Topic: numerical error

topics > computer science > Group: algorithms

data value

computational geometry
error checking in robot programming
exceptions and undesired events
FOCUS number system
real numbers and floating point numbers
unbounded precision


All numerical functions on computers have small errors due to a finite number representation. The error varies over the function's domain, making evaluation order important. For instance subtracting two nearly equal numbers is error-prone, as is taking a value near the function's vertical asymptote. Numerical accuracy is provided by small integers, multiple precision arithmetic, string arithmetic, and FOCUS numbers (mul-div only). Error analysis is an important branch of numerical mathematics.

Most systems append a zero digit during normalization. This is correct for exact numbers, such as one, but error prone for real numbers in base 16. With real numbers the correct digit ranges from zero to the floating point number base. The appended zero is at the correct range's boundary, thereby maximizing the possible error. A better solution, tested on a numerical conversion package, is shifting in a half digit (i.e., 8 for a number base of 16) unless the previous digit was a zero (keeps integers exact). (cbb 5/80) Modern floating point hardware uses base 2 instead of base 16. (cbb 5/06)

Subtopic: errors likely up

Quote: errors occur frequently in numeric tables; corrections to the Nautical Almanac were themselves in error [»babbC_1864, OK]

Subtopic: software error up

Quote: discrepancies in commercial packages for seismic data processing traced to software errors; caused 1% precision loss per 3000 executable source lines; much worse than cross-platform errors [»hattL10_1994]

Subtopic: numeric error in geometric computations up

Quote: unrestricted geometric computation has potentially unbounded growth in the precision required [»hoffCM3_1989]
Quote: a geometric object defined by approximate data may not make sense even though it can be drawn [»hoffCM3_1989]

Subtopic: real vs floating point up

Quote: even though computers approximate real operations, ignore this until get programs that would be correct with exact operations [»straC3_1973]

Subtopic: tracking round-off errors up

Quote: AMTRAN generates a warning whenever 4 or more decimal digits are lost in a subtraction [»reinJ11_1971, OK]
Quote: MicroTool includes lower and upper limit operators to bound operations [»elshJL1_1991]
Quote: perturbed inputs tracked round-off error degradation fairly closely [»cbb_1990, OK]
Quote: Guard performs relative debugging with assertions to compare data structures, permutations to identify subarrays, and plots of error surfaces
Quote: wonglediff identifies numeric instability by running a program multiple times while randomly changing the floating-point rounding mode [»eggePR4_2005]

Subtopic: bounding numeric error up

Quote: to bound numeric error, a programmer must be able to predicate how the compiler generates floating-point operations [»farnC7_1988]
Quote: programming languages do not support the computation of estimated errors in physical measurements [»dimoT9_1986]

Subtopic: double vs. single precision up

Quote: a program may use double precision for precise products of single precision or for testing near zero results [»farnC7_1988]

Subtopic: uninitialized as infinity up

Quote: in arithmetic can use +-infinity as NIL; so uninitialized variables produce NIL results [»wileDS11_1973, OK]

Subtopic: recalibration up

Quote: if two hundred spot welds, correcting errors in three points can correct errors in all others [»craiJJ4_1987]

Subtopic: unbounded precision arithmetic up

Quote: fast arbitrary-precision addition and multiplication up to a thousand bits; uses floating point numbers; adaptive
Quote: CORE geometric library offers guaranteed accuracy; up to 150 times slower than machine accuracy [»karaV6_1999]

Subtopic: reducing floating point errors up

QuoteRef: cbb_1973 ;;3/19/74 multiplicative floating point errors: (b[n]-b[n]')*(16^(n-1))(a1+a2/16)((16)) last term if normalization (ca. a1*b1 <. 16) if no normalization, error decreases by ca a1/16

Subtopic: using error as a signal up

Quote: loss of tracking is the termination signal for Sketchpad; finish a drawing with a quick flick of the pen; the tracking program fails

Related Topics up

Group: data value   (19 topics, 433 quotes)
Group: mathematics   (23 topics, 560 quotes)

Topic: bugs (66 items)
Topic: computational geometry (20 items)
Topic: error checking in robot programming (6 items)
Topic: exceptions and undesired events (29 items)
Topic: FOCUS number system (8 items)
Topic: real numbers and floating point numbers (37 items)
Topic: unbounded precision
(9 items)

Updated barberCB 9/04
Copyright © 2002-2008 by C. Bradford Barber. All rights reserved.
Thesa is a trademark of C. Bradford Barber.