Numerical Methods
Math · Advanced · Lesson 4 of 7

Computer Arithmetic

Why 0.1 + 0.2 ≠ 0.3

An interactive lesson with 6 steps, about 5 minutes: you predict, drag and solve, with instant feedback on every answer. It runs in your browser and needs JavaScript.

What you’ll learn

  • Computers store numbers in binary floating point with limited precision.
  • Decimals like 0.1 can’t be stored exactly, so tiny errors appear.
  • Compare floating-point numbers with a tolerance, and watch for accumulated errors.

Steps

  1. Binary fractions
  2. Checking equality
  3. Catastrophic cancellation
  4. Order matters
  5. Digits of precision
  6. A deadly drift