Equality Comparison of Floating-Point Numbers in C# (and Others)
Published: 2021-07-11 13:04:58

## tl;dr

You should compare two floating-point numbers for equality in C# as follows:

```csharp
bool Equals(double x, double y, double tolerance)
{
    var diff = Math.Abs(x - y);
    return diff <= tolerance ||
           diff <= Math.Max(Math.Abs(x), Math.Abs(y)) * tolerance;
}
```

`tolerance` must be much larger than `double.Epsilon`.

## The problem in the equality operator

You should avoid using the equality operator == to compare two floating-point numbers for equality. This is because the equality operator can return false for two intuitively equal numbers. The following list and the output indicate the problem.

```csharp
ar one = 0d;
for (var i = 0; i < 10; i++)
    one += 0.1;
var pairs = new[]
{
    (1.0 - 0.9, 0.1),
    (0.15 + 0.15, 0.1 + 0.2),
    (one, 1.0)
};
foreach (var p in pairs)
{
    var (x, y) = p;
    Console.WriteLine($"x: {x:f18}, y: {y:f18},\r\n    " +
        $"x == y: {x == y}, Abs(x-y): {Math.Abs(x - y):f18}");
}
```
```yml
x: 0.099999999999999978, y: 0.100000000000000006,
    x == y: False, Abs(x-y): 0.000000000000000028
x: 0.299999999999999989, y: 0.300000000000000044,
    x == y: False, Abs(x-y): 0.000000000000000056
x: 0.999999999999999889, y: 1.000000000000000000,
    x == y: False, Abs(x-y): 0.000000000000000111
```

The equality comparisons between 1.0 - 0.9 and 0.1, between 0.15 + 0.15 and 0.1 + 0.2, and the sum of ten 0.1 and 1.0 unexpectedly result in false. The errors in the floating-point numbers cause these results.

## Tolerating absolute errors

The well-known way to resolve this problem is to compare the absolute error of two numbers with a minuscule number. The following results are as expected because the absolute errors are less than 1e-10.

```csharp
const double tolerance = 1e-10;
foreach (var p in pairs)
{
    var (x, y) = p;
    var r = Math.Abs(x - y) <= tolerance;
    Console.WriteLine($"x: {x:f18}, y: {y:f18},\r\n    " +
        $"TolerateAbsError: {r}, Abs(x-y): {Math.Abs(x - y):f18}");
}
```
```yml
x: 0.099999999999999978, y: 0.100000000000000006,
    TolerateAbsError: True, Abs(x-y): 0.000000000000000028
x: 0.299999999999999989, y: 0.300000000000000044,
    TolerateAbsError: True, Abs(x-y): 0.000000000000000056
x: 0.999999999999999889, y: 1.000000000000000000,
    TolerateAbsError: True, Abs(x-y): 0.000000000000000111
```

This approach has a problem where the absolute error between two large numbers can easily exceed the minuscule number. For example, the comparison between 0.1 added to 1e6 ten times and 1e6 plus 1.0 results in false because the absolute error exceeds 1e-10.

```csharp
const double tolerance = 1e-10;
double x, y;
x = y = 1e6;
for (var i = 0; i < 10; i++)
    x += 0.1;
y += 1.0;
var r = Math.Abs(x - y) <= tolerance;
Console.WriteLine($"x: {x:f18}, y: {y:f18},\r\n    " +
    $"TolerateAbsError: {r}, Abs(x-y): {Math.Abs(x - y):f18}");
```
```yml
x: 1000000.999999999767169356, y: 1000001.000000000000000000,
    TolerateAbsError: False, Abs(x-y): 0.000000000232830644
```

## Tolerating relative errors

You can resolve this problem by tolerating the relative error instead of the absolute error. A relative error of two numbers is the absolute error divided by the maximum of their absolute values.

The following comparison tolerant of the relative error between the two additions to 1e6 returns true because the relative error is less than 1e-10. Please notice `a / b <= c` forms `a <= b * c` to avoid division by zero.

```csharp
x and y are the same as above.
var r = Math.Abs(x - y) <= Math.Max(Math.Abs(x), Math.Abs(y)) * tolerance;
var error = Math.Abs(x - y) / Math.Max(Math.Abs(x), Math.Abs(y));
Console.WriteLine($"x: {x:f18}, y: {y:f18},\r\n    " +
    $"TolerantRelativeError: {r}, RelativeError: {error:f18}");
```
```yml
x: 1000000.999999999767169356, y: 1000001.000000000000000000,
    TolerantRelativeError: True, RelativeError: 0.000000000000000233
```

This technique causes another problem. The relative error between two minuscule numbers almost equal to zero can become very large. For example, the comparison tolerant of the relative error between 1e-11 and 1e-12 returns false.

```csharp
const double tolerance = 1e-10;
var x = 1e-11;
var y = 1e-12;
var r = Math.Abs(x - y) <= Math.Max(Math.Abs(x), Math.Abs(y)) * tolerance;
var error = Math.Abs(x - y) / Math.Max(Math.Abs(x), Math.Abs(y));
Console.WriteLine($"x: {x:f18}, y: {y:f18},\r\n    " +
    $"TolerantRelativeError: {r}, RelativeError: {error:f18}");
```
```yml
x: 0.000000000010000000, y: 0.000000000001000000,
    TolerantRelativeError: False, RelativeError: 0.900000000000000022
```

## Torelating both errors

The equality comparison must tolerate both the absolute and relative errors of two numbers to solve those problems.

```csharp
bool Equals(double x, double y, double tolerance)
{
    var diff = Math.Abs(x - y);
    return diff <= tolerance ||
           diff <= Math.Max(Math.Abs(x), Math.Abs(y)) * tolerance;
}
```

The above equality comparison returns true for both of the corner cases.

```csharp
const double tolerance = 1e-10;
double x, y;
bool r;
x = y = 1e6;
for (var i = 0; i < 10; i++)
    x += 0.1;
y += 1.0;
r = Equals(x, y, tolerance);
Console.WriteLine($"x: {x:f18}, y: {y:f18},\r\n    "+
                  $"TolerateRelativeAndAbsError: {r}");
x = 1e-11;
y = 1e-12;
r = Equals(x, y, tolerance);
Console.WriteLine($"x: {x:f18}, y: {y:f18},\r\n    "+
                  $"TolerateRelativeAndAbsError: {r}");
```
```yml
x: 1000000.999999999767169356, y: 1000001.000000000000000000,
    TolerateRelativeAndAbsError: True
x: 0.000000000010000000, y: 0.000000000001000000,
    TolerateRelativeAndAbsError: True
```

## The myth of machine epsilon

I have arbitrarily chosen 1e-10 as the `tolerance` so far, but there is a mathematically rigid number misunderstood as an appropriate value in this case. It is [machine epsilon](https://en.wikipedia.org/wiki/Machine_epsilon).

Many programming languages define machine epsilon as the difference between 1 and the next larger floating-point number, even though the formal definition is different from it. In C#, `Double.Epsilon` has machine epsilon.

You can't use machine epsilon as the `tolerance`. When adopting `Double.Epsilon` as the `tolerance`, the comparison between the sum of ten 0.1 and 1.0 in the first example returns **false**. Machine epsilon is too small, and cumulative errors of arithmetic operations easily exceed it.

```csharp
var one = 0d;
for (var i = 0; i < 10; i++)
    one += 0.1;
var r = Equals(one, 1.0, Double.Epsilon);
Console.WriteLine($"x: {one:f18}, y: {1.0:f18},\r\n    "+
                  $"UseMachineEpsilonAsTolerance: {r}");
```
```yml
x: 0.999999999999999889, y: 1.000000000000000000,
    UseMachineEpsilonAsTolerance: False
```

After all, the `tolerance` should be based on the precision necessary to your application. It also should be much larger than machine epsilon.

## Conclusion

The equality comparison of floating-point numbers should take into account both the absolute and relative errors. Furthermore, the error tolerance in the comparison should be chosen based on the required accuracy in your application. If you would like to know more details of this topic, you should consult [Comparing Floating Point Numbers, 2012 Edition](http://randomascii.wordpress.com/2012/02/25/comparing-floating-point-numbers-2012-edition/).

[This GitHub repository](https://github.com/fujieda/DoubleComparer) has the above examples and the implementation of `EqualityComparer<double>` and `IComperer<double>` based on this article.