In the case where the key is a double, Tolerance.Empty will get replaced by the default tolerance for doubles, which can be set by the user using GlobalSettings.DefaultFloatingPointTolerance. In fact, that's why we use ref tolerance.

I think new Tolerance(0) would work for the tests you have, without running this risk. I'm not sure how that would effect DateTime key, for example, without looking at the  code. It's possible we need something like Tolerance.Empty, which doesn't get replaced... maybe Tolerance.Zero or Tolerance.None?

Take a look at how tolerance is used and if changing is a can of worms, we can file a separate issue. Otherwise, if it works, please use Tolerance(0) at least for numeric types. (See Numerics.cs)