Repository: go
Issue: all: ordering numeric values, including NaNs, and the consequences thereof
Labels: NeedsInvestigation

There are thorny issues around ordering of numerical values, and rather than add to existing conversations I thought it worth pulling them into their own conversation.

The questions here touch on `math.Compare` (issue #56491) as well as the proposed `ordered` constraint (issue #59488). That said, I feel there are higher level issues around comparison that should be discussed and resolved before making any other changes to the library or language. I also feel that `math.Compare` should be pulled for now, not released, until these issues are better understood.

Go provides less than and less-than-or-equal operators that apply to strings, integers, and scalar numbers. (There is no ordering for complex values.) The proposal to add an `ordered` constraint somewhere is _a priori_ reasonable as it allows one to write comparators, finders, and sorters that apply to all such types.

Except they don't, not quite. The problem is IEEE-754 NaNs. It is a peculiarity of IEEE-754 and its implementation in both hardware and most languages that NaNs do not behave like other numbers under comparison. All "less than" and related comparison operations, including "equals", return `false` when comparing NaNs, even when the bit pattern of the operands is identical. This is specified by IEEE-754 and, although odd, provides an easy way to see if a value is a NaN: if it is neither equal nor unequal to itself, it is not a number.

This behavior is the key reason why the map `clear` builtin was added: One cannot pull out all the values of a map to clear it as it is never possible to match a NaN key.

Similarly, sorting and such will misbehave in the presence of NaNs. This is why `math.Compare` was added, to provide a total ordering on floating point. But of course, the `sort` package does not use it, nor could a generic sorter, at least absent a _generic_ compare, which is possibly what should happen instead.

Mischievously, this total ordering of NaNs is also defined by IEEE-754 but is **not** the standard behavior of less than, either as a hardware instruction or in a programming language.

Thus NaNs are ordered but not by the comparison operator. _The consequences of this must be clear before defining any ordering constraint._

That is, the addition of an `ordered` constraint to the language, however it is done, bumps up against this conflict. If it is defined, as suggested, to be any type that implements the `<` less than operator, any function that uses that constraint may fail unpredictably if a NaN arises, as IEEE-754 demands. On the other hand, if it is instead defined using the total ordering of `math.Compare`, programs will always work reliably but the `<` operator is then not allowed, which is bizarre for a constraint defining an ordering.

Neither of these paths is satisfactory. So what should we do? Short answer, I don't know, but we certainly should not change the language or libraries until we have thoroughly thought this through.

I do not, at least at the moment, propose a solution. Instead I am isolating the issue in a discussion to make sure we do the right thing with `ordered`, so we don't end up in a situation like the one for maps where the language had to be changed to make them behave properly.

Discuss.