MathUtil: fix IsQNAN()

The constants were one nibble too short and the lower 51 bits don't
actually have to be zero.
Interpreter: less duplicate code in float compares