correct hashing of large floats [fix #8619]

Unfortunately, this introduces a 30% slowdown for Float64 hashing
and a 12% slowdown for Int64 hashing. Perhaps something cleverer
can be done, but for now, at least this fixes the bug.