Simplify big endian double handling