Disable floating point support only for arm

--enable-fixed-point is required for NEON optimizations, and NEON
optimizations are available only on ARM.