Avoid endian-specific code

The few places that still used code that was endian dependent can easily be converted not to require setting a define to indicate the endianness.

This doesn't incur any significant measurable slowdown in my tests.