Clean up endian-conversion code

This fixes issues on platforms that require byte-aligned reads and writes,
such as ARMv5 and earlier.
Merge pull request #1 from balr0g/patch-1

Clean up endian-conversion code