Set byte order on all buffers

makes sure we see the same going in as coming out