Revert some byte_buffer changes because unpack generates tons of garbage