ENH Faster decoding time

Read in large block and special case the common path.

Measuring reading & writing random data, this brought the time down by 40%.