Delay byte swapping to until after we know that the header matches.

ok pedro@