Remove the cached little endian variable. We can get it easily off
of the DataLayout.