Don't fetch pointers from a InMemoryStruct.

InMemoryStruct is extremely dangerous as it returns data from an internal
buffer when the endiannes doesn't match. This should fix the tests on big
endian hosts.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@178875 91177308-0d34-0410-b5e6-96231b3b80d8