Fix some assumptions that all pointers are in AS 0