Correctly fill the pointed_to_size for pointers.