Make sure the GC doesn't invalidate existing pointers when allocating