[GC] Almost get rid of unsafe.Pointrs