Fix memory leak in HostFromGpu when allow_gc=False!