Add CudaMemManager. It queued up calls of cudaFree.