CUDA out of memory
#37
by
worldsapart
- opened
Consistently getting this issue on Windows with RTX 3070 Ti. I noticed GPU memory utilization shoots up to 100% quickly until the process is killed. I played around with PYTORCH_CUDA_ALLOC_CONF without success.
torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 1024.00 MiB. GPU 0 has a total capacty of 8.00 GiB of which 0 bytes is free. Of the allocated memory 6.11 GiB is allocated by PyTorch, and 451.52 MiB is reserved by PyTorch but unallocated. If reserved but unallocated memory is large try setting max_split_size_mb to avoid fragmentation. See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF
Any suggestions?
Try enabling cpu offload,
pipe.enable_sequential_cpu_offload()
This should allow you to run it