mosaicml

/

mpt-30b-chat

like 203

Follow

Mosaic ML, Inc. 351

config.init_device how to use all gpu。I config config.init_device = 'cuda:0',it report error：torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 98.00 MiB (GPU 0; 15.77 GiB total capacity; 14.74 GiB already allocated; 86.88 MiB free; 14.75 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation. See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CON