intfloat

/

e5-mistral-7b-instruct

like 401

Hello, I'm facing an cuda memory error while trying to embed documents (less than 4096 tokens).
I'm using sentence transformers to load the model, I'm using a Tesla V100-PCIE-32GB GPU.

Here is the error :

torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 64.00 MiB (GPU 0; 31.74 GiB total capacity; 23.36 GiB already allocated; 11.06 MiB free; 23.37 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation. See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF

any idea how to solve this ? or my GPU doesn't have enough memory ?