OutOfMemoryError: CUDA out of memory.
#204
by
RohitSuresh15
- opened
I have two GPUs
[0] NVIDIA GeForce RTX 3090
[1] NVIDIA GeForce RTX 3090
but when i try to load the model
model_name = 'meta-llama/Meta-Llama-3-8B'
model = AutoModelForCausalLM.from_pretrained(model_name, token=access_token)
I get out of memory error
Did you check the cuda availability and that the model is properly load into the gpu?