Colab is getting crashed due to Memory while accessing Llama-3-8B
Hello Brilliant Minds,
I am trying to access Llama-3-8B using Google Colab & transformers with accelerate, but every time it is getting crashed giving error of RAM fully utilized, I have used T4-GPU as hardware accelerator but no success, can anyone please suggest something. I am able to access it through Ollama though locally.
Regards,
Fellow Learner
can you share what your resources look like along w/ any errors the system spits out?
@sayanroy07
try quantization
It worked for me when i used the bitsandbytes library on T4
can you share what your resources look like along w/ any errors the system spits out?
Hi this has been resolved now, just had to re-start the kernal & colab session, it worked now. thanks all
@sayanroy07 try quantization
It worked for me when i used the bitsandbytes library on T4
Hi this has been resolved now, just had to re-start the kernal & colab session, it worked now. thanks all