CPU or GPU
#76
by
lalit34
- opened
I am trying to load the model for analysis but each time i load it the process is getting killed. I am pretty sure theres enough space . But I am not able to figure out why my model is getting killed. Is it CPU or GPU which one to use here for tiiuae/falcon-40b-instruct.
It is first loaded from disk to CPU memory (or RAM), then transfered to GPU memory (or VRAM).
If it is killed it usually means you don't have enough RAM. A VRAM oom won't kill your progress.