Minimum system requirements?
#1
by
kusoge
- opened
Any idea how much RAM is needed to preload the model and VRAM to actually use it?
The 4bit 13B models I tried fills my 16GB RAM and during usage my VRAM completely fills my 12GB VRAM.
Will 32GB RAM and 18GB VRAM be enough to load and use this? Thanks
Hey! Sorry. I didn't notice this until just now.
For a better overview of the exact requirements see: https://www.reddit.com/r/LocalLLaMA/comments/11o6o3f/how_to_install_llama_8bit_and_4bit/
But TL;DR: No. Minimum VRAM needed is 20GB, 23GB with full context. 24GB RAM is needed, so on the RAM side you'd be set.