Will this run on a GTX 3060 with 12 GB of RAM
#16
by
webslug
- opened
Hi
Will this run on a GTX 3060 with 12 GB of RAM?
Specs: Intel I7 6700
RAM: 16GB DDR 4
GPU: GTX 3060 (12 GB VRAM)
No.Your only option would be to use use a GGML instead, partially loaded on to the GPU and the rest in RAM. But it would have to be a small GGML, like q2_K or q3_K_S at the most, as you don't have much RAM either.
You may need to stick to 13B at the most, unless you can get more RAM or a bigger GPU.
Noted thank you. I will purchase some more RAM.
Closed
webslug
changed discussion status to
closed