Hardware Requirements for Q4_K_M
#23
by
ShivanshMathur007
- opened
What is the minimum hardware required and what is the recommended hardware for Q4_K_M with a decent speed. Any idea?
What is the minimum hardware required and what is the recommended hardware for Q4_K_M with a decent speed. Any idea?
if you're a Mac M-chips user, 32GB ram should be ok to run inference with Q4_K_M. I'm playing with the Q8 version, and it can take up to 52GB ram and output at 26 t/s on my MBP (M2-max 64GB ram),