System requirements

#1
by Igor7777 - opened

How much video memory is required to run the model?

what configuration is recommended to run?

P.S.
Thank you very much for your work!

LM Studio Community org

It'll largely depend on what size you want to run and how fast

If you have more than a combined 50 gb of RAM/VRAM (so a 3090 + 32GB of RAM) you should be able to run Q4_K_M, not super fast though

The less you have available (or the faster you want to run it) the lower the quant level you should use

Q2_K will run pretty nicely if given 24GB of VRAM and 8-16GB of RAM

It'll largely depend on what size you want to run and how fast

If you have more than a combined 50 gb of RAM/VRAM (so a 3090 + 32GB of RAM) you should be able to run Q4_K_M, not super fast though

The less you have available (or the faster you want to run it) the lower the quant level you should use

Q2_K will run pretty nicely if given 24GB of VRAM and 8-16GB of RAM

Thank you πŸ™πŸ™πŸ™

Sign up or log in to comment