Failed to use Q8 model in LM Studio

#3
by rorrerror - opened

Hi. I download and load the Q8 model in LM Studio 0.2.18 on M2 MAX MAC with 64GB RAM. However, the model cannot generate meaningful responses. Any idea? Thanks for any suggestions.

图片.png

Try to lower the Temperature to .9 or 1, and raise the Top K Sampling to around 40. Use around .95 for Top P sampling. See if that helps

Sign up or log in to comment