Failed to use Q8 model in LM Studio
#3
by
rorrerror
- opened
Hi. I download and load the Q8 model in LM Studio 0.2.18 on M2 MAX MAC with 64GB RAM. However, the model cannot generate meaningful responses. Any idea? Thanks for any suggestions.
Try to lower the Temperature to .9 or 1, and raise the Top K Sampling to around 40. Use around .95 for Top P sampling. See if that helps