Surprisingly good for stories
I downloaded the model without understanding much of what's on the model card. I was initially surprised that it chugged along a story I've been fiddling with for a while, and after a few tweaks, the model consistently keeps my story on track and engaging. It does very well with expanding or creating dialogue. I see far fewer spelling errors than some of the other 70B models I've tried, and it is more grammatically correct than 22 or 34B models I've used.
Great job with this!
Which quant did you use @cliffwalls ? The IQ2_XS seems to perform worse than a Q4_K_M of Nous-Capybara-limarpv3-34B for me. In fact, I've yet to find a 2-bit 70B that performs better than a good 4-bit 34B for RP purposes.
@cliffwalls : DrShotgun made this model, but removed the fp16. Sad, because WinterGoddess with 32k context was quite the hit at the time. Glad you enjoy it.
@OrangeApples : You can try the IQ_XS quants of Miqu or Miqumaid. These models are vastly superior to this version of WinterGoddess.
But if you guys want to use this model at its best aptitude, set the rope scale / linear scale at 2.5.
It offers 10240 context size, and a way better quality that the default rope scale / linear rope 8.
@Nexesenex thank you! Will be sure to try out Miqu/Miqumaid. Also have my sights on Senku at the moment.
@OrangeApples I'm using Q3K_M. After fiddling with it some more, I'm seeing very good context retention that isn't included in the pre-prompt/system prompt.
@Nexesenex Thanks for those suggestions.