Repetition issues
Hi. I've started to notice increasing repetition issues with newer versions. Didn't seem an issue around v0.60.
I tried both Silly Tavern sampler setting that's recommended on the model card, also using updated context and instruction settings. But unfortunately is still unbearable.
In what form does repetition take place in your case (looping, word salad, excessive use of specific sentences, etc.) and after how many tokens in general?
With Limamono (I use data from that) I've often seen the model fail with word salad for some unknown reason, but switching to Mistral-Instruct as a base model appeared to have solved that. I've recently also used ShoriRP to generate rather long chats (20-30k tokens) with Mikupad and I haven't seen that sort of looping using Temperature ~2.35-2.45 and Min-P 0.4. However on top of that I've also added Frequency Penalty in the 0.10-0.15 range, so try checking if that solves your issues.
Oftentimes degeneration (looping, word salad, etc) can be due to excessive repetition penalty (or presence/frequency penalty), so it's best to avoid using them if possible.
It did occur within 2k tokens. Word salad and excessive use of specific sentences eg. for skin and hands.
Previously, I used default sampler settings. As I usually do for Mistral models.
I'll try Mistral-Instruct and maybe try on LM Studio if work is slow tomorrow :)
I'll try Mistral-Instruct and maybe try on LM Studio if work is slow tomorrow
I meant that Limamono (which used Mistral-7B-v0.1 as a base) had a "word salad" issue (⇒ after a while, punctuation disappeared and eventually the model began to generate random words) and I've also observed that when I tried to use Mistral-7B-v0.1 as a base for ShoriRP, but after I switched to Mistral-7B-Instruct-v0.2 (same finetuning data, everything, just different base model), those problems disappeared.
If you're just seeing plain old repetition, try adding some form of repetition penalty (but not too much) with the suggested temperature=2.35/Min-P=0.4 settings.
Another possible suggestion I can make here is to avoid banning/ignoring the EOS token, if you're doing that (in other words, you should keep EOS enabled).
Sometimes models may not act as intended (and can end up generating word salad) if they're expecting the EOS token, but its generation is prevented.
Heya. ShoriRP-v0.75d seems fine now, even with default samplers.
That's great to know, but I'm not sure what change exactly might have solved that.