The model ends every message with </s>
It's a really good model but when I talk with my roleplay chat bot it ends every message with this for some reason.
Huh, that's weird. Did you make sure this is in your stopping strings? <|end_of_text|>
I never had that issue, looking into my training files and gguf quants <|end_of_text|> was the EOS and PAD tokens. I'm also unsure of where the </s> is leaking from.
If the issue still persists then you can set </s> as a custom stopping string, that works too.
i was testing a lot by doing all kinds of things like adding the <|end_of_text|> in the text separators and in between example messages but nothing worked. then i checked of the AI assistant had the same issue but it actually didn't which made me wonder if it was because of the example messages i had in the Author's note. So I took all example messages out of one of my bots and it worked. But when i put them back, it all still worked fine. I can't get it to appear anymore. The only thing slightly weird about my example messages was i feel is that they were separated by spaces but putting those back didn't make it appear again. even putting the separator back to the way it was which was *** doesn't change anything. So it all works fine now which is great, just don't know why it was there and why it disappeared.
After restarting the webui it's back, I can't find a way to remove it and the assistant has it now too. there must be something unique to this model because other models like Fimbulvetr-11B-v2.i1-Q6_K.gguf and DevsDoCode-LLama-3-8b-Uncensored.Q8_0.gguf don't have the same problem. I don't think any of the models have had this happen except for this one.
I'm also getting the thing now and then, plus some run-on paragraphs that never shut up. At this point I prefer Fimbul 11b. I'm using the Faraday app, with the setting of "Use model default". Would using ChatML help? Or something else I should paste into some box?
On the </s> issue: There seems to be some confusion on if Llama3 should have eos_token (in tokenizer_config.json, in the unquantized model) set to <end_token> or <eot_id>, and I think this is currently impacting Oobagooba's Web Generation UI specifically. Setting eos_token to <eot_id> will likely fix the generation. Some other L3 models have made this change (e.g. https://huggingface.co/ChaoticNeutrals/Poppy_Porpoise-v0.4-L3-8B)