Returned to this model again after some time and realized how amazing it

#2
by sergkisel3v - opened

The first time I tried to write stories with this model, I used the chat mode in oobaboga and the standard settings. At first I thought that the model had become dumbed down from RP datasets, since it only returned one phrase per message.

However, when I tried Q4_K_M with Shortwave preset and notebook mode, it started to generate very good texts.

Funny, I had a very similar experience with it.
My first impression of MiquMaid was "meh." I thought the 8x7B Noromaids did better in most things I wanted to try, but ... 10 days later ...
I ran across some EXL2 flavors, and gave it another try. It was spectacular 9/10 gens!

I do think there are some very narrow bands for settings on this one. It's like that bad shower valve that goes from freezing cold to boiling hot in just a tiny twist? This model seems to go from "mind blowing" to "WTF?" with very small tweaks. Once you find a setup that works, save that shit ...

So... what are those settings that worked for both of you ?

So... what are those settings that worked for both of you ?

Shortwave preset in oobabooga textgen webui and q4_k_m

max_new_tokens
512

temperature
1,53

top_p
0,64

min_p
0

top_k
33

repetition_penalty
1,07

presence_penalty
0

frequency_penalty
0

typical_p
1

tfs
1

mirostat_mode
mode=1 is for llama.cpp only.
0

mirostat_tau
5

mirostat_eta
0.1

Thank you !

Sign up or log in to comment