Quick question.
Hello Crestfall, I noticed that you removed the Crestfall-Toppy-Rats-M-7B from your repository and updated this one, was there something wrong with it? Should I replace with this Crestfall-Toppy-M-7B instead?
Thank you for your work.
It was an experiment and the results were inconclusive. If you liked it, that is valuable feedback. I would switch to the one you mentioned for now, although the next versions will all have some form of neutral instruction set in them, probably.
It was an experiment and the results were inconclusive. If you liked it, that is valuable feedback. I would switch to the one you mentioned for now, although the next versions will all have some form of neutral instruction set in them, probably.
I must inform you that I haven't found any oddities with the Crestfall-Toppy-Rats-M-7B merge and in fact, I've been using it for a while now, of course, my main use is RP~ERP sessions.
I must inform you that I haven't found any oddities with the Crestfall-Toppy-Rats-M-7B merge and in fact, I've been using it for a while now, of course, my main use is RP~ERP sessions.
That's great feedback, thanks! Would you mind playing with the crestfall-toppy-m-7b version for awhile and seeing which one you like the best? I would be very eager to hear your thoughts.
That's great feedback, thanks! Would you mind playing with the crestfall-toppy-m-7b version for awhile and seeing which one you like the best? I would be very eager to hear your thoughts.
Alright, I'll play with this one for a couple of days and then let you know how it went. I must inform you firstly though, my experiences are based on the Q8_0 Quant, that means that I don't know how it would be for lower ranked Quants.
I'll also let you know the settings I use on KoboldCPP later.
There is a q8_0 quant available for this model as well. Generally speaking just ask me if you want a different quant for some model and I'll upload it.
There is a q8_0 quant available for this model as well. Generally speaking just ask me if you want a different quant for some model and I'll upload it.
Hello Crestfall, sorry for the late reply.
I've played with both these last few days, they're quite similar overall.
Crestfall-Toppy-Rats-M-7B: Has more quirks, nothing too grand nor bad per se.
Crestfall-Toppy-M-7b: Is similar enough that the majority of users wouldn't even notice any difference, at least not when it comes to RP~ERP, although the Toppy-Rats-M-7B seems to have a bit more vocabulary on it in very specific scenarios (which doesn't necessarily means a plus, it's specific enough)
My verdict would be that they're similar enough, meaning that they're both good, you probably did well onto pulling the other merge out.
My settings goes has follow:
KoboldCPP = Min-P (0.05-0.1 depending on my objective with the writing.) Temperature (1.6-2.0 in accordance to my Min-P value, also depending on my objective.) Standard value is 0.05 and 1.6.
Every other sampling method disabled (Top-p, Top-K, Top-A, Typ., TFS, Mirostat) including Repetition Penalty which I only enable when necessary.
Sorry for late response.
Thanks for the details, that's super helpful to know. I'm going to keep the Rats models on hold for now but might revisit the idea at a later time.