Feedback/General Discussion
Hello, in a moment I'll give a few thoughts from using this model for a while. To be honest, this is the first time I tried a 7b, as I was used to 13b and 20b's well when I heard that 7b's are becoming the new mainstay, I figured I'd give it a shot. In general, using the model is a nice experience.
Firstly though here's my setup: I use the latest staging SillyTavern on my android. I used Lewdiculous' Q5_K_M imatrix gguf quant of this at 10k context, the backend is koboldcpp on free colab. I used roleplay alpaca for formatting with no system prompt and I use a peculiar character card system wherein I put example dialogue in desc:
And in Author note, the info in ali chat format(It's the case for this character):
Now for general feedback from minimal testing, here's what I observed:
Consistent formatting: Figured this will be a recurring feature across many 7b's I'll check out in the future.
Nice sfw potential: I played around a bit and I saw that this is really nice for sfw purposes.
Follows instructions well: While it does detract from example dialogue a bit(In the case of inner thoughts) it does follow well in all gens.
model gets a bit confused analyzing semi-complicated author note: In another character card I tested which is similiar to what I mentioned in setup, model confuses character's anatomy(It wasn't mentioned in author note that char had paws of any sort. and gender for some reason. (In one response, when character gets aroused, model assumes character who is supposed to be female as a futa and I was like: Bruh since when 💀).
minor logical incoherencies and feels like model glances over responses by user: Some responses by model are tad strange and does not make sense context wise. Though that's probably sampling, (min_p: 0.053, smoothing: 0.08).
good personality grasping: Not particularly as great as other higher parameter models I tried but good enough.
Traces of phrase repetition: I don't use repetition penalty, and I did see one instance of this happening in one response.
Trace of gptism: in some responses, variations of 'shivers down your spine' or 'pleasure coursing through your body' tend to appear in slight nsfw situations.
Nice metaphor usage: though uncommon, the model does make use of it. This reminds me of a model I know that uses this characteristic which is Dark Forest V2 20b.
Simplistic Prose: I was getting that feeling when using this model, probably just me being spoiled by 20b's.
Sampling:
Responses:
Strange responses:
gender confusion:
Anatomy confusion:
In this case, phrase repetition of 'a low growl of pleasure':
For what it's worth, it does a much better job unquantized, and it runs fast on a 4090. It's actually one of the better turn-taking RP models I've used. It's not as great for long-form creative content, but I'm really impressed at how well it responds (speed and quality) in back-and-forth RP.
This model is superb! Are you planning a Llama 3 version ?