Deterministic?
Swipe diversity was literally zero in my testing in Marinara Engine. Swipes produced verbatim exact copies of the same message. Tried other Gemma-4-31B merges and isolated MeroMero as the issue. Other merges including MeroMero produced more diversity but recognizably similar paths. Merges not including MeroMero would produce vastly different responses per swipe.
What are your sampler settings ?
Temp = 1.0
Top_p = 0.95
Top_k = 100
Freq_pen = 0.05
Pres_pen = 0.0
Pretty neutral settings... it also seems deterministic to me when I use the model. Just to be clear, this is a fine tune using the same 3 datasets than meromero, not a merge. So yeah, the deterministic effect might come from these datasets, or one of them. I could try to retrain the model and eliminate some of the datasets
I've had some issues from Zerofata models in the past — basically every PaintedFantasy model had really bad article-dropping issues in my testing. That wasn't unique to Zerofata MS3.2 models, of course, but it seemed exacerbated in any model containing that lineage. Not sure what's up there ¯_(ツ)_/¯