Kitchoon-V5-Clean-26B-A4B

Kitchoon retrained from scratch on Gemma-4-26B-A4B-it-heretic-antislop, on the same dataset as V3, with a fixed training template.

Thinking is fixed at the root. Earlier Kitchoon versions were trained without the empty thought channel that Gemma 4 expects before every reply, so training slowly taught the model to skip reasoning. V5 is trained the way Google recommends for non-thinking data, and in testing thinking opened and closed in every run, with no reasoning budget.

"Clean" means no third-party or extra LoRAs baked in: no Gemory and Anthropos l07. Just Kitchoon on the antislop base.

The Kitchoon series is designed as an RP- and ERP-oriented model, focusing on character development and featuring a neutral writing style.

What was changed

	Base (heretic-antislop)	Kitchoon-V5-Clean
Attention OV (v_proj + o_proj)	stock	LoRA, layers 10–29, 36 targets, baked at 1.0
Dense MLP (gate_proj + up_proj + down_proj)	stock	LoRA, layers 10–20, 33 targets, baked at 0.5
lm_head, QK, MoE experts, router, embeddings, vision tower	stock	identical

Compared to V3: OV is baked twice as strong (1.0 vs 0.5). In V3 a stronger LoRA broke thinking. In V5 it doesn't, so more of the dataset could go in.

---

☕ Support

If this model was useful to you, you can support my work with a small donation.
It helps me pay for GPU time and keep releasing new models and datasets.

Gram (TON network): UQB6cmEHWIcuiwsjgO_C0Uz4fyX3Puj09BAB3hEVVpDDcQ7K
Boosty donation: Link

---

Details

• Empty thought channel. Every model turn in the training data starts with <|channel>thought\n<channel|>. The block is visible to the model but excluded from the loss, so it learns to reply without being taught to skip thinking.
• GEM loss instead of plain cross-entropy (Li et al., ICLR 2025), β = 0.7. It keeps the output distribution from collapsing onto one "best" continuation. With the linear variant used here, its gradient is exactly that of cross-entropy at training temperature 0.7, so it costs nothing extra.
• OV and dense MLP were trained together as one LoRA, but baked with separate strengths.
• MoE experts and the router were never touched — checked by module name and by matrix shape. In testing, ~93% of expert choices stay the same as in the base.
• Baked with stochastic bf16 rounding, so the baked strength matches the tested one.
• The GGUF was checked against the safetensors model: identical tokens (including all special tokens), matching perplexity.

LoRA details

Parameter	Value
Rank / alpha	OV 32 / 64, MLP 16 / 32
Learning rate	2.5e-5, cosine
Epochs	1 (756 steps)
Context	8192 tokens
Loss	GEM, β = 0.7
Eval loss (cross-entropy)	2.53 → 1.47

The eval loss starts much lower than in V3 (2.53 vs 4.32): with the correct template, the base already reads the dataset as normal replies.

What was masked out of the loss

The model still sees these spans in context — it just gets no gradient for reproducing them:

• Slop phrases. Stock cliché phrases plus ~400 n-grams that showed up across many different characters in the dataset ("voice dropping to a", "a flicker of", "tilts her head"...). Ordinary phrases like "in front of" are filtered out of this list.
• Slop constructions. "Not just X, but Y", "It wasn't an X; it was a Y".
• "X — or Y?" endings. The final either-or question sentence of a reply.
• Echoing the user. A first sentence that repeats the user's words back.
• Card echo. Text copied verbatim from the character card.
• New in V5 — repeating its own earlier replies. Any 8+ word span copied from the model's earlier replies in the same chat. About 14% of the dataset's replies had one. Replies that were mostly a copy of an earlier one were not trained at all.

Dataset

The same English RP dataset as Kitchoon-V3 by Indexnusrefather: multi-turn conversations built on character cards, split into 8192-token windows (~3k windows).

Cleaned before training:

• removed cards with underage characters and real people;
• fixed system prompts that instructed the model to write for the user;
• long conversations split into chunks instead of being truncated;
• the first and/or last paragraph randomly trimmed in part of the replies, to break up identical openings and hook-question endings;
• character name prefixes (Name:) stripped from replies.

I would like to express my immense gratitude to Indexnusrefather, with whom I work, for providing the dataset.

Using it

Recommended settings:

Parameter	Value
Temperature	0.85
Min-P	0.05
Repetition penalty / DRY	off
Reasoning budget	any, including auto

Thinking works with and without a reasoning budget, including at 5–7K tokens of context. Expect 500–700 words of reasoning before the reply. Replies run long (400 words) — give it a response length of at least 900 tokens.

What to expect

In testing, compared to the base:

• thinking opened and closed in every run, both on fresh cards and on long chats;
• swipes open differently much more often: the same first three words in ~12% of swipe pairs, vs ~43% for the base;
• far fewer hook endings and no "X — or Y?" final questions;
• strict formatting rules are followed at least as well as by the base.

Known quirks:

• swipes vary in how the scene is delivered more than in what happens: the story beat itself often stays the same. This is what the retrained l07 is meant to fix (see below);
• Gemma phrasing is still around in places ("emerald eyes sparkling with mischief", smirk, the em dash).

Coming next: l07 for V5

My decision-point LoRA, Anthropos l07, makes swipes go in different directions, not just open differently. In testing on V5 it did exactly that: impersonate and swipes became clearly more varied, and side characters came to life. But the current l07 (and Gemory) were trained without the empty thought channel. On top of V5 they make the model skip reasoning in about a quarter to a third of long thinking runs, and when that happens the reply can fall into word loops.

So V5 ships clean. l07 will be retrained specifically for this model: with the correct template, using V5 itself as the reference, and with thinking-mode examples where it learns to leave the reasoning alone. It will come as a separate release.

Feedback

This is an experiment, and I mostly test it on my own cards, so my coverage is narrow. If you try it, I would really like to know:

• How it holds up in long sessions — scene logic, who is where, who is talking to whom.
• How thinking behaves for you, and at which reasoning budget.
• How it compares to V3 and V4 in your ERP scenes.
• Your sampler settings, if you got clearly better or worse results than with the defaults.

Open a thread in the Community tab, I read all of them.

Credits

Thanks to Tetratype for the antislop base, to coder3101 for the heretic version it is built on, and to Sam Paech for the auto-antislop pipeline. Thanks to FrenzyBiscuit for pointing out the empty thought channel. Thanks to the entire 26B-Suite team for their intellectual support. Special thanks to Naphula and redaihf. You guys are awesome!

And special thanks, of course, to Indexnusrefather for his excellent dataset.