Amazing tune! Some feedback, thanks and inspiration.
Heya, Gryphe!
Thank you, first of all, for your continuous contributions to the space over the years. I've been a lurker for a while, and this tune is so good it drew me out of hiding. For the first time my local experience feels comparable to, and in some cases better than, cloud models.
I'm running the model at Q6_K_M, lower quantization tends to break down relatively soon but this one is pretty stable up to roughly 25K token context, which is about where the base model at qat_q4_0 breaks down as well. Largest quant that fits neatly into 16GB VRAM and 32GB RAM with enough space left over to not impede doing other stuff at he same time.
Somehow, despite the way you trained it supposedly leaving the central parts of the model alone during training, reasoning, instruction following and general emotional intelligence of the model have improved pretty notably in narrative context, at least from my impressions of it. I've not tested it exhaustively, but have tried it in a large variety of scenarios, with about 20 different prompts and presets, and it managed a few that the gemma 4 base didn't parse correctly, and the only ones it completely bungled were ones that most other models, even much larger ones also failed at. As others have already suggested, training on the qat base might be a good call, but I'd also be interested in seeing you try this process with different datasets first.
I had been toying with similar ideas of training primarily for style, and slowly curated a dataset from human written text over the years. Your success with this approach makes me think it might be The One for the job. I'm prepping a test run on unsloth with gemma 4 E4B at the moment, which I will start as soon as I can figure out how to target only lm_head. If you have any recommendations, or additional details (your dataset size and formatting I am immensely curious about for one) you can share, I would love to hear about them, as I'm very much just an enthusiastic newbie.
Have a great day, and keep doing what you do!