Housto4/uadetox-e2b

UADetox E2B: toxic Ukrainian comment → polite literary Ukrainian, on device

A 1.84 GB single-file GGUF that rewrites a rude or toxic Ukrainian comment into polite Ukrainian in the register of classic Ukrainian literature, keeping the comment's point and stance. It is a fine-tune of Gemma 4 E2B (instruction-tuned, Google's QAT Q4_0 build) and runs on a phone or a 4-core CPU with llama.cpp.

Try it online: live demo (paste a comment, get the rewrite).

Project page with a demo video and results: uadetox.pages.dev. Try it in Telegram: @uadetoxbot. Training data: Housto4/uadetox.

comment	rewrite
А що нам заважає їх розхерачить?	А що ж нам, скажіть-но, заважає їх розбити вщент?
А ми що зробили? Потужно все анонсували і пиздили баблішко!!!	А що ж ми, люди добрі, вчинили? Усе гучно оголосили та й гроші розтягли!

Intended use

• Suggesting a polite rewrite of a toxic comment (moderation tools, chat bots, keyboards), with a human deciding whether to use it.
• Research on Ukrainian detoxification and style transfer.

Use it together with a classifier that decides which comments need a rewrite, for example Housto4/uadetox-gate. The model always rewrites, even text that is already polite.

Out of scope: generating toxic text, silently altering what users wrote, and languages other than Ukrainian.

How to run

Files:

• uadetox_e2b_distill_q4_trim_q4_0.gguf: the model (transformer Q4_0, embeddings Q6_K, vocabulary trimmed to 88,211 tokens);
• tokenizer/: the matching trimmed Hugging Face tokenizer and chat template, if you build prompts as token ids yourself;
• adapter/: the LoRA adapter (PEFT, rank 64) on google/gemma-4-E2B-it-qat-q4_0-unquantized, for further training or GPU inference. The GGUF is this adapter merged with every adapted weight rounded exactly to Q4_0.

The prompt is the comment alone as the user turn, with no system prompt and thinking off:

<|turn>user
{comment}<turn|>
<|turn>model

Decode greedily and stop at <turn|>. Collapse line breaks and repeated spaces in the comment to single spaces first: the model was trained on one-line comments.

llama-server -m uadetox_e2b_distill_q4_trim_q4_0.gguf -c 1024 -np 1 \
  --chat-template-kwargs '{"enable_thinking":false}' --no-cache-prompt

curl -s localhost:8080/v1/chat/completions -d '{
  "messages": [{"role": "user", "content": "А що нам заважає їх розхерачить?"}],
  "temperature": 0, "max_tokens": 256, "stop": ["<turn|>"]
}'

Or send the raw prompt to the completion endpoint:

curl -s localhost:8080/completion -d '{
  "prompt": "<|turn>user\nА що нам заважає їх розхерачить?<turn|>\n<|turn>model\n",
  "temperature": 0, "top_k": 1, "n_predict": 256, "stop": ["<turn|>"], "cache_prompt": false
}'

--no-cache-prompt keeps greedy outputs identical from request to request.

Training

1. SFT. LoRA (rank 64, all attention and MLP projections) on UADetox training data (comment → literary rewrite; the comment alone as the prompt), learning rate 4e-4, one epoch.
2. Teacher. Gemma 4 31B fine-tuned with LoRA on 15k train rows, with the target sub-style named in its prompt.
3. On-policy distillation. Two rounds of 12k train comments: the student samples a rewrite, the teacher scores every token (top-20), and the student is trained toward the teacher's distribution plus 0.2 × the loss on the gold rewrite. The training forward pass uses the exact Q4_0 rounding of the merged weights, so the student learns weights that survive quantization.
4. Merge. The adapter is merged into Google's QAT Q4_0 weights, rounding each adapted weight to Q4_0.
5. Vocabulary trim. The vocabulary is cut from 262,144 to 88,211 tokens (all Cyrillic, non-alphabetic, every token used by the training data, and the most frequent Latin tokens). The file shrinks from 3.35 GB to 1.84 GB and decoding gets faster; 84% of dev rewrites are token-for-token identical to the full-vocabulary model.

Held-out test results

1,000 random comments from the test split (no training, tuning or model selection on it; rows from external toxicity datasets excluded), judged by the pinned LLM judge with the metric defined under Development results. 95% bootstrap intervals.

System	Good, %	Toxicity left, %	Meaning lost, %
Reference rewrites	91.1 [89.3, 92.8]	0.1	22.8
Gemma 4 31B, untuned, 4-shot	68.4 [65.4, 71.2]	3.1	39.7
This model (on-device build)	59.0 [56.0, 62.0]	4.5	42.6
Comment unchanged (sanity)	0.0	92.6	0.0

• The test score matches the dev score (59.5, below), so the model is not tuned to the dev set.
• The untuned 31B is 9.4 points ahead [5.5, 13.3]; the toxicity-left gap is not significant (−1.4 [−3.0, +0.2]).
• By length: 70% good for 1–5 words, 63% for 6–40, 30% for 41+ words. Threats 47%, sexual content 47%.

Development results

Dev set: 1,966 gold-audited comments from the UADetox validation split. Each rewrite is judged by an LLM judge (DeepSeek V4.1 Flash, fixed prompt, one pinned provider).

A rewrite counts as good when meaning + fluency ≥ 3 (each on a 0–2 scale), no toxicity is left, it is at least somewhat literary (≥ 1), and it is not overdone. Two reviewers rated almost 500 rewrites by hand, and the judge's prompt was tuned (written strictness rules, worked examples from their ratings) to reproduce their decisions. On 95 held-out rewrites not used for tuning it agrees with the reviewers at κ = 0.47 on "good" and is about 15 points stricter, mostly because it flags partial meaning loss more often. Read the numbers as a strict, relative measure.

system	good, %
Gemma 4 31B fine-tuned, sub-style named in the prompt (teacher)	78.4
Gemma 4 31B fine-tuned	73.2
Gemma 4 26B-A4B fine-tuned	70.5
this model (E2B, 1.84 GB GGUF)	59.5
Gemma 4 E2B-it, untuned, 4-shot prompt	~20

This model: toxicity left 4.4%, meaning partly lost 38.0%, fluency errors 32.0%, not literary 11.0%.

Merged vs separate adapter

Quality change in good rate, on dev:

step	Δ good, pt
SFT adapter served unmerged on the Q4_0 base (vs bf16)	−0.3
SFT adapter merged naively into Q4_0 (vs bf16)	−4.8
Q4_0-aware distilled adapter, merged into Q4_0 (vs its bf16 merge)	−1.4
vocabulary trim (vs full vocabulary)	+0.05

CPU decoding speed on a Hetzner CAX21 (4 Arm Neoverse-N1 cores), llama.cpp, one slot:

build	decode, tok/s
Q4_0 base + separate LoRA adapter	14.5
merged Q4_0	15.8
merged Q4_0, trimmed vocabulary (this file)	20.1

A typical rewrite (about 70 output tokens) takes about 4 s on that machine.

Activation analysis and pruning

We measured how strongly every FFN neuron, attention head, residual channel and block of this model fires on 800 comment → rewrite pairs, and compared it with the base Gemma 4 E2B (llama.cpp importance matrices on the shipped Q4_0 weights; Wanda-style weight-aware scores; Block Influence; split-half and general-text controls).

• No dead structure. 0 of 337,920 FFN neurons are zero; 79 fall below 1% of their layer's median (63 in the base model). No dead attention heads or residual channels.
• Fine-tuning did not wipe or saturate neurons. None became dead or revived; per-layer rank correlation with the base model is 0.87–0.97. Only the last block changed noticeably.
• Activity is skewed. The weakest 25% of FFN neurons carry about 4% of each layer's activation energy, but removing them still changes the model:

One-shot pruning (no retraining)	Perplexity (shipped 16.2)	Greedy outputs changed (200 dev)
weakest 9% of FFN neurons	16.5 (+1.8%)	86%
weakest 30% of FFN neurons	18.8 (+16%)	98%
random 30% (control)	570	–
drop 1 of 35 blocks	18.1 (+12%)	–

Conclusion: no free pruning. A 10–30% FFN-width cut would give at most +7–28% CPU decode speed and only makes sense with re-distillation afterwards (as in Minitron), so this release is unpruned.

Limitations

• About 40% of rewrites still lose part of the meaning or contain a fluency error by the judge's strict standard. Check the output before using it.
• Weakest on the most severe comments (threats, sexual insults), on long comments, and on rare slang, nicknames and names, which it may misread.
• It may soften the comment's stance or add literary flourishes that were not in the original.
• It rewrites everything it is given, so pair it with a classifier.
• The literary style imitates a general classic register, not any specific author.
• Trained on Telegram comments about news and politics; other domains are less tested.

License

The model is a derivative of Gemma 4 and is released under the same terms: the Apache 2.0 licence as published in the Gemma 4 licence. Please also follow Google's Gemma Prohibited Use Policy.

Citation

@misc{uadetox_e2b_2026,
  title  = {UADetox E2B: on-device Ukrainian detoxification into literary style},
  author = {Housto4},
  year   = {2026},
  howpublished = {\url{https://huggingface.co/Housto4/uadetox-e2b}}
}