kessenma/gemma4-e2b-german-tutor-4bit

Gemma 4 E2B German Tutor (4-bit MLX)

A German grammar-correction fine-tune of Gemma 4 E2B, 4-bit MLX quantized (3.4 GB on disk, **2.7 GB peak RAM** during a normal turn). It is the small sibling of kessenma/gemma4-e4b-german-tutor-4bit, trained on the identical dataset, and exists so phones that can't afford the ~4.3 GB E4B model still get a usable tutor.

Trained for one narrow job: look at one sentence a learner wrote and either accept it or return a correction, in a fixed format. Four target areas — verbs with fixed prepositions (warten auf), separable verbs (aufstehen), reflexive verbs incl. the accusative/dative split, and da-/wo-compounds (darauf/worauf).

⚠️ Read this before you use it: you must filter echoed corrections

This model's dominant failure is emitting FIX: <the input, unchanged> with a confabulated reason attached, instead of saying OK. On a 32-item held-out set of already-correct sentences, 19 came back "corrected" — and 18 of those 19 were the input verbatim.

That single behaviour is the difference between this model being unusable and being good:

scored as	core suite	extension suite	false corrections	missed errors
raw output	68%	70%	59%	19%
with an echo filter	83%	85%	3%	19%

The filter is a few lines and cannot cost you anything — it only ever converts a no-op "fix" into an accept:

import re

def apply_echo_filter(response: str, original: str) -> str:
    """A FIX line that restates the input is not a correction."""
    def norm(s: str) -> str:
        s = re.sub(r"\s+", " ", s.strip().strip("\"“”„'"))
        return s.rstrip(".!?").strip().casefold()

    m = re.search(r"FIX:\s*(.+)", response)
    if m and norm(m.group(1)) == norm(original):
        return "OK"
    return response

Two things to get right if you reimplement it: keep it diacritic-sensitive (Madchen → Mädchen is a real correction, not an echo), and fold trailing punctuation (models echo the sentence and add a period).

This is not unique to this model — stock Gemma 4 E2B, stock E4B, Ministral 8B, Aya Expanse 8B and Apple's on-device model all show it, several far worse. It just matters most here, because without the filter the headline number looks disqualifying and with it the model is shippable.

Prompt format

System prompt (the correction task it was trained on):

You are a meticulous German teacher reviewing one line a student said during a spoken conversation.
The student's level is B1. Correct clear grammar mistakes, but ignore minor style issues.
The student uses the du form.
They may have mixed in an English word they didn't know — in your correction, replace it with the correct German word.

If the sentence is already correct and natural German, reply with exactly:
OK

Otherwise reply in EXACTLY this format and nothing else:
FIX: <the full corrected sentence in natural German>
WHY: <one short explanation in English, at most 18 words>

User turn: The student said: "<sentence>"\n\nEvaluate the student's sentence.

It was also trained on multi-turn tutor dialogue and flashcard-JSON generation, but the correction task is the one that's measured here.

Results

Held-out suites (60-item core over the four target areas, 61-item extension over ten untargeted areas), against the stock model, both as shipped 4-bit quantizations. All numbers with the echo filter applied to both sides:

	stock E2B	this model
core four areas	73%	83%
extension (untargeted)	85%	85%
false corrections (n=32)	0%	3%
missed / mis-fixed errors (n=69)	28%	19%

The fine-tune buys ~10 points of accuracy and cuts missed errors by a third, for one spurious correction in 32. Untargeted areas held level, so the general-instruction mix-in did its anti-forgetting job.

Known weaknesses

• Separable verbs regressed against the stock model (11/15 → 8/15). It sometimes asserts a separable verb is inseparable. If your use case centres on separable verbs, measure before adopting.
• One genuine bad rewrite survives the echo filter in the eval: it "corrects" Kannst du bitte das Fenster zumachen? to …zuschlagen? — swapping "close" for "slam", changing the meaning.
• The WHY: explanations are less reliable than the FIX: line. It produces confabulated rules (e.g. calling aufstehen inseparable). Treat the explanation as the weakest part of the output — in the app it originates from, the note is discarded whenever the fix is.
• Reflexives improved most (8/15 → 12/15) but the accusative/dative distinction remains the hardest area in the family.
• Judged only on German correction. No claims about anything else.

Training

QLoRA via Unsloth on unsloth/gemma-4-E2B-it: rank 8, alpha 16, lr 2e-4, 2 epochs, 362 steps, 1,447 examples (960 targeted German items + ~35% general German instruction data). Train loss 0.57 → low 0.2s, held-out eval loss flat ~0.75. ~26 minutes on a rented RTX 4000 Ada.

Data is synthetic but gated: a curated 268-entry verb+preposition table, 6,526 separable and 1,604 reflexive verbs from German Wiktionary, and seed sentences mined from Tatoeba, generated in the app's exact prompt formats and validated with self-hosted LanguageTool + spaCy checks, dedup, and a guard against held-out eval overlap.

Converted with mlx_vlm convert — mlx_lm.convert cannot convert Gemma 4 (it looks for KV-projection weights the checkpoint legitimately omits).

Notes for MLX Swift users

An inference repo must contain exactly one model.safetensors; MLX Swift recursively merges every *.safetensors it finds, so a stray adapter folder breaks loading with Unhandled keys ["base_model"]. This repo is clean.

License & attribution

Apache 2.0, inherited from Gemma 4. Built with Google's Gemma; fine-tuned with Unsloth. Training data derives from German Wiktionary (CC-BY-SA) and Tatoeba (CC-BY 2.0 FR).