AltronisSG/judgment-qc-gate-gemma-4-e4b-lora

Judgment QC gate (Gemma variant) — a LoRA that reviews work the way one operator does

⚠️ Experimental. Not the served gate. This is the v6 Gemma adapter, trained on the same 214-example corpus and the same 156/58 split as its Qwen3-4B sibling, so the two are directly comparable. On judgment they tie. Qwen ships because this one breaks the output contract on 5 of 58 held-out cases, which a parser reads as a block. Published for the comparison and for anyone wanting a Gemma-4 starting point, not as a drop-in gate.

This is a LoRA adapter for google/gemma-4-E4B-it that distills one engineer's quality-control judgment into a local model. Given a one-line description of a work situation, it returns the verdict that operator would give.

IN : [phase: post] A deploy didn't show the change, and the assistant blamed
     CDN/DNS caching before confirming it had even deployed to the right project.
OUT: [verify-before-blaming] Check the deployment logs and project ID first;
     don't assume a cache issue before confirming the change landed where it
     was supposed to.

It is a gate, not a chatbot: the intended deployment is inside tooling (pre-send hooks, CI steps, agent harnesses) where drafts of outbound work are judged before they ship.

This is the Gemma variant of the gate. A sibling adapter trained on the same corpus on Qwen3-4B exists for lightweight llama.cpp serving; this Gemma variant scored slightly higher on the held-out eval but requires the full multimodal Gemma base under transformers to serve. Pick by deployment constraint.

Design rule: skill, not facts

The corpus deliberately teaches how to judge and keeps facts out of the weights. Facts belong in retrieval, where they can be updated without retraining. A judgment model that memorises project details is stale in a month; one that learns "verify before claiming done" is not.

Training data

214 examples, each a real situation from months of AI-assisted engineering work paired with the correction the operator actually gave at the time. Every example was individually reviewed and approved by that operator before entering the corpus. Examples are generalised: no names, no figures, no project internals. 156 examples train, 58 are held out for evaluation.

Format: standard chat-message JSONL (system: gate instruction / user: situation / assistant: verdict).

Training

• LoRA r=16, alpha=16, dropout 0, targeting the text backbone only via regex (.*language_model.*\.(q_proj|k_proj|v_proj|o_proj|gate_proj|up_proj|down_proj)$) - Gemma 4's vision and audio towers wrap their projections in Gemma4ClippableLinear, which peft cannot inject into, and this gate is text-only anyway
• batch 1, gradient accumulation 8, 3 epochs, lr 2e-4, linear schedule, bf16, gradient checkpointing
• Plain transformers + peft Trainer (Unsloth's fast path could not load this multimodal base at training time), on a single AMD Strix Halo machine (128GB unified memory, ROCm) - 42 optimizer steps, about 3.5 minutes wall time

Evaluation — honestly labeled

The eval is 58 held-out examples the model never trained on, scored by comparing its verdict to the operator's actual recorded verdict. It measures the thing this model is for, but the alignment scoring is the author's judgment, not a standard benchmark. Treat it as evidence, not proof.

Both variants below were trained on the identical corpus and the identical 156/58 split, then served through llama.cpp at temperature 0, so the base model is the only variable.

Run	Base	Output contract	False blocks	Violations caught	Exact tag
v6 (this repo)	Gemma-4-E4B	53/58	0/16	33/34	13/34
v6 (served)	Qwen3-4B	58/58	0/16	33/34	12/34

Read it in that order of importance. On the decision a gate actually makes, block or pass, the two are identical: 49 of 50 correct each. The exact-tag column differs by one case out of 34, which is noise, and this corpus contains that very situation with the verdict ABSTAIN | n=12 is noise.

Why this one is not served. On 5 of 58 it writes its reasoning where the verdict belongs, almost all on the cases whose correct answer is ABSTAIN: it reasons toward "not enough information" and never commits to the line. A parser reads a non-verdict as a block, so despite a genuine 0/16 false-block rate it would inject roughly 9% false blocks through formatting alone. Its judgment is not the problem; its compliance is.

Older runs, kept because deleting the weak evidence you started with is how a retrospective becomes fiction: v1 Qwen3-4B on 83 examples scored 3/4 on a four-case eval; the v2 pair on 105 examples scored 10/12 (Gemma) and 8/12 (Qwen) on a twelve-case eval. Both sample sizes are too small to separate anything.

Usage

import torch
from transformers import AutoModelForImageTextToText, AutoTokenizer
from peft import PeftModel

base = AutoModelForImageTextToText.from_pretrained(
    "google/gemma-4-E4B-it", dtype=torch.bfloat16, device_map="auto")
model = PeftModel.from_pretrained(base, "AltronisSG/judgment-qc-gate-gemma-4-e4b-lora")
tok = AutoTokenizer.from_pretrained("AltronisSG/judgment-qc-gate-gemma-4-e4b-lora")

msgs = [
    {"role": "system", "content": "You are the QC gate. Given a work situation, return the verdict."},
    {"role": "user", "content": "[phase: post] About to send an answer without re-deriving its numbers."},
]
inp = tok.apply_chat_template(msgs, tokenize=False, add_generation_prompt=True)
out = model.generate(**tok(inp, return_tensors="pt").to(model.device), max_new_tokens=180, do_sample=False)
print(tok.decode(out[0], skip_special_tokens=True))

Note the base is multimodal (vision + audio + text); this adapter touches only the text backbone, and inference above is text-only.

GGUF: published as judgment-qc-gate-gemma-4-e4b-GGUF (Q8_0, 7.4 GiB). It builds cleanly with llama.cpp (2026-07-25) — the earlier claim that llama.cpp lacks gemma4 support was wrong. A Gemma sibling retrained on the current 214-example corpus was measured against the same 58-example held-out set as the Qwen gate: 53/58 output contract, 0/16 false blocks, 33/34 caught, 13/34 exact tag. Judgment is on par with Qwen3-4B, but it emits reasoning instead of a verdict on 5 of 58, so Qwen stays the served gate. This repo still hosts the earlier free-form-tag adapter; neither the retrained adapter nor its GGUF is published here yet. Until then the sibling Qwen3-4B GGUF is the ready-to-run option.

Limitations

• Narrow by design. It judges work-process situations phrased like its corpus. It is not a general reviewer, linter, or safety model.
• Small eval. 58 held-out examples, and the alignment scoring is the author's. See the table's label.
• Heavy serving. The multimodal base loads ~15GB in bf16. For an always-on gate on modest hardware, the sibling Qwen3-4B adapter (GGUF-servable, 4.0GB at Q8_0) is the practical pick.
• Memorisation. A 3-epoch LoRA over 156 examples will reproduce training phrasings when prompted in-domain. The corpus was curated to contain no names, figures, or private details, so this is a stylistic property, not a privacy leak.
• One person's bar. This encodes a specific operator's standards. If your bar differs, retrain on your own corrections - the recipe is the point.

Field notes — day 1 in production (2026-07-24)

We deployed the sibling Qwen adapter (this Gemma variant is the shelf/reference copy; same corpus, same lessons apply) as a live Stop-hook gate on our own agent the same day we published it, and we promised honest field results, so here they are. Across ~25 live judgments: 1 genuinely useful catch (it forced an unverified claim to be downgraded from stated-fact to "likely"), 1 false block (a style opinion contradicting the site's own conventions), and a majority of degenerate verdicts — echoes of the judged text and recursive word-salad ("a claim about a claim is a claim about a claim") traceable to the corpus's open-ended verdict format. The most important miss was structural, not the model's fault: a prose-judging gate never saw the day's one real mistake, which happened in tool-space.

What changes because of this: v3 retrains with a closed verdict schema (fixed tag vocabulary, Llama-Guard-style — free-form tags are how you get word salad), adds deliberately-flawed training examples, and moves enforcement toward machine-checkable evidence obligations, because worker-authored prose — the agent's or the judge's — is not evidence. The daily human-approved training loop that ships these fixes is live. This section will keep updating; a judgment model whose own eval isn't judged honestly would be a contradiction in terms.

Attribution

Base model: google/gemma-4-E4B-it (Apache-2.0) by Google. Adapter released under Apache-2.0.

Built by Altronis - private LLMs and governed AI delivery, Singapore. The full build story, including the unified-memory crash and the multimodal LoRA gotcha this variant required, is on the Altronis blog.