grammar-v7 (run #1 exam-push) -- FAILED the do-no-harm gate; NOT shipped

Educational / provenance release. This checkpoint is not used in production. The live site keeps shipping mydeskmate/grammar-v5.

What it was

Run #1 of the GPU exam-push recipe: warm start from grammar-v5, 1 epoch on the standard mix (coedit + jfleg x4 + agentlans x0.8), 1250 steps on a Google Colab Tesla T4, official scorers, pre-registered selection rule.

Measured (official scorers, GPU run)

benchmark v5 (live) this run target verdict
CoNLL-2014 test F0.5 47.35 52.10 >= 65 improved (+4.75)
BEA-2019 dev F0.5 33.90 35.48 >= 70 improved (+1.58)
JFLEG test GLEU 62.78 63.25 >= 62.30 pass
Do-No-Harm (clean touched) 2.39% 5.00% (25/500) < 5% FAIL

Verdict

The pre-registered rule (CoNLL/BEA up AND GLEU >= 62.30 AND Do-No-Harm < 5%) failed on the last condition by one sentence of 500 (24/500 would have passed). Reported as measured; nothing hidden. The checkpoint is kept as the warm start for run #2, the do-no-harm calibration -> mydeskmate/grammar-v7b.

Downloads last month
32
Safetensors
Model size
0.2B params
Tensor type
F32
·
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for mydeskmate/grammar-v7

Finetunes
1 model