Instructions to use mydeskmate/grammar-v7b with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use mydeskmate/grammar-v7b with Transformers:
# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, AutoModelForSeq2SeqLM tokenizer = AutoTokenizer.from_pretrained("mydeskmate/grammar-v7b") model = AutoModelForSeq2SeqLM.from_pretrained("mydeskmate/grammar-v7b", device_map="auto") - Notebooks
- Google Colab
- Kaggle
grammar-v7b (run #2 do-no-harm calibration) -- FAILED the rule; NOT shipped
Educational / provenance release. This checkpoint is not used in
production. The live site keeps shipping mydeskmate/grammar-v5.
What it was
Run #2: v7 (run #1) calibrated on its own mined hard negatives -- 478 clean
wikitext-2-train sentences it wrongly edited (7.97% of 6000 scanned),
upsampled x3, mixed with jfleg x2 / wikitext x0.1 / coedit x0.05 /
agentlans x0.01, --identity_weight 3.0, lr 8e-5, 2 epochs, 250 steps on a
Google Colab Tesla T4. Checkpoint selected by best validation loss.
Measured (official scorers, GPU run; local CPU re-verification of the same
pipeline on run #1 matched all four metrics exactly)
| benchmark | v5 (live) | v7 (run #1) | this run | rule | verdict |
|---|---|---|---|---|---|
| CoNLL-2014 test F0.5 | 47.35 | 52.10 | 40.25 (P 52.70 / R 20.69) | > 47.35 | FAIL |
| BEA-2019 dev F0.5 | 33.90 | 35.48 | 29.65 (P 40.39 / R 14.37) | > 33.90 | FAIL |
| JFLEG test GLEU | 62.78 | 63.25 | 59.64 | >= 62.30 | FAIL |
| Do-No-Harm (clean touched) | 2.39% | 5.00% | 1.40% (7/500) | < 5% | PASS |
Verdict
Over-calibration: the do-no-harm gate was fixed (1.40%, better than the live v5) but correction recall collapsed (CoNLL recall 34.87 -> 20.69, BEA recall 22.24 -> 14.37, JFLEG below the 62.30 hold-line). The pre-registered rule (CoNLL/BEA up AND GLEU >= 62.30 AND Do-No-Harm < 5%) fails on three of four conditions. Reported as measured; nothing hidden. The lesson: 3x identity weight on an identity-heavy mix overshoots; a future run needs a much lighter calibration nudge (or checkpoint selection by JFLEG GLEU instead of validation loss, which is biased toward copying on an identity-heavy mix).
- Downloads last month
- 16