nyxhuang/ednoda-grade-cefr-mapping

text-to-grade

End-to-end English text → US K-6 grade level predictor.

Given an English sentence, it returns:

• a continuous CEFR score in [1.0, 6.0] (1 = A1 … 6 = C2),
• the discrete CEFR level (A1–C2),
• a continuous US grade in [0.0, 6.0] (0 = preschool, 1–6 = grades 1–6),
• the nearest grade word (preschool, one, … six).

It is a two-stage model:

1. CEFR head — a 3-layer truncated bert-base-uncased with a single-linear regression head. This stage is loaded verbatim from SNALYF/CEFR_Bert_Fine-tuned (Ace-CEFR baseline, arXiv:2506.14046 §4.5.1).
2. Grade head — a piecewise-linear isotonic calibrator that maps the CEFR score onto a numeric US grade. Calibrated on the ednoda K-6 snapshot (2,448 sentences, ~2,989 (text, grade) pairs after multi-grade rows are expanded).

Usage

The model uses a custom modeling.py, so loading needs trust_remote_code or a local download of the repo. Either form works:

# Option A: clone or hf_hub_download then load locally
from modeling import TextToGradePredictor
model = TextToGradePredictor.from_pretrained("YourUser/text-to-grade")
model.eval()

results = model.predict([
    "I'm from Canada.",
    "The committee unanimously ratified the long-debated treaty.",
])
for r in results:
    print(r)
# {'text': "I'm from Canada.",  'cefr_score': 2.313, 'cefr_level': 'A2',
#  'grade_num': 4.05,            'grade_word': 'four'}

predict() batches under the hood (batch_size=32 by default). For raw forward access you can call model(input_ids, attention_mask, token_type_ids) or model.cefr_score(list_of_texts) to skip the grade head.

Outputs and label maps

CEFR score	CEFR level
1	A1
2	A2
3	B1
4	B2
5	C1
6	C2

grade num	grade word
0	preschool
1	one
2	two
3	three
4	four
5	five
6	six

Calibration data

The grade head was fit on the ednoda snapshot — a 2,448-row corpus of K-6 English-learner expressions, vocabulary, and question prompts, each labelled with one or more grade words. Multi-grade rows were exploded so each (text, grade) becomes its own row (~2,989 pairs).

Per-grade CEFR distribution from the calibration set:

grade	n	mean CEFR	mean level
preschool (0)	5	1.34	A1
one (1)	2	1.39	A1
three (3)	717	1.69	A2
four (4)	757	1.68	A2
five (5)	860	1.88	A2
six (6)	648	2.00	A2

The fit was an sklearn.IsotonicRegression(increasing=True) from CEFR score to grade number, serialised as a 51-point piecewise-linear table inside config.json. At inference the model interpolates with np.interp — no scikit dependency is required to load.

Metrics (train-set, on ednoda)

• MAE on grade: 0.92
• Exact-grade match: 30.0 %
• Within ±1 grade: 82.0 %

These are training-set fit metrics; they are an upper bound on the calibrator's ceiling, not a held-out estimate. The CEFR head's own held-out metrics (445-row ACE-CEFR test) are MSE 0.567, 51.5 % exact level, 93.9 % within ±1 level — see the source repo.

Caveats

• Saturation at high CEFR. ednoda content is K-6 English-learner material; almost all of it scores A1–A2. As a result the calibrator is essentially flat for B1+: a B2 sentence and a C2 sentence both clip to grade_word = "six". When the upstream CEFR score is ≥ ~3, treat the grade as "≥ six" rather than a literal value.
• Weak ends of the range. Only 5 preschool and 2 grade-1 rows exist, so predictions in [0, 1] are unreliable.
• Single-sentence input. Both stages were trained on sentence-level text; do not pass paragraph- or document-length inputs without segmenting first.
• English only. The base BERT is bert-base-uncased.

Files

modeling.py         # TextToGradePredictor — defines model + load/save
config.json         # bert hyperparams + isotonic table + grade-word map
pytorch_model.bin   # 45.7M params (3-layer BERT + regressor)

Citation / credits

• CEFR head: SNALYF/CEFR_Bert_Fine-tuned
• ACE-CEFR baseline: https://arxiv.org/abs/2506.14046
• Calibration data: ednoda K-6 snapshot

License

MIT (same as the upstream CEFR head). Re-uploading the upstream weights here is done with credit to the original author; if you are not the same author, please obtain their consent before publishing this repo.