SocraTeach Impl-3 (a, T=8) — LoRA adapter for OLMo-2-1B-Instruct

KL-reweighted SFT ("Impl-3") on Socratic tutoring dialogues. The per-token loss is reweighted by a temperature-T softmax over the token's forward KL against the frozen base model, so tokens the tutoring data would move furthest get downweighted. T -> inf recovers vanilla SFT; lower T concentrates weight more aggressively.

Variant a, temperature T = 8, step 923 (1 epoch)
Base: allenai/OLMo-2-0425-1B-Instruct · LoRA r=16, alpha=32, dropout=0.05, 7 projections
Data: meric533/socrateach-sft
Final training loss: 0.4729

Measured behaviour

metric	value	what it means
KL vs base, pedagogy + SI	0.6622	how far the policy moved in the condition it was trained in
KL vs base, pedagogy no SI	0.0671	the KL that best predicts forgetting in our sweep
Pedagogy NLL (held-out, with SI)	0.9665	tutoring fit
GSM8K, hinted prompt	0.652	prior-task retention (primary)
GSM8K, bare prompt	0.616	prior-task retention (no `\boxed{}` hint)
Answer-commit rate, hinted	0.992	fraction of items it actually attempts

Math is 250 GSM8K items, greedy, no pedagogy system instruction. "Hinted" appends the \boxed{} instruction; it is the primary number because without it a Socratically-tuned model often answers with a question instead of an answer, which scores as wrong for a reason that has nothing to do with arithmetic.

Pedagogy judge status — read this before quoting a score

Judged OVERALL (no SI, deduped, n=12): 0.370, against base 0.358 and vanilla SFT 0.685 scored in the same batch.

All existing judge numbers for this project were generated with no system instruction in context, because gen_pedagogy.py's --si flag defaults to none. The headline SFT number from the earlier POC (0.84) is the +SI condition, so it is not comparable to the numbers here. A re-judge in the +SI condition is pending and will supersede these.

The judged sample is also smaller than it looks: the held-out file stores several dialogues per problem, and with greedy decoding those duplicates produce byte-identical responses, so the "40 contexts" are 12 distinct problems. Values here are deduped to those 12.

Two noise floors measured on that data, both as paired bootstraps over the 12 contexts:

Judge round-to-round: re-judging the same generations gives +0.002, 95% CI [-0.027, +0.045]. Aggregate scores are reproducible.
Retraining: two runs of the same vanilla-SFT recipe differing only in run-to-run nondeterminism scored 0.685 and 0.527 — a gap of +0.157, 95% CI [+0.018, +0.310].

So a single-seed pedagogy gap below roughly 0.15 is not evidence of a method difference.

This adapter is SI-gated. With no system instruction in context it behaves essentially like the base model: it scored 0.370 against base's 0.358, and it produced step-level guidance on 0 of 12 held-out contexts. That is exactly what the no-SI condition cannot distinguish from "learned nothing" — its pedagogy NLL (measured with the SI in context) says it captured ~81% of vanilla SFT's improvement. Judge it with the SI before drawing any conclusion about its teaching behaviour.

Usage

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base = AutoModelForCausalLM.from_pretrained("allenai/OLMo-2-0425-1B-Instruct")
tok = AutoTokenizer.from_pretrained("allenai/OLMo-2-0425-1B-Instruct")
model = PeftModel.from_pretrained(base, "meric533/socrateach-impl3-a-T8-olmo2-1b-lora")

Put a tutoring system instruction in context. This adapter was trained with one, and at least the a variants behave very differently without it.