SocraTeach Impl-3 (b, T=0.5) — LoRA adapter for OLMo-2-1B-Instruct

KL-reweighted SFT ("Impl-3") on Socratic tutoring dialogues. The per-token loss is reweighted by a temperature-T softmax over the token's forward KL against the frozen base model, so tokens the tutoring data would move furthest get downweighted. T -> inf recovers vanilla SFT; lower T concentrates weight more aggressively.

Variant b, temperature T = 0.5, step 923 (1 epoch)
Base: allenai/OLMo-2-0425-1B-Instruct · LoRA r=16, alpha=32, dropout=0.05, 7 projections
Data: meric533/socrateach-sft
Final training loss: 0.3819

Measured behaviour

metric	value	what it means
KL vs base, pedagogy + SI	0.1756	how far the policy moved in the condition it was trained in
KL vs base, pedagogy no SI	0.0731	the KL that best predicts forgetting in our sweep
Pedagogy NLL (held-out, with SI)	0.9587	tutoring fit
GSM8K, hinted prompt	0.620	prior-task retention (primary)
GSM8K, bare prompt	0.612	prior-task retention (no `\boxed{}` hint)
Answer-commit rate, hinted	1.000	fraction of items it actually attempts

Math is 250 GSM8K items, greedy, no pedagogy system instruction. "Hinted" appends the \boxed{} instruction; it is the primary number because without it a Socratically-tuned model often answers with a question instead of an answer, which scores as wrong for a reason that has nothing to do with arithmetic.

Pedagogy judge status — read this before quoting a score

Not yet judged. No pedagogy-judge score exists for this adapter in any condition.

All existing judge numbers for this project were generated with no system instruction in context, because gen_pedagogy.py's --si flag defaults to none. The headline SFT number from the earlier POC (0.84) is the +SI condition, so it is not comparable to the numbers here. A re-judge in the +SI condition is pending and will supersede these.

The judged sample is also smaller than it looks: the held-out file stores several dialogues per problem, and with greedy decoding those duplicates produce byte-identical responses, so the "40 contexts" are 12 distinct problems. Values here are deduped to those 12.

Two noise floors measured on that data, both as paired bootstraps over the 12 contexts:

Judge round-to-round: re-judging the same generations gives +0.002, 95% CI [-0.027, +0.045]. Aggregate scores are reproducible.
Retraining: two runs of the same vanilla-SFT recipe differing only in run-to-run nondeterminism scored 0.685 and 0.527 — a gap of +0.157, 95% CI [+0.018, +0.310].

So a single-seed pedagogy gap below roughly 0.15 is not evidence of a method difference.

Never judged in any condition — it finished after the judging batch had been built.

Usage

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base = AutoModelForCausalLM.from_pretrained("allenai/OLMo-2-0425-1B-Instruct")
tok = AutoTokenizer.from_pretrained("allenai/OLMo-2-0425-1B-Instruct")
model = PeftModel.from_pretrained(base, "meric533/socrateach-impl3-b-T0.5-olmo2-1b-lora")

Put a tutoring system instruction in context. This adapter was trained with one, and at least the a variants behave very differently without it.