slm-125m-sft

A 125.8M-parameter grounded reader: it answers a question using only a passage you supply, and refuses when the passage does not contain the answer. Full fine-tune of narendraalluri/slm-125m-base, which was itself pretrained from random weights.

Prompt format (required)

The model was trained on one exact template. Deviating from it degrades output badly:

<|bos|><|system|>
Answer the question using only the provided context. If the context does not contain the answer, say so.<|eos|>
<|user|>
<context>
{passage}
</context>

Question: {question}<|eos|>
<|assistant|>

from transformers import AutoModelForCausalLM, AutoTokenizer

tok = AutoTokenizer.from_pretrained("narendraalluri/slm-125m-sft")
model = AutoModelForCausalLM.from_pretrained("narendraalluri/slm-125m-sft")

SYSTEM = "Answer the question using only the provided context. If the context does not contain the answer, say so."
prompt = (f"<|bos|><|system|>\n{SYSTEM}<|eos|>\n<|user|>\n<context>\n{passage}\n"
          f"</context>\n\nQuestion: {question}<|eos|>\n<|assistant|>\n")

ids = tok(prompt, return_tensors="pt", add_special_tokens=False).input_ids
out = model.generate(ids, max_new_tokens=64, min_new_tokens=4, do_sample=False)
print(tok.decode(out[0][ids.shape[1]:], skip_special_tokens=True))

Greedy decoding is recommended. Sampling makes fabrication worse at this scale.

When the answer is absent the model emits exactly: The provided context does not contain the answer to that question.

Measured behaviour

On 58 unanswerable and 75 answerable held-out questions, greedy decoding:

Metric	Value	Meaning
refusal recall	67.2%	unanswerable questions correctly refused
false refusal rate	12.0%	answerable questions wrongly refused
refusal precision	81.2%	emitted refusals that were correct
number fidelity	92.6%	numeric answers with no figure absent from the passage
val loss / ppl	1.1449 / 3.14	cross-entropy on answer tokens only

Read the first row as the honest headline: roughly a third of the time, when the answer is not in the passage, this model invents one anyway. Treat its output as a draft.

Training

Knob	Value
data	11,563 train / 608 val records, avg 306 tokens
loss	cross-entropy on `{answer}<
refusal share	10.0% of training records
epochs / steps	2 / 361
hardware / cost	4x H100, $0.22
LR	5e-05 -> 5e-06 cosine, 5% warmup
precision	bf16

Everything before <|assistant|> is masked to -100, so the model is never trained to reproduce the passage or the question — only to answer.

Data provenance and synthetic-data disclosure

The instruction data is synthetic. Passages were sampled from the base model's own cleaned pretraining corpus (US case law, SEC filings, general web text); the questions and answers were generated by Google Gemini (gemini-3.6-flash) and then filtered: every number in an answer must appear in its passage, answers must overlap the passage by >=60% of content words, plus exact dedup and 13-gram decontamination against CaseHOLD and LexGLUE.

Unlike the base model, which involved no teacher at all, this model is the product of sequence-level distillation of a capability (grounded reading) — though not of the teacher's knowledge, since the filter discards facts absent from the passage.

Limitations

• Not an assistant. It answers questions about a supplied passage. With no <context> it has nothing to read.
• It fabricates confidently when pushed past the passage, including plausible-looking figures, percentages and directions of change ("a decrease of 6%" for a passage describing an increase).
• It over-refuses on roughly one in eight answerable questions.
• No retrieval. Supplying the right passage is the caller's job; pair it with a retriever for RAG.
• Not legal or financial advice. This is a teaching demonstration of the SFT method at 125M, not a production tool. Reliability improves most from a larger base model or retrieval-plus-verification, not more SFT.

Base-model caveats (including a measured CaseHOLD contamination gap) carry over: see narendraalluri/slm-125m-base.