ssurface/cot-dialect-olmo3-7b-think-grpo-l1

Olmo-3-7B-Think — L1 dialect (Verbose explanation)

A LoRA adapter that makes allenai/Olmo-3-7B-Think reason at compression level L1 — full natural-language reasoning.

Results

	Accuracy
After SFT	88.5%
After GRPO (this adapter)	88.9%
Difference	+0.5 pp

GSM8K test (n=1317), greedy decoding, single-turn, no exemplars, no self-consistency.

Also evaluated on (out-of-domain, not the headline metric):

Benchmark	n	Accuracy
AIME	60	6.7%
BBH	250	52.4%
SVAMP/transfer	300	90.3%

Training data

GSM8K train, re-expressed at level L1 by a teacher model: 6913 examples, median chain length 532 characters inside <think>.

Across the family the median chain runs from 532 characters at L1 to 16 at L5 — a 33x span. An L1 chain looks like this:

Madeline has $48. Her brother has half as much, so the brother's
amount is $48 divided by 2, equaling $24. Adding Madeline's $48 to her
brother's $24 gives $72.

Training setup

GRPO on top of the merged level-1 SFT model.

Engine	trl.GRPOTrainer on stock transformers, attention sdpa
Reward	correctness, format
Loss type	dapo
Generations per prompt	8
Batch	64 x 1 accum
Max completion	256 tokens
Learning rate	1e-05
KL coefficient (beta)	0.0
Prompt set	gsm8k_grpo_balanced_1k.json
Trained on	merged_olmo/l1
LoRA	r=16, alpha=32
Hardware	1x NVIDIA A100 80GB

Reward components

• correctness — +/- the gold solution's step count on an answer match, so harder problems are worth more
• format — the response must be one <think>...</think> block then #### <answer>

Engine note. Stock transformers with sdpa attention, not a fused-kernel wrapper. The fused path produced adapters whose lora_B matrices were all zero — mathematically inert despite loading without error. Every adapter in this collection was verified lora_B != 0 before publishing; 13 that failed that check were withheld.

Usage

Solve this using Level 1 (Verbose).
Problem: {your problem}

Stacks on the SFT model, not the raw base. Trained against the merged SFT model, so loading it straight onto allenai/Olmo-3-7B-Think will not reproduce the number above.

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

model = AutoModelForCausalLM.from_pretrained("allenai/Olmo-3-7B-Think", torch_dtype="bfloat16", device_map="auto")
model = PeftModel.from_pretrained(model, "ssurface/cot-dialect-olmo3-7b-think-sft-l1")   # 1. SFT for this level
model = model.merge_and_unload()
model = PeftModel.from_pretrained(model, "ssurface/cot-dialect-olmo3-7b-think-grpo-l1")   # 2. this adapter
tok = AutoTokenizer.from_pretrained("allenai/Olmo-3-7B-Think")

Limitations

• Trained and evaluated on math word problems only.
• Accuracy falls with problem difficulty, fastest at the compressed levels.
• Single seed unless the repo name says otherwise; differences of a couple of points are within noise (95% half-width ~2.7 pp at n=1317, ~4.4 pp at n=500).

Citation

@misc{cot-compression-dialects,
  title  = {Chain-of-Thought Compression Dialects},
  author = {Frolov, Anatolii},
  year   = {2026}
}