openai/gsm8k
Benchmark • Updated • 17.6k • 1.06M • 1.57k
How to use ssurface/cot-dialect-olmo3-7b-think-conditioned-grpo-matched with PEFT:
from peft import PeftModel
from transformers import AutoModelForCausalLM
base_model = AutoModelForCausalLM.from_pretrained("merged_cond_single")
model = PeftModel.from_pretrained(base_model, "ssurface/cot-dialect-olmo3-7b-think-conditioned-grpo-matched")The alternative design: one adapter asked to serve every verbosity level by naming the level in the prompt, instead of one adapter per level. This is the comparison that motivates the per-level family.
| Requested level | Accuracy |
|---|---|
| L1 | 90.3% |
| L2 | 89.1% |
| L3 | 88.5% |
| L4 | 88.7% |
| L5 | 89.1% |
Flat across every requested level — but flat because the model emits roughly the same chain whatever level is asked for. It does not collapse; it also does not compress. Read alongside the per-level adapters, where accuracy and length actually move with the level.
| Stage | GRPO on the merged conditioned SFT model |
| Conditioning | the level is named in the prompt, not selected by adapter |
| Engine | trl.GRPOTrainer on stock transformers, sdpa |
| LoRA | r=16, alpha=32 |
| Hardware | 1x NVIDIA A100 80GB |
Solve this using Level {N} ({Verbose|Concise|Symbolic|Shorthand|Extreme}).
Problem: {your problem}
Stacks on the conditioned SFT model, not the raw base.
from transformers import AutoModelForCausalLM
from peft import PeftModel
model = AutoModelForCausalLM.from_pretrained("allenai/Olmo-3-7B-Think", torch_dtype="bfloat16", device_map="auto")
model = PeftModel.from_pretrained(model, "ssurface/cot-dialect-olmo3-7b-think-conditioned-sft")
model = model.merge_and_unload()
model = PeftModel.from_pretrained(model, "ssurface/cot-dialect-olmo3-7b-think-conditioned-grpo-matched")
Base model
allenai/Olmo-3-1025-7B