basta/lightning-mcqwen3

Lightning McQwen3

A LoRA adapter for Qwen3-0.6B that thinks shorter: 47.3% fewer output tokens at −0.88 pp accuracy on BottleCapAI's ThinkingCap test (SVAMP, MBPP+, IFEval, BFCL; 8 samples per question, paired with the base model). Thinking stays on, and none of the four benchmarks was seen in training.

• Code and recipe: github.com/basta/lightning-mcqwen3
• Write-up: Half the thinking, minus one point

Files

• adapter_config.json, adapter_model.safetensors: the LoRA (r16, alpha 32, all linear layers), trained from Qwen3-0.6B at revision c1899de289a04d12100db370d81485cdf75e47ca.
• gate.pt: an optional think gate for tool calls. A logistic probe on the adapted model's layer-21 hidden state at the end of the prompt; where it fires, the reply starts with an empty thinking block. It only applies to prompts with a function list and is used through the repository's evaluate.py --gate.

Results

Against Qwen3-0.6B, 8 samples per question, 95% intervals from a paired bootstrap over questions:

	Tokens saved	Accuracy change
SVAMP (maths)	66.3%	−3.8 pp
MBPP+ (code)	39.1%	−1.9 pp
IFEval (instructions)	32.0%	+1.4 pp
BFCL (tool calls)	44.6%	+0.8 pp
All	47.3% [45.8, 48.7]	−0.88 pp [−1.61, −0.15]

Without the gate, the adapter alone saves 43.2% at −0.85 pp; the gate only changes BFCL (25.3% → 44.6% saved there). The headline combines the ungated runs on SVAMP, MBPP+ and IFEval, which the gate doesn't touch, with a gated BFCL run. A full gated rerun read 46.6% / −1.33 pp, within rerun noise.

Use

With transformers and peft, without the gate:

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

tok = AutoTokenizer.from_pretrained("Qwen/Qwen3-0.6B")
model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3-0.6B", torch_dtype="auto")
model = PeftModel.from_pretrained(model, "basta/lightning-mcqwen3")

messages = [{"role": "user", "content": "A train travels 60 km in 45 minutes. What is its speed in km/h?"}]
ids = tok.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt")
out = model.generate(ids, max_new_tokens=1024, do_sample=True, temperature=0.6, top_p=0.95, top_k=20)
print(tok.decode(out[0][ids.shape[1]:], skip_special_tokens=True))

The evaluation samples with temperature 0.6, top-p 0.95, top-k 20, Qwen3's recommended thinking-mode settings. To score it on the benchmarks, with the gate:

git clone https://github.com/basta/lightning-mcqwen3 && cd lightning-mcqwen3
hf download basta/lightning-mcqwen3 --local-dir weights
python evaluate.py --out runs/final --adapter weights --gate weights/gate.pt

Training

Every step is a LoRA continued from the one before, on the challenge's suggested training data only:

1. Prompt distillation. SFT on Chain-of-Draft traces with the prompt removed (11,247 targets).
2. On-policy oracle DPO. The distilled model's correct traces are cut at 10–90% of their thinking and re-answered 4 times from each cut. DPO prefers the earliest cut from which all 4 re-answers stay right (9,034 pairs, stopped at step 100 of 561).
3. Anchored correctness DPO. The model's own right maths trace over its own wrong one at matched length, with an NLL term on the right one (732 pairs, 42 steps).
4. Think gate. Fitted on 1,500 ToolACE prompts, labelled by whether thinking changes the sampled tool call; no correctness labels and no benchmark data.

The repository's scripts/reproduce.sh runs every step with its settings.

Credit

The challenge, evaluation harness and training data selection are by BottleCapAI. The prompt is Chain of Draft (Xu et al., 2025). The truncation oracle with guard pairs is close to ThoughtFold (ICML 2026), and the probe gate is a label-free variant of ThinkSwitcher (EMNLP Findings 2025).