Dory-Beam-4B-Accurate

NVIDIA's looped Dory1.0-4B, distilled from Reflection's Beam-1.0-501B-A23B. It matches the base model's accuracy on every benchmark we ran while writing 1.8× fewer tokens.

This is the accuracy-first sibling of Dory-Beam-4B:

• It blends 70% base Dory with 30% of the same Beam fine-tune, so it keeps more of Dory's own reasoning.
• It has no reasoning_effort setting. At this ratio the effort levels wash out.
• For the effort setting, or for the cheapest answers, use Dory-Beam-4B.

Results

Accuracy (%) · mean output tokens per answer:

Model	Loops	GPQA-Diamond	MATH-500	AIME 2024	AIME 2025	AMC 2023	Held-out
Dory1.0-4B (base)	2	72.5 · 14.3k	98.4 · 3.7k	91.2 · 13.5k	92.1 · 15.7k	100.0 · 4.8k	79.0 · 9.4k
Dory-Beam-4B-Accurate	2	74.7 · 7.4k	97.6 · 1.9k	92.1 · 7.9k	89.2 · 9.6k	98.8 · 2.8k	80.0 · 4.8k
Dory1.0-4B (base)	4	74.5 · 13.8k	98.6 · 3.5k	92.9 · 12.6k	93.8 · 14.9k	99.4 · 4.5k	79.0 · 8.9k
Dory-Beam-4B-Accurate	4	75.0 · 7.1k	98.0 · 1.7k	93.8 · 7.5k	92.1 · 9.3k	99.4 · 2.7k	81.7 · 4.5k

Against base Dory at the same loop count, no benchmark shows a significant difference (95% paired bootstrap CIs over problems):

• At 2 loops: AIME 2025 is −2.9 (CI −6.7 to +0.8), and AMC −1.2 is borderline.
• At 4 loops: AIME 2025 is −1.7 (CI −7.1 to +3.8).

It writes 1.8× fewer tokens overall at both loop counts. At 4 loops it uses about half the tokens of base Dory at 2 loops, and scores higher or within a point of it on every benchmark.

The setup matches Dory-Beam-4B's: vLLM, temperature 1.0, top-p 0.95, up to 64k output tokens, GPQA ×2, AIME ×8 and AMC ×4 samples, plus 300 held-out training-distribution problems. Every graded answer is in the dataset's eval configs as dory-beam-4b-accurate. The model repo's code/scripts/audit_eval.py recomputes these numbers.

How it was made

The fine-tune is Dory-Beam-4B's first stage: full-parameter SFT on 52,660 verified Beam traces at reasoning_effort=medium, with the loop count sampled per step. Alone, it reasons 2–3.5× shorter than base Dory but drops 7–10 points on AIME. Interpolating weights between base and fine-tune, then re-normalizing onto nGPT's hypersphere, trades those off continuously:

• 50/50 (Dory-Beam-4B v1) gives 2.3× fewer tokens, with AIME 2025 7 points lower.
• 70/30 base to fine-tune (this model) gives 1.8× fewer tokens with no measurable loss.

Training, merge and evaluation code are in the Dory-Beam-4B repo (merge_dory.py --t 0.3).

Usage

The weights, config and chat template match the base model, so the Dory quick start applies with the model id swapped. With vLLM from PR #60149:

vllm serve --model ProCreations/Dory-Beam-4B-Accurate \
    --hf-overrides '{"n_recurrent_loops": 2, "recurrent_kv_cache_mode": "per_loop"}' \
    --max-model-len 65536 --enable-prefix-caching --reasoning-parser nemotron_v3

Use 2 loops, or 4 when accuracy matters most. GGUF builds (Q5_K_M and Q8_0) for llama.cpp are in ProCreations/Dory-Beam-4B-GGUF, as dory-beam-4b-accurate-*.gguf.

Limitations

Trained only on math and science data; coding, tool calling and long-context use were not evaluated. English only.

License

OpenMDW-1.1, inherited from Dory1.0-4B; a copy is in LICENSE. Base model © NVIDIA. Training data generated with Beam-1.0-501B-A23B (Apache-2.0).