PursuitOfDataScience/Argonne-3.0-think

Argonne 3.0-think

Argonne 3.0-think is a 2.88B-parameter reasoning model from the Argonne 3.x family. It supports an explicit chain-of-thought "think" mode (<think> … </think>) and answers directly in no-think mode. It is built entirely from Argonne 3.0-base through a multi-stage pipeline that ends in two training-free weight-soups — one to repair the pretraining base, one to reconcile reasoning with general chat.

On a 40-item internal 4-quadrant probe ({math, general} × {no-think, with-CoT}) it scores 33/40, with strong arithmetic in both modes while remaining a loop-free general chat model. (This is a small hand-checked probe — for held-out benchmark numbers and how to get the most out of the model, see Getting more accuracy: test-time compute below.)

Update (2026-07-08) — this checkpoint is v2. The model was further improved by a round of STaR self-improvement (rejection-sampling fine-tuning on the model's own verified-correct GSM8K <think> traces) followed by a third training-free soup that recovers the no-think arithmetic that step regressed — improving held-out math (on the clean SVAMP/ASDiv benchmarks, greedy 9%→18% / 18%→23%; see Benchmarks) while keeping general chat and no-think math intact. (Earlier versions of this card reported a GSM8K 2%→~7.5% gain; GSM8K is now known to be contaminated for this model — see the note under Benchmarks — so held-out SVAMP/ASDiv are used instead.) See step 8 in How it was trained. Also: config.json now ships an auto_map, so AutoModelForCausalLM.from_pretrained(..., trust_remote_code=True) loads it directly (no manual import model needed).

Decoding note (important): for math/arithmetic, use greedy decoding and do not apply an aggressive repetition_penalty / no_repeat_ngram_size. Anti-repeat decoding corrupts multi-digit arithmetic (it blocks the model from re-emitting digits it just used, e.g. turning 80 / 2 into 8 / 2) and collapses math accuracy. See Recommended inference settings below.

How it was trained

Argonne 3.0-think is the endpoint of the following pipeline. Every stage runs at 13,568-token context with RoPE θ = 1,000,000 (the base is RoPE-extrapolated from a 1,024-ctx pretraining run).

#	Stage	Init from	Data	What it does
1	Pretraining (from scratch)	—	HuggingFaceFW/fineweb (~76B tokens, 1,024 ctx)	The 2.88B base, Argonne 3.0-base.
2	Intermix midtraining	Argonne 3.0-base	FineWeb (crawl CC-MAIN-2025-21) + HuggingFaceTB/finemath subset finemath-4plus, mixed 50:50 → 60:40 by document	Injects grade-school + competition numeracy the pure-FineWeb base lacked.
3	Soup base (training-free)	—	(weight interpolation, no data)	0.35 · base + 0.65 · intermix. A linear weight-soup of the two same-lineage checkpoints — recovers general knowledge the midtraining LR over-wrote while keeping the math. This is the reasoning-ready base.
4	SFT	Soup base	HuggingFaceH4/ultrachat_200k (train_sft)	Instruction / multi-turn chat.
5	DPO	SFT model	KatoHF/chatbot_arena_binarized	Preference alignment on real user comparisons.
6	CoT-SFT	DPO model	cot_sft_mix_v3 (see table below)	Teaches explicit <think> chain-of-thought; brings math to 10/10 but regresses general chat (over-reasons / loops).
7	Final weight-soup (training-free)	—	(weight interpolation, no data)	0.15 · (DPO model) + 0.85 · (CoT-SFT model). Fractionally "un-applies" the CoT stage's general-chat regression while keeping its math. The soup_blend_a085 reasoning model (v1).
8	STaR self-improvement + soup-recovery (v2, 2026-07-08)	step 7 model	Rejection-sampled, verified-correct GSM8K <think> traces (the model's own, filtered by a \boxed{} checker) + a general/no-think anchor; then a training-free soup 0.4 · (step 7) + 0.6 · (STaR model)	STaR: fine-tunes on the model's own correct solutions → GSM8K greedy 2%→7.5%. The third soup recovers the no-think multi-step arithmetic the STaR step regressed (e.g. divisor-counting), keeping general chat intact. This checkpoint.

The two soups are the key ideas. The CoT-SFT model (step 6) and the DPO model (step 5) live in the same optimization basin (step 6 fine-tunes step 5), so CoT-SFT = DPO + Δ for a chain-of-thought weight-delta Δ. Blending back 15% of the DPO weights (0.85 · CoT-SFT + 0.15 · DPO) scales that delta down just enough to remove the loop/forgetting pathology the CoT stage introduced — the grammar loop disappears, lost facts (e.g. "the Red Planet is Mars") return — without hurting the 10/10 math. α = 0.85 is a knee: lower values recover even more general chat but start breaking <think> trace-closure (the CoT format itself lives in Δ).

Chain-of-thought SFT data (cot_sft_mix_v3, ~113k examples)

Tier	Rows	Source
direct_tulu (no-think chat)	34,000	allenai/tulu-3-sft-mixture
synth_arith	15,000	Synthetic, correct-by-construction
gen_ultrachat (CoT-augmented)	15,000	Derived from HuggingFaceH4/ultrachat_200k
hard_strict	12,000	PursuitOfDataScience/MiniMax-M2.1-Mixture-of-Thoughts (open-ended, strict-filtered)
easy_gsm8k	8,402	openai/gsm8k (main), curated with <think> / \boxed{}
med_math	5,729	nlile/hendrycks-MATH-benchmark (levels 1–3)
ms_algebra	5,000	Synthetic multi-step (Python-verified)
ms_series	5,000	Synthetic multi-step (Python-verified)
ms_geometry	5,000	Synthetic multi-step (Python-verified)
med_openmath	4,620	Problems from nvidia/OpenMathReasoning (CoT subset); solutions regenerated
hq_opus	2,300	nohurry/Opus-4.6-Reasoning-3000x-filtered
ms_divisors	1,290	Synthetic multi-step (Python-verified)

Every example is capped at 4,000 tokens; all synthetic and ms_* traces are re-verified with a \boxed{} answer extractor.

Key hyperparameters

Stage	LR	Epochs	Effective batch	Ctx	θ
SFT	2e-5	1	18	13,568	1e6
DPO	1e-6 (β = 0.03)	1	8	13,568	1e6
CoT-SFT	1e-5	1	12 (3× H200 DDP)	13,568	1e6

Datasets used (all stages)

1. HuggingFaceFW/fineweb — pretraining + intermix general half (crawl CC-MAIN-2025-21).
2. HuggingFaceTB/finemath (finemath-4plus) — intermix math half.
3. HuggingFaceH4/ultrachat_200k — SFT, and the gen_ultrachat CoT tier.
4. KatoHF/chatbot_arena_binarized — DPO preferences.
5. allenai/tulu-3-sft-mixture — CoT direct_tulu no-think tier.
6. openai/gsm8k (main) — CoT easy_gsm8k tier.
7. nvidia/OpenMathReasoning — CoT med_openmath tier.
8. nlile/hendrycks-MATH-benchmark — CoT med_math tier.
9. nohurry/Opus-4.6-Reasoning-3000x-filtered — CoT hq_opus tier.
10. PursuitOfDataScience/MiniMax-M2.1-Mixture-of-Thoughts — CoT hard_strict tier.

Fully synthetic (no external dataset): synth_arith, ms_algebra, ms_series, ms_geometry, ms_divisors.

Model architecture

Component	Specification
Parameters	2,882,162,688 (~2.88B)
Layers	24 transformer blocks
Hidden size	3,072
Attention heads	12 query / 4 key-value (GQA)
Head dimension	256
Feed-forward	SwiGLU MLP, 8,192 intermediate dim
Attention pattern	Interleaved local/global causal attention
Normalization	RMSNorm with QK / V / sandwich norms
Position encoding	RoPE (θ = 1,000,000)
Logit stabilization	Final logit softcap = 15.0
Context length	13,568 tokens (RoPE-extrapolated from a 1,024-ctx base)
Vocabulary size	151,669
Tied embeddings	Yes (input ↔ output)
Precision	bf16 safetensor shards + model.safetensors.index.json

Tokenizer

Reuses the Qwen3 tokenizer (vocab 151,669) via the Qwen2Tokenizer compatibility class. The chat template (chat_template.jinja) supports enable_thinking and parses <think> … </think>. Tokenizer files are bundled — no extra download needed.

Evaluation

Internal 4-quadrant probe, 10 items each, graded leniently (greedy for no-think, sampled for with-CoT):

Quadrant	Score
Math, no-think (greedy)	10 / 10
Math, with-CoT (sampled)	10 / 10
General, no-think (greedy)	7 / 10
General, with-CoT (sampled)	6 / 10
Total	33 / 40

The final weight-soup lifts general no-think from 5→7 vs. the pre-soup CoT model (the grammar loop is gone; the "Red Planet = Mars" fact is restored) while keeping math perfect. The residual general misses (e.g. naming the third primary color, a taller-than transitivity puzzle) are genuine capability gaps of the 2.88B base — present already before CoT training — and are not fixable by souping, decoding, or data changes at this scale.

v2 note: this 4-quadrant probe was measured on v1. v2 (blend_star_a06) preserves the general/no-think axes (verified head-to-head) and improves the math path — on the clean held-out SVAMP/ASDiv benchmarks, greedy 9%→18% / 18%→23% and pass@32 70%→73% — with the no-think multi-step arithmetic (e.g. divisor counting) that a pure-STaR checkpoint had regressed fully recovered by the recovery soup. (The card previously cited a GSM8K 2%→7.5% figure; GSM8K is contaminated for this model — see Benchmarks.)

Benchmarks (lm-evaluation-harness)

⚠ GSM8K contamination (disclosed 2026-07-10). The CoT-SFT data tier easy_gsm8k was built from a curated GSM8K shard that pooled the train and test splits with no split filter, so a large fraction of the GSM8K test set — with worked <think>/\boxed{} solutions — was seen during training; the v2 STaR step also trained on GSM8K problems. GSM8K is therefore not a valid held-out benchmark for this model; its rows below are marked ⚠ and should not be read as held-out. For an honest held-out math signal, see the SVAMP / ASDiv numbers under Getting more accuracy: test-time compute. The knowledge/commonsense tasks (ARC, HellaSwag, MMLU, TruthfulQA, WinoGrande, SciQ, PIQA, …) are unaffected — standard academic benchmarks not used in any training stage.

Standard academic benchmarks via EleutherAI lm-evaluation-harness v0.4.11, run through its vLLM backend (continuous batching) — bf16, greedy for generative tasks (no repetition penalty; see the decoding note above), completion-style prompting (no chat template, for comparability).

v1 → v2. Both checkpoints are shown: the previous release (soup_blend_a085, v1) and this upload (blend_star_a06, v2). The STaR + soup-recovery step (pipeline step 8) lifts the math path — GSM8K 5-shot 6.2 → 7.2, and greedy with-<think> roughly 2% → 7.5% — while every knowledge/commonsense task is unchanged (the STaR delta is math-only).

Open LLM Leaderboard v1 + GSM8K

Benchmark	Setup	v1	v2
ARC-Challenge	25-shot, acc_norm	34.0	34.2
HellaSwag	10-shot, acc_norm	58.7	58.6
MMLU	5-shot, acc	25.0	25.0
TruthfulQA (MC2)	0-shot	45.1	45.4
WinoGrande	5-shot, acc	57.9	57.8
GSM8K ⚠contaminated	5-shot, exact-match	6.2	7.2
Average (incl. contaminated GSM8K⚠)		37.8	38.0

Additional benchmarks (0-shot)

Benchmark	Metric	v1	v2
SciQ	acc	82.9	83.2
PIQA	acc	72.3	72.4
BoolQ	acc	62.3	62.4
ARC-Easy	acc	55.3	55.7
LAMBADA (OpenAI)	acc / ppl	44.6 / 16.7	45.3 / 16.8
OpenBookQA	acc_norm	35.2	34.6
CommonsenseQA	acc	20.1	20.1

Reading these numbers (2.88B, from scratch):

• Real commonsense/science signal: SciQ 82.9, PIQA 72.3, HellaSwag 58.7, BoolQ 62.3, ARC-Easy 55.3 — the from-scratch base is not hollow.
• At/near chance where knowledge-dense pretraining is required: MMLU 25.0 and CommonsenseQA 20.1 (≈ 5-way random) — the base was pretrained on FineWeb + math only.
• GSM8K is contaminated — do not read it as held-out (see the note above). For honest held-out math the model reaches self-consistency ~36% (SVAMP) / ~51% (ASDiv) and pass@32 ~73–74% in <think> mode; see Getting more accuracy: test-time compute.
• These are honest reference points for a small from-scratch reasoner, not a leaderboard flex.

(MathQA and LogiQA were skipped — their loaders use legacy dataset scripts unsupported by current datasets; GPQA is a gated dataset.)

Inference

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

model_id = "PursuitOfDataScience/Argonne-3.0-think"

tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    trust_remote_code=True,
    dtype=torch.bfloat16,
)
device = "cuda" if torch.cuda.is_available() else "cpu"
model = model.to(device).eval()

# --- Math / factual: GREEDY, no anti-repeat penalty ---
messages = [{"role": "user", "content": "What is 15% of 80?"}]
prompt_ids = tokenizer.apply_chat_template(
    messages,
    tokenize=True,
    add_generation_prompt=True,
    enable_thinking=True,          # set False to answer directly (also 10/10 on math)
)
input_ids = torch.tensor([prompt_ids], dtype=torch.long, device=device)

output_ids = model.generate(
    input_ids,
    max_length=input_ids.shape[1] + 512,
    do_sample=False,               # greedy — best for arithmetic
)
print(tokenizer.decode(output_ids[0], skip_special_tokens=True))

For open-ended general chat, mild sampling is fine (do_sample=True, temperature=0.7, top_p=0.9).

Recommended inference settings

Task	do_sample	temperature	top_p	repetition_penalty	notes
Math / arithmetic	False (greedy)	—	—	1.0 (off)	Do not use no_repeat_ngram — it corrupts multi-digit numbers.
No-think factual Q&A	False (greedy)	—	—	1.0
Open-ended chat / brainstorming	True	0.7	0.9	≤ 1.1	Keep any repetition penalty mild.

• Load with trust_remote_code=True so the custom ArgonneModel / argonne2 classes (model.py) register.
• Use apply_chat_template(..., enable_thinking=True) to elicit a <think> … </think> trace, or enable_thinking=False for a direct answer. Math is strong on the internal probe in both modes; for held-out benchmark numbers and how to get the most out of the model, see Getting more accuracy: test-time compute below.
• The custom generate uses max_length (total length), not max_new_tokens.
• Weights are bf16 safetensor shards with a model.safetensors.index.json weight map for sharded loading.

Getting more accuracy: test-time compute

The 33/40 internal probe (and "10/10 math") is a small, hand-checked sanity probe (10 items per quadrant): it shows the capability is present, not competition-grade math. For an honest held-out signal we use SVAMP and ASDiv — elementary math-word-problem benchmarks that appear in none of the training stages (unlike GSM8K, which is contaminated for this model — see the note under Benchmarks). The model holds substantial latent capability that cheap test-time compute unlocks:

decoding (v2, with-think)	SVAMP (n=300)	ASDiv (n=300)
greedy, single sample	18%	23%
+ budget-forcing (force-close </think> past a think-token budget)	21%	29%
self-consistency (sample K=32, majority-vote the \boxed{} answer)	36%	51%
pass@32 (a correct answer is somewhere in 32 samples)	73%	74%

Read: on clean, never-trained problems the model is a genuine small reasoner — pass@32 of 73–74% shows the capability is real, not memorized. The dominant single-shot failure is non-termination (~half of greedy traces never close </think>), which is exactly why budget-forcing (force a stop) and self-consistency (vote over K samples) are the biggest free wins.

Recommended recipe for best accuracy: sample K ≈ 16–32 traces (temperature 0.8, top_p 0.95), force-close the <think> block if it runs past a token budget, then majority-vote the extracted \boxed{} answers — roughly 2× greedy at zero training cost. (A learned same-base verifier does not beat majority vote here — reliably judging a solution needs capability this 2.88B base doesn't have — so simple voting is the right choice.)

Fast inference with vLLM

The custom argonne2 architecture (a Gemma2-style sandwich-norm layer + Qwen3 qk-norm + an extra value-norm + final logit soft-cap, full-causal, tied embeddings) ports cleanly to vLLM — validated numerically exact (token-for-token greedy) against the reference model.py. vLLM's continuous batching makes the large-K sampling above cheap (~65× the naive per-token HF decode loop, and it actually fills the GPU). The custom-model class is in the training repo.

Limitations

• 2.88B parameters — far smaller than frontier models; expect weaker performance on hard reasoning, long-form knowledge, and code.
• Base-capability gaps persist: some simple general questions (e.g. listing all three primary colors, multi-hop comparison puzzles) are answered incorrectly regardless of think mode — these are limits of the underlying pretraining, not of the reasoning recipe.
• Anti-repeat decoding breaks math (see notes above) — a property of this small model's arithmetic being token-repetition-heavy.
• Context extended via RoPE extrapolation; very-long-context retrieval may degrade.
• English-centric SFT/DPO data; limited multilingual ability. No safety filtering or content moderation has been applied.

Source code

All training code is on the main branch of the Argonne repo: https://github.com/PursuitOfDataScience/ArgonneAI

The custom architecture (ArgonneModel, model_type = argonne2) is bundled with this model as model.py. The exact scripts that produced this checkpoint, by pipeline stage:

Stage	Script(s)
Architecture	model.py
Pretraining (base)	pretrain.py
Intermix midtraining	preprocess_finemath.py → reasoning/build_intermix.py → midtraining.py
Soup base (training-free)	reasoning/build_soup_base.py
SFT	sft.py
DPO	dpo.py
CoT-SFT data	reasoning/build_sft_mix.py, reasoning/build_mix_v3.py
CoT-SFT training	reasoning/cot-sft.py
Final weight-soup (training-free)	reasoning/build_ckpt_soup.py
Evaluation	reasoning/eval_numeracy.py, reasoning/eval_intermix_base.py
Full writeup / recipe	reasoning/thinking_training.md

The SLURM launcher scripts that wire these together with per-stage hyperparameters are intentionally untracked in the repo (they carry cluster-specific paths); every stage's hyperparameters are recorded in reasoning/thinking_training.md and summarized in the tables above.

Citation

@misc{argonne30think,
  author = {PursuitOfDataScience},
  title  = {Argonne 3.0-think},
  year   = {2026},
  publisher = {Hugging Face},
  url = {https://huggingface.co/PursuitOfDataScience/Argonne-3.0-think}
}