maram-350m

A 339,267,586-parameter decoder-only language model trained from scratch on one 48 GB GPU to run K = 3 reasoning streams in parallel inside one sequence (block-interleaved every N = 64 tokens) under a tree attention mask, with a 2,100,226-parameter judge head that reads a periodic <chk> token and predicts which stream is right. This repository holds both the trunk (maram-350m-v2, the second and valid 350M training run) and the same trunk with the judge head fine-tuned on the model's own streams (maram-350m-v2-judge-v2), which is the checkpoint to use for native decoding.

Paper: MARAM: a free in-weights verifier for parallel reasoning streams — selection beats voting, matches beam at equal tokens, and pruning doesn't help (Angurajan, 2026), DOI 10.5281/zenodo.23857054. Every number below is copied from the paper and its RESULTS.md; see MODEL_CARD_maram-350m-v2.md for the full card and MODEL_CARD.md for the 125M ablation models.

What it does

On held-out 5–7-step arithmetic (synth-hard, n = 2,000, 95 % Wilson intervals), with judge-v2:

method	accuracy	tokens per problem
greedy decompose	0.7095 [0.689, 0.729]	145
best-of-3 anchored, value vote	0.7325 [0.713, 0.751]	421
beam 2 on the same trunk	0.7625 [0.743, 0.781]	366
judge selector over 3 streams, no pruning	0.7730 [0.754, 0.791]	425
gated native, K3 N64, τ 0.35 δ 0.1	0.7755 [0.757, 0.793]	425
beam 3 on the same trunk	0.8070 [0.789, 0.824]	617

Judge selection beats the value vote on identical samples and matches beam-2 at equal tokens; beam-3 is higher at 1.45× the tokens. Easy 2–5-step arithmetic is near saturation (greedy 0.9310); Countdown and GSM8K are at chance for every method. The margin exists only at the trained K = 3 and N = 64. Details and limitations: the paper, §4–5.

Use

pip install -r maram_inference/requirements.txt
cd maram_inference
python generate.py --model ../maram-350m-v2-judge-v2 --tokenizer ../tokenizer.json --mode greedy1 --strat decompose \
    --prompt "Compute the value of ((39 - 11) - 20) + 865) * 6."
python generate.py --model ../maram-350m-v2-judge-v2 --tokenizer ../tokenizer.json --mode native --K 3 --N 64 \
    --strats decompose,decompose,decompose --temps 0,0.3,0.3 --strat_pool decompose --fork_temp 0.5 \
    --prune gated --tau 0.35 --delta 0.1 --prompt "Compute the value of ((39 - 11) - 20) + 865) * 6."

--prune none gives the selector-only rule. The model answers prompts of the form "Compute the value of ." and is not a general assistant.

Files

• maram-350m-v2/, maram-350m-v2-judge-v2/: model.safetensors (fp32), config.json, train_config.json, export_info.json (source checkpoint and its SHA-256), CHECKSUMS.txt.
• tokenizer.json: byte-level BPE, 16,384 entries, every digit a single token.
• maram_inference/: mask, model, judge head, decoder (--prune {always,gated,none}), tokenizer wrapper, loader, CLI. No training code is released.
• heldout/{synth,synth_hard,countdown}.jsonl: the 2,000-problem held-out sets of the paper, with CHECKSUMS.txt. GSM8K test is public.
• maram-125m/, maram-125m-judge-v2/, maram-flat-125m/: the 125M ablation models.
• MODEL_CARD_maram-350m-v2.md, MODEL_CARD.md.

Licence

Weights, tokenizer, inference package, model cards and held-out files: Timegravity Research-Only Licence v1.0, see LICENSE (SPDX LicenseRef-Timegravity-Research-Only-1.0; commercial licences on request). Paper: CC BY 4.0.

Citation

@misc{angurajan2026maram,
  author    = {Angurajan, Vignesh},
  title     = {MARAM: a free in-weights verifier for parallel reasoning streams -- selection beats voting, matches beam at equal tokens, and pruning doesn't help},
  year      = {2026},
  publisher = {Zenodo},
  doi       = {10.5281/zenodo.23857054},
  url       = {https://doi.org/10.5281/zenodo.23857054}
}