Cymela/Monarch-Latent-Math-7B

Monarch Latent Math 7B

Monarch Latent Math 7B solves math word problems by thinking in hidden steps before it answers. Instead of writing out its reasoning, it takes a few internal steps (6 by default) and then writes a short answer of about 6 tokens.

It is a set of adapter weights on Ai2's open mixture-of-experts model OLMoE-1B-7B-0125-Instruct (7B parameters, about 1.3B active per token). The base model is not included; you download it from Ai2.

Results (GSM8K test, greedy decoding)

Setting	Accuracy	Output tokens per answer
No hidden steps	17.1%	about 9
6 hidden steps	35.1%	about 6
6 steps replaced by an average step	12%	about 6
6 steps taken from another problem	8%	about 6
Base model writing out its full reasoning	67.8%	about 265

• Thinking for 6 hidden steps doubles accuracy compared with answering at once, while writing about the same number of tokens or fewer.
• Replacing the steps with an average step, or with another problem's steps, removes the gain, so the steps carry problem-specific computation that the answer uses.
• Longer problems keep gaining from more steps: two-step problems need one, four-step problems still improve at six.
• The base model writing out its reasoning is still far more accurate. Closing that gap at a few percent of the tokens is the goal of this line of work.
• Caveat: this checkpoint was selected on GSM8K test, so these figures may be optimistic for unseen problems.
• These figures were measured in fp32 on a GPU. The script's default bf16 run may give slightly different answers.

Results on new problems (SVAMP)

SVAMP, 1,000 problems that do not appear in our training data (checked by exact and near-exact match against our training questions). Measured in bf16 on a CPU, greedy decoding.

Setting	Accuracy
No hidden steps	39.9%
6 hidden steps	43.0%

• The gain from hidden steps is also present on these new problems: +3.1 points (one-sided 97.5% lower bound +1.0 point), compared with +18.0 points on GSM8K (17.1% to 35.1%).
• SVAMP is easier for this model than GSM8K even without hidden steps (39.9%).

How to run it

Requirements: Python 3.10+, torch, transformers 5.x (tested with 5.16.1), safetensors, tokenizers. Memory: about 16 GB of RAM in bf16 on a CPU, or a GPU with 16 GB or more. On an 8-core desktop CPU it loads in under a minute and answers in about 2 to 3 seconds per question.

pip install torch "transformers>=5.16" safetensors tokenizers
hf download Cymela/Monarch-Latent-Math-7B --local-dir monarch-latent-math-7b
hf download allenai/OLMoE-1B-7B-0125-Instruct --revision b89a7c4bc24fb9e55ce2543c9458ce0ca5c4650e --local-dir olmoe-base
cd monarch-latent-math-7b
python run_latent_math.py --base ../olmoe-base --question "Tom has 3 boxes of 12 pencils. He gives away 7. How many are left?"

Options: --steps N sets the number of hidden steps (0 answers at once, for comparison), --file questions.txt answers one question per line, --json prints one JSON object per answer, --dtype fp32 runs in full precision.

Intended use and limits

• Research on latent (hidden-step) reasoning. It is not a general assistant.
• Math word problems only, in English, in the style of grade-school arithmetic problems. Other inputs give poor or meaningless answers.
• It is far less accurate than the base model writing out its reasoning (35.1% vs 67.8% on GSM8K). Do not rely on its answers.
• The number of hidden steps is fixed by you; the model does not choose its own depth.

Files

• model.safetensors: the adapter weights and the three control-token rows.
• config.json: the settings run_latent_math.py needs to rebuild the model on its base.
• run_latent_math.py: a standalone script that loads the base, applies the adapters and answers questions.
• LICENSE: the Cymela Model License, Open tier 1.0 (CML-Open-1.0), which covers these files.
• LICENSE-BASE-Apache-2.0 and NOTICE: the base model's license and attribution.

Acknowledgments

Trained on AMD Instinct MI300X GPUs, with compute credits from the AMD Developer Cloud. Thank you to AMD for supporting this work.

License

These files are released under the Cymela Model License, Open tier 1.0 (CML-Open-1.0): you may use the model for any purpose, including commercially, and share your fine-tunes free of charge under the same license; you may not sell the weights or re-host the original weights. See LICENSE. The base model is Ai2's OLMoE-1B-7B-0125-Instruct under the Apache License 2.0, which governs the base.

Credit: "Monarch Latent Math 7B by Cymela". Contact: contact@cymela.com