Built with Llama. The tokenizer files in this repository are Meta's Llama 3.1 tokenizer.

braid-commons-mamba3-1.5b: the Braid commons on Mamba-3

The commons is the foundation Braid's Threads run behind. Braid is a language model made of strands under one reed: the commons, run whole, and any number of Threads. Each Thread is one corpus with its own trained blocks 0..<cut in front of the commons' tail: blocks cut..<L, the embedding and the final norm, the same module instances, never copied. At every token a synchronization layer weighs each Thread by its recent lift over the commons and mixes the strands' distributions. Each token's probability is split exactly among the strands, and each Thread's part is cited to its own text. The session log is the artifact: every weight, probability and share recomputes from it.

This repository is the commons alone. It holds no Thread. Threads belong to their authors. Rao Studios will publish specialized Thread models of its own as they come, and Braid lets anyone build their own Thread from their own texts in a few commands (see Uses).

Model details

Weights. state-spaces/mamba3-mimo-1.5b at revision bc6b5d0f7994fe4cb3478242e92da8daf9ee29ec, Mamba-3 by its authors. They are re-packaged unmodified: upstream's BF16 values widened exactly to F32, under the checkpoint's mamba_ssm names, in safetensors. Rao Studios did not train or change them.
Architecture. 24 layers of width 2,048. Each layer is a Mamba-3 MIMO mixer (64 heads of 64, state 128, one group, rank 4, 32 rotary angles) and a gated MLP, each RMS-normed. No attention, and no KV cache. Embeddings are tied: the embedding reads tokens in and scores them out.
A's activation. Heavy-tail, written into config.json as braid_a_activation. Upstream changed it from softplus on 2026-06-01, before the release, and the release installs mamba from main. A held-out reading favoured heavy-tail on all 24 documents but by less than its own margin, so the pin is Rao Studios' decision on the release's history, recorded as taken after that reading.
Parameters. 1,497.1M. A Thread's head, blocks 0..<12, is 617.2M. The commons' tail is 879.9M: blocks 12..<24, the embedding (262.7M) and the final norm.
Vocabulary. 128,256 tokens, the Llama 3.1 tokenizer. A stream opens with BOS 128000.
The pack. braid import at cut 12 gives these hashes:

	SHA-256
NAME	`04e49e933e8918a6c59d90425dc19d8c69aafe28b721f07ddaca71f566f2a7ce`
SEAM, at cut 12	`d2697c4e71e0671dab7aea93c7ee30bc4ac2e7e18f8f2a57ce00ad2e16084db2`
VOCABULARY	`729d34c9531f80cb14cf13cf91fdc86e1d2644e89d3dbfbd2073d7d62972c48b`

Training details

Upstream's: 100B tokens of FineWeb-Edu at context 2,048. Rao Studios did no training of the commons.

Uses

With Braid. Import it, then build Threads behind it:

braid import rao-studios/braid-commons-mamba3-1.5b --cut 12 --check
braid init --base braid-commons-mamba3-1.5b
braid thread create <thread> --corpus <dataset:…|dir:…|app:…>
braid

Every Thread holds its own corpus, its own trained blocks and its own index. The commons is loaded once and serves them all. braid pack show prints the NAME, SEAM and VOCABULARY above.

Without Braid. No transformers class exists for Mamba-3. To run the language model alone, use the upstream checkpoint with mamba_ssm; its values are these.

Out of scope. The commons is not a chat model and is not instruction-tuned. It answers no question about a corpus on its own; the Threads do that, and the trace cites them. Nothing in this repository was trained on any user's data.

Threads attaching and detaching mid-answer, from one logged run on Braid's Mamba2 130m bench braid, 2026-11-10. Its weights are that run's, not this commons'.

How provenance is derived

The commons is frozen, and it is the reference. A Thread is trained on one corpus as a delta: new blocks 0..<12 in front of the commons' own tail, which never changes. When training ends, the delta is frozen too. At every token each strand gives its own probability, and a Thread's lift is its probability against the commons'. Where a Thread has learned the text, its lift is large and its entropy sits below the commons'. That window is the detection. The Thread's index then says which document the tokens follow, and that is the citation. Nothing retrains the commons, and detaching a Thread takes its part away.

The ablations back it, on this commons at cut 12 (2026-11-11). A Thread's delta tells its own texts from unseen ones at AUROC 1.000, and on the same texts it scores 0.966 where a head of the commons' own blocks scores 0.566 (the proof's D1 and D2). The holder has the largest weighted lift on 99.6% of positions where any lift is positive, and its chain is a live one on 100% of 9,644 coherent positions (ER1, ER2). Shuffle the lift across Threads and the holder's lead falls to chance, 34.0% against 33.3%.

Every strand carries its own history, and it has to survive a Thread attaching or detaching mid-answer. On a state-space model that history is a fixed state: 54.07 MB at any length, restored exactly after a detach (worst state difference 0.0 at cuts 12 and 8). A transformer of this shape keeps a key and a value for every token, 384 KiB a token at F32 (48 KiB at F16 with 8 key-value heads), for the commons and for every attached Thread, and reads all of it at every token. It passes 54 MB within 137 tokens at F32. That figure is arithmetic: no transformer arm was run, and the bounded hybrid that was run failed the lock-out rule (SY1). See the preprint outline in Braid's Docs/ABLATION.

Evaluation

Braid's REAPER rules, read on 2026-11-10 with this commons in a braid of three Threads (braid-n3's ambient, craft and veil, 40 documents each). Every rule and bar was fixed before its run. The datasets are synthetic, and every claim is bounded to these benches. The synchronization constants are Braid's defaults (window 16, β 1.5, commons logit ln 4, top-k 3, lift clamp ln 10, linear pooling), tuned on a 130m commons; a retune for this commons has not run.

The publication gate.

Rule	Bar	Result
Conformance to a pure-PyTorch oracle of the reference implementation	max \|Δ logits\| ≤ 1e-3	4.005e-5 over 3 prompts; token-by-token step against sequence 1.669e-5
SY6, the layer's invariants	the suites pass	pass
ER1, lift says who	the holder has the largest ω·lift on ≥ 70% of positive positions	99.6% of 7,150
ER2, the trajectory says which document	≥ 70% of coherent positions on the holder's chain	100.0% of 9,644
ER4′, no false provenance	the commons keeps ≥ 0.70 of the share on foreign and generic text	0.771 over 6,748 positions
ER5, the log carries it	ω and m recompute exactly from the log	0 mismatches over 348 texts
FR, fact recall	≥ 80% of fact prompts answered exactly	28 of 30 (93.3%)

Both sessions behind ER′ replay with braid verify --keys, 473 generations, with no issue.

Also read, not part of the gate.

Rule	Bar	Result
W0, the state's bytes	equal at every length	pass
W1, the commons and three Threads at F32	peak under 16 GiB	13.26 GiB
T1–T3, the trace	the trace's bars	pass
L1, nobody's text	the commons leads on ≥ 90%	100% of 28
L2, inherited knowledge	Threads take ≤ 0.10 of commons prompts' tokens; the owner ≥ 0.90 of fact answers	fail: 0.196; 0.858
L4, unknown subjects	the commons leads on ≥ 80%; Threads' credit ≤ 0.25	fail: 70%; 0.899
Q1, Q2, Q3, Q5a, Q5b, the question path	as written in Braid's REAPER.md	Q3 and Q5b pass; Q1, Q2 and Q5a fail
AT1, a node caught up in one forward against one advanced token by token	the state within 1e-4 and the same top-16 hits	fail over 98 positions

Not yet read on this commons: the remaining synchronization arms (SY1–SY5), saliency, recall, the proof, and the constants' retune.

Limitations

On this commons, Threads take credit for some tokens the commons already knows (L2) and for subjects no Thread holds (L4). The datasets' shared sentence templates are one known cause.
The question path rewrites a question into a sentence stem that matches the dataset's stored stem on 30% of questions (Q1). Retold facts are decided on 5 of 23 (Q5a), mostly because template questions match neither retelling.
A node caught up on a prefix in one forward pass does not yet equal one advanced token by token over 98 positions on this commons (AT1). Sessions logged and verified as Braid runs them replay exactly.
Training a Thread's head on a 128 GB Apple Silicon Mac fits only at micro-batch 1: 41.37 GiB peak, 14.9 s per step of 8 rows of 512 tokens.

Technical specifications


Files	`model.safetensors`, F32, 5.99 GB; `config.json`, the checkpoint's fields with `model_type: mamba3` and Braid's `braid_*` fields
State per stream	54.1 MB at F32, fixed at any context length: per layer, the SSM state, the rotation angles, the previous keys and the previous input
Key per position	16,384 B at F32, `residual:12` with the final hidden state
Four nodes at F32	13.61 GB computed; 13.26 GiB measured peak
Run on	MLX on Apple Silicon, through Braid

braid.json holds the three hashes and the source. conversion.json records the conversion from upstream's pytorch_model.bin: every input's and output's SHA-256, the tokenizer's provenance, the activation's pin and the training memory reading.

License

The weights are under the Apache License, Version 2.0 (LICENSE). The tokenizer files (tokenizer.json, tokenizer_config.json, special_tokens_map.json) are governed by the Llama 3.1 Community License (LICENSE-llama3.1) and its Acceptable Use Policy (USE_POLICY.md). They are byte-identical to meta-llama/Llama-3.1-8B at revision d04e592bb4f6, taken from the mirror NousResearch/Meta-Llama-3.1-8B. NOTICE carries both attributions.

The license also asks that a model created, trained, fine-tuned or improved with Llama Materials carry "Llama" at the beginning of its name. These weights were trained upstream, not by Rao Studios, and Rao Studios did not train or improve them with the tokenizer, so the name carries no "Llama" on that reading.

Citation

The weights:

@misc{lahoti2026mamba3improvedsequencemodeling,
  title         = {Mamba-3: Improved Sequence Modeling using State Space Principles},
  author        = {Aakash Lahoti and Kevin Y. Li and Berlin Chen and
                   Caitlin Wang and Aviv Bick and J. Zico Kolter and
                   Tri Dao and Albert Gu},
  year          = {2026},
  eprint        = {2603.15569},
  archivePrefix = {arXiv},
  primaryClass  = {cs.LG},
  url           = {https://arxiv.org/abs/2603.15569}
}

The signal Braid computes was formalised elsewhere as REAP: Lasby, Lazarevich, Sinnadurai, Lie, Ioannou and Thangarasa, "REAP the Experts: Why Pruning Prevails for One-Shot MoE Compression", arXiv:2510.13999, ICLR 2026. Braid has no router and no pruning step; the correspondence is Braid's reading, not theirs.