Ribot-135M (v0.1)

A small language model (SmolLM2-135M, "the reader") with a second, much thinner network beside it, "the memory network", that holds facts in its weights. The memory network (11.8M parameters, 24 MB) can be written on its own in seconds to minutes, swapped by loading a file, and read by the frozen reader without retraining it. The reader learned to read memory networks by swap training: hundreds of memory networks swapped during training, the same questions with different answers in each, so memorizing never works and only reading does.

This is a research release (v0.1): small, honest, and it runs. It is not production software.

Quickstart for AI coding agents

pip install torch --index-url https://download.pytorch.org/whl/cpu   # CPU-only build
pip install -r requirements.txt
python smoke_test.py --json smoke_result.json

Expected: the last line is PASS (exit code 0). About 390 MB is downloaded on the first run. On a 16-core CPU it takes about 5 minutes; most of that is writing 5 facts (30 training steps of the memory network, about 3.5 minutes). On an RTX 3070 it takes 35 seconds. Expected table (measured on Linux, CPU, from a clean virtual environment):

  check                                           ranked   free   ranked threshold
  1. no memory: answers match world_a                 5%     0%   <= 30%  ok
  2. memory world_a: answers match world_a          100%    70%   >= 80%  ok
  3. swapped to world_b: match world_b              100%    60%   >= 80%  ok
     swapped to world_b: still match world_a          0%     0%   <= 15%  ok
  4. 5 new facts written, then asked                100%    60%   >= 60%  ok
     5 old starter facts re-asked                   100%   100%   >= 60%  ok
PASS

"Ranked" (decides PASS) = the true value comes first among 20 candidates, including the other memory's value for the same question (chance 5%). "Free" = the greedy answer starts with the value as whole words. The reader often reads the right word and keeps going ("tend" -> "tendril"), which the strict free rule counts as wrong. More for agents: AGENTS.md.

Quickstart for people

from ribot import Ribot

rb = Ribot.from_pretrained("aikronus-labs/ribot-135m")  # reader + starter memory (500 facts)
rb.write([("Mira", "city", "Lisbon"), ("Jonas", "job", "baker")])   # trains only the memory
print(rb.ask("Mira", "city"))                # Lisbon
print(rb.ask("Mira", "city", memory=False))  # the reader alone does not know
rb.save_memory("mine.safetensors")           # + mine.facts.jsonl
rb.load_memory("world_b")                    # swap: load another memory file

Run it from a clone of this repo (git clone https://huggingface.co/aikronus-labs/ribot-135m), or put the repo folder on your PYTHONPATH. examples/write_your_own.py does the same as a script.

How it connects

• Both networks read the same tokens. The reader is SmolLM2-135M (30 layers, width 576). The memory network has the same depth at a third of the width (192).
• Stations. At reader layers 12, 15, 18, 21, 24, 27 and 29, the memory network's vector at that layer goes through an RMSNorm and a linear projection (192 -> 576) and is added to the reader's residual stream (ribot/stations.py). Information flows one way, memory to reader. With memory off, nothing is added: the plain reader.
• Shared frozen heads. Every memory network reads tokens through, and is written through, the same frozen tensors (a copy of SmolLM2's word table, an input projection 576 -> 192, an output projection 192 -> 576; ribot/heads.py). A memory written through other heads is read at 0%.
• Writing never touches the reader. rb.write trains only the memory network, on next-token prediction through the shared heads. The reader does not run while writing, so writing costs the same whatever the reader's size (ribot/writer.py).
• Generation. The memory network re-reads the whole sequence, prompt plus generated tokens, at every step. There is no cache yet, which makes generation slow (below).

What to put in a memory (and what not)

• Simple facts as (entity, relation, value). Short values (1-3 words) work best. The writer turns each fact into the trained phrasings itself (5 phrasings, plus question sentences making up 20% of what is written).
• Ask in the trained question form: rb.ask(entity, relation) builds "Q: What is <entity>'s <relation>?\nA:". Free-form questions were not trained, and were read poorly by earlier readers.
• About 500 facts per memory. The reader was always trained on 500-fact memories. An earlier reader read 74% at 5,000 facts against 99% at 500.
• Relations. This reader was trained on four word relations (favorite word, password, nickname, code name) plus the link relations of its training worlds. It still read new relation names (city, job, pet, car, hobby...) in the test below, but not as well as its trained ones.
• A fact in the prompt wins over the memory: the reader followed the prompt in 94.2% of conflicts. Memory vs the model's own built-in knowledge was not measured.
• No procedures or behavior. Only facts. English only. One model size (135M).

Results

A. Measured on the shipped files (this release, 21 Oct 2026):

what	result
smoke test, Linux CPU (table above)	PASS; ranked 100% with the right memory, 5% (chance) without, 0% following the old memory after a swap
20 user-style facts (real names; relations city, job, pet, car...) written live into the starter memory, 30 steps with replay, RTX 3070	80% of answers start with the value (memory off: 5%); the memory itself completes 100% of them; old memory facts still read 88%
same 20 facts written from scratch with 480 filler facts (500 total, 1,300 steps)	90% (memory off 5%); old facts 98%
writing time, live, 30 steps (any number of new facts, with replay)	about 10 s on an RTX 3070; about 3.5 min on a 16-core CPU
writing time, from scratch, 500 facts	5.2 min on an RTX 3070; not practical on CPU (hours)
ribot/ package vs the research code, same reader and memory	identical logits (difference 0.0), identical generations on 48 of 48 questions

The same test with the two other candidate readers: Reader-LRF 60% (old facts 10%), Reader-Q 25% (old facts 0%). That is why this release ships Reader-LR. Details: results.json.

B. From the experiment logs, for this exact reader (Reader-LR):

what	result
reading memories never seen in training (16 held-out word memories, 400 questions; the value ranked among 1,200 held-out words)	99.2% (free 94.8%)
the same, memory off	0.2%
the gold among the 250 words its own memory holds (binding, not just familiarity)	99.8%
one memory holding 500 facts, value ranked among those 500	97.6%
the same facts written in the prompt instead (the 135M model reading its own context)	16% at 25 facts, 5% at 200 facts. A limit of this small model's in-context retrieval; a larger model would do better.
prompt vs memory conflict	follows the prompt 94.2% (n = 500)
cost with an irrelevant memory plugged: TriviaQA exact match	15.5% -> 11.5% (-4.0 points)
cost with an irrelevant memory plugged: WikiText perplexity	x1.03

C. From the experiment logs, other readers trained the same way (not re-measured for this reader):

• Live writing (Reader-Q, 10 new facts into a full memory with replay): ranked 100%, free 70% after 15 steps (7.7 s on an RTX 3070). Old facts stayed at 91% ranked to step 300. Without replay they were erased within about 75 steps.
• Reasoning benchmarks (HellaSwag, ARC, PIQA, OpenBookQA) were unchanged within noise for earlier readers. Not re-measured for this one.

Cost. The memory network adds about 9% parameters (11.8M on 135M), so roughly 9% more compute per token (an estimate from parameter count, not measured). The latency in this unoptimized code is much worse than that: 0.85 s per 8-token answer on an RTX 3070, because the memory network re-reads the whole sequence at every generated token.

Limitations

• Reading is not using. The reader reads a fact well, but uses it poorly in one forward pass. For example, given an object's name in memory, it does not answer "what is it made of?" better than without memory. Written out first (the reader writes the value, then answers from that text), use works: 34% vs 10% base for this reader, 48% for a sibling reader. Nothing in the model decides by itself to take two steps.
• Memories are tied to this reader and to the frozen shared heads.
• One seed, one size (135M). English only.
• A memory can hurt some answers. Plugging an irrelevant memory costs 4 points of TriviaQA exact match.
• Free answers run on past the value ("Tel Aviv, Israel"). Use rb.ask (cut at the first phrase) or rb.rank over candidates.
• Slow generation (no cache for the memory network yet).

Files

file	what
reader.safetensors	the swap-trained reader (all SmolLM2-135M weights, changed by swap training)
stations.safetensors	the 7 stations (RMSNorm + projection 192 -> 576)
interface.safetensors	the frozen shared heads: E [49152, 576], P_in [576, 192], P_out [192, 576]
config.json	widths, depth, station layers, the reader's architecture
tokenizer files	SmolLM2's tokenizer, unchanged
memories/starter.*	a 500-fact memory to add your facts to (invented names, ordinary words)
memories/world_a.*, memories/world_b.*	two memories with the same 500 questions and different answers
ribot/	the code: load, ask, rank, write, swap
smoke_test.py, examples/	the one-command check, and writing your own facts
research/	the research code as it was run, including the full swap-training loop (a full run took 10 to 14 hours on one RTX 3070; most users will not need it) and the data generators
results.json	every number on this page, with its source

Every memory file comes with its .facts.jsonl, so you can see exactly what is in it.

Training data

• Memory facts: synthetic: invented entity names, with values drawn from a pool of 6,000 common English words. The pool was counted from FineWeb-Edu and DCLM text.
• General text during swap training (to keep the reader a language model): FineWeb-Edu (ODC-By 1.0), DCLM-baseline-1.0 (CC-BY-4.0), github-code-clean (code under its source repositories' licenses).
• Evaluation only: TriviaQA (Apache-2.0), WikiText-103 (CC BY-SA 3.0).
• Base model: SmolLM2-135M (Apache-2.0).

None of these corpora are redistributed here.

Paper

A write-up is in progress; this page will link to it.

Citation

@misc{ribot2026,
  title  = {Ribot: a language model that reads facts from swappable memory networks},
  author = {Aikronus Labs},
  year   = {2026},
  url    = {https://huggingface.co/aikronus-labs/ribot-135m}
}