tetiai/dynamic-engram-350m

Dynamic Engram 350M

A 350M-parameter multilingual model that answers from a memory you inject at run time, with the question alone in the prompt. Instead of having particular memories written into it, the model was trained once to handle memories in general: any new memory is injected with no training, no optimization and no change to the weights, and the facts stay in the memory, outside the weights. It generalizes to new memories, not yet to new forms of question (other phrasings, other languages). It is a research release that demonstrates the approach: it works best on short personal facts asked in the memory's own language, and it does not yet stay silent when the memory lacks the answer but holds similar items (see Limitations).

• Unbounded memory, written without training. Add any number of texts; every item is kept and indexed in time proportional to its length. Longer documents are best read in hybrid mode, with the recalled items as sources in the prompt.
• No passage in the prompt. For each question, exact n-gram counts over the recalled items become candidates that a pointer copies from at the output and that also enter the model through Engram gates (layers 2 and 13). The pointer carries the facts; the gates mainly reduce wrong answers (on personal memories, 3.9% wrong instead of 8.3% without them).
• One model. The vectors that search the memory by meaning come from the model's own layer 8, distilled from a multilingual embedding model used only during training.
• An empty answer for "not in memory". Trained on answerable, unsupported, natively unanswerable and counterfactual questions in six languages; your application shows the empty answer as "I don't know" in the user's language. It works on most unsupported questions, not on the case in Limitations.
• CPU-first. ONNX graphs with the memory inside, int8 and int4 weights, memory-drafted decoding; about 0.4 s per answer on two CPU threads with the question alone.

Paper: Dynamic Engram: Answering from an Exact n-gram Memory Injected at Inference Time (Marcello Violini, Lorenzo Nargiso, TetiAI S.r.l.). Paper and code: doi:10.5281/zenodo.23803703 (Zenodo); the weights are in this repository.

Quick start

pip install huggingface_hub
python -c "from huggingface_hub import snapshot_download; snapshot_download('tetiai/dynamic-engram-350m', local_dir='dynamic-engram-350m')"
pip install ./dynamic-engram-350m/dynamic_engram

from engramlm import EngramMemory

mem = EngramMemory("dynamic-engram-350m", onnx_dir="dynamic-engram-350m/onnx", precision="int8", threads=4)
mem.inject([("Sara gave me a purple mug.", "2025-03-02T18:00:00+01:00"),
            ("Giorgio now lives in Naples.", "2026-10-15T10:00:00+02:00")])

mem.answer("Who gave me the purple mug?")
# {"answer": "Sara", "abstained": False, "sources": [{"memory": "Sara gave me a purple mug.", ...}], ...}

mem.answer("Who won the hockey championship?")
# {"answer": None, "abstained": True, ...}

mem.answer("Where does Giorgio live?", mode="hybrid")   # depends on time: the newest memory goes in the prompt
# {"answer": "Naples", ...}

mem.export("memories.jsonl")    # the raw log; import_ rebuilds the same state

These are the outputs of the released weights. The same works in Italian (Sara mi ha regalato una tazza viola / Chi mi ha regalato la tazza viola? → Sara), the language of most of the synthetic evaluation data. mode="engram" (default) answers from the memory alone; mode="hybrid" also puts the best k ≤ 3 items in the prompt as sources and cites them (Naples [1]). Without onnx_dir the model runs in PyTorch. strict=True (default) also silences an answer whose words are not in the recalled memory.

How it works

1. Store (tier 1). A raw log of ⟨text, date⟩ pairs, indexed by exact word n-grams and stems, plus one vector per item from the model's meaning head. A recall fuses the lexical score, the meaning similarity and explicit time expressions ("yesterday", "the last time", "in 2024").
2. Count memory (tier 2). Built per question from the top recalled items, in milliseconds and with no model call: what follows every n-gram (continuation) and which words occur near the question's cues (association; an answer that precedes its cue is reachable). The words of the items closest in meaning add candidates when the question shares no word with its memory (paraphrase, other language).
3. Reading and pointing. The top candidates enter as the model's own token embeddings through Engram's context-aware gate before layers 2 and 13. At the output, p = (1-λ)·p_LM + λ·(α·s_continuation + (1-α)·s_question), where s is the model's choice among the candidates with their counts as a prior (a pointer network): copying a fact is one decision (λ→1), an empty answer needs no memory mass (λ→0).

Base	Pleias-RAG-350M (Llama architecture, 26 layers, d=1024)
Parameters	365.4M: base 353.4M + 12.0M (Engram readers 8.4M, candidate choice 2.1M, meaning head 1.4M, pointer gates)
Meaning vectors	384-d, mean of layer-8 states + head; prefixes query: / memory:
Languages	trained: en, it, fr, de, es, pt; evaluated without training: ro, el, ru
Files	model.safetensors (bf16), onnx/engram_lm.{int8,int4}.onnx, onnx/embedder.{int8,int4}.onnx, dynamic_engram/ (library)

Results

Held-out data (a synthetic life never seen in training, held-out QA test sets); one training run; 95% intervals and every other number in the paper.

Memory-only mode — the prompt holds the question alone (with today's date):

Questions	Correct	Wrong	Silent
Personal memories, "who gave me…?" / "with whom did I see…?", in the memories' language, training phrasings (n=110)	95.5%	4.5%	0%
The same kind of questions in English, held-out phrasing (n=70)	8.6%	2.9%	88.6%
Encyclopedic QA, 6 fine-tuning languages (n=1,642)	2.2%	2.9%	94.9%

• The facts come from the memory: the same data and steps without memory are correct on 2.7% of the first row (they guess names); with the memory emptied, nothing is answered.
• The pointer carries the facts: on the personal questions the memory can surface, trained without it the model falls from 73.8% to 6.7%; without the readers inside the network, it is about as accurate.
• Scale: with 100,000 unrelated memories added to the store, memory-only accuracy on personal memories is 73.0% (73.8% without them, on the questions the memory can surface).
• Silence: silent on 98.5% of the unsupported evaluation questions (mostly encyclopedic). But when the memory that answers a first-row question is removed and similar memories remain, the model still answers all 110, usually with the giver of another gift.

Hybrid mode — 3 sources in the prompt plus the memory: 48.0% exact match on answerable questions (personal memories 93.7%), 27.7% of unsupported questions answered, 70.6% of counterfactuals answered with the injected fact. The same fine-tuning without memory scores the same (47.9%, 28.8%, 68.2%): with the text in the prompt, the memory adds nothing measurable.

Against RAG — same personal-memory questions, realistic retrieval, every answerable question counted. Memory-only mode answers 61.7% correctly with a 25-token prompt; the same data without memory reading the top 1 or 3 memories (RAG) answers 71.7% and 72.8%, with 51 and 103 tokens; the gap is mostly on the questions in English. On 2 CPU threads (int8) an answer takes 396 ms against 600 and 829 ms: 1.5 and 2.0 times faster, crediting RAG with the count memory it does not build (times measured on this model's graph with the memories in the prompt). On encyclopedic passages the gap is 2.7–5.7 times, but there RAG answers far more (32.7% against 1.8%).

A second base model — the same recipe on Qwen3-0.6B-Base improves the search by meaning (for questions in another language, the gold memory ranks first among the recalled items 70% of the time, against 46%), and the memory still brings facts in (19% correct against 4% without memory), but reading is far weaker than here and, with sources in the prompt, the memory hurts. That model is not released.

Related work — User as Engram (Li, 2026) shares the goal (per-user facts in an n-gram memory, asked about without the facts in the prompt) by writing optimized vectors into an Engram-pretrained model; here nothing is written into the model, an off-the-shelf model is fine-tuned once, and the question need not end like the stored fact.

CPU inference

Final weights, 2 CPU threads, 40 held-out questions, memory-only and hybrid mixed; time per answer covers building the count memory, reading the prompt and decoding (recall from the store is not included). Measured apart (40 personal and 40 encyclopedic questions, int8): with the question alone in the prompt an answer takes about 0.4 s; with three memories in the prompt, about 0.8 s; with three encyclopedic passages, about 2.5 s.

Runtime	Same answers as PyTorch	ms per answer	Tokens/s	Language model file
ONNX fp32	40 / 40	1,055	27.4	1.7 GB
ONNX int8	40 / 40	1,082	26.4	434 MB
ONNX int4	35 / 40	973	65.3	296 MB

int8 gives the same answers as fp32 at a quarter of the size, but on two threads it is not faster. int4 generates fastest but changes 5 of 40 answers (exact match 46.2% instead of 53.8%). Answers are a few tokens long, so most of the time per answer is spent outside generation. Memory-drafted decoding did not change any answer.

Training

• Data. Extractive QA, keeping answers that occur verbatim in their text: SQuAD v2 (en, CC BY-SA 4.0), SQuAD-it (machine translation of SQuAD), SQuAD v1.1-pt (nunorc/squad_v1_pt), SQAC (es, CC BY-SA 4.0), GermanQuAD (de, CC BY 4.0), PIAF (fr, MIT); plus synthetic multi-year personal memories generated by code (dates, moves, gifts, dinners, unknown people). No real user conversation was used. The training set is not released.
• Examples. Answerable, unsupported (the supporting text removed from sources and memory), natively unanswerable (SQuAD v2) and counterfactual (the answer replaced in the injected text; the target follows the text); hybrid and memory-only modes. A memory-only answer is trained only where the count memory can surface it (the first content word of the answer is among its candidates); elsewhere the target is the empty answer. An audit checks the data before training.
• Meaning channel. Layers 0–8 and the head are distilled from granite-embedding-97m-multilingual-r2 (Apache-2.0) for 2,000 steps, then trained jointly with the language-model loss. The teacher is not needed at inference.
• Optimization. One epoch (54,377 examples, 3,399 steps, effective batch 16), AdamW, cosine schedule, fp16, one T4 GPU.

Limitations

• It can invent among similar memories. If the memory that answers a question is missing but similar ones are there (other gifts, other films), the model usually answers with a name from them instead of staying silent, and strict=True lets all of these answers through (110 of 110 in our probe), since each name occurs in another recalled memory. We attribute it to the training data, which lacks this case; the training code has an option to add it (--support-negatives), not used for these weights.
• Memory-only answers are short facts that the memory associates with the question's words or meaning; multi-sentence answers and reasoning across items need the hybrid mode. On encyclopedic questions the memory-only mode barely works (2.2% correct): use the hybrid mode. A personal memory asked about in another language is mostly met with silence.
• In the hybrid mode the model answers about a quarter of the unsupported questions, like an ordinary fine-tuned reader.
• The count memory does not order events in time: for "where does Giorgio live now?" use the hybrid mode, which puts the most recent memory in the prompt.
• It answers with short spans or stays silent: it is a memory reader, not a conversational assistant.
• Among many similar memories, a question in another language reaches the right one less often than a question in the memory's own language.
• Synthetic personal memories are templated (80% Italian, 20% English) and the QA data is extractive; one run per system. The base model was trained for retrieval-augmented answering; on Qwen3-0.6B-Base the same recipe transfers only in part (see the paper).
• The support check is lexical: a correct paraphrase that is not in the memory is silenced.
• The meaning channel inherits the biases of its teacher.

License

Apache-2.0, as the base model. The weights were fine-tuned on the datasets listed above under their own licenses.

Citation

@misc{violini2026dynamicengram,
  title     = {Dynamic Engram: Answering from an Exact n-gram Memory Injected at Inference Time},
  author    = {Violini, Marcello and Nargiso, Lorenzo},
  year      = {2026},
  publisher = {Zenodo},
  doi       = {10.5281/zenodo.23803703},
  url       = {https://doi.org/10.5281/zenodo.23803703},
  note      = {TetiAI S.r.l.}
}