textilelabs/Loom-Weave-3

Loom Weave 3

31.5M parameters · 16 layers · 1024 context · Textile Labs

The first Loom that reads real prose. Successor to Loom Weave 2 — and deliberately smaller than it (Weave 2 was 59.65M). Trained from scratch: randomly initialised weights, nothing fine-tuned from anyone's checkpoint.

It decides when to look something up, writes the query, reads what comes back from live Wikipedia, and answers from it — saying it checked rather than knew. No earlier Loom could read real, messy prose; the family's whole weakness was that curated results read well and live articles didn't. This is the model that closes that gap.

you            who wrote dracula
Loom Weave 3   <lookup>dracula</lookup>
harness        ← Dracula is an 1897 Gothic horror novel by Irish author Bram Stoker.
Loom Weave 3   I think Irish author Bram Stoker. Worth saying I checked rather than knew it.

Why it's smaller than Weave 2

Weave 2 (59.65M) failed for method, not size — a thin instruction veneer over a pretraining register, train and eval written by the same hand. Weave 3 is half the size and far better, because tokens-per-parameter and how it's trained matter more than parameter count. That has precedent: Loom Spark 3 (12.2M) succeeded and beat Spark 2 (19.9M). Bigger is not the goal; behaviour is.

What changed

The reading skill is a real circuit, and it only formed once two things were right:

lever	what went wrong before	fix
learning rate on the attention matrices	trained 8× too low, so the copy/reading circuit never formed — the model was fluent but couldn't pull "Paris" out of a sentence containing "Paris"	Muon LR set to the family standard (0.025); reading appeared
what the loss grades	it graded every token equally, spending most of its effort on raw web text	loss is masked to the reply tokens; whole conversations packed into each block
how long it trained	over-trained a tiny corpus into memorised, garbled output	short training, kept the best-validation checkpoint
admitting personal limits	learned "I can't know your name" but not "I can't know what you did" — so it hallucinated on other personal questions	added varied personal-unknowable examples; it now declines all of them

The search harness

The model decides a search is needed and writes the query. harness.py does the rest:

• searches the model's query and the subject it can see in your question
• prefers the real article over lists, films, albums and disambiguation pages
• reads the intro first, further only when the intro has no answer of the right kind
• strips brackets and pronunciation guides, so real text looks like training text
• hands back one sentence, not a paragraph — a model this size misreads paragraphs

Measured behaviour

Same harness and settings a user gets. Run on 2026-09-20.

The acceptance battery, row by row (total ties the best in the Loom family):

row	Loom Weave 3
A · says its own name	10/12
B · its own name under rough typing	10/12
C · 5-turn conversation stays on thread	5/5
D · answers from a search result	5/5
E · follow-up answered from the same result	2/5
F · says it looked, after a lookup	5/5
G · never claims a lookup it didn't make	16/16
H · admits what it can't know about you	8/8
I · says when a result doesn't contain the answer	0/5
J · never leaks a search tag with tools off	28/28
K · stops on its own	12/12
L · searches when it should, not for your private things	19/20
total	120/133

End to end, everyday questions it had never seen, live Wikipedia, the model writing its own query, scored on the final answer:

set	decided to search	wrote its own query	answer reached the model	answered right
tuning	20/20	20/20	14/20	11/20
held-out	20/20	20/20	12/20	6/20

It is the first Loom to read live prose above a coin-flip on the tuning set — but read the held-out number honestly: on unfamiliar, messy passages it is right about a third of the time.

Read this before you use it

Every point here was measured.

• Held-out live reading is ~30%. "I looked that up" means it searched — not that it read the result correctly. Run harness.py --show and trust the sentence it read.
• It is not a calculator. Small sums are out of scope (0/6). Use a tool for arithmetic.
• Follow-up questions from the same result are weaker (~40%).
• It never says a result doesn't contain the answer — it answers from whatever it read.
• It rarely asks a clarifying question on an ambiguous request.
• It reads a supplied result far better than it reads live search (row D is 100%).
• Harness search is Wikipedia only — time, weather, news and prices can't be answered.
• It is a small reader/assistant, not a chat stylist. Voice is plain and brief by design.

What it does do, reliably: knows what it is, stops on its own, holds a conversation, admits what it can't know about you (8/8), and never claims a lookup it didn't make (16/16).

Usage — the harness

python3 harness.py "who wrote dracula"
python3 harness.py                          # interactive
python3 harness.py --show "how tall is mount everest"   # see what it searched and read
python3 harness.py --no-tools "who are you"

Stdlib + certifi. Wikipedia needs no API key. Never feed a failed lookup back as a result — the model will answer from the error text; harness.py fails loudly instead.

Usage — Ollama

ollama run hf.co/textilelabs/Loom-Weave-3 "who are you"

template and params are read automatically.

Usage — transformers

import torch
from transformers import AutoTokenizer, AutoModelForCausalLM

tok = AutoTokenizer.from_pretrained("textilelabs/Loom-Weave-3")
model = AutoModelForCausalLM.from_pretrained("textilelabs/Loom-Weave-3").eval()
eot = tok.convert_tokens_to_ids("<|eot|>")

def ask(message, tools=False):
    p = f"<tools:{'on' if tools else 'off'}>\n<user>\n{message}\n<|eot|>\n<loom>\n"
    ids = tok(p, return_tensors="pt", add_special_tokens=False).input_ids
    with torch.no_grad():
        out = model.generate(ids, max_new_tokens=64, do_sample=False, eos_token_id=eot,
                             pad_token_id=tok.convert_tokens_to_ids("<|pad|>"))[0]
    return tok.decode(out[ids.shape[1]:], skip_special_tokens=False).replace("<|eot|>", "").strip()

Prompt format is exact: <tools:off>\n<user>\n{message}\n<|eot|>\n<loom>\n.

How it was built

architecture	Llama — 16 layers × 384d, GQA (6 heads / 2 KV), SwiGLU, RoPE, tied embeddings
parameters	31,469,952
context	1,024
vocabulary	16,384 custom BPE
optimiser	Muon (LR 0.025) on the 2D hidden matrices, AdamW on embeddings and norms
loss	masked to the reply tokens; whole conversations packed per block (FFD)
schedule	warmup → stable → cosine decay (WSD); best-validation checkpoint kept
corpus	173,043 conversations; grounded-reading rows carry real encyclopedic prose
training	~1,280 steps · ~336M tokens seen · 10 epochs · from random init
hardware	Kaggle dual T4 (GPU) · ~2 hours

Files

config.json / model.safetensors           the model
tokenizer.json / tokenizer_config.json    custom BPE tokenizer, 16,384 tokens
loom-weave-3-f16.gguf                      for Ollama / llama.cpp (lookup tags USER_DEFINED)
harness.py                                 runnable search harness
template / params                          read automatically by `ollama run hf.co/...`
Modelfile                                  for building locally
ATTRIBUTION.md                             required credits for the training corpora

Training data

slice	source
grounded reading of real prose	SQuAD 2.0 (CC BY-SA 4.0) · Wikipedia (CC BY-SA)
when to reach for a tool	MASSIVE (CC BY 4.0) · CLINC150 (CC BY 3.0)
instruction following	databricks-dolly-15k (CC BY-SA 3.0)
multi-turn dialogue structure	OpenAssistant OASST1 (Apache 2.0)
tokenizer coverage of real prose	HuggingFaceFW/fineweb-edu (ODC-By 1.0)
identity, limits, warmth, attribution	Textile Labs — written for Loom

No language model wrote any training query, and nothing is fine-tuned from anyone's checkpoint. No real user data was used. See ATTRIBUTION.md.

License

Model: MIT. Training data retains its original licences and attribution.