textilelabs/Loom-Spark-2

Loom Spark 2

19.9M parameters · 512 context · Textile Labs

A small conversational model that knows what it is, knows what it can't know, and asks for a tool instead of guessing. Successor to Loom Spark 1.8.

Trained from scratch in one hour on a 2013 desktop CPU. Randomly initialised weights, trained end to end — nothing fine-tuned from a pretrained base.

who are you              →  Loom, a small model from Textile Labs.
WHO ARE YOU???           →  I'm Loom — Textile Labs built me.
what is my sisters name  →  I can't know that unless you tell me.
i had a rough day        →  That sounds heavy. Want to talk about it?

Measured against Spark 1.8

Same probe structure 1.8 published, so this is like-for-like rather than cherry-picked.

	Spark 1.8	Spark 2
offline <lookup> leak	16/30 · 53%	0/30 · 0%
identity probes leaking <lookup>	0/12	0/12
clean single online lookup	8/10	10/10
identity correct	—	12/12
identity under CAPS / typos / filler	not trained for	6/6
self-termination without a Modelfile	needed one	12/12
parameters · context	18.85M · 256	19.87M · 512

The offline leak result is the one that matters most in practice. 1.8 reached for a lookup on more than half of all offline questions; Spark 2 did not do it once in thirty.

Honest limitations — read before relying on it

It can be given a tool result and asked to answer from it. That part is weak:

	score
answers correctly from a supplied <result>	2/5
says the result doesn't contain the answer	0/4
admits an unknowable personal fact	4/8

It does not reliably say "the result doesn't say" — it fabricates instead. If you feed it results, validate the output; do not treat a grounded answer as trustworthy.

It also has almost no world knowledge. With tools off it will decline factual questions, which is the intended behaviour, not a bug.

Why: one hour of training gives ~809 optimiser steps, and validation accuracy was still climbing steeply (0.47 → 0.54 over the final 200 steps) when the clock ran out. The model is under-trained rather than under-sized.

Two modes

<tools:off> (the default) — conversational. Identity, limits, warmth, brevity. No harness needed.

<tools:on> — it emits <lookup>query</lookup> and stops. Your harness runs the lookup and continues with a <result> block:

<tools:on>
<user>
what is the capital of Peru
<|eot|>
<loom>
<lookup>what is the capital of Peru</lookup><|eot|>
<result>
Lima is the capital and largest city of Peru.
<|eot|>
<loom>

Note the persona slice was trained entirely under tools:off, so identity questions asked with tools on will often be turned into a lookup. Keep tools off for chat.

Usage — the harness

harness.py in this repo is a working harness: it runs the lookup Loom asks for and feeds the result back. Wikipedia is used because it is free and needs no key — swap the search() function for anything else; the contract is just text in, text out.

python3 harness.py "who wrote Dracula"          # with lookups
python3 harness.py                              # interactive
python3 harness.py --no-tools "who are you"     # chat only

you  > who wrote Dracula
    [loom wants: 'who wrote Dracula']
    [result: Dracula is an 1897 Gothic horror novel by Irish author Bram Stoker...]

Three things any harness for this model needs, learned the hard way:

• Never feed a failed lookup back as a <result>. The model will earnestly try to answer from the error text. Fail loudly instead — harness.py does.
• Wikipedia returns 403 without a descriptive User-Agent.
• macOS system Python often needs certifi for TLS.

And the honest warning: with the grounded score at 2/5, the final answer is frequently wrong even when the lookup and the result are both perfect. The example above returns "Scrucula" from that passage. Treat the retrieved <result> as the trustworthy part and the model's summary of it as unreliable.

Usage — Ollama

ollama run hf.co/textilelabs/Loom-Spark-2 "who are you"
# Loom, a small model from Textile Labs.

Ollama reads the template and params files in this repo — nothing to set up. The template defaults to tools:off. To build locally: ollama create loom-spark-2 -f Modelfile.

Usage — transformers

import torch
from transformers import AutoTokenizer, AutoModelForCausalLM

tok = AutoTokenizer.from_pretrained("textilelabs/Loom-Spark-2")
model = AutoModelForCausalLM.from_pretrained("textilelabs/Loom-Spark-2").eval()
eot = tok.convert_tokens_to_ids("<|eot|>")

def ask(message, tools=False):
    p = f"<tools:{'on' if tools else 'off'}>\n<user>\n{message}\n<|eot|>\n<loom>\n"
    ids = tok(p, return_tensors="pt", add_special_tokens=False).input_ids
    with torch.no_grad():
        out = model.generate(ids, max_new_tokens=64, do_sample=False,
                             eos_token_id=eot,
                             pad_token_id=tok.convert_tokens_to_ids("<|pad|>"))[0]
    return tok.decode(out[ids.shape[1]:], skip_special_tokens=True).strip()

ask("who are you")                       # -> 'Loom, a small model from Textile Labs.'
ask("what is the capital of Peru", True) # -> '<lookup>what is the capital of Peru</lookup>'

Prompt format is exact: <tools:off>\n<user>\n{message}\n<|eot|>\n<loom>\n.

Files

config.json / model.safetensors           the model
tokenizer.json / tokenizer_config.json    custom BPE tokenizer, 4,096 tokens
loom-spark-2-f16.gguf                     40MB, for Ollama / llama.cpp
harness.py                                runnable harness — runs lookups, feeds results back
template / params                         read automatically by `ollama run hf.co/...`
Modelfile                                 for building locally
ATTRIBUTION.md                            required credits for the training corpora

Training data

Built from openly licensed corpora of real human text, plus a persona curriculum written for Loom. See ATTRIBUTION.md — several of these licences require credit.

slice	source
grounded reading, and "the result doesn't say"	SQuAD 2.0 (CC BY-SA 4.0)
when to reach for a tool	MASSIVE (CC BY 4.0) · CLINC150 (CC BY 3.0)
instruction following	databricks-dolly-15k (CC BY-SA 3.0)
multi-turn dialogue structure	OpenAssistant OASST1 (Apache 2.0)
identity, limits, warmth, brevity	Textile Labs — written for Loom

~11.4M tokens, 43% multi-turn. Validation is a held-out split of the same corpora.

License

Model: MIT. Training data retains its original licences and attribution.