textilelabs/Loom-Spark-1.5

Loom Spark 1.5

Second of the Loom models · Textile Labs · successor to Loom Spark v1 (which remains available, unchanged)

Loom Spark 1.5 keeps everything that made the first Loom different — trained from scratch to trade actual knowledge for wisdom: self-aware, honest about its limits, warm, and skilled at exactly one real superpower — forming clean web-search queries for its agent harness to execute.

What changed from v1

	v1	1.5
parameters	7.6M	12.3M (384d × 6 layers × 6 heads)
offline discipline	occasional <lookup> leaks	zero leaks in testing (mode markers on every turn)
fake <result> blocks	possible outside harness	structurally impossible (turns end at the query)
non-sequitur answers	sometimes	fixed at the curriculum level
math & small facts	shaky	heavier training weight

Validation loss 0.326 vs v1's 0.337 — and the wins that matter are behavioral: in a 30-question offline battery, v1 leaked tool markup into plain chat constantly; 1.5 leaked zero times, and answers identity/restraint questions correctly from every angle we tried.

Prompt format (IMPORTANT — it is sensitive to format)

Same protocol as v1, but the mode marker now rides on EVERY user turn:

<tools:off>
<tools:off><user> what year did the Titanic sink?
<loom>

With tools on, the model ends its turn with <lookup>query</lookup> followed by <|endoftext|>; YOUR harness executes the search and splices <result>…</result> before continuing with <loom>. Stop generation at <|endoftext|> or <user>.

Option A — plain transformers

from transformers import GPT2LMHeadModel, AutoTokenizer
import torch

tok = AutoTokenizer.from_pretrained("TextileLabs/Loom-Spark-1.5")
model = GPT2LMHeadModel.from_pretrained("TextileLabs/Loom-Spark-1.5")

prompt = "<tools:off>\n<tools:off><user> who are you?\n<loom>"
ids = tok(prompt, return_tensors="pt").input_ids
out = model.generate(ids, max_new_tokens=90, do_sample=True, temperature=0.8,
                     top_k=50, pad_token_id=tok.eos_token_id)
print(tok.decode(out[0][ids.shape[1]:], skip_special_tokens=False))

This is a standalone second-generation model — v1 keeps its own repo and card. Tested on transformers ≥ 4.40 and Python 3.9–3.13. The widget above prefills the correct format — keep both <tools:…> markers or quality drops sharply.

Option B — llama.cpp / Ollama (offline persona)

loom-spark-1.5-f32.gguf carries the weights plus the custom BPE tokenizer with all nine special tokens embedded. The ollama/Modelfile in this repo produces the offline persona (greetings, identity, honest deferrals):

ollama pull hf.co/textilelabs/Loom-Spark-1.5
ollama create loom-spark-15 -f Modelfile
ollama run loom-spark-15 "hi"

Raw runners can't execute searches — if a reply contains a <lookup>…</lookup> query, that's the model saying "I'd look this up". For real internet answers, use the harness.

Option C — the harness (with internet)

Same package as v1 (harness/ folder here) — it auto-detects 1.5's format:

pip install ./harness
loom-chat                    # terminal agent, live DuckDuckGo searches
loom-web --port 7860         # local chat GUI with tools toggle + lookup feed

Architecture & training

Decoder-only transformer, pre-LN GELU blocks, tied embeddings, learned positions. 6 layers / 6 heads / d_model 384 / context 256 / vocab 4096 (fresh custom BPE). 12,318,... ≈ 12.3M params.

Trained entirely on our procedurally generated curriculum (zero external datasets) on a CPU-only Dell OptiPlex i5-4690: 3,914 steps ≈ 32M tokens over ~5.8 hours, final validation loss 0.343 (best checkpoint 0.326, shipped).

Limitations (by design)

Still knows almost nothing — that remains the point. Not for facts, medicine, law, finance, or anything where being wrong costs more than company.