spectral-embed-v1-140m

A closed-form, CPU-only linear-spectral embedding model. Deterministic, dependency-light, and roughly 65× faster than a small neural teacher on single-sentence inference.

spectral-embed-v1-140m replaces a neural transformer backbone with a fixed, deterministic feature extractor plus a single trained linear operator. The entire inference path is $z = Wx + b$, followed by an L2 normalization. No GPU, no PyTorch, no ONNX runtime, no tokenizer, no learned vocabulary. The runtime dependency is NumPy.

---

Model summary

Field	Value
Architecture	Linear spectral operator ($z = Wx + b$, then L2-normalized)
Parameters	139,471,360 (~139.5 M) — $W$ (2048×65536) + P_raw + $b$ + aux
Output dimension	2048
Serving dtype	float32
Artifact size	558 MB (.npz, uncompressed)
Runtime dependencies	Python 3.9+, NumPy
Hardware target	Any CPU (edge, IoT, serverless, on-prem)
Deterministic	Yes — identical inputs produce byte-identical outputs
License	Custom Community License v1.0 (see License)

---

Intended use

• Semantic search and retrieval on CPU-only infrastructure.
• Text classification and clustering where latency and footprint matter.
• Edge and on-prem deployments where no GPU is available.
• Batch embedding pipelines where a neural teacher is too slow or too heavy.

Out of scope

• Long-context reasoning or structured generation.
• Tasks requiring token-level cross-attention behaviour.
• Claims of benchmark parity with 300 M–600 M neural encoders.

---

Quickstart

Local Python

from app import SpectralEncoder

enc = SpectralEncoder.from_npz("spectral-embed-v1-140m.npz")

v = enc.embed([
    "How do I reset my account password?",
    "Steps to change login credentials and recover access.",
    "The weather forecast calls for rain tomorrow.",
])

sim = v @ v.T
print(sim[0, 1])   # anchor vs. paraphrase   → high
print(sim[0, 2])   # anchor vs. unrelated    → low

Docker + Ollama-compatible HTTP API

docker build -t spectral-embed-v1-140m .
docker run -d -p 7860:7860 --name spectral-embed spectral-embed-v1-140m

curl http://localhost:7860/api/embed \
  -H "Content-Type: application/json" \
  -d '{"model": "spectral-embed-v1-140m", "input": "How do I reset my account password?"}'

Response:

{
  "embeddings": [[-0.0124, 0.4512, 0.0891, -0.6120, 0.2319, "..."]],
  "dim": 2048,
  "model": "spectral-embed-v1-140m"
}

The server also exposes:

Endpoint	Purpose
GET /	HTML playground (index.html)
GET /health	Liveness probe
GET /api/tags	Ollama-style model list
POST /api/embed	Ollama-compatible embedding endpoint

---

Evaluation

MTEB results

Evaluated with the official mteb harness (v2.22.2) using a custom NumPy encoder wrapper, at Hub revision 39169ba7aa8d1de29da32d97a6129bca790e9193. Scores are the task's main metric, ×100.

Task	Type	Metric	Score
STS12	STS	Spearman (cosine)	61.31
STS13	STS	Spearman (cosine)	65.44
STS14	STS	Spearman (cosine)	69.79
STS15	STS	Spearman (cosine)	77.76
STS16	STS	Spearman (cosine)	70.88
STSBenchmark	STS	Spearman (cosine)	70.13
Banking77Classification.v2	Classification	Accuracy	69.45
AmazonCounterfactualClassification	Classification	Accuracy	69.80
ArguAna	Retrieval	nDCG@10	30.92
NFCorpus	Retrieval	nDCG@10	14.59
SciFact	Retrieval	nDCG@10	32.06

Category	Tasks	Average
STS	6	69.22
Classification	2	69.63
Retrieval	3	25.86

Read this carefully:

• This is a hand-picked subset, not the official MTEB(eng, v2) average, and should not be compared directly to leaderboard averages.
• NFCorpus is part of this model's training mix, so its score is not zero-shot (declared as training data in the model metadata). It is included in the Retrieval average below.
• Results for 35 tasks have been computed so far: 31 of the 41 tasks in MTEB(eng, v2) plus 4 extra tasks outside it. The table shows an 11-task selection from those, and the official MTEB(eng, v2) average cannot be computed until the remaining 10 tasks are run.
• Retrieval is the weakest category by a wide margin. For retrieval-critical workloads, evaluate on your own data before adopting this model.

Internal proxy metrics

Measured on an internal mini-MTEB proxy (10 tasks spanning Classification, Clustering, PairClassification, Reranking, Retrieval, STS, Summarization). These are not official MTEB numbers.

Metric	Value
Mini-MTEB average (10 tasks, internal proxy)	43.9
Gate E1 semantic triples	100 % (10/10)
Effective dimensional rank	1342 / 2048
Operator condition number	70.3
Inference latency (CPU, single text)	~0.45 ms
Memory footprint (serving)	~560 MB

Per-source fidelity (held-out report split)

Source	N	Score
equation	50	0.561
emotion	88	0.542
news	2,110	0.456
universe	431	0.448
wiki	208	0.443
nfcorpus	1,000	0.437
science	76	0.385

Small-N sources (equation, emotion, science) have wide confidence intervals. The 2,110-row news slice is the tightest estimate.

What this model does not do

This model is trained on a mixed-domain corpus covering the sources above. It is not a general-purpose replacement for a neural encoder. On tasks outside that distribution — long-form reasoning, code, multilingual retrieval, or anything requiring sub-word compositionality beyond the featurizer's channels — performance will be far below a neural baseline.

Comparison to the teacher

Distilled from Qwen3-Embedding-4B. The spectral student is ~65× faster and ~14× smaller in memory footprint than the teacher. It is not a substitute for the teacher on benchmark-critical workloads.

	Qwen3-Embedding-4B (teacher)	spectral-embed-v1-140m
Latency	~30 ms / sentence (CPU)	~0.45 ms / sentence
Footprint	~8 GB	~560 MB
Output dim	2560	2048
GPU required	Recommended	No

Reproducing the MTEB results

The model is not a Sentence-Transformers or transformers model, so MTEB runs it through a thin wrapper (spectral_mteb_wrapper.py) that exposes an encode method, attaches an MTEB ModelMeta, and returns L2-normalised embeddings.

pip install numpy mteb
python run_mteb_smoke.py     # STS12 only, about 20 s
python run_mteb_subset.py    # the 11-task subset reported above

import mteb
from spectral_mteb_wrapper import SpectralEmbedEncoder

model = SpectralEmbedEncoder(
    npz_path="spectral-embed-v1-140m.npz",
    revision="39169ba7aa8d1de29da32d97a6129bca790e9193",  # otherwise results are filed under "local"
)
tasks = mteb.get_tasks(tasks=["STS12"])
results = mteb.evaluate(model, tasks)

---

Architecture

1. Deterministic multiscale featurizer

Text is lifted into a sparse 65,536-dimensional coordinate space using seven hash-based channels:

Channel	Description	Weight
w:	word unigrams	2.0
bi:	word bigrams	3.0
ch:	character n-grams (n = 3, 4)	1.0
wp:	positionally-binned unigrams	1.0
sk:	skip-grams with distance decay (d ≤ 3)	1.5 / d
tri:	word trigrams	2.0
mh:	modifier-head pairs (adj/det + noun)	2.5

Each feature key is hashed to a coordinate with CRC32("<seed>:<key>") mod 65536. Term frequencies are dampened sub-linearly and scaled by the channel weight $\alpha_c$:

$$w=(1+\ln \mathrm{t}\mathrm{f}){\alpha }_{c}w\; =\; \backslash bigl(1\; +\; \backslash ln\; \backslash mathrm\{tf\}\backslash bigr)\backslash ,\backslash alpha\_c$$

The resulting 65,536-dim vector $x$ is L2-normalized per input.

2. Linear operator

The featurized vector is projected onto the teacher's principal semantic subspace via a single $2048 \times 65536$ matrix, followed by a learned bias and L2 normalization:

$$z=Wx+b,\widehat{z}=\frac{z}{\parallel z{\parallel }_2}z\; =\; Wx\; +\; b,\; \backslash qquad\; \backslash hat\{z\}\; =\; \backslash frac\{z\}\{\backslash lVert\; z\; \backslash rVert\_2\}$$

with $x \in \mathbb{R}^{65536}$, $W \in \mathbb{R}^{2048 \times 65536}$, and $b \in \mathbb{R}^{2048}$.

3. Anisotropic diagonal metric

Semantic distance is decoded through a calibrated per-dimension metric tensor $M \in \mathbb{R}^{2048}$ (shipped as M_diag):

$$D(a,b)=\sqrt{\sum _{k=1}^{2048}{M}_k({\widehat{a}}_k-{\widehat{b}}_k{)}^2}D(a,\; b)\; =\; \backslash sqrt\{\backslash sum\_\{k=1\}^\{2048\}\; M\_k\; \backslash ,\backslash bigl(\backslash hat\{a\}\_k\; -\; \backslash hat\{b\}\_k\backslash bigr)^2\}$$

Default usage for cosine similarity uses only $\hat{z}$ and is unaffected by $M$:

$$\cos (a,b)=\widehat{a}\cdot \widehat{b}\backslash cos(a,\; b)\; =\; \backslash hat\{a\}\; \backslash cdot\; \backslash hat\{b\}$$

Downstream tasks that consume distances can use M_diag directly for slightly better ranking behaviour on calibrated domains.

---

Training procedure (summary)

• Teacher: Qwen3-Embedding-4B, unit-normalized, 2560-dim.
• Featurizer: fixed — the seven-channel hash described above.
• Operator: ridge regression on the teacher's top-2048 principal subspace, followed by per-dimension gain calibration and metric estimation on a held-out split.
• Curriculum and refit: a private multi-stage pipeline produces the final artifact. The pipeline itself is not distributed.

The teacher cache, the distillation corpus, and the build scripts are not released. Only the fused operator is published.

---

Files in this repository

File	Purpose
spectral-embed-v1-140m.npz	The model — W, b, k, H, hash_seed, M_diag, …
app.py	HTTP server and SpectralEncoder (NumPy + stdlib only)
spectral_mteb_wrapper.py	MTEB-compatible encoder wrapper
run_mteb_smoke.py	MTEB smoke test (STS12)
run_mteb_subset.py	MTEB 11-task subset run
index.html	Interactive playground served at /
Dockerfile	Container build
requirements.txt	numpy only (MTEB scripts additionally need mteb)
LICENSE	Custom Community License v1.0
README.md	This file

---

License

Licensed under the Spectral Embeddings Community License v1.0.

• ✅ Free for personal, academic, research, and internal-evaluation use.
• ✅ Free for commercial production use by individuals and organizations whose total annual revenue is below USD 14,000,000.
• ⚠️ Organizations at or above that threshold must obtain a commercial license from the licensor before any production or revenue-generating use.

This license is not OSI-approved and is not Apache 2.0. See LICENSE for full terms.

The model weights may not be used to train a competing commercial embedding service without a commercial license.

Commercial licensing contact: admin@betaprecision.com

---

Citation

@misc{spectral_embed_v1_140m,
  author = {Mullings, J.},
  title  = {spectral-embed-v1-140m: A CPU-only linear-spectral embedding model},
  year   = {2026},
  url    = {https://huggingface.co/JMullings/spectral-embed-v1-140m}
}