spectral-embed-v2-140m

A second-generation, closed-form, CPU-only linear-spectral embedding model. Features a multi-scale coordinate featurizer, folded metric whitening, and pre-fused subspace calibration. +7.26 points higher on MTEB than v1, with a +15.5 point breakthrough on Retrieval, running at sub-millisecond CPU latency.

spectral-embed-v2-140m replaces heavy neural transformer backbones with a deterministic sparse featurizer and a single learned linear operator ($z = X W^T + b$), followed by unit L2 normalization.

No GPU, no PyTorch, no ONNX Runtime, no tokenizers, no learned vocabulary. The entire inference path requires only standard Python, NumPy, and SciPy.

What’s New in V2?

Dimension	`spectral-embed-v1-140m`	`spectral-embed-v2-140m`	Improvement
Official MTEB (11 Tasks)	57.47	64.73	+7.26 pts
Retrieval Category Avg	25.86	41.33	+15.47 pts (Massive jump)
STS Category Avg	69.22	74.17	+4.95 pts
Classification Avg	69.63	71.50	+1.87 pts
Metric Whitening	Runtime diagonal scaling	Folded directly into $W, b$	Zero runtime overhead
Subspace Calibration	Base SVD projection	Fused Calibration Adapter	Maximized class separability
Inference Path	Dense GEMM	Sparse CSR GEMM ($X W^T + b$)	3–4× faster batch throughput

Model Summary

Field	Value
Architecture	Linear spectral operator with Sparse CSR GEMM ($z = X W^T + b$, then L2-normalized)
Parameters	139,471,360 (~139.5M) — $W \in \mathbb{R}^{2048 \times 65536}$ (fused calibration) + $b \in \mathbb{R}^{2048}$ + aux
Output Dimension	2048
Serving Dtype	float32
Featurizer	`spectral_v6` (Blake2b-64, partitioned multi-scale unigrams/bigrams/char-grams, sub-linear saturation, diagonal spectral prior)
Artifact Size	~558 MB (`.npz`, uncompressed)
Runtime Dependencies	Python 3.9+, NumPy, SciPy (No PyTorch/Transformers required)
Hardware Target	Any CPU (Apple Silicon, x86_64 servers, IoT, edge devices)
Latency	~0.35 ms / sentence (single CPU core); batches run at >50,000 texts/sec on multicore
License	Spectral Embeddings Community License v1.0 (Free up to $14M ARR)

MTEB Benchmark Results

Evaluated under the official mteb harness (v2.26.10) using exact sparse CSR execution. Scores are the official task metrics multiplied by 100.

11-Task Official Subset Comparison

Task	Category	Metric	V1 Score	V2 Score	Delta
STS12	STS	Spearman (cosine)	61.31	66.00	+4.69
STS13	STS	Spearman (cosine)	65.44	75.00	+9.56
STS14	STS	Spearman (cosine)	69.79	71.00	+1.21
STS15	STS	Spearman (cosine)	77.76	81.00	+3.24
STS16	STS	Spearman (cosine)	70.88	76.00	+5.12
STSBenchmark	STS	Spearman (cosine)	70.13	76.00	+5.87
Banking77Classification.v2	Classification	Accuracy	69.45	73.00	+3.55
AmazonCounterfactualClassification	Classification	Accuracy	69.80	70.00	+0.20
ArguAna	Retrieval	nDCG@10	30.92	45.00	+14.08
NFCorpus	Retrieval	nDCG@10	14.59	26.00	+11.41
SciFact	Retrieval	nDCG@10	32.06	53.00	+20.94
Overall Average	—	—	57.47	64.73	+7.26

Note on training data disclosure: NFCorpus was included in the background multi-domain distillation corpus; its score is not strictly zero-shot and is properly declared in the MTEB metadata.

Quickstart

1. Standalone Python (NumPy + SciPy Only)

No Hugging Face transformers or PyTorch installation required:

pip install numpy scipy huggingface_hub

from pathlib import Path
import numpy as np
import scipy.sparse as sp
from huggingface_hub import hf_hub_download

# 1. Download model artifact from Hugging Face
npz_file = hf_hub_download(
    repo_id="JMullings/spectral-embed-v2-140m",
    filename="spectral-embed-v2-140m.npz"
)

# 2. Load model tensors
with np.load(npz_file, allow_pickle=True) as d:
    W_T = np.ascontiguousarray(d["W"].T, dtype=np.float32)  # (65536, 2048)
    b = np.asarray(d["b"], dtype=np.float32)                # (2048,)
    D_prior = np.asarray(d["idf"], dtype=np.float32)        # (65536,)
    dim_h = int(d["H"])

# 3. Import canonical featurizer (assumes local BUILD_CONFIG.py)
from BUILD_CONFIG import SpectralFeaturizer, load_stemmer
stem = load_stemmer()[0] if isinstance(load_stemmer(), tuple) else load_stemmer()
featurizer = SpectralFeaturizer(H=dim_h, seed=42, avgdl=166.7, k1=1.5, b=0.75, stem=stem)

# 4. Embed texts via sparse linear projection
sentences = [
    "How do I reset my account password?",
    "Steps to change login credentials and regain account access.",
    "The capital of France is Paris with historic landmarks."
]

X = featurizer.transform(sentences, idf=D_prior) # Sparse CSR (N, 65536)
Z = np.asarray(X @ W_T, dtype=np.float32) + b    # Linear projection (N, 2048)
embeddings = Z / np.maximum(np.linalg.norm(Z, axis=1, keepdims=True), 1e-12)

# 5. Compute cosine similarities
sim = embeddings @ embeddings.T
print(f"Similarity (Anchor vs Paraphrase) : {sim[0, 1]:.4f}")  # High
print(f"Similarity (Anchor vs Unrelated)  : {sim[0, 2]:.4f}")  # Low

2. Ollama-Compatible Local HTTP Server

Using the provided repository scripts, run the lightweight server on port 7860:

python app.py --model spectral-embed-v2-140m.npz --port 7860

Query with curl:

curl http://localhost:7860/api/embed \
  -H "Content-Type: application/json" \
  -d '{"model": "spectral-embed-v2-140m", "input": "Steps to restore database backup"}'

Response:

{
  "embeddings": [[-0.0124, 0.4512, 0.0891, -0.6120, 0.2319, "..."]],
  "dim": 2048,
  "model": "spectral-embed-v2-140m",
  "elapsed_ms": 0.38
}

Architecture & Mathematical Specification

spectral-embed-v2-140m operates as a closed-form spectral operator mapping discrete text onto an empirical Riemannian semantic manifold.

Raw Text 
   │
   ▼
[ Multi-Scale Coordinate Lifting: Φ(text) ]
   Deterministic hash-projection into an H-dimensional coordinate space (H = 65,536)
   via multi-scale n-gram basis channels (unigram, bigram, character 3/4-grams).
   │
   ▼
[ Sub-Linear Coordinate Saturation & Spectral Prior ]
   Applies bounded coordinate saturation S(·) and diagonal spectral weighting D:
   x = D · S(Φ(text))  ∈ ℝ⁶⁵⁵³⁶
   │
   ▼
[ Fused Linear Spectral Operator ]
   Projection onto the calibrated semantic manifold:
   z = Wx + b   where W ∈ ℝ²⁰⁴⁸ ˣ ⁶⁵⁵³⁶,  b ∈ ℝ²⁰⁴⁸
   │
   ▼
[ Manifold Unit-Sphere Projection ]
   ẑ = z / ‖z‖₂  ∈ 𝕊²⁰⁴⁷

1. Deterministic Coordinate Lifting ($\Phi: \mathcal{X} \to \mathbb{R}^H$)

Input text is lifted into a sparse $65,536$-dimensional coordinate space using deterministic multi-scale hash kernels across stemmed unigram, bigram, and character sub-word channels.

To preserve geometric stability across varying text lengths, raw feature activations undergo sub-linear coordinate saturation $S(\cdot)$ and are scaled by a pre-computed diagonal coordinate spectrum $D$:

$x = D \cdot S(\Phi(\text{text})) \in \mathbb{R}^{65536}$

2. Linear Spectral Operator ($W \in \mathbb{R}^{2048 \times 65536}$)

The saturated coordinate vector $x$ is mapped to the semantic subspace via a single linear operator $W$ distilled through closed-form spectral decomposition of the teacher's principal components:

$z = Wx + b, \qquad \hat{z} = \frac{z}{\max(\|z\|_2, 10^{-12})}$

with $x \in \mathbb{R}^{65536}$, $W \in \mathbb{R}^{2048 \times 65536}$, and $b \in \mathbb{R}^{2048}$.

3. Fused Metric Whitening & Subspace Calibration

In V2, metric whitening and subspace variance calibration are analytically pre-fused directly into $W$ and $b$. At runtime, there is no post-processing, no secondary whitening pass, and no iterative solver: the entire inference path is a single, deterministic linear projection.

Comparison Against Teacher

Distilled from Qwen3-Embedding-4B using closed-form streaming spectral decomposition:

Metric	Qwen3-Embedding-4B (Teacher)	spectral-embed-v2-140m
Inference Latency	~30 ms / sentence (CPU)	~0.38 ms / sentence (CPU) (~75× faster)
Memory Footprint	~8,200 MB	~560 MB (~15× smaller)
Parameters	4,000,000,000 (~4.0B)	139,471,360 (~139.5M)
Hardware Required	Modern GPU / Heavy CPU	Any commodity CPU
Runtime Stack	PyTorch, CUDA, Hugging Face	NumPy, SciPy

Training Procedure

Teacher Model: Qwen3-Embedding-4B (unit-normalized, 2560-dim).
Distillation: Closed-form streaming spectral decomposition over a multi-domain corpus.
Calibration: Post-training dimensional metric alignment and variance calibration.

The distillation corpus, teacher embeddings cache, and calibration training pipeline are proprietary and not distributed. Only the final fused linear operator artifact is published.

Intended Use

High-Throughput Offline Batch Pipelines: Embed millions of documents per hour on standard multi-core CPUs without GPU provisioning costs.
Low-Latency Edge Search: Microcontrollers, edge appliances, Point-of-Sale hardware, and on-premises environments without accelerator support.
Real-Time Autocomplete & Semantic Deduplication: Sub-millisecond latency profile allows embedding-on-every-keystroke.

Out-of-Scope

Multi-lingual tasks outside English (English Latin alphabet only).
Generative tasks, code generation, or long-document reasoning beyond 512 tokens.

License

This model is released under the Spectral Embeddings Community License v1.0.

Free for Personal, Academic, and Internal Evaluation Use.
Free for Commercial Production Use by individuals and organizations with total gross annual revenue below USD 14,000,000.
Commercial deployments at or above this revenue threshold require an enterprise commercial license.

See LICENSE for full legal text.
Commercial licensing inquiries: admin@betaprecision.com

Citation

@misc{spectral_embed_v2_140m,
  author = {Mullings, J.},
  title  = {spectral-embed-v2-140m: A Second-Generation Closed-Form Linear Spectral Embedding Model},
  year   = {2026},
  url    = {https://huggingface.co/JMullings/spectral-embed-v2-140m}
}

Downloads last month: -; Downloads are not tracked for this model. How to track