pocket-mind/pocket-embed-22m-128

Pocket Embed 22M — 128-dimensional English search

A frozen MiniLM encoder with a small fitted projection for compact search indexes. 128-dimensional vectors use 512 bytes each in float32: one third of the original 384-dimensional vector storage. Index metadata and overhead are additional.

On two confirmation tasks (ArguAna and SciDocs), it retained 95.9% of base384 mean nDCG@10 and scored 7.0% above raw base128 truncation. These are measured comparisons on this small evaluation set, not a general quality guarantee. Results for the original four tasks appear below.

Additional tests showed lower retention on FiQA: 88.6% for FP32 and 86.4% for INT8. Quora retention was 98.7% and 98.3%. BGE Small scored higher on both additional tasks; Pocket FP32 also scored slightly below raw128 on Quora. See the full additional audit for scores and confidence intervals. The compact index is a storage/quality tradeoff, not a claim of stronger retrieval than these full-dimensional encoders.

On two later fixed-model diagnostics, INT8 retained 95.9% of full MiniLM nDCG@10 on TREC-COVID and 91.2% on CQADupStack Webmasters. Both INT8 versus Pocket FP32 paired intervals included zero. Full BGE Small scored much higher on TREC-COVID (0.7465 versus Pocket FP32 0.4477); Webmasters favored full MiniLM. These results add coverage and limitations, not new confirmation of a fitted model. The later audit includes scores, uncertainty, batch settings and 132-byte vector storage checks.

Two more FP32 diagnostics show 82.9% retention of MiniLM on Touche2020 and 93.8% on CQADupStack Gaming. Pocket scored below full MiniLM and BGE Small on both tasks; its paired intervals versus MiniLM excluded zero. Touche has only 49 queries, and Gaming belongs to a previously measured task family. These tests did not measure INT8. The FP32 audit includes per-query results, uncertainty and reproduction code. These add limitations, not new confirmation of the fitted model.

Run a complete local example: eight downloadable passages, four labeled queries, and tested commands for indexing, search, storage inspection and evaluation in a private environment without model frameworks. The supplied fixtures demonstrate the workflow, not general retrieval quality.

Install the local CLI from a pinned 24 KB wheel in this repository to use pocket-search, pocket-search-evaluate, and pocket-search-compare without copying scripts or setting framework flags. Both graph variants passed exact parity checks against the original scripts. Version 0.1.1 defers model imports, allowing index inspection and compaction in a NumPy-only environment. Validation and startup scope preserve the original encoding results; this does not improve retrieval quality.

Try browser search: a free static demo that runs inference on your device. The first search downloads about 23 MB of INT8 model weights plus runtime files.

The full hosted demo suite passed in Chromium 141.0.7390.37 and Firefox 142.0.1 on headless Linux, including cache reuse, file import/export, stale-input handling, and token-limit warnings. Firefox required an isolated hosted rerun after a reload failure. WebKit has no full pass because its model-cache reload assertion fails. Versions, scope, and evidence; mobile viewport checks do not establish Safari, iOS, macOS, or physical-device compatibility.

The demo uses ONNX Runtime Web's pinned CPU-only WASM entry point. A matched Linux WebKit diagnostic reduced session initialization from about 159–162 seconds to 1.6–2.0 seconds with the same model and inputs. This measures one environment, not general browser speed; the cache failure remains. Measurements, source, and limitations.

Load a local .txt file with one short passage per line (up to 32), then download the complete ranked results as JSON with the query, cosine scores and pinned model revisions. The file and search text stay on your device. Editing inputs clears the old downloadable ranking. Local and hosted validation. If a query or passage exceeds the 256-token limit, the demo identifies the affected inputs. Downloaded rankings include full token counts and truncation metadata while retaining the original passage text. Counts include special tokens; exactly 256 is within the limit. Token coverage validation.

Quality retention varies by task: 97.8% on ArguAna, 91.4% on SciDocs. Both score lower than the full 384-dimensional baseline. The gain over raw 128-dimensional truncation on SciDocs is uncertain: its paired 95% interval includes zero. At 64 dimensions, SciDocs also scores slightly below raw baseline truncation. Use 128 dimensions for the evaluated storage/quality tradeoff.

The encoder architecture is unchanged. The fitted projection reduces embedding storage and the dimensions used in vector comparison. It does not reduce encoder parameter count or token encoding work. The optional INT8 ONNX export separately reduces the weight download; the default PyTorch weights remain float32. This uses established PCA and similarity distillation methods, with no claim of a new architecture.

Usage

pip install sentence-transformers==5.1.2 transformers==4.57.1

from sentence_transformers import SentenceTransformer

model = SentenceTransformer("pocket-mind/pocket-embed-22m-128", device="cpu")
documents = ["Bike repairs are available on Saturdays.",
             "The library lends books and audiobooks."]
vectors = model.encode(documents, normalize_embeddings=True)
query = model.encode(["Where can I fix my bicycle?"], normalize_embeddings=True)
best = int((query @ vectors.T).argmax())
print(documents[best])
print(vectors.shape)  # (2, 128)

No prompt or prefix is required. Always use this same model for queries and documents; these vectors are incompatible with the original MiniLM vectors. Re-encode your corpus when migrating. Longer inputs are truncated at 256 word pieces. Chunk long documents before encoding.

Optional 64-dimensional output:

vectors64 = model.encode(documents, truncate_dim=64, normalize_embeddings=True)

64 dimensions trade more retrieval quality for storage; the release gate applies only to 128 dimensions. The projection is evaluated at both sizes.

Search a local corpus

The CPU indexing example builds an exact-search index from JSONL passages using the pinned INT8 graph, then searches it locally. It stores model revisions and checksums with the index and can search offline once the graph and tokenizer are cached. The chunk command splits long documents into bounded overlapping passages and preserves source character offsets. The original FP32 graph is also supported. Corpus text and queries are processed on your device.

Use repeated --contains filters to require literal text such as an error code or name in each result, then rank matching passages by cosine score. Matching ignores case and combines with source restrictions, grouping and batch queries. Existing indexes need no rebuilding; see the example for details. The integration validation covers literal matching, Unicode, source restrictions, batch queries and both storage formats. New indexes also checksum their vector files; older float32 indexes without that checksum remain readable. Equal scores use corpus order as the tie breaker. The index reliability validation covers corruption checks, legacy compatibility and tied rankings. Use inspect to check an index's file layout and recorded checksums and report passage counts and storage size without loading model assets. It works offline with an empty model cache. Inspection validation. Inspection and search also reject unsupported model/tokenizer revisions, dimensions, token limits and storage declarations before reading vector files or initializing assets. Legacy float32 indexes without vector checksums remain supported; metadata checks do not authenticate an index. Compatibility validation. Use compact INPUT OUTPUT to convert an existing float32 index into the 132-byte passage-vector format without re-encoding text or loading model assets. It preserves passage bytes and model pins, validates unit vectors, and leaves the source index unchanged. Rounding can change rankings; it does not switch graph precision or restore truncated text. Usage and validation.

Add --reject-truncation to index, query, or batch-query to reject inputs longer than 256 tokens, including special tokens, before loading the graph. Batch queries check every query before producing results. The option adds a tokenizer pass and needs cached tokenizer assets in offline mode; query checks do not establish coverage of existing index passages. See the usage guide and validation.

Use the local evaluation tool with your own labeled queries to report per-query rankings and mean recall, MRR and graded nDCG at a chosen k. It uses the same CLI ranking for passages or source documents and checks judged IDs before encoding. Unlisted IDs count as irrelevant; incomplete labels affect the results. This is a domain assessment tool, not a new model benchmark. Evaluation validation. The guide also includes an offline comparison of two saved evaluation runs on the same corpus and labels, with paired query changes and percentile bootstrap intervals. It checks metric consistency and requires no model downloads or inference; intervals are limited by your query sample. Comparison validation.

python examples/pocket_search.py chunk documents.jsonl passages.jsonl
python examples/pocket_search.py index passages.jsonl my-index
python examples/pocket_search.py --offline query my-index "Where can I fix my bicycle?"

See the example README for dependency versions, JSONL input format, memory limits and 256-token truncation. Download the example files before running.

The example also supports --distinct-sources to return each document's best passage and batch-query to reuse one model load for multiple searches. Optional index ... --vector-dtype int8 stores each passage vector in 132 bytes, including its normalization value, versus 512 bytes in float32—a 74.2% smaller vector payload. Text, file headers and metadata are extra; float32 storage remains the default. Six-task storage studies retained 99.92% of mean task nDCG@10 with the FP32 encoder and 100.01% with the INT8 graph; each task exceeded 99% in both studies. These measured diagnostic results do not guarantee zero loss or quality on your corpus. See the storage audit for batch settings, paired intervals and reproduction. Stored vector compression changes neither model weights nor encoder work, and no query speed improvement is claimed.

Browser and direct ONNX inference

The browser export contains a float32 ONNX opset 17 graph of the entire pipeline, including pooling, projection, and normalization. It returns normalized 128-dimensional sentence embeddings. It is validated against PyTorch across variable batches, empty inputs and 256-token truncation (maximum observed absolute difference 3.2e-7). This export is for direct ONNX Runtime inference; do not load it as SentenceTransformer backend="onnx", which expects a raw transformer graph and applies the remaining layers separately. Input names and reproduction instructions are in browser/README.md.

The optional browser/model_int8.onnx graph uses dynamic QInt8 weights and returns float32 normalized 128-dimensional vectors. It is 23.1 MB, versus 90.6 MB for FP32, a 74.5% smaller graph download. It retains 99.52% of the FP32 projection mean nDCG@10 on the existing ArguAna and SciDocs tasks, or 95.45% of the base384 mean. These are deployment regression checks on the same tasks, not additional confirmation evidence. No encoding speed claim.

Task	FP32 Pocket 128 nDCG@10	INT8 Pocket 128 nDCG@10	INT8 / base384
ArguAna	0.4932	0.4911	97.44%
SciDocs	0.1979	0.1966	90.83%
SciFact	0.6104	0.5977	92.66%
NFCorpus	0.2938	0.2963	93.57%

INT8 passed the predefined regression gate: >=99.5% of FP32 confirmation mean,

=95% of base384 confirmation mean, >=90% of base384 on every measured task, and >=2% gain over raw base128 confirmation mean. It is close to the FP32 retention threshold; the table shows task-specific changes. See browser/quantized_decision.json and browser/quantized_results.json for exact values, recipe, size and checksums. The browser demo uses INT8; the Python SentenceTransformer example and retrieval table below use FP32. Use the same export for query and document encoding when building an index.

Retrieval measurements

nDCG@10 uses linear relevance gains, matching the trec_eval convention. Full official BEIR test queries and passage corpora, cosine search, title plus text for documents, max sequence length 256. No reranking.

Task	Role	Base 384	Base 128 truncation	Pocket 128	Pocket 64
arguana	confirmation	0.5041	0.4547	0.4932	0.4515
scidocs	confirmation	0.2164	0.1909	0.1979	0.1552
scifact	diagnostic	0.6451	0.6016	0.6104	0.4937
nfcorpus	diagnostic	0.3167	0.2741	0.2938	0.2408

Paired bootstrap 95% intervals for Pocket128 minus base128 nDCG@10:

• arguana: +0.0385, CI [+0.0275, +0.0503].
• scidocs: +0.0069, CI [-0.0002, +0.0142].
• scifact: +0.0088, CI [-0.0187, +0.0365].
• nfcorpus: +0.0196, CI [+0.0064, +0.0340].

The full metrics, 2,000-resample intervals, corpus/query counts and archive hashes are in results.json. release_decision.json records the thresholds fixed before this run and the observed outcome.

SciFact and NFCorpus are diagnostic tasks: their earlier results informed the decision to freeze the encoder. They are not fresh confirmation evidence.

How it was fitted

Base revision: 1110a243fdf4706b3f48f1d95db1a4f5529b4d41. SQuAD revision: 7b6d24c440a36b6815f21b70d25016731768db1f. Fit projection candidates on normalized original-model embeddings of unique SQuAD train questions and passages. Candidates: centered PCA, uncentered PCA, and uncentered-PCA initialized cosine-similarity distillation at 128 and 64 dimensions. The original transformer weights were frozen. Selected distill_500, using 512 deterministic SQuAD validation queries against the complete validation passage corpus. The candidate was selected and saved before evaluating confirmation labels. A standard Dense projection and Normalize module produce native 128-dimensional outputs. Model export and reload were checked numerically. No custom remote code.

Reproduce with the files in reproduce. In an environment with PyTorch and a GPU:

pip install -r reproduce/requirements.txt
python reproduce/project.py --config reproduce/config.json --output ./results --cache ./data

See selection.json, environment.txt and experiment_README.md for the protocol.

Efficiency and limits

Total parameters: 22,762,496. Native embedding dimension: 128. One-thread CPU encoding measured 24.4 passages/s versus 25.0 for the baseline, on 128 SQuAD validation passages, batch 32, median of three repetitions. Timing is specific to this cluster CPU; no encoding-speed improvement is claimed.

A separate fixed-model CPU deployment study compares MiniLM384 PyTorch and Pocket128 PyTorch/FP32 ONNX/INT8 ONNX on one Xeon core. For synthetic 128-token passages, batch 16, warm median throughput was 48.0 / 50.1 / 47.1 / 128.3 texts/s respectively. The report includes every workload, five raw timings, cached startup, peak process RSS, artifact hashes and reproduction code. INT8 speed depends on runtime, hardware and inputs; its retrieval-quality tradeoffs still apply. The smaller projection does not reduce encoder work, and these timings do not measure browser performance.

For the standalone ONNX CLI, a tested runtime recipe disables unused Transformers framework imports before startup. On one AMD CPU, the recorded OS RSS high-water counter dropped from about 1,150 MiB to 420–425 MiB and import time from 8.2 to 1.4 seconds, with identical vectors and passed CLI regressions. Warm throughput was nearly unchanged. The RSS counter can include pre-exec supervisor memory; it is not an isolated-worker or steady-state memory estimate. These settings are specific to tokenizer-only Transformers use and do not unload frameworks already imported.

An exact-index cost study separately measures cached file validation/loading and scanning of 10,000 to 1,000,000 synthetic 128-dimensional vectors. At one million rows, required files were about 565.8 MB with float32 storage and 185.8 MB with compact storage including short placeholder text. Speed ordering reversed between two runs; both runs and corrected memory snapshots are retained. No model encoding, retrieval-quality or general speed claim follows from this study.

English short-text retrieval only. No multilingual, instruction-following, long-document, adversarial robustness or safety-critical evaluation. Compression can change ranking and degrade quality on your domain. MiniLM base pretraining includes SQuAD and many other datasets; base-model benchmark overlap is unknown. Exact fine-tuning overlap checks do not rule out paraphrases or upstream pretraining overlap. This is a limited evaluation, not an MTEB leaderboard submission. Test on your own queries before migrating.

Licenses and attribution

Original MiniLM encoder weights remain Apache-2.0; see LICENSE_BASE and BASE_MODEL_CARD.md. Our fitted projection is released under CC BY-SA 4.0, the license used in this repository’s metadata. Research code is Apache-2.0. See NOTICE for attribution.

Projection fitting data: SQuAD by Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang (CC BY-SA 4.0). Adaptation: dimensionality reduction fitted to encoder representations of question/context text; no raw dataset text is included in this release.

Evaluation: BEIR, ArguAna (Wachsmuth et al.), SciDocs (Cohan et al.), SciFact (Wadden et al.), NFCorpus (Boteva et al.). The BEIR dataset cards list CC BY-SA 4.0. They were used for evaluation only.

Built by Pocket Mind. Report reproducible retrieval failures in this model’s Community tab; include task, dimensions, library version and an example you have permission to share.