neural-sparse-encoding-splare-multilingual-v1

A multilingual learned sparse retrieval model for OpenSearch, trained with SPLARE. A LoRA adapter on Qwen/Qwen3-1.7B-Base feeds the layer-10 residual stream into the frozen Qwen-Scope sparse autoencoder Qwen/SAE-Res-Qwen3-1.7B-Base-W32K-L0_100, and the 32,768 SAE features are pooled into one sparse vector per text. The features are shared across languages, so a query in one language finds documents in another.

Results

nDCG@10 through OpenSearch, on the evaluation sets described in RESULTS.md, which also has the scores per language pair. XPQA and MLQA are out of domain, MIRACL is in domain. Same-language pairs have the query and the passages in one language, cross-lingual pairs in two: XPQA has 3 of each, MLQA 7 and 41; MIRACL covers fr, ja and de.

Model	XPQA same-language	XPQA cross-lingual	MIRACL	MLQA same-language	MLQA cross-lingual
this model, GPU (bf16)	0.648	0.639	0.724	0.590	0.561
this model, CPU (int8 ONNX)	0.643	0.635	0.723	0.574	0.543
opensearch-neural-sparse-encoding-multilingual-v1	0.614	0.300	0.734	0.528	0.188

Speed, encoding only, one instance (RESULTS.md has the method and more batch sizes):

	GPU, bf16 (SageMaker ml.g5.xlarge)	CPU, int8 ONNX (Docker, 8 cores of an EC2 m7i.4xlarge)
Query, median	32 ms	19 ms
Queries per second	36	54
MLQA passages per second, one per request	32	12
MLQA passages per second, batches of 32	21	12
XPQA answers per second, batches of 32	246	45

Size, against opensearch-neural-sparse-encoding-multilingual-v1 (mlv1) in its doc-only mode. This model uses only 12 of the backbone's 28 layers and the encoder half of the SAE:

	this model, GPU (bf16)	this model, CPU (int8 ONNX)	mlv1
Parameters used	0.98B	0.93B	168M
Download	4.0 GB (backbone 3.4 GB, SAE 537 MB, adapter 51 MB)	935 MB	670 MB
Weights in memory	about 2.0 GB	about 0.9 GB	about 0.7 GB
Model at query time	yes	yes	no (mbert-uncased analyzer)

Usage

With PyTorch (the adapter):

# pip install torch transformers==5.5.4 peft==0.19.1        (tested with torch 2.6)
import sys
from huggingface_hub import snapshot_download
path = snapshot_download("aswath86/neural-sparse-encoding-splare-multilingual-v1", ignore_patterns=["onnx/*"])   # this repository, without the ONNX file
sys.path.insert(0, path)
from inference import SparseEncoder
enc = SparseEncoder(path)                        # downloads the backbone and the SAE on first use; bf16 on CUDA, fp32 on CPU
q = enc.encode("où acheter un casque sans fil ?")[0]                                    # queries: the full vector
d = enc.encode("Wireless headphones with active noise cancelling.", top_k=400)[0]       # documents: top 400 features
score = sum(w * d.get(f, 0.0) for f, w in q.items())     # the dot product a rank_features index computes

On CPU (the int8 ONNX file). Encode one text per call: the int8 model quantizes activations per call, so batching changes the vectors.

# pip install onnxruntime tokenizers huggingface_hub numpy
import os, numpy as np, onnxruntime as ort
from huggingface_hub import snapshot_download
from tokenizers import Tokenizer
path = snapshot_download("aswath86/neural-sparse-encoding-splare-multilingual-v1", allow_patterns=["tokenizer.json", "onnx/*"])
tok = Tokenizer.from_file(os.path.join(path, "tokenizer.json")); tok.enable_truncation(max_length=512); tok.no_padding()   # the CPU limit
sess = ort.InferenceSession(os.path.join(path, "onnx", "model.int8.onnx"), providers=["CPUExecutionProvider"])
text = "où acheter un casque sans fil ?"
vec = {}                                                 # an empty text gives an empty vector
if text.strip():
    n = lambda t: len(tok.encode(t).ids)
    if n(text) <= 3:                                     # very short inputs: see "Short inputs"
        text += "." if n(text + ".") > 1 else " ."
    e = tok.encode(text)
    w = sess.run(None, {"input_ids": np.asarray([e.ids], np.int64), "attention_mask": np.asarray([e.attention_mask], np.int64)})[0][0]
    vec = {f"f_{i:05d}": round(float(w[i]), 4) for i in np.nonzero(w)[0] if round(float(w[i]), 4) > 0}

Short inputs

A one-token text such as coffee gives a degenerate vector of about 21,000 features. Both examples therefore append "." to texts of three tokens or fewer. SparseEncoder(pad_short=False) turns this off; RESULTS.md was measured without it.

OpenSearch

Store each document's top 400 features in a rank_features field and search with the full query vector. A query vector can have more features than the default clause limit of 1,024, so raise it first. There are two ways to get the vectors in: (1) encode the texts yourself with inference.py or the ONNX file and send the vectors, or (2) register the model in ml-commons, so that an ingest pipeline encodes the documents and neural_sparse encodes the query. The results were measured the second way; it needs the model served behind an ml-commons connector, and the connector and serving code come with the code release. The feature ids and weights below are placeholders.

PUT _cluster/settings
{ "persistent": { "indices.query.bool.max_clause_count": 4096 } }

PUT my-index
{ "mappings": { "properties": {
  "text": { "type": "text" }, "text_sparse": { "type": "rank_features" } } } }

# (1) Vectors from inference.py or the ONNX file
PUT my-index/_doc/1
{ "text": "Wireless headphones with active noise cancelling.",
  "text_sparse": { "f_00412": 1.8125, "f_20733": 0.9531 } }

GET my-index/_search
{ "query": { "neural_sparse": { "text_sparse": {
  "query_tokens": { "f_00412": 1.4219, "f_31002": 0.7344 } } } } }

# (2) The model, registered in ml-commons, encodes documents and queries
PUT _ingest/pipeline/splare-ingest
{ "processors": [{ "sparse_encoding": {
  "model_id": "<model_id>", "field_map": { "text": "text_sparse" },
  "prune_type": "top_k", "prune_ratio": 400 } }] }

PUT my-index/_doc/2?pipeline=splare-ingest
{ "text": "Wireless headphones with active noise cancelling." }

GET my-index/_search
{ "query": { "neural_sparse": { "text_sparse": {
  "query_text": "wireless headphones", "model_id": "<model_id>" } } } }

Training

Setting	Value
Data	neural-sparse-encoding-splare-multilingual-v1-training-data, rounds 1 and 2: 88,165 MIRACL and Mr.TyDi training queries in 16 languages, each appearing twice with different hard negatives (176,330 rows)
Each row	a query, its positive passage and 8 hard negatives, every passage scored by Qwen/Qwen3-Reranker-8B (the teacher)
Objective	KL distillation from the teacher scores (temperature τ = 80), plus FLOPS regularization (λ = 1e-4 for queries and documents)
Fine-tuning	LoRA r = 32, α = 64, dropout 0, on q/k/v/o; backbone (bf16) and SAE frozen; bidirectional attention
Epochs and steps	1 epoch over the 176,330 rows, 11,020 steps
Batch size	16 (4 GPUs × 4)
Optimizer	AdamW (PyTorch defaults), learning rate 5e-5, constant, no warmup
Max length	192 tokens for queries and passages
Hardware	4× NVIDIA L40S (SageMaker ml.g6e.12xlarge), 2 h 14 min

License

Apache-2.0, as are Qwen3-1.7B-Base and the Qwen-Scope SAE it builds on. NOTICE has the attributions.

Citation

Formal, Louis, Déjean and Clinchant. Learning Retrieval Models with Sparse Autoencoders. ICLR 2026. arXiv:2603.13277