mitra-qwen35-2b-embedder

The compact (2B) retrieval embedder of the Dharmamitra Qwen3.5 family — a fast alternative to mitra-qwen35-embedder (9B) for classical Buddhist cross-lingual retrieval. It outperforms general-purpose embedders on this domain by an order of magnitude and even exceeds the 9B-class gemma-2-mitra-e on verse→commentary retrieval.

Prompting template

Identical to the 9B embedder — queries wrapped with the special tokens:

<instruct>{instruction}\n<query>{query text}

with the training instruction Please find the semantically most similar text. (optionally ... in Tibetan. etc.). Passages are raw text. EOS is appended automatically by the bundled tokenizer; pooling is last-token with left padding, L2-normalized. Embedding dimension 2048.

Script conventions — note the difference from the 9B embedder: this model is script-native. Tibetan should be given in native Tibetan script (not Wylie); Sanskrit and Pāli in IAST; Chinese in Chinese script.

Usage

import torch
from transformers import AutoModel, AutoTokenizer

tok = AutoTokenizer.from_pretrained("buddhist-nlp/mitra-qwen35-2b-embedder")
tok.padding_side = "left"
model = AutoModel.from_pretrained(
    "buddhist-nlp/mitra-qwen35-2b-embedder", dtype=torch.bfloat16, device_map="cuda"
).eval()

INSTR = "Please find the semantically most similar text."

def embed(texts, is_query=False, batch_size=32):
    if is_query:
        texts = [f"<instruct>{INSTR}\n<query>{t}" for t in texts]
    out = []
    with torch.no_grad():
        for i in range(0, len(texts), batch_size):
            enc = tok(texts[i:i+batch_size], padding=True, truncation=True,
                      max_length=512, return_tensors="pt").to(model.device)
            h = model(**enc).last_hidden_state[:, -1]
            out.append(torch.nn.functional.normalize(h.float(), dim=-1).cpu())
    return torch.cat(out)

Evaluation

R@1 on the mitra retrieval benchmark (gold pools | +25k in-language distractors per pool):

task	this model	gemma-2-mitra-e (9B-class)	BGE-M3
Multilingual parallels (sa/bo/zh)	0.770 | 0.744	0.850 | 0.841	0.007 | 0.001
Multilingual asymmetric (sentence→window)	0.553 | 0.464	0.654 | 0.581	0.002 | 0.000
English → classical	0.796 | 0.792	0.886 | 0.879	0.204 | 0.068
Verse → commentary	0.428 | 0.390	0.413 | 0.384	0.298 | 0.138
English question → passage	0.346 | 0.328	0.534 | 0.473	0.077 | 0.020

Model details

• Base: Qwen3.5-2B with Buddhist-domain continued pretraining and stage-2 SFT
• Contrastive finetuning: LoRA (r=32) on script-native data, InfoNCE with cross-device negatives, temperature 0.02, fully-annealed 500-step schedule
• Embedding dimension: 2048

Citation

If you use this model, please cite the Dharmamitra project (https://dharmamitra.org). A technical report is in preparation.