Llama-Karmx-Indic-Embedding-bge-m3

An Indic embedding model for search and RAG in Hindi, Bengali, Tamil, Telugu, Marathi, Gujarati, Kannada, Malayalam, Punjabi and Odia. It is BAAI/bge-m3 fine-tuned for Indian languages. It is a drop-in replacement with the same size, speed, 1,024-dimensional vectors and Sentence Transformers API, and it retrieves better in every one of the ten languages.

• +2.05 nDCG@10 over bge-m3 on IndicQARetrieval (73.77 vs 71.72), with gains in all 10 languages
• +0.85 nDCG@10 over bge-m3 on Belebele retrieval (89.85 vs 88.99), with no language worse
• No query prefix or instruction needed; normalized dense vectors for cosine / dot-product search
• ONNX versions for ONNX Runtime and transformers.js, including a 570 MB int8 file that still beats bge-m3

The quickstart runs semantic search over real passages in all ten languages, then retrieve-and-rerank.

Quick start

Works with Sentence Transformers 3.0 or later (verified identical embeddings on 3.0.1, 5.1.0 and 6.1.0).

from sentence_transformers import SentenceTransformer

model = SentenceTransformer("karmx/Llama-Karmx-Indic-Embedding-bge-m3")
model.max_seq_length = 512

queries = ["भारत की राजधानी क्या है?"]                      # Hindi query
passages = ["नई दिल्ली भारत की राजधानी है।",               # relevant
            "मुंबई महाराष्ट्र की राजधानी है।"]              # not relevant
q = model.encode(queries, normalize_embeddings=True)
p = model.encode(passages, normalize_embeddings=True)
print(q @ p.T)  # higher = more relevant

LangChain:

from langchain_huggingface import HuggingFaceEmbeddings

emb = HuggingFaceEmbeddings(model_name="karmx/Llama-Karmx-Indic-Embedding-bge-m3",
                            encode_kwargs={"normalize_embeddings": True})

ONNX: ONNX Runtime and transformers.js

The onnx/ folder has two exports with inputs input_ids and attention_mask and output last_hidden_state. Take the CLS token (position 0) and L2-normalize it:

File	Size	Belebele nDCG@10	Best for
onnx/model_quantized.onnx	570 MB	0.8981 (PyTorch 0.8985)	CPU servers, browsers, Node (transformers.js dtype: "q8")
onnx/model_fp16.onnx	1.13 GB	same as PyTorch (cosine 0.99999)	GPUs and WebGPU (transformers.js dtype: "fp16")

The int8 file uses per-channel dynamic quantization. It is 4× smaller than the PyTorch weights and still beats bge-m3 (0.8899) on Belebele.

transformers.js (browser or Node):

import { pipeline } from "@huggingface/transformers";

const extractor = await pipeline("feature-extraction", "karmx/Llama-Karmx-Indic-Embedding-bge-m3", { dtype: "q8" });
const embeddings = await extractor(["भारत की राजधानी क्या है?", "नई दिल्ली भारत की राजधानी है।"],
                                   { pooling: "cls", normalize: true });

ONNX Runtime (Python):

import numpy as np, onnxruntime as ort
from huggingface_hub import hf_hub_download
from transformers import AutoTokenizer

repo = "karmx/Llama-Karmx-Indic-Embedding-bge-m3"
tokenizer = AutoTokenizer.from_pretrained(repo)
session = ort.InferenceSession(hf_hub_download(repo, "onnx/model_quantized.onnx"))
enc = tokenizer(["भारत की राजधानी क्या है?"], padding=True, truncation=True, max_length=512, return_tensors="np")
cls = session.run(["last_hidden_state"], {"input_ids": enc["input_ids"], "attention_mask": enc["attention_mask"]})[0][:, 0]
embeddings = cls / np.linalg.norm(cls, axis=1, keepdims=True)

For best end-to-end quality, rerank the top 50 results with BAAI/bge-reranker-v2-m3. This embedder raises that two-stage pipeline from 80.41 to 80.76 nDCG@10 on IndicQARetrieval.

Results

Both benchmarks were held out from training and model selection. Scores are nDCG@10 (pytrec_eval) on the complete official corpora. The baseline is bge-m3 dense under the identical protocol (512 tokens, exact top-100 search, no prompts). Intervals come from a paired bootstrap over queries within each language (10,000 samples).

IndicQARetrieval (MTEB, 16,775 queries):

Language	bge-m3	this model	Δ
Hindi	0.6688	0.6854	+0.0165
Bengali	0.7374	0.7560	+0.0186
Tamil	0.6407	0.6616	+0.0210
Telugu	0.6896	0.7285	+0.0389
Marathi	0.6971	0.7124	+0.0154
Gujarati	0.7554	0.7710	+0.0156
Kannada	0.7507	0.7658	+0.0152
Malayalam	0.7247	0.7431	+0.0185
Punjabi	0.7181	0.7418	+0.0238
Odia	0.7896	0.8115	+0.0219
Macro	0.7172	0.7377	+0.0205 (95% CI +0.0178 to +0.0233)

MRR rises from 0.6768 to 0.6992 and recall@100 from 0.9802 to 0.9828.

Belebele retrieval (facebook/belebele, question → passage, 9,000 human-translated questions):

Language	bge-m3	this model	Δ
Hindi	0.8927	0.8932	+0.0005
Bengali	0.9148	0.9204	+0.0056
Tamil	0.8997	0.9091	+0.0094
Telugu	0.9020	0.9151	+0.0131
Marathi	0.9159	0.9273	+0.0115
Gujarati	0.8508	0.8594	+0.0086
Kannada	0.8768	0.8919	+0.0150
Malayalam	0.9065	0.9186	+0.0122
Punjabi	0.8499	0.8572	+0.0073
Odia	0.8902	0.8921	+0.0019
Macro	0.8899	0.8985	+0.0085 (95% CI +0.0054 to +0.0116)

How it was trained

The stronger cross-encoder BAAI/bge-reranker-v2-m3 teaches bge-m3 how to rank its own top candidates (listwise knowledge distillation):

• Loss: KL divergence between the reranker's softmax over each query's candidates and the embedder's softmax over cosine similarities / 0.02. The candidates are the labeled passage plus bge-m3's 3 highest-ranked passages. An in-batch InfoNCE term is added, with same-article passages masked.
• Data: 28,502 questions over Wikipedia paragraphs in the ten languages (ai4bharat/Indic-Rag-Suite, article-level train split) plus MIRACL training judgments for Hindi, Bengali and Telugu. Benchmark questions, passages and source articles were removed first.
• Setup: full fine-tuning with frozen word embeddings, lr 1e-5 (5% warmup, linear decay), 32 queries per batch, BF16, one pass (885 steps, about 21 minutes on one RTX 5090). The checkpoint was chosen on a 5,000-query article-held-out dev split, where nDCG@10 rose from 0.8154 to 0.8454.

Limitations

• Dense retrieval only. bge-m3's sparse and multi-vector (ColBERT) heads are not included.
• Tuned for the ten listed languages; English and other languages were not evaluated.
• Default max_seq_length is 512 tokens, the evaluated setting. The architecture accepts up to 8,192, but longer inputs were not evaluated.
• Single training run (one seed).
• No claim for romanized Hindi or Hindi–English code-mixed text.

License and attribution

MIT, like the base model. Built with Llama: the training questions in Indic-Rag-Suite were generated by Meta-Llama-3.3-70B-Instruct. The Llama 3.3 Community License requires models trained on such outputs to carry a name beginning with "Llama". Passages are Wikipedia text (CC BY-SA 4.0). MIRACL is Apache-2.0. The base model bge-m3 is MIT, and the teacher bge-reranker-v2-m3 is Apache-2.0, both by BAAI.

Citation

@misc{karmx2026indicembedding,
  title  = {Llama-Karmx-Indic-Embedding-bge-m3: Reranker-Distilled BGE-M3 for Ten Indic Languages},
  author = {Karmx},
  year   = {2026},
  url    = {https://huggingface.co/karmx/Llama-Karmx-Indic-Embedding-bge-m3}
}