KinyaEmbed model card content (click to expand)

KinyaEmbed: Contrastive Sentence Embeddings for Kinyarwanda

Ireddi Rakshitha (Software Engineer, Barclays) · Devavarapu Yashwanth (Software Engineer, Barclays) · Pierre Ntakirutimana (Research Associate, Carnegie Mellon University)

arXiv preprint · August 2026 · Paper · Code

---

Model Description

KinyaEmbed is the first dedicated sentence embedding model for Kinyarwanda (ISO 639-1: rw), a Bantu language spoken by 12 million people in Rwanda. Built on KinyaBERT-large and trained via a four-stage curriculum with MultipleNegativesRankingLoss (MNRL).

Training Stages

Stage	Data	Scale (1/τ)	Checkpoints
1 — Gazette Paraphrases	~18,000 Official Gazette pairs	30 / 35 / 40	sc30, sc35, sc40
2 — MNLI Triplets	715 translated NLI triplets	35	v12
3 — OPUS-100 Alignment	English–Kinyarwanda pairs	20	step22A
4 — KinyaCOMET	2,936 human-quality pairs (≥0.8)	20	step23A

Ensemble: all5+23A×2 — 7 checkpoints averaged, step23A double-weighted.

---

Performance

Model	SemRel2024-rw ρ	Wiki-RW-STS ρ	Clustering Silhouette
LaBSE	0.4535	0.2197	0.1882
mE5-large	0.6039	0.5337	0.0794
AfriE5-instruct	0.6037	0.5391	0.1104
OpenAI text-emb-3-large	0.5175	0.5319	0.0846
KinyaEmbed (ours)	0.7298	0.6005	0.2146

+20.9% over mE5-large on SemRel2024-rw · +8.6% on Wiki-RW-STS · Best document clustering across all 7 models.

---

Usage

from sentence_transformers import SentenceTransformer
import numpy as np

# Ensemble (recommended)
checkpoints = [
    "TabuLM-Research/KinyaEmbed/sc30",
    "TabuLM-Research/KinyaEmbed/sc35",
    "TabuLM-Research/KinyaEmbed/sc40",
    "TabuLM-Research/KinyaEmbed/v12",
    "TabuLM-Research/KinyaEmbed/step22A",
    "TabuLM-Research/KinyaEmbed/step23A",
    "TabuLM-Research/KinyaEmbed/step23A",  # double-weighted
]
models = [SentenceTransformer(c) for c in checkpoints]

def ensemble_encode(sentences):
    embs = [m.encode(sentences, normalize_embeddings=True) for m in models]
    avg = np.mean(embs, axis=0)
    return avg / np.linalg.norm(avg, axis=1, keepdims=True)

# Single checkpoint (lower latency)
model = SentenceTransformer("TabuLM-Research/KinyaEmbed/sc35")
embeddings = model.encode(["Kinyarwanda text here"], normalize_embeddings=True)

---

Included Files

┌───────────────────────────┬─────────────────────────────────────────────────────┐
│           File            │                     Description                     │
├───────────────────────────┼─────────────────────────────────────────────────────┤
│ sc30/, sc35/, sc40/       │ Stage 1 checkpoints (gazette, 3 temperature scales) │
├───────────────────────────┼─────────────────────────────────────────────────────┤
│ v12/                      │ Stage 2 checkpoint (MNLI triplets)                  │
├───────────────────────────┼─────────────────────────────────────────────────────┤
│ step22A/                  │ Stage 3 checkpoint (OPUS-100)                       │
├───────────────────────────┼─────────────────────────────────────────────────────┤
│ step23A/                  │ Stage 4 checkpoint (KinyaCOMET)                     │
├───────────────────────────┼─────────────────────────────────────────────────────┤
│ kinyacomet_filtered.jsonl │ 2,936 high-quality Kinyarwanda–English pairs        │
├───────────────────────────┼─────────────────────────────────────────────────────┤
│ wiki_rw_sts.jsonl         │ Wiki-RW-STS benchmark (300 pairs, CC-BY-SA 4.0)     │
└───────────────────────────┴─────────────────────────────────────────────────────┘

---

Citation

bibtex
@article{ireddi2026kinyaembed,
  title   = {{KinyaEmbed}: Contrastive Sentence Embeddings for {Kinyarwanda}
             via Multi-Stage Curriculum Training},
  author  = {Ireddi, Rakshitha and Devavarapu, Yashwanth and Ntakirutimana, Pierre},
  journal = {arXiv preprint arXiv:XXXX.XXXXX},
  year    = {2026}
}

License

Code: MIT · KinyaCOMET filtered pairs: CC-BY 4.0 · Wiki-RW-STS: CC-BY-SA 4.0

</details>