Bheri/ithasa-labse-en-sa

English–Sanskrit LaBSE with Mitrasamgraha

SentenceTransformer fine-tuned from sentence-transformers/LaBSE on cleaned English–Sanskrit parallel text, including all retained Mitrasamgraha training pairs. This run starts from the listed pretrained base encoder.

• Embedding dimension: 768
• Maximum input length: 512 tokens
• Pooling: CLS pooling
• Similarity: cosine
• Training objective: CachedMultipleNegativesRankingLoss with duplicate-free batches

Usage

from sentence_transformers import SentenceTransformer

model = SentenceTransformer("Bheri/ithasa-labse-en-sa")
sentences = ["The teacher speaks.", "गुरुः वदति।"]
embeddings = model.encode(sentences, normalize_embeddings=True)
print(embeddings @ embeddings.T)

Training data

609,187 training pairs. Source files were normalized, checked for structural and script problems, deduplicated, and separated from validation and test by normalized sentence identity. Mitrasamgraha uses the original source split assignments. Unfiltered NLLB was excluded.

Source	Training pairs
Mitrasamgraha	376,925
Translation data	120,557
BPCC	94,441
Sanskrit Llama	17,264

Script and length filtering do not certify the semantic correctness of every translation. One representative per sentence side was retained for contrastive training. Raw source datasets were preserved locally.

Validation results

Validation contains 2,000 pairs: 500 per source. Each source is evaluated separately with a 500-candidate translation pool, in both English-to-Sanskrit and Sanskrit-to-English directions. Values below average the two directions.

Source	Mean bidirectional accuracy
BPCC	1.0000
Sanskrit Llama	0.3810
Translation data	0.9600
Mitrasamgraha	0.9340
Macro average	0.81875

These are validation results used for checkpoint selection, not held-out test results or general semantic-search benchmark scores.

Held-out test results

Evaluated on 2,000 separate test pairs, 500 per source, using a separate 500-candidate pool per source and both translation-retrieval directions. These records were excluded from training and checkpoint selection.

Source	Mean bidirectional accuracy
BPCC	0.9980
Sanskrit Llama	0.3990
Translation data	0.9620
Mitrasamgraha	0.9220
Macro average	0.82025

Evaluation identifier: 20261025T114733918539Z. The macro test accuracy is 82.025%. These are translation-retrieval results, not a general semantic-search or NER benchmark.

Training configuration

• Learning rate: 2e-05
• Maximum epoch budget: 3; early stopping enabled
• Contrastive batch size: 128
• Cached forward mini-batch size: 16
• Seed: 42
• Selected checkpoint: step 6,500; run stopped at step 7,250 (epoch 1.523).
• Run identifier: 20261025T084226097073Z

The validation gate compares against the starting base model. Passing it does not establish superiority over the trained control. Assess the per-source results and your actual retrieval queries when selecting a deployment model.