uthayamurthy/origin-task3-indictrans2-lora

Origin SciHigh 2026 Task 3 IndicTrans2 LoRA

This repository contains a LoRA adapter for English-to-Bengali scientific-paper title translation. It is the Run 2 system submitted by team Origin to SciHigh 2026 Task 3.

The adapter must be loaded on top of ai4bharat/indictrans2-en-indic-1B at commit 10e65a9951a1e922cd109a95e8aba9357b62144b; it is not a standalone full model.

Model details

Field	Value
Base model	ai4bharat/indictrans2-en-indic-1B
Base revision	10e65a9951a1e922cd109a95e8aba9357b62144b
Method	LoRA for SEQ_2_SEQ_LM
Source / target	English (eng_Latn) → Bengali (ben_Beng)
LoRA modules	q_proj, k_proj
LoRA rank / alpha / dropout	16 / 32 / 0.1
Trainable parameters	3,538,944 of 1,119,082,496 (0.3162%)
Developed by	Origin

Training data and procedure

The adapter was trained only on the 60 official training triplets from the SciHigh 2026 Task 3 SpringerSSAT-Tiny-Multilingual split. Each input was the English title and each target was its expert Bengali translation. No synthetic examples or test labels were used.

The 20 official validation examples were held out from gradient optimization and used once per epoch for checkpoint selection.

Hyperparameter	Value
Epochs run / selected epoch	10 / 10
Batch size	8
Learning rate	2e-4
Weight decay	0.01
Warmup ratio	0.1
Maximum source / target length	128 / 128
Generation	5 beams, maximum 96 new tokens
Seed	42
Hardware	NVIDIA RTX PRO 6000 Blackwell, 96 GB
Software	Python 3.12.3, PyTorch 2.13.0+cu130, Transformers 4.53.2, PEFT 0.17.1

Validation results

Scores below were calculated on the 20 held-out validation titles. ROUGE-L uses whitespace-delimited Bengali tokens

Metric	LoRA adapter	Zero-shot IndicTrans2 1B	Delta
ROUGE-L F1	0.455671	0.429462	+0.026209
chrF++	51.1995	51.0039	+0.1956
SacreBLEU (tokenize=none)	17.1911	15.0960	+2.0951
Exact match	0.00%	0.00%	0.00 pp

Usage

pip install "transformers==4.53.2" "peft>=0.16,<0.18" \
  "protobuf>=5,<7" "sentencepiece>=0.2,<0.3" torch
pip install "IndicTransToolkit @ git+https://github.com/VarunGumma/IndicTransToolkit.git@3efb8418d0721b4ce267c2b3586899d313191357"

import torch
from IndicTransToolkit import IndicProcessor
from peft import PeftModel
from transformers import AutoModelForSeq2SeqLM, AutoTokenizer

adapter_id = "uthayamurthy/origin-task3-indictrans2-lora"
base_id = "ai4bharat/indictrans2-en-indic-1B"
base_revision = "10e65a9951a1e922cd109a95e8aba9357b62144b"
device = "cuda" if torch.cuda.is_available() else "cpu"

tokenizer = AutoTokenizer.from_pretrained(
    base_id, revision=base_revision, trust_remote_code=True
)
base_model = AutoModelForSeq2SeqLM.from_pretrained(
    base_id,
    revision=base_revision,
    trust_remote_code=True,
    torch_dtype=torch.bfloat16 if device == "cuda" else torch.float32,
)
model = PeftModel.from_pretrained(base_model, adapter_id).to(device).eval()
processor = IndicProcessor(inference=True)

title = "A framework to measure microaggressions in the mathematics classroom"
processed = processor.preprocess_batch(
    [title], src_lang="eng_Latn", tgt_lang="ben_Beng"
)
inputs = tokenizer(processed, return_tensors="pt").to(device)
with torch.inference_mode():
    tokens = model.generate(**inputs, num_beams=5, max_new_tokens=96)
decoded = tokenizer.batch_decode(tokens, skip_special_tokens=True)
print(processor.postprocess_batch(decoded, lang="ben_Beng")[0])