Baakaa/sincerox-kashmiri

Sincerox — English → Kashmiri Translation

QLoRA adapter for sarvamai/sarvam-translate (Gemma 3 4B IT), fine-tuned for English → Kashmiri (kas_Arab, Perso-Arabic script).

Built for the KATHE 2026 shared task on English–Kashmiri machine translation.

• Author: Muhammad Ammaar , HF USER: Baakaa
• Code & full methodology: github.com/amaarx/kathe-2026-english-kashmiri
• Adapter size: 131 MB (base model is loaded separately)

---

Quick start

from transformers import AutoTokenizer, AutoModelForCausalLM, BitsAndBytesConfig
from peft import PeftModel
import torch

bnb = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_compute_dtype=torch.float32,   # see "Hardware notes" below
    bnb_4bit_quant_type="nf4",
    bnb_4bit_use_double_quant=True,
)

tok = AutoTokenizer.from_pretrained("sarvamai/sarvam-translate")
base = AutoModelForCausalLM.from_pretrained(
    "sarvamai/sarvam-translate", quantization_config=bnb, device_map="auto")
model = PeftModel.from_pretrained(base, "Baakaa/sincerox-kashmiri")
model.eval()

prompt = tok.apply_chat_template(
    [{"role": "user",
      "content": "Translate the text below to Kashmiri.\n\nI go to my school daily."}],
    tokenize=False, add_generation_prompt=True)

x = tok(prompt, return_tensors="pt").to(model.device)
out = model.generate(
    **x,
    max_new_tokens=128,
    num_beams=6,
    repetition_penalty=1.15,      # required — see "Generation settings"
    no_repeat_ngram_size=3,
    do_sample=False,
    pad_token_id=tok.eos_token_id,
)
print(tok.decode(out[0][x.input_ids.shape[1]:], skip_special_tokens=True))

Sample output

English	Kashmiri
I go to my school daily.	بہٕ چُھس پرٛٮ۪تھ دۄہ سکوٗل گژھان۔
The tailor promised to finish the coat by Thursday.	سٕژن دِژ ووٚعدٕ زِ سُہ مۄکلاوِ بٔدرِوار تانۍ کوٹھ۔
My grandfather refused to sell the walnut tree.	میٲنۍ بٕڈۍببَن کوٚر ڈونٛگٕلُک کُل کٕنٛنہٕ نِش اِنٛکار۔
She counted the money twice before leaving.	سۄ گٔژھنہٕ برٛونٛہہ گۄنٛجۍ پونٛسہٕ دۄیہِ لٹہِ۔

---

Results

KATHE 2026 scores the geometric mean of BLEU and chrF++, with both hypothesis and reference normalised by KashmiriNormalizer:

score = sqrt(BLEU * chrF++)

Progression across the project (public leaderboard):

Approach	Score
IndicTrans2-1B + LoRA on BPCC	9.00
— with two data-pipeline bugs fixed	9.07
— with conversational data added (upsampled 4×)	11.26
Sarvam-Translate, zero-shot, no fine-tuning	14.50
Sarvam-Translate + this adapter (step 2250)	15.0+

On the final number. should be read with care. The conversational training corpus and the public test set are both derived from the same publicly available GAASH Lab collection, and spot checks found test sentences appearing verbatim in the training data. Much of that score therefore reflects recall of seen pairs rather than translation of unseen input. The 14.50 . figure is one of the first measure of the base model's ability here, and the 11.26 is the cleanest measure of a fine-tune on genuinely disjoint data. Anyone evaluating this adapter on a different corpus should expect results far closer to the mid-teens .

The single most informative result in the project is that switching base models — IndicTrans2 to Sarvam-Translate, with no training at all — outperformed two days of fine-tuning work on the original. Base model choice dominated every other decision.

---

Training

Base model	sarvamai/sarvam-translate (Gemma 3 4B IT)
Method	QLoRA — 4-bit NF4, double quantization
LoRA rank / alpha / dropout	16 / 32 / 0.05
Target modules	q_proj k_proj v_proj o_proj gate_proj up_proj down_proj
Excluded	vision_tower (Gemma 3 is multimodal; training is text-only)
Trainable parameters	~0.5% of total
Data	8,730 conversational English–Kashmiri pairs
Max sequence length	192 tokens
Effective batch size	16 (batch 4 × grad accumulation 4)
Learning rate	1e-4, linear decay, 6% warmup
Steps	2,250 (~4 epochs)
Loss	1.16 → 0.27
Hardware	Kaggle P100 (16 GB), ~6.5 hours

Only the Kashmiri completion contributes to the loss; prompt tokens are masked with -100.

Data

Conversational pairs extracted from the GAASH Lab Kashmiri corpus — everyday sentences averaging 7.3 English words, at 73.3 diacritics per 1,000 characters.

This register matters. The task's test set averages ~6 words per sentence, while AI4Bharat's BPCC — the largest available Kashmiri parallel corpus — is encyclopedic Wikipedia prose averaging ~25 words. A model trained only on BPCC produces fluent but formal Kashmiri that scores poorly on everyday speech: character overlap stays reasonable (chrF++ ≈ 34) while exact word matches collapse (BLEU ≈ 2). Adding conversational data was the change that moved the score.

---

Generation settings

num_beams=6, repetition_penalty=1.15, no_repeat_ngram_size=3,
max_new_tokens=128, do_sample=False

Repetition control is not optional. Without repetition_penalty and no_repeat_ngram_size, the model falls into a token loop on short inputs (... کان کان کان کان).

Hardware notes

Use bnb_4bit_compute_dtype=torch.float32 on pre-Ampere GPUs (Kaggle T4 and P100, and anything older than RTX 30xx / A100). These have no bfloat16 support, and with float16 compute Gemma 3 produces NaN logits and generate() returns an empty string — a failure that looks like a prompting bug and is not one. float32 compute is slower but numerically stable.

On Ampere or newer, torch.bfloat16 is both faster and stable.

Loading the base model in 4-bit requires bitsandbytes. Without it the model loads in full precision and needs roughly 16 GB of VRAM instead of about 5 GB.

---

Limitations

• Domain. Trained on a small conversational corpus. Formal, technical, literary and long-form text is out of distribution.
• Diacritics. Output is largely undiacritized, matching the convention of the available Kashmiri corpora. Where references carry harakat, BLEU is penalised.
• Dialect. Regional coverage of the training data is not characterised.
• Evaluation. See the note above on training/test overlap. Scores on this task's public leaderboard are not a reliable estimate of general translation quality.
• Not suitable for high-stakes translation without review by a native speaker.

Acknowledgements

• Sarvam AI — sarvam-translate, the base model.
• Google DeepMind — Gemma 3.
• AI4Bharat — IndicTrans2 and the BPCC corpus.
• GAASH Lab / Tajamul21 — the conversational Kashmiri corpus.
• Abdul Muizz — organising KATHE 2026, authoring KashmiriNormalizer, and correctly diagnosing a train/validation contamination issue in my earlier pipeline from a single score.

Citation

@misc{sincerox2026,
  title  = {Sincerox: English-Kashmiri Translation Adapter for Sarvam-Translate},
  author = {Muhammad Ammaar},
  year   = {2026},
  note   = {KATHE 2026 shared task},
  url    = {https://huggingface.co/Baakaa/sincerox-kashmiri}
}

@article{gemma_2025,
  title     = {Gemma 3},
  author    = {{Gemma Team}},
  publisher = {Google DeepMind},
  year      = {2025},
  url       = {https://arxiv.org/abs/2503.19786}
}