Aounali/nllb600m-sindhi-dsm5-terms

English → Sindhi DSM-5-TR Neurodevelopmental Disorder Terms (NLLB-200 LoRA)

A LoRA adapter for facebook/nllb-200-distilled-600M that translates 18 DSM-5-TR neurodevelopmental disorder terms from English into the expert-consensus Sindhi (Arabic script) renderings established in:

Ali, A., Dahri, B. A. and Jillani, S. H. A. S. (2026) 'Evaluating large language models for clinical terminology translation in low-resource languages: evidence from English-to-Sindhi neurodevelopmental disorder terms', International Journal of Advanced Research, 3(5), pp. 248–258. https://doi.org/10.5281/zenodo.22684171

Authors

• Aoun Ali (corresponding) — Government College of Education, Federal B Area, Block 15, Karachi, Pakistan
• Barkat Ali Dahri — Government College of Education, Federal B Area, Block 15, Karachi, Pakistan
• Syed Hussain Ali Shah Jillani — Institute of Professional Psychology, Bahria University, Karachi, Pakistan

Contributions to the terminology and this model

• Aoun Ali — translated the 18 terms and created the Sindhi terminology, carried out all statistical analysis, and trained and evaluated this model (training and evaluation scripts were written with AI assistance and verified by the author).
• Barkat Ali Dahri — proofread the Sindhi terms and obtained their finalisation from the relevant departments.
• Syed Hussain Ali Shah Jillani — contributed to writing the discussion of the article.

Background

The article found that ChatGPT, Gemini, Microsoft Copilot and DeepSeek reproduced none of the 18 expert Sindhi terms exactly. This adapter is a follow-up: a small open model fine-tuned on the expert-consensus terms so that it outputs the agreed terminology.

Intended use

• Reference and research on standardised Sindhi terminology for neurodevelopmental disorders.
• Starting point for further fine-tuning with more Sindhi clinical text.

Not for clinical decision-making. Output must be reviewed by a qualified Sindhi-speaking professional.

How to use

from transformers import AutoTokenizer, AutoModelForSeq2SeqLM
from peft import PeftModel

base = "facebook/nllb-200-distilled-600M"
repo = "Aounali/nllb600m-sindhi-dsm5-terms"
tok = AutoTokenizer.from_pretrained(base, src_lang="eng_Latn", tgt_lang="snd_Arab")
model = PeftModel.from_pretrained(AutoModelForSeq2SeqLM.from_pretrained(base), repo)

enc = tok(["Autism Spectrum Disorder"], return_tensors="pt")
out = model.generate(**enc, forced_bos_token_id=tok.convert_tokens_to_ids("snd_Arab"),
                     num_beams=4, max_new_tokens=64)
print(tok.batch_decode(out, skip_special_tokens=True)[0])
# پاڻ پڻي جي تسلسل ۾ بي ترتيبي

Training data

The 18 English source terms from the DSM-5-TR Neurodevelopmental Disorders chapter, paired with the consensus Sindhi reference built for the article: three Sindhi-language experts (education, psychology and translation) translated each term independently and then agreed one rendering per term. The 18 consensus terms appear in the article's Appendix A and in the gold column of the files in results/. Gold file SHA-256: c91f34fe9841366da033eca66e6025516a7f6b720a6656947aa3660fedff0177

Training

LoRA r=16, alpha=32, dropout 0.05 on q/k/v/out_proj and fc1/fc2 (8.65M trainable parameters, 1.39%). AdamW lr 1e-3, batch 6, 40 epochs, seed 13, CPU. Decoding: beam 4.

Evaluation

Exact match is strict character identity after trimming leading/trailing whitespace (no further normalisation). chrF++ via sacreBLEU.

Condition	n	Exact match	Wilson 95% CI	chrF++
Base NLLB-600M, no training	18	0	0.00–0.18	14.7
LoRA, 3-fold cross-validation (terms not seen in training)	18	0	0.00–0.18	51.7
LoRA trained on all 18 (terms seen in training)	18	17 (18 ignoring spacing)	0.74–0.99	100.0

Per-term predictions are in results/.

Limitations

• Use full term names, not abbreviations. Abbreviations were not in training and give wrong diagnoses: ASD returns a tic-disorder-like term and ADHD returns an incorrect term. Write "Autism Spectrum Disorder", "Attention-Deficit/Hyperactivity Disorder", etc.
• Use only for these clinical terms, not general Sindhi text. Fine-tuning on 18 terms degraded ordinary translation compared with the base model: an English word leaked into Sindhi output ("pleasant" → ڏا pleasantي), "door" was mistranslated, and the adapter replaced ه (U+0647) with ھ (U+06BE) in common words (e.g. آهي → آھي). For general text use the base facebook/nllb-200-distilled-600M model without the adapter.
• Casing and apostrophe variants (e.g. "tourette's disorder", "Tourette’s Disorder") work correctly, and in three test sentences the exact term was produced inside the Sindhi sentence (rest of each sentence not expert-reviewed). Full QA output: results/qa_check.json.
• Trained on 18 terms only. It reproduces those terms, but produced no unseen term exactly in cross-validation. Expect errors on other terms and on full sentences.
• The 18/18 result reflects learning of the training terms, not generalisation.
• The consensus reference is one agreed rendering; other renderings may also be acceptable.
• The NLLB tokenizer collapses a double space in one term; no letters are changed.
• Not yet evaluated on terms within running clinical text or rated by practising clinicians.

License

The base model is CC-BY-NC-4.0, so this adapter is released for non-commercial use under the same license.

Citation

Model DOI: https://doi.org/10.57967/hf/10399

@misc{ali2026sindhimodel,
  author    = {Ali, Aoun},
  title     = {nllb600m-sindhi-dsm5-terms: English to Sindhi DSM-5-TR Neurodevelopmental Disorder Terms (NLLB-200 LoRA)},
  year      = {2026},
  publisher = {Hugging Face},
  doi       = {10.57967/hf/10399},
  url       = {https://huggingface.co/Aounali/nllb600m-sindhi-dsm5-terms}
}

Please also cite the article:

@article{ali2026sindhi,
  author  = {Ali, Aoun and Dahri, Barkat Ali and Jillani, Syed Hussain Ali Shah},
  title   = {Evaluating Large Language Models for Clinical Terminology Translation in Low-Resource Languages: Evidence from English-to-Sindhi Neurodevelopmental Disorder Terms},
  journal = {International Journal of Advanced Research},
  volume  = {3},
  number  = {5},
  pages   = {248--258},
  year    = {2026},
  doi     = {10.5281/zenodo.22684171}
}