Aadhil96/gemma3-4b-sinhala-v1

Gemma 3 4B Sinhala v1 (experimental)

Experimental, not suitable for real use. In its one recorded test answer, this first version produced mostly meaningless Sinhala, and it did not meet its quality target. See "Known problems" below.

A LoRA adapter for unsloth/gemma-3-4b-it-bnb-4bit, fine-tuned on machine-translated Sinhala instructions.

Training notebook: https://www.kaggle.com/code/aadhilimam/sinhala-llm-fine-tuning-on-kaggle

Known problems

• Poor output: asked for the capital of Sri Lanka, the model replied with a mostly meaningless sentence containing stray non-Sinhala characters.
• Missed quality target: mean perplexity was 23.11 on 50 held-out rows, against a target of 15 or lower.
• Broken spelling in the training data: the translation step dropped the zero-width joiner, leaving a space in its place (for example ප් රධාන instead of ප්‍රධාන). This affected 4,392 of the 5,000 translated rows, and the model was trained on that text. The data was not repaired before training.
• Other data noise: about 7.7% of translated answers repeat themselves, and 228 were less than half Sinhala.
• No native-speaker review: no Sinhala speaker has reviewed the output.
• Limited training record: training loss was not captured in the run log.

Training

Setting	Value
Base model	unsloth/gemma-3-4b-it-bnb-4bit
Data	First 5,000 rows of yahma/alpaca-cleaned, translated with facebook/nllb-200-distilled-600M, plus 2 hand-written rows
Split	4,642 train, 245 held out, after cleaning
Method	QLoRA with Unsloth and TRL (r = 16, alpha = 16)
Training	2 epochs, 582 steps, batch size 2 with 8 accumulation steps, learning rate 2e-4
Hardware	One NVIDIA Tesla T4 on Kaggle

License

Use of this adapter with the base model is subject to the Gemma terms of use: https://ai.google.dev/gemma/terms