Minuri/sinhala-llama-1b-sft-diverse

Model Description

An instruction-tuned Sinhala language model fine-tuned on Minuri/sinhala-llama-1b-corpus-diverse (diversity-optimised continually pretrained LLaMA 3.2 1B). This is the best-performing model in the series, built on top of the CPT model that achieved the lowest perplexity (10.49). Part of a diversity-driven Sinhala language model adaptation study.

SFT model variants in this series:

• Minuri/sinhala-llama-1b-sft-baseline - SFT on base LLaMA 3.2 1B (no CPT)
• Minuri/sinhala-llama-1b-sft-news - SFT on sinhala-llama-1b-corpus-news
• Minuri/sinhala-llama-1b-sft-random - SFT on sinhala-llama-1b-corpus-random
• Minuri/sinhala-llama-1b-sft-diverse - SFT on sinhala-llama-1b-corpus-diverse - this repo

This model is the result of supervised fine-tuning (SFT) of Minuri/sinhala-llama-1b-corpus-diverse on the Minuri/sinhala-sft-dataset (~213K Sinhala instruction pairs). The sinhala-llama-1b-corpus-diverse model was continually pretrained on a diversity-optimised Sinhala corpus prior to SFT, achieving a perplexity of 10.49 - the lowest among the three CPT variants.

Training Data

Dataset	Description
Minuri/sinhala-sft-dataset	~213K Sinhala instruction pairs merged from three source datasets

Usage

from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("Minuri/sinhala-llama-1b-sft-diverse")
model = AutoModelForCausalLM.from_pretrained("Minuri/sinhala-llama-1b-sft-diverse")

Intended Uses

• Sinhala instruction following
• Low-resource NLP research and benchmarking
• Base for further Sinhala LLM research

Limitations

• 1B parameter model with limited reasoning capability

Related Repositories

Repo	Description
Minuri/sinhala-llama-1b-corpus-diverse	Base model
Minuri/sinhala-sft-dataset	SFT training dataset (~213K pairs)
Minuri/sinhala-llama-3.2-1b-tokenizer	Extended Sinhala tokenizer
Minuri/sinhala-llama-1b-sft-baseline	SFT baseline
Minuri/sinhala-llama-1b-sft-random	SFT on sinhala-llama-1b-corpus-random model
Minuri/sinhala-llama-1b-sft-diverse	SFT on sinhala-llama-1b-corpus-diverse model

License

This model is derived from meta-llama/Llama-3.2-1B and is subject to the LLaMA 3.2 Community License.