dubspai/chinna-small-40m

Chinna-Small (40M) — a Tamil language model trained from scratch

Chinna (சின்னா) is a small Tamil-first decoder-only language model trained from random initialization. No pretrained weights were loaded at any point. This repository holds the chat (instruction-tuned) checkpoint small-003-sft-003.

Status: experimental research model. Its answers are often factually wrong. Do not use it where accuracy matters.

Parameters	40,315,392
Architecture	Decoder-only Transformer, Llama-style block: pre-RMSNorm, RoPE, SwiGLU, no biases, tied embeddings
Size	10 layers, d_model 512, 8 heads, d_ff 1408, context 1024
Tokenizer	chinna-sp-v1: SentencePiece BPE, 16,000 pieces, byte fallback (sha256 63daa5bedea9)
Pretraining	Tamil Wikipedia (corpus v1, 245.8M tokens), then corpus v2 (Wikipedia + Sangraha + CulturaX) for a further 1.29B tokens
Author	Aravind Raghunathan
Instruction tuning	Tamil Aya Dataset subset, synthetic step-by-step arithmetic, QA from Tamil Wikipedia, grounded QA over retrieved Wikipedia sentences and Wikidata fact lines, abstention examples

See also: chinna-base-97m (the other Chinna size).

Files

File	What it is
chinna-small-40m-f16.gguf	llama.cpp / Ollama export (architecture llama, f16). Tokenization and greedy generation were checked against the PyTorch checkpoint.
Modelfile	Ollama Modelfile (prompt template, stop sequence, sampling defaults)
model.safetensors	PyTorch weights (fp32), Chinna parameter names
config.json	Model config, chat format, tokenizer hash, training provenance
tokenizer.model	SentencePiece model
code/chinna/ + example.py	Minimal inference code for the safetensors weights
eval_results.json	The measured numbers below
SHA256SUMS	Checksums

Usage

Ollama

ollama create chinna -f Modelfile
ollama run chinna "இந்தியாவின் தலைநகரம் எது?"

llama.cpp

llama-cli -m chinna-small-40m-f16.gguf -p "பயனர்: திருக்குறளை எழுதியவர் யார்?
சின்னா:" -n 200 --temp 0.3 --repeat-penalty 1.25 -r "பயனர்:"

PyTorch (pip install torch sentencepiece safetensors pydantic pyyaml)

python example.py "திருக்குறளை எழுதியவர் யார்?"

Prompt format (single turn; the model was tuned on single question–answer pairs):

பயனர்: <question>
சின்னா:

Stop at பயனர்: or end-of-sequence. For arithmetic, use repeat penalty 1.0.

Evaluation

All numbers come from the project's evaluation scripts.

Test	Result
Arithmetic exact match (300 held-out problems)	90.7%
– add / sub / compare / word problems	100% / 97% / 97% / 97%–100%
– multiplication / division	59% / 81%
QA over Wikidata fact lines (198 held-out items)	77.8%
Grounded QA over retrieved local Wikipedia evidence (held-out set)	74.5%
Answers with web evidence (Wikidata + live Tamil Wikipedia): dev 40 / held-out 40	55.0% / 32.5%
Closed-book general knowledge, no evidence: dev 40 / held-out 40	35.0% / 27.5%
Pretraining validation loss (corpus v2, nats/token)	3.1798

Retrieval and web evidence are provided by the separate Chinna inference code (BM25 over Tamil Wikipedia, Wikidata and Tamil Wikipedia APIs); the model alone only sees the prompt.

Limitations

• Small model. Expect fluent-looking Tamil with frequent factual errors and invented details. Open-ended answers (essays, advice, explanations) are largely invented.
• Closed-book facts are unreliable. Use it with retrieved evidence and show sources.
• Arithmetic is reliable only for the trained formats. Multiplication and division depend on memorized 1–12 tables.
• Register: mostly encyclopedic and web Tamil. Colloquial and conversational Tamil are weak.
• Bias: inherits coverage biases of Wikipedia and web text (topics, regions, gender).

Training data and licensing

• Tamil Wikipedia (2023-11-01 dump), CC BY-SA 4.0 — Wikipedia contributors.
• Sangraha verified Tamil (AI4Bharat), CC BY 4.0.
• CulturaX Tamil (gated dataset; inherits mC4 / OSCAR terms).
• Aya Dataset, Tamil subset (Cohere Labs), Apache-2.0.
• Wikidata facts, CC0. Synthetic arithmetic generated in-project.

Code: Apache-2.0. Weights: released under CC BY-SA 4.0 (Creative Commons Attribution-ShareAlike 4.0). If you share or adapt the weights, give attribution (Aravind Raghunathan, Chinna, and the data sources above) and release adaptations under the same license.

Author

Aravind Raghunathan

Website	aravind-r.com
GitHub	github.com/ai-aravind
LinkedIn	linkedin.com/in/aravindai
X / Twitter	@AI_Aravind
YouTube	@AravindAI
Instagram	aravind.aiml
Facebook	Aravind Raghunathan
Book a call	calendly.com/aravind-ai

Citation

@misc{raghunathan2026chinna,
  author = {Aravind Raghunathan},
  title  = {Chinna: a Tamil language model trained from scratch (Chinna-Small (40M))},
  year   = {2026},
  url    = {https://huggingface.co/dubspai/chinna-small-40m}
}

Acknowledgements

Tamil Wikipedia contributors and the Wikimedia Foundation; AI4Bharat (Sangraha); CulturaX authors; Cohere Labs (Aya Dataset); SentencePiece; llama.cpp / GGUF.