Minmini-1B

Beats 105-billion-parameter Sarvam on Tamil inference and reading comprehension, at 1/100th the size. IndicXNLI 76.0% vs 58.2%; IndicQA F1 52.4 vs 44.5 (1,000 items each, same prompts).

Minmini (மின்மினி, "firefly") is a 1-billion-parameter Tamil language model built on Google's Gemma-3-1B and trained entirely on one laptop. On Tamil-Bench's leaderboard items it scores higher than Gemini 3.8 Flash and Claude Sonnet 5.5 on Tamil natural-language inference (78.5%) and reading comprehension (IndicQA F1 48.5). It also translates English↔Tamil (FLORES chrF 44.7 / 45.6).

These are supervised, in-format results: Minmini was fine-tuned on these task formats, while the models it is compared with were not. It is a small specialist, not a general assistant. See Limitations.

What Minmini is best for

Minmini is best at tasks that work from text you give it: answering from a passage, checking a claim against a statement, and translating. It is not a knowledge source and cannot do maths or multi-step reasoning.

Use	Fit	Evidence
Answering questions from a Tamil passage (support docs, FAQs, forms)	Best	IndicQA F1 52.4; says "no answer in the passage" reliably (AUC 0.88)
Checking whether a Tamil statement follows from, contradicts or is unrelated to a text	Best	IndicXNLI 76.0%, robust to answer-option order (75.2–76.3%)
English↔Tamil translation of short sentences	Good	FLORES chrF 44.7 / 45.6
Drafting short Tamil text (messages, lists, short paragraphs)	Fair	Fluent, grammatical Tamil; drifts off topic in longer replies
On-device or offline Tamil NLP	Good	1B parameters, 2 GB in bf16; runs on a laptop
Facts, general knowledge, exam questions	Not suitable	MILU 25.7%, Belebele 24.1% (chance is 25%); invents facts
Maths, logic and step-by-step reasoning	Not suitable	Failed all 5 simple test questions (e.g. 12 − 5 + 3)
Long-form writing or multi-turn chat	Not suitable	Loops under greedy decoding; loses the thread after a few sentences

How to use

The weights are standard Gemma-3 safetensors (bf16) and load in both transformers and MLX. Chat format is Gemma's; a system message is folded into the first user turn. Generation stops at <end_of_turn>.

transformers

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

repo = "Felamity-Technologies/Minmini-1B"
tok = AutoTokenizer.from_pretrained(repo)
model = AutoModelForCausalLM.from_pretrained(repo, dtype=torch.bfloat16)

msgs = [{"role": "user", "content": "Translate to Tamil:\nThe weather is very hot today."}]
inputs = tok.apply_chat_template(msgs, add_generation_prompt=True, return_tensors="pt", return_dict=True)
out = model.generate(**inputs, max_new_tokens=100, do_sample=False)
print(tok.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))
# இன்று வானிலை மிகவும் சூடாக உள்ளது.

For free-form replies longer than a sentence or two, sample instead of greedy decoding, which can loop: do_sample=True, temperature=0.7, top_p=0.95, repetition_penalty=1.1.

MLX (Apple silicon)

from mlx_lm import load, generate

model, tok = load("Felamity-Technologies/Minmini-1B")
ids = tok.apply_chat_template([{"role": "user", "content": "Translate to Tamil:\nThe weather is very hot today."}],
                              add_generation_prompt=True)
print(generate(model, tok, prompt=ids, max_tokens=100))

Answering from a passage (best results)

Minmini was trained on this exact prompt. Send it as a single user message:

You are a helpful assistant that answers reading comprehension questions in Tamil.

Read the following passage and answer the question. Reply ONLY with a short phrase from the passage, in Tamil. No explanation.

Passage: {context}

Question: {question}
Answer:

An empty reply means "the answer is not in the passage". The reported scores use a fixed rule: take the probability of <end_of_turn> (token id 106) as the first generated token; if it is above 0.5, treat the question as unanswerable; otherwise generate with that token blocked for the first step.

Natural-language inference (best results)

You are a careful reasoner. Answer with the single letter A, B, or C.

பின்வரும் வாக்கியத்தைப் படிக்கவும்:

முன்னுரை: {premise}

கூற்று: {hypothesis}

முன்னுரையைப் பொறுத்து, கூற்று எந்த நிலையில் உள்ளது?
A) கண்டிப்பாக உண்மை (முன்னுரையால் உறுதிப்படுத்தப்படுகிறது)
B) கண்டிப்பாக தவறு (முன்னுரையுடன் முரண்படுகிறது)
C) முடிவு செய்ய முடியாது (முன்னுரையில் தெளிவான தகவல் இல்லை)

Reply with ONLY the single letter A, B, or C.

A = entailment, B = contradiction, C = neutral. Comparing the probabilities of the three letter tokens gives a confidence score.

Results

Tamil-Bench leaderboard items

The exact items Tamil-Bench's leaderboard uses: the first 200 IndicXNLI pairs and first 100 IndicQA questions of its fixed shuffle (seed 42). Same prompts and answer parsers for every model.

Model	IndicXNLI accuracy % (n=200)	IndicQA EM / F1 (n=100)
Minmini-1B	78.5	42.0 / 48.5
Gemini 3.8 Flash	75.8	20.0 / 42.4
Claude Sonnet 5.5	75.5	22.0 / 43.1
GLM 5.3 Flash	71.5	19.0 / 36.8
GPT-6 Luna	65.0	16.0 / 40.5
Gemma 4 26B A4B	64.8	20.2 / 39.3
Liquid LFM2.5 2.6B	64.6	18.0 / 35.3
DeepSeek V4.1 Flash	62.0	18.0 / 38.9
gpt-oss-20b	62.0	20.0 / 40.7
GPT-5.6 Luna	61.0	17.0 / 40.7
Sarvam-105B (measured by us)	45.5 strict / 60.0 lenient	20.0 / 38.8

Other models' scores are from the Tamil-Bench leaderboard (generated 2026-10-02). At n=200 the 95% interval on XNLI is about ±6 points, so the XNLI lead is within noise; the IndicQA lead is larger. Much of Minmini's IndicQA edge comes from correctly declining unanswerable questions, which chat models usually attempt.

Larger samples (n=1000)

Task	Minmini-1B	Sarvam-105B
IndicXNLI accuracy	76.0%	43.0% strict / 58.2% lenient
IndicQA EM / F1, all questions	44.3 / 52.4	29.1 / 44.5
IndicQA F1, answerable questions only	41.4	54.0
MILU accuracy (5-shot)	25.7%	—

Translation and general benchmarks

lm-eval 0.4.13, full test sets, translation and MILU 5-shot, others 0-shot, plain prompts (no chat template).

Task	Minmini-1B
FLORES en→ta / ta→en (chrF)	44.7 / 45.6
IN22-Gen en→ta / ta→en (chrF)	40.1 / 42.6
IN22-Conv en→ta / ta→en (chrF)	34.9 / 39.1
MILU / Belebele / MMLU (ta)	26.8 / 24.1 / 26.2
XCOPA / MultiBLiMP (ta)	58.0 / 97.1

Training

All training ran on one 24 GB Apple M5 Pro MacBook with MLX, using LoRA (rank 64 on all attention and MLP projections).

1. Continued pre-training on 664M tokens of Tamil from IndicCorpV2 (about 77 hours). Tamil bits per character fell from 1.283 to 1.054.
2. Five supervised fine-tuning stages (about 20 hours in all), each added to fix a weakness found by evaluation:
   • chat format, NLI, QA and general instructions (IndicXNLI, IndicSQuAD, Aya, IndicAlign);
   • QA rebuilt so the model learns real "no answer" cases;
   • NLI with shuffled answer options, twice, to remove a letter-position bias;
   • English↔Tamil translation (IndicAlign sentence pairs).
3. Weight blend: the released model is 0.3 × the stage-4 model (best at NLI and QA) + 0.7 × the stage-5 model (translation), averaged weight by weight. The blend keeps 83–95% of the translation gain while holding NLI at 76%.

All training text was n-gram decontaminated against MILU, IndicXNLI-test, IndicQA, FLORES and IN22. No test items were used.

Findings

• Translated QA data carries a shortcut. After the first SFT stage the model answered "no answer" to every IndicQA question. The machine-translated unanswerable questions in IndicSQuAD carry a surface cue that human-written questions lack. Building unanswerables from other paragraphs of the same article fixed it (abstention AUC 0.50 → 0.88).
• Benchmark scores can hide position bias. A model scoring 73.5% in the benchmark's fixed option order dropped to 11.7–38.3% when the same options were reordered: it had learned "entailment means A". Training with shuffled options fixed this.
• Continued pre-training erased few-shot translation, and a targeted SFT stage plus weight blending restored most of it without losing the other skills.

Limitations

• Supervised, in-format: 50–100% of the NLI and 60–75% of the QA fine-tuning examples used Tamil-Bench's prompt formats. The comparison models were evaluated zero-shot. The results show a small specialist can match large general models on these two tasks, not that it is generally stronger.
• No knowledge or reasoning: knowledge benchmarks are at chance, and the model fails simple arithmetic and logic. Open-ended factual answers are often wrong.
• Not evaluated: safety, toxicity, bias, long-form generation and multi-turn chat. There was no human evaluation of fluency.
• Tokenizer: Gemma's tokenizer uses about 2.4 tokens per Tamil word, so Tamil text is slower and costlier than with a Tamil-specific tokenizer.
• Do not use Minmini for medical, legal, financial or other high-stakes decisions.

Licence and attribution

Minmini-1B is a derivative of Gemma and is distributed under the Gemma Terms of Use. Gemma is provided under and subject to the Gemma Terms of Use found at ai.google.dev/gemma/terms. Use of this model must comply with the Gemma Prohibited Use Policy, and anyone you share it with is bound by the same terms.

Training data:

• IndicCorpV2 (AI4Bharat, CC0)
• IndicXNLI (CC0)
• IndicSQuAD (CC-BY-4.0)
• IndicAlign (AI4Bharat, CC-BY-4.0)
• Aya Dataset (Cohere For AI, Apache-2.0)

Evaluation uses Tamil-Bench, FLORES-200 and IN22.

Creator

Nafize Ahamed F

Citation

@misc{minmini2026,
  title  = {Minmini-1B: a small Tamil language model trained on a laptop},
  author = {Nafize Ahamed F},
  year   = {2026},
  url    = {https://huggingface.co/Felamity-Technologies/Minmini-1B}
}