Bharat-Tiny-LLM 🇮🇳

Hinglish AI — trained on a Mac Mini M4, runs on ₹8,000 phones.

---

The Story

I trained an AI to speak Hinglish on a Mac Mini. No cloud compute. No GPU cluster. Just $750 of Apple Silicon and a lot of chai.

This is Bharat-Tiny-LLM — a Qwen2.5-1.5B model fine-tuned on 376K Hinglish conversations. Quantized to 828 MB (Q4), it delivers ~57 tokens/second on a Mac Mini M4 and is small enough to run on an ₹8,000 smartphone.

Most AI models today are skyscrapers — built by billion-dollar companies, requiring data centers, and serving only English. We built a bicycle. A tiny, efficient, open-source model that speaks the way 600 million Indians actually talk: Hinglish.

What Makes This Different

	Big AI	Bharat-Tiny-LLM
Hardware	$100M GPU clusters	Mac Mini M4 (16GB)
Cloud Cost	Millions	$0
Language	English only	Hinglish (हिंग्लिश)
Model Size	70B–405B params	1.5B (828 MB Q4)
Device	Data centers	₹8,000 phones
Speed	API latency	57 tok/s on-device
License	Proprietary/Research	Apache 2.0

---

Model Details

Architecture

• Base Model: Qwen2.5-1.5B (Apache 2.0)
• Fine-Tuning Method: LoRA (16 layers, rank 8, alpha 16)
• Training Hardware: Mac Mini M4, 16GB unified memory
• Training Duration: ~3.3 days for 76K iterations
• Quantization: 4-bit (Q4) via MLX — 4.501 bits/weight
• Total Trainable Parameters: 5.276M (0.342% of base)

Base model details (click to expand)

This model is fine-tuned from Qwen2.5-1.5B (Apache 2.0), one of the few small models with native multilingual support (29 languages including Hindi). Unlike English-only models (like SmolLM2, which we tried first — val loss plateaued at 1.32), Qwen2.5 already understands Hindi morphology. LoRA fine-tuning on Hinglish data was enough to make it conversational.

Performance

Metric	Value
Training Data	376,040 conversations
Validation Loss	0.781 (best at 75K iters)
Inference Speed	~57 tok/s (Mac Mini M4)
Model Size	828 MB (Q4 quantized)
Memory Usage	~1.2 GB at inference
Context Length	512 tokens

---

Quick Start

Using mlx-lm (macOS / Apple Silicon)

pip install mlx-lm

python3 -c "
from mlx_lm import load, generate
from mlx_lm.sample_utils import make_sampler

model, tokenizer = load('eulogik/Bharat-Tiny-LLM')
sampler = make_sampler(temp=0.7)

prompt = '<|im_start|>user\nChai peete hain?<|im_end|>\n<|im_start|>assistant\n'
response = generate(model, tokenizer, prompt=prompt, max_tokens=200, sampler=sampler)
print(response)
"

Platform Support

Platform	Status	Instructions
macOS (Apple Silicon)	✅ Works	Use mlx-lm (above)
Linux / Windows	🔄 Coming	Need transformers-compatible bf16 export
Android	🔄 Coming	MLX Android bindings in development
Raspberry Pi 5	🔄 Coming	Testing in progress

---

Example Outputs

Prompt	Response	Quality
"Kal interview hai, nervous ho raha hoon"	"Chill maaro, you'll be alright. Bas apne aap ko confident rakhna aur jo aata hai woh bol dena. Sab theek hoga."	✅
"Biryani kaise banate hain?"	Gives structured recipe with ingredients list and step-by-step instructions in Hinglish	✅
"Delhi me rehne ke liye kya karna padega?"	Specific advice on metro, parking, traffic, and areas to live	✅
"Weekend pe kya karein?"	10 ideas with examples — museums, parks, street food, movies	✅
"Mujhe Hindi seekhni hai"	Practical listening tips, YouTube channels, and practice strategies	✅
"Chai peetey hain?"	Full chai recipe with proportions and tips	✅
"Mera phone charge nahi ho raha"	Troubleshooting steps (though slightly repetitive)	⚠️
"Job nahi mil rahi"	Can get confused — still improving on emotional/career topics	❌

---

Training Data

We curated 376,040 conversations from 5 open-source datasets, deduplicated and formatted as chat messages:

Dataset	Samples	Domain
Hinglish Conversations	201,633	Natural Hinglish dialogue
IndicVault (Hindi)	74,053	20-topic Hindi QA
IndicVault (Hinglish)	77,210	20-topic Hinglish QA
Hinglish Instruct	10,378	Instruction tuning
Yojana Sahayak	6,828	Government scheme info
cookGPT	5,938	Indian recipes

All data was:

• Trimmed to 512 characters per message (to fit within model context)
• Converted to messages format with user/assistant roles
• Split 95% train / 5% validation

---

Training Details

Hyperparameters

Base model:    Qwen2.5-1.5B (bf16 from mlx-community)
Method:        LoRA (16 layers, rank 8, alpha 16)
Batch size:    4
Seq length:    512
Learning rate: 5e-5 (with cosine schedule)
Iterations:    76,420
Optimizer:     AdamW
Warmup:        100 steps
Seed:          42

Training Curve

Val Loss
1.25 |  █
1.20 |  ██
1.15 |  ███
1.10 |  ████
1.05 |  █████
1.00 |  ██████
0.95 |  ████████
0.90 |  ██████████
0.85 |  ████████████
0.80 |  ██████████████
0.78 |  ████████████████  ← Best at 75K iters
     +------------------------
     0   20K   40K   60K   80K

Speed Optimization

Config	it/s	Memory	Notes
batch=2, seq=1024	0.30	10.7 GB	Initial config
batch=8, seq=256	0.24	10.0 GB	Too much overhead
batch=4, seq=512	0.30	10.3 GB	Final — best balance
batch=4, seq=1024	OOM	—	Exceeded Metal GPU

---

Limitations

1. Not fully converged — Only 76K iterations; val loss was still decreasing at 0.781. More training (target: 110K) would help.
2. Occasional English drift — Some responses switch entirely to English. Needs more Hinglish-only examples.
3. Telugu leakage — IndicVault Hindi data contains some Telugu examples that leak into responses.
4. Emotional/career topics — Responses to queries about jobs, relationships, or mental health can be confused or repetitive.
5. Short context — 512 token limit prevents multi-turn conversations or long-form generation.
6. MLX format only — The Q4 quantized model is in Apple MLX format. We plan to release a transformers-compatible version.
7. Not instruction-tuned — This is a base model fine-tuned on conversations, not RLHF/DPO-tuned. Responses are helpful but not aligned.

---

Roadmap

• Phase 0: Fine-tune Qwen2.5-1.5B with LoRA on Hinglish data
• Phase 1: Quantize to Q4 (828 MB) for edge deployment
• Phase 2: Extend to 8 Indian languages (Tamil, Telugu, Bengali, Marathi, etc.)
• Phase 3: DPO/RLHF for quality improvement
• Phase 4: BharatTiny-Bench evaluation suite
• Phase 5: Web demo with Gradio
• Phase 6: Android / Raspberry Pi 5 deployment

---

Why "Bharat-Tiny-LLM"?

600 million Indians speak Hinglish — a natural mix of Hindi and English. But every LLM today is either English-only or pure Hindi. There is no model that understands "Yaar, kal ka match dekh liya? Rohit ne toh maara!"

We built this because:

• No one builds for edge devices in India. Everyone targets data centers.
• No one speaks Hinglish. Everyone speaks either English or "pure" Hindi.
• No one does it on a budget. Everyone burns millions in cloud compute.

We're building the bicycle while everyone else builds skyscrapers.

---

Citation

@misc{bharat-tiny-llm-2026,
  title = {Bharat-Tiny-LLM: Hinglish Edge AI Trained on a Mac Mini},
  author = {Gautam Kishore},
  year = {2026},
  publisher = {Hugging Face},
  url = {https://huggingface.co/eulogik/Bharat-Tiny-LLM}
}

---

License

Apache 2.0 — Free for commercial and research use. The base model (Qwen2.5-1.5B) is also Apache 2.0.

---

Built with chai ☕, patience 🧘, and a Mac Mini M4 💻 — somewhere in India.