pankajpandey-dev/gemma-4-e4b-hindi-instruct-lora

🇮🇳 Gemma-4-E4B-Hindi-Instruct — LoRA adapter

LoRA adapter (r=16) for Hindi instruction-tuning of Gemma 4 E4B. Apply on top of the base model, or use the ready-made merged / GGUF builds.

Part of my Hindi LLM Series.

---

Usage (PEFT)

from transformers import AutoModelForCausalLM
from peft import PeftModel

base = AutoModelForCausalLM.from_pretrained("unsloth/gemma-4-E4B-it", device_map="auto")
model = PeftModel.from_pretrained(base, "pankajpandey-dev/gemma-4-e4b-hindi-instruct-lora")

Or with Unsloth:

from unsloth import FastModel
model, tok = FastModel.from_pretrained("pankajpandey-dev/gemma-4-e4b-hindi-instruct-lora")

---

Training details

Base model	unsloth/gemma-4-E4B-it
Method	LoRA (r=16, α=16), response-only loss
Framework	Unsloth
Data	~10k Hindi instruction pairs (AI4Bharat indic-instruct: anudesh + dolly, hi splits)
Epochs	2
LR / schedule	1e-4, cosine
Precision	bf16 (4-bit QLoRA base)
Hardware	Single NVIDIA L4 (24 GB)
Final train loss	~0.29

---

Related repos

• 16-bit model: pankajpandey-dev/gemma-4-e4b-hindi-instruct
• GGUF (Q4/Q5/Q8): pankajpandey-dev/gemma-4-e4b-hindi-instruct-GGUF

---

Provenance & license (please read)

Mixed-license lineage — review all before redistribution or commercial use:

• Adapter trained on Gemma 4 (Gemma Terms of Use).
• Data from AI4Bharat indic-instruct-data-v0.1: Dolly (CC-BY-SA-3.0) + Anudesh (responses from Llama-2-70B, Llama 2 Community License).

You are responsible for complying with the Gemma, Llama 2, and CC-BY-SA terms.

Acknowledgements

Base model by Google (Gemma 4). Data by AI4Bharat. Fine-tuning with Unsloth. 🙏