SinhalaMMLU-SharedTask/team-Sub4-gemma4-e4b-lora

Team Sub4 - Sinhala MMLU Shared Task (ICTer 2026): LoRA adapter

LoRA adapter (rank 32, alpha 64) for google/gemma-4-E4B-it, fine-tuned on Sinhala school-curriculum multiple-choice questions. It adapts the attention and MLP projections of the language model only (258 layers, 69.8M trainable parameters).

• Base model: google/gemma-4-E4B-it, loaded text-only (audio and vision towers removed).
• Inference parameters: 7.46B text backbone + 0.07B adapter = 7.53B (limit 8B).
• Use: see inference.py in the team repository SinhalaMMLU-SharedTask/team-Sub4. One forward pass per question, a fixed 4-example prompt, the answer is the highest of the four digit logits ("1".."4").
• Training: see training/ and README.md in the team repository. Training data: Sinhala Global MMLU, school past-paper questions, and questions generated from textbook passages with Gemini 3.8 Flash.
• Stage: this adapter is the state after continuation stage 2 (stage 1: rows unseen by the first run; stage 2: all extracted and generated questions with shuffled answer positions).

from peft import PeftModel
model = PeftModel.from_pretrained(base_model, "SinhalaMMLU-SharedTask/team-Sub4-gemma4-e4b-lora")