chaturvedipriyanshu17/muril-lora-codemixed-sentiment

MuRIL Fine-tuned for Code-Mixed Sentiment Analysis

Fine-tuned google/muril-base-cased on code-mixed (Hindi-English, Bengali-English) sentiment classification using LoRA (Low-Rank Adaptation).

Model Description

• Base model: MuRIL (Multilingual Representations for Indian Languages) — pre-trained on 17 Indic languages
• Fine-tuning method: LoRA (r=8, alpha=16) via the PEFT library
• Task: 3-class sentiment classification — Negative (0), Neutral (1), Positive (2)
• Dataset: md-nishat-008/Code-Mixed-Sentiment-Analysis-Dataset — 99,999 code-mixed samples

Results

Evaluated on 19,998 held-out test samples. Compared against zero-shot and few-shot LLM prompting (Llama 3.1 8B via Groq API):

Method	Accuracy	Macro F1	Avg Latency
MuRIL zero-shot (no training)	0.3334	0.1667	—
MuRIL + LoRA fine-tuned (this model)	0.7127	0.7142	~5ms (local)
LLM zero-shot (Llama 3.1 8B)	0.6307	0.6106	3934ms/call
LLM 5-shot (Llama 3.1 8B)	0.7186	0.7158	4944ms/call

Key finding: Fine-tuned MuRIL matches 5-shot LLM prompting on F1 (0.7142 vs 0.7158) while running locally at ~1000x lower latency and zero per-query API cost.

Usage

from transformers import pipeline

classifier = pipeline(
    "text-classification",
    model="chaturvedipriyanshu17/muril-lora-codemixed-sentiment",
)

texts = [
    "yaar ye product bilkul bakwaas hai, waste of money",
    "bahut achha product hai, highly recommend!",
    "ठीक है, काम चला सकते हैं but nothing special",
]

for text in texts:
    result = classifier(text)[0]
    print(f"{result['label']} ({result['score']:.2f}): {text}")
# LABEL_0 = Negative, LABEL_1 = Neutral, LABEL_2 = Positive

Training Details

• Epochs: 3
• Batch size: 32
• Learning rate: 2e-4
• Max sequence length: 128
• Precision: fp16
• Hardware: Kaggle P100
• Trainable parameters: 297,219 / 237,855,750 (0.125%) — LoRA keeps base model frozen

Label Map

Label	Sentiment
LABEL_0	Negative
LABEL_1	Neutral
LABEL_2	Positive

Limitations

• Trained primarily on product/service reviews — may not generalise to other domains
• Dataset contains Bengali-English mixing in addition to Hindi-English; performance may vary by language pair
• Not evaluated on pure Hindi or pure English text

Citation

arxiv: 2309.10272 — Mixed-Distil-BERT: Code-mixed Language Modeling for Indo-Aryan Languages
arxiv: 2103.07792 — Multilingual Code-Switching for Zero-Shot Cross-Lingual Intent Prediction