Aru-Niya/tanglish-offensive-lora

Tanglish Offensive Language Classifier

A LoRA adapter fine-tuned on Qwen2.5-3B-Instruct for 6-class offensive language classification in Tamil-English code-mixed text ("Tanglish") — the way people actually write on social media in Tamil-speaking communities.

Model Details

• Developed by: Aruniya Nadeshan
• Base model: Qwen/Qwen2.5-3B-Instruct
• Language(s): Tamil, English (code-mixed)
• Task: Text classification — offensive language detection
• Training method: QLoRA (4-bit quantization + LoRA)
• GitHub: https://github.com/Aruniya-Nadeshan/tanglish-offensive-lora

Dataset

DravidianCodeMix Tamil-English split (Chakravarthi et al., 2021) — 44,000 manually annotated YouTube comments across 6 offensive language categories.

Class	% of training data
Not_offensive	72.3%
Offensive_Untargetede	8.3%
Offensive_Targeted_Insult_Group	7.3%
Offensive_Targeted_Insult_Individual	6.7%
not-Tamil	4.1%
Offensive_Targeted_Insult_Other	1.3%

Training Details

• LoRA rank: 16, alpha: 32
• Target modules: q_proj, k_proj, v_proj, o_proj
• Trainable parameters: 7.37M / 3.09B total (0.24%)
• Training steps: 300
• Effective batch size: 16
• Learning rate: 2e-4 with cosine decay
• Experiment tracking: Weights & Biases

Results

Evaluated on 500-example stratified sample of official test split:

Model	Weighted F1	Macro F1
TF-IDF + Logistic Regression	0.676	0.426
Zero-shot Qwen2.5-3B-Instruct	0.569	0.161
This model (QLoRA fine-tuned)	0.648	0.252

Fine-tuning improved macro F1 by 56% over zero-shot (0.161 → 0.252).

How to Use

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen2.5-3B-Instruct")
base_model = AutoModelForCausalLM.from_pretrained(
    "Qwen/Qwen2.5-3B-Instruct",
    torch_dtype=torch.bfloat16,
    device_map="auto"
)
model = PeftModel.from_pretrained(base_model, "Aru-Niya/tanglish-offensive-lora")

Citation

@inproceedings{chakravarthi2021findings,
  title={Findings of the Shared Task on Offensive Language Identification
         in Tamil, Malayalam, and Kannada},
  author={Chakravarthi, Bharathi Raja and others},
  booktitle={Proceedings of the First Workshop on Speech and Language
             Technologies for Dravidian Languages},
  year={2021}
}

Framework versions

• PEFT 0.19.1