HaniaEmaan/code-switching-codesaviours-si26-hania

Code-Switching Token Classification Model (Roman Urdu & English)

Model Overview

This model is a fine-tuned XLM-RoBERTa (base) architecture trained to perform word/token-level Language Identification on code-mixed social text containing Roman Urdu and English.

• Developed by: Hania Emaan
• Model Type: Token Classification / Sequence Labeling
• Base Model: xlm-roberta-base
• Languages: Roman Urdu, English

---

Label Scheme & Categories

Label	Description	Example Words
URD	Roman Urdu tokens	aaj, kal, kaam, hai
ENG	English tokens	project, submission, file
MIX	Hybrid code-mixed tokens	Hybrid words with local inflections

---

Evaluation Results

The model was evaluated on a held-out test set using token-level precision, recall, and F1-scores via seqeval:

Class / Label	Precision	Recall	F1-Score
ENG	0.99	0.99	0.99
URD	0.91	0.89	0.90
MIX	0.00	0.00	0.00
Overall (Micro Avg)	0.98	0.97	0.97

---

How to Get Started

You can load and test this model directly using Hugging Face's pipeline:

from transformers import pipeline

nlp = pipeline(
    "token-classification", 
    model="HaniaEmaan/code-switching-codesaviours-si26-hania", 
    aggregation_strategy="simple"
)

sentence = "aaj mera project submission complete ho gaya hai"
predictions = nlp(sentence)

for pred in predictions:
    print(f"Token: {pred['word']} | Label: {pred['entity_group']} | Score: {pred['score']:.4f}")