AmnaNoor123/code-switching-codesaviours-si26-amna

Code-Switching Language ID Model — Code Saviours SI-26 (Amna)

Model Description

This model fine-tunes xlm-roberta-base for token classification on Roman Urdu–English code-switched text. Given a sentence like "Aaj mera mood nahi hai for anything", it labels every word as:

• URD — Roman Urdu
• ENG — English
• MIX — a hybrid token (Urdu morphology applied to an English root, e.g. tension-wention)

Training Data

Fine-tuned on the code-switching-codesaviours-si26-amna dataset — 160 real, naturally occurring Roman Urdu-English sentences (1,677 labelled words) collected from Pakistani Twitter/X, Reddit (r/pakistan), YouTube comments, and everyday WhatsApp-style phrasing.

Training Procedure

• Base model: xlm-roberta-base
• Task: Token classification (3 labels: URD, ENG, MIX)
• Split: 128 training sentences / 32 test sentences (80/20)
• Epochs: 5
• Batch size: 16

Evaluation Results

Evaluated on the held-out test set (334 labelled word tokens):

Label	Precision	Recall	F1-score	Support
URD	0.990	0.981	0.985	206
ENG	0.946	0.984	0.965	125
MIX	0.000	0.000	0.000	3

Overall accuracy: 0.973 · Weighted F1: 0.969

Limitations

• MIX scored 0 because the dataset only contains 11 MIX examples in total (a genuinely rare linguistic phenomenon — Urdu echo-reduplication on an English root), leaving too few training/test examples for the model to learn the pattern. More MIX-labelled data is needed to improve this.
• Trained on a small (160-sentence), single-annotator dataset skewed toward casual, everyday topics — may not generalise well to formal or domain-specific text.

How to Use

from transformers import pipeline

classifier = pipeline(
    "token-classification",
    model="AmnaNoor123/code-switching-codesaviours-si26-amna",
    aggregation_strategy=None,
)

result = classifier("Bhai kal ka plan finalize ho gaya, I'm super excited")
for token in result:
    print(token['word'], '->', token['entity'])

Intended Use

Token-level language identification for Roman Urdu-English code-switched text — a building block for NLP tools (spell-checkers, sentiment models, chatbots) that need to understand how Pakistanis actually write online.

Citation

If you use this model, please credit: Code Saviours SI-26 Internship — Amna, 2026.