chandanaau/distilbert-telugu-ner

DistilBERT Telugu NER

Fine-tuned distilbert-base-multilingual-cased for Named Entity Recognition on Telugu, trained on the Naamapadam benchmark (Mhaske et al., ACL 2023).

Research context

Telugu (~80M speakers) is a low-resource language for NLP. Existing SOTA Telugu NER models like IndicNER achieve ~0.80 F1 but rely on 180M+ parameter models pretrained specifically on Indian languages. This work explores whether a smaller, distilled multilingual model (66M params, 40% smaller than BERT) can deliver acceptable NER performance via transfer learning — addressing the accuracy-vs-efficiency tradeoff that decides real-world deployability.

Results

Entity-level F1 on the manually annotated Naamapadam Telugu test set (847 sentences):

Metric	Score
F1	0.7803
Precision	0.7827
Recall	0.7779
Accuracy	0.9389

Trained on 50k sentences (10%) of Naamapadam Telugu with fp16 mixed precision on a single Colab T4 GPU. Total training time: ~11 min.

Model details

• Model size: 66M parameters (40% smaller than BERT-base)
• Base: distilbert-base-multilingual-cased
• Task: Token classification (7-label BIO scheme)
• Labels: O, B-PER, I-PER, B-ORG, I-ORG, B-LOC, I-LOC

Usage

from transformers import pipeline

ner = pipeline(
    "token-classification",
    model="chandanaau/distilbert-telugu-ner",
    aggregation_strategy="first",
)

text = "నరేంద్ర మోదీ న్యూ ఢిల్లీలో మాట్లాడారు"
print(ner(text))

Known limitations

• Domain sensitivity: Naamapadam is auto-projected from English news, so tech and space-industry entities (Infosys, ISRO, Skyroot) are underrepresented and often missed.
• Training data volume: 10% of the full corpus used — full-data training likely reaches ~0.80 F1.

Live demo

Streamlit app (link coming after deployment)

Citation

If you use this model, please also cite the underlying dataset:

@inproceedings{mhaske-etal-2023-naamapadam,
  title = "Naamapadam: A Large-Scale Named Entity Annotated Data for Indic Languages",
  author = "Mhaske, Arnav and Kedia, Harshit and Doddapaneni, Sumanth and others",
  booktitle = "ACL 2023",
  year = "2023"
}

About

Trained by Chandana Nuthangi. Full training code, evaluation scripts, and a live Streamlit app are available at the project repository: chandana-nuthangi/telugu-ner-thesis.

License: MIT