shahbakhsh/balochi-pos-tagger

Balochi POS Tagger

A Part-of-Speech tagger for Balochi language written in Arabic script, fine-tuned from distilbert-base-multilingual-cased on the Balochi CoNLL-U dataset.

Test Set Results

Metric	Score
Accuracy	0.7002
Precision	0.5994
Recall	0.5289
F1 Macro	0.5360
F1 Weighted	0.6541

Labels

{ "ADJ": 0, "ADP": 1, "ADV": 2, "AUX": 3, "CCONJ": 4, "DET": 5, "INTJ": 6, "NOUN": 7, "NUM": 8, "PART": 9, "PRON": 10, "PROPN": 11, "PUNCT": 12, "SCONJ": 13, "VERB": 14, "X": 15 }

Usage

from transformers import AutoTokenizer, AutoModelForTokenClassification
import torch

tokenizer = AutoTokenizer.from_pretrained("shahbakhsh/balochi-pos-tagger")
model = AutoModelForTokenClassification.from_pretrained("shahbakhsh/balochi-pos-tagger")

sentence = 'ایش انت ملّاہانی حکومت ءِ سوگات'
words = sentence.split()
inputs = tokenizer(words, is_split_into_words=True, return_tensors='pt')

with torch.no_grad():
    outputs = model(**inputs)

predictions = outputs.logits.argmax(-1)[0]
word_ids = inputs.word_ids()

for idx, word_id in enumerate(word_ids):
    if word_id is not None:
        print(words[word_id], model.config.id2label[predictions[idx].item()])

Built by

shahbakhsh — AI/ML Engineer, Balochistan, Pakistan