Enhancing NER Performance in Low-Resource P akistani Languages using Cross-Lingual Data Augmentation (2025)

Abstract: Named Entity Recognition (NER), a fundamental task in Natural Language Processing (NLP), has shown significant advancements for high-resource languages. However, due to a lack of annotated datasets and limited representation in Pre-trained Language Models (PLMs), it remains understudied and challenging for low-resource languages. To address these challenges, in this paper, we propose a data augmentation technique that generates culturally plausible sentences and experiments on four low-resource Pakistani languages; Urdu, Shahmukhi, Sindhi, and Pashto. By fine-tuning multilingual masked Large Language Models (LLMs), our approach demonstrates significant improvements in NER performance for Shahmukhi and Pashto. We further explore the capability of generative LLMs for NER and data augmentation using few-shot learning.

## introduction the performance of named entity recognition (ner) in low-resource languages faces challenges due to the scarcity of annotated datasets and insufficient coverage in masked large language models (llms) @xcite . causal llms, on the other hand, demonstrate their performance by achieving moderate scores for ner @xcite . these challenges make it difficult to develop effective nlp applications and highlight the need of focused effort to improve the applicability of these models on available datasets for low-resource languages. data augmentation approaches could be effective to enhance the ner datasets for low-resource languages. one such approach is the easy data augmentation (eda) @xcite , that offers simple and effective techniques, including synonym replacement, random insertion, random swap, and random deletion @xcite @xcite . however, eda can produce linguistically implausible text lacking verbal agreement based on gender and number. additionally, eda may produce out-of-context or offensive data for culturally sensitive content. this can affect the generalizability and learning of ner models. we aim to enhance ner performance for pakistani low-resource languages by employing effective data augmentation as shown in figure 1 . the sentences 1 and 2, could result in disagreements. momina (nom.fem.sg) is a feminine name that has agreement with the verb parhti (study.hab.fem.sg), while chaudhry muhammad sarwar (nom.masc.sg) is a masculine name that agrees with the verb gae (go.past.masc.sg.hon). replacing these named entities can produce implausible text; for instance, the sentence chaudhry muhammad sarwar studies in government girls high school would violate the verbal agreement rules of the language. the named entities in the last two sentences are considered opposites within the community, and replacing such named entities can produce text that is very offensive to the native community. the generated sentences remain grammatically correct but create contextual ambiguity. we propose a cross-lingual data augmentation technique by clustering named entities as shown in figure 1 . this technique improves the quality of culturally sensitive content and grammar of the augmented text. we performed unsupervised entity clustering and entity replacement by aligning clusters for the source and candidate named entities of each type. ner experiments were conducted for low-resource settings as well as for entire datasets. we compared the results with eda-based and generative augmentation methods for mono-and multilingual settings by fine-tuning the glot500 @xcite and xlm-roberta @xcite models. shahmukhi and pashto datasets demonstrated significant improvements, producing f 1 scores of 88.06 and 88.29 with increases of 5.53 and 1.81 points, respectively. zero-or few-shot learning is relevant in lowresource scenarios where even augmented datasets are limited in size. we explore the capabilities of causal llms to perform ner and data augmenta-tion for our low-resource languages using few-shot learning. the key contributions of this paper are as follows: ## related work manually annotated corpora are crucial for achieving state-of-the-art results in ner @xcite . cross-lingual transfer also supports generalization and enhances the performance of models @xcite @xcite @xcite . data augmentation techniques enhance the size and learning capabilities of datasets for low-resource languages @xcite @xcite . for the task of ner, three data augmentation methods are mainly used; easy data augmentation (eda) @xcite and its variants, translation-based methods and generative llms. eda-based techniques demonstrate enhanced ner performance for low-resource languages @xcite . the data augmentation quality can be enhanced by using contextualized word embeddings @xcite and cosine similarity @xcite . data augmentation based on back-translation has shown improvements for code-switched ner @xcite . the translation-based data augmentation technique that performs cross-lingual entity augmentation also improves the performance of ner models @xcite @xcite . the capabilities of causal llms are being explored for data augmentation @xcite and underlying nlp tasks such as ner @xcite @xcite . generative data augmentation techniques have demonstrated improvements @xcite @xcite . w_0 iob_0 w_1 iob_1 w_2 iob_2 w_3 iob_3 . . . . . w_n iob_n original dataset w_0 iob_0 w_1 iob_1 w_2 iob_2 w_3 iob_3 . . . . . w_n iob_n augmented dataset augmentation process entity dictionary k-means pre-trained word embeddings clusters source entity find cluster id cross-lingual dictionary entity clustering alignment randomized dictionary pre-trained bert emb. entity ranking five aug. candidates x_1/ x_2 augs. ## cross-lingual data augmentation the languages selected in this work are topologically related and culturally similar. in terms of named entities, they share similar names, locations and organizations. given these similarities, cross-lingual representation could be helpful in improving the performance of ner for the regional languages. additionally, data augmentation techniques have shown improvements for low-resource languages, but eda-based methods are blunt and may produce culturally offensive and/or ungrammatical sentences by replacing entities with the other entities of the same type without any additional semantic information. out of 100 randomly selected sentences, 32 instances of verbal disagreements and three of sensitive religious named entities were found. these percentages estimate the occurrence of such issues in the augmented data. to address these issues, we propose a data augmentation technique that generates more sensible sentences and produces competitive ner performance for the selected low-resource languages. the next section describes our proposed technique, followed by descriptions of eda-based random replacement and generative approaches. ## languages and datasets pakistan is home to many widely spoken languages, each with unique linguistic characteristics and cultural significance. urdu is the national language of pakistan that has 232 million speakers worldwide. shahmukhi (punjabi), sindhi, and pashto have 67, 30 and 40 million speakers, respectively (eberhard and fennig, 2024). these languages pose several challenges for the task of ner, such as absence of capitalization, contextual ambiguity, flexible wordorder, and agglutinating nature @xcite @xcite . the statistics of the selected datasets are shown in urdu: being in the vital category (eberhard and fennig, 2024), urdu is relatively resource-rich compared to the regional languages. several ner datasets are available for urdu with different data annotations and sizes @xcite @xcite . however, we experimented with urdu-wikiann @xcite and mk-pucit @xcite , which are larger datasets annotated with coarse-grained named entities; person, location and organization. shahmukhi: there is only one ner dataset available for shahmukhi, which has been annotated using person, location, and organization types (ahmad et al., 2020). the quality of the dataset was further enhanced by using the bio annotation scheme @xcite . the dataset contained some annotation inconsistencies. to ensure the validity of our ner results, we manually reviewed and corrected the annotations in one thousand sentences from the test set. while this review process was conducted to enhance the reliability of our evaluation. sindhi: ali et al. (2020) released the first large annotated dataset for the sindhi language called siner. we experimented with three coarsegrained entity types to make it compatible with the other datasets. pashto: pashto lacks in fundamental language processing tools (eberhard and fennig, 2024). we used the pashto dataset from wikiann @xcite that contains 100 sentences for train, test and validation sets. since the dataset was automatically annotated and exhibited some annotation inconsistencies, we reviewed the test set manually to ensure valid ner results. ## experimental setup we conducted ner experiments designed to improve performance for low-resource languages, where supervised models often struggle due to limited annotated datasets. this research addresses three key questions; 1) how effective are data augmentation techniques to enhance ner for low-resource languages? 2) do cross-lingual data representations improve ner performance in multilingual settings? 3) how does few-shot learning compare to fully supervised models as an alternative to data augmentation? we hypothesize that cross-lingual representations, combined with multilingual datasets improve ner results for topologically related and culturally similar languages. ## results and discussion we use micro f-scores to ensure a balanced evaluation of ner performance across all entity types. our data augmentation technique improved ner results for low-resource languages by reducing the generation of grammatically implausible and culturally offensive content. the augmentation technique helps maintain semantics and cultural appropriateness, that highly impacted the model performance. the model trained on the augmented datasets demonstrated higher generalizability due to less exposure to the contextually implausible in- urdu-wikiann the urdu-wikiann dataset demonstrates inconsistent performance for different augmentation techniques, which is caused by three main reasons. first, urdu is a resource-rich language compared to the other three regional languages and fine-tuning using cross-lingual data augmentation enhances its diversity, but does not significantly impact ner results due to the large size of the dataset. second, causal llms, such as llama3 have better support for urdu compared to the other three languages as urdu dataset shows improvements using generative aug. method. third, the urdu-wikiann dataset is an automatically annotated dataset that may have some inconsistencies @xcite which can limit the effectiveness of cross-lingual augmentation. ## conclusion this study explored various data augmentation techniques and their effect on the task of ner for low-resource languages. we used pre-trained llms on mono-and multilingual setups. our findings highlight that cluster-based data augmentation improves ner performance for shahmukhi, sindhi and pashto datasets by incorporating linguistically plausible text and cross-lingual diversity. urdu-wikiann, an automatically annotated dataset, does not take advantage of cross-lingual augmentations. generative augmentation shows improved results on urdu, while have a negative impact on the other three regional languages. few-shot learning with causal models reveal their current limitations for low-resource languages when used for data augmentation and ner. overall, the research emphasizes the potential of hybrid data augmentation techniques to enhance ner performance for low-resource languages.