Zero-Shot Cross-Lingual NER Using Phonemic Representations for Low-Resource Languages (2024)

Abstract: Existing zero-shot cross-lingual NER approaches require substantial prior knowledge of the target language, which is impractical for low-resource languages.In this paper, we propose a novel approach to NER using phonemic representation based on the International Phonetic Alphabet (IPA) to bridge the gap between representations of different languages.Our experiments show that our method significantly outperforms baseline models in extremely low-resource languages, with the highest average F1 score (46.38%) and lowest standard deviation (12.67), particularly demonstrating its robustness with non-Latin scripts. Ourcodes are available at https://github.com/Gabriel819/zeroshot_ner.git

## introduction named entity recognition (ner) plays a crucial role in many natural language processing (nlp) tasks. achieving high performance in ner generally requires extensive resources for both sequence labeling and gazetteer training @xcite . however, access to training resources for many low-resource languages (lrls) is very limited, motivating zero-shot approaches to the task. while various strategies have been explored to enhance zero-shot ner performance across languages, they required either parallel data or unlabeled corpora in the target language, which is difficult and sometimes impossible to obtain. our work tackles zero-shot ner under a strict condition that disallows any target language training data. we decided to approach this condition by projecting data into an international phonetic alphabet (ipa) space. since different languages often share similar pronunciations for the same entities, such as geopolitical entities and personal names (e.g., the word for china is / > tsajn@/ in english and * these authors contributed equally to this work. / > tsina/ in sinhala), the model trained on one language can be transferred to others without targetlanguage training in ner. as shown in figure 1 , we first convert orthographic scripts into ipa, and then fine-tune a pre-trained model on the phonemes of the source language, i.e., english. by using a shared notation system-ipa-we can apply the model to target languages directly. our findings show that fine-tuning phoneme-based models outperforms traditional grapheme-based models (e.g., mbert @xcite ) by a large margin for lrls not seen during pre-training. furthermore, our approach demonstrates robustness with non-latin scripts, exhibiting stable performance across languages with different writing systems. ## experiments here we provide information about the datasets and models we employed for the experiments. more implementation details including hyperparameters are provided in appendix a. ## conclusion this paper presents the novel method of employing phonemes for identifying named entities for lowresource languages in zero-shot environments. our experiments compared the results of phoneme-based models with grapheme-based models in a strict zero-shot setting, and have shown that phonemes exhibit the best performance over low-resource languages unseen by all models. the results particularly demonstrate robustness towards non-latin scripts, which is crucial in context of multilingual ner since languages are written in diverse writing systems.