Cross-Lingual Named Entity Recognition via F ast A lign: a Case Study (2021)

Abstract: Named Entity Recognition is an essential task in natural language processing to detect entities and classify them into predetermined categories. An entity is a meaningful word, or phrase that refers to proper nouns. Named Entities play an important role in different NLP tasks such as Information Extraction, Question Answering and Machine Translation. In Machine Translation, named entities often cause translation failures regardless of local context, affecting the output quality of translation. Annotating named entities is a time-consuming and expensive process especially for low-resource languages. One solution for this problem is to use word alignment methods in bilingual parallel corpora in which just one side has been annotated. The goal is to extract named entities in the target language by using the annotated corpus of the source language. In this paper, we compare the performance of two alignment methods, Grow-diag-final-and and Intersect Symmetrisation heuristics, to exploit the annotation projection of English-Brazilian Portuguese bilingual corpus to detect named entities in Brazilian Portuguese. A NER model that is trained on annotated data extracted from the alignment methods, is used to evaluate the performance of aligners. Experimental results show the Intersect Symmetrisation is able to achieve superior performance scores compared to the Grow-diag-final-and heuristic in Brazilian Portuguese.

## introduction word alignment is a natural language processing (nlp) task that can be applied to a parallel text corpus to find the word-to-word correspondences in a sentence pair. it can be used in the machine translation (mt) pipeline to analyse the output of mt or to improve the quality of translation memory @xcite . although word alignment is not a necessary step in the mt pipeline, in special cases it helps the model to improve the translations. for example, word alignment is useful to translate domain-specific terminology and low-frequency content words @xcite . word alignment can also be used to match the alignments with the source annotations to determine the projection on the target text. it helps the named entity recognition (ner) system to detect named entities (nes) on the target text based on the annotation information of the source text. ner is a classification task that tries to identify and tag nes in a given sentence. the benefits of performing ner during translation tasks range from improved translation quality to customer data protection. the performance of a machine learning-based ner system depends on multiple factors such as the amount of labelled data used to train the model. the availability of labelled data is one of the key points for training the ner systems. it is a basic challenge, especially for low-resource languages. one of the possible solutions for this challenge is to use cross-lingual approaches. these approaches use parallel bilingual corpus to extract the annotation information from languages with rich labelled data. there are two different approaches to extract nes from bilingual corpora: direct transfer approach, and annotation projection approach. approaches based on the direct transfer try to use language-independent features to train the model on the source side and then directly apply it on the target side. different crosslingual features such as word-embeddings @xcite , word clusters @xcite and wikifier @xcite can be used in the training process. these approaches suffer from sense ambiguity and word order differences that lead to noise in the output of the model @xcite . methods based on the annotation projection use word alignment information to project annotations from the source language to the target side in bilingual parallel corpora. different approaches can be used to extract the alignments between sequences of words in the source and target languages. the most common approaches of word alignments are based on the ibm approach that is a classic word alignment model @xcite . in the ibm models, every word in the source language can be aligned at most with one word in the target language that is called many-to-one mapping. but real-word alignment approaches that are based on the ibm model support different types of mappings such as many-to-many. the simplest approach to produce a many-to-many mapping is the symmetrization heuristic @xcite . the symmetrization heuristic uses alignment in both directions. there are different methods to implement a symmetrization heuristic. intersection and union alignments are the most popular methods that merge alignments of both directions. the intersection of two models expresses just a one-to-one relationship between words and it misses some of the alignments. so, it has a higher precision of alignment points but at the cost of losing in recall. union of two models can capture all complementary information of both models. unlike the intersection heuristic, union has a higher recall but lower precision. there are some extended versions for intersection to improve its performance such as the gdfa heuristic. it grows the intersection heuristic by adding neighbouring alignment points from the union and unaligned points to the intersection @xcite . the gdfa heuristic includes three steps. the first step is grow-diag that intersects two-directional alignments and gradually considers the neighbourhood of each alignment point between the source and target languages. the second step is (-final) covers non-neighbour alignment points of intersection alignment points. the final step (-and) adds alignment points between two unaligned words. this paper is organized as follows. section 2 introduces ner and alignment methods. section 3 describes experimental settings and results. finally, section 4 presents our conclusions and future work. ## methods word alignment provides useful information for several applications of nlp. mt is one of these applications that can use the benefits of word alignment methods to improve translations. although the neural mt models (nmt) do not rely on the word alignment approaches, they still play an important role to improve the output quality of the model. they can be used to extract an external lexicon and apply it in the inference process of mt @xcite . it can help the mt model to better use domain-specific terminology to adapt the model with a new domain or to improve the translations of out-of-vocabulary content words. ner is another application to use word alignments in mt to improve its performance. ner is an information extraction task to automatically detect nes in text and classifying them into predefined entity types such as person, organization, location, time, date, etc. today's ner systems are based on supervised machine learning models including maximum entropy markov models (memms) @xcite , conditional random fields (crfs) @xcite , and neural networks @xcite . although the ner model that is based on neural architectures show high performance, they need a large amount of manually annotated ner data that is not available for low-resource languages @xcite . in addition, the process of annotating by a human is a time-and money-consuming task. cross-lingual ner is an effective solution to tackle these challenges. it means transferring annotated information from a high-resource language that has enough annotated resources to a low-resource language with less or no annotated data. there are two main groups of cross-lingual nlp, direct transfer and annotation projection @xcite . in direct transfer approaches, the ner model is trained on a language with rich labelled data and then applied to a text in a different language to detect nes @xcite . these approaches attempt to use language-independent features. but some features are dependent on the type of language. so, selecting a suitable set of features plays an important role in the quality of these approaches. annotation projection is another approach that is based on a parallel corpus between source and target languages @xcite . these methods attempt to annotate the target side by using the annotation information of the source language. the quality of the annotation task is related to the quality of labelled data on the source side, the quality of alignments, and the size of the parallel data. this paper is a part of our research on cross-lingual ner transfer with minimal resources in the pipeline of mt. we focus on evaluating the performance of alignment methods in the annotation projection approach, where there is only one source language with rich label and no labelled data in the target language. in this paper, the accuracy of two alignment heuristics, grow-diag-final-and (gdfa) and intersect symmetrisation are evaluated on the english-brazilian portuguese parallel corpus. ## experiments in this paper, the ner model is based on the flair foot_0 framework to train it on the training dataset and extract ner annotations from the test dataset. before training the ner model, we used aligner methods to extract the annotation projection for the target language, brazilian portuguese, because we want to figure out ways of taking annotation benefits of the source language, english, that already is available. ## conclusion & future work during this study, we focused on ner as a crucial task in the machine translation pipeline. the availability of labelled data for training the model is a main challenge of the ner systems. the focus of this project was to address this problem by using the aligner algorithms. the aligners can extract annotations in a target side (low-resource language) using annotation projection from the source side (high-resources language). the experiment shows the ner model that was trained on the intersect heuristic has a better performance than gdfa. it seems that the performance of the aligners directly impacts on the performance of the ner model. using state-of-art approaches for the alignment part can be a potential plan for future projects. in this project, restrictions on access to parallel bilingual dataset in english-brazilian portuguese by the source side's annotations impacts on the performance of the ner model as well as aligners. this project can be considered as a starting point for our study on the aligner approaches for annotation projection in low-resource languages.