Sakura at S em E val-2023 Task 2: Data Augmentation via Translation (2023)

Abstract: We demonstrate a simple yet effective approach to augmenting training data for multilingual named entity recognition using translations. The named entity spans from the original sentences are transferred to translations via word alignment and then filtered with the baseline recognizer. The proposed approach outperforms the baseline XLM-Roberta on the multilingual dataset.

## introduction named entity recognition (ner) is an essential task in natural language processing (nlp) that involves identifying and classifying entities such as people, organizations, locations, and more recently, complex entities such as events, creative works, diseases. multilingual complex named entity recognition (multiconer) shared task @xcite is organized to promote research in multilingual ner for complex entities. multi-coner ii @xcite features one multilingual track and twelve monolingual tracks for the following languages: english, spanish, hindi, bangla, chinese, swedish, farsi, french, italian, portuguese, ukrainian, and german. the datasets contain fine-grained tagsets that mainly cover three domains (wiki sentences, questions, and search queries) and include six top-level tags: location (loc), creative work (cw), group (grp), person (per), product (prod), and medical (med). complex named entities are highly ambiguous and can take the form of any linguistic constituency: "sir" (creative-work/visual-work), "catch me if you can" (creative-work/visual-work), "jeremy corbyn" (person/politician). thereby, making a granular distinction between entity classes (e.g., politician vs. scientist) and identifying rare entities solely from the context are challenging problems. to alleviate this challenge, we propose to augment multilingual training data with translations, which we believe can help the ner model to disambiguate more entities by learning from diverse and noisy contexts. the proposed augmentation strategy involves three steps: translating training example, transferring entity annotations from the source sentences to the target, and filtering out low-quality annotations. to produce translations, we use publicly-available mbart-50 @xcite . to transfer entity annotations, we compute the token alignment between the source and its translation using simalign @xcite and then build an entity alignments on top of the token alignments. to ensure high-quality annotations, we use xlm-roberta (xlm-r) @xcite finetuned on the original training data to filter out low-quality annotations from the translated data. we find that xlm-r fine-tuned on the augmented dataset outperforms the same model fine-tuned on the original training dataset. ## related work ner is one of the most popular nlp task and is widely explored across domains: news @xcite , biomedical @xcite , social media @xcite . with the recent advances in representation learning, models such as xlm-r @xcite and bert @xcite achieve the state-of-the-art ner performance with minimal feature-engineering efforts. in this work, we use xlm-r as our base model for fine-tuning on the ner task. data augmentation via translation has been widely used. an example of this is backtranslation @xcite , a methodology of obtaining parallel data by translating sentences from the target language to the source language. although such data augmentation approaches may introduce noise, increasing the lexical diversity has been effective @xcite @xcite . translation have been successfully applied to cross-lingual ner problem under unsupervised transfer settings with limited resources @xcite . schäfer et al. (2022) use machine translation and word alignment techniques to transfer inferred named entity annotations from a rich resource language to a low resource language. we draw inspiration from these works and employ the same approach to augmenting the training data for ner, in addition we assume that we have enough data in the target language to train a filtering model. ## data augmentation via translation we employ mbart-50 1 @xcite to generate translations for every sentence in the original dataset @xcite . mbart-50 is trained on 50 languages including the 12 languages of the shared task. we pivot translations via english, as we generally observe that pivoting produces better translations than the direct approach, e.g., @xmath0. @xmath1. the translations are generated from every source language to every target language except english, as our focus is on languages with less resources. after generating the translations we align the tokens using simalign @xcite . through the token alignment we transfer the entity annotations from the source sentences to their translations. although the word order varies in each language, name entities comprise mostly indivisible spans in different languages, which makes transfer generally feasible. to align the entity boundaries, we collect all the target tokens aligned to any of the tokens in the source entity and confirm the transfer only if the target tokens form a continuous span in the target sentence (see figure 1 ). although mbart-50 supports all the task lan-1 https://github.com/facebookresearch/fairseq/ tree/main/examples/multilingual#mbart50-models guages, the translation quality is not guaranteed especially for the low-resource languages, besides the alignment quality also contributes to the compounding error. thus, to ensure the quality of the augmented dataset, we first retain the translations that only have matching number of entities with the sources. then, we apply the baseline ner model fine-tunted on the original dataset to the translated data and retain the sentences that agree with baseline on the number of entities and their boundaries. ## named entity recongition we formulate named entity recognition (ner) as a sequential labelling task, where the goal is to assign each token in a given text a label that corresponds to its entity type. for example, in the sentence "john smith works at new york", the label "b-per" would be assigned to "john", the label "i-per" would be assigned to "smith", the label "b-loc" would be assigned to "new", and the label "i-loc" would be assigned to "york". the sequential labeling is a well-studied tasks and enables us to easily leverage standard model architectures. we use xlm-roberta @xcite as our baseline model. as the premise of our exploration is the data augmentation strategy only, we make no modifications to the xlm-r architecture and use it as-is for fine-tuning on the data variants. ## conclusion in this work we have shown that data augmentation via translation is an useful approach to improve the performance of ner models. on the multilingual test set we observe an improvement over the baseline xlm-r by 0.85 f1. our approach uses open-source components and can potentially benefit ner in various domains. in the future we plan to explore how different translation configurations can boost ner models. reducing the noise of augmented sentences should have a positive impact on the performance. this can be achieved by improving the performance of the translation model. alternatively, the dataset could be augmented with translations from several models. therefore, using a selection of sentences from different sources @xcite to augment the data could also be investigated.