On Cross-Language Entity Label Projection and Recognition
Authors: Paolo Gajo, Alberto Barrón-Cedeño (Università di Bologna, 2024)
Topic: Machine Translation

Abstract
Most work on named entity recognition (NER) focuses solely on %named entities using English. %models and corpora. Through the use of training data augmentation via machine translation (MT), multilingual NER can become a powerful tool for information extraction in multilingual contexts. In this paper, we augment NER data from culinary recipe ingredient lists by means of MT and word alignment (WA), following two approaches: \Ni translating each entity separately, while taking into account the full context of the list and \Nii translating the whole list of ingredients and then aligning entities using three types of WA models: Giza++, Fast Align, and BERT, fine-tuned using a novel entity-shuffling approach. We depart from English data and produce Italian versions via MT, span-annotated with the entities projected from English. Then, we use the %synthetic data produced by the two approaches to train mono- and multilingual NER BERT models. We test the performance of the WA and NER models on an annotated dataset of ingredient lists, %obtained from Italian partially out-of-domain compared to the training data. The results show that shuffling entities leads to better BERT aligner models. The higher quality NER data created by these models %in turn enables NER models to achieve better results, with multilingual models reaching performances equal to or greater than their monolingual counterparts.

Introduction
Named entity recognition (NER) is a sequence labeling task with a long history of works mainly focusing on the recognition of %named entities such as people, locations, and organizations. Multilingual NER has also attracted research efforts, with %the two most recent SemEval campaigns including tasks on multilingual complex NER (MultiCoNER) AUTHOR. Despite its popularity %of the task and various mono- and multilingual NER datasets being available, specific domains such as the culinary one likely require new annotated data. In addition, NER is often the first step in information extraction for knowledge graph construction and, to the best of our knowledge, all literature in the domain of cuisine on this topic solely focuses on English data AUTHOR. Therefore we argue that, given cuisine's multicultural nature, more research in this direction is warranted. Entity label projection AUTHOR aims to address this scarcity by automating the data generation process for NER. This task consists in taking the labels associated with spans from a source text and automatically applying them to its translation in another language, i.e. the target text. Through this task, we attempt to find an efficient automatic way of developing models for entity projection across languages to produce high-quality multilingual data for recipe Named Entities (r-NE) AUTHOR. Departing from an English-language dataset containing ingredients from culinary recipes, annotated at the span level with entity category labels, we first rely on a MT engine to translate each source entity s_i individually into Italian, while keeping the full context into account. This results in a first entity-wise (EW) translated EN--IT--ES dataset where entities are linked across languages.% . Using these synthetic alignments, we train BERT models to align source and target entities, shuffling the latter %in order to prevent the model from learning to simply predict the original entity order. We then test the models on two novel entity alignment datasets, partially out-of-domain compared to the training data, e.g., as regards the used food products, units of measure, and cooking processes. As baselines to evaluate the BERT alignment models, we use Giza++ AUTHOR and Fast Align AUTHOR, two statistical word alignment (WA) models. In order to produce higher-quality r-NE data, we %then machine translate %MT the ingredient lists across their whole length, predicting target entity spans %by using with the best BERT models %obtained from the previous step, along with the baseline models. We thus obtain various sentence-wise (SW) translated datasets in Italian, trading some alignment accuracy for better translations. Both types of training data, EW and SW, are then used to fine-tune mono- and multilingual BERT NER models on the task of recognizing entities in food recipes. The %NER models are trained on various combinations of mono- and multilingual data and are tested on the entity annotations from the two aforementioned novel testing datasets. Our contribution is %thus three-fold: % \item \Ni We show the efficacy of fine-tuning alignment models by shuffling entities in contexts where most of the information depends on the presence of lexical items rather than the dependencies linking them. % \item \Nii We %demonstrate showcase the performance delta between mono- and multilingual NER models when fine-tuning %them on the synthetic data produced by our alignment. %approach. These %multilingual models can be used to %automatically label large datasets in multiple languages at a finer granularity level compared to currently available monolingual resources. % \item \Niii We release %all code and data %hereby presented, to produce %more training data in multiple languages.% The rest of the paper is structured as follows. Section~sec:related-word presents relevant past research on the subjects of cross-lingual entity alignment and recognition. Section~sec:data introduces the datasets and corpora used in the experiments, along with their annotation process. Section~sec:models presents architecture, training, and evaluation details for the models comprising our pipeline. Section~sec:experiments-results discusses the conducted experiments and their results. Finally, Section~sec:conclusions summarizes the paper and draws conclusions. Appendix~app:es shows further results including Spanish. Appendix~app:xlwa presents statistics and gives insight on the additional training data used. Appendix~app:comp-res lists information on the computational requirements.

Conclusion
We explored a simple novel technique to automatically generate high-quality multilingual NER data by combining machine translation and cross-language entity linking. For our experiments, we relied on the English-language \tasteset{} %a dataset, which includes %of recipes whose %containing lists of ingredients are span-annotated for entity recognition. Moreover, we manually curated a novel English--Italian cross-language dataset, %\pg{in English and Italian}, featuring the same kind of %entity annotation, with the addition of cross-language %entity alignments. We machine translated the entities %contained in \tasteset's recipes individually and shuffled them within ingredient boundaries. Leveraging this augmented data, we then fine-tuned BERT entity-alignment models. Using statistical word-alignment models as baselines, we tested these BERT models on our English--Italian parallel corpus. % manually curated by us, containing entity-aligned parallel instances. The results %of this experiment showed that models fine-tuned using our novel approach consistently outperform those trained on unshuffled data, along with %\pg{the two statistical baselines. We then created additional synthetic data by first translating \tasteset's recipes in their entirety, and then aligning the entities in the machine-translated target text using the best models obtained from the first part of the study. These data allowed us to obtain better NER models, compared to the ones we would have obtained by using the original recipes translated entity by entity. We tested monolingual English and Italian BERT models against \mbert, and showed that the latter is capable of obtaining the same performance as its monolingual counterparts when tested on monolingual NER data. In future work, we plan to extend the annotation of our datasets, both in terms of number of instances and annotators. We will also prioritize solving the token-to-character conversion issues encountered in this study. Furthermore, we plan to leverage this data augmentation technique in order to improve multilingual text-to-graph models, since all of the literature in this regard focuses on English-only data AUTHOR. \appendix