Frustratingly Easy Label Projection for Cross-lingual Transfer (2023)

Abstract: Translating training data into many languages has emerged as a practical solution for improving cross-lingual transfer. For tasks that involve span-level annotations, such as information extraction or question answering, an additional label projection step is required to map annotated spans onto the translated texts. Recently, a few efforts have utilized a simple mark-then-translate method to jointly perform translation and projection by inserting special markers around the labeled spans in the original sentence. However, as far as we are aware, no empirical analysis has been conducted on how this approach compares to traditional annotation projection based on word alignment. In this paper, we present an extensive empirical study across 57 languages and three tasks (QA, NER, and Event Extraction) to evaluate the effectiveness and limitations of both methods, filling an important gap in the literature. Experimental results show that our optimized version of mark-then-translate, which we call EasyProject, is easily applied to many languages and works surprisingly well, outperforming the more complex word alignment-based methods. We analyze several key factors that affect the end-task performance, and show EasyProject works well because it can accurately preserve label span boundaries after translation. We will publicly release all our code and data.

## introduction zero-shot cross-lingual transfer, where models trained on a source language (e.g., english) are directly applied to other target languages, has the potential to extend nlp systems to many languages @xcite @xcite @xcite ). yet, its performance still lags behind models that are directly fine-tuned on labeled data (if available) from the target language. recent work has shown that combining training data in a source language together with its automatic translation to the target language leads to consistent performance improvements @xcite . however, for nlp tasks that involve span-level annotations, an additional label projection step is needed to map the span annotations onto the translated texts (see figure 1 ). traditionally, this annotation projection step is performed based on word alignment after machine translation @xcite . to avoid the use of complex word alignment models, several recent efforts @xcite ) directly translated sentences with span annotations wrapped between special markers (e.g., <@xmath0. however, due to limited analysis presented in prior work, it is unclear (1) how well this approach works across different language families, (2) how robust mt systems are in handling special markers, as inserting markers inevitably degrades the translation quality, and (3) how well marker-based projection works in comparison to traditional alignment-based methods. in this paper, we present the first systematic study of the mark-then-translate annotation projection technique, which includes careful evaluation of the choice of markers, projection accuracy, impact on translation quality, robustness to different mt systems, as well as a comparison to traditional alignment-based method across 57 languages (including 18 from africa) on 5 datasets and 3 nlp tasks. we also propose an improved variant of marker-based projection, easypro-ject, that consistently outperforms the alignmentbased approach, while being incredibly easy to use to project a variety of annotations (qa, entities, pipeline method based on word alignment: starting with machine translation of the english sentence to chinese, followed by word-to-word alignment. then, labeled spans are projected using heuristics. (b) mark-then-translate: markers are inserted around the annotated spans in the text. the modified sentence with markers inserted is then fed as input to an mt system, projecting the label span markers to the target sentence as a byproduct of translation. our main findings include (1) the marker-based method is surprisingly robust across different translation systems and languages, but the choice of markers matters ( §3.2); (2) easyproject can project annotated spans more accurately and is better at preserving span boundaries than the alignmentbased methods, which is key to its success ( §5.1); ## analysis of marker-based projection the idea of marker-based label projection is straightforward -wrap labeled spans with special marker tokens, then translate the modified sentence (see an example in figure 1b ). the projected spans can be directly decoded from the translation if the markers are retrained. however, inserting markers inevitably degrades the translation quality. in this section, we analyze several questions left open by prior work @xcite , including (1) how well are the special markers being preserved in translation, (2) the impact of different marker choices on translation quality and the performance of cross-lingual transfer. ## easyproject based on our analysis, we develop an optimized version of the mark-then-translate method, which we call easyproject. 5 our improvements target the two weaknesses of the marker-based approach: ## conclusion in this paper, we presnet a thorough empirical assessment of various approaches for cross-lingual label projection. we also design an improved variant of the mark-and-translate method, which we call easyproject. experiments on 57 target languages and three well-studied nlp tasks show that easyproject consistently outperforms the alignment-based methods and effectively improves the performance of cross-lingual transfer.