Named Entity Annotation Projection Applied to Classical Languages (2023)

Abstract: In this study, we demonstrate how to apply cross-lingual annotation projection to transfer named-entity annotations to classical languages for which limited or no resources and annotated texts are available, aiming to enrich their NER training datasets and train a model to perform NER tagging. Our method uses sentence-level aligned parallel corpora ancient texts and the translation in a modern language, for which high-quality off-the-shelf NER systems are available. We automatically annotate the text of the modern language and employ a state-of-the-art neural word alignment system to find translation equivalents. Finally, we transfer the annotations to the corresponding tokens in the ancient texts using a direct projection heuristic. We applied our method to ancient Greek, Latin, and Arabic using the Bible with the English translation as a parallel corpus. We used the resulting annotations to enhance the performance of an existing NER model for ancient Greek

## introduction named entity recognition (ner), like other nlp tasks, has benefited from the advances in language modeling and the availability of large annotated corpora. numerous high-quality ner models are available for modern languages. however, classical and ancient languages lack adequate annotated data and language models essential for training ner models. therefore, annotation projection can be employed over parallel text corpora to overcome this problem and transfer the annotation from modern languages for which accurate off-the-shelf ner systems are available. the core concept of annotation projection is to perform automatic or manual linguistics annotation on a text and project the annotation to its translation using mapping heuristics that link the entities with their correspondences. translation alignment has been used for this purpose to transfer various linguistic annotations, such as semantic role labels @xcite , part-of-speech @xcite @xcite , named entities tags @xcite @xcite , relations and arguments @xcite @xcite , semantic parsing @xcite , syntactic and dependency parsing @xcite @xcite . recently, neural translation alignment models were able to produce accurate alignments for a variety of modern and classical languages, even with no or a small amount of training data profiting from contextualized multilingual language models. in this paper, we present a processing pipeline to transfer ne annotations from a text in modern languages to parallel texts in classical or lowresourced languages. we use accurate ner models for modern languages and employ state-of-the-art neural alignment models at the word level to find the translation equivalents. further, we propose a direct projection heuristic that maps the annotations from source to target tokens considering various alignment types. we used the obtained entities to improve the accuracy of existing ner models for ancient greek. the proposed approach works for any language pair provided the parallel corpora are available and aligned at the sentence or paragraph level. ## related work cross-lingual annotation projection of named entities in a parallel corpus has two main scenarios: the first scenario incorporates machine translation to translate the detected named entities of the source text and tries to look up the translated entities in the corresponding parallel sentence using various matching heuristics based on orthographic and phonetic similarity and edit distance text similarity @xcite . the second scenario employs automatic word alignment to find the translation equivalents of the detected entities in the parallel sentence and project the annotation @xcite . on ner in digital classics: the classical language toolkit (cltk) is the largest python library to perform nlp tasks on ancient languages, including ner @xcite . however, the lack of adequately annotated datasets for most classical languages is a fundamental hindrance to the high performance of this task. other efforts have been made, starting from large annotated datasets of specific sources, using semantic annotation platforms and machine learning @xcite 3 methodology figure 1 illustrates the proposed pipeline, it consists of three main components. we start with collecting and preparing a parallel corpus of ancient languages and at least one modern language, such as english, for which a high-quality off-theshelf ner annotation tool is available. the corpus must be aligned at the sentences or paragraph level. then we use a ner tool such as spacy foot_0 , allennlp foot_1 @xcite , or flairnlp foot_2 @xcite to annotate the text of the modern language. in parallel, we use a state-of-the-art automatic alignment model to align the parallel sentences and extract the translation equivalents. an unsupervised fine-tuning using the parallel corpus can be employed using different training objectives to improve the word alignment accuracy. subsequently, we find the corresponding translations of the detected named entities and project the annotation using a direct mapping heuristic. in our experiment, we used the bible in ancient greek, latin, and english. ## results we employed the projection approach to the 7950 verses of the new testament and resulted in 6,567 ancient greek entities (6,104 single-token and 463 multi-token entities) and 6481 latin entities (5940 single-token an 541 multi-token). we performed qualitative evaluation to estimate the quality of the produced annotations on two language pairs: english-ancient greek and english-latin. two domain experts manually assessed 100 random verses, which corresponded to about 550 extracted entities per language, and assigned a score to each detected entities. the automatic ner annotation of english text achieved over 94% accuracy and the entities alignment on ancient greek-english achieved the highest accuracy (91.9%), since the alignment model is optimized for this language pair. however, the entities classification errors were common for personal names classified as locations and vice versa. in some cases, a greek or latin noun would be misclassified as a consequence of the english translation, which adopted a different type of entity: many ethnonyms, which would be classified as misc in our dataset, were translated in english as location names, and therefore classified as loc. additionally, incomplete or partial alignments were frequent in the dataset (9 cases in ancient greek, 28 in latin)especially in multi-token entities such as "jesus christ", "simon zelotes", or "pontius pilate". further, we used the resulted annotations to extend the available ner training dataset for ancient greek foot_6 and fine-tune the existing ancient greek ner models proposed by @xcite 9 . the obtained model achieved a higher f1, and a better performance on multi-token entities as reported in ## conclusion in this experiment, we used translation alignment to project ner annotations from texts in modern languages to texts in ancient languages in order to create ner datasets for such languages, enrich the available datasets, or annotate texts where the existing ner models fail to create accurate annotations. the proposed approach can be employed to any parallel corpus, not only the bible. however, many factors might affect the annotation performance, such as the translation quality, text genre, and performance of the ner tool of the modern language used in the parallel corpus. also, the proposed method can be applied to low-resourced modern languages to enrich the annotated ner training dataset. the automatic alignment accuracy varies between language pairs; it is not surprising that the english-ancient greek alignments are more accurate than the english-latin since the language model used in the experiment is mainly fine-tuned on ancient greek texts. this experiment is a proof of concept, and due to limited computational resources, we used a subset of the bible corpus (new testament only). using the entire corpus with other parallel corpora will result in more named entities and accurate ner models.