Using a multilingual literary parallel corpus to train NMT systems (2024)

Abstract: This article presents an application of a multilingual and multidirectional parallel corpus composed of literary texts in five Romance languages (Spanish, French, Italian, Portuguese, Romanian) and a Slavic language (Croatian), with a total of 142,000 segments and 15.7 million words. After combining it with very large freely available parallel corpora, this resource is used to train NMT systems tailored to literature. A total of five NMT systems have been trained: Spanish-French, Spanish-Italian, Spanish-Portuguese, Spanish-Romanian and Spanish-Croatian. The trained systems were evaluated using automatic metrics (BLEU, chrF2 and TER) and a comparison with a rule-based MT system (Apertium) and a neural system (Google Translate) is presented. As a main conclusion, we can highlight that the use of this literary corpus has been very productive, as the majority of the trained systems achieve comparable, and in some cases even better, values of the automatic quality metrics than a widely used commercial NMT system.

## introduction parallel multilingual corpora have a wide use and are known for their application in different kinds of linguistic research (contrastive linguistics, translation studies, phraseology, lexicography, etc.) @xcite , translation training (lópez @xcite and training of machine translation systems @xcite , as well as terminology extraction @xcite . the parallel corpus romcro @xcite was created taking into account all these possible applications. this project started in autumn 2019 and it is financed by the faculty of humanities and social sciences of the university of zagreb. romcro is a multilingual and multidirectional parallel corpus, which is aligned and annotated with msd (morpho-syntactic description) tags. it is composed of original literary texts written in five romance languages (spanish, french, italian, portuguese, romanian) as well as croatian, and their respective published translations into the other five languages. even though lemmatization and annotation are not relevant for the task at hand, they were completed in order to allow for different uses of the corpus, such as extracting desired structures and their translations for contrastive analysis or translator training. most previous studies about machine translation (mt) of literary texts are quite recent (from 2012 onwards). according to @xcite , a key challenge in literary translation is preserving not only the meaning, but also the reading experience. this is a key difference to other domains, for example, technical or legal texts. @xcite evaluate a customized mt system tailored for a literary translator specializing in fiction. the study demonstrates that fine-tuning a base model with a smaller subset of custom training data can yield translations closer to human references, despite the raw output still falling short of human quality. other studies @xcite also suggest the idea of training author-tailored nmt systems for literary texts. most of the studies remark the idea that translation technologies are still not mature enough for translation of literary works ready to be published and that human translators are needed for this task. bur nmt systems for literary works can still have several interesting uses, as for example (1) produce draft translation for editorial teams to decide whether to publish a translation of a novel in a given market, promoting the cultural interchange and the visibility of authors writing in smaller languages; (2) produce bilingual electronic books where users can access the translation of difficult sentences or paragraphs, promoting reading in the original language, among others. we will dedicate the first part of this paper to describe the corpus romcro, in order to understand its characteristics as well as the process of its creation, and then proceed to explore its application in neural machine translation (nmt). ## building romcro the corpus contains 27 original titles: seven in spanish, six in french, four in italian, four in romanian, three in portuguese and three in croatian, as it is shown in there were six stages in building the corpus: 1) selection and collection of texts, 2) digitization of texts, 3) preparation for segmentation and sentence alignment, 4) segmentation, alignment and manual correction, 5) lemmatization and morphosyntactic annotation (with msd tags), and 6) access to the corpus. one of the main challenges was to find high quality material translated from the original language into the rest of the languages, which is why literary texts were chosen. the uneven divide between the number of originals in each language ( once the selection of texts was completed, digitization of those not available in digital format was initiated. they were scanned and then an optical character recognition using abbyy finereader was performed. in the next stage, the material was prepared for segmentation and alignment by manually correcting texts in ms word. several undergraduate and master level students collaborated on the project, reviewing and correcting the results of this digitization, that is, preparing the texts for automatic alignment. the segmentation and alignment was performed using lf aligner, foot_3 a freely available tool based on hunalign @xcite . the results were again revised and corrected manually. the lemmatization and morphosyntactic annotation was done using the annotators available via sketch engine, which were freeling @xcite for spanish, french, italian and portuguese, and multext-east @xcite for romanian and croatian. a lemmatized and pos tagged version of the corpus containing 159 texts is available on sketch engine. for direct access to the untagged tmx and tsv versions used in this experiment (com-n. lang. prising 157 texts) under the cc-by-nc-4.0 license, please refer to the elrc (european language resource coordination) platform. 5 in both formats, the order of languages is spanish (es), french (fr), italian (it), portuguese (pt), romanian (ro), croatian (hr). all the versions of the corpus contain notes about the original language, writer, and the original title of the text the segment is from. additionally, segment order was scrambled to protect copyright. ## comparison to similar corpora many parallel corpora are freely available on the internet. the main collection can be found in opus corpora @xcite . however, only a minority of these parallel corpora are created from literary texts, and when available, they do not con-5 https://elrc-share.eu/repository/search/ ?@xmath0, or are created from individual or a small number of works (as salome foot_5 ). to the best of our knowledge, this is the first multilingual and almost completely multidirectional parallel corpus that aligns literary texts in several romance languages and one slavic. in other similar corpora, the slavic language was the pivot language @xcite @xcite , which means that all texts were translated from or to the slavic language, but not necessarily between each other. croatian is present in some multilingual literary corpora, such as translitex @xcite or intercorp @xcite , which also includes literary texts. however, translitex contains translations of a single book into 23 languages and in-tercorp is made up of 40 languages, including all those that form part of romcro, but has czech as the pivot language. as an example of use of romcro, we explore the training of neural machine translation systems tailored to literature. a total of five nmt systems have been trained, combining the spanish subcorpus (original and translated material) with subcorpora in the other five languages. in other words, the trained nmt systems were spanish to french, italian, portuguese, romanian and croatian. in this section all the processes performed to train and evaluate these systems are described.