System Description of the N ordics A lps Submission to the A mericas NLP 2024 Machine Translation Shared Task (2024)

Abstract: This paper presents the system description of the NordicsAlps team for the AmericasNLP 2024 Machine Translation Shared Task 1. We investigate the effect of tokenization on translation quality by exploring two different tokenization schemes: byte-level and redundancy-driven tokenization. We submitted three runs per language pair. The redundancy-driven tokenization ranked first among all submissions, scoring the highest average chrF2++, chrF, and BLEU metrics (averaged across all languages). These findings demonstrate the importance of carefully tailoring the tokenization strategies of machine translation systems, particularly in resource-constrained scenarios.

## introduction the participation of the nordicsalps team in the americasnlp 2024 machine translation shared task builds directly on the previous contributions by the helsinki team. the main goal of the shared task, as in the previous editions, is to build machine translation (mt) systems capable of translating spanish into eleven american languages. with limited training data, the mt solutions need to leverage cross-lingual transfer and data-efficient approaches to achieve a good level of performance on the translation tasks. previous contributions of the helsinki team performed cross-lingual transfer by pre-training a spanish-english model, and transferring the knowledge learned to the language pairs of the task, i.e., spanish-target (any of the eleven indigenous target languages), by continued training. the previous helsinki submissions primarily focused on increasing the data size by collecting additional sources and applying data augmentation techniques, but data efficiency was not directly addressed. our submission builds on the previous findings and focuses on the data efficiency aspect of the challenge. ## data following de gibert et al. (2023), we train multilingual one-to-many models that translate from spanish to the eleven indigenous target languages and and include english as an additional high-resource target language. spanish-english data we use spanish-english parallel data from a subset of the sources mentioned in de gibert et al. (2023): europarl, glob-alvoices, newscommentary, tedtalks, and tatoeba collected from opus @xcite . in contrast to de gibert et al. (2023) and due to time constraints, we do not include bibles nor opensubtitles. validation data for pre-training comes from the spanish-english wmt-news corpus. the spanish-english parallel data underwent cleaning with opusfilter @xcite , as described in de gibert et al. @xcite . namely, this consisted in deduplication and a set of filters based on length difference, script identification and language identification. spanish-indigenous language data our models include all eleven indigenous american languages for which data was provided in the shared task: asháninka (cni), aymara (aym), bribri (bzd), chatino (ctp), guarani (gn), hñähñu (oto), nahuatl (nah), quechua (quy), raramuri (tar), shipibo-konibo (shp), and wixarika (hch). we used all spanish-indigenous language training and development data provided by the shared task organizers @xcite @xcite @xcite @xcite . whenever available, we also included the extra and synthetic datasets provided by the shared task organizers @xcite . the data used for this year's submissions differ from those described in de @xcite in two crucial aspects. first, we did not include bible data, since bibles did not improve translation quality in earlier editions @xcite and were not part of the organizer-provided datasets. second, due to time constraints, we did not apply any filtering or cleaning to the parallel data. no preprocessing has been applied to the bytelevel models. some general preprocessing was carried out on the spanish-indigenous language data for the bpe-based models. this consisted in whitespace normalization, unicode character normalization, and separation of punctuation from words. separation of punctuation from words was done using the moses tokenizer as well as handcrafted rules to prevent tokenization at apostrophes that actually represented glottal stops. as documented in @xcite , we also applied some spelling normalization scripts to the data for wixarika and raramuri. since all our models are multilingual models with several target languages, we include a target language tag at the beginning of the source sentence. we did not use the additional variant and quality tags proposed by de @xcite , and opted for simply relying on the target language for the tags. ## methods our subword-level settings follow previous model architectures and training regimes closely with a few updates. the main difference here is the tokenization. in the byte-level settings, we work with different architectures. ## results the different tokenization strategies resulted in different vocabulary sizes of the mt models, as can be seen in ## conclusions this paper presents the nordicalps submissions to the americasnlp 2024 machine translation shared task. our contribution focuses on data efficiency, and in particular on optimizing subword-level tokenization. we trained four systems: a baseline system with a previously trained sentencepiece tokenizer (u-sp), a subword-level system based on the proposed minimized text redundancy bpe approach (bpe-mr), and two byte-level systems differing in their decoder architectures (byte-semd with language-specific decoders and byte-sesd with a single shared decoder). we did not submit the u-sp system. the bpe-mr system reached the first rank in terms of average scores across all languages. it reached a top-five ranking for all languages except quechua. the byte-semd and byte-sesd systems performed less well, but this is most likely due to undertraining.