Lightweight neural translation technologies for low-resource languages (2024)

Abstract: The LiLowLa (“Lightweight neural translation technologies for low-resource languages”) project aims to enhance machine translation (MT) and translation memory (TM) technologies, particularly for low-resource language pairs, where adequate linguistic resources are scarce. The project started in September 2022 and will run till August 2025.

## project overview the lilowla 1 ("lightweight neural translation technologies for low-resource languages") project, funded by the spanish government and the european regional development fund, aims to enhance machine translation (mt) and translation memory (tm) technologies, particularly for lowresource language pairs, 2 where adequate linguistic resources are scarce. 3 additionally, the project seeks to optimize web crawling methods to gather relevant data for low-resource languages effectively while avoiding unnecessary downloads, thereby reducing crawling times and enabling the acquisition of larger parallel corpora. the scarcity of linguistic resources is often a result of the low commercial interest in the language pair in question, frequently stemming from the economic constraints of the speaking communities. this also implies that translation technologies developed for low-resource language pairs are likely to be utilized in environments with limited computing capabilities; for this reason, the project focuses on lightweight technologies. being useless after their processing. to attain these objectives, the project focuses on investigating how to make neural mt significantly more robust and efficient by distilling the knowledge in large pre-trained neural models initially developed for high-resource language pairs, such as nllb-200, and researching new lightweight data augmentation techniques to make the most of the scarce resources available. it also concentrates on the development of smart focus crawlers to improve current corpus crawling methods, and on the integration of cross-lingual sentence embeddings into cat tools to permit the search of translation proposals in monolingual corpora. ## languages of interest in addition to the improvement of translation technologies for low-resource language pairs, lilowla seeks to build corpora and translation models for a number of languages selected on the basis of social impact and the preservation of their cultural heritage: ## expected results as a result of the execution of the project we plan to deliver: 1. a smart bilingual focus crawler guiding the crawling towards webpages more likely to contain parallel content; thus facilitating quicker discovery and reducing the time and bandwidth needed for acquiring parallel corpora. ## resources corpora and software developed as part of the project will be released under free/open-source licenses. in what follows we provide an incomplete list of software and corpora released so far: foot_0 matilda: multitask data augmentation approach able to improve translation performance by generating synthetic training samples with non-fluent target segments (sánchez-cartagena et al., 2024). 5 crosslingualneuralfms: method for obtaining translation proposal from target-language monolingual corpora in cat tools (esplà-gomis et al., 2022). 6 tune 'n' distill: pipeline to tune the mbart50 model to low-resource language pairs, and distill the resulting system to obtain a lightweight model (galiano-jiménez et al., 2023). 7 pilar: collection of parallel and monolingual corpora for low-resource languages of the iberian peninsula. 8 mayanv: parallel corpora between several mayan languages and spanish (lou et al., 2024). 9 url2lang: tool to infer the language of a document from the url linking to it. 10 parallel urls classifier: tool to infer whether a pair of urls link to parallel documents. 11