BSC Submission to the A mericas NLP 2024 Shared Task (2024)

Abstract: This paper describes the BSC’s submission to the AmericasNLP 2024 Shared Task. We participated in the Spanish to Quechua and Spanish to Guarani tasks. In this paper we show that by using LoRA adapters we can achieve similar performance as a full parameter fine-tuning by only training 14.2% of the total number of parameters. Our systems achieved the highest ChrF++ scores and ranked first for both directions in the final results outperforming strong baseline systems in the provided development and test datasets.

## introduction in this paper we present the submission of the barcelona super computing center to the workshop on natural language processing (nlp) for indigenous languages of the americas (americ-asnlp) 2024 shared task on machine translation systems for indigenous languages. we participated in two directions: spanish to guarani and spanish to quechua. the quechuan language family, comprising a diverse range of dialects, is spoken by over 6 million individuals across south america. it has a wide geographic spread, extending from colombia through ecuador, peru, bolivia, and argentina. quechua is an oral language, which results in a scarcity of textual resources. in this work we use an amalgamated data set for quechua from cuzco (quz) and ayacucho (quy), which are the most widely spoken. guarani belongs to the tupi-guarani branch of the tupian language family. it is one of the official languages of paraguay and has more than 9 million speakers. the data used in this work contains a mix of dialects, from pure guarani to more mixed jopara. in this work we fine-tuned various versions of @xcite as in previous year winning submission @xcite . originally, nllb-200 had been trained using parallel data in quechua, guarani, and spanish. our experiments aimed to answer the following questions: (1) how beneficial is fine-tuning with multilingual data in low resource scenarios? (2) does translation quality improve with a greater model size for low resource languages? (3) can we use lora for fine-tuning multilingual neural machine translation (mnmt) systems? the remainder of this paper is organized as follows: section 2 describes the data collection and preprocessing, section 3 introduces our training methodology, section 4 presents the results of our experiments and comments the results on the test sets, and finally, section 5 summarizes the main findings. ## data collection and preparation in this section we detail our data sources and the data cleaning pipeline used for cleaning training data. we utilize the parallel data provided by the organizers and additional sources for training our systems. ## conclusions in this paper we describe the barcelona supercomputing center submission to the americasnlp 2024 shared task. we took part in the spanish to quechua and spanish to guarani tracks. we finetuned different versions of the nllb-200 model. our systems ranked in first place in both translation directions outperforming the provided baselines. our experiments show that increasing model size does not yield superior performance when data is scarce. also, we show that training with multilingual data combined with synthetic data improves translation quality for the spanish to quechua direction. finally, we show that by fine-tuning the models using lora, we can obtain a similar performance as a full parameter fine-tuning while training only 14.2% of the parameters.