The UCF Systems for the L o R es MT 2021 Machine Translation Shared Task (2021)

Abstract: We present the University of Central Florida systems for the LoResMT 2021 Shared Task, participating in the English-Irish and English-Marathi translation pairs. We focused our efforts on constrained track of the task, using transfer learning and subword segmentation to enhance our models given small amounts of training data. Our models achieved the highest BLEU scores on the fully constrained tracks of English-Irish, Irish-English, and Marathi-English with scores of 13.5, 21.3, and 17.9 respectively

## introduction in this paper, we describe the systems developed at the university of central florida for our participation in the marathi and irish tasks of the loresmt 2021 shared task for low-resource supervised machine translation of covid-19 related texts @xcite . for these tasks, participants were asked to develop systems for the english to irish, irish to english, english to marathi, or marathi to english translation directions. submissions are split into three tracks based on data constraints: a constrained track using only provided data, an unconstrained track allowing for publicly available corpora, and an unconstrained track allowing for both publicly available corpora and pre-trained models. we utilize the neural machine translation (nmt) approach, due to its prevalence in current research. while they are able to achieve state-of-the-art results in high-resource translation tasks, nmt systems tend to particularly struggle in low-resource scenarios. to alleviate this, our experiments focus primarily on data augmentation and transfer learning. we tried different techniques such as back-translation and subword segmentation, although they yielded little to no improvement in most cases. our best performing systems during development for the irish task utilized transfer learning from english-marathi models. for the marathi task, the best performances came from models trained on text pre-processed with subword segmentation via a unigram language model @xcite . we submitted six systems for evaluation, one for each translation direction in the constrained track and two unconstrained marathi models, which achieved the highest bleu scores in the constrained tracks of english-irish, irish-english, and marathi-english. ## data we only utilize the data provided by the shared task organizers @xcite for our experiments. all text was pre-processed using moses tokenizer @xcite and lowercased prior to training. we also experimented with a pre-trained sentencepiece @xcite tokenization model for marathi from inltk @xcite . final translation results were true-cased with moses prior to submission. we filtered out training sentences in the english-marathi data that were also found in the development set from the training set to better evaluate our models during development. the parallel english-marathi data was much larger than the english-irish set, at 20,470 and 8,112 sentences respectively. a summary of the basic statistics of the final dataset used can be found in ## system description we implement our models using opennmt-py @xcite . we initially tested both the transformer @xcite and lstm @xcite architectures to obtain baseline results, but we found lstms to consistently yield better results despite attempts in optimizing transformer parameters. as such, all of our experiments utilize lstms in a standard encoder-decoder setup. we keep the majority of the parameters as their default values: we use 2 lstm layers with 500 hidden units, an initial learning rate of 1, a dropout rate of 0.3, and stochastic gradient descent as the optimizer. we found the default step count of 100,000 to work well with the provided corpora with a batch size of 32. ## experiments and results we first trained models using each technique to establish the effectiveness of a technique in each translation direction. techniques that obtained a higher score than the baseline were then jointly used to develop additional models. we evaluated each model using the sacrebleu @xcite implementation of bleu @xcite . bleu scores in the development stage are presented in ## conclusion we present systems for machine translation of irish and marathi to and from english. we improved over a developed baseline by incorporating transfer learning between language tasks and subword segmentation into our models. we also experimented with synthetic data generation via back-translation, which did not show any notable improvements during development. at test time, our models achieved the highest bleu scores in the constrained tracks of english-irish, irish-english, and marathi-english.