Ayuuk- S panish Neural Machine Translator (2021)

Abstract: This paper presents the first neural machine translator system for the Ayuuk language. In our experiments we translate from Ayuuk to Spanish, and fromSpanish to Ayuuk. Ayuuk is a language spoken in the Oaxaca state of Mexico by the Ayuukjä’äy people (in Spanish commonly known as Mixes. We use different sources to create a low-resource parallel corpus, more than 6,000 phrases. For some of these resources we rely on automatic alignment. The proposed system is based on the Transformer neural architecture and it uses sub-word level tokenization as the input. We show the current performance given the resources we have collected for the San Juan Güichicovi variant, they are promising, up to 5 BLEU. We based our development on the Masakhane project for African languages.

## introduction in recent years the efforts to preserve and promote the creation of nlp tools for the native languages of the americas have increased, particularly addressing the challenges that this endeavour requires @xcite . machine translation (mt) has become one of the main goals to pursue since in the long term it might offer benefits to the communities that speak such languages. for instance, it might provide access to knowledge in their native language and facilitate access to services such legal, medical and finance assistance. in this work, we explore this avenue for the san juan güichicovi variant of the ayuuk language, mainly because one of the authors is a native speaker of this variant. to our knowledge there has not been a construction of such a system for the ayuuk although other variants 1 are available in the jw300 corpus @xcite . in this work we rely in multiple previous work. at the core of our proposal we follow the steps from 1 coatlán mixe (iso 639-3 mco), ayuuk of the coatlán region. the masakhane project foot_0 which focuses on african languages @xcite . we also rely on the following libraries: with these tools we developed our code base that can be consulted online together with the part of the corpus which is freely available foot_4 . ## ayuuk from san juan güichicovi ayuukjä'äy can be translated as people of the mountains, most them can be located in 24 municipalities of the oaxaca state. they are the native speakers of the ayuuk language with approximately 139, 760 speakers in mexico. the ayuuk language, which has an iso 639-3 code mir, belongs to the mixezoqueana linguistic family. this linguistic family is composed by the mixe and zoque subfamilies 7 . in particular, the mixe subfamily also includes mixe of oaxaca, sayula popoluca and oluta popoluca languages. for ayuuk there are six main variants of the language, among these the mixe bajo to which the san juan güichicovi variant belongs to. at this municipality it can be estimated there is approximately 18, 298 speakers of the variant. it is important to notice that it is estimated that only 3, 205 are monolinguist. the san juan güichicovi's ayuuk variant does not has a normalized orthography, there are efforts to agree on orthographic conventions however there are strong positions related to number of consonants. one of these positions, it is known as the "bodegeros" position which proposes 20 consonants (see 1b.a) @xcite vs "petakeros" which proposes a reduction to 13 (see 1b.b) @xcite . in terms of vowels, this variant has six (see 2) which contrast with the other variants of ayuuk which can have up to nine vowels. the following are examples of san juan güichicovi'sayuuk these were taken from short stories recollected and written by albino pedro juan a native speaker and preserver of the language. the bunny become happy. el conejo se puso feliz. when everything become silence. cuando todo se silencia. ## the parallel corpus for the creation of the parallel corpus we collected samples from different sources for which there was a available translation between ayuuk and spanish, see since we have a diverse source of linguistic sources it was necessary to normalize the orthography. for this we follow the proposal from sagi-vela @xcite who has followed the unification of the ayuuk language avoiding taking sides on the controversy about the number of consonants. mainly we made two replacements: ñ/ny and ch/tsy some of the works were already aligned, others not. for those not aligned we created automatic alignments using the yasa tool @xcite . we discarded all empty and double alignments. normalization and automatic alignments were manually verified by one of the authors. the corpus keep differences among both normalization variants: petakeros and bodegeros. finally, we randomly split the sentences into training, development and testing sets. for our experimentation we created two split versions, one strict and one random. in the strict version we use all the phrases from the national archive of indigenous languages @xcite as a test. since these sentences are linguistically motivated and aim to show linguistic aspects of the language they tend to be harder to translate; this split resulted in 5, 847/700/912 (train/dev/test). in the random split we randomly sample sentences from our sources, the final split resulted in 5, 941/700/912 (train/dev/test). notice that amount of phrases among splits changes, this is because after separating the test phrases, we remove repeated or similar phrases for the train/dev sets. our intuition was to have a more uniform training/validation for the random split while the test follows the distribution of the original sources. we mimic this procedure for the strict sample. ## neural architecture our translation model is based on the transformer architecture @xcite . we use an encoder-decoder setting. for our experiments we have two configurations for both encoder and decoder: a number of layers: 3, number of heads: 4, input embedding dimensionality: 64, embedding dimensionality: 64, batch size: 128. b number of layers: 6, number of heads: 4, input embedding dimensionality: 256, embedding dimensionality: 256, batch size: 128. these models were trained in a server with two tesla v100 gpus. to obtain a model it usually take us around 2h for a 100 epochs. we also were able to reproduce the experiments in the colaboratory platform. ## experiments and results as described in the previous section we have two different versions of our splits, strict and random. per split we performed five experiments, two for configuration with fewer layers (a), and three for the configuration with more layers (b). we also modified: a) the maximum length of the phrase (50 or 70) b) the vocabulary of the bpe sub-word algorithm (we tested 2000 or 4000). figure 1 shows the perplexity and the bleu score in the development set during training for the direction spanish (es) to ayuuk (mir). the first part of the figure 2 shows the lerning curve on the direction of translation ayuuk (mir) to spanish (es). the second part of the ## conclusions and further work previous experiences on mt based on deep learning architecture, particularly on seq2seq settings, for native languages of the americas have not been promising @xcite . in particular, because there is little to none training data. however, our work shows that a standard model based on the transformer architecture and under in order to improve the performance of the system future work will focus on: