# Automatic Translation between Mixtec to Spanish Languages Using Neural Networks
## 4 Methodology
### Phase 2: Developing and Applying the Corpus Processing Module
**Transformer Network**

Figure 7 shows the proposed transformer architecture. Table 11 shows the number of layers and hyperparameters. Each \(X_{i}\) in the input layer denotes a sentence in Mixtec and Spanish, and subsequently, each word of the sentence was converted into an embedding. For the Transfomer model, we use word embeddings of size 512. We utilized the OpenNMT-py library due to its capabilities in sequence generation, support for transformer models, recurrent neural networks, and features for automatic evaluation. Our work was conducted within the Google Colab environment. Figure 6: Architecture of the recurrent neural network. Described as \(X\) are the input sentences in Mixtec and Spanish, followed by LSTM cells for the recurrent neural network, while \(Y\) denotes the target or output text of the architecture. Figure 7: Architecture of the Transformer network. \(X\) denotes the corpus in Mixtec and Spanish. Next, we use the Transformer with encoder and decoder layers. The variable \(Y\) denotes the output of the Transformer. Algorithm 2 presents the pseudocode for the training process. The instructions on lines 2, 3, and 4 utilize commands provided by OpenNMT-py, with the input parameters specified in a configuration file. ```
1:Start
2:Input: Text file in Mixtec and Spanish in utf8, YAML configuration file
3:Output: Meta text file
4:Specify the path of the YAML configuration file. 5:Call _ommt build vocab_ to generate vocabularies in Mixtec and Spanish. 6:Call _ommt train_ to initiate the training process. 7:Call _ommt translate_ to conduct translation tests. 8:End
```

**Algorithm 2** Training algorithm

The parameters listed in Table 11 are specified within a YAML configuration file. This file includes paths for the training and validation corpora, along with paths for logs that detail the training process. Figure 8 shows in more detail the architecture of the translation training algorithm. The result is called the target text or translated text. The target text is generated in a plain text file. Figure 8: The architecture of the translation training algorithm involves several key steps. Initially, vocabularies for both Mixtec and Spanish are created. This is followed by the execution of the training command. Finally, the process concludes with the execution of the command to perform testing. \begin{table}
\begin{tabular}{c c c c} \hline \hline
**No.** & **Parameter** & **RNN and BRNN** & **Transformer** \\ \hline
1 & Batch size & 16 & 4096 \\
2 & Train steps & 1000 & 700 \\
3 & Valid steps & 500 & 350 \\
4 & Optimizer & Adam & Adam \\
5 & Word vector size & 128 & 512 \\
6 & Layers & \(1,3\) & 6 \\
7 & Learning rate & 0.15 & 2 \\
8 & Dropout & 0.0,0.1 & 0.1, 0.3 \\
9 & Hidden size & 512 & 512 \\
10 & Heads & 0 & 8 \\
11 & Activation function & relu & relu \\ \hline \hline \end{tabular}
\end{table} 
Table 11: Input parameters of the neural network. ## 5 Results

In this section, we explore the results of the Mixtec-Spanish translation experiments. The evaluation is twofold; it includes an automatic assessment utilizing the BLEU metric and a manual review conducted by an expert in the Mixtec language. As detailed by [PERSON] et al. [12], the BLEU metric facilitates the automatic comparison of a translation system's output against a reference translation, ideally crafted by a human expert. BLEU scores range from 0 to 1, with 0 indicating no resemblance between the candidate and reference translations and 1 signifying a perfect match. On the other hand, the resulting TER score represents the edit distance between the translated text and the reference text. A lower TER score indicates a translation that is closer to the reference and thus generally of higher quality. Conversely, a higher TER score suggests that more edits are needed to match the reference translation, indicating a lower-quality translation. In the experiments conducted, a total of 4568 Mixtec-Spanish sentence pairs were utilized.