# Neural Network-Based Bilingual Lexicon Induction for Indonesian Ethnic Languages
## 4 Results  
### Neural Network Performance  
This study uses two scenarios to find the optimal seq2 seq model with the best performance. When comparing the character-level and SentencePiece approaches with the seq2 seq model, the character-level seq2 seq method generates a more accurate translation of word pairs.  
As shown in Tables 4 and 5, the results demonstrate that character-level tokenization, as opposed to BPE tokenization, is more suitable for translating word-to-word. The vocabulary size has minimum and maximum values. The minimum value necessary for this experiment's data is 33. The experiment was run seven times with different vocabulary sizes; the maximum vocabulary size used was 300. When utilizing a minimal vocabulary size in BPE, it indicates that the number of tokens is approximately the same as the character-level-based method. However, as shown in Table 5, the tokenization outcomes from the source and target pairs will vary more as the vocabulary size increases, which has an impact on the BPE performance outcomes. This shows that because the vector length is shortened, the data are likely to be less informative, making it more difficult for the model to recognize. In general, the larger the vocabulary size, the higher the results due to the data being word-to-word pair translations instead of sentence-to-sentence.  
\begin{table}
\begin{tabular}{c c c c c c c} \hline \hline \multirow{2}{*}{**Method**} & \multicolumn{6}{c}{**K-Fold Cross-Validation Indonesian–Minangkabau**} \\ \cline{2-7}  & **K = 1** & **K = 2** & **K = 3** & **K = 4** & **K = 5** & **Average Precision** \\ \hline Bi-LSTM (encoder) & & & & & & \\ LSTM (decoder) & 84.72 & 83.70 & 83.31 & 83.60 & 84.30 & 83.92 \\ LSTM (encoder–decoder) & 76.79 & 74.56 & 77.82 & 78.21 & 75.87 & 76.65 \\ \hline \hline \end{tabular}
\end{table}
Table 4: Evaluation of the character-level model.  
\begin{table}
\begin{tabular}{c c c c c c c} \hline \hline \multirow{2}{*}{**Vocab Size**} & \multicolumn{6}{c}{**K-Fold Cross-Validation Indonesian–Minangkabau**} \\ \cline{2-7}  & **K = 1** & **K = 2** & **K = 3** & **K = 4** & **K = 5** & **Average Precision** \\ \hline
33 & 79.96 & 76.55 & 78.84 & **81.71** & 80.78 & **79.56** \\
35 & 76.11 & 76.89 & 79.42 & 74.31 & 80.73 & 77.49 \\
40 & 72.12 & 72.88 & 75.23 & 75.99 & 71.64 & 73.59 \\
50 & 67.12 & 62.15 & 66.97 & 67.41 & 64.29 & 65.58 \\
80 & 58.73 & 59.32 & 53.35 & 54.12 & 56.47 & 56.39 \\
100 & 49.36 & 48.24 & 49.46 & 49.70 & 48.78 & 49.10 \\
300 & 34.85 & 34.93 & 30.31 & 35.76 & 36.19 & 34.40 \\ \hline \hline \end{tabular}
\end{table}
Table 5: Evaluation of SentencePiece with the BPE model.  
Figure 6 illustrates that the character-level method has a shorter vector length (29) compared to the SentencePiece with the BPE method (300) when representing the same word, i.e., \"_adolab_\". As shown in Table 5, the larger the vocabulary size, the lower the translational accuracy results. For an example of the Minangkabau word **adolab**, if the vocabulary size = 300, the number of tokens decreases, while the lengths of the vectors representing the tokens become longer because the vectors need more expression power.