# Automatic Speech Recognition Advancements for Indigenous Languages of the Americas
## 5 Conclusions and Future Work  
In this work, we have fine-tuned and reported the very first ASR models for the Wa'ikhana and Kotiria languages and have additionally established new benchmarks within the dataset of the Americas NLP Challenge 2022 for Guarani, Quechua, and Bribri, marking a significant step forward in this field. These results represent an important contribution to the study of ASR in Indigenous languages, in which only a few dozen studies are reported for these languages. Additionally, we have published our best models in a repository and the full hyperparameter experiments in the SI. We show that pre-trained models are very promising when fine-tuning them to some unknown domain with a very low amount of data but that bigger architectures do not always achieve better results. This may be due to the fact that, when the amount of data is very small and also very far from the pre-trained domain, it is more difficult to converge to an optimal solution if the number of trainable parameters is too large. This would imply that for some corner cases and minority languages, the race to use architectures with an increasing number of parameters may not be the most effective. Surprisingly, we did not find a clear relationship between the number of hours available for each language and the accuracy of the fine-tuned models. This may be related to the phonemic distance between the pre-trained languages and the fine-tuned domains, which may have made the quantized latent pre-trained representations unsuitable for all the target languages. The application of Sobol' sensitivity analysis has allowed us to identify critical hyperparameters that impact language model performance in ASR tasks. These insights inform future hyperparameter tuning efforts, enhancing the accuracy and efficiency of ASR systems for under-resourced languages. We unveil that, when considering internal correlations, the number of epochs where the pre-trained layers of the models are frozen is significantly more important than the total number of trained epochs. This indicates that, when a low amount of data is available for fine-tuning, the greatest effort has to be made in inefficiently adjusting the weights of the neural network in the last layers rather than in adjusting its weights across the architecture. The second most important factor unveiled by [PERSON]' analysis is the dropout rate, which shows that, when the number of trainable parameters is so large and the amount of data is so scarce, the risk of overtraining is very high. This comprehensive evaluation of hyperparameters provides a valuable tool for optimizing ASR models, which is particularly crucial for languages with limited resources and data availability. While our proposed ASR system demonstrates significant advancements for the Indigenous languages of the Americas, several limitations remain. First, the relatively small size of the training datasets limits the model's ability to generalize across diverse speech patterns and dialects. Additionally, the quality and variety of the data used for training and evaluation can impact the performance, as some languages had fewer resources available. Another limitation is the dependency on pre-trained models, which may not fully capture the unique phonetic and grammatical structures of these languages. Future work should focus on expanding the datasets with more diverse and representative samples, improving data collection methodologies, using transfer learning and knowledge distillation, and exploring different model architectures--such as others based on conformers--that can better handle the linguistic complexities of Indigenous languages [85]. Additionally, bimodal or \begin{table}
\begin{tabular}{c c c} \hline \hline
**Parameter** & **S1** & **ST** \\ \hline Freeze fine-tune updates & 0.32 & 0.41 \\ Activation dropout & 0.35 & 0.37 \\ Mask prob & 0.54 & 0.29 \\ Mask channel prob & 0.54 & 0.28 \\ Learning rate (lr) & 0.57 & 0.26 \\ Max update & \(\sim\)0 & 0.13 \\ \hline \hline \end{tabular}
\end{table}