## 4 Results & Discussion
### Mitigation of Random Over-Fitting Using Development Data  
A possible way to avoid selecting configurations that randomly over-fit training and testing data is to use development folds. As such, we performed cross-validation with development fold to choose the best model, and tested it using a testing fold. The results from these experiments are listed in Table 15 and depicted graphically in Figure 10. In this case, the experiments with random features are repeated six times with six different new sets of random coefficients, and average accuracy is reported. In each trial, the testing data performs worse than the development. This is to be expected because development accuracy is the greatest observed over the entire grid search, while testing accuracy could be considered as a more balanced measure of classifier performance in new data not seen in training or validation. However, when using this technique with small corpora like Neurovoz or ItalianPVS, including a development fold also leads to less training data (one fold less), which is not desirable. Notably, the trials using random coefficients with corpus sizes of Neurovoz and ItalianPVS perform the most differently between the development and testing folds. This can be explained by the absolute lack of meaning of random features other than coincidental trends that correlate with condition label vectors. \begin{table}
\begin{tabular}{c c c c c c c} \hline  & & **AVCA** & \multicolumn{2}{c}{**DARTH-VAT**} & \multicolumn{2}{c}{**Random**} \\ \hline
**Corpus** & **Vowel** & **Dev.** & **Test** & **Dev.** & **Test** & **Dev.** & **Test** \\ \hline Neurovoz & /a:/ & 71.0 & 62.5 & 70.5 & 66.5 & 62.8 (1.97) & 48.6 (5.57) \\  & /e:/ & 71.0 & 61.4 & 73.1 & 67.6 & 62.5 (3.09) & 52.4 (5.50) \\  & /i:/ & 79.4 & 71.8 & 79.4 & 70.0 & 62.3 (2.08) & 48.8 (2.41) \\  & /o:/ & 72.8 & 67.5 & 77.5 & 62.7 & 64.2 (1.85) & 52.6 (2.22) \\  & /u:/ & 75.5 & 61.8 & 85.5 & 75.1 & 67.6 (2.65) & 57.1 (3.63) \\ \hline ItalianPVS & /a:/ & 99.0 & 89.9 & 99.0 & 97.0 & 68.5 (2.74) & 51.2 (3.48) \\  & /e:/ & 98.0 & 93.9 & 98.0 & 97.0 & 67.9 (2.95) & 51.2 (3.04) \\  & /i:/ & 98.0 & 93.9 & 99.0 & 95.0 & 67.2 (2.62) & 55.6 (1.83) \\  & /o:/ & 98.0 & 93.9 & 98.9 & 95.7 & 70.2 (2.59) & 57.7 (3.67) \\  & /u:/ & 98.0 & 93.9 & 99.0 & 96.0 & 66.3 (4.36) & 50.2 (7.69) \\ \hline mPower & /a:/ & 61.6 & 60.1 & 60.1 & 58.4 & 56.5 (0.163) & 55.3 (0.415) \\ \hline \end{tabular}
\end{table} 
Table 15: Classification accuracy (%) per vowel and corpus for repeated baseline experiments, including development folds. This scenario is described in Section 3.4.5. The experiments relative to random features are repeated six times with six different new sets of random coefficients, and average accuracy is reported along with standard deviation in parenthesis. Figure 9: Accuracy of random forest classifiers trained with DARTH-VAT coefficients (**a**) as well as replacing those with random numbers between zero and one (**b**). When comparing the results from the baseline (Table 12) with those from the tests using development folds (Table 15), the classification accuracy using the development fold nearly matches the baseline's reported \"best accuracy\". On the other hand, following up with the testing fold produces a much lower classification accuracy on average. Table 16 contextualizes these differences: the mean difference between development and baseline accuracy (in corpora with five vowels) does not exceed 1.00% across all vowels. Taking the average magnitude of the difference garners a similar result, not exceeding 1.50%. However, there is not a single case where testing accuracy exceeds the baseline or development accuracy. In fact, these differences are significantly more pronounced than those between development and baseline accuracy, reaching as high of a decrease as 8.46%.