## 4 Experiments
### Classification of Matched-Subject Groups  
In order to address the first of the potential causes for the low sensitivity, we age-matched a balanced group of 40 subjects, 20 vocally healthy (non-teachers) and 20 vocally fatigued (early career teachers), from a total of 88 subjects. Additionally, to address the second cause--lack of amplitude normalization--the subjects were matched on skinkold thickness [24]. The descriptive statistics for both groups are shown in Table 4. All vocally healthy and vocally fatigued subjects had a VFI-1 \(\leqslant\) 10 and VFI-1 \(\geqslant\) 15, respectively--therefore, no subjects were in the gray area of VFI-1 scores. This age- and skinkold-thickness-based matching allowed us to eliminate all potential issues discussed earlier, since now the classes were balanced; amplitude normalization was approached via skinkold thickness; and incorrect labels were alleviated by leaving subjects in the \"gray area\" out of the experiment.  
We performed the same LOO test as described in Section 4.1. The total numbers of Positive and Negative samples were 3270 and 3202, respectively, and Table 5 shows the final classification results. This time, an even higher validation accuracy 97.51% was obtained--demonstrating that the imbalanced data and normalization indeed played a role in the classification. Similarly, after performing the LOO test, the overall accuracy of testing also improved from 58.13% to 62.36%, and the true positive from 22.62% to 71.76%. Both of these highly promising improvements show that the assumptions on imbalanced data and lack of normalization were correct. However, these changes had an impact on the specificity of the classifier, which decreased to 0.53.  
\begin{table}
\begin{tabular}{c c c} \hline \hline  & **Vocally Fatigued** & **Vocally Healthy** \\ \hline Number of subjects & 20 & 20 \\ Age (21–39 years) & 25.6 \(\pm\) 4.3 & 25.3 \(\pm\) 4.7 \\ Suprahyoid (3.2–16.7 mm) & 7.0 \(\pm\) 3.4 & 5.4 \(\pm\) 1.3 \\ Infrahyoid (2.7–15.0 mm) & 6.4 \(\pm\) 3.1 & 5.1 \(\pm\) 1.3 \\ VFI-1 (0–28) & 18.2 \(\pm\) 5.4 & 2.1 \(\pm\) 1.7 \\ \hline \hline \end{tabular}
\end{table}
Table 4: Descriptive statistics for age, neck skinkold thickness (supra- and infrahyoid), and Vocal Fatigue Index factor 1 (VFI-1) scores for 40 matched test subjects.  
\begin{table}
\begin{tabular}{c c c} \hline \hline  & **Actual Positive** & **Actual Negative** \\ \hline
**Predicted Positive** & 93.40\% & 6.49\% \\
**Predicted Negative** & 6.6\% & 93.51\% \\
**Validation Accuracy** & 93.48\% & \\ \hline \hline \end{tabular}
\end{table}
Table 3: Confusion matrix for positive vs. negative detection of vowel productions based on self-reported VFI-1 scores among 88 test subjects using leave-one-out cross-validation.  
In summary, these results demonstrated that, as long as the data are balanced, normalized and appropriately labeled, the classification of normal and vocally fatigued vowel productions as indicated by self-reported vocal fatigue can be achieved, and yet the classifier can reasonably generalize on a completely new test subject. However, further investigation into the generalization ability of the classifier is warranted.