# Which Utterance Types Are Most Suitable to Detect Hypernasality Automatically?
## 3 Results
### Combining Classifiers: HN Score  
As explained in the Method section, by combining the results of multiple utterances we obtained a HN Score per participant. In order to facilitate the interpretation of these results we assume that HN Score \(>60\%\) indicates that the HN has been confirmed, while HN Score \(<40\%\) indicates that it is rejected; finally, scores in the 40-60% range do not allow to confirm or discard HN. Figure 5 shows the results obtained when using: (1) forty-four classifiers (i.e., one per utterance; HN Score (44)); note that for each utterance we chose the classifier with the highest accuracy (i.e., SVM, RF or AMM),(2) the sixteen classifiers with accuracy above 70% (i.e., HN Score (16)) and (3) the seven classifiers with accuracy above 75% together with the optimal SVM list shown in Table 2 (i.e., HN Score (7 + _Sel_)). In all three cases, the majority of the patients scored higher than 60% and the majority of the controls scored lower than 40%. However, there were some clear differences between the three selections. \begin{table}
\begin{tabular}{c c} \hline \hline
**Algorithm** & **Optimal Lists and Cumulative Accuracy** \\ \hline \multirow{7}{*}{SVM} & Base list: T5 dedo (81\%) \\  & +T3 fiff (88\%) \\  & +T2 pa (92\%) \\  & +T6 Susi sale sola (92\%) \\ \multirow{7}{*}{RF} & T6 [PERSON] David (91\%) \\  & Base list: T2 ka (84\%) \\ \cline{1-1}  & T5 dedo (86\%) \\ \cline{1-1}  & (2) +T3 f (83\%) \\ \cline{1-1}  & Base list: T2 pi (79\%) \\ \cline{1-1} ANN & +T6 [PERSON] (86\%) \\ \cline{1-1}  & +T2 ka (76\%) \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Optimal utterance lists. Figure 4: Accuracy of the 44 single-utterance SVM, RF and ANN tests. The utterances are sorted, from left to right, according to the accuracy in the best classifier. The group mean for the patients was 66% when using the forty-four one-utterance classifiers (HN Score (44)) and it increased to 77% in HN Score (16) and to 81% in HN Score (7+_Sel_). In the case of the controls the mean values were, respectively, 31%, 24% and 20% (for HN Scores 44, 16 and 7+_Sel_). The increased discriminability of HN Score (7+_Sel_) can also be observed by examining the results qualitatively (Figure 5 bottom): In this case there were only three speakers in the 40-60% region (Figure 5 bottom) and only one false negative (i.e., a patient scoring below 40%) and one false positive (i.e., a control speaker scoring above 60%). Thus, the discriminability improves as the number of tests used to compute the HN Score was reduced. Table 3 provides some details of the utterances and features used in the test selected for HN Score (7+_Sel_). Note that in Table 3 the list of features included in each case are those which were selected in the five folds during the cross-correlation train-evaluation process (see Section 2). Figure 5: HN Score using forty-four single-utterance (**top**), using the best 16 utterances (i.e., with accuracy \(>\) 70%) (**middle**), and using the best 7 utterances + the SVM optimal list (**bottom**). Each blue bar represents one control participant. Each orange bar represents one patient. The grey box shows the speakers with intermediate scores (i.e., 40–60%). Note that the blue bars to the left of the grey box and the orange bars to the right are, respectively, true negatives (TN) and true positives (TP). On the contrary, the orange bars to the left are false negatives (FN) and the blue bars to the right are false positives (FP). Given that the controls' and the patients' recordings differed in the amount of background noise, we analyzed whether this had any effect on the HN Score. For this end, we divided the patients into two subgroups: those with background noise (N = 14) and those without background noise. Note that if the presence of background noise biased the results the patients with background noise might be classified as hypernasal more frequently that the remaining patients.