## 3 Results  
### Classification Results  
Table 3 presents the performance of the different classification systems in the task of automatically predicting the expert judgments. The different cases displayed in the table (case A to F) are based on the different subcorpora. These subcorpora were recorded with a different version of the video game and a different recording context and they were not balanced in terms of sample size and number of speakers, so it was important to know how these differences would affect the classifier accuracy. Therefore, the results of using the recordings of the three subcorpora, as well as all the combinations of these subcorpora, were compared. The SVM classifier works better with all subcorpora and the worst results are obtained using the DT classifier (best case is 79.3% vs. 64.9% baseline). The best results are obtained in Cases A and D by using any of the three classifiers (UAR 0.83 with SVM classifier). The classification accuracy decreases when the C3 corpus is entered (C, E, F and G cases), as the number of speakers substantially increases. On average, we obtain 89.9% of true positives. This will be discussed in the next section as a positive result for real time situations.  
\begin{table}
\begin{tabular}{l c c c c c c c c c c c c c c} \hline \hline  & & & & \multicolumn{3}{c}{**DT**} & \multicolumn{3}{c}{**SVM**} & \multicolumn{3}{c}{**MLP**} \\ \cline{3-13}  & **Corpora** & **BL** & **CR** & **AUC** & **UAR** & **CR** & **AUC** & **UAR** & **CR** & **AUC** & **UAR** & **\#Util** & **\#Feat.** & **\#SPK** \\ \hline Case A & C1 & 65.8\% & 69.6\% & 0.68 & 0.74 & 78.5\% & 0.74 & 0.83 & 73.2\% & 0.7 & 0.79 & 605 & 21 & 5 \\ Case B & C2 & 61.9\% & 60.3\% & 0.58 & 0.61 & 72.7\% & 0.7 & 0.79 & 68.5\% & 0.67 & 0.73 & 168 & 16 & 5 \\ Case C & C3 & 50.8\% & 65.8\% & 0.66 & 0.66 & 61.6\% & 0.62 & 0.69 & 63.7\% & 0.64 & 0.64 & 193 & 7 & 13 \\ Case D & C1+C2 & 64.9\% & 70.8\% & 0.68 & 0.75 & 79.3\% & 0.76 & 0.83 & 72.6\% & 0.7 & 0.78 & 773 & 21 & 10 \\ Case E & C1+C3 & 62.2\% & 66.3\% & 0.65 & 0.69 & 72.3\% & 0.7 & 0.79 & 67.2\% & 0.65 & 0.74 & 798 & 20 & 18 \\ Case F & C2+C3 & 56\% & 60.9\% & 0.6 & 0.64 & 66.5\% & 0.66 & 0.75 & 64\% & 0.63 & 0.69 & 361 & 13 & 18 \\ Case G & C1+C2+C3 & 62.1\% & 66.9\% & 0.66 & 0.71 & 74.3\% & 0.71 & 0.81 & 69.4\% & 0.66 & 0.76 & 996 & 20 & 23 \\ \hline \hline \end{tabular}
\end{table}
Table 3: Classification results depending on the corpus and the classifier used. The prosody expert judgments were used to train the classifiers. This table shows the performance baseline (BL) of each group of samples (number of samples of the most populated class divided by all the samples), Decision trees (DT), Support vector machines (SVM), Multilayer Perceptron (MLP), classification rate (CR), Area Under the Curve (AUC) and Unweighted Average Recall (UAR). The number of samples (utt.), the number of speakers (SPK) and the number of features (Feat.) are presented. The output of the different classifiers are Right or Wrong, based on prosody expert scoring.  
Table 4 shows the prosodic features with more influence in the utterance assessment of each C1 subcorpus speaker. The data for all the speakers of the C1 subcorpus were used to remove the highly correlated features (above 0.8 of Pearson correlation). After this redundant feature deletion, 22 of the 34 features were selected. Within these 22 features, only 10 present a significant ROC area value (above 0.6).