# Cross-Corpus Training Strategy for Speech Emotion Recognition Using Self-Supervised Representations
## 6 Results and Discussion
### Experiment 1: Cross-Corpus Strategy for Training
#### 6.1.1 Analysis of the Performance among Representations and Classifiers  
The findings of our experiments seem to indicate that WavLM outperforms HuBERT in terms of results, although the performance difference between them is not big enough to give a definite answer. Throughout most experimental configurations, the performance of WavLM representation either matches or surpasses that of HuBERT. The observed difference between them amounts to approximately 1.5%. These findings are consistent with the results presented by [PERSON] et al. [17], who showed how WavLM outperforms HuBERT in several alternative speech-processing tasks.  
However, it is noteworthy that, in the experiments conducted on the EmoDb database, the advantage of WavLM over HuBERT is less pronounced. In fact, in this dataset, most combinations demonstrate superior performance with HuBERT compared to WavLM. This particular outcome may be attributed to the limited size of the EmoDb database, rendering the performance disparities between the two models irrelevant.  
Regarding the classifiers, there is not a clear outperformance among them due to their results being very similar. However, analyzing further, the CNNSelfAtt gets better results no matter the representation employed or the kind of training for the largest database (IEMOCAP). On the other side, despite the CT-Transformer being the largest and most complex model, its results are not correspondingly improved, indicating that this model is not using its full potential, which may be related to the scarcity of audio data for fulfilling the complexity of the model to be properly trained.  
The following experiments are performed with the WavLM representation and CNN with self-attention to classification.  
Analysis of the System Errors among Emotions  
The confusion matrix in Table 6 shows that the most common failure is confusing neutrality with happiness and vice versa. This can be due to the lack of contextual information the classifier has. Another common source of confusion is anger and happiness. A possible reason for this is both are high-valence emotions according to [PERSON]'s emotional model [26].  
The recall per label for the optimal combination for IEMOCAP is shown in Table 7. The \"Neutral\" label gets considerably worse results than the average (7% below). On the other side, the accuracy of \"anger\" is 6% better than the average. This is consistent with the findings of [PERSON] [27], who found that anger is easier to detect in speech for humans than other emotions of the spectrum, with \"normal state\" (Neutral) being the most difficult to identify among the ones used in this work.  
\begin{table}
\begin{tabular}{c c c c c c} \hline  & & \multicolumn{4}{c}{**Predicted Value**} \\ \cline{3-6}  & & **Neutral** & **Happiness** & **Anger** & **Sadness** \\ \hline \multirow{3}{*}{Real Value} & Neutral & 976 & 345 & 136 & 251 \\  & Happiness & 338 & 981 & 167 & 150 \\ \cline{1-1}  & Anger & 142 & 155 & 771 & 35 \\ \cline{1-1}  & Sadness & 220 & 116 & 9 & 709 \\ \hline \end{tabular}
\end{table}
Table 6: Confusion matrix for IEMOCAP with the model based on WavLM-CNNSelfAtt on the EXTENDED TRAIN configuration.  
\begin{table}
\begin{tabular}{c c c c c} \hline
**Emotion** & **Neutral** & **Happiness** & **Anger** & **Sadness** \\ \hline Recall & 59.37\% & 65.06\% & 71.81\% & 66.72\% \\ \hline \end{tabular}
\end{table}
Table 7: Recall by class for IEMOCAP with the model based on WavLM-CNNSelfAtt on the EXTENDED TRAIN configuration.