# Cross-Corpus Training Strategy for Speech Emotion Recognition Using Self-Supervised Representations
## 6 Results and Discussion
### Experiment 3: Role of In-Domain Data in the Training Set  
The third experiment conducted in this study aimed to evaluate the results of training a model using data from a different database and then fine-tuning it with increasing amounts of audio in-domain. The objective was to assess the feasibility of using a trained model as a baseline for a specific task, which can reduce the amount of speech required for training the model and the development time. This can be useful in circumstances in which obtaining labelled data is difficult or expensive. One example of this is a call-center service where the satisfaction of the customer is required to be measured. In this experiment, a model was initially trained with the IEMOCAP database and subsequently retrained with progressively increasing amounts of EmoDb and RAVDESS audio data. Each step involved adding a specific number of speakers to the database, namely, two in EmoDb and four in RAVDESS maintaining the gender balance. This corresponds to approximately three minutes of audio per iteration for EmoDb and seven minutes for RAVDESS. Figures 2 and 3 depicted the accuracy in terms of UAR of the sequence of experiments. The obtained results reveal a notable improvement when incorporating in-domain audio into the train set, corresponding to approximately three minutes of speech in each step. This addition leads to a significant enhancement in the model's performance (see the percentage of improvement with respect to the baseline trained with out-of-domain data in Table 9), demonstrating the value of including in-domain data. \begin{table}
\begin{tabular}{c c c c c} \hline \hline \multirow{2}{*}{**Classifier**} & \multicolumn{2}{c}{**MATCHED TRAIN**} & \multicolumn{2}{c}{**EXTENDED TRAIN**} \\ \cline{2-5}  & **HuBERT** & **WavLM** & **HuBERT** & **WavLM** \\ \hline \hline \multicolumn{5}{c}{Evaluation dataset: IEMOCAP} \\ \hline MLP & 63.52\% & 63.38\% & 62.56\% & 63.28\% \\ CNNSelfAtt & 64.94\% & **65.90\%** & 64.72\% & 65.06\% \\ Transformer & 60.82\% & 62.08\% & 62.56\% & 62.85\% \\ \hline \hline \multicolumn{5}{c}{Evaluation dataset: CREMA-D} \\ \hline MLP & 81.64\% & 80.85\% & 80.20\% & 79.19\% \\ CNNSelfAtt & 80.75\% & 78.51\% & 79.80\% & 78.09\% \\ Transformer & **83.04\%** & 79.57\% & 72.99\% & 78.83\% \\ \hline \hline \end{tabular}
\end{table} 
Table 8: Accuracy in terms of UAR for the SER system using SS representations (HuBERT and WavLM) with DNN-based classifiers. Matched train: the database for training and test is the same; extended training: the database for training is IEMOCAP + RAVDESS + EmoDb + CREMA-D. Bold text indicates the best performance for each database. These findings highlight the potential of leveraging out-domain training with a different database as a starting point for specific emotion recognition tasks. The ability to  
Figure 3: Accuracy in terms of UAR for RAVDESS database with a progressive increment of audio in-domain in the training set. Figure 2: Accuracy in terms of UAR for EmoDb database with a progressive increment of audio in-domain in the training set. \begin{table}
\begin{tabular}{c c c c c c c} \hline \hline  & **Step 1** & **Step 2** & **Step 3** & **Step 4** & **Step 5** & **Step 6** \\ \hline EmoDb & Baseline & 28.82\% & 36.59\% & 38.38\% & 43.99\% & – \\ RAVDESS & Baseline & 15.31\% & 22.18\% & 27.03\% & 29.68\% & 35.15\% \\ \hline \hline \end{tabular}
\end{table} 
Table 9: Improvement of the sequence of experiments progressively adding in-domain data over the baseline trained only with out-of-domain data. achieve substantial performance gains with minimal additional in-domain audio data showcases the efficacy of out-domain training in reducing the data requirements and development time of emotion recognition models. This approach offers a promising avenue for leveraging existing resources and leveraging knowledge from related tasks to enhance performance in targeted domains.