# Cross-Corpus Training Strategy for Speech Emotion Recognition Using Self-Supervised Representations
###### Abstract  
Speech Emotion Recognition (SER) plays a crucial role in applications involving human-machine interaction. However, the scarcity of suitable emotional speech datasets presents a major challenge for accurate SER systems. Deep Neural Network (DNN)-based solutions currently in use require substantial labelled data for successful training. Previous studies have proposed strategies to expand the training set in this framework by leveraging available emotion speech corpora. This paper assesses the impact of a cross-corpus training extension for a SER system using self-supervised (SS) representations, namely HuBERT and WavLM. The feasibility of training systems with just a few minutes of in-domain audio is also analyzed. The experimental results demonstrate that augmenting the training set with EmoDB (German), RAVDESS, and CREMA-D (English) datasets leads to improved SER accuracy on the IEMOCAP dataset. By combining a cross-corpus training extension and SS representations, state-of-the-art performance is achieved. These findings suggest that the cross-corpus strategy effectively addresses the scarcity of labelled data and enhances the performance of SER systems.  
speech emotion recognition; cross-corpus; data augmentation; self-supervised representation +
Footnote †: journal:  
has been utilized in certain SER systems as a means to overcome the challenges posed by the aforementioned dataset limitations [4; 5].  
This paper explores the cross-corpus strategy as an extension of the training set, building upon the study conducted by [PERSON] et al. [6]. In the previous work, the authors combined three datasets (EmoDb, RAVDESS, and IEMOCAP) to train a DNN-based SER system, utilizing the HuBERT SS representation. The results demonstrated an improvement in SER accuracy, even in the presence of language mismatch between training and testing datasets. This finding highlighted the potential of the cross-corpus strategy in enhancing the robustness of SER systems. Expanding on these findings, this paper investigates the cross-corpus strategy in greater detail. It explores additional language variability by incorporating a dataset in a different language, and it evaluates the system's performance by gradually increasing the amount of matched data in the training set. Furthermore, another SS representation, WavLM, is assessed, considering its favorable performance in previous studies.  
In summary, this paper makes the following contributions:  
1. It assesses the effectiveness of the cross-corpus strategy for SER systems.
2. It evaluates the incorporation of multiple languages within the cross-corpus strategy.
3. It investigates the feasibility of training SER systems using predominantly out-domain data with a limited amount of in-domain data.  
In the following, Section 2 provides a review of prior research on the cross-corpus strategy for training extension within the context of SER. Subsequently, Section 3 introduces the databases and performance metrics utilized in the experimental setup. Section 4 offers a comprehensive description of the SER system's architecture, while Section 5 outlines the experiments conducted in this study. Finally, Section 6 presents and discusses the obtained results, and Section 7 concludes the paper.