###### Abstract  
Automatic Speech Emotion Recognition (SER) plays a vital role in making human-computer interactions more natural and effective. A significant challenge in SER development is the limited availability of diverse emotional speech datasets, which hinders the application of advanced deep learning models. Transfer learning is a machine learning technique that helps address this issue by utilizing knowledge from pre-trained models to improve performance on a new task in a target domain, even with limited data. This study investigates the use of transfer learning from various pre-trained networks, including speaker embedding models such as d-vector, x-vector, and r-vector, and image classification models like AlexNet, GoogLeNet, SqueezeNet, ResNet-18, and ResNet-50. We also propose enhanced versions of the x-vector and r-vector models incorporating Multi-Head Attention Pooling and Angular Margin Softmax, alongside other architectural improvements. Additionally, reverberation from the Room Impulse Response datasets was added to the speech utterances to diversity and augment the available data. Notably, the enhanced r-vector model achieved classification accuracies of 74.05% Unweighted Accuracy (UA) and 73.68% Weighted Accuracy (WA) on the IEMOCAP dataset, and 80.25% UA and 79.81% WA on the CREMA-D dataset, outperforming the existing state-of-the-art methods. This study shows that using cross-domain transfer learning is beneficial for low-resource emotion recognition. The enhanced models developed in other domains (for non-emotional tasks) can further improve the accuracy of SER. speech emotion recognition; IEMOCAP; CREMA-D; transfer learning image classification; speaker embeddings; affective computing +
Footnote †: journal: Medicalcan be negatively impacted by background acoustical noise or the quality of the audio recording. Furthermore, emotion expressed through speech might not always align with the true emotional state of speaker, leading to potential misinterpretations. Despite these challenges, SER has emerged as a critical component in the development of emotion-aware applications, with the potential to significantly enrich human-machine interaction. While humans are naturally adept at discerning emotions through speech, replicating this capability in machines remains a challenging task. SER seeks to bridge this gap by utilizing speech signal processing and machine learning to decode emotional states. There is a vast range of SER applications, from personal virtual assistants and emergency call centres, where understanding the emotional state of a caller can be crucial [2, 3], to educational tools, driver safety systems, therapeutic diagnostics, and immersive gaming experiences [4, 5, 6]. Despite the potential of SER, developing systems that can reliably capture subtle nuances in human speech remains a significant challenge. This difficulty arises from the diverse ways people express emotions, and the limited availability of large, diverse datasets needed to train state-of-the-art Deep Neural Network (DNN) models. Previous studies have explored various approaches to facilitate the development of the SER systems and improve their performance. Some have focused on learning from basic sound representations, such as raw waveforms or spectrograms, employing models based on Convolutional Neural Networks (CNNs) and Recurrent Neural Networks (RNNs) [7, 8, 9]. Standard speech features like MFCCs and a range of spectral, temporal, and statistical features (e.g., the OpenSMILE set) have been utilized to improve DNN models [10, 11]. A key strength of DNN-based approaches is their ability to learn generic representations that can be transferred across different tasks and domains [12]. A major challenge in developing SER systems is the lack of diverse and sufficiently large emotional speech datasets, which limits the use of advanced DNNs. Transfer learning (TL) helps overcome this issue by using knowledge from pre-trained models in a source domain to improve performance on new tasks in a target (downstream) domain.