## 2 Emotion Recognition and Related Works
### Related Works in SER
Combining SER with other modalities like facial expressions and physiological signals improves accuracy and provides a more comprehensive understanding, which is crucial for safety and productivity [29]. However, challenges such as background noise in real-world environments can affect SER performance. Additionally, integrating multiple modalities increases computational complexity and may impact real-time processing capabilities [30]. One well-known characteristic of DNNs is their requirement for a substantial amount of data to achieve satisfactory performance. In several studies [31; 32; 33; 34], TL approaches usingpre-trained models have been applied to tasks such as speech recognition, acoustic event detection, music analysis, and emotion recognition, aiming to address the challenge posed by limited datasets. Numerous studies have investigated the suitability of a TL approach from the image recognition domain for the SER task. [PERSON] et al. [35] employed deep spectrum features derived from the image classification of spectrograms using a pre-trained DCNN model called AlexNet [36]. The performance of deep spectrum features was compared to various acoustic feature representation sets specifically designed for emotion recognition and paralinguistic tasks, including eGeMAPS [37], COMPARE [38], and bag-of-audio word (BoAW) [39]. The results indicate that deep spectrum features either matched or outperformed traditional speech feature representations on the FAU-AIBO Emotion Corpus. [PERSON] et al. [40] proposed employing three channels of log Mel-spectrograms (static, delta, and delta-delta coefficients) as input for DCNN. The DCNN extracts segment-level features, which are then pooled into an utterance-level representation using the Discriminant Temporal Pyramid Matching strategy. The obtained results suggest that the pre-trained and fine-tuned AlexNet outperformed the simple DCNN, which was trained from scratch. [PERSON] et al. [41] combined DCNN pre-trained on the ImageNet dataset and BLSTM with an attention mechanism. The 3D Mel spectrograms were used as input to the DCNN model, where deep features were extracted at the utterance level and then fed into BLSTM for additional temporal modelling. The proposed pre-training improved accuracy by approximately 6% and 4% for the EmoDB and IEMOCAP datasets, respectively. The utilization of TL from the speaker recognition domain aims to leverage the similarities between speaker recognition and emotion recognition. This approach could work both ways: integrating emotion and speaker embedding information enhanced the performance of speaker identification in [42]. [PERSON] et al. [43] introduced a residual adapter architecture, in which the base trained ResNet model underwent pre-training for a speaker verification task using an extensive speaker database, VoxCeleb2. Modules containing domain-agnostic parameters (trained on a speaker database) were utilized alongside adapter modules containing domain-specific parameters for adaptation to the target emotion dataset. The proposed residual adapter outperformed the ResNet20 trained from scratch on emotional data. The fine-tuning of the ResNet20 approach did not yield clear results across various datasets, also resulting in a decrease in performance. This decline could be attributed to the potential forgetting of learned speaker information or the smaller size of the dataset used. In the study by [PERSON] et al. [16], it was demonstrated that features of emotional speech extracted from pre-trained models on speaker recognition, such as the ResNet-based x-vector, outperformed randomly initialized ResNet models. Furthermore, fine-tuning for emotion classification led to a significant improvement in accuracy for datasets such as IEMOCAP, MSP-Podcast, and CREMA-D. [PERSON] et al. [44] employed a ResNet model pre-trained on extensive datasets labelled with speaker information. Input representations in the form of spectrograms were utilized and augmented to mitigate overfitting.