## 3 Materials and Methods
### Feature Extraction  
For our experiments, the audio data was processed at a 16 kHz sampling rate across all datasets. The EMO-DB dataset was downsampled from 48 kHz to 16 kHz, while the RAVDESS and IEMOCAP datasets were downsampled to 16 kHz from their original 44.1 kHz sampling rate. The SAVEE dataset was already at 16 kHz. For the public datasets (EMO-DB, RAVDESS, SAVEE, IEMOCAP), the audio lengths ranged from approximately 1 s to 15 s. The SHEIE dataset was specifically segmented into 3-5 s intervals. SER extracts features from speech waves to determine a person's emotional state because transforming speech into features or parameters allows the determination of emotional characteristics. In this study, a multi-feature approach was employed to capture various aspects of the speech signals. The audio data were segmented into frames of 32 ms with an overlap of 50%, and a Hamming window of 64 ms was applied to extract the Mel-frequency cepstral coefficient (MFCC) features. The extracted features were represented in a one-dimensional format, capturing the temporal dynamics of the speech signal. Features that can be extracted from the time domain include ZCR, energy, and amplitude, which can be used to identify anger and excitement by revealing speech rate and volume [29]. Pitch, formant, and other features can be extracted from the frequency domain, with the ZCR, chroma, and roll-off representing some of these features, as shown in Figure 5. Formants (resonant frequencies of the vocal tract) reveal the shape of the vocal tract, which determines the characteristics of articulated speech sounds. Pitch (the perceived fundamental frequency of a sound) can indicate emotions, such as fear, neutral [30]. This is further supported by [PERSON] [31], who found that spectral listeners, who focus on higher harmonics, per-form better in emotion judgment tasks. [PERSON] [32] found that vocal expressions of different emotions are characterized by specific acoustical changes, such as spectral and segmental changes, and variations in pitch levels. The MFCC, spectral contrast, and chroma also affect the spectral domain [33]. The MFCC, with 40 coefficients being used in this study, offers the best approximation of the human ear's nonlinear frequency perception and represents a sound's short-term power spectrum. Thus, they allow systems to more effectively understand emotions owing to the systems acting like humans. This implies that combining features from different domains guarantees a complete speech signal representation, allowing SER systems to identify emotions accurately and precisely. The ZCR measures how often a signal crosses the zero axis by determining the signal sign changes per frame [33]. As such, it denotes the total number of times the wave flips  
Figure 4: Effects of data augmentation on SER: (**a**) original audio with additive white Gaussian noise; (**b**) original audio with time stretching; and (**c**) original sound with pitch shifting. from positive to negative, and it is distributed contrariwise by frame. Mathematically, the ZCR can be determined using the following equation:  
\[zcr=\frac{1}{T-1}\sum_{t=1}^{T-1}\quad\mathds{1}_{\mathbb{R}_{<0}}(s_{t}s_{t-1}), \tag{1}\]  
where \(s\) is a signal of length (\(T\)) and \(\mathds{1}_{\mathbb{R}_{<0}}\) is a sign function. A chromatogram visualizes audio by mapping frequencies onto 12 bins that match the 12 semitones of an octave using chroma features [34]. This compresses pitch content into time windows, enabling music analysis applications to recognize chords and harmonic similarities despite timbre and instrumentation changes. A Mel spectrogram visualizes audio signals by mapping frequencies onto the Mel scale, which is aligned with human hearing. This technique captures important sound characteristics, thereby facilitating its widespread use in speech and audio processing. The spectral contrast, centroid, bandwidth, and roll-off are features extracted from sound signals.