## 3 Methods
### Emotion Detection Based on Voice Features
#### 3.2.2 Data Processing and Feature Extraction  
All speech data manipulation is performed using the librosa library(v0.10) for Python3 [82]. All data recordings are brought to the same sampling frequency of 44.1 kHz before feature extraction. A window of approximately 25 milliseconds (around 1102 samples at the chosen frequency) is used. This is approximated to the nearest power of 2 for computational efficiency, resulting in 1024 samples per window. Hop size was chosen at half the value of the window size, which is 512 samples.  
**The following features** are extracted and used for the presented algorithm: 20 MFCC coefficients, 128 values from Mel Spectrograms (Figure 4), 12 values for chroma STFT (Figure 5), chroma deviation, zero crossing rate, root mean square, spectral centroid, spectral spread, spectral flux, and spectral rolloff, for a total of 168 features that are combined to form each feature vector. The extracted features are detailed in the following paragraphs:  
1. Mel frequency cepstral coefficients (MFCCs): MFCCs are a set of coefficients that describe the shape of the power spectrum of a sound signal. The raw audio signal is transformed into the frequency domain using a discrete Fourier transform (DFT) and then the Mel scale is applied to approximate the human auditory perception of sound frequency. Cepstral coefficients are computed from the Mel-scaled spectrum using a discrete cosine transform. The first 20 MFCCs are used in the current work and included in the feature vector.
2. Mel spectrograms extracted features: A Mel spectrogram represents the short-term power spectrum of a sound, with the frequencies converted to the Mel scale. The intensity of various frequency components in the audio signal can be extracted from this spectrogram. 128 features are extracted from each computed Mel spectrogram and included in the feature vector.
3. Chroma-STFT (short-time Fourier transform): There are 12 Chroma-STFT features, representing the 12 different musical pitch classes of the signal. They represent spectral energy and are useful in capturing harmonic and melodic characteristics of the signal. They can also be employed in differentiating the pitch class profiles between audio signals [83].
4. Chroma deviation: Chroma deviation measures the standard deviation of the Chroma features over time, indicating the variability of the pitch.
5. Zero crossing rate (ZCR): ZCR is defined as the rate at which a signal changes from positive to zero to negative, or vice versa. It serves as a crucial feature in identifying short and sharp sounds, effectively pinpointing minor fluctuations in signal amplitude. The count of zero crossings serves as an indicator of the frequency at which energy is concentrated within the signal spectrum [84].
6. Root mean square (RMS): RMS refers to the average loudness of the signal, taking into account the energy of the wave. The steps to compute the RMS of an audio frame are: computing the square of each sample of the signal, calculate the mean of those squared values and finally take the square root of the mean value.  
\begin{table}
\begin{tabular}{c c c c} \hline \hline
**Emotions** & **Number of Records** & **Class** & **Number of Records** \\ \hline Disgust & 1923 & - & _discarded_ \\ \hline Happy & 1923 & Positive & 2575 \\ Surprise/ & 652 & & \\ Positive surprise & 1923 & Negative & 5769 \\ Sad & 1923 & & \\ \hline Calm & 192 & & \\ Neutral & 1703 & Neutral & 1895 \\ \hline \hline \end{tabular}
\end{table}
Table 5: Redistribution of emotion recordings in the three classes of interest for the study.