# Speech Emotion Recognition through Hybrid Features and Convolutional Neural Network
## 3 Materials and Methods
### Feature Extraction  
The recognition performance of the model is largely determined by the quality of the feature set. Therefore, unsuitable features may lead to poor recognition results. In the context of machine learning (ML), extracting a relevant feature set is a crucial task for achieving reasonable recognition performance. Ref. [38] reported that feature extraction is an important phase in ML, as the failure or success of the SER model largely depends on the variability of features utilized in the recognition task. Recognition becomes accurate if the derived features are highly correlated with the emotion class, while it becomes difficult and inaccurate if the derived features do not strongly correlate with emotion. In SER, the quality of the feature set greatly affects recognition performance. Therefore, useful features must be extracted from collected audio files, which are suitable to learn recognition rules. Feature extraction is an essential step in this process and requires significant effort and creativity. It involves extracting suitable features from emotional audio files and transforming them into an MFV. The MFV is then utilized by a ML or DL technique to develop a recognition system. It is more challenging to derive features than classification due to its domain-specific nature. In this paper, a new feature extraction technique was implemented to derive effective and suitable features, called MFCCT features, from audio files for constructing an accurate SER model. Details of the derived MFCCT features are presented in the following subsection. The Proposed MFCCT Features  
The hybrid features were derived from audio files into two steps: (1) extracting MFCC features, and (2) fusion of time-domain and MFCC features. (a) Extracting the MFCCs features  
MFCCs features are derived to capture the vocal tract characteristics of an emotion. The process of MFCCs feature extraction involves framing, windowing, the discrete Fourier transform (DFT), taking the log of magnitude, frequency warping on Mel scale, and applying discrete cosine transform (DCT). To prevent information loss, each utterance was divided into frames of 25 ms with a 10 ms overlap between successive frames. The total frames for single audio can be computed using Equation (3), while the number of samples (\(N\)) can be calculated using Equation (4). \[Total_{frames}=\frac{NumberofSamples}{\left(Frame_{step}\right)\times\left(SampleRate\right)} \tag{3}\]  
\[N=\left(Frame_{length}\right)\times SampleRate \tag{4}\]  
Once the framing steps were completed, each individual frame underwent hamming windowing to ensure that the edges of the frame were smoothed by Equation (5). \[w(n)=0.54-0.46\cos(\frac{2\pi n}{N-1}),0\leq n\leq N \tag{5}\]  
where \(N\) represents the total samples per frame. Next, the magnitude spectrum of all frames was computed through DFT. These magnitude spectrums were passed to the multiple Mel-filter banks, where Mel represents the perceived frequency of ears. The Mel can be computed as:  
\[Mel(f)=2595\times\log_{10}\left(1+\frac{f}{700}\right) \tag{6}\]  
where \(f\) denotes the actual frequency and the \(Mel(f)\) denotes the corresponding perceived frequency. To mimic the frequency perception of ears, frequency axis was warped through Equation (6). The Mel-frequency warping with a triangular filter bank is commonly used to achieve this. The Mel-spectrum was then obtained by multiplying all magnitude spectra of triangular filters, \(X(k)\), as given below. \[s(m)=\sum_{k=0}^{N-1}\left[\left|X(k)\right|^{2}\times H_{m}(k)\right];0\leq m \leq M-1 \tag{7}\]  
where \(M\) represents the triangular filters. \(H_{m}(k)\) denotes the weight assigned to \(k\)th bin of spectrum, which contributes to the \(m\)th output band. Mathematically it is written as:  
\[H_{m}(k)= \begin{cases}0,&k<f(m-1)\\ \frac{2(k-f(m-1))}{f(m)-f(m-1)},&f(m-1)\leq k\leq f(m)\\ \frac{2(f(m+1)-k)}{f(m+1)-f(m)},&f(m)<k\leq f(m+1)\\ 0,&k>f(m+1)\end{cases}  \tag{8}\]  
where m differs from 0 to \(M-1\).