# Speech Emotion Recognition: Comparative Analysis of CNN-LSTM and Attention-Enhanced CNN-LSTM Models
## 3 Methodology
### Models Description
#### 3.3.2 Stacked Time Distributed 2D CNN-Bidirectional LSTM with Attention  
The second model builds on the first model Section 3.3.1 by adding a 2D LSTM and an attention layer. The primary enhancement is the addition of a bidirectional LSTM, which processes speech data in both forward and reverse directions, allowing the model to gain a deeper understanding of emotions in speech (Architecture is shown in Figure 4). Additionally, an attention mechanism is included to improve performance. This mechanism focuses on the most crucial segments of the speech, assigning weights to portions that carry significant emotional cues. This focused approach helps the model identify emotional signals more accurately, thereby improving its emotion classification accuracy. The bidirectional LSTM layer processes the input sequence in both forward and backward directions. This is represented mathematically as follows:  
\[\overrightarrow{h}_{t}=\text{LSTM}(x_{t},\overrightarrow{h}_{t-1}) \tag{10}\]  
\[\overleftarrow{h}_{t}=\text{LSTM}(x_{t},\overleftarrow{h}_{t+1}) \tag{11}\]  
where \(\overrightarrow{h}_{t}\) and \(\overleftarrow{h}_{t}\) are the hidden states of the forward and backward LSTMs at time step \(t\). The final hidden state \(h_{t}\) is obtained by concatenating these two states:  
\[h_{t}=[\overrightarrow{h}_{t};\overleftarrow{h}_{t}] \tag{12}\]  
This bidirectional processing allows the model to capture dependencies from both past and future contexts, providing a more comprehensive understanding of the sequential data. Moreover, the attention mechanism is applied to the output of the bidirectional LSTM. The attention mechanism computes a context vector \(c\) as a weighted sum of the hidden states, where the weights \(\alpha_{t}\) are determined by the relevance of each hidden state:  
\[\alpha_{t}=\frac{\exp{(e_{t})}}{\sum_{i=1}^{T}\exp{(e_{i})}} \tag{13}\]  
\[e_{t}=\tanh(W_{a}h_{t}+b_{a}) \tag{14}\]\[c=\sum_{t=1}^{T}\alpha_{t}h_{t} \tag{15}\]  
where \(W_{a}\) and \(b_{a}\) are learnable parameters. The context vector \(c\) captures the most relevant information from the entire sequence, allowing the model to focus on the crucial parts of the input when making predictions. Finally, the context vector \(c\) is passed through a dense layer, followed by a softmax activation to produce the final emotion classification probabilities:  
\[y=\text{softmax}(W_{c}c+b_{c}) \tag{16}\]  
While developing proposed models, it was observed that the balanced data were causing overfitting, meaning the models were performing well on the training data but not generalizing well to new, unseen data (test). To address this issue, we introduced noise to the data, which is an important step in enhancing the model's generalization capabilities, unlike simpler methods like LIWC, which rely on linguistic categorization [32]. The noise was added during the data augmentation stage. We used Additive White Gaussian Noise (AWGN) and uniform noise to augment the training data, unlike Google's attention model, which focuses on sequence weighting without noise handling [33]. The process involves generating white Gaussian noise and adding it to the original signal. First, the signal and the generated noise were normalized. The normalization is based on the number of bits used to represent each sample in the audio signal, typically 16 bits for high-quality audio. This normalization and addition of noise in both models is mathematically given by the following:  
\[\text{norm\_constant}=2^{(\text{num\_bits}-1)} \tag{17}\]  
\[\text{signal\_norm}=\frac{\text{signal}}{\text{norm\_constant}} \tag{18}\]  
\[\text{noise\_norm}=\frac{\text{noise}}{\text{norm\_constant}} \tag{19}\]  
Next, the power of the signal and the noise was computed:  
\[\overleftarrow{h}_{t}=\text{LSTM}(x_{t},\overleftarrow{h}_{t+1}) \tag{11}\]0  
\[\overleftarrow{h}_{t}=\text{LSTM}(x_{t},\overleftarrow{h}_{t+1}) \tag{11}\]1  
\[\overleftarrow{h}_{t}=\text{LSTM}(x_{t},\overleftarrow{h}_{t+1}) \tag{11}\]2 Model 2: Proposed system schema showing 2D CNN-BiLSTM architecture.