# Hybrid LSTM-Attention and CNN Model for Enhanced Speech Emotion Recognition
## 3 Proposed Model
### Model Development  
In this subsection, we outline our approach to designing an emotion recognition model that harnesses the power of both the CNN and LSTM-attention models, running them in parallel. Figure 2 and Table 2 illustrate an overview of the proposed deep network structure combining CNN and LSTM-attention mechanisms. \begin{table}
\begin{tabular}{c c c c c} \hline \hline \multirow{2}{*}{**Features**} & \multicolumn{2}{c}{**TESS**} & \multicolumn{2}{c}{**RAVDESS**} \\ \cline{2-5}  & **Original** & **Augmented** & **Original** & **Augmented** \\ \hline ZCR & 2800 & 19,600 & 1440 & 10,080 \\ \hline RMS & 2800 & 19,600 & 1440 & 10,080 \\ \hline MFCC & 2800 & 19,600 & 1440 & 10,080 \\ \hline Total & 8400 & 58,800 & 4320 & 30,240 \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Features extracted from original and augmented audio. Figure 2: The classification process of the proposed methodology. While CNNs are traditionally associated with identifying specific states or objects, the addition of the attention LSTM is strategic. The LSTM, with its attention mechanism, is adept at tracking temporal sequences, capturing the nuanced shifts in emotions over a period. By intertwining these two powerful architectures--the spatial feature recognition of CNN and the time-sensitive capabilities of LSTM--we aim to offer a more holistic representation of emotional progressions. Our ultimate goal with this synergistic model is to investigate whether such a combined approach can significantly enhance the precision of emotion recognition tasks. We developed a network structure aimed at enhancing accuracy by leveraging prior input data. By reconfiguring the input dimensions, we crafted a system that effectively harnesses the power of convolutions to assess features, ensuring a holistic and integrative analysis of the data. This innovative design intends to capture both the sequential patterns and inherent characteristics within the input. The rationale behind employing bidirectional LSTM stems from the understanding that emotional cues are influenced not just by preceding signals but also by the subsequent autonomic nervous system reactions lasting several seconds post-stimulation. Hence, it is integrated to account for both prior and subsequent temporal steps. The primary hidden layer in our architecture is designed with a bidirectional LSTM consisting of 128 units. This choice ensures that the model can effectively process sequences of data from both past and future points, enhancing its predictive accuracy. We have chosen the rectified linear unit (ReLU) as the activation function due to its efficiency and capability to handle non-linearities. Recognizing the common challenges of overfitting in deep learning models, where the model becomes too closely adapted to the training data and loses generalization capability on unseen data, we have integrated a dropout technique. By applying a dropout rate of 0.2, a fraction of the neurons in the network are randomly ignored during training, adding a layer of robustness and reducing the risk of overfitting. Subsequently, we incorporated an attention mechanism, whose computation is detailed as follows. \[\mathrm{H_{t}^{{}^{\prime}}=H_{t}W_{a}} \tag{5}\]  
\[\mathrm{a_{v}=\frac{exp\left(h_{t}^{{}^{\prime}}\right)}{\sum_{i=1}^{t}exp \left(h_{i}^{{}^{\prime}}\right)}} \tag{6}\] \begin{table}
\begin{tabular}{c c c} \hline
**Layer** & **Output Shape** & **Parameters** \\ \hline Input layer & (None, 339, 15) & 0 \\ \hline Conv1D & (None, 327, 8) & 1568 \\ \hline Maxpooling1D & (None, 109, 8) & 0 \\ \hline Dropout & (None, 109, 8) & 0 \\ \hline Conv1D & (None, 97, 16) & 1680 \\ \hline Maxpooling1D & (None, 32, 16) & 0 \\ \hline GlobalAveragePooling1D & (None, 16) & 0 \\ \hline LSTM & (None, 339, 32) & 6144 \\ \hline Dropout & (None, 339, 32) & 0 \\ \hline Attention & (None, 32) & 371 \\ \hline Concatenate & (None, 48) & 0 \\ \hline Dense & (None, 256) & 12,544 \\ \hline Dropout & (None, 256) & 0 \\ \hline Dense & (None, 8) & 2056 \\ \hline \end{tabular}
\end{table}