# Hybrid LSTM-Attention and CNN Model for Enhanced Speech Emotion Recognition
## 4 Experiments and Discussion
### Results of the Model Training and Testing  
Our model underwent a rigorous training and testing process, spanning 400 epochs with batch sizes of 32. The computational infrastructure backing this extensive training was robust, featuring the high-performance RTX 3090 Ti 24 GB graphics card and the Intel Core i9-13900K Processor. Further, we ensured smooth operation and processing speed by employing the Windows 11 operating system, complemented by a substantial 64 GB of RAM. To refine the model's performance and enhance its predictive capabilities, we employed the Adam optimizer. This optimizer was selected with a learning rate set at a meticulous value of 0.0001, making it a pivotal tool for hyperparameter tuning throughout the training phase. To ensure an impartial evaluation of our model using the TESS and RAVDESS datasets, we adopted an exhaustive end-to-end training approach, modifying the initial data as detailed in Ref. [43]. We strategically split our dataset. A majority, 80% to be precise, was dedicated to training the model, allowing it ample data for learning and adjusting. The remaining 20% was reserved strictly for testing, ensuring that we had a significant amount of data to validate our model's performance in unseen scenarios. \begin{table}
\begin{tabular}{c c c} \hline \hline \multirow{2}{*}{**Hardware**} & **GPU** & **GeForce RTX 3090 Ti, 24 GB GDDR6X, 384-bit** \\ \cline{2-3}  & **RAM** & **DDR4 64 GB** \\ \cline{2-3}  & **CPU** & **Intel Core i9-13900K** \\ \hline Software & Windows 11, Python 3.6, Librosa 0.10.1, Tensorflow 1.15, CUDA 9.0, Cudnn 7.1 \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Hardware and software specifications. Given the depth and extent of our training process, it was essential to safeguard against the risk of overfitting--a common pitfall in which the model becomes too tailored to the training data, reducing its generalization capability. To combat this, we implemented several countermeasures like early stopping, dropout with a rate of 0.2, and L2-regularization techniques. Together, these techniques ensured that our model was robust, accurate, and capable of generalizing well to new data. In our experiments, we only focused on models that demonstrated high accuracy and good fit. The accuracy in the validation phase of the model serves as a crucial indicator of its ability to generalize to new data. Figures 4(a)-4(a) illustrate that the model was successful in discerning the key features vital for accurately categorizing speech emotions. After the 100 th epoch, the loss values steadily approached zero. The training process also monitored the model's accuracy, which is depicted in Figures 4(b)-4(b). These figures reveal an improvement in accuracy as the training epochs progress. The selected learning rate ensures that this improvement is consistent. Notably, the model's accuracy peaks after the 25 th and 150 th iteration for the TESS and RAVDESS datasets, respectively. Moreover, the validation accuracy plateaus, suggesting that the model has effectively learned from the training data and is primed to perform well on the test set. It is worth mentioning that using data augmentation during training led to a notable boost in accuracy. Consequently, the model achieved an average recognition accuracy of 99.8% and 95.7% with the TESS and RAVDESS datasets, respectively, for the emotions correctly identified in the training and validation sample. Additionally, Figures 10 and 10 offer an insightful assessment of the model's ability to discern various emotions in the TESS and RAVDESS datasets. The metrics showcased, such as precision, recall, and F1-score, act as strong measures of the model's effectiveness and its proficiency in delivering precise predictions. Considering these metrics in tandem allows for a comprehensive assessment of the model's strengths and potential areas of improvement. Such an approach is invaluable, as it paints a holistic picture of the model's adaptability and reliability, especially when faced with diverse and potentially challenging datasets.