# Enhancing Multimodal Emotion Recognition through Attention Mechanisms in BERT and CNN Architectures
## 4 Results and Discussion
### Recognition Performances  
In multimodal emotion recognition, several studies have used various datasets and modalities to evaluate the effectiveness of their methodologies. A crucial aspect of these evaluations lies in performance metrics such as accuracy and F1-score, which provide insights into their efficacy and reliability.  
Table 2 and Figure 6 present a comparative analysis of emotion-recognition studies using multimodal data. Focusing on the performance metrics of the Accuracy and F1-score, we observed a range of results. [PERSON] et al. [39] reported a commendable accuracy of 81.57% and an F1-score of 81.16%. Close on its heels came [PERSON] et al. [40] with an accuracy of 81.52%, though their F1-score is not provided. A slight dip is seen in [PERSON] et al. [41], who recorded an accuracy of 75.0% and an F1-score of 74.5%. [PERSON] et al. [24] only offered an F1-score of 81.0%, omitting accuracy data. Remarkably, our method, denoted as Ours, outperforms the referenced studies with an accuracy of 83.2% and an F1-score of 82.9%. Moreover, training and validation Loss (a) and accuracy (b) performances on the CMU-MOSEI dataset were illustrated in the Figure 7. This suggests that our methodology is competitive, and also potentially sets a new benchmark in the realm of emotion recognition using the specified modality.  
Figure 6: Visualization of performances on the CMU-MOSEI dataset [24; 39; 40; 41].  
Following our earlier evaluation of the CMU-MOSEI dataset, we further expanded our comparative analysis to include the results of the MELD dataset, another key benchmark in the domain of emotion recognition. This additional comparison is vital to ensure the robustness and adaptability of the proposed system across diverse datasets.  
Table 3 and Figure 8 present the performance metrics of various studies on the MELD dataset. Each of these studies consistently employs the \"Speech + Text\" modality. [PERSON] et al. [11] developed a list with a WA of 54.79% and WF1 of 48.96%. Interestingly, while [PERSON] et al. [42] and [PERSON] et al. [43] only disclosed their WF1 scores at 65.8% and 53.42%, respectively, [PERSON] et al. [44] presented a more comprehensive result with a WA of 65.59% and a WF1 of 64.50%. Significantly, our approach surpasses these metrics with a leading WA of 66.81% and WF1 of 66.12%. Moreover, training and validation Loss (a) and accuracy (b) performances on the MELD dataset were illustrated in the Figure 9. This further consolidation of the MELD dataset underscores the efficiency and superiority of the proposed methodology for multimodal emotion recognition.  
\begin{table}
\begin{tabular}{c c c c} \hline \hline \multirow{2}{*}{**Research**} & \multirow{2}{*}{**Modality**} & \multicolumn{2}{c}{**Metrics**} \\ \cline{3-4}  & & **Accuracy** & **F1** \\ \hline [PERSON] et al. [39] & Speech + Text & 81.57 & 81.16 \\ \hline [PERSON] et al. [40] & Speech + Text & 81.52 & - \\ \hline [PERSON] et al. [41] & Speech + Text & 75.0 & 74.5 \\ \hline [PERSON] et al. [24] & Speech + Text & - & 81.0 \\ \hline Ours & Speech + Text & 88.4 & 87.9 \\ \hline \hline \end{tabular}
\end{table}
Table 2: Comparison of the recognition performances in the CMU-MOSEI dataset.  
Figure 7: Training and validation Loss (**a**) and accuracy (**b**) performances on the CMU-MOSEI dataset.