## 5 Conclusions and Future Work  
This study presents a novel approach to advancing emotional understanding in communication by merging speech emotion and text sentiment using a two-layered dynamic Bayesian mixture model (2L-DBMM). Our methodology, integrating handcrafted audio features with deep learning and classical machine learning models, has demonstrated remarkable success in accurately capturing emotional nuances in speech. By incorporating diverse statistical attributes and deep learning techniques, our framework achieved promising accuracy rates on publicly available datasets. We demonstrated a significant improvement in classifying second-level emotional states, surpassing existing methods and highlighting the potential of combining SER and SA for a more comprehensive understanding of affective communication. Our contributions extend beyond developing a high-performing model. By defining powerful handcrafted features, we have advanced SER capabilities. The adaptation and extension of the DBMM and the proposal of the 2L-DBMM facilitated not only the merging of multiple classifiers for each modality but also the successful fusion of SER and SA, opening new avenues for affective communication analysis. Extensive evaluation on datasets, encompassing both Portuguese (EmoUERJ) and English (ESD) languages, has revealed valuable insights into the cross-linguistic capabilities of our approach. Our experimental results on EmoUERJ showed an average accuracy of 94% for SER, 85% for SA, and 96% for the fused 2L-DBMM. On the ESD dataset, the extended DBMM achieved 97% accuracy for SER, 95% for SA, and an impressive 98% when merging both modalities with 2L-DBMM. These results highlight the power of combining acoustic and textual features in a dynamic Bayesian framework to enhance the recognition of complex emotional states in speech, even across different languages. Future research will focus on further optimizing the fusion process for even more accurate and nuanced \begin{table}
\begin{tabular}{l c c c} \hline \hline \multicolumn{1}{c}{\multirow{2}{*}{**Dataset**}} & \multirow{2}{*}{**Comparison**} & \multirow{2}{*}{\(p\)**-Value**} & \multicolumn{1}{c}{\multirow{2}{*}{**Significant**}} \\  & & & **Difference (\(p<0.05\))** \\ \hline ESD & DBMM vs. BERT & \(6.54\times 10^{-8}\) & Yes \\ \hline ESD & DBMM vs. GPT-2 & \(1.37\times 10^{-62}\) & Yes \\ \hline ESD & DBMM vs. SVM & \(2.19\times 10^{-5}\) & Yes \\ \hline ESD & DBMM vs. RF & \(1.84\times 10^{-6}\) & Yes \\ \hline ESD & DBMM vs. LR & \(5.42\times 10^{-6}\) & Yes \\ \hline ESD & DBMM vs. MLP & \(2.38\times 10^{-54}\) & Yes \\ \hline ESD & DBMM vs. 1D-CNN & \(1.01\times 10^{-71}\) & Yes \\ \hline EmoUERJ & DBMM vs. BERT & \(2.11\times 10^{-2}\) & Yes \\ \hline EmoUERJ & DBMM vs. GPT-2 & \(4.36\times 10^{-13}\) & Yes \\ \hline EmoUERJ & DBMM vs. SVM & \(3.98\times 10^{-2}\) & Yes \\ \hline EmoUERJ & DBMM vs. RF & \(1.59\times 10^{-2}\) & Yes \\ \hline EmoUERJ & DBMM vs. LR & \(2.87\times 10^{-2}\) & Yes \\ \hline EmoUERJ & DBMM vs. MLP & \(1.19\times 10^{-11}\) & Yes \\ \hline EmoUERJ & DBMM vs. 1D-CNN & \(2.51\times 10^{-24}\) & Yes \\ \hline \hline \end{tabular}
\end{table} 
Table 18: McNemar’s test results for SA (DBMM vs. individual classifiers). \begin{table}
\begin{tabular}{l c c c} \hline \hline \multicolumn{1}{c}{\multirow{2}{*}{**Dataset**}} & \multirow{2}{*}{**Comparison**} & \multirow{2}{*}{\(p\)**-Value**} & \multicolumn{1}{c}{\multirow{2}{*}{**Significant**}} \\  & & & **Difference (\(p<0.05\))** \\ \hline ESD & 2L-DBMM vs. MLP (SER) & \(3.98\times 10^{-14}\) & Yes \\ \hline ESD & 2L-DBMM vs. BERT (SA) & \(1.07\times 10^{-7}\) & Yes \\ \hline EmoUERJ & 2L-DBMM vs. MLP (SER) & \(1.88\times 10^{-12}\) & Yes \\ \hline EmoUERJ & 2L-DBMM vs. BERT (SA) & \(2.11\times 10^{-2}\) & Yes \\ \hline \hline \end{tabular}
\end{table} 
Table 19: McNemar’s test results for 2L-DBMM (fusion) vs. best individual classifiers. emotion classification.