## 4 Experiments
### Performance and Result Analysis  
The experimental results are presented in Table 4, where Concat represents the feature concatenation integration method and Add represents the feature superposition integration method. To ensure the accuracy of the entire experiment, a five-fold cross-validation method was adopted. The accuracy and network inference time were recorded in the experiment. As a comparison, the machine learning method Random Forest and the fusion  
Figure 11: PCAN architecture diagram. These entry arrows represent the input sources involved in the corresponding operation, and the output arrows represent the output results of the operation. scheme based on GRU and attention (GRU + Attention) model [12] was utilized for data fusion in the experiment. As depicted in Table 4, the recognition accuracy for learning emotions using Random Forest (Concat) is 93.96%, which is only 0.28% higher than the cascade attention-based method based on video signals described in Section 3.2.1. This demonstrates that, compared to deep learning feature representation, machine learning methods perform less effectively in fusing the contextual information of different modal data in the context of current emotion recognition tasks. On the contrary, the deep learning method is generally better. The GRU + Attention model is suitable for the Concat method in integrating different modal features, while the Add method has slightly poorer performance. Our PCAN model is more suitable for integration of different modal features through the Add method, and achieves optimal classification accuracy. This is because through multi-branch connections and multi-scale attention modules, contextual information of spatial and temporal structures of different modal data can be fused, resulting in strong temporal correlation of the fused features, thus achieving complementary advantages of different modal data. In terms of inference speed, the single-modal cascade attention-based method is faster, while the GRU + Attention method outperforms PCAN. To further verify the performance of PCAN, we also compare the confusion matrix and ROC curve in the experiment. Firstly, we take the GRU + Attention Concat method with good performance as an example for comparison. In Figure 12, the horizontal axis denotes the predicted labels produced by the network, while the vertical axis indicates the actual labels of the data. The point of intersection between the predicted and actual labels indicates the confusion prediction rate for each category. The GRU + Attention Concat method can correctly identify all Bored categories, and the accuracy rate of Interest category identification is 93.90%, and the proportion of confusion with Neutral category is 6.10%. However, the recognition accuracy of the Neutral \begin{table}
\begin{tabular}{l c c} \hline \hline
**Methods** & **Accuracy** & **Reasoning Speed (s)** \\ \hline Random Forest (Concat) & 93.96\% & 2 \\ \hline Cascade Attention-based [33] & 93.68\% & 0.06/epoch \\ \hline GRU + Attention (Add) & 93.62\% & 0.17 s/epoch \\ \hline GRU + Attention (Concat) & 94.69\% & 0.17 s/epoch \\ \hline PCAN (Concat) & 94.27\% & 0.20 s/epoch \\ \hline PCAN (Add) & 95.75\% & 0.21 s/epoch \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Results of multimodal fusion emotion recognition. Figure 12: Comparison of confusion matrix. category is only 91.20%, and the confusion rate between this category and the Bored category is relatively large, at 5.90%. The confusion ratio for the Interest category accounts for 2.90%. However, 6.10% of the sample data of the Interest category are identified as the Neutral category, so it can be seen that neutral emotions are not easy to identify, and network models easily identify neutral emotions as the Bored category. In comparison, PCAN's recognition accuracy of neutrality reached 94.1%, which effectively reduced the confusion rate between this category and the Bored category, accounting for 2.90%.