# Collaborative Analysis of Learners' Emotional States Based on Cross-Modal Higher-Order Reasoning
## 2 Related Works
### Attention Mechanism  
Attention mechanism, as an important neural network technology, was inspired by the human visual and cognitive systems. Its earliest proposal can be traced back to the field of cognitive psychology in the 1980s. However, the application of attention mechanisms in the field of deep learning began in recent years. Early work laid the foundation for the introduction of attention mechanisms into neural networks. These studies introduced attention mechanisms into autoencoders and neural machine translation models, demonstrating their potential in improving model performance and generalization ability.  
Figure 2: Late fusion diagram.  
The attention mechanism achieves targeted focus and weight allocation of information through a mechanism called Query-Attention-Key. The core concept of this mechanism is to compute the relevance between a query and a set of keys and use this relevance to determine the weight distribution of the final output values. The computation process of the attention mechanism is illustrated in Figure 4. Specifically, it first computes the relevance between the query and each key using some calculation method (e.g., dot product). Then, it uses some normalization function (e.g., softmax function) to transform these relevances into a weight distribution, ensuring that the sum of all weights is 1. Finally, it uses these weight allocations to aggregate all values, generating the final output result.  
Figure 4: The computational process diagram of the attention mechanism.  
Figure 3: Hybrid fusion diagram.  
The specific calculation methods as shown in Formulas (1) to (3):  
\[Sim_{i}(Query,Key_{i})=Query\cdot Key_{i} \tag{1}\]  
\[a_{i}=Softmax(Sim_{i})=\frac{e^{Sim_{i}}}{\Sigma_{j=1}^{L_{x}}e^{Sim_{j}}} \tag{2}\]  
\[Attention=\sum_{i=1}^{L_{x}}a_{i}\cdot Value_{i} \tag{3}\]  
The attention mechanism plays a crucial role in multimodal feature fusion, effectively capturing the correlations between different modalities and dynamically adjusting the features' weights to enhance the model's performance and expressive power. Introducing the attention mechanism into modal fusion allows the model to flexibly focus on essential information from different modalities, achieving more precise fusion. The attention mechanism enables the model to dynamically learn the weights of each feature when processing each modality and adjust the fusion weights based on their importance, making significant features more prominent and suppressing less relevant ones. This dynamic adjustment mechanism enhances the model's flexibility and generalization ability. For instance, consider a multimodal task involving vision and text. In early fusion, the feature vectors of vision and text can be concatenated, and the attention mechanism dynamically adjusts the weights of different modal features to reflect their importance in the task. In late fusion, after each modality undergoes independent processing, the attention mechanism is used to fuse different modal features to better capture inter-modal correlations. In practical applications, attention mechanisms can be implemented in various ways, such as fully connected layers, convolutional layers, or self-attention mechanisms. Self-attention mechanisms are particularly effective as they consider the relationships between different positions in the input sequence, better capturing long-distance dependencies within the sequence. This mechanism has achieved significant success in fields like natural language processing and computer vision and has been widely applied in multimodal learning tasks.