# Research on Personalized Course Resource Recommendation Method Based on GEMRec
## 3 Graph-Enhanced Multimodal Recommendation Method: GEMRec
### Multimodal Feature Fusion and Entity Extraction Strategy
#### 3.2.3 Multimodal Feature Fusion Strategy  
The effective integration of multimodal features requires careful consideration of both feature alignment and cross-modal interactions. Our fusion strategy addresses these challenges through a comprehensive multi-stage approach that ensures optimal information utilization from each modality while preserving their complementary characteristics.  
Initially, we address the dimensional inconsistency among different modal features through a specialized feature alignment network. This network projects features from each modality into a shared semantic space while preserving their essential characteristics. For each modality m, we define a projection function as follows:  
\[\text{h}_{\text{m}}=\text{W}_{\text{m}}\text{x}_{\text{m}}+\text{b}_{\text{m}} \tag{10}\]  
where \(W_{m}\) and \(b_{m}\) are learnable parameters specific to each modality, and \(x_{m}\) represents the original features.  
To capture the complex interactions between different modalities, we implement a cross-modal attention mechanism that allows each modality to focus on relevant information from others. The attention mechanism is formulated as a multi-head structure that can capture different aspects of cross-modal relationships, as follows:  
\[\text{Attention}_{\text{Cross}}(\text{Q}_{i},\text{K}_{j},\text{V}_{j})=\text {softmax}(\frac{\text{Q}_{i}\text{K}_{j}^{\text{T}}}{\sqrt{\text{d}_{k}}}) \text{V}_{j} \tag{11}\]  
where \(Q_{i}\), \(K_{j}\), and \(V_{j}\) represent the query, key, and value matrices from modalities i and j, respectively, and \(d_{k}\) is the dimension of the key vectors.  
To adaptively integrate information from different modalities while maintaining their unique contributions, we introduce a gated fusion mechanism. This mechanism learns to weigh the importance of each modality dynamically based on the current context.  
\[\mathrm{g_{m}}=\sigma\big{(}\mathrm{W_{g}[h_{heat};h_{video};h_{audio}]+b_{g} \big{)}} \tag{12}\]  
\[\mathrm{h_{fused}}=\sum\mathrm{g_{m}\odot h_{m}} \tag{13}\]  
where \(g_{m}\) represents the importance weight for modality m, \(\sigma\) is the sigmoid activation function, and \(\odot\) denotes element-wise multiplication.  
The fusion process is further enhanced by incorporating residual connections to preserve the original information and facilitate gradient flow during training as follows:  
\[\mathrm{h_{final}}=\mathrm{LayerNorm}(\mathrm{h_{fused}}+\alpha\mathrm{sum_{m} h_{m}}) \tag{14}\]  
where \(\alpha\) is a learnable parameter that controls the contribution of the residual connection, and LayerNorm represents layer normalization to stabilize the learning process.  
This sophisticated fusion strategy ensures that the model can effectively leverage complementary information from different modalities while being robust to potential noise or missing data in any single modality. The resulting fused representations capture rich semantic relationships that might be overlooked when considering each modality in isolation.