# UPGCN: User Perception-Guided Graph Convolutional Network for Multimodal Recommendation
## 3 Methods
### Multimodal Two-Step Enhanced Fusion Method  
To effectively differentiate the similarities among multimodal features while fully utilizing their complementarity, we propose the multimodal two-step enhanced fusion method. This approach employs a strategy of pre-enhancement followed by fusion, wherein the embedding of item ID is initially enhanced based on modal features across different modalities to separate the similarities between these diverse modal features. The second enhanced signal is derived from Equation (9), which aims to enhance the item ID embedding by utilizing the item modality information \(h_{i,m}^{(l+1)}\) after fusing the user modality information. In summary, the enhancement process involves extracting modality-specific information from both the original item modal features and the item modal features that incorporate user modality features, subsequently reflecting this information in the embedding of item ID. Specifically, after deriving the modal feature \(h_{m}\) with dimensions matching those of the ID embedding using Equation (7), we utilize this modality feature \(h_{m}\) as a guiding signal to enhance the initial embedding of item ID \(e_{u}^{(0)}\), thereby distinguishing similarities between modalities and emphasizing the unique characteristics inherent to each modality. The calculation equation is as follows:  
\[\bar{e}_{m}=e_{i}^{(0)}\mathcal{O}\sigma(h_{m}W_{id}^{item}+b_{id}^{item}) \tag{13}\]where \(W_{id}^{item}\in\mathbb{R}^{d\times d}\) and \(b_{id}^{item}\in\mathbb{R}^{d}\) represent the transformation matrix and bias, respectively, both of which are learnable parameters, \(\sigma\) denotes the sigmoid function, while \(\bigcirc\) represents the element-wise product. Guided by modal feature \(h_{m}\), the initial embedding of item ID \(e_{i}^{(0)}\) was enhanced to yield an improved embedding of item ID \(\overline{e}_{m}\). Following the calculation of user modal information \(h_{u,m}\) using Equation (7) and given that we continue to utilize \(h_{u,m}\) as the neighbor node, the item modal information \(h_{i,m}^{(l+1)}\) obtained through the GCN aggregation of neighboring nodes contains certain user preference information. To fully leverage this information, we opt to enhance \(\overline{e}_{m}\) using item modal information \(h_{i,m}^{(l+1)}\) after the initial enhancement step. The enhancement process is analogous to the methodologies presented in Equations (10) and (11), wherein \(h_{i,m}^{(l+1)}\) serves as the input for computing the attention weights \(\gamma_{i}^{m}\), followed by a weighted summation of \(\overline{e}_{m}\) to derive \(\overline{e}_{i}\):  
\[\overline{e}_{i}=\sum_{m\in M}\gamma_{i}^{m}\overline{e}_{m} \tag{14}\]  
To further distinguish the similarities between items across different modalities and facilitate subsequent multimodal fusion, we apply graph convolution to the item-item graph across various modalities established in Section 3.1.2, thereby obtaining a more precise representation of an item within a single modality. The graph convolution executed on the item-item relationship graphs across various modalities is defined as follows:  
\[\tilde{h}_{i,m}^{(l+1)}=\sum_{j\in N_{i}}\bar{g}_{ij}^{m}\tilde{h}_{j,m}^{(l)} \tag{15}\]  
where \(\mathcal{N}_{i}\) denotes the set of all neighboring nodes for item \(i\), \(\tilde{h}_{j,m}^{(l)}\) signifies the node representation of item \(j\) within the graph convolutional network at layer \(l\) in modality \(m\), and \(\tilde{h}_{j,m}^{(0)}\) is derived from Equation (9) as the item modality information \(h_{i,m}\). To fully leverage the complementarity inherent in multimodal features, the subsequent step involves effectively integrating item information from various modalities to derive a unified item representation, thereby facilitating subsequent embedding learning and further uncovering user preferences. Initially, we integrate the item representations \(\tilde{h}_{i,m}\) derived from Equation (15) across different modalities.