# GC4 MRec: Generative-Contrastive for Multimodal Recommendation
## 3 Methodology
### Multimodal Fusion
#### 3.5.1 Metadata and Meta-Preference Modeling  
We focus on the importance of metadata within user preferences or item attributes. As shown in Figure 3, we define \(h^{u}\) and \(h^{i}\) as the metadata chunk features for users and items, respectively, aiming to model atomic user preferences through these metadata blocks, such as material attributes, color characteristics, price features, and more. For instance, this might include a user's preferred color, their favorite product category, or an item's type and size. Naturally, each metadata element warrants varying degrees of attention. User  
Figure 2: Multimodal data-driven user preferences.  
preferences exhibit diverse focuses, while item attributes, representing their selling points, are similarly subject to different levels of consideration. Therefore, we employ a dynamic weighted summation of metadata to capture the latent representations embedded within these interactions. This process is defined as follows:  
\[\begin{array}{l}H=\{h_{1},h_{2},\ldots,h_{|H|}\},\\ h_{i}=\frac{s_{i}}{\sum_{j=1}^{|H|}s_{j}}\cdot d,\\ s_{i}\sim\mathcal{N}(0,1),\quad i=1,2,\ldots,|H|.\end{array} \tag{12}\]  
\(s_{i}\) represents a randomly sampled value from a normal Gaussian distribution with a mean of 0 and a variance of 1. \(|H|\) signifies the number of metadata, which corresponds to the number of samples. \(h_{i}\) represents the length of each metadata point. To isolate these distinct attribute features within each sample and map them to separate vector spaces for processing, we employ multi-head attention. For each metadata index \(i\in|H|\), the corresponding feature \(e_{i}^{m}\) with length \(h_{i}\) is linearly projected into \(|H|\) different subspaces, resulting in \(Q_{i}^{|H|},K_{i}^{|H|},V_{i}^{|H|}\) as follows:  
\[\begin{array}{l}Q_{i}^{|H|}=W_{Q}^{|H|}e_{i}^{m},\\ K_{i}^{|H|}=W_{K}^{|H|}e_{i}^{m},\\ V_{i}^{|H|}=W_{V}^{|H|}e_{i}^{m}.\end{array} \tag{13}\]  
where \(W_{Q}^{|H|},W_{K}^{|H|},W_{V}^{|H|}\) are learnable weight matrices, and \(|H|\) is the number of attention heads. We calculate the attention weights between each metadata point: for each head \(h\), we calculate the attention weight between each metadata point \(i\) and other metadata points \(j\):  
\[Attention(Q_{i},K_{j},V_{j})_{h}=softmax(Q_{i}^{h}{K_{j}^{h}}^{T}/\sqrt{d_{k}} )V_{j}^{h}, \tag{14}\]  
where \(d_{k}\) is the dimension of \(K_{j}^{h}\), and the softmax function is used to normalize the attention weights. We perform a weighted summation of the output for each metadata point: for each head \(h\), we perform a weighted summation of the output \(Attention(Q_{i},K_{j},V_{j})_{h}\) for all metadata \(i\), resulting in the final output for each metadata point \(Attention(Q_{i},K,V)_{h}\):  
\[Attention(Q_{i},K,V)h=\sum_{j=1}^{|H|}Attention(Q_{i},K_{j},V_{j})_{h}. \tag{15}\]  
We integrate the outputs of all metadata, concatenate the outputs of all heads \(h\), and perform a linear projection using a learnable weight matrix \(W^{O}\), resulting in the final output:  
\[e_{i}^{m}=W^{O}[Attention(Q_{i},K,V)_{1},\ldots,Attention(Q_{i},K,V)_{h}]. \tag{16}\]  
\[e^{m}=\left[e_{1}^{m},e_{2}^{m},\ldots,e_{|H|}^{m}\right] \tag{17}\]  
\(e_{i}^{m}\) represents the output of the \(i\)-th metadata point calculated across \(|H|\) attention heads, while \(e^{m}\) denotes the final output after processing all metadata.