# GC4 MRec: Generative-Contrastive for Multimodal Recommendation
###### Abstract
Some methods lack optimization for this drawback, while others utilize different approaches to mitigate the problem; for instance, LightGCN [8] mitigates over-smoothing by linearly reducing the contribution factor of higher-order graph convolution propagation. Recently, contrastive learning (a form of self-supervised learning) has been widely applied in recommender systems [20; 21]. It enhances model representation capabilities by maximizing the similarity between positive samples and minimizing the similarity between negative samples [22; 23]. However, the sparsity of samples and the design of pretext tasks significantly impact the training effectiveness of contrastive learning. To address the aforementioned challenges, we propose a novel multimodal recommendation learning framework. Regarding data sparsity and pretext task setting in contrastive learning, we first project the graph features using a linear layer, followed by graph reconstruction using a Variational Graph Autoencoder (VGAE) [24], resulting in original graph G and augmented graph G'. The original graph and the augmented graph are treated as a pair of positive samples, while others are treated as negative samples. The InfoNCE loss function is employed to optimize the representation learning ability of the linear layer. The introduction of VGAE enhances the model's ability to explore latent user features and narrows the distance in feature space between nodes with similar preferences. To mitigate over-smoothing, we employ cross-residual connections between the input and output layers of the two GCNs. This counteracts the tendency of individual GCNs to produce homogenized representations. Due to the generative capability of VGAE, it can mitigate the over-smoothing effect of GCN. Regarding multimodal fusion, we hypothesize that user preferences exhibit a long-tail effect, meaning that users pay significant attention to a small portion of item features. Therefore, we propose a metadata slicing generation method: we segment the features of users and items into chunks of varying lengths, where each chunk is mapped to a specific attribute entity (color, size, etc. ), referred to as \"metadata\". The length of the metadata reflects the intensity of preference for a particular attribute. We hypothesize that users are typically interested in only a subset of attributes; therefore, we determine the length of metadata by sampling from a Gaussian distribution. This ensures that a small portion of metadata captures the majority of the feature length, thereby effectively identifying the users' primary points of interest. The channel attention mechanism [25], originally used for convolutional graph attention calculation in image processing, is introduced into GC4 MRec for fusing different modal data. In this paper, each metadata point is treated as a channel. First, we pool the metadata, cross-add the modalities, and perform channel-wise convolution to obtain channel weights. The channel weights are multiplied by the original metadata channel to obtain the final metadata value. Finally, the different modal data are summed and averaged to obtain the final modality fusion result. In summary, our contributions are as follows:  
* A novel multimodal learning paradigm employing enhanced self-supervised learning is introduced to uncover latent user preferences and enable effective multimodal fusion. * Latent user preferences are captured more effectively via a dual spatio-temporal cross-residual module, which integrates information from both the original and enhanced graph at various temporal stages. * A metadata-aware fusion method provides a flexible mechanism for fusing multimodal data. * Experimental validation is performed on several public datasets. Following this introductory section, Section 2 reviews the relevant literature on multimodal recommendation, contrastive learning, and generative models, identifying limitations in existing approaches.