Attribute-driven Disentangled Representation Learning for Multimodal Recommendation

Venue: acmmm_org_ACMMM_2024_Conference
Published: 2024-04-06 17:48:13

Abstract: Recommendation algorithms forecast user preferences by correlating user and item representations derived from historical interaction patterns. In pursuit of enhanced performance, many methods focus on learning robust and independent representations by disentangling the intricate factors within interaction data across various modalities in an unsupervised manner. However, such an approach obfuscates the discernment of how specific factors (e.g., category or brand) influence the outcomes, making it challenging to regulate their effects. In response to this challenge, we introduce a novel method called Attribute-Driven Disentangled Representation Learning (short for AD-DRL), which explicitly incorporates attributes from different modalities into the disentangled representation learning process. By assigning a specific attribute to each factor in multimodal features, AD-DRL can disentangle the factors at both attribute and attribute-value levels. To obtain robust and independent representations for each factor associated with a specific attribute, we first disentangle the representations of features both within and across different modalities. Moreover, we further enhance the robustness of the representations by fusing the multimodal features of the same factor. Empirical evaluations conducted on three public real-world datasets substantiate the effectiveness of AD-DRL, as well as its interpretability and controllability.

Main paper: 
 related work 2.1 multimodal collaborative filtering: 
 Traditional Collaborative Filtering (CF) [4, 27, 28] methods primarily rely on user-item interactions to learn representations of users and items. Consequently, the recommendation performance is negatively impacted when encountering users and items with limited interactions. To mitigate the challenges posed by data sparsity [14, 18, 22, 29] , recent research has incorporated multimodal information into recommendation systems. The multimodal features, such as reviews and images, provide valuable information for user preference and item characteristics, which can supplement historical user-item interactions and thus enhance recommendation performance [8, 17, 19, 37, 38, 41] . Previous studies integrate multimodal information into the matrix factorization-based method in a straightforward manner. For instance, VBPR [8] directly concatenates the visual features learned from item images with collaborative features as the joint item representation and feeds it into the MF module. With the success of deep learning techniques in modeling complex interaction behaviors [10] and the relation between various multimodal features [24] , a lot of deep learning techniques have been introduced into multimodal recommendation [10, 30] . For example, MAML [19] first fuses the item's multimodal features and user features, then feeds it into an attention module to capture users' diverse preferences. VECF [1] constructs a visually explainable collaborative filtering model using a multimodal attention network, facilitating the integrated coupling of diverse feature modalities. More recently, graph convolutional networks (GCNs) [7] have demonstrated their power capability of representation learning for recommendation [9, 34, 36, 43] . Based on the GCN structure, MMGCN [38] constructs a user-item bipartite graph to learn representations of each modality and then fuse the modality representation together as the final representation. GRCN [37] utilizes the rich multimodal content of items to refine the structure of the interaction graph in order to mitigate the effect of false-positive edges on recommendation performance.Disentangled representation learning, which seeks to identify and separate underlying explanatory factors within data, has garnered significant interest, especially in the field of computer vision [5, 11, 20] . For instance, the beta-VAE method [11] employs a constrained variational framework to learn disentangled representations of fundamental visual concepts. Meanwhile, IPGDN [20] automatically uncovers independent latent factors present in graph data.Owing to the successful application of disentangled representation learning in various domains, numerous studies have concentrated on learning disentangled representations for users and items in recommendation systems in recent years [17, 21, 23, 31, 33, 35] . Following the previous work in computer vision, the initial attempts take Variational Auto-encoder (VAE) [16] to learn disentangled representations. For example, MacridVAE [21] captured user preferences regarding the different concepts associated with user intentions separately. Beyond the disentangled representations derived from user-item interaction modeling, ADDVAE [31] acquires an additional set of disentangled user representations from textual content and subsequently aligns both sets of representations. For studying the diversity of user intents on adopting the items, DGCF [35] employs a graph disentangled module to iteratively refine the intent-aware interaction graph and factorial representations for recommendations. In order to model the users' various preferences on different factors of each modality, DRML [17] estimates the user's attention weight to underlying factors of different modalities with an attention module. As the learned disentangled representations lack clear meanings, KDR [23] harnesses the power of knowledge graphs (KGs) to guide the disentangled representation learning process, ensuring that the resulting disentangled representations are associated with semantically meaningful information extracted from the KGs.Although the existing models achieve performance improvement by disentangled representation learning, they all face the problem that they all disentangle the user and item representation into latent factors without clarifying the semantic meaning of each factor. 
method 3.1 preliminaries: 
 3.1.1 Problem Setting. Given a set of users U ∈ {𝑢} and items I ∈ {𝑖}, we utilize two types of information to learn user and item representations: (1) A user-item interaction matrix 𝑹, where each entry 𝑟 𝑢𝑖 ∈ 𝑹 represents the implicit feedback (e.g., clicks, likes or purchases) of user 𝑢 to item 𝑖. 𝑟 𝑢𝑖 = 1 indicates that user 𝑢 has interacted with item 𝑖 and 𝑟 𝑢𝑖 = 0 indicates that there is no interaction between user 𝑢 and item 𝑖 in the observed data; and (2) Multimodal features, which mainly consist of two types of features associated with items, namely reviews and images. By utilizing user-item interactions and multimodal features, we aim to learn robust representations of users and items, thereby predicting a user's preference for items they have not yet interacted with.3.1.2 Notations. Similar to previous work [17, 37] , each user 𝑢 and item 𝑖 are assigned with a unique ID and respectively represented by a vector 𝒗 𝑢 ∈ R 𝑑 and 𝒗 𝑖 ∈ R 𝑑 , which are randomly initialized in our model. For the review and image information, we use the BERT [2] and the ViT [3] to extract the raw textual features 𝒆 𝑡 ∈ R 𝑑 0 and visual features 𝒆 𝑣 ∈ R 𝑑 0 , respectively. To tailor the recommendationoriented features, we adopt two non-linear transformations to cast 𝒆 𝑡 and 𝒆 𝑣 into the same feature space:EQUATIONwhere 𝑾 𝑡 , 𝑾 𝑣 ∈ R 𝑑 ×𝑑 0 and 𝒃 𝑡 , 𝒃 𝑣 ∈ R 𝑑 denote the weight matrices and bias vectors for textual and visual modality, respectively. 𝜎 (•) is the activation function.Following the disentangled representation learning process in previous work [17, 35] , we first split the feature vector into 𝐾 chunks. Each chunk corresponds to a specific item attribute, such as price, brand, etc. For simplicity, we equally split the representation of each modality into 𝐾 continuous chunks. Take item ID embedding as an example:EQUATIONwhere𝒗 𝑘 𝑖 ∈ R 𝑑 𝐾 is the item ID embedding corresponding to the 𝑘-th attribute. Analogously, 𝒗 𝑡 = (𝒗 1 𝑡 , 𝒗 2 𝑡 , • • • , 𝒗 𝐾 𝑡 ), 𝒗 𝑣 = (𝒗 1 𝑣 , 𝒗 2 𝑣 , • • • , 𝒗 𝐾 𝑣 ) and 𝒗 𝑢 = (𝒗 1 𝑢 , 𝒗 2 𝑢 , • • • , 𝒗 𝐾 𝑢 )are defined as the embeddings of textual feature, visual feature and user ID embedding, respectively.Our work not only aims to recommend accurate items to users but also advocates for attribute-driven disentanglement in multimodal recommender systems to enhance the interpretability and controllability of recommendations. Specifically, unlike the existing method that disentangles the latent factors in an unsupervised manner, we leverage the semantic labels of item attributes to learn attribute-specific subspaces for each attribute to obtain disentangled representations. In this paper, the vector of each modality is composed of 𝐾 chunks, with the assumption that each chunk is associated with an attribute (such as price or brand). The achievement of attribute-driven disentangling requires two necessary conditions. Firstly, each chunk should have a clear attribute reference, and chunks associated with different attributes should be distinguishable. For example, chunk 𝑗 is associated with the price, while chunk 𝑘 represents the vector of the brand. Secondly, each chunk of a specific attribute should accurately capture the specific attribute value. For example, chunk 𝑗 should reflect the price level (i.e., expensive or cheap) of the product.In this section, we elaborate on our proposed model, termed AD-DRL, an acronym for Attribute-Driven Disentangled Representation Learning model. We aim to improve the interpretability and controllability of recommendation models by assigning a specific attribute to each factor in multimodal features. Specifically, AD-DRL comprises two disentangling modules: high-level and low-level attribute-driven disentangled representation learning. In the highlevel attribute-driven disentangled representation learning module, we exploit the difference between attribute factors within the same modality feature and the consistency of the same attribute factor across different modalities. In contrast, the low-level attribute-driven disentangled representation learning module leverages the intrinsic relationships between items sharing the same attribute value.To obtain robust and independent representations for each attribute factor in multimodal features, AD-DRL enables disentangling within each modality feature and ensures consistency of representations across modalities. Next, we detail the intra-modality disentanglement and inter-modality disentanglement.Intra-Modality Disentanglement. After obtaining the feature vectors of each modality, we split these vectors into several chunks. Nevertheless, different attribute factors are still entangled within the chunks. In other words, a chunk may contain both priceand brand-related features. Therefore, in order to disentangle attribute factors in each modality feature, we employ an attribute classifier to encourage each chunk to predict the corresponding attribute, as shown in Figure 1a . Taking the 𝑘-th chunk 𝒗 𝑘 𝑡 of item textual embedding as an example,EQUATION)where 𝑾 𝑖𝑛𝑡𝑟𝑎,𝑡 ∈ R 𝐾 × 𝑑 𝐾 and 𝒃 𝑖𝑛𝑡𝑟𝑎,𝑡 ∈ R 𝐾 are the weight matrix and bias vector of the classifier, respectively. 𝒛 𝑘 is the predicted logits and 𝑧 𝑘 𝑛 ∈ 𝒛 𝑘 . z𝑘 𝑛 denotes the ground truth (attribute label) of chunk 𝒗 𝑘 𝑡 . For all the chunks in textual embedding 𝒗 𝑡 , we have the following loss for all attribute factors in textual features: Similarly, we can define the losses 𝑙 𝑖𝑛𝑡𝑟𝑎,𝑢 , 𝑙 𝑖𝑛𝑡𝑟𝑎,𝑖 and 𝑙 𝑖𝑛𝑡𝑟𝑎,𝑣 to encourage the features of each attribute factor to be concentrated in the corresponding chunk of user ID, item ID and visual feature, respectively. Finally, the total loss for the intra-modality disentanglement is formulated as:EQUATIONe t v 1 t v 2 t v K t v 2 t v 1 v v 1 t v 2 v v K v v K t v 1 i v 1 t v 1 v v 2 i v 2 t v 2 v v K i v K t v K v v 1 y v 2 y v K yEQUATIONInter-Modality Disentanglement. In addition to separately disentangling each modality, a serious challenge in disentangling representation learning for multi-modal features is handling the inter-relationship between the factors disentangled from multiple modalities. Intuitively, the chunks of the same attribute factor from different modality features should be consistent. For example, for the brand attribute, an item should have the same brand information in both the visual and textual features. In other words, chunks that share the same attribute across different modalities should be highly similar, while chunks that represent different attributes across modalities should be dissimilar.To further achieve robust representation in multimodal recommendations, we disentangle attribute factors in different modality features by ensuring consistency of the same attribute across modalities. Inspired by this, we design a cross-modal contrastive loss as shown in Figure 1b . Specifically, for any two modality representations, such as 𝒗 𝑡 and 𝒗 𝑣 , we take two chunks of the same attribute factor (i.e., (𝒗 𝑘 𝑡 , 𝒗 𝑘 𝑣 )) as positive pairs, and two chunks corresponding to different attributes (i.e., (𝒗 𝑘 𝑡 , 𝒗 𝑛≠𝑘 𝑣 ), (𝒗 𝑛≠𝑘 𝑡 , 𝒗 𝑘 𝑣 )) as negative pairs. After that, the cross-modal contrastive loss between textual and visual features is defined as follows:EQUATIONwhere • represents the dot product and 𝜏 ∈ R + is a scalar temperature parameter. By applying this contrastive learning constraint to any two out of the three modalities, we can achieve disentanglement and alignment across features of various modalities: ), where 𝐴 𝐾 is the total number of possible values for that attribute. For example, the attribute value of popularity can be stratified into five distinct levels: Super Popular, Popular, Moderate, Emerging, and Unknown. The representation disentanglement could benefit from the attribute values by exploiting their relationships of the same factor. Therefore, to learn more robust representations, we encourage disentangled representations based on attributes to predict specific attribute values of the item.EQUATIONIt is obvious that any single modality may not contain sufficient information for a particular attribute. For example, images may more intuitively reflect attributes such as color and brand of an item, but may not directly reflect attributes such as price or popularity. Therefore, in order to comprehensively and accurately depict the attribute values of the item, we first integrate the features from all modalities together. To achieve this, we apply a multimodal attention mechanism to measure the emphasis of different modalities on different attributes. Specifically, for the 𝑘-th attribute, the attention weights assigned to different modalities are estimated by a two-layer neural network:EQUATIONwhere 𝑾 𝑎1 ∈ R 3× 𝑑 𝐾 and 𝑾 𝑎2 ∈ R 3×3 are the weight matrices corresponding to the first and second layers of the neural network, respectively. 𝒃 𝑎 denotes the bias vector and tanh is the activation function. 𝑆𝑜 𝑓 𝑡𝑚𝑎𝑥 is adopted to normalize â𝑘 to a probability distribution.Thereafter, we obtain the final representation of the 𝑘-th attribute factor for the item as follows: where 𝑎 𝑘 𝑖 , 𝑎 𝑘 𝑡 , 𝑎 𝑘 𝑣 ∈ 𝒂 𝑘 are the attention weights of item ID embedding, textual feature and visual feature, respectively.EQUATIONSimilar to the disentangling at the intra-modality disentanglement in Section 3.2.1, we aim to achieve disentangled representation learning at the low level through attribute value prediction via a classification layer (as shown in Figure 1c ):EQUATION)where 𝑾 𝑘 ∈ R 𝐾 × 𝑑 𝐾 and 𝒃 𝑘 ∈ R 𝐾 are the weight matrices and bias vector of the classifier, respectively. We supervise the training of each attribute subspace via independent attribute classification tasks defined in the form of a cross-entropy loss.𝑢 = (𝒗 1 𝑢 , 𝒗 2 𝑢 , • • • , 𝒗 𝐾 𝑢 ) and 𝒗 𝑦 = (𝒗 1 𝑦 , 𝒗 2 𝑦 , • • • , 𝒗 𝐾 𝑦 )for the user 𝑢 and item 𝑖, respectively. By assigning each disentangled representation with an attribute as described above, we are able to predict users' preferences at the attribute level. This enhances the interpretability and controllability of our model. Specifically, to estimate a user 𝑢's preference for an item 𝑖, it is crucial to consider her preference for each attribute of the item. To achieve this, we first compute the user's preference score for each attribute of the item, and subsequently, integrate the scores of each attribute to estimate the user's overall preference for the item:EQUATIONwhere • symbolizes the dot product and 𝜎 (•) denotes the activation function. In this paper, we use a softplus function to ensure the resultant score 𝑠 𝑢,𝑖,𝑘 is positive. 𝑠 𝑢,𝑖,𝑘 denotes the user 𝑢's preference score for the attribute 𝑘 of the item 𝑖.Based on the predicted preference score above, we recommend a list of top-𝑛 ranked items that match the target user's preferences. The Bayesian Personalized Ranking (BPR) loss function [27] is employed to optimize the model parameters 𝚯,EQUATION)where 𝜆 is the coefficient controlling 𝐿 2 regularization; D denotes the training set; 𝑖 + and 𝑖 -are the observed and unobserved items in the interaction records of user 𝑢, respectively. Overall, the total loss of AD-DRL is formulated as,EQUATIONwhere 𝛼, 𝛽 and 𝛾 are the hyperparameters to control the weight of three disentanglement modules. 
experiments 4.1 experimental setup: 
 4.1.1 Datasets. We use the widely adopted real-world recommendation dataset, the Amazon review dataset1 [22] , for evaluation in our experiments. Apart from user-item interaction data, this dataset also includes multimodal information (i.e., reviews and images) and various attributes (i.e., price, brand, etc.) of the items on 24 product categories. Three product categories from this dataset are used in the evaluation: Baby, Toys Games and Sports. Following [17] , for all datasets, unpopular items and inactive users are filtered out to ensure that all the items and users have at least 5 interaction records. The basic statistics of the three datasets are shown in Table 1 .In our setting, in addition to the user-item interaction data and multimodal information of items, multiple attributes and their associated attribute values of items are required to guide the disentangled representation learning process. Specifically, we adopt four typical attributes (i.e., price, popularity2 , brand and category), and the attribute values for each attribute are compiled based on the metadata provided by the Amazon dataset: for brand and category values, we directly used the values provided by Amazon; for price and popularity, following the methods used by CoHHN [40] , we discretize them into five levels according to their numerical values. Table 1 shows the specific statistical information for each attribute in our experiments. It is worth noting that our method can include a wide range of attributes that objectively reflect the character of an item, beyond the four attributes mentioned.4.1.2 Baselines. We compared our method with the state-of-theart methods, including both the CF-based methods (i.e., NeuMF [10] , NGCF [34] and DGCF [35] ) and Multimodal CF-based methods (JRL [42] , MMGCN [38] , MAML [19] , GRCN [37] , DMRL [17] and BM3 [43] ). Each type of method includes a disentangled representation learning model (i.e., DGCF and DMRL) for comparison. In addition to the methods mentioned above, we also created a variant of our model, denoted as AD-DRL 𝐼 𝐷 , which excludes the utilization of multimodal information, thus facilitating a fair comparison with CF-based models.For each dataset, we randomly split the interactions from each user with an 8 : 2 ratio to construct training and testing sets. From the training set, 10% of interactions are randomly chosen as a validation set for tuning hyperparameters. We evaluate performance on the top-𝑛 recommendation task, aiming to recommend the top-𝑛 items, using Recall@𝑛 and NDCG@𝑛 as metrics for accuracy, with 𝑛 defaulting to 20.The Pytorch toolkit [25] is utilized to implement our models. To ensure fairness, all methods are optimized using the Adam optimizer [15] , with a default learning rate of 0.0001 and batch size of 1024. We fixed the embedding size of each factor to 32 for AD-DRL and its variants on all datasets. The Xavier initializer [6] is used to initialize the model parameters. The 𝛼, 𝛽, 𝛾 and 𝐿 2 regularization coefficient are searched in{1𝑒 -3 , 5𝑒 -3 , 1𝑒 -2 , • • • , 5𝑒 +0 , 1𝑒 +1 }.The number of negative examples is searched in {2, 4, 8}. Besides, The symbol * denotes that the improvement is significant with 𝑝 -𝑣𝑎𝑙𝑢𝑒 < 0.05 based on a two-tailed paired t-test.model parameters are saved every five epochs. Early stopping strategy [34] is performed, i.e., premature stopping if Recall@20 does not increase for 50 successive epochs.We summarize the overall performance comparison results in Table 2. The methods in the first block solely use the user-item interactions, while the methods in the second block also exploit textual and visual information along with the user-item interactions. From this table, we can have the following observations:• NeuMF, NGCF, DGCF and AD-DRL 𝐼 𝐷 are methods trained exclusively on user-item interactions. Among them, NeuMF outperforms traditional MF-based methods [10] , benefiting from deep neural networks' ability to model the non-linear interactions between users and items. NGCF and DGCF achieve state-of-the-art results by utilizing high-order information, with DGCF excelling over NGCF by leveraging disentangled representation to capture diverse user intents, showcasing its effectiveness in enhancing the robustness of user and item representations. Furthermore, although both DGCF and AD-DRL 𝐼 𝐷 employ disentangled representation learning, the performance of AD-DRL 𝐼 𝐷 is significantly better than that of DGCF. This highlights the superiority of our attribute-driven disentangled representation learning approach. • In general, the multimodal recommendation methods perform better than those only using user-item interactions, demonstrating the effectiveness of leveraging multimodal information on learning user and item representations. Although the simple neural network structure results in lower performance of JRL compared to the graph-based methods (NDCG and DGCF), incorporating rich multi-modal information enables it to outperform NeuMF. By exploiting user-item interactions to guide the representation learning in different modalities, MMGCN yields better performance over NGCF on all the datasets. MAML outperforms NGCF by modeling users' diverse preferences by using multimodal features of items. GRCN surpasses MAML by employing modality features to discover and prune potential false-positive edges on the user-item interaction graph. DMRL captures the different contributions of features from diverse modalities for each disentangled factor, achieving superior results. • Furthermore, our proposed AD-DRL model consistently outperforms all baselines over all three datasets by a large margin. We credit this to the joint effects of the following two aspects. Firstly, robust user and item representations can be derived using attributes at different levels to guide the disentangled representation learning process. Secondly, incorporating the additional attribute information into representation learning can alleviate the data sparsity problem in the recommendation.AD-DRL conducts disentangled representation learning for user and item representations at two levels of attribute granularity: high and low. To further confirm the effectiveness of disentangled representation learning by AD-DRL, we use t-SNE [32] to cluster and visualize the disentangled vectors of users and items from the Sports dataset at each granularity level. 9) in Figure 3 . The dots of different colors represent different attribute values 3 . As observed, the representations of different attribute values learned from AD-DRL are well separated and the representations of the same attribute value are concentrated. These results demonstrate that AD-DRL can achieve finer-grained disentanglement at the attribute value level, allowing AD-DRL to capture better user preferences.To gain a deeper insight into the interpretability of our model, in this section, we provide some qualitative examples in Figure 4 . As an illustrative example, we randomly sample two users (u18629 3 For brand and category, since there are too many attribute values in the dataset, making it difficult to display all of them in Figure 3 . Therefore, for these two attributes, we only selected the top 5 attribute values with the most corresponding items in the dataset for demonstration. and u6805) from the Sports dataset who purchased the same items (i2099 and i3460). Since we have disentangled the representation of users and items based on the attributes, we can analyze how each attribute contributes to recommendation results based on Equation 11. This greatly enhances the interpretability of our model. For example, price contributes most to the interaction (u10826, i2099). This suggests that u10826 prefer i2099 due to its price. However, u6805 is more likely to purchase the i2099 because of her preference for its brand. Such observation indicates that different users have diverse preferences for the same product. In addition, we can also see that user preferences exhibit consistency. For example, u6905 values the brand of the product more than its popularity or price, which is different from u10826.In this section, we evaluate the controllability of AD-DRL by manipulating user preferences for certain attributes. Specifically, we change the user 𝑢's preference score for a specific attribute 𝑎 in Equation 11to assess if such change yields the anticipated variation in recommendation results. We modify the formula for 𝑠 𝑢,𝑖 in Equation 11 as follows:EQUATIONwhere 𝜉 represents the scaling factor for 𝑠 𝑢,𝑖,𝑎 , used to adjust the impact of attribute 𝑎 on 𝑠 𝑢,𝑖 . For a specific attribute (in this section, we selected price and popularity for study), we systematically vary 𝜉 through values of 2, 1, 0.5, 0, and -1. This process allows us to meticulously examine the resultant shifts in our method's recommendation outputs, with findings illustrated in Figure 5 . We conduct a detailed analysis using the price attribute as an example. As shown in Figure 5a , we select 100 users from Baby dataset who always purchase inexpensive items based on their purchase records. Various colors in Figure 5a denote distinct price levels, showcasing the distribution of items recommended by AD-DRL across these levels for a given value of 𝜉. By comparing the recommendations made by AD-DRL under different values of 𝜉, we have the following observations: • When 𝜉 = 1, that is, using the original AD-DRL method, it can be seen that AD-DRL is capable of capturing the preference of these users for inexpensive items and recommends more affordable items to them. • When 𝜉 = 2, meaning we have increased the impact of price on the user preference score, it can be observed that AD-DRL tends to recommend more inexpensive items to these users. • When 𝜉 = 0.5 or 0, meaning we reduce or even eliminate the impact of price on the user preference score, we find that the recommended items from AD-DRL are more diverse in terms of price. It is worth noting that even when 𝜉 = 0, the items recommended by AD-DRL still tend to be inexpensive. This could be attributed to the retained chunk associated with the brand, and there are correlations between brand and price. For example, some high-end brands offer products at a premium price, while others provide more affordable options. • More interestingly, when 𝜉 = -1, meaning we have AD-DRL recommend items that are opposite to the users' price preferences, it can be observed that AD-DRL indeed recommends some more expensive items.The above results reveal that adjusting the value of 𝜉 enables us to align AD-DRL's recommendation results with our expectations, thereby illustrating the controllability of AD-DRL. Similar results can be seen in another group of users who prefer high-price level items. To validate the effects of the key components in AD-DRL, we set up the following model variants: 1) AD-DRL 𝑤/𝑜 disentangling : we do not perform disentangled representation learning during the training process; 2) AD-DRL 𝑤/𝑜 intra : we only remove the intra-modality disentanglement module; 3) AD-DRL 𝑤/𝑜 inter : we only remove the inter-modality disentanglement module; 4) AD-DRL 𝑤/𝑜 high : we remove the high-level attribute-driven disentangled representation learning module; 5) AD-DRL 𝑤/𝑜 low : we remove the low-level attribute-driven disentangled representation learning module. Table 3 shows the experimental results for all variants. From the results, we have the following observations:• Using either high-level or low-level disentangled representation learning independently can significantly improve the performance. This indicates that both disentangled representation learning within modalities and between modalities have a positive effect. • The AD-DRL combines high-level and low-level disentangled representation learning modules, resulting in significantly improved performance compared to the devised two variants. This demonstrates the necessity and rationality of combining the attribute and attribute value levels. • Delving deeper into the high-level disentangled representation learning module, removing the intra-modality disentanglement module results in a greater decrease in model performance compared to removing the inter-modality disentanglement module. This indicates that intra-modality disentanglement is the fundamental aspect of disentangled representation learning, while inter-modality disentanglement can further enhance model performance. 
conclusion: 
 In this paper, we highlight the limitations of existing disentangled representation learning techniques in recommender systems. The current methods disentangle the underlying factors behind useritem interactions in an unsupervised manner, leading to limited interpretability and controllability. To overcome this limitation, we propose an attribute-driven disentangled representation learning method, termed AD-DRL, which disentangles attribute factors in various multimodal features. More specifically, the proposed AD-DRL enables disentangling within each modality feature and ensures consistency of representations across modalities at the attribute level. Moreover, it leverages the intrinsic relationships between items sharing the same attribute value. Experimental results demonstrate the superiority of our proposed AD-DRL and showcase its capability in terms of interpretability and controllability.  
: 
 these models can easily encounter the sparsity problem in practical scenarios due to their exclusive dependence on interaction data. To alleviate the data sparsity problem in recommendation, side information (e.g., attributes, user reviews and item images), which contains rich information associated with users or items, is often used to enhance the representation learning of users and items [17, 19, [37] [38] [39] 41] .It is well recognized that the entangled representations of users and items are infeasible to directly capture fine-grained user preferences across diverse factors, thereby constraining both the efficacy and interpretability of recommender systems [21, 35] . In recent years, the study of disentangled representation learning has garnered significant attention in diverse fields, notably in computer vision [11, 13, 20] , due to its capability to identify and disentangle the underlying factors behind data. Empirical evidence suggests that disentangled representations exhibit enhanced robustness, particularly in complex application contexts. Hence, many recommendation methods adopt disentangled representation learning techniques to learn robust and independent representations, ultimately enhancing recommendation performance [17, 21, 23, 31, 35] . For example, Ma et al. [21] employed disentangled representations to capture user preferences regarding different concepts associated with user intentions. Wang et al. [35] introduced a GCN-based model that produces disentangled representations by modelling a distribution over intents for each user-item interaction, exploring the diversity of user intentions on adopting items. However, these methods only concentrate on disentangling the user and item representations based on their ID embeddings. To exploit the difference between the factors behind data of various modalities, Liu et al. [17] estimated the users' attention weight to underlying factors of different modalities, utilizing a sophisticated attention-driven module.Despite the considerable advancements brought about by disentangling techniques in recommender systems, existing studies often disentangle both users and items into latent factors. This methodology inherently constrains the model's interpretability and controllability. For example, consider using disentangled representations to elucidate the diverse factors, such as style, brand, popularity, and price, that shape user preferences in dress selection. The inherent abstraction of latent factors from current disentangling methods makes it difficult to pinpoint which factor represents each specific dress attribute. In particular, one factor might be loosely related to a combination of brand and price, while another factor might represent a mixture of style and popularity. This ambiguity in the latent factors limits the interpretability of the recommendation system, making it difficult to understand why a particular item was recommended to a user. Moreover, this lack of clarity also affects the controllability of the recommendation system. Suppose a user wants to receive recommendations specifically for dresses without considering her preferences for price and popularity. It would be challenging to adjust the recommender system to focus on those specific preferences, as the latent factors are not clearly tied to these attributes.Item attributes manifested in various modalities can enrich the recommendation system by offering diverse and complementary information. For example, the textual data might explicitly mention the brand or price, while visual data can reveal visual attributes, such as the category of items. Additionally, the popularity of items can be derived from the statistical information of interaction data. In fact, attributes represent specific, meaningful properties or characteristics of items. Consequently, using attributes to guide the disentanglement process is a promising way to improve the interpretability and controllability of conventional multimodal recommendation methods. In this paper, we propose an Attribute-Driven Disentangled Representation Learning method (AD-DRL for short), which disentangles factors in user and item representations across various modalities at different levels of attribute granularity. To obtain robust and independent representations for each factor associated with a specific attribute, we first disentangle the representations of features within and across different modalities. This process is guided by high-level attributes (e.g., category and popularity), which help reveal the underlying relationships between factors. Following this, we further enhance the robustness of the representations by fusing the multimodal features of the same factor. This step involves exploiting the relationships among representations of the same factor by leveraging low-level attributes (e.g., category attribute values for a clothing dataset, such as jeans, jackets, and dresses), resulting in finer-grained and more comprehensive disentangled representations. To validate the effectiveness of our method, we conduct extensive experiments and ablation studies on three real-world datasets. Experimental results demonstrate the superiority of our method AD-DRL compared to existing methods and showcase its capability in terms of interpretability and controllability.In summary, the contributions of this paper are threefold:• In this paper, we highlight the limitations of traditional disentangled representation learning in multimodal recommendation systems. To overcome these shortcomings, we introduce AD-DRL, which improves interpretability and controllability by employing attributes to disentangle factors in user and item representations. • To achieve robust and independent representations, we assign a specific attribute to each factor in multimodal features and disentangle factors at both the attribute and attribute-value levels. • We conduct extensive experiments on three real-world datasets to validate the effectiveness of our method. The experimental analysis demonstrates the interpretability and controllability of our model. 
 
 Appendix: http://jmcauley.ucsd.edu/data/amazon.Popularity is calculated by counting the number of times each item is purchased. 


Decision: This paper is well-organized and achieves high performance. Although one reviewer gives a borderline rejection score, this paper gets overall positive scores and three reviewers raise their scores after the rebuttal phase. To this end, I recommend accepting this paper.

Reviews: review: paper_summary: paper_summary: This paper introduces a disentangling method, which attempts to make learned representations includes information about the item features explicitly. Specifically, this paper aligns the same information within and between modalities based on the self-supervised loss. In addition, this paper uses attention mechanism to adjust the weights of different modal embedding. Finally, this paper conducts experiments on real-world datasets, and the experimental results demonstrate that the proposed method outperforms existing baseline methods.
strength_weakness: strength_weakness: 1. This paper is well organized and makes it easy for the reader to understand the core ideas.

2. The experimental results demonstrate that the proposed method outperforms existing baseline methods.
limitations: limitations: 1. If the attributes of two items are very similar, why they need to be treated as the negative pair in the inter-modality disentanglement part?

2. Does it make sense to split embedding into chucks and then align them? For example, if the embedding itself doesn't include information (e.g., an embedding generated from background in an image or a non-solid word in text), does such an embedding have the ability to predict attributes?

3. How do we verify the correspondence between the learned representation and the attributes? In addition, how to ensure that the embedding explicitly learns information about features?

4. Additional sensitivity analysis experiments are needed for $\alpha$, $\beta$, $\gamma$, and a scalar temperature parameter.
paper_summary: paper_summary: This paper introduces a novel method called Attribute-Driven Disentangled Representation Learning (AD-DRL), which explicitly incorporates attributes from different modalities into the disentangled representation learning process. Specifically, the article divides the representations of users, items, texts, and images into multiple chunks, each chunk corresponding to a specific item attribute. In each module, this paper uses a self-supervised loss function for alignment. Experiments demonstrate the effectiveness of the proposed method.
strength_weakness: strength_weakness: S1: This paper attempts to add interpretability to the disentangled features, which is an interesting topic.

S2: The organization of the paper is clear and makes it easy to understand.

S3: The paper provides many experimental details to enhance the reproducibility.
limitations: limitations: W1: In Section 3.1, the authors say that the data in this paper includes multimodal features and a user-item interaction matrix, do we need additional information about the item's features such as price and brand?

W2: In the inter-modality disentanglement module, take textual data as an example, if two items have the same attributes, why the textual embedding of these two items is regarded as the negative pair?

W3: In the case of textual information, for example, the embedding of each passage might have a specific meaning. For example, suppose a user evaluates an item “I like this watch because it's a red color” or, “I like this watch because it's a cheap price”. At this point, it doesn't make sense to divide the text embedding into several chunks so that each chuck can predict the attribute. If you do so, maybe the embedding of the "red color" or the "cheap price" will predict the brand. The same thing holds for the image information.

W4: Are the results presented in Figure 4 verifiable? How to conclude that the contributions of each attribute are correct?

W5: How stable is the ranking order for features? For example, does the ordering of “price, popularity, brand” and the ordering of “brand, price, popularity” have any effect on the results? Because the interpretation of embedding will change at this point.

I'd like to modify my rating based on the rebuttal.
paper_summary: paper_summary: This paper introduces a novel method, Attribute-Driven Disentangled Representation Learning (AD-DRL), to address challenges in recommendation algorithms that predict user preferences based on historical data. Traditional approaches often struggle to discern the influence of specific factors, like product categories or brands. AD-DRL explicitly incorporates attributes from various modalities into the learning process, allowing for the disentanglement of factors at both attribute and attribute-value levels. This method enhances the interpretability and robustness of the system. Empirical evaluations on three public datasets demonstrate AD-DRL's effectiveness and improved control over recommendation outcomes, proving it a significant advancement in recommendation system technology.
strength_weakness: strength_weakness: 1.The paper is well-structured and concise without sacrificing necessary detail, making it easy to follow and understand the key contributions and findings.
2.The proposed AD-DRL, involving multi-level and multi-modal disentanglement, can improve the clarity and usability of disentangled representations in recommendation systems, thereby enhancing both the interpretability and the controllability of the items provided to the user.
3.The descriptions of the datasets and baselines are thoroughly detailed, and the experimental setup is appropriately structured and reasonable.
limitations: limitations: 1.Although AD-DRL is able to enhance the performance of existing models, there is no provided code/implementation link to reproduce the results, making it impossible to use as a baseline for future methods.
2.In the Interpretability Study section, the use of merely two examples to demonstrate the advantages of AD-DRL in terms of interpretability is not convincing.
3.In terms of Interpretability and Controllability, the manuscript does not compare the proposed method with baseline models, which is a necessary analysis to establish the efficacy and advancements of the new approach over existing methods.

1.Indicator evaluations or a substantial number of case studies are required for a robust Interpretability Study.
2.It is crucial to include a comparative analysis with established baseline models to validate the claims regarding the interpretability and controllability of the proposed AD-DRL model, presenting both quantitative metrics and qualitative insights to demonstrate its efficacy.
paper_summary: paper_summary: The paper introduces AD-DRL, a method for improving multimodal recommendation systems by using attribute-specific disentangling of user and item features, which enhances both interpretability and recommendation performance.
strength_weakness: strength_weakness: 1. The paper extends existing disentangled representation learning approaches by incorporating both high-level and low-level attribute disentanglements. This dual-level disentanglement is designed to enhance the robustness and independence of feature representations, thereby improving the system's performance.
2. The evaluation of AD-DRL is thorough, with extensive experiments conducted on three real-world datasets to demonstrate its superiority over existing methods.
limitations: limitations: 1. In Sec. 3.1, you mention the use of a user-item interaction matrix and multimodal features for learning robust representations. However, how do you ensure that the model does not overfit to specific attributes of user-item interactions, especially in sparse data scenarios?
2. In Sec. 3.2, it's stated that the AD-DRL model disentangles factors at both attribute and attribute-value levels. Could the overlapping of attributes in multimodal data lead to ambiguous or conflicting representations?
3. In Sec. 4, the empirical results show improvements over traditional methods. However, how do you address the variability in results that might stem from different dataset characteristics or external factors affecting the datasets?
4. Given the complexity introduced in the model by handling both high-level and low-level disentanglement, as discussed in Sec. 3.2.2, what is the computational overhead of AD-DRL compared to standard multimodal recommendation systems?
5. In Sec. 4.2, you present a comparison of AD-DRL with several baseline models. How does the performance of AD-DRL scale with increasing data dimensions and what are the implications for larger, more diverse datasets?
6. Considering the attribute-driven approach outlined in Sec. 3.2.1, is there a potential for the model to propagate or amplify biases present in the attribute data, and how might this impact the fairness of the recommendations?