Paper: Disentangled Heterogeneous Collaborative Filtering (ICLR-2024)

Reviewer comment (Experiments):
The model consists of multiple components, which are difficult to optimize and converge. It is hard to be applied in the real-world case.

Author response:
1. The practicality of the proposed DHCF model (W1)

- Model Efficiency. In our efficiency study, as presented in Section 4.6 and Table 2, we demonstrate that the per-epoch training and testing time of DHCF is comparable to existing baseline models. This indicates that DHCF is applicable in real-world scenarios and is efficient in terms of computational resources.

- Model Convergence. Regarding the convergence of the model, our DHCF follows the same maximum training epochs as other baseline methods, which is set to 100 epochs. This suggests that there are no disadvantages in terms of convergence speed for DHCF compared to other models.

Reviewer comment (Evaluation):
The robustness analysis is not convincing. To demonstrate the model's robustness, we expect it to achieve stable performance on difficult tasks, where baseline methods perform poorly compared to easy tasks. However, in Figure 3, the baseline methods achieve similar performance on different user groups. This comparison is not convincing evidence of the method's robustness.

Author response:
2. Further clarification of the NCF's results on different user groups (in Figure 3).  (W2 & Q1)
    
    Thank you for your insightful feedback. We appreciate your comment and would like to address it with the following points:
 - **Increment of supervision signals does not always improve performance**. The observation that basic methods like NCF achieve similar performance on different user groups, as shown in Figure 3, is consistent with findings from previous works on multi-behavior recommendation, such as [1-2]. This phenomenon highlights that simply increasing supervision signals does not necessarily lead to improved model performance. The lack of high-quality self-supervision signals can limit the effectiveness of increasing supervision, and this issue can also affect self-supervised learning methods based on random augmentations.
 - **Consistent performance gain of DHCF**. In contrast to existing supervision-enhancing methods like SGL, our DHCF achieves performance improvement across different data sparsity levels through intent disentanglement and modeling of heterogeneous relation dependencies. This design effectively enhances the overall model performance. The consistent performance superiority of DHCF over the baseline models further validates the effectiveness of these designs.

    [1] "Knowledge Enhancement for Contrastive Multi-Behavior Recommendation"

    [2] "Graph Meta Network for Multi-Behavior Recommendation"

Reviewer comment (Evaluation):
It is strange that the basic methods like NCF achieve similar performance on different user groups (in Figure 3). Instead of evaluating on tailed items, why did the authors test the performance on different user groups for robustness analysis?

Author response:
2. Further clarification of the NCF's results on different user groups (in Figure 3).  (W2 & Q1)
    
    Thank you for your insightful feedback. We appreciate your comment and would like to address it with the following points:
 - **Increment of supervision signals does not always improve performance**. The observation that basic methods like NCF achieve similar performance on different user groups, as shown in Figure 3, is consistent with findings from previous works on multi-behavior recommendation, such as [1-2]. This phenomenon highlights that simply increasing supervision signals does not necessarily lead to improved model performance. The lack of high-quality self-supervision signals can limit the effectiveness of increasing supervision, and this issue can also affect self-supervised learning methods based on random augmentations.
 - **Consistent performance gain of DHCF**. In contrast to existing supervision-enhancing methods like SGL, our DHCF achieves performance improvement across different data sparsity levels through intent disentanglement and modeling of heterogeneous relation dependencies. This design effectively enhances the overall model performance. The consistent performance superiority of DHCF over the baseline models further validates the effectiveness of these designs.

    [1] "Knowledge Enhancement for Contrastive Multi-Behavior Recommendation"

    [2] "Graph Meta Network for Multi-Behavior Recommendation"

Reviewer comment (Theory):
Is the meta-learning process considered while calculating the complexity in section 3.7?

Author response:
4. Is the meta-learning process considered while calculating the complexity in section 3.7? (Q2)
- We appreciate your thoughtful question. Although we did not explicitly mention the complexity of the meta network in our paper for the sake of simplicity, it involves a straightforward linear transformation. This transformation incurs a cost of $O((I+J)\times d^2)$ for each of the $K$ heterogeneous relations. Empirically, we have found that this complexity is close to $O(|\mathcal{X}|\times d)$, which is smaller than the complexity of the graph relation learning process. The graph relation learning process has a cost of $O(|\mathcal{X}|\times d\times L)$, where $|\mathcal{X}|$ represents the size of the dataset and $L$ denotes the number of graph convolutional layers.

Reviewer comment (Theory):
Any evidence to support the convergence of the learnable hypergraphs?

Author response:
5. Discussion of model convergence with the learnable hypergraphs? (Q3)
- Thank you for your question. During the training process of DHCF, we have noticed that the performance on the validation set remains stable in the last few epochs. This consistent behavior can be replicated using the code we have released, providing strong evidence for the convergence of the learnable hypergraphs.

Reviewer comment (Novelty):
The proposed Dynamic Hypergraph Collaborative Filtering (DHCF) approach presents a unique take on recommendation systems; however, its distinctiveness and advancements over existing methodologies in the literature are not sufficiently highlighted. To strengthen the paper, the authors should conduct a more comprehensive comparison of DHCF with prevailing models, pinpointing exact areas of improvement and innovation. Integrating and discussing the influence of more contemporary trends in recommendation systems, such as applications of deep learning or graph neural networks, would further enrich the paper's relevance and depth.

Author response:
We appreciate your feedback. In our work, we address two important challenges that have been overlooked in the field of collaborative filtering: interaction heterogeneity and fine-grained modeling of user intents. To bridge these gaps, we propose DHCF, a method that integrates intent disentanglement and multi-behavior modeling using a parameterized heterogeneous hypergraph architecture. Furthermore, we introduce a novel approach to heterogeneous contrastive learning, leveraging the encoded disentangled heterogeneous interaction patterns.

Reviewer comment (Novelty):
The methods proposed in the paper lack innovation significantly. Sections 3.1-3.5 follow very common design paradigms, and their method designs exhibit a certain degree of similarity to HCCF, ICL, and others. The paper should discuss the differences in the technical details between them.

Author response:
Our methodology comprises two primary components: the encoding part, which utilizes a heterogeneous hypergraph neural network, and the self-supervised learning part, employing heterogeneous contrastive learning. In order to address these concerns and highlight the technical novelty of our DHCF model, we provide the following justifications:

- **Uniqueness of DHCF's Encoding Part**. While our DHCF model and existing works such as HCCF share some commonalities (e.g., the use of hypergraph neural networks), there are two significant technical differences in the design of our encoder: **heterogeneity** and **disentanglement**. Unlike existing hypergraph-based methods like HCCF, which primarily focus on homogeneous data, our DHCF model addresses the challenge of heterogeneity by considering interaction types and effectively preserving heterogeneity information in the hypergraph-based encoder. Furthermore, the adoption of a parameterized hypergraph neural network in our DHCF model facilitates efficient and learnable intent disentanglement. This disentanglement is vital not only during the encoding process but also in the subsequent self-supervised learning part, as discussed below. These technical distinctions highlight the unique contributions of our DHCF model, enabling it to handle heterogeneity and disentangle user intents more effectively compared to existing approaches.

- **Novelty of Our Contrastive Learning and Its Relation to Encoder**. Our paper introduces a significant technical contribution in the form of the proposed disentangled heterogeneous contrastive learning method. This method incorporates several key components, including a parameter personalization module, a node-level heterogeneous self-discrimination task, and a graph-level contrastive learning task. These components leverage hypergraph-based global representation and shuffling-based negative sample construction techniques. By employing these designed techniques, our method greatly enhances the training of models for heterogeneous collaborative filtering. It achieves this by facilitating adaptive supervision enhancement while considering global intent disentanglement. Notably, our approach outperforms vanilla graph contrastive learning methods such as SGL, primarily due to the valuable heterogeneity and disentanglement information extracted by our designed encoding part.


- **Theoretical Contribution**. Our paper makes a contribution by providing an in-depth theoretical analysis that highlights two important advantages of DHCF that distinguish it from existing works: i) hypergraph-based disentanglement enhances the adaptability of contrastive supervision signals, ii) graph-level contrastive learning injects global disentangled representations into node-level similarities. Thank you for your insightful feedback. We will further highlight the novelty and uniqueness of our encoding component in future versions.

Reviewer comment (Theory):
In section 3.6, there are two loss functions proposed for relationship learning but in reality, they belong to the same paradigm. The paper lacks sufficient theoretical justification for their validity.

Author response:
- The two SSL loss functions employed in our study follow the contrastive learning paradigm. However, they differ in their specific focuses and objectives. The first SSL loss function concentrates on micro-scale learning, targeting the individual nodes within the graph. It enables fine-grained learning at the node level, capturing local heterogenous collaborative relationships among users and items. The second SSL loss function places emphasis on macro-scale learning, aiming to maximize learning for the overall graph structure. This macro-scale approach ensures that the model captures the global context and structural information within the multi-behavior data. Furthermore, since natural negative samples are lacking at the graph level, we introduce a shuffling-based negative sampling generation method, which effectively enlarges the technical similarity between the two loss functions.

Reviewer comment (Presentation):
The font size in Figure 1 is too small.

Author response:
- Thank you for your thoughtful suggestion. We will improve the clarity by updating the figure in later versions.

Reviewer comment (Novelty):
My major concern lies in the technical details. Many of the described methods bear a resemblance to existing approaches. It is crucial to clearly explain the distinctions and improvements made by DHCF in comparison to these existing methods.

Author response:
Our methodology comprises two primary components: the encoding part, which utilizes a heterogeneous hypergraph neural network, and the self-supervised learning part, employing heterogeneous contrastive learning. In order to address these concerns and highlight the technical novelty of our DHCF model, we provide the following justifications:

- **Uniqueness of DHCF's Encoding Part**. While our DHCF model and existing works such as HCCF share some commonalities (e.g., the use of hypergraph neural networks), there are two significant technical differences in the design of our encoder: **heterogeneity** and **disentanglement**. Unlike existing hypergraph-based methods like HCCF, which primarily focus on homogeneous data, our DHCF model addresses the challenge of heterogeneity by considering interaction types and effectively preserving heterogeneity information in the hypergraph-based encoder. Furthermore, the adoption of a parameterized hypergraph neural network in our DHCF model facilitates efficient and learnable intent disentanglement. This disentanglement is vital not only during the encoding process but also in the subsequent self-supervised learning part, as discussed below. These technical distinctions highlight the unique contributions of our DHCF model, enabling it to handle heterogeneity and disentangle user intents more effectively compared to existing approaches.

- **Novelty of Our Contrastive Learning and Its Relation to Encoder**. Our paper introduces a significant technical contribution in the form of the proposed disentangled heterogeneous contrastive learning method. This method incorporates several key components, including a parameter personalization module, a node-level heterogeneous self-discrimination task, and a graph-level contrastive learning task. These components leverage hypergraph-based global representation and shuffling-based negative sample construction techniques. By employing these designed techniques, our method greatly enhances the training of models for heterogeneous collaborative filtering. It achieves this by facilitating adaptive supervision enhancement while considering global intent disentanglement. Notably, our approach outperforms vanilla graph contrastive learning methods such as SGL, primarily due to the valuable heterogeneity and disentanglement information extracted by our designed encoding part.


- **Theoretical Contribution**. Our paper makes a contribution by providing an in-depth theoretical analysis that highlights two important advantages of DHCF that distinguish it from existing works: i) hypergraph-based disentanglement enhances the adaptability of contrastive supervision signals, ii) graph-level contrastive learning injects global disentangled representations into node-level similarities. Thank you for your insightful feedback. We will further highlight the novelty and uniqueness of our encoding component in future versions.