MPHIL: Multi-Prototype Hyperspherical Invariant Learning for Graph Out-of-Distribution Generalization

Authors: Xu Shen, Yixin Liu, Yili Wang, Rui Miao, Yiwei Dai, Shirui Pan, Xin Wang
Primary area: learning on graphs and other geometries & topologies
Keywords: graph out-of-distribution generalization, invariant learning, hyperspherical space

Abstract:
Out-of-distribution (OOD) generalization has emerged as a critical challenge in graph learning, as real-world graph data often exhibit diverse and shifting environments that traditional models fail to generalize across. A promising solution to address this issue is graph invariant learning (GIL), which aims to learn invariant representations by disentangling label-correlated invariant subgraphs from environment-specific subgraphs. However, existing GIL methods face two major challenges: (1) the difficulty of capturing and modeling diverse environments in graph data, and (2) the semantic cliff, where invariant subgraphs from different classes are difficult to distinguish, leading to poor class separability and increased misclassifications. To tackle these challenges, we propose a novel method termed Multi-Prototype Hyperspherical Invariant Learning (MPHIL), which introduces two key innovations: (1) invariant learning in hyperspherical space, enabling robust invariant feature extraction and prototypical learning in a highly discriminative space, and (2) class prototypes as intermediate variables, which eliminate the need for explicit environment modeling in GIL and mitigate the semantic cliff issue through multi-prototype-based classification. Derived from the theoretical framework of GIL, we introduce two novel objective functions: the invariant prototype matching loss to ensure samples are matched to the correct class prototypes, and the prototype separation loss to increase the distinction between prototypes of different classes in the hyperspherical space. Extensive experiments on 11 OOD generalization benchmark datasets demonstrate that MPHIL achieves state-of-the-art performance, significantly outperforming existing methods across graph data from various domains and with different distribution shifts. The source code of MPHIL is available at https://anonymous.4open.science/r/MPHIL-23C0/.

Decision: reject
Reviewer scores: 5, 5, 5, 6

Reviews and discussion:
<thread>
creation date:
1729130901428

thread_type:
official_review

<thread_object>
creator: reviewer_s53c

<content>
summary:
This paper introduces MPHIL, a novel method for graph out-of-distribution (OOD) generalization. Key features include: 1）Invariant learning in hyperspherical space. 2) Multi-prototype classification approach. MPHIL eliminates the need for explicit environment modeling and addresses the semantic cliff issue. It outperforms existing methods on 11 OOD benchmark datasets across various domains. The approach offers a more robust solution for real-world graph learning applications where data distributions often shift.

strengths:
1. Originality: The idea is relatively novel, using Hyperspherical Learning to deal with OOD generalization on graphs, and achieving promising experimental results.

2. Clarity & Significance: The logic of the manuscript is clear. In particular, the example of molecular graphs given in the introduction is very helpful for contextualization, and does a good job of describing the problem the authors were trying to solve and the significance of the study.

3. The manuscript is substantial and essentially encompasses all of the research that should be discussed.

weaknesses:
1. There are some terms and formulas that are not presented accordingly, such as I(z;y) in Eq. 2, which I understand to mean mutual information. However, to increase readability I would suggest that these symbols should be explained the first time they appear. Apart from this, there is no explanation of the significance of some of the designed variables, such as separation score S. Why add it additionally to H? What is the benefit of doing so?

2. Figure 3 does not significantly show the advantage of your proposed method over baseline, can you consider a different visualization example that more intuitively reflects the difference between the two?

3.  You included a link to the code in your manuscript, but I found that many of the modules are called, but the relevant code doesn't exist and doesn't work.

questions:
1. Is graph invariant learning only applicable to graph classification task? Can it be processed if it is a node classification task?

2. In Section 3.3, it is mentioned that K prototypes are assigned to each category, so what is the basis for the final categorization? Further, since invariant representation can correspond to a set of prototypes, does it happen that these prototypes do not belong to the same class at the test phase? (after softmax operation, although one of the prototypes has the highest probability, but the other classes contain multiple prototypes with relatively high probability, can it be considered that the confidence of the classification result is lower?) I'm a bit confused here, can you explain the rationale for designing prototypes? Is it relying exclusively on the several losses that have been subsequently proposed to be corrected to avoid the situation I have described?  (I completely agree with your statement at the beginning of section 3.3 about the risk of overfitting caused by a single prototype in the previous approach)
</content>
</thread_object>

</thread>
<thread>
creation date:
1730491704909

thread_type:
official_review

<thread_object>
creator: reviewer_pafp

<content>
summary:
This paper studies the problem of graph Out-of-distribution (OOD) generalization and then proposes two objective functions: the invariant prototype matching loss to ensure samples are matched to the correct class prototypes, and the prototype separation loss to increase the distinction between prototypes of different classes in the hyperspherical space. Extensive experiments on 11 OOD generalization benchmark datasets demonstrate the effectiveness of the proposed method.

strengths:
- The paper studies an important research question.
- The experimental results are extensive.

weaknesses:
- This paper seems to closely follow the recent research "HYPO: Hyperspherical Out-Of-Distribution Generalization", e.g., "intra-class variation and inter-class separation principles". The novelty of the paper seems to be limited. 
-  The visualization results do not seem clear (fig 3). Why the proposed method is better than the compared method
- Some SOTA methods seem to be missing.

questions:
See above.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730538501116

thread_type:
official_review

<thread_object>
creator: reviewer_x7hk

<content>
summary:
The authors address the challenges of out-of-distribution (OOD) generalization on graphs and propose a model named MPHIL in this paper. MPHIL builds on graph invariant learning (GIL) by disentangling label-correlated invariant subgraphs from environment-specific subgraphs, which helps overcome difficulties in capturing diverse environments and the semantic cliff. For more details, MPHIL first introduces invariant learning within a hyperspherical space using multiple class prototypes to enhance classification without explicit environment modeling. Then, two loss functions ( invariant prototype matching loss and prototype separation loss ) are proposed to ensure accurate class association and inter-class distinction within the hyperspherical space. Extensive experiments conducted across 11 OOD generalization benchmark datasets demonstrate the effectiveness of MPHIL.

strengths:
Different from traditional methods, this paper introduces a novel approach by applying invariant learning in a hyperspherical space to enhance out-of-distribution (OOD) generalization in graph-based tasks. Specially, the invariant prototype matching loss and prototype separation loss proposed in Section 3.4 are well-designed to ensure both accurate class association and inter-class distinction, improving model robustness in hyperspherical space.

weaknesses:
- The novelty of this paper is limited. This paper introduce invariant learning into a hyperspherical space and introduces multiple prototypes to improve class separability, these ideas extend previously established techniques in a relatively straightforward manner. However, the use of hyperspherical embeddings for enhancing model performance of OOD has been explored [1], [2] and [3].

- The experimental results are not convincing, and the baseline methods are not strong enough. Although the authors compare MPHIL with some widely used baseline methods like ERM, CIGA and GSAT, these are neither the latest nor the most competitive, which undermines the significance of the reported performance. More recent OOD methods for graphs should be introduced in the experimental part for fair comparisons, such as OOD-GCL [4], GALA [5] and LECI [6].

[1] Ming Y, Sun Y, Dia O, et al. How to Exploit Hyperspherical Embeddings for Out-of-Distribution Detection?[C]//The Eleventh International Conference on Learning Representations.

[2] Bai H, Ming Y, Katz-Samuels J, et al. HYPO: Hyperspherical Out-Of-Distribution Generalization[C]//The Twelfth International Conference on Learning Representations.

[3] Du X, Gozum G, Ming Y, et al. Siren: Shaping representations for detecting out-of-distribution objects[J]. Advances in Neural Information Processing Systems, 2022, 35: 20434-20449. 

[4] Li H, Wang X, Zhang Z, et al. Disentangled Graph Self-supervised Learning for Out-of-Distribution Generalization[C]//Forty-first International Conference on Machine Learning. 2024.

[5] Chen Y, Bian Y, Zhou K, et al. Does invariant graph learning via environment augmentation learn invariance?[J]. Advances in Neural Information Processing Systems, 2024, 36.

[6] Gui S, Liu M, Li X, et al. Joint learning of label and environment causal independence for graph out-of-distribution generalization[J]. Advances in Neural Information Processing Systems, 2024, 36.

questions:
see weakness
</content>
</thread_object>

</thread>
<thread>
creation date:
1730725302182

thread_type:
official_review

<thread_object>
creator: reviewer_grw6

<content>
summary:
The paper presents a novel method MPHIL to address two challenges in existing graph invariant learning methods: the difficulty of capturing environmental information and the semantic cliff  across different classes. MPHIL operates in hyperspherical space with class prototype as intermediate variable to improve representation separability and robustnes without explicit environment modeling. 
Two innovative loss functions are derives to enhance intra-class invariance and inter-class separability. Extensive experiments show that MPHIL outperforms state-of-the-art methods on various benchmark datasets, demonstrating its effectiveness for OOD generalization.

strengths:
This paper propose a new method for graph invariant learning. 
- The paper is well-organized.

weaknesses:
-  Intuitively, the accuracy the class prediction relies on the fastest prototype point in the same class prototype set.  What if use the mean/sum/min for predicting the class? More ablation study can make your work more convincing.
- The novelty of this proposed method is limited. I don't think the projection to the hypersphere here is novel. Actually, it is a simple cosine distance. How about using other distances like L2, L1, more analysis should be included.
- I wonder the performance when only using one protype. 
- The prototype updating mechanism is complicated, what if directly use the representation of the sample from training set after training.
- The explaination of some symbols is missing, e.g. the $\omega_k^{(c)}$ in Eq (12).

questions:
See weakness.
</content>
</thread_object>

</thread>
<thread>
creation date:
1731982244722

thread_type:
official_comment

<thread_object>
creator: authors

<content>
title:
Response  Summary

comment:
We sincerely thank all the reviewers for their valuable and insightful comments. We are delighted that the reviewers acknowledged the importance of the research problem addressed in this paper (Reviewer pafp), recognized the originality of our proposed method (Reviewers s53C and X7hK) and the extensive experiments conducted (Reviewers pafp and s53C), and found the paper to be logically clear and well-organized (Reviewers gRW6 and s53C).

To the best of our efforts, we have provided detailed responses below to address each of the reviewers' concerns. Additionally, we have carefully revised the manuscript based on their feedback and highlighted the changes in blue.

Specifically, the main responses and modifications we made are as follows.
-  We have further emphasized the **unique motivation and contributions** of the proposed MPHIL framework to demonstrate its sufficient novelty.
-  We incorporated **three state-of-the-art baseline methods** and updated the experimental results, showing that MPHIL consistently achieves the best average performance.
-  We have further explained the Fig. 3 and included **additional quantitative analyses** to better illustrate the advantages of our method.
-  We conducted **more detailed ablation studies** to validate the effectiveness of MPHIL's design and provided further clarification on the principles of multi-prototype classification.
</content>
</thread_object>

</thread>


Metareview:
metareview:
This paper proposes a method for graph out-of-distribution generalization using hyperspherical invariant learning and multi-prototype classification, supported by experiments across various benchmarks. The paper addresses a meaningful problem and is well-structured. The reviewers raised concerns about the method's novelty and the competitiveness of its baseline comparisons. Despite the authors’ thorough rebuttal and additional clarifications, three reviewers rated the paper below the acceptance threshold. Respecting the majority opinion, I lean towards rejection.