Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: This paper aims to improve the generalizability of hypergraph neural networks using self-supervised contrastive learning in a few-shot learning scenario. The paper's main contributions can be summarized as follows: first, using domain knowledge, they fabricated augmentations, and second, for the first time, they proposed hypergraph generative models to generate augmented views using contrastive learning and novel variational hypergraph auto-encoder architecture which is an adaptation of VAE.  
strengths_and_weaknesses: Strengths:
* Paper is well written, flows smoothly, and easy to understand.
* Even though the idea is a combination of well-known techniques, overall it is quite original.
* The proposed method is tested with a high number of datasets and experiments even for robustness and fairness. I believe the experimental sections of the paper are quite strong but could be improved (see weaknesses).

Weaknesses:
* Proposed augmentations from A0 to A5 were not investigated in detail. Are these augmentations independent or is there any correlation between them? The selection of augmentations and coverage of the augmentations should be further investigated. Furthermore, there are not any combinations such as A1 and A2 together. This lack of various combinations of augmentations may be the reason for the proposed method to be more successful, this is not clear.
* Computational cost analysis of the proposed model is missing.
* Generated augmentations are not investigated enough. Are these augmentations, combinations of the fabricated augmentations?
questions: 
1. Are these augmentations extensive enough or is there any correlation between them?

2. Can you provide information about computational costs?

3. Are generated augmentations, combinations of the fabricated augmentations? Can you provide further analysis?

4. Eq5: I believe you need a “max” rather than a “min”, as ELBO is to be maximized.

5. Please explain briefly L_cl as the contrastive learning loss after the corresponding equation in the main text, and say further details are in the Appendix.

limitations: The authors stated two limitations of the study: the additional computational cost due to hypergraph generative model, and how to design powerful hypergraph generators, in the Appendix B4.
In Appendix B5, no negative societal impact is reported. When the application domain includes Recommender systems, I believe society is always at risk under harmful/unfair content spread/recommendation due to current ML/DL methods. This should be acknowledged.
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: The authors present a novel approach to perform contrastive learning by investigating several fabricated view augmentations of the hypergraph and a variational generative model. Their exhausting experiments demonstrated that the proposed methodology results in the SOTA for hypergraph, especially in low data regimens.
strengths_and_weaknesses: Strengths:

1. Well written, good flow of ideas.
2. Extensive evaluation:
    1. 13 datasets with different data regimens 10%, 1%. 
    2. Different self-supervision mechanisms
    3. Comparison with approaches to convert the hypergraph to graph
3. Novel idea to construct augmented views of hypergraphs for contrastive learning, especially the generative model

Weaknesses:

1. One of the arguments all around the paper is that the proposed method works for a low data regime. However, the authors performed experiments with 10% and 1%. What do the results look like when using ˜100% data for training? In other words, could you show the upper bound?

2. The results from the robustness part of the experiments are inconclusive. The authors explain that in some cases, the performance even increased. Given that these perturbations on the hypergraphs were adopted for attacks on normal graphs. The authors could either provide some insights into their results or perform the robustness study using suitable hypergraph perturbations

questions: 1. Section 3.2, when describing the different fabricated augmentations. Although it is clear that augmentations A3-A5 come from previous work, it would be nice to have a figure where A3-A5 are depicted 
2. Line 191-192. ‘We train one VHGAE to produce one generative view, with the other view G ̃p is kept as prefabricated.’ Do they mean the other view? Otherwise, do you generate one view from another view? Please explain
3. Table 2 is never referred to in the text.
4. Since Self and Con are the methods, which are compared, it would be a good idea to include some information about these approaches in the related work.
limitations: Authors have not adequately commented on the known limitations.
review_title: null

final_score: rating: 4: Borderline reject: Technically solid paper where reasons to reject, e.g., limited evaluation, outweigh reasons to accept, e.g., good evaluation. Please use sparingly.
review_content: summary: This paper investigates the view augmentations in hypergraph contrastive learning. In particular, it explores several domain knowledge-guided fabrication augmentation methods and proposed hypergraph generative models. The authors' experiments show that generative augmentations perform well at preserving higher-order information and thus presents better performance.
strengths_and_weaknesses: Strengths:
1.The paper is easy to follow.
2. The proposed hypergraph generative modeling is a good strategy for augmenting a view and the results seem quite promising.

Weaknesses:
1. The reason or analysis why the proposed hypergraph generative modeling works better than other view augmenting methods are not clear. 
2. I think the hypergraph generative modeling could potentially increase training difficulty compared to other methods.
3. If with different augmentations, the contrast manner should be accordingly tuned?

Others:
I'm not quite sure if the topic (hypergraph contrastive learning) is important or not (I have very bad experience with hypergraph learning, as they cannot work well in real applications).
questions: 1. if "current attack algorithms are far from effective because they are not specifically designed for hypergraphs", then why you claim the designed hyperGCL can provide robustness?
2. Why statistical parity ΔSP and equalized odds ΔEO can represent fairness in hypergraph learning?
3. Are the Hypergraph Generative Models hard to train (converge)?
limitations: Yes
review_title: null

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: The paper proposes augmentations for learning representation on HyperGraphs using contrastive learning. They introduce two classes of augmentations: fabricated and generative augmentations and show that using these augmentations it is possible to learn good representations over the input hypergraphs. The paper is well-written and easy-to-follow.  
strengths_and_weaknesses: Strengths:
- The paper tackles a problem (representation learning on HyperGraphs) that can be used in various applications. 
- The paper is well-written and easy-to-follow.
- The experimental setup is fair and the model shows good performance compared to baselines.

Weaknesses:
- The paper simply adopts previous work on contrastive graph representation learning [1,2,3] and applies it to hyper-graphs. There's not much insights on hypergraphs but rather authors map hypergraphs into bipartite graphs and apply recent graph augmentations to learn representations. This is not a bad thing per se, but again it doesn't give any new insights about hypergraphs.
- Experimental studies need to be expanded to show for example how the augmentation mechanism works with other losses beside InfoNCE.

[1] https://arxiv.org/abs/2201.09830
[2] https://arxiv.org/abs/2006.06830
[3] https://arxiv.org/abs/2106.07594
 

questions: 1- How does the proposed augmentations interact with other types of contrastive loss?
2- Is it possible to define the augmentations over the hypergraph space rather than converting it to a a bipartite graph? 
limitations: NA
review_title: null

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: In the paper, the author studies the problem of how to construct contrastive views of hypergraphs via augmentations. Its contributions include: (1) Fabricating two schemes to augment hyperedges with higher-order relations encoded, and adopt three vertex augmentation strategies from graph-structured data. (2) Proposing hypergraph generative models to generate augmented views, and then an end-to-end differentiable pipeline to jointly perform hypergraph augmentation and contrastive learning.
strengths_and_weaknesses: Strengths:
The quality and clarity of the paper are good. The motivation is clear and the author does enough experiments. The paper is sound in improving the generalizability of hypergraph neural networks in the low-label regime.
Weaknesses:
The innovation is of HyperGraphCL is limited. The augmentations like perturbing hyperedges and subgraph, are just the same as that in graph contrastive learning [1][2][3]. Why does the generative model that the author proposes work well? I think the author needs to explain more clearly in theory. The description and analysis in the experiment section are ambiguous, and the writing in this part is hard to follow. The compared baselines are not advanced enough, and I recommend the author conduct more SOTA methods for comparisons, e.g., SOTA graph contrastive methods.

[1] You, Y., Chen, T., Sui, Y., Chen, T., Wang, Z., & Shen, Y. (2020). Graph contrastive learning with augmentations. Advances in Neural Information Processing Systems, 33, 5812-5823.
[2] Suresh, S., Li, P., Hao, C., & Neville, J. (2021). Adversarial graph augmentation to improve graph contrastive learning. Advances in Neural Information Processing Systems, 34, 15920-15933.
[3] Xia, J., Wu, L., Chen, J., Hu, B., & Li, S. Z. (2022, April). SimGRACE: A Simple Framework for Graph Contrastive Learning without Data Augmentation. In Proceedings of the ACM Web Conference 2022 (pp. 1070-1079).
questions: About adversarial robustness in experiments: the authors mentioned that " Meanwhile current attack algorithms are far from effective because they are not specifically designed for hypergraphs ." So are the experimental results convincing? About Table7: Why does not the table contain experimental results of A1, A3 and A5?
limitations: In the experiments, one have to select the data augmentation strengths manually. The fairness of experiments needs more datasets to support.
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: The submission introduces a generative modelling approach to obtain learnable hypergraph augmentations. These augmentations are used to generate views for contrastive learning. Empirically, the benefit of the learned augmentations over several fabricated types of augmentations, as well as over two baselines, is shown. Additional experiments investigate adversarial robustness and fairness aspects.
strengths_and_weaknesses: ### Strengths

+ Compared to other types of data, (e.g. image data), sensible hypergraph augmentations are not obvious and difficult to design.
As far as I am aware, this is the first work to introduce learnable hypergraph augmentations.

+ While the submission focuses on utilizing such augmentations in a contrastive learning setting, the method to generate the augmentations is not limited to it. Application in other settings seem possible.

+ The method consistently outperforms the baselines on the studied vertex classification tasks. The learned augmentations outperform the fabricated ones.

+ The method is not only checked for classification accuracy, also adversarial robustness and fairness aspects are evaluated.

### Weaknesses

- The method is computational expensive. In consequence, only one augmentation in each pair of views is generated (the other is fabricated, cf. l.192)
This might be the reason why the empirical evaluation focuses on the low-label regime, where fewer augmentations are required.

- Some parts of the paper are difficult to understand, particularly the description of the augmentation procedure. See questions.

- The adversarial attack experiments are not very expressive, considering that for some attacks model accuracy even increases. Maybe the adaptions of Random Noise and Minmax Attack could even serve as hypergraph augmentations.


**Post rebuttal edit**

While my questions have been adequately addressed, several reviewers have pointed at limited novelty in comparison with existing works from the graph augmentation literature. I will thus keep my initial rating of **6: Weak accept**.

questions: - Did you experiment with the augmentations in different settings, e.g., hypergraph classification, link prediction, ...

- How does the method perform in not low-label regimes (e.g. 50% of vertexes as training set? Does it become infeasible due to the high computational cost of VHGAE? Is the benefit over the baselines still as pronounced?

- Could you clarify the difficulty in constructing hypergraph augmentations.
  >(l.33) However, it is non-trivial to build hypergraph views due to their overly complicated topology, i.e., there are $\sum_e \binom{N}{e}$ possibilities for one hyperedge on $N$ vertices, versus $\binom{N}{2}$ for one edge in graphs.  

  At first sight, this property seems to facilitate the construction of hypergraph augmentations.
  What is the "combinatorial challenge" (l.119?) Does this refer to checking the set of hyperedges for duplicates, i.e. whether two finite sets of vertices are equal / correspond to permuted tuples?

- Could you expand on Eq. (2)? How is the bipartite graph determined by the sample $z_{\mathcal V}$ and $z_{\mathcal E}$. What do $z_e$ and $z_v$ correspond to, how are they connected to $z_\mathcal V$ and $z_\mathcal E$? Similarly, $w_{\mathcal {VE}}$ and $w_{ve}$. What is meant with the "topological" distribution of the bipartite graph?

- Regarding equation (4) and the corresponding text. What is meant with $T(G) \circ G$, i.e., how is the "vector applied to perform augmentation". Is $T(G)$ not only a vector, but also a function?

I-  do not understand Figure 3. What do the green arrows indicate?

- In experiments (e.g. Table 3) what is the difference between SetGNN (the backbone) and A0 (identity augmentations).
limitations: The (presumably) high computational cost of the method is not adequately addressed, i.e., it is not quantified. A comparison of runtimes between generated augmentations, prefabricated augmentations and no augmentations would provide clarity.
review_title: null

Meta-review: metareview: First of all, the majority of reviewers very clearly acknowledged the clarity of presentation and writing within the manuscript. There might not  be a large degree of novelty in the paper, but the approach is well evaluated and exhibits good performance. After carefully reading the author responses to all reviewer comments, my impression is that the authors spent quite some time addressing all raised questions in a satisfactory manner. While the scores of the paper are certainly borderline, I think this paper warrants acceptance which I do recommend at this point.
confidence: Less certain
Decision: Accept