Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: The paper proposes a hypergraph neural network model exploiting a double attention mechanism in the message passing scheme. The overall architecture is designed to process sub-hypergraphs once representations are computed for the nodes and edges. The learning objective includes a regularization term based on the hypergraph laplacian.
The proposed model is evaluated on disease classification based on gene-genetic pathways data, showing higher F1 values with respect to a set of competitors.
Finally, due to the attention mechanism intepretions in terms of gene pathways (hyperedges) can be derived from the model outputs.
strengths_and_weaknesses: Strengths
- the model incorporates a dual attention mechanism applied to nodes and hyperedges respectively, exploiting the same attention context vector. This desing choice is claimed to prevent overfitting reducing the number of parameters
- the architecture includes an attention module to derive subgraph representations. This scheme allows the application of the mode in a inductive setting
- the model compares favourably with respect to the considered competitors on two benchmarks in genetic medicine

Weaknesses
- The presentation of the model is tightly interconnected with the proposed application in genetic medicine, making it appear less general
- Given the focus on bioinformatics, the paper is hard to follow for readers not completely familar with this topic.
- The effect of the number K of layers is not investigated (maybe I missed it, but the number of layers used in the experiments is not reported). It is known that (Convolutional) Graph Neural Network suffer from oversmoothing when the depth of the network increases. This may hinder the results in some applications since uniform representation of nodes are developed. Attention may perhaps limit this phenomenon, but on the other side node/edge regularization may produce add a related effect. The ablation study show a positive effect of regularization, but some discussion/analysis should be provided.


questions: How performances are affected by the number K of layers?
limitations: The authors discuss some limitations that need futher work in section of the paper both for the model architecture and the specific application considered for the evaluation.
As listed in the weaknesses, a more general description (not tailored for the considered task in genetic medicine) would have improved the presentation of the proposed hypergraph neural network architecture.
review_title: null

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: The authors propose a GNN approach to learn embeddings of sub-hyper-graphs. The approach has an explicit treatment of hyper edges, e.g., it does not resort to clique expansion, and makes use of a regulariser based on the hyper graph laplacian.  The application chosen is that of disease prediction for patients (modelled as sub hyper graphs) given a pathway network (modelled as a hyper graph with genes as nodes and sets/pathways as hyper edges). 
strengths_and_weaknesses: Pos
Explicitly treating hyper edges as first class citizens in the GNN modelling is of interest, since in this was hyper edges can be the subjects of notions of regularisation or attention.

Neg
The relative importance of the various ideas introduced is not clear, i.e., a better experimental design with clearer baselines and an ablation study is warranted. More specifically:

1. is the regularisation effective or of importance? (ablation case)

2. is the proposed architecture much different from using a standard graph neural network with attention on a pre-processed hyper graph? In particular the pre-processing could consist in representing an hyper graph as a bi-partite graph where hyper edges are materialised as the nodes of one part and the genes are the nodes of the other part.  (experiment)
Note that the whole discussion regarding strong duality would follow automatically in this case.

3. is the introduction of WSA (the weighted  subgraph attention) needed? what happens if we replace the whole subgraph treatment by Mji directly? I.e., what if we consider a subgraph simply the sum of the nodes (genes) that are of interest (with mutations) for each patient (subgraph), that is, we could learn directly the embedding of the nodes/hyperedges for the classification task when they are simply summed up for each patient. (experiment/ablation)
questions: Additional experiments to quantify the   relative importance of the various ideas introduced would strengthen the paper.
In particular consider offering support to reply to the points 1, 2 and 3 raised as negative elements in the Strengths And Weaknesses section.
limitations: yes.
review_title: null

final_score: rating: 4: Borderline reject: Technically solid paper where reasons to reject, e.g., limited evaluation, outweigh reasons to accept, e.g., good evaluation. Please use sparingly.
review_content: summary: This paper suggests sub-hypergraph representation learning, a niche problem related to subgraph and hypergraph learning. To tackle this problem, the authors propose the SHINE (SubHypergraph Inductive Neural nEtwork) model, which consists of three modules: strongly dual attention message passing, hypergraph regularization, and weighted subgraph attention. Experiments on two real-world datasets demonstrate the superiority of SHINE on performance (against baselines including GNNs for hypergraphs) and interpretation (using attention).
strengths_and_weaknesses: ## Strengths

The authors present a novel and niche problem of sub-hypergraph representation learning which has not been explored in the GNN community. A specific example (cancer patients as subgraphs of genes in hypergraphs) can be a practical application for this task. The performance improvement by the authors’ approach is significant.

## Weaknesses

However, I think this paper is not ready for publication for the following reasons.

First, the technical novelty of SHINE is limited. This model consists of several parts, and each of them is a slightly modified version of existing approaches. Using the attention to both nodes and (hyper) edges is presented in HyperGAT, and the authors are aware of it. Nevertheless, the idea of (strongly) dual attention is aligned with the dual form of hypergraphs, and I can see the novelty of this paper here. However, explicit regularization by Laplacian (Hypergraph regularization) [1] and pooling by attention weights (Weighted Subgraph Attention) [2] are well-known methods in using GNNs. In this case, SHINE's novelty is limited to Strongly Dual Attention, and a more detailed analysis of this part is required.

Second, related to the first paragraph, there are no rigorous ablation studies on the architecture. As many submodules make up the model, it is necessary to study where the performance gain comes from. In the supplementary material, only the ablation study on hypergraph regularization is presented, and the study on dual attention message passing is presented by comparison with HyperGAT. However, there are other differences in attention forms between HyperGAT and SHINE, and comparing these two does not provide a fully controlled ablation study of dual attention message passing. I recommend authors retain all parts except parameter-sharing in the attention. In addition, the performance comparison between SHINE with/without WSA and other GNNs with/without WSA also should be presented.

Third, it is skeptical that model interpretation by attention is an exclusive strength of SHINE. There are learned attentions in other attentional models like HyperGAT. Can these models provide interpretation at the same level as SHINE? Can you compare interpretations between models? Does SHINE give more precise explanations than other models?

Lastly, there are missing baselines for subgraph classification; in particular, the SubGNN can be a strong baseline. Of course, SubGNN is not designed for hypergraphs, but it is straightforward to create graphs from hypergraphs such as clique expansion. The transformation from hypergraphs to graphs is done only once before training; thus, it has a low overhead. Comparing SHINE and GNNs-for-subgraphs can justify that these specific problems in this work should be represented as a hypergraph.

## References

- [1] Learning with Hypergraphs: Clustering, Classification, and Embedding
- [2] GATED GRAPH SEQUENCE NEURAL NETWORKS

questions: My questions are summarized in the weaknesses section.

limitations: The authors do not address the potential negative societal impact of their work. This paper targets a high-level machine learning problem called sub-hypergraph representation learning; however, all datasets are related to a particular area, genes, pathways, and diseases. There could be a potential societal impact that should be considered in real-world applications in this area (e.g., privacy). It would be nice if the authors addressed this point.
review_title: null

final_score: rating: 4: Borderline reject: Technically solid paper where reasons to reject, e.g., limited evaluation, outweigh reasons to accept, e.g., good evaluation. Please use sparingly.
review_content: summary: Hypergraph neural networks can exploit multi-way connections in relational datasets but they are underexplored in domains such as genetic medicine. In this paper, a hypergraph attention-based message passing neural network is proposed for sub(hyper)graph-level tasks, e.g., 
* genes: nodes, 
* pathways: hyperedges, 
* patients: subgraphs, 
* predict cancer type of patient: task. 

Experiments on genetic medicine datasets demonstrate the effectiveness of the proposed method SHINE: SubHypergraph Inductive Neural nEtwork. 
strengths_and_weaknesses: **Originality**

Even though the paper explores an underexplored research topic in an interesting domain (subgraph representation learning for hypergraphs in genetic medicine), the methods proposed are incremental extensions of existing methods and not novel combinations of existing techniques.

Specifically in section 3.3., the ideas of hyperedge attention over nodes and node attention over hyperedges with parameter sharing are incremental extensions of well-known hypergraph attention networks.

By viewing the nodes and hyperedges as two types of vertices of a (bipartite) heterogeneous graph, the ideas of strongly dual attention mechanisms would be incremental extensions of existing attention-based methods for heterogeneous graphs, e.g., see "A Survey on Heterogeneous Graph Embedding: Methods, Techniques, Applications and Sources"

\
**Quality**

The authors have discussed interesting weaknesses of their work (in addition to highlighting the strengths).

Moreover, baseline comparison (Table 3), interpretability analysis (Table 4), and ablation study (Table 3 in supplementary) support the claims made in the paper empirically to an extent.

However, formalising the key differences with existing similar methods (e.g., HyperGAT in lines 159-169) and confirming the differences with convincing (synthetic/real-world) experiments, e.g., on a dataset chosen cleverly to show clear failure of HyperGAT but success of SHINE, would improve the paper's quality. 

\
**Clarity**

The paper is well organised.

Details on datasets and hyperparameter tuning could help an expert to reproduce the results of the paper and build effective models (those with the best hyperparameters) from scratch.

A discussion on computational complexity and an algorithm/pseudo code would further enhance the clarity of the paper.

\
**Significance**

It is unclear from the paper why modelling genetic medicine datasets with hypergraphs, despite being a natural choice, is the best choice compared to straightforward alternatives.

More specifically, it is unclear why a (bipartite) heterogenous graph with genes: nodes of type 1, pathways: nodes of type 2, patients: (sub) heterogeneous graph would not be a reasonable choice.

The paper can be improved by positioning and comparing with set-based methods for exploiting hyperedges in hypergraphs, e.g., You are AllSet: A Multiset Function Framework for Hypergraph Neural Networks, In ICLR'22.
questions: 1. How are the attention mechanisms in HyperGAT and SHINE different from existing attention mechanisms in heterogeneous graph neural networks with two different node types, e.g., treat genes and pathways as two different node types?
2. How does a simple feed-forward neural network with hypergraph regularisation and subgraph attention perform? This baseline is basically an ablated baseline with strongly dual attention message passing removed from SHINE and replaced with an MLP.
3. Is the strongly dual attention message passing scheme of SHINE an instantiation of the multiset formulation of "You are AllSet: A Multiset Function Framework for Hypergraph Neural Networks, In ICLR'22"?
4. What are the training times of all the methods (SHINE and all baselines)?
5. How sensitive are the hyperparameters (e.g., hidden dimensions) to the performance of SHINE? What is the optimal number of hidden layers in SHINE?
limitations: The authors have addressed the limitations and potential negative societal impacts adequately.
review_title: null

Meta-review: metareview: The paper proposed a GNN that explicitly treats hyperedges, and makes use of strongly dual attention, hypergraph regularization, and weighted subgraph attention.  The proposed method shows better performance than existing baselines on two genetic medicine datasets.  Explainability is also demonstrated.

Reviewers originally raised many concerns on presentation (too specialized for the target application), lack of ablation (effectiveness of each proposed component is not clearly shown), novelty (combination of small modifications of existing methods), and explainability (existing methods can do the same).  The authors made an amazing job to address most of the concerns:  They reported additional ablation results and baseline results, and showed that the proposed method still performs better, and each proposed component plays a significant role.

Two reviewers have been convinced by the author's response, while the other two have not, insisting that the novelty issue remains, and with the limited novelty, more careful investigation is required for publication.

This is a borderline paper, and I recommend acceptance because I think adjusting existing methods to target applications is important research even if the modifications are small.  The proposed method significantly outperforms existing baselines (including the ones reviewers suggested), and the additional ablation study shows each of the proposed components is effective.

On the other hand, I also sympathize with the reviewers with negative evaluations on the following comments: 

"formalising the key differences with existing similar methods (e.g., HyperGAT in lines 159-169) and confirming the differences with convincing (synthetic/real-world) experiments, e.g., on a dataset chosen cleverly to show clear failure of HyperGAT but success of SHINE, would improve the paper's quality."

"The paper can be strengthened by positioning strongly dual attention in SHINE with different attention mechanisms in heterogeneous graph neural network literature (some are listed below):

Heterogeneous Graph Attention Network, In WWW'19
HetGNN: Heterogeneous Graph Neural Network, In KDD'19
Metapath enhanced graph attention encoder for HINs representation learning, In BigData'19.
MAGNN: Metapath Aggregated Graph Neural Network for Heterogeneous Graph Embedding, In WWW'20
Heterogeneous Graph Transformer, In WWW'20.

There is no need to empirically compare and run them as baselines but explaining the key differences conceptually to make hypergraphs a more compelling choice for genetic medicine than heterogeneous graphs can strengthen the paper."

I hope the authors would make a bit more effort to incorporate these suggestions in the final version.


confidence: Less certain
Decision: Accept