Venue: ICLR 2022 Conference

Reviews:

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: This paper proposes a generalization of the hypergraph neural networks using multiset functions.
main_review: This paper proposes a generalization of the hypergraph neural networks using multiset functions. Instead of doing clique expansion as in many existing methods, the proposed method learns explicit hypergraph representations, and uses two multiset functions to perform message propagations, one from nodes to hyperedge and another from hyperedge to nodes. The method is shown to be a general framework that encompasses existing GNN methods. Experimental results show that the AllSetTransformer achieves competitive results or outperforms existing methods on a large set of benchmark datasets. 

The proposed method unifies various methods for hypergraph neural networks, and gives a competitive formulation across a wide spectrum of problems. This is a nice contribution to the community.

summary_of_the_review: The paper makes a nice contribution to the community.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: The paper proposes a framework for hyper graph neural networks.  It argues that existing work propagates a hyper graph by first transforming it into a regular graph trough clique expansion, which may lose information.  Alternatively, the propagation is based on tensors.  The proposed framework seeks to combine these different ways of propagations into one unified framework.  The proposed framework is called AllSet, with two instantiations AllDeepSets and AllSetTransformer.  Experiments are conducted on several graph datasets.
main_review: Strengths of the paper:

S1. The problem is well-motivated, with significant coverage of existing work based on clique expansion and tensor-based propagation.

S2. Experiments include a significant number of baselines (ten models), as well as on ten different datasets.

S3. There are theoretical analyses showing how the proposed framework is a generalisation of some existing works.


Weaknesses of the paper:

W1. As the proposed framework is in some sense a generalisation or a unification of existing propagation mechanisms, such as clique expansion and tensor-based propagation, there should be an ablation analysis to understand the contribution of each component.

W2. The experimental discussion are focused on just showing numerical comparisons.  A more discursive analysis that explains why the method works better than the existing methods would help to reveal where the sources of improvements are.

W3. Since the core is to derive hyper graph representations, experimenting with a single task is rather limited.  It would be good to have additional evaluation tasks beyond semi-supervised classification.
summary_of_the_review: The paper proposes a framework for hyper graph neural networks, which is an interesting problem.  Though rigorous in its treatment and experiments, further analysis on what makes the work better then the baselines would help in better appreciating the significance of the contributions.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: Transferring standard graph operators to the hypergraph setting is non-trivial and several message passing operators have been considered in the setting of hypergraphs, including clique-expansion and tensor based. In this paper the authors propose a general framework where learnable multiset functions are used in order to learn the hypergraph propagation map from the data. In particular, this framework contains  many previously considered propagation maps as particular cases
main_review: Strengths:

- Overall, I think this is a well-organized paper and I enjoyed reading it. The motivations for the model design are well discussed as is the related work. 
- The proposed idea is simple and neat. Many hypergraph Laplacian-like mappings have been considered and this gives a possible data-centric answer to the question "which one should I use?"
- The experiments consider a wide range of hypergraph datasets, which is somewhat rare in hypergraph classification literature. Moreover, the authors have collected several new hypergraph datasets, which I believe will be very useful for the community

Weaknesses:
- The experiments are not completely clear to me. In particular: (1) I think you should specify more clearly what is the depth of MLP and Set Transformer layers in AllDeepSets and AllSetTransformer and (2) you should more clearly specify what is the overall amount of known labels you used in the experiments and what is the amount used for training the AllSets architectures
- The notation in equation (8) should be better specified (e.g. what is LN or || )
- Other propagations which are "in-between" CE and tensors have been considered recently, e.g. [1-3]. Would these be particular cases of your proposed framework too?    
[1] Tudisco et al - Node and edge nonlinear eigenvector centrality for hypergraphs  
[2] Arya et al - Adaptive Neural Message Passing for Inductive Learning on Hypergraphs  
[3] Prokopchik - A nonlinear diffusion method for semi-supervised learning on hypergraphs  

summary_of_the_review: Overall, this is a solid paper which proposes one simple, yet novel and efficient idea to generalize propagation maps on hypergraphs. Solid experimental results are provided in the paper.

I like this paper and I would like to see it accepted. I will raise my score if the weaknesses are properly addressed

-----

I am happy with the feedback provided to my concerns and those of other reviewers so I will raise my score
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: Hypergraphs can capture group/set relationships in real-world data. 
Several hypergraph neural networks have been proposed in the literature to exploit both group relationships among nodes and node features for learning with hypergraph data. 
The contributions of the paper are 
1) generalisation of most existing methods into a single framework (named AllSet), 
2) exploration of AllSet based on Deep Sets [NeurIPS'17] and Set Transformer [ICML'19], and 
3) empirical evaluation on existing benchmarks and three curated hypergraph datasets 
main_review: ### **Strengths**

1. **Quality of Paper**

The paper is technically sound. 
The methods used (AllSetTransformer, AllDeepSets) are appropriate for the downstream task (node classification in hypergraphs). 
It is interesting to see that AllSet generalises several neural networks on hypergraphs and message passing neural networks on graphs. 

2. **Clarity of Presentation**

The paper is clearly written and well organised. 
The appendix is comprehensive with both theoretical proofs and empirical details (e.g., dataset construction, hyperparameters, etc.). 
The authors have also released the source code as part of the supplementary material. 



### **Weaknesses**

1. **Originality of Contributions**

The key idea of AllSet is to use rules of the form $f_{\mathcal{V}\rightarrow\mathcal{E}}$ and $f_{\mathcal{E}\rightarrow\mathcal{V}}$ as given in equation 5. 
These update rules are well-known ideas in both (i) hypergraph neural network literature [e.g., Be More with Less: Hypergraph Attention Networks for Inductive Text Classification, EMNLP'20] and (ii) bipartite graph neural network literature [e.g., Hierarchical Representation Learning for Bipartite Graphs, IJCAI'19].
Moreover, AllSetTransformer and AllDeepSets are straightforward applications of SetTransformer and DeepSets in the AllSet framework.


2. **Significance of Empirical Evaluation**

From Table 2 it is clear that the proposed methods show marginal or no improvements on all but three datasets viz., zoo, a tiny dataset, and Yelp, Walmart both curated by authors.  The Yelp dataset is about review rating prediction for restaurants where all methods achieve less than 37% accuracy: it is unclear how significant hypergraph modelling and the curated node features (e.g., latitude, longitude, city, state, etc.) are for rating prediction. The Walmart (and House) datasets do not come with node features, so it would be more compelling if hypergraph-only methods are used as baselines [e.g., Re-revisiting Learning on Hypergraphs: Confidence Interval and Subgradient Method, ICML'17].


3. **Positioning with Prior Work**

Experiments are restricted to transductive node classification in hypergraphs. The idea of $f_{\mathcal{V}\rightarrow\mathcal{E}}$ and $f_{\mathcal{E}\rightarrow\mathcal{V}}$ update rules have been successfully used in other tasks, e.g., (i) inductive text classification  [e.g., Be More with Less: Hypergraph Attention Networks for Inductive Text Classification, EMNLP'20], (ii) hyperlink prediction [e.g., Principled Hyperedge Prediction with Structural Spectral Features and Neural Networks], and (iii) recommendation [e.g., Session-based Recommendation with Hypergraph Attention Networks].
It would be interesting to see how AllSetTransformer and AllDeepSets would perform in these tasks.


### Update
After reading the responses to all the reviews, I have updated my recommendation from 5 -> 6
summary_of_the_review: Overall the paper is clear and of good quality in which claims are well supported by theoretical analysis. However, the novelty is incremental and experimental results are marginally significant. Positioning with missing prior work and evaluation on other hypergraph tasks would improve the paper.
review_title: null

Meta-review: This paper proposes a hypergraph representation learning based on multiset encoding, which  covers most existing propagation methods for hypergraph neural networks. The authors provide theoretical proofs that both CE-based and tensor-based propagation rules can be represented as a composition of two multiset functions, and propose two different multiset encoding functions, based on DeepSets and SetTransformer. The authors validate their method for its semi-supervised node classification performance on multiple benchmark datasets, showing that it is superior or comparable to a large number of existing works on hypergraph representation learning. 

The following is the summary of the pros and cons of the paper mentioned by the reviewers:

Pros
- The proposed framework generalizes existing message passing methods for hypergraphs, and authors provide theoretical proofs on how it can generalize to two different types of propagation rules for hypergraph representation learning. 
- The paper is well-organized and is clearly written, and the code is provided for reproduction. 
- The experiments consider a wide range of hypergraph datasets and baselines, and the proposed method either outperforms them or at least achieves comparable performance.

Cons
- It is still unclear where the benefits come from, due to lack of ablation studies and deeper analysis. 
- Experiments are only restricted to the semi-supervised node classification task. 

While the initial reviews were split due to these negative points, all reviewers unanimously recommended for acceptance after the discussion period, as they found the responses from the authors satisfactory.  

In summary, this is a well-written paper that proposes a neat framework for hypergraph representation learning that generalizes to most existing methods, backed up by compelling performance on benchmark datasets, which will make it a strong addition to ICLR. However, as mentioned by the reviewers there should be more ablation studies and in–depth analysis of what makes the proposed multiset function more effective, as this is lacking in the current version of the paper. It would be worthwhile to also validate the proposed framework on other tasks (e.g. graph classification tasks). 
 
One minor thing that I want to point out is regarding the claim that this is the first attempt to connect the problem of learning multiset functions with hypergraph neural networks.  [Jo et al. 21], which is a hypergraph-based framework for edge representation learning, utilized GMT [Baek et al. 21], which performs multiset encoding using SetTransformer for hypergraph representation learning, and thus I suggest the authors to tone down on the claim that this is the first work that connects multiset encoding with hypergraph neural networks, and properly acknowledge this. 

[Jo et al. 21] Edge Representation Learning with Hypergraphs, NeurIPS 2021
Decision: Accept (Poster)