HyperSAGE: Generalizing Inductive Representation Learning on Hypergraphs
Authors: Devanshu Arya
Deepak Gupta
Stevan Rudinac
Marcel Worring
Venue:  (2021)
Keywords: Hypergraph
Representation Learning
Inductive Learning
Geometric Deep Learning
Aggregation Methods

Abstract: Graphs are the most ubiquitous form of structured data representation used in machine learning. They model, however, only pairwise relations between nodes and are not designed for encoding the higher-order relations found in many real-world datasets. To model such complex relations, hypergraphs have proven to be a natural representation. Learning the node representations in a hypergraph is more complex than in a graph as it involves information propagation at two levels: within every hyperedge and across the hyperedges. Most current approaches first transform a hypergraph structure to a graph for use in existing geometric deep learning algorithms. This transformation leads to information loss, and sub-optimal exploitation of the hypergraph's expressive power. We present HyperSAGE, a novel hypergraph learning framework that uses a two-level neural message passing strategy to accurately and efficiently propagate information through hypergraphs. The flexible design of HyperSAGE facilitates different ways of aggregating neighborhood information. Unlike the majority of related work which is transductive, our approach, inspired by the popular GraphSAGE method, is inductive. Thus, it can also be used on previously unseen nodes, facilitating deployment in problems such as evolving or partially observed hypergraphs. Through extensive experimentation, we show that HyperSAGE outperforms state-of-the-art hypergraph learning methods on representative benchmark datasets. We also demonstrate that the higher expressive power of HyperSAGE makes it more stable in learning node representations as compared to the alternatives. 

Under review as a conference paper at ICLR 2021
HYPERSAGE: GENERALIZING INDUCTIVE REPRESENTATION LEARNING ON HYPERGRAPHS
Anonymous authors Paper under double-blind review
ABSTRACT
Graphs are the most ubiquitous form of structured data representation used in ma- chine learning. They model, however, only pairwise relations between nodes and are not designed for encoding the higher-order relations found many datasets. To model such complex relations, hypergraphs have proven to be natu- ral representation. Learning the node representations in hypergraph is more com- plex than in graph as it involves information propagation at two levels: within every hyperedge and across the hyperedges. Most current approaches first trans- form hypergraph structure to graph for use in existing geometric deep learning algorithms. This transformation leads to information loss, and sub-optimal ex- ploitation of the hypergraph's expressive power. We present HyperSAGE, novel hypergraph learning framework that uses two-level neural message passing strat- egy accurately and efficiently propagate information through hypergraphs. The flexible design of HyperS AGE AGE facilitates different ways of aggregating neighbor- hood information. Unlike the majority of related work which transductive, our approach, inspired by the popular GraphSAGE method, is inductive. Thus, it can also be used on previously unseen nodes, facilitating deployment in prob- lems such as evolving or partially observed hypergraphs. Through extensive ex- perimentation, we show that HyperSAGE outperforms state-of-the-ar hypergraph learning methods representative benchmark datasets. We also demonstrate that the higher expressive power of HyperSAGE makes it more stable in learning node representations compared to the alternatives.
INTRODUCTION
Graphs are considered the most prevalent structures for discovering useful information within a net- work, especially because of their capability to combine object-level information with the underlying inter-object relations (Wu et al.) 2020). However, most structures encountered in practical appli- cations form groups and relations that cannot be properly represented using pairwise connections alone, hence graph may fail to capture the collective flow of information across objects. In ad- dition, the underlying data structure might be evolving and only partially observed. Such dynamic higher-order relations occur in various domains, such social networks (Tan et al. (011), computa- tional chemistry (Gu et al.) 2020). neuroscience (Gu et al.] (017) and visual arts (Arya et al.) 2019). among others. These relations can be readily represented with hypergraphs, where an edge can connect an arbitrary number of vertices as opposed to just two vertices in graphs. Hypergraphs thus provide a more flexible and natural framework to represent such multi-way relations (Woll et al., 2016), however, this requires representation learning technique that exploits the full expressive power of hypergraphs and can generalize on unseen nodes from a partially observed hypergraph.
Recent work in the field of geometric deep learning have presented formulations on graph struc- tured data for the tasks of node classification (Kipf & Welling 2016), link prediction (Zhang & Chen 2018), or the classification of graphs (Zhang et al. 2018bb. Subsequently, for data containing learning approaches on similar tasks (Yadati et al.) 2019 Feng et al. 2019). A common implicit premise in these pa- pers is that a hypergraph can be viewed as a specific type of regular graph. Therefore, reduction of hypergraph learning problem to that of graph should suffice. Strategies to reduce hypergraph to a graph include transforming the hyperedges into multiple edges using clique expansion (Feng et al. 2019 Jiang al. 2019 /Zhan et al. 2018a), converting to heterogeneous graph using star
higher-order relations, a few recent papers have presented hypergraph-bart

---

Under review as a conference paper at ICLR 2021
star expansion
clique expansion
Hypergraph
(a)
(b)
Figure 1: (a) Example showing reduction of a hypergraph to a graph using clique and star expansion methods. The clique expansion loses the unique information associated with the hyperedge defined by the set of nodes {u2, v3). and cannot distinguish it from the hyperedge defined by the nodes {11, 12, v.). Star expansion creates a heterogeneous graph that is difficult to handle using most well-studied graph methods (Hein et af.) 2013). (b) Schematic representations of two Fano planes comprising 7 nodes and 7 hyperedges (6 straight lines and I circle.). The second Fano plane is copy of the first with nodes V2 and v3 permuted. These two hypergraphs cannot be differentiated when transformed to graph using clique expansion.
expansion (Agarwal et al. 2006), and replacing every hyperedge with an edge created using a cer- tain predefined metric (Yadati et al. 2019). Yet these methods are based on the wrong premise, motivated chiefly by a larger availability of graph-based approaches. By reducing a hypergraph to regular graph, these approaches make existing graph learning algorithms applicable hypergraphs. However, hypergraphs are not a special case of regular graphs. The opposite is true, regular graphs are simply a specific type of hypergraph (Berge Minieka 1976). Therefore, reducing the hyper- graph problem to that of a graph cannot fully utilize the information available in hypergraph. Two schematic examples outlining this issue are shown [ To address tasks based on complex structured data, hypergraph-based formulation is needed that complies with the properties of hypergraph.
A major limitation of the existing hypergraph learning frameworks is their inherently transductive nature. This implics that these methods can only predict characteristics nodes that were present in the hypergraph training time, and fail to infer previously unseen nodes. The transductive nature of existing hypegraph approaches makes them inapplicable in, for example, finding the most promising target audience for marketing campaign making movie recommendations with new movies appearing all the time. An inductive solution would pave the way to solve such problems using hypergraphs. The inductive learning framework must be able identify both the node's lo- cal role in the hypergraph, as well as its global position (Hamilton et al. 2017). This is important for generalizing the learned node embeddings that the algorithm has optimized on newly ob- served hypergraph comprising previously unseen nodes, thus, making inductive learning a far more complex problem compared to the transductive learning methods.
In this paper, we address the above mentioned limitations of the existing hypergraph learning meth- ods. We propose simple yet effective inductive learning framework for hy hypegrraphs that readily applicable to graphs as well. Our approach relies neural message passing techniques due to which can be used on hypergraphs of any degree of cardinality without the need for reduction to graphs. The points below highlight the contributions of this paper:
We address the challenging problem of representation learning on hypergraphs by propos- ing HyperSAGE, comprising a message passing scheme which is capable of jointly captur- ing the intra-relations (within hyperedge) as well as inter-relations (across hyperedges).
The proposed hypergraph learning framework is inductive, i.e. it can perform predictions on previously unseen nodes, and can thus be used to model evolving hypergraphs.
HyperSAGE facilitates neighborhood sampling and provides the flexibility in choosing dif- ferent ways to aggregate information from the neighborhood.
HyperSAGE is more stable than state-of- of-te--f-thearr methods, thus provides more accurate re- sults node classification tasks hypergraphs with reduced variance in the output.
2

---

Under review as a conference paper at ICLR 2021
2
RELATED WORK
Learning node representations using graph neural networks has been a popular research topic in the field of geometric deep learning (Bronstein et al., 2017). Graph neural networks can be broadly classified into spatial (message passing) and spectral networks. We focus on a family of spatial message passing graph neural networks that take a graph with some labeled nodes as input and learn embeddings for each node by aggregating information from its neighbors (Xu et al. 2019). Message passing operations in graph simply propagate information along the edge connecting two nodes. Many variants of such message passing neural networks have been proposed. with some popular ones including Gori et al. (2005); Li et al. (2015); Kipf & Welling (2016); Gilmer et al. (2017);
Hamilton et al. 2017).
Zhou et al. 2007) introduced learning hypergraphs to model high-order relations for semi- supervised classification and clustering of nodes. Emulating graph- based message passing frame- work for hypergraphs is not straightforward since hyperedge involves more than two nodes which makes the interactions inside each hyperedge more complex. Representing a hypergraph with a ma- trix makes rigid in describing the structures higher order relations (Li al. 2013). On the other hand, formulating message passing on a higher dimensional representation of hypergraph using ten- sors makes computationally expensive and restricts it to only small datasets (Zhang et al. 2019). Several tensor based methods do perform learning on hypergraphs (Shashua et al. 2006 Arya et al., 2019) however they are limited to uniform hypergraphs only.
To resolve the above issues. Feng et al. (2019) and |ai et al. (2020) reduce hypergraph to graph using clique expansion and perform graph convolutions on them. These approaches cannot uti- lize complete structural information in the hypergraph and lead to unreliable learning performance for e.g. classification, clustering and active learning (Li & Milenkovic| 2017 Chien et al. 2019). Another approach by Yadati et al. (2019), named HyperGCN, replaces hyperedge with pair- wise weighted edges between vertices (called mediators). With the use of mediators, HyperGCN can be interpreted
improved approach of clique expansion, and the best of our knowledge, is also the state-of-the-art method for hypergraph representation learning. However, for many cases such as Fano plane where each hyperedge contains at most three nodes, HyperGGN becomes equivalent to the clique expansion (Dong et al. 2020). In spectral theory of hypergraphs, methods have been pro- posed that fully exploit the hypergraph structure using non-linear Laplacian operators (Chan al., 2018 Hein et al. 2013). In this work, we focus on message passing frameworks. Drawing inspi- ration from GraphSAGE (Hamilton et al. 2017), we propose to eliminate matrix (or tensor) based formulations in our neural message passing frameworks, which not only facilitates utilization of all the available information in hypergraph, but also makes the entire framework inductive nature.
PROPOSED MODEL: HYPERSAGE
The core concept behind our approach is to aggregate feature information from the neighborhood of a node spanning across multiple hyperedges, where the edges can have varying cardinality. Be- low, first define some preliminary terms, and then describe our generic aggregation framework. This framework performs message passing at two-levels for a hypergraph. Further, for any graph- structured data, our framework emulates the one-level aggregation similar to GraphSAGE (Hamilton ett al. 2017). Our approach inherently allows inductive learning, which makes it also applicable on hypergraphs with unseen nodes.
3.1 PRELIMINARIES
Definition 1 (Hypergraph). A general hypergraph H can be represented as H[ (V, E, where
V = {11, 02..,, UN denotes set of N nodes (vertices) and E {01, C2 ..... Cg denotes a set of hyperedges, with each hyperedge comprising non-empty subset from X E R N denote the feature matrix, such that x, E X is the feature vector characterizing node v; E V. The maximum cardinality of the hyperedges in K is denoted as M = max/e].
eEE
Unlike in a graph, the hyperedges of H can contain different number of nodes and M denotes the largest number. From the definition above, we see that graphs are special case of hypergraphs with
3

---

Under review as a conference paper at ICLR 2021
M=2. Thus, compared to graphs, hypergraphs are designed model higher nodes. Further, we define three types of neighborhoods a hypergraph:
-igher-order order relations between
Definition 2 (Intra-edge neighborhood). The intra-edge neighborhood of node v, E V for any hyperedge e E & is defined as the set of nodes vj belonging to e and is denoted by N(U; e)
Further, let E(u)) {c E & Us c) be the sets of hyperedges that contain node U/3-
Definition 3 (Inter-edge neighborhood). The inter-edge neighborhood of a node v E V also re- ferred as its global neighborhood, is defined as the neighborhood of Us spanning across the set of hyperedges E(v;) and is represented by N(u;) UeEE(e)) N(v,,e)
Definition 4 (Condensed neighborhood). The condensed neighborhood of any node v; E V is a sampled set of < |e] nodes from hyperedge Eus) denoted by N e; C N(u,,
3.2 GENERALIZED MESSAGE PASSING FRAMEWORK
We propose interpret the propagation of information in a given hypergraph as two-level ag- gregation problem, where the neighborhood of any node is divided into intra-edge neighbors and inter-edge neighbors. For message aggregation, we define aggregation function F(-) as permuta- tion invariant set function on a hypergraph H = (V, E, X) that takes as input countable unordered message set and outputs a reduced or aggregated message. Further, for two-level aggregation, let Ti (.) and F2(-) denote the intra-edge and inter-edge aggregation functions, respectively. Schematic representation of the two aggregation functions is provided in Fig Similar to we also define Z as the encoded feature matrix built using the outputs z; of aggregation functions. Message passing node v; for aggregation of information the Td layer can then be stated as
x + Fi ((x by E N(u,, e;a))),
(1)
x + X + F2 ((x),)
(o) bl E E(u))),
(2)
(e)
where, x refers the aggregated feature set at v/ obtained with intra-edge aggregation for edge c.
The combined two-level message passing is achieved using nested aggregation function = F2. To ensure that the expressive power of a hypergraph is preserved or at least the loss is minimized, the choice of aggregation function should comply with certain properties.
360
02
Firstly, the aggregation function should able to capture the features of neighborhood ver- tices in a manner that is invariant to the per- mutation of the nodes and hyperedges. Many graph representation learning methods use per- mutation invariant aggregation functions, such as mean, sum and mar functions (Xu et al. 2019). These aggregations have proven be successful for node classification problems. For the existing hypergraph frameworks, reduction to simple graphs along with a matrix- message passing framework limits the possibil- ities of using different types of feature aggrega- tion functions, and hence curtails the potential to explore unique node representations.
Figure 2: Schematic representation of the two- level message passing scheme of HyperSAGE, with aggregation functions Fi and F2('). It shows information aggregation from two hyper- edges е and eg. where the intra-edge aggrega- tion from sampled sets of nodes (a = 5) for each hyperedge. For node U, x; and z; denote the input and encoded feature vector, respectively.
Secondly, the aggregation function should also preserve the global neighborhood invariance at the 'dominant nodes' of the graph. Here, dominant nodes refer to nodes that contain important features, thereby, impacting the learning process relatively more than their neighbors. The aggregation func- tion should ideally be insensitive the input, whether the provided hypergraph contains a few large
4

---

Under review as a conference paper at ICLR 2021
hyperedges, or larger number of smaller ones obtained from splitting them. Generally, hyperedge would be split in a manner that the dominant nodes are shared across the resulting hyperedges. In such cases, global neighborhood invariance would imply that the aggregated output at these nodes before and after the splitting of any associated hyperedge stays the same. Otherwise, the learned representation of node will change significantly with each hyperedge split.
Based on these considerations, we define the following properties for a generic message aggregation function that should hold for accurate propagation of information through the hypergraphs.
Property 1 (Hypergraph Isomorphic Equivariance). A message aggregation function F(-) is equivariant to hypergraph isomorphism, if for two isomorphic hypergraphs H. (V,E, E, X) and = OC = (V*, e. x²), given that D(" = H. and Z and Zz represent the encoded feature matrices
obtained using F-)) on H and JC*, the condition Z a o holds. Here, a denotes a permutation operator on hypergraphs.
Property 2 (Global Neighborhood Invariance). A message aggregation scheme F(-) satisfies global neighborhood invariance at any node operation ((), such that K* = =THH), and z; and " denote the encoded fecodedeeature vectors obtained using F()) at node Vs on H and HC, the condition zi 2, holds. Here T(H) could refer to operations such as hyperedge contraction or expansion.
V for a given hypergraph H = (V. &) for any
The flexibility of our message passing framework allows us go beyond the simple aggregation functions on hypergraphs without violating Property I. We introduce a series of power mean func- tions as aggregators, which have recently been shown generalize well graphs (Li et al. 2020). We perform message aggregation in hypergraphs using these generalized means, denoted by Mr and provide in section 4.2, study their performances. We also show that with appropriate combina- tions of the intra-edge and inter-edge aggregations Property 2 is also satisfied. This property ensures that the representation of node after message passing is invariant the cardinality of the hyper- edge, i.c., the aggregation scheme should not be sensitive to hyperedge contraction or expansion, as long as the global neighborhood of a node remains the same in the hypergraph.
Aggregation Functions. One major advantage of our strategy is that the message passing module decoupled from the choice of the aggregation itself. This allows our approach to be used with a broad set of aggregation functions. We discuss below few such possible choices.
Generalized means. Also referred to as power means, this class of functions are very commonly used for getting an aggregated measure over a given set of samples. Mathematically, generalized
means can be expressed as Mい = ( Σi= xf , where n refers to the number of samples in the aggregation, and p denotes its power The choice of p allows providing different interpretations to the aggregation function. For example, mean squared estimate and large value of p corresponds to max pooling from the group. Similarly, Mp can be used for geometric and harmonic means with p - 0 and -1, respectively.
denotes arithmetic mean aggregation, p = 2 refers to
Similar to the recent work of Li et al.] (2020), we use generalized means for intra-edge as well as inter-edge aggregation. The two functions F-)-) and F2(-) for aggregation at node v; is defined as
510 (s)
Σ
x
(3)
Niw e)(s](
N(u,, em)
By .n)
(s) (s)(s))
Σ (s)
(4)
E(v;)]
OE ((R)
where we use 's' for concise representation of the unordered set of input as shown in Eq Here and henceforth in this paper, remove the superscript index '(i)" for the sake of clarity and further occurrences of the two aggregation functions shall be interpreted in terms of node U;- Note that in Eq. 3 and Eq. we have chosen the power term p to be same for F1 and F2 so as to satisfy the global neighborhood invariance as stated in Property 2. Note, the scaling term added to Fj is added to balance the bias in the weighting introduced in intra-edge aggregation due varying
5

---

Under review as a conference paper at ICLR 2021
cardinality across the hyperedges. These restrictions ensure that the joint aggregation F2(+) satisfies the property of global neighborhood invariance at all times. Proof the two aggregations satisfying Property 2 is stated in Appendix
Sampling- based Aggregation. Our neural message passing scheme provides the flexibility to adapt the message aggregation module to fit the desired computational budget through aggregating infor- mation from only a subset (U; C; a:) of the full neighborhood N N c). if needed. We propose to apply sub-sampling only on the nodes from the training set, and use information from the full neigh- borhood for the test set. The advantages of this are twofold. First, reduced number of samples per aggregation training time reduces the relative computational burden. Second, similar to dropout (Srivastava et al., (014)), it serves to add regularization to the optimization process. Using the full neighborhood on test data avoids randomness in the test predictions, and generates consistent output.
3.3 INDUCTIVE LEARNING ON HYPERGRAPHS
HyperSAGE is general framework for learning node representations on hypergraphs, on even unseen nodes. Our approach uses neural network comprising I. layers, and feature-aggreeation is performed at each of these layers, as well as across the hyperedges.
Algorithm describes the forward propagation mechanism which implements the aggregation function F(-) F2(-) described above. At each iteration, nodes first aggregate information from their neighbors within specific hyper- edge. This is repeated over all the hyperedges across all the L layers of the network. The trainable weight matrices W' with E L are used aggregate information across the feature dimension and propagate it through the various layers of the hypergraph.
Algorithm HyperSAGE Message Passing
Input : (V, E, X); depth L; weight matri- ces W for 1 L: non- non-linearity o;
intra-edge aggregation function Fı(-;; inter-edge aggregation function F2(-)
Output: Node embeddings 2;| v. V
h + x, X for 1o. do
0/- E V
for E & do
h + h for U; e
Generalizability of HyperSAGE. Hyper- SAGE can be interpreted generalized for- mulation that unifies various existing graph- based as well as hypergraph formulations. Our approach unifies them, identifying each of these as special variants/cases of our method. We dis- cuss here briefly the two popular algorithms.
h + h (s)
end
end
h a (W
8 U
end + h E
Graph Convolution Networks (GCN), The GCN approach proposed by Kipf & Welling (2016) a graph-based method that can be derived as special case of HyperSAGE with maximum
cardinality [M| = 2, and setting the agggregation function F2 = Mp with = 1. This being a graph-based method, F, will not be used.
GraphSAGF. Our approach, when reduced for graphs using M = 2, similar to GraphS For exact match, the aggregation function F. should be one of mean, mar L.TM. Further, the sampling term can be adjusted to match the number of samples per aggregation as in GraphSAGE.
EXPERIMENTS
4.1 EXPERIMENTAL SETUP
For the experiments in this paper, we use co-citation and co-authorship network datasets: CiteSeer, PubMed, Cora (Sen al. 2008) and DBLP (Rossi & Ahmed, 2015). The task for each dataset is to predict the topic to which a document belongs (multi-class classification). For these datasets, x; corresponds to bag of words such that vijj E x; represents the normalized frequency of occurence of the jth word. Additional details related the hypergraph topology are presented in Appendix

---

Under review as a conference paper at ICLR 2021
Table 1: Performance HyperSAGE and other hypergraph learning methods on co-authorship and
co-citation datasets.
Co-authorship Data
Co-citation Data
Method
DBLP
Cora
Pubmed
Citeseer
Cora
MLP + HLR
63.6 + 4.7
59.8 + 4.7
64.7 + 3.1
56.1 + 2.6
61.0 + 4.1
HGNN
69.2 5.1
63.2 3.1
66.8 3.7
56.7 3.8
70.0 2.9
FastHyperGCN
68.1 + 9.6
61.1 + 8.2
65.7 + 11.1
56.2 8.1
61.3 + 10.3
HyperGCN
70.9 8.3
63.9 + 7.3
68.3 9.5
57.3 + 7.3
62.5 = 9.7
HyperSAGE (p = 2)
71.5 4.4
69.8 + 2.6
71.3 2.4
59.8 + 3.3
62.9 = 2.1
HyperS AGE (p- 1)
77.2 + 4.3
72.4 + 1.6
72.6
61.8 + 2.3
69.3 + 2.7
HyperS AGE (p- = 0.01)
77.4
3.8
72.1 + 1.8
72.9 1.3
61.3 + 2.4
68.2 + 2.4
HyperSAGE (p- -1)
70.9 + 2.3
67.4 + 2.1
68.3
+ 3.1
59.8 + 2.0
62.3 + 5.7
A.2
Further, for all experiments, we use a neural network with 2 layers. All models are implemented Pytorch and trained using Adam optimizer. See Appendix A.2 for implementation details.
4.2 SEMI-SUPERVISED NODE CLASSIFICATION ON HYPERGRAPIS
Performance comparison with existing methods. We implemented HyperSAGE for the task of semi-supervised classification of nodes hypergraph, and the results are compared with state- oh-the art methods. These include (a) Multi-layer perceptron with explicit hypergraph Laplacian regularisation (MIP HLR), (b) Hypergraph Neural Networks (HGNN) (Feng al. (019) which uses a clique expansion, and (c) HyperGCN and its variants (Yadati et al.) 2019) that collapse the hyperedges using mediators. For HyperSAGE method, we use 4 variants of generalized means Mp with p 1, 2, -1 and 0.01 with complete neighborhood i.c., a = -el. For all the cases, 10 data splits over 8 random weight initializations are used, totalling experiments per method and for every dataset. The data splits are the same as in HyperGCN described in Appendix A.
Table sshows the results obtained for the node classification task. We see that the different variants of HyperSAGE: consistently show better scores across our benchmark datasets, except Cora co- citation where no improvement is observed compared to HGNN. Cora co-citation data relatively small in size with cardinality of 3.0 + 1.1, and we speculate that there does not exist enough scope of improving with HyperS beyond what HGNN can express with the clique expansion.
For the larger datasets such DBLP and Pubmed, we see that the improvements obtained in per- formance with HyperSAGE over the best baselines are 6.3% and 4.3% respectively. Apart from its superior performance, HyperSAGE is also stable, and is less sensitive to the choice of data split and initialization of the weights. This is evident from the scores of standard deviation (SD) for the vari- experiments in Table [ and there is a significant gain performance compared to HyperGCN. Another observation that the HyperGCN method is very sensitive to the data splits well as initializations with very large errors in the predictions. This is even higher for the FastHyperGCN variant. Also, we have found that all the choices of p work well with HyperSAGE for these datasets. We further perform more comprehensive study analyzing the effect of on model performance later in this section.
We that the SD scores for our method are lower than other methods,
Stability analysis. We further study the sta- bility of our method in terms of the variance observed in performance for different ratios of train and test splits, and compare results with
Acouncy
ACOMFOC
that of HyperGCN implemented under similar
settings. Fig. 3 shows results for the two learn- ing methods on 5 different train-test ratios. We that the performance of both models im- proves when a higher fraction of data is used for training, and the performances are approx-
TradecTent Barto
Trainceedt
Figure 3: Accuracy scores for HyperSAGE and HyperGCN obtained for different train-test ratios for multi-class classification datasets.
7

---

Under review as a conference paper at ICLR 2021
Table Performance of HyperSAGE for multiple values of generalized means aggregator (Mp) on varying number of neighborhood samples (a:).
DBLP
Pubmed
2
a
3
5
10
Q
2
Q =
3
5
10
-1
59.6
61.2
69.9
70.9
60.1
60.2
67.9
66.4
p = 0.01
61.2
64.8
73.1
77.4
65.5
67.4
73.4
72.9
1
62.3
64.5
73.1
77.2
64.8
64.3
72.2
72.6
= 63.1
63.8
71.9
71.5
63.7
63.9
70.8
71.3
62.7
63.6
71.3
71.4
62.2
61.3
70.1
67.9
= 62.8
63.3
69.4
70.6
62.1
60.4
69.3
68.0
Table 3: Performance of HyperSAGE and its variants on nodes which part of the training hypergraph (seen) and nodes which were not part of the training hypergraph (unseen).
DBLP
Pubmed
Citeseer
Cora (citation)
Method
Seen
Unseen
Seen
Unseen
Seen
Unseen
Seen
Unseen
MLP + HLR
64.5
58.7
66.8
62.4
60.1
58.2
65.7
64.2
HyperSAGE (p- 0.01)
78.1
73.1
81.0
80.4
69.2
67.1
68.2
65.7
HyperSAGis (p 1)
78.1
73.2
78.5
76.4
69.3
67.9
71.3
66.8
HyperSAGE (p 2) 76.1
70.2
71.2
69.8
65.9
63.8
65.9
64.5
imately the same at the train-test ratio of 1/3.
However, for smaller ratios, we see that HyperSAGE outperforms HyperGCN by significant mar- gin across all datasets. Further, the standard deviation for the predictions of HyperSAGE are signif- icantly lower than that of HyperGCN. Clearly, this implies that HlyeerSAGB is able to better exploit the information contained in the hypergraph compared to HyperGCN, and can thus produce more accurate and stable predictions. Results Cora and Citeseer can be found in Appendix C
Effeet of generalized mean aggregations and neighborhood sampling, We study here the effect of different choices of the aggregation functions (-) and F2(-) the performance of the model. Further, we also analyze how the number of samples chosen for aggregation affect its performance. Aggregation functions from M, are chosen with p 1, 2, 3, 0.01 and -1, and to comply with global neighborhood invariance, we use aggregation function as in Eq. 4 The number of neighbors for intra-edge aggregation are chosen to be 2, 3, and 10. Table 2 shows the accuracy scores obtained for different choices of and a on DBLP and Pubmed datasets. For most cases, higher value of p reduces the performance of the model. For a 2 on DBLP, performance seems to be independent of the choice of A possible explanation could be that the number of neighbors is very small, and change in p does not affect the propagation information significantly. An exception is
1, where the performance drops for all cases. For Pubmed, the choice of p seems to be very important, and we find that p = 0.01 seems to fit best.
We also see that the number of samples per aggregation can significantly affect the performance of the model. For DBLP, model performance increases with increasing value a. However, for Pubmed, observe that performance improves up to 5, but then slight drop is observed for larger sets of neighbors. Note that for Pubmed, the majority of the hyperedges have cardinality less than equal 10. This means that during aggregation, information will most often be aggregated from all the neighbors, thereby involving almost no stochastic sampling. Stochastic sampling of nodes could serve ever, at = 10, is almost absent, due to which the noise in the data affects the performance of the model which is not the case in DBLP.
regularization mechanism and reduce the impact of noisy hyperedges. How-
4.3 INDUCTIVE LEARNING ON EVOLVING GRAPHS
For inductive learning experiment, we consider the case of evolving hypergraphs. We create inductive learning datasets from DBLP, Pubmed, Citeseer and Core (co-citation) by splitting each
8

---

Under review as a conference paper at ICLR 2021
of the datasets into train-test ratio of 1:4. Further, the test data is split into two halves: seen and unseen. The seen test set comprises nodes that are part of the hypergraph used for representation learning. Further, unseen nodes refer to those that never a part of the hypergraph during training. To study how well HyperSAGis generalizes for inductive learning, we classify the unseen nodes and compare the performance with the scores obtained on the seen nodes. Further, results on unseen nodes with those of MLP+HLR. The results are shown in Table [ We see that results obtained with HyperSAGE on unseen nodes are significantly better than the baseline method. Further, these results seem to not differ drastically from those obtained on the seen nodes, thereby confirming that HyperSAGE can work with evolving graphs as well.
also compare our
CONCLUSION
We have proposed HyperSAGE, generic neural message passing framework for inductive learning on hypergraphs. The proposed approach fully utilizes the inherent higher-order relations in a hyper- graph structure without reducing to a regular graph. Through experiments several representative datasets, we have shown that HlyperSAGE outperforms the other methods for hypergraph learning. Several variants of graph-based learning algorithm such as GCN and GraphSAGE from the flexible aggregation and neighborhood sampling framework, thus making HyperSAGE universal framework for learning node representations on hypergraphs as well as graphs.
be derived
REFERENCES
Proceedings of the 23rd international conference on Machine learning, pp. 17-24, 2006. Sameer Agarwal, Kristin Branson, and Serge Belongie. Higher order learning with graphs In
Devanshu Arya, Stevan Rudinac, and Marcel Worring. Hyperleam:: a distributed approach for repre- sentation learning in datasets with many modalities. In Proceedings of the 27th ACM International Conference on Multimedia, pp. 2245-2253 2019.
Song Bai, Feihu Zhang, and Philip HS Torr. Hypergraph convolution and hypergraph attention. Pattern Recognition, pp. 107637, 2020.
C. Berge and E. Minicka. Graphs and Hypergraphs. North-Holland mathematical library. Holth-Holland Publishing Company, 1976. ISBN 9780720424539. URL https: ggooge
Michael M Bronstein, Joan Bruna, Yann LeCun, Arthur Szlam, and Pierre Vandergheynst, Geomet- ric deep learning: going beyond euclidean data. IEEE Signal Processing Magazine, 34(4):18-42, 2017.
T-H Hubert Chan, Anand Louis, Zhihao Gavin Tang, and Chenzi Zhang. Spectral properties of hypergraph laplacian and approximation algorithms. Journal of the ACM (JACM), 65(3):1-48, 2018.
I Eli Chien, Huozhi Zhou, and Pan Li. h.s: Active learning over hypergraphs with pointwise and pairwise queries. In The 22nd International Conference on Artificial Intelligence and Statistics, pp. 2466-2475, 2019.
Yihe Dong, Will Sawin, and Yoshua Bengio. HNHN: Hypergraph networks with hyperedge neurons. arXiv preprint arXiv:2006.12278 2020.
Yifan Feng, Haoxuan You, Zizhao Zhang, Rongrong Ji, and Yue Gao. Hypergraph neural networks. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 33, pp. 3558-3565, 2019.
Justin Gilmer, Samuel Schoenholz, Patrick F Riley, Oriol Vinyals, and George E Dahl. Neural message passing for quantum chemistry. Proceedings of the 34th International Conference on Machine Learning. 2017.
Marco Gori, Gabriele Monfardini, and Franco Scarselli. A new model for learning in graph domains. In Proceedings. 2005 IEEE International Joint Conference on Neural Nenworks, 2005., volume 2, pp. 729-734. IEFE, 2005.

---

Under review as a conference paper at ICLR 2021
Shi Gu, Muzhi Yang. John D Medaglia. Ruben C Gur, Raquel E Gur, Theodore D Satterthwaite, and Danielle Bassett. Functional hypergraph uncovers novel covariant structures over neurodevel- opment. Human brain mapping. 38(8))3823-3835, 2017.
Xuemei Gu, Lijun Chen, and Mario Krenn. Quantum experiments and hypergraphs: Multiphoton sources for quantum interference, quantum computation, and quantum entanglement. Physical Review A, 101(3)003386, 2020.
Will Hamilton, Zhitao Ying, and Jure Leskovec. Inductive representation learning large graphs. In Advances in neural information processing systems, pp. 1024-1034, 2017.
Matthias Hein, Simon Setzer, Leonardo Jost, and Syama Sundar Rangapuram. The total variation on hypergraphs-lcarning on hypergraphs revisited. In Advances in Neural Information Processing Systems, pp. 2427-2435, 2013.
Jianwen Jiang, Yuxuan Wei, Yifan Feng. Jingxuan Cao, and Yue Gao. Dynamic hypergraph neural networks. In IJCAI, pp. 2635-2641, 2019.
Thomas N Kipf and Max Welling. Semi-supervised classification with graph convolutional net- works. ICLR, 2016.
Guohao Li, Chenxin Xiong, Ali Thabet, and Bernard Ghanem. Deepergen: All you need to train deeper gens, arXiv preprint arXiv:2006.07739, 2020.
Guoyin Li, Liqun Qi, and Gaohang Yu. The 7-eigenvalues of a symmetric tensor and its application to spectral hypergraph theory. Numerical Linear Algebra with Applications, 101 2013.
Pan Li and Olgica Milenkovic. Inhomogeneous hypergraph clustering with applications. In Ad- vances in Neural Information Processing Systems, pp. 2308-2311 2017.
Yujia Li, Daniel Tarlow, Mare Brockschmidt, and Richard Zemel. Gated graph sequence neural networks. arXiv preprint arXiv:1511.0549) 2015.
Ryan A. Rossi and Nesreen K. Ahmed. The network data repository with interactive graph analytics and visualization. In AAAI, 2015. URL.pttpp//nettworkrpppp itory. com
Prithviraj Sen, Galileo Namata, Mustafa Bilgic, Lise Getoor, Brian Galligher, and Tina Eliassi-Rad. Collective classification in network data. Al magazine, 29(3):33-93, 2008
Amnon Shashua, Ron Zass, and Tamir Hazan. Multi-way clustering using super-symmetric non- negative tensor factorization. In European conference on computer vision, pp. 595-608. Springer, 2006.
Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, llya Sutskever, and Ruslan Salakhutdinov. Dropout: a simple way to prevent neural networks from overfitting. The journal of machine learning research, 15(1):1929-199 2014.
Shulong Tan, Jiajun Bu, Chun Chen, Bin Xu, Can Wang, and Xiaofei He. Using rich social media information for music recommendation via hypergraph model. ACM Transactions on Multimedia Computing, Communications, and Applications (TOMM), 7(1):1-22, 2011.
Michael Wolf, Alicia M Klinvex, and Daniel Dunlavy. Advantages to modeling relational data using hypergraphs versus graphs. In 2016 IEEE High Performance Extreme Computing Conference (HPEC), pp. 1-7. IEEE, 2016.
Zonghan Wu, Shirui Pan, Fengwen Chen, Guodong Long, Chengqi Zhang, and S Yu Philip. A comprehensive survey on graph neural networks. IEEE Transactions on Neural Networks and Learning Systems, 2020.
Keyulu Xu. Weihua Hu, Jure Leskovee, and Stefanie Jegelka. How powerful are graph neural networks? International Conference Learning Representations (ICLR), 2019.
10

---

Under review as a conference paper at ICLR 2021
Naganand Yadati, Madhav Nimishakavi, Prateek Yadav, Vikram Nitin, Anand Louis, and Partha Talukdar. Hypergen: A new method for training graph convolutional networks on hypergraphs. In Advances in Neural Information Processing Systems, pp. 1511-1522, 2019.
Muhan Zhang and Yixin Chen. Link prediction based graph neural networks. In Advances in Neural Information Processing Systems, pp. 5165-5175, 2018.
Muhan Zhang. Zhicheng Cui, Shali Jiang, and Yixin Chen. Beyond link prediction: Predicting hyperlinks in adjacency space. AAAI, volume 1, pp. 2018a.
Muhan Zhang, Zhicheng Cui, Marion Neumann, and Yixin Chen. An end-to-end deep learning architecture for graph classification. In Thirty-Second AAAI Conference Artificial Intelligence, 2018b.
Songyang Zhang, Zhi Ding, and Shuguang Cui. Introducing hypergraph signal processing: the- oretical foundation and practical applications. IEEE Internet of Things Journal, 7(1):639-660, 2019.
Dengyong Zhou, Jiayuan Huang, and Bernhard Schölkopf. Learning with hypergraphs: Clustering, classífication, and embedding. In Advances in neural information processing systems, pp. 1601 1608, 2007.

---

Under review as a conference paper at ICLR 2021
APPENDICES
A EXPERIMENTS: ADDITIONAL DETAILS
perform multi-class classification on co-authorship and co-citation datasets, where the task is to predict the topic (class) for each document.
A.1 DATASET DESCRIPTION
Hypergraphs are created on these datasets by assigning each document as a node and each hyperedge represents (a) all documents co-authored by an author in co-authorship dataset and (b) all documents cited together by document in co-citation dataset. Each document (node) is represented by bag- of-words features. The details about nodes, hyperedges and features is shown Table 4 We use the same dataset and train-test splits as provided by Yadati et al. (2019) in their publically available implementation
Table 4: Details of real-world hypergraph datasets used in work
Co-authorship Data
Co-citation Data
DBLP
Cora
Pubmed
Citeseer
Cora
Nodes (V)
43413
2708
19717
3312
2708
Hyperedges ((E)
22535
1072
7963
1079
1579
average hyperedge size
4.7:16.1
4.2:14.1
4.3 + 5.7
3.2:+2.00
3.0 + 1.1
number of features, |
1425
1433
500
3703
1433
number of classes
6
7
3
A.2 IMPLEMENTATION DETAILS
We the following of hyperparameters similar the prior work by Kipf & Welling (2016) for all the models.
hidden layer size: 32
dropout rate: 0.5
learning rate: 0.01
weight decay: 0.0005
number of training epochs: 150
A for explicit Laplacian regularisation: 0.001
B CHOICE OF INTER-EDGE AND INTRA-EDGE AGGREGATIONS
Proof. For any given hypergraph H, (V, E, X), e denote node which global neighbor- hood equivariance exists. The aggregation output 3 (s) at eú can then be written using generalized means Mn as
Pi
(i(s)=
[N()))
Σ
(5)
(yEN(
To reiterate here, denotes the unordered set of input shown in Eq. aggregation F2(-) can he stated as
5 Further, the inter-edge
'HyperGCN Implementation: http:///ithaaaaoommmlllaaallttht
12

---

Under review as a conference paper at ICLR 2021
mm
(b)
(a)
Figure 4: (a) Example showing node v shared across hyperedges. (b) Hyperedge is split into r hyperedges to reduce the cardinality of eg. Note that the global neighborhood of es still remains the same, however its intra-edge neighborhood has changed due to such splitting.
P
P2
F2(s)
Σ
(6)
[B(c))]
[N(
eEE(ve
eNEN
This equation can be rewritten as
(Ariiiee Call
12
F2(s)
Σ x mg)
Σ
Σ
x5"
[[vs]]
No e)
e E(ve)
MyEM(N
Further, let
22
[N(us,e)
Σ
x
(8)
eEB(rd),
()EN(o
then Eq. can be rewritten
(M cx) N)
F2(s) = [(wi)]
(9)
Let assume that hyperedge Bg is split into hyperedges given by El eg) (e, egp eg. Stating the aggregation on the new set of hyperedges as F2(s), we assemble the contribution from this new set of hyperedges with added weight terms wy as stated below.
22
F2(s) = [E(s))
(p((us.e) e)
Σ
wyx?
+ V
(10)
eEE(v..),)
eyEN(e,,e)
For the property of global neighborhood invariance to hold at Us the following condition should be satisfied: (r;)) = F2()). Based this, we would like to solve for the weights wy. For this, equate the two terms and obtain
22
Σ
x!
Σ
Σ
wyxy
(1I)
(N(cr, en)l
e]]
(yEN(eLee))
ecE(e,
by e)
13

---

Under review as a conference paper at ICLR 2021
We further solve for the variables P1. P2 and w where Eq. 11 holds. For the sake of clarity, first simplify Eq. 11 using the following substitutions:
Pi [N(r and Bmg ])
where the index m here is used to refer to the m hyperedge from among the hyperedges obtained on splitting ey. Further, let 2j x for Vy E N(v,, e4) and žmj = x for vj EN(U,, em) and
Cm E(U; Cg).
Based on these substitutions, Eq. I can be restated as
(31 + 22 +... 2N
(8 + $1222 + ... ... 8152, + .... ... NIN=N) 2N)
+ (82121 + B2222 + + B2zz t... + B2N 2
+ (Br121 + Br222 +... Brgzz + +...+ BrNNN)
+
(12)
We seek general solutions for wj and a which holds for all values of 2j E [0, 1] since every element the normalized feature vectors X, lies in [0, 1].
For generalized solution, the coefficients of 2, the right should be equal to the coefficient of 2j on the left. The term on the left can reformulated as
g (21 +
2 80 + (22 + +... )
(13)
Consider the case when |21| real co-efficients.
132 + 23 + ... we expand Eq. 13 using binomial expansion for
(21 (2) + (22 + 23 + ....)) 3r (0)( o + (a)
21
(22 23
+
a
+ a-
21 (22 + 23 + +.. zN))
(2) + az a- + 2 .....4 20-1222 +x
+ 1 + 23 +
222 1-
(14)
Without any loss of generality, we consider splitting of hyperedge eg into r hyperedges such that nodes Uy and Vy are not contained in the same hyperedge anymore. This implies that RHS in Eq. 14 should not contain product terms of ž and 22. Hence, the term 2 2011 22 should be such that a
pl p2
(15)
Putting 1 and comparing the coefficients in Eq 12 we get
811 + B12 + ... 321 + 322
Br1 + B,2 +...
Σ
(16)
NN eg)
Niu em)
-1
Wy [N(e;, eg)l * Σ N(ee, em)
(17)
Thus, an edge e, is split into multiple edges Elv,, eq), then for the two aggregations to hold, the conditions are Pl P2 and w, [e
A c e E(v, c,).
While we provide above description related to splitting a certain hyperedge e, into r hyperedges, the derived results can be used to compute global neighborhood itself on any given node v;. Similar to cy above, node v; together with its global neighborhood (counted as N(u;)) can be interpreted as virtual hyperedge that has been split into number of hyperedges that actually exist and contain 2; These resultant hyperdges are equivalent to the r hyperdges obtained after splitting, as stated above.
14

---

Under review as a conference paper at ICLR 2021
C
STABILITY TEST ON CORA AND CITESEER
Cora
Citeseer
90-
80
70-
Accuracy
60
1330
1/20
1/15
1/10
M/3
40
1750
1/30
1/25
1/10
Train:Test Ratio
Train:Test Ratio
Figure 5: Results on cora and citeseer for multiple train test ratio
15


Reviews:
[{"review_id": "lgKRXBb45Th", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "pros.\n-The paper proposed a simple and flexible message passing method for hypergraphs.\n-By performing semi-supervised node classification on 4 popular datasets, authors show their method outperforms existing methods.\n-The paper is well-organized and clearly written. To the best of my knowledge, the method is technically sound.\n\ncons\n-The novelty of the paper seems limited. The proposed method defined in Equ. (1) and (2) is a straightforward extension of the message-passing method for ordinary graphs. \n- In Equ. (3): N(v, e) -> |N(v_i, e)|, N(v_i) -> |N(v_i)|\n\ncomments\nFrom Tab. (1),  it seems the reported performance of hypergraph-based methods is not better than the results of GAT, GCN. So, why should we use hypergraph-based methods?", "extra_scores": {}, "extra_text": {}}, {"review_id": "VsrOjoyx3Iz", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Summary \nIn this paper, the authors study the problem of learning node embeddings for hypergraphs. While most of the existing studies consider reducing hyper-graphs into graphs, this paper studies learning embeddings directly on the hypergraphs using two stages of aggregations / sampling. The efficacy of the proposed method is illustrated in a semi-supervised as well as inductive setting, where the method achieves better performances than the baselines. \n\nOriginality. \nThis paper may be interpreted as extension of GraphSage towards hypergraphs. The main contribution is in the aggregation function, which has two levels: one over intra-edge neighbourhood, and other over inter-edge neighbourhood. This method is thus suitable for both transductive as well as inductive settings. \n\nSignificance.\nThe experimental results show that HyperSage achieves better performance than HyperGCN in the semi-supervised tasks, while also extends to inductive settings and compares favourably against the MLP+HLR baseline.  In this aspect, the results are significant and relevant to the community. \n\nClarity: \nThis paper is well written and easy to follow. The relevant baselines and related methods are discussed appropriately. There are quite a few typos and some lack of clear notations, which are listed below:\n\n\t- Intra edge neighborhood - definition unclear - How is v_i used in the definition of N(v_i, e). \n\n\t- Typo in eqn 2, should have been e \\in E(v_i)\n\n\t- Typo in eqn 3, the second equation has ratio of two sets, which ideally should have been ratio of cardinality of two sets. \n\n\t- Algorithm 1, line 4, initialization of h_i^l from h_i^{l-1} should be outside the for loop, otherwise it may get reset again repeatedly. \n\n\nQuestions to authors:\n\n\t- HyperSage has less deviation in the results than other methods such as HyperGCN. Any reasons ?\n\n\t- In the stability analysis paragraph, it is shown that as the train to test ratio improves to 1/3, HyperGCN and HyperSage have nearly same performances. Would be interesting to see how it behaves as we increase the training ratio further. Also, any insights into why this behaviour happens would be useful.\n\n\t- Also, what is the specific train-test ratio which is used in reporting Table 1? It would be better to include that in the main paper, since there is a discussion on train-test already present.\n\n        - What are the other possible baselines for comparison in the inductive setting (Bai.et.al (Pattern Recognition, 2020) ? )\n\n        - From the results presented in Tables 1,2,3, it appears that the best performance is shared between p = 0.001 (equivalent to that  of Geometric mean aggregator) and p = 1(arithmetic mean aggregator). Any explanations of this would be useful. \n\n\nPros:\n\n       - An effective and simple extension of GraphSage towards HyperGraphs, which is suitable for inductive and transductive settings.\n\n       - Experiment results illustrating better performances than the baselines.\n\nCons:\n\n       - The novelty of the proposed method is limited to that of the aggregation functions.\n\n       - Some of the results mentioned lack explanation. \n", "extra_scores": {}, "extra_text": {}}, {"review_id": "PPk6-7r4s1G", "reviewer": "AnonReviewer5", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Overall, this paper is well-written. However, the intuition of the proposed framework is not very clear, and the model efficiency needs to be analyzed and evaluated compared to other baselines.\n\nThis paper analyzed the key issues of the existing message-passing graph convolutional networks. That is, the multiply stacked layer might be over-fitting and over-smoothing. Thus it proposed to choose the neighbors from the entire graph based on the structure-aware and feature-aware relatedness rather than simply choosing the local neighborhood. However, the motivations of the proposed structure-aware and feature-aware teleport functions are not very convincing, and Table 3 shows that the performance improvement of TeleGCN might largely be induced by model architecture rather than the proposed TeleGCL. \n\nSpecifically, the pros and cons of this paper are summarized as follows.\nPros:\n[1] It proposed a two-level message-passing strategy for neighborhood aggregation in the hypergraph.\n[2] The proposed HyperSAGE framework is inductive.\n[3] The experiments demonstrate its effectiveness and robustness compared to baselines.\n\nCons:\n[1] It required the message aggregation scheme had the property of hypergraph isomorphic equivariance. Two question are that can the examples of Figure 2(b) be distinguished by the proposed scheme in Eq. (3), and can the proposed be guaranteed to distinguish the hypergraphs which not isomorphic?\n[2] It is not clear what MLP+HLR is in Table 3?\n[3] In Section 4.3, it is confusing why to split each of the datasets into a train-test ratio of 1:4. Figure 3 has shown that with such a train-test ratio, HyperSAGE might work just like the HyperGCN.\n[4] The efficiency of HyperSAGE needs to be confirmed empirically compared to baselines as it argued that the condensed neighborhood could help improve the training efficiency.\n[5] There is a typo in Eq. (3). Should it be |N(v_i, e)|/|N(v_i)|?\n", "extra_scores": {}, "extra_text": {}}, {"review_id": "4g4jDDFvJke", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper introduces HyperSAGE, a method to address the problem of representation learning for hypergraphs.\n\nThe chosen approach consists in defining aggregation functions adapted to this specific type of data. It allows inductive learning while methods based on reduction to regular graphs are inherently transductive.\n\nThe algorithm introduced in this manuscript achieves good accuracy scores compared to other recent methods for hypergraphs on several real datasets. It is also more stable.\n\n\nMain concerns:\n\n1/ The paper deserves a better presentation: the authors should be more precise and also more formal at some places of the paper, in particular to describe the problem under consideration and the model they develop.\n\nFor example:\n- The reader is expected to guess that nodes have features that are stored in a matrix (which supposes that all the nodes have the same features and are real-valued). Indeed, hypergraphs are only defined through their topology (see Def. 1). The feature matrix is then introduced without any detail and explanation on what it models (see Subsection 3.2). \n\n- The concept of aggregation function should be introduced in the paper (for example at the beginning of Subsection 3.2). The two aggregation functions are not defined. In eq. (1) and (2), the reader can understand that the input of an aggregation function\nis a subset of features. In Prop. 1, the input is a hypergraph (which has no feature if one reads Def. 1 to the letter).  In eq. (3), the input is a node.\n\n- Property 2 should be written in a formal way. In particular, I do not fully understand the relation between what is claimed to be proven in Appendix B and Prop. 2. In addition, if I understand correctly, the proof does not show that p_1 is equal to p_2:\n\"we first assume p_1=p_2\" under eq. (10), while the case p_1\\neq p_2 is not dealt with. Finally, an additional condition seems to be assumed in the proof while not mentioned in the main document (see sentence under eq. (11)).\n\nFor these reasons, Sections 2 and 3 (as well as Appendix B) are difficult to read and understand.\n\n2/ It seems to me that the recent literature on hypergraph-based learning approaches is not properly cited. The authors should have a look at the following papers, in particular at the second one that proposes inductive (and transductive) learning methods for hypergraphs.\n- \"Hyperedge2vec: Distributed representations for hyperedges\", A Sharma, S Joty, H Kharkwal, J Srivastava (Preprint, 2018)\n- \"Deep Hyperedges: a Framework for Transductive and Inductive Learning on Hypergraphs\", J Payne, published in Sets & Partitions Workshop at NeurIPS 19.\n\n\nQuestions:\n- What is a uniform hypergraph as mentioned in Section 2?\n- What is the global neighborhood of a node?\nIs that the union of intra-edge and inter-edge neighborhoods?\n\n\nFew typos:\n- page 3: space missing\nactive learning etc.(Li \n- page 5 (property 1)\nisomporphism\n- page 5\nThe two functions (...) is defined", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper proposes a learning framework for Hypergraphs. The proposed method can be viewed as generalisation of GraphSAGE to hyper graphs. Though the paper emphasises that there is significant differences between Hypergraphs and Graphs and hence new methods are required. However, the proposed methods are not significantly different than that used for Graphs. Thus the novelty seems to be limited and hence it is difficult to strongly argue for acceptance.

Decision: Reject