Equivariant Hypergraph Diffusion Neural Operators
Authors: Peihao Wang
Shenghao Yang
Yunyu Liu
Zhangyang Wang
Pan Li
Venue: ICLR 2023 poster (2023)
Keywords: Hypergraph Neural Network
Hypergraph Diffusion
Equivariant Network

Abstract: Hypergraph neural networks (HNNs) using neural networks to encode hypergraphs provide a promising way to model higher-order relations in data and further solve relevant prediction tasks built upon such higher-order relations. However, higher-order relations in practice contain complex patterns and are often highly irregular. So, it is often challenging to design an HNN that suffices to express those relations while keeping computational efficiency. Inspired by hypergraph diffusion algorithms, this work proposes a new HNN architecture named ED-HNN, which provably approximates any continuous equivariant hypergraph diffusion operators that can model a wide range of higher-order relations. ED-HNN can be implemented efficiently by combining star expansions of hypergraphs with standard message passing neural networks. ED-HNN further shows great superiority in processing heterophilic hypergraphs and constructing deep models. We evaluate ED-HNN for node classification on nine real-world hypergraph datasets. ED-HNN uniformly outperforms the best baselines over these nine datasets and achieves more than 2%$\uparrow$ in prediction accuracy over four datasets therein. Our code is available at: https://github.com/Graph-COM/ED-HNN.

Published as a conference paper at ICLR 2023
EQUIVARIANT HYPERGRAPH DIFFUSION NEURAL OP- ERATORS
Peihao "a Shenghao Yang 3, Yunyu Liu Zhangyang Wang's 1 Pan Li Universiity of Texas at Austin, GGorgiaa Tech, 'University of Waterloo, "Purdue University atw.:eaaann.ltsssangeeeeeeee
edu, waannghha.yygnuuwtteee..ee 1iu31 1iu3154@purdue.c edu, panl menli@@@techhech
ABSTRACT
Hypergraph neural networks (HNNs) using neural networks to encode hyper- graphs provide promising way to model higher-order relations in data and further solve relevant prediction tasks built upon such higher-order relations. However, higher-order relations in practice contain complex patterns and are often highly ir- regular. So, it is often challenging to design an HNN that suffices to express those relations while keeping computational efficiency. Inspired by hypergraph diffu- sion algorithms,, this work proposes a new HNN architecture named ED-HNN, which provably approximates any continuous equivariant hypergraph diffusion operators that can model wide range of higher-order relations. ED-HNN can implemented efficiently by combining star expansions of hypergraphs with standard message passing neural networks. ED-HNN further shows great supe- riority in processing heterophilic hypergraphs and constructing deep models. We evaluate ED-HNN for node classification nine real-world hypergraph datasets. ED-HNN uniformly outperforms the best baselines over these nine datasets and achieves more than 2%f in prediction accuracy over four datasets therein. Our https://github. http///gtttb..eteee cow www.:hah..oo//aaaaccooooooo COM/DD-NN
code is available at: https:
INTRODUCTION
Machine learning on graphs has recently attracted great attention in the community due to the ubiqui- tous graph-structured data and the associated inference and prediction problems (Zhu) 2005 Hamil- 2020 Nickel et al. 2015). Current works primarily focus graphs which can model only pairwise relations in data. Emerging research has shown that higher-order relations that involve
ton
more than entities often reveal more significant information in many applications (Benson al.
2021 Schaub et al.] 2021] Battiston et al.) 2020 Lambiotte et al. [0099] Lee et al. 2021). For exam- ple, higher-order network motifs build the fundamental blocks of many real-world networks (Man- gaa & Alon 2003) Benson et al. 2016: Tsourakakis et al. 2017 Li et al. 2017: Li & Milenkovic, web users more precise ways (Xia et al. [2021 Wang et al.) 2020 2021, 2022). To capture these higher-order rela- tions, hypergraphs provide dedicated mathematical abstraction (Berge 1984). However, learning algorithms on hypergraphs are still far underdeveloped as opposed to those on graphs.
2017). Session-based (multi-step step) behaviors often indicate the preferences
Recently, inspired by the success of graph neural networks (GNNs), researchers have started investi- gating hypergraph neural network models (HNNs) (Feng et al. 2019) Yadati et al. 2019 Dong et al.) 2020 Huang Yang 2021: Bai et al. 2021 Arya al., 2020). Compared with GNNs, designing HNNs more challenging. First, as aforementioned, higher-order relations modeled by hyperedges could contain complex information. Second, hyperedges real-world hypergraphs are often of large and irregular sizes. Therefore, how to effectively represent higher-order relations while efficiently processing those irregular hyperedges is the key challenge when to design HNNs.
In this work, inspired by the recently developed hypergraph diffusion algorithms (Li et al. 2020a Liu et al. 20216; Fountoulakis et al. 202 Takai et al.) 20200: Tudisco et al.) (021a) we design a novel HNN architecture that holds provable expressiveness to approximate a large class of hyper- graph diffusion while keeping computational efficiency. Hypergraph diffusion is significant due to
Correspondence Peihao Wang and Pan L.i.

---

Published as a conference paper at ICLR 2023
Hyperedge potentials
Hypergraph diffusion
ED-HNN
gl a ).n(H.)) where [tt]ee
hp
[Vg (H.)), (H=))
Feed with node feature AN
(+(M,))- p(( X (Aa
Features nodes
0.7
0:1
0,2
which universally represents permutation- aqquaaaaionnniiaiann
(halv E V]
0.7
0.6
0.6
D.5
diffusion (Vg(H,))r
Hyperadge
potentials permutation-inaan inwaariant functions to model higher-arder relations.
E.g. Diftusion appears between the top-2 ranted nodes in each hyperadge
Running message passing neural networks the bipartite representation the hypergraph
Hypergraph diffusion often uses ietpuutttttooiiniiiinna hyperedge potentials to model higher-orde relations. The gradients of those potentials determine the diffusion process and termed diffusion operators. Our ED-HNN can universally represent such operators by feeding node representations into the message from hyperedges to nodes. Such small changes make big differences in model performance.
Figure 1:
its transparency and has been widely applied to supervisd learning (Hein et al.
2013 Zhang
et al. 2017a| Tuuiiso et al. 2021a), ranking aggregation (Li & Milenkovic 2017 Chitra & Raphael
2019 network analysis (Liu et 2021b Fountoulakis al.. [022]] Takai et al.) 2020) and signal
processing (Zhang et al.) 2019 Schaub et al. [202]] and so on. However, traditional hypergraph dif-
fusion needs to first handcraft potential functions to model higher-order relations and then use their
gradients or some variants of their gradients as the diffusion operators to characterize the exchange of diffused quantities on the nodes within one hyperedge. The design of those potential functions often requires significant insights into the applications, which may not be available in practice.
We observe that the most commonly-used hyperedge potential functions are permutation invariant, which covers the applications where none of the nodes in a higher-order relation are treated as in- herently special. For such potential functions, we further show that their induced diffusion operators must be permutation equivariant. Inspired by this observation, we propose NN-parameterizeed architecture that is expressive to provably represent any permulation-equuvviiine edge diffusion operators, whose NN parameters can be learned in data-driven way. We also in- troduce an efficient implementation based on current GNN platforms Fey & Lenssen (2019); Wang al. (2019): We just need to combine a bipartite representation (or star expansion Agarwal et al. (2006); Zien et al. (1999). equivalently) of hypergraphs and the standard message passing neural network (MPNN) Gilmer et al. (0117). By repeating this architecture by layers with shared parame- ters, we finally obtain our model named Equivariant Diffusion- based HNN (ED-HNN). Fig. shows an illustration of hypergraph diffusion and the key architecture in ED-HNN.
continuous hyper-
To the best of our knowledge, we are the first one to establish the connection between the general class of hypergraph diffusion algorithms and the design of HNNs. Previous HNNs were either
less expressive to represent equivariant diffusion operators (Feng et al. 2019 Yadati et al. 2019
Dong et al. 2020; Huang & Yang. 2021 Chien et al., 2022 Bai et al., 2021 or needed to learn the representations by adding significantly many auxiliary nodes (Arya et al. 2020; Yadati 2020) Yang et al., 2020). We provide detailed discussion of them in Sec, 3.4 We also show that due to the capability of representing equivariant diffusion operators, ED-HNN is by design good at predicting node labels over heterophilic hypergraphs where hyperedges mix nodes from different classes. Moreover, ED-HNN can go very deep without much performance decay.
As an extra theoretical contribution, our proof of expressiveness avoids using equivariant polyno- mials as a bridge, which allows precise representations of continuous equivariant set functions by compositing a continuous function and the sum of another continuous function on each set entry, while previous works (Zaheer et al. 2017 (Segol & Lipman| 2020) have only achieved an approxi- mation result. This result may be of independent interest for the community.
We evaluate ED-HNN by performing node classsification over 9 real-world datasets that cover both heterophilic and homophilic hypergraphs. ED-HNN uniformly outperforms all baseline methods across these datasets and achieves significant improvement (>2% 1) over 4 datasets therein. ED- HNN also shows super robustness when going deep. We also carefully design synthetic experiments to verify the expressiveness of ED-HNN to approximate pre-defined equivariant diffusion operators.
PRELIMINARIES: HYPERGRAPHS AND HYPERGRAPH DIFFUSION
Here, we formulate the hypergraph diffusion problem, along the way, introduce the notations.
2

---

Published as a conference paper at ICLR 2023
Definition 1 (Hypergraph). Let G = (V,E,)) be attributed hypergraph where V, E are the node set and the hyperedge set, respectively. Each hyperedge e = [v] ..., 2 (e) is a subset of V. Unlike graphs, a hyperedge may contain more than two nodes. X == .... Fey 1 RR denotes the node attributes and v, denotes the attribute of node v. Define dy = Ief E E u eHl as the degree of node v. Let D, De denote the diagonal degree matrix for u V and the sub-matrix v E e.
Here, we use 1-dim attributes for convenient discussion while our experiments often have multi- dim attributes. Learning algorithms will combine attributes and hypergraph structures into (latent) features defined as follows, which can be further used to make prediction for downstream tasks.
Definition 2 (Latent features). Let he E R denote the (latent) features of node E V. The feature vector H [.... be. -.. E RR includes node features entries. Further, collect the features into hyperedge feature vector He -[ho.1
@Rlel, where he, corresponds the feature go for some node (0) E.. For any E,e, there is one corresponding index i E {1..... lel). Later, we may use subscripts (e, i) and (e, v) interchangeably if they cause no confusion.
he.ee] T
A widely- heuristic to generate features H is via hypergraph diffusion algorithms.
Definition 3 (Hypergraph Diffusion). Define node potential functions f(; R - for E V and hyperedge potential functions ge(-) : Rlel + R. for each E E. The hypergrah diffusion combines the node attibutes and the hypergraph structure and asks to solve
mim Σ f(hri xx,) -eec(H))
aEE
In practice, 9e is often shared across hyperedges of the same size. Later, we ignore the subscript e.
The two potential functions are often designed via heuristics in traditional hypergraph diffusion literatures. Node potentials often correspond to some negative-log kernels of the latent features and the attributes. For example, f(h,, x.,) could be (h,, xx,) when to compute hypergraph PageRank diffusion (Li et al. 2020a Takai et al. 2020). Hyperedge potentials are more significant and complex, as they need to model those higher-order relations between more than two objects, which makes hypergraph diffusion very different from graph diffusion. Here list a few examples. Example 1 (Hyperedge potentials). Some practical g (He) may be chosen as follows.
Clique Expansion (CE, hyperedges reduced to cliques) plus pairwise potentials Zhou et al. 2007;: (h. hh,) or with degree normalization En,oeo h- Vd.
h- (2 g He)).
Divergence to the mean (Tudicco et al.
202la Eveo (he
where || llp com-
putes the nn-norm
Total Variation (TV) Hein et al.
2013
Zhang et al. 2017a maax,, hh, h P. p E (1,
Lovász Extension Lovász
1983) for cardinality-based set functions (LEC) (Jegelka et al. 2013
Li et al. 2020a Liu et al.
2021bb s(He))), p E {1,2} where y [..., y.....
E R is
constant vector and ((H.)
sorts the values of H. in a decreasing order.
One may reproduce TV by using LEC and setting 211 = -Veel = 1. To reveal more properties of these hyperedge potentials g(-) later, we give the following definitions.
Definition 4 (Permutation Invariance & Equivariance). Function v/ RK + R is permutation- invariant if for any K-dim permutation matrix P E [[[K], (PPZ) = (Z)) for all Z E K
Function 3b EK + BK is germutation-equivarian if for any K dim permutation matrix P e
I1[K,, J (PZ) = ( for all Z E k
We may easily verify permutation invariance of the hyperedge potentials in Example I The under- lying physical meaning is that the prediction goal of an application is independent of node identities in hyperedge so practical often keep invariant w.r.t. the node ordering (Veldt et al. 2021
3 NEURAL EQUIVARIANT DIFFUSION OPERATORS AND ED-HNN
Previous design of hyperedge potential functions is tricky. Early works adopted clique or star expan- sion by reducing hyperedges into edges (Zhou al. 2007 Agarwal et al. 2006 Li & Milenkovic,
3

---

Published as a conference paper at ICLR 2023
2017) and further used traditional graph methods. Later, researchers proved that those hyperedge reduction techniques cannot well represent higher-order relations (Li & Milenkovic 201 Chien
et al. 2019). Therefore, Lovász extensions of set-based cut-cost functions
hyperedges have been
proposed recently and used as the potential functions (Hein et al. 2013 Li & Milenkovic 201 8 Li
et al., 2020a Takai al. 2020 Fountoulakis et al. 2021 Yoshida 2019). However, designing those
set-based cut costs is practically lyy hard and needs a lot of trials and errors. Other types of handcrafted
hyperedge potentials to model information propagation can also be found in (Neuhäuser al., 2022
2021]. which again are handcrafted and heavily based on heuristics and evaluation performance.
Our idea uses data-driven approaches to model such potentials, which naturally brings us to HNNs. On one hand, we expect to leverage the extreme expressive power of NNs to learn the desired hypergraph diffusion automatically from the data. On the other hand, we are interested having novel hypergraph NN (HNN) architectures inspired by traditional hypergraph diffusion solvers.
To achieve the goals, next, we show by the gradient descent algorithm (GD) or alternating direction method of multipliers (ADMM) (Boyd et al.) 2011), solving objective Eq. amounts to iteratively applying some hyperedge diffusion operators. Parameterizing such operators using NNs by each step can unfold hypergraph diffusion into an HNN. The roadmap is as follows.
In Sec. 3.1 we make the key observation that the diffusion operators are inherently permutation- equivariant. To universally represent them, in Sec. 3.2 we propose an equivariant NN-based opera- tors and use it to build ED-HNN via an efficient implementation. In Sec. 3.3 we discuss the benefits of learning equivariant diffusion operators. In Sec. 3.4 we review and discuss that previous
HNNs allows efficiently modeling such equivariant diffusion operators via their architecture.
3.1 EMERGING EQUIVARIANCE IN HYPERGRAPH DIFFUSION
We start with discussing the traditional solvers for Eq. | If ʃ and g are both differentiable, one straightforward optimization approach is to adopt gradient descent. The node-wise update of each iteration can be formulated as below:
h (++1) h n (VV x.) + [V for v e V,
(2)
eiDEe
where [V denotes the gradient w.r.t. h, for v E e. We use the supscript t to denote the number of the current iteration, h!! By is the initial features, and 1j is known as the step size.
For general f and 9, we may adopt ADMM: For each e E E, we introduce an auxiliary variable Qu = [qe,1
qesee]] E Blel, We initialize ! I. and (0) -H."" And then, iterate
Q (1+1) + prox ng (2H
Q!1) ( + ( e
(3)
h (t-1) - prox (/d Σ q
(1+1) /d,), for V,,,
(4)
exvEe is the proximal operator. The detailed derivation can be found in Appendix A The iterations have convergence guarantee under closed convex as- sumptions of f.g g (Boyd al. 2011). However, our model does rely on the convergence, as our model just runs the iterations with a given number of steps (aka the number of layers in ED-HNN). reviewed in Proposition I which enables the possibil- ity of NN-based approximation even for the case when ʃ and g are not differentiable. One noted non-differentiable example is the LEC case when g( H.)) ) = {y,5l H)))² 2 in Example
where prox (h) _ arg ming (022) + = hil2
The operator prox, (.) has nice properties
Proposition 1 (Parikh & Boyd 2014; Polson et al. 2015 If (-)) IR - IR is lower semi- continuous convex convex function, then prox is L-lipschitz continuous.
The node-side operations, gradient V (:; a.)) and proximal gradient prox, are relatively easy to model, while the operations on hyperedges are more complicated. We name gradient Vg(-)= Riel + (el and proximal gradient prox (r) Rla
Rl as hyperedge diffusion operators, since they summarize the collection of node features inside hyperedge and dispatch the aggregated information to interior nodes individually. Next, we reveal one crucial property of those hyperedge diffusion operators by the following propostion (see the proof in Appendix B.T.:
Proposition 2. Given any permutation-invarion hyperedge potential function g(-). hyperedge dif- fusion operators proxng(? (-) and Vg(,) are permutation equivariant.
4

---

Published as a conference paper at ICLR 2023
Algorithm I: ED-HNN
Initialization: H = X and three MLPs é, P, q (shared across layers).
For 2,1, L..,- I, do:
1. Designing the messages from &:
m my = (( ( for all u E V.
(t) (8) = Duee ma- for all E E. 2. Sum V- & messages over hyperedges m
Broadcast me and design the messages from & to V: m p(h,)) ( m for all v E e.
4. Update h. (t+1) = (h) (t) Dewcce M Lv dy), for all v V.
(!)
states that an permutation invariant hyperedge potential leads to an operator that should process different nodes in permutationneqqiiaaiiee way.
3.2 BUILDING EQUIVARIANT HYPEREDGE DIFFUSION OPERATORS
Our design of permutation-cquiiaaiann diffusion operators is built upon the following Theorem We leave the proof in Appendix [8.2]
Theorem 1. (+-+) : [0, 1] - Rx is a continuous permutation-equivaiinn function, if and only if
can be represented as [ []) ; = p( Zi: Σ ((2)),, i for any Z [..., Zi, Zi,... E [0.1]
where p R R, Φ R- RK are two continuous functions, and K' 2 K.
Remark on an extra theoretical contribution: Theorem [ indicates that any continuous cermutation-equivarine operators where each entry of the input Z has 1-dim feature channel can be precisely written as a composition of continuous function and the sum of another continuous function & each input entry. This result generalizes the representation of permutationninvaiaan functions by p(E K
((z)) in (Zaheer al., 2017) to the equivariant case. An architecture in similar spirit was proposed in (Segol & Lipman 2020). However, their proof only allows approx-
imation, i.c., small |/ Z) + Z) instead of precise representation in Theorem Also, Zaheer jet al. (2017); Segol & Lipman (2020); (Sannai et al. (2019) focus on representations a single set instead of hypergraph with coupled sets.
The above theoretical observation inspires the design of equivariant hyperedge diffusion operators. Specifically, for an operator (5()) Telel + Rlel that may denote either gradient Vg(-) or proximal gradient prox,n9 (.) for each hyperedge e, we may parameterize it
[(b(H))), = p(he, Eg(hw)) for v E e, where ò. e are multi-layer perceptions (MLPs).
(5)
Intuitively, the inner sum collects the Ộ-encoding node features within a hyperedge and then X com- bines the collection with the features from each node further to perform separate operation.
The implementation of the above & is not trivial. A naive implementation is generate auxillary node to representation each (v, E) e--iir for v E V and e E E and learn its representation as adopted in (Yadati 2020 Arya al. 2020 Yang et al. 2020). However, this may substantially increase the model complexity. Our implementation is built upon the bipartite representations (or star expan- sion (Zien al.] 1999; Agarwal al. (006), equivalently) of hypergraphs paired with the standard message passing NN (MPNN) (Gilmer et al. (2017) that can be efficiently implemented via GNN platforms (Fey Lenssen 2019 Wang et al. 2019) or sparse-matris multiplication.
Specifically, we build bipartite graph g = (V,E). The node set V contains two parts VUV& where is the original node set while Ve contains nodes that correspond original hyperedges e E. With this bipartite graph representation, the
Then, add an edge between E V and e E & if model ED-HNN is implemented by following Algorithm 1.
The equivariant diffusion operator & can be constructed via steps 1-3, The last step is to update the node features to accomplish the first two terms in Eq. 2 or the ADMM update Eq. [ We leave more detailed discussion on how Algorithm aligns with the GD and ADMM updates in Sec. C The initial attributes 2, and node degrees are included match the diffusion algorithm by design. As the diffusion operators are shared across iterations, ED-HNN shares parameters across layers. Now, we summarize the joint contribution of our theory and efficient implementation as follows. Proposition 3. MPNNs on bipartite representations (or star expansions) of hypergraphs are expres- sive enough to learn any continuous diffusion operators induced by invariant hyperedge potentials.
5

---

Published as a conference paper at ICLR 2023
Simple while Significant Architecture Difference. Some previous works (Dong al. 2020 Huang & Yang 2021 Chien et al. 2022) also apply GNNs over bipartite representations of hy- pergraphs to build their HNN models, which look similar to ED-HNN. However, ED-HNN has
simple but significant architecture difference: Step in Algorithm needs to adopt 5 as one input to compute the message m This is crucial step to guarantee that the hyperedge operation by combining steps 1-3 forms an equivariant operator from (h0)) Juse
mm However,
previous models often by default adopt (*) = m which leads to an invariant operator. Such a simple change significant as it guarantees universal approximation of equivariant operators, which leads to many benefits as to be discussed in Sec. 3.3 Our experiments will verify these benefits.
Extension. Our ED-HNN can be naturally extended to build equivariant node operators because of the duality between hy peredges and nodes, though this is not necessary in the traditional hypergraph
diffusion problem Eq. | Specifically, we call this model ED-HNNII, which simple revises the step
o( (hh. m. ( for any e such that u E e. Due to the page limit, we put
1 in ED-HNN as ma-te
some experiment results on ED-HNNII in Appendix F.1
3.3 ADVANTAGES OF ED-HNN IN HETEROPHILIC SETTINGS AND DEEP MODELS
Here, we discuss several advantages of ED-HNN due to the design of equivariant diffusion operators.
Heterophily describes the network phenomenon where nodes with the same labels and attributes are less likely to connect to each other directly (Rogers) 2010). Predicting node labels in heterophilic networks is known to be more challenging than that in homophilic networks, and thus has recently become an important research direction (Pei et al. 2020; Zhu et al. 2020 Chien et al. 2021 Lim et al. 2021). Heterophily has been proved as more common phenomenon in hypergraphs than in graphs since it is hard to expect all nodes in a giant hyperedge to share a common label (Veldt al.] 2022). Moreover, predicting node labels in heterophilic hypergraphs is more challenging than that in graphs as a hyperedge may consist of the nodes from multiple categories.
Learnable equivariant diffusion operators are expected to be superior in predicting heterophilic node labels. For example, if a hyperedge e of size 3 is known to cover two nodes v, u from class C1 while one node w from class Co. We may use the LEC potential (H,)) by setting the parameter y = [1,-1,0] Suppose the three nodes' attibutes are He = [h.. hu, hw] IT = [0.7, 0.3]T where hv, h close as they are from the same class. One may check the hyperedge diffusion operator gives (H. = [0.4, - 00.4, OOT. One-step gradient descent 12,, 1y| g g H.)]. with a proper step size (1) < 0.5) drags h. and h. closer while keeping h., unchanged. However, invariant diffusion by forcing the operation on hyperedge e to follow (Vg(H)),, = [V (HH))) = WV (H)) w (invariant messages from the nodes in e) will allocate every node with the same change and cannot deal with the heterogeneity of node labels in e. Moreover, a learnable operator is important. To see this, Suppose we have different ground-trutt truth labels, say wt, u from the same class and from the other. Then, using the above parameter y may increase the error. However, learnable operators can address the problem, e.g., by obtaining a more suitable parameter y = [0, 1, 11] via training.
Moreover, equivariant hyperedge diffusion operators are also good at building deep models. GNNs tend to degenerate the performance when going deep, which is often attributed their oversmooth- ing (Li et al. 2018 Oono& & Suzuki 2019) and overfitting problems (Cong et al. (021). Equivariant operators allocating different messages across nodes helps with overcoming the oversmoothing is- sue. Moreover, diffusion by sharing parameters across layers may reduce the risk of overfitting.
3.4 RELATED WORKS: PREVIOUS HNNS FOR REPRESENTING HYPERGRAPH DIFFUSION
ED-HNN is the first HNN inspired by the general class of hypergraph diffusion and can provably achieve universal approximation of hyperedge diffusion operators. So, how about previous HNNs representing hypergraph diffusion? We temporarily ignore the big difference in whether parameters are shared across layers and give some analysis as follows. HGNN (Feng et al., 2019) runs graph convolution (Kipf & Welling, 2017) on clique expansions of hypergraphs, which directly assumes the hyperedge potentials follow CE plus pairwise potentials and cannot learn other operations on hyperedges. HyperGCN (Yadati et al. (011)) essentially leverages the total variation potential by adding mediator nodes to each hyperedge, which adopts a transformation technique of the total
variantion potential in (Chan et al. 2018 Chan & Liang 2020). GHSC (Zhang et al.) 2022) is

---

Published as a conference paper at ICLR 2023
Table 1: Dataset statisties, CE homophily is the homophily score Pei et al. 2020) based on CE of hypergraphs. Here, four hypergraphs Congress, Senate, Walmart, House are heterophilic.
Corn
Citesoer
Pubrned
Coe--CA
DRL DRLP-CA
Congress
Senate
Walmer
House
nodes edges
2708
3512
19717
2708
41302
1718
282
88860
1250
1072
22363
83105
340
classes
2
2
avg. In]
3.03
3.200
4,349
4.277
4.452
8.656
17.168
6.589
54.730
CE loooophily
0.897
0.893
0.952
0,500
0.869
0.555
0.498
0,530
0.509
hypergraph diffusion proposed in (Chitra & Raphael 2019). A few works view each hyperedge as a multi-set of nodes and each node as a multi-
to represent a specific digge-dpeendentnodd--ee
set of hyperedges (Dong et al.; [220 Huang & Yangl [021] Chien et al. [022] Bai et al. 2021
Arya et al. 2020 Yadati, 2020 Yang et al. 2020; Jo et al.
2021). Among them, HNHN DDng
al.] 2020). UniGNNs (Huang & Yang 2021). EHGNN (J al. 202T) and AllSet (Chien al.,
2022 build two invariant set-pooling functions on both the hyperedge and node sides, which cannot
represent equivariant functions. HCHA (Bai et al., (02T) and UniGAT (Yang et al.) (020) compute attention weights as a result of combining node features and hyperedge messages, which looks like our operation in step 3. However, scalar attention weight is too limited to represent the potentially complicated p. HyperSAGE (Arya et al.) 2020). MPNN-R (Yadati 2020) and LEGCN (Yang al.) 2020) may learn hyperedge equivariant operators in principle by adding auxillary nodes to represent node-hyperedge pairs as aforementioned. However, because of the extra complexity, these models are either too slow or need to reduce the sizes of parameters to fit in memory, which constrains their actual performance in practice. Of course, none of these works have mentioned any theoretical arguments on universal representation of equivariant hypergraph diffusion operators as ours.
One subtle point is that GraphiNN,G link in each layer to pass node features from the former layer directly to the next and may ex- pect to use complicated node-side invariant function (((h!)) -eivee m!!) to approximate (Ar (t)
(Huang & Yang. 2021 adopt jump
Eevee m = (hh (t) Eewee p(h!)) (t) )m in Step 4 of Algorithm 1 directly. This may be doable in theory. However, the jump-link solution may expect to have a higher dimension of
me (t) (dim=l Ueevee el) so that the sum pooling Eevee me does not lose anything from the neigh-
boring nodes of e before interacting with h while in ED-HNN, m! (1) needs dim=/e] according to Theorem []
Our empirical experiments also verify more expressiveness of ED-HNN
4 EXPERIMENTS
4.1 RESULTS ON BENCHMARKING DATASETS
Experiment Setting. In this subsection, we evaluate ED-HNN on nine real-world benchmarking hypergraphs. We focus on the semi-supervised node classification task. The nine datasets include co-citation networks (Cora, Citeseer, Pubmed), co-authorship networks (Cora-CA, DBLP-CA) Ya- dati et al. (001)). Walmart (Ambrrg et al. (020)), House Chodrow et al. (202I), Congress and Sen- ate Fowler (2006b a). More details of these datasets can be found in Appendix F22 Since the last four hypergraphs do not contain node attributes, we follow the method of Chien et al. (2022) to generate node features from label-dependent Gaussian distribution Deshpande et al. (2018).
As we show in Table [ these datasets already cover sufficiently diverse hypergraphs in terms of scale, structure, and hotoo-heterophily. We compare method with top-performing models on these benchmarks, including HGNN (Feng et al. 2019). HCHA (Bai et al., 2021), HNHN (Dong et al. 2020). HyperGCN (Yadati et al.) 2019). UniGCNII (Huang & Yang. (02T), AllDeepSets (Chien et al., 2022). Trrserrrasrfrmrr (Chien et al. (002)), and a recent diffusion method Hy- perND (Prokopchik et al. 2022). All the hyperparameters for baselines follow from (Chien al. 2022) and we fix the learning rate, weight decay and other training recipes same with the baselines. Other model specific hyperparameters are obtained via grid search (see Appendix F.3). We ran- domly split the data into training/validation/ee samples using 50%/25%//254 splitting percentage by following CCien et al. (2022). We choose prediction accuracy as the evaluation metric. We run each model for ten times with different trainingvvalidatin Appendix
splits to obtain the standard deviation. In F.7] we also provide the analysis of the model sensitivity to different hidden dimensions.
Performance Analysis. Table 2 shows the results. Our ED-HNN uniformly outperforms all the compared models on all the datasets. We observe that the top-performing baseline models are AllDeepSets and UniGCNII. As having been analyzed in Sec. 3.4. they model
7

---

Published as a conference paper at ICLR 2023
Table 2: Prediction Accuracy (%). Bold font highlights when ED-HNN significantly (difference in means >> 0.5 x std) outperforms all baselines. The best baselines are underlined. The training and testing times are test on Walmart by using the same server with one GPU NVIDIA RTX A6000.
Homophilic Hypergaapss
Cora
Ciesventlomo.)
Pubmed
CuniCAA
DBLF-CA
Thiining Time (10
HGNN Rugase Yaor 2011
79.39
1.36
72.45 + 1.16
56.44 = 0.44
82.64
1.65
91.03 + (1.20
0.24 + 0.51
HCh
79
1
1.02
72.42 1: 1.42
86,41 = 0.36
K2,55
0.97
90.92 1 0.22
0.24 a 0.01
HNHN
76.36
56,90 0.30
77.19
1.49
M6.78
0.30 0.56
78.45
$2.84 8.67
70:48
2.08
89.38
+:2++1 1.51
UnGCNII hont
78.81
58.25 0.40
83.00
91
HypeND ripisass RUMF
79.20
72.02
56.08 0.43
M0.2
1.32
1.32
AllDeepSann Ponnee
BUzzs
76.88
1.80
70.83
1.63
0.33 58.75
91
1.50
91 11.27:1 +-0.7
123
78.58
73.08
1.20
53.73
83.63 + 1.47
91.53 0-33
+
ED HINN (ours)
80,31 1 1.35
73,70 1. 1.38
89.03 1: 0.53'
83.97 a: 1.55
91.90 1: 0.19'
1.71 1.13
Ileerophilic Hypergaphs
Congress
Senate
Walmurt
House
Aug. Rank
Inference Time (10) 5)
HGNN
Hoane & Youn 5021
91 + 1.15
48.59 + 4.52
62.00 = 0.24
61.39 + 2.96
5.22
1.01 + 0.04
ICC
90.43 1.20
68.62
1:441
62.35 = 0.26
61.36 2.53
5.89
1:54 0.118
HNHN
2010
53.35
47.18 0.35
67.80
2.59
6.67
0.05
HyperiCS
44.74 3.81
48.12
2.93
8.22
4-006
UmGCNI
54,45 0.37
67.25
2.57
3.89
21.32
HyperND Tiginoo
52322 3.20
38.10 3.86
51.701
3.37
6.00
0.01
AllDreeSultt
(L mist Fil mizze
91 0180-11
48.17
64.55 0.33
67 82 2.40
5.22
5
AlSSeThansfomee [meee or 2022 2022
92.16
1.05
51.83 1 5.22
63.46
0.25
59.33
2.20
2.88
6.06 1. 0.67
ED-HNN (ours)
95.00
1000+0090 0.99
64.79 5.14
66.91 0.41
72,45 + 2.28
1.00
587 +.336
Table 3: Prediction Accuracy (%) over Synthetic Hypergraphs with Controlled Heterophily
Homophily
Hotemphily
HGNN (Huung & Yanz
99.86
0.05
90:42 1.14
66.60
1.60
$7.00
50.77
1.94
48.68
HGNN Jumpr.m Harrig & Tanr 2021
99.29 + 0.19
92.61 + 0.97
79.59
+ 2.32
64.39
1.55
55.39 + 2.01
51.14
2.05
AllDeepSees Chaa 2012
99.86
0.07
99.13 2 0.31
93.52
2: 1.31
K3.45
1.42
66.38
1.27
70.94
0.14
AllDeepSas JumpLink
2022
99.95
0.07
99.08 0.24
96.61
1.45
$6.45
1.30
74.08
1.02
71.52
0.94
AllSetTransloneer Chien
0.01
99.81
96.83
78.25
0.16
74.68
0.09
ED-HIN (ours)
99.96 1 0.01
99.87
0.02'
98,85 1 0.25
98.45 1 0.23
90.48
0.70!
79.71 1. 0.27
invariant set functions on both node and hyperedge sides. UniGCNII also adds initial and jump links, which accidentally resonates with our design principle (the step 4 in Algorithm 1). How- ever, their performance has large variation across different datasets. For example, UniGCNII attains promising performance on citation networks, however, has subpar results on Walmart dataset. In contrast, our model achieves stably superior results, surpassing AllSet models by 12.9% on Senate and UniGCNII by 12.5% on Walmart. We owe our empirical significance to the thcoretical design of exact equivariant function representation. Compared with HyperND, the SOTA hypergraph dif- fusion algorithm,
ED-HNN achieves superior performance on every dataset. HyperND provably reaches convergence to the "divergence to the mean" " hyperedge potential (see Example []. which gives it good performance on homophilic networks while only subpar performance heterophilic datasets. Regarding the computational efficiency, we report the wall-clock times for training (for 100 epochs) and testing of all models over the largest hypergraph Walmart, where all models use the same hyperparameters that achieve the reported performance on the left. Our model achieves efficiency comparable to AllSet models (Chien et al. 2022) and much faster than UnIGCNII. So, the implementation of equivariant computation in ED-HNN is still efficient.
4.2 RESULTS ON SYNTHETIC HETEROPHILIC HYPERGRAPH DATASET
Experiment Setting. As discussed, ED-HNN is expected to perform well on heterophilic hyper- graphs. We evaluate this point by using synthetic datasets with controlled heterophily. We generate data by using contextual hypergraph stochastic block model (Deshpande et al., 2018 Ghoshdastidar & Dukkipati| 2014, Lin & Wang 2018). Specifically, we draw two classes of 2, 500 nodes each and then randomly sample 1,000 hyperedges. Each hyperedge consists of 15 nodes, among which a many sampled from class i. We use minfaj, a:2) to denote the heterophily level. Af- terwards, we generate label-dependent Gaussian node features with standard deviation 1.0. We test both homophilic (a = 1,2 2 or CE homophily 0.7) and heterophilic (a = 4 ~ 7 or CE homophily < 0.7) cases. We compare ED-HNN with HGNN, AllSet models and their variants with jump links. We follow previous 50%/25%//5% data splitting methods and repeated 10 times the experiment.
Results. Table shows the results. On homophilic datasets, all the models can achieve good results, while ED-HNN keeps slightly better than others. Once a surpasses 3, i.e., entering the heterophilic regime, the superiority of ED-HNN is more obvious. The jump-link trick indeed also helps, while building equivariance as ED-HNN does directly provides more significant improvement.
8

---

Published as a conference paper at ICLR 2023
Pubmed
Sonate
Heuse
70
(N6
60
Deepods
40
HCBB
10
Figure 2: Comparing Deep Achitectures across Different Models: Test Acc, V.S. Layers
Clique Expansion
Total Varistion tp
Lovisz Extension Cardinality
Iwettrt
42
MAE
MAD
32
255
Figure 3: Comparing the Powers to Represent known Diffusion: MAE v.s. Latent Dimensions.
4.3 BENEFITS IN DEEPENING HYPERGRAPH NEURAL NETWORKS
We also demonstrate that by using diffusion models and parameter tying, ED-HNN can benefit from deeper architectures, while other HNNs cannot. Fig. 2 Ilsttraess the performance of different mod- els versus the number of network layers. We compare with HGNN, AllSet models, and UnIGCNII. UniGCNII inherits from (Chen et al. 2020) which is known to be effective to counteract over- smoothness. The results reveal that AllSet models suffer from going deep. HGNN working through more lightweight mechanism has better tolerance to depth. However, none of them can benefit from deepening. On the contrary, ED-HNN successfully leverages deeper architecture achieve higher accuracy. For example, adding more layers boosts ED-HNN by "1% in accuracy on Pubmed and House, while elevating ED-HNN from 58.01% % to 64.79% on Senate dataset.
4.4 EXPRESSIVENESS JUSTIFICATION ON THE SYNTHETIC DIFFUSION DATASET
We are to evaluate the ability of ED-HNN to express given hypergraph diffusion. We generate semi-synthetic diffusion data using the Senate hypergraph (Chodrow et al. 2021) and synthetic node features. The data consists 1,000 pairs (H))), H1)) The initial node features H (0) are sampled from I-dim Gaussian distributions. To obtain (1) we apply the gradient step in Eq. For non-differential cases, we adopt subgradients for convenient computation. We fix node potentials as [hh;; complexities: a) CE, b) TV (p=2) and e) LEC. The goal is to let one-layer models D E V to recover (1), We compare ED-HNN with our implemented baseline (Invariant) with parameterized invariant set functions on both the node and hyperedge sides, and AllSet models (Chien et al. 2022) that also adopt invariant set functions. We keep the scale of all models almost the same give fair comparison, of which more details are given in Appendix F.5 The results are reported Fig. The CE case gives invariant diffusion all models learn it well. The TV case mostly shows the benefit of the equivariant architecture, where the error almost does not increase even when the dimension decreases to 32. The LEC is challenging for all models, though ED-HNN is still the best. The reason, we think, is that learning the sorting operation in LEC via the sum pooling in Eq, 5 empirically challenging albeit theoretically doable. A similar phenomenon has been observed in previous literatures (Murphy et al. 2018 Wagstaff et al. 2019).
(h, - )T,)² and consider 3 different edge potentials in Example [ with varying
CONCLUSION
This work introduces new hypergraph neural network ED-HNN that can model hypergraph diffu- sion process. We show that any hypergraph diffusion with permulation-invarian potential functions can be represented by iterating equivariant diffusion operators. ED-HNN provides an efficient way to model such operators based on the commonly-use -sed GNN platforms. ED-HNN shows superiority in processing heterophilic hypergraphs and constructing deep models. For future works, ED-HNN can be applied to other tasks such regression tasks to develop counterpart implicit models.

---

Published as a conference paper at ICLR 2023
ACKNOWLEDGMENTS
We would like to express our deepest appreciation Dr. David Gleich, Dr. Kimon Fountoulakis for the insightful discussion on hypergraph computation, and Dr. Eli Chien for the constructive advice on doing the experiments.
REFERENCES
Sameer Agarwal, Kristin Branson, and Serge Belongie. Higher order learning with graphs. In International Conference on Machine Learning (ICML). 2006.
I. Amburg, N. Veldt, and A. R. Benson. Clustering in graphs and hypergraphs with categorical edge labels. Proceedings of The Web Conference, 2020.
Devanshu Arya, Deepak K Gupta, Stevan Rudinac, and Marcel Worring. Hypersage: Generalizing
inductive representation learning on hypergraphs. arXiv preprint (tt::2000004558 2020.
Song Bai, Feihu Zhang, and Philip HS Torr. Hypergraph convolution and hypergraph attention.
Pattern Recognition, 110:107637, 2021.
Federico Battiston, Giulia Cencetti, lacopo lacopini, Vito Latora, Maxime Lucas, Alice Patania, Jean-Gabria Young, and Giovanni Petri. Networks beyond pairwise interactions: structure and dynamics. Physics Reports, 2020.
Austin R Benson, David F Gleich, and Jure Leskovec. Higher-order organization of complex net- works. Science, 2016.
Austin R Benson, David Gleich, and Desmond Higham. Higher-order network analysis takes off, fueled by classical ideas and new data. arXiv preprint urXiv:2103.0503: 2021.
Claude Berge. Hypergraphs: combinatorics of finite sets, volume 45, Elsevier, 1984.
Stephen Boyd, Neal Parikh, Eric Chu, Borja Peleato, Jonathan Eckstein, et al. Distributed optimiza- tion and statistical learning via the alternating direction method of multipliers. Foundations and Trends in Machine learning, 2011.
Ben Chamberlain, James Rowbottom, Maria I Gorinova, Michael Bronstein, Stefan Webb, and Emanuele Rossi. Grand: Graph neural diffusion. International Conference on Machine Learn- ing. pp. 1407-1418. PMLR, 2021.
S. H. Chan, X. Wang, and O. A. Elgendy. Plug-and-pla admm for image restoration: Fixed-poiie convergence and applications. IEEE Transactions on Computational Imaging, 3(1):84-98, 2016
T-H Hubert Chan and Zhibin Liang. Generalizing the hypergraph laplacian via a diffusion process with mediators. Theoretical Computer Science, 2020.
T-H Hubert Chan, Anand Louis, Zhihao Gavin Tang, and Chenzi Zhang. Spectral properties of hypergraph laplacian and approximation algorithms. Journal of the ACM (JACM), 2018.
J. H. R. Chang, C.-L. Li, B. Poczos, B. V. Kumar, and A. C. Sankaranarayanan. One network to solve them all-solving linear inverse problems using deep projection models. In International Conference on Computer Vision (ICCV), 2017.
Siheng Chen, Yonina C Eldar, and Lingxiao Zhao. Graph unrolling networks: Interpretable neural networks for graph signal denoising. IEEE Transactions on Signal Processing, 69:3699-3713.
2021a
Tianlong Chen, Xiaohan Chen, Wuyang Chen, Howard Heaton, Jialin Liu, Zhangyang Wang, and Wotao Yin. Learning to optimize: primer and a benchmark. arXiv preprint arXiv:2103.1288 2021b.
Ting Chen, Simon Korbblith, Mohammad Norouzi, and Geoffrey Hinton. A simple framework for contrastive learning of visual representations. In International Conference on Machine Learning (ICML), 2020.
10

---

Published as a conference paper at ICLR 2023
X. Chen, J. Liu, Z. Wang, and W. Yin. Theoretical linear convergence of unfolded ista and its practical weights and thresholds. 2018.
Yunjin Chen and Thomas Pock. Trainable nonlinear reaction diffusion: A flexible framework for fast and effective image restoration. IEEE transactions on pattern analysis and machine intelligence, 39(6): (2(6):125-11222 1272, 2016.
Eli Chien, Jianhao Peng, Pan Li, and Olgica Milenkovic. Adaptive universal generalized pagerank graph neural network. International Conference on Learning Representations (ICLR), 2021.
Eli Chien, Chao Pan, Jianhao Peng, and Olgica Milenkovic. You are allset: A multiset function framework for hypergraph neural networks. In International Conference Learning Represen- tations, 2022.
I Eli Chien, Huozhi Zhou, and Pan Li. hs² Active learning over hypergraphs with pointwise and pairwise queries. In International Conference on Artificial Intelligence and Statistics (AISTATS), 2019.
Uthsav Chitra and Benjamin Raphael. Random walks hypergraphs with edge-dependent vertex weights. In International Conference on Machine Learning (ICML), 2019.
Philip S Chodrow, Nate Veldt, and Austin R Benson. Hypergraph clustering: from blockmodels to modularity. Science Advances, 2021.
Weilin Cong, Morteza Ramezani, and Mehrdad Mahdavi. On provable benefits of depth in training graph convolutional networks. Advances in Neurul Information Processing Systems (NeurIPS), 2021.
H. Dai, Z. Kozareva, B. Dai, A. Smola, and L. Song. Learning steady-states of iterative algorithms over graphs. In International Conference on Machine Learning (ICML), 2018.
Y. Deshpande, Sen, A. Montanari, and E. Mossel. Contextual stochastic block models. In Ad- vances in Neural Information Processing Systems (NeurIPS), 2018.
Yihe Dong, Will Sawin, and Yoshua Bengio. Hnhn: Hypergraph networks with hyperedge neurons.
arXiv preprint arXiv:2006.1222 2020.
Yifan Feng, Haoxuan You, Zizhao Zhang, Rongrong Ji, and Yue Gao. Hypergraph neural networks. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 33, pp. 3558-3565, 2019.
Matthias Fey and Jan E. Lenssen. Fast graph representation learning with PyTorch Geometric. In International Conference on Learning Representations (ICLR) Workshop on Representation Learning Graphs and Manifolds, 2019.
Kimon Fountoulakis, Pan Li, and Shenghao Yang. Local hyper-flow diffusion. Advances in Neural Information Processing Systems, 34, 2021.
J. H. Fowler. Connecting the congress: A study of cosponsorship networks. Political Analysis, 14 (4):456-48) 2006a.
H. Fowler. Legislative cosponsorship networks in the us house and senate. Social networks, 28 (4):4544465, 2006b.
D. Ghoshdastidar and A. Dukkipati. Consistency of spectral partitioning of uniform hypergraphs under planted partition model. Advances in Neural Information Processing Systems (NeurIPS), 2014.
Justin Gilmer, Samuel S Schoenholz, Patrick Riley, Oriol Vinyals, and George Dahl. Neu- ral message passing for quantum chemistry. In International Conference on Machine Learning (ICML), 2017.
K. Gregor and Y. LeCun. Learning fast approximations of sparse coding. In International Confer- on Machine Learning (ICML., 2010.
11

---

Published as a conference paper at ICLR 2023
F. Gu, H. Chang. W. Zhu, S. Sojoudi, and L El Ghaoui. Implicit graph neural networks. Advances in Neural Information Processing Systems (NeurIPS), 2020.
William L Hamilton. Graph representation learning. 2020.
Matthias Hein, Simon Setzer, Leonardo Jost, and Syama Sundar Rangapuram. The total variation on hypergraphs-lcarning on hypergraphs revisited. In Advances in Neural Information Processing Systems, pp. 2422-22222222 -435,, 2013.
R. Hershey, J. L.. Roux, and F. Weninger. Deep unfolding: Model-based inspiration of novel deep architectures. arXiv preprint arXiv:1409.2577 2014.
Jing Huang and Jie Yang. Unignn: a unified framework for graph and hypergraph neural networks.
arXiv preprint XXiv:2105500056 2021.
Stefanie Jegelka, Francis Bach, and Suvrit Sra. Reflection methods for user-friendly submodular optimization. In Advances in Neural Information Processing Systems (NeurIPS), 2013.
Jachyeong Jo, Jinheon Back, Seul Lee, Dongki Kim, Minki Kang, and Sung Ju Hwang. Edge representation learning with hypergraphs. In Advances in Neural Information Processing Systems (NeurIPS), 2021.
Thomas N Kipf and Max Welling. Semi-supervised classification with graph convolutional net- works. In International Conference on Learning Representations (ICLR), 2017.
Johannes Kliepera, Aleksandar Bojchevski, and Stephan Günnemann. Predict then propagate: Graph neural networks meet personalized pagerank. In International Conference Learning Representations, 2019.
Renaud Lambiotte, Martin Rosvall, and Ingo Scholtes. From networks to optimal higher-order models of complex systems. Nature physics, 2019.
Geon Lee, Minyoung Choe, and Kijung Shin. hypergraphs?-paateene measures, and generators. In Proceedings of the Web Conference 2021, pp. 3396-3407, 2021.
How do hyperedges overlap in real-world
Pan Li and Olgica Milenkovic. Inhomogeneous hypergraph clustering with applications. In Ad- vances in Neural Information Processing Systems, pp. 2305-2315, 2017.
Pan Li and Olgica Milenkovic. Submodular hypergraphs: p-laplacians, cheeger inequalities and spectral clustering. In International Conference on Machine Learning (ICML), 2018.
Pan Li, Hoang Dau, Gregory Puleo, and Olgica Milenkovic. Motif clustering and overlapping clustering for social network analysis. In INFOCOM 2017-IEEE Conference on Computer Com- munications, IEEE, pp. 1-9. IEEE, 2017.
Pan Li, Niao He, and Olgica Milenkovie. Quadratic decomposable submodular function minimiza- tion: Theory and practice. Journal of Machine Learning Research (JMLR), 2020a.
Qimai Li, Zhichao Han, and Xiao-Ming Wu. Deeper insights into graph convolutional networks for semi-supervised learning. In AAAI conference artificial intelligence (AAAI), 2018.
Yuelong Li. Mohammad Tofighi. Junyi Geng, Vishal Monga, and Yonina C Eldar. Efficient and interpretable deep blind image deblurring via algorithm unrolling. IEEE Transactions on Com- putational Imaging, 6:666-681, 2020b.
Derek Lim, Felix Hohne, Xiuyu Li, Sijia Linda Huang, Vaishnavi Gupta, Omkar Bhalerao, and Ser Nam Lim. Large scale learning non-homophilous graphs: New benchmarks and strong simple methods. In Advances in Neural Information Processing Systems (NeurIPS), 2021.
I. Chien C.-Y. Lin and I.-H. Wang. Community detection in hypergraphs: Optimal statistical limit and efficient algorithms. In International Conference on Artificial Intelligence and Statistics (AIS- TATS), 2018
12

---

Published as a conference paper at ICLR 2023
J. Liu, K. Kawaguchi, B. Hooi, Y. Wang, and X. Xiao. Eignn:
Efficient infinite- depth graph neural networks. Advances in Neural Information Processing Systems (NeurIPS), 2021a.
Jialin Liu and Xiaohan Chen. Alista: Analytic weights are good as learned weights in lista. In International Conference on Learning Representations (ICLR), 2019.
Meng Liu, Nate Veldt, Haoyu Song, Pan Li, and David F Gleich. Strongly local hypergraph diffu- sions for clustering and semi-supervised learning. Proceedings of the Web Conference 2021, pp. 2092-2103, 2021b.
X. Liu, W. Jin, Y. Ma, Y. Li, H. Liu, Wang, M. Yan, and Tang. Elastic graph neural networks. In International Conference on Machine Learning (ICML), 2021c.
Ziwei Liu, Xiaoxiao Li, Ping Luo, Chen Change Loy, and Xiaoou Tang. Deep learning markov random field for semantic segmentation. IEEE transactions pattern analysis and machine intelligence, 40(8):1814-188) 2017.
László Lovász. Submodular functions and convexity. In Mathematical Programming The State of the Art. 1983.
Shmoolik Mangan and Uri Alon. Structure and function of the foed-forward loop network motif. National Academy of Sciences, 2003.
Tim Meinhardt, Michael Moller, Caner Hazirbas, and Daniel Cremers. Learning proximal operators: Using denoising networks for regularizing inverse imaging problems. In International Conference on Computer Vision (ICCV), 2017.
Vishal Monga, Yuelong Li, and Yonina C Eldar. Algorithm unrolling: Interpretable, efficient deep learning for signal and image processing. IEEE Signal Processing Magazine, 2021.
R. L. Murphy, B. Srinivasan, V. Rao, and B. Ribeiro. Janossy pooling: Learning deep permutation- invariant functions for variable-size inputs. In International Conference on Learning Representa- tions (ICLR), 2018.
Leonie Neuhäuser, Michael T Schaub, Andrew Mellor, and Renaud Lambiotte. Opinion dynam- ics with multi-body body interactions. In International Conference on Network Games, Control and Optimization, 2021.
Leonie Neuhäuser, Renaud Lambiotte, and Michael T Schaub. Consensus dynamics and opinion formation on hypergraphs. In Higher-Order Systems. 2022.
Maximilian Nickel, Kevin Murphy, Volker Tresp, and Evgeniy Gabrilovich. A review of relational machine learning for knowledge graphs. Proceedings of the IEEE, 2015.
Kenta Oono and Taiji Suzuki. Graph neural networks exponentially lose expressive power for node classification. In International Conference on Learning Representations (ICLR), 2019.
Neal Parikh and Stephen Boyd. Proximal algorithms. Foundations and Trends in optimization, 2014.
Hongbin Pei, Bingzhe Wei, Kevin Chen-Chuan Chang, Yu Lei, and Bo Yang. Geom-gen: Geometric graph convolutional networks. In International Conference on Learning Representations (ICLR), 2020.
Nicholas G Polson, James G Scott, and Brandon T Willard. Proximal algorithms in statistics and machine learning, Statistical Science, 2015.
Konstantin Prokopchik, Austin R Benson, and Francesco Tudisco. Nonlinear feature diffusion on
hypergraphs. In International Conference on Machine Learning, pp. 17945-17995 PMLR, 2022.
Everett M Rogers. Diffussio of innovations. 2010.
E. Ryu, J. Liu, S. Wang, X. Chen, Z. Wang, and W. Yin. Plug-and-Pg and-payd- methods provably converge with properly trained denoisers. In International Conference Machine Learning (ICMI.), 2019.
13

---

Published as a conference paper at ICLR 2023
Akiyoshi Sannai, Yuuki Takai, and Matthieu Cordonnier. Universal approximations of permutation invariand/equivariat functions by deep neural networks. arXiv preprint arXiv:1903.01939, 2019.
Michael T Schaub, Yu Zhu, Jean- Sann-Bapisste Seby, Mitchell Roddenberry. and Santiago Segarra. Signal processing on higher-order networks: Livin' on the edge... and beyond. Signal Processing, 2021.
Christian J Schuler, Michael Hirsch, Stefan Harmeling, and Bernhard Schölkopf. Learning to deblur. IEEE transactions on pattern analysis and machine intelligence, 38(7):1439-1444 2015.
Nimrod Segol and Yaron Lipman. On universal equivariant set networks. In International Confer- ence on Learning Representations (ICLR), 2020.
Yuuki Takai, Atsushi Miyauchi, Masahiro Ikeda, and Yuichi Yoshida. Hypergraph clustering based on pagerank. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 1970-1978. 2020.
A. M. Teodoro, J. M. Bioucas-Dias, and M. A. T. Figueiredo. Scene-adapted plug-and-play algo- rithm with convergence guarantees. In International Workshop on Machine Learning for Signal Processing (MLSP), 2017.
Matthew Thorpe, Tan Minh Nguyen, Hedi Xia, Thomas Strohmer, Andrea Bertozzi, Stanley Osher, and Bao Wang. Grand++: Graph neural diffusion with source term. In International Conference on Learning Representutions, 2022.
Charalampos E Tsourakakis, Jakub Pachocki, and Michael Mitzenmacher. Scalable motif-aware graph clustering. In Proceedings of the Web Conference, pp. 1451-1460, 2017.
Francesco Tudisco, Austin R Benson, and Konstantin Prokopchik. Nonlinear higher-order label spreading. In Proceedings of the Web Conference, 2021a.
Francesco Tudisco, Konstantin Prokopchik, and Austin R Benson. A nonlinear diffusion method for semi-supervised learning on hypergraphs. arXiv preprint 1rXiv:2103.14880 2021b.
Nate Veldt, Austin Benson, and Jon Kleinberg. Hypergraph cuts with general splitting functions. SIAM Review, 2021.
Nate Veldt, Austin R Benson, and Jon Kleinberg. Higher-order homophily is combinatorially im- possible. SIAM Review (SIREV), 2022.
Edward Wagstaff, Fabian Fuchs, Martin Engelcke, Ingmar Posner, and Michael A Osborne, On the limitations of representing functions on sets, In International Conference on Machine Learning (ICML), 2019.
Jianling Wang, Kaize Ding, Liangjie Hong, Huan Liu, and James Caverlee. Next-item recommenda- tion with sequential hypergraphs. In Proceedings of the 43rd international ACM SIGIR conference research and development in information retrieval, pp. 1101-1110, 2020.
Jianling Wang, Kaize Ding, Ziwei Zhu, and James Caverlee. based recommendation with hypergraph attention networks. In Proceedings of the 2021 SIAM International Conference on Data Mining (SDM), pp. 82-90. SIAM, 2021.
Minjie Wang, Da Zheng, Zihao Ye, Quan Gan, Mufei Li, Xiang Song, Jinjing Zhou, Chao Ma, Lingfan Yu, Yu Gai, Tianjun Xiao, Tong He, George Karypis. Jinyang Li. and Zheng Zhang. Deep graph library: A graph-centrie, highly-performant package for graph neural networks. arXiv preprint
2019.
Nan Wang, Shoujin Wang, Yan Wang, Quan Z. Sheng, and Mehmet A Orgun. Exploiting intra-and inter-session dependencies for session- based recommendations, World Wide Web, 25(1):425-44 2022.
Zhong-Qin Wang, Jonathan Le Roux, DeLiang Wang, and John R Hershey. End-to-end speech separation with unfolded iterative phase reconstruction. In Conference of the International Speech Communication Association (INTERSPFEECH), 2018.
14

---

Published as a conference paper at ICLR 2023
Xin Xia, Hongzhi Yin, Junliang Yu, Qinyong Wang, Lizhen Cui, and Xiangliang Zhang. Self- supervised hypergraph convolutional networks for session-bas based recommendation. In Proceed- ings of the AAAI Conference on Artificial Intelligence, volume 35, pp. 4503-4511. 2021.
Bo Xin, Yizhou Wang, Wen Gao. David Wipf, and Baoyuan Wang. Maximal sparsity with deep networks? Advances in Neural Information Processing Systems, 29, 2016.
Naganand Yadati. Neural message passing for multi-relational ordered and recursive hypergraphs. Advances in Neural Information Processing Systems, 33:3275- -2299 2020.
Naganand Yadati, Madhav Nimishakavi, Prateck Yadav, Vikram Nitin, Anand Louis, and Partha Talukdar. Hypergen: A new method for training graph convolutional networks on hypergraphs. Advances in neural information processing systems, 32, 2019.
Chaoqi Yang, Ruijie Wang. Shuochao Yao, and Tarek Abdelzaher. Hypergraph learning with line expansion. arXiv preprint arXiv:2005.0488 04843, 2020.
Y. Yang, Liu, Wang, Zhou, Q. Gan, Z. Wei, Z. Zhang, Z. Huang, and Wipf. Graph neu- ral networks inspired by classical iterative algorithms. In International Conference Machine Learning (ICMI.), 2021a.
Yongyi Yang, Yangkun Wang, Zengfeng Huang, and David Wipf. Implicit vs unfolded graph neural networks. arXiv preprint arX65:2111.06592, 2021b.
Yuichi Yoshida. Cheeger inequalities for submodular transformations. In Proceedings of the Thirti- eth Annual ACM-SIAM Symposium on Discrete Algorithms, pp. 2582-2601, SIAM, 2019.
Manzil Zaheer, Satwik Kottur, Siamak Ravanbakhsh, Barnabas Poczos, Russ R Salakhutdinov, and Alexander J Smola. Deep sets. In Advances in Neural Information Processing Systems (NeurIPS),
2017.
Chenzi Zhang, Shuguang Hu, Zhihao Gavin Tang, and TH Hubert Chan. Re-revisiting learning on hypergraphs: confidence interval and subgradient method. In International Conference on Machine Learning, pp. 4026 4034, 2017a.
Jiying Zhang, Fuyang Li. Xi Xiao, Tingyang Xu, Yu Rong, Junzhou Huang, and Yatao Bian. Hyper- graph convolutional networks via equivalency between hypergraphs and undirected graphs arXiv preprint 2rxi::2203. 16939, 2022.
K. Zhang, W. Zuo, Gu, and L.. Zhang. Learning deep cnn denoiser prior for image restoration. In Conference Computer Vision and Pattern Recognition (CVPR), 2017b.
Songyang Zhang, Zhi Ding, and Shuguang Cui. Introducing hypergraph signal processing: Theo- retical foundation and practical applications. IEEE Internet Things Journal, 2019.
Shuai Zheng, Sadeep Jayasumana, Bernardino Romera- Paanerr-Prrede,,, Vibhav Vineet, Zhizhong Su, Da- long Du, Chang Huang, and Philip HS Torr. Conditional random fields as recurrent neural net- works. In International Conference on Computer Vision (ICCV), 2015.
Denny Zhou, Jiayuan Huang, and Bernhard Schölkopf. Learning with hypergraphs: Clustering, classification, and embedding. In Advances in Neural Information Processing Systems, pp. 1601- 1608, 2007.
Jiong Zhu, Yujun Yan, Lingxiao Zhao, Mark Heimann, Leman Akoglu, and Danai Koutra. Beyond homophily in graph neural networks: Current limitations and effective designs. In Advances in Neural Information Processing Systems (NeurIPS), 2020.
Xiaojin Zhu. Semi-supervised learning with graphs. Carnegie Mellon University, 2005.
Jason Y Zien, Martine DF Schlag, and Pak K Chan. Multilevel spectral hypergraph partitioning with arbitrary vertex sizes. IEEE Transactions computer-aided aided design of integrated circuits and systems, 1999.
15

---

Published as a conference paper at ICLR 2023
A DERIVATION OF EQ. 3 AND 4
We derive iterative update Eq. 3 and as follows. Recall that our problem defined as:
min H
fhov; xx)) +
E9e(H,
NEV
3EE
where E RN is the node feature matrix. H. = associated node features inside edge e, where h corresponds the node features of the i-th node in edge e. We use E, to denote the (initial) attribute of node v. Here, neither f(,; nor ge(.) necessarily continuous. To solve such problem, we borrow the idea from ADMM (Boyd al.)
he.lel E R lel collects the
2011]. We introduce an auxiliary variable R. He for every E as constrained optimization problem:
& and reformulate the problem
min Σ f(hoi + Σ Ie (R.) DEV
CEE
subject to R. H = Ve E
By the Augmented Lagrangian Method (ALM), we assign a Lagrangian multiplier Se (scaled by 1/A) for each edge, then the objective function becomes:
A
max min JeE8
Σ fh:: +
Σ (
R
H. + Selle
IS.... (6)
(S.)eee H.
2
VEV
REE
EEE
3EE
We can iterate the following primal-dual steps to optimize Eq. in block-wise sense:
The primal step can be computed
R <- arg min
R.
H + s s g(Ra)
2
R.
-poxx g/A
H
)( We E E.
h 5tt R((+1) / ((hni
h (++1) arg min h,
Σ Levee(S! +R!(t!))
= -rrxx (:aa.)/Ad.
(8)
du
The dual step can be computed as:
s(1+1) + S(1) + (1+1)) (1+1), (H VEEE.
(9)
Denote Q: (
S( R (++1) and 1) = 1/x, then the iterative updates become:
(!(-1) prox ng (2H Q!!)
+Q(!) -H!"), for E E
(10)
Eereeo Q (++1)
h (L-1) prox prox,,((:x..//d
for v V.
(11)
d,
B DEFERRED PROOFS
B. PROOF PROPOSITION
Proof. Define [K] [K] be an index mapping associated with the permutation matrix P
(() such that PZ = "*(1), ... =*(k]] To prove that Vg(+) is permutation equivariant, we
show by using the definition of partial derivatives. For any Z [z] zk] and permutation
we have:
g(=a(1): Pa(() ..... Fx(K)) ) g ((=)(1))"" "., Fn()), 2x zx(K))
V (PZ), lim
5-+0
8
",,,,) g(21, ... =x(1) + 8, =x)
g g( ... "'
lim
8-0
8
[Vg(Z)<<())
(12)
16

---

Published as a conference paper at ICLR 2023
where the second equality Eq. 12 is due to the permutation invariance of 9(-). To prove Proposition for the proximal gradient, we first define: H prox, (2) = arg min h g H) + 2 H Z P
for some Z. For arbitrary permutation matrix P E I1(K), we have
1 2
prox, (PZ) arg mig g H) +
H 1--PZ 1/-- = arg min g (
P(P H 2)1
H
H
+ 1 2
arg mrg g H) +
P H - = arg min ( H
P H - 2
H
2
H
(13)
PH"
where Eq. 13 is due to the permutation invariance of g(.).
B.2 PROOF OF THEOREM I
Proof. To prove Theorem
we first summarize one of our key results in the following Lemma
function if and only if there is function p(.) [0, 1] + R that is permutation invariant to the last entries, such that [bb (]) p(2;, Zi41," =K,' zi-1) for any i.
Lemma 2. (b(-) : 10.1]K + RK is a permutation-equivaate
Proof. (Sufficiency) Define [K] + [K] be an index mapping associated with the permuta- tion matrix P E [TK)) such that PZ = (I)))) xx()]] Then [vb(=(11)= >n(k))];
p(za(1), (x+1))), (K)) ()) 1)). Since p(.) is invariant to the last K - entries,
[P)); = ((=+(()) (x()+1 2K, (x))) = [eb Z (xx())
function & [0, + R we first ex- pand it to the following form: [bb [Z)] = P=(21, 2x). Permutation-qquivarianee means (2) ... =k) = Pi(zx(1),'' =x(K))- Suppose given an index i, consider any per- mutation π. [K] + [K]. where ti) = i. Then, we have p=(21.... Z.. ,zk) = (=(0 Et, zxk) p=(=)(1)s'' ...... , ==(K)), which implies Ps R
(Necessity) Given
R must
- elements other than the i-th element. Now, consider a permutation æ where (1) = Then P=(21, 22, =kk (a(1) (21, 22, =k) (I(z)(1)) (2) ... kx(K) (21, _..... : 21-1), where the last equality is due to our previous argument. This im- plies two results. First, for all i, p; ø2, ... Zz kK) ). Vi E [K] should be written in terms of p1 21+1," ...
be invariant to the K-
=K. ... 25-1). Moreover, P1 is permutation invariant to its last K 1 entries.
Therefore, we just need to set p p and broadcast accordingly to all entries. We conclude the proof.
To proceed the proof, we bring in the following mathematical tools (Zaherr al.
2017
Definition 5. Given a vector = [2|, 2x]] T E RRK we define power mapping Ou R R as @N (2) = [z x² ... =M] and sum-of-power power mapping 4M : R
+ R M as (2) =
ΣK, (2)),), where M is the largest degree.
Lemma 3. Let X-
fiEi.... =x] E [0,1]K such that 21 <2
zk). ). We define
mapping Φb R. - RK++ as x(2) 220 zt 22
2K and mapping tb RRKK + R KK+ as
b(z)= = ΣK H(z)), where M is the largest degree. Then t restricted on X, iee, tb X - R2 K+1, is homeomorphism.
Proof. Proved in Lemma 6 in (Zaheer al. 2017
We note that Definition is slightly different from the mappings defined in Lemma 3 Zaheer al. 2017) as it removes the constant (zero-order) term. Combining with Lemma Zaheer et al. 2017 and results in (Wagstaff et al. 2019), we have the following result:
17

---

Published as a conference paper at ICLR 2023
Lemma 4. Let X {[ =k] [0, 1| such that 21 22 <... < Zkk,, then there exists a homeomorphism ass : + IR M such that (z) = EE OM (zi) where ΦM R - RM if M 2 K.
Proof. For M = K, we choose őк and @x to be the power mapping and power sum with largest degree K defined in Definition We note that ((z) K
@ [x(=)]] Since K
is constant,
there exists a homeomorphism between the images of ((z) and &x (z). By Lemma 3 ④ X RK+1 is a homeomorphism, which implies &x (x)=) X' + R is also a homeomorphism.
For M K, we first pad every imput 2 E X' with constant > 1 to be an M-dimension & E Rd, Note that padding is homeomorphic since it is constant. All such & form subset
[M) such that 21 < 22 <.. < 2M}. We choose Φ : R + RM to be power mapping and a X" + IR-M to be sum-of-power mapping restricted on 2", respectively. Following (Wagstaff et al. 2019). we construct & (x) below:
X" C {[=1.-... _
K
M Σ k=++1
M (M( = Σ (2,) Σ
K Sw Σ (k)
① (x)))
Σ (z))
+
(14)
M
Σ (pnr -k)(())) +
Σ
- @ar(k)) SM (A:) = Σ -(mm(z))-@ar((1)
+ M (k), (15)
i=1
which induces Σx= (2m1(2)) - Sas (l:) क M (8) MoN (R)). Let (2) (xr(2;) dm (k)),
and (2) 1==1 PM (z.). Since [0, k]] is naturally homeomorphic to [0, 1]. by our argument
for M = K, 4 X" +
RM is a homeomorphism. This implies 4M : X' + RM also
homeomorphism.
is straightforward to show the sufficiency of Theorem by verifying that for arbitrary permutation
π [K] + [K] [bb(=x(1)) >>(K))]; = p(=*(s): Σ/= o(z,)) [b< ... =x = With
Lemma 2 and we can conclude the necessity of Theorem by the following construction:
1. By Lemma any permutation equivariant function (221.... , zx] can be written as
[+)(-]] = T T(Z,, Zi _K, z;-1) such that r(-) is invariant to the last K - 1 ele- ments.
2. By Lemma 4 we know that there exists homeomorphism mapping dx which is continu- ous, invertible, and invertibly continuous. For arbitrary z E [0, 1] the difference between z and क (2) is up to a permutation.
3. Since 7(-) is permutation invariant to the last K 1 elements, we can construct the func- tion T (Zi, Zi41, _K, -1=1) 72z, x K- xx- (Zi+1, ... =K, zi-1)) = p(zi, Σx ΦR (z;)) where ((x v) = T(x, $ xx=- (v)) and p(z, v) = p(x, - (z))
j# Ej 7(z; E
Since 2 K- and Φ all continuous functions, their composition p is also continuous.
Remark 1. Sannai et al. 2019) [Corollary 3.2] showed similar results to our Theorem 7 However, their provided exact form of equivariant function is written p(zi, Ejz; 0(2,)). We note the difference: the summation in Theorem 7 pools over all the nodes inside hyperedge while the summation in (Sannai et al. [ Corollary 3.2] pools over elements other than the central node. To implement the latter equation in hypergraphs which have coupled sets. has to maintain each node- hooe-dy pair, which is computationally prohibitive in practice. An easier proof can be shown by combining results in Sannai et al. 2019) with our proof technique introduced in the Step 3.
18

---

Published as a conference paper at ICLR 2023
d
d
Vg.
V92
V2
b
a
Hypergraph
Invariant Diffusion
Equivariant Diffusion
Figure 4: Comparing Invariant Diffusion and Equivariant Diffusion. The direction of the arrows indicates information flow. The same color of arrows denotes the same passed message while different color of arrows means different passed messages.
C ALIGNING ALGORITHM I WITH GD/ADMM UPDATES
ED-HNN can be regarded as an unfolding algorithm to optimize objective Eq. 1 where / and are node and edge potentials, respectively. Essentially, each iteration of ED-HNN simulates one-step equivariant hypergraph diffusion. Each iteration of ED-HNN has four steps. In the first step, ED- HNN point-wiscly transforms the node features via an MLP 6. Next ED-HNN sum-pools the node features onto the associated hyperedges as me = Evee ((h.). ). The third step is crucial to achieve equivariance, where each node sum-pools the hyperedge features after interacting its feature with every connected hyperedge: Dervee p(h,, m.). The last step is to update the feature information via another node-wse transformation yp According to our Theorem 1, p(h., Euee ((hw)) can represent any equivariant function. Therefore, the messages aggregated from hyperedges pl (h., m.) can be learned to approximate the gradient Vg Eq.
ED-HNN can well approximate the GD algorithm (Eq, 2 while ED-HNN does not perfectly match
ADMM unless assume Q!" -H!') in Eq. 3 for practical consideration. This assumption may reduce the performance the hypergraph diffusion algorithm while our model ED-HNN has already achieved good enough empirical performance. Tracking Q means recording the messages from to V for every iteration, which is not supported by the current GNN platforms (Fey & Lenssen, 2019 Wang et al. 2019) and may consume more memory. We leave the study of the algorithms that can track the update of Q as a future study. co-design of the algorithm and the system may be needed to guarantee the scalability of the algorithm.
D INVARIANT DIFFUSION VERSUS EQUIVARIANT DIFFUSION
Although our Proposition states hypergraph diffusion operator should be inherently equivariant, many exising frameworks design it to be invariant. In invariant diffusion, hypergraph diffusion op- erator V-(-) or prox (-)(-) corresponds to a permutation invariant function rather than permutation equivariant function (see Definition 4). Mathematically, Vg(H.) 1 ((H.) for some invariant function 9 x
R.. In contrast to equivariant diffusion. the invariant diffusion opera- tor summarizes the node information to one feature vector and passes this identical message to all nodes uniformly. We provide Fig. 4 to illustrate the key difference between invariant diffusion and equivariant diffusion. Considering implementing such an invariant diffusion by DeepSet (Zaheer) al. 2017), invariant diffusion should have the following message passing: The difference in
Example: Invariant Diffusion by DeepSet Initialization: H = X. For f 0, 2, 2,., L 1, do:
1. Designing the messages from to E: m ((h (1) for all V.
2. Sum V + messages over hyperedges m.
(1) Ene: ma-be. se. for all E.
3. Broadcast m and design the messages from & to V: m Mmm for all v E
4. Update h (1+1) = ((h! Ee'ute mi Fv, du), for all u E V.
19

---

Published as a conference paper at ICLR 2023
the third step is worth noting. ý is independent of the target node features such that each hyperedge passes every node an identical message. HyperGCN (Yadati et al.) 2019), HGNN (Feng et al. 2019) and other GCNs running clique expansion can all be reduced to invariant diffusion.
E FURTHER DISCUSSION ON OTHER RELATED WORKS
Our ED-HNN is actually inspired by a line of works on optimization-inspieee NNs. Optimization-
inspired NNs often get praised for their interpretability (Monga et al. [ Chen et al. 2021b) and
certified convergence under certain conditions of the learned operators (Ryu et al. 2019 Teodoro
er al. 2017; Chan et al., 2016). Optimization-inppinn
NNs mainly focus on the applications such
as compressive sensing (rreorr & LeCun 2010, Xin et al.
2016 Liu & Chen 2019 Chen et al,
2018 1) speech processing (Hershey et al. 2014; Wang et al. 2018 image denoising (Zhan et al,
2017b. Meinhardt et al.
Pock 2016), partitioning (Zheng et al, 2015 Liu al. 2017). deblurring (Schuler et al. 2015: Li et al. 2020b) and so on. Only a few
et al. 2017] Chen Chen
works recently have applied optimization-insnee
NNs to graphs as discussed in Sec. 3.4 while our NNs on hypergraphs.
work is the first one to study optimization-inspee
Parallel to our hypergraph diffusion inspired HNNs, there are also some models to represent diffu- sion graphs. However, there are only pairwise relations in graphs, the model design is often much simpler than that for hypergraphs.
pairwise relations is trivial (corresponding particular, to construct an expressive operator with for the case K in Theorem [ These models unroll either learnable (Yang et al., 2021a la. Chen et al.] 2021a), fixed f-normm (Liu et al. 2021c) or fixed 2--norm pair-wise potentials (Kliepera et al. 2019) to formulate GNNs. Some works also view the graph diffusion as an ODE (Chamberlain et al. 202 Thorpe et al. (022). which essentially corre-
sponds the gradient descent formulation Eq. Implicit GNNs (Dai et al. [0188] Liu et al., 2021a]
Gu et al., 2020 Yang et al. (2021b) are to directly parameterize the optimum of optimization problem while implicit methods are missing for hypergraphs, which a promising future direction.
F ADDITIONAL EXPERIMENTS AND IMPLEMENTATION DETAILS
F.1 EXPERIMENTS ON ED-HNNII
As we described in Sec. 3.2 an extension our ED-HNN is to consider - & message passing and 3 + V message passing as two equivariant set functions. The detailed algorithm is illustrated Algorithm 2, where the red box highlights the major difference with ED-HNN.
Algorithm 2: ED-HNNII
Initialization: H 0 X and three MLPs é, é, & (shared across L layers).
For ( 2, L - 1, do:
1. Designing the messages from V to E: m ~ (mmu-re (1)),, for all E
(1-1)
Sum -~> messages over hyperedges m!!) = Enco multer for all E E
3. Broadcast m! and design the messages from & to V: m j(h (1) _m!),, for all E
Update (t+1) ((h Ee mo. T.. d,), for all E V.
In our tested datasets, hyperedges do not have initial attributes. In our implementation, we assign common learnable vector for every hyperedge as their first-layer features m The performance of ED-HNNII and the comparison with other baselines are presented in Table Our finding is that ED-HNNII can outperform ED-HNN on datasets with relatively larger scale or more heterophily. And the average accuracy improvement by ED-HNNII is around 0.5%. We argue that ED-HNNII inherently has more complex computational mechanism, and thus tends to overfit on small datasets (e.g., Cora, Citeseer, etc.). Moreover, superior performance heterophilic datasets also implies that injecting more equivariance benefits handling heterophilic data. From Table 4 the measured computational efficiency is also comparable to ED-HNN and other baselines.
20

---

Published as a conference paper at ICLR 2023
Table 4:
Additional experiments and updated leaderboard with ED-HNNII. Prediction accuracy (%). Bold font highlights when ED-HNNII outperforms the original ED-HNN. Other details are kept consistent with T
Citeseer
Paloedd
Cond-CA
DBLP-CA
Training Time (10
HGNN [hane & Yogr 2001
79.39 1: 1.36
72.45
1.16
86.44 1 0.44
K2.64
1.65
91.03 1 0.20
0.24 1: 0.51
HCHA Bear
79.14 +: 1.02
72.43
= 1.2
86.41
0.36
82.55
0.9.7
90.92
0.22
0.34 0.01
INNIIN Dong
2020
76.36
1.92
72.61
1.57
86.90
0.30
77.19
1 1.49
86.76
0.29
0.30 0.55
HyperGCX
1.26
71.28
0.82
82.84
1:208
0.42 1.51
UniOCNII Thiings & Hill pura
78.81 +: 1.05
73.05
2.21
88.25
33.60-2:..1
91.69.1:00
4.36 1.18
HyperNij Tiniiooe oe min piaan
79.20
+ 1.14
72.62
1.49
86.68
0.43
80.62 + 1.32
90.35
0.26
0.15 + 1.32
AllDeepseer Caagn a 2003
76.88 + 1.80
70:83
1.63
88.75 + 0.33
81.97
1.50
91.27 + 0.27
1.23 1.09
AUSSetEnandormrr (Chien 2022
78.58
1.47
73.08
88.72
0.37
1.47 83.03
91.53
0.23
1.64 1.63
ED-HNN
80.31 + 1.35
73:70
99.03
83,97
91.90
1.31 1.13
ED HNNI
78.47 4. 1.62
1.56 72.65
0.62 89.56
82:17
1.68
91.93 1: 0.29
1.85 4: 1.09
Congress
Semiie
Walmont
House
Avg. Rank
Inference Time (10 -0
HGNN
Huang & Yanr 2022
91.26 + 1:15
48.59
4.52
62.00
0:24
61.39 + 2.96
6.00
1.01 +: 0.04
HCHA
90.43 + 1:20
48.62
4.41
62.35
0.26
36
2.53
6.67
1.54 + 01.18
HNHN
53.35
1.45
50.93
6.33
47.18
67.50
+ 2.59
7.67
0.05
HyperGCX
55.12
1.96
42.45
44.24
0,06
UniGCN11
Hiang & Vane 2012
94.81 1: 0.81
49.30
= 4.25
$4.45
0.37
67:25 2.57
4.56
21,22 4: 0.13
HyyparD (Tplise Er 2i 2011
74.63 1 3.62
$2.82
3.20
38.10 1 3.86
$1.70 3.37
6,89
0.02 1: 0.01
AilDeupSet Sight
91.80 1: 1.53
48
5.67
64.55
0.33
67.82
2,440
6.22
5.35 4: 0.33
AlSSeTTannfeemrr (Chien
5.22
60.33
2:20
0.67
ED-HNN
95.00 + 0.99
64.79
5.14
66.91
0:41
72.45
1,56
5.87 0.36
ED-HNNII
95.19 + 1.34
63.81 = 6.17
67.24 + 0.45
73.95 + 1.97
2.67
6.07 + 0.40
Table 5: More dataset statistics. CE homophily is the homophily score Pei et al. 2020) based on
CE of hypergraphs.
Cora
Citeseer
Puhmed
Cora- CA
DBLP-CA
Congress
Senate
Walmart
House
# nodes
2708
3312
19717
2708
41302
1718
282
88860
1290
hyperedges
1579
1079
7963
1072
22363
83105
315
69906
340
# features
1433
3703
500
1433
1425
100
100
100
100
classes
2.
do
1,767
1.044
1,756
1.693
2,411
427.237
19.177
5.184
9.181
avg. |e] 3.03
3,200
4.349
4.277
4.452
8.656
17.168
6.589
34.730
CE Homophily
0.897
0.893
0,952
0.803
0.869
0.555
0.498
0.530
0.509
F.2 DETAILS OF BENCHMARKING DATASETS
Our benchmark datasets consist of existing seven datasets (Cora, Citeseer, Pubmed, Cora-CA, DBLP-CA, Walmart, and House) from Chien et al. (202). and two newly introduced datasets (Congress (Fowler 2006a) and Senate (Fowler 2006b)). J. For existing datasets, we downloaded the processed version by byCee et al. al.022)) For co-citation networks (Cora, Citeseer, Pubmed), all doc- uments cited by document are connected by hyperedge (Yadati et al. 2019). For co-authorship networks (Cora-CA, DBLP), all documents co-authored by an author are in one hyperedge (Yadati) et al. 2019). The node features in citation networks are the bag-of-words representations of the cor- responding documents, and node labels are the paper classes. In the House dataset, each node is a member of the US House of Representatives and hyperedges group members of the same committee. Node labels indicate the political party of the representatives. (Chien et al.) 2022). In Walmart, nodes represent products purchased at Walmart, hyperedges represent sets of products purchased together, and the node labels are the product categories (Chien et al. (022). For Congress (Fowler) (006a) and Senate(Fowler) 2006b), we used the same setting as (Veldt et al. 2022). In Congress dataset, nodes are US Congresspersons and hyperedges are comprised of the sponsor and co-sponsors of legislative bills put forth in both the House of Representatives and the Senate. In Senate dataset, nodes are US Congresspersons and hyperedges are comprised of the sponsor and co-sponsors of bills put forth in the Senate. Each node in both datasets is laheled with political party affiliation. Both datasets were from James Fowler's data (Fowler, 2006b a). We also list more detailed statistical information on the tested datasets in 5
F.3 HYPERPARAMETERS FOR BENCHMARKING DATASETS
For a fair comparison, we use the same training recipe for all the models. For baseline models, we precisely follow the hyperparameter settings from (Chien et al. 2022). For ED-HNN, we adopt Adam optimizer with fixed learning rate=0.001 and weight 0.cay=0.0. and train for 500 epochs for all datasets. The standard deviation is reported by repeating experiments on ten different data splits. We fix the input dropout rate to be 0.2, and dropout rate to be 0.3. For internal MI.Ps we add
21

---

Published as a conference paper at ICLR 2023
LayerNorm for each layer similar (Chien et al. 2022). Other parameters regarding model sizes are obtained by grid search, which are enumerated in Table The search range of layer number is {1, 2, 4, 6, 8) and the hidden dimension is {96, 128. 256, 512). We find the model size is proportional to the dataset scale, and in general heterophilic data need deeper architecture. For ED- HNNII, due to the inherent model complexity, need prune model depth and width to fit each dataset.
Algorithm 3: Contextual Hypergraph Stochastic Block Model
Initialization: Empty hyperedge set & Draw vertex set V of 2,500 nodes with class Draw vertex set V/2 of 2,500 nodes with class 2.
For = 0, 1, ..., 1, 000, do:
1. Sample subset ei with 01 nodes from V/-
2. Sample a subset 62 with 022 nodes from V22
3. Construct the hyperedge t E {ei Ue2).
Table Choice of hyperparameters for ED-HNN. short "number of",, hd. stands for hidden dimension, cls. means the classifier. When number of MLP layers equals to 0, the MLP boils down to be an identity mapping.
ED HNN
Cora
Citeseer
Puhmed
Cora-CA
DBLP-CA
Congress
Senate
Walmart
House
iterations
8
4
8
6
8
layers of à
0
0
2
2
2
2
layens of
1
2
2
2
2
2
layers
layer cls.
1
2
2
2
2
MLP hd.
256
256
512
128
128
156
512
512
256
256
256
ED HNNI
Cora
Citeseer
Pubmed
Cora-CA
DBLF-CA
Congress
Senate
Walmart
House
iterations
4
4
4
4
layers of à
2
2
2
2
2
layens of
1
2
2
2
2
2
layers
layer cls.
1
2
MLP hd.
128
128
512
128
128
156
512
512
256
cls.
96
256
128
256
256
128
F.4 SYNTHETIC HETEROPHILIC DATASETS
We use the contextual hypergraph stochastic block model (Deshpande et al. 2018 Ghoshdastidar Dukkipati 2014 Lin & Wang 2018) to synthesize data with controlled heterophily. The gen- erated graph contains 5,000 nodes and two classes in total, and 2,500 nodes for each class. We construct hyperedges by randomly sampling a1 nodes from class and a2 nodes from class 2 with- out replacement. Each hyperedge has fixed cardinality |e| a1 + a2 15. We draw 1,000 hyperedges in total. The detailed data synthesis pipeline is summarized in Algorithm 3. We use min (a1, 022 to characterize the heterophily level of the hypergraph. For
intuitive illus- tration, we list the CE homophily corresponding to different a Table Experiments on synthetic
Table 7: Correspondence between Heterophily and CE Homophily.
3
7
Homophily 0.875
0.765
0.672
0.596
0.495
0.474
heterophilic datasets fix the training hyperparameters and hidden dimension=256 guarantee fair parameter budget ("IM). Baseline HNNs are all of one-layer architecture as they are not scalable with the depth shown in Sec. [4.3 Since our ED-HNN adopts parameter sharing scheme, we can easily repeat the diffusion layer twice to achieve better results without parameter overheads.
F.5 SYNTHETIC DIFFUSION DATASETS AND ADDITIONAL EXPERIMENTS
In order to evaluate the ability of ED-HNN to express given hypergraph diffusion, we generate semi-synthetic diffusion data using the Senate hypergraph (Chodrow et al. 2021 and synthetic node
22

---

Published as a conference paper at ICLR 2023
Total Variation (p = 2)
Lovász Extension- Cardinality
14-
Equivariant
Invariant
12-
AllDespSets
AllSe(Taansformer
MAE (x10-1)
MAE (x11
8-
15
6-
10
256
128
32
256
128
64
32
Figure 5: Comparing the Powers to Represent known Diffusion (using ADMM with the proximal
operators in Eq. and 4: MAE v.s. Latent Dimensions
features. The data consists of 1,000 pairs (H()) H()). The initial node features (0) are sampled from 1-dim Gaussian distributions with mean and variance randomly drawn between and 100.
That is, to generate single instance of H we first pick uniformly from [1,10], and then sample the coordinate entries as !o ~ (O, o²). Then we apply the gradient step in Eq. 2 to obtain the corresponding (1) For non-differentiable node or edge potentials, we adopt subgradients for convenient computation. We fix the node potential f(h;; v.) = (h, xz,²² where E. (0) We consider different edge potentials from Example [ with varying complexities: a) CE, b) TV (p 2) and c) LEC (p = 2). For LEC, we set y as follows: if [e] is even, then y; 2/ if S lel/2 and 2/|le] otherwise; if ee is odd, then y; 2/( 1) if S (le) 1)/2, y/i 0 if 1))/+
and 3/0 2/ 1) otherwise. In order to apply the gradient step in Eq. 2 we need to specify the lcarning rate 77. We choose 7 in a way such that (ar(HH )/Va(( Var H (o)) does not vary too much among the three different edge potentials. Specifically, we set r = 0.5 for CE, = 0.02 for TV and 77 0.1 for LEC.
Beyond semi-synthetic diffusion data generated from the gradient step Eq. we also considered synthetic diffusion data obtained from the proximal operators Eq. and 4 We generated a random uniform hypergraph with 1,000 nodes and 1,000 hyperedges of constant hyperedge size 20. The diffusion data on this hypergraph consists of 1,000 pairs (H) ()). The initial node features (O)) are sampled in the same way as before. We apply the updates given by Eq, and and 4 to obtain H)),, We consider the same node potential and 2 edge potentials TV (p- = 2) and LEC (p 2). We set 1) 1/2 for both cases. We show the results in Figure The additional results resonate with our previous results Figure Again, our ED-HNN outperforms other baseline HNNs by significant
margin when hidden dimension is limited.
F.6 MORE COMPLEXITY ANALYSIS
In this section, we provide more efficiency comparison with HyperSAGE (Arya et al. 2020) and LEGCN (Yang al. 2020). As we discussed in 3.4 HyperSAGE and LEGCN may be able to learn hyperedge cquivariant operator. However, both HyperSAGE and LEGCN need to build node- hyperedge pairs, which is memory consuming and unfriendly efficient message passing imple- mentation. To demonstrate this point, we compare our model with HyperSAGE and LEGCN in terms of training and inference efficiency. We reuse the official code provided in OpenReview to benchmark HyPerSAGE. Since LEGCN has not released code, we reimplement it using Pytorch Ge- ometric Library. We can only test the speed of these models on Cora dataset, because HyperSAGE and LEGCN cannot scale up there is no efficient implementation yet for their computational pro- hibitive preprocessing procedures. The results are presented in Table [ We note that HyperSAGE cannot easily employ message passing between V and &, since neighbor aggregation step in Hyper- SAGE needs to rule out the central node. Its official implementation adopts a naive "for"-loop for each forward pass which is unable fully utilize the GPU parallelism and significant deteriorates the speed. LEGCN can be implemented via message passing, however, the graphs expanded over node-edge pairs are much denser thus cannot scale up larger dataset.
23

---

Published as a conference paper at ICLR 2023
Table 8: Performance and Efficiency Comparison with HyperSAGE and LEGCN. The prediction
accuracy of HyperSAGE is copied from the original manuscript (Arya et al. 2020
HyperSAGE At 2020
LEGCN (Yang al. 2020
ED-HNN
ED-HNNI
Training Time (10 s) Inference Time (10
43.93
0.56 0.71
0.15 0.68
0.25 + 0.48 0.20 + 0.08
297.57 30.57
0.42 0.06
0.15 0.03
Prediction Accuracy (%)
69.30 2.70
73.34 + 1.06
80.31 # 1.35
78.47 + 1.62
Table 9: Sensitivity Test on Hidden Dimension on Pubmed Dataset.
Model
512
256
128
64
AetDeepSets (Chien 2022
88.75 0.33
88.41 + 0.37
87.50 + 0.42
86.78 0.40
AllSeCTransfommer (Cheen 2022
88.72
0.37
88.16
1 0.24
87.36 1: 0.23
86.21 1 0.25
ED-HINN (ours)
89.03 0.53
88.74 1 0.38
88.84 1: 0.38
88.76 1 0.24
F.7 SENSITIVITY HIDDEN DIMENSIONS
We conduct expressivity vs hidden size experiments in Tab. g where we control the hidden dimen- sion of hypergraph models and test their performance on Pubmed dataset. We choose top-performers AllDeepSets and ranforrrr as the compared baselines. On real-world data, our 64-wids 44-width ED-HNN can even achieve on-par results with 512-width AllSet models. This implies our ED- HNN has better tolerance of low hidden dimension and we owe this to higher expressive power by achieving an equivariance.
24


Reviews:
[{"review_id": "scTmrFdvg47", "reviewer": "Reviewer_n5MN", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper proposed ED-HNN, a GNN for hypergraphs inspired by the algorithm for solving hypergraph diffusion (optimization problem on hypergraphs). ED-HNN approximates the gradient-based optimization algorithm for hypergraph diffusion by a message passing on a bipartite graph that is equivalent to the original hypergraph. As a theoretical analysis, this paper showed the representation theorem for equivariant functions and justified the architectural design of ED-HNN. Finally, this paper applied ED-HNN to various graph data as an empirical evaluation.", "summary_of_the_review": "The proposed method shows good prediction accuracy in numerical experiments. On the other hand, on the theoretical side, the proposed method is justified by associating it with hypergraph diffusion. However, I have questions about the validity of the arguments and would like to clarify them. Also, if I understand correctly, the representation theorem for equivariant functions, one of this paper's contributions, is known in existing studies.\n\n# Post-rebuttal comments\n\nAfter the discussion with the authors, I increase my score (5 -> 6). (Note that there is a possibility that my final evaluation could change after the further discussion with other reviewers and area chairs)\n\nI raised two weak points in the first review comment. One is that the relationship between the hypergraph diffusion problem and the proposed model is unclear. The other is that the universality result (Theorem 1) is known. The question related to the first question was solved through the discussion with the authors. I understand that the proposed model is a gradient-based optimization of a hypergraph diffusion problem for some unknown function $f$ and $g$. Also, making $m^{(t)}_{e\\to v}$ equivariant, which is inspired by the hypergraph diffusion, certainly improved the empirical performance.\nOn the other hand, my concerns about the second question were correct. However, the second point was not strong compared with the first point.\nIn conclusion, since I recognize the novelty and significance, I increase my score from 5 to 6, tending to accept.", "strength_and_weaknesses": "Strengths\n- Numerical evaluation used various types of graph data. It demonstrates the broad applicability of the proposed model.\n\nWeaknesses\n- The parameterization of NNs somewhat hinders the relationship between the proposed method and underlying hypergraph diffusion (see the Quality section for details).\n- Similar results exist for the representation theorem for equivariant functions (Theorem 1) in existing studies.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "mC5kBFuG7mf", "reviewer": "Reviewer_z25x", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper proposes a hyper-graph neural network model, ED-HNN, that can model hypergraph diffusion process. They claimed that ED-HNN shows superiority in processing heterophilic hypergraphs and constructing deep models. ", "summary_of_the_review": "Please refer to the Strength And Weaknesses. If the authors could address the problems in the weakness, I’d like to raise my scores.", "strength_and_weaknesses": "**Strength:** \n - 1. The presentation is clear. \n - 2. The overall experiments show the superiority of their proposed algorithm, ED-HNN.\n\n**Weakness:** \n\nThe main claims of the paper are not well explained or supported. In particular, section 3.3 tried to discuss the advantages of ED-HNN for the design of equivariant diffusion operators. However, I have several questions in this section. \n- 1. For the claim that “Invariant diffusion by forcing the operation on hyperedge e to follow that the partial derivation to each dimension is the same”, could you please refer to some reference or give proof on this? It is not obvious for a reader that is not familiar to the invariant diffusion, \n- 2. “Moreover, a learnable operator is important.” This conclusion is weak, since it only tells that it exists such parameters that can help but didn’t say whether it is difficult to optimize to such parameters. Could you give some empirical experiments on the sensitivity of the parameters to show whether the learnable operator is difficult to optimize? \n- 3. “equivariant hyperedge diffusion operators are also good at building deep models” is not well supported and explained. Although Figure 2 shows an interesting result that ED-HNN successfully leverages deeper architecture to achieve higher accuracy, there is no clear trend to show that How about the performance of ED-GNN when the number of the layers go to larger than 8? Meanwhile, the claim that “Equivariant operators allocating different messages across nodes helps with overcoming the oversmoothing issue.” is not obvious to me. I could see those equivalent operators may reduce the risk of overfitting, but how do equivalent operators help the over-smoothing issue? Is there any insight on this point?", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "lXH4BmO2lnV", "reviewer": "Reviewer_GNhA", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper proposes Equivariant Diffusion-Hypergraph Neural Network (ED-HNN) for hypergraph structured data.\n\nED-HNN is implemented as a message-passing neural network on the star expansion of the hypergraph and can provably represent any continuous hypergraph diffusion operator.\n\nED-HNN competes with recent HNNs on hypergraph node classification datasets.\n\n___", "summary_of_the_review": "While the proposed method is theoretically grounded and effective on standard benchmarks, the paper can be improved in terms of clarity and quality.\n\n___", "strength_and_weaknesses": "\\\n**Strengths**\n\n\\+ The paper is well-organised and relevant to the ICLR community.\n\n\\+ The theory in the paper significantly establishes connections between HNNs and hypergraph diffusion.\n\n\\+ ED-HNN works well in practice on several node classification datasets in comparison with several existing HNNs.\n\n\n\\\n**Weaknesses**  \n\n\\- The key innovation in the design of ED-HNN (i.e., Step 3 in Algorithm 1) is marginal with respect to prior art.\n\n\\- ED-HNN (and message passing on star expansion) can potentially be problematic for real-world hypergraphs in which the vast majority of hyperedges are of size two, e.g., contact networks, online forum thread networks.\n\n___", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "iT_CreMfAD", "reviewer": "Reviewer_TN8f", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper studies the learning on hyper graphs. Targeting the problem that existing hyper graph networks often require domain knowledge in designing the mechanism to process the higher-order relations in hyper graphs, this paper proposes a hypergraph diffusion based model that can model a wide range of higher-order relations without explicitly designing based on domain knowledge. Theoretical analysis is also provided on the proposed model. Expeiments are conducted on 9 datasets and are compared with 8 baselines.\n", "summary_of_the_review": "Overall, this paper has a clear motivation targetng a meaningful problem, which is the good aspect. However, there is also some unlcear part regarding the experiments. Therefore, I would rate it as slightly below threshold currently", "strength_and_weaknesses": "Strengths:\n\nThis paper has a clear motivation on the proposed model.\n\nThe proposed model is accompanie with theoretical analysis.\n\nThe experiments are comprehensive. Mltiple datasets and baselines are included. \n\nWeakness:\n\n1. The adopted datasets seem not to be hypergraph datasets. As introduced in the paper, in a hypergraph, a hyperedge connects a set of nodes. However, in datasets such as cora, citeseer, pubmed, each edge only connect two nodes, and they are just standard graph datasets. Isn't there any dataset that contain true hyper edges？It is not convincing that the simple dataset like cora requires many higher-order relations to capture during classification.\n\n2. Since some of the adopted datasets are standard graph datasets, which were also widely adopted in works on standard GNNs, like GCN [1], the compared baselines should also include these methods. As far as I know, the performance of GCN on Cora is higher than any method shown in Table 2.\n\n[1] Kipf, Thomas N., and Max Welling. \"Semi-supervised classification with graph convolutional networks.\" arXiv preprint arXiv:1609.02907 (2016).", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Decision: Accept: poster