Augmentations in Hypergraph Contrastive Learning: Fabricated and Generative
Authors: Tianxin Wei
Yuning You
Tianlong Chen
Yang Shen
Jingrui He
Zhangyang Wang
Venue: NeurIPS 2022 Accept (2022)
Keywords: hypergraph contrastive learning
generative augmentation

Abstract: This paper targets at improving the generalizability of hypergraph neural networks in the low-label regime, through applying the contrastive learning approach from images/graphs (we refer to it as HyperGCL). We focus on the following question: How to construct contrastive views for hypergraphs via augmentations? We provide the solutions in two folds. First, guided by domain knowledge, we fabricate two schemes to augment hyperedges with higher-order relations encoded, and adopt three vertex augmentation strategies from graph-structured data. Second, in search of more effective views in a data-driven manner, we for the first time propose a hypergraph generative model to  generate augmented views, and then an end-to-end differentiable pipeline to jointly learn hypergraph augmentations and model parameters. Our technical innovations are reflected in designing both fabricated and generative augmentations of hypergraphs. The experimental findings include: (i) Among fabricated augmentations in HyperGCL, augmenting hyperedges provides the most numerical gains, implying that higher-order information in structures is usually more downstream-relevant; (ii) Generative augmentations do better in preserving higher-order information to further benefit generalizability; (iii) HyperGCL also boosts robustness and fairness in hypergraph representation learning. Codes are released at https://github.com/weitianxin/HyperGCL.

Augmentations in Hypergraph Contrastive Learning: Fabricated and Generative
Tianxin Wei' Yuning You?*, Tianlong Chen Yang Shen', Jingrui He', Zhangyang Wang² University of Illinois Urbana- Saahann--hampaign, Texas A&M University, University of Texas at Austin you, ysuu,s yshen)/tamu. Stamu. edu, {tianlong, chen, ataaaaaaaaa///uteeaaa.... edu
{twei10, jiniuo,iinnuueeee ligrui)@@llinois edu, (yuning, momingg-oo,,,
Abstract
This paper targets at improving the generalizability of hypergraph neural networks in the low-label regime, through applying the contrastive learning approach from images/graphs (we refer to it as HyperGCL). We focus on the following question: How to construct contrastive views for hypergraphs via augmentations? We pro- vide the solutions in two folds. First, guided by domain knowledge, we fabricate schemes augment hyperedges with higher-order relations encoded, and adopt three vertex augmentation strategies from graph-structured data. Second, search of more effective views in data-driven manner, we for the first time propose a hypergraph generative model to generate augmented views, and then end-to-end differentiable pipeline jointly learn hypergraph augmentations and model parameters. Our technical innovations are reflected in designing both fabricated and generative augmentations of hypergraphs. The experimental findings include: (i) Among fabricated augmentations in HyperGCL, augmenting hyper- edges provides the most numerical gains, implying that higher-order information in structures is usually more downstream-rellvant (ii) Generative augmentations do better in preserving higher-order information further benefit generalizability; (iii) HyperGCL also boosts robustness and fairness in hypergraph representation learning. Codes are released at tttpps///ithhan wwwm//eitianiii//yyeeeaeaeee
1 Introduction
Hypergraphs have raised a surge of interests in the research community 2 3 due to their innate capability of capturing higher-order relations [4). They offer powerful tool model complicated topological structures in broad applications, c.g., recommender systems [ 6) financial analyses
7. R. and bioinformaties [ g 10). Concomitant with the trend, hypergraph neural networks
(HyperGNNN) have recently been developed
2 for hypergraph representation learning.
This paper focuses on the few-shot scenarios of hypergraphs, i.c., task-specific labels are scarce, which are ubiquitous real-world applications of hypergraphs [5] 7
9 and empirically restrict the generalizability of HyperGNNs. Inspired by the emerging self-supervised learning images/graphs
12
13
14 15, 16), especially the contrastive approaches [12 14 17 18 19] 20 22
25
set out leverage contrastive self-supervision address the problem.
Nevertheless, one challenge stands out: How to build contrastive views for hypergraphs? The success of contrastive learning hinges on the appropriate view construction, otherwise it would result in "negative transfer" [12 [4]. However, it is non-trivial build hypergraph views due their overly intricate topology, i.e., there are E possibilities for one hyperedge on N vertices, versus (N) for one edge in graphs. To date, the only way of contrasting is between the representations
'Equal contribution.
36th Conference on Neural Information Processing Systems (NcurIPS 2022).

---

of hypergraphs and their multiple neural networks of different modalities (hypergraphs and variants of expanded graphs) need to be optimized. More importantly, contrasting on clique expansion has the risk of losing higher-order information via pulling representations of hypergraphs and graphs close.
graphs |6,27], which is computationally expensive as
Contributions. Motivated by [12] [14] that appropriate data augmentations suffice for the effective contrastive views, and intuitively they are more capable of preserving higher-order relations in hypergraphs compared to clique expansion, we explore on the question in this paper, how to design augmented views of hypergraphs in contrastive learning (HyperGCL). Our answers are in two folds.
We first assay whether fabricated augmentations guided by domain knowledge are suited for Hy- perGCL. Since hypergraphs are composed of hyperedges and vertices, to augment hyperedges, propose two strategies that (i) directly perturb on hyperedges, and (ii) perturb on the "edges" between hyperedges and vertices in the converted bipartite graph; To augment vertices, adopt three schemes of vertex dropping, attribute masking and subgraph from graph-structured data [4]. Our finding is that, different from the fact that vertex augmentations benefit more on graphs, hypergraphs mostly ben- efit from hyperedge augmentations (up to 9% improvement, revealing that higher-order information encoded in hyperedges is usually more downstream-relevant (than information in vertices).
Furthermore, in search of even better augmented views but in a data-driven manner, we study whether/how augmentations of hypergraphs could be learned during contrastive learning. To this end, for the first time, we propose a novel variational hypergraph auto- onto-en architecture, as a hypergraph generative model, to parameterize certain augmentation space of hypergraphs. In addi- tion, we propose an end-to-end differentiable pipeline utilizing Gumbel-Softtax (28]. to jointly learn hypergraph augmentations and model parameters. Our observation is that generative augmentations can better capture the higher-order information and achieve state-of-the-art performance on most of the benchmark data sets (up 20% improvement.
The aforementioned empirical evidences (for generalizability) are drawn from comprehensive experi- ments 13 datasets. Moreover, we introduce the robustness and fairness evaluation for hypergraphs, and show that HyperGCL in addition boosts robustness against adversarial attacks and imposes fairnsss with regard to sensitive attributes.
The rest of the paper is organized as follows. We discuss the related work in Section 2 introduce HyperGCL in Section 3 present the experimental results in Section 4 and conclude in Section
2 Related Work
Hypergraph neural networks. Hypergraphs, which are able encode higher-order relationships, have attracted significant attentions in recent years. In the machine learning community, hypergraph neural networks are developed for effective hypergraph representations. HGNN (I adopt the clique expansion technique and designs the weighted hypergraph Laplacian for message passing. HyperGCN (2] proposes the generalized hypergraph Laplacian and explores adding the hyperedge information through mediators. The attention mechanism (29) 30 is also designed to learn the importance within hypergraphs. However, the expanded graph will inevitably cause distortion and lead to unsatisfactory performance. There is also another line of works such as UniGNN [I and HyperSAGE [32) which try to perform message passing directly on the hypergraph to avoid the information loss. recent work 3 provides an AllSet framework to unify the existing studies with high expressive power and achieves state-of-the-art performance on comprehensive benchmarks. The work utilizes deep multiset functions 33 to identify the propagation and aggregation rules in a data-driven manner.
has achieved un- precedented success in computer vision. The core idea is to learn an embedding space where samples from the same instance are pulled closer and samples from different instances are pushed apart. Re- cent works start to cross- cooss-poll between contrastive learning and graph neural networks to for more generalizable graph representations. Typically, they design some fabricated augmentations guided by domain knowledge, such as edge perturbation, feature masking or vertex dropping, etc. Nevertheless, contrastive learning on hypergraphs remains largely unexplored. Most existing works (6
Contrastive self-supervised learning. Contrastive self-supervision 12 34 35)
36
26 37
design pretext tasks for hypergraphs and mainly focus on recommender systems 338
39
40
41 via
contrasting between graphs and hypergraphs which might lose important higher-order information.
In this work, explore on the structure of hypergraph itself to construct contrastive views.

---

Contrastive Views
o ft
Input g
A1 A1-A
Shared
Node Contrst
o ft
Figure 1: The framework of hypergraph contrastive learning (HyperGCL). The ellipses represent the hyperedges. Two contrastive views are generated by hypergraph augmentations A1 and A2 from the augmentation collection A. f()) and h(-) are shared encoder and projection head respectively. In the figure, we show two examples of hypergraph augmentations. At the top, the dotted ellipse denotes the deleted hyperedge. At the bottom, one vertex in the dotted hyperedge is removed.
Methods
3.1 Hypergraph Contrastive Learning
hypergraph denoted G (V,E) e where V {VI, ..., flyB is the set of vertices and = (ei is the set of hyperedges. Each hyperedge en {r1; ... "/e,.) represents the higher-order interaction among a set of vertices. State-of-the-art approaches to encode such complex structures hypergraph neural networks (HyperGNNs) ( 3], mapping the hypergraph
D-dimension latent space via f G + IR" with higher-order message passing.
adopt contrastive learning further improve the generalizability of HyperGNNs in the low-label regime (HyperGCL). Main components of our Hy- perGCL, similar images/graphs [4] include: (i) hypergraph augmentations for contrastive views, (ii) HyperGNNNs as hypergraph encoders, (iii) projection head h(-)) for representations, and (iv) contrastive loss for optimization. The overall pipeline is shown in Figure T Detailed descriptions and training procedure are shown Appendix B. The main challenge here is how to effectively augment hypergraphs build contrastive views.
Motivated from learning images/graphs.
3.2 Fabricated Augmentations for Hypergraphs
We first explore whether manually designed augmen- tations are suited for Hyper- GCL. Since hyperedges and vertices compose hyper- graph, augmentations are fabricated with regards to topology and node features, respectively.
Vertex
Transform
Hyperedge
A1. Perturbing hyper- edges. The most direct aug- mentation on higher-order interactions is to perturb on set of hyperedges. Since adding hyperedge is confronted with the combinatorial challenge (see Sec. of of introduction). here we focus on randomly removing the existing hyperedges following an i.i.d. Bernoulli distribution. The underlying assumption is that the partially missing higher-order relations do not significantly affect the semantic meaning of hypergraphs.
Figure 2: Conversion from hypergraph to equivalent bipartite graph.

---

A2. Perturbing edges in equivalent bipartite graph. To augment higher-orden relations in more fine-grained way, we first convert the hypergraph into the equivalent bipartite graph, where two disjoint sets of vertices represent vertices and hyperedges in the hypergraph, respectively (see Figure 2). On top of the bipartite graph, we perform random removal of edges. A2 disrupts the higher-order relations via randomly kicking out vertices from hyperedges, enforcing the semantics of hypergraph representations to be robust to such disruption. A2 is essentially the generalized version of Al.
Moreover, we find that vertex augmentations for graph-structured data 141 are applicable to hyper- graphs. Therefore, we adopt three additional schemes of vertex dropping (A3), attribute masking (A4) and subgraph (A5) into our experiments, with similar prior knowledge incorporated as in [14].
3.3 Generative Models for Hypergraph Augmentations
Manually designing augmentation operators requires wealth of domain knowledge, and might lead to sub-optimal solutions even with extensive trial-and-crrors. We next study whether/how augmentations of hypergraphs could be learned during contrastive learning. Two questions need to be answered here: (i) How to parameterize the augmentation space of hypergraphs? (ii) How to incorporate the learnable augmentations into contrastive learning?
3.3.1 Hypergraph Generative Models for Augmentations
Considering an augmentation operator defined as the stochastic mapping between two hypergraph manifolds that y G - G, natural thought is to adopt the generative model to parameterize the augmentation space, which in general is composed of deterministic encoder h1 G + D and stochastic decoder (or sampler) fy IR D' G. In this way, 9 hı h22.
Following this thought, inspired by the well-studied generative models with variational inference 42 45 we propose novel variational hypergraph auto-encoder architecture (VHGAE). To the best of our knowledge, this is the first hypergraph generative model for generating augmentations of hypergraphs, which will be used as A6. Notice that here it only parametrizes the augmentation space of edge perturbation, and the future node perturbation would be included. VHGAE consists of the encoder and decoder neural networks. The overall framework is shown in Figure 3
Encoder. The encoder embeds hypergraphs into latent representations. Instead of embedding
hypergraph into a single vector, we follow VGAE 43 to embed it into a set of vertex and in additional hyperedge representations, facilitate the further decoding process of non- non-Euclidean structures. We adopt two HyperGNNs, h² and h", to encode the mean and the logarithmic standard deviation for variational distributions of vertex and hyperedge representations žy q (2) N (py, oB), ze ~ as follows (please refer to Appendix B for the detailed computing pipeline):
(ze (G) Nuus, o3) Jv. Mg (9), log(ov), log(oz) h (9),
(1)
where E D' log(o) E D'xMM We here leverage the higher-order message passing in HyperGNNs for a better encoding capability.
Decoder. With the learned vertex and hyperedge variational distributions, the decoder attempts to reconstruct the higher-order relations of hypergraphs. However, modeling the space of higher-order interactions encounters the combinatorial challenge (see Section T.. Adopting the similar strategy as in the augmentation A2 (see Section (32), designate the decoder recover the relations on the converted bipartite graph 9 (V,E) for approximation. Mathematically, we formulate decoding as:
M
[
((9)z,,z8) p(r]=v, 28)
II
((E,,clzz, ze)
II
Sigmoid (T[=z =e)s
where Wee =T= ze is the learned edge logit. On the decoded topological distribution of the bipartite graph, we perform sampling and then convert the sample back to the hypergraph (the conversion between hypergraphs and bipartite graphs is lossless).
Generator optimization. With variational inference
42
45
we optimize the hypergraph
generator on the evidence lower bound (ELBO) as follows:
ELBO- = (ss (g) E. (zvlg) Po Iw., ze)]
(L|yo(zv|
9 |((x))] - KL 9) |((E)),
(3)

---

Ogen
Encoder Rotzelo)
Sample 2g zv
Decoder ((lay)
geen (Ø) B Lot ( 6)
Figure 3:
Framework of the proposed variational hy hypergraph auto-encoder (VHGAE). The green lines indicate these modules participated in the optimization process.
where (z<2)()) ( and (o) are their variational distribution, p(zy) and p(zg) are default Gaussian priors with p(zy) (0, I), p(ze) (O, I). When generating hypergraphs, the generator would sample the relations on the converted bipartite graph with probability p(g]2y,
3.3.2 Jointly Augmenting and Contrasting with Gumbel-Softmax
With hypergraph augmentations parametrized with generative models, the next step is to incorporate augmentation learning into HyperGCL. The main barrier results from the discrete sampling of hyperedges which is non-differentiable. To tackle it, we leverage the Gumbel-Softmax trick (28 for the hyperedge distribution as:
T(g) Gumbel-Softmax//e æv, 2v)
Sigmoid(( + log(8) - log(1 - 5))/T)
(4)
ggm T(9) G.
where wyg denotes the learned edge logits (before Sigmoid) and 8 ~ Uniform(O, 1). When hyperpa- rameter temperature - 0, the results get closer to being binary. T is the sampled one-hot vector for each hyperedge-vertes, interaction in the hypergraph G. Then the sampled vector will be applied to perform augmentation. During the Gumbel-Softmax, we leverage the reparametrization trick [2] to smooth the gradient and make the sample operation differentiable. Thus, this objective can be optimized an end-to-end manner as:
mingCgen(d)) - 3 La(9, Lal9, 10,
(5)
where Lgen ELBO is the generator loss to be minimized, / is the tradeoff factor. Due to the computational cost of collaboratively optimizing two generative views, we train one VHGAE to produce one generative view, with the other view Gp is kept as fabricated. To be specific, (i) independently optimizing two hypergraph generators is of reasonable budgets but would lead to distribution collapse (i.c., two hypergraph generators output the same distribution) [1,2] which results in less effective generative views, while (ii) the collaborative optimization techniques for graph generators (e.g. REINFORCE on the rewards of generative graph structures) are not directly applicable HyperGCL due the combinatorial challenge of hypergraph structures (which is computationally expensive). The goal of this multi-task loss is to generate stronger augmentation (maximize contrastive loss) push HyperGNN avoid capturing redundant information during the representation learning, while at the same time learning the hypergraph data distribution.
4 Experiments
4.1 Setup
We examine our methods on the most comprehensive hypergraph benchmarks with 13 data sets, with statistics shown Table Please refer Appendix for detailed information. We focus on semi-supervised vertex classification in the transductive setting. Different from the existing work

---

Table 1: Data statistics: hes hr are the node homophily and hyperedge homophily in hypergraph. Higher value indicate the hypergraph is more homogeneous. Details can be found in Appendix C.
Corn
Citeseer
Puhmed
Com-CA
DBLP.CC
7ap
20News
Mushroom
NTU/0112
ModelNer40
Yelp
House
Walmart
[
2708
3312
19717
2708
41302
101
16242
8124
2012
12311
50758
1290
88860
[
1579
1079
7963
1072
22363
43
100
298
2012
12311
679302
341
69906
Teature
3703
1425
100
1862
# class
7
6
3
7
6
7
4
2
67
40
9
2
11
he 0.86
0.83
0.88
0.88
0.93
0.66
0.87
0.92
0.58
bo 0.84
0.78
0.79
0.79
0.88
0.35
0.49
0.87
0.81
0.88
0.26
0.52
0.55
3] that leverages 50% of all vertexes as the training set, we focus on the more low-label regime of challenging and practical applications. By default, we split the data into training/validationn samples using (10% splitting percentages. Each experiment is run for 20 different data splits and initialization with mean and standard deviation reported. We adopt state-of-the-e SetGNN [3] as the backbone HyperGNN architecture. For basclines, we compare two existing hypergraph
self-supervised approaches 36 and [26] in recommender systems, denoted as Self and Con. They conduct self-supervvsed learning between the hypergraph and conventional graph. By default, we adopt multi-task training to incorporate contrastive self supervisoon because it performs the best as shown in the comparison Section 4.2 All the implementation details are listed in Appendix C.
More experiments of the hyperparameters study are given in Appendix A.
4.2 Results
Comparison among different hypergraph aug- mentations. The augmentation operations are summarized in Table 2 Please refer to Appendix C.5 for detailed descriptions. We first conduct experiments compare different contrastive op- erations on hypergraphs, with results shown in Table In general. generalized hyperedge aug- mentation (A2) works the best among fabricated augmenting operators, but not naïvely perturbing hyperedge (A1). Specifically, among all fabricated augmentations, A2 performs the best in 10 of 13 data sets. This indicates the nature that higher- order information structures is usually more downstream-relevant
Table 2: The proposed augmentation operations for contrastive learning framework HyperGCL and their corresponding names.
Name
Operation
AO
Identity
A1
Naïve Hyperedge Perturbation
A2
Generalized Hyperedge Perturbation
A3
Vertex Dropping
A4
Attribute Masking
AS
Subgraph
A6
Generative Augmentation
For our generative augmentation (A6), we find it performs the best in all the data sets. In our joint augmenting and contrasting framework, we generate stronger augmentation while keeping the hypergraph distribution with adversarial learning. This illustrates the importance of exploring the hypergraph structure. We also test our method on 1% label setting in Table 4 In this setting, Zoo and NTU2012 data sets are not shown because of the extremely small data size (each case has less than one training sample). We can find that in the 1 % label setting A4 (mask) method performs the best in Cora, Citeseer and DBLP-CA. These data sets are all originally graphs, and are constructed as hypergraphs in different ways. So on these data sets, relatively little higher-order structural information can be explored with hypergraph structure perturbation-bart contrastive learning.
Comparison between multi-task learning and pretraining. We then compare the multi-task training method with the pretraining method in Table 5 Pretrain L adopts the linear evaluation protocol where linear classifier is trained on top of the fixed pretrained representations, Pretrain _F follows fully finetuning protocol that uses the weights of the learned hypergnn encoder as initialization while finetuning all the layers. MTL denotes the multi-task learning method which trains the supervised classification loss and contrastive loss together. For all the methods, we use A2 (generalized hyperedge perturbation) as it performs the best among fabricated augmentations. From the table, we find MTL achieves the best performance in nearly all data sets. Pretrain_L- and Pretrain_F can only obtain better performance on two small data sets: Zoo and House. We find on most data sets, Pretrain makes the model perform worse, which shows that the linear classifier is not enough to represent the higher-order information in the hypergraph. Pretrain F has much better performance compared

---

Table 3: Results on the test data sets: Mean accuracy (%) - standard deviation. Bold values indicate the best result. Underlined values indicate the second best. 10% of all vertices are used for training.
Cora
Citeseer
Puhmed
Cora-CA
DBLP-CA
7no
20Newsgroups
Mushroom
SeIGNN
67.93
1.27
63.53
1.32
84,33
0.36
72.21
+ 1,51
19.5140.1
65,06 + 12.82
79.37 a
99.75 0.11
Self
68.24
1.12
62.49
1448
84.38 + 0.38
72.74
1.53
89.51
0.23
57.35 + 18.32
79.45
0.332
95.83 + 0.23
Con
68.89
1.80
62.82
84.56
73.22
1.65
0.
61.05
14.54
0.45
95.85 0.31
AO
68.59
1.33
62.25
2.15
0,454
71,85
1.62
0.22
62.57
13.84
79.07 0.46
99.77 + 0.17
A1
72.39
1.34
66.28
1.27
0,.1
75.45 1.54
0..8
65.80
13.31
0.33
99.80 +
A2
72.58
1.09
66.40=
1.35
0,38
1.42
0.22
66.35
13.26
79.56
99.80 1 0.17
A3
72.33
1.23
65.79
1.18
85.24
0.28
75.34
a 1.40
89.85 a 0.16
65.79
14.05
79.47
0.334
99.81 1: 0.10
A4
72.95
1.19
66.22
0.95
84.88
0.38
75.29
+ 1.56
90.10
0.
62.59
12.77
79.45 4 0.48
99.80 0.14
A5
67.90
0.99
63.21
= 1.25
84.48
+ 0.40
72.61
+ 1.86
89.75
+ 0.24
62.47
+ 12.39
79.42
4- 0.52
99.79 + 0.10
A6
1.48
66.94
1.00
85.72 0.38
76.21
1.26
90.28 + 0.19
66.89 12.44
79.78 0.40
99.86 0.10
NTU2012
ModelNev40
Yelp
House (0.6)
House (1.0)
Walmart (0.6)
Walmart (1.0)
Avg. Rank
SeIGNNN
73.86 =: 1.62
95.85 = 0.38
78.78.2 a 1.51
68,54 a 1,89
58,34 a 2.25
74.97 a: 0.22
$9.13 +: 0.20
7.71
Self
73.41
1.65
0.23
4,15
67.75
3.29
74.76
Con
73:27
1.53
95
26.14
1.86
58.56 6.21
68.50 + 2.52
2.42
0.21
59,39 0.20
7.07
AO
73.54
1.95
95.92
29:43
: 1.42
57.39
2.37
0.21
4.66
8.21
A1
7
95.87
27.18
68.64
2,99
3.220
0.13
60,09 a 0.25
4.50
A2
74.88
1.66
0.356
31.39
2.45
69.73
58,90
1.97
0.18
60.19
2.29
A3
74.68
1.74
96.48
27.57
1.00
67.88
2..21
75.29
0.20
4.71
A4
74.83
1.75
95.86
0.28
29.64
+ 1,93
69.56 2.89
58.91
2.69
75.43 0.18
$9,90 0.24
4.14
A5
74.11
1.86
96
0.33
29.24
1.42
68.14
57.70
2.98
75.26
59.81 0.22
A6
75.34
96.93
0,33
34,64
# 0,39
70.9 + 2.27
59.93
1.99
75.62 0.16
60.46 + 0.20
1,00
Table 4: Results on the test data sets: Mean accuracy (%) = standard deviation. Bold values indicate the best result. 1 5 of all vertexes are used for training.
Cora
Cileveer
Pubmed
Cora-Ch CA
DBLP-CA
20Newsgroups
Mushzoom
SeIGNN
46.48 + 1.62
47.01 4,31
1 1.19
52.29 = 4.18
85,52 + 0.54
73.83 = 1.40
97.73 a 1
Self
45.79
44.22
76.71
0.90
5.37
84,42
73.91 = 0.90
92.25
0.89
Con
4.330
4,88
77,51
1.08
52.37
86.47
74.39 1.23
92.43
0.87
AD
48.50
4..43
78.83
1.79
49.87
5.08
0.773
74.43 1.11
97.32
1.33
5.04
3.96
80.13
1.44
6.98
87.53 0.30
74.68 1.31
97.95
1.15
A2
56.81
46.81
56.
1. 2.86
80.22
1.24
6.36
88.3
74.72
1.16
98.05
1.18
3.694
3.40
80.13
60.51
1.00
98.04
A4
5.15
57.16
4,62
10.11:
1.02
60.91
88,91 + 0.29
74.67
1.39
97.72
1.12
46.23
45.07
4.89
75.95
1.32
53.26 4.86
87.12 0.43
74.81
1.04
97.
1.25
A6
5.000
3.227
81.10
0.80
61.76
= 4.94
88.55 4 0.41
75.52 0.93
98.28 1.03
ModelNet40
Yelp
(0.6)
(1.0)
Walmart (0.6)
Walmart (1.0)
Arg. Rank (L)
SeIGNN
88.34
2.69
27.64
1.10
$3.69
$1.85
1.64
65.48 + 0.45
51.15 = 0.52
7.62
Self
86.85
3.03
20.77
5.15
53.42
51.14
1.75
65,23 a 0.43
51.00 0.41
9.69
Con
47.00
24.23
0.43
53.58
3.04
51.96
1.87
65.47 + 0.44
51.13 0.46
7.31
2.775
51.70 2.76
0.47
51.10
7.46
A1
89.34
26.
18 0.51
3.29
52.23
2.46
65.96 + 0.36
51.22 0.35
4.08
89.37
27.67
54.42
52.31
1.44
0.33
2.69
A3
89.31
26.98
0.66
23.71
52.11
65,88 + 0.50
0.53
4.38
A4
89.03 2
27.45
0.81
51.77 2..64
65.55 0.51
51.04 0.47
4.54
AS
89,43
2.68
28.09
0.96
54.07
51.94
1.84
65.52
à 0.39
50.97 0.47
6.00
A6
90.23
2.72
29.61
+ 0.71
56.27
52.55
2.18
66.42 + 0.40
51.82 = 0.39
1.23
with Pretrain_L, which indicates the effects of using contrastive learning. However, the method switches the objective during finetuning, which would lead to the memorization problem and the loss of pre-trained knowledge. Therefore, all our other experiments adopt MTL setting to incorporate contrastive sulevvupevv.
Comparison to the converted graph. Next we investigate on which graph should we contrast on. We first convert the original hypergraph into a conventional graph using the clique expansion technique, and we choose the representative HGNN[]] as the backbone network for learning on the converted graph. We compare it with two representative augmentations: A2 (Here, edge perturbation on the conventional graph) and A4 (feature masking). In Table 66, we find that HGNN performs poorly compared with SeIGNN. Apparently contrastive self-supervision on converted graphs does not bring much benefit. The reason is that part of structural information, important to hypergraph representation learning, lost when hypergraphs are converted to graphs. These results indicate the importance of designing HyperGNN and contrastive strategies directly on hypergraphs.
7

---

Table 5: Results of different self-supervised mechanisms: Mcan accuracy (%) - standard deviation. Bold values indicate the best result. 10% of all vertexes are used for training.
Cora
Citeseer
Puhmed
Corn-CA
DBLP--CA
7on
20Newsgroups
SeIGNN
67.93 a 1.27
63,53 a 1.32
84.33 a 0.36
72:21 + 1.51
89.51 = 0.18
65.06
12.82
7 0.35
Pretrain
52.59 + 2.33
53.29 + 2.01
0.41
48.00 + 4.79
87.59 + 0.43
66.82 = 13.48
71.93 2
Pretrain_F
68.39 + 1.20
63.83
1.68
84.47 0.40
73.12
1.37
69.75
65.43
13.38
79.44 0.39
MTL.
72.58 + 1.10
66.40 + 1.35
85.16 + 0.38
75.82 + 1.42
90.22 + 0.23
66.35
13.26
79.56 + 0.42
Mushroom
NTU2012
ModelNe140
Yelp
House (0.6)
House (1.0)
Walmart (0.6)
Walmart (1.0)
73,86 9
+
68.54
58.34
74.97 + 0.22
59.13
93.77 + 2.20
70.06 + 2.42
96,23 0.31
26.68 + 0.30
61.22 + 3.09
54.81 + 2.39
40.35 + 4.30
33.30 2.72
99.77 = 0.15
24.03 + 1.86
28.19 0.34
1.42
09.02 4.00
$9.20 + 2.54
75,01 + 0.27
59.87 0.28
99.80 a 0.17
74.88 a 1.66 96.56 1 0.34
31.39 1 2.45
69.73 2.60
58.90 1.97
75.50 0.18
60.19 0.20
Table 6: Results on converted conventional graphs: Mean accuracy (%) + standard deviation. Bold values indicate the best result. 10% of all vertices are used for training.
Citeseer
Puhmed
Cora CA
DBLP-CA
Zoo
20Newsgroups
HGNN A2
67.37 1.45
62.76 1.42 82.16 + 0.38 66.80 1.79
85.28 0.29
47.84 6.87
70.27 0.73
67.18 1.42
63.52 + 2.35
82.37 0.34
67.14 1.79
85.22
0.26
46.85 10.15
70,46 1.21
67.24
2.56
0.43
85.16 a 0.25
46.85 9,93
1.24
Mushroom
NTU2012
ModelNe140
Yelp
House (0.6)
House (1.0)
Walmart (0.6)
Walmart (1.0)
97.15 a 0.47
70.26 + 1,70
67.60
26.91 + 0.37
58.01 + 2:47
$7.65 a 2.69
59.48 a 0.19
$3.97 = 0.29
97.29 + 0.45
69.91
0.33
26.72
58.08
57.53
59.49 a 0.22
54.04
97.15 a 0.55
69.94 a 1.54
87.65 0.366
26.66 a 0.45
58.47 = 2.99
57.73 2.84
59.53 0.22
53.98 0.26
Analysis of generative augmentation. Here we analyze our proposed generative hypergraph augmentation (A6). We select ModelNet40 and Yelp as the representatives of high-homophily and low-homophily data sets, respectively.
First, we examine the training dynamics of keep ratio and find that they are highly related to the data set homophily (Figure (a)). For ModelNet40, the generator keeps only 20% relations in the early training stage. This is because the homophily of the data set is very high (0.92/0. 88) and delcting a lot of edges relatively randomly at the beginning would not have a large impact on the model but rather can help model learn structural information. Then at a later stage, the generator begins to keep more than 80% relations, which means that the model has learned higher-order information and only removes the unnecessary relations. For Yelp, similar conclusion holds. Specifically, as its homophily is pretty low (0.57/0.26), the generator keeps most of the relations at the early stage for training and then just keeps very low ratio of related relations at the later training stage.
Next, we investigate what our generator has learned. We visualize the hyperedges in the Yelp data set in Figure Yelp is restaurant-rating data set and the restaurants visited by the same user are connected by a hyperedge. We find that some vertices with different labels are removed, which could remove extraneous information and improve the hypergraph homophily. This indicates that our generator does grasp the higher-order information in the hypergraph.
Adversarial robustness. Besides generalizability, we here show hypergraph contrastive learning also boosts robustness. Since there is no existing work developing adversarial attack algorithms designated for hypergraphs, we adapt two state-of-the-art attackers from the graph domain. We presume the graph attackers are applicable hypergraphs to certain extent, both of which data structures with sharing properties. The experiments are performed on five real-world data sets: Cora, Citeseer, ModelNet40,, NTU2012 and House. We regard each hypergraph as a bipartite graph and leverage those algorithms to conduct attacks to the vertices and hyperedges. The methods include an untargeted attack method, minmax attack [46] which poisons the graph structures by adding and removing relations to reduce the overall performance; and a targeted attack method, nettack [47] which leads the HyperGNN to mis-classify target vertices. Beyond these, we also include a random hypergraph perturbation baseline which will randomly drop numerous relations in the hypergraph. These three methods are denoted as Net, Minmax and Random. Following previous works, for each attack, we perturb 10% of vertices/relations. The results in T show that Random and Minmax attacks can decrease the performance of the original model a little on all the data sets, while net attack can decrease the performance most data sets and surprisingly increase the performance on
-ucluucidd
8

---

1.0
SEEACH
0.8
CIT
5
Keep Ratio
0.6
ModelNet40
Yelp
Generative Angmanntiion
0.4
0.0
100
200
300
400
500
Train Epochs
Figure 4: (a) Training dynamics of the relation keep ratio. (b) Illustration of our proposed generative augmentation on the Yelp data Each icon represents a restaurant in the data set, and the number near the icon is the label of this restaurant. The ellipse denotes the hyperedge.
Table Results on the test data sets with regard to robustness. Bold values indicate the best result. 10% of all vertexes are used for training.
Citeseer
ModelNes40
Random
Net
Minmax
Random
Net
Minmax
Random
Net
Minmax
SeGGNN
66.87 + 1.33
66.26
1.34
66.58 + 1.02
62,89 + 1.57 02.81 1.32
62.21 = 1.64 95.74 0.22
95.41 0.28
93.33 0.26
166
71.16
70.86
93.35 # 0.26
A4
72.11
+ 1.60
70.49
+ 1.29
70.52
1.33
65.94
1.24
65.15 1.70
64.12 1.19
95.79 = 0.27
95.44 0.25
0.25
A6
72.15 1.70
71.94 1.48
71.98
1.36
66.60 + 1.61
65.68 = 1.09
65.51 = 1.13
96.58 = 0.24
96.23 = 0.23
94.82 = 0.33
NTU2012
House
House (1.0)
Random
Net
Minmax
Random
Net
Minmax
Random
Net
Minmax
SaGGN
73.84 1 2.18
73.38 1 1.36
70.71 + 1.89
6
68.88 a 2.68
64.78 = 2.20
56.86 = 1.93
59.95 = 1.22
56.52 2 2.52
A2
74.50 2.03
73.86
1.84
71.40
1.64
67.71 + 2.94
69.59 2.32
65.23 + 2.89
57.74 2.70
60.73 + 2.30
57.00 1,94
A4
73.73 1.59
73.72
1.59
71.06
+ 1.53
67.55
68.85
1.38
64.97
3.35
2.72
60.10
1.74
56.65 + 2.26
1.97
1.99
72.09
6.288
2.71
68.84
2,71
the House data set. This performance gain indicates HyperGNNN more robust to structure attack compared with GNN as it leverages higher-order information. Based on these, HyperGCII with generalized hyperedge augmentation (A2) performs better than feature perturbation (A4), and our proposed generative augmentation (A6) can surpass these fabricated baselines on all the data sets and thus is the best to defend attacks. We believe this')l be beneficial complement to our main experiments and we hope for more works on hypergraph attacks.
Fairness. Furthermore, we claim that hypergraph contrastive self-supervision also benefits fairness. There was no related data set before. So we introduce three newly curated hypergraph data sets: German 48), Recidivism 49 and Credit [50]. The hypergraph construction follows the setting [ The top 5 similar objects in each data set are built hyperedge. For the accuracy metrics, we use F1-score and AUROC value for the binary classification task. For measuring fairness, we adopt the statistical parity Asr and equalized odds Ao. Please refer to Appendix C for detailed information about the data sets and metrics. The experimental results Table show that our generative method still achieves better comparable performances while imposing more fairness.
5 Conclusion
In the paper, we study the problem of how to construct contrastive views of hypergraphs via augmen- tations. We provide the solutions by first studying domain knowledge-guided fabrication schemes. Then, in search of more effective views in a data-driven manner, we are the first to propose hypergraph generative models to generate augmented views, well an end-to-end differentiable pipeline to jointly perform hypergraph augmentation and contrastive learning. We find that generative augmenta- tions perform better at preserving higher-order information to further benefit generalizability. The proposed framework also boosts robustness and faimesss of hypergraph representation learning. In the future, we plan design more powerful hypergraph generator and HyperGNN while addressing more real-world hypergraph data challenges and more hypergraph learning models.

---

Table 8: Results on the test data sets with regard to fairness. 10% of all vertexes are used for training. For fairness metries Asp and Ao lower values indicate better performance.
data set
Method
AUROC
F1
Asr(U)
Aro(U)
SeIGNN
59.16 2.51
819
4,06 2.65 6.76
German Credit
A2
59,81 3,00
82.26
0.55 0.95
0.78 0.70
A4
59.66
+ 3,83
80.54
4
5.07 = 7.81
59.88
8.3
0.9
0.47
SeIGNN
96.51
+ 0,48
89984
0.50
4.16 0.51
Recidivism
A2
96.34
0 0.39
90.09 0.53
8.53
3,92 = 0.68
A6
96,55 + 0.54
0.55
8.51
3.13 0.25
0.64
SetGINN
13.4620.11
8.27
2.79
0,98 0.69
Credit defanlter
73.43 0.27 87.82
2.64 1.32
0.93
A4
73.58
87.92
2.84
1.14
0.32
A6 73.78 0.16 88.03 0.14
2.58
a
0.91
0.81 0.37
Acknowledgments and Disclosure of Funding
This work is supported by National Science Foundation under Award No. IIS-1947203, IIS-2117902, IIS-2137468, 1CF-0993008; US Army Research Office Young Investigator Award N111NNFF1122440 National Institute of General Medical Seiences under grant R35GMI Food Research Initiative grant no. 2020-67021-32799/ba National Institute of Food and Agriculture. The views and conclusions are those of the authors and should not be interpreted as representing the official policies of the government agencies.
and Agriculture and
accession no. no. 10241778 from the USDA
References
[1] Yifan Feng. Haoxuan You, Zizhao Zhang, Rongrong Ji, and Yue Gao. Hypergraph neural networks. Proceedings of the AAAI Conference on Artificial Intelligence, volume 33, pages 3565,56655222 2019.
[2] Naganand Yadati, Madhav Nimishakavi, Prateek Yadav, Vikram Nitin, Anand Louís, and Partha Talukdar. Hypergen: A new method for training graph convolutional networks on hypergraphs. Advances neural information processing systems, 32, 2019.
[3] Eli Chien, Chao Pan, Jianhao Peng, and Olgica Milenkovic. You are allset: A multiset function framework for hypergraph neural networks. ICLR, 2022.
[4] Alain Bretto. Hypergraph theory. An introduction. Mathematical Engineering. Cham: Springer, 2013.
[5] Shuyi Ji, Yifan Feng, Rongrong Ji, Xibin Zhao, Wanwan Tang, and Yue Gao. Dual channel hypergraph collaborative filtering- Proceedings of the 26th ACM SIGKDD International Conference Knowledge Discovery & Data Mining, pages
2020.
[6] Junliang Yu, Hongzhi Yin, Jundong Li, Qinyong Wang, Nguyen Quoc Viet Hung, and Xi- angliang Zhang. Self-supervised multi-channel hypergraph convolutional network for social recommendation. In Proceedings of the Web Conference 2021, pages 413-424, 2021.
[7] Ramit Sawhney, Shivam Agarwal, Arnav Wadhwa, and Rajiv Ratn Shah. Spatiotemporal hypergraph convolution network for stock movement forecasting, In 2020 IEEE International Conference Data Mining (ICDM), pages 482-491. IEEE, 2020.
[8] Yuning You and Yang Shen. Cross-modality and self-supervised protein embedding for compound-protein affinity and contact prediction. bioRxiv, 2022.
[9] Ruochi Zhang, Tianming Zhou, and Jian Ma. Multiscale and integrative single-cell hi-c analysis with higashi. Nature biotechnology, 40(2):254-261) 2022.
[10] Meng Liu, Youzhi Luo, Kanji Uchino, Koji Maruhashi, and Shuiwang Generating 3d molecules for target protein binding, arXiv preprint arXiv:2204.09411 2022.
10

---

[11] Priya Goyal, Dhruv Mahajan, Abhinav Gupta, and Ishan Misra. Scaling and benchmarking self-supervised visual representation learning. In Proceedings of the ieee/evf International Conference computer vision, pages 6391 6401, 2019.
[12] Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. A simple framework for contrastive learning of visual representations. In International conference on machine learning, pages 1597-1607. PMLR, 2020.
[13] Weihua Hu, Bowen Liu, Joseph Gomes, Marinka Zimik, Perey Liang, Vijay Pande, and Jure Leskovec. Strategies for pre-training graph neural networks, arXiv preprint arXiv: 12265, 2019.
[14] Yuning You, Tianlong Chen, Yongduo Sui, Ting Chen, Zhangyang Wang. and Yang Shen. Graph contrastive learning with augmentations. Advances in Neural Information Processing Systems, 33:5812 5823, 2020.
[15] Yuning You, Tianlong Chen, Zhangyang Wang, and Yang Shen. When does self-supervision help graph convolutional networks? arXiv preprint arXiv:2006.0911 09136, 2020.
[16) Wei Jin, Tyler Derr, Haochen Liu, Yiqi Wang, Suhang Wang, Zitao Liu, and Jiliang Tang. Self-supervised learning on graphs: Deep insights and new direction. arXiv preprint arXiv:2006. 10141, 2020.
[17] Yaochen Xie, Zhao Xu, Jingtun Zhang, Zhengyang Wang, and Shuiwang Ji. sulf-super learning of graph neural networks: A unified review. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022.
[18] Petar Velickovic, William Fedus, William Hamilton, Pietro Liò, Yoshua Bengio, and Devon Hjelm. Deep graph infomax. ICLR (Poster), 2(3):4, 2019.
[19] Yuning You, Tianlong Chen, Yang Shen, and Zhangyang Wang. Graph contrastive learning automated. In International Conference on Machine Learning, pages 12121-12132. PMLR, 2021.
[20] Yuning You, Tianlong Chen, Zhangyang Wang, and Yang Shen. Bringing your own view: Graph contrastive learning without prefabricated data augmentations, arXiv preprint arX1v:2001001702. 2022.
[21] Yanqiao Zhu, Yichen Xu, Feng Yu, Qiang Liu, Shu Wu, and Liang Wang. Deep graph contrastive representation learning, arXiv preprint arXiv:2006. 04131, 2020.
[22] Puja Trivedi, Ekdeep Singh Lubana, Yujun Yan, Yaoqing Yang, and Danai Koutra. Augmenta- tions in graph contrastive learning: Current methodological flaws & towards better practices. In Proceedings of the ACM Web Conference 2022, pages
2022.
[23] Haonan Wang, Jieyu Zhang, Qi Zhu, and Wei Huang. Augmentation-ffes graph contrastive learning. arXiv preprint arXiv:2204.04877 2022.
[24] Shengyu Feng. Baoyu Jing, Yada Zhu, and Hanghang Tong. Adversarial graph contrastive learning with information regularization, In Proceedings of the ACM Web Conference 2022, pages 1372-1371, 2022.
[25] Xinyi Xu, Cheng Deng, Yaochen Xie, and Shuiwang Ji. Group contrastive self-supervised learning on graphs. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022.
[26] Lianghao Xia, Chao Huang, Yong Xu, Jiashu Zhao Dawei Yin, and Jimmy Xiangji Huang. Hypergraph contrastive collaborative filtering arXiv preprint arXiv:2204.1220 12200. 2022.
[27] Derun Cai, Chenxi Sun, Moxian Song. Baofeng Zhang. Shenda Hong, and Hongyan Li. Hy- pergraph contrastive learning for electronic health records. In Proceedings of the 2022 SIAM International Conference on Data Mining (SDM), pages 127-135. SIAM, 2022.
(28) Eric Jang, Shixiang Gu, and Ben Poole. Categorical reparameterization with gumbel-softmax, arXiv preprint 1rXiv:1611.01144 2016.

---

[29] Ruochi Zhang, Yuesong Zou, and Jian Ma. Hyper-sagnn: a self-attention based graph neural network for hypergraphs. arXiv preprint arXiv:1911.02615. 2019.
[30] Song Bai, Feihu Zhang, and Philip HS Torr. Hypergraph convolution and hypergraph attention. Pattern Recognition, 110: 107637, 2021.
[31] Jing Huang and Jie Yang. Unignn: unified framework for graph and hypergraph neural networks. arXiv preprint arXiv:2105.00956, 2021.
[32] Devanshu Arya, Deepak K Gupta, Stevan Rudinac, and Marcel Worring. Hypersage: Gener- alizing inductive representation learning on hypergraphs. arXiv preprint 10..45000 2020.
[33] Manzil Zaheer, Satwik Kottur, Siamak Ravanbakhsh, Barnabas Poczos, Russ R Salakhutdinov, and Alexander J Smola. Deep sets. Advances in neural information processing systems, 30, 2017.
[34] Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross Girshick. Momentum contrast for unsupervised visual representation learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 9729-9733 2020.
[35] Jean-Bastien Grill, Florian Strub, Florent Altché, Corentin Tallec, Pierre Richemond, Elena Buchatskaya, Carl Doersch, Bernardo Avila Pires, Zhaohan Guo, Mohammad Gheshlaghi Azar, et al. Bootstrap your own latent-a new approach to self- learning. Advances in Neural Information Processing Systems, 33:21271-22 -21244, 2020.
[36] Xin Xia, Hongzhi Yin, Junliang Yu, Qinyong Wang, Lizhen Cui, and Xiangliang Zhang. Sele-super hypergraph convolutional networks for session-based recommendation. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pages 4501-4511,
2021.
[37] Boxin Du, Changhe Yuan, Robert Barton, Tal Neiman, and Hanghang Tong. Hypergraph pre-training with graph neural networks. arXiv preprint arXiv:2105. 10862, 2021.
[38] Xiangnan He, Lizi Liao, Hanwang Zhang, Liqiang Nie, Xia Hu, and Tat-Seng Chua. Neural collaborative filtering. In Proceedings of the 26th international conference on world wide web, pages 173-182, 2017.
[39] Tianxin Wei, Fuli Feng, Jiawei Chen, Ziwei Wu, Jinfeng Yi, and Xiangnan He. Model-agnosse counterfactual reasoning for eliminating popularity bias in recommender system. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining, pages 1791 1800, 2021.
[40] Tianxin Wei and Jingrui He. Comprehensive fair meta-learned recommender system. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 1989-1999, 2022.
[41] Yang Zhang, Fuli Feng, Xiangnan He, Tianxin Wei, Chonggang Song, Guohui Ling, and Yongdong Zhang. Causal intervention for leveraging popularity bias in recommendation. In Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 11 11-20, 2021.
[42] Diederik P Kingma and Max Welling. Auto-encoding variational bayes. arXiv preprint 131.v61312.6114 14, 2013.
[43] Thomas N Kipf and Max Welling. Variational graph auto-encoders. arXiv preprint 1rX.v.1611.07308 2016.
[44] Diederik Kingma, Max Welling, et al. An introduction to variational autoencoders. Founda- tions and Trends@ in Machine Learning, 12(4):307-392, 2019.
[45] Yuning You, Yue Cao, Tianlong Chen, Zhangyang Wang, and Yang Shen. Bayesian modeling and uncertainty quantification for learning to optimize: What, why, and how, In International Conference Learning Representations, 2021.
12

---

[46] Daniel Zügner and Stephan Günnemann. Adversarial attacks on graph neural networks via meta learning. arXiv preprint arXiv:1902.08412, 2019.
[47] Daniel Zügner, Amir Akbarnejad, and Stephan Günnemann. Adversarial attacks on neural networks for graph data. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pages 2847-2856, 2018.
[48] Arthur Asuncion and David Newman. Uci machine learning repository. 2007.
[49] Kareem L Jordan and Tina L Freiburger. The effect of race/ethnicity sentencing: Examining sentence type, jail length, and prison length. Journal of Ethnicity Criminal Justice, 13(3):199- 196, 2015.
[50] I-Cheng Yeh and Che-hui Lien. The comparisons of data mining techniques for the predictive accuracy of probability of default of credit card clients. Expert systems with applications, 36(2):2473-2480, 2009.
Checklist
The checklist follows the references. Please read the checklist guidelines carefully for information on how to answer these questions. For each question, change the default [TODO] [Yes]. [No] or [N/A] You are strongly encouraged to include justification to your answer, either by referencing the appropriate section of your paper or providing a brief inline description. For example:
Did you include the license to the code and datasets? [Yes] See Section.
Did you include the license the code and datasets? [No] The code and the data are proprictary.
Did you include the license to the code and datasets? [N/A]
Please do not modify the questions and only use the provided macros for your answers. Note that the Checklist section does not count towards the page limit. In your paper, please delete this instructions block and only keep the Checklist section heading above along with the questions/answers below.
For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope? [Yes]
(b) Did you describe the limitations of your work? [Yes] Please check Apendix B.
(c) Did you discuss any potential negative societal impacts of your work? [N/A] We are aware of any potential ethical issues regarding our work. More discussions can be found in Appendix B.
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes]
2. If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results? [N/A]
(b) Did you include complete proofs of all theoretical results? [N/A]
3. If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main experi- mental results (either in the supplemental material or as a URL.)? [Yes] We add them in the supplementary materials.
(b) Did you specify all the training details (e.g., g.. data splits. hyperparameters, how they were chosen)? [Yes] We write the implementation details in Appendix C.
(c) Did you report error bars (e.g., with respect to the random seed after running experi- ments multiple times)? [Yes] We report the standard deviation of 20 runs with different splits.
(d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)? [Yes] We specify the computing resouce in Appendix C.
13

---

4. If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators? [Yes]
(b) Did you mention the license of the assets? [Yes] We discuss it in Appendix C.
(c) Did you include any new assets either in the supplemental material or as URL? [Yes]
We submit the code in the supplementary materials.
(d) Did you discuss whether and how consent was obtained from people whose data you're using/curating? [Yes] We discuss it in Appendix C.
(e) Did you discuss whether the data you are using/curating contains personally identifi- able information offensive content? [Yes] The data sets we use don't 't contain any personally identifiable information.
5. If you used crowdsourcing conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, if applicable? [N/A] We don't conduct research with human subjects.
(b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, applicable? [N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation? [N/A]
14


Reviews:
[{"review_id": "tHZfJWLAkc", "reviewer": "Reviewer_jPDp", "summary": "This paper aims to improve the generalizability of hypergraph neural networks using self-supervised contrastive learning in a few-shot learning scenario. The paper's main contributions can be summarized as follows: first, using domain knowledge, they fabricated augmentations, and second, for the first time, they proposed hypergraph generative models to generate augmented views using contrastive learning and novel variational hypergraph auto-encoder architecture which is an adaptation of VAE.  ", "questions": "\n1. Are these augmentations extensive enough or is there any correlation between them?\n\n2. Can you provide information about computational costs?\n\n3. Are generated augmentations, combinations of the fabricated augmentations? Can you provide further analysis?\n\n4. Eq5: I believe you need a “max” rather than a “min”, as ELBO is to be maximized.\n\n5. Please explain briefly L_cl as the contrastive learning loss after the corresponding equation in the main text, and say further details are in the Appendix.\n", "limitations": "The authors stated two limitations of the study: the additional computational cost due to hypergraph generative model, and how to design powerful hypergraph generators, in the Appendix B4.\nIn Appendix B5, no negative societal impact is reported. When the application domain includes Recommender systems, I believe society is always at risk under harmful/unfair content spread/recommendation due to current ML/DL methods. This should be acknowledged.", "rating": 6, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n* Paper is well written, flows smoothly, and easy to understand.\n* Even though the idea is a combination of well-known techniques, overall it is quite original.\n* The proposed method is tested with a high number of datasets and experiments even for robustness and fairness. I believe the experimental sections of the paper are quite strong but could be improved (see weaknesses).\n\nWeaknesses:\n* Proposed augmentations from A0 to A5 were not investigated in detail. Are these augmentations independent or is there any correlation between them? The selection of augmentations and coverage of the augmentations should be further investigated. Furthermore, there are not any combinations such as A1 and A2 together. This lack of various combinations of augmentations may be the reason for the proposed method to be more successful, this is not clear.\n* Computational cost analysis of the proposed model is missing.\n* Generated augmentations are not investigated enough. Are these augmentations, combinations of the fabricated augmentations?", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "pz0BPCnKpbN", "reviewer": "Reviewer_9Xc8", "summary": "The paper proposes augmentations for learning representation on HyperGraphs using contrastive learning. They introduce two classes of augmentations: fabricated and generative augmentations and show that using these augmentations it is possible to learn good representations over the input hypergraphs. The paper is well-written and easy-to-follow.  ", "questions": "1- How does the proposed augmentations interact with other types of contrastive loss?\n2- Is it possible to define the augmentations over the hypergraph space rather than converting it to a a bipartite graph? ", "limitations": "NA", "rating": 5, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n- The paper tackles a problem (representation learning on HyperGraphs) that can be used in various applications. \n- The paper is well-written and easy-to-follow.\n- The experimental setup is fair and the model shows good performance compared to baselines.\n\nWeaknesses:\n- The paper simply adopts previous work on contrastive graph representation learning [1,2,3] and applies it to hyper-graphs. There's not much insights on hypergraphs but rather authors map hypergraphs into bipartite graphs and apply recent graph augmentations to learn representations. This is not a bad thing per se, but again it doesn't give any new insights about hypergraphs.\n- Experimental studies need to be expanded to show for example how the augmentation mechanism works with other losses beside InfoNCE.\n\n[1] https://arxiv.org/abs/2201.09830\n[2] https://arxiv.org/abs/2006.06830\n[3] https://arxiv.org/abs/2106.07594\n \n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ["I don’t know"]}, "extra_text": {}}, {"review_id": "oN2k4DePaH5", "reviewer": "Reviewer_RMPZ", "summary": "In the paper, the author studies the problem of how to construct contrastive views of hypergraphs via augmentations. Its contributions include: (1) Fabricating two schemes to augment hyperedges with higher-order relations encoded, and adopt three vertex augmentation strategies from graph-structured data. (2) Proposing hypergraph generative models to generate augmented views, and then an end-to-end differentiable pipeline to jointly perform hypergraph augmentation and contrastive learning.", "questions": "About adversarial robustness in experiments: the authors mentioned that \" Meanwhile current attack algorithms are far from effective because they are not specifically designed for hypergraphs .\" So are the experimental results convincing? About Table7: Why does not the table contain experimental results of A1, A3 and A5?", "limitations": "In the experiments, one have to select the data augmentation strengths manually. The fairness of experiments needs more datasets to support.", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\nThe quality and clarity of the paper are good. The motivation is clear and the author does enough experiments. The paper is sound in improving the generalizability of hypergraph neural networks in the low-label regime.\nWeaknesses:\nThe innovation is of HyperGraphCL is limited. The augmentations like perturbing hyperedges and subgraph, are just the same as that in graph contrastive learning [1][2][3]. Why does the generative model that the author proposes work well? I think the author needs to explain more clearly in theory. The description and analysis in the experiment section are ambiguous, and the writing in this part is hard to follow. The compared baselines are not advanced enough, and I recommend the author conduct more SOTA methods for comparisons, e.g., SOTA graph contrastive methods.\n\n[1] You, Y., Chen, T., Sui, Y., Chen, T., Wang, Z., & Shen, Y. (2020). Graph contrastive learning with augmentations. Advances in Neural Information Processing Systems, 33, 5812-5823.\n[2] Suresh, S., Li, P., Hao, C., & Neville, J. (2021). Adversarial graph augmentation to improve graph contrastive learning. Advances in Neural Information Processing Systems, 34, 15920-15933.\n[3] Xia, J., Wu, L., Chen, J., Hu, B., & Li, S. Z. (2022, April). SimGRACE: A Simple Framework for Graph Contrastive Learning without Data Augmentation. In Proceedings of the ACM Web Conference 2022 (pp. 1070-1079).", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "m_lwMHcMfWJ", "reviewer": "Reviewer_7pvs", "summary": "This paper investigates the view augmentations in hypergraph contrastive learning. In particular, it explores several domain knowledge-guided fabrication augmentation methods and proposed hypergraph generative models. The authors' experiments show that generative augmentations perform well at preserving higher-order information and thus presents better performance.", "questions": "1. if \"current attack algorithms are far from effective because they are not specifically designed for hypergraphs\", then why you claim the designed hyperGCL can provide robustness?\n2. Why statistical parity ΔSP and equalized odds ΔEO can represent fairness in hypergraph learning?\n3. Are the Hypergraph Generative Models hard to train (converge)?", "limitations": "Yes", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n1.The paper is easy to follow.\n2. The proposed hypergraph generative modeling is a good strategy for augmenting a view and the results seem quite promising.\n\nWeaknesses:\n1. The reason or analysis why the proposed hypergraph generative modeling works better than other view augmenting methods are not clear. \n2. I think the hypergraph generative modeling could potentially increase training difficulty compared to other methods.\n3. If with different augmentations, the contrast manner should be accordingly tuned?\n\nOthers:\nI'm not quite sure if the topic (hypergraph contrastive learning) is important or not (I have very bad experience with hypergraph learning, as they cannot work well in real applications).", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ["I don’t know"]}, "extra_text": {}}, {"review_id": "DVyow2KGQl0", "reviewer": "Reviewer_V7YT", "summary": "The authors present a novel approach to perform contrastive learning by investigating several fabricated view augmentations of the hypergraph and a variational generative model. Their exhausting experiments demonstrated that the proposed methodology results in the SOTA for hypergraph, especially in low data regimens.", "questions": "1. Section 3.2, when describing the different fabricated augmentations. Although it is clear that augmentations A3-A5 come from previous work, it would be nice to have a figure where A3-A5 are depicted \n2. Line 191-192. ‘We train one VHGAE to produce one generative view, with the other view G ̃p is kept as prefabricated.’ Do they mean the other view? Otherwise, do you generate one view from another view? Please explain\n3. Table 2 is never referred to in the text.\n4. Since Self and Con are the methods, which are compared, it would be a good idea to include some information about these approaches in the related work.", "limitations": "Authors have not adequately commented on the known limitations.", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n\n1. Well written, good flow of ideas.\n2. Extensive evaluation:\n    1. 13 datasets with different data regimens 10%, 1%. \n    2. Different self-supervision mechanisms\n    3. Comparison with approaches to convert the hypergraph to graph\n3. Novel idea to construct augmented views of hypergraphs for contrastive learning, especially the generative model\n\nWeaknesses:\n\n1. One of the arguments all around the paper is that the proposed method works for a low data regime. However, the authors performed experiments with 10% and 1%. What do the results look like when using ˜100% data for training? In other words, could you show the upper bound?\n\n2. The results from the robustness part of the experiments are inconclusive. The authors explain that in some cases, the performance even increased. Given that these perturbations on the hypergraphs were adopted for attacks on normal graphs. The authors could either provide some insights into their results or perform the robustness study using suitable hypergraph perturbations\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "7SkKwmkjHf", "reviewer": "Reviewer_1bGo", "summary": "The submission introduces a generative modelling approach to obtain learnable hypergraph augmentations. These augmentations are used to generate views for contrastive learning. Empirically, the benefit of the learned augmentations over several fabricated types of augmentations, as well as over two baselines, is shown. Additional experiments investigate adversarial robustness and fairness aspects.", "questions": "- Did you experiment with the augmentations in different settings, e.g., hypergraph classification, link prediction, ...\n\n- How does the method perform in not low-label regimes (e.g. 50% of vertexes as training set? Does it become infeasible due to the high computational cost of VHGAE? Is the benefit over the baselines still as pronounced?\n\n- Could you clarify the difficulty in constructing hypergraph augmentations.\n  >(l.33) However, it is non-trivial to build hypergraph views due to their overly complicated topology, i.e., there are $\\sum_e \\binom{N}{e}$ possibilities for one hyperedge on $N$ vertices, versus $\\binom{N}{2}$ for one edge in graphs.  \n\n  At first sight, this property seems to facilitate the construction of hypergraph augmentations.\n  What is the \"combinatorial challenge\" (l.119?) Does this refer to checking the set of hyperedges for duplicates, i.e. whether two finite sets of vertices are equal / correspond to permuted tuples?\n\n- Could you expand on Eq. (2)? How is the bipartite graph determined by the sample $z_{\\mathcal V}$ and $z_{\\mathcal E}$. What do $z_e$ and $z_v$ correspond to, how are they connected to $z_\\mathcal V$ and $z_\\mathcal E$? Similarly, $w_{\\mathcal {VE}}$ and $w_{ve}$. What is meant with the \"topological\" distribution of the bipartite graph?\n\n- Regarding equation (4) and the corresponding text. What is meant with $T(G) \\circ G$, i.e., how is the \"vector applied to perform augmentation\". Is $T(G)$ not only a vector, but also a function?\n\nI-  do not understand Figure 3. What do the green arrows indicate?\n\n- In experiments (e.g. Table 3) what is the difference between SetGNN (the backbone) and A0 (identity augmentations).", "limitations": "The (presumably) high computational cost of the method is not adequately addressed, i.e., it is not quantified. A comparison of runtimes between generated augmentations, prefabricated augmentations and no augmentations would provide clarity.", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "### Strengths\n\n+ Compared to other types of data, (e.g. image data), sensible hypergraph augmentations are not obvious and difficult to design.\nAs far as I am aware, this is the first work to introduce learnable hypergraph augmentations.\n\n+ While the submission focuses on utilizing such augmentations in a contrastive learning setting, the method to generate the augmentations is not limited to it. Application in other settings seem possible.\n\n+ The method consistently outperforms the baselines on the studied vertex classification tasks. The learned augmentations outperform the fabricated ones.\n\n+ The method is not only checked for classification accuracy, also adversarial robustness and fairness aspects are evaluated.\n\n### Weaknesses\n\n- The method is computational expensive. In consequence, only one augmentation in each pair of views is generated (the other is fabricated, cf. l.192)\nThis might be the reason why the empirical evaluation focuses on the low-label regime, where fewer augmentations are required.\n\n- Some parts of the paper are difficult to understand, particularly the description of the augmentation procedure. See questions.\n\n- The adversarial attack experiments are not very expressive, considering that for some attacks model accuracy even increases. Maybe the adaptions of Random Noise and Minmax Attack could even serve as hypergraph augmentations.\n\n\n**Post rebuttal edit**\n\nWhile my questions have been adequately addressed, several reviewers have pointed at limited novelty in comparison with existing works from the graph augmentation literature. I will thus keep my initial rating of **6: Weak accept**.\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}]

Author rebuttal:

Decision: Accept