Original: Augmentations in Hypergraph Contrastive Learning: Fabricated and Generative
Revised: Augmentations in Hypergraph Contrastive Learning: Fabricated and Generative

Original: Anonymous Author(s)

Original: AffiliationAddress email

Original: Abstract
Revised: Abstract

Original: This paper targets at improving the generalizability of hypergraph neural networksin the low-label regime, through applying the contrastive learning approach fromimages/graphs.
Revised: This paper targets at improving the generalizability of hypergraph neural networks in the low-label regime, through applying the contrastive learning approach from images/graphs (we refer to it as HyperGCL ).

Original: The question we focus on here is: How to construct contrastiveviews of hypergraphs via augmentations?
Revised: We focus on the following question: How to construct contrastive views for hypergraphs via augmentations?

Original: We provide the solutions in two folds.
Revised: We provide the solutions in two folds.

Original: First, guided by domain knowledge, we fabricate two schemes to augment hyperedges with higher-order relations encoded, and adopt three vertex augmentationstrategies from graph-structured data.
Revised: First, guided by domain knowledge, we fabricate two schemes to augment hyperedges with higher-order relations encoded, and adopt three vertex augmentation strategies from graph-structured data.

Original: Second, in search of more effective viewsin a data-driven manner, we are the first to propose  hypergraph generative modelsto generate augmented views, and then an end-to-end differentiable pipeline tojointly perform hypergraph augmentation and contrastive learning.
Revised: Second, in search of more effective views in a data-driven manner, we for the first time propose a hypergraph generative model to generate augmented views, and then an end-to-end differentiable pipeline to jointly learn hypergraph augmentations and model parameters.

Original: Our technicalinnovations are reflected in designing both fabricated and generative augmentations of hypergraphs.
Revised: Our technical innovations are reflected in designing both fabricated and generative augmentations of hypergraphs.

Original: The experimental findings include: (i) Among fabricatedaugmentations, augmenting hyperedges performs thebest in most cases, implyingthat higher-order information in structures is usually more downstream-relevant;(ii) Generative augmentations do better in preserving higher-order informationto further benefit generalizability; (iii)The proposed framework HyperGCL alsoboosts robustness and fairness of hypergraph representation learning.
Revised: The experimental findings include: (i) Among fabricated augmentations in HyperGCL, augmenting hyperedges provides the most numerical gains, implying that higher-order information in structures is usually more downstream-relevant; (ii) Generative augmentations do better in preserving higher-order information to further benefit generalizability; (iii) HyperGCL also boosts robustness and fairness in hypergraph representation learning.

Original: Codes willbe made public upon acceptance.
Revised: Codes are released at https://github.com/weitianxin/HyperGCL .

Original:  Introduction
Revised:  Introduction

Original: Hypergraphs, generalization of graphs, have raised a surge of interest in the research community[1, 2, 3] due to their innate capability of capturing higher-order interactions [4].
Revised: Hypergraphs have raised a surge of interests in the research community [1, 2, 3] due to their innate capability of capturing higher-order relations [4].

Original: They offer a powerfultool to model sophisticated structures in broad applications, e g., recommender systems [5], financialanalyses [6] and bioinformatics [7].
Revised: They offer a powerful tool to model complicated topological structures in broad applications, e g., recommender systems [5, 6], financial analyses [7, 8], and bioinformatics [9, 8, 10].

Original: Concomitant with the trend, hypergraph neural networks(HyperGNNs) have recently been developed [1, 2, 3] for hypergraph representation learning.
Revised: Concomitant with the trend, hypergraph neural networks (HyperGNNs) have recently been developed [1, 2, 3] for hypergraph representation learning.

Original: This paper focuses on the few-shot scenario, i e., task-specific labels are scarce, which are ubiquitousin real-world applications of hypergraphs [5, 6, 7] and empirically weaken the generalizability of HyperGNNs.
Revised: This paper focuses on the few-shot scenarios of hypergraphs, i.e., task-specific labels are scarce, which are ubiquitous in real-world applications of hypergraphs [5, 7, 9] and empirically restrict the generalizability of HyperGNNs.

Original: Inspired by the emerging self-supervised learning on images/graphs [8, 9, 10, 11],especially the contrastive approaches [9, 11, 12, 13, 14, 15, 16],  we set out to leverage contrastiveself-supervision to address the problem.
Revised: Inspired by the emerging self-supervised learning on images/graphs [11, 12, 13, 14, 15, 16], especially the contrastive approaches [12, 14, 17, 18, 19, 20, 21, 22, 23, 24, 25], we set out to leverage contrastive self-supervision to address the problem.

Original: Nevertheless, one challenge stands out: How to construct contrastive views for hypergraphs?
Revised: Nevertheless, one challenge stands out: How to build contrastive views for hypergraphs?

Original: Thesuccess of contrastive learning hinges on the appropriate view construction, otherwise it wouldpromote “negative transfer” [9, 11].
Revised: The success of contrastive learning hinges on the appropriate view construction, otherwise it would result in “negative transfer” [12, 14].

Original: However, it is non-trivial to build hypergraph views due totheir overly complicated topology, i.e., there are (cid:80) Ne =1 (cid:0)
Revised: However, it is non-trivial to build hypergraph views due to their overly intricate topology, i.e., there are (cid:80) Ne =1 (cid:0)

Original: Ne (cid:1) possibilities for one hyperedge on Nvertices, versus (cid:0) N 2 (cid:1) for one edge in graphs.
Revised: Ne (cid:1) possibilities for one hyperedge on N vertices, versus (cid:0) N 2 (cid:1) for one edge in graphs.

Original: To date, the only way of contrasting is between therepresentations of hypergraphs and their clique-expansion graphs [17, 18], which is computationallyexpensive as multiple neural networks of different modalities (hypergraphs and variants of expanded graphs) need to be optimized.
Revised: To date, the only way of contrasting is between the representations of hypergraphs and their clique-expansion graphs [26, 27], which is computationally expensive as multiple neural networks of different modalities (hypergraphs and variants of expanded graphs) need to be optimized.

Original: More importantly,  clique expansion has the risk of losing higher-orderinformation via pulling representations of hypergraphs and graphs close.
Revised: More importantly, contrasting on clique expansion has the risk of losing higher-order information via pulling representations of hypergraphs and graphs close.

Original: Contributions.
Revised: Contributions.

Original: Motivated by [9, 11] that appropriate data augmentations suffice for the effectivecontrastive views, and intuitively they are more capable of preserving higher-order relations inhypergraphs compared to clique expansion,in this paper we dive into the question on how to designaugmented views of hypergraphs for contrastive learning.
Revised: Motivated by [12, 14] that appropriate data augmentations suffice for the effective contrastive views, and intuitively they are more capable of preserving higher-order relations in hypergraphs compared to clique expansion, we explore on the question in this paper, how to design augmented views of hypergraphs in contrastive learning ( HyperGCL ).

Original: We contribute to the answers  in two folds.
Revised: Our answers are in two folds.

Original: We first explore whether fabricated augmentations guided by domain knowledge are suited forhypergraph contrastive learning.
Revised: We first assay whether fabricated augmentations guided by domain knowledge are suited for HyperGCL.

Original: As hyperedges and vertices compose a hypergraph, to augmenthyperedges, we propose two strategies that (i) directly perturb on hyperedges, and (ii) perturb onthe “edges” between hyperedges and vertices in the converted bipartite graph; to augment vertices,we adopt three schemes of vertex dropping, attribute masking and subgraph from graph-structureddata [11].
Revised: Since hypergraphs are composed of hyperedges and vertices, to augment hyperedges, we propose two strategies that (i) directly perturb on hyperedges, and (ii) perturb on the “edges” between hyperedges and vertices in the converted bipartite graph; To augment vertices, we adopt three schemes of vertex dropping, attribute masking and subgraph from graph-structured data [14].

Original: Our finding is, different from the fact that vertex augmentations benefit more on graphs,hypergraphs mostly benefit from hyperedge augmentations , revealing that higher-order informationencoded in hyperedges is usually more downstream-relevant (than information in vertices).
Revised: Our finding is that, different from the fact that vertex augmentations benefit more on graphs, hypergraphs mostly benefit from hyperedge augmentations (up to 9% improvement), revealing that higher-order information encoded in hyperedges is usually more downstream-relevant (than information in vertices).

Original: Furthermore, in search of even better augmented views but in a data-driven manner, we studywhether/how augmentations of hypergraphs could be learned during contrastive learning.
Revised: Furthermore, in search of even better augmented views but in a data-driven manner, we study whether/how augmentations of hypergraphs could be learned during contrastive learning.

Original: To thisend, for the first time, we propose a novel variational hypergraph auto-encoder architecture, as ahypergraph generative model, to parameterize the augmentation space of hypergraphs.
Revised: To this end, for the first time, we propose a novel variational hypergraph auto-encoder architecture, as a hypergraph generative model, to parameterize a certain augmentation space of hypergraphs.

Original: In addition,we propose an end-to-end differentiable pipeline utilizing Gumbel-Softmax [19], to jointly performhypergraph augmentation and contrastive learning.
Revised: In addition, we propose an end-to-end differentiable pipeline utilizing Gumbel-Softmax [28], to jointly learn hypergraph augmentations and model parameters.

Original: Our finding is that generative augmentation canwell capture the higher-order information and achieve state-of-the-art performance on most of thebenchmark data sets.
Revised: Our observation is that generative augmentations can better capture the higher-order information and achieve state-of-the-art performance on most of the benchmark data sets (up to 20% improvement).

Original: Comprehensive experiments on 13 data sets validate the effectiveness of our proposed HyperGCLframework.
Revised: The aforementioned empirical evidences (for generalizability ) are drawn from comprehensive experiments on 13 datasets.

Original: Our proposed generative augmentation shows substantial improvements of up to 20.36%compared with baselines and 10.35% compared with fabricated augmentations.

Original: We also introduce therobustness and fairness evaluation for hypergraphs for the first time. And we find that our proposedmethod additionally boosts robustness against adversarial attacks and imposes fairness with regard tosensitive attributes.
Revised: Moreover, we introduce the robustness and fairness evaluation for hypergraphs, and show that HyperGCL in addition boosts robustness against adversarial attacks and imposes fairness with regard to sensitive attributes.

Original: The rest of the paper is organized as follows.
Revised: The rest of the paper is organized as follows.

Original: We discuss the related work in Section 2 and introducethe proposed HyperGCL in Section 3.
Revised: We discuss the related work in Section 2, introduce

Original: Then we present the experimental results in Section 4. In theend, we concludeour paper in Section 5.
Revised: HyperGCL in Section 3, present the experimental results in Section 4, and conclude in Section 5.

Original:  Related Work
Revised:  Related Work

Original: Hypergraph Learning.
Revised: Hypergraph neural networks.

Original: Hypergraphs, which are able to represent higher-order relationships, haveattracted significant attention in recent years.
Revised: Hypergraphs, which are able to encode higher-order relationships, have attracted significant attentions in recent years.

Original: In the machine learning community, hypergraph neuralnetworks arewell developed for effective hypergraph representations.
Revised: In the machine learning community, hypergraph neural networks are developed for effective hypergraph representations.

Original: A natural solution for learningon hypergraphs is to transform a hypergraph into a conventional graph for easy adaptation.

Original: Cliqueexpansion technique is widely used to convert the hypergraph. HGNN [1] first adopts the hypergraphtransformation and designs the weighted hypergraph Laplacian for graph convolution.
Revised: HGNN [1] adopt the clique expansion technique and designs the weighted hypergraph Laplacian for message passing.

Original: HyperGCN[2] proposes the generalized hypergraph Laplacian and explores adding the hyperedge informationthrough mediators.
Revised: HyperGCN [2] proposes the generalized hypergraph Laplacian and explores adding the hyperedge information through mediators.

Original: The attention mechanism [20, 21] is also designed to learn the importance withinhypergraphs.
Revised: The attention mechanism [29, 30] is also designed to learn the importance within hypergraphs.

Original: However, the expanded graph will inevitably cause distortion and lead to undesiredperformance.
Revised: However, the expanded graph will inevitably cause distortion and lead to unsatisfactory performance.

Original: There is also another line of works such as UniGNN [22] and HyperSAGE [23] whichtry to perform message passing directly on the hypergraph to avoid the information loss.
Revised: There is also another line of works such as UniGNN [31] and HyperSAGE [32] which try to perform message passing directly on the hypergraph to avoid the information loss.

Original: A recentwork [3] provides an AllSet framework to unify the existing studies with high expressive power andachieves state-of-the-art performance on comprehensive benchmarks.
Revised: A recent work [3] provides an AllSet framework to unify the existing studies with high expressive power and achieves state-of-the-art performance on comprehensive benchmarks.

Original: The work makes use of deeplearning methods for learning multiset functions  to identify the propagation and aggregation rules ina data-driven manner.
Revised: The work utilizes deep multiset functions [33] to identify the propagation and aggregation rules in a data-driven manner.

Original: However, few work focuses on the few-shot scenario, i.e. task-specific labelsare scarce.

Original: In this paper, we generate more effective hypergraph representation by first exploringnovel contrastive self-supervised strategies.

Original: Contrastive Self-supervision.
Revised: Contrastive self-supervised learning.

Original: Contrastive self-supervision [9, 24, 25] has achieved unprecedentedsuccess in computer vision.
Revised: Contrastive self-supervision [12, 34, 35] has achieved unprecedented success in computer vision.

Original: The core idea is to learn an embedding space where samples from thesame instance are pulled closer and samples from different instances are pushed apart.
Revised: The core idea is to learn an embedding space where samples from the same instance are pulled closer and samples from different instances are pushed apart.

Original: Recent worksbegan to marry the power of contrastive learning and graph neural networks to obtain generalizable graph representations.
Revised: Recent works start to cross-pollinate between contrastive learning and graph neural networks to for more generalizable graph representations.

Original: Typically, they design some fabricated augmentations guided by domainknowledge, such as edge perturbation, feature masking or vertex dropping, etc.
Revised: Typically, they design some fabricated augmentations guided by domain knowledge, such as edge perturbation, feature masking or vertex dropping, etc.

Original: With the rapiddevelopment of graph contrast learning, self-supervised learning on hypergraphs remains largelyunexplored.
Revised: Nevertheless, contrastive learning on hypergraphs remains largely unexplored.

Original: Most existing works [26, 27, 17] design pretext tasks for hypergraphs and mainlyfocus on recommender systems. They all conduct self-supervised learning between the convertedhypergraphs and graphs, which will lose important structural information.Moreover, they solve a verydifferent problem by generating hypergraph from the conventional graph based on the characteristicsof the data. The hypergraph only serves as a complementary function.
Revised: Most existing works [6, 36, 26, 37] design pretext tasks for hypergraphs and mainly focus on recommender systems [38, 39, 40, 41], via contrasting between graphs and hypergraphs which might lose important higher-order information.

Original: In this work, we discover thestructure of hypergraph itselfand identify how to construct contrastive views for hypergraphs.  Methods
Revised: In this work, we explore on the structure of hypergraph itself to construct contrastive views.

Original: [Figure]
Revised: [Figure]

Original: ×

Original: In the figure, we show two examples of hypergraph augmentations.

Original: At the top, the dotted ellipse denotes the deleted hyperedge.

Original: At the bottom, one vertex in the dotted is removed.

Original:   3.1 Hypergraph Contrastive Learning
Revised:  Methods 3.1 Hypergraph Contrastive Learning

Original: [Figure]

Original: A hypergraph is denoted as G =
Revised: A hypergraph is denoted as G =

Original: {V , E} ∈ G where V = { v 1 , ..., v |V| } is the set of vertices and
Revised: {V , E} ∈ G where V = { v 1 , ..., v |V| } is the set of vertices and

Original: E = { e 1 , ..., e |E| } is the set of hyperedges.
Revised: E = { e 1 , ..., e |E| } is the set of hyperedges.

Original: Each hyperedge e n = { v 1 , ..., v | e n | } represents thehigher-order interaction among a set of vertices.
Revised: Each hyperedge e n = { v 1 , ..., v | e n | } represents the higher-order interaction among a set of vertices.

Original: State-of-the-art approaches to encode such complexstructures are hypergraph neural networks (HyperGNNs)
Revised: State-of-the-art approaches to encode such complex structures are hypergraph neural networks (HyperGNNs)

Original: [1, 2, 3], mapping the hypergraph to a
Revised: [1, 2, 3], mapping the hypergraph to a

Original: D -dimension latent space via f : G → R D with higher-order message passing.
Revised: D -dimension latent space via f : G → R D with higher-order message passing.

Original: Motivated from learning on images/graphs, we adopt contrastive learning to further improve the generalizability of HyperGNNs in the low-label regime.
Revised: Motivated from learning on images/graphs, we adopt contrastive learning to further improve the generalizability of HyperGNNs in the low-label regime (HyperGCL).

Original: Main components of our hypergraph contrastivelearning (HyperGCL), similar to images/graphs [9, 11] include: (i) hypergraph augmentations forcontrastive views , (ii) HyperGNNsas hypergraph encoders, (iii) projection head h
Revised: Main components of our HyperGCL, similar to images/graphs [12, 14] include: (i) hypergraph augmentations for contrastive views , (ii) HyperGNNs

Original:  ( · ) for representations, and (iv) contrastive loss for optimization, with the overall pipeline  shown in Figure 1.L cldenotes the contrastive loss.
Revised: as hypergraph encoders, (iii) projection head h ( · ) for representations, and (iv) contrastive loss for optimization. The overall pipeline is shown in Figure 1.

Original: Detailed descriptions and training procedure are shown in Appendix B.
Revised: Detailed descriptions and training procedure are shown in Appendix B.

Original: The main challenge here is how to effectively augment hypergraphs to construct contrastive views.
Revised: The main challenge here is how to effectively augment hypergraphs to build contrastive views.

Original: 3.2 Fabricated Augmentations of Hypergraphs
Revised: 3.2 Fabricated Augmentations for Hypergraphs

Original:  We first explore whether manually designed augmentations are suited for hypergraph contrastivelearning.
Revised: [Figure] We first explore whether manually designed augmentations are suited for HyperGCL.

Original: Since hyperedges and vertices compose a hypergraph, augmentations are fabricated withregards to topology and node features, respectively.
Revised: Since hyperedges and vertices compose a hypergraph, augmentations are fabricated with regards to topology and node features, respectively.

Original: A1.
Revised: A1.

Original: Perturbing hyperedges.
Revised: Perturbing hyperedges.

Original: The most direct augmentation on higher-order interactions is to perturbon the set of hyperedges.
Revised: The most direct augmentation on higher-order interactions is to perturb on the set of hyperedges.

Original: Since adding a hyperedge is confronted with the combinatorial challenge(see Sec.
Revised: Since adding a hyperedge is confronted with the combinatorial challenge (see

Original:  1 of introduction), here we focus on randomly removing the existing hyperedges following an i.i.d.
Revised: Sec. 1 of introduction), here we focus on randomly removing the existing hyperedges following an i.i.d.

Original: Bernoulli distribution.
Revised: Bernoulli distribution.

Original: The underlying assumption is that the partially missing higher-orderrelations do not significantly affect the semantic meaning of hypergraphs.
Revised: The underlying assumption is that the partially missing higher-order relations do not significantly affect the semantic meaning of hypergraphs.

Original: A2.
Revised: A2.

Original: Perturbing edges in equivalent bipartite graph.
Revised: Perturbing edges in equivalent bipartite graph.

Original: To augment higher-order relations in a morefine-grained way, we first convert the hypergraph into the equivalent bipartite graph, where twodisjoint sets of vertices represent vertices and hyperedges in the hypergraph, respectively (see Figure2).
Revised: To augment higher-order relations in a more fine-grained way, we first convert the hypergraph into the equivalent bipartite graph, where two disjoint sets of vertices represent vertices and hyperedges in the hypergraph, respectively (see Figure 2).

Original: On top of the bipartite graph, we perform random removal of edges.
Revised: On top of the bipartite graph, we perform random removal of edges.

Original: A2 disrupts the higher-orderrelations via randomly kicking out vertices from hyperedges, enforcing the semantics of hypergraphrepresentations to be robust to such disruption.
Revised: A2 disrupts the higher-order relations via randomly kicking out vertices from hyperedges, enforcing the semantics of hypergraph representations to be robust to such disruption.

Original: A2 is essentially the generalized version of A1.
Revised: A2 is essentially the generalized version of A1.

Original: [Figure] [Figure] Moreover, we find thatvertex augmentations forgraph-structured data [11]are applicable to hypergraphs.
Revised: Moreover, we find that vertex augmentations for graph-structured data [14] are applicable to hypergraphs.

Original: Therefore we adoptthree additional schemes ofvertex dropping ( A3 ), attribute masking ( A4 ) andsubgraph ( A5 ) into ourexperiments, with similarprior knowledge incorporated as in [11].
Revised: Therefore, we adopt three additional schemes of vertex dropping ( A3 ), attribute masking ( A4 ) and subgraph ( A5 ) into our experiments, with similar prior knowledge incorporated as in [14].

Original: 3.3 Generative Models for Hypergraph Augmentations
Revised: 3.3 Generative Models for Hypergraph Augmentations

Original: Manually designing augmentation operators requires a wealth of domain knowledge, and mightlead to sub-optimal solutions even with extensive trial-and-errors.
Revised: Manually designing augmentation operators requires a wealth of domain knowledge, and might lead to sub-optimal solutions even with extensive trial-and-errors.

Original: We next study whether/howaugmentations of hypergraphs could be learned during contrastive learning.
Revised: We next study whether/how augmentations of hypergraphs could be learned during contrastive learning.

Original: Two questions needto be answered here: (i) How to parameterize the augmentation space of hypergraphs?
Revised: Two questions need to be answered here: (i) How to parameterize the augmentation space of hypergraphs?

Original: (ii) How toincorporate the learnable augmentations into contrastive learning?
Revised: (ii) How to incorporate the learnable augmentations into contrastive learning?

Original: 3.3.1 Hypergraph Generative Models for Augmentations
Revised: 3.3.1 Hypergraph Generative Models for Augmentations

Original: Considering an augmentation operator defined as the stochastic mapping between two hypergraphmanifolds that g : G → G , a natural thought is to adopt the generative model to parameterize theaugmentation space, which in general is composed of a deterministic encoder h 1 : G → R D ′ and astochastic decoder (or sampler) h 2 : R D ′ → G .
Revised: Considering an augmentation operator defined as the stochastic mapping between two hypergraph manifolds that g : G → G , a natural thought is to adopt the generative model to parameterize the augmentation space, which in general is composed of a deterministic encoder h 1 : G → R D ′ and a stochastic decoder (or sampler) h 2 : R D ′ → G .

Original: In this way, g = h 1 ◦ h 2 .
Revised: In this way, g = h 1 ◦ h 2 .

Original: Following this thought, inspired by the well-studied generative models with variational inference[28, 29], we propose a novel variational hypergraph auto-encoder architecture (VHGAE).
Revised: Following this thought, inspired by the well-studied generative models with variational inference [42, 43], we propose a novel variational hypergraph auto-encoder architecture (VHGAE).

Original: To thebest of our knowledge, this is the first hypergraph generative model for generating augmentations ofhypergraphs, which will be used as A6 .
Revised: To the best of our knowledge, this is the first hypergraph generative model for generating augmentations of hypergraphs, which will be used as A6 .

Revised: Notice that here it only parametrizes the augmentation space of edge perturbation, and in the future node perturbation would be included.

Original: Following convention, VHGAE consists of the encoder anddecoder neural networks.
Revised: VHGAE consists of the encoder and decoder neural networks.

Original: The overall framework is shown in Figure 3.
Revised: The overall framework is shown in Figure 3.

Original: Encoder.
Revised: Encoder.

Original: The encoder embeds hypergraphs into latent representations.
Revised: The encoder embeds hypergraphs into latent representations.

Original: Instead of embedding ahypergraph into a single vector, we follow graph generative models [29] to embed it into a set ofvertex and  hyperedge representations, to facilitate the further decoding process of non-Euclideanstructures.
Revised: Instead of embedding a hypergraph into a single vector, we follow VGAE [43] to embed it into a set of vertex and in additional hyperedge representations, to facilitate the further decoding process of non-Euclidean structures.

Original: We adopt two HyperGNNs, h µ 1 and h σ 1 , to encode the mean and the logarithmic standarddeviation for variational distributions of vertex and hyperedge representations z
Revised: We adopt two HyperGNNs, h µ 1 and h σ 1 , to encode the mean and the logarithmic standard deviation for variational distributions of vertex and hyperedge representations z

Original: V ∼ q ϕ
Revised: V ∼ q ϕ

Original: ( z V |G ) =
Revised: ( z V |G ) =

Original: N ( µ V , σ 2 V ) , z E ∼ q ϕ
Revised: N ( µ V , σ 2 V ) , z E ∼ q ϕ

Original: ( z E |G )
Revised: ( z E |G )

Original: =
Revised: =

Original: N ( µ E , σ 2 E ) as follows (please refer to Appendix B for the detailedcomputing pipeline):
Revised: N ( µ E , σ 2 E ) as follows (please refer to Appendix B for the detailed computing pipeline):

Original: [Equation]
Revised: [Equation]

Original: where µ ∈ R D ′ ×|V| , log( σ ) ∈ R D ′ ×|V| .
Revised: where µ ∈ R D ′ ×|V| , log( σ ) ∈ R D ′ ×|V| .

Original: We here leverage the higher-order message passing in
Revised: We here leverage the higher-order message passing in

Original: HyperGNNs for a better encoding capacity.
Revised: HyperGNNs for a better encoding capability.

Original: Decoder.
Revised: Decoder.

Original: With the learned vertex and hyperedge variational distributions, the decoder attempts toreconstruct the higher-order relations of hypergraphs.
Revised: With the learned vertex and hyperedge variational distributions, the decoder attempts to reconstruct the higher-order relations of hypergraphs.

Original: However, modeling the space of higher-orderinteractions encounters the combinatorial challenge (see Section 1).
Revised: However, modeling the space of higher-order interactions encounters the combinatorial challenge (see Section 1).

Original: Adopting the similar strategy asin the augmentation A2 (see Section 3.2), we designate the decoder to recover the relations on theconverted bipartite graph ˜ G = { ˜ V , ˜ E} for approximation.
Revised: Adopting the similar strategy as in the augmentation A2 (see Section 3.2), we designate the decoder to recover the relations on the converted bipartite graph ˜ G = { ˜ V , ˜ E} for approximation.

Original: Mathematically, we formulate decoding as:
Revised: Mathematically, we formulate decoding as:

Original: [Equation]
Revised: [Equation]

Original: where w ve =
Revised: where w ve =

Original: z Tv z e is the learned edge logit.
Revised: z Tv z e is the learned edge logit.

Original: On the decoded topological distribution of the bipartitegraph, we perform sampling and then convert the sample back to the hypergraph (the conversionbetween hypergraphs and bipartite graphs is lossless).
Revised: On the decoded topological distribution of the bipartite graph, we perform sampling and then convert the sample back to the hypergraph (the conversion between hypergraphs and bipartite graphs is lossless).

Original: Generator optimization.
Revised: Generator optimization.

Original: With variational inference [28, 30], we optimize the hypergraph generator175 on the evidence lower bound (ELBO) as follows:
Revised: With variational inference [42, 44, 45], we optimize the hypergraph generator on the evidence lower bound (ELBO) as follows:

Original: [Equation]
Revised: [Equation]

Original:  where q ϕ ( z E |G ) and q ϕ ( z V |G ) are their variational distribution, p ( z V ) and p ( z E ) are default Gaussianpriors with p ( z V ) ∼ N (0 , I ) , p ( z E ) ∼ N (0 , I ) .
Revised: [Figure] where q ϕ ( z E |G ) and q ϕ ( z V |G ) are their variational distribution, p ( z V ) and p ( z E ) are default Gaussian priors with p ( z V ) ∼ N (0 , I ) , p ( z E ) ∼ N (0 , I ) .

Original: When generating hypergraphs, the generator wouldsample the relations on the converted bipartite graph with probability p ( ˜
Revised: When generating hypergraphs, the generator would sample the relations on the converted bipartite graph with probability p ( ˜

Original: G| z V , z E ) .
Revised: G| z V , z E ) .

Original: 3.3.2 Jointly Augmenting and Contrasting with Gumbel-Softmax
Revised: 3.3.2 Jointly Augmenting and Contrasting with Gumbel-Softmax

Original: With hypergraph augmentations parametrized with generative models, the next step is to incorporateaugmentation learning into HyperGCL.
Revised: With hypergraph augmentations parametrized with generative models, the next step is to incorporate augmentation learning into HyperGCL.

Original: The main barrier results from the discrete sampling ofhyperedges which is non-differentiable.
Revised: The main barrier results from the discrete sampling of hyperedges which is non-differentiable.

Original: To tackle it, we leverage the Gumbel-Softmax trick [19] forthe hyperedge distribution as:
Revised: To tackle it, we leverage the Gumbel-Softmax trick [28] for the hyperedge distribution as:

Original: [Equation]
Revised: [Equation]

Original: where w VE denotes the learned edge logits (before Sigmoid) and δ ∼ Uniform (0 , 1) .
Revised: where w VE denotes the learned edge logits (before Sigmoid) and δ ∼ Uniform (0 , 1) .

Original: When hyperparameter temperature τ → 0 , the results get closer to being binary.
Revised: When hyperparameter temperature τ → 0 , the results get closer to being binary.

Original: T is the sampled one-hot vector foreach hyperedge-vertex interaction in the hypergraph G .
Revised: T is the sampled one-hot vector for each hyperedge-vertex interaction in the hypergraph G .

Original: Then the sampled vector will be applied toperform augmentation.
Revised: Then the sampled vector will be applied to perform augmentation.

Original: During the Gumbel-Softmax, we leverage the reparametrization trick [28]to smooth the gradient and make the sample operation differentiable.
Revised: During the Gumbel-Softmax, we leverage the reparametrization trick [42] to smooth the gradient and make the sample operation differentiable.

Original: Thus, this objective can beoptimized in an end-to-end manner as:
Revised: Thus, this objective can be optimized in an end-to-end manner as:

Original: [Equation]
Revised: [Equation]

Original: where L gen = − ELBO is the generator loss to be minimized, β is the tradeoff factor.
Revised: where L gen = − ELBO is the generator loss to be minimized, β is the tradeoff factor.

Original: Due to thecomputational cost of VHGAE, we train one VHGAE to produce one generative view, with the otherview ˜
Revised: Due to the computational cost of collaboratively optimizing two generative views, we train one VHGAE to produce one generative view, with the other view ˜

Original: G p is kept as fabricated.
Revised: G p is kept as fabricated.

Revised: To be specific, (i) independently optimizing two hypergraph generators is of reasonable budgets but would lead to distribution collapse (i.e., two hypergraph generators output the same distribution)

Revised: [1,2] which results in less effective generative views, while (ii) the collaborative optimization techniques for graph generators (e.g. REINFORCE on the rewards of generative graph structures) are not directly applicable to HyperGCL due to the combinatorial challenge of hypergraph structures (which is computationally expensive).

Original: The goal of this multi-task loss is to generate stronger augmentation(maximize contrastive loss) to push HyperGNN to avoid capturing redundant information during therepresentation learning, while at the same time learning the hypergraph data distribution.
Revised: The goal of this multi-task loss is to generate stronger augmentation (maximize contrastive loss) to push HyperGNN to avoid capturing redundant information during the representation learning, while at the same time learning the hypergraph data distribution.

Original:  Experiments 
Revised:  Experiments 4.1 Setup

Original: 4.1 Setup

Original: We examine our methods on the most comprehensive hypergraph benchmarks with thirteen data sets,with statistics shown in Table 1.
Revised: We examine our methods on the most comprehensive hypergraph benchmarks with 13 data sets, with statistics shown in Table 1.

Original: Please refer to Appendix C for detailed information.
Revised: Please refer to Appendix C for detailed information.

Original: We focus on[Table] [Table] semi-supervised vertex classification in the transductive setting.
Revised: We focus on semi-supervised vertex classification in the transductive setting.

Original: Different from the existing work
Revised: Different from the existing work

Original: [Table]
Revised: [Table]

Original: [3] that leverages 50% of all vertexes as the training set, we focus on the more low-label regimeof challenging and practical applications.
Revised: [3] that leverages 50% of all vertexes as the training set, we focus on the more low-label regime of challenging and practical applications.

Original: By default, we split the data into training/validation/testsamples using (10%/10%/80%) splitting percentages.
Revised: By default, we split the data into training/validation/test samples using (10%/10%/80%) splitting percentages.

Original: Each experiment is run for 20 different datasplits and initialization with mean and standard deviation reported.
Revised: Each experiment is run for 20 different data splits and initialization with mean and standard deviation reported.

Original: We adopt state-of-the-art SetGNN[3] as the backbone HyperGNN architecture.
Revised: We adopt state-of-the-art SetGNN [3] as the backbone HyperGNN architecture.

Original: For baselines, we compare two existing hypergraphself-supervised approaches [27] and [17] in recommender systems, denoted as Self and Con.
Revised: For baselines, we compare two existing hypergraph self-supervised approaches [36] and [26] in recommender systems, denoted as Self and Con.

Original: Theyconduct self-supervised learning between the hypergraph and conventional graph.
Revised: They conduct self-supervised learning between the hypergraph and conventional graph.

Original: By default, weadopt multi-task training to incorporate contrastive self-supervision because it performs the best asshown in the comparison in Section 4.2.
Revised: By default, we adopt multi-task training to incorporate contrastive self-supervision because it performs the best as shown in the comparison in Section 4.2.

Original: All the implementation details are listed in Appendix C.
Revised: All the implementation details are listed in Appendix C.

Original: More experiments of the hyperparameters study are given in Appendix A.
Revised: More experiments of the hyperparameters study are given in Appendix A.

Original: 4.2 Results
Revised: 4.2 Results

Original: [Table]

Original: [Table]
Revised: [Table]

Original: Comparison among different hypergraph augmentations.
Revised: Comparison among different hypergraph augmentations.

Original: The augmentation operations are summarizedin Table 2.
Revised: The augmentation operations are summarized in Table 2.

Original: Please refer to Appendix C.5 for detaileddescriptions.
Revised: Please refer to Appendix C.5 for detailed descriptions.

Original: We first conduct experiments to comparedifferent contrastive operations on hypergraphs, withresults shown in Table 3.
Revised: We first conduct experiments to compare different contrastive operations on hypergraphs, with results shown in Table 3.

Original: In general, generalized hyperedge augmentation (A2) works the best among fabricated augmenting operators, but not naïvely perturbinghyperedge (A1).
Revised: In general, generalized hyperedge augmentation (A2) works the best among fabricated augmenting operators, but not naïvely perturbing hyperedge (A1).

Original: Specifically, among all fabricated augmentations, A2 performs the best in 10 of 13 data sets.
Revised: Specifically, among all fabricated augmentations, A2 performs the best in 10 ofdata sets.

Original: This indicates the nature that higher-order informationin structures is usually more downstream-relevant.
Revised: This indicates the nature that higherorder information in structures is usually more downstream-relevant.

Original: For our generative augmentation (A6), we find it performs the best in all the data sets.
Revised: For our generative augmentation (A6), we find it performs the best in all the data sets.

Original: In our joint augment- [Table] [Table] ing and contrasting framework, we generate stronger augmentation while keeping the hypergraphdistribution with adversarial learning.
Revised: In our joint augmenting and contrasting framework, we generate stronger augmentation while keeping the hypergraph distribution with adversarial learning.

Original: This illustrates the importance of exploring the hypergraphstructure.
Revised: This illustrates the importance of exploring the hypergraph structure.

Original: We also test our method on 1% label setting in Table 4.
Revised: We also test our method on 1% label setting in Table 4.

Original: In this setting, Zoo and NTU2012data sets are not shown because of the extremely small data size (each case has less than one trainingsample).
Revised: In this setting, Zoo and NTU2012 data sets are not shown because of the extremely small data size (each case has less than one training sample).

Original: We can find that in the 1% label setting A4 (mask) method performs the best in Cora, Citeseer and DBLP-CA.
Revised: We can find that in the 1% label setting A4 (mask) method performs the best in Cora, Citeseer and DBLP-CA.

Original: These data sets are all originally graphs, and are constructed as hypergraphsin different ways.
Revised: These data sets are all originally graphs, and are constructed as hypergraphs in different ways.

Original: So on these data sets, relatively little higher-order structural information can beexplored with hypergraph structure perturbation-based contrastive learning.
Revised: So on these data sets, relatively little higher-order structural information can be explored with hypergraph structure perturbation-based contrastive learning.

Original: Comparison with Pretraining.
Revised: Comparison between multi-task learning and pretraining.

Original: We then compare the multi-task training method with the pretrainingmethod in Table 5.
Revised: We then compare the multi-task training method with the pretraining method in Table 5.

Original: Pretrain_L adopts the linear evaluation protocol where a linear classifier is trainedon top of the fixed pretrained representations.
Revised: Pretrain_L adopts the linear evaluation protocol where a linear classifier is trained on top of the fixed pretrained representations.

Original: Pretrain_F follows a fully finetuning protocol that usesthe weights of the learned hypergnn encoder as initialization while finetuning all the layers.
Revised: Pretrain_F follows a fully finetuning protocol that uses the weights of the learned hypergnn encoder as initialization while finetuning all the layers.

Original: MTLdenotes the multi-task learning method which trains the supervised classification loss and contrastiveloss together.
Revised: MTL denotes the multi-task learning method which trains the supervised classification loss and contrastive loss together.

Original: For all the methods, we use A2 (generalized hyperedge perturbation) as it performs thebest among fabricated augmentations.
Revised: For all the methods, we use A2 (generalized hyperedge perturbation) as it performs the best among fabricated augmentations.

Original: From the table, we find MTL achieves the best performancein nearly all data sets.
Revised: From the table, we find MTL achieves the best performance in nearly all data sets.

Original: Pretrain_L and Pretrain_F can only obtain better performance on two smalldata sets: Zoo and House.
Revised: Pretrain_L and Pretrain_F can only obtain better performance on two small data sets: Zoo and House.

Original: We find on most data sets, Pretrain_L makes the model perform worse,which shows that the linear classifier is not enough to represent the higher-order information in thehypergraph.
Revised: We find on most data sets, Pretrain_L makes the model perform worse, which shows that the linear classifier is not enough to represent the higher-order information in the hypergraph. Pretrain_F has a much better performance compared [Table]

Original: Pretrain_F has a much better performance compared with Pretrain_L, which indicates theeffects of using contrastive learning.
Revised: [Table] with Pretrain_L [Table] which indicates the effects of using contrastive learning.

Original: However, the method switches the objective during finetuning,which would lead to the memorization problem and the loss of pre-trained knowledge.
Revised: However [Table] the method switches the objective during finetuning [Table]

Revised: which would lead to the memorization problem and the loss of pre-trained knowledge.

Original: Therefore, allour other experiments adopt MTL setting to incorporate contrastive self-supervision.
Revised: Therefore, all our other experiments adopt MTL setting to incorporate contrastive self-supervision.

Original: Comparison to the Converted Graph.
Revised: Comparison to the converted graph.

Original: Next we investigate on which graph should we contraston.
Revised: Next we investigate on which graph should we contrast on.

Original: We first convert the original hypergraph into a conventional graph using the clique expansiontechnique, and we choose the representative HGNN[1] as the backbone network for learning on the
Revised: We first convert the original hypergraph into a conventional graph using the clique expansion technique

Original: [Table]
Revised: [Table]

Original: [Figure] converted graph.
Revised: and we choose the representative HGNN[1] as the backbone network for learning on the converted graph.

Original: We compare it with two representative augmentations: A2 (Here, edge perturbationon the conventional graph) and A4 (feature masking).
Revised: We compare it with two representative augmentations: A2 (Here, edge perturbation on the conventional graph) and A4 (feature masking).

Original: In Table 6, we find that HGNN performspoorly compared with SetGNN.
Revised: In Table [Table] we find that HGNN performs poorly compared with SetGNN.

Original: Apparently contrastive self-supervision on converted graphs doesnot bring much benefit.
Revised: Apparently contrastive self-supervision on converted graphs does not bring much benefit.

Original: The reason is that part of structural information, important to hypergraphrepresentation learning, is lost when hypergraphs are converted to graphs.
Revised: The reason is that part of structural information, important to hypergraph representation learning, is lost when hypergraphs are converted to graphs.

Original: These results indicate theimportance of designing HyperGNN and contrastive strategies directly on hypergraphs. 
Revised: These results indicate the importance of designing HyperGNN and contrastive strategies directly on hypergraphs. [Table] [Table]

Original: Analysis of Generative Augmentation.
Revised: Analysis of generative augmentation.

Original: Here we analyze our proposed generative hypergraphaugmentation (A6).
Revised: Here we analyze our proposed generative hypergraph augmentation (A6).

Original: We select ModelNet40 and Yelp as the representatives of high-homophily andlow-homophily data sets, respectively.
Revised: We select ModelNet40 and Yelp as the representatives of high-homophily and low-homophily data sets, respectively.

Original: First, we examine the training dynamics of keep ratio and find that they are highly related to the dataset homophily (Figure 4 (a)).
Revised: First, we examine the training dynamics of keep ratio and find that they are highly related to the data set homophily (Figure 4 (a)).

Original: For ModelNet40, the generator keeps only 20% relations in the earlytraining stage.
Revised: For ModelNet40, the generator keeps only 20% relations in the early training stage.

Original: This is because the homophily of the data set is very high (0.92/0.88) and deleting alot of edges relatively randomly at the beginning would not have a large impact on the model butrather can help model learn structural information.
Revised: This is because the homophily of the data set is very high (0.92/0.88) and deleting a lot of edges relatively randomly at the beginning would not have a large impact on the model but rather can help model learn structural information.

Original: Then at a later stage, the generator begins tokeep more than 80% relations, which means that the model has learned higher-order information andonly removes the unnecessary relations.
Revised: Then at a later stage, the generator begins to keep more than 80% relations, which means that the model has learned higher-order information and only removes the unnecessary relations.

Original: For Yelp, a similar conclusion holds.
Revised: For Yelp, a similar conclusion holds.

Original: Specifically, as itshomophily is pretty low (0.57/0.26), the generator keeps most of the relations at the early stage fortraining and then just keeps a very low ratio of related relations at the later training stage.
Revised: Specifically, as its homophily is pretty low (0.57/0.26), the generator keeps most of the relations at the early stage for training and then just keeps a very low ratio of related relations at the later training stage.

Original: Next, we investigate what our generator has learned.
Revised: Next, we investigate what our generator has learned.

Original: We visualize the hyperedges in the Yelp dataset in Figure 4 (b).
Revised: We visualize the hyperedges in the Yelp data set in Figure 4 (b).

Original: Yelp is a restaurant-rating data set and the restaurants visited by the same userare connected by a hyperedge.
Revised: Yelp is a restaurant-rating data set and the restaurants visited by the same user are connected by a hyperedge.

Original: We find that some vertices with different labels are removed, whichcould remove extraneous information and improve the hypergraph homophily.
Revised: We find that some vertices with different labels are removed, which could remove extraneous information and improve the hypergraph homophily.

Original: This indicates that ourgenerator does grasp the higher-order information in the hypergraph.
Revised: This indicates that our generator does grasp the higher-order information in the hypergraph.

Original: Adversarial Robustness.
Revised: Adversarial robustness.

Original: Besides generalizability, we here show hypergraph contrastive learning alsoboosts robustness.
Revised: Besides generalizability, we here show hypergraph contrastive learning also boosts robustness.

Original: Since there is no existing work developing adversarial attack algorithms designatedfor hypergraphs, we adapt two state-of-the-art attackers from the graph domain.
Revised: Since there is no existing work developing adversarial attack algorithms designated for hypergraphs, we adapt two state-of-the-art attackers from the graph domain.

Original: We presume thegraph attackers are applicable to hypergraphs to a certain extent, both of which are non-Euclideandata structures with sharing properties.
Revised: We presume the graph attackers are applicable to hypergraphs to a certain extent, both of which are non-Euclidean data structures with sharing properties.

Original: The experiments are performed on five real-world data sets:
Revised: The experiments are performed on five real-world data sets:

Original: Cora, Citeseer, ModelNet40, NTU2012 and House.
Revised: Cora, Citeseer, ModelNet40, NTU2012 and House.

Original: We regard each hypergraph as a bipartite graphand leverage those algorithms to conduct attacks to the vertices and hyperedges.
Revised: We regard each hypergraph as a bipartite graph and leverage those algorithms to conduct attacks to the vertices and hyperedges.

Original: The methods include[Table] [Table] an untargeted attack method, minmax attack [31] which poisons the graph structures by adding andremoving relations to reduce the overall performance; and a targeted attack method, nettack [32]which leads the HyperGNN to mis-classify target vertices.
Revised: The methods include an untargeted attack method, minmax attack [46] which poisons the graph structures by adding and removing relations to reduce the overall performance; and a targeted attack method, nettack [47] which leads the HyperGNN to mis-classify target vertices.

Original: Beyond these, we also include a randomhypergraph perturbation baseline which will randomly drop numerous relations in the hypergraph.
Revised: Beyond these, we also include a random hypergraph perturbation baseline which will randomly drop numerous relations in the hypergraph.

Original: These three methods are denoted as Net, Minmax and Random.
Revised: These three methods are denoted as Net, Minmax and Random.

Original: Following previous works, for eachattack, we perturb 10% of vertices/relations.
Revised: Following previous works, for each attack, we perturb 10% of vertices/relations.

Original: The results in Table 7 show that Random and Minmaxattacks can decrease the performance of the original model a little on all the data sets, while netattack can decrease the performance on most data sets and surprisingly increase the performance onthe House data set.
Revised: The results in Table 7 show that Random and Minmax attacks can decrease the performance of the original model a little on all the data sets, while net attack can decrease the performance on most data sets and surprisingly increase the performance on [Figure] [Figure] [Table]

Original:  This performance gain indicates HyperGNN is more robust to structure attackcompared with GNN as it leverages higher-order information.
Revised: the House data set. This performance gain indicates HyperGNN is more robust to structure attack compared with GNN as it leverages higher-order information.

Original: Based on these, HyperGCL withgeneralized hyperedge augmentation (A2) performs better than feature perturbation (A4), and ourproposed generative augmentation (A6) can surpass these fabricated baselines on all the data setsand thus is the best to defend attacks.
Revised: Based on these, HyperGCL with generalized hyperedge augmentation (A2) performs better than feature perturbation (A4), and our proposed generative augmentation (A6) can surpass these fabricated baselines on all the data sets and thus is the best to defend attacks.

Original: We believe this’ll be a beneficial complement to our mainexperiments and we hope for more works on hypergraph attacks.
Revised: We believe this’ll be a beneficial complement to our main experiments and we hope for more works on hypergraph attacks.

Original: Fairness.
Revised: Fairness.

Original: Furthermore, we claim that hypergraph contrastive self-supervision also benefits fairness.
Revised: Furthermore, we claim that hypergraph contrastive self-supervision also benefits fairness.

Original: There was no related data set before.
Revised: There was no related data set before.

Original: So we introduce three newly curated hypergraph data sets:
Revised: So we introduce three newly curated hypergraph data sets:

Original: German [33], Recidivism [34] and Credit [35].
Revised: German [48], Recidivism [49] and Credit [50].

Original: The hypergraph construction follows the setting in [1].
Revised: The hypergraph construction follows the setting in [1].

Original: The top 5 similar objects in each data set are built as a hyperedge.
Revised: The top 5 similar objects in each data set are built as a hyperedge.

Original: For the accuracy metrics, we use
Revised: For the accuracy metrics, we use

Original: F1-score and AUROC value for the binary classification task.
Revised: F1-score and AUROC value for the binary classification task.

Original: For measuring fairness, we adopt thestatistical parity ∆ SP and equalized odds ∆ EO .
Revised: For measuring fairness, we adopt the statistical parity ∆ SP and equalized odds ∆ EO .

Original: Please refer to Appendix C for detailed informationabout the data sets and metrics.
Revised: Please refer to Appendix C for detailed information about the data sets and metrics.

Original: The experimental results in Table 8 show that our generative methodstill achieves better or comparable performances while imposing more fairness.
Revised: The experimental results in Table 8 show that our generative method still achieves better or comparable performances while imposing more fairness.

Original:  Conclusion
Revised:  Conclusion

Original: In the paper, we study the problem of how to construct contrastive views of hypergraphs via augmentations.
Revised: In the paper, we study the problem of how to construct contrastive views of hypergraphs via augmentations.

Original: We provide the solutions by first studying domain knowledge-guided fabrication schemes.
Revised: We provide the solutions by first studying domain knowledge-guided fabrication schemes.

Original: Then, in search of more effective views in a data-driven manner, we are the first to propose hypergraphgenerative models to generate augmented views, as well as an end-to-end differentiable pipeline tojointly perform hypergraph augmentation and contrastive learning.
Revised: Then, in search of more effective views in a data-driven manner, we are the first to propose hypergraph generative models to generate augmented views, as well as an end-to-end differentiable pipeline to jointly perform hypergraph augmentation and contrastive learning.

Original: We find that generative augmentations perform better at preserving higher-order information to further benefit generalizability.
Revised: We find that generative augmentations perform better at preserving higher-order information to further benefit generalizability.

Original: Theproposed framework also boosts robustness and fairness of hypergraph representation learning.
Revised: The proposed framework also boosts robustness and fairness of hypergraph representation learning.

Original: Inthe future, we plan to design more powerful hypergraph generator and HyperGNN while addressingmore real-world hypergraph data challenges and more hypergraph learning models. 
Revised: In the future, we plan to design more powerful hypergraph generator and HyperGNN while addressing more real-world hypergraph data challenges and more hypergraph learning models. [Table]

Revised: Acknowledgments and Disclosure of Funding

Revised: This work is supported by National Science Foundation under Award No.

Original: Checklist
Revised: IIS-1947203, IIS-2117902, IIS-2137468, CCF-1943008; US Army Research Office Young Investigator Award W911NF2010240; National Institute of General Medical Sciences under grant R35GM124952; and Agriculture and Food Research Initiative grant no. 2020-67021-32799/project accession no.1024178 from the USDA

Revised: National Institute of Food and Agriculture.

Revised: The views and conclusions are those of the authors and should not be interpreted as representing the official policies of the government agencies.

Revised: Checklist

Original: The checklist follows the references.
Revised: The checklist follows the references.

Original: Please read the checklist guidelines carefully for information onhow to answer these questions.
Revised: Please read the checklist guidelines carefully for information on how to answer these questions.

Original: For each question, change the default [TODO] to [Yes] , [No] or[N/A] .
Revised: For each question, change the default [TODO] to [Yes] , [No] or [N/A] .

Original: You are strongly encouraged to include a justification to your answer , either by referencingthe appropriate section of your paper or providing a brief inline description.
Revised: You are strongly encouraged to include a justification to your answer , either by referencing the appropriate section of your paper or providing a brief inline description.

Original: For example:
Revised: For example:

Original: • Did you include the license to the code and datasets?
Revised: • Did you include the license to the code and datasets?

Original: [Yes] See Section.•
Revised: [Yes] See Section.

Original:  Did you include the license to the code and datasets?
Revised: • Did you include the license to the code and datasets?

Original: [No] The code and the data are proprietary.
Revised: [No] The code and the data are proprietary.

Original: • Did you include the license to the code and datasets?
Revised: • Did you include the license to the code and datasets?

Original: [N/A]
Revised: [N/A]

Original: For all authors...
Revised: Please do not modify the questions and only use the provided macros for your answers. Note that the Checklist section does not count towards the page limit. In your paper, please delete this instructions block and only keep the Checklist section heading above along with the questions/answers below.

Original:  (a) Do the main claims made in the abstract and introduction accurately reflect the paper’scontributions and scope?
Revised: For all authors... (a) Do the main claims made in the abstract and introduction accurately reflect the paper’s contributions and scope?

Original: [Yes] (b) Did you describe the limitations of your work?
Revised: [Yes] (b) Did you describe the limitations of your work?

Original: [Yes] Please check Apendix B. 
Revised: [Yes] Please check Apendix B. (c)

Original: (c) Did you discuss any potential negative societal impacts of your work?
Revised: Did you discuss any potential negative societal impacts of your work?

Original: [N/A] We are not aware of any potential ethical issues regarding our work.
Revised: [N/A] We are not aware of any potential ethical issues regarding our work.

Original: More discussions can befound in Appendix B.(d) Have you read the ethics review guidelines and ensured that your paper conforms tothem?
Revised: More discussions can be found in Appendix B. (d) Have you read the ethics review guidelines and ensured that your paper conforms to them?

Original: [Yes] 
Revised: [Yes] 2.

Original: If you are including theoretical results... (a) Did you state the full set of assumptions of all theoretical results?
Revised: If you are including theoretical results... (a) Did you state the full set of assumptions of all theoretical results?

Original: [N/A] (b) Did you include complete proofs of all theoretical results?
Revised: [N/A] (b) Did you include complete proofs of all theoretical results?

Original: [N/A] 
Revised: [N/A] 3.

Original: If you ran experiments... (a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?
Revised: If you ran experiments... (a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?

Original: [Yes] We add them in the supplementary materials.
Revised: [Yes] We add them in the supplementary materials.

Original: (b) Did you specify all the training details (e g., data splits, hyperparameters, how theywere chosen)?
Revised: (b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)?

Original: [Yes] We write the implementation details in Appendix C. (c) Did you report error bars (e g., with respect to the random seed after running experiments multiple times)?
Revised: [Yes] We write the implementation details in Appendix C. (c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?

Original: [Yes] We report the standard deviation of 20 runs with different splits.
Revised: [Yes] We report the standard deviation of 20 runs with different splits.

Original: (d) Did you include the total amount of compute and the type of resources used (e g., typeof GPUs, internal cluster, or cloud provider)?
Revised: (d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)?

Original: [Yes] We specify the computing resouce in Appendix C.
Revised: [Yes] We specify the computing resouce in Appendix C.

Original: If you are using existing assets (e g., code, data, models) or curating/releasing new assets... (a) If your work uses existing assets, did you cite the creators?
Revised: If you are using existing assets (e.g., code, data, models) or curating/releasing new assets... (a) If your work uses existing assets, did you cite the creators?

Original: [Yes] (b) Did you mention the license of the assets?
Revised: [Yes] (b) Did you mention the license of the assets?

Original: [Yes] We discuss it in Appendix C. (c) Did you include any new assets either in the supplemental material or as a URL?
Revised: [Yes] We discuss it in Appendix C. (c) Did you include any new assets either in the supplemental material or as a URL?

Original: [Yes] We submit the code in the supplementary materials.
Revised: [Yes] We submit the code in the supplementary materials.

Original: (d) Did you discuss whether and how consent was obtained from people whose data you’re using/curating?
Revised: (d) Did you discuss whether and how consent was obtained from people whose data you’re using/curating?

Original: [Yes] We discuss it in Appendix C. (e) Did you discuss whether the data you are using/curating contains personally identifi- able information or offensive content?
Revised: [Yes] We discuss it in Appendix C. (e) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content?

Original: [Yes] The data sets we use don’t contain any personally identifiable information.
Revised: [Yes] The data sets we use don’t contain any personally identifiable information.

Original: If you used crowdsourcing or conducted research with human subjects... (a) Did you include the full text of instructions given to participants and screenshots, if applicable?
Revised: If you used crowdsourcing or conducted research with human subjects... (a) Did you include the full text of instructions given to participants and screenshots, if applicable?

Original: [N/A] We don’t conduct research with human subjects.
Revised: [N/A] We don’t conduct research with human subjects.

Original: (b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable?
Revised: (b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable?

Original: [N/A] (c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation?
Revised: [N/A] (c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation?

Original: [N/A]
Revised: [N/A]