Self-supervised Heterogeneous Graph Pre-training Based on Structural Clustering
Anonymous Author(s)
AffiliationAddress email
Abstract
Recent self-supervised pre-training methods on Heterogeneous Information Networks (HINs) have shown promising competitiveness over traditional semisupervised Heterogeneous Graph Neural Networks (HGNNs).
Unfortunately, theirperformance heavily depends on careful customization of various strategies for generating high-quality positive examples and negative examples, which notably limitstheir flexibility and generalization ability.
In this work, we present SHGP, a novel
Self-supervised Heterogeneous Graph Pre-training approach, which does not needto generate any positive examples or negative examples.
It consists of two modulesthat share the same attention-aggregation scheme.
In each iteration, the Att-LPAmodule produces pseudo-labels through structural clustering, which serve as theself-supervision signals to guide the Att-HGNN module to learn object embeddingsand attention coefficients.
The two modules can effectively utilize and enhanceeach other, promoting the model to learn discriminative embeddings.
Extensiveexperiments on four real-world datasets demonstrate the superior effectiveness of
SHGP against state-of-the-art unsupervised baselines and even semi-supervisedbaselines.
We will release our source code at GitHub once the manuscript isaccepted.
Introduction
Over the past few years, various semi-supervised graph neural networks (GNNs) have been proposedto learn graph embeddings.
They have achieved remarkable success in many graph analytic tasks.
This success, however, comes at the cost of a heavy reliance on high-quality supervision labels.
Inreal-world scenarios
, labels are usually expensive to acquire, and sometimes even impossible due toprivacy concerns.
To relieve the label scarcity issue in (semi-) supervised learning and take full advantage of a largeamount of easily available unlabeled data, the self-supervised learning (SSL) paradigm has recentlydrawn considerable research interest in the computer vision research community.
It leverages thesupervision signal from the data itself to learn generalizable embeddings, which are then transferredto various downstream tasks with only a few task-specific labels.
One of the most common SSLparadigms is contrastive learning, which learns representations by estimating and maximizing themutual information between the input and the output of a deep neural network encoder [8].
For graphs, some recent graph contrastive learning methods [29, 7, 41, 25, 20, 19, 43, 36, 40] haveshown promising competitiveness compared with semi-supervised GNNs.
They usually require threetypical steps: (1) constructing positive examples (semantically correlated structural instances) bystrategies such as node dropping, edge perturbation, and negative examples (uncorrelated instances) bystrategies such as feature shuffling, mini-batch sampling; (2) encoding these examples through graphencoders such as GCN [16]; (3) maximizing/minimizing the similarity between these positive/negative examples.
Nevertheless, in the real world, graphs often contain multiple types of objects and multipletypes of relationships between them, which are called heterogeneous graphs, or heterogeneousinformation networks (HINs) [27].
Due to the challenges caused by the heterogeneity, existing SSLmethods on homogeneous graphs cannot be straightforwardly applied to HINs.
Very recently, severalworks have made some efforts to conduct SSL on HINs [33, 23, 18, 15, 13, 14, 37, 10].
In comparisonwith SSL methods on homogeneous graphs, the key difference is that they usually have differentexample generation strategies, so as to capture the heterogeneous structural properties in HINs.
The strategies of generating high-quality positive/negative examples are critical to the performance ofexisting methods [34, 4, 41, 36].
Unfortunately, whether for homogeneous graphs or heterogeneousgraphs, the example generation strategies are dataset-specific, and may not be applicable to allscenarios.
This is because real-world graphs are abstractions of things from various domains, e.g.,social networks, citation networks, etc.
They usually have significantly different structural propertiesand semantics.
Previous works have systematically studied this, and found that different strategiesare good at capturing different structural semantics.
For example, study [41] observed that edgeperturbation benefits social networks but hurts some biochemical networks, and study [36] observedthat negative examples benefit sparser graphs.
Consequently, in practice, the example generationstrategies have to be empirically constructed and investigated through either trial-and-error or rulesof thumb.
This significantly limits the practicality and general applicability of existing methods.
In this work, we focus on HINs which are more challenging, and propose a novel SSL approach,named SHGP.
Different from existing methods, SHGP requires neither positive examples nor negativeexamples , thus circumventing the above issues.
Specifically, SHGP adopts any HGNN model that isbased on attention-aggregation scheme as the base encoder, which is termed as the module Att-HGNN.
The attention coefficients in Att-HGNN are particularly used to combine with the structural clusteringmethod LPA (label propagation algorithm)
[21], as the module Att-LPA.
Through performingstructural clustering on HINs, Att-LPA is able to produce clustering labels, which are treated aspseudo-labels.
In turn, these pseudo-labels serve as guidance signals to help Att-HGNN learn betterembeddings as well as better attention coefficients.
Thus, the two modules are able to exploit andenhance each other, finally leading the model to learn discriminative and informative embeddings.
Insummary, we have three main contributions as follows:
• We propose a novel SSL method on HINs, SHGP.
It innovatively consists of the Att-LPA moduleand the Att-HGNN module.
The two modules can effectively enhance each other, facilitating themodel to learn effective embeddings.
• To the best of our knowledge, SHGP is the first attempt to perform SSL on HINs without anypositive or negative examples.
Therefore, it can directly avoid the laborious investigation ofexample generation strategies, improving the model’s generalization ability and flexibility.• We transfer the object embeddings learned by SHGP to various downstream tasks.
The experimental
results show that SHGP can outperform state-of-the-art baselines, even including some semisupervised baselines, demonstrating its superior effectiveness.
Related work
SSL on HINs.
There are several existing methods [33, 23, 18, 15, 37, 13, 14, 10] that conduct
SSL on HINs.
Determined by their contrastive loss functions, all these methods require high-qualitypositive and negative examples to effectively learn embeddings.
Thus, their effectiveness andperformance hinge on the specific strategies of generating positive examples and negative examples,which limits their flexibility and generalization ability.
SSL on homogeneous graphs.
Existing SSL methods on homogeneous graphs [29, 7, 41, 25,20, 19, 43, 36, 35, 30] also need to generate sufficient positive and negative examples to effectivelyperform graph contrastive learning.
They only handle homogeneous graphs and cannot be easilyapplied to HINs.
In this work, we seek to perform SSL on HINs without any positive examples ornegative examples.
GNN+LPA methods.
There exist several methods [31, 1, 24] that combine LPA [21] with GNNs.
However, they are all supervised learning methods, and only deal with homogeneous graphs.
In thiswork, we study SSL on HINs.
Others.
DeepCluster [2] uses K -means to perform clustering in the embedding space.
Differently,our SHGP directly performs structural clustering in the graph space.
JOAO [40] explores theautomatic selection of positive example generation strategies, which is still not fully automatic.
HuBERT [9] is an SSL approach for speech representation learning.
GIANT [3] leverages graphstructured self-supervision to extract numerical node features from raw data.
MARU [12] learnsobject embeddings by exploiting meta-contexts in random walks.
Different from them, in this work,we study how to effectively conduct SSL on HINs.
Graph pooling methods e.g. [22, 39] learn softcluster assignment to coarsen graph topology in each model layer.
Differently, our method propagatesinteger (hard) cluster labels in each layer to perform structural clustering.
Preliminaries
We first briefly introduce some concepts about HINs, and then formally describe the problem westudy in this paper.
Heterogeneous Information Network.
An HIN is defined as: G = ( V , E , A , R , ϕ, ψ ) , where Vis the set of objects, E is the set of links, ϕ : V → A and ψ : E → R are respectively the objecttype mapping function and the link type mapping function, A denotes the set of object types, and Rdenotes the set of relations (link types), where |A| + |R| > 2 .
Let X
=
X 1 , ..., X |A| } denote theset containing all the feature matrices associated with each type of objects.
A meta-path P of lengthl is defined in the form of A 1 R 1 −−→
A 2 R 2 −−→ · · · R l −→ A l +1 (abbreviated as A 1 A 2 · · · A l +1 ), whichdescribes a composite relation R = R 1 ◦ R 2 ◦ · · · ◦ R l between object types
A 1 and A l +1 , where ◦denotes the composition operator on relations.
We show a toy HIN in the left part of Figure 1.
It contains four object types: “Paper” ( P ), “Author”( A ), “Conference” ( C ) and “Term” ( T ), and three relations: “Publish” between P and C , “Write”between P and A , and “Contain” between P and T .
APC is a meta-path of length two, and a 1 p 2 c 2is such a path instance, which means that author a 1 has published paper p 2 in conference c 2 .
SSL on HINs.
Given an HIN G , the problem is to learn an embedding vector h i ∈
R d for eachobject i ∈ V , in a self-supervised manner, i.e., without using any task-specific labels.
The pre-trainedembeddings are expected to capture the general-purpose information contained in G , and can beeasily transferred to various unseen downstream tasks with only a few task-specific labels.
[Figure]
[Equation]
[Figure]
Methodology
In this section, we present the proposed method SHGP, which consists of two key modules.
The AttHGNN module is instantiated as any HGNN model that is based on attention-aggregation scheme.
The
Att-LPA module combines the structural clustering method LPA [21] with the attention-aggregationscheme used in Att-HGNN.
The overall model architecture is shown in Figure 1 and explained in thefigure caption.
In the following, we describe the procedure of SHGP in detail.
4.1 Initialization
At the beginning, we randomly initialize all the model parameters in Att-HGNN by the Xavier uniformdistribution [6].
To obtain initial pseudo-labels, we use the original LPA [21] to perform a thoroughstructural clustering on the input HIN G .
Specifically, LPA randomly associates each object with aunique integer as its initial label, and lets them iteratively propagate along the links in G .
In eachiteration, each object updates its label to the label that appears most frequently in its neighborhood.
After convergence, the final label indicates the cluster to which each object belongs.
We treat theseclustering labels returned by LPA as the initial pseudo-labels, and re-organize them as a one-hot labelmatrix Y [0] ∈
R |V|× K , where K denotes the cluster size, which is not a hyper-parameter but dependson the uniqueness of these labels.
Thus, in the subsequent steps, the propagation of the pseudo-labelscan be easily implemented via the matrix multiplication operation.
Under the guidance of these obtained initial pseudo-labels, we first train Att-HGNN for severalepochs as the model “warm-up”, to learn the initial meaningful attention coefficients as well as othermodel parameters.
We use W [0] to denote all these initial parameters.
Then, we proceed with thefollowing iterations.
4.2 Iteration
In the t -th iteration, we compute the object embeddings H [ t ] by the Att-HGNN module.
Att-HGNNis parameterized by W [ t − 1] , and its inputs include: the HIN topology G , the object features X .
Thisis formulated as follows:
[Equation]
Meanwhile, we update the pseudo-labels by the creatively proposed Att-LPA module.
In comparisonwith Att-HGNN, Att-LPA does not input X , but instead inputs Y [ t − 1] , i.e., the pseudo-labels of theprevious iteration.
This is formulated as follows:
[Figure]
[Equation]
Att-HGNN and Att-LPA perform one forward pass of attention-based aggregation in the same way.
The only difference between them is that Att-HGNN aggregates the (projected) features of neighborswhile Att-LPA aggregates the pseudo-labels of neighbors produced in the previous iteration, bothweighted by exactly the same attention coefficients.
Now, we input H [ t ] into a softmax classifier which is built on the top layer of the Att-HGNN to makepredictions P [ t ] .
The loss is computed as the cross-entropy between the predictions P [ t ] and thepseudo-labels Y [ t ] , as follows:
[Equation]
where C [ t − 1] ⊂ W [ t − 1] denotes the parameter matrix of the classifier.
Having the loss, we can then optimize all the model parameters by gradient descent, as follows:
[Equation]
where η denotes the learning rate.
As the optimization proceeds, the model will learn better modelparameters (including attention coefficients).
This leads Att-HGNN and Att-LPA to respectivelyproduce better embeddings (and predictions) and better pseudo-labels in the next iteration, which,in turn, promotes the model to learn further better parameters.
Thus, the two processes are able toclosely interact with each other, as well as enhance each other, finally resulting in discriminative andinformative embeddings.
The overall procedure is shown in Algorithm 1.
4.3 Model discussion
4.3.1 Encoder of Att-HGNN
The Att-HGNN module can be specifically instantiated as any them, ie-HGCN is simple and efficient.
It has shown superior performance over the other threemodels [38].
Therefore, in this work, we adopt it as the base encoder of Att-HGNN, which is in detaildescribed as follows.
Let N i denote the set of object i ’s neighbors.
Specially, object i itself is also added to N i .
Accordingly,an edge ( i, i ) is added to E , and a dummy self-relation ψ ( i, i ) is added to R .
In each model layer,object i ’s new representation ⃗h ′
i is computed as follows:
[Equation]
(5) where σ is the non-linear activation function, and ⃗h j is neighbor j ’s current representation.
W ψ ( i,j )is the projection parameter matrix, (cid:98) a ψ ( i,j ) i,j is the normalized link weight ( (cid:98) a ψ ( i,i ) i,i = 1 ), and β ψ ( i,j ) i isthe normalized attention coefficient, all of which are specific to relation ψ ( i, j ) .
With Att-HGNN, we can formulate our Att-LPA in a similar way.
Specifically, in each layer, objecti ’s pseudo-label is updated as follows:
[Equation]
where ⃗y j is neighbor j ’s pseudo-label vector in the one-hot form.
Object i first aggregates itsneighbors’ pseudo-labels according to the same normalized link weights and attention coefficients asin Eq.
Then, its new pseudo-label vector ⃗y i ′ is obtained through the argmax operator and theone-hot encoding function.
The above two equations describe the computational details of one model layer, where the iterationindices and the layer indices are omitted for notation brevity.
Here, we can further use the superscript[ x, y ] to index a symbol with respect to x -th iteration and y -th layer.
Assume the model has N layers.
In the t -th iteration, for the input layer of Att-HGNN, we set: ⃗ h [ t, 0] i = ⃗x i (i.e., the object featurevector), while for Att-LPA, we set: ⃗y [ t, 0] i = ⃗y
[ t − 1 ,N ] i .
At the end of the t -th iteration, we properlyorganize all the ⃗h [ t,N ] i and ⃗y [ t,N ] i as H
[ t ] in Eq.
(1) and Y [ t ] in Eq.
(2) respectively.
In this way, basedon the ie-HGCN encoder, we establish our Att-HGNN module and Att-LPA module.
4.3.2 Time complexity
As analyzed in the literature, both LPA [21] and ie-HGCN [38] have quasi-linear time complexity.
Therefore, our SHGP also has quasi-linear time complexity, i.e., O ( |V| + |E| ) .
Please see Appendix
A.2 for the efficiency study of SHGP.
4.3.3 Label consistency
For SHGP, in the initialization phase, we perform the thorough LPA process until convergence toobtain the initial pseudo-labels, and warm-up Att-HGNN module to learn initial meaningful attentioncoefficients.
In each subsequent iteration, we no longer need to perform LPA from scratch, butinstead perform one forward pass of Att-LPA.
Therefore, the clustering results are basically consistent with the results of the previous iteration.
Thus, our SHGP does not need to perform alignmentbetween clustering labels and predictions.
This can avoid the issue in DeepCluster [2] that there is nocorrespondence between two consecutive clustering assignments, and can also avoid the alignmentprocess in M3S [26].
4.3.4 Class space
In this work, we cluster all types of objects into one common class space.
We note that in most existing
HIN datasets, various types of objects can indeed share the same class space.
They typically show a“star” network schema [28], since they are usually constructed according to one “hub” type of objects.
The other types of objects are actually the attributes of the “hub” objects.
For example, on DBLP, the“hub” objects are paper ( P ) objects.
These papers and their associated authors, conferences, and etc.can all be categorized into four research areas: DM, IR, DB, and AI.
In the following extensive experiments, we demonstrate that this strategy works well on existingwidely used HIN benchmark datasets.
In future work, we will further investigate the problem ofclustering different types of objects into different class spaces.
Experiments
In this section, we verify the generalization ability of the proposed SHGP by transferring thepre-trained object embeddings to various downstream tasks including object classification, objectclustering, and embedding visualization.
5.1 Datasets
In the experiments, we use four publicly available HIN benchmark datasets, which are widely used inprevious related works [38, 32, 18, 23, 33].
Their statistics are summarized in Table 1.
Please see
Appendix A.1 for more details of these datasets.
[Table]
, A (
[Table]
), S
[Table]
P ⇌ A , P
[Table]
S
[Table]
A (
[Table]
), P (14328), C (20), T (8898)
P ⇌ A , P ⇌ C , P ⇌ T
IMDB
M (3328), A (42553), U (2103), D (2016)
M ⇌
A
[Equation]
M ⇌ U
[Equation]
M ⇌ D
• MAG is a subset of Microsoft Academic Graph.
It contains four object types: Paper ( P ), Author( A ), Institution ( I ) and Field ( F ), and eight relations between them.
Paper objects are labeled asfour classes according to their published venues: IEEE Journal of Photovoltaics, Astrophysics, LowTemperature Physics, and Journal of Applied Meteorology and Climatology.
• ACM is extracted from ACM digital library.
It contains three object types: Paper ( P ), Author ( A )and Subject ( S ), and four relations between them.
Paper objects are divided into three classes: DataMining, Database, and Computer Network.• DBLP is extracted from DBLP bibliography.
It contains four object types:
Author ( A ), Paper ( P ),Conference ( C ) and Term ( T ), and six relations between them.
Author ( A ) objects are labeledaccording to their four research areas: Data Mining, Information Retrieval, Database, and ArtificialIntelligence.• IMDB is extracted from the online movie rating website IMDB.
It contains four object types:Movie ( M ), Actor ( A ), User ( U ) and Director ( D ), and six relations between them.
Movie ( M )objects are categorized into four classes according to their genres: Comedy, Documentary, Drama,and Horror.
5.2 Baselines
We compare our SHGP with seven HIN-oriented baselines, including two semi-supervised methods,and five unsupervised (self-supervised) methods:
• Semi-supervised : HAN [32], and ie-HGCN (abbreviated as HGCN)
• Unsupervised : metapath2vec (abbreviated as M2V) [5], DMGI [18], HDGI [23], HeCo [33], andHGCN-DC (abbreviated as H-DC).
Here, HAN and HGCN are semi-supervised HGNN methods.
M2V is a traditional unsupervised HINembedding method.
DMGI, HDGI, and HeCo are state-of-the-art SSL methods on HINs.
Note that,in the experiments, we adopt the baseline HGCN’s encoder as the base encoder of our Att-HGNN.
H-DC is a variant of our SHGP.
It also uses HGCN as the base encoder, but unlike SHGP whichperforms structural clustering in the graph space, H-DC uses K -means to perform clustering in theembedding space, like DeepCluster [2].
We can use baselines HGCN and H-DC to investigate theeffectiveness of the proposed self-supervised pre-training scheme based on structural clustering.
5.3 Implementation details
For the proposed SHGP, in all the experiments, we use two HGCN layers as the Att-HGNN encoder,and search the dimensionalities of the hidden layers in the set {64, 128, 256, 512}.
All the modelparameters are initialized by the Xavier uniform distribution [6], and they are optimized throughthe Adam optimizer.
The learning rate and weight decay are searched from 1e-4 to 1e-2.
Forthe number of warm-up epochs, we search its best value in the set {5, 10, 20, 30, 40, 50}.
Wepre-train SHGP with up to 100 epochs and select the model with the lowest validation loss as thepre-trained model.
Then, we freeze the model and transfer the learned object embeddings to variousdownstream tasks.
For baselines, we reproduce their experimental results on our datasets.
Theirhyper-parameters are searched based on their papers and their official source codes.
Some of thebaseline methods [32, 5, 18, 33] need users to input several meta-paths, which are specified in
Appendix A.1 in detail.
For all the methods, we randomly repeat all the evaluation tasks for ten timesand report the average results.
All the experiments are conducted on an NVIDIA GTX 1080Ti GPU.
[Table]
DBLP
Mic-F 4% 90.
Mac-F 4% 90.
IMDB
Mic-F 4% 56.
Mac-F 4% 39.
5.4 Object classification
We conduct object classification on the object embeddings learned by all the semi-supervised baselinesand unsupervised baselines.
On each dataset, for the objects that have ground-truth labels, werandomly select { 4% , 6% , 8% } objects as the training set.
The others are divided equally as thevalidation set and the test set.
For all the unsupervised methods, they don’t use any class labelsduring learning object embeddings.
After finishing the pre-training, the output object embeddingsand their corresponding labels are used to train a linear logistic regression classifier.
For all thesemi-supervised methods, we directly report the classification results output by their own classifiers.
We adopt Micro-F1 and Macro-F1 as evaluation metrics.
The results are reported in Table 2.
We can see that our proposed SHGP achieves the best overall performance, even exceeding severalsemi-supervised learning methods, indicating its superior effectiveness.
On MAG, SHGP achievesvery high performance, i.e., over 98% Micro-F1/Macro-F1 scores, which are close to saturation.
Recall that SHGP adopts HGCN as the base encoder, and here, SHGP achieves better performancethan HGCN.
This indicates the effectiveness of the proposed strategy, i.e., pre-training HGNNs in aself-supervised manner based on structural clustering.
SHGP also performs better than H-DC.
Thisis reasonable because H-DC uses K -means to perform clustering in the embedding space to obtainpseudo-labels, and thus it cannot effectively exploit the structural information which naturally residesin the graph space.
Most unsupervised pre-training methods outperform traditional semi-supervised methods.
Thisverifies the superiority of the recent advances in the self-supervised pre-training paradigm.
Amongthe unsupervised methods, M2V performs much worse than the others.
This is because M2V canonly exploit a single meta-path, unlike DMGI, HDGI and HeCo which can well fuse the informationconveyed by multiple meta-paths through regularization or attention mechanism.
Among the semisupervised methods, HGCN outperforms HAN, probably because HAN can only exploit user-specifiedmeta-paths while HGCN can automatically discover and exploit the most useful meta-paths [38].
5.5 Object clustering
We evaluate the embeddings through object clustering.
In this task, we only consider the unsupervisedmethods, and the semi-supervised methods are excluded because they have exploited class labelsduring learning embeddings.
On each dataset, we use K -means to cluster the embeddings of thelabeled objects, and report normalized mutual information (NMI) and adjusted rand index (ARI) toquantitatively assess the clustering quality.
[Table]
As shown in Table 3, our SHGP achieves the best overall performance in this task.
Especially,on MAG, SHGP outperforms other baselines by a large margin.
This demonstrates its superiorcompetitiveness against other baselines in terms of learning discriminative embeddings.
On IMDB, allthe methods have shown limited performance.
This may be because the class labels and the structuralinformation on this dataset are not significantly correlated.
M2V shows the worst performance again,which may be due to its limitation as we analyzed in the previous experiment.
5.6 Visualization
For a more intuitive comparison, on MAG, we visualize the paper objects’ embeddings pre-trainedby three unsupervised methods, i.e., DMGI, HeCo, and our SHGP.
The embeddings are further embedded into the 2-dimensional Euclidean space by the t-SNE algorithm [17], and they are coloredaccording to their ground-truth labels.
[Figure]
The results are shown in Figure 2.
We can see that DMGI shows blurred boundaries between differentclasses.
HeCo performs relatively better than DMGI, but the green (left) class objects and theblue (bottom) class objects are still mixed to some extent.
Our SHGP shows the best within-classcompactness and the clearest between-class boundaries, which demonstrates its superior effectiveness.
5.7 Hyper-parameter study
In this subsection, we investigate the sensitivity of the warm-up epochs, which is the main hyperparameter that we have introduced in Section 4.1.
Specifically, on all the datasets, we explore thechange of object classification performance as the number of warm-up epochs gradually increases.
The results are shown in Figure 3.
[Figure]
As shown, overall, this hyper-parameter is not very sensitive.
The general pattern is that the performance increases first, and then declines gradually.
This is because, at the beginning, the modelhas not made full use of the information contained in the initial pseudo-labels yet.
After that, themodel may gradually overfit these initial pseudo-labels, which prevents the model from continuouslyimproving its performance in the iterations.
The overall inflection points are 10, 20, 20, and 20 for
MAG, ACM, DBLP, and IMDB respectively, which are the default values in the other experiments.
Conclusion
In this paper, we propose a novel self-supervised pre-training method on HINs, named SHGP.
Itconsists of two key modules which share the same attention-aggregation scheme.
The two modulesare able to utilize and enhance each other, promoting the model to effectively learn informativeembeddings.
Different from existing SSL methods on HINs, our SHGP does not require any positiveexamples or negative examples, thereby enjoying a high degree of flexibility and ease of use.
Weconduct extensive experiments to demonstrate the superior effectiveness of SHGP against state-ofthe-art baselines.
Checklist
For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper’s
contributions and scope?
[Yes] See Section 1.
(b) Did you describe the limitations of your work?
[Yes] See Section 4.3.4.
(c) Did you discuss any potential negative societal impacts of your work?
[Yes] SeeAppendix A.5.
(d) Have you read the ethics review guidelines and ensured that your paper conforms tothem?
[Yes] We have read and ensured.2.
If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results?
[N/A]
(b) Did you include complete proofs of all theoretical results?
[N/A]
If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main experi-
mental results (either in the supplemental material or as a URL)?
[Yes] See Section 5and the supplemental material.
(b) Did you specify all the training details (e.g., data splits, hyperparameters,
how they
were chosen)?
[Yes] See Section 5 and the supplemental material.
(c) Did you report error bars (e
g., with respect to the random seed after running experi-
ments multiple times)?
[Yes] See Appendix A.3.
(d) Did you include the total amount of compute and the type of resources used (e.g., typeof GPUs, internal cluster, or cloud provider)?
[Yes] See Section 5.3.
If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators?
[Yes] We have cited the
creators.
(b) Did you mention the license of the assets?
[No] We only used publicly available assets.
(c) Did you include any new assets either in the supplemental material or as a URL?
[Yes]
We have included our source codes in the supplemental material.
(d) Did you discuss whether and how consent was obtained from people whose data you’re
using/curating?
[No] We only used publicly available assets.
(e) Did you discuss whether the data you are using/curating contains personally identifiable
information or offensive content?
[No] Our datasets are public and do not contain
personally identifiable information or offensive content.
If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, if
applicable?
[N/A]
(b) Did you describe any potential participant risks, with links to Institutional Review
Board (IRB) approvals, if applicable?
[N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount
spent on participant compensation?
[N/A]