Self-supervised Heterogeneous Graph Pre-training Based on Structural Clustering
Abstract
Recent self-supervised pre-training methods on Heterogeneous Information Networks (HINs) have shown promising competitiveness over traditional semisupervised Heterogeneous Graph Neural Networks (HGNNs).
Unfortunately, their performance heavily depends on careful customization of various strategies for generating high-quality positive examples and negative examples, which notably limits their flexibility and generalization ability.
In this work, we present SHGP, a novel
Self-supervised Heterogeneous Graph Pre-training approach, which does not need to generate any positive examples or negative examples.
It consists of two modules that share the same attention-aggregation scheme.
In each iteration, the Att-LPA module produces pseudo-labels through structural clustering, which serve as the self-supervision signals to guide the Att-HGNN module to learn object embeddings and attention coefficients.
The two modules can effectively utilize and enhance each other, promoting the model to learn discriminative embeddings.
Extensive experiments on four real-world datasets demonstrate the superior effectiveness of SHGP against state-of-the-art unsupervised baselines and even semi-supervised baselines.
We release our source code at: https://github.com/kepsail/SHGP .
Introduction
Over the past few years, various semi-supervised graph neural networks (GNNs) have been proposed to learn graph embeddings.
They have achieved remarkable success in many graph analytic tasks.
This success, however, comes at the cost of a heavy reliance on high-quality supervision labels.
In real-world scenarios, labels are usually expensive to acquire, and sometimes even impossible due to privacy concerns.
To relieve the label scarcity issue in (semi-) supervised learning and take full advantage of a large amount of easily available unlabeled data, the self-supervised learning (SSL) paradigm has recently drawn considerable research interest in the computer vision research community.
It leverages the supervision signal from the data itself to learn generalizable embeddings, which are then transferred to various downstream tasks with only a few task-specific labels.
One of the most common SSL paradigms is contrastive learning, which learns representations by estimating and maximizing the mutual information between the input and the output of a deep neural network encoder [8].
For graphs, some recent graph contrastive learning methods [29, 7, 41, 25, 20, 19, 43, 36, 40] have shown promising competitiveness compared with semi-supervised GNNs.
They usually require three typical steps: (1) constructing positive examples (semantically correlated structural instances) by strategies such as node dropping, edge perturbation, and negative examples (uncorrelated instances) by strategies such as feature shuffling, mini-batch sampling; (2) encoding these examples through graph encoders such as GCN [16]; (3) maximizing/minimizing the similarity between these positive/negative examples.
Nevertheless, in the real world, graphs often contain multiple types of objects and multiple types of relationships between them, which are called heterogeneous graphs, or heterogeneous information networks (HINs) [27].
Due to the challenges caused by the heterogeneity, existing SSL methods on homogeneous graphs cannot be straightforwardly applied to HINs.
Very recently, several works have made some efforts to conduct SSL on HINs [33, 23, 18, 15, 13, 14, 37, 10].
In comparison with SSL methods on homogeneous graphs, the key difference is that they usually have different example generation strategies, so as to capture the heterogeneous structural properties in HINs.
The strategies of generating high-quality positive/negative examples are critical to the performance of existing methods [34, 4, 41, 36].
Unfortunately, whether for homogeneous graphs or heterogeneous graphs, the example generation strategies are dataset-specific, and may not be applicable to all scenarios.
This is because real-world graphs are abstractions of things from various domains, e.g., social networks, citation networks, etc.
They usually have significantly different structural properties and semantics.
Previous works have systematically studied this and found that different strategies are good at capturing different structural semantics.
For example, study [41] observed that edge perturbation benefits social networks but hurts some biochemical networks, and study [36] observed that negative examples benefit sparser graphs.
Consequently, in practice, the example generation strategies have to be empirically constructed and investigated through either trial-and-error or rules of thumb.
This significantly limits the practicality and general applicability of existing methods.
In this work, we focus on HINs which are more challenging, and propose a novel SSL approach, named SHGP.
Different from existing methods, SHGP requires neither positive examples nor negative examples , thus circumventing the above issues.
Specifically, SHGP adopts any HGNN model that is based on attention-aggregation scheme as the base encoder, which is termed as the module Att-HGNN.
The attention coefficients in Att-HGNN are particularly used to combine with the structural clustering method LPA (label propagation algorithm)
[21], as the module Att-LPA.
Through performing structural clustering on HINs, Att-LPA is able to produce clustering labels, which are treated as pseudo-labels.
In turn, these pseudo-labels serve as guidance signals to help Att-HGNN learn better embeddings as well as better attention coefficients.
Thus, the two modules are able to exploit and enhance each other, finally leading the model to learn discriminative and informative embeddings.
In summary, we have three main contributions as follows:
• We propose a novel SSL method on HINs, SHGP.
It innovatively consists of the Att-LPA module and the Att-HGNN module.
The two modules can effectively enhance each other, facilitating the model to learn effective embeddings.
• To the best of our knowledge, SHGP is the first attempt to perform SSL on HINs without any positive or negative examples.
Therefore, it can directly avoid the laborious investigation of example generation strategies, improving the model’s generalization ability and flexibility.
• We transfer the object embeddings learned by SHGP to various downstream tasks.
The experimental results show that SHGP can outperform state-of-the-art baselines, even including some semisupervised baselines, demonstrating its superior effectiveness.
Related work
SSL on HINs.
There are several existing methods [33, 23, 18, 15, 37, 13, 14, 10] that conduct
SSL on HINs.
Determined by their contrastive loss functions, all these methods require high-quality positive and negative examples to effectively learn embeddings.
Thus, their effectiveness and performance hinge on the specific strategies of generating positive examples and negative examples, which limits their flexibility and generalization ability.
SSL on homogeneous graphs.
Existing SSL methods on homogeneous graphs [29, 7, 41, 25, 20, 19, 43, 36, 35, 30] also need to generate sufficient positive and negative examples to effectively perform graph contrastive learning.
They only handle homogeneous graphs and cannot be easily applied to HINs.
In this work, we seek to perform SSL on HINs without any positive examples or negative examples.
GNN+LPA methods.
There exist several methods [31, 1, 24] that combine LPA [21] with GNNs.
However, they are all supervised learning methods, and only deal with homogeneous graphs.
In this work, we study SSL on HINs.
Others.
DeepCluster [2] uses K -means to perform clustering in the embedding space.
Differently, our SHGP directly performs structural clustering in the graph space.
JOAO [40] explores the automatic selection of positive example generation strategies, which is still not fully automatic.
HuBERT [9] is an SSL approach for speech representation learning.
GIANT [3] leverages graphstructured self-supervision to extract numerical node features from raw data.
MARU [12] learns object embeddings by exploiting meta-contexts in random walks.
Different from them, in this work, we study how to effectively conduct SSL on HINs.
Graph pooling methods e.g. [22, 39] learn soft cluster assignment to coarsen graph topology in each model layer.
Differently, our method propagates integer (hard) cluster labels in each layer to perform structural clustering.
Preliminaries
We first briefly introduce some concepts about HINs, and then formally describe the problem we study in this paper.
Heterogeneous Information Network.
An HIN is defined as: G = ( V , E , A , R , ϕ, ψ ) , where V is the set of objects, E is the set of links, ϕ : V → A and ψ : E → R are respectively the object type mapping function and the link type mapping function, A denotes the set of object types, and R denotes the set of relations (link types), where |A| + |R| > 2 .
Let X
=
X 1 , ..., X |A| } denote the set containing all the feature matrices associated with each type of objects.
A meta-path P of length l is defined in the form of A 1 R 1 −−→
A 2 R 2 −−→ · · · R l −→ A l +1 (abbreviated as A 1 A 2 · · · A l +1 ), which describes a composite relation R = R 1 ◦ R 2 ◦ · · · ◦ R l between object types
A 1 and A l +1 , where ◦ denotes the composition operator on relations.
We show a toy HIN in the left part of Figure 1.
It contains four object types: “Paper” ( P ), “Author” ( A ), “Conference” ( C ) and “Term” ( T ), and three relations: “Publish” between P and C , “Write” between P and A , and “Contain” between P and T .
APC is a meta-path of length two, and a 1 p 2 cis such a path instance, which means that author a 1 has published paper p 2 in conference c 2 .
SSL on HINs.
Given an HIN G , the problem is to learn an embedding vector h i ∈
R d for each object i ∈ V , in a self-supervised manner, i.e., without using any task-specific labels.
The pre-trained embeddings are expected to capture the general-purpose information contained in G , and can be easily transferred to various unseen downstream tasks with only a few task-specific labels.
[Figure]
[Equation]
[Figure]
[Equation]
[Figure]
[Equation]
[Figure]
[Equation]
[Figure]
[Equation]
[Figure]
Methodology
In this section, we present the proposed method SHGP, which consists of two key modules.
The AttHGNN module is instantiated as any HGNN model that is based on attention-aggregation scheme.
The
Att-LPA module combines the structural clustering method LPA [21] with the attention-aggregation scheme used in Att-HGNN.
The overall model architecture is shown in Figure 1 and explained in the figure caption.
In the following, we describe the procedure of SHGP in detail.
4.1 Initialization
At the beginning, we randomly initialize all the model parameters in Att-HGNN by the Xavier uniform distribution [6].
To obtain initial pseudo-labels, we use the original LPA [21] to perform a thorough structural clustering on the input HIN G .
Specifically, LPA randomly associates each object with a unique integer as its initial label, and lets them iteratively propagate along the links in G .
In each iteration, each object updates its label to the label that appears most frequently in its neighborhood.
After convergence, the final label indicates the cluster to which each object belongs.
We treat these clustering labels returned by LPA as the initial pseudo-labels, and re-organize them as a one-hot label matrix Y [0] ∈ R |V|× K , where K denotes the cluster size, which is not a hyper-parameter but depends on the uniqueness of these labels.
Thus, in the subsequent steps, the propagation of the pseudo-labels can be easily implemented via the matrix multiplication operation.
Under the guidance of these obtained initial pseudo-labels, we first train Att-HGNN for several epochs as the model “warm-up”, to learn the initial meaningful attention coefficients as well as other model parameters.
We use W [0] to denote all these initial parameters.
Then, we proceed with the following iterations.
4.2 Iteration
In the t -th iteration, we compute the object embeddings H [ t ] by the Att-HGNN module.
Att-HGNN is parameterized by W [ t − 1] , and its inputs include: the HIN topology G , the object features X .
This is formulated as follows:
[Equation]
Meanwhile, we update the pseudo-labels by the creatively proposed Att-LPA module.
In comparison with Att-HGNN, Att-LPA does not input X , but instead inputs Y [ t − 1] , i.e., the pseudo-labels of the previous iteration.
This is formulated as follows:
[Equation]
Att-HGNN and Att-LPA perform one forward pass of attention-based aggregation in the same way.
The only difference between them is that Att-HGNN aggregates the (projected) features of neighbors while Att-LPA aggregates the pseudo-labels of neighbors produced in the previous iteration, both weighted by exactly the same attention coefficients.
Now, we input H [ t ] into a softmax classifier which is built on the top layer of the Att-HGNN to make predictions P [ t ] .
The loss is computed as the cross-entropy between the predictions P [ t ] and the pseudo-labels Y [ t ] , as follows:
[Equation]
where C [ t − 1] ⊂ W [ t − 1] denotes the parameter matrix of the classifier.
Having the loss, we can then optimize all the model parameters by gradient descent, as follows:
[Equation]
where η denotes the learning rate.
As the optimization proceeds, the model will learn better model parameters (including attention coefficients).
This leads Att-HGNN and Att-LPA to respectively produce better embeddings (and predictions) and better pseudo-labels in the next iteration, which, in turn, promotes the model to learn further better parameters.
Thus, the two processes are able to closely interact with each other, as well as enhance each other, finally resulting in discriminative and informative embeddings.
The overall procedure is shown in Algorithm 1.
4.3 Model discussion 4.3.1 Encoder of Att-HGNN
The Att-HGNN module can be specifically instantiated as any attention-based HGNN encoders.
Existing possible choices include: HAN [32], HGT [11], GTN [42], and ie-HGCN [38].
Among them, our previously proposed ie-HGCN [38] is simple and efficient.
It has shown superior performance over the other three models.
Therefore, in this work, we adopt it as the base encoder of Att-HGNN, which is in detail described as follows.
Let N i denote the set of object i ’s neighbors.
Specially, object i itself is also added to N i .
Accordingly, an edge ( i, i ) is added to E , and a dummy self-relation ψ ( i, i ) is added to R .
In each model layer, object i ’s new representation ⃗h ′
i is computed as follows:
[Equation] [Equation]
where σ is the non-linear activation function, and ⃗h j is neighbor j ’s current representation.
W ψ ( i,j ) is the projection parameter matrix, (cid:98) a ψ ( i,j ) i,j is the normalized link weight ( (cid:98) a ψ ( i,i ) i,i = 1 ), and β ψ ( i,j ) i is the normalized attention coefficient, all of which are specific to relation ψ ( i, j ) .
With Att-HGNN, we can formulate our Att-LPA in a similar way.
Specifically, in each layer, object i ’s pseudo-label is updated as follows:
[Equation]
where ⃗y j is neighbor j ’s pseudo-label vector in the one-hot form.
Object i first aggregates its neighbors’ pseudo-labels according to the same normalized link weights and attention coefficients as in Eq.
Then, its new pseudo-label vector ⃗y i ′ is obtained through the argmax operator and the one-hot encoding function.
The above two equations describe the computational details of one model layer, where the iteration indices and the layer indices are omitted for notation brevity.
Here, we can further use the superscript [ x, y ] to index a symbol with respect to x -th iteration and y -th layer.
Assume the model has N layers.
In the t -th iteration, for the input layer of Att-HGNN, we set: ⃗ h [ t, 0] i = ⃗x i (i.e., the object feature vector), while for Att-LPA, we set: ⃗y [ t, 0] i = ⃗y
[ t − 1 ,N ] i .
At the end of the t -th iteration, we properly organize all the ⃗h [ t,N ] i and ⃗y [ t,N ] i as H [ t ] in Eq.
(1) and Y [ t ] in Eq.
(2) respectively.
In this way, based on the ie-HGCN encoder, we establish our Att-HGNN module and Att-LPA module.
4.3.2 Time complexity
As analyzed in the literature, both LPA [21] and ie-HGCN [38] have quasi-linear time complexity.
Therefore, our SHGP also has quasi-linear time complexity, i.e., O ( |V| + |E| ) .
Please see Appendix
A.2 for the efficiency study of SHGP.
4.3.3 Label consistency
For SHGP, in the initialization phase, we perform the thorough LPA process until convergence to obtain the initial pseudo-labels, and warm-up Att-HGNN module to learn initial meaningful attention coefficients.
In each subsequent iteration, we no longer need to perform LPA from scratch, but instead perform one forward pass of Att-LPA.
Therefore, the clustering results are basically consistent with the results of the previous iteration.
Thus, our SHGP does not need to perform alignment between clustering labels and predictions.
This can avoid the issue in DeepCluster [2] that there is no correspondence between two consecutive clustering assignments, and can also avoid the alignment process in M3S [26].
4.3.4 Class space
In this work, we cluster all types of objects into one common class space.
We note that in most existing
HIN datasets, various types of objects can indeed share the same class space.
They typically show a “star” network schema [28], since they are usually constructed according to one “hub” type of objects.
The other types of objects are actually the attributes of the “hub” objects.
For example, on DBLP, the “hub” objects are paper ( P ) objects.
These papers and their associated authors, conferences, and etc.
can all be categorized into four research areas: DM, IR, DB, and AI.
In the following extensive experiments, we demonstrate that this strategy works well on existing widely used HIN benchmark datasets.
In future work, we will further investigate the problem of clustering different types of objects into different class spaces.
Experiments
In this section, we verify the generalization ability of the proposed SHGP by transferring the pre-trained object embeddings to various downstream tasks including object classification, object clustering, and embedding visualization.
5.1 Datasets
In the experiments, we use four publicly available HIN benchmark datasets, which are widely used in previous related works [38, 32, 18, 23, 33].
Their statistics are summarized in Table 1.
Please see
Appendix A.1 for more details of these datasets.
[Table] [Table]
[Table] [Table]
P ,
[Table]
F , P ⇌ A , A ⇌
I
[Table]
P (4025), A (7167), S
[Table]
P ⇌ A , P ⇌ S
[Table]
A (4057), P (14328), C (20), T (8898)
P ⇌
[Table]
• MAG is a subset of Microsoft Academic Graph.
It contains four object types: Paper ( P ), Author ( A ), Institution ( I ) and Field ( F ), and eight relations between them.
Paper objects are labeled as four classes according to their published venues: IEEE Journal of Photovoltaics, Astrophysics, Low Temperature Physics, and Journal of Applied Meteorology and Climatology.
• ACM is extracted from ACM digital library.
It contains three object types: Paper ( P ), Author ( A ) and Subject ( S ), and four relations between them.
Paper objects are divided into three classes: Data Mining, Database, and Computer Network.
• DBLP is extracted from DBLP bibliography.
It contains four object types:
Author ( A ), Paper ( P ), Conference ( C ) and Term ( T ), and six relations between them.
Author ( A ) objects are labeled according to their four research areas: Data Mining, Information Retrieval, Database, and Artificial Intelligence.
• IMDB is extracted from the online movie rating website IMDB.
It contains four object types: Movie ( M ), Actor ( A ), User ( U ) and Director ( D ), and six relations between them.
Movie ( M ) objects are categorized into four classes according to their genres: Comedy, Documentary, Drama, and Horror.
5.2 Baselines
We compare our SHGP with seven HIN-oriented baselines, including two semi-supervised methods, and five unsupervised (self-supervised) methods:
• Semi-supervised : HAN [32], and ie-HGCN (abbreviated as HGCN)
Unsupervised : metapath2vec (abbreviated as M2V) [5], DMGI [18], HDGI [23], HeCo [33], and HGCN-DC (abbreviated as H-DC).
Here, HAN and HGCN are semi-supervised HGNN methods.
M2V is a traditional unsupervised HIN embedding method.
DMGI, HDGI, and HeCo are state-of-the-art SSL methods on HINs.
Note that, in the experiments, we adopt the baseline HGCN’s encoder as the base encoder of our Att-HGNN.
H-DC is a variant of our SHGP.
It also uses HGCN as the base encoder, but unlike SHGP which performs structural clustering in the graph space, H-DC uses K -means to perform clustering in the embedding space, like DeepCluster [2].
We can use baselines HGCN and H-DC to investigate the effectiveness of the proposed self-supervised pre-training scheme based on structural clustering.
5.3 Implementation details
For the proposed SHGP, in all the experiments, we use two HGCN layers as the Att-HGNN encoder, and search the dimensionalities of the hidden layers in the set {64, 128, 256, 512}.
All the model parameters are initialized by the Xavier uniform distribution [6], and they are optimized through the Adam optimizer.
The learning rate and weight decay are searched from 1e-4 to 1e-2.
For the number of warm-up epochs, we search its best value in the set {5, 10, 20, 30, 40, 50}.
We pre-train SHGP with up to 100 epochs and select the model with the lowest validation loss as the pre-trained model.
Then, we freeze the model and transfer the learned object embeddings to various downstream tasks.
For baselines, we reproduce their experimental results on our datasets.
Their hyper-parameters are searched based on their papers and their official source codes.
Some of the baseline methods [32, 5, 18, 33] need users to input several meta-paths, which are specified in
Appendix A.1 in detail.
For all the methods, we randomly repeat all the evaluation tasks for ten times and report the average results.
All the experiments are conducted on an NVIDIA GTX 1080Ti GPU.
[Table]
DBLP
Mic-F 4% 90.
Mac-F 4% 90.
IMDB
Mic-F 4% 56.
Mac-F 4% 39.
5.4 Object classification
We conduct object classification on the object embeddings learned by all the semi-supervised baselines and unsupervised baselines.
On each dataset, for the objects that have ground-truth labels, we randomly select { 4% , 6% , 8% } objects as the training set.
The others are divided equally as the validation set and the test set.
For all the unsupervised methods, they don’t use any class labels during learning object embeddings.
After finishing the pre-training, the output object embeddings and their corresponding labels are used to train a linear logistic regression classifier.
For all the semi-supervised methods, we directly report the classification results output by their own classifiers.
We adopt Micro-F1 and Macro-F1 as evaluation metrics.
The results are reported in Table 2.
We can see that our proposed SHGP achieves the best overall performance, even exceeding several semi-supervised learning methods, indicating its superior effectiveness.
On MAG, SHGP achieves very high performance, i.e., over 98% Micro-F1/Macro-F1 scores, which are close to saturation.
Recall that SHGP adopts HGCN as the base encoder, and here, SHGP achieves better performance than HGCN.
This indicates the effectiveness of the proposed strategy, i.e., pre-training HGNNs in a self-supervised manner based on structural clustering.
SHGP also performs better than H-DC.
This is reasonable because H-DC uses K -means to perform clustering in the embedding space to obtain pseudo-labels, and thus it cannot effectively exploit the structural information which naturally resides in the graph space.
Most unsupervised pre-training methods outperform traditional semi-supervised methods.
This verifies the superiority of the recent advances in the self-supervised pre-training paradigm.
Among the unsupervised methods, M2V performs much worse than the others.
This is because M2V can only exploit a single meta-path, unlike DMGI, HDGI and HeCo which can well fuse the information conveyed by multiple meta-paths through regularization or attention mechanism.
Among the semisupervised methods, HGCN outperforms HAN, probably because HAN can only exploit user-specified meta-paths while HGCN can automatically discover and exploit the most useful meta-paths [38].
5.5 Object clustering
We evaluate the embeddings through object clustering.
In this task, we only consider the unsupervised methods, and the semi-supervised methods are excluded because they have exploited class labels during learning embeddings.
On each dataset, we use K -means to cluster the embeddings of the labeled objects, and report normalized mutual information (NMI) and adjusted rand index (ARI) to quantitatively assess the clustering quality.
[Table]
As shown in Table
[Table]
our SHGP achieves the best overall performance in this task.
Especially, on MAG, SHGP outperforms other baselines by a large margin.
This demonstrates its superior competitiveness against other baselines in terms of learning discriminative embeddings.
On IMDB, all the methods have shown limited performance.
This may be because the class labels and the structural information on this dataset are not significantly correlated.
M2V shows the worst performance again, which may be due to its limitation as we analyzed in the previous experiment.
5.6 Visualization
For a more intuitive comparison, on MAG, we visualize the paper objects’ embeddings pre-trained by three unsupervised methods, i.e., DMGI, HeCo, and our SHGP.
The embeddings are further embedded into the 2-dimensional Euclidean space by the t-SNE algorithm [17], and they are colored according to their ground-truth labels.
[Figure]
The results are shown in Figure 2.
We can see that DMGI shows blurred boundaries between different classes.
HeCo performs relatively better than DMGI, but the green (left) class objects and the blue (bottom) class objects are still mixed to some extent.
Our SHGP shows the best within-class compactness and the clearest between-class boundaries, which demonstrates its superior effectiveness.
5.7 Hyper-parameter study
In this subsection
[Figure]
we investigate the sensitivity of the warm-up epochs, which is the main hyperparameter that we have introduced in Section 4.1.
Specifically, on all the datasets, we explore the change of object classification performance as the number of warm-up epochs gradually increases.
The results are shown in Figure 3.
[Figure]
As shown, overall, this hyper-parameter is not very sensitive.
The general pattern is that the performance increases first, and then declines gradually.
This is because, at the beginning, the model has not made full use of the information contained in the initial pseudo-labels yet.
After that, the model may gradually overfit these initial pseudo-labels, which prevents the model from continuously improving its performance in the iterations.
The overall inflection points are 10, 20, 20, and 20 for
MAG, ACM, DBLP, and IMDB respectively, which are the default values in the other experiments.
Conclusion
In this paper, we propose a novel self-supervised pre-training method on HINs, named SHGP.
It consists of two key modules which share the same attention-aggregation scheme.
The two modules are able to utilize and enhance each other, promoting the model to effectively learn informative embeddings.
Different from existing SSL methods on HINs, our SHGP does not require any positive examples or negative examples, thereby enjoying a high degree of flexibility and ease of use.
We conduct extensive experiments to demonstrate the superior effectiveness of SHGP against state-ofthe-art baselines.
Societal impact
Heterogeneous Information Networks (HINs) widely exist in our society, such as social networks, power networks, virus networks, etc.
In this work, we develop a method to learn the representations for objects in HINs.
The learned representations can be used for various analytical tasks such as object classification, object clustering, link prediction, etc.
We believe that our method contributes to our society in many aspects.
However, there is also some risk that our method could be abused illegally or unethically for some undesirable purposes.
We hope those bad things would not happen
Funding transparency statement
This work was supported by the National Natural Science Foundation of China under Grants 62133012, 61936006, 62103314, 62203354, 61876144, 61876145, 62073255, 61876138 and 62002255, the Key Research and Development Program of Shaanxi under Grant 2020ZDLGY04-07, and the Innovation Capability Support Program of Shaanxi under Grant 2021TD-05.
Checklist
For all authors
(a) Do the main claims made in the abstract and introduction accurately reflect the paper’s contributions and scope?
[Yes] See Section 1.
(b) Did you describe the limitations of your work?
[Yes] See Section 4.3.4.
(c) Did you discuss any potential negative societal impacts of your work?
[Yes] See the text after Section 6.
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them?
[Yes] We have read and ensured.
If you are including theoretical results... (a) Did you state the full set of assumptions of all theoretical results?
[N/A] (b) Did you include complete proofs of all theoretical results?
[N/A] 3.
If you ran experiments... (a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?
[Yes] See Abstract.
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)?
[Yes] See Section 5, the supplemental material, and our source code.
(c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?
[Yes] See Appendix A.3.
(d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)?
[Yes] See Section 5.3.
If you are using existing assets (e.g., code, data, models) or curating/releasing new assets... (a) If your work uses existing assets, did you cite the creators?
[Yes] We have cited the creators.
(b) Did you mention the license of the assets?
[No] We only used publicly available assets.
(c) Did you include any new assets either in the supplemental material or as a URL?
[Yes] We have released our source code.
Please see Abstract for the URL.
(d) Did you discuss whether and how consent was obtained from people whose data you’re using/curating?
[No] We only used publicly available assets.
(e) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content?
[No] Our datasets are public and do not contain personally identifiable information or offensive content.
If you used crowdsourcing or conducted research with human subjects... (a) Did you include the full text of instructions given to participants and screenshots, if applicable?
[N/A] (b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable?
[N/A] (c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation?
[N/A]
A Appendix A.1 More dataset details
On MAG and ACM, Paper (P) objects are associated with 128-dimensional real features.
On IMDB,
Movie (M) objects are associated with 14-dimensional real features.
On DBLP, all the objects have no real object features.
For the objects that are not associated with real features, following our previous work [38], we generate random features for them by the Xavier uniform distribution [6].
For some baseline methods [32, 5, 18, 33], they need users to input several useful meta-paths, which are specified in Table 4.
Our SHGP does not require meta-paths.
[Table]
Note that, although the datasets used in the papers of some baselines may have the same name as ours, they may not be identical.
Please refer to their dataset statistics.
For example, DBLP in [18] and DBLP in [33] are quite different, and IMDB in [18] and IMDB in [23] are also different.
We reproduce the experimental results of all the baselines on our datasets.
A.2 Efficiency of SHGP
We follow GCN [16] to show the efficiency of our SHGP.
Specifically, we construct eight HINs with different number of links, and use SHGP to pre-train each of them for 100 epochs on our GPU.
The actual total time cost is reported in Figure 4.
As shown, overall, the time cost increases roughly linearly with respect to the number of links, which demonstrates the high efficiency of our SHGP.
[Figure]
[Figure]
A.3 Impact of training ratio
To more intuitively observe the impact of training ratio on object classification performance.
In
Figure 5, we show the error bar of SHGP with respect to ten random repeats of object classification.
We can see, overall, the classification performance improves as the training ratio increases, slightly.
A.4 Convergence of SHGP
In Figure
[Figure]
we plot the pre-training loss curves of SHGP on the four benchmark datasets.
As we can see, our SHGP converges fast.
Specifically, it converges in about 20 iterations on ACM, 30 iterations on DBLP, and 80 iterations on MAG and IMDB.
[Figure]