Zero-shot Transfer Learning within a Heterogeneous Graph via Knowledge Transfer Networks
Anonymous Author(s)
AffiliationAddress email
Abstract
Data continuously emitted from industrial ecosystems such as social or commerceplatforms are commonly represented as heterogeneous graphs (HG) composedof multiple node/edge types.
State-of-the-art graph learning methods for HGsknown as heterogeneous graph neural networks (HGNNs) are applied to learndeep context-informed node representations.
However, many HG datasets fromindustrial applications suffer from label imbalance between node types.
As thereis no direct way to learn using labels rooted at different node types, HGNNs havebeen applied on only a few node types with abundant labels.
We propose a zeroshot transfer learning module for HGNNs called a Knowledge Transfer Network(KTN) that transfers knowledge from label-abundant node types to zero-labelednode types through rich relational information given in the HG.
KTN is derivedfrom the theoretical relationship, which we introduce in this work, between distinctfeature extractors for each node types given in a HGNN model.
KTN improvesperformance of 6 different types of HGNN models by up to 960% for inference onzero-labeled node types and outperforms state-of-the-art transfer learning baselinesby up to 73% across 18 different transfer learning tasks on HGs.
Introduction
Large technology companies commonly maintain large relational datasets, derived from their internallogs, that can be represented as or joined into a massive heterogeneous graph (HG) composed of nodes and edges with multiple types [30].
For instance, in e-commerce networks, there are product,user and review nodes, all interconnected by many edge types that represent forms of interactionssuch as spending (user-product), reviewing (user-review) and reviews-of (product-review).
To learnpowerful features representing the complex multimodal structure of HGs, various heterogeneousgraph neural networks (HGNN) have been proposed [15, 26, 35, 43]
A common issue in these industrial applications of HGNNs is the label imbalance among differentnode types.
For instance, publicly available content nodes – such as those representing video, text,and image content – are abundantly labelled, whereas labels for other types (such as user or accountnodes) may be much more expensive to collect (or even not available, e.g. due to privacy restrictions).
This means that in most standard training settings, HGNN models can only learn to make goodinferences for a few label-abundant node types, and can usually not make any inferences for theremaining node types, given the absence of any labels for them.
If there is a pair of label-abundant and zero-labeled node types which share an inference task, couldwe transfer knowledge between them?
One body of work has focused on transferring knowledgebetween nodes of the same type from two different HGs (i.e., graph-to-graph transfer learning) [16, 40].
However, these approaches are not applicable in many real-world scenarios for three reasons
First,any external large-scale HG that could be used in a graph-to-graph transfer learning setting would almost surely be proprietary.
Second, even if practitioners could obtain access to an external industrial
HG, it is unlikely the distribution of that (source) graph would match their target graph well enoughto apply transfer learning.
Finally, node types suffering label scarcity are likely to suffer the sameissue on other HGs (e.g. user nodes).
In this paper, we introduce a zero-shot transfer learning approach for a single HG (assumed to befully-owned by the practitioners), transferring knowledge from labelled to unlabelled node types.
This setting is distinct from any graph-to-graph transfer learning scenarios, since the source and targetdomains exist in the same HG dataset, and are assumed to have different node types.
Our modelutilizes the shared context between source and target node types; for instance, in the e-commercenetwork, the latent (unknown) labels of user nodes can be strongly correlated with spending/reviewingpatterns that are encoded in the cross-edges between user nodes and product/review nodes.
We proposea novel zero-shot transfer learning problem for this HG learning setting as follows:
Informal Problem Definition 1.
Zero-shot cross-type transfer learning running on a HG:
Given a heterogeneous graph G with node types { s , t , · · · } with abundant labels for source type s butno labels for target type t , can we train HGNNs to infer the labels of target-type nodes?
A naïve solution to this problem would be to re-use an HGNN pre-trained on the source nodes fortarget node inference, given that both domains exist in the same HG.
However, as we show in ourpaper, HGNNs have distinct parameter sets for each node type [15], edge type [26], and meta-pathtype [8, 35].
These facts cause HGNNs to learn entirely different feature extractors for nodes andedges of different types – in other words, the final embeddings for source and target nodes arecomputed by different sets of parameters in HGNNs.
Thus, a classifier pre-trained on source nodeswill fail to perform well on inference tasks for target nodes.
The field of domain adaptation (DA)targets this setting, seeking to transfer knowledge from a source domain with abundant labels to atarget domain which lacks them [9, 19, 20, 27].
However, distinct feature extractors across nodetypes in HGNNs break a standard assumption of DA setting, namely that source and target domainsshare the same feature extractors (e.g., CNNs for both source and target image domains).
As wedemonstrate in this paper, in our problem setting, DA approaches fail to achieve the outstandingperformance they are known for in computer vision and NLP.
In our work, we first dissect the gradient path of HGNN models to see how feature extractors aredesigned independently for each node type, and some empirical consequences.
Then we theoreticallyanalyze how feature extractors across node types relate to each other and how their output distributionscould be represented in terms of each other.
We model this theoretical relationship between twofeature extractors as a Knowledge Transfer Network (KTN) which can be optimized to transformtarget embeddings to fit the source domain distribution.
We perform an extensive evaluation of ourmethod on 18 different transfer learning tasks on HGs where we compare against state-of-the-artdomain adaptation baselines.
Additionally, in order to understand which environments are ideal fortransferring knowledge between different node types for HGs, we formulate a synthetic heterogeneousgraph generator that allows us to study the sensitivity of these methods.
Our main contributions are:
• Novel and practical problem definition:
To the best of our knowledge, KTN is the first zero-shotcross-type transfer learning method running on a heterogeneous graph — transfer knowledgeacross different node types within a heterogeneous graph.
• Generality: KTN is a principled approach analytically induced from the architecture of HGNNs,thus applicable to any HGNN models, showing up to 960% performance improvement for zerolabeled node inference across 6 different HGNN models.
• Effectiveness: We show that KTN outperforms state-of-the-art domain adaptation methods, beingup to 73 .
3% higher in MRR on 18 different transfer learning tasks on HGs.•
Sensitivity Analysis: We provide a HG generator model to analyze how the node attribute and
edge distributions of HGs affect the performance of KTN and other methods on the task.
Related Work
Various transfer learning problems have been defined on the graph domain.
[21, 22, 42, 38] constructsynthetic graphs from unstructured data and transfer knowledge over the graphs using GNNs.
Onthe other hand, [13, 14, 24, 37] focus on extracting knowledge from the existing graph structures.
They pretrain a GNN model on a source graph and re-use the model on a target graph.
While thosemethods focus on homogeneous graphs, [16, 40] transfer HGNNs across different HGs.
However, none of them can be directly applied to our cross-type transfer learning problem running on a single
HG.
Here we cover two classes of learning approaches that are related to our problem.
As HGNNsare the models to which our method can be applied, we cover them extensively in Section 3.
Zero-shot domain adaptation (DA) Approaches in the traditional zero-shot domain adaptationspace transfer knowledge from a source domain with abundant labels to a target domain which lacksthem.
Zero-shot DA can be categorized into three groups — MMD-based methods, adversarialmethods, and optimal-transport-based methods.
All three categories rely on two networks — afeature extractor network and a task classifier network.
Adversarial and OT-based methods use anadditional domain classifier network.
Due to the assumption that source and target domains have thesame modality 1 , the standard DA setting assumes identical feature extractors across domains.
Moredescriptions can be found in Appendix A.10.
Label propagation (LP) Label propagation approaches [e.g. 45] use message-passing to pass eachnodes’ labels to their neighbors according to normalized edge weights.
Similar to LP, embeddingpropagation (EP) recursively propagates source embeddings (computed using source labels) untilthey reach the target domain.
EP exploits both node attribute information and the node adjacencies,but only uses the source node embeddings.
Preliminaries
In this section we review heterogeneous graphs and heterogeneous graph neural networks (HGNNs).
3.1 Heterogeneous graph
Heterogeneous graphs (HGs) are an important abstraction for modeling the relational data of multimodal systems.
Formally, a heterogeneous graph is defined as G =
( V , E , T , R ) where the node set
V ; the edge set E consisting of ordered tuples e ij :=
( i, j ) with i, j ∈ V , where e ij ∈ E iff an edgeexists from i to j ; the set of node types T with associated map τ : V (cid:55)→ T ; the set of relation types
R with associated map ϕ : E (cid:55)→ R .
This flexible formulation allows directed, multi-type edges.
Weadditionally assume the existence of a node attribute vector x i ∈
X τ ( i ) for each i ∈ V , where X t isan attribute matrix specific to nodes of type t .
3.2 Heterogeneous Graph Neural Networks (HGNN)
Various HGNN models have been proposed [26, 35, 43, 15, 41].
Fully-specified HGNN modelshave specialized parameters for each node type [15], edge type [26], and meta-path type [8] to mosteffectively utilize the complex relationships encoded in the HG data structure.
In this paper, we use aflavor of HGNN known as a Heterogeneous Message-Passing Neural Network (HMPNN) as our basemodel on which to demonstrate KTN (though KTN can be implemented in almost any HGNN, aswe show in experiments in Section 6).
The HMPNN merely extends the standard MPNN [10] byspecializing all transformation and message matrices in each node/edge type.
With its generality,
HMPNN is itself a base model for RGCN [26] and HGT [15], and is also widely used as a default
HGNN model in popular GNN libraries (e.g., pyG [7], TF-GNN [6], DGL [34]).
In a HMPNN, for any node j , the embedding of node j at the l - th layer is obtained with the followinggeneric formulation:
[Equation]
(1) where E ( j ) = { ( i, j
) ∈
E : i, j ∈ V} denotes all the edges which connect (directionally) to j .
Theabove operations typically involve type-specific parameters to exploit the inherent multiplicity ofmodalities in heterogeneous graphs.
First, we define a linear Message function:
[Equation]
where M ( l ) r are the specific message passing parameters for each r ∈ R and each of L GNNlayers.
Then defining E r ( j ) as the set of edges of type r pointing to node j , the Aggregate function mean-pools messages by edge type, and concatenates:
[Equation]
Finally, the Transform function maps the message into a type-specific latent space:
[Equation]
By stacking L layers, HMPNN outputs highly contextualized final node representations, and the finalnode representations can be fed into another model to perform downstream heterogeneous networktasks, such as node classification or link prediction.
3.3 Problem definition
Using notations defined above, we formalize our novel transfer learning problem on HGs.
Problem Definition 1.
Zero-shot cross-type transfer learning running on a HG:
In a given heterogeneous graph G =
( V , E , T , R ) with node attributes X = ∪ t ∈T X t , assume nodetypes s and t share a classification task { ( i, y i ) : i ∈ V s , V t } .
During the training phase, using labels{ ( i, y i ) : i ∈ V s } only for source-type nodes, we train an HGNN model f : f ( G , X ) =
h i to get nodeembeddings
h i for all nodes i ∈ V and a classifier g : g ( h i )
=
ˆ y i to predict labels ˆ y i from the nodeembeddings
h i .
During the test phase, our task is to predict labels { ( j, y j ) : j ∈ V t } of target-typenodes (where none of labels of target-type nodes were used for training).
Cross-Type Feature Extractor Transformations in HGNNs
We define f t : G (cid:55)→ R d to be the “feature extractor" of a HGNN, which is composed of parametersparticipating to map input node attributes of type t into a shared feature space R d .
In this section,we derive a strict transformation between feature extractors in HMPNNs.
Specifically, for any twonodes i, j with types τ ( i ) =
s and τ ( j )
=
t , we derive an expression for f s in terms of f t , and usethat expression to inspire a trainable transfer learning module called KTN in the following section.
For simplicity, throughout this section we ignore the activation α ( · ) within the Transform functionin Equation (4).
4.1 Feature extractors in HMPNNs
We first reason intuitively about the differences between f s and f t when s ̸ = t , using a toy heterogeneous graph shown in Figure 1(a).
Consider nodes v 1 and v 2 , noticing that τ (1) ̸
= τ (2) .
Using
HMPNN’s equations (2)-(4) from Section 3.2, for any l ∈ { 0 , . . .
, L − 1 } we have
[Equation]
where h (0) j = x j .
From these equations, we see that h ( l ) 1 and h ( l ) 2 , which are features of differenttypes, are extracted using disjoint sets of model parameters at l -th layer.
In a 2-layer HMPNN,this creates unique gradient backpropagation paths between the two node types, as illustrated in
Figures 1(b)-1(c).
In other words, even though the same HMPNN is applied to node types s and t , thefeature extractors f s and f t have different computational paths.
Therefore they project node featuresinto different latent spaces, and have different update equations during training.
4.2 Empirical gap between f s and f t
Here we study the experimental consequences of the above observation via simulation.
We firstconstruct a synthetic graph extending the 2-type graph in Figure 1(a) to have multiple nodes per-type,and multiple classes.
To maximize the effects of having different feature extractors, we sample sourceand target nodes from the same feature distributions and each classes are well-separated in the boththe graph and feature space (more details available in Appendix A.8.1).
On such a well-aligned heterogeneous graph, without considering the observation in Section 4.1,there may seem to be no need for domain adaptation from f t to f s .
However, when we train the
[Figure] [Figure] HMPNN model solely on s -type nodes, as shown in Figure 2(a) we find the test accuracy for s -typenodes to be high ( 90% , blue line) and the test accuracy for t -type nodes to be quite low ( 25% , greenline).
Now if instead we make the t -type nodes use the source feature extractor f s , much moretransfer learning is possible ( ∼ 65% , orange line).
This shows that the different feature extractorspresent in the HMPNN model result in the significant performance drop, and simply matching inputdata distributions can not solve the problem.
To analyze this phenomenon at the level of backpropagation, in Figures 2(b)-2(c) we show themagnitude of gradients passed to parameters of source and target node types.
As illustrated in
Figures 1(b)-1(c)
, we find that the final-layer Transform parameter W (2) t for type- t nodes havezero gradients (Figure 2(b)), and similarly for the final-layer Message parameters (Figure 2(c)).
Additionally, those same parameters in the first-layer for t -type nodes have much smaller gradientsthan their s -type counterparts: W (1) t (blue line in Figure 2(b)), M (1) st and M (1) tt (blue and orange linesin Figure 2(c)) appear below than other lines.
This is because they contribute to f s less than f t
This case study shows that even when an HGNN is trained on a relatively simple, balanced, andclass-separated heterogeneous graph, a model trained only on the source domain node type cannottransfer to the target domain node type.
4.3 Relationship between feature extractors in HMPNNs
We show that a HMPNN model provides different feature extractors for each node type.
However,still, f s and f t are built inside one HMPNN model and interchange intermediate feature embeddingswith each other.
Both H ( L ) s and H ( L ) t (
the output of f s and f t ) are computed using the previouslayer’s intermediate embeddings H ( L − 1) s , H ( L − 1) t , and any other connected node type embeddings
H ( L − 1) x at the L -th HMPNN layer.
Therefore H ( L ) s and H ( L ) t can be mathematically presented byeach other using the ( L − 1) -th layer embeddings as connecting points, so do f s and f t .
Based on thisintuition, we derive a strict transformation between f s and f t , which will motivate the core domainadaptation component of our proposed KTN model. Require:
heterogeneous graph G =
( V , E , A , R ) , node feature matrices X , source node type s , target node type t , adjacency matrix A ts , source node label matrix Y s .
Ensure: HGNN f , classifier g , KTN t KTN 1: H ( L ) s , H ( L ) t = f ( G , H (0) = X ) 2: H ∗ t = t KTN ( H ( L ) t ) = A ts H ( L ) t T ts 3: L KTN =
(cid:13)(cid:13)(cid:13) H ( L ) s − H ∗ t (cid:13)(cid:13)(cid:13)4:
L = L CL ( g ( H ( L ) s ) , Y s )
+ λ L KTN 5:
Update f , g , t using ∇L
Require: pretrained HGNN f , classifier g , KTN t
KTN Ensure: target node label matrix Y t 1: H ( L ) t = f
( G , H (0) =
X ) 2: H ∗ t =
H ( L ) t T ts 3: return g ( H ∗ t )
Theorem 1.
Given a heterogeneous graph G = {V , E , T , R} .
For any layer l > 0 , define the set of( l − 1) - th layer HMPNN parameters as
[Equation]
Let A be the total n × n adjacency matrix.
Then for any s, t ∈ T there exist matrices A ∗ ts = a ts ( A )and Q ∗ ts
=
q ts ( Q ( l − 1) ) such that
[Equation]
(8) where a ts ( · ) and q ts ( · ) are matrix functions that depend only on s, t .
The full proof of Theorem 1 can be found in Appendix A.1.
Notice that in Equation 8, Q ∗ ts acts as amacro- Transform operator that maps H ( L ) t into the source domain, then A ∗ ts aggregates the mappedembeddings into s -type nodes.
In other words, Q ∗ ts acts as a mapping matrix from the target domainto the source domain.
To examine the implications, we run the same experiment as described in
Section 4.2, while this time mapping the target features H ( L ) t into the source domain by multiplyingwith Q ∗ ts in Equation 8 before passing over to a task classifier.
We see via the red line in Figure 2(a)that, with this mapping, the accuracy in the target domain becomes much closer to the accuracy in thesource domain ( ∼ 70% ).
Thus, we use this theoretical transformation as a foundation for our trainable
HGNN domain adaptation module, introduced in the following section.
KTN: Trainable Cross-Type Transfer Learning for HGNNs
Inspired by these derivations we introduce our primary contribution, Knowledge Transfer Networks .
We begin by noting Equation 8 in Theorem 1 has a similar form to a single-layer graph convolutionalnetwork [17] with a deterministic transformation matrix ( Q ∗ ts ) and a combination of adjacencymatrices directing from target node type t to source node type s ( A ∗ ts ).
Instead of hand-computingthe mapping function Q ∗ ts for arbitrary HGs and HGNNs (which would be intractable), we learn themapping function by modelling Equation 8 as a trainable graph convolutional network, named the
Knowledge Transfer Network, t KTN ( · ) .
KTN replaces Q ∗ ts and A ∗ ts in Equation 8 as follows:
[Equation]
where A ts is an adjacency matrix from node type t to s , and T ts is a trainable transformation matrix.
By minimizing L KTN , T ts is optimized to a mapping function of the target domain into the sourcedomain. [Table]
5.1 Algorithm
We minimize a classification loss L CL and a transfer loss L KTN jointly with regard to a HGNN modelf , a classifier g , and a knowledge transfer network t KTN as follows:
[Equation]
where λ is a hyperparameter regulating the effect of L KTN ; and f ( G , X ) s and f ( G , X ) t denote H ( L ) sand
H ( L ) t , respectively.
Algorithm 1 describes a training step on the source domain.
After computingthe node embeddings H ( L ) s and H ( L ) t , we map H ( L ) t to the source domain using t KTN and compute
L KTN .
Then, we update the models using gradients of L CL (computed using only source labels) and
L KTN .
Algorithm 2 describes the test phase on the target domain.
After we get node embeddings H ( L ) tfrom the trained HGNN model, we map H ( L ) t into the source domain using the trained transformationmatrix T ts .
Finally we pass the transformed target embeddings H ∗ t into the classifier which wastrained on the source domain.
Experiments6.1 Datasets
Open Academic Graph (OAG).
A dataset introduced in [44] composed of five types of nodes:papers (P), authors (A), institutions (I), venues (V), fields (F) and their corresponding relationships.
Paper and author nodes have text-based attributes, while institution, venue, and field nodes havetext- and graph structure-based attributes.
Paper, author, and venue nodes are labeled with researchfields in two hierarchical levels, L1 and L2.
We construct three field-specific subgraphs from OAG:computer science, computer networks, and machine learning academic graphs.
PubMed.
[39] A network composed of of four types of nodes: genes (G), diseases (D), chemicals(C), and species (S), and their corresponding relationships.
Gene and chemical nodes have graphstructure-based attributes, while disease and species nodes have text-based attributes.
Each gene ordisease is labeled with a set of diseases they belong to or cause.
Synthetic heterogeneous graphs.
We generate stochastic block models [1] with multiple node/edgetypes.
We label each node types with the same set of classes.
Then we control feature/edge distributions within/between node types by manipulating feature/edge signal-to-noise ratio within/betweenclasses.
A complete definition of the generative model is given in Appendix A.8.
6.2 Baselines
We compare KTN with two MMD-based DA methods (DAN [18], JAN [20]), three adversarial
DA methods (DANN [9], CDAN [19], CDAN-E [19]), one optimal transport-based method (WDGRL [27]), and two traditional graph mining methods (LP and EP [45]).
For KTN and DA methods,we use HMPNN as their base HGNN model.
More information of each method is in Appendix A.10. [Table] [Table]
6.3 Zero-shot transfer learning
We run 18 different zero-shot transfer learning tasks across three OAG and PubMed graphs.
Werun each experiment 3 times and report the average value.
Due to the space limitation, we reportthe standard deviations and results on OAG-computer networks and OAG-machine learning in
Appendix A.4.
Each heterogeneous graph has node classification tasks for both source and targetnode types.
During training, we are given 1) the heterogeneous graph structure information (i.e.,adjacency matrices), 2) input node attribute matrices for all node types, and 3) labels on source-typenodes for the classification task.
During the test phase, we predict labels on target-type nodes forthe same classification task.
The performance is evaluated by NDCG and MRR — widely adoptedranking metrics [14, 15].
In Tables 1 and 2, our proposed method KTN consistently outperforms all baselines on all tasks andgraphs by up to 73 .
3% higher in MRR (P-A(L1) task in OAG-CS, Table 1).
When we compare withthe base accuracy using the model pretrained on the source domain without any domain adaptation( 3 rd column, Base ), the results are even more impressive.
We see our method KTN provides relativegains of up to 340% higher MRR without using any labels from the target domain.
These results showthe clear effectiveness of KTN on zero-shot transfer learning tasks on a heterogeneous graph.
Wemention that venue and author node types are not directly connected in the OAG graphs (Figure 5(b)in Appendix), but KTN successfully transfer knowledge by passing the intermediate nodes.
Analysisfor the baseline performance can be found in Appendix A.5.
6.4 Generality of KTN
Here, we use 6 different HGNN models, R-GCN [26], HAN [35], HGT [15], MAGNN [8],
MPNN [10], and HMPNN.
MPNN maps all node types to the shared embedding space usingprojection matrices at the beginning then applies MPNN layers designed for homogeneous graphs.
In
Table 3, KTN improves accuracy on the target nodes across all HGNN models by up to 960% .
Thisshows the strong generality of KTN.
More results and analysis can be found in Appendix A.6.
6.5 Sensitivity analysis
Using our synthetic heterogeneous graph generator, we generate non-trivial 2-type heterogeneousgraphs to examine how the feature and edge distributions affect the performance of KTN and otherbaselines.
We generate a range of test-case scenarios by manipulating (1) signal-to-noise ratio σ e ofwithin-
For instance, in Figure 3, for each edge type ( s ↔ s , t ↔ t ,and s ↔ t , differentiated by colors), there are two differenttypes of edges, edges within the same class (plain line) andedges across different classes (dotted line).
For each edgetype, we manipulate σ e by changing the ratio of withinclass and cross-class edges, and σ f by diverging featuredistributions between classes.
Thus there will be 6 signalto-noise ratios in total.
A higher signal-to-noise ratio fora particular data dimension (edges vs features) across aparticular relationship r ∈ { s ↔ s, t ↔ t, s ↔ t } meansthat classes are more separable in that data dimension, when comparing within r , and hence easierfor HGNNs.
Note that while tuning one σ on the range [1 . 0 , 10 . 0] , the remaining five σ are held at10 . 0 .
Additionally, we vary σ across two scenarios: (I) “easy": source and target node types havesame number of clusters and same feature dimensions, (II) “hard" source and target node types havedifferent number of clusters and feature dimensions.
Figures 4(a) and 4(c) show results from changing σ e across the three relation types.
We see that
KTN is affected only by σ e across the s ↔ t relationship, which accords with our theory, since KTNexploits the between-type computation (adjacency) matrix.
Surprisingly, as seen in Figures 4(b) and4(d), we do not find a similar dependence of KTN on σ f , which shows that KTN is robust by learningpurely from edge homophily in the absence of feature homophily.
Regarding the performance ofother baselines, EP shows similar tendencies as KTN— only affected by cross-type σ e — because
EP also relies on cross-type propagation along edges.
However, its accuracy is bounded abovedue to the fact that it does not exploit the (unlabelled) target features.
DAN and DANN, which donot exploit cross-type edges, are not affected by cross-type σ e .
However, they show either low orunstable performance across different scenarios.
DAN shows especially poor performance in the“hard" scenarios (Figure 4(c) and 4(d)), failing to deal with different feature spaces for source andtarget domains. [Figure]
Conclusion
In this work, we proposed the first cross-type zero-shot transfer learning method for heterogeneousgraphs.
Our method, Knowledge Transfer Networks (KTN) for Heterogeneous Graph Neural
Networks, transfers knowledge from label-abundant node types to label-scarce node types.
Weillustrate KTN handily improves HGNN performances up to 960% for zero-labeled node types across6 different HGNN models and outperforms many challenging baselines up to 73% higher in MRR.
Future work in the area includes filtering noisy edges between source and target domains and making
KTN more robust and less dependent on structure of given noisy heterogeneous graphs.
Limitation Statement Our transfer learning method is limited to node types sharing the same task(i.e.
, the same classifier).
We plan to extend our work to transfer knowledge between different tasksrunning on different node types on a heterogeneous graph.
Societal Impact Statement
KTN allows organizations to learn better from their own graph data,leveraging its structure without requiring external information.
We believe this has a number ofpositive applications (preserving model quality without needing extra datasets).
However like allmodeling improvements, its true impact depends on what modeling tasks the technique is applied to.
Checklist
For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper’s
contributions and scope?
[Yes]
(b) Did you describe the limitations of your work?
[Yes] We describe it in Section 7.
We also show which environments are ideal for our method via sensitivity test in
Section 6.5.
(c) Did you discuss any potential negative societal impacts of your work?
[Yes]
We
describe it in Section 7.
We also mention about possible private information leakage in
graph-to-graph transfer learning in Section 1.
By proposing transfer learning between
node types in one heterogeneous graph, we can alleviate this issue.
(d) Have you read the ethics review guidelines and ensured that your paper conforms to
them?
[Yes]
If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results?
[Yes] We mention
the full set of assumptions for Theorem 1 in Section 4.3.
(b) Did you include complete proofs of all theoretical results?
[Yes] The proof for Theo-
rem 1 is described in Appendix A.1.
If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main experi-
mental results (either in the supplemental material or as a URL)?
[Yes] They are allincluded in the supplemental material.
(b) Did you specify all the training details (e.g., data splits, hyperparameters,
how they
were chosen)?
[Yes] They are specified in Appendix A.9 and A.12.
(c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?
[Yes] They are specified in Appendix A.4 and A.6.(d)
Did you include the total amount of compute and the type of resources used (e.g., type ofGPUs, internal cluster, or cloud provider)?
[Yes] They are specified in Appendix A.12.
If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators?
[Yes] We cite public domain
adaptation library ADA and heterogeneous graph neural network library OpenHGNN
that we have used in our experiment in Appendix A.12.
(b) Did you mention the license of the assets?
[N/A] All codes and data we used are public.
(c) Did you include any new assets either in the supplemental material or as a URL?
[Yes]
We include our code in the supplemental material.
(d) Did you discuss whether and how consent was obtained from people whose data you’re
[Table]
using/curating?
[N/A] We do not use any personal data.
[Table]
(e) Did you discuss whether the data you are using/curating contains personally identifiable
[Table]
information or offensive content?
[N/A]
We do not use any personal data.
[Table]
If you used crowdsourcing or conducted research with human subjects...
[Table]
(a) Did you include the full text of instructions given to participants and screenshots, if
[Table]
applicable?
[N/A]
[Table]
(b) Did you describe any potential participant risks, with links to Institutional Review
[Table]
Board (IRB) approvals, if applicable?
[N/A]
[Table]
(c) Did you include the estimated hourly wage paid to participants and the total amount
[Table]
spent on participant compensation?
[N/A]
[Table]