Mind the Gap: Mitigating the Distribution Gap in Graph Few-shot Learning
Authors: Chunhui Zhang
Hongfu Liu
Jundong Li
Yanfang Ye
Chuxu Zhang
Venue: Accepted by TMLR ()

Abstract: Prevailing supervised deep graph learning models often suffer from the issue of label scarcity, leading to performance degradation in the face of limited annotated data. Although numerous graph few-shot learning (GFL) methods have been developed to mitigate this problem, they tend to rely excessively on labeled data. This over-reliance on labeled data can result in impaired generalization ability in the test phase due to the existence of a distribution gap. Moreover, existing GFL methods lack a general purpose as their designs are coupled with task or data-specific characteristics. To address these shortcomings, we propose a novel Self-Distilled Graph Few-shot Learning framework (SDGFL) that is both general and effective. SDGFL leverages a self-distilled contrastive learning procedure to boost GFL. Specifically, our model first pre-trains a graph encoder with contrastive learning using unlabeled data. Later, the trained encoder is frozen as a teacher model to distill a student model with a contrastive loss. The distilled model is then fed to GFL. By learning data representation in a self-supervised manner, SDGFL effectively mitigates the distribution gap and enhances generalization ability. Furthermore, our proposed framework is task and data-independent, making it a versatile tool for general graph mining purposes. To evaluate the effectiveness of our proposed framework, we introduce an information-based measurement that quantifies its capability. Through comprehensive experiments, we demonstrate that SDGFL outperforms state-of-the-art baselines on various graph mining tasks across multiple datasets in the few-shot scenario. We also provide a quantitative measurement of SDGFL’s superior performance in comparison to existing methods.

Published in Transactions on Machine Learning Research (06/2023)
Mind the Gap: Mitigating the Distribution Gap in Graph Few-shot Learning
Chunhui Zhang
chunhuizhang@brandeissdd
Brandeis University, MA, USA
Hongfu Liu
hongfulis@@brandeiseecd
Brandcis University, MA, USA
Jundong Li University of Virginia, VA, USA
edu
Yanfang Ye
yye 7wed. edu
University of Notre Dame, IN, USA
Chuxu Zhang
chuzuchang@rrondeis.co edu
Brandeis University, MA, USA
Reviewed on OpenReview: https: // //
openneviiew. net/ forum? id- LEVONNNLEL
Abstract
Prevailing supervised deep graph learning models often suffer from the issue of label scarcity, leading to performance degradation in the face of limited annotated data. Although numerous graph few-shot learning (GFL) methods have been developed to mitigate this problem, they tend to rely excessively on labeled data. This over-reliance on labeled data can result in impaired generalization ability in the test phase due to the existence of a distribution gap. Moreover, cxisting GFL methods lack general purpose as their designs are coupled with task or data-specifie characteristics. To address these shortcomings, we propose novel Self- Distilled Graph Few-shot Learning framework (SDGFL) that is both general and effective. SDGFL, leverages self-distileed contrastive learning procedure to boost GFL. Specifically, our model first pre-trains a graph encoder with contrastive lcarning using unlabeled data. Later, the trained encoder is frozen as a teacher model to distill a student model with a contrastive loss. The distilled model is then fed to GFL. By learning data representation in self-supervised manner, SDGFL effectively mitigates the distribution gap and enhances generalization ability. Furthermore, our proposed framework is task and data-independent,, making it versatile tool for general graph mining purposes. To evaluate the effectiveness of our proposed framework, we introduce an information- based measurement that quantifies its capability. Through comprehensive experiments, we demonstrate that SDGFL perferos state-of-the-art baselines on various graph mining tasks across multiple datascts in the few-shot scenario. We also provide quantitative measurement of SDGFL's superior performance in comparison existing methods.
1 Introduction
Deep graph learning, such as graph neural networks (GNNs), has garnered significant atlention for its exceptional performance in various domains, such as information systems (Kipf Welling, 2017; Hamilton et al., 2017; Zhang al., 2022a; Liu al., 2023), molecular themistry/boologe /bioloyy (Jin al., 2017: Hao et al., 2020), and recommendation (Ying al., 2018; Fan et al., 2019). However, the success of GNNs typically requires substantial amount. of annotated data, which be prohibitively expensive obtain. To address

---

Published in Transactions on Machine Learning Research (06/2023)
this challenge, graph few-shot learning (GFL) (Zhang et al., 2022b) has emerged area of active research aimed at improving performance in the face of limited labeled data.
Previous GFL models have been developed using meta-learning (or few-shot learning) techniques, such as metric- based approaches ( (Vinyals et al., 2016; Snell et al., 2017) or optimization- based algorithms (Finn et al., 2017). These models aim to quickly learn an effective GNN for new tasks with only a few labeled samples. GFL has been applied to variety of graph mining tasks, including node classification (Zhou et al., 2019; Huang & Zitnik, 2020), relation prediction (Xiong et al., 2018; Lv et al., 2019; Zhang et al., 2020a), and graph classification (Chauhan et al., 2020; Ma et al., 2020). Despite considerable progress, most existing GFL, models suffer from the following limitations: (i) Impaared generalization. Current GFL methods often rely too heavily on labeled data, which can result in limited generalization ion and transferability to new tasks due to the existence of a distribution shift between non-overlapping meta-training and meta-testing data. Without supervision signals from ground-truth labels, GFL, may not learn an effective GNN for novel classes of test data, thus limiting its overall performance. (ti) Constrained design. Most current GFL methods lack general purpose, as they assume that the designated lask is universally the same prior across different. graph tasks datasets, which is not always guaranteed. For instance, GSM (Chauhan et al., 2020) requires the manual definition of a superclass of graphs, which cannot be extended to node-level tasks. The task or data-specific design limits the applicability of GFL to different graph mining tasks.
The challenges highlighted earlier underscore the need for a novel, generic GFL framework that can learn a transferable, effective, and generalizable GNN for various graph mining tasks with limited labeled data, Fortunately, contrastive learning has emerged as promising approach to reduce the dependence on labeled data and learn label-independent yet transferable representations from unsupervised pretext tasks for vision, language, and graphs (Chen et al., 2020; Gao et al., 2021; You et al., 2020; Sohn et al., 2020). Consequently, we propose a novel Self- Deff-Distilled Graph Few-shot Learning framework (SDGFL) that leverages contrastive learning to enhance GFL.
Specifically, our proposed framework pre-trains a GNN by minimizing the contrastive loss between the embeddings of two views generated from two augmented graphs. Then, order to learn more general and transferable representations for fast adaptation in GFL, we introduce a self-distillation step, an implicit ensemble of two pre-trained models (teacher and student), which reducos the variance (Allen-Zhu & Li, 2023) to improve the pre-training: the pre-trained GNN is frozen as a teacher model and utilized in the contrastive framework to distill a randomly initialized student model by minimizing the agreement of the embeddings generated by the two models. Both pre-training and the distillation steps can operate without requiring labeled data before the meta-learning phase. Finally, the distilled student model is taken as the initialized model for GFT, in few-shot graph mining tasks. SDGFL, learns graph representation in a self-supervised manner, effectively mitigating the negative impact of distribution shift, while producing transferable and discriminative graph representation for new tasks in the test data. Furthermore, framework is applicable to different graph mining tasks. To quantitatively measure the capability of SDGFL, we introduce an information- on each layer of the model. Specifically, we assign each node a learnable variable as a noise and train these variables to maximize the entropy while keeping the change of output as small as possible.
simple and generic
method that mcasures the quality of learned node (or graph) emboddings
To summarize, our contributions in this work arc:
We develop a general and effective SDGFL framework that leverages a self-distilled contrastive learning procedure to enhance GFL. SDGFL mitigates the impact of distribution shift and has a task and data-independent capacity for general graph mining purposes.
We introduce an information-basee method to quantitatively measure the capability of SDGFL by evaluating the quality of learned node (or graph) embeddings. To our knowledge, this is the first study that explores GFL model measurement.
Comprehensive experiments on multiple graph datasets demonstrate that SDGFL outperforms state-of-the- art methods for both node classification and graph classification tasks in the few-shot scenario. Additional measurement results further confirm that SDGFL learns better node (or graph) embeddings than baseline methods.
2

---

Published in Transactions on Machine Learning Research (06/2023)
2
Related Work
Few-Shot Learning on Graphs. In recent years, several GFL models have been proposed to solve various graph mining problems in the face of label sparsity, such as node classification (Yao et al., 2020; Ding et al., 2020; Huang & Zitnik, 2020; Wang et al., 2022; Zhang et al., 2022b;c), relation prediction (Xiong et al., 2018; Lv et al., 2019; Chen et al., 2019; Zhang et al., 2020a;b; 2022d), and graph classification (Chauhan et al., 2020; Ma et al., 2020; Guo et al., 2021; Wang el al., 2021; Zhang et al., 2023b). These models are built on meta-learning (or few-shot learning) techniques that can be categorized into two major groups: (1) metric-based approaches, which learn effective similarity metrics between few-shot support data and query data (Vinyals et al., 2016; Snell et al., 2017); (2) optimization-baste algorithms (Finn et al., 2017), which aim to learn well-initialized GNN parameters that can be quickly adapted to new graph tasks with few labeled data, For example, GPN (Ding et al., 2020) conducts node informativeness propagation to build weighted class prototypes for a distancc-basod node classificr. The second group proposes to learn well-initialized GNN parameters that can be fast adapted to new graph tasks with few labeled data. For instance, G-Meta (Huang & Zitnik, 2020) builds local subgraphs to extract subgraph-speciffc information and optimizes GNN via MAML (Finn et al., 2017). While previous efforts have relied on labeled data and had task and data-specific designs, we aim to develop novel framework that explores unlabeled data and has generic design for general graph mining purposes.
Self-Supervised Learning on Graphs. Self-supervised graph learning (SGL) has recently received considerable attention due to its effectiveness in pre-training GNNs and its competitive performance in various graph mining applications. Previous SGL models can be categorized into two main groups: generative learning and contrastive learning (Liu et al., 2020; Sohn et al., 2020; Zhao et al., 2021; Yu al., 2022; Yuc et al., 2022; Qian et al., 2022; Zhang al., 2023a; Tian et al., 2023). The generative models learn the graph representation by recovering feature structural information on the graph. The task can recover only the adjacency matrix alone (You et al., 2018) or together with the node features (Hu et al., 2020b). As for the contrastive methods, they first define the node context, which can be node-level or graph-level instances. Then, they perform contrastive learning either by maximizing the mutual information between the node-context pairs (Hassani & Ahmadi, 2020; Velickovic et al., 2019; Sun et al., 2020) or by discriminating the context instances (Qiu al., 2020; Zhu et al., 2021). In addition to the above strategy, random propagation has recently applied graph augmentation (Rong et al., 2020) for semi-supervised learning (Feng et al., 2020). Motivated by the success of SGL, we propose to use it to improve GFL.
3 Preliminary
GNNs. A graph is represented as = (V, X), where V is the set of nodes, E E V x V is the set of edges, and X is the set of node attributes. GNNs (Hamilton al., 2017; Xu et al., 2019; Zhang al., 2019) learn compact representations (embeddings) by considering both graph structure E' and node attribute X. To be specific, let fe(-) denote GNN encoder with parameter 0, the updated embedding of node at the 1-th layer of GNN can be formulated as:
h M(N (T((-)) | Vu E NN):
(1)
where Mr, denotes the neighbor set of v; M(-) is the message passing function for neighbor information aggregation, such as mean pooling layer followed by fully-connected (FC) layer; h is initialized with node attribute Xy. The whole graph embedding can be computed over all nodes' embeddings as:
k2 = READOUT(A Vo V},
(2)
where the READOUT function can be a simple permutation invariant function such as summation.
GFL Setting and Problem. Let Chase and Cnovet denote the base classes set and novel (new) classes set in training data Traan and testing data Trest; respectively. Similar the general meta-lcarning problem (Finn et al., 2017), the graph few-shot learning (GFL) aims train a GNN encoder fo(-) over Chase that can be quickly adapted to Crowet with limited labeled data per class. The base and novel classes are disjoint, denoted
3

---

Published in Transactions on Machine Learning Research (06/2023)
Cbase Cnovel Ø. In K-shot setting, batch of classes (tasks) is randomly sampled from Coase during the meta-training phase, where K labeled instances are used to form the support set 8 for model training and the remaining instances are taken as the query set Q for model evaluation. After sufficient training, the model further transferred to the mota-testing phase to conduct N-way classification over Cnovel (N the number of novel classes), where each class is only with K labeled instances. The GFL. framework can be applied various graph mining problems, such as node classification or graph classification, depending on the class definition. In this study, we consider both node classification and graph classification problems under the few-shot setting, formally defined as follows:
Problem Few-Shot Node Classification. Given a graph G E, and labeled nodes of Crases the problem is to learn a GNN fo(-) to classify nodes of Cnovel, where each class in Cnovel only has few labeled nodes.
Problem 2 Few-Shot Graph Classification. Given a set of graphs G and labeled graphs of Coases the problem is to learn GNN fo(-) to classify grophs of Crovel, where each class Crovel only has few labeled graphs.
contrast to previous studies that rely solely on labeled data of Traai and Trest for GFL model training and adaptation, our approach leverages both labeled data and unlabeled graph information to learn GFL model for solving the above problems.
4 Methodology
Figure 1 depicts the proposed SDGCL framework, which comprises two main phases: self-distileed graph contrastive learning and graph few-shot learning (GFL). The first phase (Figure (())) involves pre-training a GNN encoder with contrastive learning, followed by knowledge distillation to enhance the pre-trained GNN in self-supervised manner. The distilled GNN is subsequently fed into the GFL phase (Figure 1(b) for few-shot graph mining tasks. Additionally, we introduce an information-baset approach to quantify the superiority of SDGCL.
4.1 Self-Distilled Graph Contrastive Learning
GNN Contrastive Pre-training. In the first. phase, we employ contrastive learning to pre-train the GNN. Inspired by the representation bootstrapping technique (Grill et al., 2020), our method learns node (or graph) representations by discriminating context instances. We introduce two GNN encoders: an online GNN fa(-) and a target GNN fe(-), to encode two randomly augmented views of a given graph. The online GNN is supervised under the target GNN's output, while the target GNN is updated by the online GNN's exponential moving average. Figure (a) shows the contrastive pre-training step.
Graph Augmentation: We process the given graph with random data angmentations to generate contrastive pair (G', G") as input for the two GNN branches (online branch and target branch) used in GNN training. We apply combination of stochastic node feature masking, edge removal, and node dropping with constant probabilities for graph augmentation.
GNN Update: Using the generated graph pair (G', G"), the online GNN fo(.) and the target GNN fe(.) are respectively utilized to process G' and G" for node (or graph) embedding generation. Both GNNs have the same architecture, while two-layer FC (one-layer FC) is attached after an online GNN (target GNN) refine embedding. To prevent the prediction of the ouline model from being exactly the same the output of the target model and avoid the learned representation collapse, two branches have different FC layers. To enforce the online GNN's embeddings ze to approximate the larget GNN's embeddings heg formulate the mean squared error between them as the objective function:
20. hg
Lo.5 = 1|=0 - -hell2 2.
(3)
1|00||2 1
4

---

Published in Transactions on Machine Learning Research (06/2023)
inner loop update
stop grad
mncta-trainnng set
loss
auggminn
2. viewss
GNN 4(()
FC
emboakdig ftg
support data
support loss
exponenial moving averaee
updatc weights by
support
support protocypes
querty dataa
query loss
GNN
6"
FC
embeddinn Fe
query
outer loop update
online GNN fa(+)
dropping Edge dropping Atmibute mask
shared graph sugmentation sitalogy
copy pre-trained model distillation
copy metn-trained weights
copy distilled model for GFL
stop grad
inner loop update
BOTTI
migmeent 2 views
C)
embedding R'a
support data
support loss
A
stop weight sharing
support set
support prototypes
unlabeled
6
query
query
query
GNN
student GNN 6.()
FC
FC
embedding z's,
mota-lcsting set
self-distilled graph contrastive learning phuse
(b) graph few-shot learning phase
Figure 1: The SDGCL framework consists two phases: (a) self-distileed graph contrastive learning, whore a GNN encoder pre-trained with stop-grad contrastive learning (Grill et al., 2020; Chen He, 2021) and further evaluated with knowledge distillation a self-supevvied manner; (b) graph few-shot learning, which takes the distilled student network as the initialized model and uses meta-learning algorithm for model optimization.
We update the parameters 0 of the online GNN using the Adam optimizer (Kingma & Ba, 2015):
θ - Adam(6, VeLo.g,
(4)
where 77 is the learning rate. We use the target GNN to provide the regression target to supervise the online GNN, and its parameters & are updated as the exponential moving average (EMA) of the online GNN parameters 0. More precisely, & is updated as follows:
E- & TE (1-7)0,
(5)
where r E [0, 1] is the decay rate. Note that the target GNN stops the backpropagation from Ca and it is only updated by EMA.
Contrastive Distillation. With the pro-trained GNN fo(-) obtained in the previous step, propose method called Contrastive Distillation lo enhance the performance of a pre-trained GNN fa(.). Our approach is inspired by the Born-again strategy (Furlanello et al., 2018), which posits that well-trained teacher can improve randomly initialized identical student. The distillation step adopts a similar contrastive framework as the previous step, as shown in Figure 1aa). Specifically, we first use the pre-trained GNN fo(-) as the teacher model (.), and then generate two augmented views (G*, G**) graph to be fed into both the teacher and the student models (fe, (-) and fe,()) (+)), respectively. The teacher model is then frozen and used to distill the student model fo,(.). (-). To ensure that the student model approximates the teacher model, use contrastive framework, illustrated in Figure 1 (a). Specifically, we force the student's normalized output to approximate the teacher's 's normalized output, which is represented as follows:
11 2/6, 1//
Le. =1126 nio, 2-2
(6)
21 1// ll2
5

---

Published in Transactions on Machine Learning Research (06/2023)
28, h/.
he.
20,
(7)
2z 1l
||t, lla
where 20, and he, are the output embeddings of the student and teacher models, respectively. The student model is then updated using the Adam optimizer, as follows:
0g - Ad, Fa,Ca,, 17)
In contrast to the target GNN update in contrastive pre-training that uses the EMA method (Equ. 5), the teacher model in our method is frozen and can be seen special case of EMA, which is represented follows:
02 - -0, + (1 7)0,, ==1
(9)
4.2 Graph Few-Shot Learning
In the GFL phase, we take the distilled student GNN (o) generated in the previous phase as the initialized GNN model using an optimization- 2017). During meta-training, we compute the task-specific paramcters 6',: for task T; using a number of gradient descent updates over the support set S, of Te (i.e., inner-loop), as follows:
algorithm called model-agnostic meta-learning (MAML) (Finn et al.,
- oo. (FF...
(10)
where is the learning step size, and C% denotes the downstream task loss over S. To perform node graph classification, we use the protolypical loss (Snell et al., 2017), which utilizes embeddings extracted from the support set by a neural network as the class prototype, and the query set is classified according to the distance between its embeddings and prototypes. We then utilize the task-specific paramcter 8',, compute the loss over the query set Q, of Tr which is denoted as C. (for (for,). The losses of batch of randomly sampled lasks are then summed up to update the model parameters 0g (i.e., outer-loop), as follows:
0, - - NV, Σ Co:
(11)
where 33 is the learning step size. During meta-testing, we apply the same procedure, but using the final mnta-e parameter 02 for novel tasks, without the outer-loop. 0, is learned from knowledge across meta-training tasks and represents the optimal parameter for quickly adapting to novel tasks. It is important to note that the GFL algorithm can be applied to various graph mining problems by changing the meaning of each task. Specifically, for node classification (or graph classification), each task corresponds to a node class (or graph class).
4.3 Quantitative Measurement of GFL
The previous GFL studies target developing better methods in performance while none of them has thought about model capability measurement. To fill the gap, extend the neural network model capability measurement method proposed by Ma et al. (Ma et al., 2019) to graph data and use mutual information (MT) to measure the information of the input graph G encoded by the hidden state Z of a GFL model f. Specifically, the mutual information MT(G; Z) is given by:
MI(G; Z) (GG) HG]))),
(12)
where H-)) denotes the entropy, HG) is a constant, and H(])Z) represents the amount of discarded information after G is processed by o and encoded by Z. We can compute Z[ by decomposing it into the node level as follows:
H(GIZ)
Jzez (Giz)d)
(13)
where z = (xx) denotes the hidden state corresponding attribute of node. To disentangle information components of individual nodes from the whole graph, assume that each node is independent of the others and have:
H(Glz) = Σ H(zlz)
(14)
6

---

Published in Transactions on Machine Learning Research (06/2023)
where 2, denotes random variable of the i-th node attribute in the graph. Then, we introduce noise pertarbation-based method to approximate (2za2)) Specifically, let T zi 6 (ee ~ (0,, E; rf and we aim to optimize the following loss function:
((o) E. [x])((x)) 2
H la
(15)
where o = 02. .nn] are learnable parameters and 3 is a trade-off weight. In particular, The first term of the above objective minimizes the difference between the encoded embodding of noisy input and the hidden state, while the second term encourages high conditional entropy according to the maximum entropy principle. In this way, we have p(Z)|z) p(e) and Hxx,zz) is approximated by H H(z.(z) as follows:
H( p(. (2) log pl (( CC
(16)
where C= 1 Log((TTe) By taking Eqn. 16 into Equ. 15, we can optimize [(o) using the Adam optimizer to obtain the optimal a for computing the overall discarded information H(G Comparing the discarded information of different GFL models allows us to measure their capabilities. Ideally, we want the GFL model to encode valid node (or graph) embeddings as much as possible, which means discarding as little information as possible.
2
5 Experiments
We have performed comprehensive experiments on multiple graph datasets to evaluate our model's performance against tfate-of-tho-att models. In this section, we first describe our experimental settings and then present our findings by comparing the performance of various models. Finally, we compute the discarded information to demonstrate the capabilities of different GFL models. Additional experimental results can be found in Appendix C.
5.1 Experimental Setup
Datasets. To conduct our experiments, we used multiple graph datasets. Specifically, for the node classification task, we used ogbn-arxiv (Hu et al., 2020a), Tissue-PPI (Hamilton al., 2017), Fold-PPI (Zitnik & Leskovec, 2017), Cora. (Sen et al., 2008), and Citescer (Sen et al., 2008). For the graph classification task, we used the dalasets in (Chauban et al., 2020), namely, Legher-High, Triangles, Reddit-12K, and Enzymes. Appendix A provides more detailed information about the datasets.
We employ a diverse range of bascline methods for model comparison in the two tasks. For few-shot node classification, we use node2vec (Grover & Leskovec, 2016), DeepWalk (Perozzi et al., 2014), Meta-GNN (Zhou et al., 2019), FS-GIN (Xu et al., 2019), FS-SGC (Wu al., 2019), No-Finetune (Triantafillou et al., 2019), Finetune (Triantafillou et al., 2019), KNN (Triantafillou et al., 2019), ProtoNet (Snell et al., 2017), MAML (Finn et al., 2017), G-Meta (Huang & Zitnik, 2020), and TENT (Wang al,, 2022). For few-shot graph classification, we utilize WT. (Shervashidze et al., 2011). Graphlet (Shervashidwe et al., 2009), AWE (Ivanov & Burnaev, 2018), Graph2Vec (Narayanan et al., 2017), Diffpool (Lee et al., 2019), CapsGNN (Xinyi & Chen, 2018), GIN (Xu et al., 2019), GIN-KNN (Xu et al., 2019), GSM-GCN (Chauhan et al., 2020), GSM-GAT (Chauhan et al., 2020), and AS-MAMI, (Ma et al., 2020). Details of the baseline methods are presented in Appendix B.
Bascline Methods.
Experimental Settings. In our proposed SDGCL approach, adopt GCN (Kipf & Welling, 2017) as the GNN backbone for the node classification task, which includes two-layer graph convolution and one-layer fully connected layer. For the graph classification task, we add an additional average pooling operation as a readout layer. In the pretraining phase, we pretrain the GNN on the unlabeled graph datasel using contrastive learning. To augment graph data, we randomly drop 15% of the nodes, remove 15% of the edges, and mask 20% of the node features. The mini-batch size set to 2,048, and we use a lcarning rate of 0.05 with a decay factor of 0.9. Furthermore, we set the 7 value for exponential moving average to 0.999. We consider both inductive and transductive settings for our experiments. In the inductive setting (SDGCI-II)
7

---

Published in Transactions on Machine Learning Research (06/2023)
Few-shot node classification results. Baselines are designed for inductive setting only. The best results in our methods are highlighted in bold, while the best results in baselines are underlined. Also, baselines are implemented under inductive settings as stated in their original papers. -I and -T denote inductive and transductive settings of SDGCL, respectively. Teacher indicates the SDGCL model without knowledge distillation.
Table 1:
Tissue-PPI
Fold-PPI
Cora
Citeseer
ogba-arxiv
Method
3-shot
5-shot
3-shot
5-shot
3-shol
5-shot
3-shot
5-shol
3 S-oho
5-shot
uode2vee
3..5
3.9
36.6
37.4+19
1.3
36.9
20.0+2.5
21.7+2.9
28.9
29.5+3.7
DeepWalk
46.244.88
3..443.6
35.0:+4.4
36.3+3.2
25.6+0.88
26.7
21.2+0.6
22.6 2.7
30.3#2.1
31.5+3.4
Meta: GNN
8..
5..511.5
30.815.4
33.5+2.1
7..8
79.2:1..
69.411.4
1..6+1.9
2..311.
30.213.6
FS-GIN
49.2:224
31.5++3.0
36.7#2.1
39.1+144
53.5+16
56.2+2.8
50.2+2.6
53.2+3.8
33.6#:4.2
36.8+2.5
FS-SGC
49.8+3.8
52.3:22.
38.0#1.6
30.9
27.2+2.11
60.3+
02.012.11
54.422.5
34.7=0.5
37.3+1.0
No- Finetune
61.6+0.6
55.04:2.1
37.6+1.7
39.943.6
61.241.2
64.5+1.3
64.9+1.7
58.3425
36.4+14
38.842.0
Finetune
52.11:13
54.3122.4
37.0+2.2
40.01:2.6
63.5+008
65.7:2.1
57.8118
59.0:2.9
35.9110
38.6
KNN
61.9+225
65.2433.
33.3#3.4
16.2+119
67.8+144
30.3
60.6+14
63.2+1.6
39.2#115
12.3+188
ProtoNet
54.6:12.55
$7.54129
38.22331
1..
42.6433.7
56.6122.
1..
58.043377
3.7
39.7
MAML
74.5#t..1
77.4+277
4..2#662
51.3+:3.3
65.7+00.9
68.8+1.1
63.141.6
65.7+1.7
38.9#2.1
41.34224
GPN
77.3++3.0
79.0+3.6
57.0
38.2+3.7
73.14:2.4
7..1
1..
71.11:2.0
34.4
48.2
RALE
36.6+33.
3..
57.8++1.5
3.3
62.8+3.1
6..9+3.2
29.
71.3:222
25.1
1..8+1.5
G-Meta
76.84:2.9
79.442..6
56.19:5.9
59.0422.5
719429
74.5+2..
67.8422.
70.84:3.8
13.1
48.2433.1
TENT
64.8+4.11
69.2
54.2+3.4
62.042.3
55.6+3.11
02.9+3.77
Teacher-1
77.912.6
80.81177
58.813.4
61.313.6
18.2+13
80.9199
70.0E13
12.711.8
$2.0=2.0
54.9116
S
78.7::2.8
81.513.6
59.521411
62.012.0
78.5+1.5
81.2115
70.6+1.2
73.112.0
52.8
55.611.1
Teacher-T
79.5#3.1
62.9+1.3
63.
65.6+222
80.0+2.7
82.611.9
72.1
75.0+1.5
54.322.7
58. 3.399
SDGCL-T
80.9:330
84.1+33.
66.9433.4
69.0
80.7+1.9
83.5+3.0
72.6#1.6
75.342.0
55.242.5
58.7=2.7
we only use the unlabeled data in the training set, whereas in the transductive setting (SDGCI-T), we use the unlabeled data in both the training and testing sets. For pre-trained models without knowledge distillation, we denote them as Teacher-I and Teacher-T for the two settings, respectively. In the GFL phase, we utilize MAML for fine-tuning the model. Our SDGCL implementation is based on PyTorch, and we train it on NVIDIA V100 GPUs
5.2 Few-Shot Node Classification
Overall Performance. Table 1 presents the performances of all models for 3/5-shot node classification. Our observations are as follows: 1) SDGCL outperforms all baseline methods all datasets, demonstrating its superiority for few-shot node classification; (2) The improvement of SDGCL-I over baseline methods ranges from 1.1% to 50% (3-shot) and from 2.1% to 51% (5-shot), while the improvement of SDGCL-T ranges from 4.8% to 55.0% (3-shot) and from 4.5% to 56.6% (5-shot). These significant improvements demonstrate the effectiveness of contrastive pre-training and self-distilatiion in learning rich node representations from unlabeled graph data and addressing the label-hungry issue; (3) SDGCL-T (or Teacher-T) outperforms SDGCL-I (or Teacher-I), indicating that the model benefits from unlabeled data in the meta-testing set, thus improving generalization ability over testing data; (4) The contrastive distillation employed in SDGCL leads to an additional boost compared with the pre-trained teacher model without any label cost in distillation, demonstrating its effectiveness.
Impact of Shot Number. In Figure 2a, we present the performance of our model (SDGCT-T) under different shot numbers (1 to 5) compared seleeted basclines. SDGCL almost outperforms other methods across all shot numbers, demonstrating the stability of our model for node classification. Note that we show results for two datasets only, while results for other datasets are presented in Appendix C.1.
Impact of Training Label Rate. To evaluate SDGCL's performance under different training label rates (10%, 20%, 30,, 40%, 50%, 100%), we compare it with baseline methods for 3-shot node classification, as shown in Figure 2b. Our model nearly consistently outperforms the baseline models across all label rates. Furthermore, the performance improvement of SDGCL over bascline models is more significant when the label rate bocomes lower (c.g., 10%), demonstrating that the contrastive pre-training and self-distillation of SDGCL are effective in cases of label sparsity.
8

---

Published in Transactions on Machine Learning Research (06/2023)
65
ogbn-arxiv
90
Tissue-PPI
60
ogbn-arxiv
90
Tissue-PPI
55
80
75
Accuracy
45
Accuracy
Accuracy
Accurac/
70
60
35
50
d-Meta
45
Ours
G-Mata
MAMI
C-ffute NAHL
HAML
40
Protefeet
MANL
TENT
30
2
10 20 30 40 50
100
20
30
40
100
Shot number
Shot number
Label rate
Label rate
(a) Impact of shot number.
(b) Impact label rate.
Figure 2: Impact of shot number and label rate on node classification.
Impact of Data Augmentation. Graph augmentation is a crucial step in contrastive learning of SDGCL, and it significantly affects model performance. We conduct experiments evaluate the model performance with different augmentation strategies, including node dropping (ND), feature masking (FM), edge removing (ER), and their combinations. In Figure 3, report the performances of SDGCL under these augmentation strategies. The combination of all three augmentation strategies outperforms single or two augmentations, demonstrating that various graph augmentations generate sufficiett contrastive pairs to improve the model performance.
ND
ER+FM
ND+ER+FM
Table Info. loss for node classification.
ER
ND+ER
ND+FM
56
82
Method
Layer
ogbu-arxivv
Tiasue-PPI
80
54
GNN-1
385.15
788.30
Accuracy
G-Meta
GNN.2
FC
374.42
733.85
GNN-1
315.70
TENT
GNN-2
296.67
FC
293.55
74
GNN-I
305.29
663.38
SDGCL-I
GNN-2
300,29
6500.59
290.99
612.98
48
ogbn-arxiv
Tissue-PPI
GNN-1
276.30
SDGCL-T
GNN-2
267.33
525.42
PC
264.23
503.86
Figure 3: Impact data aug. on node classification.
Loss Information Comparison. In Section 4.3, we propose to compute the amount of graph information discarded in the GFL model. Here compare the results between SDGCL and selected baseline method (G-Meta) in Table 2. It can be observed that the amount of discarded information in each layer of SDGCL is smaller than that of baseline models. This may be due to the fact that SDGCL can learn more label- independent information from unlabeled graph data, which demonstrates the superiority of SDGCL in lcarning node embeddinggs for node classification.
5.3 Few-Shot Graph Classification
Overall Performance. The presented results in Table 3 demonstrate the superiority of SDGCL over all baseline models in few-shot graph classification. Specifically, the improvement of SDGCL-1 over baseline models ranges from 0.95% to 38.36% (5-shot) and from 0.8% to 34.33% (10-shot), while the improvement of SDGCL-T ranges from 5.65% to 46.21% (5-shot) and from 2.18% to 41.51% (10-shot). These results demonstrate the effectiveness of contrastive pre-training and oolf-distillation learning informative graph embeddings from unlabcled data. Furthermore, the transductive sctting of SDGCL-T outperforms the inductive setting of SDGCL-I, indicating the benefit of using unlabeled data in the testing to improve generalization performance. Additionally, it worth noting that the SDGCL model with contrastive
9

---

Published in Transactions on Machine Learning Research (06/2023)
Table 3: Results of few-shot graph classification. Baselines are designed for inductive setting only. The best results in our methods are highlighted in bold, while the best results in baselines are underlined. Also, baselines are implemented under inductive settings stated in their original papers. The sulfixes -I and -T denote the inductive and transductive settings of SDGCL, respectively. The term "Teacher' refers to the SDGCL model without knowledge distillation.
Letter High
Triangles
Reddit-12K
Enzyınes
Method
5-shot
10-shot
5-shot
10-shot
5-shot
10-shot
5-shot
10-shot
WL
05.27747.600
68.39
51.25-4-.02
23.26
50.26:45.17
42.57 #366
55.78:14.77
58.47:+3.88
Graphlet
33.76.16.99
37. 4.66
10.17433.1
43.76=3.09
6.94
37.59
53.1745.92
55.30
AWE
30.60:43.91
22.20:22.7
39.36
42.58
30.24:1..34
33.441:.04
13.75:1..85
45.58
Vee
56.1245.21
68, (..27
38.3843.85
50.16::4115
27 27.85:44.21
29.9713.17
55.88+44.80
58.22::4.30
Diffpool
6..69
61.59=5.21
64.1715.87
67.12::.29
5..24+5.669
37.43
45.64+-4.56
49.644:4.23
CapsGNN
56.60+1..88
6.26
6..40-16.13
68.37=3.67
36.58:14.225
3.776
5..67:1.51
55.31
GIN
7.173:17.17
69.16:5.114
63.50:15.61
67.30
40.36:144.69
43.70
55.73+55800
58.831.5.32
GIN-KNN
63.52:17.27 65.66188.9
58.34413.99
3.19
41.31:12.4
43.58 2.880
57.241
59.3411.24
GSM-GCN
68.69:+6.50 42.80:4.12
69.37:1:4.02
73.11=3.94
40.77:14.32
44.28
5..3445.64
58.16144.39
GSM-GAT
5.99 73.28=3.46
71.40+4..34
75.60=3.67
41.59:1.12
45.67
55.42++..4
60.64+3.84
ASMMAML
7..2
1.11
7..66+1117
2..56
1.69
1.16
56.03+1.85
60.79+2.74
Teacher-1
71.43:55.23
73.6242.93
71.9343.61
76.21=2.87
12.32:14.48
16.81
57.53+3.10 3,16
61.12+4.80
SDGCL-1
72.12+4488
74.08+3.30
72.34+3.42
76.91=2.98
42.85:14.62
46.89+4.96
57.94+2.85
61.66+4.61
Teacher-T
7..20=+4.34
1..35+1995
78.55+3.75
$1.03=3.37
44.80:244885
48.95+4.03
59.85+2.34
62.30+3.29
SDGCL-T
7..97:5.002
79.10:14.23
79.32214400
1.330
45.55
49.32:1.221
60.34:14.04
62.97
Letter-High
Triangles
Letter-High
Triangles
80
73
Accuracy
Accuracy
Accuracy
70
75
76
70
50GFL
SDGFL
63
SDGTL
50GFL
GAT
68
GSM-GAT
MAML
MAML
65
64
55
10
15
40
Shot number
Shot number
Label rate
Label rate
(a) Impact of shot number.
(b) Impact of label rate.
Figure 4: Impact of shot number and label rate graph classification.
distillation (Teacher-T) outperforms the SDGCL model without distillation, highlighting the effectiveness of the proposed knowledge distillation method.
Impact of Shot. Number. In Figure 4a, we present the results of our model (SDGCL) under varying shot numbers (5, 10, 15, 20) compared to select baseline methods. Similar to Figure 2a, SDGCL consistently outperforms the baseline models across varying shot numbers, demonstrating the robustness of SDGCL. We note that the results are presented for two datasets, namely Better-High and Triangles, and the results for the other datasets are shown Appendix C.2.
Impact of Training Label Rate. In Figure 4b, we show the performance of SDGCL under different training label rates in comparison with baseline models for 5-shot graph classification. Consistent with the results in the node classification task, SDGCL exhibits better accuracy across various label rates, with the performance gaps between SDGCL and bascline models being more pronounced for lower label rates. These results underscore the significance of SDGCL in scenarios where labeled data is scarce.
Impact of Data Augmentation. We also study the influence of graph augmentation on the few-shot graph classification task. As shown in Figure 5, the combination of three augmentation strategics yields the best performance, highlighting the significance of generating enough contrastive pairs during model training.
10

---

Published in Transactions on Machine Learning Research (06/2023)
ND+ER
ND+FM
Table 4: Info. loss for graph classification.
76
80
Method
Letter-High
Triangles
74
78
GNN-1
1286.67
1520.91
Accuracy
GSM-GAT
GNN-2
1250.54
1487.42
72
76
GNN-1
ASMAML
GNN-2
901.00
1221.45
70
GNN-3
899.12
1102.12
SDGCL-1
GNN-2
920.58
1219.91
FC-3
892.03
1107.56
Letter-High
Triangles
GNN-I
792.65
923.35
SDGCL-T
Figure 5: Impact of data aug. on graph classification.
FC
765.73
885.35
Loss Information Comparison. We also compare the amount of discarded graph information between our proposed model and a selected baseline method (GSM-GAT), as shown Table 4. Notably, our model (SDGCT.) discards less graph information than the baseline, which suggests that SDGCL, can effectively capture and leverage more relevant information for graph classification tasks. This finding supports the superiority of our model in lcarning informative graph embeddings.
6 Conclusion
In this paper, we proposed a novel framework called SDGCL (Self-Distilled Graph Few-shot Learning) to address the limitations of existing Graph Few-shot Learning (GFL) models. Our framework is designed to learn generalized graph representations and overcome constraints in task-speeific design. SDGCL leverages a self-distilled contrastive learning approach to enhance GFL by pre-training the GNN encoder with contrastive learning and then further improving it with knowledge distillation in self-supervised manner. We also introduced an information-basel method to compute the amount of discarded graph information by the GFL model. Our extensive experiments on multiple graph datasets demonstrated that SDGCL outperforms state-of-the-art baseline methods for both node classification and graph classification tasks in the few-shot scenario. The discarded information value further validates the superiority SDGCL in learning node and graph embeddings.
Acknowledgement
This work is partially supported by the NSF under grants CMMI-2146076 and Brandeis University. Any opinions, findings, conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of any funding agencies.
References
Zeyuan Allen-Zhu and Yuanzhi Li. Towards understanding ensemble, knowledge distillation and self-distillation in deep learning. In ICLR, 2023.
Jatin Chauhan, Deepak Nathani, and Manohar Kaul. Few-shot learning on graphs via super-classes based on graph spectral measures. In ICLR, 2020.
Mingyang Chen, Wen Zhang, Wei Zhang, Qiang Chen, and Huajun Chen. Meta relational learning for few-shot link prediction in knowledge graphs. In EMNLP-IJCNLP, 2019.
Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. simple framework for contrastive learning of visual representations. In ICML, 2020.
Xinlei Chen and Kaiming He. Exploring simple siamese representation learning. In CVPR, 2021.
11

---

Published in Transactions on Machine Learning Research (06/2023)
Kaize Ding, Jianling Wang, Jundong Li, Kai Shu, Chenghao Liu, and Huan Liu. Graph prototypical networks for few-shot learning on attributed networks. In CTKM, 2020.
Wenqi Fan, Yao Ma, Qing Li, Yuan He, Eric Zhao, Jiliang Tang, and Dawei Yin. Graph neural networks for social recommendation. In WWW, 2019.
Wenzheng Feng, Jie Zhang, Yuxiao Dong, Yu Han, Huanbo Luan, Qian Xu, Qiang Yang, Evgeny Kharlamov, and Jie Tang. Graph random neural networks for nemi-supervised learning on graphs. In NeurlPS, 2020.
Chelsea Finn, Pieter Abbeel, and Sergey Levine. agdestag meta-learning for fast adaptation of deep networks. In ICML, 2017.
Tommaso Furlanello, Zachary Lipton, Michael Tschannen, Laurent Itti, and Anima Anandkumar. Born again neural networks. In ICML, 2018.
Tianyu Gao, Xingcheng Yao, and Danqi Chen. SimCSE: Simple contrastive learning of sentence embeddings. In EMNLP, 2021.
Jean-Bastien Grill, Florian Strub, Florent Altché, Corentin Tallec, Pierre H Richemond, Elena Buchatskaya, Carl Doersch, Bernardo Avila Pires, Zhaohan Daniel Guo, Mohammad Gheshlaghi Azar, et al. Bootstrap your own latent: A new approach to self-supervised learning. In NeurIPS, 2020.
Aditya Grover and Jure Leskovec. node2vec: Scalable feature learning for networks. In KDD, 2016.
Zhichun Guo, Chuxu Zhang, Wenhao Yu, John Herr, Olaf Wiest, Meng Jiang, and Nitesh Chawla. Few-shot graph learning for molecular property prediction. In WWW, 2021.
William L Hamilton, Rex Ying, and Jure Leskovec. Inductive representation learning on large graphs. In NewrIPS, 2017.
Zhongkai Hao, Chengqiang Lu, Zhenya Huang, Hao Wang, Zheyuan Hu, Qi Liu, Enhong Chen, and Checkong Lee. Asgn: An active semi-supervised graph neural network for molecular property prediction. In KDD, 2020.
Kaveh Hassani and Amir Hosein Khas Ahmadi. Contrastive multi-view representation learning on graphs. In ICML, 2020.
Weihua Hu, Matthias Fey, Marinka Zitnik, Yuxiao Dong, Hongyu Ren, Bowen Liu, Michele Catasta, and Jure Leskovec. Open graph benchmark: Datasets for machine learning on graphs. In NewrIPS, 2020a.
Ziniu Hu, Yuxiao Dong, Kuansan Wang, Kai- Wei Chang, and Yizhou Sun. Gpt-gun: Generative pre-training of graph neural networks. In KDD, 2020b.
Kexin Huang and Marinka Zitnik. Graph meta learning via local subgraphs. In NewrIPS, 2020.
Sergey Ivanov and Evgeny Burnaev. Anonymous walk embeddings. In ICML, 2018
Wengong Jin, Connor W Coley, Regina Barzilay, and Tommi Jaakkola. Predicting organic reaction outcomes with weiseeiler-ehhman network. In NewrIPS, 2017.
Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In ICLR, 2015.
Thomas N Kipf and Max Welling, Semi-supervised classification with graph convolutional networks. In ICLR, 2017.
Junhyun Lee, Inyeop Lee, and Jaewoo Kang. Self-attention graph pooling. In ICML, 2019.
Xiao Liu, Fanjin Zhang, Zhenyu Hou, Zhaoyu Wang, Li Mian, Jing Zhang, and Jie Tang. Self-supervised learning: Generative contrastive. arXiv preprind arXiv:2006.000 00022006.0218,, 2020.
Zheyuan Liu, Chunhui Zhang, Yijun Tian, Erchi Zhang, Chao Huang, Yanfang Ye, and Chuxu Zhang, Fair graph representation learning via diverse mixture-of-experts. In WWW, 2023.
12

---

Published in Transactions on Machine Learning Research (06/2023)
Xin Lv, Yuxian Gu, Xu Han, Lei Hou, Juanzi Li, and Zhiyuan Liu. Adapting meta knowledge graph information for multi-hop reasoning over few-shot relations. In EMNLP-IJCNTLP, 2019.
Haotian Ma, Yinqing Zhang, Fan Zhou, and Quanshi Zhang. Quantifying layerwise information discarding of neural networks. areie preprint arXiv: 0..006001000 04109, 2019.
Ning Ma, Jiajun Bu, Jieyu Yang, Zhen Zhang, Chengwei Yao, Zhi Yu, Sheng Zhou, and Xifeng Yan. Adaptive-step graph meta-learner for few-shot graph classification. In CIKM, 2020.
Annamalai Narayanan, Mahinthan Chandramohan, Rajasekar Venkatesan, Lihui Chen, Yang Liu, and Shan- tanu Jaiswal. graph2vec: Learning distributed representations of graphs. arXiv preprint Xrx: 707.05005, 2017.
Bryan Perozzi, Rami Al-Rfou, and Steven Skiena. Deepwalk: Online learning of social representations. In KDD, 2014.
Yiyue Qian, Chunhui Zhang, Yiming Zhang, Qianlong Wen, Yanfang Ye, and Chuxu Zhang. Co-modality graph contrastive learning for imbalanced node classification. In NewrIPS, 2022.
Jiezhong Qiu, Qibin Chen, Yuxiao Dong, Jing Zhang, Hongxia Yang, Ming Ding, Kuansan Wang, and Jie Tang. GCC: graph contrastive coding for graph neural network pre-training. In KDD, 2020.
Yu Rong, Wenbing Huang, Tingyang Xu, and Junzhou Huang. Dropedge: Towards deep graph convolutional networks on node classification. In ICLR, 2020.
Prithviraj Sen, Galileo Namata, Mustafa Bilgic, Lise Getoor, Brian Galligher, and Tina Eliassi-Rat Collective classification in network data. Al magazine, 2008
Nino Shervashidze, SVN Vishwanathan, Tobias Petri, Kurt Mehlhorn, and Karsten Borgwardt. Efficient graphlet kernels for large graph comparison. In AISTATS, 2009.
Nino Shervashidze, Pascal Schweitzer, Erik Jan Van Leeuwen, Kurt Mchlhorn, and Karsten M Borgwardt. Weisfeiler-lehman graph kernels. Journal of Machine Learning Research, 2011.
Jake Snell, Kevin Swersky, and Richard Zemel. Prototypical networks for few-shot learning. In NewrIPS, 2017.
Kihyuk Sohn, David Berthelot, Nicholas Carlini, Zizhao Zhang, Han Zhang, Colin A Raffel, Ekin Dogus Cubuk, Alexcy Kurakin, and Chun-Liang Li. Fixmatch: Simplifying semi-supervised learning with consistency and confidence. NewrIPS, 2020.
Fan-Yun Sun, Jordan Hoffmann, Vikas Verma, and Jian Tang, Infograph: Unsupervised and semi-supervised geveh representation learning via mutual information maximization. In ICLR, 2020.
Yijun Tian, Kaiwen Dong, Chunhui Zhang, Chuxu Zhang, and Nitesh V Chawla. Heterogeneous graph masked autocncoders. In AAAT, 2023.
Eleni Triantafillou, Tyler Zhu, Vincent Dumoulin, Pascal Lamblin, Utku Evci, Kelvin Xu, Ross Goroshin, Carles Gelada, Kevin Swersky, Pierre-Antoine Manzagol, et al. Meta-dataset: A dataset of datasets for learning to learn from few examples. arXie preprint 2xXi::1933000000 2019.
Petar Velickovic, William Fedus, William I, Hamilton, Pietro Liò, Yoshua Bengio, and R Devon Hjelm. Deep graph infomax. In ICLR, 2019.
Oriol Vinyals, Charles Blundell, Timothy Lillicrap, Daan Wierstra, et al. Matching networks for one shot learning. In NeurIPS, 2016.
Song Wang, Kaize Ding, Chuxu Zhang, Chen Chen, and Jundong Li. Task-adapa few-shot node classification. Tn KDD, 2022.
13

---

Published in Transactions on Machine Learning Research (06/2023)
Yaqing Wang, Abulikemu Abuduweili, Quamming Yao, and Dejing Dou. Property-aware relation networks for few-shot molecular property prediction. In NewrIPS, 2021.
Felix Wu, Amauri Souza, Tianyi Zhang, Christopher Fifty, Tao Yu, and Kilian Weinberger. Simplifying graph convolutional networks. In ICML, 2019.
Zhang Xinyi and Lihui Chen. Capsule graph neural network. In JCLR, 2018.
Wenhan Xiong, Mo Yu, Shiyu Chang, Xiaoxiao Guo, and William Yang Wang. One-shot relational learning for knowledge graphs. In EMNLP, 2018.
Keyulu Xu, Weihua Hu, Jure Leskovec, and Stefanie Jegelka. How powerful are graph neural networks? In ICLR, 2019.
Huaxiu Yao, Chuxu Zhang, Ying Wei, Meng Jiang, Suhang Wang, Junzhou Huang, Nitesh Chawla, and Zhenhui Li. Graph few-shot learning via knowledge transfer. In AAAI, 2020.
Rex Ying, Ruining He, Kaifeng Chen, Pong Eksombatchai, William L Hamilton, and Jure Leskovec. Graph convolutional neural networks for web-scale recommender systems. In KDD, 2018.
Jiaxuan You, Rex Ying, Xiang Ren, William Hamilton, and Jure Leskovec. Graphrmm: Generating realistic graphs with deep auto-regressive models. In ICML, 2018.
Yuning You, Tianlong Chen, Yongduo Sui, Ting Chen, Zhangyang Wang, and Yang Shen. Graph contrastive learning with augmentations. In NewrIPS, 2020.
Lu Yu, Shichao Pei, Lizhong Ding, Jun Zhou, Longfei Li, Chuxu Zhang, and Xiangliang Zhang. Sail: Self-angmented graph contrastive learning. In AAI, pp. 8927-8933 2022.
Han Yue, Chunhui Zhang, Chuxu Zhang, and Hongfu Liu. Label-invariant augmentation for semi-supervised graph classification. In NewrIPS, 2022.
Chunhui Zhang, Chao Huang, Youhuan Li, Xiangliang Zhang, Yanfang Ye, and Chuxu Zhang, Look twice much as you say: Scene graph contrastive learning for self-supervieed image caption generation. In CIKM, 2022a.
Chunhui Zhang, Chao Huang, Yijun Tian, Qianlong Wen, Zhongyu Ouyang, Youhuan Li, Yanfang Ye, and Chuxu Zhang. When sparsity meets contrastive models: Less graph data bring better class-balanced representations. In ICML, 2023a.
Chuxu Zhang, Dongjin Song, Chao Huang, Ananthram Swami, and Nitesh V Chawla. Heterogeneous graph neural network. In KDD, 2019.
Chuxu Zhang, Huaxiu Yao, Chao Huang, Meng Jiang, Zhenhui Li, and Nitesh V Chawla. Few-shot knowledge graph completion. In AAAI, 2020a.
Chuxu Zhang, Lu Yu, Mandana Sacbi, Meng Jiang, and Nitesh Chawla. Few-shot multi-hop relation reasoning over knowledge bases. In FMNLP, 2020b.
Chuxu Zhang, Kaize Ding, Jundong Li, Xiangliang Zhang, Yanfang Ye, Nitesh V Chawla, and Huan Liu. Few-shot learning on graphs. In IJCAI, 2022b.
Qiannan Zhang, Xiaodong Wu, Qiang Yang, Chuxu Zhang, and Xiangliang Zhang. Hg-meta: Graph meta-learning over heterogeneous graphs. In SDM, 2022c.
Qiannan Zhang, Shichao Pei, Qiang Yang. Chuxu Zhang, Nitesh Chawla, and Xiangliang Zhang. Cross-domain few-shot graph classification with a reinforced task coordinat. In AAAI, 2023b.
Yiming Zhang, Yiyue Qian, Yanfang Ye, and Chuxu Zhang. Adapting distilled knowledge for few-shot relation reasoning over knowledge graphs. In SDM, 2022d.
14

---

Published in Transactions on Machine Learning Research (06/2023)
Jianan Zhao, Qianlong Wen, Shiyu Sun, Yanfang Ye, and Chuxu Zhang. Multi-view self-supervised heteroge- neous graph embedding. In ECML/PKDD, 2021.
Fan Zhou, Chengtai Cao, Kumpeng Zhang, Goce Trajcevski, Ting Zhong, and Ji Geng. Meta-gnn: On few-shot node classification in graph meta-learning. In CIKM, 2019.
Yanqiao Zhu, Yichen Xu, Feng Yu, Qiang Liu, Shu Wu, and Liang Wang. Graph Contrastive Learning with Adaptive Augmentation. WWW, 2021.
Marinka Zitnik and Jure Leskovec. Predicting multicellular function through multi-layer tissue networks. Bioinformatics, 2017.
15

---

Published in Transactions on Machine Learning Research (06/2023)
A Dataset Details
We evaluate various models for few-shot node classification using five diverse datasets, including ogbn- arxiv (Hu et al., 2020a), Tissue-PPI (Hamilton et al., 2017), Fold-PPI (Zitnik & Leskovoe, 2017), Cora (Sen et al., 2008), and Citeseer (Sen et al.; 2008). These datasets range from citation networks to biochemical graphs. We provide detailed information on these datasets Table 5.
Table 5: Statistics of datasets used in the node classification task.
Dataset
# Graph
# Node
# Edge
# Feat.
# Label
ogbn-arxiv
1
169,343
1,166,243
128
40
l'ssse-PPI
24
51,194
1,350,412
50
10
Fold-PPI
144
274,606
3,666,563
Cora
1
2,708
10,556
1,433
7
Citeseer
1
3,327
9,228
3,703
6
For few-shot graph classification, we employ four distinct datasets (Chauhan et al., 2020); Reddit- 12K, ENZYMES, Letter-High, and TRIANGLES, to conduct extensive empirical evaluations of various modcls. These datasets exhibit variations in terms of average graph size, ranging from small (e.g., ketter-High) large (e.g., Reddit-12K). The statistics of datasets reported in Table 6.
Table 6: Statistics of datasets used in the graph classification task.
Class #
Graph #
Dataset
Train
Test
Training
Validation
Test
Letter-High
1,330
320
600
Triangles
7
3
1,126
271
603
Reddit-122
566
141
404
Enzymes
4
2
320
80
200
B Baseline Method Details
B.1 Node Classification
Graph embedding models:
mode2vec (Grover & Leskovec, 2016): We use node2vec to generate node embeddings, then employ FC layer as predictor to classify nodes. We use the code at this link."
Deep Walk (Perozzi et al., 2014): Similar to node2vec, we use DeepWalk to generate node embeddings, then employ an FC layer as predictor classify nodes. We use the code at this link.
GNN-based models:
Meta- Meta-GNN (Zhou et al., 2019): It combines MAML and simple graph convolution (SGC) to learn node enbeddings. We use the code at this link.
FS-GIN (Xu et al., 2019): This method uses GIN to learn node embeddings and only uses few-shot nodes lo propagate loss and enable training. We use the code for GIN backbone at this link.*
FS-SGC (Wu et al,, 2019): This model is similar FS-GIN while changing GIN SGC as GNN backbone. We the code of SGC this link.3
hhtpps://hootuuraaal
httpp:///ggtuub.om///eeeeeee Mcta-GNN
hhttps//ggthuub.cmm//eeeiet 16 poweerul-wgnnn 5 https://ghhabbcmm///iiggg/////
16

---

Published in Transactions on Machine Learning Research (06/2023)
No-Finctune (Huang & Zitnik, 2020): This method trains GCN the support set and uses the trained backbone to classify samples in the meta-testing set. We use the code of GCN at this link.
Finetune (Triantafillou et al., 2019): This method trains GCN on the meta-training set, and the model is fine-tuned on the meta-testing set. We use the code of GCN at this link." 6
KNN (Triantafillon et al., 2019): This method trains a GNN on meta-training set. Then, uses the label of the K-closest examples in the support set for each query example. We use the related code this link.?
ProtoNet (Triantafillou et al., 2019): This method applies prototypical network on node embeddings processed by a neural network, which is trained under the standard meta-learning setting, We use the related code at this link.
MAML (Finn et al., 2017): It similar to ProtoNet but changes meta-learner from ProtoNet to MAML. We use the code at this link."
G-Mela (Huang & Zilmik; 2020): This is strong baseline for few-shot node classification. It uses GCN GNN backbone to learn node embeddings based local subgraphs. It further combines prototypical loss and MAML for model training. We use the code at this link."
TENT (Wang el al., 2022): This is also a state-of-the-art baseline for few-shot node classification. Tt. proposes task-adase adaptive node classification framework to make node-level, class-level, and task-level adaptations. We utilize the code at this link."
B.2 Graph Classification
Graph embedding models:
WL (Shervashidze al., 2011): 1t uses KNN search on the output embeddings of WL. We use the code of WL at this link."!
Grophlet (Shervashidze et al., 2009): Tt uses Graphlet Kernel to decompose graph and generates graph embeddings. We use the code of Graphlet at this link.
AWE (Ivanov & Burnaev, 2018): It uses KNN search on the output embeddings AWE. We use the code of AWE at this link. 12
Graph22 Vee (Narayanon et al., 2017): This method applies KNN search on the output embeddings of Graph2Vec. We use the code of Graph2Vec at this link." 12
GNN-based models:
Diffpool (Lee el al., 2019): It uses Difspool with supervised loss to generate graph embeddings. We use the code of Diffpool at this link." 13
CapsGNN (Xinyi Chen, 2018): This method applics CapsGNN generate graph emboddings with supervised training. We use the code of CapsGNN backbone at this link. 11
GIN (Xu et al., 2019): This model applies GIN to generate graph embeddings with supervised training, We use the code of GIN backbone at this link.4
GIN-KNN (Xu et al., 2019): Similarly, this model implements GIN to generate graph embeddings while it switches the MLP classifier to the KNN algorithm. We use the code of GIN backbone at this link.
hhttps//shorturrla//////P
Thttps:/ httpp:://ssorturraa///1000
17

---

Published in Transactions on Machine Learning Research (06/2023)
GSM-GCN (Chauhan et al., 2020): This is strong model (with GCN backbone) for few-shot graph classification. We follow the default settings in the original paper and use the code at this link.
GSM-GAT (Chauhan et al., 2020): This is strong model (with GAT as backbone) for few-shot graph classification. We follow the default settings in the original paper and use the code at this link. 15
AS-MAML (Ma et al., 2020): It a state-of-the-art model for few-shot graph classification. We follow the default settings in the original paper and use the code at this link.
C Additional Experiment Results
The following section presents supplementary experiment outcomes on three extra datasets for few-shot node classification (Fold-PPI, Cora, and Citeseer) and two additional datasets for few-shot graph classification (Reddit-12K and Enzymes).
C.1 Few-Shot Node Classification Results
Impact of Shot Number. As shown in Figure 6, we present our model's performance on the other three datasets (Fold-PPI, Cora, and Citeseer) under different shot numbers (1 to 5) compared with selected baseline methods. It is evident that SDGCL outperforms bascline models consistently across various shot numbers, demonstrating its effectiveness and generalization ability.
Fold-PPI
Cora
Citeseer
70
90
80
75
Accuracy
55
Accuracy
75
Accuracy
65
40
60
55
G-M
G-Meta
G-Meta
ProtoNet
ProtoNet
ProtoNet
45
2
1
2
3
5
2
5
Shot number
Shot number
Shot number
Figure 6: Impact of shot number on node classification.
Impact of Training Label Rate. Figure 7 displays the performance of our model under different training label rates compared to baseline models for 3-shot node classification. It is evident that our model achieves better accuracy across different label rates.
Impact of Data Augmentation. Figure 8 presents the impact of graph augmentation on node classification. It can be observed that the combination of three augmentation strategies yields the best performance.
Discarded Information Comparison. The discarded graph information of different models shown in Table 7. Obviously, the amount of information discarded by SDGCL is less than that by the baseline models.
C.2 Few-Shot Graph Classification Results
Impact of Shot Number. Figure 9 shows the performance of SDGCL under different shot numbers (5, 10, 15, 20) compared to some selected baselines. From this figure, it evident that our model consistently outperforms bascline methods across different shot numbers.
18

---

Published in Transactions on Machine Learning Research (06/2023)
Fold-PPI
Cora
Citeseer
70
85
75
60
75
65
Accuracy
Accuracy
Accuracy
55
45
SDGFL G-Meta
G-Meta
45
G-Meta
ProtoNet
ProtoNet
ProtoNet
35
1020 30 40
100
10 30 40
100
10 20
30 40
100
Label rate
Label rate
Label rate
Figure Impact of training label rate on node classification.
ND
FM
ER+FM ND+FM
ND+ER+FM
ER
ND+ER
67
81
73
65
79
72
Accuracy
Accuracy
Accuracy
77
71
75
70
69
Fold-PPI
Cora
Citeseer
Figure 8: Impact of data augmentation on node classification.
Impact of Training Label Rate. Figure 10 presents the performance of SDGCL under different training label rates for 5-shot graph classification, compared to baseline models. As can be from the figure, SDGCL achieves better performance across different label rates.
Table 7: Discarded information for node classification
Discarded Information
Method
Layer
Fold-PPI
Cora
Citeseer
GNN-1
618.52
361.70
134.20
Meta-GNN
GNN-2
612.11
357.47
126.75
FC
591.13
327.49
116.55
GNN-1
585.31
356.57
129.04
G-Meta
GNN-2
592.38
355.63
119.72
FC
553.25
318.87
100.10
GNN-1
509.10
340.72
111.90
SDGCL-I
GNN-2
511.10
326.35
105.01
FC
461.52
316.21
92.95
GNN-1
426.23
332.39
104.04
SDGCL-T
GNN-2
418.90
314.65
88.33
FC
384.23
306.98
78.25
19

---

Published in Transactions on Machine Learning Research (06/2023)
Reddit-12K
Enzymes
55
65
50
Accuracy
Accuracy
60
45
55
40
Ours
Ours
GSM-GAT
GSM-GCN
GSM-GCN
AS-MAML
AS-MAML
20
10
15
Shot number
Shot number
Figure 9: Impact of shot number graph classification.
Reddit-12K
Enzymes
50
65
45
60
Accuracy
Accuracy
55
35
50
SDGFL
SDGFL
30
GSM-GAT
45
GSM-GCN
GSM-GCN
AS-MAML
AS-MAML
40
10 10 30 40
100
10 20 50 Label rate
100
Label rate
Figure 10: Impact of training label rate on graph classification.
The effect of graph augmentation on graph classification is demonstrated in Figure 11. We observe that combining the three augmentation strategies results in the best performance.
Impact of Data Augmentation:
Discarded Information Comparison: We present the discarded graph information of various methods in Table 8. The table shows that SDGCL discards less information compared to the baseline models.
ND
ER+FM
ND+ER+FM
ER
ND+ER
ND+FM
Accuracy
Accuracy
40
53
38
51
Reddit-12K
Enzymes
Figure 11: Impact of data augmentation graph classification.
20

---

Published in Transactions on Machine Learning Research (06/2023)
Table 8: Discarded information for graph classification.
Discarded Information
Method
Layer
Reddit-12K
Enzymes
GNN-1
5455.46
2455.54
AS-MAML
GNN-2
5024.15
2243.25
FC
4937.99
2125.91
GNN-1
5401.30
2273.80
GSM-GAT
GNN-2
5183.87
2128.0M
FC
4936.76
1958.18
GNN-1
5323.04
1864.98
SDGCL-I
GNN-2
5098.54
1788.97
FC
4802.03
1759.87
GNN-1
4823.91
1787.98
SDGCL-T
GNN-2
4546.23
1698.35 35
FC
4329.39
1585.33
21


Reviews:
[{"review_id": "VHwHtiXDuZ", "reviewer": "Reviewer_Ywqz", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "**Strengths**:\n\n- The paper is generally well writte, clearly structured and quite easy to follow.\n- The empirical results seem good on most of the performed tasks.\n\n\n**Weaknesses**:\n\n- The paper seems to be a simple ad-hoc combintation of contrastive learning and meta-learning on graphs. Most importantly, why these two methods are suited for each other is not well explained and justified.\n- I think the roles of contrastive pretraining and meta-learning are somehow contraditive. Both methods are trying to learn some kind of initialization (i.e. an initial model). Why they are combined togethor needs more justifications.\n- The distillation stage also seems questionable. I don't see why we actually need this. Could you elaborate on this?\n- Many important ablation studies are missing. Is every single component (contrastive pretraining, distillation and meta-learning) in this pipeline really necessary? It is not convincing if there is no ablation on these components.\n- Since the novelty is not in specific components of this pipeline, why this pipeline can improve performance requires deeper understanding. It will strengthen the paper if the authors can provide a theoretical justification or a comprehensive empirical study.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper proposes a general procedure for training graph neural networks for few-shot node/graph classification. The first stage is constrastive pretraining, followed by the model-agnostic meta learning as the second stage. Empirical imporvements over some baselines are observed.", "claims_and_evidence_yn": "No", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "See the weakness section.", "broader_impact_concerns": "n/a", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "BdUaYmqRxX", "reviewer": "Reviewer_tbQa", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "# Strengths\n\n* The authors did comprehensive experiments and ablation studies. I appreciate that.\n* Paper is well-written and well-organized.\n\n# Weaknesses\n1. I completely don’t understand the purpose of the self-distillation step.\n2. Why not use the teacher model for the meta-training phase? I know authors studied teacher-only in Tables 1 and 3, but why does the self-distillation step can enhance the pre-training, and in what sense? I carefully check the paper and I cannot find lines that specifically explain the reason. If I missed anything please correct me.\n3. What does this line mean? “Both pre-training and the distillation steps can operate at the meta-training and meta-testing phases without requiring labeled data.” I thought the pre-training/self-distillation phase and the meta-learning/testing phase are completely separated?\n4. I think the MAML-based Graph Few-Shot Learning is mainly a plug-and-play study and is not novel.\n5. The \"stop grad\" part in contrastive learning is not new. The authors should at least properly cite and give credit to [1].\n\n[1] \"Bootstrap Your Own Latent A New Approach to Self-Supervised Learning\" 2020", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper proposes a novel framework called Self-Distilled Graph Few-shot Learning (SDGFL) to address the issue of label scarcity in deep graph learning models. SDGFL uses a self-distilled contrastive learning procedure to pre-train a graph encoder with unlabeled data, which is then used to distill a student model with a contrastive loss. This distilled model is fed to GFL to enhance generalization ability and mitigate the distribution gap. The proposed framework is task and data-independent, making it a versatile tool for general graph mining purposes. The effectiveness of SDGFL is evaluated using an information-based measurement, and comprehensive experiments show that it outperforms state-of-the-art baselines on various graph mining tasks in the few-shot scenario.", "claims_and_evidence_yn": "No", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "To be accepted, I think the bottom line is to provide a convincing rationale and motivation about why using the self-distillation step.", "broader_impact_concerns": "I think this work on pretraining + meta-learning for graph few-shot learning in general has an impact on the GNN community.\nI do have concerns about the novelty of the method (both contrastive/distillation and MAML part), but I suppose it is a bit hard to improve in a short period of time.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "XDSTvJCUey", "reviewer": "Reviewer_D1oC", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strength:\n1. The target issues of the paper are meaningful and worth exploring. The motivation is clear. This submission gives a valuable implementation of such an idea and presents good results.\n2. The proposed method has been validated on different datasets and tasks.\nWeakness:\n1.\tThe proposed method is not novel, and a similar self-supervised graph learning method has already been explored in Infograph[A] and GCA[B]. This paper only uses the distillation method for improving the GFL.\n2.\tThe paper lacks experiments in DBLP and Amazon-E. The compared method TENT in Table 1 has been evaluated in these datasets. More comparison with TENT is needed.\n3.\tIn Table 1, TENT performs better in both 3-shot and 5-shot settings, more explanation is needed. \n\n[A] Fan-Yun Sun, Jordan Hoffmann, Vikas Verma, and Jian Tang. Infograph: Unsupervised and semi-supervised graph-level representation learning via mutual information maximization. In ICLR, 2020.\n[B] Yanqiao Zhu, Yichen Xu, Feng Yu, Qiang Liu, Shu Wu, and Liang Wang. Graph Contrastive Learning with Adaptive Augmentation. In WWW, 2021.\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper proposes an SDGCL framework that overcomes the negative impacts of the distribution shift. SDGCL is task and data-independent, which can be used for general graph mining purposes. An information-based method is also proposed to evaluate the quality of learned embeddings of the model. This paper conducts extensive experiments to demonstrate the effectiveness of the paper.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "1.\tThe paper should give a more explanation of the mentioned weakness.", "broader_impact_concerns": "None", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "bGQMalkpwy", "reviewer": "Reviewer_Gaj3", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "## Strengths\n- The practical design and combination of self-distillation contrastive learning strategies seem to be effective for Graph Few-shot learning. \n- A new measurement is proposed to evaluate graph few-shot representation. \n- The experiments are generally comprehensive and somehow verify the effectiveness of the proposed method for two settings and the proposed measurement. \n- The paper is easy to follow. \n\n## Weaknesses\n- Since this paper re-designs/modifies existing contrastive learning techniques which have already been verified effective in image classification/graph learning. The technical contribution is not quite significant. But overall, it is also good to understand contrastive learning especially self-distillation behaviors in a few-shot graph context. \n- My main concern is the unfair and insufficient comparisons in the experimental section. \n> - In Table 1 and Table 2, the proposed method has both Inductive and Transductive settings, but all baselines are not marked 'I' or 'T.' Since the Transductive setting has been proven to improve the few-shot performance (Liu et al. ICLR 2019), this is unfair without indicating the I or T of the baselines.   \n> - In Figure 2, TENT is not compared, which is the best baseline method of Table 1.   \n> - In Table 2, TENT is not compared.   \n> - In Table 4, the strong baseline ASMAML is not compared.   \n- Typo or format issues.\n> - The second paragraph of Related work starts from a new line, while the first paragraph does not. It is better to be consistent. \n> - ``generative or contrastive, according to their learning tasks (Liu et al., 2020; Sohn et al., 2020)..\"\n> - In Table 1, ``The best results are highlighted in bold while the best baseline results are underlined.\". This is not the case. Please fix in Table 1. ", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper deals with the Graph Few-shot Learning (GFL) problem with the help of self-supervised contrastive learning. \n\nSpecifically, self-supervised contrastive graph learning makes use of existing contrastive learning good practices (Grill et al., 2020; Furlanello et al., 2018; You et al., 2020). The process consists of two stages: pure self-supervised learning and self-distillation graph contrastive learning.   \nAfter self-supervised learning, general graph few-shot learning techniques are used for the problem utilizing MAML for the graph few-shot learning phase.   \nMoreover, a Quantitative Measurement of GFL using Mutual information is proposed to evaluate the layerwise information for the learned representation model. \n\nThe technique contributions lie in (1) designing and varying the effectiveness of self-distillation contrastive learning in the Graph Few-shot Learning scenario, (2) quantitative measurement of GFL utilizing Mutual information. \n\nExperimentally, the proposed method is conducted on two problem settings: node classification and graph classification. Some of the experiments verify the effectiveness. \n", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "Please see the above experimental weaknesses and typos. ", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Decision: Accept (poster)