FEW-SHOT LEARNING ON GRAPHS VIA SUPER-CLASSES BASED ON GRAPH SPECTRAL MEASURES

We propose to study the problem of few-shot graph classification in graph neural networks ( GNNs ) to recognize unseen classes , given limited labeled graph examples . Despite several interesting GNN variants being proposed recently for node and graph classification tasks , when faced with scarce labeled examples in the few-shot setting , these GNNs exhibit significant loss in classification performance . Here , we present an approach where a probability measure is assigned to each graph based on the spectrum of the graph ’ s normalized Laplacian . This enables us to accordingly cluster the graph base-labels associated with each graph into super-classes , where theLp Wasserstein distance serves as our underlying distance metric . Subsequently , a super-graph constructed based on the super-classes is then fed to our proposed GNN framework which exploits the latent inter-class relationships made explicit by the super-graph to achieve better class label separation among the graphs . We conduct exhaustive empirical evaluations of our proposed method and show that it outperforms both the adaptation of state-ofthe-art graph classification methods to few-shot scenario and our naive baseline GNNs . Additionally , we also extend and study the behavior of our method to semi-supervised and active learning scenarios . 1 INTRODUCTION . The need to analyze graph structured data coupled with the ubiquitous nature of graphs ( Borgwardt et al. , 2005 ; Duvenaud et al. , 2015 ; Backstrom & Leskovec , 2010 ; Chau et al. , 2011 ) , has given greater impetus to research interest in developing graph neural networks ( GNNs ) ( Defferrard et al. , 2016 ; Kipf & Welling , 2016 ; Hamilton et al. , 2017 ; Velikovi et al. , 2018 ) for learning tasks on such graphs . The overarching theme in GNNs is for each node ’ s feature vector to be generated by passing , transforming , and recursively aggregating feature information from a given k-hop neighborhood surrounding the node . However , GNNs still fall short in the ” few-shot ” learning setting , where the classifier must generalize well after seeing abundant base-class samples ( while training ) and very few ( or even zero ) samples from a novel class ( while testing ) . Given the scarcity and difficulty involved with generation of labeled graph samples , it becomes all the more important to solve the problem of graph classification in the few-shot setting . Limitations and challenges : Recent work by Xu et . al . ( Xu et al. , 2019 ) indicated that most recently proposed GNNs were designed based on empirical intuition and heuristic approaches . They studied the representational power of these GNNs and identified that most neighborhood aggregation and graph-pooling schemes had diminished discriminative power . They rectified this problem with the introduction of a novel injective neighborhood aggregation scheme , making it as strong as the Weisfeiler-Lehman ( WL ) graph isomorphism test ( Weisfeiler & Leman , 1968 ) . Nevertheless , the problem posed by extremely scarce novel-class samples in the few-shot setting remains to persist as a formidable challenge , as it requires more rounds of aggregation to affect larger neighborhoods and hence necessitate greater depth in the GNN . However , when it comes to GNNs , experimental studies have shown that an increase in the number of layers results in dramatic performance drops in GNNs ( Wu et al. , 2019 ; Li et al. , 2018b ) . Our work : Motivated by the aforementioned observations and challenges , our method does the following . We begin with a once-off preprocessing step . We assign a probability measure to each graph , which we refer to as a graph spectral measure ( similar to ( Gu et al. , 2015 ) ) , based on the spectrum of the graph ’ s normalized Laplacian matrix representation . Given this metric space of graph spectral measures and the underlying distance as the Lp Wasserstein distance , we compute Wasserstein barycenters ( Agueh & Carlier , 2011 ) for each set of graphs specific to a base class and term these barycenters as prototype graphs . With this set of prototype graphs for each base class label , we cluster the spectral measures associated with each prototype graph in Wasserstein space to create a super-class label . Utilizing this super-class information , we then build a graph of graphs called a super-graph . The intuition behind this is to exploit the non-explicit and latent inter-class relationships between graphs via their spectral measures and use a GNN on this to also introduce a relational inductive bias ( Battaglia et al. , 2018 ) , which in turn affords us an improved sample complexity and hence better combinatorial generalization given such few samples to begin with . Given , the super-classes and the super-graph , we train our proposed GNN model for few-shot learning on graphs . Our GNN consists of a graph isomorphism network ( GIN ) Xu et al . ( 2019 ) as a feature extractor Fθ ( . ) to generate graph embeddings ; on which subsequently acts our classifier C ( . ) comprising of two components : ( i ) Csup : a MLP layer to learn and predict the super class associated to a graph , and ( ii ) CGAT : a graph attention network ( GAT ) to predict the actual class label of a graph . The overall loss function is a sum of the cross-entropy losses associated with Csup and CGAT . We follow initialization based strategy ( Chen et al. , 2019 ) , with a training and fine-tuning phase , so that in the fine-tuning phase , the pre-trained parameters associated with Fθ ( . ) and Csup are frozen , and the few novel labeled graph samples are used to update the weights and attention learned by CGAT . Our contributions : To the best of our knowledge , we are the first to introduce few shot learning on graphs for graph classification . Next , we propose an architecture that makes use of the graph ’ s spectral measures to generate a set of super-classes and a super-graph to better model the latent relations between classes , followed by our GNN trained using an initialization method . Finally , we conduct extensive experiments to gain insight into our method . For example , in the 20-shot setting on the TRIANGLES dataset , our method shows a substantial improvement of nearly 7 % and 20 % over DL-based and unsupervised baselines , respectively . 2 RELATED WORK . Few-shot learning in the computer vision community was first introduced by ( Fei-Fei et al. , 2006 ) with the intuition that learning the underlying properties of the base classes given abundant samples can help generalize better to unseen classes with few-labeled samples available . Various learning algorithms have been proposed in the image domain , among which a broad category of initialization based methods aim to learn transferable knowledge from training classes , so that the model can be adapted to unseen classes with limited labeled examples ( Finn et al. , 2017 ) ; ( Rusu et al. , 2018 ) ; ( Nichol et al. , 2018 ) . Recently proposed and widely accepted Initialization based methods can broadly be classified into : ( i ) methods that learn good model parameters with limited labeled examples and a small number of gradient update steps ( Finn et al. , 2017 ) and ( ii ) methods that learn an optimizer ( Ravi & Larochelle , 2017 ) . We refer the interested reader to Chen et . al . ( Chen et al. , 2019 ) for more examples of few-shot learning methods in vision . Graph neural networks ( GNNs ) were first introduced in ( Gori et al. , 2005 ) ; ( Scarselli et al. , 2009 ) as recurrent message passing algorithms . Subsequent work ( Bruna et al. , 2014 ) ; ( Henaff et al. , 2015 ) proposed to learn smooth spectral multipliers of the graph Laplacian , but incurred higher computational cost . This computational bottleneck was later resolved ( Defferrard et al. , 2016 ) ; ( Kipf & Welling , 2016 ) by learning polynomials of the graph Laplacian . GNNs are a natural extension to Convolutional neural networks ( CNNs ) on non-Euclidean data . Recent work ( Velikovi et al. , 2018 ) introduced the concept of self-attention in GNNs , which allows each node to provide attention to the enclosing neighborhood resulting in improved learning . We refer the reader to ( Bronstein et al. , 2016 ) for detailed information on GNNs . Despite all the success of GNNs , few-shot classification remains an under-addressed problem . Some recent attempts have focused on solving the few-shot learning on graph data where GNNs are either trained via co-training and self-training ( Li et al. , 2018a ) , or extended by stacking transposed graph convolutional layers imputing a structural regularizer ( Zhang et al. , 2019 ) - however , both these works focus only on the node classification task . To the best of our knowledge , there does not exist any work pertaining few-shot learning on graphs focusing on the graph classification task , thus providing the motivation for this work . Comparison to few-shot learning on images : Few shot learning ( FSL ) has gained wide-spread traction in the image domain in recent years . However the success of FSL in images is not easily translated to the graph domain for the following reasons : ( a ) Images are typically represented in Euclidean space and thus can easily be manipulated and handled using well-known metrics like cosine similarity , Lp norms etc . However , graphs come from non-Euclidean domains and exhibit much more complex relationships and interdependency between objects . Furthermore , the notion of a distance between graphs is also not straightforward and requires construction of graph kernels or the use of standard metrics on graph embeddings ( Kriege et al. , 2019 ) . Additionally , such graph kernels dont capture higher order relations very well . ( b ) In the FSL setting on images , the number of training samples from various classes is also abundantly more than what is available for graph datasets . The image domain allows training generative models to learn the task distribution and can further be used to generate samples for data augmentation , which act as very good priors . In contrast , graph generative models are still in their infancy and work in very restricted settings . Furthermore , methods like cropping and rotation to improve the models can ’ t be used for graphs given the permutation invariant nature of graphs . Additionally , removal of any component from the graph can adversely affect its structural properties , such as in biological datasets . ( c ) The image domain has very well-known regularization methods ( e.g . Tikhonov , Lasso ) that help generalize much better to novel datasets . Although , they dont bring any extra supervised information and hence can not fully address the problem of FSL in the image domain . To the best of our knowledge , this is still an open research problem in the image domain . On the other hand , in the graph domain , our work would be a first step towards graph classification in an FSL setting , which would then hopefully pave the path for better FSL graph regularizers . ( d ) Transfer learning has led to substantial improvements on various image related tasks due to the high degree of transferability of feature extractors . Thus , downstream tasks like few-shot learning can be performed well with high quality feature extractor models , such as Resnet variants trained on Imagenet . Transfer learning , or for that matter even good feature extractors , remains a daunting challenge in the graph domain . For graphs , there neither exists a dataset which can serve as a pivot for high quality feature learning , nor does there exist a Graph NN which can capture the higher order relations between various categories of graphs , thus making this a highly challenging problem .

Summary: This paper proposed a few-shot graph classification algorithm based on graph neural networks. The learning is based on a large set of base class labeled graphs and a small set of novel class labeled graphs. The goal is to learn a classification algorithm over the novel class based on the sample from the base class and novel class. The learning process constitutes of the following steps. First, the base class is classified into K super classes based on the spectral embedding of the graph (onto distributions over the corresponding graph spectrum) and the k-means algorithm with the Wasserstein metric. Second, for each super class, the classification is done through a feature extractor and a classifier. In the training of the feature extractor and classifier, the author introduces a super-graph with each node representing a super class. Finally, in the fine-tuning stage, the feature extractor is fixed, and the classifier is trained based on the novel class.