Few-shot Learning via Dirichlet Tessellation Ensemble

1 INTRODUCTION . Recent years have witnessed a tremendous success of deep learning in a number of data-intensive applications ; one critical reason for which is the vast collection of hand-annotated high-quality data , such as the millions of natural images for visual object recognition ( Deng et al. , 2009 ) . However , in many real-world applications , such large-scale data acquisition might be difficult and comes at a premium , such as in rare disease diagnosis ( Yoo et al. , 2021 ) and drug discovery ( Ma et al. , 2021b ; 2018 ) . As a consequence , Few-shot Learning ( FSL ) has recently drawn growing interests ( Wang et al. , 2020 ) . Generally , few-shot learning algorithms can be categorized into two types , namely inductive and transductive , depending on whether estimating the distribution of query samples is allowed . A typical transductive FSL algorithm learns to propagate labels among a larger pool of query samples in a semi-supervised manner ( Liu et al. , 2019 ) ; notwithstanding its normally higher performance , in many real world scenarios a query sample ( e.g . patient ) also comes individually and is unique , for instance , in personalized pharmacogenomics ( Sharifi-Noghabi et al. , 2020 ) . Thus , we in this paper adhere to the inductive setting and make on-the-fly prediction for each newly seen sample . Few-shot learning is challenging and substantially different from conventional deep learning , and has been tackled by many researchers from a wide variety of angles . Despite the extensive research All four authors are corresponding authors . on the algorithmic aspects of FSL ( see Sec . 2 ) , two challenges still pose an obstacle to successful FSL : ( 1 ) how to sufficiently compensate for the data deficiency in FSL ? and ( 2 ) how to make the most use of the base samples and the pre-trained model ? For the first question , data augmentation has been a successful approach to expand the size of data , either by Generative Adversarial Networks ( GANs ) ( Goodfellow et al. , 2014 ) ( Li et al. , 2020b ; Zhang et al. , 2018 ) or by variational autoencoders ( VAEs ) ( Kingma & Welling , 2014 ) ( Zhang et al. , 2019 ; Chen et al. , 2019b ) . However , in each way , the authenticity of either the augmented data or the feature is not guaranteed , and the out-of-distribution hallucinated samples ( Ma et al. , 2019 ) may hinder the subsequent FSL . Recently , Liu et al . ( 2020b ) and Ni et al . ( 2021 ) investigate supportlevel , query-level , task-level , and shot-level augmentation for meta-learning , but the diversity of FSL models has not been taken into consideration . For the second question , Yang et al . ( 2021 ) borrows the top-2 nearest base classes for each novel sample to calibrate its distribution and to generate more novel samples . However , when there is no proximal base class , this calibration may utterly alter the distribution . Another line of work ( Sbai et al. , 2020 ; Zhou et al. , 2020 ) learns to select and design base classes for a better discrimination on novel classes , which all introduce extra training burden . As a matter of fact , we still lack a method that makes full use of the base classes and the pretrained model effectively . In this paper , we study the FSL problem from a geometric point of view . In metric-based FSL , despite being surprisingly simple , the nearest neighbor-like approaches , e.g . ProtoNet ( Snell et al. , 2017 ) and SimpleShot ( Wang et al. , 2019 ) , have achieved remarkable performance that is even better than many sophisticatedly designed methods . Geometrically , what a nearest neighbor-based method does , under the hood , is to partition the feature space into a Voronoi Diagram ( VD ) that is induced by the feature centroids of the novel classes . Although it is highly efficient and simple , Voronoi Diagrams coarsely draw the decision boundary by linear bisectors separating two centers , and may lack the ability to subtly delineate the geometric structure arises in FSL . To resolve this issue , we adopt a novel technique called Cluster-induced Voronoi Diagram ( CIVD ) ( Chen et al. , 2013 ; 2017 ; Huang & Xu , 2020 ; Huang et al. , 2021 ) , which is a recent breakthrough in computation geometry . CIVD generalizes VD from a point-to-point distance-based diagram to a cluster-to-point influence-based structure . It enables us to determine the dominating region ( or Voronoi cell ) not only for a point ( e.g . a class prototype ) but also for a cluster of points , guaranteed to have a ( 1 + ) -approximation with a nearly linear size of diagram for a wide range of locally dominating influence functions . CIVD provides us a mathematically elegant framework to depict the feature space and draw the decision boundary more precisely than VD without losing the resistance to overfitting . Accordingly , in this paper , we show how CIVD is used to improve multiple stages of FSL and make several contributions as follows . 1 . We first categorize different types of few-shot classifiers as different variants of Voronoi Diagram : nearest neighbor model as Voronoi Diagram , linear classifier as Power Diagram , and cosine classifier as spherical Voronoi Diagram ( Table 1 ) . We then unify them via CIVD that enjoys the advantages of multiple models , either parametric or nonparametric ( denoted as DeepVoro -- ) . 2 . Going from cluster-to-point to cluster-to-cluster influence , we further propose Cluster-to-cluster Voronoi Diagram ( CCVD ) , as a natural extension of CIVD . Based on CCVD , we present DeepVoro which enables fast geometric ensemble of a large pool of thousands of configurations for FSL . 3 . Instead of using base classes for distribution calibration and data augmentation ( Yang et al. , 2021 ) , we propose a novel surrogate representation , the collection of similarities to base classes , and thus promote DeepVoro to DeepVoro++ that integrates feature-level , transformation-level , and geometry-level heterogeneities in FSL . Extensive experiments have shown that , although a fixed feature extractor is used without independently pretrained or epoch-wise models , our method achieves new state-of-the-art results on all three benchmark datasets including mini-ImageNet , CUB , and tiered-ImageNet , and improves by up to 2.18 % on 5-shot classification , 2.53 % on 1-shot classification , and up to 5.55 % with different network architectures . 2 RELATED WORK . Few-Shot Learning . There are a number of different lines of research dedicated to FSL . ( 1 ) Metricbased methods employ a certain distance function ( cosine distance ( Mangla et al. , 2020 ; Xu et al. , 2021 ) , Euclidean distance ( Wang et al. , 2019 ; Snell et al. , 2017 ) , or Earth Mover ’ s Distance ( Zhang et al. , 2020a ; b ) ) to bypass the optimization and avoid possible overfitting . ( 2 ) Optimization-based approaches ( Finn et al. , 2017 ) manages to learn a good model initialization that accelerates the optimization in the meta-testing stage . ( 3 ) Self-supervised-based ( Zhang et al. , 2021b ; Mangla et al. , 2020 ) methods incorporate supervision from data itself to learn a robuster feature extractor . ( 4 ) Ensemble method is another powerful technique that boosting the performance by integrating multiple models ( Ma et al. , 2021a ) . For example , Dvornik et al . ( 2019 ) trains several networks simultaneously and encourages robustness and cooperation among them . However , due to the high computation load of training deep models , this ensemble is restricted by the number of networks which is typically < 20 . In Liu et al . ( 2020c ) , instead , the ensemble consists of models learned at each epoch , which , may potentially limit the diversity of ensemble members . Geometric Understanding of Deep Learning . The geometric structure of deep neural networks is first hinted at by Raghu et al . ( 2017 ) who reveals that piecewise linear activations subdivide input space into convex polytopes . Then , Balestriero et al . ( 2019 ) points out that the exact structure is a Power Diagram ( Aurenhammer , 1987 ) which is subsequently applied upon recurrent neural network ( Wang et al. , 2018 ) and generative model ( Balestriero et al. , 2020 ) . The Power/Voronoi Diagram subdivision , however , is not necessarily the optimal model for describing feature space . Recently , Chen et al . ( 2013 ; 2017 ) ; Huang et al . ( 2021 ) uses an influence function F ( C , z ) to measure the joint influence of all objects in C on a query z to build a Cluster-induced Voronoi Diagram ( CIVD ) . In this paper , we utilize CIVD to magnify the expressivity of geometric modeling for FSL . 3 METHODOLOGY . 3.1 PRELIMINARIES . Few-shot learning aims at discriminating between novel classes Cnovel with the aid of a larger amount of samples from base classes Cbase , Cnovel∩Cbase = ∅ . The whole learning process usually follows the meta-learning scheme . Formally , given a dataset of base classes D = { ( xi , yi ) } , xi ∈ D , yi ∈ Cbase with D being an arbitrary domain e.g . natural image , a deep neural network z = φ ( x ) , z ∈ Rn , which maps from image domain D to feature domain Rn , is trained using standard gradient descent algorithm , and after which φ is fixed as a feature extractor . This process is referred to as metatraining stage that squeezes out the commonsense knowledge from D. For a fair evaluation of the learning performance on a few samples , the meta-testing stage is typically formulated as a series of K-way N -shot tasks ( episodes ) { T } . Each such episode is further decomposed into a support set S = { ( xi , yi ) } K×Ni=1 , yi ∈ CT and a query set Q = { ( xi , yi ) } K×Q i=1 , yi ∈ CT , in which the episode classes CT is a randomly sampled subset of Cnovel with cardinality K , and each class contains onlyN andQ random samples in the support set and query set , respectively . For few-shot classification , we introduce here two widely used schemes as follows . For simplicity , all samples here are from S and Q , without data augmentation applied . Nearest Neighbor Classifier ( Nonparametric ) . In Snell et al . ( 2017 ) ; Wang et al . ( 2019 ) etc. , a prototype ck is acquired by averaging over all supporting features for a class k ∈ CT : ck = 1 N ∑ x∈S , y=k φ ( x ) ( 1 ) Then each query sample x ∈ Q is classified by finding the nearest prototype : ŷ = arg minkd ( z , ck ) = ||z − ck||22 , in which we use Euclidean distance for distance metric d. Linear Classifier ( Parametric ) . Another scheme uses a linear classifier with cross-entropy loss optimized on the supporting samples : L ( W , b ) = ∑ ( x , y ) ∈S − log p ( y|φ ( x ) ; W , b ) = ∑ ( x , y ) ∈S − log exp ( W Ty φ ( x ) + by ) ∑ k exp ( W T k φ ( x ) + bk ) ( 2 ) in which Wk , bk are the linear weight and bias for class k , and the predicted class for query x ∈ Q is ŷ = arg maxk p ( y|z ; Wk , bk ) .

Summary: This paper provides a new geometric point of view for few-shot learning (FSL). In this view, the widely used ProtoNet can be regarded as a Dirichlet Tessellation (Voronoi Diagram) in the feature space. Furthermore, the authors use the recent Cluster-induced Voronoi Diagram (CIVD) for FSL and propose an ensemble approach to achieve a stronger FSL model. Extensive experimental results on three standard benchmarks demonstrate the effectiveness of the proposed method.