Assessing two novel distance-based loss functions for few-shot image classification

1 INTRODUCTION . Despite the advances in deep learning research , it remains a challenge for the standard supervised learning to achieve satisfactory results when learning from just a small amount of labeled data . Current deep learning algorithms tend to overfit when they are given a small dataset for training , reducing their generalization capabilities . Moreover , there are many problem domains where obtaining labeled data can be very difficult or imply a lot of manual work to get the data with its ground truth , representing a problem for real world applications as it is time consuming and costly . Few-shot learning ( FSL ) methods has been proposed ( Koch et al . ( 2015 ) ; Snell et al . ( 2017 ) ; Vinyals et al . ( 2016 ) ; Sung et al . ( 2017 ) ; Finn et al . ( 2017 ) ; Nichol et al . ( 2018 ) ) to classify previously unseen data into a set of new classes , given only a small amount of labeled instances per class . The main challenge for FSL is to apply a fine-tuning process to an existing embedding network to adapt to new classes , with the problem that this could easily lead to overfitting due to the few labeled samples available for each class . There are two main FSL approaches : The first one is Meta-learning based methods ( Finn et al . ( 2017 ) ; Andrychowicz et al . ( 2016 ) ; Li & Malik ( 2016 ) ; Chen et al . ( 2017 ) ) , where the basic idea is to learn from diverse tasks and datasets and adapt the learned algorithm to novel datasets . The second are Metric-learning based methods ( Xing et al . ( 2003 ) ; Koch et al . ( 2015 ) ) , where the objective is to learn a pairwise similarity metric such that the score is high for similar samples and dissimilar samples get a low score . Later on , these metric learning methods started to adopt the meta learning policy to learn across tasks ( Snell et al . ( 2017 ) ; Vinyals et al . ( 2016 ) ; Sung et al . ( 2017 ) ) . The main objective of these methods is to learn an effective embedding network in order to extract useful features of the task and discriminate on the classes which we are trying to predict . From this basic learning setting , many extensions have been proposed to improve the performance of metric learning methods . Some of these works focus on pre-training the embedding network ( Chen et al . ( 2019a ) ) , others introduce task attention modules ( Chen et al . ( 2020 ) ; Li et al . ( 2019a ) ; Zheng et al . ( 2019 ) ) , whereas other try to optimize the embeddings ( Lee et al . ( 2019 ) ) and yet others try to use a variety of loss functions ( Zheng et al . ( 2019 ) ) . In this work , we propose two different loss functions based on the concepts of inter-class and intraclass distance . As showed in Figure 1 , these loss functions allow us to optimize the embedding network and learn more discriminative features across tasks . For one of the proposed loss functions , we take as inspiration one of the most widely used losses in metric-learning : the triplet loss . For the second one , we adopt an algorithm based on nearest neighbors distance . We demonstrate the effectiveness of our loss functions and show our competitive results compared with state-of-the-art methods . The rest of the paper is organized as follows : In Section 2 , we describe the related work in the few-shot learning problem . In Section 3 we described the proposed model . We first explain the Proto-triplet loss function , and then we explain the ICNN Loss function . Then we discuss some of the design choices we needed to take for the ICNN loss . In Section 4 we detail the experimental setup used for the implementation of the models . In Section 5 we show the results obtained and discuss about its performance . Finally , in Section 5 we present our conclusions and discuss the future work . 2 MOTIVATION AND RELATED WORK . 2.1 DEEP METRIC LEARNING . The goal of metric learning is to learn a similarity function from the data . More specifically , it aims to learn feature embeddings in a way that reduce the distance between embeddings corresponding to instances of the same class ( intra-class ) and increase the distance between embeddings corresponding to instances of different class . Deep metric learning uses an embedding network to learn the discriminative features that will be used to compute the similarity metric . Below we review the more relevant deep metric learning methods . One of the fundamental methods for metric learning is the Siamese Networks ( Koch et al . ( 2015 ) ) , which is a symmetric neural network architecture that consists on two subnetworks both having the same parameters . These networks learn its parameters by calculating a distance metric between the feature embeddings of each subnetwork each with a different input . The loss function used in Siamese Networks is the contrastive loss or pairwise ranking loss , which seeks for the distance of samples from the same class to be small and from different class to be large . The second important metric learning method is the Triplet Network ( Schroff et al . ( 2015 ) ) , which is also a symmetric neural network architecture but this method consist of three identical subnetworks sharing the same parameters . The input of the three subnetworks consist on three different images : The first one is the anchor ( the baseline image ) , the second is the positive sample ( an instance that belongs to the same class as the anchor ) , and the third is the negative sample ( an instance that belongs to a different class than the anchor ) . This network use the triplet loss to learn discriminative feature embeddings , and it works by ensuring that the anchor image is close to the positive images and far away from the negative images . These metric learning methods have been widely used for different purposes as image retrieval ( Wang et al . ( 2014 ) ) , face recognition ( Schroff et al . ( 2015 ) ; Taigman et al . ( 2014 ) ; Hu et al . ( 2014 ) ) , person re-identification ( Xiao et al . ( 2017 ) ) , video surveillance ( Huang et al . ( 2018 ) ) , threedimensional modelling ( Dai et al . ( 2017 ) ) , signature verification ( Bromley et al . ( 1993 ) ) , medical image analysis ( Annarumma & Montana ( 2017 ) ) , text understanding ( Mueller & Thyagarajan ( 2016 ) ; Benajiba et al . ( 2018 ) ) , among other problems . 2.2 META-LEARNING FOR FEW-SHOT LEARNING . As deep learning started to produce good results in many machine learning problems , some works proposed to use the meta-learning policy in order to optimize deep models Andrychowicz et al . ( 2016 ) ; Li & Malik ( 2016 ) ; Chen et al . ( 2017 ) . The meta-learning policy refers to learn across tasks and then adapt to new tasks , instead of learning to the level of samples . The meta-learning objective is to learn the parameters θ that minimize the loss across all tasks . Few-shot learning is the perfect process in which we can test meta-learning algorithms , because of the few-labeled data given to each task . The meta-learning approach to tackle a few-shot learning problem is divided into two stages : metatrain and meta-test . The meta-learning setup consists of episodic tasks , which can be seen as batches in traditional deep learning . A few-shot K-way C-shot image classification task is given K classes and C images per class . The task-specific dataset can be formulated as D = { Dtrain , Dtest } , where Dtrain = { ( Xi , yi ) } Ntraini=1 denotes the classes reserved for the training phase and Dtest = { ( Xi , yi ) } Ntesti=1 denotes the classes reserved for the testing phase . For each meta-train task T , K class labels are randomly chosen from Dtrain to form a support set and a query set . The support set , denoted by S , contains K × C samples ( K-way C-shot ) and the query set , denoted by Q , contains n number of randomly chosen samples from the K classes . The training phase use an episodic mechanism , where each episode E is loaded with a new random task taken from the training data . For the meta-test , the model is tested with a new task T constructed with classes that weren ’ t seen during the meta-train . We can summarize the few-shot learning methods based on what the model seeks to meta-learn . Some approaches consists on having a base-learner and a meta-learner , where meta-learner parameters are optimized by gradual learn across tasks to facilitate the fast learning of the base-learner for each specific task . MAML ( Finn et al . ( 2017 ) ) , is one of these methods and have the idea to search for a good parameter initialization such that the base learner can rapidly generalize with this initialization . Then , REPTILE ( Nichol et al . ( 2018 ) ) incorporates an L2 loss to simplify the computation of MAML . Further on , LEO ( Rusu et al . ( 2018 ) ) is proposed as a network to learn low dimension latent embedding of the model . Meta-SGD ( Li et al . ( 2017 ) ) also learns the base learner update direction and learning rate on the meta-learning process . Meta-Leearner LSTM ( Ravi & Larochelle ( 2017 ) ) propose to finetune the base learner by a LSTM-based meta-learner , which takes as input the loss and gradient of base learner with respect to each support sample . Other approaches seek to learn the similarity metric that is expected to be transferrable across different tasks . 2.3 METRIC META-LEARNING FOR FEW-SHOT LEARNING . There is a whole branch of meta-learning approaches to solve the few-shot learning problem by inheriting the main idea of metric learning . These approaches adopts the meta-learning setup to learn the similarity metric expected to generalize across different tasks . There are baseline methods which achieved important milestones for few-shot learning , such as Prototypical Networks ( Snell et al . ( 2017 ) ) , Matching Networks ( Vinyals et al . ( 2016 ) ) and Relation Networks ( Sung et al . ( 2017 ) ) . Prototypical Networks is the model which we are using as a basis , and it works by taking the center of support samples ’ embeddings from each class to create the class prototypes . Then , the model use a distance metric ( typically the euclidean distance ) to predict the probabilities for each query sample . The Matching Networks predicts the probability of query samples by measuring the cosine similarity between the query embedding and each support sample embedding . The Relation Networks adopts a learnable CNN as the pairwse similarity metric , which takes the concatenation of feature maps of support sample and query sample as input and outputs the relation score . These three methods can be considered as the base metric learning approaches for few-shot learning . Further on , some recent works focus on introducing task attention modules ( Chen et al . ( 2020 ) ; Li et al . ( 2019a ) ; Zheng et al . ( 2019 ) ) , whereas others try to optimize the embeddings ( Lee et al . ( 2019 ) ) and others add a second term to the loss function ( Zheng et al . ( 2019 ) ) . Up until now , there is a lack of research for loss functions which work for the problem of few-shot learning tackled from a metric-learning perspective . Our proposed model take part of these meta-learning approaches based on metric learning , by adopting the idea of the intra-class and inter-class variance into two different loss functions , which will help us to better optimize an embedding network to obtain more discriminant feature vectors .

Summary: This paper applies two losses to the few-shot learning model based on metric learning (similar to ProtoNet), which aims to utilize the intra- and inter-class distances. The first one is based on the original triplet loss and adjusted for the prototype network. The second one is based on the recently proposed Inter and Intra Class Nearest Neighbors Score (ICNN Score) (Garc´ıa Ram´ırez (2021)). The experimental results on miniImageNet show the proposed models achieve better performance than several previous metric-based approaches.