MAML is a Noisy Contrastive Learner

1 INTRODUCTION . Humans can learn from very few samples . They can readily establish their cognition and understanding to novel tasks , environments , or domains even with very limited experience in the corresponding circumstances . Meta-learning , a subfield of machine learning aims at equipping machines with such capacity to effectively accommodate new scenarios ( Vilalta & Drissi , 2002 ; Grant et al. , 2018 ) . Machines learn to extract task-agnostic information so that their performance on unseen tasks can be improved ( Hospedales et al. , 2020 ) . One highly influential meta-learning algorithm is Model Agnostic Meta-Learning ( MAML ) ( Finn et al. , 2017 ) , which has inspired numerous follow-up extensions ( Nichol et al. , 2018 ; Rajeswaran et al. , 2019 ; Liu et al. , 2019 ; Finn et al. , 2019 ; Jamal & Qi , 2019 ; Javed & White , 2019 ) . MAML estimates a set of model parameters such that an adaptation of the model to a new task only requires some updates to those parameters . We take the few-shot classification task as an example to review the algorithmic procedure of MAML . A few-shot classification problem refers to classifying samples from some classes ( i.e . query data ) after seeing a few examples per class ( i.e . support data ) . In a meta-learning scenario , we consider a distribution of tasks , where each task is a few-shot classification problem and different tasks have different target classes . MAML aims to meta-train the base-model based on training tasks ( i.e. , the meta-training dataset ) and evaluate the performance of the base-model on the testing tasks , sampled from a held-out unseen dataset ( i.e . the meta-testing dataset ) . In meta-training , MAML follows a bi-level optimization scheme composed of the inner loop and the outer loop , as shown in Appendix A ( please refer to Section 2 for detailed definition ) . In the inner loop ( also known as fast adaptation ) , the base-model θ is updated to θ′ using the support set . In the outer loop , a loss is evaluated on θ′ using the query set , and its gradient is computed with respect to θ to update the base-model . Since the outer loop requires computing the gradient of gradient ( as the update in the inner loop is included in the entire computation graph ) , it is called second-order MAML ( SOMAML ) . To prevent computing the Hessian matrix , Finn et al . ( 2017 ) propose first-order MAML ( FOMAML ) that uses the gradient computed with respect to the inner-loop-updated parameters θ′ to update the base-model . The widely accepted intuition behind MAML is that the models are encouraged to learn a generalpurpose representations which are broadly applicable not only to the seen tasks but also to novel tasks ( Finn et al. , 2017 ; Raghu et al. , 2020 ; Goldblum et al. , 2020 ) . Raghu et al . ( 2020 ) confirm this perspective by showing that during fast adaptation , the majority of changes being made is in the final linear layers , while the convolution layers ( as the feature encoder ) remain almost static . This implies the models trained with MAML learn a good feature representation and that they only have to change the linear mapping from features to outputs during the fast adaptation . Similar ideas of freezing feature extractor during the inner loop have also been explored ( Lee et al. , 2019 ; Bertinetto et al. , 2019 ; Liu et al. , 2020 ) , and have been held as an assumption in theoretical works ( Du et al. , 2021 ; Tripuraneni et al. , 2020 ; Chua et al. , 2021 ) . While this intuition sounds satisfactory , we step further and ask the following fundamental questions : ( 1 ) In what sense does MAML guide any model to learn general-purpose representations ? ( 2 ) How do the inner and outer loops in the training mechanism of MAML collaboratively prompt to achieve so ? ( 3 ) What is the role of the support and query data and how do they , if any , interact with each other ? In this paper , we answer these questions and give new insights on the working mechanism of MAML , which turns out to be closely connected to supervised contrastive learning ( SCL ) 1 . Here , we provide a sketch of our analysis in Figure 1 . We consider a setting of ( a ) a 5-way 1-shot paradigm of few-shot learning , ( b ) the mean square error ( MSE ) between the one-hot encoding of groundtruth label and the outputs as the objective function , and ( c ) MAML with a single inner-loop update . At the beginning of the inner loop , we set the linear layer w0 to zero . Then , the inner loop update of w0 is equivalent to adding the support features to w0 . In the outer loop , the output of a query sample q1 is actually the inner product between the query feature ϕ ( q1 ) and all support features ( the learning rate is omitted for now ) . As the groundtruth is an one-hot vector , the encoder is trained to either minimize the inner product between the query features and the support features ( when they are from different classes , as shown in the green box ) , or to pull the inner product between the query features and the support features to 1 ( when they have the same label , as shown in the red box ) . Therefore , the inner loop and the outer loop together manifest a SCL objective . Particularly , as the vanilla implementation of MAML uses non-zero ( random ) initialization for the linear layer , we will show such initialization leads to a noisy SCL objective which would impede the training . In this paper , we firstly review a formal definition of SCL , present a more general case of MAML with cross entropy loss in classification , and show the underlying learning protocol of vanilla MAML as an interfered SCL in Section 2 . We then experimentally verify the supervised contrastiveness of MAML , and propose to mitigate the interference with our simple but effective technique of the zeroinitialization and zeroing trick ( cf . Section 3 ) . In summary , our main contributions are three-fold : • We show that MAML is implicitly an SCL algorithm in classification and that the noise comes from the randomly initialized linear layer and the cross-task interaction , interfering the training of the encoder . • We verify the inherent contrastiveness of MAML based on the cosine similarity analysis . • We experimentally validate our analysis and show that applying the zeroing trick induces a notable improvement in testing accuracy during training . We also show that during meta-testing , a pronounced increase in accuracy occurs when the zeroing trick is applied . 2 WHY MAML IS IMPLICITLY A NOISY SUPERVISED CONTRASTIVE ALGORITHM ? . 2.1 PRELIMINARY : SUPERVISED CONTRASTIVE LEARNING . In this work , we aim to bridge MAML and supervised contrastive learning ( SCL ) and attribute the success of MAML to SCL ’ s capacity in learning good representations . Thus , we would like to briefly introduce SCL . 1We use the term supervised contrastiveness to refer to the setting of using groundtruth label information to differentiate positive samples and negative samples ( Khosla et al. , 2020 ) . This setting is different from ( unsupervised/self-supervised ) contrastive learning . Supervised contrastive learning , proposed by Khosla et al . ( 2020 ) , is a generalization of several metric learning algorithms , such as triplet loss and N-pair loss ( Schroff et al. , 2015 ; Sohn , 2016 ) and has shown the best performance in classification compared to SimCLR and CrossEntropy . In Khosla et al . ( 2020 ) , SCL is described as “ contrasts the set of all samples from the same class as positives against the negatives from the remainder of the batch ” and “ embeddings from the same class are pulled closer together than embeddings from different classes. ” For a sample s , label information is leveraged to indicate positive samples ( i.e. , samples having the same label as sample s ) and negative samples ( i.e. , samples having different labels to sample s ) . The loss of SCL is designed to increase the similarity ( or decrease metric distance ) of embeddings of positive samples and to decrease the similarity ( or increase the metric distance ) of embeddings of negative samples ( Khosla et al. , 2020 ) . In essence , SCL combines supervised learning and contrastive learning and differs from supervised learning in that the loss contains a measurement of the similarity or metric distance between the embedding of a sample and embedding of its positive/negative sample pairs . Now we give a formal definition of SCL . For a set of N samples drawn from a n-class dataset . Let i ∈ I = { 1 , ... , N } be the index of an arbitrary sample . Let A ( i ) = I \ { i } , P ( i ) be the set of indices of all positive samples of sample i , and N ( i ) = A ( i ) \ P ( i ) be the set of indices of all negative samples of sample i . Let zi indicates the embedding of sample i . Definition 1 Let Msim be a measurement of similarity ( e.g. , inner product , cosine similarity ) . Training algorithms that adopt loss of the following form belong to SCL : LSCL = ∑ i ∑ p∈P ( i ) c−p , iMsim ( zi , zp ) + ∑ i ∑ n∈N ( i ) c+n , iMsim ( zi , zn ) + c ( 1 ) where c−p , i < 0 and c + n , i > 0 for all n , p and i ; and c is a constant independent of samples . We further define that a training algorithm that follows Eq . ( 1 ) , but with either ( a ) c+n , i < 0 for some n , i or ( b ) c is a constant dependent of samples , belongs to noisy SCL . 2.2 PROBLEM SETUP . We provide the detailed derivation to show that MAML is implicitly a noisy SCL , where we adopt the few-shot classification as the example application . In this section , we focus on the meta-training period . Consider drawing a batch of tasks { T1 , . . . , TNbatch } from a meta-training task distribution D. Each task Tn contains a support set Sn and a query set Qn , where Sn = { ( sm , tm ) } Nway×Nshot m=1 , Qn = { ( qm , um ) } Nway×Nquery m=1 , sm , qm ∈ RNin are data samples , and tm , um ∈ { 1 , ... , Nway } are labels . We denote Nway the number of classes in each task , and { Nshot , Nquery } respectively the number of support and query samples per class . The architecture of our base-model comprises of a convolutional encoder ϕ : RNin → RNf ( parameterized by φ ) , a fully connected linear head w ∈ RNf×Nway , and a Softmax output layer , where Nf is the dimension of the feature space . We denote the kth column of w as wk . Note that the base-model parameters θ consist of φ and w. As shown in Appendix A , both FOMAML and SOMAML adopt a training strategy comprising the inner loop and the outer loop . At the beginning of a meta-training iteration , we sample Nbatch tasks . For each task Tn , we perform inner loop updates using the inner loop loss ( c.f . Eq . ( 2 ) ) evaluated on the support data , and then evaluate the outer loop loss ( c.f . Eq . ( 3 ) ) on the updated base-model using the query data . In the ith step of the inner loop , the parameters { φi−1 , wi−1 } are updated to { φi , wi } using the multi-class cross entropy loss evaluated on the support dataset Sn as L { φi , wi } , Sn = E ( s , t ) ∼Sn Nway∑ j=1 1j=t [ − log exp ( ϕi ( s ) ⊤wj i ) ∑Nway k=1 exp ( ϕ i ( s ) ⊤wki ) ] ( 2 ) After Nstep inner loop updates , we compute the outer loop loss using the query data Qn : L { φNstep , wNstep } , Qn = E ( q , u ) ∼Qn [ − log exp ( ϕ Nstep ( q ) ⊤wu Nstep ) ∑Nway k=1 exp ( ϕ Nstep ( q ) ⊤wkNstep ) ] ( 3 ) Then , we sum up the outer loop losses of all tasks , and perform gradient descent to update the base-model ’ s initial parameters { φ0 , w0 } . To show the supervised contrastiveness entailed in MAML , we adopt an assumption that the Encoder ϕ is Frozen during the Inner Loop ( the EFIL assumption ) and we discuss the validity of the assumption in Section 2.6 . Without loss of generality , we consider training models with MAML with Nbatch = 1 and Nstep = 1 , and we discuss the generalized version in Section 2.6 . For simplicity , the kth element of model output exp ( ϕ ( s ) ⊤wk 0 ) ∑Nway j=1 exp ( ϕ ( s ) ⊤wj0 ) ( respectively exp ( ϕ ( q ) ⊤wk 1 ) ∑Nway j=1 exp ( ϕ ( q ) ⊤wj1 ) ) of sample s ( respectively q ) is denoted as sk ( respectively qk ) .

Summary: The paper analyzes MAML algorithms. Assuming in the inner loop the encoder is fixed and only last linear layer is updated, they analyze the gradient update and loss terms in the inner loop and outer loops (sec 2.3). Through this effort, the authors claim that there are noisy supervised contrastive term in the outer loop loss (eqn (7) and (8)). They further claim that there are additional interference terms which may degrade the performance of MAML at the beginning of training when the linear layer weights are largely random. To overcome this, they propose a simple zeroing trick by zeroing the initial linear layer weights after each outer loop update, essentially removing the interference terms (eqn (9) and (10)). They conduct experiments to support the contrastiveness in MAML and performance improvement using zeroing trick.