Original: A DVERSARIAL M ETA -L EARNING
Revised: A DVERSARIAL M ETA -L EARNING

Original: Anonymous authors
Revised: Anonymous authors

Original: Paper under double-blind review
Revised: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: Meta-learning enables a model to learn from very limited data to undertake a new task.
Revised: Meta-learning enables a model to learn from very limited data to undertake a new task.

Original: In this paper, we study the general meta-learning with adversarial samples.
Revised: In this paper, we study the general meta-learning with adversarial samples.

Original: We present a meta-learning algorithm, ADML (ADversarial Meta-Learner), which leverages clean and adversarial samples to optimize the initialization of a learning model in an adversarial manner.
Revised: We present a meta-learning algorithm, ADML (ADversarial Meta-Learner), which leverages clean and adversarial samples to optimize the initialization of a learning model in an adversarial manner.

Original: ADML leads to the following desirable properties: 1) it turns out to be very effective even in the cases with only clean samples; 2) it is robust to adversarial samples, i.e., unlike other meta-learning algorithms, it only leads to a minor performance degradation when there are adversarial samples; 3) it sheds light on tackling the cases with limited and even contaminated samples.
Revised: ADML leads to the following desirable properties: 1) it turns out to be very effective even in the cases with only clean samples; 2) it is robust to adversarial samples, i.e., unlike other meta-learning algorithms, it only leads to a minor performance degradation when there are adversarial samples; 3) it sheds light on tackling the cases with limited and even contaminated samples.

Original: It has been shown by extensive experimental results that
Revised: It has been shown by extensive experimental results that

Original: ADML outperforms several representative meta-learning algorithms in the cases involving adversarial samples generated by different attack mechanisms, on two widely-used image datasets, MiniImageNet and CIFAR100, in terms of both accuracy and robustness.
Revised: ADML outperforms several representative meta-learning algorithms in the cases involving adversarial samples generated by different attack mechanisms, on two widely-used image datasets, MiniImageNet and CIFAR100, in terms of both accuracy and robustness.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: Deep learning has made tremendous successes and emerged as a de facto approach in many application domains, such as computer vision and natural language processing, which, however, depends heavily on huge amounts of labeled training data.
Revised: Deep learning has made tremendous successes and emerged as a de facto approach in many application domains, such as computer vision and natural language processing, which, however, depends heavily on huge amounts of labeled training data.

Original: The goal of meta-learning is to enable a model (especially a Deep Neural Network (DNN)) to learn from only a small number of data samples to undertake a new task, which is critically important to machine intelligence but turns out to be very challenging.
Revised: The goal of meta-learning is to enable a model (especially a Deep Neural Network (DNN)) to learn from only a small number of data samples to undertake a new task, which is critically important to machine intelligence but turns out to be very challenging.

Original: Currently, a common approach to learn is to train a model to undertake a task from scratch without making use of any previous experience.
Revised: Currently, a common approach to learn is to train a model to undertake a task from scratch without making use of any previous experience.

Original: Speciﬁcally, a model is initialized randomly and then updated slowly using gradient descent with a large number of training samples.
Revised: Speciﬁcally, a model is initialized randomly and then updated slowly using gradient descent with a large number of training samples.

Original: This kind of time-consuming and data-hungry training process is quite different from the way how a human learns quickly from only a few samples and obviously cannot meet the requirement of meta-learning.
Revised: This kind of time-consuming and data-hungry training process is quite different from the way how a human learns quickly from only a few samples and obviously cannot meet the requirement of meta-learning.

Original: Several methods (Finn et al.
Revised: Several methods (Finn et al.

Original: (2017); Vinyals et al.
Revised: (2017); Vinyals et al.

Original: ; Snell et al.
Revised: ; Snell et al.

Original: ; Sung et al.
Revised: ; Sung et al.

Original: (2018)) have been proposed to address meta-learning by ﬁxing the above issue.
Revised: (2018)) have been proposed to address meta-learning by ﬁxing the above issue.

Original: For example, a well-known work (Finn et al.
Revised: For example, a well-known work (Finn et al.

Original: (2017)) presents a novel meta-learning algorithm called MAML (Model-Agnostic Meta-Learning), which trains a model’s initial parameters carefully such that it has the maximal performance on a new task after its parameters are updated through one or just a few gradient steps with a small amount ofnew data.
Revised: (2017)) presents a novel meta-learning algorithm called MAML (Model-Agnostic Meta-Learning), which trains and optimizes the initialization of model parameters carefully such that it achieves the maximal performance on a new task after its parameters are updated through one or just a few gradient steps with a small amount of data.

Original: This method is claimed to be model-agnostic since it can be directly applied to any learning model that can be trained with gradient decent.
Revised: This method is claimed to be model-agnostic since it can be directly applied to any learning model that can be trained with gradient descent.

Original: Robustness is another major concern for machine intelligence.
Revised: Robustness is another major concern for machine intelligence, especially for the safety-critical applications, such as facial recognition, algorithmic trading and copyright control.

Revised: It has been shown that such learning models can be easily fooled by adversarial manipulation to cause serious security threats (Zhao et al.

Original: It has been shown by Szegedy et al.
Revised: ; Goldblum et al. Saadatpanah et al.

Original: (2014) that learning models can be easily fooled by  adversarialmanipulation of actual training data to cause incorrect classiﬁcations.
Revised: (2019)), which, however, can be properly and effectively handled by conventional adversarial training and pre-processing defenses (Madry et al. ; Zhang et al.

Original: Therefore, adversarial samples pose a serious security threat to learning tasks, which need to be properly and effectively handled by learning models and training algorithms.
Revised: ; Samangouei et al. Nonetheless, if the data is limited (e.g., face recognition from few images), the aforementioned pipelines, which require a large amount of training data, suffers from serious performance degradation.

Original: We show via experiments that existing meta-leaning algorithms (such as MAML Finn et al.
Revised: Although meta-learning based approaches show great potential on dealing with few-shot tasks, we show via experiments that existing meta-leaning algorithms (such as MAML (Finn et al.

Original: (2017), Matching Networks Vinyals et al.
Revised: , Matching Networks (Vinyals et al.

Original: and Relation Networks Sung et al.
Revised: and Relation Networks (Sung et al.

Original: (2018)) are also vulnerable to adversarial samples, i.e., adversarial samples can lead to a signiﬁcant performance degradation for meta-learning.
Revised: (2018))) are also vulnerable to adversarial samples, i.e., adversarial samples can lead to a signiﬁcant performance degradation for meta-learning.

Original: An adversarial approach, called MetaGAN, was presented by Zhang et al.

Original: (2018) for few-shot classiﬁcation, nevertheless, in addition to learning better decision boundaries by leveraging fake data with the power of GAN (Goodfellow et al.

Original: , it paid no attention to deal with the cases involving adversarial samples.

Original: To the best of our knowledge,none of existing works on meta-learning have well addressed adversarial samples, which, however, are the main focus of this paper.
Revised: To the best of our knowledge, existing works on meta-learning have not yet addressed adversarial samples, which, however, is the main focus of this paper.

Original: In this paper, we extend meta-learning to a whole new dimension by studying how to quickly train a model (especially a DNN) for a new task using a small dataset with both clean and adversarial samples.
Revised: In this paper, we extend meta-learning to a whole new dimension by studying how to quickly train a model (especially a DNN) for a new task using a small dataset with both clean and adversarial samples.

Original: Since both meta-learning and adversarial learning have been studied recently, a straightforward solution is to simply combinean existing meta-learning algorithms (e.g., MAML Finn et al.
Revised: Since both meta-learning and adversarial learning have been studied recently, a straightforward solution is to simply combine MAML (Finn et al.

Original:  with adversarial training (e.g., Goodfellow et al.
Revised: algorithm with adversarial training (e.g., Goodfellow et al.

Original: However, we show such a approach does not work well by our experimental results.
Revised: However, we show such an approach does not work well by our experimental results.

Original: We present a novel ADversarial Meta-Learner (ADML), which utilizes antagonistic correlations between clean and adversarial samples to let the inner gradient update arm-wrestle with the meta-update to obtain a good and robust initialization of model parameters.
Revised: We present a novel ADversarial Meta-Learner (ADML), which utilizes antagonistic correlations between clean and adversarial samples to let the inner gradient update arm-wrestle with the meta-update to obtain a good and robust initialization of model parameters.

Original: Hence, “adversarial” in ADML refers to not only adversarial samples but also the way of updating the learning model.
Revised: Hence, “adversarial” in ADML refers to not only adversarial samples but also the way of updating the learning model.

Original: The design of ADML leads to several desirable properties.
Revised: The design of ADML leads to several desirable properties.

Original: First, it turns out to be very effective even in the cases with only clean samples.
Revised: First, it turns out to be very effective even in the cases with only clean samples.

Original: Second, unlike other meta-learning algorithms, ADML is robust to adversarial samples since it only suffers from a minor performance degradation when encountering adversarial samples, and it outperforms several representative meta-learning algorithms (Finn et al.
Revised: Second, unlike other meta-learning algorithms, ADML is robust to adversarial samples since it only suffers from a minor performance degradation when encountering adversarial samples, and it outperforms several representative meta-learning algorithms (Finn et al.

Original: (2017); Vinyals et al.
Revised: (2017); Vinyals et al.

Original: ; Sung et al.
Revised: ; Sung et al.

Original:  Bertinetto et al.
Revised: ; Bertinetto et al.

Original: in such cases.
Revised: in such cases.

Original: In addition, ADML is agnostic to the attack mechanism, which is responsible for the adversarial samples generation.
Revised: In addition, ADML is agnostic to the attack mechanism, which is responsible for the adversarial samples generation.

Original: Most importantly, it opens up an interesting research direction and sheds light on dealing with the cases with limited and even contaminated samples, which are common in real life.
Revised: Most importantly, it opens up an interesting research direction and sheds light on dealing with the cases with limited and even contaminated samples, which are common in real life.

Original: We conducted a comprehensive empirical study for performance evaluation using two widely-used image datasets, MiniImageNet (Vinyals et al.
Revised: We conducted a comprehensive empirical study for performance evaluation using two widely-used image datasets, MiniImageNet (Vinyals et al.

Original: and CIFAR100 (Krizhevsky et al.
Revised: and CIFAR100 (Krizhevsky et al.

Original: Experimental results well justify the effectiveness and superiority of ADML in terms of both accuracy and robustness.
Revised: Experimental results well justify the effectiveness and superiority of ADML in terms of both accuracy and robustness.

Original:  R ELATED W ORK
Revised:  R ELATED W ORK

Original: Meta-Learning: Research on meta-learning has a long history, which can be traced back to some early works (Naik & Mammone (1992); Thrun & Pratt (1998)).
Revised: Meta-Learning: Research on meta-learning has a long history, which can be traced back to some early works (Naik & Mammone (1992); Thrun & Pratt (1998)).

Original: Meta-learning, a standard methodology to tackle few-shot learning problems, has recently attracted extensive attention due to its important roles in achieving human-level intelligence.
Revised: Meta-learning, a standard methodology to tackle few-shot learning problems, has recently attracted extensive attention due to its important roles in achieving human-level intelligence.

Original: Several specialized models (Vinyals et al. (2016);
Revised: Several specialized models (Vinyals et al. (2016);

Original: Koch et al.
Revised: Koch et al.

Original: ; Snell et al.
Revised: ; Snell et al.

Original: ; Sung et al.
Revised: ; Sung et al.

Original: (2018)) have been proposed for meta-learning, particularly for few-shot classiﬁcation, by comparing similarity among data samples.
Revised: (2018)) have been proposed for meta-learning, particularly for few-shot classiﬁcation, by comparing similarity among data samples.

Original: Speciﬁcally, Koch et al.
Revised: Speciﬁcally, Koch et al.

Original: (2015) leveraged a Siamese Networks to rank similarity between input samples and predict if two samples belong to the same class.
Revised: (2015) leveraged a Siamese Networks to rank similarity between input samples and predict if two samples belong to the same class.

Original: In addition, Relation Networks (Sung et al.
Revised: In addition, Relation Networks (Sung et al.

Original: (2018)) was proposed to classify query images by computing relation scores, which can be extended to few-shot learning.
Revised: (2018)) was proposed to classify query images by computing relation scores, which can be extended to few-shot learning.

Original: Vinyals et al.
Revised: Vinyals et al.

Original: (2016) presented a neural network model, Matching Networks, which learn an embedding function and use the cosine distance in an attention kernel to measure similarity.
Revised: (2016) presented a neural network model, Matching Networks, which learn an embedding function and use the cosine distance in an attention kernel to measure similarity.

Original: Another popular approach to meta-learning is to develop a meta-learner to optimize key hyperparameters (e.g., initialization) of the learning model.
Revised: Another popular approach to meta-learning is to develop a meta-learner to optimize key hyperparameters (e.g., initialization) of the learning model.

Original: Speciﬁcally, Finn et al.
Revised: Speciﬁcally, Finn et al.

Original: (2017) presented a model-agnostic meta-learner, MAML, to optimize the initialization of a learning model with the objective of maximizing its performance on a new task after updating its parameters with a small number of samples.
Revised: (2017) presented a model-agnostic meta-learner, MAML, to optimize the initialization of a learning model with the objective of maximizing its performance on a new task after updating its parameters with a small number of samples.

Original: Several other methods (Andrychowicz et al. (2016); Ravi & Larochelle (2017);
Revised: Several other methods (Andrychowicz et al. (2016); Ravi & Larochelle (2017);

Original: Santoro et al.
Revised: Santoro et al.

Original: ; Mishra et al.
Revised: ; Mishra et al.

Original: (2018)) utilized an additional neural network, such as LSTM, to serve as the meta-learner.
Revised: (2018)) utilized an additional neural network, such as LSTM, to serve as the meta-learner.

Original: A seminal work (Andrychowicz et al.
Revised: A seminal work (Andrychowicz et al.

Original: (2016)) developed a meta-learner based on LSTMs and showed how the design of an optimization algorithm can be cast as a learning problem.
Revised: (2016)) developed a meta-learner based on LSTMs and showed how the design of an optimization algorithm can be cast as a learning problem.

Original: Ravi & Larochelle (2017) proposed another LSTM-based meta-learner to learn a proper parameter update and a general initialization for the learning model.
Revised: Ravi & Larochelle (2017) proposed another LSTM-based meta-learner to learn a proper parameter update and a general initialization for the learning model.

Original: A recent work (Mishra et al.
Revised: A recent work (Mishra et al.

Original: (2018)) presented a class of simple and generic meta-learners that use a novel combination of temporal convolutions and soft attention.
Revised: (2018)) presented a class of simple and generic meta-learners that use a novel combination of temporal convolutions and soft attention.

Original: Adversarial Learning: DNN models have been shown to be vulnerable to adversarial samples.
Revised: Adversarial Learning: DNN models have been shown to be vulnerable to adversarial samples.

Original: Particularly, Szegedy et al.
Revised: Particularly, Szegedy et al.

Original: (2014) showed that they can cause a DNN to misclassify an image by applying a certain hardly perceptible perturbation, and moreover, the same perturbation can cause a different network (trained on a different subset of the dataset) to misclassify the same input.
Revised: (2014) showed that they can cause a DNN to misclassify an image by applying a certain hardly perceptible perturbation, and moreover, the same perturbation can cause a different network (trained on a different subset of the dataset) to misclassify the same input.

Original: It has also been shown by Goodfellow et al.
Revised: It has also been shown by Goodfellow et al.

Original: (2015) that injecting adversarial samples during training can increase the robustness of DNN models.
Revised: (2015) that injecting adversarial samples during training can increase the robustness of DNN models.

Original: Papernot et al.
Revised: Papernot et al.

Original: (2017) introduced the ﬁrst practical demonstration of a black-box attack controlling a remotely hosted DNN without either the model internals or its training data.
Revised: (2017) introduced the ﬁrst practical demonstration of a black-box attack controlling a remotely hosted DNN without either the model internals or its training data.

Original: In addition, Kurakin et al.
Revised: In addition, Kurakin et al.

Original: (2017) studied adversarial learning at scale by proposing an algorithm to train a large scale model, Inception v3, on the ImageNet dataset, which has been shown to signiﬁcantly increase the robustness against adversarial samples.
Revised: (2017) studied adversarial learning at scale by proposing an algorithm to train a large scale model, Inception v3, on the ImageNet dataset, which has been shown to signiﬁcantly increase the robustness against adversarial samples.

Original: To the best of our knowledge, meta-learning has not been well studied in the setting with adversarial samples.
Revised: To the best of our knowledge, meta-learning has not been well studied in the setting with adversarial samples.

Original: We not only show a straightforward solution does not work well but also present a novel and effective method, ADML.
Revised: We not only show a straightforward solution does not work well but also present a novel and effective method, ADML.

Original:  A DVERSARIAL M ETA -L EARNING 3. P ROBLEM S TATEMENT
Revised:  A DVERSARIAL M ETA -L EARNING 3. P ROBLEM S TATEMENT

Original: The regular machine learning problem seeks a model that maps observations x to output y ; and a training algorithm optimizes the parameters of the model with a training dataset, whose generalization is then evaluated on a testing dataset.
Revised: The regular machine learning problem seeks a model that maps observations x to output y ; and a training algorithm optimizes the parameters of the model with a training dataset, whose generalization is then evaluated on a test dataset.

Original: While in the setting of meta-learning, the learning model is expected to be trained with limited data to be able to adapt to a new task quickly.
Revised: While in the setting of meta-learning, the learning model is expected to be trained with limited data to be able to adapt to a new task quickly.

Original: Metalearning includes meta-training and meta-testing.
Revised: Meta-learning includes meta-training and meta-testing.

Original: In the meta-training , we use a set T of T tasks, each of which has a loss function L i , and a dataset D i (with limited data) that is further split into D i and D (cid:48)
Revised: In the meta-training phase, we use a set T of T tasks, each of which has a loss function L i , and a dataset D i (with limited data) that is further split into D i and D (cid:48)

Original: i for training and testing respectively.
Revised: i for training and testing respectively.

Original: For example, in our experiments, each task is a 5-way classiﬁcation task.
Revised: For example, in our experiments, each task is a 5-way classiﬁcation task.

Revised: Note that the 5-way represents 5 different classes.

Original: We aim to develop a meta-learner (i.e., a learning algorithm) that takes as input the datasets D
Revised: We aim to develop a meta-learner (i.e., a learning algorithm) that takes as input the datasets D

Original: = {D 1 , · · · , D T } and returns a model with parameters θ that maximizes the average classiﬁcation accuracy on the corresponding testing sets D (cid:48)
Revised: = {D 1 , · · · , D T } and returns a model with parameters θ that maximizes the average classiﬁcation accuracy on the corresponding testing sets D (cid:48)

Original: = { D (cid:48) 1 , · · · , D (cid:48) T } .
Revised: = { D (cid:48) 1 , · · · , D (cid:48) T } .

Original: Note that here these testing data are also used for meta-training.
Revised: Note that here these testing data are also used for meta-training.

Original: In the meta-testing , we evaluate the generalization of the learned model with parameters θ on new tasks, whose corresponding training and testing datasets may include adversarial samples.
Revised: In the meta-testing , we evaluate the generalization of the learned model with parameters θ on new tasks, whose corresponding training and test datasets may include adversarial samples.

Original: The learned model is expected to learn quickly from just one (1-shot) or K ( K -shot) training samples of a new task and deliver highly-accurate results on its testing samples.
Revised: The learned model is expected to learn quickly from just one (1-shot) or K ( K -shot) training samples of a new task and deliver highly-accurate results on its testing samples.

Original: An ideal meta-learner is supposed to return a learning model that can deal with new tasks with only clean samples; and suffers from only a minor performance degradation for new tasks with adversarial samples.
Revised: An ideal meta-learner is supposed to return a learning model that can deal with new tasks with only clean samples; and suffers from only a minor performance degradation for new tasks with adversarial samples.

Original: We believe the proposed ADML can be easily extended to other scenarios as long as adversarial samples can be properly generated.
Revised: We believe the proposed ADML can be easily extended to other scenarios as long as adversarial samples can be properly generated.

Original: 3. A DVERSARIAL M ETA -L EARNER (ADML)
Revised: 3. A DVERSARIAL M ETA -L EARNER (ADML)

Original: We formally present the proposed ADML as Algorithm 1 for meta-training .
Revised: We formally present the proposed ADML as Algorithm 1 for meta-training .

Original: We consider a model f θ parameterized by θ , which is updated iteratively.
Revised: We consider a model f θ parameterized by θ , which is updated iteratively.

Original: Here, an updating episode includes an inner gradient update process (Line 8–Line 12) and a meta-update process (Line 14).
Revised: Here, an updating episode includes an inner gradient update process (Line 8–Line 12) and a meta-update process (Line 14).

Original: Unlike MAML, for each task, additional adversarial samples are generated and used to enhance the robustness for metatraining.
Revised: Unlike MAML, for each task, additional adversarial samples are generated and used to enhance the robustness for metatraining.

Original: Note that our algorithm is not restricted to any particular attack mechanism.
Revised: Note that our algorithm is not restricted to any particular attack mechanism.

Original: Here, we use the Fast Gradient Sign Method (FGSM) proposed by Goodfellow et al.
Revised: Here, we use the Fast Gradient Sign Method (FGSM) proposed by Goodfellow et al.

Original: (2015) for illustration.
Revised: (2015) for illustration.

Original: For task T i , given a clean sample ( x c , y c ) from D i , its corresponding adversarial sample ( x adv , y adv ) is generated using the following equations:
Revised: For task T i , given a clean sample ( x c , y c ) from D i , its corresponding adversarial sample ( x adv , y adv ) is generated using the following equations:

Original: [Equation]
Revised: [Equation]

Original: where J ( f θ pre , x c , y c ) represents the cost used to train a classiﬁcation model f θ pre parameterized by θ pre , and (cid:15) speciﬁes the size of the adversarial perturbation (the larger the (cid:15) , the higher the perturbation).
Revised: where J ( f θ pre , x c , y c ) represents the cost used to train a classiﬁcation model f θ pre parameterized by θ pre , and (cid:15) speciﬁes the size of the adversarial perturbation (the larger the (cid:15) , the higher the perturbation).

Original: Note that the classiﬁcation model f θ pre is pre-trained based on the corresponding dataset and its parameters θ pre are ﬁxed during the meta-training (Algorithm 1) and meta-testing.
Revised: Note that the classiﬁcation model f θ pre is pre-trained based on the corresponding dataset and its parameters θ pre are ﬁxed during the meta-training (Algorithm 1) and meta-testing.

Original: The key idea behind ADML is to utilize antagonistic correlations between clean and adversarial samples to let the inner gradient update and the meta-update arm-wrestle with each other to obtain a good initialization of model parameters θ , which is robust to adversarial samples.
Revised: The key idea behind ADML is to utilize antagonistic correlations between clean and adversarial samples to let the inner gradient update and the meta-update arm-wrestle with each other to obtain a good initialization of model parameters θ , which is robust to adversarial samples.

Original: Speciﬁcally, in the inner gradient update, we compute the new model parameters (updated in two directions) θ (cid:48)
Revised: Speciﬁcally, in the inner gradient update, we compute the new model parameters (updated in two directions) θ (cid:48)

Original: adv i and θ (cid:48)
Revised: adv i and θ (cid:48)

Original: c i based on generated adversarial samples D
Revised: c i based on generated adversarial samples D

Original: adv i , and clean samples D
Revised: adv i , and clean samples D

Original: c i in training set D i of task T i respectively using gradient decent (Line 10).
Revised: c i in training set D i of task T i respectively using gradient descent (Line 10).

Original: In the meta-update process, we update the model parameters θ by optimizing the losses L i ( f θ (cid:48) advi ) and L i ( f θ (cid:48) ci ) of the model with updated parameters θ (cid:48)
Revised: In the meta-update process, we update the model parameters θ by optimizing the losses L i ( f θ (cid:48) advi ) and L i ( f θ (cid:48) ci ) of the model with updated [Equation] [Equation] [Equation]

Original: adv i and θ (cid:48)
Revised: [Equation]

Original: c 
Revised: c i

Original: i with respect to θ based on the clean samples D (cid:48) c i in testing set D (cid:48)
Revised: Sample containing k adversarial samples and k clean samples; 13: end for 14:

Original: i of task
Revised: [Equation]

Original: Sample [Equation] [Equation] [Equation]
Revised: 15: end while parameters θ (cid:48)

Original: D c i

Original: Sample

Original: Generate clean samples from D to a dataset D i :=

Original: D (cid:48) adv i , D (cid:48)
Revised: adv i and θ (cid:48)

Original: c
Revised: c

Original: (cid:111) for themeta-update, containing k adversarial samples and k clean samples; 13: end for 14: [Equation]
Revised: i with respect to θ based on the clean samples D (cid:48) c i in testing set D (cid:48)

Original: 15: end while
Revised: i of task

Original: T i and the corresponding adversarial samples D (cid:48) adv i respectively:
Revised: T i and the corresponding adversarial samples D (cid:48) adv i respectively:

Original: [Equation]
Revised: [Equation]

Original: Note that in the meta-update, θ is optimized in an adversarial manner: the gradient of the loss of the model with θ (cid:48)
Revised: Note that in the meta-update, θ is optimized in an adversarial manner: the gradient of the loss of the model with θ (cid:48)

Original: adv i (updated using adversarial samples D adv i ) is calculated based on clean samples D (cid:48)
Revised: adv i (updated using adversarial samples D adv i ) is calculated based on clean samples D (cid:48)

Original: c i , while the gradient of the loss of the model with θ (cid:48) c
Revised: c i , while the gradient of the loss of the model with θ (cid:48) c

Original: i (updated using D c i ) is calculated based on adversarial samples D (cid:48)
Revised: i (updated using D c i ) is calculated based on adversarial samples D (cid:48)

Original: adv i .
Revised: adv i .

Original: The arm-wrestling between the inner gradient update and the meta-update brings an obvious beneﬁt: the model adapted to adversarial samples (through the inner gradient update using adversarial samples) is made suitable also for clean samples through the optimization of θ in the meta-update based on the clean samples, and vice versa.
Revised: The arm-wrestling between the inner gradient update and the meta-update brings an obvious beneﬁt: the model adapted to adversarial samples (through the inner gradient update using adversarial samples) is made suitable also for clean samples through the optimization of θ in the meta-update based on the clean samples, and vice versa.

Original: So “ adversarial ” in ADML refers to not only adversarial samples but also the way of meta-training.
Revised: So “ adversarial ” in ADML refers to not only adversarial samples but also the way of meta-training.

Revised: Please note that in

Revised: Line 9, the set of adversarial samples D

Revised: adv i can be generated exactly from the set of clean samples

Revised: D c i , and it also holds for D (cid:48)

Revised: adv i and D (cid:48)

Original:  The meta-update of the model parameters θ is performed as the last step of each episode (Line 14).
Revised: c i in Line 12. The meta-update of the model parameters θ is performed as the last step of each episode (Line 14).

Original: Through the arm-wrestling between the inner gradient update and the meta-update in the metatraining process, θ will be updated to a certain point, such that the average loss given by both adversarial samples and clean samples of all the tasks is minimized.
Revised: Through the arm-wrestling between the inner gradient update and the meta-update in the metatraining phase, θ will be updated to a certain point, such that the average loss given by both adversarial samples and clean samples of all the tasks is minimized.

Original: In addition, we set the step sizes α 1 = α 2 = 0 .
Revised: In addition, we set the step sizes α 1 = α 2 = 0 .

Original: 01 , β 1 = β 

Revised: 01 , β 1 = β

Original: =
Revised: =

Original: 001 , and set L i ( · ) of each classiﬁcation task T i to be the crossentropy loss.
Revised: 001 , and set L i ( · ) of each classiﬁcation task T i to be the crossentropy loss.

Original: K and k are task-speciﬁc, whose settings are discussed in the next section.
Revised: K and k are task-speciﬁc, whose settings are discussed in the next section.

Original: Note that
Revised: Note that

Original: ADML preserves the model-agnostic property of MAML because both the inner gradient update and the meta-update processes are fully compatible with any learning model that can be trained by gradient descent.
Revised: ADML preserves the model-agnostic property of MAML because both the inner gradient update and the meta-update processes are fully compatible with any learning model that can be trained by gradient descent.

Original: We further illustrate the design philosophy of our algorithm in Figure 1.
Revised: We further illustrate the design philosophy of our algorithm in Figure 1.

Original: For each task T i , in the inner gradient update, ADML ﬁrst drags θ via gradient descent to the direction of the subspace that is favorable for adversarial samples (marked with the purple color) as well as another subspace that  is favorable for clean samples (marked with the red color) to reach two points θ (cid:48)
Revised: For each task T i , in the inner gradient update, ADML ﬁrst drags θ via gradient descent to the direction of the subspace that is favorable for adversarial samples (marked with the purple color) as well as another subspace that [Figure] is favorable for clean samples (marked with the red color) to reach two points θ (cid:48)

Original: adv i and θ (cid:48) c i respectively (i.e., Line 10).
Revised: adv i and θ (cid:48) c i respectively (i.e., Line 10).

Original: Then in the meta-update, based on θ (cid:48)
Revised: Then in the meta-update, based on θ (cid:48)

Original: adv i and θ (cid:48)
Revised: adv i and θ (cid:48)

Original: c i , ADML further optimizes θ to its antithetic subspaces respectively (i.e., Line 14), and hopefully θ can reach the optimal point[Figure] θ ∗ i , which is supposed to fall into the intersection of the subspace pair and is able to achieve a good trade-off between clean and adversarial samples to boost the overall performance on both samples.
Revised: c i , ADML further optimizes θ to its antithetic subspaces respectively (i.e., Line 14), and hopefully θ can reach the optimal point θ ∗ i , which is supposed to fall into the intersection of the subspace pair and is able to achieve a good trade-off between clean and adversarial samples to boost the overall performance on both samples.

Original: Note that here we only show the updates via a single task.
Revised: Note that here we only show the updates via a single task.

Original: Using all the tasks in T , θ can be optimized to a point with the smallest average distance to the intersections of all the subspace pairs, and thus can be quickly adapted to new tasks even with adversarial samples.
Revised: Using all the tasks in T , θ can be optimized to a point with the smallest average distance to the intersections of all the subspace pairs, and thus can be quickly adapted to new tasks even with adversarial samples.

Original: As mentioned before, a rather straightforward solution to the above adversarial meta-learning problem is to simply combine a meta-learner (e.g., MAML Finn et al.
Revised: As mentioned before, a rather straightforward solution to the above adversarial meta-learning problem is to simply combine a meta-learner (e.g., MAML Finn et al.

Original: with adversarial training (e.g., Goodfellow et al.
Revised: with adversarial training (e.g., Goodfellow et al.

Original: Speciﬁcally, we mix adversarial and clean samples to form both
Revised: Speciﬁcally, we mix adversarial and clean samples to form both

Original: D i (used in the inner gradient update) and D (cid:48)
Revised: D i (used in the inner gradient update) and D (cid:48)

Original: i (used in the meta-update), which are then used to calculate θ (cid:48) i and update θ using the following equations (just like MAML) respectively:
Revised: i (used in the meta-update), which are then used to calculate θ (cid:48) i and update θ using the following equations (just like MAML) respectively:

Original: [Equation]
Revised: [Equation]

Original: We call this method MAML-AD , which is used as a baseline for performance evaluation.
Revised: We call this method MAML-AD , which is used as a baseline for performance evaluation.

Original: However, it has been shown by our experimental results that although MAML-AD can slightly mitigate the problem, it still suffers from a signiﬁcant performance degradation for new tasks with adversarial samples.
Revised: However, it has been shown by our experimental results that although MAML-AD can slightly mitigate the problem, it still suffers from a signiﬁcant performance degradation for new tasks with adversarial samples.

Original: This clearly shows that simply involving adversarial samples during the meta-training does not necessarily enhance the model’s robustness; and well justiﬁes that our idea of doing the inner gradient update and the meta-update in an adversarial way is necessary.
Revised: This clearly shows that simply involving adversarial samples during the meta-training does not necessarily enhance the model’s robustness; and well justiﬁes that our idea of doing the inner gradient update and the meta-update in an adversarial way is necessary.

Original:  P ERFORMANCE E VALUATION
Revised:  P ERFORMANCE E VALUATION

Original: The goal of our evaluation is to test and verify three properties of ADML: 1) ADML can learn quickly from limited data via a few gradient updates for a new task, and it is effective even in the cases with only clean samples; 2) ADML suffers from a minor performance degradation and yields much better performance than other meta-learning algorithms when encountering adversarial samples; 3) ADML works with adversarial samples generated by different attack mechanisms; and 4)
Revised: The goal of our evaluation is to test and verify three properties of ADML: 1) ADML can learn quickly from limited data via a few gradient updates for a new task, and it is effective even in the cases with only clean samples; 2) ADML suffers from a minor performance degradation and yields much better performance than other meta-learning algorithms when encountering adversarial samples; 3) ADML works with adversarial samples generated by different attack mechanisms; and 4)

Original: ADML maintains stable performance when the perturbation of adversarial samples (i.e., (cid:15) ) escalates.
Revised: ADML maintains stable performance when the perturbation of adversarial samples (i.e., (cid:15) ) escalates.

Original: In this section, we ﬁrst introduce the experimental setup, and then present and analyze the results.
Revised: In this section, we ﬁrst introduce the experimental setup, and then present and analyze the results.

Original: 4. E XPERIMENTAL S ETUP
Revised: 4. E XPERIMENTAL S ETUP

Original: In our experiments, we employed two commonly-used image benchmarks, MiniImageNet (Vinyals et al.
Revised: In our experiments, we employed two commonly-used image benchmarks, MiniImageNet (Vinyals et al.

Original: (2016)), and CIFAR100 (Krizhevsky et al.
Revised: (2016)), and CIFAR100 (Krizhevsky et al.

Original: MiniImageNet is a benchmark for fewshot learning, which includes 100 classes and each of them has 600 samples.
Revised: MiniImageNet is a benchmark for fewshot learning, which includes 100 classes and each of them has 600 samples.

Original: CIFAR100 was created originally for object recognition tasks, whose data are suitable for meta-learning, and same as
Revised: CIFAR100 was created originally for object recognition tasks, whose data are suitable for meta-learning, and same as

Original: MiniImageNet, it has 100 classes, each of which contains 600 images.
Revised: MiniImageNet, it has 100 classes, each of which contains 600 images.

Original: Both MiniImageNet and
Revised: Both MiniImageNet and

Original: CIFAR100 were divided into 64, 16 and 20 classes for training, validation and testing respectively.
Revised: CIFAR100 were divided into 64, 16 and 20 classes for training, validation and testing respectively.

Original: Here we consider 5-way 1-shot and 5-way 5-shot classiﬁcation tasks.
Revised: Here we consider 5-way 1-shot and 5-way 5-shot classiﬁcation tasks.

Original: 5 samples per class were used for the inner gradient update during meta-training of a 5-shot learning model (1 sample for 1-shot learning model).
Revised: 5 samples per class were used for the inner gradient update during meta-training of a 5-shot learning model (1 sample for 1-shot learning model).

Original: Thus, K in ADML was set to 25 for 5-shot learning and 5 for 1-shot learning. [Table] samples per class were used for the meta-update, thus we set k = 75 .
Revised: Thus, K in ADML was set to 25 for 5-shot learning and 5 for 1-shot learning.samples per class were used for the meta-update, thus we set k = 75 .

Original: During the meta-testing, the learning model was trained on samples of 5 unseen classes, then tested by classifying new instances into these 5 classes.
Revised: During the meta-testing, the learning model was trained on samples of 5 unseen classes, then tested by classifying new instances into these 5 classes.

Original: Attack mechanisms including FGSM (Goodfellow et al. (2015)), FFGSM (Wong et al.
Revised: Attack mechanisms including FGSM (Goodfellow et al. (2015)), FFGSM (Wong et al.

Original: RFGSM (Tram ` er et al.
Revised: RFGSM (Tram`er et al.

Original: and RPGD (Madry et al.
Revised: and RPGD (Madry et al.

Original: (2017)) were leveraged to generate adversarial samples in the experiments.
Revised: (2017)) were leveraged to generate adversarial samples in the experiments.

Original: A well-trained VGG16 network (pre-trained on ImageNet and
Revised: A well-trained VGG16 network (pre-trained on ImageNet and

Original: CIFAR100 respectively) for image classiﬁcation served as the ﬁxed classiﬁcation model.
Revised: CIFAR100 respectively) for image classiﬁcation served as the ﬁxed classiﬁcation model.

Original: To validate the performance of ADML with both strongly-attacked and slightly-attacked samples, the parameter (cid:15) was set to a large value (i.e., 2) and a small value (i.e., 0.2) for FGSM for both meta-training and meta-testing.
Revised: To validate the performance of ADML with both strongly-attacked and slightly-attacked samples, the parameter (cid:15) was set to a large value (i.e., 2) and a small value (i.e., 0.2) for FGSM for both meta-training and meta-testing.

Original: See values of (cid:15) set for the other three attack mechanisms in supplementary material.
Revised: See values of (cid:15) set for the other three attack mechanisms in supplementary material.

Original: ADML was compared with four representative meta-learning algorithms, including MAML (Finn et al.
Revised: ADML was compared with four representative meta-learning algorithms, including MAML (Finn et al.

Original: Matching Networks (Vinyals et al.
Revised: Matching Networks (Vinyals et al.

Original: Relation Networks (Sung et al.
Revised: Relation Networks (Sung et al.

Original: and R2D2 (Bertinetto et al.
Revised: and R2D2 (Bertinetto et al.

Original: Moreover, for fair comparisons, we compared ADML with another adversarial meta-learner MAML-AD (introduced in Section 3), which can be considered as a rather straightforward extension of MAML.
Revised: Moreover, for fair comparisons, we compared ADML with another adversarial meta-learner MAML-AD (introduced in Section 3), which can be considered as a rather straightforward extension of MAML.

Original: 4. E XPERIMENTAL R ESULTS
Revised: 4. E XPERIMENTAL R ESULTS

Original: To fully test the effectiveness of ADML, we conducted a comprehensive empirical study, which covers various possible cases.
Revised: To fully test the effectiveness of ADML, we conducted a comprehensive empirical study, which covers various possible cases.

Original: The experimental results on MiniImageNet and CIFAR100 with FGSM attack mechanism are presented in Tables 1–2 and Tables 3–4 (in supplementary material) respectively.
Revised: The experimental results on MiniImageNet and CIFAR100 with FGSM attack mechanism are presented in Tables 1–2 and Tables 3–4 (in supplementary material) respectively.

Original: Experimental results on MiniImageNet with the other three attack mechanisms (i.e., FFGSM,
Revised: Experimental results on MiniImageNet with the other three attack mechanisms (i.e., FFGSM,

Original: RFGSM and RPGD) are shown in Tables 5–10 (in supplementary material).
Revised: RFGSM and RPGD) are shown in Tables 5–10 (in supplementary material).

Original: Each entry in these tables gives the average classiﬁcation accuracy (with 95% conﬁdence intervals) of the corresponding test case, and the best results for each test case are marked in bold.
Revised: Each entry in these tables gives the average classiﬁcation accuracy (with 95% conﬁdence intervals) of the corresponding test case, and the best results for each test case are marked in bold.

Original: The experiments were conducted in six different test cases (combinations): “ Clean-Clean ”, “ Clean-Adversarial ”, “ Adversarial-Clean ”, “ Adversarial-Adversarial ”, “ 40%-Clean ” and “ 40%-
Revised: The experiments were conducted in six different test cases (combinations): “ Clean-Clean ”, “ Clean-Adversarial ”, “ Adversarial-Clean ”, “ Adversarial-Adversarial ”, “ 40%-Clean ” and “ 40%-

Original: Adversarial ”.
Revised: Adversarial ”.

Original: The ﬁrst part of each combination (corresponding to a row) represents the training data used in the inner gradient update (or support set for Matching Networks and Relation Networks) during the meta-testing, while the second part (corresponding to a column) represents the testing data (or query set for Matching Networks and Relation Networks) for evaluation.
Revised: The ﬁrst part of each combination (corresponding to a row) represents the training data used in the inner gradient update (or support set for Matching Networks and Relation Networks) during the meta-testing, while the second part (corresponding to a column) represents the testing data (or query set for Matching Networks and Relation Networks) for evaluation.

Original: “Clean” means clean samples only; “Adversarial” means adversarial samples only; and “40%” means that 40% samples of each class are adversarial and the rest 60% are clean, which represents intermediate cases.
Revised: “Clean” means clean samples only; “Adversarial” means adversarial samples only; and “40%” means that 40% samples of each class are adversarial and the rest 60% are clean, which represents intermediate cases.

Original: Note that the combinations, “40%-Clean” and “40%-Adversarial”, do not exist for 1-shot learning since there is only one sample per class.
Revised: Note that the combinations, “40%-Clean” and “40%-Adversarial”, do not exist for 1-shot learning since there is only one sample per class.

Original: Based on the results in Tables 1–2, we can make the following observations:
Revised: Based on the results in Tables 1–2, we can make the following observations:

Original:  Just like MAML, ADML is indeed an effective meta-learner since it leads to quick learning from a small amount of new data for a new task.
Revised: 1) Just like MAML, ADML is indeed an effective meta-learner since it leads to quick learning from a small amount of new data for a new task.

Original: In the “Clean-Clean” cases, the general condition of meta-learning, ADML delivers desirable results, which are very close to the state-of-the-art given[Table] by R2D2, Relation Networks and MAML [Table]
Revised: In the “Clean-Clean” cases, the general condition of meta-learning, ADML delivers desirable results, which are very close to the state-of-the-art given by R2D2, Relation Networks and MAML, and consistently better than those of MAML-AD and Matching Networks.

Original: and consistently better than those of MAML-AD and Matching Networks. For example, in the case of 5-way 1-shot classiﬁcation with (cid:15) = 2 (Table 1), ADML gives an average classiﬁcation accuracy of 48.00% which is very close to that given by R2D (i.e., 49.52% and 51.80%), Relation Networks (49.67%) and MAML (48.47%) and it performs better than MAML-AD (43.13%) and Matching Networks (43.87%).
Revised: For example, in the case of 5-way 1-shot classiﬁcation with (cid:15) = 2 (Table 1), ADML gives an average classiﬁcation accuracy of 48.00% which is very close to that given by R2D (i.e., 49.52% and 51.80%), Relation Networks (49.67%) and MAML (48.47%) and it performs better than MAML-AD (43.13%) and Matching Networks (43.87%).

Original: Note that the proposed ADML focuses on the cases with adversarial samples, and it is reasonable that it performs slightly worse on the cases with only clean samples.
Revised: Note that the proposed ADML focuses on the cases with adversarial samples, and it is reasonable that it performs slightly worse on the cases with only clean samples.

Original: 2) ADML is robust to adversarial samples since it only suffers from a minor performance degradation when encountering adversarial samples.
Revised: 2) ADML is robust to adversarial samples since it only suffers from a minor performance degradation when encountering adversarial samples.

Original: For example, for the 5-way 5-shot classiﬁcation with (cid:15) = 2 (Table 2), ADML gives classiﬁcation accuracies of 57.03%, 58.06%, 55.27%, 58.12% and 55.22% in the ﬁve test cases respectively.
Revised: For example, for the 5-way 5-shot classiﬁcation with (cid:15) = 2 (Table 2), ADML gives classiﬁcation accuracies of 57.03%, 58.06%, 55.27%, 58.12% and 55.22% in the ﬁve test cases respectively.

Original: Compared to the “Clean-Clean” case (i.e., 59.38%), the performance degradation is only 4.16% in the worst-case and 2.64% on average.
Revised: Compared to the “Clean-Clean” case (i.e., 59.38%), the performance degradation is only 4.16% in the worst-case and 2.64% on average.

Original: However, the classiﬁcation accuracies given by the other meta-learning algorithms, including MAML-AD, drop  substantially when there are adversarial samples.
Revised: However, the classiﬁcation accuracies given by the other meta-learning algorithms, including MAML-AD, drop [Table] [Table] substantially when there are adversarial samples.

Original: For example, for MAML, the accuracy drops from 61.45% (“Clean-Clean”) to 36.65% (“Clean-Adversarial”) when injecting adversarial samples for testing, which represents a signiﬁcant degradation of 24.80%.
Revised: For example, for MAML, the accuracy drops from 61.45% (“Clean-Clean”) to 36.65% (“Clean-Adversarial”) when injecting adversarial samples for testing, which represents a signiﬁcant degradation of 24.80%.

Original: Similar observations can be made for MAML-AD, Matching Networks, Relation Networks, R2D2 (64C) and R2D2, which represent substantial degradations of 15.48%, 22.26%, 25.48%, 36.14% and 38.47% respectively.
Revised: Similar observations can be made for MAML-AD, Matching Networks, Relation Networks, R2D2 (64C) and R2D2, which represent substantial degradations of 15.48%, 22.26%, 25.48%, 36.14% and 38.47% respectively.

Original: 3) ADML generally outperforms all the other meta-learning algorithms in the test cases with adversarial samples.
Revised: 3) ADML generally outperforms all the other meta-learning algorithms in the test cases with adversarial samples.

Original: For instance, in the “Clean-Adversarial” cases of 5-way 5-shot learning with (cid:15) =(
Revised: For instance, in the “Clean-Adversarial” cases of 5-way 5-shot learning with (cid:15) =(

Original: Table 2), ADML achieves an accuracy of 57.03%, which represents 20.38%, 15.38%, 23.30%, 18.66%, 27.69% and 27.08% improvements over MAML, MAML-AD, Matching Networks, Relation Networks, R2D2 (64C) and R2D2 respectively.
Revised: Table 2), ADML achieves an accuracy of 57.03%, which represents 20.38%, 15.38%, 23.30%, 18.66%, 27.69% and 27.08% improvements over MAML, MAML-AD, Matching Networks, Relation Networks, R2D2 (64C) and R2D2 respectively.

Original: This clearly shows the superiority of the adversarial meta-training procedure of the proposed ADML compared to MAML-AD, the straightforward adversarial meta-learner.
Revised: This clearly shows the superiority of the adversarial meta-training procedure of the proposed ADML compared to MAML-AD, the straightforward adversarial meta-learner.

Original: Even so, MAML-AD still generally performs better than the rest meta-training algorithms, when dealing with adversarial samples.
Revised: Even so, MAML-AD still generally performs better than the rest meta-training algorithms, when dealing with adversarial samples.

Original: Note that an exception occurs in the case “40%-Clean”, where the accuracy of ADML is slightly lower than that of R2D2, even with 40% adversarial samples.
Revised: Note that an exception occurs in the case “40%-Clean”, where the accuracy of ADML is slightly lower than that of R2D2, even with 40% adversarial samples.

Original: This implies that in the 5-shot learning cases with adversarial samples, the clean samples (even with a number smaller than 5) in the training data play an important role when classifying the testing data composed of clean samples.
Revised: This implies that in the 5-shot learning cases with adversarial samples, the clean samples (even with a number smaller than 5) in the training data play an important role when classifying the testing data composed of clean samples

Original: 4) When the perturbation of adversarial samples escalates, ADML maintains stable performance.
Revised: 4) When the perturbation of adversarial samples escalates, ADML maintains stable performance.

Original: For example, for 5-way 1-shot learning, when (cid:15) increases from 0 .
Revised: For example, for 5-way 1-shot learning, when (cid:15) increases from 0 .

Original: 2 to 2 (Table 1), ADML only leads to minor degradations of 0 .
Revised: 2 to 2 (Table 1), ADML only leads to minor degradations of 0 .

Original: 2% , 3 . 9% and 0 .
Revised: 2% , 3 . 9% and 0 .

Original: 5% in the corresponding three cases involving adversarial samples.
Revised: 5% in the corresponding three cases involving adversarial samples.

Original: However, much more signiﬁcant degradations can be observed when the other meta-learning algorithms are applied.
Revised: However, much more signiﬁcant degradations can be observed when the other meta-learning algorithms are applied.

Original: For instance, the accuracies of MAML suffer from 13 .
Revised: For instance, the accuracies of MAML suffer from 13 .

Original: 30% and 9 .
Revised: 30% and 9 .

Original: 44% drops in these three cases when increasing (cid:15) from 0 .
Revised: 44% drops in these three cases when increasing (cid:15) from 0 .

Original: 2 to 2 .
Revised: 2 to 2 .

Original: 5) As expected, when lower perturbations exerted, the adversarial samples are not signiﬁcantly different from the corresponding clean samples, which brings about relatively close results in different test cases, not only for ADML, but also for the other meta-learning algorithms.
Revised: 5) As expected, when lower perturbations exerted, the adversarial samples are not signiﬁcantly different from the corresponding clean samples, which brings about relatively close results in different test cases, not only for ADML, but also for the other meta-learning algorithms.

Original: For instance, for the 5-way 5-shot classiﬁcation with (cid:15) = 0 .
Revised: For instance, for the 5-way 5-shot classiﬁcation with (cid:15) = 0 .

Original: 2 (Table 2), MAML gives a smaller gap of 8 .
Revised: 2 (Table 2), MAML gives a smaller gap of 8 .

Original: 42% between “Clean-Clean” and “Clean-Adversarial”, compared with that of 24 .
Revised: 42% between “Clean-Clean” and “Clean-Adversarial”, compared with that of 24 .

Original: 80% when (cid:15) = 2 .
Revised: 80% when (cid:15) = 2 .

Original: Similar observations can be made for the results corresponding to CIFAR100 and the other three attack mechanisms ((i.e., Tables 3–10 in supplementary materials)).
Revised: Similar observations can be made for the results corresponding to CIFAR100 and the other three attack mechanisms ((i.e., Tables 3–10 in supplementary materials)).

Original: In addition, we show how the loss and Top-1 accuracy change in Figures 2 and 3 during the meta-testing.
Revised: In addition, we show how the loss and Top-1 accuracy change in Figures 2 and 3 during the meta-testing.

Original: Speciﬁcally, these two ﬁgures show that when ADML, MAML and MAML-AD are applied on MiniImageNet, how the losses and Top-1 accuracies change with the gradient update step during the meta-testing in the four test cases of 5-way 1-shot learning with (cid:15)
Revised: Speciﬁcally, these two ﬁgures show that when ADML, MAML and MAML-AD are applied on MiniImageNet, how the losses and Top-1 accuracies change with the gradient update step during the meta-testing in the four test cases of 5-way 1-shot learning with (cid:15)

Original: = 2 .
Revised: = 2 .

Original: We observe that, for all the cases, the losses of the models learned with ADML drop sharply after only several gradient updates, and stabilize at small values during the meta-testing, which are generally lower than those of the other two methods.
Revised: We observe that, for all the cases, the losses of the models learned with ADML drop sharply after only several gradient updates, and stabilize at small values during the meta-testing, which are generally lower than those of the other two methods.

Original: Moreover, the Top-1 accuracies of the models learned with ADML rise sharply after only several gradient updates, and stabilize at values, which are generally higher than those of the other two methods (a little bit lower than that of MAML in “Clean-Clean” case).
Revised: Moreover, the Top-1 accuracies of the models learned with ADML rise sharply after only several gradient updates, and stabilize at values, which are generally higher than those of the other two methods (a little bit lower than that of MAML in “Clean-Clean” case).

Original: Similar trends can be observed in Figures 4–5 in supplementary materials for 5-way 5-shot learning with (cid:15)
Revised: Similar trends can be observed in Figures 4–5 in supplementary materials for 5-way 5-shot learning with (cid:15)

Original: = 2 .
Revised: = 2 .

Original: These observations further conﬁrm that ADML is suitable for meta-learning since it can quickly learn and adapt from small data for a new task through only a few gradient updates.
Revised: These observations further conﬁrm that ADML is suitable for meta-learning since it can quickly learn and adapt from small data for a new task through only a few gradient updates.

Original:  C ONCLUSIONS
Revised:  C ONCLUSIONS

Original: In this paper, we proposed a novel method called ADML (ADversarial Meta-Learner) for metalearning with adversarial samples, which features an adversarial way for optimizing model parameters θ during meta-training through the arm-wrestling between inner gradient update and metaupdate using both clean and adversarial samples.
Revised: In this paper, we proposed a novel method called ADML (ADversarial Meta-Learner) for metalearning with adversarial samples, which features an adversarial way for optimizing model parameters θ during meta-training through the arm-wrestling between inner gradient update and metaupdate using both clean and adversarial samples.

Original: The extensive experimental results have showed that 1) ADML is an effective meta-learner even in the cases with only clean samples; 2) a straightforward adversarial meta-learner, namely, MAML-AD, does not work well with adversarial samples; in addition, 3) ADML is robust to adversarial samples generated by different attack mechanisms, and outperforms other meta-learning algorithms including MAML on adversarial meta-learning tasks; and most importantly, 4) it opens up an interesting research direction and sheds light on dealing with the difﬁcult cases with limited and even contaminated samples.
Revised: The extensive experimental results have showed that 1) ADML is an effective meta-learner even in the cases with only clean samples; 2) a straightforward adversarial meta-learner, namely, MAML-AD, does not work well with adversarial samples; in addition, 3) ADML is robust to adversarial samples generated by different attack mechanisms, and outperforms other meta-learning algorithms including MAML on adversarial meta-learning tasks; and most importantly, 4) it opens up an interesting research direction and sheds light on dealing with the difﬁcult cases with limited and even contaminated samples