Adversarial Attacks are a Surprisingly Strong Baseline for Poisoning Few-Shot Meta-Learners
Abstract
This paper examines the robustness of deployed few-shot meta-learning systems when they are fed an imperceptibly perturbed few-shot dataset.
We attack amortized meta-learners, which allows us to craft colluding sets of inputs that are tailored to fool the system’s learning algorithm when used as training data.
Jointly crafted adversarial inputs might be expected to synergistically manipulate a classifier, allowing for very strong data-poisoning attacks that would be hard to detect.
We show that in a white box setting, these attacks are very successful and can cause the target model’s predictions to become worse than chance.
However, in opposition to the well-known transferability of adversarial examples in general, the colluding sets do not transfer well to different classifiers.
We explore two hypotheses to explain this: “overfitting” by the attack, and mismatch between the model on which the attack is generated and that to which the attack is transferred.
Regardless of the mitigation strategies suggested by these hypotheses, the colluding inputs transfer no better than adversarial inputs that are generated independently in the usual way.
Introduction
Standard deep learning approaches suffer from poor sample efficiency [Krizhevsky et al., 2012] which is problematic in tasks where data collection is difficult or expensive.
Few-shot learners have been developed to address this shortcoming by supporting rapid adaptation to a new task using only a few labeled examples
[Finn et al., 2017, Snell et al., 2017].
This success has made few-shot learners more attractive for increasingly sensitive applications where the repercussions of confidentlywrong predictions are severe, such as clinical risk assessment [Sheryl Zhang et al., 2019], glaucoma diagnosis [Kim et al., 2017] and diseases identification in skin lesions [Mahajan et al., 2020].
As few-shot learners gain popularity, it is essential to understand how robust they are and how they may be exploited.
It is well known that standard classifiers are vulnerable to adversarial inputs which have been purposefully and imperceptibly modified to cause incorrect predictions [Biggio and Roli, 2017].
Such examples may be presented to a model either at test time, called evasion attacks [Biggio et al., 2017] or adversarial examples [Szegedy et al., 2014], or at training time, which is referred to as poisoning [Newsome et al., 2006, Rubinstein et al., 2009].
The concepts of evasion and poisoning can be re-stated in terms of few-shot learners.
At meta-test time, a few-shot learner is presented with an unseen task containing a few labeled examples, the support set , and a number of unlabeled examples to classify, called the query set .
We can perpetrate a “standard” adversarial attack by perturbing an image in the query set or perform a poisoning attack by perturbing images in the support set.
In this paper, we propose a poisoning attack against amortized meta-learners that jointly optimizes attack images through the meta-learner’s adaptation mechanism .
We produce a colluding set of adversarial images that is explicitly tailored to fool the learning algorithm.
We consider as a baseline the insertion of standard adversarial attack images into the support set.
We expect our attack to perform better than this baseline, since standard adversarial attacks are optimised individually to cause misclassification by a fixed classifier (rather than being curated as a set to fool the adaptation mechanism itself).
We may also expect the new attack to transfer to other learners, especially considering the well-known transferability of adversarial examples in literature Goodfellow et al. [2014], Papernot et al.
[2016a], Szegedy et al.
Our contributions are as follows
We define a poisoning attack on few-shot classifiers, referred to as adversarial support poisoning (ASP) or simply as support attacks .
This applies coordinated adversarial perturbations to produce a colluding set of inputs, generated by jointly backpropagating through the meta-learner’s adaptation process to minimize model accuracy over a set of query points.
To the best of the authors’ knowledge, ours is the first work to perpetrate poisoning attacks on trained few-shot classifiers.
In a white box setting ASP is a very strong attack, far more effective than the baseline.
Surprisingly, we show that ASP attacks do not transfer significantly better than the swap attack baseline, which inserts adversarial examples generated “in the usual way” into the support set.
We observe that transferability of support attacks are highly dependent on the feature extractor used by the meta-learner.
Although we propose various mitigation strategies to improve the transferability of support set attacks, we do not succeed at significantly improving over the efficacy of a swap attack, which is simpler and cheaper.
The last two findings are unexpected: we expect the ASP attack to be significantly stronger than the swap attack baseline because it is tailored to fool the learner’s adaptation process, is optimized over a set of query points and is able to collude within the adversarial set.
Background
We focus on image classification, denoting input images by x ∈ R ch × W × H where W is the image width, H the image height, ch the number of image channels and image labels y ∈ { 1 , . . .
, C } where
C is the number of image classes.
We use bold x and y to denote sets of images and labels.
We consider the few-shot image classification scenario using a meta-learning approach.
Rather than a single, large dataset D , we assume access to a dataset D = { τ t } Kt =1 comprising a large number of training tasks τ t , drawn i.i.d.
from a distribution p ( τ ) .
An example task is shown in Fig.
The data for a task consists of a support set D S
= { ( x n , y n ) } Nn =1 comprising N elements, with the inputs x n and labels y n observed, and a query set D Q = { ( x ∗ m , y ∗ m ) }
Mm =1 with M elements for which we wish to make predictions.
We may use the shorthand D S
= { x , y } and D Q = { x ∗ , y ∗ } .
The meta-learner g takes as input the support set D S and produces task-specific classifier parameters ψ
= g ( D S ) which are used to adapt the classifier f to the current task.
The classifier can now make task-specific predictions f ( x ∗ , ψ = g ( D S )) for any test input x ∗ ∈ D Q .
Here the inputs x ∗ are observed and the labels y ∗ are only observed during meta-training (i.e. training of the meta-learning algorithm).
Note that the query set examples are drawn from the same set of labels as the examples in the support set.
The majority of modern meta-learning methods employ episodic training [Vinyals et al., 2016], as detailed in A.1.
At meta-test time, the classifier f is required to make predictions for query set inputs of unseen tasks, which will often include classes that have not been seen during meta-training, and
D S will contain only a few observations.
The canonical example for modern gradient-based few-shot learning systems is MAML [Finn et al., 2017].
Another widely used class of meta-learners are amortized-inference or black box based approaches e.g,
V ERSA [Gordon et al., 2019] and CNAP S
[Requeima et al., 2019a].
In these methods, the task-specific parameters ψ are generated by one or more hyper-networks , g [Ha et al., 2016].
Prototypical Networks (ProtoNets)
[Snell et al., 2017] is a special case of this which is based on metric learning and employs a nearest neighbor classifier.
We focuses on Simple CNAP S Bateni et al.
[2020] – which improves on the performance of CNAP S – and uses a ProtoNets classifier head with an adaptive feature extractor like that of CNAP S ;
but we have also considered attacks against
MAML, ProtoNets, and CNAP S with similar outcomes.
Attacking Few-Shot Learners
We summarize the threat model in terms of the adversary’s goal, capabilities and knowledge.
In this work, we develop poisoning attacks that degrade the model availability (i.e. affect prediction results indiscriminately such that it is useless)
[Jagielski et al., 2018].
We allow the attacker to manipulate some fraction of the support set and further constrain pattern modifications to be imperceptible (i.e. within some ϵ of the original image, measured using the ℓ ∞ norm).
When considering transfer attacks, we call the model used generate the attack the surrogate and the model to which we transfer the attack the target .
We presume access to the surrogate model’s gradients and internal state.
We don’t assume any access to the target model’s internal state, though some of the experiments do access a limited number of the target’s predictions.
We also assume access to enough data to form a query set.
Consider a target model that has been trained and deployed as a service.
As described in Section 1, a malicious party could perpetrate two kinds of attacks: a “standard” adversarial attack (hereafter referred to as a query attack) by perturbing elements in the query set of a task; or perform a poisoning attack by perturbing elements in the support set.
We expand on these below:
Query Attack Given a meta-learner that has already adapted to a specific task, the attacker perturbs a single test input x ∗ in such a way that it will be misclassified.
This corresponds to solving arg max δ L ( f ( x ∗ + δ, g ( x , y )) , y ∗ ) .
Refer to Appendix A.6 for details.
These attacks are essentially evasion attacks as considered in Biggio et al.
[2017] and many algorithms can be used to generate adversarial examples [Madry et al., 2017, Carlini and Wagner, 2017, Chen et al., 2017].
Query attacks have been perpetrated successfully against few-shot learners [Goldblum et al., 2019, Yin et al., 2018].
Adversarial Support Poisoning (ASP)
The attacker perturbs the dataset that the target metalearner will learn from so as to influence all future test predictions .
The attacker thus computes a perturbed support set ˜ D S
= { ˜ x , y } whose inputs are jointly optimized to fool the system on a specific query set, which we call the seed query set , with the goal of generalizing to unseen query sets.
This corresponds to solving arg max δ L ( f ( x ∗ , g ( x + δ , y )) , y ∗ ) such that ∥ δ ∥ ∞ <
ϵ , where D Q = { x ∗ , y ∗ } denotes the seed query set and ϵ is the maximum size of the perturbation.
Refer to Algorithm 1 for details.
Our attack is a poisoning attack, since the attacker is manipulating data that the model will use to do inference.
However, it is important to note the the attack is perpetrated at meta-test time, after the meta-learner has already been meta-trained.
Unlike a query attack, which is generated by backpropagating through the adapted learner with respect to a single point, our attack backpropagates jointly through the adaptation process with respect to the loss on the entire query set and should thus be a much stronger poisoning attack than say, inserting a query point into the support set.
Without loss of generality, we use Projected Gradient Descent (PGD)
[Madry et al., 2017] to generate perturbed support sets because it is effective, simple to implement, and easily extensible to sets of inputs.
Algorithm 1 PGD for ASP
Require:
I min , I max : Min/Max image intensity, γ : Step size, L : Number of iterations,
D S ≡ { x , y } , D Q ≡ { x ∗ , y ∗ } , L ≡ cross-entropy, ϵ : Perturbation amount 1: procedure PGDS( D S , D Q , f, g ) 2: δ ∼ U ( − ϵ, ϵ ) 3: ˜ x ← clip ( x + δ , I min , I max ) 4: for i ∈ 1 , ..., L do 5: δ ← sgn ( ∇ ˜ x L ( f ( x ∗ , g (˜ x , y )) , y ∗ )) 6: ˜ x ← clip (˜ x + γ δ , I min , I max ) 7: ˜ x ← x + clip (˜ x − x , − ϵ, ϵ ) 8: return ˜ x
Experiments
The experiments presented in the main body of the paper are carried out on Simple CNAP S [Bateni et al., 2020] using the challenging M ETA -D ATASET benchmark (refer to Appendix A.2 and for details on the meta-data training protocols and M ETA -D ATASET ).
The Simple CNAP S model uses a
ProtoNets head in conjunction with an adaptive feature extractor.
The ProtoNets head uses Euclidean distance and the feature extractor is a Resnet18 endowed with Feature-wise Linear Modulation (FiLM) layers [Perez et al., 2018], which scales and shifts a feature map x by ζ x + β .
In the
ILSVRC
Aircraft
Birds
Quick Draw VGG Flower MSCOCO
MNIST
CIFAR
CIFAR100  97 .100 .
ASP Specific
ASP General
Swap
R e l a t i v e
D e c r e a s e i n A cc u r a c y ( % ) meta-learning setting, FiLM parameters are generated by an adaptation network as part of ψ , the learner’s adaptation to a new task, and allows the feature extractor to adapt flexibly to each new with relatively few additional parameters [Requeima et al., 2019a].
In M ETA -D ATASET , task support sets may be large — up to 500 images across all classes.
In a realistic scenario, an attacker would not likely be able to perturb all the images in such a large support set, so we only perturb a specified fraction of the support set in each experiment.
Each task is composed of a support set, a seed query set and up to 50 unseen query sets used for attack evaluation (some M ETA -D ATASET benchmarks do not have sufficiently many patterns available to form 50 query sets and thus were excluded from the experiments).
The unseen query sets are all disjoint from the seed query set to avoid information leakage.
For each task, we generate an adversarial support set using the original support set and corresponding seed query set.
The adversarial support set is then evaluated on the task’s unseen query sets.
We refer to the average classification accuracy on the seed query sets as the ASP Specific attack accuracy, and when evaluating the attack on unseen query sets we refer to it as the ASP General attack accuracy.
An important baseline is the swap attack , which is generated by using the task’s support set and query set to produce a query attack, then “swap” the role of the adversarial query set by using it as a support set.
Like our ASP attack, the swap attack is evaluated on unseen query sets.
Note that the swap attack is far less expensive to compute than the ASP attack, since it only requires backpropagation through the adapted learner and not through the entire meta-learning process.
4.1 White Box Attack
We first consider the simplest scenario, a white box attack in which the surrogate and target model are the same.
Fig.
1 shows the relative decrease in accuracy of Simple CNAP S on M ETA -D ATASET .
In general, the clean accuracy of the surrogate and target models may be quite different and so we normalise the effect an attack has by computing the percentage relative decrease in classification accuracy as follows: 100% × ( a clean − a attack )
/a clean where a clean is the clean classification accuracy before the attack, and a attack is the classification accuracy after the attack.
Unnormalised results are in A.5.
Although it is expected for ASP Specific to be very effective, the ASP attack remains highly effective even on unseen query sets (see ASP General ), easily out-performing the
Swap baseline, in spite of the fact that only 20% of the support set shots are poisoned.
At least in the white box setting, ASP is very strong attack.
4.2 Attack Transfer
We now turn our attention to the more difficult scenario, in which the target and surrogate models are different.
We fix our surrogate model as Simple CNAP S with a particular Resnet18 feature extractor (hereafter referred to Resnet18 A).
We consider a Simple CNAP S model with a slightly different feature extractor: Resnet18 B, which was trained on the same data with similar training conditions, but with no FiLM parameters.
We also consider turning the FiLM parameters for {+Resnet
+} A on and off.
Although this seems a rather trivial extension, the results shown in Fig.
2 indicate that the ASP attack no longer outperforms the Swap baseline by a statistically significant margin.
Fig.
2 considers three scenarios in increasing order of difficulty.
For Scenario A, the perturbation size is very large ( ϵ = 1 ) and the entire support set has been adversarially perturbed.
At this scale, tampering could be clearly inferred by inspection and it is unlikely that an adversary would have control over the entire support set, so this is the weakest scenario.
In this setting, we see good transfer between the surrogate model, Resnet18 A with FiLM, and the target models (Resnet18 A without FiLM and Resnet18 B without FiLM).
Scenario B reduces the perturbation size to be mostly imperceptible ( ϵ = 0 . 3 ), but still perturbs the entire support set.
Here, we start to see a reduction in attack effectiveness when
[Figure] [Figure]
transferring to Resnset18 B. In Scenario C, we use ϵ = 0 . 3 and also reduce the proportion of poisoned samples in the support set to 50% .
Here we see significant reduction in transfer, even to Resnet18 A without FiLM.
We also note that unlike the white box attack, we do not perform significantly better than the Swap attack, and possibly perform a little worse.
We know from literature that adversarial attacks transfer well and should thus be fairly robust to the kind of feature extractor being used.
Since we’re only changing the feature extractor and keeping the type of learner the same, we expect the support attacks, which are tailored to fool the learning mechanism, to have a benefit over the swap attack.
However, this is not borne out in experiments.
We consider two avenues of explanation that account for the poor transfer between feature extractors: mismatch between the target and surrogate models, and “overfitting” of the attack to the support set and/or surrogate model.
We consider a number of mitigation strategies intended to address these two problems below:
Decision Boundary Alignment Since the target and surrogate models have different feature extractors, it may be that their embedding spaces and consequently also their decision boundaries are very different.
We can try to “re-align” the surrogate model’s decision boundaries by training it to produce the same predictions as the target model, an approach which has been used in literature when perpetrating transfer attacks [Papernot et al., 2016b].
In a meta-learning setting, this corresponds to relabeling the support set presented to the surrogate model based on the target model’s predictions.
The labels of the support set when presented to the surrogate model will thus not be the true labels, but rather the labels assigned to the support images by the target model.
The relabeled support set is also used to generate the swap attack so that the comparison is fair.
As shown in Fig. 3(a), relabeling the support set does significantly improve transfer compared to the “blind” scenario in which the surrogate model has no knowledge about the target model’s decision boundaries.
However, ASP is still not significantly better than the swap attack.
Hot Start The ASP optimization procedure may be finding a different minimum, which is less robust to different feature extractors than that found by the swap attack.
We propose to hot-start the
ASP attack, where given a support set D S , we spend half the optimization budget performing a query attack to produce D ′ S , which is then used as input to an adversarial support set attack to produce ˜ D S with the other half of the budget.
In this way, we start the ASP attack off in the region of the local minimum found by the query attack.
However, as shown in Fig.
1, the hot start does not improve on the normally initialized ASP attack and may be worse in some scenarios.
Support Set
Shuffling Simple CNAP S makes use of a set encoder to produce a representation of the entire support set and so the attack must effectively manipulate the poisoned examples so that the embedded representation of the poisoned support set is sufficiently different to affect predictions.
This may explain why we observe that the poisoning attack requires a high proportion of poisoned examples in the support set in order to be effective.
The potency of the poisoned examples thus depend on the rest of the (unperturbed) support set.
When transferred, the support set may embed very differently, especially if a large proportion of it is not adversarial, negating the effects of the poisoned points.
In order to generate solutions that are less dependent on the rest of the support set we propose a procedure whereby the clean elements in the support set are not kept fixed for the duration of attack generation, but instead shuffle different clean images in and out of the support set.
Fig.
3(b) considers two configurations: each varying the percentage of the support set that is poisoned (50% and 20%, respectively) and the proportion of clean images in the support set during attack generation (the clean : poisoned ratio is 0 .
5 : 1 and 1 . 5 : 1 , respectively).
Note that we are careful not to assume access to more data than any of the other scenarios.
The first setting performs no better than an attack generated without shuffling.
The second setting performs worse – although the attack has access to a more varied support set, this comes at the price of poisoning a smaller proportion of the support set since we cannot access additional data.
Dropout If the attack optimization procedure is overfitting to the surrogate model, then regularization by dropout is an established remedy.
By applying dropout to the feature extractor, we may hope for the resulting attack to generalize better to new feature extractors.
We drop out the FiLM parameters of the surrogate’s feature extractor using one of three different strategies: (1) Block-Wise - the FiLM parameters for an entire Resnet block is either on/off, i.e. either computed as usual or we set ζ = 1 . 0 , β = 0 ; (2) Layer-Wise - each FiLM layer within a Resnet block may be switched on or off individually, i.e. compute ζ and β as usual, but apply a dropout mask; (3) Gaussian Dropout - the FiLM parameters are smoothed by a Gaussian with standard deviation given by (cid:112) d/ (1 − d ) where d is the dropout probability.
Although Fig. 3(c) shows slight improvement in transferability, the difference between the ASP and swap attacks remain statistically insignificant.
Conclusion
In this paper, we propose ASP, an attack against amortized meta-learners that jointly crafts sets of colluding points which can manipulate a meta-learner when used as training data.
We expect
ASP attacks to be strong because the attack images are generated by backpropagating through the learner’s adaptation process with respect to the loss over an entire set of test inputs and are thus able to collude to fool the classifier.
We may also expect ASP enjoy the same robustness to different feature extractors as a standard evasion attack.
We show that although ASP is very effective in a white box setting, it does not transfer well, even to the same learner with a different feature extractor.
We explore a number of strategies to improve the transferability of ASP, but in all cases ASP was not able to transfer significantly better than the swap attack baseline (which performs a query or evasion attack in the usual way and simply swaps it into the meta-learner’s support set).
Even though the swap attack should be much weaker, since it is generated to fool the learner only for the image in question, it proves a very effective baseline for poisoning few-shot learners.
Since an attacker will rarely have access to the internal workings of the target model, the transferability of an attack is an important consideration, which prevents the deployment of ASP in real-life.
But future work may glean insight into making models more robust.
Acknowledgments and Disclosure of Funding
This work was performed using resources provided by the Cambridge Service for Data Driven
Discovery (CSD3) operated by the University of Cambridge Research Computing Service ( www. csd3.cam.ac.uk ), provided by Dell EMC and Intel using Tier-2 funding from the Engineering and
Physical Sciences Research Council (capital grant EP/T022159/1), and DiRAC funding from the
Science and Technology Facilities Council ( www.dirac.ac.uk ).
Richard E. Turner is supported by funding from Google, Amazon, ARM, Improbable and Microsoft. The paper built on software developed under Prosperity Partnership EP/T005386/1 between Microsoft Research and the University of Cambridge. A Appendix A.1 Episodic Training
There has been an explosion of meta-learning based few-shot learning algorithms proposed in recent years.
For an in-depth review see Hospedales et al.
The majority of modern meta-learning methods employ episodic training [Vinyals et al., 2016].
During meta-training, a task τ is drawn from p ( τ ) and randomly split into a support set D S and query set D Q .
Fig.
4 depicts an example few-shot classification task.
The meta-learner g takes as input the support set D S and produces task-specific classifier parameters ψ
= g ( D S ) which are used to adapt the classifier f to the current task.
The classifier can now make task-specific predictions f
( x ∗ , ψ = g ( D S )) for any test input x ∗ ∈ D Q .
Refer to Clean in Fig.
A loss function L ( f ( x ∗ , ψ ) , y ∗ ) then computes the loss between the predictions for the label f
( x ∗ , ψ ) and the true label y ∗ .
Assuming that L , f , and g are differentiable, the meta-learning algorithm can then be trained with stochastic gradient descent by back-propagating the loss and updating the parameters of f and g .
A.2 Few-shot Learner Meta-training Protocols
In the following, the meta-training protocols for the few-shot learners used in the experiments.
A.2.1 Datasets
mini ImageNet mini ImageNet is a subset of the larger Imagenet dataset [Russakovsky et al., 2015] created by Vinyals et al.
It consists of 60,000 color images that is sub-divided into 100 classes, each with 600 instances.
The images have dimensions of 84 × 84 pixels.
Ravi and Larochelle [2017] standardized the 64 training, 16 validation, and 20 test class splits.
mini ImageNet has become a defacto standard dataset for benchmarking few-shot image classification methods with the following classification task configurations: (i) 5-way, 1-shot; (ii) 5-way, 5-shot.
[Equation]
al., 2020] is composed of ten (eight train, two test) image classification datasets.
We augment Meta-Dataset with three additional held-out datasets: MNIST [LeCun et al., 2010], {+CIFAR
+} [Krizhevsky and Hinton, 2009], and CIFAR [Krizhevsky and Hinton, 2009].
The challenge constructs few-shot learning tasks by drawing from the following distribution.
First, one of the datasets is sampled uniformly; second, the “way” and “shot” are sampled randomly according to a fixed procedure; third, the classes and support / query instances are sampled.
Where a hierarchical structure exists in the data (ImageNet or Omniglot), task-sampling respects the hierarchy.
In the meta-test phase, the identity of the original dataset is not revealed and the tasks must be treated independently (i.e. no information can be transferred between them).
Notably, the meta-training set comprises a disjoint and dissimilar set of classes from
[Equation]
clock
S upp o r t
S e t
Q u e r y
S e t 𝑔 𝑓 𝐷 𝑆 = {𝒙,𝒚} ෥𝒙 ∗ 𝒚 ∗ 𝝍
Query ෥𝒙 ∗ = PGDQ(𝐷 𝑆 ,𝐷 𝑄, 𝑓,𝑔) 𝑔 𝑓 ෩𝐷 𝑆 = {෥𝒙,𝒚} 𝒙 ∗ 𝒚 ∗ 𝝍
ASP ෥𝒙 = PGD𝑆(𝐷 𝑆 ,𝐷 𝑄, 𝑓,𝑔) 𝑔 𝑓 𝐷 𝑆 = {𝒙,𝒚} 𝒙 ∗ 𝒚 ∗ 𝝍
Clean 𝑔 𝑓 ෩𝐷 𝑆 = {෥𝒙 ∗ ,𝒚 ∗ } 𝒙 𝒚 𝝍
Swap ෥𝒙 ∗ =
PGDQ(𝐷 𝑆 ,𝐷 𝑄, 𝑓,𝑔) 𝑔
𝑓 ෩𝐷 𝑆 = {𝒙,෥𝒚} 𝒙 ∗ 𝒚 ∗ 𝝍
Label Shift ෤𝑦 𝑛 = 𝑦 𝑛+
mod𝐶 those used for meta-test.
M ETA -D ATASET is presently, the "gold standard" for evaluating few-shot classification methods.
Full details are available in Triantafillou et al.
In our experiments, we excluded the Omniglot, Textures, Fungi, and Traffic Signs datasets from evaluation because their test splits are too small to allow for a fair assessment of the attack’s generalization, even though the attacks reduced the classification accuracy on those datasets to approximately zero in the ASP Specific case.
For M ETA -D ATASET , we meta-trained Simple CNAP S using the code from Requeima et al.
[2019b] with FiLM feature adaptation.
We made modifications to the code to enable various adversarial attacks.
The meta-trained model attained the following results: ilsvrc 2012: 55 .
1 ± 1 . 1 , omniglot: 90 . 8 ± 0 .
6 , aircraft: 82 . 3 ± 0 .
6 , cu birds: 74 . 0 ± 0 .
9 , dtd: 63 .
4 ± 0 .
7 , quickdraw: 75 . 3 ± 0 . 8 , fungi: 44 .
6 ± 1 . 0 , vgg flower: 90 . 3 ± 0 .
5 , traffic sign: 67 . ± 0 . 8 , mscoco:
8 ± 1 . 0 , mnist: 91 .
4 ± 0 .
5 , cifar10: 72 .
5 ± 0 .
8 , cifar100:
± 1 . 0 .
A.3 Attack Summary
We summarize the types of attacks we perform in Fig.
The first scenario, Clean , illustrates how the meta-learner g performs a test-time task, taking the support set D S as input to produce parameters ψ
= g ( D S ) which are used to adapt the classifier f to the task.
The classifier makes task-specific predictions f ( x ∗ , ψ = g ( D S )) for any test input x ∗ ∈ D Q .
ASP and Query illustrate the ASP and Query attacks as discussed in Section 3.
Swap illustrates a swap attack, which is used as a baseline comparison for ASP, where a set of images are perturbed with a query attack and then inserted into the support set.
Query attacks are typically cheaper to compute, since they do not require back-propagation through the meta-learner, so it is an important baseline to consider.
A.4 Additional Experimental Details
In our experiments, all the input images were re-scaled to have pixel values between − 1 and 1 .
We considered perturbations using the ℓ ∞ norm, on a scale of [ − 1 , 1] , so that ϵ = 0 .
1 corresponds to allowing ± 10% or an absolute change of ± 0 .
2 to the intensity of each pixel in an image.
We calculated the perturbation step size γ to depend on ϵ and the maximum number of iterations, so that γ = r ϵL , where r is a scaling coefficient.
A.5 Unnormalised Large-Scale Attack Results
This section of the appendix presents the unnormalized results for all figures, along with 95% confidence intervals.
In Table 1 we present the unnormalized numbers for Fig.
1, which shows the efficacy of ASP in a white box setting.
Table 2 considers transfer between a surrogate model and target models with different feature extractors as presented in Fig.
2 (Left) .
Tables 3 and 4 consider the effect of the hot-start mitigation strategy for different perturbation sizes as presented in Fig.
2 (Right) .
Tables 5,and 7 correspond to Fig.
3 and show the effect of other mitigation strategies, specifically decision boundary re-alignment, shuffling different clean inputs into the support set, and dropout.
All numbers are percentages and the ± sign indicates the 95% confidence interval.
[Table]
set adversarial; Scenario B: ϵ = 0 .
3 with entire support set adversarial; Scenario C: ϵ = 0 .
3 with 50% of support set adversarial.
Attacks are generated using PGD with L = 100 .
These results correspond to Fig.
2 ( Left ).
Clean
ASP
[Table]
Surrogate
Scenario
[Table]
FiLM)
[Table]
to
Scenario
[Table]
Resnet
[Table]
Transfer to
Scenario
[Table]
Clean
ASP
Swap
Hot Start
Surrogate
[Table]
Resnet
[Table] [Table]
Transfer to Resnet18 A (with FiLM)
[Table]
Resnet
[Table]
Clean
ASP
[Table]
Surrogate
[Table]
Transfer
[Table]
Resnet
[Table]
with FiLM
[Table]
Transfer to Resnet
[Table]
FiLM)
[Table] [Table]
[Table]
The support set is 50% adversarial.
Attacks are generated using PGD with ϵ = 0 .
3 , γ = 0 . 0167 ,
L = 100 .
This table corresponds to Fig. 3(c).
[Table]
A.6 Query Attacks
We present our algorithm for performing query attacks with PGD in Algorithm 2. Algorithm PGD for Attack
Require
−