Smoothed Embeddings for Certiﬁed Few-Shot Learning
Abstract
Randomized smoothing is considered to be the state-of-the-art provable defenseagainst adversarial perturbations.
However, it heavily exploits the fact that classiﬁers map input objects to class probabilities and do not focus on the ones thatlearn a metric space in which classiﬁcation is performed by computing distances toembeddings of classes prototypes.
In this work, we extend randomized smoothingto few-shot learning models that map inputs to normalized embeddings.
We provideanalysis of Lipschitz continuity of such models and derive robustness certiﬁcateagainst ` 2 -bounded perturbations that may be useful in few-shot learning scenarios.
Our theoretical results are conﬁrmed by experiments on different datasets.
Introduction
Few-shot learning is a setting in which a classiﬁcation model is evaluated on the classes not seenduring the training phase.
Nowadays quite a few few-shot learning approaches based on neuralnetworks are known.
Unfortunately, neural network based classiﬁers are intimately vulnerable toadversarial perturbations [40, 10] – accurately crafted small modiﬁcations of the input that maysigniﬁcantly alter the model’s prediction.
This vulnerability poses a restriction on the deployment ofsuch approaches in safety-critical scenarios, so the research interest in the ﬁeld of attacks on neuralnetworks has been great in recent years.
Several works studied this phenomena in different practical applications – image classiﬁcation[2, 33, 32, 39], object detection [17, 26, 46, 48], face recognition [18, 5, 52], semantic segmentation[7, 12, 48].
These studies show that it is easy to force a model to behave in the desired way byapplying imperceptible change to its input.
As a result, defenses, both empirical [4, 53, 14] andprovable [49, 21, 3, 45, 50, 15, 44, 34], were proposed recently.
Although empirical ones can be (andoften are) broken by more powerful attacks, the provable ones are of a big interest since they make itpossible to provide guarantees of the correctness of the work of a model under certain assumptions,and, thus, possibly broaden the scope of tasks which may be trusted to the neural networks.
Randomized smoothing [21, 3, 25] is the state-of-the-art approach used for constructing classiﬁcationmodels provably robust against small-norm additive adversarial perturbations.
This approach isscalable to large datasets and can be applied to any classiﬁer since it does not use any assumptionsabout model’s architecture.
Generally, the idea is following.
Suppose, we are given a base neuralnetwork classier f : R D !
[0 , 1] K that maps an input image x to a ﬁxed number of K classprobabilities.
Its smoothed version with the standard Gaussian distribution is:
[Equation]
As shown in [3], the new (smoothed) classiﬁer is provably robust at x to ` 2 -bounded perturbations ifthe base classiﬁer f is conﬁdent enough at x .
However, the proof of certiﬁcation heavily exploits
[Figure]
Smoothed embeddings
[Figure]
Lipschitz constant L
[Figure]
Yes
No
Abstain from classiﬁcation
Certify all the fact that classiﬁers are restricted to map an input to a ﬁxed number of class probabilities.
Thus,directly applying randomized smoothing to classiﬁers in metric space, such as in few-shot learning,is a challenging task.
There are several works that aim at improving the robustness for few-shot classiﬁcation [20, 9, 51, 28].
However, the focus in such works is either on the improvement of empirical robustness or probabilisticguarantees of certiﬁed robustness; none of them provide theoretical guarantees on the worst-casemodel behavior.
In this work we ﬁll this gap, by generalizing and theoretically justifying the idea of randomizedsmoothing to few-shot learning.
In this scenario, provable certiﬁcation needs to be obtained not in thespace of output class probabilities, but in the space of descriptive embeddings.
This work is the ﬁrst,to our knowledge, where the theoretical robustness guarantees for few-shot scenario is provided.
Our contributions are summarized as follows:
• We provide the ﬁrst theoretical robustness guarantee for few-shot learning classiﬁcation.
• Analysis of Lipschitz continuity of such models and providing the robustness certiﬁcatesagainst ` 2  bounded perturbations for few-shot learning scenarios.
• We propose to estimate conﬁdence intervals not for distances between the approximationof smoothed embedding and class prototype but for the dot product of vectors which hasexpectation equal to the distance between actual smoothed embedding and class prototype.
Problem statement
2.1 Notation
We consider a few-shot classiﬁcation problem where we are given a set of labeled objects( x 1 , y 1 ) , . . .
, ( x m , y m ) where x i 2 R D and y i 2 { 1 , . .
, K } are corresponding labels.
We follow the notation from [38] and denote S k as the set of objects of class k.
2.2 Few-shot learning classiﬁcation
Suppose we have a function f : R D !
R d that maps input objects to the space of normalizedembeddings.
Then, d  dimensional prototypes of classes are computed as follows (expression isgiven for the prototype of class k ):
[Equation]
In order to classify a sample, one should compute the distances between its embedding and classprototypes – a sample is assigned to the class with the closest prototype.
Namely, given a distancefunction ⇢ : R d ⇥ R d ! [0 , + 1 ) , the class c of the sample x is computed as below:
[Equation]
Given an embedding function f , our goal is to construct a classiﬁer g provably robust to additiveperturbations  of a small norm.
In other words, we want to ﬁnd a norm threshold t such that equality
[Equation]
will be satisﬁed for all  : k  k 2  t.
In this paper, the solution of a problem of constructing a classiﬁer that satisﬁes the condition in (4) isapproached by extending the analysis of the robustness of smoothed classiﬁers described in (1) tothe case of vector functions.
The choice of the distance metric in (4) is motivated by an analysis of
Lipschitz-continuity given in the next section.
Randomized smoothing3.1 Background
Randomized smoothing [21, 3] is described as a technique of convolving a base classiﬁer f with anisotropic Gaussian noise such that the new classiﬁer g ( x ) returns the most probable prediction of fof a random variable ⇠ ⇠ N ( x,  2 I ) , where the choice of Gaussian distribution is motivated by therestriction on g to be robust against additive perturbations of bounded norm.
In this case, given aclassiﬁer f : R D !
[0 , 1] and smoothing distribution N (0 ,  2 I ) , the classiﬁer g looks as follows:
[Equation]
One can show by Stein’s Lemma that given the fact that the function f in (5) is bounded (namely,8 x 2 D ( f ) , | f ( x ) |  1 ), then the function g is L  Lipschitz:
[Equation]
with L = q 2 ⇡ 2 , what immediately produce theoretical robustness guarantee on g.
Although this approach is simple and effective, it has a serious drawback: in practice, it is impossibleto compute the expectation in (5)
exactly and, thus, impossible to compute the prediction of thesmoothed function g at any point.
Instead the integral is computed with the use of Monte-Carloapproximation with n samples to obtain the prediction with an arbitrary level of conﬁdence.
Notably,to achieve an appropriate accuracy of the Monte-Carlo approximation, the number of samples nshould be large enough that may dramatically affect inference time.
In this work, we generalize the analysis of Lipschitz-continuity to the case of vector functionsg : R D !
R d and provide robustness guarantees for classiﬁcation performed in the space ofembeddings.
The certiﬁcation pipeline is illustrated in the Figure 1.
3.2 Randomized smoothing for vector functions
Lipschitz-continuity of vector function.
In the work of [37], the robustness guarantee from [3] isproved by estimating the Lipschitz constant of a smoothed classiﬁer.
Unfortunately, a straightforwardgeneralization of this approach to the case of vector functions leads to the estimation of the expectationof the norm of a multivariate Gaussian which is known to depend on the number of dimensions ofthe space.
Instead, we show that a simple adjustment to this technique may be done such that theestimate of the Lipschitz constant is the same as for the function in (5).
Our results are formulated inthe theorems below proofs of which are moved to the Appendix in order not to distract the reader.
Theorem 1.
(Lipschitz-continuity of smoothed vector function) Suppose that f : R D !
R d is adeterministic function and g ( x ) =
E " ⇠ N (0 ,  2 I )
f
( x + " ) is continuously differentiable for all x .
Iffor all x , k f ( x ) k 2 = 1 , then g ( x ) is L  Lipschitz in l 2  norm with L = q 2 ⇡ 2 .
Remark 1.
We perform the analysis of Lipschitz-continuity in the Theorem 1 in l 2  norm, so thedistance metric in (4) is l 2  distance.
We do not consider other norms in this paper.
Robust classiﬁcation in the space of embeddings.
To provide certiﬁcation for a classiﬁcation in the space of embeddings, one should estimate the maximum deviation of the classiﬁed embedding that does not change the closest class prototype.
In the Theorem 2, we showhow this deviation is connected with the mutual arrangement of embedding and class prototype.
[Figure]
[Equation]
[Figure] [Figure]
[Figure]
(Adversarial embedding risk)
[Figure]
Given an input image x 2 R D and the embed-
[Figure]
ding g : R D !
R d the closest point on to decision boundary in the embedding space (see
Figure 2) is located at a distance (deﬁned asadversarial embedding risk):
[Equation]
(7) where c 1 2 R d and c 2 2 R d are the two closestprototypes.
The value of  is the distance between classifying embedding and the decisionboundary between classes represented by c 1 andc 2 .
Note that this is the minimum l 2  distortionin the embedding space required to change theprediction of g.
Two theorems combined result in an l 2  robustness guarantee for few-shot classiﬁcation as:
Theorem 3.
(Robustness guarantee)
L 2 -robustness guarantee r for an input image x
in then  dimensional input metric space under classiﬁcation by a classiﬁer g from the Theorem 1 isr =
L , where L is the Lipschitz constant from the Theorem 1 and  is the adversarial risk from the
Theorem 2.
The value of r is the certiﬁed radius of g at x , or, in other words, minimum l 2  distortionin the input space required to change the prediction of g. The proof of this fact straightforwardlyfollows from the deﬁnition (6) and results from Theorems 1-2.
Certiﬁcation protocol
In this section, we describe the numerical implementation of our approach and estimate its failureprobability.
4.1 Estimation of prediction of smoothed classiﬁer
As mentioned previously, the procedure of few-shot classiﬁcation is performed by assigning an objectx to the closest class prototype.
Unfortunately, given the smoothed function g in the form from the
Theorem 1 and class prototype c k from (2), it is impossible to compute the value ⇢ ( g ( x ) , c k ) explicitlyas well as to determine the closest prototype, since it is in general unknown how does g ( x ) look like.
We propose both to estimate the closest prototype for classiﬁcation and to estimate the distance tothe closest decision boundary  from Theorem 2 as the largest class-preserving perturbation in theembedding space by computing two-sided conﬁdence intervals for random variables
[Equation]
(8) where ˆ g ( x ) = 1 n
P ni =1 f ( x + " i ) is the estimation of g ( x ) computed as empirical mean by n samplesof noise, and one-sided conﬁdence interval for  from the Theorem 2, respectively.
Pseudo-code forboth procedures is presented in Algorithms 1-2.
Algorithm 1 Closest prototype computation algorithm.
GIVEN: base classiﬁer f , noise  , object x , number of samples n for ˆ g ( x ) , class prototypes { c i } Ki =1 , maximum number of samples T and conﬁdence level ↵ ,
RETURNS: index A of the closest prototype.
Function CLOSEST( f,  , x, n, { c i } Ki =1 , T, ↵ ) fs ; i n 0 = n while n  T do " 1 , . .
, " n ⇠ N (0 ,  2 I )
ˆ g ( x ) 1 n
P ni =1 f ( x + " i ) fs fs
[ { ˆ g ( x ) } for all i 2 [1 , . . .
, K ] do
C = c
i dsToC ⇢ ( fs, C ) ( l C , u C )
TwoSidedConfInt ( dsToC, ↵ ) l i l C , u i u C end for
A arg min { l i } Ki = if u A < min( { l i } Ki =1 ,i 6 = A ) then Return A, fs else n n + n 0 {Increase number of samples used for computing an approximation ˆ g ( x ) until the number of observations is large enough to determine two leftmost intervals or until n = T } end if end while
EndFunction
The Algorithm 1 describes an inference procedure for the smoothed classiﬁer from the Theorem 1;the Algorithm 2 uses Algorithm 1 and, given input parameters, estimates an adversarial risk from the
Theorem 2 – it determines two closest to smoothed embedding g ( x ) prototypes A and B and producesthe lower conﬁdence bound for the distance between g ( x ) and the decision boundary between A and
B. Combined with analysis from the Theorem 1, it provides the certiﬁed radius for a sample – thesmallest value of l 2  norm of perturbation in the input space required to change the prediction of thesmoothed classiﬁer.
In the next subsection, we discuss in detail the procedure of computing conﬁdence intervals in
Algorithms 1-2.
4.2 Applicability of algorithms
The computations of smoothed function and distances to class prototypes and decision boundary in
Algorithms 1-2 are based on the estimations of corresponding random variables, thus, it is necessaryto analyze applicability of the algorithms.
In this section, we propose a way to compute conﬁdenceintervals for squares of the distances between the estimates of embeddings and class prototypes.
Computation of conﬁdence intervals for the squares of distances.
Recall that one way to estimatethe value of a parameter of a random variable is to compute a conﬁdence interval for the correspondingstatistic.
In this work, we construct intervals by applying well-known Hoeffding inequality [13] inthe form
[Equation]
where X and E ( X ) are sample mean and population mean of random variable X , respectively, n isthe number of samples and numbers a i ,
b i are such that P ( X i 2 ( a i , b i ))
= 1 .
Algorithm
GIVEN: base classiﬁer f , noise  , object x , number of samples n for ˆ g ( x ) , class prototypes { c i } Ki =1 , maximum number of samples T for ˆ g ( x ) and conﬁdence level ↵
RETURNS: lower bound  for the adversarial risk  .
Function EMBEDDING-RISK( f,  , x, n, { c i } Ki =1 , T, ↵ )
A, fs A
CLOSEST( f,  , x, n, { c i } Ki =1 , T, ↵ )
B, fs B
CLOSEST( f,  , x, n, { c i } Ki
[Equation]
[Equation] [Equation]
, T, ↵ ) fs
[Equation]
[Equation]
; for all f 2 fs do
[Equation]
s [ {  } end for
LowerConfBound (  s, ↵ )
Return
EndFunction
However, a conﬁdence interval for the distance
k ˆ g
( x )  c k k 2 with a certain conﬁdence covers 162 an expectation of distance
E ( k ˆ g ( x )  c k k 2 ) , not the distance for expectation k E
(ˆ g ( x )  c k ) k 2 =k g ( x )  c k k 2 .
To solve this problem, we propose to compute conﬁdence intervals for the dot product of vectors.
Namely, given a quantity ⇠ x,k = h g ( x )  c k , g ( x )  c k
i , we sample its unbiased estimate with 166 at maximum 2 n samples of noise (here we have to mention that the number of samples n from
Algorithm 1 actually doubles since we need a pair of estimates ˆ g ( x ) of smoothed embeddings):
[Equation]
and compute conﬁdence interval ( l x,k , u x,k ) such that given
[Equation]
the population mean E (ˆ ⇠ x,k ) is most probably located within it, or, equivalently,
P ⇣ l x,k  E (ˆ ⇠ x,k )  u x,k ⌘  1  ↵ .
We have to mention that there are three conﬁdence intervals for three terms (one with quadratic number of samples and two with the linear numbers ofsamples) in the expression (10), that is why there is fraction 13 in the equation (11).
Also note that the population mean E (ˆ ⇠ x,k ) is exactly k g ( x )  c k k 22 , since
[Equation]
since f ( x + " i ) and f ( x + " j ) are independent random variables for i
= j .
Finally, note that theconﬁdence interval ( l x,k , u x,k ) for the quantity k g ( x )  c k k 22 implies conﬁdence interval
[Equation]
[Equation]
for the quantity k g ( x )  c k k 2 .
Thus, the procedures TwoSidedConfInt and LowerConfBound from 177 algorithms return an interval from (13) and its left bound for the random variable representingcorresponding distance, respectively.
[Figure] [Figure]
Experiments
5.1 Datasets
For the experimental evaluation of our approach we use several well-known datasets for few-shotlearning classiﬁcation.
Cub-200-2011 [43] is a dataset with 11 , 788 images of 200 bird species, where5864 images of 100 species are in the train subset and 5924 images of other 100 species are in the testsubset.
It is notable that a lot of species presented in dataset have degree of visual similarity, makingclassiﬁcation of ones a challenging task even for humans.
mini ImageNet [42] is a substet of imagesfrom ILSVRC 2015 [36] dataset with 64 images categories in train subset, 16 categories in validationsubset and 20 categories in test subset with 600 images of size 84 ⇥ 84 in each category.
CIFAR FS[1] is a subset of CIFAR 100 [19] dataset which was generated in the same way as mini ImageNet andcontains 37800 images of 64 categories in the train set and 11400 images of 20 categories in the testset.
Experimental setup for all the datasets is presented in the next section.
5.2 Experimental settings and computation cost
Following [3], we compute approximate certiﬁed test set accuracy to estimate the performance of thesmoothed model prediction with the Algorithm 1 and embedding risk computation with the Algorithm2.
The baseline model we used for experiments is a prototypical network introduced in [38] with
ConvNet-4 backbone.
Compared to the original architecture, an additional fully-connected layer wasadded in the tail of the network to map embeddings to 512-dimensional vector space.
The model wastrained to solve 1-shot and 5-shot classiﬁcation tasks on each dataset, with 5-way classiﬁcation oneach iteration.
Parameters of expeiments.
For data augmentation, we applied Gaussian noise with zero mean,unit variance and probability 0 .
3 of augmentation.
Each dataset was certiﬁed on a subsample of 500images with default parameters for the Algorithm 1: number of samples n = 1000 , conﬁdence level↵ = 0 .
001 and variance  = 1 . 0 , unless stated otherwise.
For our settings, it may be shown from simple geometry that values ( a i , b i ) from (9) are such that b i  a i 
so we use b i  a i = 4 .
Themaximum number of samples T in the Algorithm 1 is set to be T = 5 ⇥ 10 5 .
Computation cost.
In the table below, we report the computation time of the certiﬁcation procedureper image on Tesla V100 GPU for Cub-200-2011 dataset.
Standard deviation in seconds appearsto be signiﬁcant because the number of main loop iterations required to separate the two leftmostconﬁdence intervals varies from image to image in the test set.
[Table]
5.3 Results of experiments
In this section, we report results of our experiments.
In our evaluation protocol, we computeapproximate certiﬁed test set accuracy, CA .
Given a sample x , a smoothed classiﬁer g ( · ) from the
Algorithm 1 with an assigned classiﬁcation rule h ( x ) =
arg min i 2 { 1 ,...,K } k g ( x )  c
k k 2 , thresholdvalue " for l 2  norm of additive perturbation and the robustness guarantee r = r ( x ) from the
Algorithm 3, we compute CA on test set S as follows:
[Equation]
In other words, we treat the model g ( · ) as certiﬁed at point x under perturbation of norm " if x iscorrectly classiﬁed by g ( · ) (what means that the procedure of classiﬁcation described in the Algorithm1 does not abstain from classiﬁcation of x ) and g ( · ) has the value of certiﬁed radius r ( x ) > " .
Visualization of results.
The ﬁgures 3-4 from below represent dependencies of certiﬁed accuracyon the value of norm of additive perturbation (attack radius) for different learning settings (1-shotand 5-shot learning).
The value of attack radius corresponds to the threshold " from (14).
For
Cub-200-2011 dataset we provide a dependency of certiﬁed accuracy for different number of samplesn for the Algorithm 1 (in the Figure 5).
Limitations
In this section, we describe limitations of our approach.
Namely, we provide failure probability of
Algorithms 1-2 and discuss abstains from classiﬁcation in the Algorithm 1.
6.1 Estimation of errors of algorithms
Note that the value of ↵ from (11) is the probability of the value of ⇢ x,k = k g ( x )  c k k 2 not tobelong to the corresponding interval of the form from (13).
Given a sample x , the procedure in the Algorithm 1 returns two closest prototypes to the g ( x ) .
Notethat to determine two leftmost conﬁdence intervals, all the distances ⇢ x,k have to be located withincorresponding intervals, thus, according to the independence of computing these two intervals, theerror probability for the Algorithm 1 is q 1 = K ↵ , where K is the number of classes.
Similarly, the procedure in the Algorithm 2 outputs the lower bound for the adversarial risk withcoverage at least 1  ↵ and depend on the output of the Algorithm 1 inside, and, thus, has errorprobability q 2 = 1  (1  ↵ )(1  K ↵ ) = ↵ + K ↵ + K ↵ 2 that corresponds to returning anoverestimated lower bound for the adversarial risk from the Theorem 2.
6.2 Abstains from classiﬁcation
It is crucial to note that the procedure in the Algorithm 1 may require a lot of samples to distinguishtwo leftmost conﬁdence intervals and sometimes does not ﬁnish before reaching threshold value T forsample size.
As a result, there may be input objects at which the smoothed classiﬁer can be neither evaluated nor certiﬁed.
In this subsection, we report the numbers of objects in which the Algorithm 1abstains from determining the closest class prototype.
The fractions of abstained samples for differentvalues of conﬁdence level ↵ for both 1-shot and 5-shot scenarios are reported in the Tables 2-3 below.
n of the Algorithm 1, CIFAR-FS dataset, 5shot case.
Is is notable that relatively small number of samples may be used to achieve satisfactory level of certiﬁed accuracy.
[Table] [Table]
Related work
Breaking neural networks with adversarial attacks and empirical defending from them have a longhistory of cat-and-mouse game.
Namely, for a particular proposed defense against existing adversarialperturbations, a new more aggressive attack is found.
This motivated researchers to ﬁnd defenses thatare mathematically provable and certiﬁably robust to different kinds of input manipulations.
Severalworks proposed exactly veriﬁed neural networks based on Satisﬁability Modulo Theories solvers[16, 6], or mixed-integer linear programming [30, 8].
These methods are found to be computationallyinefﬁcient, although they guarantee to ﬁnd adversarial examples, in the case if they exist.
Anotherline of works use more relaxed certiﬁcation [45, 11, 35].
Although these methods aim to guaranteethat an adversary does not exist in a certain region around a given input, they suffer from scalabilityto big networks and large datasets.
The only scalable to large datasets provable defense againstadversarial perturbations is randomized smoothing .
Initially, it was found as an empirical defense tomitigate adversarial effects in neural networks [29, 47].
Later several works showed its mathematicalproof of certiﬁed robustness [21, 25, 3, 37].
Lecuyer et al [21] ﬁrst provided proof of certiﬁcatesagainst adversarial examples using differential privacy.
Then Li et al [25] proposed tighter boundsof guarantees using Renyi divergence.
Later, Cohen et al [3] provided the tightest bound using
Neyman-Pearson lemma.
Interestingly, alternative proof using Lipschitz continuity was found [37].
The scalability and simplicity of randomized smoothing attracted signiﬁcant attention, and it wasextended beyond l 2  perturbations [22, 41, 27, 24, 23, 20, 31, 49].
Conclusion and future work
In this work, we extended randomized smoothing as a defense against additive norm-boundedadversarial attacks to the case of classiﬁcation in the embedding space that is used in few-shotlearning scenarios.
We performed an analysis of Lipschitz continuity of smoothed normalizedembeddings and derived a robustness certiﬁcate against l 2  attacks.
Our theoretical ﬁndings aresupported experimentally on several datasets.
There are several directions for future work: ourapproach can possibly be extended to other types of attacks, such semantic transformations; also, it isimportant to reduce the computational complexity of the certiﬁcation procedure.