Smoothed Embeddings for Certified Few-Shot Learning
Abstract
Randomized smoothing is considered to be the state-of-the-art provable defenseagainst adversarial perturbations.
However, it heavily exploits the fact that classifiers map input objects to class probabilities and do not focus on the ones thatlearn a metric space in which classification is performed by computing distances toembeddings of classes prototypes.
In this work, we extend randomized smoothingto few-shot learning models that map inputs to normalized embeddings.
We provideanalysis of Lipschitz continuity of such models and derive robustness certificateagainst ℓ 2 -bounded perturbations that may be useful in few-shot learning scenarios.
Our theoretical results are confirmed by experiments on different datasets.
Introduction
In the regime of scarce data or when new classes emerge constantly, such as in face recognition,few-shot learning is required.
Modern computer vision methods of learning from a few images arebased on deep neural networks.
These neural networks are intriguingly vulnerable to adversarialperturbations [40, 10] – accurately crafted small modifications of the input that may significantlyalter the model’s prediction.
For safety-critical scenarios, these perturbations present a serious threat. Hence, it is important to investigate ways to protect neural networks from undesired scenarios.
Several works studied this phenomena in different applications of neural networks – image classification [2, 33, 32, 39], object detection [17, 26, 46, 48], face recognition [18, 5, 52], semantic segmentation [7, 12, 48].
This shows how easy for adversaries to maliciously force a model to behave in the desired way.
As a result, defenses, both empirical [4, 53, 14] and provable [49, 21, 3, 45, 50, 15, 44, 34],were proposed recently.
Although empirical ones can be (and often are) broken by more powerfulattacks, the provable ones are of a big interest since they make it possible to provide guarantees ofthe correctness of the work of a model under certain assumptions, and, thus, possibly broaden thescope of tasks which may be trusted to the neural networks.
Randomized smoothing [21, 3, 25] is the state-of-the-art approach used for constructing classificationmodels provably robust against small-norm additive adversarial perturbations.
This approach isscalable to large datasets and can be applied to any classifier since it does not use any assumptionsabout model’s architecture.
Generally, the idea is following.
Suppose, we are given a base neuralnetwork classier f : R n → [0 , 1] K that maps an input image x to a fixed number of K classprobabilities.
Its smoothed version with the standard Gaussian distribution is:
[Equation]
Interestingly, as shown in [3], the new (smoothed) classifier is provably robust at x to ℓ 2 -boundedperturbations if the base classifier f is confident enough at x .
However, the proof of certification
[Figure]
Smoothed embeddings
[Figure]
Lipschitz constant
[Figure]
Certify
The model g ( · ) is treated as certified at x for all additive perturbations heavily exploits the fact that classifiers are restricted to map an input to a fixed number of classprobabilities.
Thus, directly applying randomized smoothing to classifiers in metric space, such as infew-shot learning, is a challenging task.
There are several works that aim at improving the robustness for few-shot classification [20, 9, 51, 28].
However, the focus in such works is either on the improvement of empirical robustness or probabilisticguarantees of certified robustness; none of them provide theoretical guarantees on the worst-casemodel behavior.
In this work, we fill this gap, by generalizing and theoretically justifying the idea of randomizedsmoothing to few-shot learning.
In this scenario, provable certification needs to be obtained not in thespace of output class probabilities, but in the space of descriptive embeddings.
This work is the first,to our knowledge, where the theoretical robustness guarantees for few-shot scenario is provided.
Our contributions are summarized as follows:
• We provide the first theoretical robustness guarantee for few-shot learning classification.• Analysis of Lipschitz continuity of such models and providing the robustness certificates against ℓ 2 − bounded perturbations for few-shot learning scenarios.•
We propose to estimate confidence intervals not for distances between the approximationof smoothed embedding and class prototype but for the dot product of vectors which hasexpectation equal to the distance between actual smoothed embedding and class prototype.
Problem statement2.1 Notation
We consider a few-shot classification problem where we are given a set of labeled objects( x 1 , y 1 ) , . . .
, ( x m , y m ) where x i ∈ R n and y i ∈ { 1 , . .
, K } are corresponding labels.
We follow the notation from [38] and denote S k as the set of objects of class k.
2.2 Few-shot learning classification
Suppose we have a function f : R n → R d that maps input objects to the space of normalizedembeddings.
Then, d − dimensional prototypes of classes are computed as follows (expression isgiven for the prototype of class k ):
[Equation]
In order to classify a sample, one should compute the distances between its embedding and classprototypes – a sample is assigned to the class with the closest prototype.
Namely, given a distancefunction ρ : R d × R d → [0 , + ∞ ) , the class c of the sample x is computed as below:
[Equation]
Given an embedding function f , our goal is to construct a classifier g provably robust to additiveperturbations ∆ of a small norm.
In other words, we want to find a norm threshold t such that equality
[Equation]
will be satisfied for all δ : ∥ δ ∥ 2 ≤ t.
In this paper, the solution of a problem of constructing a classifier that satisfies the condition in (4) isapproached by extending the analysis of the robustness of smoothed classifiers described in (1) tothe case of vector functions.
The choice of the distance metric in (4) is motivated by an analysis of
Lipschitz-continuity given in the next section.
Randomized smoothing3.1 Background
Randomized smoothing [21, 3] is described as a technique of convolving a base classifier f with anisotropic Gaussian noise such that the new classifier g ( x ) returns the most probable prediction of fof a random variable ξ ∼ N ( x, σ 2 I ) , where the choice of Gaussian distribution is motivated by therestriction on g to be robust against additive perturbations of bounded norm.
In this case, given aclassifier f : R n → [0 , 1] and smoothing distribution N (0 , σ 2 I ) , the classifier g looks as follows:
[Equation]
One can show by Stein’s Lemma that given the fact that the function f in (5) is bounded (namely,∀ x ∈ D ( f ) , | f ( x ) | ≤ 1 ), then the function g is L − Lipschitz:
[Equation]
with L = (cid:113) 2 πσ 2 , what immediately produce theoretical robustness guarantee on g.
Although this approach is simple and effective, it has a serious drawback: in practice, it is impossibleto compute the expectation in (5)
exactly and, thus, impossible to compute the prediction of thesmoothed function g at any point.
Instead the integral is computed with the use of Monte-Carloapproximation with n samples to obtain the prediction with an arbitrary level of confidence.
Notably,to achieve an appropriate accuracy of the Monte-Carlo approximation, the number of samples nshould be large enough that may dramatically affect inference time.
In this work, we generalize the analysis of Lipschitz-continuity to the case of vector functionsg : R n → R d and provide robustness guarantees for classification performed in the space ofembeddings.
The certification pipeline is illustrated in the Figure 1.
3.2 Randomized smoothing for vector functions
Lipschitz-continuity of vector function.
In the work of [37], the robustness guarantee from [3] isproved by estimating the Lipschitz constant of a smoothed classifier.
Unfortunately, a straightforwardgeneralization of this approach to the case of vector functions leads to the estimation of the expectationof the norm of a multivariate Gaussian which is known to depend on the number of dimensions ofthe space.
Instead, we show that a simple adjustment to this technique may be done such that theestimate of the Lipschitz constant is the same as for the function in (5).
Our results are formulated inthe theorems below proofs of which are moved to the Appendix in order not to distract the reader.
Theorem 1 (Lipschitz-continuity of smoothed vector function) Suppose that f : R n → R d is adeterministic function and g ( x ) =
E ε ∼N (0 ,σ 2 I ) f
( x + ε ) is continuously differentiable for all x .
Iffor all x , ∥ f ( x ) ∥ 2 = 1 , then g ( x ) is L − Lipschitz in l 2 − norm with L = (cid:113) 2 πσ 2 .
Remark {+
+}
We perform the analysis of Lipschitz-continuity in the Theorem 1 in l 2 − norm, so thedistance metric in (4) is l 2 − distance.
We do not consider other norms in this paper.
Robust classification in the space of embeddings.
To provide certification for a classification in the space of embeddings, one should estimate the maximum deviation of the classified embedding that does not change the closest class prototype.
In the Theorem 2, we showhow this deviation is connected with the mutual arrangement of embedding and class prototype.
[Figure]
Adversarial embedding
Given an input image x ∈ R n and the embedding g : R n → R d the closest point on todecision boundary in the embedding space (see
Figure 2) is located at a distance (defined asadversarial embedding risk):
[Equation]
where c 1 ∈ R d and c 2 ∈ R d are the two closestprototypes.
The value of γ is the distance between classifying embedding and the decisionboundary between classes represented by c 1 andc 2 .
Note that this is the minimum l 2 − distortionin the embedding space required to change theprediction of g.
Two theorems combined result in an l 2 − robustness guarantee for few-shot classification as:
Theorem 3 (Robustness guarantee)
L 2 -robustness guarantee r for an input image x in then − dimensional input metric space under classification by a classifier g from the Theorem 1 isr = γL , where L is the Lipschitz constant from the Theorem 1 and γ is the adversarial risk from the
Theorem 2.
The value of r is the certified radius of g at x , or, in other words, minimum l 2 − distortionin the input space required to change the prediction of g. The proof of this fact straightforwardlyfollows from the definition (6) and results from Theorems 1-2.
Certification protocol
Here, we describe the numerical implementation of our approach and estimate its failure probability.
4.1 Estimation of prediction of smoothed classifier
As mentioned previously, the procedure of few-shot classification is performed by assigning an objectx to the closest class prototype.
Unfortunately, given the smoothed function g in the form from the
Theorem 1 and class prototype c k from (2), it is impossible to compute the value ρ ( g ( x ) , c k ) explicitlyas well as to determine the closest prototype, since it is in general unknown how does g ( x ) look like.
We propose both to estimate the closest prototype for classification and to estimate the distance tothe closest decision boundary γ from Theorem 2 as the largest class-preserving perturbation in theembedding space by computing two-sided confidence intervals for random variables
[Equation]
(8) where ˆ g ( x ) = 1 n
[Equation]
ni =1 f ( x + ε i ) is the estimation of g ( x ) computed as empirical mean by n samplesof noise, and one-sided confidence interval for γ from the Theorem 2, respectively.
Pseudo-code forboth procedures is presented in Algorithms 1-2.
Algorithm
GIVEN: base classifier f , noise σ , object x , number of samples n for ˆ g ( x ) , class prototypes { c i } Ki =1 , maximum number of samples T and confidence level α ,
RETURNS: index A of the closest prototype.
Function CLOSEST( f, σ, x, n, { c i } Ki =1 , T, α ) fs ← ∅ i ←n 0 = n while n ≤ T do ε 1 , . . . , ε n ∼ N (0 , σ 2 I )
ˆ g ( x ) ← 1 n (cid:80) ni
=1 f ( x + ε i ) fs ← fs ∪ ˆ g ( x )
CI ← ∅ for all C ∈ { c i } Ki =1 do dsToC ← ρ ( fs, C ) ( α C , β C ) ← TwoSidedConfInt ( dsToC, α )
CI ← CI ∪ ( α C , β C ) end for
A ← arg min { α i } Ki = if β A < min( { α i } Ki =1 ,i ̸ = A ) then Return A, fs else n ← n + n 0 {Increase number of samples used for computing an approximation ˆ g ( x ) until the number of observations is large enough to determine two leftmost intervals or until n = T } end if end while
EndFunction
The Algorithm 1 describes an inference procedure for the smoothed classifier from the Theorem 1;the Algorithm 2 uses Algorithm 1 and, given input parameters, estimates an adversarial risk from the
Theorem 2 – it determines two closest to smoothed embedding g ( x ) prototypes A and B and producesthe lower confidence bound for the distance between g ( x ) and the decision boundary between A and B. Combined with analysis from the Theorem 1, it provides the certified radius for a sample – thesmallest value of l 2 − norm of perturbation in the input space required to change the prediction of thesmoothed classifier.
In the next subsection, we discuss in detail the procedure of computing confidence intervals in Algorithms 1-2.
4.2 Applicability of algorithms
The computations of smoothed function and distances to class prototypes and decision boundary in
Algorithms 1-2, respectively, are numerical and operate with estimations of a random variable, thus,it is necessary to analyze their applicability.
In this section, we propose a way to compute confidenceintervals for squares of the distances between estimates of embeddings in the form from of samplemean and class prototypes.
Computation of confidence intervals for the squares of distances.
Recall that one way to estimatethe value of a parameter of a random variable is to compute a confidence interval for the correspondingstatistic.
In this work, we construct intervals by applying well-known Hoeffding inequality [13] inthe form
[Equation]
where X and E ( X ) are sample mean and population mean of random variable X , respectively, n isthe number of samples and numbers a i ,
b i are such that P ( X i ∈ ( a i , b i ))
= 1 .
Algorithm
GIVEN: base classifier f , noise σ , object x , number of samples n for ˆ g ( x ) , class prototypes { c i } Ki =1 , maximum number of samples T for ˆ g ( x ) and confidence level α
RETURNS: lower bound Γ for the adversarial risk γ .
Function EMBEDDING-RISK( f, σ, x, n, { c i } Ki =1 , T, α )
A, fs A ← CLOSEST( f, σ, x, n, { c i } Ki =1 , T, α )
B, fs B ← CLOSEST( f, σ, x, n, { c i } Ki =1 ,i ̸
=
A , T, α ) fs ← fs
[Equation]
fs
[Equation]
γs ← ∅ for all f ∈ fs do
[Equation]
γs ← γs ∪ γ end for Γ ← LowerConfBound ( γs, α )
Return Γ
EndFunction
However, a confidence interval for the distance ∥
ˆ g ( x ) − c k ∥ 2 with a certain confidence coversan expectation of distance E ( ∥ ˆ g ( x ) − c k ∥ 2 ) , not the distance for expectation ∥
E (ˆ g ( x ) − c k ) ∥ 2 =∥ g ( x ) − c k ∥ 2 .
To solve this problem, we propose to compute confidence intervals for the dot product of vectors.
Namely, given a quantity ξ x,k = ⟨ g 1 ( x ) − c k , g 2 ( x ) − c k ⟩ , we sample its unbiased estimate withat maximum 2 n samples of noise (here we have to mention that the number of samples n from
Algorithm 1 actually doubles since we need a pair of estimates ˆ g ( x ) of smoothed embeddings):
[Equation]
and compute confidence interval ( α x,k , β x,k ) such that given
[Equation]
the population mean E (ˆ ξ x,k ) is most probably located within it, or, equivalently,
P (cid:16) α x,k ≤ E (ˆ ξ x,k ) ≤ β x,k (cid:17) ≥ 1 − α.
We have to mention that there are three confidence intervals for three terms (one with quadratic number of samples and two with the linear numbers ofsamples) in the expression (10), that is why there is fraction 1 3 in the equation (11).
Also note that thepopulation mean E (ˆ ξ x,k ) is exactly ∥ g ( x ) − c k ∥ 22 , since
[Equation]
since f ( x + ε i ) and f ( x + ε j ) are independent random variables for i [-
-]̸
= j .
Finally, note that theconfidence interval ( α x,k , β x,k ) for the quantity ∥ g ( x ) − c k ∥ 22 implies confidence interval
[Equation]
[Equation]
for the quantity ∥ g ( x ) − c k ∥ 2 .
Thus, the procedures TwoSidedConfInt and LowerConfBound fromalgorithms return an interval from (13) and its left bound for the random variable representingcorresponding distance, respectively.
[Figure] [Figure]
Experiments
5.1 Datasets
For the experimental evaluation of our approach we use several well-known datasets for few-shotlearning classification.
Cub-200-2011 [43] is a dataset with 11 , 788 images of 200 bird species, where5864 images of 100 species are in the train subset and 5924 images of other 100 species are in thetest subset.
It is notable that a lot of species presented in dataset have degree of visual similarity,making classification of ones a challenging task even for humans.
mini ImageNet [42] is a substet ofimages from ILSVRC 2015 [36] dataset with 80 images categories in train subset and 20 categories intest subset with 600 images of size 84 × 84 in each category.
CIFAR FS [1] is a subset of CIFAR 100[19] dataset which was generated in the same way as mini ImageNet and contains 37800 images of 64categories in the train set and 11400 images of 20 categories in the test set.
Experimental setup forall the datasets is presented in the next section.
5.2 Experimental settings and computation cost
Following [3], we compute approximate certified test set accuracy to estimate the performance of thesmoothed model prediction with the Algorithm 1 and embedding risk computation with the Algorithm2.
The baseline model we used for experiments is a prototypical network introduced in [38] with
ConvNet-4 backbone.
Compared to the original architecture, an additional fully-connected layer wasadded in the tail of the network to map embeddings to 512-dimensional vector space.
The model wastrained to solve 1-shot and 5-shot classification tasks on each dataset, with 5-way classification oneach iteration.
Parameters of expeiments.
For data augmentation, we applied Gaussian noise with zero mean,unit variance and probability 0 .
3 of augmentation.
Each dataset was certified on a subsample of 500images with default parameters for the Algorithm 1: number of samples n = 1000 , confidence levelα = 0 .
001 and variance σ = 1 . 0 , unless stated otherwise.
For our settings, it may be shown from simple geometry that values ( a i , b i ) from (9) are such that b i − a i ≤
so we use b i − a i = 4 .
Themaximum number of samples T in the Algorithm 1 is set to be T = 5 × 10 5 .
Computation cost.
In the table below, we report the computation time of the certification procedureper image on Tesla V100 GPU for Cub-200-2011 dataset.
Standard deviation in seconds appearsto be significant because the number of main loop iterations required to separate the two leftmostconfidence intervals varies from image to image in the test set.
[Table]
5.3 Results of experiments
In this section, we report results of our experiments.
In our evaluation protocol, we computeapproximate certified test set accuracy, CA .
Given a sample x , a smoothed classifier g ( · ) from the
Algorithm 1 with an assigned classification rule h ( x ) =
arg min i ∈{ 1 ,...,K } ∥ g ( x ) − c k ∥ 2 , thresholdvalue ε for l 2 − norm of additive perturbation and the robustness guarantee r = r ( x ) from the
Algorithm 3, we compute CA on test set S as follows:
[Equation]
In other words, we treat the model g ( · ) as certified at point x under perturbation of norm ε if x iscorrectly classified by g ( · ) (what means that the procedure of classification described in the Algorithm1 does not abstain from classification of x ) and g ( · ) has the value of certified radius r ( x ) > ε.
Visualization of results.
The figures 3-4 from below represent dependencies of certified accuracyon the value of norm of additive perturbation (attack radius) for different learning settings (1-shotand 5-shot learning).
The value of attack radius corresponds to the threshold ε from (14).
For Cub-200-2011 dataset we provide a dependency of certified accuracy for different number of samplesn for the Algorithm 1 (in the Figure 5).
Limitations
In this section, we describe limitations of our approach. Namely, we provide failure probability of Algorithms 1-2 and discuss abstains from classification in the Algorithm 1.
6.1 Estimation of errors of algorithms
Note that the value of α from (11) is the probability of the value of ρ x,k = ∥ g ( x ) − c k ∥ 2 not tobelong to the corresponding interval of the form from (13).
Given a sample x , the procedure in the Algorithm 1 returns two closest prototypes to the g ( x ) .
Notethat if two leftmost confidence intervals are determined, two closest to g ( x ) prototypes are establishedwith probability at least 1 − α for each one, thus, according to the independence of computing thesetwo intervals, the error probability for the Algorithm 1 is q 1 = 1 − (1 − α ) 2 .
This value correspondsto returning a pair of class prototypes at least one of which is not actually among the two closest tog ( x ) prototypes given the fact that the Algorithm 1 does not abstain.
Similarly, the procedure in the Algorithm 2 outputs the lower bound for the adversarial risk withcoverage at least 1 − α and depend on the output of the Algorithm 1 inside, and, thus, has errorprobability q 2 = 1 − (1 − α ) 3 that corresponds to returning an overestimated lower bound for theadversarial risk from the Theorem 2.
6.2 Abstains from classification
It is crucial to note that the procedure in the Algorithm 1 may require a lot of samples to distinguishtwo leftmost confidence intervals and sometimes does not finish before reaching threshold value T for sample size.
As a result, there may be input objects at which the smoothed classifier can be neitherevaluated nor certified.
In this subsection, we report the numbers of objects in which the Algorithm 1abstains from determining the closest class prototype.
The fractions of abstained samples for differentvalues of confidence level α for both 1-shot and 5-shot scenarios are reported in the Tables 2-3 below.
n of the Algorithm 1, CIFAR-FS dataset, 5shot case.
Is is notable that relatively small number of samples may be used to achieve satisfactory level of certified accuracy.
[Table] [Table]
Related work
Breaking neural networks with adversarial attacks and empirical defending from them have a longhistory of cat-and-mouse game.
Namely, for a particular proposed defense against existing adversarialperturbations, a new more aggressive attack is found.
This motivated researchers to find defenses thatare mathematically provable and certifiably robust to different kinds of input manipulations.
Severalworks proposed exactly verified neural networks based on Satisfiability Modulo Theories solvers[16, 6], or mixed-integer linear programming [30, 8].
These methods are found to be computationallyinefficient, although they guarantee to find adversarial examples, in the case if they exist.
Anotherline of works use more relaxed certification [45, 11, 35].
Although these methods aim to guaranteethat an adversary does not exist in a certain region around a given input, they suffer from scalabilityto big networks and large datasets.
The only scalable to large datasets provable defense againstadversarial perturbations is randomized smoothing .
Initially, it was found as an empirical defense tomitigate adversarial effects in neural networks [29, 47].
Later several works showed its mathematicalproof of certified robustness [21, 25, 3, 37].
Lecuyer et al [21] first provided proof of certificatesagainst adversarial examples using differential privacy.
Then Li et al [25] proposed tighter boundsof guarantees using Renyi divergence.
Later, Cohen et al [3] provided the tightest bound using Neyman-Pearson lemma.
Interestingly, alternative proof using Lipschitz continuity was found [37].
The scalability and simplicity of randomized smoothing attracted significant attention, and it wasextended beyond l 2 − perturbations [22, 41, 27, 24, 23, 20, 31, 49].
Conclusion and future work
In this work, we extended randomized smoothing as a defense against additive norm-boundedadversarial attacks to the case of classification in the embedding space that is used in few-shotlearning scenarios.
We performed an analysis of Lipschitz continuity of smoothed normalizedembeddings and derived a robustness certificate against l 2 − attacks.
Our theoretical findings aresupported experimentally on several datasets.
There are several directions for future work: ourapproach can possibly be extended to other types of attacks, such semantic transformations; also, it isimportant to reduce the computational complexity of the certification procedure.
Checklist
For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper’scontributions and scope?
[Yes]
(b) Did you describe the limitations of your work?
[Yes] See Section
(c) Did you discuss any potential negative societal impacts of your work?
[N/A](d)
Have you read the ethics review guidelines and ensured that your paper conforms tothem?
[Yes]2.
If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results?
[Yes](b) Did you include complete proofs of all theoretical results?
[Yes] The proofs areincluded in the supplementary material.3.
If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?
[Yes] The code with instructions is included in the supplementary material.
(b) Did you specify all the training details (e
g., data splits, hyperparameters, how theywere chosen)?
[Yes] Applicable training details are described in Section 5.2.
(c) Did you report error bars (e
g., with respect to the random seed after running experiments multiple times)?
[N/A](d)
Did you include the total amount of compute and the type of resources used (e.g., typeof GPUs, internal cluster, or cloud provider)?
[Yes] See Section 5.2.4.
If you are using existing assets (e g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators?
[Yes] See Sections 5.1 and5.
(b) Did you mention the license of the assets?
[No]
(c) Did you include any new assets either in the supplemental material or as a URL?
[No]
(d) Did you discuss whether and how consent was obtained from people whose data you’re
using/curating?
[No] The datasets and models are publicly available.
(e) Did you discuss whether the data you are using/curating contains personally identifiable
information or offensive content?
[N/A]
If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, if
applicable?
[N/A]
(b) Did you describe any potential participant risks, with links to Institutional Review
Board (IRB) approvals, if applicable?
[N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount
spent on participant compensation?
[N/A]