Smoothed Embeddings for Certiﬁed Few-Shot Learning
Abstract
Randomized smoothing is considered to be the state-of-the-art provable defense against adversarial perturbations.
However, it heavily exploits the fact that classiﬁers map input objects to class probabilities and do not focus on the ones that learn a metric space in which classiﬁcation is performed by computing distances to embeddings of classes prototypes.
In this work, we extend randomized smoothing to few-shot learning models that map inputs to normalized embeddings.
We provide analysis of Lipschitz continuity of such models and derive robustness certiﬁcate against ` 2 -bounded perturbations that may be useful in few-shot learning scenarios.
Our theoretical results are conﬁrmed by experiments on different datasets.
Introduction
Few-shot learning is a setting in which a classiﬁcation model is evaluated on the classes not seen during the training phase.
Nowadays quite a few few-shot learning approaches based on neural networks are known.
Unfortunately, neural network based classiﬁers are intimately vulnerable to adversarial perturbations [41, 10] – accurately crafted small modiﬁcations of the input that may signiﬁcantly alter the model’s prediction.
This vulnerability poses a restriction on the deployment of such approaches in safety-critical scenarios, so the research interest in the ﬁeld of attacks on neural networks has been great in recent years.
Several works studied this phenomena in different practical applications – image classiﬁcation [2, 34, 33, 40], object detection [17, 26, 48, 50], face recognition [18, 5, 54], semantic segmentation [7, 12, 50].
These studies show that it is easy to force a model to behave in the desired way by applying imperceptible change to its input.
As a result, defenses, both empirical [4, 55, 14] and provable [51, 21, 3, 47, 52, 15, 46, 35], were proposed recently.
Although empirical ones can be (and often are) broken by more powerful attacks, the provable ones are of a big interest since they make it possible to provide guarantees of the correctness of the work of a model under certain assumptions, and, thus, possibly broaden the scope of tasks which may be trusted to the neural networks.
Randomized smoothing [21, 3, 25] is the state-of-the-art approach used for constructing classiﬁcation models provably robust against small-norm additive adversarial perturbations.
This approach is scalable to large datasets and can be applied to any classiﬁer since it does not use any assumptions about model’s architecture.
Generally, the idea is following.
Suppose, we are given a base neural
[Table]
n
[Figure] [Figure]
Smoothed
[Figure]
constant
[Figure]
Certify network classier f : R D ! [0 , 1] K that maps an input image x to a ﬁxed number of K class probabilities. Its smoothed version with the standard Gaussian distribution is:
[Equation]
As shown in [3], the new (smoothed) classiﬁer is provably robust at x to ` 2 -bounded perturbations if the base classiﬁer f is conﬁdent enough at x .
However, the proof of certiﬁcation heavily exploits the fact that classiﬁers are restricted to map an input to a ﬁxed number of class probabilities.
Thus, directly applying randomized smoothing to classiﬁers in metric space, such as in few-shot learning, is a challenging task.
There are several works that aim at improving the robustness for few-shot classiﬁcation [20, 9, 53, 28].
However, the focus in such works is either on the improvement of empirical robustness or probabilistic guarantees of certiﬁed robustness; none of them provide theoretical guarantees on the worst-case model behavior.
In this work, we ﬁll this gap, by generalizing and theoretically justifying the idea of randomized smoothing to few-shot learning.
In this scenario, provable certiﬁcation needs to be obtained not in the space of output class probabilities, but in the space of descriptive embeddings.
This work is the ﬁrst, to our knowledge, where the theoretical robustness guarantees for few-shot scenario is provided.
Our contributions are summarized as follows:
• We provide the ﬁrst theoretical robustness guarantee for few-shot learning classiﬁcation.
• Analysis of Lipschitz continuity of such models and providing the robustness certiﬁcates against ` 2  bounded perturbations for few-shot learning scenarios.
• We propose to estimate conﬁdence intervals not for distances between the approximation of smoothed embedding and class prototype but for the dot product of vectors which has expectation equal to the distance between actual smoothed embedding and class prototype.
Problem statement 2.1 Notation
We consider a few-shot classiﬁcation problem where we are given a set of labeled objects ( x 1 , y 1 ) , . . .
, ( x m , y m ) where x i 2 R D and y i 2 { 1 , . .
, K } are corresponding labels.
We follow the notation from [39] and denote S k as the set of objects of class k.
2.2 Few-shot learning classiﬁcation
Suppose we have a function f : R D !
R d that maps input objects to the space of normalized embeddings.
Then, d  dimensional prototypes of classes are computed as follows (expression is given for the prototype of class k ):
[Equation]
In order to classify a sample, one should compute the distances between its embedding and class prototypes – a sample is assigned to the class with the closest prototype.
Namely, given a distance function ⇢ : R d ⇥ R d ! [0 , + 1 ) , the class c of the sample x is computed as below:
[Equation]
Given an embedding function f , our goal is to construct a classiﬁer g provably robust to additive perturbations  of a small norm.
In other words, we want to ﬁnd a norm threshold t such that equality
[Equation]
will be satisﬁed for all  : k  k 2  t.
In this paper, the solution of a problem of constructing a classiﬁer that satisﬁes the condition in (4) is approached by extending the analysis of the robustness of smoothed classiﬁers described in (1) to the case of vector functions.
The choice of the distance metric in (4) is motivated by an analysis of
Lipschitz-continuity given in the next section.
Randomized smoothing 3.1 Background
Randomized smoothing [21, 3] is described as a technique of convolving a base classiﬁer f with an isotropic Gaussian noise such that the new classiﬁer g ( x ) returns the most probable prediction of f of a random variable ⇠ ⇠ N ( x,  2 I ) , where the choice of Gaussian distribution is motivated by the restriction on g to be robust against additive perturbations of bounded norm.
In this case, given a classiﬁer f : R D !
[0 , 1] and smoothing distribution N (0 ,  2 I ) , the classiﬁer g looks as follows:
[Equation]
One can show by Stein’s Lemma that given the fact that the function f in (5) is bounded (namely, 8 x 2 D ( f ) , | f ( x ) |  1 ), then the function g is L  Lipschitz:
[Equation]
with L = q 2 ⇡ 2 , what immediately produce theoretical robustness guarantee on g.
Although this approach is simple and effective, it has a serious drawback: in practice, it is impossible to compute the expectation in (5) exactly and, thus, impossible to compute the prediction of the smoothed function g at any point.
Instead the integral is computed with the use of Monte-Carlo approximation with n samples to obtain the prediction with an arbitrary level of conﬁdence.
Notably, to achieve an appropriate accuracy of the Monte-Carlo approximation, the number of samples n should be large enough that may dramatically affect inference time.
In this work, we generalize the analysis of Lipschitz-continuity to the case of vector functions g : R D !
R d and provide robustness guarantees for classiﬁcation performed in the space of embeddings.
The certiﬁcation pipeline is illustrated in the Figure 1.
3.2 Randomized smoothing for vector functions
Lipschitz-continuity of vector function.
In the work of [38], the robustness guarantee from [3] is proved by estimating the Lipschitz constant of a smoothed classiﬁer.
Unfortunately, a straightforward generalization of this approach to the case of vector functions leads to the estimation of the expectation of the norm of a multivariate Gaussian which is known to depend on the number of dimensions of the space.
Instead, we show that a simple adjustment to this technique may be done such that the estimate of the Lipschitz constant is the same as for the function in (5).
Our results are formulated in the theorems below proofs of which are moved to the Appendix in order not to distract the reader.
Theorem 1.
(Lipschitz-continuity of smoothed vector function) Suppose that f : R D !
R d is a deterministic function and g ( x ) =
E " ⇠ N (0 ,  2 I )
f
( x + " ) is continuously differentiable for all x .
If for all x , k f ( x ) k 2 = 1 , then g ( x ) is L  Lipschitz in l 2  norm with L = [Equation]
Remark 1.
We perform the analysis of Lipschitz-continuity in the Theorem 1 in l 2  norm, so the distance metric in (4) is l 2  distance.
We do not consider other norms in this paper.
Robust classiﬁcation in the space of embeddings.
To provide certiﬁcation for a classiﬁcation in the space of embeddings, one should estimate the maximum deviation of the classiﬁed embedding that does not change the closest class prototype.
In the Theorem 2, we show how this deviation is connected with the mutual arrangement of embedding and class prototype.
[Figure]
{+The direction of adversarial risk in the space of embeddings is always parallel to the vector c 1  c 2 . This is also true for the case of k c 1 k 2 
+}
= k c 2 k 2 .
Theorem 2.
(Adversarial embedding risk)
Given an input image x 2 R D and the embedding g : R D !
R d the closest point on to decision boundary in the embedding space (see
Figure 2) is located at a distance (deﬁned as adversarial embedding risk):
[Equation]
where c 1 2 R d and c 2 2 R d are the two closest prototypes.
The value of  is the distance between classifying embedding and the decision boundary between classes represented by c 1 and c 2 .
Note that this is the minimum l 2  distortion in the embedding space required to change the prediction of g.
Two theorems combined result in an l 2  robustness guarantee for few-shot classiﬁcation as:
Theorem 3.
(Robustness guarantee)
L 2 -robustness guarantee r for an input image x in the n  dimensional input metric space under classiﬁcation by a classiﬁer g from the Theorem 1 is r =  L , where L is the Lipschitz constant from the Theorem 1 and  is the adversarial risk from the
Theorem 2.
The value of r is the certiﬁed radius of g at x , or, in other words, minimum l 2  distortion in the input space required to change the prediction of g. The proof of this fact straightforwardly follows from the deﬁnition (6) and results from Theorems 1-2.
Certiﬁcation protocol
In this section, we describe the numerical implementation of our approach and estimate its failure probability.
4.1 Estimation of prediction of smoothed classiﬁer
As mentioned previously, the procedure of few-shot classiﬁcation is performed by assigning an object x to the closest class prototype.
Unfortunately, given the smoothed function g in the form from the
Theorem 1 and class prototype c k from (2), it is impossible to compute the value ⇢ ( g ( x ) , c k ) explicitly as well as to determine the closest prototype, since it is in general unknown how does g ( x ) look like.
We propose both to estimate the closest prototype for classiﬁcation and to estimate the distance to the closest decision boundary  from Theorem 2 as the largest class-preserving perturbation in the embedding space by computing two-sided conﬁdence intervals for random variables
[Equation]
where ˆ g ( x ) = 1 n
P ni =1 f ( x + " i ) is the estimation of g ( x ) computed as empirical mean by n samples of noise, and one-sided conﬁdence interval for  from the Theorem 2, respectively.
Pseudo-code for both procedures is presented in Algorithms 1-2.
Algorithm 1 Closest prototype computation algorithm.
GIVEN: base classiﬁer f , noise  , object x , number of samples n for ˆ g ( x ) , class prototypes { c i } Ki =1 , maximum number of samples T and conﬁdence level ↵ ,
RETURNS: index A of the closest prototype.
Function CLOSEST( f,  , x, n, { c i } Ki =1 , T, ↵ ) fs ; i n 0 = n while n  T do " 1 , . .
, " n ⇠ N (0 ,  2 I )
ˆ g ( x ) 1 n
P ni =1 f ( x + " i ) fs fs
[ { ˆ g ( x ) } for all i 2 [1 , . . .
, K ] do
C = c
i dsToC ⇢ ( fs, C ) ( l C , u C )
TwoSidedConfInt ( dsToC, ↵ ) l i l C , u i u C end for
A arg min { l i } Ki = if u A < min( { l i } Ki =1 ,i 6 = A ) then Return A, fs else n n + n 0 {Increase number of samples used for computing an approximation ˆ g ( x ) until the number of observations is large enough to determine two leftmost intervals or until n = T } end if end while
EndFunction
The Algorithm 1 describes an inference procedure for the smoothed classiﬁer from the Theorem 1; the Algorithm 2 uses Algorithm 1 and, given input parameters, estimates an adversarial risk from the
Theorem 2 – it determines two closest to smoothed embedding g ( x ) prototypes A and B and produces the lower conﬁdence bound for the distance between g ( x ) and the decision boundary between A and B. Combined with analysis from the Theorem 1, it provides the certiﬁed radius for a sample – the smallest value of l 2  norm of perturbation in the input space required to change the prediction of the smoothed classiﬁer.
In the next subsection, we discuss in detail the procedure of computing conﬁdence intervals in Algorithms 1-2.
4.2 Applicability of algorithms
The computations of smoothed function and distances to class prototypes and decision boundary in
Algorithms 1-2 are based on the estimations of corresponding random variables, thus, it is necessary to analyze applicability of the algorithms.
In this section, we propose a way to compute conﬁdence intervals for squares of the distances between the estimates of embeddings and class prototypes.
Computation of conﬁdence intervals for the squares of distances.
Recall that one way to estimate the value of a parameter of a random variable is to compute a conﬁdence interval for the corresponding statistic.
In this work, we construct intervals by applying well-known Hoeffding inequality [13] in the form
[Equation]
(9) where X and E ( X ) are sample mean and population mean of random variable X , respectively, n is the number of samples and numbers a i ,
b i are such that P ( X i 2 ( a i , b i ))
= 1 .
Algorithm 2 Adversarial embedding risk computation algorithm.
GIVEN: base classiﬁer f , noise  , object x , number of samples n for ˆ g ( x ) , class prototypes { c i } Ki =1 , maximum number of samples T for ˆ g ( x ) and conﬁdence level ↵
RETURNS: lower bound  for the adversarial risk  .
Function EMBEDDING-RISK( f,  , x, n, { c i } Ki =1 , T, ↵ )
A, fs A
CLOSEST( f,  , x, n, { c i } Ki =1 , T, ↵ )
B, fs B
CLOSEST( f,  , x, n, { c i } Ki =1 ,i 6 =
A , T, ↵ ) fs fs A [ fs B  s ; for all f 2 fs do
[Equation]
[Equation]
s [ {  } end for
LowerConfBound (  s, ↵ )
Return
EndFunction
However, a conﬁdence interval for the distance
k ˆ g
( x )  c k k 2 with a certain conﬁdence covers an expectation of distance E ( k ˆ g ( x )  c k k 2 ) , not the distance for expectation k E
(ˆ g ( x )  c k ) k 2 = k g ( x )  c k k 2 .
To solve this problem, we propose to compute conﬁdence intervals for the dot product of vectors.
Namely, given a quantity ⇠ x,k = h g ( x )  c k , g ( x )  c k
i , we sample its unbiased estimate with at maximum 2 n samples of noise (here we have to mention that the number of samples n from
Algorithm 1 actually doubles since we need a pair of estimates ˆ g ( x ) of smoothed embeddings):
[Equation]
and compute conﬁdence interval ( l x,k , u x,k ) such that given
[Equation]
the population mean E (ˆ ⇠ x,k ) is most probably located within it, or, equivalently,
P [Equation]  ↵ .
We have to mention that there are three conﬁdence intervals for three terms (one with quadratic number of samples and two with the linear numbers of samples) in the expression (10), that is why there is fraction 13 in the equation (11).
Also note that the population mean E (ˆ ⇠ x,k ) is exactly k g ( x )  c k k 22 , since
[Equation]
since f ( x + " i ) and f ( x + " j ) are independent random variables for i
= j .
Finally, note that the conﬁdence interval ( l x,k , u x,k ) for the quantity k g ( x )  c k k 22 implies conﬁdence interval
[Figure]
[Figure]
[Equation]
for the quantity k g ( x )  c k k 2 .
Thus, the procedures TwoSidedConfInt and LowerConfBound from algorithms return an interval from (13) and its left bound for the random variable representing corresponding distance, respectively.
Experiments 5.1 Datasets
For the experimental evaluation of our approach we use several well-known datasets for few-shot learning classiﬁcation.
Cub-200-2011 [45] is a dataset with 11 , 788 images of 200 bird species, where 5864 images of 100 species are in the train subset and 5924 images of other 100 species are in the test subset.
It is notable that a lot of species presented in dataset have degree of visual similarity, making classiﬁcation of ones a challenging task even for humans.
mini ImageNet [44] is a substet of images from ILSVRC 2015 [37] dataset with 64 images categories in train subset, 16 categories in validation subset and 20 categories in test subset with 600 images of size 84 ⇥ 84 in each category.
CIFAR FS [1] is a subset of CIFAR 100 [19] dataset which was generated in the same way as mini ImageNet and contains 37800 images of 64 categories in the train set and 11400 images of 20 categories in the test set.
Experimental setup for all the datasets is presented in the next section.
5.2 Experimental settings and computation cost
Following [3], we compute approximate certiﬁed test set accuracy to estimate the performance of the smoothed model prediction with the Algorithm 1 and embedding risk computation with the Algorithm 2.
The baseline model we used for experiments is a prototypical network introduced in [39] with
ConvNet-4 backbone.
Compared to the original architecture, an additional fully-connected layer was added in the tail of the network to map embeddings to 512-dimensional vector space.
The model was trained to solve 1-shot and 5-shot classiﬁcation tasks on each dataset, with 5-way classiﬁcation on each iteration.
Parameters of expeiments.
For data augmentation, we applied Gaussian noise with zero mean, unit variance and probability 0 .
3 of augmentation.
Each dataset was certiﬁed on a subsample ofimages with default parameters for the Algorithm 1: number of samples n = 1000 , conﬁdence level ↵ = 0 .
001 and variance  = 1 . 0 , unless stated otherwise.
For our settings, it may be shown from simple geometry that values ( a i , b i ) from (9) are such that b i  a i 
so we use b i  a i = 4 .
The maximum number of samples T in the Algorithm 1 is set to be T = 5 ⇥ 10 5 .
Computation cost.
In the table below, we report the computation time of the certiﬁcation procedure per image on Tesla V100 GPU for Cub-200-2011 dataset.
Standard deviation in seconds appears to be signiﬁcant because the number of main loop iterations required to separate the two leftmost conﬁdence intervals varies from image to image in the test set.
[Table]
5.3 Results of experiments
In this section, we report results of our experiments.
In our evaluation protocol, we compute approximate certiﬁed test set accuracy, CA .
Given a sample x , a smoothed classiﬁer g ( · ) from the
Algorithm 1 with an assigned classiﬁcation rule h ( x ) =
arg min i 2 { 1 ,...,K } k g ( x )  c
k k 2 , threshold value " for l 2  norm of additive perturbation and the robustness guarantee r = r ( x ) from the
Algorithm 3, we compute CA on test set S as follows:
[Equation]
In other words, we treat the model g ( · ) as certiﬁed at point x under perturbation of norm " if x is correctly classiﬁed by g ( · ) (what means that the procedure of classiﬁcation described in the Algorithm 1 does not abstain from classiﬁcation of x ) and g ( · ) has the value of certiﬁed radius r ( x ) > " .
Visualization of results.
The ﬁgures 3-4 represent dependencies of certiﬁed accuracy on the value of norm of additive perturbation for different learning settings (1-shot and 5-shot learning).
The value of attack radius corresponds to the threshold " from (14).
For Cub-200-2011 dataset we provide a dependency of certiﬁed accuracy for different sample size n for the Algorithm 1 (in the Figure 5).
Limitations
In this section, we provide failure probability of Algorithms 1-2, discuss abstains from classiﬁcation in the Algorithm 1 and speculate on the application of our method in other few-shot scenarios.
6.1 Estimation of errors of algorithms
Note that the value of ↵ from (11) is the probability of the value of ⇢ x,k = k g ( x )  c k k 2 not to belong to the corresponding interval of the form from (13).
Given a sample x , the procedure in the Algorithm 1 returns two closest prototypes to the g ( x ) .
To determine two leftmost conﬁdence intervals, all the distances ⇢ x,k have to be located within corresponding intervals, thus, according to the independence of computing these two intervals, the error probability for the Algorithm 1 is q 1 = K ↵ , where K is the number of classes.
Similarly, the procedure in the Algorithm 2 outputs the lower bound for the adversarial risk with coverage at least 1  ↵ and depend on the output of the Algorithm 1 inside, and, thus, has error probability q 2 = 1  (1  ↵ )(1  K ↵ ) =
↵ + K ↵  K ↵ 2 that corresponds to returning an overestimated lower bound for the adversarial risk from the Theorem 2.
6.2 Abstains from classiﬁcation and extension to other few-shot approaches
It is crucial to note that the procedure in the Algorithm 1 may require a lot of samples to distinguish two leftmost conﬁdence intervals and sometimes does not ﬁnish before reaching threshold value T for sample size.
As a result, there may be input objects at which the smoothed classiﬁer can be neither evaluated nor certiﬁed.
In this subsection, we report the fraction of objects in which the Algorithm 1 abstains from determining the closest class prototype (see Tables 2-3).
n of the Algorithm 1, CIFAR-FS dataset, 5shot case.
Is is notable that relatively small number of samples may be used to achieve satisfactory level of certiﬁed accuracy.
[Table] [Table]
Since our method is based on randomized smoothing, among the approaches presented in [43] it is applicable for matching networks and to some extent to MAML networks.
In case of matching networks, new sample is labeled as weighted cosine distance to support samples, so it is easy to transfer guarantees for l 2  distance to the ones for cosine distance in case of normalized embeddings.
For MAML, the smoothing may be applied to the embedding function as well, but theoretical derivations of certiﬁcates are required.
Related work
Breaking neural networks with adversarial attacks and empirical defending from them have a long history of cat-and-mouse game.
Namely, for a particular proposed defense against existing adversarial perturbations, a new more aggressive attack is found.
This motivated researchers to ﬁnd defenses that are mathematically provable and certiﬁably robust to different kinds of input manipulations.
Several works proposed exactly veriﬁed neural networks based on Satisﬁability Modulo Theories solvers [16, 6], or mixed-integer linear programming [30, 8].
These methods are found to be computationally inefﬁcient, although they guarantee to ﬁnd adversarial examples, in the case if they exist.
Another line of works use more relaxed certiﬁcation [47, 11, 36].
Although these methods aim to guarantee that an adversary does not exist in a certain region around a given input, they suffer from scalability to big networks and large datasets.
The only scalable to large datasets provable defense against adversarial perturbations is randomized smoothing .
Initially, it was found as an empirical defense to mitigate adversarial effects in neural networks [29, 49].
Later several works showed its mathematical proof of certiﬁed robustness [21, 25, 3, 38].
Lecuyer et al [21] ﬁrst provided proof of certiﬁcates against adversarial examples using differential privacy.
Later, Cohen et al [3] provided the tightest bound using Neyman-Pearson lemma.
Interestingly, alternative proof using Lipschitz continuity was found [38].
The scalability and simplicity of randomized smoothing attracted signiﬁcant attention, and it was extended beyond l 2  perturbations [22, 42, 27, 24, 23, 20, 32, 51].
Conclusion and future work
In this work, we extended randomized smoothing as a defense against additive norm-bounded adversarial attacks to the case of classiﬁcation in the embedding space that is used in few-shot learning scenarios.
We performed an analysis of Lipschitz continuity of smoothed normalized embeddings and derived a robustness certiﬁcate against l 2  attacks.
Our theoretical ﬁndings are supported experimentally on several datasets.
There are several directions for future work: our approach can possibly be extended to other types of attacks, such semantic transformations; also, it is important to reduce the computational complexity of the certiﬁcation procedure.
Checklist
For all authors... (a) Do the main claims made in the abstract and introduction accurately reﬂect the paper’s contributions and scope?
[Yes] (b) Did you describe the limitations of your work?
[Yes] See Section(c)
Did you discuss any potential negative societal impacts of your work?
[N/A] (d) Have you read the ethics review guidelines and ensured that your paper conforms to them?
[Yes]
If you are including theoretical results... (a) Did you state the full set of assumptions of all theoretical results?
[Yes] (b) Did you include complete proofs of all theoretical results?
[Yes] The proofs are included in the supplementary material.
If you ran experiments... (a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?
[Yes] The code with instructions is included in the supplementary material.
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)?
[Yes] Applicable training details are described in Section 5.2.
(c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?
[N/A] (d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)?
[Yes] See Section 5.2. 4.
If you are using existing assets (e.g., code, data, models) or curating/releasing new assets... (a) If your work uses existing assets, did you cite the creators?
[Yes] See Sections 5.1 and 5.
(b) Did you mention the license of the assets?
[No] (c) Did you include any new assets either in the supplemental material or as a URL?
[No] (d) Did you discuss whether and how consent was obtained from people whose data you’re using/curating?
[No] The datasets and models are publicly available.
(e) Did you discuss whether the data you are using/curating contains personally identiﬁable information or offensive content?
[N/A] 5.
If you used crowdsourcing or conducted research with human subjects... (a) Did you include the full text of instructions given to participants and screenshots, if applicable?
[N/A] (b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable?
[N/A] (c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation?
[N/A]