C ERTIFIED ROBUSTNESS AGAINST PHYSICALLYREALIZABLE PATCH ATTACKS VIA RANDOMIZED CROPPING
Anonymous authors
Paper under double-blind review
A BSTRACT
This paper proposes a certiﬁable defense against adversarial patch attacks on image classiﬁcation.
Our approach classiﬁes random crops from the original image independently and classiﬁes the original image as the majority vote over predicted classes of the crops.
This process minimizes changes to the training process, as only the crop classiﬁcation model needs to be trained, and can be trained in a standard manner without explicit adversarial training.
Leveraging the fact that a patch attack can only inﬂuence a certain number of pixels in the image, we derive certiﬁed robustness bounds for the classiﬁer.
Our method is particularly effective when realistic transformations are applied to the adversarial patch, such as afﬁne transformations.
Such transformations occur naturally when an adversarial patch is physically introduced in a scene.
Our method improves upon the current state of the art in defending against patch attacks on CIFAR10 and ImageNet, both in terms of certiﬁed accuracy and inference time.
I NTRODUCTION
Despite their incredible success in many computer vision tasks deep neural networks are known to be sensitive to adversarial attacks; small perturbations to an input image can lead to large changes in the output.
A wide range of defenses against adversarial attacks have been conducted in image classiﬁcation, where the goal of the attacker is simply to change the predicted label(s) of an image (Kurakin et al., 2016a; Szegedy et al., 2013; Madry et al., 2017).
But these defenses have typically considered a relatively unrealistic threat model that does not easily extend to the physical settings.
In particular, these works have mainly considered the so-called (cid:96) p -norm threat model, where an attacker is allowed to perturb the intensity at all pixels of the input image by a small amount.
In contrast, adversarial patch attacks are considered as physically-realizable alternatives, modeling scenaria where a small object is placed in the scene so as to alter or suppress classiﬁcation results (Brown et al., 2017).
Here, the attack is spatially compact, but can change the pixel value to any value within an allowable range.
This paper develops a practical and provably robust defense against patch attacks.
Inspired by the randomized smoothing defense (Cohen et al., 2019; Levine & Feizi, 2019) for the (cid:96) p -norm threat model, our approach classiﬁes randomly sampled sub-regions or crops of an image independently and outputs the majority vote across these crops as the class prediction of the input image.
This approach has numerous beneﬁts.
First, given the size of adversarial patches, we can compute the probability of a sampled crop overlapping with the attacked region (patch), and use this probability to determine if the classiﬁcation outcome of an image can be guaranteed (certiﬁed) to not be changed by any adversarial patch.
Second, this approach is highly practical, as the crop classiﬁer can be trained using standard architectures such as VGG (Simonyan & Zisserman, 2014) or ResNet (He et al., 2016) without the need for adversarial training.
Indeed, random cropping is already a common data augmentation strategy for training machine learning models, and thus the method can be trained via standard techniques.
This is different from most existing work on certiﬁable defenses against patch attacks (Levine & Feizi, 2020; Xiang et al., 2020; Chiang et al., 2020) which need extra computation for certiﬁcation during training.
Third, the proposed approach separates the training procedure from the patch threat model, thus making the method more robust against realistic settings of patch attacks, for example, patch transformations including rotation in x-y plane and aspect ratio
[Table]
changes.
This is in sharp contrast to prior work that either have ﬁxed strategies to exclude parts of the image or extract features from ﬁxed parts of the image (Levine & Feizi, 2020; Xiang et al.,
We summarize our main results on CIFAR10 and ImageNet in Table 1 in comparison with the current state of the art certiﬁable defense against patch attack (Xiang et al., 2020) with patch transformation.
We report certiﬁed accuracy, which is the percentage of test images for which classiﬁcation outcome equals to the ground truth label and is guaranteed to not change under patch attack.
Our method is better in both speed and certiﬁed accuracy compared to De-randomized smoothing (Levine & Feizi, 2020) and PatchGuard (Xiang et al., 2020) under patch attack with possible afﬁne transformations of the patch.
In addition, our method outperforms these past approaches on ImageNet (though not on CIFAR10) in the setting where the patch aligns with coordinates of the image axes and does not undergo afﬁne transformations as in Table 2, which was the setting considered in this past work.
We have made several contributions in this paper: ﬁrst, we propose a defense against patch attack for image classiﬁcation with certiﬁed robustness; second, the proposed method is fast in computing image certiﬁcation and robust against patch transformation; third, the proposed method can be applied to any image classiﬁcation model with only minimal changes to the training process.
B ACKGROUND AND RELATED WORK
Adversarial attacks Adversarial attacks on image classiﬁcation have been known for some time, with original work coming out of the ﬁeld of robust optimization (Ben-Tal et al., 2009).
Testtime attacks on ML models in general were studied in (Dalvi et al., 2004; Biggio et al., 2013), though the area gained momentum considerably when these methods were applied to deep learning systems to demonstrate that deep classiﬁers could be easily fooled by imperceptible changes to images (Szegedy et al., 2013; Goodfellow et al., 2014).
In the following years many defenses against such attacks were proposed (Tram`er et al., 2017; Papernot et al., 2016), although most heuristic approaches were later found to be ineffective (Athalye et al., 2018).
Amongst the defense strategies that have stood the test of time are: 1) adversarial training (Goodfellow et al., 2014; Kurakin et al., 2016b; Madry et al., 2017), now commonly carried out using a projected gradient descent based approach for synthesizing adversarial attacks and then incorporating them into the training; and 2) provably robust training (Wong & Kolter, 2017; Raghunathan et al., 2018).
Our approach is more related to the randomized smoothing-based methods (Cohen et al., 2019; Levine & Feizi, 2019) in the latter direction, in which random points around the original input are sampled and classiﬁed, and the predicted class of the original input is declared as the aggregation of these outputs.
The majority of the attacks mentioned above focus on attacks with bounded (cid:96) ∞ -norm, where attacks are permitted to modify any pixel in the image by (at most) some ﬁxed amount (and are usually permitted to design a new adversarial perturbation for each new input image, though the so-called “universal” (cid:96) ∞ attacks have been studied as well (Moosavi-Dezfooli et al., 2017)).
If we consider real-life attackers, however, this level of freedom on the attacker’s side seems uncommon: attackers would need to have access to the deep learning system at the pixel level, which if true, also means that the system has already been compromised, making it unclear what beneﬁt such an attack would have.
In contrast, physically-realizable attacks such that can be implemented in the real world have been primarily modeled by the so-called patch attacks (Brown et al., 2017; Eykholt et al., 2018), where a particular ‘pattern’ is designed to fool a deep learning system.
We consider patch attacks as the threat model to defend against for it being physically realizable.
Defense against patch attacks There are also two threads of research in defending against patch attacks: 1) empirical defense strategies which show stronger empirical robustness but no analytical guarantees (McCoyd et al., 2020; Naseer et al., 2019; Hayes, 2018); most of these methods rely on the fact that a patch attack has to produce strong local features that are very different from its neighborhood to change classiﬁcation outcome; by analyzing local features, one can detect patch location or smooth out such feature values to reduce effectiveness of the patch, and 2) certiﬁed defenses which provide analytical lower bounds of classiﬁcation accuracy under patch attack.
Certiﬁable defense against patch attacks Relevant to our work is the defense which provides provable lower bounds of classiﬁcation accuracy.
In Chiang et al.
(2020), interval bound propagation was used to provide certiﬁcation, but the method is not applicable to commonly used image classiﬁcation networks such as ResNet50 ( He et al., 2016) or VGG16 (Simonyan & Zisserman, 2014) because the depth of these architectures renders the bound trivial, and thus cannot provide useful certiﬁcation.
Another line of research utilizes the fact that an adversarial patch can only have local inﬂuence on the image. To reduce such inﬂuence, De-randomized smoothing (Levine & Feizi, 2020) uses ablation to exhaustively block all possible patch locations while (Zhang et al., 2020) uses
Bagnets (Brendel & Bethge, 2019) with limited receptive ﬁeld to contain the number of affected features.
Both methods then aggregate logits from ablated versions of images or local regions for ﬁnal classiﬁcation. PatchGuard
(Xiang et al., 2020) adds additional detection of patch location and then sets features extracted from the detected locations to be zero.
However all three methods require special image classiﬁer structures – Levine & Feizi (2020) require three additional channels to express ablation, Zhang et al.
(2020) is limited to Bagnets, and Xiang et al.
(2020) uses either of the former structures.
Our proposed method instead can use any image classiﬁcation architecture and does not require any additional steps to compute certiﬁcation during model training except cropping images.
This makes our method much more practical and easier to deploy onto existing systems.
We will compare our proposed method with De-randomized smoothing (DRS) and PatchGuard (PG) in the experiments and an overview of the two prior arts can be found in Appendix A.
P ROVABLE PATCH DEFENSE WITH RANDOMIZED CROPPING
A physically-realizable adversarial patch attack can be found by solving the optimization problem:
[Equation]
where h θ : X → Y denotes some (presumably deep-network-based) hypothesis function; θ denotes parameters of the model, x ∈ X denotes the original input to the network and δ ∈ X the perturbation to the input; ∆ ⊆
X denotes the set of allowable perturbations; y ∈ Y denotes the true label; (cid:96) :
Y × Y → R denotes a loss function that measures the performance of the image classiﬁer; and T : X → X is the set of transformations that the perturbation δ might go through.
The feasible set ∆ denotes a simple allowable set of values for the patch, which typically would just be constrained to lie in valid RGB space.
Patches overwrite a portion of the image with the patch perturbation itself, at a given location with a given set of transformations, such as scaling, rotation, and other transformations in T .
We refer to this combination as the patch application function A :
X ×X ×T → X , where A ( x, δ, t ) denotes the application of patch δ to image x with transformation t .
Attack budget considered for patch attack is the percentage of area that the patch covers over the whole image.
Throughout this paper, we consider the patch being one connected area bounded by a rectangle with size p i × p j .
3. RANDOMIZED CROPPING DEFENSE
Although a patch attack can change pixel values to any arbitrary value, it can only inﬂuence the pixels within the patch itself.
Therefore, networks extracting features with compact receptive ﬁelds
[Figure]
and aggregating such local features for ﬁnal classiﬁcation are more robust against patch attacks. An example of such methods is Bagnets (Brendel & Bethge, 2019), where 1x1 convolutional kernels are used to restrict the size of the receptive ﬁelds.
Several existing techniques (Zhang et al., 2020;
Xiang et al., 2020) utilize Bagnets for adversarial defense.
However, the major drawback of Bagnets is that it extracts local features at ﬁxed locations, therefore a patch may be placed in particular locations to inﬂuence more number of local features than other locations, thus reducing the worstcase robustness of Bagnets.
In contrast to this architecture, we propose to extract local features at random locations by sampling random crops; this greatly reduces variations in the number of local features an adversarial patch can inﬂuence.
Based on above observations, we propose a randomized cropping approach as shown in Fig. 1: given a full-size test image x as input, we ﬁrst randomly select n crops uniformally over all possible locations, where crop size k i × k j is smaller than image size
m i × m j in both dimensions.
Each of the sampled crops ( (cid:98) x i ) then goes through the crop-based classiﬁer g θ and its predicted class ( (cid:98) y i ) is obtained.
The ﬁnal classiﬁcation of x is the majority of { (cid:98) y i } ni =1 . The equivalent pseudo code of
Fig. 1 is in Appendix B. Training randomized cropping classiﬁer
The robust classiﬁer f θ has three components: (1) sampling n crops with uniform distribution, (2) classifying crops with g θ , and (3) majority voting.
Since both random sampling of the crops via a uniform distribution and majority voting have no trainable parameters, the set of trainable parameters of f θ is the same as g θ . Therefore training the robust classiﬁer f θ only involves training the crop classiﬁer g θ , which can be trained in standard ways with only data augmentation of randomly cropping the input x as in Algorithm 1.
Other variables are either pre-determined hyperparameters (crop size k i , k j ), or can be adjusted at test time (number of crops n ).
Since the training procedure of randomized cropping classiﬁer f θ only requires training the crop classiﬁer g θ without any assumption on attack parameters (size, shape, and location), nor does it depend on hand-crafted rules or thresholds, our method is robust against different patch shapes.
We show supportive experimental results in Section 4 for this claim. function
Input: Full size image data x , labels y for t = 1, ..., do for m = 1, ..., M do Set (cid:98) x as a crop of size k i × k j from image x at a random location; θ = θ − ∇ θ (cid:96) ( f θ ( (cid:98) x, y ) // Update model weights with some optimizer, e.g. SGD end end
3. C ERTIFIABLE ROBUSTNESS BOUND
Here we compute the probability that an image x is certiﬁed robust, i.e., the classiﬁcation outcome of x cannot be changed by patch attack of given size. Given a clean image, we randomly sample n crops { (cid:98) x i } ni =1 and obtain the set of n predicted classes { (cid:98) y i } ni =1 of the crops. Let n 1 be the number crops that are predicted as the majority in { (cid:98) y i } n i =1 , and n 2 be the number crops that are predicted as the second majority in { (cid:98) y i } ni =1 .
Then if there exists a patch attack that can change the classiﬁcation outcome of x , then the patch must overlap with at least n 2 to 1 = (cid:98) n 1 − n 2 (cid:99)+ 1 of the crops.
On the other hand, if there are only less than n 2 to 1 crops that overlap with the patch, then x is certiﬁed robust. Therefore, p c can be represented as the probability that, out of the n sampled crops, there are less than n 2 to 1 crops that overlap with the adversarial patch. The below derivation assumes that the random selection is uniform over all crop locations with replacement, i.e., the same crop can be selected again.
Derivations for uniform sampling crops without replacement can be found in Appendix C. Following the argument above, we ﬁrst compute the probability that a single sampled crop x i overlaps the adversarial patch, and use this probability to compute p c . This probability can be computed as the total number of crops overlapping with the patch (denoted by n adv ) divided by the number of all possible crops ( n all ).
Although a physically-realizable adversarial patch can be at any location in the image, to guarantee robustness, we consider the worst-case scenario where the patch is at the location that it can inﬂuence maximum number of crops (usually at the center of the image).
When the patch is at the center of the image,
[Equation] [Equation]
After we sample n crops { (cid:98) x i } ni =1 from the original input x and { (cid:98) y i } ni
{+=
 = { g θ ( (cid:98) x i ) } are+} the predicted classes of the crops. Suppose in { (cid:98) y i } ni =1 , n 1 of them are the majority class y (cid:48) , and n 2 of them are the second-majority class y (cid:48)(cid:48) .
If an adversarial patch changes the majority class, the minimal number of crop-based predicted classes that it has to change will be
[Equation]
to make y (cid:48)(cid:48) the majority class.
The probability of certiﬁcation p c is equal to the probability that out of { (cid:98) x i } ni =1 , at most n 2 to 1 −of them overlaps with the adversarial patch:
[Equation]
where C ni is the binomial coefﬁcient ( n choose i ).
If p c is close to 1.0, the input image is certiﬁably robust under patch attack. In the experiments we select p c ≥ 0 .
95 for an image to be considered certiﬁed.
Throughout this paper, we consider certiﬁed accuracy , meaning only data with predicted class y (cid:48) being the ground truth class will be considered for certiﬁcation. Note that because the crops are randomly sampled, n 2 to 1 and hence p c is an instance of a random distribution. However we argue that when n is large enough, there n 2 to 1 and p c will have very small variance. Experiments to verify this point are in Appendix D.
The above computation of probability of certiﬁcation in Eq.
6 again shows that p c is monotonically increasing with n 2 to 1 .
So to maximize number of certiﬁed robust images, the randomized cropping classiﬁer should maximize n 2 to 1 , which is equivalent to maximizing classiﬁcation accuracy of g θ . This is another way to view that, to train the proposed robust classiﬁer f θ , one only needs to train g θ .
E XPERIMENTAL RESULTS
Datasets, models, and device. We conduct experiments on two benchmark image classiﬁcation datasets: CIFAR10 and ImageNet.
For both datasets we evaluate over the whole evaluation set and report certiﬁed accuracy as percentage of images that are classiﬁed correctly and can be certiﬁed [Table] with probability higher than 0.95.
For CIFAR10, we use ResNet9 without the ﬁnal pooling layer as the crop model [Table] and for ImageNet we use ResNet34 as the crop model and resize and pad the image to 224x224.
Experiments and timing are done on single Nvidia
Ti GPU, and 16-core Intel i7-5960X CPU.
For both models we use cyclic learning rate with 10 − 3 initial learning rate. The models are trained 30 epochs and training time is 14 GPU-hours and 328 GPU-hours for CIFAR and ImageNet, respectively. we chose crop size of 10x10 (10% total area) for CIFAR and 80x80 (13% total area) for ImageNet; for these experiments we assume patch shape is square with no rotation .
For each input image in CIFAR10, we randomly sample 128 crops and for each image in ImageNet we sample 256 crops. Positional encoding. Because both CIFAR10 and ImageNet images have a certain region of interest which is usually in the center of the image, some parts of the images contain more information than others.
For example, an image with label airplane usually has the airplane in the center, while the corners and edges have the background of sky or airport.
Therefore, crops sampled at different position of the image contain different information.
To represent such information, we add learnable positional encoding (Vaswani et al., 2017) to the ﬁrst layer of our classiﬁer. Metric.
Although the certiﬁcation probability p c can be computed to any clean image despite of which data split this image is, in the experiment we follow the literature and report the certiﬁcation accuracy, i.e., percentage of test images that are certiﬁed and classiﬁed correctly .
This metric directly reﬂects how robust and accurate the classiﬁer is – for example, the ideal classiﬁer with 100% certiﬁed accuracy should classify all test images correctly and all test images’ predicted class cannot be change by patch attack.
We consider an image with p c ≥ 0 .
95 to be certiﬁed.
4. C LEAN ACCURACY AND INFERENCE TIME
Clean model accuracy and inference time per image compared with the above mentioned prior work are shown in Table 1. Note that clean model accuracy and inference time remain the same with and without afﬁne transformation of the adversarial patch. In terms of inference time our method is faster under all cases. This is because although multiple forward passes are needed, the number of crop samples is smaller than equivalent sub-regions in Bagnets and much smaller than number of all band-smoothed images as in De-randomized smoothing (DRS).
PatchGuard with Bagnets (PG-BN) is slower than our method in ImageNet for the additional reason of having to go through all logits of all possible classes sequentially. In addition, both De-randomized smoothing (DRS) and PatchGuard with De-randomized smoothing (PG-DRS) as the base structure are an order of magnitude slower than our method because De-randomized smoothing passes the whole image through the classiﬁer, which is much more expensive then passing crops (10% or 13% of the whole image) through the classiﬁer. [Table] [Table]
4. W ITHOUT PATCH TRANSFORMATION
We ﬁrst present the certiﬁcation accuracy in Table 2 of our randomized cropping method along with De-randomized smoothing and PatchGuard on patch size ranging from 0 .
4% to 3% . Position encoding used here is learnable position encoding with values up to 10% of normalized input value range.
We mainly compare our results with two prior arts: De-randomized smoothing (DRS) (Levine & Feizi, 2020) and PatchGuard (PG)
(Xiang et al., 2020).
For all PatchGuard results there are two sets of numbers separated by slash: the ﬁrst set of numbers use De-randomized smoothing(Levine & Feizi, 2020) as base structure and the second set of numbers use Bagnets (Brendel & Bethge, 2019). Methods with the highest certiﬁed accuracy are highlighted in bold.
We can see that although our method is sub-optimal for CIFAR10, for ImageNet we have the highest certiﬁed accuracy except for patch size 0 . We also show the certiﬁcation accuracy with different thresholds for p c Table 3.
As discussed in the previous subsection, our method is the fastest among the three methods.
Although using Bagnets (PG-BN) as base structure makes PatchGuard much faster than using Derandomized smoothing (PG-DRS), it pays the price of degrading certiﬁed accuracy.
For example, certiﬁcation accuracy degrades from 58.1% (PG-DRS) to 31.7% (PG-BN) for CIFAR10 with 2.4% patch size while our randomized cropping defense is as fast as PatchGuard with Bagnets (PG-BN) as show in Table 1 (inference time remains the same with or without patch transformation), but achieves much better (52.3%)
certiﬁed accuracy.
Moreover, on ImageNet our proposed method is the fastest and with the highest certiﬁed accuracy as well as clean accuracy.
Combining the certiﬁed accuracy on ImageNet in Table 2 and the short inference time in Table 1, our experiments show that the proposed randomized cropping defense is practical in the aspects of fast certiﬁcation and high certiﬁed accuracy for the dataset that is closer to real-life pictures.
4. W ITH PATCH TRANSFORMATION
As described in Section 3, a physically-realizable patch can be subject to certain transformations, such as scaling and rotation, before overwriting pixels in an image.
These transformations can be used to mimic relative camera angle and distance to the scene in the real physical world.
While it is reasonable to restrict the maximal area that an adversarial patch can inﬂuence in an image (since it corresponds to how big the physical patch is), one cannot assume such area would always align with coordinate axes of the image – even if the physical patch itself is square, when the scene is or stretched (rotation in depth).
Therefore it is important that the patch defense can also provide guarantees when the patch is rotated or if the aspect ratio is varied, given the same patch size.
In Table 4 we compare the proposed randomized cropping defense with De-randomized smoothing (Levine & Feizi, 2020) and PatchGuard (Xiang et al., 2020) when a square patch is rotateddegrees and with aspect ratio (AR) 1:1, 6:1, 2.7:1, 1:2.7, 1:6, respectively.
We highlight the highest certiﬁed accuracy under worst transformation for both datasets.
Aspect ratio 1:1 without rotation is the same as Table 2 but listed here as reference.
We chose 45 degrees because it is the worst case for a square patch, as the same number of pixels within the patch but occupying the largest rectangle area that aligns with coordinate axes of the image.
Therefore rotation of 45 degrees provides a lower bound of robustness under patch rotation.
For all transformations of the patch, we consider the same patch size.
We can clearly see that in Table 4, with the same patch size, transformation brings down the certiﬁed accuracy of all three competing methods, however, the proposed randomized cropping defense has the highest certiﬁed accuracy under worst-case patch transformation as listed in the last row of Table 4.
This is because of our random sampling strategy that we have neither ﬁxed the locations of subregions as in Bagnets, nor ﬁxed smoothing strategy as in De-randomized smoothing.
On the other hand, the ﬁxed column-smoothing strategy (ablating columns of images) in Derandomized smoothing (Levine & Feizi, 2020) which has the highest certiﬁable accuracy for square patches aligning coordinate axes of the image suffers particularly because as the patch gets shorter and longer (aspect ratio 1:3 and 1:6), the number of columns needed to ablate all the patch gets higher, and leaves fewer pixels of the image for classiﬁcation.
Also rotating the patch 45 degrees is equivalent to increasing the number of columns the patch occupies by 1.4 times, and hence significantly reduces certiﬁed accuracy for column-smoothing strategy.
While there are other smoothing techniques in De-randomized smoothing (block smoothing), similar effects can be expected as some aspect ratio or rotation degree will cause signiﬁcant degradation of certiﬁed accuracy.
Same level of degradation is also seen in PatchGuard with De-randomized smoothing base structure (numbers in
PatchGuard column before slash symbol).
How to choose crop size? Assuming that the number of crops n is ﬁxed, then in general larger crops leads to a better clean performance, as each crop contains more information when it covers more pixel area. Also with larger crops, the crop classiﬁcation accuracy of g θ would be better, indicating that n 2 to 1 in Eq.
5 could be larger and increases the probability of certiﬁcation p c .
However, larger crops also means that the probability of overlapping the adversarial patch is higher (Eq. 7) which will decrease p c . Therefore, for a given size of adversarial patch, there exists an optimal crop size which maximizes the certiﬁcation probability. To demonstrate the inﬂuence of crop size on certiﬁcation accuracy and clean accuracy, clean and certiﬁed accuracy with or without positional encoding, with regards to different crop sizes are shown in Figure 2 for 2.4% patch on CIFAR10 and 2.0% patch on ImageNet – we use square crops and square patches aligning with coordinate axes of the image, i.e., no patch transformation. Comparing clean accuracy with and without positional encoding, we can see that although clean accuracy still increases as crop size increases when crop classiﬁer includes positional encoding, but not as much as without positional encoding.
Such results show positional encoding does provide extra information for crops sampled from different locations.
On the other hand, comparing clean accuracy with certiﬁed accuracy, it is clear that the certiﬁed accuracy actually gets lower when crop size crosses some threshold as discussed above.
Note that similar experiments as in Figure 2 can be conducted for different sizes of adversarial patches to ﬁnd optimal crop size. However we used ﬁxed crop size in Table 1, 2, 4 to compare with DRS and PG to have a fair comparison, as these two approaches do not have tune-able ablation/kernel size.
How to choose number of crops? We show certiﬁed accuracy and inference time with different number of sampled crops for 2.4% patch on CIFAR10 and 2.0% patch on ImageNet. Certiﬁed accuracy of using all crops, i.e., selecting crops at all locations once without sampling, is plotted
[Figure] [Figure]
C ONCLUSION
This paper proposes a new architecture for defense against physically-realizable patch attacks.
The proposed approach decomposes an image into a random assortment of crops, each of which is processed by a classiﬁer, and the majority across the crops is used as the classiﬁcation outcome for the input image.
This approach provides a signiﬁcant advance in the form of improved certiﬁed accuracy, while maintaining a high clean accuracy when compared to prior art.
We have shown that the proposed approach can be easily incorporated into standard training and test pipelines with minimal change to the underlying codebase.
We believe this approach, by relying on a simple yet effective architectural change, provides a compelling alternative to traditional defense strategies and, in this sense, advances robust machine learning. A O VERVIEW OF D E - RANDOMIZED SMOOTHING (DRS) AND P ATCH G UARD (PG)
De-randomized smoothing (Levine & Feizi, 2020) ablates all possible parts of the image and aggregates logits of the ablated images for certiﬁcation.
The method compares number of ablated images with high logit value of the majority predicted class with the number of ablated images with high logit value of the second majority predicted class.
If the difference between the two is larger than two times possible number of ”ablation blocks” affected by adversarial patch, then this image is certiﬁed robust against patch attack.
The paper proposes two modes of ablation: block smoothing and band smoothing.
Block smoothing ablates square blocks while band smoothing ablates one column of the image.
To represent ablated regions, the image classiﬁer accept three additional channels representing ablated pixels of the original RGB channels.
The main idea of PatchGuard (Xiang et al., 2020) is to detect possible patch location and mask these locations for downstream robust classiﬁcation.
To detect patch location, the method identify if there are any local regions that contribute abnormally strongly to a class.
If there does exist such a region, it is considered as the potential location of a patch and the features extracted from the region is masked/discarded.
Because such robust masking procedure can be combined with other robust classiﬁcation approaches, the paper evaluates its robust masking with two classiﬁcation models:
De-randomized smoothing (PG-DRS) and BagsNet (PG-BN).
B D ETAILED CERTIFICATION PROCEDURE
We summarize the certiﬁcation procedure for a single image in Algorithm 2.
A more robust classiﬁer should be able to certify and correctly classify a higher percentage of clean images in the test set.
Algorithm 2: Certify if patch attack can change the predicted class of an image x
Input i = 1 ...n do
set (cid:98) x
i as a k i × k j crop of x at random location; (cid:98) y
i =
g θ (
end
; if (cid:98) y (cid:48) (cid:54) =
y then return not certiﬁed else compute p c using Eq.
6, if p c is close to 1.0 then return certiﬁed, else return not certiﬁed end
C U NIFORM SAMPLING WITHOUT REPLACEMENT
In this section we derive certiﬁcation probability and experimental results with uniform sampling without replacement.
Let n iall be the number of all possible location for the i th sampled crop, n adv be the number of crop locations that overlaps with the adversarial patch, and p ia is the probability that the i th crop overlaps with the adversarial patch, then
[Equation]
[Equation]
The probability of the image being certiﬁed p c is then the probability of less than n 2 to 1 crops overlaps with the patch.
The closed-form expression of p c is complicated yet not informative and hence omitted here.
Comparing p ia with the p a in Section 3, we can see that when sampling without replacement, the probability of sampling a crop that overlaps with the adversarial patch increases with
[Table]
[Table]
the number of crops sampled, and hence decreases the probability of certiﬁcation p c .
On the other hand, sampling without replacement enlarges the expected area that crops would cover, so the clean performance will be better than sampling with replacement.
We compare the certiﬁed and clean accuracy with and without replacement in Table 5.
As number of crops increases, we can see that the gain of clean accuracy for sampling without replacement decreases because when number of crops increases, even sampling with replacement is likely to cover most of the pixels, and the gain is slightly more signiﬁcant in ImageNet than CIFAR10.
This may be because ImageNet images are in general more complex than CIFAR10 and having the crops covering more pixels over the image could help the overall classiﬁcation.
The certiﬁed accuracy for sampling without replacement gets worse than with replacement since the probability of sampling a crop increases more signiﬁcantly as the number of crops sampled increases.
This is particularly true for CIFAR10 – with image size 32x32 and patch size 10x10, the number of non-overlapping crops that does not overlap with the patch is only 484, out of 1024 all possible locations.
This means when sampling 512 crops, there are at least 28 crops overlapping with the adversarial patch, which signiﬁcantly decrease p c .
D E XPERIMENTS ON p c INTERVAL
In this section we show the interval of n 2 to 1 and p c with different number of crops sampled.
We run the certiﬁcation process over test set 200 times to obtain the interval and variance of n 2 to 1 and p c for each test image.
Interval is deﬁned as difference between the highest value and the lowest value.
We ran this experiment over patch size 2.4% for CIFAR10 and 2.0% for ImageNet.
As shown in Table 6, with increasing number of crops, both interval and variance of n 2 to 1 and p c decrease, to negligible values.