Certified Defences Against Adversarial Patch Attacks on Semantic Segmentation
Authors: Maksym Yatsura
Kaspar Sakmann
N. Grace Hua
Matthias Hein
Jan Hendrik Metzen
Venue: ICLR 2023 poster (2023)
Keywords: certified defences
adversarial robustness
adversarial patch attacks

Abstract: Adversarial patch attacks are an emerging security threat for real world deep learning applications. We present Demasked Smoothing, the first approach (up to our knowledge) to certify the robustness of semantic segmentation models against this threat model. Previous work on certifiably defending against patch attacks has mostly focused on image classification task and often required changes in the model architecture and additional training which is undesirable and computationally expensive. In Demasked Smoothing, any segmentation model can be applied without particular training, fine-tuning, or restriction of the architecture. Using different masking strategies, Demasked Smoothing can be applied both for certified detection and certified recovery. In extensive experiments we show that Demasked Smoothing can on average certify 63% of the pixel predictions for a 1% patch in the detection task and 46% against a 0.5% patch for the recovery task on the ADE20K dataset. 

Published as a conference paper at ICLR 2023
CERTIFIED DEFENCES AGAINST ADVERSARIAL PATCH ATTACKS ON SEMANTIC SEGMENTATION
Maksym Yatsura!.; Kaspar Sakmann', N. Grace Hua', Matthias Hein Jan Hendrik Metzen' Bosch Center for Artificial Intelligence, Robert Bosch GmbH, Univerritty of Tübingen, 'Tübingen AI Center
ABSTRACT
Adversarial patch attacks are an emerging security threat for real world deep learning applications. We present DEMASKED SMOOTHING, the first approach (up to our knowledge) to certify the robustness of semantic segmentation models against this threat model. Previous work on certifiably defending against patch attacks has mostly focused on image classification task and often required changes in the model architecture and additional training which is undesirable and computationally expensive. In DEMASKED SMOOTHING, any segmentation model can be applied without particular training, fine-tuning, or restriction of the architecture. Using different masking strategies, DEMASKED SMOOTHING can be applied both for certified detection and certified recovery. In extensive experiments we show that DEMASKED SMOOTHING can average certify 63% of the pixel predictions for 1% patch in the detection task and 46% against a 0.5% patch for the recovery task the ADE20K dataset.
INTRODUCTION
Physically realizable adversarial attacks are a threat for safety-critical semii)aautooomon systems such self-driving cars robots. Adversarial patches (Brown al. 2017: Karmon et al. 2018)
the most prominent example of such an attack. Their realizability has been demonstrated repeatedly, for instance by Lee & Kolter (2019): attacker places printed version of adversarial patch in the physical world to fool a deep learning system. While empirical defenses (Hayes) 2018 Naseer et al. 2019 (Selvrrauu et al. 2019 Wu al. 2020) may offer robustness against known attacks, does not provide any guarantees against unknown future attacks (Chiang et al. 2020). Thus, certified defenses for the patch threat model, which allow guaranteed robustness against all possible attacks
for the given threat model, are crucial for safety-critical applications.
Research on certifiable defenses against adversarial patches can be broadly categorized into certified
recovery and certified detection. Certified recovery (Chiang et al. [2200] Levine & Feizi| 2020 Zhang
ct al.
2020 Xiang et al.
2021 Metzen Yatsura| 2021 Lin et al.
202 |
[Xiang et al. 2022a Salman
et al.
2021 Chen et al.
2022 has the objective to make a correct prediction on an input even in the
presence an adversarial patch. contrast, certified detection (MeCoyd et al. 2020) Xiang Mittal
2021b Han et al. 2021 Huang & Li 2021) provides weaker guarantee by only aiming at detecting inputs containing adversarial patches. While certified recovery is more desirable in principle,
typically comes at high cost of reduced performance clean data. In practice, certified detection might be preferable because it allows maintaining high clean performance. Most existing certifiable defenses against patches are focused image classification, with the exception of DetectorGuard (Xiang & Mittal 2021a) and ObjectSeeker (Xiang et al. 2022b) that certifiably defend against patch hiding attacks on object detectors. Moreover, existing defences are not easily applicable arbitrary downstream models, because they assume either that the downstream model is trained explicitly for being certifiably robust (Levine & Feizi, 2020 Metzen & Yatsura, 2021). or that the model has certain network architecture such as BagNet (Zhang et 20200 Metzen & Yatsura 2021 Xiang & 2021). A notable exception is
al. 202T) or a vision transformer (Salman et al. al.]2021||Huaa PatchCleanser (Xiang et al. (2022a), which can be combined with arbitrary downstream models but is restricted to image classification.
"Correspondence to: Maksym Yatsura cmaksym.yatsuraa bow.boocccoom>>

---

Published as a conference paper at ICLR 2023
Input with patch
Patch visible
Patch masked
(b)
Masking
Demasking
Segmentation
Aguregation
Input
Segmentation mup Certification map
(c)
Figure 1: (a) A simple patch attack on the Swin transformer (Liu al. 2021) manages switch the prediction for big part of the image. (b) Masking the patch. (c) A sketch of DEMASKED SMOOTHING for certified image segmentation. First, we generate set of masked versions of the image such that each possible patch only affect certain number of masked images. Then image inpainting to partially recover the information lost during masking and then apply an arbitrary segmentation method. The output is obtained by aggregating the segmentations pixelwise. The masking strategy and aggregation method depend on the certification mode (detection or recovery).
Adversarial patch attacks were also proposed for the image segmentation problem (Nesti et al. 2022 mostly for attacking CNN-base models that use a localized receptive field (Zhao et al. 2017 However, recently self-attention based vision transformers (Dosovitskiy et al. 202 1) have achieved new state-of-the-ar in the image segmentation task (Liu et al. [221]] [Bousselham et al. 202T). Their output may become more vulnerable to adversarial patches if they manage to manipulate the global self-attention (Lovisotto al.) 2022). We demonstrate how significant parts of the segmentation output may be affected by a small patch for Swin tranfromer Liu et al. (2021) in Figure Tal Full details the attack available in Appendix [D] We point out that preventive certified defences important because newly developed attacks can immediately be used to compromise safety-critical applications unless they are properly defended.
In this work, we propose the novel framework DEMASKED SMOOTHING (Figure Ic to obtain the first (to the best of our knowledge) certified defences against patch attacks on semantic segmentation models. Similarly to previous work (Levine & Feizi 2020). we mask different parts of the input (Figure (b) and provide guarantees with respect every possible patch that is not larger than certain pre-defined size. While prior work required the classification model to deal with such masked inputs, we leverage recent progress in image inpainting (Dong et al., (202)) reconstruct the input before passing it to the downstream model. This decoupling of image demasking from the segmentation task allows us to support arbitrary downstream models. Moreover, we can leverage state of the art methods for image inpainting. We also propose different masking schemes tailored for the segmentation task that provide the dense input allowing the demasking model to understand the scene but still satisfy the guarantees with respect the adversarial patch.
We summarize our contributions as follows:
We propose DEMASKED SMOOTHING which is the first (to the best of our knowledge)
certified recovery or certified detection based defence against adversarial patch attacks on semantic segmentation models (Section 4
DEMASKED SMOOTHING can do certified detection and recovery with any off-the-shelf segmentation model without requiring finetuning or any other adaptation.
We implement DEMASKED SMOOTHING, evaluate it for different certification objectives
and masking schemes (Section 5) We can certify 63% of all pixels in certified detection for
a 1 1 patch and 46% in certified recovery for 0.5% patch for the BEiT-B (Bao et al. 2022) segmentation model on the ADE20K Zhou et (201 dataset.
2

---

Published as a conference paper at ICLR 2023
2 RELATED WORK
Certified recovery. The first certified recovery defence for classification models against patches was proposed Chiang et al. (2020), who adapted interval-bound propagation (Gowal et al. 2019)
to the patch threat model. Levine & Feizi (2020) proposed De-Randomized Smoothing (DRS),
which provides significant accuracy improvement when compared to [Chiang et al. (2020) and scales to the ImageNet dataset. In DRS, base classifier is trained images where everything but small local region is masked (ablated). At inference time, a majority vote of all specified ablations is taken as the final classification. If this vote has large enough margin to the runner-up class, the prediction cannot be shifted by any patch that does not exceed a pre-defined size. A similar approach was adopted in Randomized Cropping (Lin et al. 202Ip. general drawback of these approaches is that the classifier needs to be trained to process masked/cropped inputs, which (in contrast to our work) prohibits the usage of arbitrary pretrained models. A further line of work studies network architectures that are particularly suited for certified recovery. For instance, models with small receptive fields such as BagNets (Brendel & Bethge| 2019) have been explored, either by combining them with some fixed postprocessing (Zhang et al. 2020 Xiang et al.) 222T] or by training them end-to-end for certified recovery (Metzen & Yatsura 2021). |Salman et al. (2021) propose to apply DRS to Vision Transfomers (ViTs). In contrast to the aforementioned works, our Demasked Smoothing can be applied to models with arbitrary architecture. This is a property shared with PatchCleanser (Xiang al. 2022a), J which however is limited to image classification and it is not clear how it can be extended to semantic segmentation where a class needs to be assigned to every pixel including the masked ones. Certified recovery against patches has also been extended object detection, specifically to defend against patch hiding attacks. Two notable works in this direction are DetectorGuard (Xiang & Mittal 2021b), an extension of PatchGuard (Xiang et al., 202T) to object detection, and ObjectSecker (Xiang et al.) (022b). Randomized smoothing (Cohen et al. (019) has been applied to certify semantic segmentation models against fornorm bounded adversarial attacks (Fischer et al., (021). However, to the best of our knowledge, no certified defence against patch attacks for semantic segmentation has been proposed so far.
Certified detection. An alternative to certified recovery is certified detection. Here, an adversarial patch is allowed to change the model prediction. However, if it succeeds doing so, there is a mechanism that detects this attack certifiably with zero false negatives. Minority Reports (McCoyd ctt al., 2020) was the first certified detection method against patches, which is based on sliding a mask over the input in way that ensures that there will be one mask position that completely hides the patch. PatchGuard++ (Xiang & Mittal 20216) is an extension of Minority Reports where the sliding mask is not applied the input but the feature maps of BagNet-type feature extractor. This reduces inference time drastically since the feature extractor needs to be executed only once per input. ScaleCert (Han al. 2021) tries to identify "superficial important neurons", which allows pruning the network in a way that the prediction needs to be made for fewer masked inputs. Lastly, Patch Veto (Huang Li 2021) is a recently proposed method for certified detection that is tailored towards ViT models. It implements masking by removing certain input patches of the ViT. In this work, we propose a novel method for certified detection in the semantic segmentation task that can be used for any pretrained model.
Image reconstruction. The problem of learning to reconstruct the full image from inputs where parts have been masked out was pioneered by Vincent et al. (2010). It recently attracted attention as proxy task for self-supervised pre-training, especially for the ViTs (Bao et al. 2022 He et al. 202T). Recent approaches to this problem are using Fourier convolutions (Suvorov et al. 2022) and visual transformers (Dong et al.] 2022). SPG-Net (Song et al.] (018) trains a subnetwork to reconstruct the full semantic segmentation directly from the masked input a part of the image inpainting pipeline. In this work, we use the state-of-the-aae ZITS (Dong et al. 2022) inpainting method.
3 PROBLEM SETUP
3.1 SEMANTIC SEGMENTATION
In this work, focus the semantic segmentation task. Let X be set of rectangular images. Let x E X' be an image with height H. width W and the number of channels C. We denote y to be a finite label set. The goal is to find the segmentation map E > for æ. For each pixel æijj the
3

---

Published as a conference paper at ICLR 2023
corresponding label $i,j denotes the class of the object which Fi belongs. We denote & to be set of segmentation maps and f X S to be a segmentation model.
3.2 THREAT MODEL
Let us consider untargeted adversarial patch attack on a segmentation model. Consider an image E [0. 1 xC and its ground truth segmentation map s. Assume that the attacker can modify an arbitrary rectangular region of the image z which has a size of H' W' We refer to this modification as a patch. Let / E {0. 1) }}HxW be binary mask that defines the patch location in the image in which ones denote the pixels belonging
the patch. Let L be set of possible patch locations Let p E [0, 1 x xC be the modification itself. Then we define an operator A as A(x, p, 1) = (1 (1- @ the H' x W subregion of p defined by binary mask / to the image I while keeping the rest of the image unchanged. We denote P := [0, 1] 1| XWCC x L to be a set of all possible patch configurations (p, 1) that define an H' W" patch. Let S be the ground truth segmentation for Let (f) s) be some quality metric such as global pixel accuracy or mean intersection over union (mloU). The goal of an attacker is to find (p', 1*) (p", 1+) arg min Q(J(A(x,p,))) 1))))
for given image
I O p. where O is glement-wiss product. The operator A applies
(p. DEP
3.3 DEFENCE OBJECTIVE
In this paper, we propose certified defences against patch attacks. It means that we certify against any possible attack from p including (p", I*). We consider two robustness objectives.
Certified recovery For a pixel Fi,5 our goal is to verify that the following statement is true
A (p. 1) P: f(A(I, p. 1)), f(x)..)
(1)
Certified detection We consider a verification function e defined on X such that v(x) E {0, #x If u(z)s,) 1, then the adversarial patch attack on Izy be detected by applying the function to the attacked image x' = A(x, p..).
vx)); A (p. DEP.PPP.. uA(E, -1 f(A(a, p. M)s = f(x)as)
(2)
u(x')s,) = means an alert on pixel x. However, if x' not an adversarial example, then this is a false alert. In that case the fraction of pixels for which return false alert is called false alert ratio (FAR). The secondary objective is to keep FAR as small as possible.
Depending on the objective our goal is certify one of the conditions for each pixel si.j- This provides us an upper bound on an attacker's effectiveness under any adversarial patch attack from P.
4 DEMASKED SMOOTHING
DEMASKED SMOOTHING (Figure Tc) consists of several steps. First, we apply a predefined set of masks with specific properties the input image to obtain set of masked images. Then we recon- struct the masked regions of each image based on the available information with an inpainting model After that we apply segmentation model f to the demasked results. Finally, aggregate the segmen- tation outcomes and make a conclusion for the original image with respect to the statements (T) or (
4.1 INPUT MASKING
Motivation. Like in previous work (Section 2 we apply masking patterns to the input image and use predictions on masked images to aggregate the robust result. If an adversarial patch is completely masked, it has effect further processing. However, in semantic segmentation, a single whole-image label like the classification task, but a separate label for each pixel. Thus, making prediction masked image must allow to predict the labels also for the masked pixels.
predict not
Preliminaries. Consider an image x E [o, 1] [TXXXXG XC We define "*" to special masking symbol that does not correspond to any pixel value and has the property Vz IR x needs to be different from 0 since 0 a valid pixel value unmasked inputs. Let m E {*, 1) XXX xW be mask. We call the element- wise product & ④ masking of In masking, subset of
Please note that
4

---

Published as a conference paper at ICLR 2023
1
2
4
5
1
2
2
3
1
2
4
7
1
1
1
2
4
5
3
6
7
(a) original image
(b)
T
=
2
(c) column mask
(d) T 3
(c) 3-mask
3
3
7
9
8
1
4
( =
(g) 4-mask
(h) detection column
(i) detection row
Figure 2: examples of for the column masks: T = 2 (b. c). 3-mask: T 3 (d. e), and 4-mask: T
(f, g) with the number of masks K = 7.7.9 respectively. The number on a block denotes in which mask it is visible (there is only one such mask for each block). For each mask set, we show one of the locations / in which an adversarial patch (p, 1) affects T different maskings.
pixels becomes and the rest remains unchanged. We consider the threat model P with patches of size H' x W" (Section 3.2). To define the structure of our masks, break m into array B of non-intersecting blocks, each having the same size H' x W" as the adversarial patch. We index W We say that the block Blg, r] is visible
Blg. r). in a mask m if (i, 1) B r] mij 1 Consider an array M of K masks. We define each mask [[]] by a of blocks that are visible in it. For certified recovery, each block is visible in exactly one mask and masked in the others. We say that mask m is affected by a patch (p, 1) if A(z, p. 2) C m * O m. We define T(M) = max Hm E M AA(x, p. 1) @ m & x O ml|- That is: T(M) is the largest number of masks affected by some patch. If M is defined, refer to the value ( as T' for simplicity.
the blocks
q H <
Certified recovery. We define column masking M for which T 2. We assign every k-th block column to be visible in the mask [K] (Figure Any (p, () E P can intersect at most two adjacent columns since (p. 1) has the same width as a column. Thus, it can affect at most two masks (Figure 2b). A similar scheme can be proposed for the rows. Due to the block size in B, the patch (p, I) cannot intersect more than four blocks at once. We define a mask set that we call 3-mask s. t. for any four adjacent blocks two are visible in the same mask (Figures 2d]. Hence, patch for 3-mask can affect no more than masks, = 3. To achieve T = 4 any assignment of visible blocks to the masks works. We consider 4-mask that allows uniform coverage of the visible blocks in the image (Figure 21). See details on masking schemes in Appendix B
Certified detection. We define Ma to be set of masks for certified detection (we use subscript d for distinction). Md should have the property: (p.1) E P 3 m E Ma A(x, p. 1) O m = 20 @ m i. e. for every patch exists at least consider K = W W + 1 masks such that the mask Ma[k] masks a column of width W starting the horizontal position l: in the image (Figure 2h To obtain the guarantee for the same P with
mask not affected by this patch. For a patch of size H' W we
smaller K, we consider a set of strided columns of width W > W' and stride W" - W' + 1 that also satisfy the condition (see the proof adapted from Xiang al. (2022a) in Appendix A). A similar scheme can be proposed for the rows (Figure 21). Alternttvvly, we could use a set of block masks of size H' x W'. Then the number of masks grows quadratically with the image resolution. Hence, in the experiments we focus on the column and the row masking schemes.
Let g be demasking model, g( @ m) E [0, XX The goal of g to make the reconstruction g( ④ m) as close as possible (in some metric) to the original image z. For segmentation model f we define segmentation array S(M, g, f). S(M) == f(g(x o M(k)),, S S K.
4.2 CERTIFICATION
Certified recovery. For the threat model P consider a set M of K masks. We define a function h X S that assigns class to the pixel vi via majority voting over class predictions of each
5

---

Published as conference paper at ICLR 2023
Algorithm Demasked Smoothing
Input: image E [0, 10 xc patch size (H', W"), certification type CT (recovery or detection), mask type MT (column, row, 3-mask, 4-mask), inpainting model g, segmentation model f Output: segmentation map h E y certification (or verification) map E {0, 1)
I: M <- CreateMaskArray (H, W, H,',, W MT)
according to section 4.1
2: for t 1.-1,,... I,..., [M] do
this loop can be paralellized 3: S[k] t f(g(1 M())) mask input, inpaint the masked regions, and apply segmentation
4: end for
5: if CT 'recovery" then h Majority Vote(S)
over the classes predicted for each pixel > in detection case, output clean segmentation
6: else h + f(x)
7: end if
8: e f AllEqual(S,A))
> assign for the pixels where all S[k] agree with / and 0 otherwise
9: Return h,
original image
(b) masking
(c) demasking
segmentation
(c) certification
Figure 3: Reconstructing the masked images with ZITS Dong et al. 2002
reconstructed segmentation in S. A class for the pixel that is predicted by the largest number of segmentations is assigned. We break the ties by assigning a class with a smaller index.
Theorem 1. If the number of masks K satisfies K 2T(M) + 1 and for pixel Fış we have A SKK S:: S[k]aj M(x)ss
(i.e. all the votes agree), then (p. 1) P h(A(r,p,))u, h(x)..)
See the proof in Appendix A
Certified detection. Consider My [M][ For of demasked segmentations we define the verification map v(z),, := [f(r).,) = S'ikk = S[K]. i.c. the original segmentation equal all the other segmentations on masked-demasked inputs, including the one which the potential patch was completely masked.
Theorem 2. Assume that v(r).,) 1. Then
(p, 1) PPP: P: pA) =1 f(A(z, P. ()),, = f(x)
See the proof in Appendix A For given image x the verification map v(x) is complementary to the model segmentation output (xx) that stays unchanged. Thus, there is no drop in clean performance however we may have some false positive alerts the verification map v the clean setting. We present the Demasked Smoothing procedure in Algorithm
EXPERIMENTS
In this section, we evaluate DEMASKED SMOOTHING with the masking schemes proposed in Section compare our approach with the direct application of Derandomized Smoothing Levine Feizi (2020) to the segmentation task and evaluate the performance on different datasets and models. Certified recovery and certified detection provide certificates different strength (Section which not comparable. We evaluate them separately for different patch sizes.
5.1
EXPERIMENTAL SETUP
We evaluate DEMASKED SMOOTHING on two challenging semantic segmentation datasets: ADE20K (Zhou et al. 2017) (150 classes, 2000 validation images) and COCO-Stuff-108, (Caesar al. 2018

---

Published as a conference paper at ICLR 2023
(a) original image
(b) recovery map
(c) detection map
(d) ground truth
Figure 4: Certification maps for recovery and detection
(171 classes, 1000 validation images). For demasking
the ZITS Dong al. (2022) inpainting
model with the checkpoint provided in the official paper repository [ The model was trained on Places2 (Zhou et al. (2066) dataset with images resized to 256 x56x256 As segmentation model f we Bao et al. (2022). Swin Liu et al. (2021). PSPNet Zhao et al. (2017) and DeepLab v3 (Chen et 2018). We note that the first two models are based on transformers and obtain near state- of-the-art results. PSPNet and DeepL.bb v3 are CNN-based segmentation methods that we consider architectures. We
use BETT-B B
to demonstrate that DEMASKED SMOOTHING not specific to transformer-
use the model implementations provided in the mmsegmentation framework (Contributors (2020). An illustration of the image reconstruction and respective segmentation can be found in Figure 3 We run the evaluation in parallel on 5 Nvidia Tesla V V100-32GB GPUs. The certification for the whole ADE20K validation set with ZITS and BEiT-B takes around 1.2 hours for certified recovery and hours for certified detection (due to larger number of masks).
5.2 EVALUATION METRICS
For both certified recovery and certified detection, we generate standard segmentation output (without any abstention) and a corresponding certification map (Figure 4. In case of certified detection, the segmentation output remains the same as for the original segmentation model, however, there may be false alerts in the certificaton map. For the certified recovery, the output is obtained by a majority vote over the segmentations of demasked images (Section (4.2). We evaluate the mean intersection over union (mloU) for these outputs. The certification map is obtained by assigning to each certified pixel the corresponding class from the segmentation output and assigning a special uncertified label all non-certified pixels. For each image we evaluate the fraction of pixels which are certified and correct (coincide with the ground truth). %C is mean of these fractions over all the images in the dataset. In semantic segmentation task, the class frequencies are usually skewed, therefore global pixel-wise accuracy alone is an insufficient metric.
Matching the certification map separately for each class y y with the ground truth segmentation for y in the image æ allows us to compute the guaranteed lower bound (c)'P on the number of true positive pixel predictions (TPy(x)) i.e. those that were correctly classified into y. If pixel was certified with a correct class, then this prediction cannot be changed by a patch (or, alternatively, the change will be detected by the verification function in certified detection). We (a)
consider recall (x) = (FN where (x)( is the number of false negative predictions for y in x. (x(x) = (Py()) + FN, is the total area of y in the ground truth and does
depend on our prediction. We can evaluate certified recall cRy
(z(z) cTP,(x) a lower bound on
the recall Rg (z). Total recall and certified total recall of class y in a dataset are TRy
E.CD (P)(a) and (D) D Py Eeen Pa(x) respectively. Then, obtain mean recall mR(D) [ Eyer (D) and certified mean recall cmR(D) 13 Eyey CargeRRRl (D). Evaluating lower bounds on other popular metrics such as mean precision or mloU this way results in vacuous upper bound since they depend on the upper bound on false positive (FP) predictions. For the pixels that are not certified we cannot guarantee that they will not be assigned to a certain class, therefore, a non-trivial upper bound on PP1 is not straightforward. We leave this direction for future work. In certified detection, we additionally consider false alert ratio (FAR) which is the fraction of correctly classified pixels for which we return alert on a clean image. Smaller FAR is preferable.
Exer Pz(z)
7

---

Published as a conference paper at ICLR 2023
Table 1: Comparison of different masking schemes proposed in Section mloU mean intersection over union, mR mean recall, cmR certified mean recall. %C mean percentage of certified and correct pixels in the image. For detection, we provide clean mloU since the output unaffected and mean false alert rate (FAR) (lower is better). See additional results in Appendix
big
all
mode
dataset
segm
mask
mloU
mR
cmR
mR
cmR
%C
FAR +
column
57.33
32.55
63.55
20.04
ADE20K
BEIT-B row
53.08
70.92
64.45
detection
50.05
26.65
58.34
25.24
1% patch
column
56.86
26.80
47.09
21.43
COCOIOK
PSPNet
row 37.76
71.71
49.65
51.05
23.51
42.78
25.74
column
24.92
60.77
41.26
29.84
12.98
46.22
ADE20K
BEIT-B
row
16.33
46.91
16.72
19.51
4.83
31.71
3-mask
19.90
56.90
26.51
23.86
7.54
38.64
recovery
4 4-mask
18.82
52.96
23.75
22.56
5.87
34.36
N/A
0.5% patch
column 21.94
61.56
36.67
29.94
11.13
29.51
COCOIOK
PSPNet
row
18.87
58.04
20.90
26.16
6.14
19.31
3- 3-mask
18.82
59.26
29.00
25.85
7.56
25.21
4-mask
17.46
58.47
23.63
24.35
5.51
20.36
Table Comparison of our method with an adaptation of Levine Feizi (2020) to segmentation. We use Swin Liu et al. (222)) 200 ADE20K images. See details and illustrations in Appendix G
big
all
method
mloU
%C
mR
cmR
mR
cmR
Demasked Smoothing (our)
19.09
66.03
52.71
23.02
12.66
47.05
DRS-S DRS-E
0.42
11.35
9.08
1.04
0.83
28.01
9.12
54.67
41.78
11.04
7.86
45.03
Due to our threat model, certifying small objects in the scene can be difficult because they can be partially or completely covered by an adversarsial patch way that there is not chance recover the prediction. To provide an additional perspective on our methods, we also evaluate mR and emR specifically for the "big" classes, which occupy on average more than 20% of the images in which they appear. These are, for example, road, building, train, and sky, which are important for understanding the scene. The full list of such classes for each dataset provided the Appendix H
5.3 DISCUSSION
Evaluation of all the models with all the masking schemes consistent with these results and can be found in Appendix We see that column masking achieves better results in both certification modes. Effectiveness of column masking for classification task was also empirically observed by Levine & Feizi (2020). We attribute the effectiveness of column masking to the fact most of the images in the datasets have clear horizont line, therefore having visible column provides slice of the image that intersects most of the scene background objects.
In Table I we compare different masking schemes proposed in Section 4.1
In Table we extend Derandomized Smoothing (DRS) proposed by Levine & Feizi (0200 for certified recovery in classification to the segmentation task and compare it to our method Direct adaptation requires training model that able to predict the full image segmentation from a small visible region. Since it is not clear what architectural design and training procedure would be needed for that, we consider two alternative baselines. DRS-S predicts the segmentation directly from the masked image and DRS-E uses inpainting method to first reconstruct the image and then obtain the segmentation. See the implementation details in Appendix G DRS with column smoothing performs poorly the segmentation task, which emphasizes the need for specific masking schemes.
In Table 3 we evaluate our method with column masking on different models. For certified detection certify more than 60% of the pixels with all models ADE20K and more than 46 %
8

---

Published as a conference paper at ICLR 2023
Table 3: Demasked Smoothing results with column masking for different models
big
all
mode
dataset
segm
mloU
mR
cmR
mR
cmR
%C
FAR 4
BEiT-B
53.08
70.92
57.33
64.45
32.55
63.55
20.04
ADE20K
PSPNet
44.39
61.83
50.02
54.74
26.37
60.57
20.08
detection
Swin-B
48.13
68.51
55.45
59.13
29.06
61.44
20.31
1 % patch
PSPNet DeepLab v3
37.76
71.71
$6.86
49.65
26.80
47.09
21.43
COCO1OK
37.81
72.52
56.54
49.98
26.86
46.55
21.89
BEiT-B
24.92
60.77
41.26
29.84
12.98
46.22
ADE20K
PSPNet
19.17
51.90
34.11
23.66
10.76
44.90
recovery
Swin-B
22.43
59.75
34.88
27.09
11.70
46.14
N/A
0.5 %
PSPNet Deepl v3
21.94
61.56
36.67
29.94
11.13
29.51
COCO1OK
patch
23.12
62.60
33.84
31.59
11,55
28.71
mR
cmK
50
%6
(big)
40
40
%C
30
mR (big)
30
mloU
cmRg(big)
20
mioU
FAR
0.5
1.0
2.0
0.1
0.5
2.0
patch (%)
patch size (%)
(a) certified detection
(b) certified recovery
Figure 5: Performance for different adversarial patch sizes evaluated on 200 ADE20K images.
COCO1OK. False alert ratio correctly classified pixels is around 20%. In certified recovery, certify more than 44% pixels on ADE20K and more than 28% pixels on COCO10K. the patch size for the BEiTT- model.
how the performance of DEMASKED SMOOTHING depends We see that certified detection metrics remain high even for a patch as big as 5% of the image surface and for the recovery they slowly deteriorate as we increase the patch size to 2%. Ablations with respect to inpainting can be found in Appendix DEMASKEDSMOOTHING illustrations procedure are provided in Appendix K
5.4 LIMITATIONS
The performance of Demasked Smoothing certified recovery may be insufficient for the downstream task if we certify against big patches (Figure unless robustness is prioritized over clean performance. We point out that robustly segmenting small objects is fundamentally difficult under the adversarial patch threat model since the objects themselves can be completely or partially covered by an adversarial patch which makes impossible to properly segment them even for a human being. Demasked Smoothing certification requires an upper bound on the expected patch size (Section 4p.
6 CONCLUSION
In this work, we propose DEMASKED SMOOTHING, the first (up to our knowledge) certified defence framework against patch attacks on segmentation models. Due its novel design based on masking schemes and image demasking, DEMASKED SMOOTHING is compatible with any segmentation model and can on average certify 63% of the pixel predictions for 1 % patch the detection task and 46% against 0.5% patch for the recovery task on the ADE20K dataset.

---

Published as a conference paper at ICLR 2023
7 ETHICS STATEMENT
This work contributes to the field of certified defences against physically-realizabl adversarial attacks. The proposed approach allows certify robustness of safety-critical applications such as medical imaging or autonomous driving. The defence might be used to improve robustness of systems used for malicious purposes such as jemm-naauonomous weaponry or unauthorized surveillance. This danger may be mitigated e.g. by using system of sparsely distributed patches which makes certifying the image more challenging. activities our organization are carbon neutral, so the experiments performed our GPUs do not leave any carbon dioxide footprint.
REPRODUCIBILITY STATEMENT
We provide the details of our experimental setup in Section 5.1 We discuss the evaluation metrics and their computation in Section 5.2
ACKNOWLEDGEMENTS
We thank Chong Xiang for the suggestions on extending our evaluation protocol. Matthias Hein is member of the Machine Learning Cluster of Excellence, EXC number 2064/1 Project number 390727645 and acknowledges support of the Carl Zeiss Foundation in the project "Certification and Foundations of Safe Machine Learning Systems in Healthcare"
REFERENCES
Hangbo Bao, Li Dong, Songhao Piao, and Furu Wei. BEit: BERT pre-training of image transformers. In International Conference on Learning Representations (ICLR), 2022. URL
https:// http://eww.ee..tttooeeeeeee p BiZSz590
Walid Bousselham, Guillaume Thibault, Lucas Pagano, Archana Machireddy, Joe Gray, Young Hwan Chang, and Xubo Song. Efficient self-ensemble for semantic segmentation, 2021. URL https
7/arxiv. otg/ abs/ /0lbs/2110 2111. 13280
Wieland Brendel and Matthias Bethge. Approximating CNNs with bag--ur--cal--cctture models works surprisingly well on ImageNet. In International Conference Learning Representations (ICLR), 2019.
Tom Brown, Dandelion Mane, Aurko Roy, Martin Abadi, and Justin Gilmer. Adversarial patch. In Advances Neural Information Processing System (NeurIPS), 2017. URL htt.hh:tt///arttll org/pdff//// http//ff/11111/ 11712. 09665. pdf arXiv: 1712.09665.
Holger Caesar, Jasper Uijlings, and Vittorio Ferrari. Coco-stuff: Thing and stuff classes in context. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018.
siang-Chieh Chen, Yukun Zhu, George Papandreou, Florian Schroff, and Hartwig Adam. Encoder- decoder with atrous separable convolution for semantic image segmentation. In ECCV, 2018.
Zhaoyu Chen, Bo Li, Jianghe Xu, Shuang Wu, Shouhong Ding, and Wenqiang Zhang Towards practical certifiable patch defense with vision transformer. CVPR, 2022.
Ping-yeh Chiang, Renkun Ni, Ahmed Abdelkader, Chen Zhu, Chris Studor, and Tom Goldstein. Certified defenses for adversarial patches. In International Conference on Learning Representations (ICLR), 2020.
Jeremy Cohen, Elan Rosenfeld, and Zico Kolter. Certified adversarial robustness via randomized smoothing. In Proceedings of Machine Learning Research, pp. 1310-1320, 2019, URL http:
//proceedings.co ml wwwpsprspppee.ee. httpppppeeee///eeee///000 html
MMSegmentation Contributors. MMSeggmentation: Openmmlab semantic segmentation toolbox and 2020.
benchmark. htttps:://i http:/ggttttbub http:///ww.....oommm////////eeee
10

---

Published as a conference paper at ICLR 2023
Qiaole Dong, Chenjie Cao, and Yanwei Fu. Incremental transformer structure enhanced image inpainting with masking positional encoding. IEEE/CVE Conference on Computer Vision and Pattern Recognition (CVPR), 2022.
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. image is worth 16x16 words: Transformers for image recognition at scale. In International Conference Learning Representations (ICLR), 2021. httpss //openreview.
abF dNTTY
Marc Fischer, Maximilian Baader, and Martin T. Vechev. Scalable certified segmentation via randomized smoothing. In ICMI, 2021.
Sven Gowal, Krishnamurthy (Di) Dvijotham, Robert Stanforth, Rudy Bunel, Chongli Qin, Jonathan Uesato, Relja Arandjelovic, Timothy Mann, and Pushmeet Kohli. Scalable verified training for provably robust image classification. In International Conference on Computer Vision (ICCV), October 2019.
Husheng Han, Kaidi Xu, Xing Hu, Xiaobing Chen, Ling Liang, Zidong Du, Qi Guo, Yanzhi Wang, and Yunji Chen. Scalecert: Scalable certified defense against adversarial patches with sparse superficial layers. In Advances Neural Information Processing Systems (NeurIPS), 2021.
Jamie Hayes.
visible adversarial perturbations digital watermarking. In IEEE Conference Computer Vision and Puttern Recognition Workshops (CVPR), 2018. URL http::://ppeacceesss thecvf. com/ content evpr 2018 workshops/w32// html Hayes On Visible Adversarial CVPR 2018 paper paper.html
Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Dollár, and Ross Girshick. Masked autoencoders are scalable vision learners, 2021. arXiv:211100631
Yuheng Huang and Yuanchun Li. Zero-shot certified defense against adversarial patches with vision transformers, 2021. arXiv:2111.10483
Danny Karmon, Daniel Zoran, and Yoav Goldberg. LaVAN: Localized and visible adversarial noise. In International Conference on Machine Learning (ICML), pp. 2507-2515, 2018. URL
mwr. v80 karmon 1 8a html
Mark Lee and J. Zico Kolter. On physical adversarial patches for object detection. International Conference on Machine Learning (Workshop), 2019. URL httpttttt:/aaaae wwwpw///wggga/aa/aeoeoooo /ttt:://araa...ogaaab////1 11897
Alexander Levine and Soheil Feizi. Randomind Smoothing for Certifiable Defense against Patch Attacks. In Advances in Neural Information Processing Systems (NeurIPS), volume 33, 2020.
Chu-Tak Li, Wan-Chi Siu, Zhi-Song Liu, Li-Wen Wang, and Daniel Pak- Kong Lun. Deepgin: Deep generative inpainting network for extreme image inpainting, 2020.
Wan- Yi Lin, Fatemeh Sheikholeslami, jinghao shi, Leslie Rice, and J Zico Kolter. Certified robustness against physically-realizabl patch attack via randomized cropping, 2021. URL https:/ //
openreview net forum? www.oruuwwiiittooooo.oo VV
Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. Swin transformer: Hierarchical vision transformer using shifted windows. In International Conference on Computer Vision (ICCV), 2021.
Giulio Lovisotto, Nicole Finnie, Mauricio Munoz, Chaithanya Kumar Mummadi, and Jan Hendrik Metzen. Give me your attention: Dot-product attention considered harmful for adversarial patch robustness. CVPR, 2022.
Michael McCoyd, Won Park, Steven Chen, Neil Shah, Ryan Roggenkemper, Minjune Hwang, Jason Xinyu Liu, and David Wagner. Minority reports defense: Defending against adversarial patches, 2020. arXiv:2004.33799
11

---

Published as a conference paper at ICLR 2023
Jan Hendrik Metzen and Maksym Yatsura. Efficient certified defenses against patch attacks image classifiers. In International Conference on Learning Representations (ICLR), 2021. URL
htttp::/cpperevvee
net/ Forummid-dii 3PMV-pPPMDpppp
Muzammal Nascer, Salman Khan, and Fatih Porikli. Local gradients smoothing: Defense against localized adversarial attacks. In JEEE Winter Conference Applications of Computer Vision (WACV), 2019. WtRLbttps:/// /ooi org/10. 109///CCCC 2019. 2019.00143
Federico Nesti, Giulio Rossolini, Saasha Nair, Alessandro Biondi, and Giorgio C. Buttazzo. Evaluat- ing the robustness of semantic segmentation for autonomous driving against real-world adversarial patch attacks. 2022 IEEE/CVE Winter Conference Applications of Computer Vision (WACV), pp. 2826-28225555 2022.
Hadi Salman, Saachi Jain, Eric Wong, and Aleksander Madry, Certified patch robustness via smoothed vision transformers, 2021. arXiv:2110.07719
Ramprasaath R. Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra. Grad-CAM: Visual Explanations from Deep Networks via Gradient-Basse Localization. International Journal of Computer Vision, October 2019. ISSN 1573-1405.
Yuhang Song, Chao Yang, Yeji Shen, Peng Wang, Qin Huang, and C.-C. Jay Kuo. Spg-net: Segmen- tation prediction and guidance network for image inpainting. In BMVC, 2018.
Roman Suvorov, Elizaveta Logacheva, Anton Mashikhin, Anastasia Remizova, Arsenii Ashukha, Aleksei Silvestrov, Naejin Kong, Harshith Goka, Kiwoong Park, and Victor S. Lempitsky. Resolution-robust large mask inpainting with fourier convolutions. 2022 IEEE/CVF Winter Conference Applications of Computer Vision (WACV), pp. 3172-3182, 2022.
Pascal Vincent, Hugo Larochelle, Isabelle Lajoie, Yoshua Bengio, and Pierre-Antoine Manzagol. Stacked denoising autoencoders: Learning useful representations in deep network with local denoising criterion. Journal of Machine Learning Research, 1 11(110)33371-444 2010. URL
http:// http:///wwm...rg//pppeessssss/// htw.appersslll..ieeeltlll.c
Tong Wu, Liang Tong, and Yevgeniy Vorobeychik. Defending against physically realizable attacks on image classification. In International Conference on Learning Representations (ICLR), 2020.
URL httppss//accl...rr/aaaasss////// wttp:/iwww..rr//////...... 09552
Chong Xiang and Prateek Mittal. Detectorguard: Provably securing object detectors against localized patch hiding attacks. In ACM Conference on Computer and Communications Security (CCS), 2021a.
Chong Xiang and Prateek Mittal. Patchguard++:: Efficient provable attack detection against adversarial patches, 2021b. arXiv:2104 12609.
Chong Xiang, Arjun Nitin Bhagoji, Vikash Sehwag, and Prateek Mittal, Patchguard: A provably robust defense against adversarial patches via small receptive fields and masking. In 30th USENIX Security Symposium (USENIX Security), 2021.
Chong Xiang, Saeed Mahloujifar, and Prateek Mittal. Patchcleanser: Certifiably robust defense against adversarial patches for any image classifier. In 31st USENIX Security Symposium (USENIX Security), 2022a.
Chong Xiang, Alexander Valtchanov, Saeed Mahloujifar, and Prateek Mittal. Objectseeker: Cer- tifiably robust object detection against patch hiding attacks via patch-agnostic masking, 2022b. arXiv:2202.01811
Zhanyuan Zhang, Benson Yuan, Michael McCoyd, and David Wagner. Clipped BagNet: Defending Against Sticker Attacks with Clipped Bag-of-features In 3rd Deep Learning and Security Workshop (DLS), 2020.
Hengshuang Zhao, Jianping Shi, Xiaojuan Qi, Xiaogang Wang, and Jayy Jia. Pyramid scene parsing network. In IEFF/CVF Conference Computer Vision and Pattern Recognition (CVPR), 2017.
12

---

Published as a conference paper at ICLR 2023
Bolei Zhou, Aditya Khosla, Agata Lapedriza, Antonio Torralba, and Aude Oliva. Places: An image database for deep scene understanding. Journal of Vision, 17, 10 2016. doi: 10.1167/17.10.266
Bolei Zhou, Hang Zhao, Xavier Puig, Sanja Fidler, Adela Barriuso, and Antonio Torralba. Scene parsing through ade20k dataset. In Proceedings of the IEEE Conference on Computer Vision and Puttern Recognition, 2017.
13

---

Published as a conference paper at ICLR 2023
A PROOFS (SECTION 4
In this section, we provide the proofs for the theorems stated in Section 4
Lemma 1. (Section 4.1) Consider image of the size H x W. Let H' x W' be fixed adversarial patch size, Let Md be a set of K masks where each mask is masking an H x W vertical column, wT 2 W' Let the stride between the columns in two adjacent masks be W W + 1. Then for any location I E L of the patch, there exists a mask that covers it completely.
Proof. (Adapted from the proof of Lemma 4 in PatchCleanser Xiang et al. (2022a). Without loss of generality, we consider the first two adjacent column masks. The first one covers the columns from to W The second mask covers the columns from 1 - (W" - W + W W' WW' +2 to (W" W + 2) + (W" 1) = 2W" W' (See Figure 6 Now consider an adversarial patch of size H' x W'. Let us find the smallest possible start index of this patch so that it does not get covered by the first mask. For that should be visible at the column W" + 1 and, therefore, start at the column with index not smaller than (W" + 1) W + W W' + 2. However, it is the same column in which second mask starts. Therefore, given that W 2 W we have that the patch is completely masked by the second mask. Then for a patch which is only partially masked by the second mask from the left we use an analogous argument to show that it is completely masked by the third mask and so on.
Certified recovery. For the threat model P (Section consider set M of K masks. We define function h : X' + S that assigns a class to the pixel Viy via majority voting over class predictions of each reconstructed segmentation in S. A class for the pixel that is predicted by the largest number of segmentations is assigned. We break the ties by assigning a class with a smaller index.
Theorem (Section 4.2) If the number of masks K satisfies K 2TM and for pixel mi. we have
V STK]] S: STA|L h(z)as i.e. all the votes agree, then (p. 1) P:: MA(x, = h(x)ss
Proof. We prove the statement by contradiction. Assume that
(p, I) P:P: ((A(x,p,))) + h())as
Let us denote x' == A(x, p. and S' to be the segmentation array for x'. We denote the class h(z)s, predicted for the pixel Fi, as C. h(A(I, p. 1))..) * C means that the class C did not get the majority in the votes over the segmentation array S" However, by definition of T(M) we know that (p. 1) could affect at most T(M) segmentations out of and change their vote. Since all K segmentations of S have voted for C, then at least K T(M) of them are still voting for C in St. And by our assumption K 2 (M( + 1, we have that K ((M) 2 T(M) + 1. Thus, the class C for æıj still gets the majority vote in S'. Therefore h(x²),, a.j C h(z))..- We have arrived to contradiction.
W
2W" W +
W" W' + +2
Figure 6: The masked columns of the first two adjacent masks (blue for the first one and red for the second onc). If the patch is not completely masked by the first mask, it should be visible at the column 4" + (the masked part of the patch is dark-grey and the visible part is in light-grey). However then the patch will be completely masked by the second mask.
14

---

Published as a conference paper at ICLR 2023
Cortified rocovery
segmentations pixel st.j
clean majority vate
output h(x)).
clean input imaage
masking, demasking, segmentation
atlacked segmentations pixel AI.
attacked majority vote
oulput hfx Uu
attacked input image x'
Figure 7: Illustration for Theorem with K
masks and = 2. majority voting across five masks is univocal for the clean image, the patch can only affect two out of five segmentations and, thus, cannot shift the majority. The green shade schematically shows which parts were masked in the masking step.
Certified detection
segmentation pixel xru
clean
input image
masking, de masking, segmentation
attacked segmentation pixel Xij
attacked
Input
Image.x
alert
There exists at least one segmentation for which the patch was masked. If the patch has managed to affect the segmentations in which it is not masked, there will be an inconsistency. The green shade schematically shows which parts were masked the masking step.
Figure 8: Illustration for Theorem 2
A schematic illustration for the certified detection mechanism is provided in Figure
Certified detection. Consider My {Ma[k)) For of demasked segmentations we define the verification map v(z),, := [f(x),,) = SiT,, .... = S[K], i.e. the original segmentation coincides with all the other segmentations including the one in which the potential patch was completely masked.
Theorem (Section Assume that u(r)a,) Then
(p, (p. : = = f(A(z, D)). = f(x)
Proof. We prove the statement by contradiction. Assume that
(p, 1) P: v(A(z,p. = S(A(z, D))s + f(x)ss
Let denote x' A(x, p,1) and 5' be the segmentation set for s'. By definition of Ma. Malk] E Ma s. t. Ma[k] masks the patch (p, I) Hence,
g( @ Ma(k)) g(x @ Ma(k)),
Sik] f(g(= Ma(k)) f(g(a' o Ma(k))) S'(k),
Since v(z)4, 1, we have f()), S[k],, Since v( 1, we have f f(x')s) S'(k)e Thus. f(x'),, = fx)). We have arrived to contradiction.
schematic illustration for the certified recovery mechanism is provided in Figure
15

---

Published as a conference paper at ICLR 2023
4
5
1
3
4
5
1
3
4
5
4
5
5
(a) original image
(b)
T
2
(c) column mask
2
3
4
5
5
6
7
1
1
2
8
9
7
8
3
4
2
3
1
N
7
6
(d) T =
(e) 3-mask
(f) = 4
(g) 4-mask
Figure 9: (a) examples mask for the column masks with T = 2 (b, c). 3-mask with T' 3 (d, c). and 4-mask with T (f. g) with the number of masks K = 5, 7,9 respectively. The number of block denotes in which mask it is not masked (there only one such mask for each block). For each mask set, we show of the locations/ / in which adversarial patch affects T different maskings.
B DETAILED DESCRIPTION OF MASKING STRATEGIES
In this section, provide additional details for constructing certified recovery masks proposed in Section [4.]] We define mask sets M that satisfy different values T. We divide the image x into a set of non-intersecting blocks B of the same size adversarial patch, H' W (see Figure 9 1 I q S [H/H", H", < S [W/W]] In each mask, each of these blocks will be either masked or not masked (i. visible). Moreover, for each block there exists only mask in which it is visible. For set M of K masks we define the mapping μM B - {l...., K}. If u(Blg, (g, r)) = k. then B[g, r] is not masked in [k]. Therefore, each mask M [k] is defined by B₁ B s. for b Bş (bb) k.
We define set M that we call 3-mask for which T(M)
We assign the blocks in each row the 5]) ([1 3
masks as follows: (1 1]) = 1: μ (B[1 2)) = (B[[1, 3)) = 2; p ([[[ 4])
and so on until we reach the end of the row. If we finish the first row with the value k, then we start
the second row as follows y [2, 1])
[B[22, 2)) = k + 1; 1 B[2, 3)) n B 4)) = 1 + 2: If we finish the second row on n, we start the third row similarly to the first: p(B[3, 1]) = n +
([B(3, 2) = [3, 3]) = + 2: ... When we reach the number K, we start from again (Figure 9d). Due to the block size, the patch cannot intersect more than four blocks at once. Our murit-alltrnating block sequence ensures that in any such intersection of four blocks either the top ones or the bottom ones will belong to the same masking, so at most three different maskings can be affected.
We define a set M that we call 4-mask for which T(M) = 4. Due to our block size any assignment of masks will work because the patch cannot intersect more than four blocks. We consider the that allows uniform distribution of the unmasked blocks (Figure (g). We point out that for the described methods each masking keeps approximately 1/K of the pixels visible and the unmasked regions are uniformly distributed in the image. This means that for any masked pixel there exists an unmasked region located close enough to this pixel. It is the core difference between our masks and the ones proposed for certified classification such as block or column smoothing Levine & Feizi] (2020). It was observed that the image demasking is facilitated when the visible regions are uniformly spread in the masked image He et al. (2021).
C TEST-TIME INPUT CERTIFICATION
In this section, we discuss how certified recovery (Theorem I can be applied to guaranteed verification of the robustness can be evaluated by using a dataset of clean images such as ADE20K (Zhou et al. 2017) or COCO- Stuff-10K (Caesar et al. 2018).
test image. We also discuss how robustness guarantees for the test-time images
16

---

Published as a conference paper at ICLR 2023
C.I TEST-TIME CERTIFIED RECOVERY
Let x' be test-time input which can be either a clean image or an image attacked with an adversarial patch. We know that there exists a clean image J corresponding to x' which removes the patch if present. We have either x' = x' E A(r), where A(a)) == (A(x.p.1) (D,1) E P). However, at test time we do not have access to the clean image æ.
Our goal is to certify that for our segmentation model h and pixel Fi we have h(=')i,; h(x)).) We can achieve this result by applying the recovery certification (Theorem
D to the lest-time image. It allows to verify whether V (p. 2) E P h( M(A(x', p,,)),, h(x"),,, ) We also know that if x' E A(x), then x E A(x') (Figure 10a). Indeed,
is only different from x by one patch, then x can be be obtained from x' by removing this patch. Therefore, by obtaining the guarantee for A(x'). we implicitly ohtain the guarantee also for the image x even though we do not have direct access to it.
We note that this test-time guarantee is only possible for certified recovery. In certified detection, would need to evaluate the verification function (Theorem 2 for both the clean image x and the attacked image x' obtain the result. This cannot be done if implicit.
C.2 ROBUSTNESS GUARANTEES EVALUATION
The typical certified robust error for given test data set (and pixel (i,j) in the segmentation case) is an estimate for
Ex~Dl max IMAA(XP-D)):H/XX))... (p,))P
where D is the data generating probability measure and we assume that test set to be i.i.d. sample of it. This is the expected robust error (worst case over our threat model for clean inputs) for given pixel (i. Using the test sample to get estimate of this quantity, we get probabilistic guarantee that the corresponding pixel (i.j) of new clean test sample x' drawn ii.d. from P will have its whole "patch" "patch"-neighborhhn neighborhood certified.
However, more important for practical security analysis is that we can certify a given instance, which can be even potentially adversarially perturbed. Formally, this means that for an input z E A(x), where r ~ P is an unknown sample from P. that we guarantee
(p, E P h(A(2, L,j = h(2)a,)
and E A(z, 1) this implies that we certify that the pixel (i, of the potentially manipulated image is classified the same as pixel (f,j) of the unperturbed image
However, is now tricky to get even a probabilistic estimate the quantity
E
max [ max 1 M(A(..p.))..l. (p.))P (g.))e
as the outer maximization process cannot be simply simulated by doing adversarial patch attacks clean test dataset.
We propose a way to evaluate guaranteed lower bound on the fraction of certified test-time inputs by using a dataset of clean images. Instead of considering a standard one one-patch neighbourhood
A(z) defined by our threat model (Section 3.2p, we propose to consider a neighbourhood A (2(x)
of two independent patches (Figure 10b). (() contains all the images x' E A(x) as well as
their respective patch neighbourhoods A(x'). ). Therefore, by verifying that V (p1, h1, (pz. 12)
P : h(A(A(z, x. Pl, 41,, P2, 12)) = h(*)ij we guarantee that x' E A(z) V (p. 1) E h((Aa', p, I)) h(x')a.-
We note that corresponding reasoning could be applied to certification in lp models. Then A² would correspond to doubling the radius the e-ball instead of adding a second patch.
Note that Theorem I can be directly extended to threat model of N patches. In the worst case each of the patches can affect different maskings. Therefore, we need to change the condition of Theorem to K 2 2NT + 1. We apply the described method to evaluating the test-time certification guarantees for a toy example of a 0.1% patch in Table We also illustrate how a column mask looks in this case in Figure 10
17

---

Published as a conference paper at ICLR 2023
A(X)
A(3)
A00
x
(a)
(b)
(C)
(d)
Figure 10:
(a) certified inference; (b) double adversarial neighbourhood: (c) original image (d) certification against two patches
Table 4: Inference recovery robustness estimate. To illustrate our point we certify example for
0. 0.1% patch.
big
all
dataset
segm
mask
mloU
%C
mR
cmR
mR
cmR
ADE20K
19.73
36.95 69.63
16.64
24.23
9.24
41.96
BETT-B
col
COCO10K
26.36
35.34
34.92
11.13
28.17
D ADVERSARIAL PATCH EXAMPLE
In this section, we demonstrate an example of a real adversarial patch for a semantic segmentation model similar to the one illustrated in the Figure Ia and show how it is handled by certified defences. We illustrate for the Swin (Liu et al. 2021) model on one of the images from the ADE20K (Zhou et al. 2017) dataset.
D.1 PATCH OPTIMIZATION
We set the patch size to 1% of the image surface. We select fixed position for patch on the rear window of a car (Figure Tla. For each pixel we extract a list of predicted logits corresponding each class and apply multi-margin loss with respect the ground truth label of the respective pixel. We use random patch initialization without restarts. As an optimizer we use projected gradient descent (PGD) with 1000 steps and initial step size of 0.01. We use cosine step size schedule and momentum for the gradient with the rate of 0.9. The optimization plot and the patch efficiency at different iterations the PGD illustrated in the Figure
D.2 CERTIFIED RECOVERY
We denote the original image as r and the patched image x'. The voting-based segmentation function h (Section 4.2) provides the majority-vote prediction h(z) and the corresponding certification map which shows the pixels where all the votes agree. In Figure we see that part of the building and the road is certified which means that this prediction cannot be affected by an adversarial patch. Figure demonstrates h(x²) which correctly segments those regions in presence an adversarial patch that fools the original model.
D.3 CERTIFIED DETECTION
We perform our analysis by evaluating the verification map (Section 4.2) for the original image x and for the patched image x'. We see that in v(z) major part of the building certified i. e. for a part of pixels æi, that belong the building and the road we have v(x),,) 1. However, u(z'),, = for those pixels. It means that we have detected that the prediction on this input is potentially affected by adversarial patch.
18

---

Published as a conference paper at ICLR 2023
mult, www.amaaaanaaaa.
loss value
200
10
600
800
1000
step
(a) clean image
(b) patched image (patch marked)
(c) optimization plot
(d) clean segmentation
(c) 300 PGD steps
(f) 1000 PGD steps
Figure 11: Patch attack illustration with Swin Liu et al. 2021 and an ADE20K image. A patch
occupying 1% of the image surface changes the segmentation.
(a) segmentation of the (b) certification map of the (c) segmentation of the (d) certification map of the original image, 1(x) original image
patched image, 1(x²) patched image x'
Figure 12: Certified recovery for % patch used the attack. The majority vote function
recovers the prediction in presence of adversarial patch that fools the undefended model. The segmentation for the original and patched image in (a) and (c) are the same for the regions certified in the certification maps (b) or (d). The certification maps (b) and (d) are also almost the same.
E ADDITIONAL EXPERIMENTS
In Tables and 6 provide additional experimental results for evaluating different masking schemes proposed in Section 4.1 on different models.
F INPAINTING ABLATION STUDIES
We perform ablation studies with respect to the demasking step. The results are in Table Figure 14 provides additional illustrations. As can be seen from the results, our method heavily benefits from having available stronger inpainting models that allow achieving better clean and certified accuracy. We consider this property actually strength of our method since it will automatically benefit from future research and developments of stronger inpainting methods. For certified recovery, we also compare it to GIN Li et al. (2020) based on generative model that trained ADE20K (without using style losses based on ImageNet trained VGG). The results are in Table g Illustrations can be found in Figure 15
19

---

Published as a conference paper at ICLR 2023
Table 5: The certified detection results(%) for patch occupying no more than 1% of the image. mloU mean intersection over union, mR mean recall, cmR certified mean recall. %C mean percentage of certified and correct pixels in the image.
big
all
dataset
segm
mask
mloU
mR
cmR
mR
cmR
%C
col
61.15
35.88
65.44
BEiT-L
56.33
74.26
68.40
row
52.77
30.25
60.48
PSPNet col
50.02
26.37
60.57
row 44.39
61.83
54.74
ADE20K
42.44
19.88
54.62
Swin col
55.45
29.06
61,44
48.13
68.51
59.13
row
47.21
22.04
55.93
col
56.86
26.80
47.61
PSPNet
37.76
71.71
49.65
COCO10K
row
51.05
23.51
43,40
56.54
26.86
47.17
DeepLab v3
37.81
72.52
49.98
row
50.51
23.89
43.19
Table 6: The certified recovery results against 0.5% patch. 3-mask and 4-mask correspond to 3 and T respectively (Figure
big
all
dataset
segm
mask
mIoU
%C
mR
cmR
mR
cmR
col
28.64
71.95
50.84
34.65
16.04
47.76
BETT-L
row
18.82
53.77
21.24
22.74
5.95
32.30
3-mask
22.40
64.83
33.96
26.89
8.97
39.59
4-mask
19.93
60.90
25.03
24.22
6.43
35.01
col
19.17
51.90
34.11
23.66
10.76
44.90
ADE20K
PSPNet
row
12.00
36.26
12.03
15.03
3.74
28.29
3-mask
15.00
44.93
19.55
18.41
5.58
35.85
4-mask
12.74
40.41
15.86
15.87
4.14
31.22
col
22.43
59.75
34.88
27.09
11.70
46.14
13.58
42.88
15.13
16.70
4.46
30.64
Swin
3-mask
17.06
51.03
24.15
20.74
6.65
38.27
4-mask
14.77
46.67
17.74
18.05
4.72
34.04
col
21.94
61.56
36.67
29.94
11.13
29.51
PSPNet
row
18.87
58.04
20.90
26.16
6.14
19.31
3-mask
18,82
59.26
29.00
25.85
7.56
25.21
4-mask
17.46
58.47
23.63
24.35
5.51
20.36
COCO1OK
col
23.12
62.60
33.84
31.59
11.55
28.71
DeepLab v3
row
20.04
55.71
17.80
27.89
6.28
17.04
3-mask
20.14
58.02
27.14
27.82
8.05
24.30
4-mask
19.35
58.22
22.01
26.74
5.79
19.38
20

---

Published as a conference paper at ICLR 2023
Table 7: Comparison for demasked smoothing with and without demasking step. mloU mean intersection over union, mR mean recall, cmR certified mean recall. %C mean percentage of certified and correct pixels in the image. We Swin model on 200 ADE20K images with column masking for certified detection and certified recovery. We compare masking the columns with solid black color without demasking ZITS demasking.
patch size
big
all
mode
demasking
mloU
%C
mR
cmR
mR
cmR
/
58,85
23.35
62.89
detection 1.0%
X 38.56
67.25
53.37
19.49
3.09
21.19
/
19.09
66.03
52.71
23.02
12.66
47.05
recovery
0.5%
X 1.10
15.09
7.71
1.79
0.72
18.59
Table 8:
Comparison of our two demasking methods: ZITS and GIN. mloU mean intersection over union, mR mean recall, emR certified mean recall. %C mean percentage of certified and correct pixels in the image. We use Swin model on 200 ADE20K images.
big
all
demasking
trained on
mloU
%C
mR
cmR
mR
cmR
ZTTS Dong al. 2022
Places2 ADE20K
19.09
66.03
52.71
23.02
12.66
47.05
GIN
(Li et al 2020)
5.46
32.27
19.05
7.62
3.52
32.08
Table 9: Comparison of different inpainting models: LAMA Suvorov et al. 2022 and ZITS Dong al. (022). mloU- mean intersection union, mR mean recall, cmR certified mean recall. %C mean percentage of certified and correct pixels in the image. For detection, we provide clean mloU since the output is unaffected and mean false alert rate (FAR) (lower better).
big
all
mode
mask
demasker
mloU
%
FAR
mR
cmR
mR
cmR
ZITS
57.33
32.55
63.55
20.04
column
53.08
70.92
64.45
detection
LAMA
56.99
31.67
64.21
19.37
1 1%
ZITS
50.05
26.65
58.34
25,24
patch
TOW
LAMA
53.08
70.92
49.06
64.45
26.58
59.21
24.38
ZITS
24.92
60.77
41.26
29.84
12.98
46.22
column
LAMA
22.48
58.20
37.51
26.49
11.49
45.95
ZITS
16.33
46.91
16.72
19.51
4.83
31.71
recovery
row
LAMA
15.64
43.07
16.51
18.78
4.95
32.84
N/A
0.5%
ZITS
19.90
56.90
26.51
23.86
7.54
38.64
3-mask
patch
LAMA
18.54
53.59
27.39
22.12
7.58
39.52
ZITS
18.82
52.96
23.75
22.56
5.87
34.36
4-mask
LAMA
17.00
50.60
18.18
20.22
5.23
35.98
21

---

Published as a conference paper at ICLR 2023
(a) ((x) (original)
(b) v(æ) (original verified) (c) o(2') (patched verified)
(d) f(x') (patched)
Figure 13: f is segmentation model (Swin Liu et al. (2021) and is the verification function
(Section 4.2). For attacked image x' u(x') detects the region of f(x') which was (potentially)
affected by an adversarial patch.
(a) original
(b) detection col (c) segmentation
(d) recovery col (e) segmentation
(f) ground truth
Figure 14: Results without image demasking. The solid color inpainting is treated as a separate object
in the scene because need to classify every pixel in semantic segmentation task. Therefore, it hard to achieve a situation where all the demasked segmentation agree on some pixel which is represented in the Table
G COMPARISON TO SIMPLIFIED DERANDOMIZED SMOOTHING
Derandomized Smoothing (DRS) Levine & Feizi 2020) was proposed for certified recovery, therefore in this section we focus on this task. Direct adaptation of derandomized smoothing to semantic segmentation task requires training model that is able to predict the full image segmentation from small visible region. Since is not immediately clear to what architectural design and training procedure would be needed to train such model, we consider simplified version of DRS that we call DRS-S. In this version, we consider an off-the-shelf semantic segmentation model and evaluate how it performs with column masking from DRS. Therefore, do not encode the masked regions with the special 'UULL' value like DRS but use black color instead. That is because an off-the-shelf model cannot work with 'NULL' values.
We run our experiments on ADE20K dataset. We consider the DRS parameters from the recent SOTA version of Derandomized Smoothing by Salman al. Salman et al. (202I). They column width b= 19 and stride & 10 for certified classification of 224x224 ImageNet images. To account for the fact that ADE20K images have larger resolution than ImageNet, scale the parameters column width b = 42 and stride 22. To make the comparison consistent with the rest of our results, we use the patch occupying 0.5% of the image.
From Table 10 we can see that DRS-S performs poorly on semantic segmentation task. The reason for that illustrated in Figure T6 Processing the column region in T6c would probably be sufficient for classification model to classify the image into the class "house". But it is clearly not sufficient reconstruct the whole segmentation map I6e as can be seen the Figure [6g Whether doing this would be possible with model specifically trained to reconstruct the segmentation map from very small visible region an open research question (up to our knowledge).
We point out that the value %C of certified and correctly classified pixels in the Table 10 is still surprisingly high for DRS-S compared to other metires. We attribute this to the fact that the solid black regions are usually treated as wall by the segmentation model, therefore the images segmented as a wall by the DRS majority voting. And the wall is a common part of both indoor and outdoor scenes in ADE20K as be implied from the Table of "big" ADE20K classes. Therefore, always classifying the output as a wall provides decent fraction of correctly classified pixels because of the skewed classes.
usually
22

---

Published as a conference paper at ICLR 2023
(a) original
(b) mask
(c) gin
(d)
(e) ground truth
(f) gin segm
(g) zits segm
Figure 15: Comparison between GIN and ZITS inpainting
original
(b) mask
mask
mask
(c) ground truth
(f) segmentation
(g) segmentation
(h) segmentation
Figure 16
However, to provide a better comparison with DRS, we emulate the model which able to reconstruct the whole segmentation map from the column masking proposed in DRS. We do this by applying the demasking approach proposed in this work. We first try to reconstruct the whole image from one column and then segment with off-the-shelf model paper. We call this approach DRS-E E and the results can be found in Table 10
did with the masks proposed this
Table 10: Comparison of our method with simplified Derandomized Smoothing Levine & Feizi (2020). mloU mean intersection over union, mR mean recall, cmR certified mean recall. %C mean percentage of certified and correct pixels in the image. We use Swin model on 200 ADE20K images.
big cmR
all
method
mIoU
%C
mR
mR
cmR
Demasked (our)
19.09
66.03
52.71
23.02
12.66
47.05
DRS-S
0.42
11.35
9.08
1.04
0.83
28.01
DRS-E
9.12
54.67
41.78
11.04
7.86
45.03
23

---

Published as a conference paper at ICLR 2023
Table 11: The list of 19 "big" classes for ADE20K (Zhou et al. 2017) (out of 150 classes in total) with their average fraction of occupied pixels in the images where they are present (%) and index in the list of dataset classes. We define a class be "big" it occupies on average more than 20% of the pixels in the images in which this class appears.
index
name
fraction
index
name
fraction
wall
25.88
11
79
hovel
25.93
building
32.36
12
88
booth
23.91
3
sky
21.54
13
96
escalator
20.96
bed
21.25
14
103
ship
26.81
21
water
22.10
15
104
fountain
28.81
29
field
22.97
16
107
washer
22.07
46
sand
21.22
17
109
swimming pool
28.87
8
48
skyscraper
42.92
18
114
tent
34.57
54
runway
28.05
19
128
lake
34.57
10
55
case
37.57
Table 12: The list of "big" classes for
(Caesar al., 2018) (out of 171 classes
in total) with their average fraction of occupied pixels in the images where they are present (%) and index the list of dataset classes. We define class be "big" if occupies on average more than 20% of the pixels in the images in which this class appears.
#
index
name
fraction
index
name
fraction
1
6
bus
21.46
11
105
floor-stone
20.10
2
7
train
23.11
12
111
fruit
20.48
3
20
cow
24.17
13
113
grass
23.25
4
21
elephant
28.50
14
134
playingfield
38.64
5
49
sandwich
23.99
15
137
river
40.01
6
51
broccoli
20.18
16
143
sand
26.37
7
54
pizza
25.86
17
144
36.51
8
60
bed
36.86
18
146
sky-other
22.94
9
61
dining table
21.71
19
148
snow
51.60
10
95
clouds
24.11
20
1599
vegetable
20.35
21
167
water-other
21.67
H A LIST OF BIG CLASSES
we suggest another perspective the evaluation of our DEMASKED SMOOTHING by specifically considering its performance on "big" semantic classes. The object of these classes occupy on average more than 20% of the images in which they appear. Correctly segmenting these classes is important for understanding the scene. In Tables I1 and 12 we provide the full list of such classes in ADE20K (Zhou al. 2017) and COCO-Stuff-100 (Caesar al. 2018) respectively together with the average fraction of pixels that they occupy in the images in which they are present. We point out that for COCO-Stuff- 10K some typically smaller classes such as "sandwich" or "fruit" get included in the list of big classes because of the macro-scale images in which they occupy big part of the scene.
In Section 5.2
T COMPLEXITY ANALYSIS AND PARALLELIZATION
In DEMASKED SMOOTHING, propose set of K masks that are applied to the original image (denote the cost applying a single masking by M). As illustrated in Figure Ic the masked images demasked (denote the cost of demasking image by D) and segmented (denote the cost of segmenting an image by S); thereupon per-mask segmentations are aggregated into a final segmentation and certification (cost of aggregation proportional to K). Asymptotically, compute
24

---

Published as a conference paper at ICLR 2023
grows thus with O(K(M + + K) while the cost of standard segmentation is O(S). Thus, for large or M D > 5, real-time applicability would actually be impractical. However, we note that:
1. M D is roughly of the same size S for typical DL--based inpainting and segmentation models.
2. For certified recovery, we operate in setting where K small (K E 5, 7, 9) and does not grow with the image resolution. This is unlike Derandomized Smoothing and its derivatives, where the number of masks in the recovery task grows with the image resolution (or randomized smoothing with thousands of samples per input). This small value of K benefits our the method in time-sensitive applications. For certified detection, we can adjust the number of masks for the computational speed by using strided masking suggested in Section 4.1
3. Moreover, masking, demasking, and segmenting for different masks do not use any shared data and can thus be fully parallelized if sufficiently powerful hardware is available. Only the aggregation step requires the results of all the previous stages. However, aggregation time small compared to the other stages. terms of latency, fully parallelized version of our procedure would thus have a latency proportional to O(M + D + $ + K). For small and M + D S, application real-time video can be facilitated by means of parallelization.
J USED DATA
In this work, we only use the datasets published under formal licenses: ADE20K (Zhou et al. 2017) and COCO-Stuff-luee 10K (Caesar al. 2018). To the best of our knowledge, data used this project do not contain any personally identifiable information offensive content. The models ZITS (Dong al. 2022) and Swin (Liu al., 202T are published under 2pa license. The text of the
license for PSPNet (Zhao al. 2017) can be found here: hctps: /ttess//gittu..tt com haanm/hsaaa /
PSPNet/blole #SPNet/bloo/matte:// http///aatttiieeeee
K DEMASKED SMOOTHING VISUALIZATION
In this section, we provide additional illustrations our method (Figures 17
18
20p. Similarly
to the Table we certify against 1 % patch for the detection task and against 0.5% patch for the recovery task. For each mask type we illustrate all the stages summarized in the Figure [c provide examples of certification maps for certified recovery and certified detection with different
We also
images (Figure 21 22
25

---

Published as a conference paper at ICLR 2023
(a) input
(b) segmentation
certification
(d) ground truth
Figure 17: DEMASKED SMOOTHING detection column masking illustration for an image from ADE20K (Zhou et al. 2017). We illustrate five masks of twenty.
26

---

Published as a conference paper at ICLR 2023
(a) input
(b) segmentation
certification (d) ground truth
Figure 18: DEMASKED SMOOTHING recovery column masking illustration for an image from ADE20K (Zhou et al. 2017).
27

---

Published as a conference paper at ICLR 2023
(a) imput
(b) segmentation
(c) certification
(d) ground truth
Figure 19: DEMASKED SMOOTHING recovery masking for T = 3, K = masks (Section 4.1 illustration for image from ADE20K.
28

---

Published as a conference paper at ICLR 2023
input
(b) segmentation
(c) certification
(d) ground truth
Figure 20: DEMASKED SMOOTHING recovery masking for T 4, K masks (Section 4.1
illustration for an image from ADE20K (Zhou et al. 2017).
29

---

Published as a conference paper at ICLR 2023
(a) original image
(b) recovery map
(c) detection map
(d) ground truth
Figure 21: Certification map examples on ADE20K Zhou et 2017 with ZITS Dong et al. 2022 and Swin Liu et al. 2021).
30

---

Published as a conference paper at ICLR 2023
(a) original image
(b) recovery map
(c) detection map
(d) ground truth
Figure 22: Certification map examples ADE20K Zhou et al. 2017 with ZITS Dong al. 2022 and Swin Liu et al. (2021).
31


Reviews:
[{"review_id": "Rp35c2VA4G", "reviewer": "Reviewer_XUU4", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper presents a new method to certify the robustness of semantic segmentation models against adversarial patch attacks without extra training.\n", "summary_of_the_review": "It implements certified defense under patch attacks on semantic segmentation. \n", "strength_and_weaknesses": "Overall, the paper is interesting and extend the certification for semantic segmentation. \n\nStrength:\n1) This paper is well-written. \n2) This is the first method to certifiably defend against adversarial patch attack on semantic segmentation.  \n3) This method can be applied on any off-the-shelf segmentation model without finetuning or any other adaptation.\n\nWeakness:\n1. One minor weakness is that this paper did not apply to the STOA transformer based methods such as Segformer. Hope to see the performance on Segformer. \n2. Hope the author can provide the code to check the reproducibility of this paper. \n\n[1] SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers.  ", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "PbkEvz0yDk", "reviewer": "Reviewer_F6Ro", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The work provides their \"DEMASKED SMOOTHING\" approach, which the authors argue provide defence framework against patch attacks for segmentation models. They evaluate their model on semantic segmentation datasets ADE20k and COCO-Stuff-10K. They use models trained on existing benchmarks, evaluated on various masking schemes. The authors discuss the results and approach compared to baselines in Section 5.", "summary_of_the_review": "Overall the work is well organized, described clearly and makes an original contribution of reasonable significance. The empirical results look promising, although it seems authors skipped some detailed on limitations and expanding theoretical claims.", "strength_and_weaknesses": "Strength:\n* The work is well organized, clear to follow with no major grammatical errors\n* The empirical evaluation seems reasonable, along with the prior work.\n* The results on the benchmarked datasets seem promising.\n\n\nWeaknesses:\n* The theoretical grounding is not clearly explained in the main paper.\n* Limitations are not clearly described, evaluation limited to semantic segmentation domain\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "FrZLcnUf77", "reviewer": "Reviewer_TG16", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper proposed Demasked Smoothing (DS), a certified defense against adversarial patch attacks on semantic segmentation. The empirical performance outruns baseline certified defenses such as DRS.", "summary_of_the_review": "Sufficient novely on certified defense for semantic segmentation, though the certification seems rather restrictive (e.g., requiring very high consistency of image recovery rather than majority voting). More discussions on the working mechanism, limitations, and additional comparisons are needed.", "strength_and_weaknesses": "While to my knowledge I agree that this paper is one of the first certified defenses proposed for semantic segmentation, I have several questions about the current presentation of the results (see the following sections).", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "3CdrDgaVJ-s", "reviewer": "Reviewer_FoVC", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The manuscript addresses adversarial patch attacks for segmentation models. The proposed approach extends previous work on certified patch robustness for image-wide models. It is capable both of certified detection and certified recovery and it can be applied to any semantic segmentation model. The main idea is to apply a set of K masks to the input image, inpaint the void areas with ZITS, and then to detect or undo adversarial patch attack by analyzing all predictions at the given pixel.\n", "summary_of_the_review": "The reviewer perceives moderate novelty, tough reading and several weaknesses.", "strength_and_weaknesses": "Strengths\n1. adversarial robustness is an important problem\n2. a previous image-wide approach [salman21arxiv] has been extended for the case of dense prediction.\n\nWeaknesses\n1. This approach is likely going to strongly diminish semantic accuracy at small objects.\n2. The majority of input pixels is inpainted. The manuscript does not seem to consider opportunities of attacking the inpainting algorithm either through adversarial patches or through data poisoning. \n3. The second to last sentence of the proof to theorem 1 is not convincing. \n4. Unclear implied guarantees of certification. Consider the example from Figure 18. An adversarial patch has opportunity to affect only T=4 of K=9 masks. Still, the model is going to make mistakes in the remaining K-T=5 masks. Could these mistakes lead to adversarial patch pass unnoticed?\n5. Unclear relation with respect to adversarial training. Would it make sense to combine these two approaches?\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Decision: Accept: poster