Original: Certiﬁed Defences Against Adversarial Patch Attacks on Semantic Segmentation
Revised: Certiﬁed Defences Against Adversarial Patch Attacks on Semantic Segmentation

Original: Abstract
Revised: Abstract

Original: Adversarial patch attacks are an emerging security threat for real world deeplearning applications.
Revised: Adversarial patch attacks are an emerging security threat for real world deep learning applications.

Original: We present D EMASKED S MOOTHING , the ﬁrst approach (upto our knowledge) to certify the robustness of semantic segmentation models againstthis threat model.
Revised: We present D EMASKED S MOOTHING , the ﬁrst approach (up to our knowledge) to certify the robustness of semantic segmentation models against this threat model.

Original: Previous work on certiﬁably defending against patch attackshas mostly focused on image classiﬁcation task and often required changes in themodel architecture and additional training which is undesirable and computationallyexpensive.
Revised: Previous work on certiﬁably defending against patch attacks has mostly focused on image classiﬁcation task and often required changes in the model architecture and additional training which is undesirable and computationally expensive.

Original: In D EMASKED S MOOTHING , any segmentation model can be appliedwithout particular training, ﬁne-tuning, or restriction of the architecture.
Revised: In D EMASKED S MOOTHING , any segmentation model can be applied without particular training, ﬁne-tuning, or restriction of the architecture.

Original: Usingdifferent masking strategies, D EMASKED S MOOTHING can be applied both forcertiﬁed detection and certiﬁed recovery.
Revised: Using different masking strategies, D EMASKED S MOOTHING can be applied both for certiﬁed detection and certiﬁed recovery.

Original: In extensive experiments we show that
Revised: In extensive experiments we show that

Original: D EMASKED S MOOTHING can on average certify 63% of the pixel predictions for a1% patch in the detection task and 46% against a 0.5% patch for the recovery taskon the ADE20K dataset.
Revised: D EMASKED S MOOTHING can on average certify 63% of the pixel predictions for a 1% patch in the detection task and 46% against a 0.5% patch for the recovery task on the ADE20K dataset.

Original:  Introduction
Revised:  Introduction

Original: Physically realizable adversarial attacks are a threat for safety-critical (semi-)autonomous systemssuch as self-driving cars or robots.
Revised: Physically realizable adversarial attacks are a threat for safety-critical (semi-)autonomous systems such as self-driving cars or robots.

Original: Adversarial patches [1, 2] are the most prominent example ofsuch an attack.
Revised: Adversarial patches [1, 2] are the most prominent example of such an attack.

Original: Their realizability has been demonstrated repeatedly, for instance by Lee and Kolter[3] : an attacker places a printed version of an adversarial patch in the physical world to fool a deeplearning system.
Revised: Their realizability has been demonstrated repeatedly, for instance by Lee and Kolter [3]: an attacker places a printed version of an adversarial patch in the physical world to fool a deep learning system.

Original: While empirical defenses [4–7] may offer robustness against known attacks, it doesnot provide any guarantees against unknown future attacks [8].
Revised: While empirical defenses [4–7] may offer robustness against known attacks, it does not provide any guarantees against unknown future attacks [8].

Original: Thus, certiﬁed defenses for the patchthreat model, which allow guaranteed robustness against all possible attacks for the given threatmodel, are crucial for safety-critical applications.
Revised: Thus, certiﬁed defenses for the patch threat model, which allow guaranteed robustness against all possible attacks for the given threat model, are crucial for safety-critical applications.

Original: Research on certiﬁable defenses against adversarial patches can be broadly categorized into certiﬁedrecovery and certiﬁed detection.
Revised: Research on certiﬁable defenses against adversarial patches can be broadly categorized into certiﬁed recovery and certiﬁed detection.

Original: Certiﬁed recovery [8–16] has the objective to make a correctprediction on an input even in the presence of an adversarial patch.
Revised: Certiﬁed recovery [8–16] has the objective to make a correct prediction on an input even in the presence of an adversarial patch.

Original: In contrast, certiﬁed detection[17–20] provides a weaker guarantee by only aiming at detecting inputs containing adversarialpatches.
Revised: In contrast, certiﬁed detection [17–20] provides a weaker guarantee by only aiming at detecting inputs containing adversarial patches.

Original: While certiﬁed recovery is more desirable in principle, it typically comes at a high cost ofreduced performance on clean data.
Revised: While certiﬁed recovery is more desirable in principle, it typically comes at a high cost of [Figure] reduced performance on clean data.

Original: In practice, certiﬁed detection might be preferable because itallows maintaining high clean performance.
Revised: In practice, certiﬁed detection might be preferable because it allows maintaining high clean performance.

Original: Most existing certiﬁable defenses against patches arefocused on image classiﬁcation.
Revised: Most existing certiﬁable defenses against patches are focused on image classiﬁcation.

Original: DetectorGuard [21] and ObjectSeeker [22] that certiﬁably defendagainst patch hiding attacks on object detectors.
Revised: DetectorGuard [21] and ObjectSeeker [22] that certiﬁably defend against patch hiding attacks on object detectors.

Original: Moreover, existing defences are not easily applicableto arbitrary downstream models, because they assume either that the downstream model is trainedexplicitly for being certiﬁably robust [9, 12], or that the model has a certain network architecture suchas BagNet [10, 12, 11] or a vision transformer [15, 20].
Revised: Moreover, existing defences are not easily applicable to arbitrary downstream models, because they assume either that the downstream model is trained explicitly for being certiﬁably robust [9, 12], or that the model has a certain network architecture such as BagNet [10, 12, 11] or a vision transformer [15, 20].

Original: PatchCleanser [14], which can be combinedwith arbitrary downstream models but is restricted to image classiﬁcation.
Revised: PatchCleanser [14], which can be combined with arbitrary downstream models but is restricted to image classiﬁcation.

Original: [Figure] Adversarial patch attacks were also proposed for the image segmentation problem [23], mostlyfor attacking CNN-based models that use a localized receptive ﬁeld [24].
Revised: Adversarial patch attacks were also proposed for the image segmentation problem [23], mostly for attacking CNN-based models that use a localized receptive ﬁeld [24].

Original: However, recently selfattention based vision transformers [25] have achieved new state-of-the-art in the image segmentationtask [26, 27].
Revised: However, recently self-attention based vision transformers [25] have achieved new state-of-the-art in the image segmentation task [26, 27].

Original: Their output may become more vulnerable to adversarial patches if they manage tomanipulate the global self-attention [28].
Revised: Their output may become more vulnerable to adversarial patches if they manage to manipulate the global self-attention [28].

Original: We demonstrate how signiﬁcant parts of the segmentationoutput may be affected by a small patch for Swin tranfromer [26] in Figure 1a. Full detailson theattack and defending against it with our method are available in Appendix E.
Revised: We demonstrate how signiﬁcant parts of the segmentation output may be affected by a small patch for Swin transformer [26] in Figure 1a (see details in Appendix E).

Original: We point out thatpreventive certiﬁed defences are important because newly developed attacks can immediately be usedto compromise safety-critical applications unless they are properly defended.
Revised: We point out that preventive certiﬁed defences are important because newly developed attacks can immediately be used to compromise safety-critical applications unless they are properly defended.

Original: In this work, we propose the novel framework D EMASKED S MOOTHING (Figure 1c) to obtainthe ﬁrst (up to our knowledge) certiﬁed defences against patch attacks on semantic segmentationmodels.
Revised: In this work, we propose the novel framework D EMASKED S MOOTHING (Figure 1c) to obtain the ﬁrst (up to our knowledge) certiﬁed defences against patch attacks on semantic segmentation.

Original: Similarly to previous work [9], we mask different parts of the input (Figure 1b) and provideguarantees with respect to every possible patch that is not larger than a certain pre-deﬁned size.
Revised: Similarly to previous work [9], we mask different parts of the input (Figure 1b) and provide robustness guarantees.

Original: While prior work required the classiﬁcation model to deal with such masked inputs, we leveragerecent progress in image inpainting [29] to reconstruct the input before passing it to the downstreammodel.
Revised: While prior work required the classiﬁcation model to deal with such masked inputs, we leverage recent progress in image inpainting [29] to reconstruct the input before passing it to the downstream model.

Original: This decouplingof image demasking from the segmentation task allows us to support arbitrarydownstream models.
Revised: This decoupling allows us to support arbitrary downstream models.

Original: Moreover, we can leverage state of the art methods for image inpainting.
Revised: Moreover, we can leverage state of the art methods for image inpainting.

Original: Wealso propose different masking schemes tailored for the segmentation task that provide the denseinput allowing the demasking model to understand the scene but still satisfy the guarantees withrespect to the adversarial patch.
Revised: We also propose different masking schemes tailored for the segmentation task that provide the dense input allowing the demasking model to understand the scene but still satisfy the guarantees with respect to the adversarial patch.

Original: We summarize our contributions as follows:
Revised: We summarize our contributions as follows:

Original: • We propose D EMASKED S MOOTHING which is the ﬁrst (to the best of our knowledge)certiﬁed recovery or certiﬁed detection based defence against adversarial patch attacks onsemantic segmentation models (Section 4).
Revised: • We propose D EMASKED S MOOTHING which is the ﬁrst (to the best of our knowledge) certiﬁed recovery or certiﬁed detection based defence against adversarial patch attacks on semantic segmentation models (Section 4).

Original: • D EMASKED S MOOTHING can do certiﬁed detection and recovery with any off-the-shelfsegmentation model without requiring ﬁnetuning or any other adaptation.
Revised: • D EMASKED S MOOTHING can do certiﬁed detection and recovery with any off-the-shelf segmentation model without requiring ﬁnetuning or any other adaptation.

Original: • We implement D EMASKED S MOOTHING , evaluate it for different certiﬁcation objectivesand masking schemes (Section 5).
Revised: • We implement D EMASKED S MOOTHING , evaluate it for different certiﬁcation objectives and masking schemes (Section 5).

Original: We can certify 63% of all pixels in certiﬁed detection fora 1% patch and 46% in certiﬁed recovery for a 0.5% patch for the BEiT-B [30] segmentationmodel on the ADE20K [31] dataset.
Revised: We can certify 63% of all pixels in certiﬁed detection for a 1% patch and 46% in certiﬁed recovery for a 0.5% patch for the BEiT-B [30] segmentation model on the ADE20K [31] dataset.

Original:  Related Work
Revised:  Related Work

Original: Certiﬁed recovery.
Revised: Certiﬁed recovery.

Original: The ﬁrst certiﬁed recovery defence against patches was proposed by Chianget al.
Revised: The ﬁrst certiﬁed recovery defence against patches was proposed by Chiang et al.

Original: [8] for classiﬁcation models .
Revised: [8] for classiﬁcation models .

Original: De-Randomized Smoothing (DRS)
Revised: De-Randomized Smoothing (DRS)

Original: [9] signiﬁcantly improved certiﬁed accuracy.
Revised: [9] signiﬁcantly improved certiﬁed accuracy.

Original: Models with small receptive ﬁelds such as BagNets [32] were adopted for this taskeither by combining them with some ﬁxed postprocessing [10, 11] or by training them end-to-endfor certiﬁed recovery [12].
Revised: Models with small receptive ﬁelds such as BagNets [32] were adopted for this task either by combining them with some ﬁxed postprocessing [10, 11] or by training them end-to-end for certiﬁed recovery [12].

Original: DRS was also applied [15] to Vision Transfomers (ViTs) [25].
Revised: DRS was also applied [15] to Vision Transfomers (ViTs) [25].

Original: In contrastto these works, our Demasked Smoothing can be applied to models with arbitrary architecture.
Revised: In contrast to these works, our Demasked Smoothing can be applied to models with arbitrary architecture.

Original: PatchCleanser [14] has this property as well but it is limited to image classiﬁcation.
Revised: PatchCleanser [14] has this property as well but it is limited to image classiﬁcation.

Original: Certiﬁed recoveryagainst patches has also been extended to object detection to defend against patch hiding attacks[18, 22].
Revised: Certiﬁed recovery against patches has also been extended to object detection to defend against patch hiding attacks [18, 22].

Original: Randomized smoothing [33] has been applied to certify semantic segmentation modelsagainst (cid:96) 2 -norm bounded adversarial attacks [34].
Revised: Randomized smoothing [33] has been applied to certify semantic segmentation models against (cid:96) 2 -norm bounded adversarial attacks [34].

Original: However, to the best of our knowledge, no certiﬁeddefence against patch attacks for semantic segmentation has been proposed so far.
Revised: However, to the best of our knowledge, no certiﬁed defence against patch attacks for semantic segmentation has been proposed so far.

Original: Certiﬁed detection.
Revised: Certiﬁed detection.

Original: In this alternative to certiﬁed recovery, an adversarial patch is allowed to changethe model prediction.
Revised: In this alternative to certiﬁed recovery, an adversarial patch is allowed to change the model prediction.

Original: However, if it succeeds in doing so, the attack is certiﬁably detected.
Revised: However, if it succeeds in doing so, the attack is certiﬁably detected.

Original: Minority
Revised: Minority

Original: Reports [17] was the ﬁrst certiﬁed detection method against patches.
Revised: Reports [17] was the ﬁrst certiﬁed detection method against patches.

Original: PatchGuard++ [18] is hassigniﬁcantly improved the inference time.
Revised: PatchGuard++ [18] is has signiﬁcantly improved the inference time.

Original: ScaleCert [19] uses “superﬁcial important neurons” todatect an attack.
Revised: ScaleCert [19] uses “superﬁcial important neurons” to datect an attack.

Original: Lastly, PatchVeto [20] implements masking by removing certain input patches of the
Revised: Lastly, PatchVeto [20] implements masking by removing certain input patches of the

Original: ViT. In this work, we propose a novel method for certiﬁed detection in the semantic segmentation.
Revised: ViT. In this work, we propose a novel method for certiﬁed detection in the semantic segmentation.

Original: Image reconstruction.
Revised: Image reconstruction.

Original: The problem of learning to reconstruct the full image from masked inputswas pioneered by Vincent et al.
Revised: The problem of learning to reconstruct the full image from masked inputs was pioneered by Vincent et al.

Original: It recently attracted attention as proxy task for self-supervisedpre-training, especially for the ViTs [30, 36].
Revised: It recently attracted attention as proxy task for self-supervised pre-training, especially for the ViTs [30, 36].

Original: Recent approaches to this problem are using Fourierconvolutions [37] and ViTs [29].
Revised: Recent approaches to this problem are using Fourier convolutions [37] and ViTs [29].

Original: SPG-Net [38] trains a subnetwork to reconstruct the full semanticsegmentation directly from the masked input as a part of the image inpainting pipeline.
Revised: SPG-Net [38] trains a subnetwork to reconstruct the full semantic segmentation from the masked input.

Original: In this work,we use the state-of-the-art ZITS [29] inpainting method.

Original:  Problem Setup
Revised:  Problem Setup

Original: Semantic segmentation .
Revised: Semantic segmentation .

Original: In this work, we focus on the semantic segmentation task.
Revised: In this work, we focus on the semantic segmentation task.

Original: Let X be a setof rectangular images.
Revised: Let X be a set of rectangular images.

Original: Let x ∈ X be an image with height H , width W and the number of channels
Revised: Let x ∈ X be an image with height H , width W and the number of channels

Original: C .
Revised: C .

Original: We denote Y to be a ﬁnite label set.
Revised: We denote Y to be a ﬁnite label set.

Original: The goal is to ﬁnd the segmentation map s ∈ Y H × W for x .
Revised: The goal is to ﬁnd the segmentation map s ∈ Y H × W for x .

Original: For each pixel x i,j , the corresponding label s i,j denotes the class of the object to which x i,j belongs.
Revised: For each pixel x i,j , the corresponding label s i,j denotes the class of the object to which x i,j belongs.

Original: We denote S to be a set of segmentation maps and f : X → S to be a segmentation model.
Revised: We denote S to be a set of segmentation maps and f : X → S to be a segmentation model.

Original: Threat model .
Revised: Threat model .

Original: Let us consider an untargeted adversarial patch attack on a segmentation model.
Revised: Let us consider an untargeted adversarial patch attack on a segmentation model.

Original: Consider an image x ∈ [0 , 1] H × W × C and its ground truth segmentation map s .
Revised: Consider an image x ∈ [0 , 1] H × W × C and its ground truth segmentation map s .

Original: Assume that theattacker can modify an arbitrary rectangular region of the image x which has a size of H (cid:48)
Revised: Assume that the attacker can modify an arbitrary rectangular region of the image x which has a size of H (cid:48)

Original: × W (cid:48) .
Revised: × W (cid:48) .

Original: We refer to this modiﬁcation as a patch .
Revised: We refer to this modiﬁcation as a patch .

Original: Let l ∈ { 0 , 1 } H × W be a binary mask that deﬁnes thepatch location in the image in which ones denote the pixels belonging to the patch.
Revised: Let l ∈ { 0 , 1 } H × W be a binary mask that deﬁnes the patch location in the image in which ones denote the pixels belonging to the patch.

Original: Let L be aset of all possible patch locations for a given image x .
Revised: Let L be a set of all possible patch locations for a given image x .

Original: Let p ∈
Revised: Let p ∈

Original: [0 , 1] H × W × C be the modiﬁcationitself.
Revised: [0 , 1] H × W × C be the modiﬁcation itself.

Original: We deﬁne an operator
Revised: We deﬁne an operator

Original: A ( x, p, l ) =
Revised: A ( x, p, l ) =

Original: (1 − l ) (cid:12) x + l (cid:12) p , where (cid:12) is element-wise product.
Revised: (1 − l ) (cid:12) x + l (cid:12) p , where (cid:12) is element-wise product.

Original: The operator A applies the H (cid:48)
Revised: The operator A applies the H (cid:48)

Original: × W (cid:48) subregion of p deﬁned by a binary mask l to the image xwhile keeping the rest of the image unchanged.
Revised: × W (cid:48) subregion of p deﬁned by a binary mask l to the image x while keeping the rest of the image unchanged.

Original: We denote P :=
Revised: We denote P :=

Original: [0 , 1] H × W × C × L to be a setof all possible patch conﬁgurations ( p, l ) that deﬁne an H (cid:48)
Revised: [0 , 1] H × W × C × L to be a set of all possible patch conﬁgurations ( p, l ) that deﬁne an H (cid:48)

Original: × W (cid:48) patch.
Revised: × W (cid:48) patch.

Original: Let s ∈ S be the groundtruth segmentation for x and Q ( f ( x ) , s ) be some quality metric.
Revised: Let s ∈ S be the ground truth segmentation for x and Q ( f ( x ) , s ) be some quality metric.

Original: The attacker’s goal is to ﬁnd( p (cid:63) , l (cid:63) ) =
Revised: The attacker’s goal is to ﬁnd ( p (cid:63) , l (cid:63) ) =

Original: arg min ( p, l ) ∈P Q ( f ( A ( x, p, l )) , s ) .
Revised: arg min ( p, l ) ∈P Q ( f ( A ( x, p, l )) , s ) .

Original: In this paper, we propose certiﬁed defences againstany possible attack from P including ( p (cid:63) , l (cid:63) ) .
Revised: In this paper, we propose certiﬁed defences against any possible attack from P including ( p (cid:63) , l (cid:63) ) .

Original: We consider two robustness objectives.
Revised: We consider two robustness objectives.

Original: Certiﬁed recovery.
Revised: Certiﬁed recovery.

Original: For a pixel x i,j our goal is to verify that the following statement is true
Revised: For a pixel x i,j our goal is to verify that the following statement is true

Original: [Equation]
Revised: [Equation]

Original: Certiﬁed detection.
Revised: Certiﬁed detection.

Original: We deﬁne a veriﬁcation function v : X → { 0 , 1 } H × W .
Revised: We deﬁne a veriﬁcation function v : X → { 0 , 1 } H × W .

Original: If v ( x ) i,j = 1 , then theadversarial patch attack on x i,j can be detected by applying v to the attacked image x (cid:48)
Revised: If v ( x ) i,j = 1 , then the adversarial patch attack on x i,j can be detected by applying v to the attacked image x (cid:48)

Original: =
Revised: =

Original: A ( x, p, l ) .
Revised: A ( x, p, l ) .

Original: [Equation]

Original: ⇒

Original: [Equation]
Revised: [Equation]

Original: v ( x (cid:48) )
Revised: v ( x (cid:48) )

Original: i,j = 0 means an alert on pixel x (cid:48)
Revised: i,j = 0 means an alert on pixel x (cid:48)

Original: i,j .
Revised: i,j .

Original: However, if x (cid:48) is not an adversarial example, then this is afalse alert.
Revised: However, if x (cid:48) is not an adversarial example, then this is a false alert.

Original: In that case the fraction of pixels for which we return false alert is called false alert ratio(FAR).
Revised: In that case the fraction of pixels for which we return false alert is called false alert ratio (FAR).

Original: The secondary objective is to keep FAR as small as possible.
Revised: The secondary objective is to keep FAR as small as possible.

Original: Depending on the objective our goal is to certify one of the conditions 1, 2 for each pixel x i,j .
Revised: Depending on the objective our goal is to certify one of the conditions 1, 2 for each pixel x i,j .

Original: Thisprovides us an upper bound on an attacker’s effectiveness under any adversarial patch attack from P .
Revised: This provides us an upper bound on an attacker’s effectiveness under any adversarial patch attack from P .

Original:  Demasked Smoothing
Revised:  Demasked Smoothing

Original: D EMASKED S MOOTHING (Figure 1c) consists of several steps.
Revised: D EMASKED S MOOTHING (Figure 1c) consists of several steps.

Original: First, we apply a predeﬁned setof masks with speciﬁc properties to the input image to obtain a set of masked images.
Revised: First, we apply a predeﬁned set of masks with speciﬁc properties to the input image to obtain a set of masked images.

Original: Then wereconstruct the masked regions of each image based on the available information with an inpaintingmodel g .
Revised: Then we reconstruct the masked regions of each image based on the available information with an inpainting model g .

Original: After that we apply a segmentation model f to the demasked results.
Revised: After that we apply a segmentation model f to the demasked results.

Original: Finally, we aggregatethe segmentation outcomes and make a conclusion for the original image with respect to the statements(1) or (2).
Revised: Finally, we aggregate the segmentation outcomes and make a conclusion for the original image with respect to the statements (1) or (2).

Original: See Algorithm 1 in Appendix B.
Revised: See Algorithm 1 in Appendix B.

Original: 4.1 Input masking
Revised: 4.1 Input masking

Original: Motivation.
Revised: Motivation.

Original: Like in previous work (Section 2) we apply masking patterns to the input image and usepredictions on masked images to aggregate the robust result.
Revised: Like in previous work (Section 2) we apply masking patterns to the input image and use predictions on masked images to aggregate the robust result.

Original: If an adversarial patch is completelymasked, it has no effect on further processing.
Revised: If an adversarial patch is completely masked, it has no effect on further processing.

Original: However, in semantic segmentation, we predict nota single whole-image label like in the classiﬁcation task, but a separate label for each pixel.
Revised: However, in semantic segmentation, we predict not a single whole-image label like in the classiﬁcation task, but a separate label for each pixel.

Original: Thus,making prediction on a masked image must allow us to predict the labels also for the masked pixels.
Revised: Thus, making prediction on a masked image must allow us to predict the labels also for the masked pixels.

Original: Preliminaries.
Revised: Preliminaries.

Original: Consider an image x ∈ [0 , 1] H × W × C .
Revised: Consider an image x ∈ [0 , 1] H × W × C .

Original: We deﬁne " ∗ " to be a special masking symbolthat does not correspond to any pixel value and has the property ∀ z ∈ R : z × ∗ = ∗ .
Revised: We deﬁne " ∗ " to be a special masking symbol that does not correspond to any pixel value and has the property ∀ z ∈ R : z × ∗ = ∗ .

Original: Note that ∗needs to be different from 0 since 0 is a valid pixel value in unmasked inputs.
Revised: Note that ∗ needs to be different from 0 since 0 is a valid pixel value in unmasked inputs.

Original: Let m ∈ {∗ , 1 } H × Wbe a mask .
Revised: Let m ∈ {∗ , 1 } H × W be a mask .

Original: We call the element-wise product x (cid:12) m a masking of x .
Revised: We call the element-wise product x (cid:12) m a masking of x .

Original: In a masking, a subset of pixelsbecomes ∗ and the rest remains unchanged.
Revised: In a masking, a subset of pixels becomes ∗ and the rest remains unchanged.

Original: We consider the patches of size at most H (cid:48) × W (cid:48) .
Revised: We consider the patches of size at most H (cid:48) × W (cid:48) .

Original: Certiﬁed recovery.
Revised: Certiﬁed recovery.

Original: We break m into an array B of non-intersecting blocks, each having the samesize H (cid:48)
Revised: We break m into an array B of non-intersecting blocks, each having the same size H (cid:48)

Original: × W (cid:48) as the adversarial patch.
Revised: × W (cid:48) as the adversarial patch.

Original: We index the blocks as B [ q, r ] , 1 ≤ q ≤ (cid:100) HH (cid:48) (cid:101) , 1 ≤ r ≤ (cid:100) WW (cid:48) (cid:101) .
Revised: We index the blocks as B [ q, r ] , 1 ≤ q ≤ (cid:100) HH (cid:48) (cid:101) , 1 ≤ r ≤ (cid:100) WW (cid:48) (cid:101) .

Original: We say that the block B [ q, r ] is visible in a mask m if ∀ ( i, j ) ∈ B [ q, r ] : m i,j = 1 .
Revised: We say that the block B [ q, r ] is visible in a mask m if ∀ ( i, j ) ∈ B [ q, r ] : m i,j = 1 .

Original: Consider anarray M of K masks.
Revised: Consider an array M of K masks.

Original: We deﬁne each mask M [ k ] by a set of blocks that are visible in it.
Revised: We deﬁne each mask M [ k ] by a set of blocks that are visible in it.

Original: Each block isvisible in exactly one mask and masked in the others.
Revised: Each block is visible in exactly one mask and masked in the others.

Original: We say that a mask m is affected by a patch ( p, l )
Revised: We say that a mask m is affected by a patch ( p, l )

Original: if A ( x, p, l ) (cid:12) m (cid:54) =
Revised: if A ( x, p, l ) (cid:12) m (cid:54) =

Original: x (cid:12) m .
Revised: x (cid:12) m .

Original: We deﬁne T ( M )
Revised: We deﬁne T ( M )

Original: =
Revised: =

Original: max ( p,l ) ∈P |{ m ∈ M |
Revised: max ( p,l ) ∈P |{ m ∈ M |

Original: A ( x, p, l ) (cid:12) m (cid:54) =
Revised: A ( x, p, l ) (cid:12) m (cid:54) =

Original: x (cid:12) m }| .
Revised: x (cid:12) m }| .

Original: That is: T ( M ) is the largest number of masks affected by some patch.
Revised: That is: T ( M ) is the largest number of masks affected by some patch.

Original: If M is deﬁned, we refer tothe value T ( M ) as T for simplicity.
Revised: If M is deﬁned, we refer to the value T ( M ) as T for simplicity.

Original: We deﬁne column masking M for which T = 2 .
Revised: We deﬁne column masking M for which T = 2 .

Original: We assignevery k -th block column to be visible in the mask M [ k ] (Figure 2b).
Revised: We assign every k -th block column to be visible in the mask M [ k ] (Figure 2b).

Original: Any ( p, l ) ∈ P can intersect atmost two adjacent columns since ( p, l ) has the same width as a column.
Revised: Any ( p, l ) ∈ P can intersect at most two adjacent columns since ( p, l ) has the same width as a column.

Original: Thus, it can affect at mosttwo masks (Figure 2b).
Revised: Thus, it can affect at most two masks (Figure 2b).

Original: A similar scheme can be proposed for the rows.
Revised: A similar scheme can be proposed for the rows.

Original: Due to the block size thepatch ( p, l ) cannot intersect more than four blocks at once.
Revised: Due to the block size the patch ( p, l ) cannot intersect more than four blocks at once.

Original: We deﬁne a mask set that we call 3-masks. t. for any four adjacent blocks two are visible in the same mask (Figures 2c).
Revised: We deﬁne a mask set that we call 3-mask s. t. for any four adjacent blocks two are visible in the same mask (Figures 2c).

Original: Hence, a patch for3-mask can affect no more than 3 masks, T = 3 .
Revised: Hence, a patch for 3-mask can affect no more than 3 masks, T = 3 .

Original: To achieve T = 

Revised: To achieve T =

Original: any assignment of visible blocksto the masks works.
Revised: any assignment of visible blocks to the masks works.

Original: We consider 4-mask that allows uniform coverage of the visible blocks in theimage (Figure 2d).
Revised: We consider 4-mask that allows uniform coverage of the visible blocks in the image (Figure 2d).

Original: See details in Appendix B.
Revised: See details in Appendix B.

Original: Certiﬁed detection.
Revised: Certiﬁed detection.

Original: We deﬁne M d to be a set of masks for certiﬁed detection (we use subscript dfor distinction).
Revised: We deﬁne M d to be a set of masks for certiﬁed detection (we use subscript d for distinction).

Original: M d should have the property: ∀ ( p, l ) ∈ P ∃ m ∈ M d : A ( x, p, l ) (cid:12) m =
Revised: M d should have the property: ∀ ( p, l ) ∈ P ∃ m ∈ M d : A ( x, p, l ) (cid:12) m =

Original: x (cid:12) m i.e. for every patch exists at least one mask not affected by this patch.
Revised: x (cid:12) m i. e. for every patch exists at least one mask not affected by this patch.

Original: See details in Appendix B.
Revised: See details in Appendix B.

Original: 4.2 Certiﬁcation
Revised: 4.2 Certiﬁcation

Original: Certiﬁed recovery.
Revised: Certiﬁed recovery.

Original: For the threat model P consider a set M of K masks.
Revised: For the threat model P consider a set M of K masks.

Original: We deﬁne a functionh : X → S that assigns a class to the pixel x i,j via majority voting over class predictions of eachreconstructed segmentation in S .
Revised: We deﬁne a function h : X → S that assigns a class to the pixel x i,j via majority voting over class predictions of each [Figure] [Figure]

Revised: reconstructed segmentation in S .

Original: A class for the pixel that is predicted by the largest number ofsegmentations is assigned.
Revised: A class for the pixel that is predicted by the largest number of segmentations is assigned.

Original: We break the ties by assigning a class with a smaller index.
Revised: We break the ties by assigning a class with a smaller index.

Original: [Figure]

Original: [Figure]

Original: Theorem 1.
Revised: Theorem 1.

Original: If the number of masks K satisﬁes K ≥ 2 T ( M ) + 1 and for a pixel x i,j we have ∀ S
Revised: If the number of masks K satisﬁes K ≥ 2 T ( M ) + 1 and for a pixel x i,j we have ∀ S

Original: [ k ] ∈ S : S [ k ] i,j = h
Revised: [ k ] ∈ S : S [ k ] i,j = h

Original: ( x ) i,j (i.e. all the votes agree), then ∀ (
Revised: ( x ) i,j (i.e. all the votes agree), then ∀ (

Original: p, l ) ∈ P : h ( A ( x, p, l ))
Revised: p, l ) ∈ P : h ( A ( x, p, l ))

Original: i,j = h ( x ) i,j .
Revised: i,j = h ( x ) i,j .

Original: Certiﬁed detection.
Revised: Certiﬁed detection.

Original: Consider M d = { M d [ k ] } Kk =1 .
Revised: Consider M d = { M d [ k ] } Kk =1 .

Original: For a set of demasked segmentations S we deﬁnethe veriﬁcation map v ( x ) i,j :=
Revised: For a set of demasked segmentations S we deﬁne the veriﬁcation map v ( x ) i,j :=

Original: [ f ( x ) i,j = S [1] i,j = . . .
Revised: [ f ( x ) i,j = S [1] i,j = . . .

Original: = S
Revised: = S

Original: [ K ] i,j ] i.e. the original segmentationis equal to all the other segmentations on masked-demasked inputs, including the one in which thepotential patch was completely masked.
Revised: [ K ] i,j ] i.e. the original segmentation is equal to all the other segmentations on masked-demasked inputs, including the one in which the potential patch was completely masked.

Original: Theorem 2.
Revised: Theorem 2.

Original: Assume that v ( x ) i,j = 1 .
Revised: Assume that v ( x ) i,j = 1 .

Original: Then
Revised: Then

Original: [Equation]
Revised: [Equation]

Original: See the proofs for both theorems in Appendix A. For a given image x the veriﬁcation map v ( x ) iscomplementary to the model segmentation output f ( x ) that stays unchanged.
Revised: See the proofs for both theorems in Appendix A. For a given image x the veriﬁcation map v ( x ) is complementary to the model segmentation output f ( x ) that stays unchanged.

Original: Thus, there is no dropin clean performance however we may have some false positive alerts in the veriﬁcation map v in theclean setting.
Revised: Thus, there is no drop in clean performance however we may have some false positive alerts in the clean setting.

Original:  Experiments
Revised:  Experiments

Original: In this section, we evaluate D EMASKED S MOOTHING with the masking schemes proposed in
Revised: In this section, we evaluate D EMASKED S MOOTHING with the masking schemes proposed in

Original: Section 4, compare our approach with the direct application of Derandomized Smoothing [9] to thesegmentation task and evaluate the performance on different datasets and models.
Revised: Section 4, compare our approach with the direct application of Derandomized Smoothing [9] to the segmentation task and evaluate the performance on different datasets and models.

Original: Certiﬁed recoveryand certiﬁed detection provide certiﬁcates of different strength (Section 4) which are not comparable.
Revised: Certiﬁed recovery and certiﬁed detection provide certiﬁcates of different strength (Section 4) which are not comparable.

Original: We evaluate them separately for different patch sizes.
Revised: We evaluate them separately for different patch sizes.

Original: Experimental Setup .
Revised: Experimental Setup .

Original: We evaluate D EMASKED S MOOTHING on two challenging semantic segmentation datasets: ADE20K [31] (150 classes, 2000 validation images) and COCO-Stuff-10K [39] (171classes, 1000 validation images).
Revised: We evaluate D EMASKED S MOOTHING on two challenging semantic segmentation datasets: ADE20K [31] (150 classes, 2000 validation images) and COCO-Stuff-10K [39] ( classes, 1000 validation images).

Original: For demasking we use the ZITS [29] inpainting model with thecheckpoint trained on Places2 [40] from the ofﬁcial paper repository 1 .
Revised: For demasking we use the ZITS [29] inpainting model with the checkpoint trained on Places2 [40] from the ofﬁcial paper repository 1 .

Original: As a segmentation model fwe use BEiT [30], Swin [26], PSPNet [24] and DeepLab v3 [41].
Revised: As a segmentation model f we use BEiT [30], Swin [26], PSPNet [24] and DeepLab v3 [41].

Original: We use the model implementationsprovided in the mmsegmentation framework [42].
Revised: We use the model implementations provided in the mmsegmentation framework [42].

Original: An illustration of the image reconstruction andrespective segmentation can be found in Figure 3.
Revised: An illustration of the image reconstruction and respective segmentation can be found in Figure 3.

Original: Evaluation .
Revised: Evaluation .

Original: We compute mIoU, mean recall (mR) and certiﬁed mean recall (cmR).
Revised: We compute mIoU, mean recall (mR) and certiﬁed mean recall (cmR).

Original: See detailedexplanation of these metrics in Appendix C. In certiﬁed detection, we additionally consider false alertratio (FAR) which is the fraction of correctly classiﬁed pixels for which we return an alert on a cleanimage.
Revised: See detailed explanation of these metrics in Appendix C. In certiﬁed detection, we additionally consider false alert ratio (FAR) which is the fraction of correctly classiﬁed pixels for which we return an alert on a clean image.

Original: Smaller FAR is preferable.
Revised: Smaller FAR is preferable.

Original: Due to our threat model, certifying small objects in the scene canbe difﬁcult because they can be partially or completely covered by an adversarial patchn.
Revised: Due to our threat model, certifying small objects in the scene can be difﬁcult because they can be partially or completely covered by an adversarial patch.

Original: To providean additional perspective on our methods, we also evaluate mR and cmR speciﬁcally for the “big”classes, which occupy on average more than 20% of the images in which they appear.
Revised: To provide an additional perspective on our methods, we also evaluate mR and cmR speciﬁcally for the “big” classes, which occupy on average more than 20% of the images in which they appear.

Original: These are, forexample, road, building, train, and sky, which are important for understanding the scene.
Revised: These are, for example, road, building, train, and sky, which are important for understanding the scene.

Original: The full listis provided in the Appendix I. We run the evaluation in parallel on 5 Nvidia Tesla V100-32GB GPUs.
Revised: The full list is provided in the Appendix I. See execution time in Table 11 in the Appendix J.

Original: The certiﬁcation for the whole ADE20K validation set with ZITS and BEiT-B takes around 1.2 hoursfor certiﬁed recovery and 2 hours for certiﬁed detection (due to a larger number of masks).

Original: Discussion .
Revised: Discussion .

Original: In Table 1, we compare different masking schemes proposed in Section 4.1.
Revised: In Table 1, we compare different masking schemes proposed in Section 4.1.

Original: Evaluationof all the models with all the masking schemes is consistent with these results and can be found in
Revised: Evaluation of all the models with all the masking schemes is consistent with these results and can be found in

Revised: Appendix F. We see that column masking achieves better results in both certiﬁcation modes.

Revised: We

Original: [Table] [Table]
Revised: [Table] [Table]

Original: Appendix F. We see that column masking achieves better results in both certiﬁcation modes.

Original: Weattribute the effectiveness of column masking to the fact most of the images in the datasets have aclear horizont line, therefore having a visible column provides a slice of the image that intersectsmost of the scene background objects.
Revised: attribute the effectiveness of column masking to the fact most of the images in the datasets have a clear horizon line, therefore having a visible column provides a slice of the image that intersects most of the scene background objects.

Original: In Table 2, we evaluate our method with column masking on different models.
Revised: In Table 2, we evaluate our method with column masking on different models.

Original: For certiﬁed detectionwe can certify more than 60% of the pixels with all models on ADE20K and more than 46 % on
Revised: For certiﬁed detection we can certify more than 60% of the pixels with all models on ADE20K and more than 46 % on

Original: COCO10K. False alert ratio on correctly classiﬁed pixels is around 20%.
Revised: COCO10K. False alert ratio on correctly classiﬁed pixels is around 20%.

Original: In certiﬁed recovery,we certify more than 44% pixels on ADE20K and more than 28% pixels on COCO10K. See thecomparison with DRS [9] adaptded for segmentation in Appendix H.
Revised: In certiﬁed recovery, we certify more than 44% pixels on ADE20K and more than 28% pixels on COCO10K. See the comparison with DRS [9] adaptded for segmentation in Appendix H.

Original: We evaluate the performance ofour method for different patch sizes in Section C. Ablations with respect to inpainting can be foundin
Revised: We evaluate the performance of our method for different patch sizes in Section C. Ablations with respect to inpainting can be found in Appendix G. D

Original: Appendix G. D EMASKED S MOOTHING illustrations procedure are provided in Appendix L.
Revised: EMASKED S MOOTHING illustrations procedure are provided in Appendix L.

Original:  Conclusion
Revised:  Conclusion

Original: In this work, we propose D EMASKED S MOOTHING , the ﬁrst (up to our knowledge) certiﬁed defenceframework against patch attacks on segmentation models.
Revised: In this work, we propose D EMASKED S MOOTHING , the ﬁrst (up to our knowledge) certiﬁed defence framework against patch attacks on segmentation models.

Original: Due to its novel design based on maskingschemes and image demasking, D EMASKED S MOOTHING is compatible with any segmentationmodel and can on average certify 63% of the pixel predictions for a 1% patch in the detection taskand 46% against a 0.5% patch for the recovery task on the ADE20K dataset.
Revised: Due to its novel design based on masking schemes and image demasking, D EMASKED S MOOTHING is compatible with any segmentation model and can on average certify 63% of the pixel predictions for a 1% patch in the detection task and 46% against a 0.5% patch for the recovery task on the ADE20K dataset.

Original: Ethical and Societal Impact
Revised: Ethical and Societal Impact

Original: This work contributes to the ﬁeld of certiﬁed defences againstphysically-realizable adversarial attacks.
Revised: This work contributes to the ﬁeld of certiﬁed defences against physically-realizable adversarial attacks.

Original: The proposed approach allows to certify robustness ofsafety-critical applications such as medical imaging or autonomous driving.
Revised: The proposed approach allows to certify robustness of safety-critical applications such as medical imaging or autonomous driving.

Original: The defence mightbe used to improve robustness of systems used for malicious purposes such as (semi-)autonomousweaponry or unauthorized surveillance.
Revised: The defence might be used to improve robustness of systems used for malicious purposes such as (semi-)autonomous weaponry or unauthorized surveillance.

Original: This danger may be mitigated e.g. by using a system ofsparsely distributed patches which makes certifying the image more challenging.
Revised: This danger may be mitigated e.g. by using a system of sparsely distributed patches which makes certifying the image more challenging.

Original: All activities in ourorganization are carbon neutral, so our experiments do not leave any carbon dioxide footprint.
Revised: All activities in our organization are carbon neutral, so our experiments do not leave any carbon dioxide footprint.

Revised: Acknowledgements

Revised: We thank Chong Xiang for the suggestions on extending our evaluation protocol.

Revised: Matthias Hein is a member of the Machine Learning Cluster of Excellence, EXC number 2064/1 – Project number 390727645 and of the BMBF Tübingen AI Center, FKZ: 01IS18039B.

Original: A Proofs (Section 4)
Revised: A Proofs (Section 4)

Original: In this section, we provide the proofs for the theorems stated in Section 4.
Revised: In this section, we provide the proofs for the theorems stated in Section 4.

Original: Certiﬁed recovery.
Revised: Certiﬁed recovery.

Original: For the threat model P (Section 3) consider a set M of K masks.
Revised: For the threat model P (Section 3) consider a set M of K masks.

Original: We deﬁne afunction h : X → S that assigns a class to the pixel x i,j via majority voting over class predictions ofeach reconstructed segmentation in S .
Revised: We deﬁne a function h : X → S that assigns a class to the pixel x i,j via majority voting over class predictions of each reconstructed segmentation in S .

Original: A class for the pixel that is predicted by the largest number ofsegmentations is assigned.
Revised: A class for the pixel that is predicted by the largest number of segmentations is assigned.

Original: We break the ties by assigning a class with a smaller index.
Revised: We break the ties by assigning a class with a smaller index.

Original: Theorem 1 .
Revised: Theorem 1 .

Original: (Section 4.2)
Revised: (Section 4.2)

Original: If the number of masks K satisﬁes K ≥ 2 T ( M ) + 1 and for a pixel x i,j we have
Revised: If the number of masks K satisﬁes K ≥ 2 T ( M ) + 1 and for a pixel x i,j we have

Original: [Equation]
Revised: [Equation]

Original: i.e. all the votes agree, then ∀ ( p, l
Revised: i.e. all the votes agree, then ∀ ( p, l

Original: ) ∈ P : h( A ( x, p,
Revised: [Equation] ∈ P [Equation] h

Original: [Equation]
Revised: [Equation]

Original: i,j = h ( x
Revised: A

Original: [Equation]
Revised: [Equation]

Original: i,j
Revised: x, p, l

Original: [Equation]
Revised: [Equation]

Original: Proof.
Revised: Proof.

Original: Assume that
Revised: Assume that

Original: [Equation]
Revised: [Equation]

Original: Let us denote x (cid:48) :
Revised: Let us denote x (cid:48) :

Original: = A ( x, p, l ) and S (cid:48) to be the segmentation array for x (cid:48) .
Revised: = A ( x, p, l ) and S (cid:48) to be the segmentation array for x (cid:48) .

Original: Then the class h ( x ) i,j didnot get the majority vote for S (cid:48) .
Revised: Then the class h ( x ) i,j did not get the majority vote for S (cid:48) .

Original: However, by deﬁnition of T ( M ) we know that ( p, l ) could affectat most T ( M ) segmentations.
Revised: However, by deﬁnition of T ( M ) we know that ( p, l ) could affect at most T ( M ) segmentations.

Original: Since all K segmentations of S have voted for h ( x ) i,j , then at least
Revised: Since all K segmentations of S have voted for h ( x ) i,j , then at least

Original: K − T > K 2 of them are still voting for h ( x ) i,j in S (cid:48)
Revised: K − T > K 2 of them are still voting for h ( x ) i,j in S (cid:48)

Original: meaning that h ( x ) i,j still has the majority votein S (cid:48) .
Revised: meaning that h ( x ) i,j still has the majority vote in S (cid:48) .

Original: Therefore h ( x (cid:48) )
Revised: Therefore h ( x (cid:48) )

Original: i,j = h ( x ) i,j
Revised: i,j = h ( x ) i,j

Original: Certiﬁed detection.
Revised: Certiﬁed detection.

Original: Consider M d = { M d [ k ] } Kk =1 .
Revised: Consider M d = { M d [ k ] } Kk =1 .

Original: For a set of demasked segmentations S we deﬁnethe veriﬁcation map v ( x ) i,j :=
Revised: For a set of demasked segmentations S we deﬁne the veriﬁcation map v ( x ) i,j :=

Original: [ f ( x ) i,j = S [1] i,j = . . .
Revised: [ f ( x ) i,j = S [1] i,j = . . .

Original: = S
Revised: = S

Original: [ K ] i,j ] i.e. the original segmentationcoincides with all the other segmentations including the one in which the potential patch wascompletely masked.
Revised: [ K ] i,j ] i.e. the original segmentation coincides with all the other segmentations including the one in which the potential patch was completely masked.

Original: Theorem 2 .
Revised: Theorem 2 .

Original: (Section 4.2) Assume that v ( x ) i,j = 1 .
Revised: (Section 4.2) Assume that v ( x ) i,j = 1 .

Original: Then ∀ ( p, l ) ∈ P : v ( A ( x, p, l ))
Revised: Then ∀ ( p, l ) ∈ P : v ( A ( x, p, l ))

Original: i,j = 1 ⇒ f
Revised: i,j = 1 ⇒ f

Original: ( A ( x, p, l ))
Revised: ( A ( x, p, l ))

Original: i,j = f ( x ) i,j
Revised: i,j = f ( x ) i,j

Original: Proof.
Revised: Proof.

Original: Assume that ∃ ( p, l ) ∈ P s. t. v ( A ( x, p, l )) i,j = 1 and f ( A ( x, p, l ))
Revised: Assume that ∃ ( p, l ) ∈ P s. t. v ( A ( x, p, l )) i,j = 1 and f ( A ( x, p, l ))

Original: i,j (cid:54) =
Revised: i,j (cid:54) =

Original: f ( x ) i,j .Let us denote x (cid:48) :
Revised: f ( x ) i,j .Let us denote x (cid:48) :

Original: = A ( x, p, l ) and S (cid:48) to be the segmentation set for x (cid:48) .
Revised: = A ( x, p, l ) and S (cid:48) to be the segmentation set for x (cid:48) .

Original: By deﬁnition of M d , ∃ M d [ k ] ∈ M d s. t. M d [ k ] masks the patch ( p, l ) Hence, g ( x (cid:12) M d [ k ])
Revised: By deﬁnition of M d , ∃ M d [ k ] ∈ M d s. t. M d [ k ] masks the patch ( p, l ) Hence, g [Equation]

Original: = g ( x (cid:48) (cid:12) M d [ k ]) , S[ k ] = f
Revised: S

Original: ( g ( x (cid:12) M d [ k ])) = f ( g ( x (cid:48) (cid:12) M d
Revised: [Equation]

Original: [ k ])) =
Revised: [Equation] ) =

Original: S (cid:48)
Revised: S (cid:48)

Original: [ k ] ,
Revised: [ k ] ,

Original: Since v ( x ) i,j = 1 , we have f ( x ) i,j = S [ k ] i,j .
Revised: Since v ( x ) i,j = 1 , we have f ( x ) i,j = S [ k ] i,j .

Original: Since v ( x (cid:48) )
Revised: Since v ( x (cid:48) )

Original: i,j = 1 , we have f ( x (cid:48) )
Revised: i,j = 1 , we have f ( x (cid:48) )

Original: i,j = S (cid:48)
Revised: i,j = S (cid:48)

Original: [ k ] i,j .
Revised: [ k ] i,j .

Original: Thus,f ( x (cid:48) ) i,j = f ( x ) i,j .
Revised: Thus, f ( x (cid:48) ) i,j = f ( x ) i,j .

Original: B Detailed description of masking strategies
Revised: B Detailed description of masking strategies

Original: In this section, we provide additional details for constructing certiﬁed recovery masks proposed in
Revised: In this section, we provide additional details for constructing certiﬁed recovery masks proposed in

Original: Section 4.1.
Revised: Section 4.1.

Original: Certiﬁed recovery .
Revised: Certiﬁed recovery .

Original: We deﬁne mask sets M that satisfy different values of T .
Revised: We deﬁne mask sets M that satisfy different values of T .

Original: We divide the image xinto a set of non-intersecting blocks B of the same size as an adversarial patch, H (cid:48)
Revised: We divide the image x into a set of non-intersecting blocks B of the same size as an adversarial patch, H (cid:48)

Original: × W (cid:48) (see Figure5), 1 ≤ q ≤ (cid:100) H/H (cid:48) (cid:101) , 1 ≤ r ≤ (cid:100) W/W (cid:48) (cid:101) .
Revised: × W (cid:48) (see Figure 5), 1 ≤ q ≤ (cid:100) H/H (cid:48) (cid:101) , 1 ≤ r ≤ (cid:100) W/W (cid:48) (cid:101) .

Original: In each mask, each of these blocks will be either maskedor not masked (i. e. visible ).
Revised: In each mask, each of these blocks will be either masked or not masked (i. e. visible ).

Original: Moreover, for each block there exists only one mask in which it isvisible.
Revised: Moreover, for each block there exists only one mask in which it is visible.

Original: For a set M of K masks we deﬁne the mapping µ M : B → { 1 , . . .
Revised: For a set M of K masks we deﬁne the mapping µ M : B → { 1 , . . .

Original: , K } .
Revised: , K } .

Original: If µ ( B [ q, r ]) =
Revised: If µ ( B [ q, r ]) =

Original: k ,then B [ q, r ] is not masked in M [ k ] .
Revised: k , then B [ q, r ] is not masked in M [ k ] .

Original: Therefore, each mask M [ k ] is deﬁned by a B k ⊂ B s. t. forb ∈ B k µ ( b ) = k .
Revised: Therefore, each mask M [ k ] is deﬁned by a B k ⊂ B s. t. for b ∈ B k µ ( b ) = k .

Original: We deﬁne a set M that we call 3-mask for which T ( M )
Revised: We deﬁne a set M that we call 3-mask for which T ( M )

Original: = 3 .
Revised: = 3 .

Original: We assign the blocks in each row to themasks as follows: µ ( B [1 , 1]) = 1 ; µ ( B [1 , 2])=
Revised: We assign the blocks in each row to the masks as follows: µ ( B [1 , 1]) = 1 ; µ ( B [1 , 2])

Original:  µ ( B [1 , 3]) = 2 ; µ ( B [1 , 4])
Revised: = µ ( B [1 , 3]) = 2 ; µ ( B [1 , 4])

Original: =
Revised: =

Original: µ ( B [1 , 5]) =
Revised: µ ( B [1 , 5]) =

Original: 3and so on until we reach the end of the row.
Revised: and so on until we reach the end of the row.

Original: If we ﬁnish the ﬁrst row with the value k , then we startthe second row as follows µ ( B [2 , 1])
Revised: If we ﬁnish the ﬁrst row with the value k , then we start the second row as follows µ ( B [2 , 1])

Original: =
Revised: =

Original: µ ( B [2 , 2])
Revised: µ ( B [2 , 2])

Original: =
Revised: =

Original: k + 1 ; µ ( B [2 , 3])
Revised: k + 1 ; µ ( B [2 , 3])

Original: =
Revised: =

Original: µ ( B [2 , 4])
Revised: µ ( B [2 , 4])

Original: =
Revised: =

Original: k
Revised: k

Original: + 2 :. . .
Revised: + 2 : . . .

Original: If we ﬁnish the second row on n , we start the third row similarly to the ﬁrst: µ ( B [3 , 1]) = n + 1 ;µ ( B [3 , 2])
Revised: If we ﬁnish the second row on n , we start the third row similarly to the ﬁrst: µ ( B [3 , 1]) = n + 1 ; µ ( B [3 , 2])

Original: =
Revised: =

Original: µ ( B [3 , 3]) =
Revised: µ ( B [3 , 3]) =

Original: n + 2 ; . .
Revised: n + 2 ; . .

Original: When we reach the number K , we start from 1 again
Revised: When we reach the number K , we start from 1 again

Original: (Figure 5d).
Revised: (Figure 5d).

Original: Due to the block size, the patch cannot intersect more than four blocks at once.
Revised: Due to the block size, the patch cannot intersect more than four blocks at once.

Original: Ourparity-alternating block sequence ensures that in any such intersection of four blocks either the topones or the bottom ones will belong to the same masking, so at most three different maskings can beaffected.
Revised: Our parity-alternating block sequence ensures that in any such intersection of four blocks either the top ones or the bottom ones will belong to the same masking, so at most three different maskings can be affected.

Original: We deﬁne a set M that we call 4-mask for which T ( M )
Revised: We deﬁne a set M that we call 4-mask for which T ( M )

Original: = 4 .
Revised: = 4 .

Original: Due to our block size any assignment ofmasks will work because the patch cannot intersect more than four blocks.
Revised: Due to our block size any assignment of masks will work because the patch cannot intersect more than four blocks.

Original: We consider the one thatallows uniform distribution of the unmasked blocks (Figure 5g).
Revised: We consider the one that allows uniform distribution of the unmasked blocks (Figure 5g).

Original: We point out that for the describedmethods each masking keeps approximately 1 /K of the pixels visible and the unmasked regions areuniformly distributed in the image.
Revised: We point out that for the described methods each masking keeps approximately 1 /K of the pixels visible and the unmasked regions are uniformly distributed in the image.

Original: This means that for any masked pixel there exists an unmaskedregion located close enough to this pixel.
Revised: This means that for any masked pixel there exists an unmasked region located close enough to this pixel.

Original: It is the core difference between our masks and the onesproposed for certiﬁed classiﬁcation such as block or column smoothing [9].
Revised: It is the core difference between our masks and the ones proposed for certiﬁed classiﬁcation such as block or column smoothing [9].

Original: It was observed that theimage demasking is facilitated when the visible regions are uniformly spread in the masked image[36].
Revised: It was observed that the image demasking is facilitated when the visible regions are uniformly spread in the masked image [36].

Original: We present the full demasked smoothing procedure in Algorithm 1.
Revised: We present the full demasked smoothing procedure in Algorithm 1.

Original: Certiﬁed detection .
Revised: Certiﬁed detection .

Original: We deﬁne M d to be a set of masks for certiﬁed detection (we use subscript dfor distinction).
Revised: We deﬁne M d to be a set of masks for certiﬁed detection (we use subscript d for distinction).

Original: M d should have the property: ∀ ( p, l ) ∈ P ∃ m ∈ M d : A ( x, p, l ) (cid:12) m =x (cid:12) mi.
Revised: M d should have the property: ∀ ( p, l ) ∈ P ∃ m ∈ M d : A ( x, p, l ) (cid:12) m =

Original:  e. for every patch exists at least one mask not affected by this patch.
Revised: x (cid:12) m i. e. for every patch exists at least one mask not affected by this patch.

Original: See deteils in Appendix B.
Revised: See deteils in Appendix B.

Original: For a patch of size H (cid:48) × W (cid:48)
Revised: For a patch of size H (cid:48) × W (cid:48)

Original: we consider K = W − W (cid:48)
Revised: we consider K = W − W (cid:48)

Original: + 1 masks such that the mask M d [ k ] masksa column of width W (cid:48) starting at the horizontal position k in the image (Figure 2e).
Revised: + 1 masks such that the mask M d [ k ] masks a column of width W (cid:48) starting at the horizontal position k in the image (Figure 2e).

Original: To obtain theguarantee for the same P with a smaller K , we consider a set of strided columns of width W (cid:48)(cid:48) ≥ W (cid:48)and stride W (cid:48)(cid:48) − W (cid:48) + 1 that also satisfy the condition.
Revised: To obtain the guarantee for the same P with a smaller K , we consider a set of strided columns of width W (cid:48)(cid:48) ≥ W (cid:48) and stride W (cid:48)(cid:48) − W (cid:48) + 1 that also satisfy the condition.

Original: Lemma 1. Consider an image of the size H × W .
Revised: Lemma 1. Consider an image of the size H × W .

Original: Let H (cid:48) × W (cid:48) be a ﬁxed adversarial patch size.
Revised: Let H (cid:48) × W (cid:48) be a ﬁxed adversarial patch size.

Original: Let
Revised: Let

Original: M d ( K, L ) be a set of masks where each mask is masking an H × W (cid:48)(cid:48) vertical column, W (cid:48)(cid:48) ≥ W (cid:48) .
Revised: M d ( K, L ) be a set of masks where each mask is masking an H × W (cid:48)(cid:48) vertical column, W (cid:48)(cid:48) ≥ W (cid:48) .

Original: Let the stride between the columns in two adjacent masks be W (cid:48)(cid:48)
Revised: Let the stride between the columns in two adjacent masks be W (cid:48)(cid:48)

Original: − W (cid:48)
Revised: − W (cid:48)

Original: + 1 .
Revised: + 1 .

Original: Then for any locationl ∈ L of the patch, there exists a mask that covers it completely.
Revised: Then for any location l ∈ L of the patch, there exists a mask that covers it completely.

Original: Proof.
Revised: Proof.

Original: (Adapted from the proof of Lemma 4 in PatchCleanser [14]).
Revised: (Adapted from the proof of Lemma 4 in PatchCleanser [14]).

Original: Without loss of generality,we consider the ﬁrst two adjacent column masks.
Revised: Without loss of generality, we consider the ﬁrst two adjacent column masks.

Original: The ﬁrst one covers the columns from 1 to
Revised: The ﬁrst one covers the columns from 1 to

Original: W (cid:48)(cid:48) .
Revised: W (cid:48)(cid:48) .

Original: The second mask covers the columns from 1 + ( W (cid:48)(cid:48) − W (cid:48)
Revised: The second mask covers the columns from 1 + ( W (cid:48)(cid:48) − W (cid:48)

Original: + 1)
Revised: + 1)

Original: =
Revised: =

Original: W (cid:48)(cid:48) − W (cid:48) + 2 to( W (cid:48)(cid:48) − W (cid:48) + 2) + ( W (cid:48)(cid:48) − 1)
Revised: W (cid:48)(cid:48) − W (cid:48) + 2 to ( W (cid:48)(cid:48) − W (cid:48) + 2) + ( W (cid:48)(cid:48) − 1)

Original: = 2 W (cid:48)(cid:48)
Revised: = 2 W (cid:48)(cid:48)

Original: − W (cid:48)
Revised: − W (cid:48)

Original: + 1 (See Figure 4).
Revised: + 1 (See Figure 4).

Original: Now consider an adversarial patchof size H (cid:48)
Revised: Now consider an adversarial patch of size H (cid:48)

Original: × W (cid:48) .
Revised: × W (cid:48) .

Original: Let us ﬁnd the smallest possible start index of this patch so that it does not getcovered by the ﬁrst mask.
Revised: Let us ﬁnd the smallest possible start index of this patch so that it does not get covered by the ﬁrst mask.

Original: For that it should be visible at the column W (cid:48)(cid:48) + 1 and, therefore, start atthe column with index not smaller than ( W (cid:48)(cid:48) + 1) − W (cid:48)
Revised: For that it should be visible at the column W (cid:48)(cid:48) + 1 and, therefore, start at the column with index not smaller than ( W (cid:48)(cid:48) + 1) − W (cid:48)

Original: + 1 = W (cid:48)(cid:48) − W (cid:48)
Revised: + 1 = W (cid:48)(cid:48) − W (cid:48)

Original: + 2 .
Revised: + 2 .

Original: However, it is thesame column in which second mask starts.
Revised: However, it is the same column in which second mask starts.

Original: Therefore, given that W (cid:48)(cid:48) ≥ W (cid:48) we have that the patchis completely masked by the second mask.
Revised: Therefore, given that W (cid:48)(cid:48) ≥ W (cid:48) we have that the patch is completely masked by the second mask.

Original: Then for a patch which is only partially masked by thesecond mask from the left we use an analogous argument to show that it is completely masked by thethird mask and so on.
Revised: Then for a patch which is only partially masked by the second mask from the left we use an analogous argument to show that it is completely masked by the third mask and so on.

Original: A similar scheme can be proposed for the rows (Figure 2f).
Revised: A similar scheme can be proposed for the rows (Figure 2f).

Original: Alternatively, we could use a set of blockmasks of size H (cid:48) × W (cid:48) .
Revised: Alternatively, we could use a set of block masks of size H (cid:48) × W (cid:48) .

Original: Then the number of masks grows quadratically with the image resolution.
Revised: Then the number of masks grows quadratically with the image resolution.

Original: Hence, in the experiments we focus on the column and the row masking schemes.
Revised: Hence, in the experiments we focus on the column and the row masking schemes.

Original: [Figure]
Revised: [Figure]

Original: Input: image x ∈ [0 , 1] H × W × C , patch size ( H (cid:48) , W (cid:48) ), certiﬁcation type CT (recovery or detection), mask type MT (column, row, 3-mask, 4-mask), inpainting model g , segmentation model f
Revised: Input: image x ∈ [0 , 1] H × W × C , patch size ( H (cid:48) , W (cid:48) ), certiﬁcation type CT (recovery or detection), mask type MT (column, row, 3-mask, 4-mask), inpainting model g , segmentation model f

Original: Output: segmentation map h ∈ Y H × W , certiﬁcation (or veriﬁcation) map v ∈ { 0 , 1 } H × W 1: M ← CreateMaskArray( H, W, H (cid:48) , W (cid:48) , CT , MT) (cid:46) to section 4.
Revised: Output: segmentation map h ∈ Y H × W , certiﬁcation (or veriﬁcation) map v ∈ { 0 , 1 } H × W 1: M ← CreateMaskArray( H, W, H (cid:48) , W (cid:48) , CT , MT) (cid:46) to section 4.

Original: 2: for k ← 1 , . . .
Revised: 2: for k ← 1 , . . .

Original: , | M | do (cid:46) loop paralellized 3:
Revised: , | M | do (cid:46) loop paralellized 3:

Original: S [ k ] ← f ( g ( x (cid:12) M [ k ]))
Revised: S [ k ] ← f ( g ( x (cid:12) M [ k ]))

Original: (cid:46) mask input, inpaint the masked regions, and apply segmentation 4: end for 5:
Revised: (cid:46) mask input, inpaint the masked regions, and apply segmentation 4: end for 5:

Original: if CT = ’recovery’ then h ← MajorityVote ( S ) (cid:46) vote over the classes predicted for each pixel 6: else h ← f ( x ) (cid:46) in detection case, output clean segmentation 7: end if 8: v ← AllEqual ( S, h ) (cid:46) assign 1 for the pixels where all S [ k ] agree with h and 0 otherwise 9: Return h , v
Revised: if CT = ’recovery’ then h ← MajorityVote ( S ) (cid:46) vote over the classes predicted for each pixel 6: else h ← f ( x ) (cid:46) in detection case, output clean segmentation 7: end if 8: v ← AllEqual ( S, h ) (cid:46) assign 1 for the pixels where all S [ k ] agree with h and 0 otherwise 9: Return h , v

Original: C Evaluation metrics
Revised: C Evaluation metrics

Original: For both certiﬁed recovery and certiﬁed detection, we provide a standard segmentation output (withoutany abstention) and a corresponding certiﬁcation map (Figure 3).
Revised: For both certiﬁed recovery and certiﬁed detection, we provide a standard segmentation output (without any abstention) and a corresponding certiﬁcation map (Figure 3).

Original: In case of certiﬁed detection, thesegmentation output remains the same as for the original segmentation model, however, there may befalse alerts in the certiﬁcaton map.
Revised: In case of certiﬁed detection, the segmentation output remains the same as for the original segmentation model, however, there may be false alerts in the certiﬁcation map.

Original: For the certiﬁed recovery, the output is obtained by a majorityvote over the segmentations of demasked images (Section 4.2).
Revised: For the certiﬁed recovery, the output is obtained by a majority vote over the segmentations of demasked images (Section 4.2).

Original: We evaluate the mean intersectionover union (mIoU) for these outputs.
Revised: We evaluate the mean intersection over union (mIoU) for these outputs.

Original: The certiﬁcation map is obtained by assigning to each certiﬁedpixel the corresponding class from the segmentation output and assigning a special uncertiﬁed labelto all non-certiﬁed pixels.
Revised: The certiﬁcation map is obtained by assigning to each certiﬁed pixel the corresponding class from the segmentation output and assigning a special uncertiﬁed label to all non-certiﬁed pixels.

Original: For each image we evaluate the fraction of pixels which are certiﬁed andcorrect (coincide with the ground truth).
Revised: For each image we evaluate the fraction of pixels which are certiﬁed and correct (coincide with the ground truth).

Original: %C is a mean of these fractions over all the images in thedataset.
Revised: %C is a mean of these fractions over all the images in the dataset.

Original: In semantic segmentation task, the class frequencies are usually skewed, therefore globalpixel-wise accuracy alone is an insufﬁcient metric.
Revised: In semantic segmentation task, the class frequencies are usually skewed, therefore global pixel-wise accuracy alone is an insufﬁcient metric.

Original: Matching the certiﬁcation map separately for each class y ∈ Y with the ground truth segmentationfor y in the image x allows us to compute the guaranteed lower bound ( cTP y ( x ) ) on the numberof true positive pixel predictions ( TP y ( x ) )
Revised: Matching the certiﬁcation map separately for each class y ∈ Y with the ground truth segmentation for y in the image x allows us to compute the guaranteed lower bound ( cTP y ( x ) ) on the number of true positive pixel predictions ( TP y ( x ) )

Original: i.e. those that were correctly classiﬁed into y .
Revised: i.e. those that were correctly classiﬁed into y .

Original: If apixel was certiﬁed with a correct class, then this prediction cannot be changed by a patch (or,alternatively, the change will be detected by the veriﬁcation function v in certiﬁed detection).
Revised: If a pixel was certiﬁed with a correct class, then this prediction cannot be changed by a patch (or, alternatively, the change will be detected by the veriﬁcation function v in certiﬁed detection).

Original: Weconsider recall R y ( x ) =
Revised: We consider recall R y ( x ) =

Original: TP y ( x ) TP y ( x )+ FN y
Revised: TP y ( x ) TP y ( x )+ FN y

Original: ( x ) where FN y ( x ) is the number of false negative predictionsfor y in x .
Revised: ( x ) where FN y ( x ) is the number of false negative predictions for y in x .

Original: P y ( x ) =
Revised: P y ( x ) =

Original: TP y ( x ) + FN y ( x ) is the total area of y in the ground truth and does notdepend on our prediction.
Revised: TP y ( x ) + FN y ( x ) is the total area of y in the ground truth and does not depend on our prediction.

Original: We can evaluate certiﬁed recall cR y ( x ) =
Revised: We can evaluate certiﬁed recall cR y ( x ) =

Original: cTP y ( x ) P y ( x )  a lower bound onR y x ) TR D ) =
Revised: cTP y ( x ) P y ( x ) , a lower bound on

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: x
Revised: x

Original: [Figure]

Original: D P y ( x ) and cTR 
Revised: D P y ( x ) and cTR y

Original: [Figure]

Original: ( D
Revised: ( D

Original: [Figure]

Original: (cid:80) 
Revised: (cid:80) x

Original: [Figure]

Original:  cTP y ( x [Figure] x ∈ D P y( x ) respectively.
Revised: D cTP y ( x ) (cid:80) x ∈ D P y x ) respectively.

Original: Then, we obtain mean recall mR
Revised: Then, we obtain mean recall mR

Original: ( D ) =
Revised: ( D ) =

Original: 1 |Y| (cid:80) y ∈Y TR y ( D ) and certiﬁed mean recall cmR ( D )
Revised: 1 |Y| (cid:80) y ∈Y TR y ( D ) and certiﬁed mean recall cmR ( D )

Original: = 1 |Y| (cid:80) y ∈Y cTR y ( D ) .
Revised: = 1 |Y| (cid:80) y ∈Y cTR y ( D ) .

Original: Evaluating lowerbounds on other popular metrics such as mean precision or mIoU this way results in vacuous upperbound since they depend on the upper bound on false positive ( FP ) predictions.
Revised: Evaluating lower bounds on other popular metrics such as mean precision or mIoU this way results in vacuous upper bound since they depend on the upper bound on false positive ( FP ) predictions.

Original: For the pixels thatare not certiﬁed we cannot guarantee that they will not be assigned to a certain class, therefore, anon-trivial upper bound on FP is not straightforward.
Revised: For the pixels that are not certiﬁed we cannot guarantee that they will not be assigned to a certain class, therefore, a non-trivial upper bound on FP is not straightforward.

Original: We leave this direction for future work.
Revised: We leave this direction for future work.

Original: Incertiﬁed detection, we additionally consider false alert ratio (FAR) which is the fraction of correctlyclassiﬁed pixels for which we return an alert on a clean image.
Revised: In certiﬁed detection, we additionally consider false alert ratio (FAR) which is the fraction of correctly classiﬁed pixels for which we return an alert on a clean image.

Original: Smaller FAR is preferable.
Revised: Smaller FAR is preferable.

Original: Figure 6 shows how the performance of D EMASKED S MOOTHING depends on the patch size for the
Revised: Figure 6 shows how the performance of D EMASKED S MOOTHING depends on the patch size for the

Original: BEiT-B model.
Revised: BEiT-B model.

Original: We see that certiﬁed detection metrics remain high even for a patch as big as 5% ofthe image surface and for the recovery they slowly deteriorate as we increase the patch size to 2%.
Revised: We see that certiﬁed detection metrics remain high even for a patch as big as 5% of the image surface and for the recovery they slowly deteriorate as we increase the patch size to 2%.

Original: D Test-time input certiﬁcation
Revised: D Test-time input certiﬁcation

Original: In this section, we discuss how certiﬁed recovery (Theorem 1) can be applied to guaranteed veriﬁcationof the robustness on a test image.
Revised: In this section, we discuss how certiﬁed recovery (Theorem 1) can be applied to guaranteed veriﬁcation of the robustness on a test image.

Original: We also discuss how robustness guarantees for the test-time imagescan be evaluated by using a dataset of clean images such as ADE20K [31] or COCO-Stuff-10K [39].
Revised: We also discuss how robustness guarantees for the test-time images can be evaluated by using a dataset of clean images such as ADE20K [31] or COCO-Stuff-10K [39].

Original: D.1 Test-time certiﬁed recovery
Revised: D.1 Test-time certiﬁed recovery

Original: Let x (cid:48) be a test-time input which can be either a clean image or an image attacked with an adversarialpatch.
Revised: Let x (cid:48) be a test-time input which can be either a clean image or an image attacked with an adversarial patch.

Original: We know that there exists a clean image x corresponding to x (cid:48) which removes the patch if it ispresent.
Revised: We know that there exists a clean image x corresponding to x (cid:48) which removes the patch if it is present.

Original: We have either x (cid:48)
Revised: We have either x (cid:48)

Original: = x or x (cid:48) ∈
Revised: = x or x (cid:48) ∈

Original: A ( x ) , where A ( x ) :=
Revised: A ( x ) , where A ( x ) :=

Original: { A ( x, p, l ) | ( p, l ) ∈ P} .
Revised: { A ( x, p, l ) | ( p, l ) ∈ P} .

Original: However, attest time we do not have access to the clean image x .
Revised: However, at test time we do not have access to the clean image x .

Original: Our goal is to certify that for our segmentation model h and a pixel x i,j we have h ( x (cid:48) )
Revised: Our goal is to certify that for our segmentation model h and a pixel x i,j we have h ( x (cid:48) )

Original: i,j = h ( x ) i,j .
Revised: i,j = h ( x ) i,j .

Original: We can achieve this result by applying the recovery certiﬁcation (Theorem 1) to the test-time image.
Revised: We can achieve this result by applying the recovery certiﬁcation (Theorem 1) to the test-time image.

Original: It allows us to verify whether ∀ ( p, l ) ∈ P : h ( A ( x (cid:48) , p, l ))
Revised: It allows us to verify whether ∀ ( p, l ) ∈ P : h ( A ( x (cid:48) , p, l ))

Original: i,j = h ( x (cid:48) )
Revised: i,j = h ( x (cid:48) )

Original: i,j .
Revised: i,j .

Original: We also know that ifx (cid:48) ∈
Revised: We also know that if x (cid:48) ∈

Original: A ( x ) , then x ∈ A ( x (cid:48) )
Revised: A ( x ) , then x ∈ A ( x (cid:48) )

Original: (Figure 7a).
Revised: (Figure 7a).

Original: Indeed, if x (cid:48) is only different from x by one patch, then xcan be be obtained from x (cid:48) by removing this patch.
Revised: Indeed, if x (cid:48) is only different from x by one patch, then x can be be obtained from x (cid:48) by removing this patch.

Original: Therefore, by obtaining the guarantee for A ( x (cid:48) ) ,we implicitly obtain the guarantee also for the image x even though we do not have direct access to it.
Revised: Therefore, by obtaining the guarantee for A ( x (cid:48) ) , we implicitly obtain the guarantee also for the image x even though we do not have direct access to it.

Original: We note that this test-time guarantee is only possible for certiﬁed recovery.
Revised: We note that this test-time guarantee is only possible for certiﬁed recovery.

Original: In certiﬁed detection, wewould need to evaluate the veriﬁcation function v (Theorem 2) for both the clean image x and theattacked image x (cid:48) to obtain the result.
Revised: In certiﬁed detection, we would need to evaluate the veriﬁcation function v (Theorem 2) for both the clean image x and the attacked image x (cid:48) to obtain the result.

Original: This cannot be done if x is implicit.
Revised: This cannot be done if x is implicit.

Original: D.2 Robustness guarantees evaluation
Revised: D.2 Robustness guarantees evaluation

Original: The typical certiﬁed robust error for a given test data set (and pixel ( i, j ) in the segmentation case) isan estimate for
Revised: The typical certiﬁed robust error for a given test data set (and pixel ( i, j ) in the segmentation case) is an estimate for

Original: E [Table]
Revised: [Equation]

Original: [Figure]
Revised: [Figure]

Original: where D is the data generating probability measure and we assume that our test set to be an i.i.d.sample of it.
Revised: where D is the data generating probability measure and we assume that our test set to be an i.i.d. sample of it.

Original: This is the expected robust error (worst case over our threat model P for clean inputs)for a given pixel ( i, j ) .
Revised: This is the expected robust error (worst case over our threat model P for clean inputs) for a given pixel ( i, j ) .

Original: Using the test sample to get an estimate of this quantity, we get a probabilisticguarantee that the corresponding pixel ( i, j ) of a new clean test sample x (cid:48) drawn i.i.d.
Revised: Using the test sample to get an estimate of this quantity, we get a probabilistic guarantee that the corresponding pixel ( i, j ) of a new clean test sample x (cid:48) drawn i.i.d.

Original: from P willhave its whole “patch”-neighborhood certiﬁed.
Revised: from P will have its whole “patch”-neighborhood certiﬁed.

Original: However, more important for a practical security analysis is that we can certify a given instance, whichcan be even potentially adversarially perturbed.
Revised: However, more important for a practical security analysis is that we can certify a given instance, which can be even potentially adversarially perturbed.

Original: Formally, this means that for an input z ∈ A ( x ) ,where x ∼ P is an unknown sample from P , that we guarantee∀ ( p, l
Revised: Formally, this means that for an input z ∈ A ( x ) , where x ∼ P is an unknown sample from P , that we guarantee

Original: ) ∈ P : h ( A ( z, p, l ))
Revised: [Equation]

Original: i,j = h ( z ) i,j , and as x ∈ A ( z, p, l ) this implies that we certify that the pixel ( i, j ) of the potentially manipulatedimage is classiﬁed the same as pixel ( i, j ) of the unperturbed image x .
Revised: and as x ∈ A ( z, p, l ) this implies that we certify that the pixel ( i, j ) of the potentially manipulated image is classiﬁed the same as pixel ( i, j ) of the unperturbed image x .

Original: However, it is now tricky to get even a probabilistic estimate of the quantity
Revised: However, it is now tricky to get even a probabilistic estimate of the quantity

Original: [Equation]
Revised: [Equation]

Original: as the outer maximization process cannot be simply simulated by doing adversarial patch attacks on aclean test dataset.
Revised: as the outer maximization process cannot be simply simulated by doing adversarial patch attacks on a clean test dataset.

Original: We propose a way to evaluate a guaranteed lower bound on the fraction of certiﬁed test-time inputsby using a dataset of clean images.
Revised: We propose a way to evaluate a guaranteed lower bound on the fraction of certiﬁed test-time inputs by using a dataset of clean images.

Original: Instead of considering a standard one-patch neighbourhood
Revised: Instead of considering a standard one-patch neighbourhood

Original: A ( x ) deﬁned by our threat model (Section 3), we propose to consider a neighbourhood
Revised: A ( x ) deﬁned by our threat model (Section 3), we propose to consider a neighbourhood

Original: A 2 ( x )of two independent patches (Figure 7b).
Revised: [Equation] ) of two independent patches (Figure 7b).

Original: A 2 ( x ) contains all the images x
Revised: A 2 ( x ) contains all the images x

Original: (cid:48) ∈
Revised: (cid:48) ∈

Original: A ( x ) as well astheir respective patch neighbourhoods A ( x (cid:48) ) .
Revised: A ( x ) as well as their respective patch neighbourhoods A ( x (cid:48) ) .

Original: Therefore, by verifying that ∀ ( p 1 , l 1 ) , ( p 2 , l 2 ) ∈
Revised: Therefore, by verifying that ∀ ( p 1 , l 1 ) , ( p 2 , l 2 ) ∈

Original: P : h ( A ( A ( x, p 1 , l 1 ) , p 2 , l 2 ))
Revised: P : h ( A ( A ( x, p 1 , l 1 ) , p 2 , l 2 ))

Original: i,j = h ( x ) i,j , we guarantee that ∀ x (cid:48) ∈
Revised: i,j = h ( x ) i,j , we guarantee that ∀ x (cid:48) ∈

Original: A ( x ) ∀ ( p, l
Revised: A ( x ) ∀ ( p, l

Original: ) ∈ P :h ( A ( x (cid:48) , p, l ))
Revised: ) ∈ P : h ( A ( x (cid:48) , p, l ))

Original: i,j = h ( x (cid:48) )
Revised: i,j = h ( x (cid:48) )

Original: i,j .
Revised: i,j .

Original: We note that corresponding reasoning could be applied to certiﬁcation in (cid:96) p models.
Revised: We note that corresponding reasoning could be applied to certiﬁcation in (cid:96) p models.

Original: Then A 2 ( x )would correspond to doubling the radius of the (cid:15) -ball instead of adding a second patch.
Revised: Then A 2 ( x ) would correspond to doubling the radius of the (cid:15) -ball instead of adding a second patch.

Original: Note that Theorem 1 can be directly extended to a threat model of N patches.
Revised: Note that Theorem 1 can be directly extended to a threat model of N patches.

Original: In the worst case eachof the N patches can affect T different maskings.
Revised: In the worst case each of the N patches can affect T different maskings.

Original: Therefore, we need to change the condition of
Revised: Therefore, we need to change the condition of

Original: Theorem 1 to K ≥ 2 NT + 1 .
Revised: Theorem 1 to K ≥ 2 NT + 1 .

Original: We apply the described method to evaluating the test-time certiﬁcationguarantees for a toy example of a 0 .
Revised: We apply the described method to evaluating the test-time certiﬁcation guarantees for a toy example of a 0 .

Original: 1% patch in Table 3.
Revised: 1% patch in Table 3.

Original: We also illustrate how a column mask looksin this case in Figure 7.
Revised: We also illustrate how a column mask looks in this case in Figure 7.

Original: [Table]
Revised: [Table]

Original: E Adversarial patch example
Revised: E Adversarial patch example

Original: In this section, we demonstrate an example of a real adversarial patch for a semantic segmentationmodel similar to the one illustrated in the Figure 1a and show how it is handled by our certiﬁed
Revised: In this section, we demonstrate an example of a real adversarial patch for a semantic segmentation model similar to the one illustrated in the Figure 1a and show how it is handled by our certiﬁed

Original: [Figure]
Revised: [Figure]

Original: defences.
Revised: defences.

Original: We illustrate it for the Swin [26] model on one of the images from the ADE20K [31]dataset.
Revised: We illustrate it for the Swin [26] model on one of the images from the ADE20K [31] dataset.

Original: E.1 Patch optimization
Revised: E.1 Patch optimization

Original: We set the patch size to 1% of the image surface.
Revised: We set the patch size to 1% of the image surface.

Original: We select a ﬁxed position for a patch on the rearwindow of a car (Figure 8a).
Revised: We select a ﬁxed position for a patch on the rear window of a car (Figure 8a).

Original: For each pixel we extract a list of predicted logits corresponding to eachclass and apply multi-margin loss with respect to the ground truth label of the respective pixel.
Revised: For each pixel we extract a list of predicted logits corresponding to each class and apply multi-margin loss with respect to the ground truth label of the respective pixel.

Original: We userandom patch initialization without restarts.
Revised: We use random patch initialization without restarts.

Original: As an optimizer we use projected gradient descent (PGD)with 1000 steps and initial step size of 0.01.
Revised: As an optimizer we use projected gradient descent (PGD) with 1000 steps and initial step size of 0.01.

Original: We use cosine step size schedule and momentum for thegradient with the rate of 0.9.
Revised: We use cosine step size schedule and momentum for the gradient with the rate of 0.9.

Original: The optimization plot and the patch efﬁciency at different iterations ofthe PGD are illustrated in the Figure 8.
Revised: The optimization plot and the patch efﬁciency at different iterations of the PGD are illustrated in the Figure 8.

Original: E.2 Certiﬁed recovery
Revised: E.2 Certiﬁed recovery

Original: We denote the original image as x and the patched image as x (cid:48) .
Revised: We denote the original image as x and the patched image as x (cid:48) .

Original: The voting-based segmentationfunction h (Section 4.2) provides the majority-vote prediction h ( x ) and the corresponding certiﬁcationmap which shows the pixels where all the votes agree.
Revised: The voting-based segmentation function h (Section 4.2) provides the majority-vote prediction h ( x ) and the corresponding certiﬁcation map which shows the pixels where all the votes agree.

Original: In Figure we see that a part of the buildingand the road is certiﬁed which means that this prediction cannot be affected by an adversarial patch.
Revised: In Figure we see that a part of the building and the road is certiﬁed which means that this prediction cannot be affected by an adversarial patch.

Original: Figure demonstrates h ( x (cid:48) ) which correctly segments those regions in presence of an adversarial patchthat fools the original model.
Revised: Figure demonstrates h ( x (cid:48) ) which correctly segments those regions in presence of an adversarial patch that fools the original model.

Original: E.3 Certiﬁed detection
Revised: E.3 Certiﬁed detection

Original: We perform our analysis by evaluating the veriﬁcation map v (Section 4.2) for the original imagex and for the patched image x (cid:48) .
Revised: We perform our analysis by evaluating the veriﬁcation map v (Section 4.2) for the original image x and for the patched image x (cid:48) .

Original: We see that in v ( x ) a major part of the building is certiﬁed i. e.for a part of pixels x i,j that belong to the building and the road we have v ( x ) i,j = 1 .
Revised: We see that in v ( x ) a major part of the building is certiﬁed i. e. for a part of pixels x i,j that belong to the building and the road we have v ( x ) i,j = 1 .

Original: However,v ( x (cid:48) )
Revised: However, v ( x (cid:48) )

Original: i,j = 0 for those pixels.
Revised: i,j = 0 for those pixels.

Original: It means that we have detected that the prediction on this input ispotentially affected by an adversarial patch.
Revised: It means that we have detected that the prediction on this input is potentially affected by an adversarial patch.

Original: F Additional experiments
Revised: F Additional experiments

Original: In Tables 4 and 5, we provide additional experimental results for evaluating different masking schemesproposed in Section 4.1 on different models.
Revised: In Tables 4 and 5, we provide additional experimental results for evaluating different masking schemes proposed in Section 4.1 on different models.

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: For an attacked image  detects the region of f (  ) which was (potentially) affected by an adversarial patch.
Revised: For an attacked image x (cid:48) v ( x (cid:48) ) detects the region of f ( x (cid:48) ) which was (potentially) affected by an adversarial patch.

Original: G Inpainting ablation studies
Revised: G Inpainting ablation studies

Original: We perform ablation studies with respect to the demasking step.
Revised: We perform ablation studies with respect to the demasking step.

Original: The results are in Table 6.
Revised: The results are in Table 6.

Original: Figure 11provides additional illustrations.
Revised: Figureprovides additional illustrations.

Original: As can be seen from the results, our method heavily beneﬁts fromhaving available stronger inpainting models that allow achieving better clean and certiﬁed accuracy.
Revised: As can be seen from the results, our method heavily beneﬁts from having available stronger inpainting models that allow achieving better clean and certiﬁed accuracy.

Original: We consider this property actually as a strength of our method since it will automatically beneﬁt fromfuture research and developments of stronger inpainting methods.
Revised: We consider this property actually as a strength of our method since it will automatically beneﬁt from future research and developments of stronger inpainting methods.

Original: For certiﬁed recovery, we alsocompare it to GIN [43] based on a generative model that we trained on ADE20K (without using stylelosses based on ImageNet trained VGG).
Revised: For certiﬁed recovery, we also compare it to GIN [43] based on a generative model that we trained on ADE20K (without using style losses based on ImageNet trained VGG).

Original: The results are in Table 7.
Revised: The results are in Table 7.

Original: Illustrations can be found in
Revised: Illustrations can be found in

Original: Figure 12.
Revised: Figure 12.

Original: [Table] [Table]
Revised: [Table] [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table] [Table]
Revised: [Table] [Table]

Original: GIN [43]
Revised: GIN [43]

Original: ADE20K 5.
Revised: ADE20K 5.

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: H Comparison to simpliﬁed Derandomized Smoothing
Revised: H Comparison to simpliﬁed Derandomized Smoothing

Original: Derandomized Smoothing (DRS) [9] was proposed for certiﬁed recovery, therefore in this sectionwe focus on this task.
Revised: Derandomized Smoothing (DRS) [9] was proposed for certiﬁed recovery, therefore in this section we focus on this task.

Original: Direct adaptation of derandomized smoothing to semantic segmentation taskrequires training a model that is able to predict the full image segmentation from a small visibleregion.
Revised: Direct adaptation of derandomized smoothing to semantic segmentation task requires training a model that is able to predict the full image segmentation from a small visible region.

Original: Since it is not immediately clear to us what architectural design and training procedure wouldbe needed to train such a model, we consider a simpliﬁed version of DRS that we call DRS-S. Inthis version, we consider an off-the-shelf semantic segmentation model and evaluate how it performswith column masking from DRS.
Revised: Since it is not immediately clear to us what architectural design and training procedure would be needed to train such a model, we consider a simpliﬁed version of DRS that we call DRS-S. In this version, we consider an off-the-shelf semantic segmentation model and evaluate how it performs with column masking from DRS.

Original: Therefore, we do not encode the masked regions with the special’NULL’ value like in DRS but use black color instead.
Revised: Therefore, we do not encode the masked regions with the special ’NULL’ value like in DRS but use black color instead.

Original: That is because an off-the-shelf model cannotwork with ’NULL’ values.
Revised: That is because an off-the-shelf model cannot work with ’NULL’ values.

Original: We run our experiments on ADE20K dataset.
Revised: We run our experiments on ADE20K dataset.

Original: We consider the DRS parameters from the recent SOTAversion of Derandomized Smoothing by Salman et al.
Revised: We consider the DRS parameters from the recent SOTA version of Derandomized Smoothing by Salman et al.

Original: They use column width b = 19 andstride s = 10 for certiﬁed classiﬁcation of 224x224 ImageNet images.
Revised: They use column width b = 19 and stride s = 10 for certiﬁed classiﬁcation of 224x224 ImageNet images.

Original: To account for the fact that
Revised: To account for the fact that

Original: ADE20K images have larger resolution than ImageNet, we scale the parameters to column widthb = 42 and stride s = 22 .
Revised: ADE20K images have larger resolution than ImageNet, we scale the parameters to column width b = 42 and stride s = 22 .

Original: To make the comparison consistent with the rest of our results, we use thepatch occupying 0 .
Revised: To make the comparison consistent with the rest of our results, we use the patch occupying 0 .

Original: 5% of the image.
Revised: 5% of the image.

Original: From Table 8 we can see that DRS-S performs poorly on semantic segmentation task.
Revised: From Table 8 we can see that DRS-S performs poorly on semantic segmentation task.

Original: The reason forthat is illustrated in Figure 13.
Revised: The reason for that is illustrated in Figure 13.

Original: Processing the column region in 13c would probably be sufﬁcient fora classiﬁcation model to classify the image into the class "house".
Revised: Processing the column region in 13c would probably be sufﬁcient for a classiﬁcation model to classify the image into the class "house".

Original: But it is clearly not sufﬁcient toreconstruct the whole segmentation map 13e as can be seen in the Figure 13g. Whether doing thiswould be possible with a model speciﬁcally trained to reconstruct the segmentation map from a verysmall visible region is an open research question (up to our knowledge).
Revised: But it is clearly not sufﬁcient to reconstruct the whole segmentation map 13e as can be seen in the Figure 13g. Whether doing this would be possible with a model speciﬁcally trained to reconstruct the segmentation map from a very small visible region is an open research question (up to our knowledge).

Original: We point out that the value %C of certiﬁed and correctly classiﬁed pixels in the Table 8 is stillsurprisingly high for DRS-S compared to other metircs.
Revised: We point out that the value %C of certiﬁed and correctly classiﬁed pixels in the Table 8 is still surprisingly high for DRS-S compared to other metircs.

Original: We attribute this to the fact that the solidblack regions are usually treated as a wall by the segmentation model, therefore the images are usuallysegmented as a wall by the DRS majority voting.
Revised: We attribute this to the fact that the solid black regions are usually treated as a wall by the segmentation model, therefore the images are usually segmented as a wall by the DRS majority voting.

Original: And the wall is a common part of both indoor andoutdoor scenes in ADE20K as can be implied from the Table 9 of "big" ADE20K classes.
Revised: And the wall is a common part of both indoor and outdoor scenes in ADE20K as can be implied from the Table 9 of "big" ADE20K classes.

Original: Therefore,always classifying the output as a wall provides a decent fraction of correctly classiﬁed pixels becauseof the skewed classes.
Revised: Therefore, always classifying the output as a wall provides a decent fraction of correctly classiﬁed pixels because of the skewed classes.

Original: However, to provide a better comparison with DRS, we emulate the model which is able to reconstructthe whole segmentation map from the column masking proposed in DRS.
Revised: However, to provide a better comparison with DRS, we emulate the model which is able to reconstruct the whole segmentation map from the column masking proposed in DRS.

Original: We do this by applying thedemasking approach proposed in this work.
Revised: We do this by applying the demasking approach proposed in this work.

Original: We ﬁrst try to reconstruct the whole image from onecolumn and then segment it with an off-the-shelf model as we did with the masks proposed in thispaper.
Revised: We ﬁrst try to reconstruct the whole image from one column and then segment it with an off-the-shelf model as we did with the masks proposed in this paper.

Original: We call this approach DRS-E and the results can be found in Table 8.
Revised: We call this approach DRS-E and the results can be found in Table 8.

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: I A list of big classes
Revised: I A list of big classes

Original: In Section C we suggest another perspective on the evaluation of our D EMASKED S MOOTHING byspeciﬁcally considering its performance on “big” semantic classes.
Revised: In Section C we suggest another perspective on the evaluation of our D EMASKED S MOOTHING by speciﬁcally considering its performance on “big” semantic classes.

Original: The object of these classes occupyon average more than 20% of the images in which they appear.
Revised: The object of these classes occupy on average more than 20% of the images in which they appear.

Original: Correctly segmenting these classes isimportant for understanding the scene.
Revised: Correctly segmenting these classes is important for understanding the scene.

Original: In Tables 9 and 10 we provide the full list of such classes in
Revised: In Tables 9 and 10 we provide the full list of such classes in

Original: ADE20K [31] and COCO-Stuff-10K [39] respectively together with the average fraction of pixelsthat they occupy in the images in which they are present.
Revised: ADE20K [31] and COCO-Stuff-10K [39] respectively together with the average fraction of pixels that they occupy in the images in which they are present.

Original: We point out that for COCO-Stuff-10Ksome typically smaller classes such as “sandwich” or “fruit” get included in the list of big classesbecause of the macro-scale images in which they occupy a big part of the scene.
Revised: We point out that for COCO-Stuff-10K some typically smaller classes such as “sandwich” or “fruit” get included in the list of big classes because of the macro-scale images in which they occupy a big part of the scene.

Original: J Complexity analysis and parallelization
Revised: J Complexity analysis and parallelization

Original: In D EMASKED S MOOTHING , we propose a set of K masks that are applied to the original image(denote the cost of applying a single masking by M ).
Revised: In D EMASKED S MOOTHING , we propose a set of K masks that are applied to the original image (denote the cost of applying a single masking by M ).

Original: As illustrated in Figure 1c, the maskedimages are demasked (denote the cost of demasking an image by D ) and segmented (denote thecost of segmenting an image by S ); thereupon per-mask segmentations are aggregated into a ﬁnalsegmentation and certiﬁcation (cost of aggregation proportional to K ).
Revised: As illustrated in Figure 1c, the masked images are demasked (denote the cost of demasking an image by D ) and segmented (denote the cost of segmenting an image by S ); thereupon per-mask segmentations are aggregated into a ﬁnal segmentation and certiﬁcation (cost of aggregation proportional to K ).

Original: Asymptotically, computegrows thus with O ( K ( M + D + S )
Revised: Asymptotically, compute grows thus with O ( K ( M + D + S )

Original: + K ) while the cost of a standard segmentation is O ( S ) .
Revised: + K ) while the cost of a standard segmentation is O ( S ) .

Original: Thus,for large K or M + D (cid:29) S , real-time applicability would actually be impractical.
Revised: Thus, for large K or M + D (cid:29) S , real-time applicability would actually be impractical.

Original: However, we notethat:
Revised: However, we note that:

Original: M + D is roughly of the same size as S for typical DL-based inpainting and segmentation models.
Revised: M + D is roughly of the same size as S for typical DL-based inpainting and segmentation models.

Original: For certiﬁed recovery, we operate in a setting where K is small ( K ∈ 5 , 7 , 9 ) and does not grow with the image resolution.
Revised: For certiﬁed recovery, we operate in a setting where K is small ( K ∈ 5 , 7 , 9 ) and does not grow with the image resolution.

Original: This is unlike Derandomized Smoothing and its derivatives, where the number of masks in the recovery task grows with the image resolution(or randomized smoothing
Revised: This is unlike Derandomized Smoothing and its derivatives, where the number of masks in the recovery task grows with the image resolution

Original: with thousands of samples per input).

Original: This small value of K beneﬁts our the

Original: method in time sensitive

Original: [Table] [Table]
Revised: [Table] [Table]

Original: broccoli 20.
Revised: broccoli 20.

Original: sand 26.
Revised: sand 26.

Original: pizza 25.
Revised: pizza 25.

Original: sea 36.
Revised: sea 36.

Original: bed 36.
Revised: bed 36.

Original: sky-other 22.
Revised: sky-other 22.

Original: dining table 21.
Revised: dining table 21.

Original: snow 51.
Revised: snow 51.

Original: clouds 24.
Revised: clouds 24.

Original: vegetable 20.
Revised: vegetable 20.

Original: water-other 21.
Revised: water-other 21.

Original:  applications.
Revised: (or randomized smoothing with thousands of samples per input). This small value of K beneﬁts our the method in time-sensitive applications.

Original: For certiﬁed detection, we can adjust the number of masks for the computational speedby using strided masking as suggested in Section 4.1.3.
Revised: For certiﬁed detection, we can adjust the number of masks for the computational speed by using strided masking as suggested in Section 4.1.

Original: Morover, masking, demasking, and segmenting for different masks do not use any shared dataand can thus be fully parallelizedif sufﬁciently powerful hardware is available.
Revised: Moreover, masking, demasking and segmenting for different masks do not use any shared data and can thus be fully parallelized

Revised: if

Revised: sufﬁciently powerful hardware is available

Original: Only the aggregationstep requires the resultsof all the previous stages.
Revised: Only the aggregation step requires the results

Revised: of all

Revised: the

Revised: previous stages

Original: However, aggregation time is small compared tothe other stages.
Revised: However aggregation time is small compared to the other stages

Original: In terms of latency, a fully parallelized version of our procedure would thus have alatency proportional to O 
Revised: In terms of latency a fully parallelized version of our procedure would thus have a latency proportional to O (

Original: ( M + D + S + K) .
Revised: M + D + S + K

Original: For small K and M + D ≈ S , application to real-timevideo can be facilitated by means of parallelization.
Revised: For small K and M + D ≈ S , application to real-time video can be facilitated by means of parallelization.

Original: K Used data
Revised: K Used data

Original: In this work, we only use the datasets published under formal licenses: ADE20K [31] and COCOStuff-10K [39].
Revised: In this work, we only use the datasets published under formal licenses: ADE20K [31] and COCOStuff-10K [39].

Original: To the best of our knowledge, data used in this project do not contain any personallyidentiﬁable information or offensive content.
Revised: To the best of our knowledge, data used in this project do not contain any personally identiﬁable information or offensive content.

Original: The models ZITS [29] and Swin [26] are publishedunder Apache-2.0 license.
Revised: The models ZITS [29] and Swin [26] are published under Apache-2.0 license.

Original: The text of the license for PSPNet [24] can be found here: https://github.com/hszhao/PSPNet/blob/master/LICENSE
Revised: The text of the license for PSPNet [24] can be found here: https: //github.com/hszhao/PSPNet/blob/master/LICENSE [Table]

Original: L Demasked Smoothing Visualization
Revised: L Demasked Smoothing Visualization

Original: In this section, we provide additional illustrations of our method (Figures 14, 15, 16, 17).
Revised: In this section [Table] we provide additional illustrations of our method (Figures 14, 15, 16, 17).

Original: Similarlyto the Table [-
-] we certify against a 1% patch for the detection task and against 0.5% patch for therecovery task.
Revised: Similarly to the Table [-
-]1 we certify against a 1% patch for the detection task and against 0.5% patch for the recovery task.

Original: For each mask type we illustrate all the stages summarized in the Figure 1c.
Revised: For each mask type we illustrate all the stages summarized in the Figure 1c.

Original: We alsoprovide examples of certiﬁcation maps for certiﬁed recovery and certiﬁed detection with differentimages (Figure 18, 19).
Revised: We also provide examples of certiﬁcation maps for certiﬁed recovery and certiﬁed detection with different images (Figure 18, 19).

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: [Figure] [Figure]
Revised: [Figure] [Figure]