Original: A model is strictly local if each of its outputs f n only operates on a well-deﬁned subset of the input data.
Revised: A model is strictly local if each of its outputs f n only operates on a well-deﬁned subset of the input data.

Original: To encode this strict locality, Schuchardt et al.
Revised: To encode this strict locality, Schuchardt et al.

Original: (2021) associate each output f n with an indicator vector ψ ( n ) and an indicator matrix Ψ ( n ) that fulﬁll
Revised: (2021) associate each output f n with an indicator vector ψ ( n ) and an indicator matrix Ψ ( n ) that fulﬁll

Original: [Equation]
Revised: [Equation]

Original: [Equation]
Revised: [Equation]

Original: for any perturbed graph G (cid:48) =
Revised: for any perturbed graph G (cid:48) =

Original: ( X (cid:48) , A (cid:48) ) .
Revised: ( X (cid:48) , A (cid:48) ) .

Original: Eq.
Revised: Eq.

Original: 116 expresses that the prediction of output f n remains unchanged if all inputs in its receptive ﬁeld remain unchanged.
Revised: 116 expresses that the prediction of output f n remains unchanged if all inputs in its receptive ﬁeld remain unchanged.

Original: Conversely, it expresses that perturbations outside the receptive ﬁeld can be ignored.
Revised: Conversely, it expresses that perturbations outside the receptive ﬁeld can be ignored.

Original: Unlike in our work, Schuchardt et al.
Revised: Unlike in our work, Schuchardt et al.

Original: (2021) describe their base certiﬁcates as sets in adversarial budget space.
Revised: (2021) describe their base certiﬁcates as sets in adversarial budget space.

Original: That is, some certiﬁcation procedure is applied to each output f n to obtain a set
Revised: That is, some certiﬁcation procedure is applied to each output f n to obtain a set

Original: [Equation]
Revised: [Equation]

Original: ] (117) with [ k ] = { 0 , . . . , k } .
Revised: ] (117) with [ k ] = { 0 , . . . , k } .

Original: If (cid:2) c + X c − X c
Revised: If (cid:2) c + X c − X c

Original: + A c − A (cid:3) T ∈ K ( n ) , then prediction y n is robust to any perturbed input with exactly c + X attribute additions, c − X attribute deletions, c + A edge additions and c − A edge deletions.
Revised: + A c − A (cid:3) T ∈ K ( n ) , then prediction y n is robust to any perturbed input with exactly c + X attribute additions, c − X attribute deletions, c + A edge additions and c − A edge deletions.

Original: A more detailed explanation can be found in Section 3 of (Schuchardt et al., 2021).
Revised: A more detailed explanation can be found in Section 3 of (Schuchardt et al., 2021).

Original: Note that the base certiﬁcates only depend on the number of perturbations, not their location in the input.
Revised: Note that the base certiﬁcates only depend on the number of perturbations, not their location in the input.

Original: Only combining them using the receptive ﬁeld indicators from Eq.
Revised: Only combining them using the receptive ﬁeld indicators from Eq.

Original: 116 makes it possible to obtain a collective certiﬁcate that is better than a na¨ıve combination of the base certiﬁcates (i.e. counting how many predictions are certiﬁably robust to the collective threat model).
Revised: 116 makes it possible to obtain a collective certiﬁcate that is better than a na¨ıve combination of the base certiﬁcates (i.e. counting how many predictions are certiﬁably robust to the collective threat model).

Original: The resulting collective certiﬁcate is
Revised: The resulting collective certiﬁcate is

Original: [Equation]
Revised: [Equation]

Original: [Equation]
Revised: [Equation]

Original: The variables deﬁned in Eq.
Revised: The variables deﬁned in Eq.

Original: 120 model how the adversary allocates their adversarial budget, i.e. how many attributes are perturbed per node and which edges are modiﬁed.
Revised: 120 model how the adversary allocates their adversarial budget, i.e. how many attributes are perturbed per node and which edges are modiﬁed.

Original: Eq.
Revised: Eq.

Original: 119 ensures that this allocation in compliant with the collective threat model.
Revised: 119 ensures that this allocation in compliant with the collective threat model.

Original: Finally, in Eq.
Revised: Finally, in Eq.

Original: 118 the indicator vector and matrix ψ ( n ) and Ψ ( n ) are used to mask out any allocated perturbation budget that falls outside the receptive ﬁeld of f n before evaluating its base certiﬁcate.
Revised: 118 the indicator vector and matrix ψ ( n ) and Ψ ( n ) are used to mask out any allocated perturbation budget that falls outside the receptive ﬁeld of f n before evaluating its base certiﬁcate.

Original: To solve the optimization problem, Schuchardt et al.
Revised: To solve the optimization problem, Schuchardt et al.

Original: (2021) replace each of the indicator functions with binary variables and include additional constraints to ensure that they have value 1 i.f.f.
Revised: (2021) replace each of the indicator functions with binary variables and include additional constraints to ensure that they have value 1 i.f.f.

Original: the indicator function would have value 1 .
Revised: the indicator function would have value 1 .

Original: To do so, they deﬁne one linear constraint per point separating the set of certiﬁable budgets K ( n ) from its complement K ( n ) in adversarial budget space (the ”pareto front” discussed in Section 3 of (Schuchardt et al., 2021)).
Revised: To do so, they deﬁne one linear constraint per point separating the set of certiﬁable budgets K ( n ) from its complement K ( n ) in adversarial budget space (the ”pareto front” discussed in Section 3 of (Schuchardt et al., 2021)).

Original: From the above explanation, the main drawbacks of this collective certiﬁcate compared to our localized randomized smoothing approach and corresponding collective certiﬁcate should be clear.
Revised: From the above explanation, the main drawbacks of this collective certiﬁcate compared to our localized randomized smoothing approach and corresponding collective certiﬁcate should be clear.

Original: Firstly, if the classiﬁer f is not strictly local, i.e. the receptive ﬁeld indicators ψ and Ψ only have non-zero entries, then all base certiﬁcates are evaluated using the entire collective adversarial budget.
Revised: Firstly, if the classiﬁer f is not strictly local, i.e. the receptive ﬁeld indicators ψ and Ψ only have non-zero entries, then all base certiﬁcates are evaluated using the entire collective adversarial budget.

Original: It thus degenerates to the na ¨ ıve collective certiﬁcate.
Revised: It thus degenerates to the na ¨ ıve collective certiﬁcate.

Original: Secondly, even if the model is strictly local, each of the output may assign varying levels of importance to different parts of its receptive ﬁeld.
Revised: Secondly, even if the model is strictly local, each of the output may assign varying levels of importance to different parts of its receptive ﬁeld.

Original: Their method is incapable of capturing this additional soft locality.
Revised: Their method is incapable of capturing this additional soft locality.

Original: Finally, their means of evaluating the base certiﬁcates may involve evaluating a large number of linear constraints.
Revised: Finally, their means of evaluating the base certiﬁcates may involve evaluating a large number of linear constraints.

Original: Our method, on the other hand, only requires a single constraint per prediction.
Revised: Our method, on the other hand, only requires a single constraint per prediction.

Original: Our collective certiﬁcate can thus be more efﬁciently computed.
Revised: Our collective certiﬁcate can thus be more efﬁciently computed.

Original: D. P ROOF OF SUBSUMPTION
Revised: D. P ROOF OF SUBSUMPTION

Original: In the following, we show that any robustness certiﬁcate obtained by using the collective certiﬁcate of Schuchardt et al.
Revised: In the following, we show that any robustness certiﬁcate obtained by using the collective certiﬁcate of Schuchardt et al.

Original: (2021) with sparsity-aware randomized smoothing base certiﬁcates can also be obtained by using our proposed collective certiﬁcate with an appropriately parameterized localized smoothing distribution.
Revised: (2021) with sparsity-aware randomized smoothing base certiﬁcates can also be obtained by using our proposed collective certiﬁcate with an appropriately parameterized localized smoothing distribution.

Original: The fundamental idea is that, for randomly smoothed models, completely randomizing all input dimensions outside a receptive ﬁeld is equivalent to masking out any perturbations outside the receptive ﬁeld.
Revised: The fundamental idea is that, for randomly smoothed models, completely randomizing all input dimensions outside a receptive ﬁeld is equivalent to masking out any perturbations outside the receptive ﬁeld.

Original: First, we derive the certiﬁcate of Schuchardt et al.
Revised: First, we derive the certiﬁcate of Schuchardt et al.

Original: (2021) for predictions obtained via sparsityaware smoothing.
Revised: (2021) for predictions obtained via sparsityaware smoothing.

Original: Schuchardt et al.
Revised: Schuchardt et al.

Original: (2021) require base certiﬁcates that guarantee robustness when (cid:2) c
Revised: (2021) require base certiﬁcates that guarantee robustness when (cid:2) c

Original: + X c − X c
Revised: + X c − X c

Original: + A c − A (cid:3) T ∈ K ( n ) , where the c indicate the number of added and deleted attribute and adjacency bits.
Revised: + A c − A (cid:3) T ∈ K ( n ) , where the c indicate the number of added and deleted attribute and adjacency bits.

Original: That is, the certiﬁcates must only depend on the number of perturbations, not on their location.
Revised: That is, the certiﬁcates must only depend on the number of perturbations, not on their location.

Original: To achieve this, all entries of the attribute matrix and all entries of the adjacency matrix, respectively, must share the same distribution.
Revised: To achieve this, all entries of the attribute matrix and all entries of the adjacency matrix, respectively, must share the same distribution.

Original: For the attribute matrix, they deﬁne scalar distribution parameters p + X , p − A ∈ [0 , 1] .
Revised: For the attribute matrix, they deﬁne scalar distribution parameters p + X , p − A ∈ [0 , 1] .

Original: Given attribute matrix X ∈ { 0 , 1 } N × D , they then sample random attribute matrices Z X that are distributed according to sparsity-aware smoothing distribution S (cid:0) X , 1 · p + X , 1 · p − X (cid:1) (see Section B.2.2), i.e.
Revised: Given attribute matrix X ∈ { 0 , 1 } N × D , they then sample random attribute matrices Z X that are distributed according to sparsity-aware smoothing distribution S (cid:0) X , 1 · p + X , 1 · p − X (cid:1) (see Section B.2.2), i.e.

Original: [Equation]
Revised: [Equation]

Original: Given input adjacency matrix A , random adjacency matrices Z A are sampled from the distribution
Revised: Given input adjacency matrix A , random adjacency matrices Z A are sampled from the distribution

Original: S (cid:0) A , 1 · p + A , 1 · p − A (cid:1) .
Revised: S (cid:0) A , 1 · p + A , 1 · p − A (cid:1) .

Original: Applying Theorem 2 (to the ﬂattened and concatenated attribute and adjacency matrices) shows that smoothed prediction y n = f n ( X , A ) is robust to the perturbed graph
Revised: Applying Theorem 2 (to the ﬂattened and concatenated attribute and adjacency matrices) shows that smoothed prediction y n = f n ( X , A ) is robust to the perturbed graph

Original: [Equation]
Revised: [Equation]

Original: if
Revised: if

Original: [Equation]
Revised: [Equation]

Original: 121) with γ + X = ln
Revised: 121) with γ + X = ln

Original: [Equation]
Revised: [Equation]

Original: p −
Revised: p −

Original: [Equation]
Revised: [Equation]

Original: p
Revised: p

Original: [Equation]
Revised: [Equation]

Original: where µ ( n ) is the mean and σ ( n ) is the variance of the base classiﬁer’s output distribution, given the input smoothing distribution.
Revised: where µ ( n ) is the mean and σ ( n ) is the variance of the base classiﬁer’s output distribution, given the input smoothing distribution.

Original: Since the indicator functions for each perturbation type in Eq.share the same weights, Eq.
Revised: Since the indicator functions for each perturbation type in Eq.share the same weights, Eq.

Original: 121 can be rewritten as
Revised: 121 can be rewritten as

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: where c + X , c − X , c + A , c − A are the overall number of added and deleted attribute and adjacency bits, respectively.
Revised: where c + X , c − X , c + A , c − A are the overall number of added and deleted attribute and adjacency bits, respectively.

Original: Eq.
Revised: Eq.

Original: 122 matches the notion of base certiﬁcates deﬁned by Schuchardt et al.
Revised: 122 matches the notion of base certiﬁcates deﬁned by Schuchardt et al.

Original: (2021), i.e. it corresponds to a set K ( n ) in adversarial budget space for which we provably know that prediction y n is certiﬁably robust if (cid:2) c
Revised: (2021), i.e. it corresponds to a set K ( n ) in adversarial budget space for which we provably know that prediction y n is certiﬁably robust if (cid:2) c

Original: + X
Revised: + X

Original: [Equation]
Revised: [Equation]

Original: When this base certiﬁcate is used, i.e. we insert the base certiﬁcate Eq. 122 into objective function Eq.
Revised: When this base certiﬁcate is used, i.e. we insert the base certiﬁcate Eq. 122 into objective function Eq.

Original: 118, the collective certiﬁcate of
Revised: 118, the collective certiﬁcate of

Original: Schuchardt et al.
Revised: Schuchardt et al.

Original: (2021) becomes equivalent to
Revised: (2021) becomes equivalent to

Original: [Equation]
Revised: [Equation]

Original: s.t.
Revised: s.t.

Original: N
Revised: N

Original: [Equation]
Revised: [Equation]

Original: m
Revised: m

Original: [Equation]
Revised: [Equation]

Original: (cid:88)
Revised: (cid:88)

Original: [Equation]
Revised: [Equation]

Original: N (cid:88)
Revised: N (cid:88)

Original: j
Revised: j

Original: [Equation]
Revised: [Equation]

Original: B
Revised: B

Original: [Equation]
Revised: [Equation]

Original: Next, we show that obtaining base certiﬁcates through localized randomized smoothing with appropriately chosen parameters and using these base certiﬁcates within our proposed collective certiﬁcate (see Section 6) will result in the same optimization problem.
Revised: Next, we show that obtaining base certiﬁcates through localized randomized smoothing with appropriately chosen parameters and using these base certiﬁcates within our proposed collective certiﬁcate (see Section 6) will result in the same optimization problem.

Original: Instead of using the same smoothing distribution for all outputs, we use different distribution parameters for each one.
Revised: Instead of using the same smoothing distribution for all outputs, we use different distribution parameters for each one.

Original: For the n ’th output, we sample random attributes matrices from distribution S (cid:16) X , Θ + X ( n ) , Θ − X ( n ) (cid:17) with Θ + X ( n ) , Θ − X ( n ) ∈
Revised: For the n ’th output, we sample random attributes matrices from distribution S (cid:16) X , Θ + X ( n ) , Θ − X ( n ) (cid:17) with Θ + X ( n ) , Θ − X ( n ) ∈

Original: [0 , 1] N × D .
Revised: [0 , 1] N × D .

Original: Note that, in order to avoid having to index ﬂattened vectors, we overload the deﬁnition of sparsity-aware smoothing to allow for matrix-valued parameters.
Revised: Note that, in order to avoid having to index ﬂattened vectors, we overload the deﬁnition of sparsity-aware smoothing to allow for matrix-valued parameters.

Original: For example, the value Θ + X ( n ) n,d indicates the probability of ﬂipping the value of input attribute X n,d from 0 toand the value Θ − X ( n ) n,d indicates the probability of ﬂipping the value of input attribute X n,d from 1 to 0 .
Revised: For example, the value Θ + X ( n ) n,d indicates the probability of ﬂipping the value of input attribute X n,d from 0 toand the value Θ − X ( n ) n,d indicates the probability of ﬂipping the value of input attribute X n,d from 1 to 0 .

Original: We choose the following values for these parameters:
Revised: We choose the following values for these parameters:

Original: [Equation]
Revised: [Equation]

Original: (127) where ψ ( n ) is the receptive ﬁeld indicator vector deﬁned in Eq.
Revised: (127) where ψ ( n ) is the receptive ﬁeld indicator vector deﬁned in Eq.

Original: 116 and p + X , · p − X ∈ [0 , 1] are the same ﬂip probabilities we used for the certiﬁcate of Schuchardt et al.
Revised: 116 and p + X , · p − X ∈ [0 , 1] are the same ﬂip probabilities we used for the certiﬁcate of Schuchardt et al.

Original: Due to this parameterization, attribute bits inside the receptive ﬁeld are randomized using the same distribution as in the certiﬁcate of Schuchardt et al.
Revised: Due to this parameterization, attribute bits inside the receptive ﬁeld are randomized using the same distribution as in the certiﬁcate of Schuchardt et al.

Original: (2021), while attribute bits outside are set to either
Revised: (2021), while attribute bits outside are set to either

Original: 0 or 1 with equal probability.
Revised: 0 or 1 with equal probability.

Original: Similarly, we sample random adjacency matrices from distribution
Revised: Similarly, we sample random adjacency matrices from distribution

Original: S (cid:16) A , Θ + A ( n ) , Θ − A ( n ) (cid:17) with Θ + A ( n ) , Θ − A ( n ) ∈ [0 , 1] N × D and
Revised: S (cid:16) A , Θ + A ( n ) , Θ − A ( n ) (cid:17) with Θ + A ( n ) , Θ − A ( n ) ∈ [0 , 1] N × D and

Original: [Equation]
Revised: [Equation]

Original: where Ψ ( n ) is the receptive ﬁeld indicator matrix deﬁned in Eq. 116.
Revised: where Ψ ( n ) is the receptive ﬁeld indicator matrix deﬁned in Eq. 116.

Original: Note that, since we only alter the distribution of bits outside the receptive ﬁeld, the smoothed prediction y n = f n ( X , A ) will be the same as the one obtained via the smoothing distribution used by Schuchardt et al.
Revised: Note that, since we only alter the distribution of bits outside the receptive ﬁeld, the smoothed prediction y n = f n ( X , A ) will be the same as the one obtained via the smoothing distribution used by Schuchardt et al.

Original: Applying Theorem 2 (to the ﬂattened and concatenated attribute and adjacency matrices) shows that smoothed prediction y n = f n ( X , A ) is robust to the perturbed graph ( X (cid:48) , A (cid:48) )
Revised: Applying Theorem 2 (to the ﬂattened and concatenated attribute and adjacency matrices) shows that smoothed prediction y n = f n ( X , A ) is robust to the perturbed graph ( X (cid:48) , A (cid:48) )

Original: if
Revised: if

Original: N
Revised: N

Original: [Equation]
Revised: [Equation]

Original: m
Revised: m

Original: [Equation]
Revised: [Equation]

Original: Because we only changed the distribution outside the receptive ﬁeld, the scalar η ( n ) , which depends on the output distribution’s mean and variance µ and σ will be the same as the one obtained via the smoothing scheme used by Schuchardt et al.
Revised: Because we only changed the distribution outside the receptive ﬁeld, the scalar η ( n ) , which depends on the output distribution’s mean and variance µ and σ will be the same as the one obtained via the smoothing scheme used by Schuchardt et al.

Original: et al.
Revised: et al.

Original: Due to Theorem 2 and the deﬁnition of our smoothing distribution parameters in Eqs.
Revised: Due to Theorem 2 and the deﬁnition of our smoothing distribution parameters in Eqs.

Original: (126) to (129) , the scalars τ + X m,d , τ − X m,d , τ
Revised: (126) to (129) , the scalars τ + X m,d , τ − X m,d , τ

Original: + A i,j , τ − A i,j have the following values:
Revised: + A i,j , τ − A i,j have the following values:

Original: [Equation]
Revised: [Equation]

Original: where the γ are the same weights as those of the base certiﬁcate Eq.
Revised: where the γ are the same weights as those of the base certiﬁcate Eq.

Original: 121 of Schuchardt et al.
Revised: 121 of Schuchardt et al.

Original: Inserting the above values of τ into the base certiﬁcate Eq.
Revised: Inserting the above values of τ into the base certiﬁcate Eq.

Original: 130 and using the fact
Revised: 130 and using the fact

Original: that ln
Revised: that ln

Original: [Equation]
Revised: [Equation]

Original: +
Revised: +

Original: [Equation]
Revised: [Equation]

Original: = ln
Revised: = ln

Original: [Equation]
Revised: [Equation]

Original: = 0 results in
Revised: = 0 results in

Original: N
Revised: N

Original: [Equation]
Revised: [Equation]

Original: While our collective certiﬁcate derived in Section 6 only considers one perturbation type, we have already discussed how to certify robustness to perturbation models where there are multiple perturbation types in Section B.2.2:
Revised: While our collective certiﬁcate derived in Section 6 only considers one perturbation type, we have already discussed how to certify robustness to perturbation models where there are multiple perturbation types in Section B.2.2:

Original: We use a different budget variable per input dimension and perturbation type.
Revised: We use a different budget variable per input dimension and perturbation type.

Original: Furthermore, the attribute bits of each node share the same noise level.
Revised: Furthermore, the attribute bits of each node share the same noise level.

Original: Therefore, we can use the method discussed in Section A.3, i.e. use a single budget variable per node instead of using one per node and attribute.
Revised: Therefore, we can use the method discussed in Section A.3, i.e. use a single budget variable per node instead of using one per node and attribute.

Original: Modelling our collective problem in this way, using Eq. 135 as our base certiﬁcates and rewriting the ﬁrst two sums using inner products results in the optimization problem
Revised: Modelling our collective problem in this way, using Eq. 135 as our base certiﬁcates and rewriting the ﬁrst two sums using inner products results in the optimization problem

Original: [Equation]
Revised: [Equation]

Original: This optimization problem is identical to that of Schuchardt et al.
Revised: This optimization problem is identical to that of Schuchardt et al.

Original: (2021) from Eqs.
Revised: (2021) from Eqs.

Original: (123) to (125).
Revised: (123) to (125).

Original: The only difference is in how these problems would be mapped to a mixed-integer linear program.
Revised: The only difference is in how these problems would be mapped to a mixed-integer linear program.

Original: We would directly model the indicator functions in the objective using a single linear constraint.
Revised: We would directly model the indicator functions in the objective using a single linear constraint.

Original: Schuchardt et al.
Revised: Schuchardt et al.

Original: (2021) would use multiple linear constraints, each corresponding to one point in the adversarial budget space.
Revised: (2021) would use multiple linear constraints, each corresponding to one point in the adversarial budget space.

Original: To summarize: For randomly smoothed models, masking out perturbations using a-priori knowledge about a model’s strict locality is equivalent to completely randomizing (here: ﬂipping bits with probability 50% ) parts of the input.
Revised: To summarize: For randomly smoothed models, masking out perturbations using a-priori knowledge about a model’s strict locality is equivalent to completely randomizing (here: ﬂipping bits with probability 50% ) parts of the input.

Original: While Schuchardt et al.
Revised: While Schuchardt et al.

Original: (2021) only derived their certiﬁcate for binary data, it is conceivable that their approach could be applied to strictly local models for continuous data.
Revised: (2021) only derived their certiﬁcate for binary data, it is conceivable that their approach could be applied to strictly local models for continuous data.

Original: Considering our certiﬁcates for Gaussian (Proposition 1) and (Proposition 2) uniform smoothing, where the base certiﬁcate weights are 1 σ 2 and 1 λ , respectively, it should again be possible to perform the same masking operation as Schuchardt et al.
Revised: Considering our certiﬁcates for Gaussian (Proposition 1) and (Proposition 2) uniform smoothing, where the base certiﬁcate weights are 1 σ 2 and 1 λ , respectively, it should again be possible to perform the same masking operation as Schuchardt et al.

Original: (2021) by using σ → ∞ and λ → ∞ .
Revised: (2021) by using σ → ∞ and λ → ∞ .

Original: Finally, it should be noted that the certiﬁcate by Schuchardt et al.
Revised: Finally, it should be noted that the certiﬁcate by Schuchardt et al.

Original: (2021) allows for additional constraints, e.g. on the adversarial budget per node or the number of nodes controlled by the adversary.
Revised: (2021) allows for additional constraints, e.g. on the adversarial budget per node or the number of nodes controlled by the adversary.

Original: As all of them can be modelled using linear constraints on the budget variables (see Section C of their paper), they can be just as easily integrated into our mixed-integer linear programming certiﬁcate.
Revised: As all of them can be modelled using linear constraints on the budget variables (see Section C of their paper), they can be just as easily integrated into our mixed-integer linear programming certiﬁcate.

Original: E E XPERIMENTAL SETUP AND ADDITIONAL EXPERIMENTS
Revised: E E XPERIMENTAL SETUP AND ADDITIONAL EXPERIMENTS

Original: In the following, we ﬁrst explain the metrics we use for measuring the strength of certiﬁcates, and how they can be applied to the different types of randomized smoothing certiﬁcates used in our experiments.
Revised: In the following, we ﬁrst explain the metrics we use for measuring the strength of certiﬁcates, and how they can be applied to the different types of randomized smoothing certiﬁcates used in our experiments.

Original: We then discuss the experimental setup, as well hyperparameters and additional experimental results for our semantic segmentation Section E.2 and node classiﬁcation Section 8.
Revised: We then discuss the experimental setup, as well hyperparameters and additional experimental results for our semantic segmentation Section E.2 and node classiﬁcation Section 8.

Original: experiments.
Revised: experiments.

Original: E. M ETRICS
Revised: E. M ETRICS

Original: We use two metrics for measuring certiﬁcate strength: Certiﬁed accuracy (i.e. the percentage of correct and certiﬁably robust predictions) and certiﬁed ratio (i.e. the percentage of certiﬁably robust predictions, regardless of correctness).
Revised: We use two metrics for measuring certiﬁcate strength: Certiﬁed accuracy (i.e. the percentage of correct and certiﬁably robust predictions) and certiﬁed ratio (i.e. the percentage of certiﬁably robust predictions, regardless of correctness).

Original: We use Monte Carlo randomized smoothing (see Section C.
Revised: We use Monte Carlo randomized smoothing (see Section C.

Original: Therefore, we may have to abstain from making predictions.
Revised: Therefore, we may have to abstain from making predictions.

Original: Abstentions are counted as non-robust and incorrect.
Revised: Abstentions are counted as non-robust and incorrect.

Original: In the case of center smoothing, either all or none predictions abstain (this is inherent to the method.
Revised: In the case of center smoothing, either all or none predictions abstain (this is inherent to the method.

Original: In our experiments, center smoothing never abstained).
Revised: In our experiments, center smoothing never abstained).

Original: In the following, let Z = { d ∈ { 1 , . . .
Revised: In the following, let Z = { d ∈ { 1 , . . .

Original: , D out } | y n = ˆ y n } be the indices of correct predictions, given an input x .
Revised: , D out } | y n = ˆ y n } be the indices of correct predictions, given an input x .

Original: Na¨ıve collective certiﬁcate .
Revised: Na¨ıve collective certiﬁcate .

Original: The na¨ıve collective certiﬁcate certiﬁes each prediction independently.
Revised: The na¨ıve collective certiﬁcate certiﬁes each prediction independently.

Original: Like with our base certiﬁcates, let H ( n ) be the set of perturbed inputs y n is robust to.
Revised: Like with our base certiﬁcates, let H ( n ) be the set of perturbed inputs y n is robust to.

Original: Let B x be the collective perturbation model.
Revised: Let B x be the collective perturbation model.

Original: Then L = (cid:8) d ∈ { 1 , . . .
Revised: Then L = (cid:8) d ∈ { 1 , . . .

Original: , D out } | B x ⊆ H ( n ) (cid:9) is the set of all certiﬁably robust predictions.
Revised: , D out } | B x ⊆ H ( n ) (cid:9) is the set of all certiﬁably robust predictions.

Original: The certiﬁed ratio can be computed as | L | D out and the certiﬁed accuracy as | L ∩ Z | D out .
Revised: The certiﬁed ratio can be computed as | L | D out and the certiﬁed accuracy as | L ∩ Z | D out .

Original: Center smoothing Center smoothing does not determine which predictions are robust, but only how many are.
Revised: Center smoothing Center smoothing does not determine which predictions are robust, but only how many are.

Original: Let k be the number of certiﬁably robust predictions.
Revised: Let k be the number of certiﬁably robust predictions.

Original: The certiﬁed ratio can be easily computed as kD out .
Revised: The certiﬁed ratio can be easily computed as kD out .

Original: For the certiﬁed accuracy, we have to make the worst-case assumption that the non-robust predictions are correct predictions, i.e. compute max(0 ,k − ( D out −| Z | ))
Revised: For the certiﬁed accuracy, we have to make the worst-case assumption that the non-robust predictions are correct predictions, i.e. compute max(0 ,k − ( D out −| Z | ))

Original: D out
Revised: D out

Original: Collective certiﬁcate .
Revised: Collective certiﬁcate .

Original: Let l ( T ) be the optimal value of our collective certiﬁcate and set of targeted nodes T .
Revised: Let l ( T ) be the optimal value of our collective certiﬁcate and set of targeted nodes T .

Original: Then both the certiﬁed ratio and certiﬁed accuracy can be computed via l ( T ) D out with T = { 1 , . . .
Revised: Then both the certiﬁed ratio and certiﬁed accuracy can be computed via l ( T ) D out with T = { 1 , . . .

Original: , D out } and T =
Revised: , D out } and T =

Original: Z , respectively.
Revised: Z , respectively.

Original: E. S EMANTIC SEGMENTATION
Revised: E. S EMANTIC SEGMENTATION

Original: We ﬁrst list all parameters and details of our training and certiﬁcation procedures, before discussing additional experimental results in Section E.2.3.
Revised: We ﬁrst list all parameters and details of our training and certiﬁcation procedures, before discussing additional experimental results in Section E.2.3.

Original: E.2. E XPERIMENTAL SETUP AND HYPERPARAMETERS
Revised: E.2. E XPERIMENTAL SETUP AND HYPERPARAMETERS

Original: Model.
Revised: Model.

Original: As the base model for the semantic segmentation task, we use a U-Net model (Ronneberger et al., 2015) with a ResNet-18 (He et al., 2016) backbone, as implemented by the Pytorch Segmentation Models library (version 0 . 13 ) (Yakubovskiy, 2020).
Revised: As the base model for the semantic segmentation task, we use a U-Net model (Ronneberger et al., 2015) with a ResNet-18 (He et al., 2016) backbone, as implemented by the Pytorch Segmentation Models library (version 0 . 13 ) (Yakubovskiy, 2020).

Original: We use the library’s default parameters.
Revised: We use the library’s default parameters.

Original: In particular, the inputs to the U-Net segmentation head are the features of the ResNet model after the ﬁrst convolutional layer and after each ResNet block (i.e. after every fourth of the subsequent layers).
Revised: In particular, the inputs to the U-Net segmentation head are the features of the ResNet model after the ﬁrst convolutional layer and after each ResNet block (i.e. after every fourth of the subsequent layers).

Original: The U-Net segmentation head uses (starting with the original resolution) 16 , 32 , 64 ,and 256 convolutional ﬁlters for processing the features at the different scales.
Revised: The U-Net segmentation head uses (starting with the original resolution) 16 , 32 , 64 ,and 256 convolutional ﬁlters for processing the features at the different scales.

Original: To avoid dimension mismatches in the segmentation head, all input images are zero-padded to a height and width that is the next multiple of 32 .
Revised: To avoid dimension mismatches in the segmentation head, all input images are zero-padded to a height and width that is the next multiple of 32 .

Original: Data and preprocessing.
Revised: Data and preprocessing.

Original: We evaluate our certiﬁcates on the Pascal-VOC 2012 segmentation validation set.
Revised: We evaluate our certiﬁcates on the Pascal-VOC 2012 segmentation validation set.

Original: We do not use the test set, because evaluating metrics like the certiﬁed accuracy requires access to the ground-truth labels.
Revised: We do not use the test set, because evaluating metrics like the certiﬁed accuracy requires access to the ground-truth labels.

Original: For training, we use the 10582 Pascal segmentation masks extracted from the SBD dataset (Hariharan et al., 2011) (referred to as ”Pascal trainaug” or ”Pascal augmented training set” in other papers).
Revised: For training, we use the 10582 Pascal segmentation masks extracted from the SBD dataset (Hariharan et al., 2011) (referred to as ”Pascal trainaug” or ”Pascal augmented training set” in other papers).

Original: SBD uses a different data split than the ofﬁcial PascalVOC 2012 segmentation dataset.
Revised: SBD uses a different data split than the ofﬁcial PascalVOC 2012 segmentation dataset.

Original: We avoid data leakage by removing all training images that appear in the validation set.
Revised: We avoid data leakage by removing all training images that appear in the validation set.

Original: We downscale both the training and the validation images and ground-truth masks to 50% of their original height and width, so that we can use larger batch sizes and thus use our compute time to more thoroughly evaluate a larger range of different smoothing distributions.
Revised: We downscale both the training and the validation images and ground-truth masks to 50% of their original height and width, so that we can use larger batch sizes and thus use our compute time to more thoroughly evaluate a larger range of different smoothing distributions.

Original: The segmentation masks are downscaled using nearest-neighbor interpolation, the images are downscaled using the INTER AREA operation implemented in OpenCV (Bradski, 2000).
Revised: The segmentation masks are downscaled using nearest-neighbor interpolation, the images are downscaled using the INTER AREA operation implemented in OpenCV (Bradski, 2000).

Original: Training and data augmentation.
Revised: Training and data augmentation.

Original: We initialize our model weights using the weights provided by the Pytorch Segmentation Models library, which were obtained by pre-training on ImageNet.
Revised: We initialize our model weights using the weights provided by the Pytorch Segmentation Models library, which were obtained by pre-training on ImageNet.

Original: We train our models for 256 epochs, using Dice loss with batch size 64 and Adam( lr = 0 .
Revised: We train our models for 256 epochs, using Dice loss with batch size 64 and Adam( lr = 0 .

Original: 001 , β 1 = 0 .
Revised: 001 , β 1 = 0 .

Original: 9 , β 2 = 0 .
Revised: 9 , β 2 = 0 .

Original: 999 , (cid:15) =
Revised: 999 , (cid:15) =

Original: 10 − 8 , weight decay = 0 .
Revised: 10 − 8 , weight decay = 0 .

Original: Every 8 epochs, we compute the mean IOU on the validation set.
Revised: Every 8 epochs, we compute the mean IOU on the validation set.

Original: After training, we use the model that achieved the highest validation mean IOU.
Revised: After training, we use the model that achieved the highest validation mean IOU.

Original: We apply the following train-time augmentations: Each image is randomly shifted by up to 10% of its height and width, scaled by a factor from [0 . 5 , 2 . 0] and rotated between − 10 and 10 degrees using the ShiftScaleRotate augmentation implemented by the Albumentations library (version 0 .
Revised: We apply the following train-time augmentations: Each image is randomly shifted by up to 10% of its height and width, scaled by a factor from [0 . 5 , 2 . 0] and rotated between − 10 and 10 degrees using the ShiftScaleRotate augmentation implemented by the Albumentations library (version 0 .

Original: (Buslaev et al., 2020).
Revised: (Buslaev et al., 2020).

Original: The images are than cropped to a ﬁxed size of 128 × 160 .
Revised: The images are than cropped to a ﬁxed size of 128 × 160 .

Original: Where necessary, the images are padded with zeros.
Revised: Where necessary, the images are padded with zeros.

Original: Padded parts of the segmentation mask are ignored during training.
Revised: Padded parts of the segmentation mask are ignored during training.

Original: After these operations, each input is randomly perturbed using a Gaussian distribution with a ﬁxed standard deviation σ ∈ { 0 , 0 . 01 , . . .
Revised: After these operations, each input is randomly perturbed using a Gaussian distribution with a ﬁxed standard deviation σ ∈ { 0 , 0 . 01 , . . .

Original: All samples are clipped to [0 , 1] to retain valid RGBimages.
Revised: All samples are clipped to [0 , 1] to retain valid RGBimages.

Original: Certiﬁcation.
Revised: Certiﬁcation.

Original: We evaluate all certiﬁcates on the ﬁrst 50 images from the validation set that – after downscaling – have a resolution of 166 × 250 .
Revised: We evaluate all certiﬁcates on the ﬁrst 50 images from the validation set that – after downscaling – have a resolution of 166 × 250 .

Original: For our experiments, we use Monte Carlo randomized smoothing (see discussion in Section C).
Revised: For our experiments, we use Monte Carlo randomized smoothing (see discussion in Section C).

Original: We use 12288 samples for making smoothed predictions and 153600 samples for certiﬁcation.
Revised: We use 12288 samples for making smoothed predictions and 153600 samples for certiﬁcation.

Original: We use the signiﬁcance parameter α to 0 .
Revised: We use the signiﬁcance parameter α to 0 .

Original: 01 , i.e. all certiﬁcates hold with probability 0 .
Revised: 01 , i.e. all certiﬁcates hold with probability 0 .

Original: For the center smoothing baseline, we use the default parameters suggested by the authors ( ∆ = 0 .
Revised: For the center smoothing baseline, we use the default parameters suggested by the authors ( ∆ = 0 .

Original: 05 , β = 2 , α 1 =
Revised: 05 , β = 2 , α 1 =

Original: α 2 ).
Revised: α 2 ).

Original: For the na¨ıve collective certiﬁcate baseline, we use Holm correction to account for the multiple comparisons problem, which yields strictly better results than Bonferroni correction.
Revised: For the na¨ıve collective certiﬁcate baseline, we use Holm correction to account for the multiple comparisons problem, which yields strictly better results than Bonferroni correction.

Original: For our localized smoothing certiﬁcates, we use Bonferroni correction.
Revised: For our localized smoothing certiﬁcates, we use Bonferroni correction.

Original: For our localized smoothing distribution, we partition the input image into a regular grid of size 4 × 6 and deﬁne minimum standard deviation σ min and maximum standard deviation σ max .
Revised: For our localized smoothing distribution, we partition the input image into a regular grid of size 4 × 6 and deﬁne minimum standard deviation σ min and maximum standard deviation σ max .

Original: Let J ( k,l ) be the set of all pixel coordinates in grid cell ( k, l ) .
Revised: Let J ( k,l ) be the set of all pixel coordinates in grid cell ( k, l ) .

Original: To smooth outputs in grid cell ( i, j ) , we use a smoothing distribution N ( 0 , diag( σ ) with, ∀ k ∈ { 1 , . .
Revised: To smooth outputs in grid cell ( i, j ) , we use a smoothing distribution N ( 0 , diag( σ ) with, ∀ k ∈ { 1 , . .

Original: , 4 } , l ∈ { 1 , . . .
Revised: , 4 } , l ∈ { 1 , . . .

Original: , 6 } , d ∈ J ( k,l ) , σ d = σ min + ( σ max − σ min ) · max ( | i − k | , | l − j | ), (139) i.e. we linearly interpolate between σ min and σ max based on the l ∞ distance of grid cells ( i, j ) and ( k, l ) .
Revised: , 6 } , d ∈ J ( k,l ) , σ d = σ min + ( σ max − σ min ) · max ( | i − k | , | l − j | ), (139) i.e. we linearly interpolate between σ min and σ max based on the l ∞ distance of grid cells ( i, j ) and ( k, l ) .

Original: All results are reported for the relaxed linear programming formulation of our collective certiﬁcate (see Section A.4).
Revised: All results are reported for the relaxed linear programming formulation of our collective certiﬁcate (see Section A.4).

Original: For each grid cell, we use 124 = 14 · 6 of the samples, which corresponds to 512 samples for prediction and 6400 samples for certiﬁcation.
Revised: For each grid cell, we use 124 = 14 · 6 of the samples, which corresponds to 512 samples for prediction and 6400 samples for certiﬁcation.

Original: The collective linear program is solved using MOSEK (version 9.2.46) (MOSEK ApS, 2019) through the CVXPY interface (version 1.1.13) (Diamond & Boyd, 2016).
Revised: The collective linear program is solved using MOSEK (version 9.2.46) (MOSEK ApS, 2019) through the CVXPY interface (version 1.1.13) (Diamond & Boyd, 2016).

Original: E.2. H ARDWARE
Revised: E.2. H ARDWARE

Original: All experiments on image segmentation were performed using a Xeon E5-2630 v4 CPU @
Revised: All experiments on image segmentation were performed using a Xeon E5-2630 v4 CPU @

Original: 2.20GHz, an NVIDA GTX 1080TI GPU and 128 GB of RAM.
Revised: 2.20GHz, an NVIDA GTX 1080TI GPU and 128 GB of RAM.

Original: E.2. C OMPLETE EXPERIMENTAL EVALUATION
Revised: E.2. C OMPLETE EXPERIMENTAL EVALUATION

Original: As explained in Section 8.1, the objective of our experiments on semantic segmentation is to verify our claim that localized smoothing allows for a better accuracy-robustness trade-off than randomized smoothing with i.i.d.
Revised: As explained in Section 8.1, the objective of our experiments on semantic segmentation is to verify our claim that localized smoothing allows for a better accuracy-robustness trade-off than randomized smoothing with i.i.d.

Original: distributions, i.e. it allows us to retain a higher prediction quality and simultaneously provide stronger collective robustness guarantees.
Revised: distributions, i.e. it allows us to retain a higher prediction quality and simultaneously provide stronger collective robustness guarantees.

Original: The ﬁrst step of our evaluation procedure is to evaluate the na ¨ ıve and the center smoothing baselines for Gaussian standard deviations σ ∈ { 0 . 1 , 0 . 2 , 0 . 3 , 0 . 4 , 0 . 5 } and – for each σ – ﬁnd a combination of localized smoothing parameters σ min , σ out that result in equal or higher accuracy and stronger certiﬁcates, as measured by certiﬁed accuracy and certiﬁed ratio.
Revised: The ﬁrst step of our evaluation procedure is to evaluate the na ¨ ıve and the center smoothing baselines for Gaussian standard deviations σ ∈ { 0 . 1 , 0 . 2 , 0 . 3 , 0 . 4 , 0 . 5 } and – for each σ – ﬁnd a combination of localized smoothing parameters σ min , σ out that result in equal or higher accuracy and stronger certiﬁcates, as measured by certiﬁed accuracy and certiﬁed ratio.

Original: Figs.
Revised: Figs.

Original: 5 to 8 show the certiﬁed accuracy and certiﬁed ratio curve for all 10 pairs of baseline and locally smoothed models ( 5 per baseline).
Revised: 5 to 8 show the certiﬁed accuracy and certiﬁed ratio curve for all 10 pairs of baseline and locally smoothed models ( 5 per baseline).

Original: The AUC of the certiﬁed ratio and certiﬁed ratio curves for all smoothed models are summarized in Tables 1 and 2.
Revised: The AUC of the certiﬁed ratio and certiﬁed ratio curves for all smoothed models are summarized in Tables 1 and 2.

Original: Safe for center smoothing with σ = 0 .
Revised: Safe for center smoothing with σ = 0 .

Original: 2 , which has a slightly higher certiﬁed ratio but lower accuracy, we can outperform all baselines.
Revised: 2 , which has a slightly higher certiﬁed ratio but lower accuracy, we can outperform all baselines.

Original: The second step is to ﬁx the found localized smoothing parameters and perform a ﬁne-grained search over σ ∈ [0 , 1] .
Revised: The second step is to ﬁx the found localized smoothing parameters and perform a ﬁne-grained search over σ ∈ [0 , 1] .

Original: This is to ensure that our previous results were not caused by a particularly poor choice of baseline parameters.
Revised: This is to ensure that our previous results were not caused by a particularly poor choice of baseline parameters.

Original: Table 5 summarizes the results.
Revised: Table 5 summarizes the results.

Original: It shows that there is no σ in our search space for which the baselines outperform the locally smoothed model w.r.t.
Revised: It shows that there is no σ in our search space for which the baselines outperform the locally smoothed model w.r.t.

Original: our metrics of certiﬁcate strength.
Revised: our metrics of certiﬁcate strength.

Original: Figure Fig.
Revised: Figure Fig.

Original: 4 demonstrates why there is no σ that outperforms our baselines, using the certiﬁed ratio of the na¨ıve collective certiﬁcate baseline as an example.
Revised: 4 demonstrates why there is no σ that outperforms our baselines, using the certiﬁed ratio of the na¨ıve collective certiﬁcate baseline as an example.

Original: The certiﬁcate strength, as measured by the certiﬁed ratio’s AUC, increases with σ .
Revised: The certiﬁcate strength, as measured by the certiﬁed ratio’s AUC, increases with σ .

Original: But the model’s accuracy decreases with σ .
Revised: But the model’s accuracy decreases with σ .

Original: Therefore, strengthening the certiﬁcate requires sacriﬁcing some accuracy.
Revised: Therefore, strengthening the certiﬁcate requires sacriﬁcing some accuracy.

Original: But the locally smoothed models were already as accurate or more accurate than the baselines, which makes it impossible for the baselines to simultaneously attain a higher accuracy and stronger certiﬁcates.
Revised: But the locally smoothed models were already as accurate or more accurate than the baselines, which makes it impossible for the baselines to simultaneously attain a higher accuracy and stronger certiﬁcates.

Original: To summarize: Localized smoothing appears to indeed offer a better trade-off between certiﬁed accuracy and robustness.
Revised: To summarize: Localized smoothing appears to indeed offer a better trade-off between certiﬁed accuracy and robustness.

Original: In Section 8.1, we mentioned that we compare base classiﬁers with different trained weights.
Revised: In Section 8.1, we mentioned that we compare base classiﬁers with different trained weights.

Original: For i.i.d.
Revised: For i.i.d.

Original: smoothing with σ , we loaded models trained with σ train =
Revised: smoothing with σ , we loaded models trained with σ train =

Original: σ .
Revised: σ .

Original: For localized smoothing, we loaded models trained with σ train =
Revised: For localized smoothing, we loaded models trained with σ train =

Original: σ min .
Revised: σ min .

Original: To show that this is not the reason why we outperform the baselines, we compare the same i.i.d.
Revised: To show that this is not the reason why we outperform the baselines, we compare the same i.i.d.

Original: and localized smoothing distributions as before, but use σ train
Revised: and localized smoothing distributions as before, but use σ train

Original: =
Revised: =

Original: σ min for all models.
Revised: σ min for all models.

Original: The results are summarized in Appendix E.2.3 and Table 4.
Revised: The results are summarized in Appendix E.2.3 and Table 4.

Original: Overall, there is only a single instance in which the baselines are better than the localized smoothing certiﬁcate in any of our metrics:
Revised: Overall, there is only a single instance in which the baselines are better than the localized smoothing certiﬁcate in any of our metrics:

Original: For σ = 0 .
Revised: For σ = 0 .

Original: 3 , the na¨ıve baseline attains an accuracy of 79 .
Revised: 3 , the na¨ıve baseline attains an accuracy of 79 .

Original: 5% that is 0 .
Revised: 5% that is 0 .

Original: 4 p .
Revised: 4 p .

Original: p .
Revised: p .

Original: higher than that of the locally smoothed model.
Revised: higher than that of the locally smoothed model.

Original: The AUC of its certiﬁed accuracy and certiﬁed ratio curves – 0 . 492 and 0 . 5615 , respectively – are however much smaller than the 0 . 5869 and 0 .
Revised: The AUC of its certiﬁed accuracy and certiﬁed ratio curves – 0 . 492 and 0 . 5615 , respectively – are however much smaller than the 0 . 5869 and 0 .

Original: 6579 guaranteed by localized smoothing.
Revised: 6579 guaranteed by localized smoothing.

Original: So, even with this different choice of weights, localized smoothing appears to guarantee a better trade-off between accuracy and provable robustness.
Revised: So, even with this different choice of weights, localized smoothing appears to guarantee a better trade-off between accuracy and provable robustness.

Original: On average, each collective linear program only took 0 .
Revised: While it resulted in signiﬁcantly stronger certiﬁcates, the proposed collective certiﬁcate added very little additional computational cost, compared to the na¨ıve collective certiﬁcate.

Original: 68 s to solve.
Revised: The na¨ıve collective certiﬁcate requires taking Monte Carlo samples and then performing a small number of vector operations to compute the per-prediction Gaussian smoothing certiﬁcates. The proposed collective certiﬁcates requires taking Monte Carlo samples and computing the same type of per-prediction certiﬁcate, but then solves a linear program on top.

Original: This computational overhead is small compared to amount of time needed to obtain the randomized smoothing certiﬁcates through Monte Carlo sampling, an average 460 s per image. [Figure]
Revised: The sampling dominated the runtime, with an average 460 s per image (both for the baseline and the proposed certiﬁcate, as we used the same number of samples for both). Averaged over all experiments and images, computing the per-prediction certiﬁcates took 15 .

Original: ﬁ e d r a ti o AU
Revised: 85 s for the baseline and 0 .

Original: C
Revised: 15 s for the proposed certiﬁcate (the increased cost for the baseline is due to the more complicated Holm correction – with Bonferroni correction both are equally fast). Averaged over all images and tested adversarial budgets, solving each collective linear program only took 0 . 68 s , i.e. much less than the Monte Carlo sampling that is necessary for both the baseline and our method.

Original: [Figure]
Revised: [Figure]

Original: [Table] [Table]
Revised: [Table] [Table]

Original: [Table]
Revised: [Table]

Original: baseline and localized smoothing for σ ∈ { 0 . 1 , 0 . 2 , 0 . 3 , 0 . 4 , 0 . 5 } and different σ min and σ max .
Revised: baseline and localized smoothing for σ ∈ { 0 . 1 , 0 . 2 , 0 . 3 , 0 . 4 , 0 . 5 } and different σ min and σ max .

Original: Unlike in Table 1, we also load the model trained with σ min for the baselines.
Revised: Unlike in Table 1, we also load the model trained with σ min for the baselines.

Original: Safe for σ = 0 .
Revised: Safe for σ = 0 .

Original: 3 , where the baseline has a higher accuracy but much weaker certiﬁcates, localized smoothing again offers a higher accuracy and stronger certiﬁcates.
Revised: 3 , where the baseline has a higher accuracy but much weaker certiﬁcates, localized smoothing again offers a higher accuracy and stronger certiﬁcates.

Original: [Table]
Revised: [Table]

Original: [Table] [Table]
Revised: [Table] [Table]

Original: C e r ti ﬁ e d r a ti o
Revised: C e r ti ﬁ e d r a ti o

Original: [Table]
Revised: [Table]

Original: LPlocalized
Revised: LPlocalized

Original: [Table]
Revised: [Table]

Original: C e r ti ﬁ e d r a ti o
Revised: C e r ti ﬁ e d r a ti o

Original: [Table]
Revised: [Table]

Original: C e r ti ﬁ e d r a ti o
Revised: C e r ti ﬁ e d r a ti o

Original: LPlocalized
Revised: LPlocalized

Original: Na¨ıvelocalized
Revised: Na¨ıvelocalized

Original: Na¨ıve i.i.d.
Revised: Na¨ıve i.i.d.

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: C e
Revised: C e

Original: [Table]
Revised: [Table]

Original: ti ﬁ e d acc
Revised: ti ﬁ e d acc

Original: [Table]
Revised: [Table]

Original: r ac y
Revised: r ac y

Original: [Table]
Revised: [Table]

Original: C
Revised: C

Original: [Table]
Revised: [Table]

Original: ti ﬁ
Revised: ti ﬁ

Original: [Table]
Revised: [Table]

Original: d acc
Revised: d acc

Original: [Table]
Revised: [Table]

Original: r
Revised: r

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: LPlocalized
Revised: LPlocalized

Original: Na¨ıvelocalized
Revised: Na¨ıvelocalized

Original: Na¨ıve i.i.d.
Revised: Na¨ıve i.i.d.

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: C e r ti ﬁ e d r a ti o
Revised: C e r ti ﬁ e d r a ti o

Original: [Table]
Revised: [Table]

Original: C e r ti ﬁ e d r a ti o
Revised: C e r ti ﬁ e d r a ti o

Original: [Table]
Revised: [Table]

Original: C
Revised: C

Original: [Table]
Revised: [Table]

Original: ti ﬁ
Revised: ti ﬁ

Original: [Table]
Revised: [Table]

Original: d r a ti o
Revised: d r a ti o

Original: LPlocalized
Revised: LPlocalized

Original: [Figure]
Revised: [Figure]

Original: Centersmoothing
Revised: Centersmoothing

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: ti ﬁ
Revised: ti ﬁ

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: LPlocalized
Revised: LPlocalized

Original: Na¨ıvelocalized
Revised: Na¨ıvelocalized

Original: Centersmoothing
Revised: Centersmoothing

Original: [Figure]
Revised: [Figure]

Original: E. N ODE CLASSIFICATION
Revised: E. N ODE CLASSIFICATION

Original: We ﬁrst discuss the experimental setup and the necessary parameters for our training and certiﬁcation procedure before we show additional experimental results in Section E.3.3.
Revised: We ﬁrst discuss the experimental setup and the necessary parameters for our training and certiﬁcation procedure before we show additional experimental results in Section E.3.3.

Original: E.
Revised: E.

Original: . E XPERIMENTAL S ETUP AND H YPERPARAMETERS
Revised: . E XPERIMENTAL S ETUP AND H YPERPARAMETERS

Original: Metric
Revised: Metric

Original: In the node classiﬁcation setting we are generally only concerned with certiﬁed accuracy.
Revised: In the node classiﬁcation setting we are generally only concerned with certiﬁed accuracy.

Original: To calculate this accuracy, we only consider nodes that are correctly classiﬁed, where the method does not abstain, and is certiﬁably robust for the given perturbation.
Revised: To calculate this accuracy, we only consider nodes that are correctly classiﬁed, where the method does not abstain, and is certiﬁably robust for the given perturbation.

Original: Model We test two different models: 2 -layer APPNP (Klicpera et al., 2019) and 6 -layer GCN (Kipf & Welling, 2017).
Revised: Model We test two different models: 2 -layer APPNP (Klicpera et al., 2019) and 6 -layer GCN (Kipf & Welling, 2017).

Original: For both models we use a hidden size of 64 and dropout with a probability of 0 . 5 .
Revised: For both models we use a hidden size of 64 and dropout with a probability of 0 . 5 .

Original: Furthermore, for the propagation step of APPNP we use 10 for the number of interactions and 0 .as value for α .
Revised: Furthermore, for the propagation step of APPNP we use 10 for the number of interactions and 0 .as value for α .

Original: Data and preprocessing.
Revised: Data and preprocessing.

Original: We evaluate our approach on the Cora-ML node classiﬁcation dataset.
Revised: We evaluate our approach on the Cora-ML node classiﬁcation dataset.

Original: We perform standard preprocessing, i.e., remove self-loops, make the graph undirected and select the largest connected component.
Revised: We perform standard preprocessing, i.e., remove self-loops, make the graph undirected and select the largest connected component.

Original: For the localized smoothing approach we perform Metis clustering (Karypis & Kumar, 1998) to partition the graph into 5 clusters.
Revised: For the localized smoothing approach we perform Metis clustering (Karypis & Kumar, 1998) to partition the graph into 5 clusters.

Original: We create an afﬁnity ranking by counting the number of edges which are connecting cluster i and j .
Revised: We create an afﬁnity ranking by counting the number of edges which are connecting cluster i and j .

Original: This ranking is used to select the noise parameter for smoothing the attributes of cluster j while classifying a node of cluster i .
Revised: This ranking is used to select the noise parameter for smoothing the attributes of cluster j while classifying a node of cluster i .

Original: Training and data augmentation All models are trained with a learning rate of 0 .
Revised: Training and data augmentation All models are trained with a learning rate of 0 .

Original: 001 and weight decay of 0 .
Revised: 001 and weight decay of 0 .

Original: The models we use for sparse smoothing are trained with the noise distribution that is also reported for certiﬁcation.
Revised: The models we use for sparse smoothing are trained with the noise distribution that is also reported for certiﬁcation.

Original: The localized smoothing models are trained on the their minimal noise level, i.e., not with localized noise but with only θ +min and θ − min .
Revised: The localized smoothing models are trained on the their minimal noise level, i.e., not with localized noise but with only θ +min and θ − min .

Original: Certiﬁcation In the node classiﬁcation setting the noise parameter space is large.
Revised: Certiﬁcation In the node classiﬁcation setting the noise parameter space is large.

Original: Instead of only one noise parameter as used in the image segmentation scenario, we can vary ﬂip probabilities for addition and deletion.
Revised: Instead of only one noise parameter as used in the image segmentation scenario, we can vary ﬂip probabilities for addition and deletion.

Original: Apart from that we need lower and upper values for each of those.
Revised: Apart from that we need lower and upper values for each of those.

Original: Therefore, we just selected the minimum parameters and focused on optimising the baseline around these noise levels as described in Section 8.2.
Revised: Therefore, we just selected the minimum parameters and focused on optimising the baseline around these noise levels as described in Section 8.2.

Original: The speciﬁc noise parameters and baseline search space regions can be seen in the captions of the respective ﬁgures.
Revised: The speciﬁc noise parameters and baseline search space regions can be seen in the captions of the respective ﬁgures.

Original: The collective linear program is solved using MOSEK (version 9.2.46) (MOSEK ApS, 2019) through the CVXPY interface (version 1.1.13) (Diamond & Boyd, 2016).
Revised: The collective linear program is solved using MOSEK (version 9.2.46) (MOSEK ApS, 2019) through the CVXPY interface (version 1.1.13) (Diamond & Boyd, 2016).

Original: E.3. H ARDWARE
Revised: E.3. H ARDWARE

Original: All experiments on image segmentation were performed using an AMD EPYC 7543 CPU @ 2.80GHz, an NVIDA A100 GPU and 32 GB of RAM.
Revised: All experiments on image segmentation were performed using an AMD EPYC 7543 CPU @ 2.80GHz, an NVIDA A100 GPU and 32 GB of RAM.

Original: E.3. A DDITIONAL E XPERIMENTAL E VALUATION
Revised: E.3. A DDITIONAL E XPERIMENTAL E VALUATION

Original: In Fig.
Revised: In Fig.

Original: 9, we can see a comparison of our localized smoothing approach to sparse smoothing for a
Revised: 9, we can see a comparison of our localized smoothing approach to sparse smoothing for a

Original: GCN model with 6-layers.
Revised: GCN model with 6-layers.

Original: Here, the smoothing distribution is the same as the one used in Fig.of Section 8.2.
Revised: Here, the smoothing distribution is the same as the one used in Fig.of Section 8.2.

Original: In Fig.
Revised: In Fig.

Original: 10, we can see a comparison for an APPNP model but with smaller minimal noise levels.
Revised: 10, we can see a comparison for an APPNP model but with smaller minimal noise levels.

Original: We can see that in all experiments for deletion, the localized smoothing approach outperforms the baselines.
Revised: We can see that in all experiments for deletion, the localized smoothing approach outperforms the baselines.

Original: In the deletion setting we increase the certiﬁed accuracy curve’s AUC from an average 12 .
Revised: In the deletion setting we increase the certiﬁed accuracy curve’s AUC from an average 12 .

Original: 28 to 18 .
Revised: 28 to 18 .

Original: 93 (i.e. by 54% ).
Revised: 93 (i.e. by 54% ).

Original: In the addition scenario the AUC, averaged over all experiments, decreases from 6 .
Revised: In the addition scenario the AUC, averaged over all experiments, decreases from 6 .

Original: 85 to 6 .
Revised: 85 to 6 .

Original: 08 (i.e. by 11 . 25% ).
Revised: 08 (i.e. by 11 . 25% ).

Original: This is mainly due to the results of the experiment that can be seen in Fig.
Revised: This is mainly due to the results of the experiment that can be seen in Fig.

Original: In the baseline evaluation process both noise parameters for sparse smoothing are signiﬁcantly larger than our min-values.
Revised: In the baseline evaluation process both noise parameters for sparse smoothing are signiﬁcantly larger than our min-values.

Original: This shows that our values may be too small in this scenario.
Revised: This shows that our values may be too small in this scenario.

Original: Fig.
Revised: Fig.

Original: 11 compares the variance certiﬁcate to the sparse smoothing certiﬁcate of Bojchevski et al.
Revised: 11 compares the variance certiﬁcate to the sparse smoothing certiﬁcate of Bojchevski et al.

Original: That is, we use only one cluster and the same noise levels and models for our approach and the baseline, i.e., θ + = 0 .
Revised: That is, we use only one cluster and the same noise levels and models for our approach and the baseline, i.e., θ + = 0 .

Original: 01 and θ − = 0 .
Revised: 01 and θ − = 0 .

Original: We observe that for both models the variance certiﬁcate yields better results for deletion.
Revised: We observe that for both models the variance certiﬁcate yields better results for deletion.

Original: However, in the addition setting, it is outperformed by the baseline.
Revised: However, in the addition setting, it is outperformed by the baseline.

Original: If we look at Theorem 2, we can see that we multiply the adversarial budget for addition with γ +
Revised: If we look at Theorem 2, we can see that we multiply the adversarial budget for addition with γ +

Original: [Figure]
Revised: [Figure]

Original: C e r ti ﬁ e d acc u r ac y
Revised: C e r ti ﬁ e d acc u r ac y

Original: LP localized
Revised: LP localized

Original: Na¨ıve localized
Revised: Na¨ıve localized

Original: Na¨ıve i.i.d.
Revised: Na¨ıve i.i.d.

Original: [Figure] [Figure]
Revised: Number of adversarial deletions 0 .0 .0 .0 .0 .1 .

Original: C e r ti ﬁ e d acc u r ac y
Revised: C e r ti ﬁ e d acc u r ac y

Original: LP localized
Revised: LP localized

Original: Na¨ıve localized
Revised: Na¨ıve localized

Original: Na¨ıve i.i.d.
Revised: Na¨ıve i.i.d.

Original: In the deletion scenario we outperform the baseline with an AUC of 21 .
Revised: In the deletion scenario we outperform the baseline with an AUC of 21 .

Original: 76 non-collective).
Revised: 76 non-collective).

Original: For addition the sparse smoothing performs slightly better than our certiﬁcate with AUC of 9 .
Revised: For addition the sparse smoothing performs slightly better than our certiﬁcate with AUC of 9 .

Original: ( 6 . 39 non-collective).
Revised: ( 6 . 39 non-collective).

Original: However, the our approach performs signiﬁcantly better in small perturbation regions and also in clean accuracy (0 perturbation).[Figure] i.i.d.
Revised: However, the our approach performs signiﬁcantly better in small perturbation regions and also in clean accuracy (0 perturbation).

Original: Number of adversarial deletions 0 .0 .0 .0 .0 .1 .

Original: C e r ti ﬁ e d acc u r ac y

Original: LP localized

Original: Na¨ıve localized

Original: Na¨ıve i.i.d.

Original: and the one for deletion with γ − .
Revised: and the one for deletion with γ − .

Original: As γ + has θ + in the denominator the small noise level for addition results in larger values for γ + .
Revised: As γ + has θ + in the denominator the small noise level for addition results in larger values for γ + .

Original: For these parameters we have γ + = 2 .
Revised: For these parameters we have γ + = 2 .

Original: 795 and γ − = 0 .
Revised: 795 and γ − = 0 .

Original: 491 which shows why the certiﬁcate is less robust to adversarial additions.
Revised: 491 which shows why the certiﬁcate is less robust to adversarial additions.

Original: This is consistent with the other experiments where our results are on par or worse than the baseline for additions but we outperform it for deletions.
Revised: This is consistent with the other experiments where our results are on par or worse than the baseline for additions but we outperform it for deletions.

Original: On average, each collective linear program only took 10 .
Revised: The Monte Carlo sampling dominated the runtime. Averaged over all experiments, it took 1034 s (both for the baseline and the proposed certiﬁcate, as we used the same number of samples for both). Averaged over all experiments, the per-prediction certiﬁcates took 0 . 11 s for the baseline and 0 .

Original:  9 s to solve.
Revised: 66 s for the proposed certiﬁcate (note that the baseline uses the sparsity-aware smoothing certiﬁcate of Bojchevski et al. (2020), while the proposed collective certiﬁcate uses our novel variance smoothing certiﬁcate (see Theorem 2), which requires estimating both the mean and variance of softmax scores). Averaged over all tested adversarial budgets, solving each collective linear program only took 10 . 9 s , i.e. less than the Monte Carlo sampling that is necessary for both the baseline and our method.

Original: The reason that the linear programs for graphs required more time than those for image, even though they involve fewer constraints and variables, is that a different, not as well-vectorized implementation was used.
Revised: The reason that the linear programs for graphs required more time than those for image, even though they involve fewer constraints and variables, is that a different, not as well-vectorized implementation was used.

Original: [Figure]
Revised: [Figure]

Revised: C e r ti ﬁ e d acc u r ac y

Revised: LP localized

Revised: Na¨ıve localized

Revised: Na¨ıve i.i.d.

Revised: Number of adversarial deletions 0 .0 .0 .0 .0 .1 .

Original: C e r ti ﬁ e d acc u r ac y
Revised: C e r ti ﬁ e d acc u r ac y

Revised: LP localized

Revised: Na¨ıve localized

Revised: Na¨ıve i.i.d.

Original: [Figure]
Revised: [Figure]

Revised: ﬁ e d acc u r

Original: [Figure] 
Revised: [Figure] Var. cert., del. Sparse, del.

Revised: Var.

Revised: cert., add.

Revised: Sparse, add.

Revised: Number of adversarial perturbations 0 .0 .0 .0 .0 .1 .

Revised: C e r ti ﬁ e d acc u r ac y

Revised: Var.

Revised: cert., del.

Revised: Sparse, del.

Revised: Var.

Revised: cert., add.

Revised: Sparse, add.