Original: Invariance-Aware Randomized Smoothing Certificates
Revised: Invariance-Aware Randomized Smoothing Certificates

Original: Anonymous Author(s)
Revised: Anonymous Author(s)

Original: AffiliationAddress email
Revised: AffiliationAddress email

Original: Abstract
Revised: Abstract

Original: Building models that comply with the invariances inherent to different domains,such as invariance under translation or rotation, is a key aspect of applying machinelearning to real world problems like molecular property prediction, medical imaging, protein folding or LiDAR classification.
Revised: Building models that comply with the invariances inherent to different domains,such as invariance under translation or rotation, is a key aspect of applying machinelearning to real world problems like molecular property prediction, medical imaging, protein folding or LiDAR classification.

Original: For the first time, we study how theinvariances of a model can be leveraged to provably guarantee the robustness of itspredictions.
Revised: For the first time, we study how theinvariances of a model can be leveraged to provably guarantee the robustness of itspredictions.

Original: We propose a gray-box approach, enhancing the powerful black-boxrandomized smoothing technique with white-box knowledge about invariances.
Revised: We propose a gray-box approach, enhancing the powerful black-boxrandomized smoothing technique with white-box knowledge about invariances.

Original: First, we develop a post-processing-based gray-box certification procedure that canbe applied to arbitrary models with invariance under permutation and Euclideanisometries.
Revised: First, we develop a post-processing-based gray-box certification procedure that canbe applied to arbitrary models with invariance under permutation and Euclideanisometries.

Original: Then, we derive provably tight gray-box certificates.
Revised: Then, we derive provably tight gray-box certificates.

Original: We experimentallydemonstrate that the provably tight certificates can offer much stronger guarantees,but that in practical scenarios the post-processing method is a good approximation.
Revised: We experimentallydemonstrate that the provably tight certificates can offer much stronger guarantees,but that in practical scenarios the post-processing method is a good approximation.

Original:  Introduction
Revised:  Introduction

Original: It is well-established that machine learning models are susceptible to adversarial attacks [1–4].
Revised: It is well-established that machine learning models are susceptible to adversarial attacks [1–4].

Original: Evenwithout malevolent actors, adversarial attacks can be considered worst case scenarios in environmentswith noisy, erroneous or otherwise corrupted data, thus necessitating robust machine learning methods.
Revised: Evenwithout malevolent actors, adversarial attacks can be considered worst case scenarios in environmentswith noisy, erroneous or otherwise corrupted data, thus necessitating robust machine learning methods.

Original: Invariance is a central design principle that has so far received little dedicated attention in the realmof adversarially robust machine learning.
Revised: Invariance is a central design principle that has so far received little dedicated attention in the realmof adversarially robust machine learning.

Original: Over the past decades, there has been ongoing researchinto developing machine learning models that comply with the invariances inherent to different datatypes and tasks.
Revised: Over the past decades, there has been ongoing researchinto developing machine learning models that comply with the invariances inherent to different datatypes and tasks.

Original: Prominent recent examples include Deep Sets [5], PointNet [6], Group Equivariant
Revised: Prominent recent examples include Deep Sets [5], PointNet [6], Group Equivariant

Original: CNNs [7], Spherical CNNs [8] and Graph Convolutional Networks [9], but the study of invariantmodels significantly precedes the surge in popularity of deep learning methods [10–16].
Revised: CNNs [7], Spherical CNNs [8] and Graph Convolutional Networks [9], but the study of invariantmodels significantly precedes the surge in popularity of deep learning methods [10–16].

Original: For the first time, we explore the following question: Can a-priori knowledge about invariancesbe leveraged in deriving provable guarantees for a model’s robustness to adversarial attacks?
Revised: For the first time, we explore the following question: Can a-priori knowledge about invariancesbe leveraged in deriving provable guarantees for a model’s robustness to adversarial attacks?

Original: Going by a loose categorization of prior work, we could adopt one of two possible approaches for ourexploration: A white-box or black-box one.
Revised: Going by a loose categorization of prior work, we could adopt one of two possible approaches for ourexploration: A white-box or black-box one.

Original: White-box certificates (e.g. [17–24]) analyze a model’sinternals, such at its weights and non-linearities, to provably guarantee that a prediction does notchange under adversarial attack.
Revised: White-box certificates (e.g. [17–24]) analyze a model’sinternals, such at its weights and non-linearities, to provably guarantee that a prediction does notchange under adversarial attack.

Original: Black box certificates – specifically randomized smoothing [25–27]– use statistical methods to provide provable guarantees that hold for all models sharing the sameprediction probabilities under a random input distribution, irrespective of their internals.
Revised: Black box certificates – specifically randomized smoothing [25–27]– use statistical methods to provide provable guarantees that hold for all models sharing the sameprediction probabilities under a random input distribution, irrespective of their internals.

Original: We opt for a randomized smoothing approach, as it allows us to focus on the interplay betweeninvariances and robustness, rather than the specific means of implementing these invariances.
Revised: We opt for a randomized smoothing approach, as it allows us to focus on the interplay betweeninvariances and robustness, rather than the specific means of implementing these invariances.

Original: Bycombining white-box knowledge about invariances with black-box knowledge about predictionprobabilities we obtain gray-box certificates.
Revised: Bycombining white-box knowledge about invariances with black-box knowledge about predictionprobabilities we obtain gray-box certificates.

Original: We first derive a gray-box certificate that follows apost-processing paradigm: It takes an existing black-box certificate and extends the certified regionusing information about the model’s invariances.
Revised: We first derive a gray-box certificate that follows apost-processing paradigm: It takes an existing black-box certificate and extends the certified regionusing information about the model’s invariances.

Original: This post-processing-based certificate serves asa baseline for the provably tight gray-box certificates we derive in subsequent sections.
Revised: This post-processing-based certificate serves asa baseline for the provably tight gray-box certificates we derive in subsequent sections.

Original: For our exploration, we focus on models operating on spatial data, rather than structured data (e.g. images and sequences) – both because spatial symmetries can be elegantly formalized using algebraicconcepts and because there is an ongoing trend towards using machine learning for real-worldapplications with inherent spatial invariances, e.g. molecular property prediction [28–32], LiDARclassification [33–36], drug discovery [37–39], particle physics [40–42] and protein folding [43–47].
Revised: For our exploration, we focus on models operating on spatial data, rather than structured data (e.g. images and sequences) – both because spatial symmetries can be elegantly formalized using algebraicconcepts and because there is an ongoing trend towards using machine learning for real-worldapplications with inherent spatial invariances, e.g. molecular property prediction [28–32], LiDARclassification [33–36], drug discovery [37–39], particle physics [40–42] and protein folding [43–47].

Original: Our main contributions are
Revised: Our main contributions are

Original: • the first study on the interplay of invariance and certifiable robustness,
Revised: • the first study on the interplay of invariance and certifiable robustness,

Original: • a principled method for deriving tight invariance-aware randomized smoothing certificates,•
Revised: • a principled method for deriving tight invariance-aware randomized smoothing certificates,•

Original: a tight certificate for models invariant to translations in arbitrary dimensions,• tight certificates for models invariant to rotations in 2 D and elemental rotations in 3 D.
Revised: a tight certificate for models invariant to translations in arbitrary dimensions,• tight certificates for models invariant to rotations in 2 D and elemental rotations in 3 D.

Original: We further demonstrate experimentally that the post-processing-based certificates offer a goodapproximation of our tight certificates, if the variance of the smoothing distribution is small.
Revised: We further demonstrate experimentally that the post-processing-based certificates offer a goodapproximation of our tight certificates, if the variance of the smoothing distribution is small.

Original:  Related work
Revised:  Related work

Original: Invariant machine learning.
Revised: Invariant machine learning.

Original: Given the diversity of approaches to invariant machine learning, and thefact that our approach is model-agnostic, we refrain from attempting a survey and instead refer to [48]for a principled, high-level introduction into the realm of learning with invariances and equivariances.
Revised: Given the diversity of approaches to invariant machine learning, and thefact that our approach is model-agnostic, we refrain from attempting a survey and instead refer to [48]for a principled, high-level introduction into the realm of learning with invariances and equivariances.

Original: Invariance and robustness.
Revised: Invariance and robustness.

Original: Two recent empirical studies [49, 50] demonstrate that data augmentation meant to increase robustness to l p -norm adversarial attacks reduces robustness to semanticallymeaningful transformations (e.g. rotation) and vice-versa, suggesting an inherent invariancerobustness trade-off.
Revised: Two recent empirical studies [49, 50] demonstrate that data augmentation meant to increase robustness to l p -norm adversarial attacks reduces robustness to semanticallymeaningful transformations (e.g. rotation) and vice-versa, suggesting an inherent invariancerobustness trade-off.

Original: However, neither studies models that are invariant by design.
Revised: However, neither studies models that are invariant by design.

Original: In [51], thenegative effect of shift-invariance on the robustness of image classifiers is investigated.
Revised: In [51], thenegative effect of shift-invariance on the robustness of image classifiers is investigated.

Original: Note that ourwork is not meant to resolve potential trade-offs, but to tightly bound the actual robustness of models.
Revised: Note that ourwork is not meant to resolve potential trade-offs, but to tightly bound the actual robustness of models.

Original: Gray-box robustness certificates.
Revised: Gray-box robustness certificates.

Original: While we propose the first gray-box certificate for invariantmodels, there exists prior work on combining other white-box knowledge with black-box certificates.
Revised: While we propose the first gray-box certificate for invariantmodels, there exists prior work on combining other white-box knowledge with black-box certificates.

Original: In [52] and [53], knowledge about a classifier’s gradients is used to derive tighter randomizedsmoothing certificates.
Revised: In [52] and [53], knowledge about a classifier’s gradients is used to derive tighter randomizedsmoothing certificates.

Original: In [54], knowledge about a graph neural network’s receptive fields is combinedwith randomized smoothing to derive collective robustness certificates for multiple predictions.
Revised: In [54], knowledge about a graph neural network’s receptive fields is combinedwith randomized smoothing to derive collective robustness certificates for multiple predictions.

Original: Adversarial attacks on spatial data.
Revised: Adversarial attacks on spatial data.

Original: Adversarial attacks on spatial data either modify [55–63], insert [64, 62, 65] or delete [66–68] points in space and have been particularly actively studied for pointcloud-classification.
Revised: Adversarial attacks on spatial data either modify [55–63], insert [62, 64, 65] or delete [66–68] points in space and have been particularly actively studied for pointcloud-classification.

Original: While the earliest work simply adopted gradient-based attacks from the image domain [55] more recent work has developed a rich assortment of domain-specific methods, for exampleto preserve object smoothness [61, 69], leverage critical points [66, 67] or craft physically realizable attacks (e.g. to attack models through LiDAR sensors)
Revised: While the earliest work simply adopted gradient-based attacks from the image domain [55] more recent work has developed a rich assortment of domain-specific methods, for exampleto preserve object smoothness [61, 69], leverage critical points [66, 67] or craft physically realizableattacks (e.g. to attack models through LiDAR sensors)

Original: Of particular note are attacks viaisometries (e.g. rotations and translations) [73, 74], whose effectiveness motivates the use of invariantmodels.
Revised: Of particular note are attacks viaisometries (e.g. rotations and translations) [73, 74], whose effectiveness motivates the use of invariantmodels.

Original: Note that we certify the robustness of invariant models to arbitrary point modifications attacks– not just isometry attacks (see also “orthogonal research directions” below).
Revised: Note that we certify the robustness of invariant models to arbitrary point modification attacks– not just isometry attacks (see also “orthogonal research directions” below).

Original: Like in other domains,empirical defenses have been proposed [56, 75, 76] and subsequently broken [77–79], motivatingthe development of black-box [80–82] and white-box [83] robustness certificates for spatial data.
Revised: Like in other domains,empirical defenses have been proposed [56, 75, 76] and subsequently broken [77–79], motivatingthe development of black-box [80–82] and white-box [83] robustness certificates for spatial data.

Original: Itshould be noted that Gaussian randomized smoothing, without invariance information, has alreadybeen used in prior work – either as a baseline [80] or as a special case of the respective certificate [82].
Revised: Itshould be noted that Gaussian randomized smoothing, without invariance information, has alreadybeen used in prior work – either as a baseline [80] or as a special case of the respective certificate [82].

Original: Orthogonal research directions.
Revised: Orthogonal research directions.

Original: One related but orthogonal research direction is transformationspecific certification [24, 82–90].
Revised: One related but orthogonal research direction is transformationspecific certification [24, 82–90].

Original: There, a model is assumed to potentially change its predictionunder adversarial parametric transformations (e.g. rotations) and one certifies robustness for specificparameter ranges.
Revised: There, a model is assumed to potentially change its predictionunder adversarial parametric transformations (e.g. rotations) and one certifies robustness for specificparameter ranges.

Original: White-box methods [24, 83–86] overapproximate the set of inputs reachable by atransformation and then propagate it through a model using existing white-box techniques.
Revised: White-box methods [24, 83–86] overapproximate the set of inputs reachable by atransformation and then propagate it through a model using existing white-box techniques.

Original: Black-boxmethods – namely transformation-specific randomized smoothing [87, 91] – randomize the transformation parameters to provide robustness guarantees for arbitrary models via statistical methods.
Revised: Black-boxmethods – namely transformation-specific randomized smoothing [87, 91] – randomize the transformation parameters to provide robustness guarantees for arbitrary models via statistical methods.

Original: Whileapplying this paradigm to additive transformations (e.g. brightness changes) is relatively straightforward, non-composable transformations (e.g. rotations with interpolation) require a specializedtoolset, e.g. statistical bounds on interpolation errors [87] or computation of transformation preimages [91].
Revised: Whileapplying this paradigm to additive transformations (e.g. brightness changes) is relatively straightforward, non-composable transformations (e.g. rotations with interpolation) require a specializedtoolset, e.g. statistical bounds on interpolation errors [87] or computation of transformation preimages [91].

Original: Later work generalized this principle to spatial data [82], vector-field deformations [89]and multiplicative parameters [90].
Revised: Later work generalized this principle to spatial data [82], vector-field deformations [89]and multiplicative parameters [90].

Original: Different from these methods, we assume our model to be invariant under a set of transformations, i.e. never change its prediction.
Revised: Different from these methods, we assume our model to be invariant under a set of transformations, i.e. never change its prediction.

Original: We use this property as a tool for certification against arbitrary perturbations.
Revised: We use this property as a tool for certification against arbitrary perturbations.

Original: Aside from that, there exist white-box certification techniquesfor specific operations with invariances (global max-pooling [83], message passing [92, 93] and batchnormalization [94]).
Revised: Aside from that, there exist white-box certification techniquesfor specific operations with invariances (global max-pooling [83], message passing [92, 93] and batchnormalization [94]).

Original: But prior work does not use or even discuss this property - it treats these invariantoperations as coincidental building blocks of the models it is trying to certify.
Revised: But prior work does not use or even discuss this property - it treats these invariantoperations as coincidental building blocks of the models it is trying to certify.

Original: Our work is the first tostudy invariance itself in the context of provable robustness and how to leverage it for certification.
Revised: Our work is the first tostudy invariance itself in the context of provable robustness and how to leverage it for certification.

Original:  Background on randomized smoothing
Revised:  Background on randomized smoothing

Original: Randomized smoothing is a black-boxcertification technique that can be adapted to various datatypes, tasks and threat models [95–100, 88, 101–103].
Revised: Randomized smoothing is a black-boxcertification technique that can be adapted to various datatypes, tasks and threat models [95–100, 88, 101–103].

Original: Instead of directly certifying a classifier g , itconstructs a smoothed classifier f that returns the most likely prediction under random perturbationsof its input.
Revised: Instead of directly certifying a classifier g , itconstructs a smoothed classifier f that returns the most likely prediction under random perturbationsof its input.

Original: It then certifies the robustness of this smoothed classifier.
Revised: It then certifies the robustness of this smoothed classifier.

Original: We present the tight Gaussiansmoothing certificate derived by Cohen et al.
Revised: We present the tight Gaussiansmoothing certificate derived by Cohen et al.

Original: [27] and its generalization to matrices [80, 82].
Revised: [27] and its generalization to matrices [80, 82].

Original: Assume a continuous ( N × D ) -dimensional input space R N × D , label set Y and base classifier
Revised: Assume a continuous ( N × D ) -dimensional input space R N × D , label set Y and base classifier

Original: [Equation]
Revised: [Equation]

Original: Let µ X ( Z ) =
Revised: Let µ X ( Z ) =

Original: (cid:81)
Revised: (cid:81)

Original: Dd =1 N (cid:0) Z : ,d | X : ,d , σ 2 I N (cid:1) be the isotropic matrix normaldistribution with mean X and standard deviation σ .
Revised: Dd =1 N (cid:0) Z : ,d | X : ,d , σ 2 I N (cid:1) be the isotropic matrix normaldistribution with mean X and standard deviation σ .

Original: Let p X ,y =
Revised: Let p X ,y =

Original: Pr Z ∼ µ X [ g ( Z ) =
Revised: Pr Z ∼ µ X [ g ( Z ) =

Original: y ] be theprobability of g predicting class y under this smoothing distribution .
Revised: y ] be theprobability of g predicting class y under this smoothing distribution .

Original: One can then define a smoothedclassifier f ( X ) =
Revised: One can then define a smoothedclassifier f ( X ) =

Original: argmax y ∈ Y p X ,y that returns the most likely prediction of g under µ X .
Revised: argmax y ∈ Y p X ,y that returns the most likely prediction of g under µ X .

Original: Let y ∗ =
Revised: Let y ∗ =

Original: f ( X ) be a smoothed prediction and X ′ =
Revised: f ( X ) be a smoothed prediction and X ′ =

Original: X + ∆ a perturbed input.
Revised: X + ∆ a perturbed input.

Original: One can showf ( X ′ ) =
Revised: One can showf ( X ′ ) =

Original: y ∗ by proving that, for perturbed input X ′ , y ∗ is more likely than all other classes combined.
Revised: y ∗ by proving that, for perturbed input X ′ , y ∗ is more likely than all other classes combined.

Original: That is, p X ′ ,y ∗ > 0 .
Revised: That is, p X ′ ,y ∗ > 0 .

Original: A tighter certificate can be obtained by proving that y ∗ is more likely than thesecond most likely class, i.e. p X ′ ,y ∗ > max y ′ ̸
Revised: A tighter certificate can be obtained by proving that y ∗ is more likely than thesecond most likely class, i.e. p X ′ ,y ∗ > max y ′ ̸

Original: =
Revised: =

Original: y ∗ p X ′ ,y ′ .
Revised: y ∗ p X ′ ,y ′ .

Original: For the sake of exposition, we use the firstapproach throughout the main text and generalize all results to the second one in Appendix F. Onecan lower-bound p X ′ ,y ∗ by finding the worst-case classifier from a set of functions H with g ∈ H :
Revised: For the sake of exposition, we use the firstapproach throughout the main text and generalize all results to the second one in Appendix F. Onecan lower-bound p X ′ ,y ∗ by finding the worst-case classifier from a set of functions H with g ∈ H :

Original: [Equation]
Revised: [Equation]

Original: For H = (cid:8) h : R N × D
Revised: For H = (cid:8) h : R N × D

Original: → Y | Pr Z ∼ µ X [ h ( Z ) =
Revised: → Y | Pr Z ∼ µ X [ h ( Z ) =

Original: y ∗ ] ≥ p X ,y ∗ (cid:9) , the classifiers that are at least aslikely as g to predict y ∗ , the exact solution is given by the Neyman-Pearson lemma [104] .
Revised: y ∗ ] ≥ p X ,y ∗ (cid:9) , the classifiers that are at least aslikely as g to predict y ∗ , the exact solution is given by the Neyman-Pearson lemma [104] .

Original: Theoptimal value is Φ (cid:16)
Revised: Theoptimal value is Φ (cid:16)

Original: Φ − 1 ( p X ,y ∗ ) − || ∆ || 2 σ (cid:17) , where Φ is the standard-normal CDF and || · || 2 is the
Revised: Φ − 1 ( p X ,y ∗ ) − || ∆ || 2 σ (cid:17) , where Φ is the standard-normal CDF and || · || 2 is the

Original: Frobenius norm.
Revised: Frobenius norm.

Original: If || ∆ || 2 < σ Φ − 1 ( p X ,y ∗ ) , then p X ′ ,y ∗ > 0 .
Revised: If || ∆ || 2 < σ Φ − 1 ( p X ,y ∗ ) , then p X ′ ,y ∗ > 0 .

Original: and the prediction is provably robust.
Revised: and the prediction is provably robust.

Original: Because Eq.
Revised: Because Eq.

Original: (1) was solved exactly, this is a tight certificate , i.e. the best possible certificate thatcan be obtained by only using black-box knowledge about prediction probability p X ,y ∗ .
Revised: (1) was solved exactly, this is a tight certificate , i.e. the best possible certificate thatcan be obtained by only using black-box knowledge about prediction probability p X ,y ∗ .

Original: Probabilistic certificates.
Revised: Probabilistic certificates.

Original: For neural networks, the prediction probability p X ,y ∗ can usually not becomputed analytically.
Revised: For neural networks, the prediction probability p X ,y ∗ can usually not becomputed analytically.

Original: Instead, one has to use Monte Carlo sampling to compute a lower confidencebound p x ′ ,y ∗ that holds with high probability 1 − α .
Revised: Instead, one has to use Monte Carlo sampling to compute a lower confidencebound p x ′ ,y ∗ that holds with high probability 1 − α .

Original: The resulting certificate is a probabilistic one.
Revised: The resulting certificate is a probabilistic one.

Original:  Problem setting
Revised:  Problem setting

Original: We consider a similar setup to that described in the previous section, i.e. we have a smoothed classifierf : R N × D
Revised: We consider a similar setup to that described in the previous section, i.e. we have a smoothed classifierf : R N × D

Original: → Y that is the result of randomly smoothing a base classifier g with an isotropic matrixnormal distribution µ X .
Revised: → Y that is the result of randomly smoothing a base classifier g with an isotropic matrixnormal distribution µ X .

Original: Given a clean prediction y ∗
Revised: Given a clean prediction y ∗

Original: =
Revised: =

Original: f
Revised: f

Original: ( X ) with clean prediction probabilityp X ,y ∗ , we want to determine whether f ( X ′ )
Revised: ( X ) with clean prediction probabilityp X ,y ∗ , we want to determine whether f ( X ′ )

Original: =
Revised: =

Original: y ∗ for an adversarially perturbed input X ′ =
Revised: y ∗ for an adversarially perturbed input X ′ =

Original: X + ∆ .
Revised: X + ∆ .

Original: Different from all prior work, we additionally assume that the base classifier g (not f ) is invariantunder a group of transformations T , i.e. a set of functions that contains the identitiy function as wellas the inverse of each element, and is associative and closed under function composition.
Revised: Different from all prior work, we additionally assume that the base classifier g (not f ) is invariantunder a group of transformations T , i.e. a set of functions that contains the identitiy function as wellas the inverse of each element, and is associative and closed under function composition.

Original: Definition 1 (Group-invariance) .
Revised: Definition 1 (Group-invariance) .

Original: A function h :
Revised: A function h :

Original: A → B is invariant under a group of transformations 130 T if ∀ x ∈ A , ∀ τ ∈ T : h ( x )
Revised: A → B is invariant under a group of transformations 130 T if ∀ x ∈ A , ∀ τ ∈ T : h ( x )

Original: = h ( τ ( x )) .
Revised: = h ( τ ( x )) .

Original:  Post-processing-based gray-box certificates
Revised:  Post-processing-based gray-box certificates

Original: Our first method for leveraging invariances for certification is based on the insight that certificatescorrespond to sets of inputs B ⊆ R N × D that preserve clean prediction y ∗ and invariances correspondto sets of transformations that preserve predictions.
Revised: Our first method for leveraging invariances for certification is based on the insight that certificatescorrespond to sets of inputs B ⊆ R N × D that preserve clean prediction y ∗ and invariances correspondto sets of transformations that preserve predictions.

Original: Consequently, post-processing B by applying thetransformations from T yields a new set of inputs ˜ B ⊇ B that provably preserve the clean prediction.
Revised: Consequently, post-processing B by applying thetransformations from T yields a new set of inputs ˜ B ⊇ B that provably preserve the clean prediction.

Original: [Figure]
Revised: [Figure]

Original: Theorem 1.
Revised: Theorem 1.

Original: Let h ∈ R N × D
Revised: Let h ∈ R N × D

Original: → Y be invariant under a group of transformations T .
Revised: → Y be invariant under a group of transformations T .

Original: Let y ∗
Revised: Let y ∗

Original: =
Revised: =

Original: h ( X )be a prediction that is certifiably robust to a set of perturbed inputs B ⊆ R N × D , i.e. ∀ Z ∈ B :h ( Z ) =
Revised: h ( X )be a prediction that is certifiably robust to a set of perturbed inputs B ⊆ R N × D , i.e. ∀ Z ∈ B :h ( Z ) =

Original: y ∗ .
Revised: y ∗ .

Original: Let ˜ B
Revised: Let ˜ B

Original: = { τ ( Z ) | Z ∈ B , τ ∈ T } .
Revised: = { τ ( Z ) | Z ∈ B , τ ∈ T } .

Original: Then ∀ X ′ ∈ ˜ B : h ( X ′ )
Revised: Then ∀ X ′ ∈ ˜ B : h ( X ′ )

Original: =
Revised: =

Original: y ∗ .
Revised: y ∗ .

Original: To simplify the comparison with the tight gray-box certificates we shall derive shortly, we adopt anequivalent, pre-processing-based perspective for the special case of Frobenius norm certificates: Input
Revised: To simplify the comparison with the tight gray-box certificates we shall derive shortly, we adopt anequivalent, pre-processing-based perspective for the special case of Frobenius norm certificates: Input

Original: X ′ is provably not an adversarial example, if it can be mapped into B via a transformation from T .
Revised: X ′ is provably not an adversarial example, if it can be mapped into B via a transformation from T .

Original: Theorem 2.
Revised: Theorem 2.

Original: Let h ∈ R N × D
Revised: Let h ∈ R N × D

Original: → Y be invariant under a group of transformations T .
Revised: → Y be invariant under a group of transformations T .

Original: Let y ∗
Revised: Let y ∗

Original: =
Revised: =

Original: h ( X )be a prediction that is certifiably robust to a set of perturbed inputs B = { Z | ||
Revised: h ( X )be a prediction that is certifiably robust to a set of perturbed inputs B = { Z | ||

Original: Z − X
Revised: Z − X

Original: || 2 < r } , i.e.∀ Z ∈ B : h ( Z ) =
Revised: || 2 < r } , i.e.∀ Z ∈ B : h ( Z ) =

Original: y ∗ .
Revised: y ∗ .

Original: If min τ ∈ T || τ ( X ′ ) − X
Revised: If min τ ∈ T || τ ( X ′ ) − X

Original: || 2 < r , then h ( X ′ )
Revised: || 2 < r , then h ( X ′ )

Original: =
Revised: =

Original: y ∗ .
Revised: y ∗ .

Original: Importantly, note that Theorem 2 is equivalent to Theorem 1 and not a separate certificate.
Revised: Importantly, note that Theorem 2 is equivalent to Theorem 1 and not a separate certificate.

Original: It issimply a different, implicit way of characterizing the region ˜ B .
Revised: It issimply a different, implicit way of characterizing the region ˜ B .

Original: Fig.
Revised: Fig.

Original: 1 compares the two perspectives.
Revised: 1 compares the two perspectives.

Original: Applying it to translation invariance shows that h ( X ′ ) =
Revised: Applying it to translation invariance shows that h ( X ′ ) =

Original: y ∗ if (cid:12)(cid:12)(cid:12)(cid:12) ∆ − 1 N ∆ (cid:12)(cid:12)(cid:12)(cid:12) 2 < r , where
Revised: y ∗ if (cid:12)(cid:12)(cid:12)(cid:12) ∆ − 1 N ∆ (cid:12)(cid:12)(cid:12)(cid:12) 2 < r , where

Original: X , ∆ ∈ R 1 × D are column-wise averages.
Revised: X , ∆ ∈ R 1 × D are column-wise averages.

Original: For rotation invariance, Theorem 2 guarantees robustnessif (cid:12)(cid:12)(cid:12)(cid:12)
Revised: For rotation invariance, Theorem 2 guarantees robustnessif (cid:12)(cid:12)(cid:12)(cid:12)

Original: X ′ R T − X (cid:12)(cid:12)(cid:12)(cid:12) 2 < r , where R is an optimal rotation matrix defined by the singular value decomposition of X T X ′
Revised: X ′ R T − X (cid:12)(cid:12)(cid:12)(cid:12) 2 < r , where R is an optimal rotation matrix defined by the singular value decomposition of X T X ′

Original: In Appendix C, we provide proves and discuss other invariances.
Revised: In Appendix C, we provide proves and discuss other invariances.

Original: 5.1 Application to randomized smoothing certificates
Revised: 5.1 Application to randomized smoothing certificates

Original: While the post-processing approach may seem straightforward, it has two practical limitations thatwe shall resolve shortly.
Revised: While the post-processing approach may seem straightforward, it has two practical limitations thatwe shall resolve shortly.

Original: Firstly, one may not be able to obtain a certified region B .
Revised: Firstly, one may not be able to obtain a certified region B .

Original: For example, nowhite-box robustness certificate for rotation-invariant models has been proposed thus far.
Revised: For example, nowhite-box robustness certificate for rotation-invariant models has been proposed thus far.

Original: Secondly,randomized smoothing, which can be applied to arbitrary models, only certifies the predictions ofsmoothed classifier f .
Revised: Secondly,randomized smoothing, which can be applied to arbitrary models, only certifies the predictions ofsmoothed classifier f .

Original: Recall from Section 4, that we only know base classifier g to be invariantunder group T .
Revised: Recall from Section 4, that we only know base classifier g to be invariantunder group T .

Original: There is no guarantee that f shares the same invariances, i.e. postprocessing B with 158 T may yield invalid certificates.
Revised: There is no guarantee that f shares the same invariances, i.e. postprocessing B with 158 T may yield invalid certificates.

Original: We address this issue by proving the following result:
Revised: We address this issue by proving the following result:

Original: Theorem 3.
Revised: Theorem 3.

Original: Let base classifier g : R N × D
Revised: Let base classifier g : R N × D

Original: → Y be invariant under a group of transformations T ⊆{
Revised: → Y be invariant under a group of transformations T ⊆{

Original: X (cid:55)→ P η ( X ) | P ∈ S ( N ) , η ∈ E ( D ) } , where S ( N ) is the symmetric group in N dimensions, 161 E ( D ) is the Euclidean group in D dimensions and η : R D
Revised: X (cid:55)→ P η ( X ) | P ∈ S ( N ) , η ∈ E ( D ) } , where S ( N ) is the symmetric group in N dimensions, 161 E ( D ) is the Euclidean group in D dimensions and η : R D

Original: → R D is applied row-wise.
Revised: → R D is applied row-wise.

Original: Then theisotropically smoothed classifier f , as defined in Section 3, is also invariant under T .
Revised: Then theisotropically smoothed classifier f , as defined in Section 3, is also invariant under T .

Original: In other words: Isotropic Gaussian smoothing preserves invariance to any group of transformationscomposed of permutation and Euclidean isometries, i.e. rotation, reflection and translation.
Revised: In other words: Isotropic Gaussian smoothing preserves invariance to any group of transformationscomposed of permutation and Euclidean isometries, i.e. rotation, reflection and translation.

Original: Theorem 3follows from the fact that we smooth by adding zero-mean isotropic noise, meaning that applyingan isometry (e.g. rotation) and then randomizing X is equivalent to first randomizing X and then applying the isometry – which has no effect on an isometry-invariant classifier.
Revised: Theorem 3follows from the fact that we smooth by adding zero-mean isotropic noise, meaning that applyingan isometry (e.g. rotation) and then randomizing X is equivalent to first randomizing X and then applying the isometry – which has no effect on an isometry-invariant classifier.

Original: The formal proof can befound in Appendix C. We may thus instantiate Theorems 1 and 2 with h = f and r = σ Φ − 1 ( p X ,y ∗ ) .
Revised: The formal proof can befound in Appendix C. We may thus instantiate Theorems 1 and 2 with h = f and r = σ Φ − 1 ( p X ,y ∗ ) .

Original:  Tight gray-box certificates
Revised:  Tight gray-box certificates

Original: Now that we have derived the first gray-box certificates for invariant models, one may naturallywonder about their optimality.
Revised: Now that we have derived the first gray-box certificates for invariant models, one may naturallywonder about their optimality.

Original: We answer this by deriving tight gray-box certificates, i.e. the bestcertificates that can be obtained for prediction y ∗ using only the invariances of base classifier g under group T and its prediction probability p X ,y ∗ under clean smoothing distribution µ X .
Revised: We answer this by deriving tight gray-box certificates, i.e. the bestcertificates that can be obtained for prediction y ∗ using only the invariances of base classifier g under group T and its prediction probability p X ,y ∗ under clean smoothing distribution µ X .

Original: To do so, wefind the worst-case invariant classifier, i.e. solve min h ∈ H T Pr Z ∼ µ X
Revised: To do so, wefind the worst-case invariant classifier, i.e. solve min h ∈ H T Pr Z ∼ µ X

Original: [ h ( Z ) =
Revised: [ h ( Z ) =

Original: y ∗ ] with
Revised: y ∗ ] with

Original: [Equation]
Revised: [Equation]

Original: where [ Z ] T is the equivalence class of Z with respect to invariance under group T :
Revised: where [ Z ] T is the equivalence class of Z with respect to invariance under group T :

Original: Definition 2 (Equivalence classes) .
Revised: Definition 2 (Equivalence classes) .

Original: The equivalence class of an input X ∈ R N × D w.r.t.
Revised: The equivalence class of an input X ∈ R N × D w.r.t.

Original: invarianceunder a group of transformations T is [ x ] T = { τ ( x ) | τ ∈ T } .
Revised: invarianceunder a group of transformations T is [ x ] T = { τ ( x ) | τ ∈ T } .

Original: 6.1 Certification methodology
Revised: 6.1 Certification methodology

Original: To work with invariance constraints, it is convenient to not think of h as a function, but a familyof variables ( h Z ) ∈ Y indexed by R N × D .
Revised: To work with invariance constraints, it is convenient to not think of h as a function, but a familyof variables ( h Z ) ∈ Y indexed by R N × D .

Original: The invariance constraint states that all variables froman equivalence class should have the same value, i.e. ∀ Z , ∀ Z ′ ∈
Revised: The invariance constraint states that all variables froman equivalence class should have the same value, i.e. ∀ Z , ∀ Z ′ ∈

Original: [ Z ] T : h Z
Revised: [ Z ] T : h Z

Original: =
Revised: =

Original: h Z ′ .
Revised: h Z ′ .

Original: Intuitively, suchconstraints can be enforced by replacing each occurrence of a variable with a distinct representativeof its equivalence class.
Revised: Intuitively, suchconstraints can be enforced by replacing each occurrence of a variable with a distinct representativeof its equivalence class.

Original: We propose to formalize this idea by using canonical maps:
Revised: We propose to formalize this idea by using canonical maps:

Original: Definition 3 (Canonical map) .
Revised: Definition 3 (Canonical map) .

Original: A canonical map for invariance under a group of transformations Tis a function γ : R N × D
Revised: A canonical map for invariance under a group of transformations Tis a function γ : R N × D

Original: → R N × D with
Revised: → R N × D with

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: In Appendix D.3, we prove that canonical maps let us discard the invariance constraints:
Revised: In Appendix D.3, we prove that canonical maps let us discard the invariance constraints:

Original: Theorem 4.
Revised: Theorem 4.

Original: Let g : R N × D
Revised: Let g : R N × D

Original: → Y be invariant under group T and let H T be defined as in Eq.
Revised: → Y be invariant under group T and let H T be defined as in Eq.

Original: Ifγ : R N × D
Revised: Ifγ : R N × D

Original: →
Revised: →

Original: R N × D is a canonical map for invariance under T , then
Revised: R N × D is a canonical map for invariance under T , then

Original: [Equation]
Revised: [Equation]

Original: H T
Revised: H T

Original: [Equation]
Revised: [Equation]

Original: The unconstrained r.h.s.
Revised: The unconstrained r.h.s.

Original: optimization problem in Theorem 4 could now be solved exactly usingthe Neyman-Pearson lemma – if we could find the distribution of γ ( Z ) , i.e. the distribution ofrepresentatives (which, by Definition 3, is equivalent to a distribution over equivalence classes).
Revised: optimization problem in Theorem 4 could now be solved exactly usingthe Neyman-Pearson lemma – if we could find the distribution of γ ( Z ) , i.e. the distribution ofrepresentatives (which, by Definition 3, is equivalent to a distribution over equivalence classes).

Original: We doso by simplifying the probability integrals in Theorem 4 via carefully chosen substitutions that allowsus to express the distribution over equivalence classes as a marginal distribution over some alternativeparameterization of R N × D .
Revised: We doso by simplifying the probability integrals in Theorem 4 via carefully chosen substitutions that allowsus to express the distribution over equivalence classes as a marginal distribution over some alternativeparameterization of R N × D .

Original: We present a more formal discussion of our method in Appendix D.2.
Revised: We present a more formal discussion of our method in Appendix D.2.

Original: 6.2 Translation invariance
Revised: 6.2 Translation invariance

Original: Applying our procedure to translation invariance (see Appendix D.4) shows the following:
Revised: Applying our procedure to translation invariance (see Appendix D.4) shows the following:

Original: Theorem 5.
Revised: Theorem 5.

Original: Let g : R N × D
Revised: Let g : R N × D

Original: → Y be translation invariant and H T be defined as in Eq.
Revised: → Y be translation invariant and H T be defined as in Eq.

Original: Then
Revised: Then

Original: [Equation]
Revised: [Equation]

Original: where ∆ ∈ R 1 × D are the column-wise average of ∆ =
Revised: where ∆ ∈ R 1 × D are the column-wise average of ∆ =

Original: X ′ − X
Revised: X ′ − X

Original: and σ is the standard deviation ofthe isotropic matrix normal smoothing distribution µ X .
Revised: and σ is the standard deviation ofthe isotropic matrix normal smoothing distribution µ X .

Original: Substituting into robustness condition min h ∈ H T
Revised: Substituting into robustness condition min h ∈ H T

Original: Pr Z ∼ µ X ′
Revised: Pr Z ∼ µ X ′

Original: [ h ( Z ) =
Revised: [ h ( Z ) =

Original: y ∗ ] > 12 shows that f ( X ′ ) =
Revised: y ∗ ] > 12 shows that f ( X ′ ) =

Original: y ∗if (cid:12)(cid:12)(cid:12)(cid:12) ∆ − 1 N ∆ (cid:12)(cid:12)(cid:12)(cid:12) 2 < σ Φ − 1 ( p X ,y ∗ ) .
Revised: y ∗if (cid:12)(cid:12)(cid:12)(cid:12) ∆ − 1 N ∆ (cid:12)(cid:12)(cid:12)(cid:12) 2 < σ Φ − 1 ( p X ,y ∗ ) .

Original: This is identical to the post-processing-based certificate fortranslation invaraince from Section 5, i.e. the post-processing-based certificate is provably tight.
Revised: This is identical to the post-processing-based certificate fortranslation invaraince from Section 5, i.e. the post-processing-based certificate is provably tight.

Original: 6.3 Rotation invariance in 2D
Revised: 6.3 Rotation invariance in 2D

Original: Considering the previous result, one may also suspect the post-processing-based certificate for rotationinvariance, i.e. invariance under (cid:8) X → XR T | R ∈
Revised: Considering the previous result, one may also suspect the post-processing-based certificate for rotationinvariance, i.e. invariance under (cid:8) X → XR T | R ∈

Original: SO
Revised: SO

Original: ( D ) (cid:9) , to be tight.
Revised: ( D ) (cid:9) , to be tight.

Original: This is not the case:
Revised: This is not the case:

Original: Theorem 6.
Revised: Theorem 6.

Original: Let g : R N × D
Revised: Let g : R N × D

Original: → Y be rotation invariant and H T be defined as in Eq.
Revised: → Y be rotation invariant and H T be defined as in Eq.

Original: Furtherassume that ∄ R ∈
Revised: Furtherassume that ∄ R ∈

Original: SO
Revised: SO

Original: ( D ) :
Revised: ( D ) :

Original: X ′ R T =
Revised: X ′ R T =

Original: X
Revised: X

Original: and that p X ,y ∗ ∈ (0 , 1) .
Revised: and that p X ,y ∗ ∈ (0 , 1) .

Original: Then, for all R ∈ SO ( D ) :
Revised: Then, for all R ∈ SO ( D ) :

Original: [Equation]
Revised: [Equation]

Original: Proof in Appendix E.
Revised: Proof in Appendix E.

Original: Our certification procedure, however, lets us derive the tight, strictly strongercertificates for rotation invariance in 2D (see Appendix D.5).
Revised: Our certification procedure, however, lets us derive the tight, strictly strongercertificates for rotation invariance in 2D (see Appendix D.5).

Original: In the following, let ⟨ A , B ⟩ F be the
Revised: In the following, let ⟨ A , B ⟩ F be the

Original: Frobenius inner product and R ( θ ) ∈
Revised: Frobenius inner product and R ( θ ) ∈

Original: SO (2) the matrix that rotates counter-clockwise by angle θ .
Revised: SO (2) the matrix that rotates counter-clockwise by angle θ .

Original: Theorem 7.
Revised: Theorem 7.

Original: Let g : R N × 2 → Y be rotation invariant and H T be defined as in Eq.
Revised: Let g : R N × 2 → Y be rotation invariant and H T be defined as in Eq.

Original: Then
Revised: Then

Original: [Equation]
Revised: [Equation]

Original: h ∈ H
Revised: h ∈ H

Original: [Equation]
Revised: [Equation]

Original: where I 0 is the modified Bessel function of the first kind and order 0 , and the entries of m (1) , m (2) ∈ 213 R, Σ ∈ R 4 × 4 are linear combinations (see Appendix D.5) of clean data norm
Revised: where I 0 is the modified Bessel function of the first kind and order 0 , and the entries of m (1) , m (2) ∈ 213 R, Σ ∈ R 4 × 4 are linear combinations (see Appendix D.5) of clean data norm

Original: ||
Revised: ||

Original: X
Revised: X

Original: || 2 , perturbationnorm ||
Revised: || 2 , perturbationnorm ||

Original: [Equation]
Revised: [Equation]

Original: || 2 and parameters
Revised: || 2 and parameters

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: Certificate parameters.
Revised: Certificate parameters.

Original: Different from the black-box certificate, which only depends on || ∆ || 2 , 216 this certificate also depends on the clean data norm
Revised: Different from the black-box certificate, which only depends on || ∆ || 2 , 216 this certificate also depends on the clean data norm

Original: ||
Revised: ||

Original: X
Revised: X

Original: || 2 .
Revised: || 2 .

Original: It further depends on ϵ 1 and ϵ 2 , Frobenius 217 inner products between ∆ , and the clean input X before and after a rotation by − π 2 .
Revised: It further depends on ϵ 1 and ϵ 2 , Frobenius 217 inner products between ∆ , and the clean input X before and after a rotation by − π 2 .

Original: Theseparameters capture the orientation of X ′ relative to X , as one would expect from a rotation-invarianceaware certificate.
Revised: Theseparameters capture the orientation of X ′ relative to X , as one would expect from a rotation-invarianceaware certificate.

Original: Note that ϵ 1 and ϵ 2 always fulfill (cid:112) ϵ 21 + ϵ 22 ≤
Revised: Note that ϵ 1 and ϵ 2 always fulfill (cid:112) ϵ 21 + ϵ 22 ≤

Original: ||
Revised: ||

Original: X
Revised: X

Original: || 2 || ∆ || 2 (see Appendix H).
Revised: || 2 || ∆ || 2 (see Appendix H).

Original: Certificate evaluation.
Revised: Certificate evaluation.

Original: Evidently, we do not have a closed-form expression for the probabilitiesin Eqs.
Revised: Evidently, we do not have a closed-form expression for the probabilitiesin Eqs.

Original: (6) and (7).
Revised: (6) and (7).

Original: However, recall from Section 3 that randomized smoothing already involves anintractable probability, namely p X ,y ∗ .
Revised: However, recall from Section 3 that randomized smoothing already involves anintractable probability, namely p X ,y ∗ .

Original: One has to use sampling to compute a lower confidence boundthat hold with high probability 1 − α .
Revised: One has to use sampling to compute a lower confidence boundthat hold with high probability 1 − α .

Original: We adopt the same approach:
Revised: We adopt the same approach:

Original: First, we bound threshold tfrom Eq.
Revised: First, we bound threshold tfrom Eq.

Original: Then, we bound probability Pr q ∼N ( m (1) , Σ ) [ ρ ( q ) ≤ t ] from Eq.
Revised: Then, we bound probability Pr q ∼N ( m (1) , Σ ) [ ρ ( q ) ≤ t ] from Eq.

Original: We discuss thefull algorithm and how to ensure that all bounds simultaneously hold in Appendix D.9.
Revised: We discuss thefull algorithm and how to ensure that all bounds simultaneously hold in Appendix D.9.

Original: Because theprobabilities do not depend on base classifier g , one canuse a large number of samples to obtain narrowbounds at little computational cost (e.g. 0 .
Revised: Because theprobabilities do not depend on base classifier g , one canuse a large number of samples to obtain narrowbounds at little computational cost (e.g. 0 .

Original: 59s for 100000 samples on an Intel Xeon E5-2630 CPU).
Revised: 59s for 100000 samples on an Intel Xeon E5-2630 CPU).

Original: Connection to prior work.
Revised: Connection to prior work.

Original: In Appendix D.5 we show that the worst-case invariant classifier, i.e. theminimizer of our optimization problem, is a function that predicts y ∗ if and only if β ( X ′ , Z ) β ( X , Z ) ≤ t withβ ( X , Z ) =
Revised: In Appendix D.5 we show that the worst-case invariant classifier, i.e. theminimizer of our optimization problem, is a function that predicts y ∗ if and only if β ( X ′ , Z ) β ( X , Z ) ≤ t withβ ( X , Z ) =

Original: (cid:82)
Revised: (cid:82)

Original: [Equation]
Revised: [Equation]

Original: (cid:10) X , ZR (
Revised: (cid:10) X , ZR (

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: (cid:11) F (cid:1) dθ.
Revised: (cid:11) F (cid:1) dθ.

Original: Different from black-box randomized smoothing,where the worst-case classifier is a linear model [27], this classifier evaluates Gaussian kernels of( X , Z ) and ( X ′ , Z ) and averages them over all possible rotations.
Revised: Different from black-box randomized smoothing,where the worst-case classifier is a linear model [27], this classifier evaluates Gaussian kernels of( X , Z ) and ( X ′ , Z ) and averages them over all possible rotations.

Original: This group averaging is a keytechnique for building invariant models [7, 107–111] Group-averaged kernels have been proposedin [16].
Revised: This group averaging is a keytechnique for building invariant models [7, 107–111] Group-averaged kernels have been proposedin [16].

Original: It is fascinating to see them naturally materialize from nothing but an invariance constraint.
Revised: It is fascinating to see them naturally materialize from nothing but an invariance constraint.

Original: 6.4 Generalizations
Revised: 6.4 Generalizations

Original: In Appendix D.6 we generalize our result to elemental rotations in 3D, i.e. rotations from SO (3 , d ) :
Revised: In Appendix D.6 we generalize our result to elemental rotations in 3D, i.e. rotations from SO (3 , d ) :

Original: ={ R | R ∈
Revised: ={ R | R ∈

Original: SO
Revised: SO

Original: (3) , R d = e d } with indicator vector e d .
Revised: (3) , R d = e d } with indicator vector e d .

Original: The certificate additionally depends on the 238 amount of perturbation along rotation axis d , i.e.
Revised: The certificate additionally depends on the 238 amount of perturbation along rotation axis d , i.e.

Original: || ∆ : ,d || 2 .
Revised: || ∆ : ,d || 2 .

Original: Note that every model that is invariant 239 to arbitrary rotations is also invariant to elemental rotations.
Revised: Note that every model that is invariant 239 to arbitrary rotations is also invariant to elemental rotations.

Original: In Appendices D.7 and D.8 we furtherprove that additionally enforcing translation invariance is equivalent to centering X and ∆ beforeevaluating the certificates for rotation invariance, which is consistent with our results from Section 6.2.
Revised: In Appendices D.7 and D.8 we furtherprove that additionally enforcing translation invariance is equivalent to centering X and ∆ beforeevaluating the certificates for rotation invariance, which is consistent with our results from Section 6.2.

Original:  Limitations and broader impact
Revised:  Limitations and broader impact

Original: The main limitation of our work lies in its exploratory nature.
Revised: The main limitation of our work lies in its exploratory nature.

Original: In Section 5, we have derivedthe first certificates for models invariant under groups composed of permutation and arbitrary
Revised: In Section 5, we have derivedthe first certificates for models invariant under groups composed of permutation and arbitrary

Original: Euclidean isometries.
Revised: Euclidean isometries.

Original: However, there is a vast swath of invariances that we have not covered, suchas spatio-temporal invariances [42], invariance under graph isometries [9] or invariances for planarimages [7].
Revised: However, there is a vast swath of invariances that we have not covered, suchas spatio-temporal invariances [42], invariance under graph isometries [9] or invariances for planarimages [7].

Original: Furthermore, we have not yet derived tight certificates for all Euclidean isometries(though our approach based on canonical maps could conceivably be applied to them).
Revised: Furthermore, we have not yet derived tight certificates for all Euclidean isometries(though our approach based on canonical maps could conceivably be applied to them).

Original: Broader impact.
Revised: Broader impact.

Original: With the growing prevalence of machine learning in safety-critical and sensitivedomains like autonomous driving [112, 113] or healthcare [114, 115], trustworthy models promise to become increasingly important.
Revised: With the growing prevalence of machine learning in safety-critical and sensitivedomains like autonomous driving [112, 113] or healthcare [114, 115], trustworthy models promise to become increasingly important.

Original: Certification is one pillar of trustworthiness, alongside concepts likefairness [116, 117] or differential privacy [118, 119].
Revised: Certification is one pillar of trustworthiness, alongside concepts likefairness [116, 117] or differential privacy [118, 119].

Original: Unique to our work is that we certify modelswith spatial invariances.
Revised: Unique to our work is that we certify modelswith spatial invariances.

Original: These invariances naturally occur in the physical sciences, including thosewith a direct societal impact like pharmacology and biochemistry.
Revised: These invariances naturally occur in the physical sciences, including thosewith a direct societal impact like pharmacology and biochemistry.

Original: Our work can be seen as a first steptowards provable trustworthiness for tasks like drug discovery [37–39] or protein folding [43–47].
Revised: Our work can be seen as a first steptowards provable trustworthiness for tasks like drug discovery [37–39] or protein folding [43–47].

Original:  Experimental evaluation
Revised:  Experimental evaluation

Original: We know that the post-processing-based certificate for translation invariance is tight and certifiesrobustness for an infinitely larger volume than black-box randomized smoothing (see Fig. 1).
Revised: We know that the post-processing-based certificate for translation invariance is tight and certifiesrobustness for an infinitely larger volume than black-box randomized smoothing (see Fig. 1).

Original: We therefore focus our experiments on the certificates for rotation invariance.
Revised: We therefore focus our experiments on the certificates for rotation invariance.

Original: Recall that thepost-processed certificate guarantees robustness for the set (cid:8) ZR T | R ∈
Revised: Recall that thepost-processed certificate guarantees robustness for the set (cid:8) ZR T | R ∈

Original: SO
Revised: SO

Original: ( D ) , || Z − X
Revised: ( D ) , || Z − X

Original: || 2 ≤ r (cid:9) 261 with r = σ Φ − 1 ( p X ,y ∗ ) .
Revised: || 2 ≤ r (cid:9) 261 with r = σ Φ − 1 ( p X ,y ∗ ) .

Original: In other words: It certifies robustness for perturbations with rotationalcomponents that can be eliminated to bring X ′ into distance r of X .
Revised: In other words: It certifies robustness for perturbations with rotationalcomponents that can be eliminated to bring X ′ into distance r of X .

Original: We want to understand whetherthe tight certificates offer any benefit beyond that, or if the strict inequality in Theorem 6 is due tosome negligible ϵ .
Revised: We want to understand whetherthe tight certificates offer any benefit beyond that, or if the strict inequality in Theorem 6 is due tosome negligible ϵ .

Original: To this end, we first thoroughly examine the four-dimensional parameter spaceof the tight certificate for 2D rotation invariance, before applying our certificates to rotation invariantpoint cloud classifiers.
Revised: To this end, we first thoroughly examine the four-dimensional parameter spaceof the tight certificate for 2D rotation invariance, before applying our certificates to rotation invariantpoint cloud classifiers.

Original: All parameters and experimental details are specified in Appendix B. Areference implementation is made available with the supplementary material.
Revised: All parameters and experimental details are specified in Appendix B. Areference implementation is made available with the supplementary material.

Original: We use 10000 samplesper confidence bound and set α = 0 .
Revised: We use 10000 samplesper confidence bound and set α = 0 .

Original: 001 , i.e. all certificates hold with 99 . 9% probability.
Revised: 001 , i.e. all certificates hold with 99 . 9% probability.

Original: 8.1 Tight certificate parameter space
Revised: 8.1 Tight certificate parameter space

Original: Recall that the tight certificate for rotation invariance depends on || X
Revised: Recall that the tight certificate for rotation invariance depends on || X

Original: || 2 , ||
Revised: || 2 , ||

Original: ∆ || 2 and parameters ϵ 1 andϵ 2 , which capture the orientation of the perturbed point cloud and
Revised: ∆ || 2 and parameters ϵ 1 andϵ 2 , which capture the orientation of the perturbed point cloud and

Original: fulfill (cid:112) ϵ 

Revised: fulfill (cid:112) ϵ

Original: + ϵ 22 ≤
Revised: + ϵ 22 ≤

Original: ||
Revised: ||

Original: X
Revised: X

Original: || 2 · || ∆ || 2 .
Revised: || 2 · || ∆ || 2 .

Original: To avoid clutter, we define ˜ ϵ
Revised: To avoid clutter, we define ˜ ϵ

Original: k :=
Revised: k :=

Original: ϵ k / (
Revised: ϵ k / (

Original: || X || 2 · || ∆ || 2 ) .
Revised: || X || 2 · || ∆ || 2 ) .

Original: As our metric for this section, we report 273 p min , the smallest probability p X ,y ∗ for which a prediction can still be certified
Revised: As our metric for this section, we report 273 p min , the smallest probability p X ,y ∗ for which a prediction can still be certified

Original: [Figure]
Revised: [Figure]

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: ||
Revised: ||

Original: ∆ ||
Revised: ∆ ||

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: ˜ (cid:15)
Revised: ˜ (cid:15)

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: ||
Revised: ||

Original: [Figure]
Revised: [Figure]

Original: |
Revised: |

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: Adversarial scaling.
Revised: Adversarial scaling.

Original: First, we assume that X ′ =
Revised: First, we assume that X ′ =

Original: (1 + β ) X , i.e. the input is adversarially scaled.
Revised: (1 + β ) X , i.e. the input is adversarially scaled.

Original: In this case, we have ˜ ϵ
Revised: In this case, we have ˜ ϵ

Original: = 1 and ˜ ϵ
Revised: = 1 and ˜ ϵ

Original: 2 = 0.
Revised: 2 = 0.

Original: We then vary || ∆ || 

Revised: We then vary || ∆ ||

Original: and || X
Revised: and || X

Original: || 

Revised: ||

Original: and evaluate our certificates.
Revised: and evaluate our certificates.

Original: 276 Note that such attacks have no rotational component, i.e. the post-processing-based certificate isidentical to the black-box one.
Revised: 276 Note that such attacks have no rotational component, i.e. the post-processing-based certificate isidentical to the black-box one.

Original: Fig.
Revised: Fig.

Original: 2 shows that, even in the absence of rotations, the tight certificatecan yield significantly stronger guarantees.
Revised: 2 shows that, even in the absence of rotations, the tight certificatecan yield significantly stronger guarantees.

Original: For σ = 0 .
Revised: For σ = 0 .

Original: 5 and || X
Revised: 5 and || X

Original: || 2 = 0 .
Revised: || 2 = 0 .

Original: 01 , post-processing can only 279 certify robustness for a prediction with p
Revised: 01 , post-processing can only 279 certify robustness for a prediction with p

Original: X ,y ∗ = 0 .
Revised: X ,y ∗ = 0 .

Original: if || ∆ || 2 ≤ 0 . 4 .
Revised: if || ∆ || 2 ≤ 0 . 4 .

Original: The tight certificate can certify 280 robustness up to || ∆ ||
Revised: The tight certificate can certify 280 robustness up to || ∆ ||

Original: 2 = 0 .
Revised: 2 = 0 .

Original: However, the gap shrinks, as the norm of the clean data increases.
Revised: However, the gap shrinks, as the norm of the clean data increases.

Original: Effect of data norm.
Revised: Effect of data norm.

Original: We would like to see if this is a pervasive pattern.
Revised: We would like to see if this is a pervasive pattern.

Original: To this end, we fix || ∆ || 2 , 282 gradually increase ||
Revised: To this end, we fix || ∆ || 2 , 282 gradually increase ||

Original: X
Revised: X

Original: || 2 , and evaluate the tight certificate for ˜ ϵ 1 , ˜ ϵ 2 on a 100 × 100 rasterization of 283 [0 , 1] × [0 , 1] .
Revised: || 2 , and evaluate the tight certificate for ˜ ϵ 1 , ˜ ϵ 2 on a 100 × 100 rasterization of 283 [0 , 1] × [0 , 1] .

Original: We then measure the difference d ( p min ) to the black-box certificate (we will discuss
Revised: We then measure the difference d ( p min ) to the black-box certificate (we will discuss

Original: -   ˜ (cid:15)-   ˜ (cid:15)|| X
Revised: -   ˜ (cid:15)-   ˜ (cid:15)|| X

Original: || =0.
Revised: || =0.

Original: -   ˜ (cid:15)-  1 ||
Revised: -   ˜ (cid:15)-  1 ||

Original: X
Revised: X

Original: || =
Revised: || =

Original: -   ˜ (cid:15)-   ||
Revised: -   ˜ (cid:15)-   ||

Original: X
Revised: X

Original: || =
Revised: || =

Original: -   ˜ (cid:15)-  1 ||
Revised: -   ˜ (cid:15)-  1 ||

Original: X
Revised: X

Original: || =10.
Revised: || =10.

Original: .d
Revised: .d

Original: ( p min ) how these results relate to the post-processing-based one shortly).
Revised: ( p min ) how these results relate to the post-processing-based one shortly).

Original: Fig.
Revised: Fig.

Original: 4 shows that for small ||
Revised: 4 shows that for small ||

Original: X
Revised: X

Original: || 2 , 285 the tight certificate outperforms the black-box certificate for arbitrary perturbations of norm ||
Revised: || 2 , 285 the tight certificate outperforms the black-box certificate for arbitrary perturbations of norm ||

Original: ∆ || 2 . 

Revised: ∆ || 2 .

Original: But, as || X || 2 increases, the regions where it outperforms the black-box one shrink.
Revised: But, as || X || 2 increases, the regions where it outperforms the black-box one shrink.

Original: Rotational components.
Revised: Rotational components.

Original: Simple algebra (see Appendix H) shows that for any combination of || X || 2 288 and || ∆ || 2 , adversarial rotations, i.e. X ′ =
Revised: Simple algebra (see Appendix H) shows that for any combination of || X || 2 288 and || ∆ || 2 , adversarial rotations, i.e. X ′ =

Original: XR T such that || X
Revised: XR T such that || X

Original: ′ − X
Revised: ′ − X

Original: ||
Revised: ||

Original: =
Revised: =

Original: ||
Revised: ||

Original: ∆ || , correspond totwo specific points with (cid:112) ˜ ϵ
Revised: ∆ || , correspond totwo specific points with (cid:112) ˜ ϵ

Original: 21 + ˜ ϵ 

Revised: 21 + ˜ ϵ

Original: = 1 in the certificate’s parameter space.
Revised: = 1 in the certificate’s parameter space.

Original: In Fig.
Revised: In Fig.

Original: 3 we fix σ = 0 .
Revised: 3 we fix σ = 0 .

Original: 5and
Revised: 5and

Original: ||
Revised: ||

Original: X
Revised: X

Original: || 

Revised: ||

Original: = 1 , vary ||
Revised: = 1 , vary ||

Original: ∆ || and highlight these two points.
Revised: ∆ || and highlight these two points.

Original: We observe that the tight certificate 291 only outperforms the black-box certificate for values of ˜ ϵ 1 , ˜ ϵ 2 that are close to adversarial rotations.
Revised: We observe that the tight certificate 291 only outperforms the black-box certificate for values of ˜ ϵ 1 , ˜ ϵ 2 that are close to adversarial rotations.

Original: However, robustness to such attacks can also be readily certified via post-processing.
Revised: However, robustness to such attacks can also be readily certified via post-processing.

Original: Combined withour previous observations, we expect the tight and the post-processing-based certificate to performsimilarly well, assuming that the smoothing standard deviation σ is small relative to ||
Revised: Combined withour previous observations, we expect the tight and the post-processing-based certificate to performsimilarly well, assuming that the smoothing standard deviation σ is small relative to ||

Original: X
Revised: X

Original: || 2 .
Revised: || 2 .

Original: 8.2 Application to point cloud classification
Revised: 8.2 Application to point cloud classification

Original: To verify whether our observations hold in practice, we apply our certificates to point cloud classification.
Revised: To verify whether our observations hold in practice, we apply our certificates to point cloud classification.

Original: We consider two datasets: 3D point cloud maps of ModelNet40 [120], which consists of CADmodels from 40 different categories, and 2D point cloud maps of MNIST [121].
Revised: We consider two datasets: 3D point cloud maps of ModelNet40 [120], which consists of CADmodels from 40 different categories, and 2D point cloud maps of MNIST [121].

Original: We apply the samepre-processing steps as in [6].
Revised: We apply the samepre-processing steps as in [6].

Original: As our base classifiers, we use rotation and translation (i.e. SE ( D ) )invariant versions of two well-established models: PointNet [6] and DGCNN [122].
Revised: As our base classifiers, we use rotation and translation (i.e. SE ( D ) )invariant versions of two well-established models: PointNet [6] and DGCNN [122].

Original: To implementthe invariances, we center the input data, perform principal component analysis, apply the modelto all possible poses (see discussion in [123, 124]) and average the output logits (EnsPointNet and
Revised: To implementthe invariances, we center the input data, perform principal component analysis, apply the modelto all possible poses (see discussion in [123, 124]) and average the output logits (EnsPointNet and

Original: EnsDGCNN).
Revised: EnsDGCNN).

Original: In addition, we consider a more refined model [123] that combines canonical posesvia a self-attention mechanism (AttnPointNet).
Revised: In addition, we consider a more refined model [123] that combines canonical posesvia a self-attention mechanism (AttnPointNet).

Original: Certification is performed on the default test sets.
Revised: Certification is performed on the default test sets.

Original: Practical smoothing parameters.
Revised: Practical smoothing parameters.

Original: Fig.
Revised: Fig.

Original: 5 shows the test set accuracies of randomly smoothed modelsunder varying standard deviations σ .
Revised: 5 shows the test set accuracies of randomly smoothed modelsunder varying standard deviations σ .

Original: Values of σ that preserve an accuracy above 50% are small,
Revised: Values of σ that preserve an accuracy above 50% are small,

Original: [Figure]
Revised: [Figure]

Original: .||
Revised: .||

Original: ∆ || 0 .0 .0 .
Revised: ∆ || 0 .0 .0 .

Original: [Figure]
Revised: [Figure]

Original: [Figure]
Revised: [Figure]

Original: C e r ti fi e d
Revised: C e r ti fi e d

Original: A cc .
Revised: A cc .

Original: SE
Revised: SE

Original: [Figure]
Revised: [Figure]

Original: relative to the average norm of the test sets ( 10 .
Revised: relative to the average norm of the test sets ( 10 .

Original: 67 for MNIST, 19 .
Revised: 67 for MNIST, 19 .

Original: 17 for ModelNet40).
Revised: 17 for ModelNet40).

Original: Going by ourprevious results, we expect the tight and post-processing based certificates to perform similarly well.
Revised: Going by ourprevious results, we expect the tight and post-processing based certificates to perform similarly well.

Original: Adversarial scaling.
Revised: Adversarial scaling.

Original: In Fig.
Revised: In Fig.

Original: 6 we again consider adversarial scaling, i.e. attacks without rotationalcomponents, but applied to the MNIST point cloud dataset with σ = 0 .
Revised: 6 we again consider adversarial scaling, i.e. attacks without rotationalcomponents, but applied to the MNIST point cloud dataset with σ = 0 .

Original: We report the certifiedaccuracy, i.e. the percentage of correct and provably robust predictions, for certification with ( SE (2) )and without ( SO (2) ) translation invariance.
Revised: We report the certifiedaccuracy, i.e. the percentage of correct and provably robust predictions, for certification with ( SE (2) )and without ( SO (2) ) translation invariance.

Original: The tight and post-processing-based certificate yieldsimilar results.
Revised: The tight and post-processing-based certificate yieldsimilar results.

Original: Note that the certified accuracies of the tight method are in fact marginally lower,because we bound the certificates using Monte Carlo sampling (see Section 6.3).
Revised: Note that the certified accuracies of the tight method are in fact marginally lower,because we bound the certificates using Monte Carlo sampling (see Section 6.3).

Original: We further observethat additionally enforcing translation invariance (i.e. SE (2) ) strengthens both approaches.
Revised: We further observethat additionally enforcing translation invariance (i.e. SE (2) ) strengthens both approaches.

Original:      θ 0 .0 .0 .

Revised:      θ 0 .0 .0 .

Original: [Figure]
Revised: [Figure]

Original: [Figure]
Revised: [Figure]

Original: θ
Revised: θ

Original: [Figure]
Revised: [Figure]

Original: Rotational components.
Revised: Rotational components.

Original: Finally, we study perturbations with rotational components.
Revised: Finally, we study perturbations with rotational components.

Original: We fix || ∆ || , 317 randomly sample perturbations of the specified norm and then rotate Z = X + ∆ by a specifiedangle θ ∈ [0 , 10 ◦ ] (in the case of ModelNet40, around one randomly chosen coordinate axis).
Revised: We fix || ∆ || , 317 randomly sample perturbations of the specified norm and then rotate Z = X + ∆ by a specifiedangle θ ∈ [0 , 10 ◦ ] (in the case of ModelNet40, around one randomly chosen coordinate axis).

Original: Foreach element of the test set and each θ , we generate 10 such samples.
Revised: Foreach element of the test set and each θ , we generate 10 such samples.

Original: We then compute the percentageof samples X ′ for which f ( X ) is correct and f ( X ′ )
Revised: We then compute the percentageof samples X ′ for which f ( X ) is correct and f ( X ′ )

Original: =
Revised: =

Original: f ( X ) is provably guaranteed (‘probabilisticcertified accuracy”).
Revised: f ( X ) is provably guaranteed (‘probabilisticcertified accuracy”).

Original: Fig.
Revised: Fig.

Original: 7 shows results for MNIST and ModelNet40 evaluated with || ∆ ||
Revised: 7 shows results for MNIST and ModelNet40 evaluated with || ∆ ||

Original: = σ = 0 .
Revised: = σ = 0 .

Original: 1 322 ( SE (3 , d ) refers to elemental rotations).
Revised: 1 322 ( SE (3 , d ) refers to elemental rotations).

Original: The black-box baseline’s probabilistic certified accuracydrops close to 0 for θ = 2 ◦ .
Revised: The black-box baseline’s probabilistic certified accuracydrops close to 0 for θ = 2 ◦ .

Original: The gray-box certificates are almost constant in θ , i.e. effectivelyeliminate any induced rotation.
Revised: The gray-box certificates are almost constant in θ , i.e. effectivelyeliminate any induced rotation.

Original: However, the tight certificate did not offer any benefit beyond that.
Revised: However, the tight certificate did not offer any benefit beyond that.

Original: We did not observe a single sample for which only the tight certificate could guarantee robustness.
Revised: We did not observe a single sample for which only the tight certificate could guarantee robustness.

Original: In Appendix A.2.4 we consider arbitrary random rotations and observe that the post-processing-basedcertificate for arbitrary rotations is more effective than the tight one for elemental rotations.
Revised: In Appendix A.2.4 we consider arbitrary random rotations and observe that the post-processing-basedcertificate for arbitrary rotations is more effective than the tight one for elemental rotations.

Original: In Appendix A we repeat the experiments from this and the previous section for various othercombinations of parameter values.
Revised: In Appendix A we repeat the experiments from this and the previous section for various othercombinations of parameter values.

Original: All results are consistent with the ones presented here, confirmingthat the post-processing approach offers a good approximation of the tight certificates in practice.
Revised: All results are consistent with the ones presented here, confirmingthat the post-processing approach offers a good approximation of the tight certificates in practice.

Original:  Conclusion
Revised:  Conclusion

Original: For the first time, we have studied the use of invariances for robustness certification.
Revised: For the first time, we have studied the use of invariances for robustness certification.

Original: We proposeda gray-box approach, combining white-box knowledge about invariances with black-box randomizedsmoothing.
Revised: We proposeda gray-box approach, combining white-box knowledge about invariances with black-box randomizedsmoothing.

Original: We have derived a post-processing-based procedure for certification that can be appliedto arbitrary models with invariance to permutations and Euclidean isometries.
Revised: We have derived a post-processing-based procedure for certification that can be appliedto arbitrary models with invariance to permutations and Euclidean isometries.

Original: We have proventhat the post-processed certificate for translation invariance is tight and derived strictly strongercertificates for rotation invariance.
Revised: We have proventhat the post-processed certificate for translation invariance is tight and derived strictly strongercertificates for rotation invariance.

Original: Our experiments are to be interpreted in two ways: The fact thatit is possible to derive tight invariance-aware certificates and there exist scenarios such that they offerstronger guarantees for arbitrary perturbations should be an exciting inspiration for future work.
Revised: Our experiments are to be interpreted in two ways: The fact thatit is possible to derive tight invariance-aware certificates and there exist scenarios such that they offerstronger guarantees for arbitrary perturbations should be an exciting inspiration for future work.

Original: Thefact that post-processing yields semantically meaningful certificates that offer good approximationsof our tight certificates should invite its
Revised: Thefact that post-processing yields semantically meaningful certificates that offer good approximationsof our tight certificates should invite its

Original: Checklist
Revised: Checklist

Original: For all authors...
Revised: For all authors...

Original: (a) Do the main claims made in the abstract and introduction accurately reflect the pa-
Revised: (a) Do the main claims made in the abstract and introduction accurately reflect the pa-

Original: per’s contributions and scope?
Revised: per’s contributions and scope?

Original: [Yes] We derive the described certificates and
Revised: [Yes] We derive the described certificates and

Original: our
Revised: our

Original: experimental results are consistent with the claims in the abstract and introduction.
Revised: experimental results are consistent with the claims in the abstract and introduction.

Original: (b) Did you describe the limitations of your work?
Revised: (b) Did you describe the limitations of your work?

Original: [Yes] See Section 7.
Revised: [Yes] See Section 7.

Original: (c) Did you discuss any potential negative societal impacts of your work?
Revised: (c) Did you discuss any potential negative societal impacts of your work?

Original: [Yes] See Section 7.
Revised: [Yes] See Section 7.

Original: (d) Have you read the ethics review guidelines and ensured that your paper conforms tothem?
Revised: (d) Have you read the ethics review guidelines and ensured that your paper conforms tothem?

Original: [Yes]2.
Revised: [Yes]2.

Original: If you are including theoretical results...
Revised: If you are including theoretical results...

Original: (a) Did you state the full set of assumptions of all theoretical results?
Revised: (a) Did you state the full set of assumptions of all theoretical results?

Original: [Yes]
Revised: [Yes]

Original: (b) Did you include complete proofs of all theoretical results?
Revised: (b) Did you include complete proofs of all theoretical results?

Original: [Yes] See Appendix C for
Revised: [Yes] See Appendix C for

Original: post-processing-based certificates, Appendix D for tight certificates and Appendix Efor the non-tightness of post-processing-based certificates for rotation.
Revised: post-processing-based certificates, Appendix D for tight certificates and Appendix Efor the non-tightness of post-processing-based certificates for rotation.

Original: If you ran experiments...
Revised: If you ran experiments...

Original: (a) Did you include the code, data, and instructions needed to reproduce the main exper-
Revised: (a) Did you include the code, data, and instructions needed to reproduce the main exper-

Original: imental results (either in the supplemental material or as a URL)?
Revised: imental results (either in the supplemental material or as a URL)?

Original: [Yes] Code andinstructions are made available with the supplemental material.
Revised: [Yes] Code andinstructions are made available with the supplemental material.

Original: Links to the datasetsare provided with the readme file.
Revised: Links to the datasetsare provided with the readme file.

Original: (b) Did you specify all the training details (e.g., data splits, hyperparameters, how theywere chosen)?
Revised: (b) Did you specify all the training details (e.g., data splits, hyperparameters, how theywere chosen)?

Original: [Yes] See Appendix B.1.(c)
Revised: [Yes] See Appendix B.1.(c)

Original: Did you report error bars (e.g., with respect to the random seed after running experi-
Revised: Did you report error bars (e.g., with respect to the random seed after running experi-

Original: ments multiple times)?
Revised: ments multiple times)?

Original: [No] However, each individual robustness certificate is basedon confidence intervals, i.e. holds with 99 . 9% probability w.r.t.
Revised: [No] However, each individual robustness certificate is basedon confidence intervals, i.e. holds with 99 . 9% probability w.r.t.

Original: randomness of thesmoothing distribution.
Revised: randomness of thesmoothing distribution.

Original: (d) Did you include the total amount of compute and the type of resources used (e.g., type
Revised: (d) Did you include the total amount of compute and the type of resources used (e.g., type

Original: of GPUs, internal cluster, or cloud provider)?
Revised: of GPUs, internal cluster, or cloud provider)?

Original: [Yes] See Appendix B.3.
Revised: [Yes] See Appendix B.3.

Original: If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
Revised: If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...

Original: (a) If your work uses existing assets, did you cite the creators?
Revised: (a) If your work uses existing assets, did you cite the creators?

Original: [Yes] See Appendix B.4.
Revised: [Yes] See Appendix B.4.

Original: (b) Did you mention the license of the assets?
Revised: (b) Did you mention the license of the assets?

Original: [Yes] See Appendix B.4.
Revised: [Yes] See Appendix B.4.

Original: (c) Did you include any new assets either in the supplemental material or as a URL?
Revised: (c) Did you include any new assets either in the supplemental material or as a URL?

Original: [Yes]
Revised: [Yes]

Original: Yes, our code is made available with the supplemental material.
Revised: Yes, our code is made available with the supplemental material.

Original: (d) Did you discuss whether and how consent was obtained from people whose data you’re
Revised: (d) Did you discuss whether and how consent was obtained from people whose data you’re

Original: using/curating?
Revised: using/curating?

Original: [No] We only experimented on synthetic data.
Revised: [No] We only experimented on synthetic data.

Original: (e) Did you discuss whether the data you are using/curating contains personally identifiable
Revised: (e) Did you discuss whether the data you are using/curating contains personally identifiable

Original: information or offensive content?
Revised: information or offensive content?

Original: [No] We only experimented on synthetic data.
Revised: [No] We only experimented on synthetic data.

Original: If you used crowdsourcing or conducted research with human subjects...
Revised: If you used crowdsourcing or conducted research with human subjects...

Original: (a) Did you include the full text of instructions given to participants and screenshots, if
Revised: (a) Did you include the full text of instructions given to participants and screenshots, if

Original: applicable?
Revised: applicable?

Original: [N/A]
Revised: [N/A]

Original: (b) Did you describe any potential participant risks, with links to Institutional Review
Revised: (b) Did you describe any potential participant risks, with links to Institutional Review

Original: Board (IRB) approvals, if applicable?
Revised: Board (IRB) approvals, if applicable?

Original: [N/A]
Revised: [N/A]

Original: (c) Did you include the estimated hourly wage paid to participants and the total amount
Revised: (c) Did you include the estimated hourly wage paid to participants and the total amount

Original: spent on participant compensation?
Revised: spent on participant compensation?

Original: [N/A]
Revised: [N/A]

Original: A Additional experiments
Revised: A Additional experiments

Original: A.1 Tight certificate parameter space . . . . . . . . . . . . . . . . . . . . . . . . . . .
Revised: A.1 Tight certificate parameter space . . . . . . . . . . . . . . . . . . . . . . . . . . .

Original: A.2 Application to point cloud classification . . . . . . . . . . . . . . . . . . . . . . .
Revised: A.2 Application to point cloud classification . . . . . . . . . . . . . . . . . . . . . . .

Original: B Full experimental setup 

Revised: B Full experimental setup

Original: B.1 Models, data and training parameters . . . . . . . . . . . . . . . . . . . . . . . . .
Revised: B.1 Models, data and training parameters . . . . . . . . . . . . . . . . . . . . . . . . .

Original: B.2 Randomized smoothing parameters . . . . . . . . . . . . . . . . . . . . . . . . . .
Revised: B.2 Randomized smoothing parameters . . . . . . . . . . . . . . . . . . . . . . . . . .

Original: B.3 Computational resources . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Revised: B.3 Computational resources . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Original: B.4 Third-party assets . .
Revised: B.4 Third-party assets . .

Original: C Post-processing-based gray-box certificates 

Revised: C Post-processing-based gray-box certificates

Original: C.1 Proof of Theorem 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Revised: C.1 Proof of Theorem 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Original: C.2 Proof of Theorem 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Revised: C.2 Proof of Theorem 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Original: C.3 Proof of Theorem 3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Revised: C.3 Proof of Theorem 3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Original: C.4 Application of Theorem 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Revised: C.4 Application of Theorem 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Original: D Tight gray-box certificates 

Revised: D Tight gray-box certificates

Original: D.1 Neyman-Pearson lemmmapa . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Revised: D.1 Neyman-Pearson lemmmapa . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Original: D.2 Certification methodology . .
Revised: D.2 Certification methodology . .

Original: D.3 Proof of Theorem 4 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Revised: D.3 Proof of Theorem 4 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Original: D.4 Tight certificate for invariance under translation . . . . . . . . . . . . . . . . . . .
Revised: D.4 Tight certificate for invariance under translation . . . . . . . . . . . . . . . . . . .

Original: D.

Revised: D.

Original: Tight certificate for invariance under rotation in 2D . .
Revised: Tight certificate for invariance under rotation in 2D . .

Original: D.6 Tight certificate for invariance under elemental rotation in 3D . . . . . . . . . . . .
Revised: D.6 Tight certificate for invariance under elemental rotation in 3D . . . . . . . . . . . .

Original: D.7 Tight certificate for simultaneous invariance under rotation and translation in 2D . .
Revised: D.7 Tight certificate for simultaneous invariance under rotation and translation in 2D . .

Original: D.8 Tight certificate for simultaneous invariance under elemental rotation and translationin 3D . . .
Revised: D.8 Tight certificate for simultaneous invariance under elemental rotation and translationin 3D . . .

Original: D.9 Monte Carlo certification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Revised: D.9 Monte Carlo certification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Original: E Proof of Theorem
Revised: E Proof of Theorem

Original: F Multi-class certificates 

Revised: F Multi-class certificates

Original: G
Revised: G

Original: Inverse
Revised: Inverse

Original: certificates 

Revised: certificates

Original: H Parameter space of the tight certificate for 2D rotation invariance 

Revised: H Parameter space of the tight certificate for 2D rotation invariance

Original: H.1 Feasible parameter range . .
Revised: H.1 Feasible parameter range . .

Original: H.2 Adversarial rotations .
Revised: H.2 Adversarial rotations .

Original: A Additional experiments
Revised: A Additional experiments

Original: In the following, we repeat the experiments from Section 8 for a wider range of certificate and modelparameters.
Revised: In the following, we repeat the experiments from Section 8 for a wider range of certificate and modelparameters.

Original: In Appendix A.1, we investigate the parameter space of the tight certificate for rotationinvariance in 2D. In Appendix A.2, we again apply our certificates to rotation invariant point cloudclassifiers, this time considering all three models (EnsPointNet, AttnPointNet and EnsDGCNN) anddifferent smoothing distribution parameters.
Revised: In Appendix A.1, we investigate the parameter space of the tight certificate for rotationinvariance in 2D. In Appendix A.2, we again apply our certificates to rotation invariant point cloudclassifiers, this time considering all three models (EnsPointNet, AttnPointNet and EnsDGCNN) anddifferent smoothing distribution parameters.

Original: As before, we take 10000 samples per confidence boundand set α = 0 .
Revised: As before, we take 10000 samples per confidence boundand set α = 0 .

Original: 001 , i.e. all certificates hold with 99 . 9% probability.
Revised: 001 , i.e. all certificates hold with 99 . 9% probability.

Original: All results support our main conclusion from Section 8: The tight certificates for rotation invariancecan offer much stronger robustness guarantees than their post-processing-based counterparts.
Revised: All results support our main conclusion from Section 8: The tight certificates for rotation invariancecan offer much stronger robustness guarantees than their post-processing-based counterparts.

Original: However, in practical scenarios, where the smoothing standard deviation is small relative to the norm ofthe clean data, the post-processing approach offers a very good approximation.
Revised: However, in practical scenarios, where the smoothing standard deviation is small relative to the norm ofthe clean data, the post-processing approach offers a very good approximation.

Original: A.1 Tight certificate parameter space
Revised: A.1 Tight certificate parameter space

Original: For this section, recall that the tight certificate depends on || X
Revised: For this section, recall that the tight certificate depends on || X

Original: || 2 , ||
Revised: || 2 , ||

Original: ∆ || 2 and parameters ϵ 1 =⟨ X , ∆ ⟩ F , ϵ 

Revised: ∆ || 2 and parameters ϵ 1 =⟨ X , ∆ ⟩ F , ϵ

Original: =
Revised: =

Original: (cid:68) XR ( − π / 2) T , ∆ (cid:69) F , as well as smoothing standard deviation σ .
Revised: (cid:68) XR ( − π / 2) T , ∆ (cid:69) F , as well as smoothing standard deviation σ .

Original: As before, wedefine ˜ ϵ 1 = ϵ 

Revised: As before, wedefine ˜ ϵ 1 = ϵ

Original: || X ||·|| ∆ ||
Revised: || X ||·|| ∆ ||

Original: and ˜ ϵ 

Revised: and ˜ ϵ

Original: = ϵ 2 || X ||·|| ∆ || .
Revised: = ϵ 2 || X ||·|| ∆ || .

Original: Note that (cid:112) ˜
Revised: Note that (cid:112) ˜

Original: ϵ 

Revised: ϵ

Original: + ˜ ϵ 22 ≤ 1 (see also Appendix H).
Revised: + ˜ ϵ 22 ≤ 1 (see also Appendix H).

Original: A.1.1 Adversarial scaling
Revised: A.1.1 Adversarial scaling

Original: We begin with adversarial scaling, which corresponds to ˜ ϵ 1 = 1 , ˜ ϵ 2 = 0 .
Revised: We begin with adversarial scaling, which corresponds to ˜ ϵ 1 = 1 , ˜ ϵ 2 = 0 .

Original: We consider σ ∈{ 0 . 1 , 0 .
Revised: We consider σ ∈{ 0 . 1 , 0 .

Original: For each σ , we vary ||
Revised: For each σ , we vary ||

Original: X
Revised: X

Original: || 2 and || ∆ ||
Revised: || 2 and || ∆ ||

Original: and compute p min , the smallest 803 value of p
Revised: and compute p min , the smallest 803 value of p

Original: X ,y ∗ for which a prediction can still be certified (see Appendix G).
Revised: X ,y ∗ for which a prediction can still be certified (see Appendix G).

Original: Smaller p min mean thatthe certificate can guarantee robustness for models that are less consistent in their predictions.
Revised: Smaller p min mean thatthe certificate can guarantee robustness for models that are less consistent in their predictions.

Original: Fig.
Revised: Fig.

Original: 8 shows that, if the clean data norm is small, e.g. ||
Revised: 8 shows that, if the clean data norm is small, e.g. ||

Original: X
Revised: X

Original: || = 0 .
Revised: || = 0 .

Original: 01 || , then the tight certificate 806 yields much lower p min than the post-processing-based one, save for very small and very largevalues of || ∆ || .
Revised: 01 || , then the tight certificate 806 yields much lower p min than the post-processing-based one, save for very small and very largevalues of || ∆ || .

Original: That is, the tight certificate for rotation invariance is beneficial even for adversarial 808 perturbations that do feature any any rotational components.
Revised: That is, the tight certificate for rotation invariance is beneficial even for adversarial 808 perturbations that do feature any any rotational components.

Original: However, as || X || approaches σ , this 809 difference shrinks, i.e. both approaches offer guarantees of similar strength.
Revised: However, as || X || approaches σ , this 809 difference shrinks, i.e. both approaches offer guarantees of similar strength.

Original: [Figure]
Revised: [Figure]

Original: ||X||=0. ||X||=0.
Revised: ||X||=0. ||X||=0.

Original: (a) σ = 0 .0 .0 .0 .0 .

Revised: (a) σ = 0 .0 .0 .0 .0 .

Original: .||
Revised: .||

Original: ∆ || 0 .0 .

Revised: ∆ || 0 .0 .

Original: .p
Revised: .p

Original: m i n
Revised: m i n

Original: SO (2) Tight
Revised: SO (2) Tight

Original: SO (2) Post ||X||=0.
Revised: SO (2) Post ||X||=0.

Original: ||X||=0.
Revised: ||X||=0.

Original: ||X||=0. ||X||=0.
Revised: ||X||=0. ||X||=0.

Original: (b) σ = 0 .0 .

Revised: (b) σ = 0 .0 .

Original: .||
Revised: .||

Original: ∆ || 0 .0 .

Revised: ∆ || 0 .0 .

Original: .p
Revised: .p

Original: m i n
Revised: m i n

Original: SO (2) Tight
Revised: SO (2) Tight

Original: SO (2) Post ||X||=0.
Revised: SO (2) Post ||X||=0.

Original: ||X||=0.
Revised: ||X||=0.

Original: ||X||=0. ||X||=0.
Revised: ||X||=0. ||X||=0.

Original: (c) σ = 0 .0 .0 .0 .0 .0 .0 .

Revised: (c) σ = 0 .0 .0 .0 .0 .0 .0 .

Original: .||
Revised: .||

Original: ∆ || 0 .0 .

Revised: ∆ || 0 .0 .

Original: .p
Revised: .p

Original: m i n
Revised: m i n

Original: SO (
Revised: SO (

Original: [Table]
Revised: [Table]

Original: ) Tight
Revised: ) Tight

Original: SO (
Revised: SO (

Original: [Table]
Revised: [Table]

Original: Post ||X||=0. ||X||=0.
Revised: Post ||X||=0. ||X||=0.

Original: ||X||=0. ||X||=0.
Revised: ||X||=0. ||X||=0.

Original: (d) σ = 0 .0 . 00 0 .
Revised: (d) σ = 0 .0 . 00 0 .

Original: 1 . 25 1 .|| ∆ || 0 .0 .

Revised: 1 . 25 1 .|| ∆ || 0 .0 .

Original: .p
Revised: .p

Original: m i n
Revised: m i n

Original: SO (2) Tight
Revised: SO (2) Tight

Original: SO (2) Post ||X||=0.
Revised: SO (2) Post ||X||=0.

Original: ||X||=0.
Revised: ||X||=0.

Original: ||X||=0. ||X||=0.
Revised: ||X||=0. ||X||=0.

Original: (e) σ = 0 .0 .0 .1 .1 .2 .2 .||
Revised: (e) σ = 0 .0 .0 .1 .1 .2 .2 .||

Original: ∆ || 0 .0 .

Revised: ∆ || 0 .0 .

Original: .p
Revised: .p

Original: m i
Revised: m i

Original: [Figure]
Revised: [Figure]

Original: SO (
Revised: SO (

Original: [Figure]
Revised: [Figure]

Original: ||X||=
Revised: ||X||=

Original: [Figure]
Revised: [Figure]

Original: ||X||=
Revised: ||X||=

Original: [Figure]
Revised: [Figure]

Original: ||X||=
Revised: ||X||=

Original: [Figure]
Revised: [Figure]

Original: ||X||=0.
Revised: ||X||=0.

Original: (f) σ
Revised: (f) σ

Original: [Figure]
Revised: [Figure]

Original: A.1.2 Effect of clean data norm on certificates for arbitrary perturbations
Revised: A.1.2 Effect of clean data norm on certificates for arbitrary perturbations

Original: Next, we study whether the same effect can be observed with arbitrary perturbations, i.e. arbitraryvalues of ˜ ϵ 1 and ˜ ϵ 2 .
Revised: Next, we study whether the same effect can be observed with arbitrary perturbations, i.e. arbitraryvalues of ˜ ϵ 1 and ˜ ϵ 2 .

Original: In Fig.
Revised: In Fig.

Original: 4 from Section 8.1, we fixed a specific combination of σ and || ∆ || , 813 varied ||
Revised: 4 from Section 8.1, we fixed a specific combination of σ and || ∆ || , 813 varied ||

Original: X
Revised: X

Original: || 

Revised: ||

Original: and evaluated the tight certificate for ˜ ϵ 1 , ˜ ϵ 2 taken from a 100 × 100 rasterization of 814 [0 , 1] × [0 , 1] .
Revised: and evaluated the tight certificate for ˜ ϵ 1 , ˜ ϵ 2 taken from a 100 × 100 rasterization of 814 [0 , 1] × [0 , 1] .

Original: We then measured the difference d ( p min ) to the black-box certificate.
Revised: We then measured the difference d ( p min ) to the black-box certificate.

Original: Here, we consider different combinations of σ , || ∆ || and || X
Revised: Here, we consider different combinations of σ , || ∆ || and || X

Original: || .
Revised: || .

Original: We can reduce the dimensionality 816 of the parameter space that needs to be explored by observing that both the tight gray-box andthe black-box certificate do not depend on the absolute value of these parameters, but their valuerelative to σ .
Revised: We can reduce the dimensionality 816 of the parameter space that needs to be explored by observing that both the tight gray-box andthe black-box certificate do not depend on the absolute value of these parameters, but their valuerelative to σ .

Original: The black-box baseline has p min =
Revised: The black-box baseline has p min =

Original: Φ( || ∆ || 2 / σ ) .
Revised: Φ( || ∆ || 2 / σ ) .

Original: As specified in Appendix D.5, 819 the tight certificate depends on || X
Revised: As specified in Appendix D.5, 819 the tight certificate depends on || X

Original: || 22 / σ 2 , ||
Revised: || 22 / σ 2 , ||

Original: ∆ || 22 / σ 2 , ϵ 1 / σ 2 = 1 σ 

Revised: ∆ || 22 / σ 2 , ϵ 1 / σ 2 = 1 σ

Original: ||
Revised: ||

Original: X
Revised: X

Original: || 2 · || ∆ || 2 · cos( θ 1 ) and1 σ 2 / ||
Revised: || 2 · || ∆ || 2 · cos( θ 1 ) and1 σ 2 / ||

Original: X || 2 · || ∆ || 2 · cos( θ 1 ) , where θ 1 , θ 2 are angles between 2 N -dimensional vectors.
Revised: X || 2 · || ∆ || 2 · cos( θ 1 ) , where θ 1 , θ 2 are angles between 2 N -dimensional vectors.

Original: Therefore, 821 we can fix an arbitrary value of σ and then choose ||
Revised: Therefore, 821 we can fix an arbitrary value of σ and then choose ||

Original: X
Revised: X

Original: || 2 and || ∆ || 2 relative to it.
Revised: || 2 and || ∆ || 2 relative to it.

Original: We set set σ = 1 and consider || ∆ || 2 ∈ { 14 σ, 12 σ, σ, 2 σ, 3 σ } , i.e. perturbation norms that are much 823 smaller, similar to or much larger than the smoothing standard deviation.
Revised: We set set σ = 1 and consider || ∆ || 2 ∈ { 14 σ, 12 σ, σ, 2 σ, 3 σ } , i.e. perturbation norms that are much 823 smaller, similar to or much larger than the smoothing standard deviation.

Original: For each || ∆ || 2 , we evaluate 824 the tight certificate ||
Revised: For each || ∆ || 2 , we evaluate 824 the tight certificate ||

Original: X
Revised: X

Original: || 2 ∈ { 1100 σ, 12 σ, σ, 10 σ } , i.e. clean data norms that are much smaller, similar 825 to or much larger than the smoothing standard deviation
Revised: || 2 ∈ { 1100 σ, 12 σ, σ, 10 σ } , i.e. clean data norms that are much smaller, similar 825 to or much larger than the smoothing standard deviation

Original: [Figure]
Revised: [Figure]

Original: Fig.
Revised: Fig.

Original: 9 shows the resulting d ( p min ) .
Revised: 9 shows the resulting d ( p min ) .

Original: In the case where || ∆ || 2 ≤ 2 ||
Revised: In the case where || ∆ || 2 ≤ 2 ||

Original: X
Revised: X

Original: || 

Revised: ||

Original: we have additionally 827 highlighted the two values of (˜ ϵ 1 , ˜ ϵ 2 ) that correspond to adversarial rotations with blue crosses(see Appendix H)
Revised: we have additionally 827 highlighted the two values of (˜ ϵ 1 , ˜ ϵ 2 ) that correspond to adversarial rotations with blue crosses(see Appendix H)

Original: Note that, to improve readability, the colorbar is scaled differently for each choiceof || ∆
Revised: Note that, to improve readability, the colorbar is scaled differently for each choiceof || ∆

Original: || , i.e. each row.
Revised: || , i.e. each row.

Original: Within each row, the same colorbar is used.
Revised: Within each row, the same colorbar is used.

Original: We can make four observations.
Revised: We can make four observations.

Original: Firstly, in the case that || ∆ || 2 ≤ σ , the tight certificate yields lower (i.e. better) p min for arbitrary 831 perturbations, not just those correponding to adversarial rotations.
Revised: Firstly, in the case that || ∆ || 2 ≤ σ , the tight certificate yields lower (i.e. better) p min for arbitrary 831 perturbations, not just those correponding to adversarial rotations.

Original: Secondly, as the clean data norm ||
Revised: Secondly, as the clean data norm ||

Original: X
Revised: X

Original: || 2 increases, the region where the tight certificate outperforms 833 the black-box baseline shrinks.
Revised: || 2 increases, the region where the tight certificate outperforms 833 the black-box baseline shrinks.

Original: It concentrates around values of (˜ ϵ 1 , ˜ ϵ 2 ) corresponding to adversarialrotations.
Revised: It concentrates around values of (˜ ϵ 1 , ˜ ϵ 2 ) corresponding to adversarialrotations.

Original: Thirdly, the difference in p min is not as drastic when || ∆ || 2 is small.
Revised: Thirdly, the difference in p min is not as drastic when || ∆ || 2 is small.

Original: This is to be expected.
Revised: This is to be expected.

Original: For 836 instance, with || ∆ || 2 = σ / 4 , the black-box certificate has p min =
Revised: For 836 instance, with || ∆ || 2 = σ / 4 , the black-box certificate has p min =

Original: Φ( σ / 4) ≈ 0 . 6 .
Revised: Φ( σ / 4) ≈ 0 . 6 .

Original: Here, the 837 difference to the smallest possible value 0 .
Revised: Here, the 837 difference to the smallest possible value 0 .

Original: 5 is small, meaning there is little room for improvement.
Revised: 5 is small, meaning there is little room for improvement.

Original: We already observed this in Fig. 8.
Revised: We already observed this in Fig. 8.

Original: Finally, when || ∆ || 2 is large relative to σ , and so large that ∆ cannot be the result of an adversarial 840 rotation (e.g.
Revised: Finally, when || ∆ || 2 is large relative to σ , and so large that ∆ cannot be the result of an adversarial 840 rotation (e.g.

Original: ||
Revised: ||

Original: ∆ || 2 = 3 ,
Revised: ∆ || 2 = 3 ,

Original: ||
Revised: ||

Original: X
Revised: X

Original: || 2 = 0 .
Revised: || 2 = 0 .

Original: 01 , then the tight certificate is almost identical to the black-box 841 baseline.
Revised: 01 , then the tight certificate is almost identical to the black-box 841 baseline.

Original: But, as || X || 2 increases and the case of an adversarial rotation becomes possible (e.g.|| ∆ || 2 = 3 ,
Revised: But, as || X || 2 increases and the case of an adversarial rotation becomes possible (e.g.|| ∆ || 2 = 3 ,

Original: ||
Revised: ||

Original: X
Revised: X

Original: || 2 = 10 , then the tight certificate does again yield much better p min , but only for 843 values of (˜ ϵ 1 , ˜ ϵ 2 ) that are very close to adversarial rotations.
Revised: || 2 = 10 , then the tight certificate does again yield much better p min , but only for 843 values of (˜ ϵ 1 , ˜ ϵ 2 ) that are very close to adversarial rotations.

Original: In summary, these observations support our claim that, if || X || is large relative to σ then the tight 845 certificate only improves upon the black-box baseline for perturbations that are similar to adversarialrotations.
Revised: In summary, these observations support our claim that, if || X || is large relative to σ then the tight 845 certificate only improves upon the black-box baseline for perturbations that are similar to adversarialrotations.

Original: Combined with the fact that the post-processing-based gray-box certificate for rotationinvariance accounts for – or using the pre-processing perspective: eliminates – rotational components,this suggests that the post-processing-based certificate should be a good approximation in practice.
Revised: Combined with the fact that the post-processing-based gray-box certificate for rotationinvariance accounts for – or using the pre-processing perspective: eliminates – rotational components,this suggests that the post-processing-based certificate should be a good approximation in practice.

Original: In Fig.
Revised: In Fig.

Original: 10, we repeat the same experiments with σ ,
Revised: 10, we repeat the same experiments with σ ,

Original: ||
Revised: ||

Original: X
Revised: X

Original: || 2 and ||
Revised: || 2 and ||

Original: ∆ || 2 scaled by a factor of 12 , to verify 850 that it is indeed sufficient to only consider an arbitrary σ and the relative value of all other parameters.
Revised: ∆ || 2 scaled by a factor of 12 , to verify 850 that it is indeed sufficient to only consider an arbitrary σ and the relative value of all other parameters.

Original: As expected, the results are identical to Fig.
Revised: As expected, the results are identical to Fig.

Original: In Fig.
Revised: In Fig.

Original: 11 we perform the same experiment for fixed values of σ and ||
Revised: 11 we perform the same experiment for fixed values of σ and ||

Original: X
Revised: X

Original: || 2 and two values of || ∆ || 2 853 chosen such that adversarial rotations correspond to ˜ ϵ
Revised: || 2 and two values of || ∆ || 2 853 chosen such that adversarial rotations correspond to ˜ ϵ

Original: = ≈ 0 .
Revised: = ≈ 0 .

Original: 5 and ˜ ϵ 2 ≈ ± 0 .
Revised: 5 and ˜ ϵ 2 ≈ ± 0 .

Original: 5 , respectively (similar 854 to Fig.
Revised: 5 , respectively (similar 854 to Fig.

Original: 3 from Section 8.1).
Revised: 3 from Section 8.1).

Original: In other words: The adversarial rotations move in 30 ◦ -increments in theparameter space.
Revised: In other words: The adversarial rotations move in 30 ◦ -increments in theparameter space.

Original: Again, we can observe that the regions with large p min move with the adversarialrotations.
Revised: Again, we can observe that the regions with large p min move with the adversarialrotations.

Original: [Table]
Revised: [Table]

Original: ˜
Revised: ˜

Original: [Table]
Revised: [Table]

Original: ||
Revised: ||

Original: X
Revised: X

Original: ||
Revised: ||

Original: [Table]
Revised: [Table]

Original: ||
Revised: ||

Original: X
Revised: X

Original: ||
Revised: ||

Original: [Table]
Revised: [Table]

Original: ˜ (cid:15)
Revised: ˜ (cid:15)

Original: [Table]
Revised: [Table]

Original: ||
Revised: ||

Original: X
Revised: X

Original: ||
Revised: ||

Original: [Table]
Revised: [Table]

Original: -   ˜ (cid:15)-   ˜ (cid:15)|| X
Revised: -   ˜ (cid:15)-   ˜ (cid:15)|| X

Original: || =0.
Revised: || =0.

Original: -   ˜ (cid:15)-   ||
Revised: -   ˜ (cid:15)-   ||

Original: X
Revised: X

Original: || =
Revised: || =

Original: -   ˜ (cid:15)-   ||
Revised: -   ˜ (cid:15)-   ||

Original: X
Revised: X

Original: ||
Revised: ||

Original: =1.
Revised: =1.

Original: -   ˜ (cid:15)-  1 ||
Revised: -   ˜ (cid:15)-  1 ||

Original: X
Revised: X

Original: || =10.
Revised: || =10.

Original: 0 .0 .0 .d ( p min ) (d) ||
Revised: 0 .0 .0 .d ( p min ) (d) ||

Original: ∆ ||
Revised: ∆ ||

Original: = 2 σ -   ˜ (cid:15)-   ˜ (cid:15)|| X
Revised: = 2 σ -   ˜ (cid:15)-   ˜ (cid:15)|| X

Original: || =0.
Revised: || =0.

Original: -   ˜ (cid:15)-   ||
Revised: -   ˜ (cid:15)-   ||

Original: X
Revised: X

Original: || =
Revised: || =

Original: -   ˜ (cid:15)-   ||
Revised: -   ˜ (cid:15)-   ||

Original: X
Revised: X

Original: ||
Revised: ||

Original: =1.
Revised: =1.

Original: -   ˜ (cid:15)-  1 ||
Revised: -   ˜ (cid:15)-  1 ||

Original: X
Revised: X

Original: || =10.
Revised: || =10.

Original: 0 .0 .0 .d ( p min ) (e) ||
Revised: 0 .0 .0 .d ( p min ) (e) ||

Original: ∆ ||
Revised: ∆ ||

Original: = 3 σ
Revised: = 3 σ

Original: -   ˜ (cid:15)-   ˜ (cid:15)|| X
Revised: -   ˜ (cid:15)-   ˜ (cid:15)|| X

Original: || =0.
Revised: || =0.

Original: -   ˜ (cid:15)-  1 ||
Revised: -   ˜ (cid:15)-  1 ||

Original: X
Revised: X

Original: || =
Revised: || =

Original: -   ˜ (cid:15)-   ||
Revised: -   ˜ (cid:15)-   ||

Original: X
Revised: X

Original: || =
Revised: || =

Original: -   ˜
Revised: -   ˜

Original: (cid:15)-
Revised: (cid:15)-

Original: [Table]
Revised: [Table]

Original: ||
Revised: ||

Original: X
Revised: X

Original: || =
Revised: || =

Original: [Table]
Revised: [Table]

Original: [Table] [Table]
Revised: [Table] [Table]

Original: [Table] [Table]
Revised: [Table] [Table]

Original: d (
Revised: d (

Original: [Table]
Revised: [Table]

Original: ) (a) || ∆ ||
Revised: ) (a) || ∆ ||

Original: =
Revised: =

Original: σ /-   
Revised: σ /-

Original: ˜ (cid:15)-   ˜ (cid:15)|| X
Revised: ˜ (cid:15)-   ˜ (cid:15)|| X

Original: || =0.
Revised: || =0.

Original: -   ˜ (cid:15)-  1 ||
Revised: -   ˜ (cid:15)-  1 ||

Original: X
Revised: X

Original: || =
Revised: || =

Original: -   ˜ (cid:15)-   ||
Revised: -   ˜ (cid:15)-   ||

Original: X
Revised: X

Original: || =
Revised: || =

Original: -   ˜ (cid:15)-   ||
Revised: -   ˜ (cid:15)-   ||

Original: X
Revised: X

Original: || =
Revised: || =

Original: [Table] [Table]
Revised: [Table] [Table]

Original: d (
Revised: d (

Original: [Table]
Revised: [Table]

Original: ) (b) ||
Revised: ) (b) ||

Original: ∆ ||
Revised: ∆ ||

Original: =
Revised: =

Original: σ /-   
Revised: σ /-

Original: ˜ (cid:15)-   ˜ (cid:15)|| X
Revised: ˜ (cid:15)-   ˜ (cid:15)|| X

Original: || =0.
Revised: || =0.

Original: -   ˜ (cid:15)-  1 ||
Revised: -   ˜ (cid:15)-  1 ||

Original: X
Revised: X

Original: || =
Revised: || =

Original: -   ˜ (cid:15)-   ||
Revised: -   ˜ (cid:15)-   ||

Original: X
Revised: X

Original: || =
Revised: || =

Original: -   ˜ (cid:15)-   ||
Revised: -   ˜ (cid:15)-   ||

Original: X
Revised: X

Original: || =5.
Revised: || =5.

Original: .d
Revised: .d

Original: ( p min ) (c) ||
Revised: ( p min ) (c) ||

Original: ∆ ||
Revised: ∆ ||

Original: = σ -   ˜
Revised: = σ -   ˜

Original: (cid:15)-   ˜ (cid:15)|| X
Revised: (cid:15)-   ˜ (cid:15)|| X

Original: || =0.
Revised: || =0.

Original: -   ˜ (cid:15)-  1 ||
Revised: -   ˜ (cid:15)-  1 ||

Original: X
Revised: X

Original: || =
Revised: || =

Original: -   ˜ (cid:15)-   ||
Revised: -   ˜ (cid:15)-   ||

Original: X
Revised: X

Original: || =
Revised: || =

Original: -   ˜ (cid:15)-   ||
Revised: -   ˜ (cid:15)-   ||

Original: X
Revised: X

Original: || =5.
Revised: || =5.

Original: 0 .0 .0 .d ( p min ) (d) ||
Revised: 0 .0 .0 .d ( p min ) (d) ||

Original: ∆ ||
Revised: ∆ ||

Original: = 2 σ -   ˜ (cid:15)-   ˜ (cid:15)|| X
Revised: = 2 σ -   ˜ (cid:15)-   ˜ (cid:15)|| X

Original: || =0.
Revised: || =0.

Original: -   ˜ (cid:15)-  1 ||
Revised: -   ˜ (cid:15)-  1 ||

Original: X
Revised: X

Original: || =
Revised: || =

Original: -   ˜ (cid:15)-   ||
Revised: -   ˜ (cid:15)-   ||

Original: X
Revised: X

Original: || =
Revised: || =

Original: -   ˜ (cid:15)-   ||
Revised: -   ˜ (cid:15)-   ||

Original: X
Revised: X

Original: || =5.
Revised: || =5.

Original: 0 .0 .0 .d ( p min ) (e) ||
Revised: 0 .0 .0 .d ( p min ) (e) ||

Original: [Figure]
Revised: [Figure]

Original: ||
Revised: ||

Original: = 3 σ
Revised: = 3 σ

Original: [Table]
Revised: [Table]

Original: ˜ (cid:15)-
Revised: ˜ (cid:15)-

Original: [Table]
Revised: [Table]

Original: ˜
Revised: ˜

Original: [Table]
Revised: [Table]

Original: ||
Revised: ||

Original: ∆ || =
Revised: ∆ || =

Original: [Table] [Table]
Revised: [Table] [Table]

Original: ˜ (cid:15)
Revised: ˜ (cid:15)

Original: [Table]
Revised: [Table]

Original: ||
Revised: ||

Original: ∆ ||
Revised: ∆ ||

Original: [Table] [Table]
Revised: [Table] [Table]

Original: ˜ (cid:15)
Revised: ˜ (cid:15)

Original: [Table] [Table]
Revised: [Table] [Table]

Original: ||
Revised: ||

Original: ∆ ||
Revised: ∆ ||

Original: =1.
Revised: =1.

Original: [Table]
Revised: [Table]

Original: ˜ (cid:15)-
Revised: ˜ (cid:15)-

Original: [Table]
Revised: [Table]

Original: 1 || ∆ ||
Revised: 1 || ∆ ||

Original: =1.
Revised: =1.

Original: [Table] [Table]
Revised: [Table] [Table]

Original: [Table]
Revised: [Table]

Original: 0 .d ( p min ) (e) σ = 1 . 0 ,
Revised: 0 .d ( p min ) (e) σ = 1 . 0 ,

Original: ||
Revised: ||

Original: X
Revised: X

Original: || = 1 .-   ˜
Revised: || = 1 .-   ˜

Original: (cid:15)-   ˜ (cid:15)|| ∆ ||
Revised: (cid:15)-   ˜ (cid:15)|| ∆ ||

Original: =2.
Revised: =2.

Original: -   ˜ (cid:15)-  1 || ∆ || =3.
Revised: -   ˜ (cid:15)-  1 || ∆ || =3.

Original: 0 .0 .0 .d ( p min ) (f) σ = 1 . 0 , ||
Revised: 0 .0 .0 .d ( p min ) (f) σ = 1 . 0 , ||

Original: X
Revised: X

Original: ||
Revised: ||

Original: = 2 .
Revised: = 2 .

Original: A.2 Application to point cloud classification
Revised: A.2 Application to point cloud classification

Original: Next, we repeat the experiments from Section 8.2 for diverse combinations of parameters and classifierarchitectures.
Revised: Next, we repeat the experiments from Section 8.2 for diverse combinations of parameters and classifierarchitectures.

Original: We focus on the case σ ≤ 0 .
Revised: We focus on the case σ ≤ 0 .

Original: 25 , where the randomly smoothed models achieve at least50% accuracy (see Fig. 5), as even smaller accuracies are too low to be of any practical use.
Revised: 25 , where the randomly smoothed models achieve at least50% accuracy (see Fig. 5), as even smaller accuracies are too low to be of any practical use.

Original: We first consider adversarial scaling on the point clouds constructed from MNIST (Appendix A.2.1).
Revised: We first consider adversarial scaling on the point clouds constructed from MNIST (Appendix A.2.1).

Original: We then consider perturbations with random rotation components on MNIST (Appendix A.2.2) andrandom elemental rotation components on ModelNet40 (Appendix A.2.3).
Revised: We then consider perturbations with random rotation components on MNIST (Appendix A.2.2) andrandom elemental rotation components on ModelNet40 (Appendix A.2.3).

Original: In Appendix A.2.4 weadditionally consider arbitrary random rotations.
Revised: In Appendix A.2.4 weadditionally consider arbitrary random rotations.

Original: A.2.1 Adversarial scaling on MNIST
Revised: A.2.1 Adversarial scaling on MNIST

Original: Fig.
Revised: Fig.

Original: 12 shows the certified accuracy (i.e. the number of correct and certifiably robust predictions)under adversarial scaling (i.e. ˜ ϵ 1 = 1 , ˜ ϵ 2 = 0 for the randomly smoothed EnsPointNet with differentσ on the MNIST test set.
Revised: 12 shows the certified accuracy (i.e. the number of correct and certifiably robust predictions)under adversarial scaling (i.e. ˜ ϵ 1 = 1 , ˜ ϵ 2 = 0 for the randomly smoothed EnsPointNet with differentσ on the MNIST test set.

Original: We evaluate the tight and post-processing-based certificate for rotationinvariance ( SO (2) ) and those for simultaneous rotation and translation invariance ( SE (2) .
Revised: We evaluate the tight and post-processing-based certificate for rotationinvariance ( SO (2) ) and those for simultaneous rotation and translation invariance ( SE (2) .

Original: In all cases, the tight and post-processing-based certificates yield similar certified accuracies.
Revised: In all cases, the tight and post-processing-based certificates yield similar certified accuracies.

Original: Thecertified accuracies of the tight certificate are marginally lower, because we compute a probabilisticbound that holds with high probability 1 − α , rather than evaluating it exactly.
Revised: Thecertified accuracies of the tight certificate are marginally lower, because we compute a probabilisticbound that holds with high probability 1 − α , rather than evaluating it exactly.

Original: This effect is amplifiedin the case σ = 0 . 1 , which can be explained by our observations from Appendix A.1: The smaller σis, relative to
Revised: This effect is amplifiedin the case σ = 0 . 1 , which can be explained by our observations from Appendix A.1: The smaller σis, relative to

Original: ||
Revised: ||

Original: X
Revised: X

Original: || 2 (which is defined by each datapoint of the test set and thus constant), the smaller 875 the benefit of using it over the post-processing-based baseline becomes.
Revised: || 2 (which is defined by each datapoint of the test set and thus constant), the smaller 875 the benefit of using it over the post-processing-based baseline becomes.

Original: Furthermore, additionally enforcing translation invariance yields stronger robustness guarantees.
Revised: Furthermore, additionally enforcing translation invariance yields stronger robustness guarantees.

Original: Forinstance, with σ = 0 . 2 , the SE (2) certificates can still certify some predictions for || ∆ ||
Revised: Forinstance, with σ = 0 . 2 , the SE (2) certificates can still certify some predictions for || ∆ ||

Original: 2 = 0 .
Revised: 2 = 0 .

Original: 8 , 878 whereas the SO (2) certificates can not certify robustness beyond || ∆
Revised: 8 , 878 whereas the SO (2) certificates can not certify robustness beyond || ∆

Original: || 2 ≈ 0 .
Revised: || 2 ≈ 0 .

Original: .||
Revised: .||

Original: ∆ || 0 .0 .0 .

Revised: ∆ || 0 .0 .0 .

Original: C e r ti fi e d
Revised: C e r ti fi e d

Original: A cc .
Revised: A cc .

Original: SE (2) Tight
Revised: SE (2) Tight

Original: SE (2) Post
Revised: SE (2) Post

Original: SO (2) Tight
Revised: SO (2) Tight

Original: SO (2) Post (a) σ = 0 .0 .0 .0 .0 .0 .0 .

Revised: SO (2) Post (a) σ = 0 .0 .0 .0 .0 .0 .0 .

Original: .||
Revised: .||

Original: ∆ || 0 .0 .0 .

Revised: ∆ || 0 .0 .0 .

Original: C e r ti fi e d
Revised: C e r ti fi e d

Original: A cc .
Revised: A cc .

Original: SE (2) Tight
Revised: SE (2) Tight

Original: SE (2) Post
Revised: SE (2) Post

Original: SO (2) Tight
Revised: SO (2) Tight

Original: SO (2) Post (b) σ = 0 .0 .0 .0 .0 .

Revised: SO (2) Post (b) σ = 0 .0 .0 .0 .0 .

Original: .||
Revised: .||

Original: ∆ || 0 .0 .0 .

Revised: ∆ || 0 .0 .0 .

Original: C e r ti fi e d
Revised: C e r ti fi e d

Original: A c c .
Revised: A c c .

Original: SE (2) Tight
Revised: SE (2) Tight

Original: SE (2) Post
Revised: SE (2) Post

Original: SO (2) Tight
Revised: SO (2) Tight

Original: SO (2) Post (c) σ = 0 .0 .0 .0 .0 .0 .1 .||
Revised: SO (2) Post (c) σ = 0 .0 .0 .0 .0 .0 .1 .||

Original: ∆ || 0 .0 .0 .

Revised: ∆ || 0 .0 .0 .

Original: C e r ti fi e d
Revised: C e r ti fi e d

Original: A c c .
Revised: A c c .

Original: SE (2) Tight
Revised: SE (2) Tight

Original: SE (2) Post
Revised: SE (2) Post

Original: SO (2) Tight
Revised: SO (2) Tight

Original: SO (2) Post (d) σ = 0 .
Revised: SO (2) Post (d) σ = 0 .

Original: A.2.2 Perturbations with random rotation components on MNIST
Revised: A.2.2 Perturbations with random rotation components on MNIST

Original: Like in Section 8.2, we construct perturbations with rotational components by fixing || ∆ || , randomly 881 sampling perturbations of the specified norm and then rotating Z = X + ∆ by a specified angleθ .
Revised: Like in Section 8.2, we construct perturbations with rotational components by fixing || ∆ || , randomly 881 sampling perturbations of the specified norm and then rotating Z = X + ∆ by a specified angleθ .

Original: We obtain the original, unrotated perturbations via sampling from an isotropic matrix normaldistribution and then scaling the sample to be of the desired norm.
Revised: We obtain the original, unrotated perturbations via sampling from an isotropic matrix normaldistribution and then scaling the sample to be of the desired norm.

Original: Per value of θ , we sample ten suchperturbed inputs per element of the test set (i.e. 100000 per θ ).
Revised: Per value of θ , we sample ten suchperturbed inputs per element of the test set (i.e. 100000 per θ ).

Original: We then compare the tight and thepost-processing-based certificate for simultaneous rotation and translation invariance ( SE ), as well asthe black-box baseline w.r.t.
Revised: We then compare the tight and thepost-processing-based certificate for simultaneous rotation and translation invariance ( SE ), as well asthe black-box baseline w.r.t.

Original: probabilistic certified accuracy, i.e. the percentage of samples X ′ forwhich f ( X ) is correct and f ( X ′ )
Revised: probabilistic certified accuracy, i.e. the percentage of samples X ′ forwhich f ( X ) is correct and f ( X ′ )

Original: =
Revised: =

Original: f ( X ) is provably guaranteed.
Revised: f ( X ) is provably guaranteed.

Original: Fig.
Revised: Fig.

Original: 13 shows the results for σ ∈ { 0 . 05 , 0 . 1 , 0 . 2 } , || ∆ || 2 ∈ { σ / 2 , σ, 2 σ } and θ ∈ [0 ◦ , 90 ◦ ] .
Revised: 13 shows the results for σ ∈ { 0 . 05 , 0 . 1 , 0 . 2 } , || ∆ || 2 ∈ { σ / 2 , σ, 2 σ } and θ ∈ [0 ◦ , 90 ◦ ] .

Original: While 889 the black-box baseline fails to certify robustness even for small rotation angles, both gray-boxcertificates effectively eliminate the rotations, i.e. are constant in θ .
Revised: While 889 the black-box baseline fails to certify robustness even for small rotation angles, both gray-boxcertificates effectively eliminate the rotations, i.e. are constant in θ .

Original: However, the tight gray-boxcertificate does not offer any benefit beyond that.
Revised: However, the tight gray-boxcertificate does not offer any benefit beyond that.

Original: It never yields higher probabilistic certified accuracythan the post-processing-based certificate.
Revised: It never yields higher probabilistic certified accuracythan the post-processing-based certificate.

Original: These results are consistent with our observartions about the tight certificate’s parameter space:
Revised: These results are consistent with our observartions about the tight certificate’s parameter space:

Original: Allvalues of σ that retain high accuracy are small, relative to the average norm of the test set ( 10 . 67 ).
Revised: Allvalues of σ that retain high accuracy are small, relative to the average norm of the test set ( 10 . 67 ).

Original: [Figure]
Revised: [Figure]

Original: P r .
Revised: P r .

Original: C e r t .
Revised: C e r t .

Original: A cc .
Revised: A cc .

Original: (b) σ = 0 .
Revised: (b) σ = 0 .

Original: 05 , ||
Revised: 05 , ||

Original: ∆ ||
Revised: ∆ ||

Original: = 0 .
Revised: = 0 .

Original:    θ / deg 0 .0 .1 .
Revised:    θ / deg 0 .0 .1 .

Original: P r .
Revised: P r .

Original: C e r t .
Revised: C e r t .

Original: A cc .
Revised: A cc .

Original: (c) σ = 0 . 05 , ||
Revised: (c) σ = 0 . 05 , ||

Original: ∆ ||
Revised: ∆ ||

Original: = 0 .
Revised: = 0 .

Original:    θ / deg 0 .0 .1 .
Revised:    θ / deg 0 .0 .1 .

Original: P r .
Revised: P r .

Original: C e r t .
Revised: C e r t .

Original: A cc .
Revised: A cc .

Original: (d) σ = 0 . 1 , || ∆ ||
Revised: (d) σ = 0 . 1 , || ∆ ||

Original: = 0 .
Revised: = 0 .

Original:    θ / deg 0 .0 .1 .
Revised:    θ / deg 0 .0 .1 .

Original: P r .
Revised: P r .

Original: C e r t .
Revised: C e r t .

Original: A cc .
Revised: A cc .

Original: (e) σ = 0 . 1 , || ∆ ||
Revised: (e) σ = 0 . 1 , || ∆ ||

Original: = 0 .
Revised: = 0 .

Original:    θ / deg 0 .0 .1 .
Revised:    θ / deg 0 .0 .1 .

Original: P r .
Revised: P r .

Original: C e r t .
Revised: C e r t .

Original: A cc .
Revised: A cc .

Original: (f) σ = 0 . 1 , || ∆ ||
Revised: (f) σ = 0 . 1 , || ∆ ||

Original: = 0 .
Revised: = 0 .

Original:    θ / deg 0 .0 .1 .
Revised:    θ / deg 0 .0 .1 .

Original: P r .
Revised: P r .

Original: C e r t .
Revised: C e r t .

Original: A cc .
Revised: A cc .

Original: (g) σ = 0 .
Revised: (g) σ = 0 .

Original: 2 , || ∆ ||
Revised: 2 , || ∆ ||

Original: = 0 .
Revised: = 0 .

Original:    θ / deg 0 .0 .1 .
Revised:    θ / deg 0 .0 .1 .

Original: P r .
Revised: P r .

Original: C e r t .
Revised: C e r t .

Original: A cc .
Revised: A cc .

Original: (h) σ = 0 .
Revised: (h) σ = 0 .

Original: 2 , || ∆ ||
Revised: 2 , || ∆ ||

Original: = 0 .
Revised: = 0 .

Original:    θ / deg 0 .0 .1 .
Revised:    θ / deg 0 .0 .1 .

Original: P r .
Revised: P r .

Original: C e r t .
Revised: C e r t .

Original: A cc .
Revised: A cc .

Original: (i) σ = 0 .
Revised: (i) σ = 0 .

Original: 2 , || ∆ ||
Revised: 2 , || ∆ ||

Original: = 0 .
Revised: = 0 .

Original: A.2.3 Perturbations with random elemental rotation components on ModelNet40
Revised: A.2.3 Perturbations with random elemental rotation components on ModelNet40

Original: Next, we repeat the same experiment on ModelNet40.
Revised: Next, we repeat the same experiment on ModelNet40.

Original: After sampling a perturbation ∆ of specifiednorm || ∆ || 2 , we randomly rotate X ′ =
Revised: After sampling a perturbation ∆ of specifiednorm || ∆ || 2 , we randomly rotate X ′ =

Original: X + ∆ by angle θ around one of three coordinate axes 898 (chosen randomly).
Revised: X + ∆ by angle θ around one of three coordinate axes 898 (chosen randomly).

Original: We then compare the tight certificate for simultaneous invariance under elementalrotation and translation, as well as the post-processing-based certificate for simultaneous invarianceunder rotation and translation.
Revised: We then compare the tight certificate for simultaneous invariance under elementalrotation and translation, as well as the post-processing-based certificate for simultaneous invarianceunder rotation and translation.

Original: We evaluate the tight certificate for all three axes and pick the best one.
Revised: We evaluate the tight certificate for all three axes and pick the best one.

Original: Figs.
Revised: Figs.

Original: 14 to 16 show the results for EnsPointNet, AttnPointNet and EnsDGCNN.
Revised: 14 to 16 show the results for EnsPointNet, AttnPointNet and EnsDGCNN.

Original: In all cases, boththe tight- and post-processing-based certificate are constant in θ , but the tight certificate does notimprove upon the post-processing-based one w.r.t.
Revised: In all cases, boththe tight- and post-processing-based certificate are constant in θ , but the tight certificate does notimprove upon the post-processing-based one w.r.t.

Original: probabilistic certified accuracy.
Revised: probabilistic certified accuracy.

Original: [Figure]
Revised: [Figure]

Original: (a) σ = 0 .
Revised: (a) σ = 0 .

Original: [Figure]
Revised: [Figure]

Original: || ∆ || =
Revised: || ∆ || =

Original: [Figure]
Revised: [Figure]

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: |
Revised: |

Original: [Figure]
Revised: [Figure]

Original: (c) σ =
Revised: (c) σ =

Original: [Figure]
Revised: [Figure]

Original: |
Revised: |

Original: [Figure]
Revised: [Figure]

Original: ∆ ||
Revised: ∆ ||

Original: [Figure]
Revised: [Figure]

Original: deg
Revised: deg

Original: [Figure]
Revised: [Figure]

Original: t
Revised: t

Original: [Figure]
Revised: [Figure]

Original: (d) σ = 0 . 1 , || ∆ ||
Revised: (d) σ = 0 . 1 , || ∆ ||

Original: = 0 .
Revised: = 0 .

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: ) σ = 0 . 1 , || ∆ || = 0 .
Revised: ) σ = 0 . 1 , || ∆ || = 0 .

Original:    θ / deg 0 .0 .1 .
Revised:    θ / deg 0 .0 .1 .

Original: P r .
Revised: P r .

Original: C e r t .
Revised: C e r t .

Original: A cc .
Revised: A cc .

Original: (f) σ = 0 . 1 , || ∆ ||
Revised: (f) σ = 0 . 1 , || ∆ ||

Original: = 0 .
Revised: = 0 .

Original:    θ / deg 0 .0 .1 .
Revised:    θ / deg 0 .0 .1 .

Original: P r .
Revised: P r .

Original: C e r t .
Revised: C e r t .

Original: A cc .
Revised: A cc .

Original: (g) σ = 0 .
Revised: (g) σ = 0 .

Original: 2 , || ∆ ||
Revised: 2 , || ∆ ||

Original: = 0 .
Revised: = 0 .

Original:    θ / deg 0 .0 .1 .
Revised:    θ / deg 0 .0 .1 .

Original: P r .
Revised: P r .

Original: C e r t .
Revised: C e r t .

Original: A cc .
Revised: A cc .

Original: (h) σ = 0 .
Revised: (h) σ = 0 .

Original: 2 , || ∆ ||
Revised: 2 , || ∆ ||

Original: = 0 .
Revised: = 0 .

Original:    θ / deg 0 .0 .1 .
Revised:    θ / deg 0 .0 .1 .

Original: P r .
Revised: P r .

Original: C e r t .
Revised: C e r t .

Original: A cc .
Revised: A cc .

Original: (i) σ = 0 .
Revised: (i) σ = 0 .

Original: 2 , || ∆ ||
Revised: 2 , || ∆ ||

Original: = 0 .
Revised: = 0 .

Original: [Table]
Revised: [Table]

Original: P r .
Revised: P r .

Original: C e r t .
Revised: C e r t .

Original: A cc .
Revised: A cc .

Original: [Table]
Revised: [Table]

Original: P r .
Revised: P r .

Original: C e r t .
Revised: C e r t .

Original: A cc .
Revised: A cc .

Original: [Table] [Table]
Revised: [Table] [Table]

Original: θ /
Revised: θ /

Original: [Table] [Table]
Revised: [Table] [Table]

Original: [Table]
Revised: [Table]

Original: P r .
Revised: P r .

Original: C e r t .
Revised: C e r t .

Original: A c c .
Revised: A c c .

Original: [Table]
Revised: [Table]

Original: h
Revised: h

Original: [Table]
Revised: [Table]

Original: |
Revised: |

Original: [Table]
Revised: [Table]

Original: [Equation]
Revised: [Equation]

Original: [Table]
Revised: [Table]

Original: P r .
Revised: P r .

Original: C e r t .
Revised: C e r t .

Original: A cc .
Revised: A cc .

Original: [Table]
Revised: [Table]

Original: P r .
Revised: P r .

Original: C e r t .
Revised: C e r t .

Original: A cc .
Revised: A cc .

Original: [Table] [Table]
Revised: [Table] [Table]

Original: θ /
Revised: θ /

Original: [Table] [Table]
Revised: [Table] [Table]

Original: [Table]
Revised: [Table]

Original: P r .
Revised: P r .

Original: C e r t .
Revised: C e r t .

Original: A c c .
Revised: A c c .

Original: [Table]
Revised: [Table]

Original: h
Revised: h

Original: [Table]
Revised: [Table]

Original: |
Revised: |

Original: [Table]
Revised: [Table]

Original: [Equation]
Revised: [Equation]

Original: A.2.4 Perturbations with arbitrary random rotation components on ModelNet40
Revised: A.2.4 Perturbations with arbitrary random rotation components on ModelNet40

Original: Finally, we want to assess the ability of the tight certificate for invariance under elemental rotationand translation ( SE (3 , d ) ) to certify robustness under perturbations that have arbitrary rotationcomponents.
Revised: Finally, we want to assess the ability of the tight certificate for invariance under elemental rotationand translation ( SE (3 , d ) ) to certify robustness under perturbations that have arbitrary rotationcomponents.

Original: To this end, we sample perturbations ∆ of specified norm ||
Revised: To this end, we sample perturbations ∆ of specified norm ||

Original: ∆ || 2 and randomly rotate 908 X ′ =
Revised: ∆ || 2 and randomly rotate 908 X ′ =

Original: X + ∆ by angle θ around a randomly sampled axis.
Revised: X + ∆ by angle θ around a randomly sampled axis.

Original: We obtain this axis by randomly samplingfrom a three-dimensional isotropic normal distribution and normalizing the result.
Revised: We obtain this axis by randomly samplingfrom a three-dimensional isotropic normal distribution and normalizing the result.

Original: Fig.
Revised: Fig.

Original: 17 shows the result of applying this evaluation scheme to EnsDGCNN on ModelNet40.
Revised: 17 shows the result of applying this evaluation scheme to EnsDGCNN on ModelNet40.

Original: Thetight certificate outperforms the black-box one, as it can at least eliminate some of the rotationalcomponent induced by the sampling procedure.
Revised: Thetight certificate outperforms the black-box one, as it can at least eliminate some of the rotationalcomponent induced by the sampling procedure.

Original: However, unlike the post-processing-based certificatefor invariance under arbitrary rotations and translation ( SE (3) , it is not constant in θ and can notcertify robustness for θ ≥ 10 ◦ .
Revised: However, unlike the post-processing-based certificatefor invariance under arbitrary rotations and translation ( SE (3) , it is not constant in θ and can notcertify robustness for θ ≥ 10 ◦ .

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: (b) σ = 0 .
Revised: (b) σ = 0 .

Original: 05 , ||
Revised: 05 , ||

Original: ∆ ||
Revised: ∆ ||

Original: = 0 .
Revised: = 0 .

Original: [Figure]
Revised: [Figure]

Original: (c) σ = 0 .
Revised: (c) σ = 0 .

Original: [Figure]
Revised: [Figure]

Original: ||
Revised: ||

Original: ∆ ||
Revised: ∆ ||

Original: = 0 .
Revised: = 0 .

Original: [Figure]
Revised: [Figure]

Original: (d) σ = 0 . 1 , || ∆ ||
Revised: (d) σ = 0 . 1 , || ∆ ||

Original: = 0 .
Revised: = 0 .

Original: [Figure]
Revised: [Figure]

Original: θ
Revised: θ

Original: [Figure]
Revised: [Figure]

Original: deg
Revised: deg

Original: [Figure]
Revised: [Figure]

Original: P r .
Revised: P r .

Original: C e r t .
Revised: C e r t .

Original: A cc .
Revised: A cc .

Original: (e) σ = 0 . 1 , || ∆ ||
Revised: (e) σ = 0 . 1 , || ∆ ||

Original: = 0 .
Revised: = 0 .

Original:   θ / deg 0 .0 .1 .
Revised:   θ / deg 0 .0 .1 .

Original: P r .
Revised: P r .

Original: C e r t .
Revised: C e r t .

Original: A cc .
Revised: A cc .

Original: (f) σ = 0 . 1 , || ∆ ||
Revised: (f) σ = 0 . 1 , || ∆ ||

Original: = 0 .
Revised: = 0 .

Original:   θ / deg 0 .0 .1 .
Revised:   θ / deg 0 .0 .1 .

Original: P r .
Revised: P r .

Original: C e r t .
Revised: C e r t .

Original: A cc .
Revised: A cc .

Original: (g) σ = 0 .
Revised: (g) σ = 0 .

Original: 2 , || ∆ ||
Revised: 2 , || ∆ ||

Original: = 0 .
Revised: = 0 .

Original:   θ / deg 0 .0 .1 .
Revised:   θ / deg 0 .0 .1 .

Original: P r .
Revised: P r .

Original: C e r t .
Revised: C e r t .

Original: A cc .
Revised: A cc .

Original: (h) σ = 0 .
Revised: (h) σ = 0 .

Original: 2 , || ∆ ||
Revised: 2 , || ∆ ||

Original: = 0 .
Revised: = 0 .

Original:   θ / deg 0 .0 .1 .
Revised:   θ / deg 0 .0 .1 .

Original: P r .
Revised: P r .

Original: C e r t .
Revised: C e r t .

Original: A cc .
Revised: A cc .

Original: (i) σ = 0 .
Revised: (i) σ = 0 .

Original: 2 , || ∆ ||
Revised: 2 , || ∆ ||

Original: = 0 .
Revised: = 0 .

Original: B Full experimental setup
Revised: B Full experimental setup

Original: In the following, we provide a full specification of the used models, data sets and training parameters(Appendix B.1, randomized smoothing parameters (Appendix B.2), as well as the used computationalresources (Appendix B.3) and third-party assets (Appendix B.4).
Revised: In the following, we provide a full specification of the used models, data sets and training parameters(Appendix B.1, randomized smoothing parameters (Appendix B.2), as well as the used computationalresources (Appendix B.3) and third-party assets (Appendix B.4).

Original: The code and all configuration files needed for reproducing the experimental results are included withthe supplementary material.
Revised: The code and all configuration files needed for reproducing the experimental results are included withthe supplementary material.

Original: B.1 Models, data and training parametersB.1.1 Models
Revised: B.1 Models, data and training parametersB.1.1 Models

Original: The experiments in Section 8.2 and Appendix A.2 are performed with three models: EnsPointNet,
Revised: The experiments in Section 8.2 and Appendix A.2 are performed with three models: EnsPointNet,

Original: EnsDGCNN and AttnPointNet, which are rotation and translation invariant versions of PointNet [6]and the Dynamic Graph Convolutional Neural Network [122].
Revised: EnsDGCNN and AttnPointNet, which are rotation and translation invariant versions of PointNet [6]and the Dynamic Graph Convolutional Neural Network [122].

Original: EnsPointNet is based on a standard PointNet architecture with an input T-Net but without a feature
Revised: EnsPointNet is based on a standard PointNet architecture with an input T-Net but without a feature

Original: T-Net (we did not find the feature T-Net to improve the accuracy of the rotation invariant model).
Revised: T-Net (we did not find the feature T-Net to improve the accuracy of the rotation invariant model).

Original: Theinput T-Net uses three convolutional layers with kernel size 1 × 1 ( 64 , 128 and 1024 filters) and threelinear layers ( 512 , 256 and 9 neurons).
Revised: Theinput T-Net uses three convolutional layers with kernel size 1 × 1 ( 64 , 128 and 1024 filters) and threelinear layers ( 512 , 256 and 9 neurons).

Original: The PointNet itself three convolutional layers with kernelsize 1 × 1 ( 64 , 128 and 1024 filters) and three linear layers ( 512 , 256 and | Y | neurons).
Revised: The PointNet itself three convolutional layers with kernelsize 1 × 1 ( 64 , 128 and 1024 filters) and three linear layers ( 512 , 256 and | Y | neurons).

Original: All layers,except the last one, use BatchNorm ( ϵ = 1 e − 05 , momentum = 0 . 1 ).
Revised: All layers,except the last one, use BatchNorm ( ϵ = 1 e − 05 , momentum = 0 . 1 ).

Original: The second linear layer usesdropout ( p = 0 . 4 ).
Revised: The second linear layer usesdropout ( p = 0 . 4 ).

Original: To achieve rotation and translation invariance, the input data is centered and thetwo (for 2 D data) or three (for 3 D data) principal components are computed.
Revised: To achieve rotation and translation invariance, the input data is centered and thetwo (for 2 D data) or three (for 3 D data) principal components are computed.

Original: Principal componentsare not unique.
Revised: Principal componentsare not unique.

Original: One has to account for order and sign ambiguities to ensure rotation invariance (seediscussion in [123, 124]).
Revised: One has to account for order and sign ambiguities to ensure rotation invariance (seediscussion in [123, 124]).

Original: If two or more eigenvalues are numerically close (relative tolerance 1 e − 5 ,absolute tolerance 1 e − 8 ) , we iterate over all possible eigenvector signs and orders ( 8 · 6 ), multiplythe input data with the principal components, and pass it through the PointNet.
Revised: If two or more eigenvalues are numerically close (relative tolerance 1 e − 5 ,absolute tolerance 1 e − 8 ) , we iterate over all possible eigenvector signs and orders ( 8 · 6 ), multiplythe input data with the principal components, and pass it through the PointNet.

Original: If the eigenvaluesare distinct, we sort them in ascending order and iterate over all possible signs ( 8 ).
Revised: If the eigenvaluesare distinct, we sort them in ascending order and iterate over all possible signs ( 8 ).

Original: The 8 or 48 logitvectors are then averaged to obtain a prediction.
Revised: The 8 or 48 logitvectors are then averaged to obtain a prediction.

Original: EnsDGCNN is based on a standard DGCNN architecture with an input spatial transform.
Revised: EnsDGCNN is based on a standard DGCNN architecture with an input spatial transform.

Original: The spatialtransform uses uses three convolutional layers with kernel size 1 × 1 ( 64 , 128 and 1024 filters) andthree linear layers ( 512 , 256 and 9 neurons).
Revised: The spatialtransform uses uses three convolutional layers with kernel size 1 × 1 ( 64 , 128 and 1024 filters) andthree linear layers ( 512 , 256 and 9 neurons).

Original: The DGCNN encodes the spatially transformed inputwith three DGC layers ( 64 , 64 , 64 and 128 filters and k = 20 ).
Revised: The DGCNN encodes the spatially transformed inputwith three DGC layers ( 64 , 64 , 64 and 128 filters and k = 20 ).

Original: The encoder output and residuals areconcatenated and passed through a convolution with kernel size 1 × 1 ( 1024 filters), followed bymax-pooling and three linear layers ( 512 , 256 , | Y | neurons).
Revised: The encoder output and residuals areconcatenated and passed through a convolution with kernel size 1 × 1 ( 1024 filters), followed bymax-pooling and three linear layers ( 512 , 256 , | Y | neurons).

Original: All layers use BatchNorm ( ϵ = 1 e − 05 ,momentum = 0 . 1 ).
Revised: All layers use BatchNorm ( ϵ = 1 e − 05 ,momentum = 0 . 1 ).

Original: The first two linear layers use dropout ( p = 0 . 5 ).
Revised: The first two linear layers use dropout ( p = 0 . 5 ).

Original: We use the same ensemblingapproach as for EnsPointNet to achieve rotation and translation invariance.
Revised: We use the same ensemblingapproach as for EnsPointNet to achieve rotation and translation invariance.

Original: AttnPointNet combines PointNet with the attention-based mechanism for combining canonical posesproposed in [123].
Revised: AttnPointNet combines PointNet with the attention-based mechanism for combining canonical posesproposed in [123].

Original: It uses the same encoder as EnsPointNet.
Revised: It uses the same encoder as EnsPointNet.

Original: After passing the different PCA-basedcanonical poses through the encoder, the hidden vectors are combined via a self-attention layer ( 1024neurons each for query, key and value transform) and then passed through the same decoder as in
Revised: After passing the different PCA-basedcanonical poses through the encoder, the hidden vectors are combined via a self-attention layer ( 1024neurons each for query, key and value transform) and then passed through the same decoder as in

Original: EnsPointNet.
Revised: EnsPointNet.

Original: To reduce memory usage, we only consider sign combinations that correspond toproper rotation matrices (see discussion in [124]).
Revised: To reduce memory usage, we only consider sign combinations that correspond toproper rotation matrices (see discussion in [124]).

Original: B.1.2 Data
Revised: B.1.2 Data

Original: ModelNet40 [120] consists of 12311 CAD models from 40 categories, split into 9843 trainingsamples and 2468 test samples.
Revised: ModelNet40 [120] consists of 12311 CAD models from 40 categories, split into 9843 trainingsamples and 2468 test samples.

Original: We subdivide the original training set into 80% train data and 20%validation data.
Revised: We subdivide the original training set into 80% train data and 20%validation data.

Original: The same split is used for all experiments.
Revised: The same split is used for all experiments.

Original: Each CAD model is then transformedinto a 3D point cloud with 1024 points using the same pre-processing steps as in [6], i.e. randomlysampling from the mesh faces according to surface area and normalizing the resulting point cloudinto the unit sphere.
Revised: Each CAD model is then transformedinto a 3D point cloud with 1024 points using the same pre-processing steps as in [6], i.e. randomlysampling from the mesh faces according to surface area and normalizing the resulting point cloudinto the unit sphere.

Original: MNIST [121] consists of 70000 handwritten digits, split into 60000 training samples and 10000 testsamples.
Revised: MNIST [121] consists of 70000 handwritten digits, split into 60000 training samples and 10000 testsamples.

Original: We subdivide the original training set into 80% train data and 20% validation data.
Revised: We subdivide the original training set into 80% train data and 20% validation data.

Original: Thesame split is used for all experiments.
Revised: Thesame split is used for all experiments.

Original: Each image is transformed into a 2D point with 1024 pointsusing the same pre-processing st eps as in [6], i.e. mapping all pixels with values greater than 128 tox-y coordinates, then uniformly sub-sampling or padding with a randomly chosen point and finally normalizing into the unit sphere.
Revised: Each image is transformed into a 2D point with 1024 pointsusing the same pre-processing st eps as in [6], i.e. mapping all pixels with values greater than 128 tox-y coordinates, then uniformly sub-sampling or padding with a randomly chosen point and finally normalizing into the unit sphere.

Original: We additionally combine classes 6 and 9 , because these digitscannot be expected to be differentiated by a rotation-invariant model.
Revised: We additionally combine classes 6 and 9 , because these digitscannot be expected to be differentiated by a rotation-invariant model.

Original: B.1.3 Training parameters
Revised: B.1.3 Training parameters

Original: All models are trained on samples from the smoothing distribution (i.e. the model that is evaluatedand certified with σ is trained on data augmented with Gaussian noise sampled from an isotropicnormal distribution with standard deviation σ ).
Revised: All models are trained on samples from the smoothing distribution (i.e. the model that is evaluatedand certified with σ is trained on data augmented with Gaussian noise sampled from an isotropicnormal distribution with standard deviation σ ).

Original: Per element of a training batch, we take exactly onesample from the smoothing distribution.
Revised: Per element of a training batch, we take exactly onesample from the smoothing distribution.

Original: We do not use consistency regularization.
Revised: We do not use consistency regularization.

Original: EnsPointNet is trained with cross entropy loss and the TNet regularization from [6] (weight 0 . 0001 ).
Revised: EnsPointNet is trained with cross entropy loss and the TNet regularization from [6] (weight 0 . 0001 ).

Original: Training is performed for 200 epochs with batch size 128 using Adam ( β 1 = 0 .
Revised: Training is performed for 200 epochs with batch size 128 using Adam ( β 1 = 0 .

Original: 9 , β 2 = 0 .
Revised: 9 , β 2 = 0 .

Original: 99 ,ϵ = 1 e − 8 , weight _ decay = 1 e − 4 ).
Revised: 99 ,ϵ = 1 e − 8 , weight _ decay = 1 e − 4 ).

Original: The learning rate starts at 0 .
Revised: The learning rate starts at 0 .

Original: 001 and is multiplied with 0 .
Revised: 001 and is multiplied with 0 .

Original: 7every 20 epochs.
Revised: 7every 20 epochs.

Original: The input data is randomly scaled by a factor from [0 . 8 , 1 . 25] and then transformedvia principal component analysis, using a single set of eigenvectors with randomized sign and order.
Revised: The input data is randomly scaled by a factor from [0 . 8 , 1 . 25] and then transformedvia principal component analysis, using a single set of eigenvectors with randomized sign and order.

Original: EnsDGCNN is trained in the same manner as EnsPointNet, but with batch size 32 , 400 epochs andlearning rate decay every 40 epochs.
Revised: EnsDGCNN is trained in the same manner as EnsPointNet, but with batch size 32 , 400 epochs andlearning rate decay every 40 epochs.

Original: AttnPointNet is trained with batch size 64 , 400 epochs and learning rate decay every 40 epochs.
Revised: AttnPointNet is trained with batch size 64 , 400 epochs and learning rate decay every 40 epochs.

Original: Theother parameter values are identical to those for EnsPointNet.
Revised: Theother parameter values are identical to those for EnsPointNet.

Original: In order to train the self-attention layer,we do not transform the input with a single set of eigenvectors, but pass all possible eigenvectorscorresponding to rotations through the network.
Revised: In order to train the self-attention layer,we do not transform the input with a single set of eigenvectors, but pass all possible eigenvectorscorresponding to rotations through the network.

Original: B.2 Randomized smoothing parameters
Revised: B.2 Randomized smoothing parameters

Original: We use 1000 samples from the smoothing distribution to compute smoothed predictions.
Revised: We use 1000 samples from the smoothing distribution to compute smoothed predictions.

Original: Abstentions,i.e. predictions for which we cannot guarantee that p X ,y ∗ ≥ 0 .
Revised: Abstentions,i.e. predictions for which we cannot guarantee that p X ,y ∗ ≥ 0 .

Original: 5 , are considered as incorrect.
Revised: 5 , are considered as incorrect.

Original: For certification, we set the significance parameter α to 0 .
Revised: For certification, we set the significance parameter α to 0 .

Original: i.e. each certificate holds with probability99 .
Revised: i.e. each certificate holds with probability99 .

Original: We use 10000 samples per confidence bound, i.e. 10000 samples to bound p X ,y ∗ and – forthe tight gray-box certificates involving rotation invariance – 10000 samples to bound the threshold tand 10000 samples to bound the optimal value itself (for more details, see Appendix D.9).
Revised: We use 10000 samples per confidence bound, i.e. 10000 samples to bound p X ,y ∗ and – forthe tight gray-box certificates involving rotation invariance – 10000 samples to bound the threshold tand 10000 samples to bound the optimal value itself (for more details, see Appendix D.9).

Original: For thetight gray-box certificates involving rotation invariance, we use Holm-Bonferroni correction ([125])to ensure that all three confidence bounds hold simultaneously with probability 1 − α .
Revised: For thetight gray-box certificates involving rotation invariance, we use Holm-Bonferroni correction ([125])to ensure that all three confidence bounds hold simultaneously with probability 1 − α .

Original: B.3 Computational resources
Revised: B.3 Computational resources

Original: Training and Monte Carlo sampling of smoothed predictions was performed on a single NVIDIA
Revised: Training and Monte Carlo sampling of smoothed predictions was performed on a single NVIDIA

Original: A100 GPU ( 40 GB VRAM) with an AMD EPYC 7543 CPU @ 2 . 8 GHz .
Revised: A100 GPU ( 40 GB VRAM) with an AMD EPYC 7543 CPU @ 2 . 8 GHz .

Original: For EnsPointNet and
Revised: For EnsPointNet and

Original: AttnPointNet, 16 GB RAM were allocated.
Revised: AttnPointNet, 16 GB RAM were allocated.

Original: For EnsDGCNN, which requires additional memoryfor computation of k-nearest-neighbor graphs, 32 GB RAM were allocated.
Revised: For EnsDGCNN, which requires additional memoryfor computation of k-nearest-neighbor graphs, 32 GB RAM were allocated.

Original: The average time fortraining EnsPointNet on MNIST was 42 .
Revised: The average time fortraining EnsPointNet on MNIST was 42 .

Original: 8 min .
Revised: 8 min .

Original: The average time for training EnsPointNet, AttnPointand EnsDGCNN on ModelNet40 was 20 .
Revised: The average time for training EnsPointNet, AttnPointand EnsDGCNN on ModelNet40 was 20 .

Original: 2 min , 114 min and 158 min , respectively.
Revised: 2 min , 114 min and 158 min , respectively.

Original: The averagetime for obtaining 11000 samples from EnsPointNet on MNIST was 1 .
Revised: The averagetime for obtaining 11000 samples from EnsPointNet on MNIST was 1 .

Original: 1 s .
Revised: 1 s .

Original: The average time forobtaining 11000 samples from EnsPointNet, AttnPoint and EnsDGCNN on ModelNet40 was 1 .
Revised: The average time forobtaining 11000 samples from EnsPointNet, AttnPoint and EnsDGCNN on ModelNet40 was 1 .

Original: 59 s ,1 . 4 s and 24 .
Revised: 59 s ,1 . 4 s and 24 .

Original: 91 s , respectively.
Revised: 91 s , respectively.

Original: Computation of certificates was performed on an Intel Xeon E5-2630 v4 CPU @ 2 . 2 GHz , with16 GB RAM allocated to each experiment.
Revised: Computation of certificates was performed on an Intel Xeon E5-2630 v4 CPU @ 2 . 2 GHz , with16 GB RAM allocated to each experiment.

Original: The average time for computing a tight certificate for 2Drotation invarance (i.e. computing threshold t and bounding the optimal value, using 10000 sampleseach), was 0 .
Revised: The average time for computing a tight certificate for 2Drotation invarance (i.e. computing threshold t and bounding the optimal value, using 10000 sampleseach), was 0 .

Original: 05 s .
Revised: 05 s .

Original: The average time for computing a tight certificate for 3D rotation invariance was0 .
Revised: The average time for computing a tight certificate for 3D rotation invariance was0 .

Original: 06 s .
Revised: 06 s .

Original: The average time for computing the black-box randomized smoothing certificate and the tightcertificate for translation invariance was in the sub-milisecond range and can thus not be accuratelyreported.
Revised: The average time for computing the black-box randomized smoothing certificate and the tightcertificate for translation invariance was in the sub-milisecond range and can thus not be accuratelyreported.

Original: B.4 Third-party assets
Revised: B.4 Third-party assets

Original: Our work uses the publicly available MNIST [121] and ModelNet40 [120] datasets.
Revised: Our work uses the publicly available MNIST [121] and ModelNet40 [120] datasets.

Original: Our implementation additionally uses code from a publicly available implementation of PointNet and PointNet++ [126], as well as the author’s reference implementation of DGCNN [122].
Revised: Our implementation additionally uses code from a publicly available implementation of PointNet and PointNet++ [126], as well as the author’s reference implementation of DGCNN [122].

Original: Both are availableunder MIT license.
Revised: Both are availableunder MIT license.

Original: C Post-processing-based gray-box certificates
Revised: C Post-processing-based gray-box certificates

Original: In the following, we first prove the soundness of our post-processing-based approach to gray-boxrobustness certification (Appendix C.1).
Revised: In the following, we first prove the soundness of our post-processing-based approach to gray-boxrobustness certification (Appendix C.1).

Original: We then prove the soundness of the equivalent pre-processingperspective for Frobenius norm certificates (Appendix C.2) and that isotropic Gaussian smoothingpreserves invariance under groups composed of permutation and isometries of the Euclidean space(Appendix C.3).
Revised: We then prove the soundness of the equivalent pre-processingperspective for Frobenius norm certificates (Appendix C.2) and that isotropic Gaussian smoothingpreserves invariance under groups composed of permutation and isometries of the Euclidean space(Appendix C.3).

Original: We conclude by discussing how the optimization problem underlying the preprocessing perspective can be solved for different invariances (Appendix C.4).
Revised: We conclude by discussing how the optimization problem underlying the preprocessing perspective can be solved for different invariances (Appendix C.4).

Original: C.1 Proof of Theorem 1
Revised: C.1 Proof of Theorem 1

Original: Theorem 1.
Revised: Theorem 1.

Original: Let h ∈ R N × D
Revised: Let h ∈ R N × D

Original: → Y be invariant under a group of transformations T .
Revised: → Y be invariant under a group of transformations T .

Original: Let y ∗
Revised: Let y ∗

Original: =
Revised: =

Original: h ( X )be a prediction that is certifiably robust to a set of perturbed inputs B ⊆ R N × D , i.e. ∀ Z ∈ B :h ( Z ) =
Revised: h ( X )be a prediction that is certifiably robust to a set of perturbed inputs B ⊆ R N × D , i.e. ∀ Z ∈ B :h ( Z ) =

Original: y ∗ .
Revised: y ∗ .

Original: Let ˜ B
Revised: Let ˜ B

Original: = { τ ( Z ) | Z ∈ B , τ ∈ T } .
Revised: = { τ ( Z ) | Z ∈ B , τ ∈ T } .

Original: Then ∀ X ′ ∈ ˜ B : h ( X ′ )
Revised: Then ∀ X ′ ∈ ˜ B : h ( X ′ )

Original: =
Revised: =

Original: y ∗ .
Revised: y ∗ .

Original: Proof.
Revised: Proof.

Original: Consider an arbitrary X ′ ∈ ˜ B .
Revised: Consider an arbitrary X ′ ∈ ˜ B .

Original: Due to the definition of ˜ B , there must be a Z ∈ B and a τ ∈ Tsuch that X ′ =
Revised: Due to the definition of ˜ B , there must be a Z ∈ B and a τ ∈ Tsuch that X ′ =

Original: τ ( Z ) .
Revised: τ ( Z ) .

Original: Since τ ∈ T , we know that h ( τ ( Z ))
Revised: Since τ ∈ T , we know that h ( τ ( Z ))

Original: =
Revised: =

Original: h ( Z ) .
Revised: h ( Z ) .

Original: Since Z ∈ B , we know thath ( Z ) =
Revised: Since Z ∈ B , we know thath ( Z ) =

Original: y ∗ .
Revised: y ∗ .

Original: By the transitive property, we have h ( X ′ )
Revised: By the transitive property, we have h ( X ′ )

Original: =
Revised: =

Original: y ∗ .
Revised: y ∗ .

Original: C.2 Proof of Theorem 2
Revised: C.2 Proof of Theorem 2

Original: Theorem 2.
Revised: Theorem 2.

Original: Let h ∈ R N × D
Revised: Let h ∈ R N × D

Original: → Y be invariant under a group of transformations T .
Revised: → Y be invariant under a group of transformations T .

Original: Let y ∗
Revised: Let y ∗

Original: =
Revised: =

Original: h ( X )be a prediction that is certifiably robust to a set of perturbed inputs B = { Z | ||
Revised: h ( X )be a prediction that is certifiably robust to a set of perturbed inputs B = { Z | ||

Original: Z − X
Revised: Z − X

Original: || 2 < r } , i.e.∀ Z ∈ B : h ( Z ) =
Revised: || 2 < r } , i.e.∀ Z ∈ B : h ( Z ) =

Original: y ∗ .
Revised: y ∗ .

Original: If min τ ∈ T || τ ( X ′ ) − X
Revised: If min τ ∈ T || τ ( X ′ ) − X

Original: || 2 < r , then h ( X ′ )
Revised: || 2 < r , then h ( X ′ )

Original: =
Revised: =

Original: y ∗ .
Revised: y ∗ .

Original: Proof.
Revised: Proof.

Original: If min τ ∈ T || τ ( X ′ ) − X
Revised: If min τ ∈ T || τ ( X ′ ) − X

Original: || 2 < r , then the minimizer τ ∗ = argmin τ ∈ T
Revised: || 2 < r , then the minimizer τ ∗ = argmin τ ∈ T

Original: || τ ( X ′ )
Revised: || τ ( X ′ )

Original: − X
Revised: − X

Original: || 2 fulfills 1034 τ ∗ ( X ′ )
Revised: || 2 fulfills 1034 τ ∗ ( X ′ )

Original: ∈ B and thus h ( τ ∗ ( X ′ ))
Revised: ∈ B and thus h ( τ ∗ ( X ′ ))

Original: =
Revised: =

Original: y ∗ .
Revised: y ∗ .

Original: Since τ ∗ ∈ T , we have h ( τ ∗ ( X ′ ))
Revised: Since τ ∗ ∈ T , we have h ( τ ∗ ( X ′ ))

Original: =
Revised: =

Original: h ( X ′ ) .
Revised: h ( X ′ ) .

Original: By thetransitive property, we have h ( X ′ )
Revised: By thetransitive property, we have h ( X ′ )

Original: =
Revised: =

Original: y ∗ .
Revised: y ∗ .

Original: C.3 Proof of Theorem
Revised: C.3 Proof of Theorem

Original: Theorem 3.
Revised: Theorem 3.

Original: Let base classifier g : R N × D
Revised: Let base classifier g : R N × D

Original: → Y be invariant under a group of transformations T ⊆{
Revised: → Y be invariant under a group of transformations T ⊆{

Original: X (cid:55)→ P η ( X ) | P ∈ S ( N ) , η ∈ E ( D ) } , where S ( N ) is the symmetric group in N dimensions, 1039 E ( D ) is the Euclidean group in D dimensions and η : R D
Revised: X (cid:55)→ P η ( X ) | P ∈ S ( N ) , η ∈ E ( D ) } , where S ( N ) is the symmetric group in N dimensions, 1039 E ( D ) is the Euclidean group in D dimensions and η : R D

Original: → R D is applied row-wise.
Revised: → R D is applied row-wise.

Original: Then theisotropically smoothed classifier f , as defined in Section 3, is also invariant under T .
Revised: Then theisotropically smoothed classifier f , as defined in Section 3, is also invariant under T .

Original: Recall that
Revised: Recall that

Original: [Equation]
Revised: [Equation]

Original: with p X ,y =
Revised: with p X ,y =

Original: Pr Z ∼ µ X [ g ( Z ) =
Revised: Pr Z ∼ µ X [ g ( Z ) =

Original: y ] and µ
Revised: y ] and µ

Original: [Equation]
Revised: [Equation]

Original: We shall prove Theorem 3 by showing that ∀ τ ∈ T : p X ,y =
Revised: We shall prove Theorem 3 by showing that ∀ τ ∈ T : p X ,y =

Original: p τ ( X ) ,y , i.e. the prediction probabilities 1043 are invariant.
Revised: p τ ( X ) ,y , i.e. the prediction probabilities 1043 are invariant.

Original: To do so, we need the following simple lemma:
Revised: To do so, we need the following simple lemma:

Original: Lemma 1. Consider an arbitrary invertible matrix A ∈ R N × N , vectors z , m ∈ R N and scalarσ > 0 .
Revised: Lemma 1. Consider an arbitrary invertible matrix A ∈ R N × N , vectors z , m ∈ R N and scalarσ > 0 .

Original: Then
Revised: Then

Original: [Equation]
Revised: [Equation]

Original: AA T (cid:1) .
Revised: AA T (cid:1) .

Original: Proof.
Revised: Proof.

Original: It follows from the change of variables formula for densitiy functions that the l.h.s.
Revised: It follows from the change of variables formula for densitiy functions that the l.h.s.

Original: term isthe density function of random variable z ′ =
Revised: term isthe density function of random variable z ′ =

Original: Az with z ∼ N (cid:0) m , σ 2 1 N (cid:1) .
Revised: Az with z ∼ N (cid:0) m , σ 2 1 N (cid:1) .

Original: Due to the behaviorof multivariate Normal distributions under affine transformation, the r.h.s.
Revised: Due to the behaviorof multivariate Normal distributions under affine transformation, the r.h.s.

Original: term is also the densityfunction of z ′ .
Revised: term is also the densityfunction of z ′ .

Original: In the following, let ˆ g ( Z ) =
Revised: In the following, let ˆ g ( Z ) =

Original: [ g ( Z ) =
Revised: [ g ( Z ) =

Original: y ∗ ] indicate whether base classifier g classifies input Z as y ∗ .
Revised: y ∗ ] indicate whether base classifier g classifies input Z as y ∗ .

Original: Now, consider an arbitrary X ∈ R N × D and transformation τ ∈ T .
Revised: Now, consider an arbitrary X ∈ R N × D and transformation τ ∈ T .

Original: Since
Revised: Since

Original: T ⊆ {
Revised: T ⊆ {

Original: X (cid:55)→ P η ( X ) | P ∈ S ( N ) , η ∈ E ( D ) } and E ( D ) is composed of translations, rotationsand reflections, we must have τ ( X ) =
Revised: X (cid:55)→ P η ( X ) | P ∈ S ( N ) , η ∈ E ( D ) } and E ( D ) is composed of translations, rotationsand reflections, we must have τ ( X ) =

Original: P (cid:0) XR T + 1 N c T (cid:1) with P ∈ S ( N ) , R T ∈
Revised: P (cid:0) XR T + 1 N c T (cid:1) with P ∈ S ( N ) , R T ∈

Original: O ( D ) andc ∈ R D , where O ( D ) ⊆
Revised: O ( D ) andc ∈ R D , where O ( D ) ⊆

Original: R D
Revised: R D

Original: × D is the orthogonal group, i.e. the set of all orthonormal matrices.
Revised: × D is the orthogonal group, i.e. the set of all orthonormal matrices.

Original: Note that P is also orthonormal and that | det( P ) | = | det( R ) | = 1 .
Revised: Note that P is also orthonormal and that | det( P ) | = | det( R ) | = 1 .

Original: We thus have
Revised: We thus have

Original: [Equation]
Revised: [Equation]

Original: where the second equality is due to a substitution V = τ − 1 ( Z ) and the third equality is due to
Revised: where the second equality is due to a substitution V = τ − 1 ( Z ) and the third equality is due to

Original: τ ∈
Revised: τ ∈

Original: Tand g being invariant under T .
Revised: Tand g being invariant under T .

Original: We may now insert the definition of our isotropic matrix normaldistribution µ and then use the fact that ∀ d : N ( z | m + d , Σ ) =
Revised: We may now insert the definition of our isotropic matrix normaldistribution µ and then use the fact that ∀ d : N ( z | m + d , Σ ) =

Original: N ( z − d | m , Σ ) to eliminate 1059 the translation
Revised: N ( z − d | m , Σ ) to eliminate 1059 the translation

Original: [Equation]
Revised: [Equation]

Original: Next, we can use the fact that P is orthonormal, i
Revised: Next, we can use the fact that P is orthonormal, i

Original: [Equation]
Revised: [Equation]

Original: e
Revised: e

Original: [Equation]
Revised: [Equation]

Original: P P T =
Revised: P P T =

Original: I N and | det( P − 1 ) | = 1 , to eliminatethe permutation via Lemma 1:
Revised: I N and | det( P − 1 ) | = 1 , to eliminatethe permutation via Lemma 1:

Original: [Equation]
Revised: [Equation]

Original: Finally, we can express our isotropic matrix normal density as the product of N D -dimensionalisotropic normal densities, rather than D N -dimensional isotropic normal densities and use the fact that R is orthonormal, i.e. RR T
Revised: Finally, we can express our isotropic matrix normal density as the product of N D -dimensionalisotropic normal densities, rather than D N -dimensional isotropic normal densities and use the fact that R is orthonormal, i.e. RR T

Original: =
Revised: =

Original: I N and | det( R − 1 ) | = 1 , to eliminate the rotation via Lemma 1:
Revised: I N and | det( R − 1 ) | = 1 , to eliminate the rotation via Lemma 1:

Original: [Equation]
Revised: [Equation]

Original: By the transitive property, we have p τ ( X ) ,y ∗ =
Revised: By the transitive property, we have p τ ( X ) ,y ∗ =

Original: p X ,y ∗ , which concludes our proof.
Revised: p X ,y ∗ , which concludes our proof.

Original: C.4 Application of Theorem
Revised: C.4 Application of Theorem

Original: Theorem 2.
Revised: Theorem 2.

Original: Let h ∈ R N × D
Revised: Let h ∈ R N × D

Original: → Y be invariant under a group of transformations T .
Revised: → Y be invariant under a group of transformations T .

Original: Let y ∗
Revised: Let y ∗

Original: =
Revised: =

Original: h ( X )be a prediction that is certifiably robust to a set of perturbed inputs B = { Z | ||
Revised: h ( X )be a prediction that is certifiably robust to a set of perturbed inputs B = { Z | ||

Original: Z − X
Revised: Z − X

Original: || 2 < r } , i.e.∀ Z ∈ B : h ( Z ) =
Revised: || 2 < r } , i.e.∀ Z ∈ B : h ( Z ) =

Original: y ∗ .
Revised: y ∗ .

Original: If min τ ∈ T || τ ( X ′ ) − X
Revised: If min τ ∈ T || τ ( X ′ ) − X

Original: || 2 < r , then h ( X ′ )
Revised: || 2 < r , then h ( X ′ )

Original: =
Revised: =

Original: y ∗ .
Revised: y ∗ .

Original: In the following, we discuss how to solve the optimization problem min τ ∈ T
Revised: In the following, we discuss how to solve the optimization problem min τ ∈ T

Original: || τ ( X ′ )
Revised: || τ ( X ′ )

Original: − X
Revised: − X

Original: || 2 < r for 1071 invariance under different groups T .
Revised: || 2 < r for 1071 invariance under different groups T .

Original: Before that, let us reiterate that using Theorem 2 and
Revised: Before that, let us reiterate that using Theorem 2 and

Original: solvingthese optimization problems is not necessary for certifying robustness, i.e. specifying a set of inputs˜ B
Revised: solvingthese optimization problems is not necessary for certifying robustness, i.e. specifying a set of inputs˜ B

Original: such that ∀ X ′ ∈ ˜ B : f ( X ′ )
Revised: such that ∀ X ′ ∈ ˜ B : f ( X ′ )

Original: =
Revised: =

Original: y ∗ (see Theorem 1).
Revised: y ∗ (see Theorem 1).

Original: It is only a way of performing membershipinference, i.e. determining whether X ′ ∈ ˜ B for a specific X ′ ∈ R N × D – just like computing the
Revised: It is only a way of performing membershipinference, i.e. determining whether X ′ ∈ ˜ B for a specific X ′ ∈ R N × D – just like computing the

Original: Frobenius norm of X ′ can be used to determine whether it is part of the original certified region B .
Revised: Frobenius norm of X ′ can be used to determine whether it is part of the original certified region B .

Original: C.4.1 Translation invariance
Revised: C.4.1 Translation invariance

Original: In this case, we have T = (cid:8) X
Revised: In this case, we have T = (cid:8) X

Original: (cid:55)→ X
Revised: (cid:55)→ X

Original: [Equation]
Revised: [Equation]

Original: N
Revised: N

Original: [Equation]
Revised: [Equation]

Original: , meaning
Revised: , meaning

Original: [Equation]
Revised: [Equation]

Original: where ∆ =
Revised: where ∆ =

Original: X ′ − X and ∆ ∈ R 1 × D are the column-wise averages.
Revised: X ′ − X and ∆ ∈ R 1 × D are the column-wise averages.

Original: The second equality can beshown by computing the gradients w.r.t.
Revised: The second equality can beshown by computing the gradients w.r.t.

Original: c and setting them to zero.
Revised: c and setting them to zero.

Original: C.4.2 Rotation invariance
Revised: C.4.2 Rotation invariance

Original: In this case, we have T = (cid:8) X
Revised: In this case, we have T = (cid:8) X

Original: (cid:55)→ XR T | R ∈
Revised: (cid:55)→ XR T | R ∈

Original: SO
Revised: SO

Original: ( D ) (cid:9) , meaning
Revised: ( D ) (cid:9) , meaning

Original: [Equation]
Revised: [Equation]

Original: This is a special case of the orthogonal Procrustes problem, which can be solved via singularvalue decomposition [105].
Revised: This is a special case of the orthogonal Procrustes problem, which can be solved via singularvalue decomposition [105].

Original: The optimal rotation matrix R ∗ is given by R ∗ =
Revised: The optimal rotation matrix R ∗ is given by R ∗ =

Original: V ˆ S U T , where
Revised: V ˆ S U T , where

Original: USV = ( X ′ )
Revised: USV = ( X ′ )

Original: T X and ˆ S = diag (cid:0) 1 , . .
Revised: T X and ˆ S = diag (cid:0) 1 , . .

Original: , 1 , sign (cid:0) det (cid:0) V U T (cid:1)(cid:1)(cid:1) .
Revised: , 1 , sign (cid:0) det (cid:0) V U T (cid:1)(cid:1)(cid:1) .

Original: C.4.3 Simultaneous rotation and reflection invariance
Revised: C.4.3 Simultaneous rotation and reflection invariance

Original: In this case, we have T = (cid:8) X
Revised: In this case, we have T = (cid:8) X

Original: (cid:55)→ XR T | R ∈
Revised: (cid:55)→ XR T | R ∈

Original: O
Revised: O

Original: ( D ) (cid:9) , where O ( D ) is the orthogonal group,meaning
Revised: ( D ) (cid:9) , where O ( D ) is the orthogonal group,meaning

Original: [Equation]
Revised: [Equation]

Original: This is the orthogonal Procrustes problem, which can again be solved via singular value decomposition [105].
Revised: This is the orthogonal Procrustes problem, which can again be solved via singular value decomposition [105].

Original: The optimal rotation matrix R ∗ is given by R ∗ =
Revised: The optimal rotation matrix R ∗ is given by R ∗ =

Original: V U T , where USV =
Revised: V U T , where USV =

Original: ( X ′ )
Revised: ( X ′ )

Original: T X .
Revised: T X .

Original: Here, accounting for the sign of the determinant is not necessary.
Revised: Here, accounting for the sign of the determinant is not necessary.

Original: C.4.4 Permutation invariance
Revised: C.4.4 Permutation invariance

Original: In this case, we have T =
Revised: In this case, we have T =

Original: { X (cid:55)→ P X | P ∈ S ( N ) } , meaning
Revised: { X (cid:55)→ P X | P ∈ S ( N ) } , meaning

Original: [Equation]
Revised: [Equation]

Original: The inner optimization problem is equivalent to finding an optimal matching in a bipartite graph with2 · N nodes, whose cost matrix is given by C n,m = ||
Revised: The inner optimization problem is equivalent to finding an optimal matching in a bipartite graph with2 · N nodes, whose cost matrix is given by C n,m = ||

Original: X ′ n − X m ||
Revised: X ′ n − X m ||

Original: This problem can be solved in 1095 polynomial time, for example via the Hungarian algorithm.
Revised: This problem can be solved in 1095 polynomial time, for example via the Hungarian algorithm.

Original: C.4.5 Simultaneous rotation and translation invariance
Revised: C.4.5 Simultaneous rotation and translation invariance

Original: In this case, we have T = (cid:8) X
Revised: In this case, we have T = (cid:8) X

Original: (cid:55)→
Revised: (cid:55)→

Original: [Equation]
Revised: [Equation]

Original: , meaning
Revised: , meaning

Original: [Equation]
Revised: [Equation]

Original: It can be shown that this is equivalent to centering X ′ and X
Revised: It can be shown that this is equivalent to centering X ′ and X

Original: and then solving the orthogonal
Revised: and then solving the orthogonal

Original: Procrustes problem [106], i.e.
Revised: Procrustes problem [106], i.e.

Original: [Equation]
Revised: [Equation]

Original: As discussed in Appendix C.4.2, this problem can be solved via singular value decomposition.
Revised: As discussed in Appendix C.4.2, this problem can be solved via singular value decomposition.

Original: C.4.6 Simultaneous permutation, rotation and translation invariance
Revised: C.4.6 Simultaneous permutation, rotation and translation invariance

Original: In this case, we have T = (cid:8) X
Revised: In this case, we have T = (cid:8) X

Original: (cid:55)→ P (cid:0) XR T + 1 N c T (cid:1) | P ∈ S
Revised: (cid:55)→ P (cid:0) XR T + 1 N c T (cid:1) | P ∈ S

Original: ( N ) ,
Revised: ( N ) ,

Original: R ∈
Revised: R ∈

Original: SO ( D ) , c ∈ R D (cid:9) , mean-
Revised: SO ( D ) , c ∈ R D (cid:9) , mean-

Original: [Figure]
Revised: [Figure]

Original: ing
Revised: ing

Original: [Figure]
Revised: [Figure]

Original: [Equation]
Revised: [Equation]

Original: Different from the previously discussed problems, the above is a challenging optimization problem known as point cloud registration, which does not have an efficiently computable solution.
Revised: Different from the previously discussed problems, the above is a challenging optimization problem known as point cloud registration, which does not have an efficiently computable solution.

Original: Approximate solutions to this problem are being actively studied (for a comprehensive survey,see [127]).
Revised: Approximate solutions to this problem are being actively studied (for a comprehensive survey,see [127]).

Original: Note that, if some upper bound ˆ m with min τ ∈ T
Revised: Note that, if some upper bound ˆ m with min τ ∈ T

Original: || τ ( X ′ )
Revised: || τ ( X ′ )

Original: − X
Revised: − X

Original: || 2 <
Revised: || 2 <

Original: ˆ m fulfills ˆ m < r , 1108 then min τ ∈ T
Revised: ˆ m fulfills ˆ m < r , 1108 then min τ ∈ T

Original: || τ ( X ′ )
Revised: || τ ( X ′ )

Original: − X
Revised: − X

Original: || 2 < r .
Revised: || 2 < r .

Original: Thus, any approximate solution to the point cloud registrationproblem can be used for membership inference.
Revised: Thus, any approximate solution to the point cloud registrationproblem can be used for membership inference.

Original: If ˆ m < r , we provably know that X ′ ∈ ˜ B and thush ( X ′ )
Revised: If ˆ m < r , we provably know that X ′ ∈ ˜ B and thush ( X ′ )

Original: =
Revised: =

Original: y ∗ .
Revised: y ∗ .

Original: However, there may be some X ′ ∈ ˜ B with ˆ m > r , which would be incorrectly declaredas potential adversarial examples.
Revised: However, there may be some X ′ ∈ ˜ B with ˆ m > r , which would be incorrectly declaredas potential adversarial examples.

Original: Thus, the membership inference via approximate solutions to thepoint cloud registration problem is sound, but not optimal.
Revised: Thus, the membership inference via approximate solutions to thepoint cloud registration problem is sound, but not optimal.

Original: D Tight gray-box certificates
Revised: D Tight gray-box certificates

Original: In the following, we first restate the Neyman-Pearson lemma from statistical testing, which weshall use to exactly solve the optimization problem underlying our tight gray-box certificates (Appendix D.1).
Revised: In the following, we first restate the Neyman-Pearson lemma from statistical testing, which weshall use to exactly solve the optimization problem underlying our tight gray-box certificates (Appendix D.1).

Original: We then summarize our certification methodology (Appendix D.2) and prove the maintheorem underlying this methodology (Appendix D.3).
Revised: We then summarize our certification methodology (Appendix D.2) and prove the maintheorem underlying this methodology (Appendix D.3).

Original: This methodology is then applied to translation invariance in arbitrary dimensions (Appendix D.4, rotation invariance in 2D (Appendix D.5and invariance to elemental rotations in 3D (Appendix D.6).
Revised: This methodology is then applied to translation invariance in arbitrary dimensions (Appendix D.4, rotation invariance in 2D (Appendix D.5and invariance to elemental rotations in 3D (Appendix D.6).

Original: The latter two are further combinedwith translation invariance (Appendices D.7 and D.8).
Revised: The latter two are further combinedwith translation invariance (Appendices D.7 and D.8).

Original: We conclude by discussing how to use Monte
Revised: We conclude by discussing how to use Monte

Original: Carlo sampling to obtain narrow probabilistic bounds on the certificates involving rotation invariance(Appendix D.9).
Revised: Carlo sampling to obtain narrow probabilistic bounds on the certificates involving rotation invariance(Appendix D.9).

Original: D.1 Neyman-Pearson lemmmapa
Revised: D.1 Neyman-Pearson lemmmapa

Original: For our purposes, the Neyman-Pearson lemma [104] can be formulated as follows:
Revised: For our purposes, the Neyman-Pearson lemma [104] can be formulated as follows:

Original: Lemma 2 (Neyman-Pearson lower bound) .
Revised: Lemma 2 (Neyman-Pearson lower bound) .

Original: Let µ X ′ , µ X , be two continuous distributions over ameasurable
Revised: Let µ X ′ , µ X , be two continuous distributions over ameasurable

Original: set A such that, for all t ∈ R + ,
Revised: set A such that, for all t ∈ R + ,

Original: the set (cid:110) z | µ X ′ ( z ) µ X ( z ) =
Revised: the set (cid:110) z | µ X ′ ( z ) µ X ( z ) =

Original: t (cid:111) has measure zero.
Revised: t (cid:111) has measure zero.

Original: Consider anarbitrary label set Y , a specific class label y ∈ Y and scalar p ∈ [0 , 1] .
Revised: Consider anarbitrary label set Y , a specific class label y ∈ Y and scalar p ∈ [0 , 1] .

Original: Then
Revised: Then

Original: [Equation]
Revised: [Equation]

Original: with h
Revised: with h

Original: [Equation]
Revised: [Equation]

Original: ( z ) =
Revised: ( z ) =

Original: [Equation]
Revised: [Equation]

Original: z
Revised: z

Original: [Equation]
Revised: [Equation]

Original: (cid:21) and t
Revised: (cid:21) and t

Original: [Equation]
Revised: [Equation]

Original: Here, indicator function h ∗ corresponds to a classifier that predicts class y if and only if the likelihoodratio is below a specific certain threshold.
Revised: Here, indicator function h ∗ corresponds to a classifier that predicts class y if and only if the likelihoodratio is below a specific certain threshold.

Original: For an application of this variant of the Neyman-Pearsonlemma to black-box robustness certification, as well as a discussion of its relation most powerfulhypothesis tests, see [27].
Revised: For an application of this variant of the Neyman-Pearsonlemma to black-box robustness certification, as well as a discussion of its relation most powerfulhypothesis tests, see [27].

Original: For various other formulations of the lemma, see [128].
Revised: For various other formulations of the lemma, see [128].

Original: One can use the same approach to obtain an upper bound on the probability of predicting a specificclass.
Revised: One can use the same approach to obtain an upper bound on the probability of predicting a specificclass.

Original: This will be relevant for our discussion of multi-class certificates in Appendix F.
Revised: This will be relevant for our discussion of multi-class certificates in Appendix F.

Original: Lemma 3 (Neyman-Pearson upper bound) .
Revised: Lemma 3 (Neyman-Pearson upper bound) .

Original: Let µ X ′ , µ X , be two continuous distributions over ameasurable
Revised: Let µ X ′ , µ X , be two continuous distributions over ameasurable

Original: set A such that, for all t ∈ R + ,
Revised: set A such that, for all t ∈ R + ,

Original: the set (cid:110) z | µ X ′ ( z ) µ X ( z ) =
Revised: the set (cid:110) z | µ X ′ ( z ) µ X ( z ) =

Original: t (cid:111) has measure zero.
Revised: t (cid:111) has measure zero.

Original: Consider anarbitrary label set Y , a specific class label y ∈ Y and scalar p ∈ [0 , 1] .
Revised: Consider anarbitrary label set Y , a specific class label y ∈ Y and scalar p ∈ [0 , 1] .

Original: Then
Revised: Then

Original: [Equation]
Revised: [Equation]

Original: with h ∗ ( z ) = 1 (cid:20) µ X
Revised: with h ∗ ( z ) = 1 (cid:20) µ X

Original: [Equation]
Revised: [Equation]

Original: ( z )
Revised: ( z )

Original: [Equation]
Revised: [Equation]

Original: and t
Revised: and t

Original: [Equation]
Revised: [Equation]

Original: D.2 Certification methodology
Revised: D.2 Certification methodology

Original: Recall from Section 6 that in order to obtain tight gray-box certificates, we need to find the worst-caseinvariant classifier.
Revised: Recall from Section 6 that in order to obtain tight gray-box certificates, we need to find the worst-caseinvariant classifier.

Original: That is, we need to solve min h ∈ H T Pr Z ∼ µ X ′
Revised: That is, we need to solve min h ∈ H T Pr Z ∼ µ X ′

Original: [ h ( Z ) =
Revised: [ h ( Z ) =

Original: y ∗ ] , where H is the setof all classifiers that are at least as likely as base classifier g to predict class p X ,y ∗ and have the sameinvariances.
Revised: y ∗ ] , where H is the setof all classifiers that are at least as likely as base classifier g to predict class p X ,y ∗ and have the sameinvariances.

Original: Further recall that the invariance constraint can be eliminated by applying a canonicalmap, which maps each element of R N × D to a distinct representative of its equivalence class.
Revised: Further recall that the invariance constraint can be eliminated by applying a canonicalmap, which maps each element of R N × D to a distinct representative of its equivalence class.

Original: Theorem 4.
Revised: Theorem 4.

Original: Let g : R N × D
Revised: Let g : R N × D

Original: → Y be invariant under group T and let H T be defined as in Eq.
Revised: → Y be invariant under group T and let H T be defined as in Eq.

Original: Ifγ : R N × D
Revised: Ifγ : R N × D

Original: →
Revised: →

Original: R N × D is a canonical map for invariance under T , then
Revised: R N × D is a canonical map for invariance under T , then

Original: [Equation]
Revised: [Equation]

Original: We shall prove this main theorem in Appendix D.3.
Revised: We shall prove this main theorem in Appendix D.3.

Original: After applying Theorem 4 with an appropriate canonical map γ , we first notice that the optimizationproblem only depends on 1 [ h ( Z ) =
Revised: After applying Theorem 4 with an appropriate canonical map γ , we first notice that the optimizationproblem only depends on 1 [ h ( Z ) =

Original: y ∗ ] , i.e. whether an input is classified as y ∗ or some other class.
Revised: y ∗ ] , i.e. whether an input is classified as y ∗ or some other class.

Original: To avoid notational clutter, we may equivalently optimize over binary classifiers h :
Revised: To avoid notational clutter, we may equivalently optimize over binary classifiers h :

Original: R N × D → { 0 , 1 } ,with h ( Z ) = 1 corresponding to the case that Z is classified as y ∗ :
Revised: R N × D → { 0 , 1 } ,with h ( Z ) = 1 corresponding to the case that Z is classified as y ∗ :

Original: [Equation]
Revised: [Equation]

Original: We can then transform the expectation integrals via a carefully chosen invertible substitutionξ : R N × D
Revised: We can then transform the expectation integrals via a carefully chosen invertible substitutionξ : R N × D

Original: → A that maps all elements from R N × D into some alternative, K -dimensional parameterspace A ⊆ R K .
Revised: → A that maps all elements from R N × D into some alternative, K -dimensional parameterspace A ⊆ R K .

Original: We choose this substitution such that applying the canonical map γ is equivalent tosetting L < K parameters of the alternative parameter space to zero.
Revised: We choose this substitution such that applying the canonical map γ is equivalent tosetting L < K parameters of the alternative parameter space to zero.

Original: That is
Revised: That is

Original: [Equation]
Revised: [Equation]

Original: Since the inverse ξ − 1 is matrix-valued, we define D ξ − 1 to be the Jacobian of vec (cid:0) ξ − 1 (cid:1) .
Revised: Since the inverse ξ − 1 is matrix-valued, we define D ξ − 1 to be the Jacobian of vec (cid:0) ξ − 1 (cid:1) .

Original: Because the term involving h does not depend on the first L dimensions of v , we may marginalizethem out.
Revised: Because the term involving h does not depend on the first L dimensions of v , we may marginalizethem out.

Original: Let A :
Revised: Let A :

Original: L = ( v : L | v ∈ A , ) be the first L dimensions of the parameter space and let 1157 A
Revised: L = ( v : L | v ∈ A , ) be the first L dimensions of the parameter space and let 1157 A

Original: [Equation]
Revised: [Equation]

Original: L +1): =
Revised: L +1): =

Original: [Equation]
Revised: [Equation]

Original: v ( L +1): | v ∈ A , (cid:1) be the remaining K − L dimensions.
Revised: v ( L +1): | v ∈ A , (cid:1) be the remaining K − L dimensions.

Original: Then, 

Revised: Then,

Original: [Equation]
Revised: [Equation]

Original: where ν X is the marginal distribution over A ( L +1): , the parameter space occupied by the representatives of the different equivalence classes.
Revised: where ν X is the marginal distribution over A ( L +1): , the parameter space occupied by the representatives of the different equivalence classes.

Original: Inserting into the problem from Eq.
Revised: Inserting into the problem from Eq.

Original: (9) yields
Revised: (9) yields

Original: [Equation]
Revised: [Equation]

Original: where we have written w ∈ R K − L instead of v ( L +1): to clarify that the optimization problem nolonger depends on the entire K -dimensional parameter space.
Revised: where we have written w ∈ R K − L instead of v ( L +1): to clarify that the optimization problem nolonger depends on the entire K -dimensional parameter space.

Original: At this point, it is convenient to treat the function
Revised: At this point, it is convenient to treat the function

Original: h
Revised: h

Original: ( Z ) as a family of variables ( h Z ) indexed by
Revised: ( Z ) as a family of variables ( h Z ) indexed by

Original: R N × D .
Revised: R N × D .

Original: Because ξ is a bijection and w (cid:55)→ (cid:2) 

Revised: Because ξ is a bijection and w (cid:55)→ (cid:2)

Original: TL w T (cid:3) T is an injective function, each distinct vectorw ∈
Revised: TL w T (cid:3) T is an injective function, each distinct vectorw ∈

Original: A ( L +1): indexes a distinct variable h Z .
Revised: A ( L +1): indexes a distinct variable h Z .

Original: We may thus equivalently optimize over a family ofvariables ( h ′ w ) ∈ Y indexed by A ( L +1): :
Revised: We may thus equivalently optimize over a family ofvariables ( h ′ w ) ∈ Y indexed by A ( L +1): :

Original: [Equation]
Revised: [Equation]

Original: Finally, we can exactly solve the lower-dimensional problem from Eq.
Revised: Finally, we can exactly solve the lower-dimensional problem from Eq.

Original: (10) via Lemma 2:(cid:18)
Revised: (10) via Lemma 2:(cid:18)

Original: [Equation]
Revised: [Equation]

Original: z
Revised: z

Original: [Equation]
Revised: [Equation]

Original: [ h
Revised: [ h

Original: [Equation]
Revised: [Equation]

Original: ( z )] with h ∗ ( z ) = 1 (cid:20) ν
Revised: ( z )] with h ∗ ( z ) = 1 (cid:20) ν

Original: X ′ ( z )
Revised: X ′ ( z )

Original: [Equation]
Revised: [Equation]

Original: and t ∈ R + s.t.
Revised: and t ∈ R + s.t.

Original: E z ∼
Revised: E z ∼

Original: [Equation]
Revised: [Equation]

Original: [ h
Revised: [ h

Original: [Equation]
Revised: [Equation]

Original: ( z )] =
Revised: ( z )] =

Original: p.
Revised: p.

Original: We discuss which canonical map and substitution to use, how to derive the marginal distributionsν X and ν X ′ and how to apply the Neyman-Pearson lemma in the respective sections of the differentinvariances (see Appendices D.4 to D.8).
Revised: We discuss which canonical map and substitution to use, how to derive the marginal distributionsν X and ν X ′ and how to apply the Neyman-Pearson lemma in the respective sections of the differentinvariances (see Appendices D.4 to D.8).

Original: D.3 Proof of Theorem 4
Revised: D.3 Proof of Theorem 4

Original: Before deriving the individual certificates, we need to prove our main theorem.
Revised: Before deriving the individual certificates, we need to prove our main theorem.

Original: Theorem 4.
Revised: Theorem 4.

Original: Let g : R N × D
Revised: Let g : R N × D

Original: → Y be invariant under group T and let H T be defined as in Eq.
Revised: → Y be invariant under group T and let H T be defined as in Eq.

Original: Ifγ : R N × D
Revised: Ifγ : R N × D

Original: →
Revised: →

Original: R N × D is a canonical map for invariance under T , thenmin h ∈ H T
Revised: R N × D is a canonical map for invariance under T , thenmin h ∈ H T

Original: [Equation]
Revised: [Equation]

Original: For the proof, recall
Revised: For the proof, recall

Original: Definition 2 (Equivalence classes) .
Revised: Definition 2 (Equivalence classes) .

Original: The equivalence class of an input X ∈ R N × D w.r.t.
Revised: The equivalence class of an input X ∈ R N × D w.r.t.

Original: invarianceunder a group of transformations T is [ x ] T = { τ ( x ) | τ ∈ T } .and
Revised: invarianceunder a group of transformations T is [ x ] T = { τ ( x ) | τ ∈ T } .and

Original: Definition 3 (Canonical map) .
Revised: Definition 3 (Canonical map) .

Original: A canonical map for invariance under a group of transformations Tis a function γ : R N × D
Revised: A canonical map for invariance under a group of transformations Tis a function γ : R N × D

Original: → R N × D with
Revised: → R N × D with

Original: [Equation]
Revised: [Equation]

Original: (4) and
Revised: (4) and

Original: [Equation]
Revised: [Equation]

Original: We begin our proof of Theorem 4 by deriving two lemmeta.
Revised: We begin our proof of Theorem 4 by deriving two lemmeta.

Original: The first lemma states that, if an input Z is not the representative of its own equivalence class, then itcannot be the representative of any equivalence class.
Revised: The first lemma states that, if an input Z is not the representative of its own equivalence class, then itcannot be the representative of any equivalence class.

Original: Lemma 4.
Revised: Lemma 4.

Original: Let γ be a canonical map for invariance under T .
Revised: Let γ be a canonical map for invariance under T .

Original: Then, for all Z ∈ R N × D
Revised: Then, for all Z ∈ R N × D

Original: [Equation]
Revised: [Equation]

Original: Proof.
Revised: Proof.

Original: Proof by contraposition.
Revised: Proof by contraposition.

Original: Assume there was some Z ′ with γ ( Z ′ )
Revised: Assume there was some Z ′ with γ ( Z ′ )

Original: =
Revised: =

Original: Z .
Revised: Z .

Original: Eq.
Revised: Eq.

Original: (3) from Definition 2 would imply that Z ∈ [ Z ′ ] T .
Revised: (3) from Definition 2 would imply that Z ∈ [ Z ′ ] T .

Original: Eq.
Revised: Eq.

Original: (4) from Definition 2 would then imply that γ ( Z ′ ) =
Revised: (4) from Definition 2 would then imply that γ ( Z ′ ) =

Original: γ ( Z ) .
Revised: γ ( Z ) .

Original: By the transitive property, we would have Z = γ ( Z ′ )
Revised: By the transitive property, we would have Z = γ ( Z ′ )

Original: =
Revised: =

Original: γ ( Z ) .
Revised: γ ( Z ) .

Original: The next lemma allows us to replace all equality constraints ∀ Z
Revised: The next lemma allows us to replace all equality constraints ∀ Z

Original: ′ ∈
Revised: ′ ∈

Original: [ Z ] T : h ( Z ) =
Revised: [ Z ] T : h ( Z ) =

Original: h ( Z ′ ) involvinginput Z and element of its equivalence class with a single equality constraint h ( Z )
Revised: h ( Z ′ ) involvinginput Z and element of its equivalence class with a single equality constraint h ( Z )

Original: =
Revised: =

Original: h ( γ ( Z )) .
Revised: h ( γ ( Z )) .

Original: Lemma 5.
Revised: Lemma 5.

Original: Let γ be a canonical map for invariance under T .
Revised: Let γ be a canonical map for invariance under T .

Original: Then
Revised: Then

Original: [Equation]
Revised: [Equation]

Original: Proof.⇒ Consider an arbitrary Z ∈ R N × D .
Revised: Proof.⇒ Consider an arbitrary Z ∈ R N × D .

Original: Due to Eq.
Revised: Due to Eq.

Original: (3) from Definition 2, we know that γ ( Z ) ∈
Revised: (3) from Definition 2, we know that γ ( Z ) ∈

Original: [ Z ] T .
Revised: [ Z ] T .

Original: Therefore, ∀ Z ′ ∈
Revised: Therefore, ∀ Z ′ ∈

Original: [ Z ] T : h ( Z ) =
Revised: [ Z ] T : h ( Z ) =

Original: h ( Z ′ ) implies h ( Z )
Revised: h ( Z ′ ) implies h ( Z )

Original: =
Revised: =

Original: h ( γ ( Z )) .
Revised: h ( γ ( Z )) .

Original: ⇐ Assume that ∀ Z : h ( Z )
Revised: ⇐ Assume that ∀ Z : h ( Z )

Original: =
Revised: =

Original: h ( γ ( Z )) .
Revised: h ( γ ( Z )) .

Original: Consider an arbitrary pair of arbitrary inputs Z ∈ 1195 R N × D ,
Revised: Consider an arbitrary pair of arbitrary inputs Z ∈ 1195 R N × D ,

Original: Z ′ ∈
Revised: Z ′ ∈

Original: [ Z ] T .
Revised: [ Z ] T .

Original: Due to our assumption, we know that h ( Z )
Revised: Due to our assumption, we know that h ( Z )

Original: =
Revised: =

Original: h ( γ ( Z )) .
Revised: h ( γ ( Z )) .

Original: Due to Eq.
Revised: Due to Eq.

Original: (4)from Definition 3, we know that γ ( Z ) =
Revised: (4)from Definition 3, we know that γ ( Z ) =

Original: γ ( Z ′ ) and thus h ( γ ( Z ))
Revised: γ ( Z ′ ) and thus h ( γ ( Z ))

Original: =
Revised: =

Original: h ( γ ( Z ′ )) .
Revised: h ( γ ( Z ′ )) .

Original: Again, due to our assumption, we know that h ( γ ( Z ′ ))
Revised: Again, due to our assumption, we know that h ( γ ( Z ′ ))

Original: =
Revised: =

Original: h ( Z ′ ) .
Revised: h ( Z ′ ) .

Original: By the transitive property, we have h ( Z )
Revised: By the transitive property, we have h ( Z )

Original: =
Revised: =

Original: h ( Z ′ ) .
Revised: h ( Z ′ ) .

Original: Lemma 5 allows us to restate H T as follows:
Revised: Lemma 5 allows us to restate H T as follows:

Original: [Equation]
Revised: [Equation]

Original: Thus
Revised: Thus

Original: [Equation]
Revised: [Equation]

Original: the optimization problem min h ∈ H T
Revised: the optimization problem min h ∈ H T

Original: Pr Z ∼ µ X ′
Revised: Pr Z ∼ µ X ′

Original: [ h ( Z ) =
Revised: [ h ( Z ) =

Original: y ∗ ] can be written as
Revised: y ∗ ] can be written as

Original: [Equation]
Revised: [Equation]

Original: Since we already know from the second constraint that any feasible solution must fulfill h ( Z )
Revised: Since we already know from the second constraint that any feasible solution must fulfill h ( Z )

Original: =
Revised: =

Original: h ( γ ( Z )) , we may as well substitute h ( γ ( Z )) for h ( Z ) before solving the problem:
Revised: h ( γ ( Z )) , we may as well substitute h ( γ ( Z )) for h ( Z ) before solving the problem:

Original: [Equation]
Revised: [Equation]

Original: For each Z ∈ R N × D , we can now distinguish two cases: If Z = γ ( Z ) , then the constrainth ( Z )
Revised: For each Z ∈ R N × D , we can now distinguish two cases: If Z = γ ( Z ) , then the constrainth ( Z )

Original: =
Revised: =

Original: h ( γ ( Z )) is trivially fulfilled and can thus be dropped.
Revised: h ( γ ( Z )) is trivially fulfilled and can thus be dropped.

Original: If Z ̸ = γ ( Z ) , then Lemma 4 showsthat h ( Z ) does not appear in the objective function or first constraint of Eq.
Revised: If Z ̸ = γ ( Z ) , then Lemma 4 showsthat h ( Z ) does not appear in the objective function or first constraint of Eq.

Original: (11), because there is noother Z ′ with γ ( Z ′ )
Revised: (11), because there is noother Z ′ with γ ( Z ′ )

Original: =
Revised: =

Original: Z .
Revised: Z .

Original: We can thus ignore the second constraint, solve the optimization problem
Revised: We can thus ignore the second constraint, solve the optimization problem

Original: [Equation]
Revised: [Equation]

Original: and then let h ( Z ) ← h ( γ ( Z )) to obtain an optimal, feasible solution to Eq.
Revised: and then let h ( Z ) ← h ( γ ( Z )) to obtain an optimal, feasible solution to Eq.

Original: [Equation]
Revised: [Equation]

Original: D.4 Tight certificate for invariance under translation
Revised: D.4 Tight certificate for invariance under translation

Original: Theorem 5.
Revised: Theorem 5.

Original: Let g : R N × D
Revised: Let g : R N × D

Original: → Y be translation invariant and H T be defined as in Eq.
Revised: → Y be translation invariant and H T be defined as in Eq.

Original: Then
Revised: Then

Original: [Equation]
Revised: [Equation]

Original: where ∆ ∈ R 1 × D are the column-wise average of ∆ =
Revised: where ∆ ∈ R 1 × D are the column-wise average of ∆ =

Original: X ′ − X
Revised: X ′ − X

Original: and σ is the standard deviation ofthe isotropic matrix normal smoothing distribution µ X .
Revised: and σ is the standard deviation ofthe isotropic matrix normal smoothing distribution µ X .

Original: Following our certification methodology, we derive this tight certificate for translation invariance,i.e. invariance under T = (cid:8) X
Revised: Following our certification methodology, we derive this tight certificate for translation invariance,i.e. invariance under T = (cid:8) X

Original: (cid:55)→ X + 1 N c T | c ∈ R D (cid:9) , by first specifying a canonical map (Appendix D.4.1) and then performing a suitable substitution to find the marginal distribution over asubset of an alternative parameter space occupied by the representatives of the different equivalenceclasses (Appendix D.4.2).
Revised: (cid:55)→ X + 1 N c T | c ∈ R D (cid:9) , by first specifying a canonical map (Appendix D.4.1) and then performing a suitable substitution to find the marginal distribution over asubset of an alternative parameter space occupied by the representatives of the different equivalenceclasses (Appendix D.4.2).

Original: D.4.1 Canonical map
Revised: D.4.1 Canonical map

Original: Lemma 6.
Revised: Lemma 6.

Original: The function γ ( Z ) =
Revised: The function γ ( Z ) =

Original: Z − 1 N Z T 1 , : is a canonical map for translation invariance.
Revised: Z − 1 N Z T 1 , : is a canonical map for translation invariance.

Original: Proof.
Revised: Proof.

Original: Based on the definition of canonical maps and translation invariance, we need to show that
Revised: Based on the definition of canonical maps and translation invariance, we need to show that

Original: [Equation]
Revised: [Equation]

Original: and
Revised: and

Original: [Equation]
Revised: [Equation]

Original: The function γ performs a translation with c =
Revised: The function γ performs a translation with c =

Original: Z 1 , : , so the first criterion is fulfilled.
Revised: Z 1 , : , so the first criterion is fulfilled.

Original: Now, consider an arbitrary pair of matrices
Revised: Now, consider an arbitrary pair of matrices

Original: Z , Z ′ ∈ R N × D with Z ′ ∈
Revised: Z , Z ′ ∈ R N × D with Z ′ ∈

Original: [ Z ] T , i.e. Z ′ =
Revised: [ Z ] T , i.e. Z ′ =

Original: Z + 1 N c
Revised: Z + 1 N c

Original: Tfor some c .
Revised: Tfor some c .

Original: We know that Z ′ 1 , : =
Revised: We know that Z ′ 1 , : =

Original: Z 1 , : + c and thus
Revised: Z 1 , : + c and thus

Original: [Equation]
Revised: [Equation]

Original: meaning the second criterion is fulfilled as well.
Revised: meaning the second criterion is fulfilled as well.

Original: D.4.2 Proof for certificate
Revised: D.4.2 Proof for certificate

Original: Our canonical map γ ( Z ) subtracts the first row of Z from all rows.
Revised: Our canonical map γ ( Z ) subtracts the first row of Z from all rows.

Original: We perform a substitution intothe alternative parameter space that specifies the first row Z 1 , as well Z n − Z 1 , i.e. the differencebetween all other rows to Z 1 .
Revised: We perform a substitution intothe alternative parameter space that specifies the first row Z 1 , as well Z n − Z 1 , i.e. the differencebetween all other rows to Z 1 .

Original: In this parameter space, applying the canonical map is equivalent tosetting the first D parameters to 0 .
Revised: In this parameter space, applying the canonical map is equivalent tosetting the first D parameters to 0 .