Original: λ ( 0 , ψ ) = γ ( λ ( ω ′ , ψ ))
Revised: λ ( 0 , ψ ) = γ ( λ ( ω ′ , ψ ))

Original: =
Revised: =

Original: γ ( Z ′ ) .
Revised: γ ( Z ′ ) .

Original: Substitution.
Revised: Substitution.

Original: Next, we use Proposition 1 and the canonical map from Eq.
Revised: Next, we use Proposition 1 and the canonical map from Eq.

Original: (17) to bring the probabilities from our optimization problem in Eq.
Revised: (17) to bring the probabilities from our optimization problem in Eq.

Original: (14) into a form that is compatible with the Neyman-Pearson lemma.
Revised: (14) into a form that is compatible with the Neyman-Pearson lemma.

Original: To declutter the terms, we first introduce ˆ h ( Z ) as a shorthand for 1 [ h ( Z ) =
Revised: To declutter the terms, we first introduce ˆ h ( Z ) as a shorthand for 1 [ h ( Z ) =

Original: y ∗ ] :
Revised: y ∗ ] :

Original: [Equation]
Revised: [Equation]

Original: We then perform the substitution Z
Revised: We then perform the substitution Z

Original: = λ ( ω , ψ ) with ω ∈ Ω ⊆ R K and ψ ∈ Ψ ⊆ R N · D − K :
Revised: = λ ( ω , ψ ) with ω ∈ Ω ⊆ R K and ψ ∈ Ψ ⊆ R N · D − K :

Original: [Equation]
Revised: [Equation]

Original: where J λ is the Jacobian of λ after vectorizing its domain and codomain.
Revised: where J λ is the Jacobian of λ after vectorizing its domain and codomain.

Original: Next, we apply the canonical map defined in Eq.
Revised: Next, we apply the canonical map defined in Eq.

Original: (17) to eliminate the group parameters ω from the term involving classifier ˆ h .
Revised: (17) to eliminate the group parameters ω from the term involving classifier ˆ h .

Original: This allows us to marginalize out ω :
Revised: This allows us to marginalize out ω :

Original: [Equation]
Revised: [Equation]

Original: where ν X ( ψ ) = (cid:82) Ω µ X ( λ ( ω , ψ )) · | det( J λ ( ω , ψ )) | d ω is the marginal density of parameters ψ .
Revised: where ν X ( ψ ) = (cid:82) Ω µ X ( λ ( ω , ψ )) · | det( J λ ( ω , ψ )) | d ω is the marginal density of parameters ψ .

Original: Finally, we can insert our result into Eq.
Revised: Finally, we can insert our result into Eq.

Original: (14) to obtain the simplified optimization problem
Revised: (14) to obtain the simplified optimization problem

Original: [Equation]
Revised: [Equation]

Original: Applying the Neyman-Pearson lemma.
Revised: Applying the Neyman-Pearson lemma.

Original: Eq.
Revised: Eq.

Original: (21) is almost in the form required by the NeymanPearson lemma.
Revised: (21) is almost in the form required by the NeymanPearson lemma.

Original: But, unlike in Lemma 5, we optimize over a function defined on R N × D while only evaluating it on a subset of R N × D , namely λ ( 0 , Ψ) .
Revised: But, unlike in Lemma 5, we optimize over a function defined on R N × D while only evaluating it on a subset of R N × D , namely λ ( 0 , Ψ) .

Original: To resolve this mismatch, it is convenient to treat h as a family of variables ( h Z ) indexed by R N × D .
Revised: To resolve this mismatch, it is convenient to treat h as a family of variables ( h Z ) indexed by R N × D .

Original: As specified in Proposition 1, λ is injective almost everywhere, meaning that each tuple ( 0 , ψ ) indexes a distinct variable h λ ( 0 , ψ ) , save for sets of measure zero, which do not influence the objective and constraint in Eq.
Revised: As specified in Proposition 1, λ is injective almost everywhere, meaning that each tuple ( 0 , ψ ) indexes a distinct variable h λ ( 0 , ψ ) , save for sets of measure zero, which do not influence the objective and constraint in Eq.

Original: Therefore, we may equivalently optimize over a family of variables (˜ h ψ ) indexed by Ψ :
Revised: Therefore, we may equivalently optimize over a family of variables (˜ h ψ ) indexed by Ψ :

Original: [Equation]
Revised: [Equation]

Original: According to Lemma 5, the minimizer is a function that classifies ψ as y ∗ iff ˜ h ∗ ( ψ ) = 1 with
Revised: According to Lemma 5, the minimizer is a function that classifies ψ as y ∗ iff ˜ h ∗ ( ψ ) = 1 with

Original: [Equation]
Revised: [Equation]

Original: with κ ∈ R + such that
Revised: with κ ∈ R + such that

Original: [Equation]
Revised: [Equation]

Original: Consequently, the optimum of our original problem Eq.
Revised: Consequently, the optimum of our original problem Eq.

Original: (21) is given by any function h ∗ ( Z ) with h ∗ ( λ ( 0 , ψ ))
Revised: (21) is given by any function h ∗ ( Z ) with h ∗ ( λ ( 0 , ψ ))

Original: =
Revised: =

Original: ˜ h ( ψ ) , i.e. we can use an arbitrary classifier for all parts of the domain that do not appear in the probability terms of Eq.
Revised: ˜ h ( ψ ) , i.e. we can use an arbitrary classifier for all parts of the domain that do not appear in the probability terms of Eq.

Original: We make the following proposition for our choice of worst-case classifier h ∗ ( Z ) :
Revised: We make the following proposition for our choice of worst-case classifier h ∗ ( Z ) :

Original: Proposition 2.
Revised: Proposition 2.

Original: Consider marginal density ν X ( ψ )
Revised: Consider marginal density ν X ( ψ )

Original: = (cid:82) Ω µ X ( λ ( ω , ψ )) · | det( J λ ( ω , ψ )) | d ω .
Revised: = (cid:82) Ω µ X ( λ ( ω , ψ )) · | det( J λ ( ω , ψ )) | d ω .

Original: Let η be a right Haar measure on group T ∈ { T ( D ) , SO ( D ) , SE ( D ) } .
Revised: Let η be a right Haar measure on group T ∈ { T ( D ) , SO ( D ) , SE ( D ) } .

Original: For SO ( D ) and SE ( D ) , let
Revised: For SO ( D ) and SE ( D ) , let

Original: D ∈ { 2 , 3 } .
Revised: D ∈ { 2 , 3 } .

Original: Define the group-averaged kernel
Revised: Define the group-averaged kernel

Original: [Equation]
Revised: [Equation]

Original: Then,
Revised: Then,

Original: [Equation]
Revised: [Equation]

Original: where ∝ absorbs factors that are constant in ψ .
Revised: where ∝ absorbs factors that are constant in ψ .

Original: Assuming Proposition 2 holds, we have by transitivity of the previous equalities
Revised: Assuming Proposition 2 holds, we have by transitivity of the previous equalities

Original: that
Revised: that

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: with
Revised: with

Original: [Equation]
Revised: [Equation]

Original: and κ
Revised: and κ

Original: [Equation]
Revised: [Equation]

Original: R
Revised: R

Original: [Equation]
Revised: [Equation]

Original: such that
Revised: such that

Original: [Equation]
Revised: [Equation]

Original: Resubstitution.
Revised: Resubstitution.

Original: Next, we need to transform the expectations w.r
Revised: Next, we need to transform the expectations w.r

Original: [Equation]
Revised: [Equation]

Original: t. marginal distribution ν X over Ψ ⊆ R N · D − K back into an expectation w.r
Revised: t. marginal distribution ν X over Ψ ⊆ R N · D − K back into an expectation w.r

Original: [Equation]
Revised: [Equation]

Original: t. our original smoothing distribution µ X over R N × D .
Revised: t. our original smoothing distribution µ X over R N × D .

Original: Applying the steps from Eq.
Revised: Applying the steps from Eq.

Original: (19) to Eq.
Revised: (19) to Eq.

Original: (20) in reverse order shows that
Revised: (20) in reverse order shows that

Original: [Equation]
Revised: [Equation]

Original: Exploiting invariance.
Revised: Exploiting invariance.

Original: Finally, we use the fact that our worst-case classifier h ∗ is invariant under group T to eliminate the canonical map γ from the expectation.
Revised: Finally, we use the fact that our worst-case classifier h ∗ is invariant under group T to eliminate the canonical map γ from the expectation.

Original: Recall from Eq.
Revised: Recall from Eq.

Original: (17) that, for any
Revised: (17) that, for any

Original: Z = λ ( ω , ψ ) , we defined γ ( Z ) =
Revised: Z = λ ( ω , ψ ) , we defined γ ( Z ) =

Original: ( τ ( ω ))
Revised: ( τ ( ω ))

Original: − 1 ◦ Z , i.e. the canonical map lets a group element act on
Revised: − 1 ◦ Z , i.e. the canonical map lets a group element act on

Original: Z .
Revised: Z .

Original: Since η is a right Haar measure, we have
Revised: Since η is a right Haar measure, we have

Original: [Equation]
Revised: [Equation]

Original: where the second equality follows from the fact that ◦ is a group action The third equality holds because η is a right Haar measure, meaning we can make the substitution u
Revised: where the second equality follows from the fact that ◦ is a group action The third equality holds because η is a right Haar measure, meaning we can make the substitution u

Original: = t · ( τ ( ω )) − 1 ) without having to change the measure.
Revised: = t · ( τ ( ω )) − 1 ) without having to change the measure.

Original: Thus, we have
Revised: Thus, we have

Original: [Equation]
Revised: [Equation]

Original: Combined with Eq.
Revised: Combined with Eq.

Original: (26) and Eqs.
Revised: (26) and Eqs.

Original: (24) and (25), this proves that the optimal value of the original variance-constrained problem is
Revised: (24) and (25), this proves that the optimal value of the original variance-constrained problem is

Original: [Equation]
Revised: [Equation]

Original: ) with
Revised: ) with

Original: [Equation]
Revised: [Equation]

Original: and κ ∈ R + such that
Revised: and κ ∈ R + such that

Original: [Equation]
Revised: [Equation]

Original: The last thing we need to do in order to conclude our proof is verify that Propositions 1 and 2 hold for each of the considered groups T .
Revised: The last thing we need to do in order to conclude our proof is verify that Propositions 1 and 2 hold for each of the considered groups T .

Original: F.3.1 Verifying Propositions 1 and 2 for translation invariance
Revised: F.3.1 Verifying Propositions 1 and 2 for translation invariance

Original: For T = T ( D ) , we define λ ( ω , ψ ) =
Revised: For T = T ( D ) , we define λ ( ω , ψ ) =

Original: τ ( ω ) ◦ ξ ( ψ ) with τ : R D → T ( D ) , ξ : R ( N − 1) D
Revised: τ ( ω ) ◦ ξ ( ψ ) with τ : R D → T ( D ) , ξ : R ( N − 1) D

Original: → R N × D and
Revised: → R N × D and

Original: [Equation]
Revised: [Equation]

Original: (cid:21) , where vec − 1 : R ( N − 1) D
Revised: (cid:21) , where vec − 1 : R ( N − 1) D

Original: → R ( N − 1) × D reshapes an input vector into a matrix.
Revised: → R ( N − 1) × D reshapes an input vector into a matrix.

Original: Due to the definition of group action ◦ from Appendix C.1, we have λ
Revised: Due to the definition of group action ◦ from Appendix C.1, we have λ

Original: [Equation]
Revised: [Equation]

Original: with all-ones vector 1 N ∈ R N .
Revised: with all-ones vector 1 N ∈ R N .

Original: In other words: We represent each element of R N × D as a matrix whose first row is zero, translated by a vector ω .
Revised: In other words: We represent each element of R N × D as a matrix whose first row is zero, translated by a vector ω .

Original: The function λ is evidently a differentiable bijection with inverse
Revised: The function λ is evidently a differentiable bijection with inverse

Original: [Equation]
Revised: [Equation]

Original: meaning it is surjective and injective.
Revised: meaning it is surjective and injective.

Original: Furthermore, we have τ ( 0 D ) =
Revised: Furthermore, we have τ ( 0 D ) =

Original: 0 D , with 0 D being the identity element of T ( D ) .
Revised: 0 D , with 0 D being the identity element of T ( D ) .

Original: Lastly, we have for all t ∈ T ( D ) , ω ∈ R D , ψ ∈ R ( N − 1) D that
Revised: Lastly, we have for all t ∈ T ( D ) , ω ∈ R D , ψ ∈ R ( N − 1) D that

Original: [Equation]
Revised: [Equation]

Original: Thus, all conditions from Proposition 1 are fulfilled.
Revised: Thus, all conditions from Proposition 1 are fulfilled.

Original: To verify Proposition 2, we need to show that
Revised: To verify Proposition 2, we need to show that

Original: [Equation]
Revised: [Equation]

Original: with marginal density ν X ( ψ ) =
Revised: with marginal density ν X ( ψ ) =

Original: (cid:82) R D µ X ( λ ( ω , ψ )) · | det( J λ ( ω , ψ )) | d ω and Haar integral β X ( Z ) = (cid:82) t ∈ R D exp (cid:0) ⟨ t ◦ Z , X ⟩ F / σ 2 (cid:1) dη ( t ) , where η is an arbitrary right Haar measure on translation group T ( D )
Revised: (cid:82) R D µ X ( λ ( ω , ψ )) · | det( J λ ( ω , ψ )) | d ω and Haar integral β X ( Z ) = (cid:82) t ∈ R D exp (cid:0) ⟨ t ◦ Z , X ⟩ F / σ 2 (cid:1) dη ( t ) , where η is an arbitrary right Haar measure on translation group T ( D )

Original: =
Revised: =

Original: R D .
Revised: R D .

Original: Firstly, we see that | det( J λ ( ω , ψ ))
Revised: Firstly, we see that | det( J λ ( ω , ψ ))

Original: | = 1 , since we are only performing translations.
Revised: | = 1 , since we are only performing translations.

Original: Thus,
Revised: Thus,

Original: [Equation]
Revised: [Equation]

Original: where ∝ absorbs factors that are constant in ω .
Revised: where ∝ absorbs factors that are constant in ω .

Original: In the above equalities we have first inserted the definition of our isotropic matrix normal smoothing distribution, then removed constant terms, then expressed the product of exponential functions more compactly using the Frobenius inner product and finally used that, by definition, λ ( ω , ψ ) =
Revised: In the above equalities we have first inserted the definition of our isotropic matrix normal smoothing distribution, then removed constant terms, then expressed the product of exponential functions more compactly using the Frobenius inner product and finally used that, by definition, λ ( ω , ψ ) =

Original: τ ( ω ) ◦ ξ ( ψ ) =
Revised: τ ( ω ) ◦ ξ ( ψ ) =

Original: ω ◦ λ ( 0 D , ψ ) .
Revised: ω ◦ λ ( 0 D , ψ ) .

Original: Finally, we note that the Lebesgue measure is translation-invariant, i.e. (cid:82) R D h ( ω + c )
Revised: Finally, we note that the Lebesgue measure is translation-invariant, i.e. (cid:82) R D h ( ω + c )

Original: d ω =
Revised: d ω =

Original: (cid:82) R D h ( ω ) d ω for arbitrary functions h , meaning it is a Haar measure of the translation group.
Revised: (cid:82) R D h ( ω ) d ω for arbitrary functions h , meaning it is a Haar measure of the translation group.

Original: Since the translation group is a Lie group and thus locally compact, the Haar measure is unique up to a multiplicative constant [109, 110].
Revised: Since the translation group is a Lie group and thus locally compact, the Haar measure is unique up to a multiplicative constant [109, 110].

Original: Thus, Proposition 2 holds.
Revised: Thus, Proposition 2 holds.

Original: F.3.2 Verifying Propositions 1 and 2 for rotation invariance in 2D
Revised: F.3.2 Verifying Propositions 1 and 2 for rotation invariance in 2D

Original: In this section, let
Revised: In this section, let

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: be the matrix that rotates counter-clockwise by angle ω .
Revised: be the matrix that rotates counter-clockwise by angle ω .

Original: Note that SO (2) =
Revised: Note that SO (2) =

Original: { R ( ω ) | ω ∈ [0 , 2 π ] } .
Revised: { R ( ω ) | ω ∈ [0 , 2 π ] } .

Original: To verify our propositions for T = SO (2) , we define λ ( ω , ψ ) =
Revised: To verify our propositions for T = SO (2) , we define λ ( ω , ψ ) =

Original: τ ( ω ) ◦ ξ ( ψ ) with τ : [0 , 2 π ] → SO (2) and ξ : R + × R 2( N − 1)
Revised: τ ( ω ) ◦ ξ ( ψ ) with τ : [0 , 2 π ] → SO (2) and ξ : R + × R 2( N − 1)

Original: → R N × 2 (note that the first argument is non-negative) with
Revised: → R N × 2 (note that the first argument is non-negative) with

Original: [Equation]
Revised: [Equation]

Original: where vec − 1 : R 2( N − 1)
Revised: where vec − 1 : R 2( N − 1)

Original: → R ( N − 1) × 2 reshapes an input vector into a matrix.
Revised: → R ( N − 1) × 2 reshapes an input vector into a matrix.

Original: Due to the definition of group action ◦ from Appendix C.2, we have
Revised: Due to the definition of group action ◦ from Appendix C.2, we have

Original: [Equation]
Revised: [Equation]

Original: In other words: We represent each element of R N × 2 as a matrix whose first row is aligned with the x -axis, rotated counter-clockwise by an angle ω .
Revised: In other words: We represent each element of R N × 2 as a matrix whose first row is aligned with the x -axis, rotated counter-clockwise by an angle ω .

Original: The function λ is evidently surjective.
Revised: The function λ is evidently surjective.

Original: Any Z ∈ R N × 2 can be represented via
Revised: Any Z ∈ R N × 2 can be represented via

Original: [Equation]
Revised: [Equation]

Original: with ω ∗ =
Revised: with ω ∗ =

Original: arctan2( Z 1 , 2 , Z 1 , 1 ) .
Revised: arctan2( Z 1 , 2 , Z 1 , 1 ) .

Original: It is also injective, save for the set { Z ∈ R N × 2 | Z 1 = 0 } , which has measure zero.
Revised: It is also injective, save for the set { Z ∈ R N × 2 | Z 1 = 0 } , which has measure zero.

Original: The first row Z 1 uniquely defines ψ 1 = ||
Revised: The first row Z 1 uniquely defines ψ 1 = ||

Original: Z 

Revised: Z

Original: || 2 and ω =
Revised: || 2 and ω =

Original: arctan2( Z 1 , 2 , Z 1 , 1 ) , because polar coordinates for non-zero vectors are unique.
Revised: arctan2( Z 1 , 2 , Z 1 , 1 ) , because polar coordinates for non-zero vectors are unique.

Original: The parameter vector z 2: must fulfill
Revised: The parameter vector z 2: must fulfill

Original: [Equation]
Revised: [Equation]

Original: which has a unique solution because rotation matrix R ( ω ) is invertible.
Revised: which has a unique solution because rotation matrix R ( ω ) is invertible.

Original: In addition to surjectivity and injectivity almost everywhere, λ is differentiable and we have τ (0)
Revised: In addition to surjectivity and injectivity almost everywhere, λ is differentiable and we have τ (0)

Original: =
Revised: =

Original: I N , with identity matrix
Revised: I N , with identity matrix

Original: I N being the identity element of SO (2) .
Revised: I N being the identity element of SO (2) .

Original: Furthermore, we have for all R ( α ) , ω , ψ that
Revised: Furthermore, we have for all R ( α ) , ω , ψ that

Original: [Equation]
Revised: [Equation]

Original: Thus, all conditions from Proposition 1 are fulfilled.
Revised: Thus, all conditions from Proposition 1 are fulfilled.

Original: To verify Proposition 2, we need to show that
Revised: To verify Proposition 2, we need to show that

Original: [Equation]
Revised: [Equation]

Original: with marginal density ν X ( ψ ) =
Revised: with marginal density ν X ( ψ ) =

Original: (cid:82) [0 , 2 π ] µ X ( λ ( ω , ψ )) · | det( J λ ( ω , ψ )) | d ω and Haar integral β X
Revised: (cid:82) [0 , 2 π ] µ X ( λ ( ω , ψ )) · | det( J λ ( ω , ψ )) | d ω and Haar integral β X

Original: ( Z ) =
Revised: ( Z ) =

Original: (cid:82) R ∈ SO (2) exp (cid:0) ⟨ t ◦ Z , X ⟩ F / σ 2 (cid:1) dη ( t ) .
Revised: (cid:82) R ∈ SO (2) exp (cid:0) ⟨ t ◦ Z , X ⟩ F / σ 2 (cid:1) dη ( t ) .

Original: We begin by calculating the Jacobian
Revised: We begin by calculating the Jacobian

Original: [Equation]
Revised: [Equation]

Original: a
Revised: a

Revised: [Equation]

Original: B
Revised: B

Original: [Equation]
Revised: [Equation]

Original: , with some vector a ∈ R 2( N − 1) and block-diagonal matrix B ∈ R 2( N − 1) × 2( N − 1) with
Revised: , with some vector a ∈ R 2( N − 1) and block-diagonal matrix B ∈ R 2( N − 1) × 2( N − 1) with

Original: B =  
Revised: B =  

Original: R ( ω ) . . .
Revised: R ( ω ) . . .

Original: R ( ω )   .
Revised: R ( ω )   .

Original: Due to the block structure
Revised: Due to the block structure

Original: [Equation]
Revised: [Equation]

Original: we have
Revised: we have

Original: [Equation]
Revised: [Equation]

Original: Thus
Revised: Thus

Original: [Equation]
Revised: [Equation]

Original: our marginal density is
Revised: our marginal density is

Original: [Equation]
Revised: [Equation]

Original: where ∝ absorbs factors that are constant in ω .
Revised: where ∝ absorbs factors that are constant in ω .

Original: Because the composition of two rotations corresponds to a translation of rotation angles, the translation-invariant Lebesgue measure is an invariant measure for group
Revised: Because the composition of two rotations corresponds to a translation of rotation angles, the translation-invariant Lebesgue measure is an invariant measure for group

Original: SO (2) (in this angle-based parameterization): (cid:82) 
Revised: SO (2) (in this angle-based parameterization): (cid:82) [

Original: [Equation]
Revised: [Equation]

Original: [Equation]

Original:  ] f
Revised: π ] h

Original: [Equation]
Revised: [Equation]

Original: R
Revised: R

Original: [Equation]
Revised: [Equation]

Original: [Equation] R[Equation]
Revised: ω ) · R

Original: [Equation]
Revised: [Equation]

Original: dω
Revised: ω

Original: [Equation]
Revised: [Equation]

Original: [Equation]

Original: ] f ( R (
Revised: )) dω =

Original: [Equation]
Revised: [Equation]

Original: [Equation] =
Revised: [0 , 2 π ] h ( R ( ω + ω ′ ))

Original: [Equation] [Equation] ] f ( R ( ω )) dω  .
Revised: dω = (cid:82) [0 , 2 π ] h ( R ( ω )) dω for arbitrary functions h .

Original: Since SO (2) is a Lie group and thus locally compact, the Haar measure is unique up to a multiplicative constant [109, 110].
Revised: Since SO (2) is a Lie group and thus locally compact, the Haar measure is unique up to a multiplicative constant [109, 110].

Original: Thus, Proposition 2 holds.
Revised: Thus, Proposition 2 holds.

Original: F.3.

Revised: F.3.

Original: Verifying Propositions 1 and 2 for rotation invariance in 3D
Revised: Verifying Propositions 1 and 2 for rotation invariance in 3D

Original: In this section, let
Revised: In this section, let

Original: [Equation]
Revised: [Equation]

Original: be the matrix that performs an intrinsic rotation around the z -axis by angle ω 1 , followed by a rotation around the new y -axis by angle ω 2 and then a rotation around the new x -axis by angle ω 3 .
Revised: be the matrix that performs an intrinsic rotation around the z -axis by angle ω 1 , followed by a rotation around the new y -axis by angle ω 2 and then a rotation around the new x -axis by angle ω 3 .

Original: Note that SO (3) =
Revised: Note that SO (3) =

Original: { R ( ω ) | ω ∈ Ω } with Ω = [0 , 2 π ] × [ − π 2 , π 2 ] × [0 , 2 π ] .
Revised: { R ( ω ) | ω ∈ Ω } with Ω = [0 , 2 π ] × [ − π 2 , π 2 ] × [0 , 2 π ] .

Original: To verify our propositions for T = SO (3) , we define λ ( ω , ψ ) =
Revised: To verify our propositions for T = SO (3) , we define λ ( ω , ψ ) =

Original: τ ( ω ) ◦ ξ ( ψ ) with τ : Ω → SO (3) and ξ : R + ×
Revised: τ ( ω ) ◦ ξ ( ψ ) with τ : Ω → SO (3) and ξ : R + ×

Original: [Equation]
Revised: [Equation]

Original: ×
Revised: ×

Original: [Equation]
Revised: [Equation]

Original: + × R 3( N − 2) → R N × 3 (note that the first and third argument are non-negative) with
Revised: + × R 3( N − 2) → R N × 3 (note that the first and third argument are non-negative) with

Original: [Equation]
Revised: [Equation]

Original: where vec − 1 : R 3( N − 1) → R ( N − 1) × 3 reshapes an input vector into a matrix.
Revised: where vec − 1 : R 3( N − 1) → R ( N − 1) × 3 reshapes an input vector into a matrix.

Original: Due to the definition of group action ◦ from Appendix C.2, we have
Revised: Due to the definition of group action ◦ from Appendix C.2, we have

Original: [Equation]
Revised: [Equation]

Original: In other words: We represent each element of R N × 3 as a matrix whose first row is aligned with the x -axis, and whose second row is in the first or second quadrant of the x - y -plane (because ψ 3 is non-negative) which is then intrinsically rotated by z - y - x angles ω 1 , ω 2 , ω 3 .
Revised: In other words: We represent each element of R N × 3 as a matrix whose first row is aligned with the x -axis, and whose second row is in the first or second quadrant of the x - y -plane (because ψ 3 is non-negative) which is then intrinsically rotated by z - y - x angles ω 1 , ω 2 , ω 3 .

Original: The function λ ( ω , ψ ) is injective and surjective, save for the set { Z ∈ R N × 3 | Z 1 = 0 ∨ ∃ c ∈ R :
Revised: The function λ ( ω , ψ ) is injective and surjective, save for the set { Z ∈ R N × 3 | Z 1 = 0 ∨ ∃ c ∈ R :

Original: Z 2 = c · Z 1 } , which has measure zero.
Revised: Z 2 = c · Z 1 } , which has measure zero.

Original: That is, for any Z outside this set, Z = λ ( ω , ψ ) has a unique solution.
Revised: That is, for any Z outside this set, Z = λ ( ω , ψ ) has a unique solution.

Original: Firstly, ( ψ 1 , ω 1 , ω 2 ) are spherical coordinates of Z 1 , which are unique for Z 1 ̸ = 0 .
Revised: Firstly, ( ψ 1 , ω 1 , ω 2 ) are spherical coordinates of Z 1 , which are unique for Z 1 ̸ = 0 .

Original: Secondly, the unique angles ω 1 and ω 2 constrain the x -axis after the intrinsic rotation to be co-linear with Z 1 .
Revised: Secondly, the unique angles ω 1 and ω 2 constrain the x -axis after the intrinsic rotation to be co-linear with Z 1 .

Original: Thus, and because we assume that Z 1 and Z 2 are not co-linear, there must be unique angle ω 3 for rotation around the x -axis that ensures that Z 2 is in the first or second quadrant of the new x - y -plane.
Revised: Thus, and because we assume that Z 1 and Z 2 are not co-linear, there must be unique angle ω 3 for rotation around the x -axis that ensures that Z 2 is in the first or second quadrant of the new x - y -plane.

Original: Finally, with the rotation angles ω and parameter ψ 1 fixed, the remaining parameter values are determined by
Revised: Finally, with the rotation angles ω and parameter ψ 1 fixed, the remaining parameter values are determined by

Original: [Equation]
Revised: [Equation]

Original: The function is also surjective on the entirety of R N × 3 .
Revised: The function is also surjective on the entirety of R N × 3 .

Original: If Z 1 = 0 , we have ψ 1 = 0 and ω 1 , ωcan be chosen arbitrarily.
Revised: If Z 1 = 0 , we have ψ 1 = 0 and ω 1 , ωcan be chosen arbitrarily.

Original: If ∃ c : Z 2 = c · Z 1 , then ω 3 can be chosen arbitrarily.
Revised: If ∃ c : Z 2 = c · Z 1 , then ω 3 can be chosen arbitrarily.

Original: The remaining parameters can be chosen using the procedure described above.
Revised: The remaining parameters can be chosen using the procedure described above.

Original: Finally, the function is differentiable, we have τ ( 0 ) =
Revised: Finally, the function is differentiable, we have τ ( 0 ) =

Original: I N and τ is a surjective function into SO (3) , meaning
Revised: I N and τ is a surjective function into SO (3) , meaning

Original: [Equation]
Revised: [Equation]

Original: ψ ) for some ω ′′ ∈ Ω .
Revised: ψ ) for some ω ′′ ∈ Ω .

Original: Thus, all criteria from Proposition 1 are fulfilled.
Revised: Thus, all criteria from Proposition 1 are fulfilled.

Original: Like in previous sections, we verify Proposition 2 by first calculating the Jacobian of λ .
Revised: Like in previous sections, we verify Proposition 2 by first calculating the Jacobian of λ .

Original: In the following, we use the shorthands s i = sin( ω i ) and c i = cos( ω i ) .
Revised: In the following, we use the shorthands s i = sin( ω i ) and c i = cos( ω i ) .

Original: We have
Revised: We have

Original: [Equation]
Revised: [Equation]

Original: with
Revised: with

Original: [Equation]
Revised: [Equation]

Original: Due to the block structure and because ω 2 ∈ [ − π
Revised: Due to the block structure and because ω 2 ∈ [ − π

Original: [Equation]
Revised: [Equation]

Original: π 2 ] and ψ 3 ∈ R +
Revised: π 2 ] and ψ 3 ∈ R +

Original: [Equation]
Revised: [Equation]

Original: we have
Revised: we have

Original: [Equation]
Revised: [Equation]

Original: Thus
Revised: Thus

Original: [Equation]
Revised: [Equation]

Original: our marginal density ν X ( ψ ) is
Revised: our marginal density ν X ( ψ ) is

Original: [Equation]
Revised: [Equation]

Original: This is, up to a multiplicative constant, the unique Haar integral for this angle-based parameterization of SO ( D ) (see, for instance [137, Chapter 1]).
Revised: This is, up to a multiplicative constant, the unique Haar integral for this angle-based parameterization of SO ( D ) (see, for instance [137, Chapter 1]).

Original: Thus, Proposition 2 holds.
Revised: Thus, Proposition 2 holds.

Original: F.3.4 Verifying Propositions 1 and 2 for roto-translation invariance in 2D and 3D
Revised: F.3.4 Verifying Propositions 1 and 2 for roto-translation invariance in 2D and 3D

Original: Finally, we prove that the propositions hold for T =
Revised: Finally, we prove that the propositions hold for T =

Original: SE
Revised: SE

Original: ( D ) with D ∈ { 2 , 3 } , which amounts to combining the results from the previous sections.
Revised: ( D ) with D ∈ { 2 , 3 } , which amounts to combining the results from the previous sections.

Original: In the following, let τ rot : Ω rot → SO ( D ) with τ rot ( ω ) =
Revised: In the following, let τ rot : Ω rot → SO ( D ) with τ rot ( ω ) =

Original: R ( ω ) be the the parameterization of SO ( D ) defined in Appendix F.3.2 or Appendix F.3.3.
Revised: R ( ω ) be the the parameterization of SO ( D ) defined in Appendix F.3.2 or Appendix F.3.3.

Original: Further let ξ rot : Ψ rot → R ( N − 1) × D be the same function as in Appendix F.3.2 or Appendix F.3.3, but for matrices with N − 1 instead of N rows.
Revised: Further let ξ rot : Ψ rot → R ( N − 1) × D be the same function as in Appendix F.3.2 or Appendix F.3.3, but for matrices with N − 1 instead of N rows.

Original: We begin by defining τ : Φ rot × R D : → SE ( D ) and ξ : Ψ rot → R N × D as follows:
Revised: We begin by defining τ : Φ rot × R D : → SE ( D ) and ξ : Ψ rot → R N × D as follows:

Original: [Equation]
Revised: [Equation]

Original: Due to the definition of group action ◦ from Appendix C.4, we have
Revised: Due to the definition of group action ◦ from Appendix C.4, we have

Original: [Equation]
Revised: [Equation]

Original: In other words: We represent each element of R N × D as a matrix whose first row is zero and whose second row is aligned with the x -axis which is then rotated and finally translated.
Revised: In other words: We represent each element of R N × D as a matrix whose first row is zero and whose second row is aligned with the x -axis which is then rotated and finally translated.

Original: For D = 

Revised: For D =

Original: we z x
Revised: we z x

Original: The proof for intrinsic z - y - x rotation is analogous.
Revised: The proof for intrinsic z - y - x rotation is analogous.

Original: additionally constrain the third row to be in the first or second quadrant of the x - y plane (for more details, see Appendix F.3.3).
Revised: additionally constrain the third row to be in the first or second quadrant of the x - y plane (for more details, see Appendix F.3.3).

Original: The function λ (( ω , b ) , ψ ) is surjective and injective, save for
Revised: The function λ (( ω , b ) , ψ ) is surjective and injective, save for

Original: [Equation]
Revised: [Equation]

Original: which have measure zero
Revised: which have measure zero

Original: That is, given an Z ∈ R N × D the solution to Z = λ (( ω , b ) , ψ ) is unique.
Revised: That is, given an Z ∈ R N × D the solution to Z = λ (( ω , b ) , ψ ) is unique.

Original: Evidently, we must have b =
Revised: Evidently, we must have b =

Original: Z 1 .
Revised: Z 1 .

Original: With this parameter fixed, the remaining parameters can be found by solving the equation Z 2: − 1 N − 1 Z T 1 = τ rot ( ω ) ◦ ξ rot ( ψ ) , whose unique solution we discussed in Appendix F.3.2 and Appendix F.3.3.
Revised: With this parameter fixed, the remaining parameters can be found by solving the equation Z 2: − 1 N − 1 Z T 1 = τ rot ( ω ) ◦ ξ rot ( ψ ) , whose unique solution we discussed in Appendix F.3.2 and Appendix F.3.3.

Original: The function is also surjective on the entirety of R N × D for the reasons presented in Appendix F.3.2 and Appendix F.3.3.
Revised: The function is also surjective on the entirety of R N × D for the reasons presented in Appendix F.3.2 and Appendix F.3.3.

Original: Further note that the function λ (( ω , b ) , ψ ) is differentiable and that τ ( 0 , 0 )
Revised: Further note that the function λ (( ω , b ) , ψ ) is differentiable and that τ ( 0 , 0 )

Original: =
Revised: =

Original: ( I D , 0 D ) , which is the identity element of SE ( D ) (see Appendix C.4).
Revised: ( I D , 0 D ) , which is the identity element of SE ( D ) (see Appendix C.4).

Original: Finally, we see that
Revised: Finally, we see that

Original: [Equation]
Revised: [Equation]

Original: for some ω ′′ ∈ Ω , b ′′ ∈ R D .
Revised: for some ω ′′ ∈ Ω , b ′′ ∈ R D .

Original: Thus, all criteria from Proposition 1 are fulfilled.
Revised: Thus, all criteria from Proposition 1 are fulfilled.

Original: Next, we verify Proposition 2 by again showing that the marginal density ν X ( ψ ) is a Haar integral.
Revised: Next, we verify Proposition 2 by again showing that the marginal density ν X ( ψ ) is a Haar integral.

Original: The function λ is a composition of two functions: The function λ rot from the previous sections and a translation.
Revised: The function λ is a composition of two functions: The function λ rot from the previous sections and a translation.

Original: Thus, the Jacobian determinant is the product of the two corresponding Jacobians.
Revised: Thus, the Jacobian determinant is the product of the two corresponding Jacobians.

Original: As discussed in Appendix F.3.1, the Jacobian determinant for translation is 1 .
Revised: As discussed in Appendix F.3.1, the Jacobian determinant for translation is 1 .

Original: Let J λ rot ( ω , ψ ) be the
Revised: Let J λ rot ( ω , ψ ) be the

Original: Jacobian of λ rot .
Revised: Jacobian of λ rot .

Original: Then
Revised: Then

Original: [Equation]
Revised: [Equation]

Original: where ∝ absorbs factors that are constant in ω and b .
Revised: where ∝ absorbs factors that are constant in ω and b .

Original: This is a Haar integral, because the inner and outer integral are Haar integrals for T ( D ) and SO ( D ) , respectively.
Revised: This is a Haar integral, because the inner and outer integral are Haar integrals for T ( D ) and SO ( D ) , respectively.

Original: Let us verify this by considering an arbitrary ( R ( ω ′ ) ,
Revised: Let us verify this by considering an arbitrary ( R ( ω ′ ) ,

Original: b ′ ) ∈ SE ( D ) .
Revised: b ′ ) ∈ SE ( D ) .

Original: To avoid clutter, define the shorthand f ( Z )
Revised: To avoid clutter, define the shorthand f ( Z )

Original: = exp (cid:0) ⟨ Z , X ⟩ F / σ 2 (cid:1) .
Revised: = exp (cid:0) ⟨ Z , X ⟩ F / σ 2 (cid:1) .

Original: We have
Revised: We have

Original: [Equation]
Revised: [Equation]

Original: Here, we have first applied the definition of the group action and group operator from Appendix C. and then used the fact that we are integrating over Haar measures for T ( D ) and then SO ( D ) .
Revised: Here, we have first applied the definition of the group action and group operator from Appendix C. and then used the fact that we are integrating over Haar measures for T ( D ) and then SO ( D ) .

Original: Since
Revised: Since

Original: SE ( D ) is a Lie group and thus locally compact, the Haar measure is unique up to a multiplicative constant [109, 110].
Revised: SE ( D ) is a Lie group and thus locally compact, the Haar measure is unique up to a multiplicative constant [109, 110].

Original: This confirms that Proposition 2 holds.
Revised: This confirms that Proposition 2 holds.

Original: F.4 Group-specific certificates
Revised: F.4 Group-specific certificates

Original: In this section, we prove the results for specific invariances from Sections 6.2 to 6.5, i.e. translation invariance (Appendix F.4.1), rotation invariance in 2D (Appendix F.4.2 and 3D (Appendix F.4.3, as well as roto-translation invariance in 2D and 3D (Appendix F.4.4).
Revised: In this section, we prove the results for specific invariances from Sections 6.2 to 6.5, i.e. translation invariance (Appendix F.4.1), rotation invariance in 2D (Appendix F.4.2 and 3D (Appendix F.4.3, as well as roto-translation invariance in 2D and 3D (Appendix F.4.4).

Original: F.4.1 Translation invariance
Revised: F.4.1 Translation invariance

Original: Theorem 4.
Revised: Theorem 4.

Original: Let g : R N × D
Revised: Let g : R N × D

Original: → Y be invariant under T
Revised: → Y be invariant under T

Original: = T ( D ) and H T be defined as in Eq.
Revised: = T ( D ) and H T be defined as in Eq.

Original: Then
Revised: Then

Original: [Equation]
Revised: [Equation]

Original: where ∆ ∈ R 1 × D are the column-wise averages of ∆ =
Revised: where ∆ ∈ R 1 × D are the column-wise averages of ∆ =

Original: X ′ − X
Revised: X ′ − X

Original: and σ is the standard deviation of the isotropic matrix normal smoothing distribution µ X .
Revised: and σ is the standard deviation of the isotropic matrix normal smoothing distribution µ X .

Original: During our proof of Theorem 3 in Appendices F.3 and F.3.1, we have shown that
Revised: During our proof of Theorem 3 in Appendices F.3 and F.3.1, we have shown that

Original: [Equation]
Revised: [Equation]

Original: with
Revised: with

Original: [Equation]
Revised: [Equation]

Original: and κ ∈ R + such that
Revised: and κ ∈ R + such that

Original: [Equation]
Revised: [Equation]

Original: with ψ ∈ R ( N − 1) · D and marginal distribution
Revised: with ψ ∈ R ( N − 1) · D and marginal distribution

Original: [Equation]
Revised: [Equation]

Original: where we have inserted the definition of our isotropic matrix normal smoothing distribution for the second equality.
Revised: where we have inserted the definition of our isotropic matrix normal smoothing distribution for the second equality.

Original: Using the matrix A =
Revised: Using the matrix A =

Original: [Equation]
Revised: [Equation]

Original: R N
Revised: R N

Original: [Equation]
Revised: [Equation]

Original: N with inverse A − 1 =
Revised: N with inverse A − 1 =

Original: [Equation]
Revised: [Equation]

Original: [Equation]
Revised: [Equation]

Original: I N −(cid:21) and all-zeros vector 0 N − 1 ∈ R N − 1 , all-ones vector 1 N − 1 ∈ R N − 1 and identity matrix I N − 1 , the marginal density can equivalently be written as
Revised: I N −(cid:21) and all-zeros vector 0 N − 1 ∈ R N − 1 , all-ones vector 1 N − 1 ∈ R N − 1 and identity matrix I N − 1 , the marginal density can equivalently be written as

Original: [Equation]
Revised: [Equation]

Original: where the second equality follows from the change of variable formula for densities (see also Lemma 2) and the third equality is due to det( A )
Revised: where the second equality follows from the change of variable formula for densities (see also Lemma 2) and the third equality is due to det( A )

Original: = 1 .
Revised: = 1 .

Original: Evidently, we are marginalizing out the first dimension of each of the D densities.
Revised: Evidently, we are marginalizing out the first dimension of each of the D densities.

Original: For normal distributions, this is equivalent to dropping the first row of the mean as well as the first row and column of the covariance matrix, i.e.
Revised: For normal distributions, this is equivalent to dropping the first row of the mean as well as the first row and column of the covariance matrix, i.e.

Original: [Equation]
Revised: [Equation]

Original: with
Revised: with

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original:  and B =
Revised:  and B =

Original: σ 2 A 2: ( A 2: ) T .
Revised: σ 2 A 2: ( A 2: ) T .

Original: Inserting back into Eqs.
Revised: Inserting back into Eqs.

Original: (29) and (30), we see that the optimal value of our variance-constrained optimization problem min h ∈ H T
Revised: (29) and (30), we see that the optimal value of our variance-constrained optimization problem min h ∈ H T

Original: Pr Z ∼ µ X ′
Revised: Pr Z ∼ µ X ′

Original: [ h ∗ ( Z ) =
Revised: [ h ∗ ( Z ) =

Original: y ∗ ] equals the optimal value of the black-box certification problem for anisotropic normal smoothing distribution N ( m X , Σ ) .
Revised: y ∗ ] equals the optimal value of the black-box certification problem for anisotropic normal smoothing distribution N ( m X , Σ ) .

Original: This optimal value has been derived in prior work (see [138] and Appendix A of [94]) and is
Revised: This optimal value has been derived in prior work (see [138] and Appendix A of [94]) and is

Original: [Equation]
Revised: [Equation]

Original: We can conclude our proof by showing that
Revised: We can conclude our proof by showing that

Original: [Equation]
Revised: [Equation]

Original: To this end, we need the following result:
Revised: To this end, we need the following result:

Original: Lemma 7. Consider an arbitrary vector x ∈ R D and let x = 1 D (cid:80) Dd
Revised: Lemma 7. Consider an arbitrary vector x ∈ R D and let x = 1 D (cid:80) Dd

Original: =1 x d be its average.
Revised: =1 x d be its average.

Original: Then
Revised: Then

Original: [Equation]
Revised: [Equation]

Original: Proof.
Revised: Proof.

Original: We substract the left-hand side from the right-hand side
Revised: We substract the left-hand side from the right-hand side

Original: [Equation]
Revised: [Equation]

Original: where the last equality follows from the fact that (cid:80) Dd =1 x d = D x .
Revised: where the last equality follows from the fact that (cid:80) Dd =1 x d = D x .

Original: Now, we can proceed by using the fact that the inverse of a block-diagonal matrix is also a blockdiagonal matrix and inserting the definitions of m X ′ , m X , Σ and B to show that:
Revised: Now, we can proceed by using the fact that the inverse of a block-diagonal matrix is also a blockdiagonal matrix and inserting the definitions of m X ′ , m X , Σ and B to show that:

Original: [Equation]
Revised: [Equation]

Original: where for the second to last last equality we have used that X ′ =
Revised: where for the second to last last equality we have used that X ′ =

Original: X + ∆ .
Revised: X + ∆ .

Original: Our next step is to compute ( A 2: ) T (cid:16)
Revised: Our next step is to compute ( A 2: ) T (cid:16)

Original: A 2: ( A 2: ) T (cid:17) − 1 A 2: .
Revised: A 2: ( A 2: ) T (cid:17) − 1 A 2: .

Original: Recall that A 2: ∈ R
Revised: Recall that A 2: ∈ R

Original: ( N − 1) × D and A 2: =
Revised: ( N − 1) × D and A 2: =

Original: [ − 1 N −
Revised: [ − 1 N −

Original: I N − 1 ] .
Revised: I N − 1 ] .

Original: Matrix multiplication shows that
Revised: Matrix multiplication shows that

Original: [Equation]
Revised: [Equation]

Original: The matrix is sufficiently simple to be inverted by inspection:
Revised: The matrix is sufficiently simple to be inverted by inspection:

Original: [Equation]
Revised: [Equation]

Original: Finally, matrix multiplication shows that
Revised: Finally, matrix multiplication shows that

Original: [Equation]
Revised: [Equation]

Original: Note that the matrix in Eq.
Revised: Note that the matrix in Eq.

Original: (38) transforms a vector by subtracting its average from all entries.
Revised: (38) transforms a vector by subtracting its average from all entries.

Original: Inserting into Eq.
Revised: Inserting into Eq.

Original: (37) shows that
Revised: (37) shows that

Original: [Equation]
Revised: [Equation]

Original: where ∆ : ,d ∈ R is the average of column ∆ : ,d , ∆ ∈ R 1 × D are the column-wise averages of matrix ∆ , the second equality is due to Lemma 7 and the third equality uses the definition of inner products.
Revised: where ∆ : ,d ∈ R is the average of column ∆ : ,d , ∆ ∈ R 1 × D are the column-wise averages of matrix ∆ , the second equality is due to Lemma 7 and the third equality uses the definition of inner products.

Original: Taking the square root and using the definition of the Frobenius norm yields our desired result:
Revised: Taking the square root and using the definition of the Frobenius norm yields our desired result:

Original: [Equation]
Revised: [Equation]

Original: F.4.2 Rotation invariance in 2D
Revised: F.4.2 Rotation invariance in 2D

Original: Theorem 6.
Revised: Theorem 6.

Original: Let g : R N × 2 → Y be invariant under T =
Revised: Let g : R N × 2 → Y be invariant under T =

Original: SO (2) and H T be defined as in Eq.
Revised: SO (2) and H T be defined as in Eq.

Original: Define the indicator function ρ : R 4 → { 0 , 1 } with
Revised: Define the indicator function ρ : R 4 → { 0 , 1 } with

Original: [Equation]
Revised: [Equation]

Original: with κ ∈ R such that
Revised: with κ ∈ R such that

Original: [Equation]
Revised: [Equation]

Original: Then
Revised: Then

Original: [Equation]
Revised: [Equation]

Original: where
Revised: where

Original: [Equation]
Revised: [Equation]

Original: with clean data norm ||
Revised: with clean data norm ||

Original: X || 2 , perturbation norm
Revised: X || 2 , perturbation norm

Original: || ∆ || 2 and parameters ϵ 1 = ⟨ X , ∆ ⟩ F , ϵ
Revised: || ∆ || 2 and parameters ϵ 1 = ⟨ X , ∆ ⟩ F , ϵ

Original: [Equation]
Revised: [Equation]

Original: ⟨ XR ( − π / 2 ) T , ∆ ⟩ F .
Revised: ⟨ XR ( − π / 2 ) T , ∆ ⟩ F .

Original: We know from Theorem 3 and our derivations in Appendix F.3.2 that
Revised: We know from Theorem 3 and our derivations in Appendix F.3.2 that

Original: [Equation]
Revised: [Equation]

Original: with
Revised: with

Original: [Equation]
Revised: [Equation]

Original: and κ ∈ R + such that
Revised: and κ ∈ R + such that

Original: [Equation]
Revised: [Equation]

Original: To prove Theorem 6, we shall first calculate the integral β X ( Z ) characterizing our worst-case classifier and then use the fact that it applies an affine transformation to our random input data sampled from µ X and µ X ′ .
Revised: To prove Theorem 6, we shall first calculate the integral β X ( Z ) characterizing our worst-case classifier and then use the fact that it applies an affine transformation to our random input data sampled from µ X and µ X ′ .

Original: We begin by factoring out cos( ω ) and sin( ω ) from the exponent:
Revised: We begin by factoring out cos( ω ) and sin( ω ) from the exponent:

Original: [Equation]
Revised: [Equation]

Original: We can then show that
Revised: We can then show that

Original: [Equation]
Revised: [Equation]

Original: where the second equality follows from the fact that cos( ω ) η 1 +sin( ω ) η 2 = cos( ω + α ) (cid:112) η 21 + η 22 for some α ∈ [0 , 2 π ] (see also Eq.
Revised: where the second equality follows from the fact that cos( ω ) η 1 +sin( ω ) η 2 = cos( ω + α ) (cid:112) η 21 + η 22 for some α ∈ [0 , 2 π ] (see also Eq.

Original: 29 of [139]) and the third equality is due to the integral representation of the modified Bessel function of the first kind with order 

Revised: 29 of [139]) and the third equality is due to the integral representation of the modified Bessel function of the first kind with order

Original: (see Eq.
Revised: (see Eq.

Original: 10.32.1 of [140]):
Revised: 10.32.1 of [140]):

Original: [Equation]
Revised: [Equation]

Original: We can now insert this expression for the worst-case classifier into the expectations w
Revised: We can now insert this expression for the worst-case classifier into the expectations w

Original: [Equation]
Revised: [Equation]

Original: r
Revised: r

Original: [Equation]
Revised: [Equation]

Original: t. clean and perturbed smoothing distributions µ X , µ ′ X from Eqs.
Revised: t. clean and perturbed smoothing distributions µ X , µ ′ X from Eqs.

Original: (39) and (40) .
Revised: (39) and (40) .

Original: Note that, by definition, we have vec( Z ) ∼ N (vec( X ) , σ 2 · I 2 N ) for Z ∼ µ X .
Revised: Note that, by definition, we have vec( Z ) ∼ N (vec( X ) , σ 2 · I 2 N ) for Z ∼ µ X .

Original: Thus:
Revised: Thus:

Original: [Equation]
Revised: [Equation]

Original: with means m (1) =
Revised: with means m (1) =

Original: W vec( X ′ ) , m (2) =
Revised: W vec( X ′ ) , m (2) =

Original: W vec( X ) and covariance matrix Σ =
Revised: W vec( X ) and covariance matrix Σ =

Original: σ 2 W W T , where
Revised: σ 2 W W T , where

Original: [Equation]
Revised: [Equation]

Original: Calculating the matrix-vector and matrix-matrix products yields the values from Theorem 6, thus proving that the certificate is valid.
Revised: Calculating the matrix-vector and matrix-matrix products yields the values from Theorem 6, thus proving that the certificate is valid.

Original: F.4.3 Rotation invariance in 3D
Revised: F.4.3 Rotation invariance in 3D

Original: As discussed in Section 6.4, we can apply Theorem 3 to T =
Revised: As discussed in Section 6.4, we can apply Theorem 3 to T =

Original: SO (3) , but do not have a closed-form expression for the Haar integral β X ( Z ) characterizing our worst-case classifier.
Revised: SO (3) , but do not have a closed-form expression for the Haar integral β X ( Z ) characterizing our worst-case classifier.

Original: We can however evaluate it using numerical integration.
Revised: We can however evaluate it using numerical integration.

Original: In the following, we first show how we can reduce the number of integration variables to facilitate numerical integration.
Revised: In the following, we first show how we can reduce the number of integration variables to facilitate numerical integration.

Original: We then show that, similar to the 2D case, the Monte Carlo certification procedure presented in Section 6.3 only requires sampling from a 16 -dimensional normal distribution, which allows us to obtain tight bounds via a large number of samples at little computational cost.
Revised: We then show that, similar to the 2D case, the Monte Carlo certification procedure presented in Section 6.3 only requires sampling from a 16 -dimensional normal distribution, which allows us to obtain tight bounds via a large number of samples at little computational cost.

Original: Reducing the number of integration variables.
Revised: Reducing the number of integration variables.

Original: Recall from our proof of Theorem 3 in Appendices F.3 and F.3.3 that
Revised: Recall from our proof of Theorem 3 in Appendices F.3 and F.3.3 that

Original: [Equation]
Revised: [Equation]

Original: [Equation]
Revised: [Equation]

Original: with Ω = [0 , 2 π ] × [ − π 2 , π 2 ] × [0 , 2 π ] and rotation matrix
Revised: with Ω = [0 , 2 π ] × [ − π 2 , π 2 ] × [0 , 2 π ] and rotation matrix

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: ) =
Revised: ) =

Original: [Equation]
Revised: [Equation]

Original: with ˜ R ( ω 2: ) being the matrix that rotates around the y - and z -axis by angles ω 2 and ω 3 , respectively.
Revised: with ˜ R ( ω 2: ) being the matrix that rotates around the y - and z -axis by angles ω 2 and ω 3 , respectively.

Original: While we cannot evaluate this term analytically, we can calculate the inner integral over ω 1 in order to reduce it to a double integral.
Revised: While we cannot evaluate this term analytically, we can calculate the inner integral over ω 1 in order to reduce it to a double integral.

Original: We begin by factoring out cos( ω 1 ) and sin( ω 1 ) from the exponent:
Revised: We begin by factoring out cos( ω 1 ) and sin( ω 1 ) from the exponent:

Original: [Equation]
Revised: [Equation]

Original: where χ 1 , χ 2 , χ 3 are shorthands we introduce to avoid clutter in the following derivations
Revised: where χ 1 , χ 2 , χ 3 are shorthands we introduce to avoid clutter in the following derivations

Original: [Equation]
Revised: [Equation]

Original: Due to the entries of ˜ R ( ω 2 :)
Revised: Due to the entries of ˜ R ( ω 2 :)

Original: [Equation]
Revised: [Equation]

Original: the shorthand χ 1 : R 2 × R 3 × 3 → R is defined as
Revised: the shorthand χ 1 : R 2 × R 3 × 3 → R is defined as

Original: [Equation]
Revised: [Equation]

Original: Similarly
Revised: Similarly

Original: [Equation]
Revised: [Equation]

Original: the shorthand χ 2 : R 2 × R 3 × 3 → R is defined as
Revised: the shorthand χ 2 : R 2 × R 3 × 3 → R is defined as

Original: [Equation]
Revised: [Equation]

Original: The shorthand χ 3 : R 2 × R 3 × 3 → R is defined as
Revised: The shorthand χ 3 : R 2 × R 3 × 3 → R is defined as

Original: [Equation]
Revised: [Equation]

Original: Just like in Appendix F.4.
Revised: Just like in Appendix F.4.

Original: [Equation]
Revised: [Equation]

Original: we can use the integral formula for the modified Bessel function of the first kind and order zero to eliminate the integral w.r.t.
Revised: we can use the integral formula for the modified Bessel function of the first kind and order zero to eliminate the integral w.r.t.

Original: ω 1 : β X ( Z ) =
Revised: ω 1 : β X ( Z ) =

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: We introduce ˆ β
Revised: We introduce ˆ β

Original: ( X T Z ) to avoid clutter in the next equations and to highlight that this function characterizing our worst-case classifier only depends on X T Z ∈ R 3 × 3 .
Revised: ( X T Z ) to avoid clutter in the next equations and to highlight that this function characterizing our worst-case classifier only depends on X T Z ∈ R 3 × 3 .

Original: Efficient Monte Carlo certification.
Revised: Efficient Monte Carlo certification.

Original: Just like in Appendix F.4.2, we can now use the fact that our worst-case classifier only depends on a small number of variables that are the result of linearly transforming our randomized inputs Z ∼ µ X , namely the 9 entires of X T Z .
Revised: Just like in Appendix F.4.2, we can now use the fact that our worst-case classifier only depends on a small number of variables that are the result of linearly transforming our randomized inputs Z ∼ µ X , namely the 9 entires of X T Z .

Original: Let ζ : R 8 → R 3 ×with
Revised: Let ζ : R 8 → R 3 ×with

Original: [Equation]
Revised: [Equation]

Original: be a function that zero-pads its 8 -dimensional input vector before devectorizing into shape 3 × 3 .
Revised: be a function that zero-pads its 8 -dimensional input vector before devectorizing into shape 3 × 3 .

Original: Since vec( Z ) ∼ N (cid:0) vec( X ) , σ 2 I 3 N (cid:1) if Z ∼ µ X , we have by definition of ˆ β that
Revised: Since vec( Z ) ∼ N (cid:0) vec( X ) , σ 2 I 3 N (cid:1) if Z ∼ µ X , we have by definition of ˆ β that

Original: [Equation]
Revised: [Equation]

Original: with m X = W vec( X ) , Σ
Revised: with m X = W vec( X ) , Σ

Original: = σ 2 W W T and
Revised: = σ 2 W W T and

Original: W =  
Revised: W =  

Original: X ′ :
Revised: X ′ :

Original: [Equation]
Revised: [Equation]

Original: X
Revised: X

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: X :
Revised: X :

Original: [Equation]
Revised: [Equation]

Original: X
Revised: X

Original: [Equation] X
Revised: [Equation] X

Original: [Equation]
Revised: [Equation]

Original: X X
Revised: X X

Original: [Equation]
Revised: [Equation]

Original: X  
Revised: X  

Original: T .
Revised: T .

Original: Thus, the expectation that provides the optimal value of our variance-constrained optimization problem can be probabilistically bounded via Monte Carlo sampling from an 8 -dimensional normal distribution, regardless of the data dimensionality.
Revised: Thus, the expectation that provides the optimal value of our variance-constrained optimization problem can be probabilistically bounded via Monte Carlo sampling from an 8 -dimensional normal distribution, regardless of the data dimensionality.

Original: F.4.4 Roto-translation invariance in 2D and 3D
Revised: F.4.4 Roto-translation invariance in 2D and 3D

Original: For the roto-granslation group SE ( D ) , we prove that evaluating the tight certificate is equivalent to evaluating the tight certificate for rotation group
Revised: For the roto-granslation group SE ( D ) , we prove that evaluating the tight certificate is equivalent to evaluating the tight certificate for rotation group

Original: SO ( D ) after centering the clean and perturbed data
Revised: SO ( D ) after centering the clean and perturbed data

Original: X , X ′ by substracting their averages.
Revised: X , X ′ by substracting their averages.

Original: Theorem 7.
Revised: Theorem 7.

Original: Let ˜ X
Revised: Let ˜ X

Original: = X − 1 N X with column-wise averages X , X ′ ∈ R 1 × D .
Revised: = X − 1 N X with column-wise averages X , X ′ ∈ R 1 × D .

Original: Further let η be a right Haar measure on SO (3) and µ X our isotropic matrix normal smoothing distribution.
Revised: Further let η be a right Haar measure on SO (3) and µ X our isotropic matrix normal smoothing distribution.

Original: Let β SE
Revised: Let β SE

Original: ( D ) X
Revised: ( D ) X

Original: =
Revised: =

Original: [Equation]
Revised: [Equation]

Original: be the Haar integral characterizing the worst-case classifier for roto-translation invariance (see also Appendix F.3.4) and
Revised: be the Haar integral characterizing the worst-case classifier for roto-translation invariance (see also Appendix F.3.4) and

Original: [Equation]
Revised: [Equation]

Original: be the Haar integral characterizing the worst-case classifier for rotation invariance (see also Appendices F.3.2 and F.3.3).
Revised: be the Haar integral characterizing the worst-case classifier for rotation invariance (see also Appendices F.3.2 and F.3.3).

Original: Then, for all V ∈ R N × D and κ ∈ R + ,
Revised: Then, for all V ∈ R N × D and κ ∈ R + ,

Original: [Equation]
Revised: [Equation]

Original: Note that these (with V = X and V = X ′ ) are exactly the expectations determining the optimal value of our variance-constrained optimization problem (see Theorem 3).
Revised: Note that these (with V = X and V = X ′ ) are exactly the expectations determining the optimal value of our variance-constrained optimization problem (see Theorem 3).

Original: To prove this theorem, we first simplify the integrand of β SE ( D ) X
Revised: To prove this theorem, we first simplify the integrand of β SE ( D ) X

Original: ( Z ) by moving the rotation matrix into the second argument of the Frobenius inner product (note that, since R is a rotation matrix, we have R − 1 = R T ):
Revised: ( Z ) by moving the rotation matrix into the second argument of the Frobenius inner product (note that, since R is a rotation matrix, we have R − 1 = R T ):

Original: [Equation]
Revised: [Equation]

Original: Next, we bring our integral into the same form as in our derivation of the certificate for translation invariance (see Appendix F.4.1), so that we can easily integrate out the translation vector b .
Revised: Next, we bring our integral into the same form as in our derivation of the certificate for translation invariance (see Appendix F.4.1), so that we can easily integrate out the translation vector b .

Original: First, we eliminate the rotation matrix R T via the substitution c = R T b :
Revised: First, we eliminate the rotation matrix R T via the substitution c = R T b :

Original: [Equation]
Revised: [Equation]

Original: For the next step, let A =
Revised: For the next step, let A =

Original: [Equation]
Revised: [Equation]

Original: with inverse A
Revised: with inverse A

Original: [Equation]
Revised: [Equation]

Original: = (cid:20)
Revised: = (cid:20)

Original: [Equation]
Revised: [Equation]

Original: and all-zeros vector 0 N − 1 ∈ R N − 1 , all-ones vector 1 N − 1 ∈ R N − 1 and identity matrix
Revised: and all-zeros vector 0 N − 1 ∈ R N − 1 , all-ones vector 1 N − 1 ∈ R N − 1 and identity matrix

Original: I N − 1 .
Revised: I N − 1 .

Original: Multiplying a vector or matrix with A is equivalent to subtracting the first row from all other rows.
Revised: Multiplying a vector or matrix with A is equivalent to subtracting the first row from all other rows.

Original: We introduce the redundant factor A − 1 A and make the substitution u
Revised: We introduce the redundant factor A − 1 A and make the substitution u

Original: = c + Z 1 to show that
Revised: = c + Z 1 to show that

Original: [Equation]
Revised: [Equation]

Original: SE
Revised: SE

Original: [Equation]
Revised: [Equation]

Original: D
Revised: D

Original: [Equation]
Revised: [Equation]

Original: X
Revised: X

Original: [Equation]
Revised: [Equation]

Original: Next
Revised: Next

Original: [Equation]
Revised: [Equation]

Original: we use the behavior of multivariate normal densities under affine transformation (see also Lemma 2)
Revised: we use the behavior of multivariate normal densities under affine transformation (see also Lemma 2)

Original: [Equation]
Revised: [Equation]

Original: as well as the fact that marginalizing out one dimension of a multivariate normal density is equivalent to dropping the correspond row of the mean and corresponding row and column of the adjacency matrix
Revised: as well as the fact that marginalizing out one dimension of a multivariate normal density is equivalent to dropping the correspond row of the mean and corresponding row and column of the adjacency matrix

Original: [Equation]
Revised: [Equation]

Original: to show that
Revised: to show that

Original: [Equation]
Revised: [Equation]

Original: Finally, reusing the fact that ( A 2: ) T (cid:0)
Revised: Finally, reusing the fact that ( A 2: ) T (cid:0)

Original: A 2: ( A 2: ) T (cid:1) − 1 A 2: X = ˜
Revised: A 2: ( A 2: ) T (cid:1) − 1 A 2: X = ˜

Original: X with ˜ X
Revised: X with ˜ X

Original: = X − 1 N X from Appendix F.4.1 proves that
Revised: = X − 1 N X from Appendix F.4.1 proves that

Original: [Equation]
Revised: [Equation]

Original: The last term is the Haar integral β
Revised: The last term is the Haar integral β

Original: SO ( D ) ˜
Revised: SO ( D ) ˜

Original: X ( Z ) characterizing the worst-case classifier for SO ( D ) , after centering X .
Revised: X ( Z ) characterizing the worst-case classifier for SO ( D ) , after centering X .

Original: This means that we have
Revised: This means that we have

Original: [Equation]
Revised: [Equation]

Original: Note that, when using ∝ in the previous steps, we have only removed constant factors that appear in the enumerator and denominator and thus cancel out.
Revised: Note that, when using ∝ in the previous steps, we have only removed constant factors that appear in the enumerator and denominator and thus cancel out.

Original: The last thing we need to do is verify that not only the classifiers, but also their expectations under V and ˜ V are identical (see Theorem 7).
Revised: The last thing we need to do is verify that not only the classifiers, but also their expectations under V and ˜ V are identical (see Theorem 7).

Original: We use the fact that the worst-case classifier only depends on a low-dimensional linear projection of our random input variable Z (similar to our derivations for
Revised: We use the fact that the worst-case classifier only depends on a low-dimensional linear projection of our random input variable Z (similar to our derivations for

Original: Monte Carlo evaluation in Appendices F.4.2 and F.4.3):
Revised: Monte Carlo evaluation in Appendices F.4.2 and F.4.3):

Original: [Equation]
Revised: [Equation]

Original: By definition of our smoothing distribution, we have vec( Z ) ∼ N (vec( V ) | vec( V ) , σ 2 I D · N ) for Z ∼ µ V .
Revised: By definition of our smoothing distribution, we have vec( Z ) ∼ N (vec( V ) | vec( V ) , σ 2 I D · N ) for Z ∼ µ V .

Original: Because our worst-case classifier for SE ( D ) performs a linear transformation of vec( Z ) , we have
Revised: Because our worst-case classifier for SE ( D ) performs a linear transformation of vec( Z ) , we have

Original: [Equation]
Revised: [Equation]

Original: with m V = W vec( V ) and Σ = W W T
Revised: with m V = W vec( V ) and Σ = W W T

Original: [Equation]
Revised: [Equation]

Original: where
Revised: where

Original: [Equation]
Revised: [Equation]

Original: Similarly
Revised: Similarly

Original: [Equation]
Revised: [Equation]

Original: we have for SO (3) (after centering V )
Revised: we have for SO (3) (after centering V )

Original: [Equation]
Revised: [Equation]

Original: that
Revised: that

Original: [Equation]
Revised: [Equation]

Original: with m ˜ V
Revised: with m ˜ V

Original: = W vec( ˜ V ) and Σ = W W T .
Revised: = W vec( ˜ V ) and Σ = W W T .

Original: Finally, due to the fact (see Lemma 7) that calculating an inner product between a centered and an uncentered vector (here: columns of ˜ X and V ) is equivalent to centering both vectors before calculating the inner product (here: columns of ˜ X and ˜ V ), we have m V = m ˜ V .
Revised: Finally, due to the fact (see Lemma 7) that calculating an inner product between a centered and an uncentered vector (here: columns of ˜ X and V ) is equivalent to centering both vectors before calculating the inner product (here: columns of ˜ X and ˜ V ), we have m V = m ˜ V .

Original: Thus, both expectations are equal, which concludes our proof.
Revised: Thus, both expectations are equal, which concludes our proof.

Original: All of the discussed tight certificates, save for the one for translation invariance, are of the form
Revised: All of the discussed tight certificates, save for the one for translation invariance, are of the form

Original: [Equation]
Revised: [Equation]

Original: with κ ∈ R such that
Revised: with κ ∈ R such that

Original: [Equation]
Revised: [Equation]

Original: where g : R N × D → Y is our base classifier, µ X , µ X ′ are the clean and perturbed smoothing distribution, υ (1) and υ (2) are distributions over some set S , and ρ : S → R is some arbitrary scalar-valued function.
Revised: where g : R N × D → Y is our base classifier, µ X , µ X ′ are the clean and perturbed smoothing distribution, υ (1) and υ (2) are distributions over some set S , and ρ : S → R is some arbitrary scalar-valued function.

Original: As discussed in Section 6.3, we propose to compute a narrow probabilistic lower bound on min h ∈ H T Pr Z ∼ µ X ′
Revised: As discussed in Section 6.3, we propose to compute a narrow probabilistic lower bound on min h ∈ H T Pr Z ∼ µ X ′

Original: [ h ( Z ) =
Revised: [ h ( Z ) =

Original: y ∗ ]] by combining three confidence bounds.
Revised: y ∗ ]] by combining three confidence bounds.

Original: Before computing these bounds, we have to inspect the monotonicity of our certificate.
Revised: Before computing these bounds, we have to inspect the monotonicity of our certificate.

Original: Evidently, Pr V ∼ υ (1) [ ρ ( V ) ≤ κ ] is monotonically decreasing in κ .
Revised: Evidently, Pr V ∼ υ (1) [ ρ ( V ) ≤ κ ] is monotonically decreasing in κ .

Original: Furthermore, κ is monotonically decreasing in p X ,y ∗ : If p X ,y ∗ decreases, then a smaller κ is sufficient for fulfilling the constraint
Revised: Furthermore, κ is monotonically decreasing in p X ,y ∗ : If p X ,y ∗ decreases, then a smaller κ is sufficient for fulfilling the constraint

Original: Pr V ∼ υ (2) [ ρ ( V ) ≤ κ ] =
Revised: Pr V ∼ υ (2) [ ρ ( V ) ≤ κ ] =

Original: p X ,y ∗ .
Revised: p X ,y ∗ .

Original: Thus, we can compute a lower bound on our certificate by
Revised: Thus, we can compute a lower bound on our certificate by

Original: lower-bounding p X ,y ∗ , i.e. p X ,y ∗ ≤ p X ,y ∗ , 2.
Revised: lower-bounding p X ,y ∗ , i.e. p X ,y ∗ ≤ p X ,y ∗ , 2.

Original: lower-bounding the κ ∈ R that fulfills Pr V ∼ υ (2) [ ρ ( V ) ≤ κ ] =
Revised: lower-bounding the κ ∈ R that fulfills Pr V ∼ υ (2) [ ρ ( V ) ≤ κ ] =

Original: p X ,y ∗ , i.e. κ ≤ κ 3.
Revised: p X ,y ∗ , i.e. κ ≤ κ 3.

Original: and then lower-bounding Pr V ∼ υ (1) [ ρ ( V ) ≤ κ ] .
Revised: and then lower-bounding Pr V ∼ υ (1) [ ρ ( V ) ≤ κ ] .

Original: The random variable 1 [ g ( Z ) =
Revised: The random variable 1 [ g ( Z ) =

Original: y ∗ ] is a Bernoulli random variable.
Revised: y ∗ ] is a Bernoulli random variable.

Original: Just like other randomized smoothing methods (e.g. [26]), we can lower-bound p X ,y ∗ by evaluating g on N 1 samples from µ X and computing a binomial proportion confidence bound, such as the the Clopper-Pearson confidence bound [141].
Revised: Just like other randomized smoothing methods (e.g. [26]), we can lower-bound p X ,y ∗ by evaluating g on N 1 samples from µ X and computing a binomial proportion confidence bound, such as the the Clopper-Pearson confidence bound [141].

Original: Lower-bounding κ requires computing a lower bound on F − 1 ( p X ,y ∗ ) , where F − 1 is the quantile function of ρ ( V ) with V ∼ υ (2) .
Revised: Lower-bounding κ requires computing a lower bound on F − 1 ( p X ,y ∗ ) , where F − 1 is the quantile function of ρ ( V ) with V ∼ υ (2) .

Original: A non-parametric lower confidence bound on this quantile can be constructed by evaluating ρ ( V ) on N 2 samples from υ (2) and returning the largest order statistic
Revised: A non-parametric lower confidence bound on this quantile can be constructed by evaluating ρ ( V ) on N 2 samples from υ (2) and returning the largest order statistic

Original: R ( n ) such that (cid:2) ρ ( V ) ≤ R ( n ) (cid:3) ≤ p X ,y ∗ holds with high probability (see Section 5.2.1.
Revised: R ( n ) such that (cid:2) ρ ( V ) ≤ R ( n ) (cid:3) ≤ p X ,y ∗ holds with high probability (see Section 5.2.1.

Original: of [142] and Algorithm 1 below).
Revised: of [142] and Algorithm 1 below).

Original: The random variable [ ρ ( V ) ≤ κ ] is another Bernoulli random variable and can thus be lower-bounded by evaluating ρ ( V ) on N 3 samples from µ X ′ and computing a binomial proportion confidence bound.
Revised: The random variable [ ρ ( V ) ≤ κ ] is another Bernoulli random variable and can thus be lower-bounded by evaluating ρ ( V ) on N 3 samples from µ X ′ and computing a binomial proportion confidence bound.

Original: We want all three confidence bounds to simultaneously hold with high probability 1 − α .
Revised: We want all three confidence bounds to simultaneously hold with high probability 1 − α .

Original: We ensure this by using Holm-Bonferroni [131] correction to account for the multiple comparisons problem.
Revised: We ensure this by using Holm-Bonferroni [131] correction to account for the multiple comparisons problem.

Original: In our case, this corresponds to computing the first bound with significance α , the second one with significance α / 2 and the last one with significance α / 3 .
Revised: In our case, this corresponds to computing the first bound with significance α , the second one with significance α / 2 and the last one with significance α / 3 .

Original: Algorithm 1 summarizes our certification procedure.
Revised: Algorithm 1 summarizes our certification procedure.

Original: L OWER C OUNF B OUND refers to the ClopperPearson lower confidence bound.
Revised: L OWER C OUNF B OUND refers to the ClopperPearson lower confidence bound.

Original: B IN PV ALUE ( n, N 2 , ≥ , p X ,y ∗ ) refers to the p -value of a Binomial test with the null-hypothesis that the success probability is greater or equal p X ,y ∗ .
Revised: B IN PV ALUE ( n, N 2 , ≥ , p X ,y ∗ ) refers to the p -value of a Binomial test with the null-hypothesis that the success probability is greater or equal p X ,y ∗ .

Original: Note that only the first confidence bound requires evaluating the base classifier g .
Revised: Note that only the first confidence bound requires evaluating the base classifier g .

Original: For the other two confidence bounds a large number of samples can be evaluated at little computational cost.
Revised: For the other two confidence bounds a large number of samples can be evaluated at little computational cost.

Original: [Table]
Revised: [Table]

Original: G Proof of Theorem
Revised: G Proof of Theorem

Original: Next, we prove that the post-processing-based certificate for rotation-invariance ( T = SO ( D ) ) is not tight, which we formalized in Section 6.3 as follows:
Revised: Next, we prove that the post-processing-based certificate for rotation-invariance ( T = SO ( D ) ) is not tight, which we formalized in Section 6.3 as follows:

Original: Theorem 5.
Revised: Theorem 5.

Original: Let g : R N × D
Revised: Let g : R N × D

Original: → Y be invariant under T =
Revised: → Y be invariant under T =

Original: SO ( D ) and H T be defined as in Eq.
Revised: SO ( D ) and H T be defined as in Eq.

Original: Assume that perturbed input X ′ is not obtained via rotation of X , i.e. ∄ R ∈
Revised: Assume that perturbed input X ′ is not obtained via rotation of X , i.e. ∄ R ∈

Original: SO ( D ) :
Revised: SO ( D ) :

Original: X ′ =
Revised: X ′ =

Original: XR T .
Revised: XR T .

Original: Further assume that p X ,y ∗ ∈ (0 , 1) .
Revised: Further assume that p X ,y ∗ ∈ (0 , 1) .

Original: Then, for all R ∈ SO ( D ) :
Revised: Then, for all R ∈ SO ( D ) :

Original: [Equation]
Revised: [Equation]

Original: The right-hand side term is the optimal value of the black-box optimization problem evaluated for perturbed input X ′ R T , i.e.
Revised: The right-hand side term is the optimal value of the black-box optimization problem evaluated for perturbed input X ′ R T , i.e.

Original: [Equation]
Revised: [Equation]

Original: with H = (cid:8) h : R N × D
Revised: with H = (cid:8) h : R N × D

Original: → Y | Pr Z ∼ µ X [ h ( Z ) =
Revised: → Y | Pr Z ∼ µ X [ h ( Z ) =

Original: y ∗ ] ≥ p X ,y ∗ (cid:9) .
Revised: y ∗ ] ≥ p X ,y ∗ (cid:9) .

Original: The left-hand side term is the optimal value of the gray-box optimization problem evaluated for perturbed input X ′ .
Revised: The left-hand side term is the optimal value of the gray-box optimization problem evaluated for perturbed input X ′ .

Original: Note that H T is a set of rotation invariant classifiers.
Revised: Note that H T is a set of rotation invariant classifiers.

Original: In Appendix D we have proven that rotation of the smoothing distribution’s mean does not have an effect on the prediction probabilities of such classifiers, i.e.
Revised: In Appendix D we have proven that rotation of the smoothing distribution’s mean does not have an effect on the prediction probabilities of such classifiers, i.e.

Original: [Equation]
Revised: [Equation]

Original: Thus, we can prove Theorem 5 by proving the following, more general statement:
Revised: Thus, we can prove Theorem 5 by proving the following, more general statement:

Original: Lemma 8.
Revised: Lemma 8.

Original: Let g : R N × D
Revised: Let g : R N × D

Original: → Y be invariant under T =
Revised: → Y be invariant under T =

Original: SO ( D ) and H T be defined as in Eq.
Revised: SO ( D ) and H T be defined as in Eq.

Original: Further assume that X ′
Revised: Further assume that X ′

Original: ̸
Revised: ̸

Original: = X and that p X ,y ∗ ∈ (0 , 1) .
Revised: = X and that p X ,y ∗ ∈ (0 , 1) .

Original: Then:
Revised: Then:

Original: [Equation]
Revised: [Equation]

Original: We can do so by showing that no optimal solution to the r.h.s.
Revised: We can do so by showing that no optimal solution to the r.h.s.

Original: black-box problem from Eq.
Revised: black-box problem from Eq.

Original: (42) is a feasible solution to the l.h.s.
Revised: (42) is a feasible solution to the l.h.s.

Original: gray-box problem.
Revised: gray-box problem.

Original: More formally, let H ∗ ⊆
Revised: More formally, let H ∗ ⊆

Original: H be the set of all classifiers minimizing the r.h.s.
Revised: H be the set of all classifiers minimizing the r.h.s.

Original: black box problem.
Revised: black box problem.

Original: We have H ∗ ⊆
Revised: We have H ∗ ⊆

Original: H and H T ⊆ H .
Revised: H and H T ⊆ H .

Original: By showing that H ∗ ∩ H T = ∅ , we prove that H T ⊆
Revised: By showing that H ∗ ∩ H T = ∅ , we prove that H T ⊆

Original: H \ H ∗ , i.e. all rotation invariant classifiers from H T yield strictly larger optimal values.
Revised: H \ H ∗ , i.e. all rotation invariant classifiers from H T yield strictly larger optimal values.

Original: Recall that an optimal solution to the r.h.s.
Revised: Recall that an optimal solution to the r.h.s.

Original: black-box problem in Eq.
Revised: black-box problem in Eq.

Original: (42) is given by the NeymanPearson lemma [107].
Revised: (42) is given by the NeymanPearson lemma [107].

Original: Later work on hypothesis testing has shown that the Neyman-Pearson lemma is not only a sufficient, but a necessary condition for optimality[143]: Every most powerful test must fulfill the likelihood ratio inequalities, save for sets of zero measure.
Revised: Later work on hypothesis testing has shown that the Neyman-Pearson lemma is not only a sufficient, but a necessary condition for optimality[143]: Every most powerful test must fulfill the likelihood ratio inequalities, save for sets of zero measure.

Original: For our formulation of the
Revised: For our formulation of the

Original: Neyman-Pearson lemma (see Lemma 5), this means that any classifier h that is an optimal solution to the r.h.s.
Revised: Neyman-Pearson lemma (see Lemma 5), this means that any classifier h that is an optimal solution to the r.h.s.

Original: black-box problem from Eq.
Revised: black-box problem from Eq.

Original: (42) must fulfill
Revised: (42) must fulfill

Original: [Equation]
Revised: [Equation]

Original: for some κ ∈ R , save for sets of zero measure.
Revised: for some κ ∈ R , save for sets of zero measure.

Original: Cohen et al.
Revised: Cohen et al.

Original: [26] have shown that if X ′ ̸
Revised: [26] have shown that if X ′ ̸

Original: = X and p ∈ (0 , 1) and µ X , µ X ′ are isotropic Gaussian distributions, then these likelihood ratio inequalities correspond to a linear decision boundary, i.e. any optimal solution most fulfill
Revised: = X and p ∈ (0 , 1) and µ X , µ X ′ are isotropic Gaussian distributions, then these likelihood ratio inequalities correspond to a linear decision boundary, i.e. any optimal solution most fulfill

Original: [Equation]
Revised: [Equation]

Original: for some b ∈ R and W ∈ R N × D with w ̸ = 0 N,D , save for sets of zero measure.
Revised: for some b ∈ R and W ∈ R N × D with w ̸ = 0 N,D , save for sets of zero measure.

Original: Here, ⟨ A , B ⟩ F = vec( W ) T vec( Z ) is the Frobenius inner product.
Revised: Here, ⟨ A , B ⟩ F = vec( W ) T vec( Z ) is the Frobenius inner product.

Original: Therefore, we can prove Lemma 8 by showing that classifiers complying with Eq.
Revised: Therefore, we can prove Lemma 8 by showing that classifiers complying with Eq.

Original: (43) (save for sets of zero measure) are not rotation invariant, i.e. they are not feasible solutions to the gray-box optimization problem.
Revised: (43) (save for sets of zero measure) are not rotation invariant, i.e. they are not feasible solutions to the gray-box optimization problem.

Original: We first do this for the case that D is even and then for the case that D is odd.
Revised: We first do this for the case that D is even and then for the case that D is odd.

Original: Lemma 9.
Revised: Lemma 9.

Original: Assume that D = 2 k for some k ∈ N .
Revised: Assume that D = 2 k for some k ∈ N .

Original: Let h : R N × D
Revised: Let h : R N × D

Original: → Y be a classifier that fulfills
Revised: → Y be a classifier that fulfills

Original: [Equation]
Revised: [Equation]

Original: for some b ∈ R and w ∈ R N × D with w ̸ = 0 N · D , save for sets of zero measure.
Revised: for some b ∈ R and w ∈ R N × D with w ̸ = 0 N · D , save for sets of zero measure.

Original: Then, there is an input ˆ Z and a rotation matrix R ∈
Revised: Then, there is an input ˆ Z and a rotation matrix R ∈

Original: SO ( D ) such that h ( ˆ Z ) ̸
Revised: SO ( D ) such that h ( ˆ Z ) ̸

Original: = h ( ˆ ZR T ) .
Revised: = h ( ˆ ZR T ) .

Original: Proof.
Revised: Proof.

Original: Case 1.
Revised: Case 1.

Original: Assume that b ≥ 0 .
Revised: Assume that b ≥ 0 .

Original: Consider the set S
Revised: Consider the set S

Original: = { Z | ⟨ W , Z ⟩ F < b } .
Revised: = { Z | ⟨ W , Z ⟩ F < b } .

Original: By construction, we have ∀ Z ∈ S : ⟨ W , Z ⟩ F + b < 0 and ∀ Z ∈ S : ⟨ W , − Z ⟩ F + b > 0 .
Revised: By construction, we have ∀ Z ∈ S : ⟨ W , Z ⟩ F + b < 0 and ∀ Z ∈ S : ⟨ W , − Z ⟩ F + b > 0 .

Original: There must be at least one ˆ Z ∈ S with h ( ˆ Z ) =
Revised: There must be at least one ˆ Z ∈ S with h ( ˆ Z ) =

Original: y ∗ and h ( ˆ Z )
Revised: y ∗ and h ( ˆ Z )

Original: ̸
Revised: ̸

Original: = y ∗ .
Revised: = y ∗ .

Original: Otherwise, all points Z ∈ S would need to fulfill h ( Z ) ̸
Revised: Otherwise, all points Z ∈ S would need to fulfill h ( Z ) ̸

Original: =
Revised: =

Original: y ∗ or h ( − Z )
Revised: y ∗ or h ( − Z )

Original: =
Revised: =

Original: y ∗ , in which case we would have sets of non-zero measure violating the likelihood ratio inequalities from Eq.
Revised: y ∗ , in which case we would have sets of non-zero measure violating the likelihood ratio inequalities from Eq.

Original: Note that − ˆ Z = ˆ Z ( − I N ) T and − I N ∈
Revised: Note that − ˆ Z = ˆ Z ( − I N ) T and − I N ∈

Original: SO
Revised: SO

Original: ( D ) , because − I N is orthonormal and det( − I N )
Revised: ( D ) , because − I N is orthonormal and det( − I N )

Original: = 1 , due to D being even.
Revised: = 1 , due to D being even.

Original: Thus, we have found an input ˆ Z and a rotation matrix R ∈
Revised: Thus, we have found an input ˆ Z and a rotation matrix R ∈

Original: SO ( D ) such that h ( Z ) ̸ = h ( ZR T ) .
Revised: SO ( D ) such that h ( Z ) ̸ = h ( ZR T ) .

Original: Case 2.
Revised: Case 2.

Original: Assume that b < 0 .
Revised: Assume that b < 0 .

Original: This case follows analogously by constructing the set S
Revised: This case follows analogously by constructing the set S

Original: = { Z | ⟨ W , Z ⟩ F > b } and considering a point ˆ Z ∈ S with h ( ˆ Z )
Revised: = { Z | ⟨ W , Z ⟩ F > b } and considering a point ˆ Z ∈ S with h ( ˆ Z )

Original: ̸ = y ∗ and h ( − ˆ Z )
Revised: ̸ = y ∗ and h ( − ˆ Z )

Original: =
Revised: =

Original: y ∗
Revised: y ∗

Original: Now, we consider the case that D is odd.
Revised: Now, we consider the case that D is odd.

Original: Here, − I N / ∈ SO ( D ) , because det( − I N ) = − 1 .
Revised: Here, − I N / ∈ SO ( D ) , because det( − I N ) = − 1 .

Original: Therefore, we will need to use slightly more complicated constructions.
Revised: Therefore, we will need to use slightly more complicated constructions.

Original: Lemma 10.
Revised: Lemma 10.

Original: Assume that D = 2 k − 1 for some k ∈ N .
Revised: Assume that D = 2 k − 1 for some k ∈ N .

Original: Let h : R N × D
Revised: Let h : R N × D

Original: → Y be a classifier that fulfills
Revised: → Y be a classifier that fulfills

Original: [Equation]
Revised: [Equation]

Original: for some b ∈ R and w ∈ R N × D with w ̸ = 0 N · D , save for sets of zero measure.
Revised: for some b ∈ R and w ∈ R N × D with w ̸ = 0 N · D , save for sets of zero measure.

Original: Then, there is an input ˆ Z and a rotation matrix R ∈
Revised: Then, there is an input ˆ Z and a rotation matrix R ∈

Original: SO ( D ) such that h ( ˆ Z ) ̸
Revised: SO ( D ) such that h ( ˆ Z ) ̸

Original: = h ( ˆ ZR T ) .
Revised: = h ( ˆ ZR T ) .

Original: Proof.
Revised: Proof.

Original: In the following, let A =
Revised: In the following, let A =

Original: (cid:20)0 TD 0 D −− I D −(cid:21) .
Revised: (cid:20)0 TD 0 D −− I D −(cid:21) .

Original: We have A ∈ SO ( D ) , because A is orthonormal and det( A )
Revised: We have A ∈ SO ( D ) , because A is orthonormal and det( A )

Original: = 1 , due to D being odd.
Revised: = 1 , due to D being odd.

Original: Case 1.
Revised: Case 1.

Original: Assume that b ≥ 0 .
Revised: Assume that b ≥ 0 .

Original: We know that W ̸ = 0 N,D .
Revised: We know that W ̸ = 0 N,D .

Original: Without loss of generality, assume that W : , 2: ̸ = 0 N, ( D − 1) , i.e. at least one of the the last D − 1 columns is non-zero.
Revised: Without loss of generality, assume that W : , 2: ̸ = 0 N, ( D − 1) , i.e. at least one of the the last D − 1 columns is non-zero.

Original: Consider the set S
Revised: Consider the set S

Original: = (cid:8) Z | W T : , 1 Z : , 1 ∈ [0 , 1] ∧ ⟨ W : , 2: , Z : , 2: ⟩ F < b − 1 (cid:9) .
Revised: = (cid:8) Z | W T : , 1 Z : , 1 ∈ [0 , 1] ∧ ⟨ W : , 2: , Z : , 2: ⟩ F < b − 1 (cid:9) .

Original: By construction, we have ∀ Z ∈ S : ⟨ W , Z ⟩ F + b < 0 and ∀ Z ∈ S : (cid:10) W , ZA T (cid:11) F + b > 0 .
Revised: By construction, we have ∀ Z ∈ S : ⟨ W , Z ⟩ F + b < 0 and ∀ Z ∈ S : (cid:10) W , ZA T (cid:11) F + b > 0 .

Original: There must be at least one ˆ Z ∈ S with h ( ˆ Z ) =
Revised: There must be at least one ˆ Z ∈ S with h ( ˆ Z ) =

Original: y ∗ and h ( ˆ ZA T )
Revised: y ∗ and h ( ˆ ZA T )

Original: ̸
Revised: ̸

Original: = y ∗ .
Revised: = y ∗ .

Original: Otherwise, all points Z ∈ S would need to fulfill h ( Z ) ̸
Revised: Otherwise, all points Z ∈ S would need to fulfill h ( Z ) ̸

Original: =
Revised: =

Original: y ∗ or h ( ZA T )
Revised: y ∗ or h ( ZA T )

Original: =
Revised: =

Original: y ∗ , in which case we would have sets of non-zero measure violating Eq.
Revised: y ∗ , in which case we would have sets of non-zero measure violating Eq.

Original: Case 2.
Revised: Case 2.

Original: Assume that b < 0 .
Revised: Assume that b < 0 .

Original: This case follows analogously by constructing the set S
Revised: This case follows analogously by constructing the set S

Original: = (cid:8)
Revised: = (cid:8)

Original: [Equation]
Revised: [Equation]

Original: , 0] ∧ ⟨ W : , 2: , Z : , 2: ⟩ F > b + 1 (cid:9) and considering a point ˆ Z ∈ S with h ( ˆ Z ) ̸
Revised: , 0] ∧ ⟨ W : , 2: , Z : , 2: ⟩ F > b + 1 (cid:9) and considering a point ˆ Z ∈ S with h ( ˆ Z ) ̸

Original: = y ∗ and h ( A ˆ Z ) =
Revised: = y ∗ and h ( A ˆ Z ) =

Original: y ∗ .
Revised: y ∗ .

Original: Lemmas 9 and 10 combined prove Lemma 8, which in turn proves Theorem 5.
Revised: Lemmas 9 and 10 combined prove Lemma 8, which in turn proves Theorem 5.

Original: H Multi-class certificates
Revised: H Multi-class certificates

Original: In our discussions in Sections 3.1, 5 and 6, we have only considered binary certificates 5 .
Revised: In our discussions in Sections 3.1, 5 and 6, we have only considered binary certificates 5 .

Original: That is, prediction y ∗ of smoothed classifier is certifiably robust to a perturbed input X ′ if the probability of base classifier g predicting y ∗ under perturbed smoothing distribution µ X ′ is greater than the probability of all other classes combined.
Revised: That is, prediction y ∗ of smoothed classifier is certifiably robust to a perturbed input X ′ if the probability of base classifier g predicting y ∗ under perturbed smoothing distribution µ X ′ is greater than the probability of all other classes combined.

Original: That is, p X ′ ,y ∗ > 12 .
Revised: That is, p X ′ ,y ∗ > 12 .

Original: We have then, for different invariances, computed lower bounds p X ′ ,y ∗ ≤ p X ′ ,y ∗ .
Revised: We have then, for different invariances, computed lower bounds p X ′ ,y ∗ ≤ p X ′ ,y ∗ .

Original: If p X ′ ,y ∗ > 12 , then p X ′ ,y ∗ > 12 and the prediction is certifiably robust.
Revised: If p X ′ ,y ∗ > 12 , then p X ′ ,y ∗ > 12 and the prediction is certifiably robust.

Original: Alternatively, one can certify robustness by showing that y ∗ remains more likely than the second most likely class under the perturbed smoothing distribution [26], i.e. p X ′ ,y ∗ > max y ̸
Revised: Alternatively, one can certify robustness by showing that y ∗ remains more likely than the second most likely class under the perturbed smoothing distribution [26], i.e. p X ′ ,y ∗ > max y ̸

Original: =
Revised: =

Original: y ∗ p X ′ ,y .
Revised: y ∗ p X ′ ,y .

Original: To this end, one can compute a lower bound p X ′ ,y ∗ ≤ p X ′ ,y ∗ and an upper-bound max y ̸
Revised: To this end, one can compute a lower bound p X ′ ,y ∗ ≤ p X ′ ,y ∗ and an upper-bound max y ̸

Original: =
Revised: =

Original: y ∗ p X ′ ,y ≥ max y ̸
Revised: y ∗ p X ′ ,y ≥ max y ̸

Original: =
Revised: =

Original: y ∗ p X ′ ,y .
Revised: y ∗ p X ′ ,y .

Original: If p X ′ ,y ∗ > max y ̸
Revised: If p X ′ ,y ∗ > max y ̸

Original: =
Revised: =

Original: y ∗ p X ′ ,y , then robustness is guaranteed.
Revised: y ∗ p X ′ ,y , then robustness is guaranteed.

Original: Due to the monotonicity of randomized smoothing certificates w.r.t.
Revised: Due to the monotonicity of randomized smoothing certificates w.r.t.

Original: to the prediction probabilities, this can be achieved by showing that p X ′ ,y ∗ > p X ′ ,y ′ , where y ′ =
Revised: to the prediction probabilities, this can be achieved by showing that p X ′ ,y ∗ > p X ′ ,y ′ , where y ′ =

Original: argmax y ̸
Revised: argmax y ̸

Original: =
Revised: =

Original: y ∗ p X ,y is the second most likely class under the clean smoothing distribution.
Revised: y ∗ p X ,y is the second most likely class under the clean smoothing distribution.

Original: This second most likely class y ′ can be found via a binomial test, see [26, 144].
Revised: This second most likely class y ′ can be found via a binomial test, see [26, 144].

Original: The lower bound p X ′ ,y ∗ can be obtained using the same formulae discussed throughout the main text (e.g. p X ′ ,y ∗ =
Revised: The lower bound p X ′ ,y ∗ can be obtained using the same formulae discussed throughout the main text (e.g. p X ′ ,y ∗ =

Original: Φ (cid:16) Φ − 1 ( p X ,y ′ ) − || ∆ || 2 σ (cid:17) for black-box randomized smoothing).
Revised: Φ (cid:16) Φ − 1 ( p X ,y ′ ) − || ∆ || 2 σ (cid:17) for black-box randomized smoothing).

Original: The upper bound p X ′ ,y ′ can be found via the Neyman-Pearson upper bound discussed in Appendix F.2:
Revised: The upper bound p X ′ ,y ′ can be found via the Neyman-Pearson upper bound discussed in Appendix F.2:

Original: Lemma 6 (Neyman-Pearson upper bound) .
Revised: Lemma 6 (Neyman-Pearson upper bound) .

Original: Let µ X ′ , µ X , be two continuous distributions over a measurable set
Revised: Let µ X ′ , µ X , be two continuous distributions over a measurable set

Original: A such that, for all κ ∈ R + , the set (cid:110) z | µ X ′ ( z ) µ X ( z ) =
Revised: A such that, for all κ ∈ R + , the set (cid:110) z | µ X ′ ( z ) µ X ( z ) =

Original: κ (cid:111) has measure zero.
Revised: κ (cid:111) has measure zero.

Original: Consider an arbitrary label set Y , a specific class label y ∈ Y and scalar p ∈ [0 , 1] .
Revised: Consider an arbitrary label set Y , a specific class label y ∈ Y and scalar p ∈ [0 , 1] .

Original: Then
Revised: Then

Original: [Equation]
Revised: [Equation]

Original: with h ∗ ( z ) = 1 (cid:20)
Revised: with h ∗ ( z ) = 1 (cid:20)

Original: [Equation]
Revised: [Equation]

Original: X ′
Revised: X ′

Original: [Equation]
Revised: [Equation]

Original: and κ ∈ R + such that
Revised: and κ ∈ R + such that

Original: [Equation]
Revised: [Equation]

Original: Note that it is identical to the lower bound, save for replacing " ≤ " with " ≥ " and vice-versa.
Revised: Note that it is identical to the lower bound, save for replacing " ≤ " with " ≥ " and vice-versa.

Original: The resulting black-box upper bound is p X ′ ,y ′ =
Revised: The resulting black-box upper bound is p X ′ ,y ′ =

Original: Φ (cid:16) Φ − 1 ( p X ,y ′ )
Revised: Φ (cid:16) Φ − 1 ( p X ,y ′ )

Original: + || ∆ || 2 σ (cid:17) .
Revised: + || ∆ || 2 σ (cid:17) .

Original: It is identical to the black-box lower bound, save for replacing " − " with " + ".
Revised: It is identical to the black-box lower bound, save for replacing " − " with " + ".

Original: Substituting both bounds into p X ′ ,y ∗ > max y ̸
Revised: Substituting both bounds into p X ′ ,y ∗ > max y ̸

Original: =
Revised: =

Original: y ∗ p X ′ ,y and applying the inverse-normal CDF Φ − 1 to both sides shows that the classifier is certifiably robust if || ∆ || ≤ σ 2 (cid:0) Φ − 1 ( p X ′ ,y ∗ ) − Φ − 1 ( p X ′ ,y ′ )
Revised: y ∗ p X ′ ,y and applying the inverse-normal CDF Φ − 1 to both sides shows that the classifier is certifiably robust if || ∆ || ≤ σ 2 (cid:0) Φ − 1 ( p X ′ ,y ∗ ) − Φ − 1 ( p X ′ ,y ′ )

Original: (cid:1) [26].
Revised: (cid:1) [26].

Original: Evidently, the orbit-based versions of the multi-class certificates are identical to their binary counterparts, except that the radius of the underlying black-box certificate changes.
Revised: Evidently, the orbit-based versions of the multi-class certificates are identical to their binary counterparts, except that the radius of the underlying black-box certificate changes.

Original: For the tight gray-box approach, upper bounds can be derived by going through the same derivations as in Appendix F, but using the Neyman-Pearson upper bounds from Lemma 6, i.e. replacing " ≤ " with " ≥ " and vice-versa.
Revised: For the tight gray-box approach, upper bounds can be derived by going through the same derivations as in Appendix F, but using the Neyman-Pearson upper bounds from Lemma 6, i.e. replacing " ≤ " with " ≥ " and vice-versa.

Original: (see Lemma 6).
Revised: (see Lemma 6).

Original: In the case of translation invariance, this results in upper bound Φ (cid:16)
Revised: In the case of translation invariance, this results in upper bound Φ (cid:16)

Original: Φ − 1 ( p X ,y ′ )
Revised: Φ − 1 ( p X ,y ′ )

Original: + || ∆ − 1 N ∆ || 2 σ (cid:17) ) .
Revised: + || ∆ − 1 N ∆ || 2 σ (cid:17) ) .

Original: In the cases involving rotation invariance, this results in bounds of the form
Revised: In the cases involving rotation invariance, this results in bounds of the form

Original: [Equation]
Revised: [Equation]

Original: with κ ∈ R such that
Revised: with κ ∈ R such that

Original: [Equation]
Revised: [Equation]

Original: i.e. the same certificates, but with “ ≥ κ ” instead of “ ≤ κ ”.
Revised: i.e. the same certificates, but with “ ≥ κ ” instead of “ ≤ κ ”.

Original: Our Monte Carlo certification procedure proposed in Appendix F.5 can also easily be adapted to computing an upper bound by replacing " ≤ " with " ≥ " and vice-versa.:
Revised: Our Monte Carlo certification procedure proposed in Appendix F.5 can also easily be adapted to computing an upper bound by replacing " ≤ " with " ≥ " and vice-versa.:

Original: [Table]
Revised: [Table]

Original: Note that we still need to lower-bound threshold κ , because this increases the probability
Revised: Note that we still need to lower-bound threshold κ , because this increases the probability

Original: Pr V ∼ υ (1) [ ρ ( V ) ≥ t ] and thus leads to more pessimistic, sound multi-class certificates.
Revised: Pr V ∼ υ (1) [ ρ ( V ) ≥ t ] and thus leads to more pessimistic, sound multi-class certificates.

Original: Finally note that, because the multi-class certificates are just a combination of two binary certificates, our evaluation of binary invariance-aware certificates in in Section 8 is a good indicator of the performance of invariance-aware multi-class certificates.
Revised: Finally note that, because the multi-class certificates are just a combination of two binary certificates, our evaluation of binary invariance-aware certificates in in Section 8 is a good indicator of the performance of invariance-aware multi-class certificates.

Original: I Inverse certificates
Revised: I Inverse certificates

Original: In Section 8.1, we evaluate inverse certificates, i.e. compute the smallest prediction probability p min for which robustness can still be certified, given the remaining certificate parameters such as || ∆ || 2 or || X
Revised: In Section 8.1, we evaluate inverse certificates, i.e. compute the smallest prediction probability p min for which robustness can still be certified, given the remaining certificate parameters such as || ∆ || 2 or || X

Original: || 2 .
Revised: || 2 .

Original: In the case of the black-box randomized smoothing baseline, p min can be calculated by solving Φ (cid:16) Φ ( − 1) ( p ) − || ∆ || 2 σ (cid:17)
Revised: In the case of the black-box randomized smoothing baseline, p min can be calculated by solving Φ (cid:16) Φ ( − 1) ( p ) − || ∆ || 2 σ (cid:17)

Original: =
Revised: =

Original: 12 for p , which results in p min =
Revised: 12 for p , which results in p min =

Original: Φ (cid:16) ||
Revised: Φ (cid:16) ||

Original: ∆ || 2 σ (cid:17) .
Revised: ∆ || 2 σ (cid:17) .

Original: Similarly, the gray-box certificates for translation invariance has p min =
Revised: Similarly, the gray-box certificates for translation invariance has p min =

Original: Φ (cid:16) || ∆ − 1 N ∆ || 2 σ (cid:17) , where ∆ ∈ R 1 × D are the column-wise averages.
Revised: Φ (cid:16) || ∆ − 1 N ∆ || 2 σ (cid:17) , where ∆ ∈ R 1 × D are the column-wise averages.

Original: The tight gray-box certificates involving rotation invariance do not have closed-form analytic expressions.
Revised: The tight gray-box certificates involving rotation invariance do not have closed-form analytic expressions.

Original: Instead, they are of the form
Revised: Instead, they are of the form

Original: [Equation]
Revised: [Equation]

Original: with κ ∈ R such that
Revised: with κ ∈ R such that

Original: [Equation]
Revised: [Equation]

Original: Here, an inverse certificate can be computed by first finding a threshold κ such that
Revised: Here, an inverse certificate can be computed by first finding a threshold κ such that

Original: Pr V ∼ υ (1) [ ρ ( V ) ≤ κ ] = 12 and then evaluating Pr V ∼ υ (2) [ ρ ( V ) ≤ κ ] .
Revised: Pr V ∼ υ (1) [ ρ ( V ) ≤ κ ] = 12 and then evaluating Pr V ∼ υ (2) [ ρ ( V ) ≤ κ ] .

Original: That is,
Revised: That is,

Original: [Equation]
Revised: [Equation]

Original: with κ ∈ R such that
Revised: with κ ∈ R such that

Original: [Equation]
Revised: [Equation]

Original: To ensure a fair comparison with our baselines, we modify the Monte Carlo certification procedure proposed in Appendix F.5 to compute a probabilistic upper bound on p min that holds with high probability 1 − α .
Revised: To ensure a fair comparison with our baselines, we modify the Monte Carlo certification procedure proposed in Appendix F.5 to compute a probabilistic upper bound on p min that holds with high probability 1 − α .

Original: function I NVERSE P ROB C ERTIFY
Revised: function I NVERSE P ROB C ERTIFY

Original: ( υ (1) , υ (2) , N 1 , N 2 , α ) n
Revised: ( υ (1) , υ (2) , N 1 , N 2 , α ) n

Original: [Equation]
Revised: [Equation]

Original: ← min { n | B IN PV ALUE ( n, N 1 , ≤ , 0 . 5) < α }
Revised: ← min { n | B IN PV ALUE ( n, N 1 , ≤ , 0 . 5) < α }

Original: V (1) , . .
Revised: V (1) , . .

Original: , V ( N 2 ) ← S AMPLE ( υ (1) , N 1 )
Revised: , V ( N 2 ) ← S AMPLE ( υ (1) , N 1 )

Original: R (1) , . . .
Revised: R (1) , . . .

Original: , R ( N 2 ) ← S ORT A SCENDING ( ρ ( V (1) ) , . .
Revised: , R ( N 2 ) ← S ORT A SCENDING ( ρ ( V (1) ) , . .

Original: , ρ ( V ( N 2 ) ) ) κ ← R ( n ∗ )
Revised: , ρ ( V ( N 2 ) ) ) κ ← R ( n ∗ )

Original: ▷ First bound
Revised: ▷ First bound

Original: V (1) , . .
Revised: V (1) , . .

Original: , V ( N 2 ) ← S AMPLE ( υ (2) , N 3 ) count 2 ← (cid:80) N 2 n =1 1 [ ρ ( V ) ≤ κ ] return U PPER C ONF B OUND ( count 2 , N 2 ,− α 2 )
Revised: , V ( N 2 ) ← S AMPLE ( υ (2) , N 3 ) count 2 ← (cid:80) N 2 n =1 1 [ ρ ( V ) ≤ κ ] return U PPER C ONF B OUND ( count 2 , N 2 ,− α 2 )

Original: ▷ Second bound
Revised: ▷ Second bound

Original: Note that, different from Appendix F.5 we compute a probabilistic upper bound on threshold κ by finding the smallest order statistic R ( n ) such that (cid:2) ρ ( V ) ≤ R ( n ) (cid:3) ≥ 12 holds with high probability.
Revised: Note that, different from Appendix F.5 we compute a probabilistic upper bound on threshold κ by finding the smallest order statistic R ( n ) such that (cid:2) ρ ( V ) ≤ R ( n ) (cid:3) ≥ 12 holds with high probability.

Original: J Parameter space of the tight certificate for rotation invariance in 2D
Revised: J Parameter space of the tight certificate for rotation invariance in 2D

Original: Our tight certificates for 2D rotation invariance depend on || X
Revised: Our tight certificates for 2D rotation invariance depend on || X

Original: || 2 , || ∆ || 2 , as well as parameters ϵ 1 = ⟨ X , ∆ ⟩ F , ϵ 2 = (cid:68) XR (cid:0) − π 2 (cid:1) T , ∆ (cid:69) F , which capture the orientation of the perturbed pointcloud, relative to the clean point cloud.
Revised: || 2 , || ∆ || 2 , as well as parameters ϵ 1 = ⟨ X , ∆ ⟩ F , ϵ 2 = (cid:68) XR (cid:0) − π 2 (cid:1) T , ∆ (cid:69) F , which capture the orientation of the perturbed pointcloud, relative to the clean point cloud.

Original: In this section, we determine the feasible range of ϵ 1 and ϵ 2 and calculate the two pairs of values corresponding to adversarial rotations, which is relevant for our experiments in Section 8.1.
Revised: In this section, we determine the feasible range of ϵ 1 and ϵ 2 and calculate the two pairs of values corresponding to adversarial rotations, which is relevant for our experiments in Section 8.1.

Original: J.1 Feasible parameter range
Revised: J.1 Feasible parameter range

Original: Let ˆ X =
Revised: Let ˆ X =

Original: X
Revised: X

Original: ||
Revised: ||

Original: X
Revised: X

Original: || 2 .
Revised: || 2 .

Original: Then, parameters ϵ 1 and ϵ 2 can be equivalently stated as follows:
Revised: Then, parameters ϵ 1 and ϵ 2 can be equivalently stated as follows:

Original: [Equation]
Revised: [Equation]

Original: Note that vec( ˆ X ) and vec (cid:16) ˆ XR (cid:0) − π 2 (cid:1) T (cid:17) have norm 1 and are orthogonal to each other.
Revised: Note that vec( ˆ X ) and vec (cid:16) ˆ XR (cid:0) − π 2 (cid:1) T (cid:17) have norm 1 and are orthogonal to each other.

Original: Thus, they are the first two elements of an orthonormal basis of R 2 N .
Revised: Thus, they are the first two elements of an orthonormal basis of R 2 N .

Original: Parameters ϵ 1 and ϵ 2 are projections of vec( || X || 2 ∆ ) onto the first two elements of this basis.
Revised: Parameters ϵ 1 and ϵ 2 are projections of vec( || X || 2 ∆ ) onto the first two elements of this basis.

Original: Basis changes preserve the norm of vectors.
Revised: Basis changes preserve the norm of vectors.

Original: Thus, the values ϵ 1 , ϵ 2 must fulfill (cid:112) ϵ 21 + ϵ 22 ≤
Revised: Thus, the values ϵ 1 , ϵ 2 must fulfill (cid:112) ϵ 21 + ϵ 22 ≤

Original: ||
Revised: ||

Original: X
Revised: X

Original: || 2 · || ∆ || 2 .
Revised: || 2 · || ∆ || 2 .

Original: J.2 Adversarial rotations
Revised: J.2 Adversarial rotations

Original: Known rotation angle.
Revised: Known rotation angle.

Original: Consider an arbitrary X ∈ R N × 2 .
Revised: Consider an arbitrary X ∈ R N × 2 .

Original: Further consider an adversarially perturbed X ′ =
Revised: Further consider an adversarially perturbed X ′ =

Original: XR ( θ ) T that is the result of rotating all rows of X by angle θ .
Revised: XR ( θ ) T that is the result of rotating all rows of X by angle θ .

Original: Let ∆ =
Revised: Let ∆ =

Original: X ′ − X .
Revised: X ′ − X .

Original: Recall that, for any vector a , b ∈ R D , a T b T
Revised: Recall that, for any vector a , b ∈ R D , a T b T

Original: = ||
Revised: = ||

Original: a || 2 || b
Revised: a || 2 || b

Original: || 2 cos( ∠ ab ) .
Revised: || 2 cos( ∠ ab ) .

Original: Therefore
Revised: Therefore

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: where the last equality follows from the definition of the Frobenius norm.
Revised: where the last equality follows from the definition of the Frobenius norm.

Original: Similarly, we have
Revised: Similarly, we have

Original: [Equation]
Revised: [Equation]

Original: Unknown rotation angle.
Revised: Unknown rotation angle.

Original: Now assume that we know
Revised: Now assume that we know

Original: ||
Revised: ||

Original: X
Revised: X

Original: || 2 and || ∆
Revised: || 2 and || ∆

Original: || 2 , but do not know the rotation angle θ .
Revised: || 2 , but do not know the rotation angle θ .

Original: We have
Revised: We have

Original: [Equation]
Revised: [Equation]

Original: Evidently, such an adversarial rotation is only possible if || ∆ || ≤ 2 ||
Revised: Evidently, such an adversarial rotation is only possible if || ∆ || ≤ 2 ||

Original: X || , which corresponds to a rotation by angle π .
Revised: X || , which corresponds to a rotation by angle π .

Original: Solving for θ yields
Revised: Solving for θ yields

Original: [Equation]
Revised: [Equation]

Original: Inserting into our formulae for ϵ 1 and ϵ 2 and using that sin(arccos( a ))
Revised: Inserting into our formulae for ϵ 1 and ϵ 2 and using that sin(arccos( a ))

Original: = √ 1 − a 2 shows that
Revised: = √ 1 − a 2 shows that

Original: [Equation]
Revised: [Equation]

Original: ∆ || 22 (4 || X || 22 − || ∆ || 22 ) .
Revised: ∆ || 22 (4 || X || 22 − || ∆ || 22 ) .