Original: Before deriving the marginal distribution over this lower-dimensional space and finding the worst-caseclassifier, we need the following lemma:
Revised: Before deriving the marginal distribution over this lower-dimensional space and finding the worst-caseclassifier, we need the following lemma:

Original: Lemma 7. Consider an arbitrary vector x ∈ R D and let x = 1 D (cid:80) Dd
Revised: Lemma 7. Consider an arbitrary vector x ∈ R D and let x = 1 D (cid:80) Dd

Original: =1 x d be its average.
Revised: =1 x d be its average.

Original: Thenx
Revised: Thenx

Original: [Equation]
Revised: [Equation]

Original: x
Revised: x

Original: [Equation]
Revised: [Equation]

Original: Proof.
Revised: Proof.

Original: We substract the left-hand side from the right-hand side
Revised: We substract the left-hand side from the right-hand side

Original: [Equation]
Revised: [Equation]

Original: where the last equality follows from the fact that (cid:80) Dd =1 x d = D x .
Revised: where the last equality follows from the fact that (cid:80) Dd =1 x d = D x .

Original: Further recall the following lemma from Appendix C.3:
Revised: Further recall the following lemma from Appendix C.3:

Original: Lemma 1. Consider an arbitrary invertible matrix A ∈ R N × N , vectors z , m ∈ R N and scalarσ > 0 .
Revised: Lemma 1. Consider an arbitrary invertible matrix A ∈ R N × N , vectors z , m ∈ R N and scalarσ > 0 .

Original: Then
Revised: Then

Original: [Equation]
Revised: [Equation]

Original: We begin our proof of Theorem 5 by reformulating the canonical map γ from Lemma 6 as γ ( Z ) =
Revised: We begin our proof of Theorem 5 by reformulating the canonical map γ from Lemma 6 as γ ( Z ) =

Original: KZ with K =
Revised: KZ with K =

Original: [Equation]
Revised: [Equation]

Original: The multiplication KZ sets the first row of Z to zero andsubtracts the first row from all other rows, just like our originally defined γ .
Revised: The multiplication KZ sets the first row of Z to zero andsubtracts the first row from all other rows, just like our originally defined γ .

Original: Applying Theorem 4 with this canonical map shows thatmin h ∈ H T
Revised: Applying Theorem 4 with this canonical map shows thatmin h ∈ H T

Original: [Equation]
Revised: [Equation]

Original: We notice that the above optimization problem does not depend on the specific class predicted by h ,but only on whether it is y ∗ or y .
Revised: We notice that the above optimization problem does not depend on the specific class predicted by h ,but only on whether it is y ∗ or y .

Original: To avoid clutter, we formulate the equivalent problem
Revised: To avoid clutter, we formulate the equivalent problem

Original: [Equation]
Revised: [Equation]

Original: where h ( Z )
Revised: where h ( Z )

Original: = 1 indicates that Z is classified as y ∗ .
Revised: = 1 indicates that Z is classified as y ∗ .

Original: Recall that µ X is an isotropic matrix normal distribution with variance σ 2 .
Revised: Recall that µ X is an isotropic matrix normal distribution with variance σ 2 .

Original: We thus have:
Revised: We thus have:

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: Substitution.
Revised: Substitution.

Original: We can now use V = AZ with A =
Revised: We can now use V = AZ with A =

Original: [Equation]
Revised: [Equation]

Original: N (cid:21) ,where 1 N ∈ { 1 } N − 1 is a vector that has value 1 everywhere
Revised: N (cid:21) ,where 1 N ∈ { 1 } N − 1 is a vector that has value 1 everywhere

Original: and I N − 1 ∈ { 0 , 1 } ( N − 1) × ( N − 1) is theidentity matrix.
Revised: and I N − 1 ∈ { 0 , 1 } ( N − 1) × ( N − 1) is theidentity matrix.

Original: [Equation]
Revised: [Equation]

Original: The second equality follows from applying Lemma 1 D times.
Revised: The second equality follows from applying Lemma 1 D times.

Original: Marginalization.
Revised: Marginalization.

Original: Note that KA − 1 =
Revised: Note that KA − 1 =

Original: (cid:20)
Revised: (cid:20)

Original: [Equation]
Revised: [Equation]

Original: Since the first column is 0 everywhere,h ( KA − 1 V ) does not depend on the first row of V , meaning we can marginalize out V 1 , 1 , . .
Revised: Since the first column is 0 everywhere,h ( KA − 1 V ) does not depend on the first row of V , meaning we can marginalize out V 1 , 1 , . .

Original: , V 1 ,D .
Revised: , V 1 ,D .

Original: For multivariate normal distributions, this is equivalent to dropping the first row of the mean and thefirst row and column of the covariance matrix.
Revised: For multivariate normal distributions, this is equivalent to dropping the first row of the mean and thefirst row and column of the covariance matrix.

Original: In the following, we use A 2: ∈ R ( N − 1)
Revised: In the following, we use A 2: ∈ R ( N − 1)

Original: × D to denotethe remaining N − 1 rows of A .
Revised: × D to denotethe remaining N − 1 rows of A .

Original: [Equation]
Revised: [Equation]

Original: Let ν X be this (( N − 1) × D ) -dimensional marginal distribution with density ν X ( W ) =(cid:81)
Revised: Let ν X be this (( N − 1) × D ) -dimensional marginal distribution with density ν X ( W ) =(cid:81)

Original: Dd =1 N (cid:16) W : ,d |
Revised: Dd =1 N (cid:16) W : ,d |

Original: A 2: X : ,d , σ 2 A 2: ( A 2: ) T (cid:17) .
Revised: A 2: X : ,d , σ 2 A 2: ( A 2: ) T (cid:17) .

Original: We may then rewrite Eq.
Revised: We may then rewrite Eq.

Original: (12) as
Revised: (12) as

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: Importantly, note that Eq.
Revised: Importantly, note that Eq.

Original: (16) only depends on the function value of h evaluated for matrices whosefirst row is zero.
Revised: (16) only depends on the function value of h evaluated for matrices whosefirst row is zero.

Original: To use the family of variables analogy discussed in Appendix D.2, Eq.
Revised: To use the family of variables analogy discussed in Appendix D.2, Eq.

Original: (16) onlydepends on a subset of variables indexed by R ( N − 1)
Revised: (16) onlydepends on a subset of variables indexed by R ( N − 1)

Original: × D .
Revised: × D .

Original: We may thus equivalently solve
Revised: We may thus equivalently solve

Original: [Equation]
Revised: [Equation]

Original: Solving the lower-dimensional problem.
Revised: Solving the lower-dimensional problem.

Original: At this point, we could apply the Neyman-Pearson lemmato exactly solve Eq.
Revised: At this point, we could apply the Neyman-Pearson lemmato exactly solve Eq.

Original: However, the exact solution has already been derived in prior work.
Revised: However, the exact solution has already been derived in prior work.

Original: Notethat the density function of ν X is just the product of multiple multivariate normal density functions.
Revised: Notethat the density function of ν X is just the product of multiple multivariate normal density functions.

Original: Itcan be expressed as a single multivariate normal distribution with a block-diagonal covariance matrix.
Revised: Itcan be expressed as a single multivariate normal distribution with a block-diagonal covariance matrix.

Original: If W ∼ ν X , then vec( W ) ∼ N ( m X , Σ ) with m X ∈ R
Revised: If W ∼ ν X , then vec( W ) ∼ N ( m X , Σ ) with m X ∈ R

Original: ( N − 1) D and Σ ∈ R ( N − 1) D × ( N − 1) D + ,wherem X
Revised: ( N − 1) D and Σ ∈ R ( N − 1) D × ( N − 1) D + ,wherem X

Original: = vec ( A 2: X ) , Σ = 
Revised: = vec ( A 2: X ) , Σ = 

Original: 
Revised: 

Original: B . . . B . . .
Revised: B . . . B . . .

Original: B   with B =
Revised: B   with B =

Original: σ 2 A 2: ( A 2: ) T .
Revised: σ 2 A 2: ( A 2: ) T .

Original: Since the function vec − 1 : R ( N − 1) D → R ( N − 1)
Revised: Since the function vec − 1 : R ( N − 1) D → R ( N − 1)

Original: × D that de-vectorizesvec( W ) is injective, we may restate our optimization problem from Eq.
Revised: × D that de-vectorizesvec( W ) is injective, we may restate our optimization problem from Eq.

Original: (17) as an optimizationproblem over a family of variables indexed by R ( N − 1) D :
Revised: (17) as an optimizationproblem over a family of variables indexed by R ( N − 1) D :

Original: [Equation]
Revised: [Equation]

Original: Eq.
Revised: Eq.

Original: (18) is the problem of finding a tight certificate for a classifier smoothed with anisotropic Gaussiannoise.
Revised: (18) is the problem of finding a tight certificate for a classifier smoothed with anisotropic Gaussiannoise.

Original: Its optimal value has been derived in prior work (see [129] and Appendix A of [91]) and is
Revised: Its optimal value has been derived in prior work (see [129] and Appendix A of [91]) and is

Original: [Equation]
Revised: [Equation]

Original: We can conclude our proof by showing that
Revised: We can conclude our proof by showing that

Original: [Equation]
Revised: [Equation]

Original: First, we use the fact that the inverse of a block-diagonal matrix is also a block-diagonal matrix andinsert the definitions of m X ′ , m X , Σ and B to show that:
Revised: First, we use the fact that the inverse of a block-diagonal matrix is also a block-diagonal matrix andinsert the definitions of m X ′ , m X , Σ and B to show that:

Original: [Equation]
Revised: [Equation]

Original: where in the second to last last equation we have used that X ′
Revised: where in the second to last last equation we have used that X ′

Original: =
Revised: =

Original: X + ∆ .
Revised: X + ∆ .

Original: Our next step is to compute ( A 2: ) T (cid:16)
Revised: Our next step is to compute ( A 2: ) T (cid:16)

Original: A 2: ( A 2: ) T (cid:17) − 1 A 2: .
Revised: A 2: ( A 2: ) T (cid:17) − 1 A 2: .

Original: Recall that A 2: ∈ R
Revised: Recall that A 2: ∈ R

Original: ( N − 1) × D and A 2: =[ − 1 N −
Revised: ( N − 1) × D and A 2: =[ − 1 N −

Original: I N − 1 ] .
Revised: I N − 1 ] .

Original: Matrix multiplication shows that
Revised: Matrix multiplication shows that

Original: [Equation]
Revised: [Equation]

Original: The matrix is sufficiently simple to be inverted by inspection:
Revised: The matrix is sufficiently simple to be inverted by inspection:

Original: [Equation]
Revised: [Equation]

Original: Finally, matrix multiplication shows that
Revised: Finally, matrix multiplication shows that

Original: [Equation]
Revised: [Equation]

Original: Note that Eq.
Revised: Note that Eq.

Original: (26) transforms a vector by subtracting its average from all entries.
Revised: (26) transforms a vector by subtracting its average from all entries.

Original: Inserting into Eq.
Revised: Inserting into Eq.

Original: (25)shows that
Revised: (25)shows that

Original: [Equation]
Revised: [Equation]

Original: where ∆ : ,d ∈ R is the average of column ∆ : ,d , ∆ ∈ R 1 × D are the column-wise averages of matrix∆ , the second equality is due to Lemma 7 and the third equality uses the definition of inner products.
Revised: where ∆ : ,d ∈ R is the average of column ∆ : ,d , ∆ ∈ R 1 × D are the column-wise averages of matrix∆ , the second equality is due to Lemma 7 and the third equality uses the definition of inner products.

Original: Taking the square root and using the definition of the Frobenius norm yields our desired result:
Revised: Taking the square root and using the definition of the Frobenius norm yields our desired result:

Original: [Equation]
Revised: [Equation]

Original: D.5 Tight certificate for invariance under rotation in 2D
Revised: D.5 Tight certificate for invariance under rotation in 2D

Original: The following theorem is our tight certificate for rotation invariance in 2D, i.e. invariance under 1286 T = (cid:8) X
Revised: The following theorem is our tight certificate for rotation invariance in 2D, i.e. invariance under 1286 T = (cid:8) X

Original: (cid:55)→ XR T | R ∈ SO (2) (cid:9) .
Revised: (cid:55)→ XR T | R ∈ SO (2) (cid:9) .

Original: Different from the formulation in in Section 6.3, we also specifyits exact dependence on the four parameters
Revised: Different from the formulation in in Section 6.3, we also specifyits exact dependence on the four parameters

Original: ||
Revised: ||

Original: X
Revised: X

Original: || 2 , ||
Revised: || 2 , ||

Original: ∆ || , ϵ 1 and ϵ 2 :
Revised: ∆ || , ϵ 1 and ϵ 2 :

Original: Theorem 7.
Revised: Theorem 7.

Original: Let g : R N × 2 → Y be rotation invariant and H T be defined as in Eq.
Revised: Let g : R N × 2 → Y be rotation invariant and H T be defined as in Eq.

Original: Then
Revised: Then

Original: [Equation]
Revised: [Equation]

Original: where I 0 is the modified Bessel function of the first kind and order 0 , and
Revised: where I 0 is the modified Bessel function of the first kind and order 0 , and

Original: [Equation]
Revised: [Equation]

Original: ϵ 1 + || X
Revised: ϵ 1 + || X

Original: ||
Revised: ||

Original: ||
Revised: ||

Original: X
Revised: X

Original: || 22 .
Revised: || 22 .

Original: 
Revised: 

Original:  , with ϵ 1 = ⟨ X , ∆ ⟩ F , ϵ 2 = (cid:68) XR (cid:0) − π 2 (cid:1) T , ∆ (cid:69) F .
Revised:  , with ϵ 1 = ⟨ X , ∆ ⟩ F , ϵ 2 = (cid:68) XR (cid:0) − π 2 (cid:1) T , ∆ (cid:69) F .

Original: Here, ⟨ A , B ⟩ F = (cid:80) Nn =

Revised: Here, ⟨ A , B ⟩ F = (cid:80) Nn =

Original: A Tn B Tn = vec( A ) T vec( B ) is the Frobenius inner product.
Revised: A Tn B Tn = vec( A ) T vec( B ) is the Frobenius inner product.

Original: Following our certification methodology, we present a canonical map for invariance under rotation in2D and then use it to derive our certificate via substitutions, marginalization and application of the
Revised: Following our certification methodology, we present a canonical map for invariance under rotation in2D and then use it to derive our certificate via substitutions, marginalization and application of the

Original: Neyman-Pearson lemma.
Revised: Neyman-Pearson lemma.

Original: We additionally demonstrate that the worst-case invariant classifier is a
Revised: We additionally demonstrate that the worst-case invariant classifier is a

Original: Gaussian Haar integration kernel method (see Appendix D.5.3).
Revised: Gaussian Haar integration kernel method (see Appendix D.5.3).

Original: D.5.1 Canonical map
Revised: D.5.1 Canonical map

Original: For our derivation, recall the definition of canonical maps.
Revised: For our derivation, recall the definition of canonical maps.

Original: Definition 3 (Canonical map) .
Revised: Definition 3 (Canonical map) .

Original: A canonical map for invariance under a group of transformations Tis a function γ : R N × D
Revised: A canonical map for invariance under a group of transformations Tis a function γ : R N × D

Original: → R N × D with
Revised: → R N × D with

Original: [Equation]
Revised: [Equation]

Original: For rotation invariance, it is more convenient to work with polar coordinates.
Revised: For rotation invariance, it is more convenient to work with polar coordinates.

Original: In the following, letλ : R N × R N → R N × 2 with λ ( r , θ ) =
Revised: In the following, letλ : R N × R N → R N × 2 with λ ( r , θ ) =

Original: [ r ⊙ cos( θ ) r ⊙ sin( θ )] be the function that maps frompolar coordinates into Cartesian coordiantes.
Revised: [ r ⊙ cos( θ ) r ⊙ sin( θ )] be the function that maps frompolar coordinates into Cartesian coordiantes.

Original: Here, ⊙ is the elementwise product and cos and sin areapplied element-wise.
Revised: Here, ⊙ is the elementwise product and cos and sin areapplied element-wise.

Original: First, we define a rotation operator for matrices:
Revised: First, we define a rotation operator for matrices:

Original: Definition 4.
Revised: Definition 4.

Original: The matrix rotation operator is the function rot : R N × 2 × R → R N × 2 withrot
Revised: The matrix rotation operator is the function rot : R N × 2 × R → R N × 2 withrot

Original: ( Z , a ) =
Revised: ( Z , a ) =

Original: Z (cid:20) cos( a ) − sin( a ) sin( a ) cos( a ) (cid:21) T that rotates all rows of matrix
Revised: Z (cid:20) cos( a ) − sin( a ) sin( a ) cos( a ) (cid:21) T that rotates all rows of matrix

Original: Z by angle a .
Revised: Z by angle a .

Original: We use the matrix rotation operator to define our canonical map, which rotates all matrices Z suchthat their first non-zero row is aligned with the first coordinate axis, i.e. takes on value 0 in the secondcolumn.
Revised: We use the matrix rotation operator to define our canonical map, which rotates all matrices Z suchthat their first non-zero row is aligned with the first coordinate axis, i.e. takes on value 0 in the secondcolumn.

Original: Lemma 8.
Revised: Lemma 8.

Original: The function γ ( Z )
Revised: The function γ ( Z )

Original: = rot ( Z , − α
Revised: = rot ( Z , − α

Original: ( Z ))
Revised: ( Z ))

Original: with
Revised: with

Original: [Equation]
Revised: [Equation]

Original: Z
Revised: Z

Original: [Equation]
Revised: [Equation]

Original: where n ∗ = min { n | || Z n || 2 > 0 } is the first non-zero row of matrix Z , is a canonical map for 1312 rotation-invariance in 2 D.
Revised: where n ∗ = min { n | || Z n || 2 > 0 } is the first non-zero row of matrix Z , is a canonical map for 1312 rotation-invariance in 2 D.

Original: Proof.
Revised: Proof.

Original: We can distinguish two cases.
Revised: We can distinguish two cases.

Original: Case 1.
Revised: Case 1.

Original: All entries of Z are 0 , i.e. Z = 0 N, 2 .
Revised: All entries of Z are 0 , i.e. Z = 0 N, 2 .

Original: In this case ||
Revised: In this case ||

Original: Z ||
Revised: Z ||

Original: 2 = 0 and thus γ ( Z )
Revised: 2 = 0 and thus γ ( Z )

Original: =
Revised: =

Original: Z = 0 N, 2 .
Revised: Z = 0 N, 2 .

Original: The equivalence class of Z is [ Z ] T = (cid:8) 0 N, 2 R T | R ∈ SO (2) (cid:9) = { 0 N, 2 } .
Revised: The equivalence class of Z is [ Z ] T = (cid:8) 0 N, 2 R T | R ∈ SO (2) (cid:9) = { 0 N, 2 } .

Original: Both criteria Eqs.
Revised: Both criteria Eqs.

Original: (3)and (4) for canonical maps are thus fulfilled.
Revised: (3)and (4) for canonical maps are thus fulfilled.

Original: Case 2.
Revised: Case 2.

Original: The matrix Z has non-zero entries.
Revised: The matrix Z has non-zero entries.

Original: Let r , θ be a map of Z in polar coordinates, i.e. Z =λ ( r , θ ) .
Revised: Let r , θ be a map of Z in polar coordinates, i.e. Z =λ ( r , θ ) .

Original: Let n ∗ be the index of the first non-zero row of Z .
Revised: Let n ∗ be the index of the first non-zero row of Z .

Original: Recall that rotation in cartesian coordinatescorresponds to translation of the polar coordinate angles and thus γ ( Z ) =
Revised: Recall that rotation in cartesian coordinatescorresponds to translation of the polar coordinate angles and thus γ ( Z ) =

Original: λ ( r , θ − 1 N θ n ∗ ) .
Revised: λ ( r , θ − 1 N θ n ∗ ) .

Original: Criterion Eq.
Revised: Criterion Eq.

Original: (3) is fulfilled, because by construction of γ we have γ ( Z ) =
Revised: (3) is fulfilled, because by construction of γ we have γ ( Z ) =

Original: ZR T for some R ∈
Revised: ZR T for some R ∈

Original: SO (2) .
Revised: SO (2) .

Original: Now, consider an arbitrary Z ′ ∈
Revised: Now, consider an arbitrary Z ′ ∈

Original: [ Z ] T .
Revised: [ Z ] T .

Original: By definition of equivalence class [ Z ] T , there must be somea ∈ R such that Z ′
Revised: By definition of equivalence class [ Z ] T , there must be somea ∈ R such that Z ′

Original: =
Revised: =

Original: λ ( r , θ + a 1 N ) .
Revised: λ ( r , θ + a 1 N ) .

Original: Note that rotation does not affect the radii, i.e. n ∗ is also theindex of the first non-zero row of Z ′ .
Revised: Note that rotation does not affect the radii, i.e. n ∗ is also theindex of the first non-zero row of Z ′ .

Original: Thus
Revised: Thus

Original: [Equation]
Revised: [Equation]

Original: ( Z ) , meaning criterion Eq.
Revised: ( Z ) , meaning criterion Eq.

Original: (4) is fulfilled as well.
Revised: (4) is fulfilled as well.

Original: D.5.2 Proof for certificate
Revised: D.5.2 Proof for certificate

Original: Our canonical map γ ( Z ) rotates all rows of an input Z ∈ R N × D such that the first non-zero rowis aligned with the first coordinate-axis.
Revised: Our canonical map γ ( Z ) rotates all rows of an input Z ∈ R N × D such that the first non-zero rowis aligned with the first coordinate-axis.

Original: Similar to our proof for translation invariance, we will usesubstitutions to state our problem in an alternative parameter space that specifies the angle θ 1 of thefirst non-zero row and all angle differences to this row, i.e. θ n − θ 1 , as well as the radii r ∈ R N + .
Revised: Similar to our proof for translation invariance, we will usesubstitutions to state our problem in an alternative parameter space that specifies the angle θ 1 of thefirst non-zero row and all angle differences to this row, i.e. θ n − θ 1 , as well as the radii r ∈ R N + .

Original: Wethen find the marginal distribution over the representatives in this alternative parameter space and applythe Neyman-Pearson lemma.
Revised: Wethen find the marginal distribution over the representatives in this alternative parameter space and applythe Neyman-Pearson lemma.

Original: Finally, we transform our solution back into Cartesian coordinates anduse the behavior of multivariate Normal distributions under affine transformations to prove our result.
Revised: Finally, we transform our solution back into Cartesian coordinates anduse the behavior of multivariate Normal distributions under affine transformations to prove our result.

Original: Because the calculations for deriving the marginal distribution are somewhat more arduous, we shallhandle them separately in Lemma 9, as to interfere with the rest of the proof.
Revised: Because the calculations for deriving the marginal distribution are somewhat more arduous, we shallhandle them separately in Lemma 9, as to interfere with the rest of the proof.

Original: Lemma 9.
Revised: Lemma 9.

Original: Consider two multivariate normal distributions
Revised: Consider two multivariate normal distributions

Original: N ( M : , 1 , Σ ) , N ( M : , 2 , Σ ) with M ∈ 1337 R N × 2 that share the same symmetric covariance matrix Σ −∈ R N × D .
Revised: N ( M : , 1 , Σ ) , N ( M : , 2 , Σ ) with M ∈ 1337 R N × 2 that share the same symmetric covariance matrix Σ −∈ R N × D .

Original: Further consider arbitraryϕ ∈ R N and r ∈ R N .
Revised: Further consider arbitraryϕ ∈ R N and r ∈ R N .

Original: Define ξ : R × R N → R N with ξ ( α, ϕ ) =
Revised: Define ξ : R × R N → R N with ξ ( α, ϕ ) =

Original: ϕ
Revised: ϕ

Original: + 1 N α .
Revised: + 1 N α .

Original: Then
Revised: Then

Original: [Equation]
Revised: [Equation]

Original: with κ ( Z , M , Σ ) =
Revised: with κ ( Z , M , Σ ) =

Original: (cid:20)
Revised: (cid:20)

Original: [Equation]
Revised: [Equation]

Original: where cos and sin are applied element-wise, ⊙ is the element-wise product and ∝ absorbs factorsthat are constant in M .
Revised: where cos and sin are applied element-wise, ⊙ is the element-wise product and ∝ absorbs factorsthat are constant in M .

Original: Note that w ∈ R 2 and that I 0 is the modified Bessel function of the first kindand order 0 .
Revised: Note that w ∈ R 2 and that I 0 is the modified Bessel function of the first kindand order 0 .

Original: Proof.
Revised: Proof.

Original: We begin by identifying terms that that are constant in both M and α and can thus beeliminated, as well as terms that are constant in α and can thus be moved outside the integral.
Revised: We begin by identifying terms that that are constant in both M and α and can thus beeliminated, as well as terms that are constant in α and can thus be moved outside the integral.

Original: Wefirst eliminate the normalization constants:
Revised: Wefirst eliminate the normalization constants:

Original: [Equation]
Revised: [Equation]

Original: with
Revised: with

Original: [Equation]
Revised: [Equation]

Original: Note that cand ccorrespond to −
Revised: Note that cand ccorrespond to −

Original: [Equation]
Revised: [Equation]

Original: ϕ
Revised: ϕ

Original: [Equation]
Revised: [Equation]

Original: Σ
Revised: Σ

Original: [Equation]
Revised: [Equation]

Original: , ϕ ))
Revised: , ϕ ))

Original: and− 12 sin ( ξ ( α, ϕ ))
Revised: and− 12 sin ( ξ ( α, ϕ ))

Original: T Σ − 1 sin ( ξ ( α, ϕ )) , respectively.c 1 is constant in α and can thus be moved outside the integral.c 2 and c 3 are not constant in α and must therefore remain inside the integral.c 4 + c 5 can be shown to be constant w.r.t.
Revised: T Σ − 1 sin ( ξ ( α, ϕ )) , respectively.c 1 is constant in α and can thus be moved outside the integral.c 2 and c 3 are not constant in α and must therefore remain inside the integral.c 4 + c 5 can be shown to be constant w.r.t.

Original: both M and α and can thus be eliminated.
Revised: both M and α and can thus be eliminated.

Original: By thetrigonometric identity cos( a ) cos( b ) + sin( a ) sin( b ) = cos( a − b ) , we have
Revised: By thetrigonometric identity cos( a ) cos( b ) + sin( a ) sin( b ) = cos( a − b ) , we have

Original: [Equation]
Revised: [Equation]

Original: Recall that ξ ( α, ϕ ) =
Revised: Recall that ξ ( α, ϕ ) =

Original: ϕ
Revised: ϕ

Original: + 1 N α , i.e. “ + α ” appears in every row.
Revised: + 1 N α , i.e. “ + α ” appears in every row.

Original: Subtraction thus eliminates α forall combinations of n and m .
Revised: Subtraction thus eliminates α forall combinations of n and m .

Original: Based on these observations, we may conclude that
Revised: Based on these observations, we may conclude that

Original: [Equation]
Revised: [Equation]

Original: Next, we use the trigonometric identites cos( a + b ) = cos( a ) cos( b ) − sin( a ) sin( b ) and sin( a + b ) =sin( a ) cos( b ) + cos( a ) sin( b ) to move α outside the inner products of c 2 and c 3 :
Revised: Next, we use the trigonometric identites cos( a + b ) = cos( a ) cos( b ) − sin( a ) sin( b ) and sin( a + b ) =sin( a ) cos( b ) + cos( a ) sin( b ) to move α outside the inner products of c 2 and c 3 :

Original: [Equation]
Revised: [Equation]

Original: (32) and
Revised: (32) and

Original: [Equation]
Revised: [Equation]

Original: Stacking the Σ − 1 M terms, we can express c 2 + c 3 as follows:
Revised: Stacking the Σ − 1 M terms, we can express c 2 + c 3 as follows:

Original: [Equation]
Revised: [Equation]

Original: We can now substitute our results into Eq.
Revised: We can now substitute our results into Eq.

Original: (31) and simplify the integral using the fact that cos( α ) η 1 +sin( α ) η 2 = cos( α + ω ) (cid:112) η 21 + η 22 ) for some ω ∈ [0 , 2 π ] (see also Eq.
Revised: (31) and simplify the integral using the fact that cos( α ) η 1 +sin( α ) η 2 = cos( α + ω ) (cid:112) η 21 + η 22 ) for some ω ∈ [0 , 2 π ] (see also Eq.

Original: of [130]).
Revised: of [130]).

Original: [Equation]
Revised: [Equation]

Original: where the second to last equality is due to the fact that we integrate over an entire period of cos andthe last equality uses the integral map of the modified Bessel function of the first kind with order
Revised: where the second to last equality is due to the fact that we integrate over an entire period of cos andthe last equality uses the integral map of the modified Bessel function of the first kind with order

Original: 0(see Eq.
Revised: 0(see Eq.

Original: 10.32.1 of [131]):
Revised: 10.32.1 of [131]):

Original: [Equation]
Revised: [Equation]

Original: Restating (cid:112) η 21 + η 22 ) as
Revised: Restating (cid:112) η 21 + η 22 ) as

Original: [Equation]
Revised: [Equation]

Original: concludes our proof.
Revised: concludes our proof.

Original: Applying the canonical map.
Revised: Applying the canonical map.

Original: Now, we can turn to deriving the certificate itself.
Revised: Now, we can turn to deriving the certificate itself.

Original: Applying Theorem 4with our canonical map from Lemma 8 shows thatmin
Revised: Applying Theorem 4with our canonical map from Lemma 8 shows thatmin

Original: [Equation]
Revised: [Equation]

Original: H T
Revised: H T

Original: [Equation]
Revised: [Equation]

Original: Like in our previous proof, we notice that the above optimization problem does not depend on thespecific class predicted by h , but only on whether it is y ∗ or y .
Revised: Like in our previous proof, we notice that the above optimization problem does not depend on thespecific class predicted by h , but only on whether it is y ∗ or y .

Original: We thus formulate the equivalentproblem
Revised: We thus formulate the equivalentproblem

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: where h ( Z )
Revised: where h ( Z )

Original: = 1 indicates that Z is classified as y ∗ .
Revised: = 1 indicates that Z is classified as y ∗ .

Original: Substitution.
Revised: Substitution.

Original: We begin by going from Cartesian to polar coordinates: As defined in the previoussection, let λ : R N × R N → R N × 2 with λ ( r , θ ) =
Revised: We begin by going from Cartesian to polar coordinates: As defined in the previoussection, let λ : R N × R N → R N × 2 with λ ( r , θ ) =

Original: [ r ⊙ cos( θ ) r ⊙ sin( θ )] be the function thatmaps from polar coordinates into cartesian coordiantes.
Revised: [ r ⊙ cos( θ ) r ⊙ sin( θ )] be the function thatmaps from polar coordinates into cartesian coordiantes.

Original: [Equation]
Revised: [Equation]

Original: The r n in the last equality are due to the fact that λ transforms N rows from polar to Cartesiancoordinates, with each transformation having a Jacobian determinant r .
Revised: The r n in the last equality are due to the fact that λ transforms N rows from polar to Cartesiancoordinates, with each transformation having a Jacobian determinant r .

Original: Now, we would like to actually apply our canonical map γ , which performs a translation of the anglesθ .
Revised: Now, we would like to actually apply our canonical map γ , which performs a translation of the anglesθ .

Original: Recall from Lemma 8 that γ requires a case distinction to find the first non-zero row of λ ( r , θ ) .
Revised: Recall from Lemma 8 that γ requires a case distinction to find the first non-zero row of λ ( r , θ ) .

Original: However, due to the factor (cid:81) Nn =1 r n , the entire integrand becomes zero whenever any row of λ ( r , θ )is zero.
Revised: However, due to the factor (cid:81) Nn =1 r n , the entire integrand becomes zero whenever any row of λ ( r , θ )is zero.

Original: We can therefore always assume that we are in the case with r 1 > 0 , i.e. rotate by θ 1 , withoutinfluencing the value of the integral.
Revised: We can therefore always assume that we are in the case with r 1 > 0 , i.e. rotate by θ 1 , withoutinfluencing the value of the integral.

Original: Thus
Revised: Thus

Original: [Equation]
Revised: [Equation]

Original: Next, we apply a similar substitution to the one used in our translation certificate: Let ψ =
Revised: Next, we apply a similar substitution to the one used in our translation certificate: Let ψ =

Original: Aθwith A =
Revised: Aθwith A =

Original: (cid:20) 

Revised: (cid:20)

Original: TN
Revised: TN

Original: [Equation]
Revised: [Equation]

Original: −
Revised: −

Original: [Equation]
Revised: [Equation]

Original: N
Revised: N

Original: [Equation]
Revised: [Equation]

Original: I N −(cid:21) and A − 1 = (cid:20)0 N − 1 1 N −
Revised: I N −(cid:21) and A − 1 = (cid:20)0 N − 1 1 N −

Original: I N (cid:21) .
Revised: I N (cid:21) .

Original: Note that A − 1 ( ψ − 1 N ψ 1 ) =
Revised: Note that A − 1 ( ψ − 1 N ψ 1 ) =

Original: (cid:20)ψ 2: (cid:21) ,where ψ 2: are the last N − 1 entries of ψ .
Revised: (cid:20)ψ 2: (cid:21) ,where ψ 2: are the last N − 1 entries of ψ .

Original: In other words: Applying canonical map γ is equivalentto setting ψ 1 to 0 .
Revised: In other words: Applying canonical map γ is equivalentto setting ψ 1 to 0 .

Original: Further note that det (cid:0) A − 1 (cid:1) =
Revised: Further note that det (cid:0) A − 1 (cid:1) =

Original: and that we have to adjust the integrationboundaries, because we are now integrating over angle differences.
Revised: and that we have to adjust the integrationboundaries, because we are now integrating over angle differences.

Original: Thus
Revised: Thus

Original: [Equation]
Revised: [Equation]

Original: Marginalization.
Revised: Marginalization.

Original: The function λ that maps from polar into Cartesian coordinates is 2 π -periodicw.r.t.
Revised: The function λ that maps from polar into Cartesian coordinates is 2 π -periodicw.r.t.

Original: its second argument and, by definition, we have A −
Revised: its second argument and, by definition, we have A −

Original: [Equation]
Revised: [Equation]

Original: (cid:20)
Revised: (cid:20)

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: +
Revised: +

Original: [Equation]
Revised: [Equation]

Original: − 1 ψ 1 .
Revised: − 1 ψ 1 .

Original: This means thatfor any fixed ψ 1 , integrating ψ 2: over [ − ψ 1 , 2 π − ψ 1 ] N − 1 is equivalent to integrating over [0 , 2 π ] .
Revised: This means thatfor any fixed ψ 1 , integrating ψ 2: over [ − ψ 1 , 2 π − ψ 1 ] N − 1 is equivalent to integrating over [0 , 2 π ] .

Original: Further note that the term involving h does not depend on ψ , meaning it can be moved out.
Revised: Further note that the term involving h does not depend on ψ , meaning it can be moved out.

Original: [Equation]
Revised: [Equation]

Original: r (41)
Revised: r (41)

Original: This demonstrates that E Z ∼ µ X [ h ( γ ( Z ))]
Revised: This demonstrates that E Z ∼ µ X [ h ( γ ( Z ))]

Original: =
Revised: =

Original: E r , ψ 2: ∼ ν X (cid:20) h (cid:18) λ (cid:18) r , (cid:20)
Revised: E r , ψ 2: ∼ ν X (cid:20) h (cid:18) λ (cid:18) r , (cid:20)

Original: [Equation]
Revised: [Equation]

Original: ψ 2: (cid:21)(cid:19)(cid:19)(cid:21) with ν X being adistribution over R N × [0 , 2 π ] N − 1 with density function
Revised: ψ 2: (cid:21)(cid:19)(cid:19)(cid:21) with ν X being adistribution over R N × [0 , 2 π ] N − 1 with density function

Original: [Equation]
Revised: [Equation]

Original: dψ(43)
Revised: dψ(43)

Original: Inserting into our original optimization problem Eq.
Revised: Inserting into our original optimization problem Eq.

Original: (34) and replacing ψ 2: with ω ∈ [0 , 2 π ] N − 1 toclarify that the problem no longer depends on the entire vector of angles ψ results in
Revised: (34) and replacing ψ 2: with ω ∈ [0 , 2 π ] N − 1 toclarify that the problem no longer depends on the entire vector of angles ψ results in

Original: [Equation]
Revised: [Equation]

Original: .t
Revised: .t

Original: [Equation]
Revised: [Equation]

Original: Note that λ is an injective function from R N × [0 , 2 π ] N into R N × D , meaning that each pair ( r , ω ) inthe support of ν X and ν X ′ corresponds to a distinct function value h ( Z ) .
Revised: Note that λ is an injective function from R N × [0 , 2 π ] N into R N × D , meaning that each pair ( r , ω ) inthe support of ν X and ν X ′ corresponds to a distinct function value h ( Z ) .

Original: We may thus equivalentlyfind the worst-case classifier in R N ×
Revised: We may thus equivalentlyfind the worst-case classifier in R N ×

Original: [0 , 2 π ] N − 1 :
Revised: [0 , 2 π ] N − 1 :

Original: [Equation]
Revised: [Equation]

Original: Solving the lower-dimensional problem.
Revised: Solving the lower-dimensional problem.

Original: Applying the Neyman-Pearson lemma (Lemma 2) to theabove problem shows that its optimal value is
Revised: Applying the Neyman-Pearson lemma (Lemma 2) to theabove problem shows that its optimal value is

Original: [Equation]
Revised: [Equation]

Original: with h ∗ ( r , ω ) = 1 (cid:20) ν
Revised: with h ∗ ( r , ω ) = 1 (cid:20) ν

Original: X ′ ( r , ω
Revised: X ′ ( r , ω

Original: [Equation]
Revised: [Equation]

Original: ν X ( r ,
Revised: ν X ( r ,

Original: [Equation]
Revised: [Equation]

Original: and t ∈ R s.t.
Revised: and t ∈ R s.t.

Original: E r , ω
Revised: E r , ω

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: Transformation into Cartesian coordinates.
Revised: Transformation into Cartesian coordinates.

Original: To conclude our proof, we transform the expectationsin Eqs.
Revised: To conclude our proof, we transform the expectationsin Eqs.

Original: (46) and (47) over this lower-dimensional parameter space back to our Cartesian space 1402 R N × D .
Revised: (46) and (47) over this lower-dimensional parameter space back to our Cartesian space 1402 R N × D .

Original: Before doing so, we take a minor precaution to make the integration bounds easier to workwith.
Revised: Before doing so, we take a minor precaution to make the integration bounds easier to workwith.

Original: We define a function ˆ h : R N × R N − 1 → { 0 , 1 } that allows h to be evaluated for arbitrarypolar coordiante angles, not just those in [0 , 2 π ] : For this, we let ˆ h ( r , ω )
Revised: We define a function ˆ h : R N × R N − 1 → { 0 , 1 } that allows h to be evaluated for arbitrarypolar coordiante angles, not just those in [0 , 2 π ] : For this, we let ˆ h ( r , ω )

Original: =
Revised: =

Original: h ∗ ( r , ( χ ( ω )) , whereχ ( ω ) =
Revised: h ∗ ( r , ( χ ( ω )) , whereχ ( ω ) =

Original: ω
Revised: ω

Original: + 2 kπ s.t. ω
Revised: + 2 kπ s.t. ω

Original: + 2 k ∈ [0 , 2 π ] .
Revised: + 2 k ∈ [0 , 2 π ] .

Original: Clearly E r
Revised: Clearly E r

Original: [Equation]
Revised: [Equation]

Original: , ω )] .
Revised: , ω )] .

Original: Therefore
Revised: Therefore

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: The second equality uses the definition of ν X from Eq.
Revised: The second equality uses the definition of ν X from Eq.

Original: (43) and the last equality combines theintegration bounds of the two integrals.
Revised: (43) and the last equality combines theintegration bounds of the two integrals.

Original: We now apply the substitution θ =
Revised: We now apply the substitution θ =

Original: A − 1 (cid:20) β ω (cid:21) with A − 1 = (cid:20)0 N − 1 1 N −
Revised: A − 1 (cid:20) β ω (cid:21) with A − 1 = (cid:20)0 N − 1 1 N −

Original: I N (cid:21) to transform back intostandard polar coordinates.
Revised: I N (cid:21) to transform back intostandard polar coordinates.

Original: We then simplify the integration bounds by using the fact that, for anyfixed θ 1 all terms of the integrand are 2 π -periodic in θ 2: – similar to how we simplified the integrationbounds when transforming into our alternative parameter space in Eqs.
Revised: We then simplify the integration bounds by using the fact that, for anyfixed θ 1 all terms of the integrand are 2 π -periodic in θ 2: – similar to how we simplified the integrationbounds when transforming into our alternative parameter space in Eqs.

Original: (40) and (41):
Revised: (40) and (41):

Original: [Equation]
Revised: [Equation]

Original: Finally, recall from Eq.
Revised: Finally, recall from Eq.

Original: (47) that ˆ h depends on the likelihood ratios of the marginal distributionsν X ′ and ν X .
Revised: (47) that ˆ h depends on the likelihood ratios of the marginal distributionsν X ′ and ν X .

Original: Lemma 9 provides a closed-form expression for these likelihood ratios, up to constantfactors in X ′ and X .
Revised: Lemma 9 provides a closed-form expression for these likelihood ratios, up to constantfactors in X ′ and X .

Original: This closed-form expression lets us express the likelihood ratio as a function of λ ( r , θ ) .
Revised: This closed-form expression lets us express the likelihood ratio as a function of λ ( r , θ ) .

Original: [Equation]
Revised: [Equation]

Original: [Table]
Revised: [Table]

Original: [Equation]
Revised: [Equation]

Original: [Table]
Revised: [Table]

Original: [Equation]
Revised: [Equation]

Original: [Table]
Revised: [Table]

Original: [Equation]
Revised: [Equation]

Original: [Table]
Revised: [Table]

Original: [Equation]
Revised: [Equation]

Original: with κ : R N × D × R N × D × R N × N
Revised: with κ : R N × D × R N × D × R N × N

Original: →
Revised: →

Original: R 2 defined as in Lemma 9, I 0 being the modified Besselfunction of the first kind and order 0 and c being constant in r and θ .
Revised: R 2 defined as in Lemma 9, I 0 being the modified Besselfunction of the first kind and order 0 and c being constant in r and θ .

Original: In the second equality, we havecancelled out the the r n and used the fact that λ is 2 π -periodic in its second argument to add 1 N θ 1without changing the value of the integral.
Revised: In the second equality, we havecancelled out the the r n and used the fact that λ is 2 π -periodic in its second argument to add 1 N θ 1without changing the value of the integral.

Original: Inserting back into Eq.
Revised: Inserting back into Eq.

Original: (49) and transforming into Cartesian coordinates shows that
Revised: (49) and transforming into Cartesian coordinates shows that

Original: [Equation]
Revised: [Equation]

Original: Inserting into the optimal solution of our lower-dimensional optimization problem (Eqs.
Revised: Inserting into the optimal solution of our lower-dimensional optimization problem (Eqs.

Original: (46) and (47))shows that
Revised: (46) and (47))shows that

Original: [Equation]
Revised: [Equation]

Original: t ∈ R s.t.
Revised: t ∈ R s.t.

Original: [Equation]
Revised: [Equation]

Original: Note that we can absorb the constant c into the threshold t .
Revised: Note that we can absorb the constant c into the threshold t .

Original: [Equation]
Revised: [Equation]

Original: 52) with t ′ ∈ R s.t.
Revised: 52) with t ′ ∈ R s.t.

Original: [Equation]
Revised: [Equation]

Original: If t ′ fulfills the conditions in Eq.
Revised: If t ′ fulfills the conditions in Eq.

Original: (53), then t = t ′ c
Revised: (53), then t = t ′ c

Original: fulfills the conditions in Eq.
Revised: fulfills the conditions in Eq.

Original: Likelihood ratio distribution.
Revised: Likelihood ratio distribution.

Original: Finally, recall from Lemma 9 that
Revised: Finally, recall from Lemma 9 that

Original: [Equation]
Revised: [Equation]

Original: Inside the probability terms appearing in Eqs.
Revised: Inside the probability terms appearing in Eqs.

Original: (52) and (53), we have
Revised: (52) and (53), we have

Original: [Equation]
Revised: [Equation]

Original: and
Revised: and

Original: [Equation]
Revised: [Equation]

Original: where R (cid:0) − π 2 (cid:1) is the rotation matrix that rotates clockwise by angle π 2 .
Revised: where R (cid:0) − π 2 (cid:1) is the rotation matrix that rotates clockwise by angle π 2 .

Original: In Eq.
Revised: In Eq.

Original: (52), the random variable Z follows distribution µ X ′ .
Revised: (52), the random variable Z follows distribution µ X ′ .

Original: Thus, vec ( Z ) ∼ N (cid:0) vec( X ′ ) , σ 2 I 2 N (cid:1) .
Revised: Thus, vec ( Z ) ∼ N (cid:0) vec( X ′ ) , σ 2 I 2 N (cid:1) .

Original: Therefore, the random variable q ∈ R 4 = (cid:20) κ (cid:0) Z , X ′ , σ 2 I N (cid:1) κ (cid:0) Z , X , σ 2 I N (cid:1)(cid:21) , which is an affine transformation ofvec( Z ) follows distribution N (cid:0) m (1) , S (cid:1) with m (1) =
Revised: Therefore, the random variable q ∈ R 4 = (cid:20) κ (cid:0) Z , X ′ , σ 2 I N (cid:1) κ (cid:0) Z , X , σ 2 I N (cid:1)(cid:21) , which is an affine transformation ofvec( Z ) follows distribution N (cid:0) m (1) , S (cid:1) with m (1) =

Original: CX ′ and S
Revised: CX ′ and S

Original: = σ 2 CC T , where
Revised: = σ 2 CC T , where

Original: [Equation]
Revised: [Equation]

Original: Similarly, in Eq.
Revised: Similarly, in Eq.

Original: (53), the random variable Z follows distribution µ X .
Revised: (53), the random variable Z follows distribution µ X .

Original: Therefore, the random variableq ∈ R 4 = (cid:20) κ (cid:0) Z , X ′ , σ 2 I N (cid:1) κ (cid:0) Z , X , σ 2 I N (cid:1) (cid:21) , which is an affine transformation of vec( Z ) follows distribution
Revised: Therefore, the random variableq ∈ R 4 = (cid:20) κ (cid:0) Z , X ′ , σ 2 I N (cid:1) κ (cid:0) Z , X , σ 2 I N (cid:1) (cid:21) , which is an affine transformation of vec( Z ) follows distribution

Original: N (cid:0) m (2) , S (cid:1) with m (2) =
Revised: N (cid:0) m (2) , S (cid:1) with m (2) =

Original: CX and S =
Revised: CX and S =

Original: σ 2 CC T .
Revised: σ 2 CC T .

Original: Computing the matrix-vector and matrix-matrix products and using the definition that X ′ =
Revised: Computing the matrix-vector and matrix-matrix products and using the definition that X ′ =

Original: X
Revised: X

Original: + ∆shows that
Revised: + ∆shows that

Original: [Equation]
Revised: [Equation]

Original: with ϵ 1 = ⟨ X , ∆ ⟩ F , ϵ 2 = (cid:68) XR (cid:0) − π 2 (cid:1) T , ∆ (cid:69) F .
Revised: with ϵ 1 = ⟨ X , ∆ ⟩ F , ϵ 2 = (cid:68) XR (cid:0) − π 2 (cid:1) T , ∆ (cid:69) F .

Original: This concludes our proof.
Revised: This concludes our proof.

Original: D.5.3 Worst-case classifier
Revised: D.5.3 Worst-case classifier

Original: In
Revised: In

Original: Section 6.3, we claimed that the worst-case classifier, i.e. the minimizer ofmin h ∈ H T Pr Z ∼ µ ′ X [ h ( Z ) =
Revised: Section 6.3, we claimed that the worst-case classifier, i.e. the minimizer ofmin h ∈ H T Pr Z ∼ µ ′ X [ h ( Z ) =

Original: y ∗ ] was a Gaussian Haar integration kernel method [16], i.e. a functionthat predicts y ∗ if and only if β ( X ′ , Z ) β ( X , Z ) ≤ t with β ( X , Z ) =
Revised: y ∗ ] was a Gaussian Haar integration kernel method [16], i.e. a functionthat predicts y ∗ if and only if β ( X ′ , Z ) β ( X , Z ) ≤ t with β ( X , Z ) =

Original: (cid:82) 2 π 0 exp (cid:0) 1 σ 

Revised: (cid:82) 2 π 0 exp (cid:0) 1 σ

Original: (cid:10) X , ZR ( α )
Revised: (cid:10) X , ZR ( α )

Original: [Equation]
Revised: [Equation]

Original: where R ( α ) ∈ SO (2) is the matrix that rotates counter-clockwise by angle α .
Revised: where R ( α ) ∈ SO (2) is the matrix that rotates counter-clockwise by angle α .

Original: Here, we verify this claim.
Revised: Here, we verify this claim.

Original: In fact, we have already implicitly done so in the derivation of ourcertificate.
Revised: In fact, we have already implicitly done so in the derivation of ourcertificate.

Original: Recall from Eq.
Revised: Recall from Eq.

Original: (52) that
Revised: (52) that

Original: [Equation]
Revised: [Equation]

Original: for some t ∈ R .
Revised: for some t ∈ R .

Original: Let r , θ ∈ R N be the radii and angles of Z , i.e. Z = λ ( r , θ ) .
Revised: Let r , θ ∈ R N be the radii and angles of Z , i.e. Z = λ ( r , θ ) .

Original: During our proofof Lemma 9, we have shown that
Revised: During our proofof Lemma 9, we have shown that

Original: [Equation]
Revised: [Equation]

Original: 54) with cos( α ) λ ( r , ϕ ) T : , 1 1 σ 2 X : , 1 − sin( α ) λ ( r , ϕ ) T :
Revised: 54) with cos( α ) λ ( r , ϕ ) T : , 1 1 σ 2 X : , 1 − sin( α ) λ ( r , ϕ ) T :

Original: [Equation]
Revised: [Equation]

Original: X
Revised: X

Original: [Equation]
Revised: [Equation]

Original: 55) and sin( α ) λ ( r , ϕ ) T : , 1 1 σ 2 X : , 2 + cos( α ) λ
Revised: 55) and sin( α ) λ ( r , ϕ ) T : , 1 1 σ 2 X : , 2 + cos( α ) λ

Original: ( r , ϕ ) T : , 2 1 σ 2 X :
Revised: ( r , ϕ ) T : , 2 1 σ 2 X :

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: see Eqs.
Revised: see Eqs.

Original: (31) to (33) with X substituted for M and 1 σ 2 I N substituted for Σ − 1 .
Revised: (31) to (33) with X substituted for M and 1 σ 2 I N substituted for Σ − 1 .

Original: We have omittedterms that are constant and can thus be absorbed into classification threshold t .
Revised: We have omittedterms that are constant and can thus be absorbed into classification threshold t .

Original: By substituting Z for λ ( r , θ ) and explicitly writing out the inner products in Eqs.
Revised: By substituting Z for λ ( r , θ ) and explicitly writing out the inner products in Eqs.

Original: (55) and (56) assummations over the N rows, Eq.
Revised: (55) and (56) assummations over the N rows, Eq.

Original: (54) can be equivalently expressed as
Revised: (54) can be equivalently expressed as

Original: [Equation]
Revised: [Equation]

Original: Since ⟨ A , B ⟩ F = (cid:80) Nn =1 (cid:0)
Revised: Since ⟨ A , B ⟩ F = (cid:80) Nn =1 (cid:0)

Original: A Tn B n (cid:1) , the above is in fact identical to
Revised: A Tn B n (cid:1) , the above is in fact identical to

Original: [Equation]
Revised: [Equation]

Original: D.6 Tight certificate for invariance under elemental rotation in 3D
Revised: D.6 Tight certificate for invariance under elemental rotation in 3D

Original: Next, we generalize our certificate for rotation invariance in 2D to invariance under T = 1455 (cid:8)
Revised: Next, we generalize our certificate for rotation invariance in 2D to invariance under T = 1455 (cid:8)

Original: X (cid:55)→ XR T | R ∈ SO (3 , d ∗ ) ∈ R 3 (cid:9) , with SO (3 , d ) =
Revised: X (cid:55)→ XR T | R ∈ SO (3 , d ∗ ) ∈ R 3 (cid:9) , with SO (3 , d ) =

Original: { R ∈ SO (3) | R d ∗ = e d } , where e d isan indicator vector and d ∗ ∈ { 1 , 2 , 3 } is the rotation axis.
Revised: { R ∈ SO (3) | R d ∗ = e d } , where e d isan indicator vector and d ∗ ∈ { 1 , 2 , 3 } is the rotation axis.

Original: In the following, let A : , − d ∗ ∈ R N × 2 be the matrix composed of the two columns that are not A : ,d ∗ .
Revised: In the following, let A : , − d ∗ ∈ R N × 2 be the matrix composed of the two columns that are not A : ,d ∗ .

Original: Theorem 8.
Revised: Theorem 8.

Original: Let g : R N × 3 → Y be invariant to elemental rotations around axis d ∗ and let H T bedefined as in Eq.
Revised: Let g : R N × 3 → Y be invariant to elemental rotations around axis d ∗ and let H T bedefined as in Eq.

Original: Then
Revised: Then

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: with t ∈ R s.t.
Revised: with t ∈ R s.t.

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: and ρ ( q ) =
Revised: and ρ ( q ) =

Original: I
Revised: I

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: where I 0 is the modified Bessel function of the first kind and order 0 , and
Revised: where I 0 is the modified Bessel function of the first kind and order 0 , and

Original: [Equation]
Revised: [Equation]

Original:   ϵ 1 + ϵ− ϵ 2 ϵ 
Revised:   ϵ 1 + ϵ− ϵ 2 ϵ 

Original:  , Σ =
Revised:  , Σ =

Original: [Equation]
Revised: [Equation]

Original:  
Revised:  

Original: [Equation]
Revised: [Equation]

Original: 1 +
Revised: 1 +

Original: [Equation]
Revised: [Equation]

Original: ϵ 1 +
Revised: ϵ 1 +

Original: [Equation]
Revised: [Equation]

Original: ϵ 22 ϵ 1 +
Revised: ϵ 22 ϵ 1 +

Original: [Equation]
Revised: [Equation]

Original: 3 +
Revised: 3 +

Original: [Equation]
Revised: [Equation]

Original: − ϵϵ 1 + ϵ
Revised: − ϵϵ 1 + ϵ

Original: [Equation]
Revised: [Equation]

Original: 1 +
Revised: 1 +

Original: [Equation]
Revised: [Equation]

Original: − ϵϵ ϵϵ 1 + ϵ ϵ  .
Revised: − ϵϵ ϵϵ 1 + ϵ ϵ  .

Original: with ϵ 1 = (cid:68) ˜ X , ˜ ∆ (cid:69) F , ϵ 2 = (cid:68) ˜ XR (cid:0) − π 2 (cid:1) T , ˜ ∆ (cid:69) F ϵ 

Revised: with ϵ 1 = (cid:68) ˜ X , ˜ ∆ (cid:69) F , ϵ 2 = (cid:68) ˜ XR (cid:0) − π 2 (cid:1) T , ˜ ∆ (cid:69) F ϵ

Original: =
Revised: =

Original: ||
Revised: ||

Original: X : , − d ∗ || 

Revised: X : , − d ∗ ||

Original: and ϵ 

Revised: and ϵ

Original: = ||
Revised: = ||

Original: X : , − d ∗ || 22 .
Revised: X : , − d ∗ || 22 .

Original: The certificate is very similar to that from Appendix D.5, but additionally depends on the amount ofperturbation || ∆ : ,d ∗ || 2 along perturbation axis d ∗ .
Revised: The certificate is very similar to that from Appendix D.5, but additionally depends on the amount ofperturbation || ∆ : ,d ∗ || 2 along perturbation axis d ∗ .

Original: It further differs from the previous certificate in 1464 that the probabilities in Eqs.
Revised: It further differs from the previous certificate in 1464 that the probabilities in Eqs.

Original: (58) and (59) involve a fifth normally distributed random variable o ∈ R .
Revised: (58) and (59) involve a fifth normally distributed random variable o ∈ R .

Original: D.6.1 Canonical map
Revised: D.6.1 Canonical map

Original: The canonical map differs from that for rotations in 2D in that we have to make case distinctionsbetween
Revised: The canonical map differs from that for rotations in 2D in that we have to make case distinctionsbetween

Original: d ∗ = 1 , d ∗ = 2 and d ∗ = 3 .
Revised: d ∗ = 1 , d ∗ = 2 and d ∗ = 3 .

Original: We shall formulate these case distinctions in a way that makesthe case d ∗ = 3 as similar as possible to the canonical map for rotation invariance in 2D, which willhelp simplify the proof of our certificate.
Revised: We shall formulate these case distinctions in a way that makesthe case d ∗ = 3 as similar as possible to the canonical map for rotation invariance in 2D, which willhelp simplify the proof of our certificate.

Original: For our derivation, recall the definition of canonical maps.
Revised: For our derivation, recall the definition of canonical maps.

Original: Definition 3 (Canonical map) .
Revised: Definition 3 (Canonical map) .

Original: A canonical map for invariance under a group of transformations Tis a function γ : R N × D
Revised: A canonical map for invariance under a group of transformations Tis a function γ : R N × D

Original: → R N × D with
Revised: → R N × D with

Original: [Equation]
Revised: [Equation]

Original: For rotation invariance, it is more convenient to work with cylindrical coordinates.
Revised: For rotation invariance, it is more convenient to work with cylindrical coordinates.

Original: In the following,let l : R N × R N × R N × { 1 , 2 , 3 } → R N × 3 be the function that maps from cylindrical coordinatesinto Cartesian coordinates with
Revised: In the following,let l : R N × R N × R N × { 1 , 2 , 3 } → R N × 3 be the function that maps from cylindrical coordinatesinto Cartesian coordinates with

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: and λ ( r , θ ) =
Revised: and λ ( r , θ ) =

Original: [ r ⊙ cos( θ ) r ⊙ sin( θ ) ] defined like in Appendix D.5.
Revised: [ r ⊙ cos( θ ) r ⊙ sin( θ ) ] defined like in Appendix D.5.

Original: Next, we define an elemental rotation operator for matrices:
Revised: Next, we define an elemental rotation operator for matrices:

Original: Definition 5.
Revised: Definition 5.

Original: The elemental rotation operator is the function erot : R N × 3 × R × { 1 , 2 , 3 } → R N × 2witherot( Z , a, d ) =
Revised: The elemental rotation operator is the function erot : R N × 3 × R × { 1 , 2 , 3 } → R N × 2witherot( Z , a, d ) =

Original:  
Revised:  

Original: Z     0 cos( a ) − sin( a ) sin( a ) cos( a ) 
Revised: Z     0 cos( a ) − sin( a ) sin( a ) cos( a ) 

Original: 
Revised: 

Original: T if d =
Revised: T if d =

Original: Z   cos( a ) sin( a )   − sin( a ) 0 cos( a ) 
Revised: Z   cos( a ) sin( a )   − sin( a ) 0 cos( a ) 

Original: 
Revised: 

Original: T if d =
Revised: T if d =

Original: Z   cos( a ) − sin( a )sin( a ) cos( a )    
Revised: Z   cos( a ) − sin( a )sin( a ) cos( a )    

Original: 
Revised: 

Original: T if d =that rotates all rows of matrix
Revised: T if d =that rotates all rows of matrix

Original: Z by angle a around axis d .
Revised: Z by angle a around axis d .

Original: We use the elemental matrix rotation operator to define our canonical map, which rotates an inputmatrix Z around axis d ∗ such that the first row n ∗ with non-zero radius r n ∗ takes on angle 0 :
Revised: We use the elemental matrix rotation operator to define our canonical map, which rotates an inputmatrix Z around axis d ∗ such that the first row n ∗ with non-zero radius r n ∗ takes on angle 0 :

Original: Lemma 10.
Revised: Lemma 10.

Original: The function γ ( Z ) =
Revised: The function γ ( Z ) =

Original: erot ( Z , − α
Revised: erot ( Z , − α

Original: ( Z ) , d ∗ ) withα ( Z ) =
Revised: ( Z ) , d ∗ ) withα ( Z ) =

Original: (cid:26)if || Λ( Z , d ∗ ) ||
Revised: (cid:26)if || Λ( Z , d ∗ ) ||

Original: 2 =
Revised: 2 =

Original: arctan2 (Λ( Z ) n ∗ , 2 , Λ( Z ) n ∗ , 1 ) else where n ∗ =
Revised: arctan2 (Λ( Z ) n ∗ , 2 , Λ( Z ) n ∗ , 1 ) else where n ∗ =

Original: min { n | || Λ( Z n ) || 2 > 0 } is the first non-zero row of matrix Λ( Z ) , d ∗ withΛ( Z , d ) =
Revised: min { n | || Λ( Z n ) || 2 > 0 } is the first non-zero row of matrix Λ( Z ) , d ∗ withΛ( Z , d ) =

Original:    
Revised:    

Original: Z : ,
Revised: Z : ,

Original: Z : ,if d =
Revised: Z : ,if d =

Original: Z : ,
Revised: Z : ,

Original: Z : ,if d =
Revised: Z : ,if d =

Original: Z : ,
Revised: Z : ,

Original: Z : ,if d =
Revised: Z : ,if d =

Original:  is a canonical map for elemental rotations around axis d ∗ .
Revised:  is a canonical map for elemental rotations around axis d ∗ .

Original: Proof.
Revised: Proof.

Original: We can distinguish two cases.
Revised: We can distinguish two cases.

Original: Case 1.
Revised: Case 1.

Original: All entries of Λ( Z , d ∗ ) are 0 , i.e. Λ( Z , d ∗ ) =
Revised: All entries of Λ( Z , d ∗ ) are 0 , i.e. Λ( Z , d ∗ ) =

Original: 0 N, 2 .
Revised: 0 N, 2 .

Original: In this case α ( Z ) = 0 and thusγ ( Z )
Revised: In this case α ( Z ) = 0 and thusγ ( Z )

Original: =
Revised: =

Original: Z .
Revised: Z .

Original: The equivalence class of Z is [ Z ] T = { Z } , because rotation around d ∗ will not makeΛ( Z , d ∗ ) non-zero.
Revised: The equivalence class of Z is [ Z ] T = { Z } , because rotation around d ∗ will not makeΛ( Z , d ∗ ) non-zero.

Original: Both criteria Eqs.
Revised: Both criteria Eqs.

Original: (3) and (4) for canonical maps are thus fulfilled.
Revised: (3) and (4) for canonical maps are thus fulfilled.

Original: Case 2.
Revised: Case 2.

Original: The matrix Λ( Z , d ∗ ) has non-zero entries.
Revised: The matrix Λ( Z , d ∗ ) has non-zero entries.

Original: Let ( r , θ , z ) be a map of Z in cylindricalcoordinates, i.e. Z = l ( r , θ , z , d ∗ ) .
Revised: Let ( r , θ , z ) be a map of Z in cylindricalcoordinates, i.e. Z = l ( r , θ , z , d ∗ ) .

Original: Let n ∗ be the index of the first non-zero row of Λ( Z , d ∗ ) .
Revised: Let n ∗ be the index of the first non-zero row of Λ( Z , d ∗ ) .

Original: Recallthat rotation around axis
Revised: Recallthat rotation around axis

Original: d ∗ in Cartesian coordinates corresponds to translation of the cylindricalcoordinate angles and thus γ ( Z )
Revised: d ∗ in Cartesian coordinates corresponds to translation of the cylindricalcoordinate angles and thus γ ( Z )

Original: =
Revised: =

Original: l ( r , θ − 1 N θ n ∗ , z , d ∗ ) .
Revised: l ( r , θ − 1 N θ n ∗ , z , d ∗ ) .

Original: Criterion Eq.
Revised: Criterion Eq.

Original: (3) is fulfilled, because by construction of γ we have γ ( Z ) =
Revised: (3) is fulfilled, because by construction of γ we have γ ( Z ) =

Original: ZR T for some R ∈
Revised: ZR T for some R ∈

Original: SO (3 , d ∗ ) .
Revised: SO (3 , d ∗ ) .

Original: Now, consider an arbitrary Z ′ ∈
Revised: Now, consider an arbitrary Z ′ ∈

Original: [ Z ] T .
Revised: [ Z ] T .

Original: By definition of [ Z ] T , there must be some a ∈ R such that
Revised: By definition of [ Z ] T , there must be some a ∈ R such that

Original: Z ′ =
Revised: Z ′ =

Original: l ( r , θ + a 1 N , z , d ∗ ) .
Revised: l ( r , θ + a 1 N , z , d ∗ ) .

Original: Note that rotation does not affect the radii, i.e. n ∗ is also the index ofthe first non-zero row of Z ′ .
Revised: Note that rotation does not affect the radii, i.e. n ∗ is also the index ofthe first non-zero row of Z ′ .

Original: Thus
Revised: Thus

Original: [Equation]
Revised: [Equation]

Original: meaning criterion Eq.
Revised: meaning criterion Eq.

Original: (4) is fulfilled as well.
Revised: (4) is fulfilled as well.

Original: D.6.2 Proof for certificate
Revised: D.6.2 Proof for certificate

Original: In the following, we provide the proof of Theorem 8 for the case d ∗ = 3 .
Revised: In the following, we provide the proof of Theorem 8 for the case d ∗ = 3 .

Original: The other cases followanalogously.
Revised: The other cases followanalogously.

Original: Mapping into cylindrical coordinates is equivalent to transforming the two columns thatare not d ∗ into polar coordinates.
Revised: Mapping into cylindrical coordinates is equivalent to transforming the two columns thatare not d ∗ into polar coordinates.

Original: Similarly, applying the canonical map for elemental rotations isequivalent to applying the canonical map for rotation in 2D (see Appendix D.5) to the two columnsthat are not d ∗ into polar coordinates.
Revised: Similarly, applying the canonical map for elemental rotations isequivalent to applying the canonical map for rotation in 2D (see Appendix D.5) to the two columnsthat are not d ∗ into polar coordinates.

Original: We use this fact to reuse much of our proof from Appendix D.5.
Revised: We use this fact to reuse much of our proof from Appendix D.5.

Original: Applying Theorem 4 with our canonical map from Lemma 10 shows that
Revised: Applying Theorem 4 with our canonical map from Lemma 10 shows that

Original: [Equation]
Revised: [Equation]

Original: T
Revised: T

Original: [Equation]
Revised: [Equation]

Original: Like in our previous proof, we notice that the above optimization problem does not depend on thespecific class predicted by h , but only on whether it is y ∗ or y .
Revised: Like in our previous proof, we notice that the above optimization problem does not depend on thespecific class predicted by h , but only on whether it is y ∗ or y .

Original: We thus formulate the equivalentproblem
Revised: We thus formulate the equivalentproblem

Original: [Equation]
Revised: [Equation]

Original: where h ( Z )
Revised: where h ( Z )

Original: = 1 indicates that Z is classified as y ∗ .
Revised: = 1 indicates that Z is classified as y ∗ .

Original: Substitution.
Revised: Substitution.

Original: We begin by going from Cartesian to cylindrical coordinates: As defined in Eq.
Revised: We begin by going from Cartesian to cylindrical coordinates: As defined in Eq.

Original: (61),let l : R N × R N × R N × { 1 , 2 , 3 } → R N × 3 be the function that maps from cylindrical coordinatesinto Cartesian coordiantes.
Revised: (61),let l : R N × R N × R N × { 1 , 2 , 3 } → R N × 3 be the function that maps from cylindrical coordinatesinto Cartesian coordiantes.

Original: [Equation]
Revised: [Equation]

Original: The r n in the last equality are due to the fact that l transforms N rows from cylindrical to Cartesiancoordinates, with each transformation having a Jacobian determinant r .
Revised: The r n in the last equality are due to the fact that l transforms N rows from cylindrical to Cartesiancoordinates, with each transformation having a Jacobian determinant r .

Original: Similarity to 2D rotation invariance.
Revised: Similarity to 2D rotation invariance.

Original: Now, we use the fact that we are in case d ∗ to show that theabove integral is almost identical to the one from our proof for rotation invariance in 2D.
Revised: Now, we use the fact that we are in case d ∗ to show that theabove integral is almost identical to the one from our proof for rotation invariance in 2D.

Original: Firstly, we know from the definition of l ( · ) in Eq.
Revised: Firstly, we know from the definition of l ( · ) in Eq.

Original: (61) that l ( r , θ , Z : ,d ∗ , d ∗ ) =
Revised: (61) that l ( r , θ , Z : ,d ∗ , d ∗ ) =

Original: l ( r , θ , Z : , 3 , 3) =[ λ ( r , θ ) Z : , 3 ] , where λ ( r , θ ) =
Revised: l ( r , θ , Z : , 3 , 3) =[ λ ( r , θ ) Z : , 3 ] , where λ ( r , θ ) =

Original: [ r ⊙ cos( θ ) r ⊙ sin( θ )] is the function that maps from polarinto Cartesian coordinates.
Revised: [ r ⊙ cos( θ ) r ⊙ sin( θ )] is the function that maps from polarinto Cartesian coordinates.

Original: Secondly, let γ ′ be the canonical map for rotation invariance in 2D, as defined in Lemma 8.
Revised: Secondly, let γ ′ be the canonical map for rotation invariance in 2D, as defined in Lemma 8.

Original: We know from the definition of our canonical map for elemental rotations in Lemma 10 thatγ ([ λ ( r , θ ) Z : , 3 ]) =
Revised: We know from the definition of our canonical map for elemental rotations in Lemma 10 thatγ ([ λ ( r , θ ) Z : , 3 ]) =

Original: [ γ ′ ( λ ( r , θ ))
Revised: [ γ ′ ( λ ( r , θ ))

Original: Z : , 3 ] .
Revised: Z : , 3 ] .

Original: Thirdly, recall that µ X is an isotropic matrix normal distribution, i.e. all columns are independent.
Revised: Thirdly, recall that µ X is an isotropic matrix normal distribution, i.e. all columns are independent.

Original: Let µ X : , :2 be the distribution of the first two dimensions and µ X : , 3 be the distribution of the third 1526 dimension, i.e.
Revised: Let µ X : , :2 be the distribution of the first two dimensions and µ X : , 3 be the distribution of the third 1526 dimension, i.e.

Original: [Equation]
Revised: [Equation]

Original: Combining all three steps shows that
Revised: Combining all three steps shows that

Original: [Equation]
Revised: [Equation]

Original: This expectation is identical to the one we derived in Eq.
Revised: This expectation is identical to the one we derived in Eq.

Original: (37) of Appendix D.5, save for an independent variable Z : , 3 that contributes factor µ X : , 3 ( Z : , 3 ) to the likelihood.
Revised: (37) of Appendix D.5, save for an independent variable Z : , 3 that contributes factor µ X : , 3 ( Z : , 3 ) to the likelihood.

Original: We may thus perform 1530 exactly the same steps as in the derivation of our certificate for rotation invariance in 2D, starting at
Revised: We may thus perform 1530 exactly the same steps as in the derivation of our certificate for rotation invariance in 2D, starting at

Original: Eq.
Revised: Eq.

Original: (37), i.e. further substitutions, marginalization, application of the Neyman Pearson lemma andtransformation back into Cartesian coordinates, to show thatmin
Revised: (37), i.e. further substitutions, marginalization, application of the Neyman Pearson lemma andtransformation back into Cartesian coordinates, to show thatmin

Original: [Equation]
Revised: [Equation]

Original: with t ∈ R s.t.
Revised: with t ∈ R s.t.

Original: [Equation]
Revised: [Equation]

Original: and ρ ( q ) = I(cid:18)(cid:113)
Revised: and ρ ( q ) = I(cid:18)(cid:113)

Original: q
Revised: q

Original: [Equation]
Revised: [Equation]

Original: (cid:19)
Revised: (cid:19)

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: Likelihood ratio distribution.
Revised: Likelihood ratio distribution.

Original: The last step of our derivation is to find the distribution of thelikelihood ratios in Eqs.
Revised: The last step of our derivation is to find the distribution of thelikelihood ratios in Eqs.

Original: (63) and (64).
Revised: (63) and (64).

Original: Since X ′ =
Revised: Since X ′ =

Original: X + ∆ , we have
Revised: X + ∆ , we have

Original: [Equation]
Revised: [Equation]

Original: , see also Appendix A of [27].
Revised: , see also Appendix A of [27].

Original: The argument of the exponential function is an affine transformation of random variable Z : , 3 ,o = 12 σ
Revised: The argument of the exponential function is an affine transformation of random variable Z : , 3 ,o = 12 σ

Original: [Equation]
Revised: [Equation]

Original: (cid:0)
Revised: (cid:0)

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: [Equation]
Revised: [Equation]

Original: |
Revised: |

Original: [Equation]
Revised: [Equation]

Original: ∆ : ,
Revised: ∆ : ,

Original: [Equation]
Revised: [Equation]

Original: || 22 −
Revised: || 22 −

Original: [Equation]
Revised: [Equation]

Original: ∆
Revised: ∆

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: : , 3 (cid:1) .
Revised: : , 3 (cid:1) .

Original: In Eq.
Revised: In Eq.

Original: (63), we have Z : , 3 ∼ µ X ′ : , 3 := N (cid:0) X : , 3 + ∆ : , 3 , σ 2 I N (cid:1) and thus o ∼ N ( u (1) , s ) with meanu (1) = 12 σ 2 (cid:0) 2 || ∆ : , 3 || 22 − || ∆ : , 3 || 22 (cid:1) =
Revised: (63), we have Z : , 3 ∼ µ X ′ : , 3 := N (cid:0) X : , 3 + ∆ : , 3 , σ 2 I N (cid:1) and thus o ∼ N ( u (1) , s ) with meanu (1) = 12 σ 2 (cid:0) 2 || ∆ : , 3 || 22 − || ∆ : , 3 || 22 (cid:1) =

Original: 12 σ 2 (cid:0) || ∆ : , 3 || 22 (cid:1) and variance s 2 = 1 σ 2 ∆ T : , 3 σ 2 I N ∆ : , 3 1 σ 2 =1 σ 2 || ∆ : , 3 || 22 .
Revised: 12 σ 2 (cid:0) || ∆ : , 3 || 22 (cid:1) and variance s 2 = 1 σ 2 ∆ T : , 3 σ 2 I N ∆ : , 3 1 σ 2 =1 σ 2 || ∆ : , 3 || 22 .

Original: In Eq.
Revised: In Eq.

Original: (64) we have Z : , 3 ∼ µ X : , 3 := N (cid:0) X : , 3 , σ 2 I N (cid:1) and thus o ∼ N ( u (2) , s ) with mean u (2) =12 σ 2 (cid:0) −|| ∆ : , 3 ||
Revised: (64) we have Z : , 3 ∼ µ X : , 3 := N (cid:0) X : , 3 , σ 2 I N (cid:1) and thus o ∼ N ( u (2) , s ) with mean u (2) =12 σ 2 (cid:0) −|| ∆ : , 3 ||

Original: [Equation]
Revised: [Equation]

Original: Therefore, we have
Revised: Therefore, we have

Original: [Equation]
Revised: [Equation]

Original: with t ∈ R
Revised: with t ∈ R

Original: [Equation]
Revised: [Equation]

Original: which concludes our proof
Revised: which concludes our proof

Original: [Equation]
Revised: [Equation]

Original: D.7 Tight certificate for simultaneous invariance under rotation and translation in 2D
Revised: D.7 Tight certificate for simultaneous invariance under rotation and translation in 2D

Original: Next, we consider the case that base classifier g is rotation and translation invariant, i.e. invariantunder group T = (cid:8) X (cid:55)→ XR T + 1 N c T | R ∈ SO (2) , c ∈ R D (cid:9) .
Revised: Next, we consider the case that base classifier g is rotation and translation invariant, i.e. invariantunder group T = (cid:8) X (cid:55)→ XR T + 1 N c T | R ∈ SO (2) , c ∈ R D (cid:9) .

Original: In the following, let ˜ A ∈ R N × D be matrix A ∈ R N × D after centering,
Revised: In the following, let ˜ A ∈ R N × D be matrix A ∈ R N × D after centering,

Original: i.e. ˜ A
Revised: i.e. ˜ A

Original: =
Revised: =

Original: A − 1 N A , where
Revised: A − 1 N A , where

Original: A ∈ R 1 × D are the column-wise averages.
Revised: A ∈ R 1 × D are the column-wise averages.

Original: Further let ⟨ A , B ⟩ F = (cid:80) Nn =

Revised: Further let ⟨ A , B ⟩ F = (cid:80) Nn =

Original: A Tn B Tn = vec( A ) T vec( B )be the Frobenius inner product.
Revised: A Tn B Tn = vec( A ) T vec( B )be the Frobenius inner product.

Original: Theorem 9.
Revised: Theorem 9.

Original: Let g : R N × 2 → Y be rotation and translation invariant and H T be defined as in Eq.
Revised: Let g : R N × 2 → Y be rotation and translation invariant and H T be defined as in Eq.

Original: Then
Revised: Then

Original: [Equation]
Revised: [Equation]

Original: 69) with t ∈ R s.t.
Revised: 69) with t ∈ R s.t.

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: and
Revised: and

Original: [Equation]
Revised: [Equation]

Original: where I 0 is the modified Bessel function of the first kind and order 0 , and
Revised: where I 0 is the modified Bessel function of the first kind and order 0 , and

Original: [Equation]
Revised: [Equation]

Original: with ϵ 1 = (cid:68) ˜ X , ˜ ∆ (cid:69) F , ϵ 2 = (cid:68) ˜ XR (cid:0) − π 2 (cid:1) T , ˜ ∆ (cid:69) F .
Revised: with ϵ 1 = (cid:68) ˜ X , ˜ ∆ (cid:69) F , ϵ 2 = (cid:68) ˜ XR (cid:0) − π 2 (cid:1) T , ˜ ∆ (cid:69) F .

Original: The above certificate is identical to the tight certificate for rotation invariance in 2D (see Appendix D.5), save for the fact that the clean input X and the perturbation ∆ have been centered.
Revised: The above certificate is identical to the tight certificate for rotation invariance in 2D (see Appendix D.5), save for the fact that the clean input X and the perturbation ∆ have been centered.

Original: Since X ′ =
Revised: Since X ′ =

Original: X + ∆ , this is equivalent to centering X and X ′
Revised: X + ∆ , this is equivalent to centering X and X ′

Original: and then evaluating the certificatefrom Appendix D.5.
Revised: and then evaluating the certificatefrom Appendix D.5.

Original: D.7.1 Canonical map
Revised: D.7.1 Canonical map

Original: For our derivation, recall the definition of canonical maps:
Revised: For our derivation, recall the definition of canonical maps:

Original: Definition 3 (Canonical map) .
Revised: Definition 3 (Canonical map) .

Original: A canonical map for invariance under a group of transformations Tis a function γ : R N × D
Revised: A canonical map for invariance under a group of transformations Tis a function γ : R N × D

Original: → R N × D with
Revised: → R N × D with

Original: [Equation]
Revised: [Equation]

Original: In the following, let γ rot be the canonical map for invariance under T rot := 1564 (cid:8) X
Revised: In the following, let γ rot be the canonical map for invariance under T rot := 1564 (cid:8) X

Original: (cid:55)→ XR T | R ∈ SO (2) (cid:9) , as defined in Appendix D.5.
Revised: (cid:55)→ XR T | R ∈ SO (2) (cid:9) , as defined in Appendix D.5.

Original: Further let γ trans be the canonical map for invariance under T trans := (cid:8)
Revised: Further let γ trans be the canonical map for invariance under T trans := (cid:8)

Original: X
Revised: X

Original: (cid:55)→ 1 N c T | c ∈ R 2 (cid:9) , as defined in Appendix D.4.
Revised: (cid:55)→ 1 N c T | c ∈ R 2 (cid:9) , as defined in Appendix D.4.

Original: Composing these two canonical maps yields a canonical map for simultaneous rotation and translationinvariance.
Revised: Composing these two canonical maps yields a canonical map for simultaneous rotation and translationinvariance.

Original: Lemma 11.
Revised: Lemma 11.

Original: The function γ ( Z ) =
Revised: The function γ ( Z ) =

Original: γ rot ( γ trans ( Z )) is a canonical map for simultaneous rotationand translation invariance.
Revised: γ rot ( γ trans ( Z )) is a canonical map for simultaneous rotationand translation invariance.

Original: Proof.
Revised: Proof.

Original: Consider an arbitrary Z ∈ R N × 2 .
Revised: Consider an arbitrary Z ∈ R N × 2 .

Original: As defined in Appendices D.4 and D.5, γ performs sometranslation, followed by some rotation.
Revised: As defined in Appendices D.4 and D.5, γ performs sometranslation, followed by some rotation.

Original: That is, γ ( Z ) =
Revised: That is, γ ( Z ) =

Original: γ rot (cid:0) Z + 1 N c T (cid:1) for some c ∈ R 2 andγ rot ( Z + 1 N c T ) =
Revised: γ rot (cid:0) Z + 1 N c T (cid:1) for some c ∈ R 2 andγ rot ( Z + 1 N c T ) =

Original: (cid:0) Z + 1 N c T (cid:1) R T = ZR T + 1 N ( Rc ) T for some R ∈ SO (2) .
Revised: (cid:0) Z + 1 N c T (cid:1) R T = ZR T + 1 N ( Rc ) T for some R ∈ SO (2) .

Original: Evidentlyγ ( Z ) ∈
Revised: Evidentlyγ ( Z ) ∈

Original: [ Z ] T , meaning criterion Eq.
Revised: [ Z ] T , meaning criterion Eq.

Original: (3) is fulfilled.
Revised: (3) is fulfilled.

Original: Now, consider an arbitrary pair Z ∈ R N × 2 ,
Revised: Now, consider an arbitrary pair Z ∈ R N × 2 ,

Original: Z ′ ∈
Revised: Z ′ ∈

Original: [ Z ] T .
Revised: [ Z ] T .

Original: Since Z ∈ [ Z ] T , we must have Z ′ =
Revised: Since Z ∈ [ Z ] T , we must have Z ′ =

Original: ZR T + 1 N c T for some R ∈ SO (2) and c ∈ R 2 .
Revised: ZR T + 1 N c T for some R ∈ SO (2) and c ∈ R 2 .

Original: Recall that γ trans ( Z ) maps input Z to arepresentative by subtracting the first row from all other rows, i.e.
Revised: Recall that γ trans ( Z ) maps input Z to arepresentative by subtracting the first row from all other rows, i.e.

Original: [Equation]
Revised: [Equation]

Original: Further note that γ trans is a canonical map for translation invariance and therefore fulfills criterion
Revised: Further note that γ trans is a canonical map for translation invariance and therefore fulfills criterion

Original: Eq.
Revised: Eq.

Original: Therefore, we have γ trans ( Z ′ )
Revised: Therefore, we have γ trans ( Z ′ )

Original: =
Revised: =

Original: K (cid:0) ZR T + 1 N c T (cid:1) = K (cid:0) ZR T (cid:1) =
Revised: K (cid:0) ZR T + 1 N c T (cid:1) = K (cid:0) ZR T (cid:1) =

Original: ( KZ ) R T , wheresecond equality is due to criterion Eq.
Revised: ( KZ ) R T , wheresecond equality is due to criterion Eq.

Original: (4) and the last equality is due to associativity.
Revised: (4) and the last equality is due to associativity.

Original: Finally,note that since γ rot is a canonical map for translation invariance, we must have γ rot ( γ trans ( Z ′ ))
Revised: Finally,note that since γ rot is a canonical map for translation invariance, we must have γ rot ( γ trans ( Z ′ ))

Original: =
Revised: =

Original: γ rot (( KZ ) R T ) =
Revised: γ rot (( KZ ) R T ) =

Original: γ rot ( KZ ) =
Revised: γ rot ( KZ ) =

Original: γ rot ( γ trans ( Z )) .
Revised: γ rot ( γ trans ( Z )) .

Original: This shows that γ also fulfills criterion Eq.
Revised: This shows that γ also fulfills criterion Eq.

Original: (4),i.e.
Revised: (4),i.e.

Original: it is a canonical map for simultaneous translation and rotation invariance.
Revised: it is a canonical map for simultaneous translation and rotation invariance.

Original: D.7.2 Proof of certificate
Revised: D.7.2 Proof of certificate

Original: To derive our certificate, we shall combine the substitutions we used for the translation invariancecertificate (see Appendix D.4) with those we used for rotation invariance (see Appendix D.5).
Revised: To derive our certificate, we shall combine the substitutions we used for the translation invariancecertificate (see Appendix D.4) with those we used for rotation invariance (see Appendix D.5).

Original: Weshall see that this leads to an optimization problem that is almost identical to that for 2D rotationinvariance, save for the fact that we have anisotropic normal distributions.
Revised: Weshall see that this leads to an optimization problem that is almost identical to that for 2D rotationinvariance, save for the fact that we have anisotropic normal distributions.

Original: Like in Appendix D.4, themean and covariance of these anisotropic distributions naturally lead to a centering of the certificateparameters.
Revised: Like in Appendix D.4, themean and covariance of these anisotropic distributions naturally lead to a centering of the certificateparameters.

Original: Before proceeding, we need the following lemma:
Revised: Before proceeding, we need the following lemma:

Original: Lemma 12.
Revised: Lemma 12.

Original: Consider arbitrary vectors x , y ∈ R D and let x = 1 D (cid:80)
Revised: Consider arbitrary vectors x , y ∈ R D and let x = 1 D (cid:80)

Original: Dd =1 x d , y = 1 D (cid:80) Dd
Revised: Dd =1 x d , y = 1 D (cid:80) Dd

Original: =1 y d betheir averages.
Revised: =1 y d betheir averages.

Original: Then
Revised: Then

Original: [Equation]
Revised: [Equation]

Original: Proof.
Revised: Proof.

Original: We substract the left-hand side from the right-hand side
Revised: We substract the left-hand side from the right-hand side

Original: [Equation]
Revised: [Equation]

Original: where the last equality follows from the fact that (cid:80) Dd =1 y d
Revised: where the last equality follows from the fact that (cid:80) Dd =1 y d

Original: = D y .
Revised: = D y .

Original: Applying Theorem 4 with our canonical map γ ( Z ) =
Revised: Applying Theorem 4 with our canonical map γ ( Z ) =

Original: γ rot ( γ trans ( Z )) leads to optimization problem
Revised: γ rot ( γ trans ( Z )) leads to optimization problem

Original: [Equation]
Revised: [Equation]

Original: (72) where h ( Z )
Revised: (72) where h ( Z )

Original: = 1 indicates that Z is classified as y ∗ .
Revised: = 1 indicates that Z is classified as y ∗ .

Original: Substitution.
Revised: Substitution.

Original: First, we perform exactly the same substitution.
Revised: First, we perform exactly the same substitution.

Original: as in our proof for translationinvariance to eliminate γ trans (see Eq.
Revised: as in our proof for translationinvariance to eliminate γ trans (see Eq.

Original: (13) to Eq.
Revised: (13) to Eq.

Original: (14) in Appendix D.4).
Revised: (14) in Appendix D.4).

Original: That is, V = AZ with
Revised: That is, V = AZ with

Original: A = (cid:20) 

Revised: A = (cid:20)

Original: TN −
Revised: TN −

Original: [Equation]
Revised: [Equation]

Original: −
Revised: −

Original: [Equation]
Revised: [Equation]

Original: N −
Revised: N −

Original: [Equation]
Revised: [Equation]

Original: I N −(cid:21) .
Revised: I N −(cid:21) .

Original: [Equation]
Revised: [Equation]

Original: (cid:18) γ rot (cid:18)(cid:20) 

Revised: (cid:18) γ rot (cid:18)(cid:20)

Original: T
Revised: T

Original: V 2: (cid:21)(cid:19)(cid:19) 2 (cid:89) d =1 N (cid:0) V : ,d | A
Revised: V 2: (cid:21)(cid:19)(cid:19) 2 (cid:89) d =1 N (cid:0) V : ,d | A

Original: [Equation]
Revised: [Equation]

Original: : X : ,d , σ
Revised: : X : ,d , σ

Original: [Equation]
Revised: [Equation]

Original: A
Revised: A

Original: [Equation]
Revised: [Equation]

Original: : ( A
Revised: : ( A

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: (cid:1)
Revised: (cid:1)

Original: d V 2: := (cid:90)
Revised: d V 2: := (cid:90)

Original: R ( N − 1) × 2 h (cid:18) γ rot (cid:18)(cid:20) 

Revised: R ( N − 1) × 2 h (cid:18) γ rot (cid:18)(cid:20)

Original: T
Revised: T

Original: W (cid:21)(cid:19)(cid:19) η X ( W ) d W , where we have defined η X ( W ) =
Revised: W (cid:21)(cid:19)(cid:19) η X ( W ) d W , where we have defined η X ( W ) =

Original: (cid:81) 2 d =1 N (cid:0) W : ,d |
Revised: (cid:81) 2 d =1 N (cid:0) W : ,d |

Original: A 2: X : ,d , σ 2 A 2: ( A 2: ) T (cid:1) to be the marginaldistribution over R N × 2 .
Revised: A 2: X : ,d , σ 2 A 2: ( A 2: ) T (cid:1) to be the marginaldistribution over R N × 2 .

Original: Next, we eliminate γ rot by transforming into polar coordinates.
Revised: Next, we eliminate γ rot by transforming into polar coordinates.

Original: For this, let λ ( r , θ ) =[ r ⊙ cos( θ ) r ⊙ sin( θ )] be the function that maps from polar into Cartesian coordinates.
Revised: For this, let λ ( r , θ ) =[ r ⊙ cos( θ ) r ⊙ sin( θ )] be the function that maps from polar into Cartesian coordinates.

Original: [Equation]
Revised: [Equation]

Original: Recall from Appendix D.5 that γ rot rotates its argument such that its first non-zero row is aligned withthe first coordinate axis.
Revised: Recall from Appendix D.5 that γ rot rotates its argument such that its first non-zero row is aligned withthe first coordinate axis.

Original: Evidently, the first row of its argument is zero.
Revised: Evidently, the first row of its argument is zero.

Original: Due to the factor (cid:81) N − 1 n =1 r n ,the entire integrand is zero whenver any of the other rows are zero (recall our discussion of Eq.
Revised: Due to the factor (cid:81) N − 1 n =1 r n ,the entire integrand is zero whenver any of the other rows are zero (recall our discussion of Eq.

Original: (38)in Appendix D.5).
Revised: (38)in Appendix D.5).

Original: We may thus assume that γ rot always rotates by θ 1 :
Revised: We may thus assume that γ rot always rotates by θ 1 :

Original: [Equation]
Revised: [Equation]

Original: θ
Revised: θ

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: Since W (cid:55)→ (cid:20) 

Revised: Since W (cid:55)→ (cid:20)

Original: T
Revised: T

Original: W (cid:21) is an injective function from R ( N − 1) × 2 into R N × 2 , we may formulate anequivalent optimization problem over a family of variables ( h ′ ) indexed by R ( N − 1) × 2 , which leadsto expectation
Revised: W (cid:21) is an injective function from R ( N − 1) × 2 into R N × 2 , we may formulate anequivalent optimization problem over a family of variables ( h ′ ) indexed by R ( N − 1) × 2 , which leadsto expectation

Original: [Equation]
Revised: [Equation]

Original: Similarity to 2D rotation invariance.
Revised: Similarity to 2D rotation invariance.

Original: We now observe that this expectation is almost identicalto Eq.
Revised: We now observe that this expectation is almost identicalto Eq.

Original: (39) from Appendix D.5, the expectation we obtained after eliminating the canonical map for2D rotation invariance.
Revised: (39) from Appendix D.5, the expectation we obtained after eliminating the canonical map for2D rotation invariance.

Original: The only difference is that we have a distribution η X that is a product of twoanisotropic multivariate normal distributions, instead of µ X , which was a product of two isotropicmultivariate normal distributions.
Revised: The only difference is that we have a distribution η X that is a product of twoanisotropic multivariate normal distributions, instead of µ X , which was a product of two isotropicmultivariate normal distributions.

Original: However, our proof did not make any assumptions about theparameters of the distribution.
Revised: However, our proof did not make any assumptions about theparameters of the distribution.

Original: We may thus perform the same steps, starting at Eq.
Revised: We may thus perform the same steps, starting at Eq.

Original: (39), i.e. furthersubstitution, marginalization, application of the Neyman Pearson lemma and transformation backinto Cartesian coordinates, to show that
Revised: (39), i.e. furthersubstitution, marginalization, application of the Neyman Pearson lemma and transformation backinto Cartesian coordinates, to show that

Original: [Equation]
Revised: [Equation]

Original: with t ∈ R s.t.
Revised: with t ∈ R s.t.

Original: [Equation]
Revised: [Equation]

Original: (75) with κ defined as in Lemma 9.
Revised: (75) with κ defined as in Lemma 9.

Original: Likelihood ratio distribution.
Revised: Likelihood ratio distribution.

Original: Finally, recall from Lemma 9 that
Revised: Finally, recall from Lemma 9 that

Original: [Equation]
Revised: [Equation]

Original: We shall now use our knowledge about the distribution of W , as well as some simple algebra to showthat Eqs.
Revised: We shall now use our knowledge about the distribution of W , as well as some simple algebra to showthat Eqs.

Original: (74) and (75) are equivalent to evaluating the tight certificate for 2D rotation invariance,after centering X ′ and X .
Revised: (74) and (75) are equivalent to evaluating the tight certificate for 2D rotation invariance,after centering X ′ and X .

Original: Let B =
Revised: Let B =

Original: A 2: ( A 2: ) T .
Revised: A 2: ( A 2: ) T .

Original: First, we can absorb the parameter B into thesecond argument of κ :
Revised: First, we can absorb the parameter B into thesecond argument of κ :

Original: [Equation]
Revised: [Equation]

Original: Furthermore, we know from Eqs.
Revised: Furthermore, we know from Eqs.

Original: (74) and (75) that W ∼ η X or W ∼ η ′ X with η X ( W ) =
Revised: (74) and (75) that W ∼ η X or W ∼ η ′ X with η X ( W ) =

Original: (cid:81)d =1 N (cid:0) W | A 2: X , σ 2 A 2:
Revised: (cid:81)d =1 N (cid:0) W | A 2: X , σ 2 A 2:

Original: ( A 2: ) T (cid:1) .
Revised: ( A 2: ) T (cid:1) .

Original: Let us begin with the case that W ∼ η X .
Revised: Let us begin with the case that W ∼ η X .

Original: The random variable W is an affine transformation of random variable E ∈ R N × 2 that is distributed according toa zero-mean isotropic matrix normal distribution
Revised: The random variable W is an affine transformation of random variable E ∈ R N × 2 that is distributed according toa zero-mean isotropic matrix normal distribution

Original: [Equation]
Revised: [Equation]

Original: We may thus restate the probability from Eq.
Revised: We may thus restate the probability from Eq.

Original: (75) as
Revised: (75) as

Original: [Equation]
Revised: [Equation]

Original: Next
Revised: Next

Original: [Equation]
Revised: [Equation]

Original: we can absorb S into the second argument of κ :
Revised: we can absorb S into the second argument of κ :

Original: [Equation]
Revised: [Equation]

Original: We know from Eq.
Revised: We know from Eq.

Original: (26) of our derivation of the translation invariance certificate that
Revised: (26) of our derivation of the translation invariance certificate that

Original: [Equation]
Revised: [Equation]

Original: i.e. ( A 2: ) T B − 1 A 2: is the matrix that subtracts the mean from every vector it is multiplied with.
Revised: i.e. ( A 2: ) T B − 1 A 2: is the matrix that subtracts the mean from every vector it is multiplied with.

Original: Thus κ (cid:0) E + X ,
Revised: Thus κ (cid:0) E + X ,

Original: [Equation]
Revised: [Equation]

Original: A
Revised: A

Original: [Equation]
Revised: [Equation]

Original: B
Revised: B

Original: [Equation]
Revised: [Equation]

Original: where ˜ X =
Revised: where ˜ X =

Original: X −
Revised: X −

Original: [Equation]
Revised: [Equation]

Original: X .
Revised: X .

Original: Next, we can apply Lemma 12, which shows that computing the inner product between a centeredand an uncentered vector is identical to centering both vectors and then computing the inner product:
Revised: Next, we can apply Lemma 12, which shows that computing the inner product between a centeredand an uncentered vector is identical to centering both vectors and then computing the inner product:

Original: [Equation]
Revised: [Equation]

Original: Finally
Revised: Finally

Original: [Equation]
Revised: [Equation]

Original: inserting this result into Eq.
Revised: inserting this result into Eq.

Original: (76) and using linearity of expectation shows that
Revised: (76) and using linearity of expectation shows that

Original: [Equation]
Revised: [Equation]

Original: The case W ∼ η X ′ follows analogously.
Revised: The case W ∼ η X ′ follows analogously.

Original: We therefore have
Revised: We therefore have

Original: [Equation]
Revised: [Equation]

Original: with t ∈ R s.t.
Revised: with t ∈ R s.t.

Original: [Equation]
Revised: [Equation]

Original: This solution is identical to the one we derived for rotation invariance (see Eqs.
Revised: This solution is identical to the one we derived for rotation invariance (see Eqs.

Original: (52) and (53)in Appendix D.5), except that all occurrences of X have been replaced with the centered ˜ X andall occurrences of X ′ have been replaced with the centered ˜ X ′ .
Revised: (52) and (53)in Appendix D.5), except that all occurrences of X have been replaced with the centered ˜ X andall occurrences of X ′ have been replaced with the centered ˜ X ′ .

Original: Thus, performing the subsequentderivation steps from Appendix D.5 yields the desired certificate.
Revised: Thus, performing the subsequentderivation steps from Appendix D.5 yields the desired certificate.

Original: 3D
Revised: 3D

Original: Finally, we consider the case that base classifier g is invariant under group T
Revised: Finally, we consider the case that base classifier g is invariant under group T

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: SO
Revised: SO

Original: [Equation]
Revised: [Equation]

Original: d } , 1646 where e d is an indicator vector and d ∗ is the rotation axis.
Revised: d } , 1646 where e d is an indicator vector and d ∗ is the rotation axis.

Original: In the following, let ˜ A ∈ R N × D be matrix A ∈ R N × D after centering,
Revised: In the following, let ˜ A ∈ R N × D be matrix A ∈ R N × D after centering,

Original: i.e. ˜ A
Revised: i.e. ˜ A

Original: =
Revised: =

Original: A − 1 N A , where
Revised: A − 1 N A , where

Original: A ∈ R 1 × D are the column-wise averages.
Revised: A ∈ R 1 × D are the column-wise averages.

Original: Further let ⟨ A , B
Revised: Further let ⟨ A , B

Original: [Equation]
Revised: [Equation]

Original: F =
Revised: F =

Original: [Equation]
Revised: [Equation]

Original: = vec
Revised: = vec

Original: [Equation]
Revised: [Equation]

Original: ( B )be the Frobenius inner product.
Revised: ( B )be the Frobenius inner product.

Original: Theorem 10.
Revised: Theorem 10.

Original: Let g : R N × 3 → Y be translation invariant and invariant to elemental rotationsaround axis d ∗ and let H T be defined as in Eq.
Revised: Let g : R N × 3 → Y be translation invariant and invariant to elemental rotationsaround axis d ∗ and let H T be defined as in Eq.

Original: Then
Revised: Then

Original: [Equation]
Revised: [Equation]

Original: where I 0 is the modified Bessel function of the first kind and order 0 , and
Revised: where I 0 is the modified Bessel function of the first kind and order 0 , and

Original: [Equation]
Revised: [Equation]

Original: ∗
Revised: ∗

Original: [Equation]
Revised: [Equation]

Original: The certificate is identical to centering X and X ′ and evaluating the tight certificate for invarianceunder elemental rotation derived in Appendix D.6.
Revised: The certificate is identical to centering X and X ′ and evaluating the tight certificate for invarianceunder elemental rotation derived in Appendix D.6.

Original: D.8.1 Canonical map
Revised: D.8.1 Canonical map

Original: In the following, let γ erot be the canonical map for invariance under T erot := 1657 (cid:8) X
Revised: In the following, let γ erot be the canonical map for invariance under T erot := 1657 (cid:8) X

Original: (cid:55)→ XR T | R ∈ SO (3 , d ∗ ) (cid:9) , as defined in Appendix D.6.
Revised: (cid:55)→ XR T | R ∈ SO (3 , d ∗ ) (cid:9) , as defined in Appendix D.6.

Original: Further let γ trans be the canonical map for invariance under T trans := (cid:8)
Revised: Further let γ trans be the canonical map for invariance under T trans := (cid:8)

Original: X
Revised: X

Original: (cid:55)→ 1 N c T | c ∈ R 2 (cid:9) , as defined in Appendix D.4.
Revised: (cid:55)→ 1 N c T | c ∈ R 2 (cid:9) , as defined in Appendix D.4.

Original: Composing these two canonical maps yields a canonical map for simultaneous rotation and translationinvariance.
Revised: Composing these two canonical maps yields a canonical map for simultaneous rotation and translationinvariance.

Original: Lemma 13.
Revised: Lemma 13.

Original: The function γ ( Z ) =
Revised: The function γ ( Z ) =

Original: γ erot ( γ trans ( Z )) is a canonical map for simultaneous rotationand translation invariance.
Revised: γ erot ( γ trans ( Z )) is a canonical map for simultaneous rotationand translation invariance.

Original: Proof.
Revised: Proof.

Original: Identical to the proof of Lemma 10, replacing every occurence of “ SO (2) ” with SO (3 , d ∗ ) .
Revised: Identical to the proof of Lemma 10, replacing every occurence of “ SO (2) ” with SO (3 , d ∗ ) .

Original: D.8.2 Proof for certificate
Revised: D.8.2 Proof for certificate

Original: Like in Appendix D.6, we only consider the case d ∗ = 3 , the other cases follow analogously.
Revised: Like in Appendix D.6, we only consider the case d ∗ = 3 , the other cases follow analogously.

Original: As inour proof for simultaneous rotation and translation invariance in Appendix D.7, we first eliminateγ trans via the substitution we used for our translation invariance certificate.
Revised: As inour proof for simultaneous rotation and translation invariance in Appendix D.7, we first eliminateγ trans via the substitution we used for our translation invariance certificate.

Original: We then make the sameobservation as in our proof for elemental rotations: The optimization problem is identical to that for2D rotation invariance, save for an independent factor in the density function.
Revised: We then make the sameobservation as in our proof for elemental rotations: The optimization problem is identical to that for2D rotation invariance, save for an independent factor in the density function.

Original: We may thus reusemuch of the proof and only have to find the distribution of an independent likelihood ratio.
Revised: We may thus reusemuch of the proof and only have to find the distribution of an independent likelihood ratio.

Original: Applying Theorem 4 with our canonical map γ ( Z ) =
Revised: Applying Theorem 4 with our canonical map γ ( Z ) =

Original: γ rot ( γ trans ( Z )) leads to optimization problem
Revised: γ rot ( γ trans ( Z )) leads to optimization problem

Original: [Equation]
Revised: [Equation]

Original: where h ( Z )
Revised: where h ( Z )

Original: = 1 indicates that Z is classified as y ∗ .
Revised: = 1 indicates that Z is classified as y ∗ .

Original: Substitution.
Revised: Substitution.

Original: Performing the same steps as in Appendix D.7 to eliminate γ trans shows that
Revised: Performing the same steps as in Appendix D.7 to eliminate γ trans shows that

Original: [Equation]
Revised: [Equation]

Original: where we have defined η X ( W )
Revised: where we have defined η X ( W )

Original: = (cid:81)
Revised: = (cid:81)

Original: [Equation]
Revised: [Equation]

Original: =
Revised: =

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: d |
Revised: d |

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: X :
Revised: X :

Original: [Equation]
Revised: [Equation]

Original: d
Revised: d

Original: [Equation]
Revised: [Equation]

Original: σ 2 A 2: ( A 2: ) T (cid:1) to be the marginaldistribution over R N × 3 , with A = (cid:20)
Revised: σ 2 A 2: ( A 2: ) T (cid:1) to be the marginaldistribution over R N × 3 , with A = (cid:20)

Original: [Equation]
Revised: [Equation]

Original: Note that η X is the product of three independent multivariate normal distributions, i.e. all columnsare independent.
Revised: Note that η X is the product of three independent multivariate normal distributions, i.e. all columnsare independent.

Original: In the following, let η X : , :2 be the distribution of the first two dimensions and η X : , 3 1679 be the distribution of the third dimension, i.e.
Revised: In the following, let η X : , :2 be the distribution of the first two dimensions and η X : , 3 1679 be the distribution of the third dimension, i.e.

Original: [Equation]
Revised: [Equation]

Original: Similarity to invariance under elemental rotation.
Revised: Similarity to invariance under elemental rotation.

Original: Next, we transform into cylindrical coordinates.
Revised: Next, we transform into cylindrical coordinates.

Original: Because we are in the case d ∗ , this is equivalent to transforming Z : , :2 into polar coordinates (recallour discussion in Appendix D.6).
Revised: Because we are in the case d ∗ , this is equivalent to transforming Z : , :2 into polar coordinates (recallour discussion in Appendix D.6).

Original: For this, let λ ( r , θ ) =
Revised: For this, let λ ( r , θ ) =

Original: [ r ⊙ cos( θ ) r ⊙ sin( θ )] be the functionthat maps from polar into Cartesian coordinates.
Revised: [ r ⊙ cos( θ ) r ⊙ sin( θ )] be the functionthat maps from polar into Cartesian coordinates.

Original: E Z ∼ µ X [ h ( γ
Revised: E Z ∼ µ X [ h ( γ

Original: [Equation]
Revised: [Equation]

Original: Z ))]
Revised: Z ))]

Original: [Equation]
Revised: [Equation]

Original: This expectation is identical to Eq.
Revised: This expectation is identical to Eq.

Original: (73) from Appendix D.8, save for the independent factorη X : , 3 ( W : , 3 ) in the densitiy function.
Revised: (73) from Appendix D.8, save for the independent factorη X : , 3 ( W : , 3 ) in the densitiy function.

Original: We may thus perform the same derivations, i.e. further substi- 1686 tutions, marginalization, application of the Neyman Pearson lemma and transformation back into
Revised: We may thus perform the same derivations, i.e. further substi- 1686 tutions, marginalization, application of the Neyman Pearson lemma and transformation back into

Original: Cartesian coordinates, to show thatmin h ∈ H T
Revised: Cartesian coordinates, to show thatmin h ∈ H T

Original: [Equation]
Revised: [Equation]

Original: with t ∈ R s.t.
Revised: with t ∈ R s.t.

Original: [Equation]
Revised: [Equation]

Original: and ρ ( q
Revised: and ρ ( q

Original: [Equation]
Revised: [Equation]

Original: =
Revised: =

Original: I
Revised: I

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: Likelihood ratio distribution.
Revised: Likelihood ratio distribution.

Original: The last step of our derivation is to find the distribution of thelikelihood ratios in Eqs.
Revised: The last step of our derivation is to find the distribution of thelikelihood ratios in Eqs.

Original: (86) and (87).
Revised: (86) and (87).

Original: Recall that η : , 3 ( W : , :3 )
Revised: Recall that η : , 3 ( W : , :3 )

Original: =
Revised: =

Original: N (cid:0) W : , 3 | A 2: X : , 3 , σ 2 A 2: ( A 2: ) T (cid:1) .
Revised: N (cid:0) W : , 3 | A 2: X : , 3 , σ 2 A 2: ( A 2: ) T (cid:1) .

Original: Evidently, it is an affine transformation
Revised: Evidently, it is an affine transformation

Original: A 2: Z : , 3 of the random variable Z : , 3 ∈ R N which followsisotropic normal distribution µ X : , 3 ( Z : , 3 ) = N (cid:0) Z : , 3 | X : , 3 , σ 2 I n (cid:1) .
Revised: A 2: Z : , 3 of the random variable Z : , 3 ∈ R N which followsisotropic normal distribution µ X : , 3 ( Z : , 3 ) = N (cid:0) Z : , 3 | X : , 3 , σ 2 I n (cid:1) .

Original: We may thus restate Eqs.
Revised: We may thus restate Eqs.

Original: (86) 1692 and (87) asmin
Revised: (86) 1692 and (87) asmin

Original: [Equation]
Revised: [Equation]

Original: with t ∈ R s.t.
Revised: with t ∈ R s.t.

Original: [Equation]
Revised: [Equation]

Original: Next, we can compute the likelihood ratio.
Revised: Next, we can compute the likelihood ratio.

Original: Define B =
Revised: Define B =

Original: ( A 2: A 2: ) T and recall that X ′ =
Revised: ( A 2: A 2: ) T and recall that X ′ =

Original: X + ∆ .
Revised: X + ∆ .

Original: Then:
Revised: Then:

Original: [Equation]
Revised: [Equation]

Original: Now, recall from Eq.
Revised: Now, recall from Eq.

Original: (26) of our derivation of the translation invariance certificate that
Revised: (26) of our derivation of the translation invariance certificate that

Original: [Equation]
Revised: [Equation]

Original: i.e. ( A 2: ) T B − 1 A 2: is the matrix that subtracts the mean from every vector it is multiplied with.
Revised: i.e. ( A 2: ) T B − 1 A 2: is the matrix that subtracts the mean from every vector it is multiplied with.

Original: Further recall Lemma 12, i.e. the inner product between a centered and an uncentered vector isidentical to centering both vectors and then computing the inner product.
Revised: Further recall Lemma 12, i.e. the inner product between a centered and an uncentered vector isidentical to centering both vectors and then computing the inner product.

Original: Thus
Revised: Thus

Original: [Equation]
Revised: [Equation]

Original: The argument of the exponential function is an affine transformation of random variable Z : , 3 ,o = 12 σ 2 (cid:16) 2 ˜ ∆ T : , 3 Z : , 3 − || ˜ ∆ : , 3 || 22 − 2 ˜ ∆ : , 3 ˜ X : , 3 (cid:17) .
Revised: The argument of the exponential function is an affine transformation of random variable Z : , 3 ,o = 12 σ 2 (cid:16) 2 ˜ ∆ T : , 3 Z : , 3 − || ˜ ∆ : , 3 || 22 − 2 ˜ ∆ : , 3 ˜ X : , 3 (cid:17) .

Original: In Eq.
Revised: In Eq.

Original: (89), we have Z : , 3 ∼ µ X ′ : , 3 := N (cid:0) X : , 3 + ∆ : , 3 , σ 2 I N (cid:1) and thus o ∼ N ( u (1) , s ) with meanu (1) = 12 σ 2 (cid:16) 2 || ˜ ∆ : , 3 || 22 − || ˜ ∆ : , 3 || 22 (cid:17) =
Revised: (89), we have Z : , 3 ∼ µ X ′ : , 3 := N (cid:0) X : , 3 + ∆ : , 3 , σ 2 I N (cid:1) and thus o ∼ N ( u (1) , s ) with meanu (1) = 12 σ 2 (cid:16) 2 || ˜ ∆ : , 3 || 22 − || ˜ ∆ : , 3 || 22 (cid:17) =

Original: 12 σ 2 (cid:16) || ˜ ∆ : , 3 || 22 (cid:17) and variance s 2 = 1 σ 2 ˜ ∆ T : , 3 σ 2 I N ˜ ∆ : , 3 1 σ 2 =1 σ 2 || ˜ ∆ : , 3 || 22 .
Revised: 12 σ 2 (cid:16) || ˜ ∆ : , 3 || 22 (cid:17) and variance s 2 = 1 σ 2 ˜ ∆ T : , 3 σ 2 I N ˜ ∆ : , 3 1 σ 2 =1 σ 2 || ˜ ∆ : , 3 || 22 .

Original: Note that we have again used Lemma 12 in computing the inner product between ˜ ∆ : , 3 1704 and the mean of our random distribution.
Revised: Note that we have again used Lemma 12 in computing the inner product between ˜ ∆ : , 3 1704 and the mean of our random distribution.

Original: In Eq.
Revised: In Eq.

Original: (90) we have Z : , 3 ∼ µ X : , 3 := N (cid:0) X : , 3 , σ 2 I N (cid:1) and thus o ∼ N ( u (2) , s ) with mean u (2) =12 σ 2 (cid:16) −|| ˜ ∆ : , 3 ||(cid:17) = − 12 σ(cid:16) ||
Revised: (90) we have Z : , 3 ∼ µ X : , 3 := N (cid:0) X : , 3 , σ 2 I N (cid:1) and thus o ∼ N ( u (2) , s ) with mean u (2) =12 σ 2 (cid:16) −|| ˜ ∆ : , 3 ||(cid:17) = − 12 σ(cid:16) ||

Original: ˜ ∆ : , 3 ||(cid:17) and variance s 2 = 1 σ 2 || ˜ ∆ : , 3 || 22 .
Revised: ˜ ∆ : , 3 ||(cid:17) and variance s 2 = 1 σ 2 || ˜ ∆ : , 3 || 22 .

Original: Note that we have againused Lemma 12 in computing the inner product between ˜ ∆ : , 3 and the mean of our random distribution.
Revised: Note that we have againused Lemma 12 in computing the inner product between ˜ ∆ : , 3 and the mean of our random distribution.

Original: Therefore, we have
Revised: Therefore, we have

Original: [Equation]
Revised: [Equation]

Original: with t ∈ R s.t.
Revised: with t ∈ R s.t.

Original: [Equation]
Revised: [Equation]

Original: and ρ (
Revised: and ρ (

Original: [Equation]
Revised: [Equation]

Original: =
Revised: =

Original: I
Revised: I

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: which concludes the proof for our final tight certificate.
Revised: which concludes the proof for our final tight certificate.

Original: All of the discussed tight certificates, save for the one for translation invariance, are of the form
Revised: All of the discussed tight certificates, save for the one for translation invariance, are of the form

Original: [Equation]
Revised: [Equation]

Original: with t ∈ R s.t.
Revised: with t ∈ R s.t.

Original: [Equation]
Revised: [Equation]

Original: and
Revised: and

Original: [Equation]
Revised: [Equation]

Original: where g : R N × D → Y is our base classifier, µ X , µ X ′ are the clean and perturbed smoothingdistribution, υ (1) and υ (2) are distributions over some set S , and ρ : S → R is some arbitraryscalar-valued function.
Revised: where g : R N × D → Y is our base classifier, µ X , µ X ′ are the clean and perturbed smoothingdistribution, υ (1) and υ (2) are distributions over some set S , and ρ : S → R is some arbitraryscalar-valued function.

Original: As discussed in Section 6.3, we propose to compute a narrow probabilistic lower bound onmin
Revised: As discussed in Section 6.3, we propose to compute a narrow probabilistic lower bound onmin

Original: h ∈ H T
Revised: h ∈ H T

Original: Pr Z ∼ µ X ′
Revised: Pr Z ∼ µ X ′

Original: [ h ( Z ) =
Revised: [ h ( Z ) =

Original: y ∗ ]] by combining three confidence bounds.
Revised: y ∗ ]] by combining three confidence bounds.

Original: Before computing these bounds, we have to inspect the monotonicity of our certificate.
Revised: Before computing these bounds, we have to inspect the monotonicity of our certificate.

Original: Evidently, Pr V ∼ υ (1) [ ρ ( V ) ≤ t ] is monotonically decreasing in t .
Revised: Evidently, Pr V ∼ υ (1) [ ρ ( V ) ≤ t ] is monotonically decreasing in t .

Original: Furthermore, t is monotonicallydecreasing with p X ,y ∗ : If p X ,y ∗ decreases, then a smaller t is sufficient for fulfilling the constraint
Revised: Furthermore, t is monotonicallydecreasing with p X ,y ∗ : If p X ,y ∗ decreases, then a smaller t is sufficient for fulfilling the constraint

Original: Pr V ∼ υ (2) [ ρ ( V ) ≤ t ] .
Revised: Pr V ∼ υ (2) [ ρ ( V ) ≤ t ] .

Original: Thus, we can compute a lower bound on our certificate by
Revised: Thus, we can compute a lower bound on our certificate by

Original: lower-bounding p X ,y ∗ ,
Revised: lower-bounding p X ,y ∗ ,

Original: i
Revised: i

Original: e. p X ,y ∗ ≤ p X ,y ∗ ,
Revised: e. p X ,y ∗ ≤ p X ,y ∗ ,

Original: lower-bounding the t ∈ R that fulfills Pr V ∼ υ (2) [ ρ ( V ) ≤ t ] =
Revised: lower-bounding the t ∈ R that fulfills Pr V ∼ υ (2) [ ρ ( V ) ≤ t ] =

Original: p X ,y ∗ , i.e. t ≤ t
Revised: p X ,y ∗ , i.e. t ≤ t

Original: and then lower-bounding Pr V ∼ υ (1) [ ρ ( V ) ≤ t ] .
Revised: and then lower-bounding Pr V ∼ υ (1) [ ρ ( V ) ≤ t ] .

Original: The random variable 1 [ g ( Z ) =
Revised: The random variable 1 [ g ( Z ) =

Original: y ∗ ] is a Bernoulli random variable.
Revised: y ∗ ] is a Bernoulli random variable.

Original: Just like other randomizedsmoothing methods (e.g. [27]), we can lower-bound p X ,y ∗ by evaluating g on N 1 samples from µ Xand computing a binomial proportion confidence bound, such as the the Clopper-Pearson confidencebound [132].
Revised: Just like other randomizedsmoothing methods (e.g. [27]), we can lower-bound p X ,y ∗ by evaluating g on N 1 samples from µ Xand computing a binomial proportion confidence bound, such as the the Clopper-Pearson confidencebound [132].

Original: Lower-bounding t requires computing a lower bound on F − 1 ( p X ,y ∗ ) , where F − 1 is the quantilefunction of ρ ( V ) with V ∼ υ (2) .
Revised: Lower-bounding t requires computing a lower bound on F − 1 ( p X ,y ∗ ) , where F − 1 is the quantilefunction of ρ ( V ) with V ∼ υ (2) .

Original: A non-parametric lower confidence bound on this quantile can beconstructed by evaluating ρ ( V ) on N 2 samples from υ (2) and returning the largest order statistic
Revised: A non-parametric lower confidence bound on this quantile can beconstructed by evaluating ρ ( V ) on N 2 samples from υ (2) and returning the largest order statistic

Original: R ( n ) such that (cid:2) ρ ( V ) ≤ R ( n ) (cid:3) ≤ p X ,y ∗ holds with high probability (see Section 5.2.1.
Revised: R ( n ) such that (cid:2) ρ ( V ) ≤ R ( n ) (cid:3) ≤ p X ,y ∗ holds with high probability (see Section 5.2.1.

Original: of [133]and Algorithm 1 below).
Revised: of [133]and Algorithm 1 below).

Original: The random variable [ ρ ( V ) ≤ t ] is another Bernoulli random variable and can thus be lower-boundedby evaluating ρ ( V ) on N 3 samples from µ X ′ and computing a binomial proportion confidence bound.
Revised: The random variable [ ρ ( V ) ≤ t ] is another Bernoulli random variable and can thus be lower-boundedby evaluating ρ ( V ) on N 3 samples from µ X ′ and computing a binomial proportion confidence bound.

Original: We want all three confidence bounds to simultaneously hold with high probability 1 − α .
Revised: We want all three confidence bounds to simultaneously hold with high probability 1 − α .

Original: We ensurethis by using Holm-Bonferroni [125] correction to account for the multiple comparisons problem.
Revised: We ensurethis by using Holm-Bonferroni [125] correction to account for the multiple comparisons problem.

Original: In our case, this corresponds to computing the first bound with significance α , the second one withsignificance α / 2 and the last one with significance α / 3 .
Revised: In our case, this corresponds to computing the first bound with significance α , the second one withsignificance α / 2 and the last one with significance α / 3 .

Original: Algorithm 1 summarizes our certification procedure.
Revised: Algorithm 1 summarizes our certification procedure.

Original: L OWER C OUNF B OUND refers to the ClopperPearson lower confidence bound.
Revised: L OWER C OUNF B OUND refers to the ClopperPearson lower confidence bound.

Original: B IN PV ALUE ( n, N 2 , ≥ , p X ,y ∗ ) refers to the p -value of the Binomialtest with null-hypothesis that the success probability is greater or equal p X ,y ∗ .
Revised: B IN PV ALUE ( n, N 2 , ≥ , p X ,y ∗ ) refers to the p -value of the Binomialtest with null-hypothesis that the success probability is greater or equal p X ,y ∗ .

Original: Note that only thefirst confidence bound requires evaluating the base classifier g .
Revised: Note that only thefirst confidence bound requires evaluating the base classifier g .

Original: For the other two confidence boundsa large number of samples can be evaluated at little computational cost.
Revised: For the other two confidence boundsa large number of samples can be evaluated at little computational cost.

Original: [Table]
Revised: [Table]

Original: E Proof of Theorem 6
Revised: E Proof of Theorem 6

Original: Next, we prove that the post-processing-based certificate for rotation-invariance, i.e. invariance under 1746 T = (cid:8) X
Revised: Next, we prove that the post-processing-based certificate for rotation-invariance, i.e. invariance under 1746 T = (cid:8) X

Original: (cid:55)→ XR T | R ∈
Revised: (cid:55)→ XR T | R ∈

Original: SO
Revised: SO

Original: ( D ) (cid:9) is not tight, which we formalized in Section 6.3 as follows:
Revised: ( D ) (cid:9) is not tight, which we formalized in Section 6.3 as follows:

Original: Theorem 6.
Revised: Theorem 6.

Original: Let g : R N × D
Revised: Let g : R N × D

Original: → Y be rotation invariant and H T be defined as in Eq.
Revised: → Y be rotation invariant and H T be defined as in Eq.

Original: Furtherassume that ∄ R ∈
Revised: Furtherassume that ∄ R ∈

Original: SO
Revised: SO

Original: ( D ) :
Revised: ( D ) :

Original: X ′ R T =
Revised: X ′ R T =

Original: X
Revised: X

Original: and that p X ,y ∗ ∈ (0 , 1) .
Revised: and that p X ,y ∗ ∈ (0 , 1) .

Original: Then, for all R ∈ SO ( D ) :
Revised: Then, for all R ∈ SO ( D ) :

Original: [Equation]
Revised: [Equation]

Original: The right-hand side term is the optimal value of the black-box optimization problem evaluated forperturbed input X ′ R T , i.e.
Revised: The right-hand side term is the optimal value of the black-box optimization problem evaluated forperturbed input X ′ R T , i.e.

Original: [Equation]
Revised: [Equation]

Original: , with H = (cid:8) h : R N × D → Y | Pr Z ∼ µ X [ h ( Z ) =
Revised: , with H = (cid:8) h : R N × D → Y | Pr Z ∼ µ X [ h ( Z ) =

Original: y ∗ ] ≥ p X ,y ∗
Revised: y ∗ ] ≥ p X ,y ∗

Original: [Equation]
Revised: [Equation]

Original: The left-hand side term is the optimal value of the gray-box optimization problem evaluated for perturbed input X ′ .
Revised: The left-hand side term is the optimal value of the gray-box optimization problem evaluated for perturbed input X ′ .

Original: Note that H T is aset of rotation invariant classifiers.
Revised: Note that H T is aset of rotation invariant classifiers.

Original: In Appendix C.3 we have proven that rotation of the smoothingdistribution’s mean does not have an effect on the prediction probabilities of such classifiers, i.e.
Revised: In Appendix C.3 we have proven that rotation of the smoothingdistribution’s mean does not have an effect on the prediction probabilities of such classifiers, i.e.

Original: [Equation]
Revised: [Equation]

Original: Thus, we can prove Theorem 6 by proving the following, more general statement:
Revised: Thus, we can prove Theorem 6 by proving the following, more general statement:

Original: Lemma 14.
Revised: Lemma 14.

Original: Let g : R N × D
Revised: Let g : R N × D

Original: → Y be rotation invariant and H T be defined as in Eq.
Revised: → Y be rotation invariant and H T be defined as in Eq.

Original: Furtherassume that X
Revised: Furtherassume that X

Original: ′ ̸
Revised: ′ ̸

Original: = X and that p X ,y ∗ ∈ (0 , 1) .
Revised: = X and that p X ,y ∗ ∈ (0 , 1) .

Original: Then:
Revised: Then:

Original: [Equation]
Revised: [Equation]

Original: We can do so by showing that no optimal solution to the r.h.s.
Revised: We can do so by showing that no optimal solution to the r.h.s.

Original: black-box problem from Eq.
Revised: black-box problem from Eq.

Original: (95)is a feasible solution to the l.h.s.
Revised: (95)is a feasible solution to the l.h.s.

Original: gray-box problem.
Revised: gray-box problem.

Original: More formally, let H ∗ ⊆
Revised: More formally, let H ∗ ⊆

Original: H be the set of allclassifiers minimizing the r.h.s.
Revised: H be the set of allclassifiers minimizing the r.h.s.

Original: black box problem.
Revised: black box problem.

Original: We have H ∗ ⊆
Revised: We have H ∗ ⊆

Original: H and H T ⊆ H .
Revised: H and H T ⊆ H .

Original: By showing that 1761 H ∗ ∩ H T = ∅ , we prove that H T ⊆
Revised: By showing that 1761 H ∗ ∩ H T = ∅ , we prove that H T ⊆

Original: H \ H ∗ , i.e. all rotation invariant classifiers from H T yield strictlylarger optimal values.
Revised: H \ H ∗ , i.e. all rotation invariant classifiers from H T yield strictlylarger optimal values.

Original: Recall that an optimal solution to the r.h.s.
Revised: Recall that an optimal solution to the r.h.s.

Original: black-box problem in Eq.
Revised: black-box problem in Eq.

Original: (95) is given by the NeymanPearson lemma [104].
Revised: (95) is given by the NeymanPearson lemma [104].

Original: Later work on hypothesis testing has shown that the Neyman-Pearson lemmais not only a sufficient, but a necessary condition for optimality[134]: Every most powerful test mustfulfill the likelihood ratio inequalities, save for sets of zero measure.
Revised: Later work on hypothesis testing has shown that the Neyman-Pearson lemmais not only a sufficient, but a necessary condition for optimality[134]: Every most powerful test mustfulfill the likelihood ratio inequalities, save for sets of zero measure.

Original: For our formulation of the
Revised: For our formulation of the

Original: Neyman-Pearson lemma (see Lemma 2), this means that any classifier h that is an optimal solution tothe r.h.s.
Revised: Neyman-Pearson lemma (see Lemma 2), this means that any classifier h that is an optimal solution tothe r.h.s.

Original: black-box problem from Eq.
Revised: black-box problem from Eq.

Original: (95) must fulfill
Revised: (95) must fulfill

Original: [Equation]
Revised: [Equation]

Original: for some t ∈ R , save for sets of zero measure.
Revised: for some t ∈ R , save for sets of zero measure.

Original: Cohen et al.
Revised: Cohen et al.

Original: [27] have shown that if X ′ ̸
Revised: [27] have shown that if X ′ ̸

Original: = X andp ∈ (0 , 1) and µ X , µ X ′ are isotropic Gaussian distributions, then these likelihood ratio inequalitiescorrespond to a linear decision boundary, i.e. any optimal solution most fulfill
Revised: = X andp ∈ (0 , 1) and µ X , µ X ′ are isotropic Gaussian distributions, then these likelihood ratio inequalitiescorrespond to a linear decision boundary, i.e. any optimal solution most fulfill

Original: [Equation]
Revised: [Equation]

Original: [Equation]
Revised: [Equation]

Original: for some b ∈ R and W ∈ R N × D with w ̸ = 0 N,D , save for sets of zero measure.
Revised: for some b ∈ R and W ∈ R N × D with w ̸ = 0 N,D , save for sets of zero measure.

Original: Here, ⟨ A , B ⟩ F =vec( W ) T vec( Z ) is the Frobenius inner product.
Revised: Here, ⟨ A , B ⟩ F =vec( W ) T vec( Z ) is the Frobenius inner product.

Original: Therefore, we can prove Lemma 14 by showing that classifiers complying with Eq.
Revised: Therefore, we can prove Lemma 14 by showing that classifiers complying with Eq.

Original: (96) (save forsets of zero measure) are not rotation invariant, i.e. they are not feasible solutions to the gray-boxoptimization problem.
Revised: (96) (save forsets of zero measure) are not rotation invariant, i.e. they are not feasible solutions to the gray-boxoptimization problem.

Original: We first do this for the case that D is even and then for the case that D is odd.
Revised: We first do this for the case that D is even and then for the case that D is odd.

Original: Lemma 15.
Revised: Lemma 15.

Original: Assume that D = 2 k for some k ∈ N .
Revised: Assume that D = 2 k for some k ∈ N .

Original: Let h : R N × D
Revised: Let h : R N × D

Original: → Y be a classifier that fulfills
Revised: → Y be a classifier that fulfills

Original: [Equation]
Revised: [Equation]

Original: for some b ∈ R and w ∈ R N × D with w ̸ = 0 N · D , save for sets of zero measure.
Revised: for some b ∈ R and w ∈ R N × D with w ̸ = 0 N · D , save for sets of zero measure.

Original: Then, there is aninput ˆ Z and
Revised: Then, there is aninput ˆ Z and

Original: a rotation matrix R ∈
Revised: a rotation matrix R ∈

Original: SO ( D ) such that h ( ˆ Z ) ̸
Revised: SO ( D ) such that h ( ˆ Z ) ̸

Original: = h ( ˆ ZR T ) .
Revised: = h ( ˆ ZR T ) .

Original: Proof.
Revised: Proof.

Original: Case 1.
Revised: Case 1.

Original: Assume that b ≥ 0 .
Revised: Assume that b ≥ 0 .

Original: Consider the set S
Revised: Consider the set S

Original: = { Z | ⟨ W , Z ⟩ F < b } .
Revised: = { Z | ⟨ W , Z ⟩ F < b } .

Original: By construction, we have ∀ Z ∈ S : ⟨ W , Z ⟩ F + b < 0 1783 and ∀ Z ∈ S : ⟨ W , − Z ⟩ F + b > 0 .
Revised: By construction, we have ∀ Z ∈ S : ⟨ W , Z ⟩ F + b < 0 1783 and ∀ Z ∈ S : ⟨ W , − Z ⟩ F + b > 0 .

Original: There must be at least one ˆ Z ∈ S with h ( ˆ Z ) =
Revised: There must be at least one ˆ Z ∈ S with h ( ˆ Z ) =

Original: y ∗ and h ( ˆ Z )
Revised: y ∗ and h ( ˆ Z )

Original: ̸
Revised: ̸

Original: = y ∗ .
Revised: = y ∗ .

Original: Otherwise, all points Z ∈ Swould need to fulfill h ( Z ) ̸
Revised: Otherwise, all points Z ∈ Swould need to fulfill h ( Z ) ̸

Original: = y ∗ or h ( − Z ) =
Revised: = y ∗ or h ( − Z ) =

Original: y ∗ , in which case we would have sets of non-zeromeasure violating the likelihood ratio inequalities from Eq.
Revised: y ∗ , in which case we would have sets of non-zeromeasure violating the likelihood ratio inequalities from Eq.

Original: Note that − ˆ Z = ˆ Z ( − I N ) T and − I N ∈
Revised: Note that − ˆ Z = ˆ Z ( − I N ) T and − I N ∈

Original: SO
Revised: SO

Original: ( D ) , because − I N is orthonormal and det( − I N )
Revised: ( D ) , because − I N is orthonormal and det( − I N )

Original: = 1 ,due to D being even.
Revised: = 1 ,due to D being even.

Original: Thus, we have found an input ˆ Z and a rotation matrix R ∈
Revised: Thus, we have found an input ˆ Z and a rotation matrix R ∈

Original: SO ( D ) such thath ( Z )
Revised: SO ( D ) such thath ( Z )

Original: ̸
Revised: ̸

Original: = h ( ZR T ) .
Revised: = h ( ZR T ) .

Original: Case 2.
Revised: Case 2.

Original: Assume that b < 0 .
Revised: Assume that b < 0 .

Original: This case follows analogously by constructing the set S
Revised: This case follows analogously by constructing the set S

Original: ={ Z | ⟨ W , Z ⟩ F > b } and considering a point ˆ Z ∈ S with h ( ˆ Z )
Revised: ={ Z | ⟨ W , Z ⟩ F > b } and considering a point ˆ Z ∈ S with h ( ˆ Z )

Original: ̸ = y ∗ and h ( − ˆ Z )
Revised: ̸ = y ∗ and h ( − ˆ Z )

Original: =
Revised: =

Original: y ∗
Revised: y ∗

Original: Now, we consider the case that D is odd.
Revised: Now, we consider the case that D is odd.

Original: Here, − I N / ∈ SO ( D ) , because det( − I N ) = − 1 .
Revised: Here, − I N / ∈ SO ( D ) , because det( − I N ) = − 1 .

Original: Therefore, 1793 we will need to use slightly more complicated constructions.
Revised: Therefore, 1793 we will need to use slightly more complicated constructions.

Original: Lemma 16.
Revised: Lemma 16.

Original: Assume that D = 2 k − 1 for some k ∈ N .
Revised: Assume that D = 2 k − 1 for some k ∈ N .

Original: Let h : R N × D
Revised: Let h : R N × D

Original: → Y be a classifier thatfulfills
Revised: → Y be a classifier thatfulfills

Original: [Equation]
Revised: [Equation]

Original: for some b ∈ R and w ∈ R N × D with w ̸ = 0 N · D , save for sets of zero measure.
Revised: for some b ∈ R and w ∈ R N × D with w ̸ = 0 N · D , save for sets of zero measure.

Original: Then, there is aninput ˆ Z and
Revised: Then, there is aninput ˆ Z and

Original: a rotation matrix R ∈
Revised: a rotation matrix R ∈

Original: SO ( D ) such that h ( ˆ Z ) ̸
Revised: SO ( D ) such that h ( ˆ Z ) ̸

Original: = h ( ˆ ZR T ) .
Revised: = h ( ˆ ZR T ) .

Original: Proof.
Revised: Proof.

Original: In the following, let A = (cid:20)
Revised: In the following, let A = (cid:20)

Original: [Equation]
Revised: [Equation]

Original: I
Revised: I

Original: [Equation]
Revised: [Equation]

Original: (cid:21) .
Revised: (cid:21) .

Original: We have A ∈ SO ( D ) , because A is orthonormal and det( A )
Revised: We have A ∈ SO ( D ) , because A is orthonormal and det( A )

Original: = 1 , due to D being odd.
Revised: = 1 , due to D being odd.

Original: Case 1.
Revised: Case 1.

Original: Assume that b ≥ 0 .
Revised: Assume that b ≥ 0 .

Original: We know that W ̸ = 0 N,D .
Revised: We know that W ̸ = 0 N,D .

Original: Without loss of generality, assume that W : , 2: ̸ = 0 N, ( D − 1) , i.e. at least 1802 one of the the last D − 1 columns is non-zero.
Revised: Without loss of generality, assume that W : , 2: ̸ = 0 N, ( D − 1) , i.e. at least 1802 one of the the last D − 1 columns is non-zero.

Original: Consider the set S
Revised: Consider the set S

Original: = (cid:8) Z | W T : , 1 Z : , 1 ∈ [0 , 1] ∧ ⟨ W : , 2: , Z : , 2: ⟩ F < b − 1 (cid:9) .
Revised: = (cid:8) Z | W T : , 1 Z : , 1 ∈ [0 , 1] ∧ ⟨ W : , 2: , Z : , 2: ⟩ F < b − 1 (cid:9) .

Original: By construction, we have∀ Z ∈ S : ⟨ W , Z ⟩ F + b < 0 and ∀ Z ∈ S : (cid:10) W , ZA T (cid:11) F + b > 0 .
Revised: By construction, we have∀ Z ∈ S : ⟨ W , Z ⟩ F + b < 0 and ∀ Z ∈ S : (cid:10) W , ZA T (cid:11) F + b > 0 .

Original: There must be at least one ˆ Z ∈ S with h ( ˆ Z ) =
Revised: There must be at least one ˆ Z ∈ S with h ( ˆ Z ) =

Original: y ∗ and h ( ˆ ZA T )
Revised: y ∗ and h ( ˆ ZA T )

Original: ̸
Revised: ̸

Original: = y ∗ .
Revised: = y ∗ .

Original: Otherwise, all points Z ∈ Swould need to fulfill h ( Z ) ̸
Revised: Otherwise, all points Z ∈ Swould need to fulfill h ( Z ) ̸

Original: = y ∗ or h ( ZA T ) =
Revised: = y ∗ or h ( ZA T ) =

Original: y ∗ , in which case we would have sets of non-zeromeasure violating Eq.
Revised: y ∗ , in which case we would have sets of non-zeromeasure violating Eq.

Original: Case 2.
Revised: Case 2.

Original: Assume that b < 0 .
Revised: Assume that b < 0 .

Original: This case follows analogously by constructing the set S = 1809 (cid:8)
Revised: This case follows analogously by constructing the set S = 1809 (cid:8)

Original: Z | W T : , 1 Z : , 1 ∈ [ − 1 , 0] ∧ ⟨ W : , 2: , Z : , 2: ⟩ F > b + 1 (cid:9) and considering a point ˆ Z ∈ S with h ( ˆ Z ) ̸
Revised: Z | W T : , 1 Z : , 1 ∈ [ − 1 , 0] ∧ ⟨ W : , 2: , Z : , 2: ⟩ F > b + 1 (cid:9) and considering a point ˆ Z ∈ S with h ( ˆ Z ) ̸

Original: = y ∗and h
Revised: = y ∗and h

Original: ( A ˆ Z ) =
Revised: ( A ˆ Z ) =

Original: y ∗ .
Revised: y ∗ .

Original: Lemmas 15 and 16 combined prove Lemma 14, which in turn proves Theorem 6.
Revised: Lemmas 15 and 16 combined prove Lemma 14, which in turn proves Theorem 6.

Original: F Multi-class certificates
Revised: F Multi-class certificates

Original: In our discussions in Sections 3, 5 and 6, we have only considered binary certificates 4 .
Revised: In our discussions in Sections 3, 5 and 6, we have only considered binary certificates 4 .

Original: That is,prediction y ∗ of smoothed classifier is certifiably robust to a perturbed input X ′ if the probabilityof base classifier g predicting y ∗ under perturbed smoothing distribution µ X ′ is greater than theprobability of all other classes combined.
Revised: That is,prediction y ∗ of smoothed classifier is certifiably robust to a perturbed input X ′ if the probabilityof base classifier g predicting y ∗ under perturbed smoothing distribution µ X ′ is greater than theprobability of all other classes combined.

Original: That is, p X ′ ,y ∗ > 12 .
Revised: That is, p X ′ ,y ∗ > 12 .

Original: We have then, for different invariances,computed lower bounds p X ′ ,y ∗ ≤ p X ′ ,y ∗ .
Revised: We have then, for different invariances,computed lower bounds p X ′ ,y ∗ ≤ p X ′ ,y ∗ .

Original: If p X ′ ,y ∗ > 12 , then p X ′ ,y ∗ > 12 and the prediction iscertifiably robust.
Revised: If p X ′ ,y ∗ > 12 , then p X ′ ,y ∗ > 12 and the prediction iscertifiably robust.

Original: Alternatively, one can certify robustness by showing that y ∗ remains more likely than the secondmost likely class under the perturbed smoothing distribution [27], i.e. p X ′ ,y ∗ > max y ̸
Revised: Alternatively, one can certify robustness by showing that y ∗ remains more likely than the secondmost likely class under the perturbed smoothing distribution [27], i.e. p X ′ ,y ∗ > max y ̸

Original: =
Revised: =

Original: y ∗ p X ′ ,y .
Revised: y ∗ p X ′ ,y .

Original: Tothis end, one can compute a lower bound p X ′ ,y ∗ ≤ p X ′ ,y ∗ and an upper-bound max y ̸
Revised: Tothis end, one can compute a lower bound p X ′ ,y ∗ ≤ p X ′ ,y ∗ and an upper-bound max y ̸

Original: =
Revised: =

Original: y ∗ p X ′ ,y ≥max y ̸
Revised: y ∗ p X ′ ,y ≥max y ̸

Original: =
Revised: =

Original: y ∗ p X ′ ,y .
Revised: y ∗ p X ′ ,y .

Original: If p X ′ ,y ∗ > max y ̸
Revised: If p X ′ ,y ∗ > max y ̸

Original: =
Revised: =

Original: y ∗ p X ′ ,y , then robustness is guaranteed.
Revised: y ∗ p X ′ ,y , then robustness is guaranteed.

Original: Due to the monotonicityof randomized smoothing certificates w.r.t.
Revised: Due to the monotonicityof randomized smoothing certificates w.r.t.

Original: to the prediction probabilities, this can be achieved byshowing that p X ′ ,y ∗ > p X ′ ,y ′ , where y ′ =
Revised: to the prediction probabilities, this can be achieved byshowing that p X ′ ,y ∗ > p X ′ ,y ′ , where y ′ =

Original: argmax y ̸
Revised: argmax y ̸

Original: =
Revised: =

Original: y ∗ p X ,y is the second most likely class underthe clean smoothing distribution.
Revised: y ∗ p X ,y is the second most likely class underthe clean smoothing distribution.

Original: This second most likely class y ′ can be found via a binomial test,see [27, 135].
Revised: This second most likely class y ′ can be found via a binomial test,see [27, 135].

Original: The lower bound p X ′ ,y ∗ can be obtained using the same formulae discussed throughout the main text(e.g.
Revised: The lower bound p X ′ ,y ∗ can be obtained using the same formulae discussed throughout the main text(e.g.

Original: p X ′ ,y ∗ =
Revised: p X ′ ,y ∗ =

Original: Φ (cid:16) Φ − 1 ( p X ,y ′ ) − || ∆ || 2 σ (cid:17) for black-box randomized smoothing).
Revised: Φ (cid:16) Φ − 1 ( p X ,y ′ ) − || ∆ || 2 σ (cid:17) for black-box randomized smoothing).

Original: The upper boundp X ′ ,y ′ can be found via the Neyman-Pearson upper bound discussed in Appendix D.1:
Revised: The upper boundp X ′ ,y ′ can be found via the Neyman-Pearson upper bound discussed in Appendix D.1:

Original: Lemma 3 (Neyman-Pearson upper bound) .
Revised: Lemma 3 (Neyman-Pearson upper bound) .

Original: Let µ X ′ , µ X , be two continuous distributions over ameasurable
Revised: Let µ X ′ , µ X , be two continuous distributions over ameasurable

Original: set A such that, for all t ∈ R + ,
Revised: set A such that, for all t ∈ R + ,

Original: the set (cid:110) z | µ X ′ ( z ) µ X ( z ) =
Revised: the set (cid:110) z | µ X ′ ( z ) µ X ( z ) =

Original: t (cid:111) has measure zero.
Revised: t (cid:111) has measure zero.

Original: Consider anarbitrary label set Y , a specific class label y ∈ Y and scalar p ∈ [0 , 1] .
Revised: Consider anarbitrary label set Y , a specific class label y ∈ Y and scalar p ∈ [0 , 1] .

Original: Then
Revised: Then

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: [Equation]
Revised: [Equation]

Original: with h
Revised: with h

Original: [Equation]
Revised: [Equation]

Original: t
Revised: t

Original: [Equation]
Revised: [Equation]

Original: Note that it is identical to the lower bound, save for replacing " ≤ " with " ≥ " and vice-versa.
Revised: Note that it is identical to the lower bound, save for replacing " ≤ " with " ≥ " and vice-versa.

Original: The resulting black-box upper bound is p X ′ ,y ′ =
Revised: The resulting black-box upper bound is p X ′ ,y ′ =

Original: Φ (cid:16) Φ − 1 ( p X ,y ′ )
Revised: Φ (cid:16) Φ − 1 ( p X ,y ′ )

Original: + || ∆ || 2 σ (cid:17) .
Revised: + || ∆ || 2 σ (cid:17) .

Original: It is identical to theblack-box lower bound, save for replacing " − " with " + ".
Revised: It is identical to theblack-box lower bound, save for replacing " − " with " + ".

Original: Substituting both bounds into p X ′ ,y ∗ >max y ̸
Revised: Substituting both bounds into p X ′ ,y ∗ >max y ̸

Original: =
Revised: =

Original: y ∗ p X ′ ,y and applying the inverse-normal CDF Φ − 1 to both sides shows that the classifier iscertifiably robust if || ∆ || ≤ σ 2 (cid:0) Φ − 1 ( p X ′ ,y ∗ ) − Φ − 1 ( p X ′ ,y ′ )
Revised: y ∗ p X ′ ,y and applying the inverse-normal CDF Φ − 1 to both sides shows that the classifier iscertifiably robust if || ∆ || ≤ σ 2 (cid:0) Φ − 1 ( p X ′ ,y ∗ ) − Φ − 1 ( p X ′ ,y ′ )

Original: (cid:1) [27].
Revised: (cid:1) [27].

Original: Evidently, the post-processing-based versions of the multi-class certificates are identical to theirbinary counterparts, except that the radius of the underlying black-box certificate changes.
Revised: Evidently, the post-processing-based versions of the multi-class certificates are identical to theirbinary counterparts, except that the radius of the underlying black-box certificate changes.

Original: For the tight gray-box approach, upper bounds can be derived by going through the same derivationsas in Appendix D, but using the Neyman-Pearson upper bounds from Lemma 3, i.e. replacing " ≤ "with " ≥ " and vice-versa.
Revised: For the tight gray-box approach, upper bounds can be derived by going through the same derivationsas in Appendix D, but using the Neyman-Pearson upper bounds from Lemma 3, i.e. replacing " ≤ "with " ≥ " and vice-versa.

Original: (see Lemma 3).
Revised: (see Lemma 3).

Original: In the case of translation invariance, this results in upperbound Φ (cid:16)
Revised: In the case of translation invariance, this results in upperbound Φ (cid:16)

Original: Φ − 1 ( p X ,y ′ )
Revised: Φ − 1 ( p X ,y ′ )

Original: + || ∆ − 1 N ∆ || 2 σ (cid:17) ) .
Revised: + || ∆ − 1 N ∆ || 2 σ (cid:17) ) .

Original: In the cases involving rotation invariance, this results inbounds of the form
Revised: In the cases involving rotation invariance, this results inbounds of the form

Original: [Equation]
Revised: [Equation]

Original: i.e. the same certificates, but with “ ≥ t ” instead of “ ≤ t ”.
Revised: i.e. the same certificates, but with “ ≥ t ” instead of “ ≤ t ”.

Original: Our Monte Carlo certification procedure proposed in Appendix D.9 can also easily be adapted tocomputing an upper bound by replacing " ≤ " with " ≥ " and vice-versa.:
Revised: Our Monte Carlo certification procedure proposed in Appendix D.9 can also easily be adapted tocomputing an upper bound by replacing " ≤ " with " ≥ " and vice-versa.:

Original: [Table]
Revised: [Table]

Original: Note that we still need to lower-bound threshold t , because this increases the probability
Revised: Note that we still need to lower-bound threshold t , because this increases the probability

Original: Pr V ∼ υ (1) [ ρ ( V ) ≥ t ] and thus leads to more pessimistic, sound multi-class certificates.
Revised: Pr V ∼ υ (1) [ ρ ( V ) ≥ t ] and thus leads to more pessimistic, sound multi-class certificates.

Original: Finally note that, because the multi-class certificates are just a combination of two binary certificates,our evaluation of binary invariance-aware certificates in in Section 8 is a good indicator of theperformance of invariance-aware multi-class certificates.
Revised: Finally note that, because the multi-class certificates are just a combination of two binary certificates,our evaluation of binary invariance-aware certificates in in Section 8 is a good indicator of theperformance of invariance-aware multi-class certificates.

Original: G Inverse certificates
Revised: G Inverse certificates

Original: In Section 8.1, we evaluate inverse certificates, i.e. compute the smallest prediction probability p minfor which robustness can still be certified, given the remaining certificate parameters such as || ∆ || 2 1856 or ||
Revised: In Section 8.1, we evaluate inverse certificates, i.e. compute the smallest prediction probability p minfor which robustness can still be certified, given the remaining certificate parameters such as || ∆ || 2 1856 or ||

Original: X
Revised: X

Original: || 2 .
Revised: || 2 .

Original: In the case of the black-box randomized smoothing baseline, p min can be calculated by solvingΦ (cid:16) Φ ( − 1) ( p ) − || ∆ || 2 σ (cid:17)
Revised: In the case of the black-box randomized smoothing baseline, p min can be calculated by solvingΦ (cid:16) Φ ( − 1) ( p ) − || ∆ || 2 σ (cid:17)

Original: =
Revised: =

Original: 12 for p , which results in p min =
Revised: 12 for p , which results in p min =

Original: Φ (cid:16) ||
Revised: Φ (cid:16) ||

Original: ∆ || 2 σ (cid:17) .
Revised: ∆ || 2 σ (cid:17) .

Original: Similarly, the gray-box certificates for translation invariance has p min =
Revised: Similarly, the gray-box certificates for translation invariance has p min =

Original: Φ (cid:16) || ∆ − 1 N ∆ || 2 σ (cid:17) , where∆ ∈ R 1 × D are the column-wise averages.
Revised: Φ (cid:16) || ∆ − 1 N ∆ || 2 σ (cid:17) , where∆ ∈ R 1 × D are the column-wise averages.

Original: The tight gray-box certificates involving rotation invariance do not have closed-form analytic expressions.
Revised: The tight gray-box certificates involving rotation invariance do not have closed-form analytic expressions.

Original: Instead, they are of the form
Revised: Instead, they are of the form

Original: [Equation]
Revised: [Equation]

Original: with t ∈ R s.t.
Revised: with t ∈ R s.t.

Original: [Equation]
Revised: [Equation]

Original: Here, an inverse certificate can be computed by first finding a threshold t such that
Revised: Here, an inverse certificate can be computed by first finding a threshold t such that

Original: Pr V ∼ υ (1) [ ρ ( V ) ≤ t ] = 12 and then evaluating Pr V ∼ υ
Revised: Pr V ∼ υ (1) [ ρ ( V ) ≤ t ] = 12 and then evaluating Pr V ∼ υ

Original: [Equation]
Revised: [Equation]

Original: [ ρ ( V ) ≤ t ] .
Revised: [ ρ ( V ) ≤ t ] .

Original: That is,
Revised: That is,

Original: [Equation]
Revised: [Equation]

Original: with t ∈ R s.
Revised: with t ∈ R s.

Original: [Equation]
Revised: [Equation]

Original: To ensure a fair comparison with our baselines, we modify the Monte Carlo certification procedureproposed in Appendix D.9 to compute a probabilistic upper bound on p min that holds with highprobability 1 − α .
Revised: To ensure a fair comparison with our baselines, we modify the Monte Carlo certification procedureproposed in Appendix D.9 to compute a probabilistic upper bound on p min that holds with highprobability 1 − α .

Original: function I NVERSE P ROB C ERTIFY
Revised: function I NVERSE P ROB C ERTIFY

Original: ( υ (1) , υ (2) , N 1 , N 2 , α ) n ∗ ← min { n | B IN PV ALUE ( n, N 1 , ≤ , 0 . 5) < α }
Revised: ( υ (1) , υ (2) , N 1 , N 2 , α ) n ∗ ← min { n | B IN PV ALUE ( n, N 1 , ≤ , 0 . 5) < α }

Original: V (1) , . .
Revised: V (1) , . .

Original: , V ( N 2 ) ← S AMPLE ( υ (1) , N 1 )
Revised: , V ( N 2 ) ← S AMPLE ( υ (1) , N 1 )

Original: R (1) , . . .
Revised: R (1) , . . .

Original: , R ( N 2 ) ← S ORT A SCENDING ( ρ ( V (1) ) , . .
Revised: , R ( N 2 ) ← S ORT A SCENDING ( ρ ( V (1) ) , . .

Original: , ρ ( V ( N 2 ) ) )
Revised: , ρ ( V ( N 2 ) ) )

Original: t ← R ( n ∗ )
Revised: t ← R ( n ∗ )

Original: ▷ First bound
Revised: ▷ First bound

Original: V (1) , . .
Revised: V (1) , . .

Original: , V ( N 2 ) ← S AMPLE ( υ (2) , N 3 ) count 2 ← (cid:80) N 2 n =1 1 (cid:2)
Revised: , V ( N 2 ) ← S AMPLE ( υ (2) , N 3 ) count 2 ← (cid:80) N 2 n =1 1 (cid:2)

Original: ρ ( V ) ≤ t (cid:3) return U PPER C ONF B OUND ( count 2 , N 2 , 1 − α 2 )
Revised: ρ ( V ) ≤ t (cid:3) return U PPER C ONF B OUND ( count 2 , N 2 , 1 − α 2 )

Original: ▷ Second bound
Revised: ▷ Second bound

Original: Note that, different from Appendix D.9 we compute a probabilistic upper bound on threshold t byfinding the smallest order statistic R ( n ) such that (cid:2) ρ ( V ) ≤ R ( n ) (cid:3) ≥ 12 holds with high probability.
Revised: Note that, different from Appendix D.9 we compute a probabilistic upper bound on threshold t byfinding the smallest order statistic R ( n ) such that (cid:2) ρ ( V ) ≤ R ( n ) (cid:3) ≥ 12 holds with high probability.

Original: H Parameter space of the tight certificate for 2D rotation invariance
Revised: H Parameter space of the tight certificate for 2D rotation invariance

Original: Our tight certificates for 2D rotation invariance depend on || X
Revised: Our tight certificates for 2D rotation invariance depend on || X

Original: || 2 , || ∆ || 2 , as well as parametersϵ 1 = ⟨ X , ∆ ⟩ F , ϵ
Revised: || 2 , || ∆ || 2 , as well as parametersϵ 1 = ⟨ X , ∆ ⟩ F , ϵ

Original: [Equation]
Revised: [Equation]

Original: =
Revised: =

Original: [Equation]
Revised: [Equation]

Original: −
Revised: −

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: T , ∆
Revised: T , ∆

Original: [Equation]
Revised: [Equation]

Original: F , which capture the orientation of the perturbed pointcloud,relative to the clean point cloud.
Revised: F , which capture the orientation of the perturbed pointcloud,relative to the clean point cloud.

Original: In this section, we determine the feasible range of ϵ 1 and ϵ 2 and calculate the two pairs of valuescorresponding to adversarial rotations, which is relevant for our experiments in Section 8.1.
Revised: In this section, we determine the feasible range of ϵ 1 and ϵ 2 and calculate the two pairs of valuescorresponding to adversarial rotations, which is relevant for our experiments in Section 8.1.

Original: H.1 Feasible parameter range
Revised: H.1 Feasible parameter range

Original: Let ˆ X =
Revised: Let ˆ X =

Original: X
Revised: X

Original: ||
Revised: ||

Original: X
Revised: X

Original: || 2 .
Revised: || 2 .

Original: Then, parameters ϵ 1 and ϵ 2 can be equivalently stated as follows:
Revised: Then, parameters ϵ 1 and ϵ 2 can be equivalently stated as follows:

Original: [Equation]
Revised: [Equation]

Original: Note that vec( ˆ X ) and vec (cid:16) ˆ XR (cid:0) − π 2 (cid:1) T (cid:17) have norm 1 and are orthogonal to each other.
Revised: Note that vec( ˆ X ) and vec (cid:16) ˆ XR (cid:0) − π 2 (cid:1) T (cid:17) have norm 1 and are orthogonal to each other.

Original: Thus, theyare the first two elements of an orthonormal basis of R 2 N .
Revised: Thus, theyare the first two elements of an orthonormal basis of R 2 N .

Original: Parameters ϵ 1 and ϵ 2 are projections ofvec( || X || 2 ∆ ) onto the first two elements of this basis.
Revised: Parameters ϵ 1 and ϵ 2 are projections ofvec( || X || 2 ∆ ) onto the first two elements of this basis.

Original: Basis changes preserve the norm of vectors.
Revised: Basis changes preserve the norm of vectors.

Original: Thus, the values ϵ 1 , ϵ 2 must fulfill (cid:112) ϵ 21 + ϵ 22 ≤
Revised: Thus, the values ϵ 1 , ϵ 2 must fulfill (cid:112) ϵ 21 + ϵ 22 ≤

Original: ||
Revised: ||

Original: X
Revised: X

Original: || 2 · || ∆ || 2 .
Revised: || 2 · || ∆ || 2 .

Original: H.2 Adversarial rotations
Revised: H.2 Adversarial rotations

Original: Known rotation angle.
Revised: Known rotation angle.

Original: Consider an arbitrary X ∈ R N × 2 .
Revised: Consider an arbitrary X ∈ R N × 2 .

Original: Further consider an adversariallyperturbed X ′ =
Revised: Further consider an adversariallyperturbed X ′ =

Original: XR ( θ ) T that is the result of rotating all rows of X by angle θ .
Revised: XR ( θ ) T that is the result of rotating all rows of X by angle θ .

Original: Let ∆ =
Revised: Let ∆ =

Original: X ′ − X .
Revised: X ′ − X .

Original: Recall that, for any vector a , b ∈ R D , a T b T
Revised: Recall that, for any vector a , b ∈ R D , a T b T

Original: = ||
Revised: = ||

Original: a || 2 || b
Revised: a || 2 || b

Original: || 2 cos( ∠ ab ) .
Revised: || 2 cos( ∠ ab ) .

Original: Thereforeϵ 1 = ⟨ X ,
Revised: Thereforeϵ 1 = ⟨ X ,

Original: [Equation]
Revised: [Equation]

Original: where the last equality follows from the definition of the Frobenius norm.
Revised: where the last equality follows from the definition of the Frobenius norm.

Original: Similarly, we have
Revised: Similarly, we have

Original: [Equation]
Revised: [Equation]

Original: Unknown rotation angle.
Revised: Unknown rotation angle.

Original: Now assume that we know
Revised: Now assume that we know

Original: ||
Revised: ||

Original: X
Revised: X

Original: || 2 and || ∆
Revised: || 2 and || ∆

Original: || 2 , but do not know the rotation 1888 angle θ .
Revised: || 2 , but do not know the rotation 1888 angle θ .

Original: We have
Revised: We have

Original: [Equation]
Revised: [Equation]

Original: Evidently, such an adversarial rotation is only possible if || ∆ || ≤ 2 ||
Revised: Evidently, such an adversarial rotation is only possible if || ∆ || ≤ 2 ||

Original: X || , which corresponds to a 1890 rotation by angle π .
Revised: X || , which corresponds to a 1890 rotation by angle π .

Original: Solving for θ yields
Revised: Solving for θ yields

Original: [Equation]
Revised: [Equation]

Original: Inserting into our formulae for ϵ 1 and ϵ 2 and using that sin(arccos( a ))
Revised: Inserting into our formulae for ϵ 1 and ϵ 2 and using that sin(arccos( a ))

Original: = √ 1 − a 2 shows that
Revised: = √ 1 − a 2 shows that

Original: [Equation]
Revised: [Equation]

Original: ||
Revised: ||

Original: [Equation] [Equation]
Revised: [Equation] [Equation]