Original: where ∆ : ,d ∈ R is the average of column ∆ : ,d , ∆ ∈ R 1 × D are the column-wise averages of matrix∆ , the second equality is due to Lemma 7 and the third equality uses the definition of inner products.

Original: Taking the square root and using the definition of the Frobenius norm yields our desired result:

Original: [Equation] D.5 Tight certificate for invariance under rotation in 2D The following theorem is our tight certificate for rotation invariance in 2D, i.e. invariance under 1286 T = (cid:8) X (cid:55)→ XR T | R  ∈ SO (2) (cid:9) .
Revised: That is, the input space R N × D can be parameterized by a matrix ξ ( ψ ) ∈ R N × D that is transformed by a group element τ ( ω ) ∈ T .

Original: Different from the formulation in in Section 6.3, we also specifyits exact dependence on the four parameters || X || 2 , ||
Revised: As formalized in Eq.

Original: ∆ || , ϵ 1 and ϵ 2 :

Original: Theorem 7.

Original: Let g : R N × 2 → Y be rotation invariant and H T be defined as in Eq. Then [Equation] where I 0 is the modified Bessel function of the first kind and order 0 , and
Revised: (16), this alternative parameterization lets us neatly disentagle the effect of a group action, which is a translation and/or rotation of the coordinate system, from the (group-invariant) geometry of Z , i.e. ξ ( ψ ) .

Original: [Equation]

Original: ϵ 1 + || X

Original: || || X || 22 .
Revised: Canonical map.

Original: [-  , with ϵ 1 = ⟨ X , ∆ ⟩ F , ϵ 2 = (cid:68) XR (cid:0) − π 2 (cid:1) T , ∆ (cid:69) F . Here, ⟨ A , B ⟩ F = (cid:80) Nn =
 A Tn B Tn = vec( A ) T vec( B ) is-] the Frobenius inner product.
Revised: [-  , with ϵ 1 = ⟨ X , ∆ ⟩ F , ϵ 2 = (cid:68) XR (cid:0) − π 2 (cid:1) T , ∆ (cid:69) F . Here, ⟨ A , B ⟩ F = (cid:80) Nn =
 A Tn B Tn = vec( A ) T vec( B ) is-]Based on Proposition 1, we can define the following canonical map γ : R N × D

Original: Following our certification methodology, we present a canonical map for invariance under rotation in2D and then use it to derive our certificate via substitutions, marginalization and application of the

Original: Neyman-Pearson lemma.

Original: We additionally demonstrate that the worst-case invariant classifier is a

Original: Gaussian Haar integration kernel method (see Appendix D.5.3).

Original: D.5.1 Canonical map

Original: For our derivation, recall the definition of canonical maps. Definition 3 (Canonical map) . A canonical map for invariance under a group of transformations Tis a function γ : R N × D → R N × D with
Revised: → R N × D for Z = λ ( ω , ψ ) :

Original: [Equation]
Revised: [Equation]

Original: For rotation invariance, it is more convenient to work with polar coordinates. In the following, letλ : R N × R N → R N × 2 with λ ( r , θ ) = [ r ⊙ cos( θ ) r ⊙ sin( θ )] be the function that maps frompolar coordinates into Cartesian coordiantes. Here, ⊙ is the elementwise product and cos and sin areapplied element-wise.
Revised: Note that ( τ ( ω ))

Original: First, we define a rotation operator for matrices: Definition 4. The matrix rotation operator is the function rot : R N × 2 × R → R N × 2 withrot ( Z , a ) =
Revised: − 1 ∈ T is the inverse of group element τ ( ω ) ∈ T and not the inverse of the function τ .

Original: Z (cid:20) cos( a ) − sin( a ) sin( a ) cos( a ) (cid:21) T thatrotates all rows of matrix Z by angle a . We use the matrix rotation operator to define our canonical map, which rotates all matrices Z suchthat their first non-zero row is aligned with the first coordinate axis, i.e. takes on value 0 in the secondcolumn. Lemma 8.
Revised: Further note that by Eq.

Original: The function γ ( Z) = rot ( Z , − α ( Z )) with
Revised: (15), it holds for arbitrary Z = λ ( ω , ψ ) that

Original: [Equation]

Original: Z

Original: [Equation]
Revised: [Equation]

Original: where n ∗ = min { n | || Z n || 2 > 0 } is the first non-zero row of matrix Z , is  a  canonical map for 1312 rotation-invariance in 2 D. Proof. We can distinguish two cases. Case 1.
Revised: We can verify that γ is indeed a valid canonical map by testing the two criteria from Definition 3.

Original: All entries of Z are 0 , i.e. Z = 0 N, 2 . In this case || Z || 2 = 0 and thus γ ( Z ) 
Revised: Because γ ( Z ) applies a group action, we have γ ( Z ) ∈

Original: =

Original: Z = 0 N, 2 . The equivalence class of Z is [ Z ] T = (cid:8) 0 N, 2 R T | R ∈ SO (2) (cid:9) = { 0 N, 2 } . Both criteria Eqs. (3)and (4) for canonical maps are thus fulfilled.
Revised: [ Z ] T , where [ Z ] T is the orbit of Z w.r.t.

Original: Case 2. The matrix Z has non-zero entries. Let r , θ be a map of Z in polar coordinates, i.e. Z =λ ( r , θ ) . Let n ∗ be the index of the first non-zero row of Z .
Revised: T .

Original: Recall that rotation in cartesian coordinatescorresponds to translation of the polar coordinate angles and thus γ ( Z ) =

Original: λ ( r , θ − 1 N θ n ∗ ) .

Original: Criterion Eq.
Revised: Due to Eq.

Original: (3) is fulfilled, because by construction of γ we have γ ( Z ) = ZR T for some R ∈ SO (2) . Now, consider an arbitrary Z′ ∈
Revised: (16), we have ∀ Z

Original: [ Z ] T .

Original: By definition of equivalence class [ Z ] T , there must be somea ∈ R such that Z ′

Original: =

Original: λ ( r , θ + a 1 N ) .

Original: Note that rotation does not affect the radii, i.e. n ∗ is also theindex of the first non-zero row of Z ′ .

Original: Thus [Equation] ( Z ) , meaning criterion Eq. (4) is fulfilled as well.
Revised: , ∀ Z ∈ [ Z ] T : γ ( Z ) =

Original: D.5.2 Proof for certificate

Original: Our canonical map γ ( Z ) rotates all rows of an input Z ∈ R N × D such that the first non-zero rowis aligned with the first coordinate-axis.

Original: Similar to our proof for translation invariance, we will usesubstitutions to state our problem in an alternative parameter space that specifies the angle θ 1 of thefirst non-zero row and all angle differences to this row, i.e. θ n − θ 1 , as well as the radii r ∈ R N + .

Original: Wethen find the marginal distribution over the representatives in this alternative parameter space and applythe Neyman-Pearson lemma.

Original: Finally, we transform our solution back into Cartesian coordinates anduse the behavior of multivariate Normal distributions under affine transformations to prove our result.

Original: Because the calculations for deriving the marginal distribution are somewhat more arduous, we shallhandle them separately in Lemma 9, as to interfere with the rest of the proof.

Original: Lemma 9.

Original: Consider two multivariate normal distributions

Original: N ( M : , 1 , Σ ) , N ( M : , 2 , Σ ) with M ∈ 1337 R N × 2 that share the same symmetric covariance matrix Σ −∈ R N × D .

Original: Further consider arbitraryϕ ∈ R N and r ∈ R N .

Original: Define ξ : R × R N → R N with ξ ( α, ϕ ) =

Original: ϕ

Original: + 1 N α .

Original: Then

Original: [Equation] with κ ( Z , M , Σ ) =(cid:20) [Equation]
Revised: γ ( λ ( ω , z )) =

Original: where cos and sin are applied element-wise, ⊙ is the element-wise product and ∝ absorbs factorsthat are constant in M .

Original: Note that w ∈ R 2 and that I 0 is the modified Bessel function of the first kindand order 0 .

Original: Proof.

Original: We begin by identifying terms that that are constant in both M and α and can thus beeliminated, as well as terms that are constant in α and can thus be moved outside the integral.

Original: Wefirst eliminate the normalization constants:

Original: [Equation]

Original: with

Original: [Equation]

Original: Note that cand ccorrespond to −

Original: [Equation]

Original: ϕ

Original: [Equation]

Original: Σ [Equation] , ϕ )) and− 12 sin ( ξ ( α, ϕ ))
Revised: λ ( 0 , ψ ) = γ ( λ ( ω ′ , ψ ))

Original: T Σ − 1 sin ( ξ ( α, ϕ )) , respectively.c 1 is constant in α and can thus be moved outside the integral.c 2 and c 3 are not constant in α and must therefore remain inside the integral.c 4 + c 5 can be shown to be constant w.r.t.

Original: both M and α and can thus be eliminated.

Original: By thetrigonometric identity cos( a ) cos( b ) + sin( a ) sin( b ) = cos( a − b ) , we have

Original: [Equation] Recall that ξ ( α, ϕ ) =ϕ + 1 N α , i.e. “ + α ” appears in every row.
Revised: =

Original: Subtraction thus eliminates α forall combinations of n and m .

Original: Based on these observations, we may conclude that

Original: [Equation]

Original: Next, we use the trigonometric identites cos( a + b ) = cos( a ) cos( b ) − sin( a ) sin( b ) and sin( a + b ) =sin( a ) cos( b ) + cos( a ) sin( b ) to move α outside the inner products of c 2 and c 3 :

Original: [Equation]

Original: (32) and

Original: [Equation]

Original: Stacking the Σ − 1 M terms, we can express c 2 + c 3 as follows:

Original: [Equation]

Original: We can now substitute our results into Eq.

Original: (31) and simplify the integral using the fact that cos( α ) η 1 +sin( α ) η 2 = cos( α + ω ) (cid:112) η 21 + η 22 ) for some ω ∈ [0 , 2 π ] (see also Eq.

Original: of [130]).

Original: [Equation]

Original: where the second to last equality is due to the fact that we integrate over an entire period of cos andthe last equality uses the integral map of the modified Bessel function of the first kind with order

Original: 0(see Eq.

Original: 10.32.1 of [131]):

Original: [Equation]

Original: Restating (cid:112) η 21 + η 22 ) as

Original: [Equation]

Original: concludes our proof.

Original: Applying the canonical map.

Original: Now, we can turn to deriving the certificate itself.

Original: Applying Theorem 4with our canonical map from Lemma 8 shows thatmin

Original: [Equation]

Original: H T

Original: [Equation]

Original: Like in our previous proof, we notice that the above optimization problem does not depend on thespecific class predicted by h , but only on whether it is y ∗ or y .

Original: We thus formulate the equivalentproblem

Original: [Equation] [Equation] where h ( Z  )= 1 indicates that Z is classified as y ∗ .
Revised: γ ( Z ′ ) .

Original: Substitution.
Revised: Substitution.

Original: We begin by going from Cartesian to polar coordinates: As defined in the previoussection, let λ : R N × R N → R N × 2 with λ ( r , θ ) =

Original: [ r ⊙ cos( θ ) r ⊙ sin( θ )] be the function thatmaps from polar coordinates into cartesian coordiantes.

Original: [Equation]

Original: The r n in the last equality are due to the fact that λ transforms N rows from polar to Cartesiancoordinates, with each transformation having a Jacobian determinant r .

Original: Now, we would like to actually apply our canonical map γ , which performs a translation of the anglesθ .

Original: Recall from Lemma 8 that γ requires a case distinction to find the first non-zero row of λ ( r , θ ) .

Original: However, due to the factor (cid:81) Nn =1 r n , the entire integrand becomes zero whenever any row of λ ( r , θ )is zero.

Original: We can therefore always assume that we are in the case with r 1 > 0 , i.e. rotate by θ 1 , withoutinfluencing the value of the integral.

Original: Thus

Original: [Equation]

Original: Next, we apply a similar substitution to the one used in our translation certificate: Let ψ =

Original: Aθwith A =

Original: [-(cid:20) 
-]
Revised: [-(cid:20) 
-]

Original: TN

Original: [Equation]

Original: −

Original: [Equation] N [Equation] I N −(cid:21) and A − 1 = (cid:20)0 N − 1 1 N −
Revised: Next, we use Proposition 1 and the canonical map from Eq.

Original: I N (cid:21) .

Original: Note that A − 1 ( ψ − 1 N ψ 1 ) =

Original: (cid:20)ψ 2: (cid:21) ,where ψ 2: are the last N − 1 entries of ψ .

Original: In other words: Applying canonical map γ is equivalentto setting ψ 1 to 0 .

Original: Further note that det (cid:0) A − 1 (cid:1) =

Original: and that we have to adjust the integrationboundaries, because we are now integrating over angle differences.

Original: Thus

Original: [Equation]

Original: Marginalization.

Original: The function λ that maps from polar into Cartesian coordinates is 2 π -periodicw.r.t.

Original: its second argument and, by definition, we have A −

Original: [Equation] (cid:20) [Equation] [Equation]
Revised: (17) to bring the probabilities from our optimization problem in Eq.

Original: + [Equation] − 1 ψ 1 . This means thatfor any fixed ψ 1 , integrating ψ 2: over [ − ψ 1 , 2 π − ψ 1 ] N − 1 is equivalent to integrating over [0 , 2 π ] .
Revised: (14) into a form that is compatible with the Neyman-Pearson lemma.

Original: Further note that the term involving h does not depend on ψ , meaning it can be moved out.

Original: [Equation] r (41) This demonstrates that E Z ∼ µ X [ h (γ ( Z ))] =
Revised: To declutter the terms, we first introduce ˆ h ( Z ) as a shorthand for 1 [ h ( Z ) =

Original: E r , ψ 2: ∼ ν X (cid:20) h (cid:18) λ (cid:18) r , (cid:20)

Original: [Equation] ψ 2: (cid:21)(cid:19)(cid:19)(cid:21) with ν X being adistribution over R N × [0 , 2 π ] N − 1 with density function [Equation] dψ(43)
Revised: y ∗ ] :

Original: Inserting into our original optimization problem Eq.

Original: (34) and replacing ψ 2: with ω ∈ [0 , 2 π ] N − 1 toclarify that the problem no longer depends on the entire vector of angles ψ results in

Original: [Equation]
Revised: [Equation]

Original: .t

Original: [Equation]

Original: Note that λ is an injective function from R N × [0 , 2 π ] N into R N × D , meaning that each pair ( r , ω ) inthe support of ν X and ν X ′ corresponds to a distinct function value h ( Z ) .

Original: We may thus equivalentlyfind the worst-case classifier in R N × [0 , 2 π ] N − 1 : [Equation] Solving the lower-dimensional problem.
Revised: We then perform the substitution Z

Original: Applying the Neyman-Pearson lemma (Lemma 2) to theabove problem shows that its optimal value is

Original: [Equation]

Original: with h ∗ ( r , ω ) = 1 (cid:20) ν X ′ (r , ω[Equation] ν X ( r , 
Revised: = λ ( ω , ψ ) with ω ∈ Ω ⊆ R K and ψ ∈ Ψ ⊆ R N · D − K :

Original: [Equation]
Revised: [Equation]

Original:  and t ∈ R s.t. E r , ω [Equation] [Equation]
Revised: where J λ is the Jacobian of λ after vectorizing its domain and codomain.

Original: Transformation into Cartesian coordinates. To conclude our proof, we transform the expectationsin Eqs. (46) and (47) over this lower-dimensional parameter space back to our Cartesian space 1402 R N × D . Before doing so, we take a minor precaution to make the integration bounds easier to workwith.
Revised: Next, we apply the canonical map defined in Eq.

Original: We define a function ˆ h : R N × R N − 1 → { 0 , 1 } that allows h to be evaluated for arbitrarypolar coordiante angles, not just those in [0 , 2 π ] : For this, we let ˆ h ( r , ω )

Original: =

Original: h ∗ ( r , ( χ ( ω )) , whereχ ( ω ) = ω + 2 kπ s.t. ω + 2 k ∈ [0 , 2 π ] .
Revised: (17) to eliminate the group parameters ω from the term involving classifier ˆ h .

Original: Clearly E r [Equation] , ω )] . Therefore
Revised: This allows us to marginalize out ω :

Original: [Equation]
Revised: [Equation]

Original: [Equation] The second equality uses the definition of ν X from Eq. (43) and the last equality combines theintegration bounds of the two integrals.
Revised: where ν X ( ψ ) = (cid:82) Ω µ X ( λ ( ω , ψ )) · | det( J λ ( ω , ψ )) | d ω is the marginal density of parameters ψ .

Original: We now apply the substitution θ =
Revised: Finally, we can insert our result into Eq.

Original: A − 1 (cid:20) β ω (cid:21) with A − 1 = (cid:20)0 N − 1 1 N − I N (cid:21) to transform back intostandard polar coordinates. We then simplify the integration bounds by using the fact that, for anyfixed θ 1 all terms of theintegrand are 2 π -periodic in θ 2: – similar to how we simplified the integrationbounds when transforming into our alternative parameter space in Eqs. (40) and (41):
Revised: (14) to obtain the simplified optimization problem

Original: [Equation]
Revised: [Equation]

Original: Finally, recall from Eq.
Revised: Applying the Neyman-Pearson lemma. Eq.

Original: (47) that ˆ h depends on the likelihood ratios of the marginal distributionsν X ′ and ν X . Lemma 9 provides a closed-form expression for these likelihood ratios, up to constantfactors in X ′ and X .
Revised: (21) is almost in the form required by the NeymanPearson lemma. But, unlike in Lemma 5, we optimize over a function defined on R N × D while only evaluating it on a subset of R N × D , namely λ ( 0 , Ψ) .

Original: This closed-form expression lets us express the likelihood ratio as a function of λ ( r , θ )  .[Equation] [Table]
Revised: To resolve this mismatch, it is convenient to treat h as a family of variables ( h Z ) indexed by R N × D .

Original: [Equation] [Table]
Revised: As specified in Proposition 1, λ is injective almost everywhere, meaning that each tuple ( 0 , ψ ) indexes a distinct variable h λ ( 0 , ψ ) , save for sets of measure zero, which do not influence the objective and constraint in Eq.

Original: [Equation] [Table] [Equation] [Table]
Revised: Therefore, we may equivalently optimize over a family of variables (˜ h ψ ) indexed by Ψ :

Original: [Equation]
Revised: [Equation]

Original: with κ : R N × D × R N × D × R N × N → R 2 defined as in Lemma 9, I 0 being the modified Besselfunction of the first kind and order 0 and c being constant in r and θ .
Revised: According to Lemma 5, the minimizer is a function that classifies ψ as y ∗ iff ˜ h ∗ ( ψ ) = 1 with

Original: In the second equality, we havecancelled out the the r n and used the fact that λ is 2 π -periodic in its second argument to add 1 N θ 1without changing the value of the integral. Inserting back into Eq. (49) and transforming into Cartesian coordinates shows that
Revised: [Equation] with κ ∈ R + such that

Original: [Equation]
Revised: [Equation]

Revised: Consequently, the optimum of our original problem Eq.

Revised: (21) is given by any function h ∗ ( Z ) with h ∗ ( λ ( 0 , ψ ))

Revised: =

Original: Inserting into the optimal solution of  our lower-dimensional optimization problem (Eqs.
Revised: ˜ h ( ψ ) , i.e. we can use an arbitrary classifier for all parts of the domain that do not appear in the probability terms of Eq. We make the following proposition for our choice of worst-case classifier h ∗ ( Z ) : Proposition 2. Consider marginal density ν X ( ψ )

Original: (46) and (47))shows that
Revised: = (cid:82) Ω µ X ( λ ( ω , ψ )) · | det( J λ ( ω , ψ )) | d ω . Let η be a right Haar measure on group T ∈ { T ( D ) , SO ( D ) , SE ( D ) } . For SO ( D ) and SE ( D ) , let D ∈ { 2 , 3 } .

Revised: Define the group-averaged kernel

Original: [Equation]
Revised: [Equation]

Original: t ∈ R s.t.
Revised: Then,

Original: [Equation]
Revised: [Equation]

Original: Note that we can absorb the constant c into the threshold t . 
Revised: where ∝ absorbs factors that are constant in ψ . Assuming Proposition 2 holds, we have by transitivity of the previous equalities that [Equation]

Original: [Equation]
Revised: [Equation]

Original: 52) witht ′ ∈ R s.t.
Revised: with

Original: [Equation]
Revised: [Equation]

Original: If t ′ fulfills the conditions in Eq.
Revised: and κ [Equation] R [Equation]

Original: (53), then t = t ′ c
Revised: such that [Equation]

Original: fulfills the conditions in Eq.
Revised: Resubstitution. Next, we need to transform the expectations w.r [Equation] t. marginal distribution ν X over Ψ ⊆ R N · D − K back into an expectation w.r

Original: Likelihood ratio distribution. Finally, recall from Lemma 9 that [Equation] 
Revised: [Equation] t. our original smoothing distribution µ X over R N × D .

Original: Inside the probability terms appearing in Eqs.
Revised: Applying the steps from Eq.

Original: (52) and (53), we have [Equation] and
Revised: (19) to Eq. (20) in reverse order shows that

Original: [Equation]
Revised: [Equation]

Original: where R (cid:0) − π 2 (cid:1) is the rotation matrix that rotates clockwise by angle π 2 . In Eq. (52), the random variable Z follows distribution µ X ′ . Thus, vec ( Z ) ∼ N (cid:0) vec( X ′ ) , σ 2 I 2 N (cid:1) .
Revised: Exploiting invariance.

Original: Therefore, the random variable q ∈ R 4 = (cid:20) κ (cid:0) Z , X ′ , σ 2 I N (cid:1) κ (cid:0) Z , X , σ 2 I N (cid:1)(cid:21) , which is an affine transformation ofvec( Z ) follows distribution N (cid:0) m (1) , S (cid:1) with m (1) = CX ′ and S = σ 2 CC T , where [Equation]
Revised: Finally, we use the fact that our worst-case classifier h ∗ is invariant under group T to eliminate the canonical map γ from the expectation.

Original: Similarly, in Eq.
Revised: Recall from Eq.

Original: (53), the random variable Z follows distribution µ X . Therefore, the random variableq ∈ R 4 = (cid:20) κ (cid:0) Z , X ′ , σ 2 I N (cid:1) κ (cid:0) Z , X , σ 2 I N (cid:1) (cid:21) , which is an affine transformation of vec( Z ) follows distribution N (cid:0) m (2) , S (cid:1) with m (2) = CX and S =
Revised: (17) that, for any

Original: σ 2 CC T . Computing the matrix-vector and matrix-matrix products and using the definition that X ′ =X + ∆shows that
Revised: Z = λ ( ω , ψ ) , we defined γ ( Z ) =

Original: [Equation] with ϵ 1 = ⟨ X , ∆ ⟩ F , ϵ 2 = (cid:68) XR (cid:0) − π 2 (cid:1) T , ∆ (cid:69) F . This concludes our proof.
Revised: ( τ ( ω ))

Original: In Section 6.3, we claimed that the worst-case classifier, i.e. the minimizer ofmin h ∈ H T Pr Z ∼ µ ′ X [ h ( Z ) = y ∗ ] was a Gaussian Haar integration kernel method [16], i.e.  a functionthat predicts y ∗ if and only if β ( X ′ , Z ) β ( X , Z ) ≤ t with β ( X , Z ) =
Revised: − 1 ◦ Z , i.e. the canonical map lets a group element act on

Original: [-(cid:82) 2 π 0 exp (cid:0) 1 σ 
 (cid:10) X , ZR ( α ) [Equation] where R ( α ) ∈ SO (2) is the matrix that rotates counter-clockwise by angle α-] .
Revised: [-(cid:82) 2 π 0 exp (cid:0) 1 σ 
 (cid:10) X , ZR ( α ) [Equation] where R ( α ) ∈ SO (2) is the matrix that rotates counter-clockwise by angle α-]Z .

Original: Here, we verify this claim. In fact, we havealready implicitly done so in the derivation of ourcertificate. Recall from Eq. (52) that
Revised: Since η is a right Haar measure, we have

Original: [Equation]
Revised: [Equation]

Original: for some t ∈ R . Let r , θ ∈ R N be the radii and angles of Z , i.e. Z = λ ( r , θ ) . During our proofof Lemma 9, we have shown that 
Revised: where the second equality follows from the fact that ◦ is a group action The third equality holds because η is a right Haar measure, meaning we can make the substitution u

Original: [Equation] 54) with cos( α ) λ ( r , ϕ ) T : , 1 1 σ 2 X : , 1 − sin( α ) λ ( r , ϕ ) T :
Revised: = t · ( τ ( ω )) − 1 ) without having to change the measure. Thus, we have

Original: [Equation]
Revised: [Equation]

Original: X [Equation] 55) and sin( α ) λ ( r , ϕ ) T : , 1 1 σ 2 X : , 2 + cos( α ) λ ( r , ϕ ) T : , 2 1 σ 2 X :
Revised: Combined with Eq.

Original: [Equation] [Equation] see Eqs. (31) to (33) with X substituted for M and 1 σ 2 I N substituted for Σ − 1 .
Revised: (26) and Eqs.

Original: We have omittedterms that are constant and can thus be absorbed into classification threshold t . By substituting Z for λ ( r , θ ) and explicitly writing out the inner products in Eqs. (55) and (56) assummations over the N rows, Eq. (54) can be equivalently expressed as
Revised: (24) and (25), this proves that the optimal value of the original variance-constrained problem is

Original: [Equation]
Revised: [Equation]

Original: Since ⟨ A , B ⟩ F = (cid:80) Nn =1 (cid:0) A Tn B n (cid:1) , the above is in fact identical to
Revised: ) with

Original: [Equation]
Revised: [Equation]

Original: D.6 Tight certificate for invariance under elemental rotation in 3D

Original: Next, we generalize our certificate for rotation invariance in 2D to invariance under T = 1455 (cid:8)

Original: X (cid:55)→ XR T | R ∈ SO (3 , d ∗ ) ∈ R 3 (cid:9) , with SO (3 , d ) = { R ∈ SO (3) | R d ∗ = e d } , where e d isan indicator vector and d ∗ ∈ { 1 , 2 , 3 } is the rotation axis. In the following, let A : , − d ∗ ∈ R N × 2 be the matrix composed of the two columns thatare not A : ,d ∗ . Theorem 8.
Revised: and κ ∈ R + such that

Original: Let g : R N × 3 → Y be invariant to elemental rotations around axis d ∗ and let H T bedefined as in Eq.

Original: Then

Original: [Equation]

Original: [Equation]
Revised: [Equation]

Original: with t ∈ R s.t. [Equation] [Equation] and ρ ( q ) =
Revised: The last thing we need to do in order to conclude our proof is verify that Propositions 1 and 2 hold for each of the considered groups T .

Original: I [Equation] [Equation] where I 0 is the modified Bessel function of the first kind and order 0 , and 
Revised: F.3.1 Verifying Propositions 1 and 2 for translation invariance

Original: [Equation]

Original:   ϵ 1 + ϵ− ϵ 2 ϵ 

Original:  , Σ =[Equation]   [Equation]
Revised: For T = T ( D ) , we define λ ( ω , ψ ) =

Original: 1 +

Original: [Equation] ϵ 1 + [Equation] ϵ 22 ϵ 1 +
Revised: τ ( ω ) ◦ ξ ( ψ ) with τ : R D → T ( D ) , ξ : R ( N − 1) D

Original: [Equation]

Original: 3 +

Original: [Equation] − ϵϵ 1 + ϵ [Equation] 1 +
Revised: → R N × D and

Original: [Equation]
Revised: [Equation]

Original: − ϵϵ ϵϵ 1 + ϵ ϵ  .

Original: [-with ϵ 1 = (cid:68) ˜ X , ˜ ∆ (cid:69) F , ϵ 2 = (cid:68) ˜ XR (cid:0) − π 2 (cid:1) T , ˜ ∆ (cid:69) F ϵ 
-]
Revised: [-with ϵ 1 = (cid:68) ˜ X , ˜ ∆ (cid:69) F , ϵ 2 = (cid:68) ˜ XR (cid:0) − π 2 (cid:1) T , ˜ ∆ (cid:69) F ϵ 
-]

Original: =

Original: ||

Original: X : ,  − [-d ∗ || 
 and ϵ 
 = || X-] : , − d ∗ || 22 .
Revised: (cid:21) , where vec − [-d ∗ || 
 and ϵ 
 = || X-]1 : R ( N − 1) D

Original: The certificate is very similar to that from Appendix D.5, but additionally depends on the amount ofperturbation || ∆ : ,d ∗ || 2 along perturbation axis d ∗ .

Original: It further differs from the previous certificate in 1464 that the probabilities in Eqs.

Original: (58) and (59) involve a fifth normally distributed random variable o ∈ R . D.6.1 Canonical map The canonical map differs from that for rotations in 2D in that we have to make case distinctionsbetween d ∗ = 1 , d ∗ = 2 and d ∗ = 3 .
Revised: → R ( N − 1) × D reshapes an input vector into a matrix.

Original: We shall formulate these case distinctions in a way that makesthe case d ∗ = 3 as similar as possible to the canonical map for rotation invariance in 2D, which willhelp simplify the proof of our certificate.

Original: For our derivation, recall the definition ofcanonical maps. Definition 3 (Canonical map) . A canonical map for invariance under a group of transformations Tis a function γ : R N × D → R N × D with
Revised: Due to the definition of group action ◦ from Appendix C.1, we have λ

Original: [Equation]
Revised: [Equation]

Original: For rotation invariance, it is more convenient to work with cylindrical coordinates. In the following,let l : R N × R N × R N × { 1 , 2 , 3 } → R N × 3 be the function that maps from cylindrical coordinatesinto Cartesian coordinates with 
Revised: with all-ones vector 1 N ∈ R N . In other words: We represent each element of R N × D as a matrix whose first row is zero, translated by a vector ω . The function λ is evidently a differentiable bijection with inverse

Original: [Equation]
Revised: [Equation]

Original: [Equation]

Original:  and λ ( r , θ ) = [ r ⊙ cos( θ ) r ⊙ sin( θ ) ] defined like in Appendix D.5. Next, we define an elemental rotation operator for matrices: Definition 5.
Revised: meaning it is surjective and injective.

Original: The elemental rotation operator is the function erot : R N × 3 × R × { 1 , 2 , 3 } → R N × 2witherot( Z , a, d ) =   Z   0 cos( a ) − sin( a ) sin( a ) cos( a )  
Revised: Furthermore, we have τ ( 0 D ) =

Original: T if d = Z   cos( a ) sin( a )   − sin( a ) 0 cos( a )   T if d =
Revised: 0 D , with 0 D being the identity element of T ( D ) .

Original: Z   cos( a ) − sin( a )sin( a ) cos( a )    

Original: 

Original: T if d =that rotates all rows of matrix Z by angle a around axis d . We use the elemental matrix rotation operator to define our canonical map, which rotates an inputmatrix Z around axis d ∗ such thatthe first row n ∗ with non-zero radius r n ∗ takes on angle 0 : Lemma 10.
Revised: Lastly, we have for all t ∈ T ( D ) , ω ∈ R D , ψ ∈ R ( N − 1) D that

Original: The function γ ( Z ) =
Revised: [Equation] Thus, all conditions from Proposition 1 are fulfilled. To verify Proposition 2, we need to show that [Equation]

Original: erot ( Z  , − α
Revised: with marginal density ν X ( ψ ) = (cid:82) R D µ X ( λ ( ω , ψ )) · | det( J λ ( ω , ψ )) | d ω and Haar integral β X ( Z ) = (cid:82) t ∈ R D exp (cid:0) ⟨ t ◦ Z , X ⟩ F / σ 2 (cid:1) dη ( t ) , where η is an arbitrary right Haar measure on translation group T ( D ) = R D .

Original:  ( Z ) , d ∗ ) withα ( Z ) = 
Revised: Firstly, we see that | det( J λ ( ω , ψ )) | = 1 , since we are only performing translations. Thus, [Equation]

Original: (cid:26)if || Λ( Z , d ∗ ) ||
Revised: where ∝ absorbs factors that are constant in ω . In the above equalities we have first inserted the definition of our isotropic matrix normal smoothing distribution, then removed constant terms, then expressed the product of exponential functions more compactly using the Frobenius inner product and finally used that, by definition, λ ( ω , ψ ) = τ ( ω ) ◦ ξ ( ψ ) = ω ◦ λ ( 0 D , ψ ) .

Original: 2 = 
Revised: Finally, we note that the Lebesgue measure is translation-invariant, i.e. (cid:82) R D h ( ω + c ) d ω = (cid:82) R D h ( ω ) d ω for arbitrary functions h , meaning it is a Haar measure of the translation group. Since the translation group is a Lie group and thus locally compact, the Haar measure is unique up to a multiplicative constant [109, 110].

Original: arctan2 (Λ( Z ) n ∗ , 2 , Λ( Z ) n ∗ , 1 ) else where n ∗ =
Revised: Thus, Proposition 2 holds. F.3.2 Verifying Propositions 1 and 2 for rotation invariance in 2D In this section, let [Equation]

Original: min { n | || Λ( Z n ) || 2 > 0 } is thefirst non-zero row of matrix Λ( Z )  , d ∗ withΛ( Z , d ) =
Revised: [Equation] be the matrix that rotates counter-clockwise by angle ω . Note that SO (2) = { R ( ω ) | ω ∈ [0 , 2 π ] } .

Original:    
Revised: To verify our propositions for T = SO (2) , we define λ ( ω , ψ ) = τ ( ω ) ◦ ξ ( ψ ) with τ : [0 , 2 π ] → SO (2) and ξ : R + × R 2( N − 1) → R N × 2 (note that the first argument is non-negative) with [Equation]

Original: Z : ,
Revised: where vec − 1 : R 2( N − 1) → R ( N − 1) × 2 reshapes an input vector into a matrix. Due to the definition of group action ◦ from Appendix C.2, we have [Equation]

Original:  Z : ,if d =
Revised: In other words: We represent each element of R N × 2 as a matrix whose first row is aligned with the x -axis, rotated counter-clockwise by an angle ω . The function λ is evidently surjective. Any Z ∈ R N × 2 can be represented via [Equation]

Original: Z : , Z : ,if d =
Revised: with ω ∗ =

Original:  Z : , 
Revised: arctan2( Z 1 , 2 , Z 1 , 1 ) . It is also injective, save for the set { Z ∈ R N × 2 | Z 1 = 0 } , which has measure zero. The first row Z 1 uniquely defines ψ 1 = ||

Original: Z : ,if d ={+
 || 2 and ω =+}
Revised: Z {+
 || 2 and ω =+}

Original:  is a canonical map for elemental rotations around axis d ∗ .
Revised: arctan2( Z 1 , 2 , Z 1 , 1 ) , because polar coordinates for non-zero vectors are unique. The parameter vector z 2: must fulfill

Original: Proof.
Revised: [Equation]

Original: We can distinguish two cases. Case 1.
Revised: which has a unique solution because rotation matrix R ( ω ) is invertible.

Original: All entries of Λ( Z , d ∗ ) are 0 , i.e. Λ( Z , d ∗ ) = 0 N, 2 . In this case α ( Z ) = 0 and thusγ ( Z )
Revised: In addition to surjectivity and injectivity almost everywhere, λ is differentiable and we have τ (0)

Original: =
Revised: =

Original: Z .
Revised: I N , with identity matrix

Original: The equivalence class of Z is [ Z ] T = { Z } , because rotation around d ∗ will not makeΛ( Z , d ∗ ) non-zero.
Revised: I N being the identity element of SO (2) . Furthermore, we have for all R ( α ) , ω , ψ that [Equation]

Original: Both criteria Eqs. (3) and (4) for canonical maps arethus fulfilled.
Revised: Thus, all conditions from Proposition 1 are fulfilled.

Original: Case 2. The matrix Λ( Z , d ∗ ) has non-zero entries.
Revised: To verify Proposition 2, we need to show that

Original: Let ( r , θ , z ) be a map of Z in cylindricalcoordinates, i.e. Z = l ( r , θ , z , d ∗ ) . Let n ∗ be the index of the first non-zero row of Λ( Z , d ∗ ) . Recallthat rotation around axis
Revised: [Equation]

Original: d ∗ in Cartesian coordinates corresponds to translation of the cylindricalcoordinate angles and thus γ ( Z ) =
Revised: with marginal density ν X ( ψ ) =

Original: l ( r , θ − 1 N θ n ∗ , z ,  d ∗ ) . Criterion Eq.
Revised: (cid:82) [0 , 2 π ] µ X ( λ ( ω , ψ )) · | det( J λ ( ω , ψ )) | d ω and Haar integral β X

Original: (3) is fulfilled, because by construction of γ we have γ ( Z ) = ZR T for some R ∈ SO (3 , d ∗ ) .
Revised: ( Z ) = (cid:82) R ∈ SO (2) exp (cid:0) ⟨ t ◦ Z , X ⟩ F / σ 2 (cid:1) dη ( t ) .

Original: Now, consider an arbitrary Z ′ ∈
Revised: We begin by calculating the Jacobian

Original: [ Z ] T .
Revised: [Equation]

Original: By definition of [ Z ] T , there must be some a∈ R such that
Revised: a

Original: Z ′ =
Revised: B

Original: l ( r , θ + a 1 N , z , d ∗ ) . 
Revised: [Equation] , with some vector a ∈ R 2( N − 1) and block-diagonal matrix B ∈ R 2( N − 1) × 2( N − 1) with B =   R ( ω ) . . .

Original: Note that rotation does not affect the radii, i.e. n ∗ is also the index ofthe first non-zero row of Z ′ . 
Revised: R ( ω )   . Due to the block structure [Equation] we have

Revised: [Equation]

Original: Thus
Revised: Thus

Revised: [Equation]

Revised: our marginal density is

Original: [Equation]
Revised: [Equation]

Revised: where ∝ absorbs factors that are constant in ω .

Revised: Because the composition of two rotations corresponds to a translation of rotation angles, the translation-invariant Lebesgue measure is an invariant measure for group

Revised: SO (2) (in this angle-based parameterization): (cid:82)

Revised: [Equation]

Revised: [Equation]

Revised: ] f

Revised: [Equation]

Revised: R

Revised: [Equation]

Revised: [Equation]

Revised: R

Revised: [Equation]

Revised: [Equation]

Revised: dω

Revised: [Equation]

Revised: [Equation]

Revised: ] f ( R (

Revised: [Equation]

Revised: [Equation]

Revised: =

Revised: [Equation]

Revised: [Equation]

Revised: ] f ( R ( ω )) dω .

Revised: Since SO (2) is a Lie group and thus locally compact, the Haar measure is unique up to a multiplicative constant [109, 110].

Revised: Thus, Proposition 2 holds.

Original: {+F.3.
+}
Revised: {+F.3.
+}

Revised: Verifying Propositions 1 and 2 for rotation invariance in 3D

Revised: In this section, let

Revised: [Equation]

Revised: be the matrix that performs an intrinsic rotation around the z -axis by angle ω 1 , followed by a rotation around the new y -axis by angle ω 2 and then a rotation around the new x -axis by angle ω 3 .

Revised: Note that SO (3) =

Revised: { R ( ω ) | ω ∈ Ω } with Ω = [0 , 2 π ] × [ − π 2 , π 2 ] × [0 , 2 π ] .

Revised: To verify our propositions for T = SO (3) , we define λ ( ω , ψ ) =

Revised: τ ( ω ) ◦ ξ ( ψ ) with τ : Ω → SO (3) and ξ : R + ×

Revised: [Equation]

Revised: ×

Revised: [Equation]

Revised: + × R 3( N − 2) → R N × 3 (note that the first and third argument are non-negative) with

Revised: [Equation]

Revised: where vec − 1 : R 3( N − 1) → R ( N − 1) × 3 reshapes an input vector into a matrix.

Revised: Due to the definition of group action ◦ from Appendix C.2, we have

Revised: [Equation]

Original: meaning criterion Eq.
Revised: In other words: We represent each element of R N × 3 as a matrix whose first row is aligned with the x -axis, and whose second row is in the first or second quadrant of the x - y -plane (because ψ 3 is non-negative) which is then intrinsically rotated by z - y - x angles ω 1 , ω 2 , ω 3 . The function λ ( ω , ψ ) is injective and surjective, save for the set { Z ∈ R N × 3 | Z 1 = 0 ∨ ∃ c ∈ R : Z 2 = c · Z 1 } , which has measure zero. That is, for any Z outside this set, Z = λ ( ω , ψ ) has a unique solution.

Original: (4) is fulfilled as well. D.6.2 Proof for certificate In the following, we provide the proof of Theorem 8 for the case d ∗ = 3 .
Revised: Firstly, ( ψ 1 , ω 1 , ω 2 ) are spherical coordinates of Z 1 , which are unique for Z 1 ̸ = 0 .

Original: The other cases followanalogously.
Revised: Secondly, the unique angles ω 1 and ω 2 constrain the x -axis after the intrinsic rotation to be co-linear with Z 1 .

Original: Mapping into cylindrical coordinates is equivalent to transforming the two columns thatare not d ∗ into polar coordinates. Similarly, applying the canonical map for elemental rotations isequivalent to applying thecanonical map for rotation in 2D (see Appendix D.5) to the two columnsthat are not d ∗ into polar coordinates.
Revised: Thus, and because we assume that Z 1 and Z 2 are not co-linear, there must be unique angle ω 3 for rotation around the x -axis that ensures that Z 2 is in the first or second quadrant of the new x - y -plane. Finally, with the rotation angles ω and parameter ψ 1 fixed, the remaining parameter values are determined by [Equation]

Original: We use this fact to reuse much of our proof from Appendix D.5.
Revised: The function is also surjective on the entirety of R N × 3 . If Z 1 = 0 , we have ψ 1 = 0 and ω 1 , ωcan be chosen arbitrarily. If ∃ c : Z 2 = c · Z 1 , then ω 3 can be chosen arbitrarily. The remaining parameters can be chosen using the procedure described above.

Original: Applying Theorem 4 with our canonical map from Lemma 10 shows that
Revised: Finally, the function is differentiable, we have τ ( 0 ) = I N and τ is a surjective function into SO (3) , meaning

Original: [Equation]
Revised: [Equation]

Revised: ψ ) for some ω ′′ ∈ Ω .

Original: T
Revised: Thus, all criteria from Proposition 1 are fulfilled. Like in previous sections, we verify Proposition 2 by first calculating the Jacobian of λ . In the following, we use the shorthands s i = sin( ω i ) and c i = cos( ω i ) . We have

Original: [Equation]
Revised: [Equation]

Revised: with

Original: Like in our previous proof, we notice that the above optimization problem does not depend on thespecific class predicted by h , but only on whether it is y ∗ or y .
Revised: [Equation] Due to the block structure and because ω 2 ∈ [ − π [Equation] π 2 ] and ψ 3 ∈ R +

Original: We thus formulate the equivalentproblem
Revised: [Equation] we have [Equation] Thus

Original: [Equation]
Revised: [Equation]

Original: where h ( Z )= 1 indicates that Z isclassified as y ∗ .
Revised: our marginal density ν X ( ψ ) is

Original: Substitution.
Revised: [Equation]

Original: We begin by going from Cartesian to cylindrical coordinates: As defined in Eq.
Revised: This is, up to a multiplicative constant, the unique Haar integral for this angle-based parameterization of SO ( D ) (see, for instance [137, Chapter 1]). Thus, Proposition 2 holds.

Original: (61),let l : R N × R N × R N × { 1 , 2 , 3 } → R N × 3 be the function that maps from cylindrical coordinatesinto Cartesian coordiantes.
Revised: Finally, we prove that the propositions hold for T =

Original: [Equation]
Revised: SE

Original: The r n in the last equality are due to  the fact that l transforms N rows from cylindrical to Cartesiancoordinates, with each transformation having a Jacobian determinant r .
Revised: ( D ) with D ∈ { 2 , 3 } , which amounts to combining the results from the previous sections. In the following, let τ rot : Ω rot → SO ( D ) with τ rot ( ω ) =

Original: Similarity to 2D rotation invariance. Now, we use the fact that we are in case d ∗ to show that theabove integral is almost identical to theone from our proof for rotation invariance in 2D. Firstly, we know from the definition of l ( · )  in Eq.
Revised: R ( ω ) be the the parameterization of SO ( D ) defined in Appendix F.3.2 or Appendix F.3.3. Further let ξ rot : Ψ rot → R ( N − 1) × D be the same function as in Appendix F.3.2 or Appendix F.3.3, but for matrices with N − 1 instead of N rows.

Original: (61) that l ( r , θ , Z : ,d ∗ , d ∗ ) =

Original: l ( r , θ , Z : , 3 , 3) =[ λ ( r , θ ) Z : , 3 ] , where λ ( r , θ ) =

Original: [ r ⊙ cos( θ ) r ⊙ sin( θ )] is the function that maps from polarinto Cartesian coordinates.

Original: Secondly, let γ ′ be the canonical map for rotation invariance in 2D, as defined in Lemma 8.

Original: We know from the definition of our canonical map for elemental rotations in Lemma 10 thatγ ([ λ ( r , θ ) Z : , 3 ]) = [ γ ′ ( λ ( r , θ )) Z : , 3 ] . Thirdly, recall that µ X is an isotropic matrix normal distribution, i.e. all columns are independent.
Revised: We begin by defining τ : Φ rot × R D : → SE ( D ) and ξ : Ψ rot → R N × D as follows:

Original: Let µ X : , :2 be the distribution of the first two dimensions and µ X : , 3 be the distribution of the third 1526 dimension, i.e.

Original: [Equation]
Revised: [Equation]

Original: Combining all three steps shows that
Revised: Due to the definition of group action ◦ from Appendix C.4, we have

Original: [Equation]
Revised: [Equation]

Original: This expectation is identical to the one we derived in Eq. (37) of Appendix D.5, save for an independent variable Z : , 3 that contributes factor µ X : , 3 ( Z : , 3 ) to the likelihood. We may thus perform 1530 exactly the same steps as in the derivation of our certificate for rotation invariance in 2D, starting at Eq.
Revised: In other words: We represent each element of R N × D as a matrix whose first row is zero and whose second row is aligned with the x -axis which is then rotated and finally translated.

Original: (37), i.e. further substitutions, marginalization, application of the Neyman Pearson lemma andtransformation back into Cartesian coordinates, to show thatmin [Equation] with t ∈ R s.t. [Equation]{+For D = 
+}
Revised: {+For D = 
+}

Original: and ρ ( q ) = I(cid:18)(cid:113) q [Equation] (cid:19)
Revised: we z x

Original: [Equation] [Equation] Likelihood ratio distribution.
Revised: The proof for intrinsic z - y - x rotation is analogous.

Original: The last step of our derivation is to find the distribution of thelikelihood ratios in Eqs. (63) and (64). Since X ′ =
Revised: additionally constrain the third row to be in the first or second quadrant of the x - y plane (for more details, see Appendix F.3.3).

Original: X + ∆ , we have [Equation] , see also Appendix A of [27]. Theargument of the exponential function is an affine transformation of random variable Z : , 3 ,o = 12 σ
Revised: The function λ (( ω , b ) , ψ ) is surjective and injective, save for

Original: [Equation]
Revised: [Equation]

Original: (cid:0) [Equation] [Equation] [Equation]
Revised: which have measure zero

Original: | [Equation] ∆ : , [Equation]
Revised: That is, given an Z ∈ R N × D the solution to Z = λ (( ω , b ) , ψ ) is unique.

Original: || 22 − [Equation] ∆
Revised: Evidently, we must have b =

Original: [Equation] [Equation] : , 3 (cid:1) .In Eq.
Revised: Z 1 .

Original: (63), we have Z : , 3 ∼ µ X ′ : , 3 := N (cid:0) X : , 3 + ∆ : , 3 , σ 2 I N (cid:1) and thus o ∼ N( u (1) , s ) with meanu (1) = 12 σ 2 (cid:0) 2 || ∆ : , 3 || 22 −|| ∆ : , 3 || 22 (cid:1) = 12 σ 2 (cid:0) || ∆ : , 3 || 22 (cid:1) and variance s 2 = 1 σ 2 ∆ T: , 3 σ 2 I N ∆ : , 3 1 σ 2 =1 σ 2 || ∆ : , 3 || 22 .
Revised: With this parameter fixed, the remaining parameters can be found by solving the equation Z 2: − 1 N − 1 Z T 1 = τ rot ( ω ) ◦ ξ rot ( ψ ) , whose unique solution we discussed in Appendix F.3.2 and Appendix F.3.3.

Original: In Eq.
Revised: The function is also surjective on the entirety of R N × D for the reasons presented in Appendix F.3.2 and Appendix F.3.3.

Original: (64) we have Z : , 3 ∼ µ X : , 3 := N (cid:0) X : , 3 , σ 2 I N (cid:1) and thus o ∼ N ( u (2) , s ) with mean u (2) =12 σ 2 (cid:0) −|| ∆ : , 3 || [Equation]
Revised: Further note that the function λ (( ω , b ) , ψ ) is differentiable and that τ ( 0 , 0 ) = ( I D , 0 D ) , which is the identity element of SE ( D ) (see Appendix C.4).

Original: Therefore, we have
Revised: Finally, we see that

Original: [Equation]
Revised: [Equation]

Original: with t ∈ R [Equation] which concludes our proof
Revised: for some ω ′′ ∈ Ω , b ′′ ∈ R D .

Original: [Equation]
Revised: Thus, all criteria from Proposition 1 are fulfilled. Next, we verify Proposition 2 by again showing that the marginal density ν X ( ψ ) is a Haar integral. The function λ is a composition of two functions: The function λ rot from the previous sections and a translation. Thus, the Jacobian determinant is the product of the two corresponding Jacobians.

Original: D.7 Tight certificate forsimultaneous invariance under rotation and translation in 2D
Revised: As discussed in Appendix F.3.1, the Jacobian determinant for translation is 1 . Let J λ rot ( ω , ψ ) be the Jacobian of λ rot .

Original: Next, we consider the case that base classifier g is rotation and translation invariant, i.e. invariantunder group T = (cid:8) X (cid:55)→ XR T + 1 N c T | R ∈ SO (2) , c ∈ R D (cid:9) .
Revised: Then [Equation] where ∝ absorbs factors that are constant in ω and b . This is a Haar integral, because the inner and outer integral are Haar integrals for T ( D ) and SO ( D ) , respectively.

Original: In the following, let ˜ A ∈ R N × D be matrix A ∈ R N × D after centering,
Revised: Let us verify this by considering an arbitrary ( R ( ω ′ ) , b ′ ) ∈ SE ( D ) . To avoid clutter, define the shorthand f ( Z ) = exp (cid:0) ⟨ Z , X ⟩ F / σ 2 (cid:1) .

Original: i.e. ˜ A
Revised: We have

Original: =
Revised: [Equation]

Original: A − 1 N A , where
Revised: Here, we have first applied the definition of the group action and group operator from Appendix C. and then used the fact that we are integrating over Haar measures for T ( D ) and then SO ( D ) . Since SE ( D ) is a Lie group and thus locally compact, the Haar measure is unique up to a multiplicative constant [109, 110]. This confirms that Proposition 2 holds.

Original: A ∈ R 1 × D are the column-wise averages.
Revised: In this section, we prove the results for specific invariances from Sections 6.2 to 6.5, i.e. translation invariance (Appendix F.4.1), rotation invariance in 2D (Appendix F.4.2 and 3D (Appendix F.4.3, as well as roto-translation invariance in 2D and 3D (Appendix F.4.4). F.4.1 Translation invariance

Original: [-Further let ⟨ A , B ⟩ F = (cid:80) Nn =
 A Tn B Tn = vec( A ) T vec( B )be the Frobenius inner product.-] Theorem 9.
Revised: [-Further let ⟨ A , B ⟩ F = (cid:80) Nn =
 A Tn B Tn = vec( A ) T vec( B )be the Frobenius inner product.-] Theorem 4. Let g : R N × D

Original: Let g : R N × 2 → Y berotation and translation invariant  and H T be defined as in Eq.
Revised: → Y be invariant under T = T ( D ) and H T be defined as in Eq.

Original: Then
Revised: Then

Original: [Equation]
Revised: [Equation]

Revised: where ∆ ∈ R 1 × D are the column-wise averages of ∆ =

Revised: X ′ − X

Original: 69) witht ∈ R s.t.
Revised: and σ is the standard deviation of the isotropic matrix normal smoothing distribution µ X . During our proof of Theorem 3 in Appendices F.3 and F.3.1, we have shown that [Equation] with

Original: [Equation]
Revised: [Equation]

Revised: and κ ∈ R + such that

Original: [Equation]
Revised: [Equation]

Original:  and 
Revised: with ψ ∈ R ( N − 1) · D and marginal distribution

Original: [Equation]
Revised: [Equation]

Original: where I 0 is the modified Bessel function of  the first kind and order 0 , and
Revised: where we have inserted the definition of our isotropic matrix normal smoothing distribution for the second equality. Using the matrix A =

Original: [Equation]
Revised: [Equation]

Original: with ϵ 1 = (cid:68) ˜ X , ˜ ∆ (cid:69) F , ϵ 2 = (cid:68) ˜ XR (cid:0) − π 2 (cid:1) T , ˜ ∆ (cid:69) F . The above certificate is identical to the tight certificate for rotation invariance in 2D (see Appendix D.5), save for the fact that the clean input X and the perturbation ∆ have been centered. Since X ′ = X + ∆ , this is equivalent to centering X and X ′
Revised: R N

Original: and then evaluating the certificatefrom Appendix D.5. D.7.1 Canonical map For our derivation, recall the definition of canonical maps: Definition 3 (Canonical map) .
Revised: [Equation]

Original: A canonical map for invariance under a group of transformations Tis a function γ : R N × D

Original: → R N× D with 
Revised: N with inverse A − 1 =

Original: [Equation]
Revised: [Equation]

Original: In the following, let γ rot be the canonical map for invariance under T rot := 1564 (cid:8) X (cid:55)→ XR T | R ∈ SO (2) (cid:9) , as defined in Appendix D.5. Further let γ trans be the canonical map for invariance under T trans := (cid:8) X
Revised: [Equation]

Original: (cid:55)→ 1 N c T | c ∈ R 2 (cid:9) ,  asdefined in Appendix D.4. Composing these two canonical maps yields a canonical map for simultaneous rotation and translationinvariance. Lemma 11. The function γ ( Z ) =
Revised: I N −(cid:21) and all-zeros vector 0 N − 1 ∈ R N − 1 , all-ones vector 1 N − 1 ∈ R N − 1 and identity matrix I N − 1 , the marginal density can equivalently be written as

Original: γ rot ( γ trans ( Z )) is a canonical map for simultaneous rotationand translation invariance.

Original: Proof.
Revised: [Equation]

Original: Consider an arbitrary Z ∈ R N × 2 . As defined in Appendices D.4 and D.5, γ performs sometranslation, followed by some rotation. That is, γ ( Z )=
Revised: where the second equality follows from the change of variable formula for densities (see also Lemma 2) and the third equality is due to det( A )

Original: γ rot (cid:0) Z + 1 N c T (cid:1) for some c ∈ R 2 andγ rot ( Z + 1 N c T ) =(cid:0) Z + 1N c T (cid:1) R T = ZR T + 1 N ( Rc ) T for some R ∈ SO (2) .Evidentlyγ ( Z ) ∈ [ Z ] T , meaning criterion Eq.
Revised: = 1 .

Original: (3) is fulfilled. Now, consider an arbitrary pair Z ∈ R N × 2 , Z ′ ∈ [ Z ] T .
Revised: Evidently, we are marginalizing out the first dimension of each of the D densities.

Original: Since Z ∈ [ Z ] T , we must have Z ′ = ZR T + 1 N c T for some R ∈ SO (2) and c ∈ R 2 . Recall that γ trans ( Z ) maps input Z to arepresentative by subtracting the first row from all other rows, i.e.
Revised: For normal distributions, this is equivalent to dropping the first row of the mean as well as the first row and column of the covariance matrix, i.e.

Original: [Equation]
Revised: [Equation]

Original: Further note that γ trans is a canonical map for translation invariance and therefore fulfills criterion
Revised: with

Original: Eq. Therefore, we have γ trans ( Z ′ ) =
Revised: [Equation] [Equation]

Original: K (cid:0) ZR T + 1 N c T (cid:1) = K (cid:0) ZR T (cid:1) =
Revised:  and B = σ 2 A 2: ( A 2: ) T .

Original: ( KZ ) R T , wheresecond equality is due to criterion Eq.
Revised: Inserting back into Eqs.

Original: (4) and the last equality is due to associativity. Finally,note that since γ rot is a canonical map for translation invariance, we must have γ rot ( γ trans ( Z ′ )) = γ rot (( KZ ) R T) =
Revised: (29) and (30), we see that the optimal value of our variance-constrained optimization problem min h ∈ H T

Original: γ rot ( KZ ) =γ rot ( γ trans ( Z )) .
Revised: Pr Z ∼ µ X ′ [ h ∗ ( Z ) =

Original: This shows that γ also fulfills criterion Eq. (4),i.e. it is a canonical map for simultaneous translation and rotation invariance. D.7.2 Proof of certificate
Revised: y ∗ ] equals the optimal value of the black-box certification problem for anisotropic normal smoothing distribution N ( m X , Σ ) .

Original: To derive our certificate, we shall combine the substitutions we used for the translation invariancecertificate (see Appendix D.4) with those we used for rotation invariance (see  Appendix D.5).
Revised: This optimal value has been derived in prior work (see [138] and Appendix A of [94]) and is [Equation] We can conclude our proof by showing that

Original: Weshall see that this leads to an optimization problem that is almost identical to that for 2D rotationinvariance, save for the fact that we have anisotropic normal distributions. Like in Appendix D.4, themean and covariance of these anisotropic distributions naturally lead to a centering of the certificateparameters.
Revised: [Equation]

Original: Before proceeding, we need the following lemma:
Revised: To this end, we need the following result:

Original: Lemma 12. Consider  arbitrary vectors x, y ∈ R D and let x = 1 D (cid:80) Dd=1 x d , y = 1 D (cid:80) Dd
Revised: Lemma 7. Consider an arbitrary vector x ∈ R D and let x = 1 D (cid:80) Dd

Original: =1 y d betheir averages.
Revised: =1 x d be its average.

Original: Then
Revised: Then

Original: [Equation]
Revised: [Equation]

Original: Proof.
Revised: Proof.

Original: We substract the left-hand side from the right-hand side
Revised: We substract the left-hand side from the right-hand side

Original: [Equation]
Revised: [Equation]

Original: where the last equality follows from the fact that (cid:80) Dd =1 y d = D y .
Revised: where the last equality follows from the fact that (cid:80) Dd =1 x d = D x .

Original: Applying Theorem 4 with our canonical map γ ( Z ) = γ rot ( γ trans ( Z )) leads to optimization problem
Revised: Now, we can proceed by using the fact that the inverse of a block-diagonal matrix is also a blockdiagonal matrix and inserting the definitions of m X ′ , m X , Σ and B to show that:

Original: [Equation]
Revised: [Equation]

Original: (72) where h ( Z ) = 1 indicates that Z is classified as y ∗ .Substitution.
Revised: where for the second to last last equality we have used that X ′ = X + ∆ .

Original: First, we perform exactly the same substitution. as in our proof for translationinvariance to eliminate γ trans (see Eq. (13) to Eq.
Revised: Our next step is to compute ( A 2: ) T (cid:16) A 2: ( A 2: ) T (cid:17) − 1 A 2: .

Original: (14) in Appendix D.4). That is, V = AZ with A  =[-(cid:20) 
-]
Revised: Recall that A 2: ∈ R ( N − 1) × D and A 2: =[-(cid:20) 
-]

Original: TN − [Equation]
Revised: [ − 1 N − I N − 1 ] .

Original: −
Revised: Matrix multiplication shows that

Original: [Equation]
Revised: [Equation]

Original: N −
Revised: The matrix is sufficiently simple to be inverted by inspection:

Original: [Equation]
Revised: [Equation]

Original: I N −(cid:21) .
Revised: Finally, matrix multiplication shows that

Original: [Equation]
Revised: [Equation]

Revised: Note that the matrix in Eq.

Revised: (38) transforms a vector by subtracting its average from all entries.

Revised: Inserting into Eq.

Revised: (37) shows that

Revised: [Equation]

Revised: where ∆ : ,d ∈ R is the average of column ∆ : ,d , ∆ ∈ R 1 × D are the column-wise averages of matrix ∆ , the second equality is due to Lemma 7 and the third equality uses the definition of inner products.

Revised: Taking the square root and using the definition of the Frobenius norm yields our desired result:

Revised: [Equation]

Revised: F.4.2 Rotation invariance in 2D

Revised: Theorem 6.

Revised: Let g : R N × 2 → Y be invariant under T =

Revised: SO (2) and H T be defined as in Eq.

Revised: Define the indicator function ρ : R 4 → { 0 , 1 } with

Revised: [Equation]

Revised: with κ ∈ R such that

Revised: [Equation]

Revised: Then

Revised: [Equation]

Revised: where

Revised: [Equation]

Revised: with clean data norm ||

Revised: X || 2 , perturbation norm

Revised: || ∆ || 2 and parameters ϵ 1 = ⟨ X , ∆ ⟩ F , ϵ

Revised: [Equation]

Revised: ⟨ XR ( − π / 2 ) T , ∆ ⟩ F .

Revised: We know from Theorem 3 and our derivations in Appendix F.3.2 that

Revised: [Equation]

Revised: with

Revised: [Equation]

Revised: and κ ∈ R + such that

Revised: [Equation]

Revised: To prove Theorem 6, we shall first calculate the integral β X ( Z ) characterizing our worst-case classifier and then use the fact that it applies an affine transformation to our random input data sampled from µ X and µ X ′ .

Revised: We begin by factoring out cos( ω ) and sin( ω ) from the exponent:

Original: [-(cid:18) γ rot (cid:18)(cid:20) 
-]
Revised: [-(cid:18) γ rot (cid:18)(cid:20) 
-][Equation] We can then show that [Equation] where the second equality follows from the fact that cos( ω ) η 1 +sin( ω ) η 2 = cos( ω + α ) (cid:112) η 21 + η 22 for some α ∈ [0 , 2 π ] (see also Eq.

Original: {+29 of [139]) and the third equality is due to the integral representation of the modified Bessel function of the first kind with order 
+}
Revised: {+29 of [139]) and the third equality is due to the integral representation of the modified Bessel function of the first kind with order 
+}

Revised: (see Eq.

Revised: 10.32.1 of [140]):

Revised: [Equation]

Revised: We can now insert this expression for the worst-case classifier into the expectations w

Original: T
Revised: [Equation] r [Equation] t. clean and perturbed smoothing distributions µ X , µ ′ X from Eqs.

Revised: (39) and (40) .

Revised: Note that, by definition, we have vec( Z ) ∼ N (vec( X ) , σ 2 · I 2 N ) for Z ∼ µ X .

Revised: Thus:

Revised: [Equation]

Original: V 2: (cid:21)(cid:19)(cid:19) 2 (cid:89) d =1 N (cid:0) V : ,d | A
Revised: with means m (1) = W vec( X ′ ) , m (2) = W vec( X ) and covariance matrix Σ = σ 2 W W T , where

Original: [Equation]
Revised: [Equation]

Revised: Calculating the matrix-vector and matrix-matrix products yields the values from Theorem 6, thus proving that the certificate is valid.

Revised: F.4.3 Rotation invariance in 3D

Revised: As discussed in Section 6.4, we can apply Theorem 3 to T =

Revised: SO (3) , but do not have a closed-form expression for the Haar integral β X ( Z ) characterizing our worst-case classifier.

Revised: We can however evaluate it using numerical integration.

Revised: In the following, we first show how we can reduce the number of integration variables to facilitate numerical integration.

Revised: We then show that, similar to the 2D case, the Monte Carlo certification procedure presented in Section 6.3 only requires sampling from a 16 -dimensional normal distribution, which allows us to obtain tight bounds via a large number of samples at little computational cost.

Revised: Reducing the number of integration variables.

Revised: Recall from our proof of Theorem 3 in Appendices F.3 and F.3.3 that

Revised: [Equation]

Revised: [Equation]

Revised: with Ω = [0 , 2 π ] × [ − π 2 , π 2 ] × [0 , 2 π ] and rotation matrix

Revised: [Equation]

Revised: [Equation]

Revised: ) =

Revised: [Equation]

Revised: with ˜ R ( ω 2: ) being the matrix that rotates around the y - and z -axis by angles ω 2 and ω 3 , respectively.

Revised: While we cannot evaluate this term analytically, we can calculate the inner integral over ω 1 in order to reduce it to a double integral.

Revised: We begin by factoring out cos( ω 1 ) and sin( ω 1 ) from the exponent:

Revised: [Equation]

Revised: where χ 1 , χ 2 , χ 3 are shorthands we introduce to avoid clutter in the following derivations

Revised: [Equation]

Revised: Due to the entries of ˜ R ( ω 2 :)

Revised: [Equation]

Revised: the shorthand χ 1 : R 2 × R 3 × 3 → R is defined as

Revised: [Equation]

Revised: Similarly

Revised: [Equation]

Revised: the shorthand χ 2 : R 2 × R 3 × 3 → R is defined as

Revised: [Equation]

Revised: The shorthand χ 3 : R 2 × R 3 × 3 → R is defined as

Revised: [Equation]

Revised: Just like in Appendix F.4.

Revised: [Equation]

Revised: we can use the integral formula for the modified Bessel function of the first kind and order zero to eliminate the integral w.r.t.

Original:  :  X : ,d , σ
Revised: ω 1 : β X ( Z ) = [Equation] [Equation] We introduce ˆ β

Revised: ( X T Z ) to avoid clutter in the next equations and to highlight that this function characterizing our worst-case classifier only depends on X T Z ∈ R 3 × 3 .

Revised: Efficient Monte Carlo certification.

Revised: Just like in Appendix F.4.2, we can now use the fact that our worst-case classifier only depends on a small number of variables that are the result of linearly transforming our randomized inputs Z ∼ µ X , namely the 9 entires of X T Z .

Revised: Let ζ : R 8 → R 3 ×with

Revised: [Equation]

Revised: be a function that zero-pads its 8 -dimensional input vector before devectorizing into shape 3 × 3 .

Revised: Since vec( Z ) ∼ N (cid:0) vec( X ) , σ 2 I 3 N (cid:1) if Z ∼ µ X , we have by definition of ˆ β that

Original: [Equation]
Revised: [Equation]

Original: A
Revised: with m X = W vec( X ) , Σ = σ 2 W W T and W =   X ′ :

Original: [Equation]
Revised: [Equation]

Original:  :( A
Revised: X [Equation] [Equation] X :

Original: [Equation]
Revised: [Equation]

Revised: X

Original: [Equation]
Revised: [Equation]

Original: (cid:1)
Revised: X

Original: d V 2: := (cid:90)
Revised: [Equation] X X [Equation]

Original: [-R ( N − 1) × 2 h (cid:18) γ rot (cid:18)(cid:20) 
-]
Revised: [-R ( N − 1) × 2 h (cid:18) γ rot (cid:18)(cid:20) 
-]X   T . Thus, the expectation that provides the optimal value of our variance-constrained optimization problem can be probabilistically bounded via Monte Carlo sampling from an 8 -dimensional normal distribution, regardless of the data dimensionality.

Original: T
Revised: For the roto-granslation group SE ( D ) , we prove that evaluating the tight certificate is equivalent to evaluating the tight certificate for rotation group SO ( D ) after centering the clean and perturbed data X , X ′ by substracting their averages.

Original: W (cid:21)(cid:19)(cid:19) η X( W ) d W , where we have defined η  X ( W ) =
Revised: Theorem 7. Let ˜ X = X − 1 N X with column-wise averages X , X ′ ∈ R 1 × D . Further let η be a right Haar measure on SO (3) and µ X our isotropic matrix normal smoothing distribution.

Original: (cid:81) 2 d =1 N (cid:0) W : ,d |
Revised: Let β SE ( D ) X = [Equation]

Original: A 2: X : ,d , σ 2 A 2: ( A 2: ) T (cid:1) to be the marginaldistribution over R N × 2 .
Revised: be the Haar integral characterizing the worst-case classifier for roto-translation invariance (see also Appendix F.3.4) and [Equation] be the Haar integral characterizing the worst-case classifier for rotation invariance (see also Appendices F.3.2 and F.3.3). Then, for all V ∈ R N × D and κ ∈ R + ,

Original: Next, we eliminate γ rot by transforming into polar coordinates.
Revised: [Equation] Note that these (with V = X and V = X ′ ) are exactly the expectations determining the optimal value of our variance-constrained optimization problem (see Theorem 3). To prove this theorem, we first simplify the integrand of β SE ( D ) X

Original: For this, let λ ( r , θ ) =[ r ⊙ cos( θ ) r ⊙ sin( θ )] be the function that maps from polar into Cartesian coordinates.
Revised: ( Z ) by moving the rotation matrix into the second argument of the Frobenius inner product (note that, since R is a rotation matrix, we have R − 1 = R T ):

Original: [Equation]
Revised: [Equation]

Original: Recall from Appendix D.5 that γ rot rotates its argument such that its first non-zero row is aligned withthe first coordinate axis. Evidently, the first row of its argument is zero. Due to the factor (cid:81) N − 1 n =1 r n ,the entire integrand is zero whenver any of the other rows are zero (recall our discussion of Eq. (38)in Appendix D.5).
Revised: Next, we bring our integral into the same form as in our derivation of the certificate for translation invariance (see Appendix F.4.1), so that we can easily integrate out the translation vector b .

Original: We may thus assume that γ rot always rotates by θ 1 :[Equation] θ [Equation]
Revised: First, we eliminate the rotation matrix R T via the substitution c = R T b :

Original: [Equation]
Revised: [Equation]

Original: [-Since W (cid:55)→ (cid:20) 
 T W (cid:21) is an injective function from R ( N − 1) × 2 into R N × 2 , we may formulate anequivalent optimization problem over a family of variables ( h ′ ) indexed by R ( N − 1) × 2 , which leadsto expectation [Equation]-]
Revised: [-Since W (cid:55)→ (cid:20) 
 T W (cid:21) is an injective function from R ( N − 1) × 2 into R N × 2 , we may formulate anequivalent optimization problem over a family of variables ( h ′ ) indexed by R ( N − 1) × 2 , which leadsto expectation [Equation]-]For the next step, let A =

Original: Similarity to 2D rotation invariance.

Original: We now observe that this expectation is almost identicalto Eq.

Original: (39) from Appendix D.5, the expectation we obtained after eliminating the canonical map for2D rotation invariance.

Original: The only difference is that we have a distribution η X that is a product of twoanisotropic multivariate normal distributions, instead of µ X , which was a product of two isotropicmultivariate normal distributions.

Original: However, our proof did not make any assumptions about theparameters of the distribution.

Original: We may thus perform the same steps, starting at Eq.

Original: (39), i.e. furthersubstitution, marginalization, application of the Neyman Pearson lemma and transformation backinto Cartesian coordinates, to show that

Original: [Equation]
Revised: [Equation]

Original: with t ∈ R s.t. [Equation] (75) with κ defined as in Lemma 9. Likelihood ratio distribution.
Revised: with inverse A

Original: Finally, recall from Lemma 9 that

Original: [Equation]
Revised: [Equation]

Original: We shall now use our knowledge about the distribution of W , as well as some simple algebra to showthat Eqs.

Original: (74) and (75) are equivalent to evaluating the tight certificate for 2D rotation invariance,after centering X ′ and X . Let B = A 2: ( A 2: ) T . First, we can absorb the parameter B into thesecond argument of κ :
Revised: = (cid:20)

Original: [Equation]
Revised: [Equation]

Original: Furthermore, we know from Eqs. (74) and (75) that W ∼ η X or W ∼ η ′ X with η X ( W ) = (cid:81)d =1 N (cid:0) W | A 2: X , σ 2 A 2: ( A 2: ) T (cid:1) .
Revised: and all-zeros vector 0 N − 1 ∈ R N − 1 , all-ones vector 1 N − 1 ∈ R N − 1 and identity matrix

Original: Let us begin with the case that W ∼ η X .

Original: The random variable W is an affine transformation of random variable E ∈ R N × 2 that is distributed according toa zero-mean isotropic matrix normal distribution

Original: [Equation]

Original: We may thus restate the probability from Eq.

Original: (75) as

Original: [Equation]

Original: Next

Original: [Equation] we can absorb S into the second argument of κ : [Equation] We know from Eq.
Revised: I N − 1 .

Original: (26) of our derivation of the translation invariance certificate that

Original: [Equation] i.e. ( A 2: ) T B − 1 A2: is  the matrix that subtracts the mean from every vector it is multiplied with. Thus κ (cid:0) E + X , [Equation]
Revised: Multiplying a vector or matrix with A is equivalent to subtracting the first row from all other rows.

Original:  A [Equation] B [Equation]
Revised: We introduce the redundant factor A − 1 A and make the substitution u

Original: where ˜ X = X − [Equation] X .
Revised: = c + Z 1 to show that

Original: Next, we can apply Lemma 12, which shows that computing the inner product between a centeredand an uncentered vector is identical to centering both vectors and then computing the inner product:

Original: [Equation]
Revised: [Equation]

Original: Finally

Original: [Equation]

Original: inserting this result into Eq.

Original: (76) and using linearity of expectation shows that

Original: [Equation]

Original: The case W ∼ η X ′ follows analogously.

Original: We therefore have

Original: [Equation]

Original: with t ∈ R s.t.

Original: [Equation]

Original: This solution is identical to the one we derived for rotation invariance (see Eqs.

Original: (52) and (53)in Appendix D.5), except that all occurrences of X have been replaced with the centered ˜ X andall occurrences of X ′ have been replaced with the centered ˜ X ′ .

Original: Thus, performing the subsequentderivation steps from Appendix D.5 yields the desired certificate.

Original: 3D

Original: Finally, we consider the case that base classifier g is invariant under group T

Original: [Equation]

Original: [Equation]

Original: SO
Revised: SE

Original: [Equation]

Original: d } , 1646 where e d is an indicator vector and d ∗ is the rotation axis.

Original: In the following, let ˜ A ∈ R N × D be matrix A ∈ R N × D after centering,

Original: i.e. ˜ A

Original: =

Original: A − 1 N A , where

Original: A ∈ R 1 × D are the column-wise averages.

Original: Further let ⟨ A , B

Original: [Equation]

Original: F =

Original: [Equation]

Original: = vec

Original: [Equation]

Original: ( B )be the Frobenius inner product.

Original: Theorem 10.

Original: Let g : R N × 3 → Y be translation invariant and invariant to elemental rotationsaround axis d ∗ and let H T be defined as in Eq.

Original: Then

Original: [Equation]
Revised: [Equation]

Original: where I 0 is the modified Bessel function of the first kind and order 0 , and

Original: [Equation]

Original: ∗

Original: [Equation]

Original: The certificate is identical to centering X and X ′ and evaluating the tight certificate for invarianceunder elemental rotation derived in Appendix D.6.

Original: D.8.1 Canonical map

Original: In the following, let γ erot be the canonical map for invariance under T erot := 1657 (cid:8) X

Original: (cid:55)→ XR T | R ∈ SO (3 , d ∗ ) (cid:9) , as defined in Appendix D.6.

Original: Further let γ trans be the canonical map for invariance under T trans := (cid:8)

Original: X
Revised: D

Original: (cid:55)→ 1 N c T | c ∈ R 2 (cid:9) , as defined in Appendix D.4.

Original: Composing these two canonical maps yields a canonical map for simultaneous rotation and translationinvariance.

Original: Lemma 13.

Original: The function γ ( Z ) =

Original: γ erot ( γ trans ( Z )) is a canonical map for simultaneous rotationand translation invariance.

Original: Proof.

Original: Identical to the proof of Lemma 10, replacing every occurence of “ SO (2) ” with SO (3 , d ∗ ) .

Original: D.8.2 Proof for certificate

Original: Like in Appendix D.6, we only consider the case d ∗ = 3 , the other cases follow analogously.

Original: As inour proof for simultaneous rotation and translation invariance in Appendix D.7, we first eliminateγ trans via the substitution we used for our translation invariance certificate.

Original: We then make the sameobservation as in our proof for elemental rotations: The optimization problem is identical to that for2D rotation invariance, save for an independent factor in the density function.

Original: We may thus reusemuch of the proof and only have to find the distribution of an independent likelihood ratio.

Original: Applying Theorem 4 with our canonical map γ ( Z ) =

Original: γ rot ( γ trans ( Z )) leads to optimization problem

Original: [Equation]
Revised: [Equation]

Original: where h ( Z ) = 1 indicates that Z is classified as y ∗ . Substitution. Performing the same steps as in Appendix D.7 to eliminate γ trans shows that
Revised: X

Original: [Equation]
Revised: [Equation]

Original: where we have defined η X ( W ) = (cid:81) [Equation] =
Revised: Next

Original: [Equation]

Original: [Equation]

Original: d |

Original: [Equation]

Original: [Equation]

Original: X :

Original: [Equation]
Revised: [Equation]

Original: d [Equation]
Revised: we use the behavior of multivariate normal densities under affine transformation (see also Lemma 2) [Equation]

Original: σ 2 A 2: ( A 2: ) T (cid:1) to be the marginaldistribution over R N × 3 , with A = (cid:20)
Revised: as well as the fact that marginalizing out one dimension of a multivariate normal density is equivalent to dropping the correspond row of the mean and corresponding row and column of the adjacency matrix

Original: [Equation]
Revised: [Equation]

Original: Note that η X is the product of three independent multivariate normal distributions, i.e. all columnsare independent.
Revised: to show that [Equation]

Original: In the following, let η X : , :2 be the distribution of the first two dimensions and η X : , 3 1679 be the distribution of the third dimension, i.e.
Revised: Finally, reusing the fact that ( A 2: ) T (cid:0) A 2: ( A 2: ) T (cid:1) − 1 A 2: X = ˜ X with ˜ X = X − 1 N X from Appendix F.4.1 proves that

Original: [Equation]
Revised: [Equation]

Original: Similarity to invariance under elemental rotation. Next, we transform into cylindrical coordinates.
Revised: The last term is the Haar integral β

Original: Because we are in the case d ∗ , this is equivalent to transforming Z : , :2 into polar coordinates (recallour discussion in Appendix D.6).
Revised: SO ( D ) ˜ X ( Z ) characterizing the worst-case classifier for SO ( D ) , after centering X . This means that we have

Original: For this, let λ ( r , θ ) = [ r ⊙ cos( θ ) r ⊙ sin( θ )] be the functionthat maps from polar into Cartesian coordinates.
Revised: [Equation] Note that, when using ∝ in the previous steps, we have only removed constant factors that appear in the enumerator and denominator and thus cancel out. The last thing we need to do is verify that not only the classifiers, but also their expectations under V and ˜ V are identical (see Theorem 7).

Original: E Z ∼ µ X [ h ( γ [Equation] Z ))] [Equation]
Revised: We use the fact that the worst-case classifier only depends on a low-dimensional linear projection of our random input variable Z (similar to our derivations for

Original: This expectation is identical to Eq. (73) from Appendix D.8, save for the independent factorη X : , 3 ( W : , 3 ) in the densitiy function. We may thus perform the same derivations, i.e. further substi- 1686 tutions, marginalization, application of the Neyman Pearson lemma and transformation back into Cartesian coordinates, to show thatmin h ∈ H T
Revised: Monte Carlo evaluation in Appendices F.4.2 and F.4.3):

Original: [Equation]
Revised: [Equation]

Original: with t ∈ R s.t. [Equation] and ρ ( q [Equation]
Revised: By definition of our smoothing distribution, we have vec( Z ) ∼ N (vec( V ) | vec( V ) , σ 2 I D · N ) for Z ∼ µ V .

Original: = I [Equation]
Revised: Because our worst-case classifier for SE ( D ) performs a linear transformation of vec( Z ) , we have

Original: [Equation]
Revised: [Equation]

Original: Likelihood ratio distribution.

Original: The last step of our derivation is to find the distribution of thelikelihood ratios in Eqs.

Original: (86) and (87). Recall that η : , 3 ( W : , :3 )  =N (cid:0) W : , 3 | A 2: X : , 3 , σ 2 A 2: ( A 2: ) T(cid:1) .
Revised: with m V = W vec( V ) and Σ = W W T

Original: Evidently, it is an affine transformation

Original: A 2: Z : , 3 of the random variable Z : , 3 ∈ R N which followsisotropic normal distribution µ X : , 3 ( Z : , 3 ) = N (cid:0) Z : , 3 | X : , 3 , σ 2 I n (cid:1) .

Original: We may thus restate Eqs.

Original: (86) 1692 and (87) asmin

Original: [Equation]
Revised: [Equation]

Original: with t ∈ R s.t.
Revised: where

Original: [Equation]
Revised: [Equation]

Original: Next, we can compute the likelihood ratio.

Original: Define B = ( A 2: A 2: ) T and recall that X ′ = X + ∆ . Then:
Revised: Similarly

Original: [Equation]
Revised: [Equation]

Original: Now, recall from Eq. (26) of our derivation of the translation invariance certificate that
Revised: we have for SO (3) (after centering V )

Original: [Equation]
Revised: [Equation]

Original: i.e. ( A 2: ) T B − 1 A 2: is the matrix that subtracts the mean from every vector it is multiplied with.

Original: Further recall Lemma 12, i.e. the inner product between a centered and an uncentered vector isidentical to centering both vectors and then computing the inner product.

Original: Thus
Revised: that

Original: [Equation]
Revised: [Equation]

Original: The argument of the exponential function is an affine transformation of random variable Z : , 3 ,o = 12 σ 2 (cid:16) 2 ˜ ∆ T : , 3 Z : , 3 − || ˜ ∆ : , 3 || 22 − 2 ˜ ∆ : , 3 ˜ X : , 3 (cid:17) . In Eq. (89), we have Z : , 3 ∼ µ X ′ : , 3 := N (cid:0) X : , 3 + ∆ : , 3 , σ 2 I N (cid:1) and thus o ∼ N ( u (1) , s ) with meanu (1) = 12 σ 2 (cid:16) 2 || ˜ ∆ : , 3 || 22 − || ˜ ∆ : , 3 || 22 (cid:17) = 12 σ 2 (cid:16) || ˜ ∆ : , 3 || 22 (cid:17) and variance s 2 = 1 σ 2 ˜ ∆ T : , 3 σ 2 I N ˜ ∆ : , 3 1 σ 2 =1 σ 2 || ˜ ∆ : , 3 || 22 .
Revised: with m ˜ V

Original: Note that we have again used Lemma 12 in computing the inner product between ˜ ∆ : , 3 1704 and the mean of our random distribution. In Eq. (90) we have Z : , 3 ∼ µ X : , 3 := N (cid:0) X : , 3 , σ 2 I N (cid:1) and thus o ∼ N ( u (2) , s ) with mean u (2) =12 σ 2 (cid:16) −|| ˜ ∆ : , 3 ||(cid:17) = − 12 σ(cid:16) || ˜ ∆ : , 3 ||(cid:17) and variance s 2 = 1 σ 2 || ˜ ∆ : , 3 || 22 .
Revised: = W vec( ˜ V ) and Σ = W W T .

Original: Note that we have againused Lemma 12 in computing the  inner product between  ˜ ∆ : , 3 and  the mean of our random distribution. Therefore, we have [Equation] with t ∈ R s.t.
Revised: Finally, due to the fact (see Lemma 7) that calculating an inner product between a centered and an uncentered vector (here: columns of ˜ X and V ) is equivalent to centering both vectors before calculating the inner product (here: columns of ˜ X and ˜ V ), we have m V = m ˜ V .

Original: [Equation]

Original: and ρ (

Original: [Equation]

Original: =

Original: I [Equation] [Equation] which concludesthe proof for our final tight certificate.
Revised: Thus, both expectations are equal, which concludes our proof.

Original: All of the discussed tight certificates, save for the one for translation invariance, are of the form
Revised: All of the discussed tight certificates, save for the one for translation invariance, are of the form

Original: [Equation]
Revised: [Equation]

Original: with t ∈ R s.t.
Revised: with κ ∈ R such that

Original: [Equation]
Revised: [Equation]

Original: and [Equation] where g : R N × D → Y is our base classifier, µ X , µ X ′ are the clean and perturbed smoothingdistribution, υ (1) and υ (2) are distributions over some set S , and ρ : S → R is some arbitraryscalar-valued function.
Revised: where g : R N × D → Y is our base classifier, µ X , µ X ′ are the clean and perturbed smoothing distribution, υ (1) and υ (2) are distributions over some set S , and ρ : S → R is some arbitrary scalar-valued function.

Original: As discussed in Section 6.3, we propose to compute a narrow probabilistic lower bound onmin h ∈ H T Pr Z ∼ µ X ′
Revised: As discussed in Section 6.3, we propose to compute a narrow probabilistic lower bound on min h ∈ H T Pr Z ∼ µ X ′

Original: [ h ( Z ) =
Revised: [ h ( Z ) =

Original: y ∗ ]] by combining three confidence bounds.
Revised: y ∗ ]] by combining three confidence bounds.

Original: Before computing these bounds, we have to inspect the monotonicity of our certificate.
Revised: Before computing these bounds, we have to inspect the monotonicity of our certificate.

Original: Evidently, Pr V ∼ υ (1) [ ρ ( V ) ≤ t ] is monotonically decreasing in t .
Revised: Evidently, Pr V ∼ υ (1) [ ρ ( V ) ≤ κ ] is monotonically decreasing in κ .

Original: Furthermore, t is monotonicallydecreasing with p X ,y ∗ : If p X ,y ∗ decreases, then a smaller t is sufficient for fulfilling the constraint
Revised: Furthermore, κ is monotonically decreasing in p X ,y ∗ : If p X ,y ∗ decreases, then a smaller κ is sufficient for fulfilling the constraint

Original: Pr V ∼ υ (2) [ ρ ( V ) ≤ t ]  .
Revised: Pr V ∼ υ (2) [ ρ ( V ) ≤ κ ] = p X ,y ∗ .

Original: Thus, we can compute a lower bound on our certificate by
Revised: Thus, we can compute a lower bound on our certificate by

Original: lower-bounding p X ,y ∗ , i e. p X ,y ∗ ≤ p X ,y ∗ , 
Revised: lower-bounding p X ,y ∗ , i.e. p X ,y ∗ ≤ p X ,y ∗ , 2.

Original: lower-bounding the t ∈ R that fulfills Pr V ∼ υ (2) [ ρ ( V ) ≤ t ] =
Revised: lower-bounding the κ ∈ R that fulfills Pr V ∼ υ (2) [ ρ ( V ) ≤ κ ] =

Original: p X ,y ∗ , i.e. t ≤ t
Revised: p X ,y ∗ , i.e. κ ≤ κ 3.

Original: and then lower-bounding Pr V ∼ υ (1) [ ρ ( V ) ≤ t ] .
Revised: and then lower-bounding Pr V ∼ υ (1) [ ρ ( V ) ≤ κ ] .

Original: The random variable 1 [ g ( Z ) =
Revised: The random variable 1 [ g ( Z ) =

Original: y ∗ ] is a Bernoulli random variable.
Revised: y ∗ ] is a Bernoulli random variable.

Original: Just like other randomizedsmoothing methods (e.g. [27]), we can lower-bound p X ,y ∗ by evaluating g on N 1 samples from µ Xand computing a binomial proportion confidence bound, such as the the Clopper-Pearson confidencebound [132].
Revised: Just like other randomized smoothing methods (e.g. [26]), we can lower-bound p X ,y ∗ by evaluating g on N 1 samples from µ X and computing a binomial proportion confidence bound, such as the the Clopper-Pearson confidence bound [141].

Original: Lower-bounding t requires computing a lower bound on F − 1 ( p X ,y ∗ ) , where F − 1 is the quantilefunction of ρ ( V ) with V ∼ υ (2) .
Revised: Lower-bounding κ requires computing a lower bound on F − 1 ( p X ,y ∗ ) , where F − 1 is the quantile function of ρ ( V ) with V ∼ υ (2) .

Original: A non-parametric lower confidence bound on this quantile can beconstructed by evaluating ρ ( V ) on N 2 samples from υ (2) and returning the largest order statistic
Revised: A non-parametric lower confidence bound on this quantile can be constructed by evaluating ρ ( V ) on N 2 samples from υ (2) and returning the largest order statistic

Original: R ( n ) such that (cid:2) ρ ( V ) ≤ R ( n ) (cid:3) ≤ p X ,y ∗ holds with high probability (see Section 5.2.1.
Revised: R ( n ) such that (cid:2) ρ ( V ) ≤ R ( n ) (cid:3) ≤ p X ,y ∗ holds with high probability (see Section 5.2.1.

Original: of [133]and Algorithm 1 below).
Revised: of [142] and Algorithm 1 below).

Original: The random variable [ ρ ( V ) ≤ t ] is another Bernoulli random variable and can thus be lower-boundedby evaluating ρ ( V ) on N 3 samples from µ X ′ and computing a binomial proportion confidence bound.
Revised: The random variable [ ρ ( V ) ≤ κ ] is another Bernoulli random variable and can thus be lower-bounded by evaluating ρ ( V ) on N 3 samples from µ X ′ and computing a binomial proportion confidence bound.

Original: We want all three confidence bounds to simultaneously hold with high probability 1 − α .
Revised: We want all three confidence bounds to simultaneously hold with high probability 1 − α .

Original: We ensurethis by using Holm-Bonferroni [125] correction to account for the multiple comparisons problem.
Revised: We ensure this by using Holm-Bonferroni [131] correction to account for the multiple comparisons problem.

Original: In our case, this corresponds to computing the first bound with significance α , the second one withsignificance α / 2 and the last one with significance α / 3 .
Revised: In our case, this corresponds to computing the first bound with significance α , the second one with significance α / 2 and the last one with significance α / 3 .

Original: Algorithm 1 summarizes our certification procedure.
Revised: Algorithm 1 summarizes our certification procedure.

Original: L OWER C OUNF B OUND refers to the ClopperPearson lower confidence bound.
Revised: L OWER C OUNF B OUND refers to the ClopperPearson lower confidence bound.

Original: B IN PV ALUE ( n, N 2 , ≥ , p X ,y ∗ ) refers to the p -value of the Binomialtest with  null-hypothesis that the success probability is greater or equal p X ,y ∗ .
Revised: B IN PV ALUE ( n, N 2 , ≥ , p X ,y ∗ ) refers to the p -value of a Binomial test with the null-hypothesis that the success probability is greater or equal p X ,y ∗ .

Original: Note that only thefirst confidence bound requires evaluating the base classifier g .
Revised: Note that only the first confidence bound requires evaluating the base classifier g .

Original: For the other two confidence boundsa large number of samples can be evaluated at little computational cost.
Revised: For the other two confidence bounds a large number of samples can be evaluated at little computational cost.

Original: [Table]
Revised: [Table]

Original: E Proof of Theorem6
Revised: G Proof of Theorem

Original: Next, we prove that the post-processing-based certificate for rotation-invariance, i.e. invariance under 1746 T =(cid:8) X (cid:55)→ XR T | R ∈ SO ( D ) (cid:9) is not tight, which we formalized in Section 6.3 as follows:
Revised: Next, we prove that the post-processing-based certificate for rotation-invariance ( T = SO ( D ) ) is not tight, which we formalized in Section 6.3 as follows:

Original: Theorem 6.
Revised: Theorem 5.

Original: Let g : R N × D
Revised: Let g : R N × D

Original: → Y berotation invariant  and H T be defined as in Eq.
Revised: → Y be invariant under T = SO ( D ) and H T be defined as in Eq.

Original: Furtherassume that  ∄ R ∈
Revised: Assume that perturbed input X ′ is not obtained via rotation of X , i.e. ∄ R ∈

Original: SO ( D ) :
Revised: SO ( D ) :

Original: X ′R T = X
Revised: X ′ = XR T .

Original: and that p X ,y ∗ ∈ (0 , 1) .
Revised: Further assume that p X ,y ∗ ∈ (0 , 1) .

Original: Then, for all R ∈ SO ( D ) :
Revised: Then, for all R ∈ SO ( D ) :

Original: [Equation]
Revised: [Equation]

Original: The right-hand side term is the optimal value of the black-box optimization problem evaluated forperturbed input X ′ R T , i.e.
Revised: The right-hand side term is the optimal value of the black-box optimization problem evaluated for perturbed input X ′ R T , i.e.

Original: [Equation]
Revised: [Equation]

Original: , with H = (cid:8) h : R N × D → Y | Pr Z ∼ µ X [ h ( Z ) =
Revised: with H = (cid:8) h : R N × D → Y | Pr Z ∼ µ X [ h ( Z ) =

Original: y ∗ ] ≥ p X ,y ∗ [Equation]
Revised: y ∗ ] ≥ p X ,y ∗ (cid:9) .

Original: The left-hand side term is the optimal value of the gray-box optimization problem evaluated for perturbed input X ′ .
Revised: The left-hand side term is the optimal value of the gray-box optimization problem evaluated for perturbed input X ′ .

Original: Note that H T is aset of rotation invariant classifiers.
Revised: Note that H T is a set of rotation invariant classifiers.

Original: In Appendix C.3 we have proven that rotation of the smoothingdistribution’s mean does not have an effect on the prediction probabilities of such classifiers, i.e.
Revised: In Appendix D we have proven that rotation of the smoothing distribution’s mean does not have an effect on the prediction probabilities of such classifiers, i.e.

Original: [Equation]
Revised: [Equation]

Original: Thus, we can prove Theorem 6 by proving the following, more general statement:
Revised: Thus, we can prove Theorem 5 by proving the following, more general statement:

Original: Lemma 14.
Revised: Lemma 8.

Original: Let g : R N × D
Revised: Let g : R N × D

Original: → Y berotation invariant  and H T be defined as in Eq.
Revised: → Y be invariant under T = SO ( D ) and H T be defined as in Eq.

Original: Furtherassume that X 
Revised: Further assume that X ′

Original: ′ ̸
Revised: ̸

Original: = X and that p X ,y ∗ ∈ (0 , 1) .
Revised: = X and that p X ,y ∗ ∈ (0 , 1) .

Original: Then:
Revised: Then:

Original: [Equation]
Revised: [Equation]

Original: We can do so by showing that no optimal solution to the r.h.s.
Revised: We can do so by showing that no optimal solution to the r.h.s.

Original: black-box problem from Eq.
Revised: black-box problem from Eq.

Original: (95)is a feasible solution to the l.h.s.
Revised: (42) is a feasible solution to the l.h.s.

Original: gray-box problem.
Revised: gray-box problem.

Original: More formally, let H ∗ ⊆
Revised: More formally, let H ∗ ⊆

Original: H be the set of allclassifiers minimizing the r.h.s.
Revised: H be the set of all classifiers minimizing the r.h.s.

Original: black box problem.
Revised: black box problem.

Original: We have H ∗ ⊆
Revised: We have H ∗ ⊆

Original: H and H T ⊆ H .
Revised: H and H T ⊆ H .

Original: By showing that1761 H ∗ ∩ H T = ∅ , we prove that H T ⊆
Revised: By showing that H ∗ ∩ H T = ∅ , we prove that H T ⊆

Original: H \ H ∗ , i.e. all rotation invariant classifiers from H T yield strictlylarger optimal values.
Revised: H \ H ∗ , i.e. all rotation invariant classifiers from H T yield strictly larger optimal values.

Original: Recall that an optimal solution to the r.h.s.
Revised: Recall that an optimal solution to the r.h.s.

Original: black-box problem in Eq.
Revised: black-box problem in Eq.

Original: (95) is given by the NeymanPearson lemma [104].
Revised: (42) is given by the NeymanPearson lemma [107].

Original: Later work on hypothesis testing has shown that the Neyman-Pearson lemmais not only a sufficient, but a necessary condition for optimality[134]: Every most powerful test mustfulfill the likelihood ratio inequalities, save for sets of zero measure.
Revised: Later work on hypothesis testing has shown that the Neyman-Pearson lemma is not only a sufficient, but a necessary condition for optimality[143]: Every most powerful test must fulfill the likelihood ratio inequalities, save for sets of zero measure.

Original: For our formulation of the
Revised: For our formulation of the

Original: Neyman-Pearson lemma (see Lemma 2), this means that any classifier h that is an optimal solution tothe r.h.s.
Revised: Neyman-Pearson lemma (see Lemma 5), this means that any classifier h that is an optimal solution to the r.h.s.

Original: black-box problem from Eq.
Revised: black-box problem from Eq.

Original: (95) must fulfill
Revised: (42) must fulfill

Original: [Equation]
Revised: [Equation]

Original: for some t ∈ R , save for sets of zero measure.
Revised: for some κ ∈ R , save for sets of zero measure.

Original: Cohen et al.
Revised: Cohen et al.

Original: [27] have shown that if X ′ ̸
Revised: [26] have shown that if X ′ ̸

Original: = X andp ∈ (0 , 1) and µ X , µ X ′ are isotropic Gaussian distributions, then these likelihood ratio inequalitiescorrespond to a linear decision boundary, i.e. any optimal solution most fulfill[Equation]
Revised: = X and p ∈ (0 , 1) and µ X , µ X ′ are isotropic Gaussian distributions, then these likelihood ratio inequalities correspond to a linear decision boundary, i.e. any optimal solution most fulfill

Original: [Equation]
Revised: [Equation]

Original: for some b ∈ R and W ∈ R N × D with w ̸ = 0 N,D , save for sets of zero measure.
Revised: for some b ∈ R and W ∈ R N × D with w ̸ = 0 N,D , save for sets of zero measure.

Original: Here, ⟨ A , B ⟩ F =vec( W ) T vec( Z ) is the Frobenius inner product.
Revised: Here, ⟨ A , B ⟩ F = vec( W ) T vec( Z ) is the Frobenius inner product.

Original: Therefore, we can prove Lemma 14 by showing that classifiers complying with Eq.
Revised: Therefore, we can prove Lemma 8 by showing that classifiers complying with Eq.

Original: (96) (save forsets of zero measure) are not rotation invariant, i.e. they are not feasible solutions to the gray-boxoptimization problem.
Revised: (43) (save for sets of zero measure) are not rotation invariant, i.e. they are not feasible solutions to the gray-box optimization problem.

Original: We first do this for the case that D is even and then for the case that D is odd.
Revised: We first do this for the case that D is even and then for the case that D is odd.

Original: Lemma 15.
Revised: Lemma 9.

Original: Assume that D = 2 k for some k ∈ N .
Revised: Assume that D = 2 k for some k ∈ N .

Original: Let h : R N × D
Revised: Let h : R N × D

Original: → Y be a classifier that fulfills
Revised: → Y be a classifier that fulfills

Original: [Equation]
Revised: [Equation]

Original: for some b ∈ R and w ∈ R N × D with w ̸ = 0 N · D , save for sets of zero measure.
Revised: for some b ∈ R and w ∈ R N × D with w ̸ = 0 N · D , save for sets of zero measure.

Original: Then, there is aninput ˆ Z and a rotation matrix R ∈
Revised: Then, there is an input ˆ Z and a rotation matrix R ∈

Original: SO ( D ) such that h ( ˆ Z ) ̸
Revised: SO ( D ) such that h ( ˆ Z ) ̸

Original: = h ( ˆ ZR T ) .
Revised: = h ( ˆ ZR T ) .

Original: Proof.
Revised: Proof.

Original: Case 1.
Revised: Case 1.

Original: Assume that b ≥ 0 .
Revised: Assume that b ≥ 0 .

Original: Consider the set S
Revised: Consider the set S

Original: = { Z | ⟨ W , Z ⟩ F < b } .
Revised: = { Z | ⟨ W , Z ⟩ F < b } .

Original: By construction, we have ∀ Z ∈ S : ⟨ W , Z ⟩ F + b < 01783 and ∀ Z ∈ S : ⟨ W , − Z ⟩ F + b > 0 .
Revised: By construction, we have ∀ Z ∈ S : ⟨ W , Z ⟩ F + b < 0 and ∀ Z ∈ S : ⟨ W , − Z ⟩ F + b > 0 .

Original: There must be at least one ˆ Z ∈ S with h ( ˆ Z ) =
Revised: There must be at least one ˆ Z ∈ S with h ( ˆ Z ) =

Original: y ∗ and h ( ˆ Z )
Revised: y ∗ and h ( ˆ Z )

Original: ̸
Revised: ̸

Original: = y ∗ .
Revised: = y ∗ .

Original: Otherwise, all points Z ∈ Swould need to fulfill h ( Z ) ̸
Revised: Otherwise, all points Z ∈ S would need to fulfill h ( Z ) ̸

Original: = y ∗ or h ( − Z ) =
Revised: = y ∗ or h ( − Z ) =

Original: y ∗ , in which case we would have sets of non-zeromeasure violating the likelihood ratio inequalities from Eq.
Revised: y ∗ , in which case we would have sets of non-zero measure violating the likelihood ratio inequalities from Eq.

Original: Note that − ˆ Z = ˆ Z ( − I N ) T and − I N ∈
Revised: Note that − ˆ Z = ˆ Z ( − I N ) T and − I N ∈

Original: SO
Revised: SO

Original: ( D ) , because − I N is orthonormal and det( − I N )
Revised: ( D ) , because − I N is orthonormal and det( − I N )

Original: = 1 ,due to D being even.
Revised: = 1 , due to D being even.

Original: Thus, we have found an input ˆ Z and a rotation matrix R ∈
Revised: Thus, we have found an input ˆ Z and a rotation matrix R ∈

Original: SO ( D ) such thath ( Z ) ̸ = h ( ZR T ) .
Revised: SO ( D ) such that h ( Z ) ̸ = h ( ZR T ) .

Original: Case 2.
Revised: Case 2.

Original: Assume that b < 0 .
Revised: Assume that b < 0 .

Original: This case follows analogously by constructing the set S
Revised: This case follows analogously by constructing the set S

Original: ={ Z | ⟨ W , Z ⟩ F > b } and considering a point ˆ Z ∈ S with h ( ˆ Z )
Revised: = { Z | ⟨ W , Z ⟩ F > b } and considering a point ˆ Z ∈ S with h ( ˆ Z )

Original: ̸ = y ∗ and h ( − ˆ Z )
Revised: ̸ = y ∗ and h ( − ˆ Z )

Original: =
Revised: =

Original: y ∗
Revised: y ∗

Original: Now, we consider the case that D is odd.
Revised: Now, we consider the case that D is odd.

Original: Here, − I N / ∈ SO ( D ) , because det( − I N ) = − 1 .
Revised: Here, − I N / ∈ SO ( D ) , because det( − I N ) = − 1 .

Original: Therefore,1793 we will need to use slightly more complicated constructions.
Revised: Therefore, we will need to use slightly more complicated constructions.

Original: Lemma 16.
Revised: Lemma 10.

Original: Assume that D = 2 k − 1 for some k ∈ N .
Revised: Assume that D = 2 k − 1 for some k ∈ N .

Original: Let h : R N × D
Revised: Let h : R N × D

Original: → Y be a classifier thatfulfills
Revised: → Y be a classifier that fulfills

Original: [Equation]
Revised: [Equation]

Original: for some b ∈ R and w ∈ R N × D with w ̸ = 0 N · D , save for sets of zero measure.
Revised: for some b ∈ R and w ∈ R N × D with w ̸ = 0 N · D , save for sets of zero measure.

Original: Then, there is aninput ˆ Z and a rotation matrix R ∈
Revised: Then, there is an input ˆ Z and a rotation matrix R ∈

Original: SO ( D ) such that h ( ˆ Z ) ̸
Revised: SO ( D ) such that h ( ˆ Z ) ̸

Original: = h ( ˆ ZR T ) .
Revised: = h ( ˆ ZR T ) .

Original: Proof.
Revised: Proof.

Original: In the following, let A =(cid:20)
Revised: In the following, let A =

Original: [Equation] I [Equation] (cid:21) .
Revised: (cid:20)0 TD 0 D −− I D −(cid:21) .

Original: We have A ∈ SO ( D ) , because A is orthonormal and det( A )
Revised: We have A ∈ SO ( D ) , because A is orthonormal and det( A )

Original: = 1 , due to D being odd.
Revised: = 1 , due to D being odd.

Original: Case 1.
Revised: Case 1.

Original: Assume that b ≥ 0 .
Revised: Assume that b ≥ 0 .

Original: We know that W ̸ = 0 N,D .
Revised: We know that W ̸ = 0 N,D .

Original: Without loss of generality, assume that W : , 2: ̸ = 0 N, ( D − 1) , i.e. at least1802 one of the the last D − 1 columns is non-zero.
Revised: Without loss of generality, assume that W : , 2: ̸ = 0 N, ( D − 1) , i.e. at least one of the the last D − 1 columns is non-zero.

Original: Consider the set S
Revised: Consider the set S

Original: = (cid:8) Z | W T : , 1 Z : , 1 ∈ [0 , 1] ∧ ⟨ W : , 2: , Z : , 2: ⟩ F < b − 1 (cid:9) .
Revised: = (cid:8) Z | W T : , 1 Z : , 1 ∈ [0 , 1] ∧ ⟨ W : , 2: , Z : , 2: ⟩ F < b − 1 (cid:9) .

Original: By construction, we have∀ Z ∈ S : ⟨ W , Z ⟩ F + b < 0 and ∀ Z ∈ S : (cid:10) W , ZA T (cid:11) F + b > 0 .
Revised: By construction, we have ∀ Z ∈ S : ⟨ W , Z ⟩ F + b < 0 and ∀ Z ∈ S : (cid:10) W , ZA T (cid:11) F + b > 0 .

Original: There must be at least one ˆ Z ∈ S with h ( ˆ Z ) =
Revised: There must be at least one ˆ Z ∈ S with h ( ˆ Z ) =

Original: y ∗ and h ( ˆ ZA T )
Revised: y ∗ and h ( ˆ ZA T )

Original: ̸
Revised: ̸

Original: = y ∗ .
Revised: = y ∗ .

Original: Otherwise, all points Z ∈ Swould need to fulfill h ( Z ) ̸
Revised: Otherwise, all points Z ∈ S would need to fulfill h ( Z ) ̸

Original: = y ∗ or h ( ZA T ) =
Revised: = y ∗ or h ( ZA T ) =

Original: y ∗ , in which case we would have sets of non-zeromeasure violating Eq.
Revised: y ∗ , in which case we would have sets of non-zero measure violating Eq.

Original: Case 2.
Revised: Case 2.

Original: Assume that b < 0 .
Revised: Assume that b < 0 .

Original: This case follows analogously by constructing the set S =1809 (cid:8)
Revised: This case follows analogously by constructing the set S = (cid:8)

Original: Z | W T : , 1 Z : , 1 ∈ [ − 1 , 0] ∧ ⟨ W : , 2: , Z : , 2: ⟩ F > b + 1 (cid:9) and considering a point ˆ Z ∈ S with h ( ˆ Z ) ̸
Revised: [Equation] , 0] ∧ ⟨ W : , 2: , Z : , 2: ⟩ F > b + 1 (cid:9) and considering a point ˆ Z ∈ S with h ( ˆ Z ) ̸

Original: = y ∗and h ( A ˆ Z ) =
Revised: = y ∗ and h ( A ˆ Z ) =

Original: y ∗ .
Revised: y ∗ .

Original: Lemmas 15 and 16 combined prove Lemma 14, which in turn proves Theorem 6.
Revised: Lemmas 9 and 10 combined prove Lemma 8, which in turn proves Theorem 5.

Original: F Multi-class certificates
Revised: H Multi-class certificates

Original: In our discussions in Sections 3, 5 and 6, we have only considered binary certificates 4 .
Revised: In our discussions in Sections 3.1, 5 and 6, we have only considered binary certificates 5 .

Original: That is,prediction y ∗ of smoothed classifier is certifiably robust to a perturbed input X ′ if the probabilityof base classifier g predicting y ∗ under perturbed smoothing distribution µ X ′ is greater than theprobability of all other classes combined.
Revised: That is, prediction y ∗ of smoothed classifier is certifiably robust to a perturbed input X ′ if the probability of base classifier g predicting y ∗ under perturbed smoothing distribution µ X ′ is greater than the probability of all other classes combined.

Original: That is, p X ′ ,y ∗ > 12 .
Revised: That is, p X ′ ,y ∗ > 12 .

Original: We have then, for different invariances,computed lower bounds p X ′ ,y ∗ ≤ p X ′ ,y ∗ .
Revised: We have then, for different invariances, computed lower bounds p X ′ ,y ∗ ≤ p X ′ ,y ∗ .

Original: If p X ′ ,y ∗ > 12 , then p X ′ ,y ∗ > 12 and the prediction iscertifiably robust.
Revised: If p X ′ ,y ∗ > 12 , then p X ′ ,y ∗ > 12 and the prediction is certifiably robust.

Original: Alternatively, one can certify robustness by showing that y ∗ remains more likely than the secondmost likely class under the perturbed smoothing distribution [27], i.e. p X ′ ,y ∗ > max y ̸
Revised: Alternatively, one can certify robustness by showing that y ∗ remains more likely than the second most likely class under the perturbed smoothing distribution [26], i.e. p X ′ ,y ∗ > max y ̸

Original: =
Revised: =

Original: y ∗ p X ′ ,y .
Revised: y ∗ p X ′ ,y .

Original: Tothis end, one can compute a lower bound p X ′ ,y ∗ ≤ p X ′ ,y ∗ and an upper-bound max y ̸
Revised: To this end, one can compute a lower bound p X ′ ,y ∗ ≤ p X ′ ,y ∗ and an upper-bound max y ̸

Original: =
Revised: =

Original: y ∗ p X ′ ,y ≥max y ̸
Revised: y ∗ p X ′ ,y ≥ max y ̸

Original: =
Revised: =

Original: y ∗ p X ′ ,y .
Revised: y ∗ p X ′ ,y .

Original: If p X ′ ,y ∗ > max y ̸
Revised: If p X ′ ,y ∗ > max y ̸

Original: =
Revised: =

Original: y ∗ p X ′ ,y , then robustness is guaranteed.
Revised: y ∗ p X ′ ,y , then robustness is guaranteed.

Original: Due to the monotonicityof randomized smoothing certificates w.r.t.
Revised: Due to the monotonicity of randomized smoothing certificates w.r.t.

Original: to the prediction probabilities, this can be achieved byshowing that p X ′ ,y ∗ > p X ′ ,y ′ , where y ′ =
Revised: to the prediction probabilities, this can be achieved by showing that p X ′ ,y ∗ > p X ′ ,y ′ , where y ′ =

Original: argmax y ̸
Revised: argmax y ̸

Original: =
Revised: =

Original: y ∗ p X ,y is the second most likely class underthe clean smoothing distribution.
Revised: y ∗ p X ,y is the second most likely class under the clean smoothing distribution.

Original: This second most likely class y ′ can be found via a binomial test,see [27, 135].
Revised: This second most likely class y ′ can be found via a binomial test, see [26, 144].

Original: The lower bound p X ′ ,y ∗ can be obtained using the same formulae discussed throughout the main text(e.g. p X ′ ,y ∗ =
Revised: The lower bound p X ′ ,y ∗ can be obtained using the same formulae discussed throughout the main text (e.g. p X ′ ,y ∗ =

Original: Φ (cid:16) Φ − 1 ( p X ,y ′ ) − || ∆ || 2 σ (cid:17) for black-box randomized smoothing).
Revised: Φ (cid:16) Φ − 1 ( p X ,y ′ ) − || ∆ || 2 σ (cid:17) for black-box randomized smoothing).

Original: The upper boundp X ′ ,y ′ can be found via the Neyman-Pearson upper bound discussed in Appendix D.1:
Revised: The upper bound p X ′ ,y ′ can be found via the Neyman-Pearson upper bound discussed in Appendix F.2:

Original: Lemma 3 (Neyman-Pearson upper bound) .
Revised: Lemma 6 (Neyman-Pearson upper bound) .

Original: Let µ X ′ , µ X , be two continuous distributions over ameasurable
Revised: Let µ X ′ , µ X , be two continuous distributions over a measurable set

Original: set A such that, for all t ∈ R + , the set (cid:110) z | µ X ′ ( z ) µ X ( z ) =
Revised: A such that, for all κ ∈ R + , the set (cid:110) z | µ X ′ ( z ) µ X ( z ) =

Original: t (cid:111) has measure zero.
Revised: κ (cid:111) has measure zero.

Original: Consider anarbitrary label set Y , a specific class label y ∈ Y and scalar p ∈ [0 , 1] .
Revised: Consider an arbitrary label set Y , a specific class label y ∈ Y and scalar p ∈ [0 , 1] .

Original: Then
Revised: Then

Original: [Equation]
Revised: [Equation]

Original: [Equation]
Revised: with h ∗ ( z ) = 1 (cid:20)

Original: [Equation]
Revised: [Equation]

Original: with h
Revised: X ′

Original: [Equation]
Revised: [Equation]

Original: t
Revised: and κ ∈ R + such that

Original: [Equation]
Revised: [Equation]

Original: Note that it is identical to the lower bound, save for replacing " ≤ " with " ≥ " and vice-versa.
Revised: Note that it is identical to the lower bound, save for replacing " ≤ " with " ≥ " and vice-versa.

Original: The resulting black-box upper bound is p X ′ ,y ′ =
Revised: The resulting black-box upper bound is p X ′ ,y ′ =

Original: Φ (cid:16) Φ − 1 ( p X ,y ′ )
Revised: Φ (cid:16) Φ − 1 ( p X ,y ′ )

Original: + || ∆ || 2 σ (cid:17) .
Revised: + || ∆ || 2 σ (cid:17) .

Original: It is identical to theblack-box lower bound, save for replacing " − " with " + ".
Revised: It is identical to the black-box lower bound, save for replacing " − " with " + ".

Original: Substituting both bounds into p X ′ ,y ∗ >max y ̸
Revised: Substituting both bounds into p X ′ ,y ∗ > max y ̸

Original: =
Revised: =

Original: y ∗ p X ′ ,y and applying the inverse-normal CDF Φ − 1 to both sides shows that the classifier iscertifiably robust if || ∆ || ≤ σ 2 (cid:0) Φ − 1 ( p X ′ ,y ∗ ) − Φ − 1 ( p X ′ ,y ′ )
Revised: y ∗ p X ′ ,y and applying the inverse-normal CDF Φ − 1 to both sides shows that the classifier is certifiably robust if || ∆ || ≤ σ 2 (cid:0) Φ − 1 ( p X ′ ,y ∗ ) − Φ − 1 ( p X ′ ,y ′ )

Original: (cid:1) [27].
Revised: (cid:1) [26].

Original: Evidently, the post-processing-based versions of the multi-class certificates are identical to theirbinary counterparts, except that the radius of the underlying black-box certificate changes.
Revised: Evidently, the orbit-based versions of the multi-class certificates are identical to their binary counterparts, except that the radius of the underlying black-box certificate changes.

Original: For the tight gray-box approach, upper bounds can be derived by going through the same derivationsas in Appendix D, but using the Neyman-Pearson upper bounds from Lemma 3, i.e. replacing " ≤ "with " ≥ " and vice-versa.
Revised: For the tight gray-box approach, upper bounds can be derived by going through the same derivations as in Appendix F, but using the Neyman-Pearson upper bounds from Lemma 6, i.e. replacing " ≤ " with " ≥ " and vice-versa.

Original: (see Lemma 3).
Revised: (see Lemma 6).

Original: In the case of translation invariance, this results in upperbound Φ (cid:16)
Revised: In the case of translation invariance, this results in upper bound Φ (cid:16)

Original: Φ − 1 ( p X ,y ′ )
Revised: Φ − 1 ( p X ,y ′ )

Original: + || ∆ − 1 N ∆ || 2 σ (cid:17) ) .
Revised: + || ∆ − 1 N ∆ || 2 σ (cid:17) ) .

Original: In the cases involving rotation invariance, this results inbounds of the form
Revised: In the cases involving rotation invariance, this results in bounds of the form

Original: [Equation]
Revised: [Equation]

Revised: with κ ∈ R such that

Revised: [Equation]

Original: i.e. the same certificates, but with “ ≥ t ” instead of “ ≤ t ”.
Revised: i.e. the same certificates, but with “ ≥ κ ” instead of “ ≤ κ ”.

Original: Our Monte Carlo certification procedure proposed in Appendix D.9 can also easily be adapted tocomputing an upper bound by replacing " ≤ " with " ≥ " and vice-versa.:
Revised: Our Monte Carlo certification procedure proposed in Appendix F.5 can also easily be adapted to computing an upper bound by replacing " ≤ " with " ≥ " and vice-versa.:

Original: [Table]
Revised: [Table]

Original: Note that we still need to lower-bound threshold t , because this increases the probability
Revised: Note that we still need to lower-bound threshold κ , because this increases the probability

Original: Pr V ∼ υ (1) [ ρ ( V ) ≥ t ] and thus leads to more pessimistic, sound multi-class certificates.
Revised: Pr V ∼ υ (1) [ ρ ( V ) ≥ t ] and thus leads to more pessimistic, sound multi-class certificates.

Original: Finally note that, because the multi-class certificates are just a combination of two binary certificates,our evaluation of binary invariance-aware certificates in in Section 8 is a good indicator of theperformance of invariance-aware multi-class certificates.
Revised: Finally note that, because the multi-class certificates are just a combination of two binary certificates, our evaluation of binary invariance-aware certificates in in Section 8 is a good indicator of the performance of invariance-aware multi-class certificates.

Original: G Inverse certificates
Revised: I Inverse certificates

Original: In Section 8.1, we evaluate inverse certificates, i.e. compute the smallest prediction probability p minfor which robustness can still be certified, given the remaining certificate parameters such as || ∆ || 21856 or || X
Revised: In Section 8.1, we evaluate inverse certificates, i.e. compute the smallest prediction probability p min for which robustness can still be certified, given the remaining certificate parameters such as || ∆ || 2 or || X

Original: || 2 .
Revised: || 2 .

Original: In the case of the black-box randomized smoothing baseline, p min can be calculated by solvingΦ (cid:16) Φ ( − 1) ( p ) − || ∆ || 2 σ (cid:17)
Revised: In the case of the black-box randomized smoothing baseline, p min can be calculated by solving Φ (cid:16) Φ ( − 1) ( p ) − || ∆ || 2 σ (cid:17)

Original: =
Revised: =

Original: 12 for p , which results in p min =
Revised: 12 for p , which results in p min =

Original: Φ (cid:16) ||
Revised: Φ (cid:16) ||

Original: ∆ || 2 σ (cid:17) .
Revised: ∆ || 2 σ (cid:17) .

Original: Similarly, the gray-box certificates for translation invariance has p min =
Revised: Similarly, the gray-box certificates for translation invariance has p min =

Original: Φ (cid:16) || ∆ − 1 N ∆ || 2 σ (cid:17) , where∆ ∈ R 1 × D are the column-wise averages.
Revised: Φ (cid:16) || ∆ − 1 N ∆ || 2 σ (cid:17) , where ∆ ∈ R 1 × D are the column-wise averages.

Original: The tight gray-box certificates involving rotation invariance do not have closed-form analytic expressions.
Revised: The tight gray-box certificates involving rotation invariance do not have closed-form analytic expressions.

Original: Instead, they are of the form
Revised: Instead, they are of the form

Original: [Equation]
Revised: [Equation]

Original: with t ∈ R s.t.
Revised: with κ ∈ R such that

Original: [Equation]
Revised: [Equation]

Original: Here, an inverse certificate can be computed by first finding a threshold t such that
Revised: Here, an inverse certificate can be computed by first finding a threshold κ such that

Original: Pr V ∼ υ (1) [ ρ ( V ) ≤ t ] = 12 and then evaluating Pr V ∼ υ [Equation] [ ρ ( V ) ≤ t ] .
Revised: Pr V ∼ υ (1) [ ρ ( V ) ≤ κ ] = 12 and then evaluating Pr V ∼ υ (2) [ ρ ( V ) ≤ κ ] .

Original: That is,
Revised: That is,

Original: [Equation]
Revised: [Equation]

Original: with t ∈ R s.
Revised: with κ ∈ R such that

Original: [Equation]
Revised: [Equation]

Original: To ensure a fair comparison with our baselines, we modify the Monte Carlo certification procedureproposed in Appendix D.9 to compute a probabilistic upper bound on p min that holds with highprobability 1 − α .
Revised: To ensure a fair comparison with our baselines, we modify the Monte Carlo certification procedure proposed in Appendix F.5 to compute a probabilistic upper bound on p min that holds with high probability 1 − α .

Original: function I NVERSE P ROB C ERTIFY
Revised: function I NVERSE P ROB C ERTIFY

Original: ( υ (1) , υ (2) , N 1 , N 2 , α ) n ∗ ← min { n | B IN PV ALUE ( n, N 1 , ≤ , 0 . 5) < α }
Revised: ( υ (1) , υ (2) , N 1 , N 2 , α ) n [Equation] ← min { n | B IN PV ALUE ( n, N 1 , ≤ , 0 . 5) < α }

Original: V (1) , . .
Revised: V (1) , . .

Original: , V ( N 2 ) ← S AMPLE ( υ (1) , N 1 )
Revised: , V ( N 2 ) ← S AMPLE ( υ (1) , N 1 )

Original: R (1) , . . .
Revised: R (1) , . . .

Original: , R ( N 2 ) ← S ORT A SCENDING ( ρ ( V (1) ) , . .
Revised: , R ( N 2 ) ← S ORT A SCENDING ( ρ ( V (1) ) , . .

Original: , ρ ( V ( N 2 ) ) ) t ← R ( n ∗ )
Revised: , ρ ( V ( N 2 ) ) ) κ ← R ( n ∗ )

Original: ▷ First bound
Revised: ▷ First bound

Original: V (1) , . .
Revised: V (1) , . .

Original: , V ( N 2 ) ← S AMPLE ( υ (2) , N 3 ) count 2 ← (cid:80) N 2 n =1 1 (cid:2) ρ ( V ) ≤ t (cid:3) return U PPER C ONF B OUND ( count 2 , N 2 , 1 − α 2 )
Revised: , V ( N 2 ) ← S AMPLE ( υ (2) , N 3 ) count 2 ← (cid:80) N 2 n =1 1 [ ρ ( V ) ≤ κ ] return U PPER C ONF B OUND ( count 2 , N 2 ,− α 2 )

Original: ▷ Second bound
Revised: ▷ Second bound

Original: Note that, different from Appendix D.9 we compute a probabilistic upper bound on threshold t byfinding the smallest order statistic R ( n ) such that (cid:2) ρ ( V ) ≤ R ( n ) (cid:3) ≥ 12 holds with high probability.
Revised: Note that, different from Appendix F.5 we compute a probabilistic upper bound on threshold κ by finding the smallest order statistic R ( n ) such that (cid:2) ρ ( V ) ≤ R ( n ) (cid:3) ≥ 12 holds with high probability.

Original: H Parameter space of the tight certificate for2D rotation invariance 
Revised: J Parameter space of the tight certificate for rotation invariance in 2D

Original: Our tight certificates for 2D rotation invariance depend on || X
Revised: Our tight certificates for 2D rotation invariance depend on || X

Original: || 2 , || ∆ || 2 , as well as parametersϵ 1 = ⟨ X , ∆ ⟩ F , ϵ

Original: [Equation]

Original: =

Original: [Equation]

Original: −

Original: [Equation]

Original: [Equation] T , ∆ [Equation] F , which capture the orientation of the perturbed pointcloud,relative to the clean point cloud.
Revised: || 2 , || ∆ || 2 , as well as parameters ϵ 1 = ⟨ X , ∆ ⟩ F , ϵ 2 = (cid:68) XR (cid:0) − π 2 (cid:1) T , ∆ (cid:69) F , which capture the orientation of the perturbed pointcloud, relative to the clean point cloud.

Original: In this section, we determine the feasible range of ϵ 1 and ϵ 2 and calculate the two pairs of valuescorresponding to adversarial rotations, which is relevant for our experiments in Section 8.1.
Revised: In this section, we determine the feasible range of ϵ 1 and ϵ 2 and calculate the two pairs of values corresponding to adversarial rotations, which is relevant for our experiments in Section 8.1.

Original: H.1 Feasible parameter range
Revised: J.1 Feasible parameter range

Original: Let ˆ X =
Revised: Let ˆ X =

Original: X
Revised: X

Original: ||
Revised: ||

Original: X
Revised: X

Original: || 2 .
Revised: || 2 .

Original: Then, parameters ϵ 1 and ϵ 2 can be equivalently stated as follows:
Revised: Then, parameters ϵ 1 and ϵ 2 can be equivalently stated as follows:

Original: [Equation]
Revised: [Equation]

Original: Note that vec( ˆ X ) and vec (cid:16) ˆ XR (cid:0) − π 2 (cid:1) T (cid:17) have norm 1 and are orthogonal to each other.
Revised: Note that vec( ˆ X ) and vec (cid:16) ˆ XR (cid:0) − π 2 (cid:1) T (cid:17) have norm 1 and are orthogonal to each other.

Original: Thus, theyare the first two elements of an orthonormal basis of R 2 N .
Revised: Thus, they are the first two elements of an orthonormal basis of R 2 N .

Original: Parameters ϵ 1 and ϵ 2 are projections ofvec( || X || 2 ∆ ) onto the first two elements of this basis.
Revised: Parameters ϵ 1 and ϵ 2 are projections of vec( || X || 2 ∆ ) onto the first two elements of this basis.

Original: Basis changes preserve the norm of vectors.
Revised: Basis changes preserve the norm of vectors.

Original: Thus, the values ϵ 1 , ϵ 2 must fulfill (cid:112) ϵ 21 + ϵ 22 ≤
Revised: Thus, the values ϵ 1 , ϵ 2 must fulfill (cid:112) ϵ 21 + ϵ 22 ≤

Original: ||
Revised: ||

Original: X
Revised: X

Original: || 2 · || ∆ || 2 .
Revised: || 2 · || ∆ || 2 .

Original: H.2 Adversarial rotations
Revised: J.2 Adversarial rotations

Original: Known rotation angle.
Revised: Known rotation angle.

Original: Consider an arbitrary X ∈ R N × 2 .
Revised: Consider an arbitrary X ∈ R N × 2 .

Original: Further consider an adversariallyperturbed X ′ =
Revised: Further consider an adversarially perturbed X ′ =

Original: XR ( θ ) T that is the result of rotating all rows of X by angle θ .
Revised: XR ( θ ) T that is the result of rotating all rows of X by angle θ .

Original: Let ∆ =
Revised: Let ∆ =

Original: X ′ − X .
Revised: X ′ − X .

Original: Recall that, for any vector a , b ∈ R D , a T b T
Revised: Recall that, for any vector a , b ∈ R D , a T b T

Original: = ||
Revised: = ||

Original: a || 2 || b
Revised: a || 2 || b

Original: || 2 cos( ∠ ab ) .
Revised: || 2 cos( ∠ ab ) .

Original: Thereforeϵ 1 = ⟨ X ,
Revised: Therefore [Equation]

Original: [Equation]
Revised: [Equation]

Original: where the last equality follows from the definition of the Frobenius norm.
Revised: where the last equality follows from the definition of the Frobenius norm.

Original: Similarly, we have
Revised: Similarly, we have

Original: [Equation]
Revised: [Equation]

Original: Unknown rotation angle.
Revised: Unknown rotation angle.

Original: Now assume that we know
Revised: Now assume that we know

Original: ||
Revised: ||

Original: X
Revised: X

Original: || 2 and || ∆
Revised: || 2 and || ∆

Original: || 2 , but do not know the rotation1888 angle θ .
Revised: || 2 , but do not know the rotation angle θ .

Original: We have
Revised: We have

Original: [Equation]
Revised: [Equation]

Original: Evidently, such an adversarial rotation is only possible if || ∆ || ≤ 2 ||
Revised: Evidently, such an adversarial rotation is only possible if || ∆ || ≤ 2 ||

Original: X || , which corresponds to a1890 rotation by angle π .
Revised: X || , which corresponds to a rotation by angle π .

Original: Solving for θ yields
Revised: Solving for θ yields

Original: [Equation]
Revised: [Equation]

Original: Inserting into our formulae for ϵ 1 and ϵ 2 and using that sin(arccos( a ))
Revised: Inserting into our formulae for ϵ 1 and ϵ 2 and using that sin(arccos( a ))

Original: = √ 1 − a 2 shows that
Revised: = √ 1 − a 2 shows that

Original: [Equation]
Revised: [Equation]

Original:  || [Equation] [Equation]
Revised: ∆ || 22 (4 || X || 22 − || ∆ || 22 ) .