Original: F ULLY S TEERABLE 3D S PHERICAL N EURONS
Revised: F ULLY S TEERABLE 3D S PHERICAL N EURONS

Original: Anonymous authors Paper under double-blind review
Revised: Anonymous authors Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: Emerging from low-level vision theory, steerable filters found their counterpart in prior work on steerable convolutional neural networks equivariant to rigid transformations.
Revised: Emerging from low-level vision theory, steerable filters found their counterpart in prior work on steerable convolutional neural networks equivariant to rigid transformations.

Original: In our work, we propose a steerable feed-forward learning-based approach that consists of spherical decision surfaces and operates on point clouds.
Revised: In our work, we propose a steerable feed-forward learning-based approach that consists of spherical decision surfaces and operates on point clouds.

Original: Focusing on 3D geometry, we derive a 3D steerability constraint for hypersphere neurons, which are obtained by conformal embedding of Euclidean space and have recently been revisited in the context of learning representations of point sets.
Revised: Focusing on 3D geometry, we derive a 3D steerability constraint for hypersphere neurons, which are obtained by conformal embedding of Euclidean space and have recently been revisited in the context of learning representations of point sets.

Original: Exploiting the rotational equivariance, we show how our model parameters are fully steerable at inference time.
Revised: Exploiting the rotational equivariance, we show how our model parameters are fully steerable at inference time.

Original: We use a synthetic point set and real-world 3D skeleton data to show how the proposed spherical filter banks enable making equivariant and, after online optimization, invariant class predictions for known point sets in unknown orientations.
Revised: We use a synthetic point set and real-world 3D skeleton data to show how the proposed spherical filter banks enable making equivariant and, after online optimization, invariant class predictions for known point sets in unknown orientations.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: We present a novel feed-forward model consisting of steerable 3D neurons for point cloud classification, an important and challenging problem with many applications such as autonomous driving, human-robot interaction, and mixed-reality installations.
Revised: We present a novel feed-forward model consisting of steerable 3D neurons for point cloud classification, an important and challenging problem with many applications such as autonomous driving, human-robot interaction, and mixed-reality installations.

Original: We achieve the steerability by using geometric neurons (Melnyk et al., 2021), leading to a rotation equivariant network.
Revised: We achieve the steerability by using geometric neurons (Melnyk et al., 2021), leading to a rotation equivariant network.

Original: Besides becoming a geometrically explainable approach, further benefits of the proposed method include rotation invariant classification and, therefore, lowered data augmentation requirements during learning.
Revised: Besides becoming a geometrically explainable approach, further benefits of the proposed method include rotation invariant classification and, therefore, lowered data augmentation requirements during learning.

Original: We make use of a conformal embedding to obtain higher-order decision surfaces.
Revised: We make use of a conformal embedding to obtain higher-order decision surfaces.

Original: Following the motivation in the recent work of Melnyk et al.
Revised: Following the motivation in the recent work of Melnyk et al.

Original: (2021), we focus on 3D geometry and spherical decision surfaces, arguing for their natural suitability for problems in Euclidean space.
Revised: (2021), we focus on 3D geometry and spherical decision surfaces, arguing for their natural suitability for problems in Euclidean space.

Original: We show how a spherical neuron ( classifier ), i.e., the hypersphere neuron (Banarer et al., 2003b) or its generalization for 3D input point sets — the geometric neuron (Melnyk et al., 2021) — can be turned into a steerable neuron.
Revised: We show how a spherical neuron ( classifier ), i.e., the hypersphere neuron (Banarer et al., 2003b) or its generalization for 3D input point sets — the geometric neuron (Melnyk et al., 2021) — can be turned into a steerable neuron.

Original: We prove that the aforementioned spherical neurons in any dimension require only up to first-degree spherical harmonics to accommodate the effect of rotation.
Revised: We prove that the aforementioned spherical neurons in any dimension require only up to first-degree spherical harmonics to accommodate the effect of rotation.

Original: This allows us to derive a 3D steerability constraint for the spherical neurons and to describe a recipe for creating a steerable model from a pretrained spherical classifier.
Revised: This allows us to derive a 3D steerability constraint for the spherical neurons and to describe a recipe for creating a steerable model from a pretrained spherical classifier.

Original: Using the synthetic Tetris dataset (Thomas et al., 2018) and the skeleton data from the UTKinectAction3D dataset (Xia et al., 2012), we first verify the derived constraint and check its stability with respect to perturbations in the input.
Revised: Using the synthetic Tetris dataset (Thomas et al., 2018) and the skeleton data from the UTKinectAction3D dataset (Xia et al., 2012), we first verify the derived constraint and check its stability with respect to perturbations in the input.

Original: We further conduct an experiment in a realistic setting, where we initialize the steerable model parameters using a noisy or imperfect prediction of the transformation applied to the input and optimize the parameters in an unsupervised way.
Revised: We further conduct an experiment in a realistic setting, where we initialize the steerable model parameters using a noisy or imperfect prediction of the transformation applied to the input and optimize the parameters in an unsupervised way.

Original: The contributions of our work are as follows:
Revised: The contributions of our work are as follows:

Original: ( a ) We prove that the activation of spherical neurons on rotated input only varies by up to first-degree spherical harmonics (Section 4.1.1).
Revised: ( a ) We prove that the activation of spherical neurons on rotated input only varies by up to first-degree spherical harmonics (Section 4.1.1).

Original: ( b ) Based on a minimal set of four spherical neurons that are rotated to the corresponding vertices of a regular tetrahedron (Section 4.1.2), we derive the main result of our paper (Section 4.1.3) – a 3D steerability constraint for spherical neurons (13).
Revised: ( b ) Based on a minimal set of four spherical neurons that are rotated to the corresponding vertices of a regular tetrahedron (Section 4.1.2), we derive the main result of our paper (Section 4.1.3) – a 3D steerability constraint for spherical neurons (13).

Original: ( c ) We propose a method to turn a trained spherical classifier into a fully steerable model (Section 5.2) that produces predictions equivariant to 3D rotations, even in the presence of noise (Section 5.3), which we verify experimentally using both synthetic and real 3D data.
Revised: ( c ) We propose a method to turn a trained spherical classifier into a fully steerable model (Section 5.2) that produces predictions invariant to 3D rotations, even in the presence of noise (Section 5.3), which we verify experimentally using both synthetic and real 3D data.

Original:  R ELATED WORK 2. S TEERABILITY AND EQUIVARIANCE
Revised:  R ELATED WORK 2. S TEERABILITY AND EQUIVARIANCE

Original: Steerability is a powerful concept from early vision and image processing (Freeman et al., 1991;
Revised: Steerability is a powerful concept from early vision and image processing (Freeman et al., 1991;

Original: Knutsson et al., 1992; Simoncelli et al., 1992; Perona, 1995; Simoncelli & Freeman, 1995; Teo &
Revised: Knutsson et al., 1992; Simoncelli et al., 1992; Perona, 1995; Simoncelli & Freeman, 1995; Teo &

Original: Hel-Or, 1998) that resonates in the era of deep learning.
Revised: Hel-Or, 1998) that resonates in the era of deep learning.

Original: The utility of steerable filters and the main theorems for their construction are presented in the seminal work of Freeman et al.
Revised: The utility of steerable filters and the main theorems for their construction are presented in the seminal work of Freeman et al.

Original: Geometric equivariance in the context of computer vision has been an object of extensive research over the last decades (Van Gool et al., 1995).
Revised: Geometric equivariance in the context of computer vision has been an object of extensive research over the last decades (Van Gool et al., 1995).

Original: Equivariance is a necessary property for steerability as the group acting on the input space needs to be represented in the co-domain of the operator.
Revised: Equivariance is a necessary property for steerability as the group acting on the input space needs to be represented in the co-domain of the operator.

Original: For this reason,
Revised: For this reason,

Original: Lie theory can be used to to study steerable filters for equivariance and invariance (Reisert, 2008).
Revised: Lie theory can be used to to study steerable filters for equivariance and invariance (Reisert, 2008).

Original: Nowadays, equivariance-related research extends into deep learning, e.g., the SE (3) -equivariant models of Fuchs et al.
Revised: Nowadays, equivariance-related research extends into deep learning, e.g., the SE (3) -equivariant models of Fuchs et al.

Original: , Thomas et al.
Revised: , Thomas et al.

Original: (2018), and Zhao et al.
Revised: (2018), and Zhao et al.

Original: (2020), and the SO (3) -equivariant network of Anderson et al.
Revised: (2020), and the SO (3) -equivariant network of Anderson et al.

Original: (2019), as well as Marcos et al.
Revised: (2019), as well as Marcos et al.

Original: (2017) and Kondor et al.
Revised: (2017) and Kondor et al.

Original: Also steerable filter concepts are increasingly used in current works, e.g., Cohen & Welling (2016) considered image data and proposed a CNN architecture to produce equivariant representations with steerable features, which involves fewer parameters than traditional CNNs.
Revised: Also steerable filter concepts are increasingly used in current works, e.g., Cohen & Welling (2016) considered image data and proposed a CNN architecture to produce equivariant representations with steerable features, which involves fewer parameters than traditional CNNs.

Original: They considered the dihedral group ( discrete rotation, translation, and reflection), and the steerable representations in their work are proposed as formation of elementary feature types.
Revised: They considered the dihedral group ( discrete rotation, translation, and reflection), and the steerable representations in their work are proposed as formation of elementary feature types.

Original: One limitation of their approach is that rotations are restricted to four orientations, i.e., by π/ 2 .
Revised: One limitation of their approach is that rotations are restricted to four orientations, i.e., by π/ 2 .

Original: More recently, Weiler et al.
Revised: More recently, Weiler et al.

Original: (2018b) utilized group convolutions and introduced steerable filter convolutional neural networks (SFCNNs) operating on images to jointly attain equivariance under translations and discrete rotations.
Revised: (2018b) utilized group convolutions and introduced steerable filter convolutional neural networks (SFCNNs) operating on images to jointly attain equivariance under translations and discrete rotations.

Original: In their work, the filter banks are learned rather than fixed.
Revised: In their work, the filter banks are learned rather than fixed.

Original: Further, the work of Weiler & Cesa (2019) proposed a unified framework for E(2)-equivariant steerable CNNs and presented their general theory.
Revised: Further, the work of Weiler & Cesa (2019) proposed a unified framework for E(2)-equivariant steerable CNNs and presented their general theory.

Original: The steerable CNNs for 3D data proposed by Weiler et al.
Revised: The steerable CNNs for 3D data proposed by Weiler et al.

Original: (2018a) are closely related to our work.
Revised: (2018a) are closely related to our work.

Original: The authors employed a combination of scalar, vector and tensor fields as features transformed by
Revised: The authors employed a combination of scalar, vector and tensor fields as features transformed by

Original: SO (3) representations and presented a model that is equivariant to SE (3) transformations.
Revised: SO (3) representations and presented a model that is equivariant to SE (3) transformations.

Original: They also considered different types of nonlinearities suitable for nonscalar components of the feature space.
Revised: They also considered different types of nonlinearities suitable for nonscalar components of the feature space.

Original: The novel SE(3)-equivariant approach by Fuchs et al.
Revised: The novel SE(3)-equivariant approach by Fuchs et al.

Original: (2020) introduced a self-attention mechanism that is invariant to global rotations and translations of its input and solves the limitation of angularly constrained filters in other equivariant methods, e.g., Thomas et al.
Revised: (2020) introduced a self-attention mechanism that is invariant to global rotations and translations of its input and solves the limitation of angularly constrained filters in other equivariant methods, e.g., Thomas et al.

Original: Noteworthy, the work of Jing et al.
Revised: Noteworthy, the work of Jing et al.

Original: (2020) proposed the geometric vector perceptron (GVP) consisting of two linear transformations for the input scalar and vector features, followed by nonlinearities.
Revised: (2020) proposed the geometric vector perceptron (GVP) consisting of two linear transformations for the input scalar and vector features, followed by nonlinearities.

Original: The GVP scalar and vector outputs are invariant and equivariant, respectively, with respect to an arbitrary composition of rotations and reflections in the 3D Euclidean space.
Revised: The GVP scalar and vector outputs are invariant and equivariant, respectively, with respect to an arbitrary composition of rotations and reflections in the 3D Euclidean space.

Original: The key point distinguishing our approach from other equivariant networks is that we use conformal modeling (Li et al., 2001a; Hitzer, 2008), with which the scalar product of two points yields their
Revised: The key point distinguishing our approach from other equivariant networks is that we use conformal modeling (Li et al., 2001a; Hitzer, 2008), with which the scalar product of two points yields their

Original: Euclidean distance (allowing us to construct spherical decision surfaces) and nonlinear conformal transformations on R n can be linearized (allowing us to steer the decision spheres).
Revised: Euclidean distance (allowing us to construct spherical decision surfaces) and nonlinear conformal transformations on R n can be linearized (allowing us to steer the decision spheres).

Original: Besides, we do not constrain the space of learnable parameters (as opposed to, e.g., Thomas et al.
Revised: Besides, we do not constrain the space of learnable parameters (as opposed to, e.g., Thomas et al.

Original: , Weiler et al.
Revised: , Weiler et al.

Original: (2018a), and Fuchs et al.
Revised: (2018a), and Fuchs et al.

Original: but construct our steerable model from a freely trained base network, as we discuss in detail in Section 4.
Revised: but construct our steerable model from a freely trained base network, as we discuss in detail in Section 4.

Original: 2. C ONFORMAL MODELING AND THE HYPERSPHERE NEURON
Revised: 2. C ONFORMAL MODELING AND THE HYPERSPHERE NEURON

Original: The utility of conformal embedding for Euclidean geometry and the close connection to Minkowski spaces are thoroughly discussed by Li et al.
Revised: The utility of conformal embedding for Euclidean geometry and the close connection to Minkowski spaces are thoroughly discussed by Li et al.

Original: (2001a).
Revised: (2001a).

Original: An important result is that one can construct hyperspherical decision surfaces using representations in the conformal space (Li et al., 2001b), as done in the work of Perwass et al.
Revised: An important result is that one can construct hyperspherical decision surfaces using representations in the conformal space (Li et al., 2001b), as done in the work of Perwass et al.

Original: The hypersphere neuron proposed by Banarer et al. (2003b) is such a spherical classifier.
Revised: The hypersphere neuron proposed by Banarer et al. (2003b) is such a spherical classifier.

Original: Remarkably, since a hypersphere can be seen as a generalization of a hyperplane, the standard neuron can be considered as a special case of the hypersphere neuron.
Revised: Remarkably, since a hypersphere can be seen as a generalization of a hyperplane, the standard neuron can be considered as a special case of the hypersphere neuron.

Original: Stacking multiple hypersphere neurons in a feed-forward network results in a multilayer hypersphere perceptron (MLHP), which was shown by Banarer et al.
Revised: Stacking multiple hypersphere neurons in a feed-forward network results in a multilayer hypersphere perceptron (MLHP), which was shown by Banarer et al.

Original: (2003a) to outperform the standard MLP for some classification tasks.
Revised: (2003a) to outperform the standard MLP for some classification tasks.

Original: However, its application to point sets was not discussed.
Revised: However, its application to point sets was not discussed.

Original: This motivated the work of Melnyk et al.
Revised: This motivated the work of Melnyk et al.

Original: (2021) on the geometric neuron, where the learned parameters were shown to represent a combination of spherical decision surfaces.
Revised: (2021) on the geometric neuron, where the learned parameters were shown to represent a combination of spherical decision surfaces.

Original: Moreover, the geometric neuron activations were proved to be isometric in the 3D Euclidean space.
Revised: Moreover, the geometric neuron activations were proved to be isometric in the 3D Euclidean space.

Original: In our work, we use the latter observation as the necessary condition for deriving the steerability constraint.
Revised: In our work, we use the latter observation as the necessary condition for deriving the steerability constraint.

Original: Making spherical classifiers steerable adds to the practical value of the prior work by Melnyk et al.
Revised: Making spherical classifiers steerable adds to the practical value of the prior work by Melnyk et al.

Original:  B ACKGROUND 3. S TEERABILITY
Revised:  B ACKGROUND 3. S TEERABILITY

Original: As per Freeman et al.
Revised: As per Freeman et al.

Original: (1991), a 2D function f ( x, y ) is said to steer if it can be written as a linear combination of rotated versions of itself, i.e., when it satisfies the constraint
Revised: (1991), a 2D function f ( x, y ) is said to steer if it can be written as a linear combination of rotated versions of itself, i.e., when it satisfies the constraint

Original: [Equation]
Revised: [Equation]

Original: where v j ( θ ) are the interpolation functions, θ j are the basis function orientations, and M is the number of basis function terms required to steer the function.
Revised: where v j ( θ ) are the interpolation functions, θ j are the basis function orientations, and M is the number of basis function terms required to steer the function.

Original: An alternative formulation can be found in the work of Knutsson et al.
Revised: An alternative formulation can be found in the work of Knutsson et al.

Original: In 3D, the steering equation becomes
Revised: In 3D, the steering equation becomes

Original: [Equation]
Revised: [Equation]

Original: where f R ( x, y, z ) is f ( x, y, z ) rotated by R ∈ SO (3) , and each R j ∈ SO (3) orients the corresponding j th basis function.
Revised: where f R ( x, y, z ) is f ( x, y, z ) rotated by R ∈ SO (3) , and each R j ∈ SO (3) orients the corresponding j th basis function.

Original: Theorems 1, 2 and 4 in Freeman et al.
Revised: Theorems 1, 2 and 4 in Freeman et al.

Original: (1991) describe the conditions under which the steerability constraints (1) and (2) hold, and how to determine the minimum number of basis functions for the 2D and 3D case, respectively.
Revised: (1991) describe the conditions under which the steerability constraints (1) and (2) hold, and how to determine the minimum number of basis functions for the 2D and 3D case, respectively.

Original: 3. C ONFORMAL EMBEDDING
Revised: 3. C ONFORMAL EMBEDDING

Original: We refer the reader to Section 3 in the work of Melnyk et al.
Revised: We refer the reader to Section 3 in the work of Melnyk et al.

Original: (2021) for more details, and only briefly introduce important notation in this section.
Revised: (2021) for more details, and only briefly introduce important notation in this section.

Original: The conformal space for the Euclidean R n counterpart can be formed as ME n ≡ R n +1 , 1 = R n ⊕ R 1 , 1 , where R 1 , 1 is the Minkowski plane (Li et al., 2001a) with orthonormal basis defined as { e + , e − } and null basis { e 0 , e ∞ } representing the origin e 0 = 

Revised: The conformal space for the Euclidean R n counterpart can be formed as ME n ≡ R n +1 , 1 = R n ⊕ R 1 , 1 , where R 1 , 1 is the Minkowski plane (Li et al., 2001a) with orthonormal basis defined as { e + , e − } and null basis { e 0 , e ∞ } representing the origin e 0 =

Original: ( e − − e + ) and point at infinity e ∞ =
Revised: ( e − − e + ) and point at infinity e ∞ =

Original: e − + e + .
Revised: e − + e + .

Original: Thus, a Euclidean vector x ∈ R n can be embedded in the conformal space ME n as
Revised: Thus, a Euclidean vector x ∈ R n can be embedded in the conformal space ME n as

Original: [Equation]
Revised: [Equation]

Original: where X ∈ ME n is called normalized .
Revised: where X ∈ ME n is called normalized .

Original: The conformal embedding (3) represents the stereographic projection of x onto a projection sphere in ME n and is homogeneous , i.e., all embedding vectors in the equivalence class
Revised: The conformal embedding (3) represents the stereographic projection of x onto a projection sphere in ME n and is homogeneous , i.e., all embedding vectors in the equivalence class

Original: [ X ] =
Revised: [ X ] =

Original: [Equation]
Revised: [Equation]

Original: are taken to represent the same vector x .
Revised: are taken to represent the same vector x .

Original: Importantly, given the conformal embedding X and some Y =
Revised: Importantly, given the conformal embedding X and some Y =

Original: y + 1 2 ∥ y ∥ 2 e ∞ + e 0 , their scalar product in the conformal space corresponds to their Euclidean distance, X · Y = − 12 ∥ x − y ∥ 2 .
Revised: y + 1 2 ∥ y ∥ 2 e ∞ + e 0 , their scalar product in the conformal space corresponds to their Euclidean distance, X · Y = − 12 ∥ x − y ∥ 2 .

Original: This interpretation of the scalar product in the conformal space is the main motivation in constructing spherical classifiers.
Revised: This interpretation of the scalar product in the conformal space is the main motivation in constructing spherical classifiers.

Original: 3. S PHERICAL CLASSIFIERS
Revised: 3. S PHERICAL CLASSIFIERS

Original: By spherical classifiers , we collectively refer to hypersphere (Banarer et al., 2003a) and geometric (Melnyk et al., 2021) neurons, which have spherical decision surfaces.
Revised: By spherical classifiers , we collectively refer to hypersphere (Banarer et al., 2003a) and geometric (Melnyk et al., 2021) neurons, which have spherical decision surfaces.

Original: As discussed by Banarer et al. (2003a), by embedding both a data vector x ∈ R n and a hypersphere
Revised: As discussed by Banarer et al. (2003a), by embedding both a data vector x ∈ R n and a hypersphere

Original: S ∈ ME n in R n +2 as
Revised: S ∈ ME n in R n +2 as

Original: [Equation]
Revised: [Equation]

Original: where c = ( c 1 , . . . , c n ) ∈ R n is the hypersphere center and r ∈ R is the radius, their scalar product
Revised: where c = ( c 1 , . . . , c n ) ∈ R n is the hypersphere center and r ∈ R is the radius, their scalar product

Original: X · S in the conformal space ME n can be computed equivalently in R n +2 as X ⊤ S :
Revised: X · S in the conformal space ME n can be computed equivalently in R n +2 as X ⊤ S :

Original: [Equation]
Revised: [Equation]

Original: This result enables the implementation of a hypersphere neuron in ME n using the standard dot product in R n +2 .
Revised: This result enables the implementation of a hypersphere neuron in ME n using the standard dot product in R n +2 .

Original: The hypersphere vector components are treated as independent learnable parameters during training.
Revised: The hypersphere vector components are treated as independent learnable parameters during training.

Original: Thus, a spherical classifier effectively learns non-normalized hyperspheres of the form (cid:101)
Revised: Thus, a spherical classifier effectively learns non-normalized hyperspheres of the form (cid:101)

Original: S =
Revised: S =

Original: ( s 1 , . . .
Revised: ( s 1 , . . .

Original: , s n +2 )
Revised: , s n +2 )

Original: ∈ R n +2 .
Revised: ∈ R n +2 .

Original: Due to the homogeneity of the representation, both normalized and non-normalized hyperspheres represent the same decision surface.
Revised: Due to the homogeneity of the representation, both normalized and non-normalized hyperspheres represent the same decision surface.

Original: More details can be found in
Revised: More details can be found in

Original: Section 3.2 in the work of Melnyk et al.
Revised: Section 3.2 in the work of Melnyk et al.

Original: [Equation]
Revised: [Equation]

Original: The geometric neuron is a generalization of the hypersphere neuron for points sets as input, see
Revised: The geometric neuron is a generalization of the hypersphere neuron for points sets as input, see

Original: Figure 1.
Revised: Figure 1.

Original: A is thus the sum of the signed distances of k input points to k learned hyperspheres
Revised: A is thus the sum of the signed distances of k input points to k learned hyperspheres

Original: [Equation]
Revised: [Equation]

Original: where z ∈ R , X k ∈ R 5 is a properly embedded 3D input point, γ k ∈ R is the scale factor, i.e., the last element of the learned parameter vector (cid:101)
Revised: where z ∈ R , X k ∈ R 5 is a properly embedded 3D input point, γ k ∈ R is the scale factor, i.e., the last element of the learned parameter vector (cid:101)

Original: S k , and S k
Revised: S k , and S k

Original: = (cid:101)
Revised: = (cid:101)

Original: S k /γ k ∈ R 5 are the corresponding normalized learned parameters (spheres).
Revised: S k /γ k ∈ R 5 are the corresponding normalized learned parameters (spheres).

Original: Furthermore, Melnyk et al.
Revised: Furthermore, Melnyk et al.

Original: (2021) demonstrated that the geometric neuron activations are isometric in 3D. That is, rotating the input is equivalent to rotating the decision spheres.
Revised: (2021) demonstrated that the geometric neuron activations are isometric in 3D. That is, rotating the input is equivalent to rotating the decision spheres.

Original: This result is a necessary condition to consider rotation and translation equivariance of models constructed with geometric neurons and forms the basis for our methodology.
Revised: This result is a necessary condition to consider rotation and translation equivariance of models constructed with geometric neurons and forms the basis for our methodology.

Original: In the following sections, we use the same notation for a 3D rotation R represented in the Euclidean space R 3 , the homogeneous (projective) space P ( R 3 ) , and ME 3 ∼ = R 5 , depending on the context.
Revised: In the following sections, we use the same notation for a 3D rotation R represented in the Euclidean space R 3 , the homogeneous (projective) space P ( R 3 ) , and ME 3 ∼ = R 5 , depending on the context.

Original: This is possible since we can add the required number of ones to the diagonal of the original rotation matrix without changing the transformation representation.
Revised: This is possible since we can add the required number of ones to the diagonal of the original rotation matrix without changing the transformation representation.

Original:  M ETHOD
Revised:  M ETHOD

Original: To build a 3D steerable model, we perform the following steps: We first train an MLGP model (Melnyk et al., 2021), which consists of spherical neurons.
Revised: To build a 3D steerable model, we perform the following steps: We first train an MLGP model (Melnyk et al., 2021), which consists of spherical neurons.

Original: After optimizing the model parameters, we freeze them and transform according to the 3D steerability constraint we derive.
Revised: After optimizing the model parameters, we freeze them and transform according to the 3D steerability constraint we derive.

Original: Finally, by combining the resulting parameters in filter banks and adding interpolation coefficients as optimizable parameters, we create a steerable model.
Revised: Finally, by combining the resulting parameters in filter banks and adding interpolation coefficients as optimizable parameters, we create a steerable model.

Original: 4. F ULLY STEERABLE 3D SPHERICAL NEURONS
Revised: 4. F ULLY STEERABLE 3D SPHERICAL NEURONS

Original: In this section, we identify the conditions under which a geometric neuron as a function of its 3D input can be steered.
Revised: In this section, we identify the conditions under which a geometric neuron as a function of its 3D input can be steered.

Original: In other words, we derive an expression that gives us the response of a hypothetical geometric neuron for some input, using rotated versions of the learned geometric neuron parameters.
Revised: In other words, we derive an expression that gives us the response of a hypothetical geometric neuron for some input, using rotated versions of the learned geometric neuron parameters.

Original: We start by considering the steerability conditions for a single sphere classifying the corresponding input point X , i.e., f ( X ) =
Revised: We start by considering the steerability conditions for a single sphere classifying the corresponding input point X , i.e., f ( X ) =

Original: X ⊤ S , where X and S are embedded in ME 3 ∼ = R 5 according to (4).
Revised: X ⊤ S , where X and S are embedded in ME 3 ∼ = R 5 according to (4).

Original: Since the geometric neuron output is a linear combination of these functions, as per (6), we will use the identified conditions to build a steerable feed-forward network by reassembling modified spheres.
Revised: Since the geometric neuron output is a linear combination of these functions, as per (6), we will use the identified conditions to build a steerable feed-forward network by reassembling modified spheres.

Original: 4.1. B ASIS CONSTRUCTION
Revised: 4.1. B ASIS CONSTRUCTION

Original: To formulate a steerability constraint for a spherical neuron (sphere), first, we need to determine the minimum number of basis functions, i.e., the number of terms M in (2).
Revised: To formulate a steerability constraint for a spherical neuron (sphere), first, we need to determine the minimum number of basis functions, i.e., the number of terms M in (2).

Original: This number only depends on the degree of the spherical harmonics that are required to compute the steered result (Freeman et al., 1991).
Revised: This number only depends on the degree of the spherical harmonics that are required to compute the steered result (Freeman et al., 1991).

Original: Thus, we need to determine the required degrees.
Revised: Thus, we need to determine the required degrees.

Original: Theorem.
Revised: Theorem.

Original: Let S ∈ R n +2 be an nD classifier with center c ∈ R n ( R :=
Revised: Let S ∈ R n +2 be an nD classifier with center c ∈ R n ( R :=

Original: ∥ c ∥ ) and radius r , and x ∈ R n be a point represented by X ∈ R n +2 , see (4) .
Revised: ∥ c ∥ ) and radius r , and x ∈ R n be a point represented by X ∈ R n +2 , see (4) .

Original: Let further S ′ be the classifier that is obtained by rotating S in nD space, then X ⊤ S ′ and X ⊤ S are related by spherical harmonics up to first degree.
Revised: Let further S ′ be the classifier that is obtained by rotating S in nD space, then X ⊤ S ′ and X ⊤ S are related by spherical harmonics up to first degree.

Original: [Figure]
Revised: [Figure]

Original: Proof.
Revised: Proof.

Original: Without loss of generality, the rotation is defined by the plane of rotation πππ by the angle θ .
Revised: Without loss of generality, the rotation is defined by the plane of rotation πππ by the angle θ .

Original: Denote the projection of a vector v ∈ R n onto πππ by v πππ and define v ⊥ πππ = v − v πππ .
Revised: Denote the projection of a vector v ∈ R n onto πππ by v πππ and define v ⊥ πππ = v − v πππ .

Original: From (5) we obtain
Revised: From (5) we obtain

Original: [Equation]
Revised: [Equation]

Original: A rotation in πππ only affects the rightmost term above and there exists a ϕ ∈ [0 , 2 π ) such that
Revised: A rotation in πππ only affects the rightmost term above and there exists a ϕ ∈ [0 , 2 π ) such that

Original: [Equation]
Revised: [Equation]

Original: With a similar argument
Revised: With a similar argument

Original: [Equation]
Revised: [Equation]

Original: we obtain
Revised: we obtain

Original: [Equation]
Revised: [Equation]

Original: This result is valid in any dimension, but we are primarily interested in n = 2 and n = 3 , as illustrated in Figure 2.
Revised: This result is valid in any dimension, but we are primarily interested in n = 2 and n = 3 , as illustrated in Figure 2.

Original: Following the result of Theorem 4 in Freeman et al.
Revised: Following the result of Theorem 4 in Freeman et al.

Original: and using N = 1 , we have that M =
Revised: and using N = 1 , we have that M =

Original: ( N + 1) 2 = 4 basis functions suffice in the 3D case (2).
Revised: ( N + 1) 2 = 4 basis functions suffice in the 3D case (2).

Original: 4.1. S PHERICAL FILTER BANKS IN 3D
Revised: 4.1. S PHERICAL FILTER BANKS IN 3D

Original: In 3D, we thus select four rotated versions of the function f ( X ) , as the basis functions.
Revised: In 3D, we thus select four rotated versions of the function f ( X ) , as the basis functions.

Original: The rotations { R j } 4 j =1 must be chosen to satisfy the condition (b) in Theorem 4 (Freeman et al., 1991).
Revised: The rotations { R j } 4 j =1 must be chosen to satisfy the condition (b) in Theorem 4 (Freeman et al., 1991).

Original: Therefore, we transform f ( X ) such that the resulting four spheres are spaced in three dimensions equally, i.e., form a regular tetrahedron with the vertices (1 , 1 , 1) , (1 , − 1 , − 1) , ( − 1 , 1 , − 1) , and ( − 1 , − 1 , 1) , as shown in Figure 3 b).
Revised: Therefore, we transform f ( X ) such that the resulting four spheres are spaced in three dimensions equally, i.e., form a regular tetrahedron with the vertices (1 , 1 , 1) , (1 , − 1 , − 1) , ( − 1 , 1 , − 1) , and ( − 1 , − 1 , 1) , as shown in Figure 3 b).

Original: We stack the homogeneous coordinates of the tetrahedron vertices m j in a matrix column-wise (scaled by 1 / 2 ) to get the orthogonal matrix
Revised: We stack the homogeneous coordinates of the tetrahedron vertices m j in a matrix column-wise (scaled by 1 / 2 ) to get the orthogonal matrix

Original: [Equation]
Revised: [Equation]

Original: We will use this matrix operator M to compute the linear coefficients in the vector space generated by the vertices of the regular tetrahedron (Granlund & Knutsson, 1995).
Revised: We will use this matrix operator M to compute the linear coefficients in the vector space generated by the vertices of the regular tetrahedron (Granlund & Knutsson, 1995).

Original: This will be necessary to find the appropriate interpolation functions and formulate the steerability constraint in Section 4.1.3.
Revised: This will be necessary to find the appropriate interpolation functions and formulate the steerability constraint in Section 4.1.3.

Original: The four rotated versions of the function f ( X ) will constitute the basis functions that we call a spherical filter bank.
Revised: The four rotated versions of the function f ( X ) will constitute the basis functions that we call a spherical filter bank.

Original: To construct this filter bank, we choose the following convention.
Revised: To construct this filter bank, we choose the following convention.

Original: The originally learned spherical classifier f
Revised: The originally learned spherical classifier f

Original: ( X ) =
Revised: ( X ) =

Original: X ⊤ S is first rotated to (1 , 1 , 1) (see Figure 3 b)) with the corresponding transformation denoted as R O .
Revised: X ⊤ S is first rotated to (1 , 1 , 1) (see Figure 3 b)) with the corresponding transformation denoted as R O .

Original: Next, we rotate the transformed sphere into the other three vertices of the regular tetrahedron and transform back to the original coordinate system (see
Revised: Next, we rotate the transformed sphere into the other three vertices of the regular tetrahedron and transform back to the original coordinate system (see

Original: Figure 3 a) for the case of (1 , − 1 , − 1) ).
Revised: Figure 3 a) for the case of (1 , − 1 , − 1) ).

Original: The resulting filter bank for one spherical classifier is thus composed as the following 20 × 1 matrix:
Revised: The resulting filter bank for one spherical classifier is thus composed as the following 20 × 1 matrix:

Original: [Equation]
Revised: [Equation]

Original: where each of { R Ti } 3 i =0 is the rotation isomorphism in R 5 corresponding to a 3D rotation from (1 , 1 , 1) to the vertex i + 1 of the regular tetrahedron.
Revised: where each of { R Ti } 3 i =0 is the rotation isomorphism in R 5 corresponding to a 3D rotation from (1 , 1 , 1) to the vertex i + 1 of the regular tetrahedron.

Original: Therefore, R T 

Revised: Therefore, R T

Original: = I 5 .
Revised: = I 5 .

Original: [Figure]
Revised: [Figure]

Original: • • c (1 , − 1 , − 1) (1 , 1 , 1) c
Revised: • • c (1 , − 1 , − 1) (1 , 1 , 1) c

Original: R
Revised: R

Original: R O
Revised: R O

Original: R T
Revised: R T

Original: R > O (1 , 1 , 1) (1 , − 1 , − 1) ( − 1 , 1 , − 1) ( − 1 , − 1 , 1) c
Revised: R > O (1 , 1 , 1) (1 , − 1 , − 1) ( − 1 , 1 , − 1) ( − 1 , − 1 , 1) c

Original: 4.1. 3D STEERABILITY CONSTRAINT
Revised: 4.1. 3D STEERABILITY CONSTRAINT

Original: The steerability constraint can be formulated as follows.
Revised: The steerability constraint can be formulated as follows.

Original: For an arbitrary orientation R applied to the input of the function f ( X ) , we want the output of the spherical filter bank B ( S ) in (8) to be interpolated with v j ( R ) such that the response is equal to the original function output, i.e.,
Revised: For an arbitrary orientation R applied to the input of the function f ( X ) , we want the output of the spherical filter bank B ( S ) in (8) to be interpolated with v j ( R ) such that the response is equal to the original function output, i.e.,

Original: [Equation]
Revised: [Equation]

Original: where X ∈ R 5 is a single, appropriately embedded, 3D point, v ( R ) ∈ R 4 is a vector of the interpolation coefficients, and ⊗ denotes the Kronecker product (Horn & Johnson, 1991).
Revised: where X ∈ R 5 is a single, appropriately embedded, 3D point, v ( R ) ∈ R 4 is a vector of the interpolation coefficients, and ⊗ denotes the Kronecker product (Horn & Johnson, 1991).

Original: The coefficients v ( R ) should conform to the basis function construction (condition (b) in Theoremin Freeman et al.
Revised: The coefficients v ( R ) should conform to the basis function construction (condition (b) in Theoremin Freeman et al.

Original: (1991)), which is why they are computed with M defined in (7).
Revised: (1991)), which is why they are computed with M defined in (7).

Original: Given X ∈ R 5 as input and an unknown rotation R acting on it, the steering equation (9) implies
Revised: Given X ∈ R 5 as input and an unknown rotation R acting on it, the steering equation (9) implies

Original: [Equation]
Revised: [Equation]

Original: Given a tetrahedron rotation, e.g., R T 1 , as shown in the diagram in Figure 3 a), we can define the unknown rotation accordingly as R = R
Revised: Given a tetrahedron rotation, e.g., R T 1 , as shown in the diagram in Figure 3 a), we can define the unknown rotation accordingly as R = R

Original: ⊤ O R T 1 R O .
Revised: ⊤ O R T 1 R O .

Original: In this case, it is easy to see that to satisfy the constraint (10), v ( R ) must be (0 , 1 , 0 , 0) , i.e., the second filter in the filter bank B ( S ) must be chosen.
Revised: In this case, it is easy to see that to satisfy the constraint (10), v ( R ) must be (0 , 1 , 0 , 0) , i.e., the second filter in the filter bank B ( S ) must be chosen.

Original: This can be achieved by transforming a constant vector m 1 = ( 12 , 12 , 12 , 12 ) by rotation R T 1 and multiplying it by the basis matrix M as follows:
Revised: This can be achieved by transforming a constant vector m 1 = ( 12 , 12 , 12 , 12 ) by rotation R T 1 and multiplying it by the basis matrix M as follows:

Original: [Equation]
Revised: [Equation]

Original: Note that, in general, a geometric neuron (6) takes a set of embedded points as input.
Revised: Note that, in general, a geometric neuron (6) takes a set of embedded points as input.

Original: Therefore, with the setup above, R will be different for each input shape point k if the same v is used for all k , which contradicts that the shape is transformed by a rigid body motion, i.e., the same R B for all k .
Revised: Therefore, with the setup above, R will be different for each input shape point k if the same v is used for all k , which contradicts that the shape is transformed by a rigid body motion, i.e., the same R B for all k .

Original: Thus, we need to consider a suitable vector v k for each input shape point k , such that the resulting R B is the same for all k .
Revised: Thus, we need to consider a suitable vector v k for each input shape point k , such that the resulting R B is the same for all k .

Original: This can be achieved recalling how we construct the basis functions in the spherical filter bank (8): we need to consider the respective initial rotation R kO .
Revised: This can be achieved recalling how we construct the basis functions in the spherical filter bank (8): we need to consider the respective initial rotation R kO .

Original: The desired interpolation coefficients v k are thus computed as
Revised: The desired interpolation coefficients v k are thus computed as

Original: [Equation]
Revised: [Equation]

Original: The resulting v k
Revised: The resulting v k

Original: ( R B ) interpolate the responses of the tetrahedron-copies of the originally learned sphere S k to replace the rotated sphere.
Revised: ( R B ) interpolate the responses of the tetrahedron-copies of the originally learned sphere S k to replace the rotated sphere.

Original: By plugging (9) into (6), we can now establish the main result of our paper – the steerability constraint for a geometric neuron that takes a set of N embedded points { X k } k as input:
Revised: By plugging (9) into (6), we can now establish the main result of our paper – the steerability constraint for a geometric neuron that takes a set of N embedded points { X k } k as input:

Original: [Equation]
Revised: [Equation]

Original:  E XPERIMENTS
Revised:  E XPERIMENTS

Original: We use a single GTX 1050 Ti GPU and conduct two types of experiments to confirm our findings presented in Section 4.
Revised: We use a single GTX 1050 Ti GPU and conduct two types of experiments to confirm our findings presented in Section 4.

Original: [Figure]
Revised: [Figure]

Original: 5. D ATASETS
Revised: 5. D ATASETS

Original: 3D Tetris
Revised: 3D Tetris

Original: Following the experiments reported by Thomas et al.
Revised: Following the experiments reported by Thomas et al.

Original: , Weiler et al.
Revised: , Weiler et al.

Original: (2018a), and
Revised: (2018a), and

Original: Melnyk et al.
Revised: Melnyk et al.

Original: (2021), we use the synthetic point set of eight 3D Tetris shapes consisting of four points each (see, e.g., Figure 3 in Melnyk et al.
Revised: (2021), we use the synthetic point set of eight 3D Tetris shapes consisting of four points each (see, e.g., Figure 3 in Melnyk et al.

Original: (2021) and the numerical description in Appendix A).
Revised: (2021) and the numerical description in Appendix A).

Original: 3D skeleton data We also perform experiments on real-world data to substantiate the validity of our theoretical results.
Revised: 3D skeleton data We also perform experiments on real-world data to substantiate the validity of our theoretical results.

Original: We use the UTKinect-
Revised: We use the UTKinect-

Original: Action3D dataset introduced by Xia et al.
Revised: Action3D dataset introduced by Xia et al.

Original: (2012), in particular, the 3D skeletal joint locations extracted from Kinect depth maps.
Revised: (2012), in particular, the 3D skeletal joint locations extracted from Kinect depth maps.

Original: For each action sequence and from each frame, we extract the skeleton consisting of twenty points and assign it to the class of actions (ten categories) this frame is part of.
Revised: For each action sequence and from each frame, we extract the skeleton consisting of twenty points and assign it to the class of actions (ten categories) this frame is part of.

Original: Therefore, we formulate the task as shape recognition (i.e., a kind of action recognition from a static point cloud), where each shape is of size 20 × 3 .
Revised: Therefore, we formulate the task as shape recognition (i.e., a kind of action recognition from a static point cloud), where each shape is of size 20 × 3 .

Original: Since the orientations of the shapes vary significantly across the sequences, we perform the following standardization: We first center each shape at the origin, and then compute the orientation from its three hip joints and derotate the shape in the xy -plane (viewer coordinate system) accordingly.
Revised: Since the orientations of the shapes vary significantly across the sequences, we perform the following standardization: We first center each shape at the origin, and then compute the orientation from its three hip joints and derotate the shape in the xy -plane (viewer coordinate system) accordingly.

Original: We illustrate the effect of derotation in Figure 4.
Revised: We illustrate the effect of derotation in Figure 4.

Original: From each action sequence, we randomly select 50% of the skeletons for the test set and 20% of the remainder as validation data.
Revised: From each action sequence, we randomly select 50% of the skeletons for the test set and 20% of the remainder as validation data.

Original: The resulting data split is as follows: shapes, shapes validation, and test shapes.
Revised: The resulting data split is as follows: shapes, shapes validation, and test shapes.

Original: 5. S TEERABLE MODEL CONSTRUCTION
Revised: 5. S TEERABLE MODEL CONSTRUCTION

Original: To construct and test steerable models, we perform the same steps for both datasets.
Revised: To construct and test steerable models, we perform the same steps for both datasets.

Original: The minor differences are the choice of training hyperparameters and the presence of validation and test subsets in the 3D skeleton dataset.
Revised: The minor differences are the choice of training hyperparameters and the presence of validation and test subsets in the 3D skeleton dataset.

Original: We first train a two-layer MLGP model
Revised: We first train a two-layer MLGP model

Original: (Melnyk et al., 2021), where the hidden layer consists of geometric neurons and the output layer of hypersphere neurons, to classify the shapes.
Revised: (Melnyk et al., 2021), where the hidden layer consists of geometric neurons and the output layer of hypersphere neurons, to classify the shapes.

Original: Since the architecture choice is not the objective of the experiments, when building the MLGP, we use only one configuration with five hidden units for the Tetris data and twelve hidden units (determined by using the validation data) for the 3D skeleton dataset throughout the experiments.
Revised: Since the architecture choice is not the objective of the experiments, when building the MLGP, we use only one configuration with five hidden units for the Tetris data and twelve hidden units (determined by using the validation data) for the 3D skeleton dataset throughout the experiments.

Original: Similar to Melnyk et al.
Revised: Similar to Melnyk et al.

Original: (2021), we do not use any activation function in the hidden layer due to the nonlinearity of the conformal embedding.
Revised: (2021), we do not use any activation function in the hidden layer due to the nonlinearity of the conformal embedding.

Original: We implement both MLGP models in PyTorch (Paszke et al., 2019) and keep the default parameter initialization for the linear layers.
Revised: We implement both MLGP models in PyTorch (Paszke et al., 2019) and keep the default parameter initialization for the linear layers.

Original: We train both models by minimizing the cross-entropy loss function and use the Adam optimizer (Kingma & Ba, 2015) with the default hyperparameters (the learning rate is 0 . 001 ).
Revised: We train both models by minimizing the cross-entropy loss function and use the Adam optimizer (Kingma & Ba, 2015) with the default hyperparameters (the learning rate is 0 . 001 ).

Original: The Tetris MLGP learns to classify the eight shapes in the canonical orientation perfectly after 2000 epochs, whereas the Skeleton model trained forepochs achieves a test set accuracy of 92.9%.
Revised: The Tetris MLGP learns to classify the eight shapes in the canonical orientation perfectly after 2000 epochs, whereas the Skeleton model trained forepochs achieves a test set accuracy of 92.9%.

Original: We refer to both trained models as ancestor MLGP.
Revised: We refer to both trained models as ancestor MLGP.

Original: For both, we then freeze the trained parameters and construct a steerable model.
Revised: For both, we then freeze the trained parameters and construct a steerable model.

Original: Note that we form steerable units only in the hidden layer and keep the output, i.e., classification, layer hypersphere neurons as they are.
Revised: Note that we form steerable units only in the hidden layer and keep the output, i.e., classification, layer hypersphere neurons as they are.

Original: The steerability is not required for the subsequent layers as the output of the first layer becomes rotation-independent.
Revised: The steerability is not required for the subsequent layers as the output of the first layer becomes rotation-independent.

Original: The steerable units are formed from the corresponding frozen parameters as the (fixed) filter banks according to (8).
Revised: The steerable units are formed from the corresponding frozen parameters as the (fixed) filter banks according to (8).

Original: [Table]
Revised: [Table]

Original: The only optimizable parameters of this constructed steerable model are interpolation coefficients v k ( R B ) ∈ R 4 defined in (12), where k indexes the learned hidden layer parameters (spheres) in the ancestor MLGP model.
Revised: The only optimizable parameters of this constructed steerable model are interpolation coefficients v k ( R B ) ∈ R 4 defined in (12), where k indexes the learned hidden layer parameters (spheres) in the ancestor MLGP model.

Original: But since the only variables in (12) are the parameters of the unknown rotation R B acting on the input, we can directly optimize them if they are unknown.
Revised: But since the only variables in (12) are the parameters of the unknown rotation R B acting on the input, we can directly optimize them if they are unknown.

Original: We select the axis–angle rotation representation because it has only three parameters — the three coordinates of the rotation axis scaled by the rotation angle.
Revised: We select the axis–angle rotation representation because it has only three parameters — the three coordinates of the rotation axis scaled by the rotation angle.

Original: We propose the following basic recipe to create a steerable model: Train the ancestor MLGP →
Revised: We propose the following basic recipe to create a steerable model: Train the ancestor MLGP →

Original: Fix the learned parameters → Transform the hidden unit parameters into filter banks (8) → Add the interpolation coefficients v k as optimizable parameters to fulfill (13) → Steerable spherical classifier.
Revised: Fix the learned parameters → Transform the hidden unit parameters into filter banks (8) → Add the interpolation coefficients v k as optimizable parameters to fulfill (13) → Steerable spherical classifier.

Original: 5. K NOWN ROTATION EXPERIMENT
Revised: 5. K NOWN ROTATION EXPERIMENT

Original: Using the trained MLGP, we verify the correctness of (13).
Revised: Using the trained MLGP, we verify the correctness of (13).

Original: We first rotate the original data and then use this ground truth rotation to initialize the interpolation coefficients of the constructed steerable model according to (12).
Revised: We first rotate the original data and then use this ground truth rotation to initialize the interpolation coefficients of the constructed steerable model according to (12).

Original: Our intuition is that if the steerability constraint (13) is correct, then, given the transformed point set, the activations of the steerable units in the steerable model will be equal to the activations of the geometric neurons in the ancestor MLGP model fed with the point set in the canonical orientation.
Revised: Our intuition is that if the steerability constraint (13) is correct, then, given the transformed point set, the activations of the steerable units in the steerable model will be equal to the activations of the geometric neurons in the ancestor MLGP model fed with the point set in the canonical orientation.

Original: Hence, the classification accuracies of the ancestor and steerable models on the original and transformed datasets, respectively, should be equal.
Revised: Hence, the classification accuracies of the ancestor and steerable models on the original and transformed datasets, respectively, should be equal.

Original: We run this experiment 1000 times.
Revised: We run this experiment 1000 times.

Original: Each time, we generate a random rotation and apply it to the original point set (in case of the 3D skeleton data, we use the test split).
Revised: Each time, we generate a random rotation and apply it to the original point set (in case of the 3D skeleton data, we use the test split).

Original: We use these rotation parameters to create a steerable model and evaluate it on the transformed point set.
Revised: We use these rotation parameters to create a steerable model and evaluate it on the transformed point set.

Original: To verify the stability of the steerable unit activations, we add uniform noise to the transformed points, n ∼ U ( − a, a ) , where the range of a is motivated by the magnitude of the points in the datasets.
Revised: To verify the stability of the steerable unit activations, we add uniform noise to the transformed points, n ∼ U ( − a, a ) , where the range of a is motivated by the magnitude of the points in the datasets.

Original: For reference, we also present the accuracy of the ancestor model classifying the rotated data.
Revised: For reference, we also present the accuracy of the ancestor model classifying the rotated data.

Original: We summarize the results in Table 1.
Revised: We summarize the results in Table 1.

Original: The respective L1 distance results are presented in
Revised: The respective L1 distance results are presented in

Original: Appendix B.
Revised: Appendix B.

Original: The analogous Tetris experiment results reported in related work are 100% (over 25 runs and including random translation) (Thomas et al., 2018) and 99 ± 2% (over 17 runs) (Weiler et al., 2018a).
Revised: The analogous Tetris experiment results reported in related work are 100% (over 25 runs and including random translation) (Thomas et al., 2018) and 99 ± 2% (over 17 runs) (Weiler et al., 2018a).

Original: However, due to the significant conceptual difference between our approach and these methods, as mentioned at the end of Section 2.1, the results are not directly comparable.
Revised: However, due to the significant conceptual difference between our approach and these methods, as mentioned at the end of Section 2.1, the results are not directly comparable.

Original: 5. E STIMATED ROTATION EXPERIMENT
Revised: 5. E STIMATED ROTATION EXPERIMENT

Original: In this experiment, we feed some rotation information into the system.
Revised: In this experiment, we feed some rotation information into the system.

Original: For example, this information can come from another network branch that performs regression on the rotation, or from a dynamical model as in tracking.
Revised: For example, this information can come from another network branch that performs regression on the rotation, or from a dynamical model as in tracking.

Original: In either case, we cannot assume that the rotation is entirely accurate, unlike the experiment in Section 5.3.
Revised: In either case, we cannot assume that the rotation is entirely accurate, unlike the experiment in Section 5.3.

Original: Therefore, we check the stability with respect to the noise in the rotation.
Revised: Therefore, we check the stability with respect to the noise in the rotation.

Original: Same as before, we exploit the MLGP trained on the respective point set in the canonical orientation, as described in Section 5.2, to build the steerable model.
Revised: Same as before, we exploit the MLGP trained on the respective point set in the canonical orientation, as described in Section 5.2, to build the steerable model.

Original: At each experiment run, we randomly select one shape (from the test subset in case of the 3D skeleton data) and apply a random rotation, R GT , to it.
Revised: At each experiment run, we randomly select one shape (from the test subset in case of the 3D skeleton data) and apply a random rotation, R GT , to it.

Original: We then perturb this ground truth rotation by varying the rotation angle.
Revised: We then perturb this ground truth rotation by varying the rotation angle.

Original: We achieve this by sampling the rotation angle from a Gaussian distribution with σ = π/ 18 = 10 ° and various means, constructing a rotation of this sampled angle about a random axis, R noise , and multiplying with the ground truth: R init =
Revised: We achieve this by sampling the rotation angle from a Gaussian distribution with σ = π/ 18 = 10 ° and various means, constructing a rotation of this sampled angle about a random axis, R noise , and multiplying with the ground truth: R init =

Original: R noise R GT .
Revised: R noise R GT .

Original: We use the resulting rotation R init in the axis-angle representation to initialize the steerable model parameters.
Revised: We use the resulting rotation R init in the axis-angle representation to initialize the steerable model parameters.

Original: [Table]
Revised: [Table]

Original: Further, we perform the optimization of the rotation vector parameters, which we call online optimization.
Revised: Further, we perform the optimization of the rotation vector parameters, which we call online optimization.

Original: Since inference time, we do not have access to the labels.
Revised: Since inference time, we do not have access to the labels.

Original: Thus, we need to select a loss based on the consistency of the output.
Revised: Thus, we need to select a loss based on the consistency of the output.

Original: Here, we choose the entropy, i.e., L ( p ) = −
Revised: Here, we choose the entropy, i.e., L ( p ) = −

Original: [Equation]
Revised: [Equation]

Original: where p ∈ R K is the softmax output of the model.
Revised: where p ∈ R K is the softmax output of the model.

Original: Our intuition is that by minimizing the entropy in the output with respect to the steerable model parameters, we effectively force the model to produce a more confident prediction.
Revised: Our intuition is that by minimizing the entropy in the output with respect to the steerable model parameters, we effectively force the model to produce a more confident prediction.

Original: For this optimization procedure, we use Adam with learning rate equal to 0.01, and empirically determine the number of epochs of 300 to be sufficient for the Tetris data and 100 for the 3D skeleton data.
Revised: For this optimization procedure, we use Adam with learning rate equal to 0.01, and empirically determine the number of epochs of 300 to be sufficient for the Tetris data and 100 for the 3D skeleton data.

Original: Note that at each run, we provide the model with only one transformed shape, which implies that the accuracy per run is binary, i.e., the shape is either recognized ("1") or misclassified ("0").
Revised: Note that at each run, we provide the model with only one transformed shape, which implies that the accuracy per run is binary, i.e., the shape is either recognized ("1") or misclassified ("0").

Original: To verify the usefulness of the proposed online optimization method, we compare the model classification accuracy before and after optimization.
Revised: To verify the usefulness of the proposed online optimization method, we compare the model classification accuracy before and after optimization.

Original: The results for this experiment are summarized in Table 2.
Revised: The results for this experiment are summarized in Table 2.

Original: The respective L1 distance results are presented in Appendix B.
Revised: The respective L1 distance results are presented in Appendix B.

Original:  D ISCUSSION AND C ONCLUSION
Revised:  D ISCUSSION AND C ONCLUSION

Original: Enabled by the complete understanding of the geometry of the spherical classifiers (Banarer et al., 2003b; Perwass et al., 2003; Melnyk et al., 2021), we show in Section 4 that we only need the spherical harmonics of degree up to N = 1 to determine the effect of rotation on the activations in 3D. Using this result, we derive a novel 3D steerability constraint (13).
Revised: Enabled by the complete understanding of the geometry of the spherical classifiers (Banarer et al., 2003b; Perwass et al., 2003; Melnyk et al., 2021), we show in Section 4 that we only need the spherical harmonics of degree up to N = 1 to determine the effect of rotation on the activations in 3D. Using this result, we derive a novel 3D steerability constraint (13).

Original: The conducted experiment in
Revised: The conducted experiment in

Original: Section 5.3 shows that the derived constraint is correct since the constructed steerable model produces accurate predictions for the rotated shapes, provided that the rotation is known.
Revised: Section 5.3 shows that the derived constraint is correct since the constructed steerable model produces accurate predictions for the rotated shapes, provided that the rotation is known.

Original: From Table 1, we can see that that rotating the data without steering the neurons leads to substantial degradation of accuracy (from 99.2% to 25.2%), whereas using the steerability equation maintains the high accuracy.
Revised: From Table 1, we can see that that rotating the data without steering the neurons leads to substantial degradation of accuracy (from 99.2% to 25.2%), whereas using the steerability equation maintains the high accuracy.

Original: Moreover, the steerable model classification error only moderately increases with the level of noise in the input data, which is a clear indication of the robustness of the classifier.
Revised: Moreover, the steerable model classification error only moderately increases with the level of noise in the input data, which is a clear indication of the robustness of the classifier.

Original: The experiment presented in Section 5.4 considers a more realistic setting, where the rotation applied to the input is not known exactly but is inaccurately estimated.
Revised: The experiment presented in Section 5.4 considers a more realistic setting, where the rotation applied to the input is not known exactly but is inaccurately estimated.

Original: However, unlike the first experiment, we perform an optimization to improve the initial steerable model prediction.
Revised: However, unlike the first experiment, we perform an optimization to improve the initial steerable model prediction.

Original: The only three parameters of the model — the axis–angle representation of the unknown external rotation — determining the interpolation coefficients v k , are optimized by minimizing the entropy in the model output.
Revised: The only three parameters of the model — the axis–angle representation of the unknown external rotation — determining the interpolation coefficients v k , are optimized by minimizing the entropy in the model output.

Original: As displayed in Table 2, our steerable model, prior to the optimization, produces rotation-invariant predictions for the transformed shapes for both synthetic and real data, which points to the robustness of our method.
Revised: As displayed in Table 2, our steerable model, prior to the optimization, produces rotation-invariant predictions for the transformed shapes for both synthetic and real data, which points to the robustness of our method.

Original: The proposed online optimization further improves the model classification accuracy.
Revised: The proposed online optimization further improves the model classification accuracy.

Original: The lower accuracies for the zero-mean noise (the first row in Table 2) for the skeleton data compared to the π/ 36 case (the second row) are presumably caused by a small error in the shape orientations after the performed standardization.
Revised: The lower accuracies for the zero-mean noise (the first row in Table 2) for the skeleton data compared to the π/ 36 case (the second row) are presumably caused by a small error in the shape orientations after the performed standardization.

Original: The power of our approach lies in the geometric explainability inherent to the spherical units that constitute our model.
Revised: The power of our approach lies in the geometric explainability inherent to the spherical units that constitute our model.

Original: Seemingly complicated conformal algebra operations have a surprisingly straightforward interpretation in Euclidean space and allow us to build a steerable feed-forward neural network with a purely geometric motivation.
Revised: Seemingly complicated conformal algebra operations have a surprisingly straightforward interpretation in Euclidean space and allow us to build a steerable feed-forward neural network with a purely geometric motivation.

Original: The steerable spherical neurons discussed in our work can be employed in other architectures, e.g., CNNs, and therefore, the equivariance to transformations beyond rotation can be considered.
Revised: The steerable spherical neurons discussed in our work can be employed in other architectures, e.g., CNNs, and therefore, the equivariance to transformations beyond rotation can be considered.

Original: E THICS S TATEMENT
Revised: E THICS S TATEMENT

Original: As the nature of the present paper is mainly theoretical, no direct ethical issues arise, nor are there any immediate societal consequences.
Revised: As the nature of the present paper is mainly theoretical, no direct ethical issues arise, nor are there any immediate societal consequences.

Original: R EPRODUCIBILITY S TATEMENT
Revised: R EPRODUCIBILITY S TATEMENT

Original: We provide the implementation of the proposed method, as well as notebooks with the conducted experiments, as part of the supplementary material.
Revised: We provide the implementation of the proposed method, as well as notebooks with the conducted experiments, as part of the supplementary material.

Original: A 3D T ETRIS DATA
Revised: A 3D T ETRIS DATA

Original: The 3D Tetris dataset (Thomas et al., 2018) is the following set of point coordinates (each shape is of size× 3 ): chiral_shape_1:
Revised: The 3D Tetris dataset (Thomas et al., 2018) is the following set of point coordinates (each shape is of size× 3 ): chiral_shape_1:

Original: [(0, 0, 0), (0, 0, 1), (1, 0, 0), (1, 1, 0)], chiral_shape_2: [(0, 0, 0), (0, 0, 1), (1, 0, 0), (1, -1, 0)], square: [(0, 0, 0), (1, 0, 0), (0, 1, 0), (1, 1, 0)], line: [(0, 0, 0), (0, 0, 1), (0, 0, 2), (0, 0, 3)], corner: [(0, 0, 0), (0, 0, 1), (0, 1, 0), (1, 0, 0)], L: [(0, 0, 0), (0, 0, 1), (0, 0, 2), (0, 1, 0)], T: [(0, 0, 0), (0, 0, 1), (0, 0, 2), (0, 1, 1)], zigzag: [(0, 0, 0), (1, 0, 0), (1, 1, 0), (2, 1, 0)].
Revised: [(0, 0, 0), (0, 0, 1), (1, 0, 0), (1, 1, 0)], chiral_shape_2: [(0, 0, 0), (0, 0, 1), (1, 0, 0), (1, -1, 0)], square: [(0, 0, 0), (1, 0, 0), (0, 1, 0), (1, 1, 0)], line: [(0, 0, 0), (0, 0, 1), (0, 0, 2), (0, 0, 3)], corner: [(0, 0, 0), (0, 0, 1), (0, 1, 0), (1, 0, 0)], L: [(0, 0, 0), (0, 0, 1), (0, 0, 2), (0, 1, 0)], T: [(0, 0, 0), (0, 0, 1), (0, 0, 2), (0, 1, 1)], zigzag: [(0, 0, 0), (1, 0, 0), (1, 1, 0), (2, 1, 0)].

Original: B K NOWN AND ADAPTIVE ROTATION EXPERIMENTS : L1 DISTANCES
Revised: B K NOWN AND ESTIMATED ROTATION EXPERIMENTS : L1 DISTANCES

Original: We compare the hidden unit activations of the ancestor MLGP and constructed steerable models by computing the L1 distance (see Table 3).
Revised: We compare the hidden unit activations of the ancestor MLGP and constructed steerable models by computing the L1 distance (see Table 3).

Original: For convenience, we call the hidden unit activations of the ancestor MLGP fed with the shapes in the canonical orientation ground truth activations .
Revised: For convenience, we call the hidden unit activations of the ancestor MLGP fed with the shapes in the canonical orientation ground truth activations .

Original: From
Revised: From

Original: Table 3, we can see that the L1 distance to the ground truth activations linearly and moderately increases with the level of noise in the input data, which is a clear indication of the robustness of the classifier and corresponds well to the model accuracy results in Table 1.
Revised: Table 3, we can see that the L1 distance to the ground truth activations linearly and moderately increases with the level of noise in the input data, which is a clear indication of the robustness of the classifier and corresponds well to the model accuracy results in Table 1.

Original: Without steerability (“Ancestor” columns), the respective L1 distances increase by one order of magnitude and basically independently of the noise level, demonstrating the significant improvement by the steerability.
Revised: Without steerability (“Ancestor” columns), the respective L1 distances increase by one order of magnitude and basically independently of the noise level, demonstrating the significant improvement by the steerability.

Original: [Table]
Revised: [Table]

Original: The results presented in Table 4 indicate that the proposed online optimization, besides improving the classification accuracy (see Table 2), results in the decrease of the L1 distance to the ancestor MLGP softmax output values (referred to as ground truth output ) for the Tetris data.
Revised: The results presented in Table 4 indicate that the proposed online optimization, besides improving the classification accuracy (see Table 2), results in the decrease of the L1 distance to the ancestor MLGP softmax output values (referred to as ground truth output ) for the Tetris data.

Original: Note the low absolute level of the error already for the initial result.
Revised: Note the low absolute level of the error already for the initial result.

Original: In case of the 3D skeleton data, the absolute error level is also very low, but we observe a slight increase of the L1 distances after the optimization despite the improved accuracy (see Table 2).
Revised: In case of the 3D skeleton data, the absolute error level is also very low, but we observe a slight increase of the L1 distances after the optimization despite the improved accuracy (see Table 2).

Original: We presume it is caused by the increased number of hidden units in the ancestor model compared to the Tetris case (12 vs. 5), which might not all be equally relevant to the classification output.
Revised: We presume it is caused by the increased number of hidden units in the ancestor model compared to the Tetris case (12 vs. 5), which might not all be equally relevant to the classification output.

Original: [Table]
Revised: [Table]