Affine Steerable Equivariant Layer for Canonicalization of Neural Networks
Authors: Yikang Li
Yeqing Qiu
Yuxuan Chen
Zhouchen Lin
Venue: ICLR 2025 Poster (2025)
Keywords: equivariant networks
steerability
the affine group
equivariants
canonicalization
Primary area: unsupervised, self-supervised, semi-supervised, and supervised representation learning

Abstract: In the field of equivariant networks, achieving affine equivariance, particularly for general group representations, has long been a challenge.
In this paper, we propose the steerable EquivarLayer, a generalization of InvarLayer (Li et al., 2024), by building on the concept of equivariants beyond invariants.
The steerable EquivarLayer supports affine equivariance with arbitrary input and output representations, marking the first model to incorporate steerability into networks for the affine group.
To integrate it with canonicalization, a promising approach for making pre-trained models equivariant, we introduce a novel Det-Pooling module, expanding the applicability of EquivarLayer and the range of groups suitable for canonicalization.
We conduct experiments on image classification tasks involving group transformations to validate the steerable EquivarLayer in the role of a canonicalization function, demonstrating its effectiveness over data augmentation.

Published as a conference paper at ICLR 2025
AFFINE STEERABLE EQUIVARIANT LAYER FOR CANONICALIZATION OF NEURAL NETWORKS
Yikang Li", Yeqing Qin Yuxuan Chen', Zhouchen Lin
State Key Lab of General AI, School of Intelligence Science and Technology, Peking University
"The Chinese University of Hong Kong, Shenzhen "Shenzhen Research Institute of Big Data $ *Khoury College of Computer Sciences, Northeastern University
iInstitute for Artificial Intelligence, Peking University
Pazhou Laboratory (Huangpu), Guangzhou, Guangdong, China
ABSTRACT
In the field of equivariant networks, achieving affine equivariance, particularly for general group representations, has long been a challenge. In this paper, we propose the steerable EquivarLayer, generalization of InvarLayer (Li et al., 2024), by building on the concept of equivariants beyond invariants. The steerable EquivarLayer supports affine equivariance with arbitrary input and output repre- sentations, marking the first model to incorporate stecrability into networks for the affine group. To integrate with canonicalization. promising approach for making pre-trained models equivariant, we introduce a novel Det-Pooling module, expanding the applicability of EquivarLayer and the range of groups suitable for canonicalization. We conduct experiments on image classification tasks involv- ing group transformations to validate the steerable Equivar Layer in the role of a canonicalization function, demonstrating its effectiveness over data augmentation.
INTRODUCTION
Convolutional neural networks (CNNs) have achieved remarkable success across domains (He et al.,
2016 Chen al., 2017 Ren et al. 2016 Raissi al. (019), largely due to their property of trans-
lation equivariance. To incorporate broader symmetries into networks, Cohen & WWeling (0 26) introduce Group Equivariant CNNs (G-CNNs), which conduct convolutions
groups, preserving
rotation symmetry. Stecrable CNNs (Cohen & Welling 201 Weiler & Cesa 2019) generalize
G-CNNs, allowing for arbitrary input and output representation types by viewing features as fields.
Further works extend equivariance to subgroups of Euclidean groups across diverse domains (Es- teves et al. 2019; Weiler et al., 2018 Wang et al., 2022 Cohen et al. 2018 2019). Recently, canonicalization has emerged
novel and promising method for enforcing equivariance or invari- ance (Kaba et al. 2023). maps inputs to a canonical form, based on the group element output by a canonicalization function, before passing them through a non-equivariant network. This allows models to exhibit equivariance without architectural changes or retraining, thus enabling equivariant adaptation of pre-trained models (Mondal et al., (023). Currently, this method has been primarily applied to Euclidean groups, as the canonicalization function is typically constructed using equivari- ant networks with specific non-trivial group representations. Extending equivariant networks with steerability to more general groups can significantly broaden the applicability of canonicalization.
Achieving equivariance for more general groups, particularly the affine group, remains significant challenge. MMcaln et al. (2022) enable group convolution the affine group by converting the integral over the group into one over the Lie algebra, but it still requires group sampling, leading to exponential memory growth. Recent work by Mironenco & Forré (2024) improves sampling effi- ciency by decomposing large groups, yet limited to regular representations. Liet al. (2024) propose InvarLayer, based on differential invariants, achieving affine equivariance in networks without dis- cretizing or sampling the group for the first time. However, InvarLayer is inherently limited to the trivial representation for input and output.
still challenge design a layer that can handle affine equivariance for arbitrary input and output representation types. A more detailed literature review can be found in Appendix A
"Corresponding author.

---

Published as a conference paper at ICLR 2025
In this paper, we extend the InvarLayer (Li et al.
2024 framework to develop more general equiv-
ariant layer, called the steerable EguivarLayer, This extension analogous to the generalization
from G-CNNs (Cohen & Welling 2016a) to steerable CNNs (Cohen & Welling 2016b; Weiler &
Cesa 2019), and from PDO-eCon'e (Shen al. 2020) steerable PDOs (Jenner & Weiler (021).
While invariants, the core concept in Layer, only generate equivariant operators that produce features tied to the trivial group representation, we employ a generalized concept beyond invariants, referred to equivariants. Unlike invariants, which remain unchanged under group transforma- tions, equivariants transform according to given group representations, thus yielding more general equivariant operators that can map between different feature types. To construct equívariants, we introduce the notion of an equivariant matrix, which, when multiplied with invariants, generates equivariants. With existing methods for computing invariants (Olver) 20155 Wang et al.) 2013 Li et al. 2024), our focus turns to building equivariant matrices. We apply the tool of moving frames to construct them, similar to the process for constructing equivariant maps in (Peter Olver 2024)
Additionally. inspired by the construction of SupNorm normalized differential invariants (Li et al. 2024). introduce novel normalization technique enhance the numerical stability of equivari ant matrices. Based on equivariants, we design the steerable Equivarl enabling flexible speci- fication of input and output features corresponding to arbitrary group representations. The steerable EquivarL supports equivariance the affine group and its continuous subgroups, marking the first instance of affine steerable equivariant models that allow for flexible input and output represen- tation types, filling gap in the field of equivariant networks.
Building on this, we apply the steerable EquivarLayer as the canonicalization function within the canonicalization framework. Specifically, the steerable EquivarLayer outputs a matrix field that satisfies the equivariance property required for the canonicalization function. We then introduce novel module, Det-Pooling, to select the appropriate output matrix, enabling the model handle matrix groups such as GL + (2). This approach significantly enhances the flexibility and scope of canonicalization, extending it beyond the traditionally used rotation groups to support generalized group equivariance across broader range of applications.
We summarize main contributions as follows:
Extending the InvarLayer et al. 2024) framework, we design the steerable EquivarLayer based on equivariants, allowing arbitrary specification of input and output representations while supporting the affine group and continuous subgroups. This the first time steer- ability has been introduced to networks for the affine group.
We propose a new normalization technique that transforms relative equivariants into abso- lute equivariants, facilitating the construction of simpler equivariant matrices and enabling the generation more numerically stable equivariants.
We introduce a novel module, Det-Pooling, which allows the application of the steerable EquivarLayer to canonicalization. This extends the range of groups applicable to canoni- calization, making possible to handle more complex matrix groups.
We demonstrate the effectiveness of our approach through image classification experiments involving three -ucluuciid groups, including GT, GI,+(2), the rotation-scale group, and the scale group, showing better performance compared to data augmentation. T
2 STEERABLE EQUIVARLAYER
In this section, we introduce the steerable EquivarLayer, extending the InvarLayer framework to support affine equivariance with arbitrary input and output representations.
2.1 BASIC CONCEPTS AND NOTATIONS
In our framework, we treat the features of each network layer as a smooth vector function on continuous domain. We specifically focus on the most common case of the 2D plane, u R22 - Rs, where each point x is associated with vector u(x), referred to fiber, and is the number of channels. We consider transformation group G that acts on the input space R², specifying the symmetry aim maintain as equivariance across the layers. Moreover, the features of each layer are associated with group representation p p + GL(c), which characterizes the rype of the layer and dictates how the e channels of each feature vector u(x) transform under group actions.
The code is available at http://c....om...t//////iiiaaaa
2

---

Published as a conference paper at ICLR 2025
This property, known steerability, indicates that the layers respond transformations in manner consistent with their respective types.
Next, we formally introduce the notions and definitions related to group actions and equivariance. Given group G and function space F {u u R² + ] with type p. denote the group action on the function space corresponding to the group representation p as Lp(g), such that
(L u) (x) - p(g) ungg-t x).
(1)
Here, the group G acts on the base domain R2 in its standard way. Specifically, we focus on the affine group and its subgroups, where invertible matrix and b E R²² We consider the action of G on 18² as g X Ax + b. The group action defined in ① involves not only the transformation of positions but also the transformation of vectors at each point. Unlike the case of scalar fields, each vector is not only moved to a new position but also changes its orientation via the action of G. When the type is Po I. namely the trivial representation, the vectors retain their orientation, reducing the case that of scalar fields. With this understanding of group actions, we can now define the concept of cquivariance:
G can be represented g (A. b), with A R2x being an
Let G be group acting on function spaces F and F with types p and p', respectively. An operator vb F + F is said to be equivariant if
Definition
(Lppgg) o u] = (pp(())) o vful,, VuEF,
An important property of equivariance is its transitivity: the composition of multiple equivariant operators remains equivariant. Consequently, if the mapping between every layer in network satisfies equivariance with corresponding types, the entire network exhibits steerability.
In (Li et al. 2024), the primary concept is invariants, which also serves as one of the key concepts in this paper. We extend the definition from (Li et al. 2024) follows:
Definition Let G be a group acting on the function space F {u u: 122 + RR with type p. We define I : I2 x F + IR as an invariant if it satisfies
I(g x, Lg(gg) o u) = I(x, Vx R2, UEF,
(3)
We define I A (I) as ks-dimensional invariant if Is are all invariants of G.
Compared to the definition in (Li et al.) 22044, our formulation generalizes the group action on F. while the fundamental properties of invariants and methods for computing them remain unchanged. Both the classical approach of deriving differential invariants (Fels & Olver 1999 Olver 2003 2015) and the SupNorm normalization method described in eLi al. 2024) are still applicable.
Invariants and equivariance are closely intertwined. The following proposition reveals how to con- struct an equivariant operator from invariants:
Proposition 3 Let G be group acting the function space F = {u u lu R2 + RC} with rype p. and let I R22 x F + IR be a c'-dimensional invariant. Given F, the function I(., u) can be
viewed as an element in F {v v 2
TR. We define I F -> F as
- I I(.
(4)
Then I is equivariant operator satisfying
I(Lp(9) ul L. Tuj. YuE
(5)
where Po(g)= = I denotes the trivial group representation.
As shown in Proposition 3 Ż yields an equivariant operator that maps features of type p to features of the trivial type Pa- However, construct more general equivariant operators that map features from type p to type p', invariants alone may not suffice. This is the challenge we aim to address.
2.2 EQUIVARIANTS
To achieve steerability across network layers, we need to construct equivariant operators capable of mapping features between any specified input and output types. To this end, we introduce gen- eralized concept beyond invariants, referred as equivariants. Equivariants maintain the required transformation properties when transitioning between different feature types. The formal definition is provided below:
3

---

Published as a conference paper at ICLR 2025
Definition 4 Let G be group acting on the function space F == {u u RR - RR)
with rype p.
We define map & R F Th. as a type- p') equivariant if satisfies
Elg x,
(0( 8(x, u), VX VXERR,
(6)
It is worth noting that an equivariant can be viewed as c' e'-dimensional invariant when p' is the trivial group representation. Unlike invariants, which produce outputs that always remain un- changed under group transformations, equivariants actively transform according to specified group representations. This flexibility allows us to construct more general equivariant operators capable of mapping features between arbitrary types. The relationship between equivariants and equivariant operators can be formalized as follows:
Proposition 5 Let be a group acting on the function space F {u u: R2 and let & IR? x x + IR be type-(p. p') equivariant. Given u F, the function E(,, u) can be viewed as an element in F = {v v: IR² + RR Denote &. F + F as
R with rype p.
Elu] Eu, u).
Then & is an equivariant operator that satisfies
ElLp(g) = (g) Vue FgEE.
(8)
As demonstrated in Proposition the equivariant operator & associated with the equivariant & fa- cilitates more generalized equivariace, enabling to specify the group actions on both input and output features. The crucial question, therefore, is how to systematically construct such equivariants.
To address this challenge, we introduce the concept of an equivariant matrix, which establishes framework for constructing equivariants from invariants by creating relationship between the two. The formal definition is provided below:
Definition 6 Let G be group acting the function space F (u u R
RR} with type p.
We define map M: R
Ted xe' type-(p. p¹) equivariant matrix if it satisfies
Míg x. Lp) o = (')) M(x u), R
(9)
Using this equivariant matrix, formalized in the following theorem:
construct equivariants by combining it with invariants, as
Theorem Let G be group acting on the function space F = {u u R + R with type p.
Suppose M ; IR2 F + ge xe' is type-(p, p') equivariant matrix, and I R2 x F R is c-dimensional invariant. Denote E RRR
T
E(x, u) e M(x u) I(x,u) u).
(10)
Then, E is a type-(p.pt) p¹) equivariant, iee.
Egg x, Lp(g) o u) = p'())) - u),
UXERR
(11)
There are several well-established methods for constructing invariants, such deriving differential invariants through the classical moving frame approach (Fels & Olver 1999 Olver 2003 2015) and
utilizing the SupNorm normalization technique proposed in (Li al. 2024). With these methods, can derive a set of basis invariants and further extend them through arbitrary functional combinations to obtain a rich and comprehensive set of invariants for given group action. Once a non-trivial equivariant matrix M is obtained, we can combine it with different invariants I construct diverse set of equivariants.
Next, we tackle the challenge of constructing equivariant matrices. To do this, we will employ the tool of moving frames. First, we provide the definition of moving frame.
4

---

Published as a conference paper at ICLR 2025
Definition 8
(Olver 2015) Let G be an r-dimensional Lie group acting on an m-dimensional man- G called a moving frame if satisfies
ifold 2. map
alg- 2) a==))(())))))) 9
Yz2 E E.
(12)
The following theorem provides method for the practical computation of a moving frame.
Theorem 9 Olver 2015) Let G act freely and regularly on 2, and let K. be a cross-section. For E 2, let g a(z) be the unique group element mapping to this cross-section:
g 2 = alz) - E
(13)
Then a 2 G is moving frame for the group action.
The group action is free if, for any 2 E Z, the only group element that leaves unchanged is the identity. The group action is regular if the orbits form a regular foliation of 2. A cross-section is subset of Z defined by fixing coordinates, expressed KC = {21 C1,...g 2, cr), where Ct,...,6 Gr are constants and r = dim G. This cross-section intersects each group orbit exactly once and transversely, providing unique representation for each orbit. For more detailed and rigorous definitions, please refer (Olver 2015).
With Theorem we can now apply in our context. Consider the manifold as the jet space defined by Z = {z z (x, u(x), Vu(x).... v²u(x)). E R²). The group action on Z is given by the prolongation of the group action on the function space F. Let ug = Lp(g) o u. Explicitly, the action 2 + g is defined as follows:
(x, x Vu(x), .... u(x)) + (g x. ug x). Vug(g x)
V Vuul x)).
With this setup, we can then compute the corresponding moving frame a. The following theorem reveals how to use this moving frame to construct an equivariant matrix.
Theorem 10 Let G be group acting the function space F with type p. Consider the manifold Z as the jet space of F, and let G act on via the prolongation of its action on F. Suppose we have a moving frame a Z - G. Define M: TR.2 x F + Rd xe' as
M(x, u) = (p'(a(2)))
(14)
where: : = (x, u(x), Vu(x), ... V'((x)) E 2. Then M is a type-(p, p') equivariant matrix.
So far, have presented complete framework and theoretical foundation for constructing equiv- ariants. First, we compute the moving frame as outlined in Theorem 9 Next, we utilize Theorem TO] to derive an equivariant matrix from this moving frame. Finally, we combine the equivariant matrix with various invariants, shown in Theorem approach provides a systematic way to build equivariant operators that can map features between any specified input and output types, thereby enabling a high degree of flexibility in the design of equivariant networks.
generate wide variety of equivariants. This
2.3 SUPNORM NORMALIZED EQUIVARIANTS
In the previous subsection, we outline a thcoretical method for constructing equivariants. Building upon this foundation, we can adopt more flexible strategies to obtain equivariants that are simpler in form and exhibit improved numerical stability practice. A critical aspect of this process the construction of equivariant matrices. Similar to the problem encountered with differential invariants, discussed in (Li et al. 2024). direct computation of equivariants for the affine group may lead to complex expressions and potential division-by-zen issues. To mitigate these problems, we propose a new SupNorm normalization technique for the construction of more robust equivariant matrices.
While Thcorem 7 shows that equivariant matrices be used to construct equivariants, the rela- tionship between equivariants and equivariant matrices even more intricate: each column of an cquivariant matrix is itself an equivariant. Conversely, if every column of a matrix-valued function is an independent equivariant, the entire function qualifies as equivariant matrix. Thus, our objective is to ensure that each column of the -aluux function independently forms an equivariant.
Here, we consider the affine group G = {(> A, & > 0, Å R2x2 dex() 1 e R2), which is, more precisely, the connected component of the full affine group. The equi-affine
5

---

Published as a conference paper at ICLR 2025
group, denoted as SA(2), is the subgroup of the affine group without scaling, defined as SA(2) ((A,b) b) A R2x2 det( A) 1, R22. In (Li al. 2024), SupNorm normalized differential invariants of the affine group are constructed by normalizing relative differential invariants which can be derived from the differential invariants of the subgroup (2) Inspired by this approach, we begin with the equivariant matrix for (2),), and aim transform each column into an equivariant of the affine group. To formalize this process, we begin with the definition of a relative equivariant:
Definition Let G be the affine group acting on the function space == {u uu I22 + }R with type p. Let s(g) denote the scaling factor of the affine group element We define & IR2 x Re as rype- (p. p') relative equivariant with power if satisfies
1 Eig x. x. (s(g))m p'(g) E((x Vx
(15)
pf = Po (the trivial representation), & is called a relative invariant.
Then, we provide the following theorem to demonstrate our new SupNorm normalization technique for converting a relative equivariant into full equivariant:
R2 } denote the set of smooth bounded functions on IK2, Define the SupNorm on Fu as |lv & supxega l|v(x) Let G be the affine group acting on the function space F. with type p. Suppose
Theorem 12 Let Fs (v Iv:
R² F. + R is a type-(p, p') relative equivariant with power m1, and I IR? x Fo + IR is a relative invariant with power mz + Define E: R x Fo + Re as follows:
1
8(x, u)
E(x, u).
(16)
m m2
(II(, u) u)lp)
Then & type-(p, p¹) equivariant.
The above theorem provides a concrete method to transform relative equivariants into exact equiv- ariants using SupNorm normalization. Next, we will demonstrate a typical example that applics Theorem 12 for clearer understanding and practical usage. For instance, we define Po as the triv- ial representation, and P1 (g] A where = (A, E G. By computing the type- P1) equivariant matrix for SA(2), we obtain
Uy2lax Un May Myllay U2 Myy
Us
(17)
where By, Man, Myy and May denote the partial derivatives of the scalar function w(x, y). Al- though this function not directly exact equivariant matrix for the affine group, each column is type-(pr (Po. Pı) relative equivariant. Specifically,, the first column has a power of 2, and the second column has power of 0, making it a true equivariant on its own. We use power relative invariant Uzx Byy d to apply SupNorm normalization to (17, yielding the following matrix:
Uy Uy - Ug May
)
Uh
(18)
S(w)
My May
Ue Uyy
Uly
where S(u) = ||u Myy - u2zy /sup- This resulting matrix in (88 is type-(po- P1) equivariant for the affine group. In contrast directly computing affine equivariant matrices, which may involve complex fractional polynomial expressions prone to the division-by-zet by-zero issue, this method yields a numerically stable equivariant matrix. Such division-by-zeee problems would only occur in the rare case where is a constant function, a scenario that typically negligible practical applications.
Furthermore, this method is also applicable to continuous subgroups of the affine group. For exam- ple, in the case of the rotation-scale group, one compute the equivariant matrix for the rotation group to obtain relative equivariants, which can then be normalized to form an exact equivariant matrix using the same SupNorm normalization technique. For the scale group, one can start with the identity matrix and apply appropriate normalization to directly obtain the equivariant matrix.

---

Published as a conference paper at ICLR 2025
f
Hidden Layer
by 2/mml
Uin
Wout
M
M(ual
Figure 1: Architecture of the steerable EquivarLayer.
2.4 ARCHITECTURE OF EQUIVARLAYER
Building on the previous theoretical framework, we can derive the corresponding equivariants given the specified group representation types, leading to the construction of the desired equivariant oper- ators. In practice, our ultimate goal is to design a parameterized steerable equivariant layer that can handle arbitrary specified input and output feature types.
Let the input and output group representations be denoted by Pin. and Post. with dimensions and c', respectively. If Pout is the trivial representation, we can adopt the structure of InvarLayer from (Li et al. 2024). The equivariant layer is defined as:
Hout
hgg i [
(19)
where is a k-dimensional invariant formed by a set of basis invariants, I is the corresponding equivariant operator defined in (4) and hg is learnable multi-layer perceptron (MLP) with input
dimension and output dimension e'.
Next, we focus on the more general case where Pout is not trivial representation, requiring equiv- ariant operators defined by equivariants. In this scenario, the equivariant layer is defined as:
Hout M[u (hy I(um.))
(20)
where I and ha follow the definitions in 19p. MM a type-(pin- Pour) equivariant matrix, and M[u] & M(- u). This formulation allows the layer to handle arbitrary input and output feature types. We refer this general equivariant layer type-(Pin; Pout) EquivarLayer (see Figure 1. The layer defined in (19), where Pour is the trivial representation, can be viewed as a special case, specifically type-(Pin: Po) Equivarl
Moreover, we consider a common scenario where the input and output are multi-channel. Specifi- cally, we focus on the setting where pin T Pa and Pour = @ ps. meaning that both the input and output are direct sums of multiple channels of the group representations Pa and ps, respec- tively. While it is theoretically possible to handle this setting using the methods described earlier, can leverage the structure of the group representations to simplify the computation. We define the multi-channel version of EquivarLayer (see Figure Appendix E follows:
M (ho o I[u;)]) (1)
Hout
(21)
(Cz)
Mxluml (he I I(uun))
where My[ume] = EE M mn Here, A are learnable scalar parameters. M is type-(pa: pm) equivariant matrix, denotes the i-th channel vector function of u,.. and (5)
represents the j-th channel vector function of he D[,,]] This formulation enables efficient processing of multi-channel inputs and outputs while preserving the desired group equivariance.
(ho (u;.))

---

Published as a conference paper at ICLR 2025
3 STEERABLE EQUIVARLAYER FOR CANONICALIZATION
Canonicalization is a method for achieving invariance or equivariance without imposing constraints on the architecture of a neural network. By utilizing a canonicalization function, the input is mapped canonical sample before being passed to non-equivariant prediction network, ensuring that the output remains invariant. In the case of cquivariance, the input is mapped to its canonical form, the prediction is made, and the result is transformed back to its original position under the group action. Below, we give a formal definition of canonicalization in our context.
Definition 13 Let G be a group acting on the function space F with type p. A map q: - G is called a canonicalization function it satisfies
((Lpp9) u) = (pu) Vg G
(22)
To illustrate the process of canonicalization, we state the following theorem:
Theorem 14 Let G be a group acting on function spaces F and F' with types p and p', respectively. Given a prediction operator You : F,, define vp F + F' as follows:
(uu - L ((p(()))) vo (olL)(((u))) o ul.
(23)
Then db is equivariant, satisfying (g) o ul = L (9) (u]. If we remove L ((:(()))))) in
23
the operator becomes invariant.
The core component of canonicalization is the canonicalization function, which can be interpreted as form of equivariant mapping. In the direct approach (Kaba et al. 2023), an equivariant network that produces a group element serves as the canonicalization function. Previous works have mostly focused on rotation groups for canonicalization, largely due to the absence of more general steerable equivariant networks. We aim leverage the proposed steerable EquivarLayer to extend this concept to more general groups. Specifically, we focus on G = GL & {A A 22x2, det(A) (A) > 0}.
Thus, our goal is to construct canonicalization function yp : F R2x2 that satisfies
{(Lp(A) o u) = puu) A-1,
(24)
We proceed in two steps. Let Letpp(A) (A) = A-T, A E G. First, construct a map di F F. satisfying Φ1 (A)(() o Lp. (A) o @ [u]. where F F= (v v R2 + R2x2j. Next, construct second map db F R222 such that 52 L.p (A) o ov = (A( 2 (v]. Composing these maps, 02 Φi satisfies (A) o) = A-T d(u). Thus, o is valid canonicalization function.
For the construction of Φ1. note that the function space f = {v can be associated with the vector-valued function space F {u u R² + R. corresponding to the group representation Pe 2 Pe- Therefore, we can employ a type- P. @ p.) EquivarLayer to serve as Φ1. The output of this EquivarLayer is then reshaped to match the required form in F. In practice, this equivariant network can consist of multiple layers, as long as the input feature type of the first layer is type p and the output feature type of the final layer is type P. ① Pc-
E² + [R22] with type Pe
For the construction of Φ2, we need to select one matrix from a matrix- matrix-valued function while main- taining the equivariance condition. For this purpose, we propose module called Def-Pooling, which selects the matrix with the largest absolute determinant from the matrix-valued function. The key idea is that under the transformation T.p. (A), each point is mapped to a new location and its asso- ciated matrix are also transformed, but the matrix with the largest absolute determinant remains the largest one. The following theorem defines de and demonstrates its property:
Theorem 15 Assume v F and that |de((v(x)) has a unique maximum over x R Let Det- Pooling Φ2 : F R2x2 be defined as
Φ2(v) v argmmax /ddd/d dee((v(x)))) XER²
(25)
Then 62 satisfies the equivariance properly
22 L2 (A) o v] P0::pppAA (c) -b[[)) V G.
(26)
8

---

Published as a conference paper at ICLR 2025
Det-Pooliig effectively tracks the matrix with the largest absolute determinant, thus ensuring equiv- ariance under group transformations. In addition, reduces the likelihood of selecting singular matrix, as this would only occur if v(x) () is singular at every point, which is a rare scenario in practice.
construct Φ1 using steerable EquivarLayers and employ Det-Pooling Φ2, together forming the canonicalization function. Notably, this method can also be extended to handle subgroups of GL" aL*(( (2), making a versatile approach for broader applications.
In summary,
EXPERIMENT
To evaluate the effectiveness of the proposed steerable Equivarl we apply it to canonicalization in combination with an unconstrained prediction network. Our experiments focus on image classi- fication tasks using the MNIST dataset and its transformed versions under different group actions. Specifically, we assess the performance of our method on three non-compact continuous groups: the GL (2) group, the rotation-scat scale group, and the scale group.
4.1 EXPERIMENT SETUP
We use a ResNer50 model pre-trained on the ImageNet-1 IK dataset as the prediction network for our experiments, which has 23.5M parameters. The model is fine-tuned on the MNIST dataset us- ing three distinct strategies: (1) Vanilla: Training
the original MNIST dataset without any data augmentation. (2) Mild augmentation: Training with data augmentation, involving mild affine trans- formations, including rotation within 10 degrees, scaling between 0.9 - 1.1, and shear within +5 degrees. (3) Full augmentation: Training with the same data augmentation used in the transformed test set. These three strategics represent common training practices, and the resulting models serve as our baselines for comparison.
For the canonicalization function, integrate EquivarLayer with ResNet32 architecture, where the convolutional layers are replaced with type-(po. Po) EquivarLayers, and the output module is replaced with a type-(po. P. @ Pc) EquivarLaye followed by Det-Pooling as described in Section 3 In each experiment, each model uses EquivarLayers tailored the corresponding group action and contains fewer than 0.4M parameters. For convenience, we call the whole model Equivarl Inspired by (Mondal et al. 2023), we adopt training stage for alignment. Although the canoni- calization function can map elements within group orbit to a canonical representative, we aim to ensure that this representative is aligned with the orientation the prediction network is familiar with, such as the upright position. EquivarLayer is trained independently of the prediction network by generating random transformation matrices applied to the training images. The loss is defined as the mean squared eтror (MSE) between the identity matrix and the product of EquivarLayer's 's output and the corresponding transformation matrix. During inference, the trained EquivarLayer is connected to a pre-trained prediction network that uses mild augmentation. The EquivarLaye canonicalizes the input images (see Figure 3 in Appendix E). which are then fed into the prediction network for classification. More experiment details are given in Appendix
4.2 EVALUAION
Typically, full data augmentation helps improve a model's performance on transformed datasets, but it often comes the cost of reduced performance the original dataset. Mild augmentation, on the other hand, may enhance performance on the original dataset, but usually fails to achieve low test error on transformed datasets. Our objective is to leverage canonicalization to improve performance on transformed datasets while maintaining competitive performance the original dataset.
4.2.1 GI. (2) GROUP
Table Test error (%) on MNIST and MNIST-GL (2).
Method
MNIST
MNIST-GI. ( (2)
Vanilla
0.87+0.07
54.22+-098
Mild Augmentation
0.45+0.03
47 17.28+1.51
Full Augmentation
1.19
1.52
EquivarLayer Canonicalizer (ours)
0.81+
1.32+0.08

---

Published as a conference paper at ICLR 2025
We conduct experiments the original MNIST dataset and its transformed version, MNIST- GL"((2, which undergoes random transformations from GL (2) group including arbitrary rota- tions between - 180 to 180 degrees, scaling between 0.8 1.2, and shear within +0 degrees. Each experiment is repeated five times with independent random seeds, and we report the mean + stan-
dard deviation of test emor in Table
Mild augmentation improves performance on both MNIST and MNIST-GL* (2) compared to the vanilla model, though the results on MNIST-GL+ remain relatively sub-optimal. Full augmenta- tion achieves good performance on MNIST-GL* (2) but performs the worst on the original MNIST dataset. Our method, combining mild augmentation with canonicalization, outperforms full aug- mentation on MNIST-GL" (2) while maintaining competitive performance on the original MNIST dataset, second only to mild augmentation.
4.2.2 ROTATION-SCALE GROUP
Table Test error (%) on MNIST and MNIST-RS.
Method
MNIST
MNIST-RS
Vanilla
0.87
54.18=00993
Mild Augmentation
0.45
47.36_1.39
Full Augmentation
1.19
1.65.10.06
EquivarLayer Canonicalizer (ours)
0.71
1.06+0.07
We also evaluate our method on the rotation-scale group, where MNIST-RS includes arbitrary rota- tions between 1880 to 180 degrees and scaling between 0.8 - 1.2. The results, averaged over five independent runs, are shown in Table 2 Our method yields the best performance on MNIST-RS while maintaining low test error on MNIST, close that of mild augmentation.
4.2.3 SCALE GROUP
Table 3: Test error (%) on MNIST and MNIST-Scale.
Method
MNIST
MNIST-Scale
Vanilla
0.87
8.7242.18
Mild Augmentation
0.45
1.62
Full Augmentation
0.49
0.67
EquivarLayer Canonicalizer (ours)
0.44
0.64
We further evaluate our method on the scale group, where the MNIST-Scale dataset involves random scaling transformations with factors ranging from 0.8 to 1.6. The results are presented in Table 3 Since the scale transformation is relatively simple, mild augmentation already yields a significant improvement on MNIST-Scale and full augmentation achieves near-perfect test error on both the MNIST and ANIST-Sccae datasets. However, our method not only achieves the best performance the MNIST-Scale dataset but also slightly outperforms all baselines on the original MNIST dataset.
CONCLUSION
In this paper, we introduce the steerable EquivarLayer, extending the InvarLayer framework sup- port equivariance with arbitrary input and output feature types for the affine group and its continuous subgroups. This marks the first time steerability is achieved in networks for the affine group. We also develop the novel Det-Pooling module, which enables the steerable EquivarLayer's applica- bility to canonicalization, thus extending the method to handle more complex matrix groups. Our experiments demonstrate the effectiveness of this approach, showing that the steerable EquivarLayer, when used as a canonicalization function, outperforms traditional data augmentation methods.
Our framework holds great potential for future research, particularly in extending beyond the 2D plane, the theoretical foundations apply higher-dimente dimensional spaces. Exploring equivariance more general manifolds and applying steerable Equivarl Layers Layers a wider variety of groups is another promising direction. Additionally, expanding the application of this method to a broader range of tasks, from computer vision to scientific problems, presents valuable avenues for further study.
10

---

Published as a conference paper at ICLR 2025
ACKNOWLEDGMENT
Z. Lin was supported by National Key R&D Program of China (20222D0160300) and the NSF China (No. 62276004).
REFERENCES
Liang-Chie Chen, George Papandreou, lasonas Kokkinos, Kevin Murphy, and Alan L Yuille. Daep Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs. JEEE Transactions on Pattern Analysis and Machine Intelligence, pp- 834 848, 2017.
Taco S Cohen and Max Welling. Group equivariant convolutional networks. In International Con- ference on Machine Leurning, 2016a.
Taco S Cohen and Max Welling, Steerable CNNs. In International Conference on Learning Repre- sentations, 2016b.
Taco Cohen, Mario Geiger, Jonas Köhler, and Max Welling. Spherical CNNs. In International Conference on Learning Representations, 2018.
Taco S Cohen, Maurice Weiler, Berkay Kicanaoglu, and Max Welling. Gauge equivariant convo- lutional networks and the icosahedral CNN. In International Conference on Machine Learning. 2019.
Carlos Esteves, Avneesh Sud, Zhengyi Luo, Kostas Danilidis, and Ameesh Makadia. Cross-domain 3D equivariant image embeddings. In International Conference Machine Learning, 2019.
Mark Fels and Peter J Olver. Moving coframes: regularization and theoretical foundations. Acta Applicandae Mathematica, 55:127-208 1999.
Marc Finzi, Samuel Stanton, Pavel Izmailov, and Andrew Gordon Wilson. Generalizing convolu- tional neural networks for equivariance to Lie groups arbitrary continuous data. In Interna- tional Conference on Machine Learning, 2020.
Fabian Fuchs, Daniel Worrall, Volker Fischer, and Max Welling. SE(3))transformmrs: 3D roto- translation equivariant attention networks. In Advances in Neural Information Processing Sys- tems, 2020.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2016.
Lingshen He, Yuxuan Chen, Yiming Dong, Yisen Wang, Zhouchen Lin, et al. Efficient equivariant network. In Advances in Neural Information Processing Systems, 2021.
Lingshen He, Yuxuan Chen, Zhengyang Shen, Yibo Yang, and Zhouchen Lin. Neural ePDOs: Spatially adaptive equivariant partial differential operator based networks. In International Con- ference on Learning Representations, 2022.
Boce Hu, Xupeng Zhu, Dian Wang, Zihao Dong, Haojie Huang, Chenghao Wang, Robin Walters, and Robert Platt. Orbitgrasp: SEE(3)-euivvaiiat grasp learning. arXiv preprint 1rX1v:2407.03531, 2024.
Haojie Huang, Dian Wang, Robin Walters, and Robert Platt. Equivariant transporter network. In Roboties: Science and Systems, 2022.
Haojie Huang. Dian Wang, Xupeng Zhu, Robin Walters, and Robert Platt. Edge grasp network: A graph-based SE(3)-invariant approach to grasp detection. In 2023 IEEE International Conference on Robotics and Automation, pp. 3882-3888. IEEE, 2023.
Haojie Huang, Haotian Liu, Dian Wang, Robin Walters, and Robert Platt. Match policy: simple pipeline from point cloud registration to manipulation policies. arXiv preprint arXiv:2409.15533 2024.
11

---

Published as a conference paper at ICLR 2025
Erik Jenner and Maurice Weiler. Steerable partial differential operators for equivariant neural net- works. In International Conference on Learning Representations, 2021.
Sékou-Oumar Kaba, Arnab Kumar Mondal, Yan Zhang, Yoshua Bengio, and Siamak Ravanbakhsh. Equivariance with learned canonicalization functions. In International Conference on Machine Learning. 2023.
David M Klee, Ondrej Biza, Robert Platt, and Robin Walters. Image to sphere: Learning equivariant features for efficient pose prediction. In International Conference on Learning Representations, 2023.
Yikang Li, Yeqing Qiu, Yuxuan Chen, Lingshen He, and Zhouchen Lin. Affine equivariant networks based on differential invariants. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024.
Yi-Lun Liao and Tess Smidt. Equiformer: Equivariant graph attention transformer for 3D atomistic graphs. International Conference on Learning Representations, 2022.
Lachlan E MacDonald, Sameera Ramasinghe, and Simon Lucey. Enabling equivariance for arbi- trary Lie groups. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022.
Mircea Mironenco and Patrick Forré. Lie group decompositions for equivariant neural networks. In International Conference Learning Representations, 2024.
Arnab Kumar Mondal, Siba Smarak Panigrahi, Oumar Kaba, Sai Rajeswar Mudumba, and Siamak Ravanbakhsh. Equivariant adaptation of large pretrained models. In Advances in Neural Infor- mation Processing Systems, 2023.
Peter Olver. Moving frames. Journal of Symbolic Computation, 36(3-4)550--12,, 2003.
Peter Olver. Modern developments in the theory and applications of moving frames. London Math. Soc. Impact150 Stories, 1:1 1--450, 2015.
Jung Yeon Park, Ondrej Biza, Linfeng Zhao, Jan Willem Van De Meent, and Robin Walters. Learn- ing symmetric embeddings for equivariant world models. In International Conference on Machine Learning. 2022.
Jung Yeon Park, Sujay Bhatt, Sihan Zeng, Lawson LS Wong, Alec Koppel, Sumitra Ganesh, and Robin Walters. Approximate equivariance reinforcement learning. arXiv preprint arXiv:2411.0222 1.0425,, 2024a.
Jung Yeon Park, Lawson Wong, and Robin Walters. Modeling dynamics over meshes with gauge equivariant nonlinear message passing. In Advances in Neural Information Processing Systems, 2024b.
Peter J. Olver. Using Moving Frames Construct Equivariant Maps. preprint, 2024. 7/ /wwwwuserss cse. umn.
https:
pdd.
M Raissi, Perdikaris, and GE Karniadakis. Physics-ifformed neural networks: A deep learn- ing framework for solving forward and inverse problems involving nonlinear partial differential equations. Journal of Computational Physics (Print), 378:686-707, 2019.
Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun. Faster R-CNN: Towards real-time object detection with region proposal networks. IEEE transactions on pattern analysis and machine intelligence, 39(6) 2016.
David Romero, Erik Bekkers, Jakub Tomezak, and Mark Hoogendoorn. Attentive group equivariant convolutional networks. In International Conference on Machine Learning, 2020.
Mateus Sangalli, Samy Blusseau, Santiago Velasco-Forero, and Jesús Angulo. Differential invari- ants for SE(2)-equivariant networks. In 2022 IEEE International Conference on Image Process- ing. pp.
IEEE, 2022.
Mateus Sangalli, Samy Blusseau, Santiago Velasco-Forero, and Jesus Angulo. Moving frame net: SE(3)-equivariant network for volumes. In NeurIPS Workshop on Symmetry and Geometry in Neural Representations, pp. 81-97. PMLR, 2023.
12

---

Published as a conference paper at ICLR 2025
Zhengyang Shen, Lingshen He, Zhouchen Lin, and Jinwen Ma. PDO-eConvs: Partial differential operator based equivariant convolutions. In International Conference on Machine Learning, 2020.
Zhengyang Shen, Tao Hong, Qi She, Jinwen Ma, and Zhouchen Lin. PDO-s3DCNNs: Partial dif- ferential operator based steerable 3D CNNs. In International Conference on Machine Learning, 2022.
Ivan Sosnovik, Michal Szmaja, and Arnold Smeulders. Scale-equivariant steerable networks. In International Conference Learning Representations. 2019.
Dian Wang, Robin Walters, and Robert Platt. SQu2)-equivarian reinforcement learning. In Interna- tional Conference on Learning Representations, 2022.
Rui Wang, Robin Walters, and Rose Yu. Incorporating symmetry into deep dynamics models for improved generalization. In International Conference on Learning Representations, 2020.
Rui Wang, Elyssa Hofgard, Han Gao, Robin Walters, and Tess Smidt. Discovering symmetry break-
ing in physical systems with relaxed group convolution. In International Conference on Machine Learning, 2024.
Yuanbin Wang, Xingwei Wang, Bin Zhang, et al. Affine differential invariants of functions on the plane. Journal of Applied Mathematics, 2013, 2013.
Maurice Weiler and Gabriele Cesa. General E())-equivariant steerable CNNs. In Advances in Neural Information Processing Systems, 2019.
Maurice Weiler, Mario Geiger, Max Welling, Wouter Boomsma, and Taco S Cohen. 3D steerable CNNs: Learning rotationally equivariant features in volumetric data. Advances in Neural Information Processing Systems, 2018.
Daniel Worrall and Gabriel Brostow. Cubenet: Equivariance to 3D rotation and translation. In Proceedings of the European Conference on Computer Vision, 2018.
Han Xiao, Kashif Rasul, and Roland Vollgraf. Fashion-MNIST: novel image dataset for bench- marking machine learning algorithms. arXiv preprint 1r.iv:1708.001 2017.
Yinshuang Xu, Jiahui Lei, and Kostas Daniilidis. Equivariant light field convolution and transformer in ray space. In Advances in Neural Information Processing Systems, 2023.
Yinshuang Xu, Dian Chen, Katherine Liu, Sergey Zakharov, Rares Andrei Ambrus, Kostas Dani- ilidis, and Vitor Campagnolo Guizilini. SE(3) equivariant ray embeddings for implicit multi-view depth estimation. In Advances in Neural Information Processing Systems, 2024.
Maksim Zhdanov, Nico Hoffmann, and Gabriele Cesa. Implicit convolutional kernels for steerable CNNs. In Advances in Neural Information Processing Systems, 2023.
13

---

Published as a conference paper at ICLR 2025
A RELATED WORKS
A.1 EQUIVARIANT NETWORKS
Equivariant networks have been widely applied in various fields, such computer vision (Esteves)
al. 2019; Weiler al. 2018 Klee et al. 2023 Xu al. 2023 2024). sciences (Liao & Smidt
2022 Wang et al. 2024 Park et al. 2024b), reinforcement learning and robotics (Wang et al. 2022
Park et al. 2022 2024a Hu al. 2024: Huang et al. 2022, 2023; 2024). One primary approach to
building group equivariant networks is based on G-CN Ns (Cohen & Welling 201 where feature maps are considered as functions defined on a group and the group action on the feature is essentially permutation over the domain. From this perspective, various equivariant operators are designed for cycle or dihedral groups (Cohen & Welling 2016a He et al. 2021, Romero et al. 2020, Worrall & Brostow 2018: (hhen et al.) 2020; Cohen et al. 2019). Besides discrete groups. Cohen et al. (2018) have extended such an approach to handle inherent S3(3) equivariance for data defined on the sphere. Finzi et al. (020) further proposed LicConv that is equivariant to transformations from common Lie groups However, it fails to handle more complex Lie groups like the affine group due to the difficulty of sampling the Haar measure. Furthermore, MacDonald
al. (2022) overcome this challenge by designing special sampling strategies on the attine group. However, since the domain of feature is defined groups. it is unavoidable to sample when dealing with the affine group, which would aggravate the computation burden as the increasing of samples and layers. Mironenco & 2024)) tackles this problem by decomposing large group into smaller ones and sampling them to enhance sample efficiency. It requires sampling from the ([(()--ivvarian measure of positive definite matrices. Instead, they approximate this using a log-normal distribution, which may theoretically introduce imperfect equivariance.
Another approach to constructing equivariant networks originates from steerable CNNs (Cohen & Welling, 2016b; Weiler & Cesal 2019). In this framework, the feature maps are viewed as fields and are transtormed according to specific group representation under the action of group transfor- mation. Actually, the feature maps in the first approach precisely correspond fields with regular representation. This approach can directly handle continuous groups since the group transformation can be preserved by the linear transformation of the vectors on the fields. Based on this method, works (Cohen & Welling, 2016b Jenner & Weiler 2021 He et al. 2022) have developed operators to handle rotations on the 2D plane. WWilerr et al. (2018) : Fuchs et al. (020): Shen et al. (2022) further design equivariant layers to handle transformations in 3D space like E(3) and its subgroups. Zhdanov et al. (2023) provide a general framework for building neural networks equivariant to trans- lations and subgroups of O(n). However, it is crucial to recognize that these methods are mainly designed to tackle rigid transformations and are not well-suited for more general affine transforma- tions.
Besides these two mainstream approaches, some works construct equivariant networks based on differential invariants (Sangalli al. 2022 2023 Li al. 2024). In partieular, Li al. (2024) propose a novel equivariant layer, InverLayer, which achieves affine equivariance without the need for sampling from groups However, the input and output representation types of InvarLayer are constrained the trivial representation. Our work is a further extension of InvarLayer, capable of handling any given representation type of the affine group.
A.2 EQUIVARIANCE BY CANONICALIZATION FUNCTIONS
In equivariant networks, equivariance is typically achieved by imposing equivariance constraints on each layer. Recently, Kaba et al. (2023) propose novel method to incorporate equivariance that no longer requires layer-wise architectural constraints. Their approach introduces canonicalization
functions that learn mappings transforming inputs into canonicalized samples. In this framework, only these canonicalization functions need satisfy certain equivariance constraints incorporate additional symmetries, thereby explicitly decoupling the equivariant and non-equivariant parts of the networks. MMndal et al. (2023) further utilize canonicalizatio functions enable the adaptation of large pre-trained models to be equivariant.
14

---

Published as a conference paper at ICLR 2025
B DETAILED PROOFS
B.I PROOF OF PROPOSITION
Proof Vu FFg G, R2, we have
[ I(x, Lp(g) ou)
II x.
(27)
=I[u[(g- x)
(T (g) (x))
Thus, I satisfies the condition for equivariance, completnn the proof.
B.2 PROOF OF PROPOSITION S
Proof. Vu E Fg G, R2, we have
{(4)) o u](())
= E((x Lg(g) o u))
= p'()) E(g xx u)
(28)
(') E(u](g -x
(Tip' (g) (x).
Hence, E(Lp(9) o ul = o
B.3 PROOF OF THEOREM 7
Proof. Yu F.g E G,x R2, we have
Elg x, Lp(g) u) Mig x, 4p(g) ou) I(g x, Lp(g) u)
=g((g) MM(, I(x,
(29)
p'(g)) E(x, u)
This proves that E is a type- p') equivariant.
B.4 PROOF OF THEOREM TO
Proof. By the definition of the group action Z, we have
g = (g x. ug x). Vug(o x), ... u x)],
(30)
where uy [l( Then we will show that M(g x, Lg(g) u) (g)) M(x, u).
M(g x, 4 o = (p'(alg 2)))
(p'(a(z) g 1))
(p'(o(z)) p0(g p1(1~11)
(31)
p'(g) - (p'(a(2)))
p'()) M(x u).
Thus, M is a tapyeepppppp)) p¹) equivariant matrix.
15

---

Published as a conference paper at ICLR 2025
B.5 PROOF OF THEOREM 12
Proof. First, compute the SupNorm under the group action:
IZ( Lp(g)
uillsup = sup I(x. Lp(g) uu
XE62
sup I(g x, [p(g) ou)
XER2
sup xEE
(s(g)) m I I(x, u
(32)
sup I(x, u) xER²
(s(g))
|I lbap
(s(g))
Next, apply this property to & defined in (16
E(g x, Lp(g) ou)=
E(g x, Lg(g) ou)
ml (II(, (g) ullpp
m2
p((g) E((,
(33)
(s(g))
(+( mm
[I( u)
sup
mg
('(g) E u)
(II(;, u)mllaml m2
(g) E(x, u).
Thus, satisfies the definition of a type- p') equivariant.
B.6 PROOF OF THEOREM 14
Proof. We focus proving the equivariant case. Denote Lp(g) Ug- Vg E G, we have
((p(g) ul =
Lp ( "=))
= (pr Lp (1((((((()))))) Lp(g Lp(g) oul
(34)
= =Lr((g) Lp (1 uu]
Lp) b ul.
Thus, satisfies the equivariance condition.
B.7 PROOF THEOREM 15
Proof. Denote VA Lp. (A) i.., (A A-T. v(A-'x)
Let x* argmaxxeR? det (v(x)) We first show that Ax" argmax [x)((
AA E G and x + Ax* E R., we have A -1 x + x". Then
det((v (x))x = det(AA -T vxA-1x))
det(A - 7) - (det -1x))
(det((A-T)) ( Idet(v(x)))
(35)
-deetA -T v(A -1 -IAx)))
=dxt(v
16

---

Published as a conference paper at ICLR 2025
Thus, Ax" = argmaXxeR* (dc((vA())). Now, for the equivariance of Φ2. have:
22 p2 (A) o ov
2b2(val
vA(Ax")
A .V(A -Axx)xxx)
(36)
-A-T v(x*)
Pc(A) 22lv].
Thus, the proof is complete.
C EXPERIMENTAL DETAILS
Experiment Configuration. In all experiments in Section each training dataset consists of 50,000 images, and each test dataset consists of 10,000 images. All experiments are conducted NVIDIA RTX 3090 GPU.
Prediction Networks. We utilize ResNer50 model pre-trained on the ImageNet- 1k dataset as the prediction network for our experiments. The model is fine-tuned using SGD with a learning rate of 10-S, decay scheduler reduces the learning rate at one-third and one-half of the total epochs, multiplying it by factor of 0.1 each milestone. The batch size for datasets is set 128.
10-1, and momentum of 0.9 for a duration of epochs. The learning rate
Canonicalization Networks. We incorporate EquivarLayers into a ResNet32 architecture, replac- ing the convolutional layers with (ype-(Po, Po) Equivarl Additionally, the output module is substituted with type-(po. Pe + P.) EquivarLayer, followed by Det-Pooling, as described Sec- tion 3] During the training stage for alignment. the canonicalization network is trained by applying randomly generated transformation matrices, corresponding to the evaluation group, to the training images from the MNIST dataset. The model is trained using an AdamW optimizer with learning rate x 10-3 and employs a cosine annealing scheduler for 200 epochs. The batch size for the datasets is set to 128.
D ADDITIONAL EXPERIMENTS
D.1 MEASUREMENT OF COMPUTATIONAL COMPLEXITY
Table 4: The memory usage, FLOPs, and inference time per image.
Input Size
Memory (MB)
FLOPs
Time (ms)
32 32
0.03
6.86 x 10a
1.7
64 x 64
0.12
2.74 x 10s
1.8
128 128
0.50
1.10 106
1.8
256 x 256
2.00
4.39 x 106
1.8
512 x 512
8.00
1.76 x 107
2.8
1024 1024
32.00
7.02 107
8.7
2048 x 2048
128.00
2.81 X 10
32.4
We measure the time and space complexity of type-(po, Pe + Pa) EquivarLayer used in Sec- tion 4.2.I and present the numerical results in Table Following (Li al. (024)). we use torchstat to calculate the memory usage and the FLOPs required for model interence. and perform explicit timing measurements during inference. The results of memory usage and FLOPs indicate that the time and space complexity the model grows linearly with the input image size. For large in- put sizes, the inference time increases proportionally, while for smaller input sizes, it grows more slowly. This behavior is likely due to the underlying parallelism and optimizations in low-level computations, as noted in (Li et al. 2024).
D.2 ADDITIONAL DATASETS
As shown in Tables 6 and we include additional experiments on Fashion-MNIST (iiao et al. 2017] and its transformed variants. Specifically, construct three datasets, Fashion-Gl.
17

---

Published as a conference paper at ICLR 2025
Fashion-Re RS, and Fashion-Scale, by zero- prro-paddin original Fashion-MNEE images to 40 40 and applying corresponding group transformations, The transformation parameters are consistent with those detailed in Section + Besides, we employ the same prediction networks and canonicalization networks in S including ResNet50 with different data augmentation strategies and Equiv- arLayers for three groups. During inference, EquivarLayer is connected to a pre-trained prediction network that uses mild augmentation. ResNet50 is fine-tuned on Aithion-MNITT using SGD with learning rate of 10-3, weight decay of 5 x 10-4, momentum of 0.9, and batch size of 128 for 50 epochs. The learning rate scheduler reduces the rate by a factor of 0.1 at one- third and one-half of the total epochs. The canonicalization network is trained for alignment using an Adam W optimizer with a learning rate of 2 x 10 3 and a cosine annealing scheduler for 400 epochs. The batch size for datasets is set to 128. The experiments are repeated five times and we report mean + std of the test eπrоr.
Tables 5 demonstrate that while full augmentation enhances model's performance on trans- formed datasets, it often sacrifices performance on the original dataset; mild augmentation, on the other hand, may improve performance the original dataset but generally underperforms formed datasets. By combining prediction networks trained with mild augmentation and our Equiv- arLayer canonicalizers, we achieve improved performance on transformed datasets while maintain- ing competitive results the original dataset. Compared full augmentation, this approach de- livers better performance on the original dataset and achieves comparable results on transformed datasets. In Table 7 probably due to the simplicity of scale transformations, full augmentation also performs well on the original dataset. Nevertheless, we can still observe that canonicalization en- hances the performance on transformed datasets for models trained with mild augmentation. It is worth noting that the performance of this approach is closely tied to the capability of the prediction network. Combining powerful pre-trained model with the canonicalizer is a promising strategy, especially since off-the-shelf pre-trained models typically do not undergo full augmentation.
trans-
Table 5: Test error (%) on Fashion and Fashion-Gl. (2)
Method
Fashion
Fashion-Ci GL [LL((2) (2)
Vanilla
2.18
71.83+1.07
Mild Augmentation
5.46
66.35+2.01
Full Augmentation
7.62
8.38
EquivarLayer Canonicalizer (ours)
6.71
8.60
Table 6: Test error (%) on Fashion and Fashion-RS.
Method
Fashion
Fashion-KS
Vanilla
6.5810.19
71.6910.79
Mild Augmentation
5.4640.08
65.82+1.60
Full Augmentation
7.70
8.28.10.04
EquivarLayer Canonicalizer (ours)
6.41+0.17
8.34
Table 7: Test error (%) Fashion and Fashion-Scall
Method
Fashion
Fashion-Scall
Vanilla
6.58
18.09+1.06
Mild Augmentation
5.4610.08
10.2610.42
Full Augmentation
5.36
5.73
Equivarl.ayer Canonicalizer (ours)
5.46
6.84
D.3 ADDITIONAL BASELINES
To provide more comprehensive evaluation, we conduct additional experiments with equivariant networks of the corresponding groups as baselines, including InvarLayers (Li et al. 2024) for the three groups and the scale-equivariant model SESN (Sosnovik et al. 2019), involving various data augmentation strategies. All these baseline models adopt the ResNet50 architecture and are trained on the MNIST dataset for 50 epochs using an AdamW optimizer with a learning rate of x 10-3 and a cosine annealing scheduler. Furthermore, we combine EquivarLayers as canonicalization networks with equivariant baseline models trained using mild augmentation prediction networks. We repeat
18

---

Published as a conference paper at ICLR 2025
each experiment five times and report the mean = std of the test error in Tables 8 and I0 (Aug Augmentation, MA Mild Augmentation, FA Full Augmentation, EquivarCan an EquivarLay Layer Canonicalizer).
Regarding comparisons with equivariant network baselines, emphasize important distinction: while EquivarL and other equivariant network baselines are all equivariant models, they serve different roles in the canonicalization experiments. These approaches are not mutually exclusive; rather, they are complementary and can be effectively combined. These equivariant networks, when used as prediction networks, outperform ResNet50 under the same augmentation settings. Thus, we integrate Equivarl Layers as canonicalizers with equívariant networks as prediction networks, achiev- ing further improved performance.
As shown in Tables and 10 equivariant baseline models with mild augmentation, despite their theoretical equivariance, may exhibit suboptimal performance on transformed datasets. A possible reason is that the large number layers and the presence of several downsampling operations may weaken the equivariance in practice. However, when combined with canonicalization, these models maintain competitive performance on the original dataset while significantly enhancing their perfor- mance on transformed datasets, achieving results comparable to those of full augmentation.
Table 8: Test error (%) on MNIST and MNIST-GL GL + (2).
Model
MNIST
MNIST-GL (2)
InvarLayer w/o Aug
0.72+0.05
41.38+1.28
InvarLayer w/ MA
0.47
36.71+0.63
InvarLayer w/ FA
0.58+0.03
0.87+u.u5
EquivarCan + InvarLayer (ours)
0.63
0.92
Table Test error (%) on MNIST and MNIST-RS.
Model
MNIST
MNIST-RS
InverLayer w/o Aug
0.50
41.36+1.30
InvarLayer w/ MA
0.43
37.91
InvarLayer w/ FA
0.57+
0.75
EquivarCan + InvarLayer (ours)
0.57
0.74+0.01
Table 10: Test error (%) MNIST and MNIST-Scale.
Model
MNIST
MNIST-Scale
SESN w/o Aug
0.43
3..92
SESN w/ MA
0.34
1.03
SESN w/ FA
0.35
0.44
InvarLayer w/o Aug
0.43
4.54
InvarLayer w/ MA
0.37+0.03
1.53+0.11
InvarLayer w/ FA
0.35
0.38
EquivarCan SESN (ours)
0.33++0.05
0.43+
EquivarCan + Invarl Layer (ours)
Inverlaayrr
0.36
0.40
D.4 EOUIVARLAYERS DIFFERENT TYPES
Previously, we focused on the application of EquivarLayer to canonicalization, specifically utilizing type-(pu. Pe + Pc) EquivarLayer. Our proposed framework is general and supports EquivarLay- ers with arbitrary input and output feature types. To validate EquivarLayers of various types, one straightforward approach is to specify the feature types of the network's 's hidden layers, analogous to steerable CNNs in (Weiler& & Cesal 2019). For simplicity,
refer scalar features type-O and vector features as type-1. We implement several three-layer affine equivariant networks as image classifiers, where both the input and output consist of type-0 features, and the hidden layers can in- volve either type-0 type-I features. These models include EquivarLayers that map from type-O to type-0, type-0 to type-1, type-1 to type-0, and type-1 to type-1. We train these networks on MNIST and evaluate them on both MNIST and MNIST-GI.* (2). The results are summarized in Table 11 Models with different types exhibit varying performance. highlighting the flcxibility and expanded design space that Equivarl ayer provides for network architectures.
Layer
19

---

Published as a conference paper at ICLR 2025
Table 11: Test error (%) on MNIST and MNIST-GL**2
Model
Type
MNIST
MNIST-GL."O" (2)
InvarLayer
0000
0.96+0.07
+0.28
EquivarLayer
0100
1.06+0.09
6.5640.73
EquivarLaaeer
0010
1.20
6.54
EquivarLayer
0110
1.44+0.06
6.43+0.22
D.5 EQUIVARLAYERS FOR DYNAMICAL SYSTEM PREDICTION
To further demonstrate the increased generality of our framework, we conduct numerical experi- ments on the prediction of dynamical systems. The goal is use our equivariant network model to predict the evolution a dynamical system governed by a partial differential equation (PDE). Specifically, we consider the PDE: 8,u = Vxu u, where u(t, x) = (w, x), u2 x)) T and x= (I1, (01,,22"= _T2 This PDE deseribes vector field that exhibits affine symmetry. the initial vector field of the dynamical system undergoes an affine transformation. the vector field at each subsequent time step is transformed in the same manner. Following (Wang et al., 2020), we predict the vector field at the next time step based on the past steps of the vector field. Here we set to
For the dataset, we randomly generate functions with specific conditions as the initial values for the PDE and use the Runge- Kuttee method to iteratively compute the evolution of the dynamical system. We retain sequences by sampling every 5th step of iteration and employ rolling window approach to generate subsequences of length 4. where 3 steps are used as input to predict the next step and step serves the ground truth output. From each sequence, we extract 48 such subsequences.
Each input data point has the shape (3, 2, 64, 64), while the output has the shape (1, 2, 64, 64). We generate 2310 sequences with different initial conditions and split the dataset into 5:2 ratio, using 1650 sequences for training (corresponding to 79,200 data points) and 660 sequences for testing (31,680 data points).
We use a three-layer affine equivariant Equivarl Layer, where the input, output, and hidden layers consist of vector features. As a baseline, we employ an SO(2) equivariant steerable CNN (Weiler & Cesa 2019). which also uses vector features in each layer, and has approximately the same num- ber of parameters as EquivarLayer. The models (RMSE) between the forward predictions and the ground truth over all pixels.
evaluated using the Root Mean Square Error
reference for ab- solute values, if the output is simply set to 0, the RMSE is 5231.071 x 10 10-5, 5. As shown in Table I2 Equivarl Layer achieves significantly higher precision, demonstrating its superior performance.
Table 12: RMSE (10 5) on Test Set.
Model
RMSE
Number of Parameters
SO(2) Steerable CNN
6.88 10.150
107 K
EquivarLayer (ours)
0.21840.012
104 K
E ADDITIONAL FIGURES
i
Ouvo Jy
i Uin
Hickkn Layer
hvo ynl
Uest
Ohen Ma
Type: Aus
Type: Pia
Alult
M
AkyJaul
Figure 2: Multi-channel version of the steerable EquivarLayer.
20

---

Published as a conference paper at ICLR 2025
0
0
0
0 1
0
0
0
0
0
0
0
O
1
1
1
1
11
1
NC C 2
2 2
2
2.
2
2
22
22
2
2
2
2
2.
2
2
w 3
8
3
3
3
3
3
3
3 4
33
3
3
3
3
3
3
3 4
4
4
4
41
4
4
4
44
4
4
4
4
4 5
ho5
5
5
5
5
5
5 6
5 6
5
5
5
5
5
16
6
6
9 a 6
6
6
6
6
6
6
6
6
6
4
l
0
6
6
6 7
7 7
7
7
7 8
7
7
7
7
7
7
7
7 8
7 8 9
8
6
8
8
8
6
8
8
8
9
9 9
9
9
9
Transformed Images
Canonicalized Images
Original Images
Figure 3: Visualization of the canonicalization process.
21


Reviews:
[{"review_id": "yby9KypjIu", "reviewer": "Reviewer_8iML", "summary": "In this work, the authors propose a steerable and affine group equivariant layer, EquivarLayer. They introduce a novel pooling technique that can facilitate canonicalization. The empirical experiments presented were conducted on MNIST and MNIST scale datasets.", "questions": "1. Could you present results with additional datasets along with a comparison with existing methods? For example, above mentioned Steerable CNNs as well as ablation with image datasets like CIFAR10/CIFAR100?\n\n\n2. The addition of simple canonization as presented in Kaba et al compared to the proposed method would be helpful. \n\n\n3. How does the proposed method perform to roto-translation and scale? \n\n\n4. How does this method scale with data, dimensions of invariants as well as group size?", "limitations": "", "rating": 5, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "1. This paper presents a novel pooling layer that keeps equivariance to affine groups intact.", "weaknesses": "1. The paper conducts experiments solely on MNIST datasets and skips existing methods like [1] for comparison. \n\n2. The contribution does not seem significant and the experiments section is limited. \n\n3. The motivation for using the proposed approach given the already existing methods that conduct canonicalization, and steerable layers, is unclear to me after going over the paper. \n\n4. Definition 4, Proposition 5 as well as Theorem 7 can be improved in terms of formalizing as well as readability of equations. \n\n[1] Implicit Convolutional Kernels for Steerable CNNs, Zhdanov et al.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "7fBLe71cb0", "reviewer": "Reviewer_uxCw", "summary": "The paper proposes a framework for constructing equivariant layers based on the method of moving frames; under this framework it is shown how the class of input/output group representations that can be employed is larger than previously shown. The authors adapt the jet-space based construction to the case where we are working with an induced representation also acting on the codomain of the signals. The framework is then employed in the context of a canonicalization framework, where the proposed equivariant layer with respect to the affine group is used for the canonicalization function. A primary focus of this work is the affine group, more specifically the affine group of the plane and its subgroups that go beyond euclidean transformations (rotation-scale, $\\textnormal{GL}(2, \\mathbb{R})$). The methodology is evaluated on a set of image classification tasks employing transformed MNIST datasets, where it is contrasted with a standard data augmentation approach. The experimental validation focuses on the delta in performance on transformed/non-transformed test sets.", "questions": "Beyond the issues highlighted before:\n- In the case where the output representation acting on the codomain is not operating on scalar fields, do we have any restrictions with respect to the class of activation functions that can be used as it is the case for steerable methods which operate on vector fields? Do we have to use norm non-linearities?\n- In the appendix it is stated \"Mironenco & Forre (2024) tackles this problem by decomposing a large group into smaller ones and sampling them to enhance sample efficiency. However, it requires sampling on certain measures that may be impractical, such as GL(n)-invariant measure of positive definite matrices.\". I'm not sure I understand what limitation is described here. As far as I understand, sampling from this measure is indeed possible (see e.g. [10]) and the SPD manifold has quite extensive applications.\n- I don't understand how the Det-Pooling ensures a parametrization on the group $\\textnormal{GL}(2, \\mathbb{R})$. It seems we are mapping to the space of $\\mathbb{R}^{2 \\times 2}$ matrices and it is simply assumed that this is a group element? \n- Is there a connection between this work and Olver's work on relative invariants [8]?\n\n[10] Riemannian Gaussian Distributions on the Space of Symmetric Positive Definite Matrices, Said et. al 2015.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "- The paper is generally clear in its presentation, with each mathematical object being well-introduced and described. The authors present a framework which indeed treats a larger class of group representations than seen for this framework.\n- While there has been some recent work in this area, the problem of constructing equivariant/invariant layers with respect to larger transformation groups is generally under-explored and potentially useful. In this sense, I am also happy to see work employing the method of moving frames and the use of differential invariants, which may yield different complexity constraints and/or scaling behaviour on the spectrum of methods imposing inductive biases. \n- Intertwining canonicalization and traditional methods for constructing equivariant layers seems like a useful avenue to explore.", "weaknesses": "I will categorize the issues I find with the paper into three categories: (1) Claimed novelty; (2) Lack of presentation of concrete examples given the level of generality claimed and (3) experimental validation.\nI want to note from the onset that I am open to raising my score, and the degree to which I am penalizing (lack of) novelty here is very much based on what the authors themselves claim.\n\n- (1) The paper uses language that supports the notion where a lot of the concepts are novel (in the sense of new mathematical objects) rather than, their application in this form in the context of machine learning being novel. \n- Considering the framework presented up to section 2.3, I would like to highlight obviously the work of Olver himself in [4] where the same framework of constructing equivariant maps using moving frames based on his previous work is summarized (ending up with the abstract operator of eq. (13)). Directly related is the work of Sangalli et. al which also uses the prolongation of the group action on the jet space for invariantization (based on Olver's work) in the context of image and volumetric data [1-3]. From [1] appendix B, the general framework is directly applicable taking X = $\\mathbb{R}^2$, $Y = \\mathbb{R}^{c}$, $Z = \\mathbb{R}^{c^{'}}$, we arrive at the same principle where an equivariant operator $\\mathcal{F} \\to \\mathcal{F}^{'}$ on the function spaces is defined implicitly once a G-invariant operator on the product space $X \\times Y$ is available ($g \\cdot (x, u) = (gx, \\rho(g)u) = (gx, u)$ ($\\rho = \\rho_0$), corresponding to $g \\cdot f(x) = f(g^{-1}x)$). In this work the notation is less clear and it appears that $Y = \\mathcal{F}$ indicating a higher order operator, however the differential invariants are always (correctly) constructed with the same $x \\in X$ argument i.e. $\\hat{\\mathcal{I}}(f)(x) = I(x, f(x))$ (we make use of differential invariants dependent on $x$). It should be highlighted then that the 'equivariant' $\\mathcal{E}: X \\times Y \\to Z$ presented here is a $G$-equivariant map on the product space respecting $\\mathcal{E} \\circ \\rho(g) = \\rho^{'}(g) \\circ \\mathcal{E}$ , and would correspond to the case where $\\rho \\neq \\rho_0$ ((16) in [1]). This is indeed a different group action and the authors seek to effectively work with induced representations which also acts on the codomain $L_{\\rho}(g)(f)(x) = \\rho(g)f(g^{-1}x)$, however it should not be presented as a departure from this framework if one is simply changing the representation used.\n- The authors should also try and clarify further how their framework differs from [6] in its usage of the sup-normalized differential invariants. Are the polynomial relative invariants of [6] and the relative invariants here the same? I would also highlight [5] and [7] which also deal with differential invariants of the affine group of the plane, where the zero division errors are treated with a different formulation (see [5] (34) and (35)).\n- Regarding (2), I would have liked to see more concrete examples of the framework being employed in the context where the additional flexibility in terms of input/output representations is valuable e.g. beyond image data, see for example [9]. I am not concerned with SOTA results, simply a validation that the increased generality allows the use of the differential invariants for a larger set of applications and modalities. It would also be useful for practitioners if more concrete examples of the realized (beyond theoretical) operators is presented.\n- Regarding (3), it would be useful to quantify more clearly the relationship between the amount of data augmentation used and the size of the dataset for each experiment. More importantly, what would be very useful here is to have a thorough presentation of the time/space complexity of each layer employing this framework, see e.g. [1] appendix D or [6], with the inclusion of standard (3-channel) image data.\n\n[1] \"Moving frame net: SE(3)-equivariant network for volumes\", Sangalli et. al, 2023\n[2] \"Differential invariants for SE(2)-equivariant networks\", Sangalli et. al, 2023\n[3] \"Equivariant Deep Learning Based on Scale-Spaces and Moving Frames\", Sangalli 2023\n[4] \"Using Moving Frames to Construct Equivariant Maps\", Peter Olvar (March 2024)\n[5] \"Affine Differential Invariants for Invariant Feature Point Detection\", Tuznik et. al 2018\n[6] \"Affine Equivariant Networks Based on Differential Invariants\", Li et. al, 2024\n[7] \"Affine invariant detection: edge maps, anisotropic diffusion, and active contours\", Olver et.al 1999\n[8] \"On Relative Invariants\", Fels & Olver 1997\n[9] \"Generalizing Convolutional Neural Networks for Equivariance to Lie Groups on Arbitrary Continuous Data\" Finzi et. al 2020", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "m3S9wwRaRW", "reviewer": "Reviewer_sBuw", "summary": "This paper extends the InvarLayer (Li et al., 2024), which uses differential invariants to achieve affine equivariance, by proposing the EquivarLayer. EquivarLayer combines a novel equivariant matrix with differential invariants to generalize from invariance to equivariance. The paper also introduces Det-pooling, which enables EquivarLayer to achieve canonicalization. This canonicalization capability is assessed by using the EquivarLayer model to canonicalize transformed MNIST images, followed by classification with a ResNet-50. The results show improved performance compared to ResNet-50 with data augmentation.", "questions": "1. Can this approach be extended to handle reflections, specifically for cases where det(A) < 0?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "1. This paper introduces a novel and efficient method for extending the affine-invariant layer to an affine steerable equivariant layer. Achieving this in deep learning is challenging, as the affine group has six degrees of freedom, and sampling from this group often incurs prohibitively high computational costs.\n2. The proposed ResNet-32 model contains only 0.4 million parameters, which is impressive. This compact model size may enable the exploration of very deep equivariant networks.", "weaknesses": "1. Experiment Design: Using invariant classification to demonstrate the effectiveness of the proposed EquivarLayer for canonicalization is somewhat indirect. The true strengths or limitations of EquivarLayer are obscured by the downstream ResNet-50 prediction network, yet five pages are dedicated to deriving the EquivarLayer. Since canonicalization is a downstream task, its accuracy depends significantly on the model’s ability to preserve information—a quality typically evaluated by learning equivariant features, performing invariant classification, or computing equivariant error. Key questions remain, such as whether EquivarLayer improves upon InvarLayer (as it is a generalization) and how it compares to other steerable methods achieving subgroups of affine group equivariance.\n2. Experiment Setup and Comparability: The experiment setup limits comparability with other methods. The train/test split is not provided, and the scale factor range (0.8 to 1.2 or 1.6) differs from that of other scale-equivariant papers, which often use a range of [0.3, 1]. Additionally, key scale-equivariant works, such as [1], are not cited. These omissions make it challenging to compare this model with existing equivariant CNNs, especially since no equivariant network baselines are included in the experimental section.\n3. Clarity in Derivations: The derivations following Eq. (16) in Section 2.3 are unclear. The notation for symbols like $u_x, u_y$ is not adequately defined, and it was only by referring to Li et al. (2024) that I understood they represent gradients. While this is a minor issue, a more significant concern is the lack of discussion around $\\alpha$ from Eq. (14). Instead, the paper provides an example of an equivariant matrix (Eq. 17) without explaining the derivation of $\\alpha$.\nAdditionally, one of the advantages of equivariance is its ability to preserve information. While the proposed method is computationally efficient, it may sacrifice some information, as it limits gradients to second order and relies on differential invariants for local features. It remains unclear if the gradient order relates to information preservation. The derivations also do not clarify why only first- and second-order gradients are used or how the equivariant matrix is derived.\n\n[1] Sosnovik, Ivan, Michał Szmaja, and Arnold Smeulders. \"Scale-Equivariant Steerable Networks.\" International Conference on Learning Representations.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "JdEUePBqjq", "reviewer": "Reviewer_5XxQ", "summary": "In summary, this paper contributes the following:\n- An extension of the equivariant steerable layers for the 2D affine group proposed in [1] from type-0 (scalar-type) feature maps (in the terminology of [2, 3]) to other types utilizing equivariant moving frames [4], specifically by linear combination of the equivariants produced by frames using type-0 features as weights,\n- An adaptation of the above layer for invariant or equivariant canonicalization [5] of non-equivariant backbones for the 2D affine group, using global max-pooling of 2-channel type-1 feature with respect to an invariant quantity (absolute determinant),\n- Experiments on invariant canonicalization of a pre-trained ResNet50 backbone for MNIST image classification under affine, rotation-scaling, and scaling transformations.\n\n[1] Li et al. Affine equivariant networks based on differential invariants (2024)\n\n[2] Cohen and Welling, Steerable CNNs (2017)\n\n[3] Thomas et al. Tensor field networks: Rotation- and translation-equivariant neural networks for 3D point clouds (2018)\n\n[4] Olver, Modern developments in the theory and applications of moving frames (2015)\n\n[5] Kaba et al. Equivariance with learned canonicalization functions (2022)", "questions": "- Q1. Is it necessary that equivariant matrices are square matrices, as implied in Definition 6? As far as I understand, it should be that their number of columns can be flexible, while the number of rows is specified by the representation \\rho'.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "- S1. The use of equivariant moving frames to extend type-0 features to other types is technically sound, and original as far as I am aware in the context of steerable networks. (In a broader context, the authors may find [6] and its application to Lorentz canonicalization [7] related, as they also linearly combine equivariants using learned invariant features.)\n- S2. The considered application for canonicalization using global invariant pooling is technically sound as far as I can tell, and properly shows the utility of the proposed layer since canonicalization [5] only using type-0 features is not straightforward.\n- S3. The experiments show that the approach is able to perform canonicalization of a non-equivariant backbone for invariant MNIST classification under affine transformations, which has been unavailable in current literature before as far as I am aware.\n\n[5] Kaba et al. Equivariance with learned canonicalization functions (2022)\n\n[6] Villar et al. Scalars are universal: Equivariant machine learning, structured like classical physics (2021)\n\n[7] Nguyen et al. Learning symmetrization for equivariance with orbit distance minimization (2023)", "weaknesses": "- W1. A proper comparison against fully equivariant networks for classification (e.g., InvarPDEs-Net and InvarLayer from [1]) is not included in the experiments. Including these baselines would strengthen the paper, especially since canonicalization may benefit from pre-training in non-invariant domains (e.g., ImageNet-1k in this paper) unlike fully equivariant networks.\n- W2. It seems possible to use the moving frame in Definition 8 (or the equivariant matrix in Theorem 10) directly for canonicalization without multiplication by type-0 features (Eq. (10)), as this coincides with the definition of equivariant frames in frame averaging [8]. Adding this as a baseline would strengthen the results, as it can be understood as ablating trainability from canonicalization.\n- W3. It was unclear to me how the definition and property of moving frames given in Definition 8 and Theorem 9 logically leads to the construction of equivariants using relative equivariants given in Section 2.3.\n- W4. The paper demonstrates a single application of the proposed layer as the last layer of a canonicalization network, which extracts 2-channel type-1 features from type-0 features, while all other layers of the network map between type-0 features [1]. This misses (1) feature maps of type >1, (2) layers taking non-scalar features as input, and (3) layers mapping between non-scalar features, all of which are possible within the proposed framework. Testing (a subset) of these would strengthen the paper. A straightforward way is to modify InvarPDEs-Net and InvarLayer from [1] to have non-scalar hidden features, analogous to steerable networks in [2].\n\nMinor typo\n- Line 13: InvarLayer -> InvarLayer (Li et al. 2024)\n\n[1] Li et al. Affine equivariant networks based on differential invariants (2024)\n\n[2] Cohen and Welling, Steerable CNNs (2017)\n\n[8] Puny et al. Frame averaging for invariant and equivariant network design (2021)", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)