Steerable Partial Differential Operators for Equivariant Neural Networks
Authors: Erik Jenner
Maurice Weiler
Venue: ICLR 2022 Poster (2022)
Keywords: partial differential operators
equivariance
deep learning
steerability

Abstract: Recent work in equivariant deep learning bears strong similarities to physics. Fields over a base space are fundamental entities in both subjects, as are equivariant maps between these fields. In deep learning, however, these maps are usually defined by convolutions with a kernel, whereas they are partial differential operators (PDOs) in physics. Developing the theory of equivariant PDOs in the context of deep learning could bring these subjects even closer together and lead to a stronger flow of ideas. In this work, we derive a $G$-steerability constraint that completely characterizes when a PDO between feature vector fields is equivariant, for arbitrary symmetry groups $G$. We then fully solve this constraint for several important groups. We use our solutions as equivariant drop-in replacements for convolutional layers and benchmark them in that role. Finally, we develop a framework for equivariant maps based on Schwartz distributions that unifies classical convolutions and differential operators and gives insight about the relation between the two.


Published as a conference paper at ICLR 2022
STEERABLE PARTIAL DIFFERENTIAL OPERATORS FOR EQUIVARIANT NEURAL NETWORKS
Erik Jenner*
Maurice Weiler
University of Amsterdam er 2rikkejenner.com
University of Amsterdam
m. we weeer... ml mal@gmail. com
ABSTRACT
Recent work in equivariant deep learning bears strong similarities to physics. Fields over a base space are fundamental entities in both subjects, are equivariant maps between these fields. In deep learning, however, these maps are usually defined by convolutions with a kernel, whereas they are partial differential operators (PDOs) physics. Developing the theory of equivariant PDOs in the context of deep learning could bring these subjects even closer together and lead stronger flow of ideas. In this work, we derive a G-steerabilityy constraint that completely characterizes when PDO between feature vector fields is equivariant, for arbitrary symmetry groups G. We then fully solve this constraint for several important groups. We use our solutions as equivariant drop-in replacements for convolutional layers and benchmark them in that role. Finally, develop framework for equivariant maps based on Schwartz distributions that unifies classical convolutions and differential operators and gives insight about the relation between the two.
div + curl
(0 + og
By
858, + og + 0.)
Figure I: A vector field (left) be mapped to scalar field (right) by applying certain partial diffcrential operators (PDOs), such scalar field can be represented Laplacian of the divergence and 2D curl. Such PDO from vector to x matrix, where each of two entries a one-dimensional PDO that acts on one of the two components of the vector field, Similarly, matrices of PDOs with different dimensions map between other types of fields. Our goal is to find all PDOs for which this map becomes equivariant, for arbitrary types of fields. For the implementation, will later discretize PDOs stencils (middle).
INTRODUCTION
In many machine learning tasks, the data exhibits certain symmetries, such as translation- and sometimes onaaion--ivvriiane in image classification. To exploit those symmetries, equivariant neural networks have been widely studied and successfully applied in the past years, beginning with Group convolutional neural networks (Cohen & Welling 2016; Weiler et al. 8b). A significant generalization of Group convolutional networks given by steerable CNNs (Cohen & Welling 2017] Weiler et al. 2018a Weiler Cesa 2019), which unify many different pre-existing equivariant models (Weiler& & Cesal 2019). They do this by representing features as fields of feature vectors over a base space, such as I in the case of two-dimensional images. Layers are then linear equivariant maps between these fields.
This is very reminiscent of physics. There, fields are used to model particles and their interactions, with physical space spacetime the base space. The maps between these fields also equivariant, with the symmetries being part of fundamental physical laws.
Work done during intersship QUVA Lab

---

Published as a conference paper at ICLR 2022
It is also noteworthy that these symmetries are largely ones that appear the most often in deep learning, such as translation and rotation equivariance. These similarities have already led to ideas from physics being applied in equivariant deep learning (Lang & Weiler 202T
However, one remaining difference is that physics uses equivariant partial differential operators (PDOs) define maps between fields, such the gradient Laplacian. Therefore, using PDOs instead of convolutions in deep learning would complete the analogy to physics and could lead to even more transfer of ideas between subjects.
Equivariant PDO-based networks have already been designed in prior work (Shen et al. 2020 mmets al. 2020 Sharp et al., 2020). Most relevant for our work are PDO-eConvs (Shen al. 2020p, which can be seen as the -Doalnalooon of group convolutions. However, PDO-eConvs are only one instance equivariant PDOs and do not cover the most common PDOs from physics, such as the gradient, divergence, etc. Very similarly to how steerable CNNs (Cohen & Welling, 2017 Weiler jtt al., 20T8a; Weiler & Cesa 2019) generalize group convolutions, we generalize PDO-eConvs by characterizing the set of all translation equivariant PDOs between feature fields over Euclidean spaces. Because of this analogy, we dub these equivariant differential operators steerable PDOs.
These steerable PDOs and their similarity to steerable CNNs also raise the question of how equivariant PDOs and kernels relate to each other, and whether they can be unified. We present framework for equivariant maps that contains both stecrable PDOs and convolutions with stecrable kernels special cases. We then prove that this framework defines the most general set of translation equivariant, linear, continuous maps between feature fields, complementing recent work (Aronsson 2021) that describes when equivariant maps are convolutions. Since formally developing this framework requires the theory of Schwartz distributions, we cover it mainly in Appendix E and the main paper can be read without any knowledge of distributions. However, we reference the main results from this framework in the paper where appropriate.
In order to make stecrable PDOs practically applicable, we describe an approach to find complete bases for vector spaces of equivariant PDOs and then apply this method the most important cases. We have also implemented steerable PDOs for all subgroups of ((2) battps://ort withubbbuu com/ ejnnr Stnnr/steeralee patpaalleepppp._ee_eepe_pp__ Our code extends the E2CNN library I (Wileer & Cesa 2019), which will allow practitioners to easily use both steerable kernels and steerable PDOs within the same library, and even to combine both inside the same network. Finally, we test our approach empirically by comparing steerable PDOs to steerable CNNs. In particular, we benchmark different discretization methods for the numerical implementation.
In summary, main contributions are follows:
We develop the theory of equivariant PDOs on Euclidean spaces, giving a practical charac- terization of precisely when PDO is equivariant under any given symmetry.
We unify equivariant PDOs and kernels into one framework that provably contains all translation equivariant, linear, continuous maps between feature spaces.
We describe a method for finding bases of the vector spaces of equivariant PDOs, and provide explicit bases for many important cases.
We benchmark steerable PDOs using different discretization procedures and provide an implementation steerable PDOs as an extension of the E2CNN library.
1.1 RELATED WORK
Equivariant convolutional networks Our approach to equivariance follows the one taken by steerable CNNs (Cohen & Welling 2017; Weiler et al. 2018a Weiler & Cesa 2019 Brandstetter al., 2021). They represent each feature as map from the base space, such as R a fiber R that is equipped with representation of the point group G. Compared to vanilla CNNs, which have fiber R, steerable CNNs thus extend the codomain of feature maps.
A different approach is taken by group convolutional networks (Cohen & Welling 2016 Hoogeboom ett al. 2018 Weiler et al. 2018h). They represent each feature as map from a group H acting on the input space to R. Because the input to the network usually does not lie in H. this requires a lifting
https::/quvaa 1ab. www.b.gtttau thub. i0/e2cnn/
2

---

Published as a conference paper at ICLR 2022
map from the imput space to H. Compared vanilla CNNs, group convolutional networks can thus be understood as extending the domain of feature maps.
When H Fd x G' is the semidirect product of the translation group and pointwise group G, then group convolutions on H are equivalent to G-steerable convolutions with regular representations. For finite G. the group convolution over G simply becomes finite sum. LieConvs (Finzi al.) 2020) describe a way of implementing group convolutions even for infinite groups by using a Monte Carlo approximation for the convolution integral. Steerable CNNs with regular representations would have to use similar approximations for infinite groups, but they can instead also use (non-regular) fiite-dimensional representations. Both the group convolutional and the steerable approach can be applied to non- non-Euclideaa input spaces- LicConvs define group convolutions on arbitrary Lie groups and steerable convolutions can be defined on Riemannian manifolds (Cohen et al. 2019b Weiler et al. 202T and homogeneous spaces (Cohen et al. 2019a).
One practical advantage of the group convolutional approach employed by LieConvs is that doesn't require solving any equivariance constraints, which tends to make implementation of new groups easier. They also require somewhat less heavy theoretical machinery. On the other hand, steerable CNNs are much more general. This makes them interesting from a theoretical angle and also has more practical advantages; for example, they can naturally represent the symmetries of vector field input output. Since our focus is developing the theory of equivariant PDOs and the connection to physics, where vector fields are ubiquitous, we are taking the steerable perspective in this paper.
Equivariant PDO-based networks The work most closely related ours PDO-eConvs (Shen et al. 2020), which apply the group convolutional perspective to PDOs. Unlike LieConvs, they are not designed work with infinite groups The steerable PDOs we introduce generalizes PDO-eConvs, which are obtained as a special case by using regular representations.
different approach equivariant PDO-based networks taken by Smets et al. (2020). Instead of applying differential operator to input features, they use layers that map an initial condition for PDE its solution at fixed later time. The PDE has a fixed form but several Iearnable parameters and constraints on these
combineed with the form of the PDE- PDEE-uaranted guarantee equivariance. Sharr et al. (2020) also use a PDE, namely the diffusion equation, as part of their DiffusionNet model, which can learn on 3D surfaces. Interestingly, the time evolution operator for the diffusion equation is exp((A), which can be interpreted as an infinite power series in the Laplacian, very reminiscent of the finite Laplacian polynomials that naturally appear throughout this paper. Studying the equivariance of such infinite series of PDOs might be an interesting direction for future work. We clarify the relation and kernel-based networks in some more detail in Appendix F
between
2 STEERABLE PDOS
In this section, we develop the theory of equiv- ariant PDOs. We will represent all features as smooth fields f : Rd + R that asso- ciate feature vector f(x) IR called the fiber at x, with each point x E Rd. We write ==0 (Rd Ro:) for the space of these fields I in layer i. Additionally, we have a group of transformations acting on the input space Rd. which describes under which symmetries we want the PDOs to be equivariant. We will al- ways use a group of the form H = (Rd +) G, for some G S (d, R). Here, (Rd (R) refers to the group of translations of Rd, while G is some group of linear invertible transformations. The full group of symmetries H is the semidirect product of these two, meaning that each element E H can be uniquely written as h = tg. where E Rd is a translation and g E G a linear transformation. For example, if G {e) } is the trivial group, we consider only equivariance der translations, as in classical CNNs, while for = SO(d) we additionally consider rotational equivariance.
8
1-8
g-v /
g
g
scalar field p(g) = vector field p(g)
Figure 2: Transformation of scalar and vector fields (re-
produced with permission from Weiler & Cesa 2019)
3

---

Published as a conference paper at ICLR 2022
Each feature space F; has associated group representation p; G + GL(C,, R), which determines how each fiber I transforms under transformations of the imput space. Briefly, P; associates an invertible matrix (g(g) to each group element g. such that (;(g)
p; (gg''); ); more details on representation theory can be found in Appendix [B] To see why these representations are necessary, consider the feature space F c R2) and the group G (O) The channels could simply be two independent scalar fields, meaning that rotations of the input move each fiber but do not transform the fibers themselves. Formally, this would mean using trivial representations p(g) 1 for both channels. On the other hand, the two channels could together form a vector field, which means that each fiber would need to itself be rotated in addition to being moved. This would correspond to the representation p(g) y. These two cases are visualized in Fig. 2
In general, the transformation of feature f E Fi under an input transformation bg with E Rd and g E G' is given by
((gg P. (z) == ( (00-t)) -))
The 9 - t) t) term moves each fiber spatially, whereas the pi responsible for the individual transformation of each fiber.
For network, we will need maps between adjacent feature spaces F, and Fi+1. Since during this theory section, we only consider single layers in isolation, we will drop the index i and simply denote the layer map as & Fi - Fout- We are particularly interested in equivariant maps Φ, i.e. maps that commute with the action of H on the feature spaces:
(h in ʃ) h Dout $()) 4A E H,J Fi
We call t raanslation-equivaiin gously, & is G-equivariant if it holds for linear transformations h E G. Because H is the semidirect product of I&d and G, map is H-equivariant and only if it is both translation- and G-equivariant.
Eq. 2 holds for E (RR +), ) i.e. for pure translations. Analo-
2.1 PDOS AS MAPS BETWEEN FEATURE SPACES
We want to use PDOs for the layer map o. so we need to introduce some notation for PDOs between multi-dimensional feature fields. As shown in Fig. ! such a multi-dimmenional PDO can be interpreted as matrix of one-dimensional PDOs. Specifically, PDO from C (R4, R to C (RR, R is described by a Cour X en matrix. For example, the 2D divergence operator, which maps from IR= to R = IR' can be written the x 2 matrix (0, 02). This is exactly analogous to convolutional kernels, which can also be interpreted as Cour X Cin matrices of scalar- valued kernels.
scalar-valued
To work with the one-dimensional PDOs that make up the entries of this matrix, we use multi-index notation, so for a tuple a = (a1...., ata) E Ný. we write d := 8 ...8I 88" A general one- dimensional PDO is a sum Σ,, ca," where the coefficients ca are smooth functions ca : Rd > R (so for now spatial weight sharing is assumed). The sum ranges over all multi-indices o, but we require all but a finite number of coefficients to be zero everywhere, so the sum is effectively finite. As described, a PDO between general feature spaces is then matrix of these one-dimensional PDOs.
2.2 EQUIVARIANCE CONSTRAINT FOR PDOS
We now derive complete characterization of the PDOs that are H-equivariant in the sense defined by Eq. (2). Because a map is equivariant under the full symmetries H = (Rd,, +) x G if and only if it is both translation equivariant and G-equivariant, we split up our treatment into these two requirements.
First, we note that translation equivariance corresponds to spatial weight sharing, just like in CNNs (see Appendix G for the proof):
Proposition 1. A Caut Ca-PDOO & with matrix entries
Eded 30 is translation equivariant if and only if all coefficients eq are constants, i.e. edi cdd for all x, E Rd.
So from now on, we restrict our attention PDOs with constant coefficients and ask under which circumstances they are additionally equivariant under the action of the point group G S GL, IR). To answer that, make use of duality between polynomials and PDOs that will appear throughout this paper: for PDO Ea Oo with Co E IR there is an associated polynomial simply given by
Note that we had c, E 2 (Rd R) before. but we now restrict ourselves to constant coefficients and identify constant functions Rd + R with real numbers.
4

---

Published as a conference paper at ICLR 2022
Eacart where x" == 21 I..Id' Conversely, for any polynomial p = En Ca get PDO by formally plugging in å = (ō,..... Oa) for x, yielding p(0) := Ea Co 0n We will denote the map in this direction by D.
write D(p) == p(0).
We can extend this duality to PDOs between culli-dimmenional fields: for a matrix P of polynomials, so (PP) will be matrix of PDOs given by D(P), == ((P,)). To avoid confusion, we will always denote polynomials by lowercase letters, such as q. and matrices of polynomials by uppercase letters, like P and Q.
define (P) component
As a simple example of the correspondence between polynomials and PDOs, the Laplacian operator A is given by Á ([x=["). The gradient (i)...., 0a)² is induced by the d x 1 matrix (#1...., Fd)) and the 3D curl (d = 3) is induced by the 3 x 3 matrix
0
33
J2
0g
02
P =
33
0
T1
D(P) =
0g
0
(3)
22
T1
0
02
0
Finally, note that D is a ring isomorphism, i.c. bijection with D(p q) = D(p) D(g) and D(pq)= = D(p) ((g), allowing us to switch viewpoints at will.
We ready state main result on the G-equivariance of PDOs:
Theorem 2. For any matrix of polynomials P, the differential operator D(P) is G-equivariant if and only if it satisfies the PDO G-steerability constraint,
P P(gg = Powa P())) GgEG,
(4)
P(x) just means that x is plugged into the polynomials in each entry of P. which results in real-valued matrix P(x) E IG and similarly for P ((g ())))) Appendix C provides more intuition for the action of group elements on polynomials. Because D is an isomorphism, this result completely characterizes when a PDO with constant coefficients G-equivariant, and in conjunction with Proposition D when any PDO between feature fields is H-equivariant, The proof of Theorem can again be found in Appendix G
To avoid confusion, we would like to point out that this description of PDOs as polynomials is only a useful trick that lets us express certain operations more easily and will later let us connect steerable PDOs to steerable kernels. Convolving with these polynomials is not meaningful operation; they only become useful when 0 is plugged in for x.
2.3 EXAMPLES OF EQUIVARIANT PDOS
To build intuition, we explore the space of equivariant PDOs for two simple cases before covering the general solution. Consider IR2 as a base space with the symmetry group G = SO(2) and trivial representations Pin and Pout= i.c. maps between two scalar fields. The equivariance condition then becomes p(gr)= = p(z), so the polynomial has to be rotation invariant. This is the case if and only if it can be written as function of | i.e. p(z) <(<x(²) for some q R20 + R. Since we want be polynomial, g needs to be polynomial one variable. Because | = . + x. we get the PDOs
PDO ((p)= = (()), where A == 0f 0 is the Laplace operator. So the between two scalar fields are exactly the polynomials in the Laplacian, such as 24 + +++3.
As second example, consider PDOs that map from a vector to a scalar field, still with SO(2) as the symmetry group. There are two such PDOs that often occur the natural sciences, namely the divergence, div e := 0ivi + O202, and the 2D curl, curl2n e == 01 O27n Both of these are (quuiiaquiaa (see Appendix A). We get additional equivariant PDOs by composing with equivariant scalar-to-scalas maps, i.c. polynomials in the Laplacian. Specifically, 9(A) div and ((A) a curlæD are also equivariant vector-to-scalar PDOs, for any polynomial q. We will omit the from now
We show in Appendix A that these PDOs already span the complete space of SO(2) ( equivariant PDOs from vector to scalar fields. Explicitly, the equivariant PDOs in this setting are all of the form qi (A) div + 92( (2(AA) curlpp for polynomials qı and 92. One example of this is the PDO shown in Fig.1
5

---

Published as a conference paper at ICLR 2022
In these examples, well as in other simple cases (see Appendix Aj. the equivariant PDOs all combinations well-known operators such as the divergence and Laplacian. That the notion of equivariance reproduce all the intuitively "natural" differential operators suggests that it captures the right concept.
BASES FOR SPACES OF STEERABLE PDOS
For the purposes of deep learning. we need be able to learn steerable PDOs. To illustrate how to achieve this, consider again -eqquivrrai PDOs mapping from vector to scalar field. We have seen in Section 2.3 that they all have the form
Qi (4) div + Q2 (A) curl2D
(5)
In practice, we will need to limit the order of the PDOs we consider, so that we can discretize them. For example, we could consider only polynomials 9i and 9 of up to first order, i.c. q1 (2) = c1 + C2 and (2 cy2 C1. This leads to PDOs of up to order three (since A is second order PDO and div and curl2p are first order). The space of such equivariant PDOs is then
e A div + 02 div + C3 A curl2D + curl2D,
G E lR.
(6)
We now train the real-valued parameters c4 and thereby learn arbitrary equivariant PDOs of up to order three.
The general principle is that we need to find basis of the real vector space of steerable PDOs; then we learn weights for a linear combination of the basis elements, yielding arbitrary equivariant PDOs.
Different group representations popular in equivariant deep learning practice; for example PDO- eConvs and group convolutions correspond to so-called regular representations (see Appendix MM but quotient representations have also been used very successfully (Weiler & Cesal 2019). We therefore want to find bases of steerable PDOs for arbitrary representations Pin and Pout To do so, we make use of the existing work on solving the closely related G-steerability constraint for kernels. In this section, we will first give a brief overview of this kernel steerability constraint and then describe how transfer its solutions to ones of the PDO steerability constraint.
3.1 THE STEERABILITY CONSTRAINT FOR KERNELS
The kernel G-steerability constraint characterizes when convolution with kernel Rd Re G-equivariant, like the PDO steerability constraint Eq. 44 does for PDOs. Namely,
<(gz) = |det gl (oun(g) (z))
Vg E G
has to hold. This constraint was proven in /Weiler et al. 2018a) for orthogonal G and later in (Weiler et al. 2021) for general G. It is very similar to the PDO steerability constraint: the only differences are the determinant and the k(gz) term on the LHS where we had (g (g-1) instead of g.
An explanation for this similarity is that both constraints be as special cases of more general steerability constraint for Schwartz distributions, which we derive in Appendix F Schwartz distributions are generalization of classical functions and convolutions with Schwartz distributions can represent both classical convolutions and PDOs. As we prove in Appendix D such distributional convolutions are the most general translation equivariant, continuous, linear maps between feature spaces, strictly more general than either PDOs or classical kernels. We also show how steerable PDOs can be interpreted as the Fourier transform of steerable kernels, which we use to explain the remaining differences between the two steerability constraints.
But for the purposes of this section, we want to draw particular attention to the fact that for G < ((d), the two constraints become exactly identical: the determinant then becomes 1, and 11- g. So for this case, which is by far the most practically important one, we will use existing solutions the kernel steerability constraint to find complete basis of equivariant PDOs.
3.2 TRANSFERRING KERNEL SOLUTIONS TO STEERABLE PDO BASES
Solutions of the kernel steerability constraint have been published for subgroups of ((2) (ang & Weiler 2021 Weiler & Cesal 2019) and 0(3) (Lang & Weiler 2021 Weiler et al. 2018a), and they

---

Published as a conference paper at ICLR 2022
all use basis of the form
(Koa(T) P (1=))x (x/[²)] A,
(8)
where A and are index sets for the radial and angular part and the (a span the entire space of radial functions. The reason is that the steerability constraint Eq. constrains only the angular part if G S (d), because orthogonal groups preserve distances.
The angular functions X are only defined on the sphere 84-1, But crucially, we show in Appendix J that they can all be canonically extended to polynomials defined on R Concretely, we define xo(x) | lo X ( [x=)z where lg is chosen minimally such that X. is polynomial. What prove in Appendix D is that such an la always exists.
The radial part Sa in the kernel basis consists of unrestricted radial functions. To get basis for the space of polynomial steerable kernels, it is enough use only powers of |ar] for the radial part. Specifically, we show in Appendix that
Pkg = |002* X3 lo E No. 3 E B)
a basis for the space of polynomial steerable kernels. As final step, we interpret each polynomial as a PDO using the isomorphism 1) defined in Section 2.2 yielding a complete basis of the space of steerable PDOs. The |x|² terms become Laplacian powers A while in the examples from Section 2.3 Xa corresponds to divergence and curl.
In Appendix I
we apply this procedure to subgroups of O(2) and 0(3) to obtain concrete solutions.
4 EXPERIMENTS
Implementation We developed the theory of steerable PDOs a continuous setting, but for imple- mentation the PDOs need be discretized, just like steerable kernels. The method of discretization completely independent of the steerable PDO basis, so steerable PDOs can be combined with any discretization procedure. We compare three methods, differences (FD), radial basis function finite differences (RBF-FD) and Gaussian derivatives.
Finite differences are generalization of the usual central difference approximation and are the method used by PDO-eConvs (Shen et al./ 2020). RBF-FD finds stencils by demanding that the discretization should become exact when applied radial basis functions placed on the stencil points. Its advantage over FD is that it can be applied to structureless point clouds rather than only to regular grids. Gaussian derivative stencils work by placing a Gaussian on the target point and then evaluating its derivative on the stencil points. Like RBF-FD, this also works on point clouds, and the Gaussian also has a slight smoothing effect, which why this discretization is often used in Computer Vision. Formal descriptions of all three discretization methods can be found in Appendix D
In addition to discretization, the infinite basis of steerable PDOs or kernels needs to be restricted to a
finite subspace. For kernels, use the bandlimiting filters by Weiler & Cesal (019). For PDOs, we limit the total derivative order to two for 3 x stencils and to three for 5 x stencils (except for PDO-eConvs, where
the original basis that limits the maximum order of partial derivatives).
Finally, steerable PDOs and stecrable kernels only replace the convolutional layers in a classical CNN. To achieve equivariant network, all the other layers, such nonlincarities Batchnorm also need to be equivariant. WWiler& & Cesal (2019) discuss in details how this can be achieved for exactly the same implementation they do. Care also needs to be taken with biases in the PDO layers. Here, we again follow (Weiler& & Cesa 2019 by adding bias only to the trivial irreducible representations that make up Pout-
various types of layers. In our experiments,
"This holds for the cases we discuss here, i.e. subgroups of O(2) and 0(3). In higher dimensions, the situation is more complicated, but for those the kernel solutions have not yet been worked out anyway. "s in the original PDO-eConv paper (Shen et al. 2020). Note that their performance is better, which is simply caused by their different architecture and hyperparameters.
7

---

Published as a conference paper at ICLR 2022
Rotated MNIST We first bench- mark steerable PDOs on rotated MNIST (Larochelle et al. 2007). which consists of MNIST images that have been rotated by different angles, with 12k train and 50k test images. Our results can be found in Table I The models with 5 x 5 stencils use an architecture that Weiler & Cesa (2019) used for steerable CNNs, with six Cis-equivariant layers followed by two fully connected layers. The first column gives the representation under which the six equivariant layers transform (see Appendix B for their definitions). PDO-eConvs implicitly representations (see Appendix M but with a slightly different basis than the one we present, so we test both bases. We also tested models that are Dis-cquivariant in their first layers and Cıe-equivariant in their last one but did not find any improvements, see Appendix N For the models with 3 x 3 stencils, we use eight instead of six Cqe-equivariant layers, in order to compensate for the smaller receptive field and keep the parameter count comparable. The remaining differences between kernel and PDO parameter counts come from the fact that the basis restrictions necessarily work slightly differently (via bandlimiting filters or derivative order restriction respectively). All models were trained with 30 epochs and hyperparameters based on those by WWileer & Cesa (2019), though we changed the learning rate schedule and regularization slightly because this improved performance for all models, including kernel-based ones. The training data is augmented with random rotations. Precise descriptions of the architecture and hyperparameters can be found Appendix O
Table MNIST-rot results. Test errors + standard deviations are averaged over six runs. Vanilla CNN is solely translation equivariant model (G = {(e) with the same general architecture. See main text for details the models.
Represen- Method tation
Stencil
Error [%]
Params
Vanilla CNN
3 x
2.001 + 0.030
1.IM
x
1.959 + 0.055
3 3
0.741 + 0.036
837K
Kernels
5
0.683 + 0.021
1.IM
3 x 3
1.196 + 0.062
837K
regular (our basis)
FD
x
1.54 0.32
941K
3 x
1.313 1 0.065
837K
RBF-FD
regular
5 x
1.475 + 0.020
941K
3 3
0.795 + 0.030
837K
Gauss
5 5
0.750 + 0.017
941K
regular (PDO- eConv)
FD
5 x
1.98
+ 0.11
982K
Gauss
5 x
0.831 + 0.039
3 x 3
0.717 + 0.026
877K
Kernels
5 5
0.670 + 0.011
1.1M
3 x 3
1.143 + 0.063
877K
quotient (our basis)
5 x 5
1.347 + 0.026
951K
3 x 3
1.303 1 0.077
877K
RBF-FD
5 5
1.422 + 0.040
951K
3 x 3
0.825 1 0.053
877K
Gauss
5 5
0.744 + 0.040
951K
STL-10 The rotated MNIST dataset has global rotational symmetry by design, so it is un- surprising that equivariant models perform well. But interestingly, rotation equivariance can also help for natural images without global rotational symmetry (Weiler & Cesal 2019: Shen et al. 2020). We therefore benchmark steerable PDOs on STL-10 (Coates et al. (011)) where we only use the labeled portion of 5000 training images. The results shown in Table 2 The model ar- chitecture and hyperparameters are exactly the
Table 2: STI. results, again over six runs. All models except the vanilla CNN use regular representations, see main text for details.
Method
Groups
Error [%]
Params
Vanilla CNN
12.7 + 0.2
11M
DdD,D, DiD,Di
10.7 1 0.6
Kernels
4.2M
10.2 + 0.4
DaD4Di
12.1 0.6
FD
DiD,Di
12.1 0.7
same as in (Weiler & Cesa, 2019). namely Wide-ResNettl6-88 trained for 1000 epochs with random crops, horizontal flips and Cutout (De- Vries & Taylor (017) as data augmentation. The group column describes the equivariance group in each of the three residual blocks. For example, D,D,D, means that the first block is equivariant under reflections and rotations, the second under 4 rotations and the last one only under reflections. All layers regular representations. The Ds-equivariant layers use 5 x filters to improve equivariance, whereas the other layers use 3 x 3 filters.
D&D4D,
14.3 14.3
RBF-FD
3.2M
DaDaDi
D&DaDr
11.2 0.3
Gauss
D.D.Dr
10.6 + 0.8
8

---

Published as a conference paper at ICLR 2022
Fluid flow prediction In the previously described tasks, the input and output representations all trivial. To showcase the use of non-trivial output representations, we predict laminar fluid flow around various objects, following Ribeiro et al. 2020). In this case, the network outputs vector field, which behaves differently under rotations than scalar outputs, and whose equivariance cannot be represented using PDO-eConvs, since they only implement trivial and regular representations. We find that equivariance significantly improves performance for both kernels and PDOs, compared to vanilla non-equivariant versions. See Appendix N for details.
Equivariance errors In the continuum, steerable CNNs and steerable PDOs are both exactly equivariant. But the discretization on square grid leads to unavoidable equivariance errors for rotations that aren't multiples 2. The violation equivariance practice is thus closely connected to the discretization error. For finite differences, the discretization error is particularly easy to bound asymptotically, and as pointed out by Shen et al. (2020), this places the same asymptotic bound on the equivariance error. However, our experiments show that empirically, finite differences don't 't lead to particularly low equivariance error (kernels and all PDO discretizations perform similarly). See Table in in Appendix for details.
Locality of PDOs While all equivariant models improve significantly over the non-equivariant CNN, the method of discretization plays an important role for PDOs, The reason that FD and RBF-FD underperform kernels is that they don't make full use of the stencil, since PDOs are inherently local operators. When a 5 x 5 stencil is used, the outermost entries are all very small compared to the inner ones, and even in images of stencils to illustrate this). Gaussian discretization performs significantly better and almost as well as kernels because its smoothing effect alleviates these issues. This fits the observation that kernels and Gaussian methods profit from using 5 x 5 kernels, whereas these do not help for FD and RBF-FD (and in fact decrease performance because of the smaller number of layers).
3 kernels, the four corners tend to be closer to zero (see Appendix Moadixx[[ooo [or
5 CONCLUSION
We have described general framework for equivariant PDOs acting on feature fields over Euclidean space. With this framework, we found strong similarities between equivariant PDOs and equivariant convolutions, even unifying the two using convolutions with Schwartz distributions. We exploited these similarities to find bases for equivariant PDOs based on existing solutions for stecrable kerels..
Our experiments show that the locality of PDOs can be a disadvantage compared to convolutional ker- nels. However, our approach for equivariance can easily be combined with any discretization method, and we show that using Gaussian derivatives for discretization alleviates the issue. Equivariant PDOs could also be very useful in cases where their strong locality is desideratum rather than drawback.
The theory developed in this work provides the necessary foundation for applications where equivari- ant PDOs, rather than kernels, are needed. For example, Probabilistic Numerical CNNs (Finzi et al.. 2021) use PDOs in order to parameterize convolutions on continuous input data, Finzi et al. also derive constraint make these PDOs equivariant, which is special case of our PDO G-steerability constraint Eq. (4) The solutions to this constraint presented in this paper are the missing piece for implementing and empirically evaluating Probabilistic Numerical CNNs neither of which Finzi et al. do.
Another promising application of PDOs is an extension manifolds. Gauge CNNs (Cohen et al. 2019b; Kicanaoglu et al.] 2019 Haan et al. [202 Weiler et al. 2021) are a rather general framework for convolutions on manifolds; see Weiler et al. (2021) for a thorough treatment and literature review. As in the Euclidean case, Gauge CNNs use feature fields that transform according to some representation p. The kernels are defined on the tangent space and are still constrained by the G-steerability constraint. Because of that, our approach to equivariant Euclidean PDOs is very well-suited for generalization to manifolds and our steerable PDO solutions will still remain valid. One advantage of PDOs in this setting is that they require no Riemannian structure and can achieve equivariance with respect to arbitrary diffcomorphisms, as is common in physics, instead of mere isometry equivariance.

---

Published as a conference paper at ICLR 2022
REPRODUCIBILITY STATEMENT
All Propositions and Theorems in the main paper and in the appendix explicitly state all their assumptions. In cases where theoretical results are stated only informally in the main paper (such as our results involving Schwartz distributions), the precise claims can be found in the appendix. The appendix also contains complete proofs for all of our theoretical claims.
The code necessary reproduce our experiments can be found https::/githubb com/ @jnnr/steerable pdo experiments... The required datasets are downloaded and prepro- cessed automatically. The exact hyperparameters we used are available as pre-defined configuration options in our scripts. We also include a version lockfile for installing precisely the right versions of all required Python packages. Our implementation steerable PDOs is easy to adapt to different use cases and fully documented, allowing other practitioners to test the method on different datasets or tasks.
ACKNOWLEDGMENT
We would like to thank Gabriele Cesa and Leon Lang for discussions on integrating steerable PDOs into the E2CNN library and on solutions of the kernel G-steerability constraint. This work was supported by funding from QUVA Lab.
REFERENCES
Jimmy Aronsson. Homogeneous vector bundles and G-equivariant convolutional neural networks. arXiv:2105.0544 05400 [cs, math, star), 2021.
Johannes Brandstetter, Rob Hesselink, Elise van Pol, Erik Bekkers, and Max Welling. Geometric and physical quantities improve E(3) equivariant message passing, 2021.
Adam Coates, Andrew Ng, Honglak Lee. An Analysis Single-Laye Networks in Unsupervised Feature Learning. International Conference on Artificial Intelligence and Statistics, 2011.
Taco Cohen, Mario Geiger, and Maurice Weiler. general theory of cquivariant cnns on homogeneous spaces. In NeurlPS, 2019a.
Taco S. Cohen and Max Welling. Group Equivariant Convolutional Networks, International Conference on Machine Learning, 2016.
Taco S. Cohen and Max Welling. Stecrable CNNs. International Conference on Learning Representations, 2017.
Taco S. Cohen, Maurice Weiler, Berkay Kicanaoglu, and Max Welling. Gauge Equivariant Convolutional Networks and the Icosahedral CNN. International Conference on Machine Learning, 2019b.
Terrance DeVries and Graham W. Taylor. Improved Regularization of Convolutional Neural Networks with Cutout. arXiv: 1rxiv:/708.0455 070504452 fes), 2017.
Mare Finzi, Samuel Stanton, Pavel Izmailov, and Andrew Wilson. Generalizing convolutional neural networks for equivariance to lie groups on arbitrary continuous data. In ICML, 2020.
Mare Finzi, Roberto Bondesan, and Max Welling. Probabilistic Numeric Convolutional Neural Networks. International Conference on Learning Representations, 2021.
Bengt Fornberg and Natasha Flyer. Primer on Radial Basis Functions with Applications to the Geosciences.
CBMS-NSF Regional Conference Series in Applied Mathematics, 2015.
Pim Haan, Maurice Weiler, Taco Cohen, and Max Welling. Gauge Equivariant Mesh CNNs: Anisotropic convolutions on geometric graphs. International Conference on Learning Representations, 2021.
Emiel Hoogeboom, Jom W. Peters, Taco Cohen, and Max Welling. HexaConv. International Conference on Learning Representations, 2018
Berkay Kicanaoglu, Pim de Haan, and Taco Cohen. Gauge Equivariant Spherical CNNs, 2019. URL attps
//openrevies net/ www.:neteeottttteeeeeem http:/oorommnnneeeiiieeeeee
Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. International Conference on Learning Representations, 2015.
10

---

Published as a conference paper at ICLR 2022
Leon Lang and Maurice Weiler. A Wigner-Eckart Theorem for Group Equivariant Convolution Kernels. International Conference on Learning Representations, 2021.
Hugo Larochelle, Dumitru Erhan, Aaron Courville, James Bergstra, and Yoshua Bengio. An empirical evaluation of deep architectures on problems with many factors of variation. International Conference on Machine Learning, 2007.
Mateus Dias Ribeiro, Abdul Rehman, Sheraz Ahmed, and Andreas Dengel. Deepefd: Efficient steady-state laminar flow approximation with deep convolutional neural networks, 2020.
Burhan Sadiq and Divakar Viswanath. Finite difference weights, spectral differentiation, and supereonvergenne. Mathematics of Computation, 83, 2014.
Nicholas Sharp. Souhaib Attaiki, Keenan Crane, Maks Ovsjanikov, Diffusion You Need Learning on Surfaces. arXiv:2012.0088 [cs]. 2020.
Zhengyang Shen, Lingshen He. Zhouchen Lin, and Jinwen Ma. PDO-eConvs: Partial Differential Operator Based Equivariant Convolutions, International Conference on Machine Learning, 2020.
Bart Smets, Jim Portegies, Erik Bekkers, Remco Duits. PDE-based Group Equivariant Convolutional Neural Networks. arXiv:200100044 [cs, math, stat], 2020.
Francois Treves. Topological Vector Spaces, Distributions and Kernels. Academic Press Inc, San Diego, 1967.
Dmitrii Aleksandrovich Varshalovich, D. A. Varshalovich, A. Moskalev, and Valeriï Kel manovich Kherson- skiī. Quantum Theory of Angular Momentum: Irreducible Tensors, Spherical Harmonicy, Vector Coupling Coefficients, 3nj Symbols. World Scientific Pub., 1988.
Maurice Weiler and Gabriele Cesa. General Fquiiiaquv Steerable CNNs. Advances Neural Information Processing Systems, 2019.
Maurice Weiler, Mario Geiger, Max Welling, Wouter Boomsma, and Taco Cohen. Steerable CNNs: Learning Rotationally Equivariant Features Volumetric Data. Advances in Neural Information Processing Systems, 2018a.
Maurice Weiler, Fred A. Hamprecht, and Martin Storath. Learning Steerable Filters for Rotation Equivariant CNNs. IEEE Conference on Computer Vision and Puttern Recognition (CVPR), 2018b.
Maurice Weiler, Patrick Forré, Erik Verlinde, and Max Welling, Coordinate Independent Convolutional Networks Isometry and Gauge Equivariant Convolutions on Riemannian Manifolds. arXiv:2 2060000000000 [cs. stat), 2021.
Sergey Zagoruyko ko and Nikos Komodakis. Wide Residual Networks. British Machine Vision Conference (BMVC), 2016.

---

Published as a conference paper at ICLR 2022
SUPPLEMENTARY MATERIAL
A STEERABLE PDOS BETWEEN VECTOR AND SCALAR FIELDS
We give bases for the space steerable PDOs for many important cases Appendix K However, the generality of the description there obscures the connection to well-known PDOs such as the gradient divergence. So complement the general solutions, we discuss few simple cases in much more detail in this section. We will see that Laplacian, gradient, divergence, and curl are all rotation equivariant and, more interestingly, that all rotation equivariant PDOs can be constructed by combining these (in the simple settings we cover in this section). In many cases, we rederive the solutions even though all of them would follow immediately from the general case in Appendix [] in order to provide some intuition why these the only equivariant PDOs. Readers who are only interested in an overview of the results may wish to skip to the end this section.
A.1 SCALAR TO SCALAR PDOS
We have already argued in Section 2.3 that the (QQ))ecquivarian PDOs between scalar fields are precisely polynomials in the Laplacian, i.c. of the form g(A) for an arbitrary real polynomial q [x]... The derivation given there applies without changes to SO(d) for d > 2 and to ((d) as well, so the same holds in these cases.
A.2 SCALAR TO VECTOR PDOS
We start by considering the case where Pmn is trivial (with Gin = 1) and Pour is the vector field representation (i.e. Cout d). We can then represent PDOs by a d x 1 matrix of polynomials, i.c. column vector. The PDO steerability constraint becomes
P(g=) gP(x)
(10)
since Pmn is trivial. A good mental model for this subsection is to think of P as vector field P: Rd Rd whose entries happen be polynomials. The steerability constraint simply states that this vector field must "look the same" after rotation, see Fig. for examples.
begin by discussing the case G = SO(2), which somewhat different from O(2) and from d >>> 2. Any rotation equivariant vector field on IR² is fully determined by its values the ray {(r1, 0) [ > 0). Specifically, the rotation equivariant vector fields v are precisely those that can in polar coordinates be written
COS yp sin yp
-sin e) (fi(T) cos up (2((T)
v(r, p)
(11)
for arbitrary radial parts fi, f2. Cartesian coordinates, we can write this as
u(x, v) (x) -
(1 + +22) frr2 + +22) (
F2(2
(12)
where f((z) == LiV3., need e and 82 to be polynomials in æ, y. which means that the f need to be polynomials. If we then apply the D isomorphism, we get: Proposition 3. The -eqi)--rui differential operators from a scalar to a vector field are exactly those of the form
Vz
(0
0
82
91 (1(
(8)) + +g((A)
(-o3) θ
(A) 92 (2)
(13)
02
0,
where ai and 92 are arbitrary polynomials.
In words, the of the 2D curl, (-0g, 0,)T, with coefficients being polynomials in the Laplacian (rather than just real numbers).
PDOs are all linear combinations of the gradient and the transpose
Created wwnghttpss::wwwwdemssss.... httpa//aaaaa////////////ooooooon
12

---

Published as a conference paper at ICLR 2022
7
7
(b) More complex equivariant vector field (a linear combination of gradient and transpose curl)
(a) Vector field that induces the gradient
Figure 3: Two examples of S0( -cquivarian vector fields
The gradient is also equivariant under reflections, i.e. (22)-qquivriian, and it easily generalizes to higher dimensions. However, the transpose 2D curl only appears in this particular setting: is not reflection reflection-equivariaatt and it does not have an analogon in higher dimensions o We summarize this in the following result:
Proposition 4. Let G (dd for d 2 or (O(d) for d Then the G-equivariant differential operators from scalar to vector field are exactly those of the form
(A) grad
(14)
for an arbitrary polynomial E R[x]].
possible intuition for why SO(2) is a special case is that SO(2) ~ S whereas SO(d) ş S for d > 2 and ((d) 전 g for d > 2. Our construction above heavily makes use of the fact that rotations of R22 correspond one-to-one to angles, i.e. points on S and this construction thus does not generalize to any other cases.
Before we prove Proposition 4 we show a helpful lemma:
Lemma 5. Let > 2. Then for any linearly independent vectors v, w E Rd, there is a rotation g E (O(d) such that go but gu # w. For there is such g E O(2).
Proof of lemma. Since u and w are linearly independent, we can write Rd = span(e) + W, where
W is linear subspace containing w. Pick an element ğ E SO(d 1) (or O - 1) = 0(1) {+1} in the d = 2 case) such that ÿw A w, where ÿ acts on W (this always exists, note that w + 0). Then there is g E SO(d) (or ((d)) that restricts to ğ on W and the identity on span(v) (in terms of matrices with respect to a basis of the form {u....}, }, g would be block-diagonal, with 1 x 1 identity
block and (d 1) x (d 1) block for g).
One direction is clear: the gradient is induced by the matrix of polynomials P(x) 2, Rdd which is clearly equivariant. We have also seen that polynomials the Laplacian are equivariant, and we can compose these equivariant PDOs to get another equivariant PDO. So what remains show is that no other equivariant PDOs exist.
Proof of Proposition 4
"To avoid confusion, we remark that the 3D curl of course exists but maps between two vector fields. The 2D curl is in fact closely related to the 3D curl, and the fact that it does not have higher-dimensionn analogon corresponds to the fact that the 3D curl, as a vector to vector PDO cannot easily he generalized to higher dimensions.
13

---

Published as a conference paper at ICLR 2022
So let P be G-equivariant, with G' in Proposition 4 Let furthermore IR be arbitrary and g E G be an element in the stabilizer of æ, i.c. gr = 2. Then we have
gP(x) = P(gx) = P(x).
(15)
In other words, for any g E G with gr = z, we also have gP(x) = P(z). By the lemma, x and P(x) thus linearly dependent, i.e. P(x))
for this particular a: and some E R.
We can apply this reasoning to any x E R. so there is a function c: IR + R such that P(x) c(z)x.
Furthermore,
c=))zz=pP[c = gP(x) P(gx) cgzz)gz,
(16)
has be rotation invariant. As have already argued, this implies that c(x))())))))) qa} for some function and since e needs to be polynomial, so does Then
D(P) q( grad
(17)
as claimed.
If the lemma also holds and the proof goes through this is just the scalar case from the previous section, which is generalized here. But for G = SO(2), this argument does not work because SO(1) {1}. the decisive step the proof of the lemma fails, That is what allows the additional equivariant PDOs.
A.3 VECTOR TO SCALAR PDOS
Equivariant PDOs mapping from vector to scalar fields are simply the transpose of those mapping from scalar to vector fields (for orthogonal groups G): P is now a x d matrix and the steerability constraint is
P(gx) = -1 P(x)g"
(18)
By transposing, get
pd ((xx)=9PP (x),
(19)
which is the equivariance condition for scalar vector layers. As solutions, we get the divergence (as the transpose of the gradient) and for G = S0( also the 2D curl. They are again combined linearly with polynomials in the Laplacian as coefficients.
A.4 VECTOR TO VECTOR PDOS
For PDOs mapping between two vector fields, the steerability constraint is
P(gz) = gP(r)
(20)
Since the solution space in this case is somewhat more complicated, we will only cover G = (2 in these examples; Appendix K more solutions. In principle, could apply the same method that we already used before for (2), choosing the radial components of P freely and using the steerability constraint determine the angular components. But since the computations in this are more involved and don't t yield much additional insight, we will instead use the solutions from Appendix K Vector fields correspond frequency irreps, and writing out the solutions for those explicitly, we get that the equivariant PDOs are exactly linear combinations of
( 9 9
(i -))
O8 -8 20,0,
-20,00 0 03 02 02
(21)
20,0,
02-02
20,0,
with polynomials in the Laplacian coefficients.
The first two operators are simply the identity and a h rotation matrix, both zeroth order PDOs. Note that the rotation matrix rotates the fibers of the vector field. it does not act on the base space. The other two operators are less interpretable, but we can replace them through a change of basis:
0
0 02
20.0,
02
0,0,
+
2
20,0,
0f -2-03 -2
=
8
y
(22)
14

---

Published as a conference paper at ICLR 2022
which is the matrix describing the composition grad div. Similarly,
1
-
1 +
20,0 8
- 0% 20,0,
(
-
020g -
0 0g0,)
(23)
0
which is the matrix describing grad curlZD. So if write R for the z/z rotation matrix (interpreted as PDO), then the equivariant PDOs mapping between vector fields are exactly linear combinations of
id, R grad div, grad curl2n
(24)
as always with polynomials in the Laplacian as coefficients.
We can be even more economical and describe these PDOs with fewer building blocks. For two vectors P. Q of one-dimensional PDOs, e.g. = = (0,, 0²)², we write P 8) Q for the 2 x PDO with entries (P @ Q) P:Qj- Then furthermore note that
write for example grad div 0 @ Ō. We
1 0
0
-1
0x
(-g, 02
Rd
curl2D
(25)
Ogy
This means we can write the basis from above as
id,
R, 0 0,
0 ② Rd
(26)
A.5
SUMMARY OF RESULTS
(O())- or O(d) -qquiiaaaaa PDOs between two scalar fields are exactly polynomials in the Laplacian.
G-equivariant PDOs mapping from scalar to vector fields are exactly those of the form q(A) grad for G (O(d) with > or = Odd). For PDOs from vector to scalar fields, we similarly get (() div.
S0()) PDOs from vector to scalar fields are exactly those of the form
@i (A) div + (2(A) curlan
(27)
For PDOs from scalar to vector fields, we get grad instead of div and the transpose of the 2D curl instead.
PDOs between two vector fields are linear combinations of the identity, 2 rotation, grad div and grad o curlyp, with polynomials the Laplacian as coefficients.
B REPRESENTATIIN THEORY PRIMER
This section introduces the fundamental definitions of representation theory that we need.
B. BASIC DEFINITIONS
Definition 1. A group representation group G, or representation for short, is a group homomor- phism p: G GL(V) for some vector space V. This means that p(gg') p(g)p(g') for all g.g', so multiplication of group elements in is represented as matrix multiplication in GL(V).
For this paper, we only need V R so focus on this case. In particular, some of the following definitions make use of the fact that we only consider finite-dimensional representations.
Given multiple representations of the same group, we can "stack them together" using direct sums: Definition 2. Let P1 G + GL(( and p2 : G GL be two representations of G. Then we define the direct sum representation P1 @ P2 : G + GL by
(pi(g)
(ox e (2)
p2(9)
(28)
which independently the subspaces T and 8 of te
15

---

Published as a conference paper at ICLR 2022
This corresponds exactly to stacking feature fields of different types. For example, vector and a scalar field into one diuediiensional field, such that its vector and scalar part transform independently. The representation of this Nour-dimensional field will be the direct sum of the vector and scalar field representations.
stack
Often, two representations formally different but can be transformed into one another using a change of basis; they then behave the same in all relevant aspects. For example, S0(2) has a representation
p()) = cos yp sin yp
sin yp cos
(29)
that represents a rotation angle yp by counterelockwise used for vector fields). However, we could just well use
rotation matrix (this representation is the one
p(p) =- (- cos 4 sin up sin yp
(30)
cos (p)
where the rotation is clockwise. Using one the other is pure convention and would like treat them as "the same" representation. This is formalized as follows:
Definition Two representations Pl, P2 G >> GR.((R)) are equivalent if there matrix Q GL()R)) such that
p2(g) = p))rp(
(31)
for all g e G.
Intuitively, Pr and 02 differ only by change of basis, which is given by Q.
B.2 DECOMPOSITION INTO IRREDUCIBLE REPRESENTATIONS
We can now discuss irreducible representations, which play an important role for solving the kernel and PDO steerability constraints.
Definition 4. A linear subspace W C R is called invariant under a representation p G + GL
p(g)w E W for all E G and to E W. In this case, we can define the restriction P/W G GL(W), called a subrepresentatioon
p.
Definition 5. A representation p is called irreducible if all its subrepresentations either p itself representations Gi + GL({O)), where {0} is the trivial vector space.
For example, P1 and P2 are both subrepresentations of #1 ① P2, so direct sums are never irreducible. A natural question is the converse: if a representation is not equivalent to direct sum, does that mean that it is irreducible? ones? In general, this is false, but it holds for the cases that interest us:
other words, can all representations be split into a direct sum of irreducible
Definition 6. A topological group is a group G equipped with a topology, such that the group multiplication G G G and the inverse map G + G are continuous with respect that topology. A compact group is a topological group that is compact as a topological space.
Theorem Let G be a compact group. Then every finite-dimensionl representation of G is equivalent to direct sum of irreducible representations.
Since we only consider compact subgroups of ((2) and 0(3), this theorem applies to all the cases solve.
So we can always write
Pin Qi3
t/x
Qin
iein
(32)
Pout Q
out
thz
Quu
where the vi are irreducible representations. It is then easy show (Wilerr & Cesa kernel k solves the G-steerability constraint
2019 that
k(gx) pau(g) -1 k(xx) pio (1)
(33)
16

---

Published as a conference paper at ICLR 2022
if and only if к Qou Qin solves bioek-wise steerability constraint between irreducible representations. Concretely, <gx) (=))))
Vi,j
(34)
where (x) is the submatrix of k(x) that belongs to the and v/..
The approach to solving the steerability constraint thus to solve Eq. 34 for arbitrary irreducible representations vi and f/y. For general (not necessarily irreducible) Pour and Pin+ then first find the decompositions into irreducible representations, Each basis element n of the solution to Eq. (34) is then padded with zeros to the right size and finally transformed via = Q xiQnn to get the final basis elements. See Weiler & Cesa (019)) for more detailed discussion and visualization.
Clearly, this procedure works just as well for PDOs as does for kernels: for PDOs, we need to find the restriction of the kernel solution space to polynomials, and it does not matter whether we restrict on the level of irreducible representations and then combine them, or first combine irreducible representations and then restrict.
B.3 SPECIFIC REPRESENTATIONS
We now define all the types of representations that occur in the main paper:
As already mentioned the paper, scalar fields are described by the trivial representation p(g) := 1 and vector fields by the representation p(g) 9 (for G < GT.(R.)).
For a finite group G, the regular representation is G -> GL [TR. Gl), defined by
p(g)ez Cg''
(35)
where (Eg) is the canonical basis of [ (for some ordering of G). So this representation associates one basis vector to each group element and then acts by permuting these basis vectors. p(g) is thus always a permutation matrix.
Quotient representations are a generalization of regular representations, defined as follows: let G be finite group and H I G subgroup. Then define the quotient representation G/H Pquot : - (LI by
G Pquot
(g)e == Egg'
(36)
where we now use basis indexed by the cosets gHl E G/Hl for g E G. For H {e}. we recover regular representations. Appendix C by Weiler & Cesa (2019) provides some
intuition for these quotient representations in the case where G and H both cyclic groups CN and Car.
C INTUITION FOR THE GROUP ACTION ON POLYNOMIALS
Our work makes heavy use of terms of the form p(gr). where p is polynomial p: R + R, g G a group element, and æ E RRd We would now like to provide bit more intuition for this action of the group G on polynomials. Note that we will only cover scalar-valued polynomials in this appendix, i.e. using trivial representations. The general case is straight-forward: the group acts on each polynomial in a matrix of polynomials the way we describe here, while (m)(g) and Pout(g) act via matrix multiplication.
To prevent confusion, let us reiterate that can think of polynomials in two different ways. The first as formal expression, where a placeholder for things to plugged in. This is the approach we take when connecting polynomials to PDOs, where plug in differential operators for æ. The second is as a specific type of function on Rd in which case r IRd is simply the argument that function.
This second perspective is the one in which the group action on polynomials is easiest to understand. Specifically, the action of function on Rd x + p(gx) defines a different function on Rdd. We could think of it as composing the function p with the group action of g on Rdd
is simply matrix multiplication. Similar to how + p(x) defines
17

---

Published as a conference paper at ICLR 2022
Crucially, this new function is still polynomial in the components of just with different coefficients. For purposes of illustration, consider very simple example with and G = SO(2). We will use the polynomial p(x) 2 + 21 (this is just meant to be simple non-trivial polynomial, it does not have special equivariance properties).
We represent 9 rotation matrix parameterized by an angle 0. The action x: is then given by
gz
cos sin 0 cos 0
sin 0
x11 F2
2i cos 0 222 sin 0 21 sin 0 + x2 cos
(37)
The left-hand side is what now plug into our polynomial, which yields
p(gx) (g=)² (gz) (2) sin 0 + 22 cos + (æ) cos 22 sin
(38)
We expand this and collect the coefficients for each power of 2ı and 22, which yields p(ge) sin (0)a} + 2 sin cos(0)=11 + cos²(0)² + cos(0)az sin(0)æ2
(39)
If desired, we can now switch to the first perspective, and interpret this polynomial formal expression defined by its coefficients. In that perspective, 9 acts on p by modifying its coefficients, rather than by composition. Computing the new coefficients is straightforward analytically (given a matrix representation g): we just gave a simple example for a polynomial of order two. The general case proceeds along the same lines, it just requires using the binomial theorem (or the multinomial theorem for d > 2).
D BACKGROUND ON DISTRIBUTIONS
In Appendix E. we will describe our framework for equivariant maps using convolutions with Schwartz distributions. To facilitate that, we now give the necessary background on Schwartz distributions. We restrict ourselves to what is absolutely necessary for our purposes; for much more thorough introduction and for proofs, see e.g. Treves (167)).
D.1 BASIC DEFINITIONS
C Rd open, D(U) := (U) is called the space of lest on (C? (U) is the space of compactly supported smooth functions U - RR. A sequence (yo,) in D(U) is defined to converge to 0 iff
Definition
there a compact subset C such that the support of each yo. is contained K and
(ii) 8m Pr + uniformly for all multi-inieee multi-indices a.
One can define the so-called canonical LF topology on D(U). This topology induces the notion of convergence just given. However, constructing this topology explicitly is unnecessary for our purposes since knowing when sequences converge will enough.
Definition 8. A linear functional T: D(U) -> IR is defined continuous if for every sequence n that converges to 0 in D(U), Ty,, + 0 (in R).
Such continuous functional is called distribution on U. The space of all distributions on U is written (U).
This notion of continuity is also induced by the canonical LF topology and then D'(U') is the topological dual DU) as the notation suggests.
So intuitively, a distribution is a "reasonable" way of linearly assigning a number in IR to each test function in C
A function U + R induces distribution T3 E D'U),, defined by
Typ:= = / fpdaa
(40)
18

---

Published as a conference paper at ICLR 2022
Since vp has compact support, we don't need many restrictions f (for example, any locally integrable function f (U) works).
We will also use the duality pairing
(T,4) :- T(p)
(41)
and under slight abuse of notation also write
/ fipda".
(++) - Ty({))
(42)
Note that this coincides with the inner product on (U)),, but allows functions that are not in L2 while in exchange requiring &o to have compact support.
D.2 CONVOLUTIONS
We define the translation Tx Rd - Rd by (v) y + and set Ta := T-z for functions on Rd. This is exactly the same as the action of (R +) we used in the main paper, just with more explicit notation. Furthermore, we write f(z) == f(-x).
Then we can define the convolution between distribution and test function: for f E D(U) and E (U), the convolution * E (U) is defined by
(T (z) T(T=f)-
(43)
Explicitly, (Ta,f)u)= = f(x - y). This immediately shows that this notion of convolution extends the classical one, i.c.
Tg * g
(44)
D.3 DERIVATIVES
Definition 9. For T D'( we define the distributional derivative T the unique distribution on U for which
(TT, = (-1) (T.
(45)
The distributional derivative of all orders always exists, i.e. "distributions are infinitely differentiable", but of course only this distributional sense. At least for (U) (but also under much weaker assumptions), this definition also extends the definition of derivatives of functions, in the sense that Tye = Ty
D.4 COMPOSITION WITH DIFFEOMORPHISMS
Let F; : U + be diffeomorphism and T E DU).. Then we define the composition ToF E D'(U)
O F. (T) det DF 1p 1 F-
(46)
where det DF-1 is the inverse Jacobian.
As in the previous constructions, this extends the definition for classical functions the sense that
TjoF = Ty F.
(47)
This follows immediately from the transformation theorem for integrals:
(Tror, sp) = / (foF) Pd
(48)
f. 1.p F ( DF dd
(49)
(T' F "
(50)
(51)
19

---

Published as a conference paper at ICLR 2022
Furthermore, this type of composition is associative:
((P o = (7 o F, de /
(52)
DG (T, (dde
(53)
(T, [eee((000=aa--0000010000000011
(54)
(T, |det p
(55)
(T (F G)
(56)
We are particularly interested in the case where linear transformation, i.c. F GL((R). Then
we have
(T' :p) det F=1 (T, p FT1)
(57)
(note that in this case, we pull out the determinant because it is just a constant). For translations, we get
(T ToTA, Tx, (TTT))))=TTpoooa.... =T (T,
(58)
D.5 THE DIRAC DELTA DISTRIBUTION
A very simple but important distribution is the following: Definition 10. For any x E Rd, the Dirac delta distribution 82 E D(Rd) is defined by
[ p(x)
(59)
is clear from the definition that the distributional derivatives of the delta distribution are given by (60)
D.6 CONVERGENCE OF DISTRIBUTIONS
Definition 11. We say that a sequence Tn of distributions converges to T, written T, + T, if converges pointwise, i.e.
(Tn: vp) + (T,p) for + do
(61)
for all ip E D(U).
Abusing notation a bit, we also write fn + T for functions fn if Tf, T.
One important type of function sequences are Dirac sequences, which converge to the Delta distribu- tion:
Lemma 7. Let fn be sequence in (-(R)) such that
(i) fn 2 0.
(ii) [lfalll = and
(ii) SREV,B(()) fndxa for all >>> 0.
Then fn + dg in the sense of distributions.
We also note that convergence plays together nicely with derivatives and with convolutions: Lemma 8. I, + T, then T + T for all a.
Proof.
(8 = (-1)la (Tn, 0%
(-1) 1o ( (T
(62)
(8
20

---

Published as a conference paper at ICLR 2022
Lemma 9. If - T, then Tn + T pointwise for all f E D(U).
Proof.
(T, * ( (n(TTf)
TT(Taf)
(63)
= *T ()))
D.7 TEMPERED DISTRIBUTIONS AND THE FOURIER TRANSFORM
The Schwartz space S(R") is the space of functions for which all derivatives decay very quickly as |x| + co. More precisely, a smooth function f : IR + S(R)) iff
sup |1~000 0 f(=) <0. ZERd
(64)
Intuitively, all derivatives of f must decay more quickly than any polynomial. Examples are compactly supported functions or Gaussians.
Its dual space S' (Rd) called the space of tempered distributions and can be continuously embedded into 'D' (IR.). Tempered distributions are still very general; in particular the delta distribution, distributions induced by functions, and derivatives of tempered distributions are all tempered.
The reason we're interested in tempered distributions is that there is a Fourier transform defined by
(7) 9) =
(65)
for tempered distributions T. This is an automorphism on S'
We use the following convention for the Fourier transform on functions:
F (f) (5) (2m) d/2 [ f(x) exp()ppp) {) dr.
(66)
which means the inverse is given by
F (2)) 4/2 { [(x) ((x))xx((xx dg.
4422 (27)-a/2
(67)
Rdd
will later need the Fourier transform of derivatives the Dirac delta distribution:
(F (0 )o) sp) = (0 6o, F ((p})
(-1) {4))
0
(-i)la1 (x p]
(68)
(-i))al(2T) 4/2
x² p(z) exp(-ix 0)dr
(-i)lol(2T) f / p())d
so F 800 x xd (or more precisely, the distribution induced by the function + x²).
D.8 MATRICES OF DISTRIBUTIONS
Analogously how defined matrices of PDOs of polynomials, will need matrices of distributions. We will write (U, RCax)G)) for the space of Cout x Ci dimensional matrices with entries in DU,,). We get a pairing
(.)) : D' (U, R. Raa DUU,RA))
IR San
(69)
21

---

Published as a conference paper at ICLR 2022
defined by
(T, := E
(70)
The convolution of T (U, Xo and D(U, Re)) is defined analogously to the scalar case, Lee. (T (z) = (T, Tz.)), where we now use the more general duality pairing just defined.
We can also define the multiplication of matrices of distributions by matrices over R. For matrix E IR Sou Cou and a matrix of distributions T D'(IR, RR Cin) )), we write
(AT);; Σ AuTIY ER(
(71)
(multiplying distribution by a scalar obviously defines another distribution). Analogously, we define TB for B E Ca It immediately follows that
A B4) (ATB,p)
(72)
holds.
Composition with diffeomorphisms of Rd can simply be defined component-we wise for matrices of distributions. This commutes with multiplication by constant matrices, meaning that
F A(T F)B.
(73)
Finally, we also define the Fourier transform component-wise, and this commutes with multiplication by matrices in the same way.
E DISTRIBUTIONAL FRAMEWORK FOR EQUIVARIANT MAPS
In this section, develop two main results: first that all linear continuous translation equivariant maps between feature spaces are convolutions with some distribution, and then the equivariance constraint for such convolutions. See Appendix Dor background on distributions. We equip (UU) with the canonical LF topology throughout this section.
E. TRANSLATION EQUIVARIANT MAPS ARE CONVOLUTIONS WITH DISTRIBUTIONS
We begin by showing that the framework using convolutions with Schwartz distributions encompasses all translation equivariant continuous linear maps between feature spaces. As preparation, prove simple Lemma on the reflection map 8:
Lemma 10. The map D(U) + D(U) given by s(f) == f is linear and continuous.
Proof. Linearity is clear:
s(af - (x) = af (-x) + =b(-=)=as(f{(a)) (z)
(z(g)
(74)
For continuity, we use the fact that linear map from D(U) to itself is continuous if and only if is sequentially continuous (Treess 1967 Proposition 14.7). So take any sequence fn. - in D(U). This means that
(i) there is a compact subset c such that the support each fo contained in
(ii) 8 fn + 0 uniformly for all multi-indices a
We set -K = {- E K), then the support of s(fn) (which is just the mirror of the support of fa) is contained in - K, and - K is compact. Additionally.
0 s()) = (0
(75)
Since d f.. + 0 uniformly, the same is true for s (0" f.). follows that s is sequentially continuous, and thus continuous, which concludes the proof.
22

---

Published as a conference paper at ICLR 2022
We first prove main generality result for the special case of one-dimensional fibers:
Lemma 11. Ler + D()) - "()) be a translation equivariant continuous linear map. Then there is distribution T (U) such that Φ(f) T ff.
Proof. Let O D(U)= - D(U) be any continuous linear map, where continuity is understood with respect the canonical LF topology. We then define
T: DUU + R, =-) (
(76)
Equivalently, write T = 60 o & understood as normal composition of functions. dn and + are linear and continuous, as is by Lemma 10 Therefore, is also linear and continuous, and thus Schwartz distribution 7 (U).
We will also need that
T-If (4) (T-=f)(-o)
f(x v)
(77)
fly xT) (T)))
Now using the assumption that & is translation equivariant, i.c.
Tz = 1 TA,
(78)
it follows that convolution with T is given by
(T * (xx) (T, Ta
--a) (((T=f))())
(79)
=T-rO(f)(0)
(z))
This shows that convolution with T' is Q, which concludes the proof.
Finally, generalize multi-dimensiinal feature fields:
Theorem 12. Let & DUU Rea) - +DU, Rea)) be translation equivariant continuous linear map.
Then there is distribution T DU, IR Sour Xu)) such that Φ(J) T f.
Proof. We write ej for the j-th canonical basis vector of R For f E D(URRG)) we write fy for its j-th component. Then we have
Φ = +
Σ Sjej
= => $ j=1
(fzez)
(80)
j=1
We now define the components i D(U) - D(U) by
(,)(9) (T(ge;)),
(81)
Here, ge, E D(U, (sw) ) the map with y in its j-th output component and 0 in the other ones. We can then write
Go
(()), Σ
Σ (f)()))
(82)
j=1
is clear that the components P;j of + are still translation equivariant, continuous and linear. Therefore, by the previous Lemma, there are distributions Tij (D'(U)) such that ..)(g) = Tí * We then get
( = Σ
+ = =
(83)
3=1
where T (U, RGe is the matrix of distributions with entries Tij-
23

---

Published as a conference paper at ICLR 2022
E.2 EQUIVARIANCE CONSTRAINT FOR DISTRIBUTIONS
We will now characterize the distributions T E D'R ( for which the operator
: D R - C (R" R"
(84)
given by f + T H--isHH-eeuee H-equivariant. The codomain of T. may contain functions that are not compactly supported for some T. We are mostly interested in distributions T for which the codomain can be restricted to D(R4, RR())) but for the discussion of equivariance this does not make any difference,
keep the derivation general by not restricting the distribution T.
First, we can note that convolution with a distribution is always translation equivariant:
Proposition 13. The map f T f is translation equivariant for any distribution
Proof. The general equivariance condition is (h+ (x)))) (T (h (x)
(85)
for all f E D(R", R E H. E R More explicitly, this means
Pou (T, Th - (T, To (Pm( oh
(86)
where g is the linear component of h.
Let E IRd, i.e. a pure translation. The condition then becomes
(T,T(--))) (T, Tz T--)) (T, (T,
(87)
which always holds (since ((--1) = -t) .) So T. is translation equivariant by construction.
Therefore, it suffices consider pure linear transformations g E GL(RR... For those, we have the following result, which generalizes the steerability constraint for PDOs and the one for kernels:
Theorem 14. Let D'RRd, Rea (en). Then the map f * is G-equivariant if and only if
Tog = |det gllggg'pPpap ( Po Polg))
(88)
As is shown in Appendix D the definitions of convolution with distributions and of composition with diffcomorphism are compatible with those for classical functions. So if T is classical kernel, then the constraint on T is given by the same equation, which already gives us the steerability constraint for kernels.
Proof. We start with the constraint Eq. (86) from above with h = G and will transform this into the desired form Eq. (88). First, simplify notation, we can remove all the reflections : in Eq. (86).
This is now possible because f f g 1 = fo oog- which was not true for translations. Furthermore, we can pull the translations the other side of the duality pairing:
Pow((g) Tg f) 1 (T's o Tz. Pmm Pmngg) fg
(89)
= del gl (T
())
Now we'll use
T2 o 0 1z
(90)
since
+ gy = g(g 1x v)
(91)
Plugging this into Eq.
889
get
Pour (a) O 1z, det (T g Tg i2s Pmn()
(92)
We want to have only I on the right side of the duality pairing, so we use the notation we introduced for multiplying distributions by matrices and get
(pon(g)7 o Tg-112 f) 1 dee o (
(93)
24

---

Published as a conference paper at ICLR 2022
This holds for all f iff we have equality of distributions,
Doulg]] Ty-tx _ |det g googggg Tg-1=Pi(99).
(94)
Finally, multiplication with matrices and composition with diffeomorphisms commute, so we can cancel the Tg ix- This means our final constraint T is
Pout Pour(g)] (g) = |eet gl o g)
(95)
which we can slightly rewrite
To g - |det gl Pour " (
(96)
This is exactly Eq. 88 All steps of the derivation work equally well in the other direction, which proves the "if and only if"
E.3 DIFFERENTIAL APERATORS AS CONVOLUTIONS
It is clear that convolutions with classical kernels are special case of convolutions with distributions (see Appendix D) But we have claimed that convolutions with distributions also cover PDOs, which is what show
We have already seen Appendix D that derivatives of the delta distribution are closely related to PDOs. So we calculate the convolution with such derivatives:
(1)% * f)) = (0 TzJ)
(-1))alo
(0
(97)
Ts (rs)) (0)
0 (-z)
00 f(x).
Noting that the map T' + T. is RRliieerr we see that D(p) = p(0)6, f for polynomials p. It then also immediately follows that this is true for matrices of polynomials J' because
6n
(D(P))), D(Pig)
(98)
(0) * ff
j=1
P f
This shows that PDOs can be interpreted as convolutions with distributions, as claimed.
E.4 THE FOURIER DUALITY BETWEEN KERNELS AND PDOs
Interpreting PDOs convolutions with distributions allows us relate them to classical convolutional kernels via a Fourier transform. We have already seen in Appendix that
F F000 (2x) ( =x
(99)
which by linearity of the Fourier transform immediately implies
(P(())))) (2x)- d/2
Σ (-1))la 1
(100)
for P.j E., cd So the Fourier transform of derivatives of the delta distribution are polynomials, and of course vice versa via the inverse Fourier transform. Since PDOs convolutions with such delta distribution derivatives, we can also interpret them as convolutions with the (inverse) Fourier transform of polynomials. We will use this interpretation to give derivation of the PDO steerability constraint that sheds some light on the similarities and differences to the kernel steerability constraint.
We begin by proving basic fact about the Fourier transform of composition of functions:
25

---

Published as a conference paper at ICLR 2022
Lemma 15. Let g E GL(R")) and up D(U). Then
g det g
(101)
where we use the shorthand g -- := -
Proof. First we apply the transformation theorem for integrals:
F F(pog} (5) = (2x) 4/2 / (((x)) exp(-ix -ix E)dr
(102)
= det g (2 4/2 / (0(x)) expllii" x E)dx.
Now we just rewrite g
x g 5- z. Ts which gives the desired result.
This result holds more generally for tempered distributions (a subset of distributions for which the Fourier transform can be defined, see Appendix D
Proposition 16. For a tempered distribution T and E GL((R.).
( [gog det
(103)
Proof.
[T ()) g, (+))
= -ee ( FTT
(T.F(0og")
(104)
(F (F
= det [TT)
where we used Lemma 15 for (1).
Now note that the equivariance constraint for distributions, Eq. 88 is equivalent to the constraint we get when we take the Fourier transform on both sides. That's 's because the Fourier transform an automorphism the space of tempered distributions. By applying the result just proved, then get the constraint
det g (T)
(T) og -T- = |det g g (g) (TT P
(105)
We cancel the determinants, which gives the equivariance constraint in Fourier space:
F F og (g) (T) (1
(106)
Now let P()) so that * = D(P), for some matrix of polynomials P. We've seen above that in this case F (T) = P o m_ (up to constant coefficient), where m_, is multiplication by the negative imaginary unit -i. So ((P) is equivariant
P o m_m -T
Poar (g) P m-iPta(g-1
(107)
but since m- is invertible and commutes with the other maps, we can cancel it and get the equivari- ance condition
Pag-T TT--T-poon = (g)
P Pp
(108)
which is precisely the PDO steerability constraint. The reason that differs slightly from the kernel steerability constraint can now be traced back to Lemmal [5] Intuitively speaking, since PDOs are in sense the Fourier transform convolutional kernels, they transform differently under GL which leads to superficial differences in the steerability constraints. However, Fourier transforms commute with rotations and reflections (i.e. transformations from O(d). which is why for G < O(d), the steerability constraints coincide.
26

---

Published as a conference paper at ICLR 2022
E.5 PDOS AS THE INFINITESIMAL LIMIT OF KERNELS
Let th be any sequence of functions such that ehn - ôo. for example Dirac sequence. Then for polynomial p La Ca we also have
p - p(8)6
(109)
because of Lemma
Then Lemma 9 implies that
* f f
(110)
where the convergence is understood pointwise. Therefore, any sequence of kernels that approximates the delta distribution (by becoming "increasingly narrow") can be used approximate arbitrary PDOs by convolving with the derivatives of the kernels.
One example is a sequence of Gaussians
-0 02/6
(x()) (2m2)a/2
(111)
(now indexed by E >> instead of natural numbers). For E + O, we have v/s - do. as is easy to check with Lemma This naturally leads to the "derivative of Gaussian" discretization used in our experiments: we can approximate the PDO as convolution with derivative of a Gaussian kernel and then simply discretize this Gaussian derivative by sampling on the grid points.
The discussion in this subsection focused on 1 x PDOs and kernels, i.e. Gn Cout 1, but since convergence for multi-dimensional PDOs and kernels works component-wise. everything generalizes immediately to that setting.
F
THE RELATION BETWEEN KERNELS AND PDOS
Convolutional kernels and PDOs are closely related but also differ in some important ways. This appendix is meant briefly summarize various aspects of their relation that would otherwise only be scattered throughout this paper.
First and foremost, would like to emphasize that, in continuous setting, PDOs not just a special case convolutions with classical kernels. For example, the gradient operator is not represented by convolution with any kernel. In fact, even the identity operator (a zeroth-order PDO) is not the convolution with any function.
However, there two caveats to the above statement. First, if we allow convolutions with Schwartz distributionss rather than the usual kernels, then this generalizes both PDOs and convolutions with functions. In this broader framework, our knowledge, convolutions with Schwartz distributions have not been previously discussed in deep learning context, so for the common usage of "convolution" PDOs distinct operators.
thus interpret PDOs convolutions. But to the best of
The second caveat is that when we discretize ivanslationneeqivarrttn become convolutions in this discrete setting. Even so, the differences the continuum can matter in practice. First, we might want to discretize on point clouds or meshes. Second, even on regular grid, there are different methods for discretizing PDOs. While all of them lead to convolutions, they lead to convolutions with slightly different kernels. This shows that there is no clear one-to-one correspondence between kernels and PDOs even on discrete regular grids. Finally, the space of discretizations of equivariant PDOs is not necessarily the same the space of discretizations of equivariant kernels.
PDOs on a regular grid, they
Finally, we would like to mention connection that is outside the scope of this paper: infinite series of differential operators and convolutions can be the same even in the continuous setting. As an example, consider the diffusion equation
au((,, 1) Au(a,
(112)
Its solution can be written as
u(x, () exp(tA)u(x,
(113)
The time evolution operator cxp(tA) is an infinite series of differential operators, but can also be written convolution with heat kernel. However, these infinite series not covered by work-w we restrict ourselves to PDOs of finite order. We mention this connection between infinite PDOs and convolutions only avoid confusion for readers familiar with this correspondence.
27

---

Published as a conference paper at ICLR 2022
G PROOF OF THE PDO EQUIVARIANCE CONSTRAINT
In this section, we prove Proposition and Theorem the characterizations of translation- and G-equivariance for PDOs. Appendix already contains a proof of Theorem 2 that is arguably more insightful, but in this section we present an elementary proof that does not rely on Schwartz distributions.
G.1 TRANSLATION EQUIVARIANCE
We begin by proving that translation equivariance of PDO is equivalent to spatially constant coefficients, Proposition
Let 22j E. 00 where of (CR) and i, index the Cour Gn matrix describing the PDO. The equivariance condition Eq. (2 for the special case of translations
( D f) t out Φ(f) Vt E E Fm.
(114)
Using the definition of the i- action in Eq. (T. this becomes
c(a f(x - ca
()0 f(x Ve EERR Fm.
(115)
Here, Ca are matrix-valued, f is vector valued, and have matrix-vector product between the two. Explicitly, this means
ΣΣ 1 - -)- ΣE de (((=0)))))((-1
(116)
for all indices i. If f is zero in all but one component, the sum over j reduces to one summand. We therefore get simpler scalar constraint
Σ (1) g(1 f)= = Σ ddd 9)0 -t)
(117)
that has to hold for all indices j.j, where E C (Rd) is now scalar-valued.
Since this must hold for all functions g. we get equality of differential operators,
Σ (z) = Σ (x - t)o
Va,t E Rd.
(118)
This implies cd (z t) d for all and a. which in turn means that cd must be constants. From Eq. (I15), it is also apparent that constant coefficients are sufficient for translation equivariance, which proves the converse direction.
G.2 G-RQUIVARIANCE
Secondly we prove Theorem 2 the G-steerability constraint for PDOs. Recall that the G-equivariance condition for D(P)
(g Pin f) = Duu (D(P))),
(119)
where f E Fi and g E G. If we write out the definition of the induced action D., this becomes
(Pin(()))) gيو Pomr (pP g
(120)
We write P= Pa a where ca are constant matrix-valued coefficients Ca E Xea The LHS of Eq. 120) is then
D(P))An())fog Σ CaPi (f
(121)
In Lemma 17 we will show that the last term in Eq. 111 is given by
j o - = ( (0-=a)) 1) og 11
(122)
28

---

Published as a conference paper at ICLR 2022
where 9 T = (g-1)² and Ø understood vector that g acts on. Plugging this into Eq. 121
we get
D(P)omll)) Eee ca T0) Pl(()))) ((())))))))))))) "g
(123)
for the LHS of Eq. (120). For the matrix of polynomials
P= -m Cort we now define
g P Σ ca c)))"
(124)
which is again matrix of polynomials, each one rotated by Then Eq. 123 can be written more compactly as
((P)(a(()f = (D(g" P)
(125)
Plugging this back into Eq. (120), canceling the g and using the fact that this has to hold for all f. get
D(g P)pm(() Ponl (g)
(126)
equality of differential operators. We move the Pin to the other side and use the fact that the map D is bijective, which yields our final constraint
Pgg P') (o)))))
(127)
This concludes the proof of the G-steerability constraint for PDOs.
Finally, we prove the Lemma we just made use of, a higher-dimensional chain rule for the special case we need:
Lemma 17. Let f E (R2, and g E GL(RR).. Then any multi-index a.
0fo (0-""))) 5) gg
(128)
where g -- = (g-1)²
Proof. In general, for linear map A E GL(R"), we have
0,A,(z) Aji Vr.
(129)
Therefore,
0, offfgg)) o -0,(g
(130)
E 1g
(131)
=((6))))) 1) =g
(132)
We can apply this iteratively to show that
8( 0) og-
(133)
H TRANSFERRING STEERABLE KERNEL BASES TO STEERABLE PDO BASES
In this section, we develop the method presented in Section 3.2 in more detail and with proofs.
We fix group G < ((d) and representations Pm and Pout Then write K. for the space of G-steerable kernels. Because the steerability constraints for kernels and PDOs are identical in this setting, the space of equivariant PDOs the image under the isomorphism D of the intersection
Kpal [R n.k
(134)
29

---

Published as a conference paper at ICLR 2022
In words, the space of equivariant PDOs is isomorphic to the space of potynomial steerable kernels. Both spaces are ciinitt-dimensiona real vector spaces. The question we tackle now is how we can find a basis of pal given a basis of K. under certain conditions.
will vastly simplify our discussion to treat IC and Keul as modules over invariant kernels instead of as real vector spaces, least for now. A module is generalization of vector space, where the scalars for scalar multiplication can form a ring instead of a field. Because the radial parts of steerable kernels are unrestricted, makes sense think of IC module over the ring of radial functions. Formally:
Lemma 18. K. a C (oool- with scalar multiplicatiinn defined by
(fn)())== (1o1²) k((x)
(135)
for K E K. and f E - (Bzo). Similarly, Koot is an -modul, where r² ;= x7 +... + x and
multiplication simply multiplication of polynomials.
Proof. The steerability constraint
k(gr)) = (oul())((z))pm))
(136)
is clearly R-linear and in particular, KC is closed under addition and O E K.. Furthermore, for E K,
(fn)) [(1gx()) (0)))))))))))))) (o
(137)
so K. is also closed under the given scalar multiplication. The proof for Xpat is exactly analogous.
In Lemma 18 and the following, we write y² instead of |a|* simply to emphasize role as polynomial; so when reading 22, think of as a polynomial, and when reading | simply as function of Jr.
A basis of module is defined analogously to basis of a vector space, as set of linearly independent vectors that span the entire module. However, linear combinations now allow coefficients the ring of radial functions, instead of only real numbers. This means that fewer vectors are needed to span the entire space. because the coefficients "do more work".
In contrast to vector spaces, not every module has a basis. K. and Kpol do have a basis in the cases we consider in the paper but for this section that doesn't matter us: we will simply assume that K. has a basis with certain properties and then transfer this basis to Kgul- That the method developed here is indeed applicable to subgroups of (2) and O(3) will be the topic of Appendix
We roughly proceed in two steps:
1. We show that a basis of K. (over O (R>o)) that consists only of polynomials (and fulfills few other technical conditions) is also basis of Koot. but this time of course over R[r²].
2. We then show how turn this module basis into a vector space basis of Kppal
The first step is formalized as follows:
Proposition 19. Let B C R[x1..... xz.] [[f])....,,]]]<>XX Soa be basis of K. such that no matrix of polynomials in B is divisible by x2 and each one is homogeneous. Then B is also a basis of K as an 1R
Here, we say that b R[ za Xen a polynomial). We call it homogeneous all its entries are homogeneous polynomials of the same degree.
Xen is divisible by 12 if every component is divisible by r2 (as
Proof. Linear independence obvious: R[r2] C C Ryo), so if is linearly independent over 0 (Rzo), then it is also linearly independent over TR (r2
30

---

Published as a conference paper at ICLR 2022
To show that B generates Koot first let p Kool be homogeneous of degree I. Because p E K, there are fi E (Rzu) and K, B such that
p(x) = Σ [((l))))x(()))
(138)
We want to show that fi E R[(2), i.e. that fi is polynomial. Since each K; is homogeneous of degree 4, we get
(1 X E (es)))
(139)
because of p(Ax)= = > 2p(xx) for any X 2 O. Because the Hi are linearly independent, we must have
[1(AP1z17²) = > f((=())
(140)
Thus, fi is homogeneous of degree L- - and show in Lemma below, this implies
f((2) = (--122
(141)
or alternatively
f((1()) -/
(142)
What remains to show is that 1-4 is natural number, i.e. that - I; is even and non-negative. To prove this, divide all the into two groups: those for which 1 I, is even and those for which is odd. Then get expression of the form
2
p( xx)
[ Σ agr 2mm Kk +
Σ bar 2nk Kk k
(143)
Fach summand is rational function (the quotient of two polynomials), so both sums are rational functions. p(z) is also rational, so the first term on the RHS has to be rational. This only possible if ax for all k, otherwise could divide by the (rational) sum and should get rational function, but (x] is not rational. This shows that l is even for all i.
Now let /mxx max; 4. Then
E, Colo fmx 14 |
Kis ki(x))
p(x)=
(144)
fan
It is not possible to cancel any terms: because is not divisible by 12 and for one i, 4max - 4 = 0, the enumerator is not divisible by y² (as polynomial). Since the denominator a power of 72, the fraction can't 't be simplified. But we know that p(æ) is a polynomial. Therefore, 4max S l.
In summary, we've shown that
f((2) = ((2)); (1-11)))
(145)
where - 4 is even and non-negative. Therefore, all fi are polynomials.
Now recall that assumed p be homogeneous. But this no significant restriction: we can write any polynomial as a sum of homogeneous polynomials, each of which can be written as a linear combination of the K;, as we just showed. Adding those up leads linear combination of the K; for arbitrary polynomials. This complete the proof.
Lemma 20. Let f: R220 - Th be homogeneous of degree E IR. meaning that f(xx) Xx for all O. Then f(z) cz for some E TR. (and ifl E N, then is polynomial).
Note that this Lemma does not hold in higher dimensions in general there many more homoge- neous functions than polynomials!
Proof. For any 2 0,
f(z) ef((1)
(146)
which proves the claim by setting c:= f(1).
n
31

---

Published as a conference paper at ICLR 2022
In Appendix D we will show that the construction of angular basis elements described in Section 3.2 leads to a basis B IC with the properties required by Proposition I9 then follows that this also defines a basis of Kool vector space basis.
module. We come to the second step, turning this module basis into
Proposition 21. Let B be a basis of Kool as an -module. Then
{ k N2o. be e B}
(147)
is a basis of Koot as real vector space.
Proof. Let V E pat By assumption, there are then basis vectors .
bn E .
and coefficients
Ply...,PA Pn E R[ such that
p,b.
(148)
For each i = 1,.... there are also real coefficients dg am; such that
122
(1)
P1
(149)
Combining these equations, we get
my
0k
(i)
2k
(150)
This is real linear combination of elements of the form 2k for b B. So since v was arbitrary, the set of such elements does indeed span Kyol-
To prove linear independence, let
a bk =
(151)
for some choice of real coefficients Now we only need to note that ag E [R So we can interpret this expression as linear combination over IR [r]]]. Because B a basis, it follows that 2k 0 for all i, and thus 0. That proves linear independence.
O
As a final note, we show that the condition in Proposition 19 that the basis elements are not divisible by y2 is purely technical; any basis can easily be transformed into one that fulfills in canonical way, as formalized by the following lemma. We do not formally need this result anywher but it may prove useful if this approach is extended to other groups G because clarifies which parts of the conditions in Proposition 19 are actually important.
Lemma 22. Let B C [R[x,...., [d]] Waa)Su be basis of K. For E B, we write b 222 where k is chosen maximally such that there is (automatically unique) polynomial matrix E RiT,,.... I.]]"" XCr, Then B' := {b' b E B} a basis of K. and no 1' divisible by r?. Furthermore, if is homogeneous, then is b'.
Proof. First, note that 6' is in fact well-defined: For i = 0, = works, while for deg(h) 2 no fitting b' exists So there is some maximal 2 0 with the desired property. d' is clearly unique, namely = |x| maximally.
2h, as functions on ddd Furthermore, is not divisible by r² because k was chosen
B' is also clearly a generating set: k E, = (fir
To prove linear independence, assume that
=
(152)
Here, the degree of matrix of polynomials is the maximum of the degrees of all components (though in this case, if & larger than the minimum degree, b' exists).
32

---

Published as a conference paper at ICLR 2022
We write := max, k and then get
= r 26
Σ S.V. Σ S
(153)
This means that
for 2 - k;) 0 Vi,
(154)
which implies = 0 for all i, since 2(k k1) is non-zero everywhere except in the origin and fi is continuous. Therefore, B' is linearly independent and hence basis of K.
Now we can formulate Proposition 19 more generally: for any basis B of K. consisting only of homogeneous matrices of polynomials, the corresponding B' is basis of Kpol-
SOLUTIONS FOR IMPORTANT GROUPS
This and the next appendices describe the solutions of the PDO equivariance constraint for subgroups of O(2) and O(3). In this appendix, we describe the general form of these solutions; we recommend readers who are not interested in all the details focus on this one. Appendix [] contains proofs for some claims we make in this appendix where these proofs do not provide as much insight. Finally, Appendix contains tables with concrete solutions, is mainly relevant the implementation of steerable PDOs.
1.1 SOLUTIONS FOR SUBGROUPS OF O(2)
To solve the steerability constraint for all (compact) subgroups G < 0(2) and for arbitrary represen- tations Pin and Pout- WWiler & Cesa (2019) derive explicit bases only for irreducible representations, since the general case can then easily be computed (see Appendix for details). This works exactly the same for PDOs as well.
The angular basis elements Xa that they describe for irreps are all matrices with entries of the form (kspk)) and sin(k:)), where k differs between basis elements but the same for all entries of one matrix. For example, for G = SO(2), the frequency n irrep. i.c. (m) = g" and Pour trivial, the angular basis elements are
XI (cos(n4p), (inp)) and
X2 = (- cos(ne))
(155)
We will show how to find the steerable PDO basis using this example, but the method works exactly the same in all cases. Tables with all explicit solutions be found in Appendix K
As described in Section 3.2 we now need to multiply these matrices with the smallest power of [ar] such that all entries become polynomials. We show in Appendix [] that the necessary coefficient for cos(n)) and sin(p is [x]" so in the example above, we get
X1 = (|=|" (os())))
|a]] sin(np))
and
8 (- sin(n:), |0c| cos(n)p))
(156)
The entries are written in polar coordinates here, but they are in fact polynomials in the Cartesian coordinates 2 and æ2. More precisely, we show in Appendix ! that they are closely related to Chebyshev polynomials, based on which we derive the following explicit expressions:
[xr] R cos())p)=
= ((1))
x7 and |-s" sin(p)))
= -E-1) xf 22 (157)
(?)
iS even
is odd
This Cartesian form then allows to interpret the polynomial as differential operator, by applying the ring isomorphism D, i.e. plugging in Ø and 0g for 21 and Ø2.
If we set = 1 in our example above, which corresponds to vector field, we get simply |ær cos({p)
=
Ii and [x|] sin( = 32, so we recover the angular PDO basis
D (01, 02) div and
D(72) (-02, a,) curlap
(158)
that we already derived in Section 2.3 To get complete basis, we combine these PDOs with powers of the Laplacian.
33

---

Published as a conference paper at ICLR 2022
1.2 SOLUTIONS FOR 0(3) AND SO(3)
We now turn to the other cases for which steerable kernel solutions have been published, namely ((3) and SO(3). Like for O2), we only need consider pairs of irreducible representations. As described in [Weiler et al. 2018a Lang & Weiler (02T), we can build the corresponding angular parts out of real spherical harmonics Yim using Clebsch-Gordan coefficients. We show Appendix [ that we can apply this procedure to |x] Ymm instead of Yim to obtain the corresponding X.a. We then build the basis by combining with powers of [x|? as we described in Section 3.2 and already did for O(2). To find the corresponding differential operators D(xs), we only need Cartesian representation of the polynomials |x: Ymm which is fortunately well-known Vaarshllvvcch et al. 1988). This then again leads complete basis of the space of stecrable PDOs, with the same general form containing powers of the Laplacian.
J PROOFS FOR SOLUTIONS FOR SUBGROUPS OF 0(2) AND 0(3)
In this section, we apply the method from Appendix H to subgroups of ((2) and 0(3), by describing bases for the space of steerable kernels that satisfy the conditions Proposition 19
J.1 ((2)
As described in Appendix [1 it is sufficient to consider irreducible representations Pio and Pour and for these representations, the angular part of the kernel has a basis consisting of matrices with entries of the form cos(rep) and sin (nup) (where E 2). Crucially, is the same for all entries of one matrix (though it may differ between basis elements).
We claim that multiplying each angular basis element with plبi gives a basis of the space of poly- nomial steerable kernels Kpol (as [[(r)]-module) as described in Appendix H This follows from Proposition
19 if can prove that pll cos(p)p) and (1l sin(n)))
(i) polynomials (in x, y).
iii) homogeneous (of degree nt),
(iii) and not divisible by y²
To show that plll (nip)))) and plب sin (n:p) are polynomials, we use Chebyshev polynomials, which can among other things be seen as a generalization of the addition theorems for sin(2pp and cos(2.p). Specifically, they are families of polynomials Tù (first kind) and Un (second kind) defined for with the property that
cos(nip) = (cos up)
(159)
sin(nep) Un-1 up) sin 4p
(160)
We extend the definition to negative n by setting
T-n n Tn
for n > 0
(161)
U_ = U U-n-1 == -Un-1
(162)
for n. 2 1.
(163)
Then Eq. (159) holds for all n E Z,, as follows immediately from the parity of sin and cos.
Motivated by the close relation to Chebyshev polynomials, we then define the following notation for the matrix entries we are considering:
T, := Inl (cos = nlll cos(nip)
(164)
Ü, 1nl sin(pp)nn- (p) p |1 sin(nup).
(165)
We now prove claim (i), that T; and Ü. are polynomials in the Cartesian coordinates x, y, by using a few well-known facts about Chebyshev polynomials: First, T, has degree [n], so the highest order term in T, is plnl (cos Inl = g!!ll Similarly, Un-1 has degree |n| and the highest order term
34

---

Published as a conference paper at ICLR 2022
in Un is thus (cos (cos sin y x/ Lower order terms have additional powers of that aren't "matched" by a cosine or sine. But the second fact about Chebyshev polynomials that they (i.e. powers of y²) and thus themselves
either or odd, so all of these powers of r polynomials.
That Tn and Ün are homogeneous degree n - claim (ii) immediately clear from their definition.
It remains to show claim (iii), that they are not divisible by r². For that, use the following Lemma: Lemma 23. Let p be non-zero, hurmonic, homogeneous polynomial. Then p is not divisible by r². Here, p is harmonic if its Laplacian vanishes.
Proof. For any homogeneous polynomial p, there is a unique decomposition
+ h,
(166)
such that h is harmonic and homogeneous. Since q = 0, = p is one such decomposition for
harmonic, there is no solution with h 0 (unless p 0). So non-zero homogeneous harmonic polynomials are not divisible by y².
Now we only need to show that Tn and Ün are in fact harmonic. This can be done with brief calculation in polar coordinates:
1 0, + - 08) (r" cos(ne)) AT, = 02 +
r
(167)
= n(n - 1)r cos(nup) + n- cos(n()) - n n-2 cos(mip)
0.
The same holds for Un:
1 0, + , 08) (r" sin(np))
1
AÜ, 03
(168)
nnn 1-2 sin(n)) nr sin(n() n-2 =>?n==>in((pp)
= 0.
In summary, we have shown that T, and Ü, are homogeneous polynomials not divisible by r2, which makes Proposition 19 applicable.
In order to make the basis practically applicable, we also give explicit Cartesian expressions for T and Ü.. We restrict ourselves to non-negative to keep the notation less cluttered. Equation (161 immediately gives the case for < 0. An explicit formula for the Chebyshev polynomials of the first. kind is
[
T, (z)
Σ
n
(22
1) n-2k
(169)
2k
k=0
follows that
"T, (cosspp
22 "r"".
Th
/x2 + y2
1
12.
(-y²²) -ー
(170)
2k
k=
Σ
(-1)* 2 2 n-
2k
0=
35

---

Published as a conference paper at ICLR 2022
Similarly, the Chebyshev polynomials of the second kind are given by
Un(2) = Σ
2k + 1) (x² 1)a zn-28
+
(171)
which means that
Un = r" sin 1
= =yV x² + y2² 1-
x
Un-1
x02 yo²
(-12))²22-111- 11
(172)
y
2k + 1
"
n
(- (-1) (2
2k + 1
k=
J.2 0(3)
As already mentioned in Appendix 1.2 for 0(3) and SO(3), the irreps angular basis between irreducible representation Gordan coefficients. We refer to Weiler et al.] (2018a); Lang & Weiler (02T) for details on how this works since it exactly the same procedure whether one uses kernels or PDOs. The only fact we need to know here is that each basis element is built from spherical harmonics with the same (.
be built by combining real spherical harmonics Yim using Clebsch-
The necessary steps now very similar to those for O(2) and its subgroups. We will use r'Y.o. where we had T, and U, for O2)). Here becomes important that inside one basis element, only one appears, because this means we can multiply the entire matrix of polynomials by r!.
We would then need show that these are homogeneous polynomials not divisible by r2 but for spherical harmonics, it already very well known that Y'm are homogeneous harmonic polynomials of degree Lemma then implies that they are not divisible by 7². For explicit Cartesian expressions, we refer to e.g. Varshaloviih et al. (1988).
K SOLUTION TABLES
Using the results from Appendices H and we now very easily get complete bases for all subgroups of (()) and for all irreducible representations by transferring the solutions by Weiler & Cesal 2019). The appendix of (Weiler & Cesa 2019) contains tables with all kernel solutions; we simply replace every term of the form cos()y) with T₁ and sin(ksp) with UR to get the following tables.
For the sake of readability, we only write the polynomials T. and Üg inside the tables, though the PDOs themselves should of course be D(T,) and D(U)). Explicit formulas for Tç and Up were given in Eq. 157).
All bases described here are module bases for the space of steerable PDOs as an R[A] See Section 3.2 and Appendix for details.
K.1 SPECIAL ORTHOGONAL GROUP SO(2)
The irreducible representations of SO( are the trivial representation you and those of the form
1 SO(2) - GT.(R
9 + gg
(173)
The bases for all the combinations of these irreps are:
36

---

Published as a conference paper at ICLR 2022
Vnn
N>0
Out
(1)
(In Un), (-Un Tn)
Tm-n
-Um-n)
Um-n
-Tm-n
Un-n
mm-n
Tm-n
-Um-ni
Vm. N
(im Um
(-Um
(mmtn
Ú Untn
(-mm+n
Tm+n
Tmm
Untn
-Tm-n/
Tmnn
Umein
K.2 ORTHIGGONAL GROUP O(2)
Elements of O(2) be written tuple (r consisting of rotation SO(2) and flip E [+1]. The irreducible representations with frequency k: >> 0 are similar to those for SO(2), only with the additional flip:
1 SO(2) - GL(RR",
(r, + os.
(174)
Here, 8 is understood as a map acting on R² either as the identity or by flipping along a certain axis. In contrast to SO(2), there are now two irreducible representations for k O, namely the trivial representation (r, s) 1 and the representation s)),n(r,s) The solutions for all possible combinations of these irreducible representations are as follows:
In
10000
$1.0
Vi,n n E N
Out
$0000
(1)
0
(-ūp Tn)
11100
0
(1)
(In Ün)
11,m
(-um
Tm Um
Tm-n
-mm-n
Tmtn Unta
Um-n -Tmtn
m E N
T
Üm-n
Tm-n)
K.3
REFLECTION GROUP
The reflection group has only two irreducible representations: the trivial representation to and the representation & (s) = s. Both are one-dimensional, so all the PDOs are only x 1 matrices:
In
2/o
Out
v/u
(T,)
(Ü,)
1
(Ü,)
(T,)
To get the full basis, μ needs to take on all natural numbers (in practice, we use y up to some maximum value). Note that we assume that the reflection is with respect to the Ij -axis, both here and later for DN. To get other reflection axes, the PDOs simply need to rotated.
K.4 CYCLIC GROUP CN
The irreducible representations of Cy the same those of SO(2) but only up frequency of k N-1 J. If N is even, there is an additional one-dimensional irreducible representation, namely
N
(0)
cos
(175)
We then get the following solutions, where / ranges over Z and i over N:
37

---

Published as a conference paper at ICLR 2022
In
Mwya (if N even)
Vns Igns N/2
Out
(fir)> (Wix)
(Ranraw) (Fass/a
(-Unsuv Tasex), (Taxe UwiNN)
(Fenran)) (Ca1)/aw))
(Tyy)
(-Unsteipan PoseerppN) (Taaeera Uusterrppn))
(N even)
(0xx)
(-ummen') PmaaN
Tm-nsol Um-n
Um-n4ee mm-nten)
Um-n4tw Tm-neeN
-mm-m+
the 1sn< N/2
Um-n-enh
(mmeN Um4aN
(m+0011PN (
Tman+eN Um-nseN
Umia4ee -ImenseN
Uminass
Tm+n+4N
TmenseN
Umontel)
K.S DINEDRAL GROUP DN
Similarly to CN. the irreducible representations of Dy are the same as those of 0(2) up to frequency of k = | J. If N is even, there are two additional one-dimensional irreducible representations, namely
N
V0,N/2(0,5) 8) = 0os
0
2
(176)
N
1 (1,N/2(0, = COS
0
2
The solutions are (again with / E Z and E N):
In
voon
thoo
Vauyy2 (N even)
$1,8/2 (N even)
Ptas: 15x < N/2
Out
vl.a
(fr)
(0xx)
(Rewrnw)
(-Brew BaseN)
v1.0
(Uts)
(fxx)
(Fa-uaw)
(Fare Untax
Vo,w/2-
i (Panra))
(frr
(Onx)
(N
V1.8/2-
(Ra
(0xx
(A2x)
(N
Bom Tm min Brewween -ImanteN
Vi.-- S'
(-Umeen))
(mmser Omeeee)
-Batterrppp Tm Tntarvens
(ootterpp D (oumteree
On Tomtwee
N/2
Umimisx
L DISCRETIZATION METHODS FOR PDOS
L.1 FINITE DIFFERENCES
Finite difference methods are common in machine learning; for example, the discretization of d de as
[-1 1] or |-1 1], or of dr as [1 -2 1] all use finite difference methods. To understand where these filters come from, we need the following
result:
Proposition 24. Let II,...,,y, EN E be arbitrary but distinct grid points. Then for m S N 1, there are unique coefficients whth
(m) such that the approximation
N
p(m) (0)
~
w (m)
(177)
7=
38

---

Published as a conference paper at ICLR 2022
has error for any f E CN (R).
The coefficients wh.h) are called finite difference coefficients and approximating derivatives using Eq. (177) is the finite difference method. We will soon describe how to generalize this to higher dimensions as well.
We remark that O(hN~") is an asymptotic upper bound on the error, and it can sometimes be da
lower, even for all f. For example, the central difference discretization of dez as -2 uses
N grid points but still achieves an error of O(h2), rather than (h). For details on when such "boosted" order of accuracy occurs, see (Sadiq & Viswanath 2014).
Note that Eq. can be generalized to
(())
hm Σ
(m)) f(hx, + 2).
(178)
This follow immediately because the coefficients w(t) don't depend on the function f. so we can apply Proposition 24 T-uf.
Particularly interesting for is the case of regular grid. We can use infinitely many grid points In E Z as long as we demand that (m)) is zero for almost all n. Then we get
f(m)) (x)
w(() f(hn + x).
(179)
1m
Fixing h = 1, this is exactly the cross correlation
f(m) ((m)
(180)
we interpret (m) as function n - wn This is why, in the end, we discretize a derivative by convolving with some stencil, such as [1 -2 I]. at least regular ID grid.
Generalizing Proposition 24 to higher dimensions does not work in a straightforward way. However, if we restrict ourselves to regular grids, then finite difference methods can be easily applied PDOs. The idea very simple: a PDO such as 0g can be interpreted as first applying 0g and then 0₂ (or the other way around). So we discretize each of these with the one-dimensional finite difference method described before, and then we convolve with both filters one after the other We can also combine the two one-dimensional filter into two-dimensional filter, the outer product of the two ? The asymptotic error of this discretization will simply be the highest asymptotic error along all the dimensions, so we get similar guarantees.
1.2 RBF-FD
As mentioned, Proposition 24 does not directly generalize to higher dimensions. So to discretize a PDO on arbitrary point clouds in higher dimensions. somewhat different approach is needed.
RBF-FD is one such method and works as follows: we still want to approximate a derivative using
2th f(0) ~ Σ w fxnn)
(181)
1==1
similar to finite difference methods. Here, T, E Rdd are arbitrary (but again distinct) points. The idea now that we require this approximation to be exact if f(x) = -(I|=r - ll), where yp is an arbitrary but fixed radial basis function. In words, the approximation should become exact for a certain radial basis function centered on any of the points In. This leads to linear system, which is solved for the
coefficients w 10 For more details on both finite differences and RBF-FD, see for example Fornberg
& Flyer (2015)
"As we have seen, finite diffeerncce methods can most immediately be seen as performing a cross-correlation rather than convolution. However, we can casily switch to convolutions by flipping the filter. FFor simple PDO such as 0z this may he undesirable for computational reasons. But in practice, have PDOs that are sums such pure terms and thus don't factorize.
In practice, one often solves extended linear system containing additional low-order polynomials, hut won't 't discuss that here.
39

---

Published as a conference paper at ICLR 2022
L.3 GAUSSIAN DERIVATIVES
Discretizing PDOs using derivatives of Gaussians is very simple to describe: given grid points In E IR, we approximate using
8" f(0)
Σ (0"G(zn:o)) ((n))
(182)
where (a;; d) is a Gaussian kernel with standard deviation a centered around 0. a is a free parameter; larger will lead stronger denoising effect.
On regular grids, this again turns into a cross-correlation, with the filter being the derivative 8 G(rnn)))) evaluated the grid coordinates.
In Appendix we briefly touch on a possible interpretation of this discretization method using the distributional framework for PDOs.
M RELATION TO PDO-ECONVS
In this section, describe how PDO-eConvs (Shen et al. 2020) fit into the framework of steerable PDOs. We mostly follow the original notation from [Shen et al. al.] (2020) when describing PDO-eConvs but do make some minor changes avoid clashes and confusion with
notation.
As in our presentation, Shen et al. (2200) use polynomials to describe PDOs. One difference that they never explicitly use matrices of polynomials, because they model the feature space somewhat differently (which (where we would write e.g. p) and write x ((4) for the corresponding PDO transformed by O(d). In our notation,
we will discuss
moment). They write H for the polynomial describing PDO
(4) D(A H) D(H A-
(183)
PDO-eConvs use two types of PDO layers. The first one, V can be interpreted as a steerable PDO with Pin trivial and Pau regular. maps the scalar input to the network the internally used regular representation. The second layer type, & maps between regular representations and is used for hidden layers. At the end, pooling performed to obtain scalar output again.
The first layer type is defined as
V C (IR")) + (E(d))
(()(z, = (x()))
(184)
Here, E(d) == Rdd x S with < (d); in practice, needs to finite subgroup, i.c. C' or Dx- Elements of (d) can be uniquely written as (x, A) with æ E Rd and A E S.
There is an obvious bijection C C*0 (R(d)) = C RR),, where e e:= |S] is the order of S, i.e. the number of group elements. Concretely, define
O: 0 (B(d) + C C R*), f S (I+ AI),, ((T Aa)A
(185)
where Al...., Ao is enumeration of the group elements of S. We will therefore interpret C (B(d)) o-dimensional feature field over gd, and will show that using regular repre- sentations for this field (and trivial ones for the input) makes the PDO-eConv layers equivariant and thus steerable PDOs.
First, note that under the bijection, the first PDO-eComy layer type V becomes a C x matrix of PDOs, namely
A1 H
((Ai))
D(H)) :
(186)
A H
(Ac)
What we mean by this is that the diagram
40

---

Published as a conference paper at ICLR 2022
O (Rd)
O (E(d))
C (BR))
D(Hb))
C (R")
commutes. Concretely, we have
(z) A.) - (D(H))
(187)
So need to check whether Hy satisfies the PDO steerability constraint for trivial to regular PDOs:
H (Ax) Pregular (A) (a:)
(188)
Using the definition of Pregular (A), we can rewrite the RHS as
Pregular (A)
Pregular (A) Hz)
Σ (Ax- (H)(F)AAA
(189)
H(Ap'z))AA 2)AAAA
Σ H
Ax)eA
Here, we use basis vectors c......, CA. for R', with the same enumeration A Ag of define e. For the final step, reparameterized the sum with A == AAA.
used to
The LHS of Eq. (188) can be written as
(Ax)x) = (A1 (Ax),
A. H(x))) T
(190)
= =HH(AT Ar),
H(A_1 Az))
This is the same as the final term in Eq. (199) which proves that the PDO steerability constraint satisfied.
The second PDO-eConv layer type, mapping between regular representations, defined as
① C (E(d) + C (E(d))
(c) = Σ (x (z, AA,).
(191)
XAy different PDOs and they act E C (E(d)) by acting each of the components separately. Under the θ bijection, & becomes
(4)
O(0(e))(c), = Σ
() (5, AiA,)
XA
1=1
= Σ
(Ail (())a())) (x)
XAy
(192)
1=1
(As) 0(c)s,) (z).
(x 'd,
5=1
We therefore represent it PDO (HH) with
(Ho);, (z) H A, (A- =: Hiz
(193)
where H A A, is the polynomial that induces x (I)
' This makes the diagram
41

---

Published as a conference paper at ICLR 2022
(E
C (E(d))
-
O (RR", RRO)
D(H)) O (IR",1²)) .R²)
commute, similar to the case discussed above. So again, we need to check that H4. satisfies the PDO steerability constraint, this time for Pin and Poat both regular:
Ha (Az) Pregutar (A) (1) (A (A-)).
(194)
Writing out Hy in its components, this becomes
Hv (Ax)
Σ Progular ( (A)He)(z))a,, Prooulll (A) (A)
Hay())ess,Chn
Σ
H A (A, (A7
(195)
Σ AAA (A, A
Σ H HA A, A
Σ Hoy(A))en
The first and the last term the same, just written out in components the RHS, the steerability constraint is again satisfied. For (1), we used that Pregular (A) orthogonal, and thus
ef Pregular (A)-1 eÿj Pregular (A)" (Pregular (A)e;)*
(196)
(2) was again a reparameterization of the sum, with AA, +> A, and AA, + Aj. The other steps are only simplifications and plugging in definitions.
In conclusion, we have shown that there is a simple bijection between the feature spaces used for PDO-eConv hidden layers and the feature fields we use. and that under this bijection, PDO-eConvs correspond to steerable PDOs with regular representations (and trivial representations for the input).
It is relatively easy to adapt the argument we present for the converse direction: every equivariant PDO between two regular feature fields (or from a scalar regular one) can be interpreted as PDO-eConv layer.
N ADDITIONAL EXPERIMENTAL RESULTS
N. FLUID FLOW PREDICTION
While our classification experiments on MNIST-rot and STL-10 make use of various representations inside the networks, the input and output are always trivial. To showcase the flexibility of the steerable PDO framework, we also conduct small experiment in fluid flow prediction, where the output contains vector field.
We use the dataset provided by Ribeiro et al. (2020). It contains simulated laminar fluid flows around various objects. The inputs are all scalar fields describing the geometry of the problem. The outputs are the velocity and pressure field, the former of which is a vector field, and thus requires vector representations to get the correct equivariance (rather than only trivial and regular ones like PDO-eConvs support). In the original dataset, the fluid always flows into tube from the same direction. To better demonstrate the effects of equivariance, we randomly rotate each sample by multiple of z, making the task somewhat more challenging. During training, the rotations
42

---

Published as a conference paper at ICLR 2022
Table 3: Mean squared test error for prediction of velocity and pressure of laminar flow around different objects.
Method
Equivariance
MSE
3.20 + 0.22 2.26 + 0.14
Kernel
Cs
2.75 1 0.21
PDO
Cs
2.32 1 0.08
different each epoch to avoid disadvantaging the non-equivariant networks (i.e. we effectively use data augmentation). To ensure all inputs have the same shape, we pad the original samples with zeros to make them square before rotating them.
Our architecture and hyperparameters follow those of Ribeiro et al. 2020). The only exception is that we use a single decoder for the vector field (since we treat it as a single vector-valued output for the purposes of equivariance), whereas Ribeiro et al. (2000) used separate decoders for the two vector components some of their experiments. Just as in the STL-10 experiments, we perform no hyperparameter tuning and use the hyperparameters that [Ribeiro al. (2020) optimized for the non-equivariant network. To make the network equivariant, we replace the usual convolutions with Cs Cs-equivariant steerable kernels or PDOs. We chose the channel sizes such that all networks had approximately the same number of parameters (slightly over 800k).
T clear advantage of the equivariant networks over the non-equivariant ones Steerable PDOs perform slightly worse than steerable kernels, though the difference is within the error intervals. They still perform clearly better than any non-equivariant method. The PDO results are based on Gaussian discretization, since that performed best in our other experiments.
N.2 EQUIVARIANCE ERRORS
To check the equivariance error and indirectly the discretization eттоr, since at least equivariant layers have zero equivariance error in the continuous setting we checked how much rotating an input image changes the output of layer, compared to what the output should be under perfect equivariance. The challenge here that rotating a discrete image itself introduces some errors. To minimize those, used a large high- sigh-dimensional image, rotated it, and then scaled it down before passing into the layer, and scaled down again after that. We compared the result of this procedure to what we get by first downscaling, then applying convolutional or PDO layer, then rotating, and then downscaling again. Effectively, all rotations thus happen at large resolutions, which should minimize artifacts.
Table shows the relative equivariance errors (as multiples of lc-6). These errors are for randomly initialized layers (averaged over 10 initializations). As discussed in the main text, the asymptotic error bound for finite difference discretization does not lead to a particularly low error in practice.
N.3 RESTRICTION EXPERIMENTS
Table shows additional results on MNIST-rot. The general architecture and hyperparameters are the same as in the experiments in Section 4 with regular representations, using basis. However, in the experiments in this section, the first five layers are ceuiivivvri, while the final PDO/convolutional layer is -ieeequivariant. The motivation for this that while the input images do not have global reflectional symmetry, such symmetry occurs smaller scales, so that stronger equivariance in earlier layers might be helpful.
However, we don't 't observe clear improvements over pure Cie-equivariance. A reason could be that even the -ue-equivrriant networks are already very parameter efficient compared to classical CNNs, so that parameter efficiency and equivariance are not bottleneck anymore. It is also possible that the architecture would need to be adapted slightly to profit from the Di6 Dis-cquivariant layers.
Note that non-qquivariant performance is significantly worse than the one obtained by Ribeiro et al. this is because we randomly rotated the samples, resulting in more challenging task.
2020
43

---

Published as a conference paper at ICLR 2022
Table 4: Relative equivariance errors for C1e test image, averaged over 10 random initializations of the layer. As orientation, we also include non-qquivariant (vanilla) convolutions.
Method
Stencil
Error [1e-6]
Vanilla
3 x
32716 + 5484
convolution
x 5
32785 + 5620
x
5.0 + L.0
Kernels
x
5.0 1.3
x
4.8 + 1.1
FD
x
6.6 - 1.2
3
5.9 1.2
RBF-FD
x
6.1 1 1.1
3
4.9 1.7
Gauss
5 5x
6.4 4 0.8
Table MNIST-rot results restriction from Dis to Cie eguivariance. Test errors standard deviations are averaged over six runs. See main text for details on the models.
Method
Stencil
Error [%]
Params
3
0.717 + 0.022
709K
Kemels
x
0.710 + 0.020
1.1M
3 x
1.248 1 0.060
709K
FD
x
1.436 + 0.063
947K
3
1.396 + 0.059
709K
RBF-FD
x
1.565 1 0.048
947K
3
0.806 + 0.047
709K
Gauss
x
0.778 = 0.051
947K
44

---

Published as a conference paper at ICLR 2022
Kernel
RBF-FD
Gauss
(a) 3 x 3 stencils
Kernel
FD
RBF-FD
Gauss
(b) 5 x 5 stencils
Figure 4: Stencils of basis filters for a trivial to regular layer. Each row a different method and contains six arbitrarily selected filters from the basis (which is not the entire basis). All the settings are those that were actually used for the MNIST-rot experiments.
N.4 STENCIL IMAGES
Figure 4 contains examples of stencils used during the MNIST-rot experiments. For the x 3 stencil, the different methods yield qualitatively similar results (though FD and RBF-FD have fewer stencils that make of the four corners). But for 5 5 stencils, kernels and Gauss discretization make significantly more use of the outer stencil points than FD and RBF-FD.
45

---

Published as a conference paper at ICLR 2022
Table 6: Architecture for MNIST-rot experiments
Layer
Output fields
Conv block
16
Conv block
24
Max pooling
Conv block
32
Cony block
32
Max pooling
Cony block
48
Conv block
64
Group pooling
Global average pooling
Fully connected
64
Fully connected + Softmax
10
O DETAILS ON EXPERIMENTS
O.1 MNIST-ROT EXPERIMENTS
For the MNIST-rot experiments, use architecture similar to from (Weiler & Cesa 2019). Table/6] a listing of all the layers. Each conv block consists of a 3 x or 5 x steerable layer, either convolutional or PDO, followed by nachhnoormalization and ELU nonlinearity. The output fields are the number of Cıe-regular feature fields that is used; in the case the Vanilla CNN and for quotient representations, the number of fields is adjusted so that the parameter count is approximately preserved.
Cin/Ca 3 4ptrivial the representation, For the quotient experiments, we use 5pregular C Can/Ca $ 2qqu P
where the numbers are scaled to reach the same parameter count as the model with only regular representations. This combination of representations is the same used by Weiler & Cesal (2019) and we refer to their appendix for motivation on why we need to combine different representations when using quotients.
We trained all MNIST-rot models for 30 epochs with Adam <Kingma & Ba 2015) and a batch size of 64. The training data normalized and augmented using random rotations. For the final training runs, we used the entire set of 12k training plus validation images, as is common practice on MNIST-rot. The initial learning rate was 0.05, which was decayed exponentially after burn-in of 5 epochs at rate of 0.7 per epoch. We used dropout of 0.5 after the fully connected layer, and weight decay of 1c-7.
These hyperparameters are based on those used in (Weiler & Cesa 2019); the main difference is that we use another learning rate schedule, which works better than the original one for all models.
For the Gaussian discretization models, we use standard deviation of a = 1 for 3 x 3 stencils and 1.3 for 5 x stencils; we chose these values by visual inspection of the stencils, with the aim that full use is made of the stencil (see Appendix Nj. The RBF-FD discretization uses third-order polyharmonic basis functions, i.e. sp(r) = =
O.2 STL-10 EXPERIMENTS
For the STL.110 experiments, we used exactly the same architecture and hyperparameters as Weiler & Cesa (2019), which in turn are essentially those Dries & Taylor (017)). This means we train Wide-ResNet 16-8 (Zagoruyko & Komodakis 2016) for 1000 epochs, with SGD and Nesterov momentum of 0.9, a batch size of 128 and weight decay of 5e-4. We begin with a learning rate of 0.1 and divide it by 5 after 300, 400, 600 and 800 epochs. For data augmentation, pad the image by 12 pixels, then randomly crop a 96 x 96 pixel patch, randomly flip horizontally, and apply Cutout Vries & Taylor 2017) with cutout size of 60 60 pixels.
46

---

Published as a conference paper at ICLR 2022
O.3 COMPUTATIONAL REQUIREMENTS
We performed our experiments on an internal cluster with GeForce RTX 2080 Ti and 6 CPU cores. single run of MNIST-rot model took about 12 minutes and of the STL-10 model about 5.5h. Training the fluid flow prediction model took about 45 minutes. Multiplying this by the number of experiments we did and by six runs with different seeds, the MNIST-rot results took about 34 hours to produce, the STL-10 results about 264 hours, and the fluid flow results about 18 hours. The initial tests we did to debug and find a good learning rate schedule (on MNIST-rot) took much less time than that. So we estimate that producing this paper took around 350 GPU-hours on the GeForce RTX 2080
47


Reviews:
[{"review_id": "xuO-TiHkEgh", "reviewer": "Reviewer_p5Vs", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strength:\n+ The paper is very rigorously and well-written. The layout of the paper is very clean.\n+ The paper tries to bridges the idea between deep learning and physics, drawing attention in particular to convolution between features and PDOs between vector fields.\n+ Even though the derivation of the PDO G-steerablility constraint follows similarly from that of kernel steerability constraint, the theory developed is interesting. In particular, it is interesting to see a special example in section 2 that equivariant PDOs are combinations of well-known operators such as Laplacian and divergence.\n+ Connections between convolutions and PDOs have also been made through Schwartz distributions in the appendix, which I have to admit that I have not read in full.\n\nWeakness:\n- It seems that in Table 1 and 2, equivariant PDOs typically underperform steerable CNNs. The authors did explain the reason being the locality of PDOs, but such underwhelming results do in some sense limit the utility of the developed equivariant PDOs.\n- It would be great to see examples where PDOs have a significant advantage over steerable CNNs.", "summary_of_the_paper": "The authors developed the general theory of equivariant partial differential operators (PDOs) between feature fields on the Euclidean space. Given an arbitrary group G, the G-steerability constraint is derived that fully characterized when a PDO is equivariant between vector fields for given representations. Experiments on the rotated-MNIST and STL-10 datasets have been conducted to compare the performance of equivariant-PDOs and equivariant steerable kernels.", "summary_of_the_review": "The theoretical contribution of the paper is significant. Even though experimental results are not as satisfactory to demonstrate the utility of equivariant PDOs, I am leaning towards accepting the paper.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "fwXhB6FsSoN", "reviewer": "Reviewer_mpGT", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "This paper is exceptionally well written and covers an impressive body of work across multiple different domains. Personally, I was extremely impressed with the production quality and thoroughness of the Appendix as well as the ambitious research goal of introducing equivariant PDOs in the main paper. The main technical results are both novel and illuminating and gracefully extends prior work on $E(2)$-CNNs and their theory.\n\nI do not have many strong criticisms regarding the work but rather a few minor questions and comments. The largest of which is that while $G$-steerable equivariant networks guarantee equivariance between layers there exists another body of work that achieves equivariance in a slightly different manner. Specifically, prior works also achieve equivariance in expectation using a Monte-Carlo estimation of the group convolution integral (see [1]). These approaches differ in that they do not treat data as associated vector bundles and also equivariance is achieved over functions on the group $G$ itself necessitating a lifting operation. While this school of thought is different than the $G$-steerable variety as presented in this paper it would be great to harmonize both directions with a discussion on their differences especially when it comes to practical applications. Finally, the paper could be strengthened a bit more if the discussion on the discretization error for the various discretization methods was elevated from the appendix to the main paper. It would be good here to get some numerical quantification of the errors for the datasets and architectures considered as this is an important insight which informs practice.\n\n***Minor Comments***\n- I had a bit of a hard time understanding how a group element $g$ acts on a polynomial $x$ in the main paper. Could the authors provide a bit more exposition---if it doesn't already exist---in an appendix?\n- The notation in section 2.1 was a bit confusing. Specifically, $c_{\\alpha}: \\mathbb{R}^d \\to \\mathbb{R}$ but right after proposition 1. $c_{\\alpha} \\in \\mathbb{R}$. Is this an artifact of picking constants due to enforcing translational equivariance?\n\n\n[1] Finzi, Marc, et al. \"Generalizing convolutional neural networks for equivariance to lie groups on arbitrary continuous data.\" International Conference on Machine Learning. PMLR, 2020.", "summary_of_the_paper": "This paper introduces equivariant Partial Differential Operators as a drop in replacement for equivariant kernels in $G$-steerable networks. The paper provides a thorough treatment on the subject matter and gives the necessary equivariant constraints for PDO's in a similar flavor to $G$-steerable kernels. In addition, explicit solutions to these constraints are provided for subgroups of $O(2)$, $O(3)$ and $SO(3)$. Finally, the paper also empirically validates equivariant PDO's on rotation MNIST and STL-10 highlighting important discussion points on the discretization error.", "summary_of_the_review": "The paper studies equivariant Partial Differential Operators towards building equivariant networks. The work is both novel, interesting, and brings new perspectives to the equivariant networks literature while maintaining an exceptional level of polish in both writing and execution.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "CbRKlXAOyoW", "reviewer": "Reviewer_BPgr", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The paper provides a very solid theoretical analysis for equivariant PDO. Different from other works on equivariant PDO, this paper enforces more properties of PDO such as divergence and gradient. Also some theoretical findings are useful to me. For example, the locality of PDO is not helpful. The PDO and convolution can be unified into a single framework.\n\nHowever, I still have several questions and concerns which I will detail below:\n\n- While theoretical findings are interesting, from my point of view, it’s not surprising and novel. For example, PDO is a specific form of convolution. I think this has been revealed by previous works in particular in the context of the diffusion process.\n\n- Regarding the proposed equivariant PDO, I appreciate the deep theoretical analysis. However, I am not very attracted. First, I believe the PDO-eConv should have already revealed these analyses more or less. Second, the equivariant PDO is worse than other works experimentally. In other words, I am not very sure of the significance of the proposed PDO to our community given the limited performance and novelty.\n\n- Experimental analysis is also very limited. What are the advantages of the proposed PDO compared with previous extensive steerable convolution? I recommend having more results. \n\n- Do you have any insight why the performance of the proposed PDO is worse than convolution? Is that potentially because of the isotropic nature of PDO? If so, I recommend the analysis in this direction. \n", "summary_of_the_paper": "The paper develops the theory of equivariant partial differential operators -- a steerable PDO which is equivariant under any given symmetry. Interestingly, this work reveals the relation between convolution and PDO by unifying them into a single framework. The work also provides the rigorous theoretical analysis and proof. The paper also provides experimental results to validate its theoretical analysis. ", "summary_of_the_review": "The paper theoretically develops the steerable PDO which enlarges the family of equivariant operators. However, the proposed method is quite limited in terms of performance. Also, the theoretical findings are not very surprising to me given the previous literature. Thus, I doubt the significance of this work to our community. But in case of any misunderstanding, I would like to hear more from the authors during rebuttal. \n\n> Post-discussion: the authors addressed my concerns. So I'm very happy to improve my rating. ", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper develops steerable partial differential operator and show how it can be used to build equivariant network. Experimentation on rotated MNIST and STL10 show the merits of the proposed method. Reviewers agreed on the significance of the work and that it brings new perspective on equivariance that would be interesting to the ICLR community. Accept
Decision: Accept (Poster)