Residual Pathway Priors for Soft Equivariance Constraints
Authors: Marc Anton Finzi
Gregory Benton
Andrew Gordon Wilson
Venue: NeurIPS 2021 Poster (2021)
Keywords: symmetry
equivariance
group equivariance
priors

Abstract: Models such as convolutional neural networks restrict the hypothesis space to a set of functions satisfying equivariance constraints, and improve generalization in problems by capturing relevant symmetries. However, symmetries are often only partially respected, preventing models with restriction biases from fitting the data. We introduce Residual Pathway Priors (RPPs) as a method for converting hard architectural constraints into soft priors, guiding models towards structured solutions while retaining the ability to capture additional complexity. RPPs are resilient to approximate or misspecified symmetries, and are as effective as fully constrained models even when symmetries are exact. We show that RPPs provide compelling performance on both model-free and model-based reinforcement learning problems, where contact forces and directional rewards violate the assumptions of equivariant networks. Finally, we demonstrate that RPPs have broad applicability, including dynamical systems, regression, and classification.

Residual Pathway Priors for Soft Equivariance Constraints
Mare Finzi*
Greg Benton" New York University
Andrew Gordon Wilson New York University
New York University
Abstract
Models such as convolutional neural networks restrict the hypothesis space to a set of functions satisfying equivariance constraints, and improve generalization in problems by capturing relevant symmetries. However, symmetries are often only partially respected, preventing models with restriction biases from fitting the data. We introduce Residual Pathway Priors (RPPs) as method for converting hard ar- chitectural constraints into soft priors, guiding models towards structured solutions while retaining the ability to capture additional complexity. RPPs are resilient approximate misspecified symmetries, and models even when symmetries are exact. We show that RPPs provide compelling performance on both model-free and model-based reinforcement learning problems, where contact forces and directional rewards violate the assumptions of equivariant networks. Finally, demonstrate that RPPs have broad applicability, including dynamical systems, regression, and classification.
effective as fully constrained
Introduction
Central to the expanding application of deep learning structured data like images, text, audio, sets, graphs, point clouds, and dynamical systems, has been a search for finding the appropriate set of inductive biases to match the model the data. These inductive biases, such as recurrence [4], local connectivity [29], equivariance [10], or differential equations (8), reduce the set of explored hypotheses and improve generalization. Equivariance in particular has had large impact ruling out a large class of meaningless shortcut features in many distinct domains, such as the ordering of the nodes in graphs and sets or the coordinate system chosen for an image.
allows
disadvantage hard coding these restrictions is that this prior knowledge may not match reality. scene may have long range non-local interactions, rotation equivariance may be violated by preferred camera angle, or dynamical system may occasionally have discontinuous transitions. In particular, symmetries are delicate. A small perturbation like adding wind breaks the rotational symmetry of pendulum, and bumpy or tilted terrain could break the translation symmetry for locomotion. these cases we would like incorporate our prior knowledge in a way that admits our own ignorance, and allows for the possibility that the world is more complex than we imagined. We aim to develop an approach that is more general, and can be applied when symmetries are exact, approximate, non-ex non-existent.
The Bayesian framework provides mechanism for expressing such knowledge through priors. In much of the past work on Bayesian neural networks, the relationship between the prior distribution and the functions preferred by the prior is not transparent. While it is easy specify different variances for different channels, or to use heavy tailed distributions, it is hard know how high level properties meaningfully translate into these low level attributes. Ultimately priors should represent our prior beliefs, and those beliefs we have are about high level concepts like the locality, independence. and symmetries of the data.
"qqal Contribution
35th Conference on Neural Information Processing Systems (NcurIPS 2021).

---

EMLP Strong Inductive Blases
Inputs
Prior Mass
RPPs Flexible with bias towards
Restrictive Layer Strong inductive
Flexible Layer
Weak inductivv
blases
blases
equivariance
Constrained solutions
Unconstrained
High Flexibility
solutions
Approximaa symmelry Full Bymmetry
No Symmetry
Dutputs
(a) Priors over Equivariant Solutions
(b) Structure of RPP Models
approach to modeling. Highly flexible models like MLPs lack the inductive biases to assign high evidence to key datasets, while models with strict equivariance constraints
Figure 1: Left: RPPs encode Occam's
not flexible enough to support problems with only approximate symmetry. Right: The structure of RPPs. Expanding the layers into a sum of the constrained and unconstrained solutions while setting the prior favor the constrained solution, leads the more flexible layer explaining only the residual of what is already explained by the constrained layer.
To address the need for more interpretable priors we introduce Residual Pathway Priors (RPPs), a method for converting hard architectural constraints into soft priors. Practically, RPPs allow to tackle problems in which perfect symmetry has been violated, but approximate symmetry is still present, is the case for most real world physical systems.
RPPs have a prior bias towards equivariant solutions, but are not constrained to equivariance, The choice of RPPs can be viewed from Occam's razor perspective, shown in Figure Ta in which seek to use models that have both the correct inductive biases and level flexibility [41. As we find with problems in which symmetries exist, highly flexible models with weak inductive biases like MLPss fail to concentrate prior mass around solutions that exhibit any symmetry. On the other hand when symmetries only approximate, the strong biases of constrained models like Equivariant Multi-Layer Perceptrons (EMLP) [14) fail to provide support for the observations. As middle ground between these two extremes, RPPs combine the inductive biases of constrained models with the flexibility of MLPs to define a model class which excels when data show approximate symmetries.
In the following sections introduce method and show results across variety of domains. We list our contributions and the accompanying sections below:
1. We propose Residual Pathway Priors as a mechanism imbue models with soft inductive biases, without constraining flexibility.
2. While our approach is general, straints into soft equivariance priors (Section 4).
RPPs to show how to turn hard architectural con-
3. We demonstrate that RPPs are robust to varying degrees of symmetry (Section 5) RPPs perform well under exact, approximate, or misspecified symmetries.
4. Using RPP on the approximate symmetries in the complex state spaces of the Mujoco locomotion tasks, we improve the performance of model free RI. agents (Section
We provide PyTorch implementation of residual pathway priors https://gi thub. thub.
com/mt
2 Related Work
The challenge of equivariant models not being able to fully fit the data has been identified in number of different contexts, and with different application specific adjustments to mitigate the problem. Liu et al. 32 observe that convolutional networks can be extremely poor at tasks that require identifying or outputting spatial locations an image as result of the translation symmetry. The authors solve the problem by concatenating a coordinate grid to the input of the convolution layer. Constructing translation and rotation equivariant GCNNs, Weiler and Cesa 5x find that in order to get the best performance CIFAR- 10 and STL-10 datasets which have preferred camera orientation, they

---

must break the symmetry which they do by using equivariance to progressively smaller subgroups in the later layers. Bogatskiy et al. 66] go great lengths to construct Lorentz group equivariant networks for tagging collisions in particle colliders only to break it by introducing dummy inputs that identify the collision axis. van der Wilk al. [50) use the marginal likelihood to learn approximate invariances in Gaussian processes from data. In a similar approach, Benton et al. [5] explore related concept of learning the extent of the symmetry using the reparametrization trick and test time augmentation. While sharing some commonalities with RPP, this method is not aimed at achieving approximate equivariance and cannot bake equivariance into the model architecture.
Outside of equivariance, adding model outputs to much more restrictive base model has been a fruitful idea employed in multiple contexts. The original ResNet 20 drew on this motivation, with shortcut connections. Johannink et al. [24]. Silver et al. Learning. whereby the RI. problem split into user designed controller using engineering principles and flexible neural network policy learned with RI. Similarly, in modeling dynamical systems, one approach is to incorporate base parametric form informed by models from physics or biology, and only learn a neural network to fit the delta between the simple model and reality [25 33
45 proposed Residual Reinforcement
There have been several works tackling symmetries and equivariance in RL, such as permutation equivariance for multi agent RL [46, [3 BT. as well exploring reflection symmetry for continuous control tasks M and discrete symmetries in the more general framework of MDP homomorphisms [49]. However, in each of these applications the symmetries need to be exact, and the complexities of real data often require violating those symmetries. Although not constructed with this purpose. some methods which use regularizers to enforce equivariance [48] could be used for approximate symmetries. Interestingly, the value of approximate symmetries of MDPs has been explored in some theoretical work 42 47 but without architectures that can make use of it. Additionally data augmentation, while not able to bake in architectural equivariance, has had successful application for encouraging equivariance on image tasks 277 and recently even on tabular state vectors 30 36
Background
In order develop our method, we first review the concept of group symmetries, how representations formalize the way these symmetries act on different objects.
Group Symmetries In the machine learning context, a symmetry group G can be understood as a set of invertible transformations under which an object is the same, such reflections rotations. These symmetries can act on many different kinds of objects. A rotation could act on a simple vector, 2d array like image, a complex collection objects like the state space of robot, more abstractly on an entire classification problem or Markov Decision Process (MDP).
Representations The way that symmetries act on objects is described by a representation. Given an object in an n-dimensional vector space V, group representation is mapping p G - Rx yielding a matrix which acts on V. Vectors v E V are transformed e + p(g)v. In deep learning, each of the inputs and outputs to our models can be embedded in some vector space: m m sized rgb image exists in 333". and node valued function on graph of m elements exists within R The representation o specifies how each of these objects transform under the symmetry group G.
These representations can be composed of multiple simpler subrepresentations, describing how each object within collection transform. For example given the representation P1 of rotations acting on vector in R and a representation P2 of how rotations act on a 3 matrix, the two objects
(1(g)
concatenated together have representation given by pi(g) + p2(9)
Pz(g) where the
0
two matrices are concatenated along the diagonal. Practically this means we can represent intricate and multifaceted structures by breaking them down into their component parts and defining how each part transforms. For example, we may know that the velocity vector, an orientation quaternion, a joint angle, and control torque all transform in different ways under a left-right reflection, and one can accommodate this information into the representation.
Equivariance Given some data with representation Pin and Y with representation Pout: we may wish to learn some mapping f X + Y. A model f is equivariant [10), if applying the symmetry
3

---

transformation to the input is equivalent to applying it to the output
((Pm(g)x) = Pout (z)
In other words, it is not the symmetry of X or Y that is relevant, but the symmetry of the function mapping from X to Y. If the true relationship in the data has symmetry. then constraining the hypothesis space to functions/ f that also have the symmetry makes learning easier and improves generalization [12). Equivariant models have been developed for a wide variety of symmetries and
data types like images 10, 54] [56 [53], sets [55 35 graphs [44] point clouds 3 15 44), dynamical
systems
13 jets and other objects [5T
Residual Pathway Priors
In this section, we introduce Residual Pathway Priors. The core implementation of RPP is to expand each layer in model into a sum of both a restrictive layer that encodes the hard architectural constraints and a generic more flexible layer but penalize the more flexible path via a lower prior probability. Through the difference in prior probability, explanations of the data using only the constrained solutions are prioritized by the model; however, if the data more complex the residual between the target and the constrained layer will be explained using the flexible layer. We can apply this procedure to any restriction priors, such as linearity, locality, markovian structure, and of course equivariance.
The Residual Pathway Prior draws inspiration from the residual connections in ResNets 20 211..
whereby training stability and generalization improves by providing multiple paths for gradients to flow through the network that have different properties. One way of interpreting a residual block and shorteut connection f(x) x + h(x)) in combination with 12 regularization, either explicitly from weight decay or implicitly from the training dynamics (39] is as a prior that places higher prior likelihood on the much simpler identity mapping than on the more flexible function hzx). In this way. h(x) need only explain the the difference between what is explained in the previous layer (passed
through by 2) and the target.
We can leverage a similar approach to convert the hard constraints of a restriction prior specified through a given network architecture into a soft prior that merely places higher prior likelihood on such models. Supposing we have an equivariant layer A(x), and a more flexible non-equivariant layer Bxx) which contains A as a special case, we can allow the model to explain part of the data with A and part with B by forming the sum A(x) + Bzx). A given a prior likelihood on the size of A and B such as p(A) exp (-1IA 12 and pBB) x exp ( /2oz) with oa ob
MAP optimized model will favor explanations of the data using the more structured layer A and only resort to using layer B explain the difference between the target and what is already explained by the more structured model A. Adding these non-equivariant residual pathways to each layer of an equivariant model, we have model that has the same expressivity of network formed entirely of B layers, but with the inductive bias towards a model formed entirely with the equivariant A layers. We term this residual pathway prior (RPP).
To make the approach concrete, we first consider constructing equivariance priors using the constraint solving approach known as Equivariant Multi-Layer Perceptrons (EMLP) from Finzi al. 14]
Equivariant MLPs EMLPs provide method for automatically constructing exactly equivariant layers for any given group and representation by solving set of constraints. The way in which the vectors are equivariant is given by formal specification of the types of the input and output through defining their representations. Given some input vector space Vnn with representation Pin and some output space Vant with representation Pout. the space of all equivariant linear layers mapping Vnn + Vout satisfies
Vg E G:: Pour W WW (g)
These solutions to the constraint form subspace of matrices Bo X which can be solved for and described by a r dimensional orthonormal basis Q E _ Linear layers can then be parametrized in this equivariant basis. The elements of W can be parametrized vec(W) = Q8 for B E RR for the linear layer + We, and symmetric biases can be parametrized similarly.
Equivariance Priors with EMIP In order to convert the hard equivariance constraints in EMLP into a soft prior over equivariance that can accomodate approximate symmetries, can apply the

---

RPP procedure from above to each these linear layers in the network. Instead of parametrizing the weights W directly the equivariant basis vec(W) Q3, we can instead define W as the sum W = A + B of an equivariant weight matrix vec A) Q with Gaussian prior over B and an unconstrained weight matrix B ~ ~N(0,, of with a Gaussian prior. A Gaussian prior over (O, d is equivalent to the Gaussian prior ~ NN( o QQQQT): Since Q is an orthogonal matrix, we can breakdown the covariance of B into the covariance in the equivariant subspace Q as well as the covariance in the orthogonal complement P, of = QqQ + of Therefore the sum of the weight matrices is distributed as B = W ~ (O (o² + +Q])Q + PPTP
Regardless of the values of the prior variances of and oz. solutions in the equivariant subspace QQT are automatically favored by the model and assigned higher prior probability mass than those in the subspace PPT that violate the constraint. Even if σ > .a. the model still favors equivariance because the equivariance solutions are contained in the more flexible layer A. We show in Section (52 that RPPs are insensitive to the choice of Jg and og provided that oe is large enough be able to fit the data. By replacing each of the equivariant linear layers in an EMLP with a sum of an equivariant layer and an unconstrained layer and adding in the negative prior likelihood to the loss function, we arrive at RPP-EMLP that can accommodate approximate or incorrectly specified symmetries. |
RPPs With Other Equivariant Models While in EMLP these equivariant bases are solved for explicitly, the argument holds for the linear layers in other equivariant networks in precisely the same way. good example of this is the translationally equivariant convolutional neural network (CNN) which can be viewed as restricted subset of a fully connected network. Though the layers are parametrized as convolutions, the convolution operation can be expressed as a Toeplitz matrix residing within the space of dense matrices. Adding the convolution to a fully connected layer and choosing prior variance o² and of over each, we have the same RPP prior
W W~N + +
(I)
where Q is the basis of (bi-) (bi-)Toeplitz matrices corresponding to x 3 filters. This RPP CNN has the biases of convolution but can readily fit non translationally equivariant data. Likewise for other equivariant models like GCNNs [I0), and we can even apply the RPP principle to the breaking of a given symmetry group subgroup.
5 How and Why RPPs Work
In this section we explore how and why RPPs work on a variety of domains, applying RPPs in settings where constraints are known to be helpful, cannot fully describe the problem, and are misspecified.
5.1 Dynamical Systems and Levels of Equivariance
In order to better understand how and why residual pathway priors interact with the symmetries of the problem we move settings in which can directly control both the type of symmetry and the level to which the symmetries are violated. We examine how RPPs coupled with EMLP networks perform (RPP-EMIP) on the inertia and double pendulum datasets featured in Finzi et al. 14 in 3 experimental settings: (i) the original inertia and double pendulum datasets which preserve exact symmetries with respect to the to (()) and ((2) groups respectively; (ft) modified versions of these datasets in which add an additional terms (such as wind on the double pendulum) that lead to approximate symmetries; and (izi) versions with misspecified symmetry groups which break the symmetries entirely (described in Appendix C).
The results for these 3 settings given in Figure 4 Across all settings RPP-EMLP match the performance of EMLP when symmetries are exact, perform as well as an MLP when the symmetry is misspecified and better than both when the symmetry is approximate. For these experiments use prior variance of o² = 10s on the EMLP weights and of = on the MI. weights.
Exact Symmetries As part of the motivation, RPPs should properly attribute prior mass to both constrained and unconstrained solutions, we test cases in which symmetries are exact, and show that
For the EMLP that uses gated nonlinearities which do not always reduce to standard Swish, we likewise add a more general Swish weighted by a parameter with prior variance of.
5

---

Inertia
Modified Inertia
Inertia; SL(3)
10
MSE
10
MSE
10
MSB
10
10-1
10
RPP
MLP
EMLP
MLP
EMLP
10"
Double Pendulum
Windy Double Pendulum
10
Pendulum; SO(3)
10-a
MSE
MSE
MSE
10-2
10
5:10-
RPP
NLP
EMLP
10-4
RPP
MLP
EMLP
RPP
MLP
EMLP
(a) Exact Symmetries
(b) Approximate Symmetries
(c) Mis-specified Symmetries
Figure 2: A comparison of test performance over 10 independent trials using RPP-EMLP and equiva- lent EMLP and MLP models on the inertia (top) and double pendulum (bottom) datasets in which we have three varying levels of symmetries. The boxes represent the interquartile range, and the whiskers the remainder of the distribution. Left: perfect symmetries in which EMLP and the equivariant components of RPP-EMLP exactly capture the symmetries in the data. Center: approximate symme- tries in which the perfectly symmetric systems have been modified to include some non-equivariant components. Right: mis-specified symmetries in which the symmetric components of EMLP and RPP-EMLP do not reflect the symmetries present in the data.
RPP-EMLP is capable of performing par with EMLP which only admits solutions with perfect symmetry. The results in Figure 4a show that although the prior over models as described RPP-EMML is broader than that of EMLP (as we can admit con-equivariant solutions) in the presence of perfectly equivariant data RPP-EMIP do not hinder performance, and we are able to generalize nearly as well as the perfectly prescribed EMLP model.
Approximate symmetries To better showcase the ideas of Figure T we compare RPP-EMIPPs to EMLPs and MIPs on the modified inertia and windy pendulum (Figure ??) datasets. In these datasets we can think about the systems as primarily equivariant but containing non-equivariant contributions. As shown in 4b these problems are best suited for RPP-EMLP as MLPs have no bias towards the approximately symmetry present in the data, and EMLPs are overly constrained in this setting.
Misspecified symmetries In contrast to working with perfect symmetries and showing that RPP- EMLPs are competitive with EMLPs, we also show that when symmetries are misspecified the bias towards equivariant solutions does not hinder the performance of RPP-EMLP For the inertia dataset we substitute the group equivariance in EMLP from (33 to the overly large group SL(3) consisting of all volume and orientation preserving linear transformations, not just the orthogonal ones. For the double pendulum dataset, we substitute ((2) symmetry acting on R with the larger SO(3) rotation group that contains it but
a symmetry of the dataset.
By purposefully misspecifying the symmetry in these datasets we intentionally construct EMLP and RPP-EMLP models with incorrect inductive biases. In this setting EMLP is incapable of making accurate predictions as has a hard constraint on an incorrect symmetry. Figure 44c shows that even in cases where the model is intentionally mis-specified that RPPs can overcome poorly aligned inductive bias and recover solutions that perform as well as standard MLPs, even where EMLPs fail.
5.2 Prior Levels of Equivariance
To test the effect of prior variances we use the modified inertia dataset, which represents version of a problem in which perfect equivariance has been broken by adding new external forces to the dynamical system. Shown in Figure (right) a comparison of mean squared error on test data as function of the prior precision terms on both the equivariance and basic weights. As general trend we see that when the regularization on the equivariant weights too high (equivalent to a concentrated prior around 0) we find instability in test performance, yet when we apply a broad prior to the equivariant weights performance is typically both better in terms of MSE, and more stable to the choice of prior the basic model weights.

---

Modified Inertia Inertia
Modified Inertia Test. MSE
5x10
Epoch
If mnn
Test MSE
Bealc
13
100
2ML
10
10
10
10
30
Equivariant Precision
10:15
10 10.25
10"26
Log Equivariance Eror
Kernel density estimators of log equivariance error across training epochs for 10 independently trained networks. Here the color denotes the dataset these models were trained on. Treating these samples as proxy for posterior density, we see that on the non-equivariant Modified Inertia dataset, the posterior shifted upward to match the level of equivariance in the data during training. Right: Test MSE function of the weight decay parameters on the equivariant and basic weights on the modified inertia dataset. We observe that so long as the prior in the basis of equivariant weights is broad enough, we can achieve low test error with RPPs.
Figure 3: Left:
As the prior variances over the equivariant basis Q and the non-equivariant basis P describe our bias towards or away from equivariant solutions investigate how the choice of prior variance relates to the level of symmetry present in a given dataset. the windy pendulum dataset of Figure ?? we have control over the level of wind and thus how far system is from perfect equivariance.
5.3 Posterior Levels of Equivariance
RPPs describe a method for setting prior over equivariance, and in the presence of new data we expect the posterior distribution over equivariance to change accordingly. Using samples from deep ensemble query points of high density the posterior we estimate how the distribution over equivariance error progresses through training. Recalling that with an equivariant function f we have pr(g)s(x) = ((pi())))))))))) ())=)), we compute equivariance error as
||a bll where RellErr(a,)) ||a|| + [[b]]*
EquivErr(f.e)
We train one deep ensemble on the inertia dataset which exhibits perfect symmetry, and another on the modified inertia dataset which has only partial symmetry, with each deep ensemble being comprised of 10 individual models using the same procedure as in Section [.] In Figure 3 (left) we see that throughout training the models trained on the modified inertia concentrate around solutions with substantially higher equivariance error than models trained on the dataset with the exact symmetry. This figure demonstrates one of the core desiderata of RPPs: that we are able converge to solutions with an appropriate level of equivariance for the data.
5.4 RPPs and Convolutional Structure
Using the RPP-Conv specified by the prior in Eq we apply the model CIFAR-10 classification and UCI regression tasks where the inputs are reshaped to zero-padded two dimensional arrays and treated as images. Notably, the model is still an MLP and merely has larger prior variance in the convolutional subspace. As a result can perform well on image datasets where the inductive bias aligned, as well as on the UCI data despite While retaining the flexibility of an MLP, the RPP performs better than the locally connected MLPs trained with 3 3-lasso in Neyshabur 38 which get 14% error on CIFAR-10. The full details for the architectures and training procedure are given in Appendix
being an image dataset shown Table [
6 Approximate Symmetries in Reinforcement Learning
Both model free and model based reinforcement learning present opportunities to take advantage of structure in the data for predictive power and data efficiency. On the one hand stands the use of
7

---

CIFAR-10
Energy
Fertility
Pendulum
Wine
MLP
37.61 0.56
0.39 = 0.48
0.049 + 0.0044
4.65 = 0.50
0.66 + 0.058
RPP
12.62 + 0.34
0.73 = 0.44
0.060 = 0.0097
4.25 0.50
0.69 = 0.031
Conv
12.03 0.46
1.34 + 0.38
0.076 = 0.0157
4.63 = 0.36
0.79 = 0.092
Table 1: Mean test classification error on C1FAR-10 and MSE on 4 UCI regression tasks, with one standard deviation errors taken over 10 trials. Similar to Figure 4 we find that whether the constrained convolutional structure is helpful (CIFAR) or not (UCI), RPP-Conv performs similarly to the model with the correct level of complexity.
( ((g) x(o)))
((()) p(g))
(o(S) p(g)
(a) Walker2d Left-Right Swimmer Front-Back (c) HalfCheetah In-Out
Figure 4: Example illustrations of symmetries and representations from the Mujoco environments. Left: legh-right symmetry the Walker2d environment, center: front-back symmetry in the Swimmer environment, and right: In-out similarity in the HalfCheetah environment
model predictive control in the engineering community where finely specified dynamics models constructed by engineers and only a small number of parameters are fit with system identification to determine mass, inertia, joint stiffness, etc. On the other side of things stands the hands off approach taken in the RL community, where general and unstructured neural networks are used for both transition models 9 52 22] as well as policies and value functions [17]. The state and action spaces for these systems are highly complex with many diverse inputs like quaternions, joint angles, forces, torques that each transform in different ways under symmetry transformation like left-right reflection or rotation. As result, most RL methods treat these spaces a black box ignoring all of this structure, and as result they tend require tremendous amounts of training data, making difficult to apply to real systems without the use of simulators.
We make of this information about what kinds of objects populate the state and action spaces to encode approximate symmetries of the RL. environments. As shown in van der Pol et al. [49), exploiting symmetries in MDPs by using equivariant networks can yield substantial improvements in data efficiency. But symmetries are brittle, and minor effects like rewards for moving in one direction, gravity, or even perturbations like wind, minor tilt angle in CartPole, or other environment imperfections can break otherwise perfectly good symmetries. As shown in Table 2 broadening the scope to approximate symmetries allows for leveraging a lot more structure in the data which we can exploit with RPP. While Walker2d, Swimmer, Ant, and Humanoid have exact left/right reflection symmetries, Hopper, HalfCheetah, and Swimmer have approximate front/back reflection symmetries. Ant and Humanoid have an even more diverse set, with the D4 dihedral symmetry by reflecting and cyclicly permuting the legs of the ant, as well as continuous rotations of the Ant and Humanoid within the environment which can be broken by externa forces or rewards. Identifying this structure

---

150
Average Raturn
3000
Average Retumn
10000
Average Rteurn
7500
2000
5000
1000
0.0
0.2
0.4
0.5
0.6
1.0
0.0
0.2
0.4
0.8
1.0
Ervironment Steps (K10f)
Enviromment Staps (x10")
Eniiiromment Steps (x100))
HalfCheetah-v2
Swimmer-v2
Hopper-v2
5000
Average Return
4000
Average Return
5000
Average NerageBuuuum Return
4000
RPP-SAC
2000
3000
3000
SAC
2000
1000
1000
1.0
1.0
0.2
0.8
Environment Steps (X10f)
166
Environment Steps (x10f)
266
Environment Steps (x10F)
Walker2d-v2
Ant-v2
Humanoid-v2
Figure 5: Average reward curve of RPP-SAC and SAC trained Mujoco locomotion environments (max average reward attained at each step). Mean and one standard deviation taken over 4 trials shown in the shaded region. Incorporating approximate symmetries in the environments improves the efficiency of the model free RL agents.
in the data, we are able to use the generality of EMLP to construct an equivariant model for this data, and then turn it into soft prior using RPP.
Symmetries
Walker2d
Hopper
HalfCheetah
Swimmer
Ant
Humanoid
Exact
L2
X
X
L2
Zn
Z2
Approximate
Z2
Z2
Z2
Z2
Z2
D4 0(2)
Z2 O(2)
This work
Z2
Zz
Z2
Z2
Z2
Z4
SO(2)
Table 2: Exact and approximate symmetries of Mujoco locomotion environments of which we use the subgroups in the bottom row, see Appendix D for the detailed action and state representations.
6.1 Approximate Symmetries in Model Free Reinforcement Learning
We evaluate RPPs on the standard suite of Mujoco continuous control tasks in the context of model- free reinforcement learning. With the appropriately specified action and state representations detailed in[Appendix D we construct RPP-EMLPss which we use as a drop-in replacement for both the policy and Q-function in the Soft Actor Critic (SAC) algorithm [17] using the same number of layers and channels. contrast with van der Pol et al. 49 where equivariance is used just for policies, we find that using RPP-EMLP for the policy function alone is not very helpful with Actor Critic (see Figure 5). With the exception of the Humanoid-v2 environment where the RPP-EMLP destabilizes SAC, we find that incorporating the exact and approximate equivariance with RPP yields consistent improvements the data efficiency the RL agent as shown in Figure 5
6.2 Better Transition Models for Model Based Reinforcement Learning
We also investigate whether the equivariance prior of RPP can improve the quality the predictions for transition models in the context of model based RI. To evaluate this in way decoupled from the complex interactions between policy, model, and value function in MBRL, we instead construct static dataset of 50, 000 state transitions sampled uniformly from the replay buffer of a trained SAC

---

Swimmer-v2
Hopper-v2
Ant-v2
Rollout
MLP
RPP
MLP
RPP
MLP
RPP
10 Steps
0.51 + 0.02
0.40 + 0.04
+ 0.1
0.9 0.1
4.2
5.2 + 0.3 13.9 + 0.7 20.0 1.1
30 Steps
1.6 0.2
1.26 0.14
3.8 0.3
0.5
11.3 +
100 Steps
3.9 1.0
2.75 + 0.31
9.8 0.5
0.7
16.0 0.3
Equiv Err
46%
19%
98%
32%
36%
31%
Table 3:
Transition model rollout relative error in percent % averaged over 10, 30, and 100 step rollouts (geometric mean over trajectory). Errorbars 1 standard deviation taken over 3 random seeds. Equivariance error is computed from as the geometric mean averaged over the 100 step rollout.
agent. Since the trajectories in the replay buffer come from different times, they capture the varied dynamics MBRL transition models often encounter during training.
State of the art model based approaches on Mujoco tend to use ensemble of small MLPs that predict the state transitions [ [22 22 2). without exploiting any structure the state space. We evaluate test rollout predictions via the relative error of the state over different length horizons for the RPP model against an MLP. the method of choice. As shown in Table 3 RPP transition models outperform MLPs the Swimmer and Hopper environments, especially for long rollouts showing promise for use in MBRL. On these environments, RPP learns a smaller but non non- -oglngglii equivariance error that still enables it to fit the data.
7 Limitations
Using RPP-EMLP for the state and action spaces of the Mujoco environments required identifying the meaning of each of the components in terms of whether they are scalars, velocity vectors, joint angles, or orientation quaternions, and also which part of the robot they correspond This can be an error-prone process. While RPPs are fairly robust to such mistakes, the need to identify components makes using RPP more challenging than standard MLP. Additionally, due to the bilinear layers within EMLP, the Lipschitz constant of the network is unbounded which can lead to training instabilities when the inputs are not well normalized. We hypothesize these factors may contribute to the training instability we experienced using RPP-EMLP on Humanoid-v2.
8 Conclusion
In this work we have presented method for converting restriction priors such as equivariance constraints into flexible models that have a bias towards structure but are not constrained by it. safe choice. These RPP models are able Given uncertainty about the nature of the data, RPPs to perform as well as the equivariant models when exact symmetries are present, and as well as unstructured MLPs when the specified symmetry doesn't 't exist, and better than both for approximate symmetries. We have shown that encoding approximate symmetries can be a powerful technique for improving performance with the messy and complex state and action spaces in reinforcement learning, as well as in a more general regression setting.
We hope that RPP enables designing more expressive priors for neural networks that capture the kinds of high level assumptions that machine learning researchers actually hold when developing models, rather than low level characteristics about the parameters that are hard to interpret. Building better techniques for enforcing high level properties helps lower the cost of incorporating prior knowledge, and better accommodate the complexities of data even if they don't 't match expectations.
Acknowledgements We thank Samuel Stanton for useful discussion and feedback. This research was supported by Amazon Research Award, NSF I-DISRE 193471, NIH DA0087644-01A NSF 19--19,0266, and NSF 1926658NRT-HDR; FUTURE Foundations, Translation, and Responsibility for Data Science.
10

---

References
[1] Farzad Abdolhosseini, Hung Yu Ling, Zhaoming Xie, Xue Bin Peng, and Michiel van de Panne. On learning symmetric locomotion. In Motion, Interaction and Games, pages 1-10. 2019.
[2] Brandon Amos, Samuel Stanton, Denis Yarats, and Andrew Gordon Wilson. On the model-based stochastic value gradient for continuous reinforcement learning. arXiv preprint arXiv:2008. 12775, 2020.
[3] Brandon Anderson, Truong Son Hy, and Risi Kondor. Cormorant: Covariant molecular neural networks. In Advances in Neural Information Processing Systems, pages 15510-1559, 2019.
[4] Marcin Andrychowicz, Anton Raichuk, Piotr Staüczyk, Manu Orsini, Sertan Girgin, Raphael Marinier, Léonard Hussenot, Matthieu Geist, Olivier Pietquin, Marcin Michalski, et al. What matters in on-policy reinforcement learning? a large-scale empirical study. arXiv preprins vrXiv:2006.05990 2020.
[5] Gregory Benton, Mare Finzi, Pavel Izmailov, and Andrew Gordon Wilson. Learning invariances in neural networks. arXiv preprint arXiv:2010. 1882, 2020.
[6] Alexander Bogatskiy, Brandon Anderson, Jan Offermann, Marwah Roussi, David Miller, and Risi Kondor. Lorentz group equivariant neural network for particle physics. In International Conference on Machine Learning, pages 992-1002. PMLR, 2020.
[7] Greg Brockman, Vicki Cheung, Ludwig Pettersson, Jonas Schneider, John Schulman, Jie Tang, and Wojeiech Zaremba. Openai gym. arXiv preprint 160.00660000 2016.
[8] Ricky TQ Chen, Yulia Rubanova, Jesse Bettencourt, and David K Duvenaud. Neural ordinary differential equations. In Advances in neural information processing systems, pages 6571-6583, 2018.
[9] Kurtland Chua, Roberto Calandra, Rowan McAllister, and Sergey Levine. Deep reinforce- ment learning in a handful of trials using probabilistic dynamics models. arXiv preprint arXiv:1805.124 12114, 2018.
[10] Taco Cohen and Max Welling. Group equivariant convolutional networks. In International conference on machine learning, pages 2990-2999. PMLR, 2016.
[11] Dheeru Dua and Casey Graff. UCI machine learning repository, 2017. URL http:////aach ics.uci, www.uci.eeee edu/ml
[12] Bryn Elesedy and Sheheryar Zaidi. Provably strict generalisation benefit for equivariant models. arXiv preprint arXiv:2102.1033:3 2021.
[13] Marc Finzi, Samuel Stanton, Pavel Izmailov, and Andrew Gordon Wilson. Generalizing convolutional neural networks for equivariance to lie groups on arbitrary continuous data. In International Conference on Machine Learning, pages 31 3165-3176. PMLR, 2020.
[14] Mare Finzi, Max Welling, and Andrew Gordon Wilson. A practical method for constructing equivariant multilayer perceptrons for arbitrary matrix groups. arXiv preprint arXiv:2104.0949 09459, 2021.
[15] Fabian B Fuchs, Daniel E Worrall, Volker Fischer, and Max Welling. Se transfforrrrs:: 3d roto-translation equivariant attention networks. arXiv preprint arXiv:2006. 10503, 2020.
[16] Samuel Greydanus, Misko Dzamba, and Jason Yosinski. Hamiltonian neural networks. In Advances in Neural Information Processing Systems, pages 15379-15389, 2019,
[17) Tuomas Haarnoja, Aurick Zhou, Pieter Abbeel, and Sergey Levine. Soft actor-critic: Off- policy maximum entropy deep reinforcement learning with a stochastic actor. In International Conference on Machine Learning, pages 1861-1870. PMLR, 2018.
[18] Tuomas Haarnoja, Aurick Zhou, Pieter Abbeel, and Sergey Levine. Soft actor-critic: Off- policy maximum entropy deep reinforcement learning with a stochastic actor. In International Conference Machine Learning, pages 1861-1870. PMLR, 2018.

---

[19] Tuomas Haarnoja, Aurick Zhou, Kristian Hartikainen, George Tucker, Sehoon Ha, Jie Tan, Vikash Kumar, Henry Zhu, Abhishek Gupta, Pieter Abbeel, et al. Soft actor-critic algorithms and applications. arXiv preprini arXiv: 18/2005905 2018.
[20] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770-778, 2016.
[21] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Identity mappings in deep residual networks. In European conference computer vision. pages 630-645- Springer, 2016.
[22] Michael Janner, Justin Fu, Marvin Zhang, and Sergey Levine. When to trust your model: Model-based policy optimization. arXiv preprint 1906:0:9900.02555. 2019.
[23] Jiechuan Jiang, Chen Dun, Tiejun Huang, and Zongqing Lu. Graph convolutional reinforcement learning. arXiv preprint
09202, 2018.
[24] Tobias Johannink, Shikhar Bahl, Ashvin Nair, Jianlan Luo, Avinash Kumar, Matthias Loskyll, Juan Aparicio Ojea, Eugen Solowjow, and Sergey Levine. Residual reinforcement learning for robot control. In 2019 International Conference on Robotics and Automation (ICRA), pages 6023-6029. IEEE, 2019.
[25] K Kashinath, M Mustafa, A Albert, JL Wu, C Jiang, S Esmaeilzadeh, K Azizzadenesheli, Wang, A Chattopadhyay, A Singh, et al. Physics-infomme machine learning: case studies for weather and climate modelling. Philosophical Transactions of the Royal Society A, 379(2 20200093, 2021.
[26] Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv: 2014.
[27] Ilya Kostrikov, Denis Yarats, and Rob Fergus. Image augmentation is all you need: Regularizing deep reinforcement learning from pixels. arXiv preprint urXiv:2004.1664) 2020.
[28] Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009.
[29] Yann LeCun, Bernhard Boser, John S Denker, Donnie Henderson, Richard E Howard, Wayne Hubbard, and Lawrence D Jackel. Backpropagation applied to handwritten zip code recognition. Neural computation, 1(4):541-551, 1989.
[30] Yijiong Lin, Jiancong Huang, Matthicu Zimmer, Yisheng Guan, Juan Rojas, and Paul Weng. Invariant transform experience replay: Data augmentation for deep reinforcement learning. IEEE Robotics and Automation Letters, 5(4):6615-661) 2020.
[31] lou-Jen Liu, Raymond A Yeh, and Alexander G Schwing. Pic: permutation invariant critic for multi-agent deep reinforcement learning. In Conference on Robot Learning, pages 590-602. PMLR, 2020.
[32] Rosanne Liu, Joel Lehman, Piero Molino, Felipe Petroski Such, Eric Frank, Alex Sergeev, and Jason Yosinski. An intriguing failing of convolutional neural networks and the coordcony solution. arXiv preprint 180i 00247,032770400 2018.
[33] Ziming Liu, Yunyue Chen, Yuanqi Du, and Max Tegmark. Plysics-augmented learning: A new paradigm beyond physics-informed learning. arXiv preprint arXiv:2109. 13901, 2021.
[34] Haggai Maron, Heli Ben-Hamu, Naday Shamir, and Yaron Lipman. Invariant and cquivariant graph networks. arXiv preprint arXiv:1812.09902. 2018.
[35] Haggai Maron, Or Litany, Gal Chechik, and Ethan Fetaya. On learning sets of symmetric elements. arXiv preprint arXiv:2002.08599, 2020.
[36] Aditi Mavalankar. Goal-conditioned batch reinforcement learning for rotation invariant locomo- tion. arXiv preprint arXhv:2004.08356, 2020.
12

---

[37] Takeru Miyato, Toshiki Kataoka, Masanori Koyama. and Yuichi Yoshida. Spectral normalization for generative adversarial networks, arXiv preprint arXiv:180..05957, 2018.
[38] Behnam Neyshabur. Towards learning convolutions from scratch. arXiv preprint
arXiv:2007.13657 2020.
[39] Behnam Neyshabur, Ryota Tomioka, and Nathan Srebro. In search of the real inductive bias:
On the role of implicit regularization in deep learning. arXiv preprint arXiv:1412.6611 2014,
[40] Prajit Ramachandran, Barret Zoph, and Quoe V Le. Searching for activation functions. arXiv preprint arXiv:1710.059/0 2017.
[41] Carl Edward Rasmussen and Zoubin Ghahramani. Occam's 's razor. Advances in neural informa- tion processing systems, pages 294-300, 2001.
[42] Balaraman Ravindran and Andrew G Barto. Approximate homomorphisms: A framework for non-exact minimization in markov decision processes. 2004.
[43] David E Rumelhart, Geoffrey E Hinton, and Ronald Williams. Learning internal represen- tations by error propagation. Technical report, California Univ San Diego La Jolla Inst for Cognitive Science, 1985.
[44] Victor Garcia Satorras, Emiel Hoogeboom, and Max Welling. E (n) equivariant graph neural networks. arXiv preprint arXiv:2102.09844 2021.
[45] Tom Silver, Kelsey Allen, Josh Tenenbaum, and Leslie Kaelbling, Residual policy learning. arXiv preprint arXiv:
2018
[46] Sainbayar Sukhbaatar, Arthur Szlam, and Rob Fergus. Learning multiagent communication with backpropagation. arXiv preprint arXiv: MrXiv::605007733 2016.
[47] Jonathan Taylor, Doina Precup, and Prakash Panagaden. Bounding performance loss in ap- proximate mdp homomorphisms. Advances in Neural Information Processing Systems, 21: 1649-1656, 2008
[48] Elise van der Pol, Thomas Kipf, Frans A Oliehoek, and Max Welling. Plannable approximations to mdp homomorphisms: Equivariance under actions. arXiv preprint arXiv:2002. / 1963, 2020.
[49] Elise van der Pol, Daniel Worrall, Herke van Hoof, Frans Oliehoek, and Max Welling. Mdp homomorphic networks: Group symmetries in reinforcement learning. Advances in Neural Information Processing Systems, 33, 2020.
[50] Mark van der Wilk, Matthias Bauer, ST John, and James Hensman. Learning invariances using the marginal likelihood. arXiv preprint arXiv: 2018.
[51] Renhao Wang, Marjan Albooyeh, and Siamak Ravanbakhsh. Equivariant maps for hierarchical structures. arXiv preprint arXiv:2006.33627, 2020.
[52] Tingwu Wang and Jimmy Ba. Exploring model-based planning with policy networks. arXiv preprint arXiv: 190:::9060000 08649, 2019.
[53] Maurice Weiler and Gabriele Cesa. General <22)-cquivariant steerable cnns. arXiv preprint 1911:v99110025/1 2019.
[54] Daniel E Worrall, Stephan J Garbin, Daniyar Turmukhambetov, and Gabriel Brostow. Har- monic networks: Deep translation and rotation equivariance. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 5028-5033 2017.
[55] Manzil Zaheer, Satwik Kottur, Siamak Ravanbakhsh, Barnabas Poczos, Russ R Salakhutdinov, and Alexander Smola. Deep sets. Advances neural information processing systems, pages
[56] Yanzhao Zhou, Qixiang Ye, Qiang Qiu, and Jianbin Jiao. Oriented response networks. In Proceedings of the IEFE Conference on Computer Vision and Puttern Recognition, pages 519-528, 2017.
13


Reviews:
[{"review_id": "lJ8YSgDtbIn", "reviewer": "Reviewer_hyfb", "summary": "This paper proposes a method to introduce inductive biases into neural networks in a *soft* way. In a nutshell, it achieves this by combining strict equivariant layers with flexible non-structured layers, akin to how residual layers combine a identity skip connection with a normal linear layer. This induces soft equivariance priors that, in principle, can accomodate both exact and approximate symmetries. \n\nIn a first set of experiments, the paper demonstrates that the proposed RPPs provide a trade-off between unstructured MLPs and hard-equivariant EMLP's in datasets with exact, approximate, and mis-specified symmetries. Then, it showcases the promise of RPPs in the context of reinforcement learning, where recent deep learning-based models have typically been fully unstructured, and for which the results show RPPs provide significant improvements. ", "questions": "", "limitations": "", "rating": 7, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "This paper contains one simple main idea (soft equivariance via residual-type additive layers), but which is motivated very compellingly, executed elegantly, and validated convincingly. It is also written well, and makes for an entertaining read. Overall, I appreciate this type of simple-but-well-executed papers, and I think it makes for a valuale contribution to the conference. Most of my comments and concerns relate to clarity, minor issues, and some requests for additional results. \n\nMeta-Comments:\n* Originality. As far as I can tell, this approach to softly enforce equivariance constraints through residual pathway priors is new, although admittedly I'm not on top of all the recent equivariance literature\n* Significance. While the experimental results are somewhat limited to make a definitive conclusion on the applicability, robustness, and performance of RPP-based models, they are promising, making this paper potentially quite significant\n* Quality. In terms of quality, albeit simple and limited, the theoretical derivations are correct, and the experimental results seem reliable and replicable\n* Clarity. The clarity of the paper can be improved in some aspects, e.g., see comments below. \n\nSpecific Comments:\n* I found it counterintuitive that the solutions for the modified inertia setting in Fig 3 are less spread than those of the pure Inertia ones. Given the approximate notion of symmetry encoded in this dataset, I would have expected it to lead to solutions with more variance in their equivariance error. I would appreciate if the authors could comment on this.\n* In Sec 5.3, it would be nice if the main text could specifiy whether the exact same architecture is used for all these datasets. If so, how much padding is there on the UCI datasets?\n* Table 2 needs some more explanation. E.g., it is my understanding that the Hooper dataset does have exact symmetries, so it's not clear why it's crossed out for 'Exact' in this table. \n* The results in Table 3 and Figure 5 consider only a fully-unstructured MLP baseline. In order to get a full picture here, it would have been nice to have results for some hard-equivariance baseline.\n* While I appreciate the discussion on limitations in the Appendix, deferring it to the Supplement instead of the main text dilutes its perceived importance, and goes against the spirit of the 9-th page policy of this year's CFP\n\nQuestions:\n* In L135, is the prior on the *size*, or on the *magnitude*? \n* In L168, do you mean *less* flexible layer? (or replace A by B), otherwise this seems to contradict the intuition for A/B provided above\n* Why does the method fail for the Humanoid-v2 setting in Figure 5? The paper says it 'destabilitizes SAC'. How so? Could this be prevented? How prevalent is this phenomenon? Answering such questions would bring better understanding of the limitations of this class of models.\n\n\nTypos:\n* word in missing in L67\n* coma missing in L67\n* repeated 'the' L129\n* The definition of the space of solutions in L150.5 would be clearer wrapping it around {$ W \\in \\mathbb{R}^{n_{\\text{out}} \\times n_{\\text{in}}} : \\quad \\dots $}.\n* break<->down L162\n* A couple of typos in L194\n* The figure reference in L197 is incorrect", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Yes, albeit tangentially, in the appendix", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "5 hours"}, "extra_text": {}}, {"review_id": "YVVy-P1xLo8", "reviewer": "Reviewer_RJms", "summary": "The idea here is to use a combination of fully connected layers with equivariant layers and use a prior to bias the feedforward layer towards simpler (equivariant) linear maps. Standard residual connections are given as an extreme example where the simpler linear map is the identity operator. L2 penalty on the weights is used to implement this prior. Experiments study this type of model in settings where 1) the exact symmetry holds, 2)  it is partially violated, and 3) the data has no such symmetry. Example application domains used for experiments include physical models and applications in RL, where the correct (approximate) symmetries are known.\n", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The proposed idea is quite simple yet it seems quite powerful and flexible, in the sense that the model performs well even with misspecified or “approximate” symmetries. The experiments are chosen very well, and the results are supportive. The presentation is quite clear, the paper is an enjoyable read. I have a few (minor) questions:\n\nThe material in appendix D about the effect of prior variance seems quite central to your claim. I suggest expanding the experiments to see if similar patter (of robustness to the hyperparameters) appears across different problems and potentially add this to the main body. \n\nCould you please elaborate on the claim of lines  165-168? Are these only based on empirical observations?\n\nHow do you explain the RL experiments in which RPP fails (Hopper-v2, Ant-v2)?\n\nI wonder if presenting this idea as a generalization of residual connection is valid, given that a vanilla residual connection does not have an associated parameter.\n\nIs it necessary to package this idea with the EMLP’s method for finding equivariant linear maps, given that one could use any other method for designing equivariant kernels or tying the parameters in the feedforward layer?\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "The appendix includes some discussion on the limitations.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "5"}, "extra_text": {}}, {"review_id": "55da-t9RSoZ", "reviewer": "Reviewer_wXcB", "summary": "The authors propose to enforce soft equivariance constraints in ANN layers through a Gaussian prior on layer weights. Experimental results on a variety of problems support the benefit of the approach perhaps unsurprisingly. The overall contribution feels quite straightforward and incremental.", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Overall the contribution of the paper is to use MAP optimization to propose and intermediate between stricly equivariant layers and non-equivariant ones (MLPs typically). This is enforced by using a Gaussian prior whose convariance is a sum of unstructured (proportional to identity) and structured (contrained to a basis of equivariant matrices) matrices.\nPerhaps unsurprisingly, the resulting model achieves a trade off between unstructured approches and stricly equivariant approaches that is beneficial in several examples.\n\nThe contribution feels thus quite iterative, while there would be many ways to address the problem more in depth: e.g. can we use sparser priors in ordre to relax equivariance only where/when necessary? Can we implement this soft constraint such that computational benefits of structued models are preserved as much as possible? \n\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "I could not find a discussion on the limitations (e.g. computational complexity).", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "3 hours"}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper presents a simple solution to an important problem - deciding what kind of inductive biases to use when we don’t have full confidence about the type of symmetries that exist in the domain.  The solution is to combine a flexible model with one that has strong equivariance inductive biases.

All reviewers recognize the importance of the problem, clarity of the presentation and the simplicity of the approach.  The proposed method being simple does not diminish its value, and simple methods are typically more applicable to a wide range of problems and more easily adopted by many.

Reviewer wXcB who gave a 5 rating does have a point about the baselines though: this paper mostly contrasts the proposed approach with the two extremes - the MLP without any explicitly symmetry built in and models like convnets that have strong equivariance built in.  It would make the paper stronger to have a stronger baseline, that is for example trained with data-augmentation as a soft way to build some inductive bias into the model.  The advantage of the proposed approach, as the authors explained in the discussion, is that it will be much more data-efficient and generalize much better in data-sparse regions.

After discussion among the reviewers and calibrating across a range of borderline papers we decided to accept this paper based on its clarity and elegance, and believe it can be a useful contribution to the community.

Decision: Accept (Poster)