The Surprising Effectiveness of Equivariant Models in Domains with Latent Symmetry
Authors: Dian Wang
Jung Yeon Park
Neel Sortur
Lawson L.S. Wong
Robin Walters
Robert Platt
Venue: ICLR 2023 notable top 25% (2023)
Keywords: Equivariant Learning
Reinforcement Learning
Robotics

Abstract: Extensive work has demonstrated that equivariant neural networks can significantly improve sample efficiency and generalization by enforcing an inductive bias in the network architecture. These applications typically assume that the domain symmetry is fully described by explicit transformations of the model inputs and outputs. However, many real-life applications contain only latent or partial symmetries which cannot be easily described by simple transformations of the input. In these cases, it is necessary to learn symmetry in the environment instead of imposing it mathematically on the network architecture. We discover, surprisingly, that imposing equivariance constraints that do not exactly match the domain symmetry is very helpful in learning the true symmetry in the environment. We differentiate between extrinsic and incorrect symmetry constraints and show that while imposing incorrect symmetry can impede the model's performance, imposing extrinsic symmetry can actually improve performance. We demonstrate that an equivariant model can significantly outperform non-equivariant methods on domains with latent symmetries both in supervised learning and in reinforcement learning for robotic manipulation and control problems.

Published as a conference paper at ICLR 2023
THE SURPRISING EFFECTIVENESS OF EQUIVARIANT MODELS IN DOMAINS WITH LATENT S SYMMETRY
Dian Wang, Jung Yeon Park, Neel Sortur, Lawson L.S. Wong, Robin Walters', Robert Platt" Northeastern University {wang. di an, park. jungy, sortur. .n, w.nnn..oooo. wong, rawa www.trrs, r.w.sss..ppttttt.....c p
ABSTRACT
Extensive work has demonstrated that equivariant neural networks can signifi- cantly improve sample efficiency and generalization by enforcing inductive bias in the network architecture. These applications typically assume that the do- main symmetry is fully described by explicit transformations of the model inputs and outputs. However, many real-life applications contain only latent or partial symmetries which cannot be easily described by simple transformations of the in- put. In these cases, it is necessary to learn symmetry in the environment instead of imposing mathematically on the network architecture. We discover, surpris- ingly, that imposing equivariance constraints that do not exactly match the domain symmetry is very helpful in learning the true symmetry in the environment. We differentiate between extrinsic and incorrect symmetry constraints and show that while imposing incorrect symmetry can impede the model's performance, impos- ing extrinsic symmetry can actually improve performance. We demonstrate that an equivariant model can significantly outperform non-equivariant methods on domains with latent symmetries both in supervised learning and in reinforcement learning for robotic manipulation and control problems.
INTRODUCTION
Recently, equivariant learning has shown great success in vari- ous machine learning domains like trajectory prediction (Wal- ters al., 2020), robotics (Simeonov al.. 2022), and inforcement learning (Wang et al., 2022c). Equivariant net- works (Cohen & Welling, 2016; 2017) can improve general- ization and sample efficiency during learning by encoding task symmetries directly into the model structure. However, this requires problem symmetries to be perfectly known and mod- eled at design time something that is sometimes problematic. It is often the case that the designer knows that latent sym- metry is present in the problem but cannot easily express how that symmetry acts in the input space. For example, Figure 1b is rotation of Figure la. However, this is not a rotation of the image is a rotation of the objects present the image when they are viewed from an oblique angle. In order to model this rotational symmetry, the designer must know the viewing angle and somehow transform the data or encode projective ge- ometry into the model. This difficult and makes the entire approach less attractive. In this situation, the conventional wis- dom would be discard the model structure altogether since unconstrained model. Instead,
Object Transform
Hard to model
(b)
Helps
Model
(e)
Image Transform Easy to model
Object image trans- forms. Object transform rotates the object itself (b), while image trans- form rotates the image (c). We pro- pose use the image transform help model the object transform.
Figure 1:
is not fully known and to we explore whether it is possible to benefit from equivariant models even when the way a symmetry acts the problem input not precisely known. We show empirically that this is indeed the case and that an inaccurate equivariant model is often better than a completely unstructured model. For
"Equal Advising

---

Published as a conference paper at ICLR 2023
example, suppose we want to model function with the object-wise rotation symmetry expressed in Figure la and b. Notice that whereas it is difficult to encode the object-wise symmetry, it is easy to encode an image-wise symmetry because it involves simple image rotations. Although the image- wise symmetry model is imprecise in this situation, our experiments indicate that this imprecise model is still much better choice than completely unstructured model.
This paper makes three contributions. First, we define three different relationships between prob- lem symmetry and model symmetry: correct equivariance, incorrect equivariance, and extrin- sic equivariance. Correct equivariance means the model correctly models the problem symme- try; incorrect equivariance is when the model symmetry interferes with the problem symme- try; and extrinsic equivariance is when the model symmetry transforms the input data to out- of-distribution data. We theoretically demonstrate the upper bound performance for an incor- rectly constrained equivariant model. Second, we empirically compare extrinsic and incorrect equivariance in a supervised learning task and show that a model with extrinsic equivariance can improve performance compared with unconstrained model. Finally, we explore this idea in a reinforcement learning context and show that an extrinsically constrained model can outper- form state- -fr conventional CNN baselines. Supplementary video and code are available at https: / /ww.ntw..w...t..ooo. github. io cexxtrinsic_paate
RELATED WORK
Equivariant Neural Networks. Equivariant networks are first introduced as G-Convolution (Co- hen & Welling, 2016) and Steerable CNN (Cohen & Welling, 2017; Weiler & Cesa, 2019; Cesa et al., 2021). Equivariant learning has been applied to various types of data including images (Weiler & Cesa, 2019), spherical data (Cohen et al., 2018), point clouds (Dym & Maron, 2020), sets Maron et al. (2020), and meshes (De Haan et al., 2020), and has shown great success in tasks including molecular dynamics (Anderson et al., 2019), particle physies (Bogatskiy et al., 2020), fluid dynam- ics (Wang et al., 2020), trajectory prediction ( (Walters et al., 2020), robotics (Simeonov et al., 2022; Zhu et al., 2022: Huang et al., 2022) and reinforcement learning (Wang et al., 2021: 2022c). Com- pared with the prior works that assume the domain symmetry is perfectly known, this work studies the effectiveness of equivariant networks in domains with latent symmetries.
Symmetric Representation Learning. Since latent symmetry is not expressable as simple trans- formation of the input, equivariant networks can not be used in the standard way. Thus several works have turned to learning equivariant features which can be easily transformed. Park et al. (2022) learn an encoder which maps inputs to equivariant features which can be used by downstream equivariant layers. Quessard et al. (2020), Klee et al. (2022), and Marchetti et al. (2022) map 2D image inputs to elements of various groups including SO(3), allowing for disentanglement and equivariance con- straints. Falorsi et al. (2018) use homeomorphic VAE to perform the same task in an unsupervised manner. Dangovski al. (2021) consider equivariant representations learned in a self-supervised manner using losses to encourage sensitivity or insensitivity to various symmetries. Our method may be considered as an example of symmetric representation learning which, unlike any of the above methods, uses an equivariant neural network as an encoder. Zhou et al. (2020) and Dehmamy et al. (2021) assume no prior knowledge of the structure of symmetry in the domain and learn the symme- try transformations inputs and latent features end-to-end with the task function. In comparison, our work assumes that the latent symmetry is known but how it acts on the input is unknown.
Sample Efficient Reinforcement Learning. One traditional solution for improving sample effi- ciency is to create additional samples using data augmentation (Krizhevsky et al., 2017). Recent works discover that simple image augmentations like random crop (Laskin et al., 2020b; Yarats et al., 2022) or random shift (Yarats et al., 2021) can improve the performance of reinforcement learning. Such image augmentation can be combined with contrastive learning (Oord et al., 2018) to achieve better performance (Laskin et al., 2020a; Zhan et al., 2020). Recently, many prior works have shown that equivariant methods can achieve tremendously high sample efficiency in reinforce- ment learning (van der Pol et al., 2020; Mondal et al., 2020: Wang et al., 2021; 2022c), and realize on-robot reinforeement learning (Zhu et al., 2022: Wang et al., 2022a). However, recent equivariant reinforcement learning works are limited in fully equivariant domains. This paper extends the prior works by applying equívariant reinforcement learning to tasks with latent symmetries.
2

---

Published as a conference paper at ICLR 2023
3 BACKGROUND
Equivariant Neural Networks. A function is equivariant if it respects symmetries of its input and output spaces. Specifically, a function f: X + Y is equivariant with respect to symmetry group G if it commutes with all transformations E G. (()(gg)) ()))))()), where Pa and py are the representations of the group G that define how the group element E G acts on Z E X and y E Y, respectively, An equivariant function is mathematical way of expressing that is symmetric with respect to G: if we evaluate f for differently transformed versions of the same input, we should obtain transformed versions of the same output.
In order to use an equivariant model, we generally require the symmetry group G and representation Pa: to be known at design time. For example, in convolutional model, this can be accomplished by tying the kernel weights together so as to satisfy K (gy) - Pour (g) K (y) (g)= where Pin and Pous denote the representation of the group operator the input and the output of the layer (Cohen et al., 2019). End-to-end equivariant models can be constructed by combining equivariant convo- lutional layers and equivariant activation functions. In order to leverage symmetry this way, is common to transform the input so that standard group representations work correctly, e.g. to transform an image to a top-down view so that image rotations correspond to object rotations.
Equivariant SAC. Equivariant SAC (Wang et al., 2022c) is variation of SAC (Haarnoja et al., 2018) that constrains the actor to an equivariant function and the critic to an invariant function with respect to a group G. The policy is a network π : S + A x Ag, where Ag is the space of action stan- dard deviations (SAC models a stochastic policy). It defines the group action on the output space of the policy network network ā E A Ao as: gā ((aqquiv Ginv; az) = (Pequiv (g) Giav: Qz), where Qequiv E Aequiv is the equivariant component in the action space, aiav E Anv is the invariant component in the action space, Og E Ar, E G. The actor network æ is then defined to be a mapping r that satisfies the following equivariance constraint: T(gs) = g(((s)) gū. The critic is Q-network q S x A + I that satisfies an invariant constraint: q(gs, ga) = q(s,a)
4 LEARNING SYMMETRY USING OTHER SYMMETRIES
4.1 MODEL SYMMETRY VERSUS TRUE SYMMETRY
8
This paper focuses on tasks where the way in which the symmetry group operates on the input space is unknown. In this case the ground truth function ʃ X Y is equivariant with respect to a group G which acts on X and Y by P. and py respectively. However, the action Pe on the input space is not known and may not be a
Gj reflection corrict
Gj scaling
eetrinalc
(
rotation
(b)
simple or explicit map. Since Par is unknown, we cannot pursue the strategy of learning f using an equivariant model class fo constrained by Pa. As an alternative, we propose restricting to model class so which satisfies equivariance with respect to a different group action P.. Py (z) This paper tests the hy- pothesis that if the model is constrained to a symmetry class P.. which is related to the true symmetry Pa. then it may help learn a model satisfying the true symmetry. For example, if x an image viewed from an oblique angle and P5 is the rotation of the objects in the image, P. can be the rotation of the whole image (which different from Ps because of the tilted view angle). Section 4.4 will describe this example in detail.
B
8
8
i..., fo (px((g))
1
8
(4)
Figure 2: An example classification task for correct, incorrect, and extrinsic equivariance. The grey ring shows the in- put distribution. Circles are the training data in the distribution where the color shows the ground truth label. Crosses show the group transformed data.
4.2 CORRECT, INCORRECT, AND EXTRINSIC EQUIVARIANCE
Our findings show that the success of this strategy depends on how P.. relates to the ground truth function f and its symmetry. We classify the model symmetry correct equivariance, incorrect
3

---

Published as a conference paper at ICLR 2023
equivariance, extrinsic equivariance with respect to f. Correct symmetry means that the model symmetry correctly reflects a symmetry present in the ground truth function f. An extrinsic sym- metry may still aid learning whereas an incorrect symmetry is necessarily detrimental to learning. We illustrate the distinction with a classification example shown in Figure 2a. (See Appendix B for more in-depth description.) Let D S X be the support of the input distribution for f- Definition 4.1. The action Os has correct equivariance with respect to / if (g(g) E D for all E D,9 g E G and (()(() = Py (x))
That is, the model symmetry preserves the input space D and is equivariant with respect to it. For example, consider the action Pa of the group G, C2 acting on I by reflection across the horizontal axis and Py 1, the trivial action fixing labels. Figure 2b shows the untransformed data E D as circles along the unit circle. The transformed data Pa (g)z (shown as crosses) also lie on the unit circle, and hence the support D is reflection invariant. Moreover, the ground truth labels f() (shown as orange or blue) are preserved by this action.
Definition 4.2. The action pa has incorrect equivariance with respect to f if there exist D and E G such that Pa (g)xz E D but [((x(g)x) & (a)
In this case, the model symmetry partially preserves the input distribution, but does not correctly preserve labels. In Figure 2c, the rotation group G (Rotz) maps the unit circle to itself, but the transformed data does not have the correct label. Thus, constraining the model So by fo((n(g)=) fa(x) will force fo to mislabel data. In this example, for a v2/2. f(a.
ORANGE and
f(-a, a) = BLUE, however, fa a) = fo (a, a)) fe(-a, Definition 4.3. The action P.v has extrinsic equivariance with respect to f ffr E D, & D.
Extrinsic equivariance is when the equivariant constraint in the equivariant network fo enforces equivariance to -ut-of-distribution data. Since P=(g) æ & D. the ground truth f(² is unde- fined. An example of extrinsic equivariance is given by the scaling group G3 shown in Figure 2d. For the data & E D, enforcing scaling invariance (g)(( = fø(x) where 9 E G3 will not increase error, because the group transformed data (in crosses) are out of the distribution D of the input data shown in the grey ring, In fact, we hypothesize that such extrinsic equivariance may even be helpful for the network learn the ground truth function. For example, in Figure 2d, the network can learn to classify all points on the left as blue and all points on the right as orange.
4.3 THFORETICAL UPPER BOUND ON ACCURACY FOR INCORRECT EQUIVARIANT MODELS
Consider a classification problem over the set X with finitely many classes Y. Let G be a finite group acting on X. Consider a model fs: X + Y with incorrect equivariance constrained to be invariant to G. In this case the points in a single orbit {gr : g E G} must all be assigned the same label fo(gz) = y. However these points may have different ground truth labels. We classify how bad this situation is by measuring p(x). the proportion of ground truth labels in the orbit of which are equal to the majority label. Let ep be the fraction of points E X which have consensus proportion p(x) = p. Proposition 4.1. The accuracy of fo has upper bound acc(fo) S E, CpP
See the complete version of the proposition and its proof in Appendix A. In the example Figure 2c, we have p E {0.5} and co.s = 1, thus acc(fa) f < 0.5. In contrast, an unconstrained model with a universal approximation property and proper hyperparameters can achieve arbitrarily good accuracy.
4.4 OBJECT TRANSFORMATION AND IMAGE TRANSFORMATION
In tasks with visual inputs (X = Bexhxw), incorrect or extrinsic equivariance will exist when the transformation of the image does not match the transformation of the latent state of the task. In such case, we call Pr. the object transform and P.z the image transform. For an image input E X, the image transform Pa (g)x (r is defined as a simple transformation of pixel locations (c.g., Figure la-c where g 7/2 E SO(2)), while the object transform Pa (g)x is an implicit map transforming the objects in the image (c.g.. Figure la-b where g = x x/2 E SO(2)). The distinction between object transform and image transform is often caused by some symmetry-breakke angle, occlusion, backgrounds, and so on (e.g.. Figure l). We refer to such commetry--beaking factors symmetry corruptions.
factors such as camera
4

---

Published as a conference paper at ICLR 2023
corrupt
grid
occlusion
light effect
squeeze
reflect
side
invert label
yellow is on the loft
(a)
(b)
Figure 3: (a) The rotation estimation task requires the network to estimate the relative rotation be- tween the two input states. (b) Different symmetry corruptions in the rotation estimation experiment.
5 EVALUATING EQUIVARIANT NETWORK WITH SYMMETRY CORRUPTIONS
Although it is preferable to use an equivariant model to enforce correct equivariance, real-world problems often contain some symmetry corruptions, such as oblique viewing angles, which mean the symmetry is latent. In this experiment, we evaluate the effect of different corruptions on an equivariant model and show that enforcing extrinsic equivariance can actually improve performance. We experiment with simple supervised learning task where the scene contains three ducks of different colors. The data samples are pairs of images where all ducks in the first image are rotated by some g E Cg to produce the second image within each pair. Given the correct y, the goal is to train a network fe TR,24x + R. (Csl to classify the rotation (Figure 3a). If we have perfect top-down image observation, then the object transform and image transform are equal, and we can enforce the correct equivariance by modeling the ground truth function f as an invariant network fo(p)())=) = far) where g E SO(2) (because the rotation the two images will not change the relative rotation between the objects in the two images). To mimic symmetry corruptions in real-world applications, we apply seven different transformations to both pairs of images shown in Figure 3b (more corrpptions are considered in Appendix E.1). In particular, for invert-label, the ground truth label is inverted to -9 when the yellow duck is on the left of the orange duck in the world frame in the first input image. Notice that enforcing invert-label is an incorrect equivariant constraint because a rotation on the ducks might change their relative position in the world frame and break the invariance of the task: f(gx) + f(x), 3g SO(2). However, in all other corruptions, enforcing -ivvaiine is an extrinsic equivariance because gæ will be out of the input distribution. We evaluate the equivariant network defined in group Cs implemented using e2cnn (Weiler& & Cesa, 2019). See Appendix D.1 for the training details.
in fo under
Comparing Equivariant Networks with CNNs. We first compare the performance of an equiv- ariant network (Equi) and conventional CNN model (CNN) with similar number of trainable parameters. The network architectures are relatively simple (see Appendix C.1) as our goal is to eval- uate the performance difference between equivariant network and unconstrained CNN model rather than achieving the best performance in this task. In both models, we apply a random crop after sampling each data batch to improve the sample efficiency. See Appendix E. for the effects of ran- dom crop augmentation on learning. Figure (blue vs green) shows the test accuracy of both models after convergence when trained with varying dataset sizes. For all corruptions with extrinsic equiv- ariance constraints, the equivariant network performs better than the CNN model, especially in low data regimes. However, for invert-label which gives an incorrect equivariance constraint, the CNN outperforms the equivariant model, demonstrating that enforcing incorrect equivariance negatively impacts accuracy. In fact, based on Proposition 4.1, the equivariant network here has a theoretical upper bound performance of 62.5%. First, p E {1, 0.5). Then p 1 when fx) E {0, TT) S Cs where fx) = - fz) (i.e., negating the label won't 't change it), and e = 2/8 = 0.25. The consen- proportion p 0.5 when (x) E /4, π /2, 3z 4, 5t 3m /2, 7m / } C Cs. where half of the
5

---

Published as a conference paper at ICLR 2023
corrupt
grld
occlusion
light effect
1.0
3.2
1.0
0.
0.0
0.3
0.0
0.0
04
0.3
04
0.2
04
0.1
0/
0.1
62
0.2
02
02
68
500.00
300
2000
100303
500
1000
01
330300
500
1300
1000000
523
5003
squeeze
invert label
1.0
5.2
1.0
5.2
0.
030
0.4
0.4
04
0.4
04
02
017
00
30.00
.00
3300
100203
503
1003
"333300
1300
100.00
180
Number of Training Data
Equi
CNN
CNN Img Trans
Figure 4: Comparison of an equivariant network (blue), conventional network (green), and CNN equipped with image transformation augmentation using Cs rotations (red). The plots show the prediction accuracy in the test set of the model trained with different number of training data. In all of our experiments, we take the average over four random seeds. Shading denotes standard error.
labels in the orbit of æ will be the negation of the labels of the other half (because half of g E Cs will change the relative position between the yellow and orange duck), thus co.5 6/8 0.75. acc(fe) S 1 x 0.25 + 0.5 0.75 = 0.625. This theoretical upper bound matches the result in Figure 4. Figure 4 suggests that even in the presence of symmetry corruptions, enforcing extrinsic equivariance can improve the sample efficiency while incorrect equivariance is detrimental.
Extrinsic Image Augmentation Helps in Learning Correct Symmetry. In these experiments, we further illustrate that enforcing extrinsic equivariance helps the model learn the latent equiv- ariance of the task for nn-distribution data. As an alternative equivariant networks, we consider an older alternative for symmetry learning, data augmentation, to see whether extrinsic symmetry augmentations can improve the performance of an unconstrained CNN by helping learn latent symmetry. Specifically, we augment each training sample with Cs image rotations while keeping the validation and test set unchanged. As shown in Figure 4, adding such extrinsic data augmen- tation (CNN Img Trans, red) significantly improves the performance of CNN (green), and nearly matches the performance of the equivariant network (blue). Notice that invert-label, adding such augmentation hurts the performance of CNN because of incorrect equivariance.
6 EXTRINSIC EQUIVARIANCE IN REINFORCEMENT LEARNING
The results in Section 5 suggest that enforcing extrinsic equivariance can help the model better learn the latent symmetry the task. this section, we apply this methodology in reinforcement learning and demonstrate that extrinsic eguivariance can significantly improve sample efficiency.
6.1 REINFORCHMENT LEARNING IN ROBOTIC MANIPULATION
We first experiment in five robotic manipulation environ- ments shown in Figure 6. The state space S IR 4xxxx a 4-channel RGBD image captured from fixed cam- era pointed at the workspace (Figure 5). The action space A = R is the change in gripper pose (z, 2, 0), where is the rotation along the z-axis, and the gripper open width X. The task has latent O(2) symmetry: when rotation or reflection applied the poses the grip- per and the objects, the action should rotate and reflect accordingly. However, such symmetry does not exist in image space because the image perspective is skewed in- stead of top-down (we also perform experiments with an- other symmetry corruption caused by sensor occlusion in
Figure 5: The image state in the Block Picking task. Left image shows the RGB channels and right image shows the depth channel.

---

Published as a conference paper at ICLR 2023
(a) Block Pulling
(b) Block Pushing
(c) Block Picking
(d) Drawer Opening
(e) Block in Bowl
Figure 6: The manipulation environments from BulletArm benchmark Wang et al. (2022b) imple-
mented in PyBullet Coumans & Bai (2016). The top-left shows the goal for each task.
Block Pulling
Block Pushing
Block Picking
Drawer Opening
Block in Bowl
Discounted reward
0.4
02
00
00g
0.0
00
Number of training steps
Equi SAC RAD
CNN SAC RAD
CNN SAC DrQ
FERM
SEN RAD
Figure 7: Comparison of Equivariant SAC (blue) with baselines. The plots show the performance
of the evaluation policy. The evaluation is performed every 200 training steps.
Appendix E.3). We enforce such extrinsic symmetry (group Dạ) using Equivariant SAC (Wang al., 2022c;a) equipped with random crop augmentation using RAD (Laskin et al., 2020b) (Equi SAC + RAD) and compare it with the following baselines: 1) CNN SAC RAD: same our method but with an unconstrained CNN instead an equivariant model: 2) CNN SAC DrQ: same as 1), but with DrQ (Yarats et al., 2021) for the random crop augmentation; 3) FERM (Zhan et al., 2020): a combination of 1) and contrastive learning; and 4) SEN + RAD: Symmetric Embedding Network (Park et al., 2022) that uses conventional network for the encoder and equivariant network for the output head. All baselines are implemented such that they have a similar number of parameters as Equivariant SAC. See Appendix C.2 for the network architectures and Appendix F for the architecture hyperparameter search for the baselines. All methods use Prioritized Experience Replay (PER) (Schaul al., 2015) with pre-loaded expert demonstrations (20 episodes for Block Pulling and Block Pushing, 50 for Block Picking and Drawer Opening, and 100 for Block in Bowl). We also add an L2 loss towards the expert action in the actor to encourage expert actions. More details about training are provided in Appendix D.2.
Figure shows that Equivariant SAC (blue) outperforms all baselines. Note that the performance of Equivariant SAC in Figure 7 does not match that reported in Wang et al. (2022c) because we have harder task setting: we do have a top-down observation centered at the gripper position as the prior work. Such top-down observations would not only provide correct equivariance but also help learn translation-invariaee policy. Even the harder task setting without top-down observations, Figure suggests that Equivariant SAC can still achieve higher performance compared to baselines.
6.2 INCREASING CORRUPTION LEVELS
In this experiment, we vary the camera angle by tilting to see how increasing the gap between the image transform and the object transform affects the performance of ex- trinsically equivariant networks. When the view angle is at 90 degrees (i.e., the image is top-down), the object and image transformation exactly match. As the view angle is decreased, the gap increases. Figure 8 shows the obser- vation at 90 and 15 degree view angles. We remove the robot arm except for the gripper and the blue/white grid on the ground to remove the other symmetry- breaking components in the environment that the camera angle
Figure 8: Left: view angle at 90 de- grees. Right: view angle at 15 degrees.
wymmetry-breaking

---

Published as a conference paper at ICLR 2023
Block in Bowl
Opening
Block Picking
Block Pulling
Block Pushing
Discounted rewarr
04
64
02
02
22
0.
02
10
View angle
Equi SAC RAD
CNN SAC
RAD
Figure 9: Comparison between Equivariant SAC (blue) and CNN SAC (green) as the view angle decreases. The plots show the evaluation performance of Equivariant SAC and CNN SAC the end of training in different view angles.
Block Pulling
Block Pushing
Block Picking
Drawer Opening
Block in Bowl
Discounted reward 00
0.6
0.4
03
022
00g
5.0 1007
0.
00
Number of training steps
Equi SAC RAD
CNN SAC + RAD
Figure 11: Comparison between Equivariant SAC (blue) and CNN SAC (green) in an environment that will make Equivariant SAC encode incorrect equivariance. The plots show the performance of the evaluation policy. The evaluation is performed every 200 training steps.
the only symmetry corruption. We compare Equi SAC + RAD against CNN SAC + RAD. We evaluate the performance of each method at the end of training for different view angles in Figure 9. As expected, the performance of Equivariant SAC decreases as the camera angle is decreased, espe- cially from 30 degrees to 15 degrees. On the other hand, CNN generally has similar performance for all view angles, with the exception of Block Pulling and Block Pushing, where decreasing the view angle leads to higher performance. This may because decreasing the view angle helps the network to better understand the height of the gripper, which is useful for pulling and pushing actions.
6.3 EXAMPLE INCORRECT EQUIVARIANCE
We demonstrate an example where incorrect equivariance can harm the performance of Equivariant SAC compared an unconstrained model. We modify the environments so that the image state will be reflected across the ver- tical axis with 50% probability and then also reflected across the horizontal axis with 50% probability (see Figure 10). As these random reflections are contained D4, the transformed state reflect(s), S is affected by Equivari- ant SAC's symmetry constraint. In particular, as the actor produces a transformed action for reflect when the optimal action should actually be invariant, the extrinsic equivariance constraint now be- comes an incorrect equivariance for these reflected states. As shown in Figure 11, Equivariant SAC can barely learn under random reflections, while CNN can still learn a useful policy.
Figure 10:
The environment conducts random reflection on the state image at every step. The four images show the four possible reflections, each has 25% probability.
6.4 REINFORCEMENT LEARNING IN DEEPMIND CONTROL SUITE
We further apply extrinsically equivariant networks to continuous control tasks in the DeepMind Control Suite (DMC) (Tunyasuvunakool et al., 2020). We use subset of the domains in DMC that have clear object-level symmetry and use the D group for cartpole, cup catch, pendulum, acrobot domains, and D2 for reacher domains. This leads to total of tasks, with 4 easy and 3 medium
8

---

Published as a conference paper at ICLR 2023
Cartpole Balance
Caripole Swingup
Pendulum Swingup
Cup Catch
1401
Diecoumted
5.5
Acrobot Swingup
Reacher Easy
Reacher
600
400
013
100
3.2
Frumes
Equivariant DrQv2
Nott equivariant DrQv2
Figure 12: Comparison between Equivariant DrQv2 and Non-equivariant DrQv2 on easy tasks (top) and medium tasks (bottom). The evaluation is performed every 10000 environment steps.
level tasks as defined (Yarats al., 2022). Note that all of these domains are not fully equivariant as they include a checkered grid for the floor and random stars as the background.
We use DrQv2 Yarats et al. (2022), SOTA model-free RL algorithm for image-based control, as our base RL algorithm. We create an equivariant version of DrQv2, with an equivariant actor and invariant critic with respect to the environment's symmetry group. We follow closely the architecture and training hyperparameters used in the original paper except in the image encoder, where two max-pooling layers are added further reduce the representation dimension for faster training- Furthermore, DrQv2 uses convolution layers in the image encoder and then flattens its output to feed it into linear layers in the actor and the critic. In order to preserve this design choice for the equivariant model, do not reduce the spatial dimensions to x 1 by downsampling/pooling or stride as commonly done in practice. Rather we flatten the image using a process we term action restriction since the symmetry group is restricted from Z² D& to D3. Let I Rhxmxx denote the image feature where Dr acts on both the spatial domain and channels. Then we add a new axis corresponding to D; by i = (gT),ED, E RB We then flatten to ī = (gT) E RR x The intermediate step I is necessary to encode both the spatial and channel actions into a single axis which ensures the action restriction is D₃- Dy-equvvaria We now map back down to the original dimension with D x convolution. To the best of our knowledge, this the first equivariant version of DrQv2.
We compare the equivariant vs. the non-equivariant (original) DrQv2 algorithm to evaluate whether extrinsic equivariance can still improve training in the original domains (with symmetry corrup- tions). In figures 12, equivariant DrQv2 consistently learns faster than the non-equivariant version on all tasks, where the performance improvement is largest on the more difficult medium tasks. In pendulum swingup, both methods have 1 failed run each, leading to large standard error, see Fig- ure 27 in Appendix F.4 for plot of all runs. These results highlight that even with some symmetry corruptions, equivariant policies outperform non-equivariant ones. See Appendix E.4. for an additional experiment where we vary the level of symmetry corruptions as in Section 6.2.
DISCUSSION
This paper defines correct equivariance, incorrect cquivariance, and extrinsic equivariance, and iden- tifies that enforcing extrinsic equivariance does not necessarily increase error. This paper further demonstrates experimentally that extrinsic equivariance can provide significant performance im- provements reinforcement learning. limitation of this work that we mainly experiment in reinforcement learning and simple supervised setting but not in other domains where equivariant learning is widely used. The experimental results of our work suggest that an extrinsic equivariance should also be beneficial in those domains, but leave this demonstration to future work. An- other limitation is that we focus on planar equivariant networks. In future work, we are interested in evaluating extrinsic equivariance in network architectures that process different types of data.

---

Published as a conference paper at ICLR 2023
ACKNOWLEDGMENTS
This work supported in part by NSF 1724257, NSF 1724191, NSF 1763878, NSF 1750649, NSF 2107256, and NASA 80NSSC19K1474. R. Walters is supported by the Roux Institute and the Harold Alfond Foundation and 2134178.
ETHIC STATEMENT
Equivariant models allow us to train robots faster and more accurately in many different tasks. Our work shows this advantage can be applied even more broadly tasks in real-world conditions. Our method is agnostic to the morality of the actions which robots are trained for and, in that sense, can make it easier for robots to be used for either societally beneficial detrimental tasks.
REFERENCES
Brandon Anderson, Truong Son Hy, and Risi Kondor. Cormorant: Covariant molecular neural networks. Advances in neural information processing systems, 32, 2019.
Alexander Bogatskiy, Brandon Anderson, Jan Offermann, Marwah Roussi, David Miller, and Risi Kondor. Lorentz group equivariant neural network for particle physics. In International Confer- ence on Machine Learning, pp. 992-1002. PMLR, 2020.
Gabriele Cesa, Leon Lang, and Maurice Weiler. A program to build e (n)-equivariant steerable cnns. In International Conference on Learning Representations, 2021.
Taco Cohen and Max Welling. Group equivariant convolutional networks. In International confer- ence on machine learning. pp. 2990-2999. PMLR, 2016.
Taco S. Cohen and Max Welling, Steerable CNNs. In International Conference on Learning Rep- resentations, 2017. URL https: oopereeevvvv ew. net/ wwrumriimissooe (irrummiirrrggKY5.....
Taco Cohen, Mario Geiger, Jonas Köhler, and Max Welling. Spherical cnns. In International Conference Learning Representations, 2018
Taco S Cohen, Mario Geiger, and Maurice Weiler. A general theory of equivariant cnns homo- geneous spaces. Advances in neural information processing systems, 32, 2019.
Erwin Coumans and Yunfei Bai. Pybullet. a python module for physics simulation for games, robotics and machine learning. GitHub repository, 2016.
Rumen Dangovski, Li Jing, Charlotte Loh, Seungwook Han, Akash Srivastava, Brian Cheung, Pulkit Agrawal, and Marin Soljacic. Equivariant self-supervised learning: Encouraging equivariance in representations. In International Conference Learning Representations, 2021.
Pim De Haan, Maurice Weiler, Taco Cohen, and Max Welling. Gauge equivariant mesh cans: Anisotropic convolutions on geometric graphs. In International Conference on Learning Repre- sentations, 2020.
Nima Dehmamy, Robin Walters, Yanchen Liu, Dashun Wang, and Rose Yu. Automatic symmetry discovery with lie algebra convolutional network. Advances in Neural Information Processing Systems, 34:2503-25155 2021.
Nadav Dym and Haggai Maron. On the universality of rotation equivariant point cloud networks. In International Conference Learning Representations, 2020.
I Luca Falorsi, Pim De Haan, Tim R Davidson, Nicola De Cao, Maurice Weiler, Patrick Forré, and Taco Cohen. Explorations in homeomorphic variational auto-encoding. arXiv preprint arXiv:1807 407.04689, 2018.
Tuomas Haanoja,, Aurick Zhou, Pieter Abbeel, and Sergey Levine. Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with stochastic actor. In International confer- on machine learning, pp. 1861-1870. PMLR, 2018.
10

---

Published as a conference paper at ICLR 2023
Haojie Huang, Dian Wang. Robin Walters, and Robert Platt. Equivariant transporter network. In Robotics: Science and Systems, 2022.
Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.
David Klee, Ondrej Biza, Robert Platt, and Robin Walters. 12i: Image to icosahedral projection for (3) object reasoning from single-view images. arXiv preprint arXiv:2207.08925, 2022.
Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. Imagenet classification with deep convo- lutional neural networks. Communications of the ACM, 60(6):84-9))) 90, 2017.
Michael Laskin, Aravind Srinivas, and Pieter Abbeel. Curl: Contrastive unsupervised representa- tions for reinforcement learning. In International Conference on Machine Learning, pp. 5639- 5650. PMLR, 2020a.
Misha Laskin, Kimin Lee, Adam Stooke, Lerrel Pinto, Pieter Abbeel, and Aravind Srinivas. Rein- forcement learning with augmented data. Advances in neural information processing systems, 33: 19884-19895, 2020b.
Giovanni Luca Marchetti, Gustaf Tegnér, Anastasiia Varava, and Danica Kragic. Equivariant repre- sentation learning via class-pose decomposition. arXiv preprint arXiv:2207.03/ 2022.
Haggai Maron, Or Litany, Gal Chechik, and Ethan Fetaya. On learning sets of symmetric elements. In International Conference on Machine Learning, pp. 6734-6744. PMLR, 2020.
Mirgahney Mohamed, Gabriele Cesa, Taco S Cohen, and Max Welling. A data and compute efficient design for limited-resources deep learning. arXiv preprint arXiv:2004.0969) 2020.
Arnab Kumar Mondal, Pratheeksha Nair, and Kaleem Siddiqi. Group equivariant deep reinforce- ment learning. arXiv preprint arXiv:2007.03337 2020.
Aaron den Oord, Yazhe Li, and Oriol Vinyals. Representation learning with contrastive predic- tive coding. arXiv preprint arXiv: 2018.
Jung Yeon Park, Ondrej Biza, Linfeng Zhao, Jan-Willem Van De Meent, and Robin Walters. Learn- ing symmetric embeddings for equivariant world models. In Proceedings of the 39th International Conference on Machine Learning, volume 162 of Proceedings of Machine Learning Research, pp. 17372-17389. PMLR, 17-23 Jul 2022. URL https: https:t//prcceeeeeeee http::///ccccceee..e/// . 62 2t22ppar222a html.
Adam Paszke, Sam Gross, Soumith Chintala, Gregory Chanan, Edward Yang, Zachary VeVito, Zeming Lin, Alban Desmaison, Luca Antiga, and Adam Lerer. Automatic differentiation in PyTorch. In NIPS Autodiff Workshop, 2017.
Robin Quessard, Thomas Barrett, and William Clements. Learning disentangled representations and group structure of dynamical environments. Advances in Neural Information Processing Systems, 33:19727-19737, 2020.
Tom Schaul, John Quan, loannis Antonoglou, and David Silver. Prioritized experience replay. arXiv preprint arXiv:1511.0559 2015.
Anthony Simeonov, Yilun Du, Andrea Tagliasacchi, Joshua B Tenenbaum, Alberto Rodriguez, Pulkit Agrawal, and Vincent Sitzmann. Neural descriptor fields: (3)-equivariant object rep- resentations for manipulation. In 2022 International Conference on Robotics and Automation (ICRA), pp. 6394 6400. IEEE, 2022.
Saran Tunyasuvunakool, Alistair Muldal, Yotam Doron, Siqi Liu, Steven Bohez, Josh Merel, Tom Erez, Timothy Lillicrap, Nicolas Heess, and Yuval Tassa. dm.control: Software and tasks for continuous control. Software Impacts, 6:100022, 2020. ISSN 2665-9638. doi: https://ooioog//.....f......pppp acc.aanceerrtteeeees//////6sss9s
URL https: www..sciineeiieeettttttt...e.e.. com/

---

Published as a conference paper at ICLR 2023
Elise van der Pol, Daniel Worrall, Herke van Hoof, Frans Olichoek, and Max Welling. Mdp homo- morphic networks: Group symmetries in reinforcement learning. Advances Neural Information Processing Systems, 33, 2020.
Robin Walters, Jinxi Li, and Rose Yu. Trajectory prediction using equivariant continuous convolu- tion. arXiv preprint arXiv:2010.1 0ett20000.113344 11344, 2020.
Dian Wang, Robin Walters, Xupeng Zhu, and Robert Platt. Equivariant Q learning in spatial action spaces. In 5th Annual Conference on Robot Learning, 2021. URL net/Eorum?id 1otfforumm2d11122223111111
Dian Wang, Mingxi Jia, Xupeng Zhu, Robin Walters, and Robert Platt. On-robot learning with equivariant models. In 6th Annual Conference on Robot Learning, 2022a. URL https://
openreview net fest/forum?iii www.roorruummaunomm wtt.rrmmmm.www//0o22oo..
Dian Wang, Colin Kohler, Xupeng Zhu, Mingxi Jia, and Robert Platt. Bulletarm: An open-source robotic manipulation benchmark and learning framework. arXiv preprint arXiv:2205.42292 2022b.
Dian Wang, Robin Walters, and Robert Platt. -euuiivavaa reinforcement learning. In Interna- tional Conference Learning Representations, 2022c. URL
Rui Wang, Robin Walters, and Rose Yu. Incorporating symmetry into deep dynamics models for improved generalization. arXiv preprint arXi::2002.03061, 2020.
Maurice Weiler and Gabriele Cesa. General e (2)-equivariant steerable emns. Advances in Neural
Information Processing Systems, 32, 2019.
Denis Yarats, Ilya Kostrikov, and Rob Fergus. Image augmentation is all you need: Regularizing deep reinforcement learning from pixels. In International Conference on Learning Representa- tions, 2021. URL pttttp:://ppeeeeiieee
Denis Yarats, Rob Fergus, Alessandro Lazaric, and Lerrel Pinto. Mastering visual continuous trol: Improved data-augmented reinforcement learning. In International Conference on Learning Representations, 2022. URL https: hwwp///wwwwwwweeeooo/etee net www.omrirurr.oooooooooorr _
Albert Zhan, Philip Zhao, Lerrel Pinto, Pieter Abbeel, and Michael Laskin. framework for effi cient robotic manipulation. arXiv preprint arXiv:20/2.07975, 2020.
Allan Zhou, Tom Knowles, and Chelsea Finn. Meta-learning symmetries by reparameterization. arXiv preprint arXiv:2007.02933, 2020.
Xupeng Zhu, Dian Wang, Ondrej Biza, Guanang Su, Robin Walters, and Robert Platt. Sample efficient grasp learning using equivariant models. In Robotics: Science and Systems, 2022.
12

---

Published as a conference paper at ICLR 2023
A THEORETICAL UPPER BOUND ON ACCURACY FOR MODELS WITH INCORRECT SYMMETRY
We consider classification problem over the X with finitely many classes Y. Let m | be the number of classes. Let 1: X - Y be the true labels. Let G be a finite group acting on X. We assume the action of on X is density preserving. That is, if px is the density function corresponding to the input domain, then px (gæ) (x). Denote the orbit of point e X by Ga {gx g E G} and the stabilizer by G2 = {g gz = æ}. By the orbit-stabilizer theorem [G] G= Gral
Now consider a model f: X + Y with incorrect equivariance constrained to be invariant to G. We partition the input set into subsets X _ Xs where
Xx { X (((G)) = k].
If has correct equivariance then X X1. Incorrect equivariance implies that there are orbits Ga which are assigned more than one label. Since f is constrained to be equivariant such orbits will necessarily result in some errors. We give an upper bound on that error. Define C P(x E Xx). Note that since X3 give partition, Σk=1 C* 1. Also, X is empty for k > |G] since the number of labels assigned an orbit also upper bounded by the number points the orbit which is at most [G). Letting K = min [G)), we have X = I Xg.
Tm
Proposition A.1. The accuracy of I has upper bound acc 1 - Ca 1)/GG/
In contrast, we can choose an unconstrained model from model class with universal approx- imation property and given properly chosen hyperparameters find model with arbitrarily good accuracy.
Proof. Let y (x)) Then ace(f) E (=(f(() = y) Since the action of G is density pre- serving, applying an element of G' before sampling does not affect the expectation, Ere (z) = y)] Exex (gz) y)] and so
acc(f)
Σ ( = v)].
G
9EG
we split the expectation over the partition X = =I X2 we get
Exxx (((((x))(( = 4)).
GG|
DEG
Interchanging sums gives
Σ 8(f(gz) -1)()))=1) -0)
S 1
1
Ck
ErEX.
GG
k=1
DEG
By the orbit-stabilizer theorem,
GG G
Σ 8(f(gz) = BEG
Σ
8(f(z)) =
Σ 8(f(z') y)
G
Ga
E'G
a'EGa
which is the average accuracy over the orbit Ga. Since f is constrained to single value of the orbit, and & different true labels appear, the highest accuracy attainable is when GG |Gz| and the true labels are maximally unequally distributed such that 1 point in the orbit takes each of l 1 labels and all the other |G| (k: - 1) points receive a single label. In this case accuracy can be maximized by choosing f(x') to be this majority label, and
8(f(x') v) 1
Gr
G
2e EG
13

---

Published as a conference paper at ICLR 2023
X
5%
3x
7x
4
2
4
Cg
3
π
π
4
2
4
p 1,c 0.25
p 0.5,cos 0.75
Figure 13: Demonstration of the upper bound of an equivariant model under invert label corruption in our supervised learning experiment. The number on each partition shows the ground truth label.
Substituting back in,
K
k
1-11
acc(f)
E
1
k=
K
k -
Ck
G
k=1
since k-1 is constant over X2 and Σ
2k=1
1.
Note that the assumption that G |Gal and that the labels on given orbit are maximally un- equally distributed need not hold in general and thus this bound is not tight. In order to produce tight upper bound, consider a partition X = [Ip Xp where X, {x E X (maxylff" n Grl)/ Gx p} and define Cp P(x X,). The set X,. contains points in orbits where the majority label covers a fraction of the points. Note that although is a fraction between O and 1, there only finitely many possible values of since the numerator and denominator and bounded natural numbers. We may thus sum over the values of
Proposition A.2. The accuracy of f has upper bound ace(f) S Ep Cpp.
Proof. The proof similar to the proof Proposition A.1 replace X₃ and Cs with X, and cp respectively. For a E Xp. the term the majority label yielding
Exees 8(f(x') y) can be upper bounded by choosing Ca
|||
[1] [Ga
EveG 8(f(z') 4) p. The bound then follows as before.
This is tight upper bound since assigning any but the majority label would result in lower accuracy.
Figure 13 demonstrates the upper bound of an incorrectly constrained equivariant network with the invert label corruption in Section 5, where acc(f) 0.25 1 + 0.75 0.5 0.625.
B CORRECT, INCORRECT, AND EXTRINSIC EQUIVARIANCE EXAMPLES
this section, we describe how the model symmetry transforms data under correct, incorrect, and extrinsic equivariance and how such transformations relate to the true symmetry present in the task using the example of Section 4.2. The ground truth function f X -
a mapping from X R² to Y {ORANGE, BLUE}. Let (a, a), (-a, -a), (-a, a), (b, c) be the coordinates of four points in the data distribution on the unit circle (Figure 14a). The ground truth labels for these points are: f(a, a) ORANGE, f(=a, -a) BLUE, f( (aaa a) BLUF, (bb c) ORANGE.
14

---

Published as a conference paper at ICLR 2023
Gj reflection
60
correct
6
Gy scaling
extrinsic
(a)
Gg rotation Incorrect
X
O O
8
6
80
80
80
(C)
(d)
An example classification task for correct, incorrect, and extrinsic equivariance. The input distribution shown as gray ring. The training data samples are shown as circles, where the color is the ground truth label. Crosses represent the group transformed data. The opaque points highlight the example points while other points are semitransparent.
Figure 14:
B.1 CORRECT EQUIVARIANCE
Definition 4.1. The action Pa has correct equivariance with respect / if Px(g) E D for all E D,ge E G and f(px(g)z) (()))))))((())((a))
Consider the reflection group G C2 [1.r] (where r is the reflection along the horizontal axis) acting X by (1) ld (r(r) = (6 -1) and Y via Py Id, the trivial action fix- ing the labels (Figure 14b). If we define an equivariant model with respect to Pa and Py, then the model's symmetry preserves the problem symmetry. For example, consider the point (-a, a), E G1 is the reflection so that (r(r) ( 9) and (r) a) (-a, -a). Since the model fo is Gi-equivariant, (r)))(r))) = (a())) Substituting Py ld and = (-a, a), we obtain fo(-a, -a) fo(-a,a) a). meaning that the output of fo(-a, -a) and fo( a) constrained to be equal. Thus the invariance property in the ground truth function f where f(-a, a) = f(-a,a) = BLUE is preserved (notice that this applies to all x E X). We call this correct equivariance.
B.2 INCORRECT EQUIVARIANCE
Definition 4.2. The action Pme has incorrect equivariance with respect to f if there exist æ E D and g E G such that (g(g E but f(px(g)x) A (z).
Consider the rotation group G2 (Rotz) (Figure 14c) which acts via P. on X via rotation matrix define an equivariant model with respect to P.. and Py, the
of æ and acts on Y via Py ld. network's symmetry will conflict with the problem's symmetry. For example, consider the point (a, a) and let g E G2 be the rotation action so that (x) = (-- % and (r) a) = (-a, -a). As the model f is Gz-equivariatt, (o( = Py(g) (x)) Substituting Py Id and x = (a, a), a) (a a). However, this constraint interferes with the ground truth function f as f(-a, a) BLUE and f(a, a) ORANGE. We call this incorrect equivariance.
we get fo(-
B.3 EXTRINSIC EQUIVARIANCE
Definition 4.3. The action has extrinsic equivariance with respect to f ifr D,px(g)2 & D.
Consider the scaling group Gg acting on X by scaling the vector and on Y via Py = Id (Figure 14d). If define equivariant model with respect Px. and Py: the group-transformed data will be
15

---

Published as a conference paper at ICLR 2023
outside the imput distribution. Consider the point (b, and let G3 be the scaling action that (x(g)(b, c) = (b', d'). Since the model fo is Gg-equivariant, fo))=(g)x) = Py(g) (x) Substitut- (b, c) have fa
ing Py = Id and fa(b, c) are constrained to be equal. However, (b', c') is outside the input distribution (gray ring) and thus the ground truth S(b', c') is undefined. We call this extrinsic equivariance.
(b, c) meaning that the output of fb (0',cc c') and
Intuitively, it is easy to see this example how extrinsic equivariance would help the model learn f. If the model f equivariant the scale group G3. then can generalize "scaled" up or down versions of the input distribution and "covers" more of the input space I2?. As such, the model may lear the decision boundary (the vertical axis) more easily because of its equivariance compared to non-equivariant model, even if the equivariance is extrinsic.
C NETWORK ARCHITECTURE
Bogg hune EXB
Selaze busoo pe
begy/e Wutto pec
jood yea
1ood Xeb
tea
Begye buroo
jold xeaa
j0od sebai
Beens
Figure 15: Network architecture of the equivariant network the supervised learning experiment.
"Nuoo DIg
Food 20u
pood stu
Nuso Exg
meu
jood stu
pood sen
R2: Witeo esp
huzee
Figure 16: Network architecture of the CNN network in the supervised learning experiment.
Table I: Number of trainable parameters of the equivariant network (Equi) and conventional CNN network (CNN) in the supervised learning task.
Network
Equi
CNN
Number of Parameters
1.11 million 1.28 million
C.I SUPERVISED LEARNING
Figure shows the network architecture of the equivariant network and Figure 16 shows the net- work architecture of the CNN network in Section 5. Both networks 8-layer convolutional neural networks. The equivariant network is implemented using the e2cnn (Weiler & Cesa, 2019) library, where the hidden layers are defined using the regular representation and the output layer is defined using the trivial representation. Table shows the numbers of trainable parameters in both networks, where both networks have similar number with a slight advantage in the CNN.
C.2 REINFORCEMENT LEARNING IN ROBOTIC MANIPULATION
Figure 17 shows the network architecture of Equivariant SAC used in manipulation tasks in Sec- tion 6. 1. All hidden layers are implemented using the regular representation. For the actor (top), the output is mixed representation containing one signed representation for the action, and seven trivial representations for the (z, A) actions and the standard deviations of all action components. Figure 18 shows the network architecture of CNN SAC for both RAD and DrQ. Figure 19 shows the network architecture of FERM. Figure 20 shows the network architecture of SEN.
standard representation for the (z, y) actions,
Table 2 shows the number trainable parameters for each model. All baselines have slightly more parameters compared with Equivariant SAC.
16

---

Published as a conference paper at ICLR 2023
Eagas grg
Degns NuDo SXg
BnWge Waacgee
Eagys Nuco gAc
Degts NNDo EXS
Eagll suoc
8000 stuu
Rlod smuu
Jay
Beyat Auso grD
Begir huso gig
Geyps nuso grD
Debts huso gig
HiLL 1Hg
Iooti Jebu
pondi seba
I000 dhoul
10
poost
Begre Trueo
pood dhout
AUL
Figure 17: Network architecture of Equivariant SAC in robotic manipulation tasks.
S: gig
(0od xeu
(ood sbuu
Wuboo pig
10od new
021 Hetdo pip
992 Nuto pXp
1ood seu
882 Wuoo gi
0: Nupo LX
upou
28
P3
Pic
S: Iuuzo 200
pood xeeu
CMC
PPS huuo
[ond sinu
021 Yutzo gig
1ood xet
021 Natoo gig
9992 Nuso Dip
1ood XENu
962 INMO EX
992 Watoo
Nu00 Lx
10
2tt
852 Waoe xx
20
Figure 18: Network architecture of CNN SAC robotic manipulation tasks.
D TRAINING DETAILS
D. SUPERVISED LEARNING
We implement the environment in the PyBullet simulator (Coumans & Bai, 2016). The ducks located in a workspace with a size of 0.3m x 0.3m. The pixel size the image is 152 x 152 (and will be cropped to 128 x 128 during training). We implement the training in PyTorch (Paszke al., 2017) using a cross-entropy loss. The output of the model the score for each g E C.. We use the Adam optimizer (Kingma & Ba, 2014) with learning rate of 10-4, The batch size is 64. In all training, we perform a three-way data split with N training data, 200 holdout validation data, and 200 holdout test data. The training is terminated either when the validation prediction success rate does not improve for 100 epochs or when the maximum epoch (1000) is reached.
D.2 REINFORCEMENT LEARNING IN ROBOTIC MANIPULATION
We use the environments provided by the BulletArm benchmark (Wang et al., 2022b) implemented in the PyBullet simulator (Coumans & Bai, 2016). The workspace's size is 0.4m x 0.4m x 0.24m. The pixel size of the image observation is 152 x 152 (and will be cropped to 128 x 128 during training). The action space is Ag. Ag |-0. 0.05m, 0.05m] for the change of (x, y, position
of the gripper, Ag [- 7. 71 for the change of top-down rotation of the gripper; and A, [0, 1] for the open width of the gripper where ( means fully close and I means fully open. All environ-
17

---

Published as a conference paper at ICLR 2023
Wuu
Yuooo
100d ettu
Yuuto EMB
pood Vtu
'Wolo
99g HuDD
08
mem Joller
Japly bapoo
2i003
904
pood
+9
819.4
2:002
SOS
215
219
219
PO1
z0
Figure 19: Network architecture of FERM in robotic manipulation tasks.
9 'uuo g%
(000 vtu
gtc
1000 mtu
75 HuDo gPg
1000 ktu
82 gXg
(000 MtU
Od
XWy Eyg
75 HuDo
28
pge
9 moon 208
1000 mmu
28 HIUT 596
1000 veu
79 1uur3 spt
1000 yau
1000 yeu
75 1uu3 ges
poad dhrodd
AU1 Neado
10
Deucct
Emeutt rucD
pald dhoud
huaoo
Figure 20: Network architecture of SEN in robotic manipulation tasks.
ments have a sparse reward: +1 for reaching the goal and 0 otherwise. During training, we use parallel environments where a training step is performed after all 5 parallel environments perform an action step. The evaluation is performed every 200 training steps. We implement the training in PyTorch (Paszke al., 2017). We the Adam optimizer (Kingma & Ba. 2014) with a learning rate of 10-3 The batch size is 128. The entropy temperature for SAC is initialized at 10-2, The target entropy -5. The discount factor 7 0.99. The Prioritized Experience Replay (PER) (Schaul al., 2015) has a capacity of 100,000 transitions with prioritized replay exponent of 0.6 and prioritized importance sampling exponent 3n 0.4 as in Schaul et al. (2015). The expert transitions given priority bonus of ca 1.
The contrastive encoder of the FERM baseline has an encoding size of 50 as in Zhan et al. (2020). The FERM bascline's contrastive encoder is pre-trained for 1.6k steps using the expert data as in Zhan et (2020). In DrQ. the number of augmentations for calculating the target K and the number of augmentations for calculating the loss M are both in Yarats et al. (2021).
18

---

Published as a conference paper at ICLR 2023
Table 2: Number of trainable parameters of Equivariant SAC, CNN SAC, FERM, and SEN in the reinforcement learning task in robotic manipulation. Notice that FERM has a shared encoder be- tween the actor and the critic so the total number of parameters is smaller than the sum of the actor parameter and the critic parameter.
Network
Equi SAC
CNN SAC
FERM
SEN
Number of Actor Parameters
1.11 million
1.13 million
1.79 million
1.22 million
Number of Critic Parameters
1.18 million
1.27 million
1.90 million
1.24 million
Number of Total Parameters
2.29 million
2.40 million
2.34 million
2.46 million
D.3 REINFORCEMENT LEARNING IN DEEPMIND CONTROL SUITE
Sample images of each environment are shown in Figure 21. Environment observations are 3 con- secutive frames of RGB images of size x 85, order to infer velocity and acceleration. Note that odd-sized image sizes instead of 84 x 84 used in Yarats al. (2022), as the DrQv2 architecture contains a convolutional layer with stride 2 and this breaks equivariance for even-sized spatial inputs (Mohamed al., 2020). For each environment, an episode lasts 1000 steps where each step has a reward between and 1.
(a) Cartpole Balance
(b) Cartpole Swingup
(c) Pendulum Swingup
(d) Cup Catch
(e) Acrobot Swingup
(l) Reacher easy
(g) Reacher hard
Figure 21: DeepMind Control Suite: images of easy (top) and medium (bottom) tasks.
modify the original DrQv2 by making the encoder map down to smaller spatial output, leading to faster training. The second and third convolutional blocks have an added max-pooling layer, leading to spatial output size of x 7. As the equivariant version DrQv2 has an additional convolutional layer after the action restriction. the non-equivariant version also has additional convolutional layer the end of the encoder. We also scale the number of channels by vIG in order to preserve roughly the same number of parameters as the non-equivariant version.
The policy is evaluated by averaging the return of 10 episodes every 10000 environment steps. In all DMC experiments, we plot the mean and the standard error over 4 seeds. All other training details and hyperparameters are kept the same in Yarats et al. (2022).
19

---

Published as a conference paper at ICLR 2023
corrupt
grid
occlusion
random light color
light effect
light specular
squeeze
reflect
Label inverted if yallow is the left refeet
side
side,grid
side,light effect
side,squeeze
Figure 22: All symmetry corruptions in the rotation estimation experiment.
ADDITIONAL. EXPERIMENTS
E. SUPERVISED LEARNING WITH MORE SYMMETRY CORRUPTION TYPES
In this section, we demonstrate the experiment Section more symmetry corruptions. Fig- 22 shows the 15 different corruptions. We also show the performance of 'Equi', 'CNN', and "CNN + Img Trans' without the random crop augmentation used in Section Section (labeled as 'no Crop' variations). The result is shown in Figure 23. First, comparing blue green, and pur- ple vs orange, the equivariant network always outperforms the CNN with or without random crop augmentation, especially with fewer data. Second, comparing blue purple, and green orange, random crop generally helps both the equivariant network and the CNN network. Third, comparing red vs green, and cyan vs orange, adding the image transformation augmentation improves the per- formance of CNN. Notice that the condition reverse is an outlier because the equivariant network has incorrect equivariance, where the CNN methods (green and orange) without image transformation augmentation have the best performance.
E.2 RL IN MANIPULATION WITHOUT RANDOM CROP
In this section, we demonstrate the performance of Equivariant SAC and CNN SAC without random crop augmentation using RAD. As is shown in Figure 24, both methods work poorly without the random crop augmentation.
E.3 RL IN MANIPULATION WITH OCCLUSION CORRUPTION
In this section, we perform the same experiment as in Section 6.1 with different type of symmetry corruption: occlusion due to orthographic projection using a single camera. Instead of using an RGBD image observation as in Section 6.1, we take the depth channel from the RGBD image and perform an orthographic projection the gripper's position (Figure 25). This is the same process as in Wang et al. (2022a) to generate top-down image for equivariant learning, however, since we only have one camera instead of two as in the prior work, this orthographic projection will have missing depth values due to occlusion and thus leads to extrinsic equivariant constraint. Figure 26 shows the results. Similar as in Section 6, Equivariant SAC outperforms all baselines with a significant margin.
20

---

Published as a conference paper at ICLR 2023
corrupt
occlusion
random light
light effect
4.0
1.0
1.
10
0.3
0.
0.8
0.0
04
04
24
0.6
06
0.1
04
0.4
0
02
89
0.
02
00
100230
130
1600
330.00
20
1030
100.00
5000
08
100000
600
1000
08
132300
530
1000
Test Accurecy
two light color
two specular
squeeze
reflect
1.0
1.0
0.0
0.
0.0
100330
1000
300.00
1000
300.00
2000
1000200
500
1000
120200
side
side, grid
side, light effect
side, squeeze
side, reflect
003230
331200
100000
300200
Number Training
Equi
CMN lmg
Equi Crop
Crop
CNN Img Crop
Figure 23: Comparison of equivariant network (blue), conventional network (green), CNN equipped with image transformation augmentation (red), and their variation without random crop augmentation (purple, orange, cyan). The plots show the prediction accuracy in the test of the model trained with different number of training data. Results are averaged over four runs. Shading denotes standard error.
Block Pulling
Block Pushing
Block Picking
Drawer Opening
Block Bowl
Discouneed rewerd
0L5
0.2
3990
02
Number training steps
Equi SAC
Figure 24: Comparison between Equivariant SAC and CNN SAC without data augmentation using RAD. The plots show the performance (in terms of discounted reward) of the evaluation policy. The evaluation is performed every 200 training steps. Results averaged over four runs. Shading denotes standard error.
E.4 RL DEEPMIND CONTROL SUITE
Figure 27 is another visualization of equivariant vs coneequivariant DrQv2 on the original pendulum swingup environment. As each method has failed seed, we plot all runs with slightly different color shades. If we exclude the failed run from each method, it can easily be seen that equivariant DrQv2 learns faster than the non-equivariant version.
E.4.1 INCREASING SYMMETRY CORRUPTIONS
In these experiments, we modify some domains have different levels of symmetry-breake cor- ruptions. For cartpole and cup catch, we either remove the gridded floor and background (None) make the observation perfectly equivariant or keep the floor and background and further change the camera angle by rolling (30' 75"), increasing the level of corruption. For reacher, we use the same modifications but tilt the camera instead of rolling. See Figure for sample images. In order to see the effects of increasing corruption learning, we plot the mean discounted reward when both methods have converged (30k frames for cartpole and cup catch. 1.5M frames for reacher). Figure 28 shows that both the equivariant and non-equivariant DrQv2 surprisingly perform quite well across all corruption levels, with the exception of 75 on reacher. The equivariant policy seems converge slightly higher discounted reward than the non-equivariant version, though the dif-
21

---

Published as a conference paper at ICLR 2023
Figure 25: Left: the depth image taken from depth camera. Right: the orthographic projection centered at the gripper position generated from the left image, where the black areas are missing depth values due to occlusions.
Block Pullinng
Block Pushing
Block Picking
Drawer Opening
Block Bowl
Discounted reward 0-
no no
0.
0.4
0.2
000
Number steps
Equi
DrQ
FERM
Figure 26: Comparison of Equivariant SAC (blue) with baselines in environments with occlusion corruption. The plots show the performance (in terms of discounted reward) of the evaluation policy. The evaluation is performed every 200 training steps. Results averaged over four runs. Shading denotes standard error.
ference is not significant. On reacher, changing the camera angle may have affected both methods by making the task more difficult for both equivariant and regular CNN encoder.
F BASELINE ARCHITECTURE SEARCH
F.1 CNN SAC ARCHITECTURE SEARCH
This section demonstrates the architecture search for CNN SAC. We consider three different archi- tectures (all with a similar amount of trainable parameters): 1) conv (Figure 18); a CNN network with the same structure as Equivariant SAC, where all layers implemented using convolutional layers. 2) fel (Figure 29): CNN network that replaces some layers in 1) with fully connected lay- ers. 3) fc2 (Figure 30): similar as 2), but with fewer convolutional layers and more weights in the FC layer. We evaluate the three network architectures with SAC equipped random crop augmentation using RAD (Laskin al., 2020b).
Figure 31 shows the result, where all three variations have similar performance. We use conv in the main paper since has similar structure as Equivariant SAC.
F.2 FERM ARCHITECTURE SEARCH
This section demonstrates the architecture search for FERM. We consider four different architec- tures: 1) sim total (Figure 33) and 2) sim total 2 (Figure 19) are two different architectures with the similar amount of total trainable parameters as Equivariant SAC. 3) sim enc (Figure 32) has similar amount of trainable parameters in the encoder Equivariant SAC's encoder. Notice that since FERM share an encoder between the actor and the critic while Equivariant SAC has separate encoders, having the similar amount of parameters in the encoder will lead to fewer total parameter in FERM compared with Equivariant SAC. 4) ferm ori (Figure 34) is the same network architecture used in the FERM paper (Zhan et al., 2020).
Figure 35 shows the comparison across the four architectures. 'sim total 2' has a marginal advantage it in the main paper.
compared with the other three variations,
22

---

Published as a conference paper at ICLR 2023
Pendulum Swingup
800
Discounted
200
8.2
Frames
Equivariant
Non-equivvrrant
Figure 27: All runs of equivariant and non-equivariant DrQv2 on the DMC pendulum swingup task. Each method has failed seed the failed equivariant policy (blue) run consistently near zero reward and the failed non-equivariant policy run (red) is around 200. Overall, the equivariant DrQv2 learns faster than the non non-equivariant version when succeeds.
Cartpole Swingup
0C
CC
Cc
C
Cup Catch
Reacher hard
None
Orig
30
45
60
75
Modifications to DMC domains for varying symmetry corruption levels. The gridded floor and background removed to be fully equivariant (None) or the camera angle is modified to increase the level of symmetry corruption (roll for cartpole and cup catch, tilt for reacher).
Table 3:
F.3 SEN ARCHITECTURE SEARCH
This section shows the architecture search for SEN. We consider three variations (all with similar amount of trainable parameters): 1) SEN cony (Figure 36): all layers implemented using volutional layers. 2) SEN fcl (Figure 37) and SEN fc2 (Figure 20) replaces some layers in 1) with fully connected layers.
Figure 38 shows the comparison across the three variations. 'SEN fc2' shows the best performance.
23

---

Published as a conference paper at ICLR 2023
Cartpole Swingup
Cup Catch
Reacher Hard
Discounted reward
500
520
600
200
400
420
012
0'0
01g
Corruption Level
Equivariant DrQv2
Non-equivvrrant DrQv2
Figure 28: DMC performance comparison various levels of symmetry corruptions. Both the
equivariant and non-equivariant DrQv2 perform quite well even with increasing levels of corruption.
to Nubo gie
100d meu
85 gre
dbc Nuut eic
100d sou
85:
DDP
19 huoo ei
80 'wuto git
661 Notto etc
(6od sau
(005 yeu
u2: 'euta
(800 you
1:4
10
104
Figure 29: Network architecture of the 'fel' variation for CNN SAC.
22
19
88:
xe geg
881
Figure 30: Network architecture of the 'c2' variation for CNN SAC.
Block Pulling
Block Pushing
Block Picking
Drawer Opening
Block in Bowl
Discounted reward
0.3
0.7
0.2
2.0
1000
2003
4000
0.3
2000
00
Number of training steps
conv
fc1
fc2
Figure 31: Architecture search for CNN SAC. The plots show the performance (in terms of dis- counted reward) of the evaluation policy. The evaluation is performed every 200 training steps. Results are averaged over four runs. Shading denotes standard error.
24

---

Published as a conference paper at ICLR 2023
Nuzzo pMc
(ooi seu
73 Nuzo pop
(001) ainu
80 gng
uaiiy jadlery
JPPH
080 0a
952.04
UnSU
00
Jopay
8
882 0z
992 02
Dis
082 00
395
092 Od
L.44
Figure 32: Network architecture of the 'sim enc' variation for FERM.
28 gr
Yuaco
10od senu
BB1 wuaoo pip
1ood sen
952 gPc
mully Jallrr
e90 0dd
UReU
10od
09
868 Pd
952 Rd
PO
D2
852.0 00
96204
1922 0z
959
1:04
Figure 33: Network architecture of the 'sim total
variation for FERM.
25

---

Published as a conference paper at ICLR 2023
2:0 Da
4000 De
110
g
8
V0DI Cd
904
100 De
100 Oz
2000 Cd
1004
Figure 34: Network architecture of the 'ferm ori' variation for FERM.
Block Pulling
Block Pushing
Block Picking
Drawer Opening
Block Bowl
Discounted revard
04
130 700
2010
700
5600
1000
XCC
30C
430
5000
1000
2001 ando Wids 5ine
1300 2030
3000
1003
1000
300 430 030) 1801 1966
Number of training stops
sim total
sim total 2
sim
ferm on
Figure 35: Architecture search for FERM. The plots show the performance (in terms of discounted reward) of the evaluation policy. The evaluation is performed every 200 training steps. Results averaged over four runs. Shading denotes standard error.
06 Nubo gec
1003 yau
pdrd Mit
Wett Nutto DPC
pood meu
68c Hutoo DNC
pea maa
215 JXL
Nuto
AB
XBy
By C000 200
1000 102
pood 3t12
Bic Husto DNC
pood attu
692 DXC
pood sitaa
Zi5 HUD0 Myg
NULL supo JB
Yrubo
821
AULL rup0 Mi
Besrr
por
Figure 36: Network architecture of 'SEN conv' variation of SEN.
26

---

Published as a conference paper at ICLR 2023
(000 yoN
(0o0 yenu
78 6MC
88 "uuoo eX
901 Wuuto pxg
eXC
215 Gd
Yuato
(Bod
2
(8
Weaoo eX0
pood Xeuu
pood Jebu
EXC
pood yeau
Yuzo gi
Yeuto ere
219 Obu
(000 ahous
992
HiLL
LKL
pood groul
AUL
Figure 37: Network architecture of 'SEN fc1' variation of SEN.
Block Pulling
Block Pushing
Block Picking
Drawer Opening
Block in Bowl
Discounitd reward
0.5
64
0.4
022
00
2.0
1000
3000
4000
0.3
2000
00
Number of training steps
SEN COnV
SEN
SEN
Figure 38: Architecture search for SEN. The plots show the performance (in terms of discounted reward) of the evaluation policy. The evaluation is performed every 200 training steps. Results are averaged over four runs. Shading denotes standard error.
27


Reviews:
[{"review_id": "WJjrJ5wu6lo", "reviewer": "Reviewer_bQBQ", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper proposes to learn *extrinsic* symmetries and equivariances in settings that have latent symmetries that are challenging to manually impose with standard group equivariance. The method is evaluated mostly on pixel-based settings with convolutional networks with a significant focus on pixel reinforcement learning tasks.", "summary_of_the_review": "I recommend to accept this paper as the idea of extrinsic equivariance is appealing and the experimental evaluation is convincing and improves upon many established results.", "strength_and_weaknesses": "Strengths\n+ The experimental evaluation is extensive and significantly improves upon established results for equivariant SAC/RAD (Fig 7, 9, 11) and DrQv2 (Fig 12).\n+ The appendix includes a significant amount of ablations and details, for example the tasks in Table 3 and Figure 27 on evaluating the DMC tasks with corruptions to the symmetry/camera pose\n\nWeaknesses\n+ While the paper is motivated by unknown latent symmetries, I thought it involve learning them. However, as reading the paper, I realized that the extrinsic equivariances still need to be manually specified as auxiliary equivariances that are simply other known transformations.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works.", "empirical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works."}, "extra_text": {}}, {"review_id": "U-NgVFV4BZb", "reviewer": "Reviewer_6UwC", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": " This paper explores the settings where an equivariant model can\n improve sampling efficiency and generalization by proposing a\n distinction between equivariant models that preserve the true latent\n symmetry and those which make it impossible to represent. These\n equivariant models that preserve the true latent symmetry even while\n not explicitly modeling it are called \"extrinsic\n equivariance\". Through a combination of theoretical and empirical\n results, this paper shows how extrinsic symmetries improve models. In\n fact, some of the improved generalization may come from\n \"extrinsically equivariant\" models transforming input data out of\n distribution in a way that non-equivariant models simply don't do.\n", "summary_of_the_review": "A great contribution to the equivariant models literature, but the value is not in any one particular method.", "strength_and_weaknesses": " Overall, I view the paper as a useful contribution to the literature\n as it provides greater clarity for where equivariant models can really\n shine. While the paper does introduce an equivariant version of DrQv2,\n I feel the contribution is introducing the distinction between \"incorrect \n equivariance\" and \"extrinsic equivariance\".\n\n The main weakness of the paper is it doesn't introduce new models. All\n the theory and most of the experiments are done using existing equivariant\n models in the literature. Much of the benefits of equivariant models are\n already known as the related work highlights and there are already many\n methods for learning latent symmetries.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "Nv-PuINy18", "reviewer": "Reviewer_ZXUL", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This work defines correct equivariance, incorrect equivariance, and extrinsic equivariance. Correct symmetry means that the model symmetry correctly reflects a symmetry present in the ground truth function, for which it is correct to enforce equivariance constraints. Extrinsic equivariance is when the equivariant constraint in the equivariant network enforces equivariance to out-of-distribution data. The authors theoretically demonstrate the upper bound performance for an incorrectly constrained equivariant model. In a supervised classification setting, they empirically show that a model with extrinsic equivariance can aid learning compared with an unconstrained model, especially in a low-data regime. Then, they explore this idea in a RL context and show that an extrinsically constrained model can outperform state-of-the-art conventional CNN baselines.", "summary_of_the_review": "I lean to acceptance but would like the authors to improve the clarify and comment on effectiveness in large-data regime.", "strength_and_weaknesses": "Strength\n* This work empirically shows that extrinsic equivariance can still aid learning, which is very well-motivated contribution to model architecture with image transformation augmentation. \n* Empirical evaluations are strong and comprehensive, covering many kinds of transformations, supervised learning, and reinforcement learning.\n\nWeaknesses\n* The scale of the model and data used in these experiments are still considered small. Given the results in slightly larger data-regime, it's not clear how much it could be useful with large model and large data.\n* When the term correct, incorrect, and extrinsic equivariance/symmetry appear the first time in the paper, they are not well-explained. Even in the following, it's still unclear what the definition is and examples are somewhat unclear. I wonder if it would be useful to at least have section in appendix to clarify is the main paper space is too constrained.\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "Nki4nVDBoa", "reviewer": "Reviewer_a4PJ", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper defines three types of equivariance: correct, incorrect and extrinsic. The authors prove an upper bound on the accuracy when incorrect equivariance is used. Then the authors show empirically that using incorrect equivariance (e.g. using an equivariant neural network with incorrect labels) is worse than using a neural network, without the equivariant constraint. Finally, the authors shows that extrinsic equivariance can be helpful for policies in reinforcement learning. The authors experiment in two domains: 1) simple synthetic domain and 2) reinforcement learning tasks on robotic manipulation and DeepMind Control Suite.", "summary_of_the_review": "I like the paper, because the problem of equivariance for neural networks is widely recognized, the proposed studies and theory are quite useful, and further research in that direction is promising, and may benefit from the findings in that paper.", "strength_and_weaknesses": "Strengths:\n\n1. Very important idea that the equivariance may be latent, and not necessarily captured in the input transformation.\n\n2. Useful definitions of correct, incorrect and extrinsic equivariances that capture the intuition of the problem.\n\n3. Really neat proof of Proposition A.1.\n\nWeaknesses:\n\n1. The authors address it in the end of their paper, but it would be nice to consider the method in other tasks / settings. I wonder if the power of the equivariant policy would be even further demonstrated in a realistic scenario, where the states are actual natural images. In that setting the extrinsic equivariances will be \"natural.\" Might be a cool experiment to try.\n\n2. Some minor comments: \n\n* It would be great if you could put Figure 2 on the first page to visualize what you mean in the discussion.\n\n* In Section 4.1. it is not clear what the relationship between $\\hat{\\rho}_x$ and $\\rho_x$ is. Can you give some examples for intuition early in the paper?\n\n* Small typo in the proof of Proposition A.1. \"labels and maximally\" <- \"labels are maximally\".\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Decision: Accept: notable-top-25%