Original: T HE S URPRISING E FFECTIVENESS OF E QUIVARIANT M ODELS IN D OMAINS WITH L ATENT S YMMETRY
Revised: T HE S URPRISING E FFECTIVENESS OF E QUIVARIANT M ODELS IN D OMAINS WITH L ATENT S YMMETRY

Original: Anonymous authors
Revised: Anonymous authors

Original: Paper under double-blind review
Revised: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: Extensive work has demonstrated that equivariant neural networks can significantly improve sample efficiency and generalization by enforcing an inductive bias in the network architecture.
Revised: Extensive work has demonstrated that equivariant neural networks can significantly improve sample efficiency and generalization by enforcing an inductive bias in the network architecture.

Original: These applications typically assume that the domain symmetry is fully described by explicit transformations of the model inputs and outputs.
Revised: These applications typically assume that the domain symmetry is fully described by explicit transformations of the model inputs and outputs.

Original: However, many real-life applications contain only latent or partial symmetries which cannot be easily described by simple transformations of the input.
Revised: However, many real-life applications contain only latent or partial symmetries which cannot be easily described by simple transformations of the input.

Original: In these cases, it is necessary to learn symmetry in the environment instead of imposing it mathematically on the network architecture.
Revised: In these cases, it is necessary to learn symmetry in the environment instead of imposing it mathematically on the network architecture.

Original: We discover, surprisingly, that imposing equivariance constraints that do not exactly match the domain symmetry is very helpful in learning the true symmetry in the environment.
Revised: We discover, surprisingly, that imposing equivariance constraints that do not exactly match the domain symmetry is very helpful in learning the true symmetry in the environment.

Original: We differentiate between extrinsic and incorrect symmetry constraints and show that while imposing incorrect symmetry can impede the model’s performance, imposing extrinsic symmetry can actually improve performance.
Revised: We differentiate between extrinsic and incorrect symmetry constraints and show that while imposing incorrect symmetry can impede the model’s performance, imposing extrinsic symmetry can actually improve performance.

Original: We demonstrate that an equivariant model can significantly outperform non-equivariant methods on domains with latent symmetries both in supervised learning and in reinforcement learning for robotic manipulation and control problems.
Revised: We demonstrate that an equivariant model can significantly outperform non-equivariant methods on domains with latent symmetries both in supervised learning and in reinforcement learning for robotic manipulation and control problems.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: Recently, equivariant learning has shown great success in various machine learning domains like trajectory prediction (Walters et al., 2020), robotics (Simeonov et al., 2022), and reinforcement learning (Wang et al., 2022c).
Revised: Recently, equivariant learning has shown great success in various machine learning domains like trajectory prediction (Walters et al., 2020), robotics (Simeonov et al., 2022), and reinforcement learning (Wang et al., 2022c).

Original: Equivariant networks (Cohen & Welling, 2016; 2017) can improve generalization and sample efficiency during learning by encoding task symmetries directly into the model structure.
Revised: Equivariant networks (Cohen & Welling, 2016; 2017) can improve generalization and sample efficiency during learning by encoding task symmetries directly into the model structure.

Original: However, this requires problem symmetries to be perfectly known and modeled at design time – something that is sometimes problematic.
Revised: However, this requires problem symmetries to be perfectly known and modeled at design time – something that is sometimes problematic.

Original: It is often the case that the designer knows that a latent symmetry is present in the problem but cannot easily express how that symmetry acts in the input space.
Revised: It is often the case that the designer knows that a latent symmetry is present in the problem but cannot easily express how that symmetry acts in the input space.

Original: For example, Figure 2b is a rotation of Figure 2a.
Revised: For example, Figure 1b is a rotation of Figure 1a.

Original: However, this is not a rotation of the image – it is a rotation of the objects present in the image when they are viewed from an oblique angle.
Revised: However, this is not a rotation of the image – it is a rotation of the objects present in the image when they are viewed from an oblique angle.

Original: In order to model this rotational symmetry, the designer must know the viewing angle and somehow transform the data or encode projective geometry into the model.
Revised: In order to model this rotational symmetry, the designer must know the viewing angle and somehow transform the data or encode projective geometry into the model.

Original: This is difficult and it makes the entire approach less attractive.
Revised: This is difficult and it makes the entire approach less attractive.

Original: In this situation, the conventional wisdom would be to discard the model structure altogether since it is not fully known and to use an unconstrained model.
Revised: In this situation, the conventional wisdom would be to discard the model structure altogether since it is not fully known and to use an unconstrained model.

Original: Instead, we explore whether it is possible to benefit from equivariant models even when the way a symmetry acts on the problem input is not precisely known.
Revised: Instead, we explore whether it is possible to benefit from equivariant models even when the way a symmetry acts on the problem input is not precisely known.

Original: We show empirically that this is indeed the case and that an inaccurate equivariant model is often better than a completely unstructured model.
Revised: We show empirically that this is indeed the case and that an inaccurate equivariant model is often better than a completely unstructured model.

Original: For example, suppose we want to model a function with the object-wise rotation symmetry expressed in Figure 2a and b. Notice that whereas it is difficult to encode the object-wise symmetry, it is easy to encode an image-wise symmetry because it involves simple image rotations.
Revised: For example, suppose we want to model a function with the object-wise rotation symmetry expressed in Figure 1a and b. Notice that whereas it is difficult to encode the object-wise symmetry, it is easy to encode an image-wise symmetry because it involves simple image rotations.

Original: Although the image-wise symmetry model is imprecise in this situation, our experiments indicate that this imprecise model is still a much better choice than a completely unstructured model.
Revised: Although the image- wise symmetry model is imprecise in this situation, our experiments indicate that this imprecise model is still a much better choice than a completely unstructured model.

Original: This paper makes three contributions.
Revised: This paper makes three contributions.

Original: First, we define three different relationships between problem symmetry and model symmetry: correct equivariance , incorrect equivariance , and extrinsic equivariance .
Revised: First, we define three different relationships between problem symmetry and model symmetry: correct equivariance , incorrect equivariance , and extrinsic equivariance .

Original: With incorrect equivariance, the model structure interferes with themodeling problem whereas with extrinsic equivariance, it may not.
Revised: Correct equivariance means the model correctly models the problem symmetry; incorrect equivariance is when the model symmetry interferes with the problem symmetry; and extrinsic equivariance is when the model symmetry transforms the input data to out-of-distribution data.

Original: We theoretically demonstrate the upper bound performance for an incorrectly constrained equivariant model.
Revised: We theoretically demonstrate the upper bound performance for an incorrectly constrained equivariant model.

Original: Second, we empirically compare extrinsic and incorrect equivariance in a supervised learning task and show that a model with extrinsic equivariance can improve performance compared with an unconstrained model.
Revised: Second, we empirically compare extrinsic and incorrect equivariance in a supervised learning task and show that a model with extrinsic equivariance can improve performance compared with an unconstrained model.

Original: Finally, we explore this idea in a reinforcement learning context and show that an extrinsically constrained model can outperform state-of-the-art conventional CNN baselines.
Revised: Finally, we explore this idea in a reinforcement learning context and show that an extrinsically constrained model can outperform state-of-the-art conventional CNN baselines.

Original:  R ELATED W ORK
Revised:  R ELATED W ORK

Original: Equivariant Neural Networks.
Revised: Equivariant Neural Networks.

Original: Equivariant networks are first introduced as G-Convolution (Cohen & Welling, 2016) and Steerable CNN (Cohen & Welling, 2017; Weiler & Cesa, 2019; Cesa et al., 2021).
Revised: Equivariant networks are first introduced as G-Convolution (Cohen & Welling, 2016) and Steerable CNN (Cohen & Welling, 2017; Weiler & Cesa, 2019; Cesa et al., 2021).

Original: Equivariant learning has been applied to various types of data including images (Weiler &
Revised: Equivariant learning has been applied to various types of data including images (Weiler &

Original: Cesa, 2019), spherical data (Cohen et al., 2018), point clouds (Dym & Maron, 2020), sets Maron et al.
Revised: Cesa, 2019), spherical data (Cohen et al., 2018), point clouds (Dym & Maron, 2020), sets Maron et al.

Original: (2020), and meshes (De Haan et al., 2020), and has shown great success in tasks including molecular dynamics (Anderson et al., 2019), particle physics (Bogatskiy et al., 2020), fluid dynamics (Wang et al., 2020), trajectory prediction (Walters et al., 2020), robotics (Simeonov et al., 2022;
Revised: (2020), and meshes (De Haan et al., 2020), and has shown great success in tasks including molecular dynamics (Anderson et al., 2019), particle physics (Bogatskiy et al., 2020), fluid dynamics (Wang et al., 2020), trajectory prediction (Walters et al., 2020), robotics (Simeonov et al., 2022;

Original: Zhu et al., 2022; Huang et al., 2022) and reinforcement learning (Wang et al., 2021; 2022c).
Revised: Zhu et al., 2022; Huang et al., 2022) and reinforcement learning (Wang et al., 2021; 2022c).

Original: Compared with the prior works that assume the domain symmetry is perfectly known, this work studies the effectiveness of equivariant networks in domains with latent symmetries.
Revised: Compared with the prior works that assume the domain symmetry is perfectly known, this work studies the effectiveness of equivariant networks in domains with latent symmetries.

Original: Symmetric Representation Learning.
Revised: Symmetric Representation Learning.

Original: Since latent symmetry is not expressable as a simple transformation of the input, equivariant networks can not be used in the standard way.
Revised: Since latent symmetry is not expressable as a simple transformation of the input, equivariant networks can not be used in the standard way.

Original: Thus several works have turned to learning equivariant features which can be easily transformed.
Revised: Thus several works have turned to learning equivariant features which can be easily transformed

Original: Parket al.
Revised: Park

Original: (2022) learn an encoder which maps inputs to equivariant features which can be used by downstream equivariant layers.
Revised: ) learn an encoder which maps inputs to equivariant features which can be used by downstream equivariant layers.

Original: Quessard et al.
Revised: Quessard et al.

Original: (2020), Klee et al.
Revised: (2020), Klee et al.

Original: (2022), and Marchetti et al.
Revised: (2022), and Marchetti et al.

Original: (2022) map 2D image inputs to elements of various groups including SO(3) , allowing for disentanglement and equivariance constraints.
Revised: (2022) map 2D image inputs to elements of various groups including SO(3) , allowing for disentanglement and equivariance constraints.

Original: Falorsi et al.
Revised: Falorsi et al.

Original: (2018) use a homeomorphic VAE to perform the same task in an unsupervised manner.
Revised: (2018) use a homeomorphic VAE to perform the same task in an unsupervised manner.

Original: Dangovski et al.
Revised: Dangovski et al.

Original: (2021) consider equivariant representations learned in a self-supervised manner using losses to encourage sensitivity or insensitivity to various symmetries.
Revised: (2021) consider equivariant representations learned in a self-supervised manner using losses to encourage sensitivity or insensitivity to various symmetries.

Revised: Our method may be considered as an example of symmetric representation learning which, unlike any of the above methods, uses an equivariant neural network as an encoder.

Original: Zhou et al.
Revised: Zhou et al.

Original: (2020) and Dehmamy et al.
Revised: and Dehmamy et al.

Original: (2021) assume no prior knowledge of the structure of symmetry in the domain and learn the symmetry transformations on inputs and latent features end-to-end with the task function.
Revised: (2021) assume no prior knowledge of the structure of symmetry in the domain and learn the symmetry transformations on inputs and latent features end-to-end with the task function.

Original: Our method may be considered as an example of symmetric representation learning which, unlike any of the above methods, uses an equivariant neural network as encoder.
Revised: In comparison, our work assumes that the latent symmetry is known but how it acts on the input is unknown.

Original: Sample Efficient Reinforcement Learning.
Revised: Sample Efficient Reinforcement Learning.

Original: One traditional solution for improving sample efficiency is to create additional samples using data augmentation (Krizhevsky et al., 2017).
Revised: One traditional solution for improving sample efficiency is to create additional samples using data augmentation (Krizhevsky et al., 2017).

Original: Recent works discover that simple image augmentations like random crop (Laskin et al., 2020b; Yarats et al., 2022) or random shift (Yarats et al., 2021) can improve the performance of reinforcement learning.
Revised: Recent works discover that simple image augmentations like random crop (Laskin et al., 2020b; Yarats et al., 2022) or random shift (Yarats et al., 2021) can improve the performance of reinforcement learning.

Original: Such image augmentation can be combined with contrastive learning (Oord et al., 2018) to achieve better performance (Laskin et al., 2020a; Zhan et al., 2020).
Revised: Such image augmentation can be combined with contrastive learning (Oord et al., 2018) to achieve better performance (Laskin et al., 2020a; Zhan et al., 2020).

Original: Recently, many prior works have shown that equivariant methods can achieve tremendously high sample efficiency in reinforcement learning (van der Pol et al., 2020; Mondal et al., 2020; Wang et al., 2021; 2022c), and realize on-robot reinforcement learning (Zhu et al., 2022; Wang et al., 2022a).
Revised: Recently, many prior works have shown that equivariant methods can achieve tremendously high sample efficiency in reinforcement learning (van der Pol et al., 2020; Mondal et al., 2020; Wang et al., 2021; 2022c), and realize on-robot reinforcement learning (Zhu et al., 2022; Wang et al., 2022a).

Original: However, recent equivariant reinforcement learning works are limited in fully equivariant domains.
Revised: However, recent equivariant reinforcement learning works are limited in fully equivariant domains.

Original: This paper extends the prior works by applying equivariant reinforcement learning to tasks with latent symmetries.
Revised: This paper extends the prior works by applying equivariant reinforcement learning to tasks with latent symmetries.

Original:  B ACKGROUND
Revised:  B ACKGROUND

Original: Equivariant Neural Networks.
Revised: Equivariant Neural Networks.

Original: A function is equivariant if it respects symmetries of its input and output spaces.
Revised: A function is equivariant if it respects symmetries of its input and output spaces.

Original: Specifically, a function f : X → Y is equivariant with respect to a symmetry group
Revised: Specifically, a function f : X → Y is equivariant with respect to a symmetry group

Original: G if it commutes with all transformations g ∈ G ,
Revised: G if it commutes with all transformations g ∈ G ,

Original: [Equation]
Revised: [Equation]

Original: where ρ x and ρ y are the representations of the group G that define how the group element g ∈ G acts on x ∈ X and x ∈ Y , respectively.
Revised: where ρ x and ρ y are the representations of the group G that define how the group element g ∈ G acts on x ∈ X and x ∈ Y , respectively.

Original: An equivariant function is a mathematical way of expressing that f is symmetric with respect to G : if we evaluate f for differently transformed versions of the same input, we should obtain transformed versions of the same output.
Revised: An equivariant function is a mathematical way of expressing that f is symmetric with respect to G : if we evaluate f for differently transformed versions of the same input, we should obtain transformed versions of the same output.

Original: In order to use an equivariant model, we generally require the symmetry group G and representation ρ x to be known at design time.
Revised: In order to use an equivariant model, we generally require the symmetry group G and representation ρ x to be known at design time.

Original: For example, in a convolutional model, this can be accomplished by tying the kernel weights together so as to satisfy K ( gy ) =
Revised: For example, in a convolutional model, this can be accomplished by tying the kernel weights together so as to satisfy K ( gy ) =

Original: ρ out ( g ) K ( y ) ρ in ( g ) − 1 , where ρ in and ρ out denote the representation of the group operator at the input and the output of the layer (Cohen et al., 2019).
Revised: ρ out ( g ) K ( y ) ρ in ( g ) − 1 , where ρ in and ρ out denote the representation of the group operator at the input and the output of the layer (Cohen et al., 2019).

Original: End-to-end equivariant models can be constructed by combining equivariant convolutional layers and equivariant activation functions.
Revised: End-to-end equivariant models can be constructed by combining equivariant convolutional layers and equivariant activation functions.

Original: In order to leverage symmetry in this way, it is common to transform the input so that standard group representations work correctly, e.g., to transform an image to a top-down view so that image rotations correspond to object rotations.
Revised: In order to leverage symmetry in this way, it is common to transform the input so that standard group representations work correctly, e.g., to transform an image to a top-down view so that image rotations correspond to object rotations.

Original: Equivariant SAC.
Revised: Equivariant SAC.

Original: Equivariant SAC (Wang et al., 2022c) is a variation of SAC (Haarnoja et al., 2018) that constrains the actor to an equivariant function and the critic to an invariant function with respect to a group G .
Revised: Equivariant SAC (Wang et al., 2022c) is a variation of SAC (Haarnoja et al., 2018) that constrains the actor to an equivariant function and the critic to an invariant function with respect to a group G .

Original: The policy is a network π : S → A × A σ , where A σ is the space of action standard deviations (SAC models a stochastic policy).
Revised: The policy is a network π : S → A × A σ , where A σ is the space of action standard deviations (SAC models a stochastic policy).

Original: It defines the group action on the output space of the policy network network ¯ a∈ A ×
Revised: It defines the group action on the output space of the policy network network ¯ a

Original: A σ as:g ¯ a = g
Revised: [Equation] σ as:

Original: ( a equiv , a inv , a σ ) =
Revised: [Equation]

Original: ( ρ equiv ( g ) a equiv , a inv , a σ ) , where a equiv ∈
Revised: , where a equiv ∈

Original: A equiv is the equivariant component in the action space, a inv ∈
Revised: A equiv is the equivariant component in the action space, a inv ∈

Original: A inv is the invariant component in the action space, a σ ∈
Revised: A inv is the invariant component in the action space, a σ ∈

Original: A σ , g ∈ G .
Revised: A σ , g ∈ G .

Original: The actor network π is then defined to be a mapping s
Revised: The actor network π is then defined to be a mapping s

Original: (cid:55)→ ¯ a that satisfies the following equivariance constraint: π ( gs ) =
Revised: (cid:55)→ ¯ a that satisfies the following equivariance constraint: π ( gs ) =

Original: g ( π ( s ))
Revised: g ( π ( s ))

Original: =
Revised: =

Original: g ¯ a .
Revised: g ¯ a .

Original: The critic is a Q -network q : S ×
Revised: The critic is a Q -network q : S ×

Original: A → R that satisfies an invariant constraint: q ( gs, ga ) =
Revised: A → R that satisfies an invariant constraint: q ( gs, ga ) =

Original: q ( s, a ) .
Revised: q ( s, a ) .

Original:  L EARNING S YMMETRY USING OTHER S YMMETRIES 4.M ODEL S YMMETRY VERSUS T RUE S YMMETRY 
Revised:  L EARNING S YMMETRY U SING O THER S YMMETRIES 4.

Revised: [Figure]

Original: This paper focuses on tasks where the way in which the symmetry group operates on the input space is unknown.
Revised: This paper focuses on tasks where the way in which the symmetry group operates on the input space is unknown.

Original: In this case the ground truth function f : X → Y is equivariant with respect to a group G which acts on X and Y by ρ x and ρ y respectively.
Revised: In this case the ground truth function f : X → Y is equivariant with respect to a group G which acts on X and Y by ρ x and ρ y respectively.

Original: However, the action ρ x on the input space is not known and may not be a simple or explicit map.
Revised: However, the action ρ x on the input space is not known and may not be a simple or explicit map.

Original: Since ρ x is unknown, we cannot pursue the strategy of learning f using an equivariant model class f ϕ constrained by ρ x .
Revised: Since ρ x is unknown, we cannot pursue the strategy of learning f using an equivariant model class f ϕ constrained by ρ x .

Original: As an alternative, we propose restricting to a model class f ϕ which satisfies equivariance with respect to a different group action ˆ ρ x , i.e., f ϕ
Revised: As an alternative, we propose restricting to a model class f ϕ which satisfies equivariance with respect to a different group action ˆ ρ x , i.e., f ϕ

Original: (ˆ ρ x ( g ) x ) =
Revised: (ˆ ρ x ( g ) x ) =

Original: ρ y ( g ) f ϕ ( x ) .
Revised: ρ y ( g ) f ϕ ( x ) .

Original: This paper tests the hypothesis that if the model is constrained to a symmetry class ˆ ρ x which is related to the true symmetry ρ x , then it may help learn a model satisfying the true symmetry.
Revised: This paper tests the hypothesis that if the model is constrained to a symmetry class ˆ ρ x which is related to the true symmetry ρ x , then it may help learn a model satisfying the true symmetry.

Revised: For example, if x is an image viewed from an oblique angle and ρ x is the rotation of the objects in the image, ˆ ρ x can be the rotation of the whole image (which is different from ρ x because of the tilted view angle).

Revised: Section 4. will describe this example in detail.

Original: 4. C ORRECT , I NCORRECT , AND E XTRINSIC E QUIVARIANCE
Revised: 4. C ORRECT , I NCORRECT , AND E XTRINSIC E QUIVARIANCE

Original: Our findings show that the success of this strategy depends on how ˆ ρ x relates to the ground truth function f and its symmetry.
Revised: Our findings show that the success of this strategy depends on how ˆ ρ x relates to the ground truth function f and its symmetry.

Original: We classify the model symmetry as correct equivariance , incorrect equivariance , or extrinsic equivariance with respect tothe true function f .
Revised: We classify the model symmetry as correct equivariance , incorrect equivariance , or extrinsic equivariance with respect to f .

Original: Correct symmetry means that the model symmetry correctly reflects a symmetry present in the ground truth function f .
Revised: Correct symmetry means that the model symmetry correctly reflects a symmetry present in the ground truth function f .

Original: An extrinsic symmetry may still aid learning whereas an incorrect symmetry is necessarily detrimental to learning.
Revised: An extrinsic symmetry may still aid learning whereas an incorrect symmetry is necessarily detrimental to learning.

Original: We illustrate the distinction with a classification example shown in Figure 1a.
Revised: We illustrate the distinction with a classification example shown in Figure 2a.

Revised: (See Appendix B for a more in-depth description.)

Original: Let D ⊆
Revised: Let D ⊆

Original: X be the support of the input distribution for f .
Revised: X be the support of the input distribution for f .

Original: Definition 4.1.
Revised: Definition 4.1.

Original: The action ˆ ρ x has correct equivariance with respect to f if ˆ ρ x ( g ) x ∈ D for all x ∈ D, g ∈ G and f (ˆ ρ x ( g ) x ) =
Revised: The action ˆ ρ x has correct equivariance with respect to f if ˆ ρ x ( g ) x ∈ D for all x ∈ D, g ∈ G and f (ˆ ρ x ( g ) x ) =

Original: ρ y ( g ) f
Revised: ρ y ( g ) f

Original: ( x ) .
Revised: ( x ) .

Original: That is, the model symmetry preserves the input space D and f is equivariant with respect to it.
Revised: That is, the model symmetry preserves the input space D and f is equivariant with respect to it.

Original: For example, consider the action ˆ ρ x of the group G 1 =
Revised: For example, consider the action ˆ ρ x of the group G 1 =

Original: C 2 acting on R 2 by reflection across the horizontal axis and ρ y = 1 , the trivial action fixing labels.
Revised: C 2 acting on R 2 by reflection across the horizontal axis and ρ y = 1 , the trivial action fixing labels.

Original: Figure 1b shows the untransformed data x ∈ D as circles along the unit circle.
Revised: Figure 2b shows the untransformed data x ∈ D as circles along the unit circle.

Original: The transformed data ˆ ρ x ( g ) x (shown as crosses) also lie on the unit circle, and hence the support D is reflection invariant.
Revised: The transformed data ˆ ρ x ( g ) x (shown as crosses) also lie on the unit circle, and hence the support D is reflection invariant.

Original: Moreover, the ground truth labels f
Revised: Moreover, the ground truth labels f

Original: ( x ) (shown as orange or blue) are preserved by this action.
Revised: ( x ) (shown as orange or blue) are preserved by this action.

Original: Definition 4.2.
Revised: Definition 4.2.

Original: The action ˆ ρ x has incorrect equivariance with respect to f if there exist x ∈ D and g ∈ G such that ˆ ρ x ( g ) x ∈ D but f (ˆ ρ x ( g ) x ) ̸
Revised: The action ˆ ρ x has incorrect equivariance with respect to f if there exist x ∈ D and g ∈ G such that ˆ ρ x ( g ) x ∈ D but f (ˆ ρ x ( g ) x ) ̸

Original: =
Revised: =

Original: ρ y ( g ) f ( x ) .
Revised: ρ y ( g ) f ( x ) .

Original: In this case, the model symmetry partially preserves the input distribution, but does not correctly preserve labels.
Revised: In this case, the model symmetry partially preserves the input distribution, but does not correctly preserve labels.

Original: In Figure 1c, the rotation group G 2 = ⟨ Rot π ⟩ maps the unit circle to itself, but the transformed data does not have the correct label.
Revised: In Figure 2c, the rotation group G 2 = ⟨ Rot π ⟩ maps the unit circle to itself, but the transformed data does not have the correct label.

Original: Thus, constraining the model f ϕ by f ϕ (ˆ ρ x ( g ) x ) =
Revised: Thus, constraining the model f ϕ by f ϕ (ˆ ρ x ( g ) x ) =

Original: f ϕ ( x ) will force f ϕ to mislabel data.
Revised: f ϕ ( x ) will force f ϕ to mislabel data.

Original: In this example, for a = √ 2 / 2 , f ( a, a ) =
Revised: In this example, for a = √ 2 / 2 , f ( a, a ) =

Original: ORANGE and f ( − a, − a )
Revised: ORANGE and f ( − a, − a )

Original: =
Revised: =

Original: BLUE , however, f ϕ ( a, a ) =
Revised: BLUE , however, f ϕ ( a, a ) =

Original: f ϕ (Rot π ( a, a )) =
Revised: f ϕ (Rot π ( a, a )) =

Original: f ϕ ( − a, − a ) .
Revised: f ϕ ( − a, − a ) .

Original: Definition 4.3.
Revised: Definition 4.3.

Original: The action ˆ ρ x has extrinsic equivariance with respect to f if for x ∈ D , ˆ ρ x ( g ) x
Revised: The action ˆ ρ x has extrinsic equivariance with respect to f if for x ∈ D , ˆ ρ x ( g ) x

Original: ̸∈ D .
Revised: ̸∈ D .

Original: Extrinsic equivariance is when the equivariant constraint in the equivariant network f ϕ enforces equivariance to out-of-distribution data.
Revised: Extrinsic equivariance is when the equivariant constraint in the equivariant network f ϕ enforces equivariance to out-of-distribution data.

Original: Since ˆ ρ x ( g ) x ̸∈ D , the ground truth f
Revised: Since ˆ ρ x ( g ) x ̸∈ D , the ground truth f

Original: (ˆ ρ x ( g ) x ) is undefined.
Revised: (ˆ ρ x ( g ) x ) is undefined.

Original: An example of extrinsic equivariance is given by the scaling group G 3 shown in Figure 1d.
Revised: An example of extrinsic equivariance is given by the scaling group G 3 shown in Figure 2d.

Original: For the data x ∈ D , enforcing scaling invariance f ϕ (ˆ ρ x ( g ) x ) = f ϕ ( x ) where g ∈ G 3 will not increase error, because the group transformed data (in crosses) are out of the distribution D of the input data shown in the grey ring.
Revised: For the data x ∈ D , enforcing scaling invariance f ϕ (ˆ ρ x ( g ) x ) = f ϕ ( x ) where g ∈ G 3 will not increase error, because the group transformed data (in crosses) are out of the distribution D of the input data shown in the grey ring.

Original: In fact, we hypothesize that such extrinsic equivariance may even be helpful for the network to learn the ground truth function.
Revised: In fact, we hypothesize that such extrinsic equivariance may even be helpful for the network to learn the ground truth function.

Original: For example, in Figure 1d, the network can learn to classify all points on the left as blue and all points on the right as orange.
Revised: For example, in Figure 2d, the network can learn to classify all points on the left as blue and all points on the right as orange.

Original:  T HEORETICAL U PPER B OUND ON A CCURACY FOR  E QUIVARIANT M ODELSWITH I NCORRECT S YMMETRY
Revised: 4. T HEORETICAL U PPER B OUND ON A CCURACY FOR I NCORRECT E QUIVARIANT M ODELS

Original: Consider a classification problem over the set X with finitely many classes Y .
Revised: Consider a classification problem over the set X with finitely many classes Y .

Original: Let G be a finite group acting on X .
Revised: Let G be a finite group acting on X .

Original: Consider a model f ϕ : X → Y with incorrect equivariance constrained to be invariant to G .
Revised: Consider a model f ϕ : X → Y with incorrect equivariance constrained to be invariant to G .

Original: In this case the points in a single orbit { gx : g ∈ G } must all be assigned the same label f ϕ ( gx ) =
Revised: In this case the points in a single orbit { gx : g ∈ G } must all be assigned the same label f ϕ ( gx ) =

Original: y .
Revised: y .

Original: However these points may have different ground truth labels.
Revised: However these points may have different ground truth labels.

Original: We classify how bad this situation is by measuring p ( x ) , the proportion of ground truth labels in the orbit of x which are equal to the majority label.
Revised: We classify how bad this situation is by measuring p ( x ) , the proportion of ground truth labels in the orbit of x which are equal to the majority label.

Original: Let c p be the fraction of points x ∈ X which have consensus proportion p ( x ) = p .
Revised: Let c p be the fraction of points x ∈ X which have consensus proportion p ( x ) = p .

Original: Proposition 4.1.
Revised: Proposition 4.1.

Original: The accuracy of f ϕ has upper bound acc( f ϕ ) ≤ (cid:80)
Revised: The accuracy of f ϕ has upper bound acc( f ϕ ) ≤ (cid:80)

Original: p c p p
Revised: p c p p

Original: See the complete version of the proposition and its proof in Appendix A.
Revised: See the complete version of the proposition and its proof in Appendix A.

Original: In the example in Figure 1c, we have p ∈ { 0 . 5 } and c 0 .
Revised: In the example in Figure 2c, we have p ∈ { 0 . 5 } and c 0 .

Original: 5 = 1 , thus acc( f ϕ ) ≤ 0 .
Revised: 5 = 1 , thus acc( f ϕ ) ≤ 0 .

Original: In contrast, we can choose an unconstrained model from a model class with a universal approximation property and given properly chosen hyperparameters find a model with arbitrarily good accuracy.
Revised: In contrast, we can choose an unconstrained model from a model class with a universal approximation property and given properly chosen hyperparameters find a model with arbitrarily good accuracy.

Original: 4. O BJECT T RANSFORMATION AND I MAGE T RANSFORMATION
Revised: 4. O BJECT T RANSFORMATION AND I MAGE T RANSFORMATION

Original: [Figure] In tasks with visual inputs ( X = R c × h × w ), incorrect or extrinsic equivariance will exist when the transformation of the image does not match the transformation of the latent state of the task.
Revised: In tasks with visual inputs ( X = R c × h × w ), incorrect or extrinsic equivariance will exist when the transformation of the image does not match the transformation of the latent state of the task.

Original: In such case, we call ρ x the object transform and ˆ ρ x the image transform .
Revised: In such case, we call ρ x the object transform and ˆ ρ x the image transform .

Original: For an image input x ∈ X , the image transform ˆ ρ x ( g ) x is defined as a simple transformation of pixel locations (e.g., Figure 2a-c where g = π/ 2 ∈ SO(2) ), while the object transform ρ x ( g ) x is an implicit map transforming the objects in the image (e.g., Figure 2a-b where g = π/ 2 ∈ SO(2) ).
Revised: For an image input x ∈ X , the image transform ˆ ρ x ( g ) x is defined as a simple transformation of pixel locations (e.g., Figure 1a-c where g = π/ 2 ∈ SO(2) ), while the object transform ρ x ( g ) x is an implicit map transforming the objects in the image (e.g., Figure 1a-b where g = π/ 2 ∈ SO(2) ).

Original: The distinction between object transform and image transform is often caused by some symmetry-breaking factors such as camera angle, occlusion, backgrounds, and so on (e.g., Figure 2).
Revised: The distinction between object transform and image transform is often caused by some symmetry-breaking factors such as camera angle, occlusion, backgrounds, and so on (e.g., Figure 1).

Original: We refer to such symmetry-breaking factors as symmetry corruptions .
Revised: We refer to such symmetry-breaking factors as symmetry corruptions .

Revised: [Figure]

Original:  E VALUATING E QUIVARIANT N ETWORK WITH S YMMETRY C ORRUPTIONS
Revised:  E VALUATING E QUIVARIANT N ETWORK WITH S YMMETRY C ORRUPTIONS

Original: Although it is preferable to use an equivariant model to enforce correct equivariance, real-world problems often contain some symmetry corruptions, such as oblique viewing angles, which mean[Figure] the symmetry is latent.
Revised: Although it is preferable to use an equivariant model to enforce correct equivariance, real-world problems often contain some symmetry corruptions, such as oblique viewing angles, which mean the symmetry is latent.

Original: In this experiment, we evaluate the effect of different corruptions on an equivariant model and show that enforcing extrinsic equivariance can actually improve performance.
Revised: In this experiment, we evaluate the effect of different corruptions on an equivariant model and show that enforcing extrinsic equivariance can actually improve performance.

Original: We experiment with a simple supervised learning task where the scene contains three ducks of different colors.
Revised: We experiment with a simple supervised learning task where the scene contains three ducks of different colors.

Original: The data samples are pairs of images where all ducks in the first image are rotated by some g ∈ C 8 to produce the second image within each pair.
Revised: The data samples are pairs of images where all ducks in the first image are rotated by some g ∈ C 8 to produce the second image within each pair.

Original: Given the correct g , the goal is to train a network f ϕ : R 2 × 4 × h × w → R | C 8 | to classify the rotation (Figure 3a).
Revised: Given the correct g , the goal is to train a network f ϕ : R 2 × 4 × h × w → R | C 8 | to classify the rotation (Figure 3a).

Original: If we have a perfect top-down image observation, then the object transform and image transform are equal, and we can enforce the correct equivariance by modeling the ground truth function f as an invariant network f ϕ ( ρ x ( g ) x )
Revised: If we have a perfect top-down image observation, then the object transform and image transform are equal, and we can enforce the correct equivariance by modeling the ground truth function f as an invariant network f ϕ ( ρ x ( g ) x )

Original: =
Revised: =

Original: f ϕ ( x ) where g ∈ SO(2) (because the rotation of the two images will not change the relative rotation between the objects in the two images).
Revised: f ϕ ( x ) where g ∈ SO(2) (because the rotation of the two images will not change the relative rotation between the objects in the two images).

Original: To mimic symmetry corruptions in real-world applications, we apply seven different transformations to both pairs of images shown in Figure 3b (more corruptions are considered in Appendix D.1).
Revised: To mimic symmetry corruptions in real-world applications, we apply seven different transformations to both pairs of images shown in Figure 3b (more corruptions are considered in Appendix E.1).

Original: In particular, for invert-label, the ground truth label g is inverted to − g when the yellow duck is on the left of the orange duck in the world frame in the first input image.
Revised: In particular, for invert-label, the ground truth label g is inverted to − g when the yellow duck is on the left of the orange duck in the world frame in the first input image.

Original: Notice that enforcing SO(2) -invariance in f ϕ under invert-label is an incorrect equivariant constraint because a rotation on the ducks might change their relative position in the world frame and break the invariance of the task: f ( gx ) ̸
Revised: Notice that enforcing SO(2) -invariance in f ϕ under invert-label is an incorrect equivariant constraint because a rotation on the ducks might change their relative position in the world frame and break the invariance of the task: f ( gx ) ̸

Original: = f ( x ) , ∃ g ∈ SO(2) .
Revised: = f ( x ) , ∃ g ∈ SO(2) .

Original: However, in all other corruptions, enforcing SO(2) -invariance is an extrinsic equivariance because gx will be out of the input distribution.
Revised: However, in all other corruptions, enforcing SO(2) -invariance is an extrinsic equivariance because gx will be out of the input distribution.

Original: We evaluate the equivariant network defined in group Cimplemented using e2cnn (Weiler & Cesa, 2019).
Revised: We evaluate the equivariant network defined in group Cimplemented using e2cnn (Weiler & Cesa, 2019).

Original: See Appendix C.1 for the training details.
Revised: See Appendix D.1 for the training details.

Original: Comparing Equivariant Networks with CNNs.
Revised: Comparing Equivariant Networks with CNNs.

Original: We first compare the performance of an equivariant network (Equi) and a conventional CNN model (CNN) with a similar number of trainable parameters.
Revised: We first compare the performance of an equivariant network (Equi) and a conventional CNN model (CNN) with a similar number of trainable parameters.

Original: The network architectures are relatively simple (see Appendix B.1) as our goal is to evaluate the performance difference between an equivariant network and an unconstrained CNN model rather than achieving the best performance in this task.
Revised: The network architectures are relatively simple (see Appendix C.1) as our goal is to evaluate the performance difference between an equivariant network and an unconstrained CNN model rather than achieving the best performance in this task.

Original: In both models, we apply a random crop after sampling each data batch to improve the sample efficiency.
Revised: In both models, we apply a random crop after sampling each data batch to improve the sample efficiency.

Original: See Appendix D.1 for the effects of random crop augmentation on learning.
Revised: See Appendix E.1 for the effects of random crop augmentation on learning.

Original: Figure 4 (blue vs green) shows the test accuracy of both models after convergence when trained with varying dataset sizes.
Revised: Figure 4 (blue vs green) shows the test accuracy of both models after convergence when trained with varying dataset sizes.

Original: For all corruptions with extrinsic equivariance constraints, the equivariant network performs better than the CNN model, especially in low data regimes.
Revised: For all corruptions with extrinsic equivariance constraints, the equivariant network performs better than the CNN model, especially in low data regimes.

Original: However, for invert-label which gives an incorrect equivariance constraint, the CNN outperforms the equivariant model, demonstrating that enforcing incorrect equivariance negatively impacts accuracy.
Revised: However, for invert-label which gives an incorrect equivariance constraint, the CNN outperforms the equivariant model, demonstrating that enforcing incorrect equivariance negatively impacts accuracy.

Original: In fact, based on Proposition 4.1, the equivariant network here has a theoretical upper bound performance of 62 .
Revised: In fact, based on Proposition 4.1, the equivariant network here has a theoretical upper bound performance of 62 .

Original: First, p ∈ { 1 , 0 . 5 } .
Revised: First, p ∈ { 1 , 0 . 5 } .

Original: Then p = 1 when f ( x ) ∈ { 0 , π } ⊆
Revised: Then p = 1 when f ( x ) ∈ { 0 , π } ⊆

Original: Cwhere f
Revised: Cwhere f

Original: ( x ) = − f ( x ) (i.e., negating the label won’t change it), and c 1 = 2 / 8 = 0 .
Revised: ( x ) = − f ( x ) (i.e., negating the label won’t change it), and c 1 = 2 / 8 = 0 .

Original: The consensus proportion p = 0 .
Revised: The consensus proportion p = 0 .

Original: 5 when f ( x ) ∈ { π/ 4 , π/ 2 , 3 π/ 4 , 5 π/ 4 , 3 π/ 2 , 7 π/ 4 } ⊆
Revised: 5 when f ( x ) ∈ { π/ 4 , π/ 2 , 3 π/ 4 , 5 π/ 4 , 3 π/ 2 , 7 π/ 4 } ⊆

Original: C 8 , where half of the labels in the orbit ofx will be the negationof the labels of theother half (because half of g ∈ C acc( f ϕ ) ≤ 1 × 0 .
Revised: C 8 , where half of the labels in the orbit of will be the negation labels of the (because half of g ∈ Cwill change the relative position between the ), thus c 5 = 6 / 8 = acc( f ϕ ) ≤ 1 × 0 .

Original: 25 + 0 .
Revised: 25 + 0 .

Original: 5 × 0 .
Revised: 5 × 0 .

Original: 75 = 0 .
Revised: 75 = 0 .

Original: This theoretical upper bound matches the result in
Revised: This theoretical upper bound matches the result in

Original: Figure 4.
Revised: Figure 4.

Original: Figure 4 suggests that even in the presence of symmetry corruptions, enforcing extrinsic equivariance can improve the sample efficiencycompared with an unconstrained model while incorrect equivariance is detrimental.
Revised: Figure 4 suggests that even in the presence of symmetry corruptions, enforcing extrinsic equivariance can improve the sample efficiency while incorrect equivariance is detrimental.

Original: Extrinsic Image Augmentation Helps in Learning Correct Symmetry.
Revised: Extrinsic Image Augmentation Helps in Learning Correct Symmetry.

Original: In these experiments, we further illustrate that enforcing extrinsic equivariance helps the model learn the latent equivariance of the task for in-distribution data.
Revised: In these experiments, we further illustrate that enforcing extrinsic equivariance helps the model learn the latent equivariance of the task for in-distribution data.

Original: As an alternative to equivariant networks, we consider an older alternative for symmetry learning, data augmentation, to see whether extrinsic symmetry augmentations can improve the performance of an unconstrained CNN by helping it learn latent symmetry.
Revised: As an alternative to equivariant networks, we consider an older alternative for symmetry learning, data augmentation, to see whether extrinsic symmetry augmentations can improve the performance of an unconstrained CNN by helping it learn latent symmetry.

Original: Specifically, we augment each training sample with C 8 image rotations while keeping the validation and test set unchanged.
Revised: Specifically, we augment each training sample with C 8 image rotations while keeping the validation and test set unchanged.

Original: As is shown in Figure 4, adding such extrinsic data augmentation (CNN + Img Trans, red) significantly improves the performance of CNN (green), and nearly matches the performance of the equivariant network (blue).
Revised: As is shown in Figure 4, adding such extrinsic data augmentation (CNN + Img Trans, red) significantly improves the performance of CNN (green), and nearly matches the performance of the equivariant network (blue).

Original: Notice that in invert-label, adding such augmentation hurts the performance of CNN because of incorrect equivariance.
Revised: Notice that in invert-label, adding such augmentation hurts the performance of CNN because of incorrect equivariance.

Original:  E XTRINSIC E QUIVARIANCE IN R EINFORCEMENT L EARNING
Revised:  E XTRINSIC E QUIVARIANCE IN R EINFORCEMENT L EARNING

Original: The results in Section 5 suggest that enforcing extrinsic equivariance can help the model better learn the latent symmetry in the task.
Revised: The results in Section 5 suggest that enforcing extrinsic equivariance can help the model better learn the latent symmetry in the task.

Original: In this section, we apply this methodology in reinforcement learning and demonstrate that extrinsic equivariance can significantly improve sample efficiency.
Revised: In this section, we apply this methodology in reinforcement learning and demonstrate that extrinsic equivariance can significantly improve sample efficiency.

Original: 6. R EINFORCEMENT L EARNING IN R OBOTIC M ANIPULATION
Revised: 6. R EINFORCEMENT L EARNING IN R OBOTIC M ANIPULATION

Original: We first experiment in five robotic manipulation environments shown in Figure 6.
Revised: We first experiment in five robotic manipulation environments shown in Figure 6.

Original: The state space S =
Revised: The state space S =

Original: R 4 × h × w is a 4-channel RGBD image captured from a fixed camera pointed at the workspace (Figure 5).
Revised: R 4 × h × w is a 4-channel RGBD image captured from a fixed camera pointed at the workspace (Figure 5).

Original: The action space
Revised: The action space

Original: A = R 5 is the change in gripper pose ( x, y, z, θ ) , where θ is the rotation along the z -axis, and the gripper open width λ .
Revised: A = R 5 is the change in gripper pose ( x, y, z, θ ) , where θ is the rotation along the z -axis, and the gripper open width λ .

Original: The task has latent O(2) symmetry: when a rotation or reflection is applied to the poses of the gripper and the objects, the action should rotate and reflect accordingly.
Revised: The task has latent O(2) symmetry: when a rotation or reflection is applied to the poses of the gripper and the objects, the action should rotate and reflect accordingly.

Original: However, such symmetry does not exist in image space because the image perspective is skewed instead of top-down (we also perform experiments with another symmetry corruption caused by sensor occlusion in
Revised: However, such symmetry does not exist in image space because the image perspective is skewed instead of top-down (we also perform experiments with another symmetry corruption caused by sensor occlusion in

Original: [Figure]
Revised: [Figure]

Original: Appendix D.3).
Revised: Appendix E.3).

Original: We enforce such extrinsic symmetry (group D 4 ) using Equivariant SAC (Wang et al., 2022c;a) equipped with random crop augmentation using RAD (Laskin et al., 2020b)
Revised: We enforce such extrinsic symmetry (group D 4 ) using Equivariant SAC (Wang et al., 2022c;a) equipped with random crop augmentation using RAD (Laskin et al., 2020b)

Original: (Equi
Revised: (Equi

Original: SAC + RAD) and compare it with the following baselines: 1) CNN SAC + RAD: same as our method but with an unconstrained CNN instead of an equivariant model; 2) CNN SAC + DrQ: same as 1), but with DrQ (Yarats et al., 2021) for the random crop augmentation; 3) FERM (Zhan et al., 2020): a combination of 1) and contrastive learning; and 4) SEN + RAD:
Revised: SAC + RAD) and compare it with the following baselines: 1) CNN SAC + RAD: same as our method but with an unconstrained CNN instead of an equivariant model; 2) CNN SAC + DrQ: same as 1), but with DrQ (Yarats et al., 2021) for the random crop augmentation; 3) FERM (Zhan et al., 2020): a combination of 1) and contrastive learning; and 4) SEN + RAD:

Original: Symmetric Embedding
Revised: Symmetric Embedding

Original: Network (Park et al., 2022) that uses a conventional network for the encoder and an equivariant network for the output head.
Revised: Network (Park et al., 2022) that uses a conventional network for the encoder and an equivariant network for the output head.

Original: All baselines are implemented such that they have a similar number of parameters as Equivariant SAC.
Revised: All baselines are implemented such that they have a similar number of parameters as Equivariant SAC.

Original: See Appendix B.2 for the network architectures and Appendix E for the architecture hyperparameter search for the baselines.
Revised: See Appendix C.2 for the network architectures and Appendix F for the architecture hyperparameter search for the baselines.

Original: All methods use Prioritized Experience
Revised: All methods use Prioritized Experience

Original: Replay (PER) (Schaul et al., 2015) with pre-loaded expert demonstrations (20 episodes for Block
Revised: Replay (PER) (Schaul et al., 2015) with pre-loaded expert demonstrations (20 episodes for Block

Original: Pulling and Block Pushing, 50 for Block Picking and Drawer Opening, and 100 for Block in Bowl).
Revised: Pulling and Block Pushing, 50 for Block Picking and Drawer Opening, and 100 for Block in Bowl).

Original: We also add an L2 loss towards the expert action in the actor to encourage expert actions.
Revised: We also add an L2 loss towards the expert action in the actor to encourage expert actions.

Original: More details about training are provided in Appendix C.2.
Revised: More details about training are provided in Appendix D.2.

Original: Figure 7 shows that Equivariant SAC (blue) outperforms all baselines.
Revised: Figure 7 shows that Equivariant SAC (blue) outperforms all baselines.

Original: Note that the performance of
Revised: Note that the performance of

Original: Equivariant SAC in Figure 7 does not match that reported in Wang et al.
Revised: Equivariant SAC in Figure 7 does not match that reported in Wang et al.

Original: (2022c) because we have a harder task setting: we do not have a top-down observation centered at the gripper position as in the prior work.
Revised: (2022c) because we have a harder task setting: we do not have a top-down observation centered at the gripper position as in the prior work.

Original: Such top-down observations would not only provide correct equivariance but also help learn a translation-invariant policy.
Revised: Such top-down observations would not only provide correct equivariance but also help learn a translation-invariant policy.

Original: Even in the harder task setting without top-down observations,
Revised: Even in the harder task setting without top-down observations,

Original: Figure 7 suggests that Equivariant SAC can still achieve higher performance compared to baselines.
Revised: Figure 7 suggests that Equivariant SAC can still achieve higher performance compared to baselines.

Original: 6. I NCREASING C ORRUPTION L EVELS
Revised: 6. I NCREASING C ORRUPTION L EVELS

Original: In this experiment, we vary the camera angle by tilting to see how increasing the gap between the image transform and the object transform affects the performance of extrinsically equivariant networks.
Revised: In this experiment, we vary the camera angle by tilting to see how increasing the gap between the image transform and the object transform affects the performance of extrinsically equivariant networks.

Original: When the view angle is at 90 degrees (i.e., the image is top-down), the object and image transformation exactly match.
Revised: When the view angle is at 90 degrees (i.e., the image is top-down), the object and image transformation exactly match.

Original: As the view angle is decreased, the gap increases.
Revised: As the view angle is decreased, the gap increases.

Original: Figure 8 shows the observation at 90 and 15 degree view angles.
Revised: Figure 8 shows the observation at 90 and 15 degree view angles.

Original: We remove the robot arm except for the gripper and the blue/white grid on the ground to remove the other symmetry-breaking components in the environment so that the camera angle is the only symmetry corruption.
Revised: We remove the robot arm except for the gripper and the blue/white grid on the ground to remove the other symmetry-breaking components in the environment so that the camera angle is the only symmetry corruption.

Original: We compare Equi SAC + RAD against CNN SAC + RAD.
Revised: We compare Equi SAC + RAD against CNN SAC + RAD.

Original: We evaluate the performance of each method at the end of training for different view angles in Figure 9.
Revised: We evaluate the performance of each method at the end of training for different view angles in Figure 9.

Original: As expected, the performance of Equivariant SAC decreases as the camera angle is decreased, especially from 30 degrees to 15 degrees.
Revised: As expected, the performance of Equivariant SAC decreases as the camera angle is decreased, especially from 30 degrees to 15 degrees.

Original: On the other hand, CNN generally has similar performance for all view angles, with the exception of Block Pulling and Block Pushing, where decreasing the view angle leads to higher performance.
Revised: On the other hand, CNN generally has similar performance for all view angles, with the exception of Block Pulling and Block Pushing, where decreasing the view angle leads to higher performance.

Original: This may be because decreasing the view angle helps the network to better understand the height of the gripper, which is useful for pulling and pushing actions.
Revised: This may be because decreasing the view angle helps the network to better understand the height of the gripper, which is useful for pulling and pushing actions.

Original: 6. E XAMPLE OF I NCORRECT E QUIVARIANCE
Revised: 6. E XAMPLE OF I NCORRECT E QUIVARIANCE

Original: We demonstrate an example where incorrect equivariance can harm the performance of
Revised: We demonstrate an example where incorrect equivariance can harm the performance of

Original: Equivariant SAC compared to an unconstrained model.
Revised: Equivariant SAC compared to an unconstrained model.

Original: We modify the environments so that the image state will be reflected across the vertical axis with 50% probability and then also reflected across the horizontal axis with 50% probability (see Figure 10).
Revised: We modify the environments so that the image state will be reflected across the vertical axis with 50% probability and then also reflected across the horizontal axis with 50% probability (see Figure 10).

Original: As these random reflections are contained in D 4 , the transformed state reflect ( s ) , s ∈ S is affected by Equivariant SAC’s symmetry constraint.
Revised: As these random reflections are contained in D 4 , the transformed state reflect ( s ) , s ∈ S is affected by Equivariant SAC’s symmetry constraint.

Original: In particular, as the actor produces a transformed action for reflect when the optimal action should actually be invariant, the extrinsic equivariance constraint now becomes an incorrect equivariance for these reflected states.
Revised: In particular, as the actor produces a transformed action for reflect when the optimal action should actually be invariant, the extrinsic equivariance constraint now becomes an incorrect equivariance for these reflected states.

Original: As shown in Figure 11, Equivariant SAC can barely learn under random reflections, while CNN can still learn a useful policy.
Revised: As shown in Figure 11, Equivariant SAC can barely learn under random reflections, while CNN can still learn a useful policy.

Original: 6. R EINFORCEMENT L EARNING IN D EEP M IND C ONTROL S UITE
Revised: 6. R EINFORCEMENT L EARNING IN D EEP M IND C ONTROL S UITE

Original: We further apply extrinsically equivariant networks to continuous control tasks in the DeepMind
Revised: We further apply extrinsically equivariant networks to continuous control tasks in the DeepMind

Original: Control Suite (DMC) (Tunyasuvunakool et al., 2020).
Revised: Control Suite (DMC) (Tunyasuvunakool et al., 2020).

Original: We use a subset of the domains in DMC that have clear object-level symmetry and use the D 1 group for cartpole, cup catch, pendulum, acrobot domains, and D 2 for reacher domains.
Revised: We use a subset of the domains in DMC that have clear object-level symmetry and use the D 1 group for cartpole, cup catch, pendulum, acrobot domains, and D 2 for reacher domains.

Original: This leads to a total of 7 tasks, with 4 easy and 3 medium level tasks as defined in (Yarats et al., 2022).
Revised: This leads to a total of 7 tasks, with 4 easy and 3 medium level tasks as defined in (Yarats et al., 2022).

Original: Note that all of these domains are not fully equivariant as they include a checkered grid for the floor and random stars as the background.
Revised: Note that all of these domains are not fully equivariant as they include a checkered grid for the floor and random stars as the background.

Original: We use DrQv2 Yarats et al.
Revised: We use DrQv2 Yarats et al.

Original: (2022), a SOTA model-free RL algorithm for image-based control, as our base RL algorithm.
Revised: (2022), a SOTA model-free RL algorithm for image-based control, as our base RL algorithm.

Original: We create an equivariant version of DrQv2, with an equivariant actor and invariant critic with respect to the environment’s symmetry group.
Revised: We create an equivariant version of DrQv2, with an equivariant actor and invariant critic with respect to the environment’s symmetry group.

Original: We follow closely the architecture and training hyperparameters used in the original paper except in the image encoder, where two max-pooling layers are added to further reduce the representation dimension for faster training.
Revised: We follow closely the architecture and training hyperparameters used in the original paper except in the image encoder, where two max-pooling layers are added to further reduce the representation dimension for faster training.

Original: Furthermore, DrQv2 uses convolution layers in the image encoder and then flattens its output to feed it into linear layers in the actor and the critic.
Revised: Furthermore, DrQv2 uses convolution layers in the image encoder and then flattens its output to feed it into linear layers in the actor and the critic.

Original: In order to preserve this design choice for the equivariant model, we do not reduce the spatial dimensions to 1 × 1 by downsampling/pooling or stride as commonly done in practice.
Revised: In order to preserve this design choice for the equivariant model, we do not reduce the spatial dimensions to 1 × 1 by downsampling/pooling or stride as commonly done in practice.

Original: Rather we flatten the image using a process we term action restriction since the symmetry group is restricted from Z 2 ⋉ D k to D k .
Revised: Rather we flatten the image using a process we term action restriction since the symmetry group is restricted from Z 2 ⋉ D k to D k .

Original: Let I ∈
Revised: Let I ∈

Original: R h × w × c denote the image feature where D k acts on both the spatial domain and channels.
Revised: R h × w × c denote the image feature where D k acts on both the spatial domain and channels.

Original: Then we add a new axis corresponding to D k by ˜ I
Revised: Then we add a new axis corresponding to D k by ˜ I

Original: = ( gI )
Revised: = ( gI )

Original: g ∈ D k ∈ R
Revised: g ∈ D k ∈ R

Original: h × w ×
Revised: h × w ×

Original: c
Revised: c

Original: × 2 k .
Revised: × 2 k .

Original: We then flatten to ¯ I
Revised: We then flatten to ¯ I

Original: =
Revised: =

Original: ( gI ) g ∈ D k ∈
Revised: ( gI ) g ∈ D k ∈

Original: R 1 × 1 × hwc × 2 k .
Revised: R 1 × 1 × hwc × 2 k .

Original: The intermediate step ˜
Revised: The intermediate step ˜

Original: I is necessary to encode both the spatial and channel actions into a single axis which ensures the action restriction is D k -equivariant.
Revised: I is necessary to encode both the spatial and channel actions into a single axis which ensures the action restriction is D k -equivariant.

Original: We now map back down to the original dimension with a D k -equivariant 1 × 1 convolution.
Revised: We now map back down to the original dimension with a D k -equivariant 1 × 1 convolution.

Original: To the best of our knowledge, this is the first equivariant version of DrQv2.
Revised: To the best of our knowledge, this is the first equivariant version of DrQv2.

Original: We compare the equivariant vs. the non-equivariant (original) DrQv2 algorithm to evaluate whether extrinsic equivariance can still improve training in the original domains (with symmetry corruptions).
Revised: We compare the equivariant vs. the non-equivariant (original) DrQv2 algorithm to evaluate whether extrinsic equivariance can still improve training in the original domains (with symmetry corruptions).

Original: In figures 12, equivariant DrQv2 consistently learns faster than the non-equivariant version on all tasks, where the performance improvement is largest on the more difficult medium tasks.
Revised: In figures 12, equivariant DrQv2 consistently learns faster than the non-equivariant version on all tasks, where the performance improvement is largest on the more difficult medium tasks.

Original: In pendulum swingup, both methods have 1 failed run each, leading to a large standard error, see Figure 26 in Appendix D.4 for a plot of all runs.
Revised: In pendulum swingup, both methods have 1 failed run each, leading to a large standard error, see Figure 27 in Appendix E.4 for a plot of all runs.

Original: These results highlight that even with some symmetry corruptions, equivariant policies can outperform non-equivariant ones.
Revised: These results highlight that even with some symmetry corruptions, equivariant policies can outperform non-equivariant ones.

Original: See Appendix D.4.1 for an additional experiment where we vary the level of symmetry corruptions as in Section 6.2.
Revised: See Appendix E.4.1 for an additional experiment where we vary the level of symmetry corruptions as in Section 6.2.

Original:  D ISCUSSION
Revised:  D ISCUSSION

Original: This paper defines correct equivariance, incorrect equivariance, and extrinsic equivariance, and identifies that enforcing extrinsic equivariance does not necessarily increase error.
Revised: This paper defines correct equivariance, incorrect equivariance, and extrinsic equivariance, and identifies that enforcing extrinsic equivariance does not necessarily increase error.

Original: This paper further demonstrates experimentally that extrinsic equivariance can provide significant performance improvements in reinforcement learning.
Revised: This paper further demonstrates experimentally that extrinsic equivariance can provide significant performance improvements in reinforcement learning.

Original: A limitation of this work is that we mainly experiment in reinforcement learning and a simple supervised setting but not in other domains where equivariant learning is widely used.
Revised: A limitation of this work is that we mainly experiment in reinforcement learning and a simple supervised setting but not in other domains where equivariant learning is widely used.

Original: The experimental results of our work suggest that an extrinsic equivariance should also be beneficial in those domains, but we leave this demonstration to future work.
Revised: The experimental results of our work suggest that an extrinsic equivariance should also be beneficial in those domains, but we leave this demonstration to future work.

Original: Another limitation is that we focus on planar equivariant networks.
Revised: Another limitation is that we focus on planar equivariant networks.

Original: In future work, we are interested in evaluating extrinsic equivariance in network architectures that process different types of data.
Revised: In future work, we are interested in evaluating extrinsic equivariance in network architectures that process different types of data.

Original: R EPRODUCIBILITY S TATEMENT
Revised: R EPRODUCIBILITY S TATEMENT

Original: For the review propose, our code is anonymously available at https://anonymous.4open.
Revised: For the review propose, our code is anonymously available at https://anonymous.4open.

Original: science/r/surprising_equi/ .
Revised: science/r/surprising_equi/ .

Original: In the final submission, we will include a link to a publicavailable repository.
Revised: In the final submission, we will include a link to a publicavailable repository.

Original: E THIC S TATEMENT
Revised: E THIC S TATEMENT

Original: Equivariant models allow us to train robots faster and more accurately in many different tasks.
Revised: Equivariant models allow us to train robots faster and more accurately in many different tasks.

Original: Our work shows this advantage can be applied even more broadly to tasks in real-world conditions.
Revised: Our work shows this advantage can be applied even more broadly to tasks in real-world conditions.

Original: Our method is agnostic to the morality of the actions which robots are trained for and, in that sense, can make it easier for robots to be used for either societally beneficial or detrimental tasks
Revised: Our method is agnostic to the morality of the actions which robots are trained for and, in that sense, can make it easier for robots to be used for either societally beneficial or detrimental tasks

Original: A T HEORETICAL U PPER B OUND ON A CCURACY FOR M ODELS WITH I NCORRECT S YMMETRY
Revised: A T HEORETICAL U PPER B OUND ON A CCURACY FOR M ODELS WITH I NCORRECT S YMMETRY

Original: We consider a classification problem over the set X with finitely many classes Y .
Revised: We consider a classification problem over the set X with finitely many classes Y .

Original: Let m = | Y | be the number of classes.
Revised: Let m = | Y | be the number of classes.

Original: Let l : X → Y be the true labels.
Revised: Let l : X → Y be the true labels.

Original: Let G be a finite group acting on
Revised: Let G be a finite group acting on

Original: X .
Revised: X .

Original: We assume the action of G on X is density preserving.
Revised: We assume the action of G on X is density preserving.

Original: That is, if p X is the density function corresponding to the input domain, then p X ( gx ) =
Revised: That is, if p X is the density function corresponding to the input domain, then p X ( gx ) =

Original: p X ( x ) .
Revised: p X ( x ) .

Original: Denote the orbit of a point x ∈ X by
Revised: Denote the orbit of a point x ∈ X by

Original: Gx = { gx : g ∈ G } and the stabilizer by G x = { g ∈ G : gx = x } .
Revised: Gx = { gx : g ∈ G } and the stabilizer by G x = { g ∈ G : gx = x } .

Original: By the orbit-stabilizer theorem | G | = | G x || Gx | .
Revised: By the orbit-stabilizer theorem | G | = | G x || Gx | .

Original: Now consider a model f : X → Y with incorrect equivariance constrained to be invariant to G .
Revised: Now consider a model f : X → Y with incorrect equivariance constrained to be invariant to G .

Original: We partition the input set into subsets X
Revised: We partition the input set into subsets X

Original: = (cid:96) mk
Revised: = (cid:96) mk

Original: =1 X k where
Revised: =1 X k where

Original: [Equation]
Revised: [Equation]

Original: If f has correct equivariance then X
Revised: If f has correct equivariance then X

Original: = X 1 .
Revised: = X 1 .

Original: Incorrect equivariance implies that there are orbits Gx which are assigned more than one label.
Revised: Incorrect equivariance implies that there are orbits Gx which are assigned more than one label.

Original: Since f is constrained to be equivariant such orbits will necessarily result in some errors.
Revised: Since f is constrained to be equivariant such orbits will necessarily result in some errors.

Original: We give an upper bound on that error.
Revised: We give an upper bound on that error.

Original: Define c k = P ( x ∈ X k ) .
Revised: Define c k = P ( x ∈ X k ) .

Original: Note that since X k give a partition, (cid:80) mk =

Revised: Note that since X k give a partition, (cid:80) mk =

Original: c k = 1 .
Revised: c k = 1 .

Original: Also, X k is empty for k > | G | since the number of labels assigned to an orbit is also upper bounded by the number of points in the orbit which is at most | G | .
Revised: Also, X k is empty for k > | G | since the number of labels assigned to an orbit is also upper bounded by the number of points in the orbit which is at most | G | .

Original: Letting K = min( | Y | , | G | ) , we have X =
Revised: Letting K = min( | Y | , | G | ) , we have X =

Original: (cid:96)
Revised: (cid:96)

Original: Kk =1 X k .
Revised: Kk =1 X k .

Original: Proposition A.1.
Revised: Proposition A.1.

Original: The accuracy of f has upper bound acc( f ) ≤ 1 − (cid:80)
Revised: The accuracy of f has upper bound acc( f ) ≤ 1 − (cid:80)

Original: Kk =1 c k ( k − 1) / | G | .
Revised: Kk =1 c k ( k − 1) / | G | .

Original: In contrast, we can choose an unconstrained model from a model class with a universal approximation property and given properly chosen hyperparameters find a model with arbitrarily good accuracy.
Revised: In contrast, we can choose an unconstrained model from a model class with a universal approximation property and given properly chosen hyperparameters find a model with arbitrarily good accuracy.

Original: Proof.
Revised: Proof.

Original: Let y
Revised: Let y

Original: =
Revised: =

Original: l ( x ) .
Revised: l ( x ) .

Original: Then acc( f ) =
Revised: Then acc( f ) =

Original: E x ∈ X
Revised: E x ∈ X

Original: [ δ ( f ( x ) = y )] .
Revised: [ δ ( f ( x ) = y )] .

Original: Since the action of G is density preserving, applying an element of G before sampling does not affect the expectation, E x ∈ X [ δ ( f ( x ) = y )]
Revised: Since the action of G is density preserving, applying an element of G before sampling does not affect the expectation, E x ∈ X [ δ ( f ( x ) = y )]

Original: =
Revised: =

Original: E x ∈ X
Revised: E x ∈ X

Original: [ δ ( f ( gx ) = y )] and so
Revised: [ δ ( f ( gx ) = y )] and so

Original: [Equation]
Revised: [Equation]

Original: If we split the expectation over the partition X = (cid:96)
Revised: If we split the expectation over the partition X = (cid:96)

Original: Kk =

Revised: Kk =

Original: X k we get
Revised: X k we get

Original: [Equation]
Revised: [Equation]

Original: Interchanging sums gives
Revised: Interchanging sums gives

Original: [Equation]
Revised: [Equation]

Original: By the orbit-stabilizer theorem,
Revised: By the orbit-stabilizer theorem,

Original: [Equation]
Revised: [Equation]

Original: which is the average accuracy over the orbit Gx .
Revised: which is the average accuracy over the orbit Gx .

Original: Since f is constrained to a single value of the orbit, and k different true labels appear, the highest accuracy attainable is when | G | =
Revised: Since f is constrained to a single value of the orbit, and k different true labels appear, the highest accuracy attainable is when | G | =

Original: | Gx | and the true labels and maximally unequally distributed such that 1 point in the orbit takes each of k − 1 labels and all the other | G | − ( k − 1) points receive a single label.
Revised: | Gx | and the true labels are maximally unequally distributed such that 1 point in the orbit takes each of k − 1 labels and all the other | G | − ( k − 1) points receive a single label.

Original: In this case accuracy can be maximized by choosing f ( x ′ ) to be this majority label, and
Revised: In this case accuracy can be maximized by choosing f ( x ′ ) to be this majority label, and

Original: [Equation]
Revised: [Equation]

Original: Substituting back in,
Revised: Substituting back in,

Original: [Equation]
Revised: [Equation]

Original: since k − 1 | G | is constant over X k and (cid:80)
Revised: since k − 1 | G | is constant over X k and (cid:80)

Original: Kk
Revised: Kk

Original: =

Revised: =

Original: c k = 1 .
Revised: c k = 1 .

Original: Note that the assumption that | G | = | Gx | and that the labels on a given orbit are maximally unequally distributed need not hold in general and thus this bound is not tight.
Revised: Note that the assumption that | G | = | Gx | and that the labels on a given orbit are maximally unequally distributed need not hold in general and thus this bound is not tight.

Original: In order to produce a tight upper bound, consider a partition X
Revised: In order to produce a tight upper bound, consider a partition X

Original: = (cid:96) p X p where X p = { x ∈ X : (max y | f − 1 ( y ) ∩
Revised: = (cid:96) p X p where X p = { x ∈ X : (max y | f − 1 ( y ) ∩

Original: Gx | ) / | Gx | = p } and define c p = P ( x ∈ X p ) .
Revised: Gx | ) / | Gx | = p } and define c p = P ( x ∈ X p ) .

Original: The set X p contains points in orbits where the majority label covers a fraction p of the points.
Revised: The set X p contains points in orbits where the majority label covers a fraction p of the points.

Original: Note that although p is a fraction between 0 and 1, there are only finitely many possible values of p since the numerator and denominator and bounded natural numbers.
Revised: Note that although p is a fraction between 0 and 1, there are only finitely many possible values of p since the numerator and denominator and bounded natural numbers.

Original: We may thus sum over the values of p .
Revised: We may thus sum over the values of p .

Original: Proposition A.2.
Revised: Proposition A.2.

Original: The accuracy of f has upper bound acc( f ) ≤ (cid:80) p c p
Revised: The accuracy of f has upper bound acc( f ) ≤ (cid:80) p c p

Original: p.
Revised: p.

Original: Proof.
Revised: Proof.

Original: The proof is similar to the proof of Proposition A.1 replace X k and c k with X p and c p respectively.
Revised: The proof is similar to the proof of Proposition A.1 replace X k and c k with X p and c p respectively.

Original: For x ∈ X p , the term | 1 | | Gx | (cid:80) x ′ ∈
Revised: For x ∈ X p , the term | 1 | | Gx | (cid:80) x ′ ∈

Original: Gx δ ( f ( x ′ ) =
Revised: Gx δ ( f ( x ′ ) =

Original: y ) can be upper bounded by choosing the majority label yielding | 1 | | Gx | (cid:80) x ′ ∈
Revised: y ) can be upper bounded by choosing the majority label yielding | 1 | | Gx | (cid:80) x ′ ∈

Original: Gx δ ( f ( x ′ ) =
Revised: Gx δ ( f ( x ′ ) =

Original: y ) ≤ p .
Revised: y ) ≤ p .

Original: The bound then follows as before.
Revised: The bound then follows as before.

Original: This is a tight upper bound since assigning any but the majority label would result in lower accuracy.
Revised: This is a tight upper bound since assigning any but the majority label would result in lower accuracy.

Original: Figure 13 demonstrates the upper bound of an incorrectly constrained equivariant network with the invert label corruption in Section 5, where acc( f ) ≤ 0 .
Revised: Figure 13 demonstrates the upper bound of an incorrectly constrained equivariant network with the invert label corruption in Section 5, where acc( f ) ≤ 0 .

Original: 25 × 1 + 0 .
Revised: 25 × 1 + 0 .

Original: 75 × 0 .
Revised: 75 × 0 .

Original: 5 = 0 .
Revised: 5 = 0 .

Original: B N ETWORK A RCHITECTURE B. S UPERVISED L EARNING
Revised: B C ORRECT , I NCORRECT , AND E XTRINSIC E QUIVARIANCE E XAMPLES

Revised: In this section, we describe how the model symmetry transforms data under correct, incorrect, and extrinsic equivariance and how such transformations relate to the true symmetry present in the task using the example of Section 4.2.

Revised: The ground truth function f : X → Y is a mapping from X =

Revised: Rto Y

Revised: = { ORANGE , BLUE } .

Revised: Let ( a, a ) , ( − a, − a ) , ( − a, a ) , ( b, c ) be the coordinates of four points in the data distribution on the unit circle (Figure 14a).

Revised: The ground truth labels for these points are: f ( a, a ) =

Revised: ORANGE , f ( − a, − a ) =

Revised: BLUE , f ( − a, a ) =

Revised: BLUE , f ( b, c ) = ORANGE .

Revised: B. C ORRECT E QUIVARIANCE

Revised: Definition 4.1.

Revised: The action ˆ ρ x has correct equivariance with respect to f if ˆ ρ x ( g ) x ∈ D for all x ∈ D, g ∈ G and f (ˆ ρ x ( g ) x ) =

Revised: ρ y ( g ) f

Revised: ( x ) .

Revised: Consider the reflection group G = C 2 = { 1 , r } (where r is the reflection along the horizontal axis) acting on X by ˆ ρ x (1)

Revised: =

Revised: Id or ˆ ρ x ( r ) =

Revised: (cid:0) 1 0 0 − 1 (cid:1) and Y via ρ y = Id , the trivial action fixing the labels (Figure 14b).

Revised: If we define an equivariant model with respect to ˆ ρ x and ρ y , then the model’s symmetry preserves the problem symmetry.

Revised: For example, consider the point ( − a, a ) , r ∈ G 1 is the reflection so that ˆ ρ x ( r )

Revised: = (cid:0) 1 0 0 − 1 (cid:1) and ˆ ρ x ( r )( − a, a ) =

Revised: ( − a, − a ) .

Revised: Since the model f ϕ is G 1 -equivariant, f ϕ (ˆ ρ x ( r ) x )

Revised: =

Revised: ρ y ( r ) f ϕ ( x ) .

Revised: Substituting ρ y

Revised: = Id and x = ( − a, a ) , we obtain f ϕ ( − a, − a ) =

Revised: f ϕ ( − a, a ) , meaning that the output of f ϕ ( − a, − a ) and f ϕ ( − a, a ) are constrained to be equal.

Revised: Thus the invariance property in the ground truth function f where f ( − a, − a ) =

Revised: f ( − a, a ) =

Revised: BLUE is preserved (notice that this applies to all x ∈ X ).

Revised: We call this correct equivariance

Revised: . B. I NCORRECT E QUIVARIANCE

Revised: Definition 4.2.

Revised: The action ˆ ρ x has incorrect equivariance with respect to f if there exist x ∈ D and g ∈ G such that ˆ ρ x ( g ) x ∈ D but f (ˆ ρ x ( g ) x ) ̸

Revised: =

Revised: ρ y ( g ) f ( x ) .

Revised: Consider the rotation group G 2 = ⟨ Rot π ⟩ (Figure 14c) which acts via ˆ ρ x on X via a rotation matrix of π and acts on Y via ρ y = Id .

Revised: If we define an equivariant model with respect to ˆ ρ x and ρ y , the network’s symmetry will conflict with the problem’s symmetry.

Revised: For example, consider the point ( a, a ) and let g ∈ G 2 be the rotation action so that ˆ ρ x ( g )

Revised: = (cid:0) − 1 0 0 − 1 (cid:1) and ˆ ρ x ( g )( a, a ) =

Revised: ( − a, − a ) .

Revised: As the model f ϕ is G 2 -equivariant, f ϕ (ˆ ρ x ( g ) x )

Revised: =

Revised: ρ y ( g ) f ϕ ( x ) .

Revised: Substituting ρ y

Revised: = Id and x = ( a, a ) , we get f ϕ ( − a, − a ) =

Revised: f ϕ ( a, a ) .

Revised: However, this constraint interferes with the ground truth function f as f ( − a, − a ) =

Revised: BLUE and f ( a, a ) = ORANGE .

Revised: We call this incorrect equivariance.

Revised: B. E XTRINSIC E QUIVARIANCE

Revised: Definition 4.3.

Revised: The action ˆ ρ x has extrinsic equivariance with respect to f if for x ∈ D , ˆ ρ x ( g ) x

Revised: ̸∈ D .

Revised: Consider the scaling group G 3 acting on X by scaling the vector and on Y via ρ y

Revised: = Id (Figure 14d).

Revised: If we define an equivariant model with respect to ˆ ρ x and ρ y , the group-transformed data will be outside the input distribution.

Revised: Consider the point ( b, c ) and let g ∈ G 3 be the scaling action so that ˆ ρ x ( g )( b, c )

Revised: = ( b ′ , c ′ ) .

Revised: Since the model f ϕ is G 3 -equivariant, f ϕ (ˆ ρ x ( g ) x )

Revised: =

Revised: ρ y ( g ) f ϕ ( x ) .

Revised: Substituting ρ y

Revised: = Id and x = ( b, c ) we have f ϕ ( b ′ , c ′ )

Revised: =

Revised: f ϕ ( b, c ) meaning that the output of f ϕ ( b ′ , c ′ ) and f ϕ ( b, c ) are constrained to be equal.

Revised: However, ( b ′ , c ′ ) is outside of the input distribution (gray ring) and thus the ground truth f ( b ′ , c ′ ) is undefined.

Revised: We call this extrinsic equivariance.

Revised: Intuitively, it is easy to see in this example how extrinsic equivariance would help the model learn f .

Revised: If the model f ϕ is equivariant to the scale group G 3 , then it can generalize to “scaled” up or down versions of the input distribution and “covers” more of the input space R 2 .

Revised: As such, the model may learn the decision boundary (the vertical axis) more easily because of its equivariance compared to a non-equivariant model, even if the equivariance is extrinsic.

Revised: C N ETWORK A RCHITECTURE

Original:  Figure 14 shows the network architecture of the equivariant network and Figure 15 shows the network architecture of the CNN network in Section 5.
Revised: [Figure] [Table] [Figure] Figure 15 shows the network architecture of the equivariant network and Figure 16 shows the network architecture of the CNN network in Section 5.

Original: Both networks are 8-layer convolutional neural networks.
Revised: Both networks are 8-layer convolutional neural networks.

Original: The equivariant network is implemented using the e2cnn (Weiler & Cesa, 2019) library, where the hidden layers are defined using the regular representation and the output layer is defined using the trivial representation.
Revised: The equivariant network is implemented using the e2cnn (Weiler & Cesa, 2019) library, where the hidden layers are defined using the regular representation and the output layer is defined using the trivial representation.

Original: Table 1 shows the numbers of trainable parameters in both networks, where both networks have a similar number with a slight advantage in the CNN.[Table]
Revised: Table 1 shows the numbers of trainable parameters in both networks, where both networks have a similar number with a slight advantage in the CNN.

Original: B. R EINFORCEMENT L EARNING IN R OBOTIC M ANIPULATION
Revised: C. R EINFORCEMENT L EARNING IN R OBOTIC M ANIPULATION

Original: Figure 16 shows the network architecture of Equivariant SAC used in manipulation tasks in Section 6.1.
Revised: Figure 17 shows the network architecture of Equivariant SAC used in manipulation tasks in Section 6.1.

Original: All hidden layers are implemented using the regular representation.
Revised: All hidden layers are implemented using the regular representation.

Original: For the actor (top), the output is a mixed representation containing one standard representation for the ( x, y ) actions, one signed representation for the θ action, and seven trivial representations for the ( z, λ ) actions and the standard deviations of all action components.
Revised: For the actor (top), the output is a mixed representation containing one standard representation for the ( x, y ) actions, one signed representation for the θ action, and seven trivial representations for the ( z, λ ) actions and the standard deviations of all action components.

Original: Figure 17 shows the network architecture of CNN
Revised: Figure 18 shows the network architecture of CNN

Original: SAC for both RAD and DrQ. Figure 18 shows the network architecture of FERM.
Revised: SAC for both RAD and DrQ. Figure 19 shows the network architecture of FERM.

Original: Figure 19 shows the network architecture of SEN.
Revised: Figure 20 shows the network architecture of SEN.

Original: Table 2 shows the number of trainable parameters for each model.
Revised: Table 2 shows the number of trainable parameters for each model.

Original: All baselines have slightly more parameters compared with Equivariant SAC.
Revised: All baselines have slightly more parameters compared with Equivariant SAC.

Original: C T RAINING D ETAILS C. S UPERVISED L EARNING
Revised: D T RAINING D ETAILS D. S UPERVISED L EARNING

Original: We implement the environment in the PyBullet simulator (Coumans & Bai, 2016).
Revised: We implement the environment in the PyBullet simulator (Coumans & Bai, 2016).

Original: The ducks are located in a workspace with a size of 0 .
Revised: The ducks are located in a workspace with a size of 0 .

Original: 3 m × 0 .
Revised: 3 m × 0 .

Original: 3 m .
Revised: 3 m .

Original: The pixel size of the image is 152 × 152 (and will be cropped to 128 × 128 during training).
Revised: The pixel size of the image is 152 × 152 (and will be cropped to 128 × 128 during training).

Original: We implement the training in PyTorch (Paszke et al., 2017) using a cross-entropy loss.
Revised: We implement the training in PyTorch (Paszke et al., 2017) using a cross-entropy loss.

Original: The output of the model is the score for each g ∈ C 8 .
Revised: The output of the model is the score for each g ∈ C 8 .

Original: We use the Adam optimizer (Kingma & Ba, 2014) with a learning rate of 10 − 4 .
Revised: We use the Adam optimizer (Kingma & Ba, 2014) with a learning rate of 10 − 4 .

Original: The batch size is 64.
Revised: The batch size is 64.

Original: In all training, we perform a three-way data split with N training data, 200 holdout validation data, and 200 holdout test data.
Revised: In all training, we perform a three-way data split with N training data, 200 holdout validation data, and 200 holdout test data.

Original: The training is terminated either when the validation prediction success rate does not improve for 100 epochs or when the maximum epoch (1000) is reached.
Revised: The training is terminated either when the validation prediction success rate does not improve for 100 epochs or when the maximum epoch (1000) is reached.

Original: C. R EINFORCEMENT L EARNING IN R OBOTIC M ANIPULATION
Revised: D. R EINFORCEMENT L EARNING IN R OBOTIC M ANIPULATION

Original: We use the environments provided by the BulletArm benchmark (Wang et al., 2022b) implemented in the PyBullet simulator (Coumans & Bai, 2016).
Revised: We use the environments provided by the BulletArm benchmark (Wang et al., 2022b) implemented in the PyBullet simulator (Coumans & Bai, 2016).

Original: The workspace’s size is 0 .
Revised: The workspace’s size is 0 .

Original: 4 m × 0 . 4 m × 0 .
Revised: 4 m × 0 . 4 m × 0 .

Original: 24 m .
Revised: 24 m .

Original: [Table] The pixel size of the image observation is 152 × 152 (and will be cropped to 128 × 128 during training).
Revised: The pixel size of the image observation is 152 × 152 (and will be cropped to 128 × 128 during training).

Original: The action space is A x , A y , A z = [ − 0 .
Revised: The action space is A x , A y , A z = [ − 0 .

Original: 05 m, 0 .
Revised: 05 m, 0 .

Original: 05 m ] for the change of ( x, y, z ) position of the gripper;
Revised: 05 m ] for the change of ( x, y, z ) position of the gripper;

Original: A θ =
Revised: A θ =

Original: [ − π 4 , π 4 ] for the change of top-down rotation of the gripper; and A λ =
Revised: [ − π 4 , π 4 ] for the change of top-down rotation of the gripper; and A λ =

Original: [0 , 1] for the open width of the gripper where 0 means fully close and 1 means fully open.
Revised: [0 , 1] for the open width of the gripper where 0 means fully close and 1 means fully open.

Original: All environments have a sparse reward: +1 for reaching the goal and 0 otherwise.
Revised: All environ- ments have a sparse reward: +1 for reaching the goal and 0 otherwise.

Original: During training, we useparallel environments where a training step is performed after all 5 parallel environments perform an action step.
Revised: During training, we useparallel environments where a training step is performed after all 5 parallel environments perform an action step.

Original: The evaluation is performed every 200 training steps.
Revised: The evaluation is performed every 200 training steps.

Original: We implement the training in
Revised: We implement the training in

Original: PyTorch (Paszke et al., 2017).
Revised: PyTorch (Paszke et al., 2017).

Original: We use the Adam optimizer (Kingma & Ba, 2014) with a learning rate of 10 − 3 .
Revised: We use the Adam optimizer (Kingma & Ba, 2014) with a learning rate of 10 − 3 .

Original: The batch size is 128.
Revised: The batch size is 128.

Original: The entropy temperature for SAC is initialized at 10 − 2 .
Revised: The entropy temperature for SAC is initialized at 10 − 2 .

Original: The target entropy is − 5 .
Revised: The target entropy is − 5 .

Original: The discount factor γ = 0 .
Revised: The discount factor γ = 0 .

Original: The Prioritized Experience Replay (PER) (Schaul et al., 2015) has a capacity of 100,000 transitions with prioritized replay exponent of α = 0 .
Revised: The Prioritized Experience Replay (PER) (Schaul et al., 2015) has a capacity of 100,000 transitions with prioritized replay exponent of α = 0 .

Original: 6 and prioritized importance sampling exponent β 0 = 0 .
Revised: 6 and prioritized importance sampling exponent β 0 = 0 .

Original: 4 as in Schaul et al.
Revised: 4 as in Schaul et al.

Original: The expert transitions are given a priority bonus of ϵ d = 1 .
Revised: The expert transitions are given a priority bonus of ϵ d = 1 .

Original: The contrastive encoder of the FERM baseline has an encoding size of 50 as in Zhan et al.
Revised: The contrastive encoder of the FERM baseline has an encoding size of 50 as in Zhan et al.

Original: The FERM baseline’s contrastive encoder is pre-trained for 1.6k steps using the expert data as in Zhan et al.
Revised: The FERM baseline’s contrastive encoder is pre-trained for 1.6k steps using the expert data as in Zhan et al.

Original: In DrQ, the number of augmentations for calculating the target K and the number of augmentations for calculating the loss M are both 2 as in Yarats et al. 
Revised: In DrQ, the number of augmentations for calculating the target K and the number of augmentations for calculating the loss M are both 2 as in Yarats et al. [Table]

Original: C. R EINFORCEMENT L EARNING IN D EEP M IND C ONTROL S UITE
Revised: D. R EINFORCEMENT L EARNING IN D EEP M IND C ONTROL S UITE

Original: Sample images of each environment are shown in Figure 20.
Revised: Sample images of each environment are shown in Figure 21.

Original: Environment observations are 3 consecutive frames of RGB images of size 85 × 85 , in order to infer velocity and acceleration.
Revised: Environment observations are 3 consecutive frames of RGB images of size 85 × 85 , in order to infer velocity and acceleration.

Original: Note that we use odd-sized image sizes instead of 84 × 84 used in Yarats et al.
Revised: Note that we use odd-sized image sizes instead of 84 × 84 used in Yarats et al.

Original: (2022), as the DrQv architecture contains a convolutional layer with stride 2 and this breaks equivariance for even-sized spatial inputs (Mohamed et al., 2020).
Revised: (2022), as the DrQv architecture contains a convolutional layer with stride 2 and this breaks equivariance for even-sized spatial inputs (Mohamed et al., 2020).

Original: For each environment, an episode lasts 1000 steps where each step has a reward between 0 and 1 .
Revised: For each environment, an episode lasts 1000 steps where each step has a reward between 0 and 1 .

Original: [Figure]
Revised: [Figure]

Original: We modify the original DrQv2 by making the encoder map down to a smaller spatial output, leading to faster training.
Revised: We modify the original DrQv2 by making the encoder map down to a smaller spatial output, leading to faster training.

Original: The second and third convolutional blocks have an added max-pooling layer, leading to a spatial output size of 7 × 7 .
Revised: The second and third convolutional blocks have an added max-pooling layer, leading to a spatial output size of 7 × 7 .

Original: As the equivariant version of DrQv2 has an additional convolutional layer after the action restriction, the non-equivariant version also has an additional convolutional layer at the end of the encoder.
Revised: As the equivariant version of DrQv2 has an additional convolutional layer after the action restriction, the non-equivariant version also has an additional convolutional layer at the end of the encoder.

Original: We also scale the number of channels by (cid:112) | G | in order to preserve roughly the same number of parameters as the non-equivariant version.
Revised: We also scale the number of channels by (cid:112) | G | in order to preserve roughly the same number of parameters as the non-equivariant version.

Original: The policy is evaluated by averaging the return of 10 episodes every 10000 environment steps.
Revised: The policy is evaluated by averaging the return of 10 episodes every 10000 environment steps.

Original: In all
Revised: In all

Original: DMC experiments, we plot the mean and the standard error over 4 seeds.
Revised: DMC experiments, we plot the mean and the standard error over 4 seeds.

Original: All other training details and hyperparameters are kept the same as in Yarats et al.
Revised: All other training details and hyperparameters are kept the same as in Yarats et al.

Original: D A DDITIONAL E XPERIMENTS D. S UPERVISED L EARNING WITH M ORE S YMMETRY C ORRUPTION T YPES
Revised: E A DDITIONAL E XPERIMENTS E. S UPERVISED L EARNING WITH M ORE S YMMETRY C ORRUPTION T YPES

Original: In this section, we demonstrate the experiment in Section 5 in more symmetry corruptions.
Revised: In this section, we demonstrate the experiment in Section 5 in more symmetry corruptions.

Original: Figure 21 shows the 15 different corruptions.
Revised: Figure 22 shows the 15 different corruptions.

Original: We also show the performance of ‘Equi’, ‘CNN’, and ‘CNN + Img Trans’ without the random crop augmentation used in Section Section 5 (labeled as ‘no Crop’ variations).
Revised: We also show the performance of ‘Equi’, ‘CNN’, and ‘CNN + Img Trans’ without the random crop augmentation used in Section Section 5 (labeled as ‘no Crop’ variations).

Original: The result is shown in Figure 22.
Revised: The result is shown in Figure 23.

Original: First, comparing blue vs green, and purple vs orange, the equivariant network always outperforms the CNN with or without random crop augmentation, especially with fewer data.
Revised: First, comparing blue vs green, and purple vs orange, the equivariant network always outperforms the CNN with or without random crop augmentation, especially with fewer data.

Original: Second, comparing blue vs purple, and green vs orange, random crop generally helps both the equivariant network and the CNN network.
Revised: Second, comparing blue vs purple, and green vs orange, random crop generally helps both the equivariant network and the CNN network.

Original: Third, comparing red vs green, and cyan vs orange, adding the image transformation augmentation improves the performance of CNN.
Revised: Third, comparing red vs green, and cyan vs orange, adding the image transformation augmentation improves the performance of CNN.

Original: Notice that the condition reverse is an outlier because the equivariant network has incorrect equivariance, where the CNN methods (green and orange) without image transformation augmentation have the best performance.
Revised: Notice that the condition reverse is an outlier because the equivariant network has incorrect equivariance, where the CNN methods (green and orange) without image transformation augmentation have the best performance.

Original: D. RL IN M ANIPULATION WITHOUT R ANDOM C ROP
Revised: E. RL IN M ANIPULATION WITHOUT R ANDOM C ROP

Original: In this section, we demonstrate the performance of Equivariant SAC and CNN SAC without random crop augmentation using RAD.
Revised: In this section, we demonstrate the performance of Equivariant SAC and CNN SAC without random crop augmentation using RAD.

Original: As is shown in Figure 23, both methods work poorly without the random crop augmentation.
Revised: As is shown in Figure 24, both methods work poorly without the random crop augmentation.

Original: D. RL IN M ANIPULATION WITH O CCLUSION C ORRUPTION
Revised: E. RL IN M ANIPULATION WITH O CCLUSION C ORRUPTION

Original: In this section, we perform the same experiment as in Section 6.1 with a different type of symmetry corruption: occlusion due to orthographic projection using a single camera.
Revised: In this section, we perform the same experiment as in Section 6.1 with a different type of symmetry corruption: occlusion due to orthographic projection using a single camera.

Original: Instead of using an
Revised: Instead of using an

Original: RGBD image observation as in Section 6.1, we take the depth channel from the RGBD image and perform an orthographic projection at the gripper’s position (Figure 24).
Revised: RGBD image observation as in Section 6.1, we take the depth channel from the RGBD image and perform an orthographic projection at the gripper’s position (Figure 25).

Original: This is the same process as in Wang et al.
Revised: This is the same process as in Wang et al.

Original: (2022a) to generate a top-down image for equivariant learning, however, since we only have one camera instead of two as in the prior work, this orthographic projection will have missing depth values due to occlusion and thus leads to an extrinsic equivariant constraint.
Revised: (2022a) to generate a top-down image for equivariant learning, however, since we only have one camera instead of two as in the prior work, this orthographic projection will have missing depth values due to occlusion and thus leads to an extrinsic equivariant constraint.

Original: Figure 25 shows the results.
Revised: Figure 26 shows the results.

Original: Similar as in Section 6.1, Equivariant SAC outperforms all baselines with a significant margin.
Revised: Similar as in Section 6.1, Equivariant SAC outperforms all baselines with a significant margin.

Original: D. RL IN D EEP M IND C ONTROL S UITE
Revised: E. RL IN D EEP M IND C ONTROL S UITE

Original: Figure 26 is another visualization of equivariant vs non-equivariant DrQv2 on the original pendulum swingup environment.
Revised: Figure 27 is another visualization of equivariant vs non-equivariant DrQv2 on the original pendulum swingup environment.

Original: As each method has 1 failed seed, we plot all runs with slightly different color shades.
Revised: As each method has 1 failed seed, we plot all runs with slightly different color shades.

Original: If we exclude the failed run from each method, it can easily be seen that equivariant DrQv learns faster than the non-equivariant version.
Revised: If we exclude the failed run from each method, it can easily be seen that equivariant DrQv learns faster than the non-equivariant version.

Original: D.4. I NCREASING SYMMETRY CORRUPTIONS
Revised: E.4. I NCREASING SYMMETRY CORRUPTIONS

Original: In these experiments, we modify some domains to have different levels of symmetry-breaking corruptions.
Revised: In these experiments, we modify some domains to have different levels of symmetry-breaking corruptions.

Original: For cartpole and cup catch, we either remove the gridded floor and background (None) to make the observation perfectly equivariant or keep the floor and background and further change the camera angle by rolling ( 30 ◦ − 75 ◦ ), increasing the level of corruption.
Revised: For cartpole and cup catch, we either remove the gridded floor and background (None) to make the observation perfectly equivariant or keep the floor and background and further change the camera angle by rolling ( 30 ◦ − 75 ◦ ), increasing the level of corruption.

Original: For reacher, we use the same modifications but tilt the camera instead of rolling.
Revised: For reacher, we use the same modifications but tilt the camera instead of rolling.

Original: See Figure 3 for sample images.
Revised: See Figure 3 for sample images.

Original: In order to see the effects of increasing corruption on learning, we plot the mean discounted reward when both methods have converged ( 30 k frames for cartpole and cup catch, 1 . 5 M frames for reacher).
Revised: In order to see the effects of increasing corruption on learning, we plot the mean discounted reward when both methods have converged ( 30 k frames for cartpole and cup catch, 1 . 5 M frames for reacher).

Original: Figure 27 shows that both the equivariant and non-equivariant DrQv2 surprisingly perform quite well across all corruption levels, with the exception of 75 ◦ on reacher.
Revised: Figure 28 shows that both the equivariant and non-equivariant DrQv2 surprisingly perform quite well across all corruption levels, with the exception of 75 ◦ on reacher.

Original: The equivariant policy seems to converge to a slightly higher discounted reward than the non-equivariant version, though the difference is not significant.
Revised: The equivariant policy seems to converge to a slightly higher discounted reward than the non-equivariant version, though the dif- ference is not significant.

Original: On reacher, changing the camera angle may have affected both methods by making the task more difficult for both an equivariant and regular CNN encoder.[Table]
Revised: On reacher, changing the camera angle may have affected both methods by making the task more difficult for both an equivariant and regular CNN encoder.

Original: E B ASELINE A RCHITECTURE S EARCH E. CNN SAC A RCHITECTURE S EARCH
Revised: F B ASELINE A RCHITECTURE S EARCH F. CNN SAC A RCHITECTURE S EARCH

Original: This section demonstrates the architecture search for CNN SAC.
Revised: This section demonstrates the architecture search for CNN SAC.

Original: We consider three different architectures (all with a similar amount of trainable parameters): 1) conv (Figure 17): a CNN network with the same structure as Equivariant SAC, where all layers are implemented using convolutional layers.
Revised: We consider three different architectures (all with a similar amount of trainable parameters): 1) conv (Figure 18): a CNN network with the same structure as Equivariant SAC, where all layers are implemented using convolutional layers.

Original: 2) fc1 (Figure 28): a CNN network that replaces some layers in 1) with fully connected layers. 3)
Revised: 2) fc1 (Figure 29): a CNN network that replaces some layers in 1) with fully connected layers. 3)

Original: fc2 (Figure 29): similar as 2), but with fewer convolutional layers and more weights in the FC layer three network architectures
Revised: fc2 (Figure 30): similar as 2), but with fewer convolutional layers and more weights in the FC layer.

Original:  SAC equipped random crop augmentation  RAD (Laskin et al., 2020b).
Revised: We evaluate the three network architectures with SAC equipped random crop augmentation using RAD (Laskin et al., 2020b).

Original: Figure 30 shows the result, where all three variations have a similar performance.
Revised: Figure 31 shows the result, where all three variations have a similar performance.

Original: We use conv in the main paper since it has a similar structure as Equivariant SAC.
Revised: We use conv in the main paper since it has a similar structure as Equivariant SAC.

Original: E. FERM A RCHITECTURE S EARCH
Revised: F. FERM A RCHITECTURE S EARCH

Original: This section demonstrates the architecture search for FERM.
Revised: This section demonstrates the architecture search for FERM.

Original: We consider four different architectures: 1) sim total 1 (Figure 32) and 2) sim total 2 (Figure 18) are two different architectures with the similar amount of total trainable parameters as Equivariant SAC. 3)
Revised: We consider four different architectures: 1) sim total 1 (Figure 33) and 2) sim total 2 (Figure 19) are two different architectures with the similar amount of total trainable parameters as Equivariant SAC. 3)

Original: sim enc (Figure 31) has similar amount of trainable parameters in the encoder as Equivariant SAC’s encoder.
Revised: sim enc (Figure 32) has similar amount of trainable parameters in the encoder as Equivariant SAC’s encoder.

Original: Notice that since FERM share an encoder between the actor and the critic while Equivariant SAC has separate encoders, having the similar amount of parameters in the encoder will lead to fewer total parameter in FERM compared with Equivariant SAC. 4) ferm ori
Revised: Notice that since FERM share an encoder between the actor and the critic while Equivariant SAC has separate encoders, having the similar amount of parameters in the encoder will lead to fewer total parameter in FERM compared with Equivariant SAC. 4) ferm ori

Original: (Figure 33) is the same network architecture used in the FERM paper (Zhan et al., 2020).
Revised: (Figure 34) is the same network architecture used in the FERM paper (Zhan et al., 2020).

Original: Figure 34 shows the comparison across the four architectures.
Revised: Figure 35 shows the comparison across the four architectures.

Original: ‘sim total 2’ has a marginal advantage compared with the other three variations, so we use it in the main paper. 
Revised: ‘sim total 2’ has a marginal advantage compared with the other three variations, so we use it in the main paper. Cartpole Swingup [Table]

Original: E. SEN A RCHITECTURE S EARCH
Revised: F. SEN A RCHITECTURE S EARCH

Original: This section shows the architecture search for SEN.
Revised: This section shows the architecture search for SEN.

Original: We consider three variations (all with similar amount of trainable parameters): 1) SEN conv (Figure 35): all layers are implemented using convolutional layers.
Revised: We consider three variations (all with similar amount of trainable parameters): 1) SEN conv (Figure 36): all layers are implemented using convolutional layers.

Original: 2) SEN fc1 (Figure 36) and SEN fc2 (Figure 19) replaces some layers in 1) with fully connected layers.
Revised: 2) SEN fc1 (Figure 37) and SEN fc2 (Figure 20) replaces some layers in 1) with fully connected layers.

Original: Figure 37 shows the comparison across the three variations.
Revised: Figure 38 shows the comparison across the three variations.

Original: ‘SEN fc2’ shows the best performance.
Revised: ‘SEN fc2’ shows the best performance.