Integrating Symmetry into Differentiable Planning with Steerable Convolutions
Authors: Linfeng Zhao
Xupeng Zhu
Lingzhi Kong
Robin Walters
Lawson L.S. Wong
Venue: ICLR 2023 poster (2023)

Abstract: To achieve this, we draw inspiration from equivariant convolution networks and model the path planning problem as a set of signals over grids. We demonstrate that value iteration can be treated as a linear equivariant operator, which is effectively a steerable convolution. Building upon Value Iteration Networks (VIN), we propose a new Symmetric Planning (SymPlan) framework that incorporates rotation and reflection symmetry using steerable convolution networks. We evaluate our approach on four tasks: 2D navigation, visual navigation, 2 degrees of freedom (2-DOF) configuration space manipulation, and 2-DOF workspace manipulation. Our experimental results show that our symmetric planning algorithms significantly improve training efficiency and generalization performance compared to non-equivariant baselines, including VINs and GPPN.

Published as a conference paper at ICLR 2023
INTEGRATING SYMMETRY INTO DIFFERENTIABLE PLANNING WITH STEERABLE CONVOLUTIONS
Linfeng Zhao", , Xupeng Zhu', Lingzhi Kong', Robin Walters, Lawson L.S. Wong Khoury College of Computer Sciences, Northeastern University
ABSTRACT
In this paper, we study a principled approach on incorporating group symme- try into end-to-end differentiable planning algorithms and explore the benefits of symmetry in planning. To achieve this, we draw inspiration from equivariant con- volution networks and model the path planning problem as a set of signals over grids. We demonstrate that value iteration can be treated a linear equivariant op- erator, which is effectively a steerable convolution. Building upon Value Iteration Networks (VIN), propose that incorporates rotation and reflection symmetry using steerable convolution networks. We evaluate approach on four tasks: 2D navigation, visual navi- gation, 2 degrees of freedom (2-DOF) configuration space manipulation, and 2- DOF workspace manipulation. Our experimental results show that our symmetric planning algorithms significantly improve training efficiency and generalization performance compared to non-equivariant baselines, including VINs and GPPN.
Symmetric Planning (SymPlan) framework
M
3M
INTRODUCTION
Model-based planning algorithms can struggle to find so- lutions for complex problems, and one solution is to ap- ply planning in a more structured and reduced space (Sut- ton and Barto, 2018; Li et al., 2006; Ravindran and Barto, 2004; Fox and Long, 2002). When a task exhibits sym- metry, this structure can be used to effectively reduce the search space for planning. However, existing planning al- gorithms often assume perfect knowledge of dynamics and require building equivalence classes, which can be inef- ficient and limit their applicability to specific tasks (Fox and Long, 1999; 2002; Pochter et al., 2011; Zinkevich and Balch, 2001; Narayanamurthy and Ravindran, 2008).
SymPlunn(A)
SymaPun(88)
S-A
Figure 1: Symmetry in path planning.
Symmetric Planning approach guarantees the solutions are same up to rotations.
In this paper, we study the aath-planning problem and its symmetry structure, as shown in Figure Given a map M (top row), the objective is to find optimal actions A = SymPlan(M1 (bottom row) to given position (red dots). we rotated the map g.M (top right), its solution g.4 (shortest path) can also be connected by rotation with the original solution A. Specifically, we say the task has symmetry since the solutions SymP (a (g) = g.SymP an( [M) are related by a O 90" rotation. As a more concrete example, the action in the NW corner of A is the same the action in the SW corner of g.A, after also rotating the arrow O 90%. This is an example of symmetry appearing in a specific task, which can be observed before solving the task or assuming other domain knowledge. If can use the rotation (and reflection) symmetry in this task, we effectively reduce the search space by |Ca 4 (or [D| 8) times. Instead, classic planning algorithms like A * would require searching symmetric states (NP-hard) with known dynamics (Pochter et al., 2011).
Recently, symmetry in model-free deep reinforcement learning (RL) has also been studied (Mon- dal et al., 2020; van der Pol et al., 2020a; Wang et al., 2021). A core benefit of model-free RL
*Coresponning Author: Linfeng Zhao zhao www....ii@nooottssteeeee Second and third authors contributed equally.

---

Published as a conference paper at ICLR 2023
that enables great asymptotic performance is its end-to-end differentiabilityy However, they lack long-horin horizon planning ability and only effectively handle pixel-level symmetry, such as flipping or rotating image observations and action together. This motivates to combine the spirit of both: can we enable end-to-end differentiable planning algorithms to make use of symmetry in environments?
In this work, we propose a framework called Symmetric Planning (SymPlan) that enables planning with symmetry in an end-to-end differentiable manner while avoiding the explicit construction of equivalence classes for symmetric states. Our framework is motivated by the work in the equivariant network and geometric deep learning community (Bronstein et al., 2021a; Cohen et al., 2020; Kon- dor and Trivedi, 2018; Cohen and Welling, 2016;;b; Weiler and Cesa, 2021), which views geometric data as signals (or "steerable feature fields") over base space. For instance, an RGB image is rep- resented as signal that maps ZZ to TR:3. The theory of equivariant networks enables the injection of symmetry into operations between signals through equívariant operations, such as convolutions. Equivariant networks applied to images do not need to explicitly consider "symmetric pixels" while still ensuring symmetry properties, thus avoiding the need to search symmetric states.
We apply this intuition to the task of path planning, which is both straightforward and general. Specifically, we focus on the 2D grid and demonstrate that value iteration (VI) for 2D path planning is equivariant under translations, rotations, and reflections (which are isometries of z²). We further show that VI for path planning is type of steerable convolution network, as developed in (Cohen and Welling, 20) To implement this approach, we use Value Iteration Network (VIN, (Tamar et al., 201 6a)) and its variants, since they require only operations between signals. We equip VIN with steerable convolution to create the equivariant steerable version of VIN, named SymVIN, and use variant called GPPN (Lee et al., 2018) to build SymGPPN. Both SymPlan methods sig- nificantly improve training efficiency and generalization performance on previously unseen random maps, which highlights the advantage of exploiting symmetry from environments for planning. Our contributions include:
We introduce a framework for incorporating symmetry into path path-planning problems on 2D grids, which is directly generalizable to other homogeneous spaces.
We prove that value iteration for path planning can be treated as a steerable CNN, motivating us implement SymVIN by replacing the 2D convolution with steerable convolution.
We show that both SymVIN and a related method, SymGPPN, offer significant improvements in training efficiency and generalization performance for 21) navigation and manipulation tasks.
2 RELATED WORK
Planning with symmetries. Symmetries are prevalent in various domains and have been used in classical planning algorithms and model checking (Fox and Long, 1999; 2002; Pochter et al., 2011; Shleyfman et al., 2015; Sievers et al., 2015; Sievers; Winterer et al.; Röger et al., 2018; Sievers et al., 2019; Fišer et al., 2019). Invariance of the value function for a Markov Decision Process (MDP) with symmetry has been shown by Zinkevich and Balch (2001), while Narayanamurthy and Ravindran (2008) proved that finding exact symmetry in MDPs is graph-isomorphismm complete. However, classical planning algorithms like A have a fundamental issue with exploiting symme- tries. They construct equivalence classes of symmetric states, which explicitly represent states and introduce symmetry breaking. As a result, they are intractable (NP-hard) in maintaining symmetries in trajectory rollout and forward search (for large state spaces and symmetry groups) and are incom- patible with differentiable pipelines for representation learning. This limitation hinders their wider applications in reinforcement learning (RL) and robotics.
State abstraction for detecting symmetries. Coarsest state abstraction aggregates all symmetric states into equivalence classes, studied in MDP homomorphisms and bisimulation (Ravindran and Barto, 2004; Ferns et al., 2004; Li et al., 2006). However, they require perfect MDP dynamics and do not scale up well, typically because of the complexity in maintaining abstraction mappings (homomorphisms) and abstracted MDPs. van der Pol et al. (2020b) integrate symmetry into model- free RL based on MDP homomorphisms (Ravindran and Barto, 2004) and motivate us to consider planning. Park et al. (2022) learn equivariant transition models, but do not consider planning. Ad- ditionally, the typical formulation of symmetric MDPs in (Ravindran and Barto, 2004; van der Pol al., 2020a; Zhao al., 2022) is slightly different from our formulation here: we consider sym-
2

---

Published as a conference paper at ICLR 2023
metry between MDPs (rotated maps), instead of within single MDP. Thus, the reward or transition function additionally depends on map imput, as further discussed in Appendix B.2.
Symmetries and equivariance in deep learning. Equivariant neural networks used incor- porate symmetry in supervised learning for different domains (c.g., grids and spheres), symmetry groups (e.g., translations and rotations), and group representations (Bronstein et al., 2021 Cohen and Welling (2016b) introduce G-CNNs, followed by Steerable CNNs (Cohen and Welling, 2016a) which generalizes from scalar feature fields vector fields with induced representations. Kondor and Trivedi (2018); Cohen et al. (2020) study the theory of equivariant maps and convolutions. Weiler and Cesa (2021) propose to solve kernel constraints under arbitrary representations for E(2 and its subgroups by decomposing into irreducible representations, named E(2)-CNN.
Differentiable planning. Our pipeline is based on learning to plan in neural network in a differen- tiable manner. Value iteration networks (VIN) (Tamar et al., 2016b) is a representative approach that performs value iteration using convolution on lattice grids, and has been further extended (Niu al., 2017; Lee et al., 2018; Chaplot et al., 2021; Deac et al., 2021; Zhao et al., 2023), Other than using convolutional networks, works on integrating learning and planning into differentiable networks in- clude Oh et al. (2017); Karkus et al. (2017); Weber et al. (2018); Srinivas et al. (2018); Schrittwieser et al. (2019); Amos and Yarats (2019); Wang and Ba (2019); Guez et al. (2019): Hafner et al. (2020); Pong et al. (2018): 8): Clavera et al. (2020). On the theoretical side, Grimm et al. (2020; 2021) propose to understand the differentiable planning algorithms from a value equivalence perspective.
BACKGROUND
Pg
X2
Markov decision processes. We model the path- planning problem as Markov decision process (MDP) (Sutton and Barto, 2018). An MDP is a 5-tuple M (S, P. R, 7), with state space S, action space + Rt, reward function R S x A R. and discount fac- + and E represent expected future returns. The core component behind dynamic programming (DP)- based algorithms in reinforcement learning is the Bellman (optimality) equation (Sutton and Barto, 2018): (*(s)= max, R(s,a) a) + y E. P(s'|s, a) (s'). Value iteration is an instance of DP to solve MDPs, which iteratively ap- plies the Bellman (optimality) operator until convergence.
Spadia
Conatrut
wirh//
A, transition probability function P S
2
tor 7 E [0.1]. Value functions V.
Q S
P
Ru
Figure 2: (Left) Construction of spatial MDPs from path-planning problems, abling the use of G-invariant transition mod- els. (Right) A demonstration how an ac- tion (arrow in red circle) is rotated when a map is rotated.
Path planning. The objective of the path path-planning prob- lem is find optimal actions from every location to nav- igate to the target in shortest time. However, the original path-planning problem is not equivariant under translation due to obstacles. VINs (Tamar et al., 2016a) implicitly construct an equivalent problem with an equivariant transition function, thus CNNs can be used to inject translation equiv- ariance. We visualize the construction of an equivalent "spatial MDP" in Figure 2 (left), where the key idea is to encode obstacle information in the transition function from the map (top left) into the reward function in the constructed spatial MDP (bottom right) as "trap" states with -oc reward. functionn
Further details about this construction are in Appendices E.1 and E.3. In Figure (right), we pro- vide a visualization of the representation (r) of a rotation r of ( 90", and how an action (arrow) is rotated ( 90" accordingly.
Value Iteration Network. Tamar et al. (201 6a) proposed Value Iteration Networks (VINs) that use a convolutional network to parameterize value iteration. It jointly learns in a latent MDP on 2D grid, which has the latent reward function R 2 - RIAI and value function V z² + R, and applies value iteration on that MDP:
Ra. J' +
W.V.
max Qa...
(I)
The first equation can be written as: layer Conv2D has parameters V.
Q(K) = Ra (CCoo22((VV())) WV) where the 2D convolution
3

---

Published as a conference paper at ICLR 2023
M
Q
Vi
Qu-1
A
Rig Ra
L
C
Pa Hovia
(deer And
PM rrepttr
a
M
Qe-1
9..4
Figure 3: The commutative diagram of Symmetric Value Iteration Network (SymVIN), Every row is a full computation graph of VIN. Every column rotates the field by à 90".
We intentionally omit the details on equivariant networks and instead focus on the core idea of integrating symmetry with equivariant networks. We present the necessary group theory background in Appendix C and our full framework and theory in Appendices D and E.
4 METHOD: INTEGRATING SYMMETRY INTO PLANNING BY CONVOLUTION
This section presents algorithmic framework that can provably leverage the inherent symmetry of the path-planning problem in a differentiable manner. To make our approach more accessible, first introduce Value Iteration Networks (VINs) (Tamar et al., 2016a) the foundation for algorithm: Symmetric VIN. In the next section, we provide an explanation for why we make this choice and introduce further theoretical guarantees how exploit symmetry.
How to inject symmetry? VIN uses regular 2D convolutions (Eq. 1), which has translation equiv- ariance (Cohen and Welling, 2016b; Kondor and Trivedi, 2018). More concretely, VIN will output the same value function for the same map patches, up to 2D translation. Characterization of transla- tion equivariance requires a different mechanism and does not decrease the search space nor reduce a path-planning MDP to an easier problem. We provide complete description in Appendix E.
Beyond translation, we are more interested in rotation and reflection symmetries. Intuitively, as shown in Figure 1, if we find the optimal solution to map, it automatically generalizes the solution 8 transformed maps (4 rotations times reflections, including identity transformation). This can be characterized by equivariance of planning algorithm P Plan, such as value iteration VI: g.Plan(M. = P (g), where M maze map, and the symmetry group D4 under which 2D grids are invariant.
More importantly, symmetry also helps training of differentiable planning. Intuitively, symmetry in path planning poses additional constraints its search space: if the goal is in the north, go up; if in the east, go right. In other words, the knowledge can be shared between symmetric cases; the path- planning problem is effectively reduced by symmetry to smaller problem. This property also be depicted by equivariance of Bellman operators T, or one step of value iteration: V = Tig. Vo]. If we use (I( to denote applying Bellman operators arbitrary initialization until convergence T [V)). value iteration is also equivariant:
g. V = g. g. T [V]] [=]== = 7 [o. VI(g.M)
(2)
We formally prove this equivariance in Theorem 5.1 in next section. In Theorem 5.2, we theoreti- cally show that value iteration in path planning is a specific type of convolution: steerable convo- lution (Cohen and Welling, 201 Before that, we first use this finding to present our pipeline on how to use Steerable CNNs (Cohen and Welling, 2016a) to integrate symmetry into path planning.
Pipeline: SymVIN. We have shown that VI is equivariant given symmetry path planning. We introduce our method Symmetric Value Iteration Network (SymVIN), that realizes equivariant VI integrating equivariance into VIN with respect to rotation and reflection, addition to translation. We use instance of Steerable CNN: E(2) Steerable CNNs (Weiler and Cesa, 2021) and their package e2enn for implementation, which is equivariant under D4 rotation and reflection, and also 72 translation the 2D grid 72, In practice, to inject symmetry into VIN, mainly need to
4

---

Published as a conference paper at ICLR 2023
replace the translation-equivanie Conv2D in Eq. 1 with SteerableConv:
@!a R SteerableConv((;WW
(1) max Q
(3)
We visualize the full pipeline in Figure 3. The map and goal are represented as signals M : Z {0, 1)2. It will be processed by another layer and output to the core value iteration loop. After some iterations, the final oulput will be used to predict the actions and compute cross-entropy loss.
Figure 3 highlights our injected equivariance property: if we rotate the map (from M to g) in order guarantee that the final policy function will also be equivalently rotated (from A to g.A), we shall guarantee that every transformation (e.g., Qs + Vx and V2 + Qk+1) in value iteration will also be equivariant, for every pair of columns. We formally justify our design in the section below and provide more technical details in Appendix E.
Symmetrie GPPN. Based on same spirit, we also implement a symmetric version of Gated Path-Planning Networks (GPPN (Lee et al., 2018)). GPPNs use LSTMs to alleviate the issue of unstable gradient in VINs. Although it does not strictly follow value iteration, still follows the spirit of steerable planning. Thus, we first obtained fully convolutional variant of GPPN from Kong (2022), called ConvGPPN, which incorporates translational equivariance. It replaces the MLPs in the original LSTM cell with convolutional layers, and then replaces convolutions with equivari- ant steerable convolutions, resulting in SymGPPN that is equivariant to translations, rotations, and reflections. See Appendix G.1 for details.
Extension:
5 THEORY: VALUE ITERATION IS STEERABLE CONVOLUTION
In the last section, we showed how to exploit symmetry in path planning by equivariance from convolution via intuition. The goal of this section to (1) connect the theoretical justification with the algorithmic design, and (2) provide intuition for the justification. Even through we focus a specific task, we hope that the underlying guidelines on integrating symmetry into planning are useful for broader planning algorithms and tasks as well. The complete version is in Appendix E.
Overview. There are numerous types of symmetry in various planning tasks. We study symmetry in path planning as example, because is straightforward planning problem, and its solutions have been intensively studied in robotics and artificial intelligence (La Valle, 2006; Sutton and Barto, 2018). However, even for this problem, symmetry has not been effectively exploited in existing plan- ning algorithms, such as Dijkstra's ' algorithm, A or RRT, because it is NP-hard to find symmetric states (Narayanamurthy and Ravindran, 2008).
Why VINebased planners? There are two reasons for choosing value-based planning methods.
The expected-value operator in value iteration Σ. P(s' |s, a) (s') is (1) linear in the value function and (2) equivariant (shown in Theorem 5.1). Cohen et al. (2020) showed that any linear
equivariant operator (on homogeneous spaces, e.g., 2D grid) is group-convolution operator.
2. Value iteration, using the Bellman (optimality) operator, consists of only maps between signals (stcerable fields) over 2 (c.g., value map and transition function map). This allows to inject symmetry by enforcing equivariance to those maps. Taking Figure as an example, the 4 corner states are symmetric under transformations
Da. Equivariance enforces those 4 states to have the same value if we rotate or flip the map. This avoids the need to find if a new state is symmetric to any existing state, which is shown to be NP-hard (Narayanamurthy and Ravindran, 2008).
Our framework for integrating symmetry applics any value-based planner with the above prop- erties. We found that VIN is conceptually the simplest differentiable planning algorithm that meets these criteria, hence our decision to focus primarily on VIN and its variants.
Symmetry from tasks. we want to exploit inherent symmetry in a task to improve planning, there major steps: (1) characterize the symmetry in the task, and (2) incorporate the corresponding symmetry into the planning algorithm. The theorctical results in Appendix E.2 mainly characterize the symmetry and direct us feasible planning algorithm.
5

---

Published as a conference paper at ICLR 2023
The symmetry in tasks for MDPs can be specified by the equivariance property of the transition and reward function, studied in Ravindran and Barto (2004); van der Pol et al. (2020b):
P(s' a) P(g.s' g.8,g.a), Vg E G, Vs, Ras (s, = (g.,(g.8.g.))) Ag E G, Vs,
(4)
(5)
Note that how the group G acts states and actions is called group representution, and is decided by the space S or A, which is discussed in Equation 19 in Appendix E.2. We emphasize that the equivariance property of the reward function is different from prior work (Ravindran and Barto, 2004; van der Pol et al., 2020b): in our case, the reward function encodes obstacles as well, and thus depends map input M. Intuitively, using Figure in order to find the correct original reward R before rotation, the imput map M must also be rotated g.M. See Appendix E for more details.
example, if position is rotated to g.s,
Symmetry into planning. As for exploiting the symmetry in planning algorithms, we focus on value iteration and the VIN algorithm. We first prove in Theorem 5.1 that value iteration for path planning respects the equivariance property, motivating us to incorporate symmetry with equivariance.
Theorem 5.1 (informal). If transition is G-invariant, expected-value operator Σ P(s') s, Vs')) and value iteration are equivariant under translation, rotation, reflection on the 2D grid.
We visualize the equivariance of the central value- update step R+yP**V, V/ in Figure 4. The upper row is a value field Vj and its rotated version g. Vk, and the lower row is for Q-value fields Q& and g.Qus- The diagram shows that, if we input a rotated value g. Vs, the output R + 7P g.Vi Vv is guaranteed to be equal to rotated Q-field g.Qu. Additionally, rotating Q- field g.Qu has two components: (1) spatially rotat- ing each grid (a feature channel for an action Q(, a)) and (2) cyclically permuting the channels (black ar- rows). The red dashed line points out how a specific grid of a Q-value grid Qu(-, South) got rotated and permuted. We discuss the theoretical guarantees in Theorem 5.1 and provide full proofs in Appendix F.
Vi
V.
90'
Rt
Value Update
D
90'
However, while the first theorem provides intuition, is inadequate since it only shows the equivari- ance property for scalar-valued transition probabil- ities and value functions, and does not address the implementation of VINs with multiple feature chan- nels CNNs. To address this gap, the next theorem further proves that value iteration is general form of steerable convolution, motivating the use of steerable CNNs by Cohen and Welling (2016a) to replace regular CNNs in VIN. This is related to Cohen et al. (2020) that proves steerable convolution is the most general linear equivariant map homogeneous spaces.
Q
g. Q
Figure 4: Commutative diagram of single step of value update, showing equivariance under rota- tions. Each grid in the Q-value field corresponds to all the values Q(, a) of single action
Theorem 5.2 (informal). If transition G-invariant, the expected-value operator expressible steerable convolution k, which is equivariant under translation, rotation, and reflection on 2D grid. Thus, value iteration (with max, +. x) a form of steerable CNN (Cohen and Welling, 2016a).
We provide complete version of the framework in Section E and the proofs in Section F. This justifies why we should use Steerable CNN (Cohen and Welling, 2016 the VI itself is composed of steerable convolution and additional operations (max, +, x). With equivariance, the value function z IR is D4-invariant, thus the planning is effectively done the quotient space reduced by D4-
Summary. We study how to inject symmetry into VIN for (2D) path planning, and expect the task- specific technical details are useful for two types of readers. (i) Using VIN. If one uses VIN for differentiable planning, the resulting algorithms SymVIN or SymGPPN can be plug-in alternative, as part of a larger end-to-end system. Our framework generalizes the idea behind VINs and enables us understand its applicability and restrictions. (ii) Studying path planning. The proposed frame- work characterizes the symmetry in path planning, so it is possible to apply the underlying ideas to other domains. For example, it is possible to extend to higher-dimensionall continuous Fuclidean

---

Published as a conference paper at ICLR 2023
Visualized Panoramic L
Workspace
To C-space
End-to-end
(4 Directions)
Learned
dggorthm))
2D Novigation and Visual Novigction
Workspace Manibulation and Cuppace Monipulation
Figure 5: (Left) We randomly generate occupancy grid z {0, 1} for 2D navigation. For visual naviga- tion. each position provides
32 3 egocentric panoramic RGB images 4 directions z² I2 32x3 One location is visualized. (Right) The top-down view (left) is the workspace of a 2-DOF manipulation task. For workspace manipulation. it is converted by mapper layer to configuration space, shown in the right subfigure. For C-space manipulation, the ground-truth C-space is provided to the planner.
location
spaces or spatial graphs (Weiler et al., 2018; Brandstetter et al., 2021). Additionally, we emphasize that the symmetry in spatial MDPs is different from symmetric MDPs (Zinkevich and Balch, 2001; Ravindran and Barto, 2004; van der Pol et al., 2020a), since our reward function not G-invariant (if conditioning on obstacles). We further discuss this in Appendices B.2 and E.4.
EXPERIMENTS
We experiment with VIN, GPPN and our SymPlan methods on four path-planning tasks, using both given and learned maps. Additional experiments and ablation studies are in Appendix H.
Environments and datasets. We demonstrate the idea in four path-planning tasks: (1) 2D navi- gation, (2) visual navigation, (3) 2 degrees of freedom (2-DOF) configuration-space (C-space) manipulation, and (4) 2-DOF workspace manipulation. For each task, we consider using ei- ther given (2D navigation and 2-DOF configuration-spac manipulation) navigation and 2-DOF workspace manipulation). In the latter case, the planner needs to jointly learn mapper that converts egocentric panoramic images (visual navigation) (workspace manipulation) into a map that the planners can operate on, as in (Lee et al., 2018; Chaplot al., 2021). In both cases, randomly generate training, validation and test data of 10K /2K 222/22//K / maps for all map sizes, to demonstrate data efficiency and generalization ability of symmetrie plan- ning. Due the small dataset sizes, test maps are unlikely to be symmetric to the training maps by any transformation from the symmetry groups G. For all environments, the planning domain is the 2D regular grid as VIN, GPPN and SPT S Ω Z and the action space is move in 4 O directions'' A (north, west, south, east).
learned maps (visual
workspace states
Methods: planner networks. We compare five planning methods, two of which are our Sym- Plan methods. Our two equivariant methods are based on Value Iteration Networks (VIN, (Tamar al., 6a)) and Gated Path Planning Networks (GPPN, (Lee al., 2018)). Our equivariant ver- sion of VIN is named SymVIN. For GPPN, first obtained a fully convolutional version, named ConyGPPN (Kong, 2022), and furthermore obtain SymGPPN by replacing standard convolutions with steerable CNNs. All methods use (equivariant) convolutions with circular padding to plan in configuration spaces for the manipulation tasks, except GPPN which is not fully convolutional.
Training and evaluation. We report success rate and training curves over 3 seeds. The training process (on given maps) follows (Tamar et al., 201 6a; Lee et al., 2018), where we train 30 epochs with batch size 32, and kernel size F 3 by default. The default batch size is 32. GPPN variants need smaller number because LSTM consumes much more memory.
6.1 PLANNING ON GIVEN MAPS
Environmental setup. In the 2D navigation task, the map and goal are randomly generated, where the map size is (15, 28, 50]. In 2-DOF manipulation in configuration space, adopt the setting
Note that the MDP action space A needs to be compatible with the group action G x A + A. Since the E2CNN package (Weiler and Cesa, 2021) uses counterclockwise rotations as generators for rotation groups Cn, the action space needs be counterclackwise O

---

Published as a conference paper at ICLR 2023
Table Averaged test success rate (%) for using 10K/2K/2K dataset for all four types of tasks.
Method
Navigation
Manipulation
(10K Data)
15
28 28
50 x 50
Visual
18 18
36 x 36
Workspace
VIN
66.97
67.57 98.14
$7.92
50.83
77.82 99.98
84.32 99.36
80.44
VIN
98.99
86.20
95.50
91.10
GPPN
96.36
95.77
91.84
93.13
2.62
1.68
3.67
ConyGPPN
99.75
99.09
97.21
98.55
99.98
99.95
89.88
SymGPPN
99.98
99.86
99.49
99.78
100.00
99.99
90,50
in Chaplot et al. (2021) and train networks to take as input "maps" in configuration space, repre- sented by the state of the two manipulator joints. We randomly generate to 5 obstacles in the manipulator workspace. Then the 2 degrec-of-freede freedom (DOF) configuration space is constructed from the workspace and discretized into 2D grids with sizes {18, 36), corresponding to bins 20 and 10 respectively. All methods are trained using the same network size, where for the equivariant versions, we use regular representations for all layers, which has size Da = 8. We keep the same parameters for all methods, so all equivariant convolution layers with regular representations will have higher embedding sizes. Due to memory constraints, we use K = 30 iterations for 2D maze navigation, and K 27 for manipulation. We use kernel sizes {3, 5, 5) for m {15, 28, 50) navigation, and F = [3, 5} for m = {18, 36} manipulation.
Results. We show the averaged test results for both 2D navigation and C-space manipulation tasks on generalizing unseen maps (Table 1) and the train- ing curves for 2D navigation (Figure 6). For VIN- based methods, SymVIN learns much faster than standard VIN and achieves almost perfect asymp- totic performance. For GPPN-based methods, we found the translation-oquiivriann convolutional vari- ant ConyGPPN works better than the original one in (Lee al., 2018), especially in learning speed. SIMVIN fluctuates in some runs. We observe that the first epoch has abnormally high loss in most runs and quickly goes down in the second or third epoch, which indicates effects from initialization. SymGPPN further boosts ConyGPPN and outperforms all other methods and does not experience variance. One exception is that standard GPPN learns poorly in C-space manipulation. For GPPN, the added circular padding in the convolution encoder leads to a gradient-vanishing problem.
Successtul Rate
0.8
0.8
0.4
SymVIN
0.2
ConVGPPN
SymGPPN
5
15
25
Figure 6: Training curves on 2D navigation with 10K of 15 x 15 maps. Faded areas indi- cate standard error.
Additionally, we found that using regular representations (for D4 or C4) for state value V z CV (and for Q-values) works better than using trivial representations. This is counterintuitive since we expect the value to be scalar 2² + R. One reason is that switching between regular (for Q) and trivial (for V) representations introduces an unnecessary bottleneck. Depending on the choice of representations, we implement different max- max-pooling, with details in Appendix G.2. We also empirically found using FC only in the final layer QK A helps stabilize the training. The ablation study on this and more are described in Appendix H.
Remark. Our two symmetric planners are both significantly better than their standard counterparts. Notably, we did not include any symmetric maps in the test data, which symmetric planners would perform much better on. There several potential sources of advantages: (1) SymPlan allows parameter sharing across positions and maps and implicitly enables planning in a reduced space: every (s, a, s') generalizes to (g.s, g.a.g....... g.s') for any g G. (2) thus it uses training data more efficiently, (3) it reduces the hypothesis-class size and facilitates generalization to unseen maps.
6.2 PLANNING ON LEARNED MAPS: SIMULTANEOUSIA PLANNING AND MAPPING
Environmental setup. For visual navigation, randomly generate maps using the same strategy as before, and then render four egocentric panoramic views for each location from 3D environments produced with Gym-Mini Sorr-ddiWoolld
Boisvert, 2018) 8), which can generate 3D mazes with any
8

---

Published as a conference paper at ICLR 2023
layout. For m X m maps, all egocentric views for map represented by m x m X 4 RGB images. For workspace manipulation, we randomly generate 0 to 5 obstacles workspace as before. We mapper network to convert the 96 x 96 workspace (image of obstacles) to the m x m 2 degree- of-freedom (DOF) configuration space (2D occupancy grid). In both environments, the setup is similar Section 6.1, except here we only use map size 15 for visual navigation (training for 100 epochs), and map size m 18 for workspace manipulation (training for 30 epochs).
mapper networks and setup. For visual navigation, we implemented an equivariant mapper network based on Lee et al. (2018). The mapper network converts every image into a 256- dimensional embedding m x m
Methods:
256 and then predicts the map layout m X m X For workspace manipulation, we use U-net (Ronneberger et al., 2015) with residual-connectin (He et al., 2015) as a mapper. For more training details, see Appendix H.
Results. The results are shown in Table 1, under the columns "Visual" (navigation, 15 x 15) and "Workspace" (manipulation, 18 x 18). In visual navigation, the trends are similar to the 2D case: our two symmetric planners both train much faster. Besides standard VIN, all approaches eventually converge to near-optimal success rates during training (around 95%). but the validation and test results show large performance gaps (see Figure 23 in Appendix HD). SymGPPN has almost no generalization gap. whereas VIN does not generalize well to new 3D visual navigation environments. SymVIN significantly improves test success rate and is comparable with GPPN. Since the input is raw images and a mapper is learned end-to-end, it is potentially a source of generalization gap for some approaches. In workspace manipulation, the results similar to findings in C-space manipulation, but our advantage over baselines were smaller. We found that the mapper network was a bottleneck, since the mapping for obstacles from workspace C-space is non-trivial to learn.
6.3 RESULTS ON GENERALIZZTION TO LARGER MAPS
To demonstrate the generalization advantage of our methods, all methods are trained small maps and tested on larger maps. All methods are trained on 15 x 15 with K 30. Then we test all methods on map size 15 x 15 through 99 x 99, averaging over 3 seeds (3 model checkpoints) for each method and 1000 maps for each map size. Iterations K were set to v2M, where M is the test map size (x-axis). The results are shown in Figure 7.
Results. SYM generalizes better than VIN, although the variance is greater. GPPN tends to diverge for larger values of K. ConvGPPN converges, but it fluctuates for
different seeds. SymGPPN shows the best generaliza- tion and has small variance. In conclusion, Sym VIN and SymGPPN generalize better to different map sizes, com- pared to all non-equivariant baselines.
0.6
0.4
9ymvV
dPPN
SymS7Ph
Remark, In summary, our results show that the Sym- Plan models demonstrate end-to-end planning and learn- ing ability, potentially enabling further applications to other tasks as a differentiable component for planning. Additional results and ablation studies are in Appendix H.
Figure 7: Results for testing on larger maps, when trained on size 15 map. Our methods outperform all baselines.
7 DISCUSSION
In this work, we study the symmetry in the 2D path path-planning problem, and build a framework using the theory of steerable CNNs to prove that value iteration in path planning is actually a form of steer- able CNN (on 2D grids). Motivated by our theory, we proposed two symmetric planning algorithms that provided significant empirical improvements in several path-planning domains. Although our focus in this paper has been on 22, our framework higher-dimensional or even continuous Euclidean spaces (Weiler et al., 2018; Brandstetter et al., 2021), by using equivariant operations on steerable feature fields (such steerable convolutions, pooling, and point-wise non-linearities) from steerable CNNs. We hope that our SymPlan frame- work, along with the design of practical symmetric planning algorithms, can provide a new pathway for integrating symmetry into differentiable planning.
potentially generalize to path planning

---

Published as a conference paper at ICLR 2023
8 ACKNOWLEDGEMENT
This work was supported by NSF Grants #2107256 and #2134178. R. Walters is supported by The Roux Institute and the Harold Alfond Foundation. We also thank the audience from previous poster and talk presentations for helpful discussions and anonymous reviewers for useful feedback.
REPRODUCIBILITY STATEMENT
We provide additional details in the appendix. We also plan to open source the codebase. We briefly outline the appendix below.
1. Additional Discussion
2. Background: Technical background and concepts steerable CNNs and group CNNs
3. Method: provide full details how to reproduce it
4. Theory/Framework: we provide the complete version of the theory statements
5. Proofs: this includes all proofs
6. Experiment/ Environment Implementation details: useful details for reproducibility
7. Additional results
10

---

Published as a conference paper at ICLR 2023
REFERENCES
Richard S. Sutton and Andrew G. Barto. Reinforcement learning: an introduction. Adaptive com- putation and machine learning series. The MIT Press, Cambridge, Massachusetts, second edition edition, 2018, ISBN 978-0-262-03924-6
Lihong Li, Thomas Walsh, and M. Littman. Towards a Unified Theory of State Abstraction for MDPs. In AI&M, 2006.
Balaraman Ravindran and Andrew G Barto. An algebraic approach to abstraction reinforcement learning. PhD thesis, University of Massachusetts Amherst, 2004.
Maria Fox and Derek Long. Extending the exploitation of symmetries in planning. In Proceedings of AIPS'OR 02, pages 83-91, 2002.
Maria Fox and Derek Long. The Detection and Exploitation of Symmetry in Planning Problems. In In IJCAI, pages 956 961. Morgan Kaufmann, 1999.
Nir Pochter, Aviv Zohar, and Jeffrey S. Rosenschein. Exploiting Problem Symmetries in State- Based Planners. In Twenty-Fifth AAAI Conference on Artificial Intelligence, August 2011. URL https: / www. aaai. org/ ocs/ index..n.. wwwp..ppp/eiiei.....mmpppppppp AAAAAAAAAA1/// htt.aa//////pppeee////////3/
Martin Zinkevich and Tucker Balch. Symmetry in Markov decision processes and its implications for single agent and multi agent learning. In In Proceedings of the 18th International Conference on Machine Learning, pages 632-640, Morgan Kaufmann, 2001.
Shravan Matthur Narayamamurthy and Balaraman Ravindran. On the hardness of finding symme- tries in Markov decision processes. In Proceedings of the 25th international conference on Ma- chine learming ICML. '08, pages 688-699 Helsinki, Finland, 2008. ACM Press. ISBN 978-1- cfm?doide
60558-205-44 doi: 10/bkswe2. URL http: porttll.ll.....oo.o.o www.m.org//iiee 1390156. 1390243.
Arnab Kumar Mondal, Prathecksha Nair, and Kaleem Siddiqi. Group Equivariant Deep Reinforce- ment Learning. 2xXv:20007.03437 [es, stat), June 2020. URL http:://exxx...rrg////////////// 2007. 03437. arxiv: 2007.03437.
Elise van der Pol, Daniel E. Worrall, Herke van Hoof, Frans A. Oliehoek, and Max Welling. MDP Homomorphic Networks: Group Symmetries in Reinforcement Learning. arXiv:2006. 16908 [cs, stat], June 2020a. URL http:
http://b//20000 16908. arXiv: 1006.16908.
Dian Wang, Robin Walters, and Robert Platt. (Snattrm(5O////-SEguuviaaa Reinforcement Learn- ing. September 2021. URL httpt:p::///pee/vvviieee wtt.:/et///oteeeee
Michael M Bronstein, Joan Bruna, Taco Cohen, and Petar Veličković. Geometric deep learning: Grids, groups, graphs, geodesics, and gauges. arXiv preprint arXiv:2104.13478 2021a.
Taco Cohen, Mario Geiger, and Maurice Weiler. A General Theory of Equivariant CNNs on Ho- mogeneous Spaces. arXiv: 1111111000000 1.02017 [cs, stat], January 2020. URL
abs/1811. 02017. arXiv: 1811.02017.
Risi Kondor and Shubhendu Trivedi. On the Generalization of Equivariance and Convolution in Neural Networks the Action of Compact Groups. (at::880000360 [es, stat], November 2018. URL http:
rwwaaiivvvoooo http//aw.vv..oooaases/oo/es//.as 1802. 03690. arXiv: 1802.03690.
Taco Cohen and Max Welling. Steerable CNNs. November 2016a. URL https:// openreview woee:/orommiddd000eeee
Taco S. Cohen and Max Welling. Group Equivariant Convolutional Networks. arXiv: 1602.0060 [cs, stat], June 2016b. URL
www.aawwwww.....///a/a/a//////// 07576. arXiv: 1602.07576.
Maurice Weiler and Gabriele Cesa. General EqquiS--uuivvarian Steerrae [es, eess), April 2021. URL http://nt////aaaaoooooro/a////// 08251, arXiv: 1911.08251.
Steerable CNNs. arXiv:1911.082.

---

Published as a conference paper at ICLR 2023
Aviv Tamar, YI WU, Garrett Thomas, Sengey Levine, and Pieter Abbeel. Value Iteration Net- works. In Advances in Neural Information Processing Systems, volume 29. Curran Asso- ciates, Inc., 2016a. URL https: http:////pcccceedeeeeeee
httpp/ps....ppppe///////eeeeeeee Abst.ratt....................... html.
Lisa Lee, Emilio Parisotto, Devendra Singh Chaplot, Eric Xing, and Ruslan Salakhutdinov, Gated Path Planning Networks. arXiv: 1806, 06408 [cs, star], June 2018. URL abs/ 06408,
arXiv: 1806.06408.
Alexander Shleyfman, Michael Katz, Malte Helmert, Silvan Sievers, and Martin Wehrle. Heuris- tics and Symmetries in Classical Planning. Proceedings of the AAAI Conference Artifi- cial Intelligence, 29(1), March 2015. ISSN 2374-3468. doi: 10/gq5m5s. URL https: //ojs. wwai.aaa..........a...aaa....... org/ index. php/ /ttp/////aaareeeeleeeeeeae/a/e// 9649. Number: 1.
Silvan Sievers, Martin Wehrle, Malte Helmert, and Michael Katz. An Empirical Case Study on Symmetry Handling in Cost-Optimal Planning as Heuristic Search. Steffen Hölldobler, Rafael Peñaloza, and Sebastian Rudolph, editors, KI 2015: Advances in Artificial Intelligence, volume 9324, pages 160.180. Springer International Publishing, Cham, 2015. ISBN 978-3-319-24489 11833-319-24489111 doi: 10.10007/78-3 3-119-312-23399 13. URL http: /link. springer. 13. Series Title: Lecture Notes in Computer Sci-
ence.
Silvan Sievers. Structural Symmetries of the Lifted Representation of Classical Planning Tasks. page 8.
Dominik Winterer, Martin Wehrle, and Michael Katz. Structural Symmetries for Fully Observable Nondeterministic Planning. page 7.
Gabriele Röger, Silvan Sievers, and Michael Katz. Symmetry-Based Task Reduction for Relaxed Reachability Analysis. In Twenty-Eighth International Conference on Automated Planning and Scheduling, June 2018. URL https: //aaiiaaaaa/a org/ oos///c///////// indin. wwtp:/pppp..../pp//p ICAPS18/
paper/view/17772 17772.
Silvan Sievers, Gabriele Röger, Martin Wehrle, and Michael Katz. Theoretical Foundations for Structural Symmetries of Lifted PDDL Tasks. Proceedings of the International Conference Automated Planning and Scheduling, 29:446-454, 2019. ISSN 2334-0843, doi: 10/g45m5t. URL
https://ojs aaai. org/ index.
Daniel Fišer, Álvaro Torralba, and Alexander Shleyfman. Operator Mutexes and Symmetries for Simplifying Planning Tasks. Proceedings of the AAAI Conference on Artificial Intelligence, 33 (01):7586-7593, July 2019. ISSN 2374-3468. doi: 10/ghkkbq, URL https: //tjs////ss..//// aaai. org/index php.in.eepppppppp.pppppppppppppt www.///aateee//eaaaw.aaaaaaaaaee 4751. Number: 01.
N. Ferns, P. Panangaden, and Doina Precup. AAAI, 2004.
Metrics for Finite Markov Decision Processes. In
Elise van der Pol, Daniel Worrall, Herke van Hoof, Frans Olichock, and Max Welling. Mdp homo- morphic networks: Group symmetries in reinforeement learning. Advances Neural Information Processing Systems, 33, 2020b.
Jung Yeon Park, Ondrej Biza, Linfeng Zhao, Jan Willem van de Meent, and Robin Walters. Learning Symmetric Embeddings for Equivariant World Models, arXiv:2204. 11371 [cs], April 2022. URL arXiv: 2204.11371.
http:// /ttp:///aiivvv//ii org/ 2tt.//224.1................1..1.
Linfeng Zhao, Lingzhi Kong, Robin Walters, and Lawson L. S. Wong. Toward Compositional Generalization in Obiect-Oriented World Modeling. In ICML 2022, April 2022. URL http:
//arxiv. wrt. abs /2044 13661. arXiv: 2204.13661.
Michael M. Bronstein, Joan Bruna, Taco Cohen, and Petar Veličković. Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gauges. arXiv:2104.1 122004114777 13478 [cs, stat], April 2021b. URL http:/ / /aaxivi..v..////a/ wogg/abs......o.ooooo /200 13478. arXiv: 1104.13478.
Aviy Tamar, Yi Wu, Garrett Thomas, Sergey Levine, and Pieter Abbeel. Value iteration networks. arXiv preprint arxiv: 1602.62867. 2016h.
12

---

Published as a conference paper at ICLR 2023
Sufeng Niu, Siheng Chen, Hanyu Guo, Colin Targonski, Melissa C. Smith, and Jelena Kovačević. Generalized Value Iteration Networks: Life Beyond Lattices. arXiv: 02416 [cs], October 2017. URL http: http//axvvvv.////////////e org/ httpg/////////.////////// 02416. arXiv: 1706.02416.
Devendra Singh Chaplot, Deepak Pathak, and Jitendra Malik. Differentiable Spatial Planning using Transformers. arX1v:2112.01010 [cs]. December 2021. URL sttp://arxiv.. org. abs/
arXiv: 2112.01010.
Andreea Deac, Petar Veličković, Ognjen Milinković, Pierre-Luee Bacon, Jian Tang, and Mladen Nikolić. Neural Algorithmic Reasoners are Implicit Planners. October 2021. URL https:
Linfeng Zhao, Huazhe Xu, and Lawson L. S. Wong. Scaling up and Stabilizing Differen- tiable Planning with Implicit Differentiation. In ICLR 2023, February 2023, URL https: //openzeview.nete wwwss//ormmmiddddee 4MoHf32.
Junhyuk Oh, Satinder Singh, and Honglak Lee. Value Prediction Network. arXiv: -XXiv:777.00397 [cs], 03497, arXiv: 1707.03497,
November 2017. URL http: /ttaaaiiviv////////a//.../a/aata wwt./org/////ooooooo...oeo.
Peter Karkus, David Hsu, and Wee Sun Lee. QMDP-Net: Deep Learning for Planning under Partial Observability. 1703.06700069 [cs, stat], November 2017. URL http: /tttpi////xvv..//// abs/1703. 06692.
arXiv: 1703.06692.
Théophane Weber, Sébastien Racanière, David P. Reichert, Lars Buesing, Arthur Guez, Danilo Jimenez Rezende, Adria Puigdomênech Badia, Oriol Vinyals, Nicolas Heess, Yujia Li, Razvan Pascanu, Peter Battaglia, Demis Hassabis, David Silver, and Daan Wierstra. Imagination- Augmented Agents for Deep Reinforcement Learning. /7Xiv:/707.0000 .6200 [cs, stat], February 2018. URL http: / rxiv. www...ig..aa.aaaa..ao.aoo org/abs 1100717000000 111777062200 06203. arXiv: 1707.06203.
Aravind Srinivas, Allan Jabri, Picter Abbeel, Sergey Levine, and Chelsea Finn. Universal Planning Networks. arXiv: /004.000000000 004.00645 [cs, stat), April 2018. URL WRLLhttpp//aavvv.orgg////// 00645. arXiv: 1804.00645.
Julian Schrittwieser, Ioannis Antonoglou, Thomas Hubert, Karen Simonyan, Laurent Sifre, Si- mon Schmitt, Arthur Guez, Edward Lockhart, Demis Hassabis, Thore Graepel, Timothy Lill- icrap, and David Silver. Mastering Atari, Go, Chess and Shogi by Planning with a Learned Model. arXiv: 1911,08266 1.08265 [cs, staf], November 2019. URL http:://rxxv..rrg//// 1911. 08265.
arXiv: 1911.08265.
Brandon Amos and Denis Yarats. The Differentiable Cross-Entropy Method. September 2019. doi: 10/gg5m59. URL https:tpp/... //tp/v/vvvvv//////////////////v/ org/ abs / 1909. 12830v4.
Tingwu Wang and Jimmy Ba. Exploring Model-based Planning with Policy Networks. June 2019.
URL https:: //rxivv///xvv..a/ o abs /10/s//90..... 08649v1.
Arthur Guez, Mehdi Mirza, Karol Gregor, Rishabh Kabra, Sébastien Racanière, Théophane Weber, David Raposo, Adam Santoro, Laurent Orseau, Tom Eccles, Greg Wayne, David Silver, and Timothy Lillicrap. An investigation of model-free planning. 19Xi 03559 [cs, stat], May 2019. URL http:// htt//xxivv.o./ .rg/
1901. wwwooo/ggoooo abs /atps//91...0000 03559. arXiv: 1901.03559.
Danijar Hafner, Timothy Lillicrap, Jimmy Ba, and Mohammad Norouzi. Dream to Control: Learn- ing Behaviors by Latent Imagination. arXiv: [cs], March 2020. URL http: 01 1033, arXiv: 1912.01603.
/
Vitchyr Pong, Shixiang Gu, Murtaza Dalal, and Sergey Levine. Temporal Difference Models: Model-Free Deep RL for Model-Based Control. AxXiv:880200908 [cs], February 2018. URL http:://axiii wrgg abs// 802. 09081. arXiv: 1802.09081.
Ignasi Clavera, Violet Fu, and Pieter Abbeel. Audee--ugmeeteed Actor-Critic: Backpropagating through Paths. arXiv:2005.0 088622005.080688 [cs, stat], May 2020. URL http: /tto:///axiiv org/ abs/ 2005. 68068. arXiv: 0006008068.
13

---

Published as a conference paper at ICLR 2023
Christopher Grimm, André Barreto, Satinder Singh, and David Silver. The Value Equivalence Prin- ciple for Model-Based Reinforcement Learning. arXiv:2011.03506 [cs], November 2020. URL http://arxiv org/ abs 221112011.. 03506.
arXiv: 2011.03506.
Christopher Grimm, André Barreto, Gregory Farquhar, David Silver, and Satinder Singh. Proper Value Equivalence. arXiv:2106, 10316 [cs/. December 2021. URL
abs/2106. 10316.
arXiv: 2106.10316.
Lingzhi Kong. Integrating implicit deep learning with Value Iteration Networks. Master's thesis, Northeastern University, 2022.
Steven M. LaValle. Planning Algorithms. Cambridge University Press, May 2006. ISBN 978-1- 139-45517-6.
Maurice Weiler, M. Geiger, M. Welling, Wouter Boomsma, and Taco Cohen. 3D Stecrable CNNs: Learning Rotationally Equivariant Features in Volumetric Data. In NeurIPS, 2018.
Johannes Brandstetter, Rob Hesselink, Elise van der Pol, Erik J. Bekkers, and Max Welling. Geomet- ric and Physical Quantities Improve E(3) Equivariant Message Passing. arXiv:2110.02000 /cs, 02905.
star), December 2021. URL htwp: /ttw//ixivvvv.////// http://wwxwv....aaaa//o//e////se wwtp:////////b///o///ooo//
arXiv: 21 10.00299
Maxime Bhevvlier-Boisver..
https: Boisvert. Miniworld: Minimalistic 3d environment for rl & robotics research.
com com/maximecb wtwxxx/mmecb/mmmmmimiwoooooe /gymmmmmmmwooleee 2018.
Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomed- ical image segmentation. CoRR, 1ess1550,,457,, 2015. URL http::/aaxxi..orgg//////////////
1505. 04597.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep learnigg for image recog- nition. CoRR, abs/1 hts/1512.00385. 2015. URL http: http:p://arrdv..rrg/////5/.22.. 0338.
Taco S. Cohen, Mario Geiger, Jonas Koehler, and Max Welling. Spherical CNNs. 1011: [cs, stat), February 2018. URL http:///aa.....ooo///////001111/ 1801.10130.
arXiv:
Mario Geiger and Tess Smidt. e3nn: Euclidean Neural Networks. Technical Report
urXiv:2207.09455 arXiv, July 2022. URL arXiv:2207.0944 099453 [es] type: article.
Cohen. Equivariant convolutional networks. 2021. URL https::/darr..va...
14

---

Published as a conference paper at ICLR 2023
CONTENTS
A Outline
16
Additional Discussion
16
B.1 Limitations and Extensions
16
B.2 The Considered Symmetry and Difference Existing Work
16
B.3 Additional Discussions
17
C Background: Equivariant Networks
17
C.I. Basics: Groups and Group Representations
17
C.2 Group Representations: Visual Understanding
19
C.3 Geometric Deep Learning..
19
C,4 Steerable CNNs
20
Symmetric Planning in Practice
21
D.1 Building Symmetric VIN
21
D.2 PyTorch-style pseudocode
23
Symmetric Planning Framework
24
E.1 Steerable Planning: Planning On Steerable Feature Fields
25
E,2
Symmetric Planning: Integrating Symmetry by Convolution
26
E.3 Details: Constructing Path planning in Neural Networks
27
E,4 Details: Understanding Symmetric Planning by Abstraction
28
E,5 Note: Augmented State
Symmetric Planning Framework: Proofs
29
F.1
Proof: equivariance of scalar-valued expected value operation
29
F2 Proof: expected value operator as steerable convolution
30
E3 Proof: equivariance of expected future value
31
F4 Proof: equivariance of steerable value iteration
32
G
Implementation Details
34
G.I Implementation of SymGPPN
................................
34
G.2
Implementation of max operation
34
H Experiment Details and Additional Results
34
H.I Environment Sctup
34
H.2 Building Mapper Networks
35
H.3 Training Setup....
37
H.4 Visualization of Learned Models
37
15

---

Published as a conference paper at ICLR 2023
H.5 SPL metric
38
H.6 Ablation Study
38
A
OUTLINE
We provide table of content above.
We omit technical details on symmetry and equivariant networks in the main paper and delay them here. Specifically, for readers interested in additional details on how use equivariant networks for symmetric planning, we recommend an order as follows: (1) Basics on group representations and equivariant networks in Section C.1. (2) Practice on building Sym VIN in Section D.1. (3) Detailed formulation on SymPlan in Section E.1 and Section E.2.
The rest technical sections provide additional reading materials for the readers interested in more in-depth account on studying symmetry in reinforcement learning and planning.
B ADDITIONAL DISCUSSION
B.1 LIMITATIONS AND EXTENSIONS
Assumption on known domain structure. As in VIN, although the framework of steerable plan- ning can potentially handle different domains, one important hidden assumption that the under- lying domain Q (state space), is known. In other words, we fix the structure of learned transition kernels p(s' 8, a) and estimate coefficients of it. One potential method is to use Transformers that learn attention weights to all states in S, which has been partially explored in SPT (Chaplot et al., 2021). ). Additionally, it is also possible to treat unknown MDPs as learned transition graphs, explored in XLVIN (Deac et al., 2021). We leave the consideration of symmetry in unknown underlying domains for future work.
The curse of dimensionality. The paradigm of steerable planning still requires full expansion in computing value iteration (opposite to campling-based. since we realize the symmetric planner using group equivariant convolutions (essentially summation integral). Convolutions high- dimensional space could suffer from the curse of dimensionality for higher dimensional domains, and vastly under-explored. This is a primary reason why need sampling-based planning algorithms. If the domain (state-action transition graph) is sparsely connected, value iteration can still scale up to higher dimensions. is also unclear either when steerable planning would fail, or how sampling based algorithms could be integrated with the symmetric planning paradigm.
B.2 THE CONSIDERED SYMMETRY AND DIFFERENCE TO EXISTING WORK
We need differentiate between two types of symmetry in MDPs. Let's take spatial graph as illustrative example to understand the potential symmetry from a higher level, which means that the nodes V in the graph have spatial coordinates Z R², Our 2D path planning is a special case of spatial graph, where the actions can only move to adjacent spatial nodes.
Let the graph denoted as G (V,E). E is the set of edges connecting two states with an action. One type of symmetry is the symmetry of the graph itself. For the grid case, it means that after D4 rotation or reflection, the map is unchanged.
Another type of symmetry comes from the isometries of the space. For spatial graph, we can rotate it freely in space, while the relative positions are unchanged. For our grid case, it is shown in the Figure 1 that rotating a map resulting in the rotated policy. However, the map policy itself can never be equal under any transformation D4-
In other words, the first type is symmetry within MDP (rely the property of the MDP itself M, or Aut(M)), and the second type is symmetry between MDPs (only rely on the property of the underlying spatial space Z or Aut AZ?))
Nevertheless, we could input map M and somehow treat symmetric states between MDPs as one state. See the proofs section for more details.
16

---

Published as a conference paper at ICLR 2023
B.3 ADDITIONAL DISCUSSIONS
Potential generalization to 3D space. The goal of our work is to integrate symmetry into differ- entiable planning. While there is rich branch in math and physies that studies representation theory, we use 2D settings to keep the representation part minimal and provide an example pipeline for us- ing equivariant networks to integrate symmetry. For 3D and other cases, the key is still equivariance, but they require more technical attention on the equivariant network side.
Representation theory has extensively studied 3D rotations. SO(3)-equivariant networks have been developed in the equivariant network field by researchers like Cohen et al. (2018) (Spherical CNNs) and Geiger and Smidt (2022) (2022, e3nn: Euclidean Neural Networks). -quivvavrnr net- works avoid gimbal lock by not predicting pose but only needing equivariance to SO(3). SO(3) elements only appear in solving equivariant kernel constraints.
Inference on realistic robotic maps. Our choice of small and toyish maps (100 x 100 smaller) is in line with prior work, such as VIN, GPPN, and SPT, which mainly experimented on 15 x 15 and 28 x 28 maps. While we recognize that in robotics planning, maps can be significantly larger, we believe that integrating symmetry into differentiable planning is an orthogonal topic with the scalability of differentiable planning algorithms. However, our visual navigation experiment shows that our method can learn from panoramic egocentric RGB images of all locations, and our generalization experiment demonstrates the potential of scalability as the model can generalize to larger maps, which has not been explored in prior work. In comparison with robotic navigation works like Active Neural SLAM, our approach aims to improve only the differentiable planning module, while is systematic pipeline with hierarchical architecture of global and local planners. Our Symmetric Planners could serve as an alternative to either the global or local planner, but we not comparing approach with the whole Active Neural SLAM pipeline. Additionally, Neural SLAM uses room-like maps, which have a different structure and require less planning horizon compared to maze-like maps.
Measure of efficiency of symmetry in path planning. In Appendix Section E.4, we demonstrate how symmetry can aid path planning and provide intuition for our approach. We show that the MDPs from rotated maps are related by MDP homomorphisms, which has been previously used in Ravindran and Barto (2004). This means that states related by D4 rotations/rreleectioss can be aggregated into one state. We inject this symmetry property into the Sym algorithm by enforcing Dy-invariance of the value function on Z², which equivalent to a function on the quotient space. The global and local equivariance to D4 enables our Symmetric Planning algorithms to effectively plan on the quotient/reduced MDP with a smaller state space. By local equivariance, we mean that every value iteration step is convolution (or other equivariant) operation and is equivariant to rotation/reflection (see Figure and 4), thus the search space in planning can be exponentially shrunk with respect the planning horizon. Although we haven't 't formally proven this the sample complexity side, intuitively, this gives |G] times smaller state space and sample complexity.
Equivariance's benefits in general supervised learning tasks still being explored. Recent work includes showing improved generalization bounds for group invariant/cquivarian deep networks, as demonstrated in the work of Sannai et al. (2021), Elesedy and Zaidi (2021), and Behboodi et al. (2022), among others.
C BACKGROUND: EQUIVARIANT NETWORKS
We omit technical details in the main paper and delay them here. This section introduces the back- ground on equivariant networks and representation theory. The first subsection covers necessary basics, while the rest subsections provide additional reading materials for the readers interested in more in-depth account on the preliminaries on studying symmetry reinforcement learning and planning.
C.I BASICS: GROUPS AND GROUP REPRESENTATIONS
Symmetry groups and equivarance. A symmetry group is defined as a set together with a binary composition map satisfying the axioms of associativity, identity, and inverse. (left) group action
17

---

Published as a conference paper at ICLR 2023
Pv
trivtal
P:
R+
Connitet
AR BF
R
Figure 8: (Left) Construction of spatial MDPs from path planning problems, enabling G-invariant transition. (Middle) The group acts feature field (MDP actions). We need to find the element the original field by f(r-1x), and also rotate the arrow by p(r), where r D4. We represent one-hot actions arrows (vector field, using Pau) for visualization. (Right) Equivariance of + Q in Bellman operator on feature fields, under 3 90 E C4 rotation, which visually explains Theo- rem 1. The example simulates VI for step (see red circles; minus signs omitted) with true transition P using O N-W-S-E actions. The Q-value field are for actions and can be viewed as either 2² a ((Cohen and Welling, 2016a; Weiler and Cesa, 2021)) 2² C4 + (on p4 group, (Cohen and Welling, 2016b)). Simplified figures are presented in the main paper.
of G set X' is defined the mapping (g, x) 9.2' which is compatible with composition. Given a function f :: - y and G acting on X' and V, then f is G-equivariant it commutes with group actions: g- (x: fggx), Vg E G, Ve X. In the special case the action y is trivial g-y = y, then f(x) = (ggxx holds, and we say f is G-invariant.
Group representations. We mainly use two groups: dihedral group D4 and cyclic group C4. The cyclic group of elements is C4 (r dihedral group D₃ (r, pd r²= 5² (sr)² = 1) includes both rotations r and reflections s, and has size D4 = 8. A group representation defines how group action transforms vector space G - S. These groups have three types of representations of our interest: trivial, regular, and quotient representations, see (Weiler and Cesa, 2021). The trivial representation Puiv maps each g E G and hence fixes all s E S. The regular representation Preg of C4 group sends each g E C4 to 4x permutation matrix that cyclically permutes a 4-element vector, such as a one-hot 4- direction action. The regular representation of D4 maps each element to an x 8 permutation matrix which does not act 4-direction actions, which requires the quotient representations (quotienting out 2r reflection part) and forming 4 4 permutation matrix. It is worth mentioning the standard representation of the eyclic groups, which are 2 x 2 rotation matrices, only used for visualization (Figure middle).
1), a symmetry group of rotating square. The
Steerable feature fields and Steerable CNNs. The concept of feature fields is used in (equivariant) CNNs (Bronstein et al., 2021a; a; Cohen et al., 2020; Kondor and Trivedi, 201 8; Cohen and Welling, 2016;;;; Weiler and Cesa, 2021). The pixels of an 2D RGB image : z
domain R$
Ω = 2² is a feature field. In steerable CNNs for 2D grid, features are formed as steerable feature fields f 2 RC that associate a C-dimensional feature vector fxx e R to each element on a base space, such as Z². Defined like this, we know how to transform steerable feature field and also the feature field after applying CNN on it, using some group (Cohen and Welling, 2016a) 6a). The type of CNNs that operates on steerable feature fields is called Steerable CNN (Cohen and Welling, 2016a), which is equivariant to groups including translations as subgroup (z², +), extending (Cohen and Welling, 2016b). It needs to satisfy kernel steerability constraint, where the I² and 2² cases are considered (Weiler and Cesa, 2021). We consider the 2D grid as our domain = = z and use G p4m group the running example. The group pám (Z², +) D₃ (wallpaper group) is semi-direct product of discrete translation group z² and dihedral group D4. see (Cohen and Welling, 201 6b;a). We visualize the transformation law of pAm on feature field Ω = 22 in
18

---

Published as a conference paper at ICLR 2023
Figure 9: Visualization of the permutation representations D4 group for every element g E Dj (4 rotations each row and 2 reflections each column). They are (1) the trivial representation, (2) the regular representation, (3) the quotient representation (quotienting out rotations), (4) the quotient representation (quotienting out reflections).
Figure 8 (Middle), usually referred as induced representation (Cohen and Welling, 2016a; Weiler and Cesa, 2021).
C.2 GROUP REPRESENTATIONS: VISUAL UNDERSTANDING
A group representation is (linear) group action that defines how a group acts some space. Cohen and Welling (2016b;a); Weiler and Cesa (2021) provide more formal introduction to them in the context of equivariant neural networks. We provide visual understanding and refer the readers to them for comprehensive account.
To visually understand how the group D4 acts some vector space, visualize the trivial, regular, and quotient (quotienting out reflections sy²) representations, which are permutation matrices. apply such representation pg D4) to vector, the elements get cyclically permated. See Figure 9.
The quotient representation that quotients out reflections and has dimension 4 x 4 is what need to use on the 4-direction action space.
C.3 GEOMETRIC DEEP LEARNING
We review another set of important concepts that motivate our formulation of steerable planning: geometric deep learning and the theories connecting equivariance and convolution (Bronstein et al., 2021a; Cohen et al., 2020; Kondor and Trivedi, 2018). Bronstein et al. (2021a) use for feature fields while Cohen and Welling (2016a); Cohen al. (2020); Weiler and Cesa (2021) use f-
Convolutional feature fields. The signals are taken from set C = RR on some structured domain Q, and all mappings from the domain signals forms the space of C-valued signals X (12, C) = {f Ω + C}. or (S2) for abbreviation. For instance, for RGB images, the domain is the 2D n x grid Q Zn x Zn, and every pixel can take RGB values = R at each point in the domain E Ω represented by mapping In Zn R3, A function on images thus operates on 3n2 -imeensionll inputs.
It is argued that the underlying geometric structure of domains Ω plays key role in alleviating the curse of dimensionality, such as convolution networks in computer vision, and this framework is named Geometric Deep Learning. We refer the readers to Geometric Deep Learning (Bronstein et al., 2021a) for more details, and to more rigorous theories on the relation between equivari- ant maps and convolutions in (Cohen et al., 2020) (vector fields through induced representations) and (Kondor and Trivedi, 2018) (scalar fields through trivial representations).
Group convolution. Convolutions are shift-equivariant operations, and vice versa. This is the special case for Ω = RR which can be generalized to any group G (that we can integrate sum over). The group convolation for signals on then defined
(ʃ (g) (f.Plg)) [ ((
where (u)) is shifted copies of filter, usually locally supported on subset of Ω and padded outside. Note that although x takes u E Ω, the feature map (x e)) takes as input the elements g G instead of points the domain u Ω. All following group convolution layers take G:
19

---

Published as a conference paper at ICLR 2023
X(G) - (G). In the grid case, the domain Q is homogeneous space of the group G, i.. the group G acts transitively: for any two points u, v E Q there exists a symmetry g E G to reach u = gu.
Analogous to classic shift-equivariant convolutions, the generalized group convolution is G- cquivariant (Cohen et al., 2020). It is observed that (a, p(g)0) = (plg )ax 0), and from the defining property of group representations p(h=))p(g) p(h- g). the G-equivariance of group convolution follows (Bronstein et al., 2021a):
(p(h)xr (g) = (p(h),, = (z, p(r - p(h)
Steerable convolution kernels. Steerable convolutions extend group convolutions to more general setup and decouple the computation cost with the group size (Cohen and Welling, 2016a; Cohen, 2021). For example, E()
CNNs (Weiler and Cesa, 2021) apply it for E2) group, which is semi-direct product translations II and fiber group H. where H is a group of transformations that fixes the origin and is ((2) or its subgroups. The representation from a representation of the fiber group H. Use R2 as example, a steerable kernel only needs to be H-equivariant by satisfying the following constraint (Weiler and Cesa, 2021):
the signals/fields is induced
(hz)) = Pout (h) (1 Yh E H. E R².
(8)
C.4 STEERABLE CNNS
We still use the running example on Z and group p4m = Z² x D4-
Induced representations. We follow (Cohen and Welling, 2016a; Cohen et al., 2020) to use ı for induced representations. We still use feature fields over 2² as example.
As shown in Figure 8 middle, to transform a feature field f: : 2² - CC on base z with group p4m = 2² x D4. we need the induced representation (Cohen and Welling, 2016a; Cohen et al., 2020). The induced representation in this case denoted as =(g) A indg*xDa ((gg)) (for all g).
which means how the group action of D4 transforms a feature field on 2² x D4.
It acts on the feature field with two parts: (1) on the base space 22 and (2) the fibers (feature channels RC) by fiber group H = Da (Cohen and Welling, 2016a; Weiler and Cesa, 2021). More specifically, applying a translation & E Z and transformation x (tr)f (Cohen and Welling, 2016a; Weiler and Cesa, 2021):
D4 some field f, get
((x) + [m((r)] (x) A p(r) [1 ((tr)-+))]
p(r) is the fiber representation that transforms the fibers RCC and -tx finds the element before group action (or equivalently transforming the base space 2²).. Thus, π only depends on the fiber representation p but not the latter part, thus named induced representation by p.
Steerable convolution vs. group convolution. The steerable convolution on 22 The understand- ing of this point helps to understand how group acts on various feature fields and the design of state space for path planning problems. We use the discrete group p4 = Z²2 x C4 as example, which consists of 2 translations and 90º rotations. The only difference with p4m is p4 does not have reflections.
The group convolution with filter v and signal æ on grid (or p E 2²), which outputs signals (a function) on group p4
(+b * 27 (t,r))=
Σ ((t,r) xp))
(10)
PEZZ
A group G has a natural action on the functions over its elements; x : G + IR and g E G, the function g.v is defined as [g. := -1g-
"The definition of group convolution needs to assume that (1) signals X2( are in Hilbert space (to detine inner product (x, 8) Jg (u))d(u)dd)) and (2) the group G is locally compact (so Haar measure exists and "shift" of filter can be defined).
PTechnically, we still need to solve the linear equivariance constraint in Fq. 34 to enable weight-sharing for cquivariance, while Weiler and Cesa (2021) have implemented it for 2D case.
20

---

Published as a conference paper at ICLR 2023
For example: The group action of rotation C4 the space of functions p4 is
(p,y)pp, s) == y(r (p. s)) = ",r
(11)
where r 'p spatially rotates the pixels, r cyclically permutes the 4 channels.
The G-space (functions p4) with natural action of på it:
[[t, ((tr)).u (p. s) = v((t,r)~ (p, 8)) = (p t), (s)
(12)
The group convolution in discrete case is defined as
J [g)
0/ (g. xh)). Σ bEH
(13)
The group convolution with filter sb and signal x on p4 group is given by:
(eb + xx(,rt) = E Σ
((t,rr) x(p, s).
(14)
SECa PE2
Using the fact
v> (ptrr) = vir t) = [rreui(pp - t)
(15)
the convolution can be equivalently written into
[+ * xtt, = Σ
Σ ppe2
|naj] (p -1,
(16)
sEC₁
So (Epez s) a(p,s) can be implemented usual shift-cquivariant convolution CONV2D.
The inner sum Epez: is equivalently for the sum in steerable convolution, and the outer sum Σ implement rotation-equivariann convolution that satisfies ststeerrbillty kernel constraint. Here, the outer is essentially using the regular fiber representation of C4.
In other words. group convolution on p4 Z x C4 group is equivalent to steerable convolution on base space Z with the fiber group of C₁ with regular representation.
Stack of feature fields. Analogous to ordinary CNNs, a feature space in steerable CNNs can con- sist of multiple feature fields J; z² R The feature fields are stacked f θ fe together by concatenating the individual feature fields f. (along the fiber channel), which transforms under the directly sum p D; P: of individual (fiber) representations. Every layer will be equivariant between input and output field fin, fout under induced representations Tin, Tous For a steerable con- volution between more than oie-dimensional feature fields, the kernel is matrix-valued (Cohen al., 2020; Weiler and Cesa, 2021).
D SYMMETRIC PLANNING IN PRACTICE
D.1 BUILDING SYMMETRIC VIN
In this section, we discuss how to achieve Symmetric Planning on 2D grids with E(e)-steerable CNNs (Weiler and Cesa, 2021). We focus implementing symmetric version of value iteration, SymVIN, and generalize the methodology to make a symmetric version of a popular follow-up of VIN, GPPN (Lee al., 2018).
Steerable value iteration. We have showed that, value iteration for path planning problems on consists of equivariant maps between steerable feature fields. It can be implemented as equivari- ant steerable CNN, with recursively applying two alternating (cquivariant) layers:
Q (s) R2, (s) x [Pg (s), V*+I(s) max Q?(s), 8 EEZ?
(17)
21

---

Published as a conference paper at ICLR 2023
Semdica
Q
V
Qa+1
A
Ry Ba
Rar PAR
FC TCaaflms
L
U
(swoed pea)
PAC rogela
Rar + 15
Ra PAR
.M
V/2
Qai1
X Sammiies
Figure 10: Commutative diagram for the full pipeline of Sym VIN on steerable feature fields over 72 (every grid). If rotating the input map M by TM (g) of any g. the output action A SymVIN(M guaranteed to be transformed by æ A (g), i.c. the entire steerable SymVIN equivariant under in- duced representations тM and πA: SymVIN (g)M) л S We use stacked fea- ture fields to emphasize that SymVIN supports direct-sum representations beyond
where & E [K] indexes iteration, VR. Og: R are steerable feature fields over Z² output by equiv- ariant layers, Pg is learned kernel in neural network, and +, x are element-wise operations.
Implementation of pipeline. We follow the pipeline in VIN (Tamar et al., 201 6a). The commutative diagram for the full pipeline shown in Figure 10. The path planning task is given by a m m spatial binary obstacle occupancy map and one-hot goal map, represented
feature field M z² + {0, 1)². For the iterative process O + V + Qi++ the reward field Ry is predicted from map M (by 1 X 1 convolution layer) and the value field Vo is initialized as zeros. The network output is (logits of) planned actions for all locations". represented as A z² IR IR predicted from the final Q- Q-value field Qk (by another 1 convolution layer). The number of iterations K and the convolutional kernel size F of Po set based map size M, and the spatial dimension is kept consistent.
Building Symmetric Value Iteration Networks. Given the pipeline of VIN fully on steerable feature fields, we are ready to build equivariant version with E(2)-sterrbble CNNs (Weiler and Cesa, 2021). The idea is to replace every Conv2d with steerable convolution layer between steerable feature fields, and associate the fields with proper fiber representations p(h).
VINs ordinary CNNs and can choose the size of intermediate feature maps. The design choices in steerable CNNs is the feature fields and fiber representations (or type) for every layer (Cohen and Welling, 2016a; Weiler and Cesa, 2021). The main difference' in steerable CNNs is that also need to tell the network how to transform every feature field, by specifying fiber representations, as shown in Figure 10.
Specification of input map and output action. We first specify fiber representations for the input and output field of the network: map M and action A. For input occupancy map and goal M z + {0, 1 it does not D2 to act on the 2 channels, so we use two copies of trivial representations Paiv @ Panv. For action, the final action output A : 2² + |R is for logits of four actions A (north, west, south, east) for every location. If we use H C4. it naturally acts on the four actions (ordered ()) by cyelically O permuting the R channels. However, since D4 group has 8 elements, we need quotient representation, see (Weiler and Cesa, 2021) and Appendix G.
Specification of intermediate fields: value and reward. Then, for the intermediate feature fields: Q-values Qk. state value VE. and reward Rm.. are free to choose fiber representations, as well as the width (number of copies). For example, if we want copies of regular representation of D4, the feature field has 2 8 16 channels and the stacked representation is 16 x 16 (by direct-sum).
Pechnically it also includes values actions for obstacles, since the network needs to learn to approximate the reward Rar(s, As) with enough small reward and avoid obstacles.
22

---

Published as a conference paper at ICLR 2023
For the Q-value field (s),,, use representation PQ and its size Cq. We need at least CA 2 |A]] channels for all actions of Q(s, as in VIN and GPPN, then stacked together and denoted as Qk - ① e O with dimension Qk z² + CC Therefore, the representation direct-sum + PQ for Cд copies. The reward is implemented similarly as Ras - θ R and must have same dimension and representation to add element-wistt fiber representation as Pv and its size Cv. It has size Vig 2 - R2 Thus, the steerable kernel is matrix-valued with dimension Pg z² - RCCx+Ca)xCv, In practice, we found using regular representations for all three works the best. It can be viewed as "augmented" state and is related to group convolution, detailed in Appendix G.
For state value field, we denote the choose as
Other operations. We now visit the remained (equivariant) operations. (1) The max operation in
Qk + V4+1. While we have showed the max operation in (*) max, Q ()) is equivariant in Theorem E.3, we need to apply max(-pooling) for all actions along the "representation channel" from stacked representations CA Co one Cq. More details in Appendix G.2. (2) The final output layer QK + A. After the final iteration, the Q-value field Qx is fed into the policy layer with convolution to convert the action logit field 2² + R!A!
Gated path planning network (GPPN (Lee et al., 2018)) proposes to use I.TTM to alleviate the issue of unstable gradient in VINs. Although it does not strictly follow value iteration, it still follows the spirit of stecrable planning. Thus, we first obtained a fully convolutional variant of GPPN from [Redacted for anonymous review], called ConvGPPN. It replaces the MLPs in the original LSTM cell with convolutional layers, and then replaces volutions with equivariant steerable convolutions, resulting in a fully equivariant SymGPPN. See Appendix G.1 for details.
Extended method:
Symmetric GPPN.
Extended tasks: planning on learned maps with mapper networks. We consider two planning tasks 2D grids: 2D navigation and 2-DOF manipulation. To demonstrate the ability of handling symmetry in differentiable planning, we consider more complicated state space input: visual naviga- tion and workspace manipulation, and discuss how use mapper networks convert the state input and use end-to-end learned maps, as in (Lee et al., 2018; Chaplot et al., 2021). See Appendix H.2 for details.
D.2 PYTORCH-STYLE PSEUDOCODE
Here, write section explaining the SymVIN method with PyTorch-style pseudocode, since directly corresponds to what we propose in the method section. We try to relate (1) existing concepts with VIN, (2) what we propose in Section 4 and for SymVIN, and (3) actual PyTorch implementation of VIN and SIM aligned line-by-line based on semantic correspondence.
We provide the key Python code snippets to demonstrate how easy it is to implement SymVIN, our symmetric version of VIN (Tamar et al., 201 6a).
In the current Section (SymPlan practice), we heavily use the concepts from Steerable CNNs. Thanks to the equivariant network community and the e2cnn package, the actual implementation compact and closely corresponds to their non-equivariant counterpart, VIN, line-by-line. Thus, the ultimate goal here is to illustrate that, whatever concepts we have in regular CNNs (e.g., have whatever channels we want), we can can use steerable CNNs that incorporate desired extra symmetry (of D4 rotation+reflection or C4 rotation).
We highlight the implementation of the value iteration procedure in VIN and SymVIN:
max R + 7 pa V.
(18)
Note that we use actual code snippets to avoid hiding any details.
Defining (steerable) convolution layer. First, we show the definition of the key convolution layer for a key operation in VIN and SymVIN: expected value operator, in Listing and 2.
As proved in Theorem E.2, the expected value operator be executed by steerable convolution layer for (2D) path planning. This serves as the theoretical foundation on how we should use a steerable layer here.
23

---

Published as a conference paper at ICLR 2023
import
torch
import torch
import 62cnn
Define the sysmetzy group
gapaee o2cnn.gapaces. Flppccea. (N=4)
6
# Define featuze (Ciber) at
field_type_g_ie =2cmm.an.Fieddlype
8
gepacemgspace,
9
representationa-2 e_size Igepace. regular_repr)
10
Define regular 20 convolution
Darine steerable convolution
13 convv - torch.en. Con.vat
12
c2conv - e2can.an. an.an.an.22e Rn.o2
in_chaznele=l, out cut_chanmelas2 g_size,
13
14
www.typo-fieldd.yye.cc termel_aizenn stride=1, blas=False
kernel kernel_atzeur, _aize=F, stride=1, blas=Fa
17
16
Listing I: Define 'expected value' convolution layer for VIN.
Listing Define 'expected value' (steerable) convolution layer for SymVIN.
For the left side, regular 2D convolution is defined for VIN. The right side defines steerable convolution layer, using the library e2cnn from (Weiler and Cesa, 2021). It provides high-level abstraction for building equivariant 2D steerable convolution networks. As user, only need to specify how the fcature fields transform (as shown in Figure 10), and it will solve the G-stecrability constraints, process what needs be trained for equivariant layers, etc. We use name r_noo to highlight the difference.
Value iteration procedure. Second, we compare the for loop for value iteration updates VIN and Sym VIN, where the former one has regular 2D convolution Conv2D (Listing 3), and the latter one uses steerable convolution (Weiler and Cesa, 2021) (Listing 4).
The lines aligned based semantic correspondence. The e2cnn layers, including steerable convolution layers, operate on its Geomet r cTensor data structure, which is to wrap a Py PyTocc tensor. We denote them with gee suffix. It only additionally needs to specify how this tensor (fea- ture field) transforms under group (e.g., D4), i.c. the user needs to specify a group representation for it.
tensor direct is used to concatenate two GeometricTensoor (feature fields) and compute their associated representations (by direct-sum).
Thus, the e2enn steerable convolution layer on the right side q r_c22o can be used as regular PyTorch layer, while the input and output are Geomet r cTensor.
We also define the max operation as customized max-poo max-pooling layer, named max _poo_ The implementation similar the left side of VIN and needs additionally guarantee equivariance, and the detail is omitted.
Note that for readability, we assume we use regular representations for the Q-value field Q and the state-value field V. They are empirically found to work the best. This corresponds to the definition sield_type type_g_in in line in the SIMVIN definition listing and the comments in line 16-17 in the steerable VI procedure listing for SymVIN.
Other components are omitted.
E SYMMETRIC PLANNING FRAMEWORK
This section formulates the notion of Symmetric Planning (SymPlan). We expand the understanding of path planning in neural networks by planning as convolution on steerable feature fields (steerable planning). We use that to build steerable value iteration and show it is equivariant.
24

---

Published as a conference paper at ICLR 2023
Inous: maze and goal nap, Vitoraciona
Iaput: mazo and GO2I sap, Fitorationa K
2
wZcon.cn impozt GesneLricfenae
from eZann.cn import tensor_dizeetaun
5
cazch.cat (mnaze_nap, gosl_napl, din=l)
torch.cat ((naze_nap, gosl_nap,, din=l]
7
GeonsrricTonsorfe cype_clold_eyeeuul F_geo +_rroorr (x_gea]
r_ (x)
9
10 Init value function y
10 Init ? and Heap in ežena "oesnetric consoz!
Luzch, derus (f.
Ye Lorch. sarch.zerosiin t sixel)) V_9e0 GeonersicTense (V_ tyyed_
13
13
14
for in range [K]
14
Eor in range 181
15
Coneat ccavelve with
(dizect-sus) and convoive V wich
16
rv tarch.cat ([T, Via din-1)
rv_geo tensor dirt sun ([r_ge,, x_geo]) q_eeu q_ (ev_geo)
17
gLeonv Jev)
17
18
18
ion chenne?
group phennei
21
Gr Batel q_xizu x
G: bavelleize x (18) [gaize)
21
> Ve batel_aizo x 1 x x
21
V: batch_ x (19) + x
22
qeview 1, guxize H, I1)
22
v_ g_nex_pool (g.geo)
V. Lorch.max (4, din=l)
24 v w.viewt-l, M, R)
25
25
26 # OuspoL: 'o" (co prodoce policy nap?
26
Ouupan:: "a_geu' (LD produce pulliy sap!
Listing 3: The central value iteration procedure for VIN. Some variable names are adjusted accordingly for readability. W and H are width and height for 2D map.
Listing 4: The equivariant steerable value iteration procedure for SymVIN. Lines are aligned by semantic correspondence. Definition of other field types are similar and thus omitted.
E. STEERABLE PLANNING: PLANNING ON STEERABLE FEATURE FIELDS
We start the discussion based on Value Iteration Networks (VINs, (Tamar et al., 2016a)) and use a running example of planning on the 2D grid Zz, We aim to understand (1) how VIN-style networks embed planning and how its idea generalizes, (2) how is symmetry structure defined in path planning and how could it be injected into such planning networks.
Constructing G-invariant transition: spatial MDP. Intuitively, the embedded MDP in a VIN different from the original path planning problem, since (planar) convolutions are translation equivariant but there are different obstacles in different regions.
We found the key insight in VINs is that implicitly uses an MDP that has translation equivariance. The core idea behind the construction is that it converts obstacles (encoded in transition probability P. by blocking) into "traps" (encoded in reward R, by -c0 reward). This allows to use planar con- volutions with translation equivariance, and also enables use to further use steerable convolutions.
The demonstration of the idea is shown in Figure 8 (Left). We call spatial MDP, with different transition and reward function M (8, A, Rm, 7), which converts the "complexity" in the transition function P in M to the reward function Rn in M. The state and action space are kept the same: state S = Z and action A c Z to move As in four directions in 2D grid. We provide the detailed construction of the spatial MDP in Section E.3.
Steerable features fields. We generalize the idea from VIN, by viewing functions (in RL and planning) as steerable feature fields, motivated by (Bronstein et al., 2021a; Cohen et al., 2020; Cohen and Welling, 2016a). This is analogous to pixels on images Ω - [255] and would allow us to apply convolution on it. The state value function is expressed as field V : S + IR while the Q-value function needs a field with |A] channels: Q S + A Similarly, a policy field" has probability logits of selecting |A| actions. For the transition probability P(s' ss a), we can action to index it as (* (s), similarly for reward (s). The next section will show that we can convert the transition function to field and even convolutional filter.
'ee avoid the symbol for policy since it is used for induced representation in (Cohen and Welling, 201 6a; Weiler and Cesa, 2021).
25

---

Published as a conference paper at ICLR 2023
E.2 SYMMETRIC PLANNING: INTEGRATING SYMMETRY BY CONVOLUTION
The seemingly slight change in the construction of spatial MDPs brings important symmetry struc- ture. The general idea in exploiting symmetry in path planning is use equivariance to avoid explicitly constructing equivalence classes of symmetrie states. To this end, we construct value iteration over steerable feature fields, and show it is equivariant for path planning.
In VIN, the convolution is over 2D grid z², which is symmetric under D4 (rotations and reflec- tions). However, we also know that VIN is already equivariant under translations. To consider all symmetries, as in (Cohen and Welling, 2016a; Weiler and Cesa, 2021), we understand the group A H as constructed by a base space B = G/H H = D4. which is a slubilizer subgroup that fixes the origin 0 E z We could then formally study such symmetry in the spatial MDP, since we construct it to ensure that the transition probability function in M is G-invariant. Specifically, we can uniquely decompose any g E Z² DA as t E Z² and r E D1 (and translations act "trivially" on action), so
p4m G
(Z²,+) and a fiber group
P(s' s, a) = P(g.s' g.s, g.a) ((tr).s'' (l.),,, Vgg = tr E 2² X D4, Vs, (19)
Expected value operator as steerable convolution. The equivariance property can be shown step- by-step: (1) expected value operation, (2) Bellman operator, and (3) full value iteration. First, we use G-invariance to prove that the expected value operator >x' P(s'|s, (s') is equivariant.
Theorem E.1. If transition G-invariant, the expected value operator E over Z² is G-equivariant
The proof is in Section F.1 and visual understanding is in Figure 8 middle. However, this provides intuition but is inadequate since we do not know: (1) how to implement it with CNNs. (2) how to use multiple feature channels like VINs, since it shows for scalar-valued transition probability and value function (corresponding to trivial representation). To this end, we next prove that we can implement value iteration using steerable convolution with general steerable kernels.
Theorem E.2. If transition is G-invariant, there exists (one-argument, isotropic) matrix- valued steerable kernel pa (s - 8") (for every action), such that the expected value operator can be written as a steerable convolution and is G-equivariant:
E - pa VV Lg.[P" V][(s) = [p9. * [lg. Vs 2
x
(20)
The full derivation is provided Section F. We write the transition probability as (s, and we show it only depends on state difference P* s') (or one-argument kernel (Cohen et al., 2020)) using G-invariance, which the key step to show it is some convolution. Note that we use one kernel pa for each action (four directions), and when the group acts on E, it also acts the action pg.e (and state, so technically acting on x A). Additionally, if the steerable kernel also satisfies the Da-steerability constraint (Weiler and Cesa, 2021; Weiler et al., 2018), the steerable convolution is equivariant under p4m = Z² x D2. We can then extend VINs from Z² translation equivariance to ydm-equivariance (translations, rotations, reflections). The derivation follows the existing work on steerable CNNs (Cohen and Welling, 20;6;;a; Weiler and Cesa, 2021; Cohen et al., 2020), while this is our goal: to justify the close connection between path planning and steerable convolutions.
Steerable Bellman operator and value iteration. We can now represent all operations in Bellman (optimality) operator on steerable feature fields over 2² (or steerable Bellman operator) as follows:
V (s) = max Rd (s) 7
[Pa * (s),
(21)
where V, R Po are steerable feature fields over Z As for the operations, max, is (max) pooling (over group channel), +, x are point-wise operations, and is convolution. As the second step, the main idea is to prove every operation in Bellman (optimality) operator on steerable fields is equivariant, including the nonlinear max, operator and +, x. Then, iteratively applying Bellman operator forms value iteration and is also equivariant, as shown below and proved in Appendix F.4.
Proposition E.3. For a spatial MDP with G-invariant transition, the optimal value function can be found through G-steerable value iteration.
Remark. Our framework generalizes the idea behind VINs and enables us to understand its appli- cability and restrictions. More importantly, this allows to integrate symmetry but avoid explicitly
26

---

Published as a conference paper at ICLR 2023
building equivalence classes and enables planning with symmetry in end-to-end fashion. We empha- size that the symmetry in spatial MDPs is different from symmetric MDPs (Zinkevich and Balch, 2001; Ravindran and Barto, 2004; van der Pol et al., 2020a), since reward function is not G- invariant (if not conditioning on reward). Although we focus on Z², we can generalize to path planning on higher-dimeenional or even continuous Euclidean spaces (like TR3 space (Weiler et al., 2018) or spatial graphs in IR (Brandstetter et al., 2021)), and use equivariant operations on steerable feature fields (such steerable convolutions, pooling, and point-wise non-linearities) from steerable CNNs. We refer the readers to (Cohen and Welling, 2016b;a; Cohen, 2021; Weiler and Cesa, 2021) for more details.
E.3 DETAILS: CONSTRUCTING PATH PLANNING IN NEURAL NETWORKS
We provide the detailed construction of doing path planning in neural networks in the Section E. This further explains the visualization in Figure left.
We use the running example of planning on the 2D grid 2². We aim to understand (1) how VIN- style networks embed planning and how its idea generalizes, (2) how is symmetry structure defined in path planning and how could it be injected into such planning networks. Recall that we aim to understand (1) how VIN-style networks embed planning and how its idea generalizes, (2) how is symmetry structure defined in path planning and how could it be injected into such planning networks.
Path planning MDPs. To answer the above two questions, we first need to understand how a VIN embeds a path planning problem into a convolutional network as some embedded MDP. Intuitively, the embedded MDP in VIN is different from the original path planning problem, since (planar) convolutions are translation equivariant but there are different obstacles in different regions.
For path planning on the 2D grid S = 22, the objective is to avoid some obstacle region Coba 2² and navigate to the goal region Cgeat through free space C' C.Cbb An action a = -As E AA is to move from the current state
next free state s' = + As, where for now we limit it to be in four directions: A =. Assuming deterministic transition, the agent moves to s' with probability 1 if + As E C'.Cobs. If it hits an obstacle, stays if + As Cobs: P (s + As s, As) 0 and ((s s, As) = 1. Every move has constant negative reward R(s a) = -1 to encourage shortest path. We call this ground path planning MDP, a 5-tuple M A, P. P. R. 7).
Constructing embedded MDPs. However, such transition function is not cunslation-invariann ie. at different position, the transition probabilities are not related by any symmetry: P(s'|s. a) + P(g.s'Ig g.a). Instead, could always construct "symmetric" MDP that has equivalent optimal value and policy for path planning problems, which implicitly realized in VINs. The idea is to move the information of obstacles from transition function reward function: when we hit some action 8 + As E Coba. we instead allow transition P ((s + As 8, As) probability) while set a "trap" with negative infinity reward Rm, (s. As) = cc. The reward function needs the information from the occupancy map M, indicating obstacles Cobe and free space. For the free region, the reward still a constant RM (s, As) 11, indicating the cost of movement.
(with all other s' as O
We call it the embedded MDP, with different transition and reward function M = AS Ry,,) 77), which converts the "complexity" in the transition function P in M to the reward function R., in M. Here, map M shall also be treated as an "input", thus later we will derive how the group acts on the map g g. It has the same optimal policy and value as the ground MDP M, since the optimal policies in both MDPs will avoid obstacles in M or trap cells in M. It could be easily verified by simulating value iteration backward in time from the goal position.
The transition probability P of the embedded MDP M is for an "empty" maze and thus translation- invariant. Note that the reward function R is not not necessarily invariant. This construction is not limited to 2D grid and generalizes to continuous state space even higher dimensional space, such as R configuration space for 6-DOF manipulation.
Note, all of this is what we to conceptually understand how VIN is possible to learn. The reward cannot be negative infinity, but the network will learn to be smaller than all desired Q- values.
27

---

Published as a conference paper at ICLR 2023
E.4 DETAILS: UNDERSTANDING SYMMETRIC PLANNING BY ABSTRACTION
How do we deal with potential symmetry in path planning? How do we characterize it? We try to understand symmetrie planning (steerable planning after integrating symmetry with equivariance) and how it is difference classic planning algorithms, such A*, for planning under synmetry.
Steerable planning. Recall that we generalize the idea of VIN by considering it as a planning network that composes of mappings between steerable feature fields.
The critical point is that, convolutions directly operate on local patches of pixels and never directly touch coordinates of pixels. In analogy, this avoids a critical drawback in other explicit planning algorithms: in sampling-based planning, a trajectory (81, a1, 82, a2,,... ...) is sampled and inevitable represented by states Ω = S. However, to find another symmetric state g.8, we potentially need to compare it against all known states S' C S with all symmetries g E G. On high level, an implicit planner can avoid such symmetry breaking and is more easily compatible with symmetry by using equivariant constraints.
We can use MDP homomorphism to understand this (Ravindran and Barto, 2004; van der Pol al., 2020b).
MDP homomorphisms. An MDP homomorphism h - M is mapping from one MDP M = AS, R2 to another M = AS R,,,R, 7) (Ravindran and Barto, 2004; van der Pol et al., 2020b). h consists of tuple of surjective maps h (db {a, 8 E S}), where Φ : + S the state mapping and a, A - A the dependde action mapping. The mappings are constructed to satisfy the following conditions:
A(d(s), & R(s,a),
(() &
Σ 8" ("(o")
P(s" |s,a)
(22)
for all s E S and for all a E A.
We call the reduced MDP M the homomorphic image of M under h. I = (0, {as E S}) has bijective maps @ and {a,), J. call h an MDP isomorphism. Given MDP homomorphism h, (s, a) and (s', d') are said to be h-equivariant if (s) = a (s') and (a = ag (a').
Symmetry-indueed MDP homomorphisms. Given group G, an MDP homomorphism h is said to be group structured if any state-action pair (s, a) and its transformed counterpart g. (s, a) are mapped to the same abstract state-action pair: aa,)), = (ø(g.s), a (g.a)). for all S, a E A, E G. For convenience, we denote g.(s,, a) as (g.s,g.u g.a). where g.a implicitly depends on state s. Applied to the transition and reward functions, the transition function P is G-invariant if P satisfies (gg.s' (g..s,/g,,,, g.a) = P(s a), and reward function R is G-invariant if R(g.s, g.a) R(s, a), for all s E S, a E A. g G.
However, this only fits the type of symmetry in (van der Pol et al., 2020a; Wang al., 2021). And also, they cannot handle invariance to translation z². In our case, we need to augment the reward function with map M input:
(g.sr( = (s,
(23)
for all s S,a E E G= E pAm.
This means that, at least for rotations and reflections D,, the MDPs constructed from transformed maps M} are MDP isomorphic to each other.
We avoid the symbol π for policy since it is used for induced representation in (Cohen and Welling, 2016a; Weiler and Cesa, 2021).
'The group operation acting on action space A depends on state, since G actually acts on the product space S x A: (g. (s, a)) + gs a). while we denote it as (g.s.g.a) for consistency with / = (é, {a= E S)). As a bibliographical note, in van der Pol et al. (2020b), the group acting state and action space is denoted as state transformation T.g S S and state-dependent action transformation K5 A A.
28

---

Published as a conference paper at ICLR 2023
E.5 NOTE: AUGMENTED STATE
We derive the relationship between group convolution and steerable convolution in Section C.4,
The augmented state Z D4 I be similarly treated the group pdm Z2 D4. It is equivalent to using regular representation on the base space T²2 as ??2 - R
F SYMMETRIC PLANNING FRAMEWORK: PROOFS
We show the derivation and proofs for all theoretical results in this section.
We follow the notation in (Cohen et al., 2020) use for (one-argument) convolution and for (two-argument) multiplication:
(M[V(s) Vi(s) = (8 V('))
(24)
F.1 PROOF: EQUIVARIANCE OF SCALAR- VALUED EXPECTED VALUE OPERATION
We present the Theorem E.1 here and its formal definition.
Theorem F.1. If transition is G-invariant, the expected value operator over Z2 is G-equivariant:
Eg [E5-9 (g. V]] (s), for all g Ir E 2? D4.
Proof. E is the expected value operator. We also write the transition probability Recall the G-invariance condition of transition probability, the group element acts on 8, a, s':
P(s' s, a) = P(g.s' g.s,g.a) P ((tr)..' (tr),,,r.a), Vg = tr E z x D4 Vs, a, (25)
where we can uniquely decompose any g E 2 D1 as 2² and D₃ (Cohen and Welling, 201 Note that, since the action is the difference between states a As s 8, the translation part acts trivially on it, so g.a (tr).a r.a for r D4.
We transform the feature field and show its equivariance:
Ig. [V]](s 1g. V]](s)
(26)
Σ (r (s' (6'((1r)) ((r)) .s) V(s')
(27)
Σ (ail(r) ((tr).s' |s) v(s')
(28)
Σ (niv(r) (8') 8) V ((tr)-1s))
(29)
Σ pra (3' ()) (VV ((tr)-13)
(30)
pra [g. [s)
(31)
= E gg. (8)).
(32)
We use the trivial representation (g)v(g) Id to emphasize that (1) the group element acts on feature fields pa and V. and (2) both feature fields pa and V are scalar-valued and correspond to the one-dimensional trivial representation of E D₁.
In the third line, we use the G-invariance of transition probability.
The fourth line uses substitution 3' (tr).s', for all s' 2 and tr 2 xD. This is one-to-one mapping and the summation does does not change.
29

---

Published as a conference paper at ICLR 2023
F.2 PROOF: expected value operator AS STEERABLE CONVOLUTION
In this section, we derive how to cast expected value operator as steerable convolution. The equiv- ariance proof is in the next section.
In Theorem E.I, we show equivariance of value iteration in 2D path planning, while it is only for the case that feature fields pa and V are scalar-valued and correspond to one-dimensional trivial representation of D1-
Here, provide the derivation for Theorem E.2 show that steerable CNNs (Cohen and Welling, 2016a) can achieve value iteration since we could construct the G-invariant transition probability a steerable convolutional kernel. This generalizes Theorem E. from scalar-valued kernel (for transition probability) with trivial representation to matrix-valued kernel with any combination of representations, enabling using stack (direct-sum) of feature fields and representations.
We state Theorem E.2 here for completeness:
Theorem F.2. If transition is G-invariant, there exists a (one-argument, isotropic) matrix-valued steerable kernel Pa (s- s') (for every action), such that the expected value operator can be written as a steerable convolution and is G-equivariant:
E EV[V P v, [g-[Pa V]](s) = [pp.a *gg. [g.V]((s), Vs 22 E E D4.
(33)
Steerable kernels. our earlier definition, th and fm are transition probability and value func- tion, which are both real-valued 26ah - R, fi z + R. However, this is special case which corresponds to use one-dimensional trivial representation of the fiber group D4. the general case in steerable CNNs (Cohen and Welling, 201 Weiler and Cesa, 2021), )), we can choose the feature fields d z² + C XCo and fin : 2² + RC and their fiber representations, which we will introduce the group representations of D4 and how to choose in practice in the next section.
Weiler et al. (2018) show that convolutions with steerable kernels ei 22 + RCa is the most general equivariant linear map between steerable feature space, transforming under Pmn and Paut- In analogy to the continuous version" in (Weiler and Cesa, 2021), the convolution equivariant iff the kernel satisfies a H --steerability kernel constraint:
(hs) = Poaa (h)( (s) (4) MEH=DD D4, E
(34)
Expected value operation steerable convolution. The foremost step is show that the pected value operation is a form of convolution and is also G-equivariant. By definition, if we want write (linear) operator as a form of convolution, we need one-argument kernel. Cohen et al. (2020) show that every linear equivariant operator is some convolution and provide more details. For our case, this formally shown follows.
Proposition F.3. If the transition probability is G-invariant, it can be expressed as (one- argument) kernel pa (s'ls) = pa (s' - -s) that only depends on the difference 8.
Proof. The form of our proof is similar to (Cohen et al., 2020), while its direction is different from us. We construct MDP such that the transition probability kernel is G-invariant, while Cohen al. (2020) assume the linear operator và f is linear equivariant operator on homogeneous space, and then derive that the kernel is G-invariant and expressible as one-argument kernel. Additionally, our kernel v (s, s') and " s') both live on the base space B = 2² but not on the group = Z x D4-
We show that the transition probability only depends on the difference As = s' s' the two-argument kernel مa (s'(s) on S by an one- argumentt kernel (s'' ss) (for every action
define
Weiler and Cesa (2021) letter G denote the stabilizer subgroup H 0(2) of E((2).
30

---

Published as a conference paper at ICLR 2023
a) S z², without loss of generality:
(s - 8) P= -
(35)
-g0 -s) s)
(36)
(rr))
(37)
=pri(ns, - 88 8)) =pro(r.s).s'))
(38)
(39)
=s"ss,'',
(40)
where the second step uses G-invariance with g = sr, understood as the composition of a translation 8 E Z?2 and transformation in r D4.
Additionally, we can also derive that, for the one-argument kernel, we rotate state difference r.(s' s), the probability is the same for rotated action r.a.
(s - s) prr (r.(8' - 8)), for all r E 8,,8,8' E ez?
(41)
The expected value operator with two-argument kernel can be then written as
(sV] = [Pa. V]()) = Σ (d(ss)))))) = Pm (= = V[))))
(42)
Note that we do not differentiate between cross-correlation (s' s) and convolution (s s').
F.3 PROOF: EQUIVARIANCE OF expected future value
Our derivation follows the existing work on group convolution and steerable convolution net- works (Cohen and Welling, 2016b;a; Weiler and Cesa, 2021; Cohen et al., 2020). However, the goal of providing the proof not just for completeness, but instead to emphasize the close connec- tion between how formulate our planning problem and the literature of steerable CNNs, which explains and justifies our formulation.
Additionally, there are several subtle differences worth mention. (I) Throughout the paper, do not discuss kernels or fields that live on a group to make it more approachable. Nevertheless, group convolutions are special case of steerable convolutions with fiber representation p as regular representation. (2) We use Z as running example. Some prior work uses R² or Z², but they are merely just differ integral and summation. (3) The definition of convolution and cross-correlation might be defined and used interchangeably in the literature of (equivariant) CNNs.
Notation. To keep notation clear and consistent with the literature (Cohen and Welling, 2016a; Cohen al., 2020; Weiler and Cesa, 2021), denote the transition probability P(s'|s, a) & (s, R.. (one kernel for an action) and value function as V(s') & fin(s') TR, and the resulting expected value as fou (s) E. (
f'm) (given specific action a).
Transformation laws: induced representation. For some group acting on the base space ZZ, the signals f 2 + Re are transformed like Cohen and Welling (2016a):
[ (x) = f(g-tz)
(43)
Apply a translation and transformation D4 to f. get π ffr).. The transformation law on the input space fi is (Cohen and Welling, 2016a; Weiler and Cesa, 2021):
f(x) + [T((t) (x) p(r) [f ((tr)-'x))
(44)
The transformation law of the output space after applying Tia on input fin is given by Cohen and Welling (2016a):
b fl (z) [vb (n(t)) (I) A p(r) [I fl ((tr)-12)]
(45)
In our case, the output space is fa. 2 - R Coc and the input space is fin 2 CCC, Intuitively, we rotate vector field (fibers represent arrows) by the induced representation <trrr ) of f, we also
need rotate the direction of arrows by p(r), r D4.
31

---

Published as a conference paper at ICLR 2023
Equivariance.
Now prove the steerable convolution is equivariant: 1 * (m)())) (6) = (uu ( (s) vs E S Vg E.
(46)
The induced representation of input field fim is induced by the fiber representation Pmr expressed by Tin - ind Pin ind DD Pin. where Pio is the fiber representation of group H D4. The induced representation of output field Tout is analogously from Pout.
Weiler and Cesa (2021) proved equivariance of steerable convolutions for IR² case. while include the proof under our setup for completeness. The definition in (Weiler and Cesa, 2021) uses form of cross-correlation and we use convolution, while it is usually referred to interchangeably in the literature and is equivalent. Cohen and Welling (2016a); Weiler et al. (2018); Weiler and Cesa (2021); Cohen et (2020); Cohen (2021) provide more details and we refer the readers to them for more comprehensive account.
The convolution on discrete grids z² with input field fin transformed by the induced representation
Tin gives:
[w0 (mi((r)).in]((s)) Σ
(s 8--88)) (=M))m((()))
s'EZ?
(
-s- s() (t)
E'
Σ
Pona (r) (
Pont
(rn(r) -
s'EZZ
(47)
-ooa
16 (s - s'))sin(rr (s' (t)
s'E22
-Pou(r)
Σ (r (8-1) -1- (m)
BE22
Pour
t))
oTo (rt) (s),
where s' S 22, and thus satisfies the equivariance condition:
[vb * (rt) (s) (rt) (s). Z2, Z² D4.
(48)
1. Definition of
2. Transformation law of the induced representation Tin (Cohen and Welling, 2016a; Weiler
and Cesa, 2021)
3. Kernel steerability (s) Pour (1) 2 (Weiler and Cesa, 2021)
4. Move and cancel
5. Substitutes 3 = (s' / - t). r- r 3, so (s = r- E ZZ2 the result still p4m, is one-to-one correspondence p4m x 2 + Z2, and the summation does not change. Weiler and Cesa (2021) analogously considers the continuous case, where D4 is orthogonal transformations so the Jacobian is always 1.
5.
Since r E D4 and
6. Definition of *
7. Transform law of the induced representation nout
F.4 PROOF: EQUIVARIANCE OF STEERABLE VALUE ITERATION
As the third and final step, would like show that the full steerable value iteration pipeline equivariant under 2 x D4- We need to show that every operation in the steerable value iteration is equivariant.
The key is to prove that max, is an equivariant non-linearity over feature fields, which follows Section D.2 in (Weiler and Cesa, 2021).
32

---

Published as a conference paper at ICLR 2023
Q
V
Qa+1
Ry Pa
Rar PAR
FC TCaaflms
L
U
(swved pea)
PAC rogela
Rar 1s
Ra PAR
MM
Q
V/2
Qai1
9.A
Figure 11: We attach a copy of the commutative diagram of SymVIN to show the equivariance of steerable value iteration. Commutative diagram for the full pipeline of SymVIN on steerable feature fields over Z (every grid). rotating the input map M by I (g) of any g. the output action A = SymVIN is guaranteed to be transformed by (g), i.e. the entire steerable SymVIN is equivariant under induced representations тM and T SymVIN(T M) ) = n (g)SymYM We use stacked feature fields to emphasize that SymVIN supports direct-sum of representations beyond scalar-vall valued.
Step 1: V + Q. Here, we prove the equivariance of (+(s) = (x) let the group acts on both sides:
Pg
Vie (s).
First,
(+)s) = R& (s) + x Pg V& (s)
(49)
oanr (s)((((s) = [g) 18 [(s) + x (g) [P * Vi]] (s)
(50)
oa {}){(s) = [g) R (s) (s) + [Pg [m ( V&]] (s)
(g)(g)))
(51)
Qh (g (g-1s) -gg. (g + Phia Vo (g ss)
(52)
Q = R (3) +
P
(53)
The the last step we substitute g and g.a.
M Z - (0, 2 is the concatenation of maze occupancy map and goal map, which also lives on Z We use two copies trivial representations as fiber representation PM. and denote the induced representation of the field M as TM-
Then, we prove the equivariance: iteration should have both input V and output Q transformed. Since this is iterative process, the only input to the value iteration is actually the occupancy map M : Z + 1)?.
we transform the occupancy map (and goal map), the value
Before that, observe that the reward also has G-invariance when we have map input:
Rhg (s) Rggn (g-s).
(54)
Additionally, since the reward R (s) means the reward given position in map M after executing action a, when we transform the map, we also need to transform the action: a (s).
Since it is iterative process, let the Q-map being transformed by g:
(s) =
(55)
-R3, (g +
Pg Vv] (g
(56)
Rig. (3) + [Pg Vi] (g-1s)
(57)
RS.a R.M (s)
+ [pg. Vil] (s)
(58)
The second last step uses the G-invariance condition R9 (s) equivariance of steerable convolution.
pa.0 Ms (g.s). The last step uses the
should be understood as: (1) transforming map g.M and action g.a. is always equal to (2) trans- forming values [g. and [g. Vt). This proves the equivariance visually shown in Figure 11.
33

---

Published as a conference paper at ICLR 2023
Step 2: Q V. The second step is to show for Vs+1(() max, Q5(s).
Intuitively, we sum over every channel of each representation. For example, we have N copies of the regular representation with size [Da] (1 x 8) X m X m along the N channel. Thus, how we use the 8 x regular representation to transform the N 8 channels still holds for 8, which implies equivariance. The m spatial map channels form the base space Z² and are transformed as usual (spatially rotated).
8, transform the tensor (N 8)
to
Weiler and Cesa (2021) provide detailed illustration and proofs for equivariance of different types of non-linearities.
Step 3: multiple iterations. Since each layer is equivariant (under induced representations), Co- hen and Welling (2016b); Kondor and Trivedi (2018) 8); Cohen et al. (2020) show that stacking mul- tiple equivariant layers is also equivariant. Thus, we know iteratively applying step and (equiv- ariant steerable Bellman operator) is also equivariant (steerable value iteration).
G IMPLEMENTATION DETAILS
G.1 IMPLAMENTATION SYMGPPN
ConvGPPN (Kong, 2022) is inspired by VIN and GPPN. To avoid the training issues in VIN, GPPN proposes to use LSTM to alleviate them. In particular, it does not use max pooling the VIN. Instead, it uses CNN and LSTM to mimic the value iteration process. ConvGPPN, hand, integrates CNN into LSTM, resulting a single component convLSTM for value iteration. We found that ConyGPPN performs better than GPPN in most cases. Based ConvGPPN, SymGPPN replaces each convolutional layer with steerable convolutional layer.
the other
G.2 IMPLEMENTATION OF max OPERATION
Here, we consider how to implement the max operation (*+ max. Q9(s). The max taken over every state, so the computation mainly depends on our choice of fiber representation.
For example, if we use trivial representations for both input and output, the input would Qx z² + I-CAA and the output is state-value V₂ z R. This recovers the default value iteration since we take max over CC vector.
In steerable CNNs, we can stack of fiber representations. We can choose from regular-regular, trivial-trivial, and regular-trivial (trivial reguial is not considered).
We already covered trivial representations for both input and output, they would be Qk Z CC and Vx z + gCv with CQ = Cy = 1, since every channel would need a trivial representation.
we use regular representation for Q and trivial for V, they are Qk z > RC and V z Cv with Cq [D1 = and Cy = It degenerates that we just take max over all Co CA channels.
For both using regular representations, need make sure they use the same fiber group (such as D4 or C4), so Co = Cv. If using D4, we have Qx 7 - B+CA and V2 Z² - R² and we take max over every CA channels (for every location) and have 8 channels left, which are used as 2 + R
Empirically, we found using regular representations for both works the best overall.
H EXPERIMENT DETAILS AND ADDITIONAL RESULTS
H.1 ENVIRONMENT SETUP
Action space. Note that the MDP action space A needs be compatible with the group action x A + A. Since the E2CNN package (Weiler and Cesa, 2021) uses counterclockwise rotations generators for rotation groups C.,, the action space needs be counterclockwise.
34

---

Published as a conference paper at ICLR 2023
We show the figures for Configuration-space and Workspace manipulation in Figure 12, and the figures for 2D and Visual Navigation in Figure 13.
Figure 12: A set of visualization for a 2-joint manipulation task. The obstacles are randomly gener- ated. (1) The 2-joint manipulation task shown in top-down workspace with 96 96 resolution. This is used as the input to the Workspace Manipulation task. (2) The predicted configuration space in resolution 18 18 from a mapper module, which is jointly optimized with planner network. (3) The ground truth configuration space from handcraft algorithm in resolution 18 x 18. This is used as input to the Confiuuratio--ppace (C-space) Manipulation task and as target the auxiliary loss for the Workspace Manipulation task (as done in SPT (Chaplot et al., 2021)). )). (4) The predicted policy (overlaid with C-space obstacle for visualization) from an end-to-end trained SymVIN model that uses mapper to take the top-down workspace image and plans on a learned map. The red block the goal position.
Manipulation. For planning in configuration space, the configuration space of the DoFs ma- nipulator has no constraints in the (0,T) boundaries, i.e., joint limits. To reflect this nature of the configuration space in manipulation tasks, we use circular padding before convolution op- eration. The circular padding is applied to convolution layers in VIN, Sym VIN, ConvGPPN, and SymGPPN. Moreover, GPPN, there is a convolution encoder before the LSTM layer. We add the circular padding the convolution layers in GPPN well.
2-DOF manipulation in configuration space, we adopt the setting in (Chaplot et al., 2021) and train networks to take as input of configuration space, represented by two joints. We randomly generate 0 to 5 obstacles the manipulator workspace. Then the degree-of of-eeedomm (DOF) con- figuration space constructed from workspace and discretized into 2D grid with sizes {18, 36), corresponding to bins of 20 and 10", respectively.
We allow each joint to rotate 2Tr, so the configuration space of 2-DOF manipulation forms torus T Thus, the both boundaries need to be connected when generating action demonstrations, and (equivariant) convolutions need be circular (with padding mode) wrap around for all meth- ods. We allow each joint to rotate over 2T, so the both boundaries in configuration space need be connected when generating action demonstrations, and (equivariant) convolutions need to be circular (with padding mode) to wrap around for all methods.
H.2 BUILDING MAPPER NETWORKS
For visual navigation. For navigation, we follow the setting in GPPN (Lee et al., 2018). The input m panoramic egocentric RGB images in 4 directions of resolution 32 x 32 x3, which forms tensor of m x m x x 32 x 32 x 3. A mapper network converts every image into a 256-dimensional embedding and results in a tensor in shape m m x 256 and then predicts map layout m m 1.
For the first image encoding part, we use CNN with first layer of 32 filters of size 8 x 8 and stride of 4 4, and second layer with 64 filters of size 4 and stride of 2 x 2, with final linear layer of size 256.
The second obstacle prediction part, the first layer has 64 filters and the second layer has 1 filter, all with filter size x 3 and stride 1 x 1.
For workspace manipulation. For workspace manipulation, we use U-net Ronneberger et al. (2015) with residual-commection He et al. (2015) mapper, see Figure.14. The input is 96 x 96 top-down occupancy grid of the workspace with obstacles, and the target is to output 18 x 18 configuration space the maps for planning.
35

---

Published as a conference paper at ICLR 2023
L
Y axis
Position (x=5,y=3)
X axis
Ground truth maze
Mapper input
T-+
Figure 13: of visualization for 2D navigation and visual navigation. The maze is randomly generated. (1, top) The 3D visual navigation environment generated by illustrative x 7 map, where we highlight the panoramic view position (5, 3) with four RGB images (resolution 32 32 x 3). The entire observation tensor for this 7 x 7 example visual navigation environment is This used as the input to the Visual Navigation task. (2) Another predicted map in resolution 15 X 15 from mapper module, which is jointly optimized with planner network. We show the visualization different map used in actual training.
7 x 32
(3) The ground truth map in resolution 15 15. This is also used input to the 2D Navigation task and as target in the auxiliary loss for the Visual Navigation task (as done GPPN). (4) The predicted policy from an end- sondto--nd trained SymVIN model that uses mapper to take the observation images (formed as a tensor) and plans on a learned map. The red block is the goal position.
9) ruas gee yooea 1od
1000 DoU
YARD SoRa 9oa
pood Jou
pood Soa
pood THU
30 come 120
nX nom. o
nin con.
94 Nuaos grt sovea pog
seuic on
Traso ec cone bau
Pauure.o dn Splu
I0: Hapd ct conel sall
Up surril
Ree Black x coni
Figure 14: The U-net architecture we used as manipulation mapper.
During training, we pre-train the mapper and the planner separately for 15 epochs. Where the map- per takes manipulator workspace and outputs configuration space. The mapper is trained to mini- mize the binary cross entropy between output and ground truth configurations space. The planner is trained in the same way as described in Section 6. After pre- pre-training, we switch the input to the planner from ground truth configuration space the one from the mapper. During testing, fol- low the pipeline in Chaplot et al. (2021) that the mapper-planner only have access to the manipulator workspace.
36

---

Published as a conference paper at ICLR 2023
H.3 TRAINING SETUP
We try to mimic the setup in VIN and GPPN (Lee et al., 2018).
For non-SymPlan related parameters, we use learning rate of 10-3, batch size of 32 if possible (GPPN variants need smaller), RMSprop optimizer.
For SymPlan parameters, methods) to process the imput map. We use 100 hidden channels for Q-value for VIN (or 100 regular representations for SymVIN), and use 40 hidden channels for Q-value for GPPN and ConyGPPN (or 40 regular representations for SymGPPN on 15 x 15, and 20 for larger maps because of memory constraint).
150 hidden channels (or 150 rrivial representations for SymPlan
H.4 VISUALIZATION OF LEARNED MODELS
We visualize trained VIN and SymVIN. evaluated 15 15 map and its rotated version. For non-symin non-symmetric VIN Figure 15, the learned policy obviously not equivariant under rotation.
We also visualize SymVIN on larger map sizes: 28 x 28 and 50 50, to demonstrate its performance and equivariance.
Figure 15: A trained VIN evaluated on 15 x 15 map and its rotated version. It obvious that the learned policy not equivariant under rotation.
E
Figure 16: A trained SymVIN evaluated 15 15 map and its rotated version.
37

---

Published as a conference paper at ICLR 2023
Figure 17: A fully trained SymVIN evaluated on 28 x 28 map and its rotated version.
Figure 18: fully trained VIN evaluated on 50 x 50 map and its rotated version.
H.S SPL METRIC
We additionally provide SPL metric of the 2D navigation experiments with different seeds. The trends should be similar the results in the main paper.
Table 2: Averaged test success rate (%) over seeds for using 10K/2K/2K dataset on 2D navigation.
Methods
15 x 15
28 x 28
50 x 50
VIN
69.94:2.99
61.43 15.66
54.62 649.62.19.76
SymVIN
10.39
97.56.10.11
96.27 1..27
GPPN
96.48
9-1.35:1.14
88, 88.25:4-425
ConvGPPN
99.51 20.19
96.4448.66
96.62++035
SymGPPN
99.68 +0.00
99.98 +.0.02
99.91 4-0.05
The performance under SPL metric is pretty similar to success rate. This indicates that most paths are pretty close to the optimal length from all planners. Thus, we decided to not include this to avoid extra concept in experiment section.
H.6 ABLATION STUDY
Additional training curves. in the main text.
We also provide other training curves that we only show test numbers
38

---

Published as a conference paper at ICLR 2023
Table 3: Averaged test SPL (%) over 5 seeds for using 10K/2K/2K dataset on 2D navigation.
Methods
15 x 15
28 x 28
50 x 50
VIN
68.62+a.u2
4..77
54.14
SymVIN
98.08
97.23 +0.67
96.07 40.27
GPPN
95.55-20.61 6.
94.00
87.79
ConVGPPN
98.73
96.22
96.44.10.40
SymGPPN
98.91-9.01
99.76#00006
99.82+t0.06
1.0
0.8
Successuu Rate
0.6
model
VIN
0.4
SymVIN
GPNN
ConyGIPN
SymGPPN
Epochs
Figure 19: (Left) Accuracy evaluated unseen test maps. The x-axis is the width of the map, and the y-axis is the accuracy, reported on every map size and every size and every chose symmetry group G. (Right) Visual navigation 15 15 with 10K data.
Training efficiency with less data. Since the supervision is still dense, experiment on training with even smaller dataset to experiment more extreme setup. We experiment how symmetry may affect the training efficiency of Symmetric Planners by further reducing the size of training dataset. We compare on two environments: 2D navigation and visual navigation, with saining/validation/tee size of 1/200/2200, for all methods.
Choose of symmetry groups for navigation. One important benefit of partially equivariant net- work is that, we do not need to design the group representation of MDP action space PA(g) for different group action space. Thus, experiment several G-equivariant variants with different group equivariance: (discrete rotation group) C22 Ca. Cg, C16. and (dihedral group) D2, Da. Dx, all based on E(2)-steerable CNN (Weiler and Cesa, 2021). For all intermediate layers, representations (ge of each group, followed by final policy layer with non-equivariant x convolution.
regular
The results are reported in the Figure 19 (left). We only compare VIN (denoted as "none" symmetry) against our EVN)-VIN (other symmetry group option) on 2D navigation with 15 15 maps.
In general, the planners equipped with any G group equivariance outperform the vanilla non- equivariant VIN, and D4-squivariant steerable CNN performs the best most map sizes. Addi- tionally, since the environment has actions in 8 directions (4 diagonals), Css or Dg groups seem to take advantage of that and have slightly higher accuracy on some map sizes, while C16 over- constrained compared to the true symmetry G Da and be detrimental to performance. The non-equivariant VIN also experiences higher variance on large maps.
Choosing fiber representations. As use steerable convolutions (Weiler and Cesa, 2021) to build symmetric planners, we are free to choose the representations for feature fields, where interme- diate equivariant convolutional layers will be equivariant between them (p))(g)))) Pout (()). We found representations for some feature fields are critical to the performance: mainly V S and Q : S
We use the best setting as default, and ablate every option. As shown in Table 4, changing PV PQ to trivial representation would result in much worse results.
Fully vs. Partially equivariance for symmetric planners. One seemingly minor but critical design choice in our SymPlan networks is the choice of the final policy layer, which maps Q-values + RlAl to policy logits S + AI Fully equivariant is expected to perform better, but it has some points worth to mention. (1) We experience unstable training the beginning. where the loss can
39

---

Published as a conference paper at ICLR 2023
Successul Rate
0.0
Successtll Rate
0.8
model
model
0.6
SymVIN
VIN
GPPN
0.4
SymVIN
COnVGPPN
GPPN
SymGPPN
CanvCPPN
0.2
SymGPPN
0.0
10
15
20
25
30
10
15
20
25
30
Epochs
Epochs
Figure 20: Training curves on (Left) 2D navigation with 10K of 15 x 15 maps and on (Right) 2DoFs
manipulation with 10K of 18 18 maps in configuration space. Faded areas indicate standard error.
1.0
1.0
0.8
Successu Rate
Sucresstu nate
0.6
0.6
model
VIN
VIN
0.4
SymVIV
0.4
SymVIN
GPPN
GPPN
SymGPPN
SymGPPN
Epochs
Epochs
Figure 21: Training curves for (Left) 28
28 and (Right) 50 x 50.
1.0
0.8
0.8
Successful Rate
Successtul Rate
0.6
model
model
0.4
SymVIN
0.4
SymVIN
0.2
ConvGPPN
0.2
ConVGPPN
SymGPPN
SymMPPM
100
150
200
250
300
50
100
150
200
250
300
Epochs
Figure 22: Training curves for 15 x 15 2D navigation 1K data (Iefft) training and (Right) validation successful rate.
Sucressull Rate
Successful Rate
model
0.6
0.6
VIN
SymVIN
0.4
VIN
0.4
ConvGPPN
GPPN
SymGPPN
ConVGPPN
SymGPPN
0.0
200
400
600
800
1000
200
400
600
800
1000
Epochs
Epochs
Figure 23: Training curves for 15 x 15 visual navigation IK data (Left) training and (Right) vali- dation successful rate.
Table 4: Fiber representations
(Fiber representation)
SymVIN
Default
98.45
Hidden: trivial to regular
99.07
State-value py: regular to trivial
63.08
Q-value pQ: regular to trivial
21.30
PQ and pv: both trivial
2.814
40

---

Published as a conference paper at ICLR 2023
go up 10t in the first epoch, while did not observe it in non-equivariant partially equivariant counterparts. However, this only slightly affects training.
In summary, found
though fully equivariant version can perform slightly better in the best tuned setting, on average setting, partially equivariant version is more robust and the gap is much larger, shown in the follow table, which example of averaging three choices of represen- tations introduced in the last paragraph. On average partially equivariant version much better. In our experiments, partially equivariant version also is easier tune.
Table 5: Fully Partially equivariance
(Equivariance)
SymVIN
Partially equivariant averaged over all representations Fully equivariant averaged over all representations
91.04
42.61
Generalization additional experiment for fixed K. For fixed K setup in Figure 24 (left), we keep number of iterations to be K 30 and kernel size F for all methods.
1.0
algorithm
VIN
0.8
SymVIN
ConvCPPN
2vg success
0.6
SPM
8.6
SynVVNN GPAN
ConvENPN SymSPPN
20
80 100
20
60
80
100
size
size
Figure 24: Results for generalization larger maps for all methods. (Left) Fixed K 30 iterations. (Left) Variable K iterations, where K v2 M and M is the generalization map size (x-axis).
For VIN, it far surpasses VIN for all sizes and preserves the gap throughout the evaluation. Ad- ditionally, SIN has slightly higher variance across three random seeds (three separately trained models).
Among GPPN and its variants, SymGPPN significantly outperforms both GPPN and ConvGPPN. Interestingly. ConvGPPN has sharper drop with map size than both SymGPPN and ConyGPPN and thus has increasingly larger gap with SymGPPN and finally even got surpassed by GPPN. Across random seeds, the three trained models of ConyGPPN give unexpectedly high variance compared to GPPN and SymGPPN.
41


Reviews:
[{"review_id": "mwlLAySzVq1", "reviewer": "Reviewer_QPW3", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper proposes a novel steerable convolution for 2D path planning, where the convolution operation benefits from incorporating symmetric constraints to reduce overall search space and improve planning performance. Specifically, by manipulating the input signal in a symmetric setting (e.g. rotation and flipping for 2D space) and enforcing consensus outputs, the symmetric convolution can learn to reduce ambiguous prediction and therefore have better potential in both efficiency and accuracy. ", "summary_of_the_review": "I have several questions & concerns regarding the technique.\n1. E(2)-CNN also proposed a transformation invariant representation yield by the steerable convolution. What is the difference compared to them? Can E(2)-CNN apply to the same task with slight modification?\n2. Is the binary entropy loss applied to both raw and symmetric output? ", "strength_and_weaknesses": "Pros:\n- This paper provides a simplistic integration using VIN, which is easy to implement. \n- The motivation to use VIN and the effectiveness of symmetric constraint is discussed and proved (only in 2D grid case).\n- The experiment shows substantial improvements over baselines: VIN & GPPN.\n\nAdditionally, although I don't have experience in path & agent planning, I found this paper easy to understand.\n\nCons:\n- Differences should be highlighted compared to E(2)-CNN, as they also proposed a steerable convolution with kernel constraints. Can their steerable convolution be applied to the same task with a little modification?\n- Defining the generalized symmetric operation is challenging. Although the paper demonstrates the 2D grid case where the input signal (\\mathbb{Z}) is rotated 90 deg, extending to 3D or higher dimension might be more difficult. For example, if the Euler angle represents the rotation in 3D, rotating a 3D signal with multiple 90 deg will suffer the gimbal lock. ", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "Not applicable"}, "extra_text": {}}, {"review_id": "SDCqalzNyq", "reviewer": "Reviewer_hMHM", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper proposes to build differentiable planning methods that are equivariant to translations, rotations, and reflection transforms. This is done by replacing standard convolutional layers (which are equivariant to translations) in differentiable planning algorithms like VINs and GPPNs with steerable convolutional layers (which are additionally equivariant to rotations and reflections). The authors also theoretically show that value operators and value iteration on 2D grids are equivariant, and this can be expressed as a steerable convolution. Experiments are performed on 2D grid / visual navigation and 2 DOF manipulation and demonstrate strong improvements over non-equivariant versions of the planning methods.", "summary_of_the_review": "# Post-rebuttal update\nThe authors address the comments about the map sizes and optimality of paths for planning. I'm raising my rating to accept.\n\n# Pre-rebuttal review\nThe paper tackles an important problem and proposes to incorporate equivariances in differentiable planning methods. It provides both theoretical and empirical analyses and demonstrates strong improvements over prior work. However, the results are demonstrated only in simplistic maps and do not capture closeness of planned paths to the optimal path. ", "strength_and_weaknesses": "# Strengths\n* The paper focuses on an important problem and clearly motivates the need for equivariance for path planning (with good illustrative examples).\n* The authors theoretically analyze the problem and provide key theorems to support the proposed algorithm\n* The experiment setup is sensible and uses good baselines\n* The results are strong and demonstrate consistent improvements over the baselines\n\n# Weaknesses\n* The results are only demonstrated only small and toyish maps (100 x 100 or smaller). In robotics planning, the maps can be significantly larger (e.g., 960 x 960 in Active Neural SLAM, Chaplot et al., ICLR 2020). It is unclear how the proposed method works on more complex scenarios.\n* The metrics in Table 1, Figures 6 and 7 only capture success rate and not efficiency of the predicted plan relative to an optimal planner like A*. A common metric to measure this efficiency is SPL (Success weighted by Path Length) from Anderson et al., 2018: https://arxiv.org/pdf/1807.06757.pdf", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "Q1XoQlLHmk", "reviewer": "Reviewer_qcZN", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper demonstrates one way to exploit the rotation and reflection symmetry structure in path planning problems. More specifically, the authors use Value Iteration Networks (VINs) to solve the path finding problem on the Z2 space. They show that the steerable convolution can incorporate the symmetry structure in the value iteration process. Experiments show that their proposed symmetry regularization can improve the training speed, training success rate, as well as generalizability.", "summary_of_the_review": "This paper presents an elegant symmetry regularization to the VIN-based path finding pipeline. The methodology part reads very well and experiments are sufficient to back up their technical novelties. In summary, I think this paper looks good to me.", "strength_and_weaknesses": "This paper flows very well and smoothly. The idea of exploiting the rotation and reflection symmetry in the path finding problem is simple but also elegant. And it is a good choice to start from Z2 space and VIN with steerable convolution. The authors formally build the connections among symmetries, value iterations, and steerable convolutions. Moreover, the experiment results further verify that this symmetry regularization does help.\n\nBesides those strengths, I might have some questions in mind. \n\nFirst, although it is intuitive that symmetries could improve the performance and the experiments also demonstrate the effectiveness of such proposed inductive bias empirically, is it possible to prove or even compute quantitatively why symmetry can help the path finding problems? \n\nSecond, it is also mentioned that the symmetry structure also exists in other higher-dimensional continuous Euclidean spaces or spatial graphs (of path finding problems), can the authors provide more clues on how to generalize their method to those cases? Since the current experiments are on small scales and those mentioned more general spaces could be more practical in some real-world problems. \n\nThird, in Section 6.3, why is iteration K set to be sqrt(2)M? And why is the iteration linear to the map size?\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Decision: Accept: poster