SE(2)-Equivariant Pushing Dynamics Models for Tabletop Object Manipulations
Authors: Seungyeon Kim
Byeongdo Lim
Yonghyeon Lee
Frank C. Park
Venue: CoRL 2022 Oral (2022)
Keywords: Pushing dynamics learning
Pushing manipulation
Symmetry and Equivariance

Abstract: For tabletop object manipulation tasks, learning an accurate pushing dynamics model, which predicts the objects' motions when a robot pushes an object, is very important. In this work, we claim that an ideal pushing dynamics model should have the SE(2)-equivariance property, i.e., if tabletop objects' poses and pushing action are transformed by some same planar rigid-body transformation, then the resulting motion should also be the result of the same transformation. Existing state-of-the-art data-driven approaches do not have this equivariance property, resulting in less-than-desirable learning performances. In this paper, we propose a new neural network architecture that by construction has the above equivariance property. Through extensive empirical validations, we show that the proposed model shows significantly improved learning performances over the existing methods. Also, we verify that our pushing dynamics model can be used for various downstream pushing manipulation tasks such as the object moving, singulation, and grasping in both simulation and real robot experiments. Code is available at https://github.com/seungyeon-k/SQPDNet-public.



SE(2)- Pushing Dynamics Models for Tabletop Object Manipulations
Seungyeon Kim! Byeongdo Lim! Yonghyeon Leel Frank Chongwoo Park1.2
'eouul National University, "Saige Research
(ksy, bdlim, yhlee @robotics........ snu. ac. krr,, fcpêsnu. ac.kkr kr
For tabletop object manipulation tasks, learning an accurate pushing dynamics model, which predicts the objects' motions when robot pushes object, is very important. In this work, we claim that an ideal pushing dynam- ics model should have the SE(()--quivariiace property, i.c., if tabletop objects' poses and pushing action are transformed by some same planar rigid-body trans- formation, then the resulting motion should also be the result of the same transfor- mation. Existing tfateof-the--a data-driver approaches do not have this equiv- ariance property, resulting in less- less-than-Te tee-tthanneeee -ea--deieabble learning performances. In this paper, we propose a new neural network architecture that by construction has the above equivariance property. Through extensive empirical validations, we show that the proposed model shows significantly improved learning performances over the existing methods. Also, we verify that our pushing dynamics model can be used for various downstream pushing manipulation tasks such the object mov- ing, singulation, and grasping in both simulation and real robot experiments. Code is available at https://github. https//gituub.oom/eunggeoo
Abstract:
Keywords: Pushing dynamics learning, Pushing manipulation, Symmetry and Equivariance
1 Introduction
Robotic visual pushing manipulation - by visual manipulation, we mean that only visual obser- vations (e.g., depth camera) are available in cluttered environments including unseen objects is an important yet challenging manipulation skill that allows robot to interact with and change its environment to be suitable for performing downstream tasks. For example, pushing manipulation techniques have been used to move tabletop objects graspable [1, 2, 3, 4J, rearrange multiple objects for sorting [5, 6, 7, 8), and find target object occluded by the other objects [9, 10).
We consider model-based approaches for the pushing manipulations that consist of the fol- lowing two components: (i) construct a push- ing dynamics model which predicts the motions of the objects after a robot performs a push- ing action to the environment and (ii) to find an optimal sequence of pushing actions that achieves the goal given predesigned task cri- teria [11, 12]. Our primary focus is the first step which is to develop an accurate visual pushing dynamics model that takes a visual observation input. Analytic approaches that precisely model the physical interactions [13, 14, 15, 16] cannot be used since jects with only vision data.
Scene
Scene 2
Figure The box object and pushing vector in Scene are transformed by some same planar rigid-body transformation as those in Scene 2. An ideal pushing dynamics model should be SF((2)- cquivariant, i.c., the resulting motion in Scene is transformation of that in Scene 1.
given unseen ob-
Recently, there has been considerable interest in data-driven methods for learning pushing dynam- ies models [17, 18, 19, 20, 21, 22], but their generalization performances are still far-less-than- satisfying. We claim that one of the important reasons behind this is that neural network models
6th Conference on Robot I Learning (CoRI. 2022), Auckland, New Zealand.

---

used in existing approaches lack considering the symmetry of the physical systems, and more pre- cisely, equivariance. For example, suppose a model is trained with an experience where a robot pushes a box object into red arrow direction as shown in Figure 1 (Scene 1). And consider new situation where the same box object is located at different pose and the robot pushes the object in the same relative direction as shown in Figure 1 (Scene 2). At an intuitive level, a good model should be able to easily generalize to this type of new situation, where tabletop objects are only translated or rotated along the z-axis. In more technical terms, the pushing dynamics model needs to be equivariant to the SE(2) transformation.
In this paper, we define the SE(2) -qquivraava pushing dynamics model and deliberately design a neural network architecture that by construction has the equivariance property. The core idea to make the model equivariant is to properly transform the coordinates of the pushing action and the objects' poses as needed; details are elaborated in Section 2. This construction naturally captures the symmetry of the physical systems and significantly improves the generalization performances.
To employ the proposed equivariant pushing dynamies model in environments with only vision data and unseen objects, we need an additional module that can recognize the objects' shapes and poses. In this work, we represent 3d objects' shapes by using the shape class called the superquadrics, which can express diverse shapes ranging from boxes, cylinders, and ellipsoids to other complex symmetric shapes. We train the recognition network that predicts the objects' shapes with su- perquadrics by adopting an idea from [23]. We call our superquadric object representation-base pushing dynamics model a SuperQuadric Pushing Dynamics Network (SQPD-Net).
Experiments and benchmark comparisons against the existing state-of-the-art methods confirm that our dynamics model achieves the highest performance in predicting objects' motions after pushing action. In addition, we validate the effectiveness of our model by using it for model-based optimal controls for various pushing manipulation tasks in both simulation and real-world experiments.
2 SEiia-Equvvariant Pushing Dynamics Models
In this section, we develop a neural network architecture specialized to learn a SE(2) pushing dynamics model. We assume that multiple rigid-body objects are placed on the table whose surface is assumed to be flat and orthogonal to the gravity direction, and the robot interacts with the objects by pushing manipulations. Each object is represented by a pose parameter T E SF(3) (4 x matrix representation) and shape parameter q. where the pose parameter is described with respect some global fixed frame and the shape parameter is a vector. And the pushing action is defined as a tuple (p, v) where the tip of the end-effector moves from the position E IR to + v E R. As the tip of the end-effector moves, the robot can have contact with environments, pushes objects, and changes the poses of the objects.
Further, assume there maximally M rigid-body objects on the table that have the parameters {(T;, 9:) ) for N < M. We consider a discrete-time pushing dynamics model ʃ that outputs the object's transformed poses [T when a pushing action (p, v) is applied, i.e., [T = f(((T,, (:) ) (p. v)). where N can vary as long as N S M. Assuming the gravity direction the z-axis, we first give a precise definition of the -qquivvraa pushing dynamics model:
Definition A pushing dynamics model f is -SQ()e-qquvariimm if
{CT f(((CT,, qi) (Rot(2. ()pp + Rot(2, (2, f)v))
(1)
for all object numbers N < M and rigid-body body transformations C that have the following
[Rot(2,0) 0)
txy
C= c
(2)
where Rot(2, 8) is a 3 x 3 rotation matrix for rotations around z-axis and txy = (tz, ty. 0) E R
To build a SE(2)-equivanl) composition method that decomposes an object pose T; E SE(3) to a pose projected to the table surface denoted by C, E SE(3) and the relative rigid-body transformation U, SF(3) such that T, C,U,
neural network architecture, we first introduce an object pose de-
2

---

Scene before action
Action encoder
Scene after action
MLP
Concatenated feature Y:
C (p.v)
TgöT;
Ego encoder
MLP
(U, q:)
Scene encoder
b,
MLP,
C
(C, TT)
Motion predictor
MLP;
MLP,
8T.
Action (p, v)
(0, (T..)))
Max pooling
Figure 2: SE(e)-equivarian pushing dynamics neural network architecture for an i-th object, fi.
Object Pose Decomposition. Given object pose T E SE(3), we decompose it two 4 x matrices C, U SE(() as visualized Figure First, C defined by projecting to the table surface, which has the form in equation (2). And secondly, U is defined as C-IT. More details are in Appendix B.
Now, we explain our network architecture for the pushing dynamics model f: overall archi- tecture is described in Figure 2. The model defined L where each f; outputs the i-th object's transformed pose, i.e., TV fi([(T, 9t) 9 Jー=1, (p, v)). For f.. we first de- compose the i-th object pose T. C,U, and transform the other objects' poses (including it-
T2
U2
Base
U
self) and pushing action as follows: (i) T3 C-2T, for j = 1, N and (ii) (p.v) C (p, = (R. p RTt, RT v) where R. and t; are rotation matrix and translation vector parts of C. Then, three different multi-layer perceptron (MLP) networks are used to extract SE(2)-invariant feature vectors: (i) the MLP; takes the transformed action C; (P, v) and outputs feature vector a,, (ii) the MLP, takes the i-th ob- ject's parameter (U,, q:) and outputs a feature vector b,, and (iii) the MLP3 takes the transformed object's parameters (C, .Ty, qj) and outputs a feature vector cf for all j = 1.....
Table
C2
Figure 3: Object Pose Decomposition.
and then these
output vectors pass through some permutation invariant function h as C; = h(c!. .... ..,),, such as the element-wiss max pooling. These feature vectors concatenated y: (a, b, c,). and have the last layer MLP, that takes y; and outputs 8T; E SE(3). We note that these MLP layers shared across all i 1, 1,... N. Then, the transformed poses defined TV T,OT, for all i = 1,.......... N. As a result, this dynamics model is -uuuiiieqi by construction; the proof is in Appendix B.
for some < M and a = (p, v). In this paper, we train the pushing dynamics model given set of 3-tuples ((s, a. 1 where T₃ the next pose of the i-th object. The loss function L is defined by comparing the ground-trui next poses (TD), and the predicted poses (T
Training. Denote by s = ((T.,):)) 9.)
f(s, a) as follows:
C() Σ 116--11 L (I., R RR))
where dso(3) is distance measure between rotation matrices, I is 3 x 3 identity matrix, RC, R and t², ? are rotation matrices and translation vectors parts of Tv, Ty respectively, and a weighting parameter (for later experiments set to 0.1). The details about the used distance measure are in Appendix C.
3

---

3 Object Recognition-based Pushing Manipulations
If we have known objects and can easily estimate the poses of the objects, then it is straightforward to use the learned pushing dynamics model for pushing manipulations. However, for unseen objects, first need to recognize the objects' shapes and poses. Therefore, our overall framework consists of the following two steps: (i) to recognize objects' shapes and poses and (ii) to push objects by using the learned pushing dynamics model and pre-designed task criteria, of which details are explained in the following subsections.
3.1 Object Shape and Pose Recognition via Superquadrics
We propose to use implicit functions to represent 3d objects' shapes. In general, an implicit object surface representation is defined by level set of a function S(z, y, q, T) = O, where q is a shape parameter and T E SE(3) is pose parameter. In our framework, any implicit function approximation model (zx y, 2; q, T) can be used.
In this work, we employ the shape class called the superquadries, a family of geometric shapes that resemble ellipsoids and other quadrics, which can be used to represent diverse shapes ranging from boxes, cylinders, and ellipsoids to bi-cones, octahedra, and other complex sym- metric shapes. The implicit equation for su- perquadric surface at = I, (I4 is x 4 iden- tity matrix) has the following form:
Box
Cylinder
Ellipsoid
Octahedron
Bicone
0.2
0.2
1.0
2.0
6, 2.0
0.2
e2
1.0
1.0
0.2
6 = 1.0
Figure 4: Examples of superquadrics.
y
S(x y. 2; 4.)
0,
(4)
03
where q (01, (01,a2, d2, 03 E1, e2) Is is the shape parameter. In particular, 01, 02 0z controls the sizes and c1, E2 controls the geometric shapes. Some examples are shown in Figure 4. At T + 14, the equation S(x, y, q. T) can be written with the passive coordinate transformation of (zz y, by T, i.e. S(x, y, 2: q, T) = S(T (I. y, y, q. L4); see Appendix C for details.
The object recognition problem that we address in this paper can then be posed as follows: given visual input obtained from a depth camera that typically contains partial views of the objects, we need to predict the superquadric parameters (q. T) for each object. To bridge the gap between synthetic and real-world vision sensor data, we add noise to the visual input as done [24, 25]. The predicted object represented by (q. T) should fit the full object, although only partial view of the object is given as input. This problem has been recently tackled by [23]. where two neural network models that take point cloud data inputs are employed: (i) object segmentation network [26] and (ii) object full shape and pose recognition network [23]. We include details about the visual input noise, the network architectures, and the training methods of these networks in Appendix C.
We call our SE(2) (E())-equivariann pushing dynamics model that uses the superquadric representation a SuperQuadric Pushing Dynamics Network (SQPD-Ner).
3.2 Model-based Pushing Manipulations
Given visual observation of tabletop objects point cloud which we denote by o, goal is to find a sequence of robot pushing actions (a1, az, ay) that changes the environment for some given task. In this section, we assume that objects' poses and shapes, i.e. R(o:) = S, (throughout, we denote by s, = [(TT,i, )..)) and (ii) pushing dynamies model S a.). Given task-specific objective function J, solve the following optimal control problem:
given (i) recognition module R that outputs the
min J(01, a1, AT) Σ rst, at) ((S++1) s.t. R(o1), S f(st, a(). (5) an.... arr
For tasks we focus in this paper, we set (s:,, at) = 0 and only use terminal cost function (($++1)- We use the sampling- baaee MPCs [12] (implementation details are in Appendix D). Below, we

---

introduce three terminal cost functions for the following pushing manipulation tasks: (i) moving, (ii singulation, and (ii) grasping. We denote the translation vector and rotation matrix parts of the transformation matrix T (.) as {()) (.). respectively.
Moving is task to move objects to their desired poses. The desired poses are given as (Ta then we define a terminal cost function as
N
((T++))
= Σ (1LT+I,)
- La,i11 3 Rr.1,))
(6)
Am
Singulation is task to separate objects by more than certain distance т. We define terminal cost function as
min (min(Itt+1i-taa tr+15 -T))) (STT+1) (Ir))el.rNN)l Hiss)
Grasping is task to make a target object graspable. Given target object index i, generate candidate grasp poses for the recog- nized target object as shown in Figure and check collisions with the environment and the other recognized objects: green grasp poses are collision-free and red poses are not. The ter- minal cost (ST++1 defined to 0 if least one collision-free grasp pose exists and I other- wise. Further details are provided Appendix C.
Pre-defined grasp poses
Collision-free grasp poses
Target object
Figure 5: Sampling- Sumpling-based based grasping criteria.
4 Experiments
In this section, we empirically show that (i) our proposed pushing dynamics model, the SQPD- Net, outperforms the existing state-of-the-art data-driven pushing dynamics models, and (ii) our SQPD-Net can be used for various downstream pushing manipulation tasks, c.g., object moving, singulation, and grasping.
Environment. We use the 7-dof Franka Emika Panda robot with a parallel-jaw gripper and an Azure Kinect DK camera sensor mounted on the gripper. The raw input visual observation is depth image, which is then pre-processed to other representations (e.g., point cloud) as needed.
Pushing Manipulation Dataset. To train pushing dynamics models, we generate a pushing manipulation dataset in simula- tion (Pybullet). Throughout our experiments, we use cylinder- shaped and cube-shaped objects of various sizes, and one scene contains less than 5 objects. To execute an action (p, v) E RR (i) the robot first moves so that the gripper's ' tip is placed at p and its orientation is set as visualized in Figure 6, and then (ii) the robot moves in a way that the gripper's 's tip moves to p with a fixed orientation. We generate the pushing manipula- tion dataset as follows: (i) we place random objects random poses in the workspace, (ii) we sample action (p, v) E TR. where p is sampled near one randomly selected object and v directs the center of the object, and (ii) we execute the robot pushing action. this process, we note that the gripper's other parts than the tip can also make contact with the environment. More details are included in Appendix D.
15e
Gripper
Object
Figure 6: Execution of pushing
action.
Baseline Methods. We compare our SQPD-Net with the following baseline methods: 2DFlow and SE3-Net adopted from [17], SE3Pose-Ner adopted from [18], and 3DFlow and DSR-Ner adopted from [20]. The 2DFlow, SE3-Net, and SE3Pose-Nes take an organized point cloud as a visual input and predict the flow vectors of the points. The 3DFlow and DSR-Net take a voxelized truncated signed distance field (TSDF) as a visual input and predict the voxel flow. Our SQPD-Net takes the estimated objects' poses and superquadric shape parameters input and predicts the objects' next
5

---

poses. While. in the existing approaches, the models directly predict motions from the pee- prre-esooee raw visual observations, our model consists of two modules: (i) pre-trained recognition network R that predicts objects' poses and shape parameters and (ii) the SQPD-Net that predicts the objects' next poses. We denote these two networks together by R-SQPD--Net. For the comparison purpose, we also test the case where the ground-truth objects' poses and shape parameters are used as an input for the SQPD-Net and denote by GT-SQPD-Net.
Evaluation Metrics. Throughout, we use two types of evaluation metrics for the learned pushing dynamics models: (i) flow error (the lower the better) and (ii) mask intersection over union (mask IoU, the higher the better). First of all, we consider the visible and full flow errors The visible flow error is the root mean squared error (RMSE) between the ground -ruthh flows and predicted flows of the points on the visible surface of the objects, while the full flow enror is the RMSE computed with all points from the objects' volumes. Second, we consider the 2D and 3D mask IoUs. The 2D mask loU is computed by using the depth images and thus only visible surfaces are taken into consideration. On the other hand, the 3D mask IoU is computed with the complete 3D occupancy grid. The full flow error and mask IoU cannot be computed in 2DFlow, SE3-Net, and SE3Pose-Nee because they do not estimate the complete objects' shapes as an intermediate step of the prediction of the pushing dynamics.
4.1 Pushing Dynamics Learning
In this section, we first empirically verify the equivariance property of our method and show the performance advantages of ours over the existing methods.
Equivariance Study. For the purpose of testing the equiv-
METHOD
visible flow (L)
ariance of the models, we design the following experi- ment: we train the models with only one pushing manip- ulation data 3-tuple {o, a, o') where o and o' are cur- rent and next observations respectively and a is pushing action that the models overfit the given data. Then, we compare the models' generalization capabilities with test data that are generated by applying random SE(2)- transformation to the data. An ideal equivariant model should produce almost zero error in the test data.
2DFlow [17]
4.73
SE3-Net [17]
4.73
SE3Pose-NG [18]
4.72
R- R-SQPD-Net (ours)
0.73
GT STTSSQPD-Net (ours)
0.02
Table 1: Test visible flow error (em).
Table 1 shows average visible flow errors of the baseline methods and SQPD-Nets, obtained by running the above experiment multiple times with different training data (details in Ap-
Ground-truth
SE3Pose-Net
R-SQPD-Net
Training
pendix D). The 3DFlow and DSR-Net are omit- ted in this experiment because they cannot make estimations if the transformed actions do not belong to the pre-defined discrete of actions. The GT-SQPD-Net produces al- most zero error as expected while the R-SQPD- Net produces little error originating from the recognition error. Our SQPD-Nets are much more SE(2)-equivariant compared to the exist- ing works. Figure shows an example pre- diction result from the SE3Pose-Net and R- SQPD-Net: the blue bounding box represents
Test
Figure 7: Depth images of prediction results. For SE3Pose-Net, after the point cloud moves, the space occupied before is colored black.
the ground-truth next pose of the object. For the test data, the SE3Pose-Net predicts completely wrong motion. More example figures of experimental results are provided in Appendix D.
Pushing Dynamics Learning. We compare the learning performances of the SQPD-Net and the baseline methods with large-scale pushing dataset where the rraiiing/validationnest data consist of 12000. 1200, and 1200 numbers of 3-tuples (fo. a, o')), respectively. Figure 8 shows the predicted depth images and 3D masks for an example pushing data the test dataset. As shown in the ground-truth motions (left of Figure 8), the red, green, and gray objects are in contact with each other and these three objects move together when the red object is pushed. In this case, our R- SQPD-Net only successfully predicts the complex interactive motions of the objects. Table 2 shows

---

2DFlow
SE3-Net
SE3Pose-Net
3DFlow
DSR-Net
R-SQPD-Net
action
Depth
3D mask
action
Figure 8: Depth images and 3D masks of the ground-truth next scene and predicted scenes. Upper: Depth images where the blue bounding boxes represent the ground-truth next poses of the green and gray objects. Lower: (i) (incomplete) 3D masks converted from the depth images for 2DFlow, SE3-Net, and SE3Pose-Net and (ii) predicted complete 3D masks for 3DFlow, DSR-Net. and R- SQPD-Net.
Known
Unknown
Flow сπτоr (L)
Mask IoU
Flow error (C)
Mask IoU
METHOD
visible
full
2D
3D
visible
full
2D
3D
2lllow [17]
2.179
2.180
SE3-Net [17]
1.631
1,701
SE3Pose-Ner [18]
1.639
1.712
3DFlow [20]
1.818
1,859
0.747
0.699
1.697
1.719
0.755
0.698
DSR-Net [20]
1.325
1.331
0.720
0.705
1.531
1.524
0.665
0.632
R-SQPD-Net (ours)
0.575
0.610
0.844
0.798
0.710
0.726
0.834
0.781
GT-SQPD-Net (ours)
0.519
0.379
0.903
0.888
0.638
0.485
0.888
0.868
Table 2: Evaluation metrics computed within test dataset (the unit of flow error is cm).
the evaluation metries computed within the test data and shows that our SQPD-Nets outperform the other baseline methods by significant margins. Further experimental results with more example figures are provided in Appendix D.
4.2 Pushing Manipulation using R-SQPD-Net
In this section, we use the R-SQPD-Net trained in Section 4.1 and conduct the pushing manipu- lation tasks introduced in Section 3.2 (moving, singulation, and grasping) in both simulation and real-world. For the real-world experimen- tal setup,
Robot environment
Real-world objects
CHEEL-IT
various box- cylinder-like objects as shown in Figure 9; the same objects are used in simulation experiments. Since we directly apply the R-SQPD-Ne trained in sim- ulation to the real physical environment, it is reasonable to ask about the sim-to-real transfer issue. In our experiments, we use slow pushing motions to generate quasi-static movements of the objects and thus minimize the sim-to-real gap (for quasi-static object movements, the dynamical properties of the objects and environment, e.g.. mass, friction coefficient, become less affective [27]).
JELLO
Figure 9; Real-world experimental setting.
Figure 10 shows some real-world manipulation results for various tasks. For the moving task (first row), we set the desired positions ta, as (0.3, to.i.g. to, and / = 0 in equation (6). For the singulation task (second row), we set and fourth row), we sample about 15 to 30 candidate grasp poses for the target recognized objects. For all three examples, our approach can find a series of pushing actions that successfully perform the desired tasks. Notably, for the grasping tasks, without using ad hoc objective functions, the robot realizes how to re-configure the objects that feasible grasp poses can be found for the target objects: (i) the robot pushes the large and flat object to the edge of the table and (ii) the robot pushes surrounding objects to make the surrounded target object graspable.
20 (cm) in equation (7). For the grasping tasks (third

---

t=0
t=2
t=T+1
Scene
Recognition
Scene
Recognition
Scene
Scene
Grasping Singulation Moving
Figure 10: Real-world manipulation results using R-SQPD-Net for moving, singulation, and grasp- ing tasks (for the fourth row case, the target object is the cylinder surrounded by the three cubes). The red arrow at each recognition step means the optimal pushing action.
Failure Cases. Table 3 shows the manipulation suc- cess rates in simulation and real-world experiments. We design 10 test scenarios for each task, of which object configurations are in Appendix D. A few failure cases occur, whose underlying reasons we observe can be roughly categorized as (i) failure of shape recog- nition (simulation, real) and (ii) sim-to-real transfer issue (real). Details can be found in Appendix D.
TASK Moving Singulation
Simulation
Real
9/10
8/10
9/10
8/10
Grasping clutter
4/5
4/5
Grasping large
4/5
3/5
Table 3: Simulation and real-world manip- ulation results.
4.3 Limitations and Future Directions
First, since SQPD-Net considers single superquadric-shapel objects, it is not trivial apply it di- rectly to more complex non-convex shapes. As the researches representing objects in multiple superquadrics progress [28, 29], extending our approach to multiple superquadric-shape objects remains a future work. Second, the dynamics prediction task becomes challenging when pushing an object with non-uniform mass distribution since different mass distributions will lead to different motions. In this case. if specified poses in CAD models), our distribution. Since predicting reference poses is casy with only depth images, this is limitation of our approach. As one possible solution, additional information such as RGB images should be utilized [30, 31]. Lastly, there could be ply: in this case, our approach can be detrimental. One example is that the friction coefficients are different different regions of the table. As a research direction overcome this, a locally SE(()- equivariant model SEE(2) space is divided into several subspaces and the model is equivariant only within each subspace be considered.
consistently predict the reference poses of the objects (e.g., pre-
model applicable regardless of the mass
situations where the (ee))--uivar does ap-
Conclusion
This paper has proposed a representations of object shapes, we have proposed SuperQuadric Pushing Dynamies Network (SQPD-Net). Through extensive empirical validations, we confirm that the SQPD-Net significantly outperforms the existing state-of-the-art visual pushing dynamics models. Moreover, we have veri- fied that the SQPD-Net can be used for various pushing manipulation tasks.
pushing dynamics model. Using the superquadric

---

Acknowledgments
This work was supported in part by SRRC NRF grant 2016RIA5A1938472. IITP-MSIT grant 2021- 0-02068 (SNU AT Innovation Hub), IITP-MSIT grant 2022-0-00483 (Training and Inference Meth- ods for Goal-Oriented AI Agents), SNU-AIIS, SNU-IAMD, SNU BK21+ Program in Mechanical Engineering, SNU Institute for Engineering Research, Samsung Research, and Samsung Electronics Co.,Lid.
References
[1] A. Zeng, S. Song, S. Welker, J. Lee, A. Rodriguez, and T. Funkhouser. Learning synergies between pushing and grasping with self-supervised deep reinforcement learning. In 2018 TEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 4238- 4245. IEEE, 2018.
[2] M. Danielczuk, J. Mahler, C. Correa, and K. Goldberg. Linear push policies to increase grasp access for robot bin picking. In 2018 IEEE 14th International Conference on Automation Science and Engineering (CASE), pages 1249-1256. IEEE, 2018.
[3] K. Xu, H. Yu, Q. Lai, Y. Wang, and R. Xiong. Efficient learning of goal-oriented push-grasping synergy in clutter. IEEE Robotics and Automation Letters, 6(4):6337-6344, 2021.
[4] M. Kiatos and S. Malassiotis. Robust object grasping in clutter via singulation. In 2019 International Conference on Robotics and Automation (ICRA), pages 1596-1600, IEEE, 2019.
[5] W. Yuan, K. Hang. D. Kragic, M. Y. Wang, and J. A. Stork. End-to-end nonprehensile rear- rangement with deep reinforcement learning and simulation-to-reality transfer. Robotics and Autonomous Systems, 119:1 19-134, 2019.
[6] A. H. Qureshi, A. Mousavian, C. Paxton, M. C. Yip, and D. Fox. Nerp: Neural rearrangement planning for unknown objects. arXiv preprint arXiv:2/06.01352, 2021.
[7] E. Huang, Z. Jia, and M. T. Mason. Large-scale multi-object rearrangement. 2019 Interna- tional Conference on Robotics and Automation (ICRA), pages 21 1-218. IEEE, 2019.
[8] H. Song, J. A. Haustein, W. Yuan, K. Hang. M. Y. Wang, D. Kragic, and J. A. Stork. Multi- object rearrangement with monte carlo tree search: A case study on planar nonprehensile sort- ing. In 2020 IEEE/RSJ International Conference Intelligent Robots and Systems (IROS), pages 9433-9440. IEEF, 2020.
[9] Y. Yang, H. Liang, and C. Choi. A deep learning approach to grasping the invisible. JEEE Robotics and Automation Letters, 5(2):2232 22392, 2020.
[10] M. Danielezuk, A. Angelova. V. Vanhoucke, and K. Goldberg. X-ray: Mechanical search for an occluded object by minimizing support of learned occupancy distributions. In 2020 IEEE/RSJ International Conference Intelligent Robots and Systems (IROS), pages 9577-9588 9584. IEEE, 2020.
[11] M. LaValle and J. J. Kuffner Jr. Randomized kinodynamic planning. The international journal of roboties research, 20(5):378-44) 47778-400.22 2001.
[12] A. Nagabandi, G. Kahn, R. S. Fearing, and S. Levine. Neural network dynamics for model- based deep reinforcement learning with model-free fine-tuning. 2018 IEEE International Conference Robotics and Automation (ICRA), pages 7559-7566. IEEE, 2018
[13] M. T. Mason. Mechanics and planning of manipulator pushing operations. The International
Journal of Robotics Research, 5(3):53-7,, 1986.
[14] K. M. Lynch. Estimating the friction parameters of pushed objects. In Proceedings of 1993 IEEE/RSJ International Conference Intelligent Robots and Systems (IROS volume 1, pages 186-193. IEEE, 1993.
[15] K. M. Lynch and M. T. Mason. Stable pushing: Mechanics, controllability, and planning. The international journal of robotics research, 15(6):533-556 1996.

---

[16] J. Zhou, Y. Hou, and M. T. Mason. Pushing revisited: Differential flainess, trajectory plan- ning, and stabilization. The International Journal of Roboties Research, 38(12-13)1147-1448,, 2019.
[17] A. Byravan and D. Fox. Se3-nets: Learning rigid body motion using deep neural networks. In 2017 IEEE International Conference on Robotics and Automation (ICRA), pages 173-180. IEEE, 2017.
[18] A. Byravan, F. Leeb, F. Meier, and D. Fox. Se3-pose-nets: Structured deep dynamics models for visuomotor control. In 2018 IEEE International Conference on Robotics and Automation (ICRA), pages 3339-3346. IEEE, 2018.
[19] Y. Ye, Gandhi, Gupta, and Tulsiani. Object-centric forward modeling for model predictive control. In Conference on Robot Learning, pages 100-109. PMLR, 2020.
[20] Z. Xu, Z. He, J. Wu, and S. Song. Learning 3d dynamic scene representations for robot manipulation. arXiv preprint M.Ni::20/1.016666 2020.
[21] Wang, C. Hu, Y. Wang. and Y. Zhu. Dynamics learning with object-centric interaction networks for robot manipulation. JEEE Access, 9:68277-68288 2021.
[22] B. Huang, S. D. Han, A. Boularias, and J. Yu. Dipn: Deep interaction prediction network with application to clutter removal. In 2021 IEEE International Conference on Robotics and Automation (ICRA), pages 4694-4701. IEEE, 2021.
[23] Kim, T. Ahn, Y. Lee, Kim, M. Y. Wang, and C. Park. Dsqnet: deformable model- based supervised learning algorithm for grasping unknown occluded objects. IEEE Transac- tions on Automation Science and Engineering. 2022.
[24] Y. Lee, J. Back, Y. M. Kim, and F. C. Park. Imat: The iterative medial axis transform. In Computer Graphics Forum, volume 40, pages 162-181. Wiley Online Library, 2021.
[25] Y. Lee, S. Kim, J. Choi, and F. Park. A statistical manifold framework for point cloud data. In International Conference on Machine Learning, pages 12378-12400 PMLR, 2022.
[26] Y. Wang, Y. Sun, Z. Liu, S. E. Sarma, M. M. Bronstein, and J. M. Solomon. Dynamic graph for learning point clouds. Acm Transactions On Graphics (tog), 38(5):1-12, 2019.
[27] Z. Xu, Wu, Zeng. J. Tenenbaum, and S. Song. Densephysnet: Learning dense physical object representations via multi-step dynamic interactions. arXiv preprint arXiv:1906.00 03855, 2019.
[28] D. Paschalidou, A. O. Ulusoy, and A. Geiger. Superquadrics revisited: Learning 3d shape parsing beyond cuboids. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pages 10344-10353, 2019.
[29] W. Liu, Wu, S. Ruan, and G. S. Chirikjian. Robust and accurate superquadric recovery: a probabilistic approach. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pages 2676-2685, 2022.
[30] Y. Xiang, T. Schmidt, V. Narayanan, and D. Fox. Posecnn: A convolutional neural network for 6d object pose estimation in cluttered scenes. arXiv preprint arXiv:1711.00199, 2017.
[31] C. Xie, Y. Xiang, A. Mousavian, and D. Fox. Unseen object instance segmentation for robotic environments. IEEE Transactions on Robotics, 1343-113431335) 2021.
10


Reviews:
[{"review_id": "sytkTYDK5Lr", "reviewer": "Reviewer_PwZa", "summary": "The paper studies the SE(2)-equivariance in robotic pushing and proposes a pushing dynamics model that is designed to be SE(2)-equivariant. Experimental results in both simulation and the real world demonstrate that 1. the proposed model achieves high-precision prediction; 2. the model can be used in various tasks.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n\n1. The paper studies an interesting problem that utilizes SE(2)-equivariance to improve a pushing dynamics model.\n2. The paper has comprehensive experimental evaluation.\n3. The paper has very nice figure illustrations.\n\nWeaknesses:\n\n1. The paper does not have a limitation section (though there is a future direction subsection in Section 5).\n2. In my opinion, the idea of performing a transformation on the input before evaluating through the network to implement equivariance is not new [1][A], though the prior works are image-based and perform transformation w.r.t. the gripper frame, but this paper is vector-based and perform transformation w.r.t. the object frame. A proper comparison would be beneficial in a related work section.\n3. The paper does not discuss related works in equivariant learning and other works that implement equivariant learning in manipulation [B][C][D].\n\n\n[A] Ha, Huy, and Shuran Song. \"Flingbot: The unreasonable effectiveness of dynamic manipulation for cloth unfolding.\" *CoRL* 2021.\n\n[B] Wang, Dian, Robin Walters, Xupeng Zhu, and Robert Platt. \"Equivariant $ q $ learning in spatial action spaces.\" *CoRL* 2021.\n\n[C] Wang, Dian, Robin Walters, and Robert Platt. \"SO (2) Equivariant Reinforcement Learning.\" *ICLR* 2022.\n\n[D] Simeonov, Anthony, Yilun Du, Andrea Tagliasacchi, Joshua B. Tenenbaum, Alberto Rodriguez, Pulkit Agrawal, and Vincent Sitzmann. \"Neural descriptor fields: Se (3)-equivariant object representations for manipulation.\" *ICRA* 2022.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "The idea of the paper, making use of equivariance in robotic pushing, is interesting. I believe the paper makes a good contribution, however, the paper can be improved by adding a related work section and a limitation section. Though the paper talks about some limitations in the future directions subsection in Section 5, the discussion there seems a bit superficial. ", "technical_quality": "Very Good:  Convincing.", "clarity_of_presentation": "Good: Minor revision needed to improve organization or clarity.", "potential_impact": "", "robotics_focus": "Sufficient demonstration on hardware", "quality_of_limitations": "Limitations are not well addressed", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"impact": "3: The work is incremental and unlikely to have much impact, even though it may be technically correct and well executed.", "reviewer_expertise": "4: The reviewer is confident but not absolutely certain that the evaluation is correct"}, "extra_text": {"issues": "See the major issues in ‘Weaknesses’. \n\nMinor issues:\n\n1. line 100: $T_j \\mapsto C_i^{-1}T_j$ should be $U_j \\mapsto C_i^{-1}T_j$\n2. Is $I_3$ in Equation 3 the identity matrix? It would be better to mention it in the text. Same applies to the $I_4$ in line 138."}}, {"review_id": "SmjQCIFjQkp", "reviewer": "Reviewer_uaxP", "summary": "This paper presents SuperQuadric Pushing Dynamics Network (SQPD-Net), a learning-based simulator for modeling the interactions between objects (represented as superquadrics) on a planar surface. This work is motivated by the observation that if the whole pushing configuration is transformed by a planar rigid-body transformation, then the pushing result should also be the result of the same transformation. The authors thus propose to incorporate SE(2)-equivariance in the learning framework by considering an object-centric coordinate frame when modeling the object-object interactions.\n\nThe authors have compared the proposed method with a few existing learning-based multi-object interaction simulators and showed improved prediction accuracy thanks to the built-in SE(2)-equivariance. They have used the learned model together with model-based planning algorithms for three types of downstream manipulation tasks (i.e., moving, singulation, and grasping) both in simulation and the real world.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "**[Strengths]**\n\nThis paper is tackling an important problem of how to add priors and invariances in the learning-based simulators to improve the prediction accuracy and long-term rollout stability.\n\nThe idea of incorporating SE(2)-equivariance in the planar pushing tasks nicely captures the structure of the underlying system and has shown improved performance than the alternatives that do not consider such structure.\n\nIt is nice that the authors have shown downstream planning results using the learned model both in simulation and in the real world.\n\n**[Weaknesses]**\n\nWhile I like the general direction of this paper, I have concerns regarding the motivation, the evaluation, and the lack of comparison with more necessary baselines.\n\nI do not feel that the paper has sufficiently motivated why we want to learn the rigid-body planar pushing simulator from data. We have very good physics-based rigid body simulators. Even the proposed model itself is learned from a rigid body simulator (i.e., Pybullet) and evaluated by using Pybullet as the ground truth. If we add Pybullet as a baseline for evaluating the prediction accuracy, the error would be zero. Why don't we directly use Pybullet (or other rigid-body simulators) for real-world experiments? What value do the learning-based dynamics models bring in the specific evaluating scenario?\n\nI could think of the following potential reasons.\n- Will it be easier for the learned model to perform online adaptation given real-world observation (e.g., identify the center of mass, friction, distribution of supporting force) --- in other words, would it be more accurate in fitting the real-world observation data? If this is the case, more system identification or online adaptation experiments compared with physics-based simulators are needed for the real-world experiments.\n- Will the learned model be able to simulate systems that are otherwise hard for physics-based models to simulate? This could be true if the authors are dealing with objects with more complicated physical properties, but we have a very good understanding of rigid body dynamics. It is hard for me to imagine, in rigid-body planar pushing scenarios, how the learning-based method is better than the physics-based simulators, which already have the priors built in (e.g., SE(2)-equivariance, objects stay rigid).\n- The learning-based method can potentially learn models from observation data without the assumption of access to full-state information. However, in this paper, the authors assume a perception model that can produce the full state information of the environment (e.g., object shape and pose), which could also be the inputs to a physics-based simulator to predict the future.\n- Can the learned model provide better optimization results for downstream planning tasks? If this is the case, the authors need to show that the learning-based method is more effective at accomplishing the manipulation tasks than using the physics-based simulators (e.g., use the same sampling-based MPCs with Pybullet).\n\nDepending on the authors' motivation, more extensive experiments and comparisons with physics-based simulators are needed to justify the significance of this paper.\n\nThe real-world experiments are simple. It seems that only one of the objects is moving during the majority (or all?) of the task completion process, which only constitutes a very small portion of what can happen in the scenario. How well does the method work if there are more complicated multi-object interactions (e.g., the pushed object also pushes other objects, pushing objects in a pile)?\n\nI'm also curious about situations where the SE(2)-equivariance does not apply. For example, what if the friction coefficients are different in different regions of the table? What if the table is not perfectly flat and the distribution of the supporting force is different in different areas?\n\nThere are also a lot of works that use graph neural networks that capture the compositionality of the underlying systems and model multi-object dynamics (e.g., [1] and many follow-up works). Therefore, comparisons with these works could also be essential to show the significance of the proposed method, demonstrate the advantages of the incorporated priors on SE(2)-equivariance, and expand its impact.\n\n[1] Peter W. Battaglia, Razvan Pascanu, Matthew Lai, Danilo Rezende, Koray Kavukcuoglu, \"Interaction Networks for Learning about Objects, Relations and Physics\"", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "While I like the general direction of this paper, I have concerns regarding the motivation, the evaluation, and the lack of comparison with more necessary baselines (especially the physics-based simulators like Pybullet).\n\n=============\n\n**[Post-rebuttal recommendation]**\n\nI have read the rebuttal and the reviews from other reviewers. I appreciate the authors' effort in providing new experimental results in response to my questions, especially the comparison with PyBullet by learning models directly from real-world observations. As a result, I updated my score from Weak Reject to Weak Accept.", "technical_quality": "Fair: Significant additional evidence required to support the claims.", "clarity_of_presentation": "Good: Minor revision needed to improve organization or clarity.", "potential_impact": "", "robotics_focus": "Sufficient demonstration on hardware", "quality_of_limitations": "Additional details required", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"impact": "2: Unsure, difficult to assess.", "reviewer_expertise": "5: The reviewer is absolutely certain that the evaluation is correct and very familiar with the relevant literature"}, "extra_text": {"issues": "Concrete and comprehensive comparisons with physics-based models could be necessary to show the significance of this paper. Depending on the objective of this work, different types of experiments could be needed to justify the necessity of learning in the specific scenarios considered in this paper.\n\nI would like to see how well the model work in more complicated scenarios that involve multi-object interactions (e.g., pushing a chain or a pile of objects). Currently, the real-world experiments mostly involve pushing just one object at each manipulation step.\n\nIt would help the readers better understand the difficulty of the tasks and the performance of the method by showing video demonstrations of the task completion processes.\n\nPlease see the weaknesses section for more questions and issues."}}, {"review_id": "AuImVCSZsW8", "reviewer": "Reviewer_tXbi", "summary": "This paper presents an SE(2) equivariant representation of planar object dynamics which can be used to find pushing actions to drive a set of objects to a goal using planar pushing. The approach is trained with depth data to predict dynamic performance in a way that preserves geometric symmetry in planar motion. This approach is used to find pushing actions for a real robotic system and complete a wide variety of tasks. The results demonstrate how this approach outperforms previous paradigms to solve planar manipulation tasks with visual information.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n\n1. Paper is easy to follow and well explained overall, along with helpful descriptive figures.\n2. The approach is technically correct and \n3. This method is well validated with a variety of experiments and tasks, demonstrating its benefits.\n\nWeaknesses:\n\n1. The paper doesn't take enough time to describe the fundamental limitations of this approach and the type of tasks where equivariance might be detrimental.\n2. The limitations of superquadrics as a shape representation are not discussed.\n3. The paper contains several typos and uses passive voice very frequently, which sometimes leads to confusing statements.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "The paper is well written, presents an interesting idea, and is validated with a wide variety of experiments. The reviewer recommends acceptance. However, there are many areas of improvement when describing the limitations of the approach. More specifically, the choice of superquadrics and the impact of equivariance may lead to some fundamental limitations on object shape and mass distributions. ", "technical_quality": "Very Good:  Convincing.", "clarity_of_presentation": "Very Good:  Well organized and clear.", "potential_impact": "", "robotics_focus": "Sufficient demonstration on hardware", "quality_of_limitations": "Additional details required", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"impact": "3: The work is incremental and unlikely to have much impact, even though it may be technically correct and well executed.", "reviewer_expertise": "5: The reviewer is absolutely certain that the evaluation is correct and very familiar with the relevant literature"}, "extra_text": {"issues": "1. I would like to point out that equivariance is not always beneficial since different mass distributions will lead to different optimal policies and contact interactions. For instance, a square with a shifted center of mass will tend to rotate even when pushed by the middle. This is a natural limitation of this work and should be discussed with more care.\n2. The usage of superquadrics limits the algorithm to convex bodies, which leaves behind a wide range of objects of interest. The authors should also discuss this limitation.\n3. The authors briefly mention how they run their real-world experiments in a quasi-static fashion. How does this compare with the motion predictor? Why is this a straightforward way to close the sim-to-real gap? \n4. The proposed approach vastly outperforms previous work on this problem, it would be helpful if the paper briefly explained why the previous methods tend to perform poorly on some of these tasks. Is this related to equivariance or to the object representation?"}}, {"review_id": "-vPL1wXZbvJ", "reviewer": "Reviewer_T36J", "summary": "The main contribution of this paper is the design of an SE(2)-equivariant pushing dynamics model, which is then used in a model-based learning method. Objects are first detected by fitting superquadrics onto depth images and/or point cloud representations, which is an efficient way to improve pose definitions while simultaneously being a simpler description mechanism than the raw visual observation. Then through a series of transformations, they are concatenated and max pooled to enable equivariance, and input into a predictor model that predicts next poses. This model is then used by sampling based MPC methods to learn the sequence of actions needed to achieve the goal.  ", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n- The proposed method seems solid, easy to understand and improves upon the baselines in the paper. The equivariance property is useful for robot manipulation and it was interesting to see how this was encapsulated in the method.\n- The use of superquadrics is simplify object representations and improve pose descriptions is elegant.\n- The experiments are thorough, with successful sim-to-real transfer. Additionally, the robot learning to push flat objects to the edge of the table in order to grasp them is a good indication of the efficacy of the proposed method.\n- The paper is well-written, with proper discussion, good use of figures and clear explanations.\n\nWeaknesses:\n- The shapes of objects used seemed to be a bit simple (cylinders and cuboids). Given the use of superquadrics, it would have been great to see some complex shapes, or even a commentary on limitations if unable to.\n- Perhaps not directly a weakness of the paper, but it would have been great to see some videos of the experiments.\n\nImprovements:\n- A sentence or two explaining how long the training process takes, and a clarification on whether the actions were executed in an open loop or closed-loop manner would be good (are the actions and candidate grasp poses sampled once or resampled every so often?).", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "The proposed method seems good, both in terms of the logic of the design and the experimental results, though I can’t comment on the novelty. The paper is well-written, with extremely detailed description of the method, thorough experiments and clear discussion of the results. The back and forth between the main paper and the supplementary Appendix (especially the related works section being in the Appendix) made it a bit frustrating to read, but that’s just a minor issue. I also have not verified all the derivations in the paper, but there seemed to be occasional missing definitions of variables. The authors took a simple (yet important) observation of equivariance in learning dynamics and used it well to improve robot manipulation. In my opinion, overall, it is a solid paper.", "technical_quality": "Very Good:  Convincing.", "clarity_of_presentation": "Very Good:  Well organized and clear.", "potential_impact": "", "robotics_focus": "Sufficient demonstration on hardware", "quality_of_limitations": "Limitations are addressed clearly", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"impact": "3: The work is incremental and unlikely to have much impact, even though it may be technically correct and well executed.", "reviewer_expertise": "3: The reviewer is fairly confident that the evaluation is correct"}, "extra_text": {"issues": "- Some variable definitions seemed to be missing in the equations.\n- A brief explanation of why only simple objects were considered in this paper, or some experiments with more complex objects would strengthen the contributions of the paper.\n- Clarification on whether open-loop or closed-loop action sampling is used.\n"}}]

Author rebuttal:

Decision: Accept (oral)