A Practical Guide for Incorporating Symmetry in Diffusion Policy
Authors: Dian Wang
Boce Hu
Shuran Song
Robin Walters
Robert Platt
Venue: NeurIPS 2025 poster ()
Keywords: Robotic Manipulation
Equivariance
Diffusion Model
Primary area: deep_learning
TL;DR: We propose practical methods to incorporate symmetry into Diffusion Policy, improving performance while maintaining simplicity.

Abstract: Recently, equivariant neural networks for policy learning have shown promising improvements in sample efficiency and generalization, however, their wide adoption faces substantial barriers due to implementation complexity. Equivariant architectures typically require specialized mathematical formulations and custom network design, posing significant challenges when integrating with modern policy frameworks like diffusion-based models. In this paper, we explore a number of straightforward and practical approaches to incorporate symmetry benefits into diffusion policies without the overhead of full equivariant designs. Specifically, we investigate (i) invariant representations via relative trajectory actions and eye-in-hand perception, (ii) integrating equivariant vision encoders, and (iii) symmetric feature extraction with pretrained encoders using Frame Averaging. We first prove that combining eye-in-hand perception with relative or delta action parameterization yields inherent SE(3)-invariance, thus improving policy generalization. We then perform a systematic experimental study on those design choices for integrating symmetry in diffusion policies, and conclude that an invariant representation with equivariant feature extraction significantly improves the policy performance. Our method achieves performance on par with or exceeding fully equivariant architectures while greatly simplifying implementation.

A Practical Guide for Incorporating Symmetry in Diffusion Policy
Dian Wang'
Boce Hu?
Robin Walters Stanford University NNortheastern University
Shuran Song'
Robert Platt
https://syde http::///smmin-ddeee ietaaiindppp dn- github.
Abstract
Recently, equivariant neural networks for policy learning have shown promis- ing improvements in sample efficiency and generalization, however, their wide adoption faces substantial barriers due to implementation complexity. Equivariant architectures typically require specialized mathematical formulations and custom network design, posing significant challenges when integrating with modern policy frameworks like diffusion-basee models. this paper, we explore number straightforward and practical approaches to incorporate symmetry benefits into diffusion policies without the overhead full equivariant designs. Specifically, we investigate (i) invariant representations via relative trajectory actions and eye-in- hand perception, (ii) integrating equivariant vision encoders, and (iii) symmetric feature extraction with pretrained encoders using Frame Averaging. We first prove that combining eye-in-hand perception with relative or delta action parameteriza- tion yields inherent SE(3)-invariance, thus improving policy generalization. We then perform a systematic experimental study on those design choices for integrat- ing symmetry in diffusion policies, and conclude that invariant representation with equivariant feature extraction significantly improves the policy performance. Our method achieves performance on par with or exceeding fully equivariant architectures while greatly simplifying implementation.
Introduction
Although recent advancements incorporating symmetry robotic policy learning have shown promising results [52, 60), the practical impact of this approach remains limited due to the significant implementation challenges. While equivariant models [62, 11] can substantially improve sample efficiency and generalization, integrating these symmetric properties into modern policy learning frameworks presents several obstacles. First, symmetry reasoning must be tailored specifically for each policy formulation, requiring different mathematical analyses and architectures across different policy frameworks like Q-learning [57]. actor-critic methods [59], and diffusion models [3]. This creates steep learning curve for practitioners seeking to leverage symmetry benefits. Second, state-of-the-art equivariant architectures often introduce considerable complexity with specialized layers [4, 15] and require structured inputs [1] 34], which do not naturally align well with modern policy components like diffusion-basee action generation [5], eye-in-hand perception [7], or pre- trained vision encoders. As result, robotics researchers frequently face difficult choice: adopting complex equivariant architectures with specialized implementation, or maintaining practical imple- mentation concerns at the cost of symmetry benefits. This dilemma particularly pronounced in diffusion-basee visuomotor policies [5], which have emerged as a powerful paradigm for generat- ing smooth, multimodal robot actions. Although prior works have tried implement equivariant
Equal Advising
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

diffusion models [51, 65, 60], the diffusion diffusioondenoising nature significantly enhances the difficulty of incorporating equivariant structure.
In this paper, we present a practical guide for incorporating symmetry into diffusion policies without requiring significant design overhead or sacrificing the advantages of modern pol- icy formulations. We systematically investi- gate several straightforward approaches that achieve this balance: (i) invariant representa- tions through relative trajectory actions and aye-in-hand perception, (ii) equivariant vision encoders that can be incorporated into stan- dard diffusion frameworks, and (iii) Frame Averaging [48] techniques that enable sym- metrization with pretrained encoders. Our ap- proach offers a compelling alternative to end- to-end equivariant architectures: rather than choosing between symmetry benefits and im- plementation practicality, our methods demon- strate that these advantages can be achieved with minimal architectural changes and over- head. As shown in Figure 1, our method achieves excellent performance while maintaining implementation simplicity, positioning it an optimal balance point in the symmetry -aanttrr-rractccality tradeoff. Notably, our work using a single eye- in-hand image input reaches similar performance Wang et al. [60], which uses four cameras to reconstruct the 3D voxel grid input.
Average Performance in 12 MimicGen Tasks
70
Proposed
65
63.9
61.4
60
50
46.7
45
42
Pakg)
40
Preinine
35
P
30
Relative Trajectory
Equi
Pre-train Frame Avg
Fully
Symmetry
Encoder
Equivariant
Difficulty of Incorporating Symmetry
We propose number of practical ap- proaches for incorporating symmetry in Diffusion Policy, achieving comparable performance as fully- equivariant policies while maintaining simplicity.
Figure 1:
Our contributions can be summarized as follows:
We prove that eye-in-hand hand perception and relative trajectory action inherently possesses SE(3)- invariance, significantly improving the policy's 's generalization.
We demonstrate that transitioning from absolute action representations to relative trajectory actions provides a straightforward improvement in policy performance, both with eye-in-hand perception and extrinsic perception.
We propose novel approach that integrates a symmetric encoder into standard diffusion policy learning, achieved by either using equivariant network in end-to-end training or using Frame Averaging with pretrained encoders, and show that it can significantly improve performance.
We show that combining the invariant perception and action representations and pretrained encoder with Frame Averaging achieves the state-of-the-art results in the MimicGen [42] bench- mark while maintaining low architectural complexity and computational overhead compared to fully equivariant methods.
2 Related Work
Diffusion Policies:
Denoising diffusion models have transformed generative modeling in vision,
achieving state-of-the results in image and video synthesis [18, 55] as well as planning [29, 33]. Recently, Chi et al. [5, 6] introduced Diffussion Policy, which extends diffusion models to robotic visuomotor control by denoising action trajectories conditioned on observations. Subsequent extensions include applications to reinforcement learning [61, 49], incorporation of 3D inputs [68, 31 hicrarchical policies [64, 40, 71], and large vision-language action models [45, 2]. A key limitation of diffusion methods is their heavy demand for training data. To mitigate this, recent works have injected domain symmetries as equivariant constraints into the denoising network, thereby boosting sample efficiency and generalization [3, 51, 60, 65, 56]. However, they typically require complex equivariant denoising models. In contrast, our approach integrates symmetry by combining invariant observation and action representations with an equivariant vision encoder, greatly simplifying implementation.
Equivariant Policy Learning: Robotic policies often require generalizing across spatial transfor- mations of the environment. Traditional methods often achieve this via extensive data augmenta-

---

tion [69, 70]. Recently, equivariant models, class of methods that are mathematically constrained to be equivariant [8, 9, 62. 4, 11, 15. 34, 35], has been widely adapted to robotics automatically instantiate spatial generalization. Such models have been widely applied across robot learning, includ- ing equivariant reinforcement learning [57, 59, 58, 44, 43, 32, 37, 191, imitation learning [30, 66, 14], grasp learning [74, 75, 24, 21, 36], and pick-place policies [52, 53, 46, 50, 22, 25, 23, 27, 13, 26]. However, these methods often demand complex symmetry reasoning and equivariant layers, which can hinder scalability. By contrast, our framework introduces symmetry in a more modular fashion, making it easier to implement and adapt.
Policy Learning using Eye-in-Hand Images: Eye-in-hand perception, using a camera mounted on the robot's 's end-effector, has been popular choice in manipulation because it is simple and calibration- free. For instance, Jangir et al. [28] learn shared latent space between egocentric and external views train hybrid-input policies. Hsu et al. [20] show that eye-in-hand images (alone or combined with external cameras) yield higher success rates and better generalization, and similar findings have been captured in [41]. Consequently, many recent frameworks retain eye-in-hand imagery [72, 1, 73, 38). Another advantage is rapid data collection using handheld gripper [54, 67, 47, 7, 63, 16]. In this work, we theoretically analyze how eye-in-hand observation, when paired with relative or delta trajectory actions, yields inherent symmetry ad advantages for riffuusion-based policies.
3 Background
Problem Statement: We consider behavior cloning for visuomotor policy learning in robotic manipulation, where the goal is to learn a policy that maps an observation o to an action a, mimicking an expert policy. Both o and a may span multiple time steps, i.e.. o = (0--(m-1)..... 0z-1; 0z), a {ae; 0e+1,-..;; (n- where is the number of past observations and is the number of future action steps. At time step t, the observation O, (1,, Ti, wx) contains the visual information 4t, (e.g., images), the pose of the gripper in the world frame T, E SE((), as well as the gripper aperture Wi E R. The action a₂ (A,, w,) specifies target pose A, E SE(3) of the gripper and an open- width command we E R. To simplify the notation for our analysis, we omit the gripper command we in the action and focus on the pose command by writing
{A,, A+t1.... A (while in the actual implementation the policy controls both the pose and the aperture).
Diffusion Policy: Chi et al. [5] introduced Diffusion Policy, which formulates the behavior cloning problem as learning a Denoising Diffusion Probabilistic Model (DDPMs) [18] over action trajectories. Diffusion Policy learns a noise prediction function (, a -a+++ e k) E using network Ep, which is trained to predict the noise cx added to an action a. The training loss is L || (o, a + - 111 where e is random noise conditioned on a randomly sampled denoising step k. At inference, starting from a (0, 1), the model iteratively denoises
a² - a(a rea a², k) + +),
where ~ NO, o²1) a, 7, a are functions the denoising step (also known as the noise schedule). The action a" is the final executed action trajectory.
Equivariance: A function f: X Y is equivariant to group G if, for all g E G,
where Pa and Py are representations of G on X and Y. Equivariance ensures that applying before or after yields the same result, thus f is G-symmetric. When p is clear, we often write g gr.
For 2D images, the group SO(2) = {Rote : s < 2 of planar rotations and its subgroup Co = (Rots : 2 10 S u}} of rotations by multiples of are often used to construct equivariant neural networks [62, 4] that can capture rotated features. The regular representation Prog + Ruxu of C. is of particular interest of this paper, which defines how C, acts on a vector x E Rd by x permutation matrices. Intuitively, the vector can be viewed as containing information for each rotation in Cz. Let r E Cu = (1.r..... -"-1] and x (x1,... x.,) E R Lui L1, I2, . Iu-m) cyclically permutes the coordinates of z.
then Preg (r*) j (I (Iu-otl....
3

---

Simplicity
Equivariance
Stop k:
Step
Equi O Enc
Exiemal View
Equi Enc
Cat
Voxcl Map from Mutiple Cam
Large FOV
Hand View
Equi temporal UNet
a
aD
Eaul Rap
Hyn- Views
Robot State
Robot State
8012
Robot Stato
Peoooda UNtt
Irips
Observation
Observation
Observation
Ence
Equi A
Noise Prediction e
Dec
Equi A
Noiey Action In Gripper Frame
Noiey Action World Frame
Noisy World Frame
Ieeaatons
Output Action
Cutput Action Gripper
Action Frame
World Frame
Original Diffusion Policy
Ours
Equivariant Diffusion Policy
The difference comparing our method (middle) with the original Diffusion Policy (left) and the Equivariant Diffusion Policy (right). We use only eye-in-hand image the input, and use equivariant encoder to acquire symmetry-aware features from the input image. policy denoising, the noisy action and the noise-free action output are both in the gripper frame. Other components remain identical to the original Diffusion Policy. Compared with Equivariant Diffusion Policy (right), approach is significantly simpler while maintaining comparable experimental performance.
Figure 2:
4 Approaches for Incorporating Symmetry in Diffusion Policy
In this section, introduce three practical approaches for incorporating symmetry into diffusion policies without requiring complex end-to-end equivariant architecture design. First, we examine how invariant action and perception representations naturally induce symmetric properties. Second, we explore integrating equivariant vision encoders that extract symmetry-awen features while maintaining standard diffusion heads. Finally, we present how leverage pre-trained vision encoders in an equivariant way through Frame Averaging [48]. Together, these approaches offer a spectrum of options for balancing symmetry benefits with implementation simplicity. As shown in Figure our proposed approach (middle) requires minimal architectural change compared with the original Diffusion Policy [5] (left), and much simpler than the fully equivariant model [60] (right).
4.1 Representing Actions as Absolute, Relative, and Delta Trajectory
Choosing the right representation for actions and observations is crucial for sample-efficient policy learning in robotic manipulation. When robotic task and environment exhibit rigid-body symmetries, incorporating an equivariant or invariant representation can significantly enhance generalization to unseen object configurations and poses. In this section, explore three natural trajectory action representations: absolute, relative, and delta trajectories, highlighting their symmetry properties under global SE(3) transformations. Notice that although relative trajectories were introduced by Chi et al. [7], their symmetric advantages have not yet been explored.
Definition 1 (Absolute Trajectory Action). An absolute trajectory action specifies future gripper poses directly in the world frame as
= =
where each Acti E SE(3) is the desired pose at time t

---

Definition 2 (Relative Trajectory Action), Let Ti E SE(3) be the current gripper pose in the world frame. relative trajectory action sequence
a' = ( Air , 1
where each Ai+; SE(3) specifies the gripper's 's pose relative to its initial frame at time t. The corresponding absolute poses are recovered via
Acti T, A7
i= O.......7
(2)
Definition (Delta Trajectory Action). delta trajectory action sequence of incremental transforms expressed in moving local frame:
a² = ( A A
where Alts E (E((3 represents the incremental motion time step + expressed relative to the gripper's frame at the previous time step 1.+ 1. The absolute poses are reconstructed as:
1-1
A4t1 = T
II ==
i=
(3)
We now formalize the transformation properties of these action representations (see Appendix A for the proof):
Proposition (Equivariance and Invariance under SE(3)). Consider global transformation SE(3) applied to the world coordinate frame, which transforms the current gripper pose as T3 + g'r.. Under this transformation:
1. The absolute trajectory action transforms equivariantly, i.c., a. tgAe, gAct1......
2. The relative trajectory action is invariant, i.e., a" = a".
3. The delta trajectory action is invariant, i.e., a" = ad.
4.2 SE(3)-Invariant Policy Learning
A key advantage of using relative delta tra- jectory action representations that, if the pol-
icy being modeled is equivariant, i.e., (go))
Dam
(o)),, the learned function म o a" or o + ad becomes invariant, i.e., #(go)= = (o).
3E/9) Transform
This makes the underlying denoising network tş also invariant, significantly reducing the fune- tion space and potentially casing the training.
Moreover, when the relative delta trajectory is combined with eye-in-hand perception, it natu- rally yields an SE(3)-invalll
Gripper
Gripper
canonicalization. Specifically, consider an agent equipped with chrpper-mounted camera, which captures an eye- in-hand image J., When an arbitrary transforma- tion y E SE(3) applied the world, the vi- sual input from the eye-in-hand camera remains unchanged since the relative position and orien- tation between the camera and the world do not vary. For the input observation o, = (4,, Te, w;) consisting of the invariant image I,, gripper pose T,, and gripper open- width wt, applying the transformation g to the world frame affects only the gripper pose Tt,
Invariant
Figure 3: The invariant property of eye-in- hand perception and action representation in policy learning. Top: SE(3) transform applied the world, the action should transform accordingly. Bottom: for the policy, both the input eye-in-hand image and the output relative trajectory (in the grip- per frame) remain invariant.
open-width
g 0t (Je, Ne.gT,, w.).
Let us first assume that the policy does not depend on the gripper pose Tr, then the policy w: o +> is -333iiqqqi when using eye-in-hand perception and relative or delta action:
5

---

Proposition 2. L a" or i: : at be function mapping from the observation to the relative trajectory a' or the delta trajectory a². Assume an eye-in-hane observation is used where Equation 4 is satisfied and i. does not depend on T,. the policy π o reconstructs the absolute trajectory a using Equation or 3, then π is SE((3) -qquivarrrn i..., * <(g)) = (r)
See Appendix for the proof. This equivariance property implies that once a policy is learned, it automatically generalizes across different poses in space without additional training data, thus enhancing sample efficiency and robustness. As shown in Figure 3, when an SE(3) transformation is applied to the world, both the perception and action remain invariant.
In practice, the assumption that does not depend on T₃ will not perfectly hold because the changing T. will affect the network prediction, thus we will only have an approximate invariance property. Since T, provides important information to the policy despite breaking the symmetry, we choose not to explicitly constrain the policy to be invariant to Ti. Still, our experiments demonstrate significant performance improvements when employing this approximate invariant property.
4.3 Equivariant Vision Encoders
While the invariant representations described in Section 4.2 theoretically achieve SE(33 -qquivaraa policy learning, we experimentally found that they do not fully match the performance of end-to-end equivariant policies [60]. This is because equivariant neural networks not only guarantee global symmetric transformations, but more importantly, they extract richer local features that can capture the underlying symmetries of the problem domain. Instead of falling back to network architecture that is end-to-end equivariant, we propose novel approach that incorporates an equivariant vision encoder to extract synmetry-aware features while preserving a standard non-equivariant diffusion backbone. This approach would preserve the benefits of symmetric feature extraction without the complexity of a full equivariant model.
Specifically, can replace the standard CNN vision encoder in Diffussonn Policy with equivariant CNN that operates on the group Cn C SO(2). This encoder maps the input eye-in-hand image to feature vector that transforms according to the regular representation of Cu. providing richer representation to the diffusion head with explicit information about how features transform under rotations, significantly enhancing learning.
4.3.1 Incorporating Pretrained Vision Encoders with Frame Averaging
Although there exists a wide variety equivariant neural network architectures [8, 62, 4], they typically require defining each layer to be equivariant by constraining the weights with specialized kernels [10]. This usually implies that the network is built specifically for a task and is trained from scratch. However, modern computer vision has seen tremendous progress through large-scale pretrained models, which provide powerful general-purpose representations. To bridge the gap between equivariance and pre- trainingg, employ Frame Averaging [48] for turning an arbitrary function Φ : X - Y into a G-equivariant network by averaging over a Frame F X 2 (6) that satisfies equivariance set F(go) = F(x):
x
Σ (EF((a)
pr(g) x),
(5)
F(x)
where V : X + Y will have the equivariant property V (px(g)x)) Py (g) (x:) For finite group G, one can set the frame to be the whole group F(x) = G. and Equation 5 becomes symmetrization:
1
(p)
(6)
G
9EG
When using a pretrained encoder with Frame Averaging, obtain the benefits of both powerful pretrained representations and explicit rotational equivariance, allowing us to leverage state-of-the-art vision backbones without sacrificing symmetry properties.
5 Experiments
In this section, we conduct a systematic experimental study comparing different approaches for incorporating symmetry in diffusion policies. We investigate the following key research questions:

---

(a) Stack DI
(b) Stack Three D1
(c) Square D2
(d) Threading D2
(e) Coffee D2
(l) Three Pc. Assembly D2 (g) Hammer Cleanup DI
(h) Mug Cleanup DI
(i) Kitchen D1
(j) Nut Assembly DO
(k) Pick Place DO
(l) Coffee Preparation D1
Figure 4: The experimental environments from MimicGen [42]. The left image in each subfigure shows the initial state of the environment; the right image shows the goal state.
1. Invariant representations: How do the invariant action and perception representations analyzed in Section 4.1 impact diffusion policy learning performance?
2. Equivariant vision encoders: Can diffusion policies benefit from incorporating equivariant vision encoders?
3. Pre-trained encoders: How effectively can we leverage pre-trained encoders with Frame Averaging (Equation 5y?
4. Comparison to end-to-end equivariant diffusion: How do these approaches compare with fully equivariant diffusion policies [60]?
We evaluate our approaches on 12 robotic manipulation tasks in the MimicGen [42] benchmark, as illustrated in Figure 4. We perform an additional Robomimic [41] experiment in Appendix D.
5.1 Action and Observation Representation
We first evaluate the three action representations (absolute trajectory, relative trajectory, and delta trajectory) discussed Section 4.1 across two different observation settings, Large FOV In Hand and In-hand + External. Figure 2 illustrates the differences between these configurations.
The results, presented in Table demonstrate several key find- ings. First, relative trajectory consistently outperforms absolute trajectory in 10 out of 12 tasks in both Large FOV In-Hand and In-Hand + External observations. On average, relative tra- jectory provides 5.9% improvement with Large FOV In-Hand observations and a 7.4% improvement with In ia- + External observations. These results align with our thcoretical analysis in Section 4.1, confirming that the sym- metry properties of relative trajectory representations contribute to better performance. However, despite having the same the- oretical guarantee, delta trajectory empirically performs poorly, underperforming absolute trajectory by 2.9% on average, where only performs well relatively simple tasks. We hypothesize that this is because delta trajectory can be interpreted tion for the denoising process. Notice that similar observations the underperformance of velocity control in diffusion policy learning were also reported in prior works [5, 60].
absolute trajectory
(a) Threading
(b) Coffee Prep.
Figure 5: The failures of the in- hand observation due to occlusion and insufficient information.
sequence of velocity vectors, containing less temporal and structural informa-
7

---

Table I: The performance comparison between different action representations and different observa- tion setups. All policies are trained using 100 expert demonstrations. We perform 60 evaluations (each with 50 policy rollouts) throughout the training, and report the best average success rate. Results averaged over three seeds, + indicates standard error. Blue box indicates relative trajectory or delta trajectory outperforming absolute trajectory in the corresponding observation setting; Red box indi- cates underperforming. Bold indicates best performing method across different settings. Performance of Abs Traj in In-Hand + External is reported by [60].
Obs
Action
Mean
Stack DI
Slack Three D1
Square D2
Threading D2
Coffee D32
Three Pc. D2
Rel Traj
46.7
98.0=0.00
72.0+:122
16.0=122
16.0+122
16.7+188
4,0+:1122
Large FOV
In-Hand
Delta Traj
37.9
96.7+00.7
54.743.5
10.0
11.33:1
38.7::0.7
2.0::.22
Traj
40.8
94.0#1.2
52.742.4
7.3+0.7
10.0+000
44.0:t:4.6
2.0+:0.0
In-Hand
Rel Tmj
4
89.14.2.7
52.74:3.7
20.7
18.74355
63.3--1.3
3.34:0.7
External
Abs Traj
42.0
76.0++.00
38.0+00.0
8.0+1.2
17,3118
1..0=
4.0+00.0
Obs
Method
Hammer CI. D1
Mug C1. D1
Kitchen D1
Nut Asse. DO
Pick Place DO
Coffee DI
Tmj
60.7.10,7
5.
68.7
44.7.3..3
4..7:11.6
3.
Large FOV
Delta Traj
22.7=2,4
47.3+1188
57.3+488
17.0+155
31,7+:0.4
26.0+
In-Hand
Abs Traj
62.0#22.0
44.71188
66.7=0077
43.0+t:.55
32,0::066
30.7+2.9
In-Hand
Rel Traj
58,7=00,7
46.7+:0.7
62.0=22.0
$7.3+0.99
39.8:1.2
80.0:2.00
External
Traj
52.00.1.2
42.7.10.7
66.7.:244
54.7.2233
35.31122
65.3100.7
When comparing across observation settings using relative trajectory, we find that Large FOV In-Hand generally performs better on par with In-Hand External. However, if averaged across all tasks, the Large FOV In-Hand setup underperforms by 2.7%. This performance gap is primarily due to a sigmificant drop in the Coffee Preparation task, where the eye-in-hand view alone provides insufficient information for completing this long-horizon task. Moreover, we found that tasks like Threading sometimes encounter occlusion challenges. As shown in Figure 5, those limitations of a single eye-in-hand image constitute the majority of the failure modes. Despite these drawbacks, leveraging the invariant observation and action representations provides 4.7% improvement compared with the original Diffusion Policy, which uses In- In-Hand + External views and absolute trajectory.
5.2 Integrating Symmetry into the Vision Encoder
Having established the advantages of invariant action representations, we now investigate different approaches for incorporating symmetry into the vision encoder component of diffusion policies. We compare four methods: CNN Encoder (CNN Enc): standard R [17] without any symmetry constraints, trained from scratch; Equivariant Encoder (Equi Enc): An equivariant ResNet-18 archi- tecture implemented with equivariant layers using the esenn [4] library, enforcing Cs -geeuivvare with outputs in the regular representation of Cg; Pretrained Encoder (Pretrain): A standard ResNet- 18 pretrained on ImageNet-Ik [12], without explicit symmetry constraints; Pretrained Encoder with Frame Averaging (Pretrain FA): A pretrained ResNet-13 18 enhanced with Frame Averaging (Equation 5) to achieve Cg-equivariance without modifying the underlying network architecture.
Table 2 presents our findings across all 12 manipulation tasks. The results reveal several important insights: First, comparing non- non-pretrained encoders (Equi Enc vs. CNN Enc), we observe that incorporating equivariance improves performance in 11 out of 12 tasks, yielding a substantial 9.1 % average improvement. This confirms that explicit symmetry constraints significantly benefit diffusion policy learning. Second, in the pretrained encoder setting, adding Frame Averaging (Pretrain + FA vs. Pretrain) leads to 4.1% average performance improvement, with superior results in 7 out of 12 tasks. This demonstrates that symmetry benefits can be obtained even when leveraging powerful pretrained representations. Third, comparing our approaches to Equivariant Diffusion Policy [60] (EquiDiff), we find that both Equi Enc and Pretrain + FA achieve competitive performance.
Specifically, our Equi Enc approach outperforms image-based EquiDiff (Im) on average, while Pretrain + FA achieves results only 2.5% below voxel-based EquiDiff (Vo). This is particularly impressive considering that EquiDiff (Vo) utilizes RGBD inputs from four cameras and employs a substantially more complex architecture. In contrast, our Pretrain + FA approach requires only a single eye-in-hand RGB image and minimal equivariant reasoning, making considerably more practical for real- world deployment.
8

---

Table 2: The performance comparison between symmetric encoder and standard encoder. All policies are trained using 100 expert demonstrations. We perform 60 evaluations (each with 50 policy rollouts) throughout the training, and report the best average success rate. Results averaged over three seeds, + indicates standard error. Dark green box indicates outperforming EquiDiff with voxel inputs; Light green box indicates outperforming EquiDiff with image inputs; Yellow box indicates underperforning both. Bold indicates whether the symmetric encoder outperforms the non-symmetric version in the corresponding training setting, Performance of Equi is reported by [60].
Obs
Action
Method
Mean
Stack D1
Stack Three D1
Square D2
Threading D2
Coflee D2
Three Pc. D22
Pretrain FA
61.4
100.0:0:00
86,7:11.8
43,3:20.7
16,7+18
63,3=0.7
28.7==188
Large FOV
Rel Traj
Pretmin
57.3
100.0:1.00
78.0:22.0
33.3:1:
18.0:1122
65.3:
14.0:10.0
In-Hand
Equi Enc
55.8
99.3
75.3112.44
32.0.11.2
14.0:112
63.3
2..0
CNN
46.7
98.0=2..0
72.0::122
16.04:12
16.0:1:22
56.74 1.8
4.0::
In Hand Ext.
EquiDai (Im)
53.7
93.3
$4.7.15.2
25.3.18.7
22.0.112
60.01:2.0 64.720.7
15.31
Voxel
Abs Traj
EquiDiff (Vo)
63.9
98.7
74.71:44
38.711.
38.7::077
3.7
Obs
Method
Hammer D1
Mug
DO
DO Coffee Prep.
Pretmuin FA
7..0:12.0
6..0:1233
78.711.
74.7.1.5
50.8 0.7
58.0
Large FOV
Rel Traj
Pretmin
1.2
62.7:11.8
80.7:12.44
61.0:12.1
48.211
30.01:.11
In-Hand
Equi Enc
67.3
58.7:1
72.0113.1
69.3:10.9
47.2:12.88
2..7:1:.77
CNN Fnc
60.7:10.7
50.7:11.8
68.7
44.7.:..3
42.71:
30.7:t 1.8
In Ro-Iaand+
Ext.
EquiDiil (Im) EquiDiff (Vo)
65.3:10.7
49.3:10.7
67.3:10.7
74.01:12
41.71:32
76.7:10.7
Abs Traj
Voxel
70.0=22.0
$2.7:::3
85.3:00.7
67.3#t099
57.7=1188
80.0
Overall, these results suggest that integrating symmetry through equivariant encoders provides significant performance benefits for diffusion policies, with Frame Averaging offering an elegant way to leverage powerful pretrained representations while maintaining equivariance properties.
Discussion
In this paper, we present a practical guide for incorporating symmetry in diffusion policies, achieving performance competitive with or exceeding fully equivariant architectures while requiring signifi- cantly less implementation complexity. Notably, our method performs only 2.5% below voxel-based EquiDiff, despite using only single eye-in-hand RGB image compared to EquiDiff's four RGBD cameras. Our approach not only defines a new state-of-the-art performance for RGB eye-in-hand diffusion policy, but more importantly, it addresses the trade-off between architectural complexity and sample efficiency when introducing symmetries into policy learning.
Concretely, we investigate three straightforward approaches for incorporating symmetry: invariant representations through relative trajectory and eye-in-hand perception, integrating equivariant vi- sion encoders, and using Frame Averaging with pretrained encoders. Our extensive experimental evaluation across 12 manipulation tasks in MimicGen yields several important findings. First, we demonstrate that relative trajectory actions consistently outperform absolute trajectory. confirming our theoretical analysis that relative trajectory induces SF(()-invariance. This finding particu- larly valuable because a simple coordinate frame change in action representation can bring a 5-7% improvement. Second, we found that incorporating symmetry through equívariant vision encoders significantly enhances performance by 9.1% highlighting the value of symmetry-awan features while avoiding complex end-to-end reasoning. Lastly, we show that Frame Averaging provides an elegant solution for leveraging the power of pre-trained vision encoders while maintaining equivariance.
6.1 Limitations
There are several limitations of this work that suggest directions for future research. First, only leveraging an eye-in-hand image assumes a good coverage of the entire workspace (thus we use an enlarged FOV in our experiments); however, as shown in Figure 5, the limited view still constitutes the most significant failure mode of our system. future works, this could be addressed by using a fish-eye camera [7], or memory mechanism to maintain context across timesteps. Second, while our approaches are theoretically applicable to other policy learning frameworks beyond diffusion models, such ACT [72], we limited our investigation to diffusion policies and only experimented

---

in the MimicGen [42] and Robomimic [41] benchmarks. Third, leveraging an equivariant encoder, especially with Frame Averaging, could be computationally expensive. Our method roughly takes twice the GPU hours to train compared with the original Diffusion Policy, but is twice as fast as EquiDiff (Im). Finally, although our method is well-suited for real-world deployment on systems like UMI [7], we have not yet demonstrated this transfer to physical rohots.
Acknowledgment
This work was supported in part by NSF grants 2107256, 2134178, 2314182, 2409351, 2442658, and NASA grant 10445C19K1474.
References
[1] Jorge Aldaco, Travis Armstrong, Robert Baruch, Jeff Bingham, Sanky Chan, Kenneth Draper, Debidatta Dwibedi, Chelsea Finn, Pete Florence, Spencer Goodrich, al. Aloha 2: An enhanced low-cost hardware for bimanual teleoperation. arXiv preprint arXiv:2405.0223 020.00222, 2024.
[2] Kevin Black, Noah Brown, Danny Driess, Adnan Esmail, Michael Equi, Chelsea Finn, Niccolo Fusai, Lachy Groom, Karol Hausman, Brian Ichter, et al. 0: A vision-lagguage-action flow model for general robot control. arXiv preprint arXiv:24 10. 24 164, 2024.
[3] Johann Brehmer, Joey Bose, Pim De Haan, and Taco Cohen. EDGI: Equivariant Diffusion for Planning with Embodied Agents. arXiv preprint arXiv:2303.12410, 2023.
[4] Gabriele Cesa, Leon Lang, and Maurice Weiler. A Program to Build CNNs. In International Conference on Learning Representations, 2021.
Steerable
[5] Cheng Chi, Siyuan Feng, Yilun Du, Zhenjia Xu, Eric Cousineau, Benjamin Burchfiel, and Shu- ran Song. Diffusion Policy: Visuomotor Policy Learning via Action Diffusion. Proceedings of Robotics: Science and Systems (RSS), 2023.
[6] Cheng Chi, Zhenjia Xu, Siyuan Feng, Eric Cousineau, Yilun Du, Benjamin Burchfiel, Russ Tedrake, and Shuran Song. Diffusion policy: Visuomotor policy learning via action diffusion. The International Journal of Robotics Research, page 02783649241273668, 2023.
[7] Cheng Chi, Zhenjia Xu, Chuer Pan, Eric Cousineau, Benjamin Burchfiel, Siyuan Feng. Russ Tedrake, and Shuran Song. Universal manipulation interface: In-the-wild robot teaching without tn-the-wild robots. In Proceedings of Robotics: Science and Systems (RSS), 2024.
[8] Taco Cohen and Max Welling. Group equivariant convolutional networks. In International conference on machine learning, pages 2990-2999. PMLR, 2016.
[9] Taco S. Cohen and Max Welling, Steerable CNNs. In International Conference on Learning Representations, 2017. URL https://
[10] Taco S Cohen, Mario Geiger, and Maurice Weiler. A general theory of equivariant cnns homogeneous spaces. Advances in neural information processing systems, 32, 2019.
[11] Congyue Deng, Or Litany, Yueqi Duan, Adrien Poulenard, Andrea Tagliasacchi, and Leonidas J Guibas. Vector neurons: A general framework for so (3)-qqivvriaant networks. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 12200-12209, 2021.
[12] Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large- scale hierarchical image database. 2009 IEEE conference on computer vision and pattern recognition, pages 248-255. Ieee, 2009.
[13] Ben Eisner, Yi Yang, Todor Davchev, Mel Vecerik, Jonathan Scholz, and David Held. Deep Sqii)--quivarian geometric reasoning for precise placement tasks. In The Twelfth International Conference on Learning Representations, 2024. URL http::t//////o https://openrevieeeeeee wetpe/neeeeeeeeeee ner/forum? id=2inBuTTyL2.
10

---

[14] Chongkai Gao, Zhengrong Xue, Shuying Deng, Tianhai Liang, Siqi Yang, Lin Shao, and Huazhe Xu. RiEMann: Near real-time equivaaqin robot manipulation without point cloud segmentation. In 8th Annual Conference on Robot Learning, 2024. URL. https:
[15] Mario Geiger, Tess Smidt, Alby M. Benjamin Kurt Miller, Wouter Boomsma, Bradley Dice, Kostiantyn Lapehevskyi, Maurice Weiler, Michal Tyszkiewiez, Simon Batzner, Dylan Madisetti, Martin Uhrin, Jes Frellsen, Nuri Jung, Sophia Sanbor, Mingjian Wen, Josh Rackers, Marcel Rød, and Michael Bailey. Euclidean neural networks: e3nn, April 2022. URL https:///ol org/10. /t///10.2281/22 5201/22e0od// 6459381.
[16] Huy Ha, Yihuai Gao, Zipeng Fu, Jie Tan, and Shuran Song. Umi on legs: Making manip- ulation policies mobile with manipulation-centri whole-body controllers. arXiv preprint arXiv:2407. 10353, 2024.
[17] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep Residual Learning for Image Recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern recognition, pages 770-778, 2016.
[18] Jonathan Ho. Ajay Jain, and Pieter Abbeel. Denoising Diffusion Probabilistic Models. Advances in Neural Information Processing Systems, 33:6840-6851, 2020.
[19] Tai Hoang, Huy Le, Philipp Becker, Vien Anh Ngo, and Gerhard Neumann. Geometry-aware RL for manipulation of varying shapes and deformable objects. In The Thirteenth International Conference on Learning Representations, 2025. URL https: h/ttps///operrevien net/forum? id=7BLXhmWvwF.
[20] Kyle Hsu, Moo Jin Kim, Rafael Rafailov, Jiajun Wu, and Chelsea Finn. Vision-based manipula- tors need to also see from their hands. In International Conference on Learning Representations, 2022. URL https: / http//ppeeeeveeeeeeeeee wwwp:/eeetteedddddne.ee..o.
[21] Boce Hu, Xupeng Zhu, Dian Wang, Zihao Dong, Haojie Huang, Chenghao Wang, Robin Walters, and Robert Platt. Orbitgrasp: Se (3)-equivariant grasp learning. In 8th Annual Conference on Robot Learning, 2024.
[22] Haojie Huang, Dian Wang, Robin Walters, and Robert Platt. Equivariant Transporter Network. In Robolics: Science and Systems, 2022.
[23] Haojie Huang, Dian Wang, Arsh Tangri, Robin Walters, and Robert Platt. Leveraging Symme- tries in Pick and Place. The International Journal of Robotics Research, 2023.
[24] Haojie Huang, Dian Wang, Xupeng Zhu, Robin Walters, and Robert Platt. Edge Grasp Network: A Graph-Basee Based SE(3) Approach to Grasp Detection. In International Conference on Robotics and Automation (ICRA), 2023.
[25] Haojie Huang, Owen Lewis Howell, Dian Wang, Xupeng Zhu, Robert Platt, and Robin Walters. Fourier transporter: Bi-equivariant robotic manipulation in 3d. In The Twelfth International Conference on Learning Representations, 2024. URL https: //tp:////ee/.eeeee////mee wwt.:/eee/ffrura id-UulwvAU1WO.
[26] Haojie Huang, Haotian Liu, Dian Wang, Robin Walters, and Robert Platt. Match policy: A simple pipeline from point cloud registration to manipulation policies. arXiv preprint arXiv:2409.15517, 2024.
[27] Haojie Huang, Karl Schmeckpeper, Dian Wang, Ondrej Biza, Yaoyao Qian, Haotian Liu, Mingxi Jia, Robert Platt, and Robin Walters. IMAGINATION POLICY: Using generative point cloud models for learning manipulation policies. In 8th Annual Conference on Robot Learning, 2024.
URL https://openrreviet wwt.:eeooooooidddieeeeeeee
[28] Rishabh Jangir, Nicklas Hansen, Sambaran Ghosal, Mohit Jain, and Xiaolong Wang. Look closer: Bridging egocentric and third-person views with transformers for robotic manipulation. IEEE Robotics and Automation Letters, 7()):3046-3053, 2022.

---

[29] Michael Janner, Yilun Du, Joshua Tenenbaum, and Sergey Levine. Planning with Diffusion for Flexible Behavior Synthesis. In International Conference on Machine Learning, pages 9902-9915. PMLR, 2022.
[30] Mingxi Jia, Dian Wang, Guanang Su, David Klee, Xupeng Zhu, Robin Walters, and Robert Platt. SEIL: Simulation-augmenter Equivariant Imitation Learning. In International Conference Robotics and Automation (ICRA), 2023.
[31] Tsung-We Wei Ke, Nikolaos Gkanatsios, and Katerina Fragkiadaki. 3d diffuser actor: Policy diffusion with 3d scene representations. In 8th Annual Conference on Robot Learning, 2024. URL SotLhtts///ppeereviiee wwtp:fooruummdddgggc0o0v..
[32] Colin Kohler, Anuj Shrivatsav Srikanth, Eshan Arora, and Robert Platt. Symmetric models for visual force policy learning. arXiv preprint arXiv:2308. 14670, 2023.
[33] Zhixuan Liang, Yao Mu, Mingyu Ding, Fei Ni, Masayoshi Tomizuka, and Ping Luo. AdaptDif- fuser: Diffusion Models as Adaptive Self-evolving Planners, In International Conference on Machine Learning, 2023.
[34] Yi-Lun Liao and Tess Smidt. Equiformer: Equivariant graph attention transformer for 3d atomistic graphs. The Eleventh International Conference on Learning Representations, 2023. URL https: tettts::/opoeee http:///ppp/eeeeeeeeeee wwt.
[35] Yi-Lum Liao, Brandon M Wood, Abhishek Das, and Tess Smidt. Equiformerv2: Improved equivariant transformer for scaling higher-degree representations. In The Twelfth International Conference on Learning Representations, 2024. URL https: https:://ppnneeeee //tp///eveeeeeeeeeeeeeeeeeeeeeee wtt.iiiwwwe.eeeeeeeeeeeeeeeeeeee forum??m? id=mCOBKZmrzD.
[36] Byeongdo Lim, Jongmin Kim, Jihwan Kim, Yonghyeon Lee, and Frank C. Park. Equigraspflow: SEuiivaqna 6-dof grasp pose generative flows. 8th Annual Conference on Robot Learning, 2024. URL https: /pwprreveeeeeeeeeeeeemeo net/forummiiii merriidd55Skkvvnii
[37] Shiqi Liu, Mengdi Xu, Peide Huang, Xilun Zhang, Yongkang Liu, Kentaro Oguchi, and Ding Zhao. Continual Vision- based Reinforcement Learning with Group Symmetries. In Conference on Robot Learning. pages 222-222222222222222222 222. PMLR, 2023.
[38] Songming Liu, Lingxuan Wu, Bangguo Li, Hengkai Tan, Huayu Chen, Zhengyi Wang. Ke Xu, Hang Su, and Jun Zhu. RDT-1b: a diffusion foundation model for bimanual manipulation. In The Thirteenth International Conference on Learning Representations, 2025. URL https:
/ /openreview. httpeiiwwweeeeeeeeeeeeeeeeee wwt//eeetooiiieee
[39] Ilya Loshchilov and Frank Hutter. Decoupled Weight Decay Regularization, In International Conference Learning Representations, 2018.
[40] Xiao Ma. Sumit Patidar, Iain Haughton, and Stephen James. Hierarchical diffusion policy for kinematics-aware multi-task robotic manipulation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 18081-18090, 2024.
[41] Ajay Mandlekar, Danfei Xu, Josiah Wong, Soroush Nasiriany, Chen Wang, Rohun Kulkarni, Li Fei-Fei, Silvio Savarese, Yuke Zhu, and Roberto Martín-Martnn What Matters in Learning from Offline Human Demonstrations for Robot Manipulation. In Aleksandra Faust, David Hsu, and Gerhard Neumann, editors, Proceedings of the 5th Conference on Robot Learning, volume 164 of Proceedings of Machine Learning Research, pages 1678-1699 PMLR, 08-11 Nov 2022.
[42] Ajay Mandlekar, Soroush Nasiriany, Bowen Wen, Iretiayo Akinola, Yashraj Narang, Linxi Fan, Yuke Zhu, and Dieter Fox. MimieGen: A Data Generation System for Scalable Robot Learning using Human Demonstrations. In 7th Annual Conference on Robot Learning. 2023.
[43] Hai Nguyen, Tadashi Kozuno, Cristian C Belran-Hernandez, and Masashi Hamaya. Symmetry- aware reinforcement learning for robotic assembly under partial observability with soft wrist. arXiv preprint 2rXiv:2402.18800 2024.
12

---

[44] Hai Huu Nguyen, Andrea Baisero, David Klee, Dian Wang, Robert Platt, and Christopher Amato. Equivariant reinforcement learning under partial observability. In Conference on Robot Learning, pages 3309-3320. PMLR, 2023.
[45] Octo Model Team, Dibya Ghosh, Homer Walke, Karl Pertsch, Kevin Black, Oier Mees, Sudeep Dasari, Joey Hejna, Charles Xu, Jianlan Luo, Tobias Kreiman, You Liang Tan, Lawrence Yun- liang Chen, Pannag Sanketi, Quan Vuong, Ted Xiao, Dorsa Sadigh, Chelsea Finn, and Sergey Levine. Octo: An open-source generalist robot policy. In Proceedings of Robotics: Science and Systems, Delft, Netherlands, 2024.
[46] Chuer Pan, Brian Okorn, Harry Zhang, Ben Eisner, and David Held. TAX-Pose: Task-Specific Cross-Pose Estimation for Robot Manipulation. In Conference on Robot Learning, pages 1783-1792. PMLR, 2023.
[47] Jyothish Pari, Nur Muhammad Shafiullah, Sridhar Pandian Arunachalam, and Lerrel Pinto. The surprising effectiveness of representation learning for visual imitation. arXiv preprint arXiv:21 2021.
[48] Omri Puny, Matan Atzmon, Edward J. Smith, Ishan Misra, Aditya Grover, Heli Ben-Hamu and Yaron Lipman. Frame averaging for invariant and equivariant network design. In International Conference on Learning Representations, 2022. URL https:// http:/ee//foorooo J55xXR..55nXR.
[49] Allen Z. Ren, Justin Lidard, Lars Lien Ankile, Anthony Simeonov, Pulkit Agrawal, Anirudha Majumdar, Benjamin Burchfiel, Hongkai Dai, and Max Simchowitz. Diffusion policy policy optimization. In The Thirteenth International Conference on Learning Representations, 2025.
URL https: /ttp://////ppeeeeeeeeeee wtc.ieewweeeeeeeeeeeeeeeeeeeeeee wttps//ooomniddddmn wttp:rrrrdddddppaavvrtr...
[50] Hyunwoo Ryu, Hong in Lee, Jeong-Hoon Lee, and Jongeun Choi. Equivariant Descriptor Fields: Fquivarqua Energy-Basee Models for End-to-End Visual Robotic Manipulation Learning. In The Eleventh International Conference on Learning Representations, 2023.
[51] Hyunwoo Ryu, Jiwoo Kim, Junwoo Chang, Hyun Seok Ahn, Joohwan Seo, Taehan Kim, Jongeun Choi, and Roberto Horowitz. Diffusion-EDDFs Bi-equivariant Denoising Generative Modeling on SE(3) for Visual Robotic Manipulation. arXiv preprint arXiv:2309. 02685, 2023,
[52] Anthony Simeonov, Yilun Du, Andrea Tagliasacchi, Joshua B Tenenbaum, Alberto Rodriguez, Pulkit Agrawal, and Vincent Sitzmann. Neural Descriptor Fields: SE(3)-Equivaris Object Rep- resentations for Manipulation. In 2022 International Conference on Robotics and Automation (ICRA), pages 6394-6400 IEEE, 2022.
[53] Anthony Simeonov, Yilun Du, Yen- Yen-Chen Lin, Alberto Rodriguez Garcia, Leslie Pack Kaelbling, Tomás Lozano- Pérez, and Pulkit Agrawal. Equivvarat Relational Rearrangement with Neural Descriptor Fields. In Conference on Robot Learning, pages 835-846. PMLR, 2023.
[54] Shuran Song, Andy Zeng, Johnny Lee, and Thomas Funkhouser. Grasping in the wild: Learning 6dof closed-loop grasping from low-cost demonstrations. IEEE Robotics and Automation Letters, 5(3):4978 4985, 2020.
[55] Yang Song and Stefano Ermon. Generative Modeling by Estimating Gradients of the Data Distribution. Advances in Neural Information Processing Systems, 32, 2019.
[56) Chenrui Tie, Yue Chen, Ruihai Wu, Boxuan Dong, Zeyi Li, Chongkai Gao, and Hao Dong. Et-seed: Efficient trajectory-level se (3) equivariant diffusion policy. arXiv preprint XXXi::2411.03990 2024.
[57] Dian Wang, Robin Walters, Xupeng Zhu, and Robert Platt. Equivariant Q Learning in Spatial Action Spaces. In 5th Annual Conference on Robot Learning, 2021.
[58] Dian Wang, Mingxi Jia, Xupeng Zhu, Robin Walters, and Robert Platt. On-Robot Learning With Equivariant Models. In 6th Annual Conference on Robot Learning, 2022.
[59] Dian Wang, Robin Walters, and Robert Platt. FQuia--qnna Reinforcement Learning. In International Conference on Learning Representations, 2022.
13

---

[60] Dian Wang. Stephen Hart, David Surovik, Tarik Kelestemur, Haojie Huang, Haibo Zhao, Mark Yeatman, Jiuguang Wang. Robin Walters, and Robert Platt. Equivariant diffusion policy. In 8th Annual Conference on Robot Learning, 2024. URL ttattp:://opereviiee net/ forum? id= wD2kUVLT1g.
[61] Zhendong Wang, Jonathan J Hunt, and Mingyuan Zhou. Diffusion Policies as Expressive Policy Class for Offline Reinforcement Learning. In The Eleventh International Conference on Learning Representations, 2022.
[62] Maurice Weiler and Gabriele Cesa. General e -2uequvrari steerable cnns. Advances in Neural Information Processing Systems, 32, 2019.
[63] Ziniu Wu, Tianyu Wang. Chuyue Guan, Zhongjie Jia, Shuai Liang, Haoming Song, Delin Qu, Dong Wang. Zhigang Wang, Nieqing Cao, al. Fast-umi: A scalable and cerdware-independeee universal manipulation interface. arviv preprint arXiv:2409. 19499, 2024.
[64] Zhou Xian, Nikolaos Gkanatsios, Theophile Gervet, Tsung- Wei Ke, and Katerina Fragki- adaki. ChainedDiffuser: Unifying Trajectory Diffusion and Keypose Prediction for Robotic Manipulation. In 7th Annual Conference Robot Learning, 2023.
[65] Jingyun Yang. Ziang Cao, Congyue Deng, Rika Antonova, Shuran Song, and Jeannette Bohg. Equibot: SIM(3) Jequivaaqa diffusion policy for generalizable and data efficient learning. In 8th Annual Conference on Robot Learning, 2024. URL https /wwp///eep//e/ee////////e/ wtwpearrriie..ee/fruunnt net/forum? id=ueBmGhLOXP.
[66] Jingyun Yang. Congyue Deng, Jimmy Wu, Rika Antonova, Leonidas Guibas, and Jeannette Bohg. Equivact: Sim (3)-cquivariant visuomotor policies beyond rigid object manipulation. In 2024 IEEE international conference on robotics and automation (ICRA), pages 9249-9255. IEEE, 2024.
[67] Sarah Young, Dhiraj Gandhi, Shubham Tulsiani, Abhinav Gupta, Pieter Abbeel, and Lerrel Pinto. Visual imitation made easy. Conference on Robot learning, pages 1992-2005. PMLR, 2021.
[68] Yanjie Ze, Gu Zhang, Kangning Zhang, Chenyuan Hu, Muhan Wang, and Huazhe Xu. 3d diffusion policy: Generalizable visuomotor policy learning via simple 3d representations. In Science and Systems, 2024.
Robotics:
[69] Andy Zeng, Shuran Song, Stefan Welker, Johnny Lee, Alberto Rodriguez, and Thomas Funkhouser. Learning synergies between pushing and grasping with self-supervised deep reinforcement learning. In 2018 JEEE/RSJ International Conference Intelligent Robots and Systems (RROS), pages 4238 4245. IEEE, 2018
[70] Andy Zeng, Pete Florence, Jonathan Tompson, Stefan Welker, Jonathan Chien, Maria Attarian, Travis Armstrong, Ivan Krasin, Dan Duong, Vikas Sindhwani, et al. Transporter networks: Rearranging the visual world for robotic manipulation. arXiv preprint arXiv:2010. 14406, 2020.
[71] Haibo Zhao, Dian Wang, Yizhe Zhu, Xupeng Zhu, Owen Howell, Linfeng Zhao, Yaoyao Qian, Robin Walters, and Robert Platt. Hierarchical equivariant policy via frame transfer. In Forty- second International Conference on Machine Learning, 2025. URL https: /ppnreviie..r.ee net/forum? http:/oorumm.ddnnS5teettt 1d=nAvtketHi...
[72] Tony Z Zhao, Vikash Kumar, Sergey Levine, and Chelsea Finn. Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardware. In Proceedings of Robotics: Science and Systems (RSS), 2023.
[73] Tony Z. Zhao, Jonathan Tompson, Danny Driess, Pete Florence, Seyed Kamyar Seyed Ghasemipour, Chelsea Finn, and Ayzaan Wahid. ALOHA unleashed: A simple recipe for robot dexterity. In 8th Annual Conference on Robot Learning, 2024. URL https: / //openreview
did==xdxE777 IkHI.
[74] Xupeng Zhu, Dian Wang, Ondrej Biza, Guanang Su, Robin Walters, and Robert Platt. Sample Efficient Grasp Learning Using Equivariant Models. In Robotics: Science and Systems, 2022,
[75] Xupeng Zhu, Dian Wang, Guanang Su, Ondrej Biza, Robin Walters, and Robert Platt. On robot grasp learning using equivariant models. Autonomous Robots, 4708):1175-1193, 2023.
14

---

A Proof of Proposition
Proof. 1. Absolute trajectory equivariance: Given an absolute trajectory action = {A++) each pose Asti is defined in the world frame. Under the transformation g. each pose transforms as Thus, by definition, the absolute trajectory transforms equivariantly:
gAr+i, O,...,, n
g = {gA,, gAr+1,....
Relative trajectory invariance: Consider a relative trajectory action a" : defined in the local gripper frame at the initial time step f. The corresponding absolute poses are obtained as A++: = T Under the global transform g. the absolute pose becomes g- --A++ gT, Since the relative pose A4+s appears as right multiplication factor, it remains unchanged under the global transform. Hence, have invariance:
9 g.a" = ===""a"
3. Delta trajectory invariance: For a delta trajectory action a² = { 1n-1 each incremental transform Af+: is expressed in the gripper's local frame at time + 1. The absolute pose reconstruction is given by As+: = Te I1; Aftj- Under the global transform g. we have AA++ = I Af+z. where each incremental transform Afiy is multiplied on the right and thus remains unaffected by the global transformation g. Therefore, the delta trajectory action invariant:
g gaa =a==aa
B Proof of Proposition 2
Proof. We treat the two cases in parallel. both, the policy
T(o)) {Ac++]=0
outputs sequence of absolute poses A4+1 SE((). Internally it first predicts "local" sequence (o.) )/ 7 (o,) {A 1n-1 (either relative or delta) and then reconstructs absolute poses by anchoring to the current gripper pose Tr.
Case 1: Relative trajectories. By Definition 2,
Atti Af
i= 0, ......... 1,
where Ah4 is the ith pose in the relative sequence म (0,) = (1 Thus
T(o.) (T A1++) 1ー
By assumption, does not depend on the gripper pose T. explicitly. Thus, applying the transformation g to the observation has effect the relative trajectory prediction:
(g ot) (I. g&T, w.) म (Ie, Tr, wt) a
(7)
Reconstructing absolute poses from the transformed observation gives, for each i,
[T(g- ot]] (gT,) A : = 9 (T, A
Acti [T()))];
Because this holds for all 0, ... 1, conclude
(g 0.) = g (or).
Case 2: Delta trajectories. By Definition, the della-reconstruction
Aot: T, [II A A 0. .2
where {Att,) } is the delta-sequence from ग (o:). Again invariance of under g gives #(g-o:) = ((:)).
so
(g)), (gT))
[I A(-s) 9 [T. 1=
II ==0
At,) = g. A = g [T(O))]
Hence once again
(.) = g (0x).
In both cases we have shown that the entire trajectory satisfies T(g ot) g =(O:), i.e. x (B((3)-qqivarrt)
0
15

---

Lill
Can
Square
tool hang
Obs
Action
Method
Moan
100 PH
100 MH
100 PH
100 MH
100 PH
100 MH
100 PH
Large FOV In Hand
Rel Traj
93.4
100.0:1:000
100.0:10.0
99.3:1-077
100.0:0.00
88.12.33
92.0:1000
74.74:2.4
Voxel
Abs Traj
EquiDiff
90.4
100.0:10.0
100.0:10.0
99.3.1.0.7
84.0.1.. 96.7.0.77
76.7 76.0.10.0
In-Hand + External
Abs Traj
DifiPo
87.9
100.0:10.00
100.0=00.0
100.0::0.00
95.3:0077
05.340.7
70.740077
64.045.8
Table 3: The performance of our method compared with the baselines in Robomimic. We experiment with 100 Proficient- Human (PH) Hultii-Human (MH) demos in each environment. Results averaged over three seeds. + indicates standard error.
C Training Detail
We follow the training setup and hhper-parameee of the prior works [5, 60, 7]. Specifically, our RGB observation has a size of 3 x 84 x 84 (which will be random cropped to 3 x 76 x 76 during training), and all tasks have full 6 DoF SE(3) action space. The observation contains two steps of history observation, and the output of the denoising process is a sequence of 16 action steps. We use all 16 steps for training but only execute eight steps in evaluation. In all pretraining encoder variations, we use two steps of proprioceptive observation but only one step of visual observation, following Chi et al. [7]. The vision encoder's output dimension is 64 for for CNN Enc (following [5]), x for Equi Enc (128 channel regular representation of Cg. following [60]), and 512 for Pretrain (following [7]). The diffusion UNet has [512. 1024, 2048] hidden channels for end-to-end training variations (following Chi et al. [5]), and [256, 512, 1024] hidden channels for pretraining encoder variations (following Chi et al. [7]). We train our models with the Adam W [39] optimizer (with a learning rate of 10 and weight decay of 10-9) and Exponential Moving Average (EMA). We use cosine learning rate scheduler with 500 warm-up steps. We use DDPM [18] with 100 denoising steps for both training and evaluation. We perform training for 600 epochs, and evaluate the method every 10 episodes (60 evaluations in total). All trainings are performed on single GPU, where we perform training on internal clusters and desktops with different GPU models. Each training of the Pretrain + FA method takes from 3 hours (Stack DI) to 24 hours (Pick Place DO), due to the different sizes of the dataset. The total amount of compute used in this project roughly 3000 GPU hours.
D Robomimic Experiment
In this section, we perform an experiment in the Robomimic [41] enviromments. We compare our Pretrain + FA method against EquiDiff [60] and the vanilla Diffusion Policy [5]. As shown in Table our method generally outperforms the baselines, achieving an average improvement of 3% compared with EquiDiff.
E Pretraining and Frame Averaging with External View
In this experiment, we extend our analysis from Section 5.2 to the external view setting to verify whether our findings generalize across different observation configurations. Specifically, we perform an additional experiment on using pretrained encoders (with and without Frame Averaging) with In-Hand + External view. Similar to Section 5.2, we consider three methods in each view setting: 1) No Pretruin: A standard ResNet-18 [17] trained from scratch; 2) Pretrain: A standard ResNet- 18 pretrained on ImageNet- 1k [12], without explicit symmetry constraints; 3) Pretrain + FA: pretrained ResNet-18 18 enhanced with Frame Averaging (Fquation 5) to achieve Ca-equivariance without modifying the underlying network architecture.
As shown in Table 4, the benefits of Frame Averaging remain consistent across both observation settings. In the In-Hand + External view setting, Pretrain + FA yields 6.7% improvement compared with not using Frame Averaging (Pretrain). and a 15.5% improvement compared with training from scratch (No Pretrain). Notably, Pretrain + FA in In nn-and External view outperforms EquiDiff (Im) in all tasks, and even achieves a I 1% higher average performance compared with EquiDiff (Vo). This is particularly impressive considering the additional complexity of EquiDiff, as discussed in Section 5.2. Comparing Pretrain FA across different views, despite using additional camera, In- Hand + External only outperforms Large FOV In-Hand by 2.5%. This finding verifies our analysis in Section 4.2, and suggests Large FOV In Hand is preferable in many applications due to its simplicity
16

---

Table 4:
The performance comparison of pretrained encoder with Frame Averaging, pretrained encoder, and no pretraining. All policies are trained using 100 expert demonstrations. We perform 60 evaluations (each with 50 policy rollouts) throughout the training, and report the best average success rate. Results averaged over three seeds, + indicates standard error. Dark green box indicates outperforming Equi with voxel inputs; Light green box indicates outperforming EquiDiff with image inputs; Yellow box indicates underperforming both. Bold indicates whether the symmetric encoder outperforms the mon-symmetric version the corresponding training setting. Performance of EquiDiff is reported by [60].
Method
Stack
Square
Threading
Coffee
Pretmuin FA
64,9
100.0
0.77.10.7
42.7
28.0.11.2
72,
2.3
In-Hand Ext. Rel Traj
Pretmin
58.2
10000:10.0
72.742244
33.341:3
22.0:1:.00
63.720.7
18.01:200
No Pretrain
49.4
89.3
52.7=:3.7
20.7+11.
18.7#3.5
63.3
3.3:b0.7
Pretrain FA
61.4
100.0:0000
86,7::1.88
43,3:20.77
16.7#18
63,3=007
28.7:+188
Large FUV
In In.n
Rel Traj
Preteuin
57.3
1.0
78.0:12.00
33.31118
18.0:1122
65.3
14.01:0.0
No Pretrain
46.7
98.0.10.0
72.0:11.2
16.0:11.2
16.0.11.2
1..71
4.0.1.2
In Hand Voxel
EquiDaiT (Im)
93.3
$4.7::522
25.3:18.7
22.0::1.2 38.7+0077
60.01:2.0 64.7=0.7
15.3118 37.3#2.77
Abs Traj
EquiDitf (Vo)
63.9
98.7
74.7:4444
38.7+1.
Obs
Action
Method
Hammer CL DI
Mug CL DI
Kitchen DI
Not Asse. DO
Pick Place DO Coffee Prep. DI
1.
55.3:113
82.7110.7
78.0::155
6.6
78.01:1.2
In no.T + Ext.
Rel Traj
Pretrain
61.3110.77
54.01112
76.0.11.2
72.3:117 57.34:09
$5.5
69.3.1
58.7200.
46.740077
62.042.0
39.8 1.2
20.0::2.0
Pretrain + FA
76.0=22.0
60.0:22.3
78.7+188
74.7:115
50.8=0.7
2..0=
Large FOV
Rel Traj
Pretrain
76.01:1.
62.7:1:188
80.7::24
6100::2.11
48.2+:122
In-Hand
No Pretrain
60.7
50.7:11..8
68.713.77
44.7:1333
4..71:16
30.7.1 1.8
In-Hand Ext.
EquiDitI (Im)
65.3+0077 70.0=22.0
49.3:20077 52.7
67.3-20077 85.3:0077
74.0#11. 67.3#0099
41.7=3.2
76.7+
Voxel
EquiDiff (Vo)
$7.7 1.8
80.0
Table 5: Validating the Pretrain+FA encoder in absolute trajectory and multi-camera observations.
Obs
Action
Method
Mean
Stack Three D1
Square D2
Coffee D2
Nut Assembly DO
36.2
38.0:10.00
8.0:::1.2
44.0:1122
$4.71:2.33
In-Hand Ext.
Abs Tmi
DP + Pretmain FA
57.7
58.7.11.2
38.0.19.2
56.0.1200
78.0.12.66
EquiDil
$3.5
54.7:15.2
25.3:18.7
60.0:1:2.
74.0:1122 76.7..3.2
EquiDiff + Pretrain FA
63.2
68.7.12.3
29.3.13.0
78.0.1..0
(requiring only single eye-in-hand camera) and easier transferability to diverse robotic platforms beyond tabletop manipulation scenarios.
Pretraining and Frame Averaging with Absolute Action and Multi-Camera Observation
To demonstrate generality beyond the in-hand setup, we evaluate the proposed Pretrain+FA encoder under the same observation/actioon setting Diffusion Policy and EquiDiff (in-hand + external views and absolute actions) in four different environments. As shown in Table 5, employing the FFe encoder yields significant 21.5% and 9.7% improvement for Diffusion Policy and EquiDiff, respectively. These results confirm the encoder's 's plug-and nature across observation and action parameterizations.
G Ablation Study
G.1 Isolating Relative Trajectory and Symmetric Feature Extraction
We explicitly isolate each component in our design. As shown in Table 6, starting from the full model, replacing the relative trajectory with absolute trajectory reduces performance by 11.1% across four tasks, while replacing the encoder with non-pretrained CNN reduces performance more significantly by 19.6%. This result highlights the complementary benefits from symmetry-aware features and invariant action parameterization.
17

---

Table 6: Ablation isolating the contributions of (i) relative trajectory and (ii) symmetric feature extraction (Pretrain+FA).
Obs
Rel Tni
Pretrain
Mean
Stack Three
Square
Coffee D2
Nut Asse. DO
67.0
86.74 1.8
43.31:0.7
63.3-10.77
74.74:1.5
In-Hand
FOV
V
55.9
68.742.3
26.7115
60.0:::.0
68.0+:3.0
X
47.4
72.0
16.01.22
56.7+1.8
44.7+3..
G.2
Ablating Proprioception
We test Proposition 2's assumption by removing the gripper pose from the policy input in Table 7. As expected, this tighter symmetry assumption reduces performance-indicating that proprioception. while conmeety--breaingg provides valuable context.
Table 7: Ablating proprioception input in the policy
Stack Three DI
Square D2
proprioception
No proprioception
64.7
14.7
G.3 Ablating Symmetry Group
We vary the SO(2) discretization used by the equivariant encoder. As shown in Table reducing the cyclic group order degrades performance (C8 > C4 > C2), aligning with prior observations [62, 58] that C8 is practical sweet spot for 2D rotational symmetry.
Table Effect of SO(2) discretization.
Group
Stack Three DJ
Square D2
C8
75.3
32.0
C4
68.0
30.7
H EquiDiff with Large FOV In-Hand Only
In this experiment, we evaluate EquiDiff with only large-FOV eye-in-hand camera setting. The comparison is shown in Table 9. Compared to the original external-camera configuration, EquiDifffss mean success drops by 12.8%, underscoring the benefit EquiDiff derives from external viewpoint signals. This result also justifies our choice of using the original EquiDiff observation setting in our main results.
Table 9: Performance of EquiDiff under large FOV in-hand-only camera setting vs. the original in-hand external and voxel reduces EquiDiff performance on several tasks.
settings. Removing external cameras substantially
Action
Square Threading
Large FOV In-Hand
EquiDiff (Im) 40.9
96.0=0..0
61.325.0
8.741. 25344877
13.342.,3
47.3+33.1
3.3+1:12
In-Hand
Traj EquiDiif (Im)
93.3
54.71:52
22.0::12
60.0::2.0
15.3::1.8
Moxel
EquiDill (Vo) 63.9
98.7.10.7
7..7144.4
38.7111
38.7..0.7
64.7.10.77
37.3
Obs
Action Method
Hammer CL DI
Muy CL. D1
Kitchen DI Nut Asse. DO Pick Place DD Coffee Pre. DI
Large FOV Ia-Hand
EquiDiff (Im)
59.3-14.2
50.74223
55.3411.2
40.0:1:.0
27.7412.9 41.743.2 57.7::188
2.2
In-Hand Ext.
Abs Traj
EquiDiff (Im)
65.3:10.7
49.3 49.3:2077
67.34:077
74.0:::22 67.3::0.9
76.7-0077
Voxel
EquiDiff
70.0:22..0
$2.72:1.
85.3:0077
80.0:11.2
18

---

I
Broader Impact
This work has couple of positive and negative social impacts. First, we provide a simple policy learning framework from eye-in-hand images, which could benefit the development of household robots assistive robots that could be beneficial for society. However, since it is behavior cloning algorithm and the robots' behavior completely depends on the training data, it could also potentially be used to train robots with harmful behavior. Therefore, is important for future works to highlight safety monitoring, especially when deploying in real-world environments with human presence.
19

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: Our abstract and introduction clearly state the claims of this paper, including the contributions and the assumption that using eye-in-hand -nnd camera. The claims made match our theoretical and experimental results.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation long it is clear that these goals are not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification: We include a detailed limitation section in the paper, including the assumption and potential negative impact of using eye-in-hand camera, the limitation of the testing dataset, and the computational efficiency.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g.. independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or speech-to-text system might not be used reliably provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairesss.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't acknowledged
the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
20

---

Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
Justification: All the definitions and propositions in the paper are numbered and cross- referenced. All the assumptions clearly stated.
Guidelines:
The answer NA means that the paper does not include theoretical results. All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
Justification: We will release the code of this paper, including the code for all models and experiments. We also discuss the training detail in the appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
the paper includes experiments, a No answer to this question will not perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution a novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how replicate the results, access hosted model (e.g.. in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily new algorithm, the paper should make it clear how to reproduce that algorithm.
(b) the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (c.g., to registered users), but it should possible for other researchers to have some path to reproducing verifying the results.
21

---

5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, described in supplemental material?
Answer: [Yes]
Justification: We include the code for all experiments in the supplementary material, includ- ing data generation and all the models. All the results will be reproducible using the code. In the final version of the paper, we will provide GitHub repository for the code of the project.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (uttps://nips.. cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https:
/ttppsccc/ppp//a wwtpccccccpppccccccccooooocp gew/d//s///i////eeeeeeeeeseeese/ SCwp:/eeieeeeseeeeeeeeeeea wwwamiisssoooicioooooooccc for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing much information possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: We provide the experimental setting in both the paper and the code.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification: We include the standard error of all our experiments, which is calculated from three random seeds.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by emror bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
22

---

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not show in tables figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification:
We discuss the compute resources in the appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics https: /et::://neueiiie /nerriss....p// cc wctppblic//eceecccccceccc wtwp:tthhhhhceeeeeieeeeeeeeeeeee
Answer: [Yes]
Justification: Our work conforms with the NeurIPS Code of Ethics in every respect.
Guidelines:
The answer NA means that the authors have not reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- cration due to laws or regulations their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: We include broader impact statement in the appendix, discussing the potential positive and negative societal impacts.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
23

---

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), faimess considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: Our work does not have high risk of misuse.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but encourage authors to take this into account and make best. faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: All the prior works used are properly cited, we will also cite the codebases used in the paper in the GitHub repo
Guidelines:
The answer NA means that the paper does existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset used and, possible, include a URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
24

---

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided. If this information is available online, the authors encouraged to reach out the asset's s creators.
13. New assets
Question: Are assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: We will include those in the supplementary material as well the GitHub repo for the final submission.
Guidelines:
The answer NA means that the paper does not release assets.
Researchers should communicate the details of the dataset/code/mooel as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
This paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then much detail as possible should be included the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, curation, other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification:
This paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does involve crowdsourcing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
25

---

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Question:
Answer: [NA]
The core method development in this research does involve LLMs as any important, original, or non-standard components. We only use LLM for editing, which is denoted in Openreview.
Justification:
Guidelines:
The answer NA means that the method development in this research does not involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy (https: //tp/r/eess.///e/ee Cttpppps.cc///oooo cc/Conf//e/ceeeeec "Conferrence//2255LIM for what should or should be described.
26


Reviews:
[{"review_id": "21NBIKUTdK", "reviewer": "Reviewer_ikDP", "summary": "This paper presents a guide on how to easily incorporate symmetry into diffusion policies. They propose a combination of in-hand camera observation, relative action commands, and equivariant image encoders. The authors evaluate their approach in simulation benchmarks and show that their simple approach offers comparable performance to fully equivariant network designs.", "questions": "1. Since the design incorporates symmetry, will it perform better in low-data regimes? All of the datasets used are fairly abundant in the experiments.\n2. As mentioned in the paper that frame averaging slows down the training process. Does it also slow down the inference process? Will it limit the performance of the policy in realworld settings if the latency of the model increases?\n3. It would be interesting to see how this work can be extended to external observation cameras.", "limitations": "Yes.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 3, "originality": 2, "strengths_and_weaknesses": "Strength:\n\n1. Proposes a simple yet effective way of designing diffusion policies that incorporate symmetry.\n2. Clearly documents expeirments designs and results. Overall paper is clear and well presented.\n3. Offers a interesting proof on how in-hand observations + relative actions leads to SE3 equivariance under assumptions.\n\nWeakness:\n\n1. No real-world experiment to evaluate the proposed approach.\n2. Limited experimental results. The result from using relative actions is not very interesting as it was already proposed in the oringal DP paper and widely used in diffusion policies these days.\n3. Action selection is well explored in the community, and the use of equivariant encoders and in-hand observations are not uncommon in robot policy learning.", "was_revised": true, "final_justification": "I gave the paper a score of 4. The paper is well written, and the experiments and methods are clearly documented and explained. The authors' rebuttal also addressed almost all of my questions, particularly clarifying my misunderstanding of action selection, and included explanations of the lack of real-world experiments, as well as how this method could be applied to external observations. The only drawback I think is the novelty and performance of the method. It utilizes only equivariant encoders + relative action representation, which are both existing methods, and the final performance is slightly lower than that of more sophisticated methods. However, I believe the paper does provide a good contribution in guiding how to incorporate symmetry in diffusion policies with just simple methods.", "rebuttal": "The authors thank the reviewer for their thoughtful review. Please see our response below:\n\n> No real-world experiment to evaluate the proposed approach.\n\nWhile we do not explicitly include real-world experiments in this paper, the assumptions that we are making in our simulation experiments are well-aligned with standard real-world experimental settings (i.e., the only information that we are acquiring from the simulation is the eye-in-hand image and the proprioceptive information of the gripper). Consequently, our findings should transfer easily to the real world. Moreover, one important goal of this paper is to systematically evaluate a number of practical methods for incorporating symmetry into Diffusion Policy. This requires abundant and rigorous comparisons that are more controllable in simulation than the real-world.\n\n> Limited experimental results. The result from using relative actions is not very interesting as it was already proposed in the oringal DP paper and widely used in diffusion policies these days.\n\nThe original DP did not use the same relative action definition as our paper. Instead, their relative action is a delta action defined in the **world frame** (referred as velocity control in the original DP paper), following the RoboMimic action space convention. This action formulation does not have any inherent symmetry, thus it underperforms the absolute action in the original DP paper. The relative action studied in this paper, however, is defined in the **gripper frame**, thus its symmetric property discussed in section 4.1 enables an outperformance compared with absolute control.\n\n>  Action selection is well explored in the community, and the use of equivariant encoders and in-hand observations are not uncommon in robot policy learning.\n\nOur method uses an equivariant encoder with a non-equivariant policy, as well as frame averaging for bridging pretrained encoders and equivariance, both of which have never been done before in robot learning. Prior works in equivariant robot learning all focus on defining an end-to-end equivariant policy, which is much more complex mathematically and implementationally compared with our approach. Regarding action selection, although this is indeed a question that has been studied a lot in the community, it remains uncertain what is the best action formulation in the context of Diffusion Policy. Moreover, we believe re-thinking about this question from a symmetry perspective will also offer the community some new insights. \n\n> Since the design incorporates symmetry, will it perform better in low-data regimes? All of the datasets used are fairly abundant in the experiments.\n\nYes, it will perform better in low-data regimes. In fact, all of our experiments use 100 demonstrations, which is already in the low-data regime (e.g., compared with the 200-300 demonstrations per task used in the original DP paper and the UMI gripper paper). If we reduce the number of demonstrations further, we believe the performance of all methods will decrease, but the impact on symmetric methods will be smaller.\n\n> As mentioned in the paper that frame averaging slows down the training process. Does it also slow down the inference process? Will it limit the performance of the policy in realworld settings if the latency of the model increases?\n\nThis is a good question. In our experiments, the inference for frame averaging is the same as the baseline. One advantage of frame averaging is that it typically does not slow down inference, especially in the robot policy inference where the inference batch size is normally 1. Frame averaging essentially increases the batch size by the size of the frame (8 in our case), which does not affect inference time due to batch processing.  \n\n> It would be interesting to see how this work can be extended to external observation cameras.\n\nThis is a very important point, but we would like to clarify that we do have this experiment in Appendix E, where we found that the proposed method in external observation can also improve the performance by a significant margin. \n\nIn addition, we performed an additional experiment validating the Pretrain + FA encoder (with either equivariant diffusion head or standard diffusion head) in the external observation and absolute action space (i.e., same setting as EquiDiff). As shown in the table below,our method is also effective in the external observation and absolute action space setting.\n|                          | Stack Three D1 |   Square D2  |\n|:------------------------:|:--------------:|:------------:|\n| EquiDiff                 | 54.7           | 25.3         |\n| EquiDiff + Pretrain + FA | 68.7 (+14.0)   | 29.3 (+4.0)  |\n| Diff + Pretrain + FA     | 58.7 (+4.0)    | 38.0 (+12.7) |"}, {"review_id": "UO8BG42T0S", "reviewer": "Reviewer_zdCn", "summary": "The paper proposes and evaluates several methods to gain the benefits of symmetry-awareness in robotic manipulation policies without the high implementation complexity of fully equivariant models.\nThe authors show that through a combination of invariant representations (eye-in-hand camera with relative actions), equivariant vision encoders and symmetrizing pretrained encoders with Frame Averaging, they can achieve performance comparable to state of the art equivariant architectures while using a much simpler and more practical setup.", "questions": "- I think it would be interesting to see an experiment where the proposed Pretrain+FA was applied to the same multi-camera, absolute action setting as EquiDiff. Additionally, would it be possible to run EquiDiff in your eye-in-hand setup? I think having these two experiments would strengthen the paper", "limitations": "Yes", "rating": 4, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "STRENGTHS:\n- The authors formally prove that the combination of eye-in-hand perception and relative actions parametrisation leads to inherent SE(3)-invariance. While components are not new, this specific formalisation is novel as far as I am aware\n- The paper does a good job at systematically analysing these known techniques and how they can be practically integrated into a feasible and effective setup for robotic learning\n- The authors obtain good results given the simpler setup, effectively, they show that for a minor drop in performance, they can disregard three RGBD cameras and the complex voxelisation / equivariance network pipeline. \n- As the title suggests, the paper provides a clear, practical guide backed by systematic experimental / empirical findings. \n\nWEAKNESSES:\n- The novelty is limited, as the authors are simply combining and analysing different known methods\n- The comparisons in Table 2 are not completely fair, as they are comparing their method (which uses an eye-in-hand camera) to EquiDiff (using absolute actions and multiple external cameras). Yes, it is a simpler setup, and the authors argue the setup is a key part of their solution, but I still find the comparison a bit awkward.", "was_revised": true, "final_justification": "I appreciate the new ablations and I think they really add value to the insights of the paper. I'm still on the fence about the originality of the work, but I think the paper presents the information well and the experiments are well done.", "rebuttal": "We thank the reviewer for their insightful suggestions. Please see our response below.\n\n> The novelty is limited, as the authors are simply combining and analysing different known methods\n\nWe would like to clarify three novelties of our paper:\n1. A theoretical analysis of the symmetry of relative trajectory and eye-in-hand perception.\n2. The use of an equivariant encoder but non-equi policy, rather than a fully equivariant policy as commonly done in prior works, for incorporating symmetry. \n3. The use of Frame Averaging to bridge the benefit of equivariant encoding and pre-trained vision encoders.\n\nTo the best of our knowledge, we are the first to accomplish each of those three points in the robot learning literature. We thank the reviewer for raising this concern, we will clarify the novelty of our work in the revision.\n\n> The comparisons in Table 2 are not completely fair, as they are comparing their method (which uses an eye-in-hand camera) to EquiDiff (using absolute actions and multiple external cameras). Yes, it is a simpler setup, and the authors argue the setup is a key part of their solution, but I still find the comparison a bit awkward.\n\nThis is a good point, thanks for bringing it up. The reason for using the external camera for EquiDiff is two-folds: 1. This is the setting proposed by the prior work and 2. Using EquiDiff with only a large FOV in-hand camera will decrease its performance, due to the lack of equivariant signal from the external camera. However, we do agree with the reviewer that using a different camera setting can raise fairness concerns. To address this, we performed an additional experiment for EquiDiff using a large FOV in-hand camera. As shown in the table below, the performance of EquiDiff dropped significantly without accessing the external camera. \n|                        | Mean         | Stack | Stack Three | Square | Threading | Coffee  | Three Piece Assembly | Hammer Cleanup | Mug Cleanup | Kitchen | Nut Assembly | Pick Place | Coffee Preparation |\n|------------------------|--------------|-------|-------------|--------|-----------|---------|----------------------|----------------|-------------|---------|--------------|------------|--------------------|\n| EquiDiff IH + External | 53.7         | 93.3  | 54.7        | 25.3   | 22.0      | 60.0    | 15.3                 | 65.3           | 49.3        | 67.3    | 74.0         | 41.7       | 76.7               |\n| EquiDiff Large FOV IH  | 40.9 (-12.8) | 96.0  | 61.3        | 8.7    | 13.3      | 47.3    | 3.3                  | 59.3           | 50.7        | 55.3    | 40.0         | 27.7       | 27.3               |\n\n> I think it would be interesting to see an experiment where the proposed Pretrain+FA was applied to the same multi-camera, absolute action setting as EquiDiff. Additionally, would it be possible to run EquiDiff in your eye-in-hand setup? I think having these two experiments would strengthen the paper\n\nThis is also a great point, thank you for the suggestion. Demonstrating the performance of Pretrain + FA with multi-camera, absolute action setting as EquiDiff would indeed demonstrate the generalizability of our proposal. We performed a new experiment in this suggested setting for two methods: (1) EquiDiff + Pretrain + FA: replacing the encoder in EquiDiff with our Pretrain + FA encoder. (2) Diff + Pretrain + FA: using our Pretrain + FA with the original diffusion head, similar as our method in Table 2. As shown in the table below, the Pretrain + FA encoder can be used with either EquiDiff or the standard diffusion head, both performing better than the baseline EquiDiff. In addition, we had an experiment of Pretrain + FA in the multi-camera (with relative action though) in Appendix E. We believe these results well demonstrate the generalizability of our Pretrain + FA encoder, improving baseline performance in various different settings.\n|                          | Stack Three D1 |   Square D2  |\n|:------------------------:|:--------------:|:------------:|\n| EquiDiff                 | 54.7           | 25.3         |\n| EquiDiff + Pretrain + FA | 68.7 (+14.0)   | 29.3 (+4.0)  |\n| Diff + Pretrain + FA     | 58.7 (+4.0)    | 38.0 (+12.7) |\n\nWe also performed the requested EquiDiff in eye-in-hand setup, please see our response to the question above."}, {"review_id": "sunrRJHqnH", "reviewer": "Reviewer_yST9", "summary": "This paper proposes practical strategies to incorporate spatial symmetry into diffusion-based policy learning frameworks without the overhead of fully equivariant architectures. The authors demonstrate that using SE(3)-invariant representations—specifically relative trajectory actions and eye-in-hand perception—naturally leads to improved generalization. Additionally, the work introduces the use of equivariant vision encoders and Frame Averaging to leverage symmetry-aware features from pretrained networks. Across 12 tasks in the MimicGen benchmark, the proposed methods achieve performance competitive with or better than state-of-the-art equivariant policies while maintaining simplicity in design and implementation.", "questions": "- How much additional computational cost is introduced by Frame Averaging, and how does this compare to training from scratch with a full equivariant architecture?", "limitations": "Yes", "rating": 4, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strengths:\n- The paper presents a thorough theoretical analysis backed by extensive empirical validation on a standard benchmark. Results are clearly reported and systematically compared.\n- The writing is clear and well-organized.\n- The work addresses a critical trade-off in robot learning: the complexity of incorporating symmetry versus the practical need for deployable, performant policies. This makes the contribution broadly relevant to practitioners and researchers.\n\nWeaknesses:\n- Proposition 2 assumes the policy does not depend on the gripper pose $T_t$, yet the authors themselves acknowledge this is not strictly true in practice. However, no ablation is provided to quantify how much performance degrades due to residual $T_t$-dependence. As a result, the claim that “approximate invariance” suffices remains qualitative and lacks experimental support.\n- The experimental sections evaluate combinations of techniques (e.g., relative trajectories with equivariant encoders), but do not fully isolate the contributions of each component via comprehensive ablations.", "was_revised": true, "final_justification": "The authors’ response has addressed most of my concerns, and I have accordingly adjusted my score.", "rebuttal": "We thank the reviewer for their careful review. Please see our response below.\n\n> Proposition 2 assumes the policy does not depend on the gripper pose , yet the authors themselves acknowledge this is not strictly true in practice. However, no ablation is provided to quantify how much performance degrades due to residual -dependence. As a result, the claim that “approximate invariance” suffices remains qualitative and lacks experimental support.\n\nWe performed a new ablation study to remove the gripper pose in the policy input. The resulting ablation better satisfies Proposition 2 at a cost of the proprioceptive information. As shown in the table below, removing the gripper pose decreases the performance, justifying our design choice of keeping the pose in the input.\n|                      | Stack Three D1 | Square D2    |\n|----------------------|----------------|--------------|\n| With Gripper Pose    | 72.0           | 16.0         |\n| No Gripper Pose      | 64.7 (-7.3)    | 14.7 (-1.3)  |\n\n> The experimental sections evaluate combinations of techniques (e.g., relative trajectories with equivariant encoders), but do not fully isolate the contributions of each component via comprehensive ablations.\n\nThis is a good point. Although some of the ablations do exist in the paper, they are not organized as an ablation table. We add an additional ablation baseline that uses the Pretrain + FA encoder with absolute trajectory, and explicitly ablates the relative trajectory and equivariant encoder compared to the full model. As shown in the table below, both components contribute significantly to the overall performance.\n|                                              | Stack Three D1 | Square D2    |\n|----------------------------------------------|----------------|--------------|\n| Full (Rel Traj + Pretrain + FA in Table 2)   | 86.7           | 43.3         |\n| - Pretrain - FA (Rel Traj + CNN Enc in Table 2) | 72.0 (-14.7)   | 16.0 (-27.3) |\n| - Rel Traj (new, Abs Traj + Pretrain + FA)   | 68.7 (-18)     | 26.7 (-16.6) |\n\n> How much additional computational cost is introduced by Frame Averaging, and how does this compare to training from scratch with a full equivariant architecture?\n\nFrame averaging increases the GPU memory consumption by increasing the batch size by a factor of the frame size. In our case (8 frames), adding frame averaging to a pretrained encoder increases the GPU memory from 2970GB to 6476GB. Compared to Equi Enc’s 4942GB, frame averaging is slightly heavier (partially because the pretrained resnet-18 is slightly larger than our custom equivariant resnet-18), however, it remains relatively light and can be trained on most of the existing gaming GPU. For larger base models, one can reduce the size of the frame or the training batch size to reduce the GPU memory consumption."}, {"review_id": "iOJtgxDEfU", "reviewer": "Reviewer_jnxb", "summary": "This paper systematically studies different approaches to incorporate symmetries (invariance/equivariance) into diffusion policies.\nSpecifically, the authors compared three action representations (abs, rel, delta), and four image encoders (cnn, equi cnn, pretrained cnn, frame averaging) using 12 robotic manipulation tasks and found that, in general, incorporating symmetries improves the performance of diffusion policies.", "questions": "1. How important is FOV in terms of performance? It appears that there are no direct comparison between normal FOV and large FOV in the experiments, but only large FOV and normal FOV + external/voxel.\n2. Related to (1), would it be possible to also incorporate equivariant CNNs in the normal FOV + external setting? Would it require significant changes to the architecture?\n3. How sensitive is the method's performance to the discretization of SO(2) used in equivariant CNNs? (e.g., would $C_4$ improve/degrade performance)", "limitations": "yes", "rating": 4, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 3, "originality": 2, "strengths_and_weaknesses": "Strengths:\n\nThe paper is clearly written and easy to follow.\nThe systematic evaluation of various techniques for incorporating symmetries provides practical value and can help guide practitioners in the design of network architectures and data collection pipelines.\nThe empirical results showing that equivariant diffusion policies using a single view can perform competitively with more complex voxel-based baselines are particularly interesting and suggest promising directions for future work.\n\nWeaknesses:\n\n1. The action representations proposed in this paper appear to be specifically designed for eye-in-hand robotic setups, and it remains unclear how well these techniques would generalize to other types of tasks or embodiments.\n2. One common problem with incorporating symmetries is that the tasks usually do not fully respect the group structures in general, due to factors like imperfections of the observations. I think it would be beneficial to consider soft variants like [1].\n\n[1] Finzi, Marc, Gregory Benton, and Andrew G. Wilson. \"Residual pathway priors for soft equivariance constraints.\" Advances in Neural Information Processing Systems 34 (2021): 30037-30049.", "was_revised": true, "final_justification": "All my questions were adequately addressed.", "rebuttal": "We thank the reviewer for their helpful comments. Please see our response below.\n\n> The action representations proposed in this paper appear to be specifically designed for eye-in-hand robotic setups, and it remains unclear how well these techniques would generalize to other types of tasks or embodiments.\n\nAlthough relative actions with eye-in-hand perception can achieve a theoretical benefit as discussed in Proposition 2, our action representations are not limited in eye-in-hand robotic setups. As shown in Table 1 and Table 4, relative trajectory works very well with the in-hand + external observation setting. Regarding other types of embodiments, eye-in-hand setting might actually be better for cross-embodiment generalization, as the observation would remain invariant across different robot embodiments.\n\n> One common problem with incorporating symmetries is that the tasks usually do not fully respect the group structures in general, due to factors like imperfections of the observations. I think it would be beneficial to consider soft variants like [1].\n[1] Finzi, Marc, Gregory Benton, and Andrew G. Wilson. \"Residual pathway priors for soft equivariance constraints.\" Advances in Neural Information Processing Systems 34 (2021): 30037-30049.\n\nThis is a good thought, however, unlike the prior works in equivariant policy learning that use end-to-end equivariant networks and might encounter the symmetry breaking problems you mention, our work uses an equivariant encoder followed by a standard diffusion head (see Figure 2). As a result, our policy is not totally constrained by the symmetry of the network, and thus will not suffer from the symmetry mismatch problem. That said, we do agree with the reviewer that methods like the residual pathway would be an interesting future direction to explore fused equivariant and non-equivariant feature encoding. We will discuss this in the final version of our paper.\n\n> How important is FOV in terms of performance? It appears that there are no direct comparison between normal FOV and large FOV in the experiments, but only large FOV and normal FOV + external/voxel.\n\nWe performed a new experiment comparing the large FOV with the normal FOV. As shown in the table below, a normal FOV could work for simple tasks like Stack D1, however, when facing more challenging tasks with multiple objects in the scene like Stack Three D1, it suffers from incomplete information in the observation.\n|            | Stack D1  | Stack Three D1 |\n|------------|-----------|----------------|\n| Large FOV  | 98.0      | 72.0           |\n| Normal FOV | 90.0 (-8) | 30.0 (-42)     |\n\n> Related to (1), would it be possible to also incorporate equivariant CNNs in the normal FOV + external setting? Would it require significant changes to the architecture?\n\nIt is possible, and no changes to the architecture are required. Please notice that we did perform experiments using the proposed method with external cameras in Appendix E, where our method still outperforms the baselines with a significant margin. In fact, our implementation supports an arbitrary number of cameras, and our Table 4 and Table 2 were run using the same encoder class.\n\n> How sensitive is the method's performance to the discretization of SO(2) used in equivariant CNNs? (e.g., would  improve/degrade performance)\n\nReducing the discretization will degrade performance. We validated this with our Equi Enc variation, where we changed the group associated with the network from C8 (containing 8 discrete rotations ranging from 0 to 2pi) to C4 (containing 4 discrete rotations) or C2. Please see the results in the table below. \n|    | Stack Three D1 | Square D2 |\n|----|----------|----------------|\n| C8 | 75.3     | 32.0           |\n| C4 | 68.0     | 30.7           |\n| C2 | 63.3     | 20.7           |\n\nWe suspect that increasing the size of the sub-group from C8 to higher order might not future improve the performance. As experimented in the e2cnn paper, a cyclic group of order 8 is typically a sweet spot among the discrete subgroups of SO(2)."}]

Author rebuttal:

Meta-review: This paper explores a number of straightforward and practical approaches to incorporate symmetry benefits into diffusion policies without the overhead of full equivariant designs. This paper proves that combining eye-in-hand perception with relative or delta action parameterization yields inherent SE(3)-invariance, performs a systematic experimental study on those design choices for integrating symmetry in diffusion policies, and concludes that an invariant representation with equivariant feature extraction significantly improves the policy performance. 

Based on comments from reviewers, the merits of this paper include: (1) It is clearly written and easy to follow. (2) Provide a contribution in guiding how to incorporate symmetry in diffusion policies with just simple methods. (3) Offer a proof on how in-hand observations + relative actions leading to SE3 equivariance under assumptions. 

While there are some merits as above, some reviewers also have some concerns about the novelty and originality, and the performance of the method. For the performance issue, the concern by a reviewer is to utilizes only equivariant encoders + relative action representation, which are both existing methods, and the final performance is slightly lower than that of more sophisticated methods.

All reviewers recommend Weak Accept, which suggests that the work is valuable but may not meet the highest bar for novelty or performance. There are no recommendations for rejection. Given that the contribution is clean, well-supported, and of practical interest—even if somewhat incremental—I would lean slightly toward acceptance if room permits. However, I acknowledge that the final decision may depend on the overall selectivity and comparative strength of submissions, and I defer to the SAC or program chairs in borderline cases.
Decision: Accept (poster)