RAVEN: End-to-end Equivariant Robot Learning with RGB Cameras
Authors: David Klee
Boce Hu
Andrew Cole
Heng Tian
Dian Wang
Robert Platt
Robin Walters
Venue: ICLR 2026 Poster (2026)
Keywords: Robotic Manipulation
Policy Learning
Equivariance
Primary area: applications to robotics, autonomy, planning
TL;DR: We propose the first SE(3)-equivariant policy learning framework that operates with only RGB image observations.

Abstract: Recent work has shown that equivariant policy networks can achieve strong performance on robot manipulation tasks with limited human demonstrations.  However, existing equivariant methods typically require structured inputs, such as 3D point clouds or top-down camera views, which prevents their use in low-cost setups or dynamic environments.  In this work, we propose the first $\mathrm{SE}(3)$-equivariant policy learning framework that operates with only RGB image observations.  The key insight is to treat image-based data as collections of rays that, unlike 2D pixels, transform under 3D roto-translations. Extensive experiments in both simulation with diverse robot configurations and real-world settings demonstrate that our method consistently surpasses strong baselines in both performance and efficiency.

Published as a conference paper at ICLR 2026
RAVEN: END-TO-END EQUIVARIANT ROBOT LEARN- ING WITH RGB CAMERAS
David M. Klee* Boce Hu Hu Andrew Cole', Heng Tian', Dian Wang', Robert Platt Robin W
"Northeastern University, "Stanford University, Equal Contribution, 'Equal Advising (klee.d d, hu.boce, r.platt, r.walters/ @northeastern.edu
ABSTRACT
Recent work has shown that equivariant policy networks can achieve strong perfor- mance on robot manipulation tasks with limited human demonstrations. However, existing equivariant methods typically require structured inputs, such as 3D point clouds or top-down camera views, which prevents their use in low-cost setups or dynamic environments, In this work, we propose the first -3q)--equvari policy learning framework that operates with only RGB image observations. The key insight is to treat image-based data as collections of rays that, unlike 2D pixels, transform under 3D roto-translations Extensive experiments in both simulation with diverse robot configurations and real-world settings demonstrate that our method consistently surpasses strong baselines both performance and efficiency. Our project page is available at allhtps::// /ttp:///dmkkeeeeee gwthuh.tt..oooooo raven
INTRODUCTION
Learning effective visuomotor policies for robotic manipulation remains fundamental challenge, particularly in settings that demand strong generalization from limited data. Recent advances in iiffusion-based policy learning (Chi et al. 2023 Wang et al. 2024| (ee et al. (2024) and large-scale behavior cloning pipelines (Barreiros et al. 2025 Black et al., 2024). have demonstrated remarkable performance across diverse scenes and viewpoints. However, these methods typically require large amounts of training data to robust to sensor or scene variations, which limits their practicality in settings where data collection is expensive the environment is unstructured.
promising direction to improve generalization and sample efficiency is through equivariance, which explicitly encodes symmetries into networks (Cohen & Welling 2016). By ensuring that policies respond consistently under spatial transformations of the input, equivariant models generalize more effectively across variations in object pose and scene layout. However, existing equivariant methods impose strict requirements on input modalities, typically assuming access to structured geometric data such point clouds or sirde view RGB-D images with known alignment (Yang et al. 2024 Wang et al. 2024). Unfortunately, these assumptions are violated in many popular robot learning setups, where raw image observations from arbitrarily placed cameras (as in UMI (Chi et al. 2024).
ALOHA (Zhao et al. 2023), Mobile-ALOHA (Fu et al. 2024h) are the norm.
Our method, RAy-based equiVaria Variant ENcoder or RAVEN, is the first end-to-end SE(3) equivariant policy learning method that works with mmage-based observations. Our key insight is to express images terms of rays projected into 3D space using camera parameters. particular, each small patch of pixels in the image is encoded as feature vector with corresponding ray originating at the camera aperture and passing through the patch. Unlike raw pixels that exist only in the camera frame and therefore only support 2D transformations, rays are naturally defined in 3D and can be transformed by SE( rotations and translations. This gives us a representation of the input images that is compatible with -equiiaaua neural network layers and therefore enables to create an end-to-end (E)33equuvarria model using only RGB input while remaining agnostic to the number and placement of cameras in the robot learning setup.
RAVEN utilizes novel ray- based observation encoder that transforms multi-view images into SE(3)- equivariant geometric tokens. To decode these tokens into actions, we introduce lightweight and

---

Published as a conference paper at ICLR 2026
flexible policy head that supports both world-frame (absolute) and end-effector-frine (relative) action representations, allowing compatibility with a wide range of robotic systems and control interfaces. Unlike prior equivariant methods that often suffer from slow training due to heavy architectures, our method achieves SF((3) Diffusion Policy, but with significantly improved performance.
with a training speed that is even faster than a baseline
This paper makes the following contributions:
We introduce encoder that expresses an image as set of Tokens from different, arbitrarily placed cameras can be seamlessly combined to produce a single, consistent representation of the world.
geometric tokens.
Based on this encoder, we propose RAVEN, the first end-to-end 3E(3)-equivarian) robotic policy learning framework that can operate directly on RGB image inputs.
We characterize our method both in simulation and on physical robot. It outperforms the strongest baseline by 12% over 12 MimicGen tasks, 17% over 6 MexMinieGen tasks, and 35% over real-world tasks. Finally, while prior equivariant models are often slow to train, our method trains approximately 1.6x faster than the previous equivariant diffusion method.
2 RELATED WORK
Equivariant Networks for Image Processing Equivariant models for 2D image processing (Cohen
& Welling 2016 Dieleman et al. 2016 Worrall et al., 2017 Chidester et al., 2018: Weiler &
Cesal 2019 Maron al.) 2020, He et al.) 2021 Rahman & Yeh 2023 Li al. 2024] 2025)
achieve equivariance to planar transformation groups including SO 2)) (2), and more general affine groups, which improves generalization and sample efficiency in vision tasks involving spatial symmetries. Extending beyond such planar symmetry, recent efforts (Esteves et al. 2019 Park et al.,
2022 Klee et al., 2023alb by Howell et al. 2023 Lee & Cho 2024) seek to learn SO(3) or SE(-)- equivariant representations directly from 2D images for 3D vision tasks, such 6D pose estimation and object understanding. These methods attempt to bridge the gap between 2D observation and 3D geometric reasoning by encoding 3D symmetries into the learned features. More recently, ray-based formulations have emerged as promising direction for incorporating 3D geometry into image- based equivariant models. By interpreting each pixel a 3D ray originating from the camera, these approaches anchor features along these rays in 3D space while maintaining equivariance (Xu al.) 20233 Brehmer et al., 2024] Xu et al.) 2024). Our work builds on GTA (Miyato al. (023). which introduces an efficient attention mechanism with camera pose and pixel coordinate geometric representations. In contrast, we represent image patches as ray-based (F))3 representations, which better encode the spread of rays in the scene and allow processing with non-image data.
Equivariant Robotic Manipulation Symmetry has been widely explored in robotic manipulation
to improve generalization and sample efficiency (Huang et al. 2022 2024| Wang al. 2022
Nguyen et al.) 2023 Zhu et al. 2022; Hu et al. 2024a Jia et al. 2022 Ryu et al. 2022
2024
Gao et al. 2024 Mittal al. 2024 Eisner et al. 2024). Recent works have transitioned from
open-loop to closed-loop control by adopting generative policies such as diffusion models (Chi
al. 2023) and flow matching approaches (Hu et al. 2024b) Chisari et al.] 2024). Within this framework, incorporating symmetry further improves performance. Wang et al. (2024) introduces an -euuivequiv diffusion policy, but assumes fixed camera placement. EquiBot Yang et al.)
2024) achieves SIM(3) -3)-)vv)rvvaiinncce using object-centric point clouds, but relies on accurate object segmentation, which limits generalization diverse sensor configurations and open-world world setups. More recently, (uu et al. (2025) proposes an -0)3)-equivariant policy using only RGB inputs from a single wrist-mounted camera. Wang et al.] (2025) further highlights the benefits of pretrained vision encoders and relative action spaces in diffusion-based policies. In contrast, RAVEN is the first, as far as we know, SE(3) -eeee-eruviaan manipulation framework that operates directly on raw RGB images without relying on object segmentation assumptions about the number and placement of cameras. Additionally, our model achieves equivariance through canonicalization and is trained via flow-mat now-matching, so our model is less computationally heavy than existing equivariant methods.
2

---

Published as a conference paper at ICLR 2026
3 BACKGROUND
Equivariance and Canonicalization A function f is equivariant to a symmetry group G Pgf(x) f(PTT) for all g E G, E X where P and p' denote group actions on the input and output spaces, respectively. Equivariance is desirable in deep learning because it allows models to generalize across transformations of the data. Equivariant networks can be constructed through group convolution (Cohen & [016) [Esteves et al., 2019) or by imposing constraints on the weights of each layer, as shown by (Geiger & Smidi [002]] Cesa et al. 2022 Brehmer et al. 2024).
An alternative approach is to enforce equivariance with canonicalization: first, transform each input into a standardized, or canonical, reference frame that is consistent across samples, and then map the output back to the original frame. Canonicalization avoids the computational overhead of equivariant convolutions, but assumes access to a useful canonicalizer. In practice, such canonicalization functions can be defined analytically (e.g., aligning to principal axes), estimated (e.g., via PCA), learned from data (Esteves et al. 2017 Du et al. 2022 Kaba et al.) 2023). Formally, let G act on the input G be canonicalization function satisfying the
space X via p x X X, and let X right-compensation property:
-(0z))((())
(1)
We denote the canonicalized input by PC(x)æ. Using ① have: PC(p,z) = pe(x)a so ())( is G-invariant. To recover equivariance, we decanonicalize in the output space with action p²: F(x)= Pe(ay- (P(())z) which implies F(p,z) = Pg F(x).
Images and Rays An image typically represented a discretized 2D signal, that is, mapping from pixel coordinates to RGB values, I: Z + R.. Alternatively, an image can be thought of as collection of rays moving from the camera's optical center out into the world. This ray-based
representation better conveys 3D spatial information of the world. The ray associated with a 2D pixel coordinate determined by the camera intrinsics K E R333 and extrinsies M = [R [] E SF(3), where R E SO(3) and & E 3 denote the camera orientation and position in the world frame. Given pixel coordinate u E 2² expressed in homogeneous coordinates, we can compute the corresponding ray?
r = (t, RKR u)
where r (Fo, Ta) is tuple describing the origin To E R and direction rdd E Sa This follows from the standard camera projection equation, which maps a 3D point x E R to a pixel coordinate 22: K R R f|ær This analysis is based pinhole camera model; for other models, such as fisheye cameras, we refer the reader to Grossberg & Nayar (200T); Kannala & Brandi (2006).
Geometric Transform Attention Geometric Transform Attention (GTA) is recently proposed attention mechanism designed for spatial data (Miyato et al. 2023) to achieve equivariance. Specifi- cally, each token z; is associated with a group element 9, that encodes its geometric context (in the simplest case, an image, the context is the 2D pixel location; but the idea extends to SE(3) poses of cameras, etc.). Given some representation p G - (L)) that maps group elements to linear operators, the output of geometric transform attention between the i-th query, Q.. and the j-th key and value, Kj, Vg, is:
exp ((plos) Qi) (.) O = p(gi) Ej exp ((p(o.) (T(())) (V()))-\V,)
This construction ensures that attention scores are only affected by the relative transformation PT. Pas between tokens. Thus, if all tokens undergo the same global transformation (g: + hg; for some G), the internal similarity remains unchanged and the output transforms consistently by p(h). yielding equivariance. As noted by the authors, dot-product similarity violates equivariance when G SF(3), so they propose a variation on Equation that uses negative Fuclidean distance as a similarity metric to achieve exact SE(3) equivariance.
Policy Learning with Flow Matching Flow-based generative modeling provides an alternative to diffusion for capturing multimodal action distributions in policy learning. Unlike DDPM (Ho et al.
3

---

Published as a conference paper at ICLR 2026
SE(3)
Relative Action Decoder
Rest
Rlys
Feedforward Local Prames
Attention Giobal Fame
Extrecs
FeedForwrr in End Eftectrr Framre
Projection on
Croaa- Attention Clobal Frams
Causal Attention Giobel Frome
Greper Stane
Groper Pizs
Figure The RAVEN Architecture: image observations are converted geometric tokens by preprocessing with ResNet and assigning pose based rays. The SE(3) Geometric Transformer efficiently processes the tokens with feedforward and geometric transform attention layers. Finally, an action decoder network refines relative end-effector motions with flow matching.
2020), which predicts noise in discrete denoising process, flow matching (Lipman et al. 2022
al. 2022) directly learns a continuous-timg vector field that transports prior distribution the expert action distribution. Concretely, the policy is represented as an ordinary differential equation (ODE): /a(t)/dt (o, a(t), t) where a(0) NO I) and the terminal state a(1) follows the expert action distribution. During training, a is approximated by interpolating between a noise sample ag ~ Po and an expert action a ~ pt, and the model ny supervised with the flow matching objective:
CFM = Er.
()a0 + tai, t) (a - =
which trains by to approximate the transport field that maps the prior po to the expert distribution P1. Intuitively, (a t) tells each point how to move over time, and this direct supervision enables more efficient sampling and more stable training compared to diffusion-
policies. As shown in similar works diffusion policy (Wang et al. 2024 Yang al., 2024). flow matching be made SE(3) equivariant if the velocity field is SO(3) equivariant and the observations are centered Chisari al. 2024).
METHOD
Our method, RAVEN, is policy learning framework for image-based observations that equivariant to SE(3) transformations of the scene (e.g. robots, objects, and sensors). Unlike existing equiv- ariant methods. RAVEN does not require spatial data like point clouds and can be used with any number of stationary moving RGB cameras. In this section, we first describe novel equivariant encoder network that digests images and proprioceptive information to produce dense, equivariant representation of the scene. Second, we introduce an action decoder network that operates on this observation encoding to produce action trajectories. Finally, we analyze the equivariance properties of the method.
4.1 RAY-BASED EQUIVARIANT ENCODER
We propose an SE(3) equivariant encoder network that efficiently processes multi-image and pro- prioceptive observations. First, we map 2D image data 3D ray features, which better convey spatial information about the scene. Next, we express all input data in a unified representation, called geometric tokens, in which the features are canonicalized by their pose in the scene. Finally, we process these tokens with GTA-style transformer layers, composed of feedforward operations in local frames and attention operations in the global frame.
Ray-based SE(3) Features from Images It is difficult to integrate image inputs into 3D equivariant methods, since the 3D coto-translation action on the data is unclear. Inspired by recent works (Brehmer et al. 2024 Xu al. 2024), instead consider each pixel to represent 3D ray, according the known camera extrinsics and intrinsics. In this way, an SE(3) transformation acts on the extrinsics (camera pose), which transforms the rays even though the image data is unchanged.
4

---

Published as a conference paper at ICLR 2026
In practice, processing 3D data like rays is expensive, with a pretrained ResNet (He et al. 2016). This is useful for two reasons: (1) pretrained image models are fast and effective at extracting useful information, (2) the resulting feature map is downsampled, which reduces the burden of subsequent operations on 3D data. However, after downsampling, each grid cell in the feature map represents a local patch of pixels and corresponding local patch of rays. While single ray is symmetric about the camera's ' roll axis, a local patch of rays has a well-defined orientation. In this way, each grid cell in the feature map is associated with element of SE((), describing its origin and orientation (direction is z-axis, and image plane axes are y-axes). The following subsections explain how to use these features and their poses for fast and equivariant processing.
propose first pre-proeeee precceesssing the images
Geometric Tokens geometric token a piece of geometric information stored in a reference frame. Specifically, geometric token is a tuple, x = (Za, 9z), composed of feature vector, 2x Rd. and pose, 9x E SE(3). such that the feature vector is canonicalized by the pose, e.g., it conveys local information. The idea is similar to positional embeddings in ViT's, where each token has features describing a local patch which may be combined with the coordinate of the patch via positional embedding. The idea was described in its most general form by Miyato et al. (2023), where each token has features and group element. In contrast, our geometric tokens only contain SE(3) group elements, providing a unified representation across different input modalities.
Figure 2: pixel conveys a ray direction (left), an image patch conveys an orientation (right).
A geometric token's pose is used to convert the features into global frame: p(gx) The feature vector, Zn. a combination of scalar, 2 vector, 2n and point, = features so that:
p(gx) (p"(gz) p")))""'"(g')'>
(5)
where p"(gx) is the identity (scalar features are invariant geometric transforms), p" (gx) applies 3D rotation and (9x) applies 3D rotation and translation. We illustrate this idea in Figure 3 A helpful way to think of geometric tokens is through the lens of canonicalization. The features are canonicalized by default, and the pose is used to de-canonicalize them using a hardcoded SE(3) group action. This ability to process information in canonical decanonical frames is key to our method's ' SE(3) equivariance and computational efficiency.
A core assumption here is that all observations can be converted to geometric tokens. For image obser- vations, we discussed how we use a ResNet to gen- erate feature map where each grid cell is assigned a pose based on the camera projection. We apply 1xl convolution to the feature map to obtain the desired feature dimensionality (split among scalar, vector, and point features). For the proprioceptive information, e.g., end-effector pose and state (gripper opening or finger joint angles), we construct one ge- ometric token per robot. The features are generated with a linear mapping on the gripper state, and the pose is defined as the end-effector pose. We discuss tokenization schemes for other observation modali- ties in Appendix C
Zz = (2 2 24
p(gx)
9z
Figure 3: A geometric token encodes scalar, vector, and point features the sensor's local frame (left), which can be transformed to the global frame with the token's 's pose (right).
SE(3) Geometric Transformer Once all observations are converted into geometric tokens, we process them with several transformer blocks (Vaswani et al.] 2017) using geometric transform attention (GTA) (Miyato et al. 2023). The feedforward layers, a two-layer MLP, operate directly on the geometric tokens' features, leaving their poses unchanged. As stated above, the features are canonicalized (e.g., expressed in the local frame determined by the token's 's pose) so the feedforward layers do not break SE(3) equivariance. The attention layers in our transformer use GTA, which mod- ulates the similarity scores based on the relative transformation between tokens, using each token's SF(3) pose compute the relative transformation. We combination of dot product similarity
5

---

Published as a conference paper at ICLR 2026
(on the scalar and vector components) and negative Euclidean distance (on point components) to compute similarity. The dot product is, in our experience, more stable during training, but violates equivariance when used with point features. Since our model is used in the low data regime, use negative Euclidean distance as needed to ensure exact SF.(3 equivariance out-of-the-bon
4.2 GEOMETRIC TOKEN ACTION DECODER
The output of our encoder network is a set of geometric tokens that form 3D representation of the observed scene. To solve robotic manipulation tasks, feed this representation to a decoder network that is trained with flow-matching to generate trajectories of SE((3 gripper poses and states. We now describe novel SE(3) equivariant decoder network designed output gripper actions in the relative frame. (We also describe how to produce actions in the absolute frame in Appendix
With geometric tokens, the features remain canonicalized in some known reference frame throughout processing. This property can be used to our advantage to output end-effector trajectories arbitrary reference frames. To predict actions relative the end-effector frame, we create action geometric tokens where the pose is set to be the end-effector's 's pose at the current timestep. When these tokens are processed by a SE(3) geometric transformer, the action geometric features are naturally suited to express the robot trajectory in the global frame while still having the precision of being described in the local frame.
In practice, we train an SE(3) geometric transformer with a fow-mhachhigg loss to output the velocity field. The action geometric tokens for each timestep in the trajectory are initialized with the features sampled from Normal distribution, and the poses are all set to the current gripper pose. The model is trained using the loss from Equation aapplee to the predicted action token features only. We modify the SE(3) geometric transformer architecture from encoder to roughly match the transformer decoder from DiffPo. Each transformer block is composed of causal self-attention between action tokens, then cross-attention with the geometric tokens from the encoder, and finally feedforward layer. We use four transformer blocks in total, and a final projection layer to convert he action geometric tokens into velocities. A diagram of the relative action decoder shown the right of Figure I
4.3 EQUIVARIANCE PROPERTIES
Our method is end-to-end equivariant with respect to global SE(3) transformations of the robots, sensors, and objects. To simplify the analysis, let consider a setup with two cameras and one gripper. The observation is composed of images, J,, J2, camera intrinsics, K1, K2, camera extrinsics, M1, M2. gripper pose, P, and gripper state, X. The action of global transformation, g SE(3), on the inputs is: 11, I2, K1, K2, M2, P, X + I1, 12, K1, K2, gM,,, g.M2, A. In other words, the transformation only acts on the camera poses, M M2. and gripper pose, P. Our encoder network, call it v/ outputs geometric tokens: Z, G = (T, K, M, P, A), so the effect of g E SE(3) is:
(6) meaning the output is transformed by an SE(3) action on the geometric tokens' poses, with the tokens' features remaining unchanged. It is important note that the equivariance does not capture independent camera transformations (we test generalization to this empirically in Section 5.1)
Z.gG (1, 12, K1, K2. gM,, MMM
The equivariance of method is slightly different from prior equivariant robot learning methods that operate on structure or spatial inputs (Wang et al.) 2022 Yang et al. 2024) Those methods claim equivariance with respect to transformations of the robots and objects independent of the sensors, which results in stronger generalization. Equivariant methods that operate on RGB images, like RAVEN, cannot achieve this equivariance outside of few edge cases (see longer discussion in Appendix A.
EXPERIMENTS
We systematically evaluate RAVEN in both simulation and real-world settings. Our experiments are designed to address the following research questions: (1) Does our method outperform existing approaches general manipulation tasks? (2) Can it maintain strong performance under different

---

Published as a conference paper at ICLR 2026
(a) Representative tasks from MimicGen (left three images) and DexMimicGen (right three images).
(h) Viewpoint generalization evaluation.
Figure 4: Visualization of tasks from the MimicGen and DexMimieGen benchmarks and the viewpoint gencralization setting. Subfigure (a) shows representative tasks, while (b) illustrates viewpoint variation: in each task, the leftmost image the original view and the two on the right are sampled perturbed views. The complete set of tasks is provided in Appendix D
camera numbers and bimanual configurations? (3) Does it generalize to novel viewpoints at inference time? (4) Is its training speed faster than previous equivariant-based methods? (5) it applicable to real-world robotic manipulation tasks?
5.1 SIMULATION
Experiment Setting To address the first four questions, conduct simulation experiments on the MimicGen Mandleearr et al. (023) and DexMimieGen (Jiang et al. 2024) benchmarks. First, we evaluate RAVEN on twelve single-arm arm manipulation tasks from MimicGen, widely used suite for closed-loop policy learning (Wang et al., 2024 Hu et al.) 2025). using agent-view and eye-in-hand RGB observations. Each model is trained with 100 demos. Second, study multi-camera and bimanual settings, we evaluate RAVEN on six bimanual tasks from DexMimicGen, consisting of three gripper-based and three dexterous-hand tasks. This setting uses two eye-in-hand cameras and one agent-view, and involves controlling two arms with moving cameras, making the tasks more challenging. Models are trained with both 50 and 100 demos. Both MimicGen and DexMimicGen tasks are evaluated with 50 rollouts per task and 3 random seeds. Figure (a) illustrates representative tasks from the two benchmarks. Third, to study viewpoint generalization, we modify the MimicGen tasks by randomly perturbing the agent-view camera at the start of each episode (and demonstration), as shown in Figure 4 (b). The camera sampled within + pitch and +40" yaw relative to the original viewpoint, ensuring unique but distributionally consistent perspectives across demos and rollouts. Fourth, we compare the training speed of RAVEN with basclines to demonstrate its efficiency. Dataset configurations, visualizations of all subtasks, and training details are provided in Appendices D and
Baselines We compare against four strong baselines: (1) DiffPo (Chi et al. 2023): A diffusion- based policy without equivariance, trained from scratch, serving as the primary reference. (2) DiffPo (Pre): The same as (I) but with pretrained ResNet encoder to match the visual backbone used in RAVEN. This ensures fair comparison when both methods leverage pretrained perception. (3) EquiDifIPo (Wang et al. y024): An (2)))--quivarian) image-basee based diffusion model that incorporates both equivariant image encoder and equivariant temporal U-Net. (4) ACT (Zhao et a 2023): transformer-baat based behavior cloning method designed for closed-loop control. Additionally. we conduct experiments on each task to compare relative and absolute action prediction with RAVEN.
Due to space constraints, the full results are provided in Appendix
MimieGen Table reports the success rates across all methods and configurations. Compared to the strongest baseline result for each task, RAVEN achieves the highest performance 11 out of 12 tasks, with the remaining task only 4% behind. These results demonstrate the robustness of approach under different action representations. RAVEN exceeds the best-performing baseline
7

---

Published as a conference paper at ICLR 2026
Table I: MimicGen tasks with agent-view and eye-in-hand cameras using 100 demos. Results are
reported as the best success rate (%) across 30 evaluation checkpoints during training, each based on the same set of 50 rollouts. The final score averaged over the best checkpoint from each of 3 random seeds (some baselines from Wang et al. (2024)). Numbers in parentheses indicate the difference between RAVEN and the best baseline (improvements shown blue, drops in red).
Meihou
Averuge
Stack DI
Stack Three DI
Square D2
Thrending D2
Coffee D2
3P Assembly D2
RAVEN
66 (+12)
(43)
81 (+26)
50 (+25)
(+6)
(+1)
27 (+12)
EquiDidIPo
54
93
55
25
22
60
15
DifiPo (Pre)
97
53
21
53
14
DillPo
76
38
17
44
ACT
21
35
6
10
19
H. Cleanup DI
M. Cleanup DI
Kitchen DI
N. Assembly DD
Pick Place DO
Coffee Prep. DI
RAVEN
73 (-8)
53 (44)
(55)
74 (-)
73 (+27)
73 (-4)
EquiDiffPo
49
67
74
42
77
DiflPo (Pre)
54
47
80
71
46
65
DifIPo
52
43
67
55
35
65
ACT
Table 2: Maximum success rate (%) DexMinicGen tasks with 50 demos. Results reported using the same evaluation protocol as in Table [
Method
Average
Threading
3P Assembly
Transport
B. Cleanup
D. Cleamup
Tray Lift
RAVEN
82 (+17)
63 (+34)
69 (+19)
(-5)
(+13)
97(+1)
(434)
EquiDitPPo
62
29
45
85
94
56
DiliPo (Pre)
85
96
DidPo
20
84
91
by 12% on average. While both RAVEN and DifiPo (Pre) pretrained image encoder, RAVEN consistently outperforms it by 14%, indicating that our architectural design provides additional advantages beyond pretraining. Interestingly, DiffPo (Pre) performs on par with, or slightly worse than, EquiDifíPo on individual tasks, with an average gap of 2%, suggesting the positive effect of pretraining on model robustness. Nevertheless, all baselines still fall short by a substantial margin compared to RAVEN. Complete results, including standard deviations across three random seeds, are provided in Appendix
DexMimicGen Table reports results on DexMiniGGen with 50 demos. Complete results with 100 demos and variance across seeds are provided in Appendix H. At the time of writing, only tasks with the DO initial state distribution are publicly available, so there is minimal variation in object placement. Since generalization to spatial transformations is less useful in this benchmark, we expect the performance boost of equivariance to be less significant. Despite this, RAVEN consistently outperforms all baselines by substantial margin. Notably, as shown in Table H. RAVEN trained with only 50 demos performs comparably to or even outperforms basclines trained with 100 demos, highlighting the superior data efficiency of our method. We believe our performance in this setting is driven by the GTA layers in the encoder, which allow information from all three cameras to be fused into a better understanding of the scene. In comparison, DifiPo and EquiDifiPo simply concatenate feature vectors from each image without reasoning over their geometric relationships, limiting the integration of 3D information.
Viewpoint Generalization A key advantage of our observation encoder is its ability to form 3D representations, rather than relying solely on 2D image features. This inductive bias makes our method naturally more robust to variations in camera viewpoints compared to purely image-based approaches. This property is particularly useful in real-world deployment, where demos are often collected across a fleet of robots with varied sensors (Zhao et al.) 2024). The results (Table show that RAVEN is more robust to viewpoint perturbations than the baselines. Averaged across four tasks, RAVEN achieves 23% improvements over the strongest baseline. As expected, overall performance drops compared to Table due to distribution shifts from unseen camera poses. It is worth noting that our method is equivariant to transformations of the entire scene, including robots, objects, and cameras, but this task perturbs only single camera, which breaks global equivariance. Nonetheless,
8

---

Published as a conference paper at ICLR 2026
Table MimicGen experiments with randomly perturbed agent-view cameras. subset of tasks is selected for evaluation. The same experimental protocol as in Table is followed.
Average
Stack
Stack Three
Coffee
Hammer Cleanup
RAVEN
(+23)
99 (+21)
(+26)
53 (+2)
65 (+13)
EquiDiffPo
48
65
25
51
52
DiffPo (Pre)
44
78
42
21
36
Table 4: Ablation Study of RAVEN MimicGen tasks, success rate (%) averaged over three seeds.
Average
Stack
Stack Three
Coffee
Hammer Cleanup
RAVEN w/o SE(3) ray encoding
76
100
81
50
73
72
100
74
44
69
w/o Equi. Decoder
72
99
86
38
65
w/o Equi. Encoder Decoder
58
90
56
27
our geometry-aware feature representations enable significantly better generalization novel views than all baselines.
We also performed experiment to test RAVEN'S generalization noisy camera calibration (see Appendix B.I]. We find that RAVEN is sensitive to noise in the camera calibration, especially when introduced test time. However, by adding data augmentation to the camera parameters during testing, RAVEN is robust to calibration errors, dropping only 1% in success rate averaged across subset of MimicGen tasks.
Ablation Study We perform ablation study determine the impact of RAVEN'S equivariant layers and geometric structure have on performance. We consider three variations of our method. First, we remove the SE(3) ray encoding scheme and instead encode ray patches elements of ray-space, RR x 5 (e.g. each ray patch has a direction vector instead of an orientation). Second, remove the equivariant layers in the decoder network by replacing all GTA layers with standard dot- dot-product attention. We still provide the network with the pose of all tokens via an absolute position embedding. Third, we remove the equivariant layers in the decoder and encoder in the same manner.
For each variation, we evaluate its performance on a subset of MimicGen tasks and report success rates in Table 4 Without SE(3) ray encoding, the performance drops by 4% on average. While this drop is small, it highlights the benefits of the additional geometric structure provided by the ray patches orientation. The method's performance drops an average 4% with a non-equivariant decoder and 18% with a non-equivariant encoder and decoder. This supports the importance of equivariant layers in our method while keeping the added geometric structure of encoding images as rays. These tasks are all table-top tasks, with minimal out-of-plant motions and we expect the impact of SE(3) equivariant layers to be greater in less structured manipulation settings.
Policy Efficiency Analysis Training and deploying policies efficiently is critical in robotics appli- cations. In this context, training time is an important evaluation metric. Prior equivariant methods often suffer from longer training durations (Wang et al., 2024: Hu et al.) (025).. To evaluate training efficiency, measure the training time the MimicGen Threading D2 task under identical hard- ware and configurations. DifiPo completes the pipeline (training plus policy rollout simulation) in 3.3 hours, while EquiDifiPo requires 7.3 hours. In contrast, RAVEN finishes in just 2.8 hours. This corresponds to a 1.6x speedup over EquiDiffPo with a superior performance, highlighting our method's advantages in both effectiveness and training efficiency.
5.2 REAL WORLD
Physical Setup Our real-world experiments are conducted using Universal Robot UR5 equipped with a Robotiq-88 85 Gripper and custom- designed soft fingers. We use one mounted RealSense D455i for aient-view observations and a wrint-mounted GoPro for eye-in-hand views. Demos collected via the Gello teleoperation interface (Wu et al., 2024), with RGB observations and actions recorded 5 Hz. Figure 5 shows the four manipulation tasks. Banana Picking, Beans Scooping, and

---

Published as a conference paper at ICLR 2026
(a) Banana Picking
(b) Beans Scooping
(c) Coffee Cleanup
(d) Box Cleanup
Figure 5: Visualization of four real world tasks shown at the beginning (left) and end (right) of the episode. The randommess task initial states is illustrated Fig.
Table 5: Real-world task performance over 20 trials, reported average progress (%) and, in parentheses, success rate (%).
Banana Picking
Beans Scooping
Coffee Cleanup
Box Cleanup
# Demos
75
75
75
40
RAVEN
100 (100)
70 (45)
75 (45)
77 (60)
DifiPo (Pre)
53 (50)
30 (0)
56 (15)
43 (30)
Coffee Cleanup require precise 3D control to meet complex goal conditions, presenting challenges in spatial reasoning. The final task, Box Cleanup, uses only half the demos compared to other tasks. It emphasizes learning under limited data and involves transparent objects that require policies to adapt to low-data regimes while interpreting challenging visual cues from RGB observations. We compare RAVEN against the pretrained Diffusion Policy (Chi et al.] (023).. Further details, including hardware setup, task specifications, and success rate definition, provided in Appendix [
Results We report the results of four real-world tasks in Table 5 Numbers represent the average completion progress of each task, with overall task success rate shown in parentheses. Progress evaluation offers a more comprehensive measure of policy performance, particularly for tasks where baselines fail entirely (e.g., 0% in Beans Scooping). Across all tasks, our method consistently outperforms the pretrained DiffPo. Specifically, RAENN achieves 81% in average progress and 63% % overall completion success rate. By contrast, DifiPo (Pre) reaches only 46% and 24%. These results highlight the strong effectiivnees of our method, particularly its data efficiency, and demonstrate its practicality for real-world deployment. Progress definitions and full results with our additional absolute action variant provided in Appendix
CONCLUSION
In the space of Imitation Learning methods for robotic manipulation, the RAVEN framework is an exciting new piece technology that combines the advantages standard imitation learning methods (like baseline Diffusion Policy) with the advantages of an equivariant model. Unlike previous equivariant models like EquiDifiPo (Wang et al. 2024) or EquiBot (Yang et al., 2024), which require depth or point cloud data in order to be SE(2) - or SE(3) (e(()))-qivarian, RAVEN an end-to-end SEE3) (3q))--qquvariant method that can function with RGB input alone. Moreover, RAVEN can benefit from standard pretrained image encoders, just like non-equivariant models do. Most importantly, compared to a baseline pretrained Diffusion Policy, RAVEN outperforms significantly and has faster training times.
Limitations RAVEN has some limitations. Although RAVEN policy can generalize well to novel camera viewpoints, requires all cameras to be well calibrated with respect to world coordinate frame. Also, have far only explored this idea in the context of RGB inputs. In principle, the RAVEN encoder compatible with a variety of input modalities, including depth and point cloud data, and integrating with this type of information is something would like explore in the future.
10

---

Published as a conference paper at ICLR 2026
ACKNOWLEDGEMENTS
We would like to thank Haotian Liu for his assistance with the real-world experiments as well as all members of the Geometric Learning Lab and Helping Hands Lab for their valuable feedback on the manuscript. This work was supported in part by NSF Grants 2107256, 2134178, 2314182, 2409351, 2442658 and NASA Grant 80NSSC19K 1474. This work used GPU resources at NCSA Delta through allocation CIS250208 from the Advanced Cyberinfrastructure Coordination Ecosystem: Services Support (ACCESS) program (Boerner et al. 2023). which supported by U.S. National Science Foundation grants #2138259, #2138286, #2138307, #2137603, and #2138296.
REPRODUCIBILITY STATEMENT
Comprehensive experimental setups and training details are provided in the Appendix, Upon accep- tance, we will release our implementation publicly via GitHub repository.
ETHICS STATEMENT
This research makes use of both publicly available datasets, released under appropriate licensing agreements, and additional data collected by the authors. All newly collected data involve only robotic systems and physical objects, without any human participants or personally identifiable information. No sensitive or harmful content is present. Data collection and usage were conducted solely for scholarly research purposes and accordance with established ethical standards. Our work conforms, in every respect, with the ICLR Code of Ethics.
THE USE OF LARGE LANGUAGE MODELS (LLMs)
LLMs used exclusively for improving language quality, such grammar correction and refinement. The core method development in this research does not involve LLMs as any important, original, non-standard components.

---

flow model for Published as a conference paper at ICLR 2026
REFERENCES
Jose Barreiros, Andrew Beaulieu, Aditya Bhat, Rick Cory, Eric Cousineau, Hongkai Dai, Ching- Hsin Fang, Kunimatsu Hashimoto, Muhammad Zubair Irshad, Masha Itkina, et al. careful examination of large behavior models for multitask dexterous manipulation. arXiv preprint arXiv:2507.05331, 2025.
Kevin Black, Noah Brown, Danny Driess, Adnan Esmail, Michael Equi, Chelsea Finn, Niccolo Fusai, Lachy Groom, Karol Hausman, Brian Ichter, et al. No: A vision-language general robot control. arXiv preprint arXiv:2410.24164. 2024.
Timothy Boerner, Stephen Deems, Thomas R Furlani, Shelley Knuth, and John Towns. Access: Advancing innovation: Nsf's s advanced cyberinfrastructure coordination ecosystem: Services & support. In Practice and experience in advanced research computing 2023: Computing for the common good, pp. 173-176. 2023.
Johann Brehmer, Pim De Haan, Sönke Behrends, and Taco S Cohen. Geometric algebra transformer. Advances in Neural Information Processing Systems, 36, 2024.
Gabriele Cesa, Leon Lang, and Maurice Weiler. A program to build e (n)-equivariant steerable cnns. In International conference on learning representations, 2022.
Cheng Chi, Zhenjia Xu, Siyuan Feng, Eric Cousineau, Yilun Du, Benjamin Burchfiel, Russ Tedrake, and Shuran Song. Diffusion policy: Visuomotor policy learning via action diffusion. The International Journal of Robotics Research, pp. 02783649241273668. 2023.
Cheng Chi, Zhenjia Xu, Chuer Pan, Eric Cousineau, Benjamin Burchfiel, Siyuan Feng, Russ Tedrake, and Shuran Song. Universal manipulation interface: In-the-wild robot teaching without in-the-wild robots. arXiv preprint 200. 2024.
Benjamin Chidester, Minh N Do, and Jian Ma. Rotation equivariance and invariance in convolutional neural networks. arXiv preprint 180i 12301, 2018
Eugenio Chisari, Nick Heppert, Max Argus, Tim Welschehold, Thomas Brox, and Abhinav Val- ada. Learning robotic manipulation policies from point clouds with conditional flow matching. Conference Robot Learning (CoRL), 2024.
Taco Cohen and Max Welling. Group equivariant convolutional networks. In International conference machine learning, pp. 2990-2999. PMLR, 2016.
Congyue Deng, Or Litany, Yueqi Duan, Adrien Poulenard, Andrea Tagliasacchi, and Leonidas J Guibas. Vector neurons: A general framework for so (3)-equivariant networks. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 12200-12209, 2021.
Sander Dieleman, Jeffrey De Fauw, and Koray Kavukcuoglu. Exploiting cyclic symmetry in convolu- tional neural networks. In International conference on machine learning, pp. 1889-1898. PMLR, 2016.
Weitao Du, He Zhang, Yuanqi Du, Qi Meng, Wei Chen, Nanning Zheng, Bin Shao, and Tie-Yan Liu. Se (3) equivariant graph neural networks with complete local frames. In International Conference on Machine Learning, pp. 5583-5608. PMLR, 2022.
Ben Eisner, Yang, Todor Davchev, Mel Vecerik, Jonathan Scholz, and David Held. Deep se (3)evviiat geometric reasoning for precise placement tasks. arXiv preprint 1rXiv:2404.13478, 2024.
Carlos Esteves, Christine Allen-Blanchete Xiaowei Zhou, and Kostas Daniilidis, Polar transformer networks. arXiv preprint arXiv:1709.0188) 2017.
Carlos Esteves, Yinshuang Xu, Christine Allnn-Blanchette and Kostas Danilldiis. Equivariant multi- view networks. Proceedings of the IEEE/CVF international conference on computer vision, pp. 1568-1577, 2019.
12

---

Published as a conference paper at ICLR 2026
Zipeng Fu. Tony Z Zhao, and Chelsea Finn. Mobile aloha: Learning bimanual mobile manipulation with low-cost whole-body teleoperation. arXiv preprint arXiv:2401.02111, 2024.
Chongkai Gao, Zhengrong Xue, Shuying Deng, Tianhai Liang, Siqi Yang. Lin Shao, and Huazhe Xu. Riemann: Near real-time se (3)-equivariant robot manipulation without point cloud segmentation. arXiv preprini arXiv:2403. 19460, 2024.
Mario Geiger and Tess Smidt. e3nn: Euclidean neural networks. arXiv preprint 2022.
Michael Grossberg and Shree K Nayar. A general imaging model and a method for finding its parameters. In Proceedings Eighth IEEE International Conference Computer Vision. ICCV 2001, volume 2, pp. 108-115. IEEE, 2001.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 770-778, 2016.
Lingshen He, Yuxuan Chen, Yiming Dong, Yisen Wang, Zhouchen Lin, et al. Efficient equivariant network. Advances Neural Information Processing Systems, 14:5290-5302, 2021.
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33:6840-6851, 2020.
Owen Howell, David Klee, Ondrej Biza, Linfeng Zhao, and Robin Walters. Equivariant single view pose prediction via induced and restriction representations. Advances in Neural Information Processing Systems, (6:47221--4723,, 2023.
Boce Hu, Xupeng Zhu, Dian Wang. Zihao Dong, Haojie Huang, Chenghao Wang, Robin Walters, and Robert Platt. Orbitgrasp: Se(3)-equivarian grasp learning. arXiv preprint arXiv:2407.03531 2024a.
Boce Hu, Dian Wang, David Klee, Heng Tian, Xupeng Zhu, Haojie Huang, Robert Platt, and Robin Walters. 3d equivariant visuomotor policy learning via spherical projection. arXiv preprint 1XX::25005.1669, 2025.
Xixi Hu, Qiang Liu, Xingchao Liu, and Bo Liu. Adaflow: Imitation learning with variance-adaptive flow-based policies. Advances in Neural Information Processing Systems, 37:138836-138855 2024b.
Haojie Huang, Dian Wang, Robin Walters, and Robert Platt. Equivariant transporter network. arXiv preprint arXiv:2202.09400 2022.
Haojie Huang, Owen Howell, Dian Wang, Xupeng Zhu, Robin Walters, and Robert Platt. Fourier transporter: Bi-equivariant robotic manipulation in 3d. arXiv preprint arXiv:2401.12046 2024.
Mingxi Jia, Dian Wang, Guanang Su, David Klee, Xupeng Zhu, Robin Walters, and Robert Platt. Seil: Simulation-augmenni equivariant imitation learning. arXiv preprint arXiv:2211.00194 2022.
Zhenyu Jiang. Yuqi Xic, Kevin Lin, Zhenjia Xu, Weikang Wan, Ajay Mandlckar, Linxi Fan, and Yuke Zhu. Dexmimicgen: Automated data generation for bimanual dexterous manipulation via imitation learning. arXiv preprint arXiv:2410.24158 2024.
Oakoar Kaba, Arnb Kumar Mondal, Yan Zhang, Yoshua Bengio, and Siamak Ravanbakhsh. Equivariance with learned canonicalization functions. In International Conference on Machine Learning, pp. 15546-15566. PMLR, 2023.
Juho Kannala and Sami S Brandt. A generic camera model and calibration method for conventional, wide-angle, and fish-eye lenses. IEFE transactions on pattern analysis and machine intelligence, 18(8):1335-1340, 2006.
David Klee, Ondrej Biza, Robert Platt, and Robin Walters. Image to icosahedral projection for (3) object reasoning from single-view images. In NeurIPS Workshop on Symmetry and Geometry in Neural Representations, pp. 64 80. PMLR, 2023a.
13

---

Published as a conference paper at ICLR 2026
David M Klee, Ondrej Biza, Robert Platt, and Robin Walters. Image to sphere: Learning equivariant features for efficient pose prediction. arXiv preprint arXiv:2302.13996, 2023b.
Jongmin Lee and Minsu Cho. 3d equivariant pose regression via direct wigner-d d harmonics prediction. Advances in Neural Information Processing Systems. 37:127765-12779, 2024.
Yikang Li, Yeqing Qiu, Yuxuan Chen, Lingshen He, and Zhouchen Lin. Affine equivariant networks based on differential invariants. In Proceedings of the IEEE/CVE conference on computer vision and pattern recognition, pp. 5546-5556, 2024.
Yikang Li, Yeqing Qiu, Yuxuan Chen, and Zhouchen Lin. Affine steerable equivariant layer for canonicalization of neural networks. In The Thirleenth International Conference on Learning Representations, 2025.
Yaron Lipman, Ricky TQ Chen, Heli Ben-Hann Maximilian Nickel, and Matt Le. Flow matching for generative modeling. arXiv preprint arXiv:2210.02477, 2022.
Xingchao Liu, Chengyue Gong, and Qiang Liu. Flow straight and fast: Learning generate and transfer data with rectified flow. 2022.
Iyy Loshchilov and Frank Hutter. Decoupled weight decay regularization. arXiv preprint arXiv: (711::/711000000 2017.
Ajay Mandlekar, Soroush Nasiriany, Bowen Wen, Iretiayo Akinola, Yashraj Narang, Linxi Fan, Yuke Zhu, and Dieter Fox. Mimicgen: A data generation system for scalable robot learning using human demonstrations. arXiv preprint arXiv:2310.17500 2023.
Haggai Maron, Or Litany, Gal Chechik, and Ethan Fetaya. On learning sets of symmetric elements. In International conference on machine learning, pp. 6734-6744. PMLR, 2020.
Mayank Mittal, Nikita Rudin, Victor Klemm, Arthur Allshire, and Marco Hutter. Symmetry consid- crations for learning task symmetric robot policies. 2024 IEEE International Conference on Robotics and Automation (ICRA), pp. 7433-7439 IEEE, 2024.
Takeru Miyato, Bernhard Jaeger, Max Welling, and Andreas Geiger. Gta: A geometry-aware attention mechanism for multi-view transformers. arXiv preprint arXiv:2310. 10375, 2023.
Hai Huu Nguyen, Andrea Baisero, David Klee, Dian Wang. Robert Platt, and Christopher Amato. Equivariant reinforcement learning under partial observability. In Conference on Robot Learning, pp. 3309-3320. PMLR, 2023.
Jung Yeon Park, Ondrej Biza, Linfeng Zhao, Jan Willem van de Meent, and Robin Walters. Learning symmetric embeddings for equivariant world models. arXiv preprint arXiv:2204.11377 2022.
Md Ashiqur Rahman and Raymond A Yeh. Truly scale-equivariant deep nets with fourier layers. Advances in Neural Information Processing Systems, 36:6092-6111 -1044, 2023.
Hyunwoo Ryu, Hong-in Lee, Jeong- Hoon Lee, and Jongeun Choi. Equivariant descriptor fields: Se ensee
(3)-equivariant energy- based models for end-to-end visual robotic manipulation learning, arXiv preprint arXiv:2206 08321, 2022.
Hyunwoo Ryu, Jiwoo Kim, Hyunseok An, Junwoo Chang, Joohwan Seo, Taehan Kim, Yubin Kim, Chaewon Hwang, Jongeun Choi, and Roberto Horowitz. Diffusion-eddss: Bi-equivariant denoising generative modeling Conference on Computer Vision and Pattern Recognition, pp. 18007-18018, 2024.
(3) for visual robotic manipulation. In Proceedings of the IEEE/CVF
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones. Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention all you need. Advances in neural information processing systems, 30, 2017.
Dian Wang, Robin Walters, and Robert Platt. So(2)-equivarian reinforcement learning. arXiv preprint arXiv:2203.0442 2022.
14

---

Published as a conference paper at ICLR 2026
Dian Wang, Stephen Hart, David Surovik, Tarik Kelestemur, Haojie Huang, Haibo Zhao, Mark Yeatman, Jiuguang Wang, Robin Walters, and Robert Platt. Equivariant diffusion policy. arXiv preprint arXiv:2407.018122 2024.
Dian Wang, Boce Hu, Shuran Song, Robin Walters, and Robert Platt. A practical guide for incorpo- rating symmetry in diffusion policy. arXiv preprint arXiv:2505. 13431, 2025.
Maurice Weiler and Gabriele Cesa. General e (2)-equivariant steerable cnns. Advances in neural information processing systems, 32, 2019.
Daniel E Worrall, Stephan J Garbin, Daniyar Turmukhambetov, and Gabriel J Brostow. Harmonic networks: Deep translation and rotation equivariance. In Proceedings of the IEEE conference computer vision and pattern recognition, pp. 5028-50000 50,75077,2000 2017.
Philipp Wu, Yide Shentu, Zhongke Yi, Xingyu Lin, and Pieter Abbeel. Gello: A general, low-cost, and intuitive teleoperation framework for robot manipulators. In 2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 22156-12163, IEEE, 2024.
Yinshuang Xu, Jiahui Lei, and Kostas Danilidis. Equivariant light field convolution and transformer in ray space. Proceedings of the 37th International Conference on Neural Information Processing Systems, pp. 2463-2510, 2023.
Yinshuang Xu, Dian Chen, Katherine Liu, Sergey Zakharov, Rares Ambrus, Kostas Danillidis, and Vitor Guizilini. se(3) equivariant ray embeddings for implicit multi-view depth estimation. arXiv preprint ar 241 1. 1.07326 2024.
Jingyun Yang, Zi-ang Cao, Congyue Deng. Rika Antonova. Shuran Song, and Jeannette Bohg. Equibot: Sim equiivarrt diffusion policy for generalizable and data efficient learning, arXiv preprint arXiv:24077.0147 2024.
Yanjie Ze, Gu Zhang, Kangning Zhang, Chenyuan Hu, Muhan Wang, and Huazhe Xu. diffusion policy: Generalizable visuomotor policy learning via simple 3d representations. arXiv preprint arXiv:2403.03954, 2024.
Tony Z Zhao, Vikash Kumar, Sergey Levine, and Chelsea Finn. Learning fine-grained bimanual manipulation with low-cost hardware. arXiv preprint xxXi:23304.13005 2023.
Tony Z. Zhao, Jonathan Tompson, Danny Driess, Pete Florence, Kamyar Ghasemipour, Chelsea Finn, and Ayzaan Wahid. Aloha unleashed: A simple recipe for robot dexterity. arXiv preprint arXiv:2410.13126, 2024.
Xupeng Zhu, Dian Wang, Ondrej Biza, Guanang Su, Robin Walters, and Robert Platt. Sample efficient grasp learning using equivariant models. arXiv preprint arXiv: 09468, 2022.
15

---

Published as a conference paper at ICLR 2026
A EXTENDED EQUIVARIANCE DISCUSSION
In this section, we continue the discussion from Section about how the equivariance of RAVEN dif- fers from the traditionally accepted definition in equivariant robot learning. To restate the distinction, our method's s equivariance is with respect to joint transformations of the robots, objects and sensors whereas traditionally it is with respect to joint transformations of the robots and objects (not the sensors). We will refer to these two definitions of equivariance as rubot-object-sensso equivariance and robot-objec equivariance, respectively. It is clear to see that the rubot-object-sensse equivariance of our method is less powerful than robot-object equivariance, since there is only generalization if the sensors move with the robots and objects. The reason our method cannot achieve robot-object equivariance comes down to the input data. With RGB images, there is not obvious mapping on the input space that captures a transformation of scene independent of the camera. (In fact, there is an entire field of novel view synthesis that trains large networks to find such mapping). In contrast, for spatial inputs like point clouds, we can apply 4 x 4 transformation matrix to all points to approximate how the transformed scene would appear independent of camera pose. practice, the gap between our method's 's equivariance and robot- object equivariance is not always obvious or even present, as we show in the special cases below.
S2()) Equivariance with a Single RGB Camera: With single camera, 2D roto-translations of the image correspond to 2D roto- coto-translations of the scene (i.e. robots and objects) about the cameras roll axis, independent of the camera pose. This is the setting explored by (Wang et al., 2022; 2024).
RAVEN can match the robot-object equivariance of these methods here, so long as an 2D equivariant convolutional encoder network is used.
Discretized SO( Equivariance with a Structured RGB Camera Layout If cameras are ar- ranged uniformly around the scene's z-axis, then the group action takes the form of permuting the images between cameras. For instance, if four cameras are arranged 90 degrees apart, then cycling the images between cameras corresponds to rotations of the scene (robots and objects) by 90-degree increments. In this setting, RAVEN would be robot-object equivariant to discrete subgroup of SO(2) (or subgroups of SO(3) using camera layout shown in (Esteves al., 2019)
SE(3) Equivariance with Partial Point Clouds When only a few depth sensors are used, the point cloud will often have some gaps due to occlusion. To be precise, applying 3D roto-translations to the points corresponds to joint transformations of the robots, objects and sensors (since the occluded this case, methods can only achieve robot-object-senson equivariance.
regions also transform). This happens to be the case in several equivariant point cloud works Yang et al. (2024); Ryu et al.) (2022) 2024), although they do not discuss the nuance of their equivariance claims. So unless many depth sensors are used, our method will have the same equivariance guarantees. In practice, point cloud networks seem to generalize well on partial point clouds and this distinction is worth exploring more in the future.
B ADDITIONAL EXPERIMENTS
B.1 ROBUSTNESS TO CAMERA CALIBRATION ERROR
key assumption of RAVEN is that the camera intrinsic and extrinsics are known during training and deployment. Slight errors in camera parameters are typical in real robotic systems, due to miscalibration or sensors being bumped. Since RAVEN uses the camera parameters to map features on 2D image plane to SE( ray features, it is possible that these errors would harm RAVEN'S ability to form 3D representations of the scene and accurately predict actions. In this experiment, we directly test the effect of noisy camera parameters RAVEN'S performance.
We look at two cases of camera calibration error. In the first case, the model trained with ground- truth camera parameters but tested with noisy camera parameters. This case may arise in practice when using a fleet of robots. While the demonstration data may be generated on a carefully calibrated robot, other robots may fall out of calibration during deployment. In the second case, the model is trained and tested with the same noisy parameters. This case represents a scenario where the method is used on robot, but the calibration is done quickly and, therefore, is inaccurate. For
16

---

Published as a conference paper at ICLR 2026
these two cases, we evaluate the performance of RAVEN subset of MimicGen tasks, following same protocol as in Table [[ The ground-truth camera parameters are provided by the simulation sample some noise and add it offset to the ground-truth
environment. For noisy parameters, parameters. Specifically, we apply a + pixel (6.6%) shift to the principal point, a 2% variation in focal length, 0.5 to 1.5 em perturbation to the camera translation, and 6, -4, and degree variation to the camera roll, pitch and yaw angles, The results, shown in T
Lastly, we look at using data augmentation improve robustness camera calibration error. In the final row of Table 66, we include a comparison of RAVEN that was trained with data augmentation on the ground truth camera parameters (both intrinsic and extrinsic). and tested on noisy camera parameters. With data augmentation, the impact of noisy camera parameters is minimal, lowering performance by 3% on Coffee task and 0% on the other tasks. This experiment shows that RAVEN robust to reasonable camera calibration errors, so long data augmentation is used.
Table 6: Effect of feature map grid resolution RAVEN'S performance. Performance reported in success rate (%) on subset of MimicGen tasks, averaged over three seeds.
Training Parameters
Testing Parameters
Stack
Stack Three
Coffee
Hammer Cleanup
ground truth
ground truth noisy
50
73
noisy
100
80
49
71
ground truth
noisy
100
76
42
69
ground truth data aug
noisy
100
81
47
73
B.2 COMPARING GRID RESOLUTIONS FOR GENERATING RAYS
As stated in Section we use a pretrained ResNet encoder to preprocess image observations into feature maps. Each element in the feature map is then mapped to an SE(3) ray encoding. Since the ResNet downsamples the image, we end up with much fewer ray tokens than image pixels, which accelerates subsequent attention layers (which are quadratic in number of tokens). In this experiment, we compare performance when using different feature map grid resolutions. Specifically, we test with 3x3 feature maps, 5x5 feature maps (this is the default), and 7x7 feature maps. We vary the resolution by removing maxpool layers or reducing convolutional stride the ResNet; all other model parameters kept the same. We report the success rates subset of MimicGen tasks in Table
The results show that 5x5 grid resolution performs best. There is larger drop-off in performance with a 3x3 grid than a 7x7 grid. We believe a higher grid resolution allows the model to construct denser 3D understanding of the scene. It is worth pointing out that downsampling in the image encoder does not affect the SE(3) equivariance of our method, even though it does affect equivariance image shifts. This is because the SE(3) transformation acts on the camera extrinsines, not the image. In future work, it may be possible to achieve equivariance to image shifts (as in shift an image to model the effect of translating the camera position). This could further increase robustness to camera viewpoint changes.
Table 7: Effect of feature map grid resolution RAVEN'S performance. Performance reported in Success Rate (%) on subset of MimicGen tasks, averaged over three seeds.
Stack
Stack Three
Coffee
Hammer Cleanup
RAVEN (3x3 grid)
99
37
RAVEN (5x5 grid)
100
81
50
73
RAVEN (7x7 grid)
100
79
46
74
B.3 FLOW MATCHING vs. DIFFUSION
In this experiment, compare flow matching and diffusion for denoising action predictions. We compare performance on a subset MimicGen tasks in Table 3, We kept the model architecture the same, and only changed the loss applied and the denoising scheme inference. For flow-matching.
17

---

Published as a conference paper at ICLR 2026
followed the procedure and parameters presented in this work. For diffusion, used the approach from Wang et al. (2024). The results show that there is a small performance boost for flow-matte motthhnhinng.. although varies by task. Based our experience developing this work, diffusion perform better on tasks that require high precision. We did not focus any efforts on refining the parameters for the denoising schemes, and it is possible that additional improvements performance and speed could be realized after fine tuning.
sometimes
Table 8: Comparing performance of RAVEN using Flow-Matching and Diffusion for decoding actions. Performance reported in Success Rate (%) on subset of MimicGen tasks, averaged over three seeds.
Stack
Stack Three
Coffee
Hammer Cleanup
AVEN (Flow Matching)
100
81
50
73
RAVEN (Diffusion)
100
84
43
70
C GEOMETRIC TOKENS FOR OTHER OBSERVATION MODALITIES
In this work, focus observations with multiple images and the end-effector state. We believe that our method can be easily extended to robotic settings with other observation modalities. Integrating other inputs comes down to converting the data into geometric tokens. In most cases, the token features are generated with linear layer or a traditional network encoder (ViT or PointNet). The token poses may be less obvious, and we provide some suggestions here. We would like to empirically test these ideas in the future.
Joint Data In some tasks, we may wish to encode the current joint positions of the robot, especially for joint space action,s in ALOHA (Zhao et al.) 2024). One option is to convert the joint angles to features with a linear layer and assign the pose to be the robot base frame. Another option is to use forward kinematics to generate point cloud that represents the positions or meshes of each link, which exist in the robot base frame.
Torque/Force Data and point features could be used. It would make sense pose the data in the end-effector frame (for wrist-mounted force sensor) or a joint frame for motor torques.
This data already has a geometric structure, so a hard-coded mapping vector
Point Cloud Data A point cloud cannot be canonicalized in the same way that image end effector can. So, our recommendation to patchify the point cloud using clustering, embed each cluster into geometric features using PointNet, and assign the pose based on the cluster centroid and principal axes.
D SIMULATION EXPERIMENT DETAILS
We visualize the manipulation tasks from MimicGen (Mandlekar et al. 2023). DexMimicGen CJang al., (2024), and the Viewpoint Generalization task in Figure 7 and 88 respectively. Following prior work (Chi et al., 2023] Wang et al.) 2024 Hu et al.) 2025). we set the resolution of the image to 84 84 and adopt the same maximum episode length. In addition, we slightly modify the original simulator so that the environments can provide intrinsic and extrinsic camera parameters in real time. For the viewpoint generalization task, we select subset of tasks whose scene content is preserved under camera placement changes, ensuring that varying the viewpoint does not lead to significant information loss in the observations.
E TRAINING DETAILS
To capture temporal context, we follow prior work (Chi et al. 2023 Wang et al. 2024 and use a two-step history window on all observations input to the policy. Images encoded by pretrained ResNet-1e 18 (He et al., (0166) which was modified by replacing batchnorm with groupnorm layers (as done in DifiPo) and tweaking downsampling operations produce dense feature map (5 x grid
18

---

Published as a conference paper at ICLR 2026
(a) Stack DI
(b) Stack Three D1
(c) Square D2
(d) Threading D2
(e) Coffee D2
(f) Three Pc. Assembly D2
(g) Hammer Cleanup D1
(h) Mug Cleanup D1
(i) Kitchen D1
(i) Nut Assembly DO
(k) Pick Place DO
Coffee Preparation D1
Figure 6: The twelve simulation tasks from the MimicGen (Jiang et al. 2024) benchmark, each shown in its initial configuration.
cells from 84 84 simulation images; 7 grid cells with real-world images). The feature vector at each grid cell is mapped to the geometric token representation using linear projection. For all geometric tokens, we set dimensionality of the scalar, vector, and point features to be 64 (so there are 64 64 + 64 3 channels). The pose of each grid cell is calculated using camera projection equations (Equation)) shows the case for the z-direction, and we project the image plane axis as well get the and y-directions). Our SF(3) Geometric Transformer is composed of two transformer blocks that use multi-headed attention (8 heads).
These geometric tokens produced the encoder are fed decoder that either (i) uses EquiBot (Yang et al. 2024) to predict an absolute trajectory or (ii) our GTA Transformer to predict a relative trajectory. The decoder outputs a sequence of 16 action steps; all steps supervise training, while only the first steps are executed at evaluation time. During training, we apply random cropping to fixed resolution; evaluation, we use center cropping. In simulation, the observation size is 84 x 84. whereas for real- world experiments, we found this resolution insufficient for fine- fine-grained manipulation and therefore use 224 224. We train with AdamW (Loshchilov Hutter (017) and exponential moving average (EMA). The hyperparameters for flow matching were based on Chisari et al. (2024): 10 integration steps for training and testing, a position embedding scale of 20.0 (for time step encoding), a noise scaling of 1.0, linear flow schedule, scheme, and loss weights 10 for end-effector rotation eгror, 10 for translation eттor, and for the gripper state error. For all baselines, keep their original hyperparameters and only align the number training steps for a fair comparison. All runs use one GPÚ per experiment, executed on compute clusters and workstations equipped with multiple hirf-performance consumer-grade GPUs.
exponential scaling of 4.0, and uniform SNR sampling
19

---

Published as a conference paper at ICLR 2026
(a) Threading
(b) Piece Assembly
(c) Transport
(d) Box cleanup
Drawer Cleanup
(f) Tray Lift
Figure 7: The six simulation tasks from the DexMimicGen (Jiang et al. 2024 benchmark are shown, with the top row ((a)-(c)) showing gripper-based tasks and the bottom ((d)-(c) illustrating dexterous hand-based tasks.
(a) Coffee D2
(b) Stack Three DI
(c) Stack D1
(d) Hammer Clcanup D1
Figure 8: Viewpoint generalization experiments on four MimicGen tasks under perturbed camera views.
F RAVEN WITH ABSOLUTE ACTION PREDICTION
In this section, we introduce a variation of RAVEN that employs different action decoder to predict actions the absolute frame. We refer to this variant RAVEN-abs, The action decoder in RAVEN- abs follows the EquiBot (Yang et al. 2024) design: 1D conditional UNet in which all linear layers are made (O(3) (3)))-qquivarian using Vector Neurons (Deng al. 2021). Translational equivariance is achieved through canonicalization. where predicted offset vector is subtracted from the UNet inputs and added back to the predicted action at the output. Adapting the encoder outputs of RAVEN, set of geometric tokens, EquiBot's 's UNet requires several preprocessing steps. First, each token is decanonicalized by transforming into the global frame, after which all tokens are pooled into single feature vector. The offset vector is then obtained by mean-pooling the point components of
20

---

Published as a conference paper at ICLR 2026
velocily
p(())44
p² P'())
Conditional UNet
Mean Pool
nokey action
Figure 9: Process of feeding geometric tokens from our encoder network to the Unet Decoder from Equibot.
Table 9: Maximum success rates (%) on MimicGen Tasks with 100 demos.
Method
Average
Stack DI
Slack Three DI
Square D2
Threading
Coflee D2
3P Assembly D2
RAVEN
66
100
81
S0
28
61
27
Cleanup
Cleanup DI
Kitchen DI
N. Assembly
Place DO
Coffee Prep.
RAVEN
73
53
85
74
73
73
this vector. The scalar components are used as scalar conditioning for the UNet, while the vector components and the centered point components provide vector conditioning. This overall pipeline is illustrated in Figure 9
We conduct experiments comparing the absolute action decoding scheme with the originally proposed relative decoding approach on MimicGen (Table 9 and DexMimicGen (Table TO) tasks, using the same experimental settings as in the main paper. The results show that RAVEN-abs performs comparably to RAVEN in single-arm manipulation tasks but underperforms in bimanual settings. This gap is expected and can be attributed to the increased complexity of the bimanual action space. In such settings, absolute trajectory representations may lead to larger discrepancies between actions. For instance, when the two arms consistently move in opposite directions, their translation values differ significantly. In contrast, relative trajectories remain more stable and consistent across arms, potentially making them easier to model.
G FULL MIMICGEN RESULTS WITH STANDARD ERRORS
Table 11 shows the MimicGen benchmark results corresponding to Table I but with the standard error included.
H FULL DEXMIMICGEN RESULTS WITH STANDARD ERRORS
Table 12 shows the DexMimieGen benchmark results corresponding to Table 2 but with the standard error included.
I REAL WORLD EXPERIMENT DETAILS
Hardware Setup Figure 10 illustrates our real-world experimental setup. Two calibrated cameras (one GoPro for the eye-in-hand view and one RealSense D455i for the agent- view) used to capture the scene, both aligned to a common global frame with their intrinsics and extrinsics recorded. Demos collected using the Gello teleoperation interface (Wu et al. 2024) within 40cm x 40em square area, outlined orange. At each timestep, we synchronously record visual observations together with end-effector actions, including position, rotation, and gripper state.
Task Specifications Figure II shows averaged visualizations across multiple initial state distribu- tions for each task. Banana Picking, the mug tree translated randomly within the square area, and its orientation is varied +:45 when the branch is aligned horizontally to the robot base square. In
21

---

Published as a conference paper at ICLR 2026
Table 10: Maximum success rate (%) on DexMimicGen tasks with 50 demos.
Method
Average
Threading
3P Assembly
Transport
B. Clcanup
D. Clcanup
Tray Lift
RAVEN
82
63
69
76
98
97
90
RAVEN-abs
55
91
86
97
Table 1I: We report maximum success rates (%) on MimicGen tasks with 100 demos, averaged over three random seeds across different methods. Standard errors are indicated by +.
Method
Stack DI
Stack Three DI
Square D2
Threading D2
Coffee D2
Three Piece Assembly D2
RAVEN
100.0 0.0
80.7
50.0 4,0
28.0 2.0
60.7 4: 5.0
26.7 +: 4.2
abs
100.0 0.0
76.7
47.3 1. 6.1
31.3 1: 7.0
63.3 3,1
40.7 1: 3.1
EquiDiffPo
= 0.7
54.7 5.2
25.3 8.7
22.0 1.2
60.0 :: 2.0
15.3 1.8
DitPo(Pre)
96.7 = 1.2
52.7 = 3.1
22.0 2.0
20.7 1.7.11 1.2
52.7 + 2.3
14.0 = 5.3
DilPPo
ACT
34.7 = 0.7
6.0 2.3
6.0 + 0.0
10.0 1.2
19.3 2,4
0.0 0.0
Hammer Cleamup DI
Mug Cleanup D1
Kitchen DI
Nut Assembly DO
Pick Place DO
Coffee Preparation DJ
3.1
84.7
74.0
+: 4.6
RAVEN abs
62.7 1: 1.2
$4.0 1 3,5
79.3 1 4,2
87.7 1. 3.1
60.3 1 3.1
87.3 1: 6.1
EquiDifPPo
67.3
DiftPo(Pre)
54.0 2.0
4.0
80.0
71.0
46.3 4.7
65.3 = 1.2
DitPo
$2.0 = 122
42.7
66.7 2.4
$4.7
35.3 2.2
65.3 0.7
ACT
38.0 1: 4.2
23.3 1 0.7
37.3 4.3.
42
7.2
32.0 1: 2.0
Beans Scooping, the spoon is initialized with three discrete orientations, while bean-filled bowl and the pot are randomly placed within the square area. In Coffee Cleanup, the rag is hung on the mug tree with the branch orientation randomized +20 The coffee machine randomly placed within the square area, and its orientation is additionally perturbed by up to +00 when positioned 45 off perpendicular to the robot base square. In Box Cleanup, a screwdriver randomly placed within the square area, with its head pointing outward perpendicular +60%. We also visualize one episode for each task in Figure 12 where the subfigures illustrate the key frames corresponding critical action steps.
the robot base and perturbed by up
Progress and Subgoal Definitions In our evaluation, rather than using only binary final outcome (0 for failure and for success), we also introduce a progress reward to better capture the policy's s performance. Specifically, we define a sequence of key subgoals for each task, evenly distributing weights across them such that the total sums to 1. The progress reward is then computed by accumulating the weights completed key subgoals:
Banana Picking:
Grasp and lift the banana
Drop the banana into the transparent box
Beans Scooping:
Grasp the spoon
Scoop the beans
Transfer beans into the pot
Place the spoon back into the bowl
Coffee Cleanup:
Grasp the rag from the mug tree
Wipe the coffee machine
Close the machine cover
Press the handle to lock the cover
Box Cleanup:
Remove the box cover
22

---

Published as a conference paper at ICLR 2026
Table 12: We report maximum success rates (%) on DexMimieGen tasks with 50 and 100 demos, averaged over three random seeds across different methods. Standard errors are indicated by +.
Threading
Assembly
Transport
Avg (50 dermos)
Mothod
100
50
100
RAVEN
63.3 +: 3.1
80.0 + 2.0
69.3 +: 6.1
78.0 + 7.2
76.0 + 4,0
80.0 6.0
82.2
91.3
EquiDifPPo
28.7 + 3.1
45.3 + 4,6
62.6 2.3
62.0
DitiPo (Pre)
25.3 + 4.2
54.7
50.0
3.5
64.7
80.7
72.7
65.3
DallPo
20.0 + 2.0
42.7 +
42.7 6.4
71.3 # 6.4
78.0
+ 3.5
76.6
59.8
Cleamup
Cleanup
Tray
Avg (100 demos)
Method
100
50
100
RAVEN
98.0 1: 0.0
96.7 1: 1.2
96.7 1: 1.2
99.3 +: 1.2
90.0 1: 6.9
91.3 1.2
87.6
EquiDifPPo
85.3 + 4.2
94.0 = 5.3
7.2
DatPo (Pre)
85.3 +
88.7 + 5.8
96.0 3.5
98.7
44
80.0 7.2
76.6
DalPo
84.0 1:
88.7 1 3.1
91.3 3.1
92.7 2.23
42.7 4:3.
71.3 2.66 6.1
73.9
Figure 10:
The real-robot platform consists of UR5 robot arm equipped with a GoPro for the eye-in-hand view and an Intel RealSense D455i camera mounted on the left as the agent camera. The Gello teleoperation interface is also shown on the left.
Place the screwdriver into the box
Put the cover back on
(a) Banana Picking
(b) Beans Scooping
(c) Coffee Cleanup
(d) Box Cleanup
Figure 11: Distributions of random initial states used in the four roal-world tasks.
23

---

Published as a conference paper at ICLR 2026
Full Real-world Results Table reports the full results of the four worldworld experiments with the additional absolution action representation variant, including subgoal-level success rate (%), average progress reward (%), and overall task completion rate (%).
Discussion of RAVEN's Failure Modes In the beans scooping task, the most common failure was the robot getting stuck repeating the same dropping action sequence (e.g. tilting scooper) even though the beans were dropped already. In the coffee cleanup task, the failures were either getting stuck in the wiping stage or inaccurately pressing the handle when closing the lid. The demonstrations included three wiping motions in row, it makes sense that the robot would get stuck here since it has limited observation history (two frames) and more history may be needed to contextualize itself. For the box cleanup task, the robot would often make unsuccessful grasp attempt on the screwdriver. This is be expected since it requires a precise grasp pose and there was lot of variation in its initial placement across environment resets.
Table 13: Subgoal-level success rate (%), average progress reward (%), and overall task completion
rate (%) across four real-world tasks.
Subgoal
Subgoal2
Subgoal3
Subgoal4
Avg. Progress Reward
Overall Completion
Banana Picking
DifPPo (Pre)
55
50
N/A
N/A
53
50
RAVEN-abs
95
N/A
90
RAVEN
100
100
N/A
N/A
100
100
Beans Scooping
DifiPo (Pre)
85
0
RAVEN-abs
100
85
65
80
83
55
RAVEN
100
75
50
55
70
45
Coffee Cleanup
DiffPo (Pre)
100
90
20
15
56
15
RAVEN-abs
100
95
95
80
93
80
RAVEN
75
75
80
70
75
45
Box Cleanup
DifIPo (Pre)
65
35
30
N/A
43
30
RaVEN-abs
100
65
65
N/A
77
65
RAVEN
100
77
60
24

---

Published as a conference paper at ICLR 2026
(a) Banana Picking
(b) Beans Scooping
25

---

Published as a conference paper at ICLR 2026
(c) Coffee Cleanup
(d) Box Cleanup
Figure 12: Visualization of one episode for (a) Banana Picking, (b) Beans Scooping, (c) Coffee Cleanup, and (d) Box Cleanup. Each subfigure illustrates the trajectory of key action steps.
26


Reviews:
[{"review_id": "d97UBSqCuS", "reviewer": "Reviewer_rWp8", "summary": "This paper presents RAVEN, an end-to-end SE(3)-equivariant policy learning framework that operates using only RGB camera inputs. The key idea is to interpret 2D image patches as 3D rays defined by known camera intrinsics and extrinsics, which allows the use of SE(3)-equivariant geometric transformers to process visual observations. The encoder outputs geometric tokens that can be fused across arbitrary camera views, and the policy is trained with flow matching to predict continuous-time action trajectories. Empirically, RAVEN achieves strong results on MimicGen and DexMimicGen benchmarks and shows faster training and better viewpoint generalization compared to prior equivariant or diffusion-based policies.", "questions": "1. Camera calibration sensitivity: How robust is RAVEN to small errors in extrinsic or intrinsic calibration? Could the authors quantify the degradation in performance?\n\n2. Depth and RGB-D ablation: Have the authors tried integrating depth maps or estimated depth (e.g., from pretrained monocular models) into the ray-based representation? This would make a strong case for the claimed modality-agnostic design.\n\n3. Failure analysis: It would be useful to include qualitative failure cases — for instance, where equivariance assumptions break or where viewpoint perturbations cause prediction drift — to better understand the model’s limits.", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "The work introduces a creative and elegant approach to extend SE(3)-equivariance to RGB-only observations, bridging a long-standing gap between equivariant robotic policies (which typically rely on structured 3D inputs) and practical camera-based systems. The ray-based tokenization formulation is novel and theoretically sound.\n\nRAVEN emperically outperforms strong baselines across multiple tasks and even surpasses pretrained diffusion policies while training faster. The evaluation covers both simulation and real-world tasks.", "weaknesses": "Limited applicability in realistic deployments: The method’s core assumption — that camera extrinsics and intrinsics are precisely known and remain static — limits its applicability in unstructured real-world settings. The real-world experiments use carefully calibrated, fixed setups, so the paper does not convincingly demonstrate robustness to realistic calibration noise.\n\nMissing ablations / comparisons on input modality and architecture choices: The evaluation focuses exclusively on RGB input. Despite claiming compatibility with multimodal data, the paper lacks any ablation or comparison using non-equivariant model + depth, RGB-D, or depth-estimation-based representations, which are standard in 3D-aware robot learning literature. It is therefore unclear whether the gains stem from true equivariance or from better geometric bias in the encoder.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "op41dbmZGn", "reviewer": "Reviewer_DW7R", "summary": "The paper proposes a ray-based SE(3) equivariant visual manipulation policy that can be used end-to-end from RGB images without involving traditonal 3D modalities like point clouds. Speficially, a feature map from pretrained 2D image model is lifted to featured rays using known camera parameters. These featured rays are grouped to form a SE(3)-posed geometric token. A series of transformer blocks with Geometric transform attention (GTA) is then applied to equivariantly process visuo-proprioceptive information. Finally, these equivariant representations are decoded into flow-matching prediction. Experimental results on standard benchmarks confirm clear advantage of the proposed method.", "questions": "Please find my questions in the weaknesses section.", "limitations": "", "rating": 6, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "### [Strength 1] Novelty\nAs far as I know, this is the first ray-based equivariant robot policy. While [1] has previously explored equivariant models in Plucker space by leveraging its homogeneous space property as being SE(3)/(SO(2)xR), it was limited to computer-vision oriented applications. As ray-based representation is becoming increasingly popular in 3D vision fields [2,3,4], it is timely to explore its application to robotics field. \n\n### [Strength 2] Scalability\nMost existing SE(3) equivariant manipulation methods are not scalable as they rely on complicated and model-specific operations that are not well-optimized. On the other hand. the proposed method is more compatible with more commonly used operations that are heavily optimized. For instance, GTA is compatible with flash attention, which can greatly improve memory efficiency and training speed. \n\n[1] Xu et al. \"\"Equivariant Light Field Convolution and Transformer.\" \n\n[2] Jin et al. \"Lvsm: A large view synthesis model with minimal 3d inductive bias.\"\n\n[3] Jiang et al. \"RayZer: A Self-supervised Large View Synthesis Model.\"\n\n[4] Li et al. \"Cameras as relative positional encoding.\"", "weaknesses": "### [Weakness 1] Definition of Equivariance is Questionable\nThe claimed SE(3) equivariance property of the proposed method is defined with respective to the gripper and camera poses. This is a valid and helpful inductive bias especially when denoising multiple gripper pose trajectory or incorporating multiview information. However, it should be noted that this kind of definition is different from more accepted defintion of equivariant policy where the equivariance of the action is defined with respect to the content of the scene, not to the camera. Assuming correct point cloud registration, scene-to-gripper equivariance is a strict superset of this camera-to-gripper equivariance. As such, it is not quite true that this is \"first end-to-end equivariant policy learning method that works with image-based observations.\" In fact, this is a shared limitation of other equivariant methods based on ray [1] or spherical projection [2]. \n\n[1] Xu et al., \"Equivariant Light Field Convolution and Transformer.\" \n\n[2] Hu et al., \"3D Equivariant Visuomotor Policy Learning via Spherical Projection\"\n\n### [Weakness 2] Effectiveness of Equivariance is Questionable\nExperimental results are solid enough to support the effectiveness of the proposed method. However, it is unclear equivariance is indeed the main factor of this observed advantage. The use of GTA-like camera RoPE approach has been reported to be effective in several non-equivariant methods [3,4,5]. Hence, unless an ablation without equivariance is provided (i.e., using nonequivariant MLP), I would be inclined to believe that the performance advantage is mainly from this relative attention property, not from equivariance. In terms of benchmark, I think it is more fair to compare the baseline diffusion policy with the same GTA architecture (but with non-equivariant counterpart) than using the original Unet-based architecture. \n\nAlso, it is unclear why ray-based representation is necessary to achieve the equivariance. Unlike [1] which used homogeneous space convolution, the proposed method lifts rays onto SE(3) space without ever being grounded to ray space again. If so, why not simply tokenize a featured point cloud projected to a unit sphere in the local camera reference frame, and then lifting it onto SE(3)?\n\n[3] Miyato et al. \"Gta: A geometry-aware attention mechanism for multi-view transformers.\"\n\n[4] Kong et al. \"Eschernet: A generative model for scalable view synthesis.\"\n\n[5] Li et al. \"Cameras as relative positional encoding.\"", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "P8WLS8zBV2", "reviewer": "Reviewer_q8H3", "summary": "This paper presents RAVEN, the first SE(3)-equivariant policy learning framework that operates directly on RGB image observations. The key innovation is treating image patches as collections of 3D rays that transform under SE(3) roto-translations, enabling equivariant processing through geometric tokens.", "questions": "- **Ray Discretization**: How does the grid resolution (5×5 in your implementation) affect the quality of the ray-based representation and overall performance?\n\n- **Viewpoint Generalization Paradox**: If the method is truly camera-agnostic, why does performance drop when only the agent-view camera is perturbed (Table 3)? Doesn't this suggest the equivariance guarantee is weaker than claimed?\n\n- **Computational Cost**: What is the actual inference time compared to baselines? Training time is reported, but deployment efficiency matters for robotics.\n\n- **Flow Matching vs Diffusion**: Is the performance gain primarily from the ray-based encoder or from using flow matching instead of diffusion? An ablation would clarify this.\n\n- **Comparison with [1]**: This work also claims RGB-based SO(3) equivariance. What are the key differences, and why wasn't this included as a baseline?\n\n\n[1] Boce Hu, Xupeng Zhu, Dian Wang, Zihao Dong, Haojie Huang, Chenghao Wang, Robin Walters, and Robert Platt. Orbitgrasp: Se(3)-equivariant grasp learning. arXiv preprint arXiv:2407.03531", "limitations": "", "rating": 2, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "- The ray-based formulation for achieving SE(3) equivariance from RGB images is elegant and principled. Converting image patches to geometric tokens with SE(3) poses is a creative solution to a real problem in robotics.\n- The experiments cover a range of test settings, and the ablation studies help understand each component of the method.\n- Overall the experiment results are good, which demonstrates the effectiveness of the proposed method.", "weaknesses": "- **Limited Theoretical Analysis of Ray-Patch Representation**: While the paper states that \"each grid cell in the feature map is associated with an element of SE(3)\", the justification for how a patch of pixels/rays defines a unique orientation is somewhat hand-waved (Figure 2). A more rigorous analysis of when this representation is well-defined and how discretization affects equivariance would strengthen the work.\n\n- **Incomplete Equivariance Analysis**: The viewpoint generalization task (Section 5.1) perturbs only a single camera, which \"breaks global equivariance\" as the authors note. This seems to contradict the claimed advantage of camera-agnostic operation (\"remaining agnostic to the number and placement of cameras,\" line 48).\n\n- **Real-World Experiments Limited in Scope**:\n  - Only 4 tasks with relatively simple manipulation primitives are considered.\n  - No analysis of failure modes or edge cases is provided.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "YMmcXASWyK", "reviewer": "Reviewer_KKzi", "summary": "This paper proposes an end-to-end approach for learning SE(3)-equivariant visuomotor policies directly from RGB input. The key idea is to convert image patches into geometric tokens. Each token carries canonicalized features in a local frame and an explicit SE(3) pose derived from the pixel ray geometry, using camera intrinsics/extrinsics. They leverage geometric transform attention (GTA) layer then computes attention using relative token poses, enabling SE(3)-equivariant attention. Finally, the decoder predicts end-effector (EEF)-relative actions via action tokens anchored at the current EEF pose. Results are reported on standard simulation benchmarks and real-robot tasks.\nThe approach is logical and practically appealing. While the experimental results over baselines are sometimes limited, the paper presents a clean pathway to SE(3)-aware RGB policies. The claims would be strengthened by broader EquiDiffPo comparisons, parity with 3D-input methods.", "questions": "- Include EquiDiffPo comprehensively (at least all simulation tasks) for a fair comparison.\n- Add comparisons to 3D input policies (point-cloud/voxel) to substantiate the benefits of RGB-only SE(3)-equivariance.", "limitations": "", "rating": 6, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "-A clear and principled route to achieving SE(3)-equivariance without relying on 3D point-cloud or voxel inputs—enabled by ray-based tokenization and pose-aware attention for end-to-end equivariant policy learning from image-based observations.\n -A coherent system architecture that serves as a practical guideline for leveraging SE(3) symmetry in pixel-based visuomotor control tasks.", "weaknesses": "- Novelty granularity: The core contribution is the \"integration\" of known components (ResNet features, ray back-projection, pose-aware attention, and an EEF-relative action head). The encoder is mostly standard aside from attaching per-patch poses. The paper's strongest novelty is at the system level, rather than in the encoder/decoder design individually.\n- Experimental strength: Improvements over an equivariant baseline (e.g., EquiDiffPo) are often modest, though the baseline leverages SO(2)-symmetry. Given the emphasis on \"SE(3) equivariance from 2D\", it would be more compelling to match the performance from the 3D-input methods, (using e.g., point-cloud/voxel) under identical protocols [1,2]. \n\n<Typos>\np.6, 4.2., 3rd paragraph: convert he action geometric tokens\" -> \"convert the action geometric tokens\"\n\n[1] Wang et al., Equivariant Diffusion Policy (2024).\n[2] Zhu et al., SE(3)-Equivariant Diffusion Policy in Spherical Fourier Space (2025).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)