EquAct: An SE(3)-Equivariant Multi-Task Transformer for 3D Robotic Manipulation
Authors: Xupeng Zhu
Yu Qi
Yizhe Zhu
Robin Walters
Robert Platt
Venue: ICLR 2026 Poster (2026)
Keywords: SE(3) Equivariance; Multi-task Transformer; sample efficient
Primary area: applications to robotics, autonomy, planning
TL;DR: We propose a novel SE(3) equivariant multi-task transformer that is provably adapts action to 3D scene transformations.

Abstract: Multi-task manipulation policy often builds on transformer's ability to jointly process language instructions and 3D observations in a shared embedding space. However, real-world tasks frequently require robots to generalize to novel 3D object poses. Policies based on shared embedding break geometric consistency and struggle in 3D generation. To address this issue, we propose EquAct, which is theoretically guaranteed to generalize to novel 3D scene transformations by leveraging SE(3) equivariance shared across both language, observations, and action. EquAct makes two key contributions: (1) an efficient SE(3)-equivariant point cloud-based U-net with spherical Fourier features for policy reasoning, and (2) SE(3)-invariant Feature-wise Linear Modulation (iFiLM) layers for language conditioning. Finally, EquAct demonstrates strong spatial generalization ability and achieves state-of-the-art across $18$ RLBench tasks with both SE(3) and SE(2) scene perturbations, different amounts of training data, and on $4$ physical tasks.

Published as a conference paper at ICLR 2026
EQUACT: AN SEQUA-EQUI MULTI-TASK TRANSFORMER FOR 3D ROBOTIC MANIPULATION
Xupeng Zhu, Yu Qi", Yizhe Zhu; Robin Walters Robert Platt Khoury College of Computer Sciences Northeastern University Boston, MA, 02115 [zhu. xup, qi yu2, zhu. zhu.yizhe, r. r.walters) (natterssBBothheatemm...c rplatt@ccs neu.
ABSTRACT
Multi-task manipulation policy often builds transformer's ability to jointly process language instructions and 3D observations in a shared embedding space. However, real-world tasks frequently require robots to generalize to novel 3D object poses. Policies based on shared embedding break geometric consistency and struggle in 3D generation. To address this issue, we propose EquAct, which is theoretically guaranteed to generalize to novel 3D scene transformations by leveraging SE(3) equivariance shared across both language, observations, and action. EquAct makes two key contributions: (1) an efficient 3F(33--qqivart) point cloud-based U-net with spherical Fourier features for policy reasoning, and (2) SE(33)iivariant Feature-wise Linear Modulation (iFiLM) layers for language conditioning. Finally, EquAct demonstrates strong spatial generalization ability and achieves state-of-the-art across 18 RLBench tasks with both SE(3) and SE (2) scene perturbations, different amounts of training data, and on physical tasks. Code is available in
com/ ZXP S works. /EquAct
INTRODUCTION
Recent breakthroughs in multi-task keyframe action policy learning (Shridhar et al. 2023 Goyal et al. 2024 Fang et al. (2255) have been driven by the success of transformer architectures Vaswani et al.
2017). which excel at bridging different modalities by tokenizing language, 3D observations, and next-best keyframe actions into shared embedding space. However, real-world robotic tasks often involve substantial SE(3) variation in object poses-for example, harvesting fruit from branches diverse orientations, performing pipe work with fixtures mounted on walls ceilings, and assembling components onto pegs at angles. While transformers excel at cross-modal integration, their tokenization process discards the underlying 3D geometric structure. Consequently, existing multi-task keyframe action methods struggle to generalize to novel 3D scene configurations and require large amounts of robot data to learn geometric priors from scratch.
This paper presents EquAct, a novel multi-task transformer that is theoretically guaranteed to generalize to novel 3D scene transformations. EquAct adapts actions scene transformations and SE(3)-invariant) when language instructions remain unchanged. This adaptation is achieved by introducing a novel -Equivvana point transformer U-net with field networks for keyframe action evaluation, alongside novel 3E((3)invartn) FiLM (iFiLM) layers to condition the policy on language
with
semantically dependent yet geometrically invariant way.
EquAct is the first method to achieve continuous (G(3))eqqivariann) (covering both 3D rotation and translation) in a single unified model for multi-tase task policy. In contrast, previous (E(3) (E(3)-qquivariil eqquivariant methods (Simeonov et al. 2022 Ryu et al. 2024 Huang al. a Zhu et al.) 2025b)
single task, and multi-task methods (Zhu et al.) 2025a Gervet et al. 2023 Ke et al.) are only translationally
cquivariant. Moreover, reflect that realistic tasks, objects often have random 3D poses, rether
Equual contribution.
I Equal advising.

---

Published as a conference paper at ICLR 2026
than random 2D poses in RLBench, proposes RLBench tasks with SE(3) initialization. EquAct achieving stateoff-the-an art performance on RLBench SE(2) and (E(3) benchmarks, and on 4 physical experiments. Practically, our method leverages spherical Fourier representation, and achieves computational efficiency for training and inference, matching the computation overhead of non- equivariant baselines.
To summarize, the contributions of this paper are as follows:
1. We propose a continuous (B))333euivariaal multi-task policy with novel equivariant U-net architecture, novel invariant FiLM layers, and novel equivariant field networks.
2. We mathematically prove the relevant equivariance and invariance properties.
3. We verify that EquAct outperform baselines across on RLBench with 18 tasks and 249 language goals with SE(2) or SEE(3) initialization, and on 4 physical tasks.
2 RELATED WORKS
Keyframe action and multi-task manipulation policy. Keyframe action formulation was first introduced by (James & Davison 2022). which approximates closed-loop manipulator trajectories using sequence of discrete keyframes, thereby simplifying policy learning. Building this idea, PerAct (Shridhar et al., 2023) proposes a transformer- based agent that learns a multi-task policy-executing different keyframe actions conditioned on natural language instructions. Later, the multi-task policy learning has diverged into two main directions to evaluate translational action. The first class consists of
methods (Goyal et al., 2023 2024 Wang et al.. 2024b) Zhang et al. 2025 Fang et al. 2025), where the 3D scene is projected into three orthogonal image planes, followed by a ViT-like (Dosovitskiy et al. 2020) multi-view transformer that evaluates
translational action values. While this approach is computationally efficient, reasoning in the image
plane sacrifices geometric fidelity and requires clever strategies to project into SE(3) (Xu et al. 2024)
or SO(3) (Klee et al.) 2023 Park et al. 2022) space to achieve -quivvaanie. The second
class operates directly in 3D space (Gervet et al. 20233 Xian & Gkanatsios [023] Ke et al.) Garcia et al.] 2025), typically using boint-cloud-based transformers with densely sampled query points or diffusion models (Chi et al., 2202)) to evaluate translational actions. These methods can achieve 3D translational equivariance through 3D CNNs relative positional embeddings (Su et al. 2024), but are not 3D relationally equivariant. For rotational action prediction, existing approaches typically rely on discretized Euler angles or denoise (3) rotations. While the former suffers from gimbal lock and discontinuity issues (Zhou et al. 2019). the latter incurs significant computational overhead due to iterative refinement. In contrast, EquAct achieves both translation and rotation equivariance. also achieves fast inference by evaluating actions in one shot.
Equivariant policy learning. Previous works (Van der Pol al. 2020 Wang et al. 2022b) have shown that geometric structures are inherent in reinforcement learning problems and that iccopprating
equivariant policy learning can lead to improved performance. Building on this insight, a series of
methods (Zeng al.] 20188 Wang et al. 2021 Zhu et al. 2022; Huang et al. 2022, Wang et al.
2022a Zhu et al. 2023] Liu et al. 2023 Wang et al. 20233 Nguyen et al. 2023 Huang et al.
2023a
Zhao et al.
2023 Jia et al. 2023 Huang et al. Kohler et al. 2024 Wang et al. 2024a
Tangri
et al. 2024 Hu et al. 2025 have proposed SE eequivariant policy learning for robotic tasks. More
recently, (Simeonov et al. 2022 Ryu al. Huang al. 2023b Ryu al. 2024 Hu et al. Gao
jet al. al. 2025 Zhu al. (025b) extended equivariance to the full SE(3) group. However, all of these
2024| Zhu et al. [202a]] Yang et al. 2024 Huang et al. b Qi et al.) [025] Yang et al. Tie
equivariant policy learning methods are limited to single-policy learning. In contrast, EquAct learns
multi-task. language conditioned keyframe policies using a single unified model.
Equivariant neural networks. There are several approaches to achieving equivariance in learning- based robotic policies. A common method is data augmentation (Laskin et al. 2020). where both inputs and outputs are transformed according to the desired group symmetry during training. Another strategy is canonicalization (Zeng et al. 2018). which aligns inputs to a canonical frame prior to inference. An altermative is to leverage equivariant neural networks, which incorporate equivariance directly into the architecture through symmetry perttyy-rreeeiingg operations. Prior works (Wang et al.) 202T Zhu et al. 2022 Miller et al. 2020) have shown that such networks outperform data augmentation
2

---

Published as a conference paper at ICLR 2026
and canonicalization by significant margin. Equivariant neural networks grounded in rigorous math from group theory, enabling them to preserve symmetry while maintaining high expressiveness. One class of such networks leverages group convolutions (Cohen & Welling 2016 Weiler & Cesa
2019; Cesa et al. 2022), which typically discretize a symmetry group and apply convolution over its elements. However, these approaches may suffer from discretization artifacts. Another class operates
in the Fourier domain (Geiger& & Smidt [022 Liao & Smidt 2023; Passaro & Zitnick, [2023] Liao al. 2024), which offers a more compact and continuous representation of the group. Building on this Fourier-based framework, our method achieved natural language conditioning and fast SE(3)
action inference.
Equivariant natural language processing. Incorporating natural language into equivariant models has recently gained attention (Li et al. 2025 Roche et al. 2024, Jia et al. 2024). Li et al. (2025) and Roche et al. (2024) combine equivariant graph neural networks with invariant language embeddings and evaluate their effectiivness at scale. However, they are either limited to molecule generation or (2) equivariance. In contrast, our work is the first to explicitly identify the SE(3) invariance of natural language instructions in the context of robotic policies, and introduce simple yet effective invariant FiLM (iFiLM) layers to enforce this invariance within an SE(3) -qqiivarria policy network.
3 BACKGROUND
Equivariant policy learning. A function f is equivariant with respect to a group G' if the group action 9 E G commutes with the function, i.e., f(g x) = 9 f(x). In this paper, focus the special Euclidean group SE(3) = SO(3) x T(3), which represents 3D rigid-body transformations composed of 3D rotations SO(3) and translations T(3). An equivariant robotic policy (Wang et al. 2022c b) satisfies the property:
o) g T()),
(1)
meaning the action transforms as the observation transforms. For example, an (B(2)-qquivarian planar grasping policy (Zhu al. 2022) predicts a grasp pose from an input image; if the image
is rotated, the predicted grasp pose rotates accordingly. There are several strategies to enforce
equivariance in neural network-based policies. One common approach data augmentation (Laskin et al. 2020 Wang et al. (022c), where both observations and corresponding actions are transformed according to Equation| during training. Another method is canonicalization, which transforms the input into a standard reference frame aligned with the action space (Zeng al. 2018). More recently, robot policies that leverage equivariant neural networks (Wang et al. 2022c Zhu et al. 20022] Huang et al., 2023b Weiler & Cesa 2019 Deng et al. 1021 Zhu et al. (025b) have been shown to outperform these alternatives by embedding equivariance directly into the network architecture, but none of them studies multi-task policy learning.
Spherical harmonics. -qquiivaaaa models rely feature representations based spher- ical functions and spherical harmonics. A spherical function fs: 52 - R maps a point on the sphere u 2 to a real value y. An alternative representation of fs is its Fourier form, where the function is decomposed into spherical harmonic coefficients ct via the spherical Fourier trans- form F: fa + far such that f. {c"). }. Each coefficient c/" denotes the weight of the cor- responding spherical harmonic Y.M 52 + R. which forms an orthonormal basis for the func- tion space I (S2 .R). These basis functions
indexed by type (or degree) 1 E Zzo and order m E Z such that -IS m <. I. The inverse spherical Fourier transform reconstructs the spatial function as F (f)
Em=-, c (u). In practice, truncated spherical coefficients 1 1 -mmzz are used because they provide a good approximation of the spherical function (Liao & Smidt 2023). Spherical functions are steerable under SO(3), making them well-suited for SO(3)-
=
equivariant neural networks (Thomas et al. 2018 Liao & Smidi, 2023 Fuchs et al., 2020 Passaro & Zitnick 2023 Liao et al. 2024). Specifically, rotating the input function by g E SO(3), i.e.,
[. = g (2) = folg 2) corresponds to rotating its Fourier coefficients via the Wigner mo (g) q", where c²' are the coefficients of the rotated function f.. For example, a type-0 feature is a scalar, and its Wigner D-matrix is identity; a type-1 feature is a 3D vector, and its Wigner D-matrix is 3D rotation matrix.
D-matrices D: of - D
3

---

Published as a conference paper at ICLR 2026
Insert onto the yellow pog"
CLIP
Transforee A
FILM
Q Qapen
CRCc
or gripper pose
point cloud gripper pose
latent spherical features
V
Qr distribution over SO(3)
Open gripper open
Figure 1: Overview of EquAct. EquAct first encodes the observation o = {s, e} into latent spherical features h using a Squiiaaqui U-Net, enc,, while conditioning the natural language instruction n through invariant iFiLM layers. Based on the encoded features h, EquAct then samples and refines translational query actions and gripper open actions using equivariant field network, resulting in action value functions Q, and Qopen- Finally, a rotational field network aggregates spherical features from h centered at the predicted translation at to obtain a latent feature Φ. which subsequently convolved with learned filter v to produce the rotational action value function Qr-
Spherical CNN. Spherical Convolutional Neural Networks (Cohen et al. 2018 lift a spherical function fo to an SO(3) function fso(3) SO(3) IR by convolving it with a learnable spherical filter vp such (Js [g] Js² (u) u) du, SO(3). This spatial convolution is
equivalent to outer product in the Fourier domain: f. * ② f. G. which is more efficient than performing the convolution directly in the spatial domain (Cohen et al. 2018 Klee et al.) 2023).
Multi-task keyframe policy formulation. Following PerAct (Shridhar et al. 2023). we formulate multi-task keyframe manipulation policy learning as a mapping from an observation o and a natural language instruction to the next best keyframe action of the gripper a, denoted as <(,,, n) a motion planner generates a trajectory to reach this keyframe action. This formulation decomposes robot trajectories into a sequence of keyframe poses, thus simplifies learning. For example, PerAct learns multi- task keyframe policy using 53 demonstrations, but (Team et al.) 2025) relied on 64, 262 demonstrations learn trajectory policy. The observation o = {s,e) consists of the scene information 8, and the end-effector state e. The scene is represented as a colored point cloud of 2500 points generated from 256 x 256 RGBD images captured by calibrated front, left, right, and in-hand cameras. The end-effector state e or action a are expressed as Ş = {Er, 3. = or & = where Ç SF(33 denotes the gripper pose and Eopen E (0, 1} indicates whether the gripper closed or open. The instruction is represented as natural language string. The policy is trained via imitation learning: we first collect expert demonstrations D consisting of observations, natural language goals, and expert actions, and then train the policy to predict the expert actions. At evaluation time, we test the policy on the training tasks but with novel object poses. For more details, please see Appendix
Then
METHOD
rotate
EquAct is a multi-task keyframe action policy represented implicit function Qo, n, R that estimates the action value given an observation o, a language goal n, and query action The inference procedure is illustrated in Figure [ and has the following steps. 1) The SE(3)- Equivariant Point Transformer U-Net encodes the ob- servation o that includes a point cloud and the gripper pose e into set of latent spherical features / at each point in the cloud h. (o). 2) Invariant Feature-wise Linear Modulation layers fuse the language embedding k, which is treated type-O features, into the U-Net. Here, is the encoding of the natural language instruction n, by using frozen CLIP (Radford et al. 2021) tokenlizer and a Transformer (Vaswani et al. 2017) encoder. 3) The Equivariant Field Network takes the latent point
D0"
9-9
8-0
<(a,n)
x(g o. g-a
n plcck_ the mmon"
Figure 2: The equivariance and invari- ance of the multi-task keyframe pol- icy. Under the equivariance assumption, when the observation is transformed to g o, the predicted action transforms accordingly to g a. Under the invari- ance assumption, given a fixed natural language instruction n, the action trans- formation depends solely on the transfor- mation applied to the observation.
4

---

Published as a conference paper at ICLR 2026
cloud h and sampled query actions {at, Oopens ar as input and predicts values for each action ((a, h) E R. The final output actions a; a, and a are chosen as those with the highest action values.
During training, EquAct minimizes the following loss:
C EDEAA H((Qr(ar n2), āz) + H(Qr(ar, ar, o,n), + H(Q (Copen, Re. Ti), Ropen
where (,, a) D are expert demonstrations consisting of observations natural language instructions n, and expert actions û, and a ~ denotes uniformly sampled query actions from action space. Specifically, a, consists of 449 uniformly sampled translational actions, and a, consists of 36,864 rotational actions sampled using the HEALPix (Gorski et al. 2005 Klee et al. 2023 grid. H denotes cross-entropy loss. Intuitively, this loss treat policy learning classification problem in which the goal the policy to correctly choose the expert action from among all available actions. During training, also augment the dataset with respect equation by randomly rotating the point cloud and the action simultaneously with [15 45, +45% rotation along [x. z] axis.
4.1 EQUIVARIANCE ASSUMPTIONS IN MULTI-TASK MANIPULATION POLICY LEARNING
EquAct assumes that the keyframe action policy is equivariant with respect to the observation. That is, when the observation undergoes transformation, the predicted action should transform accordingly (Wang et al.) 2021] Zhu et al.) [022]] Huang et al. 2022 Ryu et al.). Additionally, we identify and assume that the action invariant to the natural language instruction meaning that for a fixed instruction, the action should transform solely based on SE(3) transformation of the observation.
Formally, this behavior is expressed as:
Tgg o, n) = a. g E SE(3),
where the group action g operates both the observation o and the predicted action by applying rigid-body transformations to the point cloud or the gripper poses and @r, see Figure 2 for an illustration.
Methodologically. EquAct achieves equivariance between the observation and action by employing novel SE(3) SE(3) equivariant Point Transformer U-Net (Section 4.2p and -qquivaann field networks (Section 4.4). In parallel, it enforces invariance with respect natural language instructions via the proposed (F())-invrrian layer-wise modulation (iFiL.M layers (Section
Proposition 4.1. EquAct is (B(33)eqqivarian in observation- mapping and -invarraa to nature language instruction, described in Equation
This is proved by induction; see Appendix B.
4.2 EQUIVARIANT POINT TRANSFORMER U-NET (EPTU)
N.
N/4,61
N/106,18
128
2564,
N/64,
N/16, 128
N/4,64
N.32
N,32
(a) Overview of EPTU.
3 +
lype-O
podtraus
MLP
7ype- vectors
Iypa-2
veciorr
input
outsut
Inzul
Input
(b) Detailed structure design each module. Red color indicates the magnitude of feature.
Figure 3: (E(()) Point Transformer U-net (EPTU).
5

---

Published as a conference paper at ICLR 2026
The SE(3) -Equivvaavaa Point Transformer U-Net (EPTU, Figure 3) encodes point cloud into equiv- ariant latent features by propagating both local and global information across points. Compared to non-equivariant counterparts such Point Transformer (Zhao al. 2021) and U-Net (Ronneberger 2015), EPTU achieves continuous SF(33) (e33)--quivar by leveraging spherical Fourier features in its hidden layers. EPTU further improves the computational efficiency of EquiformerV2 (Liao 2024) by adopting a U-net-style architecture (Ronneberger et al. 2015), which incorporates
(et al.,
ct al. novel spherical Fourier maxpooling layers to compress point cloud features and spherical Fourier up- sampling layers to reconstruct features back to the original resolution. These pooling and upsampling layers are interleaved with standard EquiformerV2 (Liao al.) (024) graph attention blocks, which first construct k- k-nearest-neighbor graph for each point and then apply equivariant iitention-based message passing. EPTU also incorporates skip connections (Ronneberger et al. 2015) between the downsampling and upsampling stages. Compared to prior equívariant point U-Net (Ryu et al.] [024]] Hu et al.). the proposed U-Net is straightforward to implement, by climinating the need for caching graphs, I.c., the sup block in Figure (b) does not need the maxpool graph in the smaxpool model.
Spherical Fourier maxpooling. Analogous to the maxpooling operation in convolutional neural networks (LeCun et al. 1998). the spherical Fourier maxpooling layer (Figure middle) reduces the resolution of the feature map in the spherical Fourier domain. Specifically, for a point x, the layer aggregates features from its k-nearest neighhorhood (ct.p p E kmn())) and selects the spherical Fourier coefficient with the largest magnitude at each degree l:
d smaxpool(c.,) (ctpp E knn(z)} = CL.p²
p" = arg paNnenn 11t
(3)
where the 2/- + 1 1-dimension vector Ct,p 10 O.
-1 -+1
ct denotes the type-l spherical Fourier
coefficient at point
Proposition 4.2. The spherical Fourier maxpooling operation defined in Equation SF((3- equivariant. That is. for any E SO(3) and E T(3):
D(r)- 4, ttx (ma c.plp E + knn(x)}.
This is proved by the orthogonal property of Wigner D-matrices, Appendix B.2 for proof.
Spherical Fourier upsampling. Interpolation is commonly used for upsampling feature maps (Zhao et al. 2021 Ronneberger al., 2015). To extend this operation to the spherical Fourier domain, we propose a novel spherical Fourier upsampling method (Figure 3 (b) right). Specifically, for each (b)
type-l component, we perform a coefficient-win interpolation over the k-nearest neighbors of a query coordinate x:
d/ = sup (cppp xlp E knn())) softmax
( pll
CLP:
(5)
Proposition 4.3. The spherical Fourier upsampling operation defined in Equation 5 is SE(3)-
equivariant. Specifically, for any r E SO(3) and E T()):
D(r) 6, = (up( c. + xlpett knn(z)}-
(6)
See Appendix(B.. for proof. The proof is based on Schur's 's lemma (Schur 1905) and that the linear
SO(3) action on the Fourier coefficients.
4.3 INVARIANT FEATURE-WEL LINEAR MODULATION LAYERS (IFILM)
We propose invariant Feature-wise wise Linear Modulation (iFiLM) layers (Figure 3 (b) left) to enforce the geometric invariance of natural language conditioning in the policy, as defined in Equation 2 Unlike standard FiLM layers (Perez et al. 2018), which do not guarantee equivariance or invariance, the iFiLM layer is provably SE(3) invariant with respect to the conditioning input k. Specifically, the iFiLM layer takes as input a spherical Fourier feature e and a type-O (invariant) condition feature k; and outputs a semantically modulated feature c':
d' iFiLM(c, k),
an B, MLP(k),
& ata,
for > 0,
(8)
d Bcp + 7, for 1 0,
(9)

---

Published as a conference paper at ICLR 2026
where iFiLM first uses a m perceptron to project the condition & into type-O modulation scales a, 3 and bias 7. Then, iFiLM scales the type- input feature a by og for all / >> 0, and applies an affine transformation to the type-O features using B and 7.
Proposition 4.4. The invariant feuture-wise linear modulation (iFiLM) layer is with respect to the condition input k, and SO(3) -equivaaqa with respect to the input feature c. Specifically, for any rotation E SO(3):
D(r) c' = (D())(D(r)) c. k))
(10)
See Appendix BB44for for a proof. The proof utilizes Shur's lemma (Schur 1905).
4.4 EQUIVARIANT FIELD NETWORK
EquAct evaluates actions over the entire pose action space AT C SF(3), rather than actions anchored at each point in the point cloud (Hu et al.). To achieve this, introduce equivariant field networks q that propagate features from the latent point cloud representation to any query point aT E AT. where the action is decomposed into translational and rotational components, @r at Or-
For translational action value evaluation, given the query translational action a and the latent point cloud h, the field network qi builds graph with h as the source and a₂ as the destination, then performs graph attention to aggregate spherical Fourier features from /t to dç. The graph connects the query point the k-nearest neighbor in h. The graph attention is implemented by one EquiformerV2 attention block (Liao et al. 2024). The graph building and attention operation is similar to (Gervet et al.) 2023 Ryu al.] 2024] Chatzipantazis et al. 2023), except that the output, i.c., the translation action value is invariant to rotation (Wang et al. 2021 Zhu el al. 2022): qe (a, h) = q: (a(a. 9 h).g E (O)). Therefore, the field network only takes the type-U feature from aggregated features. We evaluate the translational action in a coarse-to to-finefff fashion, where the initial resolution of action is coarse, and the subsequent sampling refines the action. The gripper open action Jopen (Sopen, Oz, h) is evaluated in the same way, except that Gopen outputs two channels of type-0 features, corresponding to the open/close action values.
For rotational action value evaluation, given the query trans-rotal action a, ar and the latent point cloud h, the field network gr first aggregates features in the same way as the translational network to obtain the spherical Fourier features è at ag. Then the action value for a, is calculated by a spherical
CNN
(Cohen et al. 2018) with a learnable filter & (a ae, h) = (ó * [ar] = - [a,]] Notice that our field network gr performs spherical convolution at a 3D location at and (E)() cquivariant, which differs from the previous 50(3) equivariant spherical convolution (Cohen et al.
2018 Klee al. 2023. Howell et al. 2023) that operates in images.
EXPERIMENTS
5.1 SIMULATION EXPERIMENTS
Task setups. We benchmark multi-task algorithms 18 RLBench (Shridhar al., 2023 James ct al., (020) tasks. The benchmark uses a Franka Panda robot equipped with a parallel gripper. Observations are captured from four RGB-D cameras positioned the front, left shoulder, right shoulder, and wrist, with resolutions of either 128² or 256² pixels. Each task includes several variations specified by natural language instructions. For example, in the "open "open_drawe" drawer" task, "open_the tatrr__th_tpppp lop_drawer" and "open_ the _tt__tihdleeee_eeeeeeeeeeeeeeee middle_drawer" are two distinct variations. Across all tasks, there are between 2 and 60 variations per task, resulting in a total of 249 variations.
Evaluation metric. Performance is measured by a binary reward, where 0% and 100% correspond to failure and successful completion of the task according the natural language instruction, respectively. We report the task success rate over 25 evaluation episodes per task, with a maximum of 25 steps per episode. During evaluation, the objects and language goals remain the same as in the training set, but the object poses are novel.
Baselines. We benchmark our method with two strong baselines. SAM2ACT (Fang et al. 2025) is the current state-of-the-ar -frt baseline on 18 RL RlBecc,, which leverages pretrained image tokenizer from
7

---

Published as a conference paper at ICLR 2026
the chocolate box the supooard"
"screw the light bub from grey holder lamp"
"pick and them mug
Slot the cylnner into sorter
orange spoke"
place the mustard container the cupboard"
place the ring onto sooke"
Screw the light bula from the green holder into the lamp"'
"move 2 from the table the
"olace the syfinder info its slot shapn
"luck the top flower."
pick up the strawbarry'
"disssemmbl the second toilitt paper
Figure 4: Experiments setups. First 18 standard RLBench tasks (Shridhar al. 2023). Second row: 18 RLBeech tasks with SF(3) random initialization. Third row: 3 SE(3) and SF physical experiments and robustness test. A language instruction specifies each variant of the task.
Table 1: Multi-task success rate (%) on 18 RLBench tasks with 249 instructions. On average,
EquAct outperforms all the baselines on all 3 settings. Furthermore, the second column shows that EquAct's training and inference time, GPU memory matches baselines. 2D/100 and 2D/10 denote 100 and 10 training demonstrations per task with object poses randomly initialized in SE(2). 3D/10 denotes task with object poses randomly initialized in SE(3) and 10 demonstrations per task.
av2.
s00r044 nite
open drower
slide black
sweep dust.
mest off grill
Methed
3D/10
20/10
2TV100
3D/10
2D/10
20/100
3D/10
2TN10
20/100
STM10
2D/10
2TV100
3D/10
2D/10
2TV100
EquAct
53.3
60.1
89.4
55
74
48
56
100
61
83
96
100
100
SAMEACT
37.0
76
37.9
Train.
Infer.
Mem.
SUTUW
pui in safe
place wine
put in cupbouml
Method
Ldo 4
(.69
(GB)
3D/10
2DV10
2D/100
3DV10
2D/L0
2D/100
3D/10
2D/10
2DV100
3TVL0
2D/10
2D/100
EquAct
240
0.7
21
36
53
76
91
100
14
95
95
36
22
89
SAMZACT
225
0.1
21
89
68
48
98
68
93
75
jar
drag stick
cups
place cups
Method
31V10
20M10
20/100
3DV10
20V10
20/100
3LV10
21N1O
21V100
30/10
21V10
2LY100
3LV10
20V10
20M100
EquAci
33
91
85
18
68
62
76
SAMZACI
8
68
99
44
100
59
16
20
76
12
78
4
47
3DDA
24
18
10
24
put drawer
shape
Insert pag
Method
3D/10
2DV10
2D/100
3D/10
2DV10
2DV100
3DV10
2D/LO
2DV100
3D/10
2D/10
2D/100
3D/L0
2DV10
2D11000
EquAct
67
100
SAMRACT
36
96
64
100
16
56
100
2N
84
3DDA
48
74
99
14
92
96
14
33
44
95
98
14
66
SAM2 Ravi et al. 2024 and projects point cloud into image planes (Goyal et al. 2024). 3DDA stands for 3D diffuser actor (Ke et al.), which takes point cloud input and leverages diffusion policy to capture multi-modality in the demonstrations. All the baselines are trained and evaluated on single RTX 4090 GPU with 24 GB memory. We report hyperparameters in Appendix G
8

---

Published as a conference paper at ICLR 2026
Experiment settings. We benchmark baselines three experiment settings with increasing diffi- culty. In the 100 setting (Shridhar et al. (202),, the model is trained with 100 demonstrations per task, then tested with randomly SE()) initialized objects. In the 10 setting, the model is trained with 10 demonstrations per task and tested in the same way as 100. In the 10 SF(3) setting, the training set contains 10 demo per task and both the training and testing scenes have randomly SF((3 initialized objects.
Results. Table I shows that on average, EquAct outperforms all the baselines on the 100 setting by 2.6%, the 10 setting by 6.2%, and the 10 SE(3) setting by 15.4%. Furthermore, the more difficult the setting is, the more EquAct outperforms the baselines, demonstrating strong sample efficiency and 3D generalization. EquAct also excels tasks requiring precisions, e.g., "place cups" and "sort shape", where other baselines struggle. This indicates that the equivariance is crucial for a policy adapting precisely to objects pose. Lastly, EquAct underperforms baselines in the tasks in which the object's pose is fixed, c.g. "sweep_ to dustpan". Besides success rate, the sceond row in Table | shows that EquAct matches the training/inference time and GPU memory consumption of other baselines.
5.2 REAL-WORLD EXPERIMENTS
We benchmark the performance of EquAct and baseline 4 physical multi-task with 11 variations and 135 demonstrations in to- tal: "disassemble pipe", "pluck_ flower", "pick_fruit" "install Biiiaall_tile___aeae toilet_roll", The pose of objects in all tasks, except "pick_fruit undergo random SE(3) transformations within the manipulator's workspace. De- tails of the experiment setting are given in Appendix F] We evaluate 10 episodes for each task and report the binary success rate. Notice that physical settings are more challenge than simulation, due to noisy demonstrations and noisy observations. We baseline with the best model 3DDA (Ke et al.) in the 10-SE(3) setting in Table I and show quantitative results in Table 2] EquAct effectively learns physical SE(3) multi-task keyframe policy from limited demonstrations, achieving 65% average success rate. In comparison, 3DDA struggles in these experiments, often skipping keyframe actions and resulting in failure.
Table 2: Real-world experiments.
"pluck_flowrr,,
avg. SR f
disass.
pluck flower
pick fruit
install toilet roll x
pipe
Var x Demo
3 10
3 15
3 x 10
Ours
65.0
90
70
50
50
20
5.3 ABLATION STUDY
We perform the ablations on the 10 demo setting: Ours:
the full EquAct model. aug. + no aug. removes data augmentation by training with the raw demonstration data. iFiLM > FiLM: ablates iFiLM layers by replacing them with standard FiLM layers (Perez et al.) 2018). / - + 2: reduces the spherical feature resolution in EquAct (reducing the spherical harmonic degree from 3 to 2). EPTU - VN: replaces the equivariant Point Transformer U-Net with a VN-DGCNN (Qi et al., (2025) Deng et al. 2021 network, which is also SE(3) equivariant. equ. + no equ.: breaks the equivariance by replacing one equivariant layer in qe and
Table 3: Ablation study.
avg. SR
place wine
reach drag
cups
peg
Ours
$2.8
45
62
90
14
ang. -> no aug.
50.5
36
85
10
iFiL FiLM
50.3
68
90
[ 1=
3 + 2
45.5
64
28
80
10
22.0
equ. oqu.
12.3
35
0
g. with Roformer transformer layer (Su et al. 2024 Gervet et al. 2023)
Table reports the multi-task success rates across 4 RLBench tasks. Even though only a single equivariant layer is replaced, equ. -> no equ. results in the largest performance drop, underscoring the critical role of geometric structure in EquAct. The EPTU - VN ablation shows that although VN-DGCNN is (E(()--quivariant,
underperforms EquAct by more than 30%. A likely reason is that VN-DGCNN operates only on vector features (type-1 irreps), whereas EquAct leverages higher-order features (up to type-3). Similarly, the / + 2 ablation highlights the importance of high-resolution spherical Fourier coefficients for accurate action reasoning. Additionally, replacing iFiLM with FiLM causes a notable drop on precision tasks (c.g.. "place_cups"). confirming iFiLM's precision advantage, though FiLM overfit on the tasks where actions nearly constant. Finally,

---

Published as a conference paper at ICLR 2026
aug. aug. indicates using data augmentation can further improve performance, hypothesize that data augmentation reduces numerical error in the equivariant neural networks.
5.4 ROBUSTNESS TEST AND EMPIRICAL EQUIVARIANCE ERROR
Robustness to occlusion. To explore the degree to which our equivariance mitigates the effects of extrinsic corruptions such as partial observability, we performed an ab-
Table 4: Robustness to occluded point cloud.
Method
PCD
Avg. SR
Place Wine
Place Cups
Reach Drag
Insert
lation study on RLBBecc tasks given 10 demonstrations per-task (see Table 4 In the occluded setup, all models are trained and tested using only the front and in-hand cameras instead of all 4 cameras, resulting in significant occlusion in the point cloud. We found that EquAct's performance decreased by only 5.8%, suggesting that the model works well under this type of uncertainty. In contrast, 3DDA (Ke) al.) struggles when the observation occluded.
Peg
EquAct EquAct
Fall
52.8
45
62
90
14
Occluded
47.0
50
43
96
3DDA
Occluded
15.8
55
5
3
Robustness to distraction. To test the robust- ness towards unseen distraction objects, we ad- ditionally evaluated the EquAct model in Sec- tion 5.2 on the "disassemble_pipp"
Table 5: Robustness distracting objects.
task with variations In Table randomly placed 10 additional distraction objects in the scene, in- cluding toy car, small soccer ball, mug, a tape, a plate, etc.), and the performance of EquAct dropped from 90% to 70% success rate, demonstrating strong robustness in the cluttered scenes.
# Dist. Obj
Avg.
Disas. the
Disas, the
Disas. Links
Link
2nd
90%
3/4
70%
3/3
2/3
2/4
Equivariance error. In addition to providing theoretical proofs of EquAct's equivariance in Sec- tion 4 we measure its empirical equivariance eтror in Appendix E EquAct achieves lower SE(3) equivariance error than 3DDA (Ke et al.)). Together, the equivariance error, the equivariance proofs, and the consistent outperformance reported in Table [ validate that equivariance is crucial for spatial generalization.
CONCLUSION AND LIMITATIONS
This paper proposes EquAct to leverage SE(3) equivariance in the multi-task keyframe policy and invariance in the language instruction. Specifically we novel equivariant point transformer U-net (EPTU) to encode the observation and use equivariant field networks to evaluate action candidates. Then we propose invariant FiLM layers to modulate the policy with natural language instructions. In the end, EquAct outperforms SOTA baselines by 2.6% and 6.2% when trained with 100 10 demos in SEE(2) setting, and by 15.4% when trained with 10 demos in SEE(3 setting. Physical experiments validated that EquAct can solve complex tasks with SEE(3 variation. Additional experiments empirically validate that EquAct is robust to distractors, resilient to occlusion, and exhibits low equivariance eтror.
There are several limitations of EquAct. Firstly, the keyframe action formulation assumes the task can be solved by several key gripper poses. This assumption is satisfied in RL.Bench tasks but could be broken fine-graind manipulation settings (Chi al. 2023). Moreover, despite EquAct scales well with training data in Table I the data efficiency and semantic generalization could be further improved by leveraging pre-trained vision models (Radford et al. 2021 Shafiullah et al. 2022 Gervet et al. 2023). Lastly, the training and the inference speed of EquAct is slower than the best baseline; a more efficient equivariant backbone can speed up the inference.
10

---

Published as a conference paper at ICLR 2026
ACKNOWLEDGMENTS
We would like thank Hyunwoo Ryu, David Klee, well all members of the Helping Hands Lab for their valuable discussions. This work was supported in part by NSF grants 2107256, 2134178, 2314182, 2409351, 2442658, and NASA grant 80NSSC19K1474.
REPRODUCIBILITY STATEMENT
Our implementation available in https: /tt:://gtttbb 7XP- http:///S-S-ow///ppaaaaaa Hyper- parameters for all simulation experiments are detailed in Appendix G
ETHICS STATEMENT
This research employs only publicly available datasets released under appropriate licenses with publisher ethical approval. We collect no personally identifiable information and use no harmful or sensitive data, All work is conducted for academic research purposes.
REFERENCES
Gabriele Cesa, Leon Lang, and Maurice Weiler. A program build (n)-equivariant steerable enns. In International conference on learning representations, 2022.
Evangelos Chatzipantazis, Stefanos Pertigkiozoglou, Edgar Dobriban, and Kostas Danilldiis Se(3)- equivariant attention networks for shape reconstruction function space. In The Eleventh Interna- tional Conference on Learning Representations, 2023.
Cheng Chi, Zhenjia Xu, Siyuan Feng, Eric Cousineau, Yilun Du, Benjamin Burchfiel, Russ Tedrake, and Shuran Song. Diffusion policy: Visuomotor policy learning via action diffusion. The International Journal of Robotics Research, pp. 02783649241273668, 2023.
Taco Cohen and Max Welling. Group equivariant convolutional networks. In International conference machine learning, pp. 2999-2999. PMLR, 2016.
Taco Cohen, Mario Geiger, Jonas Köhler, and Max Welling. Spherical enns. In International Conference on Learning Representations, 2018.
Congyue Deng, Or Litany, Yueqi Duan, Adrien Poulenard, Andrea Tagliasacchi, and Leonidas J Guibas. Vector neurons: A general framework for so (3)-cquivariant networks. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 12200-12209, 2021.
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, G Heigold, S Gelly, et al. An image is worth 16x16 words: Transformers for image recognition at scale. In International Conference on Learning Representaations, 2020.
Haoquan Fang, Markus Grotz, Wilbert Pumacay, Yi Ru Wang, Dieter Fox, Ranjay Krishna, and Jiafei Duan. Sam2act: Integrating visual foundation model with memory architecture for robotic manipulation, 2025. URL htttps:// http:////.aaa...rra////2.......
Fabian Fuchs, Daniel Worrall, Volker Fischer, and Max Welling. (3)-transformers: 3d roto- translation equivariant attention networks. Advances in neural information processing systems, 33: 1970-1981, 2020.
Chongkai Gao, Zhengrong Xue, Shuying Deng, Tianhai Liang, Siqi Yang, Lin Shao, and Huazhe Xu. Riemann: Near real-time se (3)-equivariant robot manipulation without point cloud segmentation. In 8th Annual Conference on Robot Learning. 2024.
Ricardo Garcia, Shizhe Chen, and Cordelia Schmid. Towards generalizable dision-languee robotic manipulation: A benchmark and llm-guided 3d policy. In International Conference on Robotics and Automation (ICRA), 2025.
11

---

Published as a conference paper at ICLR 2026
Mario Geiger and Tess Smidt. e3nn: Euclidean neural networks. arXiv preprint arXiv:2207.09453 2022.
Theophile Gervet, Zhou Xian, Nikolaos Gkanatsios, and Katerina Fragkiadaki. Act3d: 3d feature field transformers for multi-task robotic manipulation. In Conference on Robot Learning, pp- 3949-3965 PMLR, 2023.
Krzysztof M Gorski, Eric Hivon, Anthony Banday, Benjamin Wandelt, Frode K Hansen, Mstvos Reinecke, and Matthia Bartelmann. Healpix: A framework for high-resolution discretization and fast analysis of data distributed on the sphere. The Astrophysical Journal, 622(2):759, 2005.
Ankit Goyal, Jie Xu, Yijie Guo, Valts Blukis, Yu-Wei Chao, and Dieter Fox. transformer for 3d object manipulation. CoRL, 2023.
Rvt: Robotic view
Ankit Goyal, Valts Blukis, Jie Xu, Yijie Guo, Yu-Wei Chao, and Dieter Fox. Rv12: Learning precise manipulation from few demonstrations. RSS, 2024.
Owen Howell, David Klee, Ondrej Biza, Linfeng Zhao, and Robin Walters. Equivariant single view pose prediction via induced and restriction representations. Advances in Neural Information Processing Systems, 36:47251-47266 2023.
Boce Hu, Xupeng Zhu, Dian Wang, Zihao Dong, Haojie Huang, Chenghao Wang, Robin Walters, and Robert Platt. Orbitgrasp: Se (3)-qqivvariant grasp learning. In 8th Annual Conference on Robot Learning.
Boce Hu, Heng Tian, Dian Wang, Haojie Huang, Xupeng Zhu, Robin Walters, and Robert Platt. Push-grasp policy learning using equivariant models and grasp score optimization, 2025. URL
https: org/ abs 2504. 03053
Haojie Huang, Owen Lewis Howell, Dian Wang, Xupeng Zhu, Robert Platt, and Robin Walters. Fourier transporter: Bi-equivariant robotic manipulation 3d. The Twelfth International Conference on Learning Representations, a.
Haojie Huang, Karl Schmeckpeper, Dian Wang, Ondrej Biza, Yaoyao Qian, Haotian Liu, Mingxi Jia, Robert Platt, and Robin Walters. Imagination policy: Using generative point cloud models for learning manipulation policies. In 8th Annual Conference on Robot Learning. b.
Haojie Huang, Dian Wang, Robin Walters, and Robert Platt. Equivariant Transporter Network. In Proceedings of Roboties: Science and Systems, New York City, NY, USA, June 2022. doi: 10. 1155677/RS5.2222XX111..00
Haojie Huang, Dian Wang, Arsh Tangri, Robin Walters, and Robert Platt. Leveraging symmetries in pick and place. arXiv preprint arXiv:2308.0 07944, 2023a.
Haojie Huang, Dian Wang, Xupeng Zhu, Robin Walters, and Robert Platt. Edge grasp network: graph-based (3)-invariant approach to grasp detection. In 2023 IEEE International Conference on Robotics and Automation (ICRA), pp. 3882-3888. IEEE, 2023b.
Stephen James and Andrew J Davison. Q-attention: Enabling efficient learning for vision-based robotic manipulation. IEEE Robotics and Automation Letters, 10):1612-1619, 2022.
Stephen James, Zicong Ma, David Rovick Arrojo, and Andrew J Davison. Rlbench: The robot learning benchmark learning environment. IEEE Robotics and Automation Letters, 5(2):3019- 3026, 2020.
Stephen James, Kentaro Wada, Tristan Laidlow, and Andrew Davison. Coarse-to-fine fine q-attention: Ef- ficient learning for visual robotic manipulation via discretisation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13749, 2022.
Mingxi Jia, Dian Wang, Guanang Su, David Klee, Xupeng Zhu, Robin Walters, and Robert Platt. Seil: Simulation -ugieeeted equivariant imitation learning. In 2023 IEEE International Conference on Robotics and Automation (ICRA), pp. 1845-1851, IEEE, 2023.
12

---

Published as a conference paper at ICLR 2026
Mingxi Jia, Haojie Huang, Zhewen Zhang, Chenghao Wang, Linfeng Zhao, Dian Wang, Jason Xinyu Liu, Robin Walters, Robert Platt, and Stefanie Tellex open-vocabulaty pick and place via patch-level semantic maps. arXiv preprint arXiv:2406. 2024.
Tsung- Wei Ke, Nikolaos Gkanatsios, and Katerina Fragkiadaki. 3d diffuser actor: Policy diffusion with 3d scene representations. In 8th Annual Conference on Robol Learning.
David Klee, Ondrej Biza, Robert Platt, and Robin Walters. Image sphere: Learning equivariant features for efficient pose prediction. In The Eleventh International Conference on Learning Representations, 2023.
Colin Kohler, Amuj Shrivatsav Srikanth, Eshan Arora, and Robert Platt. Symmetric models for visual force policy learning. In 2024 IEEE International Conference on Robotics and Automation (ICRA), pp. 3101-3107. IEEE, 2024.
Misha Laskin, Kimin Lee, Adam Stooke, Lerrel Pinto, Pieter Abbeel, and Aravind Srinivas. Rein- forcement learning with augmented data. Advances in neural information processing systems, 33: 19884-19895, 2020.
Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11):2278-25 2-224, 1998.
Zongzhao Li, Jiacheng Cen, Bing Su, Wenbing Huang, Tingyang Xu, Yu Rong, and Deli Zhao. Large language-geometry model: When IIm meets equivariance. arXiv preprint arXiv:2502. 1125502.11111 11149, 2025.
Yi-Lun Liao and Tess Smidt. Equiformer: Equivariant graph attention transformer for 3d atomistic graphs. The Eleventh International Conference on Learning Representations, 2023.
Yi-Lun Liao, Brandon Wood, Abhishek Das, and Tess Smidt. Equiformerv2: Improved equivariant transformer for scaling to digher-degree representations. In The Twelfth International Conference on Learning Representations, 2024.
Shigi Liu, Mengdi Xu, Peide Huang, Xilun Zhang, Yongkang Liu, Kentaro Oguchi, and Ding Zhao Continual vision- based reinforcement learning with group symmetries. In Conference on Robot Learning, pp. 222-240 PMLR, 2023.
Benjamin Kurt Miller, Mario Geiger, Tess E Smidt, and Frank Noé. Relevance of rotationally equivariant convolutions for predicting molecular properties. arXiv preprint arXiv:2008.0846) 2020.
Hai Huu Nguyen, Andrea Baisero, David Klee, Dian Wang, Robert Platt, and Christopher Amato. Equivariant reinforcement learning under partial observability. In 7th Annual Conference Robot Learning, 2023. URL https::////// httpss::///peeeeeeeeeeee net forum? id AnDDMOgM7-
Jung Yeon Park. Ondrej Biza, Linfeng Zhao, Jan Willem van de Meent, and Robin Walters. Learning symmetric representations for equivariant world model. In International Conference on Machine Learning, 2022. htt.thttt////..t http:::://axxvv..oo..oo
2204 11371
Saro Passaro and C Lawrence Zitnick. Reducing so (3) convolutions to (2) for efficient equivariant gnns. In International conference on machine learning, pp. 27420-27438. PMLR, 2023.
Ethan Perez, Florian Strub, Harm De Vries, Vincent Dumoulin, and Aaron Courville. Film: Visual reasoning with general conditioning layer. In Proceedings of the AAAI conference on artificial intelligence, volume 32, 2018.
Yu Qi, Yuanchen Ju, Tianming Wei, Chi Chu, Lawson LS Wong, and Huazhe Xu. Two by two: Learning multi-task pairwise objects assembly for generalizable robot manipulation. arXiv preprini arXiv:2504.06s 06961, 2025.
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. International conference on machine learning, pp.
PmLR, R, 2021.
13

---

Published as a conference paper at ICLR 2026
Nikhila Ravi, Valentin Gabeur, Yuan- Juan-Ting Hu, Ronghang Hu, Chaitanya Ryali, Tengyu Ma, Haitham Khedr, Roman Rädle, Chloe Rolland, Laura Gustafson, et al. Sam 2: Segment anything in images and videos. arXiv preprint arXiv:2408.0m 240.:2040000 2024.
Rahmatullah Roche, Bernard Moussad, Md Hossain Shuvo, Sumit Tarafder, and Debswapna Bhat- tacharya. Equipnas: improved protein-nucleic acid binding site prediction using protein-language- model-informed equivariant deep graph neural networks. Nucleic Acids Research, 52(5)=227=227, 2024.
Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomedical image segmentation. In Medical image computing and computer-assisted Intervention-MICC 2015: 18th international conference, Munich, Germany, October 5-9, 2015, proceedings, part III 18, pp. 234 241. Springer, 2015.
Hyunwoo Ryu, Hong-in Lee, Jeong- Hoon Lee, and Jongeun Choi. Equivariant descriptor fields: Se (3) equiqvaaari energy- -basdd models for end-to-end visual robotic manipulation learning. In The Eleventh International Conference Learning Representations.
Hyunwoo Ryu, Jiwoo Kim, Hyunseok An, Junwoo Chang, Joohwan Seo, Tachan Kim, Yubin Kim, Chaewon Hwang, Jongeun Choi, and Roberto Horowitz. Diffusion-edfs:: Bi-equivariant denoising generative modeling on se (3) for visual robotic manipulation. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 18007-18018, 2024. 20244
Issai Schur. Neue begründung der theorie der gruppencharaktere. In Sirzungsberichte der Königlich Preußischen Akademie der Wissenschaften zu Berlin: Jahrgang 1905; Erster Halbband Januar bis Juni, pp. 402. Verlag der Königlichen Akademie der Wissenschaften, 1905.
Nur Muhammad Mahi Shafiullah, Chris Paxton, Lerrel Pinto, Soumith Chintala, and Arthur Szlam. Clip-fields: Weakly supervised semantic fields for robotic memory. arXiv preprint 22X1v:2210.05663, 2022.
Mohit Shridhar, Lucas Manuelli, and Dieter Fox. Perceiver-actor: A multi-task transformer for robotic manipulation. In Conference on Robot Learning, pp. 785-799. PMLR, 2023.
Anthony Simeonov, Yilun Du, Andrea Tagliasacchi. Joshua B Tenenbaum, Alberto Rodriguez, Pulkit Agrawal, and Vincent Sitzmann. Neural descriptor fields: Se (3)-equivariant object representations for manipulation. In 2022 International Conference on Robotics and Automation (ICRA), pp. 6394-6400. IEEE, 2022.
Jianlin Su, Murtadha Ahmed, Yu Lu, Shengfeng Pan, Wen Bo, and Yunfeng Liu. Roformer: Enhanced transformer with rotary position embedding. Neurocomputing, 568: 127063, 2024.
Arsh Tangri, Ondrej Biza, Dian Wang, David Klee, Owen Howell, and Robert Platt. Equivariant offline reinforcement learning. arXiv preprint arXiv:2406. 13961, 2024.
TRI LBM Team, Jose Barreiros, Andrew Beaulieu, Aditya Bhat, Rick Cory, Eric Cousineau, Hongkai Dai, Ching-Hsin Fang, Kunimatsu Hashimoto, Muhammad Zubair Irshad, Masha Itkina, Naveen
Kuppuswamy, Kuan-Hui Lee, Katherine Liu, Dale McConachie, Ian McMahon, Haruki Nishimura, Calder Phillips-Grafttin, Charles Richter, Paarth Shah, Krishnan Srinivasan, Blake Wulfe, Chen Xu, Mengchao Zhang, Alex Alspach, Maya Angeles, Kushal Arora, Vitor Campagnolo Guizilini, Alejandro Castro, Dian Chen, Ting-Sheng Chu, Sam Creasey, Sean Curtis, Richard Denitto, Emma Dixon, Eric Dusel, Matthew Ferreira, Aimee Goncalves, Grant Gould, Damrong Guoy, Swati Gupta, Xuchen Han, Kyle Hatch, Brendan Hathaway, Allison Henry, Hillel Hochsztein, Phoebe Horgan, Shun Twase, Donovon Jackson, Siddharth Karamcheti, Sedrick Keh, Joseph Masterjohn, Jean Mercat, Patrick Miller, Paul Mitiguy, Tony Nguyen, Jeremy Nimmer, Yuki Noguchi, Reko Ong, Aykut Onol, Owen Pfannenstichl, Richard Poyner, Leticia Priebe Mendes Rocha, Gordon Richardson, Christopher Rodriguez, Derick Seale, Michael Sherman, Mariah Smith-Jones, David Tago, Pavel Tokmakov, Matthew Tran, Basile Van Hoorick, Igor Vasiljevic, Sergey Zakharov, Mark Zolotas, Rares Ambrus, Kerri -oozellB Benjamin Burchfiel. Hadas Kress-Gazit, Siyuan Feng, Stacie Ford, and Russ Tedrake. A careful examination of large behavior models for multitask dexterous manipulation, 2025. URL httlhhtts:////m
org/ abs /507 05331
14

---

Published as a conference paper at ICLR 2026
Nathaniel Thomas, Tess Smidt, Steven Kearnes, Lusann Yang. Li Li, Kai Kohlhoff, and Patrick Riley. Tensor field networks: Rotation- and translation -qquiiarann neural networks for 3d point clouds. arXiv preprint wwx.::802.0022111 2018.
Chenrui Tie, Yue Chen, Ruihai Wu, Boxuan Dong, Zeyi Li, Chongkai Gao, and Hao Dong. Et- seed: Efficient trajectory-level se (3) equivariant diffusion policy. The Thirteenth International Conference Learning Representations, 2025.
Elise Van der Pol, Daniel Worrall, Herke van Hoof, Frans Oliehoek, and Max Welling. Mdp homomorphic networks: Group symmetries in reinforcement learning. Advances in Neural Information Processing Systems, 33:4199-4210, 2020.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Iilia Polosukhin. Attention all you need. Advances in neural information processing systems, 30, 2017.
Dian Wang, Robin Walters, Xupeng Zhu, and Robert Platt. Equivariant SqS learning in spatial action spaces. In 5th Annual Conference on Robot Learning, 2021. URL httpps://opeereee/e// net/ d= Scz42A3iCT
Dian Wang, Mingxi Jia, Xupeng Zhu, Robin Walters, and Robert Platt. On-robot learning with equivariant models. In 6th Annual Conference on Robot Learning, 2022a. URL https:// openreview net/forum wett/orrummdd-3360000 21d=33
Dian Wang, Robin Walters, and Robert Platt. -qquivaauva reinforcement learning. In Interna- tional Conference on Learning Representations, 2022b. URL
net/
Darrum1d=779cc0l
Dian Wang, Robin Walters, Xupeng Zhu, and Robert Platt. Equivariant y learning in spatial action spaces. In Alcksandra Faust, David Hsu, and Gerhard Neumann (eds.), Proceedings of the Sth Conference on Robot Learning, volume 164 of Proceedings of Machine Learning Research, pp. 1713-1723, PMLR, 08-11 Nov 2022c. URL http:tttppocceeeeeiiii
wang22]. html
Dian Wang, Jung Yeon Park, Neel Sortur, Lawson L.S. Wong, Robin Walters, and Robert Platt. The surprising effectiveness of equivariant models in domains with latent symmetry. In International Conference on Learning Representations, 2023. htLLhttp::/// http://pppneeweeeeeeeeeeeeeeeeee
id=P AcU
Dian Wang, Stephen Hart, David Surovik, Tarik Kelestemur, Haojie Huang, Haibo Zhao, Mark Yeatman, Jiuguang Wang, Robin Walters, and Robert Platt. Equivariant diffusion policy. In 8th Annual Conference on Robot Learning, 2024a. URL https: htt::///ppeeeiiiieeeeeee hwt.:eett/foooeeee
Id=wD2KUVLTIq
Weiyao Wang, Yutian Lei, Shiyu Jin, Gregory D. Hager, and Liangjun Zhang. Vihe: Virtual in-hand eye transformer for 3d robotic manipulation. 2024 JEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 403-410, 2024b. 10802366.
doi: 10.1 10.1109//00558592.200444
Maurice Weiler and Gabriele Cesa. General e (2)-equivariant steerable cnns. Advances in neural information processing systems, 32, 2019.
Zhou Xian and Nikolaos Gkanatsios. Chaineddiffuser: Unifying trajectory diffusion and keypose prediction for robotic manipulation. In Conference Robot LearningProceedings of Machine Learning Research. Proceedings of Machine Learning Research, 2023.
Yinshuang Xu, Dian Chen, Katherine Liu, Sergey Zakharov, Rares Andrei Ambrus, Kostas Daniilidis, and Vitor Campagnolo Guizilini. $SF(3)$ equivariant ray embeddings for implicit multi-view depth estimation. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024. URL TRL htttttsst//ppeeeeeeeeee
www//foruue ot yRuJqoWoCs
Jingyun Yang, Ziang Cao, Congyue Deng. Rika Antonova, Shuran Song. and Jeannette Bohg. Equibot: Sim equivvarat diffusion policy for generalizable and data efficient learning. In 8th Annual Conference Robot Learning.
15

---

Published as a conference paper at ICLR 2026
Jingyun Yang, Congyue Deng, Jimmy Wu, Rika Antonova, Leonidas Guibas, and Jeannette Bohg. Equivact: Sim equivvarra visuomotor policies beyond rigid object manipulation. 2024 IEEE international conference robotics and automation (ICRA), pp. 9249 9255. IEEE, 2024.
Andy Zeng, Shuran Song, Stefan Welker, Johnny Lee, Alberto Rodriguez, and Thomas Funkhouser. Learning synergies between pushing and grasping with self-supervised deep reinforcement learning. In 2018 IEEE/RS. International Conference on Intelligent Robots and Systems (IROS), pp. 4238- 4245. IEEE, 2018.
Xinyu Zhang, Yuhan Liu, Haonan Chang, Liam Schramm, and Abdeslam Boularias. Autoregressive action sequence learning for robotic manipulation. IEEE Robotics and Automation Letters, 2025.
Hengshuang Zhao, Li Jiang, Jiaya Jia, Philip HS Torr, and Vladlen Koltun. Point transformer. In Proceedings of the IEEE/CVF international conference computer vision, pp. 16259-16268, 2021.
Linfeng Zhao, Xupeng Zhu, Lingzhi Kong, Robin Walters, and Lawson LS Wong. Integrating symmetry into differentiable planning with steerable convolutions. In International Conference on Learning Representations. International Conference on Learning Representations, 2023.
Yi Zhou, Connelly Barnes, Jingwan Lu, Jimei Yang, and Hao Li. On the continuity of rotation representations in neural networks. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 5745-5753, 2019.
Xupeng Zhu, Dian Wang, Ondrej Biza, Guanang Su, Robin Walters, and Robert Platt. Sample efficient grasp learning using equivariant models. Proceedings of Robotics: Science and Systems (RSS), 2022.
Xupeng Zhu, Dian Wang, Guanang Su, Ondrej Biza, Robin Walters, and Robert Platt. On robot grasp learning using equivariant models. Autonomous Robots, 2023.
Xupeng Zhu, David Klee, Dian Wang, Boce Hu, Haojie Huang, Arsh Tangri, Robin Walters, and Robert Platt. Coarse-to-fine 3d keyframe transporter, 2025a. URL htttp:://axxv..or
abs/ /2ts.2502.ao 01773
Xupeng Zhu, Fan Wang, Robin Walters, and Jane Shi. Se (3)-equivariant diffusion policy in spherical fourier space. In Forty-second International Conference on Machine Learning, 2025b.
16

---

Published as a conference paper at ICLR 2026
A
THE USE OF LLM
In this work, we utilize large language models (LLMs) exclusively for language polishing and refinement our written content. We do NOT employ LLMs for any other aspects of this research, including but not limited to: conceptual development, experimental design, data analysis, result interpretation, literature review, or generation of core research content. All substantive intellectual contributions, methodological innovations, and scientific insights are entirely our own work.
B PROOFS
B. PROOF OF PROPOSITION 4.1
Proof. To prove the equivariance of EquAct with respect to o, we only need to prove that every layer of EquAct is equivariant, then by induction, EquAct is equivariant to the observation o. See Proof B.2 B.3 that proves the equivariance of the proposed spherical maxpool layers and the proposed spherical upsampling layers. Referring (Liao al. 2024) for proof Equiformer layers and (Cohen
et al. 2018 for proof of Spherical CNNs.
To prove the invariance of EquAct with respect the nature language instruction n, we only need to prove that the iFiLM layers are invariance to n, see Proof B.4
B.2 PROOF OF PROPOSITION
Proof. Focusing on the right-hand side of Equation and denoting the point with the largest manganite of Fourier coefficients after transformation g as pzz
smaxpool(D(r) lp t knn(z)) D(r) CL.p
(1I)
Expanding the equation of pÿ. and using the property that the winger-D matrices are orthogonal, we have:
Pg arg max (m)()))
[D(r) c..ll
(12)
arg max (u)
(D(() pppp) (pp))
(13)
arg maxper+
c D(r) ap)
(14)
arg max, kmal
(pp)
(15)
arg max
| 11
(16)
= I + arg max peknn(x)
11,11
(17)
t p"
(18)
Thus:
smaxpool(D(r)) c Et +kn(()) D(r) 9.r, -D(r) -4,49" = D(r) G,tt
(19)
17

---

Published as a conference paper at ICLR 2026
B.3 PROOF OF PROPOSITION 4.3
Proof. Expanding the right-hand side of Equation gives:
(Drr) C,,,, t xlp' E + knn(z))
||+ -p'll D(r) c.p (20)
softmaxx+pEr--mmaa
D(r) aatt
|| - t -pll
(21)
softmaxpekmn(a)
D(r) cletp
(22)
x pl1
(r) softmaxpekinfe
|
CLe++
(23)
D(r) c,,+
(24)
Line 23 is due to Schur's lemma (Schur 1905), which proved that any linear combination of Fourier
coeffeiints is equivariant.
B.4 PROOF OF PROPOSITION 4.4
Proof. When | 0, the wingle-D matrix is an identity matrix, thus:
(DD(r) ca) + Bco + = c) D(r)
(25)
When 0, expanding the right-hand side of Equation and applying Schur's lemma (Schur 1905
we have:
oz (D(r) ax) = D(r) (ana)) = D(r)+ c
(26)
C ADDITIONAL BACKGROUND ON KEYFRAME IMITATION LEARNING AND MULTI-TASK MANIPULATION POLICY.
The keyframe action formulation (James & Davison, 2022 James et al. 2022) defines the setting where the policy predicts the next goal pose of the gnipper based on the current observation. motion planner then generates a collision-free trajectory to reach this predicted goal. This formulation decomposes complex trajectories into a sequence keyframe poses, thereby simplifying policy learning while preserving the ability to solve wide range of manipulation tasks. Building on this, keyframe imitation learning (Shridhar et al. 2023) formulates the problem as imitation learning, where the policy (o) learns predict the expert keyframe action given observation from expert demonstrations. Multi-task keyframe manipulation policies (Shridhar et al. 2023; Goyal et al.
2023 Gervet al., 2023) Goyal al. 2024 Ke et al.)) extend this formulation to support multiple
skills by conditioning the policy on natural language goals n, enabling task-specific behavior across a diverse of instructions.
D 18 RLBENCH TASKS WITH STANDARD AND SE(3) INITIALIZATIONS
The 18 RLBench tasks Shridhar et al. (2023) James et al. (200 are initialized with objects in random SE(2) poses. In this paper, present 18 RLBench tasks with SE(3) variation, where in addition to the SE(2) initialization, the pose objects are further perturbed with SO(3) transformation. This change will leads keyframe actions change in SE(3). For detailed SO(3) perturbation range and perturbed object, see Table
E EQUIVARIANCE ERROR
18

---

Published as a conference paper at ICLR 2026
Table 6: 18 Language-conditioned tasks in RLBench (James 2020 with SE(3) initializations.
Task
Variation Type Periuree Object
50(3) Perturbation (r. p))
Language Template
open Grewer
placement
drawer
[0.5)))),[00. (..6
"open the drawer"
alide block color
plane
[-0.12. -..12-
12), [0.12, 0.1]
"slide the block to target"
eweep dustpan.
holder
[0.0.-0.9]] (..00.)
"swoop ditt the dustpan"
meat off grill
calegory
grill table
[-0.55, -0.25). [0.25.0.25]
"take the off the grill"
turn top
placement
tap
[-0.5, -0.5). (0..
tap"
put in drawer
placement
drawer
[0. -0.2).0..2...2
"pur the item in the drawer"
close jaz
color
]az
[0. -0.5].... (..6
"close the jar"
érag stick
color
plane
0..12 -0.12,, 00.12, 0.12
"use the stick drag the cube onto the target" blocks
stack blocks
color, count
plane
-0.15, -0.150.15], (0.15, 0.15
"stack
screw bulb
lamp
-0.6,-0.66 0.66
"screw
- light bulb" "put the money away in the sule on the shell"
put in safe
placement
sase
-0.25, -0.3), [0.5.
place wine
placement
rack
[0.5, -0.5) (0.5.
"stack wine bottle the
put in cupboard
category
cupboard
-0.5, 0.5.
"pur the in the cupboard"
sort shape
shape
shape sortes
00255 -0.25). [0.25, 0.25]
"put the in the shape sorter"
push buttons
color
buttons
-0.25, -0.25), (0.25, 0.25
"push the bution, |iben the button]
insert peg
color
pillars
[-0.3, -0.4), 0.43
"put the ring on the spoke"
cups
cups
-0.3, 0.3.
"stuck other cups top of cup"
place cupa
count
cups
[0, 0.5), (0.6.0.5)
"place cups on the holder"
We empirically measure the degree of equivariance error in our model. This is defined as the geodesic distance d based on Equation 2:
Table 7: Empirical equivariance error, reported as geodesic distances (in me- ters for translation and radians for rota- tion). Identity setting measures the ran- domness of inference, and SE(3) pertur- bation measures equivariance error.
equ_error = d(T(g -, n., 9 n))
Here, rotates the point cloud and proprioceptive infor- mation in the observation and the predicted action. g is uniformly sampled from a translational range of 0.1 along the X. Y, and axis, and a rotational range of 60" along the roll and pitch axis, and full 360 around the yaw axis. We compared the equivariance error in EquAct with that in 3DDA (Ke et a..) Both models those used to produce the results in Table I and both are trained with 100 demonstrations. We focus on the "open_draa" "reen_drrer"" task because its action is uni-modal, which minimizes ambiguity from multi-modal action spaces and allows for clean measurement of equivariance error. Our results in Table iindicate that our model has lower equivariance error than 3DDA, empirically verifying AquAct is equivariant.
Model
Identity
SB(3) Perturbation
EquAct
0.09 rad, 0.004
0.8 rad, 0.038
3DDA
0.07 rad, 0.006
1.4 rad. 0.132
F DETAILS OF PHYSICAL TASKS
Our real-world experiments are carried out on URS robotic arm equipped with a Robotiq 2F-85 gripper and three Intel RealSense D455 cameras (front, left, and right cameras), as shown in Figure Keyframe actions are collected using a 6-DoF 3DConnexion SpaceMouse. We collect both visual observations (from all three cameras) and robot end-effector actions (position, orientation, and gripper states). During training and evaluation, we randomize the objects' SE5(3 (or SE(()) poses by physically rotating the claeet- the magic-arm camera mount-befor each rollout.
The 4 physical tasks are visualized in Figure 5 For details of these tasks, see descriptions below.
F.1 DISASSEMBLE PIPE
Task: Disassemble the required link of the pipe: first, second, all.
Number of keyframe actions: 3-10.
Variations: "first link", "second link","all"
Objects: Pipes consisting of five sections of water pipes.
Success Metric: The robot must accurately grasp the target pipe segment and completely remove from the intact assembly.
19

---

Published as a conference paper at ICLR 2026
Short horizon task
"pluck the top flower"
"pick the strawberry"
Long horizon task
"place the small roil"
"disassemble the link"
Figure 5: Physical tasks.
Werlepsee Realsenese camera
SE(3) clamp
Figure 6: Real world experimental setup
F.2 PLUCK FLOWERS
Task:
Pluck the specified flower: top, middle, bottom.
Number of keyframe actions: 4.
Variations: "top flower". "middle flower", "bottom flower".
Objects: Three artificial flowers and vase.
20

---

Published as a conference paper at ICLR 2026
Success Metric: The robot must accurately grab the designated flower and pluck it.
F.3
PICK FRUIT
Task: Pick up the specified tuil(sttawberryppcchhh...u.)
Number of keyframe actions: 3.
Variations: "strawberry", "peach", "plum".
Objects: Three fruits of mixed types.
Success Metric: The robot must correctly identify, grasp the target fruit.
F.4 INSTALL TOILET ROLL
Task: Place the specified toilet paper roll: large, small.
Number of keyframe actions: 5.
Variations: "large roll", "small roll".
Objects: Two toilet-paper rolls and one wall-mounted holder.
Success Metric: The robot must pick up the specified roll and mount onto the holder.
G HYPERPARAMETERS
We report the following hyperparameters in Table for EquAct well baselines we compared in the paper. The differences in training iterations across baselines are primarily due to variations in computational resources (number of GPUs), which in turn affect batch sizes. For instance, EquAct was trained on single GPU with batch size of 2, so it is trained with 8e5 iterations. whereas SAM2Act was trained on 32 GPUs with a batch size of 256, so its iteration is reduced to 5e4.
Table 8: Hyperparameters. sim: the hyperparameters used in simulation experiments. phy: the hyperparameters used in physical experiments.
Name of the hyperparameter
Method
EquAct (sim)
EquAct (phy)
3DDA (sim)
3DDA (phy) SAM2ACT (sim)
(train/test)
450/3000
450/6000
None
None
None
a, 2oanse2fine levels
None
None
None
a, (train/test)
36, 0..86/223399900
36,864/2.359
None
None
None
learning rate
Ic-4
Ic-4
lc-4
1c-4
1c4
scheduler
batch size
7
training iterations
8c5
1.2c5
5.625e4
21


Reviews:
[{"review_id": "fPIjEhVYGo", "reviewer": "Reviewer_cZ5N", "summary": "The paper proposes a novel approach for adapting visuomotor policies to be SE(3) equivariant to scene changes and SE(3) invaraiant to similar language commands. This is achieved by using an SE(3)-equivariant point transformer that adapts to changes in the observation space, while having an SE(3)-invariant FiLM layers to adapt (or consequently not adapt) to changes in the language commands. They authors show the improved performance of the proposed approach for learning a next-best keyframe manipulation policy in a multi-task imitation learning setting.", "questions": "In Lines 201-202, does the Q function take in a single action sample? Why does the gripper Q function take in only the translational action and not the rotational one as well? Moreover, how are the cross-entropy losses calculated?\nThe SO(3) Perturbation (r, p) is unclear in Table 4. are they the deviations along XYZ and roll pitch yaw? if so, should it not be SE(3) Perturbations in that case?", "limitations": "", "rating": 10, "confidence": 3, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "The proposed approach does a good job at ensuring SE(3) equivariance/invariance through different parts of the system, such as the equivariant transformer arhitechture for encoding the points, and the Field network. The proposed novelties are experimentally evaluated and show good performance especially under high SE(3) variability with just a low number of samples (10 demos) and scales well with more number of demonstrations.", "weaknesses": "The policy architecture and the corresponding action refinement/selection is unclear. How is the implicit action value function used to predict the actions? Is it by randomly sampling actions and evaluating them? Or is it by having a separate policy network trained in an actor-critic approach? \nTraining details about the implementation of the cross entropy losses are missing to make the paper more complete and to improve the reproducibility aspects of the paper.\nSome discussion on not using direct behavioural cloning instead of an implicit action-value function would benefit the paper.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "QANMaqAxho", "reviewer": "Reviewer_Nxid", "summary": "This paper proposes EquAct, an SE(3)-equivariant multi-task Transformer for open-loop robotic manipulation.\nThe method introduces explicit geometric inductive bias into 3D point cloud and language-conditioned policy learning, aiming to improve generalization and sample efficiency under spatial perturbations.\nThe model consists of three main components: (1) an SE(3)-equivariant Point Transformer U-Net for geometric encoding, (2) an Invariant Feature-wise Linear Modulation (iFiLM) layer for language conditioning, and (3) an Equivariant Field Network for action candidate evaluation.\nExperiments on 18 RLBench tasks and several real-robot setups demonstrate clear advantages over non-equivariant baselines under SE(3)/SE(2) perturbations. The paper also provides quantitative analyses of equivariance error.", "questions": "1. Could the authors include comparisons or analyses with other SE(3)-equivariant models such as [1][2]? This is not mandatory, but discussing why previous methods are limited to single-task settings while EquAct handles multi-task scenarios would clarify whether this advantage arises from the keyframe-based formulation rather than the equivariance design itself.\n2. When scaling to larger datasets, does the model still rely on explicit equivariance for good performance? Are there signs of saturation?\n3. In real-world conditions, camera viewpoint shifts and optical distortions are common. It would be valuable to include experiments testing robustness under such non-ideal equivariance scenarios.\nIf the authors can effectively address these concerns, I would consider raising the score.\nReferences\n[1] Tie, Chenrui, et al. “Et-seed: Efficient trajectory-level SE(3) equivariant diffusion policy.” arXiv preprint arXiv:2411.03990 (2024).\n[2] Qi, Yu, et al. “Two by two: Learning multi-task pairwise objects assembly for generalizable robot manipulation.” CVPR 2025.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "(1)The work systematically integrates explicit SE(3) equivariance into a multi-task vision-language manipulation Transformer with modular and theoretically grounded design.\n(2)The introduction of SE(3) initialization in RLBench contributes an independent and valuable benchmark for studying spatial generalization.\n(3)The method achieves strong performance under SE(3)/SE(2) perturbations, demonstrating the benefits of geometric inductive bias in low-data regimes.", "weaknesses": "(1)Lack of comparison with other equivariant models. The paper only compares with non-equivariant baselines, leaving unclear whether EquAct’s design offers advantages within the broader class of equivariant inductive biases.\n(2)It is unclear whether explicit SE(3) constraints remain beneficial when data scale or pretraining increases, or if the performance gains would saturate.\n(3)In real robotic systems, camera pose variations, viewpoint shifts, and lens distortions break strict SE(3) symmetry. The model’s robustness to such non-ideal conditions has not been examined.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "wXcrJMbxH3", "reviewer": "Reviewer_xNKT", "summary": "The paper proposes a SE(3)-equivariant multi-task policy network with several architectural designs: (i) an equivariant U-net, (ii) an invariant FiLM layer, and (iii) an equivariant field network. The equivariance is mathematically proven. The network shows good performance in both simulation and real-world experiments.", "questions": "- How are the query actions sampled when training the equivariant field network? Is it a uniform sampling? Is the sampling space too large: rotation + translation, in a large scene?\n- In Section 4, it says: \"During training, we also augment the dataset with respect to equation 1 by randomly rotating the\npoint cloud and the action simultaneously with [±5◦,±5◦,±45◦] rotation along [x, y, z] axis.\" How does it relate to the SE(3)/SE(2) randomizations in Section 5.1 Experiment settings (line 409-414)? Also, why are these numbers chosen?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- Multi-task learning with generalization is a much harder problem than per-task learning, while most prior works with equivariance could only tackle the latter.\n- The proposed network has both theoretical foundations to guarantee its equivariance and engineering designs to improve its performance and efficiency.\n- The experiments are well-designed and well-conducted. Especially some small but important details are well-studied in the experiments:\n\t- I appreciate that the authors also show the training/inference time and memory consumption for the proposed network, and it's good that the results match the baselines -- usually, equivariant networks can easily consume more time or memory as a cost for their \"for-free\" generalization to unseen poses\n\t- The robustness and empirical equivariance errors are tested.", "weaknesses": "- Some details, especially the experimental setups, can be made clearer. See the minor points and questions below.\n\n\nSome minor points:\n- Line 157: $x = \\{x_T, x_{\\text{open}\\}, x=\\{e, a\\}$ looks confusing at first glance -- I understood it after a while, but I think it can be expressed in a clearer way.\n- Table 1 is a bit hard to read at first, and it took me a while to understand its configuration: (i) The two small regions for the avg. success rate and time/memory consumptions, perhaps mark them in different background colors, so that they look more distinct from the per-task success rates. (ii) \"10* 10 100\" looks too abbreviated for training setups, maybe make it a bit more concrete, like \"SE(2)/10\" or something, and I think the table still has some space for that.\n- Perhaps say \"real-world experiments\" instead of \"physical tasks\"? It made me confused before reading Appendix F -- what are \"physical tasks\"? force control? physical parameter related? ...", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "BA4Ak8IOan", "reviewer": "Reviewer_UBsD", "summary": "This paper proposes a multi-task SE(3) equivaraint robot policies.  The model consists of (1) a SE(3)-equivariant encoder that transforms input point cloud into spherical Fourier features, followed by (2) SE(3)-invariant feature-wise linear modulation layer that facilitates conditioning on task instructions.  The proposed method sets a new state-of-the-art on RLBench simulation benchmark, while demonstrating strong generalization to spatial variety.", "questions": "1. What is the performance of EquAct on SimplerEnv or LIBERO or CALVIN simulation benchmark?\n2. How does the proposed method work in sim2real setup?\n\nI'm open to adjust my rating based on the author's rebuttal.", "limitations": "", "rating": 6, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The proposed method sets a new state-of-the-art on the challenging RLBench simulation benchmark.  In particular, it demonstrates superior performance on testing scenes where objects are randomly SE(3) initialized.\n2. The design choice of using SE(3)-equivariant point cloud encoder is techinically sound.  Its efficacy is also supported by the ablation study.\n3. This paper reports both training time, inference speed and required GPU mem, facilitating better understanding of the proposed method.\n4. The authors have released the code base, facilitating reproducibility of the proposed method.", "weaknesses": "1. This paper only evaluates the proposed method on one RLBench simulation benchmark and a real-world benchmark.  I'd highly encourage the authors to evaluate on other benchmarks, such as SimplerEnv [1] or LIBERO [2] or CALVIN [3], to further demonstrate the generality and effectiveness of the proposed method .\n2. It's widely known that 3DDA has poor sim2real generalization due to the visual appearance gap between the simulation and the real world.  This paper uses a SE(3)-equivariant point cloud encoder, and point cloud has stronger sim2real generalization than RGB image encoders.  Along with the SE(3)-equivariance, it'd be interesting to see if the proposed method trained in simulation can be directly deployed in the real world.  Such results would intrigue the community and attract more attention in SE(3)-equivariant policies.\n\n---\n\nReference\n\n[1] Li, Xuanlin, et al. \"Evaluating real-world robot manipulation policies in simulation.\" arXiv preprint arXiv:2405.05941 (2024).\n\n[2] Liu, Bo, et al. \"Libero: Benchmarking knowledge transfer for lifelong robot learning.\" Advances in Neural Information Processing Systems 36 (2023): 44776-44791.\n\n[3] Mees, Oier, et al. \"Calvin: A benchmark for language-conditioned policy learning for long-horizon robot manipulation tasks.\" IEEE Robotics and Automation Letters 7.3 (2022): 7327-7334.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)