Rodrigues Network for Learning Robot Actions
Authors: Jialiang Zhang
Haoran Geng
Yang You
Congyue Deng
Pieter Abbeel
Jitendra Malik
Leonidas Guibas
Venue: ICLR 2026 Oral (2026)
Keywords: Robot learning
Action understanding
Neural architecture
Primary area: applications to robotics, autonomy, planning
TL;DR: We design a new neural network, the Rodrigues Network (RodriNet), that addresses the kinematic structural priors in articulated robot action learning.

Abstract: Understanding and predicting articulated actions is important in robot learning. However, common architectures such as MLPs and Transformers lack inductive biases that reflect the underlying kinematic structure of articulated systems. To this end, we propose the **Neural Rodrigues Operator**, a learnable generalization of the classical forward kinematics operation, designed to inject kinematics-aware inductive bias into neural computation. Building on this operator, we design the **Rodrigues Network (RodriNet)**, a novel neural architecture specialized for processing actions. We evaluate the expressivity of our network on two synthetic tasks on kinematic and motion prediction, showing significant improvements compared to standard backbones. We further demonstrate its effectiveness in two realistic applications: (i) imitation learning on robotic benchmarks with the Diffusion Policy, and (ii) single-image 3D hand reconstruction. Our results suggest that integrating structured kinematic priors into the network architecture improves action learning in various domains.

Published as a conference paper at ICLR 2026
RODRIGUES NETWORK FOR LEARNING ROBOT ACTIONS
Jialiang Zhang* MIT. Stanford University
Haoran Geng" UC Berkeley
Yang You" Stanford University
Congyue Deng MIT, Stanford University
Pieter Abbeel UC Berkeley
Jitendra Malik
Leonidas Guibas
UC Berkeley
Stanford University
Classical Operator
Learnable Operator
Applications
Rodrigues' Rotation Formula for robot control
Neural Rodrigues Operator for robot leurning
Motion prediction
Rotation
Message passing
Imitation
sin cos 0
learning
Fixed coefficients
Joint state
Learnable weights
Joint
pose
features
estimation
Figure We introduce the Neural Rodrigues Operator, a learnable extension of the classical Rodrigues' Rotation Formula from robot control, where the original coefficients are replaced with trainable weights and joint angles are generalized to abstract features. Built upon this operator, the Rodrigues Network leverages the kinematic structure of articulated systems advance wide range of action-learning tasks.
ABSTRACT
Understanding and predicting articulated actions is important in robot learning. However, common architectures such as MLPs and Transformers lack inductive biases that reflect the underlying kinematic structure of articulated systems. To this end, we propose the Neural Rodrigues Operator, learnable generalization the classical forward kinematics operation, designed to inject kinematics- inductive bias into neural computation. Building this operator, design the Rodrigues Network (RodriNet), a novel neural architecture specialized for pro- cessing actions. We evaluate the expressivity of our network two synthetic tasks on kinematic and motion prediction, showing significant improvements compared standard backbones. We further demonstrate its effectiveness in two realis- tic applications: (i) imitation learning on robotic benchmarks with the Diffusion Policy, and (ii) single-image 3D hand reconstruction. Our results suggest that inte- grating structured kinematic priors into the network architecture improves action learning various domains.
INTRODUCTION
We study the problem of understanding and predicting the actions of articulated actors. Articulated actors (e.g. robots (Shaw et al.] 2023) and animated characters (Romero et al. 2022; Loper et al.
2023) systems that use multiple rotational joints connect moving links. Their actions, includ- ing poses, motions, or control commands, are usually represented as values associated with all joints.
Equual contribution
Work performed visiting student Stanford University.

---

Published as a conference paper at ICLR 2026
Learning with articulated actors usually involves predicting their actions while processing diverse sensory imputs. Such a problem lies a wide spectrum of intelligent systems, from whole-bod body controllers (Moro & Sentis 2019; Kuang et al. |2025 Geng et al., 20255, ) to dexterous grasp detec- tors (Duan et 2021 Xu et al. 2023; Wan et al. 2023 Zhang et al. 2024), to motion retargeting
networks (Aberman et 2020).
These action data are inherently structured by articulated kinematics. This motivates us to design a neural network that leverages this structure an inductive bias for better understanding and infer- ence of actions. However, general architectures, such as MLPs and Transformers (Vaswani et al.) 2017]. treat actions as unstructured tokens, ignoring the kinematic relationships between joints. Pre- vious works have exploited the connectivity of robot links through graph convolutions (Yan et al.) 2018 Ci et al., 2019; Aberman et al., 2020) or masked attention (Sferrazza et al. 2024). Yet, architectures that directly exploit the computational patterns of articulated kinematics remain under- explored. This gap raises our central question: Can we design a neural network for action learning that embeds articulated kinematics
inductive bias?
To answer this question, we draw an analogy to convolutional neural networks for images. Low-level 2D image features are spatially local and translation equivariant. Traditional pattern recognition exploits this structure using hand-crafted filters to detect edges (Canny) 1986) and corners (Harris ct al. 1988). CNNs (LeCun al., 2002) build on this idea by making filters learnable, adding non- linearities, and using high-dimensional channels. This creates a deep learning framework that learns high-level semantic features while preserving the structural properties of classical image filters.
In a similar vein, we identify Rodrigues' rotation formula as the fundamental operator in articulated forward kinematics and transform into learnable form, which we call the Neural Rodrigues Op- erator. Specifically, we separate the entries in the Rodrigues' rotation formula into state-dependent parameters conditioned on the joint angles, and inteeedndden coefficients that only rely on the articulated structure. We then convert it into neural operator by treating the sttenddeendent pa- rameters as input features, and relaxing the state-independent coefficients into optimizable weights. We further extend it into multi-channel operator that applies to higher-dimensions. features rather than just 1D joint angles.
With our neural operator as the key component, we construct complete network architecture, named the Rodrigues Network (RodriNet), for encoding, understanding, and predicting articu- lated actions. It comprises three key components: a Rodrigues Layer for passing information from joints to links, built up our neural operator; a Joint Layer for passing information from links to joints; and finally, Self-Attention Layer for global information exchange. We also introduce global token for processing other task variables, such as perception inputs. Although starting with an operator in classical robotic theory, we end up with deep neural network with modern designs while maintaining the structural bias in articulated kinematics, as illustrated in Figure [
We evaluate approach through three sets of experiments: First, we demonstrate the strong pressivity of our network in representing forward kinematics and motions through toy experiments. Second, showease our effectiveness realistic robot-learning scenarios with imitation learning on 5 robot manipulation tasks. Additionally, we show our network achieving state-of-the re- sults in human hand pose estimation from images, where the articulated actor is no longer a robot, highlighting its applications in computer vision and graphics.
2 RELATED WORK: ARTICULATION-AWARR ROBOT LEARNING
Articulated robots can be naturally modeled as graphs, making graph convolution (Bruna et al.
2013
Niepert et al., (016) a popular approach for processing articulated data. This has been widely applied in character animation for tasks like skeleton
action recognition (Yan et al. 20188 Cheng et al.) 2020 Song et al. 2022, Chen et al., 2021b), pose estimation (Ci et al. 2019 Choi et al. 2020 Zeng et al. 2021). and motion retargeting (Aberman al., 2020). Graph convolution effectively captures link connectivity and spatial locality, but it does not explicitly incorporate articulated kinematics, which is an essential aspect of articulated action understanding. In contrast, our work introduces a novel operator derived from forward kinematics, providing networks with ineematics-ifforma inductive bias.
2

---

Published as a conference paper at ICLR 2026
Transformer- based architectures (Vaswani et al. 2017) widely used in policy learning (Brohan et al. 2022 Zhao et al. 2023; Shridhar et al. 2023). and recent work has explored incorporating structural blas via graph-aware positional embeddings (Hong et al., (02T) masked attention (Ster- razza et al. ).24). However, these modifications do not fundamentally adapt the self-attention mechanism suit kinematics. We instead use standard self-attention layers for network capacity, but rely on our Kinematics-inspired operator for inductive bias.
Forward kinematics provides deterministic mapping from joint space to Cartesian space, but inte- grating it into neural networks remains non-trivial. Prior methods have inserted analytical forward kinematics as a differentiable layer in neural networks (Villegas al.] 2018) help them reason about the Cartesian results of the robot action. While this introduces kinematic awareness, it con- strains the model's s flexibility. Other methods apply Cartesian-spase loss functions after computing forward kinematics on network outputs (Pavllo et al.) [020, Jiing et al., 2021 Liu et al., 2020), but these approaches do not alter the network architecture and are thus orthogonal to our focus. con- trast, our method derive a learnable operator from forward kinematics, thereby making the network kinematics-aware while maintaining the flexibility to learn high-level features.
NEURAL RODRIGUES OPERATOR
In this section, we derive the Neural Rodrigues Operator by making the Rodrigues' rotation for- mula learnable and more generalized.
3.1 BACKGROUND
Articulated robots In this paper's context, articulated robot has loop-free kinematic tree structure, with D+1 rigid links Lo. Lo connected by D one-DoF revolute joints ".,...,")) This definition encompasses wide range of platforms such robotic arms, dexterous hands, quadrupeds, and humanoids. The kinematic tree has a root, denoted as the base link Lo. which can be either fixed or free-floating. All links and joints have their local 3D frames. Each joint Jj connects a parent link Lp, with child link Le,- where rotation axis W E IR3 in the joint frame defines the rotational motion of the child link relative to the parent link, as well as a fixed transformation from the parent link's frame the joint's frame T, E SEL()..
Given the kinematic structure, the configuration of the robot is then determined by the joint angles = [....... Ap] E RR,", as well as the root pose Pu SE(3) if its base link is free-floating (not required for fixed-base robots). In most classical control systems, control commands are sent to the joints, specifying their joint angles, velocities, or torques, depending on the control modes.
Forward kinematics To obtain the position and orientation of all links, including the end-effector, given a set of joint angles, we apply forward kinematics. Below, we briefly outline its computation using homogeneous coordinates. We represent the pose of link L, as homogeneous matrix P; describing the transformation from the world frame to the link's frame:
P.
R;
t; 1
R444
O1xs
where t, R is the position and R, R3x3 is the orientation. Given the pose of the base link Po. the poses of the descendant links can be computed recursively from parents to children. More concretely, joint Jj. using T برj 0j. can compute the child-link pose Pey from the parent- link pose Pp, through two transformations: (i) a fixed coordinate change T, from the parent frame to the joint frame; (ii) dynamic rotation R(W,, 02) 3 around axis بs of angle 9, in the joint frame. Putting these together, the parent-to-children pose transformation is:
Rey
tg
-
Rp, tp,
T, [R( 0axi
01x3
O1x3
01x3
Here, transformation (i) only depends on the robot's articulated structure and thus is fixed, and transformation (ii) is state-dependent with variable 0y. Therefore, we can abbreviate Equation 2 as Pe, = P p3 ((O,, 0,)). where Rz) E R4x4 is the homogeneous matrix of the rotation. Essentially, forward kinematics is a hierarchical composition of fixed coordinate transformations and dynamic rotations in the axis-angle representation with variables O,,...
3

---

Published as a conference paper at ICLR 2026
Rodrigues' rotation formula The Rodrigues' rotation formula tells how compute the rotation matrix R( 0) E R3x3 from the axis-angle representation:
R(w, 0) 13 + sin [|اا (1 cos 0) [ا]
(3)
where [wi] E R3x3 is the skew-symmetric cross-product matrix of the rotation axis w = (wix, Wy.
3.2 NEURAL RODRIGUES OPERATOR WITH LEARNABLE PARAMETERS
Observe from Equation 3 that every entry in R( 0) is essentially a linear combination of 1, cos and sin 0, with fixed coefficients determined by the rotation axis Therefore, every en- are linear combinations of 1, cos and sin with
try in P(W; U2) and thereby T, ( f3) constant coefficients defined by the iate-indeeendent parameters Ty, W3 of joint Jj. Thus, we can re re-parameterize Equation as:
Ps. Pr, (A, Bj cos + Cy sin
where A,
Cy E 42 are coefficient matrices that only depend on the robot's articulated struc- ture. Based this, we construct our Neural Rodrigues Operator for one single joint by replacing these fixed coefficients with learnable weights W bibias Woos Wsin E B4x4, resulting in:
peur (W + W cos θ Wrin sin Q)
(5)
where E is scaler feature of joint, Fin R24x4 and pour E R4x4 are the input and output link features, corresponding to the parent and child links.
This operator generalizes the classical transformation Equation When applied recursively, defines a hierarchical message passing along the robot's kinematic tree. On one hand, in the special case when = 0g and all parameters in Equation are identical to those in Equation[] this learnable operator degenerate to the forward kinematics of the robot. On the other hand, this generalization also provides a more expressive function space that can be potentially used to encode richer, high- level features beyond joint angles and link poses. These properties enable the operator to combine the inductive bias of kinematic awareness with representational flexibility, making it well-suited for robot learning tasks involving complex actions and motions governed by articulated structures.
3.3 MULTI-CHANNEL NEURAL RODRIGUES OPERATOR
Derived from the Rodrigues' rotation formula, the neural operator defined in Equation 5 only applies to ID joint features and 4 x link features. We now extend higher-dimensionall features for a single joint. Specifically, we extend the link features from 4 x 4 matrices to pin E C pout E CL,C,,C C. are the channel numbers. Accordingly, the learnable weights are extended to Whiis BCCxxC2 x4x4 and Woos Wisn e RROxxx xCyx4x4
multi-channel operator to learn
and similarily, the joint features θ RRC, Here
[Rxxx(xxxxxxx giving multi-channel extension of Equation
U[v,]] Whin [i,
[1 (os(O(e)) + W (1,j. sin((cc))
(6)
Foul إ
pin [iU where E
E 4
For better expressivity of the network, we further learn another conjugate Ü(i.j] following Equa- tion and extend Equation 7 to include both left and right multiplications:
pour Li]
Σ (F (UU 0 (-
(8)
6 together define Multi-Channel Neural Rodrigues Operator with learnable parame- ters W T bius W'sin W/eos (W)bbas (W) (W)eos) }, where W,, W are the parameters for learn- ing U, 0 respectively. We abbreviate Equation as Fooi = Rodrigues (Fin W*, θ) and will use in the following sections.
Equation
4

---

Published as a conference paper at ICLR 2026
Rodrigues Layer update link
Joint Layer
Self-attention Layer
update joint feature
exchange link feature
Layer
Rodhigues
Layer
Linear
Rodrigues
Noers
Figure 2: Rodrigues Block. It comprises three components: Rodrigues Layer for passing infor- mation from joints to links, constructed with our Multi- Multi-Channel Neural Rodrigues Operator; Joint Layer for passing information from links joints; and exchange with all the links and the global token.
Layer for global information
4 RODRIGUES NETWORK
Given the Rodrigues Operator, we are interested in building a complete neural network that lever- ages this operator while being versatile and expressive. To achieve that, propose basic building block called Rodrigues Block (Figure 2. which comprises the following three components: (1) a Rodrigues Layer for passing information from joints links, constructed with
Multi-Channel Neural Rodrigues Operator (Section [41);: (2) Joint Layer for passing information from links to joints (Section 4.2 (3) and Self-Artention Layer (Section 4.3) for global information exchange.
4.1 RODRIGUES LAYER
With the Multi-Channel Rodrigues Operator being the core component operating on single joint, we construct the Rodrigues Layer, which extends this operator the full tree structure of an ar- ticulated robot. We define the feature graph of an articulated robot the collection of all link and joint features: [Fi]_-o for links and {0,)
joints. The Rodrigues Layer maintains a set of Rodrigues Kernels {W7 J==1, one for each joint, and computes the output link features { from the input link features {E(") and joint features { For each joint Jj. retrieve its Ro- drigues Kernels Wj. its joint feature OJ. and the feature of its parent link Fp.. We then apply the Multi-Chammi Neural Rodrigues Operator (Equation X to compute the transformed feature:
D
Firans Rodrigues
(rer Wj. Oj))
This transformed feature is added to the child link Lej input feature F and normalized:
Four
LayerNorm
Fnin Firams
(10)
For the root link, its output feature is defined its layer-normalized input. This layer updates the link features and leaves the joint features unchanged.
4.2 JOINT LAYER
While the Rodrigues Layer updates the link features, we still need a learnable mechanism to update the joint features. The Joint Layer computes its output joint features {@rt)) from its own input joint features [Gin) and link features {PiN). For each joint Jj, retrieve the feature its child link F apply a joint-specific linear transformation, and add it to the joint's 's existing feature:
+ out Linear, (Flatten (F())) + +
(11)
The transformations Lineary
R C2 x4x4 - R C, independently learned for each joint, allowing the model to capture joint-specific information. This layer updates the joint features and leaves the link features unchanged.
5

---

Published as a conference paper at ICLR 2026
4.3 OTHER COMPONENTS AND OVERALL ARCHITECTURE
Self-attention layer While the Rodrigues Layer and Joint Layer leverage the spatial locality inher- ent in articulated structures, they restrict information flow consecutive links and joints. To enable direct communication across all links, we incorporate Self-attention Layer. In this layer, each link feature first projected into a token using a linear transformation. These tokens then interact through multi-head self-attention, allowing the model to aggregate information from all links regard- less of their spatial distance. The attended tokens are subsequently projected back to the link feature space, followed by residual addition and layer normalization. Joint features are left unchanged.
To capture and utilize global context, we optionally introduce a global token G. This token represents learnable global feature and only participates in self-attention alongside the link tokens. It is processed through its own projection, residual, and normalization steps, and joins the link tokens during multi-head self-attention. The global token enables the network to store and propagate task-wide information that is not tied to any specific joint or link. We optionally enable the global token in tasks that require predicting global outputs, such as base link pose estimation.
Global token
Overall architecture We combine the above three components into unified module called the Rodrigues Block. Each Rodrigues Block takes input the link features, joint features, and an optional global token. It sequentially applies the Rodrigues Layer, Joint Layer, and Self-attention Layer to produce updated link features, joint features, and global token, where each layer's outputs serves as the next layer's 's inputs. By stacking multiple such blocks sequentially, we construct the full Rodrigues Network (RodriNet), enabling deep, hierarchical reasoning over articulated structures. The overall architecture is drawn in Figure Refer to Section of the supplementary for details on computing the first-layer features and task-specific outputs.
EXPERIMENTS
We evaluate our Rodrigues Network on a set of different tasks, ranging from forward kinematics and motion prediction (Section imitation learning in robotics (Section 5.2) to hand pose estima- tion for vision and graphics (Section (.3). The experiments focus on two main questions on action learning: (i) Does the structural prior of articulated kinematics make the Rodrigues Network better at representing robot actions? (ii) Can this inductive bias improve the understanding and predic- tion of robot actions in realistic task scenarios? Additional studies on ablations and hyperparameter sensitivity are provided in the supplementary material.
5.1 TOY EXPERIMENTS ON KINEMATICS AND MOTION
We begin by studying whether our neural operators help networks better represent the motions and actions of articulated actors with two synthetic tasks: forward kinematics fitting and (Cartesian- space) motion prediction. These synthetic tasks provide clean environment to directly evaluate network expressivity without other factors.
Forward kinematics fitting As discussed in Section 3.1 forward kinematics maps the configura- tion of an articulated robot (including the root pose and joint angles) to the pose of each link. To effectively control robot, a neural network should, at minimum, be capable of learning this map- ping. To evaluate whether the Neural Rodrigues Operator possesses strong representational capacity for kinematic modeling, we construct specialized Rodrigues Network consisting solely of Ro- drigues Layers and compare it to other neural backbones in their ability to fit the forward kinematics of single robot. We conduct this experiment on the LEAP Hand (Shaw et al. 2003). a free-floating dexterous robotic hand with 16 joints and 17 links. The network takes as input the configuration (T, R, 0) and predicts the pose matrices for all 17 links. For model hyperparameters and training settings, please refer to Sections C.1 D. of the supplementary material.
As shown in Figure Ba the Rodrigues Network achieves significantly lower prediction error than competing architectures, indicating superior precision in modeling forward kinematics. Moreover, its training loss decreases much faster (Figure (3b), demonstrating better data efficiency. We further visualize the networks" predictions single robot configuration in Figure 4 Notably, MLP and

---

Published as a conference paper at ICLR 2026
Fitting forward kinematics: MSE vs, backbones
Fitting forward kinematics: MSE vs. Iterations
10
0:
3477944
MLP
Transforme
10-4
Bedeiguss
(Rodrag was Layer ooly)
MSB
MSE
10-4
20-5
10-4
0.
2.62+07
10-4
GCN
Transformer Rodrigues
(Rodrigues Layor oely)
10k
20k
30k
40k
50%
68k
70k
80k
9.K
100k
Backbones
Berations
(a) MSE vs. backbones.
(b) MSE vs. training iters.
Figure 3: Fitting forward kinematics with different network backbones (MSE.). The Rodrigues network achieves significantly lower error (left) with faster convergence during training (right).
MLP
GCN
BoT
Transformer
Rodrigues
Ground Truth
Figure 4: Visualization of forward kinematics prediction on an example configuration. Errors plotted each link with color scales, with darker colors indicating larger errors.
GCN baselines produce visible artifacts, and all four baseline methods accumulate substantial error near the fingertips. These findings suggest that fitting forward kinematics not a trivial task; requires modeling structured function with spatial and hierarchical dependencies. In contrast, network closely matches the ground truth across all links, with minimal error, indicating that it ef- fectively captures the underlying structure of the kinematic mapping. We attribute this performance to the inductive bias introduced by the Neural Rodrigues Operator. Its structural prior equips the network with an inherent advantage learning kinematic functions.
Motion prediction in Cartesian space Although fitting forward kinematics highlights the our operator's representation capacity for kinematics, the task itself has limited practical value. In real-world robot learning scenarios, neural backbones typically process observations in 3D Carte- sian space (e.g., point clouds) and output control commands as target joint angles. evaluate propose Cartesian motion prediction task that
network's ability to bridge these modalities, challenges the model to reason about structured motion in Cartesian space while expressing its pre- 6-DoF UR5 robotic arm. We hegin by randomly
dictions in joint angle space. This task sampling two end-effector poses in 3D space and interpolating between them (linearly for transla- tion and spherically for rotation) generate 16-frame Cartesian trajectory. Then, kinematics to convert this trajectory into corresponding joint configurations. The network is given the first frames of joint angles and tasked with predicting the remaining 8, also in joint space. Refer to Sections C.2 and D.2 of the supplementary for details on training, model parameter count control, and runtime comparisons.
inverse
As shown in Table I the Rodrigues Network achieves the lowest training loss and test errors when trained on pre-collected dataset of 10 trajectories. Notably, its test MSE is lower than the train MSE of all baseline models, indicating that the Rodrigues Network not only fits the data more effectively but also generalizes better without overfitting. Figure 5b further shows that our model consistently outperforms baselines across different training set sizes. In Figure 55 we visualize the end-effector trajectories predicted by each network for a single example; our model's trajectory aligns most closely with the ground truth. These results confirm that the Rodrigues Network is more effective at bridging Cartesian space and joint angle space than conventional architectures. Please refer Section C.2 of the supplementary for metric definitions.

---

Published as a conference paper at ICLR 2026
Table 1: Motion prediction in Cartesian space with trainset size- 10
Backbone
Errory (mm)
E (*)
Errorg (*)
MSE (1c-6)
Train MSE (1c-6)
MLP
3,49 +:033
0,46 #0.05
0.17
#0000
22.52 +0.95
12.47 +0.73
GCN
3,55 #0.44
0.48
#0.05
0.17
#0001
18.52
11.7
13.68 +1.87
BoT
2.92 1:0.29
0,46
1:004
0.15
1:001
15.72
#:121
13.04 #1.4
Transformer
2.89 1:045
0.41
1:006
0.14
1:0.01
12.86
+1.25
10.50 #1.21
Rodrigues
1,21
10.17
0.16
0.004
0.06 1:0.00
2.56
1:0.39
1.93 10.34
Iraams 7: ooserved
Frmmes 3- 15:00 peedict with models
10-3
Test error vs trainset size
ML.
GUN
BoT
TrunsCormer
12
15
20-4
Rodrnguees
Framee inces
Test MSR
MTPP
RON
bxT
Tracimme
ET
107
10
10
10
End-effecterr positias prodicted by different medde (sinealieed frem lep-dioe view and intepellttd)
Trainaet
(a) Trajectory visualization. We visualize the trajectories of the end- point (marked in red) predicted by each model from the top-down view, interpolated with B-spline curves.
(b) Testset performance (MSEL)
under different amounts of training data.
Figure 5: Results for motion prediction Cartesian space.
5.2 ROBOTIC MANIPULATION WITH IMITATION LEARNING
Next, we evaluate whether our method benefits realistic robotic applications. We integrate the Ro- drigues Network as backbone into the Diffusion Policy (Chi al. (023), one of the state-of-the-art imitation learning frameworks, and test on manipulation benchmark in simulation.
Benchmark We construct suite of five manipulation tasks from ManiSkill (Mu al. 2021 using a 7-DoF Franka arm with a 1-DoF Panda gripper, simulated in SAPIEN (Xiang et al. (2200) For each task, we collect 100-500 demonstration trajectories using motion planning, with each trajectory spanning 200 steps. During training, the network receives proprioception and object state input and outputs relative joint offsets used for PD control. Performance is measured by running 100 evaluation rollouts in simulation, and all models are trained with 5 random seeds to report the mean and standard deviation of success rates.
Methods We adopt Diffusion Policy (Chi et al. 2023) as our learning framework, using 2-frame observation history and predicting 16 future steps, of which 8 are executed during deployment. To isolate the impact of the neural backbone, we keep the outer framework and all other compo- nents fixed, modifying only the denoising network. Bascline architectures include the U-Net and Transformer designs from the original Diffusion Policy paper. Our method replaces these with the Rodrigues Network, which takes the current observation, denoising timestep. and a noisy action as inputs and predicts the corresponding action noise. The gripper output is handled via the global token. All models are tuned to have approximately 17 million parameters for a fair comparison. Implementation details are provided in Section D.3 of the supplementary material.
Results and analysis As shown in Table 2] Diffusion Policy (Chi et al., 2023) with the Rodrigues Network backbone achieves overall ofate-o--the-arr performance, demonstrating that our kinematics- inspired inductive bias improves imitation learning in realistic robotic tasks. In particular, we ob- serve substantial gains in PickCube and StackCube, and comparable or slightly better performance in PushCube, PeglnsertionSide, and PlugCharger. These results suggest that the benefits of the Ro- drigues Decoder are task-dependent. PushCube is relatively simple, leading all backbones to achieve near-perfect success. In contrast, PegInsertionSide and PlugCharger involve complex contact dy- namics, such as inserting a peg into a hole, where tactile or force feedback would be beneficial. However, these sensor inputs are provided in the environment, making the neural backbone less of a limiting factor. In summary, enhancing the backbone architecture with our neural operator can yield significant gains when the network is the performance bottleneck.
8

---

Published as a conference paper at ICLR 2026
Table 2: Baseline comparisons the imitation learning benchmark. Simulated success rate.
Method
PushCube PickCube
StackCube PegInsertionSide PlugCharger
Average
Transformer-DD
0.98 1.0.02
0.63
10.05
0.38 10002
0.18 1:0.05
0.04 10002
0.44
UNet-DP
1.00 1.000
0.85
10003
0.37
0.004
0.56
1:0.06
0.13 1.006
0.58
Rodrigues-DP
1.00 +0000
0.94
0.00
0.44 0000
0.58 2:004
0.10 #0002
0.61
Table Baseline comparisons on the FreiHAND dataset. We use the standard protocol and report metrics 3D joint and 3D mesh accuracy. PA-MPVPE and PA-MPIPE numbers are in mm.
Method
PA-MPJPE 4
PA-MPVPE 4
F@5
F@ 15
12L-MeshNet (Moon & Lee
2020)
7.4
7.6
0.681
0,973
Pose2Mesh (Chor et al. 2020)
7.7
7.8
0.674
0.969
12UV-HandNer (Chen et al.
2021a
6.7
6.9
0.707
0.977
METRO (Lin et al. [202
6.5
6.3
0.731
0.984
Tang et al. (Tang et al. 202
6.7
6.7
0.724
0.981
Mesh Graphormer Lin et al.
2021b
5.9
6.0
0.764
0.986
MobRecon (Chen et al.
2022
5.7
5,8
0.784
0,986
AMVUR (Jiang al.
2023
6.2
6.1
0.767
0.987
HaMeR (Pavlakos al.
2024
6.0
5.7
0.785
0.990
HaMeR (Reproduced)
6.2
5.9
0.774
0.989
Ours
5.9
5.6
0.793
0.991
5.3 3D HAND RECONSTRUCTION
Finally, we show that Rodrigues Networks extend beyond robots to animated characters. Specifi- cally, we apply our method to 3D hand reconstruction from single-view RGB images, which involves predicting the rotations and positions of hand joints based the kinematic structure defined by the MANO (Romero et al. 2022 hand kinematics.
Our network builds upon HaMeR (Pavlakos et al. 2024 by replacing its vanilla transformer with the proposed Rodrigues Network (with modifications to suit MANO's configuration representation). Additionally, we introduce cross-attention layer following the self-attention layer to enable inter- actions between joint and link features and the input image tokens. Our network achieves a notable performance improvement while significantly reducing the number of parameters (39.5M vs. ours: 10.7M). Further architectural details are provided in Section of the supplementary material.
Table 3 presents the quantitative results. We follow the evaluation protocol and metrics established by HaMeR (Pavlakos et al., 2024), and report performance on the FreiHand ZZimmermann et al., 2019 dataset. For reference, results of our reproduced HaMeR model are also included.
As shown in Table our method achieves superior quantitative performance, surpassing the previ- ous sfate-of-the-art Compared the strongest baseline, HaMeR, our approach outperforms both the results reported in the original paper and our reproduced implementation. This underscores the effectiveness of incorporating hand joint kinematics into the network, resulting in substantial improvements kinematics-related tasks. Therefore, approach is not limited to robotic appli- cations, demonstrating its versatility and applicability to graphics-relate tasks as well.
CONCLUSIONS AND DISCUSSIONS
In this work, we design a neural network that addresses the kinematic struetural priors in articulated robot action learning. Core to it is the Neural Rodrigues Operator that extends the Rodrigues' rota- tion formula into a learnable operator of more flexible forms, providing networks with an inductive bias that better model kinematics-related features. With this neural operator as key compo- nent, we build the Rodrigues Network with additional layer designs, resulting in a powerful and embodiment- aware network architecture applicable diverse action-learning tasks.
Many state-of-ft of-the-art networks in robot learning build upon architectural designs originally devel- oped for other domains, such vision and language. With this work, aim encourage the

---

Published as a conference paper at ICLR 2026
exploration of action-centric neural network architectures that tailored to the unique characteris- tics of robot learning, particularly those aspects that are underexplored in other fields.
Limitations and future work First, while Rodrigues Networks successfully incorporate articu- lated kinematics as an inductive bias, they do not yet account for the geometry of individual links. In many settings, this information is available and could improve performance on tasks that require precise contact reasoning. Second, the current Neural Rodrigues Operator is restricted to rotational joints. Extending also handle translational joints would broaden its applicability. Third, our robot learning experiments focused on imitation learning. Exploring reinforcement learning (RI.) scenarios could further test the network's 's generality and effectiveness in closed-lood loop settings.
7 ACKNOWLEDGEMENTS
Leonidas Guibas, Yang You and Congyue Deng acknowledge support from the Toyota Research Institute University 2.0 Program, ARL grant 1911NNF211220111 Vannevar Bush Faculty Fellow- ship, and gift from the Flexiv corporation. Haoran Geng acknowledges support from the Berkeley Fellowship Award and the Qualcomm Innovation Fellowship. Yang You is also supported in part by the Outstanding Doctoral Graduates Development Scholarship of Shanghai Jiao Tong Univer- sity. Congyue Deng is also supported in part by the Tayebati Postdoctoral Fellowship the MIT Schwarzman College of Computing. Pieter Abbeel holds concurrent appointments as professor at UC Berkeley and as an Amazon Scholar, and this paper is not associated with Amazon.
REFERENCES
Kfir Aberman, Peizhuo Li, Dani Lischinski, Olga Sorkine-Hornun Daniel Cohen-Or, and Baoquan Chen. Skeleton-aware networks for deep motion retargeting. ACM Transactions Graphics (TOG), 39(4):62-1, 2020.
Anthony Brohan, Noah Brown, Justice Carbajal, Yevgen Chebotar, Joseph Dabis, Chelsea Finn, Keerthana Gopalakrishnan, Karol Hausman, Alex Herzog, Jasmine Hsu, al. Rt-1: Robotics transformer for real-world control at scale. arXiv preprint arXiv:2212.068)) 2022.
Joan Bruna, Wojciech Zaremba, Arthur Szlam, and Yann LeCun. Spectral networks and locally connected networks on graphs. arXiv preprint arXiv: 1312.62333,, 2013.
John Canny. A computational approach to edge detection. IEEE Transactions on pattern analysis and machine intelligence, (6):679-698, 1986.
Yu-Wei Chao, Wei Yang, Yu Xiang, Pavlo Molchanov, Ankur Handa, Jonathan Tremblay, Yashraj Narang, Karl Van Wyk, Umar Iqbal, Stan Birchfield, et al. Dexycb: A benchmark for capturing hand grasping objects. In Proceedings of the IEEF/CVE conference on computer vision and pattern recognition, pp. 9044-9053, 2021.
Ping Chen, Yujin Chen, Dong Yang, Fangyin Wu, Qin Li, Qingpei Xia, and Yong Tan. 12uv- handnet: Image-to-uv prediction network for accurate and high-fidelity 3d hand mesh modeling. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 12929-12938, 2021a.
Xingyu Chen, Yufeng Liu, Yajiao Dong, Xiong Zhang, Chongyang Ma, Yanmin Xiong, Yuan Zhang, and Xiaoyan Guo. Mobrecon: Mobile Mobile-friendly hand mesh reconstruction from monocular image. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pp- 2022.
Yuxin Chen, Ziqi Zhang, Chunfeng Yuan, Bing Li, Ying Deng, and Weiming Hu. Channel-wise topology refinement graph convolution for skeleton-bre based action recognition. In Proceedings of the IEEF/CVF international conference on computer vision, pp. 13359-13366 2021b.
Ke Cheng, Yifan Zhang, Xiangyu He, Weihan Chen, Jian Cheng, and Hanqing Lu. Skeleton-based action recognition with shift graph convolutional network. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 183-192, 2020.
10

---

Published as a conference paper at ICLR 2026
Cheng Chi, Zhenjia Xu, Siyuan Feng, Eric Cousineau, Yilun Du, Benjamin Burchfiel, Russ Tedrake, and Shuran Song. Diffusion policy: Visuomotor policy learning via action diffusion. The Inter- national Journal of Robotics Research, pp. 02783649241273668. 2023.
Hongsuk Choi, Gyeongsik Moon, and Kyoung Mu Lee. Pose2mesh: Graph convolutional network for 3d human pose and mesh recovery from 2d human pose. In Computer Vision-ECCV 2020: 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part VII 16, pp- 769-787. Springer, 2020.
Hai Ci, Chunyu Wang, Xiaoxuan Ma, and Yizhou Wang. Optimizing network structure for 3d human pose estimation. In Proceedings of the IEEF/CVF international conference on computer vision, pp. 2262-2271, 2019.
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, al. An image is worth 16x words: Transformers for image recognition at scale. arXiv preprint arXiv:2010. 1929, 2020.
Haonan Duan, Peng Wang, Yayu Huang, Guangyun Xu, Wei Wei, and Xiaofei Shen. Robotics dex- terous grasping: The methods based on point cloud and deep learning. Frontiers in Neurorobotics, 15:658280, 2021.
Hao-Shu Fang, Jiefeng Li, Hongyang Tang, Chao Xu, Haoyi Zhu, Yuliang Xiu, Yong-Lu Li, and Cewu Lu. Alphapose: Whole-body regional multi-person pose estimation and tracking in real- time. IEEE transactions on pattern analysis and machine intelligence, 45(6):715777173, 2022.
Haoran Geng, Feishi Wang. Songlin Wei, Yuyang Li, Bangjun Wang. Boshi An, Charlic Tianyue Cheng, Haozhe Lou, Peihao Li, Yen-Jen Wang. Yutong Liang, Dylan Goetting, Chaoyi Xu, Haozhe Chen, Yuxi Qian, Yiran Geng, Jiageng Mao, Weikang Wan, Mingtong Zhang, Jian- gran Lyu, Siheng Zhao, Jiazhao Zhang, Jialiang Zhang, Chengyang Zhao, Haoran Lu, Yufei Ding, Ran Gong, Yuran Wang, Yuxuan Kuang, Ruihai Wu, Baoxiong Jia, Carlo Sferrazza, Hao Dong, Siyuan Huang, Yue Wang, Jitendra Malik, and Pieter Abbeel. Roboverse: Towards uni- fied platform, dataset and benchmark for scalable and generalizable robot learning, 2025. URL bttps:///ariic org/ abs 2504. 18904
Shreyas Hampali, Mahdi Rad, Markus Oberweger, and Vincent Lepetit. Honnotate: A method for 3d annotation of hand and object poses. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 3196-3206, 2020.
Chris Harris, Mike Stephens, al. A combined comer and edge detector. In Alvey vision conference, volume 15, pp. 10-5244. Manchester, UK, 1988.
Sunghoon Hong, Deunsol Yoon, and Kee-Eung Kim. Structure-aware transformer policy for inho- mogeneous multi-task reinforcement learning. In International Conference on Learning Repre- sentations, 2021.
Hanwen Jiang, Shaowei Liu, Jiashun Wang, and Xiaolong Wang. Hand-object contact consistency reasoning for human grasps generation. In Proceedings the IEEE/CVF international conference on computer vision, pp. 11107-11116, 2021.
Zheheng Jiang, Hossein Rahmani, Sue Black, and Bryan M Williams. A probabilistic attention model with ocelusion-aware texture regression for 3d hand reconstruction from single rgb im- age. In Proceedings the 758-767, 2023.
the IEEE/CVF conference on computer vision and pattern recognition, pp.
Sheng Jin, Lumin Xu, Jin Xu, Can Wang, Wentao Liu, Chen Qian, Wanli Ouyang, and Ping Luo. Whole-body human pose estimation in the wild. In European Conference on Computer Vision, pp. 196-214. Springer, 2020.
Yuxuan Kuang, Haoran Geng, Amine Elhafsi, Tan-Dzung Do, Pieter Abbeel, Jitendra Malik, Marco Pavone, and Yue Wang. Skillblender: Towards versatile humanoid whole-body. body manipulaunl via skill blending. arXiv preprint arXiv:2506.09366, 2025.

---

Published as a conference paper at ICLR 2026
Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. Gradient- based learning applied to document recognition. Proceedings of the IEEE, 86(11):2278-23 2324, 2002.
Kevin Lin, Lijuan Wang. and Zicheng Liu. End-to-end human pose and mesh reconstruction with transformers. In Proceedings of the IEEE/CVF conference computer vision and pattern recog- nition, pp. 1954-1963, 2021a.
Kevin Lin, Lijuan Wang, and Zicheng Liu. Mesh graphormer. Proceedings of the IEEF/CVF international conference on computer vision, pp- 19939-12948, 2021b.
Min Liu, Zherong Pan, Kai Xu, Kanishka Ganguly, and Dinesh Manocha. Deep differentiable grasp planner for high-dof grippers. arXiv preprint arXiv:2002.015: 01500, 2020.
Matthew Loper, Naureen Mahmood, Javier Romero, Gerard Pons-Moll, and Michael J Black. Smpl: A skinned multi-person linear model. In Seminal Graphics Papers: Pushing the Boundaries, Volume 2, pp. 851-866. 2023.
Gyeongsik Moon and Kyoung Mu Lee. 121-meshnet: Image-to-lixel prediction network for accurate 3d human pose and mesh estimation from single rgb image. In Computer Vision-ECCH 2020: 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part VII 16, pp. 752-768. Springer, 2020.
Gyeongsik Moon, Shoou-I Yu, He Wen, Takaaki Shiratori, and Kyoung Mu Lee, Interhand2. 6m: A dataset and baseline for 3d interacting hand pose estimation from a single rgb image. Computer Vision-ECCV 2020: 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part XX 16, pp. 548-564 Springer, 2020.
Federico L Moro and Luis Sentis. Whole-body control of humanoid robots. Humanoid robotics: reference, pp. 1161-1188 2019.
Tongzhou Mu, Zhan Ling, Fanbo Xiang, Derek Yang, Xuanlin Li, Stone Tao, Zhiao Huang, Zhi- wei Jia, and Hao Su. Maniskill: Generalizable manipulation skill benchmark with large-scale demonstrations. arXiv preprint arXiv:2107. 14483, 2021.
Mathias Niepert, Mohamed Ahmed, and Konstantin Kutzkov. Learning convolutional neural net- works for graphs. In International conference on machine learning, pp. 2014-2023. PMLR, 2016.
Georgios Pavlakos, Dandan Shan, Ilija Radosavovic, Angjoo Kanazawa, David Fouhey, and Jitendra Malik. Reconstructing hands in 3d with transformers. In Proceedings of the IEEE/CVF Confer- on Computer Vision and Pattern Recognition, pp. 9826-9836 2024.
Dario Pavllo, Christoph Feichtenhofer, Michael Auli, and David Grangier. Modeling human motion with quaternion-base neural networks. International Journal of Computer Vision, 128:855-872,
2020.
Javier Romero, Dimitrios Tzionas, and Michael J Black. Embodied hands: Modeling and capturing hands and bodies together. arXiv preprint arXiv:2201.026/1 2022.
Carmelo Sferrazza, Dun-Ming Huang, Fangchen Liu, Jongmin Lee, and Pieter Abbeel. Body trans- former: Leveraging robot embodiment for policy learning. arXiv preprint MXXiv:2408.06316 2024.
Kenneth Shaw, Ananye Agarwal, and Deepak Pathak. Leap hand: Low-cost, efficient, and anthro- pomorphic hand for robot learning, arXiv preprint 2rXiv:2309.064400, 2023.
Mohit Shridhar, Lucas Manuelli, and Dieter Fox. Perceiver-actor: A multi-task transformer for robotic manipulation. In Conference on Robot Learning, pp. 785-799. PMLR, 2023.
Tomas Simon, Hanbyul Joo, lain Matthews, and Yaser Sheikh. Hand keypoint detection in single images using multiview bootstrapping. In Proceedings of the IEEE conference on Computer Vision and Pattern Recognition, pp. 1145-1153, 2017.
Yi-Fan Song, Zhang Zhang, Caifeng Shan, and Liang Wang. Constructing stronger and faster base- lines for skeleton- based action recognition. IEEE transactions on pattern analysis and machine intelligence, 45(2): 14::144444445 1488, 2022.
12

---

Published as a conference paper at ICLR 2026
Xiao Tang, Tianyu Wang. and Chi-Cinn Wing Fu. Towards accurate alignment in real-time 3d hand- reconstruction. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 1117169881777,, 2021.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural informa- tion processing systems. 30, 2017.
Ruben Villegas, Jimei Yang, Duygu Ceylan, and Honglak Lee. Neural kinematic networks for unsupervised motion retargetting. In Proceedings of the IEEE conference computer vision and pattern recognition, pp. 8639-8648, 2018.
Weikang Wan, Haoran Geng, Yun Liu, Zikang Shan, Yaodong Yang. Li Yi, and He Wang. Unidex- grasp++: Improving dexterous grasping policy learning via geometry-aware curriculum and iter- ative generalist-specialist learning. arXiv preprint arXiv:2304.00400 00464, 2023.
Donglai Xiang, Hanbyul Joo, and Yaser Sheikh. Monocular total capture: Posing face, body, and hands in the wild. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 10965-10974 2019.
Fanbo Xiang, Yuzhe Qin, Kaichun Mo. Yikuan Xia, Hao Zhu, Fangchen Liu, Minghua Liu, Hanxiao Jiang. Yifu Yuan, He Wang, et al. Sapien: A simulated part-based interactive environment. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 11097- 11107, 2020.
Yinzhen Xu, Weikang Wan, Jialiang Zhang, Haoran Liu, Zikang Shan, Hao Shen, Ruicheng Wang, Haoran Geng, Yijia Weng, Jiayi Chen, et al. Unidexgrasp: Universal robotic dexterous grasping via learning diverse proposal generation and goal-conditioned policy. arXiv preprint arXiv:2303.0099 00938, 2023.
Sijie Yan, Yuanjun Xiong, and Dahua Lin. Spatial temporal graph convolutional networks for skeleton-based action recognition. In Proceedings of the AAAI conference artificial intelli- gence, volume 32, 2018
Ailing Zeng, Xiao Sun, Lei Yang, Nanxuan Zhao, Minhao Liu, and Qiang Xu. Learning skeletal graph neural networks for hard 3d pose estimation. Proceedings of the IEEE/CVF international conference on computer vision, pp. 11146-11444 1444, 2021.
Jialiang Zhang, Haoran Liu, Danshi Li, Xinqiang Yu, Haoran Geng, Yufei Ding, Jiayi Chen, and He Wang. Dexgraspnet 2.0: Learning generative dexterous grasping large-scale synthetic clut- 23004
tered scenes, 2024. BtRLhttps: http:///axvvvv.v / http///aww..ogabb///aaaaa//
Tony Z Zhao, Vikash Kumar, Sergey Levine, and Chelsea Finn. Learning fine-graine grained bimanual manipulation with low-cost hardware. arXiv preprint arXiv:2304. 13705, 2023.
Christian Zimmermann and Thomas Brox. Learning to estimate 3d hand pose from single rgb images. In Proceedings of the IEEE international conference on computer vision, pp. 4903-4911, 2017.
Christian Zimmermann, Duygu Ceylan, Jimei Yang, Bryan Russell, Max Argus, and Thomas Brox. Freihand: dataset for markerless capture of hand pose and shape from single rgb images. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 813-822, 2019.
13

---

Published as a conference paper at ICLR 2026
Supplementary Material
This supplementary material provides additional details to support the experiments presented the main paper. Section describes how adapt the Neural Rodrigues Operator for animated char- acters, as used in the 3D hand reconstruction experiment. Section B provides additional details network architecture. Section [ outlines experimental settings, including data collection, training procedures, evaluation protocols, and all training hyperparameters. Section D details the architecture and configurations of both baseline models and our proposed method across all exper- iments. These details are provided to facilitate reproducibility of our results. We will release our code, datasets, and pretrained checkpoints upon acceptance further support the community in reproducing and building upon our work.
Additionally, Section presents further experiments designed to investigate the properties of our model and provide additional support for the claims made in the main paper. Finally, Section F describes our CUDA implementation of the Multi-Chanme Rodrigues Operator, which accelerates training and inference.
A RODRIGUES NETWORK FOR ANIMATED CHARACTERS
Beyond articulated robots, animated characters such as MANO (Romero al. 2022) and SMPI. (Loper et al., (023) also exhibit articulated structures and employ forward kinematics, which can similarly serve as inductive biases for neural networks. However, fundamental distinction ex- ists between these models and articulated robots: each joint in MANO and SMPL is free rotational joint with three degrees of freedom, as opposed to the single-axis, I-DoF rotational joints typically found in robots. As result, adapting our approach to animated characters requires fundamental modification of the Rodrigues Operator to accommodate unconstrained 3-DoF rotations. We need to identify the basic operation in forward kinematics for animated characters, and develop neural operator by turning that operation learnable.
Forward kinematics At each 3-DoF joint, we can parameterize configuration as a unit quater- nion q (gw: Ks qz- 9t) where |/|ll acters is similar to articulated robots. There are also two transformations at each joint Jj: (i) fixed coordinate change T, from the parent link frame to the joint frame; (ii) dynamic rotation Ragmar 12x3 in the joint frame. The parent-to-children pose transformation can also be writ-
Following this, forward kinematics for animated char-
ten as:
Rey
tey
R 01x3
tp 1
R Rymma (q)
03x1
(12)
01x3
01x3
0
Forward kinematics for animated characters is essentially hierarchical combination of Equation 12
Quaternions to matrix conversion mated characters lies in converting a joint's unit quaternions into a 3 by rotation matrix. This can be done using the following formula:
The key difference of computing forward kinematics for ani-
1 2(g g
2(q:4 9.
2 9;9w)
R Ra2mat (q)
2(q:9; + )
1 2(g + g
2 9:9w)
(13)
2(g:9k ()
2(g,y: qdpw)
- 2( + of)
Neural Rodrigues Operator for quaternions Observe from Equation every entry Ragmar (q)
a linear combination of ones and all 10 quadratic terms of qwo 2i qj Qk. with constant coefficients. Thus, we can mimic what we have done for the Rodrigues' rotation formula and re-parameterize
Equation 12 as:
P.j
Pp5
A
Σ
Anyla
Σ
Ana
(14)
are (w...j.A)
where A, Ary. Agr E 4x4 are coefficient matrices that only depend on the joint's structure. Similar what have done for axis angles, construct our Neural Rodrigues Operator for
14

---

Published as a conference paper at ICLR 2026
quaternions by replacing these fixed coefficient with learnable weights Whis Way, W R resulting in:
Fon pin Whiss
Σ
In
Σ War
(15)
3.96 (ww.g..)).arfff
where pin E IR1x4 and Foout E IRxx4 are the input and output link features, corresponding to the parent and child links. Similar to the original Neural Rodrigues Operator, we also extend this operator to support multi link feature channels and left multiliccation. But since there are already four feature channels for each joint, we do not extend further. We also add learnable weights W E I444 to Equation 15 to support representing linear terms of qw, Qs Qj. -k-
B ADDITIONAL DETAILS ON NETWORK ARCHITECTURE
This section describes how task observations are mapped to the Rodrigues Network's 's inputs and how the network's outputs are mapped to task actions.
Input embeddings In all experiments except 3D hand reconstruction, task inputs are low- dimensional vectors. We use a linear input embedding layer to map the input observation feature the Rodrigues Network's input. Specifically, given an observation feature of dimension dabs- we apply separate linear transformations to produce the initial link features, joint features, and, if applicable, global token. These serve as inputs to the first Rodrigues Block of the Rodrigues Network. After processing through the stacked Rodrigues Blocks, we obtain output features for all links, joints, and the global token.
For 3D hand reconstruction, the inputs are RGB images. We first apply a VIT encoder to transform the image into visual tokens, then insert cross-attention layer into each Rodrigues Block so that link features and the global token can attend to these tokens. As for the input to the first Rodrigues Block in this case, learnable embeddings are used.
Output heads For joint-level outputs, each joint's final output feature is concatenated with the output feature of its child link, then passed through a joint-specific linear transformation to produce the final output. For global outputs (those associated with a specific joint), we apply separate linear transformation to the final global token.
C EXPERIMENT SETTINGS
C.I FORWARD KINEMATICS FITTING
Data preparation In this experiment, the models learn to fit the forward kinematics mapping of robot. Therefore, we do not need to generate a fixed-size training set. However, we do create fixed-size validation set for selecting the best checkpoint, and separate test set for evaluation. We conduct this experiment the LEAP hand, which is fully-actuated robotic hand with four fingers, 16 rotational joints, and 17 links. Each data point includes the input: the root link position e R the root link orientation matrix R E 3x3 and joint angles 0 R16 The output is the position vectors and orientation matrices of all 17 links. We sample uniformly in [-0.05 em, 0.05 cm]3, R uniformly from SO(3), and 0 uniformly within the joint limits. We then compute all link poses using forward kinematics. The validation and test sets each contain 10,000 input-output pairs, generated using different random seeds.
Training details We use the Adam optimizer with Ir = 0.0003 and without weight decay and train each model for 10,000 steps. each training step, we generate new batch of 1024 input- output pairs using the same method for the validation and test sets, instead of sampling from fixed-size training set. The model takes the input and predicts 17 position vectors and 17 orientation matrices, totaling 17 (3 9) values. We compute the mean squared error (MSE) loss between the predicted and ground truth outputs. The loss is averaged over the batch and all output values, and then used to update the model through gradient descent. We evaluate the model on the validation
15

---

Published as a conference paper at ICLR 2026
set for every 500 steps, and pick the model with the lowest validation loss for final evaluation. We also save a checkpoint every 1,000 iterations for plotting the training curve. All hyperparameters are summarized in Table 4
Table 4: Training hyperparameters for forward kinematics fitting experiment.
Parameter name
Value
Parameter name
Value
Parameter
Value
Training iterations
100,000
Optimizer Validate every
Adam
Learning rate Weight decay
0.0003
Batch size
1024
500 iterations
C.2 MOTION PREDICTION IN CARTESIAN SPACE
Data preparation This experiment is conducted on fixed-base 6-DoF UR5 robotic arm. To make the inverse kinematics solution unique for all reachable end-effector poses, we limit the six joint ranges to [0, /2), [- 1/2,0, 0). [0, x x/2]. [0, x //4]. [0, π //4], and [0, //4.. We sample two joint configurations 0um. Pnd E R6 within these limits, which serve as the start and end poses. We then use forward kinematics to compute the corresponding 3D end-effector poses Psnn: Pend E SE(3). Next, we interpolate 16- 2 intermediate 3D poses between the start and end to form a trajectory of 16 poses: P1. P1s- Positions are interpolated linearly, and orientations are interpolated using spherical linear interpolation (slerp). Finally, joint configurations 0, G1e for the 3D pose trajectory. The first 8 frames of joint configurations used as input, and the models are asked to predict the last 8 frames. In Cartesian space, the motion pattern is clear: positions follow a straight line, and orientations change smoothly with slerp. However, this pattern is not obvious in joint space. The model needs to learn to reason about this motion pattern from the joint configurations. We generated four training sets with 10a, 10 10", and 10 input-output pairs. The validation and test sets each contain 10 input-output pairs.
apply inverse kinematics to get the corresponding
Training details We use the Adam optimizer with learning rate of 0.0001 and no weight decay, and train each model for 10,000 steps. Each training session uses a fixed training set. In each step, we randomly sample a batch of 1,024 input-output pairs from this set. The model takes the input and predicts compute the mean squared error (MSH) loss between the predicted and ground truth outputs, and use to perform gradient descent. We evaluate the model on the validation set every 500 steps, and select the checkpoint with the lowest validation loss for final testing. Unlike the forward kinematics fitting experiment, this experiment trains models on a fixed-size training set and evaluates them on separate test set drawn from the same distribution. Therefore, it evaluates not only the model's s ability to fit the training data, but also its generalization performance. All training hyperparameters listed in 5
values, which represent the joint configurations for the last frames. We
Table 5: Training hyperparameters for motion prediction in Cartesian space experiment.
Parameter name
Value
Parameter name
Value
Parameter
Value
Training iterations
100,000
Optimizer Validate every
Adam 500 iterations
Learning rate
0.0001
Batch size
1024
Weight decay
Input frames
8
Output frames
8
DoFs
6
Evaluation We apply forward kinematics to the predicted joint configurations to obtain the pre- dicted end-effector 3D poses. Then, we compare both the predicted joint configurations and the predicted end-effector poses with the ground truth. We report the following evaluation metrics: 1) Errorr (mm): End-effector position error on the test set, averaged over the 8 predicted frames. 2) Errorn ("): End-effector orientation error on the test set, averaged over the 8 predicted frames. We compute the relative rotation between the predicted and ground truth orientations, convert it to axis- angle representation, and take the angle. 3) Error, (*): Absolute joint configuration error on the test set, averaged over the 8 predicted frames and 6 joints. 4) MSE ( (x10-9g -9: Mean squared error on the test set, multiplied by 10 for better readability. 5) Train MSE (x10- Gy: Mean squared error on the training set, also scaled by 10 - The first four metrics measure the model's 's generalization ability, while the last one reflects its fitting ability. For all metries, lower values indicate better performance.
16

---

Published as a conference paper at ICLR 2026
C.3 ROBOTIC MANIPULATION WITH IMITATION LEARNING
Data generation We evaluate on five representative manipulation tasks from the ManiSkill Bench- mark (Mu et al. (02T): PushCube, PickCube, StackCube, PegInsertionSide, and PlugCharger. For each task, we collect expert demonstration trajectories using the ManiSkill data collection API. These trajectories are generated via motion planning algorithms and are limited to maximum of 200 simulation steps per episode. The number of demonstrations collected per task is listed in Table [
During testing, the initial scene configurations randomized using the same distribu- tion employed during training data collection. For additional details, please refer to the ManiSkill benchmark (Mu et al. 2021).
Training details We follow the training setup in Chi et al. (2023). including optimizer choice, learning rate scheduling, and exponential moving average (EMA) for network stabilization. Specif- ically, we use the AdamW optimizer with a learning rate of 0.0001, 3 = (0.95, 0.999). and a weight decay of 1 10 6 A cosine learning rate scheduler with 500 warm-up steps applied. To enhance training speed and stability, we maintain an EMA of the model weights with decay factor of 0.75. During training, we sample batch of 1024 expert observation-action pairs from the demonstra- tion dataset in each iteration and perform a gradient update using the diffusion loss. All training hyperparameters are summarized Table 6 The number of training iterations and demonstration trajectories varies across tasks, depending implementation details, please refer to Chi et al. (2023).
task complexity; see Table 7 for details. For further
Table 6: Training hyperparameters for imitation learning experiment (following Chi al. 2023 's settings).
Parameter name
Value
Parameter name
Value
Parameter
Value
Optimizer
Adam W Cosine scheduler
Learning rate
0.0001
Weight decay
1e-6
LR scheduler
Batch size
1024
Episode steps
200
Table 7: Demo trajectories and training iterations for each task in imitation learning experi- ment.
Task name
PushCube
PickCube
StackCube
PegInsertionSide
PlugCharger
Demo trajectories
100
100
100
500
500
Training iterations
30k
30k
60k
100k
300k
C.4 3D HAND RECONSTRUCTION
Dataset preparation Following Hamer, train model large number of datasets that provide 2D or 3D hand annotations. Specifically, we use FreiHAND (Zimmermann et al.) 2019).
HO3D (Hampali al., (2020), MTC (Xiang et al. 2019). RHD (Zimmermann & Brox 201 7). In-
terHand2.66 2.6M (Moon et | H203D (Hampali et al. 2020). DEX YCB (Chao et al. 202
COCO WholeBody (Jin et al. 2020). Halpe (Fang et al. 2022) and MPII NZSL (Simon al.
201
Training Details We follow the training protocol described in HaMeR (Pavlakos et al. 2024
We use the AdamW optimizer with learning rate le-5 and weight decay 1e-4, and train the model for 1,000,000 steps. The batch size is set to 64. Our model takes in a square image around the target hand, resized to 256 x 256, and output the 58 MANO (Romero et al., 2022) parameters of the human hand, specifically 48 pose parameters and the 10 shape parameters. We evaluate the model on the validation set for every 1,000 steps, and pick the model with the lowest validation loss for final evaluation. Similar Hamer, camara ı corresponds to a translation t e R3 that projects the 3D mesh and joints into the image. Hyperparameters summarized in Table
additionally estimate the camera parameters The
17

---

Published as a conference paper at ICLR 2026
Table 8: Training hyperparameters for 3D hand reconstruction experiment.
Parameter
Value
Parameter
Value
Parameter
Value
Training iterations 1,000,000 64
Optimizer
AdamW
Learning rate
le-5
Batch size
Validate every
1,000 iterations
Weight decay
le-4
D METHOD AND IMPLEMENTATION DETAILS
D.1 FORWARD KINEMATICS FITTING
We construct four baseline methods using existing neural network backbones for comparison: MLP, Graph Convolution Network (GCN), Transformer, and Body Transformer (BoT) (Sterrazza et al. 2024
MLP The MLP baseline concatenates the input root position, root orientation, and joint angles into a (3 + 9 + Dome-ddimessonnl vector (DoF = 16 for the LEAP hand), and feeds it into a 7-layer MLP with the following shape: [28, 768, 768, 768, 768, 768, 768, 204). The output represents the positions and orientations of all DoF - 1 = 17 links, All hidden layers use ReLU activation, and no normalization layers are applied.
The GCN baseline uses 1 + DoF separate linear transformations to encode the root pose and each joint angle into + DoF feature embeddings, each with 512 dimensions. Each embedding corresponds one robot link: the one derived from the root pose represents the root link, and each joint's embedding corresponds its child link. We represent the robot as an undirected tree graph with DoF nodes (links) and DoF edges (joints). The GCN applies 11 layers of graph convolution to update the link features. All hidden layers have 512 dimensions. Finally, separate linear transformations used predict the pose (position and orientation) of each link.
GCN
Transformer The Transformer baseline uses the same approach as the GCN baseline to encode the input root pose and joint angles into DoF link feature embeddings, each with 256 dimen- sions. After applying positional encoding, these embeddings are passed through Transformer backbone consisting of Transformer blocks. Each block includes feed-forward layer with a 256- dimensional hidden layer. The Transformer outputs 1 + DoF updated link features. As in the GCN baseline, separate linear transformations are used to predict the pose of each link from these features.
Body Transformer (BoT) (Sferrazza et al. 2024) The BoT baseline shares a similar architecture with the Transformer baseline but replaces the Transformer backbone with a Body Transformer backbone. uses the same robot connectivity graph as in the GCN baseline to capture the structural relationships between links.
Rodrigues Network (ours) Our method uses specially customized Rodrigues Network as the neural backbone. The network consists of 12 Rodrigues Blocks, and each block contains only a single Rodrigues Layer. No Joint Layers, Self-Attention Layers, Global Tokens are used. Each Rodrigues Layer has 1 joint channel (C, = 1) and 3 link channels (C₁. = 3). We first concatenate the input root position, root orientation, and joint angles into a (3 + -dmmensiia vector. This vector is passed through DoF separate linear transformations to produce the input joint fea- tures, and through Rodrigues Network updates the joint and link features through its 12 Rodrigues Blocks, resulting in the output joint and link features. Finally, we concatenate all output link features and apply + DoF separate linear transformations to predict the pose of each link.
DoF separate linear transformations to produce the input link features. The
Parameter count comparison All baseline networks have around million parameters, while network has only 0.2 million parameters. Since this experiment focuses only fitting ability and not generalization, this setup gives an advantage to the bascline methods with more parameters. Even so, network still performs better.
18

---

Published as a conference paper at ICLR 2026
Compute resources We train each model on single Quadro RTX 6000 graphics card, and the approximate training time for 100,000 training iterations are listed in Table
Table 9: Approximate training time of different methods for fitting forward kinematics
Method
MLP
GCN
Transformer
BoT
Rodrigues (ours)
Time
17min
1h 50min
2h 20min
20min
18min
D.2 MOTION PREDICTION IN CARTESIAN SPACE
This experiment uses the same four neural backbones as in the forward kinematics fitting experi- ment: MLP, Graph Convolution Network (GCN), Transformer, and Body Transformer (BoT).
MLP
The MLP baseline takes all 8 x 6 48 input values from the input frames as input and feeds them into a 7-layer MLP with the following layer sizes: [48, 768, 768, 768, 768, 768, 768, 48). The output represents the joint configurations of the 8 predicted frames. All hidden layers use ReLU activation, and no normalization layers are applied.
The GCN baseline uses 1 + DoF separate linear transformations encode the input into + DoF feature embeddings, each with 512 dimensions. As in the forward kinematics fitting ex- periment, each embedding corresponds to one robot link, and the robot modeled as an undirected tree graph with + DoF nodes (links) and DoF edges (joints). The GCN applies 11 layers of graph convolution to update the link features. Finally, for each joint, we extract the output feature of its corresponding child link and apply separate linear transformation to predict the joint's 's angles for all output frames.
GCN
Transformer The Transformer baseline follows the procedure the GCN baseline to code the imput into 1 DoF link feature embeddings, each with 250 dimensions. Positional encoding then applied to these embeddings, which are processed by Transformer backbone consisting of Transformer blocks. Each block contains feed-forwar layer with a 250-dimensional hidden layer. DoF updated link features. in the GCN baseline, we extract the
The Transformer outputs output feature corresponding to each joint's 's child link and apply a separate linear transformation to predict the joint's output angles.
Body Transformer (BoT) (Sterrazza et al. 2024) The BoT baseline replaces the Transformer backbone of the Transformer baseline with a Body Transformer backbone, using the same number of blocks and hidden dimensions.
Rodrigues Network (ours) Our method uses a full Rodrigues Network as the neural backbone, consisting of 12 Rodrigues Blocks. Each block contains a Rodrigues Layer, Joint Layer, and Self-attention Layer. In this experiment, Global Tokens are also not used. Each Rodrigues Layer configured with 4 joint channels (C3 = 4) and link channels (C₁. = 8). Each Self-attention Layer operates 356-dimensional embeddings with 8 attention heads. To prepare the input, apply DoF separate linear transformations to produce the input joint features, and 1 + DoF separate linear transformations to generate the input link features. These features Rodrigues Network, which updates them through all 12 blocks to produce the final joint and link features. For each joint, concatenate its output joint feature with the output link feature of its child link, and apply separate linear transformation to predict the joint's 8-frame output angles.
then processed by the
Parameter count comparison All baseline methods and our approach have approximately mil- lion parameters. This ensures a fair comparison of both fitting and generalization abilities.
Compute resources We train each model on single Quadro RTX 6000 graphics card, and the approximate training time for 100,000 training iterations are listed in Table TO
19

---

Published as a conference paper at ICLR 2026
Table 10: Approximate training time of different methods for motion prediction experiment
Method
MLP
GCN
Transformer
BoT
Rodrigues (ours)
Time
27min
Ih 12min
Ih 20min
Ih 20min
2h 22min
D.3 ROBOTIC MANIPULATION WITH IMITATION LEARNING
This experiment follows the Diffusion Policy (DP) framework proposed in Chi et al. (2023). and evaluates different neural backbones for denoising action samples. The original paper (Chi et al. 2023) provides two backbone options, convolutional UNet and Transformer, which we directly adopt as baselines. In the DP framework, the policy observes history of prior time steps, predicts sequence of 16 future actions, and executes the first 8 during deployment. Each action vector consists of 8 values: 7 for joint position control of the 7-DoF Franka and I for the 1-DoF Panda gripper.
The UNet-DP baseline uses a convolutional UNet architecture that performs temporal convolution over the 16-step action sequence. We follow the default structure, with the hidden dimensions the downsampling layers set to [320, 320, 344). For further implementation details, please refer to Chi et al. 2023).
UNet-DP
Transformer-DP The Transformer-DP baseline applies causal self-attention to the 16 action-step tokens. We use embedding dimension of 320, total of 10 Transformer blocks. Additional details are available Chi et al. (2023).
hidden dimension of 1280, and
Rodrigues-DD (ours) Our method adopts Rodrigues Network as the denoising backbone, com- posed of Rodrigues Blocks with 16 link feature channels (C2 16), joint feature channels (C, = 8), and self-attention embedding dimension of 256. We also introduce Global Tokens of dimension 128 to model gripper actions. The denoising network takes three inputs: time step, a noisy action, and observation vector. We first embed the time step and concatenate it with the observation and noisy action to form an input feature vector. This vector is then passed through DoF separate linear transformations to generate the input joint features, + DoF separate linear transformations to generate the input link features, and one additional linear transformation to pro- duce the input global token. The Rodrigues Network updates all features through its 12 Rodrigues Blocks, resulting in output joint features, output link features, and output global token. For each joint, we concatenate its output joint feature with the output link feature of its corresponding child link, and apply a separate linear transformation to predict the joint's 16-frame denoised action tra- jectory. For the gripper, we use linear transformation on the output global token to predict the 16-frame denoised gripper actions.
Parameter count comparison All baseline networks and our network have approximately 17 million parameters to make comparisons fair.
D.4 3D HAND RECONSTRUCTION
Architecture wise, we employ a Vision Transformer (Dosovitskiy al., 2020) as the image- processing backbone, followed by our Rodrigues Network (RodriNet) head to regress both hand and camera parameters. Since there are 17 links and 16 joints in the MANO parameter model, to make the information pass through the entire kinematic chain, our RodriNet head comprises 18 Rodrigues blocks, each of which sequentially applies a Rodrigues Layer, a Joint Layer, and a Self-attention Layer to update the link features. joint features. The Rodrigues Layer uses link channels, and the Self-attention Layer uses 64 embed dimensions. We enable Global Tokens in this experiment to model the MANO's shape parameters.
In addition to the standard components, we append a Cross-Attention Layer at the end of each block. This layer processes the link features and the input image tokens through standard cross attention transformer module, enriching the link features with visual information from the input image.
The training takes approximately 7 days single Quadro RTX 6000 graphics card.
20

---

Published as a conference paper at ICLR 2026
Table 11: Ablation studies for motion prediction in Cartesian space with trainset size 10".
We remove the Rodrigues Layer (R Layer), Joint Layer (J Layer), Self-attention Layer (S Layer) respectively from the original Rodrigues Network, and evaluate the MSE on train/test sets.
R Layer
J Layer
S Layer
Params (M)
Train MSE (le-6)
Test MSE (le-6)
3.04
1.93 40.34
2.56 +0.39
V
1.44
1.94 1:026
2.33 #0.26
V
3.01
2.33 40056
2.80 #0.62
V
1.69
5.57 10.55
6.19 10.57
E ADDITIONAL RESULTS
We conduct additional experiments to further support the claims made in the main paper.
E ABLATION STUDIES
In the main paper, we benchmark the Rodrigues Network against several baselines for motion pre- diction in Cartesian space using a fixed training set of 10 trajectories. Our method demonstrates a significant performance advantage over all baselines. To investigate the contributions of different components of our model, we perform an ablation study by individually removing the Rodrigues Layer, Joint Layer, or Self-attention Layer from the original architecture, and evaluate the resulting performance changes.
The results are presented in Table I1] First, we observe that the Self-attention Layers comprise more than half of the model parameters. However, removing them results in only slight increase in train- ing error and even a slight decrease in test error. This suggests that while the Self-attention Layer enhances the model's ability fit the training data, may slightly hinder generalization. Nonethe- less, we retain this component in our default architecture to ensure sufficient model capacity for more complex tasks. Second, the Joint Layers contribute negligibly to the overall parameter count, yet their removal consistently degrades both training and test performance. This highlights their critical role the effectiveness of the Rodrigues Network. Third, the Rodrigues Layers constitute a substantial portion of the model's parameters. Removing them causes the greatest drop in both training and test performance among all ablation settings. This indicates that the Rodrigues Layer is the most critical component for the model's success. Since the Rodrigues Layer encodes the core inductive bias of our architecture, these findings strongly support our central claim: embedding structural prior of articulated kinematics directly into the network architecture improves learning the actions and motions of articulated actors.
E.2 TUNING THE BASELINES FOR MOTION PREDICTION
While our method outperforms all baselines on the motion prediction task under the 3-million- parameter setting, we further investigate whether these baselines have been sufficiently tuned. Therefore, we construct three additional configurations for each of the four baselines (GCN, MLP, BoT, and Transformer) with approximately 1M, 10M, and 30M parameters.
The results are shown in Figure 6 For all baselines, the training error consistently decreases as model size increases. However, both GCN and MLP begin to overfit with larger models, as in- dicated by rising test errors. contrast, BoT and Transformer exhibit test error saturation, but not overfitting, as parameter count increases to 30M. Importantly, across all parameter scales, our 3M-parameter model outperforms all baseline configurations on both training and test errors. This confirms that the benchmark results reported in the main paper are reliable and that the baselines have been appropriately tuned. Furthermore, even the best-performing baseline (Transformer) with 30M parameters yields training error that remains higher than the test error of our 3M model. This highlights the superior fitting capability, generalization performance, and parameter efficiency of our proposed approach.
21

---

Published as a conference paper at ICLR 2026
Train MSE vs model parameters
Test MSE model parameters
MSF
10
MLP
MLP
GCN
GCN
BoT
BoT
2.56e-06
Transformer
Transformer
Rodrigues (3M)
1.93e-06
Rodrigues (3M)
1M
10M
30M
1M
3M
10M
30M
Model parameters
Model parameters
Figure Comparing our method to different baseline configurations in motion prediction with trainset size = 10
Table 12: Hyperparameter sensitivity analysis for the Rodrigues Network on motion prediction in Cartesian space with trainset size = 10
Variation
C5
CL
Params (M)
Train MSE (1c-6)
Test MSE (1e-6)
Default
3.04
1.93
2.56
Joint channels
2.43
2.18
3.00
Joint channels
4.26
1.32
1.96
Link channels
1.19
3.64
4.35
Link channels
16
8.73
1.51
2.18
Num blocks
1.55
2.24
2.76
Num blocks
24
6.03
3.47
4.15
E.3 HYPERPARAMETER SENSITIVITY ANALYSIS FOR THE RODRIGUES NETWORK
We investigate the robustness of our network's performance with respect to variations in key archi- tectural hyperparameters. Specifically, we focus three components of the Rodrigues Net- work: the number of joint channels (C.)), link channels (CL.) and the number of Rodrigues Blocks (B). Starting from the default configuration used in the main paper, we create six variants by either halving or doubling the value of each hy peppaaaraeeter. We then evaluate these configurations on the motion prediction task determine whether the network maintains stable performance across these variations. This analysis helps assess the sensitivity of our model to design choices and the general robustness of its architecture.
The results, summarized in Table I2 show that increasing the number of joint channels from 2 to 8 consistently improves both training and test errors. A similar trend observed for link channels, where increasing from 4 to 16 leads to better performance. In contrast, increasing the number of Ro- drigues Blocks from to 24 initially improves train and test errors, but further depth degrades them both, indicating optimization challenges in deeper configurations. Overall, across all hyperparam- eter variations (up to 4x changes), performance remains within reasonable range, demonstrating the robustness of our architecture. Furthermore, all variants significantly outperform the strongest baseline from the main paper, suggesting that our method is not only robust but also easily tunable for high performance.
22

---

Published as a conference paper at ICLR 2026
F ACCELERATING THE MULTI-CHANNEL RODRIGUES OPERATOR WITH CUDA
The Multi-Channel Rodrigues Operator is novel component in architecture, and found that implementing it directly in Tyrchh led to suboptimal speed and memory efficiency. To address this, developed custom CUDA kernel that significantly accelerates both the forward and backward passes while reducing memory overhead.
In the forward pass, assign each CUDA thread block to compute a single output channel over sub-batch of link features. Each thread accumulates contributions from all imput joint and link channels to produce the output feature. This design eliminates the need explicitly compute and store intermediate tensors such as U and Ù (see Equation 6 in the main paper), thereby saving memory bandwidth and improving runtime performance.
The backward pass adopts similar strategy of accumulation to avoid intermediate computations. However, additionally requires explicit gradient computations for both the input joint features and the Rodrigues Kernels, which we handle within the same memory-efficient framework.
This CUDA implementation significantly improves efficiency of our experiments. For instance, training 12-block, 16-DoF Rodrigues Network composed solely of Rodrigues Layers, with 16 joint channels and 16 link channels (approximately 52 million parameters), for 100,000 iterations at batch size of 1024, would take over 100 hours using our PyTorch implementation Quadro RTX 6000 GPU. In contrast, using our CUDA kernel to compute the Multi-Channel Rodrigues Operator reduces the training time approximately 15 hours (over 6x speed-up).
Nonetheless, we acknowledge that existing operators (such as graph convolutions, temporal convo- lutions, and multi-head attention) have benefited from extensive optimization current implementation does not yet reach that level of maturity. We will release the CUDA source code upon acceptance and hope it serves as a foundation for further research into optimized imple- mentations of the Multi-Channe Rodrigues Operator.
the years. Our
23


Reviews:
[{"review_id": "ZpYgdlPP4q", "reviewer": "Reviewer_fHxR", "summary": "This paper introduces RodriNet, a neural architecture that embeds articulated-kinematics structure directly into its computation by learning a generalized version of Rodrigues’ rotation formula. Instead of treating joint values as flat vectors like MLPs or Transformers, the proposed Neural Rodrigues Operator replaces fixed trigonometric coefficients in classical forward-kinematics equations with learnable weights, enabling structured message passing along a robot's kinematic tree. RodriNet stacks these operators with joint-update and attention layers, yielding a model that naturally learns hierarchical motion patterns. Experiments show large gains in fitting forward kinematics, predicting 3D joint trajectories from Cartesian motions, improving imitation-learning performance when used as the backbone in Diffusion Policy, and achieving state-of-the-art results on 3D hand pose estimation, all with fewer parameters. Overall, the work argues for architectural priors tailored to robot embodiment as a path to more efficient and generalizable action learning.", "questions": "-", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- Introduces a learnable generalization of Rodrigues rotation to embed articulated-kinematics structure into neural networks, giving a principled inductive bias for articulated action learning\n\n- Presents a clear architecture (Rodrigues Layer, Joint Layer, self-attention) that mixes local kinematic structure with global information exchange\n\n- Demonstrates strong improvements across domains: forward kinematics, Cartesian-to-joint motion prediction, robotic imitation learning, and 3D hand reconstruction\n\n- Shows faster convergence and better data efficiency than standard architectures like MLPs, GCNs, and Transformers in modeling articulated motion", "weaknesses": "- Requires per-joint learnable parameters and tree-structured computation, which may introduce higher architectural complexity compared to simpler universal backbones\n\n- No ablation studying how much each component (Rodrigues layer vs joint layer vs attention) contributes, leaving uncertainty about which parts drive performance", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "erICPkVuUU", "reviewer": "Reviewer_LiRT", "summary": "The authors investigate incorporating structural inductive biases from kinematic models in neural network design. They propose the Rodriguez layer, which combines projecting joint features through the learned Rodriguez operator and aggregating features globally with self-attention. Experimentally, the authors show that for problems where agents have a rigid body structure (e.g., forward kinematics or solving manipulation tasks with imitation learning), their model design performs substantially better than neural architectures that lack these inductive biases.", "questions": "Q1: What's the reason for not comparing it to the Denavit–Hartenberg formulation of kinematic structure? A quick search reveals that the Rodrigues formulation does have some technical advantages, but this seems the more appropriate comparison. \n\nQ2: What kind of features are provided to the Rodriguez layers? Do the transformations process non-kinematic related features in some way? How would these layers work with non-kinematic associated features, for instance? \n\nQ3: What about the author's current work that limits their representation to only spherical joints as opposed to translational joints? \n\nQ4: What distinguishes the Rodriguez layer from running system identification to find the model's kinematics?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "> Incorporating learnable kinematic models in robotics makes intuitive sense and seems to provide measurable improvements for applying machine learning on rigid body structures\n\n> The paper is well written and presents the proposed model clearly. The diagrams help summarize the approach. \n\n> The experiments show compelling results to validate the efficacy of the Rodriguez layer for rigid body agents. The inclusion of both robots and animated characters helps demonstrate the model's potential across several settings of embodied agents.", "weaknesses": "> Some aspects of the work are a bit obvious. The results for forward kinematics are not surprising, particularly as the Rodriguez matrix is one of two major approaches to modelling rigid-body kinematics—the other being Denavit-Hartenberg parameters. It could strengthen the paper to show how directly predicting the kinematic model's parameters compares in this problem. \n\n> Some results could benefit from statistical hypothesis testing to confirm whether the observed benefits are statistically significant (e.g. the success rates in Table 2). \n\n> Something that might be helpful would be to include results for failure cases. Presumably, there are data modalities or scenarios in which assuming a kinematic relation negatively impacts performance. \n\nWriting Opinions:\n\nLine 063 - 068:  This paragraph can be cut with almost no loss of meaningful content to the paper. \n\nRelated Work: Consider looking at cross-embodiment works as potential related work. These models also consider agent structure as necessary to model explicitly. \n\n[1] Gupta, Agrim, et al. \"Metamorph: Learning universal controllers with transformers.\" arXiv preprint arXiv:2203.11931 (2022).\n\n[2] Xiong, Zheng, Jacob Beck, and Shimon Whiteson. \"Universal morphology control via contextual modulation.\" International Conference on Machine Learning. PMLR, 2023.\n\n> Figure 3: We suggest replacing \"backbone\" with \"architecture\" unless these models are pretrained, which, in our experience, is where this term is used more frequently. \n \n>line 353: replace backbone with \"architecture\" \n\n> Figure 5: Provide more information in the caption on what is shown in the diagram.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "1oGfdEj4Dn", "reviewer": "Reviewer_4VZY", "summary": "This paper introduces Rodrigues Network (RodriNet), a neural architecture that explicitly embeds articulated kinematic structure into its design. The key component is the Neural Rodrigues Operator, derived from Rodrigues’ rotation formula by separating “state-dependent” (sin θ, cos θ) terms from “structure-dependent” coefficients and making the latter learnable. This transforms parent–child relations in a kinematic tree into a learnable message-passing rule. A multi-channel variant updates per-link 4×4 features from per-joint inputs using both left and right multiplications. Stacking these operators forms Rodrigues Layers, complemented by a Joint Layer and a Self-Attention Layer for global interaction, with an optional global token handling shared variables such as base pose or gripper state.\nEmpirically, RodriNet (1) achieves superior fitting accuracy on forward kinematics for the LEAP Hand and improved Cartesian-trajectory prediction on the UR5 arm, (2) boosts simulated success rates in Diffusion Policy across five ManiSkill tasks, and (3) improves single-image 3D hand reconstruction on FreiHAND by replacing the transformer head in HaMeR, yielding small but consistent SOTA gains with significantly fewer parameters.", "questions": "- Equivariance and structure validity: Does RodriNet satisfy any formal invariance or equivariance guarantees under changes of the base frame or re-rooting of the kinematic tree? Could initializing weights from analytical FK coefficients and applying soft orthogonality constraints improve SE(3) consistency while maintaining flexibility?\n\n- Comparison to SE(3)-aware models: How does RodriNet perform against SE(3)-equivariant architectures (e.g., SE(3)-Transformer, E3NN) or MLP/Transformer baselines with Fourier angle encodings and identical kinematic-graph wiring? This would clarify whether the Neural Rodrigues Operator contributes beyond periodic encodings and structural locality.\n\n- Cross-morphology generalization: Can the per-joint kernels be shared across joints of the same type or conditioned on metadata (axis, link transforms) to enable zero-shot transfer to unseen kinematic trees? A small multi-embodiment study would help test generality.\n\n- Feature interpretation: Do the learned 4×4 “link features” converge toward SE(3)-like structures in practice? Visualizing the 3×3 rotation blocks (e.g., via polar decomposition) could reveal whether the network implicitly maintains orthogonality.\n\n- Compute tradeoffs: RodriNet requires longer training time in the UR5 task. With the CUDA kernel acceleration, what is the effective wall-clock speedup compared to a Transformer at matched accuracy? Are there memory or scaling limits for large CL/CJ?\n\n- Simulation-to-real transfer: In tasks where simulated gains are limited (e.g., PegInsertionSide, PlugCharger), how does RodriNet handle tactile or force-feedback inputs? Any early evidence of transfer performance on hardware would strengthen the empirical case.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- Principled inductive bias: The formulation of a learnable operator directly from Rodrigues’ rotation rule is mathematically clean and physically grounded. By explicitly modeling the basis (1, sin θ, cos θ), RodriNet captures rotational structure while preserving the kinematic tree topology.\n\n- Strong expressivity for kinematic mapping: On forward-kinematics fitting, RodriNet (Rodrigues Layers only) achieves orders-of-magnitude lower MSE and faster convergence than MLP, GCN, Transformer, or BoT baselines, demonstrating that the inductive bias aligns well with articulated geometry.\n\n- Effective bridging of Cartesian and joint spaces: The UR5 trajectory prediction task is a well-controlled test of inverse kinematics. RodriNet achieves lower train/test MSE and maintains accuracy under reduced data regimes, showing improved data efficiency.\n\n- Practical gains in imitation learning: Integrated into Diffusion Policy, RodriNet improves average success rate (0.61 vs. 0.58 for UNet-DP and 0.44 for Transformer-DP) across five ManiSkill tasks, with the largest gains on PickCube and StackCube. The global token cleanly handles gripper control.\n\n- Cross-domain transfer: When replacing the transformer head in HaMeR, RodriNet attains consistent SOTA on FreiHAND with far fewer parameters (10.7 M vs. 39.5 M), indicating that the inductive bias generalizes beyond robotics.\n\n- Thorough analysis and solid engineering: Ablations identify the Rodrigues Layer as the main contributor. Performance is stable under architectural scaling, and the custom CUDA implementation yields ~6× speedup in large configurations.", "weaknesses": "- Lack of equivariance guarantees: The learnable operator applies unconstrained 4×4 left/right multiplications, offering no guarantee of SE(3) consistency or frame equivariance. Layer composition may not preserve physically valid transformations. The method serves as an inductive bias rather than a structured constraint; adding formal regularization (e.g., orthogonality or SE(3)-aware constraints) could strengthen the theoretical foundation.\n\n- Limited novelty relative to existing basis encodings: The operator performs a learned linear combination over {1, sin θ, cos θ} (and quadratic quaternion terms for MANO), similar in spirit to Fourier or harmonic encodings. The absence of comparisons to SE(3)-equivariant architectures (e.g., SE(3)-Transformer, EGNN) or differentiable FK layers leaves it unclear whether gains arise from kinematic-tree structure or from providing the correct trigonometric basis.\n\n- Restricted embodiment generalization: Kernels are defined per joint within a fixed kinematic tree, and experiments are conducted per robot (LEAP Hand, UR5, Franka). There is no evaluation of cross-morphology transfer, shared parameters across repeated structures, or conditioning on structural metadata for multi-embodiment generalization.\n\n- No real-world validation: Results in imitation learning are limited to simulation. For contact-rich tasks such as PegInsertionSide or PlugCharger, performance gains are minimal, suggesting limited robustness to real sensor or actuation noise.\n\n- Compute tradeoffs insufficiently analyzed: Training RodriNet for motion prediction requires longer runtime (2 h 22 m vs. ~1 h 20 m for Transformer/BoT), yet compute–accuracy tradeoffs are not fully discussed.\n\n- Scope limitations: The model handles only rotational joints and omits link geometry or prismatic motion. Its focus on imitation learning rather than closed-loop control narrows applicability to contact-rich or mobile-manipulation domains.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "NmdgR02Wt9", "reviewer": "Reviewer_wx1x", "summary": "This paper proposes Rodrigues network (rodrinet), a novel graph neural network architecture that encodes the kinematics-aware inductive bias. The authors evaluate the proposed architecture on forward/inverse kinematic fitting, robot manipulation, and hand reconstruction tasks, which show that rodrinet outperforms or is on par with existing popular architectures.", "questions": "(I list all questions in the weakness section)", "limitations": "", "rating": 2, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1. The overall presentation is very clear and straightforward, give enough background to understand the proposed method. \n2. The forward/inverse kinematic fitting experiments clearly demonstrate that the proposed architecture captures the inductive bias of articulated chains. Before that, I hesitated on this because GNN message passing is very different from sequential forward kinematic computation. \n3. The Rodriguez equation is an interesting perspective, and the authors conduct experiments over a diverse range of tasks. I very appreciate the authors' efforts.", "weaknesses": "My major concerns (questions) are two below, which also link to the core weaknesses of this paper in my opinion.\n\n1. Why robot-kinematic-inductive bias matters? \n2. Does the performance gain from RodriNet justify the cost of an unorthodox design?\n\n[Q1]: Most of the current robotic applications use task space control (i.e., predicting the end-effector pose), aka the eef pose, which can be easily translated to joint torques with an op-space controller. In this case, the model does not need to consider the robot's own kinematic structure and can leave that to controller. \n\nIn this paper's experiment results, the most compelling results come from the forward/inverse kinematics fitting, which, however, has faster and flexible analytical solutions. On other tasks, the benefits are very marginal. Like in the robot manipulation experiments, even when Rodrinet outperforms the diffusion policy on PickCube and StackCube, I am not entirely convinced because these two tasks should not heavily depend on the robot kinematic chain. In the hand reconstruction task, the authors explicitly mention that modifications are made to RodriNet to suit MANO's configuration space (for example, fitting the hand shape beta parameters, which I assume are not tied to joint/link). This even make the results look less convincing. \n\n\n[Q2] Even if robot-kinematic-inductive bias may not matter in every task, having them for free does not hurt. However, this proposed architecture, which looks like another GNN variant, has limitations and cannot be used for free. \n\nFor example, all robot manipulation experiments are conducted in state space, rather than using visual inputs. This is a critical constraint. Additionally, using a GNN instead of a general transformer could make it challenging to interpolate with other transformer-based backbones. Using joint space and not task space actions can also make the prediction less visually grounded. In sum, these constraints outweigh the marginal performance gain.\n\nA side note: joint controllers are widely used in dexterous hands; maybe this technique can find its place in hand manipulation tasks, instead of parallel grippers.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Oral)