Scaling Manipulation Learning with Visual Kinematic Chain Prediction

Authors: xinyu  zhang
yuhan  liu
haonan  chang
abdeslam  boularias
Venue: robot-learning_org_CoRL_2024_Conference
Published: 2024-05-24 00:22:53

Abstract: Learning general-purpose models from diverse datasets has achieved great success in machine learning. In robotics, however, existing methods in multi-task learning are typically constrained to a single robot and workspace, while recent work such as RT-X requires a non-trivial action normalization procedure to manually bridge the gap between different action spaces in diverse environments. In this paper, we propose the visual kinematics chain as a precise and universal representation of quasi-static actions for robot learning over diverse environments, which requires no manual adjustment since the visual kinematic chains can be automatically obtained from the robot’s model and camera parameters. We propose the Visual Kinematics Transformer (VKT),  a convolution-free architecture that supports an arbitrary number of camera viewpoints, and that is trained with a single objective of forecasting kinematic structures through optimal point-set matching. We demonstrate the superior performance of VKT over BC transformers as a general agent on Calvin, RLBench, ALOHA, Open-X, and real robot manipulation tasks. Video demonstrations and source code can be found at https://mlzxy.github.io/visual-kinetic-chain.

Main paper: 
 introduction: 
 There are numerous techniques in machine learning and computer vision that can successfully learn a single general-purpose model from multiple diverse datasets [1] . In robotics, however, despite the recent advances in multi-task learning that enable a single policy to perform various tasks through imitation learning with language instructions [2, 3] , these methods are typically constrained to a single robot and workspace. Some recent work, such as RT-X [4, 5] , leverages a large visionlanguage model (VLM) to directly train policies with Behavioral Cloning (BC) on the Open-X Embodiment [5] , a collection of datasets crowd-sourced from various environments and robots. However, these techniques require a non-trivial action normalization procedure to bridge the gap between the different action spaces in diverse setups, such as end-effector poses, joint positions, and velocities in various world frames. This manual engineering procedure is currently custom-designed for each training dataset, which affects the generalization and interpretability of these models. Therefore, a key question is: can we find an action representation that is precise and universal for various setups and robots? To this end, we propose the visual kinematic chain, which is the projection of the robot's high-dimensional kinematic structure into the image plane as pixels. Instead of predicting the low-level robot actions, we propose to learn and visually forecast the movement of robots' kinematic chains in the image plane. This visual approach provides a unified action representation for different robots, and requires no manual adjustment since the visual kinematic chains can be automatically obtained from the robot's model and camera parameters.To forecast the kinematic structures of various robots, we render the kinematic chains into point sets by sampling points along links and performing optimal matching [6] to minimize the earth moving Figure 1 : Overview of the proposed framework. We use the visual kinematic chain as a universal action representation across diverse robots and setups. We propose the Visual Kinematics Transformer (VKT), an architecture based solely on attention layers, which predicts the future movements of the visual kinematic chain in images from multiple viewpoints. Our VKT is trained with the earth-moving distance as the single learning objective without knowing any low-level robot states or actions. When deployed in a specific environment, we freeze the VKT as a backbone and attach a tiny head to project the VKT output to actual robot commands. distance [7, 8] between the predicted point sets and ground-truth kinematic structures. Further, we propose the Visual Kinematics Transformer (VKT), a convolution-free architecture that is solely based on the attention mechanism in the RGB space, which supports an arbitrary number of camera viewpoints. Our VKT is trained with a single objective of forecasting kinematic structures through optimal point set matching without seeing any low-level robot actions. VKT is deployed to a specific environment by simply training a tiny head while freezing the backbone. VKT demonstrates superior performance over BC transformers as a general agent on Calvin (average length 1.46 vs 0.48), RLBench (success rate 61.7% vs 24.3%), ALOHA (success rate 55.3% vs 12%), Open-X, and real robot manipulation tasks.To summarize, our contributions are threefold. (1) We propose the visual kinematic chain as a precise and universal representation of quasi-static actions for learning from diverse robot configurations. (2) We propose VKT, a convolution-free architecture that supports an arbitrary number of camera viewpoints and is trained with a single objective of forecasting kinematic structures through optimal point set matching. (3) We present a thorough empirical study of the performance of VKT on specialized and general agents over a diverse set of language-conditioned tasks and environments. 
related work: 
 Robot Learning from Diverse Environments. Learning a single universal multi-task policy on various robots, cameras, and task configurations is a challenging problem that is receiving increasing attention [9] . One recent strategy to solving this problem consists in training a network that takes as input a robot's kinematic structure [10, 11] . For instance, MetaMorph [12] tokenizes the robot's kinematic tree, which is then used as a transformer prompt to produce per-joint action. However, these methods are only evaluated in simple walking environments of Mujoco [13] without any variations in the task, the camera viewpoint, and the workspace. Several recent works have also looked into leveraging a large vision-language model (VLM) to directly train robots, through behavior cloning, on a collection of crowd-sourced datasets such as Open-X Embodiment [14, 15, 5] . Despite their impressive results, these methods require a non-trivial action normalization procedure to reduce the gap between action spaces such as end-effector poses, joint positions, and velocities in various world frames. However, this manual engineering procedure is custom-designed for each training dataset, which severely affects the generalization and interpretability of these models. In contrast, our method directly learns and visually predicts the movement of robots' kinematic chains.Transformers for Manipulation Learning. Transformers are widely used in manipulation learning [4, 16] . However, existing architectures require a fixed number of camera viewpoints, which is not guaranteed when learning is performed across various environments. Both RT-1/2 and RT-X select one canonical view from each Open-X dataset [5, 4, 17] . RVT [18] supports multiple but predefined viewpoints. PerAct [3] trains transformers over point-clouds and is not limited by camera viewpoints. But methods based on point clouds still suffer from the lack of datasets with depth information, as well as scale discrepancies [19, 20] . In contrast, our visual kinematics transformer (VKT) is a convolution-free architecture that is solely based on attention mechanisms in the RGB space, without the need for depth data. VKT also supports an arbitrary number of camera viewpoints.Intermediate Action Representations. Instead of only using low-level robot commands, incorporating intermediate actions into control policies has been shown to increase data efficiency [21] . RT-H is a hierarchical policy that predicts language motions such as "move arm forward" [22] . SWIM uses affordance maps to unify grasping actions of humans and robots [21] . General Flow employs a labeling process to translate end-effector motions into point-cloud movements [23] . IMOP [24] uses invariant regions to describe key manipulation poses. RT-Trajectory predicts the future 2D trajectory of the end-effector's center in a given image [25] . In contrast, our method forecasts the movements of the entire kinematic chain in multiple camera viewpoints. This representation provides a visually grounded action abstraction, while capturing detailed and small motions at the same time.Recently, designing universal action representation has received growing attention. ATM [26] designs a track-guided policy that accepts a set of point trajectories on the image plane as guidance for predicting robot actions, which enables policy learning from both robot and human videos. However, compared to our visual kinematic chain, point trajectory is ambiguous because trajectories are not guaranteed to correlate with the desired robot actions. UniPi [27] solves robot tasks by first generating videos and then deriving robot actions from the video generated. Yet, using videos as an action representation is inefficient because video generators are orders larger than common policy networks [28, 29] . Moreover, UniPi [27] is evaluated on a single planar environment [30] because deriving 3D actions or joint positions from videos is not trivial. We consider the problem of learning a single universal multi-task policy from diverse robots and workspaces through a unified action space. Our key insight lies in the use of the visual kinematic chain, which is the projection of the robot's high-dimensional kinematic structure into a set of pixels in the image 2D plane. The visual kinematic chain can be analytically generated with camera parameters and robot models without any manual engineering. We show that this simple representation allows us to train a single visual language-conditioned policy over multiple environments. The VLM policy is trained with a single objective of forecasting the future visual kinematic chain movements without knowing any low-level robot actions. Our proposed transformer architecture is built entirely with attention layers and can predict consistent For each camera input, we encode the language instruction and RGB image as text-and-vision tokens with CLIP [32] . Then, we concatenate the text tokens and the kinematics tokens as query tokens. The kinematics tokens are learned parameters. Next, the query and vision tokens are interweaved with a sequence of our proposed multi-view dual attention block. For each block, the query tokens are first updated with self-attention (orange). Then, cross attention is applied with the query tokens as queries and the vision tokens as keys and values (green). A cross-attention layer updates queries with keys and values, we use ➝ to denote queries and to denote keys and values. Then, both the query tokens and the vision tokens are updated through cross-attentions with tokens from other camera viewpoints (blue). Next, the vision tokens are updated by cross-attention with query tokens as keys and values (green). Finally, the T kinematics tokens are projected into T point sets through an MLP, representing the visual kinematic chain in the current and the future T -1 steps. The predicted point-sets are optimized through point-set matching with the ground-truth, as shown in Equation 1. 
visual kinematic chain forecasting: 
 multi-view chains from an arbitrary number of camera viewpoints. To deploy the multi-task policy in a specific environment, one simply needs to freeze the transformer and train a tiny head to project the actions into actual robot commands. An overview of our framework is illustrated in Figure 1 .Fitting any Structure with Point-Set Matching. The kinematic structures of various robots differ significantly. To forecast different structures, we propose to render them to point sets by uniformly sampling points along links, as illustrated in Figure 2 . Let P = {p i } M i=1 denote the point-set rendered from the ground-truth configuration of a kinematic chain, and Q = {q i } N i=1 denote the point-set predicted by VKT, where p i ∈ R 2 , q i ∈ R 2 . We first compute the pairwise euclidean distance matrix C ∈ R M×N , where C i j =∥ p iq j ∥. We then minimize the earth-moving distance EMD(P, Q) = i, j γ * i j C i j , where γ * is a stochastic assignment matrix that matches points in P and Q. We find γ * using the Sinkhorn-Knopp algorithm to solve the following optimization problem online for each mini-batch [31, 8] .EQUATION)Visual Kinematics Transformer (VKT). Our VKT network accepts a language instruction and an RGB image as input and predicts a sequence of T point-sets. The dimensions of the kinematics tokens are T × D, where D denotes the embedding size, and each token predicts one point-set. The first point-set in the predicted sequence describes the current state of the kinematic chain in the given RGB image. The remaining point-sets are predictions of the future states of the kinematic chain in the next T -1 time-steps, i.e., future robot movements. The network is trained to match each point-set to the ground-truth kinematic structure at the corresponding time-step by minimizing the earth-moving distance. The same process is applied to multiple RGB images taken from different viewpoints. The network is also trained to make the future point-sets predicted from different viewpoints consistent with each other. The kinematics tokens are concatenated with the text tokens as query tokens. The concatenated query and vision tokens are forwarded to our proposed multiview dual attention block, to interweave the information between query and vision tokens and across multiple camera viewpoints.The multi-view dual attention block performs dual attentions between query and vision tokens within each single view, indicated by the green blocks in Figure 3 , and multi-view attentions for query and vision tokens independently but across multiple camera viewpoints, shown as the blue blocks in Figure 3 . This dual attention enables the query token to learn the visual kinematic chain movements. The multi-view attention enables a spatially-consistent prediction across multiple viewpoints. The output kinematics tokens are projected to T point-sets using a small MLP network. Each point-set contains N 2D points on the input image. The VKT architecture is illustrated in Figure 3 .Compared to existing robot learning transformers [18, 17] , our proposed VKT is convolution-free and built solely with attention layers. Therefore, VKT supports an arbitrary number of camera viewpoints, which is an important advantage as camera setups are different among various environments. Deploying to a Specific Environment. We freeze the trained VKT, drop the point-set prediction branch, and only use the kinematics tokens. We apply a 1D convolution to project the T kinematics tokens into low-level robot actions for the next T timesteps. Unlike the VKT backbone which is trained from multiple environments, the 1D convolution head is trained separately for each environment.The corresponding proposed architecture is illustrated in Figure 4 .  
experiments: 
 We evaluate VKT on Calvin [33] , RLBench [34] , and ALOHA [35] . Calvin and RLBench are standard benchmarks in language-conditioned multi-task manipulation learning. ALOHA is a bimanual manipulation environment. Further, we evaluate VKT in a subset of Open-X Embodiment [4] and real robot experiments. We aim to answer the following questions: (1) Can visual kinematics forecasting improve specialized agents in each environment? (2) Can VKT serve as a strong general agent in multiple environments? (3) Can VKT be efficiently trained with real-world demonstrations and solve manipulation tasks with real robots?Setup. We adopt the setting of RT-X [5] for imitation learning from RGB images without depth. RL-Bench contains various task categories, such as pick-and-place, tool use, high-precision operations, screwing, tasks with visual occlusion, and long-term manipulation. RLBench provides five RGBD cameras. We use the RGB signal from the front, left, and right cameras and exclude the cameras installed on the shoulder and gripper of the robot because they do not take images of the robot. We choose 17 tasks based on camera visibility and the task categorization of Hiveformer [36] . We use 100 recorded trajectories per task for training, evaluate the trained agent on 25 independent trials, and report the average success rate. Calvin requires policies to accomplish multiple tasks sequentially in each episode. We use the training set and evaluation procedure of the Calvin D environment and report the average number of successful tasks per episode (average length) [33, 37] . ALOHA requires a high-frequency control of two robot arms through joint positions to solve tasks collaboratively. We chose the cube transfer task that requires picking up the cube and transferring it to the other arm. We follow the convention [37, 38] to report the metrics on Calvin and ALOHA with three independent runs of 1000 and 100 episodes, respectively. We report the success rate of RLBench We adopt LoRA finetuning [39, 40] on the vision encoder. The original weights of the ViT-B vision encoder are kept frozen. The convolution head has only 96K parameters, less than 0.1% of the entire network. The structure of the convolution head is illustrated in Figure A4 . We train the VKT for 50000 steps with a batch size of 96, which takes roughly 22 hours on a dual NVIDIA A6000 GPU server. The inference throughput is 0.02 secs/img on a NVIDIA 3060 GPU.Baselines. We compare the performance of our visual kinematics transformer (VKT) with a behavioral cloning transformer (BCT). The BCT is directly trained to predict the robot actions such as 6-DoF end-effector poses and joint values. In contrast, VKT is trained to forecast visual kinematic chains first, then project the kinematics tokens to robot actions using a convolution head with the backbone frozen, as shown in Figure 4 . Note that the BCT shares the same neural architecture with VKT, including the convolution head for each specific environment. The specialized agent is trained on each environment separately. The general agent is trained using demonstrations from all environments. We use the 6-DoF end-effector pose and binary gripper states as the robot action space for both Calvin and RLBench, and the radian values of the 14 robot joints for ALOHA. We also compare the performance of our VKT with existing environment-specific methods. In Calvin, recent methods go beyond our imitation learning setup and adopt model learning for long-horizon latent planning. Therefore, we choose the goal-condition behavior cloning (GCBC) baseline from [41] . In RLBench, we choose PerAct [3] and retrain the model with the same camera setup as our VKT. In ALOHA, we choose the action chunking transformer (ACT) from the origin ALOHA paper [35] .Results. Table 1 and A3 summarize the comparison between VKT, BCT, and environment-specific methods. By forecasting the visual kinematic chains, VKT outperforms BCT in both specialized and general agents. This indicates that the kinematics tokens capture sufficient information about the robot's action, despite not seeing any low-level robot actions. When trained in all environments, the general BCT agent suffers a catastrophic performance drop (1.36 → 0.48, 36.4% → 24.3%, 55.3% → 12%). This indicates the challenge of policy learning from different action spaces. In Failure Case Analysis. We list the per-task success rates of RLBench and Calvin in Table A4 and A5. Overall, VKT outperforms BCT. We empirically find that VKT is more robust to changes in object layout because of the visually grounded action space. However, VKT has a lower success rate in the "turn tap" task. This task requires some specific end-effector rotations which involve large variations in the Euler angles of the end-effector but small changes in the visual kinematic chain from all camera viewpoints, as illustrated in the fourth row of Figure A5 . Figure A5 summarizes the four categories of failure cases of VKT.Setup. Open-X Embodiment (Open-X) [5] is a collection of video demonstrations of real-world manipulation tasks crowd-sourced from diverse environments. Since Open-X does not provide robot models such as URDF, and camera parameters are not available in most datasets, we select four datasets from Open-X and manually annotate the robot joint positions from 80 demonstration videos with 1, 1359 frames. In future work, it is possible to apply robot keypoint detectors [42, 43] to extract the kinematic chain without URDFs. Using the annotated videos, we train a single VKT for the four datasets. Then, we train the network to predict low-level robot actions as in Figure 4 . However, instead of freezing the VKT, we continue finetuning the entire network with the trained VKT as initial weights. For the BCT, we use the same training setup but randomly initialize the network.Results. Figure 6 shows qualitative examples of the predicted kinematic chains (left) and compares the relative precision of BCT and VKT on the validation set (right). Let L vkt and L bct denote the L1 error of VKT and BCT, the relative precision of VKT is computed as |L vkt -L bct |/L vkt . Figure 6 shows that our VKT learns to predict kinematic chains from diverse real-world data, and visual kinematics forecasting can serve as a pre-training objective to improve imitation learning.Setup. We evaluate our VKT in a real-world language-conditioned pick-and-place task. The task input includes an RGB image and text instructions such as "put the cup into the blue box". We adopt nine YCB objects such as balls, chips, and two boxes as place targets. Examples of this task and the forecasted kinematic chains are shown in Figure 7 . We implement a scripted policy using a DINOv2-based object detector [44] to collect 90 demonstrations as training data, with only 5 demonstrations for each object-container pair. We automatically compute the ground-truth visual kinematic chain with the robot's URDF and the calibrated camera parameters. We use a Kuka LBR iiwa robot. For the convolution head, we train the model to directly predict the pick and the place poses instead of predicting the full trajectory for simplicity, and use MoveIt [45] for path generation.Results. Table 2 shows the success rates of VKT in the real-world task. The overall success rate is the average of 65 independent runs with different object layouts. The BC Transformer (BCT) shares the same training and evaluation setup with VKT, but VKT outperforms BCT by a significant margin. Further, we discover that visual kinematic forecasting enables the use of 2D image augmentation for manipulation learning because the action space also resides in image planes, which is important to prevent overfitting in our experiment due to the small training set. We randomly augment the images through size scaling, translation, rotation, and perspective transform. In comparison, existing manipulation learning augmentations are applied in the 3D space [46] , which requires depth that is less available than RGB data, and has fewer categories than 2D image augmentations [47] .Details of analyses and ablation studies are included in Appendix A.1 due to space limit. 
discussion and future possibilities: 
 We have shown that visual kinematics forecasting of quasi-static robot movements can be a building block technique for developing general agents across diverse robot learning environments. In the following, we discuss the limitations and potential future directions.3D Kinematic Chain. In Figure A5 , VKT shows an inferior performance when the desired robot actions only produce small visual changes in the kinematic chain. This discrepancy is caused by the ambiguity of the 3D to 2D projection and prohibits VKT from more dexterous or precise tasks. One possible solution is to project the 2D kinematic chain back to a normalized 3D space with a virtual camera. This ensures the 3D space is consistent across different environments, provides the necessary precision in the 3D space, and still be compatible with our 2D kinematic chain method.General Visual Encoding of Robot Actions. From an encoding perspective, the visual kinematic chain can be viewed as a visual encoding strategy for robot actions. The benefit of visual encoding is that it transforms robot learning tasks into vision tasks, where more generalized and robust solutions are often discovered. One possible future direction is to explore more visual encoding strategies with better learnability, higher precision, and reliable or even analytical translation into robot actions.Beyond Universal Action Representation. Despite our advancements in unifying the action spaces across diverse environments, there is still a considerable gap in performance between our general agent VKT and those environment-specific state-of-the-art solutions. Different environments not only differ greatly in action spaces but more so in types of tasks and the required robotics skills. These skills are acquired separately by those environment-specific solutions. It is a potentially promising future direction to not just focus on universal action representation, but designing a universal learning architecture that can acquire various robot skills for diverse environments.A.1 Ablation Studies Predicting the Full Chain versus Predicting Only the End-Effector. We compare the effects of forecasting the full chain and only the end-effector in Table A1 . Table A1 shows that forecasting the full chain improves the performance on all environments and especially the bimanual manipulation ALOHA benchmark. Moreover, despite the reduced capacity to represent rotations, end-effector VKT also delivers stronger performance as a general agent than BCT. This indicates that encoding robot actions visually in the image plane is the key to countering distribution shifts from different action spaces when building general agents across diverse environments. Furthermore, the endeffector predictions can be useful in special cases when robot models are unavailable. This suggests a broader applicability for visual kinematics forecasting. VKT Performance and Imitation Learning Convergence. We study the impacts of VKT performance on imitation learning convergence for the low-level robot actions in Figure A1 . We train the VKT for a different number of steps. Next, we use the pretrained VKTs as backbones and train the convolution head for Calvin. Figure A1 demonstrates a strong correlation between the VKT pretrained steps and the convergence quality of imitation learning on robot actions.Earth Moving Distance versus Chamfer Distance. Figure A2 compares the predicted visual kinematic chains using our proposed EMD and Chamfer Distance [48] . Unlike EMD, Chamfer Distance is not density-aware and only minimizes the distance between each point and its nearest neighbor.Figure A2 shows EMD produces fuzzier but more accurate kinematic chains, but Chamfer Distance produces sharper but sparser kinematic chains that lack structural integrity.Percentages of Training Data for the Convolution Head. Figure A3 shows the average lengths on Calvin with the convolution heads trained on different data percentages. Similar levels of performance can be obtained with 5% data, or even 1% data at a minor performance drop. Calvin includes 34 tasks and 5123 episodes as the training set for D → D challenge. 1% data only leaves roughly 1 episode per task. This shows that kinematics tokens already encode robot action information. Only a small amount of data is needed to train a decoder to map kinematics tokens to actions.Multi-View versus Single-View Predictions. We study the impacts of multi-view forecasting in Table A2 . We remove the multi-view attention layers to create the single-view VKT. We average the low-level robot actions predicted from each view as the final prediction. Table A2 shows that VKT improves its 3D understanding significantly with our multi-view dual attention block by predicting consistent kinematic chains in different camera viewpoints. 8th Conference on Robot Learning (CoRL 2024), Munich, Germany. 
 
 Appendix:  


Decision: This paper introduces the Visual Kinematic Chain as a more general action space to scale up learning in robotic manipulation. The reviewers all recognize the novelty and importance of this approach.

Before the rebuttal, the reviewers raised several concerns. The experimental analysis is criticized for unfair comparisons and lack of detailed analysis. The paper does not clearly show the quantitative benefits of predicting the entire kinematic chain versus just the end-effector. Additionally, the claim of bridging different action spaces is not well-supported, and the reason for VKT's improvement over BCT is unclear.

The evaluation methods and metrics also need clarification, especially the success length metric. Important details are missing, and the reliance on URDF files might limit generalizability. Comparisons with other action spaces like track points and video are insufficient.


During the rebuttal, the authors did an excellent job addressing the reviewers’ concerns. As a result, all reviewers now rate the paper as a Weak Accept. However, Reviewer t5uW still has concerns regarding the explanation of Figure 6 and the claims about performance improvements. Nonetheless, the reviewer acknowledges that the overall idea is interesting and valuable for the field. I strongly encourage the authors to consider the reviewers’ feedback both during and after the rebuttal to strengthen their paper.

I’m including Reviewer t5uW’s comment from the discussion below.

========

Thank you to the authors for their response.

One of my main concerns previously was the ablation study in Table A1 which previously showed no improvement for forecasting the full kinematic chain instead of only the end-effector. I stated that “the success rate for predicting the full chain is 55.5% and 54.1% for predicting the end-effector. It’s unclear for this table how many trials were used, but assuming 1000 trials were used, the p-value for this result is 0.5 which is not statistically significant.” I’m also not convinced that for the other environment a difference in average length of 1.24 vs 1.46 is a meaningful improvement. The authors need to perform a statistical comparison, or at the very least, provide a measure of variance of their reported performance. Despite this, the authors kept the claim in their revised draft that “forecasting the full chain improves performance on all environments” which I strongly disagree with given the p-value analysis I provided. The authors should remove the sentence in lines 411-413. I appreciate the new results on the ALOHA environment that show a meaningful improvement in performance for predicting the full kinematic chain. However, this means that predicting the full kinematic chain only leads to improvement in 1 out of the 3 tasks studied and no explanation was provided on why that is. The paper would be stronger if more analysis was provided to explain when and why predicting the full kinematic chain actually helps.

The authors’ response to my other concerns in W3 were helpful and I agree with their comments.

My other major concern are the results presented in Figure 6. I asked for more explanation from the authors for this figure, but the additional explanation provided merely repeated what was already said. In its current state, Figure 6 does not make sense. The text says that Figure 6 shows that VKT can predict kinematic chains from real world data. But all Figure 6 shows is a comparison to the BCT baseline.

The “relative precision” metric does not make sense — according to 4.1.2, it’s a relative comparison between two methods. By this definition, VKT and BCT could both be equally terrible. No data is provided to evaluate the absolute performance of either method, therefore a comparison between them is not helpful.

Also, if relative precision is indeed defined as in 4.1.2, it doesn’t make sense to show BCT and VKT as separate bars. Isn’t the whole point of the relative precision metric to produce a single scalar comparing the two methods?

The text in the authors’ rebuttal also does not make sense to me.

> “Figure 6 compares the relative precision (an inverse of L1 error on regression of robot actions, explained in lines 234-235) of BCT and VKT on validation splits of the selected datasets.”

- “relative precision” as defined in 4.1.2 is not the inverse of L1 error.

> “It demonstrates that visual kinematics forecasting, as a pre-training objective, contributes to the accuracy of action regression.”

- The authors do not show this. Are they trying to argue VKT is better than BCT? The actual paper text does not make this claim, and the figure does not support it.

I am clearly still missing what the authors are trying to show. The current figure and text in 4.1.2 do not make sense together even after the authors attempted to clarify, in either the rebuttal or the text. If the goal is for the authors to show that VKT can produce kinematic chains, why do they not simply report L_vkt?

This is unfortunate because Figure 6 is fairly incidental to the overall story the authors are telling. I am fine with accepting this paper if the authors can either explain Figure 6 in a coherent fashion or simply delete it from the paper.

I am changing my evaluation to a weak accept because I think the ideas proposed in the paper are interesting and valuable for the field and the ALOHA results have strengthened the paper, but given my remaining concerns on the quality of the execution of the paper, I would be very hesitant to publish this paper unless

(1) the authors can coherently explain Figure 6 (or simply delete it)

(2) the authors remove the claim that a success rate of 55.5 versus 54.1 is meaningful in Table A1 (lines 411-413). It is simply not.

Reviews: review: title: title: A good paper that might needs more clarification and analysis
paper_summary: paper_summary: This paper proposes the visual kinematics chain as a novel action representation to enable manipulation policy training on diverse data with various embodiment and environments. The authors also introduce the Visual Kinematics Transformer (VKT) to predict kinematics structure and use it to generate actions for downstream tasks.
main_review: main_review: - strength:
  - The proposed representation and method are novel and interesting
  - There are sufficient experiments demonstrating the effectiveness of this method
  - The writing is clear and easy to read overall.
- weakness:
  - Potential unfair comparison. The authors compare the specialized agent and the general agent in the paper. For the general agent, the training details are not quite clear. My current assumption is:
    - General VKT: train VKT on all tasks and train the tiny head with one task (as described in L135).
    - General BCT: train BCT on all tasks.

    If it is true, I think the general agent comparison is unfair. Because BCT can only have one model to consume all data, while VKT can train separate tiny heads for each task. If my assumption is wrong, please also clarify it.
  - More analysis is needed for experiments. Table 2 lists the success rate of different tasks. However, the analysis is not clear. It could be more insightful if the authors could summarize the failure mode of BCT and VKT. For example, one failure pattern of VKT is the motion requiring rotation. There are also some interesting data. For example, BCT performs drastically worse on "put in drawer". Analyzing why VKT performs better could be interesting.
  - Tasks it can accomplish are still preliminary, such as pick-and-place and pour. Extending this framework to more dexterous tasks could be more impressive.
- comments:
  - Table 2: The layout can be a bit confusing at first glance. I thought it was comparing "Specialized BCT" vs. "Agent VKT" or "General BCT" vs. "Agent VKT". Maybe "Specialized Agent" and "General Agent" can be summarized as "Specialized" and "General". Then they should be aligned vertically.
  - All image captions and table captions will be easier to notice and read by adding an image or table summary in **bold** fonts. Now captions look a bit mixed with the main text
questions: questions: - Has the authors tried different losses for VKT? Is earth-moving loss significantly better than other loss, like Chamfer distance?
- Is N (the number of predicted points) a fixed number?
- For specific environment deployment, how much data is needed to train the tiny head?
- How scalable this method is with respect to the number of camera viewpoints? For example, why only three viewpoints out of five are chosen in RLBench?
- Is it possible to raise visual kinematics to 3D to resolve the rotation motion problem?
limitations: limitations: 3
review_summary: review_summary: This paper is overall a good paper and novel. If authors could clarify their experiment setting and add more experiment analysis, I lean to accept it.
title: title: Review for VKT
paper_summary: paper_summary: Action spaces are frequently not aligned between different robotics datasets, and this paper proposes a method to address this problem. Instead of training a model to predict the 6DOF pose of the end-effector, they train a model to predict, in pixel space, the motion of the kinematic chain of the robot. This kinematic chain can be obtained using the robot’s URDF and a calibrated camera. The performance of this method is evaluated both in simulation and on real hardware and is compared to a BC transformer baseline that predicts the robot’s end-effector pose directly.
main_review: main_review: This paper addresses an important problem, and shows positive results with their method in certain cases. However, I have several concerns about the paper: 

The paper includes an important ablation study to investigate whether there really is a benefit to predicting the motion of the entire kinematic chain of the robot vs just predicting the motion of the end-effector of the robot in pixel space. The results of this study show that the success rate for predicting the full chain is 55.5% and 54.1% for predicting the end-effector. It’s unclear for this table how many trials were used, but assuming 1000 trials were used, the p-value for this result is 0.5 which is not statistically significant. Therefore, the paper does not show any quantitative benefit to predicting the entire kinematic chain of the robot relative to the end-effector, which significantly weakens the paper given that the paper’s main emphasis is on predicting the full kinematic chain.

Additionally, the paper motivates the proposed approach by stating the need to “bridge the gap between different action spaces” in different datasets. However, for the simulation experiment considering the CALVIN and RL bench environments, the paper states that they both use the 6DOF end-effector pose as the action space. I don’t understand how this simulation study is able to study how well this method bridges the gap between different action spaces without having different action spaces. It seems instead like the main difference between the two environments is that the cameras are in different locations and the scenes and robots look different. 

The paper shows significant improvement using the VKT model over the BCT model for the distribution shift between the two environments in the general agent case. This is fairly surprising given that predicting the full kinematic chain is not any better than predicting only the end-effector motion and only predicting the end-effector motion (which performs as well as VKT with kinematic chain prediction) does not capture rotations. How can a method that cannot capture end-effector rotations perform better than one that can? Additionally, while both VKT and BCT have the same architecture, in the current implementation, only VKT gets access to a specialized head that has been finetuned on each environment separately. This specialized head has learned to convert actions in the camera frame (i.e. pixels in the image plane) to the robot’s frame. Therefore, an important baseline to evaluate VKT vs BCT would be to have BCT predict actions in the camera frame and then convert those actions into the frame of the robot (which is straightforward to do with known camera parameters which is an assumption VKT makes) before passing them to the robot.

The success length metric is confusing. The paper cites reference [28] as an example of reporting number of successful steps per episode, but I was not able to find where this metric is reported in this paper. If the success length is the average number of successful steps per episode, why is the average close to 1 or less than 1 for all methods? This seems very low. What is the length of the demonstration trajectories? When reporting this metric, the standard deviation of the average length should also be reported to evaluate statistical significance of the results.

Each results table needs to clearly indicate how many evaluation trials were used for each experiment. The current paper has conflicting information on this topic. For example, in Table 2, the caption states that “success rates are measured from five independent runs.” However, the success rates reported in Table 2 have three significant digits, which is impossible to obtain from 5 trials. Not knowing how many trials were actually used in each experiment makes it difficult to properly evaluate the statistical significance of the results. 

The bar chart on Figure 6 is difficult to interpret. The text needs to explain this plot in more detail.
questions: questions: Lines 215 and 216 refer to 2D image augmentation for manipulation learning. What augmentations are used?
limitations: limitations: 3
review_summary: review_summary: The paper addresses an important problem and shows some positive results. The emphasis on predicting the entire visual kinematic chain despite the negative results of the ablation study is very confusing. The paper needs some clarification on some of the results as described in the main review.
title: title: Intersting and novel idea for an universal action space for multi-embodiment robot learning with lacking experimental evaluation and too short limitation section..
paper_summary: paper_summary: The paper introduces Visual Kinematic Chains as a universal action space for multi embodiment robot training. The main idea is to track the kinematic chain of the robot arm that is robot and end-effector agnostic and can be pre-trained on diverse robot datasets such as Embodiment. They use a transformer to track the kinematic chain and for domain adaption finetune a small action head. The proposed method is evaluated on several simulation and real robot environments.
main_review: main_review: Strengths

- The proposed idea is novel and creative and tackles a relevant  problem in scaling robot learning with heterogeneous data consisting of various action spaces 
- One big plus is the easy adaptability towards setups with multiple cameras and it also does not require depth 
- several experiments in sim and real robot setups as well as general visualization on OXE show the performance of VKT to better scale than default BC Transformer 
- Illustrative figures and website to demonstrate the proposed method 

Weaknesses

- The method requires URDF files to get relevant kinematic information. In the OXE experiments, the authors already mention that they are not able to train on more data since this information is missing. 
- the CALVIN results are very low compared to sota models such as MDT [1] for CALVIN D or 3DDA [2] for CALVIN ABC benchmarks. An average rollout length of 1 is not good on any of these benchmarks and does not provide any arguments for the proposed method. Further, it is not mentioned which dataset split the authors use for their experiments on CALVIN. In addition, no baselines are shown for RL-bench experiments. Adding some relevant 2D baselines would underline the effectiveness of the proposed method. 
- the proposed method is not tested against other universal action spaces used in recent works such as: track points [3], video [4] and is only tested against default BC. These makes it hard to evaluate how well the proposed method compares against similar methods. Overall, relevant prior work is missing in the Intermediate Action Representation Section of the paper. 
- No information about training time and inference speed
- the limitation section is not very detailed and is missing information


[1]: Reuss, Moritz, et al. "Multimodal Diffusion Transformer: Learning Versatile Behavior from Multimodal Goals." RSS 2024

[2]: Ke, Tsung-Wei, Nikolaos Gkanatsios, and Katerina Fragkiadaki. "3d diffuser actor: Policy diffusion with 3d scene representations." _arXiv preprint arXiv:2402.10885_ (2024).

[3]: Wen, Chuan, et al. "Any-point trajectory modeling for policy learning."  RSS 2024

[4]: Du, Yilun, et al. "Learning universal policies via text-guided video generation." _Advances in Neural Information Processing Systems_ 36 (2024).
questions: questions: - How would you try to solve the issue of missing URDF files to enable training VKT on OXE on more diverse data for future work? 
- Can you add some relevant baselines mentioned in the weakness section?
- Why is the average performance of VKT on CALVIN so low compared to other methods? 
- Did you ever try to combine your kinematic chain prediction with a separate more powerful low-level transformer policy as used in [1,2] and only use the kinematic chain as a visual conditioning for the low level policy? Similar how [3] does it with their track conditioning? While I understand the idea to have a single model for everything, I believe that the performance of the proposed method can be enhanced a lot with a better low level policy. In my experience a small action head is not enough for challenging benchmarks as used in the paper.


—-

Post rebuttal 

The added experiments and the updated limitations section further strengthen the paper. While the performance gap to specialist models remain on some benchmarks, I believe it’s mostly related to architecture design decisions and are not related to the proposed method itself. Overall, I believe the paper has timely and relevant contributions for generalizable action spaces that are interesting for the wider robot learning community. Thus I changed to weak accept. For a strong accept the authors would have required better performance across benchmarks. However, I believe the paper is still timely and relevant.
limitations: limitations: 2
review_summary: review_summary: Overall, I really like the creative and novel contributions of the paper and appreciate the evaluation across several benchmarks. However, missing relevant baselines and low performance on sim environments and the limitation section are open issues, that need to be addressed. Thus, I only recommend weak reject for now.