Kati8/ICLR_2023_2025 huggingface dataset:
authors: Po-Chen Ko
authors: Jiayuan Mao
authors: Yilun Du
authors: Shao-Hua Sun
authors: Joshua B. Tenenbaum
reviews: 
Comments: The paper presents a method called Actions from Video Dense Correspondences (AVDC) that learns to perform robotic tasks without ever accessing labels of robot actions. The method first trains a text-conditioned diffusion video prediction model on videos of robot data. It then computes optical flows for adjacent frames of the prediction, and by segmenting out the object of interest and finding a rigid body transformation that best fits the corresponding points, computes an object (manipulation) or robot (navigation) transformation that enables the robot to follow the plan. Experiments are conducted on several environments: simulated meta-world and iTHOR tasks, visual pusher for evaluating learning from a human embodiment, and bridge data/a real Panda arm.
Strengths: - Learning from action-free videos is an important and challenging problem, which could enable data-driven robotics to access scale through sources like Youtube.
- The proposed method makes sensible design decisions, including the video model and correspondence computation strategy. 
- The evaluation is very thorough in terms of the number of environments considered, and AVDC appears to yield consistent performance gains.
- The authors have committed to making their video model, which requires significantly fewer resources than prior works, open source. I think this is also a valuable contribution (although less related to the main message of the work).
- The presentation is quite good, the writing is clear, and main ideas are communicated directly.
Weaknesses: - The most apparent weakness of this work is that not all tasks that a robot might want to solve can be solved by a trajectory of target object poses. For example, it is unclear how to plan a task that would require the robot to use another object as a tool. It is also unclear how deformable objects could be handled. There also may be tasks such as pressing a button on a microwave, which do not involve the microwave moving, but require a particular amount of force to be applied, for which this may not be applicable.
- The work “Zero-Shot Robot Manipulation from Passive Human Videos” by Bharadhwaj et al. presents very similar (although not identical) ideas and is not discussed in the prior work or cited. Could the authors discuss and ideally perform a comparison to some of the ideas in that work?
- Related to the previous point, the ideas presented in this paper are not entirely novel. However, I believe that this particular combination/instantiation of them, as well as the evaluation and exploration of them that is provided, is a valuable contribution to the community.
Recommendation: 6: marginally above the acceptance threshold
Reviewer confidence: 4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.
Soundness/correctness: 4 excellent
Clarity: 4 excellent
reviews: 
Comments: The paper introduces a method for constructing a video-based robot policy, capable of performing diverse tasks across different environments. This approach doesn't require action annotations but uses images for a task-agnostic representation. Text is employed for specifying robot goals. By synthesizing videos to predict robot actions and employing dense correspondences between frames, the model infers actions without explicit training labels. It can leverage the large-scale RGB videos on the internet for training, and use this knowledge for robotic manipulation. The paper showcases the effectiveness of this approach in tabletop manipulation and navigation tasks and also provides an open-source framework for efficient video modeling.
Strengths: 1. The paper proposed a new correspondence based method to obtain robot action in forecasted robot videos. It proves that a latent dynamic model is not needed if the forecasted video has good quality.
2. The authors proposed a new method to generate future videos using a diffusion model, which achieves efficient training. It provides a promising toolbox for the community.
3. The method is evaluated on two tasks, table-top manipulaion and in-door navigation, demonstrating its effectiveness in different domains.
4. The paper is well-written and solid.
Weaknesses: 1. The selected robot tasks are relatively toy, and the potential of such kind of video prediction method is not evaluated. However, this is not the weakness of this paper, but a common practice for video prediction based robot control.
Recommendation: 8: accept, good paper
Reviewer confidence: 4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.
Soundness/correctness: 3 good
Clarity: 3 good
reviews: 
Comments: The goal of this paper is to learn robot policies from action-free video data. The motivation is that there exists a lot of video data, but very little action data. Video prediction methods are often over-dependent on actions but have the benefit of being task agnostic. AVDC aims to solve this challenge by learning a video generation model (via diffusion) on the robot video data. From the generated sequence of images, the optical flow is estimated, which conditioned on some 3D knowledge as well as masks/segmentations of different objects gives an idea of the actions that are taken. The robot actions are then taken. To avoid accumulating errors, AVDC allows for replanning after a few actions. The approach is tested on manipulation (Meta-World) and navigation (iThor) setups, as well as qualitative results on a cross-embodiment visual pusher dataset and some robot arm data. Experiments and ablations find that (1) AVDC outperforms inverse dynamics+video prediction and BC baselines (2) all individual components are important.
Strengths: - The paper tackles an important problem of learning from action-free videos 
- The method, to my knowledge is novel
- The approach significantly outperforms baselines on many different tasks 
- The ablations are well analyzed 
- The paper is easy to follow and well written
Weaknesses: - I think one of the main limitations is the setting: AVDC needs videos of robots performing the task. I believe this is a contrived setting as it is very likely that if video + 3D information is available, then this was a robot demonstration, and one can just collect action data. To me, it is unclear how this approach will scale beyond robot data. 

- I am concerned by the reported results for the BC baseline. Due to action data being available, as well as the robot data being in-domain for the task a simple BC or kNN baseline should work very well. There are many cases where the results are < 5% success. This should be addressed. I would be willing to increase my score if this weakness is addressed. 

- AVDC relies on object/robot masks - a simple baseline would be to use those as a proxy for the actions. One could get pseudo action labels from the videos and train a policy. 

- AVDC assumes that all objects are going to be directly manipulated by the robot directly but this is not the when one object as a tool. 

- Navigation approaches have many action free baselines which should be explored as well

- It would be good to see real world experiments

- It would be good to have more of an analysis on the quality of the video prediction model. I suspect it has a
Recommendation: 6: marginally above the acceptance threshold
Reviewer confidence: 4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.
Soundness/correctness: 3 good
Clarity: 3 good