Dreamitate: Real-World Visuomotor Policy Learning via Video Generation

Venue: robot-learning_org_CoRL_2024_Conference
Published: 2024-05-20 19:37:56

Abstract: A key challenge in manipulation is learning a policy that can robustly generalize to diverse visual environments. A promising mechanism for learning robust policies is to leverage video generative models, which are pretrained on large-scale datasets of internet videos. In this paper, we propose a visuomotor policy learning framework that fine-tunes a video diffusion model on human demonstrations of a given task. At test time, we generate an example of an execution of the task conditioned on images of a novel scene, and use this synthesized execution directly to control the robot. Our key insight is that using common tools allows us to effortlessly bridge the embodiment gap between the human hand and the robot manipulator. We evaluate our approach on 4 tasks of increasing complexity and demonstrate that capitalizing on internet-scale generative models allows the learned policy to achieve a significantly higher degree of generalization than existing behavior cloning approaches.

Main paper: 
 introduction: 
 Learning visuomotor policies that decide actions from perception has been a longstanding challenge in robotics because they need to generalize to diverse physical situations [1] . Recently, behavior cloning from human demonstrations [2, 3] has become the method of choice for many manipulation tasks, where visuomotor policy learning is formulated as a regression problem to map visual observations to actions supervised by human demonstrations. However, behavior cloning requires ground truth robot actions, which makes scaling to diverse situations challenging.We aim to learn generalizable visuomotor policies by leveraging video generation. Trained on large-scale datasets of internet videos, video generation provides a promising avenue for generalization because extensive priors from human behavior can be transferred into robot behavior. Many recent methods have also explored how to capitalize on these priors, such as synthesizing videos of human behavior [4] or synthesizing videos of robot behavior [5] . However, while videos of people capture the diversity of behavior, generated human actions are difficult to transfer to robots due to the embodiment gap. While directly synthesizing videos of robots would be more realizable physically, the scale of available data is many orders of magnitude smaller than videos of humans, making the learned policy less robust in an in-the-wild environment.How else can we leverage video generation for policy learning? In this work, we introduce Dreamitate, 1 shown in Figure 1 , a visuomotor policy that controls robot behavior via conditional video generation. Our key insight is that the end-effector is the most important part of the embodiment for manipulation: its interaction with diverse objects is challenging to model explicitly and can benefit most from common sense knowledge from video generation. The rest of robot embodiment can be solved through inverse kinematics. Given a visual observation, our policy synthesizes a video of a person performing tasks with a tool. To translate into robot behaviors, we 3D track the tool in the synthesized video, and simply transfer the trajectory into explicit robot actions.Our formulation offers several major advantages compared to traditional visuomotor policies:• Generalizability. Our underlying video generation model is pretrained on a massive amount of videos available on the internet, including a wide variety of human manipulation in all kinds of environments. By formulating policy learning as video model finetuning, the prior knowledge learned from pre-taining can be preserved, allowing our model to learn generalizable manipulation skills with the same amount of demonstration data.• Scalability. Our finetuning videos are recordings of human demonstrations instead of teleoperation, making our dataset collection more scalable.• Interpretability. During inference time, our video model predicts future execution plans in the form of videos before the actual robot executions are performed. Compared to a black-box endto-end policy, our formulation offers an intermediate representation of policy that is interpretable to humans, a key feature for human-robot interaction applications.We evaluated our approach across four real-world tasks, including bimanual manipulation, precise 3D manipulation, and long-horizon tasks, using a small number of expert human demonstrations. We found that the video model consistently outperformed the baseline behavior cloning model in generalizing to unseen scenarios. Additionally, we analyzed the model's performance when scaling down the training dataset and found it maintained strong generalization performance even with fewer demonstrations. We will release the code and data for reproducing our results. 
related work: 
 Behavior Cloning. Behavior cloning (BC) has emerged as a leading approach in manipulation tasks, learning policies from demonstrations through supervised learning. Early BC efforts involved end-to-end models mapping states to actions, but struggled with multimodal behaviors and highprecision tasks [6, 7, 8] . Subsequently, Energy-Based Models (EBMs) were explored, predicting actions by minimizing energy in sequence optimization [9, 10, 11, 12] . Recently, conditional generative models have shown promise in capturing multimodal demonstration behaviors, enhancing task success rates [2, 3, 13] . Different from previous models, we integrate video prediction and 3D tracking to predict action prediction.Visual Pretraining for Policy Learning. Prior works have extensively explored various ways to pre-train the perception model in a visuomotor policy for learning more robust visual representation. One of the most popular pertaining objectives has been video prediction [14, 15, 16, 17, 18] . By learning to predict future based on the current observation, a model can learn the dynamic and causality of the world crucial for physical interaction. Contrastive learning [15, 19, 20, 21] as well as masked autoencoding [22, 23, 24] are two very popular self-supervised learning objectives for learning visual representation for robotics. Besides, another line of work [25, 26, 27] studies learning a generalizable reward function through visual pertaining for reinforcement learning.Video Models for Decision-Making. More recently, in light of the fast progress of text-tovideo generative models [28, 29, 30, 31, 32] , the hope to leverage internet-scale video pertaining for robotics is rekindled [33] . A line of work uses video generative model as a world simulator [34, 5, 28] , which can predict future video conditioned on an action. Another line of work uses video-language model for longer-horizon planning [35, 36, 37] . Different from prior works on this topic which typically use video prediction models as a world model or simulator in order to perform planning and decision making, Dreamitate uses video prediction model as part of a visuomotor policy and directly uses the predicted video for action prediction. 
overview: 
 Robot Tools Human Tools Push-Shape (Fig. 7 ) Sweeping (Fig. 6 ) Scooping (Fig. 5 ) Rotation (Fig. 4 )Figure 3 : Aligning Human and Robot Behavior via Trackable Tools. We use tools with known CAD models for human demonstrations to facilitate precise 3D tracking across four real-world manipulation tasks. We design unique tools (e.g., a spoon for scooping) to show the generalizability of our approach for varying tasks.Given a visual observation of the scene v 0 , our goal is to plan and execute robot actions a t ∈ SE(3) via video generation. We approach this problem through the framework, visualized in Figure 2 :a t = T (v t ) where {v t } = f θ (v 0 )where f θ (•) is a generative video model with learnable parameters θ. We train f to generate videos of a human doing the task using a tool that we can track in 3D. T tracks the trajectory of the tool in the generated video, which we can directly use to control the end-effector state to perform manipulation tasks. This composition of synthesize-then-track facilitates cheap data collection without teleoperation, and aligns with pretaining video data, which is composed of mostly videos showing human behavior.Starting with a pre-trained video generator, we fine-tune it on a small video demonstration dataset in order to combine the priors from internet videos with behaviors from demonstration videos. To collect videos for fine-tuning, we build a tabletop setup with two calibrated cameras positioned 45 degrees apart for better visibility. We capture several pair of stereo videos (v 1 , v2 ) ∈ V of humans performing the task using a 3D printed tool (see Figure 3 for details).We use these demonstration videos to fine-tune the video generative model. Since we will imitate the trajectory of the synthesized tool in the physical world, we generate stereo videos to make action possible in 3D. Let (v 1 t , v2 t ) be the predicted stereo video frames at time t and (v 1 t , v 2 t ) be the ground truth stereo frames from demonstration videos. We optimize the video prediction objective:EQUATIONWe initialize θ to the pre-trained weights learned from large-scale internet video datasets (Stable Video Diffusion [29] ). We train a separate f θ for each task (e.g. sweeping or scooping).Following the implementation from [38] , the encoder and decoder are frozen such that only the spatial/temporal attention layers are fine-tuned. The per-frame image embedding input to the model is modified based on the viewing angle of the output frame to facilitate stereo video generation. This ensures that the first half of the frames generate the first view and the half generates the second view, using the initial image embedding from the respective view.At test time, we will be given a stereo image pair of the scene that we wish the robot to manipulate. We apply the fine-tuned video model f (v 0 ) and obtain a generated stereo video frames vt of the task being performed.The synthesized video frames {v t } serve as the intermediate representation of our policy to obtain an action trajectory, {a t }, that we then execute on the robot. Actions a t ∈ SE(3) are represented as the 6D pose of the tool relative to the camera. By using a known CAD model, we can efficiently and accurately track the tool in each frame of the generated videos and therefore obtain precise locations of the tool. Each action a i in the action trajectory corresponds to one frame from the synthesized video. For 3D consistency, we use the generated stereo pair, along with the calibrated camera parameters to resolve 6D pose of the tool. For tracking the tool from an RGB image, we use Megapose [39] and operate on 768 × 448 resolution video frames with camera intrinsic derived from the default values of the Intel Realsense camera. Finally, during execution, the customized tool is mounted on the robot arm and actions a 0 , ..., a T are executed by the robot with the trajectory smoothed using impedance control. The entire data-collection, training, and fine-tuning pipeline are agent-independent. This is possible because we use tool's as manipulators and track known tools in generated videos, removing the need for learning any agent-specific action spaces. 
experiments: 
 In this section, we evaluate the performance of our video model in real-world robot manipulation tasks. The evaluation, summarized in Table 1 , encompasses four distinct tasks including object rotation, granular material scooping, table top sweeping and shape pushing (see Figure 3 for a visualization of the used tools). The details of each task are further explained in the corresponding sections below. For an assessment of our model's capabilities, we benchmark its performance against Diffusion Policy [40] , Action Chunking Transformer (ACT) [3] and Vector-Quantized Behavior Transformer (VQ-BeT) [13] as baselines.To evaluate the model's generalizability, the data was collected by a human demonstrator at different locations for training and evaluation using the same camera setup, though the exact tabletop and lighting conditions varied. To further test the model's robustness in real-world scenarios, we ensured that the sets of objects used in training and evaluation are non-overlapping. To initialize tool tracking with Megapose [39] , background subtraction and hand removal based on skin color were employed to obtain the tool bounding box. In rare cases where this process failed, human corrections were applied to the bounding boxes (both for our method and for the baseline).We used the same training data to train the baselines with stereo image input. The videos were preprocessed using Megapose for tool tracking, providing target trajectories for training. For Diffusion Policy we used a pretrained ResNet-18 variant [40] as the baseline, as the pretrained CLIP encoder [41] variant showed lower performance in our tasks. We trained the baselines to predict the entire action trajectory for the next 12 time steps without replan as an open-loop system, similar to the video model. Task. We design a rotation task, shown in Figure 4 , to test our policy's ability to coordinate endeffectors and choose appropriate grasping points to manipulate real-world objects. In each training demonstration, we randomly place an object on the table, and use a gripper, shown in Figure 3 (top), to grasp the object at appropriate points and rotate it counterclockwise up to 45 degrees.Setup. We collect training data with 31 objects, with 14 real-world objects (such as boxes or a hammer) and 17 custom colored shapes made out of foam. We use 10 unseen real-world objects for We compare our method to ACT, VQ-BeT and Diffusion Policy on four tasks quantitatively. We report success rates (successful trials / total trials) for rotation, scooping, and sweeping tasks. For Push-Shape, we report mean intersection-over-union (mIoU) and average rotation error (Rot. Error). Our approach performs well across all tasks, whereas baselines shows worse performance overall and degrades in the more challenging sweeping and Push-Shape scenarios.evaluation. We mark a trajectory as successful if the robot makes and maintains contact with the object throughout the rotation, and rotates it at least 25 degrees counterclockwise.Results. As shown in Table 2 , our policy significantly outperforms baselines (92.5% vs. 55% at best). Fig. 4 illustrates cases where our policy succeeds while Diffusion Policy fails. We observe that baselines can fail to move the end-effector into contact with the object. In more challenging cases, baselines are prone to selecting unstable grasping points causing the object to slip during manipulation. In contrast, our policy consistently makes contact with the object, with limited failures when selecting appropriate grasping points for particularly challenging shapes, such as a transparent bag with toys inside.Task. This task, shown in Figure 5 , requires scooping granular material (e.g., beans) from a full container to an empty one while avoiding distractor objects. This task requires the policy to perform precise manipulation of a scooping tool, identify the full and the empty containers, along with their precise locations, in arbitrary positions, and ignore distractors.Setup. We collect demonstrations with 17 bowls and 8 colored beans with only 1 distraction object in the scene at a time. At test time, we use an unseen set of 8 bowls, 4 new colored particles, and 5 distractor objects per trial sampled from a fixed set of 15 everyday objects. We mark a trajectory as successful if the robot transfers any particles to the empty bowl.Results. As shown in Table 2 (middle), our policy significantly outperforms baselines (85% vs. 55% at best). This is a particularly challenging task for our approach, due to the small target (the scooper) for object tracking and video generation. This demonstrates that stereo video generation can accurately determine the object's pose to perform 3D manipulation. By comparison, baselines are often distracted by objects in the scene, placing material in the wrong container, and failing more frequently in scooping due to misjudging the bowl's height on the table, as illustrated in Fig. 5 .Policy (Execution)Step 1Step 2Step 3Step 4 Generated VideoTask. The sweeping task, shown in Figure 6 , requires policies to use a brush to sweep randomly placed particles to a target location marked by a randomly placed star, while avoiding obstacles. This task is designed to test policy's ability to handle multi-modal distributions in the training data. The training data contains some variations with multiple ways to achieve the goal, such as choosing which particle to sweep in the scene. Due to the small size of the particles, we remove the final pooling layer from Diffusion Policy ResNet-18 encoder to provide a higher spatial resolution and improve baseline performance. We mark a trajectory as successful if any particle is transferred to be within 50 mm from the target.Results. In Table 2 (right) we observe that the video model maintains strong performance in this task with a 92.5% success rate. In contrast, baselines generate trajectories that often collide with obstacles and fail to sweep to the target (see Fig. 6 ), achieving only a 12.5% success rate at best with Diffusion Policy. This experiment demonstrates that the capitalizing on internet-scale pre-trained video generation models allows to better handle multi-modal demonstrations and achieve a much larger degree of generalization in this challenging scenario.Task. Push-Shape, shown in Figure 7 , is a challenging version of! the long-horizon Push-T task [2] : we place a foam shape on the table, and task the robot with pushing the object to a specified target goal mask (given as input to the policy) over consecutive steps. This is a challenging task as it requires adjusting the position and orientation of the shape, requiring predicting the shape's movement, which depends on the contact between the object and the end-effector, as well as material properties of the table.Setup. We train with a set of 26 foam objects each in the shape of a letter from the alphabet, and test on 8 unseen foam shapes (including digits and polygons). As this is a challenging, multi-step task, we score the best of 4 rollouts for each trial, and report the mean intersect-over-union (mIoU) with the target mask as well as the average rotation error from the target.Results. As shown in Table 2 , over the 32 trials, the video model achieved a significantly higher mIoU of 0.731 compared to 0.550 mIoU (at best with Diffusion Policy), and an average rotation error of 8.0 degrees compared to 44.1 degrees (at best with ACT). We find that baselines tend to push the object to the target, but fails to effectively rotate the object to match the mask. By contrast, the video model produces appropriate pushing actions to adjust the object's position and orientation, thereby reducing the rotation error. Qualitative examples of the model rollouts are illustrated in Fig. 7 . We investigated the impact of pretraining and stereo camera setups on model generalization using the rotation task. The results are shown in Table 3 . Without pretraining, the model produces blurry videos with significantly distorted tools, but the tracking model still tracks the tool's pose with stereo projection correcting depth estimation. We also explored the effects of the stereo camera setup. In the stereo input monocular output (SIMO) model, we track only one view of the stereo output in the full model. Additionally, we trained a monocular input and monocular output (MIMO) model. The SIMO model showed a performance decline due to greater depth estimation errors without stereo projection. The MIMO model exhibited an even further reduction in performance, indicating the importance of stereo input conditioning. 
limitations: 
 Our approach is limited by the need for visually trackable tools, and the model is task-specific, lacking conditional text embeddings to resolve ambiguities in input images. Future work could explore integrating text embeddings to train a unified policy across tasks. The reliance on rigid tools and video generation restricts the model's use in tasks requiring fine-grained force control. Additionally, using tools with known CAD models and camera calibration reduces flexibility, while tracking small objects remains a challenge for pose estimation models, affecting robot trajectory planning. Finally, the computational demands of video models make real-time control impractical, with current processing times of 33.5 seconds per video and 7.5 seconds for object pose tracking on an A100 GPU. 
conclusion: 
 In this study, we explore how video generative models can learn generalizable visuomotor policies.We propose to fine-tune a video diffusion model on human demonstrations to synthesize execution videos, which are then used to control the robot. Our key insight is that using common tools helps bridge the gap between human demonstrations and the robot manipulator. Experiments show that leveraging pretrained video diffusion models on Internet-scale data leads to greater generalization compared to state-of-the-art behavior cloning methods. 
 
 Appendix: In the videos, the tool is tracked using MegaPose [39] . Utilizing a stereo setup, the center of the tracked object from each camera are projected into 3D space as a straight line. The translation component of the object in 3D space is determined by finding the midpoint between the projected lines from the two cameras. The rotation component of the object is obtained by averaging the object rotations from the two views. This refined object pose from the stereo setup enhances the accuracy of the object's depth measurement from the cameras. In the scooping task, only the handle of the scooper is tracked to avoid inaccuracies due to occlusion by particles. In the sweeping and Push-Shape tasks, the tool without the handle is tracked, as the handle is occluded by the human hand. To obtain the tool trajectories for training Diffusion Policy, the same stereo tracking is applied to the demonstration videos.We use a CNN-based Diffusion Policy as our baseline, employing two pretrained ResNet-18 [42] image encoders to process the stereo images of the scene. The input images have a resolution of 384×224, similar to the original implementation resolution. We found that higher resolution input images did not improve model performance. We trained Diffusion Policy for 200 epochs to predict actions for the next 12 time steps.We use the ACT implementation from LeRobot [43] as our baseline, utilizing a single pretrained ResNet-18 [42] image encoder to process stereo images of the scene, as the implementation supports multi-image conditioning by default. The input images have a resolution of 768×448. We trained ACT for approximately 90k steps to predict actions for the subsequent 12 time steps.We use the VQ-BeT implementation from LeRobot [43] as our baseline, modifying it to incorporate two pretrained ResNet-18 [42] image encoders to process stereo images of the scene, since the original implementation does not support multi-image conditioning. The input images have a resolution of 768×448. Through experimentation, we found that training the VQ-BeT VAE for approximately 5k steps, followed by training the entire model for around 85k steps, yielded the best results in predicting actions for the subsequent 12 time steps.Ours (Trajectory) Diffusion Policy (Trajectory) Start Enddream•i•tate (verb) \drēm e tāt\. To behave in a way that imitates a dream. 


Decision: Strengths:
+ Dreamitate is a novel approach for using video-diffusion models in manipulation.
+ The approach is scalable, interpretable, and agent-agnostic. 
+ The approach outperforms Diffusion Policies on real-world tasks. 
+ The paper is well written.

Weaknesses:
- Missing video-conditioned baselines such as UniPi.
- Evaluations are missing prehensile tasks like grasping.
- Some details on the evaluation setup are missing.
- The method uses custom tools during data collection, which might hinder general applicability. 
- The limitations section could benefit from a discussion on interactions with multiple objects, robustness to perturbations, and failure cases.

Post rebuttal:  
Overall, the reviews are mixed; some key concerns regarding the generalizability of the approach remain. However, "video diffusion for robotics" is a relatively new field, so limitations could be addressed in future works.

Reviews: review: title: title: Interesting Paper and Idea but Baselines and Ablations are Missing
paper_summary: paper_summary: The authors demonstrate how to use video prediction models as a way to train robot policies. First, an off-the-shelf video model is finetuned on a few human demonstrations of a tabletop task. Stereo cameras are used to collect demonstrations and subsequently, the generated pairs of videos are used to estimate the 3D position of a tool handled by the human. This 3D position subsequently serve as robot action which are executed. The method is tested on table top tasks like shape pushing, sweeping and scooping tasks.
main_review: main_review: Strengths:
1. The paper is well written and easy to understand
2, It addresses an important problem, that of utilizing free form human demonstrations obtained from an internet scale dataset to improve robot learning. 
3. The method is straight forward and makes realistic assumptions

Weaknesses
1. **Lack of Baselines**: My biggest issue with the paper in its current state is the lack of baselines utilized by the authors. Works like Video as a Language for  https://arxiv.org/pdf/2402.17139, Video Language Planning https://arxiv.org/pdf/2310.10625 , UniPy https://proceedings.neurips.cc/paper_files/paper/2023/file/1d5b9233ad716a43be5c0d3023cb82d0-Paper-Conference.pdf have studied this problem before with varying degrees of success. The authors should utilize these baselines and compare how their method performs.
2. **Lack of Robot Foundation Models:** Currently, there are robot foundation model results in the paper. These are important as they serve as a skyline for the method, providing a metric for the potential performance if only robot data was used to train the behavior model.
3. **Lack of ablations:** The paper currently does not include any ablations and it is unclear which design decisions affect performance the most. This is necessary for completeness.
questions: questions: 1. How does changing the camera angle affect the video model? Is it still robust to such differences?
2. What is the speed of policy execution? What is the inference speed of the video model? 
3. Can you please describe the error modes of the work?
limitations: limitations: 3
review_summary: review_summary: I think the paper presents excellent ideas. However, I believe the experiments section needs improvement. Baselines and ablations are essential and are currently missing, which is why I am recommending a weak reject. With these improvements, I think this paper will be useful to many in the community.
title: title: Official Review of Submission46 by Reviewer gdSH
paper_summary: paper_summary: This paper presents Dreamitate, a novel approach to visuomotor policy learning that leverages large-scale video generation models. The key idea is to fine-tune a video diffusion model on human demonstrations with a tool, track the tool's trajectory in the generated video, and translate the tool trajectory to robot actions. The authors demonstrate their approach on four manipulation tasks, showing improved generalization compared to a state-of-the-art baseline.
main_review: main_review: The paper offers a compelling case for leveraging internet-scale video data for robust policy learning, though it also raises questions about scalability and practical implementation constraints. 

**Strengths:**

- **Effective use of Internet-Scale Data:** The paper effectively demonstrates how leveraging pre-trained video generative models trained on internet-scale data can lead to more robust and generalizable policies. 

- **Scalability:** By using human demonstrations instead of teleoperation data, the method significantly reduces the cost and complexity of data collection. 

- **Interpretability:** The intermediate representation in the form of generated videos provides an interpretable output, which can be useful for understanding the robot's planned actions. 

- **Agent-Agnostic:** The method's use of tool tracking makes the training pipeline agent-agnostic, allowing for transfer between different environments. 

- **Performance:** The method demonstrates strong performance across multiple tasks, consistently outperforming the Diffusion Policy baseline, especially in generalizing to unseen objects and scenarios. 

**Weaknesses:**

- **Scalability Concerns:** The lead over baselines appears to decrease as demonstration data increases (Fig. 8). 

- **Requirement for Structured Environment:** The method requires calibrated camera extrinsics during fine-tuning and evaluation, which could be a practical limitation in unstructured environments. The method also requires CAD models of each tool, which may limit real-world applicability. 

- **Single Task per Tool Assumption:** The approach seems to be designed for single-task policies per tool, which feels somewhat restrictive and may limit its flexibility in multi-task scenarios. 

- **Baseline Comparison:** The baseline diffusion policy might not be optimally configured, especially regarding handling distractors, which raises concerns about the fairness of the comparison. 

While the approach shows promise, I have some reservations about the method's long-term viability and scalability. The reliance on specific tools and the apparent diminishing returns with increased data raise questions about how far this method can be pushed. Furthermore, the computational demands of video generation models may be prohibitive for real-time applications. Despite these concerns, I believe this work represents an important step forward in leveraging large-scale generative models for robotics. It opens up new avenues for research and challenges us to think creatively about how we can harness the power of internet-scale data for robotic learning.
questions: questions: 1. Why did you choose to track the tool instead of the end-effector? Why not, for example, use something like the UMI gripper (https://arxiv.org/abs/2402.10329) for data collection? 

2. Why do some tasks have significantly more demonstrations than others? Did some tasks require more demonstrations for the video generation fine-tuning to work?

3. Are all policies single-task? If not, how is the task specified for the model? 

4. What are the success criteria for the shape pushing task? 

5. The results in Figure 8 show decreased performance with 2/3 of the data. What could be the reason for this decline? Was there significant variance in policy performance? 

6. Are actions generated by simply transforming the tool pose from the camera frame to the robot frame? Is there any additional control mechanism, such as PID controllers or filters? Is the policy closed loop?

7. Your method segments out the background and hands to obtain object poses using Megapose. However, the diffusion baseline does not seem to have a corresponding mechanism to encourage generalization to various backgrounds. Could this difference have impacted the fairness of the comparison? 

8. What computational resources are needed for training and evaluation? How long does the video generation take?
limitations: limitations: 3
review_summary: review_summary: This paper presents an approach that shows promise in leveraging internet-scale data for robot learning. While I have some concerns about its scalability and real-world applicability, the results are impressive and the ideas are exciting, and I believe it could inspire important follow-up work. I think the robot learning community would benefit from engaging with these ideas, and I will consider revising my recommendation if my questions and concerns are addressed.
title: title: Review of submission 46
paper_summary: paper_summary: This paper introduces a framework for visuomotor policy learning by fine-tuning a video generative model on human demonstrations. This approach leverages large-scale datasets of internet videos to generalize robot manipulation tasks across diverse environments. The framework synthesizes execution videos of tasks based on initial scene images, allowing robots to track and execute these tasks directly. Evaluated on four tasks, the method demonstrates better generalization compared to traditional behavior cloning approaches. The key insight is that using common tools bridges the gap between human and robot manipulation, enabling effective learning from minimal demonstrations.
main_review: main_review: # Clarity

The paper is well-written and easy to follow. The authors provide a strong introduction to the problem statement and convincingly explain how their method can improve performance over the current state-of-the-art.

# Strengths

The paper addresses a compelling problem: how robots can increase their training data without explicitly collecting expensive human demonstrations. The authors tackle this by employing video generation models that can create new videos in unseen circumstances using only a single initial state picture. They effectively combine the efficiency of behavior cloning in learning from data with video generation models to synthetically generate data, demonstrating a novel and promising approach.

# Weaknesses

My primary concern is that the paper relies heavily on having known tools to leverage generated videos. This constraint could significantly impact the method's ability to scale in the wild. Despite this limitation, the method works well within the parameters laid out by the authors.

The authors should also acknowledge the potential ambiguity in the initial image in their limitations section. For example, if there are two bowls with sprinkles and the objective is to scoop from only one, the method might struggle to determine the correct bowl. Additionally, the method's applicability to force-reliant tasks, such as cutting, is unclear and should be addressed as a potential limitation.
questions: questions: I think the authors should expand their limitations section to include that their method may not work effectively when there are multiple objects that can be interacted with similarly using the chosen tool. Additionally, their method may also fail with tasks that require precise force application, such as cutting or other force-dependent manipulations. Addressing these limitations would provide a more comprehensive understanding of the method's constraints and potential areas for improvement.
limitations: limitations: 2
review_summary: review_summary: The paper is well-written and introduces a compelling approach to enhance robot training data through video generation models, which can synthesize new videos from a single initial state image. This method effectively combines behavior cloning's efficiency with video generation to create synthetic data, demonstrating improvement over current state-of-the-art techniques. However, the reliance on known tools could limit scalability in real-world scenarios.
title: title: Dreamitate: Real-World Visuomotor Policy Learning via Video Generation
paper_summary: paper_summary: The authors present Dreamitate, a visuomotor policy learning framework that fine-tunes a video diffusion model on human demonstrations for dexterous manipulation tasks. The method generates videos of task execution conditioned on novel scenes and uses these videos to directly control robots. The key insight is using common tools to bridge the embodiment gap between human and robot manipulators. Dreamitate demonstrates superior generalization compared to behavior cloning methods, validated across four real-world tasks: rotation, scooping, sweeping, and shape pushing.
main_review: main_review: ### Strengths:
1. **Generalizability:** The use of video generative models pretrained on internet-scale datasets significantly enhances the generalizability of the learned visuomotor policies.
2. **Scalability:** The framework uses human demonstrations for training, which is more scalable than teleoperation-based data collection.
3. **Interpretability:** The intermediate representation of policy as generated videos offers better interpretability compared to end-to-end black-box policies.

### Weaknesses:
1. **Non-Prehensile Tasks:** The evaluated tasks are non-prehensile, limiting the applicability to more complex tasks requiring grasping or picking.
2. **Robustness Testing:** The paper does not thoroughly address how the method handles perturbations during task execution.
3. **Error Propagation:** There is a potential for error propagation in the video generation and tool tracking phase, which can significantly impact the performance of the generated policy.
4. **Lack of Comparative Analysis:** The paper lacks a comprehensive comparison with other state-of-the-art methods beyond behavior cloning, including other hybrid approaches or categorical manipulations such as [1-4].

[1] Peter R. Florence, Lucas Manuelli, and Russ Tedrake. "Dense Object Nets: Learning Dense Visual Object Descriptors By and For Robotic Manipulation."

[2] Leonard Bruns and Patric Jensfel. "SDFEst: Categorical Pose and Shape Estimation of Objects from RGB-D using Signed Distance Fields."

[3] Bowen Wen, Wenzhao Lian, Kostas Bekris, and Stefan Schaal. "You Only Demonstrate Once: Category-Level Manipulation from Single Visual Demonstration."

[4] Lucas Manuelli, Wei Gao, Peter Florence, and Russ Tedrake. "kPAM: KeyPoint Affordances for Category-Level Robotic Manipulation."
questions: questions: 1. Task Complexity: All the tasks are non-prehensile. How does the method perform on tasks that require picking or grasping?
2. Failure Cases: The tasks can be solved by the proposed method, but what are the typical failure cases for your method and the diffusion policy?
3. Performance Comparison: The Push-Shape task appears similar to the push-T task in the diffusion policy paper. The performance in the Push-Shape task is much worse. Why?
4. Training Data Subsampling: What does "open-loop system without subsampling the training data" in line 155 mean?
5. Task Difficulty: The hardest task, Push-Shape, has been well-solved with both learning-based and model-based methods. Can you perform more realistic and harder tasks?
6. Robustness to Perturbations: How does the system handle perturbations during the trajectory execution?
limitations: limitations: 2
review_summary: review_summary: While the paper presents a novel approach to visuomotor policy learning using video generation and shows promising results, there are several areas that could benefit from further development. The focus on non-prehensile tasks and limited real-world complexity are significant weaknesses. Addressing these issues and expanding the evaluation to include prehensile and more realistic tasks would significantly strengthen the contribution.