GEVRM: Goal-Expressive Video Generation Model For Robust Visual Manipulation

Authors: Hongyin Zhang, Pengxiang Ding, Shangke Lyu, Ying Peng, Donglin Wang
Primary area: applications to robotics, autonomy, planning
Keywords: Robot Manipulation; Vision Language Action Model

Abstract:
With the rapid development of embodied artificial intelligence, significant progress has been made in vision-language-action (VLA) models for general robot decision-making. However, the majority of existing VLAs fail to account for the inevitable external perturbations encountered during deployment. These perturbations introduce unforeseen state information to the VLA, resulting in inaccurate actions and consequently, a significant decline in generalization performance. The classic internal model control (IMC) principle demonstrates that a closed-loop system with an internal model that includes external input signals can accurately track the reference input and effectively offset the disturbance. We propose a novel closed-loop VLA method GEVRM that integrates the IMC principle to enhance the robustness of robot visual manipulation. The text-guided video generation model in GEVRM can generate highly expressive future visual planning goals. Simultaneously, we evaluate perturbations by simulating responses, which are called internal embeddings and optimized through prototype contrastive learning. This allows the model to implicitly infer and distinguish perturbations from the external environment. The proposed GEVRM achieves state-of-the-art performance on both standard and perturbed CALVIN benchmarks and shows significant improvements in realistic robot tasks.

Decision: accept (poster)
Reviewer scores: 6, 6, 6, 6

Reviews and discussion:
<thread>
creation date:
1730673713854

thread_type:
official_review

<thread_object>
creator: reviewer_ehog

<content>
summary:
This paper introduces GEVRM, a novel close-loop VLA model that adopts the Internal Model Control principle to a learning-based framework and improves robustness to perturbations from external environments. This method leverages text-conditioned video generation to infer image goal states, aligns the current and goal states through prototypical contrast learning, and performs goal-guided action prediction using a diffusion policy. Evaluation results demonstrate the strong generalization capability of image goal generation across simulated and real-world datasets, as well as the robustness through high task success rates under external perturbations in the visual manipulation task.

strengths:
1. This paper is well-motivated, clearly written, and easy to follow.
2. The paper properly discussed the related work that leverages text-conditioned video models to generate visual plans for decision-making.
3. GEVRM innovatively uses several techniques to improve the quality of generated visual goals and aligns state representations to enhance the robustness under perturbations
4. The authors provide extensive evaluation results on both behavior planning quality and task completion in perturbed environments, showing the effectiveness of the proposed method.

weaknesses:
1. The underlying framework – composing a text-conditioned video model and an action predictor has been widely explored by previous work, in which most works adopt a relatively lightweight method to extract inverse dynamics. This work employs a diffusion policy for action prediction and interacts with the environment in a close-loop manner, which may help learn multi-modal action distribution but can be time-consuming and often not desirable in real-time deployment.
2. The thoroughness of behavior planner evaluation can be further improved, for example, the effectiveness of techniques like random masking compared to simply masking out the last few frames for decision-making scenarios, should be supported by ablations.
3. It would be great if the authors could thoroughly discuss the limitations of the proposed method.

questions:
1. When GEVRM uses prototypical contrastive learning for state alignment, the positive pairs become the current and goal states from the same trajectory, instead of different augmented views from the same image. I wonder if this contrastive learning process will learn to focus on only trivial consistent features in the background, which are likely to be similar across all timesteps, and ignore some differences (e.g. the position change of objects or the robot arm) between the current and goal states that are essential to help the model make correct decisions.
2. Would the input states be perturbed by any transformations while training the state alignment encoders? If not, are these encoders directly used to process the perturbed images during inference?
3. During the testing time in CALVIN evaluation, apart from the visual discrepancy of the environments, are the instructions for each (sub)task seen during training, or are novel instructions tested to demonstrate task generalization capabilities?
</content>
</thread_object>

</thread>
<thread>
creation date:
1730712347342

thread_type:
official_review

<thread_object>
creator: reviewer_9iwm

<content>
summary:
This paper considers the problem of vision-language-action model for robotics. The proposed approach leverages a text-video generation model to generate the goal images conditioned the current observation and task description. Then the action policy leverages the generated goal and the current state to predict the robot action. To aggress the problem of disturbance encountered during deployment, the paper leverages the idea of IMC from classic control by using internal embeddings (optimized through prototype contrastive learning) as the policy model input.

strengths:
The paper considers an important topic in VLM for robotics. The paper is well-written, and the logic is clear. The proposed approach outperforms the baseline in terms of goal generation when the input image is perturbated and in terms of closed-loop eval in simulations.

weaknesses:
The connection to IMC. The IMC in control uses “diff” between the predicted output from the process model and the actual output from the actual robot, induced by the same action, to generate the feedback signal to compensate the controller. But the proposed approach seems to use the current robot state and the desired output to generate the control signal. I think the proposed structure is just a goal conditioned policy, and I don’t see the strong connection to the IMC.


Image encoder. The image encoder is trained through the discriminating if a goal image and the current state image are from the same trajectory. The experiment is missing ablation studies on the encoder. Would be interested in how an off-the shelf encoder like DinoV2 performs.

questions:
Please see my questions above.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730712594791

thread_type:
official_review

<thread_object>
creator: reviewer_sih9

<content>
summary:
This paper presents a goal-guided policy conditioned on future predictions generated by a video diffusion model. The authors try to integrate the classic principle of internal model control (IMC) into visuomotor control, thereby enhancing the robot's capability to resist environmental perturbations.

strengths:
1. The method appears to be reasonably designed. It utilizes a video diffusion model as a visual planner to generate future predictions that guide the policy in action output. In terms of detailed design, the proposed method enhances the generation consistency through efficient video spatiotemporal compression and random mask strategies. Furthermore, the authors utilize prototypical contrastive learning to align the goal states with its current state, thereby enabling the model to implicitly infer and distinguish perturbations from the external environment.
2. The authors propose generalization experiments on perturbed environments in CALVIN (train A, B, C → perturbed test D), which can assist the research community in examining the the robustness of policies against external environmental disturbances.

weaknesses:
1. This paper did not undertake real-world robotic experiments. Instead, it focused exclusively on analyzing video prediction quality using the Bridge dataset and conducting action execution tests within the simulated environment of the Calvin benchmark. However, the effectiveness of GEVRM on real-world robots was not validated.
2. In Table 2, the authors compare their proposed GEVRM method with other baseline approaches within the CALVIN ABC-D environment, utilizing the settings of RGB input only. However, they did not involve powerful baseline methods for comparison. While GR-1 is a comparatively strong baseline, its reproduced performance is notably low. Could the authors provide further details regarding the reproduction of GR-1? Furthermore, would it be possible for the authors to incorporate additional powerful methods for comparison by adjusting their input modalities to RGB only? Additionally, I would like to mention that Susie's results on Calvin ABC-D should be included in Table 2.
3. The authors should give more credits to the paper "Closed-Loop Visuomotor Control with Generative Expectation for Robotic Manipulation" (https://arxiv.org/abs/2409.09016). The design philosophy behind GEVRM and the presentation of the article, such as the layout in Figure 1, seem to be inspired by this paper; however, it is not mentioned in the introduction or related works sections.

questions:
1. The integration of a video diffusion model might introduce considerable computational overhead. However, the paper lacks a comprehensive analysis of the computational requirements. How about the inference efficiency in Calvin simulation environment?
2. On the goal-conditioned policy, the authors propose a diffusion policy conditioned on the generated goals. In such goal-conditioned policy framework, does the diffusion policy exhibit significant superiority compared to a simple MLP?
3. In Table 3, the generalization experiments are conducted in perturbed environments. How do the other baseline methods, apart from Susie, perform in this setting?
</content>
</thread_object>

</thread>
<thread>
creation date:
1730719656316

thread_type:
official_review

<thread_object>
creator: reviewer_tjew

<content>
summary:
This paper introduces GEVRM, a method for robust visual manipulation through video generation. It trains a video generator for goal generation, and a goal-conditioned behavioral cloning policy. The video generator takes a sequence of observations and a language instruction, and predicts the subsequent image observations as the goal. The policy head is a goal-conditioned diffusion policy. GEVRM uses an auxiliary state alignment loss to improve learned representations. Results show that GEVRM outperforms previous methods on Bridge and CALVIN data, and is more robust to perturbations and generalizable than baselines in the simulated environment CALVIN.

strengths:
- Video generation for goal-conditioned behavioral cloning is well-motivated.
- GEVRM outperforms previous methods on Bridge and CALVIN for video generation.
- GEVRM outperforms HiP, UniPi, and GR-1 on generalization in CALVIN, and outperforms SuSIE on generalization with visual perturbations.

weaknesses:
- Limited/inconsistent evaluation. GEVRM is compared with AVDC on Bridge and with GR-1 on CALVIN for video generation, with HiP/UniPi/GR-1 on unperturbed generalization, and with SuSIE on perturbed generalization. It would be good to see GEVRM compared with a consistent set of baselines for video generation and policy rollouts in a few more environments.
- If the core problem setting is robust manipulation, it would be convincing to see GEVRM compared with HiP, UniPi, GR-1, and pure behavioral cloning, each with data augmentation.
- It is unclear what the key contributions of GEVRM are and why it outperforms prior work. A more comprehensive ablation with downstream task success rate could justify each component in the design of GEVRM and clarify its improvement over prior state of the art.

questions:
- Baseline (video generation): is AVDC/GR-1/GEVRM trained with data augmentation?
- Baseline (CALVIN rollout): how does GEVRM compare to HiP/UniPi/GR-1/behavioral cloning with data augmentation?
- Ablation: how does setting the state alignment loss $\lambda = 0$ affect task success rate? Quantitative results in addition to the t-SNE plot would be convincing.
- Ablation/Clarification: video generation for control and goal-conditioned behavioral cloning are relatively well-studied problems. Could you clarify the key contribution of GEVRM? (How is GEVRM different from prior work in its architecture, and what design choices contributes to the improved performance?) A more comprehensive ablation studying the importance of each component in GEVRM would be convincing to see.
</content>
</thread_object>

</thread>
<thread>
creation date:
1732524306519

thread_type:
official_comment

<thread_object>
creator: authors

<content>
title:
General Response (1)

comment:
We would like to thank the reviewers for their appreciation of our work. Reviewers TJEW and Ehog recognized our method was "**well-motivated**", and reviewers 9iWM and Ehog appreciated our manuscript was "**clearly written**", "**logically clear**" and "**easy to follow**". We also thank all the reviewers for their professional guidance, constructive comments, and insightful suggestions, which enabled us to improve the manuscript further. We have thoroughly performed additional experiments and revised the manuscript to address all your comments and concerns. 
## 1. The core contribution of our method.
Our **motivation** is that VLA models trained in ideal environment data are inevitably subject to external perturbations when deployed, resulting in fragile and unstable actions, which leads to a significant decrease in their generalization performance. We aim to explore how to instantiate the classic internal model control (IMC) principle in the VLA framework to improve the robustness of decision-making actions. Therefore, **the core contribution of our work lies in the necessary components in the VLA framework, mainly including goal state generation and goal alignment**. The widespread principle of IMC suggests that a closed-loop system with an internal model that includes external input signals can accurately track the reference input and effectively cancel out disturbances. We, therefore, borrow this control idea for VLA robust action generation. After the DiT video model generates the goal state, prototype contrastive learning is utilized to enhance the state representation to compensate the low-level controller. 
## 2. Comparison with more advanced baseline algorithms.
To compare with the baseline algorithms more comprehensively, we added the SuSIE algorithm to the goal generation task (Table 1 of the manuscript); added the SuSIE algorithm to the CALVIN ABC-D generalization task (Table 2 of the manuscript); added RoboFlamingo (pure imitation learning) and GR-1 algorithms to the perturbed CALVIN ABC-D generalization task (Table 3 of the manuscript). The specific results are shown in the following three tables. The results show that compared with the baseline algorithms SuSIE and GR-1, our method GEVRM shows superior performance in the goal generation task, the environment generalization task, and the perturbed environment generalization task. In addition, SuSIE is a method based on single-frame image state generation, while HiP, UniPi and our GEVRM are methods based on multi-frame video state generation. Therefore, our task is more challenging, aiming to allow the model to learn more temporal consistency of the robot arm and object motion.

### Goal generation task 
| **Benchmark**  | **Algorithms** | **FID (↓)**         | **FVD (↓)**         | **LPIPS (↓)**         | **SSIM (↑)**        | **PSNR (↑)**        |
|:--------------:|:--------------:|:-------------------:|:-------------------:|:---------------------:|:-------------------:|:-------------------:|
| **BridgeData**  | AVDC           | 246.45 ± 39.08      | 22.89 ± 4.99        | 0.23 ± 0.03           | 0.73 ± 0.05         | 18.22 ± 2.53        |
| **BridgeData**  | SuSIE          | 114.79 ± 21.38      | -                   | 0.217 ± 0.082         | 0.706 ± 0.070       | 16.388 ± 2.901      |
| **BridgeData**  | GEVRM (Ours)   | **35.70 ± 10.77**   | **4.16 ± 1.35**     | **0.06 ± 0.03**       | **0.89 ± 0.04**     | **22.36 ± 2.75**    |
| **CALVIN**      | GR-1           | 236.75 ± 38.87      | 12.83 ± 2.60        | 0.20 ± 0.02           | 0.65 ± 0.03         | 18.59 ± 0.95        |
| **CALVIN**      | SuSIE          | 214.14 ± 45.45      | -                   | 0.150 ± 0.041         | 0.750 ± 0.045       | 18.115 ± 2.289      |
| **CALVIN**      | GEVRM (Ours)   | **94.47 ± 22.54**   | **3.80 ± 1.20**     | **0.09 ± 0.04**       | **0.80 ± 0.05**     | **21.10 ± 3.29**    |

### CALVIN ABC-D generalization task (Only static camera is used)
| **Algorithms** | **1**   | **2**   | **3**   | **4**   | **5**   |
|-----------------|:---------:|:---------:|:---------:|:---------:|:---------:|
| **HiP**            | 0.08    | 0.04    | 0.00    | 0.00    | 0.00    |
| **UniPi**          | 0.56    | 0.16    | 0.08    | 0.08    | 0.04    |
| **GR-1**           | 0.75    | 0.45    | 0.20    | 0.15    | 0.10    |
| **SuSIE**          | 0.87    | 0.69    | 0.49    | 0.38    | 0.26    |
| **GEVRM (Ours)**   | **0.92**    | **0.70**    | **0.54**    | **0.41**    | **0.26**    |
</content>
</thread_object>

</thread>
<thread>
creation date:
1732757945608

thread_type:
official_comment

<thread_object>
creator: authors

<content>
title:
Friendly Reminder Regarding Upcoming Rebuttal Deadline

comment:
Dear reviewers, I hope this message finds you well. This is a gentle reminder regarding the review of our manuscript.
We deeply appreciate the invaluable comments and feedback provided by reviewers. They are instrumental in enhancing the quality of our research. As per the schedule, the rebuttal phase is drawing to a close. We understand that you have a demanding schedule and a multitude of responsibilities, but we are keen to receive your feedback before the deadline. This will afford us the opportunity to address any questions or concerns you may have raised in a timely manner. We are eager to incorporate your insights to refine our work and would be grateful if you could share your thoughts prior to the rebuttal deadline.

Thank you very much for your hard work and support. Your dedication to the review process is greatly appreciated.
</content>
</thread_object>

</thread>


Metareview:
metareview:
The paper introduces GEVRM, a closed-loop vision-language-action (VLA) model based on the internal model control (IMC) principle, designed to enhance robustness to external perturbations through the integration of text-guided video generation and prototype contrastive learning. The model achieves state-of-the-art performance on both standard and perturbed CALVIN benchmarks and demonstrates strong results in realistic robot tasks.

The reviewers unanimously acknowledged the paper's contributions, noting its (1) clear and well-motivated approach, (2) extensive experimental results demonstrating advantages over existing methods, (3) valuable insights from evaluations under external environmental disturbances, and (4) clear and well-structured presentation.

During the Author-Reviewer Discussion phase, the authors provided detailed responses that successfully addressed many of the reviewers' concerns, leading to score increases from multiple reviewers. With all reviewers in unanimous agreement to accept the paper, the AC recommends acceptance while encouraging the authors to carefully address both pre- and post-rebuttal comments to further strengthen the final version.