FLIP: Flow-Centric Generative Planning as General-Purpose Manipulation World Model
Authors: Chongkai Gao
Haozhuo Zhang
Zhixuan Xu
Cai Zhehao
Lin Shao
Venue: ICLR 2025 Poster (2025)
Keywords: World Model
Long-Horizon Planning
Robot Manipulation
Flow Generation
Primary area: applications to robotics, autonomy, planning
TL;DR: We train an interactive world-model that enables model-based planning on the image flow and video spaces for diverse manipualtion tasks with only language-annotated video datasets.

Abstract: We aim to develop a model-based planning framework for world models that can be scaled with increasing model and data budgets for general-purpose manipulation tasks with only language and vision inputs. To this end, we present FLow-CentrIc generative Planning (FLIP), a model-based planning algorithm on visual space that features three key modules: 1) a multi-modal flow generation model as the general-purpose action proposal module; 2) a flow-conditioned video generation model as the dynamics module; and 3) a vision-language representation learning model as the value module. Given an initial image and language instruction as the goal, FLIP can progressively search for long-horizon flow and video plans that maximize the discounted return to accomplish the task. FLIP is able to synthesize long-horizon plans across objects, robots, and tasks with image flows as the general action representation, and the dense flow information also provides rich guidance for long-horizon video generation. In addition, the synthesized flow and video plans can guide the training of low-level control policies for robot execution. Experiments on diverse benchmarks demonstrate that FLIP can improve both the success rates and quality of long-horizon video plan synthesis and has the interactive world model property, opening up wider applications for future works. Video demos are on our website: https://nus-lins-lab.github.io/flipweb/.

Published as a conference paper at ICLR 2025
FLIP: FLOW-CENTRIC GENERATIVE PLANNING AS GENERAL-PURPOSE MANIPULATION WORLD MODEL
Chongkai Gao
Haozhuo Zhang
National University of Singapore gaochongkat @u. nus edu
Peking University
21 21000131 32 320.tu..2..uu.... pku edu.
Zhixuan Xu, Zhehao Cai National University of Singapore (zhixuanxu, e1l.sr
Lin Shao
National University of Singapore 1 linshao@nus edu. sg
edu
ABSTRACT
We aim to develop a model-based planning framework for world models that can be scaled with increasing model and data budgets for general-purpose manipula- tion tasks with only language and vision inputs. To this end, we present FLoww- Centric generative Planning (FLIP), model-based planning algorithm on visual space that features three key modules: 1) multi-modal flow generation model as the general- purpose action proposal module; flow-conditioned video gen- eration model as the dynamics module; and 3) vision-language representation learning model as the value module. Given an initial image and language instruc- tion the goal, FLIP can progressively search for long-horizon flow and video plans that maximize the discounted return to accomplish the task. FLIP is able to synthesize morizon plans across objects, robots, and tasks with image flows as the general action representation, and the dense flow information also provides rich guidance for long-horizon video generation. In addition, the synthesized flow and video plans can guide the training of low-level control policies for robot ex- ecution. Experiments both the success rates and quality of long- horizon video plan synthesis and has the interactive world model property, opening up wider applications for future works. Video demos are on our website: lntips://nus lins lans-lab.gge lab. wwthgi.ii//iiiweeeee..
diverse benchmarks demonstrate that FLIP can improve
INTRODUCTION
World models refer to learning- basning-based representations or models that learn to simulate the environ- ment (LeCun) 2024 Ha & Schmidhuber, 2018). With world models, agents can imagine, reason, and plan inside world models to solve tasks more safely and efficiently. Recent advancements in generative models, especially in the area of video generation (Brooks et al.) [0244] Blattmann al.] 2023 Yang et al.] 2023), have demonstrated the application of generating high-quality videos as world simulators with internet-scale training data. World models have opened new avenues across various fields, particularly in the domain of robotic manipulation (Yang et al. 2023 Mendonca et al.
2023
Seo et al. 2023), which is the focus of this paper.
The intelligence of generalist robots involves two levels of abilities (Caucheteux & King 2022 Manto et al. )012): 1) high-level planning of the abstraction sequence of the task with multi-moaal inputs, and 2) lovelevel execution of the plan by interacting with the real world. A well-designed world model could serve as an ideal way to realize the first function, for which it should enable model-based planning. This requires the world model to be interactive, i.e., can simulate the world according to some given actions. The core of this framework is to find a scalable action repre- sentation that serves as the connection between high-level planning and low-level control. This representation should: 1) be able to represent various kinds of movements across diverse objects, robots, and tasks in the whole scene: 2) be easy to obtain or label large amount of training data for scaling up. Regarding this, Yang et al. (2023); Du et al. (2023); Zhou et al. (2024) use lan- guages from VLMs (Driess et al. 2023) as high-level actions, while Wu et al. (2024) directly use low-level robot actions to interact with the world model. However, they either require extra datasets or task-specific high-level action labeling processes for training the interactive world model, or their representations cannot deseribe sophisticated subtle movements in the whole scene. For example, they cannot describe the detailed movements of a dexterous hand spinning a pen. These limit their application scalable interactive world model and inspire to find other action representations,

---

Published as a conference paper at ICLR 2025
Input Image
Flow Generation as Action
Instrucionn
prove the grean stor Jo the green cirole
bant aptsber
Language Annotated Video Datasets
Foid the
Tut tie dire sou
basker
Flow-conditonee Generation Dynamics
Iterative Planning
FLIP
the
unfold (be cloth
Vislon-language Similarity Function
the spic pint
pael the Hg2
Figure I: Overview of our method. Left: FLIP is trained on video datasets across different tasks, objects, and robots, with only one language description for each video as the goal. Right: we train an interactive world model consisting of an action module for flow generation, dynamics module for video generation, and a value module for assigning value at each step. These modules can perform flow-centric model-basee planning for manipulation tasks on the flow and video space.
Image flow, a dynamic representation of pixel-level changes over time, is a concise yet general representation of all kinds of movements in images for different robots and objects and can describe more subtle changes than language. More importantly, image flow can be completely obtained by off-the-shelf trackers (Karaev et al. 2023) from pure video datasets. Meanwhile, recent works also show that flows are effective representations for training low-level manipulation policies (Wen al., 2023 Xu et al.) al.]]]]222222 (2224 b). These make flow a good choice for action representation of world models. However, remains unclear how to leverage flows for planning manipulation tasks.
In this work, we present Flow-Centric General Planning (FLIP) for general-purposs robot manip- ulation tasks. As shown in Figure [ train flow-centrie world model purely from language- annotated video data from diverse tasks. This world model contains three modules: 1) flow gener- ation network the action module, 2) a flow-conditioned video generation model as the dynamics module, and 3) a visual-language representation learning model as the value module. Specifically, we design our action and dynamics module based on CVAE (Kingma) 2013) and DiT (Peebles & Xie, 2023) architectures respectively and propose a new training mechanism for leveraging LIV (Ma al. (2233) as our value module. The trained modules enable model-based planning by progres- sively searching successful long- horghhorzzon plans on the flow and video spaces: given an initial image and language instruction as the goal, the action module will propose several flow candidates, and the dynamics module will generate the short-horizon future videos. The value module will access the fa- vorability of generated videos that maximize the discounted returns and perform tree search (Selman & Gomes 2006) to synthesize Long-horizen plans for solving the task.
In experiments, we show that FLIP can perform model-basee planning to solve tasks for both sim- ulation manipulation tasks (LIBERO (Liu et al.) 2024a)) and real-world tasks (including FMB (Luo) et al. 2023), cloth folding, unfolding. and Bridge V2 (Walke et adl]|202)) We also show that FLIP can generate high-quality long-horizon videos for these tasks. Meanwhile, the generated flow and video plans can guide the training of low-level policies. We also show that the three modules of FLIP superior to their respective baselines (Wen et al. 2023 Zhu al. 2024; Ma al., 2023). We quantitatively show that FLIP can simulate diverse complex manipulation tasks across objects and robots. The trained world model also demonstrates interactive properties, zero-shot transfer ability, and scalability. In summary, our contributions are:
We propose flow-centric generative planning (FLIP) an interactive world model for general-purpose model-based planning for manipulation tasks.
We design new flow generation network, a new flow-conditioned video generation net- work, and a new training method for an existing vision-language representation learning network as the three key modules of FLIP.
In our experiments, we show FLIP can perform general-purpose model-based planning, synthesize long-horing horizon videos, guide the training of low-level policy, and other promising properties, well the superiority of the three modules of FL.P compared to baselines.
2

---

Published as a conference paper at ICLR 2025
2 RELATED WORKS
2.1 WORLD MODELS FOR DECISION MAKING
Early works of world models learn system dynamics in low dimensional state space Leesort et al. 2018 Ferns et al. 2004), perform planning in latent space (Naiiriany et al. 2019 ,. or train networks to predict the future observations (Finn Levine 2017) and actions (Kaiser et al. 2019). Modern
model-bused reinforcement learning methods (Hafner al. 2020 2023 Hansen et al. 2023 Baker
ct al. 2022 Micheli et al. 2022 focus on latent space imagination with coupled dynamics and action
modules. Recent works leverage powerful scalable video generation architectures like Diffusion
Transformer (Peebles & Xie, 2023) and large-scale training data (Grauman et al.. (2022) to develop
video generation networks simulate an interactive environment (Yang et al. [203]] Bruce et al.
2024] Shridhar et al., [024] Valevski et al., 2024) Wu et al. [024]] Zhu et al. 20244) Wu et al. (2024).
In this work, we build world model with separate flow-centric action and dynamics modules as well as a vision-language value model for model-bases planning for robot manipulation tasks,
2.2 FLOW AND VIDEO MODELS FOR MANIPULATION
Flows are the future trajectories of query points on images or point clouds. They are universal de- scriptors for motions in the video, while video data contains rich knowledge of behaviors, phy and semantics, and have unparalleled scalability in terms of both content diversity and data acqui- sition. For robotics, people have been trying to use flows as policy guidance (Wen al., 2023 Bharadhwaj et al.) 2024), learn dense correspondence (Jiang et al. (024b), tool using (Seita et al..
2023j, or cross-embodiment representations (Xu et al. 2024a| Zhu et al. 2024]] Yuan et al. 2024).
Videos are usually used for learning inverse dynamics (Du et al. [2244] Finn & Levine 2017
Brand-
fonbrener ct al. 2024 Gao et al. 2021), rewards (Ma et al. 2022] 2023] Nair et al.
2022
Zakka
ct
transferrable visual representations such as latent embeddings (Sermanet ct al.
2018
Nair et al. 2022 Liu et al.
2024a). key points (Huang et al., 2024 Di Palo & Johns 2024). affor-
dance (Bahl al., 2023 Shu et al. 2017), flows ( (Wen al. [203 Xu et al.
[024a]] Bharadhwaj
jet al. 2024). scene graphs (Zhang et 2004 Jiang et al. [024a]] |
Kumar et al. 2023 or acquire
similar manipulation knowledge from human videos (Wang et 2023] Mendonca al., 2023
Shao et al. 202T; Liang et al., 2024). Recent works also use video generation techniques as visual
simulation Yang et al.] [023] Liu et al., 2224b). In this work, we build our action, dynamics, and
value modules all based on video and language inputs, enabling the scalability of our framework.
3 THREE FUNDAMENTAL MODULES OF FLIP
3.1 PROBLEM FORMULATION
We model a manipulation task T as a goal-conditioned Partially Observable Markov Decision Pro- cess (POMDP) parameterized by (S, A, P, RR 7, where A, O are state, action, and ob- servation spaces, o : S + O is the satte-observation mapping function, P S x A -> S is the transition function, R S A + R is the reward function, 7 is the discount factor, and g is the goal state. In this work, the observation space is the image space: O = IR XWXx where H and W the height and width of the image, and R(s, g) T(s g) is a goal-conditioned sparse reward. The task is solved if the agent maximizes the return 2 ((((()))).
We aim to solve this problem by learning a world model and low-level policy. The world model performs model-based planning on image and flow spaces to maximize the return, synthesizing long- horizon plans, and the low-level policy executes the plan in the real environment. We aim to train the world model only on language-annott annottted vidco datasets to make it general- general-purpose purpose and scalable, and train the low-level policy on few action-labeled datasets. To enable model-based planning, our world model contains three key modules, as introduced in the following sections.
3.2 FLOW GENERATION NETWORK AS ACTION MODULE
Overview, The action module of FLIP is a flow generation network Ty that generates image flows (future trajectories on query points) actions for planning. The reason why use generation model rather than a predictive model is that we are doing model-based planning, where the action module should give different action proposals for sampling- planning. Formally, given h step
3

---

Published as a conference paper at ICLR 2025
Flow Generation as Action
Flow-condltioned Video Generation as Dynamics
MLP
[CLS]
2
Scale Direction
Decodes
MLP
ottention
Cross-
Trare Trarofermer former Encoder
Transfomer Encoder
MLP
Encode
Patchity
Spatral
Attertion
Tempporal
Attention
Lineon Umgattchis
bohk Imosge Mtlutoy
Nobse: E
Patchity
addiL.- Zero
Lossos
24d best We CrEam choose ELRDE The
Liana
T
Reconstructed Flows
MLP
MLLP
XL
Figure 2: The action module and dynamics module of FLIP. Left: the tokenizing process of different modalities in training data. Middle: we use a Conditional VAE to generate flows as actions. sep- arately generates the delta scale and directions each query point for flow reconstruction. Right: we use a DiT model with the apatial-tempore attention mechanism for flow-conditioned video gen- eration. Flows (and observation history) conditioned with attention, while languages and timestep are conditioned with AdaLN-
image observation history Or-hat at timestep f, the language goal g, and a set of 2D query points K coordinates P, = {p/1 k=1, where Pt,k (u, v) is the k-th query point coordinate o,, the flow generation network f generates coordinates of query points in future L timesteps (including the current step): Ptt-L = y (t.c--...P..))) Pr.g)= E LxXXX
Training Data Annotation. The flows of query points can be extracted from pure video data by the off-the-shelf point tracking models. The problem is how to select query points. Previous works either use SAM (Ravi et al. 2024) to select query points on the region of interest or select query points active/stationary regions with predefined ratio (Wen al. (023). These methods face two problems: 1) for diverse kinds of videos with complex scenes, it hard for modern segmentation models (Ravi et al., 2024) to segment perfect regions of interest with human assistance; 2) for long-horizon videos, there may be objects points only from the initial frame become problematic. To this end, in this work, we uniformly sample dense grid query points for the whole image (for the first problem) at each timestep, and track them for only short-horizon video clip, i.e., tracking on video clips starting from every frame of the long- hori-on video (for the second problem). This can mitigate the second problem because even some objects appear/disappear, their influences are restricted in a short horizon. Formally, for each frame in the dataset, we uniformly sample grid of N₂ points, then use Co-Tracker (Karaev) N. 2023 to generate the flows (pf. 1 within a future video clip of L steps.
in the video, and using query
ct al.
Model Design. We design Conditional VAE (Kingma 2013) with transformer [Vaswani 2017) architecture for flow generation, illustrated in Figure 2 As opposed to previous flow prediction works (Wen et al.) 2023] Xu et al. 2024a Bharadhwaj et al. 2024), we observe enhanced per- formance when predicting relative displacements rather than absolute coordinates, i.c. predict App P p for the k-th point at each time step.
For the VAE encoder, we encode ground truth flow {p.)- patchify observation history Ot-hsts and encode language embedding from Llama 3.1 8B (Dubey al., 2024) tokens, concatenate them with a CLS token for gathering the information, and then send them to a transformer encoder extract the output the CLS token position the latent variable of VAE. For the VAE decoder, we first encode the query points {p: 1
2 at only timestep to query tokens, concatenate them with image and language tokens well as the sampled latent variable from reparameterization. and send them to another transformer encoder. We extract the output at the query tokens and use two MLPs to predict the delta scale ö, E 1
Lxk and delta direction 5g E 221 K for L future horizons.
Thus can get Apt 8th 8i8. and the whole future flow be reconstructed step by step. We also decode the output at the image token positions as an auxiliary image reconstruction task (Wen) et al. 2023 He et al. 2022ab. which we find useful for improving the training accuracy.
4

---

Published as a conference paper at ICLR 2025
3.3 FLOW-CONDITIONED VIDEO GENERATION NETWORK AS DYNAMICS MODULE
Overview. The flo-ccnditioned video generation network D generates the following L frames based on current image observation history Ot-h:t, the language goal g. and the predicted flow Pt:+L to enable iterative planning for the next planning step: 01+1:1+L = D(or-aat: 9. Pta+L).
Model Design. We design a new latent video diffusion model that can effectively take as input dif- ferent kinds of conditions such as images, flows, and language. This model is built on the DiT (Pec- 2023 architecture with spatial-temporal attention mechanism (Ma et al,, 2024, Bruce
bles & Xie
et al. 2024 Zhu et al., (024). The background knowledge of latent video diffusion models is in
Appendix A.I Here we introduce the design of the multi-modal condition mechanism.
In the original DiT (Peebles & Xie, 2023) and previous trajectory-conditional video diffusion pa- per (Zhu at. 2024), they use adaptive layer norm zero (AdaL.N-Zero blocks to process conditional inputs (such as diffusion timestep and class labels), which regress the scale and shift parameters of the layer norm layers from all conditions with a zero-initialized MLP. However, AdaLN will compress all conditional information to scalars, and cannot enable fine-grained interaction between different parts of conditions with the inputs. Thus, this mechanism not suitable for complex con- ditions such image and flow (Zhang et al. 2023 Bao et al.) 2023). To this end, we propose mixed conditioning mechanism for multi-modal conditional generation. We use cross attention for fine-grained interactions between flow conditions (tokenized as N,, tokens) and observation condi- tions and noisy frames. For image history conditions, we concatenate them on the Gaussian noise frames. We use AdaLN-Zero to process the global conditions including the diffusion timestep and language instruction, as shown in Figure 2 To keep the observation condition clean, we do not add noise to Ot-h:t during the diffusion process and do not perform denoising on them either.
3.4 VISION-LANGUAGE REPRESENTATION LEARNING AS VALUE MODULE
Overview. The value module V assigns estimated value VÚ for each frame o, to en- able model-based planning on the image space, based on the language goal g: V. V(or,00 g). In this work, we adopt LIV (Ma al. 2023) to instantiate the value function. LIV first learns shared language-vision representation from action-free free videos with language annotations. then computes the similarity between current frame o, and g as the value for timestep t: V, c- (ox), (g)). where biy and b are the encoding network for image and language respectively, and S is the y-weighted cosine similarity metric.
Vision-Language Similarity as Value Function
clip clip
Instruction
'put hoth the cream m cheese box the butter the baskel"
clip
(1br(o)), (g))
Ours
Figure 3: Top: The value module of FLIP. We fol- low the idea of (Ma et al.] 2023) and use time- contrastive learning for the visual-language rep- resentation, but we treat each video clip (rather than each frame) state. Bottom: the fine-tuned value curves of et et al. 2023) and ours.
The pretrained LIV model needs to be fine- tuned give good value representation new tasks (Ma et al. 2023). The original fine-tuning loss LLIV Cr(w1 + Li th)) is cal- culated on sampled sub-trajectory batch data
{o'..... :....... of.g''] 1 from each task Tú where E - S k T. For
V task i, (741) will use time contrastive learning to increase the similarity S(o of,)) between the sampled start frame and the end frame and keep the embedding distance between two adjacent frames as (n-discounted) 1, and L,, encourages the image goal of and language goal gd have the same embedding for the same task Details of this process can be found in Appendix A.2
Finetuning LIV on Long-Horizon Imperfect Videos. Finetuning LIV with the original training objective works well on short-horizon perfect videos (about 50 frames in their original papers (Mal (et al.) 2022; 2023). However, we find that it does not work well for our long- long-horizon imperfect videos, shown in Figure where the fine-tuned value curve exhibits numerous jagged peaks.
5

---

Published as a conference paper at ICLR 2025
Algorithm 1 Flow-Centric Generative Planning
1: Input: Current observation history 00-1:0. language goal query points p. flow prediction
network f,, flow-conditioned video generation network DD vision-language value module V. 2: Hyperparameters: Flow candidates number A, planning Beams B, planning horizon H.
3: Initialization: Flow plans fp + [Po- E 1....). .l. video plans Up f -00-h:0=+ E 1....... B].
4: for h 1. H do
5: for = 1... do
6:
Get the Latest Observation History in the Plan Beam & Generate Different Flow Actions Generate Corresponding Different Video Clips id f- arg max R(v, g) for in (1 AI)..)
[p|b][-h:]]
7:
01:A f- P.9)
8:
VI:A t D(o, (4.g) for in.1... ...
9:
fy (bbp.append(aa)) up 1p[b] apppnddd
b Add Plans with Highest Value
11:
end for
12:
mar idr, min idæ <- arg max V), arg min( V) Sp m/mmn_id)) <- Sp amar__daa up oplmin_dda) &- Up (marridd))
13:
D Periodically Replace
end for
15: f + fylarg max V)], f Up aarg max V)]
Return Highest Value Plan
This is disastrous for sampling-based planning algorithms since most planning algorithms expect smoothing value curve be effective (Selman & Gomes 2006
Ma
al. 2023).
We point out that this problem is caused by imperfect long-horizon videos, where the task does not necessarily progress smoothly the video progresses. For example, the robot arm may hesitate in the air during the task. To mitigate this problem, we replace the concept of adjacent frames in the original loss to adjacent states, where we define states as short-horizon video clips. Formally, we di- vide mon-hhoizoon video into small segments of fixed length and treat each clip solip as the smallest unit of the video. The original 0g, OT, Oj, Ok+1 seamlessly replaced by sclip clip clip clip spectively. As shown in Figure 3, this simple strategy is surprisingly useful and makes the fine-tuned vale curve much smoother than the originally fine-tuned ones.
ST ,8 8x41
4 FLOW-CENTRIC GENERATIVE PLANNING
4.1 MODEL-BASED PLANNING WITH FLOWS, VIDEOS, AND VALUE FUNCTIONS
Directly generating Long-hrizzon videos autoregressively is usually not accurate (Wen et al. 2023 Yang al. 2023 Du et al., 2024) due compounding errors. In this work, we use model-based planning to search for a sequence of flow actions and video plans that maximizes the discounted return:
6o. arg max
7 R(o,.g)-
o0.1. D
According to Bellman Equation (Sutton 2018), this equals stepping towards the next state that max- each time step given an optimal value function V*. In our problem, = -1 is constant for every step before reaching the goal, and we assume our learned value function V V*, thus our problem simplified find the next state that maximizes V at each time step. Note this reward design also encourages finding the shortest plan. We use hill climbing (Sel-) man & Gomes 2006) to solve this problem. It initializes B plan beams. At each timestep 1,, given current image history 0r-h:t and the language goal g. it employs T to generate multiple flow ac- tions 1++1 = n. (or-Ait Pt, gt.g) on uniformly sampled query points as candidates for tree search, then use D to generate corresponding short-horizon videos 0.4 = D(or-hit. 1ith 9, Pr+ L.). The value module V is then used to select the generated video with the highest reward among A videos to enable the next iteration of generation for each beam. In order prevent exploitative planning routes that over-exploit on an irregular state, we periodically replace the lowest value plan among the beams with the beam with the highest value. The algorithm summarized in Algorithm
imizes Fz + >V*
(st+1,9) +V**(*++19)))

---

Published as a conference paper at ICLR 2025
LIBERO-LONG (Liu et 2024a
FMB-S (Luo et al. 2023
FMB-M -M
Folding
Unfolding
UniPi (Du et al. 2024
2%
0%
0%
20%
10%
FLIF-NV
78%
52%
40%
100%
70%
FLIs(
100%
86 %
78%
100%
90%
Table 1: Success rates of model- planning long-horizon tasks.
LIBERO-LONE
(Li et
2024
FMB
Luo al.
2025
Bridge V2
Walke el.
2023
Latent L2
FVD J
PSNR
Latent L2 L
FVD
PSNR 1
Latent L2
FVD L
PSNR 1
LVDM al. (022b
610.98
358.22
12.349
0.373
153.41
16.481
IRASim 7hu ala M0Z4
0.407
206.28
12.205
0.395
172.45
13.157
0.325
138.97
16.796
FLTROOu))
35.62
43.712
25.531
0.173
36.15
33.485
Table 2: Quantitative results long horizon video generation.
4.2 PLAN-CONDITIONED LOW-LEVEL POLICY
The low-level policy are given the image observation history 0t-h:1, the language goal 9, and the predicted flow plan Pr.++, r. as well as the video plan 01+1s4 1. = (ott- h:t: 9: 1t41:41))) to predict the low-level robot action 1 that drive the robot to operate in the environment. We train different policies that take as input different kinds of condition information, with all of them trained on a few demonstrations with action labels. The policy architectures similar to diffusion policy (Chi et al. 2023). Details can be found in Appendix A.3
5 EXPERIMENTS
In this section, we first demonstrate that FLIP can: 1) perform model-based planning for different manipulation tasks; 2) synthesize long-horizon videos (2 200 frames); and can guide the low- level policy for executing the plan for both simulation and real-world tasks. We also evaluate the action, dynamics, and value modules separately compared corresponding baselines and show the interactive, zero-shot, scalability properties of FLIP. More results and videos are on our website
5.1 MODEL-BASER PLANNING FOR MANIPULATION TASKS
Setup. this section, we train FLIP on four benchmarks show its model-based planning ability. The model is given initial image and language instruction, and it is required to search the flow and video spaces to synthesize the plan for this task. The first one is LIBERO-LONG (Liu et al. 2024a), long-hone long-horizon table-top manipulation benchmark of 10 tasks in simulation. We train FLIP on 50 x 10 long-horizon videos with a resolution of 128 x 128 x and test on 50 x 10 new random initializations. The second one is the FMB benchmark (Luo et al. 2023), long-horizon object manipulation and assembly benchmark with varying object shapes and appearances. We train FLIP on 1K single-object multi-stage videos and 100 multi-object multi-stage videos with resolution of 128 x 128 x 3 and test on 50 new initialization for each. The third and fourth suites are cloth folding and cloth unfolding. These two datasets are collected by ourselves. We train each suite on 40 videos with varying viewpoints and test on 10 new viewpoints for each with resolution of 96 128 x 3.
We follow previous works Du et al. 2023 Zhu et al. 2024 and evaluate our model-based planning results by human evaluating the correctness of generated video plans. That is, we visually assess the percentage time the video successfully solved the given task. We compare FLIP to two baselines: 1) UniPi (Du et al. 2024), text-to-vides generation method with long-horizon text goals. 2) FLIP- NV, an ablation of FT.IP that performs the same beam search but with no value module as guidance.
Results. Table shows the results. We can see that UniPi achieves low success rates across all tasks, which shows that directly synthesizing long-horizon videos is difficult. FLIP-NV achieves better results than UniPi. This shows that with dense flow information as guidance, the performance of the video generation model is improved. FLIP outperforms all baselines, pointing out the ef- fectiveness of using value functions for model-basee planning. This can eliminate incorrect search routes during planning. We show such incorrect search routes website.
7

---

Published as a conference paper at ICLR 2025
Figure 4: Model-basee based planning results LIBERO-LONG, FMB, cloth folding, and cloth unfold- ing. All the flows, images, and values shown are generated by FLIP.
5.2 LONG-HORIZON VIDEO GENERATION EVALUATION
Setup. In this section, we quantitatively evaluate the long-horizon video generation quality of FLIP compared to other video generation models. We choose the same datasets as in Section [.]]jo well as Bridge-V2 V2 (Wakke al. 2023) the evaluation benchmarks. Here all videos are longer than 200 frames except for Bridge- V2. For Bridge-V2, we train on 10k videos and test on 256 videos with resolution 96 x 128 3. We choose two baselines: LVDM (He et al. 2022b), a state-of-the- text-to-video method for video generation; 2) IRASim (Zhu al. 2024). conditional video gen- eration method with the end-effector trajectories as the condition. We use SAM2 (Ravi et al. 2024) to label the end-effector trajectory for IRASim. We choose model-based metrics including Latent L2 loss and FVD (Unterthiner et al. 2018) as well as a computation based metric PSNR (Hore & Ziou 2010). Latent 1.2 loss and PSNR measure the L. distance between the predicted video and the ground-truth video in the latent space and pixel space, and FVD assess video quality by analyzing the similarity of video feature distributions
Results. 2 shows the results. We can see that our method consistently outperforms baselines all datasets. LVDM performs badly on LIBERO-LONG and FMB, and better on Bridge-V2. This is because the videos in Bridge-VE V2 are shorter than the previous two benchmarks. IRASim performs better than LVDM, which shows the importance of trajectory guidance. However, it generates long- horizon videos in auto-regressive manner, which has worse results than our method. showing that mosellbased planning can also help generate high-quality videos by concatenating short-horizon videos generated with rich flow guidance. The results the FMB benchmark are the worst for all methods. This is because the training videos have many discontinuous transitions, where the robot gripper instantly moves to where the next stage begins. Since our model leverages history observa- tions as input conditions, it can sometimes overcome this discontinuous gap. We qualitatively show the model-based planning results on the four tasks in Figure 4
Since FLIP is a universal framework for all manipulation tasks as long as they have language- annotated video datasets, here we qualitatively show FLIP can be used for complex long-horizon video generation including the ALOHA tasks (Aldaco et al. (024)), pen spinning (Wang et al., 2024], robot pilling (Chen al. 2024). tying plastic bags (Gao al., 2023), and human peeling eggs, as shown in Figure 7 More video demos are on our website.
5.3
GUNNGUIDED LOW-LEVEI POLICY
Setup. In this evaluation we explore how the generated flow and video plans can be used con- ditions for training a manipulation policy to accomplish the task. We aim to answer the question: which one, flow video (or both at the same time), is more suitable be used as the condition to guide the learning of the underlying strategy? We use LIBERO-LONG (Liu et al. 2024a) for evalu- ation, where for each task in LIBERO-LONG, we use 10 demonstrations with action labels and 50 demonstrations without action labels, as done in the baseline method ATM (Wen et al., 2023). Dur- ing inference, FLIP a close-loop policy that will replan after every action chunking. We compare FLIP to ATM Wen et al. (2023) and its diffusion-polli et a.. (024) (with both zeros-shot and fine-tuned version) and policies with pretrained FLIP on LIBERO-90 the planner. Please Appendix B.2 for these results.
version. We also compare Open' AppenVLA Kim
8

---

Published as a conference paper at ICLR 2025
LIBERO-LONG
Success Rate (%)
Ap
MM
FC-Fullset
DP
ATM DP Ours Curs-V Ours
Tasks
PreTrained
Finetune-LIV
Finetune-Ours
Figure 5: Success rates of different low-level Figure 6: Value curves from the pretrained LIV, fine- tuned by LIV, and fine-tuned by FLIP.
policies on LIBERO-LONG.
LIBERO-10
Language- Table
Bridge-V2
Latent L. L
FVD L
PSNR
Latent L2. L
FVD
PSNR
Latent L. J
FVD
PSNR 1
LVDM al. IRASim
0.366
0.364
124.75
111.34
Zhu al 202-
0.307
92.76
19.205
0.335
132.56
18.156
0.318
107.89
19.967
SL
0.271
89.77
20.089
0.304
137.89
18,904
0.316
127.65
18.375
FLIP(OUrs)
0.197
27.62
28.602
0.159
21.23
33.632
0.171
38.41
34,576
Table 3: Quantitative results on short-horizon video generation.
Results. The results are in Figure We can see that our dan-guided policies achieve higher success rates than diffusion policies and ATM-DP, showing that dense flow information and high- quality future videos are better be used as conditions than sparse flow information. The flow- video-guided policy (Ours-FV) achieves the best average success rates across all methods, showing the advantage of using multi-modality information as conditions. Although video-guided policies (Ours-V) V) achieve competitive mean success rate, they have high variance, showing that using videos as conditions is unstable. This may come from that the generated future vidcos can become low-quality if the robot deviates from the trained trajectories. Instead, with the flow extra condi- tions, the variance becomes lower, showing the stability of dense image flow predictions.
5.4 EXPERIMENTS ON FUNDAMENTAL MODULES OF FLIP
Action Module Experiments. We use two metrics to assess the flow generation model Ty quantitatively (Jiang et al., 2024b): 1) Average Distance Error (ADE) between the generated and the ground truth flows in pixel units on all query points; 2) Less Than Delta Ratio (LTDR)): the average percentage of points within the dis- tance threshold of 1, 2, 4, and 8 pixels between the reconstructed and the ground truth flows at each time step. Since most of the points are stationary points, in order to better demonstrate the results, we only calculate points with 6, 2 1. We also do experiments that compare using CVAE and diffusion models as the action module in Appendix B.3
LIBERO-10
Bridge V2
ADE
LTDRR
ADE
LTDR 1
ATM Wen et al 2023
19.6
53.8%
18.4
66.1%
Uurs-ABS
20.5
17.9
Ours-NoAUX
14.5
73.2%
12.7
75.6%
76.5%
Table 4: Quantitative results of the action model.
We use LIBERO-LONG (Liu et al.) 2024a) and Bridge- V2 (Walke et al., 2023) for evaluation. We compare our method with 3 baselines: ATM (Wen et al. (023)) the statc-oi-ther thet--rr flow prediction module for manipulation tasks; 2) Ours-ABS: directly generating absolute flow coordinates each timestep rather than generating the scale and direction; 3) Ours-NoAUX: the same architecture of ours with no auxiliary training losses (the flow and image reconstruction losses).
From Table we can see that Ours-ABS generally achieves the same results ATM, and predicting the scale and directions are better than ATM and Ours-ABS, showing that directly regressing the absolute coordinates is worse than predicting the delta of flows at each timestep. We can also see that the auxiliary losses can help improve the final results.
Dynamics Module Experiments. We evaluate our dynamics module separately with the ground truth flows conditions on short-horizon video generation. We use PSNR (Hore & Ziou 2010 latent L2 loss, and FVD (Unterthiner et al. 2018) as metrics. We use LIBERO-LONG (Liu et al, 2024a), Bridge- V2 (Walke et al. 2023), and Language- Table (Lynch al. 2023) as the evaluation datasets. We use three baselines (as introduced in Section 5.2): 1) LVDM (He et al., 2022b); 2) IRAsim (Zhu et al. 2024); 3) Ours-SC: using AdaLN- for all kinds of conditions.
Results are in Table The result trends across methods are generally consistent with the long- horizon video generation results in Table 2 FL.P--C generally achieves the same performance with

---

Published as a conference paper at ICLR 2025
Figure 7: FLIP is general framework for diverse kinds of manipulation tasks across objects and robots, even for human hands. All of the flows and images are generated.
ADE
Locore 12 Loss
Thaining Ships
Figure 8: Interactive ability.
Figure 9: Zero-shot transfer.
Figure 10: Scalability.
IRASim, showing that even if the model is given dense flow information, requires a fine-grained
mechanism to leverage the condition for video generation.
Value Module Experiments. We here qualitatively show the fine-tuned value curves of our method compared to the original LIV (Ma et al. 2023) method on two different tasks consisting of Language-Tabll (Lynch et al. (023) and cloth folding in Figure|6) We also show the value curves before fine-tuning. We see our method consistently gets smoother value curves than the original LIV method, where the value curves have violent oscillations.
5.5 APPLICATIONS AND SCALING
Finally, train FLIP LIBERO-90 90, large-scale simulation manipulation dataset to show three properties FLIP. We use 50 videos for each task in the resolution of 3 x x 64.
Interactive World Model. We first show that the trained dynamics module is interactive: it can generate corresponding videos given image flows specified by humans. We use SAM2 (Ravi al.) 2024) to select the region of the robot arm and manually give flows in different directions. Results We can see the robot arm can move left or right according to the given flow.
are shown Figure
Zero-Shot Generation. Secondly, we show that the trained FLIP has zero-shot transfer ability. We test the trained model on LIBERO-LONG. Results are shown in Figure 9 Interestingly, we can see that the pretrained model, without fine-tuning, can generate natural movement for the robot arm with unseen observations and instructions. This shows FLIP has certain knowledge transfer ability.
Model Scaling, We show that the action and dynamics module are scalable with increasing model sizes. Figure I0 shows the smoothed ADE and Latent L2 loss on the validation set. shows that increasing the model size can consistently help achieve better performance for both modules.
6 CONCLUSION AND LIMITATION
In this work, we present FLIP, a flow-centric generative planning method for ulation tasks. FLIP is trained only video and language data, can perform model-based planning the trained world model to synthesize long-horizon plans, and can guide low-level policy learning. FLIP has the potential to scale up with increasing data and computation budgets in the future.
manip-
major limitation of FLIP is the slow speed of planning, which is restricted by extensive video gen- eration processes during the planning phase. This restricts method quasi-stati manipulation tasks. Another limitation is that FLIP does not use physical properties and 3D information of the scene. Future works develop physical 3D world models and extend FL.P 3D scenarios.
10

---

Published as a conference paper at ICLR 2025
REFERENCES
Jorge Aldaco, Travis Armstrong, Robert Baruch, Jeff Bingham, Sanky Chan, Kenneth Draper, De- bidatta Dwibedi, Chelsea Finn, Pete Florence, Spencer Goodrich, al. Aloha 2: An enhanced low-cost hardware for bimanual teleoperation. arXiv preprint arXiv:2405.00292. 2024.
Shikhar Bahl, Russell Mendonca, Lili Chen, Unnat Jain, and Deepak Pathak. Affordances from hu- man videos as a versatile representation for robotics. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13778-13790, 2023.
Bowen Baker, 1ige Akkaya, Peter Zhokov, Joost Huizinga, Jie Tang, Adrien Ecoffet, Brandon Houghton, Raul Sampedro, and Jeff Clune. Video pretraining (vpt): Learning to act by watching unlabeled online videos. Advances in Neural Information Processing Systems, 15:24639-24654 2022.
Fan Bao, Shen Nie, Kaiwen Xue, Yue Cao, Chongxuan Li, Hang Su, and Jun Zhu. All are worth words: A vit backbone for diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 22669-22679, 2023.
Homanga Bharadhwaj, Roozbeh Mottaghi, Abhinav Gupta, and Shubham Tulsiani. Track2act: Pre- dicting point tracks from internet videos enables diverse zero-shot robot manipulation. arXiv preprint arXiv:2405.01522 2024.
Andreas Blattmann, Tim Dockhorn, Sumith Kulal, Daniel Mendelevitch, Maciej Kilian, Dominik Lorenz, Yam Levi, Zion English, Vikram Voleti, Adam Letts, et Stable video diffusion: Scaling latent video diffusion models to large datasets. arXiv preprint arXiv:2311. 2023.
David Brandfonbrener, Ofir Nachum, and Joan Bruna. Inverse dynamics pretraining learns good representations for multitask imitation. Advances Neural Information Processing Systems, 36, 2024.
Tim Brooks, Bill Peebles, Connor Holmes, Will DePue, Yufei Guo, Li Jing, David Schnurr, Joe Taylor, Troy Luhman, Eric Luhman, Clarence Ng. Ricky Wang, and Aditya Ramesh. Video generation models as world simulators. 2024. URL htttts:://pp..taee
Pideo-geneeatto moneatilln-mmmlssss as wor Id simulators
Jake Bruce, Michael D Dennis, Ashley Edwards, Jack Parker-Holder, Yuge Shi, Edward Hughes, Matthew Lai, Aditi Mavalankar, Richie Steigerwald, Chris Apps, et al. Genie: Generative inter- active environments. In Forty-first International Conference on Machine Learning, 2024.
Charlotte Caucheteux and Jean-Rémi King. Brains and algorithms partially converge in natural language processing. communications biology, (1), 134, 2022.
Tao Chen, Eric Cousineau, Naveen Kuppuswamy, and Pulkit Agrawal. Vegetable peeling: A case study in constrained dexterous manipulation. arXiv preprint arXhv:2407.07884 2024.
Cheng Chi, Zhenjia Xu, Siyuan Feng. Eric Cousineau, Yilun Du, Benjamin Burchfiel, Russ Tedrake, and Shuran Song. Diffusion policy: Visuomotor policy learning via action diffusion. The Inter- national Journal of Robotics Research, pp. (2783649241273666 2023.
Norman Di Palo and Edward Johns. Keypoint action tokens enable in-context imitation learning in robotics. arXiv preprint 2xXv:24433.1957 2024.
Danny Driess. Fei Xia, Mehdi SM Sajjadi, Corey Lynch. Aakanksha Chowdhery, Brian Ichter, Ayzaan Wahid, Jonathan Tompson, Quan Vuong, Tianhe Yu, al. Palm-e: An embodied multi- modal language model. arXiv preprint arXiv:2303.0333 2023.
Yilun Du, Mengjiao Yang, Pete Florence, Fei Xia, Ayzaan Wahid, Brian Ichter, Pierre Sermanet, Tianhe Yu, Pieter Abbeel, Joshua B Tenenbaum, et al. Video language planning. arXiv preprint arXiv:2310.1062. 2023.
Yilun Du Sherry Yang, Bo Dai, Hanjun Dai, Ofir Nachum, Josh Tenenbaum, Dale Schuurmans, and Pieter Abbeel. Learning universal policies via text-guided video generation. Advances in Neural Information Processing Systems, 36, 2024.

---

Published as a conference paper at ICLR 2025
Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang. Angela Fan, et al. The llama 3 herd of models. arXiv preprint arXhv:240..21783, 2024.
Norm Ferns, Prakash Panangaden, and Doina Precup. Metries for finite markov decision processes. In UAI, volume 4, pp. 162-169, 2004.
Chelsea Finn and Sergey Levine. Deep visual foresight for planning robot motion. In 2017 IEEE International Conference on Robotics and Automation (ICRA), pp- 2786-2793, IEEE, 2017.
Chongkai Gao, Haichuan Gao, Shangqi Guo, Tianren Zhang. and Feng Chen. Cril: Continual robot imitation learning via generative and prediction model. In 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 6747-5754. IEEE, 2021.
Chongkai Gao, Zekun Li, Haichuan Gao, and Feng Chen. Iterative interactive modeling for knotting plastic bags. In Conference on Robot Learning, pp. 571-582, PMIR, 2023.
Kristen Grauman, Andrew Westbury, Eugene Byrne, Zachary Chavis, Antonino Furnari, Rohit Gird- har, Jackson Hamburger, Hao Jiang, Miao Liu, Xingyu Liu, et al. Ego4d: Around the world in 3,000 hours of egocentric video. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 18995-19012, 2022.
David Ha and Jürgen Schmidhuber. Recurrent world models facilitate policy evolution. Advances in neural information processing systems, 31, 2018.
Danijar Hafner, Timothy Lillicrap. Mohammad Norouzi, and Jimmy Ba. Mastering atari with dis- crete world models. arXiv preprini arXiv:2010.00 193, 2020.
Danijar Hafner, Jurgis Pasukonis, Jimmy Ba, and Timothy Lillicrap. Mastering diverse domains through world models. arXiv preprint arXiv:2301 000044,0004,,,... 2023.
Nicklas Hansen, Hao Su, and Xiaolong Wang. Td-mpe2: Scalable, robust world models for contin- uous control. arXiv preprint arXiv:2310. 16828, 2023.
Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Dollár, and Ross Girshick. Masked au- toencoders are scalable vision learners. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 16000-16009, 2022a.
Yingqing He, Tianyu Yang, Yong Zhang, Ying Shan, and Qifeng Chen. Latent video diffusion models for high-fidelity long video generation. arXiv preprint arXiv:2211.33211, 2022b.
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in
neural information processing systems, 33:6840-6851, 2020.
Alain Hore and Djemel Ziou. Image quality metrics: Psnr vs. ssim. In 2010 20th international conference on pattern recognition, pp. 2366-2369, IEEE, 2010.
Wenlong Huang, Chen Wang, Yunzhu Li, and Fei-fei Li. Rekep: Spatio-temporal reasoning of relational keypoint constraints for robotic manipulation. 2024.
Hanxiao Jiang, Binghao Huang, Ruihai Wu, Zhuoran Li, Shubham Garg, Hooshang Nayyeri, Shen- long Wang, and Yunzhu Li. Roboexp: conditoond scene graph via interactive exploration for robotic manipulation. arXiv preprint
2024a.
Zhenyu Jiang, Hanwen Jiang, and Yuke Zhu. Doduo: Learning dense visual correspondence from unsupervised semantic-aware flow. In 2024 IEEE International Conference on Robotics and Automation (ICRA), pp. 12420-12427. IEEE, 2024b.
Lukasz. Kaiser, Mohammad Bahaeizadeh, Piotr Milos, Blazej Osinski, Roy H Campbell, Konrad Czechowski, Dumitru Erhan, Chelsea Finn, Piotr Kozakowski, Sergey Levine, et al. Model-based reinforcement learning for atari. arXiv preprint arXiv:1903.00374 2019.
Nikita Karaev, Ignacio Rocco, Benjamin Graham, Natalia Neverova, Andrea Vedaldi, and Christian Rupprecht. Cotracker: It is better to track together. arXiv preprint arXiv:2307.07635, 2023.
12

---

Published as a conference paper at ICLR 2025
Moo Jin Kim, Karl Pertsch, Siddharth Karamcheti, Ted Xiao, Ashwin Balakrishna, Suraj Nair, Rafael Rafailov, Ethan Foster, Grace Lam, Pannag Sanketi, et al. Openvla: An open-source lision-language-actin model. arXiv preprint arXiv:2406.0922 2024.
Diederik Kingma. Auto-encoding variational bayes. arXiv preprini arXiv: 1312.6111,..14, 2013.
Sateesh Kumar, Jonathan Zamora, Nicklas Hansen, Rishabh Jangir, and Xiaolong Wang. Graph inverse reinforcement learning from diverse videos. In Conference on Robot Learning, pp. 55- 66. PMLR, 2023.
Yann LeCun. A path towards autonomous machine intelligence. 2022. URL https://openreview: net/pdf, 2024.
Timothée Lesort, Natalia Díaz-Rodríguez, Jean-Franois Goudou, and David Filliat. State represen- tation learning for control: An overview. Neural Networks, 108:379-392, 2018.
Junbang Liang, Ruoshi Liu, Ege Ozguroglu, Sruthi Sudhakar, Achal Dave, Pavel Tokmakov, Shuran Song, and Carl Vondrick. Dreamitate: Real-world visuomotor policy learning via video genera- tion. arXiv preprint arXiv:2406. /6862, 2024.
Bo Liu, Yifeng Zhu, Chongkai Gao, Yihao Feng, Qiang Liu, Yuke Zhu, and Peter Stone. Libero: Benchmarking knowledge transfer for lifelong robot learning. Advances in Neural Information Processing Systems, 36, 2024a.
Shaowei Liu, Zhongzheng Ren, Saurabh Gupta, and Shenlong Wang. Physgen: Rigid-body physics- grounded image-to-video generation. In European Conference Computer Vision, 2024b.
Jianlan Luo, Charles Xu, Fangchen Liu, Liam Tan, Zipeng Lin, Jeffrey Wu, Picter Abbeel, and Sergey Levine. Fmb: A functional manipulation benchmark for generalizable robotic learning. The International Journal of Robotics Research, pp. 02783649241276017 2023.
Corey Lynch, Ayzaan Wahid, Jonathan Tompson, Tianli Ding, James Betker, Robert Baruch, Travis Armstrong, and Pete Florence. Interactive language: Talking to robots in real time. IEEE Robotics and Automation Letters, 2023.
Xin Ma, Yaohui Wang, Gengyun Jia, Xinyuan Chen, Ziwei Liu, Yuan-Fang Li, Cunjian Chen, and Yu Qiao. Latte: Latent diffusion transformer for video generation. arXiv preprint 2rXi::2401.03048, 2024.
Yecheng Jason Ma, Shagun Sodhani, Dinesh Jayaraman, Osbert Bastani, Vikash Kumar, and Amy Zhang. Vip: Towards universal visual reward and representation via value-implicit pre-training. arXiv preprint arXiv:2210.000300 2022.
Yecheng Jason Ma. Vikash Kumar, Amy Zhang, Osbert Bastani, and Dinesh Jayaraman, Liv: Language-image representations and rewards for robotic control. In International Conference on Machine Learning, pp. 23301-23320. PMLR, 2023.
Mario Manto, James M Bower, Adriana Bastos Conforto, José M Gelgado-Garcíal Suzete Nasci- mento Farias Da Guarda, Marcus Gerwig, Christophe Habas, Nobuhiro Hagura, Richard B Ivry, Peter Mariën, al. Consensus paper: roles of the cerebellum in motor control-the diversity of ideas on cerebellar involvement in movement. The Cerebellum, 1 11:45-4877, 2012.
Russell Mendonca, Shikhar Bahl, and Deepak Pathak. Structured world models from human videos. arXiv preprint arXiv:2308. 10901, 2023.
Vincent Micheli, Eloi Alonso, and François Fleuret. Transformers are sample-efficient world mod- cls. arXiv preprint arXiv:2209.05588 2022.
Suraj Nair, Aravind Rajeswaran, Vikash Kumar, Chelsea Finn, and Abhinav Gupta. R3m: A univer- sal visual representation for robot manipulation. arXiv preprint arXh::2203.12601 2022.
Soroush Nasiriany, Vitchyr Pong, Steven Lin, and Sergey Levine. Planning with goal-conditioned policies. Advances in neural information processing systems, 32, 2019.
13

---

Published as a conference paper at ICLR 2025
Alexander Quinn Nichol and Prafulla Dhariwal. Improved denoising diffusion probabilistic models. In International conference on machine learning, pp. 8162-8171, PMLR, 2021.
William Peebles and Saining Xie. Scalable diffusion models with transformers. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 4195-4205, 2023.
Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas Müller, Joe Penna, and Robin Rombach. Sdxl: Improving latent diffusion models for high-resolution image synthesis. arXiv preprint arXhv:2307.01952 2023.
Nikhila Ravi, Valentin Gabeur, Yuan-Ting -ing Hu, Ronghang Hu, Chaitanya Ryali, Tengyu Ma, Haitham Khedr, Roman Rädle, Chloe Rolland, Laura Gustafson, et al. Sam 2: Segment anything in images and videos. arXiv preprint arXiv:2408.0077 2024.
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björ Ommer. High- resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF confer- ence on computer vision and pattern recognition, pp. 10684-10699 2022.
Daniel Seita, Yufei Wang, Sarthak Shetty, Edward Yao Li, Zackory Erickson, and David Held. Toolflownet: Robotic manipulation with tools via predicting tool flow from point clouds. In Conference Robot Learning, pp. 1038-1049, PMLR, 2023.
Bart Selman and Carla P Gomes. Hill-climbing search. Encyclopedia of cognitive science, 81 (333-335):10, 2006.
Younggyo Sco, Junsu Kim, Stephen James, Kimin Lee, Jinwoo Shin, and Picter Abbeel. Multi-view masked world models for visual robotic manipulation. In International Conference on Machine Learning, pp. 1663-20332. PMLR, 2023.
Pierre Sermanet, Corey Lynch, Yevgen Chebotar, Jasmine Hsu, Eric Jang, Stefan Schaal, Sergey Levine, and Google Brain. Time-contrastive networks: Self-supervised learning from video. In 2018 IEEE international conference on robotics and automation (ICRA), pp. 1134-1141. IEEE, 2018.
Lin Shao, Toki Migimatsu, Qiang Zhang, Karen Yang, and Jeannette Bohg. Concept2robot: Learn- ing manipulation concepts from instructions and human demonstrations. The International Jour- nal of Robotics Reseurch, ((12-14):14199444)
2021.
Mohit Shridhar, Yat Long Lo, and Stephen James. Generative image as action models. arXiv preprint arXiv:2407.07875. 2024.
Tianmin Shu, Xiaofeng Gao, Michael Ryoo, and Song-Chun Zhu. Learning social affordance grammar from videos: Transferring human interactions human-robot interactions. In 2017 IEEE international conference on robotics and automation (ICRA), pp. 1669-1676. IEEE, 2017.
Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. arXiv preprint MrXiv:2011.13456 2020.
Richard S Sutton. Reinforcement learning: An introduction. A Bradford Book, 2018.
Thomas Unterthiner, Sjoerd Van Steenkiste, Karol Kurach, Raphael Marinier, Marcin Michalski, and Sylvain Gelly. Towards accurate generative models of video: A new metric & challenges. arXiv preprint ar 1r/iv:18/2.01117, 2018.
Dani Valevski, Yaniv Leviathan, Moab Arar, and Shlomi Fruchter. Diffusion models are real-time game engines. arXiv preprint arXiv:2408. 14837, 2024.
A Vaswani. Attention is all you need. Advances in Neural Information Processing Systems, 2017.
Homer Rich Walke, Kevin Black. Tony Z Zhao, Quan Vuong, Chongyi Zheng, Philippe Hansen- Estruch, Andre Wang He, Vivek Myers, Moo Jin Kim, Max Du, et al. Bridgedata v2: dataset for robot learning scale. In Conference Robot Learning, pp. 1723-1736. PMLR, 2023.
14

---

Published as a conference paper at ICLR 2025
Chen Wang, Linxi Fan, Jiankai Sun, Ruohan Zhang. Li Fei-Fei, Danfei Xu, Yuke Zhu, and An- ima Anandkumar. Mimicplay: Long-horizal horizon imitation learning by watching human play. arXiv preprint arXiv:2330..12422 2023.
Jun Wang, Ying Yuan, Haichuan Che, Haozhi Qi, Yi Ma, Jitendra Malik, and Xiaolong Wang. Lessons from learning to spin" pens". arXiv preprint arXiv:2407.18902 2024.
Chuan Wen, Xingyu Lin, John So, Kai Chen, Qi Dou, Yang Gao, and Pieter Abbeel. Any-point trajectory modeling for policy learning. arXiv preprint arxiv:2401.00025, 2023.
Jialong Wu, Shaofeng Yin, Ningya Feng, Xu He, Dong Li, Jianye Hao, and Mingsheng Long. ivideogpt: Interactive videogpts scalable world models. arXiv preprint 23X 2024.
Mengda Xu, Zhenjia Xu, Yinghao Xu, Cheng Chi, Gordon Wetzstein, Manuela Veloso, and Shuran Song. Flow the cross-domain manipulation interface. arXiv preprint arXiv:2407. 15200., 2024a.
Zhixuan Xu, Chongkai Gao, Zixuan Liu, Gang Yang, Chenrui Tie, Haozhuo Zheng, Haoyu Zhou, Weikun Peng, Debang Wang, Tianyi Chen, et al. Manifoundation model for general-purpose robotic manipulation of contact synthesis with arbitrary objects and robots. arXiv preprint SEf.:2200500664., 2024b.
Mengjiao Yang. Yilun Du Kamyar Ghasemipour, Jonathan Tompson, Dale Schuurmans, and Pieter Abbeel. Learning interactive real-world simulators. arXiv preprint 1 14, 2023.
Chengbo Yuan, Chuan Wen, Tong Zhang, and Yang Gao. General flow as foundation affordance for scalable robot learning. arXiv preprint arXiv:2401. 11439, 2024.
Kevin Zakka, Andy Zeng, Pete Florence, Jonathan Tompson, Jeannette Bohg, and Debidatta Dwibedi. Xirl: Cross-embodiment inverse reinforcement learning. In Conference Robot Learning, pp. 537-546. PMLR, 2022.
Kaifeng Zhang, Baoyu Li, Kris Hauser, and Yunzhu Li. Adaptigraph: Material-adaptive graph-based neural dynamics for robotic manipulation. arXiv preprint arXiv:2407.07889, 2024.
Lvmin Zhang, Anyi Rao, and Mancesh Agrawala. Adding conditional control text-to-image diffusion models. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 3836-3847, 2023.
Siyuan Zhou, Yilun Du, Jiaben Chen, Yandong Li, Dit- Yan Yeung, and Chuang Gan. Robodreamer: Learning compositional world models for robot imagination. arXiv preprint arXiv:2404.12377, 2024.
Fangqi Zhu, Hongtao Wu, Song Guo, Yuxiao Liu, Chilam Cheang, and Tao Kong. Irasim: Learning interactive real-robot action simulators. arXiv preprint arXiv:2406. 14540, 2024.
15

---

Published as a conference paper at ICLR 2025
A METHOD DETAILS
A.1 LATENT DIFFUSION MODELS
Diffusion Models. Diffusion models (Song et al. 2020 Ho et al. 2020) typically contain a forward nosing process and reverse denoising process. During the forward process, grad- ually apply noise to real data zy; (( = N(xx va,zu, Tu (1 a.) over timesteps, where constants ae hyperparameters. By applying the reparameterization trick,
sam-
ple 24 = Va + vI- CgCt, where 6 ~ (0. During the reverse process, it starts from Gaussian noise ØT (O,, I) and gradually removes noises recover zn: po( (x+) ; Eg (x))) With reparameterizing μs as a noise prediction network 6p. the model can be trained with Csample (0) = < Eell.. We also learn the covariance 2g following Nichol & Dhariwal 2021); Peebles & Xie (2023) with the full KL loss.
Latent Diffusion and Tokenization. Latent diffusion models (Rombach et al. 2022; Ma et al, 2024) perform diffusion process in low-dimensional latent space 2 rather than the original pixel space. We leverage the pre-trained VAE in SDXL. (Podell et al.] 2023 to compress each frame or to latent representations: 2t Enc(or), and after the denoising process, the latent representation can be decoded back to the pixel space with the VAE decoder: oz Dec(z(d). For each z'd it is divided into image patches and tokenized by convolutional networks P tokens with D dimensions (hidden size). Sequencing the image tokens of all ' frames, we get the video token in the shape of T X D.
Attention Mechanism. We leverage transformers (Vaswani 2017) to imple- ment the dynamics module, and use the memory-
attention mechanism (Ma) al. 2024 Bruce et al.) 2024; Zhu et al. (024), where each attention block consists of spatial attention block and a temporal attention block. The spatial attention operates on the x P tokens within each frame, and the temporal attention operates on the 1 tokens across timesteps the same location.
A.2 LANGUAGE- VISION REPRESENTATION
The original fine-tuning loss LLVV = (w(())) + LL (v'r, 1/L) is calculated on sampled sub-trajectory
batch data (o....., : of of of. } from each task Tr, where s [0. T. 11, s T,. They have the following forms:
E((WT) = 1-7
(1 log
explS(> vy +1 +1 (0(
c (0>2)((((0))))))
1-7
Ex(env))
Σ
log
B
B Σ (c
A.3 LOW-LEVEL POLICY
In this work, use a low-level policy with a similar structure diffusion policy CChi We show the architecture of this policy in Figure We employ a spatial-temporal attention mech- anism. Specifically, the input contains the agent view observation history of
2023
x3
and the eye in hand observation history of-12:t 122x3x128 128x 128 at timestep t, the proprioception
state history 8t-12:4 E 12x123 the language tokens extracted from Meta Llama 3.1 8B (Dubey et al. 2024 g E RT, 4409
the predicted flow for both the agent view P E IR 16x629x2 and E 121xx29x2, and the predicted future videos for both the agent view 16 E R 11663x128XX12 128 and eye in hand view o² 10 E I11x3x128x128 The low-level policies are a denoising model that will predict the gradient field the action chunking (E(,),, and after 500 denoising steps, can get the future action sequences E 116x7 where 7 is the action size. We use the action in a receding horizon manner where we only execute 8 steps and we replan the future flow and videos and predict another 16 action steps iteratively.
eye in hand view 16
16

---

Published as a conference paper at ICLR 2025
Eye Hand Flows
MLP
Gradient Field (E(A)
Agent View Flows
MLP
Eye Hand
ResNet
Transformer Encoder
Transformer Decoder
Fustor
Agent Hustory+Future
ResNet
Positional Embedding
State History
MLP
Instruction
Llama
Action Chunking A;
Figure 11: Low-level policy architecture. This is Ours-FV. Ours-F and Ours-V are in the same
architecture without corresponding input conditions.
B EXPERIMENT DETAILS
B. TRAINING DETAILS
We report the hyperparameters of the models we trained in Table and Table We train all data with observation history equals to 16 and future flow horizon equals to 16.
CVAE-S
CVAE-B
CVAE-L
Encoder Layer
4
8
Decoder Layer
6
12
Hideen Size
384
768
1024
Learning Rate
le-4
5e-5
le-5
Image Patch Size
8
8
Head Number
4
8
12
Table 5: Hyperparameters of CVAE.
D-S
D-B
D-L
Layers
12
16
Hideen Size
384
768
1024
Learning Rate
le-4
le-4
le-4
Head Number
12
16
Table Hyperparameters of the dynamics module.
B.2 MORE POLICIES RESULTS
Besides the low-level experiments in Section 5.3 we also perform experiments in the LIBERO-
LONG task suite with two other models:
1. VLe Kim et al. 2024): this large-scale retained that is designed for general- purpose vision- vision-langue lanoongggaaggtiiionn prediction. We use the pretrained checkpoint from the of- ficial paper and test with both zero-shot and fine-tuned models. We use 50 demonstrations for each task in LIBERO-LONG fine-tune the OpenVLA model. For consistency, resolution of 128 128 for fine- fine-tuning, For fair comparisons, we only use the agent view images as input because the original VLe is only trained with third-view images.
2. Pretrained FLIP LIBERO-90 and fine-tuned on LIBERO-10 10 as the high-level planner. In this setting, we train FLIP on 50x90 action-less demonstrations with a resolution of 64 from LIBERO-90 and finetune it with 50x 10 from LIBERO-LONG, and use this FLIP model as the planner to train the same low-level policy as in the main paper. Here we only test the flw-conditionee policy version, and call it Ours-90.
17

---

Published as a conference paper at ICLR 2025
Results. We show the results together with the main paper results in Figure 12 We that OpenVLA cannot handle the long-horizon tasks of LIBERO-LONG either with zero-shot or fine- tuned models, showing there is still long way go for ceneral-purposs vision-language-acn models. Ours-90 performs similarly to Ours-F and Ours-FV, showing that pretraining in other tasks may not bring significant improvement for low-level policy learning. This comforts with the life- long learning results in the original LIBERO paper (Liu et al.) 2024a), where they also show that pretraining cannot help (sometimes even hurt) the policy training results.
It is worth noting that, in the original Open VLA paper, they also fine-tuned the pretrained model on LIBERO-LONG tasks and archived a 53.7 1.3% success rate. We think the success of their results comes from two aspects, which cannot be true in our setting: 1) we are using a resolution of x28x 128, which may not be large enough represent the details in the scene. In comparison, OpenVLA uses resolution of 256x256. 2) We are using the official demonstrations provided by the LIBERO paper, which may not be as good as the re-collected demonstrations in their demon- strations.
LIBERO-LONG
Success
UpenViA OpenMAH
IK-hulbet
Ourstt
Oursey
Durs- SE
Figure 12: Low-level policy results in LIBERO-LONG.
B.3 ABLATION STUDIES OF ACTION MODULE
To verify if diffusion models can generate better results for image flow prediction (the action module of FLIP) than a CVAE model, here we design a diffusion model for the flow generation task. We use the same architecture the CVAE decoder for the diffusion model (as illustrated in Figure where a transformer encoder takes as input the query points tokens, the image history tokens, the language embedding tokens, and noisy inputs, and outputs the scale and direction gradient fields for each query point. The network structure is shown in Figure [3 We perform comparison experiments in both LIBERO-LONG and Bridge-V2 V2 data.
Results. Table 7 shows the results. We can see that CVAE performs better on LIBERO-LONG
tasks and the diftusion model performs better Bridge-V2 tasks, but their performance difference is minor. We can see the main improvement of the action module comes from: 1) predicting the delta action (scale and direction) rather than predicting the absolute coordinates of the image flows; 2) auxiliary losses as stated in our main paper.
Although using the diffusion model may perform better in some tasks, the inference speed is slower than CVAE models. Given that their performance is not too different, we still use CVAE as our default action module.
B.4 REAL WORLD EXPERIMENTS
In order to show the application of FLIP on real-world tasks, here we design two long-horizon real-world manipulation tasks for testing. We use a 6-DOF X-arm as the robot arm, and use two RealSense D435i cameras to get the visual inputs. We put one camera the other robot arm the third-view camera and another camera on the wrist of the robot arm as the eye-in- hn-d camera. The control frequency of both tasks is set to 10Hz. We also train diffusion policy and ATM (Wen)
18

---

Published as a conference paper at ICLR 2025
Query Points
Diffusion Model as the Action Module
ML P
Scale & Direction Notse
Scale & Direction
Transformer Encoder
MLP Decoders
MLP
Scale & Direction
Instruction Image History
Denoised
Patchify
creem cheese box and the butter the besket"
Liama
Aux Losses
XK
Reconstructed Flows
Figure 13: Using diffusion model for the action module. The model also generates the scale and directions for each query point and the full flow trajectory can be reconstructed by them.
LIBERO-LONG
Bridge V2
ADE
LTDR f
ADE 4
LTDR f
ATM (Wen et al. 2023
19.6
53.8%
18.4
66.1 1%
Ours-ABS
20.5
57.3%
17.9
59.3%
14.5
73.2%
12.7
75.6%
Ours(CVAE) 12.7
76.5%
11.9
80.2%
Diffusion
13.4
75.9%
10.2
82.7
Table 7: Comparison results of CVAE and Diffusion models the action module for FLIP.
et al. shown
2023
policy as the baseline for each task. We perform experiments in table-top setting, as Figure 14 We test both tasks with 20 random initialized episodes. The two tasks are:
1. Tea Scooping Task. In this task, there is white bowl of dry tea leaf (Tieguanyin), a yellow bowl with an iron spoon, and an empty cup the table, shown in Figure [5] The robot is trained to first pick up the spoon from the yellow bowl, then use the spoon to scoop the tea
Third View
Camera
Figure 14: Our real-world experiment setting. We use the right X-arm as the manipulator, and use two RealSense D435i cameras visual inputs.
19

---

Published as a conference paper at ICLR 2025
Random Initial State
Pick Up the Spoon
Scopa the Tea Leaf
Pour the Cup
Put Back the Spotn
Random Initial State
Grasp and Drag
Grasp and Drag
Finish
Figure 15: Two tasks in our real- experiments.
leaf from the white bowl pour the tea leaf into the cup, and finally put back the spoon into the yellow bowl. The positions both the bowls and the cup are randomized in the space that the robot arm can reach. The task is successful if all three stages are accomplished. The action space of the robot 7-dim, including the 6-DOF pose and the gripper action. We collect 40 demonstrations with scripted policy to train FLIP and flow-guided low-level policy. We choose this task because it is both and manipulation. It also requires tool use and interaction between rigid body and granular objects.
and requires precise grasping
2. Cloth Unfolding Task, this task, a pair of denim shorts was randomly thrown on the table, and the robot is trained to unfold it make it flat, shown in Figure 15] The task is successful the projection area of the jeans on the table is greater than 85% of its maximum area. The action space of the robot is 6-dim, which consists of a 2-dim end- effector coordinate in the x-y plane, a 1-dim grasping orientation angle, a 2-dim dragging coordinate in the x-y plane, and the gripper aperture. We fix the grasping height to 22 mm for this task. We collect 50 demonstrations for this task with a scripted policy. We choose this task because is both long-horizon and difficult because of the deformation of the jeans, which requires a long- horghhorizon task planning ability for the policy. The robot has to finish the task in 15 grasps and drags.
Results. Table 8 shows the results of both tasks. We see that our policy is significantly better than the baselines. These two tasks are very difficult in out setting since we only use no more than 50 demonstrations for each of them, while modern image-based imitation learning algorithms (such as Diffusion Policy (Chi et al. (023)) usually require more than 500 demonstrations for such kinds of long-horizon tasks. In contrast, our method benefits from the high-level model-based planning advantages and can correct back to the good trajectory when it deviates from it, thus the success rates are improved.
Diffusion Policy (Ch et al. 2023
ATM Wen et al. 2023
Ours
Tea Scooping Cloth Unfolding
15.0%
25%
55% 50.0 %
0.0%
0.0%
Table 8: Success rates of real robot experiments over 20 evaluations.
B.5
DATA SCALABILITY
To show the performance change of FLIP along with different amounts of training data, here we perform an experiment on LIBERO-LONG to show the planning success rates change with different
20

---

Published as a conference paper at ICLR 2025
amount of demonstrations. Results shown in Table From the results, that with more data for each task, the planning success rates become better.
LIBERO-LONG (Liu et al. 2024a)
FLIP-10
0%
FLIP-20
5%
FLIP-30
40%
FLIP-40
90%
FLIP-50
100 %
Table 9: Planning results change along with data amounts for LIBERO-LONG. The number shown in the left column is the demonstration number for each task during training.
B.6 GENERATIVE PLANNING WITH VISUAL DISTRACTION
In this section, perform qualitative experiment see how will our FLIP be affected in the presence of noise or visual obstructions. To this end, we manually add an image of an apple in the initial image of the LIBERO-LONG tasks, shown in Figure and Figure 17 and how FLIP performs generative planning in this setting.
Results. We add the visual distraction image in different areas of the initial image, and we can see that, the video generation model will first fail after several planning steps and generative distorted cups and make the whole image grey. However, before the model fails, the flow generation model still performs very well, which shows that the flow generation model can resist visual distractions, while the video generation model susceptible to visual distractions.
Step
Step 3
Step
Step fail
Figure 16: The apple is put the target cup, and the generative planning fails in steps (the cup distorted, by which we call the planning fails). Note, the flow model performs well before the image is distorted and turns to grey, which shows that the flow model can capture the geometry of the objects and understand the physical movement requirements of the task.
21

---

Published as a conference paper at ICLR 2025
Step
Step 3
Step 5
Step
Step.9 9
Step fail
Figure 17: The apple is put away from the target object, and the generative planning can exist more steps.
22


Reviews:
[{"review_id": "vmijIVPo4j", "reviewer": "Reviewer_H4xH", "summary": "In this work, the authors consider the task of high-dimensional generative planning for manipulation tasks. Specifically, they seek to design a prediction task which is relevant for manipulation planning and can also operate in a task-agnostic way on widely-available data. The model-based planning primitives are: an action generation module (generating 2D flow), a state transition model (generating next frame of video given flow + current observations), and a value function for determining how close a state is to the goal (based on language-conditioned visual encodings). This allows them to search for a flow-based plan, which at execution time can be executed by a learned plan-conditioned low-level policy. They evaluate the ability of their model to make coherent / correct plans on several benchmark tasks, and evaluate its utility for policy execution on the LIBERO-LONG task.", "questions": "Why doesn’t this method offer significant downstream benefits compared to ATM, which has no planning, on the selected benchmark?\n\nHow much does the action space affect downstream performance? If you were to retrain ATM with this adjusted action space, would the ranking change significantly? Do the marginal benefits of your model come from this component (unrelated to the planning contribution)?\n\n\n---\n# Post-Rebuttal Update\n\nThe authors have addressed many of my concerns adequately, and greatly strengthened their paper with the modifications to their downstream policy architecture (as well as additional details in the appendix). Bumped score to Accept.", "limitations": "", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "# Originality\n\nThe high-level approach is a reasonable combination of ideas in planning, visual representations, and generative modeling. On the flow generation component, It’s difficult to establish the level of novelty as there are a number of concurrent works proposing generative flow models for manipulation (e.g. GeneralFlow, Track2Act, etc.), but I think the combination of ideas here - for planning - is original/novel.\n\n# Quality\n\nThe quality of this paper is high. The experiments are thorough and comprehensive (with two caveats, discussed later).\n\nThe design of each component is sound, and there are lots of details / modifications that the authors conducted (and motivated!) which I found interesting and commendable. For instance, the ideas about video chunking (instead of per-frame) for value prediction, and the details about conditioning a model on flow, action space ablation, etc. are all solid contributions for other practitioners.\n\n# Clarity\n\nThe paper is quite clear - each step of the algorithm is well explained.\n\n# Significance\n\nThis paper is of moderate significance. I think the flow-based (particle-based) generative modeling approach as an action space has a lot of potential to be powerful, and clearly shows improvements for reconstruction / video generation.", "weaknesses": "The primary weakness of this paper is that, while the representation + models they built are quite interesting and useful for modeling the actual video domain they are imitating, the actual downstream utility of their method is not sufficiently characterized. Specifically, the method simply does not substantially outperform ATM, which has no planning at all and uses a similar generative representation, on the actual downstream manipulation task. Moreover, the inclusion of flow only seems to hurt the downstream policy in comparison to video conditioning. I’m not saying this method can’t show meaningful improvements over other approaches, but either the LIBERO setting chosen, or the particular low-level policy chosen, yield results that do not support the claim that this generative flow modeling + planning method provides downstream utility. Especially given the overhead. Of particular concern is the Ours-FV results, which are not well-explained.\n\nAnother weakness is that the experiments seem to be restricted to single domains, even though the method seems to have been designed to be task-agnostic - I would have liked to see the authors leverage this property more, e.g. by training predictive models on the full LIBERO benchmark and then finetuning on a subset of tasks for policy learning, or similar (even internet-scale pretraining… although I realize this is out of scope for this contribution / infeasible if there are resource constraints).\n\nAnother existential issue is that this paper is framed as a “flow generation for manipulation” paper, but the bulk of the experimentation+analysis is geared towards video prediction which has little to do with manipulation. I don’t think it stands on its own as simply a video prediction paper (at least on these tasks alone) - and I would have liked to see a larger emphasis on the actual analysis for manipulation tasks (e.g. geometric precision is a visual prediction problem, or feasibility, or other downstream metrics).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "iwrzLC0CtA", "reviewer": "Reviewer_9xux", "summary": "This paper proposes a new model-based planning framework called FLIP. It consists of 3 components: 1) a flow generation network that generates action flows given the current image observation and a language instruction 2) a video generation (dynamics model) that conditions on the flow and generate future video frames 3) a value function that evaluates the task progress (reward) given the image observation and language description. The key is the introduction of using flow as an action representation and condition the video generation model on the flow. The whole system can be used for model-based planning to generate video plans given a task, and the flow itself can also be used for guiding low-level policy execution. Various experiments are performed in both simulation environments and real-world videos.", "questions": "Please see the weakness section", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- This paper is overall clearly written.\n- The experiments cover a range of test settings, and the ablation studies help understand each component of the method.\n- Overall the experiment results are good, which demonstrates the effectiveness of the proposed method.", "weaknesses": "- I feel the paper could have compared to some stronger baselines, for some of the experiments. E.g., for experiments in section 5.1 and section 5.2, a stronger baseline than UniPi could be UniSim [1]. The reviewer understands that the code may not be open-sourced, in this case, at least some discussion to the paper should be included. There is another very recent work DIAMOND [2] that can do very long-horizon and detailed video prediction into the future conditioned on actions. Both of these paper use just a diffusion model without the need to first extract action flows, which seems to contradict to the key proposal of this paper, which is to make the model flow-centric. Some discussion on this would be appreciated.\n- For the flow generation model -- why using a C-VAE instead of a diffusion model? Some discussion on this design choice would be appreciated. \n- For all experiments, please specify the quality of the video demonstrations, e.g., are they collected using a random policy, or are they expert videos? This would be important to understand, e.g., in section 5.1., if the system can learn from sub-optimal data for planning or does it need optimal demonstrations. \n\n[1]  LEARNING INTERACTIVE REAL-WORLD SIMULATORS, Yang et al, ICLR 2024\n[2] DIAMOND💎Diffusion for World Modeling: Visual Details Matter in Atari, Alonso et al, NeurIPS 2024", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "zJA8DLTBEt", "reviewer": "Reviewer_R3ys", "summary": "This paper proposes FLIP, a model-based planning algorithm on visual space.\nThe algorithm consists of three main modules 1) a flow generation model as an action proposal module, 2) a flow-conditioned video generation model as a dynamic module, and 3) a value module.\nThe flow generation model generates flows based on the observation and a language instruction.\nThe video generation model generates videos with a Diffusion Transformer based the predicted flows.\nThe value module, built on a vision-language representation model, is used to assign values for each frame in a video to enable model-based planning in the image space.\nModel-based planning algorithm leverages the three modules to search for a sequence of flow actions and video plans that maximizes the discounted return.\nExperiments were performed to evaluate the capability of the proposed method on generating video plans, generating long-horizon videos, and accomplishing manipulation tasks.\nThe proposed method outperforms comparing baseline methods and showcases interactive properties, zero-shot transfer capability, and scalability.", "questions": "1. In Sec. 5.1, the paper compares with FLIP-NC, an ablation of the proposed method FLIP which has no value module as guidance. Is it possible to provide more details on how FLIP-NC performs beam search without a value guidance?\n\n2. Is it possible to provide a more detailed description on the typical failure modes of FLIP in policy learning (Sec. 5.3) ?\n\n3. In the Dynamics Module Experiments in Sec. 5.4, the paper compares with LVDM and IRASim on short-horizon video generation. The proposed method is provided with ground-truth flows to generate videos. What information is provided for the two comparing baseline methods?\n\n4. Are the flow generation model and the video generation model trained individually or jointly?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The paper is well-written and clear. \nLeveraging CVAE to address the multi-modality of flows is well-motivated.\nAdditionally, the paper proposes to use a mixed-conditioning mechanism for multi-modal conditional video generation.\nFor the value module, the paper modifies the original LIV method and uses video clips instead of video frames as a unit to account for the noisy value prediction.\nExperiments show that the proposed method surpasses comparing baseline methods in generating video plans, long-horizon videos, and performing manipulation tasks.\nThe paper also showcases interesting interactive properties, zero-shot transfer ability, and scalability of the proposed method.", "weaknesses": "1. There are no real-robot experiments to validate the effectiveness of the proposed method in policy learning in the real world. Incorporating such experiments and comparing with recent policy learning methods (e.g. ATM, OpenVLA [1] or Octo [2]) would provide a more comprehensive understanding of FLIP's performance in real-world policy learning.\n\n2. The paper only evaluates the policy learning capability on a suite of LIBERO, i.e. LIBERO-Long. Conducting an evaluation to assess the generalization capability of the proposed method, e.g. on the LIBERO-Object suite, would be beneficial. Also, including a comparison with recent policy learning methods, like OpenVLA [1] or Octo [2], would further strengthen the paper.\n\n2. In Sec. 5.2, the paper compares with IRASim on a text-to-video task. However, IRASim generates video based on a trajectory instead of a text. For the text-to-video task, it would be better to compare with a text-to-video method (e.g. [3]).\n\n[1] Kim, Moo Jin, et al. \"OpenVLA: An Open-Source Vision-Language-Action Model.\" arXiv preprint arXiv:2406.09246 (2024).\n\n[2] Team, Octo Model, et al. \"Octo: An open-source generalist robot policy.\" arXiv preprint arXiv:2405.12213 (2024).\n\n[3] Ma, Xin, et al. \"Latte: Latent diffusion transformer for video generation.\" arXiv preprint arXiv:2401.03048 (2024).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "DJIKNKWRSx", "reviewer": "Reviewer_6B9z", "summary": "The paper introduces a model-based planning framework (FLIP) designed for general-purpose robotic manipulation tasks using language and vision inputs. The framework allows for the progressive synthesis of long-horizon action plans, starting from an initial image and language instruction. FLIP effectively uses image flows to represent complex movements, enhancing the planning process for manipulation tasks across various objects and robots. They propose a multi-modal flow generation model predicting actions by generating dynamic representations of movements, simulating future video sequences based on the proposed actions, and evaluate the generated videos to maximize the task's success probability.", "questions": "- How does FLIP compare to other state-of-the-art planning frameworks in terms of efficiency and effectiveness?\n- What are the limitations of using image flow as an action representation? In which scenarios/tasks, this would not be ideal? \n- How does FLIP handle ambiguities in language instructions?\n- Are there any results on the real robot that show the applicability on world models even for quasi-static tasks? \n- How will the model's performance be affected in the presence of noise or visual obstructions? \n- How does the performance of FLIP depend upon or scale with the size of the available data, numerical analysis for the same would be helpful to understand the efficiency of the proposed approach.\n\nPlease also address other comments in the weaknesses section above.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The paper is well-written and methodically presented. \n- FLIP introduces a novel approach to model-based planning by integrating multi-modal inputs, which enhances its versatility for various manipulation tasks\n- FLIP is designed to scale with increasing model and data budgets, making it suitable for a variety of applications and capable of leveraging more extensive datasets as they become available.\n- The generated plans can be used to inform and train low-level control policies which can support several hierarchical policies that require strategic decision-making and planning.", "weaknesses": "- The framework may struggle with ambiguities in language instructions or unexpected changes in the environment. \n- Depending on the computational demands of the multi-modal modules, real-time performance in dynamic environments could be a concern. Evaluating the speed and efficiency of planning under real-time constraints would be crucial.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)