EVLP: Learning Unified Embodied Vision-Language Planner with Reinforced Supervised Fine-Tuning
Authors: Xinyan Cai
Qiang Guan
Shiguang Wu
DaFeng Chi
Yuzheng Zhuang
Xingyue Quan
Jianye HAO
Venue: ICLR 2026 Poster (2026)
Keywords: Embodied intelligence
multimodal large language models
reinforcement learning
long-sequence planning
Primary area: applications to robotics, autonomy, planning
TL;DR: An unified multimodal planning framework for complex long-horizon tasks with dynamic learning and reinforced alignment

Abstract: In complex embodied long-horizon manipulation tasks, effective task decomposition and execution require synergistic integration of textual logical reasoning and visual-spatial imagination to ensure efficient and accurate operation. Current methods fail to adopt a unified generation framework for multimodal planning, leading to inconsistencies in multimodal planning. To address this challenge, we present EVLP (Embodied Vision-Language Planner), an innovative multimodal unified generation framework that jointly models linguistic reasoning and visual generation. Our approach achieves multimodal planning for long-horizon tasks through a novel training pipeline incorporating dynamic pretraining and reinforced alignment. Our core innovations consist of three key components: 1. Unified Multimodal Generation Framework: For understanding, we integrate semantic information with spatial features to provide comprehensive visual perception. For generation, we directly learn the joint distribution of discrete images for one-step visual synthesis, enabling coordinated language-visual modeling through learnable cross-modal attention mechanisms. 2. Dynamic Perception Pretraining: We propose a bidirectional dynamic alignment strategy employing inverse dynamics tasks and forward dynamics tasks, effectively strengthening multimodal correlations within a unified feature space. 3. Reinforced Supervised Fine-Tuning: While conducting instruction-based fine-tuning in the unified generation space, we construct a reinforce loss to align the spatial logic between textual actions and generated images, enabling the model to acquire spatio-aware multimodal planning capabilities.Comprehensive evaluations on multiple complex tasks demonstrate that EVLP significantly outperforms competitive baselines in both instruction execution accuracy and task success rate, benefiting from its unified multimodal architecture and well-designed training pipeline. Extensive ablation studies further validate the rationality of our framework design.

Published as a conference paper at ICLR 2026
EVLP: LEARNING UNIFIED EMBODIED LANGUAGE PLANNER WITH REINFORCED SUPERVISED
VISION-
Xinyan Cai
Qiang Guan Shiguang Wu
Dafeng Chi
Yuzheng Zhuang
Jianye Hao Xingyue Quan
Institute of Automation, Chinese Academy of Sciences (CASIA)
School of Artificial Intelligence, University of Chinese Academy of Sciences
Huawei Noah's 's Ark Lab
ABSTRACT
In complex embodied long-horizon manipulation tasks, effective task decompo- sition and execution require synergistic integration of textual logical reasoning and visual-spatial imagination to ensure efficient and accurate operation. Current methods fail to adopt a unified generation framework for multimodal planning, lead to inconsistent in multimodal planning. To address this challenge, we present Planner), an innovative multimodal unified
EVLP (Embodied Vision-Languee generation framework that jointly models linguistic reasoning and visual genera- tion. Our approach achieves multimodal planning for long-horizon tasks through a novel training pipeline incorporating dynamic pretraining and reinforced alignment. Our core innovations consist of three key components: 1) Unified Multimodal Generation Framework: For understanding, We integrate semantic information with spatial features to provide comprehensive visual perception. For genera- tion, directly learn the joint distribution of discrete images for one-step visual synthesis, enabling coordinated language-visual modeling through learnable cross- modal attention mechanisms. 2) Dynamic Perception Pretraining: We propose bidirectional dynamic alignment strategy employing inverse dynamics tasks and forward dynamics tasks, effectively strengthening multimodal correlations within a unified feature space. 3) Reinforced Supervised Fine- Tuning: While conduct- ing instruction-based fine-tuning in the unified generation space, we construct a reinforce loss to align the spatial logic between textual actions and generated images, enabling the model to acquire spatio-av spatio-awared awared multimodal planning capa- bilities. Comprehensive evaluations on multiple complex tasks demonstrate that EVLP significaayy outperforms competitive baselines in both instruction execution accuracy and task success rate, benefiting from its unified multimodal architecture and well-designed training pipeline. Extensive ablation studies further validate the rationality of our framework design.
INTRODUCTION
In the realm of long-term planning for embodied intelligence, current task decomposition methodolo- gies predominantly adopt two distinet technical paradigms: language planning parses high-level
instructions (e.g.. "tidy up the room"") into sequential atomic actions (e.g.. "pick up clothes -> place
them in the closet"" to explicitly specify what to do (What) (Ahn al. 2022 Mu al. 2023a
Guo et al. 2024: Zhang et al. 2025b Zhao et al. 2025), while visual planning resolves how to accomplish (How) through generating intermediate visual representations (c.g.. depicting "clothes inside the closet") (Black et al. 2023b Du et al. 2023 Soni et al.) (2025). In contrast, emerging multi- modal planning frameworks (Ni al. (024b) synergistically produce linguistic action sequences and corresponding visual targets, thereby bridging the critical gap between procedural execution and goal
*[k] Corresponding author: gjang-guan gia.auinannccccm
T Corresponding author: wushiguang@ huawei.com

---

Published as a conference paper at ICLR 2026
achievement. This represetttennnnnnn approach demonstrates significant potential for overcoming existing planning limitations in embodied systems (Black et al. 2023b] [Ni et al. 2024b), positioning
them as a pivotal research direction for cext-generation intelligent agents.
Recent advances unified multimodal generative models (Ge et al. 2023b a
Team 2025) have
demonstrated unprecedented cross-modal generation capabilities. differing from traditional multi- modal understanding models (Liu et al. 2023a b 2024 Li al., (022) or architectures that generate through external diffusion (Ge et al.) 2023c). These architectures unify text and image generation within a single Transformer backbone, showcasing unified understanding and generation capabilities
across different modalities (Ma et al.) [025|] Zhou al., 2024] Shi et al., 2025] Wu et al. 2025b).
Their inherent scalability (Wu ct al., 2024] Chen et al. (0255 and emergent cross cross-modal synergy
(Xie et al., 2024) enable mutual reinforcement between text-image comprehension and synthesis tasks. This naturally raises an important research question: Can such unified multimodal architectures be leveraged to construct Vision-Languag planners capable of resolving robotic manipulation tasks that require complex multi-step instruction decomposition?
Developing a unified multimodal planner presents three key technical hurdles. First, while existing multimodal models excel at matching images with text descriptions (e.g., generating a cat image from "a cat on a sofa"), ) embodied planning requires deeper understanding. The model must not only recognize objects in images (e.g.. "cup on table") but also precisely grasp where objects located in ppaciccritical for planning physical actions like grasping or moving. Second, traditional multimodal tasks (e.g., image captioning visual QA) focus static understanding. Embodied planning adds a temporal dimension: successful models must track how actions transform the environment. Imagine predicting how "pouring water" changes scene from (I) cup upright to (2) cup tilted-this demands reasoning about state transitions rather than just recognizing static patterns. Third, conventional maximum likelihood training treats all visual details equally. While this works for generating photorealistic images, operational tasks prioritize functional consistency over visual perfection. For example, when planning "put clothes in closet", we care more about the clothes' final position (in closet) than their exact folds or shading. Current training paradigms lack mechanisms to prioritize such task-relevant aspects over irrelevant visual details.
To address this challenge, we present EVLP (Embodied Vision-Language Planner). novel framework that seamlessly unifies language reasoning and visual imagination within a single multi- modal architecture for long-term manipulation tasks. EVLP innovatively integrates: 1) A dual-tower vision module that decouples understanding and generation employingg SigLIP's s semantic encoder enhanced by trainable detail compensators mitigate systematic visual blind spots, paired with discrete tokenization via MAGVIT2 (Yu et al.] 2024) for direct image synthesis; 2) A bidirectional dynamics-awan pre-training paradigm that equips the model with coherent ceasoning-imagine capabilities by learning from forward/inverse dynamic prediction tasks; 3) Reinforced Supervised Vine-Tuning (RSFT), which uses maximum likelihood to supervise the entire token distribution while dynamically reinforcing spatial consistency between language actions and generated images through policy gradients. Compared to existing methods, EVLP demonstrates significant improvements in instruction following and task success rates in complex manipulation tasks, establishing a promising and inspiring paradigm for long-term operations. We also conducted extensive ablation experiments to provide references for future research.
METHOD
EVLP leverages pre-trained LLMs through a unified transformer architecture for multimodal genera- tion, processing both stepwise linguistic instructions and visual subgoal images, which providing linguistic milestones and perceptual anchors to guide action sequences in long-horizon tasks. Our technical exposition proceeds systematically: Section 2. details the multimodal architecture enabling unified generation; Section 2.2 examines dynamic perception pre-training for learning environmental dynamics; and Section 2.3 introduces reinforced supervised fine-tuning phase, which enabling the model to acquire spatio-award multimodal planning capabilities. Our overall framwork is illustrated
in Figure]] T
2

---

Published as a conference paper at ICLR 2026
Dynamic Perception Pretraining
Language Labels
Language
Image Labels
Image Loghts
<action>: place the notebook on the center of the table
LLM
LLM
prppppp0o. Tokenzter
Adoptor
Adoptor
Tokenzier
Adaptor Encader
Image Tokens
Siglp
Encoder
Siglip
Encoder
<action>: place the
<action>> notebook
table
Reinforced Supervised Fine-
Language
Language Logits
Image Logits
Labels
<action>:
place the notebook on the center of the table
Less
CE
LLM
Leggrobs
Reinforce
Tokenaler
Adoptor Encoder
Takenzler
Image Takens
Sampled Images
Siglip
Advantage
<action>:
up meeting
place
Reward Pn
supplies
Figure 1: Our overall framework diagram. In terms of the model architecture, we adopt vision tower design that integrates understanding and generation. For image understanding, combine SigLIP with a learnable spatial encoder, while for image generation, we introduce image tokens to achieve one-step generation. Regarding the training pipeline, we design two-stage framework: dynamic perception pretraining (illustrated above) and reinforced supervised fine-tuning (illustrated below). The black arrows represent the forward process, while the red arrows indicate the backward process.
2.1
UNIFIED MODEL FOR MULTIMODAL GENERATION
Modern unified multimodal generative models predominantly adopt two architectural paradigms:
diffusion- and autoregressive approaches. Diffusion architectures (Ho et al. 2020 Peebles &
Xie 2023 Xie et al., 2024) aim adapt large language models (LLMs) for image denoising through
mulli-step iterative refin refinement (Zhou et al. 2024), while autoregressive methods (Liu et al. 2025
Team 2025 Qu al. 2024 Yu et al. 2024
Wu et 2025a) employ unified tokenization strategies
For both modalities under single learning objective
(uu et al. 2024 20256).
We identify critical limitations in both frameworks: the diffusion objective exhibits fundamental discrepancy with LLMs' pre-training tasks (Wu et al. 2025a), creating optimization conflicts during multimodal adaptation, while autoregressive modeling imposes artificial sequential dependency on inherently non-sequential visual data. Notably, when applying reinforcement learning for prefer- ence optimization (von Werra et al., 2020; Hu et al., [2024] Fan al. 2023 Black et al., (202a)), both architectures require computationally prohibitive multi-step sampling to generate independent candidates bottleneck that scales poorly with model size.
Our solution introduces a lightweight multimodal architecture that seamlessly integrates multimodal capabilities into existing LLM frameworks. The key innovation lies a cumpling-efficiees generator that produces multiple independent samples through fingl-foorarr propagation, enabling joint optimization of supervised and reinforcement learning objectives during fine-tunm tne-ttnnig as detailed in Section 2.3
3

---

Published as a conference paper at ICLR 2026
Xn
X MA000000
0
step Generation
Multi-step Generation
Ste-step Generation t
Mиممم000 MADم000 condition
MтDمоо0a condition
condition
Xam
Aam--
Image Tokens
(1) Diffusion- -baied Model
(2) Autoregressive -based Model
(3) Ours
Figure 2: (1) Diffusion-based Model formulates image generation as Z0.N p(-le, Jo.N). When sampling n samples from distribution p(-(c), the model requires n. x T forward passes, where T denotes the diffusion denoising steps. (2) Autoregressive-basee Model formulates image generation as T p(-lc, vo..). When sampling n samples from distribution p(-le), the model requires n x N forward passes, where N represents the token count. (3) Our Model directly models p(-le), enabling the sampling of n samples with only one forward pass.
To enable large language models to understand and generate images, we developed Vision Tower. For image understanding. we utilize Siglip to extract high-level semantic information from images. Additionally, to address the spatial detail information that Siglip may overlook (Tong et al., 2024b), we incorporated a low-level visual encoder pre-trained with an image reconstruction loss, focusing on the intricacies within the images. During training, we keep Siglip's s weights frozen while allowing the low-level visual encoder to participate in the process. Our goal is for both components to focus on different levels of visual signals, which are then fed into the LLM through an adapter. For image generation, to seamlessly integrate with the existing training objectives of the LLM, we trained lookup-free quantizer within the Open-MAGVITY framework (Lo et al. 2025). This quantizer maintains a codebook of size K = 262, 144 and encodes images with resolution of 256x256 into 16x16 discrete tokens.
Vision Tower
Generation Architecture To build a unified modality model, we combine the Vision Tower with pretrained large language model (LLM).. For images generation, adopt a one-step generation approach: as shown in Figure 2 We assume that images can be processed as N discrete or continuous tokens, The Diffusion-based Model and Autoregressive-basee
Model model the generation process as #0:N ~ p(-lc, 26.x) and IN p(-|c, 40m-11 respectively. To obtain the complete conditional distribution Ta.N ~ p(-lc), the model needs perform multiple autoregressive forward passes. In this paper, we take direct approach: we allow the LLM to model Fo.N ~ p(-lc) directly. Practically, we introduce a set of learnable image tokens, which are input into the transformer along with the conditions, and use the image tokens to directly predict the discrete token IDs of the image. This modeling approach avoids introducing unnecessary priors and seamlessly integrates with the original learning objectives of the LLM. By integrating the Vision Tower with the LLM, we have developed unified multimodal generation model.
f-1
2.2 DYNAMIC PERCEPTION PRETRAINING
To enable the model to learn the understanding and generation of multimodal interactions, we designed a dynamic perception pre-training phase. For embodied agents, understanding the environment is crucial. This not only requires agents to comprehend images and text individually but also to grasp the interactions between them. The model must understand the differences between two distinct states and be able to infer how taking a specific action will alter the state.
Based on these considerations, we collected dataset D TT ....., ; Ti} with transitions defined as T {It, #t, where represents image observations and a denotes language actions. Building on this foundation, we designed pre-training phase based on Inverse Dynamic Learning and Forward Dynamic Learning.
Inverse Dynamic Task To train the model's perceptual capabilities while maintaining its text generation abilities, we designed the Inverse Dynamic task. Specifically, given the dataset D and prompt P such as: "What is the action between <Img> <ImageHere> </Img> and <Img> <lmageHere> </Img>? Please infer the actions that took place", the objective function is the conditional log-likelihood of the action token sequence:
4

---

Published as a conference paper at ICLR 2026
L log P(ay a a It, x2,,
Livverse Dynamic
E (m 1)D
(I)
L
where L. represents the token length of the action sequence dge dt denotes the i-th token in the action sequence, of a(<)) refers all historical tokens before position i, and represents the model's trainable parameters. By optimizing the Inverse Dynamic Loss Linverse Dynamic. we enhance the model's ss image understanding and dynamic comprehension abilities while maintaining its text generation capabilities.
Forward Dynamic Task To enable the model learn image generation and understanding while enhancing its reasoning capabilities, we designed the Forward Dynamic task. The model needs to infer the next observation It-1 based on the current observation I₁ and the language action at- Specifically, given the dataset D and prompt P such as: "What will happen if <Img> <Imagellere> </Img> takes the action like <Actionlerre> Please generate an image of the next state", the objective function is the conditional oog-liklihood of the image token sequence:
Forward Dynamic - - log P(z(41) Zt, ay; 0)
(0:N)
(2)
where N represents the token length of the image sequence 2t, a, denotes the action sequence, and represents the model's trainable parameters. By optimizing the Forward Dynamic Loss Forward Dynamic+ enhance the model's ability for image generation and understanding while improving its dynamic reasoning capabilities.
Joint Pretraining During the pre-training phase, we employed a dual-task curriculum where inverse dynamics modeling and forward dynamics prediction were co-trained identical multimodal datasets. This co-optimization strategy establishes a unified framework for processing multimodal inputs and generating coordinated outputs, effectively forming the foundation for the model's emergent world modeling capabilities through integrated cross cross-modal modal reasoning.
2.3
REINFORCED SUPERVISED FINE-TUNING
After pre-training, the model has acquired the ability to unify the processing of visual and textual information, along with preliminary dynamic understanding capability. Next, we need to lever- age the model's perception and reasoning abilities to construct multimodal planning agent. We require the model not only to understand the objective environmental transition functions but also to make proactive inferences based on given goals, breaking down complex tasks into simpler actions. Based on these considerations, we collected a dataset D = (To. Ti...., Ti) containing high-level instructions and low-level language actions, where the basic elements are T {g. Tt. de z++1 }. with representing the high-level instruction, x denoting the image observation, and a indicating the language action. On this basis, designed Reinforced Supervised Fine-Tiie Tuning(RSFT) phase.
Maximum Likelihood for Vision-Language Planning To learn the model's perception and reasoning abilities while applying our multimodal framework for vision-language joint planning, we first perform supervised fine-tuning (SFT) on the model. Specifically, the model needs infer the appropriate language action aę based on the given high-level instruction and the current observation Tt, well as the observation Iz-1 that results from taking action a;- Given the dataset D and prompt P such as: "You are given the current image observation <Img> <lmageHere>> </Img> and the given task instruction: <TaskHeree. Please make the next step action decision and generate the next state image", the objective function is the joint conditional liglliiel of the action tokens and image tokens:
log P(a | (<5)) 2x210) + log P(x (0:N) g. xz, at OLL, ;0
CSFT = - - -(g
5

---

Published as a conference paper at ICLR 2026
where L represents the token length of the action sequence at, dj ( denotes the i-th token in the action sequence, ar (<0) indicates all historical tokens before position i, N is the token length of the image sequence 2t, and are the model's is trainable parameters. By optimizing the joint conditional log-likelihood LSFT, the model learns multimodal reasoning capabilities.
Policy Gradient for Dynamic Aligument The super- vised objective Lser minimizes cross-entropy between the model's conditional joint distribution pe (Zo.N (c) and target data distribution. While effective for static pat- tern alignment, maximum likelihood estimation suffers from two critical limitations in dynamic reasoning tasks: (I) perceptual pver-speciffcation enforcing unnecessary constraints on task-irrelevant details (c.g., table texture consistency), and (2) causal under-constroun model physical dynamics that govern state transitions. Re- inforcement learning (Sutton & Barto 1998) addresses this mismatch through reward-weighted policy gradients, where human preferences guide the learning of physically plausible trajectories. Our key innovation seamlessly in- tegrates this paradigm by exploiting the model's 'ss single- forward multi-sample generation capability to compute seaantgge-wiightee gradients without additional compu- tational overhead.
I can measure the distribution, but I can't assess preferences.
direct gradient
failing to
I con't measure the distribution, but can assess preferences.
policy gradient
(2)
I can both measure the distribution and assess preferences.
direct gradient
Since our model directly models V0:N ~> p(-le), we can in- dependently sample multiple samples in a single forward pass. Specifically, we designed a Dynamic Alignment reward function R(x), used to measure whether the dynamics of the generated image are consistent with the real dynamies, the specific definition of the reward func- tion can be found in the Appendix [..]] During the forward pass, sample K samples x² P(z (0:N) g, zt, at a.=L) and obtain the reward for each sample using the reward function. We normalize these rewards within each batch to calculate the advantage for each sample and optimize with the equation
policy gradient
(3)
Figure 3: (1) SFT optimize model by minimizing KL divergence between model outputs and dataset distributions. It lacks per-sample preference align- ment. (2) Reinforcement Learning (RL) aligns preferences through per- sample feedback, focusing on reward maximization but risking distribution shifts. (3) Reinforcement Supervised Fine- Tuning (RSFT) combines distribu- tion constraints with sample optimiza- tion, enforcing preference alignment der maximum likelihood constraints.
By explicitly encoding the dynamic alignment prior through the reward function R(-), strengthen the model's dynamic alignment capabilities. However, due the use of high-variance gradient estimators, reinforce- ment learning exhibits instability during training, which can lead to training collapse. Therefore, we combine max- imum likelihood with policy gradients to achieve stable joint optimization.
K
CRL= = -E Bt
A log P( g,
2t, at O.L, 0)
(4)
K
Joint optimization In practice, we constrain the model's 's overall distribution using maximum likeli- hood and leverage policy gradient to enhance dynamic alignment capabilities (Figure B). Specifically, we optimize our model with the following objective:
C E ( [SSFT + X LRu].
The first term enforces global alignment between language and vision, while the second improves dynamic consistency via preference-awar sampling. Together, they enable physically plausible action

---

Published as a conference paper at ICLR 2026
Table
The evaluation success rate between baselines we report the mean and variance across 5 seeds.
Blocks
Letters
Model
Stacking
Sort
Matching
Shape
Orders
Spell
CLIPort
18.4 3.2
19.2 4.6
17.8 + 2.9
9.8 + 1.4
8.1 1.1:2 2.7
2.3 0.8
PAR
34.7 3.5
32.8 + 6.3
31.1 + 4.4
31.5 + 5.8
30.7 + 4.9
27.3 + 7.2
EmbodicdGPT
48.6 6.7
49.1 5.9
43.4 7.8
40.9 = 6.4
48.2 7.5
52.7 = 6.2
SuSIE
34.1
32.6 4.1
33.2 + 5.7
37.8 6.6
35.2 4.3
34.1 + 7.4
CoTDiffusion
47.9 6.0
4.6
56.6 5.2
46.1
5..9 4.8
44.8 + 7.9
PERIA
0..9
65.0 1.00:.4. 6.4
7.13
60.6 5.2
6..2
71.1 + 7.5
EVLP (ours)
79.4 + 7.9
77.3 +
75.3 8.1
4.4
78.2 7.3
81.8 6.5
sequences and subgoal-condittoned images from high-level instructions, supporting autonomous planning under real dynamics. Implementation details and pseudocode are provided in Appendix C.3
EXPERIMENTS
3.1 EXPERIMENT SETUP
Benchmark & Settings We evaluate across long-horizon manipulation environments, configuring the model to generate language-guiie guided actions and visual subgoals from high-level instructions, with low-level policy executing the plans (implementation details in Appendix G). Experiments use LoHoRavens (Zhang et af.] 2023). a Ravens based benchmark where we consider 11 language- conditioned block tasks and extend with 9 challenging letter tasks, and Meeting Preparation, an in-house simulator for arranging desktop objects under varied backgrounds, table types, and object categories, enabling deeper analysis of planning in realistic, diverse settings.
Baselines We compare against four planning paradigms: (i) end-to-end imitation learning with CLI- Port (Shridhar al. 2022), which directly maps high-level instructions to actions without an explicit planner, (ii) language planning, including PAR (Zhang et al. 2023), which uses VLM reporter and an LLM planner, and EmbodiedGPT Muu et al. (023b), which replaces the LLM+ VLM stack with stronger MILM after instruction tuning; (iii) vision planning, represented by SuSIE (Blackl et al. 2023b), which edits images to form subgoals for simple steps, and CoTDiffusion (Ni et al. 2024a), which introduces semantic alignment module for chain-of-thought subgoal generation; and (iv) multimodal planning with PERIA (Ni et al.) (2024b), which jointly plans language actions and image conditions with LLM while diffusion model renders visual subgoals.
3.2 MAIN QUANTITATIVE RESULTS OF SUCCESS RATE
The results on LoHoRavens are shown in Table As expected, end-to-end learning methods performed the worst, struggling to complete tasks due to a lack of intermediate guidance. In contrast, the language planning paradigm explicitly decomposes tasks into stepwise instructions and employs hierarchical framework consisting of language planner and a language- language-conditioned conditionnd policy, demonstrating greater potential and clearly outperforning the end-to-end approach. Meanwhile, EmbodiedGPT achieved stronger performance through fine-tuning.
The visual planning paradigm generates intermediate keyframes. For example, SuSTE uses an image editing model to directly generate subgoals, while CoTDiffusion produces more refined images through Chain Thought (CoT) reasoning, enhancing the performance of visual planning. However, CofDiffusion does not explicitly reason about the instructions, which may lead to semantic inconsistencies in the generated subgoal images. In comparison, PERIA further enhances performance through multimodal planning. This method provides rich subgoal information by jointly planning with language and vision. However, PERIA is limited by its visual perception capabilities and the interaction between modalities, which may result in inaccuracies in the generated target images.

---

Published as a conference paper at ICLR 2026
place the plate Ruioolle
place the drink con the right of the plate
place the name plate the of the notebook
place the cup soucer on the right of the rame piote
piace the oo the night of drink
Action
space
GT
RL
w/o RL
Figure 4: Comparison of generation effects between RSFT and SFT shows that RSFT generates more finely detailed results with better dynamic consistency.
In contrast, EVLP algorithm integrates the processing of textual instructions and image observa- tions, introducing richer visual perception and designing framework for unified interaction between text and images. Additionally, model's 's dynamic reasoning capabilities through reinforcement learning. Our approach surpasses the baselines across all tasks, achieving optimal performance.
developed dynamic alignment reward function to optimize the
3.3 FURTHER ANALYSIS
This section trained multiple variants of EVLP and conducted two sets of experiments on the Meeting Preparation task: first, given instructions and the observation, tested the model's planning capabilities by measuring the success rate (SR), language accuracy (LA), and LPIPS, SSIM on test set(Table We also tested the image generation capabilities by providing the observation along with actions, asking the model to generate the next state, and evaluated the LPIPS and SSIM(Table
Ablation on Vision Tower To validate the ef- fectiveness of our vision tower, we created two variants: one removing the spatial encoder from vision tower (EVLP w/o En) and the other excluding Siglip from the architecture (EVLP w/o Se). We first compared their generation per- formance (Exps. A, B, and C in Table B It evident that the image generation capability of EVLP w/o En significantly declines, rely- E
Table Comparative analysis task planning perfor- mance across different variants.
Method
SR 1
LA
LPIPS J
SSIM
EVLP
67.6
0.051
w/o En
56.5
82.9
0.092
0.92
w/o Se
50.1
73.9
0.116
0.89
w/o IDM
63.9
83.6
0.052
0.95
w/o FDM
26.8
72.1
0.192
0.84
ing solely on Siglip information lacks crucial details regarding positional context. Mcanwhile, although EVLP w/o Se shows less degradation in image generation tasks, it becomes clear in specific multimodal reasoning tasks (Exps. A, B. and C in Table that the absence Siglip significantly reduces the model's language planning abilities, leading to suboptimal overall performance. This is due to the spatial encoder alone lacking essential semantic information, making challenging for the LLM to process. In contrast, the combination of both components achieves the best results.
w/o RL.
62.2
87.4
0.054
0.95
RL only
0.0
14.0
0.712
0.29
Ablation on Generation Architecture To test if one-step generation(Fig. 2 improves quality, we build EVLP-AR, which keeps the same components but generate autoregressively. fol- lowing (Wu et al. 2024). Comparing EVLP with EVLP-AR (Exps. A.D in Tab. 3 shows the AR variant markedly de- grades fidelity and increases hallucinations. We believe this is primarily due to two reasons: (i) imposing unnatural causal- sequence prior on images and (ii) cumulative AR errors that hinder precise operations.
Table 3: Comparative analysis of age generation performance across dif- ferent variants.
Method
LPIPS
SSIM 1
A
EVLP
0.046
0.95
w/n En
0.087
0.87
C
wfo Se
0.074
0.91
D
AR
0.197
0.84
8

---

Published as conference paper at ICLR 2026
.Instruction>: move a bowl to the right side spatula
Instruction>= placing stuffed animal into pan
c Instruction>: move fork between towel pot
<Action>: pick up the bowl
<Action>: grab the stuffed animal
Actioon:: lift the fork
Observation
Subgoal Image
Observation
Subgoal Image
Observation
Subgoal Image
Figure 5: Visualization of World Dataset experiments, showcasing EVLP's planning quality in complex, real-world scenes.
further disentangle the contributions of two pretraining objec- LivesIDDM for predicting action commands and FDM for predicting state transitions. Using the same architecture, we compare three settings: joint dual-task pretraining (IDM+FDM), w/o FDM and w/o IDM (Exps. A.D.E in Tab. 2). Results show that the joint model performs best overall, underscoring the necessity of combined supervision. Ablations indicate that removing IDM weakens language planning, while removing FDM markedly degrades multimodal planning. Together, these findings highlight the pivotal role of bidirectional pretraining in modality alignment.
Ablation on Pretraining
Ablation on RSFT In Section 2.3 we introduced the Reinforce- Supervised Fine-Tuning (RSFT) framework, which optimizes both likelihood and dynamic rewards. Comparative analysis (Exps. A, F, and G in Table [ of RSFT (learning by Equation 5). traditional supervised fine-tuning (SFT) (learning by Equation B). and RL-only (learning by Equation shows that RSFT performs similarly SFT on standard image metrics but significantly improves task success rates. Notably, the RL-only baseline exhibits catastrophic policy collapse without SFT regularization, underscoring the necessity of dynamic consistency constraints for stable learning - key advan- tage preserved by our hybrid approach. Reward curves on the test set (Figure (6) further confirm our algorithm's effectiveness in reward accumulation compared to traditional SFT methods. The actual gen- erated results (Figure 4 demonstrating that RSFT perform better dynamic consistency.
0.18
Henved
0.15
0.14
2000
Figure 6: The reward curve on test set during training.
World Dataset Evaluation We constructed real- world robotic manipulation dataset based on the BaidgeData v2. This dataset features diverse household tasks (e.g., object grasping, stacking) with recorded robotic arm observations and natural language instructions. We evaluated EVLP's multi- modal planning capabilities using LA and LPIPS. As summa- rized in Table EVLP achieves the best performance across both modalities. In language planning, EVLP obtains a LA of 0.78, surpassing EmbodiedGPT and PERIA, indicating stronger alignment with task intent. In visual planning, EVLP reduces LPIPS to 0. compared with 0.23 for SuSIE and 07 for PERIA, demonstrating more spatially consistent visual subgoals.
Table 4: Comparative analysis in Real- World Dataset.
Method
LAT
LPIPS
A
EVLP
0.78
0.11
B
PERTA
0.75
0.17
C
SuSIE
0.23
D
EmbuliedGPT
0.68
4 CONCLUSION
We introduced EVLP (Embodied Vision-Language Planner), unified multimodal generation framework that integrates language reasoning and visual imagination for long- hong-horizon manipulation tasks. Through dynamic perception pre-training, EVLP learns spatial relations and environmen- tal dynamics, while our reinforced supervised fine-tuning (RSFT) enables dynamically consistent multimodal planning. Extensive experiments challenging benchmarks show that EVLP achieves

---

Published as a conference paper at ICLR 2026
superior custructiio-following aceuracy and task success rates compared to strong baselines. Ad- ditionally, we conducted detailed analyses to validate the effectiveness of our design. We believe that EVLP highlights the potential of holistic language and vision planning, and we hope this novel paradigm can provide valuable insights for robotics research in long-horizon tasks with complex instructions, moving toward more open embodied scenarios.
REFERENCES
Michael Ahn, Anthony Brohan, Noah Brown, Yevgen Chebotar, Omar Cortes, Byron David, Chelsea Finn, Chuyuan Fu, Keerthana Gopalakrishnan, Karol Hausman, et al. Do can, not i say: Grounding language in robotic affordances. arXiv preprint arXiv:2204.00 01691, 2022.
Anas Awadalla, Irena Gao, Josh Gardner, Jack Hessel, Yusuf Hanafy, Wanrong Zhu, Kalyani Marathe, Yonatan Bitton, Samir Gadre, Shiori Sagawa, et al. Openflamingo: An open-source framework for training large autoregressive vision-language models. arXiv preprint arXiv:2308.01 2023.
Yuntao Bai, Andy Jones, Kamal Ndousse, Amanda Askell, Anna Chen, Nova DasSarma, Dawn Drain, Stanislav Fort, Deep Ganguli, Tom Henighan, Nicholas Joseph, Saurav Kadavath, Jackson Kernion, Tom Conerly, Sheer El-Showk, Nelson Elhage, Zac Hatfield-Dodds, Danny Hernandez, Tristan Hume, Scott Johnston, Shauna Kravec, Liane Lovitt, Neel Nanda, Catherine Olsson, Dario Amodei, Tom Brown, Jack Clark, Sam McCandlish, Chris Olah, Ben Mann, and Jared Kaplan. Training helpful and harmless assistant with reinforcement learning from human feedback, 2022. URL
wwwpxiiv.oorroooo abs/ 2204. 05862
Kevin Black, Michael Janner, Yilun Du, Ilya Kostrikov, and Sergey Levine Training diffusion models with reinforcement learning. arXiv preprint arXiv:2305. 13301, 2023a.
Kevin Black, Mitsuhiko Nakamoto, Pranav Atreya, Homer Walke, Chelsea Finn, Aviral Kumar, and Sergey Levine. Zero-shot robotic manipulation with pretrained image-editing diffusion models. arXiv preprint arXivv2310. 10639, 2023b.
Tim Brooks, Aleksander Holynski, and Alexci A Efros. Instructpix2pix: Learning to follow image editing instructions. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 18392-18402, 2023.
Xiaokang Chen, Zhiyu Wu, Xingchao Liu, Zizheng Pan, Wen Liu, Zhenda Xie, Xingkai Yu, and Chong Ruan. Janus-pro: Unified multimodal understanding and generation with data and model scaling, 2025. URL URLL:ttes://arxvv..o////b/a
Xiangxiang Chu, Hailang Huang, Xiao Zhang, Fei Wei, and Yong Wang. Gpg: A simple and strong reinforcement learning baseline for model reasoning, 2025. URL 2504. 02546
Yilun Du, Mengjiao Yang, Pete Florence, Fei Xia, Ayzaan Wahid, Brian Ichter, Pierre Sermanet, Tianhe Yu, Pieter Abbeel, Joshua Tenenbaum, Leslie Kaelbling, Andy Zeng, and Jonathan Tompson. Video language planning, 2023,
Ying Fan, Olivia Watkins, Yuqing Du Hao Liu, Moonkyung Ryu, Craig Boutilier, Picter Abbeel, Mohammad Ghavamzadeh, Kangwook Lee, and Kimin Lee. Dpok: Reinforcement learning for fine-tuning text-to-image diffusion models. Advances in Neural Information Processing Systems, 777799888778888 2023.
Yuying Ge Yixiao Ge, Ziyun Zeng, Xintao Wang, and Ying Shan. Planting seed of vision in large language model. arXiv preprint arXiv:2307.08041 2023a.
Yuying Ge, Sijie Zhao, Ziyun Zeng. Yixiao Ge, Chen Li, Xintao Wang, and Ying Shan. Making Ilama see and draw with seed tokenizer. arXiv preprint arXiv:2310.0/2//8, 2023b.
Yuying Ge. Sijie Zhao, Ziyun Zeng, Yixiao Ge, Chen Li, Xintao Wang, and Ying Shan. Making llama see and draw with seed tokenizer, 2023c. URL htttss://aaattt wttp://wwwww...p///////./... 00221
10

---

Published as a conference paper at ICLR 2026
Qiushan Guo, Shalini De Mello, Hongxu Yin, Wonmin Byeon, Ka Chun Cheung, Yizhou Yu, Ping Luo, and Sifei Liu. Regiongpt: Towards region understanding vision language model. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 13796-13806 13806, 2024.
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models, 2020. URL https: / arxiv. wwgp/r/////ssssso/oo 11239
Jian Hu, Xibin Wu, Zilin Zhu, Xianyu, Weixun Wang, Dehao Zhang, and Yu Cao. Openrlhf: An easy-to-use, scalable and high-performance rlhf framework. arXiv preprint arXiv:2405.11143 2024.
Runhui Huang, Chunwei Wang, Junwei Yang. Guansong Lu, Yunlong Yuan, Jianhua Han, Lu Hou, Wei Zhang, Lanqing Hong, Hengshuang Zhao, and Hang Xu. Illume+: Illuminating unified mllm with dual visual tokenization and diffusion refinement, 2025. URL
abs/ / 2504 01934
Yunfan Jiang, Agrim Gupta, Zichen Zhang, Guanzhi Wang. Yongqiang Dou, Yanjun Chen, Li Fei-Fei, Anima Anandkumar, Yuke Zhu, and Linxi Fan. Vima: General robot manipulation with multimodal prompts. In NeurlPS 2022 Foundation Models for Decision Making Workshop, 2022.
Junnan Li, Dongxu Li, Caiming Xiong, and Steven Hoi. Blip: Bootstrapping language-image
pre-training for unified vision-language understanding and generation. In ICMI., 2022.
Hao Liu, Wilson Yan, Matei Zaharia, and Pieter Abbeel. World model on million-length video and language with blockwise ringattention, 2025. URL httpps://arxxvv.rr///bbb
08268
Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee. Improved baselines with visual instruction tuning, 2023a.
Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning, 2023b.
Haotian Liu, Chunyuan Li, Yuheng Li, Bo Li, Yuanhan Zhang, Sheng Shen, and Yong Jae Lee. Llava-next: Improved reasoning, ocr, and world knowledge, January 2024. URL https::/
llavavvl.gi thub.. to/bllg//022-01-33
next//
Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. arXiv preprint arXiv:171..05001, 2017.
Zhuoyan Luo, Fengyuan Shi, Yixiao Ge. Yujiu Yang, Limin Wang, and Ying Shan. Open-magvit2: An open-source project toward democratizing ruto-regressive visual generation, 2025. URL
https::/ org/ abs 2409 04410
Yiyang Ma, Xingchao Liu, Xiaokang Chen, Wen Liu, Chengyue Wu, Zhiyu Wu, Zizheng Pan, Zhenda Xie, Haowei Zhang, Xingkai yu, Liang Zhao, Yisong Wang, Jiaying Liu, and Chong Ruan. Janusflow: Harmonizing autoregression and rectified flow for unified multimodal understanding and generation, 2025. URL htt.htttt///xiiiii org/ wtgg/bs////1la..o// 2411. 07975
Oier Mees, Lukas Hermann, Erick Rosete-Beas, and Wolfram Burgard. Calvin: A benchmark for language-conditionne policy learning for long-horizon robot manipulation tasks. IEEE Robotics and Automation Letters (RA-L), 7(3):7327-73333 2022.
Yao Mu, Qinglong Zhang, Mengkang Hu, Wenhai Wang, Mingyu Ding, Jun Jin, Bin Wang, Jifeng Dai, Yu Qiao, and Ping Luo. Embodiedgpt: Vision-language pre-training via embodied chain of thought. Advances in Neural Information Processing Systems, 36:25081-2509. 2023a.
Yao Mu, Qinglong Zhang, Mengkang Hu, Wenhai Wang, Mingyu Ding, Jun Jin, Bin Wang, Jifeng Dai, Yu Qiao, and Ping Luo. Embodiedgpt: Vision-language pre-training via embodied chain of thought. Advances in Neural Information Processing Systems, 36, 2023b.
Suraj Nair, Eric Mitchell, Kevin Chen, Silvio Savarese, Chelsea Finn, et al. Learning language- conditioned robot behavior from offline data and crowd-sourced annotation. In Conference on Robot Learning, pp. 1303-1315. PMLR, 2022.

---

Published as a conference paper at ICLR 2026
Fei Ni, Jianye Hao, Shiguang Wu, Longxin Kou, Liu Jiashun, Yan Zheng, Bin Wang, and Yuzheng Zhuang. Generate subgoal images before act: Unlocking the ohaingof-thought reasoning in diffusion model for robot manipulation with multimodal prompts. Computer Vision and Pattern Recognition, 2024a.
Fei Ni, Jianye Hao, Shiguang Wu, Longxin Kou, Yifu Yuan, Zibin Dong. Jinyi Liu, MingZhi Li, Yuzheng Zhuang, and Yan Zheng. Peria: Perceive, reason, imagine, act via holistic language and vision planning for manipulation. Advances in Neural Information Processing Systems, 37: 17541-17571, 2024b.
William Peebles and Saining Xie. Scalable diffusion models with transformers, 2023, URL https
7/arxiv. wwg/ abs
/ wtw.aa/////22.... 22222.9.779
Liao Qu, Huichao Zhang, Yiheng Liu, Xu Wang, Yi Jiang, Yiming Gao, Hu Ye, Daniel K. Du, Zehuan Yuan, and Xinglong Wu. Tokenflow: Unified image tokenizer for multimodal understanding and generation, 2024.
org/ wwtpabs/////ooottc.ttt..tt..w.tt 2412 03069
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning, pp. 8748-8763. PMLR, 2021.
Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, Y. K. Li. Y. Wu, and Daya Guo. Deepseckmath: Pushing the limits of mathematical reasoning in open language models, 2024. MiRLittpp://arxivorrr
2402. 03300
Weijia Shi. Xiaochuang Han, Chunting Zhou, Weixin Liang, Xi Victoria Lin, Luke Zettlemoyer, and Lili Yu. Lmfusion: Adapting pretrained language models for multimodal generation, 2025. URL.
https://arxiv.. org/ alcp//as/220 /212 15188
Mohit Shridhar, Lucas Manuelli, and Dieter Fox. Cliport: What and where pathways for robotic manipulation. In Conference on robot learning, pp. 894-906. PMLR, 2022.
Achint Soni, Sreyas Venkataraman, Abhranil Chandra, Sebastian Fischmeister, Percy Liang, Bo Dai, and Sherry Yang. Videoagent: Self-improving video generation, 2025. http::::: //arxiv
brc/abs/2410. 10076
Richard S. Sutton and Andrew G. Barto. Reinforcement Learning: An Introduction. MIT Press, Cambridge, MA, 1st edition, 1998. ISBN 978-0262193988.
Chameleon Team. Chameleon: Mixed-modal early-fusion foundation models, 2025. URL https
//arxiv.orr//abs/2405-55
Shengbang Tong, David Fan, Jiachen Zhu, Yunyang Xiong, Xinlei Chen, Koustuv Sinha, Michael Rabbat, Yann LeCun, Saining Xie, and Zhuang Liu. Metamorph: Multimodal understanding and generation via instruction tuning, 2024a. URL http://///axxio//// www.aawwwwww.../////a/aaaae////a
Shengbang Tong, Zhuang Liu, Yuexiang Zhai, Yi Ma, Yann LeCun, and Saining Xie. Eyes wide shut? exploring the visual shortcomings of multimodal Ilms, 2024b. BRLhhtpss: //arxiv axg/abs/2401.. 06209
Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.0922 2023.
Leandro von Werra, Youncs Belkada, Lewis Tunstall, Edward Beeching, Tristan Thrush, Nathan Lambert, Shengyi Huang, Kashif Rasul, and Quentin Gallouédec. Trl: Transformer reinforcement learning. /tttps::/githae
com www.///wgggggac/ee/taaaa.... 2020.
Chengyue Wu, Xiaokang Chen, Zhiyu Wu, Yiyang Ma, Xingchao Liu, Zizheng Pan, Wen Liu, Zhenda Xie, Xingkai Yu, Chong Ruan, and Ping Luo. Janus: Decoupling visual encoding for unified multimodal understanding and generation, 2024. URL httpp::////// //arxiiv. oxq/ atg/abs////e 2abs/240 /2410
13848
12

---

Published as a conference paper at ICLR 2026
Junfeng Wu, Yi Jiang, Chuofan Ma, Yuliang Liu, Hengshuang Zhao, Zehuan Yuan, Song Bai, and Xiang Bai. Liquid: Language models are scalable and unified multi-modal generators, 2025a. URL https::/aaxxx.c.omm org/ abs /2122 04332
Yecheng Wu, Zhuoyang Zhang, Junyu Chen, Haotian Tang, Dacheng Li, Yunhao Fang, Ligeng Zhu, Enze Xie, Hongxu Yin, Li Yi, Song Han, and Yao Lu. Vila-u: unified foundation model integrating visual understanding and generation, 2025b. URL aittps://arxise www.xiiiioorgaa.....oo.oaaa
2409. 04429
Jinheng Xie, Weijia Mao, Zechen Bai, David Junhao Zhang, Weihao Wang, Kevin Qinghong Lin, Yuchao Gu, Zhijie Chen, Zhenheng Yang, and Mike Zheng Shou. Show-o: One single transformer to unify multimodal understanding and generation, 2024. URL BRttps::://axxii
abs/
1088.12528
Lijun Yu, José Lezama, Nitesh B. Gundavarapu, Luca Versari, Kihyuk Sohn, David Minnen, Yong Cheng. Vighnesh Birodkar, Agrim Gupta, Xiuye Gu, Alexander G. Hauptmann, Boqing Gong, Ming-Hsuan Yang, Irfan Essa, David A. Ross, and Lu Jiang. Language model beats diffusion tokenizer is key to visual generation, 2024. URL htl.https:/// aaaaaiiiii xvv. org/ abs 2231 00.000.00 05737
Qiying Yu, Zheng Zhang, Ruofei Zhu, Yufeng Yuan, Xiaochen Zuo, Yu Yue, Tiantian Fan, Gaohong Liu, Lingjun Liu, Xin Liu, Haibin Lin, Zhiqi Lin, Bole Ma, Guangming Sheng. Yuxuan Tong, Chi Zhang. Mofan Zhang, Wang Zhang, Hang Zhu, Jinhua Zhu, Jiaze Chen, Jiangjie Chen, Chengyi Wang, Hongli Yu, Weinan Dai. Yuxuan Song, Xiangpeng Wei, Hao Zhou, Jingjing Liu, Wei-Ying Ma, Ya-Qin Zhang, Lin Yan, Mu Qiao, Yonghui Wu, and Mingxuan Wang. Dapo: An open-source Ilm reinforcement learning system at scale, 2025. URL http:: www.aaxxvv.r...... http://axxxi..oora////55555 14476
Andy Zeng, Pete Florence, Jonathan Tompson, Stefan Welker, Jonathan Chien, Maria Attarian, Travis Armstrong, Ivan Krasin, Dan Duong, Vikas Sindhwani, et al. Transporter networks: Rearranging the visual world for robotic manipulation. In Conference on Robot Learning, pp. 726-747. PMLR, 2021.
Jianke Zhang. Yanjiang Guo, Yucheng Hu, Xiaoyu Chen, Xiang Zhu, and Jianyu Chen. Up- vla: A unified understanding and prediction model for embodied agent, 2025a. URL https www.xii.oggabss///oeee
18867
Shengqiang Zhang, Philipp Wicke, Lütfi Kerem Şenel, Luis Figueredo, Abdeldjallil Naceri, Sami Haddadin, Barbara Plank, and Hinrich Schütze. Lohoravens: A long-horizon inguuage-conditionel benchmark for robotic tabletop manipulation. arXiv preprint arXiv: 12020, 2023.
Wenqi Zhang, Mengna Wang, Gangao Liu, Xu Huixin, Yiwei Jiang. Yongliang Shen, Guiyang Hou, Zhe Zheng, Hang Zhang, Xin Li, Weiming Lu, Peng Li, and Yueting Zhuang. Embodied- Synergizing visual search, reasoning, and action for embodied interactive tasks, 2025b. URL
https://arxite wwwp/ogg/////oooo//oo///....sss/
abs /bwp//03..3.22 2503.22..3. 21696
Qingging Zhao, Yao Lu, Moo Jin Kim, Zipeng Fu, Zhuoyang Zhang, Yecheng Wu, Zhaoshuo Li, Qianli Ma, Song Han, Chelsea Finn, Ankur Handa, Ming-Ya Yu Liu, Donglai Xiang, Gordon Wetzstein, and Tsung- Yi Lin. Cot-vla: Visual chain-of-thought reasoning for vision-language- action models, 2025. URL hRttps:::://asst /t/p///////////aaaaaaa/////////0 2200000000002220
Rui Zheng, Shihan Dou, Songyang Gao, Yuan Hua, Wei Shen, Binghai Wang. Yan Liu, Senjie Jin, Qin Liu, Yuhao Zhou, Limao Xiong, Lu Chen, Zhiheng Xi, Nuo Xu, Wenbin Lai, Minghao Zhu, Cheng Chang. Zhangyue Yin, Rongxiang Weng, Wensen Cheng, Haoran Huang, Tianxiang Sun, Hang Yan, Tao Gui, Qi Zhang, Xipeng Qiu, and Xuanjing Huang. Secrets of rihf in large language models part i: Ppo, 2023. https:::: httppps://rxiiii wttpvvivvivv.//.i.//.aa///.aatvs org/ wtwp/as////oooooooee.../oe.aaeee
04964
Chunting Zhou, Lili Yu, Arun Babu, Kushal Tirumala, Michihiro Yasunaga, Leonid Shamis, Jacob Kahn, Xuezhe Ma, Luke Zettlemoyer, and Omer Levy. Transfusion: Predict the next token and diffuse images with one multi-modal model, 2024. URL Stthttt:: ///a:xi///iii///// org/ abs / 2408 11039
13

---

Published as a conference paper at ICLR 2026
A RELATED WORK
A.1 UNIFIED MLLM FOR MULTIMODAL GENERATION
The success of large language models (LLMs) has spurred growing interest harnessing them for image generation, leading to variety of architectural approaches. One straightforward approach is integrating LLMs with image generation models, where the LLM produces conditions to guide the generation process (Ge et al. 2023c; Tong et al., 2024a; Huang et al., 2025). However, these models require additional generative models, which is not conducive to unified multimodal repre- sentation and model scaling. To address this, many works have attempted to use single model for multimodal understanding and generation. Some have explored combining diffusion models with LLMs (Xie et al., 2024; Zhou al. (024). Others have focused on discretizing images and using a unified autoregressive objective to model both text and images, Within this area, some works have concentrated on image discretization methods (Wu et al., 2025b aj Qu et al. (2024) while others have focused on the integration of multimodal tokens with LLMs (Liu et al. [025] Team 2025 Wu et al., 2024] Chen et al.] (025). In this paper, we propose a novel image generation method that achieves better generation quality while enabling efficient sampling, promising to become new paradigm for multimodal unified models.
A.2 HIERARCHICAL PLANNING FOR LONG-HORIZON MANIPULATION
In Long-sequence embodied operations, directly implementing end-to -ol-tt--ooe to-end actions can lead to error accumulation due to lack of intermediate guidance (Shridhar et al. [0222] Jiang et al. 2022 Nair al., 2022). Therefore, many studies have adopted hierarchical planning. breaking down complex instructions into sequential subtasks for execution. Language planning utilizes large language models (LLMs) to transform reasoning into sequentially interpretable instructions in natural language (Ahn al. 2022 Mu et al. 2023b). Some work also introduces explicit Chain of Thought (CoT) processes before generating actions (Zhang et al., 2025b; Zhao et al. 2025). Visual planning decomposes
complex instructions into sequential subgoal images (Black et al. 2023b Ni et al. (024a), with additional efforts employing video generation models for task planning (Du et al. [223]] Soni et al.
2025). Multimodal planning integrates both approaches, generating interpretable instructions and
subgoal images simultaneously, thereby providing rich planning information (Ni et al. 2024b Zhang
cet al. 2025a).
A.3 REINFORCEMENT LEARNING IN LLM AND IMAGE GENERATION
Reinforcement learning (RL) Suttonn & Barto 1998) provides gradient-based policy optimization for non-diffcrentiable learning objectives such as human preference alignment (Bai et al. (022).. However, applying RL large language models (LLMs) remains challenged by inefficient sampling (Hu et al. 2024) and training instability (Zheng et al. 2023), driving substantial research efforts to develop robust policy gradient algorithms (Shao al. 2024] Yu al. 2025 Chu et al. 2025). While RL applications in image generation (Black et al. [023a] Fan et al. 2023) have demonstrated partial success, existing approaches suffer from prohibitive computational costs due to iterative sampling requirements of autoregressive architectures, coupled with persistent convergence challenges. In this work, we address these limitations through an efficient sampling framework for unified multimodal generation, integrating RL with supervised learning objectives to enable reward maximization under strict maximum likelihood constraints.
B
MORE RESULTS AND ANALYSIS OF ADDITIONAL EXPERIMENTS
B. OTHER CASES OF RSFT
In this section, we conducted additional experiments with RSFT, referencing the experiments in DDPO (Black et al.) 2023a). We designed two simple reward functions: compressibility and incompressibility. and implemented them using the same computation method as in DDPO. In practice, we performed the SFT task using expert dataset and employed these two rewards to encourage the model to optimize for compressibility (or inconpressibility) We present the reward curves(Figure 7 from the training along with the generation results.
14

---

Published as a conference paper at ICLR 2026
The experimental curves demonstrate RSFT's robust convergence under both reward objectives, with visualization analysis revealing reward-specific generation patterns: images synthesized under the incompressibility objective exhibit sharpened features and intricate background complexity, while the compressibiliy--diies objective produces simplified outputs with attenuated background details. Crucially, these divergent behaviors remain anchored to coherent image structures through the maximum likelihood constraint a core mechanism preventing arbitrary deviation from fundamental visual semantics. This effect originates from the method's dual optimization dynamics: reward gradients steer style adaptation while the likelihood penalty preserves content fidelity, achieving stable exploration within semantically grounded manifolds regardless of reward surface geometry.
incompresbbllly
Compresibilly
-50
-165
28
Doward
26
-175
-18
-185
22
-900
-9.5
200
400
800
800
1000
1200
3400
2600
200
400
600
800
1000
1200
1400
Steps
(a) Incompressibilitty
(b) Compressibility
Figure 7: RSFT training curves with Incompressibility (a) and Compressibility (b) reward functions.
GT
Compressibility
Incompressibility
Figure The generation effects on other rewards.
15

---

Published as a conference paper at ICLR 2026
B.2
VISUALIZATION DIFFERENT VARIANTS
Our ablation study reveals critical insights through visual comparisons of four variants: EVLP, EVLP-AR, EVLP-w/o-Se, and EVLP-w/oEE, EVLP achieves superior synthesis quality with precise semantic alignment and geometrically consistent layouts, whereas EVLP-AR suffers severe perfor- mance degradation due to eπrо accumulation in autoregressive decoding each iterative prediction step propagates spatial distortions that compound across generation steps. While EVLP-w/o-Se maintains basic object recognition, it fails to preserve fine-grained part relationships, producing anatomically implausible structures. Similarly, EVLP-w/o-En generates blurred textures with incon- sistent lighting patterns, demonstrating the quantizer's role in disentangling high-frequenc" details from latent representations. These results quantitatively validate our architectural decisions through controlled component deactivation.
GT
EVLP
EVLP-AR
EVLP-w/o En
EVLP-w/o Se.
Figure 9: The generation effects of different variants.
B.3 ABLATION ON MULTIMODAL PLANNING
In this section, we designed experiments to explore the advantages of multimodal tasks compared to unimodal tasks. Specifically, we conducted the EVLP w/o Gen experiment, in which the image gen- eration task was removed from all stages, transforming the model into a purely language- planning system. We compared the success rates and language accuracy (Exps. A and Table The results clearly demonstrate that multimodal tasks not only enhance the over-
Table 5: Comparative analysis task planning performance across different variants.
sunguage-planing
Method
LA
A
EVLP
67.6
87.0
B
wwo Gen
53.7
80.1
16

---

Published as a conference paper at ICLR 2026
all success rate but also significantly improve language accuracy, underscoring the positive impact multimodal tasks on unimodal performance.
B.4
SAMPLING EFFICIENCY
EVLP's single-step generative design can draw multiple independent samples in a single forward pass, dramatically increasing sampling throughput compared with diffusion and autoregressive (AR) models. Table@rrpoor wall-clock generation time for producing vs. images. EVLP is orders of magnitude faster in both 1.SB and 7B scales, and its advantage widens as model size grows.
Table 6: Image sampling time (seconds) for generating and 8 images. Diffusion corresponds to SuSIE; AR refers to autoregressive baselines. EVLP uses a single-step generator.
Method
1 Image (s)
8 IImages (s)
Diffusion (SuSIE)
4.41
35.36
AR-1.5B
5.31
42.64
AR-7B
21.37
172.96
EVLP-1.5B
0.05
0.13
EVLP-7B
0.15
0.40
As shown, EVLP outpaces diffusion and AR by large margins (e.g., 0.15s vs. 21.37s for 7B, single image), and the gap further enlarges when generating batches (8 images: 0.40s vs. 172.96s).
C DETAILS OF METHOD
C.I REWARD FUNCTION
This reward function employs a dual-constraint mechanism to guide generative models in achieving both physical dynamic consistency and visual coherence. The core design principles are: Ensuring spatiotemporal alignment of object motion trajectories between generated and real images; (2) Enforcing that appearance changes during object motion adhere physical interaction principles. The implementation comprises three critical phases:
Dynamic Region Detection: The pipeline processes input image pairs (xt, 1 through Gaussian blurring to enhance robustness against imaging noise, followed by differential computation and morphological closing operations to extract significant change regions. Adaptive thresholding combined with contour analysis identifies candidate bounding boxes, with non-maximum suppression eliminating overlapping regions produce final dynamic region set B.
Joint Feature Matching:
For ground truth and generated dynamic regions (Biabll and Been respec- tively), we construct an loU similarity matrix and apply the Hungarian algorithm for optimal bipartite matching, Valid matches undergo pixel-level discrepancy analysis:
dy MSE
label 19j
pen
(6)
where Gj denotes normalized image patches.
Multimodal Reward Computation:
Integrating spatial alignment and pixel-wise consistency, we formulate the composite reward function:
Eo.nes (loU,; + Adzÿ) - [ + |Beenl 2|MM)
r
(7)
min( BLabel [Beam)
17

---

Published as a conference paper at ICLR 2026
Algorithm 1: Dynamic-Aware Reward Computation
Input:
Current state image
Generated image "++1
Real image real
Hyperparameters: IoU threshold т, MSE weight X, penalty coefficient 7
Output: Reward r
Initialize dynamic regions;
Blabel DeteciRegions(zz, x/+1)
gee DetectRegions(az 2++1)
Compute region matching;
M PairwiseloU(Bradd B Bgen)
noww_nnd, col ind) Hungarian M))
M = M(i, 2 r} Calculate multi-scale reward:
score
Σ (IJ)MM
[
MSE City
Jabel 2on
where Jahel = Crop(z Blabel) Eg
gon Vj
son = Crop( 1+ Bleen)
Apply penalty normalization;
score Blabel + | Been 2|
max(1, min( [Babetl |Benn 1))
return
C.2 DETAILS PRETRAINING
During the dynamic perception pretraining phase, EVLP focuses on developing foundational mul- timodal generation capabilities through inverse and forward dynamic reasoning while enhancing spatial comprehension, change detection, and dynamic imagination. For generation, we employ Open-MAGVIT22-11-22222
as our image tokenizer, utilizing pretrained weights fine-tuned on our dataset for one epoch. The understanding module integrates SigL as the semantic encoder and leverages the tokenizer's encoder for spatial encoding, with a convolutional pooling adapter (stride=2) bridging feature hierarchies. The architecture adopts Qwen2.5- 1.5B --B-Instuutttt the LLM backbone, augmented with an auxiliary image head to project features into pixel probability distributions.
Throughout pretraining, SigLIP and tokenizer weights remain frozen while fully training other components. We implement a 4,000 4,000-step training regimen (16 hours) with batch size 2048, employing the AdamW optimizer (Loshchilov Hutter (017) configured with learning rate 1e-4, 3% warmup proportion, and 0.01 weight decay. This configuration balances computational efficiency with gradient stability during large-scale multimodal alignment.
C.3 DETAILS RSFT
We introduce in the main text novel optimization algorithm termed Reinforced Supervised Fine- Tuning (RSFT), which effectively combines policy gradients with supervised fine-tuning (detailed implementation procedures are provided in Algorithm 2}. In practical implementation, following pre-training completion, we initially conduct conventional supervised fine-tuning (SFT) for 1,000) steps with batch size of 1,024, subsequently proceeding with 4,000 steps of RSFT training using the AdamW optimizer configured with a learning rate of Ic-4, 3% warmup proportion, and 0.01 weight decay- configuration that optimally balances computational efficiency and gradient stability during large-scale multimodal alignment tasks.
attps: https///arriiaeeeeeeeeeeee///sss Voken
18

---

Published as a conference paper at ICLR 2026
Algorithm 2: Reinforced Supervised Fine-Tuning (RSFT)
Input:
Training dataset D - {[g, It, dz, T(+1))
Pretrained model Pg with parameters
Reward function R(-)
Hyperparameters: batch size B, learning rate n, reward samples K, loss weight À Output: Optimized model parameters l Initialize optimizer (e.g., Adam) with learning rate r7;
while not converged do
Sample batch (g)(0, xx(1) ai ((44))b= from D;
Compute SFT Loss LSFT:
B
(<0) .g.2.) log P(1r+119, log P(ay" (3) la
LSFT BL
b=
Compute Reinforcement Loss LRI:: for each sample b [1, B do
Generate K image samples: {x {2f+1]k=1 1K ~ P [g (b) xz a (0)):
(b)
Compute rewards: F = R(=
Normalize rewards per batch: nn
Th-Pe where Mr, or are batch mean/std; dr
Compute advantage: A&- Pks
Compute policy gradient loss:
B
LRL
A log 19 xx( )
BK b km
Joint Optimization: Total loss: L LSFT + ACRL: Update + 0 VoL:
We further contextualize our algorithm against established RL baselines, specifically analyzing GPG (Chu et al., 2025) and the online variant of GRPO (Shao et al., 2024). The vanilla policy gradient method (GPG) directly optimizes action probabilities based on advantage estimates, funda- mentally lacking stabilization mechanisms against high- verrince gradient updates. GRPO-onpolicy addresses this instability through KI-divergence constraints between the optimized policy and reference model - the latter pretrained expert data via maximum likelihood estimation. While this prevents policy collapse, it introduces substantial computational overhead from maintaining dual models and propagating gradients through KL-divergence estimation. In contrast, RSFT circumvents these intermediate steps by directly incorporating expert supervision through maximum likelihood constraints demonstration actions a. seamlessly integrated into the policy gradient objective. This design eliminates the memory/computational bottlenecks of KL constraints while preserving expert alignment, as the additive log-probability terms explicitly regularize policy updates without reference model dependency. The absence of auxiliary models makes RSFT particularly advantageous in deployment scenarios requiring frequent policy updates or cross-domain adaptation, where GRPO- online's 's dependence on pretrained reference models becomes prohibitive due to potential expert data fitting errors and cascading approximation errors in KI. estimation.
RI. Method
Loss Function
Constraint
GPG
Chu et al. 2025
L. = -B
As log mola
No Constraint
GRPO onpolicy
Shao al. 2024 C. -E
Av log To s) (To(( (eer(-15))
KL Divergence
K km
RSFTYOurs)
log na(a | a) loga
Maximum Likelihood
Table Comparison of some RL methods.
19

---

Published as a conference paper at ICLR 2026
D DETAILS OF BENCHMARKS AND TASKS
LoHoRavens LoHonavens (Zhang et al. 2023) is benchmark dataset based on the Ravens robot simulator, containing various long-horizon manipulation tasks. We categorize these tasks into three types: Stacks, Sort, and Matching. In Stacks tasks, the goal is to place blocks absolute or relative positions; Sort tasks require sorting blocks bowls with similar attributes; while Matching tasks involve attribute matching, such as placing blocks of corresponding colors into matching bowls. These tasks encompass multiple aspects of long-horizon reasoning, including color, size, space, arithmetic, and reference. To successfully complete each task, the robot must effectively integrate various reasoning capabilities and develop an appropriate long-term plan. In addition, we have extended a series of Letters tasks, which require the agent to understand information such as the shape and position objects. We designed three types of tasks within this category: Shape, Orders, and Spell. Detailed task descriptions can be found in Table
Meeting Preparation Meeting Preparation is an in-house universal operational benchmark de- signed for real-world office scenarios, challenging agents to accomplish conference preparation tasks in highly diversified desktop environments. The test environment incorporates diverse office supply variants, with key item categories (c.g.. cups) exhibiting rich morphological variations (mugs, glasses, vacuum flasks, etc.), combined with multiple environmental variables: desktop backgrounds of different materials, multiple camera perspectives, and dynamic object layouts. The core task demands agents to transcend superficial feature recognition and establish a semantic conceptualiza- tion framework-foe for instance, abstracting the common characteristics of "cups" from objects with varying heights, transparencies, and handle configurations, then strategically positioning them in task-appropriate areas according conference requirements. This benchmark rigorously evaluates agents' capabilities in cross- generalization, fine- grained object recognition, and contextual reasoning, presenting critical challenges including (1) maintaining categorical cognitive stability when object functionality decouples from morphology, (2) interpreting scene intentions amidst multimodal interference. Examples of both benchmarks can be found in the Figure T0.
Vision Planning
put the blacks in bowls with matching colors
Language Planning
pick the browm block and place on the brown bow!
pick the yellow block and place it on the yellow bow!
pick the gray bleck and place on the gray bow!
pick the cyen block and place on the cyan bowl
pick up the blue block and place it on the blue bow!
Vision Planning
prepare meeting supplies for right-handed Individual
Language Planning
place notebook
place the nome plate top
place pen on right notebook
place name plate
place coffee cup right
nome plate
Figure 10: The example of benchmarks.
E DETAILS OF DATASETS
For the LoHoRavens and Meeting Preparation datasets, we utilize the provided oracle engines to collect expert demonstration data. It is worth noting that if there are multiple correct answers or multiple ways to complete the task, we only focus on whether the instruction-speciiied complex task ultimately accomplished and include all correct demonstrations training data. For the 20 tasks in LoHoRavens, we collect 2,000 demonstrations per task, resulting in a total of 40,000 long- long-horizon demonstrations with uub-task counts ranging from 2 to 10+. In the Meeting Preparation environment, we gather 3,000 demonstrations. For all collected data, we split it into an 90% training dataset and a 10% testing dataset.
20

---

Published as a conference paper at ICLR 2026
Table 8: Overview of three main task types: Blocks, Letters, and Meeting.
Task Type
Description
Horizon
Color
Size
Spatial
Semantie
Blocks
Move all the blocks to the [ABS POS] area
4 15
X
V
Move all blocks of color the red Move all blocks
15
V
Move
[ABS POSJ area the (ABS POSJ area
15
Move all the blocks on the corner/side
~ 15
X
V
Stack all the blocks
< 15
X
V
X
Stack blocks of the same size
4~ 15
X
V
Stack
Stack blocks in alternate colors Stack only the primary color blocks on left side
2~ 15
V
12
X
Put the blocks the bowls with matching colors Put the blocks the bowls with mismatching colors
2 12
X
Matching
2 12
Put blocks of the same color in the zone with matching color
~~ 12
Letters
Sort the vertically symmetrical letters to the boltom side Sort the horizontal symmetrical letters to the blank space Sort the central symmetrical letters corner
2 ~~ 15
Shape
~ 15
X
15
tables in alphabetical order Put the letters on the tables in reverse alphabetical order Sort the consonants from all letters in orders
Put letters
15
Orders
2 ~~ 15
X
~~ 15
Spell words that are long possible
~~ 15
Spell
Spell out the
top CS conference
< 10
Spell name common transportation
4~ 15
Meeting Set up
Set up meeting supplies
10
During the dataset initialization process, we document each step's utilized assets and annotate their corresponding attributes, enabling precise identification of the color, size, and spatial relationships of the manipulated objects in each subtask's 's pick-and-place operations, thereby constructing accurate language actions. However, during testing, the model cannot access underlying environmental information, such as the exact count of real blocks or letters and their various attributes. The model must directly perceive from visual observations and reason based on these critical visual details, which significantly increases the task's difficulty.
F DETAILS OF BASELINES
CLIPort CLIPort Shridhar et al. 2022 is popular end-to-end algorithm that functions language-conditione imitation learning agent, directly processing high-level language instructions without requiring planner. It integrates the broad semantic understanding of CLIP (Radford et al.) 2021) with the spatial precision of Transporter (Zeng et al. 2021). As an end-to-end baseline, we employ CLIPort without modifications, training by directly pairing high-level instructions with corresponding actions from the dataset.
PAR PAR (Zhang et al. 2023) (Planner-Actor -ctorp--rtttt-)rrreeee a paradigm that replaces the skill pre- dictor with large language model (LLM) and employs vision-language model (VLM) reporter for visual observations. Instructions and generated captions are then fed into the LLM for linguistic planning. In PAR, Llama 13B (Touvron et al.) 2023) and the VLM OpenFlamingo (Awadalla et al., 2023) serve as the planner and reporter, respectively, using few-shot prompting. Notably, the Actor (or low-level foundational model) precisely the language-conditioned CLIPort trained with step-by-sted sub-instructions, as mentioned carlier. To ensure fair comparison. we make no modifications and keep the low-level foundational model consistent with CLIPort across all other baselines.
EmbodiedGPT EmbodiedGPT (Mu et al. 2023b) represents a standard paradigm that integrates multimodal large language models (MLLMs)) for language planning. The key distinction between
21

---

Published as a conference paper at ICLR 2026
EmbodiedGPT and PAR lies in replacing the LLM+VLM combination with advanced MLLM, which exhibits superior visual reasoning capabilities. EmbodicdGPT leverages a constructed embodied thain-of-hee of-thought dataset train the MLLM, enabling it to perceive visual details in hidden layers, similar to LLaVA (Liu et al. 2023b). To ensure fair comparison, we maintain consistency in the low-level foundational model across all baselines by using CLIPort as the common base.
SuSIE SuSIE (Black et al. 2023b) introduces a hierarchical framework that employs an image- editing diffusion model as high-level planner to generate intermediate subgoals for low-level controller to execute. The method adopts InstructPix2Pix (Brooks et al.) (023) as its pre-trained
image-editing model and fine-tunes it using language-annotated video clips and robot trajectory
data from CALVIN (Mees et al. 2022). However, due to the sensitivity of image-editing models to training data, we observed limited generative performance in the Ravens domain. To address this, we conducted additional fine-tuning while strictly maintaining the same number of training iterations and dataset composition as in EVLP.
CoTDiffusion CoTDiffusion (Ni et al. 2024a) represents a standard visual planning paradigm capable of translating complex instructions prompts-into visual subgoal images through a chain-of- thought reasoning process. The most notable distinction from SuSIE lies in CoTDiffusion's 's explicit incorporation of a semantic alignment module within its diffusion model. This module ensures correspondence and semantic coherence between generated images and high-level instructions, enabling chain-of ofthhuhf- generation. Similar to SuSIE, we fine-tune CoTDiffusion on our collected dataset and employ the same low-level image-conditionel Seaiicc-spccffcallly the image-conditioned variant CLIPort for consistent implementation.
PERIA PERIA (Ni et al. 2024b) represents standard multimodal planning paradigm that inte- grates LLMs with image generation models to achieve unified multimodal planning. It supports the conversion of complex general instructions into multimodal sub-instructions (combining language actions with visual goals) while ensuring planning correctness through multimodal alignment. We fine- tune PERIA collected dataset and employ multimodal conditioning strategy- sppecffcili..lll... the multimodal-comdiiione variant of CLIPort for implementation.
G DETAILS OF LOW-LEVEL POLICY LEARNING
To execute multimodal planning, we implement low-level policy using CLIPort, chosen for its native SE(2) action space that is particularly suitable for Ravens benchmarks. We develop two variants: language-conditiones policy trained with stepwise sub-instructions and an image-goal- conditioned policy trained with coherent keyframes, both serving as foundation models for planning tasks. For multimodal integration, design novel architecture that simultancously processes image subgoals and language instructions through a 4-layer cross-attention network (4 heads, 768- dim embeddings). Training samples consist of expert trajectories a, observations o, instructions and subgoal images v from Derasn The policy & minimizes the action prediction loss: Laction =
Et lå, P (aglor e,, 222..2 optimized with AdamW (Ir le-4, 500-step warmup, weight decay 0.01) for 10a steps (batch size 64). This approach provides dual advantages: (1) explicit subgoals reduce effective horizon complexity, and (2) combined language-visn viiual conditioning enables segmented action prediction without requiring full-sequence conditioning on global instructions, significantly simplifying policy learning while maintaining execution quality.
H LIMITATION & FUTURE WORK
While EVLP demonstrates significant improvements in long-horizon manipulation tasks with complex instructions, several limitations warrant attention in future research:
Data Dependency: The current implementation relies on pre-collected datasets for training the Multimodal Large Language Model (MLLM). While this approach effectively develops reasoning and planning capabilities, it may constrain the framework's 's adaptability to novel environments or tasks that substantially deviate from the training distribution. Future research could explore online
22

---

Published as a conference paper at ICLR 2026
learning paradigms adaptive fine-tuning methods to enhance EVLP's generalization capacity in unseen scenarios.
Sim-to-Real Gap: Although EVLP exhibits strong performance in simulated environments, its effectiveness in real-world applications remains be validated. Practical implementation introduces additional challenges including sensor noise, dynamic environmental changes, and physical interaction constraints that may impact system performance. Subsequent work should investigate EVLP's deployment on physical robotic platforms and rigorously evaluate its robustness and operational efficacy in authentic settings.
Despite these limitations, EVLP establishes novel paradigm for rohotic manipulation under general task instructions. By addressing these challenges through continued refinement, we anticipate that EVLP will inspire new research directions in long-horizon task execution with free-form instructions. This progression may ultimately contribute to developing more intelligent and versatile robotic systems capable of operating effectively across diverse real-world applications.
I SOCIAL IMPACT
The ALG framework enhances human-robor collaboration in manufacturing, healtheare, and do- mestic services by enabling natural instruction comprehension, boosting productivity and quality of life while creating job opportunities. In education, ALG--poered robots guide children through adaptive learning activities like puzzle-solving, fostering cognitive development through personalized interactive tutoring. While advancing embodied AI capabilities, its deployment requires prioritized safety protocols, ethical compliance, and equitable accessibility to ensure responsible societal integration.
LLM USAGE
Large language models (LLMs) were used solely to polish the writing (e.g., grammar correction and phrasing improvements). They did not contribute to research ideation.
23


Reviews:
[{"review_id": "OJ6QUnYkxo", "reviewer": "Reviewer_SJjD", "summary": "At its core, this paper proposes EVLP, a unified multimodal planner that jointly generates (i) step‑wise language actions and (ii) visual subgoal images for long‑horizon manipulation. On LoHoRavens, it beats language‑only, vision‑only, and prior multimodal planners (avg. ~6% over SOTA), and it samples images much faster than diffusion/AR baselines.", "questions": "1. The paper does not include compute budgets (GPUs, training time per stage, memory), and code/model checkpoints are not clearly committed. Please include them and outline your code/model weight policy clearly.", "limitations": "", "rating": 8, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. Unified generation that is practically faster and simpler than AR/diffusion.\nThe paper’s well-motivated one‑step generator lets the LLM model the full image‑token distribution directly. Sampling throughput is compelling: EVLP reports 0.15 s for one image and 0.40 s for eight images, compared with 21.37 s and 172.96 s for autoregressive baselines.\n\n1. Consistent gains across long‑horizon tasks. EVLP beats PERIA and CoTDiffusion by healthy margins across sub-tasks. \n\n1. Strong ablations. While the paper has a somewhat involved architecture (SigLIP + image encoder + quantized codebook), the ablations are thorough and cover the choice of encoder ('vision tower'), RL algorithm, etc.", "weaknesses": "1. Novelty vs. prior “unified” generators is under-explained. The paper cites Janus-style unified models and diffusion‑plus‑LLM hybrids, but the precise algorithmic novelty for one‑step image sampling (beyond “learnable image tokens + direct p(x|c) modeling”) could be spelled out more crisply. Concretely, for example, what prevents degenerate modes in your quantized codebook? This is a common issue in training codebook-based models.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "kqIoNctQdR", "reviewer": "Reviewer_c8Tb", "summary": "This paper presents EVLP, a unified multimodal framework for long-horizon robo manipulation tasks. The approach combines three components: (1) a dual-tower vision architecture integrating SigLIP for semantic understanding with MAGVIT2 discrete tokenization for one-step image generation, (2) dynamic perception pretraining using inverse and forward dynamics prediction tasks, and (3) RSFT that combines maximum likelihood with policy gradients to align spatial consistency between language actions and generated images. Experiments on LoHoRavens and Meeting Preparation benchmarks show improvements over baselines including language-only (PAR, EmbodiedGPT), vision-only (SuSIE, CoTDiffusion), and multimodal (PERIA) planning methods.", "questions": "- The 'one-step generation' claim is somewhat misleading—the model generates 256 tokens in parallel, which is more accurately described as parallel decoding rather than a fundamentally different generation paradigm. Is framing it as 'parallel token generation' more accurate?\n- Your dynamic alignment reward (Equation 7) relies on frame differencing to detect moving regions. How does this work when the background is not static (for example with a moving camera) or if multiple objects are moving at the same time?\n- How much data is used for dynamic perception pertaining, and where does this data come from? Have you tested how pretraining data quantity affects performance?", "limitations": "", "rating": 2, "confidence": 4, "soundness": 3, "presentation": 1, "contribution": 2, "strengths": "- This is a well motivated problem with clear practical value. Addressing an important gap in embodied AI: the separation of language and vision.\n- Table 6 shows dramatic speed ups from one-step generation  (0.15s vs 21.37s for 7B model), which is a truly practical advantage\n- I find the idea to be quite interesting and innovative. One-step generation is important for an application like robotics. I like the idea of unifying language and visual planning.\n- There were thorough ablations and a detailed appendix. I also like the thorough multimodal evaluation.", "weaknesses": "- I believe there to be major writing quality issues which hinder presentation. There are numerous grammatical errors throughout (\"lead to inconsistent in\", \"spatio-awared\"). I think the main paper is missing key technical details like the architecture of the \"low-level visual encoder\", the \"image reconstruction loss\", the dataset of the vision tower. For the RSFT algorithm, K is never defend and it isn't explained how the advantage A_k is computed. Also missing the optimizer, learning rate, training steps, etc. A lot of this should be moved to the main paper. Notation is used inconsistently throughout the paper. For instance, model parameters θ appear explicitly in some equations (Equation 1: log P(a_t|...; θ)), as subscripts in others (P_θ(x|c)), and are completely omitted in Algorithm 2 despite describing the same probability distributions, making it difficult to track dependencies on trainable parameters.\" Minor presentation issues are fine but in this case, it makes it to difficult understand the method.\n- I think another major issue is the lack of real-robot validation. Real-world evaluation is limited to an offline dataset (LA/LPIPS) rather than physical execution with success rates on hardware; this limits claims about sim-to-real.\n- The main gains are shown on Ravens-style and \"Meeting Preparation\" simulations. I would like to see more diverse, open-world household benchmarks (multi-room, distractors, etc) which would help strengthen the claims. \n- I think an important baseline that is missing is state-of-the-art Vision-Language-Action (VLA) policies, like OpenVLA, Pi-0.5, etc.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "oHEPy0ruvr", "reviewer": "Reviewer_yA48", "summary": "The paper presents EVLP, a unified framework for embodied vision-language planning that jointly generates textual actions and visual sub-goals.\nThe authors propose: A Unified Multimodal Generation Framework that integrates language and image token generation within a single model; A Dynamic Perception Pre-training stage based on inverse and forward dynamics prediction; A Reinforced Supervised Fine-Tuning (RSFT) scheme that combines SFT with reinforcement learning for task-specific optimization.\nExperiments on multiple embodied simulation benchmarks (e.g., LoHoRavens, Meeting Preparation) demonstrate improved task success rates compared to several recent baselines such as CLIPort, EmbodiedGPT, and PERIA.", "questions": "No", "limitations": "", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The idea of generating both visual sub-goals and language actions within a single autoregressive framework is elegant and practically appealing for long-horizon embodied tasks.\n2. The paper provides comprehensive experiments and ablations, showing consistent improvements over strong baselines.\n3. Dynamic perception pre-training is a valuable design choice that improves model understanding of environment dynamics.", "weaknesses": "1. The proposed “Reinforced Supervised Fine-Tuning” is essentially a re-implementation of existing methods, without introducing new algorithmic insights or theoretical contributions. Its novelty is mostly limited to application in the embodied context.\n2. Experiments are confined to simulation. The paper does not discuss challenges in transferring the proposed unified planner to physical robots or real-world sensory noise.\n3. The method relies on a large unified transformer architecture; computational cost and scalability under constrained hardware settings are not analyzed.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "wBwHoWFJdJ", "reviewer": "Reviewer_inNa", "summary": "This paper proposes an innovative multimodal unified generation framework (EVTP), which enhances model consistency in problem-solving through dynamic perception preprocessing and reinforcement supervised fine-tuning (RSFT). In terms of the unified multimodal generation framework, it integrates semantic understanding and spatial encoding (visual tower design) to achieve more comprehensive visual perception; in dynamic preprocessing, the bidirectional alignment strategy strengthens the correlation between models; and in reinforcement supervised fine-tuning, it ensures the stability of generated images.", "questions": "See Weakness", "limitations": "", "rating": 4, "confidence": 5, "soundness": 2, "presentation": 2, "contribution": 1, "strengths": "See Summary", "weaknesses": "1. The paper treats reinforcement supervision fine-tuning (RSFT) as one of the core methodological innovations, and its combination of maximum likelihood supervision with strategy gradient optimization is effective. However, the current discussion of RSFT in the paper is mainly limited to comparisons with GPG and GRPO-onpolicy (Table 7), which does not fully place it in a broader LLM alignment research context. In recent years, the paradigm of combining reinforcement learning with supervised learning to optimize generative models has become a research hotspot, such as the combination of instruction fine-tuning (SFT) with reinforcement learning from human feedback (RLHF), and alignment algorithms like direct preference optimization (DPO) that do not require explicit reward models. It is recommended that the authors delve deeper into the differences and similarities in motivation, optimization objectives, and implementation mechanisms between RSFT and these mainstream alignment technologies (such as PPO-based RLHF, DPO, etc.) in the methodology section or related work section.\n2. The dynamic alignment reward function in the paper is the cornerstone of the RSFT framework, and its quality directly determines the optimization effect of the reinforcement learning stage. However, in Section 2.3 of the main text, its description is merely \"measuring whether the dynamics of the generated image is consistent with the real dynamics,\" which is too vague. The specific definition is placed in Appendix C.1, which weakens the integrity and readability of the methodology in the main text. Considering the importance of this reward function, it is strongly recommended that the authors move the key information from Appendix C.1, at least the simplified version of its core ideas and mathematical forms, to Section 2.3 of the main text. Specifically, the function should be clearly explained in the main text on how it calculates the reward by detecting dynamic regions, using the Hungarian algorithm for region matching, and combining IoU (spatial alignment) and MSE (pixel consistency).\n3. The experiments in the paper mainly focus on tabletop manipulation scenarios, such as LoHoRavens and Meeting Preparation. Although these are standard benchmarks in the field, they represent relatively restricted environments. The paper prospects for \"more open embodied scenarios\" in the conclusion but does not delve into the potential challenges that the EVLP framework may face when extended to more complex environments. For example, in complex 3D scenarios involving mobile navigation, is the single-step generation of complete visual sub-goals (a 2D image) still applicable? How should 3D spatial states or free-viewpoint images be characterized and generated? Moreover, when there are severe occlusions and partial observability in the environment, is the dynamic perception pretraining based on two frames sufficient? It is recommended that these potential extension issues be discussed in depth in the conclusion or a newly added \"Limitations and Future Work\" section, and a prospective analysis of the EVIP framework be conducted, exploring the improvements needed to deal with more complex scenarios (such as 3D navigation, interaction with deformable objects, etc.).\n4. The paper introduces the composition of the \"Vision Tower\" in Section 2.1, which combines a Siglip for semantic understanding with a trainable \"low-level visual encoder\" for capturing spatial details. The description of this low-level encoder is somewhat lacking, merely mentioning \"pretrained through image reconstruction loss.\" In Appendix C.2, it is mentioned that this encoder is the tokenizer's encoder of MAGVIT2, a key piece of information that should be explicitly stated in the main text.\n5. The paper emphasizes the sampling efficiency of EVLP during the inference stage, which is a significant advantage for its application in reinforcement learning. However, the paper does not mention the overall training cost. From the description, it appears that its two-stage training process (especially the pre-training stage with a batch size of 2048) requires a high amount of computational resources. It is suggested that a brief discussion of the training cost of EVLP (such as the required GPU model, number, and training duration) be included in the experimental section or the appendix, and compared with the main baseline models (especially PERIA, which also requires a generative model). Although fast inference speed is a significant advantage, a high training cost may limit its widespread application in both academia and industry.\n6. There are significant issues with Figure 1 in the paper as an overall framework diagram. Firstly, in the inverse dynamics part of the dynamic perception pre-training, there are two action description text inputs, which is very confusing and does not match the text description and formulas in the paper. Secondly, the red arrow (representing the backpropagation process) in the \"Reinforcement Supervised Fine-tuning\" section points to a rather chaotic direction. It seems that \"Reinforce Loss\" directly acts on \"Image Logits,\" but it is actually calculated after evaluating the reward function on \"Sampled Images.\" It is suggested to redraw Figure 1 to clearly and accurately show the process of the paper's model.\n7.The model is overly dependent on data preprocessing, limiting its effectiveness in application domains. In the dynamic preprocessing step shown in Figure 1, real-world object information needs to be fed into model training through the bidirectional alignment strategy. However, challenges arise when handling naturally occurring objects that have not been previously studied or discovered, and further research is needed to address this limitation.\n8.In the benchmark tests, key experimental metrics are evaluated under simulation modes, lacking real-world manifestation. This inevitably introduces data biases. Additionally, comparisons with advanced models such as VLIA-U are absent, which should be added to strengthen the persuasiveness of the experimental results.\n9.For image generation involving multiple movement processes, unified quantitative processing should be implemented, with clear comparative metrics provided to ensure the clarity and consistency of reasoning logic and experimental results.\n10.The paper compares models such as CLIPort, SuSIE, PERIA, and EmbodiedGPT in Table 1, but does not cover embodied multimodal systems such as RoboDreamer, Octo, and RT-Trajectory that have been publicly released in recent years. It is suggested that the authors supplement comparisons with these models or at least perform qualitative comparisons in the revision to demonstrate the leading position of EVLP in the research context of 2024-2025.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)