RIG: Synergizing Reasoning and Imagination in End-to-End Generalist Policy
Authors: Zhonghan Zhao
Wenwei Zhang
Haian Huang
Kuikun Liu
Jianfei Gao
Gaoang Wang
Kai Chen
Venue: ICLR 2026 Poster (2026)
Keywords: Multi-modal Embodied Agent
Unified Generative Model
Auto-Regressive World Model
Primary area: foundation or frontier models, including LLMs

Abstract: Reasoning before action and imagining potential outcomes (i.e., world models) are essential for embodied agents operating in complex open-world environments. Yet, prior work either incorporates only one of these abilities in an end-to-end agent or integrates multiple specialized models into an agent system, limiting the learning efficiency and generalization of the policy. 
Thus, this paper makes the first attempt to synergize Reasoning and Imagination in an end-to-end Generalist policy, termed RIG.
To train RIG in an end-to-end manner, we construct a data pipeline that progressively integrates and enriches the content of imagination and reasoning in the trajectories collected from existing agents. The joint learning of reasoning and next image generation explicitly models the inherent correlation between reasoning, action, and dynamics of environments. It thus exhibits more than $17\times$ sample efficiency improvements and generalization in comparison with previous works.
During inference, RIG first reasons about the next action, produces potential action, and then predicts the action outcomes, which offers the agent a chance to review and self-correct based on the imagination before taking real actions.
Experimental results show that the synergy of reasoning and imagination not only improves the robustness, generalization, and interoperability of generalist policy but also enables test-time scaling to enhance overall performance.

Published as a conference paper at ICLR 2026
RIG: SYNERGIZING REASONING AND IMAGINATION IN END-TO-END GENERALIST POLICY
Zhonghan Zhao Wenwei Zhang Gaoang Wang"1
Haian Huang
Kuikun Liu
Jianfei Gao²
Kai Chen
Zhejjang University Shangaai AT Laboratory
(zhonghan. 22, gaoangwang]e @ ntl. a.l...j.edus .du.en, en, {zhangwenwei, chenkai ab.plab.orga...
'qqual contribution.
CCorresponding author.
ABSTRACT
Reasoning before action and imagining potential outcomes (i.e. world models) are essential for embodied agents operating in complex open-world environments. Yet, prior work either incorporates only one of these abilities in an end-to-end agent integrates multiple specialized models into an agent system, limiting the learning efficiency and generalization of the policy. Thus, this paper makes the first attempt synergize Reasoning and Imagination in end-to-end Generalist policy, termed RIG. To train RIG in an end-to-end manner, we construct a data pipeline that progressively integrates and enriches the content of imagination and reasoning in the trajectories collected from existing agents. The joint learning of reasoning and next image generation explicitly models the inherent correlation be- tween reasoning, action, and dynamics of environments. It thus exhibits more than 17x sample efficiency improvements and generalization in comparison with pre- vious works. During inference, RIG first reasons about the next action, produces potential action, and then predicts the action outcomes, which offers the agent a chance to review and self-correct based on the imagination before taking real ac- tions. Experimental results show that the synergy of reasoning and imagination not only improves the robustness, generalization, and interoperability of generalist policy but also enables test-time scaling to enhance overall performance.
INTRODUCTION
To navigate the complexities of open-world environments, two quintessential human faculties are de facto to embodied agents: imagination of prospective outcomes and reasoning. Although reasoning endows agents with the ability to deconstruct task objectives into executable plans through logical inference, it inherently operates within the constraints of perceptual history. This limitation under- scores the complementarity of world models that learn the environmental dynamics, which not only allows the agent to predict action consequences but also facilitates risk-aware recision-makkin by evaluating hypothetical trajectories.
The synergistic integration of reasoning and imagination constitutes an indispensable foundation for more intelligent and robust embodied agents operating in dynamically evolving environments.
However, these two abilities are typically implemented in separate models. Specifically, reasoning
primarily exists in large vision language models (VLMs) (Zhao et al., 2023 WWang et al.] 2023a b
Zhao et al. 2024a) that parse visual input and produce textual insights and actions (Figure Ia]),
which lack explicit mechanisms for future prediction. In contrast, world models (Lin et al. 2023
Hafner et al. 2023) specialize in predicting future frames from video data (F)) which suffer from data inefficiency due to the implicit learning of concepts, physical laws, and environment dynamics. Recent attempts (Zhou et al., 20246] Zhang et al. 2023 Zhao et al. (024b) combine reasoning and imagination by connecting VLMs and visual generative models (VGMs). Yet. the integrated system (Figure [c)) prevents end-to-end optimization of the agent, leaving the mutual benefits between reasoning and world models underexplored.

---

Published as a conference paper at ICLR 2026
VLM
VLM
VLM-based agent
+
RiG
world
VGM
model
(b) world model agent
(c) hybrid-system agent
(d) RIG: synergized agent
Reasoning
low-level action
Imagination
Figure I: Comparison between conventional agents and RIG. RIG produces reasoning, actions, and imagination within single Transformer.
To bridge these gaps, this paper makes the first attempt to synergize Reasoning and Imagination in end-to-end Genralist policy. termed RIG Figure ([)) RIG learns textual reasoning, low- level action control, and image generation through the sequence-to-sequees within an autoregressive Transformer, motivation behind actions and their consequences could make RIG capture open open-world dynamics more comprehensively and improve the sample efficiency of training.
modeling objective hypothesize that the explicit modeling of the logic and
We develop RIG by adopting progressive data collection strategy because existing datasets typi- cally lack trajectories that contain interleaved image observations, precise actions, and high-quality textual reasoning. Based on initial trajectories collected from humans (Fan et al.] 2022) and exist- ing agents (Lifshitz et al. 2023) that contain only actions and image frames, we first use VLM to insert textual rationales before each action on the trajectory and train RIG-basic with the reasoning- enriched trajectories. During inference, RIG-basic generates actions purely from textual and visual inputs, without leveraging imagined future frames, as decisions are executed immediately based on current observations.
To further leverage visual imagination in reasoning to further improve the robustness of the policy, we collect unsuccessful trajectories from RIG-basic and adopt GPT-4o to review and revise these trajectories. Then, the suboptimal trajectories taken dreamed trajectories and combined with their corresponding revisions to form dream-review style trajectories for training RIG (also noted as RIG-lonkahead for clarity). In contrast RIG-basic that conduct reasoning without imagination, RIG-lookahead learns to first generate trajectory by taking the predicted images as the environ- ment states, and then review the hypothetical trajectory reasoning, and predict revised action that changes the environment. Such a design provides scalability at inference time, where the number of steps in the dream trajectory can be scaled so that the agent can more comprehensively understand the effectiveness of the action and make future-awar aware decisions.
We extensively evaluate RIG in the diverse, open-world Minecraft environment. Experimental re- sults show that RIG upgrades the state-of-the-art results embodied tasks, image generation, and reasoning benchmarks by 3.29x, 2.42x, and 1.33 respectively. Such a superior performance achieved by training RIG on only 111 hours of videos, which is 17x fewer than previous works that rely on 2000 hours of videos. Moreover, when scaling the training data, environmental interactions, and the lookahead steps during reasoning, the generalization ability and robustness of RIG consis- tently improve, which implies the potential of synergizing reasoning and imagination embodied agents. Our main contributions summarized as follows:
We introduce an end-to-end generalist policy that synergistically integrates explicit embodied rea- soning and visual imagination.
We propose a progressive data collection strategy coupled with straightforward language model- based training efficiently implement our method.
Our method naturally supports test-time scaling, enabling dynamic lookahead reasoning that en- hances action robustness and reduces trial-and-error during inference.
2

---

Published as a conference paper at ICLR 2026
2
RELATED WORK
Embodied Agents in Minecraft. Minecraft presents significantly open-ended and complex vironment (Johnson et al.) 2016; Guss et al. 2019; Fan et al. 2022] Wang et al., [023c] Cai et al.) 2023a) for embodied agents. Early approaches leveraged explicit world models predict future states (Hafner et al., 2023 Cai al. 2023b) but lack textual reasoning capabilities. Inspired by large language models (LLMs) (Brown al. 2020 Touvron et al. 2023), subsequent methods combined LLMs with low-level controllers to address long-horizon tasks. For example, Voyager (Wang et al.) 2023ap and STEVE (Zhao al. 2023) used LLMs for high-level planning integrated with code databases, while others like Jarvis-I I (Wang et al.] 2023b) paired LLMs with pre-trained low-level policy models such as VPT (Baker et al., 2022). However, these methods typically lack a world model to explicitly anticipate future visual outcomes. More recently, MineDreamer (Zhou et al.) 2024b) integrates a world model and a policy controller, yet treats vision generation and policy con- trol as separate modules, limiting coherent multi moltal reasoning. In contrast, RIG first attempts to explore an end-to-end generalist policy that simultaneously learns textual reasoning, visual imagi- nation, and low-level action predictions to achieve high generalization ability and sample efficiency.
World Models for Embodied Agents, Learning robust world models is essential for embodied
agents to effectively plan and act within simulated environments (Oh et al. 2015 Kaiser et al.)
2020). Early approaches primarily focused on action-conditioned video prediction or latent imag- ination for sample-efficient rollouts (Hafner et al., 2020 2021 Schrittwieser et al. 2020 Hansen et al. 2022: Lin et al. 2023), yet they often tightly coupled the world model with specific policies, limiting their adaptability. Inspired by recent successes in large-scale pre-traine pre-training (Wu al. [023]
Mendonca et al. 2023) and cansffrmme-basee architectures (Micheli et al. (023), several methods now leverage generalizable knowledge to model visual and textual distributions. However, these models typically overlook explicit reasoning and deeper causal relationships between actions and
resulting visual states. RIG explicitly learns to model the joint distribution of textual reasoning. actions, and their visual consequences enable more accurate predictions of complex and evolving environment dynamics.
Unified Understanding and Generation. Multi-modas Large Language models (MLLMs) aim to tackle understanding and generation tasks across different modalities (Lu et al. 2023 Zhou et al. 2024a) within unified architecture. Existing methods typically train large-scale image-text datasets to improve general visual understanding and generation capabilities (Wang et al., 2024 Yu et al. 20233 Xie al.) 2024] Zhou al.] (2024)). However, these datasets lack the interleaved action and reasoning trajectories required for training embodied agents, limiting their direct appli- cability real-world embodied scenarios. Generalist policies like GATO (Reed et al. 2022) and RT-1 (Brohan et al., (023) demonstrate multitask capabilities but optimize each task individually without fully leveraging inter-modal synergies. Our work synergizes textual reasoning, low-level action predictions, and visual generation.
3
METHOD
This paper makes the first attempt to explore the synergy of Reasoning and Imagination in end- to-end Generalist policy, termed RIG. RIG models image, textual reasoning, and textual action in sequence-t--sqquee
manner ($ 3.1. We adopt a progressive data collection strategy to first obtain RIG-basic that can reason before action but without imagination ($ (.22), then approach RIG- lookahead that learns to reason based on generated trajectories ($ 3.3
3.1 PRELIMINARY
Typical generalist policies follow an autoregressive paradigm to predict actions based on observa- tions (Hafner et al., 2023) Lin et al. 2023). RIG extends this framework by explicitly generat- ing intermediate textual reasoning before action prediction. Specifically, given multi-modal inputs X = IMG x TXT) comprising visual tokens x .ING and textual tokens x TXT, RIG learns to autore- gressively generate textual reasoning tokens Y, low-level action tokens A, and visual prediction tokens P:
(Y, A) J(X),
X = IMG TXT
3

---

Published as a conference paper at ICLR 2026
90: Refined MineRL
S1: Vision-Action
S2: Vision-Reasoning
S3: Vision-Reviewing
S4: Temporal Alignment
Dotaset
Poley
Pollyy
RIG- basic
Policy
RIG-basic
USA
Dremn
reason
Action Text
Action Text
Action Text
Action Dict
("camera"" Boxfow, hight)
forward" Discrete(?)
X
X
reason
Redinwer
Toxoualioo
front of you
3. forward...
2. Forward to get cleser
I reaвon
at IMG
Wat!! Let's re-obrerva..
revise iew
cameera: (-5, fonward...
Crosshair not aligned
Turn down the view...
camena: (10, 0)
Illustration the data collection pipeline (SO-S4). Note that at S3 Review-n, run the trained RIG-basic and policy model (STEVE-1 1 [Lifshitz et al. 2023)) in parallel, keeping instances where R RIG-basic performs poorly compared to STEVE-1.
Figure 2:
The model is trained in an end-to-end manner using only cross-entropy loss:
C
log Po(a: [ee)).
where Po(: denotes the conditional probability distribution parameterized by the weights 0 of RIG.
3.2 REASONING WITHOUT IMAGINATION
Our primary goal is to develop synergized model capable of simultaneously generating textual reasoning, precise low-level actions, and visual outcome predictions. Since existing agents mainly produce actions, existing accessible datasets typically lack comprehensive trajectories containing all these elements. Therefore, we propose a progressive data collection strategy to gradually enrich these elements in accessible agentic trajectories. Inspired by the recent success of vision-language models (OpenAl, 2023) that can conduct chain-of-thought (CoT) reasoning given images, our first step is to add reasoning into the action-image trajectories using VLMs to obtain RIG-basic that can conduct reasoning before action.
Data Collection (SO-S2). As shown in Figure 2 we first refine or collect data from relabeled human play trajectories (SO) and specialized policies (S1), unify their formats, and add reasoning contents before each action (S2). The details are below:
SO (Refined MineRL-VO): We use trajectories from MineRL-VO (Guss et al., 2019) and quantize the camera actions of the original trajectory into discrete 5-degree intervals and then represent them as textual tokens. All other discrete low-level actions retain their original semantic labels.
S1 (Vision-Action, 446K): We use a pretrained policy, STEVE-1 (Lifshitz et al. 2023), to col- lect high-resolution (384x384) image-action pairs and ensure precise visual-action alignment for learning low-level control.
$2 VisionnReasonnig, 200K): integrate reasoning in the original trajectories, we employ GPT- 4o as Reasoner to annotate explicit textual rationales conditioned on visual observations IMG and the corresponding low-level actions A, formed as Y Reasoner(x AA).
All these trajectories rigorously filtered based task success, diversity across environment seeds, and manual validation of reasoning quality. We train RIG-basic using datasets obtained from SO, S1, and S2.
Reasoning without Imagination. After training on datasets (SO, S1, S2), the resulting model, RIG- basic naturally supports multi-round interactions with the environment. As shown in Figure 3
4

---

Published as a conference paper at ICLR 2026
Conversation Flow
Y1 "loak around for free...
"mine logs..." "tum another tree...
"A birch front of you...
X
X1
X
P i+n
dec
IMG token
dec
TXT token
<Imagine:>
Basic Reason
Lookahead Reason
Dream
X
Y2 The tree ahead... Agi camera:[-5,-100 forward
Yi+1 : the tree...
A[e1 attack
camera: (5,- -10], A441 left, forward,
Figure 3: Inference process in RIG. RIG follows structured conversation flow through multi-turn interactions. consistently uses the fixed word Imagine : to clearly separate internally imagined scenarios from real observations, thereby guiding coherent reasoning, action prediction, and visual imagination.
each step, it autoregressively generates textual CoT reasoning Y, low-level actions A, and action outcomes P:
A411, P(11) & (X,, Y, A.).
(3)
This unified approach achieves significantly better generalization than traditional methods, requiring substantially fewer training samples (Figure
3.3 LOOKAHEAD REASONING
Although RIG-basic demonstrates strong baseline performance, the reasoning is still purely based on the perceptual history and does not fully exploit the generative imagination capabilities. To address this, further augment our datasets with reflective reviewing annotations in stages 3 and (S3 and S4 in Figure 2). to endow the model with the ability to conduct lookahead reasoning, i.e. internally simulate imagined trajectories first, and then take actions after reviewing the predicted future outcomes.
Data Collection (S3-S4). We collect reflective annotations and temporal alignment data through the following stages:
S3 Resien-Reviewing, 27K): We use a state-wise advantage filter where we only retain initial states for which STEVE-1 achieves higher expected return than the current RIG-basic. It avoids using a weaker policy as positive globally and ensures the positive/negative trajectories are locally comparable under identical initial conditions. We generate S3 as follows:
Negative trajectory: Generated by the previously trained RIG-basic model, yielding subopti- mal outcomes X ,Y-,A-
Positive trajectory: Generated by the anperior-performing policy STEVE-1 Lifshitz et al. 2023). yielding optimal outcomes X+, A+.
We then adopt GPT-4o as a Reviewer to explicitly compare these parallel trajectories and gener- ate refined reasoning: Y Reviewer( X- Y-, A-, A+), so that we get corrective reasoning annotations:
= (Y-, "Wait! Let's re-observe..."" Y+
This reflection annotation significantly enhances the ability of the model to review and correct reasoning mistakes.
5

---

Published as a conference paper at ICLR 2026
2200
100
2101h
Jarvis-1
Training Data Duration
2000h
2000h
Number Samples
BO
2000
1962h
MineDreamer
60
1800
40
400
20
200
0
500
1000
1500
2000
2500
3000
MinDDissmer STEVE
# Iterations
Models
Figure 4: Performance and data-efficiency comparison. RIG-basic significantly outperforms other baselines with higher sample efficiency and achieves superior performance using only 111 hours of training data (42h SO MineRL-VO and 69h S1-S4). MineDreamer (Zhou et al. 2024b),
hybrid-system model, separately trains visual generation model (139 hours) but also relies
VPT for the policy model, increasing total data requirements. Duration of VPT (Baker et al.
2022
reflects only the IDM data used, measured as video frames, while STEVE-1 Lifshitz et 2023
and Jarvis-1 (Wang et al. 2023b) also leverage the VPT dataset.
S4 (Temporal Alignment, 38K): We further generate multi-step imagined visual predictions (P) and explicitly align them with observed ground-truth truut visual tokens (a (a)) enhance long-horizon stability: P++1 + 2441
IMG
Lookahead Reasoning with Imagination. Training on datasets from stages 3 and 4 produces RIG- lookahead, model that performs reasoning conditioned on imagined futures. Stage employs Rejection Sampling Fine-tuning (RFT) to enhance reasoning through model-generatee rollouts. We apply RFT embodied agents by leveraging joint reasoning and visual generation, which enables self- prediceedition of future states, previously infeasible due to the lack of visual prediction. Only the positive trajectory Y optimized, while negative Y is excluded from loss, encouraging better self-correction.
RIG-lookahead simulates "dream trajectories" before acting. As shown in Figure 3 imagined steps are marked with fixed token "< "<Image g >> to distinguish from observations, allowing decisions to be refined by looking n steps ahead:
(Y 4 Pit1 E (X,, P.+1, Y1.,...,, Pitno Y.-)).
(5)
This lookahead mechanism enables internal review and correction, reducing trial-and-error interac- tions and enhancing decision robustness in complex embodied tasks, as shown in Figure A3
4 EXPERIMENTS
We conduct comprehensive experiments to validate the effectiveness of RIG across diverse tasks, focusing on data efficiency, scalability, and the benefits of integrating generation, reasoning, and lookahead, Evaluations on embodied tasks are performed under both Manual (hand-only) and Tool (e.g., iron pickaxe) to assess performance in varied embodied scenarios. More details about metrics are listed in Appendix A.3 and detailed qualitative studies are listed in Appendix A.8
4.1 EXPERIMENT SETUP
Embodied tasks. Six tasks set up in MineRL et al. (2019) (Collect: Wood/Seeds/Dirt; Explore: Dig/Explore/Towel under Manual (bare-hand) and Tool (iron tools) settings; success and sample definitions follow Appendix A.3 Training and evaluation seeds are disjoint.
Language & generation metrics. Reasoning and Understanding blind- gradddd; Score-Static is STEVE-2IK static QA, for generation, we report FID (lower is better) and PSNR (higher is better), Appendix A.3.

---

Published as a conference paper at ICLR 2026
Understanding
Num. Samples
2033
(Browesd)
39.4
Scoee
53 75
PS
Score Nos.
94.3
126.3
301
STDVE-3
Inder1
MovDroami
MGG DAs
RoG nokahed
OPSG
6SC bask
tadlateed
(aaV ILaVA-Hame Kaaa 3-28
GPEAD
Figure 5: Comparison across embodied, generation, and VQA/Reasoning. RIG-basic equips reasoning. RIG-lookahead further adds reviewing.
Transay Sea ahy Taurher Surplee
Frention Sokeedlyy Sarrpiees
ware Srokeliii rnlie Sarestes
Hew.a
H0nue1
309
21533
Nf Sherriee
30.00
69h
27.00
10.33
Triiiin
#l Slepe
Taining Accurssy
Hexu
180
13.08
60.56
89h
13.9h
300
Traldog 880
#1 Bkgo
Figure 6: Scalability over training data, iterations, and lookahead steps. Shaded areas denote variance. The 42h MineRL-VO pretraining is excluded from the 111h total; the training-ratio panel counts data before lookahead.
Base model. RIG builds on J.nus-1.3B (Chen et al., 2025 with 4.096 context. SigLIP-L/16- 384 (Zhai et al., 2023) encodes images; a VQ tokenizer (Sun et al.) (024) provides discrete visual IDs for the unified Transformer. Training uses sequence packing and mixed data types on XTuner- lite (Contributors 2023).
4.2 DATA EFFICIENCY
As shown in Figure 4 RIG significantly surpasses other methods in terms of collected samples per iteration while requiring drastically lower total training time. Notably, RIG achieves superior performance using only 111 hours of total data collection, considerably less than other baselines (VPT: 1962h, MineDreamer: 2101h (139h + VPT), STEVE-1 and Jarvis-1: nearly 2000h).
4.3
MAIN RESULTS
We first evaluate the data efficiency of RIG against prior approaches (VPT Baker et al. 2022 STEVE-1 (Lifshitz et al. 2023), Jarvis-1 (Wang al. 2023b), and MineDreamer (Zhou et al.

---

Published as a conference paper at ICLR 2026
Capabilities
Number of Samples
Accuracy (%)
Action
Gen.
Lookahead
wood
grass
dist
avg.
Dig
Explore
avg.
Manual (ID 0-4)
0
7.9
6.2
8.9
7.7
+0.01
9.1
11.7
4.4
8.4
+0.0
11.0
16.5
12.1
13.2
+5.5
36.8
41.8
30.3
+21.9
2
17.3
24.5
22.5
21.4
+13.8
34.2
31.8
31.8
34.6
+26.2
22.2
45.9
38.7
35.6
+27.9
29.2
65.2
37.9
44.1
+35.7
28.3
137.5
74.8
80.2
+72.5
65.8
84.2
88.7
79.6
471.2
Tool (ID 4)
0
24.6
33.1
42.4
33.4
+0.0
17.9
11.7
8.2
12.6
+0.0
25.8
29.9
48.4
34.7
27.2
36.8
41.8
35.3
2
26.9
49.9
51.0
42.6
+9.2
24.2
31.8
29.8
28.6
+16.0
79.4
115.3
108.6
101.1
100.4
128.7
295.6
315.5
246.6
+213.2
95.4
84.2
102.8
94.1
481.5
Table I: Ablation study on embodied tasks under different capability settings. We compare different combinations of Action, Generation (Gen.), Reasoning (Reason), and Reviewing (Review). The table is divided into two groups: Manual (ID 0-4) and Tool (ID 0-4). The "Num." column represents number of completed collecting tasks (wood, grass, dirt), while "Acc." denotes success rate of exploration tasks. The columns "avg." " is the average performance. For both metrics, we report the absolute values, along with the improvement (+x) over the baseline (ID 0 for Manual and Tool).
Capubilities
Generation
Understanding
Reasoning
ID
Action
Gen.
Reason
Lookahead
FID
PSNR
Score-Stc.t
Score-Env.
Score-Eav.
214.5
16.4
225.6
16.3
/
/
1
/
156.5
17.9
9.4
7.3
/
1
/
77.6
18.4
8.5
Table 2: Ablation study on Generation, Understanding, and Reasoning performance. We compare different combinations of Action, Generation (Gen.) Reasoning (Reason), and Looka- head capabilities. "Score-Env." represents the environment-specife, evaluation score from online understanding testing, while "Score-Env." denotes reasoning-specifee evaluation. "Score-Ste." is computed on the static dataset STEVE-2IK Zhao et al. (2023), and "FID" "PPSRR" measure image generation quality.
2024b)). We then benchmark its performance across three core task categories: Embodied Tasks, Generation Tasks, and Understanding & Reasoning Tasks. For more qualitative analysis, please refer the case study in Appendix
Performance in Embodied Tasks. As shown in Figure RIG-basic have surpassed all other baselines with 93.4% accuracy and 101.1 collected samples. RIG-lookahead (extended from RIG- basic) produce even greater progress than RIG-basic, with the highest accuracy of 94.1 % and 246.6 collected samples. For extended analysis, we conduct additional scalability and ablation studies, as shown in Figure and Table
Performance in Generation and Understanding. RIG-lookahead achieves the best Minecraft- style generation quality among baselines (FID 77.6, PSNR 18.4) (see Appendix A.8), general VQA remains on par with Janus-1.33 (see Table A5).
4.4 SCALABILITY
We evaluate the scalability RIG along three key dimensions, training data ratio, iteration count, and inference steps, as illustrated in Figure 6
Training Scalability. RIG exhibits strong scalability with training data volume. Increasing training data from 10% to 100% dramatically improves performance, especially at the 20% training thresh- old, where accuracy jumps substantially (manual: 1..3%%+24.06. tool: 17.0% -..55%... Data diversity significantly increases at this point, allowing the agent encounter and adapt broader
8

---

Published as a conference paper at ICLR 2026
spectrum of complex scenarios. Beyond 20% data usage, the rate of accuracy improvement stabi- lizes, indicating the training paradigm reaches a steady state and that the agent's 's action diversity nears its upper bound. With full training data (100%), RIG achieves superior results, outperforming
existing approaches such as VPT (Baker et al. 2022) and Jarvis-1 (Wang et al., (023b) in accuracy 2023 in task collection efficiency, reaching 101.13
and even surpassing STEVE-1 (Lifshitz al.
collected tasks and 93.35% accuracy. Notably, these results are attained purely through forward reasoning, without lookahead, suggesting substantial untapped potential for further enhancement by incorporating advanced reasoning mechanisms,
Iteration Scalability, RIG demonstrates robust performance growth over iterations. Under standard forward inference, task collection grows consistently from 9.25 samples at iteration 200 to 101.13 at iteration 3000, particularly pronounced in tool-assisted tasks, showing rapid convergence due to effective data utilization and stable trajectory patterns. However, variance, illustrated by shaded ar- eas, tends to increase with iterations, reflecting longer and more diverse trajectories that introduce complexity and fluctuation. Tasks involving exploration, which inherently contain more combina- torial subtasks (e.g., material gathering followed by building structures), show larger variance and complexity over time. It potentially highlights the model's adaptive response to increasingly diverse scenarios.
Inference Scalability. The results from different lookahead steps demonstrate significant benefits from lookahead reasoning. Evaluating from the baseline at 3000 iterations, increasing steps (gen- erating "dream trajectories") substantially improves performance. Task collection metrics exhibit rapid initial improvement and relatively low variance up to four steps, indicating accurate and stable trajectory predictions. However, variance increases at five steps, suggesting accumulated prediction errors or hallucinations become more prominent. For accuracy metrics, tool-assisted tasks maintain high performance (peaking at 94.12% at steps), with slight decrease afterward due to ceiling effects and increased prediction uncertainty. Conversely, manual tasks show consistent performance improvement through stepwise lookahead, significantly benefiting from iterative reasoning. reaching a peak of 79.58% accuracy at 5 steps.
4.5 ABLATIONS STUDY
Embodied tasks. Adding generation improves targeting and exploration stability; adding reason- ing reduces redundant actions and enables goal-directed plans; adding lookahead yields the largest gains with minimal extra data (27K). The full model the best sample efficiency and success (see Table I). As shown in Table [ the synergy of Ac- tion, Generation, Basic Reasoning and Lookahead Reasoning leads to the most robust performance, which enables structured learning and improves short-term decisions and long- Long-horizon task comple- tion. the Manual setting (ID 4), there are significant gains in sample collection (from 7.7 to 80.2, +72.5) and accuracy (from 8.4% to 79.6%. +71.2). In the Tool setting, the impact is even greater, with sample counts rising from 33.4 to 246.6 (+213.2) and accuracy from 12.6% to 94.1% 1% (+81.5), highlighting the effectiveness of the unified framework.
delivers
Generation quality. Action-only slightly worsens FID, but combining action with reasoning and lookahead markedly improves FID/PSNR (e.g.. 156.5 FID; 17.9->18.4 PSNR), confirming that structured reasoning stabilizes the visual predictor (see Table 2). By jointly optimizing gener- ation, reasoning, and reviewing, RIG achieves the best trade-off between action prediction, visual understanding, and environmental reasoning. The best model (II) 4) shows that lookahead reason- ing enhances decision-making. improving sample efficiency and interaction robustness. This under- scores the benefit of integrating multiple modalities for coherent perception and action. As shown in Table 2 enabling lookahead reasoning substantially enhances mnngeggeneration quality. RIG- lookahead (ID 4) attains the lowest FID (77.6) and highest PSNR (18.4) (884), significantly surpassing variants without lookahead reasoning (e.g.. ID 3, FID: 156.5, PSNR: 17.9).
4.6
CASE STUDY
Figure 7 shows detailed case study between RIG-lookahead with the latest GPT-4o image generation upgraded. It illustrates that GPT-40 misjudges the distance, issues an at attack command, and ends up "attacking air". In contrast, RIG-lookahead realizes the trunk is out of reach by imagination, adjusts plans, and finally succeeds. Note that we compare RIG with GPT-4o under identical prompts.
9

---

Published as a conference paper at ICLR 2026
GPT-4o frequently issues invalid attack actions and stalls, with valid-action rate of only 11 11, while ours achieves 98%.
The training process involves SI to S3, where imagination is predicted but used only as auxiliary target, without feedback loop. In S4, we apply an autoregressive method over successive imagined frames, rolling out the same action path in the real environment and training the paired traces. This leads to RIG-lookahead, which produces stable predictions (Figure 7 that enhance true look- ahead reasoning and improve performance. More studies are listed Appendix A.8
Current IMG
Prompt
You
Minecraft simulator. will give you the current frame and the task you need to do. You need to help me predict the best action and the next frame of the action. Action space (forward, back...).
Correct
Wrong
Current frame: <IMG>, current task: chop a tree. Predict: next action:..., next frame: <IMG>
Key
Reasoning
Gen IMG
Next IMG
RIG-lookuhead
Reasoning:
In first frame, tree trunk directly ahead. There's need adjust the camera. Simply begin collecting wood by Bltacking the tred.
Wait! Let's re-observe the imagination. the tree remains unchanged,
1. The tree is not close enoughi so you cannot attack immediately.
2. The tree on the right closer and more suitable immediate tree-
cutting.
3. From the terrain point of view, you attack the tree flat ground by slightly adjusting to the right. Next action: forward, sprint, camera:00, [0,10
<Imagine>
attack...
Wait! Let's re-observe camera:
atlack
Imagine before Reason
Reason
GPT-4o Image Generation Updated Version
According the current screen, you facing tree, very close the target, and the task is "chop tree". The best strategy should be to get close to the trunk and start attacking.
Prediction:
Next action: attack (attack trunk with your hands)
Next frame speculation:
Cracks will appear the trunk (animation effect of attacking wood)
Figure 7: Case study on lookahead ability. This ability crucial in embodied tasks such as fault review, helping agents accurately estimate distances and take correct actions in open- speldworld environments.
5 CONCLUSION
This paper introduces RIG, an end-to-end Generalist policy that integrates Reasoning and Imagination with superior adaptability and robustness in open-world environments. RIG unifies visual, action, and textual reasoning within single autoregressive Transformer and can plan and rea- son by looking ahead with dreamt trajectories to further improve its robustness. RIG achieves new state-of-the-art performance across embodied tasks, image generation, and reasoning, with higher sample efficiency and better generalization. RIG also exhibits higher scalability with training and test-time compute. As cases shown in Appendix(A.8) research on synergizing reasoning and imagination in embodied agents.
hope the results of RIG could inspire future
10

---

Published as a conference paper at ICLR 2026
ACKNOWLEDGEMENTS
We thank Jiangning Liu for insightful discussions on evaluation and for data support. We also thank Yuzhe Gu for valuable discussions and helpful comments on earlier drafts. This work was supported by the Fundamental Research Funds for the Central Universities (No. 226-2025-00167), and the National Natural Science Foundation of China (No. 62576308).
Ethics statement. We adhere to the ICLR Code of Ethics and confirm that all authors have read and complied with it. Our experiments are conducted entirely in a simulated Minecraft environ- ment using publicly available datasets and policies (e.g., MineRL-VO, model-generated rollouts. No human subjects, personally identifiable information, or sensitive at- tributes are collected; therefore IRB approval was not required. Reasoning annotations and blind grading use GPT-4o under dataset and API terms; we release the exact prompts, filtering rules (e.g., legality weighting), and sampling criteria document potential annotator/model bias. We discuss model limitations (e.g., hallucinated "imagination" frames) and restrict claims to simulation. To mit- igate dual-use risks (e.g. unsafe autonomous control), we do not deploy to physical robots, provide research-only artifacts where applicable, and include safety notes in the repository. All third-pan -arrt assets are cited and used within their licenses. We also report compute and data usage in the appendix and will provide an estimated carbon footprint to support environmentally responsible research.
and
Reproducibility statement. We prioritize reproducibility through clear documentation and arti- fact release. The training/evaluation protocol (tasks, metries, seeds, success criteria) is specified in Appendix A..3 with full seed lists and episode counts in the appendix. Data construction SO-S4 (sources, prompts, filters, reviewer settings) is detailed in the main text and supplementary. We release the exact prompts and scripts. Model and optimization details (backbone, context length, vi- sion encoders/tokenizerr. hyperparameters) appear $4.1 and the supplementary materials, with all configs provided as YAML files. We supply evaluation harnesses for embodied control, generation (FID/PSNR), and VQA, including Docker/Conda environment files, fixed random seeds, and com- mit hashes. Anonymous links in the supplemental materials include: (1) inference/training code, (2) checkpoints for basic and lookahead, (3) ablation/plot scripts to regenerate all figures/tables. Together, these materials enable end-to-end replication of results and verification of claims.
REFERENCES
Niket Agarwal, Arslan Ali, Maciej Bala, Yogesh Balaji, Erik Barker, Tiffany Cai, Prithvijit Chat- topadhyay, Yongxin Chen, Yin Cui, Yifan Ding, et al. Cosmos world foundation model platform for physical ai. arXiv preprint arXiv:2501 03555,, 2025.
Bowen Baker, 1lge Akkaya, Peter Zhokhov, Joost Huizinga, Jie Tang, Adrien Ecoffet, Brandon Houghton, Raul Sampedro, and Jeff Clune. Video pretraining (vpt): Learning to act by watching unlabeled online videos. arXiv preprint arXiv: 1xxiv22206.11995 2022.
Anthony Brohan, Noah Brown, Justice Carbajal, Yevgen Chebotar, Joseph Dabis, Chelsea Finn, Keerthana Gopalakrishnan, Karol Hausman, Alex Herzog, Jasmine Hsu, et al. RT-1: Robotics transformer for real-world control at scale. In RSS, 2023.
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D) Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. Advances in neural information processing systems, 33:1877-1901, 2020.
Shaofei Cai, Zihao Wang, Xiaojian Ma, Anji Liu, and Yitao Liang. world multi-task control through goal-aware representation learning and adaptive horizon prediction. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13734 13744, 2023a.
Shaofei Cai,, Bowei Zhang, Zihao Wang, Xiaojian Ma, Anji Liu, and Yitao Liang. Groot: Learning to follow instructions by watching gameplay videos. arXiv preprint 23.008235, 2023b.
Xiaokang Chen, Zhiyu Wu, Xingchao Liu, Zizheng Pan, Wen Liu, Zhenda Xie, Xingkai Yu, and Chong Ruan. Janus-pro: Unified multimodal understanding and generation with data and model scaling, arXiv preprint arXiv:2501.178 2025.

---

Published as a conference paper at ICLR 2026
Ethan Chern, Jiadi Su, Yan Ma, and Pengfei Liu. Anole: An open, autoregressive, native large mul- timodal models for interleaved image-text generation. arXiv preprint arXXv:24407.06135, 2024.
XTuner Contributors. Xtuner: A toolkit for efficiently fine-tuning llm. https: com/ MnternLM/xtunee 2023.
Wenliang Dai, Junnan Li, Dongxu Li, Anthony Meng Huat Tiong, Junqi Zhao, Weisheng Wang, Boyang Li, Pascale Fung, and Steven Hoi. Instructblip: Towards general-purpose vision-language models with instruction tuning, 2023.
Lijie Fan, Tianhong Li, Siyang Qin, Yuanzhen Li, Chen Sun, Michael Rubinstein, Deqing Sun, Kaiming He, and Yonglong Tian. Fluid: Scaling autoregressive text-to-image generative models with continuous tokens. arXiv preprint vrXiv:2410.13861. 2024.
Linxi Fan, Guanzhi Wang. Yunfan Jiang, Ajay Mandlekar, Yuncong Yang. Haoyi Zhu, Andrew Tang, De-An Huang. Yuke Zhu, and Anima Anandkumar, Minedojo: Building open-ended embodied agents with internet-scale knowledge. Advances in Neural Information Processing Systems, 35:
18343-1888 18362, 2022.
William Guss, Brandon Houghton, Nicholay Topin, Phillip Wang, Cayden Codel, Manuela Veloso, and Ruslan Salakhutdinov. Minerl: large-scale dataset of minecraft demonstrations. In Proceedings of the 28th International Joint Conference on Artificial Intelligence, pp. 2442- 2448, 2019.
Danijar Hafner, Timothy Lillicrap, Jimmy Ba, and Mohammad Norouzi. Dream to control: Learning behaviors by latent imagination. In ICLR, 2020.
Danijar Hafner, Timothy Lillicrap, Mohammad Norouzi, and Jimmy Ba. Mastering atari with dis- crete world models. In ICLR, 2021.
Danijar Hafner, Jurgis Pasukonis, Jimmy Ba, and Timothy Lillicrap. Mastering diverse domains through world models. arXiv preprint arXiv:2301.04104 2023.
Nicklas Hansen, Xiaolong Wang, and Hao Su. Temporal difference learning for model predictive control. In /CML, 2022.
Matthew Johnson, Katja Hofmann, Tim Hutton, and David Bignell. The malmo platform for artifi- cial intelligence experimentation. In Ijcai, pp. 4246-4247, 2016.
Lukasz Kaiser, Mohammad Babaeizadeh, Piotr Milos, Blazej Osinski, Roy H Campbell, Konrad Czechowski, Dumitru Erhan, Chelsea Finn, Piotr Kozakowski, Sergey Levine, al. Model-based reinforcement learning for atari. In ICLR, 2020.
Shalev Lifshitz, Keiran Paster, Harris Chan, Jimmy Ba, and Sheila Mcllraith. Steve-1: A generative model for text-to-behaving in minecraft. arXiv preprint arXiv:23366. 009377 2023.
Jessy Lin, Yuqing Du, Olivia Watkins, Danijar Hafner, Pieter Abbeel, Dan Klein, and Anea Dragan. Learning to model the world with language. 2023.
Hao Liu, Wilson Yan, Matei Zaharia, and Pieter Abbeel. World model on million-length video and language with blockwise ringattention. arXiv preprint arXiv:2402.08266. 2024a.
Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee. Improved baselines with visual instruction tuning. In Proceedings of the IEEE/CVF Conference Computer Vision and Patlern Recogni- tion, pp. 26296-26333 2024b.
Haotian Liu, Chumyuan Li. Qingyang Wu, and Yong Jae Lee. Visual instruction tuning, Advances in neural information processing systems, 36, 2024c.
Pan Lu, Baolin Peng, Hao Cheng, Michel Galley, Kai-Wei Chang, Ying Nian Wu, Song-Chur Zhu, and Jianfeng Gao. Chameleon: Plug-and-play compositional reasoning with large language mod- els. arXiv preprint 2rXiv:2304.09842. 2023.
12

---

Published as a conference paper at ICLR 2026
Zhuoyan Luo, Fengyuan Shi, Yixiao Ge. Yujiu Yang, Limin Wang, and Ying Shan. Open-magven An open-source project toward democratizing auto-regressive visual generation. arXiv preprint arXiv:2409.044410 2024.
Russell Mendonca, Shikhar Bahl, and Deepak Pathak. Structured world models from human videos. In RSS, 2023.
Vincent Micheli, Eloi Alonso, and François Fleuret. Transformers are sample efficient world models. In ICIR, 2023.
Kolby Nottingham, Prithviraj Ammanabrolu, Alane Suhr, Yejin Choi, Hannaneh Hajishirzi, Sameer Singh, and Roy Fox. Do embodied agents dream of pixelated sheep?: Embodied decision making using language guided world modelling. arXiv preprint, 2023.
Junhyuk Oh, Xiaoxiao Guo, Honglak Lee, Richard Lewis, and Satinder Singh. Action-conditional video prediction using deep networks in atari games. In NewrlPS, 2015.
OpenAl. Gpt-Av(ision) system card. 2023.
URL https://openai. htt.peaaaai.mmmeeeerrttiiiii
get 10:-4v- Bv-system-cas card
Scott Reed, Konrad Zolna, Emilio Parisotto, Sergio Gomez Colmenarejo, Alexander Novikov, Gabriel Barth-Maron, Mai Gimenez, Yury Sulsky, Jackie Kay, Jost Tobias Springenberg, et al. A generalist agent. arXiv preprint arXiv:2205.06177 2022.
Julian Schrittwieser, Toannis Antonoglou, Thomas Hubert, Karen Simonyan, Laurent Sifre, Simon Schmitt, Arthur Guez, Edward Lockhart, Demis Hassabis, Thore Graepel, al. Mastering atari, go, chess and shogi by planning with a learned model. Nature, 588(7839)6004-00,, 2020.
Peize Sun, Yi Jiang, Shoufa Chen, Shilong Zhang, Bingyue Peng, Ping Luo, and Zehuan Yuan. Autoregressive model beats diffusion: Llama for scalable image generation. arXiv preprint 20X2:22406.006555. 2024.
Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaci, Niko- lay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. Llama 2: Open founda- tion and fine-tuned chat models. arXiv preprint arXiv:2307.09288 2023.
Guanzhi Wang. Yuqi Xie, Yunfan Jiang, Ajay Mandlekar, Chaowei Xiao, Yuke Zhu, Linxi Fan, and Anima Anandkumar. Voyager: An open-ended embodied agent with large language models. arXiv preprint arXiv:2305. 16291, 2023a.
Xinlong Wang, Xiaosong Zhang, Zhengxiong Luo, Quan Sun, Yufeng Cui, Jinsheng Wang, Fan Zhang, Yueze Wang, Zhen Li, Qiying Yu, et al. Emu3: Next-token prediction is all you need. arXiv preprint arXiv:2409, 18869, 2024.
Zihao Wang, Shaofei Cai, Anji Liu, Yonggang Jin, Jinbing Hou, Bowei Zhang, Haowei Lin, Zhaofeng He, Zilong Zheng, Yaodong Yang, et al. Jarvis-l: Open-world multi-task agents with memory-augmented multimodal language models. arXiv preprint S0V:23311.05990 2023b.
Zihao Wang, Shaofei Cai, Anji Liu, Xiaojian Ma and Yitao Liang. Describe, explain, plan and select: Interactive planning with large language models enables open-world multi-task agents. arXiv preprint a120:2200.001560, 2023c.
Jialong Wu, Haoyu Ma, Chaoyi Deng, and Mingsheng Long. Pre-training contextualized world models with in-the-wild videos for reinforcement learning. In NeurIPS, 2023.
Yecheng Wu, Zhuoyang Zhang. Junyu Chen, Haotian Tang, Dacheng Li, Yunhao Fang, Ligeng Zhu, Enze Xie, Hongxu Yin, Li Yi, et al. Vila-u: unified foundation model integrating visual understanding and generation. arXiv preprint ar arXiv:2409 209.2499,00422, 2024.
Jinheng Xie, Weijia Mao, Zechen Bai, David Junhao Zhang, Weihao Wang. Kevin Qinghong Lin, Yuchao Gu, Zhijie Chen, Zhenheng Yang, and Mike Zheng Shou. Show-o: One single transformer to unify multimodal understanding and generation. arXiv preprint a408v:2408. /2528, 2024.
13

---

Published as a conference paper at ICLR 2026
An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao. Chengen Huang, Chenxu Lv, et al. Qwen3 technical report. arXiv preprint S0Xiv:2500500388,, 2025.
Lili Yu, Bowen Shi, Ramakanth Pasunuru, Benjamin Muller, Olga Golovneva, Tianlu Wang, Arun Babu, Binh Tang, Brian Karrer, Shelly Sheynin, et al. Scaling autoregressive multi-modal models: Pretraining and instruction tuning. arXiv preprint arXiv:2309.02591, 2(3):3, 2023.
Haoqi Yuan, Chi Zhang, Hongcheng Wang. Feiyang Xie, Penglin Cai, Hao Dong, and Zongqing Lu. Plan4me: Skill reinforcement learning and planning for open-world minecraft tasks. arXiv preprint arXiv:2303. 16563, 2023.
Xiaohua Zhai, Basil Mustafa, Alexander Kolesnikov, and Lucas Beyer. Sigmoid loss for language image pre-training In Proceedings of the IEEE/CVF International Conference Computer Vision, pp. 11975-11986, 2023.
Chi Zhang, Penglin Cai, Yuhui Fu, Haoqi Yuan, and Zongqing Lu. Creative agents: Empowering agents with imagination for creative tasks. arXiv preprint arXiv:2312.02519, 2023.
Zhonghan Zhao, Wenhao Chai, Xuan Wang, Li Boyi, Shengyu Hao, Shidong Cao, Tian Ye, Jenq- Neng Hwang, and Gaoang Wang. See and think: Embodied agent in virtual environment. arXiv preprint arXiv:2311.15200 2023.
Zhonghan Zhao, Kewei Chen, Dongxu Guo, Wenhao Chai, Tian Ye, Yanting Zhang, and Gaoang Wang. Hierarchical auto-organizing system for open-ended multi-agent navigation. arXiv preprint 0882222403.08282. 2024a.
Zhonghan Zhao, Ke Ma. Wenhao Chai, Xuan Wang, Kewei Chen, Dongxu Guo, Yanting Zhang, Hongwei Wang, and Gaoang Wang. we really need complex agent system? distill embodied agent into a single model. arXiv preprint arXiv:2404.04619 2024b.
Sipeng Zheng, Jiazheng Liu, Yicheng Feng, and Zongging Lu. Steve-eye: Equipping Ilm-based embodied agents with visual perception in open worlds. arXiv preprint arXiv:2310.1 13255, 2023.
Chunting Zhou, Lili Yu, Arun Babu, Kushal Tirumala, Michihiro Yasunaga, Leonid Shamis, Jacob Kahn, Xuezhe Ma, Luke Zettlemoyer, and Omer Levy. Transfusion: Predict the next token and diffuse images with one mudti-modal model. arXiv preprint arXiv:2408. 1009, 2024a.
Enshen Zhou, Yiran Qin, Zhenfei Yin, Yuzhou Huang, Ruimao Zhang, Lu Sheng, Yu Qiao, and Jing Shao. Minedreamer: Learning to follow instructions via chain ioaaii-of-mmaainattinn for simulated- world control. arXiv preprint arXiv:2403. 12037, 2024b.
14

---

Published as a conference paper at ICLR 2026
A APPENDIX
THE USE OF LARGE LANGUAGE MODELS (LLMS)
Our work develops multimodal LLM-based world model. Concretely, RIG builds 1.3B- parameter LLM backbone with unified visual-text decoding; we train and improve using Su- pervised Finetuning (SFT) and Rejection Sampling Fine-Tuning (RFT) to enhance reasoning and lookahead capabilities. In addition to the backbone itself, we employ an external LLM (GPT-40) only for technical purposes, like reasoner and baseline in the research pipeline: generating rea- soning annotations and conducting blind grading of text-only outputs under the dataset/APl terms. Prompts, legality filters, and sampling criteria are documented and released aid auditability.
Beyond these technical roles, LLMs did not play significant part in research ideation, experi- mental design, or manuscript writing, the authors conceived the study, designed/evaluuated experi- ments, analyzed results, and wrote the paper. Any automated assistance, if present, was limited to -unsauuttant copy-editing/L.fTee
linting. The authors take full responsibility for all contents, including verifying any aachine-generatel intermediate artifacts, and acknowledge that LLMs are not eligible for authorship. This disclosure follows the ICLR policy on LLM usage and research integrity.
The appendix is organized as follows:
Inference Pipeline (Appendix A illustrates how RIG performs end-to-end inference: generat- ing textual reasoning, imagined visual rollouts, and executable actions, with explicit use of the < <Imagine:> token to support self-review and temporal consistency. Meanwhile, we compare the inference costs STEVE-1 and MineDreamer.
Training Pipeline (Appendix A.2) presents our multi-stage training framework, including offline supervised learning, GPT-4o-ban based reasoning and review relabeling, and imagination-gen -rroundedd alignment strategies that enable lookahead-bases decision-making. Meanwhile, we list the train-
ing costs of RIG.
Evaluation Protocol and Metrics (Appendix A.3 presents detailed evaluation protocal and met- rics.
Data Distribution (Appendix A.4 analyzes the diversity of embodied tasks within datasets, and illustrates how data volume scales with task complexity, from atomic skills like collection to composite ones like exploration and construction.
Component Comparison (Appendix A.5) offers systematic comparison with existing models, emphasizing RIG's unique capabilities in multimodal alignment, action granularity, and unified policy formulation without relying on task-specific modules.
Tokenizer and Base Model Selection (Appendix (A.6) explains our design choice of combining LlamaGen's 's VQ tokenizer with Janus as the vision-language foundation, offering a lightweight and effective setup for image-text grounding Minecraft-like settings.
General VQA Benchmark Result (Appendix A.7 compares RIG with state-of-the-arts on mul- timodal understanding benchmarks.
Qualitative Results and Case Study (Appendix A.8 showcases examples where RIG performs internal reasoning, detects failure cases via self-review, and corrects actions before execution. We further compare RIG GPT-4o (from reasoning and imagination comparison in Figure A5 to lookahead comparison in Figure 7 demonstrating that strong visual generation alone does not guarantee robust policy reasoning.
Multi-Modal Understanding Evaluation (Appendix A.9 evaluates RIG's embodied knowledge across diverse functional categories using the STEVE-216 QA benchmark, covering survival, crafting, entity understanding, and more.
Multi-turn Visual Reasoning Format (Appendix A.10 details our multi-round reasoning and imagination format, which supports fine-grained learning of vision-language-action alignment through step-by-step trajectory prediction.
Environment Details (Appendix A.11 describes experimental platform based on Min- eRL (Guss et al.] 2019), featuring low-level egocentric control and programmable environment setup for robust and reproducible embodied evaluation.
15

---

Published as a conference paper at ICLR 2026
Instruction
"Collect wood for crafting planks" "
History
"Forward, Jump
enc
<Imagin:>
Reason
Review
X n
dec
Reasoning
Action
ahead... Detour and get
"Left, Forward, Camera: (0, 10]"
View the trunk..
Figure A1: Detailed inference pipeline. RIG generates imagined visual states and corresponding reasoning to simulate multiple action trajectories, enabling self-review and corrective prediction.
A.1 INFERENCE PIPELINE
As illustrated in Figure AT RIG follows multimodal autoregressive generation process. Given current observations and the task, the model produces (i) textual reasoning, (ii) low-level actions, and (iii) visual predictions of future frames. These imagined states, denoted by the fixed token <Imagi i me >, are recursively fed back for internal reviewing and decision refinement. This mech- anism allows iterative planning without environmental interaction.
Inference costs. For typical control step. the policy without visual imagination (Reasoning Action) requires approximately 1 FLOPs, while the full RIG pipeline with imagination (Rea- soning Action + Prediction) requires about 6.38 x 10 FLOPs. As summarized in Table A1 the per-step FLOPs of STEVE-1 Lifshitz et al. (2023) about
10 whereas MineDreamer Zhou et al. (222bb) requires roughly 3 10TH FLOPs per step (6 10 for the controller and 10TH for the backend image generator). Thus, among agents with an explicit world model, our synergized RIG design remains two orders of magnitude cheaper than hybrid world-model RI. controller such as MinelDreamer, while providing integrated reasoning, imagination, and lookahead. Compared low-level policy like STEVE-1, RIG's per-step inference cost higher, but offers substantially stronger performance and richer capabilities (reasoning, visual imagination, and self-review) under 17x smaller environment interaction budget.
A.2 TRAINING PIPELINE
As illustrated in Figure A2 the training of RIG proceeds in four progressive stages:
16

---

Published as a conference paper at ICLR 2026
Method
Env.
Imag.
Reason,
Look.
Summary
Infer FLOPs
STEVE-1
~2000
Low-level policy
x 10
MineDreamer
~000
V
World-model RL
3 x 10'4
RIG-basic
111
V
World-model
x 10
RIG-lookahead (Ours)
111
4
World-model
x 10
"Env. h" counts environ- ment interaction used to train each agent (our 17x factor compares RIG's 111 h to STEVE-1 h).0 "Imag." " denotes explicit visual imagination / world model, "Reason." denotes textual reasoning review head, and "Look." denotes lookahead-brss basedd self-review. Infer FLOPs. are ap- proximate per-step costs of inference under our setup.
Table AI: Comparison between STEVE-1, MineDreamer, and RIG.
S0/S1. Offline Supervised Fine-tuning (SFT); The model learns to align the dream flow (model- generated predictions) with the real flow (observed data) through supervised learning. This phase improves visual state prediction quality, enhancing the accuracy of subsequent action decisions. Inpul: past frame, past action, task. Output: subtask, next action, next frame.
S2. Reasoning Relabeling: A two-step process enhances decision quality. (1) An environment- based evaluator filters high-quality trajectories. (2) GPT-4o acts a Reviewer to generate explicit reasoning traces and refined labels. Inpul: past frame, past action, task. Output: reasoning, next action, optionally lookahead reason- ing, next frame.
The trained model interacts in the environment, and an evaluator filters poor trajectories. GPT-4o as Reviewer analyzes the imagined traces and relabels corrections for better trajectory quality, Inpul: past frame, past action, task, imagined frame (<Imagine:>) Outpul: lookahead reason- ing, corrected action, next frame.
S3. Review Relabeling:
imagined dream trajectory generated via the autoregressive model. The entire sequence is behavior-cloned into the real environment, enabling frame-by- frame alignment and relabeling via the Reasoner.
S4. Temporal Alignment:
Input: dream trace (states/actions). Output: real visual alignment, updated reasoning annotations.
Note that RIG trained on 111 hours of data by collecting, without reusing weights from STEVE-1 or its 2k hour corpus. We just focus solely on the final data used for training ensure direct metric.
Stages 0-2 are used to train RIG-basic, while Stages 3-4 further enhance RIG-lookahead with imagination-boe baased alignment and long-horizon correction.
Eq. for Stages 3-4 shows RIG-basic is co-trained ing without Imagination" means RIG-basic has not directly used imagination as input to looka- head. Stage 3 let GPT-4o prompted to generate coherent reasoning Y+ that: (1) analyzes the failure in A and then (2) justifies the correction leading to A+ X and X+ used in Stage 3 are all observation frames, X
The "Reason-
bad trial through RIG-basic interact with environment by action. Stage 4 is trained generated frame P and real interaction outcome frame X.
Reasoning before action makes RIG globally stabler on long horizons, yet occasional local mis- actions remain. STEVE-1's greedier traces supply those hard local positives.
Training costs. For the training setup reported in the paper, RIG-basic (without lookahead imag- ination) requires about 704 GPU hours to train 111 hours of interaction data, using 64 x A100 80GB GPUs. The additional lookahead stage for RIG-lookahead requires about 280 GPU hours on the same hardware.
A.3 EVALUATION PROTOCOL METRICS
17

---

Published as a conference paper at ICLR 2026
SO, S1. Offline SFT
Xa
X
X
Ao
A
Real Flow
Dream Flow
Po
Pi
P
S2./S3. Reason + Review
X
X
X
Ao
A
Real Flow
Reasoner / Reviewer
X
X1
X
S4. Temporal Align
Yo, Ao
Y, A
Real Flo
Dream Flow
Po
P1
P
SO, S1, S2
RIG basic +3, S4
RIG-lookahead
Figure A2: Training pipeline RIG. SO/SI pretrain the model by aligning real and imagined flows S2/S3 enhance reasoning and reviewing via GPT-4o relabeling. S4 aligns temporally predicted trajectories (dream flow) with environment anvroonment-rouunded grounded traces.
Aeteeammnnnc/A (AM)
Warrst
POPE: SEHDDenxt-irgg Viua2 (152,2555
Charrlee 1ea
Vg (T)
7R. 348
0.03
.0.0
vQJv (tem)
(3)
Alt
Show
Magine2
VQN2
3..77.. 2943 MASS, 1.10), 3:10, 11.31
4.01
T3rdfor
VQ (D), VALAU)
0.6
1110
Table A2: Comparison of various unified multimodal methods, categorized by their training ap- proach (Autoregressive and AR + Diffusion), detailing vision encoder type, parameter scale, vision generation quality (GenEval SD3 8B), multimodal understanding quality, and evaluation bench- marks.
Embodied tasks. tasks (Collect: Wood/Seeds/Dirt, Explore: Dig/Explore/Tower), (1) One itera- tion means one forward pass frame used. (2) Num. of samples stands for the number of collected blocks. (3) Accuracy refers to the success rate of completing tasks.
(4) Explore success: Dig z [7, 20]; Explore travel 2 300 blocks: Tower height 15. (5) Training seeds (e.g. 1-34, 701-706 for Wood) are listed in the appendix. (6) Evaluation always starts with random seeds for the random
scene. (7) Tool/Manual toggles iron tools to shorten bare-hand loops.
18

---

Published as a conference paper at ICLR 2026
2
Understanding
You are in a forest biome surrounded by mix of oak and birch trees. The terrain consists of dirt, and grass patches, with a nearby body of water to the left. The crosshair is positioned directly on the trunk of an oak tree...
3
Lookahead Reasoning
The tree trunk already centered in the crosshair, meaning the player does not need to adjust their aim.
2. The player is standing on gravel, and there is a slight gap between them and the tree.
3. Moving forward will bring the player closer, ensuring they are within breaking range to chop the tree efficiently.
4. Since the tree is directly in front, there is need for camera adjustments at this moment, only movement.
4
"Wait! Let's re-observe the frame and Imagine..."
"
1. A pit ahead...
2. Detour and get closer
3. View at the trunk...
"Left, Forward, Camera: [0, 10]"
Figure A3: Qualitative example of lookahead and review. The agent understands the environment (1-2), simulates future states (3), and refines its decision through internal review before acting (4), successfully avoiding a hidden hazard.
Trlcmorr
Oheprookin Qumiiaim
M45-OCC
TrageNo-KK KK
ITTIQ
CAUNN-IIQ
531M
(110)
PSNK D4teerafli
PSNKT
SSIMT
PSNRI
SSIM!
01D:
PSNRT
SSS201
:P101
2
V/
10.71
4.125
18.59
1445
33355
1998
03:01
35.02
0.517
Crees
150
11:1
0770
1156
1851
31.75
0756
1811
1535
0900
0.55
1737
0.11
0.301
Des
(eag m
18 10
V0
Table A3: Comparison of Tokenizers across different benchmarks. PSNR. SSIM, and rFID are measured on MS-COCO, ImageNet-IK, FFHQ, and CelebA-HQ datasets. PSNR for Minecraft images is provided separately.
Benchmarks and splits. All embodied evaluations are conducted in the Minecraft simulator fol- lowing the STEVE-1 protocol (Lifshitz al., (203). Unless otherwise specified, training seeds and evaluation seeds are disjoint (training: list in Appendix, evaluation: random seeds per episode). Each reported number is averaged over N evaluation episodes per task (we set 50).
Settings: Manual vs. Tool. Manual disables iron tools and requires bare-hand interactions: Tool enables iron tools to shorten long loops while preserving task logic. We evaluate both settings to cover distinct difficulty regimes.
Iteration, sample, and efficiency. One iteration denotes single forward step (one frame) of the policy during evaluation. A sample denotes collected unit in material maaterial-gathriing tasks (e.g., number of wood blocks). We report samples per iteration data-efficiency (higher better).
19

---

Published as a conference paper at ICLR 2026
Method
VPT at
Baaket
DEREDDCKKAREN DEPS Wang
PlanMC Yian
Voyager Wang
STEVE
[af
RIG (Ours)
5022
2025
Boshees 2i12
es ai
ai B023
2025
St
5025
Ruez 2023
et
ai.
Demsos
Videos
None
Videos
None
None
Nove
Videos
Videos
Rewards
Sparse Pixels Only
Desse
Sparse
Nove
Dosse
Now
Novc
None
Observations
Pisels & Mita
Pixels & Inveutory
Predback
Pixels & Mesa
Feedback
Pizels
Pixels Only
& Investory
Mena
Feedhack
& Inventory
& Meta
& Inventory
Actions
Keyhoard Mouse
&
Discrate
Keyhoard Mouse
&
Keyhoard Mouse
&
Disertte
Code
Code
Keyboand
&
Mouse
Reasoning
Generation
Extra Databuse
210
Table A4: Comparison between RIG (Ours), and existing works. This system-level comparison of LLM-based and RL-based methods focuses on data sources, reward setup, observation type, action representation, iterative planning, and skill database usage.
Tower
MincRLoottannlronPcckoooo
Explore 11.2%
MineRLTreechop
Easy 8.5%
Grass
10.9%
3%
11.0%
MiveRLNavigate
36.0%
5.9%
19.6% Dirt
37.5%
Wood
11.0%
47.3%
Dig
MineRLLbbaannDammond
(b) RIG-4s Task Distribution
(c) RIG-MineRL Task Distribution
Figure A4: Task distribution. Our datasets include various embodied tasks with varying complex- ity, ensuring strong generalization across downstream goals.
Task success metrics. For collection tasks we report the number of collected samples; for explo- ration tasks report success rate (%). Success is defined as: Dig: depth 2 e [7, 20]; Explore: travel distance 2 300; Tower: height 15.
Language and reasoning metrics. Reasoning is blind grade produced by GPT-4o with legal- ity weight (0.5 (illegal), (legal)) to penalize unsafe or infeasible plans. Understanding is graded via Minecraft QA. Score-Static is accuracy STEVE-21K static QA (0/1, averaged, then rescaled to 10). We release the exact prompts and grading rubric in the Appendix to ensure reproducibil- ity.
On claims vs. GPT-4o. When stating that our model "surpasses GPT-4o," we strictly refer to the above task-specific rubric and blind grading setting, where GPT-40 is evaluating its responses. We further use alternative prompts and cross-checkers to mitigate grader bias.
A.4 DATA DISTRIBUTION
Figure A4 visualizes the task distribution across our training datasets, which cover spectrum of embodied scenarios such as resource collection, tower building, and exploration. As task complex- ity increases, progressively expand the dataset size to ensure adequate supervision. Notably, harder tasks like building structures require significantly more data than simpler ones like gathering materials, highlighting the varying difficulty levels and skill composition in training corpus.
20

---

Published as a conference paper at ICLR 2026
Type
Model
Cure
VQA
Exams
POPE
MME-PT
MMB
SEED
GQA
MIMMU
MEL-V
Und. Only
Und Oudy
LL N at. 2024
1.38
54.1
1128.0
76.5
56.5
30.7
Und Ondy
LL el Mare
78
70.3
900.6
38.7
33.5
20.5
Und Only
LLaV EirE-e
MAN
78
$5.9
1310.7
64.3
58.6
78.5
62.0
30.4
31.1
Und Only
InstrziiL L1T0E:11
781
36.0
53.8
09.2
25.2
Only
Emmii-Chor Warnera
83.2
58.5
05.2
75.1
31.8
Only
InswuBLil Dareral
78.9
25,6
Und-&Gen
2022
1,38
73.8
948.4
25.1
Und.&Gen
LWW
7B
9.6
Und
VILA
10:3
701
83.5
59.0
33..5
lann 1W 2009
1.30
87.0
1358
63.7
30.5
54.3
Und.&Gen
RIG (Ours)
13020
58.0
35.0
Table A5: Comparison with state-of-the-arts on multimodal understanding benchmarks.
"Und." and "Gen." denote "understanding" and "generation", respectively.
A.5 COMPONENT COMPARISON
As summarized in Table A2 and Table A4 we compare RIG prior works along multiple dimen- sions, including input modality, action granularity, and reasoning capabilities. Unlike prior methods relying on handcrafted API actions or curated codebooks, RIG operates solely on raw pixels and out- puts keyboard-mouse controls, offering higher flexibility and lower task bias. Notably, our design unifies reasoning and generation into single transformer policy with self-review and imagination steps, offering better trajectory-level coherence and enabling multi-turn lookahead.
A.6 TOKENIZER AND BASE MODEL SELECTION
We adopt LlamaGen 16x 16 VQ tokenizer and Janus-1.4B as our vision and language backbone. Ta- ble A3 reports their favorable reconstruction quality (PSNR 27.06) and semantic alignment. Janus uses a dual loss combining RGB and SigLIP-guided feature reconstruction, while LlamaGen pro- vides discrete, compression-friendl tokens. Together, they form scalable pipeline for visual imag- ination and reasoning, trained with simple cross-entropy objectives.
A.7 GENERAL VQA BENCHMARK RESULTS
Setup. To assess whether embodied specialization compromises general multimodal ability, we evaluate RIG (L.3B) on standard VQA/exam suite covering Core perception (POPE, MME-P, MMB, SEED), VQA (VQAv2, GQA), and Exams (MMMU, MM-Vet). Thanks to Janus Chen al. (202)). all results and settings about the compared baselines are coordinated with it. And we compare with parameter-maten matched against Janus-1.3B for fairess,, see Table A5
Overall parity with slight gains on exam-style reasoning. Across VQA benchmarks, RIG re- mains on par with Janus (VQAv2: 77.1 vs. 77.3; GQA: 59.4 vs. 59.1), indicating no catastrophic forgetting of generic visual QA. On Exams that emphasize compositional reasoning and multi-hop understanding, RIG shows small but consistent improvements (MMMU: 30.9 vs. 30.5; MM- Vet: 35.0 vs. 34.3). We attribute these gains to the explicit reasoning curriculum (S2) and lookahead reviewing (S3-S4), which strengthen structured CoT and long-horizon inference even outside the embodied environment.
Core perception remains competitive. On Core perception metries, RIG is competitive with Janus (MMB: 68.9 9 vs. 69.4). Slight drops on POPE and MME-P (82.8 and 1302.0) are within typical variance for domain-specialized finetuning and can be further mitigated by short seneric-VQA refresh stage or lightweight anti-hallucination regularization (left for future work). Importantly, these changes do not translate into regressions on downstream VQA/Exam tasks.
Takeaways. (1) Embodied adaptation does not erode general VQA competence, (2) reasoning- centric training provides transferable benefits on exam-style evaluations, (3) any minor perception drift is small and local, while end-task metrics remain stable or improved. Together, Table A5 sup- ports our claim that RIG maintains broad multimodal understanding while gaining reasoning skills.
21

---

Published as a conference paper at ICLR 2026
A.8 QUALITATIVE RESULTS AND CASE STUDY
Figure A3 demonstrates the full inference cycle of RIG, where the agent understands the scene, rea- sons about its next move, simulates imagined outcomes, and conducts self-review before taking real action. In this wood-chopping task, the agent first identifies a tree in front, then reasons that moving forward seems viable. However, by simulating future states, spots a hidden pit and triggers a self- correction: "Wait! Let's re-observe..
It updates its decision to Lešt, Forward, Camera: [o, 10]: right, successfully avoiding the hazard. This highlights our agent's ability to perform proactive planning, visual forecasting, and risk-aware correction through imagi- nation and reviewing.
Comparison with GPT-4o image generation updated version. Figure A5 further compares RIG-lookahead with GPT-4o image generation updated version. Both receive similar prompt and visual input. While GPT-40 generates visually plausible prediction, it incorrectly judges the dis- lance to the tree, prematurely issuing an attack command that leads to deadlock. It continues hallucinate progress without correcting the faulty assumption. In contrast, RIG accurately de- tects that the tree is blocked and unreachable, reasons about terrain features, and adjusts its position before action. The generated image aligns with the actual environment response, showing stronger spatial consistency and robustness in long-horizon decision-making making.
Data pipeline comparison between GPT-4o and Owen3-VL--B--me
To assess whether a lighter open-source VLM can replace GPT-4o in our data pipeline, we run small-scale comparison using Qwen3-
2025) as a drop-in annotator under the same prompts and first-person observations. Figure A6 shows two representative cases from the "collect more wood" task, where the next action is "jump" (top) or "forward" (bottom). In both examples, GPT-40 (right) produces concise, first-person reasoning that is tightly grounded in the visual input, correctly describing the relative position to the target tree and why the proposed action moves the agent toward successful state. By contrast, Qwen3-VL-8B-Instte narrative ("the player"), hallucinates objects or properties (e.g., claiming the agent is holding a wooden pickaxe when the hands are empty), and generates long, redundant text that only loosely matches the current frame and task. Colored highlights mark correct statements (green), factual errors (red), and redundant or off-task content (gray). These qualitative results indicate that Qwen3- VL-SB-Instrue currently produces substantially lower-quality reasoning labels in our setting, which is why we rely on GPT-4o for the prototype. We believe that future stronger open open-source VLMs, or student models distilled from GPT-4o, may close this gap and make the pipeline more accessible to the broader community.
Yang et al.
(left) often adopts a third-person
A.9 MUL.TI-MODAI. UNDERSTANDING EVALUATION
We further evaluate RIG on the STEVE-2IK Zheng et al. 2023) benchmark, testing its general world knowledge and Minecraft-specific understanding. Drawing from the Minecraft Wiki and Red- dit corpus, the dataset spans multiple knowledge dimensions:
World Understanding: Terrain, entities, and biome behaviors.
Player Mechanics: Combat, mobility, and health systems.
Survival Strategies: Food sourcing, shelter, and threat avoidance.
Resource Management: Gathering, mining, and inventory use.
Crafting and Construction: Recipes and structural planning.
Tool Usage: Equipment selection and upgrades.
We evaluate with 1000 QA pairs, categorized as: World & Entities (332), Mechanics Survival (152), Knowledge & Discovery (108), Crafting (219), Tools (169), and Miscellaneous (20). Our model demonstrates strong accuracy and reasoning coherence across categorics.
A. MULTI-TURN VISUAL REASONING FORMAT
22

---

Published as a conference paper at ICLR 2026
Current IMG
Prompt
You are now Minecraft simulator. will give you the current frame and need help me predict best action the frame of the action, Action space (forward, back, left, right, jump, attack, sprint, camera:[lett right, down]].
Correct
you need
Wrong
Current frame: <IMG>,
current task: chop tree.
Predict: next action:.... next frame: <IMGs
Reasoning
IMG
Next IMG
RIG-lookahead
RiG
Ressoning:
In the first frame, the tree trunk in front you blocked leaves but it is hot close enought cannot attack immediately.
2 The tree on the right is closer more suitable for immediate tree- outting.
From terrain paint view, you can the on ground slightly adjusting to the right.
Next action: forward, sprint, camera:(0. 10
GPT-4o Image Generation Updated Version
According current screen, you facingg very close
target, and the task to 'chop tree". The best strategy should be to get close to the trunk and start attacking.
Prediction:
Next action: attack (attack the trunk with your hands)
Next frame speculation:
Cracks will appear the trunk (animation effect of attacking wood in
Minecraft)
Player position remains unchanged
HUD state is the same [no tools, only bare hands
trunk
Figure A5: Case study on reasoning and imagination. Given the same input and prompt (chop tree), RIG reasons and imagines future states to choose reachable tree and adjust position before acting. GPT-4o, despite high visual quality, misjudges the distance, executes an invalid action, and fails to revise its plan.
To supervise step-level visual reasoning, we define structured multi-turn dialogue format, as shown below. Each entry logs the task instruction, prior action, current frame, reasoning, next action, and imagined future frame. This design aligns with autoregressive generation and supports fine-grained analysis and supervision.
Task Instruction: Natural language goal (e.g., "build a tower").
Previous Action: Last executed action (e.g. "camera:[00
Current Frame: Visual observation from the environment.
Step Reasoning: Textual reasoning for the next decision.
Next Action: Predicted action.
Next Frame: Imagined visual result the action.
A.11 ENVIRONMENT DETAILS
We Minecraft the testbed for embodied agents due its open-ended nature and support for low-level human-like interactions. Agents act through egocentric RGB images and execute actions using keyboard and mouse inputs, making the environment ideal for sequential decision-mall making.
Our experiments are based on MineRL (Guss et al. 2019) v1.0 (Minecraft 1.16.5), which provides agents with first-person RGB observations and removes access to any privileged information. This
23

---

Published as a conference paper at ICLR 2026
version aligns with setups in prior works such as VPT (Baker et al., 2022) and STEVE-1. Agents only perceive visual inputs and interact through low-level actions, resembling human play
In our experiments, use S 64 environment seeds per task, covering 6 different biomes (plains, forest, taiga, mountains, desert, and snowy variants). After filtering, no single seed contributes more than K trajectories. The filtering is done in three steps: 1. Task success: We discard all trajectories that do not satisfy the task success criterion. And then rollout 6 successful trajectories on each node where failure happens. 2. Seed diversity: We enforce the per-seed cap (K to avoid over-representing any specific layout or biome. 3. Reasoning quality: For stratified subset of trajectories, we manually check that the textual reasoning is consistent with the visual scene and the executed actions; trajectories with hallucinated or clearly inconsistent reasoning are removed.
A.12 OBSERVATION AND ACTION SPACE
The agent receives 640x360 RGB images rendered from a first-person view with a 70-degree field of view. When the inventory is opened, the GUI and mouse cursor are visible. No voxel, depth, or structured APIs (e.g.. "craft", "smelt") are used.
As shown in Table A4 the action space includes 14 types of keyboard and mouse operations, cover- ing movement (WASD), item use, inventory management, and camera rotation (yaw, pitch). These mirror human inputs and do not rely high-level abstractions.
For the camera a action, which is originally a 2D continuous vector [a, b] representing pitch and yaw in [-180", 180%), we quantize it into discrete steps of 5 degrees to adapt language models, where a<0/a>0 denote looking up/down and b<0/>>0 denote turning left/right; all other actions are binary (0/1), indicating whether to execute them.
A.13 ENVIRONMENT SETTINGS AND RULES
To ensure diversity and generalization, each episode is initialized with random agent position and world seed, exposing the model to varying terrains, structures, and objectives.
We adapt the MineRL (Guss et al. 2019) environment to support instruction-bast based initialization via a custom chat action, enabling programmatic task injection and faster evaluation cycles.
Programmatic Evaluation. For instructionnfollloinn tasks, we fix the seed and biome across dif- ferent models to ensure a fair comparison. Each experiment runs under conditions most favorable for the given instruction, and multiple seeds used across different trials.
Long-Horizon Tasks with Command Switching. In these scenarios, agents operate in a shared en- vironment initialized with common seed and biome. To reduce noise and stabilize training/testing, we apply the following environment rules:
/difficulty peaceful: Disables hostile mobs.
/gamerule doDaylightCycLe false: Locks the environment in perpetual daytime.
/gamerule keepInventory true: Prevents item loss upon death.
These configurations help maintain consistency across runs while preserving the task's difficulty and realism. They also support efficient benchmarking of embodied reasoning and planning long- horizon, open-ended environments.
24

---

Published as a conference paper at ICLR 2026
Prompt
<System> You are an experienced Minecraft player with first-person view. You get the task and the you should make proper reasoning for the action for the task, and
Your task collecting more wood, the next action "jump"" Please make the reasoning the first-person view for the next action realize the task.
Qwen3-VL-BB-Imstrucc
GPT-4o
The player is currently holding a wooden pickaxe and is standing on grassy ledge. To get wood, they need to access more trees or wooden blocks, which are located higher up on the terrain. Jumping will allow them to reach the elevated ground where more wood is available, enabling them to gather more resources and progress toward their task. Therefore, the next logical action is to jump to access higher ground and continue collecting wood.
should jump to climb onto the dirt black directly front of me. The birch tree, which I can use to get wood, is on the higher ground ahead. By jumping up this step, move closer to the tree so that can reach Its trunk and start punching it to collect wood.
Prompt
<System> You experienced Minecraft player with the first-person view. You will get the task and the correct next action for the task, and you should make proper reasoning for the next action.
Your task collecting more wood, the next action is "forward". Please make the reasoning for the next action to realize the task.
Qwen3- Mieen--L--B--mee IL-8B-Instruct
GPT-4o
The player is currently holding a Wooden pickaxe and is standing near a tree a key source of wood in Minecraft. However, to efficiently gather wood, they need to position themselves directly in front of the tree to break its trunk or branches. The "forward" action moves the player closer to the tree, allowing them to: Better aim their pickaxe at the tree trunk or leaves.
should move forward to get closer to the tree right in front of me. From this distance still can'i reach the trunk blocks to punch them. Walking forward puts me directly next to the tree, so on the next steps can start breaking the wood blocks collect logs to complete the "getting wood" task.
Correct
Access the tree from more optimal angle for
harvesting,
Wrong
Prepare to break the tree and collect wood planks or logs,
Redundancy
Since task is to get wood, moving forward
logical next step to reach tree and begin harvesting it. This action directly contributes completing task by positioning player interaction the wood source.
Figure A6: Qualitative comparison of reasoning labels produced by Qwen3-VL-8B--nsteee (left) and GPT-4o (right). Given the same first-person observation and prompt (top: next action is "jump": bottom: next action is "forward") in wood-collecting task, GPT-4o generates grounded first-person reasoning that correctly describes the relative position to the target tree and the effect of action. Owen3-VL-SB-IIe objects (e.g., wooden pickaxe), and produces redundant off-task text. Green, red, and gray highlights denote correct, wrong, and redundant statements, respectively.
however, often adopts a third-person perspective, hallucinates
25

---

Published as a conference paper at ICLR 2026
Prompt Format for Minecraft Conversation Reasoning
SYSTEM:
You are assisting in generating step-by-step reasoning data for Minecraft embodied agents in complex tasks like "build tower". Each step should contain (1) the reasoning process, (2) the predicted next action, and (3) the generated next image frame. We follow a multi-turn conversation structure with visual context.
Input Format:
Task: Given current task like "build a tower"
Previous Action: e.g., camera:[0,10]
Current Frame: Image observation
Human Prompt (Input):
<image>
Please make reasoning action and predict the next action based the current frame, current task and previous action, and generate the next frame based on the current frame and the next action.
The current task: build tower. The previous action: camera: [0, 10).
RIG Output (Response):
The reasoning:
In the current frame, the task is to build a tower, but no suitable material is directly targeted. The player may be exploring for logs stone. The previous action moved the camera upward to scout surroundings... Next action: left, sprint, and next frame: <image>
Special Cases:
Lookahead and Imagination
When the model performs internal reflection or imagines the future, use the special token: <Imagine:> prefix the reasoning block.
Example: <Imagine: >> Reasoning forward steps into future environment outcomes.
Fields be included in JSON:
name, id, action, images, conversat ons, subtask (optional)
Example JSON Structure:
"name": "id": 10,
"action": "left, sprint", "conversations";
"from": "human",
"value": "<image>\n <Imagine:> Please make reasoning of action...
task: build tower..."
"images":
10 c0_current.ppne
"from": "RIG",
"value": "The reasoning: Next action: left, sprint, sand next frame: <image>",
"images": ["..._ _a..__0_next._.. 1o00_exx.pnggtttt [ng
26


Reviews:
[{"review_id": "jT9cpsXt0E", "reviewer": "Reviewer_DPLQ", "summary": "The paper presents RIG, a unified Transformer-based architecture that jointly learns reasoning, action prediction, and visual imagination for embodied agents. Unlike prior modular systems that combine separate reasoning and world-model components, RIG integrates both abilities end-to-end via a progressive data-collection and training pipeline (S0–S4). The method shows strong empirical gains in data efficiency (17× fewer hours) and performance across embodied control, reasoning, and image-generation benchmarks in the Minecraft environment.", "questions": "1. How much additional computation does the integrated reasoning–imagination (RIG) pipeline require compared to a pure Dreamer or STEVE-1 baseline?\n2. Can the authors provide an estimate of GPU hours or FLOPs per training run?\n3. Given that RIG employs a VQA model for annotation and reasoning supervision, how does this affect the inference-time cost when deployed?\n4. RIG claims 17× data efficiency, but what is the corresponding energy efficiency?", "limitations": "", "rating": 8, "confidence": 2, "soundness": 3, "presentation": 2, "contribution": 4, "strengths": "1. The explicit synergy between reasoning and imagination in a single model is original and intellectually appealing.\n2. The authors conduct extensive experiments showing consistent gains across multiple dimensions (accuracy, sample efficiency, FID/PSNR, and reasoning quality).\n3. The staged analysis (Action -> Generation -> Reasoning -> Lookahead) demonstrates how each component contributes to the final performance.\n4. The appendix describes data sources, seeds, and prompts, which is appreciated for transparency.", "weaknesses": "1. While RIG achieves substantial sample efficiency, it also depends on a large-scale VQA/vision-language model for reasoning annotations and imagination during both training and inference. It remains unclear how much more compute and energy this incurs compared to baselines such as STEVE-1 or MineDreamer. Quantifying this overhead (e.g., FLOPs per inference step, GPU hours, or carbon estimate) would strengthen the argument that RIG is efficient overall.\n2. The paper uses GPT-4o for reasoning and reviewing annotations. Although this is explained, it is worth clarifying how much of RIG’s improvement stems from the quality of these annotations versus the architecture itself.\n3. The “imagined” trajectories clearly improve performance, but the paper could better illustrate what these visual predictions actually look like and whether they faithfully represent plausible world dynamics rather than generic textures.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "qCyjlKN4ZH", "reviewer": "Reviewer_bvMP", "summary": "The paper proposed RIG, Reasoning and Imagination in an end-to-end Generalist policy method to jointly learn next-n-step imagination, text reasoning, and action predictions in the MineCraft environment. The method leverages the benefit of VLM styled reasoning as well as world model styled imagination aiming to combine all modalities together for a more efficient generalist embodied agent performance. More specifically, the paper proposed a progressive data collection strategy, training a RIG-base vision-action-reasoning model without imagination through a relabeled human trajectory dataset, and a pre-trained policy to collect image-action pairs. It leverages GPT-4o to annotate corresponding reasoning for training. Next, the paper paired negative trajectory and positive trajectory, adopting GPT-4o to generate review & revise annotation and aligned the frames temporally for long horizon training stability. The paper trained the RIG-lookahead model with this data and through rejection sampling fine-tuning. The paper conducted experiments on 6 MineCraft tasks, evaluating the data efficiency, scalability, and performance on embodied, generation, and VAQ reasoning tasks against multiple baselines respectively. The paper shows that the proposed RIG methods significantly improves data efficiency, achieves higher task performance, and is scalable.", "questions": "1. How generalizable is the method after training when applied on a different task or a different environment than MineCraft?\n2. In cases of multiple good answers at certain time point t, does the data collection pipeline only collects a single GT path for training? Does the model only know how to image one path forward? \n3. Figure 5 and Section 3.3 S3: when STEVE-1 only has acc 47.4% and RIG-basic has acc 93.4%, how does S3 collect sufficient positive trajectory from STEVE-1 while negative trajectories from RIG-base?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "- The paper is well motivated for embodied tasks through imagination (next frame prediction) before action prediction\n- The proposed method jointly trains text reasoning, visual prediction, and action prediction to explicitly model the imagination reasoning before action execution\n- The paper proposed a progressive data collection pipeline to enable the training of imagination and reasoning\n- The paper conducted thorough experiments to demonstrate the performance, efficiency, and scalability through multiple tasks and comparing against multiple baseline models.", "weaknesses": "Questions below.\nSuggestions:\n1. Ln 207: \"All these trajectories are rigorously filtered based on task success, diversity across environment seeds, and manual validation of reasoning quality.\" -- (minor) might be helpful to explain how 'rigorous', how 'diverse' these procedures are before claiming them.\n2. Ln 262 Eq (4): the notation is a little confusing: \"\"wait! Let's re-observe...\" is that part of Y- or Y+? \n3. In the Experiment section, the paper highlighted the importance of 'Number of Samples' through multiple figures and tables to highlight the data efficiency. It would be helpful to connect the dots and explain more what is 'number of samples' and and why a higher number indicates better data efficiency\n4. The appendix contains a large amount of details, figures, and results. It would be more helpful to organize and highlight the important and relevant material. \n5. Ln 1133: \"Our model demonstrates strong accuracy and reasoning coherence across categories\" -- what are the evidence to back up this claim? \n6. Ln 1170: space\n7. Ln 1190: VPT (?)\n8. Ln 377: As shown in Figure 4? Figure 5?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "q6BpzqgTJr", "reviewer": "Reviewer_JjLn", "summary": "This paper introduces RIG, a generalist policy that aims to synergize Reasoning (text), Imagination (future frames), and action into a single, end-to-end autoregressive model. It is trained using a progressive data pipeline that distills reasoning and review labels from large language models (GPT-4o) onto existing agent trajectories. During inference, RIG can \"dream\" potential outcomes, \"review\" these imagined futures, and \"self-correct\" its action plan, enabling a form of test-time planning. The authors claim this synergy achieves SOTA performance and a 17x sample efficiency improvement on Minecraft tasks .", "questions": "The imagination and self-review pipeline during inference is not that clear, can you elaborate it more concretely?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 3, "strengths": "1. The paper's core idea of a single, end-to-end model that unifies reasoning, imagination, and action in one autoregressive sequence is a compelling vision for generalist agents.\n2. Table 2 provides good ablation evidence that the joint training is synergistic, where adding Reason capabilities improves Generation quality, and adding Gen capabilities improves Reasoning scores.\n3. The \"dream-review\" mechanism is a novel and interesting contribution. It provides a concrete data-driven method for teaching the model to perform internal simulation, spot its own potential failures, and then self-correct its plan before execution.", "weaknesses": "1. The paper's single most prominent claim is its \"17x sample efficiency\" (111h of data) compared to baselines like STEVE-1 (2000h). However, the paper's own data pipeline explicitly requires a \"pretrained policy, STEVE-1\" to generate the S1 (Vision-Action) dataset. Furthermore, the S3 stage requires a \"superior-performing policy STEVE-1\" to generate positive trajectories for its corrective \"dream-review\" data. Therefore, RIG's 111h data budget requires a pre-existing 2000h STEVE-1 model as a prerequisite. The 2000h data cost of this \"teacher\" model is hidden and not included in the 111h total.\n2. The paper frames RIG as a novel learning agent that \"synergizes reasoning and imagination\". However, the data pipeline shows that the \"reasoning\" (S2) and \"reviewing\" (S3) are not learned from environmental interaction but are distilled from GPT-4o, and the vision-action training data are collected through a pretrained policy (STEVE-1) in S1 and S3. The agent is learning to imitate GPT-4o's reasoning and STEVE-1's actions. This is more of a distillation strategy rather than a novel, sample-efficient learning paradigm that the paper claims to be.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "cCeBLPAUWw", "reviewer": "Reviewer_tgFQ", "summary": "This paper proposes RIG (Reasoning and Imagination in Generalist policy) , an end-to-end Generalist policy framework, for the first time, text reasoning, low-level action control, and visual imagination are jointly modeled in the same Transformer architecture. The authors use GPT-4o to inject reasoning and reflective annotations into trajectories through a progressive data construction process (S 0-s 4) , and train two versions: the reasoning-only RIG-basic and the RIG-lookahead with the ability of“Forward-reflection”. In the Minecraft environment, RIG achieved much better sample efficiency (17 × boost) and task success rates than existing methods such as VPT, Steve-1, MineDreamer, with only 111 h of training data, and the performance of the proposed method was better than existing methods, test-time scaling is also supported to enhance decision robustness.", "questions": "- Is the RIG's architecture suitable for non-boxed, continuous-control, or real-world robotic environments? Can you provide preliminary migration experiments (e. g. Carla, MetaWorld) in the appendix or in future work?\n- Have you tried annotating trajectories with smaller open-source models, such as LLAMA-3-8B with visual adapters? How does the quality of its build affect final performance? This is critical for community replication.\n- The current “Imagination” is only used to generate modified inferences, not to directly optimize action strategies (such as policy gradients based on imaginary trajectories). Are there plans to integrate RIG with reinforcement learning for tighter imagination-based planning?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- Outstanding originality: unifying “Reasoning” and “World Model Imagination” into a single autoregressive Transformer rather than splicing multiple specialized modules (e.g. , VLM + VGM) is a significant innovation in architectural design. In particular, the“Dream-review” mechanism generates a correction trajectory through GPT-4o and skillfully converts failure experiences into training signals, which is a novel paradigm of data enhancement and self-monitoring.\n- The proposed four-stage data construction process (S0-S4) is logically clear and reproducible, and each stage has a clear goal (from basic alignment → Inference Injection → failure reflection → temporal alignment). The experimental design was comprehensive, covering ablation studies, scalability analysis (data volume, number of iterations, number of prospective steps), multi-task evaluation (collection/exploration), quality of generation (FID/PSNR), and reasoning ability (VQA), and the results were analyzed, providing strong support.\n- Significant practical implications and potential: exceeding the baseline of relying on 2000 hours of video with only 111 hours of data, demonstrating the tremendous improvement in sample efficiency of “Collaborative modeling, inference, and imagination”; it has important reference value for embodied intelligent scenarios with scarce data. Lookahead also provides a viable path to reducing real-world trial-and-error.\n- Clear problem localization and comparison: the paper accurately points out the shortcomings of the existing works (e.g., Voyager, MineDreamer) in“End-to-end optimization” and“Reasoning-imagination collaboration”, and provides a new perspective for future research, and through the intuitive comparison of four types of agent architectures in Figure 1, the contribution positioning is clear.", "weaknesses": "- All experiments were conducted in Minecraft, with clear rules and a highly structured (boxed) visual style, with gaps compared to real-world or more complex simulators (e.g., Habitat, Isaac Gym). The authors do not discuss the applicability of the method to continuous action spaces, real images, or dynamic physical environments, which weakens the universality of the conclusion.\n- S2(inferential tagging) and S3(reflective tagging) rely heavily on GPT-4o to artificially generate trajectories, but do not account for tagging costs (e.g., token consumption, manual screening time), and do not account for tagging costs (e.g, nor has there been any attempt to replace it with a more lightweight model, such as an open source LLM. This may affect the scalability and practicality of the method, especially for researchers with limited resources.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)