ViMo: A Generative Visual GUI World Model for App Agents
Authors: Dezhao Luo
Bohan Tang
Kang Li
Georgios Papoudakis
Jifei Song
Shaogang Gong
Jianye HAO
Jun Wang
Kun Shao
Venue: ICLR 2026 Poster (2026)
Keywords: World Model
GUI Generation
App Agent
Primary area: applications to computer vision, audio, language, and other modalities
TL;DR: A visual world model that generate App GUI to help App agent envision outcomes of action and make better decision.

Abstract: App agents, which autonomously operate mobile Apps through GUIs, have gained significant interest in real-world applications. Yet, they often struggle with long-horizon planning, failing to find the optimal actions for complex tasks with longer steps. To address this, world models are used to predict the next GUI observation based on user actions, enabling more effective agent planning. However, existing world models primarily focus on generating only textual descriptions, lacking essential visual details. To fill this gap, we propose ViMo, the first Visual world Model designed to generate future App observations as images. For the challenge of generating text in image patches, where even minor pixel errors can distort readability, we decompose GUI generation into graphic and text content generation. We propose a novel data representation, the Symbolic Text Representation (STR), to overlay text content with symbolic placeholders while preserving graphics. With this design, ViMo employs a STR Predictor to predict future GUIs’ graphics and a GUI-text Predictor for generating the corresponding text. Moreover, we deploy ViMo to enhance agent-focused tasks by predicting the outcome of actions. Experiments show that ViMo establishes visual world models as a compelling alternative to language-based approaches, producing visually plausible and functionally effective GUIs that empower App agents with more informed decisions.

Published as a conference paper at ICLR 2026
VIMO: A GENERATIVE VISUAL GUI WORLD MODEL FOR APP AGENTS
Dezhao Luo",, Bohan Tang", Kang Li, Georgios Papoudakis", Jifei Song', Shaogang Gong',
Jianye Hao Jun Wang*, Kun Shao
'Queen Mary University of London, PUniversity of Oxford, Huawei Noah's Ark Lab, AAl Centre, University College London
ABSTRACT
App agents, which autonomously operate mobile Apps through GUIs, have gained significant interest in real-world applications. Yet, they often struggle with long- horizon planning, failing to find the optimal actions for complex tasks with longer steps. To address this, world models are used to predict the next GUI observation based on user actions, enabling more effective agent planning. However, existing world models primarily focus generating only textual descriptions, lacking essential visual details. To fill this gap, we propose ViMo, the first Visual world Model designed to generate future App observations images. For the challenge of generating text in image patches, where even minor pixel errors can distort readability, we decompose GUI generation into graphic and text content generation. We propose novel data representation, the Symbolic Text Representation (STR), to overlay text content with symbolic placeholders while preserving graphics. With this design, ViMo employs a STR Predictor to predict future GUIs' graphics and GUI-text Predictor for generating the corresponding text. Moreover, we deploy ViMo to enhance agent-focused tasks by predicting the outcome of actions. Experiments show that ViMo establishes visual world models as a compelling alter- native to language-based approaches, producing visually plausible and functionally effective GUIs that empower App agents with more informed decisions. Project link: /ttp:://aaaagggtssse 2030 qithub. io/ViMo/
INTRODUCTION
Recent advancements Large Language Models (LLMss) have unlocked new possibilities for
deploying AI agents across diverse fields (Li et al. 2023) Gou et al. 2023; Rawles et al. 2024b
Zhou ct al. 2025 Tang et al., 2025c a Zhang et al. Ye et al. 2025 Xu et al.] 2026). notable
application 1s the smartphone application (App) agents (Rawles et al. 2024aj Wang et al. 2024a
Beechey et al. 2025), designed to directly interact with Graphical User Interfaces (GUIs) to perform
tasks autonomously and efficiently in mobile operating system.
However, existing agents struggle with making decisions for tasks requiring longer steps (Chae et al. 2024). To address this "long-horizon" limitation, increasing number of studies have introduced world models, which predict how GUTs evolve in response to user actions (Gu et al. 2024 Tang al., 2025b). Yet. these models typically rely on language describe future observations. These language-based descriptions often fail to capture the intricate visual details, such as the location and
colour of GUI elements, necessary for a precise representation (Chae al., 2024). D. A seemingly
straightforward solution is to execute action candidates App emulators. However, real-world
execution is impractical for scalable planning since actions like payments or repeated purchases
* Equal contribution.
Corresponding author: Baookun2@huwwein
By L.MSs, we refer to the concept of foundation models that accept various modalities (c.g., visual language models (VLMss,, multimodal L.LMs (MLLMs)) while producing textual sequences (W. contributors 2024

---

Published as a conference paper at ICLR 2026
User Action:
Enter the Email dlwssrrtch.tesst weattchhh.eettiiii id5 @gmaiilccmm
@eeshoffoctccom
Furward
GS
5 5-I44
Forward
viggnalzoo
coiding voe oetooet Mioors Acisomennt
Fun discrso so0ppuktien
Current GUI
Ul-Difluser
Textdifluse-2
IP2P
Figure I: GUIs generated by image-based methods (UI-Diffuser (Wei et al. 2024 TextDiffuser-
2 (Chen et al. 2024c), and IP2P (Brooks al. 2023 fine-tuned on GUI dataset, denoted as IP2P*).
are difficult backtrack. Similar concerns have motivated the broader world-model community to explore ML-based simulators (Li et al. 2025 Hainer et al. 2019; Hu et al. (022). We tackle the problem in the GUI domain by designing a GUI world model capable of predicting hypothetical observations in the visual modality.
To build a visual GUI world model capable of generating plausible future GUI observations that are visually consistent with user actions, straightforward approach involves generating each pixel of a GUT using image generations (Brooks et al.] 2023 Rombach et al.) (022). Although these methods demonstrate promising results, such as the GUI graphic generation
the location, style,
and colour of GUI elements (Wei et al. 22024) or scene-text generation a style that aligns the visual context (Chen et al.
2024c| Zhang al. 2024c), they still display distortions in the text rendering, particularly for small -sized text where each pixel is critical for accurately identifying and representing
the text (see Fig. [ for illustration).
To address the challenges of accurately generating figh-fidelity text content within GUI, we propose ViMo, the first visual GUI world model. ViMo decouples the generation of graphic and text content into distinct processes, using a novel data representation named Symbolic Text Representation (STR). In STR, each text content is replaced (overlayed) with a text symbol, a rectangle-shaped placeholder with a defined border and fill colours, functioning as a special GUI element. Thus, we simplify the task of text content generation to text symbol generation, which reframes the problem to the localisation of the text within a GUI. Based on STR, ViMo employs a STR Predictor and a GUI-text Predictor generate the graphic and the text content respectively. Specifically, the STR predictor is implemented as a diffusion model, taking the current STR, extracted from the given GUI, and a user action as inputs to generate the STR of the next GUI. Meanwhile, the GUl-text predictor, implemented based on an LLM, leverages the STR generated by the STR predictor to produce the corresponding text for each text symbol. Finally, the predicted STR and the generated text are combined to produce the next GUI.
We evaluated ViMo in three distinct scenarios to comprehensively demonstrate its effectiveness. First, we assessed its world model capability, where the quality of the generated GUIs was measured using visual similarity, instructional accuracy, and action readiness scores. Each score was examined through both automatic metrics and user studies. These assessments provided a robust and holistic evaluation of how visually precise and contextually plausible the generated GUIs were. Second, we tested ViMo in an agent-focused task evaluate its benefits for existing App agents and its superiority over other language-basel and image-based world models. In this setup, given a goal and the current App observation, the agent selected optimal actions to achieve the goal (Wang et al., 2024a). By accurately predicting the next GUI based on the current observation and an action, ViMo enabled the agent to better anticipate action outcomes and make more informed decisions. This experiment demonstrated the model's effectiveness in enhancing Decision-mmking for App agents. Finally. we evaluated ViMo's 's real-world applicability under online navigation and zero- shot generalisation settings. These scenarios assessed the model's ability to perform in real-time interactions and to generalise to previously unseen Apps, further demonstrating its generalisation capabilities and practical value in dynamic environments.
Our main contributions are summarised as follows:
We propose ViMo, the first generative visual GUI world model that predicts App observations in a visual modality, capable of more realistic and concrete visual GUI predictions compared to contemporary language-based methods.
2

---

Published as a conference paper at ICLR 2026
To address the challenge of strict pixel-level accuracy required to avoid distorted or blurred text generation in a GUI, we propose a Symbolic Text Representation (STR), overlaying text with uniform text symbols (placeholders) to simplify text content generation to text location generation. Then ViMo leverages an L.M to generate the corresponding text content for each text symbol.
Extensive experiments demonstrated the effectiveness of ViMo in both world model evaluation and agent-focused tasks. Specifically, ViMo achieved an average 19.14% and 182.74% relative improvement over existing world models terms of automatic metries and user studies, respectively. Moreover, ViMo boosted the step-wise action prediction accuracy of App agents, achieving 14.07% relative performance gain. In the online navigation setup, ViMo increased the task completion rate from 33.19% to 40.95% yielding a substantial improvement of 7.76%.
RELATED WORKS
2.1 APP AGENT
App agents, powered by LLMs, have advanced task automation on mobile Apps (Wen et al. 2024b Chen et al. 2024a; Zhang et al.] 22024a] by Lee et al. 2023). These agents interact with GUIs by emulating human actions. Approaches in this domain are broadly divided into language-basee
and methods rely on textual description of the App
multi-modalliy- -aased methods. Language-basse
observation and the user goal to generate appropriate actions (Wen et al.] 2024a Li et al. 2024). while -o-d-mooodadaassssseees methods enhance this capability by incorporating GUIs for more comprehensive understanding of the interface (Christianos et al. [2024] Wang et al. 2024b). However, these approaches struggle with long-horizon tasks that require multiple interdependent actions and deep understanding of dynamic environments (Chae et al. 2024). For this challenge, straightforward solution is to use real-word real-world emulators simulate GUI changes from user actions, enabling App agents to navigate complex scenarios and improve decision decision-making accuracy. However, emulators face significant drawbacks, including the safety risks from real-world interactions, such as repeatedly sending messages or making purchases. To overcome these, world models have gained attention as more efficient alternative, not only in agents (Chac et al. 2024] Gu et al. 2024), but also in broader Zhou et al. 2024) and self-driving (Hu et al. 2022)
domains such as robotics (Li al. 2025
2.2 WORLD MODEL
By observing the real world, world models can predict how the environment evolves in response to an action (LeCun) 2022. Ding et al.) 2024). For instance, GameNGen (Valevski et al. 2024) predicts how a game system will respond to user actions. Notably, the ability to anticipate potential outcomes of actions has proven to be highly beneficial in informing decision making processes (Pascanu et al.)
2017 Yang et al. 2024 Schrittwieser et al.) 2020; Hafner et al., 2019). Inspired by their success, world models have emerged to predict the next observation on websites. These models (Chae et al. 2024 Gu et al.) 2024 Liu et al. 2023) typically take a wehsite observation and an action as inputs to generate a textual description of the next observation. While websites provide multiple sources of information, including the actual site and their CSS or HTML source files, mobiles present a more limited context, as only the GUIs are typically accessible. Moreover, text-only descriptions such as Burns et al. (2024) often lack the precise visual details required for accurately predicting future observations, highlighting the need for visual world model capable of generating high-fidelity future GUI images.
2.3 GUI GENERATION
With the rapid advancements in image generation techniques (Rombach et al., 2022 Kumari et al.
2023
Cao & Gong 2024), previous methods have explored generating GUI directly in pixel space.
For instance, layout generation methods generate the location GUI elements (Lu et al. 2023 Zheng
et al. 2023 Sobolevsky et al., 2023 Zhao et al. 2019), scene-text generation methods generate
text that aligns with the visual context (Chen et al. 2024c| Zhang et al. 2024c| Chen et al.) 2024b
Zeng al. 2024), Ul-difuuser (Wei et al. 2024) fine-tune stable diffusion model to generate
mobile GUIS conditioned on text prompts. For the next GUI generation conditioned on current GUI
observation and user action, it seems straightforward to resort to image-and- frage-and-exxt-codditneee
3

---

Published as a conference paper at ICLR 2026
User Action "Click on the OK to save alarm"
11500-
0000
Dilfosion
GUI-text Predlctor
Overlay Text into STR
Text Content
Overlay with Text Symbolls
STR Predictor
Current GUI (X²)
ST
STR,
Next GUT (xk+1)
Figure 2: Framework of our ViMo. We first detect text content (actual words) in the current GUI (at)) and overlay it with text symbols (rectangle-shaped placeholders with black border and white fill), to create STR my. Then STR,
user action (a) are input to the STR predictor to generate the STR of the next GUI (STR,g. 2 Next, text symbols within STRr are located and assigned unique ID token. Then the LLM predicts the text content corresponding to each token. Finally, the next GUI image is constructed by overlaying the predicted text into the STR.
approach (Brooks al.
2023). However, we find that pixel- -ased image generation struggles with rendering text accurately. even minor pixel prediction errors can lead to distortions, particularly for small-sized text (see Fig. for examples).
In this work, we advance beyond existing approaches that generate GUI entirely (graphic and text) the pixel level. Instead, we render graphics as image pixels and generate text as language tokens, enabling more accurate method for GUI generation.
METHOD
Then, we introduce ViMo in Subection 3.2 Finally, we demonstrate how ViMo can applied to enhance existing App agents real-world scenarios (Subsection 3.3) All the prompts in this section are listed in Appendix H.l
In this section, first define setup in Subsection 3.1
3.1 PROBLEM SETUP
general, a GUI world model processes a given GUI Image Ty at step and user action a, to predict the effect of Ię and simulate the next GUL. Formally, this can be expressed as:
+
f(xx,a) a),
where f(.) represents the world model, and 24+1 denotes the predicted next (k+1) GUI image after applying to Zt. In the following, we explain in detail of our world model.
3.2 VIMO: GENERATIVE VISUAL GUI WORLD MODEL
To tackle the limitation of existing methods (Wei et al. 2024 Chen et al. 2024c Brooks et al. 2023) in generating visually plausible text for GUI, as shown Fig. 1 we propose ViMo, a novel generative visual GUI world model that decouples the graphic and text content generation. As shown in Fig. 2, we first detect and remove all the text in the GUI by overlaying it with a text symbol to create the Symbolic Text Representation (STR). Then STR predictor is leveraged for determining the STR representation of the next GUI with a baxel-based diffusion process. Finally, a GUI-text predictor is proposed to generate the text content for each symbol using LLM, followed by handcrafted design to overplay the text into the STR image to create the next GUI. Their details are specified in the following.
4

---

Published as a conference paper at ICLR 2026
3.2.1 STR: SYMBOLIC TEXT REPRESENTATION
To develop a GUI prediction model that eliminates the need to generate specific text content, we propose the Symbolic Text Representation (STR), where all the text content (actual words) within the GUI image is symbolised (overlayed) with uniform text symbols (placeholders). To be specific, we create STR representation from a given GUT image with three steps: 1) using OCR model (Shi) iet al. 2016 Qiao et al.] 2020) to detect text within the GUI; 2) masking the detected text by overlaying with box filled with white and bordered in black; 3) we leverage an LLM to filter out static text displayed on static GUI elements and preserve it in the image, as it does not involve any semantic evolution or dynamic changes and remains unchanged as part of specific elements such as a keyboard or clock face. Additionally, we empirically find that predicting this static text with complex spatial patterns poses significant challenges for the LLM.
Through the above process, GUI images are transformed into the STR representation, where the text content is abstracted into a text symbol, relaxing the task of generating semantic text content into predicting text symbols that indicate the location and size.
3.2.2 STR PREDICTOR
Building on the powerful generative capability of cnffuusionbbased models (Rombach et al. 2022). we introduce a STR predictor specifically trained to understand a given STR and a user action, enabling it to generate the corresponding next STR effectively. In particular, we fine-tune a pre- trained stable diffusion model (Rombach et al. 2022) to predict the next STR, conditioned on the STR of the current GUI and the user action. Given STR representation (STR.;) extracted from GUI (Ix)), the process starts with the encoding of ,TR,,, into a latent representation (Kingma & Welling 2013):
eSTR,,,) Gaussian noise is then added this representation to create 2; timestep L. A denoising autoencoder is subsequently trained to predict the Gaussian noise in the latent representation, aiming to reverse the noise addition. The objective defined as:
L
[|c co(2,, ESSTR,,), a)
where co is a U-Net (Ronneberger et al. 2015) architecture conditioned on a timestep f, text prompt (action), the visual input 2, and the image condition STR,,- To support the condition on images, we follow IP2P (Brooks et al.) 2023) to add additional input channels to the first convolutional layer, concatenating the image condition E(STR,,,) with the noised latent žt. After training, our STR predictor is capable of synthesising the next STR (STReg,, for STRz, with action instruction
3.2.3 GUI-TEXT PREDICTOR
Given a STR representation generated by our STR predictor, we design a GUI-text predictor to generate plausible text for the text symbols in the STR based its graphics. Specifically, first locate the text symbols in the STR by colour matching and boundary detection. This outputs the location of text symbols, along with their unique ID tokens assigned via enumeration, denoted T. Then we leverage the image processing and task understanding ability of LLM to predict the text content based its context in STR. This process can be formulated
t +1 LLM Tk, T):
where STRey., denotes the STR representation of - textag., contains the predicted text content for each text symbol, associated with its ID token. This design ensures flexible and accurate text generation tailored to the predicted GUI STRs as the context. Finally, we overlay each text content (textæp_,) to STR + to reconstruct the predieted GUI image (=f+1). To be specific, text symbols are replaced with the corresponding text based coordinates, with dynamic styling determined by the symbol's 's size and surrounding colours. More details are provided in Appendix
A.I
3.3 VIMO ENHANCED APP AGENT
Motivated by that App agents usually face limitations in long-horizon planning to make optimal decisions on action selection (Chae et al. 2024), we leverage the proposed ViMo to enhance the decision-making of App agents.
5

---

Published as a conference paper at ICLR 2026
Algorithm 1 Enhancing App Agent with Generative Visual GUI World Model
Input: Current GUI Observation Zg. A goal 9, A visual world model ViMo, A selection model S()).
Output: Action to be applied xy to achieve y. Generate action options AA with n actions {a') (Eq.
for i 1 to do
Leverage ViMo to synthesise the next GUI observation conditioned on a' and Tk. denoted as 2x+1 (Eq. (5).
end for
Use S() to identify the optimal action with predicted observation (Fq. (6)
Table 1: GUI quality evaluation. Sge indicates the GUI consistency, Sin instructional accuracy, Sor action readiness score and Sp the harmonic average between the 3 metrics. As, is the relative performance gains of our ViMo over other methods. IP2P* denotes finetuing of IP2P our dataset.
Automatic Metric
User Study
Method
Sgc
Sia
Sar
Sh
Ash
Sgc
Sia
Sar
8h
Ash
HTML-vision
0.70
85.77
62.79
0.72
5.39%
0.31
11.32
9.01
0.23
282.61%
IP2P*
0.74
63.57
70.15
0.69
10.20%
0.82
58.92
52.81
0.63
39.68%
Ul-diffuser
0.60
39.61
38.75
0.44
71.82%
0.36
14.32
8.56
0.27
225.93%
ViMo (Ours)
0.74
75.39
78.68
0.76
0.89
91.12
84.71
0.88
To be specific, we break down the process into three steps: action option generation, action outcome synthesis, and action selection. In the first step, the App agent generates action options, follows:
Agent( g).
where A {a a² .... a²) denotes the action option set, Fy is the current GUI image at step k and g is the given user goal. With these action options, our world model ViMo is leveraged to synthesise the outcome (next GUI) these actions as follows:
ViMo(x;,,
where x denotes the synthesised next GUI of applying action a' on æg. Finally, each action a² and its corresponding predicted outcome 2 are fed into an LLM-based selection model, which identifies the optimal action based on the generated GUIs. This process can be formulated as:
Ose = [(a'',
(6)
where ase denotes the selected action, and S(-) is the selection model. This procedure is outlined in Algorithm By predicting the GUI, ViMo provides the agent with the potential outcome of an action, enabling it to make more informed decisions. We build the selection model to identify the best action two steps. First, we query an LLM evaluate all the action candidates, providing judgment-eithe valid or invalid-ant and a confidence score for each action. Second, we query the LLM again to select the best action from the two highest-scoring actions. This process is motivated by our observation that, in over 70% of tasks, the difference between the top two scores is equal to or less than 0.1, indicating that both are likely optimal. By explicitly prompting the LLM to compare the top candidates, we go beyond coarse scoring and enable more detailed decision-making.
EXPERIMENTS
In this section, we begin by summarising our proposed STR dataset discussed in Subsection 3.2.1 Next, we tested the core capability of our ViMo, focusing on its GUI generation ability. Building on this, we demonstrated how the powerful GUI generation capability of ViMo enhance the decision- maciinn of App agents. Then, we studied our effectiveness in rorl-world App navigation tasks. Finally, carried out the ablation study validate the effectiveness of our model design. Specific setups and experiment details are elaborated in subsequent sections. Unless explicitly stated otherwise, GPT-44o (Hurst al. 2024) was employed the default L.LM in the following sections.

---

Published as a conference paper at ICLR 2026
Current GUT
ViMo (Ouns)
IP2P*
HTML-vision
Ul-ditTuxer
Search apo
aa
Open upp drawer
WEW
Chick
HODKC:
Set timer for 20 minutes
5:43
00h 20m
00. o0. O0.
00h 20m 00s
00. 00. 02.
00s
00h
12m
120
7
9
12m 00s
00. 00. 01.
00s
00
THeW
111-63
Set timer for hour
01h 00m 00s
G1h 00m OD8,
00.
Figure 3: GUI generation comparison in graphic generation (Top) and text generation (Bottom).
4.1 DATASET SUMMARISATION
Our STR dataset constructed using data from two widely recognised and large-scale sources: Android Control (Li et al. 2024) and Android in the Wild (AITW) (Rawles et al. 2024b). From these sources, respectively sampled 12 and 7 Apps, selecting those with rich data samples while filtering out noise. Android Control provides two types of user actions: 1) action commands: predefined actions (e.g., click, scroll) accompanied by specific parameters such as coordinates (x. y); 2) action instructions: actions described in natural languages, such as "click the plus icon". We used action instructions as conditions for our world model as this approach was more concrete and better utilised the pre-trained model in understanding natural language. For AITW, action commands were converted into action instructions using GPT-4o (Hurst et al.] 2024). total, we collected 19 Apps with 3,550 episodes, 23,620 images and 18,450 actions. To ensure both time-efficient and cost-efficient experiments, we followed prior App agent (Rawles et al., 2024b) on partial split evaluation. Specifically, randomly sampled 57 episodes across 19 distinct Apps. Details of dataset collection, split summarisation, and full-split experiments are provided in Appendices A.3 and B
4.2 WORLD MODEL ABILITY
We evaluated the GUI generation capability of ViMo by GUI quality evaluation. We included IP2P (Brooks al. 2023) and Ul-diffuser (Wei et al. 2024), both originally designed for image editing and GUI generation. We fine-tuned IP2P on our dataset to generate everything of the GUI, including the text content and the graphic, denoted as IP2P* We also leveraged an LLM predict App observations in an HTML format, which were rendered into images, denoted as HTML-vissoo.
We leveraged evaluation metrics: The GUI consistency score (Sge) assessed the visual similarity between the ground truth and the generated next GUI; Instructional accuracy score (Sia) determined whether the generated GUI adheres to the user action; Action readiness score (Sar) evaluated whether the generated GUI retains valid elements essential for subsequent actions required to achieve the user goal. Both automatic evaluation and user studies were conducted. For the automatic evaluation, we used DINO (Caron al. 2021) as the visual encoder to compute 8ge, and LLM to evaluate Sio and $ For the user study, we invited 70 voluntary participants to complete questionnaires based on 80 GUT samples, generated by all 4 compared methods. For each sample, participants were asked
7

---

Published as a conference paper at ICLR 2026
Table 2: Performance comparison of app agents on step accuracy. Apps are categorised into "Leisure", "Work", , and "System".
Agent Type
App Agent
Leisure
Work
System
Overall
ER (Li et al.
2024
31.76
46.15
34.13
34.50
AutoDroid Wen et al.
2024a
35.81
46.15
31.75
35,46
Language-Basee
T3A (Rawles et al. 2024a
41.22
51.28
42.86
43.13
T3A + ViMo (Ours)
50,00
58.97
45.24
49.20
APP-Agent (Zhang ct al. 2023
43.24
51.28
39.68
42.81
Mobilc-Agent-vZ (Wang
2024a
43.92
53.85
39.68
43.45
Multi-Modality-aase
M3A (Rawles ct af. 2024a
46.62
51.28
43.65
46.01
M3A + ViMo (Ours)
53.38
53.85
45.24
50.16
Table 3: Compare World Models.
Table Zero-shot Evaluation.
Modality
World Model
Step Acc.
App Agent
L.M
Step Acc.
w/o WM
w/o WM
46.01
SceAct
GPT-4-Turbe
33.9
Change-text
47.28
M3A
GPT-4-Turbo
42.1
Language
HTML-text
46.65
ER
Gemini 1.5 Pro
24.4
HTML-
48.89
T3A
Gemini-2.0-Flaan
41.4
Ul-diffuser
47.60
T3A+ViMo
Gemini-2.0-Fie
46.8
Vision
IP2P* 48.56
M3A
Gemini-2.0 Semmin-2..0-Flashh
44.2
ViMo (Ours)
50.16
M3A+ViMo
Gemini-2.0-Flash
47.6
three questions, one each for evaluating Sye, Sia, and Sar- Details of the prompts for the LLM-based evaluations and the full instructions for the user study are described in Appendix A.4
As shown in Table I ViMo achieved the highest score on the harmonic average of the three automatic metrics, surpassing other methods with average relative performance improvement of 29. 14%. The results of the user study were consistent with the automatic evaluations, where ViMo demonstrated the best performance. Notably, HTML-vision and Ul-diffuser performed significantly worse in human evaluation compared to their scores in the LLM- assessment. This discrepancy likely arose because human evaluators perceived the outputs of these methods visually unrealistic or functionally incoherent, leading to lower subjective scores in Sge, Sia and
Qualitative comparisons are presented in Fig. 3 under two scenarios: GUI graphic changes (Top) and text generation (Bottom, cropped for space efficiency). Experiments revealed that while the HTML-vision method exhibited greater flexibility in responding to user actions (as shown in the bottom examples), it failed to produce concrete details necessary for future actions (top). Conversely, IP2P* generated plausible GUI graphics but lacked flexibility in text content generation (also reflected by 8ge and Sia in Table TT. This trade-off highlighted the superior balance of ViMo.
4.3 WORLD MODEL ENHANCED APP AGENT
This section demonstrates that: I) ViMo enhanced the performance of App agents in decision-making: 2) ViMo outperformed other world models in enabling App agents make more accurate decisions.
Comparison with App Agents. In this experiment, we collected LLM-based App agents, which included three language-basee methods: ER, AutoDroid, and T3A, as well as three multi-modality- based methods: APP-Agent, Mobile-Agent-v2. and M3A. We applied our ViMo into M3A and T3A following the process in Subsection 3.3. Moreover, we followed the previous works (Rawles et al. 2024b) Li et al.) 2024) to use the step accuracy (the number of correct actions divided by the number overall actions) to quantify the model performance. To provide more detailed results, we categorised the Apps into three groups: "Leisure", "Work" and "System". Table 2 demonstratee that ViMo was beneficial to the App agent, achieving a relative performance gain of 9.01% for M3A and 14.07% for T3A. These findings highlighted the effectiveness of our proposed world model in providing App agents with enhanced decision- makiin capability. Additional information about the categorisation and experiments with more App agents provided in the Appendix A.4
8

---

Published as a conference paper at ICLR 2026
Table 5: Online Evaluation on Android
Table 6: Ablations on preserving static text and using action instructions, measured by step accuracy.
World (Rawles et al. 2024a)).
App Agent
LLM
Task Acc.
SeeAct
GPT-4-Turbo
15.50
App Agent
Static Text
Action Instr.
M3A
GPT-4-Turbo
25.40
T3A
M3A
M3A
Ge5ini-1.5-Pro
22.80
T3A
GPT-4-Turbo
30.60
N/A
N/A
43.13
46.01
T3A
Gemini-1.5-PP
19.40
42.81
45.05
T3A
Gemini-2.0-FP
33.19
V
47.28
48.88
T3A+ + ViMo
Gemini-2.0-FFan 40,95
V
49.20
50.16
Comparison with World Models. We evaluated the ability ViMo to enhance App agent decision- making by comparing it against existing world models. In addition to vision-based world models discussed in Subsection 4.2 we also incorporated two language-based world models (Gu et al. 2024 Chae et al.) 2024), utilising Change-text to generate textual descriptions capturing differences between consecutive observations and HTML-text to predict App observations in an HTML format. Then, we applied each world model to M3A App agents. Table together with Appendix Table 9 illustrates that vision-based methods consistently outperform language-bass world models, thereby reinforcing our motivation for developing visual GUI world models. Moreover, our approach achieves superior performance over existing world models, underscoring its effectiveness and advantage.
4.4 REAL-WORLD APPLICATIONS
Practical Deployment. ViMo was designed to be lightweight and easily deployable. The minimum requirement for deployment is GPU with 16 GB of memory. Moreover, ViMo was implemented as pllggand--ol and-play API that required only single function call, making integration straightforward. Inference time V100 GPU 8 seconds STR image generation and 30 seconds on GUI-text prediction. We collected and compared the inference time with existing methods in the Appendix C
Generalisation New Apps. Generalisation crucial capability for real-world applications. To assess the generalisation performance of our method on new Apps that were unseen during training,
we conducted a zero-shot evaluation using data from the Android Control dataset (Li al. (2024).
explicitly excluding Apps encountered during training. As shown in Table [ ViMo substantially outperformed the baseline and achieved 47.6%, underscoring robustness and adaptability to novel App environments. Additional visualisations of unseen scenarios are provided in the Appendix
Online Evaluation. To further demonstrate the effectiveness of ViMo in realistic App navigation scenarios, conducted an online evaluation using the AndroidWorld dataset (Rawles al. 2024a) which comprises 116 distinct navigation tasks. Performance was measured using the task success rate (Task Acc.). As illustrated in Table ViMo achieved a notable improvement of 7.76% over the baseline method, highlighting its effectiveness and reliability in real-world settings.
4.5 ABLATION STUDY
In this section, we ablated on three key components of ViMo: 1) preserving static text within the image to simplify the text generation task; 2) using action instructions instead of action commands as the con- ditioning input for ViMo; and 3) varying the number of iterations, where each iteration corresponds to one roll-out step into the future during GUI prediction.
Table 7: Ablation the number of iterations.
Method
Iteratoons
Step Acc. (%)
T3A
N/A
39.94
46.06
T3A+VIMo
2
46.65
45.05
Firstly, for the challenge of predicting static text from specific GUI elements, such as keyboard, number pad or clock face, which typically did not involve text changes and exhibited complex spatial patterns, we retained static text within the image (Subsection 3.2.1 This approach eliminated the need for the LLM to generate such static text while generating

---

Published as a conference paper at ICLR 2026
in pixels instead. Secondly, we proposed conditioning STR prediction action instruction rather than action commands (Subsection 4.1) Ablation results are presented in Table 66 where "Static Text" indicates whether static text was retained in the images, and " "Action Instr." denotes whether natural language instructions (" ("v") or abstract action commands ("-") were used as conditioning input to ViMo. The first row indicates the baseline where ViMo not applied. The table shows that both components contributed significantly to performance improvements across the two App agents, highlighting their critical roles in enabling ViMo to generate high-quality GUIs. Visual comparison examples are provided in the Appendix F for further illustration.
Our ViMo predicted future GUI observations, which could be recursively fed back as imput to simulate further into the future. In this ablation study, we varied the iteration number to evaluate how extended roll-outs impact prediction accuracy. We took Gemini-2.0- 2.0--lash (Hassabis & Kavukeuoglu 2024) as the LLM in this study. As shown in Table performing two iterations yielded the highest accuracy. However, this also led increased computational cost. Therefore, we selected one step as a practical trade-off between performance and efficiency. We also observed a slight decline in performance at iteration 3 relative to iterations and 2, indicating that extending the prediction horizon did not necessarily improve agent behaviour. This likely due to that longer horizons introduced not only additional foresight but also a greater accumulation of prediction errors, whose detrimental effect could outweigh the potential benefits. Further analysis of error accumulation, user examples of ViMo with App agent, and comparisons with various world models are provided in Appendices) B and F
CONCLUSION
In this work, we introduced ViMo, a novel generative visual GUI world model designed to predict App observations in a visual modality, providing more realistic and concrete approach compared to contemporary language-based models. To address the unique challenges of GUT generation, ViMo was equipped with the STR representation simplify text content generation text location prediction by overlaying text content with placeholders and delegating content generation to LLM. This innovation ensured high visual fidelity and avoided artefacts like distorted or blurred text. Through extensive experiments, we demonstrated that ViMo generated both visually plausible and functionally effective GUIs. Notably, ViMo boosted step-wise action prediction accuracy by a relative performance gain of 14.07% underscoring its potential to enhance decision-make making of App agents. Furthermore, real-world experiments demonstrated the strong generalisation ability of ViMo to unseen Apps, along with its robust performance in online navigation tasks under real-time environment interaction. Together with its superiority over language- based world models, these results highlighted the value of ViMo in ad advancing GUI world modelling in visual modality.
REFERENCES
Daniel Beechey, Derek Yuen, Jianheng Liu, Dezhao Luo, Tiantian He, Weilin Luo, Jun Wang, and Kun Shao. Darwin mobile agent: A roadmap for self-evolution. 2025.
Tim Brooks, Aleksander Holynski, and Alexei A Efros. Instructpix2pix: Learning to follow image editing instructions. In CVPR. pp. 18392-18402, 2023.
Andrea Burns, Kate Saenko, and Bryan A Plummer. Tell me what's next: Textual foresight for generic ui representations. arXiv preprint arXiv:2406.07822 2024.
Yu Cao and Shaogang Gong. Few-shot image generation by conditional relaxing diffusion inversion. In ECCV, pp. 20-37. Springer, 2024.
Mathilde Caron, Hugo Touvron, Ishan Misra, Hervé Jégou, Julien Mairal, Piotr Bojanowski, and Armand Joulin. Emerging properties in self-supervised vision transformers. In CVPR, pp. 9650- 9660, 2021.
Hyungjoo Chae, Namyoung Kim, Kai Tzu-iunn Ong, Minju Gwak, Gwanwoo Song, Jihoon Kim, Sunghwan Kim, Dongha Lee, and Jinyoung Yeo. Web agents with world models: Learning and leveraging environment dynamics in web navigation. arXiv preprint arXiv:2410. 13232, 2024.
10

---

Published as a conference paper at ICLR 2026
Jingxuan Chen, Derek Yuen, Bin Xie, Yuhao Yang, Gongwei Chen, Zhihao Wu, Li Yixing, Xurui Zhou, Weiwen Liu, Shuai Wang, et al. Spa-bench: A comprehensive benchmark for smartphone agent evaluation. In NeurlPS 2024 Workshop Open- World Agents, 2024a.
Jingye Chen, Yupan Huang, Tengchao Lv, Lei Cui, Qifeng Chen, and Furu Wei. Textdiffuser: Diffusion models as text painters. NeurIPS, 36, 2024b.
Jingye Chen, Yupan Huang, Tengchao Lv, Lei Cui, Qifeng Chen, and Furu Wei. Textdiffuser-2: Unleashing the power of language models for text rendering. In ECCV, pp. 386-308-- Springer, 2024c.
Filippos Christianos, Georgios Papoudakis, Thomas Coste, Jianye Hao, Jun Wang, and Kun Shao. Lightweight neural app control. arXiv preprint arXiv:2410. 17883, 2024.
Jingtao Ding, Yunke Zhang, Yu Shang, Yuheng Zhang, Zefang Zong, Jie Feng, Yuan Yuan, Hongyuan Su, Nian Li, Nicholas Sukiennik, et al. Understanding world or predicting future? a comprehensive survey of world models. arXiv preprins arXiv:2411.1444 2024.
Zhibin Gou, Zhihong Shao, Yeyun Gong, Yujiu Yang, Minlie Huang, Nan Duan, Weizhu Chen, et al. Tora: tool-integrated reasoning agent for mathematical problem solving. In ICLR, 2023.
Yu Gu, Boyuan Zheng, Boyu Gou, Kai Zhang, Cheng Chang. Sanjari Srivastava, Yanan Xie, Peng Qi, Huan Sun, and Yu Su. Is your IIm secretly world model of the internet? model-based planning for web agents. arXiv preprint arXiv:2411 000:2411.066522 2024.
Danijar Hafner, Timothy Lillicrap, Jimmy Ba, and Mohammad Norouzi. Dream to control: Learning behaviors by latent imagination. arXiv preprint arXiv: 2019.
Demis Hassabis and Koray Kavukcuoglu. Introducing gemini 2.0: our new ai model for the agentic era, December 2024. URL
http://bblloooooool/eeccccceeeee Loanooooyy/ooo haaenoooggggglllleooo leoooggeeeeepmee ndd gemini ai -ed--updteeddecceee
2024/
Accessed: 2025-05-04.
Anthony Hu, Gianluca Corrado, Nicolas Griffiths. Zachary Murez, Corina Gurau, Hudson Yeo, Alex Kendall, Roberto Cipolla, and Jamie Shotton. Model-based imitation learning for urban driving. NeurlPS, 35:20703-200 0777003-27166 16, 2022.
Aaron Hurst, Adam Lerer, Adam Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Os- trow, Akila Welihinda, Alan Hayes, Alec Radford, al. Gpt-4o system card. arXiv preprint arXiv:2410.21276, 2024.
Diederik P Kingma and Max Welling. arXiv:1312.6114 14, 2013.
Auto-encoding variational bayes. arXiv preprint
Nupur Kumari, Bingliang Zhang, Richard Zhang, Eli Shechtman, and Jun- Yan Zhu. Multi-concept customization of text-to-image diffusion. In CVPR, pp. 1931-1941, 2023.
Yann LeCun. A path towards autonomous machine intelligence version 0.9. 2, 2022-06-27. Open Review, 62(1):1-62, 2022.
Sunjae Lee, Junyoung Choi, Jungjae Lee, Munim Hasan Wasi, Hojun Choi, Steven Y Ko, Sangeun Oh, and Insik Shin. Explore, select, derive, and recall: Augmenting Ilm with human-likk memory for mobile task automation. arXiv preprint arXiv:2312.03003 2023.
Chenhao Li, Andreas Krause, and Marco Hutter. Robotic world model: A neural network simulator for robust policy optimization in robotics. arXiv preprint arXiv:2501. 10100, 2025.
Guohao Li, Hasan Hammoud, Hani Itani, Dmitrii Khizbullin, and Bernard Ghanem. Camel: Commu- nicative agents for" mind" exploration of large language model society. NeurlPS, 36:51991-52008, 2023.
Wei Li, William Bishop, Alice Li, Chris Rawles, Folawiyo Campbell-Ajala, Divya Tyamagundlu, and Oriana Riva. On the effects of data scale on computer control agents. arXiv preprint 0367799406..3679 2024.
11

---

Published as a conference paper at ICLR 2026
Anthony Liu, Lajanugen Logeswaran, Sungryull Sohn, and Honglak Lee. A picture is worth thousand words: Language models plan from pixels. In EMNLP, pp. (6450-16459, 2023.
Yuwen Lu, Ziang Tong, Qinyi Zhao, Chengzhi Zhang, and Toby Jia-Jun Li. Ui layout generation with ilms guided by ui grammar. arXiv preprint wrXiv:2310.1545555 2023.
Razvan Pascanu, Yujia Li. Oriol Vinyals, Nicolas Heess, Lars Buesing, Sebastien Racanière, David Reichert, Théophane Weber, Daan Wierstra, and Peter Battaglia. Learning model-based planning from scratch. arXiv preprint arXiv: 067000 2017.
Zhi Qiao, Yu Zhou, Dongbao Yang. Yucan Zhou, and Weiping Wang. Seed: Semantics enhanced encoder-decoder framework for scene text recognition. In CVPR, pp. 15528-13537 2020.
Christopher Rawles, Sarah Clinckemaillie, Yifan Chang, Jonathan Waltz, Gabrielle Lau, Marybeth Fair, Alice Li, William Bishop, Wei Li, Folawiyo Campbell-Ajala, et al. Androidworld: A dynamic benchmarking environment for autonomous agents. arXiv preprint arivv22455.4557. 2024a.
Christopher Rawles, Alice Li, Daniel Rodriguez, Oriana Riva, and Timothy Lillicrap. An- droidinthewild: A large-scale dataset for android device control. Neur/PS, 36, 2024b.
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High- resolution image synthesis with latent diffusion models. In CVPR, pp. 10684-10695, 2022.
Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomedical image segmentation. In MICCAI, pp.
Springer, 2015.
Julian Schrittwieser, Ioannis Antonoglou. Thomas Hubert, Karen Simonyan, Laurent Sifre, Simon Schmitt, Arthur Guez, Edward Lockhart, Demis Hassabis, Thore Graepel, et al. Mastering atari, go, chess and shogi by planning with learned model. Nature, 588(7839):004 609, 2020.
Baoguang Shi, Xiang Bai, and Cong Yao. An end-to-ent trainable neural network for image-based sequence recognition and its application to scene text recognition. TPAMI, 39(11):2298-230 2016.
Andrey Sobolevsky,
Bilodeau, Jinghui Cheng, and Jin LC Guo. Guilget: Gui layout generation with transformer. arXiv preprint arXiv:2304.099 09012, 2023.
Bohan Tang, Huidong Liang, Keyue Jiang, and Xiaowen Dong. On the importance task complexity in evaluating llm-based multi-agent systems. arXiv preprint arXiv:2510.04311, 2025a.
Bohan Tang, Dezhao Luo, Jingxuan Chen, Shaogang Gong. Jianye Hao, Jun Wang. and Kun Shao. Beyond syntax: Action semantics learning for app agents. arXiv preprint arXiv:2506.176% 2025b.
Shuo Tang, Xianghe Pang, Zexi Liu, Bohan Tang, Rui Ye, Tian Jin, Xiaowen Dong, Yanfeng Wang, and Siheng Chen. Synthesizing post-training data for 1lms through multi-agent simulation. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume I: Long Papers), pp. 23306-233355 2025c.
Dani Valevski, Yaniv Leviathan, Moab Arar, and Shlomi Fruchter. Diffusion models are real-time game engines. arXiv preprint a40iv:2408. /4837, 2024.
W. contributors. Large language model Wikipedia, the free encyclopedia. [Online]. Available: Lawggaggnmmoeeeee model, 2024. Accessed: 2024-11-25.
Junyang Wang, Haiyang Xu, Haitao Jia, Xi Zhang, Ming Yan, Weizhou Shen, Ji Zhang, Fei Huang, and Jitao Sang. Mobile-agent-v2: Mobile device operation assistant with effective navigation via multi-agent collaboration. arXiv preprint arXiv:2406. 01014, 2024a.
Taiyi Wang, Zhihao Wu, Jianheng Liu, Jianye Hao, Jun Wang, and Kun Shao. Distrl: An asyn- chronous distributed reinforcement learning framework for on-device control agents. arXiv preprint arXiv:2410. /4803, 2024b.
Jialiang Wei, Anne-Lise Courbis, Thomas Lambolais, Gérard Dray, and Walid Maalej. On ai-inspired ui-design. arXiv preprint arXiv:2406. 13631, 2024.
12

---

Published as a conference paper at ICLR 2026
Hao Wen, Yuanchun Li, Guohong Liu, Shanhui Zhao, Tao Yu, Toby Jia-Jun Li, Shiqi Jiang, Yunhao Liu. Yaqin Zhang, and Yunxin Liu. Autodroid: Lim-powered task automation in android. In MobiCom, pp. 543-554, 2024a.
Hao Wen, Shizuo Tian, Borislav Pavlov, Wenjie Du, Yixuan Li, Ge Chang, Shanhui Zhao, Jiacheng Liu, Yunxin Liu, Ya-Qin Zhang, et al. Autodroid-v2: Boosting sim-based gui agents via code generation. arXiv preprint arXiv:2412.18116, 2024b.
Mingyu Xu, Cheng Fang, Keyue Jiang. Yuqian Zheng, Yanghua Xiao, Baojian Zhou, Qifang Zhao, Suhang Zheng, Xiuwen Zhu, Jiyang Tang, et al. Logics-stem: Empowering 1lm rea- soning via failure-driven post-training and document knowledge enhancement. arXiv preprint arXiv:2601.01562, 2026.
Chang Yang, Xinrun Wang, Junzhe Jiang, Qinggang Zhang, and Xiao Huang. Evaluating world models with Ilm for decision making. arXiv preprint arXiv:241 1.08994, 2024.
Rui Ye, Keduan Huang, Qimin Wu, Yuzhu Cai, Tian Jin, Xianghe Pang, Xiangrui Liu, Jiaqi Su, Chen Maslab: unified and comprehensive codebase for llm-based multi-agent
Qian, Bohan Tang, systems. arXiv preprint St0i::25055669555 2025.
Weichao Zeng, Yan Shu, Zhenhang Li, Dongbao Yang, and Yu Zhou. Textetrl: Diffusion-berl bbsed scene text editing with prior guidance control. arXiv preprint arXiv:2410.10133, 2024.
Chaoyun Zhang, Shilin He, Jiaxu Qian, Bowen Li, Liqun Li, Si Qin, Yu Kang, Minghua Ma, Qingwei Lin, Saravan Rajmohan, al. Large language model--rained gui agents: A survey. arXiv preprint arXiv:2411.18279, 2024a.
Chi Zhang, Zhao Yang. Jiaxuan Liu, Yucheng Han, Xin Chen, Zebiao Huang, Bin Fu, and Gang Yu. Appagent: Multimodal agents as smartphone users. arXiv preprint arXiv:2312. 13771, 2023.
Jiwen Zhang, Jihao Wu, Yihua Teng, Minghui Liao, Nuo Xu, Xiao Xiao, Zhongyu Wei, and Duyu Tang. Android in the zoo: sedin-of-action-tee for gui agents. arXiv preprint arXi:24033.02713, 2024b.
Lingjun Zhang, Xinyuan Chen, Yaohui Wang, Yue Lu, and Yu Qiao. Brush your text: Synthesize any scene text on images via diffusion model. In AAAI, volume 38, pp. 7215-7223, 2024c.
Yuanshuo Zhang, Yuchen Hou, Bohan Tang. Shuo Chen, Muhan Zhang, Xiaowen Dong, and Siheng Chen. Flora: Guns as predictors of agentic workflow performances. In The Fourth Learning on Graphs Conference.
Bo Zhao, Lili Meng, Weidong Yin, and Leonid Sigal. Image generation from layout. In CVPR, pp. 8584 8593, 2019.
Guangcong Zheng, Xianpan Zhou, Xuewei Li, Zhongang Qi, Ying Shan, and Xi Li. Layoutdiffusion: Controllable diffusion model for layout-to-image generation. In CVPR, pp. 22490-22499, 2023.
Huichi Zhou, Yihang Chen, Siyuan Guo, Xue Yan, Kin Hei Lee, Zihan Wang, Ka Yiu Lee, Guchun Zhang, Kun Shao, Linyi Yang, al. Memento: Fine-tuning Ilm agents without fine-tuning llms. arXiv preprini ar 2508 /6/53, 2025.
Siyuan Zhou, Yilun Du, Jiaben Chen, Yandong Li, Dit- Yan Yeung, and Chuang Gan. Robodreamer: Learning compositional world models for robot imagination. arXiv preprint arXiv:2404. 12377, 2024.
13

---

Published as a conference paper at ICLR 2026
In this Appendix, we first provide detailed explanations, including prompts related to our methods, descriptions of our STR dataset, and evaluation details. Then, we present additional experimental results. Finally, we present additional visualisations of our proposed ViMo for GUI generation.
A EXPERIMENTAL DETAILS
A.1 GUI-TEXT PREDICTOR
This subsection elaborates on the design and functionality of the GUI-text predictor, summarising its key components and providing a detailed explanation of its underlying processes.
Given STR prediction, the GUI-text predictor starts by locating the text symbols. To be specific, we first detect black borders by identifying black pixels in the BGR colour space, generating binary mask that indicates whether pixel black or not. A pixel classified as black if its BGR values fall within the range [0,0,0] to [50,50,50]. Next, we identify rectangular regions within this mask by computing the ratio of the actual contour area to its corresponding bounding rectangle area. If this ratio exceeds 0.8, the region is considered a valid rectangle, allowing to extract rectangles with black borders. For these detected regions, we further analyse their internal colour distribution to determine whether they contain the desired white colour. Specifically, as those with BGR values within the range 200,,00,,00] to [255,255, 255,255]. If more than 50% of the pixels within region fall within this range, the region is classified text symbol. Thus, the locations of text symbols are extracted, and we assign a unique identifier (ID) to each symbol through enumeration.
define white pixels
Building on this, we take as inputs the current GUT image TK, an action a to be applied to this image, the predicted STR STR x the location and unique ID token of the text symbols in the STR T. Then we leverage an LLM predict the text content for each text symbol. The process begins with preprocessing the STR by overlaying the ID token for each text symbol to the corresponding position in the STR image, resulting in a modified representation denoted as STR k++1 Next, we prompt an LLM identify which text symbols will remain unchanged after the action (see the prompt in Subsection H.T.. These symbols are determined to not be affected by the action and have content identical to the previous GUT ry- Based on the resulting ID list, we retrieve the corresponding pixels from the previous GUI based on their location and update the STR representation. The updated image is still referred to as STR ID
k++1 for simplicity.
Subsequently, the LLM is prompted determine the semantic role of each text symbol by analysing its context (see the prompt in Subsection H.2p. This semantic information, combined with STR /D is then used to predict the exact text content of each symbol (see the prompt in Subsection H.3
Finally, overlay a symbol with its actual text content, we perform the following steps: 1) For a given text symbol's 's location and corresponding text, the average background colour is computed by the average colour of the area on the edge of text symbol's coordinates; 2) The text colour is set to either white or black to ensure optimal contrast with the background colour, for better visibility; 3) The font size is calculated as the maximum size that allows the text to fit entirely within the boundaries of the text symbol, ensuring optimal use of space and readability.
A.2 ACTION SELECTION
In practice, our selection model, described in Section 3.3
identifies the best action in two steps. First, we query an LLM evaluate all the action options, providing judgment-eitee valid or invalid-and a confidence score for each action (see the prompt in Subsection These judgments are transformed into scores: if an action is judged valid, its score equals the confidence; if judged invalid, its score is the confidence multiplied by -1. This scoring reflects that higher confidence in a valid action yields a higher score, while higher confidence in an invalid action results a lower (negative) score. Second, we query the LLM again to select the best action from the two highest-scoring actions (see the prompt in Subsection H.S). This step is motivated by our observation that, in over 70% of tasks, the difference between the top two scores is equal to or less than 0.1, indicating that both are likely optimal. By allowing the LLM to choose between them, we refine the selection beyond simply picking the action with the highest score.
14

---

Published as a conference paper at ICLR 2026
Table 8: Summarisation of our STR dataset.
Split
App
Episode
Image
Instrucion
Train
19
2853
19010
14852
Val
349
2290
1774
Test
19
348
2320
1824
All
19
3550
23620
18450
Table Decision optimisation comparisons on APP agent performance. Apps are categorised into "Leisure", "Work", and "System".
App Agent
World Model Modality
World Model
Leisure
Work
Syslem
Overall
w/o world model
W/o world model
41.22
51.28
42.86
43.13
Change-text
49.32
51.28
42.06
46.65
Langugac
HTML-text
47.30
48.72
43.65
46.01
T3A
HTML-viision
50.68
53.85
43.65
48.24
Ul-difluser
48.65
53.85
43.65
47.28
Vision
IP2P
48.65
53.85
45.24
47.92
VIMo (Ours)
50.00
58.97
45.24
49.20
w/o world model
W/o world model
43.24
51.28
39.68
42.81
Change-text
45.96
56.41
45.24
46.96
Langugae
HTML--text
44.59
56.41
45.24
46.33
APP-Agnet
HTML-viisio
47.97
56.41
46.03
48.24
Ul-difluser
47.30
56.41
44.44
47.28
Vision
IP2P
47.30
58.97
45.24
47.92
ViMo (Ours)
50.68
58.97
43.65
48.89
w/o world model
Wio world model
43.92
53.85
39.68
43.45
Change-text
47.30
66.67
41.27
47.28
Langugae
HTML-ttet
47.30
66.67
38.89
46.33
Mobile-Agent--2
ITMLL-vision
50.00
41.27
48.56
Ul-diffuser
49.32
61.54
41.27
47.60
Vision
IP2P
46.62
66.67
45.24
48.56
ViMo (Ours)
50.00
66.67
44.44
49.84
w/o world model
W/o world model
46.62
51.28
43.65
46.01
Change-text
51.35
51.28
41.27
47.28
Langugae
HTML-text
50.68
51.28
40.48
46.65
M3A
HTML-vision
52.03
48.72
45.24
48.89
Ul-difuuse
50.00
48.72
44.44
47.60
Vision
IP2P
52.03
48.56
VIMo (Ours)
53.38
53.85
45.24
50.16
A.3 DATA COLLECTION
To ensure the quality and diversity of data samples for each App. while minimising noise, we collected App information from both Android Control (Li et al. 2024) and Android in the Wild dataset (AITW) (Rawles al., 2024b) datasets. To be specific, out 15,274 episodes in the Android Control, only 5.697 episodes include the "open_app" "pp" action. From these episodes, we extracted their "app name". identifying 758 unique applications. However, only 13 of these Apps had more than 50 samples. To enrich the dataset, we manually collected additional samples for these 13 Apps from rest of the dataset. For AITW, we extracted App names by using the package name listed under the "current activity" field. After filtering out the noisy, 11 valid Apps remained. By combining the overlapping applications from both datasets, we obtained total of 19 unique Apps. We split our dataset into "Train", "Validation" and "Test" splits, and we summarise dataset under each split in Table g
Furthermore, converted action commands into action instructions for AITW with specific prompts in Subsection H.6 We use Paddleocr (Shi et al. 2016) for STR generation.
A.4 EVALUATION
World Model Ability. For the results under automatic metrics presented in Table we prompt LLM for the instructional accuracy and Subsection H.8 respectiivl.. A generation is considered successful if "success" appears under "Status" for 8ia and "yes" under "ready for action" for Sar. For the study, collected 80
8ia and action readiness Sar. shown Subsection H.7
15

---

Published as a conference paper at ICLR 2026
Table 10: Trajectory synthesis evaluation. "T+L" denotes the accuracy of the whole trajectory with length L.
World Model
T+1
T+2
T+3
T+4
w/o world model
22.81
14.04
7.02
Change-text
52.63
26.32
10.53
5.26
HTML-text
38.60
14.04
12.28
7.02
HTML-vision
43.86
19.30
10.53
10.53
Ul-diffuser
52.63
29.82
12.28
5.26
1P2P*
56.14
21.05
10.53
7.02
ViMo (Ours)
57.89
36.84
14.03
12.28
Table 11: Evaluation on randomness by running the experiment 3 times (r1-r3) on our sampled test split. "All" denotes the evaluation of the full test split. SgerSia and Sar the metrics same with Table 1
Sh denotes their harmonic score. STD denote the standard deviation from rl to r3.
World Model
Syc
8ia
Sar
8h
rl
0.7421
75.08
78.29
0.7582
r2
0.7323
75.63
77.64
0.7546
r3 0.7423
75.39
78.68
0.7605
STD
0.0057
0.23
0.42
0.0025
ALL
0.7389
75.37
78.20
0.7578
generated samples-22 20 from each of the four world models. We then asked 70 participants to answer three questions on each sample designed to reflect the 8ias $ge and Sar scores, as detailed in Subsection H.9 For the 8ye. participants are asked to rate on scale from to 5. These scores were then normalised to the [0,1] range in Table
World Model Enhanced App Agent. In Table we categorised APPs based on their primary functions into three groups: Leisures, Work, and System. The Leisure category includes APPs commonly used for relaxation and entertainment, such as Decathlon, eBay, Flipkart, Amazon, Adidas, Kitchen Stories, Booking.com com, YouTube, and Vimeo. The Work category comprises APPs typically associated with professional or productivity-related activities, including Gmail, Drive, and Chrome. Lastly, the System category encompasses APPs pre- rretinttlleed in the Android operating system, such as com. android. contacts, com wom..gooll..oo android, dialer, com woo.gg.ole...oo.ooo google..ndroid.. android, googleeuicksearchboo. com.android..oo seltingg wom.google..ndriidd APPs .Pt.iiidddPPP..aaaaiii mampss and com. com.androvid. vending.
A Ablation on Iteration Numbers. ViMo predicts future GUI observations, which can be recursively fed back input to simulate further into the future. Taking the generative GUI the current GUI, an agent was prompted to generate the action instructions based on the user goal (see the prompt in Subsection H. Then the action instruction and the GUI were fed into ViMo generate the next GUI. In this study, we defined the iteration number as the number times ViMo was called. We only use the final output the signals during the candidate action selection phase, guiding the final selection among potential actions.
B ADDITIONAL EXPERIMENTAL RESULTS
Comparison with World Models. Table compares our ViMo with existing world models under M3A App agent. To further highlight our superiority, Table 1 applied to T3A, APP-Agent, and Mobile-Agent- -2. The experimental results indicate that vision- based methods consistently outperform their language-basse counterparts, thereby substantiating our motivation for developing visual GUI world models. Moreover, our approach achieves superior performance existing world models, underscoring its effectiveness and advantage.
presents additional results of ViMo
Generation Error Analysis. As discussed in Subsec. 4.3 our method ViMo can iteratively generate future GUIs. However, the number of iterations increases, the accumulated error also grows. In
16

---

Published as a conference paper at ICLR 2026
2
9
9
2
2
Current STR
(a) Click at:[0.16,0.88]]
(b) Click contact icon
Figure 4: Qualitative ablation studies. Left: Static text generation. (a) Generating static text via an LLM; (b) Preserving the original text in the image by rendering it as image pixels. Right: STR generation under input formats (a) action command and (b) action instruction.
8:00
8:00
Nowd Picticadd
ADD sear
8:30
9:00
Click the plus add a file
Swipe up view the curry recipe and some shoes
Figure 5: GUI generation conditioned on a novel combination of current GUI observation and user action.
addition to the evidence presented in Table 7 conduct further experiments analyse this iteration error and compare our approach with existing world models.
To this end, design trajectory synthesis evaluation assess how well the GUIs generated by ViMo align with those observations in real-world environments over longer iterations. In this setup, the generated GUT is leveraged the input with higher-quality trajectories indicating GUI more aligned with the real-world environment. Specifically, the GUIs generated by ViMo serve as the observation input for the App agent, which generates actions aimed at achieving the user's goal. These output actions are then evaluated to reflect whether the GUI representations offer concrete and reliable information for action prediction. This process is repeated for L steps, and we calculate the success rate of the entire L-step trajectory.
App agent to generate the subsequent action,
We employ an LLM as a judge assess the alignment between the agent's simulated actions and the ground truth actions within given trajectory. Specifically, agent was prompted to generate the action instructions based on the given GUI and user goal (see the prompt in Subsection (oattioon.TTO) (O. and the LLM evaluated whether the simulated action lead to the same outcome the ground truth action (see the prompt in Subsection H.II). a "yes" " of the "Status" is calculated as a match.
As shown in Table 10 we compared ViMo against both visual- and language- world models and demonstrated that while performance decreases across all world models with more iterations, model significantly outperformed the other methods by providing more accurate and reliable information. This was reflected in higher trajectory prediction accuracy, underscoring the ability of model to generate GUIs that aligned with the real-world environment.
17

---

Published as a conference paper at ICLR 2026
Table 12: Comparison of inference runtime and step accuracy across different models. "Execution Time" denotes the per-module execution time for the vision and text components. The end-to-end time includes model execution, initialisation, loading, and communication overhead. "_" indicates that the corresponding module is not applicable.
Model
Execution Time (Vision / Text) End-to-End Time
Step Accuracy (%)
Baseline (T3A)
4 min
43.13
Change-text
- -s
5 8
46,64
IP2P*
8s /
1.5 min
47.92
ViMo (Ours)
8s / 30s
2 min
49.20
1:20
Bubtery
Hikin equ pepupprantt
Featuredt Advanced Smart
Hiking equipment io
riking equipeent reviene
Hiking opuipmint brands
Add to Cart
4.5
4.5 Starr
Fuffiled
8148:00
5 stars stars stars
In stock
U
Prod jot Image lnnan $129.99
product
Home
Figure 6: False examples where the text symbols are incorrectly represented, making them unrecog- nisable to indicate the location of text.
Randomness Study and Evaluation on Full Test Split. ViMo involves random factors, particularly from the use of LLMs. The LLM is prompted to generate plausible textual content, and in some cases, multiple reasonable options can be produced. For example, in Fig. 3 it shows "5:49 "on the top left corner for "set timer for 20 minutes" command and shows "Timer" for "set timer for 12 minutes", both are plausible and valid in the given context. However, the key functional element, the timer itself, is consistent with the user instructions in both cases. To evaluate their influence, we conducted the experiment three times, summarised in Table ( The results demonstrate that the randomness does not significantly impact the performance or consistency of our method. Additionally, we focused on randomly selected subset of examples for evaluation, with results from the full test set also included to illustrate that the observed differences are minor, as shown in Table IT (compare ALL r1-r3). We consider the subset results provide an accurate and reliable approximation for our analysis.
Qualitative Ablation Analysis. addition to the quantitative ablation results presented in Table 6 we also provide qualitative comparisons. Fig. | (left) illustrates the challenges faced by the L.MM in predicting static text under complex spatial layouts. Fig. 4 (right) displays the STR generation of the same user intent but with different action types. It demonstrated that models learned with action commands failed to predict STR that aligns with the user's intent, whereas action instructions offered more concrete description, enabling the model to better capture the intent.
Qualitative Generalisation Study. We studied the generalisation of ViMo in Fig. 5 providing user actions that were not typically encountered within the App's standard context. For example, in the Clock App, a user action to "add a file" generated a Drive-style file selection window while retaining the Clock interface. Similarly, in the Kitchen Store App, ViMo can generate content corresponding to the action. These results emphasised ViMo's generalisation ability facing novel combinations of App observations and user actions.
18

---

Published as a conference paper at ICLR 2026
(4) Inputs
(e) potent actions and the correleted predicted of MSA ViMo. selected action highlighted red.
Wette siing the thhoes Browks anp white notiications rectres aecasse miltsing the most summen updenss.
Stad
Pacdion tydo "sorcl"" "di "dirrction:: (dow)")
[Pocdion tyo"": wwwvigtte.oooo.ooooo book
C'action "oltiee, "Tddeo
Input Image
(b) Ground Truth
(d) Output without ViMo
Pocdoon type": "hodggtoobboo
("octoon wwpp: "scroll, "diroodon" "down1)
Figure 7: Example of how ViMo helps the App agent to select the correct action.
C PRACTICAL DEPLOYMENT
In this section, we report the computational efficiency of our method to demonstrate its practicality in real-world applications. The minimum hardware requirement is GPU with 16 GB of memory. As shown in Table 12 with a V100 GPU, STR image generation (Vision) takes approximately seconds, and GUI-text prediction (Text) takes around 30 seconds. These runtimes are practical and we have demonstrated an online inference setting Table
Although ViMo introduces additional latency, this is intentional design choice that is closely aligned with our objective of improving the reliability of long- hongzoorizon decision making. For example, when the planning horizon is extended to 4 steps, our ViMo pipeline achieves 2.33x higher accuracy (12.28% vs. 5.26% in Table 10 compared to Change-text and 1.75x higher to IP2P* We view the ability to construct a visual world model that accurately predict the consequences of actions as critical foundation for building robust GUI agents. While this design involves a higher computational cost, believe this trade-off is justified by the substantial gains in reliability and planning accuracy, especially when the horizon is longer.
Click the "Add label" text field
Click the checkbox
Press home button to go to home page
Input
for Monday
6:50
Sep 31C
8:00
8:
$0
8:00
M
Mi
Vitrata
8:30
W
8:30
9:00 m
Figure 8: Visualisation of ViMo in generating GUIs given a single current GUI paired with different actions.
19

---

Published as a conference paper at ICLR 2026
Open clock app
Add label as Gym
Open kitchen sturies app
Alam
6:00
6:00
9:30
OX
3:00 AlT
Eigg piart
Click the search bar the top of screen
Click the button the bottom right of screen
Click the Black PERFLY
Men Badminton Shoes
Lit DAY
Search
Fl
Fla
Blakk
Car
Click the menu top left the screen
Open gmail app
Click on the search icon
Sunta Bevern
TEIAL
2014
Etrnns Dayia
01
Q
Figure 9: Visualisation of ViMo in generating the next GUL. For each example, the action is displayed at the top, with the current GUI shown on the left and the generated GUI on the right.
D ADDITIONAL DISCUSSION
D.1 USER PRIVACY PRESERVATION
We emphasise that our method neither collects sensitive user data nor has the potential to leak such information.
For GUI graphic generation, the diffusion model is trained on our STR representation (GUI graphic), in which all textual content is removed from the GUI. The model is trained solely to predict visual and layout information. As a result, no sensitive or personally identifiable information is included in the training data generated by the model.
20

---

Published as a conference paper at ICLR 2026
click the icon
Type hgftest
Tap pizza restaurant
Contact
Crerte
Name
lighoe
Gara reterat
Emeil
tap the 'weather' search suggestion
Tap the Chrome app icon
Tap the three-dot menu icon
100E
Home
Google
Top Stories
Inooghto
Sastt Pighiatee
Bookmarks
Entertainment Finance News
Search
Trayel Guides
Tra
History
Science Discovenie News
Tap the toggle to turn on Airplane mode
Tap on work intemnt
Click the Voicemail
Scttingo
Aethoan dineat
Arei sidl louthers
Cod Powndes buth
Sans imporeet voleamaeis Viargge
Burway
No
Figure 10:
Visualisation of ViMo generating the next GUI, For each example, the action is displayed at the top, with the current GUI shown on the left and the generated GUI on the right.
For text rendering, model fine-tuning performed. At time only, LLM (such as ChatGPT) is prompted to generate textual content based on contextual information such as the previous GUI state, the user action, and the user goal. Sensitive user information that is not displayed on the screen is never seen or accessed. Sensitive information that may appear on screen (e.g., contact lists or account details) not collected, stored, or reused for training or any other purpose.
Therefore, no private user information is collected, stored, or leaked during either the training or inference stages.
D.2 FAIRNESS COMPARISONS
To evaluate whether the visual modality provides more precise information than the language modality for app agent tasks, we conduct controlled comparisons under non-finetuned setting. As shown in Tables 3 and 10 our two visual baselines (HTML-vision and Ul-diffuser), which are not fine-tuned on our task, consistently outperform the language- based based methods (Change-tex and HTML-text), which are also not fine-tuned. This result reflects the inherent advantage of visual representations for modelling fine-grained GUT details.
21

---

Published as a conference paper at ICLR 2026
To evaluate the necessity of our ViMo design, we also finetune the vision-based baseline IP2P on task using the same training data as ViMo. This ensures a fair comparison and further verifies that the observed improvements stem from model architecture rather than advantages in the training data.
E LIMITATION AND FUTURE WORKS
Fig. illustrates failure cases where text symbols are not represented as our rectangle-shaped placeholders with a black border and white fill, making them unrecognisable as text symbols. Improving the representation of text symbols remains a potential direction for future work.
Beyond visual quality, computational efficiency remains practical limitation. Improving the efficiency of robust GUI agents is critical research direction for enabling real-world deployment.
F ADDITIONAL VISUALISATION
Fig. demonstrates how ViMo helps the M3A App agent make better action decisions. Fig. show- cases results generated from single current GUI paired with different actions, further highlighting the versatility of our approach. Diverse visualisations are presented in Fig. 9 and Fig. 10 These examples illustrate how our method effectively generates the GUT based the given action and current GUI observation, showcasing its ability to produce visually coherent and contextually accurate GUI simulations.
G STATEMENT ON LLM USAGE
We disclose that large language model (LLM) tools were used solely for language refinment of the manuscript, including improving grammar and polishing phrasing. L.MMs were not used generate scientific content, rescarch ideas, experiment designs, data, analyses, code. All suggestions and modifications from these tools were made under the direct supervision and final approval of the authors, and all authors are fully aware of and consent to this usage.
H PROMPTS
H. PROMPT TO DECIDE THE TEXT SYMBOLS TO REMAIN UNCHIANGED AFTER THE ACTION
You are professiona" UT/UX analyst and your goal is to compare the two UT screenahots their overlapping layout.
3t# Inputs:
*AUurrent Screenshote::
current
image.
2. AsNext UT Layout Screanshot
An image the next mobile UI layout with all text replaced by white boxes.
Each box has unique red ID label.
3. Use action: Layout Screenshot the text elements are masked.
user act described by language
usez action
screenshot,
position, can Dsually, the system baz information ien should be included. Exclude elements Izom the result if:
please help identify those layouts that located
predict their text di rect y from the current screenshot.
The content (text) changes as result of the user action, even if the element exists screenshets.
please be very very cautious about putting 10 the list, which means you are very very confident with this task. you are unsure about some elementa, please ignore then and do not put then on the 1ist.
tes Output the list of existing elements ; Return the results the following JSON
format: ['idl', "idd"""id2"" 'id2', ## Notes:
Ensure detected elements appear in
screenshete, which
surrounding context is the same.
Ensura Ident tyy those alements that their text change by the user act on response.
exclude them Erom
Ensuze identify these elements that share similer context layout, but their absolute the and then from your response.
Ensure only reply in pure JSCN format, with no placeholdars or comments.
22

---

Published as a conference paper at ICLR 2026
H.2 PROMPT TO DETERMINE THE SEMANTIC ROLE OF EACH TEXT SYMBOL
UT/UX analyst asst sasigred structure and ara ayalyse the semantios of mobile 01 screenshots,
You are professtonal
elaments way that enhancas understanding of their roles and relshionships within the interface. Inputa:
Your goal la segment the ard
the Next u1 layout screenshet: visuel representation of the next ui layout with all the masked with white box.
Screenshot:
zegreseniation
has an In number the current UI will result in the next UT.
colour. user
Actions An action
Box lacations: a list of box locations to better help you to locate the boxes in the format rid'; id, 'tocation' (x1,yy, width, height]. "D indicates their Tn number In the UI sczeenet.ot.
Vi_size: width/height of input images. They are seme size. image you received might be resized. Please scale it back for the locations.
Task:
Structure the boxes the Next UI layout screenshot with semantics based the visual input by following these ateps:
1, Divide UT Semantic Kindows Group the into functional sections with specific aame (e.g., "Ileader Windews," "Tine Selector Panel*). slements
Structure Assign hox elements to windows based an logical, visual relationships or sementic roles.
Semantic window.
For every element, structure output
+id: corresponding box retrieved from the box list and the Next DE layout screenshot. should consider their [xl,1]]
v<Roee: brief explanatien to indicate their location, [8yh] to indicate their size to decide the role, It is important to consider the context detail to distinguish it from other items in the same catagory.
this
prediction. The
be
Output Fornat: "#Lndow Name"; "Category Name"; "id":id, "Role": "Role" *Category Name": "id":id, "Role": "Rele"
"id":id, "Rele": "Role",
Key Guidelines:
Ensure to retrieve id from the given screenshet and box list.
duplicating or onitzing 12s.
Every box element in the box location 11st must be included in the ructured output.
Easuze there is ao additionsl formatting, code blocks or placeholders in your response; only clesn JSON without any
H.3
PROMPT TO PREDICT THE EXACT TEXT CONTENT EACH SYMBOL
the
on provided inputs
instructions.
inputs:
Current Screenshot: A visuel reprecentation of the mobile UT.
Next UI layout screenahot: visual representation of the next ur layout with all light yellow boxes indicating text place. Bach box has ID number it. User Instruction: A spect action or conmand that tranaitions the current UT to the UI state.
Sementics for the masks in Next UI screenshet: & structured map.
Geal:
Predict the contert (text] for each maaked area in the next IIT layout screenshot based on the following steps:
Mар Affected Blements to the Next UI.
Align the affected elements with the yellow box coordinates on the next UI. user instruction the
Predict the text for each yellow based the current ui. you not its context.
any information about the text, predict plausible text
Ensuze to use the semantics to help you understand the layout and predict the text. If you think the semantics is wrong, please modify it your
Output:
format with structure: (*Mindow "; "Category Name "; "id id, *taxt ": "text", "rcle *: "role" "id "id", "text": ... Ensure to predict text based the context. De not include any specisl characters.
Return predictions
"text", *role*; "role
Ensure there is additionel fornatting, cede blocks placeholders in your response; return only a clean JSON without any aomments.
23

---

Published as a conference paper at ICLR 2026
H.4 PROMPT TO EVALUATE ACTIONS WITH A CONFIDENCE SCORE
operate Ardraid phone behalf usez request, you will try to complete it step by step. At each step, list of descriptions
You agent
When given
elements on the curzent screen will given to you el element can be specified by an index), together with history of what you have done in previous steps. Based on these pieces of information and the goal, you must choose perform
of the actions in the following List laction description followed by the dSoN format) by outputting the act in the correct J5ON formst: action options from the datasel
The overall usez gual/request is: (goal)
action you picked in the latest step: laction), whese sementic deseription is: Isum) Your goal is to judge wwhether the action you picked in the latest step is on the goal/requeste*.
Here history what you done (hittory) this
the succesaful execution the overall You will be given the screershote before and after you perform the action
right track
The first scieenshet correspends to
before you performed the action.
the second screeasho: cesresponds to the JL state after you performed the action. here
of detailed information for some elaments in before screenshot: lbefore_elemente
Note that, the "after" screenshot is generated by the agent's a world model. As such, UT. For instance: the simulated "after" screenshot may not exist in a real UI. Your evaluation ahould censider reliability of the predicticns. unreascnable elements, this likely indicates failure.
may faithfully represent
elementa
"after" screenshot
New provide your judgment on the seleated action JSON formst. Your response muat Include:
Reason: A detailed explanation of why the action is valid or invalid. judgment
Judgment:
either "valid" "invalid".
Cenfidence: A confidence scoze belween 0,0 and 1.0, reflects how likely your judgment correct.
You mist follow this structure exactly:
[Resson: .... Zudgement: "valid" or *Inva Id", Confidenos: ...)
Your Answe::
H.5 PROMPT TO SELECT THE OPTIMAL ACTIONS AMONG TWO IIGHEST-SCORING ACTIONS
You agent who operate Android phone behalf a vser request, you will try to complete it step by step. At each step, a 11t of descriptions
when given
Ui elements on the current sereen will given to you el alement can be speci fied by an index), together with history of what you have done previous stepa, Based on these piecea of information and the goal, you muat choose perform one of the actions in the following list (action description followed by the USON format) by outputting the action in the correct JSUK format action options from the dstaset.
The overall user goal/request 1a: [goal]
Here history what you done forstoiy)
Here 1a 11st of deacription for some IT elementa on the current screen: [before [betore_elements) elements)
Here two candidate actions:
Action 1: (aczion_0l, described semantically as (sum_0), The rationsle for this action [act_re_0]
Action 2: (action_ll, desoribed semantically as (sum_l), The rationale for this action (ect_re_i)
Binta for naking
decision: (GUTDANCS;
Both "more options" battons and screlling actions may reveal new content. Evaluate which is
suitable for the goal.
Consider the history of previous actions. If prior steps involved repeated "saroll down" act
likely that "scrall down If the user geal involves vlewing reviews or similar Lasks and the current screes siready displays such content, "scroll dosn" may zeveal more information. Your Lask is to choose the best action from the two provided.
next step,
New, provide your judgment in JSON forma: with the following structure: Reason: detailed explanation of your choice, considering the hints above.
Choice: Action or Action 2. output exactly match
format:
(Reason: Choice: Action 1 or Action 2
24

---

Published as a conference paper at ICLR 2026
H.6 PROMPT TO CONVERT ACTION COMMANDS INTO ACTION INSTRUCTIONS
You aze professiona: 01/UX enalyst specielizing in sdeatsfying the semantics of duel peint actions between mobile screenshots.
Inputs:
Current Screenshot: visual representation
nobile
Next Screenshot: A visual representation of the NEX: mobile JI.
Geal: user
Mobile interface.
touch__y: the coordinates for the touch point, percentage of the image dimensions.
lift_xy: the x,y coordinates for the lift point, percentage of the image dimensions.
describe precise action plain language and return your answer in plain string (e.g.r "slick the + icen", "scroll up").
is analyse
If the two screanshots are identical, please return empty string as "*, the Next Screenshot does not seem to be ene step away from the Current Screenshot, only interastion with the cell phone.
empty string "", atep Ensure there 1s no additionsl formatting, code blocks or placeholders in your response; any comments
return only
string
H.7 PROMPT FOR INSTRUCTIONAL ACCURACY SCORE (8ia)
You are an expert in avaluating the performance of nobile emplator. The mobile Instruction.
emulator designed navigate Inputs:
caange based
Current UT Screenshot:
present state of the cel cel_pheeess user interface.
Next VI Screenshet:
aobile emulater generated JE indieating the next state the reliphene's user interface based en human instruction.
Human instruction: The action applied the
screenshet.
Your goal la to deternine whether the mobile amolatos succesafully predicts the next UT informalion
Image with
layeut based the
the
IMPORIANTT
Format your response into a J50N nap as shown below:
"thoughts"; syour thoughts and reasoning process>,
"Statua": "sucoesa" "failure",
H.8
PROMPT FOR ACTION READINESS ACCURACY SCORE (Sar)
You expezt in evaluating the perfornace of mobile emulator. The mobile emulater is designed to navigate the UI change based on humen instruction. Inputs:
UI Screenshet: The zobile emulater generated DI indiceting the state of the cellphene's user interface.
User intent: The usar goal to achiave.
Next action: the action will be applied to this UI.
Your geal is to determine whether the action is validated the UI Screenshot.
please also indicate if it is still in the right App according to the goal. below:
<IMPORTANT Format your respoase
map
"Thoughta"; syour thoughts
reasoning process>,
"Tn the right App": "yes" or that "zeady for action"; "yes" or "yes",
H.9 INSTRUCTIONS FOR USER STUDY
The following prompt provides the instructions for the user study. An example screenshot is shown
in Fig. 11
25

---

Published as a conference paper at ICLR 2026
You seaeerdss framt Non sover
10W
0
0
0
0
0
9
0
Figure
Screenshot of user study example.
presanted with mobile ICurrant Your task is to evaluate whether the generated GUI correctly reflects the result of the user action applied Questicr. 2:
Question
current
"Yes" or each sample.
You are presented after the user pertorms the given action. In this task, you will evaluate the visual similarity between the Ground Truth Nex: GUI AT-generated
Ground
Next CUI, the correct sereens that appear
scoring
1-5.
Questien 3:
this task, you will validate whether specific user action generated SUI.
actioo is conaidered va screen. Ånswer "Yes" or "No" to each sample.
required GUT alement viaually presented the
H. PROMPT TO GENERATE THE ACTION INSTRUCTION BASED ON THE GIVEN GUI AND THE USER GOAL
You are an autonomous inte: ligent agent tasked with navigating a cell phone to accomplisa specific Lasks. You will provided with the following informatioa: 1. Initial Ci screenshot: visual representation of the initial state of the cell phone's a interface.
2. User Objective: This is the taak are trying to complete.
3. Previous Action: An action sequence perfozned on the initial UI.
4, Currens VI states: visual representatien of the interface, generated by a simulated environment. initial image
state of the cell phone's
screenshot before actuelly performing
previcus
actions.
The currant phone screenahot.
generated by applying previous actions on the initial
Your Task: Please predict a single next step action to complete the given task based vision
To be auscensful, it is very important to follow the following rules: should only
is valid based the current stazes.
2. You should anly issue one action at time. Avoid issving miltiple actions like
"do
3. Generate the action in plain text. For example, Scroll down to set the minute
15.
"stop." you think the action is already completed. Ensure you only the act, not other formata, commenta or placeholdars
26

---

Published as a conference paper at ICLR 2026
H.11 PROMPT TO EVALUATES WHETHER THE SIMULATED ACTION LEADS TO THE SAME OUTCOME AS THE GROUND TRUTH ACTION
You are an expect in evaluatiag the formancee of cell phone navigation agent. The designed help human navigate ceilphone complete task. Inputs:
phone's Interface. User Inteat: The goal the human user alms to achieve. Action History:
Current UT Screanshot:
present
of
sequence actions
you The action auggested by the agent to achieve the user's
progress.
Agenz Simulated Action:
intent.
Ground Truth Action: The correct actien is needed schieve the user's intent. Your goal is to deternine whather the agant's a simulated action leads to the some the addi=ionslly, if the simulated action does not exactly match the ground truth action correct
still progressing
achieve user intent, indicaring
that the action is "an tha right track." IMPORIANT+
Format your response into
map shown below:
"Thoughts"; syour thoughLe and reasoning processs,
"Status": "success" or "feilure", "en right success": "yes" "no"
27


Reviews:
[{"review_id": "ZYbi2oXI9h", "reviewer": "Reviewer_qAJA", "summary": "This paper introduces a generative visual world model for mobile app agents that predicts future GUI observations as images rather than textual descriptions. The core idea is to improve long-horizon planning for agents that control mobile apps, by visually simulating the outcomes of user actions. Because rendering text is challenging for diffusion models, the authors decompose generation into graphics (via diffusion) and text (using an LLM). Experiments show that ther proposed method improves the realism of generated GUI screens, particularly the text elements. Using the method for planning improves agent's accuracy (+14%), and generalizes to unseen apps. The overall contribution is a visual diffusion+LLM GUI simulator as an alternative to language-based GUI simulation.", "questions": "1) Why is the dataset subsampled from the larger one? \nThis begs the question of whether the test set was selected in such a way as to favor the proposed method over others. Why not use a standard dataset split?\n\n2) Please explain what is the task that the agent must perform\nDoes the input start with an empty screen? Is the instruction given, e.g. \"set an alarm for 9am\"? \n\n3) (Most important question) Will this type of approach be practical?\n\nThe paper says \"Inference time on V100 GPU is 8 seconds on a STR image generation and 30 seconds on GUI-text prediction.\" However the application is an agent that operates mobile apps. Mobile devices don't have GPUs! How can this method be practically applied? The \"online evaluation\" implies that the method was applied in real time, but there are very few details.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "- introduces a visual world model that predicts future app GUI observations as images. This is unlike prior language-based world models that generate only text descriptions of GUIs.\n- proposes an “STR” representation which overlays all textual content in the GUI with symbolic placeholders (white rectangles), This simplifies the text generation task into text localization while the graphics are generated with a diffusion model\n- Proposes two decoupled modules: 1) one generates future GUI structure as an image with placeholders for text (white rectangles) 2) the other one (LLM-based) fills in text content for each placeholder rectangle.\n- Good experimental results:\n   - Better GUI world model: achieves a significant improvement in automatic GUI quality metrics and in user studies versus existing world models \n   - also boosts action prediction accuracy, and, to a lesser degree, online task completion rate", "weaknesses": "The paper is generally well written, but there are some sections that are not clear or lack important details\n- The dataset section is quite short and does not explain the task that the agent performs\n- Why is the dataset subsampled from the larger one? Why not use a standard dataset split?\n- Is the 'dataset summarization' referring to training data or test data\n- what is \"split summarisation\"?\n- Unclear how successful actions are determined\n- Details on how user actions are encoded for the diffusion process are vague\n\n(Minor)\n- Table captions do not specify what the columns mean, eg accuracy? \n- Typo in \"Table 3: Comapre World Models\"", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "4MVs3pLPhu", "reviewer": "Reviewer_yGAq", "summary": "This paper proposes ViMo, the first visual GUI world model that predicts future app observations as images rather than text descriptions. The key innovation is the Symbolic Text Representation (STR), which overlays text content with rectangular placeholders to decompose GUI generation into graphics and text generation. ViMo uses a diffusion-based STR Predictor for graphics and an LLM-based GUI-text Predictor for text content. Experiments demonstrate improvements in GUI quality metrics and app agent decision-making performance.", "questions": "**(1) Comparison with GUI Agent in other paradigm:**\nGiven that most end-to-end or grounding-based agents on the current AndroidWorld leaderboard achieve up to 97.4% accuracy with per-step latency below 10 seconds, while ViMo reaches 40.95%, could the authors elaborate on the intended advantages or use cases of adopting a world-model approach in this context?\nIn particular, under what conditions might such a model complement or outperform grounding-based or end-to-end systems?\n\n**(2) Cost–benefit analysis:**\nViMo achieves only +2.56% improvement over Change-Text while being roughly 24× slower (2 min vs 5 s). For a 10-step task, this translates to about 20 minutes vs 50 seconds.\nWhat use cases justify this trade-off? Have you explored optimizations or hybrid approaches to reduce inference latency?\n\n**(3) Dataset and evaluation scale:**\nIf only the subset of 57 episodes was used, what criteria were applied when sampling these episodes to ensure representativeness across app types?\nCould the authors expand the evaluation to include a larger dataset to strengthen statistical validity and demonstrate robustness across different GUIs?\n\n**(4) Evaluation methodology:**\nSince most metrics rely on LLM-as-judge evaluations, have the authors examined inter-rater consistency or any correlation between LLM-based scores and human evaluations?\nSuch analysis would help validate whether the automatic metrics reliably reflect real-world agent performance.\n\n**(5) Presentation:**\nPlease consider addressing the issues outlined in the weaknesses section, such as improving figure clarity, table spacing, and better cross-referencing between the main text and appendix.", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "**(1) First visual world model formulation for GUI agents:**\nViMo is among the first works to explicitly formulate GUI state prediction as a generative visual world model, extending diffusion-based modeling beyond robotics and embodied domains to mobile GUI environments.\n\n**(2) Novel STR Representation:**\nThe proposed Symbolic Text Representation (STR) elegantly addresses the challenge of text generation in GUIs by decoupling location prediction (via diffusion) from content generation (via LLM).\nThis decomposition is technically sound, mitigates the multimodal alignment issue highlighted in Fig. 1, and may generalize beyond GUI generation to other structured visual reasoning tasks.\n\n**(3) Integration with existing App agents and measurable improvements:**\nViMo is integrated as a plug-in to current App agents (T3A, M3A) and achieves consistent step-level accuracy gains (+9–14%) as well as moderate task-level improvement (+7.8%) on AndroidWorld.\nThese results demonstrate that visual world models can effectively complement decision-making modules within agent pipelines.\n\n**(4) Thorough experimental analysis:**\nThe paper provides detailed evaluations, including trajectory synthesis, error-accumulation analysis, qualitative failure cases, and inference-time comparison.\nThis level of transparency and comprehensive reporting is commendable and contributes to the reproducibility of the study.", "weaknesses": "**(1) Limited evaluation data scale:**\nThe main experiments rely on relatively small-scale or sampled settings, including 19 apps with 57 episodes for world-model evaluation and 116 tasks for online testing.\nThis limited scope raises concerns about statistical significance and makes it difficult to assess robustness across diverse GUI domains and app categories.\n\n**(2) Evaluation method validity concerns:**\nMost reported metrics depend on LLM-as-judge evaluations and a small-scale human study (~20 samples per method).\nWhile these methods provide qualitative insights, they introduce potential bias, subjectivity, and circularity, especially since the same class of LLMs is used for both generation and evaluation.\n\n**(3) Modest performance gains with high computational cost:**\nAs shown in table 12, ViMo achieves 49.20% step accuracy, only 2.56 points higher than the Change-Text baseline, while requiring approximately two minutes per inference, about 24 times slower than the five-second language baseline.\nSuch computational overhead raises concerns about the model’s practical applicability given the modest improvement in accuracy.\n\n**(4) Limited competitiveness against existing GUI agents in other paradigms:**\nCompared with the current AndroidWorld leaderboard (https://docs.google.com/spreadsheets/d/1cchzP9dlTZ3WXQTfYNhh3avxoLipqHN75v1Tb86uhHo/edit?gid=0#gid=0), where most end-to-end or grounding-based agents complete steps in under 10 seconds and achieve up to 97.4% accuracy, ViMo’s 40.95% task accuracy and significantly higher latency make it less competitive.\n\n**(5) Presentation and clarity issues:**\n- The paper’s organization is dense and at times difficult to follow. \n- Some figures (e.g., Fig. 1 and Fig. 2) are overly compact, and the dense layout and small text reduce overall readability and visual clarity.\n- The spacing of several tables (e.g., between Tables 2 and 4–5, and below Table 7) is overly tight, which reduces readability and makes the layout appear visually crowded.\n- The appendix is not properly referenced or linked from the main text, making it difficult for readers to locate the corresponding details without manual navigation.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "0DCHJt0Eyu", "reviewer": "Reviewer_Wy6A", "summary": "**Summary**\n\nThis paper introduces **ViMo** (Visual GUI World Model), a novel **generative world model** designed for **App agents** that interact with mobile graphical user interfaces (GUIs). Current App agents often struggle with **long-horizon planning**—they cannot accurately predict how future screens will look after performing an action. Prior “world models” attempt to simulate such outcomes but typically rely on **text-only descriptions**, which fail to capture visual layouts and pixel-level details essential for GUI reasoning.\n\nTo overcome these limitations, ViMo formulates the problem as **visual GUI prediction**: given a current screen (x_k) and a user action (a), it generates the **next GUI image** (x_{k+1}^a = f(x_k, a)). The model’s core innovation is the **Symbolic Text Representation (STR)**, a hybrid representation that decouples **graphic** and **textual** content generation. STR replaces all text on a GUI with rectangle-shaped placeholders (“text symbols”), allowing ViMo to first predict the overall layout and visual structure (using a **diffusion-based STR predictor**) and then fill in textual content (via a **LLM-based GUI-text predictor**). This design mitigates pixel-level rendering errors that commonly distort text in diffusion models.\n\nViMo is evaluated in three major settings:\n\n1. **World-model ability:** ViMo achieves superior GUI generation quality compared with baselines such as IP2P*, HTML-vision, and UI-diffuser, measured by GUI consistency ((s_{gc})), instructional accuracy ((s_{ia})), and action readiness ((s_{ar})) under both automatic metrics and user studies.\n2. **Agent-enhanced decision making:** When integrated into existing App agents (e.g., T3A, M3A), ViMo improves step-wise action prediction accuracy by up to **14.07%** and raises online task completion rates by **7.76%**.\n3. **Real-world deployment and generalization:** ViMo runs efficiently (≤16 GB GPU, plug-and-play API) and generalizes to unseen Apps in zero-shot evaluations, achieving 47.6% step accuracy.\n\n**Contributions:**\n\n* Proposes **ViMo**, the **first generative visual GUI world model** capable of predicting future App observations as realistic images.\n* Introduces **Symbolic Text Representation (STR)** to separate layout prediction from text generation, ensuring high visual fidelity and readable on-screen text.\n* Demonstrates through extensive experiments that ViMo significantly enhances both **world-model quality** and **App-agent decision-making performance**, outperforming prior language- and vision-based baselines across multiple benchmarks.", "questions": "* Could the authors update the experiments by integrating **ViMo** with more recent models such as **UITars** or **Qwen3-VL**, which achieve around 60% accuracy on AndroidWorld?\n\n* With the rapid progress of modern **image and multimodal generation models** (e.g., *Qwen-Image-Edit, Nano-Banana, GPT-Image*), the gap in text rendering and visual fidelity is quickly shrinking. Do the authors envision a future where **digital world models** can be trained **end-to-end**, without the need for modular designs like STR and text-filling stages?\n\n* Regarding the **conceptual validity of digital world models**—humans can intuitively predict outcomes in the *physical* world, but GUI environments are often highly dynamic, with frequent content refreshes and animations that even users cannot anticipate precisely. How do the authors view the **epistemic limits** of modeling such digital environments? Do the authors see ViMo’s predictive modeling as primarily a *functional approximation* for agents, rather than a realistic simulation of human prediction?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "The paper proposes a **novel and well-motivated idea** that effectively addresses the weakness of existing generative models in handling text within GUI scenes. The introduction of **Symbolic Text Representation (STR)** is both elegant and practical, enabling clear separation between visual layout generation and text rendering.\n\nExperiments are **comprehensive and convincing**, demonstrating strong improvements in both GUI generation quality and downstream App-agent performance. The results show that ViMo not only produces more realistic and legible interfaces but also enhances action prediction and task success rates.\n\nOverall, the work is **original**, **technically solid**, and **highly significant**, offering a new perspective on world modeling for digital interfaces with clear implications for multimodal agents.", "weaknesses": "While the paper’s idea is strong, the **experimental setup lags behind the current state of the field**. Most evaluations rely on relatively **outdated models and benchmarks**, such as older App-agent frameworks and base models that no longer reflect the best-performing systems. Recent models (e.g., **UITars**, **Qwen3-VL**, and other 2025-era multimodal agents) have reported around **60% step accuracy on AndroidWorld**, significantly higher than the baselines used in this paper. As a result, it is difficult to fully gauge how ViMo would perform when integrated with these newer, stronger agents.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "MX3g0I1WCj", "reviewer": "Reviewer_udhG", "summary": "This paper presents ViMo, a generative visual world model for mobile GUI agents. It addresses the failure of text-only world models to capture essential visual details and the failure of general image models to render text legibly. The paper's core idea is to decouple GUI generation: a diffusion-based *STR Predictor* generates the non-text graphical layout, and an LLM-based *GUI-text Predictor* generates the text content for the predicted layout. ViMo is then used as a lookahead mechanism to help a base agent select the best action from a list of candidates, leading to improved decision-making accuracy.", "questions": "Please see weaknesses.", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- A Novel and Necessary Model. The idea of a visual world model for GUIs is a logical and necessary next step for the field. The authors are one of the first to propose a concrete and workable architecture for this.\n- The proposed STR is a good workaround to decompose the problem, allowing the diffusion model and the LLM to handle the sub-tasks they are respectively good at (spatial graphics vs. semantic text).\n- Clear Downstream Task Improvement. The paper doesn't just evaluate image quality; it demonstrates a clear, practical benefit. Using ViMo to \"preview\" actions improves the underlying agent's step accuracy by a significant margin. The online task success rate also improves by an absolute 7.76%", "weaknesses": "- Complexity in architecture. The framework is a complex multi-stage pipeline (OCR -> STR creation -> Diffusion prediction -> Symbol detection -> LLM text prediction -> Rendering). A failure at any one of these steps (e.g., the initial OCR misses text, the diffusion model generates a malformed placeholder, the symbol detector fails) could cause the entire prediction to fail. Can the authors elaborate more on how to effectively build an end-to-end world model in the future? What are the fundamental challenges, and is it just data? It'll also be nice to see any initial effort or blueprint for that kind of world models.\n- With 20K images, ViMo is already with pretty good capability. Do the authors have any data-centric scaling experiments or ideas if we can have more samples by simply scaling the environment and explore? Also, what would be the challenges to train a more universal digital world model that can also predict on high-res desktop envs?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)