Is Your LLM Secretly a World Model of the Internet? Model-Based Planning for Web Agents
Authors: Yu Gu
Kai Zhang
Yuting Ning
Boyuan Zheng
Boyu Gou
Tianci Xue
Cheng Chang
Sanjari Srivastava
Yanan Xie
Peng Qi
Huan Sun
Yu Su
Venue: Accepted by TMLR ()

Abstract: Language agents based on large language models (LLMs) have demonstrated great promise in automating web-based tasks. Recent work has shown that incorporating advanced planning algorithms, e.g., tree search, is advantageous over reactive planning for web agents. However, unlike simulated sandbox environments, real-world environments such as the web are rife with irreversible actions. This undermines the feasibility of backtracking, a cornerstone of (tree) search. Overly relying on test-time search also hurts efficiency. We advocate model-based planning for web agents that employs a world model to simulate and deliberate over the outcome of each candidate action before committing to one. We systematically explore this paradigm by: (1) Proposing a model-based planning framework, WebDreamer, which employs LLMs to serve as both world models and value functions; (2) Training specialized LLMs as world models with a scalable data synthesis pipeline. Empirical results demonstrate that WebDreamers achieves substantial performance improvements over reactive baselines. It is competitive, while being - times more efficient, with tree search in sandbox environments (VisualWebArena) and also works effectively on real-world websites (Online-Mind2Web and Mind2Web-Live). Furthermore, our trained world model, Dreamer-7B, performs comparable to GPT-4o, highlighting the potential of specialized world models for efficient and effective planning in complex web environments. All code, models, and data are publicly available at https://github.com/OSU-NLP-Group/WebDreamer

Published in Transactions on Machine Learning Research (10/2025)
Is Your LLM Secretly a World Model of the Internet? Masel- Planning for Web Agents
Yu Gu Kai Zhang"", Yuting Ning", Boyuan Zheng";
Boyu Gou', Tianci Xue', Cheng Chang", Sanjari Srivastava', Yanan Xie", Peng Qi², Huan Yu Su
The Ohio State University Orby Al {gu.826, 13255, sun.397, su.809) @osu.edu
Reviewed on OpenReview: http:://ppereevee.neee///e wwt::/orrur/11dd/1377/00..5
Abstract
Language agents based on large language models (LLMs) have demonstrated great promise in automating web-based tasks. Recent work has shown that incorporating advanced planning algorithms, e.g., tree search, is advantageous over reactive planning for web agents. However, unlike simulated sandbox environments, real-world environments such as the web are rife with irreversible actions. This undermines the feasibility of backtracking, cornerstone of (tree) search. Overly relying on test-time search also hurts efficiency. We advocate model-based. based planning for web agents that employs world model to simulate and deliberate over the outcome of each candidate action before committing to one. We systematically explore this paradigm by: (1) Proposing a model-based planning framework, WEDDREAMER, which employs LLMs to serve as both world models and value functions; (2) Training specialized LLMs as world models with a scalable data syuthesis pipeline. Empirical results demonstrate that WEBDREAMER achieves substantial performance improvements over reactive basclines. It is competitive, while being 4-5 times more efficient, with tree search in sandbox enviromments (VisualWebArena) and also works effectively real-world websites Mallne-Minnd2Wer and Mind2Web-Live). Furthermore, our trained world model, Dreamer-7B, performs comparably to GPTT-4,, highlighting the potential of specialized world models for efficient and effective planning in complex web environments.
1 Introduction
Planning (Mattar & Lengyel, 2022i_d on optimal action sequences to achieve goals has been fundamental artificial intelligence since its inception. Research into generalist web agents capable of planning and executing sequence of actions to complete complex tasks across diverse websites has gained significant interest. (Deng et al., 2023; Zhou al., 2024; Zheng et al., 2024; Koh el. al., 2024a), partly due to the web's potential However, applying existing planning algorithms (Yao et al., 2023a; Hao et al., 2023; Gu et al., 2023; Wang et al., 2025; Feng et al., 2023; Brown et al., 2024, inter alia) to the online web environment presents formidable challenges. Real-world environments such as the web are rife with state-changing and irreversible actions for example, a single website like Amazon.com com can involve numerous such actions, including submitting an order, creating an account, changing privacy scttings, among many others- sabers-making backtrucking, cornerstone of search-based planning (Koh et al., 2024b; Putta et al., 2024), highly challenging, if not infeasible. The latency from excessive exploration in test-time search also hurts efficiency and compromises user experience.
complex yet realistic environment for driving agent research and development.
"Equal Contribution.
See the contribution statement for details.
Equal Advising.
All code, models, and data publicly available at https://github.ccm/// cow/OSU- -It---Grooo///brreeeeer
1

---

Published in Transactions on Machine Learning Research (10/2025)
reactive
(b) search with real interactions
(c) model-based planning
Figure 1: Schematic illustration of different web agent stralegies as a search problem, where each node represents a webpage. (a) Reactive: The agent selects locally optimal actions without planning, often leading to suboptimal results. (b) Tree search with real interactions: The agent explores multiple paths via active website navigation, potentially allowing backtracking (dashed arrows). However, backtracking is often infeasible due to irreversible actions. (c) Model-based planning: The agent simulates outcomes (cloud-bordered nodes) before execution, reducing real interactions while maintaining effectiveness.
One promising solution to address these challenges is model-bused plarming (Pascanu et al., 2017; Moerland et al., 2023), which equips agents with the ability to simulate action sequences within world model a computational representation of environment dynamics. World models have achieved notable success (Ha & Schmidhuber, 2018; Hafner et al., 2020; 2021) in traditional reinforcement learning (RI) lasks within simulated environments like Atari games (Bellemare et al., 2013; Brockman et al., 2016), where environment dynamics are well-defined and the action space is small and fixed, making world model training relatively straightforward. However, building world models for web environments remains under-explored explored. In contrast to the simulated environments, the Internet is open-ended and ever-evolving, with complex and diverse page structures and wide range of possible user interactions. This raises the question: How can build effective world models for the Internel?
We propose building world models for the Internet by leveraging large language models (LLMs) as the foundation. Pretrained on web-scale data, LLMs have implicitly acquired both structural knowledge of websites and common sense needed to predict. the outcomes of proposed actions, polentially making them well-suited to simulate transitions in complex web environments. To this end, we introduce WEBDREAMER, a model-based planning framework that uses LLMs to simulate and score possible future states before executing actions, thereby enabling informed decision-making further train dedicated world model, Dreamer-7B, using over 3.1 million interaction instances synthesized by our scalable data synthesis pipeline.
illustrated in Figure 1. Building this framework, we
Empirical results demonstrate the effectiveness of our approach: WEBDREAMER, when powered by state-of- the-art LLMs such as GPT-4o, achieves significant performance improvements over reactive baselines across three benchmarks, covering both online and sandbox environments. It is also competitive with the tree search method, while being 4-5 times more efficient on sandbox environment VisualWebArena, and performs effectively real-world websites MOnline-Mind2WWa and Mind2Web-Live), where tree search methods are difficult to implement and deploy. Morcover, Dreamer-7B achieves performance comparable to GPT-4o on two online benchmarks. In WebArrna,,,,,a,, we can continue fine-tuning our Dreamer-7B with in-domain data synthesieed by our pipeline. With just 25K training instances, the resulting domain-specific world models yield even stronger results, surpassing GPT-4o. These findings not only demonstrate the potential of using LLMs for model-based planning, but also establish a practical foundation for building world models for the open web through data synthesis, training, and evaluation.
2

---

Published in Transactions on Machine Learning Research (10/2025)
2 Related Work
2.1 Web Agents
Web agents (Su et al., 2024) powered by (multimodal) language models aim to automate web-based tasks, with benchmarks evolving from MiniWoB++ (Shi et al., 2017; Liu et al., 2018) to WebShop (Yao et al., 2022), Mind2Web (Deng et al., 2023), WebArena (Zhou et al., 2024; Koh et al., 2024a), which introduce more realistic environments and visual challenges. Reactive Agents make decisions based on immediate observations without simulation or search (Yao et al., 2023b). Enhancements include prompting proprietary models (Zheng ct al., 2024; He et al., 2024; Deng et al., 2023) and training models on HTML and webpage screenshots (Lee et al., 2023; Gur et al., 2023; Furuta et al., 2023; Hong et al., 2024; Baechler et al., 2024). Grounding improvements come from action-coordinate training (You et al., 2024; Cheng et al., 2024; Gou et al., 2025), while aunan-annotated (Shaw et al., 2023; Hong et al., 2024; Deng et al., 2023; Lai et al., 2024) and synthetic exploration trajectories (Furuta al., 2023; Song et al., 2024; Patel et al., 2024; Pahuja et al., 2025) further refine agent behavior. However, these agents struggle with multi-step decision-making due to short-sightedness. Agents with Tree Scarch have been explored to enhance decision-making. GPT-4V-based reward modeling (Pan et al., 2024a) and tree search algorithms (Koh el al., 2024b; Putla et al., 2024; Zhang et al., 2024) enable multi-step planning, with variants such as best-first search (Koh et al., 2024b) and Monte Carlo Tree Scarch (Putta et al., 2024; Zhang et al., 2024). Despite performance gains, search methods significantly increase inference time, face challenges in backtracking on real-world websites, and risk unsafe behaviors like submitting private information to incorrect or unintended web elements.
2.2 World Models
World models, central to model-based reinforcement learning (RL; Moerland et al. (2023); Sutton (1991)), learn state transitions to improve sample efficiency (Ha & Schmidhuber, 2018) and support planning (Pascamu et al., 2017; Schrittwieser et al., 2020). Unlike traditional world models in RI. focusing on improving data efficiency in the agent learning process, LLM-basod world models emphasize decision- over simulation fidelity, leveraging broad world knowledge for planning (Hao et al., 2023; Kim et al., 2024). Our work extends this line by exploring LLM-based world models in complex web environments. A concurrent work (Chae et al., 2025) also explores augmenting web agents with LLM-simulated action outcomes. However, their focus is on using small-scale data to train in-domain world models within sandbox environments, while ours centers on using LLMs and training general world models for real-world websites via a scalable data synthesis pipeline. Their sandbox in-domain settings are also discussed in Section 3. Moreover, we evaluate the world model (see Appendix D) and use screenshots as the observation space.
3
WebDreamer: Model-Based Planning for Web Agents
3.1
Preliminary
Web agents tasked with automating activities in live websites face vast and complex search spaces. Formally, each task, given an instruction 1, can be formulated as a partially observable Markov decision process (POMDP): A,,,,, R2), where is the set of possible environment states, O is the set of observations available to the agent, and A represents actions such as clicking elements, entering text, or navigating URLs. S x A S is the state transition function, while R is a binary reward indicating task completion. The agent perceives only an observation O, sampled from the observation function a))
Tree search-based planning with real interactions is costly and risks irreversible actions. Model-based planning mitigates this by using a learned simulation function sim O x A + (O x A)* which generates imagined trajectories of observations and actions before execution. This enables online planning, where the agent iteratively selects actions based on simulated future trajectories. A common approach is Model Predictive Control (MPC; Garcia et al. (1989)), which simulates future states for each action over a finite horizon H, evaluates them using a scoring function score, and exocutes the action with the highest score. This process repeats after observing new states, allowing adaptive mecision-making while avoiding unnecessary interactions.
3

---

Published in Transactions on Machine Learning Research (10/2025)
Stage Simulation
Please navigate to the 'Data Storage' category and purchase the least expensive disk with B12GB of sturage-
The Office Product' category will dieplay three echossngrrion lnc 'Ofice Flectnonied Office Schnol Supplee,, Office flmmiture I Lightng.
The webpage will deplay Offirr Electronces sub-catepory reulls products the will Pinneneeoosssees and anggrais.
v-
Ccick Cffice Produce"
Electonic' categuey display subrotopories Computers Accessrtee: "Acossorris Supplies. and Card &e Sehicle Blettonies
The webpage diglay "Cmmpnee cub-calegory indudinn 1hit "Selet Arrmooriee, and others.
One Stop Market
① Click Uldrerta
The widyaage will display search meilts including but products each of which inchaden the product rela, pree, and an 'Add to Cart button.
The Bletzoniss wnngoyy will disglay thew athvaitgorass guner Suppliew,
and Car Vabscle Electonnis.
Stage LI: Execution
Eectronics
STurt
Figure 2: Illustration of WEDDREAMER simulating outcomes for three candidate actions using GPT-4o: (1) Click 'Office Products", (2) Click 'Electronies', and (3) Type "Disk" into tezthoz. Each dotted box shows I.L.-genertted state after a proposed action. Simulated trajectories are scored to identify the best action, and the optimal action Click "Electronics" with the highest score (v = 0.8) is executed. This example shows a two-step planning horizon. In practice, WEBDREAMER simulates multiple trajectorics per action to capture decisions. Here we
a wider range of possible outcomes, improving coverage and leading to only show one trajectory for cach action and the final score for brevity.
3.2 Core Design
WERDREAMER follows the planning through simulation paradigm introduced in Section 3.1. Figure 2 illustrates this process with three candidate actions, where WERDREAMER simulates two-step trajectorics for each action, selects the trajectory with the highest score, and executes its corresponding initial action. At its core, WEBDREAMER leverages LLMs as the simulation function (sim) and the scoring function (score).
Implementation for sim. Our implementation of sim has two modules. (1) A stale-change predictor approxi- mates the environment transition function T' by predict- action; (2) An action pro-
Algorithm 1: WEBDREAMER
Input: Instruction I; initial observation Op Output: Sequence of actions Co, a1.....02 or +++ 0;
ing how the webpage changes after exccuting this module is instruction-agnant poser then imagines plausible next action conditioned on the instruction and the predicted state, enabling lori-norizon planning. By alternating between these two steps, sim generates trajectories up to depth H, where H is configurable simulation horizon. Coneretely, to represent state changes we prompt an LLM (GPT-40 or our self-trained world model) to produce a concise nat- ural language description focused only on the effects of the action (Figure 2, Stage I). The action proposer then end
while True do
A₂ + get_ cett_candidate(_,,.. o:);;
Ai + self Saaffrrffine(TAAA
a, = arg score sim(T, (T, o,, a));
o++1 + execute(ar);
++1
if termination check() = True then break;
end
uses and this predicted state to propose the next action (Stage 11). In Algorithm 1, we denote this Instruction-condiiione rollout succinetly as sim(1,or, a).
Implementation for score. After collecting a trajectory T, simulated from each candidate action a; using sim, we further use LLM a scoring function for each simulation. Following Koh et al. (2024b), we
4

---

Published in Transactions on Machine Learning Research (10/2025)
prompt GPT-4o to score each simulated trajectory with three-scale response-complees complete (1.0), on track (0.5), or incorrect (0)-iniccainn its progress toward task completion. The final score for each action is averaged over multiple simulated trajectories and scorings, then used to determine the optimal action to execute (e.g., Click "Electronics"). as shown in Stage I of Figure 2.
In addition to sim and score, a prerequisite to planning is candidate action generation. We employ a two-stage approach: first sampling top-k actions following Koh et al. (2024b), then using an LLM-based self self_refine _refiine module to filter out redundant or implausible candidates before simulation. The role of oolf_refine is to remove unnecessary actions that would lead to near-duplicate or uninformative outcomes. For example, when instructed to "search PlayStation 5 controller with Deadpool shin", the top candidates may include variants such as type(" controller deadpool") and skin"), which differ in surface form but lead to nearly identical page states. Simulating all such variants wastes compute without providing additional signal for planning. self_ref rnfe prompts an LLM reason over the action set and retain only a concise, diverse subset of candidates that are most likely to affect the trajectory meaningfully. We show the pseudo code of WEADRREAMER'S 's overall design in Algorithm 1. termination, chack verifies if the model outputs a stop action, reaches max steps, or repeats an action over 3 times, also following the implementation by Koh et al. (2024b). Appendix B shows more details.
station 5 controller deadpool
3.3 World Model Data Synthesis and Model Training
Step 1: Web Random Walking
Step 2: Synthesis
Input
① Type iPhone 16 Case'
Fonn nen two weblange screnaahals websln what changis will other aclion glon Nkgh Agg-llow and fanciioooll desciption in deiel....
Click Sellers in Outlel'
Qwen2-VL Response
Whon you lik on the search ber after tysing "Phonc Amizzon webccige transiiinn
general homesage
Hend's what will hepoorc
1. Search Results Dieplay The moin content aroa will populoodd
coses the Each lising include
product, is name, prion. oustomer milngss and optione like 'Add Cart"
NNwiggnton youflione fibee
page
Figure 3: Data synthesis pipeline. The pipeline consists of (1) Web Random Walking, where we autonomously interact with web pages through actions like typing for search and clicking, and (2) Synthesis, where Qwen2- VL-72B generates textual descriptions of state changes based on visual snapshots before and after actions.
While
LLMs such as GPT-4o have the potential to serve as world models, their cost and latency may limit their feasibility for real-time planning. To explore a more deployable alternative, we also aim to train small world model that offers lower inference cost and easier adaptation to new domains.
As shown in Figure 3, we develop scalable data synthesis pipeline that autonomously interacts with web pages using lightweight heuristic guidance. The pipeline requires only pool of seed URLs (sampled from the October 2024 Common Crawl Index?) and can be run in parallel to scale to millions of interactions without human supervision. At each step, the raw HTML is parsed to enumerate feasible actions, such as clicking elements, hovering, typing into text boxes, or selecting options. To faithfully align these actions with the visual modality (screenshot), we maintain a mapping between each HTML element and its corresponding location on the rendered screenshot, ensuring that actions identified from the HTML can be accurately represented in the visual action descriptions.
To better reflect real usage patterns, we sample actions with fixed probabilities that favor frequent interactions like clicking, while still ensuring sufficient coverage of less common operations (e.g., selecting options). Furthermore, to encourage causal dependencics across steps, we explicitly track HTML changes after cach
Phttps//commonccaal..
5

---

Published in Transactions on Machine Learning Research (10/2025)
action and increase the probability of acting newly revealed elements. For instance, after hovering over menu tab, the pipeline prioritizes clicking items in the newly expanded dropdown (with 70% probability), while still allowing occasional actions on previously visible elements. For typing actions, we generate contextually relovant queries using GPT-3.5-Turbo to ensure realistic scarch interactions.
Once an interaction is performed, we capture visual snapshots before and after the action. We then prompt Qwen2-VL-775 VL-72B (Wang et al., 2024) generate textual descriptions detailing the changes in the webpage state (Figure 3 Step 2), ensuring training instance consists of the initial visual state, the action taken, and the generated textual description of the state change. After data collection, we filter out failed interactions, automation-blocker content, and potentially harmful data, resulting in a final dataset of over 3.1M interaction instances that capture rich causal relationships between user actions and web state transitions.
accurate representation of how each action impacts the visual content. Each
As we empirically find horizon step H=1 to be the most effective and efficient configuration, we focus on training the state transition function in sim, initializing it with Qwen2-VI--B VL-7B (Wang et al., 2024). The final model, Dreamer-7B, is trained to predict the next state as a natural language description after performing an action the current state, using next-token prediction objective. To efficiently monitor the progress of self-trained world models without relying on costly downstream evaluations for every checkpoint, we construct an intrinsic evaluation set for checkpoint selection, detailed in Appendix D. Appendix C provides additional details on data synthesis and training.
4 Experiments
4.1 Setup
To properly test our planning framework's real-world performance, we focus on three representative web agent benchmarks, capturing the dynamic nature of web interactions: VisualWebArena (VWA; Koh et al. (2024a)) is designed to evaluate multimodal agents in visually grounded tasks. It includes 233 tasks verified by humans across three self-hosted websites: Classifieds, Shopping, and Reddit. The metric success rate is calculated as the percentage of tasks successfully accomplished by the agent-generated trajectories. Online-Mind2Web (Xue et al., 2025) is an online benchmark derived from Mind2Web (Deng et al., 2023), including 300 updated or newly created high-quality tasks spanning 136 real-world websites. These tasks can be categorized into easy, medium, and hard based on the number of steps required for completion. To reduce cost, we use a subset of 100 tasks, randomly sampling 30 easy, 40 medium, and 30 hard tasks. The benchmark employs an automatic evaluation pipeline to measure task success rate, achieving an 85% agreement with human judgment. Mind2Web-Live (Pan et al., 2024b) consists of 104 tasks in 69 real-world websites refined from Mind2Web (Deng et al., 2023). It defines and annotates critical intermediate steps as key nodes for each task and considers a task successful only if all key nodes are completed." 3 For all benchmarks, we use screenshots as the observation space and add Set-of-Matt (Yang et al., 2023) in VWA for fair comparison with the tree scarch bascline. In our experiments, we empirically set the planning horizon H 1. A comprehensive analysis of this parameter is presented in Section 5.1.
To demonstrate the effectiveness of our framework and trained world models, we primarily compare with two baselines: a reactive agent and a tree search agent with real interactions.* 4 The reactive bascline uses GPT-4o as the policy model. Given the task instruction, current observation, and act history, it predicts the next, action directly, without any simulation, search, or self-refinement. While the LLM may internally perform chain-of-thought reasoning (Wei et al., 2022), the agent is reactive because it chooses only the immediate next step without explicit lookahead planning. We adopt the official GPT-4o with Set-of- (Yang et al., 2023) implementation from the VWA codebase and adapt it for Online-Mind2Weh and Mind2Web-Live. For the tree search baseline (Koh et al., 2024b), we report results only on VWA, performing real-interaction tree search is infeasible on real-world websites in Online-Mind2Web or Hind2Web-Live In VWA, Koh et al. (2024b) restore previous states by resetting the sandbox environment and re-executing the corresponding
We ensure no overlap between 4 For brevity, we refer to tree search with real interactions simply as tree search our experiments.
and Mind2Web-Liive for better task diversity.
6

---

Published in Transactions on Machine Learning Research (10/2025)
action sequences, mechanism unavailalee real websites. This makes tree search unsuitable beyond VWA, whereas our WEEDREAMER readily applies to all benchmarks.
4.2
Main Results
Method
World Model
Ouline-Mind2Web VisualWebArena
MindPWeb-Live
Reactive
17.6
26.0
20.2
Tree Search
26.2
GPT-4o
23.6
37.0
25.0
Qwen2- Qwen2-VL-7B VL-7B
17.2
31.0
19.2
DREAMRR
Qwen2-VL-72B VL-72B
31.0
18.3
Dreamer-7B
21.9
35.0
24.0
Table 1: Success rate (%) on Visual WeeArraa (Koh et al., 2024a), Online-Mind2Web (Xue et al., 2025), and Mind2Web-Live (Pan et al., 2024b). We implement all the baselines ourselves to avoid discrepancies due to hardware and experimental scttings prior works.
Effectiveness. We present the overall performance results in Table WEEDREEMER demonstrates substantial improvements over the reactive agent on all benchmarks. Notably, on the VWA dataset, our proposod method achieves a 34.1% relative performance gain and only trails behind tree search slightly. It is important to note that tree search not very practical provides more flexible and adaptive alternative. On Online- Men and Mind2Web-Live, WEBDREAMER outperforms the reactive baseline by relative gain of 42.3% and 23.8%, respectively. The strong results show the effectiveness of WEBDREAMER across different real-world websites.
real-world websites, whereas WEBDREAMBR
Secondly, training world models on our large-scale synthesized data proves to be effective. As shown in Table 1, fine-tuning Qwen2-VL-7B into Dreamer-7B leads to a substantial 4.7% absolute improvement outperforming
success rate on VisnalWebArena, 4.0% on Mindne, and 4.8% on Mind2Web-Lin the vanilla Qwen2-VI.-7B model and even Qwen2-VT.-72B. Furthermore, Dreamer-7B achieves performance comparable to GPT-4o on two online benchmarks, Online-Mind2Wes and Mind2Web-Live demonstrating the feasibility of training world models for web-based decision-making.
Efficiency. Another key advantage of model-based planning is its efficiency compared with tree search using actual explorations. Table 2 shows that tree search requires approxinately 3 times more steps than the reactive baseline, whereas our method maintains comparable number of action steps. Notably, compared to reactive basclines, tree search introduces about 10 times greater latency (in wall clock time) due to additional actions and rackin, while the overhead from simulation in our approach is substantially lower, making WEBDREAMER 4-5 times more efficient than the tree search baseline.
Steps
Reactive
Tree Search
WehDreamer
Seconds Reactive
Tree Search
WebDreamer
Classifieds
3.4
9.9
4.1
Classilieds
68.3
749.2
183.6
Reddit.
5.1
13.6
5.2
Reddit
83.5
972.1
233.7
Shopping
4.5
11.4
4.5
Shopping
87.7
785.7
179.4
(a) Number of action steps.
(b) Task completion wall clock time.
Table 2: Efficiency analysis VWA. All methods here GPT-4o for fair comparison.

---

Published in Transactions on Machine Learning Research (10/2025)
5 Discussions
5.1 Planning Framework
Ablation. We perform ablation studies on the simulation and self-refinmment stages of WEBDREAMER the VWA shopping human subset, which is the largest subset verified by humans. We pay special attention to the simulation stage, which is the core of modcl-basod planning. One might argue that the primary improvement stems from reranking candidate actions, irrespective of whether this ranking relies on simulation. To test this idea, we conduct an experiment where we remove the simulation stage and instead ask the reward model (score) to directly eval- uale each candidate action (Reranking). Additionally, we remove the self-refinement step after the action proposal our framework assess its impact (w/o Self- SelffReiineent).
35
Reactive
WebDreamer Self-Refinement
Reranking
30
29.8
Upon closer examination, primarily due to the self-refinement module's ability to effectively filter out less relevant candidate actions when the next optimal action is clear. In contrast, directly sim- ulating all actions may introduce additional noise that can negatively impact performance. As shown Figure 4, this modified reranking approach does lead some im- provement over the reactive baseline, but the gain is small and still falls well behind WEBDREAMER. These results confirm that the LLM-based world model simulation plays a crucial role in the planning process. Furthermore, we observe a decrease in performance when removing the self-refinement stage.
find that this decline is
Success Rate (%)
27.
25
22.8
21.1
20
15
Figure 4: Ablation study on the simulation stage and self-refinement stage.
Planning Horizon. As introduced in Section 3.2, WEBDREAMER supports configurable planning horizon (i.e., the simulation depth). To gain deeper insights into its effectiveness and current limitations, we investigate how the planning horizon affects the final performance. Using GPT-4o as the world model, we evaluate WEBDREAMER with planning horizons of 2, and on the same subset of Onlino-Mind2W1
40
40
37
Success Rate (%)
Success Rate (%)
35
35
32
32
30
30
25
Reactive Agent AAT www.
2g
20
Mind2Web-Live
Online-Mind2Web
15
2 Planning Horizon
3
0.00
0.63
1.30
3.16
Synthetic Training Data (M)
Figure 5: Performance of WEDDREAMER with GPT-4o and different planning horizons H on Online-Mind2Web.
Figure 6: Performance of WEBDREAMER with Dreamer-7B w.r.t. different sizes of training data on two online benchmarks.
As depicted in Figure 5, the performance consistently outperforms the reactive bascline across all horizon settings. Neleth, when the plamning horizon extends to 2 and 3 steps, the elfectiveness begins to diminish. Upon closer examination, this is primarily due to action proposal hallucinations within the simulation. Specifically, the action proposal within simulation biased toward generating scemingly relevant actions for task completion, even when these actions are not available based on the predicted outcome. As result, as the planning horizon increases, the trajectories simulated from different actions become less distinguishable,
8

---

Published in Transactions on Machine Learning Research (10/2025)
they all appear somewhat correct. 5 Given the complexity of the web environment, simulating multiple steps ahead is challenging due to error accumulation, which aligns with previous observations (Mendes & Ritter, 2025; Chae et al., 2025). However, achieving better performance with longer horizons is not major goal of this work; instead, we aim to show the feasibility and potential of using LLMs for model-based planning. We leave this as a venue for future improvement.
5.2 Training World Models
Scaling Trend. We investigate the scaling trend by gradually increasing the size of our synthetic training data, as shown in Figure 6. For Mind2Web-Live, performance steadily improves with larger training datasets, though the rate of improvement tapers off at higher data scales, suggesting potential diminishing returns. In contrast, Online-MindWWeb shows more consistent gains overall, following a slight performance drop with small-scale training data. These observations suggest that further scaling may continue to yield performance improvements. We omil scaling results on the sandbox environments VWA and focus our analysis on the other two online benchmarks, which more accurately reflect real-world environments and web agent tasks.
In-Domain Continual Training. For specific environments, we can syuthesize -omain-speccfic data for world model training, cnabling more specialized and contextually grounded simulations. In VWA (Koh et al., 2024a), we employ our data synthesis pipeline introduced in Section 3.3 synthesize 25K in-domain interactions for each of the three environments: Classifieds, Reddit, and Shopping. To prevent test data leakage, we filter out search actions containing queries that appear in test examples. The final in-domain checkpoints are contimually trained from the Dreamer-7B model, resulting in three separate world models, each specialized for its respective environment.
Classifieds
Reddits
Shopping
Total
Reactive Tree Search (Koh et al., 2024a)
14.3
19.3
17.6
26.8
20.6
28.9
26.2
GPT-1o
23.2
17.5
26.3
23.2
Qwen2- VL-7B
17.9
11.1
20.2
17.2
WebDreamer
Qwen2-VL-72B VL-72B
19.6
15.9
24.6
21.0
Dreamer-7B
21.4
15.9
25.4
21.9
+ In-Domain
25.0
15.9
26.3
23.2
Table 3: Success rate (%) of WEBDREAMER with various world models on VWA.
As shown in Table 3, continual training improves performance over the Dreamer-7B model and achieves results comparable or even better than GPT-4o in certain environments. The Classifieds and Shopping domains benefit. the most from in-domain adaptation, demonstrating that domain-specifie fine-tuning successfully refines model predictions to better reflect the specific environment dynamics. However, performance on Reddit remains unchanged, likely due to its dense viewport and limited functional interactions. Unlike Classifieds and Shopping, which have simpler visual organization, Reddit pages contain long, text-heavy content, making it difficult for a world model to infer meaningful state changes beyond surfaco-level text visibility. In addition, the limited perception abilities (Gou et al., 2025; Zhang et al., 2025) 7B models in dense web viewports may further constrain them simulate fine-grained changes text-heavy environments. These results underscore the effectiveness of domain-specifie adaptation while highlighting areas for further improvement in specific web domains. Future work explore representation techniques to better handle dense web layouts, further expanding the applicability of world models across diverse real-world web environments.
Case Studies. We present example to explore the differences between GPT-4o and Dreamer-7B as world models. GPT-40 provides more detailed state change descriptions and considers multiple possible outcomes, though some of these details are unimportant or irrelevant. For example, as shown in Figure 7, GPT-4o
We observed a slight improvement from planning horizon experiments performs comparably horizon while heing significantly less costly.
VWA (Appendix F.I), but chose horizon for main

---

Published in Transactions on Machine Learning Research (10/2025)
Drugs.com
GPT-40
Knsteriirn Secorh
A page section dedicated Dosage Information' for Humira will load. This section will likely include detailed information about the recommended dosages for various conditions, administration methods, and possibly warnings precuutions related to dosage. The puge might replace the current content or open in new tab'window. The page's URL will include terms related to Humira dosage information. Existing elements like the search bar, navigation menu, and advertisements might remain unchanged.
sepboundi
Consurse irformation
merts IN. inytorrtsonti contrri yru plecs pertes enco dhorciie jrent..
Dreamer-7B
The webpage will display detailed dosage information for Humira, including recommended dosages, administration instructions, and any specific precautions considerations for different patient populations.
Click "Dosage"
Figure 7: An example comparing GPT-4o and Dreamer- 7B as world models. General- models such as GPT-4o often provide detailed but sometimes noisy predictions, while our Dreamer- 7B focuses on concise, action-relevant outcomes.
successfully predicts the key state changes after clicking on "Dosage," outlining the relevant. information that may appear on the new webpage. However, its prediction also includes unimportant details such as URL and UI changes, which may introduce noise for the policy model. On the contrary, Dreamer-7B offers more concise and action-oriented predictions, offering the most important information on the new webpage.
To further clarify the role of simulation in planning,
also present case studies with GPT-4o world model, covering both positive and negative examples in Appendix F. They illustrate how simulation aids the agent in exploring the environment, as well as how inaccuracies in simulation can lead to incorrect predictions.
6 Conclusion
In this paper, we systematically explore the use of LLMs as world models for model-based planning in web environments. Our planning framework, WEBDREAMER, achieves substantial improvements over reactive baselines across three benchmarks and offers a 4-5 times more efficient yet competitive alternative to tree scarch, which is often impractical due to backtracking and efficiency constraints on rcal-world websites. Beyond leveraging off-the-shelf proprietary LLMs as world models, we train Dreamer-7B 3.1 million web interaction examples synthesized through our scalable data generation pipeline, achieving performance comparable to GPT-4o. Continual in-domain fine-tuning further allows Dreamer-7B to adapt to specific environments, improving simulation quality and downstream performance, even surpassing GPT-4o. This work lays the foundation for future research on model-based planning for efficient and effective decision-making in web environments and establishes a path toward scaling world models to handle more general web tasks, Overall, our results suggest that the answer to our title question is yes: LLMs can serve as effective world models for web planning, though there remains significant room for improvement in multi-step simulation.
Ethical Considerations
Our work focuses on model-based planning for web agents by using and training world models on web environments. Although the framework is developed for research purposes, we acknowledge potential risks if misused. For example, automated web agents could be misused for spamming, excessive scraping, or other harmful activities, and relcasing interaction data without safeguards could raise concerns about privacy or misuse. To mitigale these risks, we filter collected data to remove failed or harmful interactions, and no personally identifiable information or explicit content is retained. Our contributions are intended solely for academic rescarch, and we caution against deploying our system ways that compromise user privacy, security, or website integrity. We strongly encourage responsible use of this research within appropriate ethical and legal boundaries.
10

---

Published in Transactions on Machine Learning Research (10/2025)
References
Gilles Baechler, Srinivas Sunkara, Maria Wang, Fedir Zubach, Hassan Mansoor, Vincent Etter, Victor Carbune, Jason Lin, Jindong Chen, and Abhanshu Sharma. Screenal: A vision-language model for UI and infographics understanding. In IJCAT, 2024.
Mare G. Bellemare, Yavar Naddaf, Joel Veness, and Michael Bowling, The arcade learning environment: An evaluation platform for general agents. Journal of Artificial Intelligence Research, 47:253-270 2013.
Greg Brockman, Vicki Cheung, Ludwig Pettersson, Jonas Schneider, John Schulman, Jie Tang, and Wojeiech ZZaremba. Openai gym. arXiv preprint, Abss/600.015440 2016.
Bradley Brown, Jordan Juravsky, Ryan Ehrlich, Ronald Clark, Quoc V Le, Christopher Ré, and Azalia Mirhoseini. Large language monkeys: Scaling inference compute with repeated sampling. ArXio preprinl, abs/2447.21178, 2024.
Hyungjoo Chae, Namyoung Kim, Kai Tzu iunn Ong, Minju Gwak, Gwanwoo Song, Jihoon Kim, Sunghwan Kim, Dongha Lee, and Jinyoung Yeo. Web agents with world models: Learning and leveraging environment dynamics in web navigation. In ICLR, 2025.
Kanzhi Cheng, Qiushi Sun, Yougang Chu, Fangzhi Xu, Li YanTao, Jianbing Zhang, and Zhiyong Wu. SeoClick: Harnessing GUI grounding for advanced visual GUI agents. In ACL, 2024.
Xiang Deng, Yu Gu, Boyuan Zheng, Shijic Chen, Samual Stevens, Boshi Wang, Huan Sun, and Yu Su. Mind2web: Towards generalist agent for the web. In NewrIPS, 2023.
Xidong Feng, Ziyu Wan, Muning Wen, Stephen Marcus McAleer, Ying Wen, Weinan Zhang, and Jun Wang. Alphazero-like tree-search can guide large language model decoding and training. ArXiv preprint, hts//309.11179, 2023.
Hiroki Furuta, Kuang-Huei Huei Lee, Ofir Nachum, Yutaka Matsuo, Aleksandra Faust, Shixiang Shane Gu, and Izzoddin Gur. Multimodal web navigation with instruction-finctund foundation models. ArXiv preprint, abs/2305. 11854, 2023.
Carlos E Garcia, David M Prett, and Manfred Morari. Model predictive control: Theory and practice survey. Automatica, 25 25(3):335-344 348, 1989.
Boyu Gou, Ruohan Wang, Boyuan Zheng, Yanan Xie, Cheng Chang, Yiheng Shu, Huan Sun, and Yu Su. Navigating the digital world as humans do: Universal visual grounding for GUI agents. In ICLR, 2025.
Yu Gu, Xiang Deng, and Yu Su. Don't generate, discriminate: A proposal for grounding language models to real-world environments. In ACL, 2023.
Izzeddin Gur, Hiroki Furuta, Austin Huang, Mustafa Safdari, Yutaka Matsuo, Douglas Eck, and Aleksandra Faust. A real-world webagent with planning, long context understanding, and program synthesis. ArXiv preprint, abs/2307. 12856,, 2023.
David Ha and Jürgen Schmidhuber. Recurrent world models facilitate policy evolution. In NewrlPS, 2018
Danijar Hafner, Timothy Lillicrap, Jimmy Ba, and Mohammad Norouzi. Dream to control: Learning behaviors by latent imagination. In ICLR, 2020.
Danijar Hafner, Timothy P Lillicrap, Mohammad Norouzi, and Jimmy Ba. Mastering atari with discrete world models. In ICLR, 2021.
Shibo Hao, Yi Gu, Haodi Ma, Joshua Hong, Zhen Wang, Daisy Wang, and Zhiting Hu. Reasoning with language model is planning with world model. In EMNLP, 2023.
Hongliang He, Wenlin Yao, Kaixin Ma, Wenhao Yu, Yong Dai, Hongming Zhang, Zhenzhong Lan, and Dong Yu. WebVoyager: Building an end-to-end web agent with large multimodal models. In ACI, 2024.
11

---

Published in Transactions on Machine Learning Research (10/2025)
Wenyi Hong, Weihan Wang, Qingsong Lv, Jiazheng Xu, Wenmeng Yu, Junhui Ji, Yan Wang, Zihan Wang. Yuxiao Dong, Ming Ding, and Jie Tang. Cogagent: A visual language model for gui agents. In CVPR, 2024.
Doyoung Kim, Jongwon Lee, Jinho Park, and Minjoon Seo. Cognitive map for language models: Optimal planning via verbally representing the world model. ArXiv preprint, abs/2406.11227, 2024.
Jing Yu Koh, Robert Lo, Lawrence Jang, Vikram Duvvur, Ming Lim, Po-Yu Huang, Graham Neubig, Shuyan Zhou, Russ Salakhutdinov, and Daniel Fried. VisualWebArena: Evaluating multimodal agents on realistic visual web tasks. In ACL, 2024a.
Jing Yu Koh, Stephen McAleer, Daniel Fried, and Ruslan Salakhutdinov. Tree search for language model agents. ArXiv preprint, abs/ abs/2400.0014 2024b.
Hanyu Lai, Xiao Liu, Iat Long Iong, Shuntian Yao, Yuxuan Chen, Pengbo Shen, Hao Yu, Hanchen Zhang, Xiaohan Zhang, Yuxiao Dong, and Jie Tang. Autowebglm: A large language model-based web navigaling agent. In SIGKDD, 2024.
Kenton Lee, Mandar Joshi, Iulia Raluca Ture, Hexiang Hu, Fangyu Liu, Julian Martin Eisenschlos, Urvashi Khandelwal, Peter Shaw, Ming-Wa Chang, and Kristina Toutanova. Pix2struct: Screenshot parsing as pretraining for visual language understanding. In JCML, 2023.
Evan Zheran Liu, Kelvin Guu, Panupong Pasupat, Tianlin Shi, and Percy Liang. Reinforcement learning on web interfaces using workflow-guided exploration. In ICLR, 2018.
Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. In ICLR, 2019.
Marcelo G Mattar and Máté Lengyel. Planning in the brain. Neuron, 110(6):914 934, 2022.
Ethan Mendes and Alan Ritter. Language models can self-improve at state-value estimation for better search. arXiv preprint, fbs/2203.02878, 2025.
Thomas M Moerland, Joost Brockens, Aske Plaat, Catholijn M Jonker, et al. Model-based reinforcement learning: A survey. Foundations and Trends@ in Machine Learning, 16(1):1-118, 2023.
Vardaan Pahuja, Yadong Lu, Corby Rosset, Boyu Gou, Arindam Mitra, Spencer Whitchead, Yu Su, and Ahmed Awadallah. Explorer: Scaling exploration-drise web trajectory synthesis for multimodal web agents. arXiu preprint, abs/2502.11335 2025.
Jiayi Pan, Yichi Zhang, Nicholas Tomlin, Yifei Zhou, Sergey Levine, and Alane Suhr. Autonomous evaluation and refinement of digital agents. In COLM, 2024a.
Yichen Pan, Dehan Kong, Sida Zhou, Cheng Cui, Yifei Leng, Bing Jiang, Hangyu Liu, Yanyi Shang, Shuyan Zhou, Tongshuang Wu, and Zhengyang Wu. Webcanvas: Benchmarking web agents in online environments. ArXie preprint, //////206..2333 2024b.
Razvan Pascamu, Yujia Li, Oriol Vinyals, Nicolas Heess, Lars Buesing, Sebastion Racanière, David Reichert, Théophane Weber, Daan Wierstra, and Peter Battaglia. Learning model-based planning from scratch. Are preprint, abs/ 11077006777 2017.
Ajay Patel, Markus Hofmarcher, Claudiu Croveanu-Condeel Marius-Constanne Dinu, Chris Callison- Callison-Burch, and Sepp Hochreiter. Large language models can self-improve at. web agent tasks. ArXio preprint, abs/2405.20308 2024.
Pranav Putta, Edmund Mills, Naman Garg, Sumeet Motwani, Chelsea Finn, Divyansh Garg, and Rafael Rafailov. Agent q: Advanced reasoning and learning for autonomous ai agents. ArXiv preprint, abs/2208.07119, 2024.
Julian Schrittwieser, Ioannis Antonoglou, Thomas Hubert, Karen Simonyan, Laurent Sifre, Simon Schmitt, Arthur Guez, Edward Lockhart, Demis Hassabis, Thore Graepel, et al. Mastering atari, go, chess and shogi by planning with a learned model. Nature, 588(7839):604 6004 2020.
12

---

Published in Transactions on Machine Learning Research (10/2025)
Peter Shaw, Mandar Joshi, James Cohan, Jonathan Berant, Panupong Pasupat, Hexiang Hu, Urvashi Khandelwal, Kenton Lee, and Kristina Toutanova. From pixels to UT actions: Learning to follow instructions via graphical user interfaces. In NewrIPS, 2023.
Tianlin Shi, Andrej Karpathy, Linxi Fan, Jonathan Hernandez, and Percy Liang. open-domain platform for web-basod agents. In ICML, 2017.
World of bits: An
Yifan Song, Da Yin, Xiang Yue, Jie Huang, Sujian Li, and Bill Yuchen Lin. Trial and error: Exploration-based trajectory optimization of LLM agents. In ACI, 2024.
Yu Su, Diyi Yang, Shunyn Yao, and Tao Yu. Language agents: Foundations, prospects, and risks. Tn EMNLP: Tutorial, 2024.
Richard S Sutton. Dyna, integrated architecture for learning, planning, and reacting. ACM Sigart Bulletin, 2(4):160-163, 1991.
Evan Z Wang, Federico Cassano, Catherine Wu, Yunfeng Bai, William Song, Vaskar Nath, Ziwen Han, Scan M. Hendryx, Summer Yue, and Hugh Zhang. Planning in natural language improves LLM search for code generation. In ICLR, 2025.
Peng Wang, Shuai Bai, Sinan Tan, Shijie Wang, Zhihao Fan, Jinze Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Yang Fan, Kai Dang, Mengfei Du, Xuancheng Ren, Rui Men, Dayiheng Liu, Chang Zhou, Jingren Zhou, and Junyang Lin. Qwen2-vl: Enhancing vision-language model's perception of the world at any resolution. arXiv preprint, abs/2409.122 2024.
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed H. Chi, Quoe V. Le, and Denny Zhou. Chain-of-thought prompting clicits reasoning in large language models. In NewrIPS, 2022.
Tianci Xue, Weijian Qi, Tianneng Shi, Chan Hee Song, Boyu Gou, Dawn Song, Huan Sun, and Yu Su. An illusion of progress? assessing the current state of web agents. In COLM, 2025.
Jianwei Yang, Hao Zhang, Feng Li, Xucyan Zou, Chunyuan Li, and Jianfeng Gao. Sct-of-mark prompting unleashes extraordinary visual grounding in gpt-4v. ArXiv preprinl, abs/2310.114411, 2023.
Shunyu Yao, Howard Chen, John Yang, and Karthik Narasimhan. Webshop: Towards scalable real-world web interaction with grounded language agents. In NewrIPS, 2022.
Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Tom Griffiths, Yuan Cao, and Karthik Narasimhan. Tree of thoughts: Deliberate problem solving with large language models. In NewrIPS, 2023a.
Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik R. Narasimhan, and Yuan Cao. React: Synergizing reasoning and acting in language models. Tn ICLR, 2023b.
Keen You, Haotian Zhang, Eldon Schoop, Floris Weers, Amanda Swearngin, Jeffrey Nichols, Yinfci Yang, and Zhe Gan. Ferret-ui: Grounded mobile UI understanding with multimodal llms. In ECCV, 2024.
Kai Zhang, Jianwei Yang, Jeevana Priya Inala, Chandan Singh, Jianfeng Gao, Yu Su, and Chenglong Wang. Towards understanding graphical perception in large multimodal models. arXiv preprint, abs/2503.108. 2025.
Yao Zhang, Zijian Ma, Yunpu Ma, Zhen Han, Yu Wu, and Volker Tresp. Webpilot: A versatile and autonomous multi-agent system for web task execution with strategic exploration. ArXiv preprint, abs/2408.1199) 2024.
Boyuan Zheng, Boyu Gou, Jibyung Kil, Huan Sun, and Yu Su. Gpt-4v(ísion) is a generalist web agent, if grounded. In ICML, 2024.
Shuyan Zhou, Frank F. Xu, Hao Zhu, Xuhui Zhou, Robert Lo, Abishok Sridhar, Xianyi Cheng, Tianyue Ou, Yonatan Bisk, Daniel Fried, Uri Alon, and Graham Neubig. Webarena: A realistic web environment for building autonomous agents. In ICLR, 2024.
13

---

Published in Transactions on Machine Learning Research (10/2025)
Overview
The appendix includes the following sections:
Section A: Author Contribution Statement.
Section B: Implementation Details of WebDreamer.
Section C: Implementation Details of Data Synthesis and World Model Training.
Section D: Intrinsic Evaluation of World Models.
Section E: More Discussions.
Section F: Case Studies.
A Contribution Statement
Yu Gu conceived the project with Yu Su and developed the planning framework WEBDREAMER. Hc implemented the core codebase, led the design and execution of key experiments including ablation studies, and wrote the initial manuscript draft. He also led the public release of the resources.
Kai Zhang led the data synthcsis and world model training efforts, inchuding the training of Dreamer-77 and donain--peciiic variants. conducted experiments on VisualWebArena and played a major role editing and polishing the manuscript to its final form.
Yuting Ning led the development of the intrinsic evaluation suite and conducted experiments Online- Mind2Web and Mind2Web- Live. She also contributed significantly to the final manuscript refinement.
Boyuan Zheng contributed to the development of the planning framework, the data synthesis pipeline, and the in-domain world model training. He also provided valuable insights into the experimental design.
Orby AI team (Cheng Chang, Sanjari Srivastava, Yanan Xie, and Peng Qi) provided large-scale compute resources and data crawling support, which were essential for scaling up the data synthesis and training pipeline. Peng Qi and Yanan Xie also provided constructive feedback throughput the project.
Yu Su and Huan Sun steered the main directions throughout the project, led discussions the planning framework and experiments, and provided funding support Yu Su conceived the project with Yu Gu.
All authors reviewed the manuscript and provided feedback.
B Prompts for Four Stages in WebDreamer
B.1 Action Proposal
Action Proposal
You are an autonomous intelligent agent tasked with navigating web browser. You will be given web based lasks. These
will be accomplished through the use specific actions you can issue.
Here's the information you'll have: (Web Information)
The user's objective: (Task Objective] This is the task you're trying to complete.
The current web page screenshot: (Web Page Screenshot Image) This is screenshot of the webpage, with each interactable element assigned a unique numerical id. Each bounding box and its respective id shares the same color.
The observation, which lists the IDs all interactable elements on the current web page with their text content if any, in the format [id] [tagType] [text content]. tagType is the type of the element, such as button, link, or lextbox. text content is the text content of the element. For example, [1234] [button] [ Add to Cart'] means that there is a button with id 1234 and text content 'Add Cart' the current web page. [] [StaticText] [text] means that the element is of some text that not interactable.
14

---

Published in Transactions on Machine Learning Research (10/2025)
The web page's URL: {Web URL} This is the page you're currently navigating.
The open tabs: (Previous Tabs} Thesse are the tabs you have open.
The previous action: (Previous Action] This is the action you just performed. may be helpful to track your progress.
The actions you can perform fall into several categories:
Page Operation Actions:
-click [id]: This action clicks on an element with a specific id on webpage.
type [id] [content]; Use this to type the content into the field with id. By default, the Enter key is pressed after typing unless press_ _arrs____err___ enter_after _afterr is set to 0, i.e., type [id] [content] [o].
hover [id]: Hover
element with id.
press [kee__cmbl]] Simulates the pressing of key combination on the keyboard (c.g.; Ctrl+V)
scroll [down] scroll [up]: Scroll the page up down.
Tab Management Actions:
new_tab: tab: Open new, empty browser tab.
facu_oocus [taab_ne :ab_index]: Switch the browser's focus to a specific tab using its index.
close_tab: Close the currently active tab.
URL Navigation Actions:
goto [url]: Navigate to a specific URL.
go_back: Navigate to the previously viewed page.
go_forward: Navigate to next page (if previous _o_ back action was performed).
Completion Action:
stop [answer]: Issue this action when you believe task complete. the objective is find text-based answer, provide the answer in the bracket.
Homepage:
If you want to visit other websites, check out the homepage at http:/ http://hoompage..oom It has list of websites you can visit. httt::/homepts htthhh/mmpee.cooo www.//oow/w.......ommmmo lists the account name and password the websites. You can use them to log in to the websites.
very important follow following rules: 1. You should only issue action that is valid given the current observation
successful,
2. You should only issue one action at time.
3. You should follow the examples to
step by step and then issue the next action. 4. Generate the action the correct format, Start with a "In summary, the nert action will perform is"
phrase, followed by action. For example, In summary, the next action I will perform is click [1234].
5. Issue stop action when you think you have achieved the objective. Don't generate anything after stop.
B.2 Self-Refinement
Self- Relleeefinemnnt
You are assisting web navigation agent to help human user navigate a website to complete a task. Given the user's intent, the action history, and the current state of the webpage, the agent has proposed of candidate actions to take at the current step.
Your role to determine best action for the agent this step, but filter the actions that very likely not relevant or helpful for the agent to accomplish the task.
Please select all actions that you think that could possibly lead the agent accomplish the task. It's important to note that to accomplish a task, the agent will execute sequence of actions. So the action to take at this step does have immediately lead the completion of the task. You should select any action that could be relevant for the agent to take in the current state of the webpage. Try to be as thoughtful and comprehensive as you can! Don't miss any possible action. If there one action that clearly the best, and all other actions are clearly not very relevant, you only select one action. Please do this sparely, since some actions may helpful in a longer horizon.
15

---

Published in Transactions on Machine Learning Research (10/2025)
An action should be included long it could be relevant to the task, if it may not be the most direct action to take at this step!! Some relevant actions might seem indirect at the first glance, but could be helpful in longer horizon. Please also include those actions. Please at least select one action.
"IMPORTANT*
Format your response into two lines as shown below:
Thoughts: <your thoughts and reasoning process?. You must explicitly evaluate each action one by and imagine whether it could be relevant to the task following the format: action:... rationale:...
Selected actions: 1d0; 1d0;;dd;aain aidd;;... (please return the index of the action in the candidate actions list, starting from 0. Don't output the action description itself. Separate the indices with semicolons. Do not add spaces any other characters after the semicolons.)
Action History: dlast_actions..
Current URL: (current_ur]]
The images corresponding to the user intent are shown the FIRST (oantinnt__maaeeee images (before the User Intent).
The last {len (screenshots)) snapshots of the agent's trajectory are shown the LAST {len (screenshots)) images. The LAST IMAGE represents the current state of the webpage.
Proposed Action: inreiiondeecripttion
B.3
World Model
World Model (state transition function)
You are an agent that predicts the effect of an action on a webpage. You will be given a screenshot of a webpage, a sequence of actions and state changes applied the initial screenshot, and an operation perform on the webpage. You are required to predict the new changes that will occur on the webpage after the operation performed, such as the appearance new elements, the disappearance of existing elements, or changes in the content of existing elements. The operation type and the element to operate will be provided in the prompt. Directly output State changes:... and don't output anything else. Try to as comprehensive and detailed possible.
Based on the initial screenshot and the changes to webpage, please predict changes action:
B.4 Reward Model
Reward Model (score)
You are an expert evaluating the performance web navigation agent. The agent designed help a human user navigate website to complete task. Given the user's intent, the agent's action history, the current state of the webpage, your goal is to decide * **whether the simulated steps by the agent indicate successful execution of the user intent** In particular, if the predicted state (i.e., the current state represented by the last image plus all the predicted changes so far) corresponds to successful final state. If it is failure but it looks like the simulated steps as such. Note that, in the simulated steps, all the state changes are predicted by the agent's is world model, and they may actually be faithful the real website interactions (c.g., some proposed actions may not be available in a realistic website). You should also account for this in your evaluation (e.g., if the predicted state changes are not reasonable then it's probably failure).
the right track towards success, you should also output
16

---

Describe what you will see after {action]ing Published in Transactions on Machine Learning Research (10/2025)
"IMPORTANT*
Format your response into two lines as shown below:
Thoughts: <your thoughta and reasoning procesa>
Status: "success" or "failure"
On the right track success: "yes" "no"
C Data Synthesis and World Model Training
C.1 Data Synthesis Prompt
Prompt Templates
Here is the web screenshot <image token>, Please describe what you would see after performing (action)
on (element_deeeeeeeeeeeeeeeeeeee__ .edecriptioo......
Here is the web page you are looking at <image_token>> Please describe what you would after doing
(action] on [element_descriptiion descriptiinn.
Here is the web page are currently at <image >. Describe what you will see after {action)ing {element_ jennrppddecriptioo]]
Here is the current web page <image_token>>. token>. Briefly describe what you will see after [action]]ig
Below the current screenshot <image_token>>.
token>. Briefly describe what you will after (action]ing
Below is the current screenshot <image_tokeer token.>.
(element_dene deameettdddsrritioo)).
Below is the current screenshot <image token>> Please describe what you would see after a {action] on
(element_desel demeent_descriptionn)
Table C.1: Prompt templates used to generate language descriptions of state transitions.
In practice, we first draw red bounding box around the target element to preciscly localize it for Qwen2- VL-72B to separately describe the element using referring expression and
VL-72B. Next, we prompt Qwen2-VL-72B to describe the resulting state change. Finally, we combine the referring expression and the state change description to construct the training instance, using one of the templates randomly selected from Table C.1. We list the prompt used to syuthesize natural language descriptions of the next state below.
Despite the fact that our training data includes only few prompt templates and natural images, experiments in Section 4 have shown that the model generalizes well to unseen instruction prompts used in benchmarks like Onndne-Mind2Web (Xue et al., 2025) and Mind2Web-Live (Pan et al., 2024b) and to images with Set-of-Mark (Yang et al., 2023) in VisualWebArem (Koh et al., 2024a).
17

---

Published in Transactions on Machine Learning Research (10/2025)
Data Synthesis
Here are two webpage screenshots from the website {URL) before and after [Action) on the element within the red bounding box.
**Eeemen Description:** ** Please describe the element within bounding box to ensure user can locate this element in the webpage image only using this description (only element description, not bounding box). So DO NOT say something like the element within the bounding box, DO NOT SAY anything about red bounding box.
Instead, describe the element with referral expression like the button showing the text 'Make Appointment or the 'Search' in the search bar.
Starts with lower case and make sure the description a noun like the element that is a category link labeled 'Massage' located the sidebar the left side of the page.
** CChange Description:* Explain to the user what changes will occur on the webpage after they click on the described element. Do not say too many trivial details. Instead, give high-level and functional description in detail after the action. Focus solely on describing the changes that will happen, not the element.
C.2 Training Data Statistics
Number
Percentage
Unique URLs
1,247,960
Action
Click
2,653, 704
84.0
Hover
241,234
7.6
Type
217,692
6.9
Select
47,617
1.5
Total
3,160,247
100
Table C.2: Training data distribution.
As described in Section 3.3, develop scalable pipeline to synthesize large-scale interaction data by randomly exploring web pages across wide range of domains. This process results in over 3.1 million interaction instances spanning 1.2 million unique URLs, as summarized in Table C.2. We focus on four primary interaction lypes: click, hover, lype, and select, which collectively reflect the core user intents on modern web interfaces. Clicks dominate the dataset (84%), consistent with their central role in triggering state transitions on the web. Although our data synthosis pipeline initially included scroll actions, we empirically found that they contributed little downstream performance and thus excluded them from the final training set.
Importantly, the interactions in our dataset are not restricted to initial steps but span various stages within multi-step trajectories. Figure C.1 shows the distribution of actions by their position in the interaction sequence. The average position is 4.4, and a substantial portion of actions occur deeper in the trajectory. This distribution allows the world model to learn more informative web dynamics and have the potential to enable long-term planning.
C.3 World Model Training
(Wang et al., 2024) backbone. To align with future state prediction, we format training examples using structured prompts, such as: "Here is the web screenshol. Please describe what you would see after performing (action} on (element). All experiments are conducted on 64 H100 GPUs with S0GB memory each. The final Dreamer-7B model is trained for up epochs over the full training dataset described above. We evaluate models every 1000 steps using an intrinsic evaluation metric (described later) to ensure fair and consistent model selection across settings. We use the AecoupledAdamW (Toshchilov & Hutter, 2019) optimizer with learning rate of 1e-6, B1 = 0.9, and
All world models are fine-tuned using the Civen2-VL-7B-lmmee
18

---

Published in Transactions on Machine Learning Research (10/2025)
Average Position: 4.4
16
500
14
12
10
Percentage Tota
Count (k)
300
6
200
100
0
2
2
5
7
8
10
11
Action Position in Trajectory
Figure C.I: Distribution of the step which actions occur within multi-step trajectories. The actions used for training span across different depths, not being limited to initial steps.
82 0.95. The learning rate follows cosine decay schedule with 1000 warmup steps, decaying 10% of the base LR. We use global batch size of 192 and enable mixed-precision training.
For the three domain-specific world models (Classifieds, Reddit, and Shopping), continue training from the Dreamer-7B checkpoint for one epoch using the respective in-domain datasets. These models follow the same training sctup, except with reduced lcarning rate of 5e-7 and a shorter warmup of 100 steps to ensure stable adaptation on smaller datasets.
Intrinsic Evaluation of World Models
While downstream evaluation with real-world web tasks can provide valuable insights for the model-based planning framework, it involves many other factors (e.g., the policy model) that may influence performance other than world models. Therefore, to rigorously evaluate world models only, we construet intrinsic evaluation that provides a more controlled and independent assessment while maintaining alignment with the downstream evaluation setting. When training our world models, we use this intrinsic evaluation for model development and checkpoint selection.
D.1 Dataset Construction
We first sample tasks from Mind2Web (Deng et al., 2023) and manually annotate the trajectories that can complete the tasks ground truths. Then, generate deviation trajectories at different states in the ground truth trajectories and assess whether the world model can help distinguish the correct actions from incorrect actions at cach state. Specifically, for cach state on the ground truth trajectory, we use the ranking model in MindAct (Deng el al., 2023) to select lop-5 candidate elements from the webpage that align best with the task intention and current state, excluding the ground truth one. These deviation actions are then extended into full trajectories using web agent (i.c., SecAct (Zheng al., 2024)), automatically evaluated with an LLM-as-a-judees method (Pan al., 2024a). Since multiple paths may lead task completion, we filter out deviation actions successful trajectories to alleviate false negative issue and only retain those in failed
19

---

Published in Transactions on Machine Learning Research (10/2025)
trajectories as negative actions. Using this pipeline, we construct an intrinsic evaluation dataset of 44 tasks, 141 states with deviations, and 279 deviation actions.
D.2 Evaluation Metrics
For evaluation, we use world models to simulate possible future states for each action and score them using the same scoring function in the planning framework. We then compute three types of metrics: (1) pair-wise accuracy: For each pair consisting of ground-truth action and a deviation action within the same state, if the ground-truth action receives a score greater than or equal to the deviation action, it is counted as correct; otherwise, it is incorrect. Pair-wise accuracy is the proportion of correct pairs across the whole dataset, (2) state-level accuracy: Evaluates whether the world model consistently ranks the ground-truth action above all deviations within given state. A state is considered correct only if the ground-truti action has a score greater than equal to all its corresponding deviation actions. (3) task-level accuracy: Assesses the correctness of an entire task by ensuring that the world model helps correctly select ground-truth action in every state within the task. A task is considered correct only if all its states are correct according to state-level accuracy criteria. We use state-level accuracy as the primary metric to select the best checkpoint, which closcly aligns with the downstream task forms.
D.3 Results
World Model
Pair-wise Accuracy
State-level Accuracy
Task-level Accuracy
GPT--4omini
85.30
78.01
47.73
GPT-4o
87.10
80.85
52.27
Qwen2-VLL VL-7B
86.38
80.85
50,00
Qwen2- VL-72B
87.10
80.14
47.73
Dreamer-7B
88.53
82.98
52.27
Table D.1: Results (%) of various world models on the intrinsic evaluation set.
Table D.1 shows the results the intrinsic evaluation. Our fine-tuned 7B model achieves comparable performance on task-level accuracy with GPT-4o and even outperforms GPT-4o in terms of pair-wisc and state-level accuracy. The scaling trend of training world models can also be observed in intrinsic evaluation shown in Figure D.1.
For all world models we have evaluated on both intrinsic evaluation and Mind2Web-Live (Pan et al., 2024b), the Pearson correlation coefficient between task-level accuracy and task success rate is 0.8455, indicating a strong correlation between intrinsic evaluation and downstream performance. We hope our intrinsic evaluation can serve as a useful tool for advancing web world model development.
E
More Discussions
E.1 State Representation and Planning Horizon
In addition to the state change description used in our primary experiments, we explore alternative approaches where GPT-4o predicts either the HTML code or the accessibility tree of the resulting webpage within the simulation. For each of these state representations, we evaluate planning horizons of 1, 2, and 3 steps VWA benchmark. As depicted in Figure E.1, all three state representations significantly outperform the reactive baseline. However, their effectiveness diminishes the plamning horizon extends to 3 steps, indicating common limitation in long-horizon simulation across these approaches. Notably, the state change representation exhibits the most pronounced performance degradation as planning horizons extend. This decline particularly severe with a planning horizon of 3, where performance falls below that the reactive bascline. This vulnerability stems from its implicit specification of available interactive elements on the current webpage, requiring the model infer these elements by applying changes the initial state. In contrast, HTML and accessibility tree representations provide explicit element. information. Consequently,
20

---

Published in Transactions on Machine Learning Research (10/2025)
Mind2Web-Live
Intrinsic Evaluation
Maline-Mind2Wen
55
50
50
Success Rate (%)
45
45
40
40
35
35
30
30
25
25
20 (es
20
15
0.00
0.63
1.30
3.16
# Synthetic Training Data (M)
Figure D.1:
Performance on two downstream benchmarks and intrinsic evaluation w.r.t. different sizes of training data.
30
28
Success Rate (%)
26
State Change
Accessibility Tree
HTML
24
22
Reactive Agent
20
Planning Horizon
Figure F.1: Performance of WEBDREAMER on the human-verified shopping subset of the VWA dataset, varying both the state representation within simulations and the planning horizon. Planning with long horizon with simulation remains challenging, regardless of the state representation employed.
the state change approach is more susceptible to hallucination during extended simulations. Despite this limitation, the state change approach remains viable choice given the current capabilities of LLMs. It matches the performance of HTML and accessibility tree representations for planning horizons less than 3 while consuming fewer output tokens.
21

---

Published in Transactions on Machine Learning Research (10/2025)
F Case Studies
F.1 Error Caused by Imperfect World Model Simulation
An error case caused by imperfect world model simulation is shown in Figure F.1.
Find me printer of the same brand as the product in this picture. needs be white and have least 11 reviews with an average rating greater than
The webpage will update to display ttmm 25-36 1272 for the cument categocy or search crtieia. The product images, namess, prices, and other related details will change to reflect this nest wit thems The pagination the bettom the page will indicatte that the user is now on the thind page of items, with Page likely highlighted or noted some way. The prodlccts and thair detail from lsems 13:24 will be replaced by the information for items 25-36.
th resulls preterr
A new set of sourch noulls for 'HP peintee will appeas displaying the rest items starting from ttem 15 and onwands The pagination namber and the item count at the top of the scarch resalts may updase accordingg)- The "Resently Ordened and "Comppane Products" sections, if static, of the may shift
move view, while sections night into view. The fouter suction scrolling naches the bottom.
Scroll Down
Correction
serolling section Privacy and
displaving,
Advansed Ordens
Returne, Contect akon with stberigtion bus upclattes new products uppoar on page wher senoling.
listing "HP lnkjet 2800 Wide Format
webpage.
Stop
image, (537.6) mitngg stars, reviews pocetioned bottom right (idements through 65). semaining products would accordingly to the gallery space.
v.1
1 Click Pagy
Figure F.1: An error caused by imperfect world model simulation.
F.2 Positive Case Benefiting from World Model Simulation
A positive case where the simulation leads to correct action prediction is shown in Figure F.2.
22

---

Published in Transactions on Machine Learning Research (10/2025)
What are the two types of birds on the front of that colorful
The pape will navigate to detalled product papy for the "Mens Flowers Casual Aloha Llawalian Shirt Summer Short Steere Beach T-Shirt Regular Fit Button Down Dasa Shirh." This additional information about the product including more detalled specifictions, codcener reviews, lerger imagee, sixing possibly 'Add Cart" button, Other categury spacifii product.
will läady suntain
and prsducts mplaaed detabed
Hovering over likaly
following changes:
tooligp or additional information pepup might appear tear owr the shirts image or element. This popup could include more details such
Hover
the price, sise options, availabiiny, and possibly zounddin view the shirt.
Flover
2. The image of the shirt might changy, powiilly showing different angle model wearing the shirt, providing more context on how it looks when woen.
3. The 'Add Cart button or rating might change appearano, putentially becoming highlighted do attention..
Click
Stop
Pamott and palm will returned the
0.25
Figure F.2: A positive case where the simulation leads to correct action prodiction.
23


Reviews:
[{"review_id": "H0TZRDjaUD", "reviewer": "Reviewer_YXpy", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper presents a framework for model-based planning in web\ninteraction domains leveraging LLMs for world modeling and control; as\nwell as a new dataset for fine-tuning the LLM world model. The method\nuses bootstrapped value predictions and action pruning / refinement to\navoid expensive inference-time search. Through a series of experiments\nin challenging web interaction benchmarks, the proposed method\nachieves substantial performance improvements over reactive agents,\nand particularly, their 7B-parameter fine-tuned model on their\nproposed dataset performs roughly on par with their\nframework instantiated with GPT-4o.", "claims_and_evidence_yn": "Yes", "claims_explanation": "Despite certain instances where I felt that terms were not explicitly defined in the main text, largely I am convinced that the claims are accurate and supported by clear evidence. Particularly, the authors included very informative ablation studies to justify the effect of each of the components of their proposed methods. Moreover, they included several control studies / baselines to further paint the picture of which components are crucial for success. I have no major questions or concerns about the results or their correctness.\n\nI also really appreciated the discussion about how the fine-tuned model was evaluated/monitored during training (e.g., via evaluations on a fixed held-out set of tasks).\n\nMy one complaint regarding the evidence is that there are no confidence bounds on their empirical results, which makes it difficult to judge statistical significance. However, given the expensive nature of the experiments, perhaps only so much can be done towards this end.", "audience_yn": "Yes", "audience_explanation": "The paper studies an important problem domain that is at the frontier of agentic LLM research; surely a large portion of the TMLR audience will be interested in the findings of this paper. Moreover, while the paper focuses explicitly on planning for web interaction, the framework it presents can similarly be applied (and be interesting) for general complex planning problems that involve a large multi-modal action space.", "requested_changes": "1.  Nit: the title doesn't feel right to me, particularly w.r.t. \"Is\n    your LLM Secretly a World Model of the Internet?\". Firstly, a\n    priori, it's not clear what this question is really asking. Almost\n    trivially, an LLM is designed to be a world model of the internet,\n    so it seems that the answer should be simultaneously \"yes\" and \"it\n    is not even a secret\". Do you mean something more precise than this?\n    And then, perhaps more importantly, it is also not clear to me\n    whether or not your paper answers this question (indeed, I still\n    don't know if you intend for the answer to be \"yes\" or \"no\").\n2.  Nit: In section 3.1, it says \"using a learned simulation function\n    $\\mathtt{sim}(o, a)$\". However, $\\mathtt{sim}(o, a)$ is not the\n    function $\\mathtt{sim}$ is. It would be helpful to write instead\n    \"using a learned simulation function\n    $\\mathtt{sim}:\\mathcal{O}\\times\\mathcal{A}\\to(\\mathcal{O}\\times\\mathcal{A})^*$\",\n    which (1) is precise, and (2) also tells us the output type of this\n    function. Indeed, I originally incorrectly assumed that\n    $\\mathtt{sim}$ outputs observations, and not trajectories.\n3.  Nit: in general POMDPs, I believe $\\Omega$ is generally\n    non-deterministic, but with your notation $o = \\Omega(s)$, it\n    appears is deterministic in your setting. This is fine, but perhaps\n    it should be stated explicitly.\n4.  Similar comment as above for \"a scoring function\n    $\\mathtt{score}(\\tau)$\" (e.g., the function should just be\n    $\\mathtt{score}$). This instance is more urgent since the variable\n    $\\tau$ here has not yet been introduced (so its type is unknown).\n5.  In Algorithm 1 (and in the definitions preceding it, shouldn't\n    $\\mathtt{sim}$ and $\\mathtt{score}$ also be conditioned on\n    $I$? Particularly in the case of $\\mathtt{score}$, I would\n    expect the scoring function to highly depend on the instruction\n    (whereas perhaps with horizon 1 or 2 as you use, maybe\n    $\\mathtt{sim}$ is less sensitive…).\n6.  Algorithm 1 includes a call to `self_refine`, which isn't explicitly\n    discussed in the text. To the left of the pseudocode, there is a\n    very brief discussion on self-refinement, but no implementation /\n    explicit details are given, unlike in the cases of `sim` and\n    `score`. The only such information I could find is with the example\n    of the prompt used for self-refinement in Appendix A.2. This should\n    be discussed more explicitly in the main text, since you indeed\n    found this intervention to be very useful later on in your\n    ablations.\n7.  Why is the method called WebDreamer? Besides its model-based nature,\n    it seems that WebDreamer has effectively nothing in common with the\n    Dreamer methods of Hafner et al. (e.g., it is planning in\n    observation space, etc).\n8.  In section 3.3, you discuss your \"scalable data synthesis pipeline\".\n    This (or at least the data it generated) appears to be a primary\n    contribution, but it is not described in much detail. What makes it\n    scalable? What does it mean to \"\\[favor\\] frequent interactions like\n    clicking while ensuring sufficient coverage of others\"? What does it\n    mean to \"encourage causal dependenies\"? I understand what these mean\n    intuitively, but how do you *encourage* these things in practice?\n    How do you know which web actions are possible at any given point in\n    time?\n9.  At the end of section 3.3, explain what you mean by \"checkpoint\n    selection\".\n10. Excuse me if I missed it, but I see no description of the \"reactive\"\n    baseline. Is this just prompting an LLM (and which one?) for the\n    next action given an instruction and current observation, and\n    playing it without planning? I see the description of general\n    \"reactive agents\" above, but nothing that says explicitly which\n    reactive agent is used in your experiments.\n11. In the \"effectiveness\" subsection of Section 4.2, I don't understand\n    how you're coming up with the stated performance gains. For example,\n    it says \"on the VWA dataset, our proposed method achieves a 34.1%\n    relative performance gain\". How is that measured?", "broader_impact_concerns": "No broader impact concerns.", "additional_comments": "In order to provide some statistical confidence bounds for your results, given how many individual tasks you benchmark on, is it not plausible to use the stratified sampling approach of [1]? \n\n**References**\n1. *Deep Reinforcement Learning at the Edge of the Statistical Precipice*. Rishabh Agarwal, Max Schwarzer, Pablo Samuel Castro, Aaron Courville, Marc G. Bellemare. NeurIPS, 2021.", "extra_scores": {}, "extra_text": {}}, {"review_id": "aSoVDI11N8", "reviewer": "Reviewer_mAEQ", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper presents WebDreamer, a model-based planning framework for web agents that leverages a large language model (LLM) as a world model to simulate the outcomes of web actions prior to execution. The framework demonstrates that both GPT-4o and a domain-tuned model, Dreamer-7B, are capable of simulating web page transitions—such as those resulting from clicking or typing. Empirical results show that Dreamer-7B outperforms both tree search and reactive baselines.", "claims_and_evidence_yn": "Yes", "claims_explanation": "This paper clearly articulates its design and findings. Key design choices—such as planning horizon, scoring functions, and self-refinement—are logically motivated and well justified. The ablation studies effectively demonstrate the impact of planning depth, LLM throughput, and self-refinement. Furthermore, potential limitations, such as the compounding of hallucinations with longer planning horizons, are acknowledged and discussed transparently.", "audience_yn": "Yes", "audience_explanation": "The main theme of this paper centers on large language models (LLMs), AI agents, and world models—all of which are highly active and timely research topics in the current AI landscape.", "requested_changes": "n/a", "broader_impact_concerns": "The ability to simulate website behavior with high fidelity introduces potential risks of misuse, including: modeling phishing infrastructure, automating web scraping or bypassing authentication mechanisms, and exploiting vulnerabilities in interactive systems.\nRecommendation: The paper would benefit from a clearer and stronger disclaimer outlining appropriate use boundaries, along with a firm commitment to ethical deployment.\n\nMoreover, the proposed model could ultimately enable autonomous browsing agents capable of acting without user oversight and modifying websites in real time—for instance, submitting forms or making purchases.\nRecommendation: The authors should explicitly address safeguards such as requiring user permission, implementing sandboxed environments, or adopting human-in-the-loop mechanisms to ensure transparency, accountability, and user control.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "0ZOnnYHxLM", "reviewer": "Reviewer_QEPd", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper introduces a framework, WebDreamer, which applies an LLM to simulate next states and to score simulated trajectories before taking actions. In this framework, the LLM replaces the role of the traditional model (transition kernel) and the value function used in RL, serving as the concept of the World Model.  The proposed method empirically improves the performance over some benchmarks.\n\nStrength: Clear motivation. New idea. Include both training-free and trainable models to validate the performance.\n\nWeakness: Lack of other baselines. The apprach seems to be more general than Web Agents.", "claims_and_evidence_yn": "Yes", "claims_explanation": "**strengths:**\n1. Motivation: Figure 1 clearly explains the motivation of proposing the core idea of this paper, which addresses the existing issues appearing in the Reactive/Tree-Search agents. \n\n2. New idea: While the LLM agent is a crowded field, I definitely believe the same idea has been proposed somewhere by someone. But the specific focus on the WebAgent should be new, and the web agent seems to be more challenging than the purely text-based benchmark. \n\n3. The LLM can be used as the \"World Model\" in the Web Agent Benchmarks. The concept of world model is apparently abused in the literature; however, in this paper, the world model is clearly described as how the model \"Implementation for sim\" and \"Implementation for score\" work, which presents a strong formulation and the fundation to support this claim.\n\n     Continuing the problem setup, this paper investigates how this perspective (LLM as the World Model) helps improve the performance of Web Agent in the standard benchmarks. The experimental results include both the training-free (using GPT-4o) and training approach (using Dreamer-7B), which strongly support the claim made in this paper.\n\n\n**weaknesses:**\n\n1. Only two baselines are included in the comparison: Reactive and Tree Search. For example, in the VisualWebArena or Online-Mind2Web, it has already included multiple other baselines in their Leaderboard. I can see some baselines are super strong. \n\n    It is not necessary for the proposed method to outperform approaches that are explicitly tailored to this specific task. However, adding some discussion to help readers better understand what other people is doing given the main issue in Reactive or Tree Search. \n\n2. The described approach, \"Implementation for sim.\" + \"Implementation for score.\", appears applicable to arbitrary tasks. While the authors choose to focus on the Web Agent setting, I am curious: what distinguishes the Web Agent from other tasks? Do the authors believe the same idea (i.e., treating the LLM as the world model) would succeed or fail in other task domains?", "audience_yn": "Yes", "audience_explanation": "As LLM Agent is a hot topic, I am sure many individuals in TMLR's audience will be interested in knowing the findings of this paper.", "requested_changes": "Requested Changes:\n\n* Add more discussions for other approaches in the Web Agent: Explain on how they are related to the web agent strategies (Reactive and Tree Search described in Figure 1). Explain why the World Model approach is potentially better than these approaches. \n\n* Help me understand when the World Model approach will fail. For example, give me an example showing that the LLM cannot  serve as a world model and or a value function in some cases.", "broader_impact_concerns": "No broader impact concerns for this paper.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Decision: Accept (poster)