WARC-Bench: Web Archive based Benchmark for GUI Subtask Executions
Authors: Sanjari Srivastava
Gang Li
Cheng Chang
Rishu Garg
Manpreet Kaur
Charlene Y. Lee
Yuezhang Li
Yining Mao
Ignacio Cases
Yanan Xie
Peng Qi
Venue: ICLR 2026 Poster (2026)
Keywords: web navigation benchmark
reinforcement learning
Primary area: datasets and benchmarks
TL;DR: Establish new interactive web-archive based benchmark to measure GUI agent performance on subtasks

Abstract: Training web agents to navigate complex, real-world websites requires them to master subtasks—short-horizon interactions on multiple UI components (e.g., choosing the correct date in a date picker, or scrolling in a container to extract information). We introduce WARC-Bench (Web Archive Benchmark), a novel web navigation benchmark featuring 438 tasks designed to evaluate multimodal AI agents on subtasks. WARC-Bench enables sandboxed interactions with dynamic and realistic webpages using Web ARChive files. We show that WARC-Bench is challenging for leading computer-use models, with the highest observed success rate being 64.8%. To improve open source models on subtask, we explore two common training techniques: supervised fine-tuning (SFT) and reinforcement learning with verifiable rewards (RLVR). Experiments show that SFT models obtain a 48.8% success rate on the benchmark. Training with RLVR over SFT checkpoints, even in data-scarce settings, improves the score to 52.8% on WARC-Bench, outperforming many frontier models. Our analysis concludes that mastering these subtasks is essential for robust web planning and navigation, and is a capability not extensively evaluated by existing benchmarks. More details about WARC-Bench can be found at https://sanjari-orb.github.io/warc-bench/.

Published as a conference paper at ICLR 2026
WARC-BENCH: WEB ARCHIVE BASED BENCHMARK FOR GUI SUBTASK EXECUTIONS
Sanjari Srivastava, Gang Li, Cheng Chang, Rishu Garg, Manpreet Kaur, Charlene Y. Lee; Yuezhang Li, Yining Mao, Ignacio Cases, Yanan Xie & Peng Qi Uniphore (sansri 2 64 @gma i 1o.1..comm com, leemagpie@gmal com, cheng www.cchnnnnnnno...c.c.o com)
ABSTRACT
Training web agents to navigate complex, real-world websites requires them to master subaasks-short-hrrize choosing the correct date in a date picker. or scrolling in container to extract information). We introduce WARC-Bench (Web Archive Benchmark), a novel web navigation benchmark featuring 438 tasks designed to evaluate multimodal Al agents on subtasks. WARC-Bench enables sandboxed interactions with dy- namic and realistic webpages using Web ARChive files. We show that WARC- Bench is challenging for leading computer-use models, with the highest observed success rate being 64.8%. To improve open source models subtask, we ex- plore two common training techniques: supervised fine-tuning (SFT) and rein- forcement learning with verifiable rewards (RLVR). Experiments show that SFT models obtain a 48.8% success rate on the benchmark. Training with RLVR over SFT checkpoints, even in data-scarce settings, improves the score 52.8% WARC-Bench, outperforming many frontier models. Our analysis concludes that mastering these subtasks is essential for robust web planning and navigation, and is a capability not extensively evaluated by existing benchmarks. More details about WARC-Bench can be found at https://sanjarriont www.iaarrrrrr.ogbb.....ee gitghb.ii//waeee he//war--benchh
interactions on multiple UI components (e.g..
25
nst Actiion
35 Verification
GitHub
snapthot
observation
querySclector( aaro- suro-docopdtke=") getastirtturee calondor rocusdate")
Sendess
7
Maxtor AIRLINEE
siders. ShOOTS
Websites
WARC
Gi action
Agent
Reward
and dropdro...)
Reward-1
Revard-a
neal
Midget interaction
Interactive Webpage
click (169 4769,475)
Model Performances WARC-Bench
BB
Coperting
Dev
Alrsaa
Tex
835
80-
752
Scroll (0, 180)
Book & fight
Task "Belect March 21 2025 the departing date"
March 2025
60-
Success
48.7
H5.3
03/31/005
40-
15
17.5
Interactive Widgets
c363,
636)
Opencut-3t8
Quont 7-
Cloude Sonnet 4.0
Dure-720 720
Ours-728
(SFT RLAV)
Figure I: Overview of WARC-Bench. We record real synthetic websites as Web Archive files create interactive web environments for evaluating subtask execution and widget interactions GUT Agents. WARC- Bench uses programmatic reward functions for automatic evaluation. Subtasks remain challenging for frontier models. Our model trained via SFT and RLVR achieves mate-of-the the-hhrrt results among open-source models.
INTRODUCTION
Autonomous agent systems that can execute complex web-based tasks (Gao al. 2024 Ning al.) 2025 have attracted a lot of research interests and investment recently. Agentic web navigation
Work done during internship

---

Published as a conference paper at ICLR 2026
presents significant challenges, which researchers have approached from multiple angles (Gao et al. 2024). At one end of the spectrum, single-step visual grounding focuses on the ability to map textual descriptions to precise pixel coordinates within images (e.g. "Ouput the pixel coordinates of the button 'Japanese' under language options") (Cheng et al. 2024 Liu et al. (024a). On the other end, comprehensive web navigation benchmarks such as WehArena (Zhou et al. 2023) and Mind2Web (Deng et al., (2023a) are created to assess the performance of web agents in solving complex multi-step and long-horizon browser-based workflows (e.g. "Place an order for a pair of green women's 's Levi's 's jeans under $50 on Amazon.").
Complex browser-basee tasks typically involve constituent steps, like exploring the webpage via scrolling, interacting with UI widgets (dropdowns, date pickers, menu bars, etc.), extracting infor- mation, or recovering from navigational errors. To our knowledge, this intermediate level of agentic complexity remains unaddressed in existing realistic GUI benchmarks. In this work, we focus on investigating agent performance on these intermediate subtasks ie., smaller workflow com- ponents that correspond to simple natural language instructions within a larger web browsing task. Subtasks typically involve multiple primitive UI actions, such as click, scroll, type, and mouse move. Some examples include "Select 10/10/2025 as the start date" " or "Reply to the comment with a 'Thank you' in bold font".
To this end, develop WARC-Bench (Web ARChive Benchmark), novel benchmark designed to evaluate the accuracy of multimodal Al agents in executing subtasks on web browsers. We provide realistic web environments that fully replicate real-world websites, and serve them for Web agents to interact with and complete tasks. To achieve this, WARC-Bench uses Web Archive (WARC) files to record and replay websites in Chromium-based browser, allowing web agents interact with the webpage flexibly. Our benchmark dataset consists of (a) WARC based web environments, (b) per-task subtask goal in natural language, and (c) per-task programmatic (thus, deterministic) reward function. The verifiable reward functions are used to measure task completion at the end of the trajectory. This makes the evaluation independent of the particular path an agent takes to complete the task. WARC-Bench contains manually labeled test set of 200 real-world examples, and a train/devclopment set of 1059/238 examples consisting of synthetic and weal-world subtasks.
We use WARC-Bench to evaluate leading large vision-language models (VLMs) and computer-use agents on subtask execution in realistic websites. We find that WARC-Bench poses a significant challenge to current large foundation models. Specifically, Anthropic's Claude-4.0-Soma achieves the highest performance, with 64.8% % task completion on the test set, followed by OpenAl's GPT-5 at 51.3%. Open-source models lag considerably behind, with Qwen-2.5V1--2BB achieving the best performance among them at 37.3%.
mode
(Bai et al.
2025
To improve open source models, construct synthetic datasets fine-tune Qwen2.5-VL (Bai et al.,
2025) family of models on subtask execution. Our 72B model trained with Supervised Fine- Tuning
(SFT) and Reinforcement Learning with Verifiable Rewards (RLVR) achieves 52.3% 3 accuracy on the test set, significantly outperforming other open-source models and most frontier closed-source models. We also conduct comparative analysis of the models on other GUI navigation benchmarks like WebArena (Zhou et al.) (003), Screenspot V2 (Wu et al.) (222)) and MiniWoB++ (Liu et al.) 2018a), highlighting the unique contributions of the WARC-Bench GUT agents research.
To summarize, the key contributions of this paper are four-fold:
We formally define GUI subtasks short-horizon tasks that constitute broader browser- based workflows. We create WARC-Bench, novel, lightweight, extensible benchmark that, to the best of our knowledge, is the first Web-Archive file based benchmark designed to measure realistic GUI subtask execution accuracy.
We find that WARC-Bench poses a challenge to state-of-the-art large foundation models, with Anthropic's Claude-Sonna 4.0 achieving the highest success rate of 64.8%.
We show that high-fidelity subtask performance on WARC-Bench better tracks long- horizon web navigation capabilities when compared other benchmarks like Screenspot- V2 and MiniWoB++, demonstrating that WARC-Bende fills a gap in the existing landscape of web benchmarks.
httpp:/wwww.attttpp..cccoowwwewe hhtp:://ggpmaiicccmmmmmmmmmmmmim
2

---

Published as a conference paper at ICLR 2026
Dataset
#Tasks (Templates)
Interactive Ervs
Task Isolation
Deterministe End
State Rowards
Diverse Observation Space
Scalable Design
MultimoodalMinnd2WWl
1013 (-)
Online-Mind2Wch
300 (136)
WebArena
812 (241)
OSWorld
369 (-)
X
WARC-Bench
Table I: Comparison of Multimodal Benchmarks for Web Task Performance of GUT Agents. (-) templates
indicate that all benchmark tasks are unique.
We explore SFT and RLVR train competitive 7B/72B parameter models that outperform open-source and many leading closed-source computer-use models on subtask execution. Specifically, we demonstrate that RLVR with synthetic subtasks can improve agent perfor- mance on real-world subtasks, with enhanced visual grounding capabilities, better explo- ration, and contextual awareness.
2 WARC-BENCH: WEB ARCHIVE BASED ENVIRONMENTS FOR TESTING SUBTASK EXECUTION BY GUI AGENTS
Our work seeks to address gap between visual grounding and complex web navigation specifi- cally, short-horizon tasks that consist of more than one primitive UI action that appear as a function- ally singular instruction humans. We call such tasks GUI subtasks. They satisfy the following constraints.
They achieve self-contained and unambiguous goal using browser-based actions; They can be completed by human within 1-20 atomic UI actions (Table
GUI subtasks require skills such as navigating menu options, setting dates on datepickers, scrolling webpage to perform entity extraction, filling form fields, editing spreadsheet cells, and setting val- ues on drag-and and-drop slider bars, among many others. Compositional frameworks such as Agent-S2 ([Agashe al., 2025 achieve strong performance complex tasks by breaking them down into subtasks. This provides evidence that specialist models with high subtask accuracy can yield sub- stantial improvements in general, long-horizon GUI agents well. Our objective is to systemati- cally examine the capability of leading large VLMs in completing such subtasks by constructing new benchmark called WARC-Bench.
WARC-Bench is an extensible GUI agent evaluation suite containing realistic, interactive, and lightweight web environments that can be used perform a wide spectrum of short-horizon tasks. We require that our web environments capture real-life, complex UI widgets and densely populated webpages. To do so, we rely on Web Archive (WARC)) files to render interactive websites within our evaluation suite. Web Archives are snapshots that can preserve the complete state of the website during the period recording. They enable high-fidelity replay of archived webpages by record- ing assets like HTML, CSS and JavaScript (JS) files, embedded contents like images/videos and even HTTP headers and metadata for replicating network calls. The recorded trajectories are fully replayable, resulting in realistic and interactive clone of the webpage. This is uniquely well- suited to evaluating and training agents on short-horizon tasks. Replaying WARC files instead of live websites also makes the benchmark easy and fast run.
Using web archives as benchmark environments has the following advantages (Table High fidelity with real websites WARCs allow us realistically replicate websites. Task isolation Each task run uses unique copy of an archived benchmark environment. This ensures proper isolation and sandboxing for each task. Using WARC files also prevents from making any real write operations on actual websites while running the benchmark.
Scalable design The benchmark is designed to be extensible and easy to evaluate. Unlike prior works that are constrained by simulated environments (e.g. WebArena, OSWorld), adding a new environment to WARC-Bench simply means adding a new web recording to the test suite.
https:/cccwwihhpcccaaa///ww//// (file format)
3

---

Published as a conference paper at ICLR 2026
S502 PLA Shpp
100T LOSTRIOON 1.0
Split
Count
Dev
menu Iav gedin
form filling
table manipulation
Dev
238
25
[SYNTHETIC]
178
[REAL]
60
855 pas 001
U1000 un0es--a0
BursunouE
Test
data Pxtraction
Test [REAL] 200
Train [SYNTHETIC]
1059
32
31
47
1 40
28
16
4
13
11
11
Number Tasks for Subtask Categories
Figure 2: Statistics and Distribution of Subtasks in WARC-Bench. Each task in WARC-Bench belong multiple subtask categories, here we illustrate the category coverage in the figure on the right.
We develop lightweight WARC replayer that can simulate the archived website in Chromium- based automated browser created using Playwright This is conceptually similar to existing tools like ReplayWeb.purr vage that allow replaying web archives in the browser. WARC-Bench tasks are
fl
runnable as Gym (Brockman et al. 2016) environments, facilitating easy integration into exist- ing evaluation and training frameworks like BrowserGym (de Chezelles et al. 2025) and Volcano- Engine Reinforcement Learning library for Agents (verl-agent by Feng et al. (2055).
Limitations of WARC-based environments: Any webpage interactions that were not recorded or cannot be rendered from the stored HTML/IS contents are not replayable. If all assets required to access the target URI are found, the webpage can be rendered exactly the same as the original page, otherwise our replay server displays error. Websites that use Cloudflare or block robots are also often not archivable. The HTML/JS contents are sometimes indexed against URIs containing fields like timestamps, random numbers or session IDs that require careful handling during replays.
2.1 TASK BREAKDOWN
In this section, we dive into the details of the tasks present in WARC-Bench and the data collection strategies we used to assemble this benchmark. Each task within WARC-Bench dataset consists of the following components: (I) Environment: realistic web environment in which the agent op- erates. Each environment is backed by a single WARC file, with additional specifications including the starting URL, and optionally the timestamp of when the WARC file was captured. (2) Goal: An umambiguous, natural language description of the subtask that the agent must perform within the web environment. Given the goal, the agent should either explore the current webpage to determine a correct next action or choose when to terminate the trajectory. The goal should result in deter- ministic and unique end-state of the webpage. Some example subtask goals are, 'Enter "MacBook Pro" " in cell Al' or 'Select 02/03/2025 (mm//dd/yy format) the start date'. (3) Evaluator: Code- based evaluation criteria that determines the trajectory level reward. Each evaluator defined by the evaluator type and evaluator function, enabling deterministic measure of the task success, We support 4 evaluator types (a) IS function- based evaluator, (b) URL matcher, (c) String matcher, and (d) JSON matcher. The latter evaluators are primarily used for information seeking tasks. Examples include, document eeuerySelecteo or (' 4 4risssll ider') ) valuem=' 4' (JS Evaluator) or ( towtalltt__aee_
657) (JSON Matcher).
2.2 DATA COLLECTION METHODS
To determine which GUI subtasks to include in the benchmark, we compile a list of 15 common subtask categories that require multi-step interactions within webpages (see Figure We then se- lect 29 real-world websites that are recordable as web archives and have good representation of our task categories. The websites we selected include Github, Zoho Desk, Zendesk, Kaiser Permanente, Google Earth, Scrimba, etc. The goals and evaluator functions for real websites are collected by the authors manually.
We also create 62 synthetic websites through an LLM--baeed pipeline that can create HTML pages with a diverse representation of interesting UI widgets. This approach has also been explored in
4

---

Published as a conference paper at ICLR 2026
concurrent works like FormFactory (Li et al. 2023). The LLM-based pipeline lets automatically and cheaply generate environments, task goals, and evaluator functions for the benchmark. The goals and evaluator functions generated in this manner still need to be manually vetted for feasibility and correctness, but the process is a lot faster than end-to-end manual data labeling.
We create total of 1497 subtasks (1059 training, 238 development, 200 test) the collected environments for WARC-Bench. We constrain the test dataset (200 tasks) to only use real web environments, while the development set contains mix of 60 real and 178 synthetic tasks. We also generate an illustrative, synthetic training set for online reinforcement learning. We observe that agent performance lower on real tasks when compared to synthetically generated tasks, un- derscoring the need for including manually collected, high quality test websites in web navigation benchmarks (Table
WEB AGENTS FOR SUBTASKS
In this section, we formalize the concept of AT Agents for computer-use and web navigation (web agents) and introduce Subtask Vision Agent (SVA) generalist VLM based web agent created for subtasks. We also go over leading computer-use models that we evaluate with WARC-Benchs
3.1 FORMULATION
Given website (e.g www.dmwd..cc.ggoo and goal (e.g. "Set the language of the page to Portugese" "), we define a web agent to be a software program that can autonomously generate a sequence of actions and complete the goal on graphical user interface (e.g.web web browser on desktop/mobile). Since a website typically transforms once an atomic action (Table/ is performed on it, a web agent usually observes the state of environment, and greedily predicts the best atomic action that brings closer to achieving the goal (Qin et al. [225 Yao et al. 2023). It also terminates the trajectory once the task is complete or is deemed infeasible. Thus, we can formalize the task completion problem by web agents as: at each step of the trajectory, given initial goal, observation of the current environment, an action space, and a history of previous observations and model decisions, find the optimal next action that would eventually result in the success of the task.
3.2 SUBTASK VISION AGENT (SVA)
In order to evaluate and compare the proficiency of Hision-language models at subtask execution, we design a simple yet powerful web-navigation agent called Subtask Vision Agent (SVA) (Figure 3.. At each step of the subtask trajectory, SVA takes in the goal, current screenshot of the webpage, the action space, and a history of previous steps. It then produces a Chain-of- -ooo-uhh-Thhhht (CoT) and the next action within the response. The complete system and user prompt of the agent and CoT questions can be found in Appendices and G
SVA uses only screenshots to observe the state of the webpage. This makes textual representation of the webpage such as accessibility trees and HTML. DOM redundant. Furthermore, text observations are lengthy and can exceed the context limit of certain models. Modern HTML websites complex visual elements like icons, widgets, and canvas-based rendering, making screenshots a more faithful representation of the interface than text. The approach of using vision-only observations aligns with the methodologies employed by all major vision-language agents mentioned in Section 3.3
The history provided the agent has the screenshots and the model responses previous steps of the trajectory (more details in Appendix F.. We limit the length of the history observed by the backbone model to 5.
We include a minimal API action space in the system prompt with eight (8) types of actions: click, complete, drag_and wan.dddeeaseess hover, vided to the agent can be found in Appendix G We use BrowserGym (de Chezelles et al. 2025 as the underlying engine interact with the browser.
type, wait. The action space descriptions pro-
"We do not focus on text-only Large Language Models in this work. http:://ggthhh...mm/cceeee/////w
5

---

Published as a conference paper at ICLR 2026
Subtask Vision Agent
Web Browser
Pronppt
Prompt Generator
Action
T Text
UI Interaction System
Imoge
Goal
Hi story
Model
CoT action
Response Parser
Action
(BrowserGym)
Figure 3: Diagram of the Subtask Vision Agent (SVA) design.
We find that SVA is effective agent design with high task completion performance despite using significantly lower tokens and fewer trajectory steps on average as opposed to other computer-use agents (Table 2 Figure 5).
3.3 EXISTING AGENT DESIGNS
For baseline experiments, we evaluate proprietary computer-use agents Anthropic's Computer- Use agent (Claude CUA 8 OpenAl's Computer-Using Agent (OpenAl CUA P Bytedance's 's open source computer-use model UI-TARS 1.5 7B (Qin et al. 2025) and the open-source OpenCUA computer-use models by Wang al. (025), All these computer-use models contain their respective, pre-defined action spaces. All four agent designs follow an passerve-predic-acc loop, i.e., fetching observations from the web environment followed by selecting the next best action until task comple- tion. The actuation of predicted action for all CUA agents is implemented by using BrowserGym. We provide the prompt used in the agentic evaluation of these models in Appendices |aand]
EXPERIMENTS
To find whether subtask execution is challenging for foundational models, we perform extensive evaluations using the development and test splits of -WARC-Bench We evaluate open-source, propri- etary, and computer-use models from ByteDance, OpenAI, and Anthropic with their recommended agent design as well as with the Subtask Vision Agent (SVA). Through our analysis, we show that SVA is a competitive agent design for subtasks (Table Figure
We discover that GUI subtasks are especially challenging for smaller and medium scale open-source VLMs. To address this, we train Qwen2.5VL family of models by [ai al. (2025) (7B and 72B parameters) with curated training datasets to enhance subtask accuracy. All our models are trained with the SVA prompt architecture. All the screenshots used in the web state observation and the past history, are of the resolution 1280x720 pixels each.
Distillation from Teacher Model Trajectories: We created large scale SFT dataset using novel and scalable subtask trajectory crawling framework. This allows us to collect around 12k unique trajectories using teacher models with strong UI interaction capabilities. These trajectories are used to distill grounding, planning, error correction, and reward modeling capabilities into the Owen2.5-VI. models. The sources for websites used for trajectory collection include Common webpages for UI component libraries, and synthetically created websites.
Craw] 10
Reinforcement Learning with Verifiable Rewards: As described in Section 2.2 we de- velop a L.MM based data synthesis pipeline to automatically create WARC-Bench tasks including synthetic webpages, subtask goals and evaluation rules. This approach is used to create an illustrative training dataset of Gym-environment based subtasks (1059 tasks). To improve on the supervised fine-tuned model, (Schulman et al.) 2017) to learn from the agent rollouts with environment feedback directly. At each training step, the agent generates rollouts where the agent gets a positive reward 10 for successful trajectories and for failed or truncated trajectories due exceeding max steps. The list of hyperparameters can be found in Appendix [)]
further apply Proximal Policy Optimization (PPO) algorithm
batch of tasks. We simple reward scheme
Phaps:/coommoncrwwlorry

---

Published as a conference paper at ICLR 2026
Accuracy (%)
Model
Dev(SYNTHETIC)
Dev[REAL.]
Dev[TOTAL]
Test
OpenAl computcr-use-previee (2025-03-//)eux
62.17
49.44
58.96 +1.3
33,83
+2.58
GPT-4o (2024-11-20)
7.87
14.51
9.54 1.70
9.17 +1.04
GPT-5 (2025-08-07)
72.66
61.67
69.89 +1.21
$1.33
#3.23
Claude Sonnet 4.0 (2025-05-14)oth
79.92
76.11
78.9 1.111
47.17
12.89
Claude Sonnet 3.7 (2025-02-19)
78.89
81.93 11.51
59.83
11.26
Claude Sonnet 4.0 (2025-05-14)
84.27
81.67
83.61 +0.84
64.83 1.61
Qwen2.5-VL. 7B
16.85
11.67
15.54 +0.42
4.67 1..04
Ul-Tars 1.5 7BCUA
44.01
26.55
39.66 +0.42
10.33 #:076
OpenCUA 7B
48.03
41.67
46.43
14.00
Ours-7B-SFT
70.60
54.49
66.54 +0.57
27.33 #1.47
Ours-7B-RLVR (SFT+RLVR)
78.09
54.44
72.13 +0.64
29.17 #1.15
OpenCUA 32BCUA+
51.12
41.67
48.74
17.50
Qwen2.5-VL. 72B
51.67
61.06 +1.99
37.33
10.85
Ours-72B-SFT Ours-72B-RLVR (SFT+RLVR)
78.23
68.89
75.88 40.97
48.33
1166
87.64
76.67
84.31
+0.87
$2.33
10.76
Table 2: Trajectory-Leve Level Success Rates on WARC-Bench. Small VLMs (7B params) are in gray. Results are divided into closed (top) vs. open-source (bottom) models. CUA indicates models evaluated with the provider's computer-use agent. All other models use SVA. All values averages across 3 runs, however indicates single-run results due to prohibitive HuggingFace inferenee costs for OpenCUA. Best per benchmark is in bold. Best inside sector (small/large open source closed source) is underlined.
RESULTS AND ANALYSIS
Table shows the main results of various model-agent combinations evaluated on WARC-Bench. Our main findings are: (1) WARC-Bench is challenging benchmark for leading frontier mod- els. While Claude-based agents show superior performance to other computer-use agents and mod- els. the best Claude model only achieves 64.83% success rate on the benchmark, leaving signifi- cant room for future improvement.
(2) SVA is simple yet competitive agent design. With the same family of frontier models, GUI agents based on the SVA design significantly outperform their dedicated computer-use agent counterparts, including proprietary ones. (3) Distilling from strong frontier models can effectively boost open-source model performance on WARC-Bench. With SFT training, we see significant improvements over the original Qwen2.5VI. 7B and 72B models from 4.67% to 27.33% (Ours-7B-SFT) and 37.33% to 48.33% (Ours-72B- BFFT). (4) Agentic RLVR training significantly improves agent performance further beyond SFT models. With additional RLVR training on the synthetic training set, we see further performance boost over the SFT baselines for both the 7B (27.3 27.33% to 29. 29.17% Ours- 7B 7e-s-B--RLLEE RLVR) and 72B (48.33% to 52.33% Burs-72B-RLVV Further analysis on the development set shows that training on synthetic data not only boosts success rate synthetic tasks (+9.41 %),, but also their real-website counterparts (+7.78%)
Effects of Agentic RLVR. We analyze the behavioral patterns of our 72B SFT and RLVR models on the WARC-Bench development split. A breakdown by task categories (Figure 4 reveals that the RLVR model achieves substantial improvements in dynamic tasks such as form filling, menu navigation, table manipulation, and datepicker, while maintaining comparable or slightly higher performance in most other categories. We observe that these gains are driven by enhanced vision grounding capabilities and better exploration contextual awareness. Specifically, the RLVR model demonstrates greater precision in identifying small interface elements (e.g., calendar dates, nested menu items) and reduced errors from misplaced clicks. Agent action distribution (Figure 4 shows increased use of scrolling actions for better exploration of the webpage by the RLVR agent, and also reveals a clear efficiency gap between the two agents: the RLVR model executes fewer overall actions, reduces redundant clicks, and leverages compound operations (e.g., click-and -tt--pee more consistently. As a result, the RLVR model produces shorter trajectories on average, with approxi- mately 0.94 fewer step per task compared to the SFT model (Figure 4
Comparison to other GUI navigation benchmarks. We also compare agent performance on sev- eral other web-navigation benchmarks compare with WARC-Benchh including long- hong-horizon tasks

---

Published as a conference paper at ICLR 2026
SF
form filing
drag and drop 0.821
navigation 0.725
table manipuletion 0.712
list 0.709
400 Q Seccco
datepicker 0.655
Success
0.833
0.8
sans Antraction
dreg
navigation
table manibulation
datapicker
0.804
0.4
Number of Tasks for Subtask Categories
Success rate comparison for different subtask categories
Avg:
-0.94
Beth Fail
mouse move, screll
mouse click
message to user
Number of Tasks
100
RLVR Success
SFT Fail,
(Log Scale)
RLVR Fail,
SFT Success
Both Success
sond messaçc USe!
RLVR
mouse move, scholl
mouse click
500
1000
Action Count
Number Difference (RLVR SFT)
(b) Agent action distrihution comparison
(c) Trajectory length comparison
Figure 4: Behavioral analysis Ours-72B-SFT (SFT) v/s Ours-72B 7----222-RRRR (RLVR) model
(WebArena; Zhou et al. 2023), synthetic, low-fidelity widget tasks MliniWoB+-a
Liu et al. 201 8a).
and GUI grounding tasks (ScreenSpot V2; Wu et al. 2024 Cheng et al. 2024
WARC-Bench
WebArena (no map)
ScreenSpot
Model
Miniwob++
(test)
V2
Owen 2.5 VL
4.67%
3.07%
12.53%
51.62%
Qwen 2.5 VL 72B GPT-5 (2025-08-07)
37.33%
15.68%
53.87%
88.05 %
51.33%
34.06%
52,27%
26.39%
Claude 4 Sonnet
64.83%
37.96%
71.73%
85.06%
Ours Ours-7B-RIMV 7V-RLVR Ours-72B-RII
29.17%
7.31% 26.80%
36.27% 59.20%
75.81% 82.44%
52.33%
Table 3: SVA agent performance comparison across benchmarks. Fach number is an average of 3 runs. Best per benchmark in bold. Second places are underlined.
Table 3 shows the results of task performance comparison across benchmarks. Our main findings are: Agent performance on grounding and low-fidelity widget tasks correlate poorly with long-horizon task performance, while WARC-Bench tracks it at a system level. This can be seen by comparing the relative ranking of agents by the performance on each benchmark, well as relative performance gaps. Most notably, while vanilla Qwen 2.5 VL models achieve stronger performance than frontier models, they are nevertheless least effective long-horizon tasks. (2) Models fine-tuned over subtask data improve over their base models on all tasks (except for grounding). As can be seen, fine-tuning models on subtask data can improve agent performance on both low-fidelity widget tasks and long-horizon web navigation tasks. While the former is more expected due to the similarities in task setup, the gains on long-horizon tasks shows that training with subtask objective improves model's overall web navigation capabilities. Together with the observation that SPARC-Bench performance tracks that of WebArena, this finding corroborates our hypothesis that solving subtasks well is precursor agent improvement navigation tasks. (3) WARC-Bench provides stress testing on crucial GUT agent capability that is not well-covered by existing benchmarks. As shown in Table 3 while GPT-5 and Claude 4 Sonnet agents achieve comparable performance on WebArena (within about 11% relative per- formance gap), their performance gap on WARC-Bench is enlarged about 26%, indicating key
longer-horizon web
We omit the OpenStreetMap portion of the WebArena benchmark due to technical difficulties with the online map service. For Screenspot V2, which is visual grounding benchmark, we limit the agent action space to c click (...) only. For each evaluation, use the same prompt format built inside our Subtask Vision Agent (see Appendix(F)
8

---

Published as a conference paper at ICLR 2026
generalization capabilities uniquely captured and assessed by WARC- For long-horizon tasks like WebArena to capture the same level of comprehensiveness and diversity, great deal of efforts necessary for task creation and environment maintenance. WARC-Bench sidesteps this limita- tion with lightweight, sandboxed WARC interactive environments, and ensures ease of scaling on task diversity and difficulty stress-test GUI agents this important aspect.
Overall, these observations demonstrate that WARC-Bench is a unique benchmark for GUI agent development, and one that helps advance agent performance on long-horizon tasks, which is closer to the ultimate goal of GUI agents.
Decomposition of web navigation into planning and subtasks. One motivation for this work is to show that GUI sub-agents that specialize in executing subtasks can bring gains to end-to-end GUI navigation tasks, making subtasks critical capability to evaluate, diagnose, and improve upon in generalist web agents. A common approach to managing agent context for a oonghhorioon, end- to-end web agent to create a hierarchical agent with a Planner (P) sub-agent (prompt provided in [ppendix[] that can break down a long-horizon task into subtasks and redirect them to performant subtask model (e.g., Ours-72B-RLYN how choosing the correct agent design for web navigation tasks can bring accuracy gains, while offloading most of the token footprint to cheaper, open source models. We use a subset of 165 WebArena (Zhou al. 2023) filtered for evaluator correctness called WebArena-Lite (WA-Lite) 2024b) for these experiments.
w/ SVA) acting as an executor (E) sub-agent. Table shows
Liu et al.
For Qwen2.5-VL-72D based agents, we can significantly improve the performance (+8.46%) by adopting a hierarchical plannertexecutor agent design and using a strong executor model Dours-72B-RLVD [Rows 3].
For strong models like Claude-4.0-Sone -SSnnett, breaking down the task into high level plans v/s subtask executions does not bring any gains/downsides in overall performance [Rows 4, 6]. However, even for Vlaude-4.0-Sonne offloading most the agent tasks to a capable executor model (Ours- Bours-72--RR 72B-RLVR) with no drop in accuracy potentially lower cost of inference [Rows 4, 6, 7].
still benefit from hierarchical design by
For long-horizon tasks, the quality of the planner model plays significant role in deter- mining the success rate [Rows 2, 5 and Rows 3, 7]. Using a better planner like Claude instead of Owen2.5-VL-72B. with the same capable executor (Ours-72B-RLA) -RLVR), signifi- cantly improves WA-Lite SR by +1 +1.11% points [Rows 3, 7.
# Planner Model
Subtask Executor
Agent
WA-Lite SR %
1 N/A
Biee2.55VL-722-Inniii
SVA
16.17 + 1.14
2 Qwen2.5-WLL-72B-Inse
Qwen2.5- 1---5--V--72----n--
Hier. [P. E]
22.39 2.58
3 Qwen2.5-VL-722-Innee
Ours-72B-RLVR
Hier. [P. E]
24.63 = 0.75
4 N/A
Claude-4.0-Sonne
SVA
36.32 1.14
5 Claude-4.0-Sone
Hier. [P. E]
31.84 + 0.87
6 Claude-4.0 4.0dde-----Sonett
Claude-4.0-Sonne
Hier. [P. E]
35.32 1.14
7 Claude-4.0-Somnet
Ours-72B-RLN -LLV
Hier. [P. E]
35.82 + 1.29
Table Web navigation accuracies on a simple SVA agent v/s Hierarchical Planner+Exeeutoor agent designs. SVA Single-loop execution agent detailed in Section 3.2. Hier. [P.E] Hierarchical agent consisting of a Planner that determines what subtasks to perform, and an SVA agent acting as an Executor. Executor completes the subtask before returning control to the Planner.
Human Baseline. We performed human evaluation on the 60 realistic tasks in the development set (Dev-Real) with 3 participants. The persons who performed the evaluation were not involved the dataset creation. The human accuracy we obtained is 87.11% + 1.84% (52.3/ 60 tasks). Among the failed tasks, tasks requiring reading long text or formatted outputs are most challenging for humans. For example, they failed the towing fee extraction task where one needs scan through rules / tables of fees to come up with the final number. For tasks that require formatted outputs, sometimes the answers were written in slightly wrong format.

---

Published as a conference paper at ICLR 2026
6 RELATED WORK
Language agents for GUI automation. Large language models (LLMs) powered Al agents have shown to be capable of interacting with digital environments and completing various tasks on graph- ical user interfaces (GUIs) (Xie et al. (2024a). LLMs can understand web states (screenshots, HTML
codes, etc.) and output code to execute actions on GUIs (Xie al. 2024b Sager et al. 2025). Latest open source works like Aguvis (Xu et al. (025) and OpenCUA Wang et al. 2025 have designed vision-only GUI agents that utilize VLM Vision-Language Models) model's 's capabilities for plan- ning and reasoning to correctly solve their tasks. Furthermore, due to the verifiable, sequential and
exploratory nature of GUT tasks, many latest works such as GUI-RI (Luo et al.) (025) and UI-R1 Lu et al. (2025) have applied Reinforcement Learning Post-Training techniques to their models.
GUI agent benchmarks. Existing GUI agent benchmarks span a spectrum of task complexity from single-step grounding tasks long-horizon task completion in live or simulated environments. On the one end, visual grounding tasks (Cheng et al., 2024 Liu et al.; 2024a) can be completed with single primitive UI action (click, type, scroll) at specific pixel coordinates. While these evaluate agent capabilities to accurately identify the GUI coordinates for a single-step action, performance on these do not necessarily correlate with downstream task performance. On the other end, long- horizon tasks aim to test GUI agents on realistic tasks within practical websites or operating systems. WebArema (Zhou et al. 2023) employs five realistic, standalone and self-hostable web environments for building and evaluating autonomous agents that interact live with these websites. Multimodal- Mind2Web (Deng et al. 2023b Zheng et al., 2024) features complex web task trajectories on real websites, but adopts Mind2Web (uue et al. 2025) re-applies
offline evaluation setting for per-step action prediction accuracy. Online- tasks to a diverse set of real-world user tasks real, online websites and uses an ALM--as--juddee to evaluate task completion accu- racy. On operating systems, OS- World (Xie et al.] 2024b) is a popular benchmark containing real OS environments evaluating agent performance on open-ended computer tasks spanning multiple applications. These long-horizon tasks can better help evaluate how well GUI agents fulfill realistic user tasks on these environments, but are typically more difficult to develop on, and lack diversity and stress testing for their component subtasks. The closest family of previous GUI agent bench- marks WARC-Bench are based on simplified/synthetiiall generated web environments focusing on UI widgets or simple tasks, such as Mini Wob++ (Liu et al. 2018b) and FormFactory (Li et al., 2025). These tasks often lack the fidelity of real-world Uls, which can result generalization gap when used to develop real- world GUI agents. WARC-Bend provides high-fideligy short-horizon, and interactive environment, which marks a unique contribution on this spectrum.
Baltimodal-Mindd2We
7 CONCLUSION
We present WARC-Bench, a novel and realistic benchmark for GUI subtasks. We demonstrate that subtask execution remains challenging problem for frontier models like Claude 4.0 Son- net and GPT-5. With synthetically generated training datasets, we show that open-source mod- els trained with supervised fine-tuning (SFT) and agentic reinforcement learning with verifiable rewards (RIVR) achieve competitive performance with frontier models. WARC-Bench presents unique challenges and opportunities for agent development GUT navigation, and complements existing long-horizon benchmarks with scalable, sandboxed environments.
We believe WARC-Bench opens up several promising research directions. For future work, we plan to better automate WARC-Bench sample creation and curation, study SVA performance as part of a larger agent framework for long-term tasks, and explore advanced training methods to further improve agent performance. We hope that our work will inform future studies of GUI subtasks and eventually lead to better generalized systems of computer-use.
10

---

Published as a conference paper at ICLR 2026
REPRODUCIBILITY STATEMENT
We use several open-source software packages in this study, including BrowserGym for web nav- igation dde Chezelles al. 2025 12
Volcano Engine Reinforcement Learning for LLMs (VERL) for SFT and offline RL training and verl-agent for agentic RL training 14 Our large-scale data collection utilize the common crawl website index
The codebase for running WARC-Bench can be found at http:////tuubooooo////aaaaaa////
The codebase of training recipes for our SFT/PPO/GRPO 7B and 72B models can be found at http://hwww..ooooaaaaaarrrrrrrba SFT and RLVR training in Appendices Cnd
We separately discuss the hyperparameters of our
REFERENCES
Saaket Agashe, Kyle Wong, Vincent Tu, Jiachen Yang, Ang Li, and Xin Eric Wang. Agent s2: A compositional seneralist--pecialis framework for computer use agents, 2025. URL https:
/arxiv.org/abs/225000 00906
Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang. Jun Tang. Humen Zhong, Yuanzhi Zhu, Mingkun Yang, Zhaohai Li, Jianqiang Wan, Pengfei Wang, Wei Ding, Zheren Fu, Yiheng Xu, Jiabo Ye, Xi Zhang, Tianbao Xie, Zesen Cheng, Hang Zhang. Zhibo Yang. Haiyang Xu, and Junyang Lin. Qwen2.5-vl technical report, 2025.
URL https: /uttps://azeee //axivv...iivvvo org/ abs 250..2552.... 13923
Greg Brockman, Vicki Cheung, Ludwig Pettersson, Jonas Schneider, John Schulman, Jie Tang, and Wojciech Zaremba. Openai gym, 2016.
Kanzhi Cheng, Qiushi Sun, Yougang Chu, Fangzhi Xu, Yantao Li, Jianbing Zhang, and Zhiyong Wu. Seeclick: Harnessing gui grounding for advanced visual gui agents, 2024. URL https:
7/
10935
Thibault Le Sellier de Chezelles, Maxime Gasse, Alexandre Lacoste, Massimo Caccia, Alexan- dre Drouin, Léo Boisvert, Megh Thakkar, Tom Marty, Rim Assouel Sahar Omidi Shayegan, Lawrence Keunho Jang. Xing Han Lù, Ori Yoran, Dehan Kong, Frank F. Xu, Siva Reddy, Gra- ham Neubig, Quentin Cappart, Russ Salakhutdinov, and Nicolas Chapados. The browsergym ecosystem for web agent research. Transactions Machine Learning Research, 2025. ISSN 2835-8856. htttt::://genneeeeev.neee/eo cation.
Expert Certifi-
Xiang Deng, Yu Gu, Boyuan Zheng, Shijie Chen, Samuel Stevens, Boshi Wang, Huan Sun, and Yu Su. Mind2web: Towards generalist agent for the web, 2023a.
Xiang Deng, Yu Gu, Boyuan Zheng, Shijie Chen, Samuel Stevens, Boshi Wang, Huan Sun, and Yu Su. Mind2web: Towards a generalist agent for the web. In Thirty-seventh Conference on Neural Information Processing Systems, 2023b. /RLAtttpss//opeeeeveee... www.neettoooreeee kd=kiYgbo3wqwi
Lang Feng, Zhenghai Xue, Tingcong Liu, and Bo An. Group-in-group policy optimization for Ilm agent training. arXiv preprint wxXv:25055.1097 2025.
Minghe Gao, Wendong Bu, Bingchen Miao, Yang Wu, Yunfei Li, Juncheng Li, Siliang Tang, Qi Wu, Yueting Zhuang, and Meng Wang. Generalist virtual agents: A survey autonomous agents across digital platforms, 2024. URL httpssttts///aa/ http://awwvvvvvv.aae/// abtp///s./2111.1111 10943
Bobo Li, Yuheng Wang, Hao Fei, Juncheng Li, Wei Ji, Mong-Li Lee, and Wynne Hsu. Formfactory: An interactive benchmarking suite for multimodal form-filing agents, 2025. URL https://
a axxiv..org/abs//566..111
14 hhans///gthuooccmoooleennneveeer "htps://commonccawlorrr/
11

---

Published as a conference paper at ICLR 2026
Evan Zheran Liu, Kelvin Guu, Panupong Pasupat, Tianlin Shi, and Perey Liang. Reinforcement learning on web interfaces using workflow- workflow-guided guided exploration. CoRR, abs/1802.08802, 2018a.
URL http::///axxiv... org/ abs/ 1802. 08802
Evan Zheran Liu, Kelvin Guu, Panupong Pasupat, Tianlin Shi, and Percy Liang. Reinforcement learning on web interfaces using workflow-guided exploration. In International Conference on Learning Representations (ICLR), 2018b. URL http:::ttss/e.. //arxiv.. org/ otg/abs/////oooooooroo..o/t..brr 1802 08802
Junpeng Liu, Yifan Song, Bill Yuchen Lin, Wai Lam, Graham Neubig, Yuanzhi Li, and Xiang Yue. Visualwebbench: How far have multimodal lims evolved in web page understanding and grounding?, 2024a. URL http:tts:: /tttpp:///riiit org/ abs 2404. 05955
Xiao Liu, Tianjie Zhang, Yu Gu, lat Long long, Yifan Xu, Xixuan Song, Shudan Zhang, Hanyu Lai, Xinyi Liu, Hanlin Zhao, Jiadai Sun, Xinyue Yang. Yu Yang, Zehan Qi. Shumtian Yao, Xueqiao Sun, Siyi Cheng, Qinkai Zheng, Hao Yu. Hanchen Zhang, Wenyi Hong, Ming Ding, Lihang Pan, Xiaotao Gu, Aohan Zeng, Zhengxiao Du, Chan Hee Song, Yu Su, Yuxiao Dong, and Jie Tang- Visualagenthench: Towards large multimodal models as visual foundation agents, 2024b. URL
https://arxiv: org/ 2408. 06327
Zhengxi Lu, Yuxiang Chai, Yaxuan Guo, Xi Yin, Liang Liu, Hao Wang, Han Xiao, Shuai Ren, Guanjing Xiong, and Hongsheng Li. Ui-rl: Enhancing efficient action prediction of gui agents by reinforcement learning, 2025. (tLLhhtps:/// arxiv. httpgg/ab///00.222g/ 2160
Run Luo, Lu Wang, Wanwei He, and Xiaobo Xia. Gui-rl : A generalist r1-style vision-language action model for gui agents, 2025. URL htttt:://aa/xiv..oo wwtp:/wwwwwwww//////ee..../
Liangbo Ning, Ziran Liang, Zhuohang Jiang, Haohao Qu, Yujuan Ding, Wenqi Fan, Xiao yong Wei, Shanru Lin. Hui Liu, Philip S. Yu, and Qing Li. A survey of webagents: Towards next- generation ai agents for web automation with large foundation models, 2025. URL https:
23350
Yujia Qin, Yining Ye, Junjie Fang. Haoming Wang. Shihao Liang, Shizuo Tian, Junda Zhang, Jiahao Li, Yunxin Li, Shijue Huang, et al. Ui-tars: Pioncering automated gui interaction with native agents. arXiv preprint arXiv:2501.12326 2025.
Pascal J. Sager, Benjamin Meyer, Peng Yan, Rebekka von Wartburg-Kotteed Layan Etaiwi, Aref Enayati, Gabriel Nobel, Ahmed Abdulkadir, Benjamin F. Grewe, and Thilo Stadelmann. com- prehensive survey of agents for computer use: Foundations, challenges, and future directions, 2025. URL http::://///mm / arwiv. org/ hbtg//bb/255000 25100......110
John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. 2017. URL
Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, Y. K. Li, Y. Wu, and Daya Guo. Deepseekmath: Pushing the limits of mathe- matical reasoning in open language models. 2024. URL /ttttttss////atttt/ org/ abs/2402.
03300
Xinyuan Wang, Bowen Wang, Dunjie Lu, Junlin Yang, Tianbao Xie, Junli Wang, Jiaqi Deng. Xi- aole Guo, Yiheng Xu, Chen Henry Wu, Zhennan Shen, Zhuokai Li, Ryan Li, Xiaochuan Li, Junda Chen, Boyuan Zheng, Peihang Li, Fangyu Lei, Ruisheng Cao, Yeqiao Fu, Dongchan Shin, Martin Shin, Jiarui Hu, Yuyan Wang, Jixuan Chen, Yuxiao Ye, Danyang Zhang. Dikang Du, Hao Hu, Huarong Chen, Zaida Zhou, Haotian Yao, Ziwei Chen, Qizheng Gu, Yipu Wang, Heng Wang, Diyi Yang, Victor Zhong, Flood Sung, Y. Charles, Zhilin Yang, and Tao Yu. Opencua: Open foun- dations for computer-use agents, 2025. URL httpt:://t/aattt v. org abs 250./558.0000 09123
Zhiyong Wu, Zhenyu Wu, Fangzhi Xu, Yian Wang. Qiushi Sun, Chengyou Jia, Kanzhi Cheng. Zichen Ding, Liheng Chen, Paul Pu Liang, and Yu Qiao. Os-atlas: A foundation action model for generalist gui agents, 2024. URL (nttp:: //tp:i//aaivvvv/.a org/ abs / 2410. .32218
Junlin Xie, Zhihong Chen, Ruifei Zhang, Xiang Wan, and Guanbin Li. Large multimodal agents: A survey, 2024a. U http:////wwwwvv.o///////////////
12

---

Published as a conference paper at ICLR 2026
Tianbao Xie, Danyang Zhang, Jixuan Chen, Xiaochuan Li, Stheng Zhao, Ruisheng Cao, Toh Jing Hua, Zhoujun Cheng, Dongchan Shin, Fangyu Lei, Yitao Liu, Yiheng Xu, Shuyan Zhou, Silvio Savarese, Caiming Xiong, Victor Zhong, and Tao Yu. Osworld: Benchmarking multimodal agents for open-ended tasks in real computer environments, 2024b. URL htttss:/aaxxv..rrr
abs /404.2 07972
Yiheng Xu, Zekun Wang, Junli Wang. Dunjie Lu, Tianbao Xie, Amrita Saha, Doyen Sahoo, Tao Yu, and Caiming Xiong. Aguvis: Unified pure vision agents for autonomous gui interaction, 2025.
URL https: awwivv org/ abs/2412.... 04454
Tianci Xue, Weijian Qi, Tianneng Shi, Chan Hee Song, Boyu Gou, Dawn Song, Huan Sun, and
Yu Su. An illusion of progress? assessing the current state of web agents. 2025. URL https:
7/ wwtpai..wo/.ab//20/0000
Shunyu Yao, Howard Chen, John Yang, and Karthik Narasimhan. Webshop: Towards scalable real- world web interaction with grounded language agents, 2023. URL httts://aaxxiy org/
abs/2207.0122 01206
Boyuan Zheng, Boyu Gou, Jihyung Kil, Huan Sun, and Yu Su. Gpt-4v(ision) is generalist web agent, if grounded. 2024. URL https:t::/// /tttps://oppnreellew wwwpeenwwwwwww..eeeeeeeeeeeeeeee wtt.:eeeeetoooummcoooleee...eee
Shuyan Zhou, Frank F Xu, Hao Zhu, Xuhui Zhou, Robert Lo, Abishek Sridhar, Xianyi Cheng, Yonatan Bisk, Daniel Fried, Uri Alon, al. Webarena: A realistic web environment for building autonomous agents. arXiv preprint M:Xiv:2337.138855 2023.
A SUBTASK ENVIRONMENT AND TASK EXAMPLES
We provide 10 examples of tasks based real websites in the dataset Dev[MANUAL] below. The goals are made as unambiguous as possible to ensure validity the evaluator functions.
Go to Button types in side menu and move slider to 40
Select date 6 June 2024 in the date picker
Select the font Times from the dropdown
Select March 21 2025 the departing date and April 21 2025 the returning date
Set the adults count to 3 and add child of age 14
Continue filling the search form using the following information: "one way": True, "use miles":
True, "upgrade": "MVP Gold 75K"
Swap the From and To airports in step
Display only tickets with "open" status. Keep the filter dropdown menu open for the user to verify your selection.
Go the Organizations page
What were the total number of property crimes and property crime rate per 100000 population commited in year 2020? Give the answer in JSON format with the keys "total crimes" and "crime rate per 100000". Do not include any commas in the values.
Similarly, below are 10 examples from Dev[SYNTHETIC].
Enter 'MacBook Pro' in cell A1
Select cell in spreadsheet and apply hold formatting using the toolbar
Navigate Data Analyties tab and search for '2023" to find all employees who started in 2023
Type the formula DC10"D10" cell E10 to calculate total value
Sort the Employee Management Table by 'Status' column header in ascending order
Scroll down in employee directory to see all employecs
Drag the Marketing Budget slider to $75,000
In the Expense Reports tab, check the "Recurring Expense" checkbox in expense type options
13

---

Published as a conference paper at ICLR 2026
Fill in the employee registration form with first name 'Dr. Sarah' and last name MiilinWillimss
Delete the last row from the spreadsheet using the 'Delete Row" button
B ACCURACY VS. LATENCY TRADEOFFS
real-world applications, users often prefer faster agents over alternatives that are marginally more accurate but slower. This motivates us to perform comprehensive analysis of the efficiency- accuracy trade-offs our trained model checkpoints against the existing computer-use agents and foundational models. We conducted systematic evaluation 238 tasks from the development set of WARC-Bend examining complementary dimensions that collectively characterize agent capa- bility success rate, agent speed, and throughput (Figure 5)
Web Navigation Agent Performance Analysis on WARC-Bench
(b) Relative Throughput Performance
Success Rate Agent Speed (Per Task)
0.8
Throughput Change (%)
+205-7
Tolic Succe Batee
+387.3%
EIPO
100
*26.0%
0.2
TTENK
Neddo Pa
-3.00%
0:00
Agent Speed (lesks per minute)
L.2
2.M
3
(Cui)
Oood
Cuude- (200)
Figure 5: Accuracy-Latency Tradcoff's baseline computer-use agents v/s our models
In Figure (a), we evaluate Task Success Rate against Agent Speed capturing user-perceived re- sponsiveness. This formulation enables direct visualization of the Pareto frontier where both higher accuracy and faster exccution are simultancously desirable.
Agent Speed measures how many tasks agent complete minute of Agent Action Time. Agent Action encompasses the complete computational pipeline from receiving a browser observa- tion producing executable action including parsing the visual and textual state, constructing prompts, querying the language model, interpreting responses, and formatting the final action com- mand. For each web navigation task, track the cumulative time spent across all these operation cycles and sum them to obtain the total Agent Action Time for that task. For example, if task requires 15 steps where each agent action takes an average of 2.5 seconds, the total Agent Action Time would be 37.5 seconds. Agent Speed is then calculated as tasks per minute based on this total action time.
Our analysis reveals that our approach achieves Pareto-optimal performance, establishing new benchmarks in the efficiency-acent accurray trade-off space. Ours-RLVR variants reach the Pareto frontier with the highest success rates while maintaining competitive task speed. This positions our models as offering the best of both worlds-high accuracy without compromising on the responsiveness that users demand in practical applications.
In Figure 5 (b), we examine relative Throughput performance using Claude-3.7 as our baseline, quantifying computational efficiency gains across model families. We define Throughput tasks processed per hour based exclusively on agent action time. We observed that our model variants have substantial throughput advantages over Claude model family. Both our 7B and 72B variants demonstrate significant efficiency gains, with the RLVR models achieving the highest throughput improvements.
14

---

Published as a conference paper at ICLR 2026
C HYPERPARAMETERS FOR SFT TRAINING
We list out the hyperparameters employed for our best SFT-trained models here. The dataset used for SFT training contains 12k trajectories, which is decomposed into 50k single-step data. For both the 7B and 72B Qwen-2.5- 2.--be- models, we use a learning rate of x 10 batch size of 128, maximum prompt length of 7168, maximum response length of 1024, and train for 1 epoch. We ensure that each data point contains no more than 3 images in the prompt to fit into the maximum response length.
D HYPERPARAMETERS FOR RLVR TRAINING
This section outlines the hyperparameter configurations used for Proximal Policy Optimization (PPO) training of both 7B and 72B parameter models on subtask environments. All configurations were empirically validated to ensure training stability and convergence.
For both model scales, each training step processes 64 tasks with rollouts per task, yielding a total of 256 rollouts. These rollouts are executed across 256 concurrent environments distributed over multiple CPU nodes, with the max step of 10. Both models are trained on 32 NVIDIA H100 GPUs; for the 72B model, tensor parallelism of size 4 is additionally applied. Training proceeds for 25 epochs, and the best- best-perfonming checkpoint is selected based on validation performance.
We adopt the standard clipped PPO objective, augmented with KL regularization term:
(PPO(() (+) E. min (r, clip(r:(f), 1 ..++)),,) 3 KL[ |eel + H(mo], (1)
where (0) is the probability ratio between the and old policies, A, is the advantage estimate, the KL penalty term with coefficient 3 constrains divergence from the reference policy, and the entropy bonus term with coefficient encourages exploration. In our setup, we retain the KL penalty term and omit the entropy bonus by setting = O.
For the 7B model, we employ 4 PPO updates per step with a batch size of 128. The clipping threshold is set to e = 0.20, and the KL coefficient to 8 = 0.10. Constant learning rates (no warm-up) are used for both actor and critic, with the actor learning rate at 5 x 10-6 and the critic 1 X 10
For the 72B model, we employ PPO updates per step with batch size of 128. The clipping threshold is set to 0.15, and the KL coefficient to 8 0.03. Constant learning rates (no warm-up) are again used, with the actor x 10-6 and the critic 5 x 10-6.
E OFFLINE RL APPROACHES
In this section, compare various training approaches that experimented with, including SFT, offline reinforcement learning techniques (Group Relative Policy Optimization (GRPO), Proximal Policy Optimization (PPO)), and online reinforcement learning techniques in live WARC based environments (Proximal Policy Optimization (PPO)). Our SFT models (Ours-7B-SFT. Ours-72B- SFT) are trained with an in-house subtask trajectory dataset, collected using frontier models. The websites used to collect these subtask trajectories include real websites from the Common Crawl index, UI component libraries, and LLM -egeerttd synthetic websites. We refer to this training step as SFT[all].
For all Reinforcement Learning experiments, we begin with our SFT checkpoints and further train them in online / offline learning settings. The training data used for RI. is curated from small scale dataset of 1059 tasks on synthetically generated websites. For offline RI. algorithms, we use foundation models to collect distillation trajectories these subtasks. Our models are trained with the objective to match the teacher model on a per-action level. For online RL, we allow our model to interact with the website and learn from the end-trajectory rewards directly.
In our offline reinforcement learning framework, we use two distinct advantage estimation methods: Group Relative Policy Optimization (GRPO) (Shao et al. 2024) and traditional Proximal Policy Optimization (PPO) (Schulman et al., 2017). Our approach leverages distillation datasets by in- troducing rule-based reward function that evaluates model performance UI interaction tasks
15

---

Published as a conference paper at ICLR 2026
Model
Training Algo.
Dev[TOTAL]] Accuracy (%)
Ours-7B-SFT
SFT[al]]
66.54
Ours-7B Murs-7BGRPPOaar GRPOotfine
SFT[all] GRPO[syythhtiicofflnnn
63.86
Ours-7-PPPO.
SFT[al]] + offfffn
61.76
Ours-7B-ER RR-VRB RLVR
SFl[ + [PO[syntteticcoonine]]]
72.13
Ours-72B-SFT
SFT[al]]
75.88
Ours-72B-GRB 1an-7BBB-RO..iieee
SFT[al]] GRPO[syathetic.fflin
76.89
Ours-72B-PPOommine
SFT[al]] + PPO[fyytheticcooffinee]
76.47
Ours-72B-RLVR
SFT[al]] - PPo[oyythheiicoonline]]
84.31
Table Performance Comparison of Training algorithms on Subtasks Completion (%) models use SVA agent design.
without requiring additional human annotation or online environment interaction. Our reward func- tion employs multi-component evaluation system that assesses correctness of both the action type and the parameters. For example, for coordinate parameter click, we use a distance-based score computed using the groundtruth and predicted coordinate. For text parameter of rype, compute character-level similarity score and binarize with a threshold at 0.8.
We conduct experiments on both 7B and 72B parameter vision- language models, initializing from supervised fine-tuning checkpoints trained on the same UI subtask domain. Both model scales trained for single epoch with identical learning rates across algorithms: actor networks use x 10-6 while critic networks employ x 10-5, The 7B model utilizes a training batch size of 64 with micro-batches of 4 samples per GPU for both GRPO and PPO, whereas the 72B model batch size 32 with single-sample micro-batches to accommodate memory constraints. Maximum sequence lengths are to 8,192 tokens, with prompts capped 7,680 tokens and responses limited to 512 tokens. We employ KI. divergence regularization to prevent policy drift from the supervised baseline, with coefficients of 0.01 for 7B models and 0.001 for 72B models using low-variance KI. estimation. Entropy regularization is disabled (coefficient = 0) to encourage deterministic action selection in the structured UI environment. Training is distributed across high sech-pefformance com- puting clusters with 16 H100-80GB GPUs (2 nodes) for 7B models and 32 H100-80GB GPUs (4 nodes) for 72B models, employing tensor parallelism with degree 4 during rollout generation and Fully Sharded Data Parallel (FSDP) training with both parameter and optimizer state offloading for memory optimization. The offline GRPO PPO variants of our models obtain worse results than the online PPO version, highlighting the necessity to train models via directly interacting with the environments and learn from environment feedback.
F
SVA MODEL PROMPT
Here is the system prompt used in Subtask Vision Agent design:
You are powerful and precise web agent, helping user with their web-related tasks.
Here is the set of actions you can take on a webpage, which you can call as pythen functions. It is a documentation of all the functions, and it's impertant that you read it well and carefully:
<ACTION SPACE>
The user will provide the screenshet of the webpage, and you will need to use the actions to complete the task.
Your outputs should be single python function calls, wi thout any additional text, and should follow the correct formatting given above. Refer to the docurentation to determine appropri itte ate args. DO NOT PROVIDE MORE TEAN ONE FUNCTION CALL AT EACH TURN!
16

---

Published as a conference paper at ICLR 2026
You will alse be given the histery of your previous thoughts and actions, this correct your previous actions intelligently.
In each turn, decide whether the task is already complete or infeasible, reason with the best next action to take, and output the action.
Here the user prompt of the agent:
Please help me! We are trying to complete the following tasks:
<GOAL>
Here is the history of your previous thoughts and actions:
<HISTORY>
If previcus action failed has been repeated multiple times without pesitive outcome, avoid repeating the mistakes. Try a different approach.
The previous action can fail either because the action itself is inappropriate,
because the coordinates of the elements are not correct. Adjust your next action accordingly.
Here is the current screenshot of the webpage, which you can interact with using the actions and whose element coordinates you should remember:
<SCREENSHOT>
Pixel coordinates originate from the top left cornes of the image, where the first coordinate refers to the horizonta//wiath axis and the second refers the vertical/neight axis.
Important: explore, explore, explore! The screenshot is not the entire webpage and you may need to scroll to determine if a task is completable er whether you have gathered all the information you need.
Again, our geal is: <GOAL>
Please answer the following questions in this exact order and format: <thinking>
Bricfly provide your reasoning and explicitly answer these questions to guide your decision:
What is the current state of the webpage?
Was the last action successful and why?
the task COMPLETED right new? Answer yes/no and way.
completed and an answer required, what should be returned to the user?
If infeasible, why? You should tend to think that the goal is achievable until proven otherwise. Only say that it is infeasible after have tried all possible ways complete the task without success.
What should the next action be and why (if not complete)? How should the webpage change if the next action succeeds? Keep this concise and the point (around 80-100 words). </thinking>
<action>
Exactly ONE python function call. </action>
You MJST include the tags <thinking> and <action> in your response and provide a valid single python function cal1 <action>. Do not include any text outside of these tags.
Answer:
17

---

Published as a conference paper at ICLR 2026
Function signature
Description
click(x, Yr button "left", double False)
Move the mouse to the location (x, y) and click the specified mouse button. Optionally perform double click.
complete (answer "" infeasible reason "*)
Complete the task and optionally provide an answer or explain why it is infeasible using infeasible.reason
drag and release (xl, yl, x2, y2)
Press and hold the left mouse button (x1, y drag the mouse to (x2, y2), then release the button.
hover (x, y)
Move the mouse to (x, y) and keep it there.
key key-press (keys)
Press one more keys simultaneously the keyboard.
scroll (x, y, delta_x
Move the mouse (x, y) and scroll the wheel horizon- tally by delta_x and vertically by delta.y.
0, deltayy = 100)
type (x, y, text)
Focus at the location (x, y) and type the string text.
wa it (ms
1000)
Pause execution for ms milliseconds.
Table 6: Function signatures and descriptions of the action types in the action space.
The history (<HISTORY>> provided to the agent has the screenshots and the model responses previous steps of the trajectory. The history is limited to 5 previous steps, and information before that truncated. This approach reduces the prompt length for the foundation models being used within SVA and produces minimal adverse effect to model performance due to the short-horizon nature of GUI subtasks as per our tests.
Note that the model is prompted to answer the following questions during its Chain-of- Shain-of-Thoughe Thought reasoning:
What is the current state of the webpage?
Was the last action successful and why? Is the task completed?
If completed and an answer required, what should be returned to the user? If the task is infeasible, why?
What should the next action be and why?
How should the webpage change the next action succeeds?
G SVA ACTION SPACE
We provide the following description of the action space to the agent in the system prompt (at the <ACTION SPACE> position):
click (x: flcat, y: float, button: Literal [' ['left', (right') ] 'left', double: bool False)
Move the mouse to location and click mouse button. Can be used to click button, select checkbox, focus input field, etc.
Args:
x (flcat): The coordinate of the location to click. y (flcat): The y coordinate of the location to click. button [Literal["left", "right"]): The button to click. double (bool): Whether to double click. Examples:
18

---

Published as a conference paper at ICLR 2026
(324.5, 12) click 34, button="right") (34.1, 120, double=True) click (230, 100, button="left", double-raise)
complete (answer:
" infeasible_reason: str "()
Complete the task and opticnally provide the Fill in the answer if the completion of the task requires providing a response to the user.
feedback.
Fill in the infeasible_redd reasonn if the task is infeasible. DO NOT fill in both answer and infeasible_rease at the same time. Args:
answer (str); The answer to the task, if any.
infeasible_reason (str): : The reason the task is infeasible, if
any.
Examples:
complete (answer=' request a refund, you need to call the customer service at 123-956-7890.'")
task is infeasible because the user has not provided valid email address. complete()
complete tinfeasible_reasonn
complete (answer='") (An "name": "John", An "age"; 30, An "city": "New York"\n)"">)
float, yl: float, x2: float, y2: float)
Press down the left mouse button a location, drag the mouse another location, and release the mouse button.
Can be used for selecting section of text, dragging a slider, etc.
Args:
x1 (flcat): The x coordinate nate of the locati on to press down the left mouse button.
yl (flcat): The y coordinate of the location to press down the left button.
location release the y2 (flcat): The y coordinate of the location to release the left mouse button.
x2 (flcat): The x coordinat
left mouse button.
Examples:
drag_and_release (10.5, 200, 10.5, 230)
hover (x: flcat, float) Move the
to location and stay there.
Can be used to focus location, pop up a tooltip, navigate dropdown menu, etc.
Args:
x (flcat): The coordinate of the location to hover over. y (flcat): The y coordinate of the location hover Examples:
hover (102, 720)
key_press (keys: list [str])
Press
combination of keys at the sane time the
keyboard.
Can be used
- As various shortcuts of the current environment (e.g. ["Control", "a"I, ["Control", "g"]).
To complete
search with ["Enter"].
And any other common actions that can be performed with a keyboard in the relevant application.
This shculd NOT be used to type string of text. Use the type action for that.
The list of allowed keys are:
Fl, F2, F3, F4, F5, F6, 57, F8, F9, F10, Fl1, F12
19

---

Published as a conference paper at ICLR 2026
9, 1, 2, 3, 4, 5, 6, 7, 8, 9
b, d, 8, f, 9r h, i, 3, k, 1, m, n, or Pr qr To 8,
V, ", X, Yr
Backspace, Tab, Enter, Shift, Control, Alt, Delete ArrowUp, ArrowDown, ArrowLeft, ArrowRight Home, End, PageUp, PageDown Args:
keys (list (str]) The list of keys to press.
Examplcs:
key_press ( ["Control", "a"]) # Select all key_press (("oontro" "f"]) # Open the search bar
key_press ["Enter"]) Submit form
key_press ( [F]12"] Open the developer tools in browser
scroll (x: ficat, float, delta_x: float 0, de #loat = 100)
Move the mouse to location and scroll the mouse wheel in the and y rirectiens.
Can be used to scroll webpage, scroll a dropdown menu, etc.
Args:
(fleat): The coordinate of the location to scroll over y (flcat): The y coordinate of the location scroll delta_x (float): The amount to scroll horizontally.
delta_y (float): The amount to scroll vertically.
Examplcs:
scrcl1 320)
scrcll (102, 320, 0, 200) screll (90, 32.7, 0, -300) scrcl1 105, 68, 49.6)
type (x: float, y: float, text: str)
Focus a location and type string text in it. Can be used to type in a text field, search bar, edit a document,
etc.
Args:
(flcat): The coordinate of the location type text in. y (flcat): The y coordinate of the location to type text in. text (str): The text to type.
Examples:
type (102, 70.6, "\nThank you for the coffee! type (44, 120, "Rest sellers")
wait (ms: int - 1000)
Mait for a specified amount of time.
Can be used to wait for webpage load, long form display, etc.
Args:
ms (int): The amount of time to wait in milliseconds.
Examples:
wait ()
wait (1000)
wait (500)
H OPENCUA MODIFIED PROMPT
We used OpenCUA's recommended prompt i.e. the L2 prompt for inference. However, slightly modified the prompt to provide instructions for generating text output inside <answer> tags for the data extraction tasks in our benchmark.
You are a GUI agent. You are given a task and a screenshot of the screen. You will also be given a sequence of previous screenshots and actions taken so far. Based on the information given to you, you need
20

---

Published as a conference paper at ICLR 2026
to perform pyautogui actions to complete the task. Some tasks require you to return answer string. In that case, do the following: Send the answer back to user the computer. teruinare action Include the answer in the 44 Thought: section of the response The answer should be contained inside <answer> tags. For example, if the final answer is [10:00 AMI, the output should be 'The final is (a1:00 AM] </answer>
For each step, provide your response in this format:
Thought:
Step by step Progress Assessment:
AAnalyze completed task parts and their contribution to the overall goal
Reflect potential errors, unexpected resulta, obstacles
previous action incorrect, predict Logical recovery
step
Next Actien Analysis:
List pessible next actions based on current state
Evaluate options considering current state and previous actions
Propose most logical next action
Anticipate consequences of the proposed action
For Text Input Actions:
Note current cursor position
Consolidate repetitive actions (specify count for multiple keypresses)
Describe expected final text outcome
Use first-person perspective in reasoning
Action:
Provide clear, concise, and actionable instructionsl If the action involves interacting with a specific target: Describe target explicitly without using coordinates possible (use original language if
Specify element non con-English)
Describe features (shape, color, position) if name unavailable For windew control buttons, identify correctly (minimize "I", maximize "+", close "X")
if the action involves keyboard actions like 'press', 'write', 'hotkey':
Consolidate repetitive keypresses with count
Specity expected text outcone for typing actions
Finally, output the action as PyAutoGUI code
following
functions:
"description": "Triple click on the screen", "parameters"; ("type": "object", "properties": ["x": ["type": "number", "description": "The coordinate the triple click"), "y": ("type": "number"" "description": "The y coordinate of the triple click")), "required"; ["x", "y"]))
{"name": "computer.triipeeeelle.
["name": "computer tortinattt..miit................ "description": "Terminato the current task and repert its completion status", "parameters"; ("type"; "object", "properties": ("status": ("type": "string", "enum": ("success", "failure"), "description"; "The status of the task")), "required"; ("status"]])
Task Instruction: [1goal))
Please generate the next move according to the screenshot, task instruction and previous steps (if provided).
21

---

Published as a conference paper at ICLR 2026
I CLAUDE/OPENAL COMPUTER-USE AGENT PROMPT
Since Claude and OpenAl's Computer-Use Agent (CUA) has been fine-tuned fixed action space, we don't need to re-define it in the prompt. Instead, we add a light-weight wrapper that passes subtask -optass--peccficc instructions, helping the agent focus data extraction while preventing unwanted behaviors, such as producing a screenshot action.
I need your help to complete computer-use task. I expect to complete the task by determin ni ng which action to take based the input provided to you. will provide you with screenshot the current state of the webpage. You will also have access to the entire chat history, which contains previous screenshots and actions taken.
You have already been trained predict specific for solving computer-use tasks. I am providing you with two additional instructions/guideliii task:
actions
that I want you to follow while solving the
1. Since you will be provided with screenshot of the current state of the webpage, DO NOT output the "screenshot" action.
2. For text or string excraction tasks, the user will explicitly ask you for an answer to a question. In these types of tasks, you won't have to perferm any actions to complete the task. Instead, you should output the final answer string inside
tags. For example, if the final answer is "113:05:00 AM)", you should output "aanser>> AM] </answer> as your answer.
Here is your goal: (1goal))
Here is the current screenshot of the webpage, which you can interact with using your computer-use capabilities: cimngeccurrent_serrnhhe
Answer this question: WHAT should be the next action and WHY?
Use your computer-use capabilities to perform the appropriate action to complete the task.
Answer:
J PLANNER PROMPT FOR HIERARCHICAL AGENT
The Jinja2 template for the prompt used in the Planner sub-agent of the Hierarchical Plan- ner+Executor Agent (Hier. [P+E]) listed in Table is provided below. The Executor agent prompt the same as the SVA agent prompt.
You are a highly capable web navigation assistant.
## How This Works
You are the PLANNER. When you output 'execute subtask') EXECUTOR agent will perform that subtask.
The executor will complete the subtask and report back to you. You will then see the completed subtask in the history below with one of these status indicators:
[[SCCCSS]] The executor successfully completed the aubtask [FAILURE]: The executor could not complete the subtask (e.g., element not found, action not possible)
[TIMBOUT]: The executor took too many steps without completing the
subtask
Based on the status, you should try new strategies based on the CURRENT status of the environment, such as:
22

---

Published as a conference paper at ICLR 2026
Continue with the next logical step toward the overall goal Try alternative approach (different exploration commands, different interaction methods, etc.)
Break the subtask into smaller steps
HP Available Actions
Below is the list actions you perform. It is crucial carefully understand and utilize these actions effectively.
1. 'exccute description of next subtask') Delegate a specific subtask to the exccutor
The executor will perform this subtask and report back when done The executor should be able to complete this task within 1-5 actions.
For example, "scroll down the drop down menu and select men's M size" is geod subtask, while "shop for men's s shirt that is green, size, and less than $20" is not.
The geal needs to be single statement of WHAT to do, NOT HOW to do it or WHY we are doing it.
For example, "Scroll and explore more books until you find 'The Great Gatsby'
good goal, while "Find the book 'The Great Gatsby' to buy it" is not.
- Do NOT give options in your goal to the executor. The intent needs to be singular and clear.
'dream' in the search box" search for hope and dreams is not a good goal, while "Type 'hope' in the search box and press Enter" is a good goal.
For example, "Type 'hope'
Do NOT ask the executor to make judgments or decisions. The executor shculd only follon your instructions.
For example, "Is the book (hhe Great Gatsby' in the search results?" is not a good goal, while "Click on the first result in the search results" is a good goal.
- ABSOLUTELY DO NOT GIVE THE GOAL "Scroll down to see more <ANYTHING>". Tell the executor EXACTLY what you trying find when you it to scroll.
For example, "Scroll down the drop down menu to see more elements" is net a good goal, while "Scroll down the drop down menu to find the Master of Science in Computer Science program" is a good goal.
Be very persistent. Try different approaches before you deem the task infeasible.
Sometimes,, it the original goal is simple enough, just ask the executor to de the entire thing. Splitting up the goal into too many subtasks may be detrimental.
-Examples:
'''eecute Change the departure date in the date picker on the top right corner of the page from 2024/10/01 to 2025/10/01.')
(' down the bottom of the page.') (instead of 'execute down to see more articles.')
* (' Enter in the search b (instead of "'xecute('Press Enter to submit the search query.')
2. 'complete answer confirmation") Mark the OVERALL task
complete
Jse ONLY when the entire geal has been achieved Provide the final answer it one is required
Be very careful when deciding whether the overall task has been achieved. Look closely at the history, the current state of the environmen,, and the goal.
You MUST make sure that the geal has been completed EXACTLY as specified in the goal before calling complete.
Only YOU completes subtask, you will be called again to decide: continue with another subtask, or complete the overall task.
complete
stop the overall task. When the executor
23

---

Published as a conference paper at ICLR 2026
When previding an answer, Look closely and strictly follow any format requirements in the goal.
Examples:
'complete (' San Francisco') (when the goal asks for a city name with nothing else)
'co (when the geal asks for number with nothing else)
3. 'stop reason ('rescnn for stopping') Mark the OVERALL task as infeasible Usc when the task cannot be completed
Provide clear reason
Examples:
stop The website does not have a login option')
stop ("
The The requested information is not availables (' website does not allow cancelling orders.')
## User Instructions
The user will provide the screenshot of a webpage. Use the above actions to navigate and interact with the webpage to achieve the goal.
Required Response Format <progress>
Analyze the current state and provide a detailed sunmary of the progress made far. Include key observations and all relevant details needed achieve the goal.
</progress>
<thinking>
Based on the progress, to complete the task, what should be the next step?
</thinking>
<action>
Write a concise Python-like function call
</action>
Example Response
<progress>
have reviewed pages and 2 of purchase history and identified bags. Their colors white, yellow, and yellow.
</progress>
<thinking>
To determine the colors of all bags I've purchased, I can consider one of the below options:
Navigate to page 3 the purchase history, where more information is likely avail.ble.
2. Search for a filter or summary option to display all bag purchases at once.
3. Check for "view all" button similar feature that aggregates
purchase details across pages.
After ovaluating these options, navigating to page 3 is the most straightforwars and likely to yield inmediate results. It aligns with the structure of the task and the available interface.
</thinking>
<action>
execute ('Click on the page 3 button')
</action>
Human: Your goal [self.goal)
I| Subtask History (previous_steps. str)
## Last Progress
24

---

Published as a conference paper at ICLR 2026
law....ast_pprorreee if self. pasg_ess else 'No progress made yet.')
## Current State <image:screensee
Again, goal is: (self.goal)
## Your response (Again, DO NOT output execute (' Scroll down to see more <ANYTHING>') and be EXTREMELY exact about your answer output format)
K ANALYZING FRONTIER MODEL LIMITATIONS ON
key motivation for developing our benchmark is that existing frontier models, despite their im- pressive reasoning abilities, still struggle on specialized GUI subtasks. In this section, we analyze the performance of frontier models and identify key reasons for their repeated failures on certain types of subtasks.
Suboptimal Interaction Strategies We noticed that the frontier models deployed incorrect strate- gies solve some tasks. For example, in cases where pop-up windows were blocking critical view of the webpage, open source models like OpenCUA and even the frontier models like Claude-Sonnet-4 didn't consider closing the pop-up window in their reasoning thoughts and strategy.
Difficulty with complex input fields and widgets All open-source and frontier models struggled in their interactions with advanced date-pie sate-picker implementations which enforce verifiable format and reset/update cursor positions automatically. Further, these models also fail to identify the correct format of date-picker, for e.g. YYYY/MM/DD MM/DD/YYYY. See Figure 6 In addition, all models struggled to add text to input fields that were pre-filled with existing text. See Figure
Weak Instruction Following Some models, particularly OpenAI-CUA and OpenCUA models were often unable to follow specific instructions on how to solve data extraction tasks. As pointed in Appendices and the models were requested output the final string <answer> tags but were often unable to do so.
Limited exploration and navigation Open- models like OpenCUA and UI-Tars sometimes struggle with adequate exploration of a webpage, especially by scrolling. While the frontier models like OpenAI-CUA and Souude-Sonne-4 explore traditional webpages well, they also struggle to aanvas-based websites.
navigate adequately
Our analysis above shows various reasons why the atomic unit of subtask execution has not yet been perfected by frontier GUI models. Our unique emphasis canvas-based web environments, complex implementations of date-pickers, and adding hostile web elements such as pop-windows makes our benchmark difficult even for state-of-the-art GUI models. This makes our benchmark valuable for the community to analyze and evaluate their models over.
L OUT-OF-DISTRIBUTION EVALUATION
We carefully made sure that the training sets (details next point) do not contain websites used by the WARC-Bench dev and test set. Our training sets do not contain the 5 websites used by WebArena (OneStopShop, GitLab, OpenStreetMap. Reddit, Adobe Magento's admin portal), or any mock websites from MiniWob+H While ScreenSpot v2 did not provide the websites they used for annotation, only 1/3 of ScreenSpotV2 screenshots are from the Web, and the other 2/3 are from desktop mobile devices, which can be considered out-of-domain. For fair comparison, we compute the ScreenSpot v2 scores on desktop mobile screenshots only in Table 7 below, which are similar the numbers in Table 3
25

---

Published as a conference paper at ICLR 2026
Enterprise Resource Management System
Enterprise Resource Management System
Rightt
Comere
Date
Comera
(a) Agent action: keyboard typeey (yp((20241111)))
(b) Post-action webpage state
Figure 6: model passed incorrect date format YYYY/MM/DD
(a) Agent action: keyboard (EECC5E)))
(b) Post-action webpage state
Figure 7: Claude-Sonnet-4 model didn't 't delete pre-existing text
Table 7: OOD ScreenSpot v2 Success Rate (SR) Comparison
Model
OOD ScreenSpot v2 SR % (Desktop / Mobile Only)
Qwen 7B
52.54
Qwen 72B
87.42
GPT-5
29.67
Claude
84.64
Ours-7B-RLVR
78.81
82.74
26


Reviews:
[{"review_id": "sDV1DSh3ke", "reviewer": "Reviewer_iqHj", "summary": "This paper introduces WARC-Bench, a new benchmark for evaluating GUI agents on short-horizon subtasks. It includes 438 tasks across 29 real and 62 synthetic websites, with deterministic programmatic evaluators (e.g., JS/URL/string matchers) and sandboxed replay environments. The paper also introduces Subtask Vision Agent and reports results across models. The authors further train Qwen-based models via  SFT and RLVR.", "questions": "see above", "limitations": "", "rating": 2, "confidence": 4, "soundness": 1, "presentation": 3, "contribution": 2, "strengths": "- The paper identifies an underexplored middle ground between visual grounding (single-step) and long-horizon navigation (multi-page workflows), positioning subtasks as a meaningful intermediate capability.", "weaknesses": "- Over 70% of training and development data is synthetic. While the paper acknowledges this, it remains unclear how well these synthetic subtasks capture the diversity and ambiguity of real-world web UIs. The manual verification process is under-detailed.\n\n\n- The paper champions the high fidelity of WARC files but fails to adequately address their most significant drawback: they are static snapshots. A vast category of real-world web tasks involves interacting with dynamic, real-time, or session-dependent content (e.g., checking live stock prices, booking the last available seat on a flight). By design, WARC-Bench cannot evaluate an agent's ability to handle this dynamism. While the authors briefly mention limitations, they understate how this core architectural choice constrains the benchmark to a specific, offline subset of web interaction, making its claim of testing real-world capabilities only partially true.\n\n- The paper frames \"GUI subtasks\" as a key conceptual contribution. However, this appears to be more of an incremental refinement than a novel problem class. Benchmarks like MiniWoB++ have long focused on widget-based interactions in simplified environments. WARC-Bench can be seen as a high-fidelity, WARC-based instantiation of this existing idea rather than a new paradigm. The contribution is therefore more on the engineering side (the use of WARC).\n\n\n- The paper suggests that adding new environments is as simple as \"adding a new web recording.\" This overlooks the fact that the primary bottleneck in creating meaningful benchmark tasks is the manual and intellectually demanding process of defining a clear, unambiguous goal and, crucially, authoring the programmatic reward function (the \"evaluator\"). This manual effort is not eliminated by the WARC approach. Therefore, the claim that the benchmark design is uniquely scalable compared to others that also require manual task specification is overstated.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "yE5RYcSFG2", "reviewer": "Reviewer_f6t5", "summary": "- This paper introduces WARC-Bench, a comprehensive training and evaluation dataset to train and evaluate models’ and agents’ ability to complete GUI sub-tasks in web environments using computer-use agents. \n- The authors implement an agentic scaffold (subtask vision agent: SVA) for inference and training and train 7B and 72B LLMs using supervised fine-tuning on trajectories sampled from stronger LLMs, followed by online RL (using PPO) on the synthetic training data. \n- Experimental results demonstrate the effectiveness of their approach on the test split of WARC bench while also generalizing to other web navigation tasks/benchmarks.", "questions": "Please address each of the weaknesses.", "limitations": "", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "- The paper is well-written and easy to understand.\n- The task of completing specific sub-tasks on webpages is an interesting method of training models for web navigation tasks.\n- The authors provide extensive comparisons across open- and closed-source models, showing improvements through supervised fine-tuning and RLVR.", "weaknesses": "- Missing baselines for other web navigation datasets: For WebArena, can you please include a comparison of your work with AutoGLM (https://dl.acm.org/doi/abs/10.1145/3637528.3671620), NNet-Nav (https://openreview.net/forum?id=L5BYl0GE2F) and Go-Browse (https://arxiv.org/abs/2506.03533) that also develop training data and train open-source LLMs as web agents. In particular, I am not convinced about the claim of your 7B model generalizing to WebArena benchmark because Go-Browse reports a resolve rate of 8.3% on WebArena with Qwen-2.5-7B-Instruct, which is slightly better/comparable to the performance of your fine-tuned 7B model. \n- Generalizability of results: What are the specific websites used during training and are they semantically similar to the websites used in the evaluation benchmarks, particularly WARC-bench test set, WebArena, MiniWob++, and ScreenSpotV2. It would be helpful if authors could include additional discussion on out-of-domain generalization. For example, if the training set does not include any task dealing with shopping websites, can the trained models solve such tasks in the evaluation split? Relatedly, is there any analysis on how the trained models generalize to subtasks not seen during training? For example, if the training dataset does not include subtasks required to fill forms and manipulate tables, can it solve these tasks during inference?\n- Exact training dataset used for SFT: What are the specific tasks and websites used during the supervised fine-tuning stage? The paper initially mentions the training split of 1059 instances in the WARC bench, but then describes using 12k unique trajectories using teacher models from Common Crawl, and synthetic data? How are the natural language descriptions of tasks designed for these 12K trajectories?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "yaH41OUM66", "reviewer": "Reviewer_pxfP", "summary": "This paper introduces WARC-Bench, a new benchmark on short-horizon web browsing tasks. It is somewhere between single-step visual grounding tasks and sophisticated, long-horizon web interaction tasks. They also developed a way to synthesize more of such tasks. They evaluated multiple computer-use agents and their own Subtask Vision Agent (SVA), using frontier models, open-weights models, and custom models post-trained with SFT and RLVR.", "questions": "The definition of a \"subtask\" seems somewhat arbitrary. Authors argue this is a \"novel\" benchmark designed to evaluate AI agents in executing \"subtasks\" on web browsers. How are these \"subtasks\" different from the tasks in similar benchmarks like WebGames, WebVoyager, WebShop?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 2, "strengths": "1. Innovative way to scale the size of the dataset, including the use of WARC files to quickly reproduce exact replication of complex real-world websites.\n\n2. A good methodology to synthesize data and demonstration to show the effectiveness of training on such datasets.\n\n3. Strong results of the authors' own trained model. Discussion on behavioral analysis of SFT v.s. RLVR is insightful.", "weaknesses": "1. Baseline only used computer-use agents but not web browser agents (e.g. browser-use) or general agents with web browsing capability (e.g. OpenHands).\n\n2. Lack of comparison to other short-horizon browsing benchmarks, e.g. WebGames, WebVoyager, WebShop.\n\n3. Experiments are run 3 times but there's no error bar or other variation info.\n\n4. No human baseline. Since these are short-horizon subtasks, it seems reasonable for human baseline to reach 100%. Lack of human baseline makes it hard to evaluate the soundness of the benchmark (e.g. whether there are ambiguous instructions, wrong answers, cheating possibilities, etc.).\n\n5. Dev accuracy on synthetic dataset is universally higher than dev accuracy on real dataset, indicating the synthetic dataset might have design flaws (e.g. data contamination, overly simple designs, etc.).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)