Let's Think in Two Steps: Mitigating Agreement Bias in MLLMs with Self-Grounded Verification
Authors: Moises Andrade
Joonhyuk Cha
Brandon Ho
Vriksha Srihari
Karmesh Yadav
Zsolt Kira
Venue: ICLR 2026 Poster (2026)
Keywords: Verifiers
Verification
Digital Agents
Web Agents
GUI Agents
Robotics
Large Language Models
Test Time Scaling
WebArena
OSWorld
Reward Models
open-endedness
LLMs-as-judges
Vision Language Models
Primary area: foundation or frontier models, including LLMs

Abstract: Verifiers—functions assigning rewards to agent behavior—have been key to AI progress in domains such as math, code, and games. However, extending these gains to domains without clear-cut success criteria (e.g., computer use) remains a challenge: while humans can recognize desired outcomes, translating this intuition into scalable rules is nontrivial. Multimodal LLMs (MLLMs) emerge as a promising solution, given vast world knowledge, human-preference alignment, and reasoning capabilities. We evaluate MLLMs as verifiers across web navigation, computer use, and robotics, spanning 13+ model families, 28+ evaluation templates, curated trajectories from diverse agents and of varying lengths, and distinct verifier applications. We identify a critical limitation: a strong tendency for MLLMs to over-validate agent behavior—a phenomenon we term agreement bias. This bias is pervasive across models, resilient to test-time scaling, and can harm methods relying on MLLM evaluations, such as filtered behavior cloning and self-improvement. We provide guidance on the design and evaluation of MLLM verifiers, and introduce Self-Grounded Verification (SGV), a lightweight method that harnesses MLLMs' own sampling mechanisms by modulating (un)conditional generation to better leverage their knowledge, alignment, and reasoning. SGV operates in two steps: first, the MLLM is elicited to generate broad priors about desired behavior, independent of the data under evaluation. Then, conditioned on self-generated priors, it reasons over and evaluates a candidate trajectory. Our methods yield gains across models and environments, improving failure detection by up to 25pp and accuracy by 14pp, with benefits extending to downstream applications. In self-improvement and online supervision, SGV boosts task completion of a GUI specialist in OSWorld, a diffusion policy in robomimic, and a ReAct agent in VisualWebArena—setting a new state of the art, surpassing the previous best by 20pp. Finally, we release an updated version of VisualWebArena featuring strong agent baselines, more human-aligned evaluators, high-fidelity environment parallelism, runtime speedups exceeding 10x, and VisualWebArena-Lite, a 1/3-scale subset with comparable evaluation fidelity. Our code, models, and data are publicly available at [our project page](https://mshalimay.github.io/agreement-bias-sgv/).

Published as a conference paper at ICLR 2026
LET'S THINK IN Two STEPS: MITIGATING AGREEMENT BIAS IN MLLMS WITH SEUFNGROUNDED VERIFICATION
Moises Andrade Joonhyuk Cha Brandon Ho
Vriksha Srihari
Karmesh Yadav
Zsolt Kira
Georgia Institute of Technology
(mandrade, jcha, bho36, vriksha.srihari, kyadav32,
zkira) @egeeggatehhc.co
ABSTRACT
Verifiers fonctionn assigning rewards to agent behavior-har been key to Al progress domains such as math, code, and games. However, extending these gains to domains without clear-cut success criteria (e.g., computer use) remains a challenge: while humans can recognize desired outcomes, translating this intu- ition into scalable rules is nontrivial. Multimodal LLMs (MLLMs) emerge as promising solution, given vast world knowledge, human-preference alignment, and reasoning capabilities. We evaluate MLLMs verifiers across web navigation, computer use, and robotics, spanning 13+ model families, 28+ evaluation templates, curated trajectories from diverse agents and of varying lengths, and distinct verifier applications. We identify a critical limitation: a strong tendency for MILMs to over-validate agent behavior-a phenomenon we term agreement bias. This bias is pervasive across models, resilient to test-time scaling, and can harm methods rely- ing on MILMM evaluations, such as filtered behavior cloning and seff-improvement. We provide guidance on the design and evaluation of MLLM verifiers, and introduce Self-Grounded Verification (SGV), a lightweight method that harnesses MLLMs' own sampling mechanisms by modulating (un)conditional generation to better lever- age their knowledge, alignment, and reasoning. SGV operates in two steps: first, the MLLM is elicited to generate broad priors about desired behavior, independent the data under evaluation. Then, conditioned on self-generated priors, reasons over and evaluates candidate trajectory. Our methods yield gains across models and environments, improving failure detection by up to 25 pp and accuracy by 14 pp, with benefits extending to downstream applications. In self- geff-improvement and online supervision, SGV boosts task completion of a GUI specialist in OS diffusion policy in robomimic, and a ReAct agent in Aireal a new state of the art, surpassing the previous best by 20pp. Finally, we release an updated version of VisualWebAren featuring strong agent baselines. more human- aligned evaluators, high-fidelity environment parallelism, runtime speedups exceed- ing 10x, and VisualWehArena-L.i a 1/3-scale subset with comparable evaluation fidelity. Our code, models, and data publicly available at our project page
INTRODUCTION
Several breakthroughs in artificial intelligence can viewed through the lens of search guided by assigning rewards to agent behavior aligned with desired criteria. Notable exam- ples include seminal work Go (Campbell al. (002) and Chess (Silver et al. 2017). where search and learning guided by 0/1 rewards tied to the game's 's final outcome, and recent ad advancements in large reasoning models (LRMs), leveraging formal verifiers code and math (Shao et al. 2024b).
However, while domains such as math, code, and games benefit from relatively well-defined criteria to evaluate agent behavior, this clarity diminishes in open-ended settings. Evaluation in such scenarios often requires nuanced criteria and reasoning over possibly long sequences of multimodal inputs. For example, consider evaluating the trajectory in Fig. (top) produced by a digital agent asked to "add the least expensive opaque phone case shopping cart". Should the agent sort products by price?

---

Published as a conference paper at ICLR 2026
Perform advanced search? What is "opaque enough"? Although humans often recognize satisfactory outcomes, formalizing this intuition into precise and scalable rules remains a challenge.
Multimodal Large Language Models (MLLMs) emerge promising solution to bridge this gap. With vast world knowledge, numan-prefereen hold the potential to serve as general-purpose verifiers, capable of handling inputs and producing rewards in multiple modalities. In this work, we probe this potential through a comprehensive study of MLLLMass-erifieee on open-ended tasks, spanning diverse environments, agent architectures, state-of-the-art MLLMs and LRMs, test-time scaling techniques, verifier designs, and evaluation metrics. We consider web, desktop, and robotic environments VisualWebArena (Koh al., 2024a).
aligmment, and large context windows, MLLMs
OSWorld (Xie et al., (2244, and robomimic (Mandlekar et al. (02T) covering roughly 1,300 tasks across varied domains, where verification demands nuanced criteria and multimodal reasoning.
We identify a systematic and critical limitation: strong tendency for MLLMs to over-validate agent behavior, phenomenon we call agreement bias. As shown in Fig. | (middle), an MLLM verifier validates flawed behavior and even generates chains-of-thought (CoT) to rationalize incorrect judgments. This bias limits MLLMs' ability to fulfill a core function of a verifier-identifyinng flawed behavior and providing feedback to improve performance-posse
risks to methods that rely on MLLM-based evaluations. In particular, can limit MLLMs' effectiveness as data curators for fine-tuning and self- inff-feproeeentttt pipelines (Trabucco et al.
2025a: Pan et al. 2024a Wang et al., 2024 Yu et al. 22255 (Shinn et al., 2023 ; as providers of rewards for training, search, and agent steering (Pan et al. [2024b] Koh et al. [2024b] [Sun et al. 20255] Bai et al. (2024); and judges (Chen)
iet al. 2024 Lee et al. [2024] Zheng et al. (023) and monitors (Baker et al. (022) of agent behavior,
where failure detection is essential for balanced assessments and prevent harmful outcomes.
Notably, such failures occur despite MLLMs exhibiting human-aligned priors about desired behavior, suggesting a bottleneck in knowledge extraction catraction-potentiallly rooted in fundamental limitations in pretraining (Allen- -Allen-Zhu and Li 22024]aa a) and RLHF (Sharma et al., (025)- thatt remains unresolved by major test-time scaling techniques and training for reasoning. To address this, we propose Self- Grounded Verification (SGV), a simple yet effective method that harnesses MILMss own sampling mechanisms by modulating (un)conditional generation to better leverage their knowledge, alignment, and reasoning (Fig. bottom). SGV operates in two steps: first, an MLLM is elicited to produce broad priors about desired behavior, conditioned only on the necessary context to frame the task. Then, conditioned on seff-generated priors, the model reasons over and evaluates a candidate trajectory.
To assess the benefits and limitations of MLLM verifiers, as well as the effectiveness of SGV, we evaluate performance across three representative settings: offfine evaluation of agent performance and two downstream applications
supervision and self-improvement via ReflexionSShinn et al. 2023). Our findings and contributions can be summarized as follows:
Agreement bias is pervasive across MLLMs and LRMs, and is reflected in multiple quantitative metrics, including failure detection rates as low as 50% and output distributions skewed toward favorable judgments. This bias persists across 28+ prompt/scoring templates, techniques to mitigate biases in LLM judgments, and test-time scaling. It holds for both weak and strong agents built from models distinet from the verifier; increases with the capability gap between agents and verifiers; and exacerbated when verification is treated as a binary success-or-faiee
classification.
We elucidate how several applications rely MILLMs acting as verifiers, and sensitive to the quality of their evaluations. We show that agreement bias can impair the use of MILMS for benchmarking agent performance, behavior cloning, self-improvement, and online supervision.
Notably, these findings apply to a verifier that shows high aggregate metrics (e.g., over 90% recall) and state-of-the- art results concurrent reward benchmarks. We discuss key factors for evaluating verifiers, including the importance of fine-grained and downstream metrics, challenges arising from strictness trade-offs, and confounders such as environment bugs and agent strength.
We evaluate several design choices for building and evaluating MLLM- verifiers, including the impact of Likert and score-based scales, trajectory length, agent strength, tools, model calibration, techniques to mitigate biases in LLM judges, and periodic vs. outcome-based verification.
Our methods yield gains of up to 25 pp in failure identification, 14 pp in accuracy, and more human- aligned evaluations across models and environments, with benefits to downstream applications.
self-improvement, our stronger SGV-based verifier yields gains of up to pp (24% relative) on VisualWebArena. In online supervision, it encourages agents to backtrack and avoid greedy
2

---

Published as a conference paper at ICLR 2026
strategies, yielding gains of 9 pp (20%) for ReAct agent on VisualWebArena, pp (22%) for the GUI-Specialist UI-TARS on OSWorld, and pp (33%) for a diffusion policy on robomimic's s tool-hang task. Our agents set new state-of-the-art VisualWebArena, outperforming the previous best by 20 pp while incurring lower token overhead and only utilizing native web actions.
As byproduct of our work, release an updated version of WesualWebAArna featuring more human-aligned evaluators, high-fidelity environment parallelism, runtime speedups exceeding 10x, and VisualWebAreen-Litee a 1/3-scale subset with comparable evaluation fidelity.
expensiiee opeque my
Agant
Trajectory
type search (8)
cD
click "Aod Carf
The tosk done.
X Incomee! Nos the local eoperolvw.
Agreement Bias x
agont succeestulyy
"Phone prone kiendifee relevart product
and addedd to the cart. fully nooompidding the lanc.
MLLM
EVALUATION: success
Caneet should loeep
Trajectory
adding
doing that
Solf-Grounded Verification
First Generation
Accomplihing bask like this o-commerce webr would byskarly inroiig the follwingg staps:
Task
MILMM
Norigale rolovart catopory.....
Bort produccts by price....
REABONING:
The aspistart correctly in likte the search but ald not EVALUATION.
ensure
least oxponsivs sorting or filoring resuls.
Trajectory
Verifler
FEEDBACK: you should sort the search results by price Iind the loast oxpensive Rem.
mace matakal anocud eroure The selected case the leat experdive corting search results by semoving the Beer the
dlick "shooping cart (26)
Figure 1: 'Top example of web task and corresponding agent trajectory, Middle: an MLLM verilier validates and reinforces flawed behavior. generating reasoning ratiomalize incorrect judgments. Bottom: SGV improves verification and enables the agent to backtrack.
2 RELATED WORK
MILMs as Evaluators. (M)LLMs have been employed as evaluators of model outputs in various scenarios under various names (M)LLMs judges, critics, reward models, value functions. This work focuses on multimodal and environment-interacion
scenarios. MLLMS have been used to score and filter agent trajectories for subsequent in finetuning (Trabucco et al. 2025b Pan al. 2024a Ma et al. (024)). and test-time refinements such as to induce prompts, reflections. and tools (Wang al. 2024 Yu et al. 2025 Shiin al. 2023 Sarch et al. 2025). They have also served as source of real-time feedback by producing natural language "critiques" (Pan et al.) 2024b). scores to rank action 2025 and rewards for training (Sun et al., 2025).
proposals in search (Koh et al. 2024c Yang et al.
AI Agents. There is growing interest in building Al agents to act in various environments, including the web (Koh al. 2024a), mobile phones (Li et al. 2024), and computers (Liu al., 2023). In this
field, the work most closely related to ours is Pan et al. (2024a), which uses a GPT-4V-based evaluator, prompted with benchmark-specific rubrics, to evaluate trajectories for Reflexion (Shinn et al. (023) and behavior cloning. Following works employ a similar evaluator to guide tree search (Yu et al. Koh et al. 2024b), filter trajectories to generate text-based memories or tools (Wang ct al.
2024
2024
Yu et al. 2024 Sarch al. 2025) to boost agent performance in (Visual) Web.lleebrreeeeeee (Zhou
et al. 2023
Koh et al. 2024a), and for RL training in simpler environments (Bai et al.) 2024).
Test-time scaling (TTS) is a major paradigm to improve model performance without increasing parameters (Snell et al. [024] Welleck et al. 2024). Early work (Wei et al. [022 Kojjmaa et al.) 2022)
shows that prompting LLMSs to generate "chains-of-thought yields substantial gains in reasoning-
3

---

Published as a conference paper at ICLR 2026
oriented tasks. This idea has been extended to several settings, including multimodal (hhag et al.
2023) and irrroomenntiiteractiin (Zawalski et al. [022]] Yao et al. (003b). Orthogonal approaches scale test-time compute via sampling and search ( Yao al.
2023a Hao et al. 2023). where multiple generations are selected through heuristics (Wang et al. 2022) or verifiers (Shao et al. 2024a
Lightman et al., (022)) Recent work leverages sampling. RI., and formal verifiers to train (M)LLMS
that autonomously generate reasoning traces [DeepSeek-A] 2025) OpenAl| 2024; Kimi Team
2025 Gemini Team 2025). Extending these methods open-ended problems requires flexible and
mu multimodal verification, for which MLLMs offer an appealing solution.
Compared to this body of work, we (1) Unify applications of MLLM evaluators based on their use of multimodal rewards derived from MLLM verifiers. (2) Evaluate MLLM verifiers on multimodal settings across a range of models, benchmarks, agents, TTS techniques, evaluation templates, and applications. (3) Dissect MILM verifier performance over several fine-grained metrics, offering guidance on how to measure the quality of their evaluations and artifacts derived from them. (4) Identify agreement bias, show its resilience to TTS techniques, and the risks it poses for downstream applications. (5) Discuss several design choices for building MLLM verifiers and introduce SGV, a simple yet effective method that can be easily integrated into pipelines involving MLLM verifiers. (6) Propose methods to improve MLLM verification with benefits extending to downstream applications such as online supervision and seff-improvement, achieving new state of the art on VisualWeb
3 PROBLEM SETUP
3.1 PRELIMINARIES AND DEFINITIONS
Our goal is to study functions that approximate human judgment of agent behavior in interactive environments. Specifically, an agent is tasked complete task q E Q through a series of actions 0z- Given a history of environment states Sy:; = [sy,,,, 8z]. the agent ex- ecutes an action sampled from a policy и.: 0g = (Sr::t q). leading to a new state St+1- The repetition of this process yields a trajectory Trit = (Sy, a...... St, ax) that, along with the task serves as the basis for evaluating agent performance. To evaluate trajectory- produce information in multiple modalities, motivating our definition of verifier and the scope of tasks considered. We define a multimodal verifier as a function r T x Q - x (V U (0)), r E R, that maps trajectory-task pairs to rewards consisting of real-valued score and optional outputs in other modalities from V. These multimodal aspects are reflected in our settings. A task g can be represented as images, text, or both, states 8t screenshots DOM trees, actions Oe text (c.g. <type opaque phone>) or images from URLs, and verifier outputs as 0/1 scalars or natural language.
Agents and Multimodal Verifiers.
pairs (g. Tr:t), humans process and
Oracle Verifiers. In the benchmarks we study, instantiations of are obtained via human-writter scripts that have privileged access to task and environment states, which we refer to as oracle verifiers. They produce 0/1 rewards reflecting task completion, which we treat aligned with human judgments. These functions, however, are not perfect, and we discuss limitations in Secs. 3.2 and
Applications of Verifiers. To characterize the potential benefits and risks of MILM verifiers, we unify the view of several applications by how they use r and are therefore affected by its quality, categorizing them into offtine and online. The offline setting covers cases where r is applied to trajectories post hoc. A canonical example is agent performance evaluation, where maps trajectories to scores reflecting task completion. Composite applications include methods that use r to improve the agent's 's subsequent performance, such as filtering successful trajectories for finetuning (Pan et al.) 2024a| Ma et al., 2024). as well as Reflexion-style refinement pipelines, where filters trajectories to induce prompts e.g., memories, reflections (Shinn et al.) 2023 Sarch et al., 2025 Yu et al. 2025). and tools (Wang et al. 20244 2023)- -that are included in the agent's 's context in subsequent executions. The online setting covers cases where influences policy distribution during execution. Applications include online supervision, where r maps trajectories to scalars or text rewards to steer agents toward
task completion and possibly update policy parameters (Bai et al., 2024 Wu et al.) 2025 as well as
using r to rank state-action samples to guide (tree) search (Koh et al. 20246 Zhou et al. 2024).
3.2 MLLM VERIFIERS, AGREEMENT BIAS, AND SELF-GROUNDED VERIFICATION
Human and scripted evaluation for open-ended tasks is hard to scale, motivating the use of MLLMs as alternative. The usual approach to obtain r with an MILM is to prompt with the task q.
4

---

Published as a conference paper at ICLR 2026
trajectory Tr:t, and context C' that can include evaluation rubrics and instructions for reasoning steps:
TMLLM( m, C) = = P(uk yet, Ti, C))
e.g., a where y = V1..... y/ are MLLM outputs, and a function mapping them to regex that maps model completions to 0/1 scores natural language feedback.
However, this approach is subject to a failure mode we term agreement bias (Figs. and 10 a tendency to over- validdal agent behavior, judging flawed trajectories aligned with the task q despite the use of carefully crafted instructions C, established test-time scaling techniques, and decoding algorithms. This bias degrades the quality of MLLM-based rewards, TMLLM- and can negatively impact several applications that rely directly or indirectly on them (Sec. 3.1.
Notably, this bias occurs despite MLLMs exhibiting strong, human-aligned priors on desired behavior-e.g. first-step generations in Figs. and 7 are not properly leveraged during verification, issue that is unresolved by techniques eliciting intermediate reasoning steps. These observations align with results showing that the way a transformer encodes knowledge within its parameters during pretraining can hinder its extraction depending on how information pre- sented (Allen-Zhu and Li 2024a b), and that models may conflate truthfulness with human rater satisfaction due to inherent limitations of RLHF (Sharma et al. 2025). However, addressing these issues by modifying pretraining corpora, training recipes, or retraining large models remains both unclear and costly, highlighting the need for alternative solutions.
Motivated by this, we propose Self-Grounded Verification (SGV) (Fig. (middle)) a simple yet effective method that substantially improves the performance of MLLM verifiers. SGV operates in two steps: first, the MILMM is elicited to extract broad priors hig associated with successful completion of the task q, conditioned only on the data needed frame the task:
ky = 9 (II P(y: #0:1, C.
where So.t are states needed to define the task (e.g., an initial screenshot) and is function to select completion. In the second step, the MILMM evaluates the trajectory conditioned on the first step priors:
Tsav (Te, C, h[T P(y yea, q, Te, C, kg))
Intuitively, by modulating (un)conditional generation, SGV harnesses MLLMs' own sampling mechanisms to enable more effective use of their knowledge, alignment, and reasoning capabilities. Conditioning only essential information in the first step encourages the model to explore its probability distribution freely, extracting knowledge pertinent to the task at hand and independent of the data under evaluation. In the second step, the MILM evaluates candidate trajectory by sampling from a conditional distribution induced by its own priors, for which we expect more balanced distributions and more accurate verification. We hypothesize that MLLMs, given their extensive world knowledge, can generally produce human-aligned priors on desired behavior that can serve as impartial references for grounding the verification, leading to more truthful and accurate verification.
3.3 EVALUATING MILM VERIFIERS
This section outlines several factors considered for a reliable assessment MLLM verifiers and for empirically demonstrating agreement bias and validating our hypotheses.
Environment Diversity and Multimodality. We consider benchmarks that require nuanced ver- ification, multimodal reasoning, and collectively span diverse range of tasks. Visual WebArena (VWA) (Koh al. 2024a) emulates web browser and spans 910 tasks, many of which combine text and image instructions (e.g.. "Buy this product" <image>). OSWorld (Xie et al. (024) emulates computer and comprises 369 tasks involving widely used desktop applications across both single- and multi-application workflows. Finally, robomimic (Mandlekar al. [222]] provides long-horizug robot manipulation tasks, and we focus on tool hang, the most challenging among them, which consists of two subtasks: (1) inserting an L-shaped pencil into a base and (2) hanging wrench on it. Sec. G.T provides illustrations representative tasks and trajectories.
Trajectory Annotations. Similar to prior work (Pan et al. 2024a Xu et al. 2024 Huang al. 2023)
we use benchmark provided oracle verifiers as proxies for human judgment due to the high cost of large-scale annotation. However, observed several issues with the oracles in VWA, issue also
5

---

Published as a conference paper at ICLR 2026
noted in concurrent work (Men al.) To establish a reliable reference, we corrected non-ambiguous issues in the VWA oracles-caa string parsing bugs. mismatches between task intents and oracle requirements, and incorrect annotations. To validate the effectiveness and impartiality of these changes, we evaluated the revised oracles on external labeled trajectories from (Men et al.] (025), observing near-perfect agreement with human judgments (Tab. D). For details about these and other refinements to Visual)WebArent see Sec. [
2025) comparing rule-based evaluation with human annotations.
Agents and Trajectory Quality. Weak agents and buggy environments lead trajectories that are trivial to verify and can artificially inflate verifier performance. For instance, some trajectories in (Men et al.] 2025) exhibit long action loops, as well as "page not found" errors due bugs in the browsergym (de Chezelles et al., (225)) suite. Moreover, incorporating trajectories generated by diverse methods is crucial to ensure the generalization of our findings. Therefore, we fixed several bugs in the (Visual)WebArent) environments and considered agents built from different methods. To generate trajectories in VWA, we build a strong ReAct agent (Yao et al. 2023b) that yields a balanced ratio of successes and failures. For OSWorld, we employ the GUI-Specialist UI-TARS-1.5 (Qin) ct al. 2025), the best-perforning agent on the benchmark at the time of this work. For robomimic, train diffusion policy the expert demonstrations collected by Zawalski et al. 2024).
Choice of Verifier Applications. MLLMs have been used to approximate r' in several of the applications discussed in Sec. 3.11 While exploring this whole range is infeasible, we focus three representative cases that can inform general applicability: offline evaluation of agent trajectories, self- refinement via Reflexion, and online supervision. These applications: (1) introduce fewer confounding factors; (2) are of direct practical interest and often serve as building blocks in larger pipelines; and (3) yield informative signals for broader applicability. For instance, if F'LLM produces many false positives in trajectory evaluation, finetuning on those trajectories is likely to be affected. Similarly, if I'LLMM yields weak rewards for online supervision or Reflexion, it likely to be suboptimal for online training and more complex Reflexion-style self self-refinement pipelines.
Baselines and Ablations. To probe limitations of MLLM verifiers and set strong baselines, we build verifiers with several methods, including Maain-offtthoghhe (CoT) (Wei et al. 2022) and set- of-marks (SoM) prompting (Yang et al., (023), majority voting (Wang et al. (022), and reasoning models. For VWA, we also consider the approach from Pan et al. (2024a). where we additionally provide benchmarkk-peeifff rubrics for evaluation. MLLMS are given full trajectories represented as sequences of screenshot- PARTIAL SUCCESS, or FAILURE, mapped to [1, o, O] to align with oracle scores. For reference, Tab. T0 shows that our baseline MILM verifier is already strong, achieving state-of-the- art performance on AgentRewardBench. SGV further improves upon that, surpassing even the original VisualWebArena oracles. Sec. R ablates on all such choices, including model family and size, prompt/scoring templates, and SGV design and prior generation mechanism.
pairs and asked to assign ternary Likert label: SUCCESS,
Quantitative Metrics. Verifiers should negatively (positively) evaluate trajectories marked as failures (successes) by humans proxies to them. Evaluations consistently biased toward either side are likely undesirable. To capture the degree of alignment in MLLM responses, we evaluate (1) bias, (2) distance skewness, and (3) true positive and true negative rates:
bias- Σ, E[d]], dSkew 1- Ey |d:4ا
Exy 4:4d5T (CR(c) = E. - v P(f: = c e), CE (0
where P is the MLLM verifier reward, r the human oracle reward, and d; f; r:.
Ultimately, a verifier should also be evaluated by its downstream impact: if its feedback is effec- tive, agent performance should improve. Therefore, for downstream applications, use task completion rates (SR) of base agents with and without verifier interventions as the primary metric.
The following highlights key aspects regarding quantitative analysis of verifiers: (1) bias and dSkew, also adopted by Xu et al. (2024), are summary statistics of the distribution of MLLM responses. Positive values indicate rewards systematically higher than those given by humans, whereas values near zero indicate closer aligmment. (2) TNR measures how often MLLMs identify failures among trajectories marked as failures by humans, and is therefore an empirical estimate of the probability of classifying a trajectory as flawed when it truly is. Moreover, we use low TNR and high false- massi-positive rate interchangeably. since TNR = - False Positive (analogous for TPR). (3) While we report multiple metrics for robustness, we emphasize the practical importance of statistics such as TPR and TNR, which directly relate to the core function of verifiers: identifying flawed behavior and providing

---

Published as a conference paper at ICLR 2026
feedback to improve agent performance. Low values indicate not only misalignment but also risks to downstream applications. (4) Accuracy (ACC) is included as an auxiliary metric for interpretation and comparison, as it provides a summary of TPR-TNR trade-offs: ACC (1 SR) TNR SR TPR.
OFFLINE EVALUATION OF AGENT TRAJECTORIES
In this section we evaluate MLLM verification performance for about 1,300 trajectories generated in VisualWebArena (VWA) and OSWorld. Trajectories are based on Gemini 12. 5-F-Fsal in VWA and UI Tarr--rr----- in WSWlrl with success rates of 47% and 22%, respectively. Tabs. and report performance across range of models and test-time scaling techniques. Secs. A D. and provide breakdowns by environment and trajectory length, as well as other ablations.
Table 1: Verification of digital agent trajectories. (a) MLLMs tend to over-validate agent behavior, exhibiting positively skewed rewards, high number of false positives (I-TNR), and probability of flagging failures (TNR low as 50%). (b) SGV improves all metrics across all models and benchmarks.
(a) NGV
(h) SGV
(h) (a)
Model
Acc
TPR
TNR
Bias
dSkew
Acc
TPR
TNR
Bias
dSkew
Acc
Bias
dSkew
Gemini 2.0
61
96
42
36
34
69
94
55
27
23
+9
-9
-11
Gemini Lite
65
49
Gemini 2,5 Flash
68
94
24
80
76
12
+12
-15
-16
Gemini-2.544 2.5-aaii (T)
74
92
18
82
+8
-11
-13
Qwen3-336
69
92
21
76
71
14
+7
-11
-10
Owen3-255b-u226
93
25
76
+10
-14
-13
Seee3-235b.a22bb (T)
24
77
15
+11
-12
-12
GPT-4,1 Mini
60
96
35
74
92
17
+14
-20
-21
GPT-4.1
GPT-o1 (T)
70
80
62
22
16
78
83
73
13
+7
9
8
GPT-o4
+6
GPT-5-Nano (T)
72
84
65
16
11
76
82
73
10
+1
4
3
GPT.5
+3
I.lama4- 1armaa-4-Mavee Maverrikk 17B.11888
60
92
44
33
29
65
89
54
25
22
+5
-7
Thinking
muximum thinking budget for corresponding model.
Agreement Bias. MILLMs display a strong tendency to over-validate agent behavior. This manifests as high number of false positives (1-TNR), responses tilted toward favorable evaluations (strictly
positive bias and skewness), and a low probability of flagging failures (TNR as low as 50%). As shown in Tab. 2 this pattern persists despite techniques such as CoT and SoM (rows and 4), majority voting (row 5), and the inclusion of task-specific evaluation criteria (row 6). Figs. 7 and illluustraee a reason behind this pattern: MLLMS generate reasoning that rationalizes flawed trajectories and their wrong judgments. Sampling fails to mitigate this issue and may exacerbate it, as higher temperatures can increase the likelihood of hallucinations and spurious correlations.
Table 2: Major test-time scaling techniques fail to mitigate agreement bias, whereas SGV improves all metrics.
Table 3: (A:: Distribution MLLM evalus- averaged 28 scoring templates. (B): MLLM-oracle agreement rates over multiple samples stratified trajectory status.
YebAArae
OSWorld
(A)
PS
PPF
Method
Avc
IPR
TNR
dSkew
Acc
TPR
TNK
dSkew
Oraele 57
97
No SGV (binary)
4
2
3
CoT too SoM)
OST (hinary)
CoT
64
59
65
90
91
47
24
36
28
21
24
29
71
71
69
99
99
97
64
61
28
30
No SGV
SGV
4
6
23
C&T (M)
67
92
53
4%
27
23
69
97
62
30
Visua/WbAAema
OSWorld
Taining
All
Thinking (M)
55
76
95
SGV
48
71
93
9 SGV
76
84
71
83
92
81
13
No SGV (T)
68
55
90
79
65
SGV
77
86
70
87
91
86
SGV
75
70
85
77
*(M) majority voting; (T) = thinking enabled.
Samples
7,280
2.784
Oracle success rates are 47% in VisualWehArem and 22% in OSWorld.
"(P)F/S denotes trajectories labeled (Partial) Failure/Succoss, U Utoertaii.
Tab. further contextualizes results through the distribution of MLLM responses. The top panel shows distributions averaged across 28+ evaluation templates that incorporate commonly used bias bias-mitigation strategies, such criteria-order randomization and label rephrasing (see Sec. E.2
7

---

Published as a conference paper at ICLR 2026
for details and all histograms). The bottom panel reports oracle-MLLE agreement rates over 10,000+ samples stratified by success and failure subsets, measuring the probability of sampling correct verification from the MLLM output distribution. Tab. (top) reveals that (i) MLLMs tend to concentrate responses in high score regions; (ii) binary scales (e.g., success/failure as adopted in (Pan et al. 2024a) and subsequent work) tend to amplify this bias (e.g.. 72% SR 57% expected); and (iii) increasing label granularity (as in our ternary Likert-scale) can partially mitigate this issue. Tab. (bottom) shows that over-validation is reflected in the model's output distribution: on failure subsets, the probability of sampling a correct response from the MLLM is at or below chance (e.g.. 48% in VWA). This imbalance explains the ineffectiveness of methods like majority voting (that rely on the mode of the distribution), model calibration (Sec. [..3I). and opens opportunities for test- and training-time methods that account for during sampling (see Sec. E.3.2 for proof of concept).
Cross-Model Variations and Verification Difficulty. Agreement bias arises even when verifiers and agents are built from different model families, sizes, and methods-in Tab. [T models such as GPT-4.1, and Qwen3 belong to distinct families and are stronger than the models used to build the agents. Likewise, it occurs for trajectories produced by weak and strong agents, as evidenced by the suboptimal performance in both OSWorld (22% SR) and VWA (47% SR)sTabs, 2 6 and 7 and weaker agents in VWA (Tab. T7p. It is, however, more severe when models are weaker (e.g., 1- mini) and for trajectories generated by stronger agents. This suggests that closing the gap between agent and verifier capabilities can alleviate (though not eliminate) the issue, whereas strategies such as building verifiers with (ensembles of) models that differ from the agents are insufficient.
Adverse Impacts. The bias toward over-validation means that MLLM verifiers fail precisely when most needed-when agent behavior is flawed and requires correction- consequences extending beyond imprecise evaluation of agent performance. For example, for the task "Buy the cheapest <product> from <category> with <attribute>> ", a trajectory as in Fig. 7 where the agent searches for "<product>" ", clicks the first result, and adds it to the cart, is deemed successful despite omitting steps such as filtering, sorting, and completing the checkout. Fig. T0 illustrates an even more extreme case: the agent produces an answer unsupported by any trajectory information, yet an ML.LM verifier validates iv-n though, by construction, no information exists to justify such a judgment.
Relation to Other Biases. Agreement bias has distinct characteristics from other biases identified in LLM literature, such as "self-bias" (Xu et al. 2024 Panickssery et al., 2024| Chen et al. 2025b) where models favor their own text generations and for which external knowledge injection is typical remedy- and biases attributable to positional or phrasing dependencies in evaluation templates (Zheng
et al. 2023; Chen et al.) 2025a Ye et al., 2024). In settings, agreement bias arises despite the verifier's inputs omitting text directly produced by agents, and including multimodal inputs augmented with external information derived from truthful environment data, such as screenshots augmented with Set-of-Marks, and low-level action representations enriched with DOM data. Likewise, it persists when agents and verifiers are built from different methods and models and despite interventions such as label shuffling and rephrasing, as well as access to grounding tools (Tab. 3 and Secs. E.2 and E.8
Effectiveness of SGV. SGV improves verification across all metries, models, and benchmarks. increasing TNR by up to 25 percentage points (pp), overall accuracy by up to 14pp, and promoting responses more aligned with human preferences. This is reflected in reduced bias and skewness, and in more balanced response distributions across 28+ evaluation templates (see Tab. Tab.B(top)). as well as in the models' output distribution (Tab. (tab./oottom) These gains hold even in settings where the verifier is weaker than the generator (e.g., Gemini 2.0 and GPT-4. see also Sec. E.6 Moreover, SGV outperforms instructions with task-specific evaluation criteria (Tab. 2 row 5), indicating it enables models to generate completions to condition themselves that surpass human- offering a more scalable alternative. Finally, additional results in Sec. E.8 show that SGV outperforms grounding via web-search tools and is robust to moderate noise in priors generated in the first step; that weaker models can produce effective priors for stronger models and models of different families; and that multiple and diverse priors in the first step can further improve performance.
rubrics,
Remarkably, SGV (i) enables non-reasoning models to match the performance of reasoning counterparts, and (ii) boosts the accuracy of reasoning models by up to 1pp-- perhaps surprising result, given that LRMs are explicitly trained to produce intermediate traces, and interventions can degrade performance (OpenAl 2025). These results align with intuitions about limitations stemming from earlier phases of LLM training, raising questions about the potential benefits of augmenting
8

---

Published as a conference paper at ICLR 2026
eussoning-orienter training with SGV step. pretraining data rewriting (Allen- and or methods that account for imbalances in MLLMs' output distributions.
Li 2024a b).
Fine-Grained Metries. These results highlight the importance of reporting fine-grained metrics in works proposing MLLMs in evaluative roles and artifacts produced by them. Given the trade-offs inherent to verification, it is crucial to report statistics capturing both the probability of identifying cor- rect behavior and especially, flawed behavior. Reporting only single or aggregate metrics can be mis- leading. As shown in Tabs. [ verifiers can display about 97% recall and 70% accuracy, yet mis- classify ~50% of failed trajectories as successes, which can severely harm downstream applications.
Seniency-Strictnee Trade-offs in Verification. For some models, SGV can lead lower TPR. This pattern stems from two main causes: disagreements with lenient oracles on simplistic tasks and stricter verification. Specifically,, evaluation in these digital benchmarks is based on hard-coded rules applied to subset of states. This means that trajectories such as in Fig. where an agent "Buy the cheapest <product>," by searching for "<product>," "" and purchasing the first result are deemed successful by oracle scripts. Influenced by agreement bias, MLLMs tend to validate such trajectories, but when SGV applied, this behavior is rejected for lacking steps that confirm that the item is the cheapest, thereby reducing TPR. On the flip side, Fig. [5] illustrates case where SGV only validates an otherwise correct trajectory after the agent performs extra steps to double-check item attributes.
These examples highlight the challenges of open-endde eneeeended verification, as well the potential and limits of MLLMs as alternatives. For humans, it is readily apparent that behaviors such as in Fig, 2 do not generalize. Yet, translating this intuition into precise rules is far from trivial: stricter criteria reject valid solutions, whereas permissive ones may encourage brittle behavior. MLLMs, like humans, offer flexibility in interpretation, but this comes at the cost of formal guarantees and vulnerabilitie--withh agreement bias being a strong one. In the next section, we analyze the impact of these trade-offs on downstream applications and show that SGV interventions are typically non-disruptive (Fig. 15 ultimately improving task completion through more generalizable behavior.
DOWNSTREAM APPLICATIONS
In this section, we evaluate MILM verifiers on downstream applications, focusing on their ability to boost agent performance in self-immpovement and online supervision. These are applications of direct interest that, so far, have shown limited benefits and occasional performance degradation (Pan et al.
2024b Huang et al. 2023 [Shinn et al., 2023 Stechly et al. 2024| Pan et al. [024b] Shinn et al. 2023] Huang et al. 2023] Kamoi et al., 2024] Stechly et al. 2024). Importantly, y, these experiments
provide assessment of verifiers' net impact, given the trade-offss discussed in Sec. 4 Below we
discuss main results; for experimental details and qualitative analysis, see Secs. A.2 and C.2
Self- mete-Ipprovemet. In this setting, after each episode: (i) verifier evaluates the trajectory; (ii) the agent reflects on its previous attempt conditioned on the verifier's 's evaluation; and (iii) reflections are given to the agent in subsequent attempts to enable self-correction over time. As shown in Fig. [ under oracle verifier supervision, task success rate (SR) increases by up to 21 pps after three iterations, confirming the feasibility and defining an upper bound for self-refinement. When supervised by the (strong) bascline MLLM verifier, the base agent's 's performance quickly plateaus, yielding minimal improvements, whereas SGV enables consistent progress, boosting SR by up to 10.4 pps. This pattern demonstrates how agreement bias limits the effectiveness of MLLM verifiers. Referring back to Sec. 4, a TNR close to 50% implies that the verifier fails to provide a corrective signal in about half of the cases where agent behavior is flawed, hindering its ability to self-improve
Online Supervision. Similarly, agreement bias impairs the effectiveness of MLLM-derived rewards for online methods. Tab. 4 shows that the (strong) baseline verifier fails to meaningfully improve digital agent performance. In contrast, SGV boosts SR by 9 percentage points (pp) on VisualWe- bArena (20% relative) and by 5 pp OSWorld (22% relative). Notably, our agent sets of the art on VisualWebArena surpassing the previous best by 20 pp (58% relative) while requiring substantially fewer tokens, no access to prior trajectories, and using only native web actions.
state
few factors explain these results. First, SGV identifies suboptimal behavior and provides feedback that enables agents to backtrack and complete the task (e.g., Fig. [3)). Without SGV, agreement bias prevents verifiers from intervening precisely when agent behavior is flawed, limiting their effec- tiveness (e.g., Fig. 7). Second, while SGV can lead to stricter verification, interventions are mostly

---

Published as a conference paper at ICLR 2026
Table 4: Task success rate and token usage (in paren- thesis) on VisualWebArena and OSWold
MLLM Verifies, to SGV Uracle Veriferr
MLLM Veriferr, SGV Baseline Reflesion
VisualWebArem
OSW
63
Method
Success Rate (%)
All
S/C/R
All
57.8
58
Search Agent 2024b
29 9x)
34/22/ 30
R-MCTS Ya ét 2024
(9.3k.3)
41/29/32
Human
(-)
88/91/87
Basc Agent
45 (1x)
50/35/48
VVeriier, no SGV
(1.5x)
52/36/49
24
45.5
46.
VVerifler, SGV
(2.2.))
56/43/58
27
"Averages across runs, S/C/R denote shopping, classifieds, and reddit subsets of Visual ArbAran..
43.
Attempt Number
Table Diffusion policy steering on robomimic.
Figure 2: Self-Improvement in VisualWebA Agreement bias prevents MLLM verifiers from pro- viding corrective feedback when agent behavior is flawed, hindering its ability to self-improve. SGV re- duces agreement bias, yielding consistent gains across refinement iterations.
Method
FR
PSR
SR
Reelaans
Diffission Policy
32
Verifier, SGV
22
78
16
89
Verifier, SGV
28
72
32
96
"FR, PSR and SR are the proportions of roll-outs with D, I and 2
subtasks (out 2) completed.
non-disruptive. For example, in Fig. 15 the verifier rejects a greedy strategy that technically satisfies the benchmark, prompting the agent search for and confirm user-requested attributes, ultimately leading to task completion through a more robust approach. Fig. quauttiiiss this pattern: without SGV, the verifier endorses flawed behavior in 30% of the tasks, offering no signal for improvement. With SGV, 10% of tasks improve due to accurate failure detection, and among the 7.4% of tasks marked as false negatives, 6.6% remain successful, whereas only 0.8% transition failure. Likewise, robomimic results (Tab. 5) show that the baseline MLLM verifier is able to guide the policy to partial completion (high PSR). However, influenced by agreement bias, struggles to further guide the policy toward full completion, as evidenced by the low number of replans and SR lower than the baseline. In contrast, SGV triggers replans more often, improving over the baseline by pp. For a categorization of factors influencing success rates and the impact of verifier design choices, see Secs. C.2 and E.4
Discussion. These results illustrate the risks posed by agreement bias: MLLMs' tendency to over-validate agent behavior leads to unreliable evaluations precisely when most seeded-when behavior is flawed and requires correction- cinrttion-immiingg limiting their effectiveness for trajectory selection and online feedback. Notably, this holds for an MLLM verifier with strong numbers in judging trajectories post-hoe: over 91% recall (higher than SGV) and state-of-the-ar performance (precision) on AgentRewardBench. This highlights the importance of evaluating verifier performance using (i) fine-graie fine-grained metrics and (ii) downstream applications, as leniency-strietee trade-offs in verification can render post-hoc metrics insufficient characterize verifier's practical utility.
6 CONCLUSION, LIMITATIONS, AND FUTURE WORK
We identify agreement bias, a critical limitation that hinders MLLMs from serving as verifiers of agent behavior. We demonstrate its adverse effects on existing applications, discuss methods for evaluating and improving MLLM-based verification, and introduce SGV, a simple yet effective method that improves verification across multiple models and environments. Although SGV mitigates agreement bias, it does not climinate it. Our qualitative analysis (Sec. C.I) indicates that remaining failures often stem from limitations in base models' integration of visual perception and language. Future work may explore methods to enhance these capabilities, such as integrating visual experts for fine-grained perception, potentially yielding gains complementary to SGV. In parallel, compelling directions for open-ended verification include combining MLLMS with symbolic methods (Kambhampati et al.) 2024) and training- or best-time strategies that account for skewness in MLLM output distributions associated with agreement bias (Sec. E.3) Finally, important question concerns when to invoke verifier and the relative value of process and outcome- nutcome-based verification, particularly in digital environ- ments where state spaces are often discrete, and actions can be irreversible or destructive (Sec. E.4
10

---

Published as a conference paper at ICLR 2026
REFERENCES
Zeyuan Allen-Zhu and Yuanzhi Li. Physics of language models: Part 3.1. knowledge storage and extraction, 2024a.
attpa///rrrraaaoaaaao/a/a 2309. 2339.14316
2
32
Zeyuan Allen-Zhu and Yuanzhi Li. Physics of language models: Part 3.2, knowledge manipulation, 2024b. URL htt.:ttsss//axii..rrrreeee httprrgabb/233333 22409.44.222
32
Hao Bai, Yifei Zhou, Mert Cemri, Jiayi Pan, Alane Suhr, Sergey Levine, and Aviral Kumar. Digirl: Training in-the- wild device-control agents with autonomous reinforcement learning, 2024. URL
fttps://arxiv: org. wwt.o//g/g////oo
11896
Bowen Baker, Joost Huizinga, Leo Gao, Zehao Dou, Melody Guan, Aleksander Madry, Wojeiech Zaremba, Jakub Pachocki, and David Farhi. Monitoring reasoning models for misbehavior and the risks of promoting obfuscation. arXiv preprint vrXiv:2503.11926 2025. 2
Murray Campbell, A Joseph Hoane Jr, and Feng-hsiung Hsu. Deep blue. Artificial intelligence, 134
(1-2):57-83, 2002. I
Dongping Chen, Ruoxi Chen, Shilin Zhang, Yaochen Wang, Yinuo Liu, Huichi Zhou, Qihui Zhang, Yao Wan, Pan Zhou, and Lichao Sun. Milm-as-a-judge: Assessing multimodal llm-as-a-judge with vision-language benchmark. In Proceedings of the s1st International Conference on Machine Learning (ICML), 2024. URL https: juiennuude..... qithub. 1o/ htttps:/aaxxi org/ /ogg/bb//a 2422 0470047778 788v3
2
Meilin Chen, Jian Tian, Liang Ma, Di Xie, Weijie Chen, and Jiang Zhu. Unbiased evaluation of large language models from a causal perspective, 2025a. URL httpss:/arrivv.rrg///
06655 g 31
Wei-Lin Chen, Zhepei Wei, Xinyu Zhu, Shi Feng, and Yu Meng. Do llm evaluators prefer themselves for a reason?, 2025b. URL URL http:sttss:///xxvv.oom haaggg/as/25000 /2504. 03846
Thibault Le Sellier de Chezelles, Maxime Gasse, Alexandre Lacoste, Massimo Caccia, Alexan- dre Drouin, Léo Boisvert, Megh Thakkar, Tom Marty, Rim Assouel, Sahar Omidi Shayegan, Lawrence Keunho Jang. Xing Han Lü, Ori Yoran, Dehan Kong, Frank F. Xu, Siva Reddy, Graham Neubig, Quentin Cappart, Russ Salakhutdinov, and Nicolas Chapados. The browsergym ecosystem for web agent research. Transactions on Machine Learning Research, 2025. ISSN 2835-8856. httttts:s//ppereeee hwwpeeeeiiiieeee/rorrrrreeeeeeee net forum? 11d (0dd=29299KKmm33 3Gmv3
Expert Certification.
31
42
DeepSeek-AI. Deepseek-rl: Incentivizing reasoning capability in llms via reinforcement learning,
2025. (RRppttss://arxttt Stctps///arxiv..oc///bb//////// wwtp:/o///aaaa///aaeee 12948
Gemini Team. Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality. long context, and next generation agentic capabilities. Technical Report- -, Google DeepMind, Mountain View, CA, June 2025. Accessed July 2025.
Zhibin Gou, Zhihong Shao, Yeyun Gong, Yujiu Yang, Nan Duan, Weizhu Chen, et al. Critic: Large language models can self-correct with tool-interactive critiquing. In The Twelfth International Conference on Learning Representations. 38
Shibo Hao, Yi Gu, Haodi Ma, Joshua Jiahua Hong, Zhen Wang, Daisy Zhe Wang, and Zhiting Hu. Reasoning with language model is planning with world model, 2023. URL httpt::tt/// /otss://aaaa //axxiv
org/abs abs 2305. 14992 4
Jie Huang, Xinyun Chen, Swaroop Mishra, Huaixiu Steven Zheng, Adams Wei Yu, Xinying Song, and Denny Zhou Large language models cannot self-correct reasoning yet. arXiv preprint
arXiv:2310.01795 2023. 5
Subbarao Kambhampati, Karthik Valmeekam, Lin Guan, Mudit Verma, Kaya Stechly, Siddhant Bhambri, Lucas Saldyt, and Anil Murthy. Llms can't 't plan, but can help planning in imduoo frameworks, 2024. URL URL http:p/:///aaai..oor////////////
2.222.1110 01817
10

---

Published as a conference paper at ICLR 2026
Ryo Kamoi, Yusen Zhang, Nan Zhang, Jiawei Han, and Rui Zhang. When can Ilms actually correct their own mistakes? a critical survey of self-correction of Ilms. Transactions of the Association for Computational Linguistics, 12:1234-255,, 2024. g
Kimi Team. Kimi k1.5: Scaling reinforcement learning with llms. arXiv preprint arXiv:2501.12599 2025. 4
Jing Koh, Robert Lo, Lawrence Jang, Vikram Duvvur, Ming Chong Lim, Po- Yu Huang, Graham Neubig, Shuyan Zhou, Ruslan Salakhutdinov, and Daniel Fried. Visualwebarena: Evaluating multimodal agents on realistic visual web tasks. arXiv preprint arXiv:2401.13649, 2024a. 3
16
21
Jing Yu Koh, Stephen McAleer, Daniel Fried, and Ruslan Salakhutdinov. Tree search for language model agents. arXiv preprint arXiv:2407.01476, 2024b.
3
39 40
Jing Yu Koh, Stephen McAleer, Daniel Fried, and Ruslan Salakhutdinov. Tree search for language
model agents, 2024c. URL htttt://aaxii org. wtt./org//////////// 01476
3
Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa. Large language models are zero-shot reasoners. Advances in neural information processing systems, 35: 22199-22213, 2022. 3
Seongyun Lee, Seungone Kim, Sue Hyun Park, Geewook Kim, and Minjoon Seo. Prometheus-vision: Vision-language model as a judge for fine- grainer evaluation. arXiv preprint arXiv:2401.0655)
2024. URL https: /
com wttp:/aiistttaaaaa TawtsstAI/promtttt www.ooeethuuuueeooeemeee vision https://
a arxiv. www.vorgg/ooo abs 224.bbolvv0010..
Junnan Li, Dongxu Li. Silvio Savarese, and Steven Hoi. Blip-2: Bootstrapping Linuuage-image pre-training with frozen image encoders and large language models, 2023. URL https://
arxiv. Dexiv.org/bss 2200 12597
16
Wei Li, William Bishop, Alice Li, Chris Rawles, Folawiyo Campbell-Ajala, Divya Tyamagundlu, and Oriana Riva. On the effects of data scale computer control agents. arXiv preprint
arXiv:2406.03669, 2024. 3
Hunter Lightman. Vineet Kosaraju, Yura Burda, Harri Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. Let's verify step by step, 2023. URL
https: /ttppii///rxv.../// www.oo/gg/b.... httpg//bs/2335.. 20t../255000000
4 35
Xiao Liu, Hao Yu, Hanchen Zhang, Yifan Xu, Xuanyu Lei, Hanyu Lai, Yu Gu, Hangliang Ding, Kaiwen Men, Kejuan Yang, et al. Agentbench: Evaluating Ilms agents. arXiv preprint
arXiv:2308.03688 03688, 2023. 3
Yecheng Jason Ma, Joey Hejna, Ayzaan Wahid, Chuyuan Fu, Dhruv Shah, Jacky Liang, Zhuo Xu, Sean Kirmani, Peng Xu, Danny Driess, Ted Xiao, Jonathan Tompson, Osbert Bastani, Dinesh Jayaraman, Wenhao Yu, Tingnan Zhang, Dorsa Sadigh, and Fei Xia. Vision language models are in-context value learners, 2024. URL
/2411 C4549
3 4
Ajay Mandlekar, Danfei Xu, Josiah Wong, Soroush Nasiriany, Chen Wang, Rohun Kulkamn,, Fei- Fei, Silvio Savarese, Yuke Zhu, and Roberto Martin-Martin. What matters in learning from offline human demonstrations for robot manipulation. arXiv preprint 208.::2200802298 2021.
Tianyi Men, Zhuoran Jin, Pengfei Cao, Yubo Chen, Kang Liu, and Jun Zhao. Agent-rewarddennch Towards unified benchmark for reward modeling across perception, plamning, and safety in real-world multimodal agents, 2025. httptt/pss///xi..oo///////aa///e
40
OpenAl. Learning to reason with 1lms.
ndex/
learning stariing-to-reaale teattorreeso-aao th 11ms/ September 2024.
Accessed: 2025-77-09
OpenAl. Reasoning best practices.
https: http::::/ppaaffooo.ooo www.::..pppe.i...m com/ docs/
reasoning -e-ttningneetttt -prctt ices. 2025. Last accessed July 9, 2025. 15
12

---

Published as a conference paper at ICLR 2026
Jiayi Pan, Yichi Zhang, Nicholas Tomlin, Yifei Zhou, Sergey Levine, and Alane Suhr. Autonomous evaluation and refinement of digital agents. arXiv preprint arXiv:2404.06474, 2024a. 5
45
Yichen Pan, Dehan Kong, Sida Zhou, Cheng Cui, Yifei Leng, Bing Jiang, Hangyu Liu, Yanyi Shang, Shuyan Zhou, Tongshuang Wu, al. Webcanvas: Benchmarking web agents in online environments. arXiv preprint 2rXiv:2406.12533 2024b.
Arjun Panickssery, Samuel Bowman, and Shi Feng. Llm evaluators recognize and favor their own generations. Advances in Neural Information Processing Systems, 37:68772-68802, 2024. g
Yujia Qin, Yining Ye, Junjie Fang, Haoming Wang, Shihao Liang, Shizuo Tian, Junda Zhang, Jiahao Li, Yunxin Li, Shijue Huang, et al. Ui-tars: Pioneering automated gui interaction with native agents, arXiv preprint arXiv:2501.12222 2025.
16
Pooyan Rahmanzadehgervi, Logan Bolton, Mohammad Reza Taesiri, and Anh Totti Nguyen. Vision language models are blind: Failing to translate detailed visual features into words, 2025. URL
https:
2407 06581
28
Allen Z. Ren, Anushri Dixit, Alexandra Bodrova, Sumeet Singh, Stephen Tu, Noah Brown, Peng Xu, Leila Takayama, Fei Xia, Jake Varley, Zhenjia Xu, Dorsa Sadigh, Andy Zeng, and Anirudha Majumdar. Robots that ask for help: Uncertainty alignment for large language model planners, 2023. Utt petttp::/aarivv.rrgp/ abs 2307 928 35
Gabriel Sarch, Lawrenee Jang, Michael J. Tarr, William W. Cohen. Kenneth Marino, and Katerina Fragkiadaki. Vlm agents generate their own memories: Distilling experience into embodied programs of thought, 2025. URL
wwt.aawwww...rorrsaaa////se/saae 2406 14596
Amrith Setlur, Nived Rajaraman, Sergey Levine, and Aviral Kumar. Scaling test-time compute without verification or rl is suboptimal. arXiv preprint arXiv:2502.12118, 2025. 35
Rulin Shao, Shuyue Stella Li, Rui Xin, Scott Geng, Yiping Wang. Sewoong Oh, Simon Shaolei Du,
Nathan Lambert, Sewon Min, Ranjay Krishna, Yulia Tsvetkov, Hannaneh Hajishirzi, Pang Wei Koh, and Luke Zeutlemoye. Spurious rewards: Rethinking training signals in rlvr, 2025. URL
hnttps:/aase /ttp://aaivvvvv/aa wttp://pppp...so//pp///////// 10947
38
Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, Y. K. Li, Y. Wu, and Daya Guo. Deepseekmath: Pushing the limits of mathematical reasoning in open language models, 2024a. URL
24C2. 03300
Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Y Wu, et al. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402.03300 2024b.
35
Mrinank Sharma, Meg Tong, Tomasz Korbak, David Duvenaud, Amanda Askell, Samuel Bowman, Newton Cheng, Esin Durmus, Zac HatfieldDDodss, Scott R. Johnston, Shauna Kravec, Timothy Maxwell, Sam McCandlish, Kamal Ndousse, Oliver Rausch, Nicholas Schiefer, Da Yan, Miranda Zhang, and Ethan Perez. Towards understanding sycophancy in language models, 2025. URL
aattps://arxir ott. wtt.://wwwwwww.w......... 201..1311114411
5 32
Noah Shinn, Federico Cassano, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao. Reflexion: Language agents with verbal reinforcement learning. Advances in Neural Information Processing Systems, 36:8634 06:8634-8652.. 2023.
David Silver, Julian Schrittwieser, Karen Simonyan, loannis Antonoglou, Aja Huang, Arthur Guez, Thomas Hubert, Lucas Baker, Matthew Lai, Adrian Bolton, et al. Mastering the game of go without human knowledge. nature, (5007676):354-350 2017.
Charlie Snell, Jachoon Lee, Kelvin Xu, and Aviral Kumar. Scaling Ilm test- compute optimally can be more effective than scaling model parameters. arXiv preprint 03314, 2024.
13

---

Published as a conference paper at ICLR 2026
Kaya Stechly, Karthik Valmeekam, and Subbarao Kambhampati. On the self-verification limitations of large language models on reasoning and planning tasks. arXiv preprint arXiv:2402.008/55 2024.
Linzhuang Sun, Hao Liang, Jingxuan Wei, Bihui Yu, Tianpeng Li. Fan Yang, Zenan Zhou, and Wentao Zhang. Mm-verify: Enhancing multimodal reasoning with chain-of-thought verification, 2025. BtRLsttps::2 http:////w.wi.....aaao.o////
1383 2
Gokul Swamy, Sanjiban Choudhury, Wen Sun, Zhiwei Steven Wu, and J Andrew Bagnell. All roads lead to likelihood: The value of reinforcement learning in fine-tuning, arXiv preprint
arXiv:2503.00 0066 2025. 35
Brandon Trabucco, Gunnar Sigurdsson, Robinson Piramuthu, and Ruslan Salakhutdinov, Towards intemet-scale training for agents, 2025a. URL hhtps: htggaa/a///022222 06776
Brandon Trabucco, Gunnar Sigurdsson, Robinson Piramuthu, and Ruslan Salakhutdinov. Towards internet-scale training for agents. arXiv preprint arXivv2502.06776, 2025b. 3
Guanzhi Wang, Yuqi Xie, Yunfan Jiang, Ajay Mandlekar, Chaowei Xiao, Yuke Zhu, Linxi Fan, and Anima Anandkumar. Voyager: An open-ended embodied agent with large language models, 2023.
URL pettps://arxive
abs/2305.... 16291
Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc Le, Ed Chi, Sharan Narang, Aakanksha Chowdh- cry, and Denny Zhou. Self-consistency improves chain of thought reasoning in language models. arXiv preprint arXiv:2203.111770 2022.
Zora Zhiruo Wang, Jiayuan Mao, Daniel Fried, and Graham Neubig. Agent workflow memory, 2024.
URL htttpt:::///xxx
abs 07429
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. of-nnn prompting elicits reasoning in large language models. Advances in neural information processing systems, 45:24824-24837 2022. 3
Sean Welleck, Amanda Bertsch, Matthew Finlayson, Hailey Schoelkopf, Alex Xie, Graham Neubig, Ilia Kulikov, and Zaid Harchaoui. From decoding to meta-generaaion: Inference-time algorithms for large language models, 2024.
16838
Yilin Wu, Ran Tian, Gokul Swamy, and Andrea Bajcsy. From foresight to forethought: Vlm-in- tho-loop policy steering via latent alignment, 2025. URL http:/:://xx....//////a//// 01828
Tianbao Xic. Danyang Zhang, Jixuan Chen, Xiaochuan Li. Siheng Zhao, Ruisheng Cao, Toh J Hua, Zhoujun Cheng, Dongchan Shin, Fangyu Lei, et al. Osworld: Benchmarking multimodal agents for open-ended tasks in real computer environments. Advances in Neural Information Processing
Systems, 37:52040-52000 2024. 5
Wenda Xu, Guanglei Zhu, Xuandong Zhao, Lianggmngg Pan, Lei Li, and William Wang. Pride and prejudice: Lim amplifies self-bias in self-refinement, In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume I: Long Papers), pages 15474-15492, 2024.
5 6 g
Jianwei Yang. Hao Zhang, Feng Li, Xueyan Zou, Chunyuan Li, and Jianfeng Gao. Set-of-mark prompting unleashes extraordinary visual grounding in gpt-4v, 2023. URL htttps://arxiv
org arg/abs/240 abw.aba............ 11441
16
Ke Yang, Yao Liu, Sapana Chaudhary, Rasool Fakoor, Pratik Chaudhari, George Karypis, and Huzefa Rangwala. Agentoccam: A simple yet strong baseline for llm-based web agents, 2025. URL
htttps///ariis www.avss..gg////o/o 241 414. 0 13825 3
Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Tom Griffiths, Yuan Cao, and Karthik Narasimhan. Tree of thoughts: Deliberate problem solving with large language models, Advances in neural information processing systems, 36:1 1809-11822 2023a.
14

---

Published as a conference paper at ICLR 2026
Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. React: Synergizing reasoning and acting in language models. In International Conference on Learning Representations (ICLR), 2023b.
Jiayi Ye, Yanbo Wang, Yue Huang. Dongping Chen, Qihui Zhang, Numo Moniz, Tian Gao, Werner Geyer, Chao Huang, Pin-Yu Yu Chen, Nitesh V Chawla, and Xiangliang Zhang, Justice or prejudice? quantifying biases in llm-as-a-judge, 2024. URL /tt/:tttss//aar...rr///a/////111
Xiao Yu, Baolin Peng, Vineeth Vajipey, Hao Cheng, Michel Galley, Jianfeng Gao, and Zhou Yu. Exact: Teaching ai agents explore with reflective-mets and exploratory learning. arXiv preprint
arXiv:2410.02052, 2024. 3 10 16
Xiao Yu, Baolin Peng, Vineeth Vajipey, Hao Cheng, Michel Galley, Jianfeng Gao, and Zhou Yu. ExACT: Teaching Al agents to explore with reflectiveMMCTS and exploratory learning. In The Thirteenth International Conference on Learning Representations, 2025. URL https
//openreview.oo /ttp:/iewwweeeeeeeeeeeeeeeeeeeee wttp:eeeeooommmmmddtlttoollool
Michal Zawalski, William Chen, Karl Pertsch, Oier Mees, Chelsea Finn, and Sergey Levine. Robotic control via embodied chain-of-thought reasoning. arXiv preprint arXiv:2407.08693, 2024.
Zhuosheng Zhang, Aston Zhang, Mu Li, Hai Zhao, George Karypis, and Alex Smola. Multimodal chain-of-thought reasoning in language models. arXiv preprint arXiv:2302.00092,, 2023, 4
Lianmin Zheng, Wei-Lin Chiang, Ying Sheng. Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric P. Xing, Hao Zhang, Joseph E. Gonzalez, and Ion Stoica. Judging aajudge with mt-bench and chatbot arena, 2023. URL htttps://arxiv.orrt abs/ abbs 65685
Andy Zhou, Kai Yan, Michal Shlapentokh-Rothann Rothman, Haohan Wang, and Yu-Xiong Wang. Language agent tree search unifies reasoning acting and planning in language models, 2024. URL https:
abs wwtpv//orggabb////a/aoo
Shuyan Zhou, Frank F Xu, Hao Zhu, Xuhui Zhou, Robert Lo, Abishek Sridhar, Xianyi Cheng, Tianyue Ou, Yonatan Bisk, Daniel Fried, et al. Webarena: A realistie web environment for building autonomous agents. arXiv preprini arXiv: 13854, 2023. 3
A EXPERIMENTAL DETAILS AND PRACTICAL IMPLEMENTATION
A.1 OFFLINE EVALUATION OF AGENT TRAJECTORIES
For the experiments reported in Tab. we employ each model with its author-reoommended inference parameters. For the ablations in Tab. 2 and Sec. E we employ gemini- 2. 5- flash with zero temperature, except for the majority voting baselines, where we select the most frequent response among 8 completions for each trajectory generated with temperature of 1.5, top-p of 0.95, and top-k set to 64.
For non-thinking models, all settings include a CoT instruction (see prompts in Sec. B For thinking models we set thinking budgets to the maximum value for each model and omit CoT instructions, as they inherently produce reasoning traces and adding them can degrade performance OpenAl (025))
Models are given interleaved images and text, in the form of screenshots produced by the environment and actions generated by the agent after they are parsed by the environment. In the case ID-based actions, we overlay bounding boxes the screenshots to indicate the location of each interactable element. In the case of coordinate-baset actions, we overlay red markers to indicate the target of mouse actions. For 1.5-7,, additionally translate agent outputs to english using gemini-2 ..0 0 as the agent sometimes produces answers in chinese. Sec. G show examples of trajectories, and Sec. B contains details about prompt templates.
15

---

Published as a conference paper at ICLR 2026
A.2 DOWNSTREAM APPLICATIONS
5-flash as the MLLM to build the verifiers, with temperature set to 0, thinking disabled, and token limit of 8192. The prompt templates and inputs given to the verifier are the same as the ones in the offline scenario described in Sec. B
For our results in downstream applications, we employ
The numbers reported are averages across three runs in the digital benchmarks, and 50 rollouts with horizon of 700 time steps in robomimic.
Reflexion Implementation Details. After each episode for a given task, the same agent that generates the trajectory is prompted to reflect on performance, using the verifier's score as part of the context. This reflection is saved to external memory, that is appended to the agent's 's context window in subsequent episodes. The prompt is almost the same as the agent's prompt, and is provided in Sec. B.6
Online Supervision Verifier and Agent Implementation Details.
gemini -2 5..5-ffaass for both the ReAct agent and the verifier. For the no-SGV case, the model is given a CoT instruction as in Sec. B The environment settings follow those in Koh et (2024a): (2023) for the screenshots, with a viewport size of 1280x2048 pixels. For consistency with prior work, we use BLIP-2 Li al. (2023) to provide objectives and environment observations (though we observe little to no difference in performance when captions are omitted). We set the maximum number of steps 30, which allows the agent some room for backtracking while keeping the search budget comparable to -houghh still more constrained than-prior approaches that employ tree-search methods Koh al. (2024b): Yu et al. (2024). For the World experiments, we follow the same setup and implementation as in Qin et al. (2025) We on single RTX 3090 GPU.
For VisualWebAren experiments in Sec. 5
adopt Set-of-Mark prompting Yang et al.
captions for images making part of the task
allow up to 50 steps for the agent and run UI-TARS-1.55
For Wes.Area the MLLM acts as an outcome verifier, intervening when the agent issues stop action. If deems the task as incomplete, it generates natural language feedback that appended to the agent's ' context window to influence subsequent generations.
For OSWlll we experiment with two configurations: one where the MI.LM intervenes only after stop actions, and another where it verifies progress every steps (shown in main results). The latter configuration was inspired by preliminary experiments showing that UI-Tars- can derail from the task objective and rapidly reach points difficult to recover from. As shown in Tab. SGV improves performance both cases, although results are slightly better in the latter.
In robomimic, verifier monitors task progression and triggers the diffusion policy to regenerate an action sequence (or "replan") at determined steps. In the oracle scenario, the steps are determined by ground-truth subtask completions. In the MILM verifier scenario, the MILMM receives a natural language description of the task, the first and last screenshots, and is called every 20 time steps. If it deems the trajectory off-track, a replan is triggered.
Sec. B.4 shows how the feedback is incorporated, and figures in Sec. G provide examples of evaluations and generated feedback.
A.3 PRACTICAL IMPLEMENTATION
SGV introduces low compute overhead and generally improves performance without complex prompt engineering, making it simple to integrate into pipelines where MLLMs play evaluative roles. For instance, adding the first step with a single-line instruction to produce domain knowledge generally helps, with little to change to existing prompts (see Sec. E.7). Nonetheless, observe the following patterns that may inform general use:
1) Generating broader,
encompassing priors in the first step tends be superior.
2) Producing priors in a single, monolithic step degrades performance, as models tend to anchor them to the data under evaluation.
16

---

Published as a conference paper at ICLR 2026
3) Some models benefit when the second step includes instruction to compare against their priors- something SGV inherently enables although a standard CoT instruction typically suffices. See Sec. B for prompts and our codebase for more examples.
We also refer readers to Sec. which provides further guidance on MLLM verifier design, including analyses of the performance of Likert, numeric, and other prompt templates and interventions; the effects noise, model scale, and diversity in SGV prior generation; the potential for calibrating MLLM output distributions account for the skew introduced by agreement bias; comparisons between periodic and outcome-based verification; and the impact of trajectory length and verification difficulty MILM verification performance.
B PROMPTS
For all prompts, content enclosed in curly braces ((...)) )) is dynamically replaced at runtime with the corresponding information. Text enclosed in double angle brackets (s... provides explanatory notes for the reader and is not included in the prompts at runtime. The labels USER: and ASSISTANT: indicate role fields passed via the OpenAl API format and are not literal strings in the prompt. Text separated by pipe (1) inside curly braces ((...) indicates phrases that change depending on the context.
B. COMMON PROMPTS
The following three prompt templates common across all verifier and reflexion prompts, with slight variations to accommodate specific details such as image annotation mode and environment specifics.
Trajectory Prompt- VisualWebArena and OSWorld
JOER:
## OBJECTIVE: [cask oojective text) (task objeetive imagea]
JBER: #+f STATE "t-d" SCREENSHO": [screenahot]
ASSISTANT: igeneraled action;
USER:
SCREENSHOT: [screenshot]
ASSTSTANT: (generated action)
Image Information Prompts- VisualWebArena
<<if set-of-mork prompting la used (bagel ine) >>
Webpage screensho:s: These aze screenshots the webpage, with each interactable element assigned unigue aumerical id. Bach bounding box and its respective id shares the same color.
<cif set-cf-marks and coordinate annotatiens>> webpage scrsenshorn: ement assigned a unique numerical id. Rach bounding ng box and its reapect ive id atares the same color, The colored narkers, if present, indicate the destination of actions taken the corresponding state.
screenshats
wabpage,
interactable
ccif coordinates
#t webpage scrsenshors: these are screenshots of the webpage taken at each step of the the instination actions
navigation. Tha colored markers, present, at the correeponding state.
<cif visuel annotacions>>
# Wabpaga scrsenshors: These are sersenshota of tha vabpage taken at each step of the navigation.
17

---

Published as a conference paper at ICLR 2026
Image Information Prompts OSWorld
<cif set-cf-mark prempting is used>>
44 Screenshole:: These are vieual captuzes of the computer screen Laken at specific of the
navigation procass, with each interactable alemant assignad a unique numerical id, Each coler.
bounding box and its respective ahares
set-ef and coordinates anaotations>>
#t# Screenshots:: These are visual captures of the computer sereen taken at specific states of the
navigation process, with each interactable element assigned unique numerical id, sach hounding box and its respective shares the same coler. The colored morkers, if actions
present, indicate destination
corresponding state.
<cif coordinates
only (baseline
#t# Screenshot.s: These are visual captures of the computer sersen taken at specific states of the novigation process. The colored markers, if present, indicate the destination of actiona taken at the corraspending state. <sif coordi nates are use6s>
<<if no visuel annotacions>> screenshots:: states of the
visual captures computer screen specific navigation process.
B.2 VERIFIER PROMPTS
To maximize consistency and minimize sensitivity due to prompt variations, we construct all verifier prompts from the following parts:
The system prompt, which describes the role of the agent and environment details.
The trajectory prompt, which created in an interleaved user-assistant conversation format, and contains information on the task to be accomplished. screenshots of the environment states, and the actions taken by the agent being evaluated.
The evaluation prompt, which includes (i) the criteria for the evaluation, and (ii) instructions for step-by-step reasoning. Check Secs, B.9 and E.2 for ablations to the scoring templates. Ablations (ii) are indicated in the prompts below, and discussed in the main text (Sec. 4 and Secs. B.8 and
We aim to keep the prompts consistent possible across experiments, applying slight adjustments to accommodate environment-specific details and prompting variations. For instance, the NoCoT variant in section 4 is nearly identical to the CoT variant, differing only in the removal of the st-p-p reasoning instructions.
Verifier System Prompt- WebAreWe
You are an intelligen: agent Lasked with supervising an assistant navigating web browser accemplish web-based task. Your jeb is evaluate the assistant 's work, and provida feedback so it can progresa towards the abjective.
#t Here's the informasion you'll have: The objective: This the taak the assistant trying to corplete. immge__ito_prrmpt...
4+ The execution trace: This a sequence of webpage screenshots paired with the assistant's 's actions, detailing the veb navigation far. #tø Ceneral web knowledge: This is general description of how tasks like this are typically accomplished on
esOnly included 8EV variants>>
44 Assistant's capabilities: To effectively analyze the assistant 's work, consider the actiens it can perferm. These actions fall into the follewing categories:
Page Operation Accions:
"**alick [fd]"";; Click on an slement with a apect ric id on the wenpage. LL"Lype [id] [content] (enter_actes)""' Type the conteat enter_after is 3, the "Enter" key is not pressed afser typing: otheruise, the "Enter" key is omat pressed.
the field with id.
18

---

Published as a conference paper at ICLR 2026
element with id. .."press [key_somb]"*> "*"; Preas a keyboard key or key combination (e.g., deleta, ctrl+a).
" bover [id]"";
''scrrll [down] **. ""scroll [up]"""; Scroll the webpage
Tab Management Acstons:
browser Lab. "ocuabfoocus [tab_index]" Switch the browser's focus to specific tab using its index.
"""ew_Labb"" Open
""close_tab""" close the currently active tab.
*+# URL Navigation Accions:
gott [uri] Revigate
specific
** ""gooback"" " Navigate to the previously viewed page.
"orward"' Bevigate to the next page (if a previous 'go_back' action was parformed).
Completion Action:
"stop [answer]** This actior is issued when the task is believed to be complete. If task requires sext-based answes, answer provided within brackets. the is deemed impossible, this action is issued optionslly with
why.
44 To be successlul, it is very important to consider the following: not
correct incerrect beferehand. You should ceme with your own opinion based on the information previded. must connect
the assistant's
tha information provided. Your evalustion should be based task completien; don't worzy about efficiency for now.
between cajective
visual information provided, eapecially the sczeenshots in the execution Liace.
Give importance
knowledge contain noise information
your
specific context. Therefore, should selectively use it to guide your analysia and reasoning, making aure to Axconalder context of spect task navigation given
<<Only included for sav variants>>
Verifier System Prompt- OSWorld
are intelligen: agent
supervising
utilizing computer
to accomplish computer- based tasks. Your job is to evaluate the essistant's 's work, and provide feedback
can progress
the objessive.
44 Beze's ' the informat fon you'll have:
44 The objective: This is the task the assistant tzying to complete. imm.g__inoo_propt..pe
execution Lrace: This paired with the assis:ent's responses, detsiling the compuzer navigatien far. Computer Knexledge: are typlcal accorpl pecorplished ssed computer. <<Only included for SGV varlants>>
sequence screenshots
computer screen
is general description
# Assistant's capabilities: To effectively analyze the assistant's work, consider the actiens it can perform. These actions fall into the follewing categories:
Mouse Actiona: boo_att__ooo""/ox__xeaa_e (xl, end Left clicks at the (xlyyl)
coordinates.
lett_double tote
< [box__ndd]>" left double click at the
x-,yl) coordinates.
right_singleessart__ue (aaigglletaarttbbo""leeo.... "< (11, box__end Right click the (xl,y1) coordinatad.
crag(etarr_bbxx*1[l (taaar___bboe<ooo_aaarrraa
( _ooox___dd>', </ ps (x3,
box_ box_end(>>)>= prags the elament at the (xl,y)) coordinates to the (x3,y3) coordinates.
serall satart_boxee "xxx'<//bxx_aaaa (x1,y" <laox_end|>", deractions onw up right :eft'!: Scrolls the mouse wheel the specified direction.
Keyboard Actions:
hotkey Ikay=! Press specific key. -ype (content=" Types string following "content". If Lowlowed by "\n",
the input i5 aubmitted.
wait (): Pauses for 5s and takes a new screenshot to checx for any changes.
** Completion Actien:
finished (contents 1 <|joontntt [>'); This action issued when the task is believed to be complete. "ccontent"" la opt onal, and is uaed to provide resson why the task is complete.
To be successful, evvery importantra to follow the following rules:
19

---

Published as a conference paper at ICLR 2026
the assistant's work should come up with your own opinion based on the informotion previded. ov the information provided. 3. Give utmost impoztance to the visus= information provided, especially the screenahots the execution trace.
incerrect beferehsnd. You
Your evalustion should be based Lask completion: don't now.
about efficiency for
information rolevant your apect fic context, Therefore, you shou d se selectively use 15 to guide your ana lysis and reasoning, making
The General compute: knewledge
of specifie task compater navigstion given to youlely included for 83V varients>>
*xconaider
Full Verifier Prompt
prompt]
(trajectory prompt)
JSER:
General web knowledge: loutput from first step1 <<Only included for s3V veriants>>
New please provide your respense.
# Bere is the avalussion criteriasscThis section is changed in ablations to evaluation templates, e.g., numesical scoring2>
SUCCESS: The assistan: executed Avall ofsa what's necessary to complete the objective, The task is fully accomplished.
ofıe what's necassary completa the ovjective. The task la partially poompplsshed.
success:
assistant executed
is TAILURE: assistan; executed 4-moslly incorrect.. aleps. accomp ishad, and majo" reviaions are needed.
# Provide your response follows:
REASONING: [Comprehensive step-by-step reasering to come up with your evaluation and faedback] <<Only included for CeT variarts>>
EVALUATION: [Your evaluation following the evaluation criteria] [Feedback
FBEDDACK:
assistant car progress
objective]
Pan et al. (2024a) Verifier System Prompt
You are an expert in eval avaluat'me nnng the performarce of a web navigation agert. The agent is designed help human user navigate website complete task. intent, the agent's 's action history, the final state of the webpage, and the agent's 's responae to the user, your goal is to decide whether the agent'a execution is successful ao:.
user's
There three types tasks:
1. Information seeking: The user wants to obtain certain information from the webpage, information
product, reviews, map info, comparison map routes, etc. "he oot reaponse must contain the information the user wants, explicitly state that the informacion is not available, Otherwise, e.g. the bet encouaters an exception and respond with the careful about the sufficiency of the agent's actions. For example, when asked to list top-searched itema in shop, searchee, and then recurn the top items. If the ordering action is missing, the task is likely
such
the task considered failure. Besides,
agent should order the itema the
of
2. site navigations The user wants to navigate to specific the bot's action history and the final state of the webpage to determine whether the bot successfully completes the zask. need consider the bat's response.
Carefully examine
3. content modification: The user wants to modify tha content of a wcbpage or configuration. Carefully examine
act on hiszory and final state the webpage to determine whethe: the bot succesafully completes the Lask, No need to corsider the hot reaponse.
Format your response
lines shown below:
Thoughtes eyour shoughts reasoning
Status: "success", "pactial success", or "failure"
20

---

computera). Published as a conference paper at ICLR 2026
Full Pan al. (2024a) Verifier Prompt
(system prompt]
JS2R:
# User Intent: [tack objactive text)
44 Action History: (generated accion)
n: (generated action)
# Last anapshot of the wabpage: (screenshot]
B.3 SGV VERIFIER FIRST STEP PROMPTS
For the main experiments, we use separate prompts for the retrieval step and the evaluation step. Below. we present the prompts for the first step. As shown in Sec. [.8] Sec.B.8 similar performance can
be achieved by reusing the prompts from Sec. [.2] as long the verification process is explicitly divided into retrieval and evaluation phases.
SGV First Step System Prompt
You are a helpful assistant with deep knowledge in [web novigation compater-based workflows).
Your job is to provide description of how Lasks like the ones provided to you are typfically ly accomp Hshed on [the
# Hler'' the informazion you'll have: #tø oojective: This is english ceseription of the task, possibly accompanied by images that provide more context on what must ba accompl fahed.
(webpage computes screen), giving you context to further understand what must be acccaplished.
Screenshots:
SGV Full First Step Prompt
(aystem prompt firat-stepl
#4 овJтCттVE: Itask objectiive text
#4 TMAGFS:
Inage 0: (Lask objective image 1F
image nr (task objective image al
Tmage n+1: initial (webpage computer) screenshot: (screanshot)
please drovide your respense:
<Description of how tasks auch this are typically accomplished on (the web computers]>.
B.4 VISUALWEBARENA AGENT PROMPT
The prompt for Vibual built upon the authors' original implementation Koh al. 2024a with the following changes:
We provide the history of thoughts and actions generated by the agent in previous interactions, which serve as text-based memory to prevent loops and allow backtracking.
We refine parts of the prompt to use Markdown formatting and make the instructions more precise.
For one of the examples, we include previous thoughts and actions to match the new inputs.
21

---

Published as a conference paper at ICLR 2026
WisualWebAreen Agent Prompt
(system prompt]
(exemples]
# RATIONAGE AND ACTION HISTERY:
"t-J":
.*RATIONALB++: lagent generalions)
: [acticn parsed by the envircnment
STATE "t-1":
**ATIINNALEv:::
generations]
++ACTION++: laction parsed by the environment]
leee is the current state "L" for you to determine what to do next: "t': [text observation)
* URL: (webpage URL:
# ORJRCTTVV: Itask object ive text.
** PEEDBACK: Here is previous response at the current state "t" and v*fesdbackrA use this sevise your respense if you appropriate.
#$Ø Previcus response: (last generated actior)
#*# seedback: (verifier feedback) <<Unly included in rounds where feedback is provided by verifierx>
IXAGES:
Image 0: (task abjective image 0]
Tnage Itask objective image
Inage nil: curzent webpage screenshot at state "L"s (screenehot
B.5 OSWORLD AGENT PROMPT
OSWorld Agent Prompt
You aze a GUI agent. You are given Lask and your accica history, with screenshets. action complete task.
to perform
#4 Output Format
Thought: Actions ...
# Action Space
click settatt_oo=""<lbxxxcoo (x1,y]) box_end|>>)
(C,, <1)x//ox__edlll
Ligght_singlestttet (strrleesttrr_boeereeo ob /box__/stooooooosttttattall yilyo_oobx_eedd
drag (start_ (boxsst=(oo bax_end[>>))
box__ndd enx''s box_start bbbbbooooooossttttteeeeeaa |> (33
hotkey (key="))
Lype (content="") #IC you want submit your laput, use "la" at the end of "content".
scooll (bbox__tattt (1) left!)
disection " up right
()it() #Sleep
screenshot check any changes.
firished Icontent= =xxxx" use escape charasters V', \", and An in content part ensure we can parse
normal pythen string formot.
+ Note
Use English in 'Thought" part. plan and finally summarize your next action (with target element) in one sentence in "Thought" part.
44 User Instruction (task objective zext:
22

---

Published as a conference paper at ICLR 2026
B.6
REFLEXION PROMPT
Reflexion System Prompt
You aze autenemous intelligent agent that was tasked with navigating web brewser Lo complete web based taeka, is to anslyze your behavior
Your goal the strategy and path you tock when tzying to complete the task, understand the reason you (failed) aseceded) and devise veconcisev reflect that be helpful whan you aze solving the same task in the future.
previous task attempt. You must anslyze
it Here's the information yo have:
44 The objective: This is the task you were trying to complete.
tté The execution trace: This is sequence of webpage screenshots paired with your responses, detailing the web navigation from the beginning to the and of the pravious task attempt.
44 Previcus reflections: This is a 1ist of previous reflections you've made about failed executions
#te wabpage scrsenshors: These are screenshota of the webpage, with each interactable element assigned a unique aumerical id. Each bounding box and its respective id shares the color.
# To analyze your past task attempts, consider the actions that were available to you. follewing categories:
actions
#t# Page Oparation Actions:
""ekick [id]"*; This action clicks on an element with a specific on the webpage. eypp [id] [content] [enter_after] "*: Use this type the content into the fiald whth id. Tf "encer_aftar" "a 0, the "Enter" key la not preased after typing; otherwise, "Bater" key automatically pressed.
""oove [id]"*;; Hover over an element with id.
*ppress [key_comb] [text] "* Fress keybeard key key combination le.g., delate, strl+al with optional text input if the key conbiration requiras it (e.g., "'presa strl+f) [some_toxt]
"'coroll [down] ... "sscroll [up] seroll the webpage down
Tab Nanagement
""ee_tab""' Upen new, empty browser tab. ""taa_focuu [tab_index]""' Switch the brewser's focus specific tab using index.
""close_lab"". Close the currently active tab.
44# URL Navigation Acnio
"''got [uri]"';: Navigate
specific
""go_back"", Navigate to the previously vieved page.
goo_ "oofforsard"""" Navigate the next page previous 'go_back" action was gerformed).
Completion Action:
"ssop [answer]*"* Issue this action if you belleve the Lask is complete or infeasible.
objective is bracket. If you deam the task is infeasible, provide reason way.
text-based answer, provide the ansver
44 To be successful, it is evvery importent*. to consider the follewing: carefully analyze the execution Lrace 2. You must coanect the dats between the objective and the information provided. execution trace you should not to continue the taak nor propose actions continuing from the previous taak atrerpt.
up your reapoase,
previous task attemps
#*finishedes. Therefore,
and general sspects. Your reflections should be applicable to future attempts to this task that will be viindependent. from previous task attempts and will start
4. Prieritize
the beginning.
5. Try to to think differeatiy from previous task attempts and reflections. Integrate previous reflections
informaticn
can perzinent future attempts while chinking of possible new strateg that can lead succesaful completion of this Laek.
Reflexion Full Prompt
(aystem prompt [Lza prempt)
#t Task Status: merrffer evaluation on of the task att (e.g., PaTlUrF) Previous Reflections: (reflectiona from prior executiona of the task]
23

---

Published as a conference paper at ICLR 2026
# New please provida your responge as folloxs:
POSSTBLE REASONS (status): [9hat the posaible reasons why object
achievedl achieved) <<lete: we observe agreement bias can lead to blind agreement thth prior execu: fons
affect reflexion generation. Similar to 55V, this section helps mitigate this lasue.
REFLECTION: [Reflections for future task attempts]
B.7
ROBOMIMIC PROMPTS
robomimic SGV Prompt- First Step
are highly skilled in robotis tasks. Based on the task description and the image of the initial state, predict how the task must look 1ike at the current time stamp.
Task Descriptien:
Pick 1-shaped pencil and insert L-shaped pencil into the hole Pick tool and hang tool on & s-shaped peacil
Current time stamp: (n/N) (?mage)
robomimic SGV Prompt Second Step
Compaze prediction with actual depicted in the image.
system
Lime step
If the current state shows situation that depicts fai lure, label it as failure.
If both tasks being completed nearing completion predicted and both tasks have been completed, label as success.
If the prediction is that the task will be in progress and the zesk is in progress, it as progress.
[Cirst step generation! Current time stamps (n/N)
(image)
robomimic - Verifier Prompt, no SGV
are highly skilled
verification.
Based on the task description, the cusrent time step and the the actual state of the in the cursent time step (depicted in the image), label it as 'success', failuze! or 'in progress".
Given the time scep, if the current state is net as expectarion (with dropped tools ), label it as failure.
If the time step is Lowards completion, end the task has been complesed, label it as
Given time scep, task expected progress progress, label it as in progress.
Current Lime stamps (o/N) (image)
24

---

Published as a conference paper at ICLR 2026
B.8 ABLATION PROMPTS- SGV
As we show in Sec. B.8 similar performance for SGV can be achieved by re-utilizing the verifier prompts from Sec. B. as long as prior generation and verification are divided into two phases. Below is the prompt used in the first step. The second step prompts are identical to those in Sec. B.2
SGV Unified Full First Step Prompt
(verifier system prompt)) <<same as in Appendix B,25>
* OBJECTIVE: [task objective text.)
# IMAGES:
image 0: Itask objective image 01
Inage nt Itask objective Image nl Inage initial (webpage computer; sczeenshot: (screenshot)
Please provide the follewing
[nesoription on of how tasks auch as this are typicallly accompl ished on (the web computers]].
The following prompts are used when prior generation and verification are performed together in single, monolithic step. The system prompts are the same as in Sec. B.2 except for the rules, which are replaced by the following.
Monolithic Retrieve and Verify System Prompt Rules
# To be successtul, it is very important follow the fol Lowlewnng rulea:
1. You must not not nssume the assistent s work is correct or incerrest beferehand. You should
with your opinion based the information previded. 2. You must connect the dots between the ob jective and the information provided. evaluation now
completion:
worzy
efficiency
information provided, aspecially the screenahots in the execution trace.
Give utmost importance to
5. You should come up with datailed description of how tasks like this are typically computers).
accorpl ished (the
6. Use this General (Web Knowledge Computer Knowledge) as guide, but also consider context specific task given you.
Monolithic Retrieve and Verify Full Prompt
(system prompt)
(trtooy prompt)
please provide your response.
44 Here the evaluaston ersteria:
SUCCESS: The assistan: executed evall of+. what's necessary to complete the objective, The task is fully accomplished.
assistant jbjettive. The task is partially accomplished.
neceasary complete the FAILURE: assistans executed smootty incorrect.. steps. accomp ishad, and major reviaions are neaded.
FARTIAL suCCESS:
44 Provide your response follows:
GENERAL (WEB COmPтиR) KNOKLEUGE: <Jescription of how tasks such as this are typically accomplished on
computer)>
REASCNING: scomprehensive step-by-step reasering to come with your evaluation and faodback> <<only included CoT variarts>s
EVaLUATtoN: sYour evaluation following the evaluation eriteriaz
rEEDDACK; <Feedback so the assislant ean progress Lowards the objectives
25

---

Published as a conference paper at ICLR 2026
B.9 ABLATION PROMPTS- EVALUATION CRITERIA
The following list some examples scoring templates used in our experiments for ablations in Sec. E.2 For all the 28+ templates, please refer to codebase. The remainder of the prompts the same as in Sec. B.2] except for the evaluation criteria section, which is replaced by the corresponding templates below.
Scoring Template Examples
<clikert Ternary Scale (baseline)>> SUCCPSS:
The assistan: executed +*all of+* what'a necessary to corplete the objactiva. is fully accomplished.
PARTIAL SUCCESS: The assistant execured vamest of** what's necessary to complete the abjective,
partially accomplished. <<removed if using binary scales> FATLURE: The assistan; executed Avmost incorract steps. The task is not accomplished, and major revisions are needed.
<sscore-basen Scale, tour-ways> Provide single sccre, rating based on the confidence of its correctness for the task.
work sirictly scale to 10
Score to 2: You are very confident that the asaistant's work is comp etely incorrect for the task. Score
fairly conf'cent that the asaistant's work incorrect, but there is a small chance it could be partially correct.
assistent's work; could are fairly confident that the assistant's work correct, 1s small chance it could be partially incorrect. Score to 10: You are very confident that the assistant's work is completely correct
Score
are be right wrong.
the correctress
Score
csScore-based Seale, shree-way>>
Provide a single secre, rating the assistant's work strictly on scale of to 10 based or the confidence of its correctness the taak.
Score to 4: You are confident that the assistant's work is incorrect, but there is snall
pactially correct.
Score 5 to 5: You are uncertain about the cosrectness of the assistant's work; it could right wrang.
Score 7 to 10: You are confident that the assistant's work is completely correct for the task.
c<Renge-based Scale, implicit>>
sirictly scale to 10 based on the confidence of its correctness for the task. szore can any scalar A ++loM score** means incorsect for the Lask,
Provide a single score, rating assistant
and 10, where: are corfidert that the assistant's work is completely
å +siigh score.* correct for the task.
you very confident that the assistant's 's work is completely
you urcertain about the corractness the assistant' work; it could be right or wrong.
ssmid-range
schange-based Scale, explicit>>
Provide a single score, rating the assistant's wozk s:rictly on scale of to 10 based on the cenfidence of its correctness for the task.
The score can ba any scalor between 1 and 10, where:
very confident completely incerrect for the task. score
you
the assistant's work
means you are very confidant that the assistant's work completely correct for the task.
A score close zo 5 means you are uncertain about the correctness of the assistant's work; it could be right wrong.
ecTrue/False Stacemen:s>>
The fol following are atasements about the assistent's work. If statement is corract, p. eaees reply with T, otherwise reply with F.
A. The assistent executed mall of** what's recessary to complete the objestive. The task fully ascomplished.
B. The asslatant exeauted *사oоs ofva what'a necessary to corplete the objective. The partislly accompliahed.
c. The assiatant executed +sprrt of+* what's necessary to complete the object iva. The
task ls partially accompllahed.
26

---

represeata your Published as a conference paper at ICLR 2026
The assistent executed Ammeslly inzorresta* steps. The task is accomplished. and major revigions are needad.
executed what's necessary complete jeettivve The Lask can be fully accorplished, partially accomplished, or not accomplisaed;
list of letters for each statement, as follows: I or F, I or E, I or F, oz E, or
Provide your response
<coptior-bsaed, roman, Five-way>> Please evaluate level of confidence. what describes your confidence
aasistant's work by selecting option
assistant's work?
I. You are very confident that the assistant's 's work is completely incorrect for the task.
You fairly confident that assistant's work is incorrect, but there snall chance it could be partially correct.
assistant werk; could right corract, but is v. You are very confident that the assistant's 's work is completaly correct for the task.
uncertain about the correclness
oF wzong.
You are fairly confident
assistant's
chance could be partislly incorrest.
C QUALITATIVE ANALYSIS
C.I. AGREEMENT BIAS
Fig. iillustrates the trajectory for a failed task in VisualWebArena, along with evaluations produced by three verifier variants. Two key flaws are evident in the trajectory: (i) the agent does not perform any price comparisons, and (ii) stops after adding an item the cart, without proceeding to checkout. The CoT verifiers validate the execution, providing reasoning traces that justify it as correct despite the evident shortcomings. In contrast, the SGV verifier flags the absence of price comparison, and accurately declares the execution unsuccessful.
C.2 ONLINE VERIFICATION AND FEEDBACK
As discussed in section 5, our method enables MLLM verifier monitor agent behavior and provide natural language feedback real time, guiding the agent toward successful task completion. We show there that this mechanism has positive overall impact performance, resulting in a 5 percentage point increase success rate on Aebuaaaa (approximately 10% relative improvement) and percentage point gain in OSWorld (approximately 9% relative), compared to no-verifier baseline.
The success of such mechanism relies on three factors: (i) the accuracy of the verifier's judgment, (ii) the quality of its feedback, and (iii) the ability of the agent to interpret and act upon that feedback, including the capacity to revise or backtrack when necessary.
In this section, we explore these factors through representative cases that offer insight into both the strengths and failure modes of our method and of MLLM verifiers more broadly. The cases are as follows:
Accurate verification with helpful feedback: The verifier correctly identifies flaws and provides actionable feedback that improves task execution.
Accurate verification with helpful feedback, limited by agent ability: The verifier offers correct judgments and useful feedback, but the agent fails to act on it effectively.
Strict verification, harmless: False negatives from the verifier that do not affect the agent's final outcome and often elicit more robust behavior.
Striet verification, harmful: False negatives that lead to degraded task performance or failed executions.
False positives where the verifier fails detect clear mistakes, and contributes no useful signal. This stems from agreement bias and is the most prominent failure case for conventional MILM verifiers.
Overly lenient verification:
27

---

Published as a conference paper at ICLR 2026
Accurate Verification with Helpful Feedback Fig. 13 shows example where the verifier success- fully steers the ReAct agent to a correct execution on a challenging VisualWebArena task. The agent performs sequence of reasonable steps and finds product closely aligned with the user's request, but not the cheapest one. The verifier detects this oversight and provides feedback that prompts the agent backtrack, sort the results correctly, and select the appropriate item.
This is a particularly challenging case that is highly susceptible to agreement bias, as the initial trajectory comprises reasonable steps and ends with a product that closely aligns with the user's query.
Accurate Verification with Helpful Feedback, Limited by Agent Ability Fig. 14 presents failure case where, despite the verifier correctly identifying the issue and providing appropriate feedback, the agent unable to recover. The verifier advises the agent to calculate shipping costs before leaving a review. However, the agent instead performs a sequence of poor actions, from which it never recovers. This example demonstrates central limitation of our agent: the performance is ultimately constrained by the capabilities of the generator, regardless of the quality of the verifier.
Overly Strict Verification, Harmless Fig. T5 illustrates this dynamic in a borderline case. The agent clicks on the first visible option without searching. Although this behavior is marked correct by automated scripts, the verifier rejects it, requiring the agent to explicitly search for the product. After performing the search, the agent provides the same (correct) response, but is again rejected for not clicking on the item to inspect its details. In this task, that step is arguably unnecessary, as the required information is already visible. Still, the agent proceeds click and confirm the details, ultimately converging on the same answer a third time.
This example reveals two insights: (i) Even though the verifier is demanding, its feedback aligns well with the user's original intent, and does not prevent task completion. (ii) In fact, the strict verifier elicits robust behavior from the agent. The benchmark task is relatively simple and susceptible shortcuts, but the verifier enforces a robust and generalizable strategy.
Overly Striet Verification, Harmful Fig. presents an example where a correct initial execution is degraded into a failure. The user's s query accepts either "Ohio" or "Pennsylvania" as valid answers, but the verifier demands the cheapest option across both. The agent, however, is unable to navigate back to Ohio to perform valid comparison and runs out of steps. This example also illustrates the difficulty of interpreting real-world user intents, which are often vague and under-speciffed.
Overly Lenient Verification Our verifier remains susceptible to false positives, primarily due to limitations in the underlying MILM's s vision- laiguale perception and reasoning capabilities.
Fig. [T shows a failure on relatively simple task in Assualee/Aree that exposes this issue: both the agent and the verifier fail to count the number of elements in an image correctly, resulting in false positive. This is a known weakness of current MLLMs Rahhmnzaaehgerrvi et al. (2025) that becomes even more problematic our setting, where perception errors can compound across multi-step trajectories.
A more complex failure is illustrated in Fig. 12 which juxtaposes the verifier's strengths and weaknesses. The verifier successfully detects that the agent has terminated the task prematurely, failing to submit a required comment. However, it misses two critical issues: (i) that the agent has incorrectly counted the number of red keys in the image, and (ii) that it has posted the comment in the wrong location. Accurately verifying these mistakes would require the detection of fine-grained visual cues, knowing which small button the agent clicked on, for example.
As perceptual abilities continue to improve in future MILMss, we anticipate a corresponding increase in the effectiveness of SGV, its benefits are largely orthogonal these capabilities. Alternatively, an interesting direction for future work is to pursue modular approaches that combine models with complementary, orthogonal strengths-for example, integrating specialist models for fine-grained perception tasks such as object counting or UI element recognition, thereby improving the verification accuracy in visually demanding scenarios.
28

---

Published as a conference paper at ICLR 2026
D MAIN RESULTS DETAILED BREAKDOWNS
D.1
OFFLINE EVALUATION OF AGENT TRAJECTORIES BREAKDOWNS ENVIRONMENT
Table 6: Performance of MILM verifiers on VisualWebArena Trajectories.
(a) No SGV
(b) SGV
(b) (a)
Model
Acc.
dSkew
TPR
TNR
Bias
dSkew
Bias
dSkew
Gemini 2.0
28
-9
Gemini 2.5 Lite
91
42
28
21
+5
Gemini 2.5
65
90
26
21
76
71
11
5
+11
-15
-16
Gemini-2.5
70
12
-7
-12
GPT-4.1 Mini
59
94
31
35
31
72
84
62
11
4
+13
-24
-27
GPT-4.1
GPT-ol (T)
70
80
62
22
16
78
83
73
13
7
+7
-9
-
04-Mini
73
+8
9
8
GPT-5 Nano
13
70
76
66
2
+4
-7
GPT:5 (T)
77
81
81
81
O
+5
-7
-3
Qwen3-32b
Qwen3-235b-a222
65
86
47
23
16
73
80
68
8
+8
-14
13
Swvn3323Shha22bb (T)
+9
62
85
42
24
17
67
80
56
16
9
+5
-9
-9
((T) Thinking enabled with the maximum thinking budget for the corresponding model.
Table 7: Performance of MLLM verifiers evaluation of OoWddd trajectories.
(a) No SGV
(b) SGV
(b) - (a)
Model
Acc.
TPR
TNR
Bias
dSkew
Acc.
TPR
TNR
Bias
dSkew
Asc.
Bias
dSkew
Gemini 2.0
66
99
33
74
97
67
25
24
+8
.9
Gemini 2.5 Lite
100
96
60
31
29
+13
-15
-16
Gemini 2.5
13
+13
-15
-17
Gemini 2.5 (T)
78
95
73
20
18
87
91
86
8
S
+9
-12
-14
GPT-4.1
69
24
24
+16
-16
-16
GPT-4.1
93
16
85
84
9
6
+5
-7
-8
GPT.o4-Miis
91
2
+6
-2
GPT-5.Nano (T)
16
90
80
14
10
+5
-6
GPT:5 (T)
87
90
92
90
92
4
2
+6
-5
Qwen3-32b
20
18
+8
66
100
56
34
34
78
99
73
21
21
+13
-13
-14
Qwen3-235b-a22b (T)
73
21
21
+13
-13
-14
99
53
35
35
+5
-6
-6
*(T) Thinking enabled with the maximum thinking budget for the corresponding model.
D.2 ONLINE SUPERVISION BREAKDOWNS FOR OSWORLD
Tab. 8 shows the performance of the base agent with and without verifier intervention across each of the domains included in OSWorld.
29

---

Published as a conference paper at ICLR 2026
Base Agent
+ SGV, outcome verifier
+ SGV, verify every steps
All
22
25
27
Chrome
29
30
GIMP
24
28
31
LO Cale
15
15
16
LO Impress
25
35
LO Writer
30
44
41
Multi Apps
8
9
13
OS
36
39
42
Thunderbird
36
VLC
8
8
VS Code
57
62
57
Table 8: UI-Tars- ---Ta---1.5-bb performance across World domains with and without SGV supervision. All numbers averages across three runs. More frequent interventions help prevent the agent from derailing into hard-to- recover states, thereby delivering superior performance.
D.3 ONLINE SUPERVISION PERFORMANCE BREAKDOWN PER CONFUSION METRICS
Before Intervention
Intervention Type
After intervention
Before Intervention
Intervestion Type
After Intervention
% Sacuno
Sucoees
Sucois
The 260
244
Falure
Salli
fafure
% malure
AP
PP:75
(a) SGV
(b) No SGV
Figure 3: Online performance transitions after verifier interventions. The boxes on the left and right show the base agent's 's performance before and after the MLLM Verifier intervention. The boxes the middle categorize the interventions as false/true positives, false/true negatives. Numbers indicate tasks in the benchmark.
Fig. 3 presents breakdown of the performance gains discussed in Sec. Bec.Swiith and without SGV. The flow -Fiilure-TTN-Suure
shows the percentage of tasks that were initially unsuccessful, were correctly identified as such by the verifier, and subsequently became successful due to the verifier's feedback. The flow (0~FF-J) -P) shows the percentage of tasks that were initially unsuccessful, were incorrectly identified as successful by the verifier, and therefore remained unsuccessful. The flow (I-FN-+1 indicates the percentage of tasks that were initially successful, were classified as failure by the verifier, but remained successful after its feedback.
As discussed in Sec. 5. most of the SGV interventions are non-disruptive: (1-FN-+1) shows that the majority of "FN" evaluations SGV remain successful, while 14% of the tasks that were initially unsuccessful become successful after corrective feedback from the verifier (0 (N-+TN-)). In contrast, in the No-SGV case, 35% of the unsuccessful tasks remain unsuccessful due to the lack of corrective signal (0-FF--0). See Sec. C.2 for examples, discussion, and categorization of these patterns.
30

---

Published as a conference paper at ICLR 2026
E ABLATIONS AND ADDITIONAL RESULTS
E. EXTERNAL VALIDATION: AGENTREWARDBENCH RESULTS
Tab. 9 shows the performance of the updated oracle evaluators in AgentRewardßench Men al.] 2025 The original oracles in VisualWebArena exhibit relatively low alignment with human annotations. After applying our fixes, the oracles reach near-perfect agreement with humans.
Tab. shows that the baseline MLLM Verifier used in our analysis of agreement bias relatively strong, setting state-of-the-art AgentRewardBench, while SGV further improves upon this, surpassing even the original oracles included in Visual WebArena
Taken together, these results help validate our experimental design. First, they show that our observations on agreement bias are not artifacts of weak evaluators or flawed implementations. Second, the strong performance of our baseline implementation reinforces the motivations discussed in Sec. 3.22 for introducing stronger agents and environment corrections. Specifically, the high performance AgentRewardBench partly because its trajectories are generated by weaker agents and include bugs in the BrowserGym suite de Chezelles et al. (2025). This results in data points where agents fail due to looping indefinitely or encountering errors (e.g.. "Page not found" or non- working click actions on select elements, see Sec. F.2 which are comparatively easy for MLLM verifiers to detect.
Table 9: Alignment of the original and revised WebArrna oracles with human annotations from AgentRewardBench. Our revised oracles reach near-perfect agreement with human labels.
Original
Improved Oracle
Precision
85.2
100
TPR
58.2
92
TNR
95.9
100
Acc
85.1
98
Table 10: Comparison of our verifier to the best performing judges in AgentRewardBent for WebArena (VWA) environment. Our baseline MLLM Verifiers already achieve state-of-the-art performance. SGV further improves and surpass even the original VWA oracles.
Method
Precision
Rule-based (VWA Oracle, original)
85
Rule-based (VWA Oracle, ours)
100
No-SGV Baseline (Gemini 2.5, no Thinking)
73
WebJudge (GPT-o4, SOTA)
75
SGV (Gemini 2.5, no Thinking)
80
No-SGV Baseline (GPTT-4)
SGV (GPT-o4)
86
Note: AgentRewardBench's 's primary leaderhoard metric is precision (P), which is directly proportional to TPR
metrics used in analysis: P
<1R.. where agent success measured by humans.
E.2 DISTRIBUTION OF MILM RESPONSES ACROSS EVALUATION TEMPLATES
In Fig. 4 we show the distribution of MLLM Verifier responses across 28 scoring templates covering both Likert, score-based, and other evaluation criteria (see Sec. B.9 for examples and our codebase for all variations). We also include interventions aimed mitigating biases in LLM evaluators, such as rephrasing, criteria order shuffling and reversal, and syntactic structure changes Chen et al.] (2025a); et al. (2024). The following summarizes the main findings.
(1) MLLMs tend to concentrate their evaluations at the high end of the scale, largely independent of interventions and how the evaluation is framed.
31

---

Published as a conference paper at ICLR 2026
(2) MLLMs rarely express uncertainty, ambiguous cases, when the evaluation template explicitly includes an "uncertain" option, and reserves the highest score for high-confidence success.
(3) Likert scales tend to produce slightly more balanced distributions compared to numeric-based scales.
(4) Binary scales can exacerbate bias toward favorable evaluations. Including third option to allow the model to "offload"" evaluations into "partial success" helps mitigate this effect. Three-way scales seems sufficient achieve this effect, with no meaningful differences observed for more granular scales.
(5) SGV leads to more balanced distribution of scores for all templates.
(6) These observations align with discussions in Sec. 3.2 suggesting root causes stemming from inherent limitations of RLHF that conflates human rater satisfaction with truthfulness (Sharma et al. (2025) and knowledge extraction bottlenecks depending on how the information is presented Allen Zhu and Li (2024a b), with SGV mitigating these issues effectively.
Fallure/ / Partial Fallure
Partial Success
Uncertain
Success
75
Dresk
(nwetagee
Bissy
eveyy Q. inveted Pndd
range Sasvel
(a) Numeric-based scales
12
20
Brasle
Devenoee)
tevecoges
Awer fet ontortiad
Awads sott
fef
Bonay
Brnnys
Rewe inder
(b) Likert-bused scales
Figure 4: Distribution of MLLM verifier responses across prompt/scoring templates and interventions to mitigate biases in LLM judgments.
E.3 CALIBRATION OF MILM VERIHER DISTRIBUTIONS: LIMITATIONS AND OPPORTUNITIES
This section extends the observations in the main text regarding agreement bias and the tendency of MLLMs to assign disproportionately high confidence scores. Specifically, we examine whether post-processing calibration techniques can address the skewed distributions exhibited by MLLM verifiersSec. E2.2 and provide proof-of-concept experiment on the potentials for sampling- based techniques that take into account the skew created by agreement bias.
32

---

Published as a conference paper at ICLR 2026
E.3.1 PLATT CALIBRATION OF MLLM- VERIFIER OUTPUTS
We first evaluate Platt and Isotonic calibration of confidence scores generated by MLLMs under <<aane-bbsedd Scale, implicit> template illustrated in Sec. B.9 We focus on VisualWebArena, where the distribution of agent successes and failures is relatively balanced (47% success, 53% failure). We use our representative subset(Sec. F.I) ) for testing and train the calibrator on the remaining trajectories ( 605 examples). The threshold separating success from failure is optimized using Youden's statistic on the training set, and we additionally report the Area Under the Curve (AUC) statistic for completeness.
Tab. reports evaluation metrics before and after calibration, compared to SGV under the same template. summary:
We do not observe meaningful gains from applying Platt scaling to the generated confidences.
This is explained by the skew in MLLM score distributions discussed in Sec. E.2 and shown in Tab. 12 MLLMs overwhelmingly assign labels at the upper end of the scale, leaving Platt scaling with no granularity to exploit and causing large clusters identical scores to map the same calibrated probability. The same issue arises with other methods, such as isotonic transform.
In line with Sec. E.2 results, SGV leads to substantially more balanced distribution of scores, with corresponding gains in evaluation metrics.
We highlight these findings seems not an artifact of prompt design/template: as SGV results show, it is possible to obtain more balanced distributions under this evaluation design.
Table 11: Calibration of MLLM verifier confidence scores
Metric
Before Calibration
Platt
SGV
ACC
63
65
72
TPR
92
92
82
TNR
37
40
64
Bias
29
28
10
dSkew
25
23
4
AUC
65
65
74
Table 12: Platt scaling vs. SGV
distribution of confidence scores.
Score bucket
Before Calib.
Platt
Isotonic
SGV
[0, 0.1)
10%
11%
16%
6%
[0.1, 0.2)
2%
10%
0%
8%
[0.2, 0.3)
5%
3%
4%
13%
[0.3, 0.4)
4%
1%
4%
5%
[0.4, 0.5)
2%
1%
1%
1%
[0.5, 0.6)
2%
74%
3%
[0.6, 0.7)
1%
0%
0%
4%
[0.7, 0.8)
1%
0%%
4%
[0.8, 0.9)
0%
0%
0%
3%
[0.9. 1.0
74%
0%
55%
E.3.2 PROOF-OF-CONCEPP, CALIBRATION OF SAMPLING-DERIVED PROBABILITIES
We further explore whether calibration is more effective when applied to probabilities estimated sampling. Specifically, for each of the 910 trajectories in VisualWebArena. we sample 8 completions
We observe that letting the MLLM freely generate confidence provides inferior results compared to providing some grounding the confidence scale, as in the utilized template. Therefore, for confidence generation we adopted the latter variation, mapping 1-10 to 0-1 during calibration.
33

---

Published as a conference paper at ICLR 2026
and estimate empirical probabilities by measuring the proportion of times the MLLM labels trajectory as success or failure under the best- best-perforee perfformng template across the 28 tested the ternary Likert scale used in our baselines. To test the Platt calibrator, subset that provides the same distribution as the full benchmark, and we train on the remainder of the data (605 datapoints). In both cases, the data is roughly balanced, with 47% success and 53% failed trajectories.
utilize our 1/3-sized representative
Before discussing results, highlight this experiment is proof-of-concept under relatively ideal and computationally expensive conditions. Particularly:
The calibrator is trained and tested on datasets with roughly equal numbers of success and failures.
It assumes access the high-quality labels provided by improved VisualWebArena oracles Sec. 3.3 a limitation real-world settings that motivates the need for MLLM verifiers in the first place.
Training and test domains match; i.e., we do not consider out-of -itt-ff-iittibuttion generalization.
To estimate probabilities, sample relatively large number of completions (8 per trajectory, totaling over 7,200 generations).
The thresholds used separate classes-whies shown below, be sensible choice optimized under these same ideal conditions.
At test time, we still generate completions per new trajectory for probability estimation, and apply the calibrator to them.
Results are shown in table Tab. In summary:
Vanilla majority voting fails to deliver improvements for not taking into account the skew in MILM label distributions toward positive evaluations.
Leveraging this fact to apply calibration to MILMM implicit probabilities leads more balanced distributions and improvements in evaluation metries.
Most notably, SGV lead to distributions superior than the calibrated ones obtained in ideal scenarios through a single generation. Importantly, this arises organically by leveraging the model's own sampling mechanisms, without the need for ground-truth labels, additional training, or calibration steps. Moreover, these gains naturally extend to other domains, demonstrated by SGV's improvements across benchmarks.
SGV also avoids complicated hyperparameter tuning such calibration thresholds. As shown in columns 'p15' and '0.5', setting this threshold to either the 15th percentile (bonostrap-estimated) or naively 0.5 eliminate the gains from Platt scaling.
Taken together, these results point some important implications. First, they reinforce that SGV's effectiveness stems from mitigating the intrinsic bias present in MLLM response distributions. Secondly, they suggest potential for both training- timingg-imme and test-time approaches that explicitly account for the asymmetries introduced by agreement bias-e.g.. loss functions that penalize such imbalance, adjustments to sampling mechanisms for methods that rely sampling (e.g., GRPO). Since SGV itself induces more balanced output distributions, incorporating SGV into larger pipelines may also be fruitful-
replacement for oracle labels in methods like the one above.
We hope that our identification of agreement bias, our demonstration of the risks it poses for applications relying MILM verifiers, our evidence the benefits of mitigating this bias to downstream applications, and the insights provided help motivate further research in this direction.
Table 13: Platt scaling vs. SGV using sampling-derivve probabilities under a ternary Likert template.
Metric
NoSGV temp=0
NoSGV maj
Platt
Platt,11
Platto
SGV temp=0
SGV maj
SGV
Platt
ACC
65
65
69
71
76
75
TPR
92
93
77
85
86
84
85
79
TNR
41
40
68
56
57
71
67
78
Bias
27
28
20
16
11
11
dSkew
23
24
15
5
5
AUC
66
75
76
76
34

---

Published as a conference paper at ICLR 2026
E.4 OUTCOME VS PROCESS-BARE BASED VERIFICATION
For some domains, such as OSWorld, certain actions can be more destructive, pushing the agent into states that difficult impossible recover from-for example, deleting file. This naturally raises an important question: when should a verifier be invoked? While we leave a full exploration future work, Tab. proovides initial evidence for the promise of this direction, particularly when equipped with verifiers that know when to intervene in the first place. We examine two configurations: one in which the MLLM verifier intervenes only after STOP actions (the poutcome-based setting), and another in which it evaluates trajectories periodically every 5 steps. As observed, the qualitative conclusions mirror those in the main text: a baseline MLLM verifier yields only modest improvements. whereas SGV consistently produces substantial gains in both configurations, with slightly better results when verification is more frequent. More generally, we view the study of process and outcome- based verification (Ren et al., 2023 Lightman et al.) 2023 Shao et al., 2024bb Setlur et al., [0225] Swamy et al. 2025) in digital environments as an exciting direction for future work, particularly due to their often discrete state-space representation that makes process reward modeling more amenable.
Table 14: Agent Performance across OSWorld and Visua/WehArena under periodic and outcome-based verifica- tion.
Verification Mode
OSWorld
Vesual
Baseline Agent
21.7
45.0
No SGV
+ antcome-based
23.4
46.1
+ Every 5 Steps
24.3
46.9
SGV
+ Outcome-based
24.9
54.0
+ Every 5 Steps
26.5
54.9
E.5 IMPACT OF TRAJECTORY LENGTH ON VERIFIER PERFORMANCE
Some tasks in the digital world can require long trajectories, which raises the question of how verifier performance is affected. In Tables [5] and T6 we examine MI.LM verifier performance stratified by trajectory length in OSWorld and in an extended 60-step configuration of VesualWebArena. To increase the number of samples per bucket and improve statistical stability, we average results across three comparable MLLMs (GPT-4.1, Qwen3-VL, and Gemini 2.5). Across both benchmarks, several consistent patterns emerge. First, SGV continues to reduce bias and skewness and improves TNR and accuracy for all trajectory lengths, reinforcing the robustness of the method. Second, verifier performance tends to increase with trajectory length. As reflected by the low success rates in longer trajectories, this pattern arises largely because longer trajectories typically indicate that agents have drifted further from the task objective, making failures easier for verifiers to detect. For example, UI-TARS repeatedly opening irrelevant applications, or performing destructive operations like file deletions in OSWorld.
This observation reinforces an important methodological point raised in Sec. 3.3
to evaluate verifiers meaningfully and to surface limitations such as agreement bias-it is essential to use strong agent policies (like our Webuena agents) that produce high-quality trajectories.
While our results show no degradation in performance at longer trajectory lengths, we acknowledge that the combination of strong agents and long trajectories may introduce new challenges, including amplified agreement bias or yet-unidentified failure modes. However, a full examination of these factors under the current state of benchmarks and agents is challenging: unless tasks inherently require long trajectories and agents can reliably produce them, disentangling these factors without introducing confounders non-trivial, In Sec. E.6 we partially address this limitation, varying agent quality in VisualWebArena while keeping trajectory length fixed. Experiments reveal that (i) verifier performance degrades when the verifier is weaker than the agent, specifically due to an increase in agreement bias, and (ii) SGV consistently improves outcomes across all settings.
35

---

Published as a conference paper at ICLR 2026
Table 15: MLLM verification performance by trajectory length Webrrraa
Metric
1-5
6-19
>20
% Success
47
57
36
17
N Samples
2730
1587
1218
198
% Total.
100
58
45
7
No-SGV
ACC
68
65
TPR
90
91
88
64
TNR
40
46
65
Bias
24
22
30
22
dSkew
19
17
26
16
SGV
ACC
73
82
TPR
82
84
81
66
TNR
60
85
Bias
10
9
14
6
dSkew
4
2
A (SGV No-SGV)
ACC
+7
+5
+11
+17
Bias
-14
-13
-16
-16
dSkew
-13
-14
Table 16: MILLM verification performance by trajectory length in OSWorld.
Metric
All
1-10
11-20
21-30
:>30
% Success
22
54
20
16
2
Samples
1044
300
198
435
% Total.
100
29
19
11
42
No-SGV
ACC
76
71
59
67
89
TPR
96
97
89
67
TNR
70
41
49
63
89
Bias
22
40
dSkew
21
24
39
27
9
SGV
ACC
83
77
75
81
94
TPR
91
90
97
88
67
TNR
81
61
70
80
94
Bias
13
24
dSkew
10
8
23
14
5
A (SGV -N-SSVV
ACC
+7
+6
+16
+14
+6
Bias
-10
-13
-13
-15
-5
dSkew
-11
-16
-13
-13
-4
E.6 CROSS-MOnE VARIATIONS AND VERIFICATION DIFFICULTY
In Sec. 4 we discuss how verification is affected by verification difficulty when comparing perfor- mance of a weaker agent in OSWorld and VisualWebArena. The results in Tab, 17 complement those findings by exploring variations in model strength between the models used to build the agents (or summary,
"generator") and the verifiers while keeping the environment constant WisualWebAreen results show that:
36

---

Published as a conference paper at ICLR 2026
(i) Agreement bias occurs both when the verifier is stronger weaker than the generator, and when agents and verifiers are built from models of distinct families and size.
(ii) Verification performance is worse when the verifier is weaker than the generator (c.g., Gemini 2.0 verifies Gemini b.5-based agent), particularly due to an increase agreement bias (note TPRs actually increase).
(iii) SGV remains effective in all configurations, providing substantial improvements over the no-SGV baseline.
Table (No SGV, SGV) performance across verifier-generator configurations VisualWehArena, G-2.5 and G-2.0 refer to Gemini-2.5- and Gemini-2.0-Flash, respectively.
Verifier
G-2.5
G-2.5
G-2.5
G-2.0
G-2.0
Agent
G-2.5
G-2.0
GPT4o
G-2.5
GPT4o
Ace
(65, 76)
(70, 78)
(68, 77)
(56, 65)
(55, 65)
TPR
(90, 84)
(84, 77)
(80, 76)
(93, 90)
(85, 74)
TNR
(47, 71)
(63, 79)
(63, 77)
(28, 42)
(42, 61)
Bias
(26, 11)
(20, 7)
(20, 8)
(38, 28)
(35, 19)
dSkew
(21, 5)
(14, 2)
(13, 3)
(35, 23)
(30, 12)
Agent Success Rate
47
A Acc
11
8
8
9
9
A Bias
-15
-13
-12
-10
-16
dSkew
-16
-12
-11
-12
-18
E.7 ABLATIONS TO SGV EXTRACTION AND VERIFICATION STEPS
Tab. 18 presents ablations for results in Sec. where (1) start with conventional MLLM verifier,
(2) add prior generation step, (3) decouple prior generation and verification steps without modifying any of the prompts, and (4) add separate prompt for the prior generation step. In all cases, include CoT instruction. Prompt templates are provided in Sec. B.8
Table 18: Ablation results for SGV's two-step mechanism.
Method
Accuracy (%)
TPR (%)
TNR (%)
Bias (%)
dSkew
Base MLLM Verifier
65
90
47
26
21
First Step
49
SGV, Unified
67
4
SGV
76
84
71
11
5
The conventional MLLM-based verification yields suboptimal performance, with slight improvement when a prior generation step is added (rows 1-2). One reason for this behavior is that when steps are combined, models tend to generate priors that align to information in their context window-a manifestation of agreement bias reducing their effectiveness in grounding the verification. De- coupling the steps improves performance (rows 3-4). with SGV achieving similar results with or without specialized prompts. This suggests that SGV's effectiveness does not from prompt specialization and that it can be integrated easily with minimal modifications to existing codebases.
E.8 ADDITIONAL ABLATIONS: PRIOR DIVERSITY, GROUNDING TOOLS, AND QUALITY OF FIRST-STEP GENERATION
In this section, examine the effect of prior diversity SGV performance and analyze the robustness SGV under various coor-generation strategies. Across both sets of experiments, we observe that increasing the diversity of priors enhance verification performance, while repeated
37

---

Published as a conference paper at ICLR 2026
sampling increased temperature within single model family yields diminishing returns. We further demonstrate that alternative grounding strategies, such as providing web-search tools, fail to meaningfully mitigate agreement bias, and that SGV remains effective when prior generation is noisy or produced by weaker models, highlighting the stability of the mechanism.
Effect of Prior Diversity. To study how the diversity of first-step priors affects verification perfor- mance, table Tab. reports results for four configurations: (1) a single prior generated from the same verifier model at high temperature; (2) single prior from model of different family; (3) concatenation of three priors sampled from the same model; and (4) a concatenation of three priors produced by three distinct model families. For reference, we include the baseline SGV and No-SGV settings. In all cases, the verifier model is Gemini-2.5 1a5imi-..5-Flase 2.5-Flash with temperature.
In summary, results shows that:
Diverse priors improve performance. Incorporating priors from multiple model families (Column 5) yields the strongest results, suggesting that epistemic diversity provides additional useful signal for verification.
Increasing temperature or sampling within a model offers limited benefit. Columns and demonstrate that the first-step prior has been generated, additional variability from the same model yields marginal gains, indicating that most of the benefit is captured in the initial SGV step.
Table 19: Ablations on prior diversity.
No SGV Baseline
SGV Baseline
Setting
(5)
First Step
G(-
(()=))
GP
G (k=3)
G+Q+GP (k=3)
Verify
G
G
G
G
G
Acc
65
76
77
76
76
78
79
TPR
90
84
84
84
84
87
82
TNR
47
71
71
70
76
Bias
26
11
11
11
11
12
6
dSkew
21
5
5
5
5
7
2
Legend:
Gemini 2.5
Qwen3-VL LL
GP GPT-5
t= temperature
Effects of Grounding Tools, Noisy Priors, and First-Step Model Strength. To further investigate agreement bias, and the effectiveness and robustness of the SGV mechanism, Tab, compare MLLM verification performance in the following settings:
(1): No first-step prior generation, but provide the verifier Web Search tool to ground the verifica- tion (Gou et a
(2): SGV, where the prior generation is supported by web search tool;
(3) SGV, where prior generation is produced by a weaker variant of the same model;
(4) SGV, where we inject into the generated priors random noiseShao et al. (2025)
(5): No SGV for thinking variant of Gemini, but provide the verifier Web Search tool to ground the verification
(6): SGV with thinking enabled, where priors are generated with thinking disabled.
In summary, results indicate that:
contrast to SGV, providing grounding tools such as web search to the verifier alone does not yield meaningful mitigation of agreement bias;
Extracting knowledge from the own model seems sufficient, with the addition of web search not providing meaningful improvements;
Weaker and non- oon-thinking models can produce priors that are sufficiently informative for effective verification for stronger thinking models;
38

---

Published as a conference paper at ICLR 2026
The similar performance of SGV with and without Web Search, the near-identical performance of Gemini- Thinking when priors are generated with or without thinking (6 vs. SGV baseline), and the comparable performance when thinking is fully disabled versus fully enabled all suggest that SGV's first-step mechanism accounts for the vast majority of the gains, and largely exhausts them.
SGV remains effective even when the first-step generation is noisy, indicating that the method is robust to some degree of noise in the first-step prior generation.
Table 20: Effects of Grounding Tools, Noisy Priors, Priors with weaker and stronger models.
NoSGV Base
NoSGV
SGV
Setting
(2)
(3)
(4)
(5)
(6)
Base
Base (T)
Base (T)
First Step
G Web
G-Lite
G Noise
(T) + Web G(T)
G (T)
Verify
G + Web
G
G
G
G(T)
G (T)
(T)
Acc
65
64
76
74
74
73
69
66
77
76
81
TNR
47
44
65
69
68
55
61
70
Bias
26
29
17
11
14
23
12
dSkew
10
Legend: Base baseline; G = Gemini-2.55 G-Lite = Gemini-2..5FFaashee (T) Thinking enable with
thinking budget; Web Web Search Tool provided.
E.9 ONLINE SUPERVISION AND SGV WITH WEAKER MODELS
Tab. 21 shows the performance of online verification with and without SGV the full Ve- bArena benchmark utilizing gemin - 0-ffash, a weaker model than the one used in Sec. 5 Consistent with findings in Sec. the no-SGV verifier fails to improve performance and even results in degradation in certain domains. In contrast, SGV consistently improves results across all domains, achieving percentage point gain overall (9% relative improvement).
Table 21: Task success rates (%) Visual AebArrn with and without SGV using gemini- 12.nii-2... f-flssh as the base MLLM.
Method
All VWA
Shopping
Reddit
Classifieds
Base Agent
36
38
27
41
+ Verifier, no SGV
36
39
27
40
+ Verifier, SGV
40
29
46
F ADDITIONAL DETAILS AND RESULTS FOR VISUALWEBARENA
F.1 VISUALWEBARENA LITE
Running the full VisualWebArena benchmark is time-consuming, hindering prototyping, experimen- tation, and ablation studies. To address this issue, we release VisualWebArees Lite, a representative subset of tasks that preserves the performance trends observed on the full benchmark, while consisting of only one third of the tasks. At a high level, the subset is built by iteratively adding and removing tasks to match the distribution templates and other characteristics such as task difficulty, while ensuring that the aggregate success rate within each domain remains close to that of the full set for given agent.
To avoid introducing bias toward our implementations, construct the subset based on the perfor- mance of Search Agent Koh et al. (2024b), for which task-level scores are publicly available.
Tab. 22 compares task success rates across agents and model families both the full benchmark and the subset. The differences in success rate between the full benchmark and the subset range from 0 to 1 percentage point, while execution time and token usage reduced by approximately 67%, substantially accelerating the development cycle and lowering inference costs. Moreover, note the performance gain of the base agent Verifier over base agent is approximately 4.5 pps the full
39

---

Published as a conference paper at ICLR 2026
benchmark and the representative subset, indicating that meaningful signals of iterated versions of similar agents are preserved, allowing for effective prototyping and ablation studies.
Table 22: Task success rates (%) WisualWa for the full benchmark and the representative
subset VisualWebArena Lite (VWA Lite).
(a) Full Benchmark
(b) VWA Lite
(b-a)
Method
Model
All Shop. Reddit Class.
Shop. Reddit Class.
Search Agent Koh et al. (2024b)
GPT-4o
27
29
21
30
29
30
+0.3
ReAct
Gemini 2.0
36
38
27
41
36
35
43
-0.4
ReAct + SGV
Gemini 2.0
40
41
29
46
40
40
31
47
+0.2
ReAct
Gemini 2.5
47
49
36
52
46
47
35
52
-0.8
ReAct SGV
Gemini 2.5
54
58
44
56
54
57
42
-0.1
Number of Tasks
910
466
210
234
305
156
70
79
Token Usage (millions)*
104
54
29
21
34
18
10
6
Token usage measured for Gemini 2.0 (Base Agent + SGV).
F.2 ENVIRONMENT REFINEMENTS
As discussed inSec. 3.3 we make several improvements the Webbrrea environments to address bugs and provide more reliable evaluation setup for MILMM verifiers. The following outline some the major upgrades. For more details, please check the documentation in our codebase.
Proper Environment Paralellization. Resetting and parallelizing environments is known issue in (Visual)WebArena]] partially due to hard-coded dependencies between task configurations and Docker instances. This not only slows down evaluation but also prevents proper resets between episodes, leading to potential state leakage across tasks and unreliable evaluations. For example, some issues in the BrowserGym suite de Chezelles et al. (2025) observed in AgentRewardßeecc Men et al. 2025) trajectories arise from failures to renew cookies and session data between episodes. These failures cause agents end in unexpected states during execution, making tasks easier detect as failures during verification.
Therefore, we refactored the environment code allow proper resets and parallelization, enabling faster and more reliable evaluations. In Table Tab. 23 we report the speedup gains obtained with the enviromments we release. When considering our representative subsets, these gains can reach up to 20x, substantially reducing evaluation cost, facilitating rapid prototyping, and enabling more faithful evaluation due to proper environment resets.
Table 23: WebArraa Evaluation runtime statistics.
Speedup
Total Time
Avg / Task
Original Full 910 tasks
2d:20h:09m 3d:18h:31 11:3hh33m
04:31m 05:58m
Full 910 tasks + Reset
5 Environments in Parallel
Full 910 tasks"
7
12h:13m
00:50m
Visual Aitenn--Lte
21x x
04h:044
00:50m
Reset by default.
Updates to Oracle Evaluators and Task Configurations. We aimed to fix only issues whose resolution involved subjective judgment. Below, we summarize and illustrate the primary classes of issues and their corresponding fixes. Full documentation is provided our codebase. See Sec. E. for comparison of the original and revised oracles evaluated against human annotations.
https:: thu.. com/ web- wetr-arreaeaxeeee www.aaaa//eeaaareaaaaaaaasseeeee
40

---

Published as a conference paper at ICLR 2026
(i) Mismatch between intent and oracle requirements. Some task configurations included requirements that are not explicitly stated in the original user intent. Where possible, we added minimal instructions to the intent to clarify these requirements. Examples:
Original Template: Find me the <item> with caltributes>
The oracle requires navigating to the item's 's page to complete the task, although this not stated in the intent. We append the instruction: vi To finish the task, please make sure navigate the page of the corresponding item.
Explanation:
Revised Template: Find the <item> with <attributes> Nito finish the task, please make sure to navigate to the page of the corresponding item.
Original Template:
Navigate to my listing of <object> and change the <attribute> to
<target>.
The oracle requires submitting the changes, though the intent does not specify this. We append the instruction: viPlease make sure to submit the changes.
Explanation:
Revised Template: Navigate to my listing of <object> and change the <attribute> to <target>. VnPlesse make sure to submit the changes.
(ii) Incorrect annotations. We corrected task configurations that referenced incorrect target items, linked to the wrong pages, or included incorrect price values or strings.
(iii) String parsing bugs. We resolved issues in the handling punctuation, number formats, special characters, and timezones for evaluators based on string comparisons.
(iv) False-positive evaluators. We refined evaluators that previously misclassified failures as successes. For example:
Some tasks contained no valid targets matching the user's stated constraints. In such cases, the original oracles sometimes awarded a score of even if the agent performed only random no actions. For tasks such as Subscribe all subreddits that start with the letter [[letter)) and have a {(object)] image in their top posts, the original evaluator only checked that no new subscriptions appeared. We introduced lax evaluators that verify whether the agent at least visited pages with partial URLs corresponding to candidate subreddits.
For tasks of the form Navigate to <target> that contains a picture of <object>., the original oracle ran a BLIP-2 classifier over all images on the page and awarded success if any image matched. This led to two issues; (i) numerous small, unrelated, or out-of-viewport thumbnails were included, and the model frequently assigned a score of to at least one of them; and (ii) other requirements, such navigating to the correct <target> page (e.g.. a post's comments section), were not enforced. We therefore added checks ensuring that the trajectory contains characteristic DOM elements URLs associated with the intended <target> page, ensuring elements and URLs that are common across all instances to not introduce false negatives.
(v) Fuzzy-match evaluators. Some tasks rely on LLM-based evaluators with predefined prompts and privileged information. We refined prompt templates to reduce both false positives and false negatives, while leaving annotations unchanged.
Bug Fixes and General Improvements.
We introduce other several improvements to the VisualWebAArna environment, including enhanced action parsing, dynamic waiting for page-load completion and environment resets, refinements to the Set-of-Marks representation, correct handling of tab metadata, scroll-bar rendering, incorporation of additional models for fuzzy-man mazzy-match evaluation, and hosted inference for the captioning model. All changes are implemented at the environment level so that subsequent research can directly benefit. Tab. 24 shows agent performance before and after the environment upgrades described in this section, demonstrating relevant improvements across all configurations. We encourage readers to explore our codebase and make use of any components that may be helpful to their own research.
To illustrate, one notable upgrade involves interactions with HTML select elements. Many Visual WebArena tasks require choosing values from dropdown menus- example, sorting product
41

---

Published as a conference paper at ICLR 2026
lists applying category filters. In the original environment, these interactions could not be reliably handled using standard click actions due to Playwright limitations, requiring augmentation of the action space with specialized select actions. In the original environment, these interactions could not be reliably performed using standard c i ck actions due to Playwright-related limitations, effectively requiring augmentation of the action space with specialized select. ect operations. The later approach introduces two issues: (i) agents may still fall into infinite loops when repeatedly attempting ineffective click actions select options, and (ii) agents become less comparable to mouse-only baselines.
To address this, we upgrade the environment so that standard click actions correctly trigger and select options from select elements. F (left) shows trajectory from AgentRewardBench generated via the BrowserGym suite de Chezelles al. (025),, while the right panel shows the same webpage in our upgraded enviroment. Before the fix, options within the dropdown are not recognized as interactable elements and therefore cannot be clicked. After the fix, they are correctly exposed in the representation, receive bounding boxes, and can be selected using standard cl lick actions. We note the environment remains compatible with specialized select actions (also provided in our codebase, but not used in any of our results).
Search results for: 'chairs'
Search results for: choirs'
(a) Before fix
After fix
Figure 5: Bug fix in select elements. Left: Snapshot of trajectory included in AgendRewardBench generated with the BrowserGym suite. Right: Snapshot the same state after fixes to the handling of select elements.
Table 24: Task success rates (%) on VisualWebArena using gemini 2. 5.5-m flash as the base model, before and after environment refinements.
Method
All VWA
Classifieds
Reddit
Shopping
Base Agent Before Refinements
46
42
Base Agent After Refinements
47
49
51
Agent SGV Before Refinements
Agent + SGV After Refinements
54
56
43
58
42

---

Published as a conference paper at ICLR 2026
G ADDITIONAL FIGURES
G.1 EXAMPLES OF TASKS IN DIGITAL BENCHMARKS
I really like these shocs, but I am not satisfied with the color. Could you find me an item on this page for these shoes with a darker shade or color?
VANS
you give me the for a gift could send to my friend sbout the conirast of investing in retirement account vs brokerage sccount from the bot of #Fallsttreebbtss
Show expensive collectible historical USA that under $2000
My glasses are broken, and I'm having trouble secing small things clearly. Could you help enlarge my screon so it's essier to read?
Please set my slides upright instead sideways.
Figure 6: Examples of tasks in VisualWebArena and OSWorld. Top row: a query composed of natural language and an image, followed by the initial state screenshot. Second and third rows: natural language queries with corresponding initial state screenshots.
43

---

Published as a conference paper at ICLR 2026
G.2 AGREEMENT BIAS
Buy the cheapest deodorant from the "Deodorants & Antiperspirants" category with the phrase 'killer' on the packaging.
Seechnous for Seoderanidder Aater
Seachnosu for Aill
type
[7]
[decdorant killer]
click [52]
step [The decderant has baen added to your cart successfully.
Verifier Responses
MI.IM Verifier
REASONING: The assistant has successfully added the cheapest doodorant with "killer" on the packaging to cart. The product meets the criteria of being in the "Deodorants & Antiperspirants category and having the phrase "killer" on the packaging. The assistant has completed all necessary steps to achieve the objective.
EVALUATION: SUCCESS
MI.IM Verifier + Spec--Specffic Evaluation Instruction:Pan et nl. 20124a))
Thoughts: The agent successfully added the Men Deodorant 48-hour Odfor Protection Clean Comfort Desdorant for Men Original Antiperspirant Deodorant Odor Protection Sport Mens Deodorant Stick to the cart, which matches the user's intent to buy the cheapest deodorant with 'killer' on the packaging. The task completed successfully.
States: success
Two-Step Verifier
REASONING: The ussistani successfully searched for "decdorant killer" and adided the lirst item to the cart. However, the objective find deodorant with the wont "killer" on the packaging and to huy the cheapest onc. The assistant did not verify the product contained the word "killer" the packaging. did the results by price find the cheapest option.
EVALUATION: PARTIAL SUCCESS
SGV - First Step Generation
To accomplish the task buying the cheapest deadorant with the phrase 'killer' the packaging from the "Deodorants Antiperspirants" category, the following steps are typically taken on the web: Navvigate to Relevant Category**, * Locate and click found in the "Beauty& & Personal Care" section or similar category.
category includes deodorants and aniperspprants. This usually
**Scarch the Specific *roduct* Use the scarch bar to enter keywords such as "doodorant killer" to filter the results to only show deodorants with 'killer' on the packaging.
3. "eFilter and Sort Resulls**; Apply filiers to namrow down the search resalts to only include decuorants. Then, sort the results by price,
selecting the option to display the cheapest items first.
4. *eReview Product Listings*:: Look through the filtered and sorted list to find the cheapest decdorant that meets the criteria. Check the product images and descriptions ensure word "killer" present on the packaging.
5. + Exxaminc product details (if necessary):* ** If product's listing doesn't 't clearly show the packaging or include the keyword in the ditlodescription. you might need to click on the product to view its detailed page. This page often provides more images, a full description. and sometimes customer reviews that might mention details about the puckaging.
6. e* Add to Can Once the correct product is identiied, click the "Add to Cart" button to add to the shopping cart.
Proccod to Checkour* Navigate the car proceed checkout complete the parchase.
Figure 7: Agreement Bias. Trajectory produced by ReAct agent VisualWebArena (top). the corresponding evaluations produced by three verifier variants (midilee), and the outpul gemerated in the first step (botion).
44

---

Published as a conference paper at ICLR 2026
Buy the least expensive red blanket (in any size) from the "Blankets Throws" category.
scroll [down]
cliok [70)
type (7] [red blanket
elick [34]
click [26]
click [29)
click [32]
stop [The least
expensi red blanket has been added to the cart.]
Verifier Responses
MLLM Verifier
REASONING: The ussistant successfully navigated through the website, searched for "red blanket," and identiiied the leust expensive red blanket available. The assistant then proceded to the product page, selected the required options (color and size), and added the item to the cart. All necessary steps were executed correctly to fulfill the objective.
EVALUATION: SUCCESS
MLLM Verifier + Tpsk-Specific Evaluation Instructions (Pan et al. 2024a
Thoughts: agent successfully adds blanket
expensive blanket. agent first searches for "red blanket", then clicks on the first item, which is a red blanket. The agent then clicks on "Add to Cart". The agent's response 'The least expensive rod blanket has been added to the cart." misleading, because the agent did not explicitly compare the prices of different red blankets, However, since the agent adied red blanket the cart, and the user's intent is to buy the least expensive red blanket, I will mark this as a success.
curt. user's intent buy
Status: success
MLLM Verifier + SGV
REASONING: The assistant successfully adieed blanket the cart, which crucial step towards objective. However, is not fully accomplished as the assistant needs to ensure that the added blanket is indeed the least expensive red blanket in the "Blankets & Throws" category. The assistant should have first navigated to the "Blankets & Throws" category, filtered for rod blankets, sorted by price find the least expensive option. The current action of adding blanket to the cart without verifying in the category premature.
selative to other options
EVALUATION: PARTIAL SUCCESS
SUCCESS
Figure 8: Agreement Bias. Trajectory produced by a ReAct agent in Visual WebAreaa (top), and the corre- sponding evaluations produced by three verifier variants (bottom).
45

---

Published as a conference paper at ICLR 2026
Buy the least expensive canvas print with grapes from Posters & Prints category.
Hespen
Lype [7] with grapes]
Teaavas prinl
click [30]
click [34]
stop (1
Verifier Responses
MLLM Verifier
REASONING: 1. The objective was the least expensive cunvas print with grapes from the "Posters & Prints" category.
The assistant started on the homepuge and used the search bur to look for "canvas print with grapes," which is a reasonable appronch
since the "Posters & Prints" category not directly visible.
3. The scarch results included several canvas prints featuring grapes. The assistant correctly identified the least expensive option:
"TTSPPEI Framed Kitchen Pictures Wall Decor Green Grape and Red Wine Canvas Prints" for $6.99.
4. The assistant attempted to add the item to the car but encountered required option (color selection). The assistant correctly selected the appropriatc option and successfully added the item to the cart.
The assistant proceeded to the cart, then checkoat, selected the shipping method, and continued to the payment page.
6. The assistant placed the order, reaching the confirmatinn page, which confirms the purchase was completed. every step, the assistant verified the requirements and made the correct selections to ensure the least expensive qualifying item was purchased.
EVALUATION: SUCCESS
MI.LM Verifier + SGV
REASONING:
objective buy the least expensive canvas print grapes from the "Posters Prints" category. The ussistant then searches for "canvas print with grapes" using the search bar, which is eorrect step find relevant products directly.
3. After searching. the assistant finds product that matches the criteria (" ('QTESPERi Framed Kitchen Pictures Wall Art Decor Green Grape and Red Wine Canvas Prints...") and adds the
4. The assistant proceeds to checkout and completes the purchase. Allhough the assistant successfully purchasedd canvas print with grapes, it is not clear category or if it is the least expensive option available in that category.
the "Posters Prints"
EVALUATION: FAILURE
Figure 9: Permissive oracle evaluation, agreement bias, and SGV disagreement. Top: The agent searches for a product, clicks on the first
result, and completes purchase. The trajectory marked as succesffll oracles, despite omitiing steps ensure product least expensive, Bottom: iofluenced by agreement bias, an MLLM verifier agroes with the oracle, producing ungrounded reasoning to justify its judgment (red), SGV flags omilted steps and disagrees with the oraele (bold).
46

---

Published as a conference paper at ICLR 2026
For the item on this puge which includes Black Friday logo in the image, tell me the most specific location given of the posting.
OsClass
Legest
Publish-Ad
Your
Furniture
6407
isttngs
Show
lestngs with sesums
Anply
SANDO 64
Tectung
4G2315.00
25000.00
20000.00
Tabscibbe
10065.00
12345.0
11000.00
00
stop [Shaw (Mashington, D.C.)]
MI.IM Verifier (no SGV)
most specific two items with Black Friday logo: item 36 and item 38, Both items are titled "RH Restoration Hardware". Clicking either of these items would lead the product page where the location information would available. The assistant correctly identified the item and provided the location "Shaw (Washington, D.C.)" which is the most specific location given the product page..
REASONING:
objective
Friday logo. Looking screenshot,
EVALLATION: SUCCESS
FEEDBACK: Great You successfully identified correct item and extracted the most specific lncation.
Figure 10: Extreme example of agreement hias and permissive oracle. The dashed box highlights the target item and not provided at runtime. Top: The agent immediately returns location in the first step navigation. despite the trajectory containing no information to justify the response. In this example, the location matches the oracle's requirements and the trajectory is marked oracle, validating the trajectory though, by construction. there is evidence supporting the agent's answer. producing ungrounded reasoning (red) to justify its incorrect judgment.
success. Bottom: An MLLM verifier agrees with the
47

---

Published as a conference paper at ICLR 2026
Leave a comment in this post with text as the number of buns in the image.
type [14] [12] [0]
click [16]
stop
EVALUATION: SUCCESS
FEEDBACK: Great job! You accurately counted the huns and followed the correct steps to post the comment. Keep up the good work!
Figure 11:
Online feedback in VesualWebArrna. False positive due to a classic failure in the integration of visual perception and language (object counting).
48

---

Published as a conference paper at ICLR 2026
How many red keys are in the picture? Leave comment replying to user "liverblow" with the following format: "It has <numher of red key> 5%.00" (not including the double inverted commas), replacing with the actual number of red key. Remember to use digits.
click [60]
Eype [52] [2c has 8891, click
stop c1lck [14]
type [62] [It has $8Ø)
click [16)
stop
EVALUATION: PARTIAL SUCCESS
FEEDBACK: Ensure all required fields are filled before posting comment, Double-check the comment for accuracy and completeness to avoid errors.
Post-feedback: stop click [14]
EVALUATION: SUCCESS
FEEDBACK: Great job! You accurately identified the number of red keys and posted the comment correctly. In future tasks, consider explicitly verifying your comment before posting to ensure accuracy.
Online feedback in VisualWebArema False positive complex task due to failures on the integration of visual perception and language.
Figure 12:
49

---

Published as a conference paper at ICLR 2026
G.3 ONLINE VERIFICATION DIGITAL AGENTS
Find me the cheapest controller from the classifieds site that is meant for the console in the image on the Reddit tah
Wharare you losicin foys
Valeo
Locus [-]
type [3] [Kintenco 61
click [32]
controlleri; click 1612 cliex [30]] click [7]
Nisond d4 Nes Conok
5000
Video pimta
Navaado Ouerr and
23,05
bbbb
stop + go_beck
click [13]
stop
EVALUATION: PARTIAL SUCCESS
FEEDBACK: You correctly identified the console and searched for the controller. However, you did not sort the search results by price to find the cheapest controller. Please go back to the search results page, sort the results by price (lowest first). and then click on the listing for the cheapest Nintendo 64 controller. Remember, the objective is to find the cheapest *controller*, not the cheapest console. Post-feedback: stop + go back
EVALUATION: SUCCESS
FEEDBACK: The assistant successfully completed the task by identifying the console, searching for the controller, sorting by price, and navigating to the page of the cheapest item.
Figure 13:
Online feedback in VisualWebArena. The agent receives valid feedback and correctly modifies its approach to complete the task.
50

---

Published as a conference paper at ICLR 2026
Order the second thing in the top row. If the shipping is more than 7% of the total price, leave star review mentioning it, otherwise 5.
click [36]
click [8]
click [5]
stop (MAX
reached)
click [4]
stop
click [24]
EVALUATION: PARTIAL SUCCESS
FEEDBACK: To fully accomplish the task, ensure to leave a review after the purchase. Calculate if the shipping cost exceeds 7% of the total price and leave a 3-star review if it does, otherwise a 5-star review.
Post-feedback: stop + click [21]
Figure 14: Online feedback in VisualWebArena. The agent receives valid feedback during an incorrect execution, but is unable to finish the task. (Partial trajectory shown due to space constraints.)
51

---

Published as a conference paper at ICLR 2026
How many hours are on the engine the most recently listed red boat?
Whot are you looking for today?
click [38]
scroll
down
stop [80] + go_back
EVALUATION: FAILURE
FEEDBACK: K: The assistant did not attempt to search for "red hoat" as required by the objective. It navigated to the "Boats" category and then scrolled down page. The assistant should have used search find boats. The assistant should go back home page and use the scarch bar to search for "red boat". After searching, the assistant should look for the most recently listed red bost and click on the listing to lind the engine hours.
feedbleed stop [80] go_back
What looking for today?
500000
type [5] [red boat] [1]
stop [80] click [38]
stop [80]
EVALUATION: PARTIAL SUCCESS
FEEDBACK: The assistant successfally searched for "red boat". However. it stopped before clicking on any of the listings to find the engine
hours. The assistant needs click the recently listed red boat and then find the engine hours that page. The recently listed
rod beat is the one at the top of the list.
Post-feedback: step [60] click [38]
EVALUATION: SUCCESS
FEEDBACK: The assistant succossfully navigated to the listing for the red boat and found the engine hours.
Figure 15: Online feedback in Werual The verifier provides feedback that guides the agent toward a more robust strategy to accomplish the task.
52

---

Published as a conference paper at ICLR 2026
Navigate to the cheapest controller in the "Vidco gaming" category from Ohio or Pennsylva- nia that is predominantly white
What ace you looking for today?
Pennyyhonds
Pensyyorts
click [56]
click [36]
click [17)
Phangfrua
Sal 20 Hightep
15.00
stop Imax steps
reachad)
click (19)
click [25)
stop
go_back
EVALUATION: PARTIAL SUCCESS
FEEDBACK: You have successfully filtered by category, searched for "white controller", and sorted by price in Pennsylvania. However, you also need to check for cheapest white controller in Ohio. Go back to the search results page and filter by Ohio as well. Then, compare the cheapest results from both states to find the overall cheapest one.
Post-feedback: stop + back
Figure 16:
Online feedback in VisualWebArena. An initially correct execution is rejected by the verifier and turns into failure. (Partial trajectory shown due to space constraints.)
53

---

Published as a conference paper at ICLR 2026
G.4 ONLINE VERIFICATION ROBOT MANIPULATION
Step: 0/700
Step: 260/700 700
Stepe 520/700
Fwil une. The Loshapeel wall hole woodon the wehaped cook hanging not complctad the predictedd actiona.
Bassed the imsele
insaited
pencilll
wooden lock, the tool picked up and hrund on the L. L-ataped structure. Thorefora, a, the cask ls completed described, labetted
hwo
not picked up hunn on the designated Shope hock or holder.
Since the tasst completed the swoond baok not, the whste clow prodiction. Thercfore, **tallure".
Figure 17: Verifier output for replanning and validation the robomimic tool-hang task.
54


Reviews:
[{"review_id": "2pl9gAacSI", "reviewer": "Reviewer_187k", "summary": "This paper identifies \"agreement bias\" as a critical failure mode in Multimodal Large Language Models (MLLMs) when they are used as verifiers for agent trajectories. The authors find that MLLMs tend to favorably evaluate flawed agent behavior, even generating rationalizations for it, despite possessing strong, human-aligned priors on correct task execution. They attribute this to a retrieval bottleneck. To address this, the paper introduces Self-Grounded Verification (SGV), a two-step prompting method. SGV first elicits the MLLM's broad priors about task completion independent of the agent's trajectory, and then conditions the MLLM on these self-generated priors to evaluate the candidate trajectory. Experiments across web navigation, computer use, and robotics benchmarks show that SGV significantly improves failure detection and accuracy, boosting the performance of agents in online supervision settings.", "questions": "How does the performance benefit of SGV change with model scale and capability? The paper shows it helps both weaker and stronger \"reasoning\" models, but does the relative gain (SGV vs. baseline) shrink as models become more capable? This would provide insight into whether SGV is fixing a fundamental reasoning flaw or a specific weakness of current models.", "limitations": "", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1. This work identifies a significant and practical problem, the \"agreement bias\" of MLLM verifiers. This is an important contribution as these verifiers are increasingly proposed for data filtering, self-refinement, and online agent guidance.\n2. The paper demonstrates strong empirical results, particularly in improving the True Negative Rate (failure detection). This is a crucial metric that is more informative than overall accuracy for this problem, as the primary goal of a verifier is to catch flawed behavior.\n3. The method is validated across a diverse and challenging set of multimodal environments (VisualWebArena, OSWorld, robomimic) and application settings (offline evaluation, self-refinement, and online supervision), which strengthens the generality of the claims.", "weaknesses": "1. The core mechanism of SGV—generating \"broad priors\" in Step 1 independent of the agent's trajectory —may be a significant flaw. By being ungrounded from the specific context of the agent's current state, these priors may be overly generic or common sense hallucinations that are irrelevant to the task at hand. This could lead the verifier to be \"overly strict,\" unfairly penalizing valid or creative solutions that deviate from the generic script, a failure mode the authors acknowledge. The paper lacks a sufficient analysis of when these broad priors are helpful versus when they are harmful.\n2. The proposed solution is a heuristic-driven prompting technique. While it shows good results, it is not clear how the method will scale or interact with future model development. The paper claims the issue is a \"retrieval bottleneck\", but it is equally plausible that agreement bias is an artifact of current alignment techniques or context-window management. The paper does not provide evidence to suggest whether SGV is a durable solution or a temporary patch for a flaw that might be solved more fundamentally by future models, rendering the heuristic obsolete.\n3. The evaluation of the online supervision setting seems arbitrary in its implementation. For instance, in OSWorld, the verifier is called \"every 5 steps\". There is no justification for this hyperparameter, and it glosses over the significant trade-off between verification frequency (and thus, token/compute cost) and the ability to catch errors in real-time.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "UXXRBVFT74", "reviewer": "Reviewer_dsJd", "summary": "This paper identifies “agreement bias,” i.e., a preference of verifiers to return positive results, as a key issue for agent tasks. They then propose a simple method in which a verifier first proposes a task-specific rubric and scores trajectories according to this rubric in order to mitigate agreement bias. Finally, the paper shows that these improved verifiers can be used to improve model performance on VisualWebArena, OSWorld, and the robotic manipulation task robomimic.", "questions": "N/A", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. This paper proposes a straightforward extension of the idea in Pan, et al. (2024) which shows that automatic evaluators can be used to improve the performance of web navigation and device control agents at training or inference time\n\n2. The paper makes a compelling case that models exhibit agreement bias when evaluating agent trajectories, i.e., a bias toward positive labels (Table 1a). The paper also clearly shows that its method leads to a reduction in agreement bias (Table 1b), and that this method works across a wide range of verifier models\n\n3. Most importantly, the paper shows that stronger verifiers are more useful, e.g., at improving agents at inference time using methods like Reflexion (Figure 2)", "weaknesses": "1. It would be nice to see some comparison of the proposed method with simpler strategies, e.g., different prompts to the verifier model, or prompting models to generate confidences and applying Platt scaling\n\n2. The paper could benefit from an additional round of proofreading. For example:\n     \n   - Line 101: missing a period\n   - Line 221: “Table Table 7” -> “Table 7”\n   - Line 263: broken reference (“??”)\n   - Line 1234: “AgentRewarBench” -> “AgentRewardBench”\n   - \\citet should be replaced with \\citep in many places", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "nT2uJTmUNp", "reviewer": "Reviewer_twch", "summary": "This paper 1) identifies \"agreement bias\", a tendency for MLLMs to inappropriately favor agent trajectories in their context window, as a critical limitation for MLLM-based verifiers, and 2) proposes Self-Grounded Verification (SGV), a two-step method that retrieves task priors first then evaluates trajectories against them, achieving up to 20pp gains in verification accuracy and setting a new SOTA on VisualWebArena.", "questions": "Address the weaknesses.", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. Originality: Framing \"agreement bias\" as a distinct limitation from self-bias and targeting it via self-generated priors is a novel angle for MLLM verifiers.\n2. Quality: Experiments use diverse benchmarks (1,200+ tasks) and models, ensuring results are generalizable rather than model-specific.\n3. Clarity: The SGV method is described simply, with step-by-step breakdowns and concrete examples (e.g., Figure 5) making it easy to follow.\n4. Significance: Improving MLLM verifier reliability directly benefits downstream tasks like agent training, data filtering, and real-time supervision, a key for deploying AI agents safely.", "weaknesses": "1. SGV does not address underlying vision-language flaws (e.g., Figure 7’s counting error), and the paper lacks discussion on combining SGV with specialist models for fine-grained perception.\n2. Current studies primarily focus on moderate-length trajectories (e.g., \"We set the maximum number of steps to 30\"). However, the scalability of SGV to extremely long sequences remains unclear. Such sequences are common in computer usage scenarios, and the context window pressure under extremely long sequences may cause biases to reemerge.\n3. The ablation on SGV’s prompt design (Appendix B.6) is limited. more tests on prior generation diversity (e.g., multiple priors vs. single) would strengthen claims about SGV’s mechanism.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "yctUUAnlcC", "reviewer": "Reviewer_Uw7g", "summary": "This work investigates the issue of agreement bias in vLLMs when used as evaluators. Specifically, vLLMs are employed to assess intermediate steps in multimodal agent tasks, serving as a form of reward or signal that can facilitate test-time scaling. However, the authors find that vLLMs may generate chains of thought to rationalize potentially flawed actions, even when their evaluative knowledge is aligned with human judgment. To mitigate this issue, the authors propose having the vLLM first generate a plausible prediction of the next state before conducting evaluation, which effectively reduces agreement bias. The problem is studied across domains such as web agents, GUI agents, and robotics.", "questions": "Please check the weaknesses section.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The research question is both interesting and important, though the phenomenon of agreement bias and the proposed solution might appear somewhat intuitive or straightforward. The experiments are comprehensive and the results are solid. The paper is also very well written — I appreciate that the abstract clearly conveys the main takeaways.", "weaknesses": "A critical problem remains: I’m particularly curious whether you have any further experiments and analysis on the “generating chains of thought to rationalize flawed behavior” aspect. You also claim that MLLMs exhibit strong, human-aligned priors on desired behavior — so at which reasoning step exactly does the failure occur? Is this bias intrinsic to the model itself, and beyond prompting strategies, are there training or sampling techniques that could help mitigate this issue?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)