Title: Let's Think in Two Steps: Mitigating Agreement Bias in MLLMs with Self-Grounded Verification
Primary area: foundation or frontier models, including LLMs
Keywords: Verifiers, Verification, Digital Agents, Web Agents, GUI Agents, Robotics, Multimodal Large Language Models, Test Time Scaling, WebArena, OSWorld

Abstract: Verifiers--functions assigining rewards to agent behavior--have proven effective to guide test-time scaling and reinforcement learning, driving progress in domains such as math, code, and board games. However, extending these gains to domains without clear-cut success criteria (e.g., computer use) remains a challenge: while humans can recognize suitable outcomes, translating this intuition into scalable rules is non-trivial. Multimodal Large Language Models (MLLMs) emerge as a promising solution, given their world knowledge, human-preference alignment, and reasoning skills. We evaluate MLLMs as verifiers across web navigation, computer use, and robotic manipulation, and identify a critical limitation: a strong tendency to favorably evaluate agent behavior, often producing chains of thought to rationalize flawed behavior, a phenomenon we term agreement bias. We show that this bias persists across a range of MLLMs, reasoning models, and test-time scaling techniques, and poses risks to several methods relying on MLLM evaluations. Notably, it occurs despite MLLMs showing strong, human-aligned priors on desired behavior, suggesting a knowledge extraction bottleneck unresolved by existing techniques. We discuss how to mitigate this bias and introduce Self-Grounded Verification (SGV), a simple two-step framework that leverages MLLMs' knowledge and reasoning by modulating unconditional and conditional generation. In SGV: (i) the MLLM first generates priors conditioned only on minimal context to frame the task, and (ii) evaluates candidate trajectories conditioned on self-generated priors. SGV improves both accuracy and failure-detection rates by up to 20 percentage points, and enables MLLMs to provide real-time supervision, boosting task completion for a GUI specialist in OSWorld, a diffusion policy in robomimic, and a ReAct agent in VisualWebArena--establishing a new state of the art on the benchmark. Finally, we release an updated version of the (Visual)WebArena benchmark featuring more human-aligned evaluators, improved environment parallelization with higher execution fidelity, and runtime speedup of over 10x, thereby facilitating the development of digital agents.