Title: CUARewardBench: Benchmark for Evaluating Reward Models on Computer-using Agent Trajectories
Primary area: datasets and benchmarks
Keywords: Computer-using Agent; Reward Models; Benchmark

Abstract: Computer-using agents (CUAs) enable task completion through natural interaction with operating systems and software interfaces. While script-based verifiers are widely adopted for evaluation, they suffer from limited scalability and inability to provide step-wise assessment. Reward models offer promising alternatives, but their effectiveness on CUA evaluation remains largely underexplored. To address this gap, we present CUARewardBench, comprising four key contributions: (1) First-ever Comprehensive CUA Reward Benchmark:* We introduce the first benchmark for evaluating both outcome reward models (ORM) and process reward models (PRM) on CUA tasks, enabling systematic assessment across trajectory-level and step-level evaluation. (2) Diverse and Representative Dataset: Our benchmark encompasses trajectories spanning 10 software categories and collected from 7 agent architectures with varying performance levels (25.9%-50.8% success rates), ensuring comprehensive coverage of CUA decision-making patterns. (3) Expert-Validated Annotations: All trajectories undergo rigorous expert annotation through carefully designed trajectory selection criteria, key step identification protocols, and systematic annotation standards. Expert annotations are validated through comprehensive cross-checking and quality control processes to ensure benchmark reliability and practical applicability. (4) Comprehensive Analysis and Insights: Through extensive experiments across 7 vision-language models and 3 prompt templates, we reveal critical limitations of current CUA RMs, including insufficient visual reasoning capabilities, knowledge deficiencies, and the superiority of general VLMs over specialized CUA models for reward evaluation.
Our findings provide practical guidance for future CUA RM development and highlight potentials for advancing evaluation of CUA models.

TL;DR: CUARewardBench: A Benchmark for Evaluating Reward Models on Computer-using Agent