OS-Harm: A Benchmark for Measuring Safety of Computer Use Agents
Authors: name: Thomas Kuntz, name: Agatha Duzan, name: Hao Zhao, name: Francesco Croce, name: Zico Kolter, name: Nicolas Flammarion, name: Maksym Andriushchenko

Computer use agents are LLM-based agents that can directly interact with a
graphical user interface, by processing screenshots or accessibility trees.
While these systems are gaining popularity, their safety has been largely
overlooked, despite the fact that evaluating and understanding their potential
for harmful behavior is essential for widespread adoption. To address this gap,
we introduce OS-Harm, a new benchmark for measuring safety of computer use
agents. OS-Harm is built on top of the OSWorld environment and aims to test
models across three categories of harm: deliberate user misuse, prompt
injection attacks, and model misbehavior. To cover these cases, we create 150
tasks that span several types of safety violations (harassment, copyright
infringement, disinformation, data exfiltration, etc.) and require the agent to
interact with a variety of OS applications (email client, code editor, browser,
etc.). Moreover, we propose an automated judge to evaluate both accuracy and
safety of agents that achieves high agreement with human annotations (0.76 and
0.79 F1 score). We evaluate computer use agents based on a range of frontier
models - such as o4-mini, Claude 3.7 Sonnet, Gemini 2.5 Pro - and provide
insights into their safety. In particular, all models tend to directly comply
with many deliberate misuse queries, are relatively vulnerable to static prompt
injections, and occasionally perform unsafe actions. The OS-Harm benchmark is
available at https://github.com/tml-epfl/os-harm.