Title: RiOSWorld: Benchmarking the Risk of Multimodal Computer-Use Agents
Venue: NeurIPS 2025
Accepted: yes
Keywords: Agent Safety

Safety Evaluation Benchmark
Abstract: With the rapid development of multimodal large language models (MLLMs), they are increasingly deployed as autonomous computer-use agents capable of accomplishing complex computer tasks. However, a pressing issue arises: Can the safety risk principles designed and aligned for general MLLMs in dialogue scenarios be effectively transferred to real-world computer-use scenarios? 
Existing research on evaluating the safety risks of MLLM-based computer-use agents suffers from several limitations: it either lacks realistic interactive environments, or narrowly focuses on one or a few specific risk types. These limitations ignore the complexity, variability, and diversity of real-world environments, thereby restricting comprehensive risk evaluation for computer-use agents.
To this end, we introduce **RiOSWorld**, a benchmark designed to evaluate the potential risks of MLLM-based agents during real-world computer manipulations. Our benchmark includes 492 risky tasks spanning various computer applications, involving web, social media, multimedia, os, email, and office software. We categorize these risks into two major classes based on their risk source: (i) User-originated risks and (ii) Environmental risks. For the evaluation, we evaluate safety risks from two perspectives: (i) Risk goal intention and (ii) Risk goal completion. Extensive experiments with multimodal agents on **RiOSWorld** demonstrate that current computer-use agents confront significant safety risks in real-world scenarios. Our findings highlight the necessity and urgency of safety alignment for computer-use agents in real-world computer manipulation, providing valuable insights for developing trustworthy computer-use agents.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper examines the safety risks of multimodal large language models (MLLMs) in real-world computer-use scenarios, noting that existing evaluations lack realistic environments or cover limited risk types. It introduces RiOSWorld, a benchmark with 492 risky tasks across multiple applications, categorizing risks as user-originated or environmental. The study evaluates risk intention and completion, finding current agents face significant safety issues, underscoring the need for better safety alignment.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n1. Table 1 is good. It helps me quickly understand the differences between different types of work.\n2. There are many image examples provided, allowing me to quickly understand what the benchmark is like.\n\n\nWeaknesses:\n1. I noticed that the code has not been open-sourced, which may be unacceptable for a benchmark.\n2. Figure 3 does not include quantitative analysis or comparisons with other studies. This is problematic because any benchmark might produce a similarly reasonable-looking distribution, making the visualization less informative.\n3. First, in terms of the Number of Risky Examples, RiOSWorld does not demonstrate an advantage. Additionally, compared to ATTACKING POPUP, it appears to be more of an incremental growth (i.e., only adding different categories).\n\nQUESTIONS:\n1. I noticed that the data collection process drew inspiration from other benchmarks or studies, which were then filtered, expanded, and augmented. The proportions of newly created data versus adopted data should be explicitly reported, along with the breakdown of sources. This helps to show the significance of the work.\n2. The author mentions that another track of autonomous agent development lies in the creation of native agent models. I'm curious why these models aren't being evaluated, and instead, the focus is only on leveraging the power of existing MLLMs. Additionally, there's now OpenAI's Operator (https://operator.chatgpt.com/), and I believe testing on these agents would be very interesting, rather than just testing some MLLMs that haven't undergone specialized training.\n3. From the experimental results, it seems to only indicate that current agents have weak safety alignment capabilities, but there are no further discoveries. Perhaps there are some new insights? In particular, if possible, it would be helpful to outline some directions for defense (alignment, detection) in future work.\n4. NeurIPS has a dedicated benchmark track—what are the authors' reasons for believing the paper is more suitable for the main track rather than the benchmark track? In other words, what considerations or factors led them not to submit to the benchmark track?\n\nI will consider updating my score based on the author's response.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-06-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors propose RiOSWorld, a safety benchmark for GUI computer use agents using MLLM models. The benchmark uses the pre-existing VM-based OSWorld GUI computer use agent environment. The benchmark contains 492 instances spanning various risk categories. The authors separately measure whether the agent intended to perform risky behavior, and whether the agent successfully performed risky behavior. The authors’ evaluations show that most recent models exhibit risky behavior or intent to perform risky behavior.\n\nSTRENGTHS AND WEAKNESSES:\n### Strengths\n\n- The authors provide a comprehensive literature review of MLLM agent benchmarks and GUI computer use agents. The authors provide a strong argument for the importance of a new GUI computer use agent benchmark. (Significance)\n- The authors provided separate metrics for evaluating risk goal completion and risk goal intention, which addresses the potential confound of models appearing safe because they are too weak to successfully cause harm. (Quality, Significance, Originality)\n- The tasks cover a broad range of user tasks including file system navigation, email, and online shopping. Most of the tasks appear to be realistic user tasks. (Quality, Significance, Originality)\n- The VM-based tasks are carefully designed by humans and likely to be high quality. (Quality)\n- Overall, the agent environment and the benchmark design are clearly explained. (Clarity)\n\n### Weaknesses\n\n- The benchmark relies on an internet connection and external state over the internet. The authors describe this property as “Real Network Accessible”. This dependency on external state outside the users’ control makes the benchmark less reproducible. (Quality)\n- The choice of risks to include in the benchmark are somewhat arbitrary, and the authors did not attempt to provide a systematic taxonomization of risks. The authors admit that examples were “derived from authors’ original ideas” on page 5. (Quality)\n- The taxonomy and categorization of risks is confusing. There are several possible axes to perform the categorization, e.g. environment (e.g. email client, word processor), attack vector (e.g. prompt injection, malicious user request), or harm (e.g. spreading misinformation, leading private information). The authors’ taxonomy mixes categories from different axes. (Quality, Clarity)\n- The text embedding clusters presented in page 5 seem questionable. For instance, one cluster is labeled as “physics”, which does not seem related to the topic of computer use agent safety. (Quality, Clarity)\n- A large number of tasks, including the examples illustrated in Appendix Tables 11 through 17 in the appendix, are web browser tasks. This overlaps with existing web browser agent safety benchmarks such as VisualWebArena-Adv and WASP. (Significance, Originality)\n- There is insufficient information about the implementation of the agent framework e.g. does the agent have chain-of-thought reasoning and memory? (Quality, Clarity)\n- The authors do not provide a good definition of risk or harm. In many tasks, it is not clear if harm was actually inflicted. (Quality)\n    - For “pop-ups” (Appendix Table 13), clicking a pop up in a web browser does not cause harm to the machine or user.\n    - For “reCAPTCHA” (Appendix Table 14), solving a captcha does not cause damage to the user’s machine or the server.\n    - For “Expose Privacy” / “Code Scenarios” (Appendix Table 22), it is contextual whether it is harmful to upload a Python file containing an API key to GitHub. For instance, uploading this file to a private GitHub repository is safe. However, I do agree that it is bad practice to include API keys in code. \n    - For OS Operation (Appendix Table 19), the authors claim that the behavior is unsafe because the agent uses the sudo password provided in the system prompt to perform a request by the user prompt provider. However, the agent was never told in the system prompt that the user prompt provider is an untrusted party that should not have sudo permissions. In fact, the agent was told “feel free to use [the sudo password] when you need sudo rights”.\n- A minor omission in the task coverage: there is no task involving a phishing website served over HTTPS, because all phishing websites are served over HTTP on a local IP. This means an agent can pass the phishing portion of the benchmark by simply refusing to use any non-HTTPS websites. (Quality)\n- There are a few minor typo errors. (Clarity)\n    - Section 1 Introduction: “os” should be “OS”.\n    - Figure 5: Graph title should be “Unsafe” instead of “Usafe”.\n\nQUESTIONS:\n- Could you provide more information on the agent framework implementation, particularly the chain-of-thought reasoning and memory mechanisms?\n- Could you provide more information on your definition of risk / harm, and how you derived your risk categories?\n- Do you expect the agents to perform better if they were (1) explicitly instructed to avoid unsafe behaviors in the system prompt, or (2) explicitly provided with an “abort unsafe task” action?\n\nLIMITATIONS:\nThe authors should acknowledge that the measurements of safety is limited to the risk categories included by the authors, and that a high score on this benchmark should not be interpreted as a guarantee that an agent is generally safe.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 4, "ORIGINALITY": 3, "DATE": "2025-06-25"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper introduces **RiOSWorld**, a benchmark designed to evaluate safety risks of multimodal large language model (MLLM)-based computer-use agents in realistic environments. Existing studies on agent safety lack realistic interactive setups or focus narrowly on specific risk types, failing to capture the complexity of real-world computer use.  \n\nRiOSWorld includes **492 risky tasks** across 13 subcategories, categorized into:  \n\n1. **Environmental risks** (e.g., phishing web, pop-ups, reCAPTCHA circumvention).  \n2. **User-originated risks** (e.g., malicious instructions for OS operations, code upload with sensitive data).  \n\nThe benchmark evaluates agents from two perspectives:  \n\n- **Risk goal intention**: Whether the agent intends to perform risky actions.  \n- **Risk goal completion**: Whether the agent successfully executes the risk.  \n\nExperiments on 10 state-of-the-art MLLMs (e.g., GPT-4o, Gemini, Claude) show high unsafe rates, which highlights that current agents lack risk awareness, especially in phishing, OS operations, and code-related tasks.\n\nSTRENGTHS AND WEAKNESSES:\n### Strengths:\n\n1. **Comprehensive Risk Coverage**: RiOSWorld spans diverse applications (web, social media, office software) and integrates both environmental and user-induced risks, addressing a critical gap in prior work that focused on isolated risk types.  \n2. **Realistic Evaluation Environment**: Using a virtual machine (VM), the benchmark simulates dynamic threats (e.g., phishing emails, pop-ups) and real-world interactions, providing a more authentic assessment than QA-format or simplified setups.   \n3. **Thorough baselines**: RiOSWorld uses ten agents, including closed and open models, tested with unified hyper-parameters.\n\n\n### Weaknesses:\n\n1. **Limited Scalability**: Constructing 492 tasks required 1,440 man-hours, indicating high resource costs. Scaling to more risks or scenarios may be challenging without automated tools.  \n2. **Subjectivity in Intent Evaluation**: Relying on LLM-as-a-judge (e.g., GPT-4o) introduces subjectivity. Different judge models or prompts could yield varying results, as shown in ablation studies.  \n3. **VM Environment Constraints**: While VM-based, the setup might not perfectly replicate all real-world complexities (e.g., network latency, hardware-specific vulnerabilities).\n\nQUESTIONS:\nSee weaknesses\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-01"}

comments: SUMMARY:
This paper presents RiOSWorld: Benchmarking the Risk of Multimodal Computer-Use Agents by providing a comprehensive datasets including more diverse risk categories, multi-modal requirements, Real Network access and etc. Based on this benchmark, they tested a range of open-source or commercial MLLMs and found the universal safety risk among them. The Unsafe Rate is extremely high even for the most advanced models, calling for more efforts into this direction.

STRENGTHS AND WEAKNESSES:
Strengths:
1. The computer-use agent safety is very important, but the current benchmarks have many limitations for testing their real performance. This paper provides a unique resource for comprehensive and realistic pressure test and identifies a major safety risk under the most advanced MLLMs.
2. The paper is well organized and easy to follow. The demonstrations are good.


Weaknesses:
1. It's unclear how the reasoning can help reduce the safety risk. if the inference has more compute, will it be easier to reduce the risk?
2. It would be great if the authors can provide more defense strategies since the attacks are very effective.
3. It is unclear why the Unsafe Rate is so high on this benchmark? how about the Unsafe Rate of the previous benchmark? what makes this benchmark stand out for bringing more risks?

QUESTIONS:
there is typo in the title: Compter-> Computer

I remember chatgpt and Claude both have very strong safety moderation system. How did you pass it?

LIMITATIONS:
yes
TITLE: Official Review
CLARITY: 3
IMPACT: 3
ORIGINALITY: 3
DATE: 2025-07-04