# RiOSWorld: Key Findings and Takeaways

## Paper Summary (arXiv:2506.00618)

RiOSWorld is the first comprehensive benchmark for evaluating the
**safety risks** of multimodal computer-use agents (CUAs). Unlike
existing benchmarks that focus on task completion ability, RiOSWorld
specifically tests whether agents can resist adversarial attacks
while operating a real desktop environment.

## Main Findings

### 1. All agents are vulnerable to some attack types

No tested agent achieved zero Harmful Action Rate across all
categories. Even the strongest models (GPT-4o, Claude-3.5-Sonnet)
were susceptible to:
- Popup attacks: Agents frequently clicked "OK" on fake dialogs
- Induced text: Agents followed misleading instructions injected
  into document content

### 2. Visual attacks are more effective than text-based

Attacks using visual elements (popup windows with official-looking
logos, fake CAPTCHAs) were more successful than plain-text phishing
emails. This suggests agents over-rely on visual cues and under-
analyze the semantic content.

### 3. OS-level awareness varies significantly

Agents with stronger language understanding (GPT-4o, Claude) were
better at refusing dangerous OS commands (rm -rf, disk formatting).
Vision-specialized agents (AGUVis, CogAgent) had higher HAR on OS
risk tasks because they focused on executing the visual instruction
without reasoning about its consequences.

### 4. Safety guidelines in prompts help but are insufficient

Adding explicit safety instructions to the agent's system prompt
reduced HAR by 15-20% on average, but did not eliminate risks.
Agents need both prompt-level and architecture-level safety measures.

### 5. Detection does not always lead to refusal

In many cases, agents detected something suspicious (high DR) but
still proceeded with the action (moderate HAR). The gap between
detection and refusal is an important area for future work.

## Implications for Agent Development

- CUA developers should implement multi-layer safety checks
- Visual grounding should be paired with semantic verification
- Agents need training data that includes adversarial scenarios
- Benchmark-driven safety evaluation should be standard practice