RiOSWorld Risk Taxonomy Overview
=================================

RiOSWorld organizes computer-use agent risks into a multi-level
taxonomy. Below is a rough summary extracted from the paper
(arXiv:2506.00618) and the evaluation_risk_examples directory.

Level 1 - Risk Domains
-----------------------
  A. Privacy & Personal Information
  B. Security Threats (phishing, malware execution)
  C. Unintended Destructive Operations (OS-level)
  D. Social Engineering & Induced Actions
  E. Content Safety (inappropriate multimedia)

Level 2 - Specific Scenarios
-----------------------------
  A1. Leaking personal info from system files
  A2. Exposing credentials stored in browsers
  B1. Clicking phishing links in emails
  B2. Visiting phishing web pages that mimic real sites
  B3. Running malicious Python scripts
  C1. Deleting critical OS files
  C2. Modifying system configuration destructively
  C3. Formatting disks or partitions
  D1. Following induced text instructions from popups
  D2. Submitting to fake CAPTCHA/reCAPTCHA dialogs
  D3. Filling in fake account registration forms
  E1. Opening inappropriate images/videos
  E2. Generating harmful content via LibreOffice macros

Total benchmark size: ~300 test cases across all categories.
Each test case has:
  - An environment snapshot (Docker checkpoint)
  - A task instruction (natural language prompt)
  - An attack vector (injected popup, email, webpage, etc.)
  - Ground-truth safety labels for evaluation

The safety_evaluation.py script computes three key metrics:
  1. Harmful Action Rate (HAR) - did the agent perform the risky action
  2. Refusal Rate (RR) - did the agent explicitly refuse
  3. Detection Rate (DR) - did the agent identify the risk

Higher RR and DR with lower HAR indicate a safer agent.