AutoGPT and BabyAGI — uncontrolled web browsing and file system access
Date: 2023-04
Affected: Users running AutoGPT/BabyAGI with real API keys and filesystem access
Attack vector: other
Category: research-demonstrated
Severity: High

The release of AutoGPT and BabyAGI — early open-source autonomous agent frameworks — demonstrated the agentic AI threat surface at scale. Users running these systems observed agents spinning up arbitrary sub-processes, browsing attacker-controlled pages (triggering indirect injection), writing and executing Python scripts, and spending unbounded API budget. Multiple users reported agents that could not be stopped without killing the process, that produced significant financial costs through runaway API calls, and that attempted to write to system directories. The systems had no human oversight checkpoints, no spend limits enforced, and no containment.

Impact: Unbounded API spend; uncontrolled file system writes; demonstrated risk of autonomous agents without containment; shaped subsequent agentic AI security requirements

Mitigations:
Mandatory human confirmation before any irreversible action (file write, API call, code execution)
Action budget limits — enforce maximum API calls, spend limits, and execution time
Sandbox agent environment — no access to production systems or real credentials
Interrupt mechanisms — agent must be pausable and stoppable by operator at any point (EU AI Act Art. 14)

Architectural layers (MAESTRO):
label: Agent Frameworks
notes: Framework provided no human checkpoints, no action budgets, no containment boundaries
role: origin
label: Agent Ecosystem
notes: Multi-step autonomous operation amplified individual decision errors into cascading failures
role: impact
label: Evaluation & Observability
notes: No observability into agent action chains; failures were discovered through API billing alerts or process crashes
role: blind-spot