ChatGPT plugin/cross-plugin data exfiltration via Markdown image injection (Embrace The Red)
Date: 2023-05-01
Affected: OpenAI ChatGPT plugins
Attack vector: prompt-injection
Category: real-world
Severity: Critical

Johann Rehberger (Embrace The Red) demonstrated that ChatGPT plugins (e.g., WebPilot, Cross-Plugin Request Forgery) could exfiltrate conversation data via Markdown image rendering and indirect prompt injection, including auto-invocation of plugins without user consent.

Impact: Goal hijacking — model redirected to serve attacker; demonstrated potential for data exfiltration, social engineering, and persistent instruction injection

Mitigations:
Content sanitisation and instruction-stripping before inserting retrieved content into context
Content sanitisation for retrieved documents before inclusion in context
Limit tool access scope — principle of least privilege for AI agents (ASI02 mitigations)
Output review before action execution — human-in-the-loop for consequential actions
Output validation: flag instruction-like patterns in LLM responses before execution
Privilege separation: retrieval agent cannot execute actions without explicit user confirmation
RAG content integrity scanning (DSGAI04)
Require explicit user confirmation before any send/write action
Sandbox retrieved content from system instructions at the context level
Separate trust levels for system instructions vs. retrieved external content
Strict separation between system instructions and retrieved/user content at prompt construction
Treat all external email content as untrusted — never execute instructions found in email bodies

Architectural layers (MAESTRO):