Multimodal indirect injection — image-embedded instructions in GPT-4V
Date: 2023-10
Affected: GPT-4V; any multimodal LLM accepting image inputs — generalises to all vision-capable models
Attack vector: indirect-prompt-injection
Category: research-demonstrated
Severity: High

Following the release of GPT-4V (vision capabilities), researcher Riley Goodside and others demonstrated that adversarial instructions could be embedded in images and would be executed by the multimodal model as if they were text instructions. Text hidden in images (white text on white background, text in image metadata, instructions in image alt text) was interpreted and acted upon. This extended indirect prompt injection from text-only RAG pipelines to any multimodal input channel — photos, screenshots, scanned documents.

Impact: Extends indirect injection attack surface to all visual input channels; bypasses text-only input sanitisation; particularly dangerous for document processing pipelines

Mitigations:
Optical character recognition (OCR) preprocessing with adversarial text detection on all image inputs
Separate trust levels for user-provided images vs. system-provided images
Do not allow image content to influence tool invocations without explicit user confirmation
Multimodal content integrity scanning before processing (DSGAI09)

Architectural layers (MAESTRO):
label: Foundation Models
notes: Multimodal model processes image-embedded text as instructions — no visual/textual trust boundary distinction
role: origin
label: Data Operations
notes: Image content treated as trusted input without adversarial content scanning
role: propagation
label: Agent Frameworks
notes: Agentic deployments with vision input act on image-embedded instructions without user awareness
role: impact