Typographic Visual Prompts Injection Threats in Cross-Modality Generation
Date: 2025-03
Affected: LLaVA, Qwen-VL, large VLMs
Attack vector: prompt-injection
Category: research
Severity: Medium

Cao et al. show that typographic visual prompts injected into images cause large VLMs (LLaVA-v1.6-72B, Qwen-v2.5-VL-72B) to follow attacker text instead of the user's instruction, with larger models being more vulnerable.


Mitigations:

Architectural layers (MAESTRO):