Visual Backdoor Attacks on MLLM Embodied Decision Making via Contrastive
  Trigger Learning
Authors: name: Qiusi Zhan, name: Hyeonjeong Ha, name: Rui Yang, name: Sirui Xu, name: Hanyang Chen, name: Liang-Yan Gui, name: Yu-Xiong Wang, name: Huan Zhang, name: Heng Ji, name: Daniel Kang

Multimodal large language models (MLLMs) have advanced embodied agents by
enabling direct perception, reasoning, and planning task-oriented actions from
visual inputs. However, such vision driven embodied agents open a new attack
surface: visual backdoor attacks, where the agent behaves normally until a
visual trigger appears in the scene, then persistently executes an
attacker-specified multi-step policy. We introduce BEAT, the first framework to
inject such visual backdoors into MLLM-based embodied agents using objects in
the environments as triggers. Unlike textual triggers, object triggers exhibit
wide variation across viewpoints and lighting, making them difficult to implant
reliably. BEAT addresses this challenge by (1) constructing a training set that
spans diverse scenes, tasks, and trigger placements to expose agents to trigger
variability, and (2) introducing a two-stage training scheme that first applies
supervised fine-tuning (SFT) and then our novel Contrastive Trigger Learning
(CTL). CTL formulates trigger discrimination as preference learning between
trigger-present and trigger-free inputs, explicitly sharpening the decision
boundaries to ensure precise backdoor activation. Across various embodied agent
benchmarks and MLLMs, BEAT achieves attack success rates up to 80%, while
maintaining strong benign task performance, and generalizes reliably to
out-of-distribution trigger placements. Notably, compared to naive SFT, CTL
boosts backdoor activation accuracy up to 39% under limited backdoor data.
These findings expose a critical yet unexplored security risk in MLLM-based
embodied agents, underscoring the need for robust defenses before real-world
deployment.