Why Are Web AI Agents More Vulnerable Than Standalone LLMs? A Security
  Analysis
Authors: name: Jeffrey Yang Fan Chiang, name: Seungjae Lee, name: Jia-Bin Huang, name: Furong Huang, name: Yizheng Chen

Recent advancements in Web AI agents have demonstrated remarkable
capabilities in addressing complex web navigation tasks. However, emerging
research shows that these agents exhibit greater vulnerability compared to
standalone Large Language Models (LLMs), despite both being built upon the same
safety-aligned models. This discrepancy is particularly concerning given the
greater flexibility of Web AI Agent compared to standalone LLMs, which may
expose them to a wider range of adversarial user inputs. To build a scaffold
that addresses these concerns, this study investigates the underlying factors
that contribute to the increased vulnerability of Web AI agents. Notably, this
disparity stems from the multifaceted differences between Web AI agents and
standalone LLMs, as well as the complex signals - nuances that simple
evaluation metrics, such as success rate, often fail to capture. To tackle
these challenges, we propose a component-level analysis and a more granular,
systematic evaluation framework. Through this fine-grained investigation, we
identify three critical factors that amplify the vulnerability of Web AI
agents; (1) embedding user goals into the system prompt, (2) multi-step action
generation, and (3) observational capabilities. Our findings highlights the
pressing need to enhance security and robustness in AI agent design and provide
actionable insights for targeted defense strategies.