Title: How Brittle is Agent Safety? Rethinking Agent Risk under Intent Concealment and Task Complexity
Primary area: alignment, fairness, safety, privacy, and societal considerations
Keywords: Agent Safety, Holistic Harms, Safety Benchmark

Abstract: Current safety evaluations for LLM-driven agents primarily focus on atomic harms, failing to address sophisticated threats where malicious intent is concealed or diluted within complex tasks. 
We address this gap with a two-dimensional analysis of agent safety brittleness under the orthogonal pressures of intent concealment and task complexity. To enable this, we introduce OASIS (Orthogonal Agent Safety Inquiry Suite), a hierarchical benchmark with fine-grained annotations and a high-fidelity simulation sandbox. Our findings reveal two critical phenomena: safety alignment degrades sharply and predictably as intent becomes obscured, and a “Complexity Paradox” emerges, where agents seem safer on harder tasks only due to capability limitations. By releasing OASIS and its simulation environment, we provide a principled foundation for probing and strengthening agent safety in these overlooked dimensions.

TL;DR: Agent safety is brittle, eroding with intent concealment and masked by capability limits (Complexity Paradox), demanding multi-dimensional assessment for robust alignment.