Title: DRIFT: Dynamic Rule-Based Defense with Injection Isolation for Securing LLM Agents
Venue: NeurIPS 2025
Accepted: yes
Keywords: Prompt Injection Attack

Defense

LLM Agent
Abstract: Large Language Models (LLMs) are increasingly central to agentic systems due to their strong reasoning and planning capabilities. By interacting with external environments through predefined tools, these agents can carry out complex user tasks. Nonetheless, this interaction also introduces the risk of prompt injection attacks, where malicious inputs from external sources can mislead the agent’s behavior, potentially resulting in economic loss, privacy leakage, or system compromise.
System-level defenses have recently shown promise by enforcing static or predefined policies, but they still face two key challenges: the ability to dynamically update security rules and the need for memory stream isolation. To address these challenges, we propose DRIFT, a Dynamic Rule-based Isolation Framework for Trustworthy agentic systems, which enforces both control- and data-level constraints.
A Secure Planner first constructs a minimal function trajectory and a JSON-schema-style parameter checklist for each function node based on the user query. A Dynamic Validator then monitors deviations from the original plan, assessing whether changes comply with privilege limitations and the user's intent. Finally, an \textit{Injection Isolator} detects and masks any instructions that may conflict with the user query from the memory stream to mitigate long-term risks.
We empirically validate the effectiveness of DRIFT on the AgentDojo and ASB benchmark, demonstrating its strong security performance while maintaining high utility across diverse models—showcasing both its robustness and adaptability. The code is released at https://github.com/SaFoLab-WISC/DRIFT.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors present a new method, DRIFT, for defending against prompt injections of LLM agents. DRIFT is composed of a secure planner, a dynamic validator and an injection isolator.  The method beats SOTA on AgentDojo in preserving utility of the agents and is close to SOTA in preventing attacks.\n\nSTRENGTHS AND WEAKNESSES:\n- Clarity: \n  - The paper is clear, well-written and well-organized in general. I have below asked a question about a slightly unclear experimental setup.\n- Originality: \n  - The paper present a novel, practical method to tackle prompt injection attacks. It beats the current SOTA, CaMeL in preserving the utility of the agents. While none of the elements are very surprising, they fit together naturally. The dynamic validator seems like a good design choice to deal with the restrictiveness of advance planning, and preventing the injections from entering the context via the injection isolator also seems very helpful.\n- Quality:\n  - The experiment conducted in the paper are scientifically sound and support the claims well. The paper also presents some interesting ablations and case studies demonstrating how each element is helpful, which helps intuitive understanding of the metho and makes the paper easy to understand.\n  - The authors however did not perform any red-teaming against their method, which might have introduced new points of failure, for example by attackers fooling the isolator, also an LLM based systems. It is possible that targeted attacks to these components significantly reduce the efficacy of the method, and a discussion of possible vulnerabilities and failure points would be helpful.\n\n- Significance:\n  - The paper improves the current best solution to a very timely and pressing problem. Due the utility boost and simple setup of the method is likely to be adopted by the community and in real-life use-cases. \n  - While there is a significant boost in the utility scores, the ASR against the method is also slightly higher compared to the 0 in CaMeL. Why this is the case and whether the method has a vulnerability against some particular attack isn't discussed. This and the results of red teaming the method might reduce the significance of this work.\n\nQUESTIONS:\n- Which attacks were used in the experiments, one of the attacks included in Agentdojo or a custom one? I would be interested to see how the method holds up under targeted red-teaming of its components eg manipulating the isolator by adjusting the attack templates, as they it is LLM based and vulnerable to prompt injections itself.\n\n- It would also be interesting to see what caused the non-zero score on the ASR and whether thats a fundamental vulnerability of the system or a rare failure.\n\n- I am a bit confused by the setup with the experiment on Figure 4, do all the agents use the same model for the drift components eg a Claude agent would use a Claude planner, validator and isolator, or  would it use Qwen components and it's just the agent being different?\n\n- I would also be interested whether the authors think the validator could handle very open-ended and unpredictable plans, for example \"read the email from my boss and do what he asks\". In Figure 6, there is already one forking that seems to be fine, I am wondering how far that can be pushed.\n\n- Finally, it would be great to discuss practicality in adding DRIFT to real-life deployments, such as compute or api cost overhead and latency.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-06-29"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper proposes DRIFT, a system-level dynamic rule-based isolation framework to defend LLM agents against prompt injection attacks. DRIFT combines a Secure Planner, Dynamic Validator, and Injection Isolator to enforce control- and data-level constraints dynamically while maintaining utility during agentic operations. The method is evaluated on AgentDojo with GPT-4o, Claude, Qwen2.5 and shows strong reductions in ASR while maintaining or improving utility.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**\n- The paper is well-written and easy to follow.\n- It provides detailed evaluations across multiple models and settings.\n- The paper addresses an important and timely issue, namely securing LLM agents against prompt injection attacks in tool-augmented agent settings.\n\n**Weaknesses**\n- Prior works[1-3] on dynamic policy and context management in LLM agents are not discussed, limiting clarity on how DRIFT truly advances beyond previous methods. The authors should conduct comparative experiments or discuss the differences. \n- The proposed approach relies heavily on experiments within AgentDojo, which, while comprehensive, are still limited compared to real-world web environments [4-6] and multimodal agent settings. \n- The proposed DRIFT introduces additional modules. The overhead analysis (resource consumption) of DRIFT is missing, leaving its practical deployment cost unclear.\n\n[1] Tianneng Shi, Jingxuan He, Zhun Wang, Linyu Wu, Hongwei Li, Wenbo Guo, and Dawn Song.\nProgent: Programmable privilege control for llm agents, 2025. 1, 2, 13\n\n[2] Yifan Song, Weimin Xiong, Dawei Zhu, Cheng Li, Ke Wang, Ye Tian, and Sujian Li. Restgpt: Connecting large language models with real-world applications via restful apis. CoRR, abs/2306.06624, 2023. 13\n\n[3] Zhou, Andy, et al. \"Language agent tree search unifies reasoning acting and planning in language models.\" arXiv preprint arXiv:2310.04406 (2023).\n\n[4] Mind2Web: Xiang Deng, Yu Gu, Boyuan Zheng, et al. Mind2Web: Towards a generalist agent for the web. NeurIPS, 2023.\n\n[5]. VisualWebArena: Jing Yu Koh, Robert Lo, Lawrence Jang, et al. VisualWebArena: Evaluating multimodal agents on realistic visual web tasks. ACL, 2024.\n\n[6] . WebArena: Shuyan Zhou, Frank F. Xu, Hao Zhu, et al. WebArena: A realistic web environment for building autonomous agents. ICLR, 2024.\n\nQUESTIONS:\nRefer to Weakness\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper presents DRIFT, which addresses prompt injection attacks in LLM-based agents through system-level defenses. The framework consists of three key components: (1) a Secure Planner that constructs minimal function trajectories and JSON-Schema parameter checklists based on user queries, (2) a Dynamic Validator that monitors deviations from the original plan and assesses compliance with privilege limitations and user intent, and (3) an Injection Isolator that detects and masks conflicting instructions from memory streams to mitigate long-term risks. The authors evaluate DRIFT on the AgentDojo benchmark across multiple LLMs (GPT-4o, GPT-4o-mini, Claude-3.5-sonnet, Claude-3-haiku, and Qwen2.5-7B), demonstrating significant security improvements.\n\nSTRENGTHS AND WEAKNESSES:\n**Strength**\n\n1. Practicality of System\n\nThe paper addresses a critical real-world problem with a comprehensive system-level approach. Unlike previous methods that rely on static policies (CaMeL) or complete isolation (IsolateGPT), DRIFT achieves an effective balance between security and functionality through dynamic policy adaptation.\n\n2. Dynamic Policy Innovation\n\nThe privilege-based categorization (Read/Write/Execute) borrowed from operating systems and applied to LLM agents is intuitive and effective. The dynamic constraint updating mechanism addresses the rigidity issues of prior static approaches.\n\n\n**Weakness**\n\n1. One of the main contribution of this paper is to propose dynamic rule-based defense. However, the dynamic validator is just an LLM without specific attribute, and the rule is simple 3-categorized with Read, Write, and Execute.  I think it is very good idea in engineering perspective, but the originality is restricted in research perspective.\n\n2. Moreover, the multi-LLM architecture introduces significant complexity and computational overhead. The paper doesn't provide detailed analysis of latency implications or resource consumption compared to simpler baselines. \n\n3. Despite claims of automation on security policy, the framework still requires users to define security policies and maintain them, which could be burdensome in practice. The paper doesn't adequately address how policies scale across diverse applications.\n\n3. The evaluation is limited to the AgentDojo benchmark, which may not fully represent the diversity of real-world agentic systems and attack scenarios. More comprehensive evaluation across different domains and attack types [1] would strengthen the claims. Moreover, I have a concern on injection Isolator. It can possibly remove safe contents. Do you have any prevention for it?\n\n[1] Hanrong Zhang, Jingyuan Huang, Kai Mei, Yifei Yao, Zhenting Wang, Chenlu Zhan, Hongwei Wang, and Yongfeng Zhang. Agent security bench (ASB): Formalizing and benchmarking attacks and defenses in LLM-based agents. In The Thirteenth International Conference on Learning Representations, 2025.\n\nQUESTIONS:\nPlease refer to weakness I mentioned above\n\nLIMITATIONS:\nThe main limitation of this work is lack of discussion and consideration on scalability and evaluation scope.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 5, "CLARITY": 4, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a system-level defense framework designed to secure\nLLM-based agentic systems against prompt injection attacks. It includes a\nmulti-layered defense mechanism combining a Secure Planner, Dynamic Validator,\nand Injection Isolator to enforce both control- and data-level constraints and\ndynamically update policies during execution. Experiments on the AgentDojo\nbenchmark show the effectiveness of the proposed method.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths\n\n1. The problem studied in this paper is timely and interesting. \n\n2. Defending prompt injection attacks in LLM agent is an important problem.\n\n3. Both control-level and data-level constraints in the proposed system level defense.\n\nWeaknesses\n\n1. DRIFT assumes that the Secure Planner can extract a minimal and correct\nfunction trajectory from the initial user query in most cases. However, for\ncomplex tasks—especially those involving exploration, tool trial-and-error, or\nmultiple planning-reflection-execution cycles, perfectly extracting such a\nminimal trajectory can be highly non-trivial. The paper does not discuss how\nDRIFT performs when the initial planning is incomplete, incorrect, or\ndynamically evolves during task execution.\n\n2. The entire framework relies heavily on prompt-based logic to define\nconstraints, assign privileges, and detect injections. However, there is no\nanalysis of how sensitive the system is to variations in these prompts, nor is\nthere any ablation on their robustness. This raises questions about the\ngeneralizability and reliability of DRIFT across different data distributions.\n\n3. Many components in the proposed framework introduce extra LLM calls. Yet, the\npaper provides no measurements or discussion of the latency, compute overhead,\nor token consumption introduced by these components. This is particularly\ncritical for real-world deployment in multi-turn, high-complexity tasks, where\nfrequent function calls are the norm.\n\n4. Despite its focus on security, the paper lacks any rigorous stress test or\nadversarial evaluation. There is no analysis of DRIFT's behavior under adaptive\nattacks where the attackers are aware of DRIFT and aim at bypassing it.\n\n5. All experiments are conducted solely on the AgentDojo benchmark. While\nAgentDojo is a reasonable dataset, solely using this dataset limits the\ngeneralizability of the findings.\n\nQUESTIONS:\nn/a\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-06"}