Privacy in Action: Towards Realistic Privacy Mitigation and Evaluation for LLM -Powered Agents (2025)

Abstract: The increasing autonomy of LLM agents in handling sensitive communications, accelerated by Model Context Protocol (MCP) and Agent-to-Agent (A2A) frameworks, creates urgent privacy challenges. While recent work reveals significant gaps between LLMs’ privacy Q&A performance and their agent behavior, existing benchmarks remain limited to static, simplified scenarios. We present PrivacyChecker, a model-agnostic, contextual integrity based mitigation approach that effectively reduces privacy leakage from 36.08% to 7.30% on DeepSeek-R1 and from 33.06% to 8.32% on GPT-4o, all while preserving task helpfulness. We also introduce PrivacyLens-Live, transforming static benchmarks into dynamic MCP and A2A environments that reveal substantially higher privacy risks in practical. Our modular mitigation approach integrates seamlessly into agent protocols through three deployment strategies, providing practical privacy protection for the emerging agentic ecosystem. Our data and code will be made available at https://aka.ms/privacy_in_action .

## introduction agents powered by large language models (llms) demonstrate strong capabilities in generation, reasoning, planning, and tool use @xcite . they are rapidly transitioning from research prototypes to practical applications across domains such as customer service, healthcare, and personal assistance @xcite @xcite . this adoption is further accelerated by the emergence of agent-oriented protocols, notably the model context protocol (mcp) @xcite and the agent2agent protocol (a2a) @xcite , which facilitate integration of external tools and enable inter-agent communication. these advances are shaping a nascent agentic ecosystem, where agents autonomously employ tools and collaborate with one another to complete tasks with minimal human oversight. as agents begin to manage complex, end-toend tasks (e.g., flight booking, restaurant reservation) @xcite @xcite , privacy becomes a pressing concern: agents routinely access user documents, emails, and credentials, raising the risk of inadvertent data leaks or malicious misuse. in response, several privacypreserving techniques have been proposed to mitigate inappropriate disclosures of private information. early efforts focused on embedding privacyconscious instructions or chain-of-thought guidance within prompts @xcite . more recent approaches draw upon contextual integrity (ci) theory @xcite to inform mitigation strategies @xcite . building on this foundation and addressing a critical gap between llms' judgment and actions, we propose privacychecker, a ci-based, modular, and model-agnostic framework that identifies information flows and evaluates their appropriateness to dynamically guide agent behavior during inference. despite growing interest in mitigation, there exist limitations in current evaluation methods @xcite @xcite . most benchmarks are static and designed for single-agent settings. privacy assessments commonly rely on fixed question-answer probes or scripted dialogues, which fail to capture agent behavior in open-ended, real-world scenarios. while some recent efforts employ llm-based simulation (e.g., synthetic dialogues), they often overlook key dynamics such as evolving agent chains and inter-agent protocols. to address this gap, we propose transforming existing static benchmarks (e.g., privacylens @xcite ) into live evaluations by incorporating the mcp and a2a protocols, as illustrated in figure 1 . using this live benchmark, we evaluate the effectiveness of privacychecker relative to baseline methods, analyze differences between static and live evaluations, and explore integration strategies for priva-cychecker within the mcp and a2a protocols. to the best of our knowledge, this is the first work to extend ci based privacy mitigation into live, multiagent llm environments. our contributions can be summarized as follows: ## privacy under mcp and a2a protocols the model context protocol (mcp) @xcite is an open standard that enables large language models (llms) to interact efficiently with external tools, data sources, and services through a unified client server architecture. its effective use requires all system components to comply with the same protocol. in contrast, the agent2agent (a2a) protocol @xcite facilitates secure, asynchronous, and interoperable communication among decentralized agents. by integrating mcp and a2a, one can construct a multiagent system in which agents communicate via a2a while accessing tools seamlessly through mcp. although our experiments use mcp and a2a as concrete testbeds, we stress that privacychecker itself is protocol-agnostic. it does not depend on mcp-or a2a-specific apis or assumptions, but instead operates via modular, prompt-based reasoning and tool-call monitoring. this design allows it to generalize easily to other agent systems such as au-togen, langgraph, or custom orchestrators. we adopt mcp and a2a in this work primarily due to their growing influence in real-world agent deployments and their ability to support structured privacy evaluation across tools and agents. this section first outlines the adaptation of the static benchmark privacylens into a live benchmark privacylens-live using mcp alone and in combination with a2a. we then present the deployment of privacychecker within this framework and evaluate its performance using the live benchmark. ## conclusion we introduced privacychecker, a modular, model-agnostic mitigation framework that injects contextual-integrity reasoning into llm agent decision making. by prompting agents to identify and evaluate information flows at inference time, pri-vacychecker reduces privacy leakage rates by over 75% across diverse models (e.g., from 36.08% to 7.30% on deepseek-r1 and from 33.06% to 8.32% on gpt-4o) while preserving task helpfulness. we also presented privacylens-live, a dynamic evaluation suite that transforms existing static privacy benchmarks into realistic mcp and a2a agent workflows. our live scenarios expose substantially higher leakage risks than prior single-agent probes, underscoring the importance of evaluating and mitigating privacy in multi-agent ecosystems. by opensourcing all data and code, we aim to inspire the community to build on our benchmarks and mitigation strategies, accelerating progress toward more secure and trustworthy autonomous llm agents.