LLM s are Privacy Erasable (2025)

Abstract: The capabilities of large language models (LLMs) are advancing at an remarkable pace, along with a surge in cloud services that are powered by LLMs. Their convenience has gradually transformed the routines people work. However, for services such as document summarizing, editing, and so on, users need to upload relevant files or context to obtain the desired services, which may inadvertently expose their privacy. This paper aims to address the challenging balance between the convenience of LLMs services and user privacy concerns. Specifically, based on the structural and functional characteristics of LLMs, we have developed a strategy that safeguards user prompt while accessing LLM cloud services, even in scenarios where advanced reconstruction attacks are adopted. We comprehensively evaluate the efficacy of our method across prominent LLM benchmarks. The empirical results show that our method not only effectively thwarts reconstruction attacks but also, in certain tasks, even improves model performance, surpassing the outcomes reported in official model cards.

## introduction in recent years, research on large language models (llms) @xcite @xcite has attracted significant attention from enterprises, universities, and governments. simultaneously, their enhanced capabilities are transforming how people work @xcite , marking a milestone in humanity's progress toward artificial general intelligence. however, existing research primarily focuses on model capabilities, often overlooking user privacy. in fact, in numerous scenarios involving llm cloud services, users are required to upload relevant data @xcite , which is highly likely to involve their privacy. for instance, the popular use of llms for * corresponding author organizing meeting minutes, as well as the integration of gpts @xcite and @xcite in office (e.g., word, excel), enables these tools to directly utilize user data as context for various operations like summarizing, editing, and computing. this undoubtedly provides us with tremendous convenience. nevertheless, whether it is meeting minutes or data in office software, it is most likely to involve important privacy related to enterprises or individuals. therefore, as llms have demonstrated the capability to be deployed and provide services, we must address the privacy issues they entail. quantizing llms and deploying them locally is undoubtedly the optimal solution @xcite @xcite . however, given the limited computational power and runtime memory of local devices, as well as the performance degradation caused by low-bit quantization, this approach requires further exploration. cryptographic methods @xcite , such as secure inference over fully homomorphic encryption @xcite , have also been a research hotspot. nevertheless, ciphertext inference on llms remains challenging, both algorithmically and hardware-wise. methods based on random perturbations, such as differential privacy (dp) @xcite , ensure service providers cannot directly access user data by perturbing and replacing token embeddings @xcite @xcite . while dp provides rigorous privacy guarantees through formal proofs, we show that significant perturbations are needed to obfuscate tokens, which impair llms' effectiveness for fine-grained tasks. as a privacy mechanism for llm inference, dp may be overly rigorous for fine-grained tasks, and no satisfactory solutions have been proposed. hence, we aim to explore some empirical alternative strategies. specifically, this paper protects user prompts based on the structural and functional character-istics of llms. we deeply analyze the sources of privacy vulnerabilities in llms and, based on the results, design customized privacy protection strategy, which has the following advantages: it is simple and easy to implement, effectively resists privacy attacks, and has almost no impact on model performance. we validate these claims through extensive experiments and analysis. our contribution. we propose a highly practical distributed inference paradigm for llm cloud services. this paradigm achieves privacypreserving inference without compromising performance by deploying only a few modules on the user side, combined with a simple prompt erasure operation. we evaluate our method on mainstream benchmarks, including reading comprehension, mathematics, code, common-sense reasoning, and general benchmarks, with zero-shot, few-shot @xcite , and chain-of-thought (cot) @xcite settings. our contributions can be summarized as follows: ## experiments due to space constraints, all experimental configurations are in appendix d, including attack implementation details, the llms used, benchmarks tested, evaluation metrics, and protected prompts. we also provide detailed analysis and discussion of hyperparameter configurations in appendix d. ## conclusion in this paper, we reveal and analyze the privacy vulnerabilities in llms. based on our analysis, we propose a pipeline-parallel privacy-preserving inference paradigm. through experiments, we validate that this paradigm resists advanced privacy reconstruction attacks without compromising utility. additionally, we apply low-bit quantization to our defense method, finding high compatibility and an efficient balance between privacy, utility, and memory efficiency. finally, an intuitive discussion on why the proposed method does not significantly affect model utility is provided in the appendix i.