On the Vulnerability of Text Sanitization (2025)

Abstract: Text sanitization, which employs differential privacy to replace sensitive tokens with new ones, represents a significant technique for privacy protection. Typically, its performance in preserving privacy is evaluated by measuring the attack success rate (ASR) of reconstruction attacks, where attackers attempt to recover the original tokens from the sanitized ones. However, current reconstruction attacks on text sanitization are developed empirically, making it challenging to accurately assess the effectiveness of sanitization. In this paper, we aim to provide a more accurate evaluation of sanitization effectiveness. Inspired by the works of Palamidessi et al., we implement theoretically optimal reconstruction attacks targeting text sanitization. We derive their bounds on ASR as benchmarks for evaluating sanitization performance. For real-world applications, we propose two practical reconstruction attacks based on these theoretical findings. Our experimental results underscore the necessity of reassessing these overlooked risks. Notably, one of our attacks achieves a 46.4% improvement in ASR over the state-of-the-art baseline, with a privacy budget of \epsilon=4.0 on the SST-2 dataset. Our code is available at: https://github.com/mengtong0110/On-the-Vulnerability-of-Text-Sanitization.

## introduction with the advancement of natural language processing (nlp) @xcite , concerns related to its privacy implications @xcite have significantly intensified. for instance, samsung employees previously leaked the company's confidential meeting records by uploading documents to the cloud services of large language models (llms) @xcite . more recently, reports @xcite have disclosed a concerning flaw in llm-based chatbots @xcite , which could expose user data and put private information at risk. to address these privacy concerns, text sanitization @xcite is now regarded as a crucial solution for privacy preservation in text and has been widely adopted by companies, such as amazon @xcite and @xcite . specifically, before users upload their documents to an untrusted cloud server for model training or inference, they can employ differential privacy (dp) @xcite to replace tokens corresponding to sensitive entities in the documents with newly sampled tokens. this method minimizes alterations in the uploaded data compared to other methods, such as synthetic data generation @xcite . while dp sanitizes sensitive tokens, recent works @xcite have demonstrated that attackers can still recover them from sanitized text. this finding underscores the significance of measuring the effectiveness of text sanitization. one of the principal methods for evaluating its effectiveness in privacy protection is through reconstruction attacks @xcite , which aim to recover sensitive tokens from sanitized text, as illustrated in figure 1 . existing attacks. in this context, several reconstruction attacks have been developed to evaluate the vulnerability of text sanitization: the embedding inversion attack @xcite reconstructs tokens by selecting the token with the closest embedding distance to the sanitized token @xcite . the inversion bert attack @xcite trains an inversion model to map sanitized tokens back to their original forms. the mask token inference attack @xcite replaces each token in the sanitized text with a "[mask]" token to predict the raw one. however, these attacks are empirically developed, which makes it difficult to accurately measure the effectiveness of sanitization (as discussed in section 2). our proposal. in this paper, we aim to address the evaluation gap in privacy protection for dpbased text sanitization. inspired by the works of @xcite , who estimate privacy leakage using bayesian inference, we introduce both the contextfree optimal reconstruction attack and the contextual optimal reconstruction attack for assessing text sanitization. we also derive theoretical bounds on the attack success rate (asr), which serve as benchmarks for evaluating its performance in protecting privacy. to support real-world implementation, we propose two practical attacks based on these theoretical findings. experimental results demonstrate that our reconstruction attacks achieve higher asrs than baseline methods. our contributions. we summarize our main contributions as follows: text sanitization @xcite leverages differential privacy (dp) @xcite to replace sensitive tokens with newly sampled tokens. specifically, let x represent the set of sensitive tokens and y represent the set of sanitized tokens. an original sentence consists of sensitive tokens x 1 , x 2 , . . . , x n , where x @xmath0. its sanitized sentence is y 1 , y 2 , . . . , y n , where the token y @xmath1. additionally, we define the sanitized context c @xmath2, y 2 , . . . , y i-1 , y i+1 , . . . , y n for the sanitized token y i (or original token x i ), where c @xmath3. in addition to text sanitization, recent studies @xcite have demonstrated that llms can also protect private information. definition of sensitive tokens. in previous research @xcite @xcite , they have made varying choices on the definition of sensitive tokens. in our study, we set the sensitive tokens in accordance with the corresponding defense method of text sanitization @xcite . @xcite identifies the token closest to a sanitized token by comparing their embedding distances (song and raghunathan, 2020) as the reconstructed token. the inversion bert attack @xcite uses a trained model to convert sanitized representations back to their original tokens on a one-to-one basis. however, neither method has performed a comprehensive evaluation of sanitization effectiveness, which should consider the context of the sanitized tokens. the mask token inference attack @xcite replaces each token in sanitized text with a "[mask]" token, which is then predicted to reconstruct the raw token. this attack overlooks the impact of probability distributions of raw tokens and the sampling probabilities in dp on the attack result, leading to an inaccurate evaluation of sanitization performance. ## theoretically optimal reconstruction attacks in this section, we first present the threat model and then develop context-free optimal reconstruction attack, deriving its asr bound to enable quick evaluation of text sanitization. for a comprehensive evaluation, we propose contextual optimal reconstruction attack with its asr bound. ## conclusion in this work, we implement both context-free and contextual optimal reconstruction attacks against text sanitization. we derive their bounds on asr to guide the evaluation of sanitization effectiveness. we propose two practical attacks for real-world implementations. evaluation results validate that our attacks surpass baseline methods. this work represents an initial attempt to provide a more precise evaluation of sanitization effectiveness compared to current empirical attacks. through this endeavor, we hope our work contributes to the research of privacy-preserving nlp.