DYNTEXT : Semantic-Aware Dynamic Text Sanitization for Privacy-Preserving LLM Inference (2025)

Abstract: LLMs face privacy risks when handling sensitive data. To ensure privacy, researchers use differential privacy (DP) to provide protection by adding noise during LLM training. However, users may be hesitant to share complete data with LLMs. Researchers follow local DP to sanitize the text on the user side and feed non-sensitive text to LLMs. The sanitization usually uses a fixed non-sensitive token list or a fixed noise distribution, which induces the risk of being attacked or semantic distortion. We argue that the token’s protection level should be adaptively adjusted according to its semantic-based information to balance the privacy-utility trade-off. In this paper, we propose DYNTEXT, an LDP-based Dynamic Text sanitization for privacy-preserving LLM inference, which dynamically constructs semantic-aware adjacency lists of sensitive tokens to sample non-sensitive tokens for perturbation. Specifically, DYNTEXT first develops a semantic-based density modeling under DP to extract each token’s density information. We propose token-level smoothing sensitivity by combining the idea of global sensitivity (GS) and local sensitivity (LS), which dynamically adjusts the noise scale to avoid excessive noise in GS and privacy leakage in LS. Then, we dynamically construct an adjacency list for each sensitive token based on its semantic density information. Finally, we apply the replacement mechanism to sample non-sensitive, semantically similar tokens from the adjacency list to replace sensitive tokens. Experiments show that DYNTEXT excels strong baselines on three datasets.

## introduction llms demonstrated exceptional capabilities in nlp tasks, particularly with closed-source llms like gpt-4 (open, 2023) that exclusively provide online inference services. however, directly submitting text containing sensitive information to * corresponding authors those llms poses significant privacy risks @xcite . to ensure privacy protection, a provable theoretical guarantee is crucial. dp @xcite formally defines and quantifies privacy. consequently, most researchers apply dp to llms to safeguard privacy @xcite . to achieve dp, methods like dp-sgd @xcite and pate @xcite , mainly focus on adding calibrated noise to the model or input representations during the training so that sensitive user data are hardly inferred from the trained model. users need to send their data to llms for training under the dp framework with noise. however, they may hesitate to share their complete data due to privacy concerns, fearing that llms may not be fully trustworthy or that an intermediary eavesdropper could compromise sensitive information @xcite . to address the above issues, ldp @xcite introduces a new scenario with two phases: local processing and llm training/inference. local processing occurs on the user side, which can access and process the private data to protect them. the protected data are then transmitted to llms for training or inference. typically, these local processing methods generate perturbed text by replacing the tokens (e.g., words or n-grams) in the private text with new nonsensitive tokens @xcite . specifically, some methods @xcite inject calibrated noise with a dp guarantee into the original token embedding (high-dimensional vector) to generate a noisy embedding, then replace the original token with the token closest to the noisy embedding. however, token (i.e. text) embedding space is usually uneven and irregular since the text signals are too sparse and discrete to represent with dense embeddings so well @xcite @xcite . dp-required noises are totally randomized within a regular distribution (i.e. gaussian or laplace). applying a dp required noises to the original token embedding sometimes leads to unexpected bias to damage the semantics. to avoid the above problem, researchers propose replacing the original tokens by sampling new tokens from a pre-computed distribution. these methods, like santext+ @xcite and custext+ @xcite , leverage dp learning methods to sequentially replace sensitive words in text with new words, which are sampled from a fixed word list carrying non-sensitive words similar to the sensitive words. these methods are more reliable and interpretable, effectively avoiding unexpected bias caused by noise and thereby enhancing the practicality of the text. however, the fixed token list introduces predictable replacement patterns, making it easier for attackers to exploit this regularity of information to infer the original sensitive information @xcite . to mitigate the above vulnerability for potential attacks @xcite , researchers introduce randomness in the non-sensitive tokens list to replace each sensitive token, avoiding potential attacks and strengthening defense against privacy threats @xcite . however, adding random perturbation to non-sensitive lists still has limitations. these methods often apply perturbations with the same distribution to all tokens, ignoring the sensitivity of each token. for tokens with low semantic sensitivity, overly strict privacy protection mechanisms may lead to unnecessary semantic loss, thus affecting the quality of the generated perturbed text. to balance the privacy-utility trade-off, we argue that sanitization should consider the token's semantic-based information while maintaining antiattack capabilities. so, we should integrate the token's semantic-based information with its nonsensitive token list under privacy protection (i.e. dp), enhancing the quality of the perturbed text and adaptively adjusting the list to resist attacks. in this paper, we propose an ldp-based dynamic text (dyntext) foot_0 sanitization mechanism for privacy-preserving llm inference, which dynamically builds a semantic-aware adjacency list of sensitive tokens to sample non-sensitive tokens for perturbation. the adjacency list satisfies dp and is customized to each token's semantic density, with smaller lists in high-density areas and larger ones in low-density areas, which encourages the sampling of high-density tokens and assigning high noise to low-density tokens. specifically, we first develop a semantic-based density information modeling module under dp to extract the density information of each token in the embedding space. this module employs the gaussian noise to achieve dp and a token-level smoothing sensitivity mechanism by combining the idea of gs and ls to avoid excessive noise in gs and privacy leakage in ls. we then dynamically construct an adjacency list for each sensitive token based on noisy semanticbased density information, which adjusts the size of each sensitive token's non-sensitive adjacency token list. this strategy effectively preserves semantic information while resisting attacks. finally, we employ a sensitive token replacement to sample non-sensitive similar tokens from the adjacency list and replace the sensitive token for perturbation. our contributions are as follows: (1) we propose dyntext, an ldp-based dynamic text sanitization mechanism that replaces sensitive tokens based on semantic density, adaptively adjusting the protection level for a better privacy-utility trade-off. ## preliminaries differential privacy (dp) @xcite ) is widely regarded as the gold standard for data privacy. its definition is as follows: definition 3.1 (ε, δ)-differential privacy. let ε ≥ 0 and δ ∈ [0, 1]. a randomized algorithm m is (ε, δ)-differentially private if for any two neighboring datasets d and d ′ , which differ in only a single record, and for any set s of possible outputs: local differential privacy (ldp) is a special case of dp in which the server is untrusted, and data privatization is performed on the client side. definition 3.2 (ε, δ)-local differential privacy. let ε ≥ 0 and δ ∈ [0, 1]. a randomized mechanism m is said to satisfy (ε, δ)-ldp if for any two possible inputs x, x ′ ∈ d and any possible output @xmath0, the following condition holds: ## conclusion in summary, we proposed dyntext to sanitize text for privacy-preserving llm inference. dyn-text extracts token density using semantic-based density information modeling under dp; then dynamically constructs the adjacency list of each token based on the density information to adaptively adjust the protection level; finally, samples nonprivate tokens from the list through a replacement mechanism to replace sensitive tokens. experiments show that dyntext achieves sota performance in balancing the privacy-utility trade-off.