Title: Self-Refining Language Model Anonymizers via Adversarial Distillation
Venue: NeurIPS 2025
Accepted: yes
Keywords: privacy

anonymization

distillation

large language models

small language models
Abstract: Large language models (LLMs) are increasingly used in sensitive domains, where their ability to infer personal data from seemingly benign text introduces emerging privacy risks. While recent LLM-based anonymization methods help mitigate such risks, they often rely on proprietary models (e.g., GPT-4), raising concerns about cost and the potential exposure of sensitive data to untrusted external systems. To address this, we introduce $\textit{SElf-refining Anonymization with Language model}$ (SEAL), a novel distillation framework for training small language models (SLMs) to perform effective anonymization without relying on external models at inference time. SEAL leverages adversarial interactions between an LLM anonymizer and an inference model to collect trajectories of anonymized texts and inferred attributes, which are then used to distill anonymization and critique capabilities into SLMs through supervised fine-tuning and preference learning. The resulting models learn both to anonymize text and to evaluate their outputs, enabling iterative improvement of anonymization quality via self-refinement. Experiments on SynthPAI, a dataset of synthetic personal profiles and text comments, demonstrate that SLMs trained with SEAL achieve substantial improvements in anonymization capabilities. Notably, 8B models attain a privacy-utility trade-off comparable to that of the GPT-4 anonymizer and, with self-refinement, even surpass it in terms of privacy protection. These results highlight the effectiveness of our adversarial distillation framework for training SLMs as efficient anonymizers.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes SEAL (Self-refining Anonymization with Language models), a novel distillation framework for training small language models (SLMs) to perform privacy-preserving text anonymization without relying on proprietary, external LLMs during inference. The approach leverages adversarial interactions between anonymizers and inference models (the same LLMs) to create trajectories of anonymized texts, attribute inferences, and utility scores. These are used to train SLMs via supervised fine-tuning (SFT) and direct preference optimization (DPO). The trained SLMs can iteratively self-refine the input text to achieve the privacy-utility trade-off. Experiments on the SynthPAI dataset show that SEAL-trained models (e.g., Llama3-8B) match or surpass the privacy-utility tradeoffs of GPT-4-based anonymizers.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n1. The adversarial data generation method is innovative.\n2. Cleverly introducing DPO through the setting of privacy and utility score functions has significantly improved performance on the basis of SFT.\n3. The framework achieves iterative enhancement by simultaneously executing anonymization, adversarial inference, and utility evaluation in the same model.\n\nWeaknesses:\n1. There is no detailed explanation on how to obtain a confidence score.\n2. In the setting, user should deploy the model locally for iteratively anonymization, the cost of this part is not provided.\n3. The utility drops compared to baseline after anonymization.\n\nQUESTIONS:\nQ1. The authors are suggested to do more research on the privacy-utility tradeoff quantitatively.\n\nQ2. The self-refinement loop seems automatic. Would it be feasible to guide it based on user preferences (For example, users are particularly sensitive to certain aspects of privacy like age)?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-16"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors introduce SElf-refining Anonymization with Language model (SEAL), a framework for distilling the anonymization capabilities of LLMs into small language models (SLMs), to do anonymization at inference time. They use adversarial interactions between an LLM anonymizer and an inference model to collect anonymized texts with inferred private attributes, which they use to train a SLM anonymizer via distillation. This is done using supervised fine-tuning (SFT) and direct preference optimization (DPO). They run experiments on SynthPAI, a dataset of synthetic personal profiles and text comments, to show that their method gives better privacy-utility tradeoffs than other baselines.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths\n- The experimental results show that SEAL gives better privacy than the other baselines, for the dataset used.\n- The authors compare the performance against a wide range of baselines, and also include evaluations against a hard dataset with contextually embedded personal information.\n- The authors also present privacy-utility tradeoffs when using SEAL across different models with a range of sizes.\n\nWeaknesses\n- Only synthetic data is used for the evaluation, which may differ substantially from real data.\n- It is unclear if the results generalize to different types of data, since only one type of dataset was used. Furthermore, the dataset seems to consist of short comments, and it is unclear if SEAL works with longer text common in other applications that require privacy.\n- The evaluation of privacy and utility are done using GPT-4. This means that the results on privacy are dependent on what GPT-4 is able to detect, which may differ from actual humans, and similarly for the utility results.\n\nQUESTIONS:\n- How is the hard eval dataset constructed? It would be useful to give more details about this dataset since it is new to this paper.\n- How does SEAL perform on other types of datasets?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-06-26"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces SElf-refining Anonymization with Language model (SEAL), a novel framework to distill the capabilities of large, proprietary models into smaller, locally-run language models (SLMs) for text anonymization. The method uses data from adversarial LLM interactions to train an SLM to both anonymize text and critique its own outputs for privacy and utility. This enables the SLM to iteratively improve its anonymizations via self-refinement at inference time without external model feedback. Experiments show that an 8B model trained with SEAL can match or even surpass the privacy-utility tradeoff of a GPT-4-based anonymizer\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\nThe paper addresses the critical need for private, efficient, and accessible anonymization tools. The proposed self-refining distillation framework, which uniquely teaches a model to critique its own outputs, is a significant and original contribution.\n\nThe methodology is sound, and the experimental evaluation is comprehensive. The use of strong baselines, varied datasets, and detailed ablation studies provides convincing evidence for the framework's effectiveness.\n\nThe paper is exceptionally well-written and clearly structured, with effective figures that make the complex framework easy to understand.\n\nWeaknesses:\n\nThe practical latency of multi-step self-refinement is not discussed, leaving a gap in understanding the full efficiency trade-off compared to single API calls.\n\nThe privacy evaluation is benchmarked against a specific model (GPT-4), meaning the anonymization is not guaranteed to hold against future, more capable adversaries.\n\nThe distilled critic model may be over-reliant on the style of its teacher model (GPT-4o), potentially limiting its effectiveness on out-of-distribution text.\n\nQUESTIONS:\nQuestions:\n\n1. Can the core principle of distilling generation and critique capabilities for self-refinement be generalized to other complex tasks, such as summarization or code generation, where a model could iteratively refine its output by checking for coherence, correctness, or logical consistency?\n\n2. What is the practical inference latency for multi-step self-refinement on an 8B model, and how does this compare to the latency of a single API call to a larger model like GPT-4o?\n\n3. Table 1 shows that after one iteration, SEAL (8B) has a slightly lower utility score (0.931) than the GPT-4o baseline (0.947). However, after three iterations, SEAL's privacy is significantly better, while utility drops to 0.862. Could you comment on this privacy-utility tradeoff? Is it possible to tune the model or the self-refinement process to prioritize utility preservation while still achieving substantial privacy gains?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nIn this work, authors tackle the task of text anonymization to remove the PII information. To this end, authors curate a high-quality dataset of input-output pairs of data by using SOTA adversarial anonymization. Next, authors use distillation to impart the anonymization capabilities to SLMs via two steps, SFT followed by DPO. Authors validate the effectiveness of the distilled SLM on the SynthPAI dataset and showcase the versatility of their approach.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths\n- The proposed method is particular effective in the real-world setting as the entire anonymization happens on device without any dependency on the big models\n- The method is easy to follow and clearly explained\n- Authors also contribute to a new evaluation set of 600+ datapoints to assess the anonymization capabilities\n\nWeakness\n- Technical novelty of the paper is limited and shares the similar theme with prior works. Already works such as [a, b] show that text anonymization capabilities can be imparted to SLMs by distillation and work of [a] uses a very similar methodology of adversarial anonymization of [9] to curate distillation dataset.  \n\n- LLM-as-a-judge is used for both privacy and utility evaluation. The results especially for utility evaluation needs human validation to understand how reliable the evaluation metrics with GPT-4 and their correlation to human evaluation. \n\n- Authors use GPT-4o for dataset generation, and also use GPT-4 for offline evaluation of text anonymization. I am concerned of the bias introduced due to this setting as there are recent works which show Judge has preference to their own generations [c]. \n\n[a] IncogniText: Privacy-enhancing Conditional Text Anonymization via LLM-based Private Attribute Randomization, SafeGenAI workshop, NeurIPS 2024\n\n[b] The Fire Thief Is Also the Keeper: Balancing Usability and Privacy in Prompts, 2024\n\n[c] LLM Evaluators Recognize and Favor Their Own Generations, 2024\n\nQUESTIONS:\n- Please see weakness on human validation of results, prior works, experimental setting.\n\n-  The prompt used in inference time for the step discussed in Line 160-161 is not clear. This seems to differ from the training time prompt style, where it takes text $x_i$ and outputs either $x_j$, $P$, $U$. But at the inference, both privacy and utility are conditioned, which differs from training prompts. Please provide more details to this.\n\n-  I have concerns on the experimental setting relying on GPT-4 both for dataset traces, and also using during evaluation stage. Authors could discuss the generalization of their training dataset construction with other models.  \n\n-   The performance of the anonymizer is dependent on the architecture family. SEAL Qwen-3-8B is on par with the performance of AA with larger models. More experiments with other families will be helpful as the current  training of SLMs is very minimal with <2 epochs of SFT and DPO. \n\n- More qualitative examples showcasing the differences with AA [9] wrt to each attribute is needed to understand the quality. While paper provides objective metrics, it would benefit the reader to highlight striking differences to baselines in terms of anonymization, along with evaluation (privacy and utility) metrics.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-02"}