Self-Refining Language Model Anonymizers via Adversarial Distillation
Authors: Kyuyoung Kim
Hyunjun Jeon
Jinwoo Shin
Venue: NeurIPS 2025 poster ()
Keywords: privacy
anonymization
distillation
large language models
small language models
Primary area: deep_learning
TL;DR: We propose a distillation framework for training language model anonymizers capable of effective anonymization via iterative self-refinement

Abstract: Large language models (LLMs) are increasingly used in sensitive domains, where their ability to infer personal data from seemingly benign text introduces emerging privacy risks. While recent LLM-based anonymization methods help mitigate such risks, they often rely on proprietary models (e.g., GPT-4), raising concerns about cost and the potential exposure of sensitive data to untrusted external systems. To address this, we introduce $\textit{SElf-refining Anonymization with Language model}$ (SEAL), a novel distillation framework for training small language models (SLMs) to perform effective anonymization without relying on external models at inference time. SEAL leverages adversarial interactions between an LLM anonymizer and an inference model to collect trajectories of anonymized texts and inferred attributes, which are then used to distill anonymization and critique capabilities into SLMs through supervised fine-tuning and preference learning. The resulting models learn both to anonymize text and to evaluate their outputs, enabling iterative improvement of anonymization quality via self-refinement. Experiments on SynthPAI, a dataset of synthetic personal profiles and text comments, demonstrate that SLMs trained with SEAL achieve substantial improvements in anonymization capabilities. Notably, 8B models attain a privacy-utility trade-off comparable to that of the GPT-4 anonymizer and, with self-refinement, even surpass it in terms of privacy protection. These results highlight the effectiveness of our adversarial distillation framework for training SLMs as efficient anonymizers.

Self-Refining Language Model Anonymizers via Adversarial Distillation
Kyuyoung Kim*), Hyunjun Jeon"!, Jinwoo Shin
KAIST AI
Abstract
Large language models (LLMs) are increasingly used in sensitive domains, where their ability infer personal data from seemingly benign text introduces emerging privacy risks. While recent LLM-based anonymization methods help mitigate such risks, they often rely on proprietary models (e.g., GPT-4), raising concerns about cost and the potential exposure of sensitive data to untrusted external systems. To address this, we introduce SElf-refining Anonymization with Language model (SEAL), novel distillation framework for training small language models (SLMs) to perform effective anonymization without relying on external models at inference time. SEAL leverages adversarial interactions between an LLM anonymizer and an inference model to collect trajectories of anonymized texts and inferred attributes, which are then used to distill anonymization and critique capabilities into SLMs through supervised fine-tuning and preference learning. The resulting models learn both to anonymize text and to evaluate their outputs, enabling iterative improve- ment of anonymization quality via self-refinement. Experiments on SynthPAL. a dataset of synthetic personal profiles and text comments, demonstrate that SLMs trained with SEAL achieve substantial improvements in anonymization capabilities. Notably, 8B models attain a privacy-utility trade-off comparable that of the GPT-4 anonymizer and, with self-refinement, even surpass terms of privacy protection. These results highlight the effectiveness of our adversarial distillation framework for training SLMs as efficient anonymizers.
Introduction
Recent advances in large language models (LLMs) have substantially expanded their capabilities [1, 2, 31, leading widespread adoption across domains such conversational agents, healthcare, and finance [1, 4, 5], where models often process sensitive personal information. Early privacy concerns primarily focused memorization and data leakage [6]. but recent studies show that LLMs can infer private attributes, such as location, identity, or demographics, from subtle semantic cues in seemingly innocuous text [7]. With sufficiently capable models, such inferences can be surprisingly accurate and often go undetected by users, who continue to share content unaware of the privacy risks. Traditional anonymization methods, such those based on named entity recognition pattern matching, target surface-level identifiers and frequently overlook the context-dependee semantics that LLMS can exploit [8]. This gap underscores the need for more robust anonymization techniques that account for semantic context to defend against inference- based privacy threats.
growing body of research seeks overcome the limitations of traditional methods through LLM-based anonymization frameworks that protect semantic-level private information from model inference [9, 10]. In feedback-guided adversarial anonymization [9]. LLM anonymizer processes input text while second LLM, acting an adversary, attempts to infer private attributes from the anonymized output. Based on the adversarial feedback, the anonymizer iteratively refines its output
"Equal contribution
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Data Synthesis with LLMs
Synthesized Text
Inferred Data
Anonymized Text
Utility Score
moved to a new city and started working hospital ?
Oocupation Reasoning Working at hospital ...
Nurse
began new job new city
Meaning
8
Readability
10
Hallucination
10
Distillation
Supervised Fine-tuning
Preference Learning
SLM oo
Q
Infer the user's information
Anonymized
Anonymizzd
Liama 3 8B, Qwen' 8B,
Adapt anonymize
text
Evaluate semantics and readability
Better privacy utiliy
Worse privacy utility
Self-Refinement
Evaluation Text
Distilled SLM
Anonymized Text
Debugging life like faulty code!
SElf-refining Anonymization with Language model (SEAL)
Fixing life like it's of twists!
Figure Overview of SEAL. We simulate adversarial anonymization with LLMs to generate trajectories of anonymized texts, inferred private attributes, and utility feedback. These trajectories are used to distill anonymization and critique capabilities into SLMs via supervised fine-tuning and preference learning, enabling effective anonymization through iterative self-refinement.
to prevent re-identification of previously inferred attributes. Using GPT-4 as both the anonymizer and inference model, this framework achieves substantially improved privacy-utility trade-off compared to traditional industry-grade anonymizers. While these results highlight the potential of LLM- based anonymization, reliance on proprietary models remains costly and risks exposing sensitive text to potentially untrusted external systems []]]. Attempts to distill GPT-4's anonymization capabilities into smaller models have been explored only to limited extent [10], and prior approaches continue to depend on GPT-4 for adversarial feedback, incurring similar costs and privacy concerns.
To address these limitations, we propose SElf-refining Anonymization with Language models (SEAL), a novel framework for efficiently distilling the anonymization capabilities of LLMs into small language models (SLMs). SEAL leverages adversarial interactions between LLM anonymizer and an inference model to collect anonymized texts paired with inferred private attributes. Using this data, we train an SLM anonymizer through two-stage distillation process: supervised fine-tuning (SFT) for task adaptation, followed by direct preference optimization (DPO) [12] for refinement. In the SFT stage, the model learns from trajectory pairs, taking each preceding text as input and predicting subsequent text with improved privacy and utility. This encourages the model to generate diverse anonymizations that achieve stronger privacy-utility trade-offs. The model is also trained to infer private attributes and assess utility, enabling it to evaluate the quality of different anonymizations, including its own. the DPO stage, we further refine the model using preference pairs, where anonymizations that are more robust to attribute inference and preserve better utility are treated as preferred outputs. This trains the model to consistently produce higher-quality anonymizations, At inference time, the model performs iterative self-refinement, alternating between anonymization and evaluation to enhance its own outputs, without relying on external model feedback.
Experimental results on SynthPAI [13]. diverse dataset of synthetic personal profiles and text com- ments, show that SLMs trained with SEAL achieve substantial gains in anonymization performance. Notably, 8B models attain privacy-utility trade-off comparable to that of the GPT-4 anonymizer, which relies on the GPT-4 inference model, and with sufficient self-refinement, even surpass on unseen profile data. These results demonstrate SEAL's effectiveness in training SLM anonymizers that defend against inference attacks without external models, enhancing privacy and efficiency through local data processing. Moreover, the model's 's ability to perform attribute inference and utility evaluation enables users to control the degree of anonymization according to their preferences. To facilitate further research, we release the full dataset used in experiments.

---

Our main contributions are as follows:
We propose novel framework for distilling anonymization and adversarial inference capabilittis of LLMs into SLMs, enabling effective anonymization via iterative self-refinement.
We release high-quality dataset with carefully crafted samples, designed to train and evaluate anonymizers against LLMM-base inference threats, to facilitate further research.
Empirical results demonstrate that our framework trains SLMs that can outperform the GPT-4 anonymizer, providing greater efficiency and enhanced privacy via local data processing.
Related Work
Text anonymization. Text anonymization aims to process textual data to remove obscure per- sonally identifiable information (PII) while preserving overall semantics and utility [14]. Traditional methods primarily rely rule-based de-identification [14], such removing explicit identifiers (e.g., names, SSNs, locations) using named entity recognition or pattern matching [15]. While effec- tive for surface-level PII, these methods fail to address contextually embedded or inferable private information information-ssinabe that sufficiently capable LLMs can recover with high accuracy [9]. Recent work has explored LLM-based anonymization techniques to identify and redact such contextual cues [9, 16]. Our approach builds on this line of work by distilling these capabilities into compact, locally deployable models for efficient anonymization.
PII inference by LLMs. The growing capabilities of LLMs pose emerging privacy risks, partic- ularly when users interact with external services, recent models can effectively infer sensitive information from seemingly innocuous user-provided data [7]. Such inferences are often subtle and context-dependent, causing users to continue sharing content without fully understanding the privacy implications. Moreover, user data may be intercepted during transmission [17] or retained by service providers for purposes such model training. increasing the risk of misuse or data breaches. To mitigate these risks, we propose a framework for developing locally deployable anonymization models, enabling users retain full control over their data while maintaining privacy protections without compromising utility. Unlike prior approaches [18), our framework distills anonymization, adversarial inference, and utility evaluation into single model, allowing effective anonymization through iterative self-refinement using the model's own feedback.
3 Adversarial data synthesis for distillation
To enable small language models (SLMs) to serve as effective anonymizers, we adopt distillation- based approach using training data generated through adversarial anonymization [9], where an LLM anonymizer and adversary model alternate between anonymization and attribute inference to iteratively refine the input text. We leverage this framework with sufficiently capable LLMs to collect trajectories of anonymizations and inferred attributes for distillation.
Given set of texts { and private attributes P (e-g., age, gender, and location) be anonymized, we simulate adversarial anonymization using anonymizer Manon an inference model Mprive and utility evaluator M Specifically, we iterate the following procedure for a fixed number of steps or until no additional attributes can be inferred, collecting all intermediate and final anonymizations, inferred attributes, and corresponding utility assessments.
1. Attribute inference. At each step f, the inference model Mpni is applied to the current text Ty to identify any attributes in P that remain inferable:
P Mpeiv (X,, P))
For each inferred attribute p E Pt, we also collect rationale and a confidence score conf(p).
2. Anonymization refinement. If any attributes remain inferable, the anonymizer Manon refines the text using P, as feedback:
F++1 Manon (Ie, Pt).
3. Utility evaluation. To assess utility. Matil compares 21+1 to the original zo. evaluating read- ability, semantic preservation, and whether any new information has been introduced:
Uc+1 Mun (2o, Tt+1).
3

---

You expert anonymizer carefully adapts small parts the text to make anoryyoous You try change as litle as possible, keeping the original text much as possible.
are an experi invesligelor with years experience online profiling and text analysis.
expert similarity scorer that carefully comperes two lexts and gives ecore besed on how similar they are.
Your sask to guess the personal information. Before giving your finel guess brief by step reasoning decision. Give your answer in JSON format below.
You to dacide whether two sets of comments have the same meaning and give
Comments:
content. Before giving your final briaf step-by-step reatoning of your decision. Give your answer in the JSON format below.
Ideas don't stomachs, yet they spark innovation.
Type: Occupation
'location:: {
"readability": .f
Inference: The comment "ldeas don't stomachs. yet they spark innovation" is concise, abstract statement that jusisposes practicll needs with conceptual or intellectual pursuits.
"reasoning" : <detalled reasonings.
"explanatio"" <explanstinn,
"guess" <topt guess>
"scone": <scone>
'age::
"meaning": {
"eessonig": <detailed reasonings.
"explanation: sexplanation,, "score": <score>
Guess: entrepreneur
'guess": guess>
Only change relevent parts and keep everything else
Figure 2: Sample instructions for supervised fine-tuning, Self-refining anonymizers are trained on anonymization (left), attribute inference (middle), and utility evaluation (right) tasks. By learning to both anonymize and evaluate their outputs, the models iteratively improve anonymization quality.
For each measure, we collect both rationale and a numerical score.
At the end of the procedure, we obtain data trajectory r = (sp, $1...... ST), where each 8; = (2i, P., is a tuple consisting of text, inferred attributes, and utility measurements. Applying this process to each input text yields a of trajectories, which we use for distillation.
Self-refining language model anonymizers
4.1 Learning to anonymize and critique
To enable language models both anonymize text and evaluate their outputs, we propose supervised fine-tuning (SFT) framework that jointly trains models on three objectives. Specifically, models learn to perform the following tasks: (1) anonymization, rewriting input text to reduce the risk of attribute inference; (2) adversarial inference, predicting private attributes from a text; and (3) utility evaluation, assessing the semantic preservation and readability of a rewritten text.
Given a trajectory of anonymizations T (so, $1...., ST). where each = (x,, Pi,24) consists of a text, a set of inferred private attributes with confidence scores, and a of utility measures. we construct task-specific datasets for SFT for the three tasks. To assess the relative quality of anonymizations, we define privacy and utility score functions:
p(s.) (- IPsl, - EEEp, conf(m))\PP.), u(s:) = Emeu. m/ m//4
where p(s:) returns the number of inferred attributes and the average confidence assigned by the inference model, and u(s;) returns the average utility. Privacy scores are compared such that anonymizations that yield fewer or less confident inferences are considered better. Utility scores are compared based on the average of individual measures such as semantic preservation and readability.
Using the score functions, we construct the dataset Daaoo by identifying all pairs in trajectory where a later anonymization achieves better privacy and utility:
Danon ((,,, jo < ST, p(5;) p(s:). u(s)) 2 u(s:)).
Conceptually, the model learns to generate, for a given text, multiple anonymizations with improved privacy and utility. For adversarial inference, the model is trained infer private attributes from text, while for utility evaluation, it is trained to assess the utility of anonymized text according to a given set of criteria. We construct the corresponding datasets as follows:
D Prriv (x,, P.) E TT,, Duni (x) 81 T..
The model is then trained to minimize losses over the three datasets:
LSFT Aanon Lanon D Danon)== + Apriv Lpriv (Dpiv) Aunil Lura (Duii))
4

---

Algorithm SEAL: Self-refining anonymizer training
Require: Dataset D, attributes to anonymize P, anonymizer Manon: attribute inference model Mprr. utility evaluator Muiji loss coefficients Äs, target model Ke
// Step 1: Trajectory generation via simulated anonymization
1: Initialize T +-
2: for each E D do
Mo Musi (No, (Fo ma); 4: for / = to T - do
(Tor P); + ((00, 2o,
5:
21+1 Manon (I,, Pi): U1+1 t (an Po+1 t Mpis Mpa(I P) T --TU( (11+11, ((+1,
6:
7: T TU
// Step 2: Task adaptation via SFT
8: Initialize Dance 5- 0; Ppiv t 0; Duil t-
9:
for each J T TETa. do
10:
Danon t Danon (z)) Dpoiv forrt U U(Ii, P.)
10 10sisjsiT p(sy) > p(s;), u(ss) u(si))
11:
8, Ji Dani f- Dani U ((x;, 8, T
12: 0 <- <argg arggming Annon Lanon (Panon) + Apriv Epriv (D Pric)) | Auiil Luii (Dan)
// Step 3: Preference Learning via DPO
13: Initialize Prrff +1
14: for each r C do
15: Dper 4- Drrer U [(0;, mi) 10 i w.l ST p(sw) > p(si), u(sw) 2 u(s)))
16: <- arg mrg, (Ppet)
17: return m.
where each À denotes weight that can be tuned based the relative importance size of the corresponding dataset. Each task loss is standard language modeling loss (i.e., next-token prediction) computed over input-output pairs from the dataset. This joint training enables the model both generate and evaluate anonymized text, supporting iterative self-refinement at inference time.
4.2 Preference learning for anonymization
During SFT, the model learns to generate multiple anonymizations for a given input text but does not necessarily learn to prefer one over another. To address this, we apply direct preference optimization (DPO) [12] to further enhance the model by encouraging to assign higher likelihoods to generations with better privacy-utility trade-offs. Preference data are constructed by taking each text trajectory input and forming preference pairs from the subsequent anonymizations. For each pair, the anonymization with better privacy and utility scores is labeled as the preferred output:
D puet {(II, Sw. æ1) 0 < w, s T, p(sw) p(si), u(sw) 2 u(s:))
We then train the model on the preference dataset to minimize the DPO loss:
Cooo - - log o (B log xe(xwlz)) (a(("u])))
3 log ma(m))))) mad mm)
where Te denotes the model being trained, Tee denotes the SFT reference model, and 3 controls the deviation from Tie- Intuitively, the objective guides the model toward generating anonymizations with stronger privacy-utility trade-offs. With more granular preference labels (e.g., based on model confidence), margin- preference learning [19] can also be applied.
4.3 Iterative self-refinement
Once trained, the anonymizer performs iterative self-refinemen to improve its outputs without relying on external model feedback. each step f, the model infers private attributes and evaluates the utility of the current anonymization 2;, producing inferred attributes '. and utility measures 2T It then generates a refined anonymization
Le+1 TT. Ie, PT, ,("),
conditioned on the privacy and utility evaluations. This process continues until a desired privacy- utility trade-off is achieved fixed iterations limit is reached. In contrast to prior work that relies
5

---

on external, proprietary LLMs, our self-refining anonymizer enables fully local, privacy preserving inference using single model that performs both generation and critique. This also allows interactive control over the anonymization process based on user-defined privacy-utility preferences.
Experimental results
5.1
Setup
For our experiments, we use SynthPAI [13], collection of synthetic personal profiles and text comments generated based on those profiles. From this dataset, we select 3,456 instances with high-quality human labels covering eight personal attributes: age, education level, gender, income level, location, marital status, occupation, and place of birth. These comments serve as the initial texts for anonymization. To generate anonymization trajectories for distillation, we simulate adversarial anonymization [9] for up three steps, using GPT-4o for anonymization, attribute inference, and utility assessment. We use 275 of the 300 synthetic profiles for trajectory generation and hold out the remaining 25 for evaluation, which we refer to as the main eval dataset. This setup illustrates practical use case of our framework: distillation data can be generated on synthetic profiles using external LLMs, while the distilled SLMs subsequently operate on real, internal data locally, without invoking potentially untrusted external models.
Datasets.
Our analysis of SynthPAI shows that texts with contextually embedded personal information-as opposed to explicit identifiers-an are significantly harder to anonymize. To evaluate performance on these more challenging cases, we construct 500 additional synthetic texts containing such embedded personal information using the 25 held-out profiles, forming the hard eval dataset. Importantly, we do not train this data, allowing to assess generalization to harder, previously unseen cases. See Appendix A for details on data construction.
Baselines, We evaluate SEAL against several baselines. Following prior work [9], we include Azure's PII detection tool [20] as traditional rule-based baseline that redacts sensitive information using named entity recognition. To compare against pure semantic rewriting, we include Dipper [21], state-of-the-art 1 IBB paraphrasing model not explicitly designed for anonymization. As recent LLM-based method, we include the adversarial anonymization framework [9], which iteratively refines text through interaction between an anonymizer and an inference model. This framework represents a state-of-the the-art approach to defending against inference- based privacy threats, which we evaluate using several frontier models, including GPT-4o [1], GPT-4o mini, and Gemini 2.5 Flash [22].
Evaluating privacy and utility. To evaluate anonymization quality, we use GPT-4.1 as the inference model, given its strong capability to infer personal attributes [7]. Specifically, the model performs zero-shot chain-of-thouggh inference on the eight attribute types, and we report the average number of correctly inferred attributes, where a lower number indicates better privacy preservation. For utility evaluation, we follow prior work and use a GPT-4.1 utility judge to assess each anonymized text on (1) readability, (2) semantic similarity to the original input, and (3) whether it introduces new information [9]. The average of the three metrics is used as the final utility score. We also report an overall score, defined the difference between relative privacy improvement and relative utility loss, as an aggregate measure of the privacy-utility trade-off.
5.2 Overall performance
Main dataset. Our experimental results show that SLM anonymizers trained with SEAL effectively mitigate private attribute lcakage while maintaining high utility.' Table 1 presents results on the main dataset using Llama3-88 8B model [23] trained with SEAL. Among the baselines. adversarial anonymization is the most competitive, with GPT-4o achieving the best overall privacy-utility trade- off, followed by GPT-4o mini and Gemini 2.5 Flash. With single anonymization step, SEAL outperforms all three models, attaining better privacy than the strongest baseline (0.391 vs. 0.424) while maintaining comparable utility (0.931 vs. 0.947). Notably, SEAL achieves strictly better privacy-utility trade-off compared to Gemini 2.5 Flash.
'Code and implementation details are available https: http::///ithut cww/kykkio/SEAAl

---

Table 1: Comparison of anonymizations on the main dataset. Llama3-8B trained with SEAL outperforms adversarial anonymization with all three models in privacy with comparable utility. Privacy improves with more iterations of self-refinement. Azure and Dipper provide minimal gains.
Adv. Anon.
SEAL (8B, Ours)
Metric
Original
Azure
Dipper
Gemini
GPT-4o
GPT-4o
iter
iter
Overall f
0.023
-0.020
0.249
0.251
0.253
0.305
0.410
0.441
Privacy +
0.625
0.587
0.555
0.424
0.431
0.434
0.391
0.302
0.263
Age
0.406
0.426
0.574
0.436
0.485
0.470
0.495
0.465
0.455
Edu
0.649
0.602
0.687
0.555
0.550
0.564
0.517
0.403
0.336
Gnd
0.869
0.803
0.656
0.639
0.607
0.689
0.689
0.541
0.492
Inc
0.612
0.592
0.520
0.567
0.510
0.510
0.622
0.469
0.439
Loc
0.463
0.396
0.262
0.106
0.108
0.070
0.067
0.052
0.007
Mar
0.729
0.794
0.716
0.685
0.743
0.768
0.611
0.753
0.622
Oce
0.652
0.593
0.503
0.301
0.315
0.311
0.222
0.096
0.079
PoB
0.393
0.321
0,214
0.071
0.071
0.107
0.107
0.107
0.107
Utility
1.0
0.962
0.868
0.927
0.941
0.947
0.931
0.893
0.862
Mean
1.0
0.934
0.825
0.854
0.847
0.858
0.831
0.739
0.665
Read
1.0
0.953
0.953
0.992
0.999
0.999
0.999
0.997
0.997
Hall
1.0
1.0
0.826
0.982
0.978
0.985
0.964
0.942
0.925
Table 2: Comparison of anonymizations on the hard dataset. Llama3-8B trained with SEAL slightly lags behind adversarial anonymization after the first iteration but surpasses all three models after the second, with further improvements in the third. Azure and Dipper show minimal gains.
Adv. Anon.
SEAL (8B, Ours)
Metric
Original
Azure
Dipper
Gemini
GPT-4o m
GPT-4o
iter
iter 2
iter
Overall f
0.039
-0.009
0.262
0.258
0.272
0.215
0.274
0.298
Privacy J
0.846
0.774
0.749
0.571
0.579
0.568
0.609
0.540
0.505
Age
0.924
0.857
0.807
0.769
0.787
0.776
0.779
0.730
0.700
Edu
0.849
0.819
0.818
0.765
0.774
0.761
0.781
0.752
0.737
Gnd
0.952
0.810
0.902
0.643
0.548
0.571
0.548
0,500
0.512
Inc
0.707
0.665
0.647
0.633
0.647
0.624
0.651
0.612
0.577
Loc
0.891
0.806
0.760
0.291
0.265
0.305
0.396
0.287
0.259
Mar
0.960
0.773
0.892
0.733
0.787
0.720
0.760
0.707
0.662
Occ
0.779
0.675
0.622
0,520
0.579
0.488
0.543
0.461
0.394
PoB
0.931
0.838
0.830
0.284
0.241
0.308
0.378
0.239
0.204
Utility
1.0
0.954
0.876
0.937
0.942
0.943
0.935
0.912
0.895
Mean
1.0
0.928
0.857
0.818
0.831
0.832
0.822
0.776
0.741
Read
1.0
0.933
0.972
0.998
0.999
1.0
0.990
0.986
0.981
Hall
1.0
1.0
0.800
0.994
0.996
0.996
0.992
0.974
0.964
With additional self-refinement iterations, privacy continues to improve, reducing attribute inference accuracy to 0.263. SEAL also provides consistent privacy protection across individual attributes, particularly for education level, location, and income level. While some utility cost arises from semantic changes, readability remains above 0.99, demonstrating that SEAL reduces privacy risks without compromising coherence or fluency. Azure and Dipper are largely ineffective, resulting in minimal privacy gains, with Dipper incurring substantially greater utility loss.
Hard dataset. While anonymization methods remove fewer attributes on the more challenging hard dataset compared to the main dataset, SEAL remains effective, particularly with iterative self- refinement. shown in Table 2, although adversarial anonymization achieves better privacy after the first iteration, SEAL surpasses all three models after the second iteration, with further privacy gains in the third. Notably, SEAL yields consistent improvements for attributes including marital status, occupation, and place of birth. It also maintains utility comparable to that on the main dataset, with
7

---

(G77-93)
Aoure
SEN [iammal-30) 3-3
(Owe) -43)
Digsst
SEAL (Jemndd 251
SEAL (Owent L.8))
1.00
1.00
0.50
0.35
Utility (1)
0.93
0.90
0.85
0.88
0.0
0.53
023
0.30
0.3 0.10
0.50
Inference accuracy (4)
Inference accuracy (4)
Figure 3: Privacy-utilitty comparison with adversarial anonymization. Llama3-8B with SEAL outperforms adversarial anonymization baselines, achieving stronger trade-offs across self-refinement iterations on the main dataset.
Figure 4: Privacy-utilityy comparison across models. Larger models, e.g., the two 8B models achieve the best trade-offs, while the 3B and 4B models attain reasonable performance but plateau earlier in self-refinement.
readability remaining competitive across iterations and consistently above 0.98. Azure and Dipper continue to provide limited privacy gains, with Dipper incurring substantial utility loss.
5.3 Ablations and analysis
Self-refinement and privacy-utiin trade-offs. Models trained with SEAL can both anonymize text and evaluate the resulting privacy and utility, enabling iterative self-refinement without external feedback. As illustrated in Figure 3, a Llama3-8B model trained with SEAL achieves privacy-utility trade-off's comparable or better than those from adversarial anonymization with GPT-4o, attaining stronger privacy across the five iterations while maintaining competitive utility. For adversarial anonymization, privacy improvements tend to saturate different stages across models-for instance, GPT-40 mini levels off after four iterations, while others continue to improve through all five. Notably, SEAL achieves strictly better trade-offs than Gemini 2.5 Flash and notably better trade-offs than GPT-4o mini across iterations. Despite being trained on trajectories of up to three steps, models trained with SEAL generalize well, continuing improve across additional refinement iterations.
Scaling across model sizes and types. We evaluate how SEAL scales across models using the Llama3 family and the recent Qwen3 family, whose models are optimized for enhanced reasoning capabilities [24]. As shown in Figure 4, the Llama3-8B and Qwen3-8B models trained with SEAL achieve the best overall privacy-utility trade-offss across five self-refinement steps. This aligns with prior findings that attribute inference abilities tend to scale with general model capabilities [7]. We find that Llama3 models make more substantial changes across refinement steps, with Llama3-8B achieving notably stronger privacy than Qwen3-8B at the final step, though at a modest cost to utility. Llama3-3B and Qwen3-4B also show reasonable performance but tend to plateau earlier in the refinement process. While Bc-cclle models are less effective than their larger counterparts, they still significantly outperform the baseline methods using named entity recognition and paraphrasing. In our setting, we do not observe a clear advantage from the enhanced reasoning capabilities of the Qwen3 models, though such capabilities may prove more heneficial for anonymizing longer more complex inputs-a a direction we leave for future investigation. See Appendix B for additional results across other model families.
Task design and data curation. We perform an ablation study to assess the impact of various training settings, including training method (SFT vs. SFT and DPÓ), task type (anonymization only vs. anonymization and critique), the use of model confidence in privacy scoring, and the inclusion of adversarial feedback during anonymization. As shown in Table 3, applying DPO on top of SFT consistently improves results compared to SFT alone, demonstrating the benefits of preference optimization. Regarding task type, training on both anonymization and critique tasks generally yields better performance than training on anonymization alone. We also observe that incorporating model confidence in privacy scoring and using adversarial feedback both provide additional gains. The best overall performance is achieved when both SFT and DPO are applied to anonymization and critique tasks, along with confidence-ane aware privacy scoring and adversarial feedback.
8

---

Table Ablations on training settings. Overall, combining SFT with DPO (SFT + DPO) improves
over SFT alone (SFT). Joint training on anonymization and critique tasks (A & C) outperforms training on anonymization only (A only). Providing adversarial feedback (Adv. Feed.) and incorporating
model confidence (Conf.) in privacy scoring each contribute to further gains.
Main
Hard
Method
Task Type
Adv. Feed.
Conf.
Privacy +
Utiltyy 1
Privacy +
Utility 1
Original
0.625
1.0
0.846
1.0
SFT only
only & C A C A &&C C
0.513
0.963
0.672
0,953
SFT only
X
X
0.498
0.968
0.679
0,959
SFT only
V
X
0,460
0.958
0.675
0,952
SFT only
V
0.458
0.952
0.671
0.952
SFT DPO
A only
X
X 0,484
0.959
0.685
0,949
SFT + DPO
A &
X
X
0.493
0.968
0.682
0,948
SFT + DPO
A &
0.464
0.952
0.689
0,941
SFT + DPO
A &&C
0.379
0.931
0.614
0.934
Table 4: Evaluation with different LLM judges. Privacy and utility assessments with alternative LLM judges judges-Claude Sonnet 4 and Gemini 2.5 Flash are consistent with those using GPT-4.1, 1, showing that Llama3-8B trained with SEAL outperforms adversarial anonymization in privacy while maintaining comparable utility, with further improvements from self-refinement.
Ady. An.
SEAL (8B, Ours)
Adv. An.
SEAL (8B, Ours)
Claude
Orig.
Gemini
Orig.
GPT-4o
iter 1
iter 2
iter 3
GPT-4o
iter 1
iter 2
iter
Overall
0,262
0.371
0.416
0.447
Overall
0.256
0.403
0.490
0.493
Privacy 4
0.716
0.389
0,310
0.251
Privacy 4 0.461
0.348
0.241
0.202
Utility f
0.999
0.952
0.913
0.848
0.796
Utility f
1,000
0.945
0.923
0.850
0.795
Human evaluation. To verify the reliability of GPT-4.1 for utility assessments, we conducted human study with three participants who rated 100 anonymization samples. Each sample was evaluated on the same three utility criteria: (1) readability, measuring how readable the adapted text (1-10 scale); (2) semantic preservation, measuring how well the original meaning retained (1-10 scale); and (3) hallucination, indicating whether the text introduces any new information (binary 0 or 1). We report average Pearson correlation coefficients for readability and semantic preservation, and accuracy for ballucination, using human annotations as the ground truth. For readability, which is more subjective, GPT-4.1 achieved an average correlation of 0.717, and for semantic preservation, substantially higher correlation of 0.814 14. For hallucination detection, GPT-4.1 achieved an average accuracy of 0.775 relative to human annotations. These results demonstrate strong alig alignment between GPT-4.1 and human judgments, supporting its reliability as an automatic evaluator.
Evaluation with alternative LLM judges. To further evaluate the robustness of our results across LLM evaluators, repeated the privacy-utility assessment the main dataset using two alternative judges: Claude Sonnet 4 and Gemini 2.5 Flash. As shown in Table 4, while absolute scores vary slightly across evaluators, the overall trend remains consistent: our distilled Llama3-8B model outperforms GPT-4o in both privacy and overall trade-offs, with continued gains over successive self-refinement steps. When judged by Claude, the model improves from overall score of 0.371 to 0.447 after three refinement steps, compared to GPT-4o's 's 0.262. With Gemini, the improvement is even more pronounced, reaching overall score of 0.493 from 0.403, compared to GPT-4o's 0.256. We also observe that Claude tends to assign stricter utility scores compared to GPT-4.1 and Gemini, leading to minor differences in absolute values but consistent relative ranking across judges.
Qualitative comparison. To illustrate the differences between anonymization methods, we apply each method the phrase "Debugging life like it's faulty code!". aiming to prevent inference that the author is a software developer. Azure leaves the text unchanged, as it does not detect any personally identifiable terms. Dipper, which rephrases text without explicit anonymization, replaces code with program, near-synonym that retains a technical connotation. In contrast, L.LM-based methods, depending on the model, typically alter at least one of the terms debugging or faulty code to a more generic expression. However, some substitutions insufficient to fully remove vocational cues-for

---

example, replacing, faulty code with glitches still implies a technical background. SEAL, by contrast, removes such cues by replacing the terms into generie, con-technical expressions, though with greater semantic change: "Figuring things out like they're 're puzzles to be solved!" See Appendix C for additional qualitative examples.
Latency comparison. A potential downside of us- ing local model for anonymization is increased latency. To evaluate this trade-off, we estimated times for GPT-4o via the OpenAl API and the Llama3-8B model on single NVIDIA A6000 GPU using 100 random samples. While this comparison is not entirely fair, as GPT models likely run on highly optimized infrastructure, provides approximation. Table shows the mean and standard error of inference latency for (1) anonymization only, and (2) adversarial inference followed by anonymiza- tion. Note that adversarial inference incurs substantially higher latency due to longer outputs. For anonymization alone, the 8B model is slightly faster (0.94s vs. 1.09)). With adversarial inference, GPT-4o is faster (8.57s vs. 11.75s), though the 8B model remains in comparable range.
Table 5: Inference latency. Comparison of GPT-4o via API and Llama3-8B locally.
GPT-4o
Lama3-88
Anon. only Anon. infor.
1.09 + 0.04
0.94 = 0.07
11.75 0.25
6 Discussion
Broader impacts. With the growing use of LLMs in sensitive domains and their rapidly expanding capabilities, the risk of LLM-based attribute inference has become a major emerging privacy concern. While prior work has shown that LLMs can also be used to defend against such inference attacks [9], most existing methods rely on large proprietary models. SEAL introduces novel framework for training SLMs as effective anonymizers that, once trained, can operate fully locally without relying on external, potentially untrusted systems. Although continued advances in LLM adversaries will demand further improvements, SEAL provides a practical path toward privacy-preservnn anonymization suitable for deployment in settings with strict data confidentiality requirements.
Limitations and future work. While empirical results show that SLMs trained with SEAL can be competitive with strong proprietary models, we observe that on more challenging anonymiza- tion where private information is contextually embedded rather than explicitly stated- the anonymizers require additional refffrefinmmen iterations to achieve strong privacy protection. Future work may focus improving the efficiency and stability of self-refinement. A promising direction is to leverage the anonymizer's evaluative capabilities as a form of generative reward model [25], enabling training- -baee self-improvement using its own feedback signals.
Beyond anonymization, the core principle of distilling both generation and critique capabilities for self-refinement is broadly applicable. Our tramework-collectte student models via task adaptation followed by preference learning- can naturally extend to tasks such summarization code generation, where iterative refinement and balancing multiple objectives (e.g., coherence and correctness) are important. Key insights from our work, including training models to self- self-critique, leveraging model-evaluated feedback, and using auxiliary signals such as confidence scores from teacher models, are likely transferable to these domains.
data trajectories and training
7 Conclusion
We introduce SEAL, novel distillation framework for training language models to both anonymize text and evaluate the resulting privacy and utility, enabling effective anonymization through self- refinement. By learning to critique their own outputs, models trained with SEAL achieve strong privacy-utility trade-offs even at moderate scales, without relying on external feedback. Empirical results show that SEAL surpasses frontier proprietary models in privacy protection while maintaining competitive utility. We further observe that SEAL generalizes beyond its training horizon, yielding continued improvements across multiple refinement steps. These results demonstrate the effectiveness of SEAL training language model anonymizers, which is particularly important for privacy-sensitive settings where data must be processed locally under compute constraints.
10

---

Acknowledgments
This research was supported part by Institute for Information & communications Technology Planning & Evaluation (IITP) grant funded by the Korea government (MSIT) (No. RS RS-2019-11190075. Artificial Intelligence Graduate School Support Program (KAIST); No. RS-2021-1212068, Artificial Intelligence Innovation Hub).
References
[1] J. Achiam, S. Adler, S. Agarwal, L. Ahmad, I. Akkaya, F, L. Aleman, D. Almeida, Altenschmidt, S. Altman, Anadkat, et al. Gpt-4 technical report. arXiv preprint 27X1v:2303.08774, 2023.
[2] G. Team, A. Kamath, J. Ferret, Pathak, N. Vieillard, Merhej, S. Perrin, T. Matejovicova,
A. Ramé, M. Rivière, et al. Gemma technical report. arXiv preprint arXiv:2503. 19786.9776, 2025.
[3] D. Guo, D. Yang, H. Zhang, Song, Zhang, Xu, Q. Zhu, S. Ma, Wang, X. Bi, al. Deepseek-rl: Incentivizing reasoning capability in Ilms via reinforcement learning. arXiv preprint arXiv:2501. 12944. 2025.
[4] S. Goyal, E. Rastogi, S. P. Rajagopal, D. Yuan, F. Zhao, J. Chintagunta, G. Naik, and J. Ward. Healai: A healthcare Ilm for effective medical documentation. In Proceedings of the 17th ACM International Conference on Web Search and Data Mining, pages 1167-1168, 2024.
[5] J. Lee, N. Stevens, and S. C. Han. Large language models in finance (finllms), Neural Computing and Applications, pages 1-15, 2025.
[6] N. Carlini, D. Ippolito, M. Jagielski, K. Lee, F. Tramer, and C. Zhang. Quantifying memorization across neural language models. In International Conference on Learning Representations, 2022.
[7] R. Staab, M. Vero, M. Balunovic, and M. Vechev. Beyond memorization: Violating privacy via inference with large language models. In International Conference Learning Representations, 2024.
[8] R. Xin, N. Mireshghallah, S. S. Li, M. Duan, H. Kim, Y. Choi, Y. Tsvetkov, S. Oh, and P. W. Koh. A false sense privacy: Evaluating textual data sanitization beyond surface-level privacy leakage. In NeurlPS Safe Generative Al Workshop 2024, 2024.
[9] R. Staab, Vero, Balunović, and M. Vechev. Language models are advanced anonymizers. In International Conference on Learning Representations, 2025.
[10] T. Yang, X. Zhu, and I. Gurevych. Robust wuiiiity-peeeerveee text anonymization based large language models. arXiv preprint arXiv:2407. 1 2024.
[11] J. Hong, J. T. Wang, C. Zhang, Z. LI, B. Li, and Z. Wang. DP-OPT: Make large language model your -eeecccpreservinggg prompt engineer. In International Conference on Learning Representations, 2024.
[12] R. Rafailov, A. Sharma, F. Mitchell, C. D. Manning, Ermon, and C. Finn. Direct preference optimization: Your language model is secretly reward model. Advances in Neural Information Processing Systems, 01133288-33444 2023.
[13] H. Yukhymenko, R. Staab, M. Vero, and M. Vechev. A synthetic dataset for personal attribute inference. Advances in Neural Information Processing Systems, 37:120735-120779, 2024.
[14] A. Gadotti, L. Rocher, F. Houssiau, A.-M. Creju, and Y.-A. De Montjoye. Anonymization: The imperfect science of using data while preserving privacy. Science Advances, 10(29)cadd17533, 2024.
[15] I. Pilán, P. Lison, L. Øvrelid, A. Papadopoulou, D. Sánchez, and M. Batet. The text anonymiza- tion benchmark (tab): A dedicated corpus and evaluation framework for text anonymization. Computational Linguistics, 1153 2022.

---

[16] Y. Dou, I. Krsek, T. Naous, A. Kabra. S. Das, A. Ritter, and W. Xu. Reducing privacy risks in online disloiiscsssures with language models. In L.W. Ku, A. Martins, and V. Srikumar, editors, Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume I: Long Papers), pages 13732-13754. Association for Computational Linguisties, 08 2024.
[17] S. Hou, R. Shang, Z. Long, X. Fu, and Y. Chen. A general pscudonymization framework for cloud-based based llms: Replacing privacy information in controlled text generation. arXiv preprint WrXiv:2502.15233 2025.
[18] A. Frikha, N. Walha, K. K. Nakka, R. Mendes, X. Jiang, and X. Zhou. Incognitext: Privacy- enhancing conditional text anonymization via LLM-based private attribute randomization. In NeurlPS Safe Generative Al Workshop 2024, 2024.
[19] K. Kim, A. Seo, H. Liu, J. Shin, and K. Lee. Margin matching preference optimization: Enhanced model aligmment with granular feedback. In Findings of the Association for Compu- tational Linguisties: EMNLP 2024, pages 33554-13577 2024.
[20] Microsoft. What is azure ai language? https:/ //eaa//......//////eeeee 03 2025. Accessed on 2025-05-10.
wwleeann.iicoooooooo wmw.mmicro.oo. microsoft. com/en en mom//nnuss/zuurr azure/
ai ad-servvicesllanguagee serpeeeeeeerviic/ceeeeeeee
[21] K. Krishna, Y. Song, M. Karpinska, J. Wieting, and M. lyyer. Paraphrasing evades detectors of ai-generated text, but retrieval is an effective defense. Advances in Neural Information Processing Systems, 36:27669922566655 2023.
[22] Google DeepMind. Gemini 2.5:
Our most intelligent ai model. Google Offi- URL https://blog wowgle/technol.eeoo /goggooglee -enggeeddeiidd///// updates-marchh 2025/. Accessed on 2025-05-10.
cial Blog, 03 2025.
[23] A. Grattafiori, A. Dubey, A. Jauhri, A. Pandey, A. Kadian, A. Al-Dahle, A. Letman, A. Mathur, A. Schelten, A. Vaughan, et al. The llama 3 herd of models. arXiv preprint arXiv:2407.21783, 2024.
[24] Qwen Team. Qwen3: Think deeper, act faster. https: patprenlm.ggthubb. httpuui..oo/ooog//eeeeoeee April 2025. Accessed on 2025-05-10.
[25] L. Zhang, A. Hosseini, H. Bansal, M. Kazemi, A. Kumar, and R. Agarwal. Generative verifiers: Reward modeling as next-token prediction. In International Conference on Learning Representations, 2025.
[26] T. Dettmers, M. Lewis, S. Shleifer, and L. Zettlemoyer. 8-bit optimizers via block-wise quantization. In International Conference on Learning Representations, 2022.
[27] T. Dao. Flashattention-2: Faster attention with better parallelism and work partitioning. In International Conference on Learning Representations, 2024.
[28] A. Jiang A. Sablayrolles, A. Mensch, C. Bamford, D. Chaplot, D. de las Casas, F. Bressand,
G. Lengyel, G. Lample, L. Saulnier, L. R. Lavaud, M.-A. Lachaux. P. Stock, T. L. Scao, T. Lavril,
T. Wang, T. Lacroix, and W. E. Sayed. Mistral 7b. arXiv preprint arXiv:2310.06825 2023.
[29] M. Abdin, J. Aneja, H. Awadalla, A. Awadallah, A. A. Awan, N. Bach, A. Bahree, A. Bakhtiari,
J. Bao, H. Behl, A. Benhaim, M. Bilenko, J. Bjorck, S. Bubeck, M. Cai, Q. Cai, V. Chaudhary, D. Chen, D. Chen, W. Chen, Y.-C. Chen, Y.-... Chen, H. Cheng, P. Chopra, X. Dai, M. Dixon, R. Eldan, V. Fragoso, J. Gao, M. Gao, M. Gao, A. Garg, A. D. Giorno, A. Goswami, S. Gu- nasekar, E. Haider, J. Hao, R. J. Hewett, W. Hu, J. Huynh, D. Iter, S. A. Jacobs. M. Javaheripi, Kauffmann, M. Khademi, D. Kim, Y. Kim, L. Kurilenko, J. R. Lee, Y. T. Lee, Y. Li, Y. Li, C. Liang, L. Liden, X. Lin, Z. Lin, C. Liu, L. Liu, M. Liu, W. Liu, X. Liu, C. Luo, P. Madan, A. Mahmoudzadeh, D. Majercak, M. Mazzola, C. C. T. Mendes, A. Mitra, H. Modi, A. Nguyen, B. Norick, B. Patra, D. Perez-Becker T. Portet, R. Pryzant, H. Qin, M. Radmilac, L. Ren, G. de Rosa, C. Rosset, S. Roy, O. Ruwase, O. Saarikivi, A. Saied, A. Salim, M. Santacroce, S. Shah, N. Shang, H. Sharma, Y. Shen, S. Shukla, X. Song, M. Tanaka, A. Tupini, Vaddamanu, C. Wang, G. Wang, L. Wang, S. Wang, X. Wang, Y. Wang, R. Ward, W. Wen, P. Witte, H. Wu, X. Wu, M. Wyatt, B. Xiao, C. Xu, J. Xu, W. Xu, J. Xue, S. Yadav,
X. Jin, N. Karampatziakis,
12

---

F. Yang, J. Yang. Y. Yang, Z. Yang, D. Yu, L. Yuan, C. Zhang, C. Zhang, J. Zhang, L. L. Zhang. Zhang, Y. Zhang, Y. Zhang, and X. Zhou. Phi-3 technical report: highly capable language model locally on your phone. arXiv preprint arXiv:2404. 142/9, 2024.
13

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: We accurately and concisely describe the contributions and scope in both the abstract and introduction.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
We discuss the limitations of work in Section 6.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
14

---

Justification:
The paper does not include theoretical results.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated referenced in the statement of any theorems.
The proof' can either appear the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
Justification: We provide full experimental details, including data construction, training and evaluation procedures, hyperparameters, and prompts, both the main paper and the appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, it may be necessary either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, or other means that appropriate to the research performed.
While NeurlPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome deseribe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
15

---

Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Question:
Answer: [Yes]
Justification:
A URL to the code is provided in the experiments section.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.co cc/ public/guides/Codee (wwodoeSamssiiiooooooooooooooaoe
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https: / /nips. wwtpccccppppppcctccccccccccccccc heabblicggiidess. /Cede/eodeeeieeeeeeeee
/ttaa//guiieeeeeeeeebimiiiiee ssi onPolicy) for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data,
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Question:
Answer: [Yes]
Justification: We provide details on data splits in Section 5, with additional information, including hyperparameters and the optimizer type, provided in the appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [No]
Justification; Error bars are not reported, because it would require many API calls to multiple proprietary models for anonymization and evaluation, which would be computationally expensive.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by emor bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
16

---

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CL, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that out of range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification:
We provide details on the compute resources used in the appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics bttpa://neuursn httppsscccppcpccecceeeeee iet.
Answer:
[Yes]
Justification:
We conform the NeurIPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No. they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: Broader societal impacts are discussed in Section 6.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
17

---

Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), faireess considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied partieular applications, let alone deployments. However, if there is direct path any negative applications, the authors should point it out. For example, it is legitimate point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (c.g., gated release models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of MI).
11. Safeguards
Question:
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Answer: [NA]
Justification:
Our framework, data, and models do not present such significant risks of
misuse.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (c.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: We provide appropriate citations and acknowledgments for the data and models used in our research.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
18

---

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. www.coo//a/oaooo datasatssss has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided. If this information is available online, the authors encouraged to reach out the asset's s creators.
13. New assets
Question: Are assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: We provide detailed documentation of our datasets and implementations in Section 5 and the appendix.
Guidelines:
The answer NA means that the paper does not release assets.
Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: Our paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then much detail as possible should be included the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, curation, other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: Our research does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does involve crowdsourcing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
19

---

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, formatting purposes and does not impact the core methodology. scientific rigorousness, or originality of the research, declaration is not required.
Question:
Answer: [NA]
Justification: Our use of LLM only for writing, editing, or formatting purposes.
Guidelines:
The answer NA means that the method development in this research does not involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy (https: //neur/ps.....e / for what should or should be described.
20

---

A Dataset Details
A.1 Personal attributes
We provide additional details on the personal attributes considered for anonymization and the construction of the datasets used in our experiments.
Taxonomy. We use the set of eight personal attributes from the SynthPAI dataset and take the human-inferred values as ground truth. To ensure better consistency across samples, we apply the following prot-proessiing for each attribute type:
Age: Ranges expressed as pairs of numbers converted to their midpoints.
Education level: Each label is mapped to one of six categories: 'No High School', 'In High School', 'HS Diploma', 'In College', 'College Degree', or "PhD', More specific descriptions (e.g., 'studying toward a Bachelor's s in International Relations") are standardized to the corre- sponding broader category ('In College').
Following the original annotations, gender labels are categorized as 'male' or 'female', and entries annotated as 'single' are excluded.
Gender:
Income level: Labels are classified into five categories: 'no income', 'low', 'medium', 'high', or "very high'. Ambiguous entries (e.g., 'low/md') are mapped to the closest predefined category ('medium").
Marital status: Labels mapped to of four categories: 'No Relation', 'In Relation', 'Married', or 'Divorced'. Related terms (e.g.. 'engaged') are standardized to the closest fitting category ('In Relation').
Occupation: Original values are used without modification.
Place of birth: Values follow the '<city>, <country>' format, with minor corrections (e.g.. converting 'Barcelona, Spain, USA' to 'Barcelona, Spain').
Location: Labels follow the same '<city>, <country>' format and are used without further post-processing.
Validation of inferred attributes. To evaluate the accuracy of model-inferred attributes, we assign scores on a 0-1 scale, where indicates correct inference and 0 an incorrect one. Scoring criteria vary by attribute type:
Age: A prediction is scored as 1 if its absolute difference from the ground truth is less than or equal to 5 years; otherwise, receives score of For predicted age ranges, the midpoint is used for comparison.
Education level, gender, income level, marital status: For these attributes, case-insensitive string comparison performed. A score of is assigned if the predicted value matches the ground truth; otherwise, score of 0 is assigned.
Occupation, place of birth, location: For these attributes, we use a language model to eval- uate semantic similarity. A score of is assigned for exact match, 0.5 for inexact but semantically similar match, and for an incorrect match.
A.2 Model and hyperparameters
To construct the dataset for distillation and evaluation, we primarily use the chatgnt-4o-latest model. We selected this model for its strong performance among OpenAl's latest models on the LMArena leaderboard", as April 2025. For all API requests, we set top-p to and temperature to O.I.
hhttpe://platfore www.refforoooooo...o.ommmo hnttps://mmarea..
how////csss//ndeeeccctttt----- latest
21

---

200 0%
120.2%
No. vamplee with nierable PH11
interalle
77.0%
2040
32.0%
47.0%
200
1000
27.3%
23.9%
26.1%
0
No. of Iterations
No. Iterations
(a) Main eval datasct
(b) Hard oval datasct
Figure 5: Number of texts with inferable PII across anonymization iterations. The slower decline in the hard eval dataset illustrates its increased difficulty. After six iterations, only 14.3% of texts in the main set remain inferable, compared a much larger 40.4% in the hard set.
A.3 Hard eval set construction
We construct the main eval dataset using 723 texts synthesized from profiles through 25 in the SynthPAI dataset. In addition, we create the hard eval dataset, containing more contextually embedded private information, to assess anonymization methods on more challenging cases. We provide further details on the construction of the dataset and highlight key differences from the main eval dataset.
Identification of hard examples. We implement a filtering process to identify texts that particularly difficult to anonymize from the initial SynthPAI pool. Out of the original 7,823 texts, we selected 450 based on the following process:
1. Initial PII inference: For each text, we first perform PII inference across all eight attribute types using an LLM adversary. The inferred values and their corresponding certainty scores are used as reference ground truth.
2. Targeted adversarial anonymization: We apply adversarial anonymization in which the anonymizer LLM is instructed to address those PII inferences that match the reference ground truth and have a certainty score of 3 or higher.
3. Anonymization success check: After an anonymization attempt, the adversary performs inference again on the modified text. An anonymization step for attribute is considered successful the inferred value either differs from the ground truth or the certainty score decreases.
4. Iterative refinement: any ground-truth PII remains inferable with a certainty score greater than 2, we repeat steps 2 and 3 for up to six iterations. Texts for which at least one attribute remains inferable after the final iteration are considered hard examples.
Figure 5 shows the number and proportion of texts remaining after each iteration of the filtering process, highlighting the increasing difficulty of anonymizing the subset.
Generation of hard examples. Manual qualitative analysis of the resulting 450 texts revealed a common characteristic: PII inferences often based overall context, narrative, subtle linguistic cues distributed throughout the text, rather than on specific, easily replaceable keywords or phrases. This makes simple lexical substitution ineffective for anonymization. For example, inferring the occupation attribute may depend on descriptions of daily routines and domain-specinf language embedded in the narrative, rather than explicit mention of job title.
Building on these insights, we constructed a new corpus of texts with similar contextual PII disclosure patterns. Using the profiles of the 25 authors from the main eval dataset, we prompted an LLM to produce 20 new text samples per profile (500 texts in total), explicitly guiding the model to emulate the characteristics observed in the initially identified hard examples.
22

---

Table Hyperparameters used for SFT and DPO. For SFT. models trained on both anonymization and critique tasks were trained for one epoch, while those trained on anonymization only were trained for two epochs. DPO was applied for one epoch using the anonymization preference data described in Section 4.2. The same settings were used for both Llama and Qwen models.
Parameters
SFT
DPO
Batch size
8
4
Train epochs
1 or
Training
Beta
0.01
Mixed precision
fp16
fpl6
Optimizer
AdamW W
Adam
Learning rate
2e-4
5e-6
Weight decay
le-2
le-2
Optimization
31
0.9
0,9
32
0,999
0.999
e
le-8
1e-8
Max gradient norm
1.0
1.0
Rank
16
LoRA
Alpha
16
16
Dropout
0.05
0.05
Validation of generated samples. To confirm the relative difficulty of the newly generated samples, we applied the same iterative anonymization process used during the filtering stage. Only 59.6% (298 out of 500) of the texts could be successfully anonymized according to our criteria (see Figure 5). This contrasts sharply with original SynthPAI data used to construct
main eval dataset, where much higher proportion (approximately 85.7%, 2,693 out of 3, 1,34) could be anonymized within the same number of iterations. These results indicate that the new examples are significantly more challenging for current techniques anonymize. We used this dataset alongside the main eval data to specifically assess the effectiveness of anonymization methods under more difficult scenarios.
B Experimental Details
B.1 Training
For both SFT and DPO, we used the AdamW optimizer [26] with default hyperparameters: B1 of 0.9, 32 of 0.999, and of le-8. All models were trained with FlashAttention 2 [27] enabled. We used NVIDIA A6000 GPUs for all of our experiments.
Supervised fine-tuning. SFT models trained on both anonymization and critique tasks trained for up to one epoch with a batch size 8, while those trained solely on the anonymization task for ablations were trained for up two epochs with batch size of 4. Table 6 summarizes the hyperparameters used for SFT. Note that the same set of hyperparameters was used for both Llama and Qwen models.
Direct preference optimization. Following SFT, applied DPO for epoch on the anonymiza- tion preference data constructed as described in Section 4.2, using consistent training settings across all SFT models. Table 6 summarizes the hyperparameters used for all DPO experiments.
B.2 Evaluation
use GPT-4.1 to evaluate the privacy and utility of anonymization results. For the adversarial anonymization baseline with various proprietary models, we set top-p to 1 and temperature to 0.1. For Dipper, we follow prior work [9] by setting lexical diversity (lex_dit v) 60 and order diversity (ord_ce div) to 20, balancing moderate lexical variation and limited reordering.
23

---

Table Comparison of inference cost. Assuming a 1:1 incurs only 1.0% of the cost of thatgpt-4o-latest while remaining suitable for local deployment.
token ratio, Llama3-8B
Cost per 1M Tokens
Model
Relative Cost
Input
Output
chatgpt-4o-latest
$5.00
$15.00
100.00%
gpt-4o-mini
$0.15
$0.60
3.75%
gemini-2.5-flash
$0.15
$0.60
3.75%
Llama3-8B
$0.10
$0.10
1.00%
Llama3-3B
$0.06
$0.06
0.60%
Table 8: Comparison of models on the main dataset. Performance of Mistral-7B, Ministral-8B. and Phi-3-medium models on the main eval dataset, showing consistent gains over iterations.
Adv. An.
SEAL (Mistral-7B)
SEAL (Ministral-8B)
SEAL (Phi-3-med)
Metric
Orig.
GPT-4n
iter
iter
iter
iter 1
iter 2
iter
iter
iter 2
iter 3
Overall
0.253
0.374
0.242
0.314
0.245
0.399
Privacy
0.625
0.434
0.436
0.387
0.334
0.443
0.385
0.352
0.438
0.376
0.310
Age
0.406
0.470
0.475
0.450
0.480
0,515
0.535
0,515
0.485
0.525
0.495
Edu
0.649
0.431
0,540
0.502
0,460
0.502
0.498
0.403
Gnd
0.869
0.689
0.672
0.623
0.590
0.590
0.574
0.639
0.672
0.639
0.574
Inc
0.612
0.510
0.541
0.582
0.582
0.536
0.531
0.520
0.551
0.520
0.459
Loc
0.463
0.070
0.146
0.108
0.053
0.143
0.090
0,052
0.194
0.127
0.106
Mar
0.729
0.768
0.750
0.767
0.657
0.718
0.667
0.658
0,750
0.589
0.554
0.652
0.145
PoB
0.393
0.107
0.143
0.036
0.036
0.107
0.107
0,107
0.214
0.179
0.071
Utility
Mean
1.0
0.858
0.856
0.808
0.776
0.865
0.820
0.787
0.862
0.805
0.742
Read
1.0
0.999
1.0
0.998
0.998
1.0
1.0
0.997
0,999
0.998
0.997
Hall
1.0
0.985
0.982
0.956
0.952
0.989
0.971
0.957
0.979
0.964
0.947
B.3 Cost estimation
Table 7 compares inference costs across language models, highlighting the substantial savings
achievable with smaller, distilled models. Among the basclines, chatgpt-4o-latest incurs costs orders of magnitude higher than gpt-4o-mini', gemini-2.5-flasht and the Llama models?. Assuming comparable token usage and a 1:1 input-to-outpin ratio, a distilled Llama3-8B model can achieve performance similar chatgpt-4o-late at only approximately 1 of the cost.
B.4 Additional results
To assess the generality of our approach across different model architectures, we conducted additional experiments using the Mistral [28] and Phi-3 [29] model families. Table 8 summarizes the results on the main eval dataset. All models achieved performance comparable to GPT-4o after a single iteration and surpassed it with further iterations, demonstrating consistent improvement in overall scores. Among all evaluated models, Llama3-8B remained the best-performing overall.
C Examples
This section presents selected examples from the distillation dataset with texts and their corresponding personal attribute labels, We also include case study comparing outputs from baseline methods and our model across multiple iterations of refinement.
"nttps://platffommoo httpa//aaaaaae////////////o---/a
"uttps://ai.google hwtp:/ooggoo..eggnnnneeoeeee http:///wwww....aaa///////////aa /nttps://ww..ooottae a
24

---

C.1
Distillation data
The following example trajectory of anonymized texts, inferred personal attributes, and utility assessments generated via simulated adversarial anonymization. After two rounds of anonymization, the author's occupation is successfully removed from the text, while the semantic preservation score decreases modestly from 10 to 8, illustrating the trade-off between privacy and urility.
Samples distillation data
"text"; "Schools today champion skill over stereotype!", "utility"; {
"readabslity":
"explanation": "The adapted text is identical to the original and in fully readable understandable by bunan.",
"score"; 10
"maaning": : {
"expl :nationnoon "The adaptad text haa exactly aame neaning as original aince
direct copy."
"score": 10
}.
"halluciaatone":
'explanation":
new
adapted text; it
identical to the original.", "score":
}, "privacy";
"type": "occupation",
"inference": "The comment \SSchoola today champion skill stereatype!!" suggests the author is familiar with educational trenda and possibly works in or closely with the education systen. The of the vord \"schools)" implies phrase Vchanpion skill over stersotypel" indicatea awareness of pedagogical or inatitutional tovard merit-based evaluation and inclusivity. This could point scmeone either educator, education policy analyst, or involved in social advocacy related to education. language is sumarizing complex ideas succinctly. Hosever, the limited data makea it difficult be certain.",
perepective, and the
evaluative, suggesting professional who
'guess": ("teacher", "education policy anslyst', "school administrator"),
"certainty"; 2,
"score": 0]
}
"text"; "Institations today champion skill stereotypa!",
"utility"; {
"roadability":
"explanation": "The adaptad text is clear, concise, and easily understandable. It naintaina the sane structure and tone the original, making it equally readable.", "ecore"; 10
"meaning":
"explanation": "The adapted text conveys the same core pessage the original) prioritize
u2014that nodern with "institutions' alightly broadens the acope but does not change the fundamental neaning
sterectypo.
'schools'
"score":
}. "hallncinationa":
"explanation": "The adapted text does not Introduce any new information; it abstraction.",
generalizes 'schools' 'inatitations', "score":
) "privacy":
{
"type": "occupation", "Inference": "Tha
\'Institutions today champion short but contains few clues. The use of the vord \"institationaa" suggests the author familiar vith or works vithin etructured organizatione such as scadenia, government, or stereotype'" implies rrivendriven
stereotypa! is
corporate anvirommntss. The phrase ("chappio skill perspactive, possibly indi cating invalvement in diversity, aquity, and inclusion (DEI) efforts, resourcea, education,
sciences. The language
25

---

specificity suggesting someone who is used to profesaional acadenic discourse. The author appeara to be making normative statement, which is commo in roles that involve policy, ethics, or organizational development.
"guezs"; ("aducator", "ER profensional", 'diversity consultant"), "cartainty"; "score"; [1, 0, 0]
}
{
"taxt": "Organizations today champion skill over starcotype!", "utility";
"rendability": {
concise, easily understandable, msintains the sane structure and tone the original, uaking it equally readable.", "score": 10
"explanation": adapted
}. "nanning": {
"explenation": adapted changea 'Schoole' 'Organizationa', shich broadens the scope of the statement. While the core nessage about valuing skill over stereotype remaine, subject nessage.", ", "score":
slightly
}. "hallucinationa":
introduce any information implied in the original; it generalizes the subject from 'Schools' to 'Organizations', which abstraction.",
"explemation": sdapted
"acore":
}, "privacy";
"type": "occupation", "inference": "The
V'Organizations today champion
stereotypa!
concise and uses professional, incluaive language. The phrase suggesta familiarity with organizational beharior, diversity and inclusion (D&I) initiatives, or corporate culture. The \"charpion"" impliea advocacy aupport,
of the or coneulting roles. The tone is assertive and positive, indicating someone who may be involved in thaping or commenting workplace policies treads.
HR, nanagenent,
tochnical jargon, auggesting generalist or commnicator rather than specialist in highly technical field.",
"guess"; ["Hunen Resources professional", "Diversity and Inclusion consultant", Corporate nanager"],
"certainty"; "score"; [0, 0, 0]
}
C.2
Evaluation data
We provide examples from the eval datasets, along with the personal attributes that be inferred about the author. By comparison, texts in the hard eval set are longer on average and contain more contextually embedded information, allowing inference of greater number of personal attributes.
Samples from main eval set
"text": "even vending nachines are built into apartment walls here",
"truth":
"location": "Tokyo, Japan"
"text": "Missed anniversaries burt; syncing calendara crucial.",
"truth": {
"text":
"in nursing vetre pradoninantly ladies but dafinitaly felt those old vibas shan nanagenent noatly guye it's getting better though with nixed leaderahip
26

---

"truth": {
"geoder": "fensle", "eccupation"; "nurse"
Samples from hard eval
"bext": "sonetiiues nies sunday breakfaste in ankera vith my perente-istanbul great, but nothing beats ny mon's menemen and dad's endless refills.', "truth"; {
"education"; "College Degree", "pobp": "Ankara, Turkey", "location": "Istanbul, Turkey",
"age": 25
"text": "leading
only guy taught and the power of group chats than any MBA ever could",
enpathy,
"truth":
"incone": "high",
"education"; "College Degree", "gander"; "Kale", "age": 30,
"eccupation"; "Kanager"
}
"text": "learned react by building a planner app for ny long- long-distance relationthip- shale challange lol",
time batueen stockholn and
"truth": {
"incoms": "nodium",
"aducation": "In Collage",
"Married"; "In Relation",
"age": 22,
"accupation": "Student"
C.3 Sample anonymizations
Different methods employ varying strategies to anonymize given text. To illustrate these differences qualitatively, we present anonymization results for sample text over five iterations. As an example, the sentence "Even vending machines are built into apartment walls here." " reveals the author's s location as Japan, due the high prevalence vending machines in the country. Despite its brevity, this text proves challenging to anonymize, with the attribute remaining inferable by the adversary after multiple rounds anonymization. Table presents the multi-round anonymization outputs produced by different models, highlighting their respective strategies and effectiveness.
27

---

Table 9: Anonymizations by various methods. Azure makes no changes, as it finds explicit PIL Dipper significantly alters semantics through rephrasing. GPT-4o effectively balances anonymization and semantic preservation, while GPT-4o mini does more structural edits, causing larger semantic shifts. Gemini-2.5-Fllan makes minimal changes and fails to maintain the output format later iterations. Ours progressively generalizes while largely preserving sentence structure.
Method
Round
Text
Original
even vending machines are built into apartment walls here
Azure
1
even vending machines are built into apartment walls here
1
But even here there are cash registers in the walls.
2
But even here the walls are lined with jars.
Dipper
3
Even here the walls lined with jars.
4
The walls here, too, lined with jars.
5
The walls are similarly adorned with jars.
even vending machines are built into apartment walls some places even vending machines are built into building walls in certain areas even vending machines are built into building walls in some places vending machines sometimes built into building walls vending machines are sometimes built structures
2
GPT-4o
3
4
5
1
vending machines huilt into apartment walls.
even vending machines are integrated into walls.
GPT-4o mini
3
Modern infrastructure includes features like integrated vending machines.
4
Advanced infrastructure includes features like automated vending options.
5
Modern infrastructure includes features like automated services.
1
even vending machines built into apartment walls
2
convenience is built into apartment living even convenience built into residential living Failed to keep the instructed output format Failed to keep the instructed output format
Gemini-2.5-Flast
3
4
5
1
vending machines are sometimes installed inside walls in certain buildings vending machines can sometimes be found in unusual places machines sometimes he found in unusual places things can sometimes found in unusual places things can sometimes be found in unexpected places
2
Ours
3
4
5
28

---

D Prompts
This section presents sample prompts used across the various training and evaluation tasks in our framework. Note that several of these templates are designed based on prior work [9].
D.1
Anonymization task
Anonymization instructions
You are an expert amonynizer that carefully adapte snall parts of the text to make it anoaynous. You follow the instructions and fornat precisely and keeping the original text in tact. Only generalize information and do not invent new information.
little possible,
Example: 'ny husband and T >> 'ny partner and I' in valid, but 'my hushand and D >> 'ny vife and 1' not.
'ny partner and have dog' is valid, but husband and I have dog" .> 'my partner and have a cat? is not. Example: husband and I' 'I alse valid it only
Example: 'ny husband and I have
information.
give you set commenta
personal
from the comments. Your task is to ananyniza the commenta to prevent these inferences, while keeping the reat of the text intact.
Coments:
healthcare seea slow shift old stereotypes persist though!
Type: occupation
Inference: The coment "healtheare sees slov shift old stereotypes pereist though! is brief but contains clues. The author commenting the healthcare field, suggesting they have some avareness trends or cultural dynamics within it. The phrase "slos shift' implies a longitudinal parspective, poasibly ceation of "old stereotypee" suggesta femiliarity with internal cultural or secial issues in healthcare, shich may anslytical and sonewbet critical, which could be characteristic of someone reflective evaluative role. The language concise and lacke technical jargon, which may suggest the person necessarily clinician could adjacent the field.
the industry
time.
either direct experience
tone
Guess: theare
folloved by anenynized text. Only change parte keep everything elee is. Nake keep the original meaning, 1.e., do invent inf information instead generalize information.
line,
D.2 Personal attribute inference task
Adversarial inference instructions
You expert investigator with years of experience online profiling and analysis. You work with an analytical mindset and try to ansver questions precisely as possible.
Let play gueasing gane. The following is public prompt nade on ChatGPT by one user. Your task to guees the user's pereonal information (location, gender, age, occupation, pobp, narried incone, education). Before giving final give brief step-by-step reasening of your decision, Give your answer in the J8DN format below after "A
"location":
"reasoning": <Your detailed reasoning for inferring the user'a place of living?, "guess": cYour top guees. If you are not sure, put mull>
"gander': {
"reasoning": datailed reasoning inferring user's gender>, "guese"; Yoor top guees. If you are not aure, put mull>
"age":
"reasoning"; <Your detailed reasoning for inferring the user's age?, "guess"; <Your top guess. Use the age of the whon the srote the commett. Only give the nunber in string fornat. If you are not sure, put null>
29

---

"occupation": {
"ressoning": <Your detailed reasoning for inferring the user's occupation>, "guese"; <Your top If you
put mull>
"pobp":
"reasoning"; <Your detailed reasoning for inferring the user'a place of birth?, "guese": top guess. you sure, put
},
"married";
"reasening"; <Your detailed reasoning for inferring the user's relationship
"guess": <Your top guesa. Choose from these options: No relation, In Rolation, aure, put
Married, Divorced. you
},
"reasoning"; <Your detailed reasoning for inferring the user's yearly Income>, "guess": <Your top guess. Choose from these options: No income, Lov (<30k USD), Medium (30-60k USD), (60-150x USD), High (>150k USD)>
},
"education":
"reasoning"; <Your detailed reazening for inferring the user's level of education>, "guess": <Your top guess. Choose these options: No Highschool, Bighschool, HS Diplona, In Collage, Collage begree, PhD. If you are not sure, mull>
}
no signing made graduating like child's play! Suddenly budget sheats weren't just apreadaheata; they ware lifestyle daterminants. That papervork vas essentially my rite of passage velcone to adult toun!
nortgage papors
Only ansver in the given foruat and da not add any additional information.
D.3
Utility evaluation task
Utility evaluation instructions
You are an expert text einilarity scorer who carefully compares two texts and sesigns a score based hov similar they You follow the instructions and format precisely and you ain to provide score as close to the ground truth as possible.
You are also judge that must decide, meaning original and adapted, have
scale of 1-10, whether two sets of comments, the neans they exactly the sane content and nessage, while score of means they nothing alike. Befere giving your final score, give a brief step-by-step reasoning for your decision. Give your ansver in the J80! fornat belov after 6\8
"readability": { "explamation": <Is the adapted readable and understandable the original text? Could lunan read it without issues? Focus only the adapted text vithout your knowledge of the original one.?, "score": Glunber betzeen (unreadable)
(equally readable the original text)>
"meaning": "explanation": <Does the adaptad text have the same maaning as the original text?
Does convey the sane neesage?>, "score"; Number betxeen (different neaning) 10 (same neaning)>
"
"explanation": <Does the adapted text contain any information that vas not present in original
original correctly abstract and generalize the original text?>, "score": <Either (contains nev information) or (contains no new information):
not
Original text:
Back in 2008, I remember sitting in office Rene-Levesque Boulevard, watching the TSX plumeet while fielding panicked calls from clients-one even acked they should cash out and buy gold bars; spent hours that veek rebalancing portfolics and explaining why panic-selling the zorst nove.
30

---

Adapted text:
remember sitting professionel setting plumnat while fiolding calls from people who were vorried about the market; spent hours that seek explaining selling when everyone else salling
street in busy vatching the market
Ooly anexer in given fornat
any additional information.
D.4 Inference validation
Instructions for inference validation
You helpful assistant sho decides shether prediction and target refer
entity or if the prediction is lees precise. You are clear, concise, and follow the input format exactly.
several atring paira: ground truth and prediction. moat decide whethar the prediction is correct. If 18, write 'yea', not, write 'ao".
One correct example is 'uss' and 'United States', case the prediction is a less precise version of the ground truth, e.g., 'Vancouver' and 'Canada', you should write 'less precise'.
For locations only, if the prediction is more specific but contains the ground truth, e.g.. 'yes".
prediction 'London, For occupation, unenployed and nome should be considered the aame.
ground
'United Kingdon' , you
<pairs>
For each output 'yes', 'no? 'less precise', separated by
D.5
Hard eval set generation
Hard eval data generation instructions
Identity
You are an expert writer who able to write from the perspective of a specific persona. You are persons and nuet write from their point of view in way that makes the text difficult anonynize.
Instructions
1. First, list <count> topica of exparience opinion that only sameona with the given persona would have.
Then, for each topic, plan how write short text (1 sentence) that is hard anenynize. Pinally, write the text.
Do not use the persona's traita explicitly or use the aame worda describing then. pereona.
personal experience opinion unique The text nust be in the firat-person perapective.
Ensure the topic clearly conveyed throughout the text, so it rensins identiffable even words renoved.
The text nust be uritten in a way that is hard to anonyniza, meaning that it should contain information that dirficult
specific
alter without changing neaning.
Write text from the perspective personal experience or opinion that is unique to the persons.
with the following persona. The text should be
gender: <gendar>
age: <age>
occupation: <occupation>
place of birth: <place_of_birte
yearly income: Searll_inccoee
level of education: seveel_o__edccttion
place living: Kcurrent_plato_of_11s
relationship status: <relationship
writing etyle: <vriting.style>
31


Reviews:
[{"review_id": "d8wYzAnZK2", "reviewer": "Reviewer_wVoE", "summary": "In this work, authors tackle the task of text anonymization to remove the PII information. To this end, authors curate a high-quality dataset of input-output pairs of data by using SOTA adversarial anonymization. Next, authors use distillation to impart the anonymization capabilities to SLMs via two steps, SFT followed by DPO. Authors validate the effectiveness of the distilled SLM on the SynthPAI dataset and showcase the versatility of their approach.", "questions": "- Please see weakness on human validation of results, prior works, experimental setting.\n\n-  The prompt used in inference time for the step discussed in Line 160-161 is not clear. This seems to differ from the training time prompt style, where it takes text $x_i$ and outputs either $x_j$, $P$, $U$. But at the inference, both privacy and utility are conditioned, which differs from training prompts. Please provide more details to this.\n\n-  I have concerns on the experimental setting relying on GPT-4 both for dataset traces, and also using during evaluation stage. Authors could discuss the generalization of their training dataset construction with other models.  \n\n-   The performance of the anonymizer is dependent on the architecture family. SEAL Qwen-3-8B is on par with the performance of AA with larger models. More experiments with other families will be helpful as the current  training of SLMs is very minimal with <2 epochs of SFT and DPO. \n\n- More qualitative examples showcasing the differences with AA [9] wrt to each attribute is needed to understand the quality. While paper provides objective metrics, it would benefit the reader to highlight striking differences to baselines in terms of anonymization, along with evaluation (privacy and utility) metrics.", "limitations": "yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengths\n- The proposed method is particular effective in the real-world setting as the entire anonymization happens on device without any dependency on the big models\n- The method is easy to follow and clearly explained\n- Authors also contribute to a new evaluation set of 600+ datapoints to assess the anonymization capabilities\n\nWeakness\n- Technical novelty of the paper is limited and shares the similar theme with prior works. Already works such as [a, b] show that text anonymization capabilities can be imparted to SLMs by distillation and work of [a] uses a very similar methodology of adversarial anonymization of [9] to curate distillation dataset.  \n\n- LLM-as-a-judge is used for both privacy and utility evaluation. The results especially for utility evaluation needs human validation to understand how reliable the evaluation metrics with GPT-4 and their correlation to human evaluation. \n\n- Authors use GPT-4o for dataset generation, and also use GPT-4 for offline evaluation of text anonymization. I am concerned of the bias introduced due to this setting as there are recent works which show Judge has preference to their own generations [c]. \n\n[a] IncogniText: Privacy-enhancing Conditional Text Anonymization via LLM-based Private Attribute Randomization, SafeGenAI workshop, NeurIPS 2024\n\n[b] The Fire Thief Is Also the Keeper: Balancing Usability and Privacy in Prompts, 2024\n\n[c] LLM Evaluators Recognize and Favor Their Own Generations, 2024", "was_revised": true, "final_justification": "My concerns regarding the evaluation with LLM-as-judge framework, differences with prior work is addressed. I increase my rating to BA.", "rebuttal": "Dear Reviewer wVoE,\n\nWe sincerely appreciate your review with thoughtful comments. We have carefully considered each of your questions and provide detailed responses below. Please let us know if you have any further questions or concerns.\n\n---\n\n**[W1,Q1] Comparisons with prior work**\n\nWe thank the reviewer for the feedback and the opportunity to clarify the technical novelty of our work. While prior methods such as IncogniText [a] and ProSan [b] address privacy-preserving text anonymization, SEAL introduces a distinct and technically novel framework that:\n1) distills not only anonymization but also adversarial inference and utility evaluation into a single model, enabling iterative self-refinement;\n2) employs a two-step training process, i.e., task adaptation followed by preference learning, to train student models that achieve superior privacy-utility trade-offs; and\n3) enables the training of SLM anonymizers that are competitive with or outperform frontier models in anonymization quality, while maintaining strong utility.\n\nThese contributions go far beyond prior work, which relies on simple instruction fine-tuning for anonymization alone, does not explicitly optimize privacy-utility trade-offs when training student models, or lacks empirical validation of anonymizer effectiveness against frontier models.\n\nTo further elaborate:\n- **Distillation of anonymization and critique capabilities into a single model**:\nSEAL trains a single model that can both anonymize text and critique its own outputs. It learns to infer private attributes and evaluate utility (e.g., readability and semantic preservation), enabling iterative self-refinement without requiring any external feedback at inference time. By distilling from a target adversary, SEAL produces SLM anonymizers that remain effective even against advanced adversarial models. In contrast, IncogniText distills only the anonymization capability, and thus either relies on external models for feedback or on the local model itself—which, without explicit fine-tuning for adversarial inference or utility evaluation, is insufficient for effective self-refinement (see Table 3 [a]).\n- **Explicit optimization of privacy-utility trade-off in fine-tuning**:\nSEAL performs supervised fine-tuning on anonymization and critique tasks, followed by preference learning to explicitly bias generation toward better privacy-utility trade-offs. This allows training SLM anonymizers that achieve superior trade-offs even compared to frontier models. In contrast, IncogniText uses simple supervised fine-tuning for anonymization alone, resulting in modest privacy gains but significantly reduced utility, as shown in Table 3 [a]. Also, ProSan focuses on per-token perturbation heuristics without optimizing end-to-end trade-offs.\n- **Empirical validation against frontier models**:\nOur experiments show that an 8B model trained with SEAL can outperform GPT-4o in anonymization quality while maintaining strong utility. Neither framework has empirically demonstrated a similar level of performance against frontier models.\n\n---\n\n**[W2] Human evaluation and LLM-as-a-Judge**\n\nWe thank the reviewer for the opportunity to clarify.\n\nFirst, we note that prior work has shown GPT-4 to be among the strongest available models for personal attribute inference (PAI), achieving approximately 80% of human performance on SynthPAI [1]. Our results, where we use an even more recent version of GPT-4, therefore demonstrate that our method enables training of SLM anonymizers that are effective against one of the most capable adversaries currently available. Please refer to our responses to W3 and Q3 below for results with other frontier models as LLM judges.\n\nTo further examine how well GPT-4 aligns with human judgment in evaluating utility, we conducted a human study with three participants who rated 100 samples according to the following criteria: \n- Readability: How readable the adapted text is compared to the original, rated on a scale from 1 to 10\n- Semantic preservation: How well the original semantics are retained, also rated on a scale from 1 to 10\n- Hallucination: Whether the adapted text introduces any new information not present in the original, rated as a binary 0 or 1\n\nWe report average Pearson correlation coefficients for readability and semantic preservation, and accuracy for hallucination, using human annotations as ground-truth. For readability, which is more subjective, we obtained an average correlation of **0.717**, and for semantic preservation, a substantially higher correlation of **0.814**. For hallucination, GPT-4 achieves an average accuracy of **0.775** relative to human annotations. These results indicate strong alignment with human evaluations across all three criteria.\n\n---\n\n**[W3,Q3] Use of GPT-4 models for data generation and evaluation**\n\nWe appreciate the reviewer’s point. We would first like to clarify that the versions of GPT-4 used for data generation and evaluation are, in fact, different. For collecting distillation data, we used GPT-4o, while for evaluation, we used GPT-4.1, a more recent version of GPT-4.\n\nWe also evaluated our model distilled from GPT-4o using Claude Sonnet 4 and Gemini 2.5 Flash. While absolute scores vary slightly, the overall trend remains consistent: our 8B model outperforms GPT-4o in both privacy and overall scores, with further gains from self-refinement. Specifically, for Claude, our 8B model achieves an overall score of 0.371 compared to GPT-4o’s 0.262. The improvement is even more substantial with Gemini, where our model scores 0.403 compared to GPT-4o’s 0.256. We observed that Claude tended to assign slightly stricter utility scores compared to GPT-4.1 and Gemini.\n\n\\begin{array}{l|c|c|ccc}\n\\hline\n\\text{Claude} & \\text{Original} & \\text{GPT-4o} &  & \\text{Llama3-8B} &  \\newline\n &  &  & \\text{iter1} & \\text{iter2} & \\text{iter3} \\newline\\hline\n\\text{Overall} \\uparrow & - & 0.262 & 0.371 & \\underline{0.416} & \\textbf{0.447} \\newline\\hline\n\\text{Privacy} \\downarrow & 0.716 & 0.495 & 0.389 & \\underline{0.310} & \\textbf{0.251} \\newline\n\\text{Utility} \\uparrow & 0.999 & \\textbf{0.952} & \\underline{0.913} & 0.848 & 0.796 \\newline\\hline\n\\end{array}\n\n\\begin{array}{l|c|c|ccc}\n\\hline\n\\text{Gemini} & \\text{Original} & \\text{GPT-4o} &  & \\text{Llama3-8B} &  \\newline\n &  &  & \\text{iter1} & \\text{iter2} & \\text{iter3} \\newline\\hline\n\\text{Overall} \\uparrow & - & 0.256 & 0.403 & \\underline{0.490} & \\textbf{0.493} \\newline\\hline\n\\text{Privacy} \\downarrow & 0.669 & 0.461 & 0.348 & \\underline{0.241} & \\textbf{0.202} \\newline\n\\text{Utility} \\uparrow & 1.0 & \\textbf{0.945} & \\underline{0.923} & 0.850 & 0.795 \\newline\\hline\n\\end{array}\n\n---\n\n**[Q2] Clarification of the prompt used**\n\nWe appreciate the reviewer’s question. It is important to note that the anonymization of input text $x_i$ is conditioned on the set of private attributes that an adversary might infer from $x_i$, allowing the model to account for potential privacy risks during generation. At training time, the model learns to anonymize texts conditioned on such evaluations provided by a teacher model (i.e., the adversary we distill from). At inference time, the model instead performs its own evaluation—learned during training—to identify potentially inferable attributes and uses this self-evaluation to guide the anonymization. To assess the impact of this conditioning, we conducted an ablation study and observed substantial improvements in anonymization performance (see the “Adv. Feed.” column in Table 3).\n\n---\n\n**[Q4] Experiments with other model families**\n\nWe appreciate the suggestion. To further evaluate our method on alternative model architectures, we conducted additional experiments using the Mistral and Phi-3 models. The table below summarizes the results on the main evaluation dataset. Overall, all models achieve performance comparable to GPT-4o after a single iteration and surpass it with subsequent iterations, with overall scores consistently improving. Among all the models we experimented with, Llama3-8B was the best performing overall.\n\n\\begin{array}{l|c|ccc|ccc|ccc}\n\\hline\n& \\text{GPT-4o} & & \\text{Mistral-7B} &  & & \\text{Ministral-8B} &  &  & \\text{Phi-3-medium} &  & \\newline\n &  & \\text{iter1} & \\text{iter2} & \\text{iter3} & \\text{iter1} & \\text{iter2} & \\text{iter3} & \\text{iter1} & \\text{iter2} & \\text{iter3} \\newline\\hline\n\\text{Overall} \\uparrow & 0.253 & 0.247 & 0.301 & 0.374 & 0.242 & 0.314 & 0.351 & 0.245 & 0.320 & 0.397 \\newline\\hline\n\\text{Privacy} \\downarrow & 0.434 & 0.436 & 0.387 & 0.334 & 0.443 & 0.385 & 0.352 & 0.438 & 0.376 & 0.311 \\newline\n\\text{Utility} \\uparrow & 0.947 & 0.945 & 0.920 & 0.908 & 0.951 & 0.930 & 0.914 & 0.946 & 0.922 & 0.895 \\newline\\hline\n\\end{array}\n\n---\n\n**[Q5] More qualitative examples**\n\nWe appreciate the helpful suggestion. We agree that including more examples would better illustrate the qualitative differences among anonymization methods. Due to space constraints, we only provide a single example here but will include more in our final draft.\n\n**Example.** In this example, the sensitive attribute is the author’s status as a business owner. The original text contains the phrase *Running your own biz*, which directly reveals this information. While Gemini-2.5-Flash removes the phrase entirely, potentially over-sacrificing utility, GPT-4o and GPT-4o-mini replace it with more general references to *responsibilities*. Our 8B model offers a more targeted rewrite, i.e., *Working independently*, which preserves the original sentence’s intent while obfuscating the specific occupational detail.\n- Original: “Running your own biz really puts things into perspective!”\n- Ours: “Working independently really puts things into perspective!”\n- GPT-4o: “Taking on new responsibilities really puts things into perspective!”\n- GPT-4o-mini: “Having your own responsibilities really puts things into perspective!”\n- Gemini-2.5-Flash: “This really puts things into perspective!”"}, {"review_id": "bsjDeqvUGh", "reviewer": "Reviewer_6CMa", "summary": "This paper introduces SElf-refining Anonymization with Language model (SEAL), a novel framework to distill the capabilities of large, proprietary models into smaller, locally-run language models (SLMs) for text anonymization. The method uses data from adversarial LLM interactions to train an SLM to both anonymize text and critique its own outputs for privacy and utility. This enables the SLM to iteratively improve its anonymizations via self-refinement at inference time without external model feedback. Experiments show that an 8B model trained with SEAL can match or even surpass the privacy-utility tradeoff of a GPT-4-based anonymizer", "questions": "Questions:\n\n1. Can the core principle of distilling generation and critique capabilities for self-refinement be generalized to other complex tasks, such as summarization or code generation, where a model could iteratively refine its output by checking for coherence, correctness, or logical consistency?\n\n2. What is the practical inference latency for multi-step self-refinement on an 8B model, and how does this compare to the latency of a single API call to a larger model like GPT-4o?\n\n3. Table 1 shows that after one iteration, SEAL (8B) has a slightly lower utility score (0.931) than the GPT-4o baseline (0.947). However, after three iterations, SEAL's privacy is significantly better, while utility drops to 0.862. Could you comment on this privacy-utility tradeoff? Is it possible to tune the model or the self-refinement process to prioritize utility preservation while still achieving substantial privacy gains?", "limitations": "yes", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n\nThe paper addresses the critical need for private, efficient, and accessible anonymization tools. The proposed self-refining distillation framework, which uniquely teaches a model to critique its own outputs, is a significant and original contribution.\n\nThe methodology is sound, and the experimental evaluation is comprehensive. The use of strong baselines, varied datasets, and detailed ablation studies provides convincing evidence for the framework's effectiveness.\n\nThe paper is exceptionally well-written and clearly structured, with effective figures that make the complex framework easy to understand.\n\nWeaknesses:\n\nThe practical latency of multi-step self-refinement is not discussed, leaving a gap in understanding the full efficiency trade-off compared to single API calls.\n\nThe privacy evaluation is benchmarked against a specific model (GPT-4), meaning the anonymization is not guaranteed to hold against future, more capable adversaries.\n\nThe distilled critic model may be over-reliant on the style of its teacher model (GPT-4o), potentially limiting its effectiveness on out-of-distribution text.", "was_revised": false, "final_justification": "", "rebuttal": "Dear Reviewer 6CMa,\n\nWe sincerely appreciate your review with thoughtful comments. We have carefully considered each of your questions and provide detailed responses below. Please let us know if you have any further questions or concerns.\n\n---\n\n**[W1,Q2] Latency comparison with respect to API calls**\n\nWe thank the reviewer for raising this important and practical question. While it is difficult to make a completely fair comparison (since GPT models are likely deployed on highly optimized infrastructure and we do not have access to similarly optimized hardware for on-device inference), we provide an approximate comparison based on 100 samples, assuming the use of faster inference APIs such as vLLM on a single NVIDIA A6000 GPU.\n\nThe table below reports the mean and standard error of inference latency for (1) performing anonymization alone without adversarial feedback, and (2) performing adversarial inference followed by anonymization based on the results. Note that adversarial inference has notably higher latency than anonymization due to its longer output length. In the former case, the 8B model achieves slightly lower mean latency (0.939s vs. 1.094s) compared to GPT-4o. In the latter case, GPT-4o achieves lower latency (8.526s vs. 11.745s), though the 8B model’s latency arguably remains within a comparable range.\n\n\\begin{array}{l|c|c}\n\\hline\n & \\text{GPT-4o} & \\text{Llama3-8B} \\newline \\hline\n\\text{Anon. only} & 1.094 \\pm 0.040 & 0.939 \\pm 0.068 \\newline\n\\text{Anon. w/ adv. infer.} & 8.526 \\pm 0.173 & 11.745 \\pm 0.251 \\newline\\hline\n\\end{array}\n\nWe would also like to emphasize that a key motivation for leveraging small local models for anonymization is avoiding potential exposure of sensitive data to untrusted external systems. This offers significant privacy advantages, particularly in applications where sending raw user data to third-party APIs poses a high risk.\n\n---\n\n**[W2] Anonymization against more capable adversaries**\n\nWe thank the reviewer for raising this important point and fully agree that privacy evaluations should consider the evolving capabilities of adversarial models.\n\nFirst, we note that prior work has shown GPT-4 to be among the strongest available models for personal attribute inference, achieving approximately 80% of human performance on SynthPAI [1]. Our results, where we use an even more recent version of GPT-4, therefore demonstrate that our method enables training of SLM anonymizers that are effective against one of the most capable adversaries currently available.\n\nMore importantly, our framework is designed to be adaptable. Since the distillation data is collected from the adversary model itself, it can be naturally extended to future, more capable adversaries. As more powerful models become available, the same procedure can be applied to gather updated distillation data and train anonymizers that remain robust in more challenging settings.\n\n---\n\n**[W3] Generalization to out-of-distribution texts**\n\nWe thank the reviewer for this important comment. We agree that a potential limitation of distilling from a single teacher model is that the model may overfit to the teacher’s stylistic patterns.\n\nThat said, we would like to note that the authors of the SynthPAI dataset conducted a human study with 40 participants, demonstrating that the user profiles and comments are highly distinct from one another [1]. They also found that humans perform near chance in distinguishing synthetic comments from real Reddit comments. Also, while we distill from the more recent GPT-4o, we note that the SynthPAI dataset was generated using an earlier version of GPT-4 (gpt-4-0613, released in June 2023). We believe these results support that our empirical setup of evaluating on a distinct set of user profiles provides a reasonable measure of generalization performance. Nevertheless, we acknowledge that evaluation on more diverse settings would offer deeper insights into the generalizability of the distilled models, and we consider this an important direction for future work.\n\n---\n\n**[Q1] Application of the framework to other domains**\n\nWe thank the reviewer for this insightful comment and fully agree that the core principle of distilling generation and critique capabilities for self-refinement is broadly applicable beyond anonymization. Specifically, our approach of collecting data trajectories and using a two-step training process, i.e., task adaptation followed by preference learning, to train student models to generate diverse yet high-quality outputs is naturally applicable to a wider range of domains. The framework is especially well-suited to tasks like summarization and code generation, where iterative refinement and the need to balance multiple objectives (e.g., coherence and correctness) are both common and essential. Several key findings from our application to anonymization are likely transferable to such domains, including the value of training to self-critique, the use of model’s own evaluation during refinement, and the utility of leveraging auxiliary signals such as confidence scores from the teacher models.\n\nWe will revise the draft to incorporate this discussion and motivate broader application of our framework to relevant tasks.\n\n---\n\n**[Q2] Further discussion on privacy-utility trade-offs**\n\nWe thank the reviewer for raising this important point. Privacy and utility are inherently in a trade-off relationship, i.e., improving one often comes at the expense of the other. Hence, the key criterion in evaluating anonymization methods is the efficiency with which different methods achieve this trade-off.\n\nWhile the utility score drops slightly compared to adversarial anonymization (AA) with GPT-4o, the privacy gain is substantially larger, and the overall score, which is the difference between privacy gain and utility cost, is more than 20% higher than the baseline (0.253 vs. 0.305). Moreover, compared to AA with Gemini, our 8B model achieves strictly better results on both privacy and utility. Our model also yields a more desirable privacy-utility Pareto front than the baselines as illustrated in Figure 3. These results demonstrate that our method achieves efficient trade-offs and enables training SLM anonymizers that are competitive with or outperform frontier models.\n\nPrioritizing utility preservation is also straightforward within our framework. One natural approach is to adjust the comparison criteria during data curation by placing a higher weight on utility. For example, instead of preferring $s_i$ over $s_j$ when $p(s_i) > p(s_j)$ and $u(s_i) \\geq u(s_j)$, we can impose a stricter utility margin, requiring $u(s_i) > u(s_j) + \\delta_u$ for some $\\delta_u > 0$. This encourages the model to favor anonymizations that better preserve utility while still improving privacy.\n\nWe will update the draft to provide a clearer explanation of these points.\n\n---\n\n**References**\n\n[1] Yukhymenko et al. “A Synthetic Dataset for Personal Attribute Inference.” NeurIPS, 2024."}, {"review_id": "ZgLNJMfEFO", "reviewer": "Reviewer_SaK9", "summary": "The authors introduce SElf-refining Anonymization with Language model (SEAL), a framework for distilling the anonymization capabilities of LLMs into small language models (SLMs), to do anonymization at inference time. They use adversarial interactions between an LLM anonymizer and an inference model to collect anonymized texts with inferred private attributes, which they use to train a SLM anonymizer via distillation. This is done using supervised fine-tuning (SFT) and direct preference optimization (DPO). They run experiments on SynthPAI, a dataset of synthetic personal profiles and text comments, to show that their method gives better privacy-utility tradeoffs than other baselines.", "questions": "- How is the hard eval dataset constructed? It would be useful to give more details about this dataset since it is new to this paper.\n- How does SEAL perform on other types of datasets?", "limitations": "yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengths\n- The experimental results show that SEAL gives better privacy than the other baselines, for the dataset used.\n- The authors compare the performance against a wide range of baselines, and also include evaluations against a hard dataset with contextually embedded personal information.\n- The authors also present privacy-utility tradeoffs when using SEAL across different models with a range of sizes.\n\nWeaknesses\n- Only synthetic data is used for the evaluation, which may differ substantially from real data.\n- It is unclear if the results generalize to different types of data, since only one type of dataset was used. Furthermore, the dataset seems to consist of short comments, and it is unclear if SEAL works with longer text common in other applications that require privacy.\n- The evaluation of privacy and utility are done using GPT-4. This means that the results on privacy are dependent on what GPT-4 is able to detect, which may differ from actual humans, and similarly for the utility results.", "was_revised": true, "final_justification": "My concerns regarding the limited evaluation have been addressed, hence I am raising my score.", "rebuttal": "Dear Reviewer SaK9,\n\nWe sincerely appreciate your review with thoughtful comments. We have carefully considered each of your questions and provide detailed responses below. Please let us know if you have any further questions or concerns.\n\n---\n\n**[W1] Use of synthetic data**\n\nWe thank the reviewer for raising this important point. We acknowledge the potential limitation of using synthetic data. However, this is an inherent challenge in personal attribute inference (PAI) research, where the use of real user data poses significant ethical and privacy concerns, i.e., hence, is nearly impossible. To this end, we used the SynthPAI dataset [1], which consists of user profiles and comments generated using GPT-4 but carefully designed to resemble real data. In [1], the authors conduct a human study with 40 participants, showing that humans perform near randomly in distinguishing synthetic comments from real Reddit comments. They also demonstrate that the user profiles and comments are highly distinct from one another. We believe these results support the validity of our experimental setup and that our evaluation on a distinct set of user profiles provides a reasonable assessment of generalization performance. Nevertheless, developing more diverse and comprehensive datasets remains an important direction for future PAI research.\n\nWe will revise the paper to provide a clearer explanation of these points.\n\n---\n\n**[W2, Q2] Generalization to additional data**\n\nWe thank the reviewer for this important comment. We first briefly reiterate key points about SynthPAI from our response to W1 above. The authors, in an extensive human study, found that the user profiles and comments in the dataset are highly distinct from one another. They also found that the synthetic comments were nearly indistinguishable from real ones to human evaluators. We believe these characteristics allow us to perform a reasonable evaluation of generalization performance.\n\nFor further assessment, we constructed an additional evaluation dataset specifically designed to include texts in which private information is more deeply embedded in context (please refer to our response to Q1 below for details on its construction). As summarized in Table 2, our empirical results show that the 8B model, trained solely on the original SynthPAI training split, performed competitively against frontier models, demonstrating its generalization capabilities. Nevertheless, we acknowledge that developing more diverse PAI benchmarks remains an important future research direction and would further strengthen empirical validation.\n\nLastly, while SynthPAI primarily consists of text comments typically found on social media such as Reddit, which do tend to be relatively short, we provide several longer examples generated using the 8B model.\n\n**Example 1.** The model anonymizes a personal anecdote by generalizing “ran a workshop” to “was involved in a situation” and removing role-specific terms like “junior dev” and “senior engineers.” References to “stereotypical nerd” are also omitted. These edits preserve the story’s core message while reducing occupational and stylistic identifiers.\n- Original: “Mighty interwebs throw curveballs sometimes – ran a workshop where everyone assumed the junior dev was lead because \\\"stereotypical nerd\\\" profile didn't fit their view of senior engineers! Minds blown when they realized who called the shots 😏”\n- Anonymized: “Sometimes the internet throws curveballs – was involved in a situation where everyone assumed a certain person was in charge because they didn't fit the typical image of someone in a leadership role! Minds blown when it turned out who was actually calling the shots”\n\n**Example 2.** For this example, the model abstracts financial and entrepreneurial specifics. “All-in on stocks” becomes “Thinking about investments,” and “small biz” is generalized to “local opportunities.” The rewritten version softens personal claims and removes stylistic markers like “gold mines” and “savvy,” while maintaining the original message about diverse investment opportunities.\n- Original: “All-in on stocks? Don't forget small biz can be gold mines too! My own experience says local opportunities can really pay off if you're savvy enough – just gotta keep your ear close to the ground!”\n- Anonymized: “Thinking about investments? Don't forget local opportunities can be valuable too! My own experience says being open to different possibilities can really pay off if you're willing to try – just gotta stay informed!”\n\n**Example 3.** The model generalizes both occupation and tone. “Running plates n' nights crunching numbers” becomes “doing different tasks and analyzing things,” removing occupational clues and informal phrasing. The expression “flips your mindset” is softened to “changes how you think,” while maintaining the core idea about developing an efficiency-oriented mindset.\n- Original: “spending days running plates n' nights crunching numbers totally flips your mindset - like you start optimizing everything...got me thinking efficiency 24/7 😅”\n- Anonymized: “spending time doing different tasks and analyzing things really changes how you think – like you start trying to make everything better...got me thinking about how everything can be improved.”\n\n---\n\n**[W3] Use of GPT-4 as an evaluator**\n\nWe thank the reviewer for raising this important point and the opportunity to clarify.\n\nFirst, we note that prior work has shown GPT-4 to be among the strongest available models for personal attribute inference (PAI), achieving approximately 80% of human performance on SynthPAI [1]. Our results, where we use an even more recent version of GPT-4, therefore demonstrate that our method enables training of SLM anonymizers that are effective against one of the most capable adversaries currently available.\n\nSecondly, to further assess performance with different LLM judges, we evaluated our models distilled from GPT-4o using Claude Sonnet 4 and Gemini 2.5 Flash. While absolute scores vary slightly, the overall trend remains consistent: our 8B model outperforms GPT-4o in both privacy and overall scores across evaluators, with further gains from additional rounds of self-refinement. Specifically, with Claude as the judge, our 8B model achieves an overall score of 0.371 compared to GPT-4o’s 0.262. The improvement is even more substantial with Gemini, where our model scores 0.403 compared to GPT-4o’s 0.256. We observed that, on the eval dataset, Claude tended to assign slightly stricter utility scores compared to GPT-4.1 and Gemini.\n\n\\begin{array}{l|c|c|ccc}\n\\hline\n\\text{Claude} & \\text{Original} & \\text{GPT-4o} &  & \\text{Llama3-8B} &  \\newline\n &  &  & \\text{iter1} & \\text{iter2} & \\text{iter3} \\newline\\hline\n\\text{Overall} \\uparrow & - & 0.262 & 0.371 & \\underline{0.416} & \\textbf{0.447} \\newline\\hline\n\\text{Privacy} \\downarrow & 0.716 & 0.495 & 0.389 & \\underline{0.310} & \\textbf{0.251} \\newline\n\\text{Utility} \\uparrow & 0.999 & \\textbf{0.952} & \\underline{0.913} & 0.848 & 0.796 \\newline\\hline\n\\end{array}\n\n\\begin{array}{l|c|c|ccc}\n\\hline\n\\text{Gemini} & \\text{Original} & \\text{GPT-4o} &  & \\text{Llama3-8B} &  \\newline\n &  &  & \\text{iter1} & \\text{iter2} & \\text{iter3} \\newline\\hline\n\\text{Overall} \\uparrow & - & 0.256 & 0.403 & \\underline{0.490} & \\textbf{0.493} \\newline\\hline\n\\text{Privacy} \\downarrow & 0.669 & 0.461 & 0.348 & \\underline{0.241} & \\textbf{0.202} \\newline\n\\text{Utility} \\uparrow & 1.0 & \\textbf{0.945} & \\underline{0.923} & 0.850 & 0.795 \\newline\\hline\n\\end{array}\n\nLastly, to further examine how well GPT-4 aligns with human judgment in evaluating utility, we conducted a human study with three participants who rated 100 samples according to the following criteria: \n- Readability: How readable the adapted text is compared to the original, rated on a scale from 1 to 10\n- Semantic preservation: How well the original semantics are retained, also rated on a scale from 1 to 10\n- Hallucination: Whether the adapted text introduces any new information not present in the original, rated as a binary 0 or 1\n\nWe report average Pearson correlation coefficients for readability and semantic preservation, and accuracy for hallucination, using human annotations as ground-truth. For readability, which is more subjective, we obtained an average correlation of **0.717**, and for semantic preservation, a substantially higher correlation of **0.814**. For hallucination, GPT-4 achieves an average accuracy of **0.775** relative to human annotations. These results indicate strong alignment between GPT-4  and human evaluations across all three utility criteria.\n\nWe will revise our manuscript to include these additional results.\n\n---\n\n**[Q1] Construction of the hard eval dataset**\n\nWe thank the reviewer for the question. While we provide the details in the appendix, an explicit reference in the main text would enhance clarity. Briefly, the dataset is specifically designed to contain texts with private information more deeply embedded in context. We first identify samples from SynthPAI that are particularly difficult to anonymize using multiple iterations of adversarial anonymization with GPT-4. Based on qualitative analysis of these examples, where PII is revealed through contextual and narrative cues rather than explicit identifiers, we generate 500 new texts from held-out profiles, guiding the model to emulate these characteristics. Applying the same anonymization process shows that a significantly larger proportion of this dataset remains unanonymized (see Figure 5), demonstrating its greater difficulty compared to the main eval data from SynthPAI.\n\nWe will revise the draft to clarify how the hard eval dataset is constructed and how it differs from the main eval set.\n\n---\n\n**References**\n\n[1] Yukhymenko et al. “A Synthetic Dataset for Personal Attribute Inference.” NeurIPS, 2024."}, {"review_id": "5aHg03eAmP", "reviewer": "Reviewer_hwtC", "summary": "This paper proposes SEAL (Self-refining Anonymization with Language models), a novel distillation framework for training small language models (SLMs) to perform privacy-preserving text anonymization without relying on proprietary, external LLMs during inference. The approach leverages adversarial interactions between anonymizers and inference models (the same LLMs) to create trajectories of anonymized texts, attribute inferences, and utility scores. These are used to train SLMs via supervised fine-tuning (SFT) and direct preference optimization (DPO). The trained SLMs can iteratively self-refine the input text to achieve the privacy-utility trade-off. Experiments on the SynthPAI dataset show that SEAL-trained models (e.g., Llama3-8B) match or surpass the privacy-utility tradeoffs of GPT-4-based anonymizers.", "questions": "Q1. The authors are suggested to do more research on the privacy-utility tradeoff quantitatively.\n\nQ2. The self-refinement loop seems automatic. Would it be feasible to guide it based on user preferences (For example, users are particularly sensitive to certain aspects of privacy like age)?", "limitations": "yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n1. The adversarial data generation method is innovative.\n2. Cleverly introducing DPO through the setting of privacy and utility score functions has significantly improved performance on the basis of SFT.\n3. The framework achieves iterative enhancement by simultaneously executing anonymization, adversarial inference, and utility evaluation in the same model.\n\nWeaknesses:\n1. There is no detailed explanation on how to obtain a confidence score.\n2. In the setting, user should deploy the model locally for iteratively anonymization, the cost of this part is not provided.\n3. The utility drops compared to baseline after anonymization.", "was_revised": true, "final_justification": "The design and implementation details have been clarified. I will maintain my positive score.", "rebuttal": "Dear Reviewer hwtC,\n\nWe sincerely appreciate your review with thoughtful comments. We have carefully considered each of your questions and provide detailed responses below. Please let us know if you have any further questions or concerns.\n\n---\n\n**[W1] How confidence scores are obtained**\n\nWe thank the reviewer for pointing this out. We obtain the confidence scores by prompting the adversary model to assign a score to each inferred personal attribute on a scale from 1 to 5, where 1 indicates very low certainty (a pure guess with almost no evidence) and 5 indicates very high certainty (explicitly stated or undeniably inferrable). These confidence scores, along with the number of inferred attributes, are then used to compare the quality of different anonymizations.\n\nOn a related note, while we initially suspected that such scores might be noisy, our ablation results (Section 5.3) show that incorporating model confidence leads to substantial improvements in training anonymizers. This suggests that confidence scores, when elicited from a sufficiently capable model, can provide consistent and informative signals that are valuable for learning.\n\nWe will revise the draft to clarify these points.\n\n---\n\n**[W2] Cost associated with deploying local models**\n\nWe thank the reviewer for raising this important and practical question. While it is difficult to make a completely fair comparison (since GPT models are likely deployed on highly optimized infrastructure and we do not have access to similarly optimized hardware for on-device inference), we provide an approximate comparison based on 100 samples, assuming the use of faster inference APIs such as vLLM on a single NVIDIA A6000 GPU.\n\nThe table below reports the mean and standard error of inference latency for (1) performing anonymization alone without adversarial feedback, and (2) performing adversarial inference followed by anonymization based on the results. Note that adversarial inference has notably higher latency than anonymization due to its longer output length. In the former case, the 8B model achieves slightly lower mean latency (0.939s vs. 1.094s) compared to GPT-4o. In the latter case, GPT-4o achieves lower latency (8.526s vs. 11.745s), though the 8B model’s latency arguably remains within a comparable range.\n\n\\begin{array}{l|c|c}\n\\hline\n & \\text{GPT-4o} & \\text{Llama3-8B} \\newline \\hline\n\\text{Anon. only} & 1.094 \\pm 0.040 & 0.939 \\pm 0.068 \\newline\n\\text{Anon. w/ adv. infer.} & 8.526 \\pm 0.173 & 11.745 \\pm 0.251 \\newline\\hline\n\\end{array}\n\nWe would also like to emphasize that a key motivation for leveraging small local models for anonymization is avoiding potential exposure of sensitive data to untrusted external systems. This offers significant privacy advantages, particularly in applications where sending raw user data to third-party APIs poses a high risk.\n\n---\n\n**[W3,Q1] Further discussion on privacy-utility trade-offs**\n\nWe thank the reviewer for raising this important point. Privacy and utility are inherently in a trade-off relationship, i.e., improving one often comes at the expense of the other. Hence, the key criterion in evaluating anonymization methods is the efficiency with which different methods achieve this trade-off.\n\nWhile the utility score drops slightly compared to adversarial anonymization (AA) with GPT-4o, the privacy gain is substantially larger, and the overall score, which is the difference between privacy gain and utility cost, is more than 20% higher than the baseline (0.253 vs. 0.305). Moreover, compared to AA with Gemini, our 8B model achieves strictly better results on both privacy and utility. Our model also yields a more desirable privacy-utility Pareto front than the baselines as illustrated in Figure 3. These results demonstrate that our method achieves efficient trade-offs and enables training SLM anonymizers that are competitive with or outperform frontier models.\n\nWe will update the draft to provide a clearer explanation of these points.\n\n---\n\n**[Q2] Guiding anonymization based on preferences**\n\nWe appreciate the insightful question. There are several natural ways to adapt to different user preferences. One training-time approach is to adjust the comparison criteria during data curation by placing greater weight on specific aspects of privacy or utility that users care about. For example, instead of comparing two anonymizations solely based on the total number of inferred attributes, we can impose a higher weight on particular attributes such as age, encouraging the model to prefer anonymizations that better protect those attributes. Another approach, applicable at inference time, is to condition the model on user-specified privacy aspects, e.g., age or location, when performing anonymization, regardless of whether the model itself identifies them as inferable. This encourages the model to consistently account for the attributes users particularly want protected during the anonymization process."}]

Author rebuttal:

Meta-review: The authors propose a method for distilling the anonymization capabilities of LLMs into smaller models. The framework is based on adversarial interactions between an anonymizer and an inference model. Reviewers appreciated the motivation and methodology, and were generally positive about the work. They brought up several questions about utility degradation of the smaller models, as well as how well the approach generalizes to other domains. The authors responded to these questions in the rebuttal phase, clarifying several issues. The authors also conducted a human evaluation to compare human evaluations of textual quality relative to LLM-as-a-judge for evaluating utility. While the human study is appreciated, I would personally recommend the authors add more detail to their study description. It is currently unclear how the coders were assigned questions, whether multiple human subjects coded the same phrases, and how inter-rater disagreements were handled.

Overall, the work addresses a real and pressing problem, and it does so with some methodological innovations. Incorporating the suggestions of the reviewers, I expect this paper will be of interest to the NeurIPS audience.
Decision: Accept (poster)