Operationalizing Data Minimization for Privacy-Preserving LLM Prompting
Authors: Jijie Zhou
Niloofar Mireshghallah
Tianshi Li
Venue: ICLR 2026 Poster (2026)
Keywords: NLP
privacy
LLM
data minimization
data sanitization
Primary area: alignment, fairness, safety, privacy, and societal considerations
TL;DR: We introduce a framework that operationalizes data minimization of LLM prompting as the least privacy-revealing prompt that preserves task utility, showing more capable LLMs tolerate greater minimization and establishing a predictive baseline.

Abstract: The rapid deployment of large language models (LLMs) in consumer applications has led to frequent exchanges of personal information. To obtain useful responses, users often share more than necessary, increasing privacy risks via memorization, context-based personalization, or security breaches. We present a framework to formally define and operationalize *data minimization*: for a given user prompt and response model, quantifying the least privacy-revealing disclosure that maintains utility, and propose a priority-queue tree search to locate this optimal point within a privacy-ordered transformation space. We evaluated the framework on four datasets spanning open-ended conversations (ShareGPT, WildChat) and knowledge-intensive tasks with single-ground-truth answers (CaseHOLD, MedQA), quantifying achievable data minimization with nine LLMs as the response model. Our results demonstrate that larger frontier LLMs can tolerate stronger data minimization while maintaining task quality than smaller open-source models (*85.7%* redaction for GPT-5 vs. *19.3%* for Qwen2.5-0.5B). By comparing with our search-derived benchmarks, we find that LLMs struggle to predict optimal data minimization directly, showing a bias toward abstraction that leads to oversharing. This suggests not just a privacy gap, but a capability gap: models may lack awareness of what information they actually need to solve a task.

Published as a conference paper at ICLR 2026
OPERATIONALIZING DATA MINIMIZATION PRIVACY-PRESERVING LLM PROMPTING
FOR
Jijie Zhou' Niloofar Mireshghallah" Tianshi Li' Northeastern University *Carnegie Mellon University
j. zhou@northeasterm. edu niloofare cmu.
tia. wwiiinrteeetteen......
ABSTRACT
The rapid deployment of large language models (LLMs) in consumer applica- tions has led to frequent exchanges of personal information. To obtain useful responses, users often share more than necessary, increasing privacy risks via memorization, context-based personalization, security breaches. We present framework to formally define and operationalize data minimization: for a given user prompt and response model, quantifying the least privacy-revealing diselo- sure that maintains utility, and propose a priority-queue tree search to locate this optimal point within a privacy-ordered transformation space. We evaluated the framework on four datasets spanning open- conversations (ShareGPT, Wild- Chat) and snowledge-intensive tasks with gegle-groond-trutte answers (Case- HOLD, MedQA), quantifying achievable data minimization with nine L.MS as the response model. Our results demonstrate that larger frontier LLMs can toler- ate stronger data minimization while maintaining task quality than smaller open- source models (85.7% % redaction for GPT-5 vs. 19.3% for Qwen2.5-0.5B). By comparing with our search- benchmarks, predict optimal data minimization directly, showing a bias toward abstraction that leads to oversharing. This suggests not just privacy gap, but capability gap: models may lack awareness of what information they actually need to solve a task.
find that LLMs struggle
INTRODUCTION
Users increasingly reveal sensitive personal information to large language model (LLM) applica- tions (Mireshghallah et al. 2024a Zhang et al.) (2024). exposing themselves to privacy leaks via memorization, context-based personalization, or security breaches. Many share details believing it boosts task performance (Zhang et al. 2024),, but this benefit is often illusory: people routinely overshare beyond what utility requires (Zhou et al. 2025). We ask a fundamental question: Whar is the minimal information needed to maintain utility while preserving privacy? This question is essential to quantify oversharing-the that is, to compare actual disclosure against the true minimum.
Data minimization, defined limiting the collection of personal information to what is necessary to accomplish a specified purpose, is well-established privacy design pattern (Cavoukian et al.) 2009) and is explicitly cited in numerous privacy regulations (e.g., GDPR (Parliament & Council 2016)). Although considerable work has sought to mitigate the oversharing of sensitive information in LLM applications, few studies explicitly formalize or quantify this challenge from the perspective of data minimization. Existing approaches typically focus on detecting personal or sensitive disclosures and then apply redaction (e.g., "New York" "[GEOLOCATION""
abstraction (e.g., "New York" - "a city in the U.S.") (Dou et al. 2024 Zeng et al., 2025); related efforts develop heuristics to flag information types that are sensitive yet have low semantic relevance to the task (e.g., SSNs [Chowd-] hury et al.) 2025)) or employ LLM-as-a-Jude to assess the relevance importance of information to guide sanitization (Ma et al., 2025 Ngong et al. 2025). this work, we introduce a framework that formally operationalizes data minimization for privacy-preservee
LLM prompting, and present an algorithm that searches for the minimum privacy disclosure while preserving utility, thereby pro- viding an oracle of data minimization for any prompt and target response generation model.
Figure I illustrates our framework with a running example. Our method can be viewed as a spe- cialized tree search for data minimization. Starting from a root node that represents the most heav-

---

Published as a conference paper at ICLR 2026
Original Prompt
Sarvin
Nower Sunten
Prompt Utility
travel
for ing Einetary for travel
Mumnar and Tekkadyy Kerala,
for
have already booked lightt from
Hyderabed to Kooti for orward joumney on 28th Jan and retur joumnyy
popular hill atation South India and Tolkady (SEOLOCATIONS aleedy booked flighte from a major city in South India to coantal city in South india for crwvard joumnyy on 25th
have
Jan. group men slanning Tolkady. want you
Murnar: Abstract
Mumnar: Abstract
stayy
Abstract
Tolocddy Abstract
Retain
renurn journey 28th Jan. we group of and oaye popular Tekkady. wantt to
Kersin Redact
Kerelar Redect
Karnla: Redact
hycersbad Redact
Hydorabad: Abstract
Hydarabod: Abstract
planning South
Koont Redact
Koch:: Redact
Koch: Abstract
day
20th Redact
25th Jans Redaact
25th Jan: Retainn
Mumnar and Tekkady fowed
Redact
Jan
26th Jon Resain
most the abstract level, since redecting them would break ty
unitty Check X
Utilty Check-PAIL
Utiiity Check PASS
Figure 1:
Framework Overview. We present a running example to demonstrate how we perform tree search ranked by privacy variants, and a transformation that achieves data minimization.
ily sanitized prompt-capturing the globally most privacy-preserving formulation- wwe iteratively expand the tree. Unlike classical depth- or breadth-first search, we maintain a priority queue ordered by privacy sensitivity. At each step, we dequeue the least sensitive node, generate slightly more informative (and thus more process systematically explores the space of possible prompts in order of increasing privacy disclo- sure, enabling the identification of a minimally sufficient prompt that satisfies the target utility.
variants as its children, and enqueue them. This
Our experimental results show that even under this utility-first constraint, there remains significant room for preserving privacy with data minimization- far exceeding the level of protection typically achieved in current practice. We observe that more powerful frontier models offer greater potential for data minimization than smaller, less capable ones. On open-ended real-world LLM prompts, gpt-5 shows the strongest removal with 85.7% REDACT and 8.6% ABSTRACT (only 5.7% RE- TAIN), while the smallest model (qwen2.5-0.5b) lags with 19.3% REDACT, 11.0% ABSTRACT, and 69.7% 7% RETAIN.
By comparing with our oracles, we show that LLMs from small edge models to frontier reasoning models are poor predictors of data minimization, which bias towards ABSTRACT actions, leading to prevalent oversharing predictions. Together, these results demonstrate data minimization as promising paradigm for addressing input privacy in LLM systems, while also revealing gaps in the popular LLM-as-a-Judge method for privacy-utility assessment tasks (Ma et al., 2025 Ngong al.) 2025). This suggests not just a privacy gap, but a capability gap: models may lack awareness of what information they actually need to solve tasks. We call for research to investigate the underlying causes of the varied levels of information "redundancy" across models, with the goal of developing robust prediction methods for effective on-device data minimization.
BACKGROUND & RELATED WORK
Theoretical & regulatory foundation. LLMSs can expose memorized training data and personally identifiable information (PII) under adversarial prompting, motivating a shift toward minimizing user-side disclosure before inference rather than relying solely on post-hoe filtering. This impera- tive embodies the data minimization principle, a cornerstone of privacy laws and design guidelines. For example, data minimization 2009). foundational and widely recognized regulatory framework central to modern data protec- tion regimes such as GDPR Art. 5(1)()), which limits processing to data necessary for a specified purpose (Parliament & Council 2016).
pillar of the privacy by design framework (Cavoukian et al.)
User-led minimization for prompts. User-assisted tools help them manually sanitize inputs prior to submission (Zhou et al. 2025 Kan et al., 2023). However, these workflows hinge on subjec- tive judgments of what "feels safe," offer no guarantees of utility preservation, and rarely include attacker-based verification of residual leakage. User studies on implicit inference further show people systematically underestimate what models can infer and often choose ineffective rewrit- ing strategies (e.g., paraphrasing) (Wang et al., 2025). In contrast, [RETAIN, ABSTRACT, REDACT) in accordance with the data minimization principle by expanding a tree in increasing order of privacy disclosure, with a priority queue guiding the exploration based on
automate selection among
2

---

Published as a conference paper at ICLR 2026
this privacy order. We employed attacker LLMs tasked with type-wise and span-wist recovery of the redacted and abstracted information in the minimized prompts produced by our method, further verifying the limited recoverable signal and the efficacy of the minimization.
minimization and prompt sanitizers. Prior input sanitization methods either do not consider utility (Dou al. 2024), seek balance between privacy and utility (Li et al. (025b), or aim to maximize utility under a privacy constraint (e.g., a differential privacy budget Chowdhury al. (2025)). Data minimization, representing a class of methods that optimize privacy under strict utility constraints, has received limited attention. A related line of work relies either on heuristics (e.g., detecting tokens whose format alone indicates sensitive content, such as SSNs Chowdhhry iet al. (2025) or on LLM-as-a-Judge to assess how essential or relevant piece of information is to the task, and then transforms the less essential and sensitive information to maintain utility (Ma et al. 2025 Ngong et al. 2025). However, we caution that it remains unclear to what extent L.MM assessments align with the actual importance or necessity of the information, as this alignment depends not only the semantic meaning of the information and the task but also the capability of the target model. Our results further show that LLMs are poor predictors of data minimization, highlighting this gap.
Utility-presee
Training-stage defenses (orthogonal), Differentially private (DP) beanninggfine-tuninn (Abadi et al,, 2016) and machine unlearning (Bärbulescu & Triantafilluu 2024) offer training- side protection against the downstream harms of oversharing caused by memorization during training. These ap- proaches require access model parameters and incur utility and compute costs, and they do not address other threat models to which oversharing is also vulnerable, including inference-stage leak- age (Shao al. 2025), data breaches (Theori Research [2025] Meta Security Team 2025 Gadget Review, 2025), or uninformed consents (Zhang et al. [2024] Fast Company (0025). Our method is black-box and pre-ing inference: it operates solely on the user imput and uses output-leve level utility checks, complementing these methods by remaining compatible with closed and rapidly evolving models, when fundamentally mitigating multiple threats through protection of the initial disclosure.
DATA MINIMIZATION FOR PRIVACY-PRESERVING PRESERVING LLM PROMPTING
3.1 PROBLEM FORMULATION
Let be a user message and let D = (eI,..., } } be set of detected sensitive spans. Each span & can be transformed by an action ag chosen from some finite action space A, forming an action vector (a1....., a,)). Applying to yields transformed message (xx; a). Given target large language model F, we seek a transformation that maximizes privacy while preserving downstream utility. Because placeholders abstractions may later be replaced with their recovered context, the utility is evaluated after a context-recovery step R that reconstructs a usable output from F:
max (riv(r();;)) subject ULi(R(F(7())))))))) 2 7 aE
where
Priv is any privacy metric (e.g., risk of sensitive-entity disclosure).
Util is any downstream utility metric evaluated on the recovered output R(F(-)),
is the context-recovery operator that replaces placeholders or abstractions with the ap- propriate recovered content, and
y is minimum acceptable utility level.
This formulation is agnostic to the choice of action space. privacy/utility metrics, and search strategy.
3.2 SPECIFIC INSTANTIATION
In this instantiation, we ground the generic formulation by defining a span-level action space A (RETAIN, ABSTRACT, REDACT), which we arrange as an ordinal hierarchy reflecting increasing privacy strength. Each detected sensitive span e, is assigned one of these actions, inducing space of possible variants guided by human preferences for privacy sensitivity. The algorithm searches this preference-ordered space to identify the most -ereaeciireeeiinggg variant while ensuring that the
3

---

Published as a conference paper at ICLR 2026
utility predicate yields acceptable judgment. This construction provides the foundation for the formal definitions that follow.
Action Space. The action space is A (RETAIN, ABSTRACT, REDACT). These actions form an ordinal lattice, RETAIN < ABSTRACT REDACT, encoding increasing privacy strength. The lattice is used to define one-step relaxations for the search procedure, and identify spans that cannot be modified without violating the utility constraint (Stage 1 of our search algorithm).
F(r( For open-ended tasks, placehold- ers/abstractions in ÿ are deterministically restored to 8rh using the transformation map. A judge model then evaluates the pair (y, 0) under fixed rubric to verify that the transformation does degrade task performance, returning pass or fail. For tasks with fixed ground truths, utility is pass iff F(T (æ; a)) is correct under the task's 's scoring rule (c.g., exact match or multiple-choice accuracy). The only criterion for accepting a candidate is the utility predicate UTIL returns pass.
Utility Predicate. Let y F(x) and
We examined how sensitive the utility predicate is small relaxations of the threshold yy To test whether users can perceive such utility reductions, we conducted a user study (see Appendix (F) comparing outputs produced under different 7 settings. The results show that even minor relaxations of y lead to noticeable quality degradation from a user's perspective. This supports our choice of strict pass-fail utility predicate that requires preserving the original utility without degradation.
Privacy Comparator. To define a structured search space over privacy transformations, we intro- duce a pairwise privacy comparator C (z, TA, TB) - (TA, TB, SAME]. Given two variants of the same source message, it returns which is more privacy-preserving (or SAME).
Unlike a partial order, this relation not assumed to be transitive or total, reflecting the empirical reality that human privacy preferences may exhibit intransitivities context-dependent judgments. Our algorithm leverages this relation as an ordering signal, treating it as an oracle for guided search without requiring formal lattice properties.
ALGORITHM AND IMPLEMENTATION
This section presents both the algorithmic procedure and the practical implementation of our frame- work. The algorithm specifies two-stage search over the privacy-ordered action space, and the implementation focuses on instantiating the privacy comparator to align with human preferences. Together, they define the end-to-end system used in our experiments.
4.1 ALGORITHM: FREEZE-THEN-SAANE
Stage 1: Freeze Infiexible Entities. For each E D, probe REDACT(e) and ABSTRACT(e) in isolation while keeping all other entities RETAIN. If both probes cause utility to fail, mark e as frozen (forced RETAIN thereafter). Let D' C D be the non-frozen entities with n' 1D'; only D' participates in Stage 2. This step both preserves utility invariants and reduces the branching factor.
Stage 2:
Privacy- -oorcy-Commprator Priority-Queue Tree Search. The tree search begins root node obtained by applying to each E D' the most privacy-pre -peaacc-rreeeiingg transformation allowed by Stage 1. Each node encodes transformation action vector and its corresponding transformed message (xx; a). For any nodes, child nodes are generated by relaxing exactly one action (i.e., REDACT + ABSTRACT; ABSTRACT - RETAIN). The tree is traversed in order of decreasing privacy, guided by priority queue that uses C as the comparator. Ties (SAME) are broken by stable insertion order. The complete search procedure is given in Algorithm|
The procedure returns the first action profile a that satisfies the utility predicate. We record (i) the transformed input (x;a):: (ii) the Stage freeze set D' (entities forced to RETAIN); (iii) the per-entity action map. If no candidate passes, we return RE
3 action profiles on the non-frozen coordinates. T candidates expanded, binary-heap implementation requires at most C cT log T pairwise comparisons (many avoided by caching). With average per-call latencies te and tumn. for comparator and utility respectively, Time cT ePlog? T lc T Lutil-
Complexity. Stage 2 explores at most |M| = 3
4

---

Published as a conference paper at ICLR 2026
Algorithm 1: Privacy-Comparaty Priority Queue Tree Search (Stage 2)
Input: message æ; non-frooe con-frozen entities D'; utility predicate U; comparator
Output: first passing action profile
Initialize ag: for e E D', set REDACT unless it failed in Stage (then ABSTRACT); for D
set RETAIN;
Q + comparator-based priority queue seeded with ag (ties: stable order);
V 0; // visited
while Q not empry do
- Q.pop(): if V then
continue
V (a):
(F(x), R(F(T(r;a));a)) = pass then
return
10 foreach e E D' with a, E (REDACT, ABSTRACT) do
a' degrade de by one step (REDACT~ARSTRAAE ABSTRACT-REFAIN); a' & V then push a' into Q 14 return RETAIN!P) // fallback
12
13
4.2 IMPLEMENTATION
Privacy Transformations and Utility Check. For each prompt we detected PII spans and per-entity variants map (e.g., New York City and NYC) detected and clustered by GPT-4o; iden- tical REDACT/ABSTRACC mappings and GPT-4o-generatee abstractions are used across all models (App. D). We implement the span-level privacy transformation actions with deterministic rewriter that (i) applies per-entity actions ag E (RETAIN, ABSTRACT, REDACT} to produce 7(2; a) and replacement map, and (ii) performs strict replace-back on model outputs for evaluation (Sec. 3.2). For utility, GPT-4o acts as judge (App. E): seed-ground-teg -ruth tasks use the task's official scorer on (x; a)); open-ended tasks judged once (y, restore(ū)): single-answer QA runs k-5 independent decodes with early stop at the first mismatch, passing only if all are correct.
Privacy Comparator. We collect human ground-truth labels 150 A/B pairs sampled from PII-rich subset of the ShareGPT dataset [RyokoA] 2023). with each pair annotated by at least five annotators. Independently, we create 4.840 additional pairs and obtain teacher labels from strong zero-shot judge (OpenAI o3) for supervised LoRA finetuning (Hu et al., 2022), resulting in a latency- optimized comparator (finetuned Qwen2.5 5- 7B-Inntructt hyperparameters
App. B) Compared with the human labels, the distilled comparator achieves 71 % overall and 89 % in high-human nuuman-consema items (2 0.8) at s/decision--yieldiin
20x speedup vs. the zero-shot judges with compara- ble high-consensus accuracy [T. This choice materially reduces the c: T''oo T le term in $14 and enables practical Stage search, Consensus among the 150 human-labeled pairs varies substan- tially: 73 items reach consensus > 0.8 and 121 reach > 0.6, with only small subset achieving full agreement. Comparator accuracy improves with higher consensus, rising from 71% overall to 77%
0.6 and 89% at > 0.8.
Comparator
Accuracy (All)
Acc. @ consensus 2 0.8
Latency (s)
(zero-shot)
70%
90%
8.05
o3 (zero-shot)
70%
89%
6.37
03-mini (zero-shot)
69%
88%
4.32
Qwen2. 5- S- (finetuned)
71%
89%
0.31
Table 1: Privacy comparator alignment with human judgments and per-decision latency.
Utility Evaluator. For open-ended tasks, where utility cannot be measured deterministically. we validate GPT-4o as the utility judge using samples drawn from the oracle's 's search trace. We con- structed 150 evaluation pairs, balanced between GPT-4o PASS and FAIL decisions, and collected judgments from 75 human annotators (five per item) whether utility preserved (ACCEPT) or
5

---

Published as a conference paper at ICLR 2026
degraded (REJECT). Agreement between GPT-4o and humans increases with consensus strength, rising from approximately 0.69 at a 0.6 consensus threshold to approximately 0.94 under full agree- ment. This pattern parallels that of the privacy comparator and supports GPT-4o's reliability in the high-consensus regime where the utility predicate is most informative.
EXPERIMENTAL DETAILS
5.1 DATASETS AND PREFILTER
We sample test prompts from four datasets spanning open-ended and closed-ended tasks:
ShareGPT (RookoAl 2023) (open-ended; 176 messages), WildChat (Zhao al.! (2024) (open- ended; 139), MedQA (Jin et al., 2020) (medical MCQ; 108), and CaseHOLD (Zheng et al. 2021
(legal MCQ: 110). All prompts contain PIIs (open-ended: 2 3; close-ended: 2 I).
For closed-ended datasets. we ensure that all test models can correctly answer the selected questions five times, so that any further accuracy drop can be attributed to reduced disclosure rather than intrinsic task difficulty. Open-ended datasets are prefiltered to only include Pll-rich English text with a clear task. Detailed curation criteria are given in App. C
5.2 MODEL SELECTION
We evaluate nine target models: 8pt-4.1-nano, gpt-4.1, gpt-5, tended thinking disabled), claude-somnet--222255000 (extended thinking disabled), Igai/exaone- and qwen/qwen2.5.
(ex-
deep-32b, mistralaiimiitraalsmaall -24b--isiit pwwen/gwen2.5-bbimmstne 0.5b-instruct. This set covers wide range of capacity model families, from frontier closed- weight models to small, open models suitable for on-device deployment. Two targets ex- pose reasoning modes and reasoning.ss fortt-rreeee -medi um) and seni-exanne-eee--e (provider default reasoning mode). All other targets are instruction-uun chat models.
with their default settings: gpt-5 (default reasoning profile;
5.3 EXPERIMENT I: ESTABLISHING DATA MINIMIZATION ORACLES
We applied our framework to search data minimization, using the nine target models the response- generation model F prompts sampled from the four datasets. We report data minimization results as the optimal percentage of REDDACT/BBSTRACTREEET
actions under the utility constraint.
To verify that minimization robustly reduces recoverability of masked information (redacted or ab- stracted) from the message itself, we run two black-box adversarial audits that attempt to simu- late on-text inference by adversary (Staab et al. 2024). Type-wise recovery: Given the text and the set of rypes that were marked during minimization, the attacker must output up to three verbatim candidates per requested type with confidences, relying only the given text. We eval- uate the same attacker on both the original input æ and the minimized input ă with an identical type set. For each type, compute Hit@ 1/Hit@3 against the corresponding gold strings. Span- wise recovery: Given the minimized text ã and the list of replacement strings actually inserted by our pipeline (c.g.. [NAME1] or abstraction phrases), the attacker must, for each span, return single guess of its original string or "Unknown' with confidence O if it cannot be recovered from this message alone. We use two LLMs different from the nine target test models as attackers: one open-weight model Ameta-1a lama/ / ama-3. 10000-0----0 instruct) and one closed model
5.4 EXPERIMENT II: BENCHMARKING ZERO-SHOT LLM DATA MINIMIZATION PREDICTORS
With the oracles in place, we evaluate the selected models in the prediction setting: given an input, the model must directly choose action from {RETAIN, ABSTRACT, REDACT) for each detected span to produce the most guidance, search, any in-loop utility judge.
variant while preserving utility, without comparator

---

Published as a conference paper at ICLR 2026
The prompt provides the message. span types, span variants, and the replacement strings that would be applied if chosen. We parse the model output into an action map; invalid actions are repaired with schema-only prompt, and undecided spans marked and excluded from conditioned ratios.
For each item and predictor model m, we pair the oracle minimized prompt o with the predicted one =((m) to evalute with the same pairwise privacy comparator and utility predicate in the search process. We classify (item, m) into four disjoint categories: Overshare if prediction leaks more privacy than oracle), Undershare+ Fail if prediction is more protective but fails utility, Un- dershare+ Pass if prediction is more protective and passes utility. Fit if prediction ties the oracle on privacy and passes utility. The first two categories are considered unsuccessful minimization, whereas the latter two represent successful minimization.
6 RESULTS
6.1 DATA MINIMIZATION ORACLE
Our minimization oracles show frontier models achieve the most privacy protection without vio- lating the utility constraint (Table 2 On open-ended task prompts, gpt-5 achieves the most ag- gressive 65.7% REDACT and 8.6% ABSTRACT (only 5.7% RETAIN)-whil the smallest model (gwen2.5-0.5bh sits at the bottom with 19.3% REDACT and 11.0% ABSTRACT (69.7% RETAIN). Closed-ended tasks admit even more minimization: gpt-4.1 tops the board at 98.0% REDACT and 1.0% ABSTRACT (1.0% RETAIN), whereas qwen2.5-0.57 again trails with 32.1 REDACT and 11.7% % ABSTRACT. The scatterplot Fig. 2 shows frontier models clustered near the z+y=1 band, confirming that very little PII must be retained to preserve utility.
Overall, minimization redaction-heayy abstraction stays small (typically 1-12%), indicating that simply deleting sensitive spans is usually sufficient for the utility constraint. Smaller models accept far less minimization both settings, which is acceptable in practice because they are more feasible to be deployed on-device, posing lower leakage risks. A cross model Jaccard analysis (App. H) fur- ther shows that, despite differences in the exact minimized prompts, redaction decisions are highly consistent across model families. The majority of cross-model variation arises instead from the much smaller abstraction set, which both explains the larger fluctuations observed in abstraction and suggests that the core redactions transfer well across models.
Open-ended Abstract 1
Closed-ended Abstract t
Response Generation Model
Redact f
Retain
Redact
Retain +
gpt-5
85.7% %
8.6% %
5.7 %
97. 1%
1.8%
1.1%
gpt-4.1
82.6%
9.9%
7.6%
98.0%
1.0 %
1.0%
gpt-4.1-nano
79.6%
10.0%
10.5%
91.3%
2.0%
6.7%
74.8%
11.2%
14.0%
97.2%
1.9%
0.9%
claude-3-7-sonneett 29
2022020
77.5%
10.6%
11.9%
79.5%
10.1%
10.4%
lgoi.exxone-
60.4%
17.4%
22.2%
75.0%
10.2%
14.7%
mistro-smaal-33.-224--mm--
75.3%
12.5%
12.2%
96.4%
1.7%
1.9%
qwenn..5-7hiinsrruct gwen2.5-0.5b-inss.
69.9%
12.0%
18.1%
91.7%
4.6%
3.7%
19.3%
11.0%
69.7%
32.1%
11.7%
56.2%
Table 2: Optimal percentage of REDACT, ABSTRACT, and RETAIN actions for open-ended (ShareGPT, WildChat) and closed-ended (MedQA, CaseHold) task prompts across nine models. indicates that higher better, and indicates that lower is better. Extended thinking disabled.
Span-wise Recovery. Pooling across target mod- els and grouping spans by action (T abstrac- tion consistently yields higher overall recovery than redaction every dataset: the correct-recovery rate Peer ranges 5.6-14.99% for ABSTRACT versus only 2.7-7.7% % for REDACT. Importantly, the absolute rates are low across the board (all Peor < 0.15, with REDACT < 0.077), indicating that on-text inference
Table 3: Span-wise recovery pooled across target models: Peor by action across (rows) datasets (columns).
Action CaseHOLL
MedQA
ShareCiPT
WildChat
abstract redact
0.092
0.056
0.149
0.119
0.050
0.027
0.051
0.077

---

Published as a conference paper at ICLR 2026
CaseHOLD
MedQA
+
0.8
0.8
Abstract ato
Daude-3-7-sunnee
0.6
0.5
caude-3:2 77-sonet--202022 p--edicccttoonn
Abstrect
sonnet 20250514 oracle
0.4
0.4
Boude--oneet-4-222200 1-p2eecciiioooocccioon
0.2
0.2
gpt-i I-oracle
gpt-4.1.p 1-prececci
0.0
gpt-4.1 2et-4,1-nan-- oracle
0.0
0.5
1.0
0.0
0.5
1.0
gpt-4.1-nanc,
Redact
Redacc
gpt-S-orarle
ShareGPr90K
WildChat
apt-S-prediction
46
Igalexaone deep-320 oracle
Igelexaone-deep- Setperee-eeee----pee-eee pedictioon
qwer//een2 St-iinstruct-oradi
Abstrect retio reto
Abstact reto
Mndtraaaimistral.co small 24b instruct-arccls
0.6
0.5 &
mistralaimistrall 3.12-l----b---tttruttt prediccioo
0.4
0.4
ower/qwen2 5 --b-insttuuc--rrcce
owerlgeen2 --7-iinttrct--reeiiii
0.2
0.2
0.0
0.0
0.0
0.5
100
0.0
0.5
1.0
Redact ratlo
Redact
Figure 2: Oracle vs, Prediction REDACT and ABSTRACT Ratio.
is generally difficult under our setup. The separation is larger on open-ended data than on closed- ended data, suggesting that open-domain context leaves more clues. Redaction is more robust to on-text inference than abstraction-attackee overall recovery remains low, reinforeing a redact-first policy when minimizing leakage, especially for open-ended inputs.
both atlempl less and succeed less after REDACT-and
parallel span-level evaluation with GPT-5 as the attacker on its own leads the same conclusion. Across datasets, correct-recovery rates remain low for both abstraction and redaction spans, and masked spans are overwhelmingly labeled as UNKNOWN. Full results are reported Appendix [.2] Together, these findings show that GPT-5 is unable to reconstruct the removed private information even when attacking its own oracle-minimized prompts.
Type-wise Recovery (original vs. masked). Aggregating by entity type, masking causes sharp drop recoverability relative to the original text. For example on WILDCHAT (Hit@) %). NAME falls from 90.3 to 0.0, GEOLOCATION from 89.8 to 2.2, OCCUPATION from 85.4 4 to 8.0, and AF- FILIATION from 83.0 to 1.9; other datasets show the same pattern (Appendix L Hit@3 mirrors these trends across types. In short, masking severely limits type- wise recovery. Consistent with the span-level results, a parallel test using GPT-5 as the attacker on its own minimized prompts shows the same pattern: masked @it@1 for every type stays in the low single digits while the corresponding original values are often near the top of the scale, indicating that GPT-5 does not infer the removed PII.
Taken together, the span-wise and type-wise recovery checks confirm that our search-based data minimization method effectively strips sensitive information from prompts and prevents that infor- mation from being inferred indirectly from the remaining context.
6.2 PREDICTION VS. ORACLE
As shown in Fig. 3 single-pas. pale- predictions are generally less savvacy-preeevvngg than the gpt-5 oracle Overshare dominates across indiciinii that these direct predictions with- out comparator-guide search tend to under-protect privacy with frontier models which are most widely used and vulnerable to more privacy risks. Items counted as Undershare+ tempts to push masking beyond the oracle that break task utility. A meaningful siice--sppccalllly open-ended datasets falls into Undershare+ PASS, signaling headroom to further tighten the ora- cle's de's comparator priorities stop rule. The Fit mass (privacy tie utility pass) is small, suggesting the prediction rarely sits close to a task-wise privacy/utility frontier. Oracles are harder to surpass in the close-ended, answer-verifiahle tasks (MedQA is near-all Overshare, while CaseHOLD still
reflect at-
8

---

Published as a conference paper at ICLR 2026
shows non-trivial Undershare+PASS and Fit). Minor stochasticity in gpt-5 decoding is mitigated via replace-bace back, and k=5 repetition on verifiable tasks.
M5
MAX
MO
Figure Prediction vs. oracle minimization across datasets. Each panel shows per-model stacked proportions that sum to 1. Outcomes are interpreted relative to the gpt-5 oracle using our privacy comparator (Sec. 3.2p and utility predicate (Sec. 5): Overshare-the prediction disclosure is less privacy-preserving than the oracle; Undershare+FAAll--ttee FAll.-the prediction hides more but fails the utility check; Undershare+ PASS the prediction hides more and passes utility; and Fit-the prediction ties the oracle privacy and passes utility.
Prediction bias toward ABSTRACT. In single-pass predictions. models consistently favor AB- STRACT over REDACT, showing an abstraction-first default on everyday user prompts (e.g., trip planning). Because ABSTRACT less privacy-preservin in our setup, choosing when REDACT would still retain utility implies unnecessary disclosure. This tendency persists even when instruc- tions explicitly indicate that the protection strengths REDACT is higher than ABSTRACT; and it contrasts with the oracles that cluster in the Bingh-REDDCT/OOWWAATTEE
regime (cf. Fig. 2) We also tested whether the abstraction preference arises from our prompt design by ablating the minimization-orde instruction. As detailed in App. G.4 removing either the "prefer stronger" clause or the entire minimization order line leaves model behavior nearly unchanged: all the selected models for this further test (GPT-5, Mistral-24B, and Qwen2.5-7B) still strongly prefer ABSTRACT, indicating that the bias is model-internal rather than proupt-induced.
Ablation by model family. Results show stable, high-level biases as illustrated by the prediction- side clusters in Fig. 2 datasets-even for structured identifiers-e.g. on ShareGPT and WildChat they abstract nearly all URL/EMAII/ID NUMBER spans with <1-2% redact, with slightly lower abstraction rates for con- textual attributes such as GEOLOCATIO CINOCAAAIIINIIIIII where abstraction remains predominant. Claude adds a pronounced RETAIN tail on open-ended prompts (large fractions of GEOL.CATTOO,, TIME, AFFILIATION kept), with little redaction. By contrast, the two reasoning models GPT-5 and Exaone are the only ones that consistently redact high preciprrcision types: on closed-ended Case- HOLD/MedQA they heavily redact DRNE/TTME//EOOLCAATOON and on open-ended chats they far more willing than other families to redact URL/EMATTL/HOOEE..E NUMBER.
Mistraa/(Qwen/GPT-.111 default to an abstract-first policy across
For completeness, we also observe that fully masking all detected Plls, as would occur in a simple NER-based redaction, often breaks utility. Together with the oversharing behavior of single-past predictions, this suggests that neither extreme is adequate, and an oracle is needed to determine how much masking each model can tolerate.

---

Published as a conference paper at ICLR 2026
7 CONCLUSION AND DISCUSSION
We present framework that formally defines and operationalizes data minimization in LLM prompting: for a given user prompt and response model, it quantifies the minimal privacy-revealing disclosure required to maintain utility. Our results show that data minimization offers a significant optimization space for reducing privacy exposure without compromising task performance, partic- ularly for larger and more capable language models. However, we find that directly predicting this minimal disclosure is challenging, even for frontier models. This work lays the groundwork for re- search on quantifying data minimization and robust prediction methods, fostering both fundamental machine learning advances and interdisciplinary research in human-AI interaction.
Novel Paradigm of model is, the more feasible data minimization becomes. This result shows that data minimization is a promising approach to addressing excessive disclosure problems in user interactions with LLM systems, as users tend to trade privacy for utility and therefore often choose frontier models hosted on the cloud for sensitive tasks despite privacy concerns (Zhang et al., (024). The variances of data minimization across datasets and models suggest that model-specific predictors needed, and we advocate that LLM providers include these as part of the released model package. Such predictors naturally align with an emerging line of work that explores dual-model management approach: using small edge models for data- maaa-mmiiiiiiiiiniimiimi -uided local sanitization before sharing data with the remote model (Li et al.) 2025b) Zhou et al.) [025 Zhang et al., 2025 Chowdhury et al., 2025). Beyond these observations, our results also clarity the technical role of the oracle within this workflow. The oracle procedure identifies the upper bound of data minimization a target model can tolerate while preserving utility, providing high quality supervision for learning practical sanitization policies. This supervision can train or distill a small predictor that performs single pass span level decisions locally, complementing the dual model management approach described above. This establishes a natural path toward future on-device predictors that give users full control over the flow of private data before any interaction with a remote model.
LLM Interactions. We show that the more capable the
LLM Capabilities for Privacy Tasks, We evaluate LLM capabilities on two novel privacy tasks: data minimization prediction and privacy sensitivity ranking (by the privacy comparator), extending prior work on using LLMS for PII detection and context-aware privacy judgments (Mireshghallah ctt al.) [022b; [Shao et al.] 2025 Li et al., 2025a). We find that data minimization prediction re- mains challenging for current state-of-the-an models. For the privacy sensitivity ranking task, we found that off-the-shelff reasoning models (e.g.. GPT-ol, o3, and o3-mini) perform better than non- reasoning models (e.g., GPT-4o). Future research should further account for individual preference differences, as our results show that in over half of cases the five human raters reached a consensus score below 0.8. A failure case analysis of the pest-performing models reveals where misalignment still occurs. In these cases, humans often choose "SAME," while models prefer "A" or "B," reflecting different thresholds for saliency: models overemphasize subtle distinctions that seem significant to them but are imperceptible or irrelevant to humans. Moreover, models tend to overvalue specificity and do not align with humans on how the specificity of certain data types corresponds to sensitivity (e.g., assigning more weight to time or date information than to names).
Interpretation Methods for "What is Necessary." Foundational understanding of what informa- tion or tokens are necessary is still required to explain the variance observed in data minimization oracles across models and datasets. Current methods can reveal what information is used at infer- ence (Vig et al. 2020), but determining what is truly necessary remains an open research frontier. In addition, the potential impact of test set contamination (Oren et al. 2024) should be carefully taken into consideration in future investigations.
ETHICS STATEMENT
All datasets are publicly available under their respective terms; we do not crawl private sources. All human-subjects studies have been approved by our institution's IRB. Our human evaluation collects no PII of the human raters. Annotators only state preferences over sanitized replacements; no demographics recorded and unanonymized content is shown.
10

---

Published as a conference paper at ICLR 2026
REPRODUCIBILITY STATEMENT
We release our code at the project GitHub repository The pipeline code used in the oracle experi- ment (Section 5.3) eun.pipeline py. We also release the trained privacy comparator as LoRA adapter on HuggingPace enabling end-to-end reproduction of our action-search pipeline. The folder Prefilters datasets corresponds to the four test datasets described in Section 5.1 The file
json1 contains human annotator tallies (tal lly) and 0mmini judgments used to (i) select the best teacher model, (ii) use the teacher to generate large imput sets, and (iii) train the privacy comparator (Section (.2); we also use the same human an- notations to evaluate the comparator's accuracy. Setup notes, and example commands are provided in the repository README.
REFERENCES
Martín Abadi, Andy Chu, Ian Goodfellow, H. Brendan McMahan, Ilya Mironov, Kunal Talwar, and Li Zhang. Deep learning with differential privacy. In Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security (CCS), pp. 808, 2016. doi: 145/ 29783119
2976749.2978318. URL htttpt:/:///.... acm. www.aao...................oo.. porg/doi/100 11t.://.1..111/29990
Bärbulescu and Peter Triantafillou. To each (Textual sequence) its own: Im- proving memorized-dai unlearning in large language models. Ruslan Salakhutdinov, Zico Kolter, Katherine Heller, Adrian Weller, Nuria Oliver, Jonathan Scarlett, and Felix Berkenkamp (eds.), Proceedings of the 41st International Conference on Machine Learning, volume 235 of Proceedings of Machine Learning Research, pp. 3003-3023. PMLR, 21-27 Jul 2024. URL html
httpps:/pproceed.ngs mlr htt.://........eeeee//////aabae.
Ann Cavoukian et al. Privacy by design: The 7 foundational principles. Information and privacy commissioner of Ontario, Canada, 5(2009): 2009.
Amrita Roy Chowdhury, David Glukhov, Divyam Anshumaan, Prasad Chalasani, Nicolas Papemot, Somesh Jha, and Mihir Bellare. Prcempt: Sanitizing sensitive prompts for lims. arXiv preprint arXiv:2504.05144 2025. TRL/https://doi.orrtt
185.444550////. 2504 05147
Yao Dou, Isadora Krsek, Tarek Naous, Anubha Kabra, Sauvik Das, Alan Ritter, and Wei Xu. Re- ducing privacy risks in online self-disclosures with language models. In Lun- Lun Ku, Andre Martins, and Vivek Srikumar (eds.), Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume Long Papers), pp. 13732-13755 Bangkok, Thailand, August 2024. Association for Computational Linguistics. doi: 10. 10.18665511/20042allllooo long.
URL
Fast Company. Google is indexing conversations with httpspt. www. wtt.s//ppaa..cctooppa....... com/ 9137913777
2025.
Gadget Review. Grok's privacy disaster: 370,000 ai conversations exposed on google. Technical report, 2025.
Edward J Hu, yelong shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. LoRA: Low-rank adaptation of large language models. In International Con- ference on Learning Representations, 2022. URL htttpe://penrreview.. wttp:riwwwwwweeeeeeeeeeeeeeeeeee net/forumm
id=nzev/eeFyfs
Di Jin, Eileen Pan, Nassim Oufattole, Wei-Hung Weng, Hanyi Fang, and Peter Szolovits. What dis- ease does this patient have? lcage-scale open domain question answering dataset from medical exams. arXiv preprint arXiv:2009, 13081, 2020. URL whtttt:://// //tp:///aaivvvv wwwp:iivwrgg//////////aa//
13081
Zhigang Kan, Linbo Qiao, Hao Yu, Liwen Peng. Yifu Gao, and Dongsheng Li. Protecting user privacy in remote conversational systems: A univacy-preserving framework based text sani- tization. arXiv preprint arXiv:2306.08223, 2023. URL http:httt:///// //tp///axvvvv/// www.aaiww......a.aaaoaaoaa abbrg//bb//3350
08223

---

Published as a conference paper at ICLR 2026
Haoran Li, Wenbin Hu, Huihao Jing, Yulin Chen, Qi Hu, Sirui Han, Tianshu Chu, Peizhao Hu, and Yangqiu Song. PrivaCI-bench: Evaluating privacy with contextual integrity and legal com- pliance. In Wanxiang Che, Joyce Nabende, Ekaterina Shutova, and Mohammad Taher Pilehvar (eds.), Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume I: Long Papers), pp. 00544-10559 Vienna, Austria, July 2025a. Association for Com- doi: 10. 1/2653/112225...e. acl-long.518 URL
putational Linguistics. ISBN 2025g/2222...m acl long. 5187]
https: Setts::/aacantho..oo wttp:/hhwwoooyooog/2e2eeeee wwt.
Siyan Li, Vethavikashini Chithrra Raghuram, Omar Khattab, Julia Hirschberg, and Zhou Yu. PA- PILLON: Privacy preservation from Internetbbased and local language model ensembles. In Luis Chiruzzo, Alan Ritter, and Lu Wang (eds.), Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume I: Long Papers), pp. 3373371--11 3371-3390, A Albuquerque, New Mexico, April 2025b. Association for Computational Linguistics. ISBN
doi: 10.18653/v1/2055 naw.ll...acccle.......mm.. waw.aaaaallloon111111 173/
naacl-long. 173. /tL.tttss//aaclaattoooo..../
Hongru Ma. Wenpeng Lu, Yanjie Liang. Tianyi Wang, Qi Zhang, Yingjie Zhu, and Jiasheng Si. Alsa: Context-sensitive prompt privacy preservation in large language models. In Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining V.2, KDD '25, pp. 2042-2053, New York, NY, USA, 2025. Association for Computing Machinery. ISBN 9798400714542. doi: 10.1145/371188933336684
URL https://eci.org///
3711896. 3736840
Meta Security Team. Meta ai chatbot security flaw exposes user conversations, July 2025, Security Advisory.
Niloofar Mireshghallah, Maria Antoniak, Yash More, Yejin Choi, and Golnoosh Farnadi. Trust no bot: Discovering personal disclosures in human-I... conversations in the wild. In First Conference on Language Modeling, 2024a. WRRLhttps:///peeeeeett
net forum?id
tIpWtMYkZU
Niloofar Mireshghallah, Hyunwoo Kim, Xuhui Zhou, Yulia Tsvetkov, Maarten Sap, Reza Shokri, and Yejin Choi. Can Iims keep secret? testing privacy implications of language models via contextual integrity theory. In International Conference on Learning Representations (ICLR), 2024b. URL https:: http:::///ooceeednng........ icl.iiinnniiii.r.mmmmommmm cc saaeeer_fiieeeeee wwtp://fleeeeeeee/////.. hash/ 0asshh0000008822220999999999 63ee1133673332 ff -atpreootooo-ttttttttttt-ooooooo Cew.traactt-Cooffeencce. htm
See also arXiv:2310.17884.
Ivoline C. Ngong, Swanand Ravindra Kadhe, Hao Wang, Keerthiram Murugesan, Justin D. Weisz, Amit Dhurandhar, and Karthikeyan Natesan Ramamurthy. Protecting users from themselves: Safeguarding contextual privacy in interactions with conversational agents. In Wanxiang Che, Joyce Nabende, Ekaterina Shutova, and Mohammad Taher Pilehvar (eds.), Findings of the As- sociation for Computational Linguistics: ACL 2025, pp. 26196-26220. Vienna, Austria, July 2025. Association for Computational Linguistics. ISBN 979-8-99977-7222 doi: 10.18653/vI/ 2025 1025.findings-ace 134.3 URL https: hatpp::/acclattiooooy..oo wwgp://hhw..oooo.....ooo.o22 findings acl. 13437
Yonatan Oren, Nicole Meister, Niladri S. Chatterji, Faisal Ladhak, and Tatsunori Hashimoto. Prov- ing test set contamination black-box language models. In The Twelfth International Confer- ence on Learning Representations, 2024. URL htttp:://openreeee/ee http:/iwwww.eeeeeeeeeeeeeeeeeeee net wwtp:/runteeeeieeooomoomoomm KS8mTvetq2
European Parliament and Council. Regulation (cu) 2016/679 of the curopean parliament and of the council of 27 april 2016 on the protection of natural persons with regard to the processing of personal data and on the free movement of such data (general data protection regulation).
hhttps://eur 1ex. http://www.......oooo///.//////d
2016.
RyokoAL Sharegpt 90k conversations. /tttps: co/ httteeeete/.tttttt///kkk..k
ShareGPT52K] 2023. Accessed: 2025-09-24.
12

---

Published as a conference paper at ICLR 2026
Yijia Shao, Tianshi Li, Weiyan Shi, Yanchen Liu, and Diyi Yang. Privacylens: evaluating privacy norm awareness of language models in action. In Proceedings of the 38th International Con- ference Neural Information Processing Systems, NeurlPS 24, Red Hook, NY, USA, 2025.
Curran Associates Inc. ISBN 9798331314385.
Robin Staab, Mark Vero, Mislav Balunović, and Martin Vechev. Beyond memorization: Violating privacy via inference with large language models. In The Twelfth International Conference on net forum?id=
Learning Representations (ICLR), 2024. URL betts:://ppereet
kmnOBhok7pp Spotlight.
Theori Research. Deepseek security, privacy, and governance: Hidden risks in open-source ai. Technical report, Theori Inc., January 2025.
Jesse Vig, Sebastian Gehrmann, Yonatan Belinkov, Sharon Qian, Daniel Nevo, Yaron Singer, and Stuart Shieber. Investigating gender bias in language models using causal mediation analysis. In H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin (eds.), Ad- vances in Neural Information Processing Systems, volume 33, pp. 1248-114001. Curran Associates, Inc., 2020. URL hhttps::/proceedddnnnn neur cc/paper files/ Pappr.
Synthia Wang. Sai Teja Peddinti, Nina Taft, and Nick Feamster. Beyond PII: How users attempt to estimate and mitigate implicit LLM inference. arXiv preprint 12Xi::2509912152., 2025. URL
https://arxix: org/ /.ds/2500 2501.1212
Hang Zeng, Xiangyu Liu, Yong Hu, Chaoyue Niu, Fan Wu, Shaojie Tang, and Guihai Chen. Au- tomated privacy information annotation in large language model interactions. arXiv preprint arXiv:2505.209100 2025. URL /oi.. orq/100 2tt//110.8855550 48550/ arwiv. 2505....550..2. 20910
Juhua Zhang, Zhiliang Tian, Minghang Zhu, Yiping Song, Taishu Sheng, Siyi Yang, Qiunan Du, Xinwang Liu, Minlie Huang, and Dongsheng Li. DYNTEXT: Semantic-aware dynamic text san- itization for
LLM inference. In Wanxiang Che, Joyce Nabende, Ekaterina Shutova, and Mohammad Taher Pilehvar (eds.), Findings of the Association for Computational Linguistics: ACL 2025, pp. 20243-20255, tational Linguistics. ISBN 979-8-89176-255--
Vienna, Austria, July 2025. Association for Compu- doi: 10.18653/v 00188653/112222555 findings- a03.. URL bttps://aclanthol..s www.eeaaaannooooooooooooo22..... wttp/////22222.... http://22222...ddnggeeeeeee Findings- 1038/
Zhiping Zhang, Michelle Jia, Hao-Ping (Hank) Lee, Bingsheng Yao, Sauvik Das, Ada Lerner, Dakuo Wang, and Tianshi Li. "it's a fair game", or is it? examining how users navigate disclosure risks and benefits when using lim-based conversational agents. In Proceedings the 2024 CHI Conference on Human Factors in Computing Systems, CHI '24, New York, NY, USA, 2024. Association for Computing Machinery. ISBN 9798400703300.
doi: 10. 00.1145/361399443442383
URL httphttss://doooooo// aoooi.org//00.. 111.111.11111111
1145/ 1444/313//3444444 3642385
Wenting Zhao, Xiang Ren, Jack Hessel, Claire Cardie, Yejin Choi, and Yuntian Deng. Wildchat:
Im chatgpt interaction logs in the wild. arXiv preprint 2rXiv:2405.01477, 2024. URL https:
//aaxiv.orq/abll http://bb//244000 2400 01470
Lucia Zheng, Neel Guha, Brandon Anderson, Peter Henderson, and Daniel E Ho. When does pretraining help? assessing self-supervised learning for law and the casehold dataset. arXiv 104.00044.7. 08670,66711 2021. URL httthhtts:///aaae
preprint arxiv 044
wttg/ass////aae/ 08671
Jijie Zhou, Eryue Xu, Yaoyao Wu, and Tianshi Li. Rescriber: Smaller-LLM minimization for LLM-based chatbots. In Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems (CHI '25). ACM, 2025. doi: 10. 05.114557766998831177011 URL
user-led data
org/ abs/ 2410 11876
Also available as arXiv:2410.11876 (v3, Feb
2025).
A LLM USAGE
Outside of minor manuscript assistance, all LLM usage in this work solely for running the exper- iments reported in the main text (PII detection/aujjudication abstraction-temi synthesis, prediction runs, and adversarial audits), with model choices and procedures fully specified in Sec.
13

---

Published as a conference paper at ICLR 2026
Manuscript assistance. We used LLMs only for light language polishing and LTEX table format- ting (e.g., caption phrasing, column alignment). No technical content or analysis was delegated, and all edits were author-verified.
B PRIVACY COMPARATOR TRAINING HYPERPARAMETERS
Dwen2.5-7B-Instruue (distilled final). learning rate = 1e epochs = 2: LoRA rank = 8; context length 2048; global batch size 2048 (Fireworks Al). On human-annotated evals (consensus 0.8), accuracy = 89.04%.
C DATASET CURATION
Close-ended Dataset (MedQA and CaseHold) Each item is answered by all nine target models with five runs temperature on the original (unmasked) input; keep an item only if every model is correct in all five runs. This ensures that any accuracy drop under minimization is attributable to masking rather than intrinsic task difficulty.
Open-ended Dataset (ShareGPT and WildChat) We curate high-quality subset to study how far inputs can be minimized before utility degrades. Specifically, we (i) select PII-rich messages using Microsoft Presidio, requiring least three distinct PII categories; (ii) retain primarily English content via fastText; (iii) remove near-duplicates using MiniLM embeddings; (iv) exclude code- heavy prefixed web dumps and templated/meta prompts lacking self-contained intent; (v) discard prompts without a clear task intention; and (vi) validate PII triggers with a GPT-4o checker, confirm- ing least three unique, text-based entities per message. For WildChat, we additionally restrict to messages without pre-existing redactions. The final sets comprise over 500 ShareGPT messages and over 100 WildChat messages. The ShareGPT pool supports replacement-pait creation for the human study and comparator distillation, and both sets are later reused for pipeline operationalization.
D DETECTION AND ABSTRACTION SYSTEM PROMPT
D.1 DETECTION
You are expert in cybersecurity and data privacy. You now tasked to detect PII from the given text, using the following taxonomy only:
NAME: Name
EMAIL: Email address
THONE_NUMBERS Phone number
ID: Identifiers, including ID Number, passport number, SSN, driver's license, taxpayer, identification number
ONLINE.IDENTITY:
IP address, username, URL, password, key
GEOLOCATTON: Places and locations and addresses, such as cities, provinces, countries, international regions, named infrastructures (bus stops, bridges, etc.).
AFFILIATION: Names of organizations, such as public and private companies, schools, universities, public institutions, prisons, healthcare institutions, gon-govvrnmental organi- zations, churches, etc.
DEMOGRAPHIC.ATTRIDUU Demographic attributes of person, such as native lan- ages, sexual orientation, gender and sex.
guage, descent, heritage, ethnicity, nationality, religious political group, birthmarks,
TIME: Description of specific date, time, or duration.
HEALTH ONFORMMTOON: Details concerning an individual's health status, medical con- ditions, treatment records, and health insurance information.
FINANCIAL INFORMATION: Financial details such as bank account numbers, credit card numbers, investment records, salary information, and other financial statuses actívities.
14

---

Published as a conference paper at ICLR 2026
EDUCATIONAL.RECOR) Educational background details, including academic records, transcripts, degrees, and certification.
For the given message that user sends to chatbot, identify all the personally identifiable informa- tion using the above taxonomy only, and the ent entity-type should be selected from the all-caps categories. Note that the information should be related to real person not in public context, but okay not uniquely identifiable, Result should be in its minimum possible unit.
Return only
JSON object in the following ("entity-type" COOLDECIDE_.TE_.TT.TTT "text":
format:
"results":
D.2 ABSTRACTION
Rewrite the text to abstract the protected information, without changing other parts. Please output in JSON format only.
Input:
<Text>I graduated from CMU, and earn a six-figure salary.
Today a lady from China <.Texxt oereececetccnaaeeoooooorrrrrrrrr
Output:
("zesults": [["protected"; "CMU", "abstracted": "a prestigious American university"),
("protected": "China", "abstracted"; "A country in Asia"]]}
E UTILITY CHECK
Utility Evaluator System Prompt You are utility evaluator. Your task is to assess whether second response adequately addresses user's original message. You will be provided with: (i) the user message (what the user is asking or talking about), (ii) a reference response A (a standard answer that is known be good), and (iii) a candidate response (a second version that needs be evaluated). You must decide whether the candidate response is "PASS" or "FAIL". This is not test of exact wording or stylistic similarity; focus on whether the candidate response addresses all key points needs expressed in the user message. If it does, it passes. If it fails to address one more key points, it fails.
Input will contain three tagged sections: <response.B>.
<user message, <response_A>, and
Return JSON with the result and one-line explanation: "Reason": EXUR.EXP
"Result"; "PASS/FAIL",
F UTILITY THRESHOLD SENSITIVITY STUDY
To evaluate how sensitive our method to the choice of the utility threshold y ($3.I) in the open- ended task setting, we performed an additional experiment on the ShareGPT and WildChat datasets. In the main paper, the utility predicate is binary because the evaluator (GPT-4o) returns categorical judgments. For this study, modified the evaluator's instruction that produces numerical utility score between and 10, where the model's original response is defined as a score of 10. A candidate transformation passes the utility predicate if its score least 10 7
We evaluated three thresholds, = 0, 1, corresponding to minimum required utility scores of 10, and 8. All other components of the pipeline remained unchanged.
To determine whether the resulting quality differences are perceptible to users, we conducted hu- man evaluation. We recruited 90 English speaking participants located in the United States using
15

---

Published as a conference paper at ICLR 2026
Prolific. For each of 90 sampled prompts, generated pairwise comparisons between (i) the origi- nal response and (ii) a response produced under one of the y settings. Participants selected whether A, B, SAME was of higher quality.
The results were as follows:
7 O: SAME 36.8%, original preferred = 32.6%, masked preferred = 30.5%
7 1: SAME 30.9%, original preferred 42.5%, masked preferred 26.66
7 SAME 28.1%, original preferred 44.1%, masked preferred 27.8%
The trend is clear. As y increases, the proportion SAME judgments decreases and preference for the original output increases. This shows that small relaxations of the utility threshold produce user perceptible quality degradation. These findings justify the use of a strict binary predicate in 3.2 which prohibits any reduction utility and therefore represents a conservative and safe choice for minimizing disclosure.
G PROMPT SPECIFICATIONS FOR ZERO-SHOT PREDICTION
G.1 SMALL-MODEL. FAILURE.
We mark the smallest-capacity model (Qwen2.5-0.5B) N/A for the single-pass prediction due persistent schema/IO failures: (i) emitting actions outside {RETAIN, ABSTRACT, REDACT); (ii) us- ing PII types as keys instead of entity IDs; and (iii) remaining unstable even with reduced prompts. Notably, when we prefill an all-RETAIN map inside the prompr to constrain decoding, the model either (a) echoes the seed map (no decision (ecision-maaing) verbatim or (b) still produces noncompli- ant/irrelevant text, failing to follow the output schema. Given this degenerate behavior, single-pass prediction meaningful so we did not use this model as predictor in the experiments.
G.2 SYSTEM PROMPTS
Open-ended tasks (ShareGPT) WildChat). You are a PII Minimization Transformer. Re- turn one line: single JSON object (no markdown/prose). Goal: for each PII surface string in pii-dict, choose the most privacy-preserving action that keeps response quality for open-ended tasks. Order: RETAIN < ABSTRACT < REDACT (prefer stronger only if quality unchanged). Schema: "transformation "<<II"X<PIII""""""""" " "retain abstract redact" "explanation": "ONE SHORT SENTENCE, NO RAW PII".
Rules: (1) Keys must be the exact surface strings from pii dict (verbatim; do not addrremove/ennmme) (2) Values are exactly one of retai labst redact (lowercase). (3) If an action is chosen, it applies to all variants in variants_map (4) If the needed entry is missing in abstracttmapprert maapp///dact.aap....m. map, default to retain for that key, (5) Keep key order aligned with pi pii-dict if possible.
Deterministic tasks (MedQA / CaseHOLD). Same (A) except the Goal: choose actions that still allow selecting the correct option in multiple-choice setting. Use the same order, schema, and rules.
line JSON only: frrransfoormtiioo :f... H. Inputs: pii.dict and draft.tranee ormat on where every key (verbatim from pi si_diiiit) is preset "retain". Edit only the values one of retain| abst redact. Do not add/remove/rename any key.
Tiny-model strict prompt (values-only edits). Return
G.3 USER PAYLOADS
Regular payload (with maps). Fields: message (original text), pii.dict (fsurface: type)), variants map ([surface: [aliases] | }), redact map/abstract map (per- surface replacements).
16

---

Published as a conference paper at ICLR 2026
"Edit values only to one of ("transformation" <edited draft>}."
rredcct; do not change keys. Return
G.4 ABLATION OF MINIMIZATTON-ORDEE
To test whether the observed preference for ABSTRACT arises from our prompt design, we con- ducted two ablations of the system instruction used for single-pass prediction. The first variant (order only) removes the clause "prefer stronger only if quality is unchanged," while the second (no order/notion) removes the entire minimization-order line. We ran GPT-5, Mistral-24B, and Qwen2.5-7B on ShareGPT and MedQA using all three prompt types, keeping all other settings fixed. These three models regimes, and ShareGPT (open-ended) and MedQA (closed-ended) serve as one exemplar dataset for each task type. As shown in Table removing these instructions does not eliminate the strong pref- erence for ABSTRACT; redaction increases marginally, but the overall pattern remains unchanged. This indicates that the abstraction bias not prompt-inducee but reflects model-internal tendency toward theency-preservng transformations.
chosen to cover a representative range of capacities and training
datasel
modell
prompl-iype
redact
abstract
relain
undecided
ShareGPT
gpt-5
order+nation
663
363 (30.5%)
(0.0%)
ShareGPT
gpi-5
order only
225 (18.9%)
656 (55.1%)
309 (26.0%)
D (0.0%)
ShareGPT
gpt-5
no order/nation
(12.9%)
(61.22)
309 (26.0%)
(0.0%)
ShareGPT
mistrul-maall-24
order-nation
6 (0.5%)
1155 (97.1%)
29 (2.4%)
0 (0.0%)
ShareGPT
mistral-small-244
order only no order/notion
(0.7%))
1070 (89.9%)
100 (8.4%)
(0.0%)
ShareGPT
mistral-small saatrll-mmal-2440
20.7775%)
1027 (86.3%)
143 (12.0%
D (0.0%)
ShareGPT
qwen2.5-7b
order-nation
(6.3))))
1174 (98. 7%))
0 (0.0%)
0 (0.0%)
ShareGPT
qwen2.5-7b
order only no ordee/nntiin
38 (3.2%)
(0.4%)
ShareGPT
qwen2.5.77 7b
(11.7%)
1000 (84.0%)
46 (3.9%)
5 (0.4%)
MedQA
gpt-5
order+mation
533 (54.6%)
405 (41.5%)
(3.9%)
(0.0%)
MedQA
gpi.5
order only no order/nation
808 (82.8%)
140 (14.3%)
(2.9%)
D (0.0%)
MsdQA
gnt-5
688 (70.5%)
258 (26.4%)
(3.1%)
(0.0%)
MsdQA
mistral-amall-246
order-nation only no order/notion
(0.0%)
912 (93.4%)
64 (6.646)
0 (0.0%)
McdQA
mistral-small-240
(0.2%)
(62.4%)
365 (37.4%%)
(0.0%)
MedQA
mistril-maall224b
2 (0.2%)
531 (54.4%)
443 (45.4%%
0 (0.0%)
McdQA
qwen2.5-7b
order-nation order only
(0.0%)
973 (99.7%)
3 (0.3%)
(0.0%)
MedQA
qwen2.5-7b
(0.2%)
940 (96.3%)
30 (3.1%)
4 (0.4%)
McdQA
qwen2.5.76 7b
no order/notion
19 19.9))9%
928 (95.1%)
28 (2.9%)
(0.15)
Table 4: Ablation of minimization-order instructions across models and datasets (without total-pii
column).
H CROSS-MODEL DATA MINIMIZATION OVERLAP
This section details the cross-model overlap experiment from 6. which quantifies how similar data minimization decisions are across response generation models, and whether the minimal sufficient masking chosen by each model's oracle shows stable structure.
For each model and dataset, we compare the oracle-derived action map to the GPT-5 oracle (as reference), computing Jaccard overlap separately for REDACT and ABSTRACT. The overlap is the number of PII spans where both oracles choose the same action divided by the union of spans where either does, yielding model- and dataset-level consistency measures.
Tables S and 6 report the resulting overlaps. As summarized in $ redaction overlap is high across most models and datasets, typically at or above eighty percent, and in some datasets such as CaseHOLD it reaches above ninety percent for nearly all models. These results indicate that the majority of removed spans form a shared core of non-essential sensitive information that models broadly agree upon. In contrast, abstraction overlap is lower but abstraction itself accounts for much smaller fraction of actions, which makes its variability less consequential in practice. Taken together, these observations suggest that the essential prevaccynreserving behavior, namely which spans can be safely removed while maintaining utility, generalizes well across model families even though the exact minimally sufficient prompts remain model specific by definition.
17

---

Published as a conference paper at ICLR 2026
Dataset
Model
Overlap
ShareGPT
gpt-4.1-nano
0.802493
ShareGPT
gpt-4.1
0.845057
ShareGPT
claude337-somme-222552119
0.792776
ShareGPT
claude-sonnett-22002555144
0.754572
ShareGPT
Igai/exaone-deep-32b
0.583899
ShareGPT
mistral//misttal-ssalll.-1-22
0.743466
ShareGPT
qwen//pwen2.577--isstuuuu
0.686103
ShareGPT
qvenkkwenn25500.5--nnmtttt
0.145875
WildChat
gpt-4.1-nano
0.849807
WildChat
gpt-4.1
0.860465
WildChat
slaude3377sommee22225522111
0.797419
WildChat
claude-sonnet-42225555444
0.800771
WildChat
Igai/cxaone-deep-32bb
0.701961
WildChat
mistraliiistaalssaal----2------i
0.853816
WildChat
qwenl/qwen2.5-7--instttuc
0.794839
WildChat
qwenl/ween..--055--nnsttt
0,298128
MedQA
gpt-4.1-nano
0.875905
MedQA
gpt-4.1
0.950464
MedQA
Claud=37-sommet22225222299
0.708768
MedQA
claudessonmet-42222555144
0.952183
MedQA
Igai/exaone-dsep-32b
0.762055
MedQA
mistralimiistaammmlllll--------s
0.954451
MedQA
qwen//wwen225577-instttuu
0.889583
MedQA
qwenkk/een2.550.55--nnnttt
0.292683
CaseHOLD
g-t-no
0,945274
CaseHOLD gpt-4.1
0.985075
CascHOLD
chaude337-somett222552229
0,987531
CascHOLD
claude-someet-4222555514
0.987562
CaseHOLD
lgai/exaone-deep-32b
0.736181
CaseHOLD
0.937811
CaseHOLD qven/qwenn..5---insttutt
0.957711
CaseHOLD qven/4pwen22550055--nnrtttt
0.395522
Table 5: Cross-Model Redaction Overlap with the GPT-5 Oracle
18

---

Published as a conference paper at ICLR 2026
Dataset
Model
Overlap
ShareGPT
gpt-4.1-nano
0.203704
ShareGPT
gpt-4.1
0.256039
ShareGPT
claude3377sommee222552219
0.182648
ShareGPT
claude-oomnet-2202555144
0.178261
ShareGPT
Igai/exaone-deep-32bb
0.090625
ShareGPT
mistalalmiisraa-smll--3---bbbb
0.189189
ShareGPT
qwen/qpwen2.577--isstuuuu
0.152941
ShareGPT
qwenkkwenn25500.5--nnttttt
0.014354
WildChat
gpt-4.1-nano
0,141414
WildChat
gpt-4.1
0.132653
WildChat
slaude3377sommee22225521111
0.1 0.114035
WildChat
claude-sonnet-4222555544
0.085470
WildChat
Igai/exaone-deep-32bb
0.108974
WildChat
mistralimisttalmmmlllll---------
0.126214
WildChat
qvenn/qwen2.5-7--instuuut
0.090909
WildChat
qwenl/wenn2.50055--imsst
0.052326
MedQA
I-n-oo
0.023810
MedQA
gpt-4.1
0.093750
MedQA
claud=3--sommet2222502229
0.060403
MedQA
claudessonmet-42222555444
0.175000
MedQA
Igai/exaone-dsep-32b
0.031008
MedQA
0.111111
MedQA
qwen//wwen2255-7-inssttuud
0.090909
MedQA
qwenkk/een2.550.55--nnnttt
0.026316
CaseHOLD
g-t-o.I-nano
0.000000
CaseHOLD gpt-4.1
0.000000
CascHOLD chuud:33--ommett222552229
0,200000
CascHOLD
claude-someet-4220555514
0.000000
CaseHOLD
lgai/exaone-deep-32b
0.000000
CaseHOLD
Sssralaimmistralssmlll...22b
0.000000
CaseHOLD qven/qwenn2.5-7--nsstttt
0.000000
CaseHOLD qven//pwen22550055--nnricc
0.000000
Table 6: Cross-Model Abstraction Overlap with the GPT-5 Oracle
I PRIVACY AUDIT
1.1 POOLED PRIVACY AUDIT ACROSS GOOGLE/GEMII-FPASIIIIL55 MITA-LLLMA/LLLMMA- 1-........----RR-IEE ATTACKERS
PROMPTS
action
N
Peorr
Peorroo
Pearrhi
Punk
Punk,lo
Punk.hi
conf
abstract
679
0.119
0.097
0.146
0.323
0.288
0.359
0.627
redact
5627
0.077
0.070
0.084
0.762
0.750
0.773
0.175
Table 7: Span-wise recovery pooled across models by action on WildChat
19

---

Published as a conference paper at ICLR 2026
action
NN
Poorr
Poorllo
Poorrhi
Punk
Punk.lo
Punk.hi
conf
abstract
1376
0.149
0.131
0.169
0.310
0.286
0.335
0.630
redact
6929
0.051
0.046
0.056
0.803
0.793
0.812
0.138
Table 8: Span-wise recovery pooled across models by action ShareGPT
action
N
Peor
Peorrlo
Poorr,hi
Punk
Punk,lo
Punk,hi
conf
abstract redact
142
0.092
0.054
0.150
0.338
0.265
0.419
0.613
6430
0.050
0.045
0.056
0.731
0.720
0.742
0.
Table Span-wise recovery pooled across models by action on CaseHOLD
action
N
Peom
Peomlo
Deurchi
Punk
Punk.la
Punk,hi
conf
abstract
935
0.056
0.043
0.072
0.030
0.021
0.043
0.964
redact
12835
0.027
0.024
0.030
0.790
0.783
0.797
0.158
Table 10: Span-wise recovery pooled across models by action on MedQA
Type
@it@ (orig)
Hit@1 (mask)
Hit@3 (orig)
Hit@3 (mask)
ADDRESS
1.000
0.000
1.000
0.000
AFFILIATION
0.684
0.017
0.725
0.020
AGE
1.000
0.000
1.000
0.000
ETHNICITY
0.000
0.000
0.000
0.000
LN
0.000
0,000
0.000
0,000
GEOLOCATION
0.759
0.140
0.788
0.157
0.500
0,000
0.500
0,000
INCOME
1.000
0.000
1.000
0.000
NAME
0,915
0.000
0,997
0.000
RACE
0.735
0.000
0.735
0.000
TIME
0.870
0.004
0.916
0.004
Table 11: Sype-wise recovery pooled by type on CaseHOLD
Type
N
HB3
ADDRESS
[67 66, 100.0%]
[0.0%, 32.4%)
[67 69,6 100.0%
[0.0%; 32.4%)
1,000000
1,000000
AFFILIATION
459
[64.0% 72.5%]
[0.9%, 3.4%]
[68.3%, 76.4%)
[1.0%) 3.7%%
0.869000
0,800000
[67.6%.
167.6%.
ETHNICITY
10.0%. 29.9%
[0.0%, 29.9%
10.0%. 29.9%
[0.0%. 29.9%
0.667000
0.067000
FINANCIAL INFORMATION
[0.0%. 29.9%]
[0.0%.
1.000000
0.000000
GEOLOCATION
344
[71.1%, 8%
[10.7%,
18.0%)
[74,2%) 82.8%
[12.2%, 19.9%)
0.830000
0.530000
HEALTH INFORMATION
[21.5%. 78.5%
[0.0%. 32.4%5]
[21.5%, 78.5%
[0.0%. 32.4%
1.0000000
1,000000
INCOME
164.6%, 100.0%
(0.0%, 35.4%
164.6%.
100.0'
[0.0%, 35.4%
1.0000000
0,257000
NAME
377
[88.3%. 93.9%
10.0%. 1.0%
198.5%.
100.0%
10.0%, 1.0%
0. 0.999000
0.601000
RACE
34
[56.9%. 85.4%)
[0.0%, 10.2%)
[56.9%,
[0.0%, 10.2%)
0.735000
0,500000
TIME
238
[82.1% 19, 90.7%]
[0.1%) 2.356
[87.4%, 94.5%
[0.1%. 2.3%] 0,954000
0.633000
Table 12: Type-wise recovery on casehold: 95% confidence intervals (H@ 1/H original and minimized) and mean top-1 confidence.
20

---

Published as a conference paper at ICLR 2026
Type
@it@1 (orig)
Hit@ (mask)
Hit@3 (orig)
Hit@3 (mask)
AFFILIATION
0.000
0.000
0.000
0.000
AGE
0.990
0.000
0.994
0.000
DIETARY PREFERENCE
0.111
0.000
0.111
0.000
GENDER
1.000
0.120
1.000
0.120
GEOLOCATION
0.704
0.000
0,704
0.000
HEALTH INALTMLNFOOMAAION
0.919
0.031
0.970
0.048
MARITAL STATUS
1.000
0.000
1.000
0.000
OCCUPATION
0.768
0.000
0.768
0.000
RACE
1.000
0.015
1.000
0.015
TEXUAL.ORIENTATION
0.375
0.000
0.375
0.000
TIME
0.541
0.000
0.824
0.000
Table 13: Type-wise recovery pooled by type on MedQA
Type
H83
AFFILIATION
10.0%. 29.9%
10.0%. 29.9%
|0.0%. 29.9%)
10.0%. 29.9%
0.000000
0.000000
AGE
864
[98.0%. 99.5%]
[0.0%, 0.4%)
[98.7% 99.8%]
[0.0%, 0.4%]
1,000000
0,117000
PREFERENCE
[2.0%%
[0.0%, 29.9%8]
23.0%.43..5%
29.9%]
GENDER
475
199.2%. 100.0%
19.49: 15.2%8
[99.2%, 100.0%]
19.4%. 15.2%
1.000000
0.839000
GEOLOCATION
10.0%.
[61.2 161.2%,
10.0%%
800
[89 8%, 93.6%
[2.1%, 4.6%
[95. 696 98.0%)
6.5%%
0,999000
0.766000
MARITAL STATUS
[70.1 16, 100.0%]
[0.0%, 29.9%]
[70. 100.0%]
[0.0%; 29.9%]
1.000000
0,222000
OCCUPATION
[65.6%, 85.2%
[0.0%, 5.3%%
[65.6%, 85.2%)
10.0%, 5.3%
0.884000
0,130000
RACE
194.7%, 100.0%
10.3%, 7.9%8
194.7%. 100.093
10.3%. 7,9%
10000000
0.015000
ORIENTTORNNN
[21.2% 57.39)
10.0%. 13.8%)
[21 24,. 57.3%)
[0.0%; 13.8%]
0.90#000
0.567000
TIME
85
[43.6%, 64.3%]
[0.0%, 4.3%
[72.9%, 89.0%]
[0.0%, 4.3%]
1.000000
0,205000
Table 14: Type- wise recovery MedQA: 95% confidence intervals (H@ 1/H@3; original and min- imized) and mean top-1 confidence.
Type
Hit@ (orig)
Hit@1 1 (mask)
Hit@3 (orig)
Hit@3 (mask)
ADDRESS
1.000
0.000
1.000
0.000
AFFILIATION
0.845
0,042
0.892
0,044
AGE
0.746
0.029
0.787
0.029
EDUCATIIOAL
0.413
0.000
0.413
0.000
EMAIL
1.000
0.000
1.000
0.000
ETHNICITY
1.000
0.000
1.000
0.000
FIVANCIAL.INFORMATIION
0.833
0.148
0.852
0.148
GENDER
1,000
0.038
1.000
0.038
GEOLOCATION
0.858
0.051
0.961
0.058
HEALTH INFORMATION
0.855
0.000
0.964
0.024
INCOME
0.729
0.000
0.729
0.000
IP P.ADDRESS
0.429
0.000
1,000
0.000
MARITAL.STATUS
0.703
0,000
0.781
0,000
NAME
0.853
0.018
0.937
0.018
OCCUPATION
0.775
0.086
0.823
0.105
RACE
1,000
0.000
1.000
0.000
RELIGION
1,000
0.000
1.000
0.000
TIME
0.861
0.046
0.918
0.052
URL
0.922
0.000
0.933
0.000
VEHICLE
1,000
0.000
1,000
0.000
Table 15: Type-wise recovery pooled by type on ShareGPT
21

---

Published as a conference paper at ICLR 2026
Type
NV
Hdd
H@1
CT
ADDRESS
[67.6%%
100.0%]
32.4%)
[67.6%, 100.0%:
[0.0%; 32.4%)
0.125000
AFFILIATION
548
[81.2%, 87.3%]
[2.8%) 6.2%)
[86.4%. 91.6%5
[3.0%% 6.49
0,934000
0.703300
AGE
5.79%
5.7%%
0,982000
EDUCATIONAL
46
[28. 3%, 75%
10.0%, 7.7%)
[28.3 55.7%]
[0.0%; 7.7%)
0,900000
0,680000
EMAII.
17
[81.6% 100.0%]
18.4%)
[81 66. 100.0%]
[0.0%, 18.4%)
1,000000
O. 0.180000
ETINICITT
31
[89.0%. 100.00]
[0.0%. 11.0%]
[89.0%, 100.0%)
[0.0%. 11.0%]
1,0000000
0.226000
TIMACIAALINFORMATIONN
54
171.3%, 91.0%)
17.7%. 26.6%
173.4%. 92.3%
17.7%, 26.6%
0.852000
0:907000
GENDER
78
[95.3%,
100.0%)
[1.3%, 10.7%)
195.3 335, 100.09
[1.3%, 10.7%)
1,000000
0,342000
BBOLOCATION
935
[83.4%. 87.9%)
[3.9%. 6.7%%
[94,7%, 97.2%%
[4.5%, 7.5%
0. 0.992000
0.674000
HEALTH INTORMATTION
[76.4%. 91.5%)
[0.0%%
[0.7%%
INCOME
48
159,0/6. 83.4%]
10.09%: 7.4%
599.0% 83.4%
10.0%% 7.4%
0.833000
0.677000
IP.ADDRESS
[15.8%
75.0%)
[64.6%
0,857000
MARITAL_STATUS
64
[58. 2%, 80.1 80.1%]
[0.0% 5.7%)
[06.6%, 86.5%]
[0.0%; 5.7%
0.969000
0.262000
NAME
621
[82.3%% 87.9%)
[1.0%, 3.1%6]
[91.5%% 95.4%)
[1.0%,
0,958000
0.597000
OCCUPATION
209
171.4%, 82.6%)
[5.5%, 13.2%
[76.6%. 86.9%
[7.1%. 15.4%)
0,911000
0.588000
RACE
13
100.0%] 177.2%.
10.0%. 22.8%
177.2%, 100.0%
10.0%% 22.8%
0.154000 1.000000
RELIGION
[64.6%,
100.0%]
[0.0%, 35.4%]
[64. 6%, 100.0%
[0.0%6, 35.4%)
1,000000
0000000
TIME
656
[83. 133.33% 88.6%]
[3.2% 6.5%)
189.4%, 93.6%)
[3.7%, 7.2%
0. 0.990000
0.695000
2%)
[0.0%.
[0.0%%
VEHICLE
161.0%. 100.0%
10.0% 39.0%
161.0% 100.0%
10.059, 39.0%8
1.000000
1000000
Table 16: Type-wise recovery ShareGPT: 95% confidence intervals (H@ 1/H@3; original and minimized) and mean top-1 confidence.
Type
@it@ (orig)
Hit@1 (mask)
Hit@3 @ (orig)
Hit@3 (mask)
AFFILIATION
0.830
0.019
0.871
0.019
AGE
0.691
0,000
0.764
0,000
EDUCATIONAL
0.667
0.000
1,000
0.000
EMAIL
0,000
0,000
0,000
0,000
ETHNICITY
0.630
0.000
1,000
0.000
FINANCIAL
0.923
0.000
0.923
0.000
GENDER
1.000
0.026
1.000
0.026
GEOLOCATION
0.898
0.022
0.954
0.031
GPA
1.000
0.000
1.000
0.000
1,000
0.000
1.000
0.000
ID.NUMBER
1.000
0.000
1,000
0.000
INCOME
0.727
0.000
0.727
0.030
KEYS
1.000
0.000
1,000
0.000
NAME
0.903
0.000
0.981
0.000
OCCUPATION
0,854
0.080
0,934
0,080
NUMBER
1.000
0.000
1.000
0.000
PRODUCT
1.000
0.000
1.000
0.000
QUANTITY
0.500
0.000
0.500
0.000
RACE
1,000
0.000
1.000
0.000
TIME
0.733
0.000
0.862
0.000
URL
0.886
0.000
0.886
0.000
USERNAME
0.533
0.000
0.533
0.000
Table 17: wise recovery pooled by type on WildChat
22

---

Published as a conference paper at ICLR 2026
Type
H103
CT
AFFILIATION
179,6%5 9%:
[1 1.00% 3,4%]
[84.0% 89.7%]
[1.0%. 1.4%]
0.671000
AGE
123
[60.5%. 76.6%)
10.0%; 3.0%
[68.2%,
83.198
10.08. 3.0%)
0,927000
0.26.3000
REUCADDONALREECOD
[46.7%.
10.0%,
186.2%, 100,0%
13.8%)
0,750000
EMAIL
18
[0.0%. 6%)
[0.0%% 17.6%)
10.0% 17.098
10.0%%
17.6%)
0,200000
0.111000
ETHNICITY
[44.25 25. 78.5%]
[0.0%% 12.598]
[87,5%, 100.0%
[0.0%. 12.5%)
1,000000
0.289000
39
779.7%, 97.
10.0%; 9.0%%
[79.7%. 97.3%8
10.015. 9.0%)
1,0000000
0,949000
GENDER
190.898 100.0%
[0,5% 13.55%
190.8% 100.0%
10.5%, 13.5%
1,0000000
0.337000
GEOLOCATION
677
[87 3%6, 9%)]
[1.3%
[93.6%% 96.8%]
12.0%%.4.75
0,972000
0.577000
GPA
8
[67.6%% 100.0%)
[0.0%: 32.4%)
[67 6%, 100.0%]
[0.0%6. 32.4%)
1,0000000
0.250000
INFORMATION
[91.08, 100.0%;]
[0.0%;
100.0%
10.0%: 9.0%%
ID_NUMBER
9
170.19%. 100.09%
[0.0%* 29.9%
170.1%. 100.0%
[0.0%, 9
1,0000000
0.11
INCOME
155.8%
[0.0%.
[55.
0,758000
0.515000
KEYS
9
[70. 1%, 100.0%)
[0.0%, 29 9%)
[70. 1%, 100.0%
[0.0% 29 9%6
1,000000
0,444000
NAME
621
[87.8% 92.4%)
[0.0%; 0.6%6]
[96.7%, 98.9%]
[0.0%; 0.6%)
O. 0.986000
0.558000
OCCUPATION
137
178.5% 90.3%.
[4.5%. 13.8%
188.08, 96.5%
[4.5%, 13.8%]
1,0000000
0.531000
PHONE NUMBER
9
170. 1%. 100.0%]
[0.0%. 29.95
170.1%; 100.0%
10.055 29.9%)
1,000000
1,000000
PRODUCT
8
[67 6%, 100.0%)
[0.0%% 32.4%)
6%, 100.0%
[0.0%6 32.4%)
1,000000
0,750000
QUANTITY
18
099.0%. 71 71.0%]
[0.0%; 17.6%)
[29.00 71.0%]
[0.056, 17.6%)
1.000000
1.000000
[67.6%.
[67.6%,
[0.06, 32.4%:
TIME
536
169 4%.. 76. 96.9%)
[0.06e, 0,7%
183.09, 88.9%
[0.0% 0.7%%
U. 0,998000
0.566000
URL
176.0%. 95.09
8.0%
[76.0%, 95 95.0%)
8.0%)
0.886000
0,443000
USERNAME
15
[30.1 16, 75.2%)
[0.0%, 20.4%)
[30. 1%, 75.2%)
[0.0%6 20.4%)
0,533000
0.533000
Table 18: Type-wise recovery on WildChat: 95% confidence intervals (H@ I/H@3: 3; original and minimized) and mean top-1 confidence.
1.2 GPT-5 AS ATTACKER ON ITS OWN DRACLE-MINIMIZED PROMPTS
action
N
Peorr
Peorrlo
Poorhii
Punk
Punkl
Punk,i
conf
abstract
679
0.119
0.097
0.146
0.323
0.288
0.359
0.627
redact
5627
0.077
0.070
0.084
0.762
0.750
0.773
0. 175
Table 19: Span-wise recovery with GPT-5 action on WildChat
attacker on its own oracle-minimized prompts by
action
N
Peurr
Poonlo
Pcom,hi
Punk
Punk.lo
Punk.hi
conf
abstract
118
0.127
0,068
0.195
0.280
0.203
0.364
0.719
redact
987
0.020
0.012
0.029
0.967
0.954
0.978
0.030
Span-wise recovery with GPT-5 action on ShareGPT
Table 20:
attacker its oracle-minimized prompts by
action
N
Peorr
Poomlo
Pcorchi
Punk
Punk.lo
Punk.hi
conf
abstract
4
0.250
0,000
0.750
0.500
0,000
1.000
0.487
redact
397
0.055
0.035
0.081
0.922
0.894
0.947
0.069
Table 21: Span-wise recovery with GPT-5 action on CaseHOLD
attacker on its oracle-minimized prompts by
action
N
Peor
Peorllo
Pcomhi
Punk
Punk,lo
Punk.hi
conf
abstract
21
0.000
0.000
0.000
0.000
0.000
0.000
1.000
redact
941
0.003
0.000
0.006
0.995
0.990
0.999
0.004
Table 22:
Span-wise recovery with GPT-5 action on MedQA
attacker its
prompts by
23

---

Published as a conference paper at ICLR 2026
Type
@it@1 (orig)
Hit@1 (mask)
Hit@3 (orig)
Hit@3 (mask)
AFFILIATION
74.6%
0.0%
85.7%
0.0%
AGE
94.1% %
0.0%
94.1% 1%
0.0%
EDUCATIONAL.RECORD
66.7%
0.0%
66.7%
0.0%
EMAIL
0.0%
0.0%
0.0%
0.0%
ETHNICITY
50.0%
0.0%
75.0%
0.0%
FINANCIA.INFFRMAATONN
25.0%
0.0%
50.0%
0.0%
GENDER
100.0%
0.0%
100.0%
0.0%
GEOLOCATION
81.5%
0.0%
93.8%
0.0%
GPA
0.0%
0.0%
100.0%
0.0%
WEALTHLINFORMATIOO
20.0%
0.0%
40.0%
0.0%
ID_NUMBER
100.0%
0.0%
100.0%
0.0%
INCOME
50.0%
0.0%
50.0%
0.0%
KEYS
0.0%
0.0%
0.0%
0.0%
NAME
92.1%
0.0%
97.4%
0.0%
OCCUPATION
88.2%
0.0%
88.2%
0.0%
PHONE_NUMBER
100.0%
0.0%
100.0%
0.0%
PRODUCT
100.0%
0.0%
100.0%
0.0%
QUANTITY
0.0%
0.0%
50.0%
0.0%
RACE
100.0%
0.0%
100.0%
0.0%
TIME
73.0%
0.0%
82.5%
0.0%
URL
60.0%
0.0%
60.0%
0.0%
USERNAME
50.0%
0.0%
50.0%
0.0%
Table 23: Type-wise recovery with GPT-5 as attacker on its own oracle-minimized prompts WildChat.
Type
@it@1 (orig)
Hit@1 (mask)
Hit@3 (orig)
Hit@3 (mask)
ADDRESS
100.0%
0.0%
100.0%
0.0%
AFFILIATION
74.2%
1.5%
81.8%
1.5%
AGE
58.3%
0.0%
63.9%
0.0%
EDUCATIONAL.REEOOR
33.3%
0.0%
33.3%
0.0%
EMAIL
100.0%
0.0%
100.0%
0.0%
ETHNICITY
75.0%
0.0%
75.0%
0.0%
FINANCIAL INFORMATION
42.9%
0.0%
57.1%
0.0%
GENDER
90.9%
0.0%
100.0%
0.0%
GEOLOCATION
82.2%
1.7%
93.2%
1.7%
HEALTH SAALTHIIFOOMMAIION
50.0%
0.0%
60.0%
0.0%
INCOME
33.3%
0.0%
50.0%
0.0%
IPAADDRESS
100.0%
0.0%
100.0%
0.0%
MARITAL.STATUS
18.8%
0.0%
25.0%
0.0%
NAME
91.0%
1.3%
94.9%
1.3%
OCCUPATION
59.3%
3.7%
70.4%
3.7%
RACE
100.0%
0.0%
100.0%
0.0%
RELIGION
100.0%
0.0%
100.0%
0.0%
TIME
71.4%
2.4%
83.3%
2.4%
URL
90.9% 100.0%
0.0%
90.9%
0.0%
VEHICLE
0.0%
100.0%
0.0%
Table 24: Type-wise recovery with GPT-5 as attacker on its own oracle-minimized prompts on ShareGPT.
24

---

Published as a conference paper at ICLR 2026
Type
@it@l (orig)
Hit@1 (mask)
Hit@3 (orig)
Hit@3 (mask)
ADDRESS
100.0%
0.0%
100.0%
0.0%
AFFILIATION
74.5%
1.8%
81.8%
1.8%
AGE
0.0%
0.0%
0.0%
0.0%
ETHNICITY
0.0%
0.0%
0.0%
0.0%
100.0%
0.0%
100.0%
0.0%
GEOLOCATION
83.3%
0.0%
95.2%
2.4%
HEALTH INFORMATION
0.0%
0.0%
0.0%
0.0%
INCOME
0.0%
0.0%
100.0%
0.0%
NAME
84.4%
0.0%
88.9%
0.0%
RACE
50.0%
0.0%
50.0%
0.0%
TIME
82.8%
0.0%
93.1%
0.0%
Table 25: Type-wise recovery with GPT-5 as attacker on its oracle-minimized prompts CaseHOLD.
Type
Hit@ (orig)
Hit@1 (mask)
Hit@3 (orig)
Hit@3 (mask)
AFFILIATION
0.0%
0.0%
0.0%
0.0%
AGE
99.0%
0.0%
99.0%
0.0%
100.0%
0.0%
100.0%
0.0%
GENDER
100.0%
0,0%
100.0%
0,0%
GEOLOCATION
46.2%
0.0%
53.8%
0.0%
HEALTH INFORMATION
HEALLHHINFORMATIION
83.7%
0.0%
92.4%
0.0%
MARITAL
100.0%
0.0%
100.0%
0.0%
OCCUPATION
50.0%
0.0%
50.0%
0.0%
RACE
100.0%
0.0%
100.0%
0.0%
SEXUALOOIENTATTOON
100.0%
0.0%
100.0%
0.0%
TIME
50.0%
0.0%
80.0%
0.0%
Table 26: Type-wise recovery with GPT-5 as attacker on its own oracle-minimized prompts on MedQA.
25


Reviews:
[{"review_id": "VppmywKtWX", "reviewer": "Reviewer_ofax", "summary": "This paper introduces a framework to define and operationalize the principle of data minimization (PII redaction) for LLM prompting. The authors formulate data minimization as an optimization problem: finding the most privacy-preserving version of a user prompt (by applying a series of actions to sensitive spans) that still maintains a minimum level of task utility for a given response LLM.", "questions": "1. What PII is commonly present in MedQA, a curated medical expert QA dataset? How representative is this of PII that users would actually share, as opposed to PII that is simply embedded in the original exam questions?\n2. Could using test-time compute (Gemini Pro Thinking or Claude Extended Thinking) help close the gap in PII removal between single-pass and expensive search-based methods?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "The paper is well-written, with motivated and formalized problem settings. The introduction and related work sections do a good job of comparing the data minimization formulation within existing privacy-preserving techniques like DP-training or simple sanitization.", "weaknesses": "1. The most significant weakness is the lack of comparison against simple, non-LLM baselines. The paper compares its computationally expensive oracle to single-pass LLM predictors. However, it fails to include a much simpler and more practical baseline: using a standard PII identification or NER tool to identify and redact PIIs. \n2.  The methodology relies on an LLM-as-a-Judge (GPT-4o) to evaluate utility for open-ended tasks. While the authors commendably validate their privacy comparator LLM against human consensus, the utility judge is not validated at all. The authors provide no data on the judge's accuracy, its agreement with human assessments of utility, or the robustness of its rubric.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "BEUdXvHJfM", "reviewer": "Reviewer_nGjY", "summary": "The work addresses the problem of \"oversharing\" of personal data in LLM prompts. The authors study the data minimization problem as a constrained optimization problem where they want to maximize privacy with a strict constraint on utility degradation. The work aims to find minimal prompts for each model that does not degrade utility. The paper proposes a \"Freeze-then-Search\" algorithm to find this optimal prompt. The search space consists of span-level transformations: {RETAIN, ABSTRACT, REDACT} where RETAIN is the least private and REDACT is the most private. To obtain an full ordering on the space, the authors utility a privacy comparator model that is a fine-tuned Qwen2.5-7B-Instruct model.  The search is conduced using a priority-queue tree search and terminates once a prompt is found that satisfies the utility constraint. To measure that utility constraint, the authors use a judge model (GPT-4o) to measure whether the answer from the minimized prompt is compared to a reference response.\n\nThe work finds these minimized prompts for 9 LLMs (from small models like Qwen-0.5B to large ones like GPT-5) on 4 datasets. The authors find that large models are tolerant of far more minimization than small models. \n\nThe work also tries to understand if LLMs are good predictors of these minimized prompts and find that all LLMs are poor predictors of this. They also find that the predictors exhibit a strong bias towards ABSTRACT rather than using REDACT as in the minimized prompts.", "questions": "- Is it the case that large models require less information is probably because they are able to infer the missing pieces of information better? \n- The \"abstract\" bias seems interesting. Could it be related to how the model is instructed to generate its minimal prompt?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "- Paper does a good job of formulating the data minimization problem as a constrained optimization problem including defining a well ordered search space.\n- The generated minimized prompts seem useful to make progress on data minimization predictors.\n- The insight that models are poor predictors of predicting the minimal prompt required is interesting.", "weaknesses": "- The minimized prompts are model specific.\n-", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "9pa8iu5P0z", "reviewer": "Reviewer_Djhr", "summary": "The paper proposes a framework for reducing privacy leakage in LLM prompting. It first identifies the sensitive spans in the prompt and introduces a priority-queue tree search algorithm that explores combinations of RETAIN / ABSTRACT / REDACT operations on the spans.", "questions": "1. Could you please present how sensitive are the utility threshold gamma? From my point of view, a little gamma may lead to significant performance degradation in LLMs.\n2. I am curious about  whether the “utility predictor” can be generalized across tasks or must be retrained for each task?\n\n3.During the search process, are all intermediate prompts exposed to the main LLM? If so, how is privacy leakage mitigated? If using different LLMs, assuming a smaller LlM local deployed, can the obtained privacy-preserved still transfere effectively to the larger main LLM?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1.\tThe paper addresses a important problem as how to minimize the privacy leakage of LLM prompt.\nThe paper addresses an important and timely problem — practical data minimization for LLM usage — with clear motivation.\n2.\tThe paper includes comprehensive experimental evaluation with  multiple datasets, models, and attack evaluations.\n3.\tAlthough without strict privacy leakage guarantee, the proposed ranking-based evaluation method offers an insightful and practical perspective for quantifying privacy leakage.", "weaknesses": "1. The evaluation of privacy relies on pairwise comparison, lacking of theoretical guarantees.\n2. The paper evaluates utility by comparing the generated output with the target response, treating it as a binary classification problem. This oversimplification fails to reflect the continuous performance characteristics of modern LLMs.\n3. The practicability of the proposed method remains questionable, as it assumes there is a local deployed LLM while the aim of the paper is to protect privacy of  prompt  when using LLM via API.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "dTUDjDNt5p", "reviewer": "Reviewer_NzwN", "summary": "This paper presents a framework to formally define and operationalize data minimization, in the context of users sharing personal information to LLMs. For a given user prompt and response model, they quantify the least privacy-revealing disclosure that maintains utility, and propose a priority-queue tree search to locate this. They evaluated this on a diverse set of four datasets and nine LLMs, and show that for the same user prompts, larger frontier LLMs can have stronger data minimization while maintaining task quality. They also show that LLMs are poor predictors of data minimization and have a bias towards oversharing.", "questions": "- How could the results be applied to real applications?\n- Could more details about the PII annotation be provided?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "- This paper formally operationalizes data minimization in the context of privacy-preserving prompts for LLMs and presents an algorithm for doing this, which is different from existing approaches that mainly focus on detecting personal information and redacting or abstracting it.\n- The authors ran a systematic evaluation on a good range of datasets and LLMs and were able to show how more powerful frontier models can be better for data minimization.", "weaknesses": "- It is not clear how to directly apply these findings to real applications, since the data minimization algorithm involves querying and checking the utility for multiple variants of the original prompt that will reveal the personal information, and might also take significant cost and latency.\n- It would be good to include more details about the PII annotation, such as how the annotators were chosen, and more information about the amount of consensus. For instance, in table 1, how many examples are there with human consensus at least 0.8?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)