Hot PATE: Private Aggregation of Distributions  for Diverse Tasks
Authors: Edith Cohen
Benjamin Cohen-Wang
Xin Lyu
Jelani Nelson
Tamas Sarlos
Uri Stemmer
Venue: ICLR 2026 Poster (2026)
Keywords: Differential Privacy
Sequential Text Generation
Coordinated Ensembles
Primary area: alignment, fairness, safety, privacy, and societal considerations
TL;DR: Hot PATE is a PATE variant for generative tasks that preserves output diversity, provably transfers it without extra privacy cost, and yields orders-of-magnitude higher utility at the same privacy level.

Abstract: The Private Aggregation of Teacher Ensembles (PATE) framework enables privacy-preserving machine learning by aggregating responses from disjoint subsets of sensitive data. Adaptations of PATE to tasks with inherent output diversity such as text generation, where the desired output is a sample from a distribution, face a core tension: 
as diversity increases, samples from different teachers are less likely to agree, but lower agreement results in reduced utility for the same privacy requirements.  Yet suppressing diversity to artificially increase agreement is undesirable, as it distorts the output of the underlying model, and thus reduces output quality.
 
We propose Hot PATE, a variant of PATE designed for diverse generative settings. 
We formalize the notion of a *diversity-preserving*  *ensemble sampler* and introduce an efficient sampler that provably transfers diversity without incurring additional privacy cost.
Hot PATE requires only API access to proprietary models and can be used as a drop-in replacement for existing *Cold* PATE samplers. 
Our empirical evaluations corroborate and quantify the benefits, showing significant improvements in the privacy–utility trade-off on evaluated in-context learning tasks, both in preserving diversity and in returning relevant responses.

Published as a conference paper at ICLR 2026
HOT PATE: PRIVATE AGGREGATION OF DISTRIBU- TIONS FOR DIVERSE TASKS
Edith Cohen
Benjamin Cohen- Cohen-Wang Anthropic bencwêm t edu
Google Research and Tel Aviv University edi t cohenwang com
Xin Lyu
Jelani Nelson
Tamás Sarlós Google Research st stamsgooglee com
UC Berkeley
UC Berkeley and Google Research minileke al um mit edu
xinlya@berkeley.co edu
Uri Stemmer
Tel Aviv University and Google Research u@uri co.co...
ABSTRACT
The Private Aggregation of Teacher Ensembles (PATE) framework enables privacy- preserving machine learning by aggregating responses from disjoint subsets of sensitive data. Adaptations of PATE tasks with inherent output diversity such text generation, where the desired output is sample from distribution, face core tension: as diversity increases, samples from different teachers are less likely to agree, but lower agreement results in reduced utility for the same privacy require- ments. Yet suppressing diversity to artificially increase agreement undesirable, it distorts the output of the underlying model, and thus reduces output quality.
We propose Hot PATE, variant of PATE designed for diverse generative settings. We formalize the notion of a diversity-presevviiii ensemble sampler and introduce an efficient sampler that provably transfers diversity without incurring additional privacy cost. Hot PATE requires only API access to proprietary models and can be used as a drop-in replacement for existing Cold PATE samplers. Our empirical evaluations corroborate and quantify the benefits, showing significant improvements in the privacy-utility trade-off on evaluated in-context learning tasks, both in preserving diversity and in returning relevant responses.
INTRODUCTION
Generative models, and in particular large language models (LLMss, can perform variety of tasks without explicit supervision (Radford et al., 2019: Brown et al., 2020). Unlike conventional machine learning models, generative models support open-ended tasks with inherently diverse outputs. where many different outputs may be appropriate. This diversity, which is often essential for functionality, tunable via temperature parameter, with higher temperatures yielding greater variation in outputs.
When training or performing analytics on sensitive data such as medical records, incident reports, or emails, privacy of individual data records must be protected. Mathematical frameworks for privacy guarantees include Differential privacy (DP) (Dwork et al.) (006). considered gold standard, which requires that the probability of each output can only change little when a single record is swapped, and k-anonymity and its extensions, which require that each released record be indistinguishable from at least & - others Sweeney (2002). In practice, many large-scale analyses (e.g., Anthropic's Clio and OpenAl's usage reports Tamkin et al. (2024); OpenAl (2025b) adopt lighter privacy notions based on conimum-supppon thresholds or suppression of low-frequency categories before releasing aggregates. Ultimately, these approaches all rely high agreement, ensuring that reported outputs are supported by many data records.
Work done while the author was student at MIT

---

Published as a conference paper at ICLR 2026
A popular paradigm for privacy protection is the Private Aggregation of Teacher Ensembles (PATE)
paradigm dPaperno et al.) 2017 Bassily et al., 2018 Papernot et al.] (018),, based on Nissim et al. (2007), and described as Framework 1.1 PATE partitions sensitive data among several teachers (each of which does not preserve privacy) and aggregates their predictions to obtain
output. In the PATE framework, each data record affects most teacher and thus affects at
Framework 1.1: Cold PATE
1. Partition the dataset D into n disjoint parts: U... - For each i E [n]] train a teacher model A; on D;.
2. For each example
X:
For each teacher E [n], compute label prediction: y/ := (, EV.
Construct the histogram of votes: for E V, C = Σ Ei 1 J}.
Apply a privacy preserving aggregation mechanism to to produce a final label y E V. Abort if no confident agreement. Outpul
most one vote. A Noi NoisyArgMax DP aggregation mechanism masks these small differences by adding noise each count ej in the histogram obtain (čy)jev and returning the index arg max, čj. Implementations vary in the noise distribution and privacy analyses (see discussion in Appendix E). but ultimately, label j can returned only when the noise scale a is small relative its count cj- A light and interpretable privacy notion for histogram aggregation is threshold privacy, parametrized by T E [n]]: With threshold privacy T, the aggregator is permitted to output only labels with ej 2 T: if max; Cj < T, it must abstain (yielding
utility). Higher I means more privacy (output must be supported by more teachers) but reduced utility. A threshold of T 6(== log(1/6)) is good proxy for (c, 6DD (for our purposes, see Appendix
1.1 PATE IN THE DIVERSE SETTING
In diverse settings, such as those involving generative models, the underlying model produces a probability distribution over the vocabulary V of tokens and returns sample from the distribution. Such distributions are typically diverse, supporting open-ended responses with many plausible outcomes. In the corresponding PATE setup, each teacher i E [n] in the ensemble produces its own probability distribution (0), We formalize the aggregation step through an ensemble sampler: mechanism that maps the set of teacher distributions (p (e())) to aggregate distribution
M(( )ee()) from which the output token is sampled.
Utility of ensemble samplers As with basic PATE, henceforth Cold PATE, the design goal of an ensemble sampler is to achieve a favorable privacy-utility trade-off. We take basic utility to be the yield: returning any relevant token (e.g., whose average teacher probability exceeds threshold or is an approximate maximizer). We further propose preserving diversity as a utility criterion: Informally (formalized in the sequel), the aggregate should allocate proportional probability values to all tokens for which there is sufficient teachers support. Diversity preservation is essential in generative settings: unlike classification, where there a single ground-truth label and knowledge transfer proceeds through labeling of non-sensitive data, the entire response distribution constitutes Eiversit--rreseviing preserving sampler enables a lossless flow of that
the knowledge to be transferred. A diversity
knowledge to the student.
Cold PATE diverse settings. When cold PATE applied diverse setting, the ensemble sampler first samples histogram Hind as follows and then aggregates it y.
c Hinel (pine((p') )EEm)
(9= Σ 1(u [(== [E[D]
=) JEV
where yi (()) independently.
The histogram sampling step induces an inherent privacy-utility trade-off: as output diversity increases, basic utility (yield) drops sharply due vote-splitting. For example, if there r equally good responses, then E[c,] s n/r for each such j. so utility under threshold- requires 7' x n/r, i.e., inversely proportional to diversity. Moreover, the subsequent No s Ma
2

---

Published as a conference paper at ICLR 2026
aggregation is not ilvvesstty-preeeviing Cold PATE histogram counts concentrate (c.g.. by Chernoff) around their expectations [[[c]] = n py (where Py the average teacher probability). the noisy maximizer is disproportionately more likely be approximate maximizer of ej than token whose average probability is, say, half as large.
All prior work
of applying PATE in diverse generative settings (Tian al. 2022
Duan et al. 2023; Wu et al. 2023) either relied on the Cold PATE ensemble sampler or employed custom samplers that explicitly reduced constrained diversity (see Section A for details). Notably, these works focused primarily on basic utility (yield) and did not evaluate diversity preservation or recognize its importance in generative tasks.
In this work, we ask: Is the diversity-privacy trade-off observed in Cold PATE inherent? If not, can design an ensemble sampler that achieves high basic utility fixed privacy budget, even under substantial diversity, and (ii) preserves (transfers) diversity across teacher-suppore responses?
1.2 PATE FRAMEWORK FOR SEQUENTIAL TEXT GENERATION
A motivating application for our study, also the setting of experiments, is the generation of representative set of synthetic, privacy-preserving records from sensitive data. Such records often contain identifying elements alongside elements shared across many samples. A privacy-preservie generator must suppress the identifying elements while retaining the shared structure and variability of the data. Crucially, must also preserve diversity: without sufficient diversity, the synthetic underrepresents rare but valid patterns and fails to reflect the richness of the underlying distribution. The resulting synthetic records can support multiple downstream uses, including training (possibly non-generative) student model, fine-tuning generative model, or constructing prompts.
This motivates Framework 1.2
PATE design tailored sequential text generation, suitable for tasks such as synthetic record generation, summarization, and querying. An autoregressive model is a map A: V* p that takes sequence of tokens and outputs next-token distribution over V. The framework is parametrized by a model generator G + AA and an ensemble sampler M. For cach data partition Di, instantiate teacher model A; ((:))- Generation then proceeds in lockstep: at each step, each teacher produces its next-token distribution, and the next response token is sampled from M ((p(() )eeee))
Framework 1.2: PATE for sequential text generation
Stachee
Scripe
Algorithm PATE for Sequential Text Generation
Parameters: Vocabolary V; Instraction C V"; ensemble sampler (p (e(n)) pover V (>);
autoregressive model generator G
Input: Dotaset D
Output: Response string E V*
For E do
Randomly partition D into disjoint subscts D; ((D.)
// Construct tescher model from
R
// initialize ampty response string
repeat
for i E [n] do pl A:(C -C. R) (( (f) )ee(-)
11 Collect teachera' distributions over V // Aggregate and sample token
if <fail> then use fallback to obtain R - R-V
11 B.g., 2. sampla from public model /7 Append sampled token to response
termination me! return R
3

---

Published as a conference paper at ICLR 2026
The model generator abstraction captures two natural ways of instantiating teachers: in-context
learning and fine-tuning, With in-context learning, teacher A, is specified by a context C constructed from data part D; for few shots learning (Liu et al. 2021; Zhou et al. 2022 Garg et al. 2023). A key advantage of in-context learning is that each teacher is simply prompt provided to a shared model, requiring additional training significant storage. Scaling the number of teachers is inexpensive: prompts are cheap, and the current OpenAI API supports 10t context+output tokens for roughly US$I (OpenAl 2025a). Thus, the primary bottleneck is the amount of available sensitive data, and larger ensembles are especially attractive since under DP composition, the number of queries allowable for fixed privacy budget grows quadratically with the number of teachers. With fine- tuning, each teacher A, is model that is fine-tuned on D;. Parameter-efficient fine-tuning techniques (e.g., LoRA (Hu et al. 2022) and managed services for fine-tuning proprietary models (OpenAT 2023 Microsoft Azure 2024 Anthropic (2024) make this approach practical. Applying a PATE
wrapper on top of such fine-tuned teachers is an appealing way to obtain privacy protection.
1.3 OVERVIEW OF CONTRIBUTIONS AND ROADMAP
Our primary contribution is Ho PATE: ensemble samplers for PATE in the diverse setting that deliver high utility, both in terms of yield and in terms of diversity preservation. Hot PATE matches or exceeds the performance of the Cold PATE baseline on all inputs, with the advantage growing as output diversity increases.
Illustration of two sets of probability distributions (each shown as rectangle where the red segment indicates the probability of token j). In the left set, many teachers assign low probability q to token j: the right, few teachers assign high probability q. The average probability of token j is the same in both cases, but the underlying support differs.
Figure 1:
X0 Ago
We begin with key observation. As noted above, Cold PATE histogram counts concentrate around the scaled average probabilities. Consequently, a sampler with threshold privacy T has yield only if some token's average teacher probability exceeds T/n. However (see Figure [ the average distribution (and thus the Cold PATE histogram) collapses critical distinction: (i) high teacher support with low per-teacher probability q (transferable under privacy even when q < T /n), versus (ii) low teacher support with high (not transferable under privacy). Because this distinction is lost under averaging, any ensemble sampler that merely post-prucesses the average distribution (or a Cold PATE histogram) inherits Cold PATE's 's unfavorable diversity -pivvacy trade-off.
In Section 2 we formalize a parameterized notion of diversity preservation that captures this distinc- tion. Informally, for robustness parameter 7 E [ni]. we require:
Transfer. If a token j has per-teacher probability at least q > 0 across 2 7 teachers, then it is transferred: the aggregate assigns it probability at least c(gc/n).
Relevance. Irrelevant tokens are not amplified: for every token j. its probability in the aggregate is not much larger than its average probability across teachers.
The
utility of an ensemble sampler is captured by the smallest 7 for which the aggregate distribution is guaranteed to satisfy the two requirements above for any set of teacher distributions. For Cold PATE, no meaningful guarantee is possible. For example, if all teacher distributions are identical and uniform over support of size >> n², the probability that any histogram count exceeds is negligible, and therefore utility is possible only for threshold privacy T = (i.e., no privacy). With T = the mechanism yields no utility, and in particular fails to preserve diversity even for T = n. Our hot ensemble samplers provide the following, asymptotically tight guarantees:
Theorem 1 (Hot ensemble samplers; Informal, see Theorem 2 Corollary Corollary 3) There exist histogram- based ensemble samplers Marr and Map such that:
1 'The term 'hot' alludes to the temperature parameter that tunes diversity in L.Mss. "Tightness holds, e.g., when teachers form groups of size with identical distributions within each group and disjoint support across groups.
4

---

Published as a conference paper at ICLR 2026
(Threshold privacy) For any threshold T E [n]. Maz simultaneously satisfies T --threshold- privacy and is T-diversity- r-diversity-prseevinng with T = O(T).
(Differential privacy) For any (e, 6), Man simultaneously satisfies (e,6)-DP and is diversity-preservin with T = 1og
In Section 4 we evaluate Hot PATE on two in-context learning tasks: (i) a natural task of synthetic record generation and (ii) curated, tunable-diversity task constructed to avoid training contamination. The results demonstrate the properties and advantages of our design and corroborate the theory, showing orders-of of-maniitude improvements over the Cold PATE baseline in the privacy cost required to achieve a given level of utility (including both diversity preservation and basic utility).
In the remaining part of the introduction preview the key ideas and design of our hot ensemble samplers. Notably, our samplers are also histogram- based: they have the form M := A o Heoo- they first construct a histogram с ~ Heool (po()) )<emi)) over V with one vote per teacher, and then apply a Section 3.2,. each teacher's vote is computed without reference to other teachers' ' distributions. Hence the histogram has low sensitivity: changing one teacher's s data affects at most that single vote. aggregation mechanism A for histograms, including the DP aggregations used in Cold PATE (Papernot et al. 2017, 2018), applies unchanged to e ~ Hooo. and the sampler M inherits the privacy properties of A. further benefit of a histogram-based method is interpretability of the privacy exposure; in particular, the number of teachers supporting an output token simply count in the histogram and we can leverage threshold privacy.
aggregation A c - V U [<fail>>] to this histogram. Crucially (sce
As consequence, any
We first preview the histogram distribution Heodl (p(9)) )e(()) component of our samplers. The key reason we obtain a much better privacy-utility trade-off than with Cold PATE's Hind is the shape of the histograms: they can vary widely across samples (unlike Cold PATE's concentrated histograms) and are peaky, placing high mass on a few tokens and inducing larger margins (see Figure 13 The mechanism we use to generate these histograms, ensemble coordination, is introduced in Section 3 The idea is simple: the ensemble draws shared randomness, and each teacher i emits a token y/: as function of its distribution (t²) and the shared randomness. This makes votes positively correlated while preserving low sensitivity. Crucially, the marginal of each vote follows the teacher's distribution
Prlys = 5] = P3" exactly as in the independent ensemble (T). The coordination only affects
joint behavior: if two teachers have aatal-variation distance TV (V ,(()), then they produce the same token with probability (1 - TV (p) p(1 + (V p())) in particular, identical distributions yield identical tokens. More generally, if token has probability q across a support of T teachers, coordination creates bursts of agreement: the histogram count Cj is ((T)) with probability l(q). This burstiness is precisely what enables diversity transfer with high privacy guarantees: tokens supported by many teachers, even with small per-teacher probability, surface as high peaks with large margins in the aggregate histogram.
The second component of our ensemble samplers is the aggregation mechanism that is applied to the histogram (see Section 3.3). We consider two regimes: (i) Homogeneous ensembles: Data are randomly partitioned so that each teacher is representative (each possesses the core knowledge to be transferred). In this setting it suffices to require diversity preservation scale 7 Q(n). (ii) Heterogeneous ensembles: Teachers may correspond to single users or narrow subpopulations. so we must allow for lower agreement and T. weighted sampling aggregation from the above-T counts (under threshold privacy) or noisy counts (under DP) works for all additionally, threshold arg max and respectively No A ra suffice in the homogeneous case, which notably, matches the regime and mechanism used in Cold PATE.
In Sections and G we discuss data-dependent DP privacy analysis methods that can increase the number of queries processed for given privacy budget by orders of magnitude over naive analysis. We benefit from a high margin - separation of the maximizer, which is more likely with coordinated ensembles, and make steps with no yield "free." With heterogeneous ensembles, teachers can be individually charged (instead of the whole ensemble) when they contribute to the final token (Kaplan et al. 2021 Cohen and Lyu 2023). Related work is surveyed in Appendix A
5

---

Published as a conference paper at ICLR 2026
2 DIVERSITY-PRESERVINN AGGREGATION
We formalize a parametrized definition of diversity preservation property of ensemble samplers: p from probability distributions over V to a probability distribution over V U {<fail>} is diversity-preserving with IN, 3 E (0, 11, 2 1 if for any input (p()))E(()) (e((n) and j V
Definition DDiversitypreesevaatoo), A map
(transfer)
For all g E [0, 1]. (Gia
Σ iE|m
(0) 1e 2 2 2T Pj
2 3
Ci q.
n
2. (relevance)
S
(m(m)
The first property is that probability q across enough (r) teachers, no matter how small is q. is transferred to the aggregate distribution. The second ensures that we do not output irrelevant tokens.
Requirements are stricter (and can be harder to satisfy) when 3 and y are closer to and when is smaller. A setting of r 1 and 3 = y 1 allows only for the average distribution to be the aggregate. A larger T increases robustness in that more teachers must support the transfer.
Remark (Failures). When 1, it necessary to include a failure/abstention outcome <fail> in the support of the aggregate distribution. For example, if the prompt requests patient ID (and we assume no generalization) then teacher distributions have disjoint supports; no token attains support 2 so no valid tokn can be returned. Pructical remedies include: (i) retrying the step with different shared randomness; (ii) falling back to a non-private default prompt/model for this step; or (iii) redesigning the prompt instruction elicit non-identifying, higher-agreement responses.
ENSEMBLE COORDINATION
coordinated ensemble, similarly independent ensemble Equation ① defines probability distribution Hcoo ((p()) (ee(m)) over histograms over V with total count Ejev Cy The sampling of histogram = (cg) JEv v is described in Algorithm 2 The algorithm samples shared randomness = (uy)eev. Each teacher E [n]] then contributes a single token yı e V that is function of its distribution (1) and The frequencies C, are computed as in
The sampling method in ensemble coordination a classic technique called coordinated sampling. was first introduced in statistics works in order to obtain samples that are stable under distribution shifts (Kish and Scoti 1971 Brewer et al. 1972 [Saavedra] 1995 Rosén 1997 Ohisson 2000) and
in computer science works for computational efficiency via sampling- based sketches and a form of 1997; Broder, 2000; Indyk and Motwani 1998] Haas 201T). Its recent applications include private learning (Ghazi et al. (2021) and speculative decoding (Leviathan et al. 2023). The Gumble- Max-Trick (Gumbel 1954 Yellott 1987). when used
Locality Sensitive Hashing (LSH) (Cohen) 1994
with the same seed across teachers, produces coordinated samples from logits.
Implementation Coordinatedit is stogram is simple to implement with access to the model. With proprietary models, an enhanced API can either (i) provide the shared randomness p to the model to facilitate token selection or (ii) give the full distribution to the user. Without API enhancements, the distribution can be approximated by repeated sampling with the same prompt. This impacts computation, the number of samples needed increases with diversity, but does not impact privacy.
3.1 PROPERTIES OF COORDINATED HISTOGRAMS
Let )e(m) be probability distributions over V and let Yono and Yind be the respective distributions of votes (yt)se(n) generated by coordinated or independent ensemble with teacher distributions (p())) Let Mcon and Hind be the respective distributions of histograms.
For each token j. its expected frequency, over the sampling of histograms, is the same for coordinated and independent ensembles:

---

Published as a conference paper at ICLR 2026
Algorithm 2: CoordinatedHistogr:a
Input:
Teacher distributions (p [eim)
foreach token j do sample i..d. uj Exp[1] foreach teacher do
1/ Sample shared randomness p (uylscV Compute coordinated samples es (Whee))
69
y8 - arg max,
// hot comtomkk samp inn transform
foreach token j E V do - [n (14
// Compute frequencias
return (c))sv. (uz)
// 81 Ristogram of frequencie
Claim (Expected token frequency).
Vj E V, EewHonn = EorHind (ask = Σ Σng
(2)
Proof. The marginal distribution of y/: for teacher i is (() with both independent and coordinated ensembles and thus the claim follows from linearity of expectation.
In a coordinated ensemble, votes of different teachers are much more likely agree than in an independent ensemble (see Appendix for proof):
Claim 2 (Agreement probability). For teachers i, E [n] and token j E V, the probability Pry~Yono M/ yk j] that both samples agree on token j is
min .P
(k))
1
min(p, "o," ,p,") (2)
Σ, max (p .p.
22
Pr I y = =
Pr [y+ uk 5] p. (K)
y~ ind
with equality possible only when max (px (1',, 10,,) = 1.
3.2 PRIVACY PROPERTIES
With both independent and coordinated ensembles, aggregate the histogram in preeecygr way to select a single token. While the distribution of the histograms produced by these ensemble types is very different, the privacy properties in terms of the divergence between neighboring datasets are identical and immediate:
Observation For every fixture of the shared randomness p. changing one of the distributions ( given as input to Algorithm chaanges at most one item of the resulting histogram. That is, letting H and H' denote the resulting histograms before and after the modification, we have that H, H' are at Hamming distance 2 (viewed as vectors in MV),
The following corollary is immediate from Observation
Corollary 1. Let A be an algorithm whose input is a histogram E NIVI, such that for any two neighboring histograms H, H'' (differing by at most item) it holds that A(H) =ee,5) A(H"). Then the composed algorithm A (Coordinat edH sddggra())))))) is (c, (eiff-eiirrrenniallly private
3.3 AGGREGATORS AND ENSEMBLE SAMPLERS
Define ST(c) = (j E V ej 2 T} and Mr(c) Ejest(e) Cj. We define the thresholded maximizer and weighted sample aggregators. Observe that they trivially satisfy T-threshold privacy:
TARGMAXT(c) = arg max 9j Sr(() # Ø; <fail> otherwise.
(c) with prob. min (e) y Cat (ert)) jeSr(e) ; else y <fail>.
This corollary holds for all variants of differential privacy, and written here with (s, 6DP for concreteness.
7

---

Published as a conference paper at ICLR 2026
DP versions of these aggregators, DPARGMAX(s.8) (Appendix E.1 and (P (Appendix E.2 are presented in Appendix [ We now establish end-to-end diversity preeeevvtroooeooonnn (Definition I and privacy guarantees for ensemble samplers of the form M A Henos which, given teacher distributions (E/n), first sample coordinated histogram Hoool (eco)(p())))E()))) )ie(m) and then return A(c), yielding distribution over V.
Theorem 2 (Ensemble samplers properties). For any E [n] and y 2 1, with A TWS, ///.//2 /2 sampler X satisfies (T (T/2))threhhold privacy and is diversity preserving with parameters (T, 0.17,7).
For > n/2. with A TARGMAX (2 sampler M satisfies (T [n/2+ 11)-threshold (
privacy and is diversity preserving with (r,
For e, > 0: With = DPWS(E.f) (), sampler M is (c, () and diversity preserving with (T = 4g 1og(1 /6), 8 = -(1),==1)
For E, > 0: with A DPARGMAX sampler M is (e, 8DDP and diversity preserving with (T 0.6n 35- log(1/6), e 2).
Proof. From Corollary [ the privacy properties M inherit those of A. The diversity preservation properties for threshold privacy aggregators are established in Appendix B and those of the DP aggregators are established in Appendix B
4 EMPIRICAL DEMONSTRATION FOR SEQUENTIAL TEXT GENERATION
We compare coordinated ensembles (Hot PATE) to a baseline of independent ensembles (Cold PATE) for sequential text generation as described in Framework 1.2 We evaluate on natural and curated task. We use default temperature settings (e.g.. f = 1) and took a few minutes on a single A100 GPU.
Evaluation metrics: In our evaluation, at given generation step, corresponding to set of contexts C. R for i E [n]. we sample r 10 vote histograms (c(*))) )== from each of the coordinated and independent ensembles. Each histogram aggregates votes from n teachers, with each teacher
(h) contributing a single token. We denote by cj" the count of token j in the /th histogram (for / E [r]).
We use threshold value T [n] on token counts proxy for the inverse privacy cost We evaluate the utility of an ensemble type at a threshold value T using the following measures: (i) transferred (h)
probability mass (coverage): І Eh-n Ejev Cj 1(ch)) 2 T), the fraction of total votes assigned to
tokens with frequency least T; (ii) transferred support size: Hj V maXne(r] c > T) the number of distinct tokens that appear above threshold in at least one histogram; and (iii) average
yield per sample: f DK Ki e V: gh) TH, the average number of above-threshold tokens per histogram.
4.1 NATURAL TASK: SYNTHETIC INSTRUCTION GENERATION FROM A SENSITIVE DATASET OF INSTRUCTIONS
Dataset: We used Dolly 15K (Conover et al. 2023). a dataset of instructions and corresponding responses intended for training "chat" models like ChatGPT (in this work, we only use the instruc- tions). We filter the dataset to include only instructions without a context that are shorter than 256 characters, resulting in a pool of about 10K examples of the original 15K.
Model and setup: To generate synthetic instructions, we use the pre-trained L.1 ama 3.11-..10 1-8 (Ia 2024) base model which is capable of in-context learning. Specifically, when we present this model with a few instructions as context, it consistently generates another instruction. The data was randomly partitioned
512 teachers with initial contexts (C,) of 10 instructions. At each step of the generation, for a fixed partial response R, we sampled r = 1000 histograms. We discuss the results, additional results are reported in Appendix C
UUnder DP, token can he reliably reported only when its count exceeds the scale of the noise introduced by the privacy-preserving mechanism (e.g., Gaussian or Laplace noise).
8

---

Published as a conference paper at ICLR 2026
Gains in utility: Figure regurtrrporrts reports the coverage and support-size of the transfer for two prefixes R. Coordinated ensembles attain high coverage and support even with T = 0.5m whereas independent ensembles transfer diversity, only token, for the first prefix and fail to even have yield (return a relevant token) for the second prefix with 7 > 0n17m. This is because independent ensembles can only transfer tokens when their average probability is Z T Figure (left) shows the distribution of the maximum count in the histogram: for prefixes with diverse next-token independent ensembles require much lower I (high privacy cost) even for the basic utility of yield.
peresting
Figure The transferred support- and coverage per threshold T with coordinated and independent ensembles. Generating with prefixes R 0 (left) and R ="What. does the word "ch" (right).
Additional privacy analysis benefits: The privacy noise scale (proxied by the threshold T) is "first order" indicator for the privacy cost with basic privacy analysis. The variety of data-dependent privacy analysis techniques (Dwork et al. 2006 Papernot et al., 2018 Cohen and Lyu 2023) benefit by "not charging" for failed aggregations and "charging less" when there larger margin between the highest and second highest count. We demonstrate that coordinated ensembles reap more of these benefits well. Figure 3 (middle) demonstrates that retries (with the same noise scale) beneficial with coordinated ensembles, as the maximum count over several tries can be much larger than in a single try. With independent ensembles, counts concentrate around their expectations, and there is little benefits in retries. Additionally, Figure 3 (right) demonstrates large margins with coordinated ensembles. In independent ensembles, margins smaller when diversity higher as they simply reflect the difference in expected counts between the highest and second highest frequency in the average distribution. large margin means that the output is much more stable which is significant benefit with refined privacy analysis] Thakurta and Smith (2013); Bassily et al. (2018) Cohen and (see Appendix C
Lyu 2023)
Figure 3: Maximum token count per histogram for different prefixes R (left: single attempt, middle: max in 10 attempts). Margin between highest and second highest counts (right).
4.2 CURATED TASK
We designed task for which (i) the pre-trained model has prior exposure that the "sensitive" context must be used for generating a good response, (ii) some mechanism is necessary for protecting the "private" information, and (iii) diversity is tunable. For simplicity, the task designed return
single token. We use the instruction-tun L. 1lama-3- 3-8B (Ila 2024 (Ila 2024 Al Meta 2024 model.
Prompts:
For each experiment we use n = 10 text prompts (teachers) of the form:
On Planet Y, some numbers are edible. <name> from planet X eats the following numbers for breakfast: <random permutation of C (<priv nun>) Give example breakfast number in planet Z. Respond with just the number.

---

Published as a conference paper at ICLR 2026
The fixed set C' is uniform sample of size |C] k from the set IN000 {100..... 999] of the 900 3-digit numbers. The strings <name> and <priv num> U U0D 100 \ CJ were generated separately for cach prompt i E in]. For our purposes, the set C' is the information we want transferred whereas the <name>, <priv num?, and the ordering of C' in the prompt are -oopp--ppecfficc and scnsitive. Each prompt is designed to have l + 1 correct answers. We report results with & E {20, 1003. Llama-33 3-8B uses vocabulary V of 128k tokens and 3-digit numbers are encoded as single tokens. The distributions pd exhibited biases towards certain numbers and high variation. The probability of returning 3-digit number was 0.995; but the model generalized and returned with 25% probability numbers outside the input set. Note that our goal is simply to reflect what the model does, including biases and generalizing. See Appendix for further details.
Figure 4: Left: Average yield per sample. Middle left: Coverage. Middle right: Total Variation Distance between transferred and average distribution; all as function of T. Right: Coverage versus support-size with coordinated and independent ensembles, when sweeping the parameter T (not shown). Top: l 20. Bottom: k 100.
Utility Evaluation Figure 4 (left) shows the average yield per sample for varying T. Observe that with independent ensembles, the maximum frequency ma of (over histograms and tokens) corresponds to the maximum token average probability: for k = 20 it is 0.14n and for A = 100 it is 0.03n. With coordinated ensembles, the majority of samples contained a token with frequency above 0.25n (that is much higher than the maximum token average probability). Figure (middle right) reports the total variation distance from the average distribution and Figure Figure[mmiddllee left) reports coverage for varying T. We observe much higher coverage with coordinated ensembles compared with independent ensembles. Additionally, we observe that the coverage corresponds to the T'nt-tobust part of the distribution shown in Figure 1O] that is, it corresponds to what we can hope to transfer (see Theorem 2 and Section D.2). For k 100, see 20% coverage with T 2000 with coordinated sampling but we need T Σ 250 with independent sampling (8x privacy budget). For k = 20. we see 40% coverage with T 4000 with coordinated sampling but we need S 1000 with independent sampling (4x in privacy budget). Morcover, independent samples have 0% coverage with T 2 1500 for k = 20 and with T 2 400 for k = 100 (when T/n exceeds the maximum average frequency) whereas coordinated ensembles are effective with high T. Figure g (right) shows a parametric plot (by threshold T, not shown) relating coverage and support size for coordinated and independent ensembles. Coordinated ensembles exhibit substantially greater diversity, achieving significantly larger support sizes at the same coverage levels, often with an order-of -faggntuu- gap compared to independent ensembles.
CONCLUSION
We introduced Hot PATE, an enhancement of the PATE framework for tasks with diverse outputs. Our core technical contribution is formal notion of a robust, with the method of generation via coordinated ensembles. Compared to the baseline "Cold" PATE, which uses independent ensembles, coordinated ensembles provably achieve higher utility for privacy budget and diversity preservation. We demonstrated orders of magnituge improvements in in-context learning scenarios, such as generating privacy-pree preserving synthetic data records from sensitive inputs. The improvement stems from the shape of the ensemble votes histograms: higher top count and separation of the top count which is favorable to privacy analysis.
ppreeerving aggregation of distributions, along
Finally, our design supports not only differential privacy but also lighter privacy enhancements that offer higher utility for tasks such as synthetic record generation and summarization. By using fewer teachers, a lower robustness threshold, and omitting DP noise from the vote counts, we can have robustness at each decoding step and protection from generating non-generalized idiosyneratic sequences (those due to one or few examples), while preserving diversity.
10

---

Published as a conference paper at ICLR 2026
ACKNOWLEDGMENTS
Ben Cohen-Wane Wang: Work done while the author student at MIT. Work supported in part by Open Philanthropy. Edith Cohen: Partially supported by Israel Science Foundation (grant 1156/23). Uri Stemmer: Partially supported by Israel Science Foundation (grant 1419/24), the Blavatnik Research Foundation
REFERENCES
The llama 3 model: A deep learning approach to language understanding, meta lama 2024.
@etMa Llama model card. 2024. Btt.:ttt.e//atttbbb..o/eeeeeeeee ma wain//MDDBL CARD. md]
Authoopic. Fine-tuning for claude 3 haiku in amazon bedrock, 2024. URL httpt::t hwwps::/wwwww.oo wwt../oop......o.c..c...........
com newn//eww///e Fine-turee claude- 3- ha baiku- sikuiaaaa
Racf Bassily, Om Thakkar, and Abhradcep Guha Thakurta. Model-agnostic private learning, In S. Bengio, H. Wallach, H. Larochelle, Grauman, N. Cesa-Bianchi, and Garnett, editors, Advances Neural Infor- mation Processing Syxtems, volume 31, Curran Associates, Inc., 2018. URL httpsst/p//oooeeddiiii .p
neurips wwturps..ccpapaeee http:cccpppppe/eeeeeeeeeeee
2018 2018/ Fi File//aaeaa aa
K. R. W. Brewer, I.. Early, and S. F. loyce. Selecting several samples from single population. Australian Journal of Statistics, 14(3):231-22 1972.
A. Z. Broder. Identifying and filtering near-duplicate documents. In Proc.of the 11th Annual Symposium on Combinatorial Pattern Matching, volume 1848 of LNCS, pages 1-10. Springer, 2000.
Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herhert- Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. Language models are few-shot learners, 2020.
Mark Bun, Thomas Steinke, and Jonathan Ullman. Make Up Your Mind: The Price of Online Queries in Differential Privacy, pages 306--325. 2017. doi: s siam. org dei http:/dcc/aaabb/s
1137/1
Mark Bun, Kobbi Nissim,
Stemmer. Simultaneous private learning multiple concepts. J. Mach. Learn. 549.html
Res., 20:94:1 1 1-94:34, 2019. URL, htttt://////.rr.. http:///mwrr.rrrrpaaee/s////////
E. Cohen. Estimating the size of the transitive closure in linear time. In Proc. 35th IEEE Annual Symposium on Foundations of Computer Science, pages 190-200. IEEE, 1994.
E. Cohen. Size-estimation framework with applications transitive closure and reachability. J. Comput. System Sci., 55:441 1997.
Edith Cohen Lyu. target-charging technique for privacy accounting across interactive computttionn.. CoRR, abs/2302. 1044,
11044, 2023. doi: 10.48550/arxiw.2302.11 1104. URL
11044
Edith Cohen, Ofir Geri, Tamas Sarlos, and Uri Stemmer. Differentially private weighted sampling. In Proceedings of The 24th International Conference on Artificial Intelligence and Statistics, volume 130 of Proceedings of Machine Learning Research. PMLR, 2021. URL httpss//ppooceedingg mlr....r..... wwt..press/v130
cohen21b.
Mike Conover, Matt Hayes, Ankit Mathur, Jianwci Xic, Jun Wan, Sam Shah, Ali Ghodsi, Patrick Wendell, Matei Zaharia, and Reynold Xin. Free dolly: Introducing the world's first truly
open instruction-tuned I1m, 2023. URL httpp::////aaa www.rrittttaiiccccoo wtt.ss.cooccooooooe folly-first-oper ooaerrr-ooe--ommmmmrraall----aee htw.:l/yww..bblleeesestttt.iooi. -uneed-nnnddddnn
comm/blog/022304/115
Damien Desfontaines, James Voss, Bryant Gipson, and Chinmoy Mandayam. Differentially private parti- tion selection. Proceedings on Privacy Enhancing Technologies. 2022(1):339 352, 2022. doi: 10,2478/ popets- 202ets-2022-0017. URI. https:
2022 0017
fdf

---

Published as a conference paper at ICLR 2026
Haonan Duan, Adam Dziedzic, Nicolas Papernot, and Franziska Boenisch. Flocks of stochastic parrots: Difterentially private prompt learning for large language models, 2023.
David Durfee and Ryan M. Rogers. Practical differentially private top-k selection with pay- woy--wht-oou-ge gett composition. In Hanna M. Wallach. Hugo Larochelle. Alina Beygelzimer, Florence d' Alché-Buc, Emily Fox, and Roman Gamett, editors, Advances in Neural Information Processing Systems 32: Annual Confer- ence on Neural Information Processing Systems 2019, NewrIPS 2019, December 8- 14, 2019, Vancouver,
BC, Canada, pages -527-3522333 2019. URL https: /ptpp:::/prroeedinngs. www.inngg..eeuugee www.risus...... wwwpurips.ccppapppe.ee ptppppp///211/////////// hash/b139e104211 2ww.e44224000eeeeee Feebcce 14 49cdf6e wwwtaeeeeetaaat.....oee
Abstract. .hmml
Cynthia Dwork, Frank McSherry, Kobbi Nissim, and Adam Smith. Calibrating noise to sensitivity in private data analysis. TCC, 2006.
Cynthia Dwork, Moni Naor, Omer Reingold, Guy N. Rothblum, and Salil Vadhan. On the complexity of differentially private data release: Efficient algorithms and hardness results. In Proceedings of the Forty-First Annual ACM Symposium on Theary of Computing, STOC '09, page 381 181-90, New York, NY, USA, 2009. URL
Association for Computing Machinery. ISBN 9781605585062. doi: 10.1145/15364144155666
https: /wcc///c..... org/ 10..1114//100 00444555534444.4 1536467
Fengyu Gao, Ruida Zhou, Tianhao Wang, Cong Shen, and Jing Yang. Data-adaptive differentially private prompt
synthesis for in-context learning. In The Thirteenth International Conference on Learning Representations
(ICLR), 2025. URL[https:: //atpis://rriivi /arxiv.vvvv...///i//.ar////aaala org/ abs/ 2 2410. 12085
Shivam Garg, Dimitris Tsipras. Percy Liang. and Gregory Valiant. What can transformers learn in-context? a case study of simple function classes, 2023.
Badih Ghazi, Ravi Kumar, and Pasin Manurangsi. User-level differentially private learning via corre- lated sampling. In Mare' Aurelio Ranzato, Alina Beygelzimer, Yann N. Dauphin, Percy Liang, and Jennifer Wortman Vaughan, editors, Advances Neural Information Processing Systems 34: Annual Conference on Neural Information Processing Systems 2021, NeurlPS 2021, December 6-14, 2021, I, vir- mull pages 20172-20184 84, 2021.
www..uuu....nggggeo............. neur wttpp/pp.pppp......pppp ptp.:/apprreeeeee /001 /hash/ 6ce. 65e 39a4b Abstract.htm .tmll
/tttpss/pprrceeeeee
529c
Arpita Ghosh, Tim Roughgarden, and Mukund Sundararajan. Universally uility-maximizing privacy mecha- nisms. SIAM J. Comput., (1( 1673-6633 2012. URL htttptt/s::/dd 1137771133/ 76828X
1137/
Emil Julius Gumbel. Statistical Theory of Extreme Values and Some Practical Applications: A Series of Lectures, volume 33 of Applied Mathematics Series. U.S. Department of Commerce. National Bureau of Standards, Washington, D.C., 1954. URL petetts://nvluuteee /ww.ppus..../..///////pm......s. .ist. www/nisstpooeoo /aapttss//egaacy Legacy/ anabammm333..........3.........3 pclf Available from the National Institute of Standards and Technology (NIST) Digital Library of Mathematical Functions.
Peter J. Haas. Sketches get sketchier. Commun. ACM, 5408):00, 2011. doi: 10.1145/1978542 URL
httpp::/doo www.oogg//lo
Junyuan Hong, Jiachen T. Wang, Chenhui Zhang, Zhangheng Li, Bo Li, and Zhangyang Wang. Op-opt Make prompt engineer. In The Twelfth International Conference on Learning Representations (ICLR), 2024. URL http::: wawxiv......////.i/.ea.///avavl/ org/ abs 2 232..222222 03724 Spotlight.
large language model your
Edward Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models. In Proceedings of the International Conference Learning Representations (ICLR), 2022. URL httttps://oooee /ttp::////ppeeeeieeeeeeeom net/ forum??
1d=nzevkeoFyE3
P. Indyk and R. Motwani, Approximate nearest neighbors: Towards removing the curse of dimensionality. In Proc. 30th Annual ACM Symposium on Theory of Computing, pages 604-613. ACM, 1998.
Paul Jaccard. Étude comparative de la distribution florale dans une portion des alpes et du jura. Bulletin de la Société Vaudoise des Sciences Naturelles, 37:547-5777 1901.
James Jordon, Jinsung Yoon, Mihaela van der Schaar. Pate-gan: Generating synthetic data with differential
privacy guarantees. In International Conference on Learning Representations, 2018. URL httpss//aap semant iesaantcsschlaar
Haim Kaplan, Yishay Mansour, and Uri Stemmer. The sparse vector technique, revisited. In Mikhail Belkin and Samory Kpotufe, editors, Conference on Learning Theory, COLT 2021, 15-19 August 2021, Boulder, Colorado, USA, volume 134 of Proceedings of Machine Learning Research, pages 2747-2776. PMIR, 2021. /RI hhtttp://procee.ee ngs. https/mm.p..sess/////aaaaaa
a.htm
12

---

Published as a conference paper at ICLR 2026
L. Kish and A. Scott. Retaining units after changing strata and probabilities. Journal of the American Statistical Association, 66(335)pp. 461-470, 1971. URLhttpp:/wwwc http:///wwww..o..o.////o//o///// wwtp:/oogggttoooe.e.ee.o wttp//tbbl/////33r333333
Aleksandra Komlova, Krishnaram Kenthapadi, Nina Mishra, and Alexandros Ntoulas. Releasing search queries and clicks privately. In Juan Quemada, Gonzalo León, Yoëlle S. Maarek, and Wolfgang Nejdl. editors. Pruceedings of the 18th International Conference on World Wide Web, WWW 2009, Madrid, Spain, April org/
20-24, 2009, pages 171-180. ACM, 2009. doi: 00.1145/1526709115277733 URL
[145/1526709, 526733
Yaniv Leviathan, Matan Kalman, and Yossi Matias, Fast inference from transformers via speculative decoding. In Proceedings of the 40th International Conference on Machine Learning, ICML'23, JMLR.org, 2023.
Zinan Lin, Sivakanth Gopi, Janardhan Kulkarni, Harsha Nori, and Sergey Yekhamin. Differentially private symthetic data via foundation model APIs I: Images. The Twelfth International Conference on Learning Representations, 2024. htlLttts::// /operr// sww net w d-EEh0S8P010
Jiachang Liu, Dinghan Shen, Yizhe Zhang, Bill Dolan, Lawrence Carin, and Weizhu Chen. What makes good
in-context examples for gpt-3? CoRR, abs/2101.06600 2021.
06804
Yunhui Long, Boxin Wang. Zhuolin Yang. Bhavya Kailkhura, Aston Zhang. Carl Gunter, and Li. G-pate: Scalable differentially private data generator via private aggregation of teacher discrimina- tors. In M. Ranzato, A. Beygelzimer, Y. Dauphin, P.S. Liang, and J. Wortman Vaughan, editors, Ad- vances in Neural Information Processing Systems, volume 34, pages 2965-2977. Curran Associates, Inc., 2021. URL htttp:::/ppoocccingg..t httppa/.rrrrrppppppppparrrr.rrre 7lleebbbaal177555227777778888833 papp wwtppppp__iiaeeeeeeeee httppppppp..llllsp///////////11/ http:/ppppcr/2////1/1/
Frank McSherry and Kunal Talwar. Mechanism design via differential privacy. In 48th Annual IEEE Symposium on Foundations of Computer Science (FOCS 2007), October 20-23, 2007, Providence, RI, USA, Proceedings, https:
pages 94 103. IEBE Computer Society, 2007. doi: 10. 10.1109/FOCS.000
10. www111199Fcc.220 20000
Microsoft
Azure
Fine-tuning
models with
azure
openai service, 2024.
URL
fine-
Kobbi Nissim, Sofya Raskhodnikova, and Adam Smith. Smooth sensitivity and sampling in private data analysis. In Proceedings of the thirty-ninth annual ACM symposium on Theory of computing, pages 75-84, 2007.
Ohlsson. Coordination of pps samples over time. The 2nd International Conference on Establishment Surveys, pages 255-264. American Statistical Association, 2000.
OpenAl. Fine-tuning guide, 2023.
URL htttps://paat http://aaa.too.ooops......oo /wwiessssss/iio//eeoeooooee
Fine-tuning
OpenAl. Openai api pricing. https://opens: httpa://ppmaai..com//p///prrrpe
2025a. Accessed: 2025.
OpenAl. How people use chatgpt: Usage analysis with aggregation thresholds, 2025b.
URL
http://dd..cccc.cccccooocoocoe economi rww.arrseaaah..e chatreetttteeerccrcaacatttt-eeee
8260- 40c6 awcc-aa93fc9f1ee0...
paper .pdf Technical Report.
Nicolas Papernot, Martín Abadi, Úlfar Erlingsson, Ian Goodfellow, and Kunal Talwar. Semi-supervised
knowledge transfer for deep learning from private training data. In 5th International Conference Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings. OpenRe- view.net, 2017. URL bttps:// http///pppeeeeeeeeeeeem net. httr://rrmmmmomm http:/oorrmmmi--Hoo00000
Nicolas Papernot, Shuang Song, Ilya Mironov, Ananth Raghunathan, Kunal Talwar, and Úlfar Erlingsson. Scalable private learning with PATE. In 6th International Conference on Learning Representations, ICLR 2018, Vancouver. BC, Canada, April 30- May 3, 2018, Conference Track Proceedings. OpenReview.net, 2018. BtLLttts::/ /wtp://ieeeeeeeeeeeeemeoeeeeeeee net forum? fimmmiddd==kea 1d=rk231
Gang Qiao, Weijie J. Su, and Li Zhang. Oneshot differentially private top-k selection. In Marina Meila and Tong Zhang, editors, Proceedings of the 38th International Conference on Machine Learning, ICMI. 2021, 18-24 July 2021, Virnual Evens, volume 139 of Proceedings of Machine Learning Research, pages 8672-8681. PMLR. 2021. URL/attp://prooeeeise
.ml wwwssppeseeeee........ee https//llll//aalall/a html
Alec Radford, Jeff Wu, Rewon Child, David Luan, Dario Amodei, and llya Sutskever. Language models are unsupervised multitask learners. 2019. URI. https: //api semant cscho ar ar.org/Corpus.
1 66025533
13

---

Published as a conference paper at ICLR 2026
B. Rosén. Asymptotic theory for order sampling, J. Statistical Planning and Inference, 62(2):135-158 1997.
J. Saavedra. Fixed sample size pps approximations with a permanent random number. In Proc. of the Section Alexandria, VA, 1995. American Statistical Association.
on Survey Research Methods, pages
Latanya Sweeney. k-anonymity: A model for protecting privacy. International Journal of Uncertainty, Fuzziness and Knowledge-Buud BeaeededeeBBed Systems, 10(5):557-570, 2002. doi: 10.1142/5021844885000000
Alex Tamkin, Miles McCain, Esin Durmus, Kunal Handa, et al. Clio: Privacy- insights into real-world use. arXiv preprint arX1v:2412.13668, 2024. URL
13678
Xinyu Tang, Richard Shin, Huseyin A Inan, Andre Manoel, Fatemchsadat Mireshghallah, Zinan Lin, Sivakanth Gopi, Janardhan Kulkarni, and Robert Sim. Privacy-preservee in-context learning with differentially private few-shot generation. In The Twelfth International Conference on Learning Representations, 2024. URL https: Duttp::/ccpenee http///ppeeeeeeeeeeeee/eeeeeeeet ew. net/ forum?i watrrmmmidc//tttpmrrmm... c=ctt.0pRoO11
Abhradeep Guha Thakurta and Adam Smith. Differentially private feature selection via stability arguments. and the robustness of the lasso. In Shai Shalev-Shwartz and Ingo Steinwart, editors, Proceedings of the 26th Annual Conference on Learning Theory, volume 30 of Proceedings of Machine Learning Research, pages 819 850, Princcton, NJ, USA. 12 14 Jun 2013. PMLR. v30 V30/Guha13.htm
mlr press//
Zhiliang Tian, Yingxiu Zhao, Ziyue Huang, Yu-Xiang Wang. Nevin Zhang, and He. Seqpate: Differentially private text generation via knowledge distillation. In S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho. and A. Oh, editors, Advances in Neural Information Processing Systems, volume 35, pages 11117- 11130. Curran Associates, Inc., 2022. CRL/https: httpps//pppccccepsssccooo neuriss wa/paper siles/
Salil Vadhan. The Complexity of Differential Privacy. 04 2017. ISBN 978-3-319-57047441 300-3--11--774----
doi: 10.1007/
Tong Wu, Ashwinee Panda, Jiachen T. Wang, and Prateek Mittal. Privacy-preserving in-context learning for large language models, 2023.
Chulin Xie, Zinan Lin, Arturs Backurs, Sivakanth Gopi. Da Yu, Huseyin Inan, Harsha Nori, Haotian Jiang, Huishuai Zhang, Yin Tat Lec, Bo Li, and Sergey Yckhanin. Diffcrentially private synthetic data via foundation model APIs 2: Text. In ICLR 2024 Workshop on Secure and Trustworthy Large Language Models, 2024.
URL https: / wwwp//eeeeeeeeeeeeeepee..e.e..wp net/ forum?ide JnF 53xXmB
Yusuke Yamasaki, Kenta Niwa, Daiki Chijiwa, Takumi Fukami, and Takayuki Miura. Plausible token plification for improving accuracy differentially private in-context learning based on implicit bayesian inference. In The Forty-Second International Conference on Machine Learning (ICML), 2025. URL
https://cpenreve .et./ /eeet//orwe Eorumr serrumiid=sk. aAEuA2
John I. Yellott. The relationship between luce's choice axiom, thurstonc's 's theory of comparative judgment, and the double exponential distribution. Journal of Mathematical Psychology, 31(():2885-)) -00, 1987. doi: 10. 1001//002222996187))000 (496(87)90043-8
wtt.o/////.....//0000111 101111102010000202222222 (87) 90043-2
Hattie Zhou, Azade Nova, Hugo Larochelle, Aaron Courville, Behnam Neyshabur, and Hanie Sedghi. Teaching algorithmic reasoning via in-context learning, 2022.
A RELATED WORK
(a) Ensemble types for Hot PATE. Homogeneous en- sembles use representative data splits. Heterogencous ensembles use user-specific data (i.c., "privacy units").
(b) Diversity within teachers arises from shared knowl- edge; across teachers, from knowledge specific few teachers. With coordinated ensembles, high value suffices for the former.
*
+
$ #
Heterogeneous
Homogeneous
Diversity within teachers
Diversity teachers
14

---

Published as a conference paper at ICLR 2026
We place our contribution in the context of prior and independent concurrent works on PATE adaptations for text generation. These works either (i) did not consider diversity or (ii) recognized it and the importance of transferring it but proposed aggregation schemes where utility decreases with diversity together with methods to limit diversity as to mitigate this perceived privacy-diversio tradc-off. In some of these designs, our Hot PATE ensemble samplers can be used as a plug-in replacement to improve utility.
Tian al. 2022 proposed PATE extension for sequential text generation that employs an ensemble sampling mechanism not based on voting. Their method computes a prvecy-preserving average of the teachers' distribu- tions. However, the intermediate step of computing a DP agggregat distribution incurs the dimensionality penalty associated with DP mean estimation. As a result, utility behaves similarly to that of independent ensemble samplers and declines when diversity increases. To mitigate this penalty, they limit diversity by truncating the tail and retaining only the top-k prohahilities. The independent and concurrent work of Tan al. (2024) takes a similar approach. Each teacher distribution is truncated and rescaled to its top-k tokens, after which a preserving average distribution is computed, and a token is sampled from this aggregate. This design limits diversity in an attempt to improve utility. but still incurs penalties for any remaining diversity. Subsequent follow-up works (also later than Hot PATE), including Gao et al. (2025) and Yamasaki al. (2025), propose alternative DP aggregation mcchanisms for teacher probability vectors, aiming to alleviate the dimensionality issue inherent in DP mean estimation. Their evaluations focus primarily on classification-sti -tsyll (low (low-diversity) settings, and diversity preservation is neither a design objective nor an achieved property in these approaches. All of these methods assume full access to the teachers' probability distributions (or top-k logits). Hot PATE also requires access teacher distributions, but this may be achievable via APIs that support sampling with specified random seed, without needing full logit access.
Duan et al. 2023) explored adaptations of PATE for in-context learning via prompting, where each part D, of the data is used to create a text prompt C, The ensemble is then used to label curated queries. But while some design elements were tailored to LLMs, the workflow and privacy analysis were identical to Cold PATE (Papernot et al.
2018). and in particular, did not consider diverse responses. [Hong et al. (2024) proposed similar use of Cold PATE in pipeline for DP prompt generation.
Wu et al. 2023) (independent concurrent work) proposed approaches to private aggregation for in-context learning with diversity. They proposed to reduce the perceived diversity in sequentially geenerate text outputs by different teachers by clustering together outputs that are semantically equivalent and aggregating each cluster in semantic space. This essentially reduces the dimensionality of the output space. The aim then is to extract and transfer this common semantics in a privacy preserving way: Map responses into common low dimensional embedding space and privately aggregate embedding vectors or identify frequent keywords in diverse teachers' responses. The limitations are that the approach only addresses same-semantis semantics diversity and offers no solution for semantically distinet diverse responses and subjected to privacy diversity trade-off. Additionally and importantly, they require hand crafted tools map and curate responses back and forth from a semantic space. The added value of such mapping approach, combined with coordinated ensembles, depends on whether the reduction of diversity that is achieved is within or across teachers. The across variety (see Figure (bb (B)), where the knowledge of each teacher only contains one or limited variations of the same semantic, is not eliminated by ensemble coordination and thus there is added value by addressing it via other means. The within variety (see Figure 5b) A)) is handled effectively by ensemble coordination and can be transferred fluidly with no privacy loss and without the need for mitigation of diversity via additional engineering. We suspect that for the in-context learning use case, and for semantic similarity that can be captured by tools external to the model (such as an embedding). the diversity eliminated is anyhow encapsulated in the base model and thus present in most teacher distributions. That is, we expect the diversity overwhelmingly be the "within" variety.
Lin et al. 22024); Xic et al. 2024) (independent and concurrent work) proposed an approach called private evolution for generating synthetic examples from private data. Their method uses heterogeneous teachers, where each teacher corresponds to single sensitive example. In each iteration, the base model is sampled to generate a collection of candidate (full) responses. The teachers then vote these candidates based on nearest-neighbor matches in an embedding space. privacy-preserving vote histogram computed, and candidates are sampled from it with weights corresponding to their votes. The sampled candidates are then used to generate a new set of candidates from the base model, progressively moving closer to the private distribution. This process is repeated over multiple rounds. While elegant, this approach has inherent limitations since it depends on the base model's ability to generalize from its pretrained knowledge, it is less suitable for transferring patterns specific to the sensitive data records or ephemeral trends that emerged in these records but are absent from the base model's training corpus. Furthermore, the method requires a number of candidates that grows exponentially with the intrinsic dimensionality of the candidate space. which may become impractical. Thus, the application domains of private evolution differ from those of Hot PATE, and the methods are directly comparable. On our two experiments, we expect private evolution to perform well on the instruction instruction-eeneraioon task, since the sensitive data primarily provides high-level structure (e.g., that questions should be generated in a particular syntactic form). Hot PATE also henefits from this structure through data-dependent privacy analysis. However, on the curated task. where the sensitive data consists of a specific random subset of three-digit numbers, private
15

---

Published as a conference paper at ICLR 2026
evolution would likely require large number of iterations to infer this structure, and therefore would be less effective than sequential generation via Hot PATE.
Papermot et al. 2017) (Appendix B. discussed using additional outputs (beyond just the noisy the maximizer) in the teachers' voles histogram for distillation tasks. They concluded that it is beneficial for utility but does not justify the privacy loss. Despite the superficial resemblance, this is very different from what we do as capture diversity in the generation of the histogram where "force" the teachers agree but there is distribution on the agreement token.
Finally, there are multiple innovative adaptations of PATE non-categorical settings (aggregate vectors rather than labels) applied with generative models. The works we are aware of address different problems and use different techniques than Hot PATE, For example, image generation using generative adversarial networks [GAN):Jordon et al. (2088) proposed train student discriminator using cold-PATE like labeling approach. Long et al. (2021) proposed to train a student generator by aggregating
gradients produced by teachers discriminators. Notably, as with Hot PATE, this design does not require external generation of examples order to facilitate transfer. Instead, it uses the built-in property of generators to produce examples from random strings.
B PROPERTIES OF COORDINATED ENSEMBLES
Proof of Claim 2 The first statement in the claim follows from the denominator satisfying (k) 2 max (4)
1 Σ
ma (6) P)
.Pj 2.
(3)
The inequality follows using the more refined upper bound 33 on the denominator.
It follows from Claim 2 that the overall agreement probability of the two teachers i' (over all tokens) is:
[ys = yel = Σ, min(p, (9)
PFj (68))
Pr Yeoo
= J(p ())
Σ, max (9) .p5")
where J(p, q) = Liev min (ps:eg) distributions p.q.
(-.V(pp.) is the weighted Jaccard similarity (Jaccard 1901 of the 14TV(pp.)
2 (py17,)
In particular, when two teacher distributions are identical, the samples are the same
( (4)
Pr
[y:
yz)
YowYooo
Lemma (diversity transfer). For any token j and p. a E [0,1].
Pr Hcoooo
C
(8) p 1n
1 In(1/p)g.
2
EEn
Proof. Let be such that , 2 q. Fix the sampled min value Expig] for g part of the probability of j. The " which is stochastically smaller than Exp[1 4]. We get
distribution of the remaining part is y ~ Exp[1
that
Pr[m = 1
Pr y Exol1
[y 2] = =e-
ln is least (2(-0) 2 Fix p E [0, 1). It follows the probability that Pr(yx = j). conditioned on x < I- The respective random variables ys different teachers that may share part of the distribution (4)
only be nonnegatively correlated. Therefore, if there are <j teachers with pj 2 q then the distribution of the number of teachers with yı = j is stochastically larger than (in(e==(1-9) <..g]. which for any æ S 7 is stochastically larger than Bin(p, cy.al The median of the Binomial distribution Bin (nf Gj with probability at least 1/2 is larger than [pcs.]]. Therefore, with this conditioning on x, there
least [pcs.]] teachers with y: j.
Pr [cy 2 [pc,.a]] 2 1/2.
(4)
(u.)eej) 2< Ing
The event Ing occurs with probability at least T-q
In = - -0000pp0000000 -(mp))
Pr [2 Expl
1-q
Combining with obtain the claim the statement of the Lemma.
16

---

Published as a conference paper at ICLR 2026
To establish relevance we show that high frequency must have "backing." " The following immediate from 2 and Markov's s inequality (and is tight in the sense that for any T there are distributions where equality holds); Lemma 2 (relevance), For any token and T.
Pr er Nroo
10 2
T
1 T 5c(m)
Σ
6) Pj
Proof of Theorem 2 diiversity properties). We first consider the 7 parameter. From Claim 1 for each j. [ey] nip,. Therefore, if for 7 2 our aggregator returns with probability at most 7e//n, it sat- isfies the relevance condition of Definition Wwith the respective 7 value.
For AARGMAX1//HH/ a token j returned if and only oj > n/2. Therefore we get = 2. For 7/5 a token j is returned with probability (aal 2 /cy/n. 4j
We next establish the claim for the transfer property of Definition | For T consider token for which m 2 T teachers have ) > q. Then from Lemma 1 with p 1/2 follows that Pr(c, 2 m/2] 2 (1/2) (2)g 0.34g. In this case, the probability that it is the output overall probability that is returned by TWS+/2.7 is at least (0.34/2)4 m = Sqm/nn for B = 0.17.
least L 2 m Therefore, the
For homogeneous ensembles via TARGMAX [o/2411 aggregator, we assume >> n/22. We apply Lemma with p = n/(2r) we obtain 3 = (1/2) log((2/m)
C FURTHER DETAILS FOR THE INSTRUCTION GENERATION DEMONSTRATION
Diversity transfer: Diversity transfer with coordinated and independent ensembles for additional prefixes R are reported in F We observe that with coordinated ensembles, more of the probahility mass transferred and it is much more diverse.
Maximum count: Figure Figure (left) shows the distribution of the maximum count for additional prefixes; (right) shows the maximum count over 10 tries (histograms generated with different samplings of shared randomness). We observe that with coordinated ensembles, the maximum token count is consistently at above 0.6n with one try and above 0.9n for the maximum over 10 tries. In particular, there is significant benefit to repetitions. As for independent ensembles, we observe that when there is high diversity (many appropriate choices for the next-token), the maximum count is frequently helow 0.2n and there is nearly no henefits for retries. As explained, the noise scale of the DP aggregation depends linearly in this maximum count. This means that even with basic privacy analysis (which does not benefit from margin). coordinated ensembles require over 4 times the number of teachers (and data) for the basic utility producing an instruction. As demonstrated, the produced instruction by independent ensembles would also be much less diverse, Furthermore, by using privacy accounting with wetweeThhre Theeehrrresool.. sholds (Cohen and Lyu 2023 Bun et al. (017)) we can generate a number of tokens that is exponential in the number of teachers when histograms are such that the maximum count is either very high (say above 0.6r) or very low (say below 0.4m).
Margin: The vote histograms generated by coordinated ensembles benefit not only a higher maximum count but also from a high margin between the highest count and second highest count tokens. Additional results that show the size of the margin between the highest and second highest counts in the histogram are reported in Figure 7 We observed margin that is consistently above 0.4n, where n is the number of teachers, with coordinated ensembles whereas very small margin occurs frequently with independent ensembles.
Benefits of high margin: We explain how high margins are leveraged in data dependent data analysis using the techniques of Bassily et al. (2018) Similar benefits are reaped via other methods such as (Cohen and Lyu 2023). Informally, their technique is based on coupling argument between the distance to instability framework of Thakurta and Smith (2013) and the sparse vector technique of Dwork et al. (2009). More specifically, the algorithm of Bassily et al. (2018) uses the sparse vector technique in order continuously verify that the number of "unstable queries" seen so does not cross some predefine threshold k;; and uses the distance to instability framework to answer queries as long as the number of unstable queries is indeed below k. If we assume, as is supposed by our experiments, that the margin in our algorithm is consistently above 7) (in our experiments we observed r = 0.4), then suffices to assert that n 2 32./2 log (Ag) log (3) in order to generate m tokens while satisfying (s, 6)- This means that (with high margin histograms) the number of tokens generated for given privacy parameters increases exponentially with the number of teachers. This can be contrasted with only quadratic increase with the number of teachers obtained using standard analysis with advanced composition.
"See Algorithm 3 in Bassily et al. 2018
17

---

Published as a conference paper at ICLR 2026
Thartate 2010 2011
Hislenc Paentt and TERCO
pertzing DiEu Dall
Sundere
Figure 6: The transferred support- size and coverage per threshold T with coordinated and independent ensembles, when generating a synthetic instruction. For multiple prefixes R.
D FURTHER DETAILS ON PLANET Z DEMONSTRATION
D.1 PROPERTIES OF THE GENERATED DISTRIBUTIONS
The distributions deviated from the "intended" one a uniform distribution over the numbers in the prompt: The model exhibited bias towards certain numbers, had spurious dependencies on private components, and generalized. Note that our evaluation focuses on the effectiveness of transferring the knowledge of the model, reflected in its generated response distributions, including its biases and generalizations. We observed the following:
The probability assigned by the model to tokens that are not 3-digit numbers is negligible: The average probability (over teachers) of response token in IN 100 was Exe(n) jjeNoon Pj Ri 0.997 for A 20 and = 0.994 for A = 100.
100
Tokens C dominate other 3-digit numbers are likely: The average probability a token in C' was E(ejs) Ejec pl R 0.716 (k = 20 tokens) and R 0.75 (k- = 100). Recall that only one in numbers in the prompt was in IN 199999 \ C, therefore the probability of 25%- assigned to these tokens is explained by the model generalizing that additional 3-digit numbers are edible Planet 7.
Despite symmetric prompt construction, there is significant variability in the average probability of different tokens in C' and in the probability across teachers of the same token. This is an artifact of the model. Figure reports regure@r the average (over prompts) of the probability of each token and demonstrates variability between tokens. The error bars indicate variability in the token prohability across teachers.
18

---

Published as a conference paper at ICLR 2026
260
500
660
200
300
330
Coardhattdd
Courtinicd
Moxt mum pasding (teestel whth 10
Noxhum 0000 to thrishe with 1:9
300
200
420
InSeRenden Coordfro22d
Independere
Courd
What
mess
powerrfal
wellpon
world
Pelnn
What
most
Dowerta
vonl
7ieln
Sonorsted token
Gonerated takes
505
540
605-
600
002
200
202-
200
Whns
Wind
7010
Genia stal Wikk
Generated laker
paskig drvattde airr by
12 143
-
602
3.0
Coordhiee
Goachheed
the ciforence
betweet
thergs
hoth
Figure 7: Maximum token count per next-token vote histogram for different prefixes R in a single attempt (left) and in 10 attempts (right)
D.2 QUANTIFYING HOW MUCH IS TRANSFERABLE
Remark (Robust Average). We the T-robust part of the average of the teachers distributions as an indicative upper bound the part that is privately transferrable:
Pf(T) = Σ min (D) ((p)) Inet)) for EV
(5)
1e(n)
19

---

Published as a conference paper at ICLR 2026
Margin between top two with 10lees
Hoggh beteccs tep teo cikes with
450
Indepandenk
Coard netedd
the
most
deliiins
crlsine
thE
wolld
Push
Whee
mar
woild
Genersted tokon
Gonorated tokes
Harghe petwoee trp two tokente with 10leee
Horghn on top 200 tiler with 57
400
300
3.00
Indegenders
Cocrd
mant
Rowerful
weapon
wolld
Path
mar
Auwerfll
woild
Generated
502
Indepenzent
500
602
460
200
belveen
200
Genvented move
Indepancent
Cosreina.es
260
2002
200
402
300
Insecendent Gorthees
Hut lnck
the
Figure 8: Margin between highest and second highest counts per histogram. single try (left) and largest of 10 tries (right).
20

---

Published as a conference paper at ICLR 2026
11
-00
Figure 9: Average probability, over teachers, of the tokens in C (left kt 20, right is k 100). The error bars indicate the contribution of the token to the average total variation distance over pairs of teacher distributions.
where (Net)))) (r) is the 7th largest probability of token in teacher distribution. Note that (P)(1))sev the average distribution and the values are non- inciincraasnng with t. The 7-robust probability mass, defined as P(T) := Eeev (T( S 1, upper bounds the transferrable probabilit mass. The complement 1 P(T) is indicative lower bound the probability of <fail> the robust aggregate.
Figure 10 reports the r-robust fraction of the average distribution for varying T (sce Remark 2 This is the part of the average distribution that we can hope to transfer via coordinated ensembles with support r. Recall that variability in the same token among teachers decreases transferability whereas variability among tokens does not.
Praction of Average Distribtton is trobust 12-200
Fraction of Aucrago Diat is T-robust Sr- 1001
2.5
0.2
5.2
01
0.4
suppon
Figure 10: The 7-robust part of the distribution for varying T (see Remark Left is k = 20 right is
=== =0100.
D.3 INDEPENDENT VERSUS COORDINATED HISTOGRAMS
Figures and visualize the average probability _ Even 109)) of each token j E 189 across teacher distributions and the average frequency _ Ei... oi over the r = 10 samples from each of independent and coordinated ensembles. This demonstrates the property in Claim that the expected number of votes for each token is the same for the two ensemble types and corresponds to the average distribution. The qualitative difference between coordinated and independent ensembles (see Claim(2) is visualized in Figure which zooms on individual sampled histograms, showing one for independent sampling and two for coordinated sampling.
With independent sampling, frequency counts of each token are concentrated close to the expectation Σ, P3 (5) and are similar across different samples and to the averages shown in Figures and [2] With coordinated ensembles there is high variability in the shape of different samples and it is possible for the frequency of a token to far exceed the average value Σ, pj.
D.4 VISUALIZED HISTOGRAMS OF TRANSFERRED MASS
Figures 14 and 15 visualize the histograms of the covered votes (averaged over the samples) per token, for varying thresholds For each T we list coverage and support size. We can see that independent ensembles become ineffective with very low 1', when 1 l'/n exceeds the maximum average frequency of a token (0.14 with k 20 and 0.03 with k = 100), and transfer support-size is effectively limited to tokens with frequency at least T/n. In particular, no generalization (shown in blue) is transferred. In contrast, coordinated ensembles are effective also when 7 0.2m and transfer larger support size.
21

---

Published as a conference paper at ICLR 2026
Figure 11: h: = 20: For all tokens (tokens in Choow in read): Average probability over teachers (left). Average frequency of r 1000 samples using independent (middle) and coordinated (right) ensembles.
Figure 12: A: = 100: For all tokens (tokens in C shown in read): For all tokens (tokens C shown read): Average probability over teachers (left). Average frequency of 1000 samples using independent (middle) and coordinated (right) ensembles.
E DP AGGREGATION METHODS
We propose two meta aggregation schemes that are parametrized by and allow for additive error of L in the counts. In Appendix E.1 for the homogeneous ensembles regime (r > n//2), propose LARGMAX variation of TARGMAX. In Appendix [E.2] we propose TWS noisy version TWS which applies with т 2 4T.. We establish the diversity preservation properties per Definition | and show they can instantiated to be (e, 8)P with L = 1og(1(11
E. HOMOGENEOUS ENSEMBLES
Algorithm 3: LARGMAX, Aggregator
Input: c H4o0
Output: j E VV (<fail>}
(j,2,) - NoisyArgMaxx(c)
/1 notay maximizer with additive error at most La
maxa C LS 8, s
2 > (n/2 L) then return j else return <fail>
The LARGMAX, aggregator, a version of TARGMAX that allows for noisy histograms, is described in Algo- rithm 3 It is specified in terms of an operator Noi NoisyArgMAx, that inputs histogram and outputs (j. 2) such that max; Cs L 5 ? S cj L.
Observe that when 6 > (n/2 + L) holds that с > n/2 and therefore j arg max Ou that is, is true maximizer. Moreover, if the true maximizer satisties max; 0j > n/2 + L. it is the output of LARGMAXL.
We show that LARGMAX, is diversity preserving:
Lemma 3 pDivevsitty of LARGMAX (Algorithm) 3d. For T. <, n/30. The ensemble sampler M where I.RRMMXX, (Algorithm 3) is diversity preserving (as in Definition 7 with 7 0.6n, $ = ((1) and 7 2.
Proof. Using the same argument as in the proof of Theorem 2 a token ji can be returned only when õ3 > n/2+ + L = Cg n/2. Therefore = 2.
Consider token j with support > 0.6n for probability From Lemma 1 with p 18/17, Pricy 2 (17/30)ml 0.5 ln5ll(11/7))) Since cj n/2 + 2L the output. We obtain 3 = 0.5 0.611(18/17) (1))
êg > n/2 L. in this case, the token j
DP instantiations of Nois MoisyArgMaxi Noisy maximizer aggregation is well studied in differential privacy (McSherry and Talwar 2007 Durfee and Rogers 2019 Qiao et al. 2021). Generally, methods vary with the choice of noise distribution and there is a (high probability) additive error bound L that depends on the privacy parameters and in some cases also the support size and confidence. Concretely, by adding truncated
noise (c.g., truncated geometric (Desfontaines al. 2022)) each count, obtain I. == E ()g()))) with
22

---

Published as a conference paper at ICLR 2026
Figure 13: Frequency counts per token in individual sampled histograms. Left: Independent ensemble. Middle and Right: Coordinated ensemble. Top k = 20 bottom k = 100.
T' 100
voverage: con: 94% ind: 75%
support: 417 ind: 24
T 200
coverage: con: 91% ind: 63%
support: 365 ind:
T = 500
coverage: coo: 83% ind: 34% support: 277 ind:
1000
coverage: coo: 72% ind: 25% support:
2000
voverage: con: 57% ind: 0% support:
5000
voverage: con: 30% ind: 0%
support:
Figure 14: Coverage histograms averaged over r
103 samples.
Filter
T
[100, 200, 500, 1000, 2000, 5000). k = 20. Left: Coordinated. Right: Independent.
(e,6) 6)DD. Combining this with Lemma we obtain ensemble sampler with the following privacy and diversity preservation guarantees:
Corollary 2 (Propertics of DPARGMAN,.a)) Let 8, > O be such that e e log(1/)) < n/30. Let A = DPARGMAX(C.5) be the aggregator LARGMAX.. (Algorithm instantiated with an (E, DDP No Not srissArrgax,, (e.g. truncated geometrics and I. = e log(1/8)).
Then the ensemble sampler Mo is (e, 8)-DP and diversity preserving (as in Definition 7 with (7 0.6n, B (11,, 2).
The two most common noise distributions for DP are Gaussian and Laplace noise. (Cold) PATE was studied with both. The canussia-nnise based Confident-GNND aggregator (Paperno et al. 2018 Duan et al. 2023) empirically outperformed the Laplace-based LNMAX (Papernot et al. 2017) on Cold PATE. The advantages of Gaussian noise concentration (less noise to separate maximizer from low frequency tokens) and efficient composition. and benefit from sparsity of the histogram (with approximate DP), consideration as the key space tokens or strings of token can be quite large, there is an optimized mechanism with weighted sampling. Both benefit from data dependent privacy analysis that benefits from consistently large maximum counts or large margins using tools such as (Cohen and Lyu 2023). Our privacy analysis in Section I uses data-dependent I Laplace-basee approach.
effective data dependent privacy analysis. Laplace-based noise the other hand
23

---

Published as a conference paper at ICLR 2026
T- 50
coverage: 90% ind: 65%
support: coo: 638 ind: 77
T= = 100
coverage: 77% ind: 42%
support: coo: 576 ind: 34
T 200
coverage: 59% ind: 23%
support: coo: 509 ind: II
T 300
coverage: 52% ind:
support: coo: 468 ind:
T 400
coverage: 47% ind:
support: coo: 441 ind: O
T' 1000
voverage: cou:
support: coo: 347 ind: O
T'= 2000
voverage: cou:
support: coo: 234 ind: 0
T' 5000
coverage: coo:
support: coo: 102 ind:
Figure 15: Average of
10a sampled histograms for filter T
(50, 100, 200, 300, 400, 1000, 2000, 5000).
= 100 Left: coordinated Right: Independent
E.2 HETEROGENEOUS ENSEMBLES
Algorithm 4: LWS, Aggregator
Input: Heoo
Output: j E V U (<fail>>]
Weighted sampling of coken from e S + Select+(S) // S" C contains all tokena with 2 2L and aubset of tokena with
S sample j E V with probability St
e, <
S
@ then return <fail> else return uniform at random token from S
The LWS; aggregator, a relaxed weighted scheme version of TWS is descrihed in Algorithm It is specified in terms of Seleck select r. operators that inputs subset of indices S. retains all those with histogram counts 2 27. and possibly removes each token with 1 S çj 2L. The aggregator first obtains (non privacy preserving) weighted sample by independently including each token j with probability e, c//n. We then apply Select,. to obtain S* C S. Finally, we return a random token from S* or <fail> is 5 empty.
Temma (Diversity-preseraate of LWS (Algorithm 4). For T. 2 1, the ensemble sampler M where & diversity preserving in the sense of Definition with 7 AI, B @(1), and y 1.
24

---

Published as a conference paper at ICLR 2026
Prouf. A token j be included in 5" and hence be the ouiput with probability most ey/n. Hence, (using the same argument as in the proof of Theorem 2.
As for the diversity preservation property, consider a token j with support m 2 T= 4L for probability q. From
Lemma [ Pr(cz 2 mn/2 2 2L] 2 (1/2) log(2)g. In this case, Pr(j E S] 2 m/(2n) and since 2 2L, Príj E S'] 2 (2ne)). Now observer that conditioned on j E S. Pr[[S] S 2] 2 1/2. That is, the probability that there is most additional item in the sample is least 1/2. In this case, J the output with probability
1/2. So overall, m 2 7, the probability that is the output is at least mn (1/2) log(2)g. We therefore get
8n
3 = 12)16).
DP implementations of Sel Selecty, are discussed in Appendix G For concreteness, the privacy privacy-preserving preserving weighted sampling method of Cohen et al. (2021) gives (e, 6)DP with L = = log(1/0). Combining this with
Lemma we obtain an ensemble sampler with the following privacy and diversity preservation guarantees:
Corollary 3 (Properties of (.,))) For é > 0 define the aggregator A = (P,S to be LWSz instantiated with (e, 8DD Select with L = E 10=====oo1(0 log(1/5).
Then the ensemble sampler MX is (c, (DPP and diversity preserving (as in Definition 7 with (T =
4s 1og(1/6), @
F PRIVACY ANALYSIS CONSIDERATIONS
When performing DP sequential text generation need to consider composition over steps.
In this section (homogeneous ensembles) and Appendix G (heterogeneous ensembles) we explore data- dependent privacy analysis that allow for many more queries to be performed for the same privacy budget, compared with naive use of DP composition. We can avoid privacy loss on responses that agree with the prior distribution of the public model with public prompt. We can benefit from the particular structure of histograms generated by coordinated ensembles. The privacy loss does not depend queries with no yield, with high agreement, or with agreement with a public prior. With heterogeneous ensembles we can also gain from individualized per-teacher privacy charging.
We explore the benefits of data-dependent privacy analysis when the aggregation follows Algorithm 3 (homoge- neous ensembles). The utility depends on the number of queries with yield (token returned) that can be returned for given privacy budget. We synthetically generated teacher distributions with varying size common component (that can be arbitrarily diverse) and distinct (private) components.
Broadly speaking, with data- deneedeeennn analysis, we incur privacy loss on "bordcrline" queries where the output of the DP aggregation has two or more likely outputs. Queries that return a particular token with high probability or return <fail> with high probability incur little privacy loss.
We demonstrate that with Algorithm we can expect that only a small fraction of frequency histograms generated by coordinated ensembles are "borderline." (i) For queries with high yield (high probability of returning a token over the sampling of the shared randomness), the generated histograms tend to have dominant token (and thus lower privacy loss). This because coordinated ensembles tend to "brcak ties" between tokens. (ii) For queries with low yield (high probability of <fail> response and low probability of returning token), the total privacy loss only depends on yield responses. This means that high <fail> probabiltty does not cause performance to deteriorate.
This is important because both these regimes are likely in sequential text generation and with coordinated ensembles. We expect many of the tokens to follow the base model distribution and therefore have high agreement and not incur privacy loss. Or alternatively, instructions that require private data have no agreement and return < 1>. The dependent privacy analysis means that generally we can process many more queries for the privacy budget than if we had just used DP composition hound.
Our evaluation here uses d) differential privacy (Dwork et al. 2006):
Definition 2 ((e, Diffferrrtt Privacy). A randomized mechanism M provides (e, 8)-differential privacy if. for any two datasets D and D' differing at most one element, and for any subset of outputs S C Range[M), the following holds:
Pr(M(D) E S) < e Pr(M(D') E S) 6
Concretely we consider No HoisyArgMax using (Cohen et al. 2021 F with the maximum sanitized frequeney, with privacy parameters (Eo, 6o). For privacy analysis across queries we applied the Target Charging Technique
(TCT) of Cohen and Lyu (2023) with the houndary-wrappe method. The wrapper modifies slightly the output
We mention the related (non optimized) prenestypres methods (Bun et al. 2019 Korolova et al. Vadhan (017) and optimized but not sparsity- preservvnng (Ghosh al. 2012).
2009
25

---

Published as a conference paper at ICLR 2026
cutcomes for overoll privary c=1
cummi slive voue count distribtionn
J=1De
10
2- 10
of DMROOEE
04-
224
0.4
122
227
221
0.0
10
12"
20
20"
01
70
40
307
100
7400225
Figure 16: Left: Number of T responses for eo-DP queries for total e = 1 loss. Right: Cummulative maximum frequency for varying common part a.
distribution of the query algorithu (after conditioning on p!) include additional outcome T (target). The otherwise returns
wrapper returns with this probability (that depends on the response distribution) sample from the output distribution the wrapped algorithm. The probability of T is at most 1/3 and decreases with agreement (vanishes when there is response with probability closer to 1). The technique allows us to analyse the privacy loss by only counting target hits, that is, queries with T response. Since the probability of T most 1 /3, we get in expectation least useful responses per target hit. But in case agreements, we can get many more. Figure 16 (left) reports the number
(target) responses we can have with the boundary wrapper method as a function of En with overall privacy budget is e 1. When eo S 0.01, it is about (10so)
With Hot PATE, we are interested in yield responses, those that return a token (not <fail> >, and when we apply the boundary wrapper, also not T). We study how the yield probability behaves for histograms generated by coordinated ensembles.
Synthetic Teacher distributions: We parametrize the set teacher distributions by E (0, 1], which the probability a common part to all distribution. This component what we aim to transfer the student. The teacher distributions have prohability vectors of the form
p (0) a 8 (1 a)
where and (8) are probability vectors. That is, with probability a there is sample from the common distribution a, and with probability (1 - a),, there is a sample from an arbitrary distribution that is specific to each teacher. Note that the common component a can be arbitrarily diverse, that is, ||l]]] is permitted to be arbitrarily small.
When the histogram is generated by coordinated ensemble, then the distribution of the maximum frequency of a token is dominated hy sampling ~ Exp[a] and then Binje Bin[e-v-(1-o) -v-(-e) m]. It visualized in Figure 16 (right) for varying values of a. Note that across all weights a >> 0 of the shared component, no matter how small is. there is probability <fail> (no agreement) this case can 2 a. Therefore a parametrizes the probability of yield over sampling of the shared randomness.
of being above high threshold (and returning token). The probability of
sy
Tikkyt
Trke 1i 1110 inpeggoeerr
Ie spcenot
02-
Figure 17: Sweep of a, showing probabilities of outcomes: token, <fail>, [target hit).
Figure 17 shows the distribution of responses sweep broken down by T (target hit). <fai 1> (abort). and token (yield). The number of queries we process per target hit, which is the inverse of the probability of
26

---

Published as a conference paper at ICLR 2026
T. & eon. is lowest T/n and is very high for small and large at, meaning that the privacy cost per query is very small.
The yield (probability of returning a token) per query is Re a. Note that as a decreases, both yield and target probabilities decrease but their ratio remains the same: In the regime a S T/n, the yield per target hit is R son/2. Queries with a >>> T/n are essentially free in that the yield (token) probability is very high and the T (target hit) probability is very low.
When using n = Cs/Eu (Cs 2 log(1/))) teachers and plugging this in, we obtain that we get 전 0.005 J. n² yields for overall privacy budget E = 1. This means that pay only for yield and not for queries. Note that this holds in the "worst case" across all values, but the number of yields can be much higher when queries have large a (and "yields" do not incur privacy loss).
G DP METHODS FOR HETEROGENEOUS ENSEMBLES
We propose two DP methods to implement Algorithm 4 [Section E.2) with different trade off'. In both cases we can apply data- detendent privacy analysis so that queries that do not yield a token (that is, return <fail> 1>) are essentially "free" in terms of the privacy loss. The parameter J. depends on the privacy parameters (and logarithmically on [V].
Importantly, with the second method we can apply privacy analysis with individual charging, where instead of charging the whole ensemble as unit we only charge teachers that contributed to a response. With heterogeneous ensembles we expect the diversity to arise both from individual distributions and from differences between teachers and therefore with individual charging allows for much more efficient privacy analysis when different groups of teachers support each prediction.
Private Weighted Sampling This method gains from sparsity (histogram support being much smaller than IVD but the calculation of privacy loss is for the whole ensemble. We can do the analysis in the TCT framework (Cohen and Lyu 2023) so that privacy loss only depends on yield queries (those that return token). We perform weighted samphing mnnging by frequency of each token to obtain the sampled histogram c' and then sanitize the frequencies of sampled tokens using the end-to-end aparsity-preservin
method Cohen et al. (202T) obtain c". The sanitizing prunes out some tokens from e' with probability that depends on the Irequency ey- privacy parameters, and sampling rate. All tokens in e' with frequency above 2L, where L only depends on the privacy parameters, remain in e' The final step is to return token from e* selected uniformly at random or to return <fail> > if c' is empty.
Individual Privacy Charging This method does not exploit sparsity, but benefits from individual privacy charging (Kaplan et al. [2021] |202]|Coeen Cohen and Lyu 2023). It appropriate when 2L & n. The queries are formulated as counting queries over the set of teachers. The algorithm maintain a per-teacher count of the number of counting queries it "impacted," A teacher is removed from the ensemble when this limit is reached. Our queries are formed such that at most O(2L) teachers (instead of the whole ensemble) get "charged" for each query that yields a token.
To express Algorithm via counting queries we as follows: We sample a sampling rate v ~ U[L/n, 1] of teachers and sample a token v E V uniformly. We sample the teachers so that each one is included with probability and count the number c' of sampled teachers with y,
We then do TetweenTTreehor test on c (using (Cohen and Lyu 2023) which improves over Bun al. (2017) to check if c' 2L. For "above" or "between" outcomes we report v. If a "between" outcome we increment the loss counter all sampled teachers with yı = v (about 2L of them). We note that this process can be implemented efficiently and does not require explicitly performing this "blind" search.
Teachers that reach their charge limit get removed from the ensemble. The uniform sampling of the sampling rate and token emulates weighted sampling, where the probability that token gets selected is proportional to its frequency. The sub-sampling
teachers ensures that we only charge the sampled teachers. Teachers are charged only when the query is at the "between" regime so (with high probability) at most Ro 2L teachers are charged. Because we don't 't benefit from sparsity, there is overhead factor of log((V (nn/V(nn/ (n/L)) L)) in the privacy parameter (to bound the error of this number of queries) but gain factor of n/ L by not charging the full ensemble for each query in the heterogencous case where most teachers have different "solutions" to contribute.
We note that the method also produces sanitized (noised) frequency values oj for tokens in e² such that
lej - c5l S T.. And hence can also be used for Noi syArgMax
27


Reviews:
[{"review_id": "lqKYE660nl", "reviewer": "Reviewer_P4kw", "summary": "The paper introduces Hot PATE, a variant of the Private Aggregation of Teacher Ensembles (PATE) method, but in the setting of text generation with large language models. Unlike standard PATE (which they term \"Cold PATE”), Hot PATE replaces independent token sampling from teachers with coordinated ensemble sampling, using shared randomness and a bottom-k transform. This construction increases agreement among teachers and uses less privacy budget. Empirically, Hot PATE achieves orders-of-magnitude higher utility per privacy budget with more diverse output supports.", "questions": "1. What is the default temperature setting you mention in Section 4?\n\n2. The motivation behind the Planet Z task is somewhat hard to understand—could you please elaborate on its purpose and how it supports the evaluation of Hot PATE?", "limitations": "", "rating": 8, "confidence": 3, "soundness": 4, "presentation": 3, "contribution": 4, "strengths": "The authors introduce Hot PATE, a novel coordinated ensemble framework that preserves diversity while operating under the same privacy budget.\n\n1. I found the idea of diversity-preserving aggregation in Section 2 and the introduction of coordinated sampling particularly instructive. Theorem 1 on the utility of this coordinated approach is also theoretically sound.\n\n2. The authors demonstrate orders-of-magnitude improvements in both utility and diversity transfer across the synthetic instruction generation and Planet Z tasks—for example, achieving 20% coverage at T = 2000 while requiring eight times less privacy budget than the baseline of independent sampling.\n\nThe following figures stand out as highlights of the paper:\n\nFigures 2 and 4: Show substantial gains in coverage and support size for coordinated ensembles.\nFigures 3 and 7: Illustrate the emergence of “peaky” histogram shapes with high maximum counts (0.6n) and large margins, supporting why coordinated ensembles use less privacy budget.", "weaknesses": "1. For API access cases, i.e., when model probabilities are not available, the paper mentions that the distribution can be approximated by resampling with the same prompt. It would be helpful to clarify whether this is also a limitation for other PATE-based methods for LLM generation? \n\n2. Not a major weakness, but Figure 2 was somewhat difficult to read. The main message (diversity of tokens) appears to be conveyed more clearly in the right panel of Figure 2, which might be sufficient to illustrate the key result?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "U5J8NudODL", "reviewer": "Reviewer_67y4", "summary": "This paper introduces Hot PATE, a variant of Private Aggregation of Teacher Ensembles (PATE) designed to support _diverse generative_ settings, where the generating distribution supports many possible outcomes (e.g. _synthetic text generation_).  The main contribution is the use of _coordinated ensembles_ in the context of PATE, and a formal definition of diversity preserving aggregation. Coordinated ensembles is a method to improve agreement over the teachers. Rigorous privacy bounds of the mechanism are provided and experiments are presented in order to validate the claims.", "questions": "1. I am not sure that I understand failure. Can you confirm that failure decision is made after noising the counts so that $\\perp$ is just another DP output? If not, failure could reveal private information, for example, that the supports of the teachers are disjoints. Could you also specify whether the number of retries is hidden, fixed, or DP-accounted?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The presentation is quite clear, and I think that the formalization of what it means to transfer diversity for aggregation constitutes a big part of the contribution.\n2. The use of coordinated ensembles, to my knowledge, has not been studied in such settings before, making this work original.\n3. The empirical validation is convincing and covers multiple scenarios.\n4. The new PATE aggregator leaves the privacy analysis of original PATE unchanged: changing one distribution as teacher changes one item of the resulting histogram.", "weaknesses": "1. The empirical sections compare independent vs coordinated ensembles. However, Appendix A lists prior PATE adaptations (Tian et al. 2022, Tang et al. 2022) that \"limited diversity\". A fair SOTA evaluation should include these baselines and compare them to the proposed methods.\n2. There is no privacy accounting in the paper. Except in Appendix F, the budgets $(\\epsilon, \\delta)$ are never explicit. As a result, the effect of the privacy budget on empirical results (high-privacy vs low-privacy regimes) is unclear. My understanding is that the experiments use $T$ as a proxy for privacy, but $T$ is not a direct substitute for reporting actual $(\\epsilon,\\delta)$. Therefore, claims of \"orders-of-magnitude improvements in utility per privacy budget\" based solely on $T$ may not be rigorous enough. This paper should report privacy budgets.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "QFWYU48Akn", "reviewer": "Reviewer_hj6r", "summary": "This paper proposes a new variant of the PATE framework designed for generative tasks with diverse outputs, such as text generation using large language models. Traditional PATE methods, referred to as “Cold PATE,” struggle in such settings because diversity among teacher models reduces agreement, leading to poor utility under the same privacy constraints. To address this, the authors introduce “Hot PATE,” which employs a coordinated ensemble sampling mechanism that uses shared randomness across teachers to positively correlate their outputs, thereby preserving diversity while improving consensus. The paper formally defines diversity-preserving aggregation, proves that the coordinated approach maintains identical differential privacy guarantees as standard PATE, and demonstrates substantial empirical gains in utility—achieving up to an order-of-magnitude improvement on both natural and curated in-context learning tasks. Overall, Hot PATE extends privacy-preserving learning to diverse generative domains, balancing privacy, utility, and output diversity effectively.", "questions": "1. How sensitive is the utility gain of Hot PATE to the choice of τ and ensemble size n in heterogeneous teacher scenarios?\n2. Could the proposed coordinated sampling approach be efficiently implemented with current LLM APIs without excessive overhead?\n3. How would Hot PATE perform under stricter privacy budgets (e.g., smaller ε) or when extended to multimodal generative tasks such as image or code generation?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The paper introduces a clear and original contribution to the PATE framework by redefining it for generative and diverse-output settings through the concept of “Hot PATE.” The proposed coordinated ensemble sampling is both conceptually elegant and theoretically sound, offering a provable way to preserve diversity while maintaining differential privacy guarantees. The formalization of diversity-preserving aggregation and the demonstration that coordinated ensembles achieve higher utility without additional privacy cost show strong theoretical depth. Empirical evaluations are thoughtfully designed, illustrating consistent, significant gains in utility and diversity across different tasks. The writing is clear, structured, and connects theory and practice effectively.", "weaknesses": "While the paper is strong in theory, the experimental evaluation is somewhat limited in scope—focused mainly on synthetic or simplified text-generation tasks rather than more complex real-world applications. The computational cost and practical constraints of implementing coordinated sampling with proprietary APIs (e.g., repeated sampling requirements) are only briefly discussed. Sensitivity analyses for parameters such as the robustness threshold (τ) and ensemble heterogeneity are limited, and comparisons with other recent privacy-preserving generative frameworks (e.g., semantic aggregation, top-k filtering) could be expanded.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "hmhd9uVjtL", "reviewer": "Reviewer_sBN4", "summary": "This paper introduces Hot PATE, a PATE variant for diverse generative tasks that transfers diversity without extra privacy cost. The theoretical guarantees and empirical results show that Hot PATE can achieve better privacy-utility trade-off for in-context learning.", "questions": "- What is the computational cost of Hot PATE compared to Cold PATE?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "- The motivation is clear: existing PATE methods lose utility in high-diversity generative tasks due to low teacher agreement.\n- The paper introduces an interesting and novel idea by extending the PATE framework to generative settings through coordinated ensembles.\n- This paper provides solid theoretical analysis.", "weaknesses": "- The paper does not report in-context learning results on concrete downstream tasks. It’s unclear how much Hot PATE improves ICL performance on real end tasks.\n- I recommend the authors expand the related work section to include recent studies on differentially private in-context learning [1, 2, 3], and explain how Hot PATE connects to and differs from these methods, so that the paper’s contribution is better positioned within the existing DP-ICL literature.\n\n[1] Hong, Junyuan, et al. \"DP-OPT: Make Large Language Model Your Privacy-Preserving Prompt Engineer.\" The Twelfth International Conference on Learning Representations.\n\n[2] Gao, Fengyu, et al. \"Data-adaptive Differentially Private Prompt Synthesis for In-Context Learning.\" The Thirteenth International Conference on Learning Representations.\n\n[3] Yamasaki, Yusuke, et al. \"Plausible Token Amplification for Improving Accuracy of Differentially Private In-Context Learning Based on Implicit Bayesian Inference.\" Forty-second International Conference on Machine Learning.\n\nMinor issues:\n\n- The empirical setup does not specify DP parameters $\\epsilon$ and $\\delta$.\n- Missing figure references on Lines 694 and 696.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "P8KLu9ekSu", "reviewer": "Reviewer_TZnG", "summary": "This paper proposed a new coordinated ensemble method to increase generated sample diversity in the Private Aggregation of Teacher Ensembles (PATE) framework. Comparing with simple majority voting, the proposed Hot PATE successfully increases the diversity of the ensemble output.", "questions": "1.\tReference failure around line 694 and 694.\n2.     What's the core difference between coordinated sampling and the proposed coordinated ensemble? Is it just a direct application?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1.\tThe proposed CoordinatedSamples are simple for integration with current PATE framework.\n2.\tMultiple utility measurements are considered including average yield per sample, coverage, etc.", "weaknesses": "1.\tThe paper mentioned that their proposed Hot PATE satisfies ($\\epsilon, \\delta$)-DP but did not give a clear way for determining the exact value of $\\epsilon$ and $\\delta$. Since the value of these hyper-parameters are vital for the privacy preserving level, it’s important to know how the hyper-parameters of Hot PATE should be set to satisfy a required privacy requirement.\n2.\tThe baseline of this work is just the simple majority voting. As the paper mentioned in the related work, there exists other similar works DP-ICL. Although the author mentioned that this is a concurrent work, given that they are already published, I think it should be compared.\n3.\tBoth evaluation settings are relatively simple (synthetic instructions, toy planet-number example). I would like to see evaluation on complex open-ended generative tasks (e.g., summarization, dialogue, creative writing).\n4.\tThis method can only work with white-box models that opens access for per-token prediction probability. The generalization to closed-source black-box models remains unknown. I am curious, do you have plans or solutions for closed-source teacher settings? \n5.\tCoordinated sampling can require repeated sampling or shared randomness that may be impractical for large vocabularies. Do you have any computational cost analysis?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)