Contrastive Private Data Synthesis via Weighted Multi-PLM Fusion
Authors: Tianyuan Zou
Yang Liu
Peng Li
Yufei Xiong
Jianqing Zhang
Jingjing Liu
Xiaozhou Ye
Ye Ouyang
Ya-Qin Zhang
Venue: ICML 2025 poster (2025)
Keywords: Differentially Private Synthetic Dataset
Collaboration between Private Data and Private Model
Fusion of Pre-trained Language Model
Primary area: social_aspects->privacy

Abstract: Substantial quantity and high quality are the golden rules of making a good training dataset with sample privacy protection equally important. Generating synthetic samples that resemble high-quality private data while ensuring Differential Privacy (DP), a formal privacy guarantee, promises scalability and practicality. However, existing methods relying on pre-trained models for data synthesis often struggle in data-deficient scenarios, suffering from limited sample size, inevitable generation noise and existing pre-trained model bias. To address these challenges, we propose a novel contr**A**stive private data **S**ynthesis via **W**eighted multiple **P**re-trained generative models framework, named as **WASP**. WASP utilizes limited private samples for more accurate private data distribution estimation via a Top-*Q* voting mechanism, and leverages low-quality synthetic samples for contrastive generation via collaboration among dynamically weighted multiple pre-trained models. Extensive experiments on 6 well-developed datasets with 6 open-source and 3 closed-source PLMs demonstrate the superiority of WASP in improving model performance over diverse downstream tasks. Code is available at https://github.com/LindaLydia/WASP.

Contrastive Private Data Synthesis via Weighted Multi-PLM Fusion
Tianyuan Zou't
Yang Liu 23 Peng Li
Yufei Xiong Jianqing Zhang² Jingjing Liu' Xiaozhou Ye
Ye Ouyang
Ya-Qin Zhang'
Abstract
dance of high-quality training data (Budach et al., 2022; Wang et al., 2024), of which only limited amount of sam- ples can be harnessed in practice. To further complicate the issue, broad tasks across disciplines such as medical record summarization (Rumshisky et al., 2016), chatbots for personalized weight loss (Chew, 2022) and instruction- following LLM fine-tuning (Yu et al., 2024) all rely on high-quality private data collected from real users, which inevitably incurs non-negligible privacy issues.
Substantial quantity and high quality are the golden rules of making a good training dataset with sample privacy protection equally impor- tant. Generating synthetic samples that resemble high-quality private data while ensuring Differ- ential Privacy (DP), a formal privacy guarantee, promises scalability and practicality. However, existing methods relying on pre-trained models for data synthesis often struggle in data-deficient scenarios, suffering from limited sample size, in- evitable generation noise and existing pre- pre-trained model bias. To address these challenges, we pro- pose a novel contrAstive private data Synthesis via Weighted multiple Pre-trained language mod- els (PLM) framework, named as WASP. WASP utilizes limited private samples for more accurate private data distribution estimation via Top-Q voting mechanism, and leverages low-quality syn- thetic samples for contrastive generation via col- laboration among dynamically weighted multiple pre-trained models. Extensive experiments on 6 well-developed datasets with 6 open-source and 3 closed-source PLMs demonstrate the superiority of WASP in improving model performance over diverse downstream tasks. Code is available at https: /gg thub.
Differentially private synthetic data stands in as a promis- ing remedy (Bommasani et al., 2019; Putta et al., 2023; Flemings & Annavaram, 2024), by creating a new synthetic dataset that resembles the real private dataset while preserv- ing the privacy of each sample via guaranteeing Differential Privacy (DP) (Dwork, 2006). There are two main lines of research for generating DP synthetic datasets. The first line of works (Mattern et al., 2022; Yue et al., 2023) intro- duce DP Fine-tune Generator which involves fine- tin-ttnnig a Pre-trained Language Model (PLM) using DP-SGD (Abadi et al., 2016). However, this practice is computationally inten- sive and requires substantial data for effective fine-tuning, Another line of work, Private Evolution (PE) (Lin et al., 2024a; Xie et al., 2024; Hou et al., 2024; Lin et al., 2025), relieves the fine-tuning requirement and instead merely uses the APIs of pre-trained models for generation, under DP- protected guidance from private samples. This API-based nature is efficient in creating DP synthetic data, and can leverage both open-source and closed-source pre-trained models, making PE more applicable solution compared to its counterparts.
1. Introduction
Although proven effective, current PE methods (Lin et al., 2024a; Xie et al., 2024; Hou et al., 2024; Lin et al., 2025), still face three major challenges: (1) Limited Private Sam- ples. Existing PE methods rely on at least thousands of private samples (Lin al., 2024a; Xie et al., 2024; Hou et al., 2024) to guarantee reliable generation feedback se- lection. In practice, however, data sources may provide only a few hundred samples (Zdrazil et al., 2024; Ren et al., 2019), leading to noisy selection guidance. As shown in Fig- ure (a)), with limited private samples (100), Aug-PE (Xie et al., 2024) (PE for text) failed to generate synthetic samples resembling real samples' distribution for 3 PLMs (except Flan-T5). Similar conclusion is drawn in Lin et al. (2024a)
In the rapidly evolving landscape of Al models and AI agents, the strength of both Large Language Models (LLMs) and Small spsc-ipe Models (STMs) hinges the abun-
'Inttitute for AI Industry Research (AIR), Tsinghua University, Beijing, China the Hong Kong Polytechnic University, HongKong, China 'Shanghai Artificial Intelligence Laboratory, Shanghai, China "the Department of Mathematics, Harbin Institute of Tech- nology, Weihai, Shandong, China 'Shanghai Jiao Tong University, Shanghai, China Asiilnfo Technologics, Shanghai, China. Corre- spondence to: Yang Liu <yang-veroninn veronica. liu polyu.edu.kk
Proceedings of the 4nnd International Conference Machine Learning. Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

Contrastive Private Data Synthesis via Weighted Multi-PLM, Fusion
Liama-2 2
Vicuna
1000- Rllt-aatin
Aug P2 iES. 771
Refine (54.
OFT:
Refins 280
ChanSuM
ChatSU
Flan-T5
Faant5
ChatGLM3
lteration
Iteration Flan-T5
Openeeven- Seeveveen-Ceeggoy
GPT2
GP12
45
50
54
30
Aug PE (88 00) Refine (29
Lame22
Danking
3amm2
PE
Vianz
Visnu
Chansux
CHANGUN
Iteration
Iteration
32
32
35
12
70
80 14
80
(a)
Figure 1. (a) Comparison of the similarity of synthetic dataset to real private dataset (measured by FID (Heusel et al., 2017)) and STM performance (numbers within parenthesis) of Aug-PE (Xie et al., 2024) (dotted lines) and our refinement with Top-Q voting (dashed lines) under the same DP setting as in Table with IMDb dataset. Lower FID indicates higher similarity. (b) The performance (accuracy) of the STM trained using the generated synthetic dataset of Aug-PE using 100 private samples and under the same DP setting in Table 1.
(see Table 2 therein). This calls for a more precise guid- ance from limited private samples. (2) Noisy Synthetic Data. Although PE approaches encourage the generation of high-quality samples that are close to real private sample distribution, low-quality noisy samples are still unavoidable (see examples in Table 9 in Appendix C. which hinder the final performance when training downstream models (Ye et al., 2022; Gao et al., 2023; Zou et al., 2024). This high- lights the the importance of instructing the avoidance of generating noisy samples during data synthesis. (3) Risky PLM Selection. As shown in Figure (a)), different PL.Ms yield varying performances (some with unsatisfactory re- sults), and even the best performing model differs across various downstream tasks (see Figure 1(b)), making it hard to select the most suitable pre-trained model for specific task. Previous PE works primarily focus on single PLM setting, thus the potential of collaboration among multiple PLMs is still unexplored.
fashion, the WASP framework can generate large quantity of synthetic data that better approximate real private data distribution while observing differential privacy. Notably, this process incurs additional API queries compared to its single-PHI PLMM counterparts.
Our contributions are summarized as follows:
1) We introduce a privacy-preserving collaborative frame- work WASP to facilitate collaboration between multiple PLMs and private samples, especially benefiting scenarios with limited private data.
2) Our proposed WASP leverages differentially private Top- Q voting to improve the estimation of private distributions using limited private samples. It generates higher-quality data by contrasting high- and low-quality samples and dy- namically assigns importance weights to PLMs, ensuring that more capable PLMs of the specific task are prioritized.
3) Experiments on 7 well-defined natural language process- ing tasks with 6 open-source and 3 closed-souree PLMs demonstrate the consistent superiority of WASP over exist- ing methods, especially for challenging tasks.
To address these demanding challenges, we propose WASP, collaborative framework that fuses the knowledge from weighted multiple PLMs to synthesize DP data in a con- trastive fashion. (1) To overcome private sample scarcity, we first extend the voting mechanism for private distribution estimation used in PE from Top-1 to Top-Q (Q 1) with decaying weights, in order to get a more accurate estimation while guaranteeing private data DP. (2) To reduce noise, we further leverage the previous voting results to select both high-quality and low-quality samples, and incorpo- rate a contrastive prompt containing both types of samples to improve synthetic data quality by encouraging genera- tion that is more aligned to high-quality samples and less similar to low-quality ones. Notice that under multi-PLMM setting, these samples may originate from different PLMs. (3) To mitigate model bias, we then interfuse the capabilities of multiple PLMs with dynamically adjusted importance weight for each PLM based on the ensemble votes from pri- vate samples. The underlying principle is to assign higher weights to PLMs that generate synthetic samples that are closer to real samples on average. Operating in an iterative
2. Related Work
DP Synthetic Dataset. The goal of generating DP syn- thetic data is to mimic private dataset while protecting sen- sitive information. Although fine-tuning a PLM with DP- SGD (Abadi et al., 2016) for data generation purpose can be effective (Bommasani et al., 2019; Putta et al., 2023; Flemings & Annavaram, 2024; Mattern et al., 2022; Yue et al., 2023), it is computationally intensive and requires a large number of high-quality private samples to reach strong performance. Moreover, many state-of-the-art PLMs such as GPT series (OpenAI, 2021; 2023; Hurst et al., 2024) are also closed- making DP fine-tuning impractical.
A new line of work instead relies on generative APIs of PLMs without fine-tuning, which focuses on either itera- tive data synthesis under DP guidance (Lin al., 2024a;
2

---

Contrastive Private Data Synthesis via Weighted Multi-PL. Fusion
Private (Aggregoted)
Weighted Parallel Data Synthesis
B
P1
P2 Px
Differentially
Pre- trained Language Models
Synthetic Dotoset
B2
Top-Q Voting
Private
Cross-PLM Contrastive In-context Learning
Somple Selection
Furthest Histogram
H
D
D
Taskerelated descriptive Promet
Neorest Histogram
Dataset Subrets
W1
Wx Weights
PLM Importance Weighting
Figure 2. Overview of WASP framework.
Zhao et al., 2024; Bojkovic & Loh, 2024) or creating DP replica of a given large private dataset (Nagesh et al., 2024). Given that requiring a large global dataset for synthetic data initialization (Zhao et al., 2024) is hard to obtain in most cases, Lin et (2024a) proposes a more practical solution, Private Evolution (PE), which instead uses task- related synthetic samples. In PE, private samples are used to identify their nearest synthetic counterparts under DP protection, which then guide the growth of the DP synthetic dataset. PE is proven effective across images (Lin et al., 2024a) and text (Xie et al., 2024), and is further adapted to federated private data scenarios (Hou et al., 2024). However, all these works primarily focus on using single PLM as the generation model.
Datasels. A mechanism M satisfies (c, (). if for any neighboring datasets DD D' and any output subset E of M, it holds that (Dwork, 2006):
Pr(M(D) E E S Pr(M(DD) E E] +
(1)
Note that post-processing on the output of (c, 5)-DP does not incur additional privacy loss (Dwork et al., 2014).
Gaussian Mechanism. Gaussian Mechanism (Dwork, 2006) can be applied to guarantee (c, ODDP, for any c 0, 8 E (0, 1), by adding Gaussian noise following ((0, o²) to the transmitted statistics with A / (og(1.2//) and A being the sensitivity of M (Balle & Wang, 2018).
The combination of multiple PLMs can lead to stronger model performance (Liu et al., 2024; Du et al., 2023; Wan et al., 2024a;b; Li et al., 2024; Zou et al., 2024). Some studies fine-tune target models with token-level fu- sion from PLMs as teachers during training time (Wan et al., 2024a;b), while others use majority voting (Li et al., 2024) or logits averaging (Mavromatis et al., 2024) for knowledge fusion during inference. However, data privacy challenges persist, as training or test samples are exposed to external models. To solve this, FuseGen (Zou et al., 2024) recently proposes PLM fusion in a zero-shot learning setting, utiliz- ing only model APIs to synthesize data without accessing real private samples, thereby ensuring data privacy. How- ever, by treating all PLMs equally, it overlooks the capability difference of individual PLMs over different tasks.
PLM Fusion.
4. Methodology
4.1. Problem Definition
In this paper, we aim to generate a DP synthetic dataset D = {(x,, n) of size N using a small number of pri- vate data B [Zz,. u,)) where M denotes the number of private samples, and zy. My denote the feature and label of the private sample respectively. We consider data-scarcity setting where M is typically few hundreds at most. To achieve this, we harness the collaborative power of K black- box PL.Ms {P& k= via APIs, while protecting private data by Gaussian DP. For evaluation, we use to train a Small Task-specific Model (STM) m and evaluate model perfor- mance on dataset A containing real samples that is never used during training.
More related works considering Contrastive In-context Learning are included in Appendix F.
Note that our framework can be easily extended to the sce- nario of distributed federated data where each data source possesses an insufficient amount of private data and collab- orates on private tasks with secure aggregation (Hou et al., 2024). We present the related details in Section 4.7.
3. Preliminaries
Differential Privacy (DP). If two datasets D and D' dif- fer in single entry, they referred to as Neighboring
3

---

Contrastive Private Data Synthesis via Weighted Multi-PLM Fusion
4.2. Overall Workflow of WASP
Algorithm 1 WASP
Input:
The overall workflow of WASP is depicted in Figure 2 and Algorithm 1, where four steps are taken iteratively for T times. For given task, the first iteration begins by prompt- ing each PLM Pe with a zero-shot prompt, which describes the task and category label, to generate a synthetic data subset D) of equal size N₁. These samples do not contain information about B. The collective dataset D = K Dx then voted by each private sample using a differentially private Top-Q voting mechanism to identify high-quality and low-quality synthetic samples based on their similarity
K PL.Ss {Px) with empty synthetic dataset (D₂ + W| data party with private dataset B of size M belonging to C' categories; number of in-context samples S; number of iterations T taken to obtain in total N synthetic samples; initialized PLM weights {wx 1/K) learning rate v): DP privacy parameters 6, Siter test dataset A; random initialized STM m.(n)- Output: STM m.
1: Initialize in-context feedback samples D f- 0, D <
2: Calculate Gaussian noise / log (1.25/ VT-I
3: for = 0 to T' - do
4: for k = 1 to K in parallel do
to the distribution of B. These samples are then used to create a contrastive in-context learning prompt for the next round of PLM generation. The voting results are further exploit to dynamically adjust the importance weight wy for each PLM Pk. which determines N₂ of the next generation round. The process repeats from here, expanding 'D with DP synthetic samples. After T iterations, D is used to train an STM m. For notational simplicity, we omit the iteration index t, with D accumulated over iterations. DP guarantee of WASP is given in Theorem 4.1 with proof included in Appendix D.
5:
Dx <- NeighheddynaaatddeeerrtttooC,,,, D". D¹,
[(N/T) x wx]. C).
6:
end for
7:
D uk Dr.
8:
H", HH < DP_ "rr_Privateeotin(D... Q)
9:
D", +- SampleSelection(C I1", H², 8, C).
10: {wx) +- PLMScoring(II" (Da)
11: end for
12: m < STMTraaiinggDD m(o). 1))
nism with decaying weights. This approach maximizes the use of limited private samples by giving weighted votes to the Top-Q nearest and furthest synthetic samples relative to the private sample. Specially, distance between each of the private samples (z,, Uy) E B and each synthetic sample (x;, E D if they possess the same label, i.c. y/ = uy. Using & distance as mcasurement, we have:
Theorem 4.1. WASP (Algorithm 1) guarantees DP with DP budget e.
first compute the pair-wise
4.3. Weighted Parallel Data Synthesis
In this stage (lines 4-6 in Algorithm 1), each PLM Pa gen- erates N₂ = [(N/T) x wx] synthetic samples following:
x, ~ Pk (-17())))
di d(z,, x,) 11p(z)) - (xxx)112, V = M,; (x:;8:) E Diu)]
K where {w k=1 are the weights for [Px) [] is the rounding function, N is the expected total number of syn- thetic samples to be generated, and T--) is the generation prompt. In the initial iteration, T(-) is a zero-shot prompt that deseribes the task and provides category description, with all PLMs receiving equal weights, i.e. {wy- = * j- For later iterations, T(-) is extended to a few-shot con- trastive prompt (see Section 4.6) with in-context samples selected in Section 4.4, and {w>}K< dynamically assigned based on each PLM's capability the specific task (see Sec- tion 4.5). The collective synthetic dataset D- = U DA is then sent to the private data party.
(3)
where y denotes a pre-trained sentence embedding model and Diu;) denotes the subset of D which has label that equals to uj. Next, we use each private sample (z,, Uy) B to vote for its Top-Q nearest and Top-Q furthest synthetic samples within D.uy, based on Equation (3). The indices of the synthetic samples selected by each (2,. E B are:
[y,,,...... nool + arg topQSmallest (d(z,) xi) (iNrDii))))))
ff....... h.al t arg LLrQQeess (d(z): x:) (x).).)
(4)
4.4. Differentially Private Top-Q Voting
where functions arg lopQSmallest and arg Largear return the indices of the Top-Q samples with the smallest and largest d(z,,x;) respectively, with nj.1,..., nz.Q. ff,. fy.1,...: .... denoting the index of selected samples. To utilize the relative ranking information, as well as to guarantee a controllable function sensitivity for DP protection, we assign decreasing voting weights 1, 1 I..., ze to each of the Top-Q selected samples when producing the voting histograms, Nearest Histogram H
As shown in Figure I(a), with limited real private samples, noisy estimations of the real private data distribution cause the original PE algorithm to fail in generating synthetic samples that resemble private data. Our aim is to improve distribution estimations and generation guidance in this sce- nario. To achieve this, unlike previous works (Lin et al., 2024a; Xie et al., 2024; Hou et al., 2024) that assign only vote per private sample, we propose Top-Q voting mecha-
4

---

Contrastive Private Data Synthesis via Weighted Multi-PLN Fusion
and Furthest Histogram HJ, This can be formulated as:
iteration. That is,
H" "isal= <- ( + 20-1
1 H Shal +- J1,a +
1
(
DX..ODED $
Σ EMNDEED
20-1
8:
H'
Wk
iPol
[Pal/
v(z,.uz) E E, Vg E [I,.....)] Q},
Σ
1911
(8)
(5)
with H", H each initialized as [o,..., 0] of length D).
4.6. Cross-PLM Contrastive In-context Learning (ICL)
To further guarantee (e, for private sam- ples, Gaussian noises following o0) with
Inspired by the observation that low-quality samples still exist in DP synthetic dataset given by PE (see Table 9 in Appendix C.I), we select cross-PLM contrastive samples from Dr and DS (obtained in Section 4.5), and use them to create a contrastive task-related dese--deeeee prompr T(:) to perform eross-PIM contrastive ICI.. T-)) describes the task, provides category description, and contains explicit contrastive instructions for high- and low-quality samples. It contains the following sequential instructions: (1) ana- lyze the difference between low- and high-quality samples; (2) ensure the new sample is better in quality and closer to real private distribution than the high-quality samples, and is further away from the low-quality samples than the high-quality samples; (3) generate new sample which is diverse in expression compared to the given high-quality samples. Note that to improve the generation diversity, for each generation we perform random sample selection to draw 30% of samples respectively from D and Dr.lel to construct the final in-context samples for T(c). Also, different from PE algorithms series, we choose not to vary one existed synthetic sample each time, but to encourage diverse sample generation using S demonstrations at once. Prompt examples can be found in Table 8 in Appendix A.
= 4 2 (1.25/Bite)) WTT
are added to HH:
H c HH +N0 (p)). H' + H + N(0, 0² (pp) (6)
where lady represents the identity matrix of size [D] x [D] and öaer
6 (T-1) represents the DP typer-parameter ap-
plied within each iteration.
Based on H", H', for each category C, we select low- quality samples with the highest votes in HJ (largest dis- tance to private samples in B), denoted Dr.le, alongside high-quality samples with the highest votes in H (nearest to private samples in B), denoted as Dn.lel, following:
m/..el [+] (Xess m6) e plu)) , { (x ms) e Dit).
Be-iel (x.. a1) e pid| [() among the top-S values of H H=)
Br.el
= (x)) E pidl H [i] is among the top. S values of H
(7)
where S the amount of samples to select and Hm.ee) H.lee
denote the sets of the nearest and furthest voting results of
samples belonging to category c. D = - Dd.lel, D/ US-1 D.lee are the total sets of high- and low-quality sam- ples respectively. Note that we do not limit the origin of the selected samples, and synthetic samples generated by different PLMs can all be included in Dr and DJ.
4.7. WASP in Federated Data Setting
So far we have built our algorithms under single data-party setting, which can be easily extended to federated data sce- nario (Hou et al., 2024), where each data party possesses an insufficient amount of private data and collaborates on private tasks. This scenario is very common in the real world, such as collaborations between medical companies. In this setting, we consider L data parties {G.)= each Mr j-1 of
4.5. PLM Importance Weighting
Previous studies on API-based multi-PLM fusion (Li et al., 2024; Zou et al., 2024) often treat involved PLMs equally. However, as shown in Figure 1(b) and Figure 7 in Ap- pendix E.2, different PLMs exhibit varying generation ca- pabilities,, leading to uneven synthetic data quality. This encourages assigning customized weights for each PL.M to enhance their contributions. Therefore, we introduce a PLM weighting strategy based on the quality of their gener- ated synthetic data, which is measured by their similarity to private samples.
possessing a real private dataset B; = {(Z } size Mt. These data parties aim to collaboratively generate DP synthetic dataset while preserving local data privacy. The full algorithm is provided in Algorithm 2.
When extending to federated data setting, each party Ci uses its local private samples in B3 to perform DP Top-Q vot- ing with 4 2 (1.25/6u.-)VVTI to guarantee privacy. The produced local nearest and furthest voting histograms ( ( (21)1_1 | are then securely aggregated (Bonawitz et al., 2016) before sent a central server following:
eVT.
Since the Nearest Histogram H obtained in Equation (5) quantifies the similarity between each synthetic sample and private samples, we simply aggregate the histogram values of each synthetic sample with source PLM Pk to obtain the weight wa of the PL.M P& for the upcoming generation
L
H
Hr, H'
Σ H 1=1
(9)
5

---

Contrastive Private Data Synthesis via Weighted Multi-PL.) Fusion
Table Evaluation downstream STM accuracy using PLMs, 1. Best and second best results are marked.
Yelp
Openreview Rating
Privacy
[8
[D]
IMDb
Banking
Category
Rating
Area
OnlyPrivate
100
50.00
5.69
35.57
6.56
22.20
13.75
FuscGen
Absolutely Private
6, 000
89.07
63.38
57.96
24.70
34.57
78.75
GPT-2
1.0
100
6,00 0,000
87.44
47.45
50.04
33,20
31.25
74.88
Llama-2
4.0
100
6, 000
84.63
62.14
49.95
28.23
28.45
79.75
DP-SGD +Gen
Vicuna
4.0
100
6, 000
84.93
62.99
57.46
31.17
23.48
78.75
OPT
4.0
100
6, 000
81.47
62.61
55.68
34.57
22.00
75.75
ChatGLM3
4.0
100
6, 000
83.17
52.99
45.79
34.60
33,99
84.38
Flan-T3 T5
4.0
100
6, 000
89.14
58.59
60.85
33.52
35.35
78.13
GPT-2
4.0
6, 000
85,38
62.33
45.28
31.45
24.12
75.63
Llama-2
= 4.0
100
6, 000
83.77
60.18
47.42
32.67
34.78
84.63
Vicuna
4.0
100
6, 000
82.76
63.28
54.42
32.27
30.66
86.75
Aug-PF:
OPT
4.0
100
6, 000
83.86
62.71
50.81
34.64
25.30
79.25
CharGLM3
c 4.0
6, 000
85.82
55.06
55.17
33,81
32.49
88.50
Flan-TS
4.0
100
6, 000
89.00
62.06
58.69
34.54
35.42
81,25
WASP (Ours)
4.0
100
6, 000
||
89.52
63.91
61.21
34.99
37.10
88.75
Table 2. Evaluation of downstream STM accuracy using 6 PLMs, L = 10. Best and second best results are marked.
Yelp
Openreview Rating
Privacy
[8]
|D]
IMDb
Banking
Category
Rating
Area
OnlyPrivate
oc
300
50.00
5.90
38.76
8.86
23.55
16.75
FuseGen
Absolutely Private
6, 000
89.07
63.38
57.96
24.70
34.57
78.75
GPT-2
c = 4.0
300
6, 000
85.87
62.58
46.25
37.13
24.45
76.25
Llama-2
c 4.0
300
6, 000
86.09
60.20
51.11
34.24
36.24
85.38
Vicuna
4.0
300
6,000
83.52
64.11
54.76
36.38
30.88
86.13
Pre-Text
OPT
= 4.0
300
6, 6,0
83.98
63.65
52.44
37.67
24.73
79.75
C3
= 4.0
300
6, 000
86.32
60.24
56.94
38.14
33.35
89.38
F5an-T5
4.0
300
6, 000
89.02
62.82
61.04
38.31
36.53
81.75
WASP (Ours)
4.0
300
000
89.65
64.34
61.46
40.47
37.60
89.63
We adopt an honest Conest-buttcurine but-curious threat model where the server only has access to the aggregated histograms H and H but not individual ones. We also assume that all data parties participate the aggregation and therefore aims to ensure sample-level (e, 5DD of B (see Appendix D). Note that, WASP be easily extended ensure user-level DP, with discussions and results included in Appendix E.4.
PLMs include 2021), GPT-4-uurb--prvview (GPT-4) (OpenAI, 2023), and GPT-4o (Hurst et al., 2024). For STM, we use pre-trained bert-base-un bune-ase- (BERT) model and further fine-tune on downstream classification tasks using D. We use sentence- 15-base (Ni al., 2022) the embedding model yp.
(GPT-3.5) (OpenAI,
Datasets. We evaluate on 6 widely used tasks: 1) IMDb (Maas et al., 2011) (2 categories) for movie-review
5. Experiments
semantic analysis task; 2) Yelp-Category (Inc. Yelp. 2015) (10 categories) for business-review item field classification task: 3) Yelp-Rating (Inc. Yelp. 2015) (5 categories) for business-review rating classification task; 4) Openreview- Category (Xie al., 2024) (12 categories) for paper- review classification by research area task; 5) Openreview- Rating (Xie et al., 2024) (5 categories) for paper-review classification by review rating task; and 6) Banking (10 cat- egories selected from Banking77 (Casanueva et al., 2020)) for online-banking queries field classification task. B
5.1. Settings
Models. In this work, 6 open-source PLMs and closed-
source PLMs are considered. Open-source PLMs include GPT-2-xl (GPT-2) (Radford et al., 2019), Llamm-2-7b--haat hf (Llama-2) (Touvron et al., 2023), Vicuna-7b- (Vi- cuna) (Chiang et al., 2023), OPT-6.7b (OPT) (Zhang al., 2022), ChatGLM3-66 base (ChatGLM3) (Du et al., 2022), and Flan-T5-xl (Flan-T5) (Chung et al., 2022). Close-source
6

---

Contrastive Private Data Synthesis via Weighted Multi-PLM Fusion
Table 3. Evaluation of downstream STM accuracy using 3 closed- source PI.Ms, T. with the same DP setting in Table Best and second best results are marked.
89.5
< Performance
88.5
87.5
86.5
Privare
Only
FuseGen
GPT-3-5
Aug-
GPT-4
GPT-4o
(Ours)
85.5
84.5
Yelp-Rating
35.57
61.36
60.90
61.02
62.06
64.48
83.5
2
K
m Performance
64
GPT do
65.04
80.92
62,87
63.0
(a) Effect K
64
62.5
Rlan-15
62
GFT4
62.89
61.52
89.33
89.45
89.30
89.32
89.43
89.28
62.0
CharGLM3
62
CPT
86, 96.92
86.97
86.18
86.22
86.12
01.22
2
61.0
OPT
85. 85.85
86.23
85.80
84.32
K
GPT-3.5
GPT4
GFñ-4
Vicuna
(a) Effect of K
(b) Pair-wise Comparison
85. 85.91
85.92
82.90
Llema-2
Figure 3. Evaluation of downstrcam STM accuracy using Yclp- Rating dataset with K 1, 2, closed-source PLMs, L == The same DP setting is applied as in Table 1. In (b), results on the diagnose are single-PL.n PLMM performance (K 1) and others are pair-wise performance (K- = 2).
86.72
85.82
GPT-2
85.65
82
GPT-2
Llama-2
Vicuna
OPT
ChatGLM
Flan-T
randomly drawn from the training sets of these datasets with their test sets used to evaluate trained STM.
(b) Pair-wise Comparison
Baselines. We compare the WASP framework to 5 base- lines: 1) Aug-PE (Xie et al., 2024), the original PE al- gorithm specialized for text modality; 2) Pre-Text (Hou et al., 2024), which applies PE to federated private data setting: 3) OnlyPrivate, the centralized training method re- lying merely on B without DP (c oo), which provides performance upper-bound of using no synthetic data; 4) FuseGen (Zou et al., 2024), which generates synthetic data in zero-shot manner without accessing private samples; 5) DP-SGD+Gen (Yue et al., 2023) which first fine-tunes PI.M using centralized private data under DP guarantee and then exploit the fine-tuned model for synthetic data genera- tion. "Absolutely Private" in result tables indicates that no private sample in exploit during training.
Figure 4. Evaluation of downstream STM accuracy using IMDb with K 1,..., 6 open-source PLMs, L = 1. Other settings the same as in Figure 3.
feedback, only the last 4 iterations are sensitive to privacy. By default, we use ôster x 10-5 in our experiments and list only e alongside the results. The notion of DP is sample-level DP unless otherwise stated.
5.2. Main Results
Single Data Party Setting. Experimental results using K 6 open-source PLMs and 3 closed-source PLMs provided in Tables and which show that WASP outper- forms all baseline methods across different tasks, demon- strating its superiority. As expected, the closed-source GPT series (see Table 3), being powerful models, outper- form their open-source counterparts (see Table 1) when using baseline method Aug-PE. As baseline method DP- SGD+Gen requires training the PLM, which is inapplicable closed-soures PLMs, do not compare with this base- line when using closed-source PLMs.
Implementation Details. By default, we use 100 private samples (M = 100) for main experiments. For federated data (L > 1) scenario. we use L = 10 private data parties which control 300 private samples (M E/e [B,] = 300) altogether. To better align with real-world scenarios, each participating data-party controls private datasets that are non-i.i.d. each other, and aggregate an unbalanced dataset. We follow Dirichlet Partition (Yurochkin et al., 2019; Hsu et al., 2019; Zhang et al., 2023) to distribute pri- vate samples to each party with parameter a = 1.0. For the DP synthetic dataset, we generate total of 6, 000 samples from all participating PLMs within 5 iteration. Since the first iteration does not use private sample information for
For all tasks, with limited private samples, DP-SGD+Gen and Aug-PE performs poorly when using improper single PLM, e.g. using OPT for IMDb and using ChatGLM3 for Yelp-Category. Differently, WASP performs consistently

---

Contrastive Private Data Synthesis via Weighted Multi-PLM Fusion
86-
Table Comparison of downstream STM accuracy under w/ and w/o Contrastive In-contex: Learning and Private Data Assisted PLM Importance Weighting setting using open-source PLMs, L = 1 with the same DP setting in Table 1. "wlo both" indicates that both techniques are removed with only Top-Q Voting (Q = 8) remaining "OR- "OR-Rating" is short for "Operevvie-Raatiin
52.5
50.0
844
83
47.5
45.0
10
100
300
1000
10C00
25
100
300
1000
10000
M
dataset.
Aug Plaar-
Aug Plowees
Aug Plowen
WASP (Oun)
AugePflune
AugPfoo
AugePP PP
w/o PLM Contrastive Prompting
w/o PLM Importance Weighting
WASP (Ours)
w/o both
Figure 5. Comparison of downstream STM accuracy using differ- ent number of private samples (M) from the training set IMDb and Yelp-Rating datascts using 6 open-source PLMs, L 1 with the same DP setting as in Table 1.
IMDb
89.05
89.21
89.17
89.52
Yelp-Rating
58.72
59.65
58.94
61.21
OR-Rating 35.45
36.18
35.53
37.10
Table 6. Evaluation of downstream STM accuracy using 6 open- source PLMs, L under different DP budget setting with ôster 1 x 10 The best performing PLM is used for Aug-PB evaluation, i... Flan-T5 T5 for both tasks.
Table 5. Evaluation of downstream STM accuracy using 6 open- source Pl.Ms, I. = with the same DP setting as Table under different Q.
Q==
Q Q=
Q= 4
Q =
Q 16
00
8.0
4.0
1.0
IMDh Yelp-Rating
89.02
89.15
89.39
89.52
89.60
WASP
89.96 89.48
89.77
89.52
89.36
58.74
58.92
59.24
61.21
61.42
IMDb
Aug-PE PE
89.23
89.00
88.72
Yelp-
WASP Aug-PE
62.02
61.54
61.21
60.94
Rating
59.62
59.12
58.69
58.59
well across tasks, and achieves a lower FID value compared to baselines (see Figure in Appendix E.1), verifying its effectiveness under limited private sample setting. Also, the best performing PLM varies across tasks for baselines, high- lighting the arbitrary nature of PLM selection. In contrast, WASP consistently achieves better performance across tasks, making it PLM-agnostic without requiring prior- knowlegee for selecting specific PLMs for collaboration.
5.3. Ablation Studies
# PLMs (K). We first study the impact of the number of PLMs (K) on the final STM performance. Results of using 1, 2, 3 closed-source PLMs and 1..... 6 open-source PLMS under L = 1 and the same DP setting as in Table are reported in Figures 3(a) and 4(a) respectively. We can see that the performance of m increases simultaneously with the increase of K while the randomness (STD) decreases. This indicates that the randomness in the performance of the synthetic dataset can be mitigated by incorporating more PLMs into WASP, which simultaneously increases the per- formance expectations.
On the other hand, comparing with FuseGen, a baseline under zero-shot setting where private samples are inacces- sible, WASP leverages real private samples and utilizes more targeted PLM importance weighting method, there- fore achieving better performance. Moreover, the notably poor performance of "OnlyPrivate" shows that the trained STM relying merely on private dataset B is nearly unusable, even without applying DP during training which can further degrade STM performance.
In Figure 3(b), we further display the pair-wise combination (K = 2) results of the 3 clored-source PLMs under L = 1 and the same DP setting in Table 1. In this figure, any pair- wise collaboration (K = 2) outperforms either participating single-PLM alone (diagnose in Figure 3(b)), demonstrating that WASP performs better using the whole set of avail- able PLMs than using only a subset of them. This finding is also verified by Figure 4(b) which shows results of the pair-wise combination of 6 open-source PLMs. These find- ings show that WASP's improvements are PLM-agnostic, independent of any single PLM's inherent task capabilities. Consequently, WASP effectively mitigates the risk of select- ing the optimal PI.M by harnessing the collective strengths of all participating models.
We also conduct experiments under distributed federated data setting, with L 10 and M 300 total number of private samples. As DP- SGD+Gen requires fine-tuning the PLM in data central- ized manner, we do not compare with under this setting. Results in Table show that WASP consistently achieves better performance across different tasks and settings com- pared to Pre-Text, a baseline designed for federated data. This further demonstrates the effectiveness of WASP when extended to federated data setting. Additional results on communication cost comparison is given in Table 13 in Ap- pendix E.6, where we show that the communication increase caused by uploading additional histograms by our method minimal.
Federated Data Setting.
Contrastive ICL & PLM Importance Weighting. To evaluate the effectiveness of our proposed Contrastive In-
8

---

Contrastive Private Data Synthesis via Weighted Multi-PLM Fusion
Table 7. Evaluation of downstream STM performance (FI score) using 6 open-source ML.ss, I. = 1 with natural language generation task using SQuAD dataset with the same DP setting in Table 1. "G, I., V. O, C, F" represents PL.M "GPT-2, LJama-2, Vicuna, OPT, CharGLM3, Flan-T5" respectively. Best and second best results are marked.
Only Private
DP-SGD+Gen
Aug-PE
WASP (Ours)
FuseGen
G
L
0
F
G
L
V
0
C
F
SQuAD 5.41
9.31
6.70
7.74
8.42
7.53
8.09
9.40
7.37
7.84
8.46
8.31
8.20
9.42
11.40
context Learning and PLM Importance Weighting methods, conduct ablation experiments to see how these com- ponents impact the final STM performance. Results are reported in Table 4. By removing Contrastive In-context Learning (labeled as "w/o PLM Contrastive Prompting" in Table 4), we only select high-quality samples for the prompt (see Table 8) with the noise scale halved compared to WASP to assure fair comparison since the function sensitivity A is halved (only H is calculated). This leads to a 0.31% decrease in STM performance on the easier IMDb task, and much larger 1.56% and 0.92% decrease on the more chal- lenging Yelp-Rating and Openreview-Raie Ratiing tasks. This highlights the importance of using low-quality samples as feedback demonstrations to encourage the PLMs avoid gen- erating low-quality DP synthetic samples.
outperforms Aug-PE across different values of M for all PLMs, and the performance gaps at smaller M values (M < 1000) are much greater, underscoring the effectiveness of WASP in limited private data scenarios.
Different Private Budget (e). As illustrated in Table 6, STM performance using WASP gradually declines from 89.96% to 89. 89.36% for IMDb and from 62.02% 60.94% the privacy budget decreases from
for Yelp-Rating 0o, 8.0, 4.0 to 1.0, similar to that of Aug-PE when using the best performing single PLM for each task. This in- dicates that WASP scales well with and maintains high performance even under tight privacy constraints, just like baseline method.
On the other hand, by removing PLM Importance Weighting (labeled as "w/o PLM Importance Weighting" in Table 4), W/:- = 1/K within each generation iteration, indicating that each PLM generates equal amount of samples across it- erations. Similarly, results indicate a 0.35% decrease in STM performancee on the casier IMDb task and a 2.27% and 1.57% decline on the more challenging Yelp-Rating and Openreview-Ratte tasks. This underlines the effectiveness of weighted aggregation of PLMs with varying degrees of reliance on their capabilities for specific task. Furthermore, these results demonstrate that by generating better DP syn- thetic data, WASP is more effective than baselines when faced with more challenging tasks.
5.4. Generation Task
We also verify the effectiveness of our proposed WASP for generation tasks in Table using SQuAD dataset (Rajpurkar et al.. 2016) as the evaluation dataset. Table 7 shows that the superiority of WASP remains for a more difficult question- answering task.
6. Conclusion and Future Work
In this work, we introduce a novel DP synthetic data gen- eration framework, WASP, which leverages the collabora- tive capabilities of multiple PLMs address real-world scenarios with limited private samples, while observing differential privacy. Extensive experiments across 6 tasks demonstrate that WASP is highly effective, PLM-agnostic, scalable with respect to privacy budgets, and superior in challenging scenarios, making it a practical and scalable solution for real-world applications.
# Votes (Q) by Each Private Sample. To better esti- mate private sample distribution with limited private sam- ples, WASP exploits each private sample by increasing the amount of votes each private sample gives out (from Q = 1 in previous works to Q 8). Here we investigate how the change in Q impacts STM performance. Results in Table 5 indicate that STM performance improves with higher values of Q, but the improvement diminishes at larger Q (Q > 8). This underscores the strength of our idea in increasing the utility of each private sample to achieve a more accurate pri- vate sample distribution estimation, particularly in scenarios with limited available private samples.
Possible future work points more precise sample-level weighting or selection to further improve the quality of the DP synthetic dataset, as well as verifying the effectiveness of WASP on non-classion non-classification tasks.
A Akknowledgments
Sensitive Analysis of # Private Samples (M). We also investigate the impact of M on WASP. In Figure 5, we compare the baseline Aug-PE method with WASP across different values of M. Results show that WASP consistently
This work was supported by the National Key R&D Program of China under Grant No. 20.20220 60504, the Tsinghua University AAiiioo Technologies (China) Inc Joint Research Center.

---

Contrastive Private Data Synthesis via Weighted Multi-PLM Fusion
Impact Statement
Casanueva, I., Temcinas, T., Gerz, D., Henderson, M., and Vulic, I. Efficient Intent Detection with Dual Sen- tence Encoders. In Proceedings of the 2nd Workshop on NLP for ConvAl ACL 2020. mar 2020. URL https::/ www.aaaww..oog/g/oooa/ageeeeeeae hwrgg/abs/2000 04807. Data available at http:///iiibbbcom///yyAt/DDDllld
This paper presents work whose goal is to advance differ- entially private synthetic dataset generation via PLM APIs. On the private sample side, since DP is guaranteed for the private samples used throughout the process, do not identify any specific societal consequences that need to be highlighted. However, for the PLMs, although accessed through black-box APIs, we do not provide privacy guar- antees for their training data in WASP and assume that the target task and prompts used in WASP are benign. There- fore, in real-world deployment, the privacy issues of the PLMs should be further considered.
Cheng, A., Wang, P., Zhang. X. S., and Cheng, J. Differen- tially Private Federated Learning with Local Regulariza- tion and Sparsification. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 10122-10131, 2022.
Chew, H. J. The Use of Artificial Intelligence-based Con- versational Agents (Chatbots) for Weight Loss: Scoping Review and Practical Recommendations, JMIR medical informatics, 10(4)xe32578, 2022.
References
Abadi, M., Chu, A., Goodfellow, I., McMahan, H. B., Mironov, I., Talwar, K., and Zhang, L. Deep Learn- ing with Differential Privacy. In Proceedings of the 2016 ACM SIGSAC conference on computer and communica- tions security. pp. 308-318, 2016.
Chiang, W.-L.. Li, Z., Lin, Z., Sheng, Y., Wu, Z., Zhang, H., Zheng, L.., Zhuang, S., Zhuang, Y., Gonzalez, J. E., Stoica, I., and Xing, E. P. Vicuna: An Open-Source Chatbot Impressing GPT-4 with 90%* ChatGPT Qual- ity, March 2023. URL 2023-03-20 -3-38-vicuna/.
An, C., Feng, J., Lv, K., Kong, L., Qiu, X., and Huang, X. Cont: Contrastive neural text generation. In Advances in Neural Information Processing Systems (NeurlPS), volume 35, 2022.
Chung, H. W., Hou, L.., Longpre, S., Zoph, B., Tay, Y., Fedus, W., Li, Y., Wang, X., Dehghani, M., Brahma, S., et al. Scaling Instruction-finetuee Language Models. arXiv preprint 2xXiv:2210.114111 2022.
Arora, S., Khandeparkar, H., Khodak, M., Plevrakis, O., and Saunshi, N. A theoretical analysis of contrastive unsupervised representation learning, 2019. URL https: //arxiv.org/abaeee
Du, Y., Li, S., Torralba, A., Tenenbaum, J. B., and Mor- datch, I. Improving Factuality and Reasoning in Lan- guage Models through Multiagent Debate. arXiv preprint arXiv:2305. 14325, 2023.
09229.
Balle, B. and Wang, Y.-. Improving the Gaussian Mech- anism for Differential Privacy: Analytical Calibration and Optimal Denoising. In International Conference on Machine Learning, pp. 394-403. PMLR, 2018
Du, Z., Qian, Y., Liu, X., Ding, M., Qiu, J., Yang. Z., and Tang, J. GLM: General Language Model Pretraining with Autoregressive Blank Infilling. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume Long Papers), pp. 320-335, 2022.
Bojkovic, N. and Loh, P.-.. Differentially Private Synthetic Data with Private Density Estimation. arXiv preprint arXiv:2405.04554, 2024.
Dwork, C. Differential Privacy. In International colloquium on automata, languages, and programming, pp. 1-12. Springer, 2006.
Bommasani, R., Wu, S., and Schofield, X. Towards Private Synthetic Text Generation. In NeurIPS 2019 Machine Learning with Guarantees Workshop, 2019.
Dwork, C., Roth, A., et al. The Algorithmic Foundations of Differential Privacy. Foundations and Trends@ in Theoretical Computer Science 933-4):211 4):211-4411) 1071, 2014.
Bonawitz, K., Ivanov, V., Kreuter, B., Marcedone, A., McMahan, H. B., Patel, S., Ramage. D., Segal, A., and Seth, K. Practical Secure Aggregation for Federated Learning on User-Held Data. In NIPS 2016 Workshop on Private Multi- Multi-Parry Machine Learning, 2016
Flemings, J. and Annavaram, M. Differentially Private
Knowledge Distillation via Synthetic Text Generation. arXiv preprint arXiv:2403.009:: 2024.
Gao, J., Pi, R., Yong, I., Xu, H., Ye, J., Wu, 7., Zhang, W., Liang, X., Li, Z., and Kong, L. Self-guided Noise- free Data Generation for Efficient Zero-shot Learning. In Proceedings of The Eleventh International Conference on Learning Representations, 2023.
Budach, L., Feuerpfeil, M., Ihde, N., Nathansen, A., Noack, N. S., Patzlaff, H., Harmouch, H., and Naumann, F. The Effects of Data Quality on ML-Model Performance. CoRR 14522, 2022.
10

---

Contrastive Private Data Synthesis via Weighted Multi-PLN Fusion
Gao, T., Yao, X., and Chen, D. Simese: Simple con- trastive learning of sentence embeddings. arXiv preprint N0t.:2144008221, 2021.
Lin, Z., Baltrusaitis, T., and Yekhanin, S. Differentially Pri- vate Synthetic Data via APIs 3: Using Simulators Instead of Foundation Model. arXiv preprint arXiv:2502.05505. 2025.
Gao, X. and Das, K. Customizing Language Model Re- sponses with Contrastive In-Context Learning. In Pro- ceedings of the AAAI Conference on Artificial Intelli- gence, volume 38, pp. 18039-18044 2024.
Liu, Z., Zhang, Y., Li, P., Liu, Y., and Yang, D. A Dynamic LLM--owered Agent Network for Task-Oriented Agent Collaboration. In Proceedings of the 1st Conference Language Modeling (COLM 2024)., 2024.
Heusel, M., Ramsauer, H., Unterthiner, T., Nessler, B., and Hochreiter, S. Gans Trained by Two Time-scale Update Rule Converge to a Local Nash Equilibrium. Advances in Neural Information Processing Systems, 30, 2017.
Maas, A. L., Daly, E., Pham, P. T.,, Huang, D., Ng. A. Y., and Potts, Learning Word Vectors for Sentiment Analysis. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, pp. 142-150, Portland, Oregon, USA, June 2011. Association for Computational Lin- guistics. URL http:/ / aclweb. org/anthologe P11-1015.
Hou, C., Shrivastava, A., Zhan, H., Conway, R., Le. T., Sagar, A., Fanti, G., and Lazar, D. PrE- Text: Training Language Models on Private Federated Data in the Age of LLMss. In Forty-first International Conference on Machine Learning, 2024.
Hsu, T.-M. H., Qi, H., and Brown, M. Measuring the Effects of Non-identical Data Distribution for Federated Visual Classification. arXiv preprint arXiv:1909.06333 (9339999..00...... 2019.
Mattern, J., Jin, Z., Weggenmann, B., Schoelkopf, B., and Sachan, M. Differentially Private Language Models for Secure Data Sharing. In Proceedings of the 2022 Con- ference on Empirical Methods in Natural Language Pro- cessing, pp. 1860-4877 2022.
Hurst, A., Lerer, A., Goucher, A. P., Perelman, A., Ramesh, A., Clark, A., Ostrow, A., Welihinda, A., Hayes, A., Radford, A., et al. GPT-4o System Card. arXiv preprint arXiv:2410.21276 2024.
Mavromatis, C., Karypis, P., and Karypis, G. Pack of LLMs: Model Fusion at Test-Time via Perplexity Optimization. arXiv preprint arXiv:2404. 11531, 2024.
Inc. Yelp. Yelp Open Dataset, 2015. URL https://wwe yelp. com/dataset.
Miyanishi, Y. and Nguyen, M. L. Multimodal Contrastive In In-Context Learning. arXiv preprint 240.::900..29559 2024.
Kairouz, P., Oh, S., and Viswanath, P. The composition the- orem for differential privacy. In International conference on machine learning, pp. 1376-13300 1385. PMLR, 2015.
Mo, Y., Liu, J., Yang. Wang, Q. Zhang, S., Wang, J., and Li, Z. C-ICL: Contrastive In-Context Learning for Information Extraction. arXiv preprint 2rXiv:2402.1125.. 2024.
Kato, F., Xiong, L., Takagi. S., Cao, Y., and Yoshikawa, M. ULDP-FL: Federated Learning with Across-Sin Silo User- Level Differential Privacy. In Proceedings of the VLDB Endowment. International Conference on Very Large Data Bases, volume 17, pp. 2826, 2024.
Nagesh, S., Chen, J. Y., Mishra, N., and Wagner, T. Pri- vate Text Generation by Seeding Large Language Model Prompts. In GenAl for Health: Potential, Trust and Policy Compliance, 2024.
Li, J., Zhang, Q. Yu, Y., Fu, Q. and Ye, D. More Agents is All You Need. arXiv preprint arXiv:2402.05/20. 2024.
Liang, C., Xiang, W., and Wang. B. In-context Con- trastive Learning for Event Causality Identification. arXiv preprint ar 2055 105/2, 2024.
Ni, J., Abrego, H., Constant, N., Ma, J., Hall, K., Cer, D., and Yang. Y. Sentence-T5: Scalable Sentence Encoders from Pre-trained Text-to-Text Models. In Findings of the Association for Computational Linguistics: ACL. 2022, pp. 1864-1874, 2022.
Lin, Z., Gopi, S., Kulkarni, J., Nori, H., and Yekhanin, S. Differentially Private Synthetic Data via Foundation Model APIs Images. The Twelfth International Conference on Learning Representations, 2024a.
OpenAL. GPT-3.5- 2021. URL https:://///a.n openai. wtt. wttpc////ooo/cc////////cccccc/// 3-5- turbo.
Lin, Z., Gopi, S., Kulkarni, J., Nori, H., and Yekhanin, S. Differentially private synthetic data via foundation model apis 1: Images, 2024b. URL https: arxiiv org/abs/ 2305. 15560.
OpenAL. GPT-4-Turbo and GPT-4, 2023. URL
https://platform. oww..i.eenccc.....o openai.ccom/dcc///ooooo com//oocc//omees//
gpt-4-turbo-and-gore

---

Contrastive Private Data Synthesis via Weighted Multi-PL. Fusion
Parulekar, A., Collins, L.,, Shanmugam, K., Mokhtari, A., and Shakkottai, S. Infonce loss provably learns cluster- preserving representations. In The Thirty Sixth Annual Conference on Learning Theory, pp. 1914-1961. PMLR, 2023.
Wan, E, Huang, X., Cai, D., Quan, X., Bi, W., and Shi, S. Knowledge Fusion of Large Language Mod- els. In Proceedings of The Twelfth International Con- ference on Learning Representations, 2024a. URL 2qcz.
Putta, P., Steele, A., and Ferrara, J. W. Differentially Pri- vate Conditional Text Generation For Synthetic Data Pro- duction, 2023. URL
Wan, F., Yang, Z.,, Zhong, L.,, Quan, X., Huang, X., and Bi, W. FuseChat: Knowledge Fusion of Chat Models. arXiv preprint arXiv:2402. 16107, 2024b.
id=LUq]32OFwWD.
Wang. J., Zhang, B., Du, Q., Zhang, J., and Chu, A
Radford, A.,
Wu, J., Child, R., Luan, D., Amodei, D., Sutskever, I., et al. Language Models are Unsupervised Multitask Learners. OpenAl blog, 188):9 2019.
Survey on Data Selection for LLM Instruction Tuning. arXiv preprint arXiv:2402.057233 2024.
Wang, T. and Isola, P. Understanding contrastive represen- tation learning through alignment and uniformity on the hypersphere. In International conference on machine learning, pp. 9929 9939. PMLR, 2020.
Rajpurkar, P., Zhang, J., Lopyrev, K., and Liang, P. SQuAD: 100,000+ Questions for Machine Comprehension of Text. In Su, J., Duh, K., and Carreras, X. (eds.), Proceed- ings of the 2016 Conference on Empirical Methods in Natural Language Processing, pp. 2383-2392, Austin, Texas, November 2016. Association for Computational Linguistics. doi: 10. 10.18663/v1/D16- 1264. URL https: waclannholoog..rrgg//oo
Xic, C.,, Lin, Z., Backurs, A., Gopi, S., Yu, D., Inan, H. A., Nori, H., Jiang, H., Zhang. H., Lee, T., et al. Dif- ferentially Private Synthetic Data via Foundation Model APIs 2: Text. In Forty-first International Conference on Machine Learning, 2024.
Ren, R. and Liu, Y. Towards Understanding How Transform- ers Learn In-context Through a Representation Learn- ing Lens. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024. URL https://openrewe net//
Ye, J., Gao, J., Wu, Z.., Feng. J., Yu, T., and Kong, L. ProGen: Progressive Zero-shot Dataset Generation via In-context Feedback. In Findings of the Association for Computa- tional Linguisties: EMNLP 2022, pp. 3671-3683, 2022.
Ren, Y., Tan, X., Qin, T., Zhao, S., Zhao, Z., and Liu, T.- Y. Almost Unsupervised Text to Speech and Automatic Speech Recognition. In International conference on ma- chine learning, pp. 5410-5419, PMLR, 2019.
Yu, D., Kairouz, P., Oh, S., and Xu, Z. Instructions for Aligning Large Language Models. In Forty-first International Conference on Machine Learn- ing, 2024. URL https://openrreiiee net/ wwwp:/nee/fforr/// formmmiddd mUT1 mUT1biz09t.
Rumshisky, A., Ghassemi, M., Naumann, T., Szolovits, P., Castro, V., McCoy, T., and Perlis, R. Predicting Early Psychiatric Readmission with Natural Language Process- ing of Narrative Discharge Summaries. Translational psychiatry, 6(10):c911-9921, 2016.
Yue, X., Inan, H., Li, X., Kumar, G., McAnallen, J., Shajari, H., Sun, H., Levitan, D., and Sim, R. Synthetic Text Gen- eration with Differential Privacy: A Simple and Practical Recipe. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume I: Long Papers), pp. 1321-1342, 2023.
Steinke, Т. Composition of Differential Privacy & Privacy Amplification by Subsampling, 2022. URL https:// arxiv. wwwiv.oor/aae .rg.o/////bba.o abs//2/0b//2210 00597.
Yurochkin, M., Agarwal, M., Ghosh, S., Greenewald, K., Hoang, N., and Khazaeni, Y. Bayesian Nonparametric Federated Learning of Neural Networks. In International conference on machine learning, pp. 7252-7261. PMLR, 2019.
Su, Y., Lan, T., Wang, Y., Yogatama, D., Kong, L., and Collier, N. A contrastive framework for neural text gen- eration. In Advances in Neural Information Processing Systems (NeurIPS), volume 35, 2022.
Zdrazil, B., Felix, E., Hunter, F., Manners, E. J., Blackshaw, J., Corbett, S., de Veij. M., Ioannidis, H., Lopez, D. M., Mosquera, J. F., et al. The CbEMBL Database in 2023: A Drug Discovery Platform Spanning Multiple Bioactivity Data Types and Time Periods. Nucleic acids research, 52 (DI):D1 -0-DD1192 2024.
Touvron, H., Martin, I.,, Stone, K., Albert, P., Alma- hairi, A., Babaei, Y., Bashlykov, N., Batra, S., Bhar- gava, P., Bhosale, S., et al. Llama 2: Open Foundation and Fine-tuned Chat Models, 2023. URL https://arxiv. org/abs/2307.09222 2023.
12

---

Contrastive Private Data Synthesis via Weighted Multi-PLM Fusion
Zhang, J., Hua, Y., Wang, H., Song, T., Xue, Z., Ma, R., and Guan, H. FedALA: Adaptive Local Aggregation for Personalized Federated Learning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pp. 11237-11244, 2023.
Zhang, S., Roller, S., Goyal, N., Artetxe, M., Chen, M., Chen, S., Dewan, C., Diab, M., Li, X., Lin, X. V., Mi- haylov, T., Ott, M., Shleifer, S., Shuster, K., Simig, D., Koura, P. S., Sridhar, A., Wang, T., and Zettlemoyer, L.
OPT: Open Pre-trained Transformer Language Models, 2022.
Zhao, W., Song, S., and Zhou, C. Generate Synthetic Text Approximating the Private Distribution with Differen- tial Privacy. In Proceedings of the Thirty- Third Inter- national Joint Conference on Artificial Intelligence, pp. 6651 6651-6659, 2024.
Zou, T., Liu, Y., Li, P., Zhang, J., Liu, J., and Zhang, Y.- Q. FuseGen: PLM Fusion for Data- Pata-generation based Zero-shot Learning. In Al-Onaizan, Y., Bansal, M., and Chen, Y.-N. (eds.), Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pp. 11172-2190 November 2024.
13

---

Contrastive Private Data Synthesis via Weighted Multi-PLM Fusion
A. Contrastive Prompts and Non-contrastive Prompts
Table 8. Prompt used for synthetic dataset generation. Due to clarity, we omit the words in the parentheses in the labels of Openreview- Category and the attributes of Openreview-Rating
Datser (task)
pronge tye
peonge
label
attrib
(semantic unalysis of movic review)
IMDS
weo Contrastive
review is <somple SaletaRooed on the shove movie reviews now movie review also
"The moviie coview <oumple__ wheThe
csouple_ Del movie
positine/ negarive
None
<lobel seniiment but divens in the expression comgured to the above given samples is: "A movie review csample_ /ttpo/__//......l.. good mavic reviow is: csaoplee 182] + JSA good movic review is: <oom- plc SwiveBased on the shove exampees had and gund movie roviews clabeb semiiment, analyva the differences
Contrastive
movle review is; clompte_ [5/])/nA M'n
pooitive movie resiew that is diverse in expression compared to the given good reviews. Insure that the new review is further retined than the good reviews while maintaining the <dabeel> semiment and clarity. miking the good reviews appear lie midway between new reviow and the had resiews. The new clobels movic review
the
good riviews Geveste
Yelp
who Contrastive
The business review <sample_ Saapli__>iiiDee FhleThe business review is: <oumple Baapppl____v.______ 2>........iibbb business review commple StldwBaased
Arts & Entertaiament / Bars Reauty & Spos
10/20/55 50/
(held classification business review)
the showe business reviews hetenging the categerr of clabel, a new review for a business item also in the field of clabels with miing cattribates star(s) but diverse in the expression compured to the shove given samples is:"
4.0/5.0 5.0
Event Planning Services
Grocary Heulth Medical/ Hume & Garden
Hun's Trasel)
w/ Contrastive
A bad besiiness is coupple Joln ASn bad business review is: csasyiie [8/2 SSin 2ood
Resteurants / Shopping
review <cumplee_ 18/2 Is/...WA good business coview ple SwinBased on shove etamples had and good hosiness revious belonging to the category of <labelz, analyze the differences between the bad and good reviews. Generate review for a business ieem alsu in the field of clobels with rating cattributes star(s) new but diverse the expression compared the given good reviews. Ensure review is fuither refiied thas the good revicus while mairtaining clarity, making the good reviews appear to lis midway between the new review and the bad reviews. The new business
field clabebs
Yelp-Racing trating classitication of business reviow)
w/o Contrastive
The business review is: csoplle Phe business review is coumple_ 2ii... business review complle SaWaaRaed the above business reviows rating <label> staris,, a new resiew for business fimm is the fiek of catmiibtee also with naing <labels star(s) but diverse in the expression compored to the ahove given samples is:
L0/20/30/
Arts & Entertainmentt Bags Reauty Spus
4.0/5.0
Event Planning & &Services/ Grocery
Contrastive
bad business review csomple Jsla...nA bad business seview csupplee [Ssoplt___///eeee b/// good business review is < cemplee (5/2 +/>'n..nn good business review is: <ram- plc SurvwRosed on the ahowe examples of had and good business reviews with rating differences between new review for a business item the Sold of canrihare> also with rating <labeb> star's) but divense in the expression compared to the above given good reviews. Ensure that the review further cefiaed than the good reviews while maintaining clurity, making the good reviews appear
Health & Medicall y
<lobel> starts), analyze
bud good reviews. Generate a
Home & Garden/
Hotess & Travel
Resturants
Shopping
midway betwzen new review and the bad reviews. new business review with mting <labels starss
Openeesew- Saateesee--Catgggg Category (fiedd classification of paper review)
Weo Contrastive
The paper review is: compple The poper review coumple Zele...eThh paper review is: coungle Salvludlaedd on the shove paper reviews of paper the area daoless a review for paper also in the of clabe> with fital recommndation: "cottribttes'' but divense
Applications /Deep Learning and represatational learning! /
rejectí seject. not good enough
General Machise Learning / S. mus musiinally below the
expression compared the shove given samples
Generatite modaks / Machine Learning for Scionces / Neurrscience and Cognitive Science/ / Optimization! Probabilistic Methods /
acceptance threshod
w/ Contrastive
A bad paper review is complee_ /ohn. had paper review is: <ommple__ J//J gaod poper rnview is: commple (82) +/snn Vec pood paper
6i marginally thove the acceptince threshuld/ 8: accept, good paper
csumple SwleiRased
on the above examples of bad and good paper reviews of poper the area <lobeb, analyze the differences between the had and good riveus Generate new review for a paper also the area of <label> with final recommendaiio: 'cattiiuice" but diverse the expression compared the given good reviews. Ensure that the new review is fatther recined than the good reviews while maizaining clarity, making the good reviews appear to lie miduary herween the new seview and the had reviews. The new paper review in the
Reinforesmend Learning! !
Social Aspects of
Machine Learning
Theory/ Unsupervized and Self-surrrited learning
dlabebs
Opencesiew Kating (rating classifteation pager review)
w/o Contrastive
The paper review ie: commple The paper review is complle 25Sn...WiThe paper of final recommenation: clabvb, new resiow for poper in the field "cutiiuurr' also with final recommenda- tioe: <label> but diverse in the expression compared to the above given samples is:"
I: srong enject? I
Applications / Deep Learning and representatioal learning
review <compie
the shove puper
3: rejoes, not good enough/
marginally below the
General Machine Leaningg
acceptonce shreshold
Genenniiv models
w/ Contrastive
had poper mivew is compple 7s'n leA hud poper reviow is: emuple I/AA good puper reviow is: exomple [8/] +/shn. vA good paper review is compple SwieBlasd the ahowe exmples of had and good puper riviews of final moummensation: <labels. analyse differences betweea the bad good reviews. Generate new review for a paper in the field of "comributes' also with final mocommendaioon <label> but diverse in the expression compared to the above gives good reviews. Ennsure that the new review is funher refinel than the good rviiww while maintaining clarity, making good resiews appear to miduay betweni the new coview and the bad reviews. The new paper review of final recommenittion clabuts is:
6: marginally above the acceptonce threshold accept, paper
Seiences/ / Neurosiesse and Cugniave Science
Machine Leuning
Ogtimirationn / Probabilisiic Methodss
Reinfoccement Leurning
Sociall Aspects of Machine Leanting / /nsupervised
Theury / and Self-sperviied leamning
Rusking (feldd clasilteation of online banking
Weo Contrastive
The online banking query is: cromply_ ThweThe online banking queyy is: <oum- ple 2o/n../WiThe online banking query is: exomple SalsteBased on the above online the category of "<labets", categoey "clabels" but diverse the expression compured the above given simples is.
actionte cardd age Bonit! apple pay or google
None
bunking
online bunking query also the
queries)
uum support! top I baloner not updaued after bank transfer i balance not updated after cheque or deposit/ benefciiary not allowed cancel_ transfer/ I cord. about essire
Contrastive
online bunking query is: crample Iow....nA online banking query <mample (8/2) Slick good online banking query is: crample (8/) +/>nn Anr good
online banking query is: ssample Swivilased on the ahuve exumples of bad and good
culine banking queries the cotegory "caale"" mallye the differences between the bad good reviews. Generate new online banking query also the categury of "claboss" but diverse in the expression compared to the above given good quasiss.
Envare that the new query is forther refined than the good queins while maintaining clarity. making the good queies appear lie miduaay herween the new query and the had queries. The new online bonking query als in the calegury "clabel""" is:"
14

---

Contrastive Private Data Synthesis via Weighted Multi-PLM Fusion
Algorithm 2 WASP for Distributed Federated Data (L >>
Input:
K PLMS (P=) with empty synthetic dataset (Da + 0] private data parties controlling distributed private dataset (Bi Him of M samples in total that belongs to C' categories; number in-context samples S:
number of iterations taken to obtain total synthetic samples; initialized PLM weights (wy 1/K]
learning rate 77;
DP privacy parameters 6;
test dataset of downstream task A; random initialized STM mçoo:
Output: STM
I: Initialize co-contxtt feedback samples Dr + 0, + 0.
Calculate Gaussian noise a = log 25/25/t//
LT
3: for 0 to T do
K in parallel do 4: for WeiighheeSSyDDtttterrrrtttooooo Do, Dí, D', ((N/T) x wx). C).
5: DD
6:
end for
7:
UK_1 Di. for = to L in parallel do
8:
Hi" H!{ DP DD.PrivateVoting(D.... a).
10:
end for
11:
H" +
H H/.
12:
D", Dí + Sampleselection(2 H H², C).
13: {arx h + PLMScoring(HH" (Dx )=
14: end for
m t- STMTraining((D mmo). 1))
In Table 8, listed the prompts used in experiments, including contrastive ("w Contrastive") and non-contrastive ("w/o Contrastive") in-context learning prompts. We need to clarify that, for PE series baselines, we use their original prompt for VARIATIONAL_ANT _PPI, which is different from the listed "w/o Contrastive in-context learning" prompt in Table 8, Please refer to Xie et al. (2024) (the original work) for detailed prompts.
B. Algorithm for Distributed Private Data and Detailed Functions
Due space limitation, we include the full algorithm for L setting here in Algorithm in the Appendix. The difference between Algorithm 2 and Algorithm 1 mainly falls in line and lines 8 to 11 in Algorithm 2.
We also include pseudo-code for the functions used in Algorithms 1 and 2 here in Algorithm 3 due to space limitation.
C. Supporting Results for Introduction
C.1. Examples High-quality and Low-quality Samples
We show examples of high-quality and low-quality synthetic samples generated using Aug-PE Table 10 and Table respectively. We also include the appearance frequency of some types of low-quality samples within the generated DP synthetic dataset in Table 9.
Table 9 shows that, low-quality noisy samples often diverge from the specified task (generating movie reviews in posi- tive/negative sentiment for this table). Differently, likes shown in Table 10, high-quality samples often possess clear sentiment tendency that well accomplished the task, with some offering detailed judgments or even containing concession details.
D. Theoretical Privacy Analysis for WASP
To prove Theorem 4. in this part, prove that the WASP framework described in Algorithm 2 with distributed federated data (L > 1) satisfies (c, 8)DD, which is the general case for L = 1 setting described in Algorithm I and Theorem 4.1.
15

---

Contrastive Private Data Synthesis via Weighted Multi-PLN Fusion
Table 9. Low-quality DP synthetic samples for movie review semantic analysis with IMDb as real dataset.
Model
Low-quality Noisy Sample Text (Examples)
Label
Explain
GPT-2
"- If you are missing No. 17 (see below) -"
negative
Meaning less sentence.
Llama-2
"In an informal way, please rephrase the sentences as follows:"
positive
Repeating the prompt. Appears around 30 times in D> of size 6, 000.
Vicuna
In o A f e 2 o d o E 7 7 $ L..
& 7 +
positive
Unmatched language and translating the prompt. Appears within 9 sam- ples D of size 6, 000. Task unrelated and mislabeled.
OPT
"The sentence is positive, but is ambiguous. Would it make sense to remove
negative
ChaiGLM3
- # # 6 T
x
* # =
positive
Unmatched language and repeating the prompt.
T T Since is based on positive sentiment, then how does one use the "roonoun"'
Flan-T5
"what fantastic movie!! :0 great acting and spec-
positive
Excessively long consecutive punc- tuation marks. Appears within 4 samples in 'D of size 000.
tacular scenery. great work. i look forward to other good one year lana!!!!!!!!!!!!!!!!!!!!!!!!
: come on laura de- woohoooo!!!....
wait!
Table 10. High-quality DP synthetic samples for movie review semantic analysis with IMDb as real dataset.
Model
High-quality Sample Text (Examples)
Label
GPT-2
"This movie isn't 't for everyone, but it's worth a look. But if you have to he in the dark to enjoy it, just buy the soundtrack."
positive
GPT-2
"is that it has some semi -nteiinteeeeting world- boild-buii,ing, but the information it offers is grossly insufficient to allow one progress in the story. The central characters have little to do with the world beyond playing around with their toys and talking to themselves.
negative
Llama-2
"Incredible film! I laughed, I cried, and I left the theater feeling inspired. Every performance was top-notch and the direction was flawless. A true masterpiece that will be remembered for years to come. 10/10."
positive
Llama-2
"This film was a colossal disappointment. found myself glancing at my wrist every 10 minutes, cagerly anticipating its conclusion. The narrative was intricate and challenging to comprehend, and the personas were underdeveloped. I had no emotional investment in their plight. The performances were lacking, and the dialogue was trite and unconvineing. In summary, would not advise viewing this motion picture to any individual."
negative
Vicuna
"A movic for people who want to action action-packed adventure that will keep them on the edge of their positive
seats."
Vicuna
"Felt like a missed opportunity. Glacial pacing and shallow characters. The premise was intriguing but the exccution fell flat, Avoid this onc."
negative
OPT
--This film is enjoyable to watch because of it's well-developed storyline and most of all its storyline that can relate to and would like to have. This film also has good acting performances and the songs help the film lot. It's not a typical love story kind of film, but it does have lots interesting messages to give. Very well-done! Definitely recommend this film!"
positive
OPT
"Quote: The first sentence means nothing to me. Without context the first two sentences also mean nothing."
negative
ChatGLM3
"Attention getter. visually interesting and outstanding acting, the story of an American citizen that is in Mexico and gets involved in a murder is a good movie."
positive
ChatGLM3
"This movie is an unwatchable mess, basically a rip off that scene in Dr. Strangelove where the U.S. Air Force gets into a scramble to deliver nukes after a false alert... like pretty much every aspect of anime, but this stuff is bad. Animation is choppy, sound effects and voice acting are atrocious, and the
negative
plot makes little sense."
Flan-T5 "<pad> This is a fine historical drama about America's 's Civil Rights movement, that has William Friedkin as director and Gary Cooper an actor. Although there other excellent directors that haven't heard of. William Friedkin has become the standard many ways. Filming in New Orleans, Mr. Friedkin gives a glimpse into the African-American experience during the late 1920s. saw it at the All-African Cinema festival at Wharton Business School. 1 was awestruck by the power and impact of the film. ...Iruncated because of excessive lengthJ..."
positive
Flan-T5
"<pad> beware of this film! i can't 't recall a single horror movie that was scarier than this one. it's just negative plain cheesy and stupid. there's plot and the characters entirely cheesy. don't even bother. you'll never forget it."
16

---

Contrastive Private Data Synthesis via Weighted Multi-PLN Fusion
Algorithm 3 Functions used in Algorithms and for WASP
function weighhaelyyyaatttmmmrriiiiion.o, Dr, D', N, C): for 1 to C' do
if then
Use zcro-shot prompt as working prompt T(c).
else
Randomly sample S - [S/2] samples from De-le) and [S/2] samples from DDAlel to create few-shot prompt as working prompt
T(c).
end
Generate [N/C] samples using end for
them to Da-
if [P&] > N then
Random sample [Dx] - Ñ different samples from Dj and remove them from De.
end if return Da.
function STMTraining(D, m(o). 1)):
Initialize a trainable STM m <- m4(0).
Train using D with learning rate till convergence by using objective function C = E|2 {(m((x;).) y.). return
function DP "t__P_iaateeoting"" B,
Initialized H t- [o.......] 0]: H - 0f of length (PI and note the total DP synthetic dataset as D {(x,, y1)
for (Z), uy) in B do
D 24 {(x,))) v:) E D 1 = n³}
"..::...,:...) < LopQSmallest (dlz), xx) (o..neelll))
[f.1.... + arg BargtopTaarger d(zz, x.) bc, (x,,))))))
for go to Q do
H"(ms.o] + H [1 + 2--1 H [fsa] + H [ssal + z
end for end for
+- +N 0² [I)) H³ H² +N((,
return H H!.
function PLMScoring (H, (Dx) for k 1 to K do Calculate s; H | Σ -ー H').
Calculate model score 17
E...)) [Pel/P]
|Pal/ ΣB Pel
end for return (u Ik=1-
function SampleSelection(D. H1" H², S.C)):
Reset Dr <- 0, DJ
H" + /D
Hi H + H² E|2 H'].).
for 1 to do
c ((x,,
E DD): = c). E
n [i] (x., n:) Dle H [4c H [i] (xi, w:) E Dle]
(x,8)) Dio] H [i] among the top-S S values of Hm. contains the top-S samples from Die) ranked by Mm
DM4 (x), E Dle] [ is among the top-S values of H fee]] contains the top-S samples from DI ranked by H/-1s|
end for
D
[milll p.... Dr.(c] }
J. DJ = (Dril)) ..... 1
Di.
return
17

---

Contrastive Private Data Synthesis via Weighted Multi-PLM Fusion
Theorem D.I. Let f be a function with global Lg sensitivity A. For any e > 0, E (0, 1), the Gaussian oulpui perturbation mechanism with a = A /2log(1 25..25//5/ 25/b ensures that f satisfies (e, (e.B)
Proof of Theorem D.I can be found in Balle & Wang (2018).
Theorem D.2. The global Lg sensitivity A of WASP described in Algorithm 2 is 4.
Proof. In WASP framework (Algorithm 2), function DP_ uP_PrivateVoting is the only function that accesses the private dataset B. Thus, A of WASP equals to that of function DP_Priveeee __r.raateeootingg.... Within function DP P.iiaateoooiigg for nearest histogram and furthest histogram respectively, each private sample contributes Q votes with decaying voting weights 2....... y0-r}. Therefore, the total votes contributed by one private sample is Σ 2 2 20-1 < 2 for each histogram. Adding or removing one private sample in B will result in a change no more than 2 in the Eo norm of each histogram. Therefore, the upper bound of the sensitivity for each histogram is 2 and the upper bound of the sensitivity of WASP framework is 4 considering both histograms, i.e. A ==4, 4.
Lemma D.3. If a Gaussian mechanism satisfies (e, 6DDD, then independently repeating this mechanism for T times results in the final DP budget to increase to final VT e and the final probubility of data leak fiiaat increased to of >> T 8.
The proof of Lemma D.3 can be found in Steinke (2022).
With the above lemma and theorems, we present and prove our main theorem as follows. Theorem D.4. If each private data party performs standard Gaussian mechanism with addition noise following (0, o²) and a 4 22 ogg (225/berr//T WASP framework described in Algorithm 2 satisfies DP with privacy budget for private samples in B.
e/b
Proof. For guaranteeing (c, 6DD
DD throughout the T iterations with feedback (the first generation iteration does not - use feedback), each iteration should satisfy a differential privacy budget of vT-I" Given A = for WASP, dotal
/2log (1.25/ (1.25/b:tar/WTT-1 for each single generation iteration will guarantee (e, 6DD for the whole process with d > ōster (T -1). Further, Gaussian random variables satisfy that X + Y ~ .V(0, + oσ) if ~ NN of), Y ~ N(0, oï) are independent. Therefore, if each private data party adds i.i.d. Gaussian noise following NN o²) with
4 /2 (1.25/bue<\VT-I the total noise follows N N(0, N0,,0 (otaa) which guaranties (c, 8)-D for the whole WASP process eVL with Siter (T -T- 1).
E. Additional Results
E.1. Comparison of Synthetic Sample Resemblance for WASP
To further demonstrate the effectiveness of WASP under limited private sample setting, we additionally use FID between the generated DP synthetic dataset D and the real private dataset B to evaluate the resemblance of the former (D) to the later (B) with M 100 in Figure 6. Lower FID indicates higher distribution similarity therefore indicating better resemblance.
As shown Figure 6, the baseline method Aug-PE often fails to generate a DP synthetic dataset that closely resembles B when using an improper PLM, leading to an increased FID value over iterations. On the contrary, WASP results in a consistently decreasing FID value over iteration, demonstrating effectiveness in improving the resemblance of to B. Moreover, although WASP initially has a higher FID than using the best single PLM (Flan-T5 in Figure 6) for Aug-PE (which is reasonable due to the initialization of 'D being mixture of synthetic samples from different PLMs, making it better than the one generated solely by worst PLM but worse than the one given solely by the best PLM), it ultimately achieves a lower FID than all baseline counterparts. This indicates that our proposed WASP method better handles the limited private sample setting.
E.2. Effectiveness of Differentially Private Top-Q Voting and Contrast In-context Learning with Single PLM
We present additional results to validate the effectiveness of Differentially Private Top-Q voting and contrastive in-context learning with single PLM in Figure? 7. Starting with Aug-PE, we increase Q from 1 to to obtain the wwo Con" results, and then incorporate contrastive in-context learning samples into the prompt to achieve the "w/ Con" results (also the K = 1
18

---

Contrastive Private Data Synthesis via Weighted Multi-PL. Fusion
27.5
25.0
Aug PGarr-3
22.5
Aug-PEuuur
Aug-PEaer
17.5
Aug Nug-PEran-TT
15.0
WASP (Ours)
12.5
Iteration
Figure 6. Comparison of the resemblance of synthetic dataset real private dataset (FID) using Aug-PF. and our proposed WASP using movie review semantic analysis task and IMDb dataset.
GPT-2
Llama-2
Vicuna
$0.0
23.9
ACC
ACC
75
22
17.5
75.0-
72.5
22.4
Iteration OPT
Iteration ChatGLM3
Iteration Flan-T5
28
82.5-
FID
ACC
90-
ACC
FID
80.0
26
Iteration
Iteration
Iteration
.. Aug-PE
w/o Con
w/ Con
Aug-PE
w/o Con
w/ Con
Figure 7. Comparison of the resemblance of synthetic dataset to real private dataset (Fréchet Inception Distance, FID) and trained downstream model performance (ACC) using Aug-PE ("Aug-PE", doted lines), refinement on Q without contrastive in-context learning ("w/o Con", dashed lines) and refinement on Q = 8 with contrastive in-context learning ("w/ Con", solid lines) with single-PLM setting and L under the same DP setting in Table 1 with IMDb dataset.
setting for WASP). Note, to guarantee the same privacy budget, noise scale also increases (to twice its original value) as Q increases from to Q and is further doubled as contrastive in-context learning is incorporated.
Both of these refinement processes both display steady decrease in FID for most PLMs, demonstrating their respective effectiveness. Nonetheless, overall performance improvement is observed for all tested PLMs, both in terms of highest performance across iterations and final performance.
E.3. Sensitive Analysis of M for PE
We performed experiments analyze the sensitivity of Aug-PE (Xie et al., 2024) various M values. Results are included in Figure 8 which shows that most PLMS fail when only a limited amount of private samples (M 100) is available, with an increasing FID through iterations. Conversely, with sufficient amount of private samples (M = 10, 000), continuous decrease in FID as well as less performance fluctuation can be observed throughout the iterations.
E.4. Comparison of WASP and Pre-Text Under User-level DP
In our work, we assume a full participation setting where all L parties participate in each iteration. Based on this, we primarily focus on ensuring sample-level DP to protect each private sample (z,, u;) E B in this work. However, our proposed WASP method can be easily extended to user-level DP protection and is also effective protecting user-level DP compared to baselines (see Table 11).
19

---

Contrastive Private Data Synthesis via Weighted Multi-PLM Fusion
GPT-2
Liama-2
Vicuna
86
85
75.0-
84
84
8.3
72.5
ACC
ACC
HID
70.0
80
67.5
79
74
Iteration
Iteration ChatGLM3
Iteration Flan-T5
OPT
84-
77.5
85-
ACC
80-
82-
22
22
ACO
-24
23
ACC
FID
78-
27
76-
74
70.0
0
2
0
2
2
Iteration
Iteration
Iteration
...
ACC for 100
--- ACC for 1000
ACC for 10000
FID for 100
FID for 1000
FID for 10000
Figure& 8. Comparison of the similarity of synthetic dataset real private dataset (FID) and trained downstream model performance (ACC) with different amount of available private samples (M) using Aug-PE with L = 1 under the same DP setting as in Table I with IMDb datasct.
In Hou et al. (2024), although they also study a full participation setting with L >> 1, they focus on user-level DP with the assumption that each participating private data party in the collaboration controls only a tiny amount of private samples (8 in their work). Therefore, strictly following Hou al. (2024), to testify the effectiveness of WASP when extended to user- DP, we assume that each participating data party controls no more than 8 real private samples, i.e. Mr S 8,1 = L. These distributed private datasets still aggregate to unbalanced dataset like in Section 5.1.
Under this setting, to protect user-level DP (where adding or removing one private data party should not significantly affect the function output), following Hou et al. (2024), the function sensitively Auser should be max(M,,..... ,M,) times as large as that for protecting sample-level DP (A sample). The rational is that, the addition or removal of a private data party can result in the addition or removal of up to max(M,,
M2,) samples, leading to a change of no more than max(M;, ... M2) X Aammle in the &g distance of the produced histograms. Given that Aamppl = 4 for WASP (see Theorem D.2 in Appendix D) for details), we have Auser max mmxx(M,-.. MM,..., Mr.) sammpl S 8x A sample 8 x 32. In our experiments, we use 32, the upper bound of Auser as the function sensitivity to calculate 32 /2log (1. 2 MvT for (e, 6DP protection with 8 diter. (T'-1) 1).
eVL
Results are shown in Table 11 with a total of L = 150 private data parties controlling M = 500 private samples in total. Other experimental settings are the same with those in Table 2. Results show that, WASP continues to outperform baseline methods, including Pre-Text. This demonstrates that WASP is effectiveness not only under the need of guarantecing sample-level DP but also under the need of providing user-level DP protection compared to baseline methods.
Note that, if each user contributes more samples (e.g. 100 samples per user), the noise application strategy given in Hou al. (2024) will scale linearly, which may become impractical. Therefore, under such scenario, optimizations like norm clipping (Cheng et al., 2022; Kato et al., 2024), an off thel- and widely applied technique, can be applied to keep noise levels practical by capping the norm of voting vectors H1" H given by data party i at preset bound Ç. preventing user sensitivity A from scaling linearly with user private dataset size. Under the same Ç. using more private samples (e.g. 100 samples per user) will not degrade performance compared to using fewer (e.g. 8 samples per user).
Morcover, if a single party controls large number of samples (c.g. 100), like shown in Table 1, this party alone can achieve good performance using sample-level DP without the need of collaborating with others.
E.5. Comparison of Computational Cost and Complexity
We need to first of all, as stated previously, no additional queries are made to PLMs under the same number of required synthetic samples (N) in WASP framework compared to baseline methods although samples are generated iterative
20

---

Contrastive Private Data Synthesis via Weighted Multi-PLM Fusion
Table 11. Evaluation of downstream STM accuracy using 6 PL.Ms, I. 150.. User-level DP is guaranteed instead of sample-level DP in this table. Best and second best results are marked.
Yelp
Openreview
Privacy
[8]
[D]
IMDb
Banking
Category
Rating
Area
Rating
OnlyPrivate
500
83.61
57.27
44.15
22.76
32.79
74.56
FuseGien
Absolutely Private
6, 50000
89.07
63.38
57.96
24.70
34.57
78.75
GPTT-2
4.0
500
6, 000
83.96
63.04
45.78
27.46
24.09
75.75
Llama-2
c 4.0
500
6, 000
84.28
60.24
50.54
29.02
34.15
82.50
Vicuna
== 4.0
500
000
83.67
63.21
51.42
28.18
32.87
83.38
Pre-Text
OPT
4.0
500
6,000
84.69
62.92
50.40
28.59
24.29
81.25
C3
4.0
500
6, 6,000
85.56
57.46
51.54
29.78
32.33
84.88
Flam-T5
= 4.0
500
6, 000
88.71
58.46
58.37
29.81
34.02
74.13
WASP (Ours)
c 4.0
500
6, 000
89.15
63.49
59.78
29.96
37.10
85.25
Table 12. Comparison of computational complexity and runtime (scconds) of WASP and PE series baselines within cach iteration (averaged across iterations and also across different PLMs PE series baselines). "Others" includes DP top-Q voting and PLM importance weighting, Le. line 7 10 in Algorithm 1, with the latter being negligible time cost tensor normalization operation the Nearest Histogram H" vector.
Aug-PE
Pre-Text
WASP
Generation
Others
Generation
Others
Generation
Others
Complexity
O(N)
O(MN)
O(N)
O(MN)
O(N)
O(MN)
I 1, M 100
1.341 10*
5.846
1,970x 10*
6.090
IMDb
= 10, M 300
1.348x10
7.153
1.963 10
7.668
1, M
1.539x 10*
5.468
2.175x 10
5.528
Yelp-Rating
L = 10, M 300
x.561x10
7.376
2.235x 10*
7.608
L 1, M 100
1.268 10*
5.655
1.949 10*
5,871
Openreview-Rating
= 10, M 300
1.215x
7.713
1.982x 10
7.911
manner. As shown in Table 12, the computational cost WASP is slightly higher than that of PE baselines, mainly due to increased in prompt length (8 in-context samples instead 1) and the calculation of Furthest Histogram HJ
Moreover, the primary computational cost is driven by synthetic sample generation, which accounts for nearly 3000 times the runtime of other operations (including DP Top-Q Voting and PLM Importance Weighting). While this process is unavoidable, users can mitigate the impact of computationally expensive PLMs by choosing faster alternatives. Also, like shown in Table 12, for large-scale datasets, WASP's computational cost remains manageable.
E.6. Comparison of Communication Overhead of WASP and Pre-Text for Federated Data Setting
We compare the transmitted information for secure aggregation between the baseline method Pre-Text and our proposed WASP framework in Table 13. With the same number of participating data parties (L), WASP only requires aggregating additional L histograms of dimension !! and uploading the aggregated histogram H E IR PR These additional communicated information leads to only a minor increase in communication overhead compared to Pre-Text.
E.7. More Strict DP Guarantee
In previous experiments, we use 6 = x 10-5 which will result 5 > 4 10 for the whole process for all PE series baselines and WASP. Therefore, we compare using ōster = 1 x 1 with 1 x 10 in Table 14. With = 1 x 10-5 following Kairouz et al. (2015), dster
10-28 -3 can be applied guarantee overall (4, 1 x 10 DDD,, which results in
21

---

Contrastive Private Data Synthesis via Weighted Multi-PLM Fusion
Table 13. Comparison of the information data parties' download, internal exchange and update in Pre-Text and WASP.
Download
Exchange
Upload
Pre-Text WASP (Ours)
embedding of each (Xa, E D
{ Ji
H H
embedding of each (Xa, E D
1 {HF N H
H
Table 14, Comparison of different DP using 6 open-source PI.Ms for PE haseline and our proposed WASP with I. M = 100.
Aug-PE
WASP (Ours)
GPT-2
Llama-2
Vicuna
OPT
ChatGLM3
Flan-T5
ōiter 1 10-5
85.38
85.77
82.76
83.86
85.82
89.00
89.52
IMDb
10
84.88
85.30
82.04
83.52
85.22
88.83
89.18
ōiter 10-5
45.28
47.42
54.42
50.81
55.17
58.69
61.21
Yelp
d = 1 10
45.03
47.10
54.09
50.47
54.97
58.61
61.05
noise scale roughly 2.14 times large the original used in original experiments in the paper.
These results demonstrate that, under tighter privacy guarantee (ōtte 1 x 10 i.c. 5 = 1 x 10-5), the performance
decrease is just minor, indicating the robustness of WASP and PE baselines.
F. Additional Related Works
Due space limitation, include the discussion of previous works related to Contrastive In-context Learning (Contrastive ICL) here in the Appendix.
Contrastive In-context Learning.' The idea of using contrastive information to enrich in-context learning samples has been exploited from different aspects. Samples belonging to positive and negative classes (Liang et al., 2024), correct or wrong oelficressstions of training samples during training time (Mo et al., 2024). human-preferred and non non- non-preferred question responses Gao & Das (2024) have all been utilized contrastive samples. Our study is the first known effort to consider contrastive in-context learning for synthetic data generation, by treating synthetic samples of different qualities generated by multiple PLMMs contrastive information.
G. Theoretical Evidence for Convergence of WASP
Our goal is to prove that, each iteration of synthetic data generation produces samples that become closer to the private dataset B. To formalize and prove this iterative improvement, we need a mechanism that biases the generation process toward "better" (closer) samples and away from "worse" (more distant) samples. Contrastive Learning (CL) (Gao et al., 2021; Su al., 2022; An et al., 2022) fits naturally here: its core idea pull "positive" pairs together in the representation space while pushing "negative" pairs apart. Specifically, we denote by S C D set of high-quality samples and by set of low-quality samples.
S- C D/ In the following, we first list reasonable assumptions that serve our proof. We then prove sequentially the convergence guarantee of our proposed WASP framework without and with DP.
Assumption G.1. In order to guarantee the viability of multi- multi-round contrastive learning, we impose the following key assumptions:
1. The private datasets of L different parties, (Bi)_,, = U Bi are contained Euclidean ball of diameter D 0. That is, Xx yll D for all x,y E B.
2. We adopt the Euclidean & norm throughout, i.e. d(x,y) ||x yll2-
'Works (Ren& & Liu, 2024; Miyanishi & Nguyen, 2024) considering understanding in-context learning with contrastive learning theories are sometimes referred to using the same name, hut we do not consider them here.
22

---

Contrastive Private Data Synthesis via Weighted Multi-PLN Fusion
3. In each round O,..,, 1, we randomly choose a subset of high-quality samples S D and subset of low-quality samples S,- C Dr. These two subsets form the Prompt T-) for that round, and we then invoke taaehttedSynattseeeeraatioo
with distribution Po(- S St to generate new candidate samples x'. We assume the distribution has bounded variance or avoids extreme outliers, and each round can be treated as nearly independent via Chernoff/ Union arguments.
4. The low-quality sample pool is sufficiently large to capture potential adverse modes; extremely rare outlier low-quality samples occur with negligible probability and can be ignored or filtered in practice.
5. When (c, 6 is required, Gaussian noise with standard deviation o is injected into voting/statisties (here, o refers to the noise level in the Gaussian mechanism). Additionally, each cluster must contain at least K private samples (multiplicity K > d), ensuring that noise cannot completely nullify that cluster and undermine the success probability 7 > 0 in contrastive learning.
6. (Alignment Assumption) If the chosen high-quality subset S contains at least one sample x+ E S that is already relatively close to the private set B, then any candidate x' closer to than this x* will not receive a "low score" under the similarity function so(:;") ).. Specifically, if d(x+, ((xx+1B)) ,B) is small, then for any x' satisfying d( d(x', .B) < dd(x+, we have sa [x x') se (x* xx). Here, s)(".)) is a learned similarity measure (often inner product in hidden space without vector normalization). Consequently, such an x' is assigned nonzero probability under Po (- S ).
The Reasonableness of the Alignment Assumption (G.1.6). Assumption G.1.6 is consistent with prior theoretical work in contrastive learning (Arora et al., 2019; Wang & Isola, 2020; Parulekar et al., 2023). It has been shown that contrastive learning "forces the inner product of representations of similar pairs to be higher on average than with negative samples" (Arora et al., 2019). In addition, alignment has been highlighted as key property of contrastive representations, indicating that "similar samples have similar features" (Wang& & Isola, 2020), i.c., high mutual similarity in the embedding space. More recently, an "alignment term" was explicitly incorporated into the InfoNCE loss to maximize the similarity between an image and its augmentation (Parulekar et al., 2023), thus providing further theoretical support for our alignment hypothesis. Taken together, these studies reinforce that well-aligned positive pairs are crucial for effective contrastive representation learning.
G.1. Convergence Analysis Without Considering Noise
Under the Alignment Assumption (Assumption G.1.6), Po(x' St, indeed assigns higher probability to candidates x' that lie closer B. This guarantees strictly positive probability of surpassing the current high-quality point in each round. Hence, we can formally track how the expected distance to B decreases over multiple iterations, leading to the convergence analysis we detail below.
Building on this contrastive mechanism, our proof leverages the idea that once the InfoNCE loss converges under the assumption that "closer" samples in St are favored over "more distant" ones in S we can rigorously connect loss convergence to the probability of improvement, As a result, each iteration maintains strietly positive chance of finding a new candidate z' that outperforms the current best sample. This linkage underpins our multi-round convergence argument: across multiple iterations, the quality of synthetic data systematically moves closer to B.
Concretely, we adopt the InfoNCE loss (Gao et al., 2021) to implement contrastive learning. In each iteration, given a pair (at x ) S+ x S and the current candidate x', we compute
exp(se(x* xx) - (E x )x log exp (xt, x')) + exp(se((x xx))
(())
For simplicity, consider a single-sample loss ((f) corresponding to one such triplet (x*, x x²). Define
exp(<e(x** ,x))
e(() log exp(se(xt,x) x()) + exp(ss(xx,xx) x'))
and let
exp( (xt (xxxxx))
xx))
cxpl (0 (x)x)
p(xt)= exp(so(xt, x')) + +expl (op(so(xx (x- (x)x))))
cxp(so(x* x')) + exp( (x((x(((__) xx))
23

---

Contrastive Private Data Synthesis via Weighted Multi-PLN Fusion
that
0E(0)
0e(0)
0sg(xt, -[1 (1-1((x)]]]]
+ +(xxx)
Ose (x6(x_,1) ,²)
Applying the chain rule w.r.t. 0,
V (()) - -11 --11-((xx] V Voso(x xx)) + p(x" Vose(x ,x))
Because training involves descending Vol((), the negative coefficient - p(x*)) "pulls up" the similarity of (xx x'), while the positive coefficient +-(xx) "pushes down" the similarity of (x x'). Taking the expectation over all such
(x* xx""x" E S+ S and x' yields the global conclusion that high-quality samples' similarity x' is increased, whereas low-quality samples' similarity decreased.
Definition G.2 (Per-Sample Improvement and Worst-Case Improvement). Let d(x, B) == inf ||x 1x--0 -bbl-
(a) Per-sample improvement: For each x** S* define
~ PocIst [((xt, d(x', B))
(b) Worst-case improvement in S+: A inf xtss+ Ax.
Proposition G.3. Let xt E S be a chosen high-quality sample, and let B be the private dataset. Suppose x' is not globally optimal, i.e. d(x+, ,B) > O. If the contrastive loss is sufficiently converged and the alignment ussumption (see Assumption 1.6) holds, then under the distribution (o | the event { x' d(x', < ddxx, .B) occurs with nonzero probability sufficiently large ensure strictly positive improvement. In other words, the improvement quantity A from Definition G.2 satisfies A > 0 as long St contains at least one x with d(x* (et,1) ,B) 0.
Proof. Fix T e St with d(x ,B) 0. Let
d(x+ B) - d(I',B), s'~ P~Pol-1st.S)
Then
A = 17Z = / ZdPPllo).
Define two events:
A = {x': Z> = (x': d(a' < d(x p.att,B9 ,B), A = {x': 7 5).
Since Z > 0 on A and Z < 0 on A we have:
/ ZdPo + / ZdPo = / [Z| - /
A
|Z| dPo.
A=
Hence, proving A,- > 0 equivalent to showing
/ |2]
|z| dPo /
IZ|dPo.
A=
By alignment (Assumption 1.6),
if d(z'; B) < d(z'g, B), then Po |S 2 Po ST.S)).
In other words, the distribution Po(. St s+, favors samples that are closer to B. Hence, the set A (all x' closer to B than at is) enjoys strictly larger probability measure than the set A
On A, Z d(x () d(x'd ()) is strictly positive and can reach up to d(x B) (in the extreme case where x' is very close to B). Conversely, on A we have S O, so |Z| = d(æ', d(x* + B3) represents a "loss." " By alignment, A occupies smaller measure the distribution Po(. St (S, S-) and corresponds to samples that are comparatively farther
24

---

Contrastive Private Data Synthesis via Weighted Multi-PLM Fusion
from B. Consequently, the integral of [2] over A will surpass that of A ensuring Aa |z dPa > SA. 12| dPo which leads
to Ag = = 7 LA- 7 0.
To see this more quantitatively, consider an increment d > 0. The probability mass of x' such that Z x 8 (i.e. d(z', BB) ~ d(x+, B) 5) exceeds the probability mass of x' such that Z ~ -ō (i.e. d(x', B) ~ d(æ (B + 8).
Integrating these local comparisons from å 0 to = d(x .B) shows SA 12|dPs > SA- 1 dPg. Hence the overall positive contribution outweighs the negative one.
Since A,+ SA d/ SA- | dP' O, we have shown strictly positive expected improvement for this particular
xt. Repeating the argument for any x+ E S* with d(x+ ,B) > O, obtain Art > 0. Therefore, A innf x zlest
Agt
0, as long as there exists at least one x not globally optimal.
Finally, we employ Markov's inequality to deduce that if A > 0, then the probability that x' is closer to B than x is at least a positive constant.
Theorem G.4. Assume A as defined in Definition G.2. If > 0, then for every x E St with d(xx, BB) > 0. we have
Po Pa(d(x', B) < d(xt B) st 5-) 2 d(xx B))
> 0.
Proof. Let us fix any x+ E S+ with d(xt, ,B > 0. Define the random variable
z = d(xt B) - B), x' ~ Po(. 11,
By Definition G.2 and the assumption A > 0, we know
E 7
E [dd(xt B) d(x,B) > A.
Here we use that A S [[ for every xt E St. Note also that Z is bounded above by d(xx, ,B) > 0.
Applying Markov's inequality, we get
E E[d(x',
7
A d(xt,
p(z > 0) 2 1
d(xt,
d( d(xt, B)
Since 7 >> ( is exactly the event d(x',B) < d(xx,13) ,B), we deduce
A
Po (d(dx' B) d(x* B) 8+ st,s~)
> 0.
d(xt,B
Since the choice of x in S+ was arbitrary (as long d(x+ BB) 0), the statement holds for all such xt.
Theorem G.S. Suppose S+ C 'D' contains some xt that is already close to B fi.e. d(xx BB is small). Once the contrastive loss converges and ensures A 0 (Definition G.2), there is still probability at least Nxx .J d(x', B) < d(xxt11) ,B), meaning x' is better than x in terms of distance.
à > 0 of generating an x' with
Proof. This is an immediate corollary of Theorem G.4: for every xt in S with d(xt, BB) > O, the event d(x',B) < d(x* BB)) occurs with probability at least A/d(xt BB > Thus, even if xt is already close, there remains a nontrivial chance discover an x' that is strictly closer.
Lemma G.6. Suppose there are L private data partie whose datasets merge into
B B: U 1=1
M
B
Let 5 >> O be threshold. In each round t = O,................. O,.., 1, we can choose multiple high-quality samples {x²] } S D' and multiple low-quality samples {x, C D (potentially forming one or more Prompts). Let D(t) denote the synthesized data after round t. If there is a point & E B whose nearest neighbor x" E (Dt)) satisfies || - x:" [| &, then with probability at least 7 > 0 (via at least one Prompt in this round) we can generate a new sample x' E D(1+1) such that
a x'1l S max fo || x x=11, E) (p < 1).
25

---

Contrastive Private Data Synthesis via Weighted Multi-PLM Fusion
Proof. Whenever ||= - x=1 2 ç. we include x" in at least one Prompt's high-quality subset, i.e. x² S C (x²). By Theorem G.4 (the multiple-sample extension of the Markov-based argument), there is probability at least 7 > 0 to generate "better" candidate x' whose distance to æ is at most pl|æ x='l or below Ç. A single successful event of that kind in round t achieves the one-round contraction for æ.
Theorem G.7. Suppose p E (0, 1) and y > 0 are fixed constants, and the merged private dataset B has an initial diameter at most D. Over T rounds, in each round t o...., T 1. if there exists point æ E. B whose current nearest neighbor in D(O) exceeds distance &, then by Lemma G.6 we can contract this point with probability to p p|| - x=)) or below &.
Otherwise, if the distance is already S &. no update is needed.
If
d log())))
+ log 4)
>>
log(1 1/p)
then, with probability at least 1- the final synthesized data D(T) ensures that for every B there exists x' DD(T) satisfying | x'|l S &, thereby implying
Wp(BBD(T) S.
Here, WBB D(T) denotes the p- Wasserstein distance between the distributions (or empirical measures) of B and D(T), often defined as W, Wp(u.v) v) == (infxentee)) ||x yll dr (xx ()) where IQ,, is the set of all couplings of p. and v. In discrete setting, Wp represents the minimal "transport cost" match points in B with those in D(T).
Proof. Consider any point æ whose initial distance to D(t) does not exceed D. In each round = 0.... .,. T 1, we form one more Prompts, each Prompt containing multiple high/low-quality samples. produce new candidate points to update ()++1), Whenever | x=1l 2 & for its nearest neighbor x" E D(t), there is probability y > 0 that at least one of the newly generated candidates achieves distance contraction to ø||æ x"1 or below ç. We call this single-round success.
WeiightedSymlattaeeeeraiiiio
we
Each success reduces ||æ - x=I| by a factor of p < 1. Decreasing from down to & needs about log1/p ( successes. The
factor d arises from geometric/meassue-bbe based arguments in d dimensions. Hence, T must exceed log(D/E)) to have high probability of bringing below g.
[og(1/p))
To ensure all M private points converge simultancously within distance &, we apply union (or Chernoff) bound, allocating
at most T/M failure probability per point, thereby introducing an extra 1 log(M/r) in T. Consequently, if T >>
log (D/E) log( M/r), then with probability least T. every x e B finds some x' DDT) with || x'| S E. log 1//)) implying (B, D(1)) < E.
Comparison with existing methods. In the PE algorithm (Lin et al., 2024b), the convergence can be stated explicilly: one d (og(D/)) log Npriv where the single-round contraction probability is and the contraction
can show that T >>> ratio is log
L
4d Here, L typically denotes the number of candidate samples generated by each call VariationAPI, thus
reflecting how many Gaussian perturbations (in PE) are tested per iteration. These two parameters 1 and - log C 4d arise
from tail-bound analyses of the Var tariationAPI step (which is based on Gaussian sampling), and can therefore be expressed a tractable form (e.g. with probability at least 1/2, one sample out of L improves the distance by a factor of 1 log In contrast, our contrastive-leaming approach employs a distribution Po |t, that originates from a deep or large model, which generally lacks the convenient Gaussian-like tractability. We therefore resort to conservative inequalities to guarantee the existence of constants y 0 and p 1. Intuitively, well-trained contrastive model induces strong alignment on S+, so whenever there is a non-optimal sample x E S+, the model assigns sufficiently high probability mass to generating a better candidate x' closer to the private set B (thus boosting 7). Mcanwhile, by effectively "pushing away" negative samples in S-, the distance between x' and x can shrink by a ratio smaller than the Gaussian Deussian-bbssd making/ p smaller). Hence, if these conditions hold (in practice or under stronger alignment assumptions). then the required number of iterations for our multi-roun convergence can be made significantly smaller than that in PE.
log C (i.e. 4d
26

---

Contrastive Private Data Synthesis via Weighted Multi-PLM Fusion
G.2. Convergence Under Differential Privacy with Added Noise
In the module DP P.rivatteVoiinn(D) Br, Q, σ), we inject Gaussian noise into the statistics derived from the synthesized data D (or D Df) and private data Bş to achieve (e, 6D.. We here assume the single-round p-contraction property (Lemma G.6 its analog) still applies as long the noise does not entirely erase the high-quality sample signal. Concretely: Theorem G.8 (Convergence Under Differential Privacy). Let the algorithm run for T rounds. Suppose:
Each "cluster" or "neighborhood" of private data has multiplicity K > a, so that noise cannot fully eliminate
log (125/6)) (T-1)
We set the noise level V the Gaussian mechanism and advanced composition).
to satisfy (c, 6)DP over T rounds and L private data parties (via
2L
(see Theorem G.7 in the noise-free case)-with probability at Then, i is large enough-on the order of Ō log(D/E) least 1 - T, the output DDT) satisfies both (e, 6)DP and (p D(T) S E.
log( )
Proof. Because each cluster has multiplicity K > a, Gaussian noise cannot entirely suppress the votes in that cluster, so the "single-round contraction" argument (i.e. probability y > 0) remains valid. By the same multi-round convergence analysis (cf. Theorem G.7), we only need to ensure log( M overhead in T. Meanwhile, the selected obeys standard advanced composition bounds for (c, 6)-D,, guaranteeing privacy in each round. Consequently, once T is sufficiently large, contract the distance of every private point to the synthesized set s E, and the algorithm remains (c, 6)-
27


Reviews:
[{"review_id": "rwu8J1p4JT", "reviewer": "Reviewer_gbiW", "summary": "Two of the main strategies for generating private synthetic text or image data are private fine-tuning and private evolution. Private fine-tuning can work well but this requires access to the parameters of a generative model (many of which are closed-source) and the computational resources to fine-tune the model. Private evolution uses API access to a generative model in order to generate private synthetic data. These methods sample some synthetic data, then privately compare the synthetic data to the real data, then query the API to create new data based on the feedback generated by the real data.\n\nThe two main contributions in this work are the weighted multi-PLM fusion and the contrastive prompting. The weighted multi-plm fusion takes advantage of the fact that there are many pretrained language models available and that some of them will be better suited for generating particular types of data. The method developed by the authors (WASP) will start by querying several generative models and as it goes it will start to generate fewer samples from the models that are not performing well for a given task and more samples from models that are performing well for a given task. Contrastive prompting modifies the PE framework by privately measuring which of the synthetic data are most similar to the real data and which synthetic data are least similar to the real data points. This is used to construct contrastive prompts like “generate synthetic data that are more similar to this <good example> and less similar to this <bad example>”.\n\nThe authors perform several experiments and ablations to demonstrate how the various components of WASP contribute to its performance and how WASP compares to baseline methods. The authors also extend WASP to a federated setting. These experiments show that WASP outperforms previous techniques across various metrics.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate these insightful comments from our reviewer which help us improve our work.\n\n**Q: Different noise scale for different function sensitivity considering contrastive prompting.**\n\nYou are absolutely right, and we indeed have applied different noise scales according to different sentitivity values of different methods in our abalation study. Specifically, while the noise applied in WASP using a sensitivity of 4 (see Theorem D.2), the noise scale was halved (from 9.690 to 4.845) for \"w/o comtrastive prompting\" ablation, as the sensitivity was halved, to assure a fair comparison. Thus, histogram sensitivity changes have been accounted for in the ablation. We will clarify this in the final version. \n\n**Q: Good performance of FuseGen and performance origination of WASP.**\n\nThe success of FuseGen relies on the fusion of several PLMs which provides a broader and more diverse knowledge base for training a STM that can generalize better on the real dataset compared to using a single PLM. As shown in Table A below and in [2], FuseGen boosts the performance of the trained STM compared to the best performing single-PLM of the task (Flan-T5 for both tasks in Table A), with larger extent for more common task like IMDb resulting in superior performance. \n\nTherefore, fusing the knowledge from multiple PLMs is the first origination of the performance improvement of WASP.\n\nWhat's more, the addition of private samples also contributes to the final performance of WASP. \nPrevious work ZeroGen [1] explores the performance of STM trained on the synthetic dataset given by a single PLM without the help of private data, which reveals the base level of knowledge \"memorized\" in a single PLM. \nLike shown in Table A, using Flan-T5, ZeroGen achieves a performance around $2.5$% less than Aug-PE for both datasets with private data completely exposed ($\\epsilon=\\infty$). Similar patterns can be witnessed for multi-PLM setting by comparing FuseGen and WASP with $\\epsilon=\\infty$. Therefore, although different PLMs may have different levels of knowledge about each dataset (as shown by ZeroGen results here and in [2]), the involvement of even a small amount of private samples consistently help in further boosting the STM performance.\n\nMoreover, the \"insensitivity to privacy budget\" of our WASP (and Aug-PE) in *Table 6* demonstrates the robustness of our proposed method.\n\nTable A. Comparison of STM performance of different methods. Flan-T5 is used for ZeroGen and Aug-PE as they are single-PLM methods, while $K=6$ is used for FuseGen and WASP.\n||ZeroGen|Aug-PE ($\\epsilon=\\infty$)|FuseGen|WASP ($\\epsilon=\\infty$)|\n|:-:|:-:|:-:|:-:|:-:|\n|IMDb|87.06|89.48|89.07|89.96|\n|Yelp-Rating|57.08|59.62|57.96|62.02|\n\n**Q: Meaning of x-axis in Figure 1.b.**\n\nSorry, the x-axis of *Figure 1.b* should be the task performance (accuracy) of the STM trained on the synthetic dataset. We will add this into our final version.\n\n**Q: Lines in Figure 6.**\n\nAs stated in the caption of *Figure. 6*, top-Q voting ($Q=8$) is used for \"w/o Con\"  whereas $Q=1$ for \"Aug-PE\". Note, to guarantee the same privacy budget, noise scale also increases (to twice its original value) as $Q$ increases from $1$ to $8$. The difference between \"w/o Con\" and \"Aug-PE\" demonstrate the effectivenss of applying top-$Q$ voting with $Q>1$.\n\n**Q: Trade-off between performance and cost.**\n\nYes, WASP can be modified to take into account the API costs by adjusting the importance weighting function of each LLM with their associate API cost. For example, assuming each PLM $k$ costs $v_k$ per query on average, a trade-off function $w_k =U(w_k,v_k)$ can be applied to balance cost and performance and get the final PLM weight $w_k$. To keep $w_k,v_k$ at the same level, we can first normalize $v_k$ to make each $v_k$ range from $0.0$ to $1.0$. $U$ can be selected according to users' needs, e.g. $U(w_k,v_k)=\\lambda w_k + (1-\\lambda) v_k$ with a tunable parameter $\\lambda$.  Finally, all the $w_k$ after adjustments need to be normalized as $w_k=w_k/\\sum_{k'=1}^K w_{k'}$.\n\n**Experimental Designs Or Analyses: Other dataset results with close-source PLMs in *Table 3*.**\n\nWe show results for IMDb on closed-source PLMs in Table B, which also shows the superiority of WASP compared to baseline methods.\n\nTable B. STM performance using $M=100$ with $3$ closed-source PLMs ($K=3$ for WASP) under the same DP setting used in *Table 3*.\n||Only Private|FuseGen|Aug-PE$_{GPT-3.5}$|Aug-PE$_{GPT-4}$|Aug-PE$_{GPT-4o}$|WASP|\n|:-:|:-:|:-:|:-:|:-:|:-:|:-:|\n|ACC|50.00|84.12|84.16|83.28|85.82|86.34|\n\n---\n\n[1] Jiacheng Ye et. al., ZeroGen: Efficient Zero-shot Learning via Dataset Generation, EMNLP2022.\n\n[2] Tianyuan Zou et. al., FuseGen: PLM Fusion for Data-generation based Zero-shot Learning., EMNLP2024.", "claims_and_evidence": "Aug-PE performs poorly when using the “incorrect” PLM - The authors show that the performance of Aug-PE depends on the PLM that is being used and that the best PLM to use for one dataset is not necessarily the best for every dataset. This claim is relatively well supported and is a good motivation for multi-PLM fusion. This is further supported by the ablation study showing that performance improves with the number of PLMs included in WASP.\n\nWASP’s advantage over Aug-PE extends to the federated setting - The authors show that federated-WASP performs better than PreText (which is a version of Aug-PE that works in the federated setting). \n\nContrastive prompting and Multi-PLM fusion are both important to the performance of WASP - This claim is supported by an ablation study summarized in Table 3. However, I am dubious about the result for the importance of contrastive prompting. I have a question about this in the questions section of the review.\n\nTop-Q voting performs better than top-1 voting - Table 5 shows for 2 datasets that performance improves when Q increases.", "methods_and_evaluation_criteria": "The proposed methods and evaluation criteria do make sense for the task at hand", "theoretical_claims": "I reviewed the claim that WASP satisfies (\\epsilon, \\delta)-DP, this uses standard DP results such as the privacy of the gaussian mechanism and composition of multiple gaussian mechanisms. This claim is correct.", "experimental_designs_or_analyses": "I am curious about the ablation results in table 3 and would like to see these results for the other datasets as well if possible. I also have a question about how this experiment was performed in the questions section.", "relation_to_broader_scientific_literature": "Dp synthetic text is a rapidly developing field that has recently become much more practical with the development of LLMs. This work could be an important step that improves private evolution, which is one of the primary techniques being explored for this problem.", "essential_references_not_discussed": "I do not believe that there are essential references not discussed here.", "other_strengths_and_weaknesses": "Strengths\nThe PLM fusion technique is intuitive and well supported by the experiments, I am confident that this technique increases the quality of the DP synthetic data.\n\nThe evaluation of these methods is very comprehensive across several popular datasets and many PLMs\n\nWeaknesses\nI am not sure that the contrastive prompting is beneficial, especially if the change in histogram sensitivity is not accounted for in the ablation.\n\nI am worried that much of the performance gains could come from PLM memorization. If the various PLMs memorized some data sources more than others then this could be the reason multi-PLM fusion is beneficial, not because the PLMs are better suited for the task in some other way. The relative insensitivity to epsilon makes me worry about this more.", "other_comments_or_suggestions": "No other comments", "questions_for_authors": "The x axis of figure 1.b is not labeled or explained in the caption, what is this plot?\n\nFor the ablation study about contrastive prompting, was the noise scale adjusted for the trials without contrastive prompting? The cost of the contrastive prompting is that you must make an additional measurement of the private data to generate the furthest histogram. If you do not apply contrastive prompting you only have to measure the nearest histogram so the sensitivity is reduced and you can make the measurement with less noise. This is not discussed in the ablation section and if it was not done this would not be a fair comparison.\n\nThe fact that FuseGen can perform well for IMDb and Yelp-category without looking at the private data is very surprising, can you explain this?\n\nTable 6 shows that the performance of these methods is relatively insensitive to the privacy budget (especially on IMDb), does this imply that the PLMs may have memorized some of these datasets and most of the performance comes from that?\n\nIn figure 6, what is the difference between the Aug-PE line and the w/o Con line?\n\nThe various commercial PLMs have different costs, could your fusion framework be modified to trade off performance and API costs?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "aO3Ah1eE4T", "reviewer": "Reviewer_Eqwr", "summary": "This paper studies differentially private generation of synthetic data using LLM APIs. The general idea is based a lines of prior works Private Evolution, which generate samples tailored for synthetic data by resampling from the ones that is closer to the private dataset. This work improves upon existing works by 1) less private data enabled 2) low quality synthetic samples 3)model bias of the LLM APIs. The proposed method WASP uses (1) a Top-Q voting mechanism where private samples vote for the nearest and furthest synthetic samples, (2) dynamically weighted LLM-based data generation informed by private histograms, and (3) contrastive in-context learning to enhance the quality and relevance of generated synthetic samples. Experiments show SOTA results compared to Aug-PE when the number of private data is small.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**Q: Lemma issue and results.**\n\nThank you for pointing this out. Lemma D.3 should be added with \"with $\\delta_{total}$ increased to larger than $T\\times\\delta$\".\n\nDenote the $\\delta_{total},\\delta_{iter}$ as the final $\\delta$ and $\\delta$ for each iteration respectively. Therefore, as we perform 5 iterations to collect the total synthetic dataset with 4 iterations related to DP (the first iteration performs zero-shot generation without real sample guidance), the final $\\delta_{total}>4\\cdot\\delta_{iter}$. Therefore, as we applied $\\delta_{iter}=1\\times10^{-5}$ in our experiments, the final $\\delta_{total}>4\\times10^{-5}$ for all PE series baselines and WASP. Moreover, following Theorem 4.3 in [3], using $\\delta_{iter}=1\\times10^{-23}$ instead will guarantee overall $(4.0,1\\times10^{-5})$-DP, which results in a noise scale roughly $2.14$ times as large as the original one used in our original experiments in the paper.\n\nExperimental comparison using the original and new $\\delta_{iter}$ are included in Table C with other setting maintained the same as that for *Table 1* in our paper. These results demonstrate that, under tighter privacy guarantee ($\\delta_{iter}=1\\times10^{-23}$), the performance decrease is just minor, indicating the robustness of WASP and PE baselines.\n\nTable C. Comparison of using $\\delta_{iter}=1\\times10^{-5}$ (original experimental results) and $\\delta_{iter}=1\\times10^{-23}$ (satisfies $\\delta=1\\times10^{-5}$) with $4$ iteration steps related to DP. $\\epsilon=4.0$ is fixed as the final combined DP budget. Experiments are performed using $6$ open-source PLMs with $L=1,M=100$.\n|||Aug-PE$_{GPT-2}$|Aug-PE$_{Llama-2}$|Aug-PE$_{Vicuna}$|Aug-PE$_{OPT}$|Aug-PE$_{ChatGLM3}$|Aug-PE$_{Flan-T5}$|WASP|\n|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|\n|IMDb|$\\delta_{iter}=1\\times10^{-5}$|85.38|85.77|82.76|83.86|85.82|89.00|89.52|\n|IMDb|$\\delta_{iter}=1\\times10^{-23}$|84.88|85.30|82.04|83.52|85.22|88.83|89.18|\n|Yelp-Rating|$\\delta_{iter}=1\\times10^{-5}$|45.28|47.42|54.42|50.81|55.17|58.69|61.21|\n|Yelp-Rating|$\\delta_{iter}=1\\times10^{-23}$|45.03|47.10|54.09|50.47|54.97|58.61|61.05|\n\n---\n\n[3] Peter Kairouz et. al., The Composition Theorem for Differential Privacy, ICML2015.", "claims_and_evidence": "There are two claims. For privacy, the author theoretically proves the differential privacy of the voting mechanism. For utility, the author demonstrate the performance through experiments. The privacy proof is problematic and might require more clarity. See below for details.", "methods_and_evaluation_criteria": "Yes.", "theoretical_claims": "Yes. Issue : Lemma D.3 is supposed to be the advanced composition theorem. But it is not cited correctly. The delta composition is missing. This could significantly weaken the privacy guarantee.", "experimental_designs_or_analyses": "Yes. Note that the private dataset has size 100, which is specifically designed to highlight the advantage of this proposed method.", "relation_to_broader_scientific_literature": "This falls under a new line of research for using LLM APIs to generate synthetic data that is similar as the private dataset. The citation is adequate.", "essential_references_not_discussed": "No.", "other_strengths_and_weaknesses": "I think the overall quality is good except for this mistake in the privacy proof. The contribution is incremental compared to prior work but still great progress towards making the generation quality better.", "other_comments_or_suggestions": "None.", "questions_for_authors": "It would be good to see the experiment results after fixing this delta issue.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "DWmPfWv6he", "reviewer": "Reviewer_rN1N", "summary": "The paper proposes a novel framework called WASP, designed to generate synthetic data that mimics real private datasets while ensuring DP. WASP addresses three key challenges in existing methods: limited private samples, noisy synthetic data, and the risk of selecting the wrong PLM. It uses a Top-Q voting mechanism to estimate private data distributions more accurately, leverages contrastive learning to improve synthetic data quality, and dynamically weights multiple PLMs to mitigate model bias. The framework is tested on six datasets with both open-source and closed-source PLMs, demonstrating superior performance over existing methods.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks very much!\n\n**Q: Fair comparison with equal API budget.**\n\nWe emphasis that, as stated in Lines 105~107 on the left in our paper, no extra query to the PLMs is incurred by WASP compared to single-PLM PE baselines, when they are compared under the same number of required synthetic samples $N$. So we are making fair comparison to Aug-PE with equal API budgets (number of queries) throughout our experiments.\n\n**Q: Validation on generation tasks.**\n\nWe provide results on question answering task with SQuAD dataset in Table D. These results also testify the effectiveness of our proposed WASP for generation tasks and will be added to our final version.\n\nTable D. Evaluation of STM performance (F1) on SQuAD dataset. The same setting is used as that in *Table 1* in the paper.\n||Only Private|FuseGen|Aug-PE$_{GPT-2}$|Aug-PE$_{Llama-2}$|Aug-PE$_{Vicuna}$|Aug-PE$_{OPT}$|Aug-PE$_{ChatGLM3}$|Aug-PE$_{Flan-T5}$|WASP|\n|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|\n|F1|5.41|9.31|7.37|7.84|8.46|8.31|8.20|9.32|11.40| \n\n**Q: Ablation without both contrastive ICL and PLM importance weighting.**\n\nBy removing both components, the final STM performance are $89.05$%,$58.72$%,$35.45$% for IMDb, Yelp-Rating and Openreview-Rating respectively. Comparing with results in *Table 4*, these results show that both components are vital in boosting the final performance of WASP. See full results in Table E1 in https://anonymous.4open.science/r/WASP/re.pdf .\n\n**Q: Comparison with DP-SGD.**\n\nWe provide experimental results for first fine-tuning a single PLM with $M=100$ private samples under the same setting of *Table 1* in our paper with DP-SGD and then use the fine-tuned PLM for generation (\"DP-SGD+Gen\") in Table F (see Table F1 in https://anonymous.4open.science/r/WASP/re.pdf for more results). Results demonstrate the superiousness of WASP compared to DP-SGD method. We will add these into our final version. \n\nTable F. STM performance of WASP and \"DP-SGD+Gen\" with $N=6000,M=100,L=1$.\n|||GPT-2|Llama-2|Vicuna|OPT|ChatGLM3|Flan-T5|\n|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|\n|IMDb|DP-SGD+Gen ($K=1$)|87.44|84.63|84.93|81.47|83.18|89.14|\n||WASP ($K=6$)|||89.52||||\n|Yelp-Rating|DP-SGD+Gen ($K=1$)|50.04|49.95|57.46|55.68|45.79|60.85|\n||WASP ($K=6$)|||61.21||||\n\n**Q: Time cost comparison.**\n\nWe compare the runtime of WASP and PE methods in Table G in response for Reviewer ngfH in below due to response limitation. Results show that, the additional time overhead for WASP per iteration is minor compared to PE methods. Since WASP makes no extra PLM queries compared to PE methods, its runtime increase comes from longer prompts (8 v.s. 1 in PE methods) for generation, and from the additional calculation of the  \"furthest histogram\" for other operations during in-context learning sample selection. \n\n**Q: Efficiency of contrastive in-context learning.**\n\nResults in Table G in response for Reviewer ngfH in below show that increasing the total number of private samples $M$ or task complexity has little impact on WASP' runtime, confirming its efficiency.\n\n**Q: Optimization for user-level DP when data parties contribute more samples.**\n\nFirst, for fair comparison in federated setting, we strictly follow Pre-Text [4], assuming each data party controls no more than 8 samples samples to fit on-device setting. If users contribute more, optimizations like norm clipping [5,6], an off-the-shelf and widely applied technique, can keep noise levels practical by capping the norm of voting vectors $H^n_l,H^f_l$ given by data party $l$ at a preset bound $\\zeta$, preventing user sensitivity $\\Delta$ from scaling linearly with user private dataset size.\nUnder the same $\\zeta$, using more private samples (e.g. 100/user) will not degrade performance compared to using fewer (e.g. 8/user).\n\nMoreover, if a single party controls a large number of samples (e.g. 100), like shown in *Table 1* in our paper, the party alone can achieve good performance using sample-level DP whithout the need of collaborating with others. \n\n**Q: More different architecture PLM results with $K=2$ for testifying PLM-agnostic.**\n\nTable H shows results for $5$ open-source PLMs. These results show that pair-wise performance exceeds individual participating single-PLM alone, supporting our PLM-agnostic claim.\n\nTable H. STM performance of $K=1$ (diagnose) and $K=2$ (others) of WASP with $N=6000,M=100$. \n||GPT-2|Llama-2|Vicuna|OPT|ChatGLM3|Flan-T5|\n|:-:|:-:|:-:|:-:|:-:|:-:|:-:|\n|GPT-2|85.65|86.72|85.91|85.85|86.92|89.33|\n|Llama-2||85.82|85.92|86.23|86.97|89.45|\n|Vicuna|||82.90|85.80|86.18|89.30|\n|OPT||||84.32|86.22|89.32|\n|ChatGLM3|||||86.12|89.63|\n|Flan-T5||||||89.28|\n\n---\n\n[4] Charlie Hou et. al., PrE-Text: training language models on private federated data in the age of LLMs, ICML2024.\n\n[5] Anda Cheng et. al., Differentially private federated learning with local regularization and sparsification, CVPR2022.\n\n[6] Fumiyuki Kato et. al., Uldp-FL: Federated Learning with Across-Silo User-Level Differential Privacy, VLDB2024.", "claims_and_evidence": "The claims made in the submission are generally supported by clear and convincing evidence. However, while WASP performs well across multiple PLMs, the claim of being \"PLM-agnostic\" might be overstated. The ablation experiments primarily focus on a limited set of PLMs, specifically three GPT-based models when testing with K=2 PLMs. This limitation in the scope of PLMs tested raises questions about how well WASP would perform with a broader range of PLMs, including those from different architectures or vendors.", "methods_and_evaluation_criteria": "The proposed method, WASP, and its evaluation criteria generally make sense for the problem.", "theoretical_claims": "No critical errors were found in the provided proofs.", "experimental_designs_or_analyses": "The experimental design in WASP demonstrates methodological rigor. It tests 6 NLP tasks including sentiment analysis, rating/field classification with 6 open-source and 3 closed-source PLMs. However, there are some potential issues:\n1. Comparison Fairness: Aug-PE uses single PLMs, while WASP benefits from multi-PLM fusion; no comparison to Aug-PE with equal compute/API budgets.  \n2. Task Scope: Limited to classification; no validation on generation/sequence tasks (summarization).", "relation_to_broader_scientific_literature": "1. Enhanced PE Methods\n- Prior: PE (Lin et al., 2024; Xie et al., 2024) used Top-1 voting, struggling with data scarcity.  \n- WASP: Introduces Top-Q voting with decaying weights, improving sensitivity analysis (Δ=2 → Δ=4) and enabling robust distribution estimation with limited samples. \n2. Contrastive ICL + DP Synthetic Data\n- Prior: Contrastive ICL improved model responses (Gao & Das, 2024). \n- WASP: First to combine contrastive ICL with DP guarantees, using low-quality samples as negative examples to reduce noise (addressing Xie et al., 2024's \"noisy data\" issue).\n\n[1] Xie, C.,Lin, Z.,Backurs, A.,Gopi, S.,Yu, D.,Inan, H.A., Nori, H., Jiang, H.,Zhang, H.,Lee, Y.T., etal. Differentially Private Synthetic Datavia Foundation Model APIs 2:Text. In Forty-first International Conference on Machine Learning, 2024.\n\n[2] Lin,Z., Gopi, S., Kulkarni, J., Nori,H., and Yekhanin, S. Differentially Private Synthetic Datavia Foundation Model APIs 1: Images. In The Twelfth International Conference on Learning Representations, 2024.\n\n[3] Ye,J., Gao, J.,Wu,Z.,Feng, J.,Yu,T., and Kong, L.ProGen: Progressive Zero-shot Dataset Generation via In-context Feedback. In Findings of the Association for Computational Linguistics: EMNLP 2022, pp.3671–3683,2022.", "essential_references_not_discussed": "As far as I know, no related works that are essential to understanding the (context for) key contributions of the paper, but are not currently cited/discussed in the paper.", "other_strengths_and_weaknesses": "It would be clearer to show the ablation results if the authors can add another comparison without both contrastive prompting and importance weighting when conducting ablation study in Table 4.", "other_comments_or_suggestions": "A more thorough comparison with other DP methods, such as DP-SGD, would offer a clearer understanding of WASP's advantages. Additionally, presenting a time cost comparison between the proposed method and standard PE would strengthen the evaluation.", "questions_for_authors": "1. Is the contrastive learning process efficient in terms of computational resources, especially when dealing with large datasets or complex tasks?\n2. The federated DP analysis assumes ≤8 samples/user for Δ=32 sensitivity. What happens if users contribute more samples (e.g., 100/user)? Does WASP’s noise scale linearly, making it impractical, or are there optimizations?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "4cCcWoPlRf", "reviewer": "Reviewer_ngfH", "summary": "The paper introduces WASP, an approach for generating differentially private  synthetic data by leveraging multiple pre-trained language models (PLMs) in a collaborative manner. They tackle (1) limited private samples, (2) noisy synthetic data, and (3) risky PLM selection. They employ a Top-Q voting mechanism to improve private data distribution estimation, uses contrastive prompts to reduce noise, and dynamically weights PLMs based on their performance.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate our reviewer's insightful comments that help us improve our work.\n\n**Q: Complexity of combining multile PLMs and computational cost & complexity comparison.**\n\nFirst of all, as stated in Lines 105~107 on the left in our paper, no additional queries are made to PLMs under the same number of required synthetic samples $N$ in the WASP framework compared to baseline methods although samples are generated in a iterative manner. As shown in Table G in below, the computational cost of WASP is slightly higher than that of PE baselines, mainly due to increased in prompt length ($8$ in-context samples instead of $1$) and the \"furthest histogram\" calculation.\n\nSecond, the primary computational cost is driven by synthetic sample generation, which accounts for nearly 3000 times the runtime of other operations (including DP Top-$Q$ Voting and PLM Importance Weighting). While this process is unavoidable, users can mitigate the impact of computationally expensive PLMs by choosing faster alternatives. Also, like shown in Table G, for large-scale datasets, WASP's computational cost remains manageable.\n\nTable G. Comparison of computational complexity and runtime (seconds) of WASP and PE series baselines within each iteration (averaged across iterations and also across different PLMs for PE series baselines). \"Others\" includes DP top-$Q$ voting and PLM importance weighting, i.e. line 7~10 in Alg. 1, with the latter beging a negligible time cost tensor normalization operation on the Nearest Histogram vector.\n|||Aug-PE|Aug-PE|Pre-Text|Pre-Text|WASP|WASP|\n|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|\n|||Generation|Others|Generation|Others|Generation|Others|\n||Complexity|$O(N)$|$O(MN)$|$O(N)$|$O(MN)$|$O(N)$|$O(MN)$|\n|IMDb|$L=1,M=100$|1.341$\\times$10$^{4}$|5.846|-|-|1.970$\\times$10$^{4}$|6.090|\n||$L=10,M=300$|-|-|1.348$\\times$10$^{4}$|7.153|1.963$\\times$10$^{4}$|7.668|\n|Yelp-Rating|$L=1,M=100$|1.539$\\times$10$^{4}$|5.468|-|-|2.175$\\times$10$^{4}$|5.528|\n||$L=10,M=300$|-|-|1.561$\\times$10$^{4}$|7.376|2.235$\\times$10$^{4}$|7.608|\n|Openreview-Rating|$L=1,M=100$|1.268$\\times$10$^{4}$|5.655|-|-|1.949$\\times$10$^{4}$|5.871|\n||$L=10,M=300$|-|-|1.215$\\times$10$^{4}$|7.713|1.982$\\times$10$^{4}$|7.911|\n\n**Q: Other data modality.**\n\nWe believe that WASP framework can also be extended to other modalities. WASP's key components, including DP Top-$Q$ voting, PLM importance weighting, are general operations which can be directly applied to other data modalities. However, the process for synthetic data generation differs by modality. For example, for image generation [7,8], the applied models and APIs are different from text, with diffusion models and RANDOM_APIs oftenly used to modify existing samples with a hyper-parameter controling variation degree. Therefore, non-trivial adaptations should be made for the generation processure. Similarly, for tabular samples [9,10], non-trivial modifications are also required for crafting proper generation prompts and pipelines.\n\nAs a case in point, the original PE method [7] focuses exclusively on image data, and Aug-PE [11] makes non-trivial adaptations with carefully designed generation techniques to generalize it onto text data. Given the extensive work required, we think generalizing WASP to other modalities deserves a separate paper/papers.\n\n**Q: Further optimization on privacy budget allocation.**\n\nWe thank our reviewer for pointing out an interesting future step for our work. Since our primary focus lies in the fusion of multiple PLMs for private synthetic data generation, we decide to leave this as a possible future advancement.\n\n---\n\n[7] Zinan Lin et. al., Differentially Private Synthetic Data via Foundation Model APIs 1: Images, ICLR2024.\n\n[8] Kecen Li et. al., PrivImage: Differentially Private Synthetic Image Generation using Diffusion Models with Semantic-Aware Pretraining, USENIX Security2024.\n\n[9] Mikel Hernandez et. al., Synthetic data generation for tabular health records: A systematic review, Neurocomputing2022.\n\n[10] Eugenia Papadaki et. al., Exploring innovative approaches to synthetic tabular data generation, Electronics2024.\n\n[11] Chulin Xie et. al., Differentially Private Synthetic Data via Foundation Model APIs 2: Text, ICML2024.", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "Yes", "theoretical_claims": "Yes", "experimental_designs_or_analyses": "Yes", "relation_to_broader_scientific_literature": "Using LLM for data publishing", "essential_references_not_discussed": "Yes", "other_strengths_and_weaknesses": "（1）Pros\nThe paper introduces a novel framework (WASP) that combines multiple PLMs to generate high-quality DP synthetic data, addressing key limitations of existing PE methods.\n\nThe use of Top-Q voting and contrastive prompts is innovative and effectively improves the quality of synthetic data while maintaining privacy guarantees.\n\nThe paper provides a thorough theoretical analysis of the differential privacy guarantees of WASP, including proofs and sensitivity analysis.\n\n（2）Cons\n\nWhile the framework is effective, the complexity of combining multiple PLMs and the iterative nature of the process may raise concerns about scalability, especially for large-scale datasets or when using computationally expensive PLMs (e.g., GPT-4).\n\nThe paper does not provide a detailed analysis of the computational cost or time complexity of WASP compared to baseline methods.\n\nThe paper focuses exclusively on text data, and it is unclear how well WASP would generalize to other types of data (e.g., images, tabular data). \n\nThe paper assumes a fixed privacy budget  across all iterations. However, in practice, the allocation of the privacy budget across iterations could be optimized to further improve the quality of synthetic data.", "other_comments_or_suggestions": "NA", "questions_for_authors": "See cons.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: This paper proposed an algorithm WASP that improves private evolution for differentially private synthetic data generation. Instead of querying a single LLM API, WASP querying multiple LLM APIs and use a carefully designed mechanism to balance the APIs. 

This paper received borderline scores (3,3,3,4) and all reviewers in favor of acceptance. DP synthetic data and the private evolution family algorithms is a timely and important topic. WASP shows empirical improvement in experiments, and releases code.

Reviewers also raised concerns on the DP theory (reviewer Eqwr), fair comparison on computation (reviewer rN1N) and number of data generation setting (review Eqwr) in experiments, and the source of the improvement, especially the concerns on memorization from multiple pre-trained models (reviewer gbiW). All reviewers acknowledge reading the authors' response, and some of the reviewers raised the scores as the authors successfully address the main concerns. I would strongly encourage the authors further clarify on these important issues in the next version, and provide more clarification to address concerns on number of data generation setting (review Eqwr), and memorization from multiple pre-trained models (reviewer gbiW).
Decision: Accept (poster)