PE-SGD: Differentially Private Deep Learning via Evolution of Gradient Subspace for Text
Authors: Tianyuan Zou
Zinan Lin
Sivakanth Gopi
Yang Liu
Ya-Qin Zhang
Robert Sim
Xin Deng
Sergey Yekhanin
Venue: ICLR 2026 Poster (2026)
Keywords: Differential Privacy
Private Evolution
Generation Model
Primary area: alignment, fairness, safety, privacy, and societal considerations

Abstract: Differentially Private Stochastic Gradient Descent (DP-SGD) and its variants like DP-Adam ensure data privacy by injecting noise into per-sample gradients. Although effective with large private datasets, their performance degrades significantly when private training data is limited. Recent works leverage public data to learn a gradient subspace and project noisy private sample gradients on to this subspace, achieving improved performance. However, they have overlooked two crucial aspects: the limitation of using a fixed projection subspace throughout training and the importance of choosing where to inject noise. Therefore, we propose Private Evolution aided Stochastic Gradient Descent (***PE-SGD***), a differentially private training framework effective for scenarios with limited private data. ***PE-SGD*** uses an evolutionary strategy to update the gradient projection subspace during training process. We also identify a more effective noise injection point for better alignment between approximate DP-protected gradient and real private gradient. This enables ***PE-SGD*** to outperform DP-SGD and other baselines, particularly in the regime of limited private data and small privacy budget.

Published as a conference paper at ICLR 2026
PE-SGD: DIFFERENTIALLY PRIVATE DEEP LEARNING VIA EVOLUTION OF GRADIENT SUBSPACE FOR TEXT
Tianyuan Zou Zinan Lin?, Sivakanth Gopil, Yang Liu', Ya-Qin Zhang', Robert Sim*, Xin Deng'. Sergey Yekhanin"
Institute for AI Industry Research (AIR), Tsinghua University, "Microsoft Research, 'hhe Hong Kong Polytechnic University, *Microsoft
zty22emails 1s. tsinghua. edw..dd (zinanlin, rsim, xinde, yewkhaninnemmcrrsoffff comm si sivakanth@openai. com, yang-veronica. liu@polyu. aku. zhangyaqin@air, tsinghua. edu. cn Work done during internship at Microsoft Research. *Work done while working Microsoft Research.
ABSTRACT
Differentially Private Stochastic Gradient Descent (DP-SGD) and its variants like DP-Adam ensure data privacy by injecting noise into per-sample gradients. Although effective with large private datasets, their performance degrades signif- icantly when private training data is limited. Recent works leverage public data to learn a gradient subspace and project noisy private sample gradients on to this subspace, achieving improved performance. However, they have overlooked crucial aspects: the limitation of using a fixed projection subspace throughout train- ing and the importance of choosing where to inject noise. Therefore, we propose Private Evolution aided Stochastic Gradient Descent (PE-SGD), a differentially private training framework effective for scenarios with limited private data. PE-SGD uses an evolutionary strategy to update the gradient projection subspace during training process. We also identify a more effective noise injection point for better alignment between approximate DP-protected gradient and real private gradient. This enables PE-SGD to outperform DP-SGD and other baselines, particularly the regime of limited private data and small privacy budget. Code is open-sourced at /tttps://githu:: cow Booom/Linnayydia/PE....
INTRODUCTION
Large language models (LLMs) have rapidly emerged as powerful paradigm, enabled by training massive amounts of high-quality data (OpenAT 2025 Comanici et al. 2025
DP-SGD
Different Noise Placement
Ours
Fixed Projection Subspace
f Distance [4)
Yang et al. 2024). Their success has reshaped natural language processing and spurred widespread adoption across industries such finance (lacovides al.) 2024), healthcare (Goyal et al.
40
20
2024) and personal assistants (Chew (022).. However, this progress raises urgent concerns about training-data privacy, as LLMs are known to memorize and potentially leak sensitive information from their training corpora (Carlini et al. 202T
6
8
10
Training Steps
Lee et al. 2022 Ramakrishnan & Balaji 2025).
Figure 1: la distance between ap- proximate DP-protected gradient (with noise) and private gradient (without noise) when using dif- ferent noise placement and fixed projection subspace compared to our proposed method, showing that both components need to be care- fully selected. More results are in- cluded in Fig, I0 in Appendix [.] See Section for detail.
Differentially private stochastic gradient descent, DP- SGD (Abadi et al., 2016). is classical method to address this concern by guaranteeing formal Differential Privacy (DP) (Dwork) 2006) and has recently shown great success in LLM training (McKenna et al., 2025). Its effectiveness, however, depends on sufficient training data. Due to its noisy perturbation on per-sample gradient, DP-SGD often largely underperforms vanilla non-pri non-private SGD with limited private samples (McKenna et al. 2025 Feldman et al. 2020), e.g. a few hundreds (Zou et al.
2025

---

Published as a conference paper at ICLR 2026
DP-SGD
PE-SGD
Dataset !
Private Dateset
Add Notso
DP Projection
Selected
Gradient
DP
D(-1
Sample
Dataset D2
Synthatic
Approximate Gradient
Generate
M++1
m2
M|+1
Figure 2: Overview of PE-SGD compared to standard DP-SGD (Aaad et al. 2016). Unlike DP-SGD, which updates m with noisy private gradients, PE-SGD projects them into an evolutionary synthetic gradient subspace under DP and updates m with the resulting approximate gradients.
Gradient projection (Yu al. 2021 Zhou et al. 2021) has been proposed as a solution. This method addresses the challenge of high-dimensional noise in DP-SGD by projecting gradients onto a lower- dimensional subspace. This takes advantage of the empirical finding that gradient updates often lie in much smaller subspace than the full parameter space (Gur-Ari et al.) [018 Li et al. 2020: Yu et al. 202Tp. The projection goal to better approximate private gradients with non- non-private ones. In these works (Yu et al. 2021 Zhou et al. 2021), the eigenspace of the non-private gradients (derived from non-private samples) serves as the projection subspace.
Although this direct idea achieves better performance compared to DP-SGD, especially in the high privacy regime (tight DP budget with small e) on image data, two major challenges remain: (1) Fixed projection subspace is incompatible with dynamic training. Like shown in Fig. I using fixed non-private dataset ("Fixed Projection Subspace") results in increasing lg distance between approximate DP DP-protected gradient and private gradients as training progresses. This indicates that fixed projection subspace, i.c. fixed non non-private dataset, is not suitable across all training iterations, limitation unexplored in existing literature (Zhou et al., 2021 Yu et al.] (02T). Moreover, their reliance on large public dataset which itself has the concern of being sensitive (Tramrr et al. (024)) (2) Lack of thorough exploration for the noise placement in the projection. Existing projection methods differ in DP noise adding point. Some perturb gradient approximation coefficients Yuu et al.
2021 while others perturb private gradients (Zhou et al. 2021). or gradient residual term (Yu et al.
2021 However, the rationale behind these choices remains unclear, despite their differing effects on performance (see "Different Noise Placement" in Fig. D..
To address these demanding challenges, we propose Private Evolution aided Stochastic Gradient Descent (PE-SGD), a differentially private training framework effective for scenarios with limited private data and tight privacy budget. (1) For projection subspace that evolves during the training process, we iteratively update the non-privatt dataset using the updated generative model after each parameter update. Inspired by prior DP synthetic data works (Lin et al. 2024 Xie et al., 2024], new synthetic samples are generated based on one of the selected non- non-private samples from current iteration using the updated model. This process starts with synthetic samples randomly generated by the (non-trained) model before training. This evolution reduces the l2 distance between approximate DP-protected and real private gradients (see "Fixed Projection Subspace" in Fig. D. Notably, unlike previous works (Yu et al. 2021 Zhou et al. (02T), we avoid the need for a large public dataset. (2) For noise placement selection, we empirically examine various options in our projection method and recommend adding the DP noise to the final projection coefficients, which yields smaller ε approximation error (see "Different Noise Placement" in Fig. T and significantly higher cosine similarity (see Fig. (0) between the approximate DP- or-protected gradient and real private gradients. Also, unlike DP-SGD, we avoid direct updates using private gradients. Our contribution are summarized as follows:
1) We propose PE-SGD, a effective with limited private data and tight budgets (small e) where DP-SGD fails. Basing closed-form analysis identifying a principled gradient projection, we privately evolve the projection basis along with model fineingning for improved adaptation and empirically select better noise injection point to better align DP-pretected approximate gradients to private gradients.
training framework robust to DP noise, particularly
2) Extensive experiments in three distinct pre-traine trained models and three challenging datasets (disjoint from veetraininn) demonstrate the consistent superiority of PE-SGD, particularly in learning hard samples with large initial losses before DP training, achieving greater loss reductions.
2

---

Published as a conference paper at ICLR 2026
2 RELATED WORK
Differentially Private Training. Differentially Private Stochastic Gradient Descent (DP- SGD) (Abadi et al.] 2016) is the standard and widely applied method for training models under differ- ential privacy (DP), achieving privacy by adding isotropic noise sample-level gradients (McKennal et al. 2025p. However, DP-SGD performs poorly with limited amount of private data compared to non non-private private vanilla SGD {Zhou al., 2021). To improve utiliiy, approaches such as Projected DP-SGD (PDP-SGD) (Zhou et al. (021) and Gradient Embedding Perturbation (GEP) (Yu et al.
202Th have been proposed to reduce the dimension of the injected noise through gradient projection, while Sparsified Model Perturbation (SMP) (Hu et al. 2023) sparsifies model parameters directly. Although proven effective, a principled gradient projection method remains unrevealed.
Differentially Private Synthetic Data. With the rise of generative models such Large Language Models (OpenAT, 2025) Yang et al. 2025) and Stable Diffusion (Rombach et al. 2022) recently, synthetic data has attracted increasing attention
alternative in scenarios where large-scale, high- quality real data is scarce or sensitive. However, the long-recogen recognied low-quality issue of synthetic
data (Ye et al. 2022 Gao et al. 2023 Zou et al. 2024) limits its direct use for model training or fine- tuning. To address this, private domain data has been used as generation guidance to align synthetic data distributions. Yet, traditional DP-SGD- based methods (Bommasani et al. 2019 Mattern et al.) Putta et al., (2023) are often impractical in resource-constrainee settings (Abou Baker et al. Wornow et al. 2023) and infeasible with powerful closed-source generative models (Xie et al. Hou et al. 2024). More recently, Private Evolution (PE) (Lin et al. (2024) and its variants (Xie
2022
2024
2024
al.
2024]|oo Hou et al. 2024] Zou et al.) 2025) Lin al., [2025] Wang et al. | (0255) refine synthetic
data to resemble private domain data in the embedding space using only generative APIs. However,
they do not directly optimize samples for training or fine-tuning.
PRELIMINARIES
Differential Privacy (DP). Two datasets D and D' are referred to as Neighboring Datasets (D ~ D') if they differ in single entry. If Pr[M(D) E E] < e Pr[MM(D' E E] + 8 holds for any neighboring datasets D. D' and any output subset E a random mechanism M, then M satisfies
(c. 8DP (work 2006 Note that any post-processing the output of (c, 5)DD does not incur additional privacy loss Owork et al. 2014).
Gaussian Mechanism. For any e 0, (0, 1), following Gaussian Mechanism (Dwork 2006), (e, -DP can be satisfied by adding Gaussian noise following (0, o²) where the noise level ar is decided on the basis of e, 5. We use the PRV Accountant from Gopi al. (202I) for the calculation of a in our work.
4 METHODOLOGY
4.1 PROBLEM DEFINITION
In this paper, our goal is to train generative model m that better adapts to the user-selected private
distribution (task) from pre-trained generative model m (0), To be specific, given a private dataset B of size M, which at the same time defines a task, the final goal is to fine-tune pre-trained model (0) for learning the private dataset distribution while guaranteeing (c, 8)DP of B. In this paper, we mainly consider limited private data setting, with M < 500 following Zou et al. (2025) The fine-tuned model is evaluated on private dataset A reserved for evaluation only.
4.2 OUR METHOD PE-SGD
1) Gradient Projection. Since DP-SGD (Abadi al. 2016) suffers from severe performance degradation with limited private samples, gradient projection methods (Zhou et al. 2021 Yu et al. 2021) have been proposed to reduce the noise dimension. The core idea is to approximate private gradients H E Rpx M under DP protection using non-private gradients G E R a non non-private dataset D of size N, with p being the number of trainable parameters of current model m.
3

---

Published as a conference paper at ICLR 2026
Given set of private gradients H [h1,.... had R non--onnnnnnn non-private SGD updates the model
with their average h. Therefore, with a set of synthetic gradients G = gg...... 9N E R N serving
as the projection subspace, goal is to minimize the &g distance between the approximation and h. This can be formulated as:
M h2
min ZERV
Gz 1112, with h Σ
H
(1)
where = E EN is the coefficient vector. Solving Eq. ① equivalent to least-squares regression in regime (since N < p) where we can assume that the N x N matrix G' G is invertible Therefore, the least-squares solution (which can be obtained by setting the gradient of objective in Eq. [ to zero) is given by:
the
M Σ M 1=1
x == GG"
E [GG 1=1
1
H
(2)
The final projection function can therefore be written as:
1 M GG H) M
Gz
Σ
(3)
Current method PDP-SGD Zhou et al. 2021) instead proposed to project as 9 EE" (E H.,) with E E RPxk spans the top-k eigenspace of GGT while GEP defines E as the k components of the left singular vectors corresponds to the top-k singular values of G and additionally adds a residual
term R = H EE H to better correct the bias resulting in g = EET (Σ H.s) + E R. Note that these gradient projection methods are equal to ours given in Eq. 3 if it does not use the top-k eigenspace but the whole space. Proof is included in Appendix B.3 Intuitively, however, restricting to a subspace inevitably incurs a performance loss.
M
2) Existing Key Challenges. With the principled gradient projection given in Eq. 3 several key challenges still exist.
[CI] How to improve fixed datasets for better gradient projections? Beyond projection design, using a proper non-private dataset 'D for the projection subspace is also crucial. Intuitively, if D differs greatly from the private dataset 13, e.g. D is overlaaalgg with B in distribution, the model cannot effectively learn the knowledge in 13, which could lead to poor training results. Prior works (Yu et al., 2021 Zhou al. 202 I) rely on fixed (public) dataset as D for the projection subspace G. But as training progresses, this fixed base causes the & distance between the real private gradient and the approximate Dr-ptetected gradient to grow (see "Fixed Projection Subspase" in Fig. D
[C2] Where should the DP noise be integrated in the projection?' Another interesting unresolved issue is the injection place of the random noise for DP guarantee. DP-SGD (Abadi al. 2016) and PDP-SGD (Zhou et al., 2021) added noise (of shape RP) to aggregated per- sample private gradients H. after clipping: while there exists other works like GEP (Yu al. 2021 (and its follow up Gu et al. (2025)) that adds noise (of shape R- after projection in the projection space. However, the rationale behind these noise placement selection remains unclear, while there exist multiple possible noise injection places within the gradient projection process in Eq.
3) Our solutions. We address these two key challenges that elucidate our proposed method, PE-SGD.
[SI] Synthetic Dataset Evolution. Unlike prior work that relies on a fixed D (Zhou et al. 2021
Yu et al.) (2021, we construct D that evolves during fine-tuning to adapt to the updated model m. Inspired by PE Lin et al. 2024 Xie et al. 2024), and starting from a pre-trained generator (0) with its randomly generated synthetic samples (termed as RANDOM_ _ () Xie et al. (2024)), we prompt each updated m to produce new samples for the next iteration by varying carefully selected seed samples Doced At each iteration, each element 2; in z E R can be seen as a score of synthetic
In practice, to ensure numerical stability, add small & regularization term to the least-squares problem. Thus invert (G" G n7) with 17 = 10 instead.
4

---

Published as a conference paper at ICLR 2026
Algorithm 1 PE-SGD
Input: Pre-trained generative model m (0) with parameter di empty synthetic dataset D + 0: private datasct B of size M; number of iterations for updates T; number of synthetic samples to generate N per iteration; number of synthetic sample variation fold L; Poisson sub-sampling ratio / for private samples; learning rate y,; DP parameters e, 6;
Output: tunee or trained PLM m.
I: D f 0, D + SynDataGeneration ( seed N, L, ml )
2: m + fine-tuned m (9) using D.
3: Noisecalculation( c, 6, T. B following Gopi al. 2021)
4: for
to 7'
5: B&0) <- PoisssonSbSamplingg B, B ); .M < |.
6: G + 91. gN with g. being the sample-level gradient of x, E D on m.
7:
H + h , h s with h, being the sample-level gradient w, E ( on m.
8:
(G' (G"
o
9:
é + & ++1 7- [G=) MM] to update m.
10: aced < SampleSelection( N. L. D), D SynDataGeneration D seed N, L, m.)
11: end for
sample x; given by private samples. We therefore select top-K seed samples with probability proportional to | since gradient related information can also be extracted from negative 2;- We then expand each seed sample into L - 1 variants via a one-shot prompt (termed as VARIATIONAL_APIO in Xie et al. (2024). ) Gaussian noise NO, xxTx) is added to prior to sample selection for privacy consideration Detailed prompts for generation are included in Appendix C.2 Note this evolutionary design removes the need for large public datasets. Moreover, with synthetic dataset of small size (200 in our experiments) serving as the projection subspace, we further avoid the need of using top-k eigenspace and exploits the entire projection subspace for gradient approximation.
[S2] Systematic Study of the DP Noise Addition Place. Prior works select their own way of adding noise arbitrarily. However, from Eq. 2 we identify three natural options: (1) adding noise of shape R2 to the aggregated private gradient Σ H.g. (2) adding noise of shape R N to the gradient inner product -M hj=1 !G H.gg or (3) adding noise of shape N to the final projection coefficient
M I (G"T GGT H)).. Our empirical study (Fig. 8 shows that the third option consistently Ej=1(
yields the best performance.
4) Feasibility Discussion Naively computing GT G and GT H with G RRXN H e R in PE-SGD in Algorithm [ can look infcasible for large models since all the per-sample gradients do not even fit in the GPU memory, so we cannot materialize the matrix G H. There to get around this: (1) We can use parameter efficient fine-tuning methods like LoRA (this is the approach we take in this paper) which makes p relatively small. Therefore, per-sample gradient matrices G, H and the rest of the matrix multiplications can be computed directly. (2) If we want to do full fine-tuning, the per-sample gradient dot-products G G and G H can be efficiently obtained using GhostSuite WWang et al. 2025a) which computes gradient dot dot-products between all pairs of data points large batch with a single backpropagation.
ways
5) Overall Framework We summarize our PE-SGD algorithm in Fig. 2 and Algorithm I Functions applied in Algorithm I are included Algorithm 3 in Appendix B. Notice that normalization instead of clipping is used in Algorithm [ for sensitivity control before column aggregation, as both techniques yield similar results while normalization avoids tuning the typer-paramett clipping threshold C' used in clipping. Also, we use the same sub-sampling strategy as DP-SGD, ensuring the only difference lies in how the sub-sampled private batch (3 for each iteration i is utilized. A full privacy analysis of PE-SGD is provided in Appendix B.2 Last but not least, although Algorithm | is presented in the form of vanilla SGD (line 9., it is fully compatible with advanced optimizers such as Adam W. simply by feeding our approximate gradients into them. A mathematical convergence analysis of PE-SGD compared to previous methods are included in Appendix D showing that the convergence speed of PGD is no slower than that of previous methods.
follow "We directly follow Gopi et al. 2021) for the calculation of the noise scale a. https://gitub..om//aatcee
GhosttSite
5

---

Published as a conference paper at ICLR 2026
5 EXPERIMENTAL RESULTS
Table Performance comparison of the trained model m using PE-SGD and baseline methods, Best and second best results (except SGD and DP-SGD) are marked.
PubMed
Congressional Specch
bioRxiv
Method
c
8
00
30
Next Token Prediction Loss ())
Base Model
2,9341
3,8690
2.5311
SGD
=0.007=0.0001
2.60784g.008:
2.2756=00.0m =0.0001
DP-SGD
2.3731
2..49700.0037
3.1080
2,6868 =0,088
2.3768
2.2887=0.0021
Aug-PE
2.5817+0.11 1557
2.6070=0:2190
3,0090ga.1000
3,0120-.0.10s
2.4672++0.01d
2.4672.20.0:60
POPri
2.9344-20 0025
2.948720-0an 20.0208
2.9487
3.8656
3,8700
2.6178
00s0
2.5309220.003
PDP-SGD
2.2502
2.2482
2,8025
10.012
2,8000
20.011
2.3325
10:00052
2.3307 10.0011
GRP
2,1164 0525
2.2248
10.0141
2.9103
20.0263
2.7822
-0.1470
2,4300
a.0. coss
20.099
InPro
2.5268 20.0543
2.5229 #0.0025
2.5229
3,4693
#0.1381
3,4728
10.1433
2.4079
#0.023s
2.4081 #n.0206
PR-SGD- PR-SGID-FixSampe FixSample
2.2497 00977+0.000 0091
20.0047
2.8024
40.024
=0.0107
2.32
+0.000+000000
2.9240+0000662
PE-SGO
20.990
2.1839 20.0131
2.7532
2.7468
000009
2.3178
#0.0018
2.3165
Next Token Prediction Accuracy (ACC) (1)
Base Model
0.5217
0.4276
0.4981
SGD
0.5494_0..00m
0.4661=20.0013
0.5145 11.652000000
DP-SGD
g0.000
0..461
0.4492g00.000
0.4656_0.0000
+0.000
0.5130=00.005
Aug-PE
20:003
0.5201 20-046
0.4450.go.0110
0.4447-000013
14982220.0015
0.4982.20.0005
POPri
0.5221+0.000 00.0015
0.5192=0.00s9
0.4276
0.4276-0.0018
0,4930
#0.002s
0,4981 +0. #0.0005
PDP-SGD
0.5382 0000
0.5385
0,4546
0,4566-0.000
0.5092
0.5093
GEP
0,534420.00m
0.5423.20-000
0.4549 20.007
0.4566
-0.0000000000
0.5035
3050035a0.000
0.5095
InPro
0.5316 0015
0.5315 10:001
0.4331 2000027
0.4330
- 0028
0.5045 1..04455..001
PE-SGI-FixSamppe
0,5413 0007
0.5420.0.0010
0.4548
20.0005
0.4561
00.0024
0.509222n.oo0s
a..005
PE-SGD
0.5435 0.5435=0.000 00.0000
0.5436 :0.0005
0.4071=0.0011 #0.0011
0.4579
=0.0021
0.5095 0.5095 +0.0000
0.5097 #0.0044
5.1 EXPERIMENTAL SETTINGS
Pre-trained Language Models. Three pre-trained genrative language models are used in exper-
iments: Soven2.--3-IInsttue (Qwen2.5) Owen-TTean 2024) released on September 19th 2024; JJama-3.2-3B-Instmu (Llama3.2) (Dubey et al.) 2024 released on September 25th_ 2024; and GPT2 RRafford al. 2019) released on November 5thh 2019. We use instruction fine-tuning in our models.
experiments for the first 2 models as they are instruction-fine-
Tasks and Datasets. Considering the release date of the above models, we carefully select real- world data samples (used as private samples in both training dataset B and evaluation dataset .A) that are created after the model release dates to ensure that these models have not seen them. To be specific, three datasets are used in our experiments: 1) PubMed (NLM (U.S.) 2025) dataset containing abstracts of professional biomedical literature completed from December 2024 to January 2025; 2) Congressional Speech (Hou et al., 2025) dataset containing public speeches extracted from government institutions of the US, UK and Canada from January 2025 to June 2025; 3) bioRxiv (Hou ct al. (025)) dataset containing abstracts of articles on bioRxiv website, a pre-printing server for biology created from January 2025 to June 2025, More details are included in Appendix C.I
In this work, we compare with five baselines: 1) DP-SGD (Abadi et al. 2016) which is the basic DP training method that is widely applied; 2) Aug-PE (Xie et al. 2024) which generates DP synthetic data that further exploited for (normal, non-DP) supervised fine-tuning: 3) POPri (Hou ett al. 2025) which exploits Direct Preference Optimization (DPO) for fine-tuning on DP synthetic data; 4) PDP-SGD (Zhou al. 2021): 5) GEP (Yu et al. 2021I. We also compare with "InPro" which adopts a biased projection method using gradient inner products with 9 = M Σn= GiG H) )..g We further compare with important variant or PE-SGD, namely "PE-SGD-FiXS "iGD-FixSample" that performs the same Algorithm| Tutt without updating D (line 10), to demonstrate the power of synthetic sample evolution.
Baselines.
Evaluation Metrics. As our goal is to fine-tune a generative model that better adapts to the private data domain, we use the most commonly applied evaluation metrics in language model training as

---

Published as a conference paper at ICLR 2026
0
0.4
0.10
Base Model DP-SGn
&oLoss
-2-
0.2
Density
0.2
PE 5GD
Density
0.05
0.0
0.0
0.00
5
10
15
0.25
0.50
0.75
10
15
0.25
0.50 0.75
None-trained Loss
None-trained ACC
Loss
Next Tokan Accuracy
(b)
Figure 3: (a) Per-sample loss and accuracy (ACC) differences between PE-SGD and DP-SGD with each point corresponding to one sample. The x-axis shows the initial loss on m) and the y-axis shows the per-sample loss accuracy difference after training (PE-SGD minus DP-SGD). (b) Loss and accuracy (ACC) distribution change before and after training with PE-SGD and DP-SGD. Qwen2.5 used for Congressional Speech.
Table 2:
Average loss change of the top 10% samples with the largest initial loss on base model m)
after fine-tuning using different methods. Qwen is used for Congressional Speech.
Baselines
PE-SGD
PE-SGD-FixSample
PDP
GEP
DP-SGD
Aug-PE
InPro
POPri
A Loss
-3.49995 -3.41537
-3.32709
-3.01130
-2.03666
-1.67257
-1.28827
-0.00034
our evaluation metrics, namely, next token prediction loss and accuracy. We refer to them as Loss and Accuracy (or ACC) respectively in the following sections for brevity.
Learning Hyper-parameters. Unless otherwise stated, use M 400, 3 0.2, =10, 200, -= 2 and guarantee (1.0, D-D for LoRA fine-tuning in our experiments. More detailed hyper-parameter settings are included in Appendix C.3
5.2
MAIN RESULTS
We first comprehensively evaluate the loss and accuracy of the fine-tuned generative model m the evaluation private dataset A in Table [ Note when co, DP is not guaranteed as a = 0.0, whereas norm clipping or normalization is still performed respectively. Results show that PE-SGD outperforms all other methods when = 1.0, achieving lower loss and higher accuracy, indicating that m learns more about the private distribution while preserving data privacy under tight DP budget. For it's expected that DP-SGD achieves better results as it directly trains on norm-clipped private samples gradients. On the other hand, "PE-SGD-FixSample most baselines, making it good substitute under resource limited settings, as it saves the cost of iteratively updating the synthetic dataset D.
also competitive compared to
POPri does not exhibit good performance under small private training dataset setting (M 400), as it is hard to obtain reliable positive-negative sample pairs with limited private data. Aug-PE also performs unsatisfattory under this small private data setting, similar that shown in Zou et al.) (2025). Other gradient projection methods do not guarantee a consistent better performance than the basic baseline DP-SGD, testifying that their projection method is suboptimal (InPro) or has some information loss when using only an eigen sub-space (PDP-SGD and GEP) even with a residual term for gradient estimation bias correction (GEP).
5.3 OBSERVATIONS AND HIGHLIGHTS
Better Solving the Long Tail Problem. Learning from private data aims to extract unique knowl- edge, particularly from hard "long-tail" samples, i.e. samples with a large initial loss under the base model (0) that are often most valuable for downstream tasks. While DP constraints prevent perfect learning, the goal is still to maximize information extraction from those samples within the privacy budget. To evaluate this, we track per-sample loss and accuracy, focusing on these long-tail samples.
In Fig. we compare PE-SGD with the standard baseline DP-SGD on private evaluation set A. Fig. 3(a) shows per-sample loss and accuracy differences (PE-SGD minus DP-SGD) after training, demonstrating that PE-SGD yields lower loss (negative values) and higher accuracy (positive values), with stronger improvements samples with high initial loss under m (!) This indicates that
7

---

Published as a conference paper at ICLR 2026
PE-SGD better mitigates the long-tail problem, resulting in broader knowledge coverage. Fig. 3(b) further confirms this with a leftward shift in per-sample loss distribution.
In addition, Table 2 shows average loss reduction the 10% hardest samples with the largest initial loss under m ( PE-SGD outperforms all methods, and its variants surpass DP-SGD, underscoring the importance of using the whole gradient subspace for projection.
Better Per-step Update. Overall, PE-SGD out- performs DP-SGD. To understand the source of this advantage, further compare their be- haviors at the level of single training steps by performing single-step updates with DP-SGD and PE-SGD along the same vanilla SGD fine- tuning trajectory, and evaluating the updated models using A. Results are shown in Fig. 4 which demonstrate that, within each updation step, PE-SGD decreases loss and increases accu- racy larger extent compared DP-SGD, even when the model is near convergence at the later steps. See Appendix D.9 for more results.
3.75
SGD
0.47
Ours
0.46
0,4090
3:25
2.627
0,4689
3.00
2,426
DP.SGD
2.75
0.44
Ours
10
0
10
Training Steps
Training Steps
Figure 4: Per-step loss and accuracy (ACC) using different training methods along the same SGD trajectory on Qwen2.5 for Congressional Speech.
5.4 ABLATIONS
Different Pre-trained LLMs. Aside from Qwen2.5, we repeat our experiments on Llama3. 2 and GPT2 testify the universality of our method across different models. We can see from Fig. 5 PE-SGD still outperforms all baselines when using different models, with significantly lower loss and higher accuracy. More results included in Appendix D.2
POPY
PDR:
PDP-5GD
PDP-GGD
Inhr
Infro
DE500 RxSemple
PE SGD RiSemplc
20
3.1
3.3
2.1
0.10
0.41
0.12
0.03
0.64
8
8.6
6.26
6:30 0.10 3700 17:30 175-175
Loss
ACC
Loss
ACC
(a) Llama3.2
(b) GPT2
Figure 5: Loss and accuracy (ACC) comparison with different models m (0) for Congressional Speech.
Different Amount of Private Sam- ple (M). We also perform exper- iments with larger private training dataset to study the scalability and application range of SGD. Results
Model
PDP-SGD
FE-SCD-FxSarpp
are included in Fig. 6 Note that 40
smaller 3 0.05 and larger are applied with larger M 2 000. Results demonstrate that, with an increase in M, the performance of PE-SGD continues to improve, demon- strated by decreasing loss and increasing accuracy. Although the performance of DP-SGD increases dramatically with the increase of M, making it a very strong baseline, PE-SGD still performs better than similar to it. Other methods all exhibit a performance increase with the increase in M.
400
1000
4000
400
1000
4000
Figure 6: Loss and accuracy (ACC) comparison with dif- ferent number of private training samples (M), Qwen2.5 used for Congressional Speech.
Differrnt Differential Privacy Budget In Fig. we compare the performance of the trained model m using PGD and the most important baseline DP-SGD under c 1.0, 2.0, 4.0, 8.0. The results show that, regardless of the DP setting, PE-SGD consistently outperforms DP-SGD, achieving lower loss and higher accuracy. Results for more baseline methods are included in Appendix DD3
Different Noise Addition Place. In Algorithm we add Gaussian noise to the final projection coefficient I GG" H) to guarantee (e, S)-D.. Like mentioned in /S2/ in Section 4.2 noise also be added the private gradient Σø= H.J ("PE-SGDD-NoisyRaaGGinn and the
8

---

Published as a conference paper at ICLR 2026
DP-SG
PE-SGDDFixSample
PF-SGD
32
0,4575
30-
24
3.0
0,4550
28-
23-
3
2.9
0.4525
0.46
0.10
288
0.4500
0.53
0.15
0.975
1.0
8.0
PubMod
Congressional Speech
b.o
Figure 7: Comparison of trained model performance under different DP budge with Qwen2.5 and Con- gressional Speech.
Comparison of trained model performance under different noise addition place with Qwen2.5.
Figure
gradient inner product Σソ= G H ("PE-SGD-Nois)) NoisyInnerproduutt), Results in Fig. shows WPW.PP----NNiisIyeerrryee
performs apparently worse than its counterparts, while adding noise to the final gradient coefficient (PE-SGD) outperforms adding noise to Σ H. achieving a lower loss and a slightly higher accuracy. Since normalization (or clipping) is applied before aggregation, one key determinant of final performance is how close the column norms are each other. If the norms are identical to one anther, no information lost during respective normalization. Empirically, Table i Appendix D.4] we compare the STD-to-Meat and Min-to-Max ratios of column norms of the matrix to which noise is added to in these methods and find that PE-SGD achieves the lowest STD-to- Re-M ratio and highest Min-to-Mat ratio, confirming smaller norm disparities. In exhibits extremely high STD-to-Mean and low Min-to-Max
contrast, "PE-SGD SGD-Noisyl NoisyInnerproduce ratios, indicating much larger disparities.
Loos (Clii)
ACC (CIB)
Loss (harm)
ACC (Nmmm
0.4623
2.00
0.400
0.542
2.75
2.75
Loss
2.9
Loss
0.450
2.20
2.74
2,66
2.8-
22
100
300
Clip Thres, (PubMed)
(Congressional Speech)
(a) Clip v.s. Normalization
(b) API fold
(c)
|D] Per Iteration
Figure 9: Comparison of trained model performance: (a) using normalization or different clipping threshold; (b) under different generative API fold (L in Algorithm T): (c) under different number of
synthetic data per iteration N. Qwen2.5 is used for Congressional Speech (unless otherwise notated).
Normalization and Clipping, Both methods can be used to bound the b sensitivity in DP algo- rithms. However, like shown Algorithm I line g we choose to simply normalize each column of the gradient coefficient Z.; = GG) ( H Ji (i.e. Z.j Z..j / Z instead of clipping (i.e. Z /Z..j/ max Z.1ll2, C)) with an additional papee-par C' as clipping threshold.
See Fig. (a) for comparison of these two methods. Though one may achieve slightly better results
using clipping, it requires careful tuning of the clipping threshold C. Since normalization avoids the need of tuning additional hyper-parameter we chose normalization for simplicity.
Different Generative API fold I., Another syper-parameter in our study is I,, the fold of MARIATIONAL_APION Specifically, each selected seed sample in Deeed generates I, variations through the VARIATIONAL our experiments, we use mainly 2. Notably, "PE-SGD-FixSample corresponds to = 1, where all samples in D are chosen as Dsedd and no new variants are generated. In contrast, setting L = do results in [Deend| = and D being fully regenerated at each iteration using only the RANDOM_APIC of the updated model. As shown in Fig. 9b)) when fixing N = |D] 200 per iteration, both extremes (L. or I, = oc) degrade performance, whereas intermediate choices yield only minor performance differences.
() of the updated model to evolve ' (see Algorithm for details). In
Different Number of Synthetic Data Per Iteration (N). We further study the impact of N on PE-SGD. Results are included in Fig. ] showing that PE-SGD able to gain good performance with only few hundred samples (around 200 of samples). Therefore, PE-SGD is very sample efficient. This also verifies that the gradients of training are limited to a low-dimension space, much smaller than the number of training parameters p (Li al. 2022).

---

Published as a conference paper at ICLR 2026
Algorithm 2 PE-SGD with Data Synthesis using More Powerful Generative Models
Pre-trained generative model m (0) with parameter o; More power generative model or API with un-trainable freezing parameters, empty synthetic datasct D + 0: private datasct B of size M; number of iterations for updates T; number of synthetic samples generate per iteration; number of synthetic sample variation fold L; Poisson sub-sampling ratio / for private samples; learning rate 7p; DP parameters e, 6;
Input:
Output: Fine-tuned or trained PIMM m.
1: D aved 0, D ( ( aceds N. L. u)
2: fine-tuned m (9) using D.
3: Noisecalculation( c. 6, T. B following Gopi al 2021)
for
to 7'
5: g(() < PoissonSubSampplnng B, B .M < |.
6: G + 91, gNv with g. being the sample-level gradient of x, E 'D on m.
7:
+ h , h 3 with h, being the sample-level gradient w, E (3() on m.
8:
(G" (G" H)) o(O,,=TT))
9:
- & (/2)) update
10: D(t) < Sampleselection( N. L. D), D..D D
SynDataGeneration D mead N, L, u.
11: end for
Synthesizing Using More Powerful Generative Models. To further improve performance, PE-SGD can be easily extended to "generating synthetic samples using stronger model or API, then perform- ing DP fine-tuning on a weaker model using these synthetic samples" by freezing the parameters of the stronger generative model while fine-tuning the weaker model (see Algorithm for detail). This extension helps to exploit a more powerful model for synthesizing data of higher quality. In Table 3 we compare two settings: (1) generating synthetic data using the fine-tuned Qwen2.5-3B-Inntruue or GPT-4o-mini, while still fine-tuning the same Qwen2.5-3B --3t-Inntt target model. Results clearly show that using the stronger generation model leads to consistent performance gains over relying on Qwen2.5-323 B------IInuuttt itself alone.
itself, and (2) generating using more powerful model, namely Swwn2.5-7BBIInsee
Table 3: Fine-tuning Qwen2.5-3B- using synthetic samples generated by different generative models. M- = 400, 3 = 0.2, N 200, T = 10 is applied.
PubMed
Congressional Speech
bioRxiv
Generation Model
Loss (1)
(CC
Loss (1)
ACC
Loss ())
ACC (1)
Qwen2.5.3B.
2.1990 20:02
0.5435 1.0. 0008
2.7532 2.7532 1000166
0.4577 10.0011
2.3178 0018
0.5095 0.5095.0.0000 20.000
Qwen2.5-7B Instructt
1.0
2.187920.000 20111
0,5480 0025
2.7152 om13
0.4615 0022
2.3042 0023
0.5139
GPi-4o-mini
2.1862200000
0.5480.00.007
2.7219
2000077
0.4618
2.3013
0.5142.a.0002
Qwen2,5-38 --sttttttt
2,1839 20.0:31
0.5436 2..000s
2.7468
ao.ceas
0.4579
20.0021
2.3165
0.5097 50.0004
Qwen2.5-7B-Insstn
2.1794..o.co
0,5493
2.7104
goucnos
0.4620
2,3015
1:0 0000
0.5148
GGPT-4o-min
2.1737 20.0089
0.5495 2.0 0007
2.7198
co.case
0.4622-.o.001
2.3010 0094
0.5148 2: 0002
Due to space limitation, more experiments are included in Appendix SraundixDDooor considerig synthetic sample SFT enhanced DP-SGD, different number of training iteration T', efficiency of PE-SGD and on.
6 CONCLUSION AND FUTURE WORK
In this work, we propose PE-SGD, gradient projection based evolutionary framework for differentially private training. Extensive experiments demonstrate that PE-SGD surpasses classical DP-SGD in training high-performance models especially under tight DP budgets with limited private data. In particular, PE-SGD shows strong effectiveness in addressing the long-tail problem. Its sample efficiency makes a practical solution for real-world applications.
Future work includes extending verification to other modalities, exploring training from scratch, and refining evolutionary strategies to better control the & distance between Dr-protected approximate gradients and real private gradients (Fig.)
10

---

Published as a conference paper at ICLR 2026
REPRODUCIBILITY STATEMENT
For reproducibility, we provide the following materials:
Source code that is applied as supplementary material during submission and open- at /ttps://gitthbb cow/ con//iiddaLydiaaPE--SS
Detailed experimental settings included in Section 5.1 and Appendix C including private data curation, synthetic data generation prompt and learning sapeerparameters.
REFERENCES
Martin Abadi, Andy Chu, Ian Goodfellow, H Brendan McMahan,, Ilya Mironov, Kunal Talwar, and Li Zhang. Deep Learning with Differential Privacy. In Proceedings of the 2016 ACM SIGSAC conference on computer and communications security, pp. 308-318, 2016.
Nermeen Abou Baker, David Rohrschneider, and Uwe Handmann. Parameter-efficien Fine-tuning of Large Pretrained Models for Instance Segmentation Tasks. Machine Learning and Knowledge Extraction, 6(4):2783-2807 2024.
Rishi Bommasani, Steven Wu, and Xanda Schofield. Towards Private Synthetic Text Generation, In NeurlPS 2019 Machine Learning with Guarantees Workshop, 2019.
Nicholas
Carlini, Florian Tramer, Eric Wallace, Matthew Jagielski, Ariel Herbert-Voss Katherine Lee, Adam Roberts, Tom Brown, Dawn Song. Ulfar Erlingsson, et al. Extracting Training Data from Large Language Models. In 30th USENIX security symposium (USENIX Security 21), pp- 2633-2650, 2021.
Han Shi Jocelyn Chew. The Use of Artificial Intelligence- Conversational Agents (Chatbots) for Weight Loss: Scoping Review and Practical Recommendations, JMIR medical informatics, 10 (4):e32578, 2022.
Gheorghe Comanici, Eric Bieber, Mike Schaekermann, Ice Pasupat, Noveen Sachdeva, Inderjit Dhillon, Marcel Blistein, Ori Ram, Dan Zhang, Evan Rosen, et Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities. arXiv preprint arXiv:2507.066: 06261, 2025.
Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang. Angela Fan, al. The Llama Herd of Models. arXiv e-prints, pp. arXiv-2407, 2024.
Cynthia Dwork. Differential Privacy. In International colloquium on automata, languages, and programming, pp. 1-12. Springer, 2006.
Cynthia Dwork, Aaron Roth, et al. The Algorithmic Foundations of Differential Privacy. Foundations and Trends@ in Theoretical Computer Science, 9(3-4):211-00) 2014.
Vitaly Feldman, Tomer Koren, and Kunal Talwar. Private Stochastic Convex Optimization: Optimal Rates in Linear Time. In Proceedings of the 52nd Annual ACM SIGACT Symposium on Theory of Computing, pp. 439 449, 2020.
Jiahui Gao, Renjie Pi, Lin Yong, Hang Xu, Jiacheng Ye, Zhiyong Wu, Weizhong Zhang, Xiaodan Liang, Zhenguo Li, and Lingpeng Kong. Self-guided Noise-free Data Generation for Efficient Zero-shot Learning. In Proceedings of The Eleventh International Conference on Learning Representations, 2023.
Sivakanth Gopi, Yin Tat Lee, and Lukas Wutschitz. Numerical Composition of Differential Privacy. Advances in Neural Information Processing Systems, 34:11631-11642, 2021.
Sagar Goyal, Eti Rastogi, Sree Prasanna Rajagopal, Dong Yuan, Fen Zhao, Jai Chintagunta, Gautam Naik, and Jeff Ward. Healai: A healthcare Ilm for effective medical documentation. In Proceedings of the 17th ACM International Conference Web Search and Data Mining, pp. 1167-1 168, 2024.
11

---

Published as a conference paper at ICLR 2026
Xin Gu, Gautam Kamath, and Zhiwei Steven Wu. Choosing Public Datasets for Private Machine Learning via Gradient Subspace Distance. In 2025 IEEE Conference on Secure and Trustworthy Machine Learning (SaTML). pp. 379-900. IEEE, 2025.
Guy Gur-Ari, Daniel A Roberts, and Ethan Dyer. Gradient Descent Happens in Tiny Subspace. arXiv preprint 1812:0881 2018.
Charlie Hou, Akshat Shrivastava, Hongyuan Zhan, Rylan Conway, Trang Le, Adithya Sagar, Giulia Fanti, and Daniel Lazar. PrE-Text: Training Language Models on Private Federated Data in the Age of LLMs. In Forty-first International Conference on Machine Learning, 2024.
Charlie Hou, Mci-Yu Yu Wang, Yige Zhu, Daniel Lazar, and Giulia Fanti, Private Federated Learning using Preference- Preference-pptmmized Synthetic Data. Forty-second International Conference on Machine Learning, 2025.
Rui Hu, Yuanxiong Guo, and Yanmin Gong. Federated Learning with Sparsified Model Perturbation: Improving Accuracy under Client-LLevl Differential Privacy. IEFE Transactions on Mobile Computing, 23(8):8242-8255, 2023.
Giorgos lacovides, Thanos Konstantinidis, Mingxue Xu, and Danilo Mandic. Finllama: Llm-based financial sentiment analysis for algorithmic trading. In Proceedings of the 5th ACM International Conference on Al in Finance, pp. 134-141, 2024.
Katherine Lee, Daphne Ippolito, Andrew Nystrom, Chiyuan Zhang, Douglas Eck, Chris Callison- Burch, and Nicholas Carlini. Deduplicating Training Data Makes Language Models Better. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume
1: Long Papers), pp. 8424-8445, 2022.
Xinyan Li, Qilong Gu, Yingxue Zhou, Tiancong Chen, and Arindam Banerjee. Hessian Based Analysis of SGD for Deep Nets: Dynamies and Generalization. In Proceedings of the 2020 SIAM International Conference on Data Mining. pp. 190.198. SIAM, 2020.
Xuechen Li, Daogao Liu, Tatsunori Hashimoto, Huseyin A Inan, Janardhan Kulkarni, YinTat Lee, and Abhradeep Guha Thakurta. When Does Differentially Private Learning Not Suffer in High Dimensions? In Alice H. Oh, Alekh Agarwal, Danielle Belgrave, and Kyunghyun Cho (eds.), Advances in Neural Information Processing Systems, 2022. URL https:///ppeereeiew net/ forum?id=FR-mmkuukd
Zinan Lin, Sivakanth Gopi, Janardhan Kulkarni, Harsha Nori, and Sergey Yekhanin. Differentially Private Synthetic Data via Foundation Model APIs 1: Images. In The Twelfth International Conference Learning Representations, 2024.
Zinan Lin, Tadas Baltrusaitis, and Sergey Yekhanin. Differentially Private Synthetic Data via APIs 3:
Using Simulators Instead of Foundation Model. arXiv preprint arXiv:2502.05505, 2025.
Justus Mattern, Zhijing Jin, Benjamin Weggenmann, Bernhard Schoelkopf, and Mrinmaya Sachan. Differentially Private Language Models for Secure Data Sharing. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pp. 4860-4873, 2022.
Ryan McKenna, Yangsibo Huang, Amer Sinha, Borja Balle, Zachary Charles. Christopher A Choquette- Chho, Badih Ghazi, George Kaissis, Ravi Kumar, Ruibo Liu, et al. Scaling Laws for Differentially Private Language Models. arXiv preprint arXiv:2501. 18914, 2025.
NLM (U.S.). Pubmed dataset. FTP dataset, National Center for Biotechnology Information, by Na- tional Library of Medicine (U.S.), 2025. UtL/https://pubmmed ncbi. nlm. nih, gov/download/
OpenAl. Gpt-5 system card. OpenAl Technical Document, August 2025. Available at https
cww. - 5-------tttte---ra--- pdff (accessed 2025-09-18).
Pranav Putta, Ander Steele, and Joseph W Ferrara. Differentially Private Conditional Text Generation For Synthetic Data Production, 2023. URL https:://operreview net//forumidd41gg)]200Ff
Qwen-Team. Qwen2.5: A Party of Foundation Models, September 2024. URL https: / //qwenlwwmmmmmmmmmmmmmmm github.io/blog///oee.. httpbb/io//blog/g//....
12

---

Published as a conference paper at ICLR 2026
Alec Radford, Jeff Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. Language Models are Unsupervised Multitask Learners. 2019.
Badrinath Ramakrishnan and Akshaya Balaji. Assessing and Mitigating Data Memorization Risks in Fine-Tuned Large Language Models, arXiv preprint arXiv:2508. 14062, 2025.
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High- Resolution Image Synthesis with Latent Diffusion Models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 10684-10695, 2022.
Florian Tramèr, Gautam Kamath, and Nicholas Carlini. Position:
Considerations for Differentially Private Learning with Large-Scale Public Pretraining. In Proceedings of the 41st International Conference on Machine Learning, pp.
2024.
Jiachen T Wang, Prateek Mittal, Dawn Song, and Ruoxi Jia. Data Shapley in One Training Run. In The Thirteenth International Conference on Learning Representations, 2025a.
Shuaiqi Wang, Vikas Raunak, Arturs Backurs, Victor Reis, Pei Zhou, Sihao Chen, Longqi Yang, Zinan Lin, Sergey Yekhanin, and Giulia Fanti. Struct-bench: A benchmark for differentially private structured text generation. arXiv preprint arXiv:2509. 2025b.
Michael Wornow, Yizhe Xu, Rahul Thapa, Birju Patel, Ethan Steinberg, Scott Fleming, Michael A Pfeffer, Jason Fries, and Nigam H Shah. The Shaky Foundations of Large Language Models and Foundation Models for Electronic Health Records. npj digital medicine, 6(1) 6(1):135, 2023.
Chulin Xie, Zinan Lin, Arturs Backurs, Sivakanth Gopi, Da Yu, Huseyin A Inan, Harsha Nori, Haotian Jiang, Huishuai Zhang, Yin Tat Lee, et al. Differentially Private Synthetic Data via Foundation Model APIs 2: Text. In Forty-firsi International Conference on Machine Learning, 2024.
An Yang. Baosong Yang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Zhou, Chengpeng Li, Chengyuan Li, Dayiheng Liu, Fei Huang, Guanting Dong, Haoran Wei, Huan Lin, Jialong Tang. Jialin Wang, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Ma. Jin Xu, Jingren Zhou, Jinze Bai, Jinzheng He, Junyang Lin, Kai Dang, Keming Lu, Keqin Chen, Kexin Yang, Mei Li, Mingfeng Xue, Na Ni, Pei Zhang, Peng Wang, Ru Peng, Rui Men, Ruize Gao, Runji Lin, Shijie Wang. Shuai Bai, Sinan Tan, Tianhang Zhu Tianhao Li. Tianyu Liu, Wenbin Ge. Xiaodong Deng. Xiaohuan Zhou, Xingzhang Ren, Xinyu Zhang, Xipin Wei, Xuancheng Ren, Yang Fan, Yang Yao, Yichang Zhang, Yu Wan, Yunfei Chu, Yugiong Liu, Zeyu Cui, Zhenru Zhang, and Zhihao Fan. Qwen2 Technical Report. arXiv preprint arXiv:2407. 10671, 2024.
An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, et al. Qwen3 technical report. arXiv preprint 20380005 2025.
Jiacheng Ye, Jiahui Gao, Zhiyong Wu, Jiangtao Feng, Tao Yu, and Lingpeng Kong. ProGen: Progressive Zero-shot Dataset Generation via In-context Feedback. In Findings of the Association for Computational Linguistics: EMNLP 2022, pp. 367 1671-3683, 2022.
Da Yu, Huishuai Zhang, Wei Chen, and Tin-Yan Liu. Do not Let Privacy Overbill Utility: Gradient Embedding Perturbation for Private Learning. In 9th International Conference on Learning Representations, 2021.
Sikai Zhang, Tingna Wang, Keith Worden, Limin Sun, and Elizabeth J Cross. based Fast Feature Selection for Structural Health Monitoring. Mechanical Systems and Signal Processing, 223: 11895, 2025.
Yingxue Zhou, Zhiwei Steven Wu, and Arindam Banerjee. BYPASSING THE AMBIENT DIMEN- SION: PRIVATE SGD WITH GRADIENT SUBSPACE IDENTIFICATION, In 9th International Conference on Learning Representations, ICLR 2021, 2021.
Tianyuan Zou, Yang Liu, Peng Li, Jianqing Zhang, Jingjing Liu, and Ya-Qin Zhang. FuseGen: PLM Fusion for Data Pata-generation based Zero-shot Learning. In Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen (eds.), Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pp. 2172-2190. November 2024.
13

---

Published as a conference paper at ICLR 2026
Tianyuan Zou, Yang Liu, Peng Li, Yufei Xiong, Jianqing Zhang, Jingjing Liu, Xiaozhou Ye, Ye Ouyang, and Ya-Qin Zhang. Contrastive Private Data Synthesis via Weighted Multi-PLM Fusion. In secood International Conference Machine Learning, 2025.
14

---

Published as a conference paper at ICLR 2026
APPENDIX
A LARGE LANGUAGE MODEL USAGE STATEMENT
Large Language Models, particularly the GPT series and Gemini, have been applied to API usage demonstrations, code generation for plotting, auto-completion (with GitHub Copilot), academic paper writing refinement, and grammar correction.
B ADDITIONAL INFORMATION FOR ALGORITHM AND METHODS
B. DETAIL ALGORITHIM FUNCTIONS
In this section, we elaborate the functions used in Algorithm 1 in detail in Algorithm S
Algorithm 3 Functions used in Algorithm for PE-SGD
function SynDataGeneration Dreed m ):
if L then
Doced N samples generated from using RANDOM_API().
Dear
else
if DDreed = 0 then
Deced + [N/L] L] samples generated from mn using RANDOM_API() ().
end if
Dear +- 0.
X; Dreed
Duar Der U
samples generated from m using VARIATIONALLAPI(N.)))
end for end if
return Deoed U
function PoissonSubSamplingg B. $
B 0.
for w, E B do
X ~ Bemoulli(3).
if X= then
B - B (w,).
end if
end for return &
function SumtppormmlllzedCollumss )):
Denote (2s)) where B. N and Z., [2i.s...., 2N.
for j = 1 to .M do
for = 1 to N do
2.. + zz.1/ /l../ where |Z., + V VVN 2?, being the la norm of Z.j.
end for end for return Ad 1 Z.
function SampleSelection( 2, N. L,
Denote 12..... 2x with N.
|z|. i.. z - [|a.]....,
exp(x)
P: Eim exp(=;]
Random sample [N/L] different index S (.1..... . according the probahility given by p.-
return (x, x. E D E 5).
15

---

Published as a conference paper at ICLR 2026
B.2 PRIVACY ANALYSIS FOR PE-SGD
In this section, we prove that Algorithm guarantees (e, o)-DP
Theorem B.1. PE-SGD (Algorithm)) Ty has the exact same (e, 6)-D guarantees as usual DP-SGD for the same noise scale o, sub-sampling probubility and number of iterations T (i.e., the composition of sub- sub-sampled Gaussian mechanism).
Proof. In Algorithm[] 1 within each iteration, the only point where private information, namely the private sample gradients H, is exposed is in the computation Z = (G' GG H) (line 8). In this place, the 7th column of Z only depends on the ith column of H and thus only depends on the dd private sample. By the post-procsssing property of differential privacy (Dwork et al. 2014).
any further computation on the output of an (e, 0)D mechanism does not incur additional privacy loss. Hence, as long as DP guaranteed for the aggregated quantity 2 = Σj=1 Z: in line 8 in Algorithm T DP guaranteed for the whole iteration. To protect this sensitive result z, we therefore normalize each column of Z to have & norm 1 and add isotropic Gaussian noise or x <0( 7) to it.
Thus, the problem reduces to the composition of sub-sampled Gaussian mechanism in exactly the same way and with the same parameters as DP-SGD. This question has been carefully studied in several prior works such as Abadi et al. (0116). In particular, we use the PRV Accountan f from Gopi et al. (202 which computes the exact privacy budget to arbitrary accuracy. PRV Accountant has also been adopted as the default privacy accountant in Opacus a widely used PyTorch library for differentially private training, which is regarded as the standard DP framework in both academia and industry.
5
B.3 THE RELATIONSHIP BETWERN DIFFERENT PROJECTION METHODS
In this section, we discuss the relationship between the projection method given in PDP-SGD (Zou et al. )02T), GEP (Yu et al., 2021) and our projection method given in Eq. (
Theorem B.2. full eigenspace F E Roxr instead of the top-k (k <r) eigenspace E E Roxk of
GGT is exploited in PDP-SGD (Zhou et al. 2021), their projection function G FF H is equal our projection method G G(G G)
G H) given in Eq.
Proof. First of all, from linear algebra, rank(G) = rank G = Therefore, the singular vector decomposition (SVD) of G can be written as G UEVT where U E Roxr and V E RNr have orthonormal columns (UTU = VTV = I, E R and Σ = diag(01....,0;; with o; 0,i I.....
Then GGT = ( = U. Therefore, the eigenvectors of GGT are the columns of U. If F spans the eigenspace of GGT corresponding to the non-zero eigenvalues, then we have F = U with FF" = UUT.
Note that G VEU VE'V", thus VV-
Therefore,
G = (VE
=UEI,E"L,EU" -UIUU -UTT =FF..
Multiplying by H at the end, we get G
H)
Theorem B.3. all left singular vectors F E IRpXT G instead of E Bpxk the top-k (k < r)
left singular vectors of G corresponds to the top-k singular values (k < r), is exploited in GEP (Yr
et al. 2021), their projection function G = FFT H (without considering the residual term R) is equal to our projection method G = G(T GG" H) given in Eq.
'An open-source implementation that is available at htthh:tt/ thub. httpat/ccmm..iiisssoooooooc.oooa
Oppauss Homepage: https: //tpacss.aaaaaaa ai
16

---

Published as a conference paper at ICLR 2026
Proof. Similar as above, the singular vector decomposition (SVD) of G can be written G UEVT where E RRrr and V E I Nxr have orthonormal columns (UTU = V = I, E Rxx") and Σ diag(o)...... , σx) with o; 0, 1, ....,... If F spans the whole left singular vectors space of G, then we have = U with FF UUT.
Thus, when neglecting the residual term of GEP, the projection of GEP is the same that of PDP- SGD. From Theorem B.2 the projection method of GEP is also equal to G (G H)
given in Eq.
C IMPLEMENTATION DETAIL
C.I DATASET CURATION
PubMed (NMM (U.S.) 2025 As samples later than January 8th 2025 are not published, we use samples that has complete date between December 1sf 2024 and January 8th 2025, We also remove samples that are less than 100 characters in length. We randomly partition the remaining samples with train:dev:test=6:1.33 resulting in a total of 8613 training samples, 1435 dev samples and 4307 test samples.
Congressional Speech (Hou et al. 2023). We use samples from January 2025 to May 2025 as the training part and samples from June 2025 as the test part. We further performs a random partition the training part with train:dev=4:1. We then removes "None" samples and samples less than 5 characters, resulting in a total of 33413 training samples, 8229 dev samples and 19880 test samples.
bioRxiv (Hou al. 2025). We also samples from January 2025 to May 2025 as the training part and samples from June 2025 as the test part. We then randomly partitioned the training part with 1. This results in total of 13775 training samples, 2296 dev samples and 2507 test samples.
In our experiments, B is randomly selected from the training partition while A uses the whole test partition or randomly selected 4, 000 samples from the test partition for Congressional Speech as the dataset is too large. To align with other methods, pass the original text input directly to tokenizer, instead of using "eencode_pluse"" plus()" after "tokenize()" as originally done by POPri for GPT2 model.
C.2 API PROMPT
In Table we listed the prompts used in our experiments, including zero-shot in-context learning prompts for RANDOM_API and one-shot prompt for VARIATIONAL_ATT (OTIIIONAAAPP()) (). We mainly follow Xie et al. (2024) for the construction of these prompts. The zero-shot prompt is also used for evaluation. Note that we use chat template throughout the training and evaluation process.
C.3 LEARNING
In our experiments, we use LoRA fine-tuning with LoRA rank = 8, LoRA = 32 and LoRA dropout rate 0.1. For Qwen2.5 and Llama3.2, we perform LoRA on "q_proj" and "v "v_proj" modules, whereas for GPT2 we perform LoRA on "c_ "e_attn" and "c_proj" modules.
The learning rate 1) varies across tasks. With M 400, 7) 5e when training Qwen2.5 for PubMed and Congressional Speech; 7/ 34-4 when training Qwen2.5 for bioRxiv; 1 = 2e-1 when training Llama3.2 for Congressional Speech; 7) = 3e -4 when training Ilama3.2 for PubMed: 7 le when training GPT2 for all tasks. With M 1, 000 and M 4, 000, 77 4e-4 and 1 = 3e are used respectively for training Qwen2.5 for Congressional Speech. Warmup ratio is set to 0.08 and weight decay ratio set 0.01. AdamW is used as the optimizer. Each experiment is repeated using 3 random seeds.
If clipping is applied, unless otherwise stated, the clipping threshold is set to C = 1.0, except the clipping threshold for the residual term in GEP (Yu et al. [02T)), for which the clipping threshold is set to ()2 following the clipping threshold given in their paper Yu al. 2021).
17

---

Published as a conference paper at ICLR 2026
Table 4: Prompt used for synthetic dataset generation. Due clarity, we omit the words in the parentheses in the labels of Openreview-Cateeen Categor and the attributes of Openreview-Rating.
Dataset
Prompt Type
Prompt
<tone>
zero-shot
"Using variety of sentence structures, write abstract (only the abstract, without other parts)
PubMed
medical researcht paper:
one-shot
" "<soinple>/nBBr
the above sample, using variety of sentence structures <tone>. write un abstract (only the abstract, without other parts) for medical
"in professional way", "in a professional tone". "in professional style". "in con- cise manner", "in creative style", "using imagination" "in storytelling tone", "in formal manner"
paper:
Congressional
zero-shot
"Using variety of oral sentence structures. write line from a public debating (heginning the appropriate formal address for the chamber) speech delivered the United States Congress, the UK Parliament, or the
Speech
Canadian Parliament:
one-shot
on the above sample, using variety of oral sentence structures <tone>, write line from a public debating (beginning the uppropriate formal address for the chamber) speech delivered in the United States Congress, the UK Parliument, the Canadian Parliament:"
"ssompl>>in
"in occasionally adversarial but civil man- ner", "with "in formal and official tone". "showing deep respect and decorum", "using re- spectful and mensured style", "in per- suasive and compelling way", "adopting an argumentative tone", "with creative and imaginative language". "eemplyying rhetor- ical flourishes", "in thoughtful and rea- soned manner", "with passionate convic- tion". "using clear and direct speech", "in diplomatic and tactful style". "expressing urgency and determination" "with nuanced and balanced arguments"
sense of patriotism"
"Uking variety sentence structures, write realistic bioRxiv preprint abstract (only the abstruct, without other parts) for a life science (e.g. biology, neuroscience, cognition, etc.) research puper:
bioRxiv
one-shot
" "<sample>/nBased on the above sample, using a variety of sentence structures <tone>, write a realistic bioRxiv preprint abstruct (only the abstract, without other parts) for life science (c.g. biology. neuroscience, cognition. etc.) research paper:"
"in professional way". "in professional tonc". "in professional style". "in con- cise manner", "in creative style", "using imagination" " "in storytelling tone", "in a formal manner". "in high-quality man- ner", a low-quality manner"
D ADDITIONAL RESULTS
D.1 SIMILARITY BETWEEN APPROXIMATE OP-PROTECTED GRADIENTS AND REAL PRIVATE GRADIENTS
Fig. 10, we additionally show the cosine sim-
ilarity between approximate DP-protected gra-
Symilrty (17
1.0
Distance (1)
dients with real private gradients under different ablation methods by removing key components used in PE-SGD: empirically better noise placement, and dynamic projection subspace that evolves with training. This figure testifies the importance of combining all these two components to get good results.
SGD
0.6
Dffferent Wolse Placmeen
of the two
02
0.0
0.0
25
5.0
7.5
30.0
0.0
25
5.0
75
20.0
Training Stops
Training Stops
Figure 10: Similarity between approximate DP- protected (with noise) gradient and private gradient (without noise). See Section T for detail.
Moreover, to better understand why fixed projection subspace leads to suboptimal perfor- mance, specifically, the growing & diserepancy between the projected and true private gradients training progresses (see Fig. 10 we analyze the principal angles (Gu et al. (225)) between the private gradient subspace the initial iteration, ("),, and those at later iterations. (t) for ....., T To compute these angles, first
extract the top-50 orthonormal basis vectors of each subspace, yielding O E(t) E RPx50 We then
18

---

Published as a conference paper at ICLR 2026
form O angles follow as 8 arccos(s)). lying in [0 [00). Larger angles indicate greater divergence between subspaces, with 90" corresponding to orthogonality. As shown in Fig. I1 these principal angles increase steadily over the progression of training for all three tasks (private datasets) and different fine-tune models. This consistent increase demonstrates that the private gradient subspace evolves significantly during fine- tun-ttnnng, independent of the specific dataset fine-tuning model. These findings underscores our claim that relying on fixed projection subspace is suboptimal for fine-tuning.
E(0) (()) and compute its singular values 8, for 1,.... ,50 via SVD. The principal
Principal angle (degro0))
Yter
Frincipl snole (degras)
Ner
80
80
60
Iter
Iter 4
40-
Iter
40
Iter
20
10
Component index
Componenn index
(a) Qwen2.5 Congressional
(b) Ilama3.2 Congressional
PVlccipal (degraa)
80
Bar
80
Iae
Iter
Iter
60
Iter 4
50-
Ier
Iter
Princips
40
Ite
Iter
Iter
Iber
Iter
20
Itor
10
20
30
40
50
20
30
40
50
index
Component index
(c) Qwen2.5 PubMed
(d) Qwen2.5 bioRxiv
Figure 11: Principle angles of initial private gradient subspace and private gradient subspaces in later
fine-tuning iteration. M 400, 3 = 0.2, L = 10, N = 200 applied.
D.2 DIFFERENT PRE-TRAINED LLMS
In this section, we include more results for Llama3.2 and GPT2 Fig. the same as that for main experiments, i.e. M 400, 0.2, 10, 1e N 200, L 2.
12
Other settings remain
PE-SGD demonstrates consistent superiority compared to other methods.
D.3 DIFFERENT DP B
In this section, we examine how the performance the fine-tuned model m varies with the differential privacy budget across all baseline methods. Results are included in Fig. 13 While the curves, particularly those for methods other than DP-SGD, appear nearly flat, their values do change slightly with However, these variations are minor compared to the substantial performance gaps between methods. This suggests that the relative performance differences are primarily method- Moreover, the robustness of most gradient projection methods to DP noise (high performance under tight budge scenario) underscores the effectiveness reducing noise dimensionality through projection.
D.4 COLUMN NORM DISPARITY OF DIFFERENT MATRIX
We include the numerical results of STD-to-Mean ratio and Min-to-Max ratio of column norms of H, H. G) H) respectively for the three different noise addition strategies, namely "PE- SGD-Noixylmerprodduc and PE-SGD in Table Results shows smaller norm disparities of PE-SGD compared to others, with PDE-SGDDNNissymmeprooeee exhibiting much larger disparities.
GGD-NoisyRealGrace
19

---

Published as a conference paper at ICLR 2026
SGp
56p
Aug-FE
Aug-rE
7DP-5GD
FDR 5GD
SEP
GEP
FE-SOD-ficaansle
PEDSDD
FE-50D
0.40
0.44
425
6
6.7 eir
S.N 037e
Loss
ACC
Loss
ACC
(a) 1.ama3.2 Congressional 1.0
(h) GPT2 Congressional 0p
Aug-FE
Auge PC
POhn
GEP
InPro
Infro
Re-SGD-Fissamphh
PE-SCO-fisSupp
PS-S0D
FE FE:SGD
0,500
8:505
asio 6:15 0.000
0.50
0:51
Loss
(c) Llama3.2 2 PubMed 1.0
(d) Llama3.2 PubMed c
0P-5GD
5G0
Aug-FE
AugeIE
POP-5GD
PDR.GDD GEP
SEP
Infro
PE-SGD-ficSengle
PE-SGD-fiSSepp
PS-SCD
FE-500
Loss
ACC.
Loss
ACC
(c) GPT2 PubMed e 1.0
(f) GPT2 PubMed e oc
RE
Aug PE
POPT
POP1
GFP
GPP
Infro
Inho
SGD
4.00
9.00
0.296
0.300
0.32
9.05
9.10
9.15
0.305
0.318
gg) GPT2 bioRxiv 1.0
(h) GPT2 BioRxiv
Figure 12: Loss and accuracy (ACC) comparison with different models (n
DP-CD
POPi PDP-500
GEP
PC SGDDSFFSaaapp piu
DRSCD FE
POhi
GEP
PC SSSSSDDDiiSaaa
Ft
kho
SGD
POP-50D 50D
snhro
SGD
2.5
3.75
3.50
0.45
5.25
0.44
8.0
(a) PubMed
(b) Congressional Speech
Figure 13: Comparison of loss and accuracy (ACC) of fine-tuned model m under different DP budge c using Qwen2.5.
Table 5: Comparison of the STD-to-Mean ratio and Min-to-Max ratio of column norms in the noise-attached-to matrix across three different noise addition strategies. Results are averaged across iterations first and then across three random seeds. Best results are marked.
PE.SGD-NaisyImmeepooott
PE-SGD
Norms
[|| 1
[ H1
[lI(
G) H
10
/ Mean (L)
0.443 00.020
10.015 0..03
0.404 10.099
Min Max (t)
10.007
0.039 0.039 10.002
0.121 10.006
20

---

Published as a conference paper at ICLR 2026
D.5 IMPACT OF GENERATION PROMPT
To assess the impact of different prompt templates on data synthesis, we follow prior work (Xie al. 2024) and report final model performance as well as FID, Precision, and Recall of the synthetic dataset in Table 6 on Congressional Speech task using Qwen2.5. Note that the FID. Precision and Recall metrics are computed using 200 randomly generated samples from the fine-tuned model after the whole training process. The first row corresponds to the prompt used in our main results ("Original"), which was designed following prior work (Xie et al. 2024). "P-1" removes all tone options from the one-shot prompt while keeping the zero-shot prompt unchanged. "P-2" keeps only three tones in the one-shot prompt, again with the zero-shot prompt unchanged relative to "Original". "P-3" and "P-4" use prompts rewritten by ChatGPT, with "P-3" being more structured and "P-4" being more creativity-orienter (as judged by ChatGPT). Full prompt details are provided in Table 7
Table 6:
Comparison of final model performance, FID, Precision, Recall of the generated synthetic dataset using different prompts throughout the training process PE-SGD using Qwen2.5.
Prompt
Loss (4)
ACC (t)
FID (4)
Precision (t)
Recall (1)
Original
2.7532+0.0106
0,4577 +0.0011
0.1571 0036
0.9200 -0.0537
0.5235 1.000
P-1
2.7526:0.0136
0.4570:0.001
0.1625
0.8533
0.5416
P-2
2.7522 1.7522.0.0338 00.033
0.4572
0.1524 0048
0.8633
(6633_0.066)
0.5233 1015
P-3 2.7915 0.4499
0.4576 0.0025
0.1767 = 01.31
0.7200
-7.200
0.6483 u0.1708
P-4
+..097
40.004
0.2054=a.um
0.6933 =u.1320
0.5708 20.3621
From T we observe that using different reasonable prompts has only minor impact on final model performance (Loss and ACC). However, different prompts affect the Precision and Recall of the synthetic dataset with respect to the private dataset.
On one hand, although less diversity inspiring prompts ("P-1" and "P-2" v.s. "Original") preserve final model performance and FID, they significantly reduce Precision, indicating an increasing number of diverging samples that are less correlated to the target private distribution. On the other hand, more structured prompt ("P-3" v.s. "Original") improves Recall (covering more of the private sample space) but harms Precision (introducing more synthetic samples that deviate from the private distribution); while more creative-inspiring prompt ("P-4"" performs worse overall than the others.
D.6 VISUALIZATION SYNTHETIC AND REAL GRADIENT SUBSPACE
We include a PCA visualization of the synthetic and real gradient subspaces of PE-SGD in Figs. [4(a] [4C) and [4(] These figures show that their alignment improves substantially over the course of training compared with the initial iteration, and is better than that of the exed-projection base- line ("PE-SGD FRE-SGD-Fix Samplee in Figs. 14(b) [4(d) and [4(f)). This provides clear evidence for the effectiveness of the evolving projection mechanism PE-SGD.
D.7 THE ENHANCEMENT OF SYNTHETIC DATA SET
As shown in Algorithm D (line our pre-trained model fine-tuning procedure builds on synthetic data supervised fine-tuning (SFT) of (9) As illustrated in Table g this step provides additional performance gains for m, although not to a large extent.
D.8 SFT ENHANCED DP-SGD
Like shown in Algorithm I our pre-trained model fine-tuning is based on a synthetic data supervised fine-tuning (SFT) of m (0) with helps to improve the performance of the final model m like shown in Table 8 This would lead to straightforward improved baseline for DP-SGD, i.e. first SFT on synthetic data and then performs DP-SGD on the tuned model. We compare PE-SGD with this improved baseline ("SFT DP- in Table 9 We also display the results of synthetic data SFT (the first step of "SFT + DP-SGD") to give detailed information of this stage procedure. Results demonstrate that, with this SFT enhancement, PE-SGD still outperforms the SFT improved
21

---

Published as a conference paper at ICLR 2026
Table 7: Different prompts used for synthetic dataset generation in Table
Dataset
Prompt Type
Prompt
<tone>
P-1 for Congressional Speech
zero-shot
"Using a variety oral sentence structures, write line from public dehating (heginning with the uppropriate formal address for the chamber) or speech delivered the United States Congress, the UK Parliament. the Canadian Parliament:
one-shot
"<somple>/aaBees on the ahove sample, using a variety of oral sentence structures, write a line from public debating (beginning with the appropriate formal address for the chamber) speech delivered in the United States Congress. the UK Purliament, the Canadian Purliament:
zero-shot
"Using a vuriety of oral sentence structures, write line from a public debuting (beginning uppropriate formal address for the chamber) speech delivered in the United States Congress, the UK Parliument, or the Canadian Parliament:
P-2 Congressional Speech
one-shot
on the above sample, using variety of oral sentence structures <tone>, write line from a public debuting (beginning the appropriate formal address for the chamber) speech delivered in the United
"<somple>>
"in formal and official tone". "in persua- sive and compelling way". "using clear and direct speech"
States Congress, the UK Parliament, or the
Canadian Parliament:
P-3 for Congressional Speech
zero-shot
"Compose one line dialogue suitable for a formal public debate parbiamentary speech. The line should begin with the proper formal address for chamber
reflect the speak- ing style either the United States Congress. the UK Purliament, or the Canadian Parliament.
one-shot
Grample>/nUsingg the sample above as refer- ence, and adopting <tone> expression, pose one line of dialogue suitable for a formal public dehate or parliamentary speech, The line should begin with the proper formal address for the chamber and reflect the speaking style of either the United States Congress, the UK Par- liament, the Canadian Parliament."
"in occasionally adversarial but civil man- ner", "with "in formal and official tone", "showing deep respect and decorum", "using a re- spectful and measured style". "in per- suasive and compelling way". "adopting an argumentative tone", "with creative and imaginative language", "employing rhetor- ical flourishes", "in thoughtful and rea- soned manner", "with pussionate convic- tion". "using
of putriotism"
and direct speech", "in diplomatic and tactful style". "expressing urgency and determination" "with nunnced halanced arguments"
P-4 for Congressional Speech
zem--shol
single line as spoken public debate purliamentary speech. Begin with appropriate formal salutation for the floor, and write in the style a speech from the U.S. Congress, the British Parliament, the Canadian Parliament.
one-shot
inspiration from the given sample and using <tone> phrasing, craft single line as spoken public debate parliamentary speech. Begin with an appropri- ate formal salutation for floor, and write it the style speech from the U.S. Congress, the British Purliament, or the Canadian Parlia- ment."
"in a occasionally adversarial but civil man- ner",, "with strong sense of patriotism" "in foomaal and official tone", "showing deep respect and decorum". "using a re- spectful and measured style". "in a per- suasive and compelling way", "adopting argumentative tone". "with creative and imaginative language", "employing thetor- ical flourishes". "in thoughtful and soned manner". "with passionate convic- tion". "using clear and direct speech", "in a diplomatic tactful style". "expressing urgency and determination" "with nuanced and bulanced arguments"
DP-SGD ("SFT DP-SGD"), achieving lower loss and higher accuracy when DP is guaranteed = 1.0). Also, it's not surprising that under e xo (without DP protection), "SFT DP-SGD"
22

---

Published as a conference paper at ICLR 2026
(a) Congressional PE-SGD
(h) Congressional PE-SGD-Fixs SGDD-FiXSaapleee
(c) PubMed PE-SGD
(d) PubMed PE- TE-SGD-FixSampp
(e) bioRxiv PE-SGD
(f) bioRxiv PE-SGD-FixSamjn
Figure 14: Visualization synthetic and private gradient subspaces within each fine-tuning iteration
using PCA. M 400, B 0.2, L 2, T 10, N 200 is applied using Qwen2.5. Blue points represent synthetic gradients while orange samples represent private gradients.
Table 8: Comparison of PE-SGD with and without (w/) synthetic data SFT procedure using Qwen2.5.
Better results are marked.
Method
1.0
00
Loss (1)
Accuracy (t)
Loss (L)
Accuracy (1)
PE-SCD w/o SFT PE-SGD
2.774040.0089
0.4559-0.000
2.7471 20.0007
0.4578g0.0008
2.7532
0.457720.001
2.7468 2.746820.0005
0,4579 0.45792..000
20.0021
performs better. This is because, now trains directly on real private sample gradients that is only clipped but not perturbed, whereas PE-SGD still trains on combination of synthetic sample gradients
Table 9: Comparison of SGD and synthetic data SFT enhanced DP-SGD ("SFT + DP-SGD") using Qwen2.5, Better results between "SFT DP-SGD" and PE-SGD are marked.
PubMed
Congressional Speech ==
bioRxiv
Method
30
= 00
=
= 00
Next Token Prediction Loss (1)
Base Model
2,9341
3,8690
2.5311
SGD
2. 2.1199
2. 2.6078 20.008
2.2756+0.0001
SFT
2.5174=0.6477
3,3767 2430
2,5261 +0.0062
SFT + DP-SGD
2.2579-20.o1or
2.1621
2.8821 20-0397
2.7193
2.3773
2.289110.001
PE-SOD
2.1990-a..122
2.1839+0.ma1
2.7532
2.7468-0.000
2.317884..01s
2.3165
Next Token Prediction Accuracy (ACC) (t)
Base Model
0.5217
0.4276
0,4981
SGD
0,5494 L0.0004
0.4661 10.0013
0.5145 0.514510.0004
SFT
0.5269 10.0032
0.4482.10.0000
SFT DP-SGD
0.5419 =0.0004
0.5450
20.500
0.4537
0.4673
0.5075
0.5126:2p.omm
PE-SGD
0.5435
10.0009
0.5436 10.0000
0.457710.0011
0.4577
10.0011
0.4579 L0.0021
0.5095L0..000
10.0000
0.5097 10.044
23

---

Published as a conference paper at ICLR 2026
D.9 PER-STEP UPDATE
In this section, we include more results on other datasets to demonstrate the effectiveness of PE-SGD compared to baseline DP-SGD within each single update. Like shown in Fig. [5] results on other datasets, PubMed and bioRxiv, all demonstrate the effectiveness of PE-SGD in single step training update, resulting in lower loss and higher accuracy.
0.55
0.515
SGD
DP-SGD
DP-SGD Ours
2/45
2.6
2,40
0.505
0.53
DP SGD
DP SGD
2.2
Ours
2.30
Ours
10
0
10
0
10
10
Training Steps
Training Steps
Training Steps
Training Steps
(a) PubMed
(b) bioRxiv
Figure 15: Per-step loss and accuracy (ACC) using different training methods along the same SGD trajectory Qwen2.5.
D.10 DIFFERENT NUMBER OF TRAINING ITERATIONS
In this section, we examine effect of the number of iterations (or parameter update steps) T has on the final performance by comparing PE-SGD with the most important baseline DP-SGD. Results are included in Fig. 16 Note that, when the same (c. 6D is guaranteed, T also affects the noise level for each iteration. We regardless T. Moreover, using longer training iterations (larger 7') incurs just minor performance updates, especially for PE-SGD, while the performance increase for DP-SGD is relatively larger when < 20. Therefore, the we conclude that, PE-SGD is more effective training method, given its ability in achieving higher performance within fewer updates steps.
from Fig. [6 that PE-SGD consistently outperforms DP-SGD
We further clarify that training with 7 10, each using on average B[M] M = 80 samples. suffices for convergence given the limited private data (M 400). As shown Figs. 4 and T5 the SGD trajectory converges within 10 updates (training steps), with the evaluation loss remaining nearly unchanged.
D.11 DIFFERENT PROMPTING RESULT OF POPRI
For the POPri baseline (Hou et al. 2025), strictly follow their training prompts during DPO training. During the evaluation of the Qwen2.5 and Llama3.2 models fine-tuned with instruction, instruction is also used (i.c., cerruccioo-followiiin evaluation), with loss and accuracy computed only on tokens from evaluation samples. Since different prompts yield different performance, we report results using the better-perfee setter-performin prompt (lower loss, higher accuracy) our main paper. Our prompt (see Table for detail) consistently outperforms theirs, so we use it for main results (Table and report evaluation results using their prompts Table I0 as well.
DP-SGD
PE-SGD
31
0.460
ACC
0.455
2.8
0.450
2.7
5
10
20
40
20
40
Figure 16: Loss and accuracy (ACC) under different number of training iterations (update steps). Qwen2.5 is used for Congressional Speech.
24

---

Published as a conference paper at ICLR 2026
Table 10: Evaluation comparison of the same fine-tuned model from Qwen2.5 using POPri but with
different prompts.
Prompt Source
Method
PubMed
Congressional Speech
bioRxiv
Loss (L)
Accuracy (t)
Loss (4)
Accuracy (t)
Loss (L)
Accuracy (t)
Base Model
2.9836
0.5207
4.0727
0.4256
2.7413
0,4828
POPri
<= 1.0
2.9612
0.5221
3.9846
0.4271
2.6948
0.4861
POPri
6= 00
2.9549
0,5178
3.9922
0,4258
2.7225
0.4813
Base Model
2.9341
0.5217
3,8690
0.4276
2.5311
0.4981
Our
< 1.0
2,9344
0.5221
3.8656
0,4276
2.6178
0,4930
POPri
== 00
2,9487
0,5192
3.8700
0,4276
2.5309
0,4981
D.12 TRAINING EFFICIENCY OF PE-SGD
In this section, we compare the per-iteration runtime of exploiting one sub-sampled private batch from (i.e., a full DPO training step for POPri and single parameter update for other methods) across PE-SGD, DP-SGD, and baselines. We use the default setting M = 400, 3 = 0.2, N 200, L = 2 and fine-tune Qwen2.5 for Congressional Speech. For DP-SGD, we adopt the Opacus package with its built-in gradient accumulation to ensure exactly one model update per sub-sampled batch. Results are shown Table 11
Unsurprisingly, gradient projection based methods (PDP-SGD, GEP, PE- PE-SGD) incur higher computa- tional costs than DP-SGD, as they compute synthetic gradients G in addition to private gradients H,
with G E Rex N being significantly larger than H E R M (E[M) = 80 v.s. N 200). Compared PE-SGD, PDP-SGD and GEP reduce cost modestly by retaining only the top-k eigenvalues 50 v.s. N = 200). POPri is by far the slowest, as each iteration requires several DPO update steps. GEP little bit longer than PDP-SGD as it calculates the residual term.
There might be concerns that generating additional synthetic samples for synthetic dataset (gradient subspace) evolution is in-efficient. In fact, although evolving the synthetic dataset across iterations does introduce additional computational overhead, the required amount of synthetic data per iteration small: around 200 samples per iteration when using generative API fold = (i.e., only around 100 newly generated samples each iteration), as shown in Fig. [())
Consequently, the added cost remains modest and manageable in practice. This is further supported by the empirical generation
times for 100 synthetic samples using Qwen2.5-3B Instruct, reported in Table 12
Table Computational efficiency comparison between different methods. Time are measured for one training iteration, the exploitation of one sub-sampling batch from private dataset B (i.e. entire DPO training for POPri and one-time parameter update for others), without considering the time for synthetic data generation.
DP-SGD
POPri
PDP-SGD
GEP
PE-SGD
Time [s]
9.7
743.8
127.3
135.2
148.6
Table 12: Time costs (seconds, s) for generating 100 synthetic samples with Owen2.5-3B-Innsee Instruct model using different generation APIs on three tasks. Tested with 80G-A100.
RANDOM_API
VARIATIONAL
PubMed
26
41
Congressional Speech bioRxiv
42
D.13 DISCUSSION ON FiG. 3(A)
25

---

Published as a conference paper at ICLR 2026
One interesting phenomenon in Fig. 3(a) is that some data points fall precisely the diagonal line (y = z), as well as on several other straight lines. These patterns correspond to subsets of examples with specific token lengths in the real private dataset of Congressional Speech dataset. This dataset consists of truncated public speech segments. Some samples contained therein are very short, for example, "The hon. Member." or "I declare the motion carried.". This indicates that there only limited number of possible combinations for how many tokens are predicted correctly or incorrectly before and after training, As a result, these short samples naturally cluster lines like y = x, where the r-axis represents accuracy before training (base-moue monsl accuracy) and the y-axis represents the difference in accuracy after training using PE-SGD compared using DP-SGD, For example, the second sample contains 7 tokens, of which only token is correctly predicted by the base model, while 2 and tokens are correctly predicted after fine-tuning using DP-SGD and PE-SGD respectively, resulting in point with
Table 13: "Overlap" metric of the synthetic and private gradient subspace each iteration. Qwen2.5- 3B-Instruct used with M 400, 3 0.2, T 10, N 200. Higher values indicate higher similarity between subspaces.
PubMed
Congressional Speech
bioRxiv
Iteration
w/ evolution
w/o evolution
w/ evolution
w/o evolution
w/ evolution
w/o evolution
11.87
11.87
6.69
6.69
7.11
7.11
2
11.64
9.88
5.80
5.59
5.56
5.58
3
11.93
9.41
4.88
5.52
5.39
5.29
6.36
8.89
5.06
4.97
5.77
4.59
8.72
7.64
4.78
4,56
3.75
3,46
6
8.10
7.54
5.74
4.14
3.71
3.10
7
6,54
6.37
5.38
3.98
3.13
2.91
8
6,88
6.51
4,71
4.51
3,40
3.18
7,81
6.30
5.67
5.00
3.68
3,51
10
8.57
6.85
6.16
5.27
3.89
3.68
E
CONVERGENCE COMPARISON OF PE-SGI WITH PREVIOUS WORKS
We provide a convergence analysis below by demonstrating that "synthetic sample (gradient subspace) evolution improves the convergence of differentially private gradient projection algorithms with high probability".
First, we need to clarify that, the key point of this proof is substantially different from prior works on DP gradient projection (e.g., PDP-SGD (Hou et al. (2025) and GEP (Yu et al. 1021 l). Those works implicitly assume that all synthetic sample gradients follow same distribution as the private gradients. However, as empirically demonstrated in Table 13 (the "overlap" metric between synthetic and private gradient subspaces is far smaller than 50, which indicates that the two subspaces are identical) this assumption does not hold in practice. In contrast, the goal of PE-SGD is precisely to push the gradient subspace of synthetic samples to align with that of the private samples. Therefore, proof focuses characterizing the gradient ali alignment of evolution.
More specifically, Theorem 3.3 in GEP (Yu et al., (021) shows that the convergence rate is posi- tively correlated with the average squared norm of the gradient-projecteo central goal of our proof is to show that evolution reduces the upper bound of this average squared residual term with high probability, which directly improves the convergence behavior under the DP constraints.
residual term. Thus, the
E.1 NOTATIONS AND ASSUMPTIONS
assume that multiple of L, and we have Ñ N/L synthetic samples, each with corresponding gradient g, e x Following Algorithm after using VARIATIONAL_API for next
TThe "overlap" metric (Zhang al. 2025) is defined as the "overlap" between G(O) RPXN and (t))
RPxM Here, we use ocerlap = Σ E5 x3. with $5=1,...50 being the singular values of O E E( and
E(9 E(O heing the top-50 orthonormal basis of (()) ( respectively (Gu al. 2025
26

---

Published as a conference paper at ICLR 2026
generation synthetic dataset creation, get another (L 1) samples, with corresponding gradients Let h E [px1 be the averaged gradient from the private samples.
We make the following assumptions:
We use the clipping method to bound the sensitivity (see corresponding discussion and ex- periments in Section 5.4). and assume that the l2 norm of the gradient projection coefficients from each private sample are below the clipping threshold. this way, we can focus our analysis on the impact of the added DP noise.
We assume that the synthetic gradients have & norm I (i.e., lla.ll = 1, vi = 1, ,,, N) and are orthogonal to each other (i.e., < gt. 9j >= 0, j).
Let 2; = (g. -.)g. h = 9. h 9, gt ll.1| = 9; h the clean gradient projection coeffi-
9:
cient, and |2;] lower bounded by 0. We assume the variant sample zy of x; created using the VARIATIONAL_ANA satisfies by 2; + ( with probability & and žy z; ( with probability 5. This represents the setting where the VARIATIONAL_AFT could either improve or hurt the projection with the same probability.
To select the samples for the next iteration, we add i.i.d. DP noise v (O a²) to the projection coefficient 4i and select the samples with the top noisy coefficient from each variation group. We assume that the index the selected samples are 81,...,88z
E.2 THEOREM AND DISCUSSION
Theorem E.1. (ffigh-Probabilie Improvement from Gaussian--erttriie Variants) With large possibility (> -)), the selected N samples have a sum of squared projection coefficients not smaller than that of the original Ñ samples:
Ñ
Pr
> --
81
1=1
N where L
exp
(-14 -لللاا
{ and & is the CDP of N 1).
Discussion: This theorem (result). Theorem directly implies that the norm of the projected
gradients Σm= z:g: - E increases with high probability across iterations. Consequently, the norm of the residual gradients on the selected Ñ samples, an upper bound on the residual gradients across all N samples, decreases with high probability over iterations.
E.3 PROOF
To prove Theorem E.1 we first prove Lemma E.2 follows.
Lemma E.2. /VARIATIONAL_AR Improvement on Each Group) Let v; be a synthetic sample, with
L L-1 1 samples { 1-1 generated from the VAIRATIONAL_API using as the seed sample, i.e. Ny, is a variant of F4. Denote v; as Ta,0 for clarity. Let Zμ,1 be the gradient projection coefficient of ri,t and v4,1 = Zi,1 + Thi be the coefficient with DP noise where nag (O, g²). Let k; arg maxogisL-1
be the index of the selected sample. We have that with high probability, the selected sample improves the projection coefficient:
(L-1)
Pr([zue, (za,o)) exp
2
20
We first prove Lemma E.2
Proof. Without loss of generality, we assume that z;,0 0. For variants j such as zi.j = 01.0 + Ç. have:
Pr(vi, < v1,0) $ +
20
27

---

Published as a conference paper at ICLR 2026
where & is the CDP of of (00
As the variants generated using the VARIATIONAL_API are independent, we have:
Pr([ _ |2s,ol) - Pr (V1 S L: (Zi,
2,,0 - () or (Zi,)
21,0 + < and Vi,1 vv,,))
1 2 ト 51 Φ
S 20
1-1
1 $ 20
L-1
exp (-(--) + S
1
2
The last inequality holds as x S e- (after using first order Talyer expension). Therefore,
(L-1)
Pr( 2 (2,,o)) exp
2
20
With Lemma E.2 we prove our main theorem Theorem
E.1
Proof. Since each group of variations is operated independently, we have
(L (I-1) 1)
4 5
N/L
Pr
N Pr(v1 SisN: |2,,il 2 (2,,u))
exe
1 exp
20
2 1 exp (1-11)
C
2
20
28


Reviews:
[{"review_id": "EKe3dejrNL", "reviewer": "Reviewer_eNfj", "summary": "The paper proposes an improved subspace projection-based DP-SGD like training algorithm, with the idea of identifying the subspace of gradient using data generated by the model being trained itself. Experiments show the proposed algorithm can achieve better performance when privacy budget is limited where DP-SGD tends to fail.", "questions": "1. Could the authors comment on how generated data quality and original model capability could affect final model performance?\n2. Did authors visualized how the subspace of projection obtained from synthetic data compared with subspace of the model gradient? Also how does prompt template of data generation affect the subspace difference and final performance?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. The algorithm is built on a simple and neat idea of using samples generated by model itself to identify the gradient subspace, removing the need of a in-distribution public dataset and mitigate the potential issue of misaligned gradient subspace.\n2. The algorithm achieves better empirical performance than baselines when epsilon is relatively small.", "weaknesses": "1. The proposed algorithm need additional computation to sample new dataset across iterations, which could make the algorithm relatively slow.\n2. The algorithm performance may depends on the quality of generated data itself, which may imply the original model quality and capability performs a role here. There is a lack of discussion and ablation on this topic.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "9HEAmAiTC6", "reviewer": "Reviewer_FYzi", "summary": "This paper proposes PE-SGD, a framework for differentially private (DP) training that combines gradient subspace projection with an evolving synthetic dataset. The authors argue that existing projection-based DP methods (e.g., PDP-SGD, GEP) suffer from (1) fixed projection subspaces that fail to adapt during training, and (2) unclear choices of where to inject DP noise. PE-SGD addresses these by (a) updating the projection subspace via synthetic data generated by a pre-trained model at each iteration, and (b) empirically identifying that injecting noise into the final projection coefficients yields better performance. Experiments on text fine-tuning tasks with three pre-trained models show moderate improvements over prior methods under tight privacy budgets.", "questions": "1. Can the authors provide any theoretical intuition for why fixed projection subspaces fail — e.g., does the private gradient manifold evolve significantly during fine-tuning, or is it a data-distribution drift effect?\n\n2. Is the proposed “evolutionary” process guaranteed to improve gradient alignment, or could it destabilize the subspace under noise?\n\n3. How is the additional synthetic data generation cost handled? Does this make PE-SGD slower than DP-SGD or PDP-SGD?\n\n4. Since experiments only involve fine-tuning with LoRA, can the method extend to full-model DP training?\n\n5. Can the authors provide at least a conceptual analysis of why adding noise to the projection coefficients yields better performance than to the raw gradients?", "limitations": "", "rating": 2, "confidence": 5, "soundness": 1, "presentation": 2, "contribution": 1, "strengths": "1. The paper tackles a practically relevant problem of improving DP-SGD performance when private data is limited, and situates itself within a growing body of work on gradient subspace methods.\n\n2. The algorithm is presented clearly, and the privacy analysis (Appendix B.2) formally ensures that the DP guarantee remains equivalent to standard DP-SGD.\n\n3. The implementation details, reproducibility statement, and visualizations are carefully documented.", "weaknesses": "1. Lack of depth in problem diagnosis\n\nThe paper highlights two “overlooked” issues in prior work --- fixed projection subspaces and unclear noise placement --- but does not provide a fundamental analysis or theoretical insight into either:\n\n- For the fixed subspace issue, the paper only shows that in one experimental setup, the ℓ₂ distance between the private gradient and the projected noisy gradient increases with training (Fig. 1). This is a narrow empirical observation, not a principled explanation. There is no discussion of why the subspace mismatch arises (e.g., due to model drift, curvature changes, or gradient manifold evolution), nor analysis of whether this holds in general or depends on the dataset, model, or training regime.\n\n- For the noise placement issue, the paper claims prior work lacked rationale --- but it likewise fails to provide one. The finding that adding noise to the final projection coefficients works best is purely empirical, without any theoretical justification or conceptual understanding.\n\nHence, although the paper frames itself as addressing two conceptual gaps, it mostly reiterates them experimentally rather than resolving them.\n\n2. Weak connection between motivation and experiments\n\nThe paper’s motivation centers on differentially private training, yet all experiments are limited to fine-tuning pre-trained models using LoRA. This is a much narrower and simpler regime than full DP training from scratch. Consequently, the experimental evidence does not convincingly demonstrate the method’s broader applicability to general DP optimization, nor its scalability to large models or high-dimensional gradient spaces.\n\n3. Overstated claims of originality\n\nBoth key ideas --- evolving a projection subspace and empirically selecting a noise placement --- are relatively incremental extensions of existing work rather than conceptually new mechanisms. The “evolution” is implemented via synthetic data regeneration using a generative model API, but there is no evidence that this evolution is guided or optimal, beyond heuristic sample resampling. The connection between this process and gradient-space adaptation remains speculative.\n\n4. Limited theoretical depth\n\nThere is no analysis of convergence, bias-variance trade-offs, or the statistical efficiency of evolving subspaces under DP noise. The privacy argument relies entirely on post-processing invariance, which sidesteps the deeper question of whether iterative synthetic data generation introduces new privacy risks or affects gradient alignment stability.\n\n5. Experimental limitations\n\nWhile quantitative results show small gains under ε = 1.0, improvements are modest and primarily on fine-tuning tasks. There is no ablation for computational cost or efficiency, nor experiments beyond text data. The results, though positive, do not convincingly demonstrate a substantial advance in the state of the art.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "2QwqAMQmyk", "reviewer": "Reviewer_zNzf", "summary": "The paper presents an approach based on subspace projection using public data, namely using gradients evaluated on synthetic data that is generated using a public model. The DP-SGD algorithm is modified so that the gradients ar first projected in a certain way using these public gradients, the noise is added only to the small-dimensional objects which are then afterwards projected back to the high-dimensions. Experiments support the benefits of this approach.", "questions": "In  many existing projection methods the projection is carried out using orthonormal bases which is arguably much more stable alternative. Why this version?\n\nHow do you know which part of the generated synethic data is useful for the gradient projection? Why aren't there methods to filter out the most useful public data / gradients?", "limitations": "", "rating": 2, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "- Interesting and a novel approach of using synthetic data to generated gradient projection bases. While there are lot existing works considering the projection of DP-SGD with public bases to reduce the amount of added noise, I am not aware of this specific approach in the literature, i.e., using public models to generate synthetic data for this task.", "weaknesses": "- the work overlooks several relevant and important prior studies, e.g., the work by [Gu et al., 2025](https://arxiv.org/pdf/2303.01256) and many others works that consider adding noise only to projected DP-SGD gradients. Existing works study different aspects of these methods. E.g., Gu et al. provide tools for privately estimating which public basis is most suitable for private gradients, an aspect highly relevant to the current paper’s methodology, yet not discussed at all.\n\n- It remains unclear, why is the projection carried out the way it is done in Alg. 1. In  many existing projection methods the projection is carried out using orthonormal bases which is arguably much more stable alternative. There are no comparisons.\n\n- The scales in the experiments are very narrow: the differences in the performance are in absolute sense small (look e.g. at Fig. 6,7,8). As the contribution is purely heuristical, the experimental part should be stronger and done much more carefully.\n\n- No theoretical support for the proposed method.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "pH4AdLzEZr", "reviewer": "Reviewer_v9H2", "summary": "DP-SGD and its variants often suffer from degraded performance when the number of private samples is limited. Prior work attempts to address this issue by computing a gradient projection subspace from public data and adding perturbations within that subspace. However, there is a mismatch between the fixed projection subspace and the dynamically evolving training process, which hampers model performance. Moreover, the optimal location for injecting noise within the projection process remains underexplored.\n\nThis paper proposes PE-SGD, which dynamically updates the non-private dataset based on the evolving model during training, thereby reducing the L2 distance between the approximate gradients and the real gradients. In addition, the paper systematically compares different noise injection strategies and finds that adding noise to the final projection coefficients yields more stable and superior performance than injecting noise at other stages. Extensive experiments demonstrate the effectiveness of PE-SGD, particularly in improving performance on long-tailed samples within the private dataset.", "questions": "- The reviewer would like to understand why, in the right plot of Figure 3(a), some data points appear to lie on the same diagonal line. Is this phenomenon due to numerical precision issues, or does it reflect certain regularities inherent in the data?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The paper identifies an important limitation in existing gradient projection approaches: the fixed projection subspace is incompatible with the dynamic nature of training. Figure 1 effectively illustrates how the subspace derived by prior methods diverges from the true gradient subspace as training progresses, revealing a key reason behind their performance degradation.\n- Methodologically, the work integrates the strengths of gradient projection and differentially private synthetic data approaches (e.g., PDP-SGD and Aug-PE). It introduces a novel design that computes the projection subspace using dynamically generated samples, achieving lower next-token prediction loss and higher next-token prediction accuracy on text generation tasks.\n- The paper provides comprehensive experiments to explain why PE-SGD outperforms prior methods. For example, analyses of long-tailed samples before and after training highlight that PE-SGD captures richer knowledge from private data and mitigates the long-tail issue. Moreover, per-step update experiments clearly demonstrate that PE-SGD achieves a larger decrease in loss compared to DP-SGD.", "weaknesses": "- In the SynDataGeneration component, the authors employ VARIATIONAL_API to generate synthetic data. However, the paper lacks an evaluation of the quality of these generated samples, such as their textual quality and diversity. This step is crucial for future extensions of the work, as insufficient diversity in data generated by VARIATIONAL_API could lead to performance degradation when applied to other datasets.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)