Synthesize Privacy-Preserving High-Resolution Images via Private Textual Intermediaries
Authors: Haoxiang Wang
Zinan Lin
Da Yu
Huishuai Zhang
Venue: NeurIPS 2025 poster ()
Keywords: differential privacy
private evolution
private text bridges
Primary area: social_and_economic_aspects_of_machine_learning

Abstract: Generating high-fidelity, differentially private (DP) synthetic images offers a promising route to share and analyze sensitive visual data without compromising individual privacy. However, existing DP image synthesis methods struggle to produce high-resolution outputs that faithfully capture the structure of the original data. In this paper, we introduce a novel method, referred to as Synthesis via Private Textual Intermediaries (SPTI), that can generate high-resolution DP images with easy adoptions. The key idea is to shift the challenge of DP image synthesis from the image domain to the text domain by leveraging state-of-the-art DP text generation methods. SPTI first summarizes each private image into a concise textual description using image-to-text models, then applies a modified Private Evolution algorithm to generate DP text, and finally reconstructs images using text-to-image models. Notably, SPTI requires no model training, only inferences with off-the-shelf models. Given a private dataset, SPTI produces synthetic images of substantially higher quality than prior DP approaches. On the LSUN Bedroom dataset, SPTI attains an FID $=$ 26.71 under $\epsilon=1.0$, improving over Private Evolution’s FID of 40.36. Similarly, on MM-CelebA-HQ, SPTI achieves an FID $=$ 33.27 at $\epsilon=1.0$, compared to 57.01 from DP fine-tuning baselines. Overall, our results demonstrate that Synthesis via Private Textual Intermediaries provides a resource-efficient and proprietary-model-compatible framework for generating high-resolution DP synthetic images, greatly expanding access to private visual datasets. Our code release: https://github.com/MarkGodrick/SPTI

Synthesize Privacy-Preserving High-Resolution Images via Private Textual Intermediaries
Haoxiang Wang Peking University
Zinan Lin
Da Yu
Microsoft Research
Google Research
wanghaoxiangostu. pku, edu. en
zinanlin@mi wwwiinmmmiiiooooiommoeeeeooooiee com
dayuwork@google.com com
Huishuai Zhang?
Wangxuan Institute of Computer Technology, Peking University State Key Laboratory of General Artificial Intelligence zhanghui sethhiissuhiiie @pku. edu. cn
Abstract
Generating high-fidelity, differentially private (DP) synthetic images offers promising route to share and analyze sensitive visual data without compromising individual privacy. However, existing DP image synthesis methods struggle produce high-resolution outputs that faithfully capture the structure of the original data. In this paper, we introduce novel method, referred to as Synthesis via Private Textual Intermediarries (SPTT), that can generate high-resolution DP images with easy adoptions. The key idea is to shift the challenge of DP image synthesis from the image domain to the text domain by leveraging state-of-the-art DP text generation methods. SPTI first summarizes each private image into a concise textual description using image-to-text models, then applies a modified Private Evolution algorithm to generate DP text, and finally reconstructs images using text-to-ine rext-to-image models. Notably, SPT/ requires no model training, only inferences with off- the-shelf models. Given a private dataset, SPTI produces synthetic images of substantially higher quality than prior DP approaches. On the LSUN Bedroom dataset, SPT7 attains an FID = 26.71 under = 1.0, improving over Private Evolution's 's FID of 40.36. Similarly. on MM-CelebA-HO. SPTI achieves an FID 33.27 at 1.0, compared to 57.01 from DP fine-tuning baselines. Overall, our results demonstrate that Synthesis via Private Textual Intermediaries provides a resource-efficient and high-resolution DP synthetic images, greatly expanding access to private visual datasets. Our code release: https: /t //g/thuh. Com/MarkGodrickk/
framework for generating
1 Introduction
The past few years have seen an explosion in the capabilities of state-of-the-art generative models. Large diffusion and autoregressive models can produce photorealistic images, coherent long-form text, and convincing multi-modal outputs with little more than a prompt [36, 17, 30, 43, 29, 56, 48, 47). However, when these powerful public models are applied domain- or user-specific tasks, some form of adaptation is often required, whether it be expensive fine-tuning on private data or in- context learning with user examples. At the same time. the use of private or sensitive data raises serious privacy concerns: direct fine-tuning of a model on proprietary images or user uploads risks memorization and potential leakage of personal information [7, 34, 49, 55).
TCoresponding author.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

DP Finetuning
Private Evolution: Images
Seleched Samplex
Cenernt Semples
DP fine auno
DP-Synthetic
DP-Bynehatic
Cenertte Samples
Synthesis via Private Textual Intermediaries (Ours)
Private Data
Random Generated
DP-Synthatic
DP-Synthetlc
(Images)
Text Data
Text Data
Data
Through Aug-PE
As Reference
Model
Pro-tained
agermooi
Figure 1: Overview of the Synthesis via Private Textual Intermediaries (SPTT) framework for differentially private (DP) synthetic data generation. Top left: DP fine-tuning framework. A pretrained model is fine-tuned private data under DP constraints, and the resulting model is used to generate DP synthetic data. Top right: Private Evolution (PE) on images. This method begins by randomly generating candidate samples, which are then compared to private data. Samples are selected based on a voting mechanism and further perturbed to produce a new generation of samples. Bottom: Synthesis via Private Textual Intermediaries (SPTI) framework. Private image data is served as reference. A modified Augmented Private Evolution (Aug-PE) method [53] then applied to generate DP synthetic text data, which is subsequently transformed into DP synthetic images using diffusion model API.
Figure 2: DP-synthetic images from dataset European Art (c = 1.0).
Figure 3: DP-synthetic images
Figure 4: DP-synthetic images from dataset LSUN Bedroom 1.0).
from dataset Wave-ui-25k (e 1.0).
Broadly speaking, there are two strategies to harness private data under formal differential privacy guarantees [10]. The more direct approach is differentially private (DP) fine-uning [57, 25, 6, 23, 35, 31, 8). which gradient updates are clipped and noised (e.g., via DP-SGD [1]). While conceptually straightforward, this method demands substantial expertise and computational resources. For instance, fine-tuning state-of-the- ther-rrtt model such as DeepSeek V3 [29] requires hundreds of GPUs, not to mention the engineering complexity of per-sample gradient clipping and noise calibration of DP-SGD [15]. Moreover, many proprietary foundation models do not even permit user fine-tuning, effectively precluding this route.
alternative to generate DP synthetic dataset that mimics the distribution of the private data [27, 53, 28, 12] and then use that synthetic data for downstream pepptaatoo-eithhe by fine- tuning or by providing examples in-context. This "generate-then-adane
paradigm sidesteps the need to directly privatize large model but poses its own challenges: how can one synthesize high-fidelity images (or other modalities) that both faithfully capture the structure of the original data and satisfy strong DP guarantees, all without incurring substantial compute overhead?

---

In this work, we introduce a Synthesis via Private Textual Intermediaries framework that leverages the remarkable text understanding of modern models as a bridge between private data and high-quality generation. Our key insight is twofold:
1. Text generation is one of the most successful modalities for AI, so is the private text generation. Private text generation can achieve similar performance to directly differentially private fine-tuning LLMs [53, 59], and text models can be privately adapted far more easily than their image counterparts.
Text unifies modalities. In modern models, natural language often serves as a universal interface, with wide range of off-the-shelf models available for both generating images. video, audio, and other modalities from text [43, 40, 30], and for generating text from these modalities [47, 18].
Building on these observations, we propose Synthesis via Private Textual Intermediaries (SPTT) (Figure 1). Given a private image collection, SPTI first converts the images into concise textual summaries using standard captioning model. It then adapts Private Evolution [27,53, 28, 50], DP synthetic data algorithm that only requires model inference, to generate DP-sanitized captions that reflect the private caption distribution. Finally, it uses an off-the-shelf text-to-image models to sample candidate images conditioned on these DP captions. Notably, all steps in SPTI rely solely on the inference APIs of existing models without any additional training, allowing SPT7 to leverage state-of-thhe-ar models behind proprietary APIs something that DP fine-tuning approaches cannot achieve.
We evaluate SPT/ on standard benchmarks. On LSUN Bedroom, SPT7 achieves FIDD = 26.71, markedly improving over Private Evolution's 's prior best of 40.36 for 1. On MM-CelebA-HQ. attains FID 33.27 versus 57.01 for DP fine-tuning baselines for c = 1. In both cases, SPTI generates high-resolution synthetic images with significantly better quality than either cost-intensive DP-SGD approaches [43] or previous API-based approaches [27]. Figures 2, 3, 4 show generated images from SPTI.
Our contributions are as follows:
Conceptually, we introduce Synthesis via Private Textual Intermediaries, novel framework that uses text as an intermediate representation to bridge off-the-shelf multimodal LLMs and achieve rrvacc-preseeving. high-resolution image generation by leveraging state-of-the-art DP text generation methods.
Technically, we develop a novel Private Evolution voting mechanism that privately select text descriptions by voting in image representation space. It achieves strong privacy while ensuring strong resemblance to the private image data.
Empirically, we demonstrate that SPT7 generates high-quality, high-resolution images across diverse datasets, and can readily incorporate future advances in multimodal generation. Extensive ablation studies validate the effect of each components of the algorithm in achieving state-of-the-art performance.
The rest of the paper is organized as follows. In Section 2 we review background and preliminaries; Section 3 details the SPT/ method; Section 4 presents experimental results and analyses; Section discusses related work; Section 6 concludes with a discussion of future directions; and Section 7 faithfully describes the potential limitations of the proposed method.
Preliminaries: Differential Privacy and DP Synthetic Data via APIs
Differential Privacy. One mechanism M is said to be (c, 8DDP D and D' which differ in single entry and for any observation set S of outputs of M, one has
P if for any two neighboring datasets
PD( E S) < ef Pr(M(DD) E S) + 5.
The intuition of 6DP requires that any single sample cannot influence the mechanism's output too much.
DP Synthetic Data via APIs. The DP synthetic data problem is formulated as DP Wasserstein Approximation Lin al. [27]. Given a private dataset Dprio {ar : i E [Npriel} with Nprin samples (c.g., images), the goal is to design an (c, d)- algorithm M that outputs a synthetic dataset
3

---

Dayn faf E [Negm]] with N Neyn samples, where the Wasserstein p-distance W W (Dprie, Daya) minimized, w.r.t the distance function -( and some p 2 1.
Private Evolution (PE). [27] is a recently proposed differentially private synthetic data generation that relies solely API access off-the-shelff models. The PE algorithm consists of following steps:
Random Initialization. Call RANDOM_APE to draw an initial set of synthetic candidates from the foundation model.
Repeat the following routine several times
1. Private Voting. Using the private dataset, each real sample "votes" for its nearest
synthetic candidate under the embedding network, producing a (noisy) histogram via the DP voting function.
Resampling. Draw new batch of synthetic points by sampling from this private histogram.
Variation. For each drawn point, invoke VARIATION API to produce novel samples that are semantically similar to the selected candidate (e.g., variations of an object in an image).
This procedure can generate synthetic data whose distribution closely matches that of the private dataset while controlling privacy leakage using private histogram algorithm.
3 Method: High Quality Image Synthesis via Private Textual Intermediaries
Before describing our method, we first outline the high-level motivation. The goal of differentially private (DP) synthetic data generation mimic the distribution of private data, no matter how rare or complex the private data are, e.g., high-resolution or UI-like images, while preserving privacy, However, directly synthesizing such images by using Private Evolution [27] poses two key challenges. First, rare cases are difficult to generate if they are far from typical data distribution that the model has learned on. Second, high-resolution images require the model to correctly render all fine-grained details across large output space, which is particularly challenging for generation models.
To address these issues, we shift the problem from the image domain to the text domain by introducing textual intermediaries. Text representations are semantically rich yet lower-dimensional making them significantly more amenable to efficient DP synthesis. Recent advances have shown that privately trained language models [59] or DP synthetic text [53] can achieve strong utility while satisfying DP. Moreover, text serves as a universal interface for multimodal generation: a single caption can condition powerful off-the-shel models to generate images, videos, audio, and more. This makes our approach have the potential to be broadly applicable across modalities, though we focus on image synthesis in this work.
We propose Synthesis via Private Textual Intermediaries (SPTT), a novel framework that leverages this insight. SPTI first converts private images into captions, applies our newly-designed Private Evolution (PE) algorithm to these descriptions under a DP guarantee, and then synthesizes images using a text-to-image generator. This design ensures strong privacy while maintaining high utility. An overview is shown in Figure
3.1 The Design of Synthesis via Private Textual Intermediaries
Our SPT/ consists of three stages: (1) using image-to-text model to caption private images to obtain text descriptions, (2) applying Private Evolution the text data under DP constraints, and (3) using text-to-image model to generate synthetic images using the evolved text. The process is outlined in Algorithm 1.
Images to Texts. Given a private image dataset D, we begin by captioning each image into textual description using off-the-shelf image captioning model. This results in a corpus of private text data T, which serves as a proxy for the original image data during subsequent privacy-preserving operations.

---

Private Evolution in Text Space. Augmented Private Evolution (Aug-PE) [53] is an iterative, population- bassd method for generating synthetic text data under DP constraints. We modify by replacing its voting mechanism with new voting strategy. At each iteration, pool of candidate texts generated, evaluated, and selected. For selection, we assign each candidate a probability using Image Voting process, which is described with detail in Section 3.2. We then select the candidates by this probability to produce the next generation. This process preserves privacy while exploring the text space efficiently. We provide a detailed version of this modified algorithm 2 in Appendix A.
Evolved Texts to Images. Once we obtain the evolved synthetic text set T', we use a high-quality text-to-image diffusion model (e.g.. state-of-the-art open-source models [44] or commercial APIs) to convert each text back into an image. Since the evolution occurred entirely in the text domain, the resulting images inherently DP-compliant due to post processing properties of DP.
RANDOM API and VARIATION API. Our method follows Aug-PE [53] method, also applying RANDOM_AP! and VARIATION_AP in algorithm Aug_PE_ Bes__PE__mae_________ Lmage_Votiig For RANDOM API, prompt the LLMs to generate random image captions from scratch, denoted as BANDOM_API(A) This means RANDOM_AP will generate N samples. For VARIATION API, also prompt the LLMs to generate variations the given image captions, denoted as VARIATION VARIATION_API(C). This means VARIATION_APP will generate variants of equal quantity from text data C.
Algorithm SPTI: Privately Synthesize High- High-Resolution Images via Synthesis via Private Textual Intermediaries
Require: Private image dataset D, Aug_PPE_Image__on
Ensure: Synthetic images D'
1: Convert D to text descriptions T using captioning model
2: Apply Private Evolution: T Aug_PE_ Image_Votiinge
3: Generate images 'D' from T' using a text-to-image diffusion model
4: return D
3.2 Private Evolution in Textual Space with Voting by Image Representations
Running Aug-PE on image captions can generate synthetic captions that are similar to those of the original private images in the text domain. However, the final text-to-image models work in random and complex way. As a result, even accurate captions in the text domain may not necessarily produce images that closely match the original private ones in the image domain.
Based on this insight, we propose Image Voting (Algorithm 2), a cross-modal selection mechanism integrated into the PE process. Rather than selecting candidate texts solely based textual similarity, we first generate images from the candidate texts using the text-to-image model. These synthesized images are then embedded using image encoder (e.g., Inception-v3 [46, 39]), along with the private images.
For each private image embedding, identify its nearest neighbor among the generated image embeddings (e.g., via Euclidean distance). The corresponding synthetic text associated with the nearest image is assigned one vote. After accumulating votes across all private images and adding noise, we normalize the vote counts into a probability distribution over candidate texts, which is then used sample the next generation. This reverse-nearess neiisst- neighbor approach prioritizes synthetic texts that produce images broadly aligned with the private dataset. Additionally, the Image Voting process notably requires textual reference, it directly leverages the private images themselves as guidance during generation.
3.3 Privacy Analysis
We analyze the privacy guarantees of our proposed SPTI framework, which synthesizes image data under differential privacy (DP) by operating in the text space. Our goal is to ensure that the final synthetic images D' are generated through process that satisfies (e, DDP with respect to the private dataset D.
5

---

Privacy-Critical Step. The only step in the SPTI pipeline that accesses the private data is the Private Evolution module, specifically through the similarity-bart based voting mechanism algorithm Aug_ PE_ Image_Voting (Algorithm 2). This module operates embeddings of the private image dataset and is used to guide the sampling of synthetic text candidates. Thus, our privacy analysis focuses on this step.
Voting Mechanism. For each private image embedding, we identify its nearest neighbor among the generated image embeddings and increment vote count histogram H indexed by the candidate text that produced the nearest image. This step corresponds to a histogram query over the private data.
To ensure differential privacy, add Gaussian noise. (0, o to the vote histogram H. which is equivalent to applying the Gaussian mechanism to a function with bounded sensitivity.
Sensitivity Analysis. The histogram query has L2 sensitivity at most since each private image contributes a vote to only one candidate (i.e., changing a single image can affect the histogram by at most in one coordinate). This satisfies the precondition for the Gaussian mechanism.
According to the Gaussian mechanism [4], adding noise of variance r² per coordinate to a function with L2 sensitivity 1 ensures (c, 6)-D, provided:
(--1-11- - so S 8.
20
Privacy Composition. According to the adaptive composition theorem Gaussian mechanisms [9], applying the above Gaussian mechanism across G iterations satisfy (c, 5)-P,, provided:
VG 2x
VG 2a
80
E0
VG ef
S 6.
VG
Post-processing Immunity. All downstream steps in SPTI SPTI-including text mutation and image gen- eration via fixed diffusion model-deppee solely on the privatized output of the voting mechanism. By the post-processing property of DP, these steps incur no additional privacy cost.
Overall Guarantee. Hence, SPTI satisfies (s, ddifferrrel privacy with respect to the private image dataset D, provided that the noise scale is properly chosen satisfy Eq. 1.
Experiments
In this section, we evaluate the effectiveness of our proposed method through comprehensive experi- ments. Section 4.1 describes our experimental setup. Section 4.2 and 4.3 present the main results, where we compare our approach against state-of-the-art methods across multiple benchmarks to demonstrate its superior performance. Section 4.4 includes ablation studies analyzing the impact of the image voting mechanism and different model APIs. Our findings show that the image voting technique consistently improves performance, while the quality of the synthetic data remains robust across different language and diffusion model APIs. We also represent extended ablation study in Appendix C.
4.1 Experiment Setup
Datasets. To mitigate concerns that existing benchmarks may have been included in the pretraining data of diffusion models, we carefully select only essential benchmarks with known publication dates, prioritizing those that were introduced recently. Additionally, we create a new dataset by extracting data from the open-source Blender movie Sprite Fright. We evaluate our SPTI method on six datasets: LSUN Bedroom [58], Cat [27], European Art [41]. Wave-ui-25k [61], MM- CelebA-HQ [51, 52, 33, 21, 24), and our newly constructed Sprite Fright. These datasets are selected because each represents realistic, srivacy- scenario. For instance, LSUN Bedroom captures aspects of an individual's private living space, while Wave-ui-25k consists of screenshots that may reveal personal information from electronic devices.

---

Baselines.
We compare our method with two state-of of-the-art DP baselines:
Private Evolution [26]: A recently proposed DP image synthesis method that operates in the image embedding space via private evolutionary strategies.
DP Fine-tuning [32]: This method fine-tunes a diffusion model on private data under DP constraints using DP-SGD, allowing direct sample generation while preserving privacy.
Models. Our pipeline requires a captioning model, large language model (LLM),, and diffusion model. For the caption model, we use GPT-4o-mini [38] and Swen-VI.-Maal [3]. Our experiments show these two models produce similar performances, and detailed results are presented in Section 4.4, For the LLM and diffusion model, we use SDXL-Turbo [44] for image synthesis. Additional comparisons with newer and more advanced models are discussed in our ablation study in Section 4.4.
[2] for text generation and
Evaluation Metries. To evaluate the quality of our differentially private synthetic samples, the Fréchet Inception Distance [16] (FID) between the original private dataset and the synthetic one. Lower FII scores indicate better alignment between the two distributions in terms of visual features.
4.2 Evaluation
FID score. We compare SPT/ with two state-of tfate-of-the-an approaches: Private Evolution and DP fine-tuning, across multiple datasets. The results are presented in Table 1 and Table 2. For fair comparison, we use a Latent Diffusion Model (LDM) [42, 5] pretrained on LAION-400M [45] as backbone for all methods. The experimental results demonstrate the superior performance of SPTT. For more details on the experimental settings, please refer to Appendix E.
Voted samples and variants. In each iteration, our results generate two categories of samples. One can be categorized as voted samples selected from previous generation, while the others can be viewed as variants generated from these samples, which we call variants. During experiment evaluation, we often find voted samples perform better on FID evaluation, which can be seen in Figures 5, 6. For consistency, we present all our experiment results using FID calculated from voted samples. For more figures on experiment results, please refer to Appendix B.I.
FID over Iterations
FID over Iterations
Our mathodjvcted]
Our methodiveted)
Our methodtvaraamss
Our methodqvariamss)
PC
160
140
0
120
100
Iteration
Iteratio
Figure 5: Experiment results LSUN Bedroom
Figure 6: Experiment results European Art datasct
dataset
4.3 Downstream Task
To comprehensively evaluate the generalization and transferability of the representations Iearned by proposed SPTI. we conduct binary classification task the CelebA dataset. We select the attribute SearinggLipstine ck as the target label, and approximately 53% of the samples are without lipstick and 47% are with lipstick. We generate 2,000, 4,000, 6,000. 8,000, and 10,000 samples using SPTI, and PE, respectively, and compare their accuracy to the model trained by the corresponding number of ground-truth training samples. A WRN-40-4 model is then trained on each generated data samples under identical training settings, and the test accuracy is evaluated using the ground-truth labels.
The test accuracy in Figure shows that SPT7 is more effective than PE method. This could be due to the higher diversity of data generated by SPTI compared to that of PE (see Figures 36, 37). We also
7

---

E = 10
E == 5
6 =
SPTI (ours)
25.88
25.87
26.39
LSUN Bedroom
PE image
41.72
41.08
40.36
DP-finetune
31.28
31.34
31.76
SPTI (ours)
101.57
102.31
106.15
Cat
PE image
51.00
51.29
64.62
DP-finetune
148.05
148.46
148.75
SPTI (ours)
41.42
42.71
57.64
European Art
PE image
76.25
74,41
76.50
DP-finetune
61.10
61.82
63.97
SPTI (ours)
20.16
22.53
35.18
Wave-ui-25k
PE image
39.28
48.95
50.45
DP-finetune
49.84
52.09
62.08
SPTI (ours)
142.48
141.49
157.31
Sprite Fright
PE image
195.58
181.77
197.13
DP-finetune
148.19
151.94
161.25
Table I: FID values (lower is better) across multiple datasets compare three different DP methods: SPTI, PE Image, and DP-finetume.
Method
model
6 = 10
6
sdxl-turbo
26.02
34.17 17
SPTI (our method)
LDM pretrained model
16.59
33.41
PE
sdxl-turbo
53.32
50.45
finittutuee
LDM pretrained model
50.13
57.01
Table 2: FID values (lower is better) the MM-Celeba-HQ dataset to compare different DP generation methods: SPTI, PE Image, and DP-finetune.
give results from models trained with ground-truth training samples to show that selected number of samples is sufficient to train a good binary classifier. Additionally, we provide model performance during training and examples of generated samples in Appendix B.3.
4.4 Ablation Study
We conduct a series of ablation studies to investigate the contribution of each component of our proposed method. Specifically, analyze the impact of image voting in the main text, while additional studies on hyperparameters for the LLM and diffusion APIs are provided in appendix due space constraints. All experiments are conducted under the same conditions described Section 4.1.
4.4.1 SPTI Without Image Voting
To evaluate the effect of image voting, we substitute with a baseline approach and present comparative results in Table 3. Specifically, we compare our image voting strategy with the original Aug-PE method, where synthetic text is voted using generated private text data rather than the original private image data. We observe significant improvement in FID, suggesting that image-bascd based voting

---

Munber of Samples
Figure Test Accuracy on CelebA dataset with different training samples. SPTI achieves better accuracy under the same configuration because SPTI generates samples with more diversity than PE method. The experiment results demonstrate that our method produces data with sufficient fidelity, enabling the model to effectively leam from the generated samples.
plays an important role in enhancing the quality of the generated data. For more figures on experiment results, please refer to Appendix B.2.
10
5
1
Image Voting
27.92
29.50
38.00
LSUN bedroom
Text Voting
39.82
37.83
41.17
Image Voting
101.57
102.31
106.15
Cat
Text Voting
110.51
106.07
106.02
Image Voting
46.69
48.45
64.04
Europeart
Text Voting
74.37
74.04
74.91
Image Voting
34.39
42.02
70.87
Wave-ui-25k
Text Voting
92.53
92.11
95.77
Table 3: FID values (lower is better) across multiple datasets to compare the SPT/ method with Image Voting and that with Text Voting.
5 Related Work
Differentially private (DP) synthetic data generation has been studied extensively. Early work focused on query-based and statistical approaches, such as the Multiplicative Weights Exponential Mechanism (MWEM) algorithm, which iteratively measures selected queries under DP and synthesizes data to match those noisy answers [14]. PrivBayes learns a Bayesian network with DP noise added to its parameters, then samples synthetic records from the privatized network [60]. While these methods provide provable guarantees, they often struggle with high-dimensional data due to the curse of dimensionality.
The advent of deep generative models enabled DP-GANs and VAEs. Xie et al. [54] apply DP-SGD to GAN training bound privacy leakage, and PATE-GAN uses the Private Aggregation of Teacher Ensembles framework to train a generator with DP guarantees [20]. Although these models can capture complex distributions, they ineur utility loss from noise and training instability.
Recently, diffusion (score-based) models have shown promise under DP. Ghalebikesabi et al. [11] privately fine-tune pre-trained diffusion model on sensitive images, achieving state-of-the-art FID scores on CIFAR- 10 and medical imaging benchmarks. This demonstrates that modern diffusion architectures can yield high-quality private synthetic data. Liu et al. [31] proposed DP-LDM bench-

---

mark to train latent diffusion models for DP image generation. This provided an outstanding baseline to compare with our method.
parallel line of work investigates inference-only, API-based DP synthesis. Lin et al. [27] introduce Private Evolution, which injects noise during generation from frozen model, matching or surpassing retrained approaches on image benchmarks. Xie et al. [53] extend this idea to text, querying large language model's API with augmented DP sampling to produce synthetic text with strong utility. Lin et al. [28] further extend Private Evolution by incorporating non-neural-network data synthesizers, such as computer graphies tools. This approach delivers better results in domains where suitable pre- trained models are unavailable and unlocks the potential of powerful non-neural-nette data synthesizers for DP data synthesis. Google has also demonstrated an inference-only LLM mechanism for DP data generation [13].
concurrent study also explores generating DP synthetic images by first producing text captions of private images [22]. Their hierarchical DP fine-tuning approach trains a model to generate privatized album descriptions and then photo descriptions conditioned on those. In contrast, we adopt the PE method [53], which requires only API access to foundation models, and further specialize it by using embeddings of the generated images, rather than text captions, during the voting process.
6 Conclusion
We have presented Synthesis via Private Textual Intermediaries (SPTT), a novel framework for differentially private (DP) image synthesis by leveraging existing powerful image-to-text and text-to- image generation models. By first converting each private image into concise textual summary via off-the-shelf image-to-ter models, then applying a modified Private Evolution algorithm to guarantee (e, 6)P on the text, and finally reconstructing high-resolution images with state-of-the-art text-to- image systems, SPT/ sidesteps the challenges of training DP image generators. Importantly, our approach requires no additional model training, making it both resource-efficient and compatible with proprictary, A.Placccess-nil models. Empirical results demonstrate significant improvement over existing methods. These findings validate that shifting the DP burden to the text domain enables the generation of high-fidelity high-resolution synthetic images under strict privacy guarantees, offering practical path toward sharing and analyzing sensitive visual datasets.
7 Limitations
Domain Generalization Synthesis via Private Textual Intermediaries' performance is intrinsically tied to the coverage and robustness of the underlying image-to-text and text-to-image models. Although multimodal large language models are continually improving, they still underrepresent certain data domains (e.g.. specialized medical imagery. aanoteseensing data, niche artistic styles). In such underrepresented or distributtio settings, SPT/ may fail to capture critical visual features or produce coherent synthesis. Moreover, biases and domain shifts in the pre-trained models can further degrade image fidelity and diversity.
Computational Overhead Our full data generation pipeline involves two major components: an 8B 1B-parameter large language model (LLM) and a 3.5B ---aaammeterr diffusion model. Under our current implementation, a complete data generation run takes approximately 19 hours on single NVIDIA A800, or 15.5 hours 2xNVIDIA A100 GPUs, with a peak memory requirement of about 70 GB.
In comparison, the baseline PE method Juses only single 3.5B model, requires LLM, and completes in about 7 hours on a single NVIDIA A100 GPU, with a peak memory requirement of about 35 GB. While our method has higher compute demands, introduces a semantically enriched generation process through language-guided synthesis, which we show leads to superior data diversity and downstream performance.
We believe the additional cost is justified by the significant gains in quality and generalization, and we note that our pipeline can be modularly optimized (e.g., via model distillation prompt caching) in future work to reduce the runtime.
As for API usage, caption process will take 250M tokens for 10,000 images. During data generation, L.M will process 7.3M tokens during each iteration, and diffusion model will process 5M tokens in each iteration. A total of 10 iterations will count for 123M tokens.
10

---

Acknowledgement
The authors would like thank the anonymous reviewers for their valuable feedback and suggestions. The work of Haoxiang Wang and Huishuai Zhang was supported in part by National Natural Science Foundation of China (Grant No. 62576015).
References
[1] Martin Abadi, Andy Chu, Ian Goodfellow, H Brendan McMahan, llya Mironov, Kunal Talwar, and Li Zhang. Deep learning with differential privacy. In Proceedings of the 2016 ACM SIGSAC conference on computer and communications security, pages
2016.
[2] AltMeta. Llama 3 model card. 2024. URL httttt////gtthitt hetggittuu.co//mmtaaaeeeeee com/meta- 11ama/ aaamaallama 11amm3// blob/maihe B.lob/main/MDDE__CO md.
[3] Jinze Bai, Shuai Bai, Shusheng Yang, Shijie Wang, Sinan Tan, Peng Wang, Junyang Lin, Chang Zhou, and Jingren Zhou. Qwen-vl: A versatile vision-language model for understanding. localization, text reading, and beyond. arXiv preprint arXiv:2308. /2966 2023.
[4] Borja Balle and Yu- Yu-Xiang Wang. Improving the gaussian mechanism for differential privacy: Analytical calibration and optimal denoising, In International Conference on Machine Learning. pages 403. PMLR, 2018.
[5] Andreas Blattmann, Robin Rombach, Kaan Oktay, and Bjöm Ommer. Retrieval- auggente diffusion models, 2022. URL https: //axxvv arxiv. org/ abs 2204. 11824.
[6] Zhiqi Bu, Yu-Xiang Wang, Sheng Zha, and George Karypis. Differentially private bias-term only fine-tuning of foundation models. arXiv preprint arXiv:22/0.000366 2022.
Herbert-Voss, [7] Nicholas Carlini, Florian Tramer, Eric Wallace, Matthew Jagielski, Ariel Herbert- Voss, Kather- ine Lee, Adam Roberts, Tom Brown, Dawn Song, Ulfar Erlingsson, al. Extracting training data from large language models. USENIX Security Symposium, 2021.
[8] Tim Dockhorn, Tianshi Cao, Arash Vahdat, and Karsten Kreis. Differentially private diffusion models. Transactions on Machine Learning Research.
(9] Jinshuo Dong, Aaron Roth, and Weijie J Su. Gaussian differential privacy. arXiv preprint arXiv:1905.0233: 2019.
[10] Cynthia Dwork. Aaron Roth, et The algorithmic foundations of differential privacy. Founda- lions and Trends@ in Theoretical Computer Science, 0(3-4):111-407, 2014.
[11] Sahra Ghalebikesabi, Leonard Berrada, Sven Gowal, Ira Ktena, Robert Stanforth, Jamie Hayes, Soham De, Samuel L Smith, Olivia Wiles, and Borja Balle. Differentially private diffusion models generate useful synthetic images, 2023.
[12] Chen Gong, Kecen Li, Zinan Lin, and Tianhao Wang. Dpimagebench: unified benchmark for differentially private image synthesis. arXiv preprint arXiv:2503. 11f.:2503..4681, 2025.
[13] Google Research. An inference-only approach to differentially private Ilm data generation. Technical report, Google, 2024.
[14] Moritz Hardt, Katrina Ligett, and Frank McSherry. A simple and practical algorithm for differentially private data release. In Advances in Neural Information Processing Systems (NeurIPS), 2012.
[15) Jiyan He, Xuechen Li, Da Yu, Huishuai Zhang, Janardhan Kulkarni, Yin Tat Lee, Arturs Backurs, Nenghai Yu, and Jiang Bian. Exploring the limits of differentially private deep learning with group-wise clipping. arXiv preprint 2rXiv:2212.01539, 2022.
[16] Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. GANs Trained by Two Time-Scale Update Rule Converge to Local Nash Equilibrium. arXiv e-prints, art arXiv: 1 706.0000000 706000000. June 2017, doi: 10.48550/arXiv. 170................. 770600800.

---

[17] Jonathan Ho, Tim Salimans, Alexey Gritsenko, William Chan, Mohammad Norouzi, and David J Fleet. Video diffusion models. Advances in Neural Information Processing Systems, 35:8633-8644 8646, 2022.
[18] Aaron Hurst, Adam Lerer, Adam Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, et al. Gpt-4o system card. arXiv preprint arXiv:2410.21276 2024.
[19]
Imagen-Team-Google: :, Jason Baldridge, Jakob Bauer, Mukul Bhutani, Nicole Brichtova, Andrew Bunner, Lluis Castrejon, Kelvin Chan, Yichang Chen, Sander Dieleman, Yuging Du, Zach Eaton-Rosen, Hongliang Fei, Nando de Freitas, Yilin Gao, Evgeny Gladchenko, Sergio Gómez Colmenarejo, Mandy Guo, Alex Haig, Will Hawkins, Hexiang Hu, Huilian Huang, Tobenna Peter Igwe, Christos Kaplanis, Siavash Khodadadeh, Yelin Kim, Ksenia Konyushkova, Karol Langner, Eric Lau, Rory Lawton, Shixin Luo, Soña Mokrá, Henna Nandwani, Yasumasa Onoe, Aäron van den Oord, Zarana Parekh, Jordi Pose-Tuset, Hang Qi, Rui Qian, Deepak Ramachandran, Poorva Rane, Abdullah Rashwan, Ali Razavi, Robert Riachi, Hansa Srinivasan, Srivatsan Srinivasan, Robin Strudel, Benigno Uria. Oliver Wang, Su Wang. Austin Waters, Chris Wolff, Auriel Wright, Zhisheng Xiao, Hao Xiong, Keyang Xu, Mare van Zee. Junlin Zhang, Katie Zhang, Wenlei Zhou, Konrad Zolna, Ola Aboubakar, Canfer Akbulut, Oscar Akerlund, Isabela Albuquerque. Nina Anderson, Marco Andreetto, Lora Aroyo, Ben Bariach, David Barker, Sherry Ben, Dana Berman, Courtney Biles, Irina Blok, Pankil Botadra, Jenny Brennan, Karla Brown, John Buckley, Rudy Bunel, Elie Bursztein, Christina Butterfield, Ben Caine, Viral Carpenter, Norman Casagrande, Ming-Wei Chang. Solomon Chang, Shamik Chaudhuri, Tony Chen, John Choi, Dmitry Churbanau, Nathan Clement, Matan Cohen, Forrester Cole, Mikhail Dektiarev, Vincent Du, Praneet Dutta, Tom Eccles, Ndidi Elue, Ashley Feden, Shlomi Fruchter, Frankie Garcia, Roopal Garg. Weina Ge, Ahmed Ghazy, Bryant Gipson. Andrew Goodman, Dawid Górny, Sven Gowal, Khyatti Gupta, Yoni Halpern. Yena Han, Susan Hao, Jamie Hayes, Jonathan Heek, Amir Hertz, Ed Hirst, Emiel Hoogeboom, Tingbo Hou, Heidi Howard, Mohamed Ibrahim, Dirichi Ike-Njoku, Joana Iljazi, Vlad Ionescu, William Isaac, Reena Jana, Gemma Jennings, Donovon Jenson, Xuhui Jia, Kerry Jones, Xiaoen Ju, Ivana Kajic, Christos Kaplanis, Burcu Karagol Ayan, Jacob Kelly, Suraj Kothawade, Christina Kouridi, Ira Ktena, Jolanda Kumakaw, Dana Kurniawan, Dmitry Lagun, Lily Lavitas, Jason Lee, Tao Li, Marco Liang, Maggie Li-Calis, Yuchi Liu, Javier Lopez Alberca, Matthieu Kim Lorrain, Peggy Lu, Kristian Lum, Yukun Ma, Chase Malik, John Mellor, Thomas Mensink, Inbar Mosseri, Tom Murray, Aida Nematzadeh, Paul Nicholas, Signe Norly, João Gabriel Oliveira, Guillermo Ortiz-Jimenez, Michela Paganini, Tom Le Paine, Roni Paiss, Alicia Parrish, Anne Peckham. Vikas Peswani, Igor Petrovski, Tobias Pfaff, Alex Pirozhenko, Ryan Poplin, Utsav Prabhu, Yuan Qi, Matthew Rahtz, Cyrus Rashtchian, Charvi Rastogi. Amit Raul, Ali Razavi, Sylvestre-Alvise Rebuffi, Susanna Ricco, Felix Riedel, Dirk Robinson, Pankaj Rohatgi, Bill Rosgen, Sarah Rumbley, Moonkyung Ryu, Anthony Salgado, Tim Salimans, Sahil Singla, Florian Schroff, Candice Schumann, Tanmay Shah, Eleni Shaw, Gregory Shaw, Brendan Shillingford, Kaushik Shivakumar, Dennis Shtatnov, Zach Singer, and Evgeny Sluzhaev. Imagen 3. arXiv e-prints, art. arXiv:2408.0 070009, August 2024. doi: 10.48550/arXiv.2405 070009
[20] James Jordon, Jinsung Yoon, and Mihacla Van Der Schaar. Pate-gan: Generating synthetic data with differential privacy guarantees. In International conference on learning representations, 2018.
[21] Tero Karras, Timo Aila, Samuli Laine, and Jaakko Lehtinen. Progressive growing of gans for improved quality, stability, and variation. 2018
[22] Weiwei Kong and Umar Syed. A picture's 's worth thousand (private) words: Hierarchical generation of coherent synthetic photo albums, 2025. URL https: google/bl og/a-pictured worth-a thouse--- thousand-prrvaaee-eeeee f- eechherent-syathttic..
albums/.
[23] Alexey Kurakin, Natalia Ponomareva, Umar Syed, Liam MacDermed, and Andreas Terzis. Harnessing large-language models to generate private synthetic text. arXiv preprint 2xX1v.2306.01684, 2023.
12

---

[24] Cheng- Han Lee, Ziwei Liu, Lingyun Wu, and Ping Luo. Maskgan: Towards diverse and interactive facial image manipulation. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2020.
[25] Xuechen Li, Florian Tramer, Percy Liang, and Tatsunori Hashimoto. Large language models can be strong differentially private learners. International Conference on Learning Representations, 2022.
[26] Zinan Lin, Sivakanth Gopi, Janardhan Kulkarni, Harsha Nori, and Sergey Yekhanin. Differ- entially Private Synthetic Data via Foundation Model APIs 1: Images. arXiv e-prints, art. ucXiv:2305.15566 May 2023. doi: 10.48350/arXiv 230....50000...
[27] Zinan Lin, Sivakanth Gopi, Janardhan Kulkarni, Harsha Nori, and Sergey Yekhanin. Differen- tially private synthetic data via foundation model apis Images. International Conference on Learning Representations (ICLR), 2024.
[28) Zinan Lin, Tadas Baltrusaitis, and Sergey Yekhanin. Differentially private synthetic data via apis Using simulators instead of foundation model. arXiv preprint arXiv:2502.05500, 2025.
[29] Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, et al. Deepseek-v3 technical report. arXiv preprini arXiv:2412. 19437, 2024.
[30] Enshu Liu, Xuefei Ning, Yu Wang, and Zinan Lin. Distilled decoding 1: One-step sampling of image auto-regressive models with flow matching. arXiv preprint arXiv:2412.17153, 2024.
[31] Michael F Liu, Saiyue Lyu, Margarita Vinaroz, and Mijung Park. Differentially private latent diffusion models. Transactions on Machine Learning Research.
[32] Michael F. Liu, Saiyue Lyu, Margarita Vinaroz, and Mijung Park. DP-LDMs: Differentially Private Latent Diffusion Models. arXiv e-prints, art. arXiv:2305.1577 May 2023. doi: 10.445550/aXxi.2330.5 15759.
[33] Ziwei Liu, Ping Luo, Xiaogang Wang, and Xiaoou Tang. Deep learning face attributes in the wild. In Proceedings of International Conference Compuler Vision (ICCV), 2015.
[34] Nils Lukas, Ahmed Salem, Robert Sim, Shruti Tople, Lukas Wutschitz, and Santiago Zanella- Béguelin. Analyzing leakage of personally identifiable information in language models. In 2023 IEEE Symposium on Security and Privacy (SP), pages 346-363, IEBE Computer Society, 2023.
[35] Justus Mattern, Zhijing Jin, Benjamin Weggenmann, Bernhard Schoelkopf, and Mrinmaya Sachan. Differentially private language models for secure data sharing. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 4860-4873, 2022.
[36] OpenAl. ChatGPT. https://chat. com, 2022.
[37] OpenAl. Dall-e 3. htttps://openai..om////
2023. Accessed: 2025-06-09.
[38] OpenAl. Gpt-4o-mini model. https://openat: com, 2024. Accessed 2025-05-16.
[39] Gaurav Parmar, Richard Zhang, and Yun-Yan Zhu. On aliased resizing and surprising subtleties gan evaluation. In CVPR, 2022.
[40] Aditya Ramesh, Mikhail Pavlov, Gabriel Goh, Scott Gray, Chelsea Voss, Alec Radford, Mark Chen, and Ilya Sutskever. Zero-shot text-to-image generation. In International conference machine learning, pages 8821-8831. Pmlr, 2021.
[41] Artem Reshetnikov, Maria-Cristina Marinescu, and Joaquim More Lopez. DEArt: Dataset of European Art. arXiv e-prints, art. arXiv:2211.01223 November 2022. doi: 10.4 1.01226.
[42] Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High-resolution image synthesis with latent diffusion models, 2021.
13

---

[43] Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bjöm Ommer. High- resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684-10695, 2022.
[44] Axel Sauer, Dominik Lorenz, Andreas Blattmann, and Robin Rombach. Adversarial Diffusion Distillation. arXiv e-prints, art. arXiv:2311.10442 November 2023. doi: 10.48550/arXiv.3311 17042.
[45] Christoph Schuhmann, Richard Vencu, Romain Beaumont, Robert Kaczmarczyk, Clayton Mullis, Aarush Katta, Theo Coombes, Jenia Jitsev, and Aran Komatsuzaki. LAION-400M: Open Dataset of CLIP-Filtered 400 Million Image-Text Pairs. arXiv e-prints, art. arXiv:2111.02 November 2021. doi: 10.48550/axxvv2211 1.02114.
[46] Christian Szegedy, Vincent Vanhoucke, Sergey loffe, Jonathon Shlens, and Zbigniew Wojna. Re- thinking the Inception Architecture for Computer Vision. arXiv e-prints, art. December 2015. doi: 10.48550/ar2ivv15522.20 151.0o00....111.66....
[47] Gemini Team, Rohan Anil, Sebastian Borgeaud, Jean- Saan-Bapistte Alayrac. Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M Dai, Anja Hauth, Katie Millican, et al. Gemini: family of highly capable multimodal models. arXiv preprint arXiv:2312.11805 2023.
[48] Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaci, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint (3Vi::2307.02288 2023.
[49] Boxin Wang. Weixin Chen, Hengzhi Pei, Chulin Xie, Mintong Kang, Chenhui Zhang, Chejian Xu, Zidi Xiong, Ritik Dutta, Rylan Schaeffer, et al. Decodingtrust: A comprehensive assessment of trustworthiness in gpt models. Conference on Neural Information Processing Systems (NeurIPS), 2023.
[50] Shuaiqi Wang, Vikas Raunak, Arturs Backurs, Victor Reis, Pei Zhou, Sihao Chen, Longqi Yang, Zinan Lin, Sergey Yekhanin, and Giulia Fanti. Struct-bench: A benchmark for differentially private structured text generation. arXiv preprint arXiv:2509. 10696, 2025.
[51] Weihao Xia, Yujiu Yang, Jing-Hao Xue, and Baoyuan Wu. Tedigan: Text-guided diverse face image generation and manipulation. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2021.
[52] Weihao Xia, Yujiu Yang, Jing- Hao Xue, and Baoyuan Wu. Towards open-world text-guided face image generation and manipulation. arxiv preprint arxiv: 704.08910, 2021.
[53] Chulin Xie, Zinan Lin, Arturs Backurs, Sivakanth Gopi, Da Yu, Huseyin A Inan, Harsha Nori, Haotian Jiang, Huishuai Zhang, Yin Tat Lee, al. Differentially private synthetic data via foundation model apis 2: Text. In International Conference on Machine Learning, pages 54531-54560. PMLR, 2024.
[54] Liyang Xie, Kaixiang Lin, Shu Wang. Fei Wang, and Jiayu Zhou. Differentially private generative adversarial network. arXiv preprint arXiv:1802.06739, 2018
[55] Chejian Xu, Jiawei Zhang, Zhaorun Chen, Chulin Xie, Mintong Kang, Yujin Potter, Zhun Wang. Zhuowen Yuan, Alexander Xiong, Zidi Xiong, et al. Mmdt: Decoding the trustworthiness and safety of multimodal foundation models. arXiv preprint 250.::20003..82.. 2025.
[56] An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, et al. Qwen2. 5 technical report. arXiv preprint 2rXiv:2412.15111 2024.
[57] Da Yu, Saurabh Naik, Arturs Backurs, Sivakanth Gopi, Huseyin A Inan, Gautam Kamath, Janardhan Kulkarni, Yin Tat Lee, Andre Manoel, Lukas Wutschitz, et al. Differentially private fine-tuning of language models. International Conference on Learning Representations, 2022.
[58] Fisher Yu, Yinda Zhang, Shuran Song, Ari Seff, and Jianxiong Xiao. Lsun: Construction of a large-scale image dataset using deep learning with humans in the loop. arXiv preprint 12Xiv:556600366 2015.
14

---

[59] Xiang Yue, Huseyin Inan, Xuechen Li, Girish Kumar, Julia McAnallen, Hoda Shajari, Huan Sun, David Levitan, and Robert Sim. Synthetic text generation with differential privacy: A simple and practical recipe. In Proceedings of the 61 Annual Meeting of the Association for Computational Linguistics (Volume I; Long Papers), pages 1321-1342, 2023.
[60] Jun Zhang, Graham Cormode, Cecilia M. Procopiue, Divesh Srivastava, and Xiaokui Xiao. Privbayes: Private data release via bayesian networks. ACM Trans. Database Syst., 42(4), October 2017. ISSN 0362-5915.
[61] Longtao Zheng, Zhiyuan Huang, Zhenghai Xue, Xinrun Wang, Bo An, and Shuicheng Yan. AgentStudio: A Toolkit for Building General Virtual Agents. arXiv e-prints, art. arXiv:2403. 17918, March 2024. doi: 10.48550/ar 240.. 17918.
15

---

NeurIPS Paper Checklist
1. Claims
Question:
Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
The abstract in paper clearly states that our method can improve the quality image generation, and the results are listed the experiment part.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification: Our paper mentioned our the limitations in the last section.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g.. independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
16

---

Our work has to prove our method follows DP constraints, this theoretical result is provided with complete and detailed proof in method section.
Justification:
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: We represented our idea method section, and specified our experiment setup that anyone who read our paper can reproduce our results.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, it may be necessary either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, or other means that appropriate to the research performed.
While NeurlPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
17

---

Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Question:
Answer: [Yes]
Justification: We provided our code in the supplemental material, and we'll release more formal version on Github soon.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please the NeurIPS code and data submission guidelines (https://nips)
biic//guiddssCoddSiibiiiiee Ctt////ooeSSSiiisseeeeeeee ottp://oooooosiiiiiooooooooo//// onPolicy) for more details.
While encourage the release of code and data, we understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https: / //npss wwcppblic//eeccccccccccccccco wwtpc///g///aeieeioeooeeeees https//ooSSSuuiiseseeee
for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: The appendices in our paper specified most of our non-default hyper-parameters in our experiment.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented the core the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [No]
Justification: Our main contributions and experiment results don't 't need information about statistical significance illustrate.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by emor bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
18

---

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: Our paper provided a part about compute resources in experiment section for reproducibility.
Guidelines:
The answer NA means that the paper does include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental rums as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't itdn't make it into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics attps://neure wwtpuriis.ccpppplpee cc/public//nieeccccccccccce Ceabblcc/Ehicaee (ethicsuuiidllleeeee
Answer: [Yes]
Justification: We're sure that our work conform with the NeurlPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g.. there a special consid- eration due to laws regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: Our paper talked about potential positive socictal impacts of our work. Our work provides a method for high quality private image generation, which better preserves privacy for private data source.
Guidelines:
19

---

The answer NA means that there is no societal impact of the work performed.
If the authors answer NA No, they should explain why their work has societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (c.g., gated release models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility ML).
11. Safeguards
Question:
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Answer: [NA]
Justification: Our paper releases no data or models that have such risks for misuse.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: All creators original owners of assets used in paper are properly credited. And licenses and terms of use are explicitly mentioned and properly respected.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
20

---

For scraped data from a particular source (e.g., website). the copyright and terms of service of that souree should be provided.
assets are released, the license, copyright information, and terms of use the package should be provided. For popular datasets, paperswi thcode. com/ aan/aataets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's s creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification:
No new assets are introduced in our paper.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/mmodll as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification: Our paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification:
Our paper does not involve crowdsoureing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
21

---

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if' applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, originality of the research, declaration is not required.
Question:
Answer: [Yes]
The paper described lot about the usage of LLMs since L.MMs take a important role in our methodology.
Justification:
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non-standard components.
Please refer to our LLM policy (https: httppp///eerippssec///eeeeeeeeee ccc/ooneeeeecccc/22222.. /L02//LLM) for what should or should not be described.
22

---

A Details on our Jag_PP__Iage__ooting algorithm
We present the pseudo-code for the DE__P___ Irt__mmae___oo generated private text T is unused in Algorithm 2, this is by design. Image voting serves solely as quality enhancement technique within our pipeline. In contrast, the original text voting mechanism explicitly requires the generated private text. To maintain the completeness and integrity of the overall pipeline, we retain the captioning process.
algorithm. While it may appear that the
Algorithm 2 Aug-PE PE with Image Voting Thug__EE_mnag__oooinng
Require: Private image dataset generated private text T, population size N, number of iterations diffusion model API Diffusion(-). image encoder EncodeImage( :), DP noise multiplier o,K,
G. RANDOM_API((), TARIATION_APIE
Ensure: Differentially private synthetic text data T
1: Cn = RANDOM (P
2: Epriv EmcodeIImage())
3: for =
do then
4: if' K
5: Ig = Diffusion(C,)
6: gen Encode (Ty)
7:
else if K > 0 then
8:
C VARIATION_APIES (AI 1, 1,2, ..., K
9:
I Diffusion (*) 1,2, .
10:
ck gen
Encode Image (Th) [1 1,2,.... K
11:
Egen K Σ gk
gen
12:
end if
ON = [0, (0..,... 0]
14:
for each Epriv E Epriv do
15:
it arg min, d( d(Epries Egen
16:
H[1 H1]
17:
end for
18:
H H + NN(00 II)
19:
P+ H
20:
if = then
21:
C. + draw N samples with replacement from C
22:
Cyt1 + VARIATION_APIICA
23:
save 6oyn C g+1
24:
else if > 1 then
25:
C + rank samples by probabilities P and draw top N samples
26:
C 9+1 VARIATION_API((
1,2,...,
27:
Cy+1= = [C", C1 g+1 C1-11
28:
save 2aym
g+1 < Cg-1
end if
30:
end for
31:
return Final text candidates Ca as synthetic data T
B More on our experiment results
Here we post more results of our experiments, and provide more experiment details.
B.1 FID comparison between PE and SPTI
Here we present details of our experiment results. We use FID [16] to compare between SPTI and PE on a variaty of datasets. We give specific dataset name and DP constraint in the caption of each image. We find SP77 generally better than PE, and voted samples is generally better than variants in most iterations. Interestingly, voted samples is often slightly worse than variants in the inital iteration.
23

---

FID over Iterations
FID over Iterations
mathollycted] Our mecholloarieres)
muthod(vatee) Our method(variants) PC
56
PC
45
40
30
25
Iteration
Iteration
Figure 8: Experiment results on LSUN Bedroom
Figure 9: Experiment results LSUN Bedroom
dataset. (c 1.0)
dataset. (c 10.0)
FID Iterations
FID Iterations
mechod/ycsod) Our medhod(varaaes)
Our method(voted]
100
Our methodvvariamss)
160
180
PE
140
1.00
00
Iteration
tteraion
Figure 10:
Experiment results on European Art
Figure 11:
Experiment results on European Art
dataset. (c- = 1.0)
dataset. (c 10.0)
FID over Iterations
FID over Iterations
Our mechodlvoted) Our methodquaranes
Our method(voted)
Our metheduarramts)
250
200
150
150
50
Iteration
Iteration
Figure 12:
Experiment results on Wave-ui-25k
Figure 13: Experiment results on Wave-ui-25k
dataset. (c = 1.0)
dataset. (c = 10.0)
24

---

FID over Iterations
FID over Iterationn
190
mathodivated)
methodyvtte)
130
100
100
60
00
Iteration
Iteration
Figure 14: Experiment results Cat dataset.
Figure 15: Experiment results on Cat dataset. (c 10.0)
1.0)
FID Iterations
FID Iterations
mechod|yoood) Our medhod(varaaes)
method(voted) Our method(variamss)
260
PE
230
0
220
200
160
Iteration
tteraion
Figure/ 16:
Experiment results on Sprite Fright
Figure 17: Experiment results on Sprite Fright
dataset. (c = 1.0)
dataset. (c 10.0)
B.2 FID comparison between Image Voting and Text Voting
Here we present details of our experiment results. We use FID [16] to compare between image voting and text voting variaty of datasets. We give specific dataset name and DP constraint in the caption each image. We find image voting is generally better than text voting under most conditions. Interestingly. image voting and text voting show the same behavior in the inital iteration, ie. voted samples is slightly worse than variants in the first iteration, no matter image voting or text voting is used.
FID over Iterations
FID over Iterations
49
Image Vodngivoted)
42.8
Image voligtvarlanes)
42
40.0
Vosing(vooed)
Voting (ona)
22.6
32.6
Image Vosgotvored)
Image Vortgnattinn
364
Text vodinglvoted vodinglvariane)
215
Iteration
Iteration
Figure 18: Experiment results on LSUN Bedroom
Figure 19: Experiment results on LSUN Bedroom
dataset. (e = 1.0)
dataset. (E = 10.0)
25

---

FID over Iterations
FID over Iterationn
Image Volinglvoted]
180
Image Volingloeriares)
Text vorinn(vvote)
110
Text Veninn(variantt)
100
Volinglvonal votinglvariants)
Voring(vvoedd)
Twxt Voring[variamts)
Iteration
Iteration
Figure 20:
Experiment results Wave-ui-25k
Figure 21:
Experiment results Wave-ui-25k
dataset. (c 1.0)
dataset. (c 10.0)
FID Iterations
FID Iterations
Image Vodig[vosod)
Image Veting(voted]
00
Image Vodig]saaranes)
Image Voting(variants)
Text Votiingvooed)
Text Velingtvotod)
65
Veting(variamms
Volingtvariants)
75
70
Iteration
Iteration
Figure 22: Experiment results on European Art
Figure 23: Experiment results on European Art dataset. (c 10.0)
dataset. (c = 1.0)
FID over Iterations
FID over Iterations
Image Voting/voted)
Image Votinggvotad)
Text Voting(voted)
Text Voting(votei)
116
112.6
0
114
112
107.5
106.0
110
108
Iteration
Iteration
Figure 24: Experiment results on Cat dataset. (E
Figure 25: Experiment results on Cat dataset. (E 10.0)
1.0)
B.3 Downstream Task
Despite the high quality of generated images in our method, we also consolidate their value in usage for downstream tasks. To more specific, we test classification accuracy on CelebA dataset using WRN-40-4 model. Here provide figures of model performance during training.
26

---

(Sampie 2004)
15
-2
15%
Figure 26: Training performance (test accuracy) on CelebA dataset with the same number of generated samples. 10.0, sample, size=2000)
Figure 27: Training performance (train accuracy) on CelebA dataset with the same number of generated samples. (e 10.0, sample size=2000)
MMM
300
GROUROTRUTS
300
275
200
150
125
202
Figure 28: Training performance (test accuracy) CelebA dataset with the same number of generated samples. 10.0, size=4000)
Figure 29: Training performance (train accuracy) on CelebA dataset with the same number of generated samples. (e 10.0, sample sample_eize=400 size=4000)
MMp Mm
-2
70
35
65
275
150
125
Figure 30: Training performance (test accuracy) CelebA dataset with the same number of generated samples, 10.0, sample_siae s1000-6000))
Figure 31: Training performance (train accuracy) on CelebA dataset with the same number of generated samples. (e 10.0, sample_sized si0e00000))))000
27

---

(Sample 2214)
///
Figure 32: Training performance (test accuracy) on CelebA dataset with the same number of generated samples. 10.0, sample, size=8000)
Figure 33: Training performance (train accuracy) on CelebA dataset with the same number of generated samples. (e 10.0, sample size=8000)
300
md
GROUROTRUTS
300
200
150
125
202
Figure 34: Training performance (test accuracy) CelebA dataset with the same number of generated samples. 10.0, sample_size s00e00000))))000
Figure 35: Training performance (train accuracy) on CelebA dataset with the same number of generated samples. (e 10.0, sample_size=100 size=10000)
Figure 36: Generated samples by SPTI (e 10.0, sample sample_size-10n size=10000)
Figure 37: Generated samples by PE image (e 10.0. sample_sise size=10000)
28

---

C Extended Ablation Study
C.1 Large Language Model and Diffusion Model APIs
Along with the effect of our image voting trick, also tested our method with different LLM and diffusion model APIs along with other hyperparameters. For large language models, we choose qwen- original
plus [3] as comparison APIS, we choose stableddifussoonxllbaaee
[2] model. For diffusion model [44] to compare with our original model. We also present Figure 38, 39, 40, 41 generated from DP synthetic data by Imagen3 [19] and DALLE3 [37], i.e. we acquire DP synthetic images by running SPT/ using -aae--laaa------sttrrre
[2] and stable- ceffusion-xl-base--e [44], but generate final DP synthetic images using different mage-generaaion
API.
SDXL-Turbo
SDXL-base-1.0
Infinity
Metts-.lanaa3388--nnstruu
26.71
25.42
30.66
qwen-plus
26.65
24.44
31.28
Table 4: FID (lower is better) on LSUN Bedroom with e 1.0, tested using different LLM and diffusion APIs. All other settings are kept identical to the default configuration. The results indicate that while different API backends (e.g., SDXL-Turbo, SDXL-base-1 .,, Infinity) and language models (e.g., Leta-Llama- -etaa---BB--ma IBB-Inttuct, Qwen-Plus) introduce minor variations in FID, their overall influence on the final image quality is limited. This demonstrates the robustness of our method to the choice of model and API implementation.
Figure 38: Images generated by Imagen3 [19] from DP-synthetic text of dataset LSUN Bedroom 1.0). FID (lower better) 26.58, which is slightly better than using SDXL-Tuurb 26.71.
Figure 39: Images generated by Imagen3 [19]
from DP-synthetic text dataset Wave-ui-25k
(e = 1.0). FID (lower better) is 72.13, which worse than using SDXL-Turhe 36.52.
29

---

Figure 40: Images generated by DALLE3 [37] from Dy-synchetic text dataset LSUN Bedroom (c = 1.0). FID (lower better) is 29.96, which is slightly worse than using SDXL-Turbo 26.71
Figure 41: Images generated by DALLE3 [37] from DP-synthetic text dataset Wave-ui-25k (c 1.0). FID (lower is better) is 150.68, which is worse than SDXL-Turbo 36.52
D Explanation of Special Conditions
In the previous experiment on Cat dataset, we find our method to be less effective compared to baseline method (PE). So we conducted contrast experiment to analyze the cause behind the phenomenon.
Firstly, build small sub-dataset from ImageNet100 dataset called bird dataset. This dataset consists of 100 images labeled goldfinch and 100 images labeled indigo bird, which corresponds to label id 70 and in the ImageNet100 dataset. We found similar results Cat dataset, so we take bird dataset as an alternate baseline.
Secondly, we proposed two possible reasons for the phenomenon that SPT/ performs poorer than PE on Cat and bird dataset. One is that the composition dataset not diverse, while SPT/ uses diversity than PE image method, causing worse results. The other is that the SPT/ performs naturally poorer than PE when the total number of dataset small. We build another 3 datasets 5 according these reasons.
Aug-PE- to generate DP data, which brings
Here we reference our newly-built dataset using their composition the following table. We only use images from ImageNet100 when building all the four dataset. The experiment settings follows the experiments on Cat dataset Table 6. The experiment results in Table 5 shows that the reason why SPTI performs poorer than PE Cat and bird dataset is because diversity. SPTI performs naturally poorer on dataset that less diverse. This provides us with insight that text modality may bring more diversity on DP data generation than using image.
E Details of Experimental Configurations
To facilitate reproducibility, we detail the hyperparameter settings used for all experiments in Section For hyperparameters mentioned, the default values used. For experiment settings of ablation studies between image voting and text voting, we use the same hyperparameters in Table 6 below. For comparison experiments on SPTI, PE and DP fine-tuning, we present our configuration details in Table 7. For specific configuration of DP fine-tuning we present our settings in Table 8.
F Datasets
In this section, we summarize datasets used our experiments with their stats, i.e. total number of images, resolution, etc. We represent all information in Table 9. Here we record the number of images and resolution in our experiments.
30

---

Table 5: FID (lower is better) comparison of SPTI and PE on Different Datasets
Dataset Composition
Total Number of Images
SPTI (ours)
PE
goldfinch*100 100
+ indigo 10rd*100
200
144.18
110.96
20 kinds of different birds,
200
111.84
168.39
each kind 10 images
goldfinch 1000
2000
106.26
78.66
indigo bird 1000
20 kinds of different birds,
2000
84.25
172.93
each kind 100 images
31

---

Dataset
Configurations
SPT](ours)
PE image
wum_samples_schedule
2000
2000
PE.
iterations
10
10
ImageVotingNI ()
Rookahead_degree
0
LSUN bedroom
NNhistogram ()
lookahead__egrre
initial_rariatiioo_pp.fff
6
0
PEPopulation
next_varriatioon.pii.fou
6
1
sct_smmples_scchduleee
200
200
PE.run ()
iterations
10
10
ImageVoting!N))
dookahead_degree
4
Cat
NearestNeighbors ()
lookahead_degree
inttial_.variatioopppp...t
0
0
PEPopulation
next_variattioo_pp__foll
1
1
smm_smpplesscchdulee
2000
2000
rE.run( ()
iterations
10
10
ImageVotinglI))
bibtaheadddegreee
0
Wave-ui-25k
NNhistogram
lookahead_degre
4
6
0
PEPopulation ()
next_variiaiionnapi_.fo
6
1
num_samples__schctule
2000
2000
PE. ()
iterations
10
10
ImageVotingNN)) NNhistogram()
lookahead_degree
0
Europeart
Lookahead_degree
initial_.vrrittionn.p...ff
6
0
PEPopulation ()
next nett_wariatiion.ppi.foll.
6
wum_samplee_sshedule
1000
1000
rE. ()
iterations
10
10
ImageVotingNIN NNhistogram ()
lookahead_degre
0
Sprite Fright
lookaheaa_deerre
4
initial_variaation..sp..ffl
6
0
PEPopulation()
next_variation..pi.fol.
6
Table 6: Experiment Settings for FID evaluation.
Configurations
SPTI(ours)
PE image
gam__samplessscheduu
2000
2000
PE.run
iterations
10
10
ImageVotingUI ()
1ookahead_degree
WWhistogram()
lookahead_degres
inirial.variation.app..ee
6
0
PEPopulation()
mext_warriation.apiifff/
6
Table 7: Experiment Settings of SPT7 and PE for FID evaluation of MMM-eleba--HQ dataset
32

---

Table 8: Hyperparameters for fine-tuning diffusion models with DP constraints e 10, 1 and
ô 10 text-conditioned CelebAHQ.
6 10
6=
batch size base learning rate learning
256
256
10-7
1 X 10-7
2.6
10-5
2.6 10-5
epochs
10
10
clipping norm
0.01
0.01
noise scale ablation
0.55
1.46
-1
num of params
280M
280M
use_ use_spatial_ transformer
True caption
True
cond_stage sand_stage_key context dim
caption
1280
1280
conditioning key transformer depth
crossattn
crossattn
1
Dataset
Total number of images
Resolution
Source
LSUN
300,000
256 x 256
atsps://ggithub.comm//////
Cat
200
512 x 512
httpp://ww..aggll..ccm//aaaasss
Len/cat-coozzee-doudou
Wave-ui-25k
24,798
256 x 256
httpa///uggingfaccc.ooo/aateeee tsea/vave-uii25k
European Art
15,154
256 256
ctt.s//cccc.co/atatttts////ggggg am/eurooeaan_art
https:/hugginggface..
Sprite Fright
13,077
256 x 256
httpa://studio.blender..oot orgpbbbbbbeeeerrrrrrr/oooeetttt sprite-fright/
MM-Celeba-HQ
30,000
256 x 256
http:///gthubb.omm/IIGU//////
A-HQ-Dataset
CelebA
202,599
178 x 178
https://mmlab. ie. cuhk. edu. ht/projects/ CelebA. html
Table Statistics datasets used in our experiments.
33


Reviews:
[{"review_id": "NLonFNqaM2", "reviewer": "Reviewer_6vVq", "summary": "This paper proposes synthesis via private textual intermediaries (SPTI) that is a framework for generating high-resolution differentially private synthetic images. The main idea is to shift the privacy-preserving synthesis process from the image domain to the text domain. The process begins by transforming private images into textual descriptions through an image captioning model. It then utilizes a modified Private Evolution algorithm to produce differential privacy (DP) text, followed by reconstructing images using a text-to-image diffusion model. Experiments on multiple datasets show that SPTI achieves better image quality (in terms of FID) than Private Evolution and DP fine-tuning.", "questions": "1) How sensitive is the overall performance to the choice of the image-to-text component? For example, if the captioning model were replaced with another model, such as LLaVA, how would this impact the image quality (FID)? Is there any quantitative analysis or ablation regarding this choice?\n\n2) For datasets with image tags or class labels, what would happen if the text input were simply 11an image of [tag]'' or a similar template for each image? Would this significantly affect performance or privacy guarantees?\n\n3) Are there any alternative evaluation methods, beyond FID scores, to assess the distribution alignment between the synthetic and private data? For example, can we use techniques like t-SNE or UMAP to visualize the embedding spaces of private and synthetic images in order to assess whether their distributions are well aligned? While the FID scores show improvements, it remains unclear whether the synthetic data truly captures the diversity and structure of the private data distribution.\n\n4) What is the estimated budget, in terms of computation and cost, for processing a datasets used in the paper, specifically for the image-to-text conversion using APIs like GPT or other captioning models?", "limitations": "No. The paper recognizes the significant computational costs and API token usage but fails to provide specific figures or estimates. Without this information, it is challenging to evaluate the method's practical expense and determine if the cost is a reasonable trade-off compared to other DP approaches.", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "**Strengths**\n1) The paper introduces a new framework, Synthesis via Private Textual Intermediaries (SPTI), which transitions the challenge of differentially private image synthesis from images to text. \n2) The method uses API-based models, including image captioning and text-to-image generation, without requiring model fine-tuning. \n3) The pipeline is modular and does not require DP fine-tuning of large image generation models, which reduces technical complexity compared to DP-SGD-based approaches, such as DP Fine-tuning.\n4) Experimental results show improvements over Private Evolution and DP fine-tuning, across multiple datasets and privacy budgets.\n\n**Weaknesses**\n1) The method has several modules: image-to-text, DP text evolution, and text-to-image synthesis. The interdependencies among these components make it challenging to assess their impact on overall performance.  In Section 4.3, the ablation study shows that the method is sensitive to changes in the image generation models. \n\n2) Using image-to-text as an intermediate representation has been explored in some existing works [1]. Although the application scenarios differ, the underlying idea is similar.\n \n      [1] Image2text2image: A novel framework for label-free evaluation of image-to-text generation with text-to-image diffusion models. \n\n3) It is unclear whether text-based representations are always sufficient to preserve fine-grained image details. \n\n4) The baseline comparisons are incomplete. While the primary comparisons focus on Private Evolution and DP fine-tuning, there is no evaluation against simpler baselines.\n   * Sampling random text prompts from the LLM API (without using any private data) to generate images, or generating images solely based on class names, raises uncertainty about whether this basic approach can achieve competitive FID scores.\n\n5) The DP guarantee is limited to the Private Evolution voting step, while other processes, particularly image-to-text captioning, access private data without DP protection. Although the paper cites the post-processing property of DP as justification, this design choice warrants further examination.\n\n6) Computational costs are not fully discussed. While the method avoids DP training, it requires repeated calls to large captioning models, LLMs, and text-to-image diffusion models. The paper reports that experiments were run on high-end GPUs, but no systematic analysis of compute costs compared to prior methods is provided. There are also additional costs for calling APIs such as ChatGPT.\n\n7) The paper does not fully analyze failure modes or limitations in the semantic fidelity of the generated images, especially when the text descriptions do not align well with the images in complex scene data.", "was_revised": true, "final_justification": "During the rebuttal, the authors addressed my concerns about the paper, which led me to raise my final rating.", "rebuttal": "# To Reviewer\nWe appreciate the reviewer’s insightful feedback. Below, we provide detailed responses to each of the raised points.\n\n## Reply to Weaknesses\n### Weakness 1\n**W1:** The interdependencies among these components make it challenging to assess their impact on overall performance. In Section 4.3, the ablation study shows that the method is sensitive to changes in the image generation models.\n\n**WA1:** This statement is inaccurate. In addition to analyzing the image generation models, we also conducted a sensitivity analysis on the image captioning component. Overall, the proposed method is not particularly sensitive to the choice of these base models, provided that each performs its respective task effectively.\n\n### Weakness 2\n\n**W2:** Using image-to-text as an intermediate representation has been explored in some existing works [1]. Although the application scenarios differ, the underlying idea is similar.\n\n\n**WA2:** Thank you for referring to paper [1], which proposes a label-free evaluation framework for image captioning models. The core idea is that a well-written caption should enable a text-to-image diffusion model to reconstruct an image visually similar to the original.\n\nWhile the use of an image-to-text-to-image pipeline is conceptually related and increasingly common, its application to the task of private dataset synthesis is nontrivial and remains unclear. Our work differs substantially in terms of motivation, privacy guarantees, and evaluation methodology:\n\nObjective: Paper [1] aims to assess the quality of image captioning models. In contrast, our work is centered on synthesizing datasets that preserve data utility while offering rigorous differential privacy (DP) guarantees.\n\nEvaluation Metric: The prior work evaluates the similarity between individual original and reconstructed images. We, however, focus on measuring the similarity between the distributions of the original private dataset and the generated dataset.\n\nWe will incorporate this paper, along with other relevant literature, into the related work section and clearly articulate how our approach differs.\n\n### Weakness 3\n\n**W3:** It is unclear whether text-based representations are always sufficient to preserve fine-grained image details.\n\n**WA3:** We acknowledge that, for static evaluation, we cannot provide a definitive answer to whether text-based representations are always sufficient to preserve fine-grained image details. However, our approach demonstrates a clear improvement in preserving such details compared to existing methods. Furthermore, from a longer-term perspective, we believe this challenge will become increasingly addressable as foundation models continue to evolve. Our method is designed to seamlessly benefit from these advancements, allowing it to leverage future improvements in underlying models with minimal effort.\n\n### Weakness 4\n\n**W4:** The baseline comparisons are incomplete. \n\n**WA4:** Thanks for the suggestion, we add the baseline as follows, and provide comparison between SPTI, PE Image and DP-finetune below. We fix the experiment settings as the same, with $\\epsilon=1.0$, `num_samples=2000`. It's noticeable that we updated chat template for LLM, so we re-run all the previous experiments. Still, new results show that our method performs better than PE image and DP-finetune.\n\nAlso, we will run more experiments with different settings to fully establish the excellence in our method.\n\n||SPTI|PE Image| DP-finetune|\n|:-:|:-:|:-:|:-:|\n|LSUN Bedroom| **26.39** | 40.36 | 31.76 |\n|Wave-ui-25k| **37.39** | 50.45 | 62.08 |\n|European Art| **57.64** | 76.50 | 63.97 |\n|MM-Celeba-HQ| **34.17** | 50.45 | 57.01 |\n\n### Weakness 5\n\n**W5:**  The DP guarantee is limited to the Private Evolution voting step, while other processes, particularly image-to-text captioning, access private data without DP protection. Although the paper cites the post-processing property of DP as justification, this design choice warrants further examination.\n\n**WA5:** PE process refers to the private data, and synthesize data that satisfies DP constraints. In our pipeline, we refers to captions, and generates image data that satisfies DP constraints, so it's fine for image-to-text captioning to access private data. \n\n### Weakness 6\n\n**W6:** Computational costs are not fully discussed.\n**WA6:** We provide details of computational costs below in Question 4.\n\n\n### Weakness 7\n\n**W7:** The paper does not fully analyze failure modes or limitations in the semantic fidelity of the generated images, especially when the text descriptions do not align well with the images in complex scene data.\n\n**WA7:** We test our method on various datasets to confirm that our pipeline works well in different circumstances, and experiments did show good results. However, we admit that there are cases where text description doesn't align well with the images. This usually happens when images have such intensive information that VLMs can hardly capture all of them, or limits on text length prohibits VLMs to fully describe all the scenes. Nevertheless, we believe the performance of our method will improve as state-of-the-art VLMs and diffusion models reach better results.\n\n\n## Reply to Questions\n### Question 1\nWhen using **text-voting** in our pipeline, the quality of generated caption **determines the upper bound** of DP generated data, since text samples are iteratively getting closer to private caption in text-voting. However, when using **image-voting** in our pipeline, text samples are directly referring to representation of private images, which actually doesn't need caption text for reference, so the overall performance is **not affected** by caption models. \n\nNotably, we uploaded a new chat template for the LLM we use and run experiments after paper submission, and found image-voting still performs better than text-voting even in Cat dataset, which argues that **image-voting performs generally better** than text-voting.\n\nHowever, as text-voting may be necessary in some condition, we run systematic experiments with text-voting with fixed settings. The following table shows the results. We will provide more comparison in our newest version of paper.\n\n| caption model  | LSUN Bedroom|\n|:----------:|:---------------:|\n|  gpt-4o-mini |  41.17   |\n| qwen-vl-max |   41.42  |\n| gemini-2.0-flash |  42.48 |\n\n\n### Question 2\nIt would work fine. Actually, many of the experiments conducted are using prompts like 'An image of ...', and received good results.\n\nAs for privacy concerns, we assert that all the text prompts used in our experiments can be regarded as **public information**, which doesn't violate privacy. For example, if we consider LSUN Bedroom is a private dataset collected from sensitive user data, then 'this dataset is about bedroom' should be considered as public information, since the real sensitive part is the detail.\n\n### Question 3\nThanks for pointing out the deficiency in our work. We'll use **UMAP** for visualizing our results, and **CMMD** as an alternative metric other than **FID**. **CMMD** also show results better than **PE Image** method.\n\n**UMAP** shows our results align well with private data, and the visualization is uploaded in our newest version of paper. However, we can only provide our results in text, so only more results in **CMMD** is provided. In this table, we found our method performs the best except for the case in LSUN Bedroom. We suppose it's because LSUN Bedroom has such a large quantity of image data that improves the performance of fine-tuning, but this needs further experiments. Other conditions show the excellence of our method.\n\n| Dataset | SPTI | PE | DP-finetune |\n|:---:|:---:|:---:|:---:|\n| LSUN Bedroom | 1.744 | 2.066 | **1.376**  |\n| Wave-ui | **1.480** | 3.132 | 2.149 |\n| European_art | **0.684** | 1.016 | 2.447 |\n\n### Question 4\nOur full data generation pipeline involves two major components: an 8B-parameter large language model (LLM) and a 3.5B-parameter diffusion model. Under our current implementation, a complete data generation run takes approximately 19 hours on **a single NVIDIA A800**, or 15.5 hours on **2×NVIDIA A100 GPUs**, with a peak memory requirement of about 70 GB.\n\nIn comparison, the baseline method uses only a single 3.5B model, requires no LLM, and completes in about 7 hours on a **single NVIDIA A100 GPU**, with a peak memory requirement of about 35 GB. While our method has higher compute demands, it introduces a semantically enriched generation process through language-guided synthesis, which we show leads to superior data diversity and downstream performance.\n\nWe believe the additional cost is justified by the significant gains in quality and generalization, and we note that our pipeline can be modularly optimized (e.g., via model distillation or prompt caching) in future work to reduce the runtime.\n\nAs for API usage, caption process will take 250M tokens for 10,000 images. During data generation, LLM will process 7.3M tokens during each iteration, and diffusion model will process 5M tokens in each iteration. A total of 10 iterations will count for 123M tokens."}, {"review_id": "E69aAwlxzl", "reviewer": "Reviewer_Deqn", "summary": "Private Evolution is a promising approach to generating synthetic image and text data under differential privacy. This paper proposes a method, SPTI, which combines image and text private evolution methods. This method 1. maps private images to text using an off-the-shelf mutli-modal model, 2. uses an novel version of private evolution to obtain synthetic text, then 3. maps the synthetic text to images. The primary technical contribution is a version of private evolution that uses image voting, where both text and images are utilized to successively generate synthetic samples.", "questions": "1. In Table 1, where are the results for DP-finetune?\n2. Any thoughts on why Private Evolution works better on the Cat dataset than the proposed method? \n\nAdditional Comments\n- Rather than stating privacy results in terms of Approximate-DP, I suggest stating results in terms of RDP and provide the conversion from RDP to Approximate-DP. Moreover, it would be helpful to have a formal statement of the privacy result and a proof in either the main text or an appendix.", "limitations": "yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strengths\n- Interesting idea combining text and image approaches to private evolution. \n- Figure 1 is clear and helpful for understanding the proposed method. \n- The proposed method outperforms baseline methods with respect to FID score, and the image voting approach to private evolution outperforms using text alone. \n\nWeaknesses\n- There is not a formal proof in the main text or appendix that the proposed mechanism satisfies differential privacy. Moreover, the privacy analysis provided is insufficient. Most results such as the post-processing property are not clearly stated and are missing proper citations. Though stated on Line 188, the Advanced Composition Theorem (for Approx. DP) is not used for privacy accounting since the authors state that the privacy analysis uses RDP rather than Approx. DP.  \n- In the limitations section, the authors mention that the proposed method requires significantly more resources than baseline Private Evolution since the former requires several calls to muti-modal models; yet, there is no comparison of running time or token usage between the models.\n- There are several grammar mistakes throughout the text.", "was_revised": true, "final_justification": "The reviewers addressed my concerns about the privacy analysis in the their rebuttal.", "rebuttal": "# To Reviewer\n\nWe appreciate the reviewer’s insightful feedback. Below, we provide detailed responses to each of the raised points.\n\n## Reply to Weaknesses\n### Weakness 1\n**w1**: There is not a formal proof in the main text or appendix that the proposed mechanism satisfies differential privacy. Moreover, the privacy analysis provided is insufficient. Most results such as the post-processing property are not clearly stated and are missing proper citations. Though stated on Line 188, the Advanced Composition Theorem (for Approx. DP) is not used for privacy accounting since the authors state that the privacy analysis uses RDP rather than Approx. DP.\n\n**WA1**: We provide a detailed privacy analysis below. This analysis provides a tighter bound for DP guarantee, and is updated to the latest version of our paper.\n\n#### Privacy Analysis\n\nWe analyze the privacy guarantees of our proposed *SPTI* framework, which synthesizes image data under differential privacy (DP) by operating in the text space. Our goal is to ensure that the final synthetic images $\\mathcal{D}'$ are generated through a process that satisfies $(\\varepsilon, \\delta)$-DP with respect to the private dataset $\\mathcal{D}$.\n\n**Privacy-Critical Step.** The only step in the *SPTI* pipeline that accesses the private data is the Private Evolution module, specifically through the similarity-based voting mechanism in algorithm `Aug_PE_Image_Voting`. This module operates on embeddings of the private image dataset $\\mathcal{D}$ and is used to guide the sampling of synthetic text candidates. Thus, our privacy analysis focuses on this step.\n\n**Voting Mechanism.** For each private image embedding, we identify its nearest neighbor among the generated image embeddings , and increment a vote count histogram $H$ indexed by the candidate text that produced the nearest image. This step corresponds to a histogram query over the private data.\n\nTo ensure differential privacy, we add Gaussian noise $\\mathcal{N}(0, \\sigma^2 \\mathbf{I})$ to the vote histogram $H$ , which is equivalent to applying the \\textit{Gaussian mechanism} to a function with bounded sensitivity.\n\n**Sensitivity Analysis.** The histogram query has $L_2$ sensitivity at most $1$, since each private image contributes a vote to only one candidate (i.e., changing a single image can affect the histogram by at most 1 in one coordinate). This satisfies the precondition for the Gaussian mechanism.\n\nAccording to the Gaussian mechanism \\cite{balle2018improving}, adding noise of variance $\\sigma^2$ per coordinate to a function with $L_2$ sensitivity $1$ ensures $(\\varepsilon, \\delta)$-DP, provided:\n\n$$\n\\Phi\\left(\\frac{1}{2 \\sigma}-{\\varepsilon \\sigma}\\right)-e^{\\varepsilon} \\Phi\\left(-\\frac{1}{2 \\sigma}-{\\varepsilon \\sigma}\\right) \\leq \\delta .\n$$\n\n**Privacy Composition.** According to the adaptive composition theorem of Gaussian mechanisms \\cite{dong2019gaussian}, applying the above Gaussian mechanism across $G$ iterations satisfy $(\\varepsilon, \\delta)$-DP, provided:\n$$\n\\Phi\\left(\\frac{\\sqrt{G}}{2 \\sigma}-\\frac{\\varepsilon \\sigma}{\\sqrt{G}}\\right)-e^{\\varepsilon} \\Phi\\left(-\\frac{\\sqrt{G}}{2 \\sigma}-\\frac{\\varepsilon \\sigma}{\\sqrt{G}}\\right) \\leq \\delta .\n$$\n\n**Post-processing Immunity.** All downstream steps in *SPTI*—including text mutation and image generation via a fixed diffusion model—depend solely on the privatized output of the voting mechanism. By the *post-processing property* of DP, these steps incur no additional privacy cost.\n\n**Overall Guarantee.** Hence, **SPTI** satisfies $(\\varepsilon, \\delta)$-differential privacy with respect to the private image dataset $\\mathcal{D}$, provided that the noise scale $\\sigma$ is properly chosen to satisfy equation above.\n\n### Weakness 2\n\n**w2**: In the limitations section, the authors mention that the proposed method requires significantly more resources than baseline Private Evolution since the former requires several calls to muti-modal models; yet, there is no comparison of running time or token usage between the models.\n\n**WA2**: \nThank you for raising this point. We have conducted a systematic evaluation of the computational cost of our pipeline and will include the results in the next version of the paper.\n\nOur full data generation process involves two main components: an 8B-parameter large language model (LLM) and a 3.5B-parameter diffusion model. In practice, a complete generation run takes approximately 19 hours on a single NVIDIA A800, with a peak memory footprint of around 70 GB. In comparison, the Private Evolution baseline uses a 3.5B-parameter diffusion model, does not involve an LLM, and completes in about 7 hours on a single NVIDIA A800, with a peak memory usage of approximately 35 GB.\n\nWhile our method introduces a modest increase in computational cost—roughly doubling GPU memory and requiring ~2.7× runtime—this is a marginal trade-off given the qualitative generation gains. By leveraging multi-modal language-guided synthesis, our approach significantly improves semantic richness, data diversity, and downstream utility, justifying the additional resources.\n\n\n### Weakness 3\n\n**w3**: There are several grammar mistakes throughout the text.\n\n**WA3**: We went through our paper again and fixed most grammar mistakes.\n\n## Reply to Questions\n### Question 1\n\n\nWe ran more experiments on DP-finetune, and provide comparison between SPTI, PE Image and DP-finetune below. We fix the experiment settings as the same, with $\\epsilon=1.0$, `num_samples=2000`. It's noticeable that we updated chat template for LLM, so we re-run all the previous experiments. Still, new results show that our method performs better than PE image and DP-finetune.\n\n||SPTI|PE Image| DP-finetune|\n|:-:|:-:|:-:|:-:|\n|LSUN Bedroom| **26.39** | 40.36 | 31.76 |\n|Wave-ui-25k| **37.39** | 50.45 | 62.08 |\n|European Art| **57.64** | 76.50 | 63.97 |\n|MM-Celeba-HQ| **34.17** | 50.45 | 57.01 |\n\n### Question 2\nPE image works better than SPTI when the diversity of the private data is too low. For example, SPTI performs poorer than PE image because Cat dataset only involves images of 2 kinds of cat, Maine Coon and Siamese Cat. To validate this idea, we conducted a simple experiment. We build four datasets using images from **ImageNet100** dataset, and use exactly the same setting to run our experiment (with $\\epsilon=1.0$, `num_samples=200`, same text prompt and FID metric). We provide its name, data composition and results below.\n\n| Dataset Name | Composition | Total Num | SPTI | PE Image |\n|:--:|:--:|:--:|:--:|:--:|\n| TwoSpecies_Bird_200 | 100\\*goldfinch image+100\\*indigo bird image| 200 | 144.18 | **110.96** |\n|TwentySpecies_Bird_200| 20 kinds of birds, each 10 images| 200| **111.84** | 168.39 |\n|TwoSpecies_Bird_2K| 1000\\*goldfinch image+1000\\*indigo bird image|2000| 106.26 | **78.66** |\n|TwentySpecies_Bird_2K| 20 kinds of birds, each 100 images | 2000| **84.25** | 172.93 |\n\n`TwoSpecies_Bird_200` dataset shows the same results as Cat dataset, so `TwoSpecies_Bird_200` is capable to be an alternative of Cat dataset. In this table, PE image performs better than SPTI when there's **only 2 kinds of bird**, no matter how many images are there. In contrast, SPTI performs better than PE image when there are **more kinds of bird**, no matter the total number of images. This validate our hypothesis, SPTI is suitable for diverse data while PE image suits for image that are less diverse."}, {"review_id": "w1Zb2rWGKQ", "reviewer": "Reviewer_dX13", "summary": "This paper proposes an interesting new idea to address the problem of differentially private synthetic image generation from text captions. Generating high quality DP images is hard since this leads to a poor privacy-utility tradeoff; on the other hand, DP text generation is relatively easy. This paper proposes a really nice idea of reducing private image generation to text-generation -- by first using a pre-trained model to caption an image, then a private image, then using a DP-finetuned model to produce a synthetic differentially private caption, and finally to use this caption and a pretrained text to image model to generate a similar synthetic image. \n\nThe main idea behind the paper is quite nice and the empirical results are quite strong.", "questions": "n/a", "limitations": "One of the limitations of the paper is what happens when one of the pre-trained models are out-of-distribution wrt the private data. this could be discussed in more detail.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n\n+ Elegant idea\n+ Strong empirical results that beats existing image baselines\n+ Well-written and clear presentation\n\nWeaknesses (mostly minor):\n\n- I know it is a bit difficult to fairly compare with image-only baselines, but some discussion on this would improve the paper further\n- The related work is missing references to other image-only methods, such as [1, 2, 3]\n[1] F. Harder, M. J. Asadabadi, D. J. Sutherland, and M. Park. Differentially private data generation\nneeds better features. arXiv preprint arXiv:2205.12900, 2022\n[2] https://arxiv.org/pdf/2309.00008\n[3] ] T. Cao, A. Bie, A. Vahdat, S. Fidler, and K. Kreis. Don’t generate me: Training differentially private generative models with sinkhorn divergence. Advances in Neural Information Processing Systems, 34:12480–12492, 2021", "was_revised": true, "final_justification": "This is a solid paper and I am supportive of its acceptance.", "rebuttal": "## Response to Comment: Fair comparison with image-only baselines and missing related work on image-only methods\n\nThank you very much for pointing out these related literature. We appreciate the reviewer’s thoughtful comment on fairly comparing against image-only baselines, especially the references [1,2,3]. These methods showcase how differentially private generative models can be trained to produce synthetic image datasets le. Specifically:\n\n* \\[1] and \\[2] advocate for using public datasets or pre-trained public feature extractors to enhance DP training and achieve improved image quality.\n* \\[3] introduces Sinkhorn divergence as a robust objective for training DP generative models, with promising results on image-only datasets.\n\nOur work shares a similar high-level motivation—**leveraging public or existing models to improve private data generation**—but advances the field in several notable ways:\n\n1. **Methodologically**: Our framework can directly utilize state-of-the-art foundation models via API access, without requiring any training or fine-tuning.\n2. **Conceptually**: We bridge modalities by introducing a **text-based intermediary representation**, which allows us to exploit the strong alignment between vision and language models. This cross-modal design expands the range of generative capabilities and circumvents limitations of purely image-based pipelines.\n3. **Empirically**: To the best of our knowledge, our method is the first to demonstrate the ability to generate **high-resolution image datasets with differential privacy guarantees**, going beyond the typical CIFAR-10 resolution benchmark.\n\nWe will integrate this comparison and discussion into the related work section to clarify how our approach builds upon and differs from these prior contributions."}, {"review_id": "4r7aMJMMZf", "reviewer": "Reviewer_ocKp", "summary": "This paper introduces SPTI (Synthesis via Private Textual Intermediaries), a novel framework for privacy-preserving differentially private (DP) image synthesis. The method first captions private images using a vision-language model (VLM), then applies an Augmented Private Evolution algorithm to generate DP captions, and finally generates synthetic images from the DP text via a diffusion model. Notably, the entire pipeline is training-free and capable of producing high-resolution outputs. Experiments show favorable performance based on FID scores. The authors also contribute a new dataset, Sprite Fright, for benchmarking.", "questions": "1. Why are no downstream tasks included to assess the semantic utility of the synthesized images? (Related to Major Weakness 1)\n\n2. Given the known generalization limitations of pretrained models, can you provide examples or analysis of failure cases—i.e., images whose captions omit critical visual information? (Also related to Major Weakness 1)\n\n3. Why are the DP-finetuning results mentioned in the text but not included in Table 1? (See Major Weakness 2)\n\n4. Why was Euclidean distance selected for similarity search in embedding space, instead of cosine similarity? (See Major Weakness 3)\n\n5. The FID metric has known shortcomings [1]. Did you consider alternative or improved evaluation metrics (e.g., CMMD [1])?\n\n6. The substantial performance gap between SPTI and the PE image on the Cat dataset (Table 1) warrants further explanation.\n\n[1] Jayasumana, S., Ramalingam, S., Veit, A., Glasner, D., Chakrabarti, A., & Kumar, S. (2024). Rethinking fid: Towards a better evaluation metric for image generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 9307-9315).", "limitations": "The authors appropriately acknowledge that the captioning process may not preserve all image information and that domain-specific details could be lost. However, this limitation could be better quantified through targeted experiments, especially using downstream tasks that rely on high-fidelity semantics.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strengths:\n1. The idea of using text as an intermediary for privacy-preserving image synthesis is both novel and practical, particularly because it avoids the need for model training and leverages powerful pretrained models.\n\n2. The proposed Aug-PE algorithm for DP text generation is a meaningful technical contribution.\n\n3. The authors contribute a new dataset, Sprite Fright, which could benefit future work in DP image synthesis.\n\nWeaknesses:\nMajors:\n1. Information loss via captioning is underexplored.\nUsing VLMs to extract captions from images can introduce significant information loss, especially in specialized domains or when fine-grained details are involved. While this is briefly acknowledged in the limitations section, it deserves deeper empirical investigation. The current evaluation relies solely on the FID metric, which does not capture semantic fidelity. Including downstream tasks (e.g., classification) would better evaluate the utility of the generated images.\n\n2. Missing results for DP-finetuning.\nThe experiments section mentions a comparison against DP-finetune, but Table 1 does not report these results. This inconsistency should be addressed.\n\n3. Choice of similarity metric.\nSection 3.2 states that Euclidean distance is used for nearest-neighbor search based on embeddings. However, cosine similarity is more commonly used in text/image embedding spaces. Justifying this choice—or comparing both—would strengthen the methodology.\n\nMinors:\n1. The last paragraph of the Introduction lacks a summary of Section 5. Adding this would improve flow and completeness.\n2. Inconsistent naming of prior work—please standardize references to \"PE\" or \"PE image\" throughout the paper.", "was_revised": true, "final_justification": "All my concerns have been addressed in the rebuttal. Furthermore, the discussion between the authors and other reviewers reinforces the strength of the proposed image voting mechanism. I have therefore raised my score to 4.", "rebuttal": "We appreciate the reviewer’s insightful feedback. Below, we provide detailed responses to each of the raised points.\n\n**Response to Weakness 1: more verification of information loss via captioning.**\n\nThank you for the question. We have conducted additional experiments on downstream tasks and evaluated the proposed CMMD metric to further validate that our approach preserves semantic fidelity. The results are presented below in Question 5.\n\n**Response to Weakness 2: Missing results for DP-finetuning.**\n\nWe provide evaluation of DP-finetuning below in response to Question 3.\n\n**Response to Weakness 3: Choice of similarity metric.**\n\nWe provide our argument below in response to Question 4.\n\n## Question 1\nWe're sorry for not implementing downstream tasks, we will provide more systematic experiments in the future. We prepare to use CelebA dataset and conduct a series of 2-class classification tasks according to its features. We're currently running experiments on `Wearing_Lipstick` feature, and acquired good results. We will update this series of experiments in the future version of our paper.\n\n## Question 2\nWe test our method on various datasets to confirm that our pipeline works well in different circumstances, and experiments did show good results. However, we admit that there are cases where text description doesn't align well with the images. This usually happens when images have such intensive information that VLMs can hardly capture all of them, or limits on text length prohibits VLMs to fully describe all the scenes. Nevertheless, we believe the performance of our method will improve as state-of-the-art VLMs and diffusion models reach better results.\n## Question 3\nOur method focuses on conditions where private data may not be directly accessed. In our method, we refer to private data through it representations. In contrast, DP-finetune has direct access to private data, which can make full use of it. This is the key reason why we focus on comparison between SPTI and PE image, since our method is an improvement on PE image method.\n\nNevertheless, SPTI performs better than DP-finetune in many circumstances, and we also conducted more experiments after paper submission, and we provide experiment results below.\n\n||SPTI|PE Image| DP-finetune|\n|:-:|:-:|:-:|:-:|\n|LSUN Bedroom| **26.39** | 40.36 | 31.76 |\n|Wave-ui-25k| **37.39** | 50.45 | 62.08 |\n|European Art| **57.64** | 76.50 | 63.97 |\n|MM-Celeba-HQ| **34.17** | 50.45 | 57.01 |\n\n## Question 4\nWe adopt the same similarity metric used in the Private Evolution (PE) work. Since this component is not the focus of our technical contribution, we retain it unchanged to ensure a fair and consistent comparison with PE.\n\n## Question 5\nWe're sorry for not considering more evaluation metrics before, so we added more evaluation and re-run some of our previous experiments for results. Here we provide experiment results with both FID and CMMD evaluation. The consistency in these two metrics show that our method indeed improves PE image method.\n\nThe table below shows CMMD value for SPTI and PE method. In this table, we found our method performs the best except for the case in LSUN Bedroom. We suppose it's because LSUN Bedroom has such a large quantity of image data that improves the performance of fine-tuning, but this needs further experiments. Other conditions show the excellence of our method.\n\n| Dataset | SPTI | PE | DP-finetune |\n|:---:|:---:|:---:|:---:|\n| LSUN Bedroom | 1.744 | 2.066 | **1.376** |\n| Wave-ui | **1.480** | 3.132 | 2.149 |\n| European_art | **0.684** | 1.016 | 2.447 |\n## Question 6\nWe propose that PE image works better than SPTI when the diversity of the private data is too low. For example, SPTI performs poorer than PE image because Cat dataset only involves images of 2 kinds of cat, Maine Coon and Siamese Cat. To validate this idea, we conducted a simple experiment. We build four datasets using images from **ImageNet100** dataset, and use exactly the same setting to run our experiment (with $\\epsilon=1.0$, `num_samples=200`, same text prompt and FID metric). We provide its name, data composition and results below.\n\n| Dataset Name | Composition | Total Num | SPTI | PE Image |\n|:--:|:--:|:--:|:--:|:--:|\n| TwoSpecies_Bird_200 | 100\\*goldfinch image+100\\*indigo bird image| 200 | 144.18 | **110.96** |\n|TwentySpecies_Bird_200| 20 kinds of birds, each 10 images| 200| **111.84** | 168.39 |\n|TwoSpecies_Bird_2K| 1000\\*goldfinch image+1000\\*indigo bird image|2000| 106.26 | **78.66** |\n|TwentySpecies_Bird_2K| 20 kinds of birds, each 100 images | 2000| **84.25** | 172.93 |\n\n`TwoSpecies_Bird_200` dataset shows the same results as Cat dataset, so `TwoSpecies_Bird_200` is capable to be an alternative of Cat dataset. In this table, PE image performs better than SPTI when there's **only 2 kinds of bird**, no matter how many images are there. In contrast, SPTI performs better than PE image when there are **more kinds of bird**, no matter the total number of images. This validate our hypothesis, SPTI is suitable for diverse data while PE image suits for image that are less diverse."}]

Author rebuttal:

Meta-review: This paper proposes Synthesis via Private Textual Intermediaries, a framework for privacy-preserving differentially private (DP) image synthesis. The method first captions private images using a VLM, then applies an APE algorithm to generate DP captions, and finally generates synthetic images from the DP text via a diffusion model. The entire pipeline is training-free and capable of producing high-resolution outputs. Experiments show favorable performance based on FID scores. Another contribution is a new dataset, Sprite Fright, for better benchmarking.

The strengths of the work lie in several aspects like interesting idea and sufficient novelty, not requiring model fine-tuning, modular pipeline, good writing, contributing a new dataset for the community, etc. 

The main negative concerns include possible information loss via captioning, heavy dependence on the captioning capability of VLM, lack of baseline comparisons, sensitivity to changes in the image generation models, etc., most of which however have been resolved by the authors during rebuttal. Therefore, all the reviewers are finally satisfied with the work and give Borderline Accept (x4). Based on the above analysis, the AC think it deserves an Accept.
Decision: Accept (poster)