Paper: Differentially Private Latent Diffusion Models (ICLR-2024)

Reviewer comment (Novelty):
The most significant thing lack of novelty: The paper simply fine-tune the pre-trained public model with a similar technique in You & Zhao (2023). This is unremarkable because reducing the fine-tuning space is mentioned in You & Zhao (2023).

Author response:
Novelty: We clearly mentioned in our introduction that the use of DP-SGD to fine-tune a pre-trained model itself could be viewed unremarkable as a method. However, please pay attention to what impact this seemingly ordinary method can bring. In fact, the potential impact is well appreciated and described by other reviewers: Reviewer tmmk wrote “The paper reads incremental, but it does a decent job in finding the correct hammer, which does lead to promising results.”, and reviewer c8Le wrote “Although the proposed approach lacks novelty, DP latent diffusion models could be of great interest to the community and the results look promising. “.

Reviewer comment (Writing):
The notations $\Delta$ and $\nabla$ used for the encoder and decoder might be confusing and could be clarified.

Author response:
"encoder “ Δ and decoder ∇ ": non-standard notations: We understand that E and D are commonly used for encoders and decoders. Unfortunately, we have used D to note dataset to define differential privacy. We are considering changing encoder Δ and decoder ∇ to be Enc and Dec as used in [1,2,3]. But if the reviewer has a suggestion on which notation we should use for encoders and decoders, we’re open to it.

Reviewer comment (Reproducibility):
In Table 2, it's unclear how the privacy budget of synthetic data was handled when reporting 88.3% accuracy.

Author response:
Table 2: There is **NO synthetic data** involved in the 88.3% accuracy. Showing this accuracy (evaluated on real test data) is quite standard in DP data generation literature, as it is used as a top accuracy synthetic data could achieve given a dataset (implying that no synthetic data can be better than real data).

Reviewer comment (Reproducibility):
Can you explain how the conditioning embedder works when it has been pre-trained with public data with different labels than the private data? Does the model treat it as a class embedder, or does it treat it as random initialization?

Author response:
Conditioning embedder: Conditioning embedder refers to both **class** conditioning and other types of conditioning such as conditioning on prompts, blurred variants of input image, semantic maps, layouts, etc. For more information about conditioning in LDM, refer to [4]. When there’s a difference between the classes in private and public datasets, e.g., in imagenet32 with 1000 classes and cifar10 with 10 classes, we freeze the embedder corresponding to 9900 classes, while fine-tuning the pre-trained embedder for the first 10 classes. Then, we use only these fine-tuned components when generating class-conditioned images for CIFAR10.

Reviewer comment (Experiments):
The paper proposes to fine-tune only the attention layers and the condition embedders to reduce the number of fine-tuning parameters. One of the most commonly used approaches in both DP and non-DP communities is to do LoRA, adapter, or compacter fine-tuning (see [3] for an example). It would be better to comment on or experimentally compare with such approaches.

Author response:
Using LoRA in our model: Thanks for the suggestion. Our current submission studies the trade-off between the final accuracy based on which layer’s attention modules to be fine-tuned, and the privacy guarantee we provide. Once we include LoRA or other parameter-efficient LLM training schemes, this fore-mentioned trade-off takes an extra dimension. Namely, once we approximate the parameters by a few low-rank matrices, we reduce the dimension of the parameters to fine-tune with DP-SGD (which is a good thing). This, however, introduces a loss of information about the parameters (which is a bad thing). As a result, now we need to study the trade-off among the three: (a) which layer’s attention modules to fine-tune (so far fine-tuning the layers in out block is preferred) ; (b) how accurate the approximation gets (which prefers a higher rank); and  (c) how the resulting model performs in the presence of noise due to DP-SGD (which prefers a lower rank). This trade-off might exacerbate when there is a larger gap between the public and private datasets we consider. We are currently looking into the possibility of plugging in the LoRA package into our code base to test this trade-off. We will update this rebuttal once we get the results.

Reviewer comment (Reproducibility):
Table 7 in the appendix shows the GPU hours for training DP-LDM and DP-DM. Could you clarify if that includes the pre-training cost for both methods? If yes, it would be clearer to break down the time into pre-training and fine-tuning stages. If not, it would be better to include pre-training costs as well, as at least in the experiments of this paper, customized pre-training for each dataset has to be done.

Author response:
Clarification for the GPU hours spent: Sorry for the confusion. Table 7 does not contain the pretraining hours. There are three steps in our method, first pretraining an autoencoder takes 8 hours as listed in Table 14, second pretraining the LDM takes 6 hours as listed in Table 15, third fine-tuning the LDM to make it DP takes 10 hours, which is listed in Table 7. The DP-DM method does not contain any pretraining, they trained from scratch, which takes 192 hours in total.

Reviewer comment (Novelty):
Introduction claims that "DPSGD ... does not scale well for large models that are necessary for learning complex distributions." It is not necessarily true. Prior work has demonstrated that DP-SGD works well with large language models. See [1,2] for some examples.

Author response:
DPSGD scaling issues: We’re aware of [1,2] that the reviewer cited. The finding of these papers is that when the gradients can be characterized by a few principal components (this happens in the case of fine-tuning LLMs), the scaling issue of DP-SGD is lifted. However, in general, this issue is present in most settings of training deep neural network models.

Reviewer comment (Experiments):
Section 5.3 discusses the process of selecting pre-training datasets. However, this selection process needs to use private data and therefore is NOT DP. Please refer to [4] for an example of how to select pre-training **dataset** in a DP fashion, and [5] for an example of how to select pre-training **samples** in a DP fashion. According to the results in the prior work, I guess that the selection between SVHN, KMNIST, and MNIST would only incur a small privacy cost. Still, the paper should at least discuss this issue (i.e., the privacy cost of this dataset selection step is ignored) and the related work, if not redoing the experiments.

Author response:
FID using private data: Whether it’s necessary to add noise to FID depends on whether we release these intermediate FID scores for choosing the public dataset. In our submission, we did not consider privatizing FID, as we had no intention of releasing these FID scores. One might argue that even if the intermediate FID scores are not released, the selection itself is already embedded in the pre-trained model, which is then used for private fine-tuning. So, to be thorough, we can consider using a DP mechanism to privatize the selection step. Recall that FID is calculated by two statistics, mean and covariance of features from a pre-trained inception network, so using a Gaussian mechanism to privatize the mean and covariance of the private data before computing FID can be used [7]. A similar idea was used in DP-MEPF[8], which uses the mean feature difference between private and public data to determine early stopping. Considering the privatized mean difference only makes sense, as the noise scale for mean perturbation is O(d), while that for covariance perturbation is O(d^2).  As shown below, the privatized mean alone, and both mean and covariance do not alter the rank of the final FID scores significantly. This stems from the fact that the sensitivity of the mean and covariance is extremely small for datasets we consider, i.e., on the order of 1 over the number of training data. So privatizing or not privatizing FID does not change the conclusion about which dataset is more useful to use as public data for pre-training. We will include this point in our final version. 

Here is the FID when we privatize both mean and covariance, the eps is the total value of eps for mean and eps for covariance, basically we set eps for mean = eps for covariance.
| total eps  | 0.1         | 0.5        | 1          | 2          |
|------------|-------------|------------|------------|------------|
| SVHN       | 66.7628     | 14.7779    | 7.6202     | 4.1        |
| KMNIST     | 80.6722     | 17.2918    | 8.8239     | 4.5922     |
| **EMNIST** | **37.5071** | **7.9408** | **4.1476** | **2.1157** |

Here is the mean difference when we privatize mean only:
| eps        | 0.1        | 0.5        | 1          | 2          |
|------------|------------|------------|------------|------------|
| SVHN       | 0.2698     | 0.2668     | 0.2666     | 0.2665     |
| KMNIST     | 0.0662     | 0.064      | 0.0639     | 0.064      |
| **EMNIST** | **0.0204** | **0.0198** | **0.0197** | **0.0197** |

[7] Park, Mijung et al, DP-EM: Differentially Private Expectation Maximization, AISTATS, 2017

[8] Harder, Frederik et al, Pre-trained perceptual features improve differentially private image generation, TMLR, 2023

Reviewer comment (Presentation):
Table 8 in the appendix: what does "Best" mean in "Best CNN accuracy"?

Author response:
Table 8, we did the hyperparmeter search for each dataset, "Best CNN accuracy" is the highest score we achieved for individual dataset, which shows out choice of EMNIST will achieve best performance among those three.

Reviewer comment (Evaluation):
Table 10 shows that the results are sensitive to the selection of fine-tuning layers, especially in regimes with a high privacy budgets. It would be better to discuss hypotheses about why fine-tuning 9-16 layers is the best and provide recommendations for practitioners on how to choose this hyper-parameter for new datasets.

Author response:
Why fine-tuning 9-16 layers is the best and provide recommendations : Our results suggest that empirically finetuning attention modules in out_blocks outperform finetuning those input_blocks. This is consistent with the results of [6]. We sould recommend this strategy to users if their privacy budget is limited. We will add this discussion to our final version.

Reviewer comment (Theory):
Section 2.2 states that "A single entry difference could come from either replacing or removing one entry from the dataset D." While both definitions (replacing vs. removing) are valid and used in practice, they result in different DP bounds as the sensitivity is different. The paper should be clear which definition is used in the experiments.

Author response:
Privacy definition: Yes, we use the inclusion/exclusion definition of DP, as this is what Opacus uses. We will make this clear in our paper.

Reviewer comment (Novelty):
Missing references on model architecture: U-Net [1], transformers [2].

Author response:
We thank the reviewer for the positive and constructive feedback on our paper. Thanks for pointing out the missing references and further improvements we can make to make the paper clearer; we will add those to our final version.

Reviewer comment (Experiments):
Missing reference on dataset: CelebA-HQ [3].

Author response:
We thank the reviewer for the positive and constructive feedback on our paper. Thanks for pointing out the missing references and further improvements we can make to make the paper clearer; we will add those to our final version.

Reviewer comment (Theory):
Missing reference on properties of DP: [4].

Author response:
We thank the reviewer for the positive and constructive feedback on our paper. Thanks for pointing out the missing references and further improvements we can make to make the paper clearer; we will add those to our final version.

Reviewer comment (Evaluation):
The authors presented the accuracy results only on CIFAR-10 (in Table 2). All the remaining results on other datasets (Tables 3, 4, 5) are for FID, which can at most be regarded as a fidelity metric and serves as a proxy for the utility of the synthetic data. It would be more straightforward to directly present the utility results, i.e., accuracy on the classification task. The authors should add the accuracy results on CelebA for better interpretation. (Nit: "we see a significant drop in accuracy as shown in Table 3." — Table 3 is about FID.)

Author response:
Regarding extra evaluations suggested by the reviewer:
1. We’re currently working on implementing the experimental setups for testing classification accuracy of the synthetic CelebA64 dataset. We will upload the results here once it is done.

Reviewer comment (Experiments):
Ghalebikesabi et al. [6] evaluated the high-dimensional medical dataset camelyon17, and so does the recent [7]. The authors should consider performing evaluation on this dataset.

Author response:
Regarding extra evaluations suggested by the reviewer:
2. We agree with the reviewer that evaluating our method on a scenario with an extreme domain gap between the public and private datasets. Hence, we are working on testing our method on the Camelyon17 dataset.

Reviewer comment (Presentation):
The baseline methods do not come with a citation. Is DP-diffusion [6] or [8]? Appendix A.2 suggests [6], but the caption of Fig. 8 suggests [8], which is confusing.

Author response:
The baseline methods do not come with a cite. Is DP-diffusion [6] or [8]? Appendix A.2 suggests [6], but the caption of Fig. 8 suggests [8], which is confusing. : Thanks! We have corrected the typo.

Reviewer comment (Writing):
In the statement "inserted into the layers of the underlying UNet backbone": it is unclear exactly where this insertion occurs.

Author response:
"inserted into the layers of the underlying UNet backbone": exactly where? The original Unet does not contain attention modules, we followed the way of LDM to insert attention modules into layers as the way illustrated in Figure 1 (marked in red).  For input_blocks, attention modules are added to the Resnet block except the first in block and downsampling ones; For middle block, attention module is added between two Resnet blocks; For out_blocks, attention modules are added except the out block.

Reviewer comment (Writing):
In the sentence "It modifies stochastic gradient descent (SGD) by adding an appropriate amount of noise by employing the Gaussian mechanism to the gradients", clarity and phrasing could be improved.

Author response:
"It modifies stochastic gradient descent (SGD) by adding an appropriate amount of noise by employing the Gaussian mechanism to the gradients" We have clarified this sentence.

Reviewer comment (Writing):
The sentence "However, Lin et al. (2023) do privatize diffusion" should be "do not privatize diffusion."

Author response:
"However, Lin et al. (2023) do privatize diffusion" -> do not: Thanks, we have fixed this.

Reviewer comment (Writing):
The notation "encoder $\Delta$ and decoder $\nabla$" is non-standard and should be clarified.

Author response:
"encoder “ Δ and decoder ∇ ": non-standard notations: We understand that E and D are commonly used for encoders and decoders. Unfortunately, we have used D to note dataset to define differential privacy. We are considering changing encoder  Δ and decoder ∇  to be Enc and Dec as used in [9, 10, 11]. But if the reviewer has a suggestion on which notation we should use for encoders and decoders, we’re open to it. 

[9] ABL Larsen et al, Autoencoding beyond pixels using a learned similarity metric,  PMLR, 2015

[10] Liu, Yihong et al, On the Copying Problem of Unsupervised NMT: A Training Schedule with a Language Discriminator Loss, ACL, 2023

[11] Sprinks, Graham, and Moens, Marie-Francine, Generating Continuous Representations of Medical Texts, NAACL, 2018