Paper: Safeguarding Data in Multimodal AI: A Differentially Private Approach to CLIP Training (ICLR-2024)

Reviewer comment (Theory):
There are several factually inappropriate usages of the notion of DP. DP is not "incorporated" in a model or multimodality (as the authors mention in different ways a few times throughout the paper), DP is a property of a randomised algorithm (in this context, the training algorithm that produces the distribution of models, not the model).

Author response:
Thank you for the suggestion. We would like to note that several papers (from distinguished ML researchers) in the literature also use the wording “incorporate DP” [4,5]. In light of your comment, we will rephrase it to “incorporate privacy” for better description of our method.

Reviewer comment (Novelty):
Proposition 3.1 is a trivial consequence of the basic theorems of DP, and Algorithm 1 is a simple modification of DP-SGD (that is already available in standard DP-training libraries). The only interesting aspect is adapting the training procedure to contrastive learning, which is trivial. Therefore the novelty of these components of the paper is negligible.

Author response:
We would like to clarify that our Proposition 3.1 states that the Gaussian mechanism applied on minibatch level with this calibrated noise satisfies $(\epsilon,\delta)$-DP for per-sample. We acknowledge that the proof of this proposition is straightforward, so we named it as Proposition 3.1 rather than Theorem 3.1 in the original paper. We also would like to emphasize that our contribution is to demonstrate the efficiency of this simple approach for CLIP training that can achieve high accuracy while maintaining DP, supported by both our theory and experiments. The theoretical analysis in Section 5 is quite challenging, as we dealt with ***non-decomposable*** CLIP loss function. The technical tools employed in the existing literature to study DP-SGD do not directly apply to our proposed DP-CLIP. The non-decomposable loss required us to develop novel technical tools for theoretical analysis. In particular, we developed a novel concentration argument to provide the convergence analysis. This new analysis provides theoretical guarantees that per-batch clipping efficiently achieves high accuracy while maintaining DP. We consider this to be one of the major contributions of our work.

Reviewer comment (Experiments):
It is unclear why the zero-shot prediction of CLIP is not used as a baseline (as it would be equivalent to epsilon = 0). Furthermore, the authors are neglecting parameter efficient fine-tuning baselines, for instance like [1]. The description of the baselines is lacking and quite confusing: it's not clear why the authors have two DP-SGD baselines, and what they're exactly updating during training.

Author response:
Thank you for your suggestion. We added both zero-shot prediction results and non-private training results in the universal response. We note that our method is proposed for DP-CLIP training, while [1] focuses on differentially private fine-tuning that optimizes some task-specific loss. Thus their method is not a natural baseline to compare with. We emphasize that our proposed method uniquely addresses the DP training of contrastive vision-language models, not only limited to DP image classifications. From this point, there are no “fair” baselines to compare with. We specifically chose these benchmarks because they demonstrate State-of-the-Art results in image classification tasks on these datasets. Our intention is not to compare with their methods, but rather to evaluate our proposed approach against the established performance standards. As image classification is not one of the tasks in [1], it is not included in our baselines.

Reviewer comment (Experiments):
The experimental analysis focuses exclusively on known computer vision datasets that do not differ from the training distribution of CLIP. It would be useful if following the suggestions of [2,3] the authors could present results in settings with low data regimes and with significant distribution shift with respect to the pre-training set, which represents a more realistic application setting. Furthermore, the authors deliberately avoid settings where DP is known to be hard due to the relatively low amount of training data per class (e.g. CIFAR-100/ImageNet).

Author response:
Thank you for your comments! Following your suggestions, we ran our experiments on CIFAR-100. The mean and standard deviation of 10 trials are reported in the universal response. Following your suggestions, we also ran our experiments on a dataset with distribution shifts, as well as settings with low data regimes. Paper [3] which you cite presents the GTSRB dataset as one that has “minimal overlap with ImageNet-1K, with [. . .] 43 traffic signs aggregated into a single label in ImageNet-1K.” We ran our experiments on GTSRB and obtained the following results for 10 trials each, and our results are reported in the universal response.

Additionally, we replicated the experiments in the paper [3] under low data settings. Following the experiments demonstrated by Figure 7 in [3], we randomly subsampled 10% and 50% of GTSRB’s training set and only used that subsample for training. Then, we tested on the test set. We ran these experiments for epsilon=0.1 and 0.7 and reported the results over 10 trials below.

| Epsilon and Train Percentage Used | 0.1 at 10% | 0.1 at 50% | 0.7 at 10% | 0.7 at 50% |
|:---------------------------------:|:----------:|:----------:|:----------:|:----------:|
|                mean               |   28.33%   |   45.43%   |   37.98%   |   79.42%   |
|                std                |  4.28E-03  |  1.42E-02  |  1.04E-02  |  6.48E-03  |

As we can see when we compare our results to Figure 7 from [3] above (left hand side), in all four settings, we outperform on the test set for GTSRB.

| Epsilon and Train Percentage Used |    0.1 at 10%    |     0.1 at 50%    |     0.7 at 10%    |     0.7 at 50%    |
|:---------------------------------:|:----------------:|:-----------------:|:-----------------:|:-----------------:|
|              our mean             |      28.33%      |       45.43%      |       37.98%      |       79.42%      |
|           mean from [3]           | Approximately 5% | Approximately 20% | Approximately 27% | Approximately 56% |

Reviewer comment (Experiments):
Not accounting for the privacy loss incurred in tuning the hyperparameters is a problem. Simply because baselines have done that, it doesn't justify the authors from reporting inflated numbers. I would recommend the authors to at least assume the availability of some public data that is kept out of training and evaluations and to run all the baselines fairly in this setting.

Author response:
Thank you for your note. We noticed a lot of paper not accounting for the privacy budget of hyperparameters tuning, and due to limited computational resources, we did not include it.

Reviewer comment (Novelty):
BLIP is definitely no more state-of-the-art as the authors claim. Several more sophisticated VLMs have been released since BLIP, e.g. LLaVa 1.5.

Author response:
Thank you for your comment. We will rephrase our language from “state-of-the-art” to “extension of CLIP.”  We acknowledge the feedback and want to re-emphasize that our focus in this paper is primarily to learn the private representations through contrastive loss instead of SOTA vision-language models such as LLaVa, which does not include contrastive loss.

Reviewer comment (Evaluation):
Since the gaussian mechanism is not applied at a sample level but at a mini-batch level, are the authors making a fair comparison between the $\epsilon$ values of their technique and the baselines?

Author response:
Although the noise injection procedure is different, we ensure that the final output is $(\epsilon,\delta)$-DP by Proposition 3.1. This is through clipping the minibatch gradient. So we are in a fair comparison setting for privacy protection. However, we acknowledged that our experimental results rely on stronger pretrained models and additional information from text. Please also refer to our universal response with respect to the choice of baselines.

Reviewer comment (Experiments):
Why the baselines selection varies based on the choice of the dataset?

Author response:
Our intention is not to compare with these baseline methods, but rather to evaluate our proposed approach against the established performance standards on these datasets. Therefore, we reported some recently established performance standards on these datasets.

Reviewer comment (Experiments):
Does the training set of the pre-trained CLIP contain MNIST-like images?

Author response:
Thanks for your comments! The original CLIP paper claimed that “CLIP was pretrained on a dataset of 400M image-text pairs, covering as broad a set of visual concepts as possible,” We realize that this means that CLIP's training set might be drawn from similar distributions to datasets such as MNIST, but we include zero-shot results above to show how much we can still privately learn through our presented training process. We have also included our results on GTSRB, a dataset with “minimal overlap with ImageNet-1K, with [. . .] 43 traffic signs aggregated into a single label in ImageNet-1K,” in the universal response above [1].

Reviewer comment (Experiments):
How different is the vizwiz image captioning dataset from the training set?

Author response:
Since the dataset used to train CLIP is not disclosed by OpenAI, we cannot answer how different the Vizwiz dataset is from the training dataset of CLIP. However, our additional experiments show that non-private fine-tuned scores are around 15% or above higher than zero-shot scores, suggesting that CLIP does not learn much from the Vizwiz dataset, or a similar dataset.

Reviewer comment (Experiments):
The paper employs smaller classification datasets such as MNIST, Fashion-MNIST, CIFAR-10, and SVHN. It would have been preferable to observe results at the Imagenet scale, but I understand that the computational resources required for such experiments would have been substantial.

Author response:
Thank you so much for your feedback! Upon your suggestion and suggestions from other reviewers, we additionally ran our methods on a larger dataset (CIFAR-100), as well as another dataset GTSRB.

Reviewer comment (Evaluation):
No comparison with real-world threat models has been provided. Epsilon-utility trade-offs can be misleading without testing them against actual attacks, as epsilon guarantees are built upon numerous assumptions, as indicated in [1].

Author response:
We thank you for your comments. Previous work has suggested that DP can protect against common privacy attacks such as reconstruction attacks and membership inference attacks [2]. We agree that it would be nice to demonstrate empirical protection performance against attacks. However, most membership inference attacks rely on the shadow model technique, which requires training thousands of shadow models with the same architecture as the target model.  As it requires significant computing resources, we decide it is beyond the scope of this project, and we leave efficient attack techniques as future work.

Reviewer comment (Experiments):
Framing the algorithm as "DP-CLIP" is very misleading. You didn't train a CLIP model from scratch using DP-SGD, but rather continue pretraining a **well-trained** model in a privacy-preserving manner. This is not mentioned in the abstract, and is not made clear until the experimental section. As a side note, the comparison with Yu et al. (2023) is unfair: 1) there are obviously no "two stages" in your algorithm since you only performed continued pretraining; 2) Yu et al. (2023) only pretrained on synthesized textures to speed up the DP training process, while the starting point of your DP training is already a very strong model.

Author response:
We would like to emphasize that our proposed algorithm and the associated theory are designed for differentially private CLIP training from scratch. In the experimental section, we opted to use a pretrained model as an initialization solely to reduce computational costs. It's important to note that our intention was not to compare our method against Yu et al. (2023) due to the inherent differences in the addressed problem. Our mention of their work is specifically to highlight that, like ours, their framework also involves using a pretrained model as initialization.

Reviewer comment (Experiments):
The classification tasks are too simple given the power of the pretrained model. It might not be necessary at all to perform private continued pretraining; plus I found pretraining on datasets such as MNIST and CIFAR-10 to be super weird. Two critical baselines are currently missing: the accuracy of the vanilla CLIP model (without continued pretraining), and the accuracy of $\varepsilon=\infty$ (non-private continued pretraining).

Author response:
We have included both the zero-shot results and the non-private results. we also added two new datasets: Cifar 100 and GTSRB, where the GTSRB dataset presents a significant distribution shift with respect to the pre-training set. Please refer to our universal response.

Reviewer comment (Experiments):
The comparison with other baselines in Section 4.2 is arbitrary and careless. The authors didn't evaluate the same set of methods on all datasets, and didn't report whether the results for other algorithms are based on their own implementation or directly copied from prior works. The accuracy of DP-Sinkhorn on Fashion-MNIST does not match the one reported in the original paper. It is also unclear whether these algorithms are indeed the state-of-the-art methods in image classification. Finally, DP-Sinkhorn is not even a DP image classification method -- while it is capable of performing such task, it is essentially an algorithm of DP data synthesis (take a look at [1] if you are not familiar with this concept). It is unbelievable that the authors chose this method for comparison.

Author response:
Thank you for pointing it out. We found several versions of the DP-Sinkhorn paper online: 1) from neurips https://proceedings.neurips.cc/paper_files/paper/2021/file/67ed94744426295f96268f4ac1881b46-Paper.pdf and 2) from arxiv https://arxiv.org/pdf/2111.01177.pdf. We used results from the newer version that appeared at NeurIPS in our paper. Looking at Table 1 from neurips version, we got an accuracy of 83.2 on MNIST and 73.0 on Fashion MNIST for epsilon=10 as cited in our paper (the numbers were taken from the best results with different architectures). If we look at the arxiv version, we get 83.2 on MNIST and 74.6 on Fashion MNIST for epsilon=10. Regarding our choice of the baseline methods, please refer to our universal response.

Reviewer comment (Experiments):
I didn't buy the results from Section 4.3; particularly, I have never seen using $\varepsilon = 1e-4$ in the privacy literature, and it is hard to believe that such privacy budget could still lead to a meaningful model (in the extreme case $\varepsilon=0$, the model will become completely random). On the other hand, the authors seemed to suggest that the model's performance is mostly unaffected even in this extreme case. I insist: 1) report the $\sigma$ (the noise multiplier), and check the SNR to see whether the results make sense; 2) use $\varepsilon=1e-8$ and see whether the results are still consistent; 3) use the vanilla CLIP model (without any further continued pre-training) and see whether the results are already very good.

Author response:
Thank you for your suggestions. We attach a set of more thorough experiments, including zero-shot results, eps = 1e-4, 1, 5, and non-private results. Please refer to the table in the universal response. Our additional experimental findings align with the results presented in the paper, demonstrating that post-training consistently outperforms zero-shot performance for epsilon values ranging from 1e-4 to $\infty$ (non-private.) A consistent trend is observable across various metrics. However, we did implement the 1e-8 regime for multiple trials and found that the result was pretty random, which suggests that the noise is substantial enough to compromise the model's utility. As a side note, due to the limit of time and computational resources, this set of additional results was generated using a training set of 5K samples and a test set of 1K samples, which results in a performance gap compared to our original results. 

Regarding the state-of-the-art non-private performance provided by IBM Research AI, it is a score from the Kaggle website (https://paperswithcode.com/sota/image-captioning-on-vizwiz-2020-test). It is the only comparable baseline we could find online, although it is unclear what their model is. We will add this link to the manuscript.

Reviewer comment (Experiments):
Other issues in Section 4.3: 1) there is no description of the non-private model, I have completely no idea what "IBM Research AI" is and whether the comparison with DP-BLIP makes sense at all; 2) it is unclear why a baseline of non-private BLIP is missing.

Author response:
Thank you for your suggestions. We attach a set of more thorough experiments, including zero-shot results, eps = 1e-4, 1, 5, and non-private results. Please refer to the table in the universal response. Our additional experimental findings align with the results presented in the paper, demonstrating that post-training consistently outperforms zero-shot performance for epsilon values ranging from 1e-4 to $\infty$ (non-private.) A consistent trend is observable across various metrics. However, we did implement the 1e-8 regime for multiple trials and found that the result was pretty random, which suggests that the noise is substantial enough to compromise the model's utility. As a side note, due to the limit of time and computational resources, this set of additional results was generated using a training set of 5K samples and a test set of 1K samples, which results in a performance gap compared to our original results. 

Regarding the state-of-the-art non-private performance provided by IBM Research AI, it is a score from the Kaggle website (https://paperswithcode.com/sota/image-captioning-on-vizwiz-2020-test). It is the only comparable baseline we could find online, although it is unclear what their model is. We will add this link to the manuscript.

Reviewer comment (Experiments):
The experiments should be running over multiple random seeds, and please include the standard deviations in the tables.

Author response:
Thank you for your suggestion — we have re-run all our experiments using 10 trials and report the mean and standard deviations in the universal response.

Reviewer comment (Experiments):
The dataset paragraph should be revised. The descriptions of the datasets are way too long yet important details are missing (e.g., the dimensionality of each dataset). The metrics used in image captioning are not explained.

Author response:
We thank you for your suggestions. We will revise the description for the datasets and add explanations on evaluation metrics for the image captioning task.

Reviewer comment (Reproducibility):
"TensorFlow Privacy": which privacy accountant did you use exactly? As a side note, RDP and PRV accountant are the go-to choices nowadays.

Author response:
We are using RDP accountant implemented at tensorflow privacy package (https://github.com/tensorflow/privacy/blob/master/tensorflow_privacy/privacy/analysis/compute_noise_from_budget_lib.py .)

Reviewer comment (Writing):
Various grammatical issues, included but not limited to: "while a smaller or equal $\varepsilon$" -> "with a smaller or equal $\varepsilon$"; "allow our private approach to achieving" -> "to achieve"; "Apart from their work" -- I don't understand what you meant here; "requires... should be indistinguishable" --> "to be indistinguishable". Please do a professional proofreading before submitting your work.

Author response:
We have fixed these grammar issues and did a thorough check on the full paper.