Paper: Differentially Private Vision-Language Foundation Models via Image Captioning (ICLR-2024)

Reviewer comment (Novelty):
The paper lacks any meaningful theoretical or algorithmic contributions. The only insight of the paper is that image captioning is a suitable loss for DP training.

Author response:
The vast majority of existing work on DP training operates on a single modality, be it image classification, image-only SSL, text classification, language modeling, etc. Our work is the first to push beyond this barrier and show that DP can also be successfully applied to the currently mainstream paradigm of multimodal learning. We intentionally stuck to well-established and proven techniques for this purpose, using the standard DP-SGD algorithm and Renyi privacy accounting, and the Cap algorithm for image captioning pre-training. By doing so, we hope to establish a strong baseline for DP pre-training on multimodal data, and lay the groundwork for future research by releasing our code/model so that others can build upon our successful training recipe.

Nevertheless, our work does reveal several novel and interesting findings regarding the suitability of image captioning for large-scale DP training, including its ability to scale to massive batch sizes as large as 1.3M, and its synergy with the TAN simulation framework for efficient exploration of training algorithms and hyperparameters. These findings, although not novel in terms of techniques, are crucial for future research since they collectively form a comprehensive strategy for scaling up DP training.

Reviewer comment (Theory):
What is the reason for the *dilemma of large batch training in DP-SGD*, and how does image captioning solve this dilemma? Can you provide any insights?

Author response:
The observation that training with massive batches can deteriorate the performance with DP-SGD [SSS2023] is also present in the non private literature, e.g. with SGD. DP-Cap exhibits robust scaling capabilities concerning batch size, contributing to its success. Our intuition is that a captioner training resembles a language model training, which can usually deal with larger batches. We believe that the same observation could be made for classical (non-private) SGD.

Reviewer comment (Experiments):
Would combining the MAE and image captioning objectives lead to even better performance?

Author response:
We thank the reviewer for this interesting observation.  It would mean performing backpropagation on the sum of these two losses, from image reconstruction and captioning. Intuitively, we would expect that this joint optimization may not be inherently advantageous, given that image representations learned from reconstruction and captioning are not necessarily similar. It is also an interesting question outside of the scope of DP that is worth exploring.

Reviewer comment (Experiments):
I am not an expert in model safety, yet I am wondering that except for DP, is there any other frameworks/methods for overcoming the copyright/privacy issue? It is clear from the manuscript that DP-based models performs much inferior to non-DP-based models, yet it is not clear is there any alternative to DP. If so, why does the authors choose DP-based method at first place?

Author response:
**A2**: This is a great question. As the topic of copyright/privacy is inherently multidisciplinary, the choice of solution must be acceptable not only to the research community, but also in the eyes of law practitioners and regulators. DP stands out in this regard as the only existing solution that is not only provably rigorous and thus scientifically appealing, but also enjoyed adoption as the gold standard for privacy protection by organizations such as the US Census Bureau, Google, Apple, etc. [Des21, CDE+2023]. The goal of our paper is to push the boundary of DP foundation model training and bring this gold standard notion closer to current practice in ML.

We would like to emphasize that even though DP training indeed performs inferior to non-DP training, the latter may be prohibited by law or policy when applied to private data. Thus, it is more appropriate to view the non-DP result as the maximum attainable model performance rather than as a baseline. In practice, however, a portion of the training data may be publicly accessible. In such situations, a natural way to apply our findings is to initialize the model by non-DP training on the public data, then fine-tune with DP on the private data.

[Des21] Damien Desfontaines. A list of real-world uses of differential privacy. https://desfontain.es/privacy/real-world-differential-privacy.html, 10 2021. Ted is writing things (personal blog).

[CDE+2023] Rachel Cummings, Damien Desfontaines, David Evans, et al. Challenges towards the Next Frontier in Privacy. arXiv preprint arXiv:2304.06929.

Reviewer comment (Novelty):
As discussed in the “weakness” part, is there any other promising framework/method that could overcome the copyright/privacy issues of large foundation models? If so, why do the authors choose DP-based methods in this manuscript?

Author response:
**A4**: See answer to **Q2**. There are some alternative privacy-preserving ML notions  but they are not as rigorous and widely accepted as DP. We fully agree with the reviewer that DP has several caveats, but hope that our paper shows promise for its application in large scale settings.

Reviewer comment (Novelty):
I would expect the authors to further clarify the technical novelty of the proposed method.

Author response:
**A5**: See answer to **Q1**.

Reviewer comment (Experiments):
There may be fairness issues in experimental comparisons. For example, the paper states that DP-Cap and ViP were trained using the same training dataset and privacy budget. However, it is worth noting that ViP was trained without utilizing captions (If I understand correctly), while DP-Cap leveraged both images and captions. Consequently, the observed performance improvement of DP-Cap compared to ViP could be attributed to the additional information used during training. While this improved performance may be seen as a contribution of DP-Cap, it is important to consider that the privacy risk might have increased since the model learned caption information.

Author response:
**A1**: $\varepsilon$-DP bounds the amount of information extracted from each training sample by $\varepsilon$. Notably, for DP-Cap, each sample is made of {image + caption}, while ViP utilizes {image} only. Consequently, DP-Cap inherently offers an equivalent or better privacy guarantee for each image. One way to see it is to note that DP provides protection against membership inference attacks [SSS+2017]. Suppose $\varepsilon$-DP upper bounds the success rate of a membership inference attack (when given the image-text pair) against DP-Cap as $\leq p$. Then the MIA success rate when given only the image can be at most $p$ since the attacker has strictly less information. This is exactly the upper bound for the success rate of a membership inference attack against ViP. In other words, any attacker that can attack the {image+caption} models (such as DP-Cap) can also attack the image only models (such as ViP). 

On the other hand, since the {image+caption} models utilize the caption, the privacy leakage from the text part of the image-caption pair is non-zero for $\varepsilon>0$. It is worth noting that in our set up since we use DP, we protect the captions with the same $\varepsilon$-DP guarantee. Thus, the privacy protection for DP-Cap is neither strictly stronger nor strictly weaker than that for ViP, so the two privacy notions are not directly comparable.

We thank the reviewer for this remark, and are adding this precision in the revised manuscript (Section A.1).

Reviewer comment (Experiments):
The effectiveness of the proposed method appears to lack a thorough analysis. In Figure 1(a), the accuracies of DP-Cap are reported as 0.58 and 0.33 under 'random init' and 'syn init' on ImageNet. It is evident that 'syn init' significantly contributes to DP-Cap's performance improvement. However, it's worth considering that 'syn init' is derived from the paper proposing ViP. An important question arises: if both ViP and DP-Cap were not to utilize 'syn init,' what impact would it have on their final results?

Author response:
**A2**: We have observed that training with DP on top of synthetic initialization leads to significantly better results for DP-Cap compared to ViP. To answer the reviewer’s question, this superiority also appears when the models are trained from scratch: the improvement over ViP is even larger when training without syn-init. For instance, when trained from random initialization, ViP reaches 23.9% for full linear probing, while we achieve 47.0%. It naturally underscores the fact that our captioning approach is significantly better for differentially private representation learning for vision data. We acknowledge the significance of this observation. We have incorporated it into the updated paper for clarity, and added more numbers for comparison in Table 8 (Section B).

Reviewer comment (Writing):
Typos. For example, the citation formats for some related papers appear to be incorrect. Is the double line in Table 1 positioned correctly?

Author response:
**A3**: We thank the reviewer for these typos; we are happy to correct the citations. Could you please clarify which specific related work you are referring to? For the double line, we wanted to highlight that ViP and DP-Cap are both private models pretrained on LAION.

[SSS+2017] Reza Shokri, Marco Stronati, Congzheng Song, Vitaly Shmatikov. Membership Inference Attacks against Machine Learning Models. Proceedings of the IEEE Symposium on Security and Privacy, 2017.

Reviewer comment (Novelty):
The novelty of the paper seems not fully justified. For example, in DP guarantee, what is the difference between DP-Cap and ViP? Under what architecture or updating mechanism design, DP-Cap can claim the novelty or difference?

Author response:
**A1**: The novelty of our work lies in building the first differentially private vision-language foundation model, and demonstrating that such models can learn way better private vision encoders compared to previous state-of-the-art DP methods. Regarding the DP guarantee, $\varepsilon$-DP bounds the amount of information extracted from each training sample by $\varepsilon$. Notably, for DP-Cap, each sample is made of {image + caption}, while ViP utilizes {image} only. Consequently, DP-Cap inherently offers an equivalent or better privacy guarantee for each image. One way to see it is to note that DP provides protection against membership inference attacks [SSS+2017]. Suppose $\varepsilon$-DP upper bounds the success rate of a membership inference attack (when given the image-text pair) against DP-Cap as $\leq p$. Then the MIA success rate when given only the image can be at most $p$ since the attacker has strictly less information. This is exactly the upper bound for the success rate of a membership inference attack against ViP. In other words, any attacker that can attack the {image+caption} models (such as DP-Cap) can also attack the image only models (such as ViP). 

On the other hand, since the {image+caption} models utilize the caption, the privacy leakage from the text part of the image-caption pair is non-zero for $\varepsilon>0$. It is worth noting that in our set up since we use DP, we protect the captions with the same $\varepsilon$-DP guarantee. Thus, the privacy protection for DP-Cap is neither strictly stronger nor strictly weaker than that for ViP, so the two privacy notions are not directly comparable.

We thank the reviewer for this remark, and are adding this precision in the revised manuscript (Section A.1).

Reviewer comment (Experiments):
In the context of Point 2, the reviewer observes that the foundation model achieves better performance on some vision tasks compared to the original transformer-based architecture, which has been observed on the ImageNet-1K classification problem. Exactly on this dataset, it should first show the traditional classification task compared to other DP methods, and then move to the few-shot setting.

Author response:
**A3**: We acknowledge the reviewers' insightful suggestion to bolster the demonstration of DP-Cap's superiority through additional evaluations. In response, we have incorporated fine-tuning experiments into the revised version, Table 9 in Section B.1. As shown in the updated results, DP-Cap improves over ViP on few-shot fine-tuning tasks. 

Addressing the concern about the perceived low numbers in few-shot learning, it's crucial to underscore that these figures hold significant importance for benchmarking quality of learned image representations in the self-supervised learning literature, e.g., [ACM+2022]. While the results in Figure 1(a) and Table 4 may appear modest compared with non-private baseline methods, they represent a noteworthy stride towards closing the performance gap with the non private representations as well as improving existing state-of-the-art DP vision foundation models (ViPs). Note that, as detailed in the general comment, we have fixed a bug in the loss scaling which has considerably improved the performance of the vision encoder across all benchmarks. For instance, we now achieve a performance of 63.4% on ImageNet full linear probing—a nearly 8% improvement over ViP.

Reviewer comment (Experiments):
Currently, only reporting the ImageNet-1K on few-shot setting with very low numbers (e.g., in Table 3) cannot lead to any positive conclusion.

Author response:
**A3**: We acknowledge the reviewers' insightful suggestion to bolster the demonstration of DP-Cap's superiority through additional evaluations. In response, we have incorporated fine-tuning experiments into the revised version, Table 9 in Section B.1. As shown in the updated results, DP-Cap improves over ViP on few-shot fine-tuning tasks. 

Addressing the concern about the perceived low numbers in few-shot learning, it's crucial to underscore that these figures hold significant importance for benchmarking quality of learned image representations in the self-supervised learning literature, e.g., [ACM+2022]. While the results in Figure 1(a) and Table 4 may appear modest compared with non-private baseline methods, they represent a noteworthy stride towards closing the performance gap with the non private representations as well as improving existing state-of-the-art DP vision foundation models (ViPs). Note that, as detailed in the general comment, we have fixed a bug in the loss scaling which has considerably improved the performance of the vision encoder across all benchmarks. For instance, we now achieve a performance of 63.4% on ImageNet full linear probing—a nearly 8% improvement over ViP.

Reviewer comment (Experiments):
One consideration is to replicate the experimental settings from ViP, e.g., DP fine-tuning evaluation on ImageNet-1K, experiments on Caltech-101 and CIFAR-100, which can provide direct comparison to ViP and other DP methods, as appeared in the ViP paper. Otherwise, the paper will fail in setting consistent comparison to the literature.

Author response:
Dear reviewer, please find bellow the markdown version of the fine-tuning evaluation you have asked for.  Note that the table was placed in appendix B.1.

| Model      |           | AirCraft  |           |           | Caltech-101 |           |           | Cifar-100 |           |
|------------|-----------|-----------|-----------|-----------|-------------|-----------|-----------|-----------|-----------|
|            | 10-shot   | 20-shot   | 30-shot   | 5-shot    | 10-shot     | 30-shot   | 5-shot    | 10-shot   | 30-shot   |
| AlexNet    | 23.3%     | 34.4%     | 41.4%     | 64.7%     | 73.6%       | 81.4%     | 29.7%     | 36.3%     | 49.3%     |
| SimCLR     | 38.8%     | 56.9%     | 64.9%     | 81.7%     | 89.1%       | 94.5%     | 49.9%     | 60.2%     | 71.8%     |
| TAN        | 22.8%     | 37.9%     | 46.0%     | 49.3%     | 66.4%       | 77.9%     | 21.3%     | 27.8%     | 42.4%     |
| ViP        | 31.6%     | 53.1%     | 64.3%     | 68.1%     | 79.0%       | 88.9%     | 30.7%     | 41.0%     | 57.5%     |
|------------|-----------|-----------|-----------|-----------|-------------|-----------|-----------|-----------|-----------|
| **DP-Cap** | **37.5%** | **57.9%** | **66.7%** | **70.3%** | **81.3%**   | **90.0%** | **36.3%** | **46.3%** | **62.1%** |

The choice for the models is the same as for table1: AlexNet and SinCLR are non private, while TAN/ViP are also trained under $\varepsilon=8$.

We would be very happy to answer any additional question that the reviewer might have.

Reviewer comment (Novelty):
The main concern comes from the method novelty and the experimental settings. For details, please refer to the weakness section.

Author response:
**A2**: The vast majority of existing work on DP training operates on a single modality, be it image classification, image-only SSL, text classification, language modeling, etc. Our work is the first to push beyond this barrier and show that DP can also be successfully applied to the currently mainstream paradigm of multimodal learning. We intentionally stuck to well-established and proven techniques for this purpose, using the standard DP-SGD algorithm and Renyi privacy accounting, and the Cap algorithm for image captioning pre-training. By doing so, we hope to establish a strong baseline for DP pre-training on multimodal data, and lay the groundwork for future research by releasing our code/model so that others can build upon our successful training recipe.

Nevertheless, our work does reveal several novel and interesting findings regarding the suitability of image captioning for large-scale DP training, including its ability to scale to massive batch sizes as large as 1.3M, and its synergy with the TAN simulation framework for efficient exploration of training algorithms and hyperparameters. These findings, although not novel in terms of techniques, are crucial for future research since they collectively form a comprehensive strategy for scaling up DP training.