Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: This paper focus on a important scenario to consider differential privacy and model compression together. It considers KD and (structured and unstructed) pruning. The paper is generally well-written.
strengths_and_weaknesses: Strengths:
- it is interesting to consider DP and compression together.
- consider both KD and (structured and unstructed)  pruning.
- the paper is well-written

Weaknesses:
- some statements are slightly over-claimed
- the methods seems not that novel. For example, DPKD is like a pipline to bring many existed methods together.
- Sec. 3.4 and 3.5 do not introduce anything beneficial, but something that has nearly idential results -- strategies to make better students/teachers.


questions: In the abstract, the authors state "we initiate the study of differentially private model compression and propose frameworks for achieving 50% sparsity levels while **maintaining nearly full performance**". From Tab. 2/3/4/5, it seems it does not achieve as well as the authors claim.  A relatively better result is achieved by unstructured DPIMP, which has 83.5 vs. 85.2 (raw BERT) in AVG (there is still a tiny gap); however, unstructured pruning with 50% sparsity usually has a limited speed-up during inferencing. The above statement seems slightly overclaimed.

The paper seems to report relatively weak results in GLUE benchmark, not only for the proposed method but also for existing work. By checking other papers (e.g. DistilBERT https://arxiv.org/pdf/1910.01108.pdf or even a stronger one  TinyBERT  https://arxiv.org/abs/1909.10351), we could observe that the reported results for BERT and DistilBERT are lower than original papers. It is unclear whether the results are evaluated in DEV or TEST set. Could you please provide more details? Frankly speaking, with half the size of BERT-base parameters, such achieved performance might not be competitive with some models without pretraining (which might be much faster).
limitations: none
review_title: null

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: - The authors focused on compressing LLM in a differentially private way. Previous works mostly focused on differentially private fine-tuning but not the compression parts.
- The proposed method achieves very similar performance with 50% sparse model compression and guaranteeing differential privacy.
- The authors showed the limitation of straightforward ways of applying DPSGD to Knowledge Distillation and proposed the improved version of it to reduce the accuracy losses.
strengths_and_weaknesses: Strength:
- The authors tackled an important but under-explored problem: DP for model compression in LLM.
- The paper is well-written and easy to follow.

Weakness:
- The experiments are limited. It would be good if the authors provide more diverse experiments with varying compression rates.
- Epsilon value is set to be too high. With epsilon < 4.25, I am not sure whether the achieved model is private enough.
- The advantages from DistBERT is not convincing. Comparing with unstructured DPIMP is not fair.
questions: 1. Compression rate
- In the paper, the authors only show the experimental results with 50% sparsity.
- However, in many applications, we need much higher compression rates to increase the latency and reduce the costs.
- It would be great if the authors can provide more results with a diverse range of compression rates.

2. Epsilon value
- Based on the differential privacy definition (Section 2), epsilon determines the maximum differences between two probabilities with and without a single sample.
- If we select the epsilon value as 4 (which is < 4.25), the difference ratio is exp(4) = 55.
- In other words, the maximum probability differences would be significantly large.
- In this paper, most results are based on the epsilon value < 4.25. But I think this would be a too loose threshold. 
- It would be good if the authors provide tighter privacy thresholds and the corresponding performances.

3. DistBERT
- Even though DistBERT is computationally expensive, we only need one time for training the DistBERT (because it is KD on public data). 
- Then, we can use the DistBERT for various downstream tasks.
- In that case, I think the computational complexity of the DistBERT does not matter a lot.
- At the end, the objective is to achieve the differentially private compressed language models and the objectives seem the same between DistBERT and the proposed method.
- In that point of view, the advantages of the proposed method in comparison to DistBERT are limited.

4. Structured DPIMP vs Unstructured DPIMP
- It would be good to show the quantitative inference latency comparison between these two methods.
- Also, directly comparing between unstructured DPIMP and DistBERT is not the fair way.
- Actually, based on Table 4, DistBERT seems consistently better than Structured DPIMP. 
limitations: - As the authors said in the introduction, the proposed method is not limited to the NLP.
- In that case, it would be good to show how well those compression methods (with DP) can be applicable to other domains in the appendix.
- Also, the authors do not consider the PATE framework for model compression. However, in some applications, PATE works better than DP-SGD. To show the generalization of the proposed method, it would be better to discuss and consider PATE as well.
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: A commonly used technique in practical NLP systems is that of model distillation (or compression or sparsification). This technique converts a Large Language Model (LLM) into a more lightweight/small model so as to make the prediction faster and memory efficient.

On the other hand, Differentially Private (DP) training has recently started to become a standard notion of safety against various adversarial model attacks. 

This work lies at the intersection of the above two concepts, in that it introduces methods to obtain compressed student models, from large teacher models, that have differential privacy guarantees. To that end, the paper investigates the following approaches:

1) Obtaining student models via a distillation loss function. Here DP is introduced possibly in the fine-tuning steps of teacher and students models. 

2) Transforming teacher models into student models via Iterative Magnitude Pruning (IMP). Here DP is introduced in the fine-tuning steps.
strengths_and_weaknesses: Strengths:

1) The setup and motivation is practically quite relevant. This is because both distillation and differential privacy are critical in large-scale NLP systems.

2) It studies two distinct compression methods: one based on the distillation loss function and the other based on weight pruning. 

Weaknesses:

1) The DP guarantees for pruning based methods are entirely missing. For instance, the output of algorithm 2 is said to be a "private student model" but one doesn't know what the epsilon-privacy guarantee is. This makes the algorithm practically useless since the user doesn't have a calibration of privacy gains. 

2) The experimental results are limited only to BERT and 1/2-BERT. It would be interesting to see if their results generalize to other architectures. 

3) The technical contribution to introduce DP to model compression is a fairly straightforward application of existing tools and doesn't involve significant novelty.
questions: 1) A main motivation for introducing DPKD is mentioned in lines 118-121. While that explanation with regards to the KD loss function makes intuitive sense, is it possible to formalize it properly? The text in those lines is used to justify the introduction of DPKD but it is quite hand-wavy and doesn't cite any previous work for support as well.

2) In Algorithm 1, line 3, what does it mean to initialize the student model with a privacy budget? How can just an initialization step have a privacy guarantee? This is also not clear from the experiments as there only the teacher and student model training steps involve DP.

3) What is the motivation for choosing privacy parameters, epsilon and delta, as shown in line 144? Calibration of these quantities is a problem in current literature as we don't know which values are good, so the current choice seems arbitrary. 

4) As mentioned in the weaknesses, the DP guarantees for DPIMP methods are missing. Can we provide epsilon, delta values for the models resulting from those algorithms?

5) Related to the PATE works, there is also very recent work on introducing DP to LLMs at the prediction stage (see recent works such as https://arxiv.org/abs/2201.00971 and https://arxiv.org/abs/2205.13621). It will be helpful to add a few lines regarding these to improve the quality of presentation. 

Minor typos:

1) Phrase "for initialization compressed models" in line 54.

2) "show" should be replaced with "shows" in line 55.

3) Phrase "opens a whole in direction" in line 295.
limitations: Yes.
review_title: null

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: This paper explores the links between differential privacy and model compression. Authors propose a framework for training NLP models with high utility by leveraging DP-SGD and knowledge distillation.
This is an experimental paper.
strengths_and_weaknesses: The work is rather easy to follow, with clear motivations of why this technique should be explored, how the scientific community can benefit from private compressed models etc. The method that authors propose seems sound and can prove useful for a broad scientific community. The way the manuscript is structured really aids its interpretability: the motivation is followed by a naive implementation, followed by the comments on why such method does not perform well, followed by another iteration with limitations addressed etc. This in my view makes the paper much friendlier to larger audience.
Authors provide a number of evaluations of their technique in comparison to its non-private counterpart. What I find really commendable is the fact that authors report the results which were not entirely positive and demonstrated their understanding of the limitations of their method. The overall results of compressed models being just slightly behind the non-private counterparts look rather promising.

However, this work contains a number of shortcomings that I think that authors need to address. 
Firstly, authors reject the idea of using PATE in their evaluation as DP-SGD has better performance. I am not entirely sure A) why is this a valid metric to reject the work that is the closest DP student-teacher implementation to this manuscript and B) I am not even sure that this statement is necessarily true [4,5]. It seems to me that PATE was avoided by the authors without a strong reason behind it, making me question some parts of the evaluation of the proposed method. 
Secondly, I am not convinced that a combination of DP and KD is a particularly novel method. In addition to the aforementioned PATE, I also found a number of works such as [1,2,3] (in fact [1] even shares the name of the framework with this study), which seem to have addressed this issue before in detail (some of which were peer-reviewed previously). So what exactly is the novel contribution of this work which was not previously proposed? Because on a high level, while the compression seems like a fairly useful trait (and a promising result), combining DP-SGD with KD is fairly trivial and has been done before.
One other comment I have is the argument on the 'better models': for DP training a 'bigger' model does not necessarily outperform a smaller one, so I am not sure such argument holds [6]. 
Minor: inconsistencies with DP-SGD (the original version) and DPSGD.

[1] Lyu, Lingjuan, and Chi-Hua Chen. "Differentially private knowledge distillation for mobile analytics." Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval. 2020.
[2] Sun, Lichao, and Lingjuan Lyu. "Federated model distillation with noise-free differential privacy." arXiv preprint arXiv:2009.05537 (2020).
[3 ]Wang, Ji, et al. "Private model compression via knowledge distillation." Proceedings of the AAAI Conference on Artificial Intelligence. Vol. 33. No. 01. 2019.
[4] Papernot, Nicolas, et al. "Scalable private learning with pate." arXiv preprint arXiv:1802.08908 (2018).
[5] Uniyal, Archit, et al. "DP-SGD vs PATE: Which Has Less Disparate Impact on Model Accuracy?." arXiv preprint arXiv:2106.12576 (2021).
[6] Klause, Helena, et al. "Differentially private training of residual networks with scale normalisation." arXiv preprint arXiv:2203.00324 (2022).
questions: A number of questions arose during my evaluation of this work. I would like the authors to either point me to the section where I can find the necessary details or to provide an answer outside of the manuscript space:
- What dataset does a teacher control? There was a discussion about PATE needing a disjoint dataset, which may be unrealistic, but I did not find any mention of what are the data assumptions on the teacher model in this work? Moreover, could the authors elaborate why is a disjoint setup considered unrealistic in this setting?
- I am not entirely sure I understand section 2.1: what is the meaning of the 'sophisticate argument(s)'? It strikes me as if the authors included a list of words associated with DP without really explaining them or even linking them together.
- It was unclear to me why in section 3.2 DP-SGD was insufficient? I suspect this links to my qn on data assumptions for the teacher mode and I am unsure whether a teacher model is required to be private or not, so would you kindly clarify?
- In the experiments an epsilon of < 4.25 was chosen: what is the motivation behind it? Why this specific value and how does it relate to prior literature? If I overlooked it, then please point me to a section/reference.
limitations: In addition to the limitations outlined by authors, my comments are largely explained above and include: Similar prior work, constraints on the data of the teacher model etc.
While overall, the work shows some interesting results and is relatively easy to read, I cannot recommend acceptance until points above are addressed by the authors (particularly wrt novelty and comparison with prior works).
review_title: null

Meta-review: metareview: This work proposes and empirically evaluates algorithms for compressing and fine-tuning a large model for a downstream task, while satisfying DP for the downstream task training data. The set up is the following: we have a large pre-trained language model such as BERT. We would like fine-tune it for a task using a dataset D, as well as compress it to a smaller model. The paper studies algorithms that are DP with respect to D and do fine-tuning+compression. The authors propose and evaluate different strategies for this problem and compare the privacy-utility tradeoffs.
The reviewers found the empirical evaluation to be thorough. Some of the other concerns raised by the reviewers have been addressed to my (and in most cases, their) satisfaction.
I think the problem studied by the paper is timely and important. I view the paper largely as a solid empirical study of natural algorithms for this problem. While the paper can be improved as discussed in the reviews and rebuttal, I believe it brings attention to an important problem and makes solid progress on it. I would therefore recommend acceptance.
confidence: Certain
Decision: Accept