Venue: ICLR 2022 Conference

Reviews:

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: The paper propose a faster algorithm to learn approximate differentially private NLP models. Pretrained NLP models are often very large. Practical procedure involves fine-tuning NLP model on private data, which may leak private information. To avoid leakage, DP-SGD (and DP-AGAGRAD, DP-ADAM) uses norm clipping on each sample’s gradient, and then add isotropic noise to aggregated gradients for samples in a batch. Then the normal update steps of SGD, ADAGRAD, or ADAM are performed. This will ensure privacy under differential privacy definition. However, this procedure requires computing per-sample gradient and keep them in the memory so that the norm of the gradient can be calculated, and the per-sample gradient clipping can be performed. This will introduce memory overhead proportional to the batch_size$\times$#params, which is impossible for very large NLP models. 

This paper propose GhostClipping method to save the memory, without the need of per-sample gradient instantiated. The idea is compute the partial sum of gradient element-wise square using two small matrices of the size of $T\times T$, where $T$ is sequence length (<=1024 in practice), and aggregate them to obtain the per-sample norm (just a scale for each). And then it performs a second back-propagation to compute aggregation on the clipped gradient. It will uses almost the same memory as standard SGD (or ADAM), with one forward pass and two backward passes. 

This paper also introduce two additional techniques to improve, one is choose a larger batch size, and the other is to introduce multi-task finetuning (fine-tuning includes masked prediction task on the target dataset). These two are important in boost the performance of the final models. 

The paper evaluates on text classification, data-to-text generation, and dialog generation tasks. The results shows it gains prior DP methods. 




main_review: Strength of the paper:
1. The paper is clearly written and easy to understand. It presents the DP training problem very well and demonstrates the challenge for large pretrained NLP models. 
2. The proposed GhostClipping is effective in reduce memory consumption, at (not too big) cost of an extra backpropagation. The method is very clear and also simple to implement. 
3. The experiments show strong results on both classification and text generation tasks, better than the prior DP methods in terms of memory consumption and accuracy/generation quality. 

Weakness of the paper:
1. There are three aspects involved. GhostClipping, large batch/proper learning rate, fine-tuning with masked prediction. It seems the accuracy on text classification comes from the later two. Table 1, comparing full(RoBERTa-large) with RGP(RoBERTa-large) does not show improvement (please list the average score here). Full is actually worse. If RGP is augmented with larger batch/proper learning rate, and masked prediction for fine-tuning, it may be the best in terms of accuracy. 
2. Figure 1 is a bit mis-leading. Please show and compare with RoBERTa (non-private) on 1(a) and non-private GPT-2 on 1(b). Those are the baselines to compare. Otherwise you are comparing private RoBERTa with non-private BERT or non-private T-GEN(non Transformer). Statement in the abstract and introduction are over-stated. 
3. The organization of the paper could be improved. Ghost-Cliping is your main method, which could be moved earlier. 

summary_of_the_review: Reason to accept:
Simple DP algorithm with demonstrated reduction in memory consumption and effectiveness. It enables large pre-trained NLP models to be further fine-tuned on private data under approximate DP. Clear presentation. 

Reason to reject:
Better to identify the cause of improvement and make a fair comparison to baseline models (in particular for text classification tasks). Over-statement and misleading in abstract and introduction about comparable to strong non-private baselines.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: This paper investigated the problem of privately fine-tuning large language models for downstream NLP tasks, including sentence classification and language generation. The authors showed that by appropriately selecting hyper-parameters (including batch size, learning rate, training epochs, and clipping norm) and making the fine-tuning task aligned with pretraining tasks, directly fine-tuning large language models with DP-SGD yields strong performance, and provided an empirical guideline for setting a good training configuration. The authors also proposed ghost clipping trick for further memory saving when fine-tuning large language models. Finally the authors showed through experiments that low dimensional updates do not necessarily lead to better performance.
main_review: Overall, this is a fairly empirical paper on an important problem and shows good performance. The authors presented a set of thorough experiments investigating the impact of various hyper-parameters, which are widely known as sensitive and difficult to tune, providing a nice and informative guidelines for other researchers and practitioners who work in this area. The ghost clipping trick proposed in this paper for memory reducing in DP-SGD is simple yet effective, greatly reducing the memory cost when applying DP-SGD to large models, especially the popular large-scale pre-trained language models, potentially encouraging more research effort in the DP learning area.

The experiments presented in this paper are quite solid and clear, both well-designed and documented, and most of the claims made in the paper are reasonable and well-supported. My only complaint is that I wish to see more explanations and more principled approach to selecting the hyper-parameters in fine-tuning with DP-SGD, but I guess it is out of this paper's scope (the authors did provide some empirical discussions in the paper and the appendix, which I appreciate).
summary_of_the_review: This paper presented a detailed investigation on the training configurations of fine-tuning large language models with DP-SGD through a set of extensive experiments, which are beneficial to researchers and practitioners working on private NLP, and the proposed ghost clipping trick for DP-SGD greatly reduces memory when applied to large language models, making private NLP research more feasible. I believe both the insights and the proposed method in this paper will bring value to the private NLP community.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper adapts the widely used DP learning algorithm, DP-SGD, to language models. It achieves to fine-tune the dataset while protecting the private information in the dataset. In this paper, the authors conduct some empirical studies on language models and find some useful conclusions (e.g. fine-tuning on a part of parameters with DP is enough). The authors verify the model on sentence classification, table-to-text generation, and dialog generation tasks, using various pre-trained language models (e.g. GPT, Bert).
main_review: Pros,
1) It achieves a remarkable performance of DP algorithms on NLP tasks with a satisfactory level of privacy protection.
2) There will be a number of potential applications since this paper verify the model on multiple NLU and NLG tasks.
3) It works on several pre-trained language models (e.g. GPT, BERT, Roberta).
4) The experiments part is quite solid and covers many necessary details.

Cons,
1) The experimental results in table 1~3 compared to the non-private seems amazing, however, the comparison with the baseline methods is also required, which would show the contribution of your proposed methods. There're several possible baselines: directly apply DP-SGD and an application of DP-SGD on language models (GPT-2) [1].

2) The choice of \epsilon makes sense because the \epsilon in a range of 0.1~5 provides meaningful protection. However, the choice of \delta in experiments may cause some problems in privacy leakage. The value of \delta on the order of 1/|D_train| is very dangerous according to [2]. In that level of \delta, they permit “preserving privacy” by publishing the complete records of a small number of database participants. It's better to choose the \delta as 1/( |D_train| *100), which may reduce the utility but ensure the privacy is well protected. 

3) Will the ghost clipping increases the utility or privacy protection? Could you please provide some insights (or even quantitative analysis) about it?

4) It would be better to conduct some key experiments with a range of \espilon from 0.1 to 10 (e.g. 0.1, 0.5, 1, 2, 3, 5, 8). Many DP learning algorithms use a curve to show the variation tendency of utilities for the different \epsilon.

5) This paper did a good job on the empirical study, but the technical novelty is limited.

[1] Differentially Private Language Models Benefit from Public Pre-training. 2020

[2] The Algorithmic Foundations of Differential Privacy, 2014


After rebuttal:
Thanks for your explanation! I believe it's a good paper and I'm happy to see its acceptance.

summary_of_the_review: The paper has a clear motivation and the main idea is also interesting, but still suffers from some issues.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: In this paper, the authors propose a method that applies DP-SGD to NLP tasks. DP-SGD protects the privacy of the model training against that the individual information about the training samples is detected or inferred. The method is applied to the fine-tuning phase of the pre-trained language models (e.g. bert, gpt), thus it achieves good performances for many applications. To adapt DP-SGD to NLP models, this paper proposes ghost clipping that allows clipping in DP-SGD to run without instantiating per-example gradients for any layer in the model.
main_review: Strengths:
1. As a DP algorithm on a deep learning model, this algorithm achieves good performance on fine-tuning the language models and several real-world applications.
2. The experimental results are sufficient and solid. 
3. The performance gap between the proposed method and the non-private method is quite small, which shows applying the proposed DP algorithm would not decrease the utility so much.
4. Some of the conclusions from the experiments seem to be adaptive to other applications. For example, the relations among parameter size (for fine-tuning), privacy leakage, and performance.

Weaknesses:
1. The \delta in (\epsilon, \detla)-DP is too large for training a DP model with a meaningful level of privacy protection. Normally, \detla should be much smaller than the inversed dataset size (cannot be in the same scale of the inversed dataset size), some researchers choose \detla = 1/(|D| * log(|D|)), where |D| stands for the dataset size.

2. It would be better to conduct an ablation study to verify the promotion of the proposed methods (ghost clipping technique, fine-tuning on only a part of the parameters, and DP-adam). It's better to make it clear how much does each strategy contribute.

3. In this paper, the authors introduce a lot about the selection of hyper-parameters, thus hyper-parameters play an important role in model training. However, tuning hyper-parameters requires additional information about the private information (accessing the validation set or testing set), which leads to private leakage. So, how to count the information leakages of users? How to avoid those private leakages.

4. Is the DP-Adam proposed by yourself? If not, the corresponding reference and explanation are needed. Does it come from [Wang 2019]? If it is proposed by yourself, I suggest the author make it clear as it's one of the contributions. 

[Wang 2019] DP-LSSGD: An Optimization Method to Lift the Utility in Privacy-Preserving ERM.
summary_of_the_review: The paper is quite good on experiments and their possible applications but faces some questions.
review_title: null

Meta-review: This work adapts the widely used DP learning algorithm to language models. Reviewers all agreed that this work tackles an important problem with clear motivation and thorough experiments, and achieved strong performance (memory reduction and effectiveness) on NLP tasks.  Thus, we recommend an acceptance.
Decision: Accept (Oral)