Differentially Private Fine-tuning of Language Models
Authors: Da Yu
Saurabh Naik
Arturs Backurs
Sivakanth Gopi
Huseyin A Inan
Gautam Kamath
Janardhan Kulkarni
Yin Tat Lee
Andre Manoel
Lukas Wutschitz
Sergey Yekhanin
Huishuai Zhang
Venue: ICLR 2022 Poster (2022)
Keywords: differential privacy
large language models
fine-tuning

Abstract: We give simpler, sparser, and faster algorithms for differentially private fine-tuning of large-scale pre-trained language models, which achieve the state-of-the-art privacy versus utility tradeoffs on many standard NLP tasks. We propose a meta-framework for this problem, inspired by the recent success of highly parameter-efficient methods for fine-tuning. Our experiments show that differentially private adaptations of these approaches outperform previous private algorithms in three important dimensions: utility, privacy, and the computational and memory cost of private training. On many commonly studied datasets, the utility of private models approaches that of non-private models. For example, on the MNLI dataset we achieve an accuracy of $87.8\%$ using RoBERTa-Large and $83.5\%$ using RoBERTa-Base with a privacy budget of $\epsilon = 6.7$. In comparison, absent privacy constraints, RoBERTa-Large achieves an accuracy of $90.2\%$. Our findings are similar for natural language generation when privately fine-tuning GPT-2. Our experiments also show that larger models are better suited for private fine-tuning: while they are well known to achieve superior accuracy non-privately, we find that they also better maintain their accuracy when privacy is introduced.

Published as a conference paper at ICLR 2022
DIFFERENTIALLY PRIVATE FINE-TUNING OF LANGUAGE MODELS
Da Yu Saurabh Naik Arturs Backurs Sivakanth Gopi Huseyin A. Inan 3.* Gautam Kamath S.* Janardhan Kulkarni Yin Tat Lee Andre Manoel Lukas Wutschitz A.* Sergey Yekhanin Huishuai Zhang
'Sun Yat-sen University Miccosooft Research Asia 3 *Microsoft Research, "Microsoft
CCheriton School of Computer Science, University of Waterloo
University of Washington
"yuda38 (adda38maa1...pp edu "nuzhanggnnlcrosottt.com a ((arturs. backurs,, sigopi, Lnseyyhn.Lanyeenccrooooff com
(jakul, amenteircman, wekhaninn/miccrooftt.o (snaik, lukas. mutschitta)miicrosokt com
"gecsail.ml edu, eyintat@uw.edu
ABSTRACT
We give simpler. sparser, and faster algorithms for differentially private fine- tuning of large-scale pre-trained language models, which achieve the state-of- the-art privacy versus utility tradeoffss on many standard NLP tasks. We pro- pose meta-framework for this problem, inspired by the recent success highly parameter-efficient methods for fine-tuning. Our experiments show that differen- tially private adaptations of these approaches outperform previous private algo- rithms in three important dimensions: utility, privacy, and the computational and memory cost of private training. On many commonly studied datasets, the util- ity of private models approaches that of non-private models. For example, the MNLI dataset we achieve an accuracy of 87.8% using RoBERTa--Lage and 83.5% using RoBERTa-Base with a privacy budget of 6.7. In comparison, absent privacy constraints, RoBERTa-Large achieves an accuracy of 90.2%. Our findings are similar for natural language generation when privately fine-tuning GPT-2. Our experiments also show that larger models are better suited for private the-tuning: while they are well known achieve superior accuracy non-prin non-privately, we find that they also better maintain their accuracy when privacy is introduced.
INTRODUCTION
Deep learning models are well known to leak sensitive information about the dataset when trained using conventional methods (Shokri et al., 2017; Carlini et al., 2019; 2021). To combat this issue, models can instead be trained to guarantee differential privacy (DP) (Dwork al., 2006b), strong notion of data privacy which limits the influence of any individual training point on the final model. While DP is one of the few approaches capable of providing machine learning models with rigorous privacy guarantees, it generally comes at cost in terms of test accuracy. One oft-cited explanation that the constraint DP necessitates much more training data (Tramer & Boneh, 2021; Feldman, 2020: Brown et al., 2021). Unfortunately, more training data may be hard to acquire, particularly in settings where privacy is a concern.
Parallel to these developments, Transformer-based (Vaswani et al., 2017) large language models (LLMs), including the BERT (Devlin et al., 2019; Liu et al., 2019) and GPT (Radford et al., 2018; 2019; Brown et al., 2020) families, have enabled significant progress in natural language process- ing, achieving state-of-the-art accuracy in almost every task considered. These models are first pre-trained
extremely large and diverse public dataset. The weights are then fine-tuned for
Authors contribute equally to this work and are listed in alphabetical order.
I The work of Da Yu is done while he an intern Microsoft Research Asia.

---

Published as a conference paper at ICLR 2022
Pre-train
DP-tune
Plug-in
DP tuned
Visiale to adversary
paramesiass
New perameters
Pre trained parsmutors
large inggung Medul
Targe anggunge Mudel
Large La Language Model
optimiter
Private optimieer
public www.cliccieeeeoo decinee
Frisent
Figure I: An illustration of our framework. First, the model is pre-trained on large, public dataset. Next, new parameters are introduced and privately fine-tuned on a smaller, private, task specifc dataset. The original parameters are frozen during this process. Finally, the fine-tuned new pa- rameters may be released publicly and plugged-in to the model for downstream tasks, while still preserving privacy of the private dataset.
Table I: Accuracy of fine-tuning for downstream tasks with RoBERTa-Large (in %). Our results achieve accuracy comparable to full fine-tuning non-privately, while simultaneously guaranteeing differential privacy. We choose =le-5 for SST-2 and QNLI and - for MNLI and QQP due their dataset sizes. Implementation details in Section 4.1.
Method
MNLI
SST-2
QQP
QNLI
Avg.
Trained params
Non-private finc-tuning results 6.7)
90.2
96.4
92.2
94.7
93.4
100%
87,8
95.3
87.4
90.8
90.3
0.94%
each task of interest using much smaller task-specific dataset. For example, single pre-trained GPT-family model may be fine-tuned for various downstream tasks, such as email reply suggestion, sentence completion in text editors, language translation, and more. This two-stage paradigm can naturally be adapted to solve tasks in private learning, automatically addressing the aforementioned data shortage issue via the massive scale of the public pre-training dataset. One may pre-train the model on public data usual," but then fine-tune the model privately.
Despite the success these models, task-specifie fine-tuning introduces a number of technical chal- lenges. In the non-private setting, the immense size of LLMs makes it impractical to fine-tune the full model and store separate copy of the parameters for hundreds of downstream tasks. Things only get worse with privacy, which leads to overheads in terms of rumning time, memory usage, and most importantly, accuracy. The magnitude of noise introduced to a model due to DP grows the model size increases (Bassily et al., 2014; Abadi al., 2016; Bun al., 2014), which can overwhelm any signal for larger models. A recent line of work in the non-private literature has proposed parameter-effiiiet methods alleviate the issues of storage and computational cost for fine-tuning (Houlsby et al., 2019; Li & Liang, 2021; Aghajanyan et al., 2020; Hu et al., 2021; Ma- habadi et al., 2021). The main focus of our work is to explore parameter-effiiinncy in the context of private learning.
1.1 OUR CONTRIBUTIONS
Our primary contribution is to show that advanced parameter-efficient methods can lead to sim- pler and significantly improved algorithms for private fine-tuning. Our framework is illustrated in Figure 1. Our findings and contributions are summarized as follows:
State-of-the-art utility and privacy. Empirical evaluation of our algorithms reveals that they achieve state-of-the-art accuracy versus privacy tradcoff's improving upon the previous best (Yu
Despite the fact that the pre-training data is public, there may nonetheless be privacy concerns related personal or copyrighted data. However, since these pre-trained models have already been released, any associated privacy loss has already been incurred.
2

---

Published as a conference paper at ICLR 2022
al., 2021b). More importantly, for many fine-tuning tasks, the utility of models trained with DP approaches that of non-private models. For example, privately fine-tuning RoBERTa-Large on the MNLI data set (Williams et al., 2018), achieve accuracy of 87.8% with a privacy budget of achieves an accuracy of 90.2% (GPT-3 is known to achieve 91.7% (Hu et al., 2021)); see Table 1 for summary. We also explore GPT-2 models on the E2E dataset (Novikova al., 2017). Again, the utility approaches non-private levels: we achieve a ROUGE-L score of 0.6755 with GPT-2-Large and (E 5.4, 5 = le-5), compared to 0.72 without privacy.
(s ===00 6.7, = 1e-6). Without privacy guarantees, RoBERTa-Large
private natural language generation tasks,
Larger models are better. Prior work has consistently shown that larger language models achieve better accuracy for downstream tasks. Our results give evidence that this phenomenon extends to the private setting. For example, on the MNLI dataset, RoBERTa-Be Base achieves an accuracy of 83.5% whereas RoBERTa-Layg achieves an accuracy of 87.8%, both under privacy budget of (s 6.7.6 = 1)-6). Similarly, privately fine-tuning with E2E, GPT-2-Small, GPT-2-Medium, and GPT-2-Large achieve ROUGE-L scores of 0.6219, 0.6645 and 0.6755 respectively, all under a privacy budget of (= = 5.4, = le-5). While established in the non-private setting, we find this phenomenon quite surprising under DP. There is often tension when choosing private model architectures: larger models may have higher capacity, but necessitate the introduction of more noise. Consequently, smaller and simpler private models achieve the better accuracy in several settings (Papernot et al., 2019; Tramèr & Boneh, 2021). In contrast, our experiments show that fine-tuning the biggest models achieves the best accuracy.
Simpler, sparser, and faster. Beyond accuracy concers, DP requirements also lead to significant overheads in terms of computation and memory usage. The large number of parameters contributes to the high cost of training LLMs, and things get worse under privacy, which has been documented to increase training time by up to two orders of magnitude (Carlini al., 2019; Subramani al., 2021). The garameter-efficiee approaches we employ partially offset this issue: as we only update a small fraction of the total number of parameters, training becomes considerably more computationally and memory efficient. Furthermore, as in the non-private setting, this framework leads to a modular design, where a single large pre-trained model can augmented with lightweight modifications for each individual downstream task.
To the best of our knowledge, we are the first to fine-tune GPT-2-Large using differential privacy, the largest model trained thus far using DP. Given our state-of-the-an results for a variety of stan- dard NLP tasks using advanced fine-tuning techniques, we believe that our paper will serve as a benchmark for further work in this direction. For example, the best average accuracy achieved by the prior work of Yu et al. (2021b) on four standard NLP tasks in Table is 83.9% using & = 8 (and the same 6 as in Table 1), whereas we can achieve an average accuracy of 90.3% using & 6.7 by combination of better algorithms, larger models, and new privacy accounting techniques.
Finally, though recently considered elsewhere (see Section 5), we put further focus the framing of public pre-training and private fine-tuning as an important conceptual direction in DP deep learning.
PRELIMINARIES AND PRIOR ALGORITHM BASELINES
Recall the formal definition of differential privacy.
Definition 2.1 (Differential Privacy (DP) (Dwork al., 2006b;a)). A randomized algorithm A is (e.fediiffrrentially private if for any two neighboring datasets D and D', which dif- fer in exactly the data pertaining to a single user, and for all sets of possible outputs: Pr[A(D) E S e Pr(A(D') E S +
We review prior techniques for private fine-tuning.
2 An alternative perspective is that what we currently think of as "large" language models are relatively small, and we yet to reach the point where the benefits of model size accuracy are outweighed by the drawbacks.
3

---

Published as a conference paper at ICLR 2022
2.1 FULL FINE-TUNING VIA DPSGD
To train a machine learning model with privacy, the most popular algorithm is the celebrated DP stochastic gradient descent (DPSGD) (Song et al., 2013; Bassily al., 2014; Abadi et al., 2016). This optimization method serves as a drop-in replacement for SGD, augmenting it with the addition of per-example gradient clipping and Gaussian noise addition steps. These two steps serve to limit and mask the contribution of a single example. Two key points to note are that a) per-example gradient clipping incurs significant computational and memory overheads in most implementations, and b) noise introduced due to privacy grows as the square-root as the number of model parameters. With this tool in place, the most basic fine-tuning strategy is to train all parameters using DPSGD.
2.2 REPARAMETRIZED GRADIENT PERTURBATION
To mitigate the limitations of DPSGD, recent work of Yu et al. (2021b) introduced elegant method called reparametrized gradient perturbation (RGP). RGP exploits the implicit low-rank structure in the gradient updates of SGD substantially improve upon DPSGD. Specifically, they reparametrize each layer's weight matrix W into LR + W, where L and R carrier matrices and W is the residual weight. The authors show that low-rank gradient- obtain low- dimensional projection of W's gradient by taking gradients only of the low-rank matrices L and R (and not the high-rank W). Privacy is introduced by clipping and noising these low-dimensional gra- dients of I. and R. While this low-dimensional projection loses some of the signal in W's gradient, turns out to contain enough to still achieve high accuracy. At the same time, the ow-diiensiiona gra- dients alleviate the aforementioned issues related to privatization, significantly reducing the memory consumption and noise introduced.
3
OUR APPROACH
3.1 A META-FRAMEWORK
We introduce our approach as a framwwrr for private deep learning, which abstracts the key principles of recent fine-tuning methods.
Suppose (WPP:; x) is a pre-trained model where WpT are the pre-trained weights and any input. We create a new fine-tuning model
frr(Wrr,,0: 0;z)
(I)
which incorporates additional trainable parameters 0, where dim(0) < dim(WpT). That is, the number of new parameters in 0 is a small fraction of the original number of parameters in the pre- trained weights WPT- Fine-tuning is done by running DPSGD on the additional parameters 6, while freezing the weights of pre-trained model Wpr. The new parameters are initialized to 6 such that
(FT 0o; x) = f(WPT) x).
The initialization condition (2) is very important, it ensures that fine-tuning starts at the praitrdined model and improves it by modifying the parameters 8. Most fine-tuning methods are additive and have the following special form:
fET(WPT, 0; = f(Wer + 2))
(3)
ie., they modify the pre-trained weights by adding a correction term (0) parametrized by 0. Recent work in the non-private literature has described concrete instantiations of this frame- work (Houlsby et al., 2019; Mahabadi et al., 2021; Hu et al., 2021), which (crucially) are effective when dim < (iin) In the non-private setting, such reparametrizations are uscful for ducing the computation and memory required for fine-tuning, and enable lightweight and plug-in modifications to the base model for different downstream tasks. At the same time, they maintain (or sometimes surpass) the accuracy achieved by full fine-tuning.
We give some intuition as why parameter-efficien methods to be more effective for private fine-tuning, especially on smaller datasets. For simplicity, we assume that the fine-tuning method is additive in (3), such that the fine-tuned weights WFT Wrr + x z0)). We can imagine that
4

---

Published as a conference paper at ICLR 2022
WeT lies on manifold passing through WPT of very small dimension (equal to the dimension of 8) compared to the dimension of WpT. Even if the parameters are very noisy due to the noise added during DPSGD, we will always stay in this manifold. In particular, the pre-trained weights in most directions (those orthogonal to the manifold near Wpr). we run DPSGD all the weights instead, then add noise in all directions, thus potentially unlearning the knowledge learned during pre-training, especially in low data regimes. However, this intuition may not always be true; see the remark the end of experiments on NLU tasks.
are not disturbing
Besides substantial gains in the accuracy, the above method of reparametrization has several other advantages:
A single pre-trained model such as BERT or GPT is generally applied to hundreds of down- stream tasks via fine-tuning. Private fine-tuning using previous methods requires updating all parameters and storing a different copy of the fine-tuned model per task. This creates substantial overheads for storing and deploying, and can be very expensive in practice. On the other hand, the reparametrization (1) means that we only need to store a single pre- trained model that can be shared across many downstream tasks. Each downstream task requires only small number of new parameters that can be plugged in.
Differentially private training requires computing and storing per-example gradients, which increases the memory footprint. In our approach, however, learning is done in a much lower dimension. hence saving on the memory cost compared to prior works.
Finally, expect that (1) also gives more communication-efficie in distributed settings such as federated learning, due to the significantly smaller number of parameters learned during fine-tuning.
method of fine-tuning
3.2 INSTANTIATING THE META-FRAMEWORK
this section, we discuss a few ways to instantiate our meta-framework. This list non-exhaustive, but covers the methods we employ in our experiments.
3.2.1 FINE-TUNING VIA LOW-RANK ADAPTATION
Low-Rank Adaptation (LoRA) (Hu et al., 2021) is an additive fine-tuning scheme as defined in (3). For each dense weight matrix W of size x in the pre-trained network, we add a low-rank correction term L'R', i.e.
W W + L'R'.
(4)
where L' gaxr, R E grxb are new trainable parameters. Hu et al. (2021) apply this repa- rameterization only the Transformer attention weights (W,, W.), and freeze all other weights (e.g.. W& and W, and those the feed-forward layers). The rank r is typically chosen to be small, e.g., 4, 16, 64. Since most parameters in Transformer architectures are dense weight matrices, choosing small r results in nearly square-root reduction in the number of parameters.
3.2.2 FINE-TUNING VIA ADAPTERS
Houlsby et al. (2019) propose adapter-based fine-tuning, in which we modify the architecture of the pre-trained model by adding new "adapter" layers after each attention and feed-forward layer. Adapter layers bottleneck layers with residual connections. Specifically, given input x, an adapter layer A performs
(5) where U is an up-projection affine linear map, D is a down-projection affine linear map, and a non-linear activation function such as the Gaussian error Linear Unit (GeLU) (Hendrycks & Gimpel, 2016). If z has dimension d, then U E Rddr D E prxd for some r & d. Thus, the num- ber of introduced parameters is significantly less than the number parameters in the pre-trained frozen, and only parameters of
U(=))(((((())) +
model. When fine-tuning, the parameters of the original model the adapter layers, as well as layer normalizations, are modified. Note that fine-tuning with adapters not additive fine-tuning framework as in (3), but is captured by the broader framework in (1).
5

---

Published as a conference paper at ICLR 2022
Table 2: Memory and speed comparison for RoBERTa-Large. The rank is chosen as r 16 for RGP and LoRA. The speed is measured by the wall-clock time for training one epoch of the SST-2 dataset on a single Tesla V100 GPU with gradient accumulation for batch size 2000.
Method
Memory (GB)
Speed (seconds per epoch)
Full fine-tuning (DPSGD)
27.9
715
RGP
9.1
296
DP LoRA
6.1
271
3.2.3 FINE-TUNING VIA COMPACTER
The recent work of Mahabadi et al. (2021) introduces Compacters (Compact adapters), a method which further improves the parameter efficiency of adapters. This done by replacing the dense matrices in the up-projection U and down- wwwn-projection..omm D by tensor products of smaller matrices, thus reducing the number trainable parameters. Specifically, they replace the dense matrix My in the adapter layer l by low-rank parameterized hypercomplex multiplication (LPHM) layer, i.c., each dense matrix Mj E Raxb is expressed as
a Me Σ
A @ (5
where A, R=x= S Ra/uxk T R and @ is the matrix Kronecker product. Note the matrices A, are not indexed by the layer l because these matrices are shared among all the adapter layers. Since each adapter layers has two dense matrices (one for up-projection and one for dton-projection), there are adapter layers, this reduces the number of parameters from L(2ab) L(2(a + b)k) + n In practice, and b chosen to be either the model dimension d or the intermediate representation dimension r in the adapters, n is typically chosen to be small constant such 2, 4, 8, 12 and is chosen to be 1.
3.2.4 WHY DOES
TUNING WORK?
Theoretical explanation of success of parameter-efficient fine-tuning methods is active area of re- search in deep learning. Indeed, since trends have consistently shown that model accuracy increases with size, how can one achieve competitive accuracy while fine-tuning less than 1% of the param- eters? One popular hypothesis is intrinsic dimensionality (Li et al., 2018), which posits that the minimum number of parameters needed to train a machine learning model may be much less than the total number of model parameters. Aghajanyan al. (2020) explore this hypothesis in the con- text of fine-tuning LLMMs, showing that one can achieve most of their accuracy by training only very small number parameters (chosen via random projection). Perhaps surprisingly, they find that as the model size increases, intrinsic dimension decreases, in the limit exhibiting zero-shot learning. While we did not explore this hypothesis in the context of DP due computational restrictions, we believe it may be an interesting lens through which one can understand the effectiveness of private narameter-efficien fine-tuning.
3.3 COMPARISION WITH BASELINE ALGORITHMS
We highlight some key algorithmic differences between our proposed methods and the baselines of full fine-tuning and RGP.
DPSGD and RGP both require updating all parameters of the pre-trained model, whereas our proposed methods update only a tiny fraction (between 0.05% and 1%). The rightmost columns of Tables 3 and 4 list the number of parameters trained by these algorithms.
RGP performs a low-rank decomposition of weight matrices which is similar to LoRA, though there are subtle differences. Recall that in RGP, at the beginning of each iteration t, the historical weight matrix Wi-1 is decomposed to find a low-rank product LR. The gradients on L. and R are then projected back to the full parameter space to perform the descent step. Hence, RGP keeps modifying the pre-trained weights during learning.
LoRA can viewed as a simplification of RGP. LoRA reparametrizes Wpr := Wpr LR, where the pre-trained weight matrix Wpr frozen during training. Hence, compared to

---

Published as a conference paper at ICLR 2022
Table Accuracy for fine-tuning with RoBERTa- (in %). The privacy parameters are 6.7, and 8 =le-5 for SST-2 and QNLI and le-6 for MNLI and QQP. Bold indicates the best accuracy with DP. Numbers for non non-private fine-tuning are from Liu et al. (2019) and Hu et al. (2021).
Method
MNLI
SST-2
QQP
QNLI
Avg.
Trained params
w/o DP
87.6
94.8
91.9
92.8
91.8
Full
100%
DP
53.1
82.6
74.4
63.9
68.5
LoRA
W/ DP
87.5
95.1
90.8
93.3
91.7
0.24%
RGP
DP
80.1
91.6
85.5
87.2
86.1
100%
Adapter
DP
83.4
92.5
85.6
87.5
87.3
1.4% (r = 48)
Compacter
DP
82.6
92.3
84.7
85.1
86.2
0.055% (r = 96, n2 = 8)
LoRA
DP
83.5
92.2
85.7
87.3
87.2
0.94% (r = 16)
RGP, LoRA eliminates the decomposition and the projection to the full parameter space at each iteration, simplifying the implementation and reducing the running time and memory cost. This is summarized in Table 2. We observe that DP LoRA reduces the memory cost by about 33% and the training speed by 8%. As we will see, this simplification also results in improved utility.
Neither full fine-tuning nor RGP fall into our meta-framewort described by (1). Thus, if pre-trained model is be applied several downstream tasks, one must store a separate set of weights for each task, incurring sigmificant memory cost and losing the plug-in functionality. In contrast, our methods are much more lightweight.
EXPERIMENTS
We experimentally evaluate our methods for DP fine-tuning to demonstrate their utility, pri- vacy, and parameter-efffcieneyy. We investigate both language understanding and text genera- tion tasks to establish that our techniques are applicable to a variety of tasks and model ar- chitectures. Our code is publicly available at https://githul: www.hhhb.ooomocooooooommm......e AnonymousAKKS// Differentially Seiiiillly-irivttee
Private- -Fine------------------ tetee-tnnnn---- of Sof-Languagee-ane Models.
4.1 FINE-TUNING FOR LANGUAGE UNDERSTANDING TASKS
We first compare our methods with state-of-the-an fine-tuning algorithms using models from the BERT family, which used in the prior work (Yu al., 2021b). Specifically, models (Liu et al., 2019), which are pre-trained on public data collected from the web. RoBERTa- Base has 125M parameters and RoBERTa-Large has 355M parameters. We choose four downstream tasks: MNLI, QQP, QNLI, and SST-2 from GLUE (Wang et al., 2018), following Yu et al. (2021b).
RoBERTa
For fine-tuning with adapters, we may choose the intermediate repre- sentation dimension r, shared across all adapter layers. For fine-tuning with Compacter, we can choose both and the Kronecker product kernel dimension in (6). For LoRA fine-tuning, we add bottleneck branches for both the attention layers and the feedforward layers, which differs slightly from the addition of bottleneck branches for only the W3 and W. matrices of the attention layers as done by Hu et al. (2021). Given the same bottleneck representation dimension in (4), our new implementation uses twice as many trainable parameters as the original paper, and achieves some improvements for learning with DP. We perform privacy accounting using the approach of Gopi et al. (2021), which currently gives the tightest bounds.
Implementation Details:
Hy perepaaareeeer choice: Given the large number of hyperparameter choices, e.g., the interme- diate representation dimension, learning rate, weight decay, privacy parameter 6, and model size, an exhaustive grid search over all hyperparameters is expensive. Our hyperparameter choices informed by prior work and are as follows. For privacy parameters, we use 8 le-5 for SST-2 and QNLI and 0.92, 0.83, 0.66 and 0.65 for SST-2, QNLI, QQP, and MNLI, respectively, which is the same as Yu al. (2021b). Appendix B, we run experiments under different privacy parameters. The proposed framework performs well under wide range choices of and 8. The clipping threshold is 10 for
le-6 for QQP and MNLI due to their dataset sizes, and noise multipliers
Numbers from httpt:/ httgg/tttbbhcco/ con/ /Di fferentiall www.rrennntaly-rrittttte-eeeeeee ty--rr-ate--eeeeeeeee learning
7

---

Published as a conference paper at ICLR 2022
Table Accuracy for fine-tuning with RoBeRTa-Large (in %). The privacy parameters are 6.7, and d = =le-5 for SST-2 and QNLI and 5 = 1e-6 for MNLI and QQP. Bold indicates the best accuracy with DP. Numbers for non- non-private fine-tuning are from Liu et (2019) and Hu et al. (2021).
Method
MNLI
SST-2
QQP
QNLI
Avg.
Trained params
Full
w/o DP
90.2
96.4
92.2
94.7
93.4
100%
LoRA
DP
90.6
96.2
91.6
94.9
93.3
0.23%
RGP
DP
86.1
93.0
86.7
90.0
88.9
100%
Adapter
DP
87.7
93.9
86.3
90.7
89.7
1.4% (r = 48)
Compacter
DP
87.5
94.2
86.2
90.2
89.5
0.053% (r 96, n = 8)
LoRA
DP
87.8
95.3
87.4
90.8
90.3
0.94% (r 16)
all methods. The batch size is 2000. In Appendix D, we show the performance of the proposed al- gorithm is stable across wide range of choices of clipping thresholds and batch sizes. For adapters and Compacter, we follow the original papers and choose r from (16, 48, 96) and from {4, 12). For LoRA, we choose the best- best-performing rank from the set {4, 16, 48, 64). The best performing hyperparameters are noted in Tables 3 and 4. We train for 20 epochs using AdamW (Loshchilov & Hutter, 2019) with weight decay le-2 and search over four learning rates {5e-4, le-3, 2e-3, 5e-3J.
Results: We report the prediction accuracy Tables 3 and 4. Our experiments using RoBERTa- Base serve a direct comparison to Yu et al. (2021b) who only trained the base model, whereas RoBERTa-Large experiments demonstrate the significance of using larger models. Our key find- ings are: (1) On all datasets, methods achieve the best accuracy while training tiny fraction of parameters; larger models give significant improvements. (2) Noticeable improvements in & ver- sus Yu al. (2021b) are primarily due to new privacy accountants based on Fourier-based numerical composition (Koskela et al., 2020; 2021: Gopi et al., 2021); we use the accountant in Gopi et al. (2021) since the most efficient.
(3) Private adapters provide the best average performance for RoBERTa-Baae Base, whereas LoRA outperforms all other methods for RoBERTa-Land
Remark:
While our experiments indicate that full fine-tuning does not achieve competitive perfor- mance, there could be a choice of hyperparameters that improves upon the reported numbers, c.g., "mega" batch sizes (in the millions) Anil et al. (2021). We note that our main message is that one does not need to fine-tune all parameters to achieve the best accuracy. Nevertheless, it is interesting to wonder if full fine-tuning with DPSGD can match the accuracy of paraaetee-effiiceet methods. A positive answer would imply that private and non-private fine-tuning conceptually mirror each other.
Update: concurrent work by Li et al (Li et al., 2022) show that using a larger batch size and training with full precision improves the performance of full fine-tuning via DPSGD, and obtains similar performance as our algorithms. Thus, poor performance of DPSGD in our experiments is due to the suboptimal choice of hyperparameters and also due to precision issues, although same hyperparameters for all the algorithms. We run new experiments with hyperparameters of (Li et al., 2022) in full precision mode, and get improvements around 1% even for algorithms. We report these findings in Appendix C.
4.2 FINE-TUNING FOR NATURAL LANGUAGE GENERATION (NLG)
Next, we study private fine-tuning for text generation problems using the GPT-2 series of models on the End-2-End (E2E) NLG challenge (Novikova et al., 2017), of the primary benchmarks used in recent works on non poivvaiv fine- aeeetunnng (Hu et al., 2021; Li & Liang, 2021). We use GPTl2-Smaill (117M parameters), GPT-2-Medium (345M parameters). and GPT-2-Large (774M parameters). To the best of our knowledge, we are the first to privately fine-tune for E2E or fine-tune GPT-2-Large- The purpose of this section is not to evaluate various fine-tuning algorithms, but show that private fine-tuning is competitive with non-private fine-tuning for text generation problems. Due to the high cost of training, we report experimental results only for fine-tuning with LoRA.
In Appendix F, we present additional experiments on NLG that include private fine-tuning of the GPT-2-XL model with 1.5 billion parameters. Other noticeable points in the additional experiments
1tml.
8

---

Published as a conference paper at ICLR 2022
Table 5: Metrics on the E2E NLG task 5.4, =le-5). Non-DP results from Hu et al. (2021).
Method
BLEU
NIST
MET
ROUGE-L
CIDEr
GPT-2-Small + DP
59.26
6.13
36.6
64.1
1.63
GPI-2-Medum + DP
64.2
7.77
40.02
66.45
2,00
GPT-22Large + DP
64.51
8.22
41.5
67.55
2.13
70.4
8.85
46.8
71.8
2,53
GPr-2-Large
70.4
8.89
46.8
72.0
2.47
include 1) we show improved performance using better hyperparameters: 2) we test different privacy parameters; consider new dataset DART (Nan et al., 2021).
The E2E dataset in Novikova et al. (2017) contains template-like information in the restaurant domain to be mapped to natural language with end-to-end training. The dataset consists of 42K training samples, 4.6K validation samples, and 4.6K test samples. We use standard metries such as BLUE, ROUGE-L, etc., used in (Hu al., 2021) for evaluation.
E2E NLG challenge:
Hyperparameter choice: For LoRA, we choose the bottleneck rank 4 in (4) and fine-tune W and W, , matrices of the attention layers as in the original paper. We optimize using AdamW with learning rate 2e-4, weight decay le-2 and train our models for 5 epochs using batch size 64. We take the gradient clipping parameter to be 1.0 and set the noise multiplier 0.5.
Results:
The results of our experiments are summarized in the Table 5, which reiterate the main themes of this paper: private fine-tuning with a parameter-efficient approach performs close to their non-private counterparts and show consistent improvement in the utility the model size increases.
RELATED WORK
Some work studies private language models on more traditional architectures such as LSTMs (Hochreiter & Schmidhuber, 1997), either training with DPSGD (McMahan et al., 2018; Carlini et al., 2019) or related heuristics (Ramaswamy et al., 2020). Though pre-training on public data is suggested (McMahan et al., 2018), 8), public data appears to only be used in one of these works for honest hyperparameter selection (Ramaswamy et al., 2020). A few more recent works consider training LLMs with DP. Anil et al. (2021) privately train BERT-Large from scratch, compared to our work which focuses on private fine-tuning. (Hoory et al., 2021; Basu et al., 2021) perform private full fine-tuning of BERT models. Hoory et al. (2021) achieve accuracy which comparable to the non-private model, but additionally supplement the public pre-training data with additional domain-relevant material, while we use off-the-shelf pre-trained models. Basu et al. (2021) observe significant drops in utility, compared to our parameter-efficient methods which do not. While Kerri- gan et al. (2020) consider public pre-trainin preitrain and private fine-tuning, their experiments are on much smaller architectures (i.e., feedforward networks with three hidden layers). A simultaneous work of Ginart al. (2022) investigates private prediction (rather than learning) for next-token predic- tion. A subsequent work by Senge et al. (2021) also investigates the effect of private fine-tuning on various NLP tasks.
In concurrent work, Li et al. (2022) also investigate DP fine-tuning of LLMSs. In several cases, their results demonstrate qualitatively similar findings as ours. While our experiments focus primarily on parameter-efficiee fine-tuning methods, interestingly, they show that private full fine-tuning can also achieve comparable utility the experimental setup is configured properly, e.g., using suitable hyperparameters. In Appendix C, experiments under the setup in Li et al. (2022). We show their setup can also improve the performance of our methods.
6 CONCLUSION
So far, DP deep learning has focused on training models from scratch. The spectacular success of transfer learning in real-world applications, however, shows that private fine-tuning! is an equally per- tinent problem to study and deserves privacy accounting, and using larger models, one can privately fine-tune
attention. We show that by combining recent advances in
NLP, parameter-efficienee models whose utility approaches that of non-private models. We hope our work inspires more study on the core problem of private fine-tuning, which we believe to be a central direction for research in private machine learning, leading more interaction between the LLM and DP communities.

---

Published as a conference paper at ICLR 2022
ACKNOWLEDGMENTS
The authors would like to thank Rabech Karimi Mahabadi for sharing hyperparameters based on experiments Mahabadi et al. (2021). Janardhan Kulkarni would like thank Edward Hu for sharing many ideas on fine-tuning, Gautam Kamath is supported by an NSERC Discovery Grant.
REFERENCES
Martin Abadi, Andy Chu, Ian Goodfellow, H Brendan McMahan, Ilya Mironov, Kunal Talwar, and Li Zhang. Deep learning with differential privacy. Proceedings of the 2016 ACM Conference on Computer and Communications Security, CCS 16, pp. 308-318, New York, NY, USA, 2016. ACM.
Armen Aghajanyan, Luke Zettlemoyer, and Sonal Gupta. Intrinsic dimensionality explains the ef- fectiveness of language model fine-tuning. arXiv preprint arXiv:2012.13255 2020.
Noga Alon, Raef Bassily, and Shay Moran. Limits of private learning with access to public data. In Advances in Neural Information Processing Systems 32, NeurlPS '19, pp. 10342-10352. Curran Associates, Inc., 2019.
Ehsan Amid, Arun Ganesh, Rajiv Mathews, Swaroop Ramaswamy, Shuang Song, Thomas Steinke, Vinith M Suriyakumar, Om Thakkar, and Abhradeep Thakurta. Public fata-assisted mirror de- scent for private model training. arXiv preprint 1rxiv:2122.00/92 2021.
Rohan Anil, Badih Ghazi, Vineet Gupta, Ravi Kumar, and Pasin Manurangsi. Large-scale differen- tially private BERT. arXiv preprint (rXix:2008001220 2021.
Raef Bassily, Adam Smith, and Abhradeep Thakurta. Private empirical risk minimization: Effi- cient algorithms and tight error bounds. In Proceedings of the 55th Annual IEEE Symposium on Foundations of Computer Science, FOCS '14, pp. 464 473, Washington, DC, USA, 2014. IEEE Computer Society.
Raef Bassily, Om Thakkar, and Abhradeep Guha Thakurta. Model-agnostic private learning. In Advances in Neural Information Processing Systems 31, NeurIPS '18, pp. 7102-7112. Curran Associates, Inc., 2018.
Raef Bassily, Albert Cheu, Shay Moran, Aleksandar Nikolov, Jonathan Ullman, and Steven Wu. Private query release assisted by public data. In Proceedings of the 37th International Conference on Machine Learning, ICML '20, pp. 695-703. JMLR, Inc., 2020a.
Raef Bassily, Shay Moran, and Anupama Nandi. Learning from mixtures of private and public populations. In Advances in Neural Information Processing Systems 33, NeurIPS '20, pp. 2947- 2957. Curran Associates, Inc., 2020b.
Priyam Basu, Tiasa Singha Roy, Rakshit Naidu, Zumrut Muftuoglu, Sahib Singh, and Fatemehsadat Mireshghallah. Benchmarking differential privacy and federated learning for BERT models. arXiv preprint arXiv:2106.. 13973, 2021.
Amos Beimel, Kobbi Nissim, and Uri Stemmer, Private learning and sanitization: Pure vs. approxi- mate differential privacy. Theory of Computing. 12(1):1-61, 2016.
Elad Ben Zaken, Shauli Ravfogel, and Yoav Goldberg. Bitfit: Simple eframeter-efficient fine-tuning for wansformer-based masked language-models, arXiv preprint arXiv:, 2021.
Gavin Brown, Mark Bun, Vitaly Feldman, Adam Smith, and Kunal Talwar. When is memorization of irrelevant training data necessary for high-accuracy learning? In Proceedings of the 53nd Annual ACM Symposium on the Theory of Computing, STOC '21, pp. 123-132, New York, NY, USA, 2021. ACM.
Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhari- wal, Arvind Neclakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert- Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M.
10

---

Published as a conference paper at ICLR 2022
Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. Language models are few-shot learners. In Advances in Neural Information Processing Systems 33, NeurlPS '20. Curran Associates, Inc., 2020.
Mark Bun, Jonathan Ullman, and Salil Vadhan. Fingerprinting codes and the price of approximate differential privacy. In Proceedings of the 46th Annual ACM Symposium the Theory of Com- puling, STOC 14, pp. 1-10, New York, NY, USA, 2014, ACM.
Han Cai, Chuang Gan, Ligeng Zhu, and Song Han. TinyTL: Reduce memory, not parameters for efficient on-device learning. In Advances in Neural Information Processing Systems 33, NeurlPS '20, pp. 11285-11229 1297. Curran Associates, Inc., 2020.
Nicholas Carlini, Chang Liu, Ülfar Erlingsson, Jernej Kos, and Dawn Song. The secret sharer: Evaluating and testing unintended memorization in neural networks. 28th USENIX Security Symposium, USENIX Security 19, pp. 267-284. USENIX Association, 2019.
Nicholas Carlini, Florian Tramèr, Eric Wallace, Matthew Jagielski, Ariel Herbert-Voor Voss, Katherine Lee, Adam Roberts, Tom Brown, Dawn Song, Ulfar Erlingsson, Alina Oprea, and Colin Raf- fel. Extracting training data from large language models. In 30th USENIX Security Symposium, USENIX Security '21, pp. 2633-26662222 2650. USENIX Association, 2021.
Jacob Devlin, Ming-Wei Wei Chang, Kenton Lec, and Kristina Toutanova. BERT; Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume I (Long and Short Papers), NAACL-HLT 19, pp. 4171-4181 2019.
Cynthia Dwork, Krishnaram Kenthapadi, Frank McSherry, Ilya Mironov, and Moni Naor. Our data, ourselves: Privacy via distributed noise generation. In Proceedings of the 24th Annual Interna- fional Conference the Theory and Applications of Cryptographic Techniques, EUROCRYPT '06, pp. 486-503, Berlin, Heidelberg, 2006a. Springer.
Cynthia Dwork, Frank McSherry, Kobbi Nissim, and Adam Smith. Calibrating noise to sensitivity in private data analysis. In Proceedings of the 3rd Conference Theory of Cryptography, TCC '06, pp. 265-284, Berlin, Heidelberg, 2006b. Springer.
Vitaly Feldman. Does learning require memorization? a short tale about a long tail. In Proceedings of the 52nd Annual ACM Symposium the Theory of Computing, STOC '20, pp. 954-959, New York, NY, USA, 2020. ACM.
Antonio Ginart, Laurens van der Maaten, James Zou, and Chuan Guo. Submix: Practical private prediction for large-scale language models. arXiv preprint arXiv:2201.0099), 2022.
Sivakanth Gopi, Yin Tat Lee, and Lukas Wutschitz. Numerical composition of differential privacy. 2021.
arXiv preprint
Dan Hendrycks and Kevin Gimpel. Gaussian error linear units (gelus). arXiv preprint
arXiv:1606.08.. 08415, 2016.
Sepp Hochreiter and Jürgen Schmidhuber. Long short-term memory. Neural Computation, 9(8):
1735-1780, 1997.
Shlomo Hoory, Amir Feder, Avichai Tendler, Alon Cohen, Sofia Erell, Itay Laish, Hootan Nakhost, Uri Stemmer, Ayelet Benjamini, Avinatan Hassidim, and Yossi Matias. Learning and evaluating a differentially private pre-trained language model. In Proceedings of the Third Workshop on Privacy in Natural Language Processing, PrivateNLP '21, pp. 21-29, 2021.
Neil Houlsby, Andrei Giurgiu, Stanislaw Jastrzebski, Bruna Morrone, Quentin De Laroussilhe, An- drea Gesmundo, Mona Attariyan, and Sylvain Gelly. Parameter-efficient transfer learning for nip. In International Conference on Machine Learning, pp. 2790-2799. PMLR, 2019.
Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zha ZZu, Yuanzhi Li, Shean Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models. arXiv preprint arXiv:2106.0966 2021.
11

---

Published as a conference paper at ICLR 2022
Zhanglong Ji and Charles Elkan. Differential privacy based on importance weighting. Machine Learning. 93(1):163-18) 18366-1-8-0000 2013.
Peter Kairouz, Mónica Ribero, Keith Rush, and Abhradeep Thakurta. (nearly) dimension indepen- dent private ERM with adagrad rates via publicly estimated subspaces. In Proceedings of the 34th Annual Conference on Learning Theory, COLT '21, pp. 2717-2746, 2021.
Gavin Kerrigan, Dylan Slack, and Jens Tuyls. Differentially private language models benefit from public pre-training. arXiv preprint arXiv:2009.0588 2020.
Antti Koskela, Joonas Jälkö, and Antti Honkela. Computing tight differential privacy guarantees using fft. In International Conference on Artificial Intelligence and Statistics, pp. 2560-2569. PMLR, 2020.
Antti Koskela, Joonas Jälkö, Lukas Prediger, and Antti Honkela. Tight differential privacy for discrete-valued mechanisms and for the subsampled gaussian mechanism using fft. Interna- tional Conference on Artificial Intelligence and Statistics, pp. 3358-3366. PMLR, 2021.
Brian Lester, Rami AI-Rfou, and Noah Constant. The power of scale for parameter-effiiien prompt tuning. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Pro- cessing (EMNLP), EMNLP '21. Association for Computational Linguistics, 2021.
Chunyuan Li, Heerad Farkhoor, Rosanne Liu, and Jason Yosinski. Measuring the intrinsic dimen- sion of objective landscapes. In Proceedings of the 6th International Conference on Learning Representations, ICLR '18 2018
Xiang Lisa Li and Percy Liang. Prefix-tuning: Optimizing continuous prompts for generation. In Proceedings of the 59th Anmual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), ACL-IJCNLP 22,, pp. 4582-4597, 2021.
Xuechen Li, Florian Tramèr, Percy Liang, and Tatsunori Hashimoto. Large language models can be strong differentially private learners. In Proceedings of the 10th International Conference Learning Representations, ICLR '22, 2022.
Terrance Liu, Giuseppe Vietri, Thomas Steinke, Jonathan Ullman, and Steven Wu. Leveraging public data for practical private query release. In Proceedings of the 38th International Conference on Machine Learning, ICML '21, pp. 6968-6977. JML,, Inc., 2021.
Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zeutlemoyer and Veselin Stoyanov. RoBERTa: A robustly optimized bert pretraining approach. arXiv preprint 19X.:1997.111999 2019.
Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. In Proceedings of the 7th International Conference Learning Representations, ICLR '19 2019.
Zelun Luo, Daniel J Wu, Ehsan Adeli, and Li Fei-Fei. Scalable differential privacy with sparse network finetuning. In Proceedings of the 2021 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, CVPR '21, pp. 5059-5066 Washington, DC, USA, 2021. IEEE Computer Society.
Rabeeh Karimi Mahabadi, James Henderson, and Sebastian Ruder. Compacter: Efficient low-rank hypercomplex adapter layers. arXiv preprint 10Xiv:2106.0460 2021.
Brendan McMahan, Daniel Ramage, Kunal Talwar, and Li Zhang. Learning differentially private recurrent language models. In Proceedings of the 6th International Conference on Learning Representations, ICLR '18, 2018.
Paulius Micikevicius, Sharan Narang, Jonah Alben, Gregory Diamos, Erich Elsen, David Garcia, Boris Ginsburg, Michael Houston, Oleksii Kuchaiev, Ganesh Venkatesh, et al. Mixed precision training. arXiv preprint arXiv: 17100377000034000 2017.
12

---

Published as a conference paper at ICLR 2022
Linyong Nan, Dragomir Radev, Rui Zhang, Amrit Rau, Abhinaand Sivaprasad, Chiachun Hsieh, Xiangru Tang, Aadit Vyas, Neha Verma, Pranav Krishna, Yangxiaokang Liu, Nadia Irwanto, Jes- sica Pan, Faiaz Rahman, Ahmad Zaidi, Mutethia Mutuma, Yasin Tarabar, Ankit Gupta, Tao Yu, Yi Chern Tan, Xi Victoria Lin, Caiming Xiong, Richard Socher, and Nazneen Fatema Rajani. DART: open-domain structured data record to text generation. In Proceedings of the 2021 Con- ference of the North American Chapter of the Association for Computational Linguisties; Human Language Technologies, NAACL-HLT 221, pp. 432-447. Association for Computational Linguis- tics, 2021.
Anupama Nandi and Raef Bassily. Privately answering classification queries in the agnostic PAC model. In Proceedings of the 31st International Conference on Algorithmic Learning Theory, ALT '20, pp. 687-703. JMLR, Inc., 2020.
Jekaterina Novikova, Ondfej Dušck, and Verena Rieser. The E2E dataset: New challenges for end-to-end generation. In Proceedings of the 18th Annual SIGdial Meeting on Discourse and Dialogue, SIGDIAL '17, pp. 201-206. Association for Computational Linguistics, 2017.
Myle Ott, Sergey Edunov, David Grangier, and Michael Auli. Scaling neural machine translation. In Proceedings of the Third Conference on Machine Transiation (WMT), 2018.
Nicolas Papernot, Martín Abadi, Ulfar Erlingsson, Ian Goodfellow, and Kunal Talwar. Semi- supervised knowledge transfer for deep learning from private training data. Proceedings of the 5th International Conference on Learning Representations, ICLR 17, 2017.
Nicolas Papernot, Shuang Song, Ilya Mironov, Ananth Raghunathan, Kunal Talwar, and Úlfar Er- lingsson. Scalable private learning with PATE. In Proceedings of the 6th International Conference on Learning Representations, ICLR 18, 2018.
Nicolas Papernot, Steve Chien, Shuang Song, Abhradeep Thakurta, and Ulfar Erlingsson. Making the shoe fit: Architectures, initializations, and tuning for learning with privacy. https:// openreview net/ forum? wetrrmmmi==Jjee d=rJg851 YWwH, 2019.
Jonas Pfeiffer, Aishwarya Kamath, Andreas Rücklé, Kyunghyun Cho, and Iryna Gurevych. Adapter- fusion: Non-destruetive task composition for transfer learning. In Proceedings of the 16th Con- ference of the European Chapter of the Association for Computational Linguistics: Main Volume, EACL '21, pp. 487-503. Association for Computational Linguistics, 2021.
Alec Radford, Karthik Narasimhan, Tim Salimans, and Ilya Sutskever. Improving language under- standing by generative pre-training, 2018
Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. Language models are unsupervised multitask learners, 2019.
Swaroop Ramaswamy, Om Thakkar, Rajiv Mathews, Galen Andrew, H Brendan McMahan, and Françoise Beaufays. Training production language models without memorizing user data, arXiv preprint arXiv:2009. 10031, 2020.
Andreas Rücklé, Gregor Geigle, Max Glockner, Tilman Beck, Jonas Pfeiffer, Nils Reimers, and Iryna Gurevych. Adapterdrop: On the efficiency of adapters in transformers. arXiv preprint D:Xiv:2010.11911 2020.
Manuel Senge, Timour Igamberdiev, and Ivan Habernal. One size does not fit all: Investigating strategies for differentially-priinn learning across nip tasks. arXiv preprint 12.0819,, 2021.
Reza Shokri, Marco Stronati, Congzheng Song, and Vitaly Shmatikov. Membership inference at- tacks against machine learning models. In Proceedings of the 38th IEEE Symposium on Security and Privacy, SP 17, pp. 3-18, Washington, DC, USA, 2017. IEEE Computer Society.
Shuang Song, Kamalika Chaudhuri, and Anand D Sarwate. Stochastic gradient descent with dif- ferentially private updates. In Proceedings of the 2013 IEEE Global Conference on Signal and Information Processing, GlobalSIP '13, pp. 245 248, Washington, DC, USA, 2013. IEEE Com- puter Society.
13

---

Published as a conference paper at ICLR 2022
Pranav Subramani, Nicholas Vadivelu, and Gautam Kamath. Enabling fast differentially private sgd via just-in-time compilation and vectorization. In Advances in Neural Information Processing Systems 34, NeurlPS '21. Curran Associates, Inc., 2021.
Zhiliang Tian, Yingxiu Zhao, Ziyue Huang, Yu-Xiang Wang, Nevin Zhang, and He He. Seq- PATE: Differentially private text generation via knowledge distillation, 2022. URL https:
/ //openzeview..
Florian Tramèr and Dan Boneh. Differentially private learning needs better features (or much more data). In Proceedings of the 9th International Conference on Learning Representations, ICLR
21, 2021.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Advances in Neural Infor- mation Processing Systems 30, NIPS '17, pp. 5998-6008. Curran Associates, Inc., 2017.
Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel R Bowman. Glue: A multi-task benchmark and analysis platform for natural language understanding. In International Conference on Learning Representations, 2018.
Adina Williams, Nikita Nangia, and Samuel Bowman. A broad broad-coverage challenge corpus for sen- tence understanding through inference. In Proceedings of the 2018 Conference of the North Amer- ican Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume I (Long Papers), NAACL-HLT '1, pp. 1112-1122 122.. Association for Computational Lin- guisties, 2018.
Da Yu, Huishuai Zhang, Wei Chen, and Tie- Yan Liu. Do not let privacy overbill utility: Gradient embedding perturbation for private learning. In Proceedings of the 9th International Conference on Learning Representations, ICLR 22,, 2021a.
Da Yu, Huishuai Zhang, Wei Chen, Jian Yin, and Tic- Yan Liu. Large scale private learning via low-rank reparametrization. In Proceedings of the 38th International Conference Machine Learning, ICML '21. JMLR, Inc., 2021b.
Yingxue Zhou, Zhiwei Steven Wu, and Arindam Banerjee. Bypassing the ambient dimension: Pri- vate SGD with gradient subspace identification. In Proceedings of the 9th International Confer- ence on Learning Representations, ICLR '21, 2021.
14

---

Published as a conference paper at ICLR 2022
Table 6: Test accuracy for fine-tuning RoBeRTa-Large with different privacy parameters. The num- ber of training samples is denoted by n. The values of a are noise multipliers. Numbers in the brackets are the changes compared the results Table (E 6.7, (1/nn))
Taks
0
1/n
== 1/10n
8 8= 1/100n
== 1/ /10000
Accuracy (in %)
MNLI
1.88
1
1.35
1.49
1.61
86.8 (-1.0%)
QQP
1.88
= 1
= 1.40
1.54
= 1.67
85.2 (-2.2%)
QNLI
3.01
1.48
Σ 1.64
1.79
88.0 (-2.8%)
SST-2
3.63
= 1
1.47
1.64
1.80
93.1 (-2.2%)
MNLI
0.91
3
4.12
4.51
4.89
87.4 (-0.4%)
QQP
0.93
= 3
4.10
4.49
4.86
86.8 (-0.6%)
QNLI
1.29
4.45
4.90
5.33
89.9 (-0.9%)
SST-2
1.52
3
4.37
4.83
5.25
94.1 (-1.2%)
A
ADDITIONAL RELATED WORK
There exist other parameter-efficien tuning methods which we did not evaluate in our work. Some of these include random subspace projection (exploiting intrinsic dimensionality (Li et al., 2018; Aghajanyan et al., 2020)), prefix and prompt tuning (Li Liang, 2021; Lester et al., 2021), tuning only biases (Cai et al., 2020; Ben Zaken et al., 2021). and other architecture variants including Adapters (Pfeiffer et al., 2021; Rücklé et al., 2020). An interesting direction for future work is to see whether parameter-efficient tuning approaches specifically designed for the private setting can achieve higher utility. We also mention zero-shot learning, in which no task-specific dataset is required and thus perfect privacy is achieved. Currently, zero-shot approaches achieve low utility compared to fine-tuning, though it is possible that future models may narrow this gap.
Finally, investigation fits more broadly into a line of work employing public data for private data analysis. Some works on image classification consider pre-training on a large public dataset and fine-tuning on smaller private dataset (Abadi et al., 2016; Papernot al., 2019; Tramèr & Boneh, 2021; Luo et al., 2021). In particular, Luo et al. (2021) investigate the role of parameter efficiency in private fine-tuning ResNet models, and propose strategies to choose which parameters to fine-tune. One line of work uses unlabeled public data to train a student model (Papernot et al., 2017: 2018; Bassily et al., 2018), including one work simultaneous to our own for natural language generation Tian et al. (2022). Another recent idea uses small amount of public data to identify lower-dimensional subspace of the gradients which perform private descent (Zhou al., 2021; Yu et al., 2021a; Kairouz et al., 2021). A simultaneous work of Amid et al. (2021) uses public data in the mirror map for private mirror descent algorithm. Finally, other works (both theoretical and experimental) investigate the role of public data in private query release, synthetic data generation, and prediction (Ji & Elkan, 2013; Beimel et al., 2016; Alon et al., 2019; Nandi & Bassily, 2020; Bassily et al., 2020a;b; Liu et al., 2021).
B EXPERIMENTS WITH DIFFERENT PRIVACY PARAMETERS
Now we test our framework under different privacy constraints. Specifically, we run LoRA on the language understanding tasks with various choices of privacy parameters and 8. We consider both RoBERTa- Base and RoBERTa-Large.
For the RoBERTa-Large model, we set 1 and with 5 being the same as those in Section 4. We use the PRV accountant (Gopi et al., 2021). After getting the noise multipliers, we also reduce the value and report the corresponding value of e. The hyperparameters are the same as those in Section 4. We run experiments on all four tasks, i.e., MNLI (n 392k), QQP (n 364k), QNLI (n ~ 104k), and SST-2 (n ~ 67k). We report the results in Table 6. The performance of our framework is decent even with very tight privacy budgets. For instance, with < 2 and = =1/1000m, the accuracy gap between the non-private baseline is only 3.8 for MNLI and 2.1 for SST-2.
15

---

Published as a conference paper at ICLR 2022
SST-2
MNLI
90
95
Accuracy
Accuracy
85
90
53
55.0
Accuracy
Accuracy
54.5
54.0
Non-private LoRA
53.5
Non-private
0.1
0.5
12
0.5
12
Figure 2: Test accuracy (in %) of fine-tuning the RoBERTa-Base model on MNLI and SST-2 with
various choices of
Table 7: Accuracy for fine-tumin tunettunig downstream tasks with RoBERTa- Base (in %). Experiments are with full-preed full-precision. We also scale up the batch size according to the dataset size compared to SST-2. The privacy parameters are & = 6.7, and =le-5 for SST-2 and QNLI and 1e-6 for MNLI and QQP.
Method
MNLI
SST-2
QQP
QNLI
Average Accuracy
w/o DP
87.6
94.8
91.9
92.8
91.8
Full
DP
83.2
85.9
86.2
84.8
85.0
Adapter
DP
84.6
92.9
87.4
89.2
88.5
LoRA
DP
84.5
92.7
87.1
88.3
88.2
For the RoBERTa-Base model, try various choices of The values of chosen from [0.1, 0.5, 3, 5, 8, 12). All other settings are the same as those Section 4. We run experiments on the MNLI and SST-2 datasets. The results presented in Figure 2. Our framework performs well for a wide range of e. We note that our algorithm achieves meaningful accuracy even for very tight privacy parameters 0.5 and Such values of are rarely explored when training deep models with differential privacy.
C FINE-TUNING FOR LANGUAGE UNDERSTANDING TASKS WITH LARGE BATCH SIZE AND FULL- CULL-PRECISION
Li al. (2022) show the performance of fine-tuming the full model can be significantly improved with proper configuration. In this section, we re-evaluate the tasks in Table 3 and under the configuration in Li al. (2022) and show such configuration also improves the performance of our methods.
The configuration in Li al. (2022) has two major differences compared that in Section 4.1. The first difference is Li et al. (2022) run experiments with full-precision while the experiments
Table 8: Accuracy for fine-tuning downstream tasks with RoBERTa- (in %). Experiments run with full-precision. We also scale the batch size according to the dataset size compared to SST-2. The privacy parameters are 6.7. and 8 =le- for SST-2 and QNLI and 8 =le-6 for
MNLI and QQP.
Method
MNLI
SST-2
QQP
QNLI
Average Accuracy
w/o DP
90.2
96.4
92.2
94.7
93.4
Full
DP
86.4
90.9
87.5
89.4
88.6
Adapter
DP
88.6
94.5
87.8
91.6
90.6
LoRA
DP
89.0
95.3
88.4
92.4
91.3
16

---

Published as a conference paper at ICLR 2022
Learning Rate 0.0005
Learning Rate 0.0010
90.8
91.3
92.1
92.4
92.7
91.0
92.0
92.1
92.3
92.7
Clipping Threshold
91.7
Clipping Threshold
1.0
3,0
91.4
91.9
3.0
91.1
92.0
5.0
90.7
91.5
91.9
92.1
92.6
5.0
91.3
92.1
92.2
92.9
92.8
10.0
90.7
91.6
92.2
92.6
92.8
10.0
91.2
92.1
92.8
92.9
92.9
200
500
1000
2000
4000
200
500
1000
2000
4000
Batchsize
Batchsize
Figure 3: Test accuracy (in %) of fine-tuning RoBERTa-BB with differentially private LoRA on
the SST-2 dataset. Our algorithm performs well on wide range of hyperparameters.
Table 9: Non-private metries on the E2E NLG task, using full fine- ane-tuning.
Method
BLEU
NIST
MЕT
ROUGE-L
CIDEr
EPT--2-Medumm
68.2
8.62
46.2
71.0
2.47
GPTg-2-Large
68.5
8.78
46.0
69.9
2.45
in Section 4.1 use half- ceff-preecsion. Using half-precision is common approach to speed up NLP experiments (Ott al., 2018). However, half calf-precision may incur underflow issue which impacts the model performance (Micikevicius et al., 2017). The second difference is they use larger batch size for larger datasets. For example, the batch size for MNLI is roughly six times larger than the batch size for SST-2 in Li et al. (2022). In Section 4.1, we use the same batch size for all datasets.
We follow the above setup and re-evaluate DP-LORA and DP-Adapter. The results are Table and 8. The results of full fine-tuning with differential privacy directly adopted from Li et al. (2022). The configuration in et al. (2022) further improves the strong results Table and 4. For example, achieve 89.0% accuracy the MNLI dataset, which is only 1.2% lower than the accuracy without DP constraint. Moreover, the benefit of the proposed framework over full fine- tuning still clear. The average accuracy of the proposed algorithms is ~3% higher than that of full fine-tuning.
D ON THE INFLUENCE OF HYPERPARAMETERS
Here we demonstrate that our algorithms perform well for a wide range of hyperparameters. We study two hyperparameters that are directly related to the variance of noise: clipping threshold and batchsize. The clipping threshold is chosen from [0.1. 1.0, 3.0, 5.0, 10.0) and the batchsize chosen from [200, 500, 1000, 2000, 4000]. We note that we keep the number of updates the same as that in Section when the batchsize changed. We fine-tune the RoBERTa-
model with differentially private LoRA (r 16) on the SST-2 dataset. The results are presented in Figure 3. DP LoRA A performs well for all the hyperparameters considered. The gap between the best accuracy and the worst accuracy only 2%.
FULL TUNING WITH GPT-2
All results in Table (in the main body). both private and non-private, perform fine-tuning using LoRA. In Table 9, we additionally report utility of non-private full fine-tuning. These numbers taken from Table Li & Liang (2021). In general, these numbers are slightly lower than those obtained by performing non-private fine-tuning with LoRA.
17

---

Published as a conference paper at ICLR 2022
Table 10: Metries on the E2E NLG task. Non-DP results from Hu al. (2021), except for GPT-2- XL, which was not reported in the paper. We ran GPT-2-XL with hyperparameters presented in Hu al. (2021). Bold indicates the best accuracy with DP. DP parameters are 6.0, 1e-5). Val perp stands for validation perplexity.
Method
Val perp
BLEU
NIST
MET
ROUGE-L
CIDEr
GPT-2-Small + DP
4.51
63.8
7.19
39.5
67.5
1.87
GPT--2Meehumm DP
4.02
65.5
8.45
42.7
67.9
2.23
GPT-2-Large + DP
3,87
66.7
8.63
44.0
67.8
2.33
GPTT-2XLL++ DP
3.79
66.1
8.53
43.0
68.1
2.28
GIPT-2-Medium
3.19
70.4
8.85
46.8
71.8
2.53
GPT-2-Large
3,06
70.4
8.89
46.8
72.0
2.47
GPT-2-XL
3.01
69.4
8.78
46.2
71.5
2.49
Table 11: Metries on the E2E NLG task. Bold indicates the best accuracy with DP. DP parameters
satisfy (s 3.0, 5 = le-5), (s = 3.4, 1/10m), (s = 3.9, 5 1/100m) and (c 4.5, 8
1/1000n). Val perp stands for validation perplexity.
Method
Val perp
BLEU
NIST
MET
ROUGE-L
CIDEr
GPT-2-Small + DP
4.59
62.7
7.03
39.2
66.4
1.85
GPT-2-Medium DP
4.08
65.2
8.31
42.2
68.1
2.22
GPT-2- + DP
3.92
66.7
8.60
43.6
68.1
2.29
GPT-2-XL DP
3.85
67.6
8.64
44.9
68.6
2.36
F ADDITIONAL EXPERIMENTS ON NATURAL LANGUAGE GENERATION
In this section, perform additional experiments private fine-tuning for text generation prob- lems using the GPT-2 series of models. This includes the private fine-tuning of the GPT-2-XL. model with 1.5B parameters. There are three main points to compared to our results in the main body: I) We show an improved performance on E2E NLG challenge using better hyperparameters: 2) We conduct experiments on E2E dataset with different privacy parameters to show that large language models perform strong even with smaller privacy budgets; 3) Finally, we conduct new experiments on DART dataset.
IMPROVING THE PERFORMANCE ON E2E NLG CHALLENGE
We improve the results of Table 5 with the following set of hyperparameters.
Hyperparameter choice: For LoRA, we choose the bottleneck rank r = in (4) and fine-tune W and W, matrices of the attention layers in the original paper. We optimize using AdamW with learning rate 4e-4, weight decay le-2 and train our models for 20 epochs. We use batch size 128. We take the gradient clipping parameter be 1.0 and the noise multiplier in Gopi et al. (2021), achieving & = 6.0, le-5..
0.6 for the accountant
Results: The results of experiments summarized in the Table 10.
F.2 EXPERIMENTS WITH DIFFERENT PRIVACY PARAMETERS
On E2E dataset, test our framework with smaller privacy budgets (s 5 and 5 < 1/n) where the number of samples the training data.
Hyperparameter choice: The hyperparameter choices are similar in Section F.I. The only difference is that we increase the noise multiplier to be 0.71 for the accountant in Gopi et al. (2021), achieving the following (s, 5) pairs: (s 3.0,8 =le-5),
3.4,8 1/10n), 3.9,
1/100m) and (c 4.5,8 8 1/1000m).
Results: The results of experiments summarized in the Table 11.
There are a couple of interesting observations comparing Table 11 with Table 10. First, we observe that although privacy budget is tight in Table 11, the results are quite similar to Table 10, which
18

---

Published as a conference paper at ICLR 2022
Table 12: Metrics on the DART dataset. Non-DP results from Hu al. (2021), except for GPT-2- XL, which was not reported in the paper. We ran GPT-2-XL with hyperparameters presented in Hu al. (2021). Bold indicates the best accuracy with DP. DP parameters are 6.8, ó 1e-5). Val perp stands for validation perplexity. Unlike all other metrics, the lower the TER metric is the better for the performance of the model.
Method
Val perp
BLEU
MET
TER
GPT-2-Small Small + DP
3.82
38.5
0.34
0.53
GPT-2-Mcdium + DP
3.30
42.0
0.36
0.51
GPT-2-Large DP
3.10
43.1
0.36
0.5
GPI-2-XL+ DP
3.00
43.8
0.37
0,5
GPT-2-Medium
2.67
47.1
0.39
0.46
GPT-2-Large
2.89
47.5
0.39
0.45
GPT-2-XL
2.83
48.1
0.39
0.46
shows that methods also perform very well under stronger privacy guarantees. A more inter- esting observation is that under smaller epsilon regimes, the performance for private fine-tuning of GPT-2-XL model improves. Observe that the performance improvement is more prominent going from GPT-2-Small to GPT-2-XL in this setting, which may indicate that larger models can be even more effective in private learning when the privacy budgets tight.
F.3 PERFORMING EXPERIMENTS ON DART DATASET
We study the DART dataset as text generation problem for private fine-tuning of GPT-2 series of models.
DART was introduced as an open-domain data-to-text dataset by Nan et al. (2021). The dataset consists of 62K training samples, 6.9K validation samples, and 12K test samples. com- parison to E2E, the dataset is larger and the task is more challenging.
DART:
Hyperparameter choice: The hyperparameter choices are similar as in the previous setting. The only difference is that use batch size 256 for the experiments DART. This achieves 6.8, =le-5 on DART using the accountant in Gopi et al. (2021).
Results: The results of our experiments summarized in the Table 12.
19


Reviews:
[{"review_id": "Z1WjbiL9Whn", "reviewer": "Reviewer_eqgr", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strength: \n+ The problem the paper aims to address is an important one. Machine learning with differential privacy is a very active research area, and this paper is a valuable addition to the literature in this field and could serve as a benchmark for further research in the area.\n\n+ The paper is well-written. The presentation is clear. It is a great pleasure to read through the manuscript. \n\n+ The experiments are well designed and executed. The results show improved privacy/utility trade-off over previous work.\n\nWeakness:\n- One major question I have is novelty. The idea to fine-tune large LMs on private datasets is not new [1]. So is the idea to hold most of the pretrained weights constant and only update a small subset of the parameters (e.g., [3]). Thus, this paper is a combination of existing state-of-the-art techniques on several fronts including LoRA [3] for dimension reduction and FFT based method (4) for privacy accounting. While this is solid engineering work and demonstrates the feasibility of integrating DP into NLP, I am not sure if it has enough novelty to be published as a separate research paper.\n\n\nReferences:\n[1] Differentially Private Language Models Benefit from Public Pre-training\n\n[2] Parameter-efficient transfer learning for nlp.\n\n[3] LoRA: Low-Rank Adaptation of Large Language Models\n\n[4] Numerical composition of differential privacy\n", "summary_of_the_paper": "This paper demonstrates the feasibility of fine-tuning large language models (pretrained on public dataset) on private datasets for various downstream tasks. It proposes a meta-framework in which most of the pretrained weights are held constant and only a small number of additional parameters are updated during fine-tuning. Combined with state-of-the-art dimension reduction and privacy accounting techniques, their method achieves good privacy/utility tradeoff on several GLEU benchmarks using RoBERTa and NLG using GPT-2.", "summary_of_the_review": "This paper presents solid engineering work for adopting DP in NLP tasks using large transformer-based language models pretrained on public datasets. Although the result is interesting, the paper is a combination of several existing techniques and do not have enough novelty to be published as a separate research paper.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "JqxAWYklDHT", "reviewer": "Reviewer_oFqs", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths\n1. The paper is well written and organized. The framework and the particular methods were explained clearly and the experiments were well executed, proving the claims in the paper.\n2. The proposed framework could have a lot of impact for NLP practitioners when training DP. Large models are notoriously difficult to train with DP from scratch. Given there are huge amounts of public texts and pretrained models in the NLP community, fine-tuning with privacy as proposed in this paper could be standard for learning on sensitive data. \n\nWeaknesses and Questions\n1. Technique novelty is a bit limited given that all the methods (Adapters, Compactors and LoRA) in the proposed framework are from prior non-private fine-tuning works. The only thing different is that this paper is more using these methods with DPSGD are effective. Could there be specific changes that are more effective just for private learning? \n2. Adapters, Compactors and LoRA are also not only parameter-efficient methods. How well might other sparsification techniques (e.g. lottery tickets [1], diff pruning [2]) compare to the proposed framework?\n3. Baseline comparison is lacking, although the authors provided the non-private utility baseline, all non-private numbers are learned with full SGD. It would make more sense to provide non-private numbers for the corresponding parameter-efficient methods.\n4. The comparison for RGP is also a bit unfair because it was originally proposed for training with privacy from scratch. Is it likely that training large models from scratch with DP could be better with the parameter-efficient frameworks as well?\n\nReferences\n\n[1] The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks. Jonathan Frankle, Michael Carbin\n\n[2] Parameter-Efficient Transfer Learning with Diff Pruning. Demi Guo, Alexander M. Rush, Yoon Kim", "summary_of_the_paper": "This paper introduces a parameter-efficient framework for privately fine-tuning large language models. The techniques were proposed in prior literature for non-private learning while the authors found that these techniques are very effective for training with differential privacy (DP), which was extremely hard for large models due to the amount of noise added. On common NLP benchmarks and with relatively small privacy budget, these parameter-efficient methods outperformed the regular DPSGD and could even be close to the non-private baseline, in terms of utility. These methods are also much more memory-efficient and faster than DPSGD as well.  \n", "summary_of_the_review": "Overall, I am leaning towards accepting this paper given the strong results and the hope for learning large models with differential privacy. The experiment section could be strengthened with better comparison to baseline methods.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "FGPqvWHO-dT", "reviewer": "Reviewer_BZJU", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Paper studies the problem of DP finetuning large language models, where DP is enforced during finetuning by using various methods (LoRA, RGP, Compacter). paper is very clearly written and easy to follow.\n\nAlthough one can say that the paper is mostly empirical and focused on showing that DP finetuning works when the curse of dimensionality for DPSGD is dealt with in one way or the other, I think it adds good value and a proof to the literature.\n\nI do not have any major concerns with the paper.", "summary_of_the_paper": "Paper studies the deferentially private fine-tuning of large language models and shows that privately finetuning language models can provide good utility. ", "summary_of_the_review": "I like the paper and think that it will add good value to the literature and the conference.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "2QTzqf9HgA", "reviewer": "Reviewer_hxyL", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Pros,\n1. The proposed method can handle various fine-tuning algorithms.\n2. The experimental results seem promising since the gaps between the proposed methods and the non-protection methods are not so large.\n3. The authors conduct experiments on multiple NLP tasks, including NLU and NLP tasks.\n4. The framework is also adaptive to other NLG methods.\n\nCons,\n1. Table 3 & 4 compare the proposed methods with the traditional usage (Full). \"Full\" acts as a baseline method. However, in Table 5 (NLG tasks), there's no baseline method for comparison.\n2. It will be better if the model can achieve good performnace when \\espilon is less than 5. Usually, \\espilon in a range of 0.1 ~ 5 can provide meaningful private protection. Also, the \\delta is large than it should be. The value of \\delta on the order of 1/|D_train| are very dangerous according to [2]. In that level of \\delta, they permit “preserving privacy” by publishing the complete records of a small number of database participants. (But in the experiments of E2E, the data size is 42000 and the \\delta is 1/50000.) Unsuitable \\epsilon and \\delta can ensure the utility (model performance) but the privacy protection becomes too weak.\n3. The authors claim that \"we are the first to fine-tune GPT-2-Large using differential privacy\". Actually, [2] also fine-tune on large GPT-2 with DP-SGD.\n4. There's some hyper-parameters play an important role for model training. But, tuning hyper-parameters requires additional information about the private information (accessing validation set or testing set), which leads to private leakage. So, how to count the information leakags of users? How to avoid it?\n5. This paper propose a meta-framework that handles several existing fine-tuning algorithms with DP-SGD. The performance is amazing. However, the contribution in terms of the technical novelty is not much.\n\n[1] The Algorithmic Foundations of Differential Privacy, 2014.\n\n[2] Differentially Private Language Models Benefit from Public Pre-training. 2020.", "summary_of_the_paper": "In this paper, the authors propose a meta-framework that applies DP-SGD to NLP tasks. As a DP learning algorithm, DP-SGD provides a practical method to protect the privacy of the training samples of the deep learning model. The framework works on the fine-tuning phase of the pre-trained language model and protects the privacy of the fine-tuning datasets. To adapt DP-SGD to NLP models, the authors propose to fine-tune only a part of the parameters in the large pre-trained language model. The proposed meta-framework can handle various fine-tuning algorithms. The experimental results show the proposed method can achieve performance very close to the non-protection algorithms.", "summary_of_the_review": "This paper proposes a meta-framework carrying some existing fine-tune methods. The performance looks good but there are still some issues. The technical novelty is limited.\n\nThe rebuttal solved most of my concerns. Thanks for the author's efforts!", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works."}, "extra_text": {}}]

Author rebuttal:

Meta-review: Discussions and additional baseline experiments added during the author response period were enough to motivate multiple reviewers to change their recommendation to an accept during the author response. Multiple reviewers felt that the technical novelty of the work was limited, but the rebuttal cleared up their concerns enough to motivate them to switch their assessments to accept.

The claim of this work is that it provides a simpler, sparser, and faster algorithms for differentially private fine tuning of LLMs, yielding SOTA privacy results vs. utility on a number of standard NLP tasks. The work proposes a meta-framework. 

In the end, all reviewers rated this paper as an accept and the AC also recommends acceptance.
Decision: Accept (Poster)