Original: D IFFERENTIALLY P RIVATE F INE - TUNING OF L ANGUAGE M ODELS
Revised: D IFFERENTIALLY P RIVATE F INE - TUNING OF L ANGUAGE M ODELS

Original: Anonymous authors

Original: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: We give simpler, sparser, and faster algorithms for differentially private ﬁnetuning of large-scale pre-trained language models, which achieve the state -ofthe-art privacy versus utility tradeoffs on many standard NLP tasks
Revised: We give simpler, sparser, and faster algorithms for differentially private finetuning of large-scale pre-trained language models, which achieve the state-ofthe-art privacy versus utility tradeoffs on many standard NLP tasks.

Original: We propose a meta-framework for this problem, inspired by the recent success of highly parameter-efﬁcient methods for ﬁne-tuning.
Revised: We propose a meta-framework for this problem, inspired by the recent success of highly parameter-efficient methods for fine-tuning.

Original: Our experiments show that differentially private adaptations of these approaches outperform previous private algorithms in three important dimensions: utility, privacy and the computational and memory cost of private training.
Revised: Our experiments show that differentially private adaptations of these approaches outperform previous private algorithms in three important dimensions utility privacy, and the computational and memory cost of private training.

Original: On many commonly studied datasets, the utility of private models approaches that of non-private models.
Revised: On many commonly studied datasets the utility of private models approaches that of non-private models.

Original: For example, on the
Revised: For example on the

Original: MNLI dataset we achieve an accuracy of using
Revised: MNLI dataset we achieve an accuracy of 87 .

Original: RoBERTa -Large and 83 .
Revised: 8% using RoBERTa-Large and 83 .

Original: 5% using RoBERTa -Base with a privacy budget of ε = 
Revised: 5% using RoBERTa-Base with a privacy budget of ε = 6 . 7 .

Original: In comparison, absent privacy constraints RoBERTa Large achieves an accuracy of 90 . 
Revised: In comparison, absent privacy constraints, RoBERTa-Large achieves an accuracy of 90 . 2% .

Original: Our ﬁndings are similar for natural language generation when privately ﬁne-tuning GPT-2.
Revised: Our findings are similar for natural language generation when privately fine-tuning GPT-2.

Original: Our experiments also show that larger models are better suited for private ﬁne-tuning: while they are well known to achieve superior accuracy non-privately, we ﬁnd that they also better maintain their accuracy when privacy is introduced
Revised: Our experiments also show that larger models are better suited for private fine-tuning: while they are well known to achieve superior accuracy non-privately, we find that they also better maintain their accuracy when privacy is introduced.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: Deep learning models are well known to leak sensitive information about the dataset when trained using conventional methods (Shokri et al., 2017; Carlini et al., 2019; 2021).
Revised: Deep learning models are well known to leak sensitive information about the dataset when trained using conventional methods (Shokri et al., 2017; Carlini et al., 2019; 2021).

Original: To combat this issue, models can instead be trained to guarantee differential privacy (DP) (Dwork et al., 2006b), a strong notion of data privacy which limits the inﬂuence of any individual training point on the ﬁnal model.
Revised: To combat this issue, models can instead be trained to guarantee differential privacy (DP) (Dwork et al., 2006b), a strong notion of data privacy which limits the influence of any individual training point on the final model.

Original: While DP is one of the few approaches capable of providing machine learning models with rigorous privacy guarantees, it generally comes at a cost in terms of test accuracy.
Revised: While DP is one of the few approaches capable of providing machine learning models with rigorous privacy guarantees, it generally comes at a cost in terms of test accuracy.

Original: One oft-cited explanation is that the constraint of DP necessitates much more training data (Tram ` er & Boneh, 2021; Feldman, 2020; Brown et al., 2021).
Revised: One oft-cited explanation is that the constraint of DP necessitates much more training data (Tram`er & Boneh, 2021; Feldman, 2020; Brown et al., 2021).

Original: Unfortunately, more training data may be hard to acquire, particularly in settings where privacy is a concern.
Revised: Unfortunately, more training data may be hard to acquire, particularly in settings where privacy is a concern.

Original: Parallel to these developments, Transformer-based (Vaswani et al., 2017) large language models (LLMs), including the BERT (Devlin et al., 2019; Liu et al., 2019) and GPT (Radford et al., 2018; 2019; Brown et al., 2020) families, have enabled signiﬁcant progress in natural language processing, achieving state-of-the-art accuracy in almost every task considered. 
Revised: Parallel to these developments, Transformer-based (Vaswani et al., 2017) large language models (LLMs), including the BERT (Devlin et al., 2019; Liu et al., 2019) and GPT (Radford et al., 2018; 2019; Brown et al., 2020) families, have enabled significant progress in natural language processing, achieving state-of-the-art accuracy in almost every task considered. These models are first

Revised: Next, new parameters are introduced and privately fine-tuned on a smaller, private,

Revised: The parameters are frozen during this process.

Revised: Finally, the fine-tuned new parameters may be released publicly and plugged-in to the model for downstream tasks preserving

Original: These models are ﬁrst pre-trained on an extremely large and diverse public dataset.
Revised: [Table] [Table] pre-trained on an extremely large and diverse public dataset.

Original: The weights are then ﬁne-tuned for each task of interest using a much smaller task-speciﬁc dataset.
Revised: The weights are then fine-tuned for each task of interest using a much smaller task-specific dataset.

Original: For example, a single pre-trained
Revised: For example, a single pre-trained

Original: GPT-family model may be ﬁne-tuned for various downstream tasks, such as email reply suggestion, sentence completion in text editors, language translation, and more.
Revised: GPT-family model may be fine-tuned for various downstream tasks, such as email reply suggestion, sentence completion in text editors, language translation, and more.

Original: This two-stage paradigm can naturally be adapted to solve tasks in private learning, automatically addressing the aforementioned data shortage issue via the massive scale of the public pre-training dataset.
Revised: This two-stage paradigm can naturally be adapted to solve tasks in private learning, automatically addressing the aforementioned data shortage issue via the massive scale of the public pre-training dataset.

Original: One may pre-train the model on public data as usual, 1{+
+} but then ﬁne-tune the model privately .
Revised: One may pre-train the model on public data as usual, {+
+} but then fine-tune the model privately .

Original: Despite the success of these models, task-speciﬁc ﬁne-tuning introduces a number of technical challenges.
Revised: Despite the success of these models, task-specific fine-tuning introduces a number of technical challenges.

Original: In the non-private setting, the immense size of LLMs makes it impractical to ﬁne-tune the Next, new parameters are introduced and privately ﬁne-tuned on a are frozen during this Finally, the-tuned new parametersmay be released publicly and plugged-in to the model for still
Revised: In the non-private setting, the immense size of LLMs makes it impractical to fine-tune the full model and store a separate copy of the parameters for hundreds of downstream tasks.

Original: [Table]

Original: Our results ( ε = 6 . 7 )

Original: [Table]

Original: full model and store a separate copy of the parameters for hundreds of downstream tasks.

Original: Things only get worse with privacy, which leads to overheads in terms of running time, memory usage, and most importantly, accuracy.
Revised: Things only get worse with privacy, which leads to overheads in terms of running time, memory usage, and most importantly, accuracy.

Original: The magnitude of noise introduced to a model due to DP grows as the model size increases (Bassily et al., 2014; Abadi et al., 2016; Bun et al., 2014), which can overwhelm any signal for larger models.
Revised: The magnitude of noise introduced to a model due to DP grows as the model size increases (Bassily et al., 2014; Abadi et al., 2016; Bun et al., 2014), which can overwhelm any signal for larger models.

Original: A recent line of work in the non-private literature has proposed parameter-efﬁcient methods to alleviate the issues of storage and computational cost for ﬁne-tuning (Houlsby et al., 2019; Li & Liang, 2021;
Revised: A recent line of work in the non-private literature has proposed parameter-efficient methods to alleviate the issues of storage and computational cost for fine-tuning (Houlsby et al., 2019; Li & Liang, 2021;

Original: Aghajanyan et al., 2020; Hu et al., 2021; Mahabadi et al., 2021).
Revised: Aghajanyan et al., 2020; Hu et al., 2021; Mahabadi et al., 2021).

Original: The main focus of our work is to explore parameter-efﬁciency in the context of private learning.
Revised: The main focus of our work is to explore parameter-efficiency in the context of private learning.

Original: 1. O UR C ONTRIBUTIONS
Revised: 1. O UR C ONTRIBUTIONS

Original: Our primary contribution is to show that advanced parameter-efﬁcient methods can lead to simpler and signiﬁcantly improved algorithms for private ﬁne-tuning.
Revised: Our primary contribution is to show that advanced parameter-efficient methods can lead to simpler and significantly improved algorithms for private fine-tuning.

Original: Our framework is illustrated in
Revised: Our framework is illustrated in

Original: Figure 1.
Revised: Figure 1.

Original: Our ﬁndings and contributions are summarized as follows:
Revised: Our findings and contributions are summarized as follows:

Original: State-of-the-art utility and privacy.
Revised: State-of-the-art utility and privacy.

Original: Empirical evaluation of our algorithms reveals that they achieve state-of-the-art accuracy versus privacy tradeoffs, improving upon the previous best (Yu et al., 2021b).
Revised: Empirical evaluation of our algorithms reveals that they achieve state-of-the-art accuracy versus privacy tradeoffs, improving upon the previous best (Yu et al., 2021b).

Original: More importantly, for many ﬁne-tuning tasks, the utility of models trained with DP approaches that of non-private models.
Revised: More importantly, for many fine-tuning tasks, the utility of models trained with DP approaches that of non-private models.

Original: For example, privately ﬁne-tuning RoBERTa-Large on the
Revised: For example, privately fine-tuning RoBERTa-Large on the

Original: MNLI data set (Williams et al., 2018), we achieve an accuracy of 87 .
Revised: MNLI data set (Williams et al., 2018), we achieve an accuracy of 87 .

Original: 8% with a privacy budget of ( ε = 6 . 7 , δ = 1e-6).
Revised: 8% with a privacy budget of ( ε = 6 . 7 , δ = 1e-6).

Original: Without privacy guarantees, RoBERTa-Large achieves an accuracy of 90 .
Revised: Without privacy guarantees, RoBERTa-Large achieves an accuracy of 90 .

Original: 2% (GPT-3 is known to achieve 91 .
Revised: 2% (GPT-3 is known to achieve 91 .

Original: 7% (Hu et al., 2021)); see Table 1 for a summary.
Revised: 7% (Hu et al., 2021)); see Table 1 for a summary.

Original: We also explore private natural language generation tasks, ﬁne-tuning GPT-2 models on the E2E dataset (Novikova et al., 2017).
Revised: We also explore private natural language generation tasks, fine-tuning GPT-2 models on the E2E dataset (Novikova et al., 2017).

Original: Again, the utility approaches non-private levels: we achieve a ROUGE-L score of 0.6755 with GPT-2-Large and ( ε = 5 . 4 , δ = 1e-5), compared to 0.72 without privacy.
Revised: Again, the utility approaches non-private levels: we achieve a ROUGE-L score of 0.6755 with GPT-2-Large and ( ε = 5 . 4 , δ = 1e-5), compared to 0.72 without privacy.

Original: Larger models are better.
Revised: Larger models are better.

Original: Prior work has consistently shown that larger language models achieve better accuracy for downstream tasks.
Revised: Prior work has consistently shown that larger language models achieve better accuracy for downstream tasks.

Original: Our results give evidence that this phenomenon extends to the private setting.
Revised: Our results give evidence that this phenomenon extends to the private setting.

Original: For example, on the MNLI dataset, RoBERTa-Base achieves an accuracy of 83 .
Revised: For example, on the MNLI dataset, RoBERTa-Base achieves an accuracy of 83 .

Original: 5% whereas RoBERTa-Large achieves an accuracy of 87 .
Revised: 5% whereas RoBERTa-Large achieves an accuracy of 87 .

Original: 8% , both under a privacy budget of ( ε = 6 . 7 , δ = 1e-6).
Revised: 8% , both under a privacy budget of ( ε = 6 . 7 , δ = 1e-6).

Original: Similarly, privately ﬁne-tuning with E2E, GPT-2-Small, GPT-2-Medium, and GPT-2-Large achieve ROUGE-L scores of 0.6219, 0.6645 and 0.6755 respectively, all under a privacy budget of ( ε = 5 . 4 , δ = 1e-5).
Revised: Similarly, privately fine-tuning with E2E, GPT-2-Small, GPT-2-Medium, and GPT-2-Large achieve ROUGE-L scores of 0.6219, 0.6645 and 0.6755 respectively, all under a privacy budget of ( ε = 5 . 4 , δ = 1e-5).

Original: While established in the non-private setting, we ﬁnd this phenomenon quite surprising under DP.
Revised: While established in the non-private setting, we find this phenomenon quite surprising under DP.

Original: There is often a tension when choosing private model architectures: larger models may have higher capacity, but necessitate the introduction of more noise.
Revised: There is often a tension when choosing private model architectures: larger models may have higher capacity, but necessitate the introduction of more noise.

Original: Consequently, smaller and simpler private models achieve the better accuracy in several settings (Papernot et al., 2019; Tram`er & Boneh, 2021).
Revised: Consequently, smaller and simpler private models achieve the better accuracy in several settings (Papernot et al., 2019; Tram`er & Boneh, 2021).

Original: In contrast, our experiments show that ﬁne-tuning the biggest models achieves the best accuracy.
Revised: In contrast, our experiments show that fine-tuning the biggest models achieves the best accuracy.

Original: Simpler, sparser, and faster.
Revised: Simpler, sparser, and faster.

Original: Beyond accuracy concerns, DP requirements also lead to signiﬁcant overheads in terms of computation and memory usage.
Revised: Beyond accuracy concerns, DP requirements also lead to significant overheads in terms of computation and memory usage.

Original: The large number of parameters contributes to the high cost of training LLMs, and things get worse under privacy, which has been documented to increase training time by up to two orders of magnitude (Carlini et al., 2019; Subramani et al., 2021).
Revised: The large number of parameters contributes to the high cost of training LLMs, and things get worse under privacy, which has been documented to increase training time by up to two orders of magnitude (Carlini et al., 2019; Subramani et al., 2021).

Original: The parameter-efﬁcient approaches we employ partially offset this issue: as we only update a small fraction of the total number of parameters, training becomes considerably more computationally and memory efﬁcient.
Revised: The parameter-efficient approaches we employ partially offset this issue: as we only update a small fraction of the total number of parameters, training becomes considerably more computationally and memory efficient.

Original: Furthermore, as in the non-private setting, this framework leads to a modular design, where a single large pre-trained model can be augmented with lightweight modiﬁcations for each individual downstream task.
Revised: Furthermore, as in the non-private setting, this framework leads to a modular design, where a single large pre-trained model can be augmented with lightweight modifications for each individual downstream task.

Original: To the best of our knowledge, we are the ﬁrst to ﬁne-tune GPT-2-Large using differential privacy, the largest model trained thus far using DP.
Revised: To the best of our knowledge, we are the first to fine-tune GPT-2-Large using differential privacy, the largest model trained thus far using DP.

Original: Given our state-of-the-art results for a variety of standard NLP tasks using advanced ﬁne-tuning techniques, we believe that our paper will serve as a benchmark for further work in this direction.
Revised: Given our state-of-the-art results for a variety of standard NLP tasks using advanced fine-tuning techniques, we believe that our paper will serve as a benchmark for further work in this direction.

Original: For example, the best average accuracy achieved by the prior work of Yu et al.
Revised: For example, the best average accuracy achieved by the prior work of Yu et al.

Original: (2021b) on four standard NLP tasks in Table 1 is 83 .
Revised: (2021b) on four standard NLP tasks in Table 1 is 83 .

Original: 9% using ε = 8 (and the same δ as in Table 1), whereas we can achieve an average accuracy of 90 .
Revised: 9% using ε = 8 (and the same δ as in Table 1), whereas we can achieve an average accuracy of 90 .

Original: 3% using ε = 6 .
Revised: 3% using ε = 6 .

Original: 7 by a combination of better algorithms, larger models, and new privacy accounting techniques.
Revised: 7 by a combination of better algorithms, larger models, and new privacy accounting techniques.

Original: Finally, though recently considered elsewhere (see Section 5), we put further focus on the framing of public pre-training and private ﬁne-tuning as an important conceptual direction in DP deep learning.
Revised: Finally, though recently considered elsewhere (see Section 5), we put further focus on the framing of public pre-training and private fine-tuning as an important conceptual direction in DP deep learning.

Original:  P RELIMINARIES AND P RIOR A LGORITHM B ASELINES
Revised:  P RELIMINARIES AND P RIOR A LGORITHM B ASELINES

Original: Recall the formal deﬁnition of differential privacy.
Revised: Recall the formal definition of differential privacy.

Original: Deﬁnition 2.1 (Differential Privacy (DP) (Dwork et al., 2006b;a)) .
Revised: Definition 2.1 (Differential Privacy (DP) (Dwork et al., 2006b;a)) .

Original: A randomized algorithm
Revised: A randomized algorithm

Original: A is ( ε , δ )-differentially private if for any two neighboring datasets D and D (cid:48) , which differ in exactly the data pertaining to a single user, and for all sets S of possible outputs:Pr[ A ( D ) [Equation]
Revised: A is ( ε , δ )-differentially private if for any two neighboring datasets D and D ′ , which differ in exactly the data pertaining to a single user, and for all sets S of possible outputs:

Original: [Equation] e ε Pr[ A ( D (cid:48) ) ∈ S ] + δ .
Revised: Pr[ A ( D ) ∈ S ] ≤ e ε Pr[ A ( D ′ ) ∈ S ] + δ .

Original: We review prior techniques for private ﬁne-tuning.
Revised: We review prior techniques for private fine-tuning.

Original: 2. F ULL F INE - TUNING VIA DPSGD
Revised: 2. F ULL F INE - TUNING VIA DPSGD

Original: To train a machine learning model with privacy, the most popular algorithm is the celebrated DP stochastic gradient descent (DPSGD)
Revised: To train a machine learning model with privacy, the most popular algorithm is the celebrated DP stochastic gradient descent (DPSGD)

Original: (Song et al., 2013; Bassily et al., 2014; Abadi et al., 2016).
Revised: (Song et al., 2013; Bassily et al., 2014; Abadi et al., 2016).

Original: This optimization method serves as a drop-in replacement for SGD, augmenting it with the addition of per-example gradient clipping and Gaussian noise addition steps.
Revised: This optimization method serves as a drop-in replacement for SGD, augmenting it with the addition of per-example gradient clipping and Gaussian noise addition steps.

Original: These two steps serve to limit and mask the contribution of a single example.
Revised: These two steps serve to limit and mask the contribution of a single example.

Original: Two key points to note are that a) per-example gradient clipping incurs signiﬁcant computational and memory overheads in most implementations, and b) noise introduced due to privacy grows as the square-root as the number of model parameters.
Revised: Two key points to note are that a) per-example gradient clipping incurs significant computational and memory overheads in most implementations, and b) noise introduced due to privacy grows as the square-root as the number of model parameters.

Original: With this tool in place, the most basic ﬁne-tuning strategy is to train all parameters using DPSGD.
Revised: With this tool in place, the most basic fine-tuning strategy is to train all parameters using DPSGD.

Original: 2. R EPARAMETRIZED G RADIENT P ERTURBATION
Revised: 2. R EPARAMETRIZED G RADIENT P ERTURBATION

Original: To mitigate the limitations of DPSGD, a recent work of Yu et al.
Revised: To mitigate the limitations of DPSGD, a recent work of Yu et al.

Original: (2021b) introduced an elegant method called reparametrized gradient perturbation (RGP).
Revised: (2021b) introduced an elegant method called reparametrized gradient perturbation (RGP).

Original: RGP exploits the implicit low-rank structure in the gradient updates of SGD to substantially improve upon DPSGD.
Revised: RGP exploits the implicit low-rank structure in the gradient updates of SGD to substantially improve upon DPSGD.

Original: Speciﬁcally, they reparametrize each layer’s weight matrix W into LR + ˜ W , where L and R are low-rank gradientcarrier matrices and ˜ W is the residual weight.
Revised: Specifically, they reparametrize each layer’s weight matrix W into LR + ˜ W , where L and R are low-rank gradientcarrier matrices and ˜ W is the residual weight.

Original: The authors show that one can obtain a lowdimensional projection of W ’s gradient by taking gradients only of the low-rank matrices L and R (and not the high-rank ˜ W ).
Revised: The authors show that one can obtain a lowdimensional projection of W ’s gradient by taking gradients only of the low-rank matrices L and R (and not the high-rank ˜ W ).

Original: Privacy is introduced by clipping and noising these low-dimensional gradients of L and R .
Revised: Privacy is introduced by clipping and noising these low-dimensional gradients of L and R .

Original: While this low-dimensional projection loses some of the signal in W ’s gradient, it turns out to contain enough to still achieve high accuracy.
Revised: While this low-dimensional projection loses some of the signal in W ’s gradient, it turns out to contain enough to still achieve high accuracy.

Original: At the same time, the low-dimensional gradients alleviate the aforementioned issues related to privatization, signiﬁcantly reducing the memory consumption and noise introduced.
Revised: At the same time, the low-dimensional gradients alleviate the aforementioned issues related to privatization, significantly reducing the memory consumption and noise introduced.

Original:  O UR A PPROACH 3. A M ETA - FRAMEWORK
Revised:  O UR A PPROACH 3. A M ETA - FRAMEWORK

Original: We introduce our approach as a meta-framework for private deep learning, which abstracts the key principles of recent ﬁne-tuning methods.
Revised: We introduce our approach as a meta-framework for private deep learning, which abstracts the key principles of recent fine-tuning methods.

Original: Suppose f ( W PT ; x ) is a pre-trained model where W PT are the pre-trained weights and x is any input.
Revised: Suppose f ( W PT ; x ) is a pre-trained model where W PT are the pre-trained weights and x is any input.

Original: We create a new ﬁne-tuning model
Revised: We create a new fine-tuning model

Original: [Equation]
Revised: [Equation]

Original: which incorporates additional trainable parameters θ , where dim( θ ) (cid:28) dim( W PT )
Revised: which incorporates additional trainable parameters θ , where dim( θ ) ≪ dim( W PT )

Original: That is, the number of new parameters in θ is a small fraction of the original number of parameters in the pretrained weights W PT .
Revised: That is, the number of new parameters in θ is a small fraction of the original number of parameters in the pretrained weights W PT .

Original: Fine-tuning is done by running DPSGD on the additional parameters θ , while freezing the weights of pre-trained model W PT .
Revised: Fine-tuning is done by running DPSGD on the additional parameters θ , while freezing the weights of pre-trained model W PT .

Original: The new parameters are initialized to θ 0 such that
Revised: The new parameters are initialized to θ 0 such that

Original: [Equation]
Revised: [Equation]

Original: The initialization condition (2) is very important, as it ensures that ﬁne-tuning starts at the pre-trained model and improves it by modifying the parameters θ .
Revised: The initialization condition (2) is very important, as it ensures that fine-tuning starts at the pre-trained model and improves it by modifying the parameters θ .

Original: Most ﬁne-tuning methods are additive and have the following special form:
Revised: Most fine-tuning methods are additive and have the following special form:

Original: [Equation]
Revised: [Equation]

Original: i.e., they modify the pre-trained weights by adding a correction term π ( θ ) parametrized by θ .
Revised: i.e., they modify the pre-trained weights by adding a correction term π ( θ ) parametrized by θ .

Original: Recent work in the non-private literature has described concrete instantiations of this framework (Houlsby et al., 2019; Mahabadi et al., 2021;
Revised: Recent work in the non-private literature has described concrete instantiations of this framework (Houlsby et al., 2019; Mahabadi et al., 2021;

Original: Hu et al., 2021), which (crucially) are effective when dim( θ ) (cid:28) dim( W PT ) .
Revised: Hu et al., 2021), which (crucially) are effective when dim( θ ) ≪ dim( W PT ) .

Original: In the non-private setting, such reparametrizations are useful for reducing the computation and memory required for ﬁne-tuning, and enable lightweight and plug-in modiﬁcations to the base model for different downstream tasks.
Revised: In the non-private setting, such reparametrizations are useful for reducing the computation and memory required for fine-tuning, and enable lightweight and plug-in modifications to the base model for different downstream tasks.

Original: At the same time, they maintain (or sometimes surpass) the accuracy achieved by full ﬁne-tuning.
Revised: At the same time, they maintain (or sometimes surpass) the accuracy achieved by full fine-tuning.

Original: However, this approach seems to be more important for achieving high accuracy in the context of private ﬁne-tuning.
Revised: However, this approach seems to be more important for achieving high accuracy in the context of private fine-tuning.

Original: For instance, full ﬁne-tuning of RoBERTa-Base with DPSGD achieves an accuracy of 53 . 1% for the MNLI dataset (Yu et al., 2021b), whereas all the algorithms that ﬁne-tune based on Equation (1) all achieve accuracy better than 80% .
Revised: For instance, full fine-tuning of RoBERTa-Base with DPSGD achieves an accuracy of 53 . 1% for the MNLI dataset (Yu et al., 2021b), whereas all the algorithms that fine-tune based on Equation (1) all achieve accuracy better than 80% .

Original: We give some intuition as to why parameter-efﬁcient methods seem to be more effective for private ﬁne-tuning.
Revised: We give some intuition as to why parameter-efficient methods seem to be more effective for private fine-tuning.

Original: For simplicity, we assume that the ﬁne-tuning method is additive as in (3), such that the ﬁne-tuned weights W FT = W PT + π ( θ ) .
Revised: For simplicity, we assume that the fine-tuning method is additive as in (3), such that the fine-tuned weights W FT = W PT + π ( θ ) .

Original: We can imagine that W FT lies on a manifold passing through W PT of very small dimension (equal to the dimension of θ ) compared to the dimension of
Revised: We can imagine that W FT lies on a manifold passing through W PT of very small dimension (equal to the dimension of θ ) compared to the dimension of

Original: W PT .
Revised: W PT .

Original: Even if the parameters θ are very noisy due to the noise added during DPSGD, we will always stay in this manifold.
Revised: Even if the parameters θ are very noisy due to the noise added during DPSGD, we will always stay in this manifold.

Original: In particular, we are not disturbing the pre-trained weights in most directions (those orthogonal to the manifold near W PT ).
Revised: In particular, we are not disturbing the pre-trained weights in most directions (those orthogonal to the manifold near W PT ).

Original: If we run DPSGD on all the weights instead, then we add noise in all directions, thus potentially unlearning the knowledge learned during pre-training, especially in low data regimes.
Revised: If we run DPSGD on all the weights instead, then we add noise in all directions, thus potentially unlearning the knowledge learned during pre-training, especially in low data regimes.

Original: Besides substantial gains in the accuracy, the above method of reparametrization has several other advantages:
Revised: Besides substantial gains in the accuracy, the above method of reparametrization has several other advantages:

Original: • A single pre-trained model such as BERT or GPT is generally applied to hundreds of downstream tasks via ﬁne-tuning.
Revised: • A single pre-trained model such as BERT or GPT is generally applied to hundreds of downstream tasks via fine-tuning.

Original: Private ﬁne-tuning using previous methods requires updating all parameters and storing a different copy of the ﬁne-tuned model per task.
Revised: Private fine-tuning using previous methods requires updating all parameters and storing a different copy of the fine-tuned model per task.

Original: This creates substantial overheads for storing and deploying, and can be very expensive in practice.
Revised: This creates substantial overheads for storing and deploying, and can be very expensive in practice.

Original: On the other hand, the reparametrization (1) means that we only need to store a single pretrained model that can be shared across many downstream tasks.
Revised: On the other hand, the reparametrization (1) means that we only need to store a single pretrained model that can be shared across many downstream tasks.

Original: Each downstream task requires only a small number of new parameters that can be plugged in.
Revised: Each downstream task requires only a small number of new parameters that can be plugged in.

Original: • Differentially private training requires computing and storing per-example gradients, which increases the memory footprint.
Revised: • Differentially private training requires computing and storing per-example gradients, which increases the memory footprint.

Original: In our approach, however, learning is done in a much lower dimension, hence saving on the memory cost as compared to prior works.
Revised: In our approach, however, learning is done in a much lower dimension, hence saving on the memory cost as compared to prior works.

Original: Finally, we expect that (1) also gives a more communication-efﬁcient method of ﬁne-tuning in distributed settings such as federated learning, due to the signiﬁcantly smaller number of parameters learned during ﬁne-tuning.
Revised: Finally, we expect that (1) also gives a more communication-efficient method of fine-tuning in distributed settings such as federated learning, due to the significantly smaller number of parameters learned during fine-tuning.

Original: 3. I NSTANTIATING THE M ETA - FRAMEWORK
Revised: 3. I NSTANTIATING THE M ETA - FRAMEWORK

Original: In this section, we discuss a few ways to instantiate our meta-framework.
Revised: In this section, we discuss a few ways to instantiate our meta-framework.

Original: This list is non-exhaustive, but covers the methods we employ in our experiments.
Revised: This list is non-exhaustive, but covers the methods we employ in our experiments.

Original: 3.2. F INE - TUNING VIA L OW -R ANK A DAPTATION
Revised: 3.2. F INE - TUNING VIA L OW -R ANK A DAPTATION

Original: Low-Rank Adaptation (LoRA) (Hu et al., 2021) is an additive ﬁne-tuning scheme as deﬁned in (3).
Revised: Low-Rank Adaptation (LoRA) (Hu et al., 2021) is an additive fine-tuning scheme as defined in (3).

Original: For each dense weight matrix W i PT of size a × b in the pre-trained network, we add a low-rank correction term L i R i , i.e.,
Revised: For each dense weight matrix W i PT of size a × b in the pre-trained network, we add a low-rank correction term L i R i , i.e.,

Original: [Equation]
Revised: [Equation]

Original: (4) where L i ∈ R a × r , 
Revised: 4) where L i ∈ R a × r , R

Original: R i ∈
Revised: i ∈

Original: R r × b are new trainable parameters.
Revised: R r × b are new trainable parameters.

Original: Hu et al.
Revised: Hu et al.

Original: (2021) apply this reparameterization only to the Transformer attention weights ( W q , W v ), and freeze all other weights (e.g., W k and W o and those in the feed-forward layers).
Revised: (2021) apply this reparameterization only to the Transformer attention weights ( W q , W v ), and freeze all other weights (e.g., W k and W o and those in the feed-forward layers).

Original: The rank r is typically chosen to be small, e.g., r = 4 , 16 , 64 .
Revised: The rank r is typically chosen to be small, e.g., r = 4 , 16 , 64 .

Original: Since most parameters in Transformer architectures are dense weight matrices, choosing a small r results in a nearly square-root reduction in the number of parameters.
Revised: Since most parameters in Transformer architectures are dense weight matrices, choosing a small r results in a nearly square-root reduction in the number of parameters.

Original: 3.2. F INE - TUNING VIA A DAPTERS
Revised: 3.2. F INE - TUNING VIA A DAPTERS

Original: Houlsby et al.
Revised: Houlsby et al.

Original: (2019) propose adapter-based ﬁne-tuning, in which we modify the architecture of the pre-trained model by adding new “adapter” layers after each attention and feed-forward layer.
Revised: (2019) propose adapter-based fine-tuning, in which we modify the architecture of the pre-trained model by adding new “adapter” layers after each attention and feed-forward layer.

Original: Adapter layers are bottleneck layers with residual connections.
Revised: Adapter layers are bottleneck layers with residual connections.

Original: Speciﬁcally, given an input x , an adapter layer A performs
Revised: Specifically, given an input x , an adapter layer A performs

Original: [Equation]
Revised: [Equation]

Original: where U is an up-projection afﬁne linear map, D is a down-projection afﬁne linear map, and τ is a non-linear activation function such as the Gaussian error Linear Unit (GeLU)
Revised: where U is an up-projection affine linear map, D is a down-projection affine linear map, and τ is a non-linear activation function such as the Gaussian error Linear Unit (GeLU)

Original: (Hendrycks &
Revised: (Hendrycks &

Original: Gimpel, 2016).
Revised: Gimpel, 2016).

Original: If x has dimension d , then U ∈ R d × r , D ∈ R r × d for some r (cid:28) d .
Revised: If x has dimension d , then U ∈ R d × r , D ∈ R r × d for some r ≪ d .

Original: Thus, the number of introduced parameters is signiﬁcantly less than the number of parameters in the pre-trained model.
Revised: Thus, the number of introduced parameters is significantly less than the number of parameters in the pre-trained model.

Original: When ﬁne-tuning, the parameters of the original model are frozen, and only parameters of the adapter layers, as well as layer normalizations, aremodiﬁed.
Revised: When fine-tuning, the parameters of the original model are frozen, and only parameters of the adapter layers, as well as layer normalizations, are

Original: Note that ﬁne-tuning with adapters is not an additive ﬁne-tuning framework as in (3), but is captured by the broader framework in (1).
Revised: [Table]

Original: 3.2. F INE - TUNING VIA C OMPACTER
Revised: F INE - TUNING VIA C OMPACTER

Original: The recent work of Mahabadi et al.
Revised: The recent work of Mahabadi et al.

Original: (2021) introduces Compacters (Compact adapters), a method which further improves the parameter efﬁciency of adapters.
Revised: (2021) introduces Compacters (Compact adapters), a method which further improves the parameter efficiency of adapters.

Original: This is done by replacing the dense matrices in the up-projection U and down-projection D by tensor products of smaller matrices, thus reducing the number of trainable parameters.
Revised: This is done by replacing the dense matrices in the up-projection U and down-projection D by tensor products of smaller matrices, thus reducing the number of trainable parameters.

Original: Speciﬁcally, they replace the dense matrix M (cid:96) in the adapter layer (cid:96) by a low-rank parameterized hypercomplex multiplication (LPHM) layer, i.e., each dense matrix M (cid:96) ∈ R a × b is expressed as
Revised: Specifically, they replace the dense matrix M ℓ in the adapter layer ℓ by a low-rank parameterized hypercomplex multiplication (LPHM) layer, i.e., each dense matrix M ℓ ∈ R a × b is expressed as

Original: [Equation]
Revised: [Equation]

Original: [Table] where A i ∈ R n × n , S (cid:96)i ∈ R a/n × k , T (cid:96)i ∈ R k × b/n and ⊗ is the matrix Kronecker product.
Revised: where A i ∈ R n × n , S ℓi ∈ R a/n × k , T ℓi ∈ R k × b/n and ⊗ is the matrix Kronecker product.

Original: Note the matrices A i are not indexed by the layer (cid:96) because these matrices are shared among all the adapter layers.
Revised: Note the matrices A i are not indexed by the layer ℓ because these matrices are shared among all the adapter layers.

Original: Since each adapter layers has two dense matrices (one for up-projection and one for down-projection), if there are L adapter layers, this reduces the number of parameters from L (2 ab ) to L (2( a + b ) k ) + n 3 .
Revised: Since each adapter layers has two dense matrices (one for up-projection and one for down-projection), if there are L adapter layers, this reduces the number of parameters from L (2 ab ) to L (2( a + b ) k ) + n 3 .

Original: In practice, a and b are chosen to be either the model dimension d or the intermediate representation dimension r in the adapters, n is typically chosen to be a small constant such as n = 2 , 4 , 8 , 12 and k is chosen to be 1 .
Revised: In practice, a and b are chosen to be either the model dimension d or the intermediate representation dimension r in the adapters, n is typically chosen to be a small constant such as n = 2 , 4 , 8 , 12 and k is chosen to be 1 .

Original: 3.2. W HY D OES P ARAMETER -E FFICIENT T UNING W ORK ?
Revised: [Equation] FFICIENT T UNING W ORK ?

Original: Theoretical explanation of success of parameter-efﬁcient ﬁne-tuning methods is active area of research in deep learning.
Revised: Theoretical explanation of success of parameter-efficient fine-tuning methods is active area of research in deep learning.

Original: Indeed, since trends have consistently shown that model accuracy increases with size, how can one achieve competitive accuracy while ﬁne-tuning less than 1% of the parameters?
Revised: Indeed, since trends have consistently shown that model accuracy increases with size, how can one achieve competitive accuracy while fine-tuning less than 1% of the parameters?

Original: One popular hypothesis is intrinsic dimensionality (Li et al., 2018), which posits that the minimum number of parameters needed to train a machine learning model may be much less than the total number of model parameters.
Revised: One popular hypothesis is intrinsic dimensionality (Li et al., 2018), which posits that the minimum number of parameters needed to train a machine learning model may be much less than the total number of model parameters.

Original: Aghajanyan et al.
Revised: Aghajanyan et al.

Original: (2020) explore this hypothesis in the context of ﬁne-tuning LLMs, showing that one can achieve most of their accuracy by training only a very small number of parameters (chosen via a random projection).
Revised: (2020) explore this hypothesis in the context of fine-tuning LLMs, showing that one can achieve most of their accuracy by training only a very small number of parameters (chosen via a random projection).

Original: Perhaps surprisingly, they ﬁnd that as the model size increases, intrinsic dimension decreases , in the limit exhibiting zero-shot learning.
Revised: Perhaps surprisingly, they find that as the model size increases, intrinsic dimension decreases , in the limit exhibiting zero-shot learning.

Original: While we did not explore this hypothesis in the context of DP due to computational restrictions, we believe it may be an interesting lens through which one can understand the effectiveness of private parameter-efﬁcient ﬁne-tuning.
Revised: While we did not explore this hypothesis in the context of DP due to computational restrictions, we believe it may be an interesting lens through which one can understand the effectiveness of private parameter-efficient fine-tuning.

Original: 3. C OMPARISION WITH B ASELINE A LGORITHMS
Revised: 3. C OMPARISION WITH B ASELINE A LGORITHMS

Original: We highlight some key algorithmic differences between our proposed methods and the baselines of full ﬁne-tuning and RGP.
Revised: We highlight some key algorithmic differences between our proposed methods and the baselines of full fine-tuning and RGP.

Original: • DPSGD and RGP both require updating all parameters of the pre-trained model, whereas our proposed methods update only a tiny fraction (between 0 .
Revised: • DPSGD and RGP both require updating all parameters of the pre-trained model, whereas our proposed methods update only a tiny fraction (between 0 .

Original: 05% and 1% ).
Revised: 05% and 1% ).

Original: The rightmost columns of Tables 3 and 4 list the number of parameters trained by these algorithms.
Revised: The rightmost columns of Tables 3 and 4 list the number of parameters trained by these algorithms.

Original: • RGP performs a low-rank decomposition of weight matrices which is very similar to LoRA, though there are subtle differences.
Revised: • RGP performs a low-rank decomposition of weight matrices which is very similar to LoRA, though there are subtle differences.

Original: Recall that in RGP, at the beginning of each iteration t , the historical weight matrix W t − 1 is decomposed to ﬁnd a low-rank product LR .
Revised: Recall that in RGP, at the beginning of each iteration t , the historical weight matrix W t − 1 is decomposed to find a low-rank product LR .

Original: The gradients computed on L and R are then projected back to the full parameter space to perform the descent step.
Revised: The gradients computed on L and R are then projected back to the full parameter space to perform the descent step.

Original: Hence, RGP does not keep the pre-trained weights frozen during the learning process.
Revised: Hence, RGP does not keep the pre-trained weights frozen during the learning process.

Revised: [Table]

Original: LoRA can be viewed as a simpliﬁcation of RGP.
Revised: LoRA can be viewed as a simplification of RGP.

Original: LoRA reparametrizes W FT :
Revised: LoRA reparametrizes W FT :

Original: = W PT + LR , where the pre-trained weight matrix W PT is frozen during training.
Revised: = W PT + LR , where the pre-trained weight matrix W PT is frozen during training.

Original: Hence, compared to RGP, LoRA eliminates the decomposition and the projection to the full parameter space at each iteration, simplifying the implementation and reducing the running time and memory cost.
Revised: Hence, compared to RGP, LoRA eliminates the decomposition and the projection to the full parameter space at each iteration, simplifying the implementation and reducing the running time and memory cost.

Original: This is summarized in Table 2.
Revised: This is summarized in Table 2.

Original: We observe that DP LoRA reduces the memory cost by about 33% and the training speed by 8% .
Revised: We observe that DP LoRA reduces the memory cost by about 33% and the training speed by 8% .

Original: As we will see, this simpliﬁcation also results in improved utility.
Revised: As we will see, this simplification also results in improved utility

Original:  • Neither full ﬁne-tuning nor RGP fall into our meta-framework described by (1).
Revised: [Table] • Neither full fine-tuning nor RGP fall into our meta-framework described by (1).

Original: Thus, if a pre-trained model is to be applied to several downstream tasks, one must store a separate set of weights for each task, incurring a signiﬁcant memory cost and losing the plug-in functionality.
Revised: Thus, if a pre-trained model is to be applied to several downstream tasks, one must store a separate set of weights for each task, incurring a significant memory cost and losing the plug-in functionality.

Original: In contrast, our methods are much more lightweight.
Revised: In contrast, our methods are much more lightweight.

Original: [Table]

Original: [Table]

Original:  E XPERIMENTS
Revised:  E XPERIMENTS

Original: We experimentally evaluate our methods for DP ﬁne-tuning to demonstrate their utility, privacy, and parameter-efﬁciency.
Revised: We experimentally evaluate our methods for DP fine-tuning to demonstrate their utility, privacy, and parameter-efficiency.

Original: We investigate both language understanding and text generation tasks, using RoBERTa and GPT-2 models, to establish that our techniques are applicable to a variety of tasks and model architectures.
Revised: We investigate both language understanding and text generation tasks to establish that our techniques are applicable to a variety of tasks and model architectures.

Original: Code for reproducibility, as well as additional experimental results, will be released within the following weeks at https://github.com/AnonymousAKES/
Revised: Our code is publicly available at https://github.com/AnonymousAKES/

Original: Differentially-Private-Fine-tuning-of-Language-Models .
Revised: Differentially-Private-Fine-tuning-of-Language-Models .

Original: 4. F INE -T UNING FOR L ANGUAGE U NDERSTANDING T ASKS
Revised: 4. F INE -T UNING FOR L ANGUAGE U NDERSTANDING T ASKS

Original: We ﬁrst compare our methods with state-of-the-art ﬁne-tuning algorithms using models from the
Revised: We first compare our methods with state-of-the-art fine-tuning algorithms using models from the

Original: BERT family, which was used in the prior work (Yu et al., 2021b).
Revised: BERT family, which was used in the prior work (Yu et al., 2021b).

Original: Speciﬁcally, we use RoBERTa models (Liu et al., 2019),3 which are pre-trained on public data collected from the web.
Revised: Specifically, we use RoBERTa models (Liu et al., 2019), which are pre-trained on public data collected from the web.

Original: RoBERTaBase has 125M parameters and RoBERTa-Large has 355M parameters.
Revised: RoBERTaBase has 125M parameters and RoBERTa-Large has 355M parameters.

Original: We ﬁne-tune the pre-trained models on four  tasks: MNLI, QQP, QNLI and SST-2 fromthe GLUEbenchmark (Wang et al., 2018), following Yu et al.
Revised: We choose four downstream tasks: MNLI, QQP, QNLI, and SST-2 from GLUE (Wang et al., 2018), following Yu et al.

Original: (2021b).
Revised: (2021b).

Original: Implementation Details: For ﬁne-tuning with adapters, we may choose the intermediate representation dimension r , shared across all adapter layers.
Revised: Implementation Details: For fine-tuning with adapters, we may choose the intermediate representation dimension r , shared across all adapter layers.

Original: Similarly, for ﬁne-tuning with Compacter, we can choose both r and the Kronecker product kernel dimension n in (6).
Revised: For fine-tuning with Compacter, we can choose both r and the Kronecker product kernel dimension n in (6).

Original: For LoRA ﬁne-tuning, we add bottleneck branches for both the attention layers and the feedforward layers, which differs slightly from the addition of bottleneck branches for only the W q and W v matrices of the attention layers as done by Hu et al.
Revised: For LoRA fine-tuning, we add bottleneck branches for both the attention layers and the feedforward layers, which differs slightly from the addition of bottleneck branches for only the W q and W v matrices of the attention layers as done by Hu et al.

Original: Given the same bottleneck representation dimension r in (4), our new implementation uses twice as many trainable parameters as the original paper, and achieves some improvements for learning with DP.
Revised: Given the same bottleneck representation dimension r in (4), our new implementation uses twice as many trainable parameters as the original paper, and achieves some improvements for learning with DP.

Original: We perform privacy accounting using the approach of Gopi et al.
Revised: We perform privacy accounting using the approach of Gopi et al.

Original: (2021), which currently gives the tightest bounds.
Revised: (2021), which currently gives the tightest bounds.

Revised: All experiments in this section are run with half-precision.

Revised: We find that using full-precision further improves the performance.

Revised: The results can be found in the full version of this work 3 .

Original: Hyperparameter choice: Given the large number of hyperparameter choices, e.g., the intermediate representation dimension, learning rate, weight decay, privacy delta, and model size,  an exhaustive grid search over all hyperparameters is expensive, due to the model sizes.
Revised: Hyperparameter choice: Given the large number of hyperparameter choices, e.g., the intermediate representation dimension, learning rate, weight decay, privacy parameter δ , and model size, [Table] an exhaustive grid search over all hyperparameters is expensive.

Original: Our hyperparameter choices are informed by prior work and are as follows.
Revised: Our hyperparameter choices are informed by prior work and are as follows.

Original: For privacy parameters, we use δ = 1efor SST-2 and QNLI and δ = 1e-6 for QQP and MNLI due to their dataset sizes, and use noise multipliers 0 . 92 , 0 . 83 , 0 . 66 and 0 .
Revised: For privacy parameters, we use δ = 1e-5 for SSTand QNLI and δ = 1e-6 for QQP and MNLI due to their dataset sizes, and use noise multipliers 0 . 92 , 0 . 83 , 0 . 66 and 0 .

Original: 65 for SST-2, QNLI, QQP and MNLI, respectively, which is the same as Yu et al.
Revised: 65 for SST-2, QNLI, QQP, and MNLI, respectively, which is the same as Yu et al.

Original: (2021b).
Revised: (2021b).

Revised: In Appendix B

Revised: [Table]

Revised: we run experiments under different privacy parameters.

Revised: The proposed framework performs well under a wide range choices of ε and δ .

Original: The clipping thresholdof per-example gradients is 10 for all methods. 
Revised: The clipping threshold is 10 for all methods. The batch size is 2000.

Revised: In Appendix C, we show the performance of the proposed algorithm is stable across a wide range of choices of clipping thresholds and batch sizes.

Original: For adapters and Compacter, we followsuggestions in the original papers and choose r froma set { 16 , 48 , 96 } and n froma set { 4 , 8 , 12 } .
Revised: For adapters and Compacter, we follow the original papers and choose r from { 16 , 48 , 96 } and n from { 4 , 8 , 12 } .

Original: For LoRA, we choose the best-performing rank r from the set { 4 , 16 , 48 , 64 } .
Revised: For LoRA, we choose the best-performing rank r from the set { 4 , 16 , 48 , 64 } .

Original: The best performing hyperparameters are noted in Tables 3 and 4.
Revised: The best performing hyperparameters are noted in Tables 3 and 4.

Original: Weuse batch size 2000 and train for 20 epochs.
Revised: We train for 20 epochs using AdamW (Loshchilov &

Original: We use the optimizer AdamW (Loshchilov & Hutter, 2019) with weight decay 1e-2 and search over four learning rates { 5e-4, 1e-3, 2e-3, 5e-3 } .
Revised: Hutter, 2019) with weight decay 1e-2 and search over four learning rates { 5e-4, 1e-3, 2e-3, 5e-3 } .

Original: Results: We report the prediction accuracyon four tasks in Tables 3 and 4.
Revised: Results: We report the prediction accuracy in Tables 3 and 4.

Original: Our experiments using RoBERTa-Base serve as a direct comparison tothe work of Yu et al.
Revised: Our experiments using RoBERTaBase serve as a direct comparison to Yu et al.

Original: (2021b) who only trained the base model, whereas RoBERTa-Large experiments demonstrate the signiﬁcance of using larger models.
Revised: (2021b) who only trained the base model, whereas RoBERTa-Large experiments demonstrate the significance of using larger models.

Original: We could not report the numbers for full ﬁne-tuning using DPSGD on RoBERTa-Large due to running time and memory costs; see the discussion at the end of this section.

Original: We summarize our key ﬁndings: • On all datasets, our methods achieve the best accuracy while training aonly tiny fraction of parameters; larger models give signiﬁcant improvements.
Revised: Our key findings are: (1) On all datasets, our methods achieve the best accuracy while training a tiny fraction of parameters; larger models give significant improvements.

Original: • Noticeable improvements inthe privacy parameter ε versus Yu et al.
Revised: (2) Noticeable improvements in ε versus Yu et al.

Original: (2021b) are primarily due to new privacy accountants based on Fourier-based numerical composition (Koskela et al., 2020; 2021; Gopi et al., 2021); we use the accountant in Gopi et al.
Revised: (2021b) are primarily due to new privacy accountants based on Fourier-based numerical composition (Koskela et al., 2020; 2021; Gopi et al., 2021); we use the accountant in Gopi et al.

Original: (2021) since it is the most efﬁcient.
Revised: (2021) since it is the most efficient.

Original: • Private adapters provide the best average performance for RoBERTa-Base, whereas LoRA outperforms all other methods for RoBERTa-Large.
Revised: (3) Private adapters provide the best average performance for RoBERTa-Base, whereas LoRA outperforms all other methods for RoBERTa-Large.

Original: Remark: While our experiments indicate that full ﬁne-tuning with DPSGD does not achieve competitive performance, there could be a choice of hyperparameters that can improve upon the reported numbers.
Revised: Remark: While our experiments indicate that full fine-tuning with DPSGD does not achieve competitive performance, there could be a choice of hyperparameters that can improve upon the reported numbers.

Original: A recent work (Anil et al., 2021) says that “mega” batch sizes (in the millions) helps
Revised: A recent work (Anil et al., 2021) says that “mega” batch sizes (in the millions) helps

Original: DPSGD.
Revised: DPSGD.

Original: Such experiments are costly for large models.
Revised: Such experiments are costly for large models.

Original: The memory consumption of DPSGD requires that batch size on a single GPU is small for large language models; on the other hand, large batch size requires gradient accumulation and running for more epochs.
Revised: The memory consumption of DPSGD requires that batch size on a single GPU is small for large language models; on the other hand, large batch size requires gradient accumulation and running for more epochs.

Original: All of this leads to substantial stretching of the training time.
Revised: All of this leads to substantial stretching of the training time.

Original: Moreover, our main message is that one does not need to ﬁne-tune all parameters to achieve the best accuracy.
Revised: Moreover, our main message is that one does not need to fine-tune all parameters to achieve the best accuracy.

Original: Nevertheless, it is interesting and natural to wonder if full ﬁne-tuning with DPSGD can match the accuracy of parameter-efﬁcient methods for some choice of hyperparameters.
Revised: Nevertheless, it is interesting and natural to wonder if full fine-tuning with DPSGD can match the accuracy of parameter-efficient methods for some choice of hyperparameters.

Original: A positive answer would imply that private and non-private ﬁnetuning conceptually mirror each other, whereas a negative answer would demonstrate that parameter efﬁciency is both necessary and sufﬁcient for private ﬁne-tuning.
Revised: A positive answer would imply that private and non-private finetuning conceptually mirror each other, whereas a negative answer would demonstrate that parameter efficiency is both necessary and sufficient for private fine-tuning.

Original: 4. F INE - TUNING FOR N ATURAL L ANGUAGE G ENERATION (NLG)
Revised: 4. F INE - TUNING FOR N ATURAL L ANGUAGE G ENERATION (NLG)

Original: Next, we study private ﬁne-tuning for text generation problems using the GPT-2 series of models on the End-2-End (E2E)
Revised: Next, we study private fine-tuning for text generation problems using the GPT-2 series of models on the End-2-End (E2E)

Original: NLG challenge (Novikova et al., 2017), one of the primary benchmarks used in recent works on non-private ﬁne-tuning (Hu et al., 2021; Li & Liang, 2021).
Revised: NLG challenge (Novikova et al., 2017), one of the primary benchmarks used in recent works on non-private fine-tuning (Hu et al., 2021; Li & Liang, 2021).

Original: We use GPT-2-Small (117M parameters), GPT-2-Medium (345M parameters), and GPT-2-Large (774M parameters).
Revised: We use GPT-2-Small (117M parameters), GPT-2-Medium (345M parameters), and GPT-2-Large (774M parameters).

Original: To the best of our knowledge, we are the ﬁrst to privately ﬁne-tune for E2E or ﬁne-tune GPT-2-Large.
Revised: To the best of our knowledge, we are the first to privately fine-tune for E2E or fine-tune GPT-2-Large.

Original: The purpose of this section is not to evaluate various ﬁne-tuning algorithms, but to show that private ﬁne-tuning is competitive with non-private ﬁne-tuning for text generation problems.
Revised: The purpose of this section is not to evaluate various fine-tuning algorithms, but to show that private fine-tuning is competitive with non-private fine-tuning for text generation problems.

Original: Due to the high cost of training, we report experimental results only for ﬁne-tuning (private and non-private) with LoRA.
Revised: Due to the high cost of training, we report experimental results only for fine-tuning with LoRA.

Original: We think that all ﬁne-tuning methods in this paper should achieve comparable accuracy.
Revised: [Table]

Original: E2E NLG challenge: The E2E dataset was introduced by Novikova et al.
Revised: E2E NLG challenge: The E2E dataset was introduced by Novikova et al.

Original: (2017), and contains template-like information in the restaurant domain to be mapped to natural language with end-toend training.
Revised: (2017), and contains template-like information in the restaurant domain to be mapped to natural language with end-toend training.

Original: The dataset consists of 42K training samples, 4.6K validation samples, and 4.6K test[Table] samples.
Revised: The dataset consists of 42K training samples, 4.6K validation samples, and 4.6K test samples.

Original: We use standard metrics such as BLUE, ROUGE-L, etc., used in (Hu et al., 2021) for measuring the quality of predictions.
Revised: We use standard metrics such as BLUE, ROUGE-L, etc., used in (Hu et al., 2021) for measuring the quality of predictions.

Original: Hyperparameter choice: For LoRA, we choose the bottleneck rank r = 4 in (4) and ﬁne-tune W q and W v matrices of the attention layers as in the original paper.
Revised: Hyperparameter choice: For LoRA, we choose the bottleneck rank r = 4 in (4) and fine-tune W q and W v matrices of the attention layers as in the original paper.

Original: We optimize using AdamW with learning rate 2e-4, weight decay 1e-2 and train our models for 5 epochs using batch size 64.
Revised: We optimize using AdamW with learning rate 2e-4, weight decay 1e-2 and train our models for 5 epochs using batch size 64.

Original: We take the gradient clipping parameter to be 1.0 and the noise multiplier to be 0.5 for the accountant in Gopi et al.
Revised: We take the gradient clipping parameter to be 1.0 and the noise multiplier to be 0.5 for the accountant in Gopi et al.

Original: (2021), achieving ε = 5 .
Revised: (2021), achieving ε = 5 .

Original: 4 , δ = 1e-5.
Revised: 4 , δ = 1e-5.

Original: Results: The results of our experiments are summarized in the Table 5, which reiterate the main themes of this paper: private ﬁne-tuning with a parameter-efﬁcient approach performs close to their non-private counterparts and show consistent improvement in the utility as the model size increases.
Revised: Results: The results of our experiments are summarized in the Table 5, which reiterate the main themes of this paper: private fine-tuning with a parameter-efficient approach performs close to their non-private counterparts and show consistent improvement in the utility as the model size increases.

Original:  R ELATED W ORK
Revised:  R ELATED W ORK

Original: More on DP learning: Some work studies private language models on more traditional architectures such as LSTMs (Hochreiter & Schmidhuber, 1997), either training with DPSGD (McMahan et al., 2018; Carlini et al., 2019) or related heuristics (Ramaswamy et al., 2020).
Revised: More on DP learning: Some work studies private language models on more traditional architectures such as LSTMs (Hochreiter & Schmidhuber, 1997), either training with DPSGD (McMahan et al., 2018; Carlini et al., 2019) or related heuristics (Ramaswamy et al., 2020).

Original: Though pre-training on public data is suggested (McMahan et al., 2018), public data appears to only be used in one of these works for honest hyperparameter selection (Ramaswamy et al., 2020).
Revised: Though pre-training on public data is suggested (McMahan et al., 2018), public data appears to only be used in one of these works for honest hyperparameter selection (Ramaswamy et al., 2020).

Original: A few more recent works consider training LLMs with DP.
Revised: A few more recent works consider training LLMs with DP.

Original: Anil et al.
Revised: Anil et al.

Original: (2021) privately train BERT-Large from scratch, compared to our work which focuses on private ﬁne-tuning.
Revised: (2021) privately train BERT-Large from scratch, compared to our work which focuses on private fine-tuning.

Original: (Hoory et al., 2021;
Revised: (Hoory et al., 2021;

Original: Basu et al., 2021) perform private full ﬁne-tuning of BERT models.
Revised: Basu et al., 2021) perform private full fine-tuning of BERT models.

Original: Hoory et al.
Revised: Hoory et al.

Original: (2021) achieve accuracy which is comparable to the non-private model, but additionally supplement the public pre-training data with additional domain-relevant material, while we use off-the-shelf pre-trained models.
Revised: (2021) achieve accuracy which is comparable to the non-private model, but additionally supplement the public pre-training data with additional domain-relevant material, while we use off-the-shelf pre-trained models.

Original: Basu et al.
Revised: Basu et al.

Original: (2021) observe signiﬁcant drops in utility, compared to our parameter-efﬁcient methods which do not.
Revised: (2021) observe significant drops in utility, compared to our parameter-efficient methods which do not.

Original: While Kerrigan et al.
Revised: While Kerrigan et al.

Original: (2020) consider public pre-training and private ﬁne-tuning, their experiments are on much smaller architectures (i.e., feedforward networks with three hidden layers).
Revised: (2020) consider public pre-training and private fine-tuning, their experiments are on much smaller architectures (i.e., feedforward networks with three hidden layers).

Original: More on Fine-tuning: There exist other parameter-efﬁcient tuning methods which we did not evaluate in our work.
Revised: More on Fine-tuning: There exist other parameter-efficient tuning methods which we did not evaluate in our work.

Original: Some of these include random subspace projection (exploiting intrinsic dimensionality (Li et al., 2018; Aghajanyan et al., 2020)), preﬁx and prompt tuning (Li & Liang, 2021; Lester et al., 2021), tuning only biases (Cai et al., 2020; Ben Zaken et al., 2021), and other architecture variants including Adapters (Pfeiffer et al., 2021; R¨uckl´e et al., 2020).
Revised: Some of these include random subspace projection (exploiting intrinsic dimensionality (Li et al., 2018; Aghajanyan et al., 2020)), prefix and prompt tuning (Li & Liang, 2021; Lester et al., 2021), tuning only biases (Cai et al., 2020; Ben Zaken et al., 2021), and other architecture variants including Adapters (Pfeiffer et al., 2021; R¨uckl´e et al., 2020).

Original: An interesting direction for future work is to see whether parameter-efﬁcient tuning approaches speciﬁcally designed for the private setting can achieve higher utility.
Revised: An interesting direction for future work is to see whether parameter-efficient tuning approaches specifically designed for the private setting can achieve higher utility.

Original: We also mention zero-shot learning, in which no task-speciﬁc dataset is required and thus perfect privacy is achieved.
Revised: We also mention zero-shot learning, in which no task-specific dataset is required and thus perfect privacy is achieved.

Original: Currently, zero-shot approaches achieve low utility compared to ﬁne-tuning, though it is possible that future models may narrow this gap.
Revised: Currently, zero-shot approaches achieve low utility compared to fine-tuning, though it is possible that future models may narrow this gap.

Original:  C ONCLUSION
Revised:  C ONCLUSION

Original: So far, DP deep learning has focused on training models from scratch.
Revised: So far, DP deep learning has focused on training models from scratch.

Original: The spectacular success of transfer learning in real-world applications, however, shows that private ﬁne-tuning is an equally pertinent problem to study and deserves more attention.
Revised: The spectacular success of transfer learning in real-world applications, however, shows that private fine-tuning is an equally pertinent problem to study and deserves more attention.

Original: We show that by combining recent advances in
Revised: We show that by combining recent advances in

Original: NLP, parameter-efﬁciency, privacy accounting, and using larger models, one can privately ﬁne-tune models whose utility approaches that of non-private models.
Revised: NLP, parameter-efficiency, privacy accounting, and using larger models, one can privately fine-tune models whose utility approaches that of non-private models.

Original: We hope our work inspires more study on the core problem of private ﬁne-tuning, which we believe to be a central direction for research in private machine learning, leading to more interaction between the LLM and DP communities.
Revised: We hope our work inspires more study on the core problem of private fine-tuning, which we believe to be a central direction for research in private machine learning, leading to more interaction between the LLM and DP communities.

Original: A A DDITIONAL R ELATED W ORK
Revised: A A DDITIONAL R ELATED W ORK

Original: Our investigation ﬁts more broadly into a line of work employing public data for private data analysis.
Revised: Our investigation fits more broadly into a line of work employing public data for private data analysis.

Original: Some works on image classiﬁcation consider pre-training on a large public dataset and ﬁnetuning on a smaller private dataset (Abadi et al., 2016; Papernot et al., 2019; Tram ` er & Boneh, 2021).
Revised: Some works on image classification consider pre-training on a large public dataset and finetuning on a smaller private dataset (Abadi et al., 2016; Papernot et al., 2019; Tram ` er & Boneh, 2021).

Original: One line of work uses unlabeled public data to train a student model (Papernot et al., 2017; 2018;
Revised: One line of work uses unlabeled public data to train a student model (Papernot et al., 2017; 2018;

Original: Bassily et al., 2018).
Revised: Bassily et al., 2018).

Original: Another recent idea uses a small amount of public data to identify a lowerdimensional subspace of the gradients in which to perform private descent (Zhou et al., 2021; Yu et al., 2021a; Kairouz et al., 2021).
Revised: Another recent idea uses a small amount of public data to identify a lowerdimensional subspace of the gradients in which to perform private descent (Zhou et al., 2021; Yu et al., 2021a; Kairouz et al., 2021).

Original: Finally, other works (both theoretical and experimental) investigate the role of public data in private query release, synthetic data generation, and prediction (Ji &
Revised: Finally, other works (both theoretical and experimental) investigate the role of public data in private query release, synthetic data generation, and prediction (Ji &

Original: Elkan, 2013; Beimel et al., 2016;
Revised: Elkan, 2013; Beimel et al., 2016;

Original: Alon et al., 2019; Nandi & Bassily, 2020; Bassily et al., 2020a;b;
Revised: Alon et al., 2019; Nandi & Bassily, 2020; Bassily et al., 2020a;b;

Original: Liu et al., 2021).
Revised: Liu et al., 2021).

Original: B E XPERIMENTS WITH D IFFERENT P RIVACY P ARAMETERS
Revised: B E XPERIMENTS WITH D IFFERENT P RIVACY P ARAMETERS

Original: Now we test our framework under different privacy constraints.
Revised: Now we test our framework under different privacy constraints.

Original: Speciﬁcally, we run LoRA on the language understanding tasks with various choices of privacy parameters ε and δ .
Revised: Specifically, we run LoRA on the language understanding tasks with various choices of privacy parameters ε and δ .

Original: We consider both
Revised: We consider both

Original: RoBERTa-Base and RoBERTa-Large.
Revised: RoBERTa-Base and RoBERTa-Large.

Original: For the RoBERTa-Large model, we set ε = 1 and 3 with δ being the same as those in Section 4.
Revised: For the RoBERTa-Large model, we set ε = 1 and 3 with δ being the same as those in Section 4.

Original: We use the PRV accountant (Gopi et al., 2021).
Revised: We use the PRV accountant (Gopi et al., 2021).

Original: After getting the noise multipliers, we also reduce the value of δ and report the corresponding value of ε .
Revised: After getting the noise multipliers, we also reduce the value of δ and report the corresponding value of ε .

Original: The hyperparameters are the same as those in Section 4.
Revised: The hyperparameters are the same as those in Section 4.

Original: We run experiments on all four tasks, i.e., MNLI ( n ∼ 392k), QQP ( n ∼ 364k),
Revised: We run experiments on all four tasks, i.e., MNLI ( n ∼ 392k), QQP ( n ∼ 364k),

Original: QNLI ( n ∼ 104k), and SST-2 ( n ∼ 67k).
Revised: QNLI ( n ∼ 104k), and SST-2 ( n ∼ 67k).

Original: We report the results in Table 6.
Revised: We report the results in Table 6.

Original: The performance of our framework is decent even with very tight privacy budgets.
Revised: The performance of our framework is decent even with very tight privacy budgets.

Original: For instance, with ε < 2 and δ = 1 / 1000 n , the accuracy gap between the non-private baseline is only 3.8 for MNLI and 2.1 for
Revised: For instance, with ε < 2 and δ = 1 / 1000 n , the accuracy gap between the non-private baseline is only 3.8 for MNLI and 2.1 for

Original: SST-2.
Revised: SST-2.

Original: For the RoBERTa-Base model, we try various choices of ε .
Revised: For the RoBERTa-Base model, we try various choices of ε .

Original: The values of ε are chosen from [0 . 1 , 0 . 5 , 1 , 3 , 5 , 8 , 12] .
Revised: The values of ε are chosen from [0 . 1 , 0 . 5 , 1 , 3 , 5 , 8 , 12] .

Original: All other settings are the same as those in Section 4.
Revised: All other settings are the same as those in Section 4.

Original: We run experiments on the MNLI and SST-2 datasets.
Revised: We run experiments on the MNLI and SST-2 datasets.

Original: The results are presented in Figure 2.
Revised: The results are presented in Figure 2.

Original: Our framework performs well for a wide range of ε .
Revised: Our framework performs well for a wide range of ε .

Original: We note that our algorithm achieves meaningful accuracy even for very tight privacy parameters ε = 0 .
Revised: We note that our algorithm achieves meaningful accuracy even for very tight privacy parameters ε = 0 .

Original: 5 and 1 .
Revised: 5 and 1 .

Original: Such values of ε are rarely explored when training deep models with differential privacy.
Revised: Such values of ε are rarely explored when training deep models with differential privacy.

Original: C O N THE I NFLUENCE OF H YPERPARAMETERS
Revised: C O N THE I NFLUENCE OF H YPERPARAMETERS

Original: Here we demonstrate that our algorithms perform well for a wide range of hyperparameters.
Revised: Here we demonstrate that our algorithms perform well for a wide range of hyperparameters.

Original: We study two hyperparameters that are directly related to the variance of noise: clipping threshold and batchsize.
Revised: We study two hyperparameters that are directly related to the variance of noise: clipping threshold and batchsize.

Original: The clipping threshold is chosen from [0 . 1 , 1 . 0 , 3 . 0 , 5 . 0 , 10 . 0] and the batchsize is chosen
Revised: The clipping threshold is chosen from [0 . 1 , 1 . 0 , 3 . 0 , 5 . 0 , 10 . 0] and the batchsize is chosen

Original: [Figure]
Revised: [Figure]

Original: SST-

Original: [Figure]
Revised: [Figure]

Original: [Figure]
Revised: n g T

Original: A cc u r a c y
Revised: h r e s h o l d

Original: MNLI 0.

Original: [Figure]
Revised: [Figure]

Original: [Figure] [Figure][Figure]
Revised: [Figure] [Figure]

Original: [Figure]
Revised: [Figure]

Original: n g T h r e s h o l d 90.
Revised: [Figure]

Original: [Figure]
Revised: [Figure]

Original: Batchsize 0.
Revised: [Figure] [Figure] [Figure] [Figure]

Original: C li
Revised: [Figure] [Figure] [Figure] [Figure]

Original: pp i n g T
Revised: [Figure] [Figure] [Figure] [Figure]

Original: h
Revised: [Figure] [Figure] [Figure] C

Revised: [Figure]

Original: r e s h 
Revised: r [Figure] h [Figure]

Original: o l d 91.
Revised: l d 91.

Original: Learning
Revised: [Figure] [Figure] [Figure] [Figure]

Original: [Figure] 
Revised: [Figure] [Figure] [Figure] [Figure]

Original: [Figure] 
Revised: [Figure] [Figure] [Figure] [Figure]

Revised: [Figure]

Revised: [Figure]

Original:  private LoRA on the SST-2 dataset.
Revised: [Figure] [Figure] of fine-tuning RoBERTa-Base with differentially private LoRA on the SST-2 dataset.

Original: Our algorithm performs well on a wide range of hyperparameters.
Revised: Our algorithm performs well on a wide range of hyperparameters.

Original: from [200 , 500 , 1000 , 2000 , 4000] .
Revised: from [200 , 500 , 1000 , 2000 , 4000] .

Original: We note that we keep the number of updates the same as that in
Revised: We note that we keep the number of updates the same as that in

Original: Section 4 when the batchsize is changed.
Revised: Section 4 when the batchsize is changed.

Original: We ﬁne-tune the RoBERTa-Base model with differentially private LoRA ( r = 16 ) on the SST-2 dataset.
Revised: We fine-tune the RoBERTa-Base model with differentially private LoRA ( r = 16 ) on the SST-2 dataset.

Original: The results are presented in Figure 3.
Revised: The results are presented in Figure 3.

Original: DP LoRA performs well for all the hyperparameters considered.
Revised: DP LoRA performs well for all the hyperparameters considered.

Original: The gap between the best accuracy and the worst accuracy is only 2%.
Revised: The gap between the best accuracy and the worst accuracy is only 2%.

Original: D F ULL F INE -T UNING WITH GPT- 
Revised: D F ULL F INE -T UNING WITH GPT-

Original: All results in Table 5 (in the main body), both private and non-private, perform ﬁne-tuning using
Revised: All results in Table 5 (in the main body), both private and non-private, perform fine-tuning using

Original: LoRA.
Revised: LoRA.

Original: In Table 7, we additionally report utility of non-private full ﬁne-tuning.
Revised: In Table 7, we additionally report utility of non-private full fine-tuning.

Original: These numbers are taken from Table 1 of Li & Liang (2021).
Revised: These numbers are taken from Table 1 of Li & Liang (2021).

Original: In general, these numbers are slightly lower than those obtained by performing non-private ﬁne-tuning with LoRA.
Revised: In general, these numbers are slightly lower than those obtained by performing non-private fine-tuning with LoRA.

Original: [Table] [Table]
Revised: [Table] [Table]

Original: [Table]
Revised: [Table]

Original: E A DDITIONAL EXPERIMENTS ON N ATURAL L ANGUAGE G ENERATION
Revised: E A DDITIONAL EXPERIMENTS ON N ATURAL L ANGUAGE G ENERATION

Original: In this section, we perform additional experiments on private ﬁne-tuning for text generation problems using the GPT-2 series of models.
Revised: In this section, we perform additional experiments on private fine-tuning for text generation problems using the GPT-2 series of models.

Original: This includes the private ﬁne-tuning of the GPT-2-XL model with 1.5B parameters.
Revised: This includes the private fine-tuning of the GPT-2-XL model with 1.5B parameters.

Original: There are three main points to note compared to our results in the main body: 1) We show an improved performance on E2E NLG challenge using better hyperparameters; 2) We conduct experiments on E2E dataset with different privacy parameters to show that large language models perform strong even with smaller privacy budgets; 3) Finally, we conduct new experiments on DART dataset.
Revised: There are three main points to note compared to our results in the main body: 1) We show an improved performance on E2E NLG challenge using better hyperparameters; 2) We conduct experiments on E2E dataset with different privacy parameters to show that large language models perform strong even with smaller privacy budgets; 3) Finally, we conduct new experiments on DART dataset.

Original: E. I MPROVING THE PERFORMANCE ON E2E NLG CHALLENGE
Revised: E. I MPROVING THE PERFORMANCE ON E2E NLG CHALLENGE

Original: We improve the results of Table 5 with the following set of new hyperparameters.
Revised: We improve the results of Table 5 with the following set of new hyperparameters.

Original: Hyperparameter choice: For LoRA, we choose the bottleneck rank r = 4 in (4) and ﬁne-tune W q and W v matrices of the attention layers as in the original paper.
Revised: Hyperparameter choice: For LoRA, we choose the bottleneck rank r = 4 in (4) and fine-tune W q and W v matrices of the attention layers as in the original paper.

Original: We optimize using AdamW with learning rate 4e-4, weight decay 1e-2 and train our models for 20 epochs.
Revised: We optimize using AdamW with learning rate 4e-4, weight decay 1e-2 and train our models for 20 epochs.

Original: We use batch size 128.
Revised: We use batch size 128.

Original: We take the gradient clipping parameter to be 1.0 and the noise multiplier to be 0.6 for the accountant in Gopi et al.
Revised: We take the gradient clipping parameter to be 1.0 and the noise multiplier to be 0.6 for the accountant in Gopi et al.

Original: (2021), achieving ε = 6 . 0 , δ = 1e-5.
Revised: (2021), achieving ε = 6 . 0 , δ = 1e-5.

Original: Results: The results of our experiments are summarized in the Table 8.
Revised: Results: The results of our experiments are summarized in the Table 8.

Original: E. E XPERIMENTS WITH D IFFERENT P RIVACY P ARAMETERS
Revised: E. E XPERIMENTS WITH D IFFERENT P RIVACY P ARAMETERS

Original: On E2E dataset, we test our framework with smaller privacy budgets ( ε < 5 and δ (cid:28) 1 /n ) where n is the number of samples in the training data.
Revised: On E2E dataset, we test our framework with smaller privacy budgets ( ε < 5 and δ ≪ 1 /n ) where n is the number of samples in the training data.

Original: Hyperparameter choice: The hyperparameter choices are similar as in Section E.1.
Revised: Hyperparameter choice: The hyperparameter choices are similar as in Section E.1.

Original: The only difference is that we increase the noise multiplier to be 0.71 for the accountant in Gopi et al.
Revised: The only difference is that we increase the noise multiplier to be 0.71 for the accountant in Gopi et al.

Original: (2021), achieving the following ( ε, δ ) pairs: ( ε = 3 . 0 , δ = 1e-5), ( ε = 3 .
Revised: (2021), achieving the following ( ε, δ ) pairs: ( ε = 3 . 0 , δ = 1e-5), ( ε = 3 .

Original: 4 , δ = 1 / 10 n ), ( ε = 3 .
Revised: 4 , δ = 1 / 10 n ), ( ε = 3 .

Original: 9 , δ = 1 / 100 n ) and ( ε = 4 .
Revised: 9 , δ = 1 / 100 n ) and ( ε = 4 .

Original: 5 , δ = 1 / 1000 n ).
Revised: 5 , δ = 1 / 1000 n ).

Original: Results: The results of our experiments are summarized in the Table 9.
Revised: Results: The results of our experiments are summarized in the Table 9.

Original: There are a couple of interesting observations comparing Table 9 with Table 8.
Revised: There are a couple of interesting observations comparing Table 9 with Table 8.

Original: First, we observe that although privacy budget is tight in Table 9, the results are quite similar to Table 8, which shows
Revised: First, we observe that although privacy budget is tight in Table 9, the results are quite similar to Table 8, which shows

Original: [Table]
Revised: [Table]

Original: that our methods also perform very well under stronger privacy guarantees.
Revised: that our methods also perform very well under stronger privacy guarantees.

Original: A more interesting observation is that under smaller epsilon regimes, the performance for private ﬁne-tuning of GPT2-XL model improves.
Revised: A more interesting observation is that under smaller epsilon regimes [Table] the performance for private fine-tuning of GPT2-XL model improves.

Original: Observe that the performance improvement is more prominent going from
Revised: Observe that the performance improvement is more prominent going from

Original: GPT-2-Small to GPT-2-XL in this setting, which may indicate that larger models can be even more effective in private learning when the privacy budgets are tight.
Revised: GPT-2-Small to GPT-2-XL in this setting, which may indicate that larger models can be even more effective in private learning when the privacy budgets are tight.

Original: E. P ERFORMING EXPERIMENTS ON DART DATASET
Revised: E. P ERFORMING EXPERIMENTS ON DART DATASET

Original: We study the DART dataset as a text generation problem for private ﬁne-tuning of GPT-2 series of models.
Revised: We study the DART dataset as a text generation problem for private fine-tuning of GPT-2 series of models.

Original: DART: DART was introduced as an open-domain data-to-text dataset by Nan et al.
Revised: DART: DART was introduced as an open-domain data-to-text dataset by Nan et al.

Original: The dataset consists of 62K training samples, 6.9K validation samples, and 12K test samples.
Revised: The dataset consists of 62K training samples, 6.9K validation samples, and 12K test samples.

Original: In comparison to E2E, the dataset is larger and the task is more challenging.
Revised: In comparison to E2E, the dataset is larger and the task is more challenging.

Original: Hyperparameter choice: The hyperparameter choices are similar as in the previous setting.
Revised: Hyperparameter choice: The hyperparameter choices are similar as in the previous setting.

Original: The only difference is that we use batch size 256 for the experiments on DART.
Revised: The only difference is that we use batch size 256 for the experiments on DART.

Original: This achieves ε = 6 . 8 , δ = 1e-5 on DART using the accountant in Gopi et al.
Revised: This achieves ε = 6 . 8 , δ = 1e-5 on DART using the accountant in Gopi et al.

Original: Results: The results of our experiments are summarized in the Table 10.
Revised: Results: The results of our experiments are summarized in the Table 10.