Large Language Models Can Be Strong
Differentially Private Learners
ICLR 2022
Xuechen Li, Florian Tramèr, Percy Liang, and Tatsunori Hashimoto

Background: User data and privacy issues in NLP-There are inherent conflicts between data collection and privacy protection
for tasks in NLP (e.g., building dialog generation systems)
Private user data is abundant and of high quality. Can we use it directly?

Public data (low quality, large quantity)

Annotator-driven data (high quality, costly)

Private user data (high quality, large quantity)

Background: User data and privacy issues in NLP-

Directly training large language models (LMs) on private user data can beProblematic-Large LMs can memorize training data
Data extraction attacks are surprisingly effective for large LMs (Carlini et al., 2021)

Background: Need for privacy-preserving techniques-We need provable guarantees that models won't leak private data
Goal-

Use private data
Do not leak them

Background: Differential privacy-

Differential privacy (DP) is a formal privacy guarantee for an algorithm used inUS census, in Google analytics, at Apple..
Past attempts at enforcing DP for vision tasks (via DP-SGD) resulted inmodels with low utility

This work-Q: Is it possible to build high quality DP NLP models on moderate amounts ofprivate training data?
A: Yes!
This work:-Leverage (public) off-the-shelf pretrained models and perform fine-tuning with DP-Adam
Surprisingly, full fine-tuning-updating all model parameters yields strong performance
Even more surprisingly, the larger the pretrained model, the better the performance of private
fine-tuning, unlike what theory for private convex learning prescribesOverview-

Overall--

Large language model (transformer-based) can achieve differential privacy

Contributions-

Effective: tricks for hyperparameter setting
Efficient: ghost clippingMethod overview-Effective--

Hyperparameters
Fine-tuning objective

Efficient-

Ghost clipping: Clipping per example gradients without instantiating per example gradients.

Effective-

Good hyperparameters--

DP learning is sensitive to choices of hyperparameters
They did a thorough study of how hyperparameters affect performance
Good hyperparameters tend to transfer across tasks – we transferred tuning results on one
task to all remaining tasks
Totally based on experiment findings

Fine-tuning objective-Objectives that make learning easy results in better private models
They want the fine-tuning objective to be close to the pretraining objective
Alignment

Hyperparameters-

Batch size and learning rate
Good batch sizes and learning rates for private learning is different from thosetypical for non-private learning
Case 1: Fixed epochs (compute bound)-

Need large batch size
Need large learning rate

Hyperparameters-

Batch size and number of epochs
Case 2: Unconstrained epochs--

Fix the update steps (large batches, each epoch less updates, more epochs)
Jointly scaling both the batch size and number of epochs is almost always better

Heuristic explanation:-, B is batch size
Signal to noise ratio:
Larger r, better performance

Hyperparameters-Batch size and number of epochs
Case 2: Unconstrained epochs-

Fix the update steps (large batches, each epoch less updates, more epochs)
Jointly scaling both the batch size and number of epochs is almost always better
Larger batch size, larger sampling rate q (just B/N), smaller, larger r, lower NLL

Fine-tuning task formulation matters--For classification, CLS-token fine-tuning introduces a discrepancy between
pretraining (masked language modeling) and fine-tuning (network on top ofCLS)
This makes the fine-tuning problem slightly difficult

Fine-tuning task formulation matters-

Instead of predicting integer labels, they ask the model to predict textualizedlabels during fine-tuning
Example: sentiment classification-Given sentence <input>, classify as positive or negative sentiment
Construct template "<input> It is [MASK]."
Predict [MASK] in the template, where [MASK] is "awesome" or "terrible"
Easier fine-tuning problem results in better private models (even with generic templates)Effective-

Good hyperparameters--

Fine-tuning objective--

DP learning is sensitive to choices of hyperparameters
They did a thorough study of how hyperparameters affect performance
Good hyperparameters tend to transfer across tasks – we transferred tuning results on one
task to all remaining tasks
Totally based on experiment findings
Large batch size
Objectives that make learning easy results in better private models
They want the fine-tuning objective to be close to the pretraining objective
Alignment
Templates

Then, how to make the model efficient?

Ghost clipping-DP optimization is costly due to clipping per example gradients
Naively implemented, this step instantiates per example gradients and can beprohibitively costly in memory
They present a technique for per example gradient clipping without
instantiating per example gradients for any linear layer in a large Transformer
model

Problem: Per-example gradient-Clip( , ) means reweighting--

Challenge:--

Scaling factor:
Reweighted loss:
ComputeTricks-

Per example gradient → Layer by layer gradient
Ghost clipping for transformer

Trick 1: Layer by layer gradient-takes a large memory to instantiating
Observations--

Neural networks have multi-layers
Vector norm:Thus-

We can instantiating one layer each time
Compute separately

Trick 2: Ghost clipping-

We still need to compute each layer norm
Linear layer--

Normal way--

S input to a linear layer
S gradient of the linear layer
F weight of the linear layer
Step 1:
Step 2:Compute the norm

Ghost clipping-

Thus not necessary to compute step 1Ghost clipping-Performance-Then, the model is effective and efficient now.

Does high dimensionality degrade performance?
-Do larger models lead to better or worse results?
--Answer: Larger models are better.

Are fine-tuning methods that privatize fewer parameters more performant?
-Answer: Not true in general.

Sentence classification

Results overview-For classification, DP fine-tuning can outperform TextHide (InstaHide for text)
For generation, DP fine-tuning can outperform strong non-private baselines
Larger and better pretrained models result in better fine-tuned performance

Generation-Epsilon, smaller, better

Dialog Generation

Summary-Large LMs can be effectively fine-tuned under DP if hyperparameters andthe fine-tuning objective are set right
Full fine-tuning large LMs under DP can be memory efficient
Better and larger pretrained models yield improved private fine-tuning results

Thank you!

Q&A