On the Benefits of Public Representations for Private Transfer Learning under Distribution Shift
Authors: Pratiksha Thaker
Amrith Setlur
Steven Wu
Virginia Smith
Venue: NeurIPS 2024 poster (2024)
Keywords: pretraining
representations
privacy
distribution shift
Primary area: privacy
TL;DR: We show that contrary to prior concerns, public features can be extremely helpful in private transfer learning even when the transfer task is significantly out of distribution. We propose a theoretical model to support our empirical results.

Abstract: Public pretraining is a promising approach to improve differentially private model training. However, recent work has noted that many positive research results studying this paradigm only consider in-distribution tasks, and may not apply to settings where there is distribution shift between the pretraining and finetuning data---a scenario that is likely when finetuning private tasks due to the sensitive nature of the data. In this work, we show empirically across three tasks that even in settings with large distribution shift, where both zero-shot performance from public data and training from scratch with private data give unusably weak results, public features can in fact improve private training accuracy by up to 67\% over private training from scratch. We provide a theoretical explanation for this phenomenon, showing that if the public and private data share a low-dimensional representation, public representations can improve the sample complexity of private training even if it is \emph{impossible} to learn the private task from the public data alone. Altogether, our results provide evidence that public data can indeed make private training practical in realistic settings of extreme distribution shift.

On the Benefits of Public Representations for Private Transfer Learning under Distribution Shift
Pratiksha Thaker Carnegie Mellon University pthaker@andrew cmu, edu
Amrith Setlur Carnegie Mellon University asetlurGandrew cmu. edu
Zhiwei Steven Wu Carnegic Mellon University zstevenvu@andrew cmu edu
Virginia Smith
Carnegie Mellon University smi thveandrew. cmu. edu
Abstract
Public pretraining is promising approach improve differentially private model training. However, recent work has noted that many positive research results studying this paradigm only consider in-distribution tasks, and may not apply settings where there is distribution shift between the pretraining and finetuning data scenario that is likely when finetuning private tasks due the sensitive nature of the data. In this work, we show empirically across three tasks that even in settings with large distribution shift, where both zero-shot performance from public data and training from scratch with private data give unusably weak results, public features can in fact improve private training accuracy by up to 67% over private training from scratch. We provide theoretical explanation for this phenomenon, showing that if the public and private data share a low-dimensional representation, public representations can improve the sample complexity of private training even if is impossible to learn the private task from the public data alone. Altogether, our results provide evidence that public data can indeed make private training practical in realistic settings of extreme distribution shift.
Introduction
Learning models from user data can potentially disclose sensitive user information, violating privacy constraints [T.] Differential privacy is standard framework that can be used when learning models from sensitive data to mitigate the risk of leaking private information [4]. However, differentially private learning may significantly degrade accuracy, which remains a barrier to adoption [5]. This has motivated recent works to explore the benefits of incorporating publicly available data into private training, e.g., by pretraining model public data and then finetuning it using private data. Empirically, this paradigm has been shown to substantially improve performance on private tasks relative to spllvtt training 13
While these results are encouraging, Tramèr et al. 14 point out that much of the existing work focuses on in-distribution tasks, where the public and private tasks are very similar. For example, many private vision models [9] use public features pretrained on ImageNet [20). CIFAR- 10 or CIFAR- 100 (21] but these works also simulate private transfer performance by finetuning
of these datasets.
In fact, Tramèr et al. [4] point out that "every single class contained the CIFAR- 10 dataset has an identical class label in the ImageNet dataset!" This is particularly problematic when attempting to understand the utility of public pretraining for private tasks, because in practice the private task is likely contain sensitive data that is perfectly represented by public data, such as in applications in medicine [2 or law 23 Indeed, data is already well-represented in a public dataset, the zero-shot
38th Conference on Neural Information Processing Systems (NeurIPS 2024).

---

performance of model trained only on public data should be good enough that no private "transfer" learning is required. potentially making these benchmark datasets uninformative for evaluating the benefits of transfer learning.
From a practical perspective, it is particularly important understand transfer learning in the private setting: if' a poon-privacy-seniiiie
task is poorly represented by the pretrained features, one solution might be simply add the data from that task into the public training dataset and learn more general set of features for downstream use. But privacy-sensitiee and individual private datasets often cannot be combined or shared. Thus, the ability to leverage public features to improve the sample dependence of private learning is critical.
data cannot be used to train a public backbone,
Our contributions. In this work, we provide evidence to alleviate these concerns, showing theoretically and empirically that public pretraining can be helpful even in settings with realistic and possibly extreme distribution shift between public (training) and private (transfer) tasks. In particular, we focus on concept shift, where the conditional distributions P(Y| [X) can vary drastically between public and private tasks. Our results are summarized as follows.
First, we conduct empirical case studies on three datasets to show that public features improve private training accuracy even under extreme distribution shift. In particular, we use a pretrained CLIP ViT-B vision model for public features and measure the accuracy of private transfer learning on datasets including the PatchCamelyon (PCam) (24), Functional Map of the World (fMoW) (25), and Remote Sensing Image Scene Classification (RESISC45) (26). On all three datasets, the pretrained model has unacceptably low zero-shot accuracy (random guessing on both PCam and fMo indicating that "perfect privacy" with zero-shot queries is likely hopeless. In comparison, on CIFAR- 10, the CLIP ViT-B/32 model achieves 91.3% zero-shot accuracy [27], making transfer learning performance far less relevant the zero-shot accuracy is already high. We observe that across all datasets, private finetuning and linear probing using public features outperform differentially training from scratch by up to 67%. In addition, private linear probing consistently outperforms private finetuming.
Motivated by our empirical results, we provide a stylized theoretical model to understand and explain our findings. We study a simple linear transfer learning model, a common theoretical model in the non-private meta- leaannen literature 22
to show the statistical benefit of learning a shared, low- dimensional representation (in our model, a low-rank linear subspace) using public data. Our transfer learning model captures an extreme form of concept shift shiftt in the sense that the target model model on private data is entirely different from those on public data, even though they are all contained in the same subspace. Analogous to the paradigm of public pre-training then private linear probing, we analyze a simple two- stage algorithm that (1) first estimates the shared, low-dimensional representation (or subspace) from diverse set of tasks in public data, and (2) performs private linear regression within the learned subspace. By leveraging the dimensionality reduction, we provide better sample complexity that scales with the rank of the shared subspace instead of the ambient dimension of the features. To complement this sample complexity hound, we also show novel lower bound that shows that our bound is tight among algorithms that search for regression parameters within a fixed low-rank subspace estimate.
In short, our findings provide optimistic insights regarding the concerns raised by Tramèr et al. [1] Specifically, Tramèr et al. [44 suggest that "current methods for large-scale pretraining may be less ef- fective." In contrast, our results indicate that pretrained features can indeed benefit private learning, even under concept shift. Additionally, our findings address another concern from Tramer al. [14 regarding the necessity of uploading private data to cloud services for finetuning large models due to high re- source requirements. We demonstrate that training linear probe privately is more effective, potentially requiring significantly fewer resources (both memory and computation) than finetuning full model.
2 Related Work
Empirical studies of public pretraining for private learning. As Tramer et al. 14 point out, existing empirical studies on public pretraining for private learning largely focus on transfer between similar datasets. For example, [15119 35) pretrain on CIFAR-100 or ImageNet and finetune on CIFAR-10 or STL-10 (a dataset very similar to CIFAR-10). ( pretrains on Places365 and finetunes
on ImageNet. [88 36) pretrain on JFT and finetune on ImageNet. Finally, (3 38 9 39 pretrain and finetune on publicly available text on the Web.
'Code will be made available
2

---

All of these works build evidence that pretraining could be beneficial for private learning. Unfortunately, because the public and private tasks are so similar. these results are unlikely to be representative of real-world private training in which the private task requires learning model on sensitive data with a very different distribution from data available on the Web.
Recent work [40] evaluates their algorithm on private learning tasks that are out-of-distributi for the feature extractor they use, including the PCam dataset that we also study, However, their algorithm requires access to (nearly) in-distribution public data in order to learn a projection matrix into a low-dimensional space. We argue that this is strong and unrealistic assumption considering the arguments put forth in Tramèr et al. [14] that private data, because of its sensitive nature, will not be well-represented by public datasets. Our work instead focuses on understanding the improvements from using off-the-ste the-shelf feature extractors, with no in-distribution public data, over aally--rrvvte learning.
Transfer or meta-learning. Our results build on the framework of Tripuraneni et al. (28] for
nonprivate transfer learning with a low-dimensional subspace. This linear, low-dimensional subspace assumption has been studied extensively in the nonprivate meta-learning literature as tractable model for real shared representation learning (28/34). However, none of these works consider the setting of public subspace estimation followed by private transfer learning. PILLAR [40] makes shared subspace assumption in the private setting, but on the input features rather than on the models.
Private algorithms that leverage public data. A number of prior works have theoretically studied the benefits of public data in other settings, including mean estimation 4 query release 42 [4] and optimization when gradients lie in a low-rank subspace 47 Kairouz et al. 45 in particular gives similar analysis using the principal angle error of the subspace, but the analysis does not apply directly as we assume that models, rather than gradients, lie in a shared low-dimensional subspace. As result, the algorithm in that work requires expensive subspace oracle calls every iteration and would be computationally suboptimal in our setting.
Finally, as discussed earlier, pretraining has empirically been shown to be useful in number of domains, including vision [68 and NLP 12] While our work does not model the complexities of neural networks, we understand our results a stylized version of finetuning in which the public network is tuned with linear regression on the last layer, potentially giving insight into these more complex models.
Theoretical analyses of pretraining for private learning. Ganesh et al. [35] provides a lower bound construction for a related setting in which public data is abundant and the private task is out of distribution, though does not consider the case where the public and private task explicitly share structure. In our setting, learning from the public data alone provides no guarantees on the transfer task, as we do not assume any bounded shift in the data distributions or target parameters between the public tasks to the private tasks: the key information enabling more efficient learning the shared structure among the tasks. PILLAR [40] incorporates public pretraining, but their analysis focuses on the benefits of dimensionality reduction using in-distribution public data, rather than transfer from out-of- distribution public data. Finally, Ke et al. [99 study the tradeoff's between linear probing and finetuning in the private setting. While their empirical results focus on the in- -ndistribution image recognition settings outlined previously, their theoretical results corroborate our findings that even under extreme distribution shift, linear probing is more effective than finetuning under differential privacy.
3 Preliminaries
Notation. Throughout the paper, we use lower-case v for vectors, upper-case V for matrices and calligraphic V for sets. Generally, we use the "hatted" notation B, â to refer to estimates of the underlying population variables. The use of O, θ is standard and Ô 5 hides polylog factors in quantities specify separately. We use ||. || for Frobenius, || Ilop for operator and I1-1Ip for Ep norms.
3.1 Differential Privacy
Differential privacy (DP) is a quantitative constraint on the information gained from a released statistic 48). Definition 3.1 restates the standard (c,,)-diiffrentis privacy introduced in
Definition 3.1 ( diifferenttt privacy 44). Given 2 0, E [0,1] and neighboring relation C, randomized mechanism X + from the set of datasets of size n to output space y is
3

---

diffeeriffeeentialll private ifr all neighboring datasets S' CX,, and all events E E Pr[M(S)EE] S e" E
Here, probabilities are taken over the random coins of M.
The "neighboring" relation differs according to the desired privacy guarantee. In this paper, we will study row-level privacy in which neighboring datasets S S" differ in a single element.
3.2 Problem Setting: Leveraging public samples for private transfer learning
We will study setting in which the learner first sees n, public samples (1)): possibly drawn from multiple different underlying tasks (i.e., sample distributions) Pi..... Pt. and then sees n2 private samples from new task Pt-1- The goal is to learn a predictor f R + y that maps inputs x E R" privacy. We aim to minimize
to outputs E Y with the constraint that f must satisfy the population loss on the private task:
(()) (((((r)
The private learner may may not use the public samples. We assume the samples drawn i.i.d. conditioned on the task, but make no other assumptions on the task distribution or the number of samples drawn from each task. In Section the public and private tasks that allows the learner to effectively leverage information from the public tasks to improve private learning.
develop theoretical model of the relationship between
Public Data Improves Out-of-Distribution Private Transfer
We begin by studying three datasets and show empirically that public data can provide benefits for private transfer learning even when the public data alone gives unusable zero-shot results on the private task. Each of the tasks evaluate on has unusably low zero-shot performance CLIP [27], indicating that these are highly citt-f-dditributiinn relative to the pretraining data. This directly contrasts with existing work: the CLIP model that we (pretrained with LAION-2B) achieves 66.6% zero-shot performance on ImageNet and 93.5% accuracy on CIFAR-10.
4.1 Datasets
The PatchCamelyon (PCam) medical images dataset is binary lymph node tumor detection task highlighted by [14. [14] point out that CLIP (27 as well as other similar text-vision mod-
PatchCamelyon.
PCam: CLIP VTT-B/32 achieves 51.2%, or close random, in our evaluation. The poor zero-shot performance (relative to tasks like ImageNet or CIFAR) indicates that the task is truly "out of distribution" in comparison to the source (public) data. Moreover, being medical image data, PCam more faithfully represents a highly privacy-sensitive dataset.
els [2 have notably poor zero-shot performance
While the next datasets not medical image datasets, they are widely studied distribution shift datasets that have poor zero-shot performance on the training data, making them suitable for understanding transfer learning performance. In particular, they are remote sensing datasets; Wang et al. [49] analyze LAION-2B and find that only 0.03% of samples are remote sensing images, another strong indication that this data is underrepresented in pretraining.
is a 62-class satellite image classification task. The pretrained CLIP VYT-B model achieves only 1.64% zero-shot accuracy, so "perfect privacy" with zero- shot classification is not possible.
IMoW. The Functional Map of the World (fMoW) dataset 125] 50
RESISC45, The Remote Sensing Image Scene Classification dataset (26 is a 45-class satellite image classification task. The pretrained CLIP ViT-B model achieves 56.3% zero-shot accuracy.
4.2 Experimental Setup
We train a ViT-B/32 model [51 on each dataset (which has output dimension 512) with a linear classification head for each task. For models trained from scratch, we use Xavier initialization

---

PCam
IMoW
RESISC45
Zero-shot CLIP
51.2
1.64
56.3
Full training from scratch
78.2
19.7
41.9
Full finetuning
82.5
58.2
93.6
Linear probing
83.5
42.1
91.7
Table 1: Test accuracy of nonprivate training each dataset that evaluate.
Pasetaring prohing
Plaeetenting probing
scratch) Plartuning prokung
50.0
30
Accuras
Accuracy (S)
30-
Accuracy
60
40
75.0
20
72.
0.4 0.6
1.0
2.0
5.0
0.4
0.6
1.0
2.0
4.0
0.4
0.6
1.0
2.0
10
Epsilon (e)
Epsilon
Epsilon (e)
(a) PCam.
(b) IMoW.
(c) RESISC45.
Figure 1: Private training three datasets. (a) PCam binary classification task which private training from scratch achieves relatively high accuracy, linear probing on the pretrained model still improves accuracy up to 4%. (b) The fMoW model trained from scratch is unusable at low privacy levels while linear probing achieves close to nonprivate accuracy. (c) On RESISC45, linear probing outperforms full finetuning by over 50% at all levels.
models initialized with weights We use the Opacus library [54] implement private training. For each training setting we performed hyperparameter sweep over learming rate ([1e-6..,,le le- 2)) and number of epochs (1-10 for full training and 1000-2500 for linear probing). and for private learning, clipping norm 1,..5,1.0.25550. For both private and nonprivate models, we evaluate training from scratch, full finetuning, and linear probing. We train private models for [0.3, 0,,,0,4,,.55,.00000 2.0. 2.0,5.0. for each training setting. For PCam and RESISC45, we use SGD with momentum (parameter 0.9). while for fMoW we found that Adam gave better performance [55] We use cosine learning rate schedule for all experiments and batch size of 32. Each finetuning run is performed on an A100 or A6000 GPU.
the weights, while for pretrained features, we use OpenCLIP
12
pretrained using LAION-20 2B (a 2B-samlee subset of LAION-5B (53
4.3 Results
We plot our private training results in Figure
and also provide nonprivate training and zero-
PCamm (Dalacompp)
Dilo (Uataccmup)
RES15C45 (Matacomu))
shot CLIP numbers for reference Table Zero-
(LAION)
shot CLIP has random accuracy on PCam (bi- nary) and IMoW (62 classes). On RESISC45, zero-shot CLIP performs better than training from scratch (nonprivately), but finetuning and linear probing have nearly 40% higher accuracy. As pointed out by Tramèr et al. [14], if the zero- shot numbers (with no knowledge of the transfer task) matched the best performance of finetun- ing, then "perfect privacy" with no finetuning would be sufficient. But in each of these settings, the zero-shot performance is considerably worse than what is achievable with finetuning in both the nonprivate and private settings.
80
Accuracy (%)
60
40
20
0.3
0.40.5
1.0
2.0
5.0
Epsilon (e)
Figure 2: Lincar probing results for ViT-B/32 pretrained on a 14M subset of Datacomp- 1B and on LAION-2B. (Solid lines are LAION results while dashed lines are Dat- acomp results.) While the linear probing results in both settings outperform training from scratch, the worse ac- curacy on the Datacomp pretrained features are reflective of the lower- quality features from the smaller pretraining set.
Across all datasets, we find that any type of finetuning significantly outperforms training pri- vately from scratch. This indicates that the pre- trained features are indeed contributing to train- ing accuracy. Further, we find across all datasets that linear probing (fixing the pretrained features) outperforms full finetuning, sometimes by large margin, in the case of RESISC45. This finding consistent with theoretical work [19] that models
5

---

the benefits of linear probing over finetuning under differential privacy. This is also consistent with earlier empirical findings on iindistribution) private finetuning (8)
The key takeaway is positive: that features that work well for nonprivate transfer learning also benefit private transfer learning even when the distribution shift is large. While the conclusions similar, these results are especially important in the private setting: training models from scratch with strong privacy simply infeasible for many tasks, resulting in only around 10% test accuracy for fMoW and RESISC at small values o
To further support results, additionally evaluate linear probing for all three datasets with features pretrained on a 14M subset of Datacomp- 1B [56] in Figure The trends in this setting are the same and linear probing still outperforms private training from scratch on all datasets, but the smaller pretraining dataset leads to lower-quality features that impact the final accuracy of linear probing.
Theoretical Model
Our empirical results show that even when distribution shift is extreme, public pretraining can indeed improve the accuracy of private training. In order to explain this observation, we study a simplified linear regression setting in which the goal is to estimate regression parameters privately for a single, unseen private task. This setting has been studied extensively the nonprivate meta-learning literature as theoretically tractable model to explain results on larger models [28] and we propose a novel extension to the private setting that helps explain our empirical findings.
We show that if the regression parameters for the private task lie in a low-dimensional subspace that shared with the public tasks, the learner can use the public data to efficiently estimate the low- dimensional subspace. project the private data into the subspace, and thus achieve private estimation error rates that match optimal private linear regression rates (up to constant factors) in & dimensions (rather than d dimensions), with an additive term that accounts for the error in estimating the subspace publicly. These results hold even when we make no assumptions on the relationship between the public and private task other than that they share the same low-dimensional subspace.
We additionally provide a novel lower bound that shows that the algorithm we analyze for our upper bound achieves the optimal rate among "two-stage" algorithms that estimate the transfer parameters within a fixed fow-dimensional subspace.
How realistic the shared subspace assump- tion? As mentioned, the theoretical model we analyze has been previously studied to explain meta-learning results in nonprivate settings. Nev- ertheless, one might ask how realistic the model for the particular settings we study, especially the assumption of a low-rank subspace shared by both the training and transfer tasks.
Eigervalues of Feature Covariance Matrix (PCAM)
0.0008
Figewwiitn
0.0006
0.0004
As a step toward understanding whether this as- sumption holds in practice, plotted the eigen- spectrum of the feature covariance matrix com- puted after extracting features of PCam images from the CLIP ViT-B-32 pretrained model (Fig- ure 3}
0.0002
0.0000
100
300
400
700
800
Index
Figure 3: Eigenspectrum of feature covariance matrix for PCam features extracted from pretrained CLIP ViT-B/32 model.
From these results, we see that the pretrained features are approximately low-rank for the out- of-distribution task PCam, yet a linear probe over these features achieves good (83.5%) performance (Table []. The fact that the representation still gives good performance when only linear layer is trained on top suggests that the data does fundamentally lie in or near the low-rank space that is identified by the pretrained model.
In Appendix A, we plot and see similar results for the fMoW and RESISC45 datasets, where linear probing is similarly successful (relative to full finetuning).

---

5.1
Model and preliminaries
We first describe our model of the data distribution for the private task, learning objective, any assumptions we make and results from prior works we use.
5.1.1 Shared task structure
We consider linear regression models in which every observation (2r.1)) for given task is generated according to:
2; (OO.Ta)
17 ~~01)
y/ = =x Bax())+7
(2)
The covariates F; and noise 7 are sampled i..dd Here, E Raxk an unknown, low rank (h < d) feature matrix with orthonormal columns. The matrix B. and consequently the subspace spanned by its columns, is shared across all tasks in our problem setting. This includes both the public tasks that may be used to derive the initial estimate of B, well as the private tasks in single-task and multi-task transfer settings.
The task vectors a, all assumed to lie in the true shared subspace B. t(i) indexes the task aj for the covariate z;: public tasks are in a o..... .t. and the transfer task is 04 1. Note that the tasks are not random variables and we do not make distributional assumptions on the tasks for our results. In Appendix we provide details on the requirements for the public tasks a1... (and also refer the reader to Tripuraneni et al. (88 ) but for now we simply require that the public tasks sufficiently "diverse" within B.
The learner sees nı samples from the public tasks (in total across all tasks) and n2 samples drawn from the private task.
We are interested in learning that minimizes the following population risk:
((w)= 2 (a [(x T w w
on the private task Bat-1-
5.1.2 Oracle for public subspace estimation
In stating our main results, we first assume access to an oracle that can output an orthonormal matrix BER Bakk that "close to" B. We measure the distance between subspaces in terms of the principal angle distance, denoted sinO(B, (B, B) =ind (B. (B (see supplement and Tripuraneni et al. (28) for more discussion).
The following identities on sin will be useful: Lemma 5.1 (subspace estimation errors). The following inequalities are satisfied for matrices with orthonormal columns B, / E ødxk (and when B, B are swapped): [[(1 Bll 2 [I(I BBT B = sin B3 B) 2 (I-B8 }BI r/VE.
Instantiating the oracle with public data. The following corollary characterizes the error incurred from estimating the underlying subspace from public data using the method method-of-moomen estimator from Tripuraneni et al. (28). We state this bound for use in subsequent results but refer the reader the supplement for the conditions required on public data in order to achieve this bound.
Theorem 5.2 ((28), Theorem 3, simplified). Let A = (a)) be the public task matrix,
σk A (AAA). and R
tr( (T(ATA)) be the average condition number. If an equal number of samples is kv
generated from each task, and R < 0(1) and v 2 a(t)). then the error of the oethend-of-moments estimator (28). Algorithm 1) is
sin0 (BB) -0 Vokk2
dk dk2/mm)
(4)
with probubility least 1 O(1 100
We will refer to 7 2 sin B, B3 upper bound on the error of the subspace estimation oracle. We give upper bounds with respect to 7 and also instantiate the bounds with the upper bound from 552
7

---

Algorithm/ 1 TWo--pase algorithm for public-private linear regression using subspace estimation
Input: ni public samples drawn according (I..81). 2, ~ (O,, y = z (a)()) +7, TJ (O.1)
and r2 private samples where 2;, '/i have the same distribution, and y/i = Bax+1 +
1: Use method-of-moment
estimator (28), Algorithm 1) to estimate B using public data
2: Project private data to k-dimensional subspace: = ! ß
3: Use DP-SGD variant of (57) on projected private data to estimate 0xt+111
Output: Parameter estimate B
5.1.3 Private linear regression in d dimensions
We use in our analysis a known upper bound for private linear regression in d-dimensions. Theorem 5.3 states an informal result from [57] that upper bounds the excess risk for variant of DP-SGD [[5] (see Appendix for more details). Furthermore, results from [58] imply that this upper bound is tight.
Theorem 5.3 (Corollary 11 from [57] simplified), Suppose we have na i..dd datapoints (s1,3)) where I; (0, and y, =z. w tt, and e; ~ (0,o²). Given sufficient private samples n2g there exists an (5,6) private estimate 10 priv such that, with high probability:
do²
(1+0
(1 NzE2
(dlog(1/b) (1/6) )
[((epri)) [(w) S
122
5.2 Private transfer learning for single task
Algorithm. Our proposed algorithm (Algorithm) [ first projects x into the estimated subspace Bpub. i.e., x BT 2, u. and then runs private linear regression in the k-dimensional subspace. This analogous linear probing in our experiments, which first uses the public encoder to compute oow-dimessional feature representation of the data and then learns a linear model using the features. While full finetuning of the model also common paradigm in the transfer learning literature, we point to [19] which shows that when the feature representation is sufficiently informative, linear probing outperforms finetuning under differential privacy - result that supports our empirical findings.
The following theorem states that Algorithm achieves rate that matches optimal rates for private linear regression in k-dimensions, up the subspace estimation error 7
Theorem 5.4 isingle-task private transfer upper bound). Assume have access subspace estimation oracle that solely uses public samples provide estimate Byubfor the unknown subspace B of private task defined by the pair (B,0(+1) in (2). Further, the estimate satisfies sin 0 B) Sni Given n2 i..dd samples from the distribution of this private task, Algorithm Z oouputs an estimate & that is (..f)ddffrrentiallyy private, and with high probability incurs risk of:
L( C(Bahhhtt) L[((e)++))))
(6)
s 00(00+111(67+
12(2)
k
k:
looog(11/)
+72
100
n
122
Y12
Proof sketch. The proof nearly follows from existing bounds on subspace estimation and private linear regression. The key difficulty is that regression on the =/ projected into the estimated subspace Bpub still leaves the residual that does not lie in Bpub, which can be treated as noise term we can show that the residual is independent of the projected zr We can show this because Bpub is orthogonal to By.. (spans null space of Bpmb),, so under the i.i.d. Gaussian assumption on x, the residual is independent of the projected x. As a result, we obtain the private linear regression rate in dimensions with variance of 1 +22 rather than and an additive x² bias.
From Theorem 5.4 we can break down the errors into an unavoidable bias due to the subspace estimation error (dependent only on the number of public samples) and the subsequent linear regression error due to privacy. For a subspace estimation error y we require ni 2 7 Given this inevitable error hope to achieve accuracy of err + + where err is the additional linear
Discussion.

---

regression enror and sin 0 (B, p Sy This requires approximately:
k
Y22
(8)
err
E everr err
samples. That is, if the subspace estimation error is zero then we achieve the rate of private linear regression dimensions, and consequently optimal non-private rates when - 00.
5.3 Lower bound for two-phase estimator
In the previous subsection, we proved an upper bound on the single-task transfer for row-level (D private algorithm, when the publicly estimated subspace Bpub is accurate. In this section, we show that our upper bound is tight among algorithms for our problem that search for solutions within fixed subspace.
In partieular, we analyze the lowest possible transfer error achieved by any (e,6)-DP algorithm that: (i) takes as input private dataset S of n2 i.i.d. samples from task 04++1 y-accurate public estimate Bpub- and (ii) outputs an estimate in the column space of B Bpub- In Theorem 5.5 we present lower bound on the risk suffered by any algorithm in such class.
Theorem 5.5 (Two-stage single-task private transfer lower bound). M be an (s,6)-DP private algorithm where e E (0.1), 8 << <1/nt*,, w >>>0 O, that takes as input: (i) publicly estimated subspace Bpab from oracle that only uses public samples; and (ii) dataset of n2 private samples. For any such M, there exists a private problem instance given by the pair (B,004) where B E GT (R), ERK sin O( B, Brub)) S and |Bo4+1ll 1|2 S such that for S sampled ii.dd from this instance using the model in (2), we have:
EM E B(a.u) (U (S, (pub)))) x)
122
= 0
(o" (0)==))))7²)
(10)
Proof Sketch. Our proof relies mainly tracing attacks in 59 588, but our analysis additionally needs to handle the misspecification of the subspace B which influences the construction the worst case problem instance. When we project imputs x + B T, we can show that the projected samples can now be treated as i.i.d. samples from a k-dimensional linear regression model with independent noise. For fixed Bpub, any choice of B,a++1 affects both the scaling of the noise (ox [| Bpub Bp.b Ba l13), and the direction the regression vector, based on how much of the true parameter Bo++1 captured in given subspace Bpub- To handle this, we first construct subclasses the adversary, where each subclass fixes the norm of [| pub Batt 2l2. Then, we lower bound the minimax risk over this subclass by via Bayes risk which we further lower bound by constructing a tracing adversary.
Pu
We show that there exists prior over Boc+1 where the probability of the intersection of the following two events very low: (i) small estimation error R. C(M(S Bpub))) and (ii) small success rate for the tracing adversary to infer the membership of some element S. Since, M has to be (c,5) private, this reults in Bayes risk lower bound.
Discussion. Our lower bound for the class of two-stage algorithms matches our upper bound in Theorem 5.4 This implies that our Algorithm ] is optimal when Apub is the estimate given by the Algorithm from (28), the estimation enror matches lower bounds (Theorem 5 in (28) upto factor of Vk.
optimal subspace estimation oracle over public samples. When
5.4 Simulated results
Finally, we complement the results in this section through simulated empirical study matching the setup described in Section 5.1
Setup. We simulate ni samples (II.9:) from t= 100 public tasks where the true dimension d= 25 but the underlying subspace has rank 5. As baselines, we compare against nonprivate linear regression, DP-SGD without a subspace estimate, and DP-SGD initialized with the true subspace B, and compare against DP-SGD initialized with the subspace estimated using the method-of-mooentss estimator 28 We use the Google Tensorflow implementation of DP-SGD for private learning [60].

---

We used grid search of hyperparameters to set the clipping norm 0.5, learning rate to 0.1, and used 50 epochs of training for DP-SGD. We use the RDP accountant to set e =11 and 1e-5.
Perameter eiror for T=100, d=25, k=S
Kenjevste
DP55ED subsoace
with MoM 2000
Our results are shown in Figure 4 We ob-
serve that, as expected, private training from scratch has high error, and additional public data
(01=500 vs 1 = 2000) improves performance, reducing the &g parameter error close to that of using DP-SGD with the true underlying subspace B (matching our intuition, for example, from Fig- ur However, we also see that when perform- ing private transfer there are diminishing returns for this more precise subspace estimation, as the noise introduced via private learning becomes dominating factor.
2000
4000
6000
5000
10000
12020
24003
Namber essempipes es private francfer tadk (rg)
Figure 4: Empirical verification of setup described in Section 5.1
6 Discussion and Limitations
Our results answer questions posed by [4] positively. Empirically, we show that across three datasets with significant shift between the public and private tasks, publicly pretrained features do make private learning far more effective, taking models from unusable when trained from scratch to close-to-nopprivate performance when trained privately with linear probing. In addition, we provide a theoretical model to explain our findings, based on models of nonprivate transfer learning. Our model supports our empirical findings, suggesting that public features should indeed reduce private sample complexity under even extreme distribution shift when the public and private tasks share low-dimensional representation. Altogether, our conclusions are optimistic and provide confidence that public data can indeed support private training even for highly sensitive tasks that cannot and should not be used in public training. However, our linear subspace model has the clear limitation of being simplified model for the neural network representations used in practice. As this is limitation shared by literature on nonprivate transfer learning [28 34, improvements in this area would contribute to both the private and nonprivate transfer learning literature.
Acknowledgements. Thanks to Shengyuan Hu, Tian Li, Qi Pang, and Anirudh Sivaraman for helpful discussions and feedback that improved the writing. This work supported in part by the National Science Foundation grants IIS2145670 and CCF2107024, and funding from Amazon, Apple, Google, Intel, Meta, and the CyLab Security and Privacy Institute. Any opinions, findings and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of any of these funding agencies. Z.S. W. was in part supported by NSF Awards #1 #1763786 and #2339775.
References
[1] Matt Fredrikson, Somesh Jha, and Thomas Ristenpart. Model inversion attacks that exploit confidence information and basic countermeasures. In Proceedings of the 22nd ACM SIGSAC conference on computer and communications security, pages 1322--1333 2015.
[2] Reza Shokri, Marco Stronati, Congzheng Song, and Vitaly Shmatikov. Membership inference attacks against machine learning models. In 2017 IEEE symposium on security and privacy (SP), pages 3-18. IEEE, 2017.
[3] Nicholas Carlini, Florian Tramer, Eric Wallace, Matthew Jagielski, Ariel Herbert- Voss, Katherine Lee, Adam Roberts, Tom Brown, Dawn Song, Ulfar Erlingsson, et al. Extracting training data from large language models. In 30th USENIX Security Symposium (USENIX Security 21), pages 2633-2650, 2021.
[4] Cynthia Dwork, Frank McSherry, Kobbi Nissim, and Adam Smith. Calibrating noise to sensitivity in private data analysis. In Theory of cryptography conference, pages 265-284. Springer, 2006.
10

---

[5] Rachel Cummings, Damien Desfontaines, David Evans, Roxana Geambasu, Matthew Jagiel- ski, Yangsibo Huang, Peter Kairouz, Gautam Kamath, Sewoong Oh, Olga Ohrimenko, et al. Challenges towards the next frontier in privacy. arXiv preprint arXiv:2304.06920 2023.
[6] Aditya Golatkar, Alessandro Achille, Yu-Xiang Wang, Aaron Roth, Michael Kearns, and Ste- fano Soatto. Mixed differential privacy in computer vision. In Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition, pages 8376-8386, 2022.
[7] Zelun Luo, D Wu, Ehsan Adeli, and Li Fei-Fei. Scalable differential privacy with sparse network finetuning. In Proceedings the IEEE/CVF Conference Computer Vision and Pattern Recognition, pages 5059-5068, 2021.
[8] Alexey Kurakin, Shuang Song, Steve Chien, Roxana Geambasu, Andreas Terzis, and Abhradeep Thakurta. Toward training at imagenet scale with differential privacy. arXiv preprint arXiv:2201.1233 2022.
[9] Da Yu, Saurabh Naik, Arturs Backurs, Sivakanth Gopi, Huseyin A Inan, Gautam Kamath, Janardhan Kulkarni, Yin Tat Lee, Andre Manoel, Lukas Wutschitz, et al. Differentially private fine-tuning of language models. In International Conference on Learning Representations, 2021.
[10] Jiyan He, Xuechen Li, Da Yu, Huishuai Zhang, Janardhan Kulkarni, Yin Tat Lee, Arturs Backurs, Nenghai Yu, and Jiang Bian. Exploring the limits of differentially private deep learning with group-wise clipping. In The Eleventh International Conference on Learning Representations, 2022.
[11] Zhiqi Bu, Yu-Xiang Wang, Sheng Zha, and George Karypis. Differentially private optimization on large model at small cost. In International Conference on Machine Learning, pages 3192-3218. PMLR, 2023.
[12] Antonio Ginart, Laurens van der Maaten, James Zou, and Chuan Guo. Submix: Practical private prediction for large-scale language models. arXiv preprint arXiv:2201.009) 2022.
[13] Yingxue Zhou, Steven Wu, and Arindam Banerjee. Bypassing the ambient dimension: Pri- vate SGD with gradient subspace identification. In 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021, 2021.
[14] Florian Tramèr, Gautam Kamath, and Nicholas Carlini. Considerations for differentially private learning with lcage-scale public pretraining. arXiv preprint arXiv:2212.06477, 2022.
[15] Martin Abadi, Andy Chu, Ian Goodfellow, H Brendan McMahan, Ilya Mironov, Kunal Talwar, and Li Zhang. Deep learning with differential privacy. In Proceedings of the ACM SIGSAC conference on computer and communications security, pages 308-33 318, 2016.
[16] Nicolas Papernot, Steve Chien,Shuang the shoe fit: Architectures, initializations, and tuning for learning with privacy. 2019.
Shuang Song, Abhradeep Thakurta, and Ulfar Erlingsson. Making
[17] Florian Tramer and Dan Boneh. Differentially private learning needs better features (or much more data). In International Conference on Learning Representations, 2020.
[18] Soham De, Leonard Berrada, Jamie Hayes, Samuel L Smith, and Borja Balle. Unlock- ing high-accuracy differentially private image classification through scale. arXiv preprint 2xXiv:2204.33650 2022.
[19] Shugi Ke, Charlie Hou, Giulia Fanti, and Sewoong Oh. On the convergence of differentially- private fine-tuning: To linearly probe or to fully fine-tune? arXiv preprint 18X0 2024.
[20] Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large- scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, pages 248-255. Ieee, 2009.
[21] Alex Krizhevsky, Geoffrey Hinton, et Learning multiple layers of features from tiny images. 2009.
11

---

[22] Hieu Pham, Zihang Dai, Golnaz Ghiasi, Kenji Kawaguchi, Hanxiao Liu, Adams Wei Yu, Jiahui Yu, Yi-Ting Chen, Minh Thang Luong. Yonghui Wu, et al. Combined scaling for zero-shot transfer learning. Neurocomputing, 555: 126658, 2023.
Minh-Thang
[23] Dan Hendrycks, Collin Burns, Anya Chen, and Spencer Ball. Cuad: An expert-annotated nip dataset for legal contract review. In Thirty-fitth Conference on Neural Information Processing Systems Datasets and Benchmarks Track, 2021.
[24] Bastiaan Veeling. Jasper Linmans, Jim Winkens, Taco Cohen, and Max Welling. Rotation equivariant CNNs for digital pathology. June 2018.
[25] Gordon Christie, Neil Fendley, James Wilson, and Ryan Mukherjee. Functional map the world. In CVPR, 2018.
[26] Gong Cheng, Junwei Han, and Xiaoqiang Lu. Remote sensing image scene classification: Benchmark and state of the art. Proceedings of the IEEE, (55(10)1885-1888 2017
[27] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning, pages 8748-8763. PMLR, 2021.
[28] Nilesh Tripuraneni, Chi Jin, and Michael Jordan. Provable meta-learning of linear representations. In International Conference on Machine Learning, pages
PMLR, 2021.
[29] Simon Shaolei Du, Wei Hu, Sham M. Kakade, Jason D. Lee, and Qi Lei. Few-shot learn- ing via learning the representation, provably. In 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021. OpenReview.net, 2021.
[30] Weisen Jiang, James Kwok, and Yu Zhang. Subspace learning for effective meta-learning. In International Conference on Machine Learning, pages 10177-10194. PMLR, 2022.
[31] Nikunj Saunshi, Arushi Gupta, and Wei Hu. A representation learning perspective the importance vaan--valiattion splitting in meta-learning. In International Conference on Machine Learning, pages 9333 93333.93...22 PMLR, 2021.
[32] Liam Collins, Aryan Mokhtari, and Sanjay Shakkottai. Why does maml outperform erm? an optimization perspective. arXiv preprint arXiv:2010.14672. page 6, 2020.
[33] Parker Knight and Rui Duan. Multi-task learning with summary statistics. Advances in Neural Information Processing Systems, 36, 2024.
[34] Ananya Kumar, Aditi Raghunathan, Robbie Jones, Tengyu Ma, and Percy Liang. Fine- tuning can distort pretrained features and underperform out-of-dissriibttin. arXiv preprins 22Xiv:2222.00054 2022.
[35] Arun Ganesh, Mahdi Haghifam, Milad Nasr, Sewoong Oh, Thomas Steinke, Om Thakkar, Abhradeep Guha Thakurta, and Lun Wang. Why is public pretraining necessary for private model training? In International Conference on Machine Learning, pages 10611-10627 PMLR, 2023.
[36] Harsh Mehta, Abhradeep Thakurta, Alexey Kurakin, and Ashok Cutkosky. Large scale transfer learning for differentially private image classification. arXiv preprint arXiv:2205.02973 2022.
[37] Da Yu, Huishuai Zhang, Wei Chen, Yin, and Tie- Yan Liu, Large scale private learning via low- rank reparametrization. In International Conference on Machine Learning, pages 12208-12218. PMLR, 2021.
[38] Xuechen Li, Florian Tramer, Percy Liang, and Tatsunori Hashimoto. Large language models can he strong differentially private learners. In International Conference on Learning Representations, 2021.
[39] Simran Arora and Christopher Ré, Can foundation models help us achieve perfectsecrecy?" arXiv preprint 273.2
12

---

[40] Francesco Pinto, Yaxi Hu, Fanny Yang, and Amartya Sanyal. Pillar: How to make semi aami-private learning more effective. arXiv preprint arXiv:2306. 033.9966..0 2023.
[41] Brendan Avent, Yatharth Dubey, and Aleksandra Korolova. The power of the hybrid model for mean estimation. Proceedings on Privacy Enhancing Technologies, 4:48 68, 2020.
[42] Terrance Liu, Giuseppe Vietri, Thomas Steinke, Jonathan Ullman, and Steven Wu. Leveraging public data for practical private query release. In International Conference Machine Learning, pages 6968 6977.9977800 PMLR, 2021.
[43] Raef Bassily, Albert Cheu, Shay Moran, Aleksandar Nikolov, Jonathan Ullman, and Steven Wu. Private query release assisted by public data. In International Conference on Machine Learning, pages 695-703. PMLR, 2020.
[44] Miguel Fuentes, Brett C Mullins, Ryan McKenna, Gerome Miklau, and Daniel Sheldon. Joint selection: Adaptively incorporating public information for private synthetic data. In International Conference on Artificial Intelligence and Statistics, pages 2404-2412, PMLR, 2024.
[45] Peter Kairouz, Monica Ribero Diaz, Keith Rush, and Abhradeep Thakurta. (nearly) dimension independent private erm with adagrad via publicly estimated subspaces. In Conference on Learning Theory, pages 2717-2746. PMLR, 2021.
[46] Ehsan Amid, Arun Ganesh, Rajiv Mathews, Swaroop Ramaswamy, Shuang Song. Thomas Steinke, Vinith M Suriyakumar, Om Thakkar, and Abhradeep Thakurta. Public data-assisted mirror descent for private model training. In International Conference on Machine Learning. pages 517-535. PMLR, 2022.
[47] Da Yu, Huishuai Zhang. Wei Chen, and Tie-Yan Liu. Do not let privacy overbill utility: Gra- dient embedding perturbation for private learning. In International Conference on Learning Representations, 2020.
[48] Cynthia Dwork, Aaron Roth, al. The algorithmic foundations of differential privacy. Founda- tions and Trends@ in Theoretical Computer Science, (33-4):21--407, 2014.
[49] Zhecheng Wang, Rajanie Prabha, Tianyuan Huang, Jiajun Wu, and Ram Rajagopal. Skyscript: A large and semantically diverse vision- language dataset for remote sensing. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 5805-5813, 2024.
[50] Pang Wei Koh, Shiori Sagawa, Henrik Marklund, Sang Michael Xie, Marvin Zhang, Akshay Balsubramani, Weihua Hu, Michihiro Yasunaga, Richard Lanas Phillips, Irena Gao, al. Wilds: benchmark of in- in-the-wild distribution shifts. In International conference on machine learning. pages 5637-5664. PMLR, 2021.
[51] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition at scale, arXiv preprint UrXiv:2010.11929, 2020.
[52] Gabriel Ilharco, Mitchell Wortsman, Ross Wightman, Cade Gordon, Nicholas Carlini, Rohan Taori, Achal Dave, Vaishaal Shankar, Hongseok Namkoong. John Miller, Hannaneh Hajishirzi. Ali Farhadi, and Ludwig Schmidt. Openclip, July 2021. htttts::://do/ org/ 10 5281/
zenodo. 5143773 If you use this software, please cite it as below.
[53] Christoph Schuhmann, Romain Beaumont, Richard Vencu, Cade Gordon, Ross Wightman, Mehdi Cherti, Theo Coombes, Aarush Katta, Clayton Mullis, Mitchell Wortsman, et al. Laion-5b: An open large-scale dataset for training next generation image-text models. Advances in Neural Information Processing Systems, 35:25278-255 252944, 2022.
[54] Ashkan Yousefpour, Igor Shilov, Alexandre Sablayrolles, Davide Testuggine, Karthik Prasad, Mani Malek, John Nguyen, Sayan Ghosh, Akash Bharadwaj. Jessica Zhao, Graham Cormode, and Ilya Mironov. Opacus: User-friendly differential privacy library in PyTorch arXiv preprint 1009 12298, 2021.
13

---

[55] Mitchell Wortsman, Gabriel Ilharco, Jong Wook Kim, Mike Li, Simon Kornblith, Rebecca Roclofs, Raphael Gontijo-Lopes Hannaneh Hajishirzi, Ali Farhadi, Hongseok Namkoong, and Ludwig Schmidt. Robust fine-tuning of zero-shot models, 2022.
[56] Samir Yitzhak Gadre, Gabriel Ilharco, Alex Fang, Jonathan Hayase, Georgios Smyrnis, Thao Nguyen, Ryan Marten, Mitchell Wortsman, Dhruba Ghosh, Jieyu Zhang, et al. Datacomp: In search of the next generation of multimodal datasets. Advances in Neural Information Processing Systems, 36, 2024.
[57] Prateek Varshney, Abhradeep Thakurta, and Prateek Jain. (nearly) optimal private linear re- gression for sub-gaussian data via adaptive clipping. In Po-Ling Loh and Maxim Raginsky, editors, Proceedings of Thirty Fifth Conference on Learning Theory, volume 178 of Pro- ceedings of Machine Learning Research, pages 1126-1166. PMLR, 02-05 Jul 2022. URL https::/ppron https://proceedings mlr
html
[58] T' T'oy Cai, Yichen Wang, and Linjun Zhang. The cost of privacy: Optimal rates of convergence for parameter estimation with differential privacy. The Annals of Statistics, 49(5): 2850, 2021.
[59] Mark Bun, Jonathan Ullman, and Salil Vadhan. Fingerprinting codes and the price of approximate differential privacy. In Proceedings of the forty-sixth annual ACM symposium on Theory of computing, pages 1-10, 2014.
[60]
Martín Abadi, Ashish Agarwal, Paul Barham, Eugene Brevdo, Zhifeng Chen, Craig Citro, Greg S. Corrado, Andy Davis, Jeffrey Dean, Matthieu Devin, Sanjay Ghemawat, Ian Goodfellow, Andrew Harp, Geoffrey Irving, Michael Isard, Yangqing Jia, Rafal Jozefowicz, Lukasz Kaiser, Manjunath Kudlur, Josh Levenberg, Dandelion Mané, Rajat Monga, Sherry Moore, Derek Murray, Chris Olah, Mike Schuster, Jonathon Shlens, Benoit Steiner, llya Sutskever, Kunal Talwar, Paul Tucker, Vincent Vanhoucke, Vijay Vasudevan, Fernanda Viégas, Oriol Vinyals, Pete Warden, Martin Wattenberg, Martin Wicke, Yuan Yu, and Xiaoqiang Zheng. TensorFlow: Large-scale machine learning on heterogencous systems, 2015. URL http::ttt/////t tensorflow org/ Software available from tensorflow.org.
[61] Camille Jordan. Essai sur la géométrie à dimensions. Bulletin de Société mathématique de France, 3:103-174, 1875.
[62] Gilbert W Stewart and Ji fi-guang Sun. Matrix perturbation theory. 1990.
[63] John C Duchi, Vitaly Feldman, Lunjia Hu, and Kunal Talwar. Subspace recovery from heteroge- neous data with non-isotropic noise. Advances in Neural Information Processing Systems, 35: 5854-5866, 2022.
[64] Thomas Steinke and Jonathan Ullman. Tight lower bounds for differentially private selection. In 2017 IEEE 58th Annual Symposium on Foundations of Computer Science (FOCS), pages 552 563. IEEE, 2017.
[65] Gautam Kamath, Jerry Li, Vikrant Singhal, and Jonathan Ullman. Privately learning high- dimensional distributions. In Conference on Learning Theory, pages 1853-1902. PMLR, 2019.
[66] Alan Edelman, Tomás A Arias, and Steven T Smith. The geometry of algorithms with or- thogonality constraints. SIAM journal on Matrix Analysis and Applications, 20(2)3303-333 1998.
[67] Vladimir Igorevich Bogachev and Maria Aparecida Soares Ruas. Measure theory, volume 1. Springer, 2007.
14

---

A Empirical evidence for shared subspace assumption
Eieevvalues Feature Coveriance
(FNoW)
0.005
0.004
Egeneuu
0.0006
Ligennelg
0.0 0.003
0.0004
0.000
0.0002
0.0000
0.000
700
300
125
600
300
100
200
300
400
530
600
700
800
Index
Index
(a) PCam.
(b) fMoW.
Eige peelvvlles of Feeture Coveriance Metris (RESIS015)
0.0020
Egenvelue
0.0055
0.0005
0.0000
100
200
330
400 Index
500
600
700
800
(c) RESISC45.
Figure 5: Eigenspectra of feature covariance matrices for features extracted from pretrained CLIP ViT-B/32 model.
is natural ask whether the assumption that public (pretraining) tasks and private tasks truly share low-dimensional subspace as we model in our theoretical analysis. In order to validate this, in figure 5) we plot the eigenspectra the feature covariance matrices for each of the datasets we evaluate in Section 4 The key takeaway is that even though the three datasets are out of distribution for the pretraining data, the extracted features are low rank. addition, these features are effective when used to train a linear probe (in contrast, if the subspace were misspecified, the features may be low-rank but lead to poor results with linear probing).
B
Additional definitions and assumptions
B.1
Preliminaries for Tripuraneni al. 28
In this section, we elaborate the assumptions required to use algorithms from Tripuraneni al. for subspace estimation using public data (i.e., instantiating the subspace oracle).
B.1.1 Principal angles
Our analysis requires a notion distance between subspaces, which we use the maximum principal Appendix A, for more details.
angle [61]. We give definition here and refer the reader to [62 (63]
Definition B.1 (Maximum principal angle). Lef U.V E d be orthogonal matrices. Let li and V
be the subspaces spanned by the columns of and V respectively. The maximum principal angle
0 [0.z/2] between U and V defined by sinO(U,V)+))) [ - -| TUlll
B.1.2 Task diversity assumptions
In our model each data point (zi.ys) is associated with a task (4g) R We do not make distribu-
tional assumptions on these tasks, but estimating the subspace accurately requires certain diversity assumptions the tasks. We inherit the following assumption from (28
15

---

Assumption B.2 (Task diversity and normalization). Define A (a) and v= == or I underlying task parameters jj satisfy ||ay|| =+ all Moreover, we assume U>
AAA
The
In the following, we will also use the average condition number A tx AI4 and the worst-case
condition number 01 (TA /v, further characterize the task diversity.
Then have:
Theorem 5.2
28 Theorem 3, simplified). Let A (0)) be the public lask matrix,
and K tr! A
Ok
be the average condition mumber. If an equal number of samples is
generated from each task, and R S O(1) and v 2 ()())) then the ervor of the estimator (28). Algorithm is
sin0 (B.B) 0 Vdk2
(4)
with probability at least 1 O
B.2 Estimation error bounds
We restate the full bound given by [57 for private SGD.
Theorem B.3 (157) Corollary 11, simplified). Suppose we have data (xr.91) such that x; (0.I0,
and y = = w" 4, where 6 (0.o²). Then, assuming 2 k(1 Vlog(1/0) we have:
1. Algorithm DP-AMBSSGD 57 Algorithm 2) with parameters 1 1/4k, /slog(1/8) (D.
is
2. The output upriv satisfies the following risk bound:
((uprio)-L(w>)
|
11
8ko²
[+0
klog(1/0)
(1I)
12222
n22
with probability 100 100).
C Technical lemmas
In this section state restate key lemmas that will refer to in the following proofs.
We will use the following lemma to argue that we can project x into the estimated subspace B and treat the residual (that lies outside of B) as i.i.d. Gaussian noise.
Lemma C.1 (Independence of residual). Consider orthonormal matrices B and B Back and E R* Let (x, y) be generated according to the model in Equation 2 where a ~ N La) and y=x Baty, Then the projection of into B, x (ßВT) Ba, is independent of the residual that lies in the complement of B, i.e. x (Ja- Bo. Moreover, this residual is i.i.d. Gaussian.
Proof. We can rewrite the distribution of y æ in terms of the projection the regression vector Box on to the column span of B, when the input is also projected in the following way: x Br:
x Ba
= Ba BB)) +1)
=BB Ba)+x T )
= =< ( Ba++
(12)
where à= TT Ba a --dimensional vector in the column span of the given subspace B.
16

---

Next, we note that the projection of input x in the column span of B and its projection into the corresponding null space are independent, i.e., x. (a..xT(BBB (BBT) Il x (Ta-BFF BBT). This is easy to show since both x (BBT) and x (Ia BBT) are jointly Gaussian and are marginally distributed as mean Gaussian random variables with their covariance:
E )aE'' T (l2-BBT)
=(BB ) |R(rx 1(a-
= = (LaBLBBBB -B =0.
This implies independence the two projections. Note that the last step in the above calculation uses the fact that T B Ja- Since the two projections are independent, we can rewrite the conditional distribution y x
TTB
y1 N((x= ((( where, 8² =' )
Lemma C.2 (58,64 651) Let M be an (efferefferrentialll private algorithm with 0 < < 1 and
8>0 O. Further, let A; = Aa (Ivr,.)) M(M()) and A'= As ((u,,= (S)) when (y,, ES and S replaces (y,,.)) with another IID draw from the same distribution. Then, for every T> 0,
EA, S 2
(P((A|)>
(13)
Proof. let - (2,0 and -- minnm,0) denote the positive and negative parts of random variable respectively. We have
0c PP -1) / PPA, >1) dt.
EA = EAT - EA,--
For the positive part, if <T < Do and OREE<<<IL 1, we have
1 P(A > t) dt=
PPAt +t)d++ [ P(A > t)dt
dt (E >t)+8) dt+ JT
P(A > t)dt
P(A! > t) +8 1 (P(AA,/s)dd
P(A =t
Similarly for the negative part,
/ P(A dt=
1 ST
P(A7 >t)dt
>1) (<T((A)>)))))) di+ 1 IP(A > t)dt
T
P(A > >))-7> 8T -8T+ 1
P(A[T>t)d >t) -25
P(A >t
?(A[)t)) 25 J. P(A >t)-
It then follows that
/ P(IAA)>1)d
EAA < P(A 11)
(P(( =1)
E.4/ 2 A, +28T+ IP(IA)>()dLL
17

---

Lemma C.3 (Stein's 's Lemma). Let Z be distributed according to some density p(2) that is continuously differentiable with respect to and let h: -R be differentiable function such that Elh((z) <c.
then:
-h(Z)p'(Z)
EA (Z) =
p(Z)
D Proofs for Section 5
D.1 Proof of Theorem 5.4
In this section prove the upper bound result the two-phase algorithm for single-task transfer learning, which first estimates the subspace publicly, projects the inputs into the estimated subspace and then privately performs linear regression the projected data.
Proof. Let sin ( h(11) and (a(rro ,Txx) -) min, B(( ()) (
Let à = min, B =))] (the best a using x projected into B3, and let @ be the output of DP-AMBSSGD on projected into the estimated B. then we have
E
(ammri _T _B x) 27
E [la BT,T)
x) ->>=)" ")²
(laprio .B x) x)) 2
((a", x) - +E ((a. aa) -E
=((Ca.TT x))
=E
We can break this into two parts: will first bound
(arrroo 1TTrio,BTT00 .T) -))= w)² minE (la.BT ")
E
(14)
and then
E
x) (a, "}
( x)
- E (la" ((aat,,Tx) xT -n) =
(15)
We first bound
14). Note that according to the model 2
y=x' Ba +
(16)
where is (0,1).
However, our algorithm first projects æ into the space of the estimated Á before performing linear regression in the lower-dimensional space, which introduces additional error.
We can rewrite as:
y x= BB Ba +x (T- )) + +c
(17)
decomposing the first term into the projection into B and the remaining error due to projection.
By Lemma C.1 this residual term is independent of the first term (with projected into B) and e.
We claim that the variance of the residual is sin())*+1|aa*
Let
= T==T(1- (1-BB) Ba te
(18)
=x B, B Ba +c
(19)
18

---

and note that the total variance is the sum of the variances because the terms are independent. Moreover, the first term is rescaled i.i.d. Gaussian with zero mean. Then the variance of is
[ B BIBI
B +o"
= EEaa" B B1B xx T
B1 Ba"
-E(a" B3 B1B BBBIBA Ba"
=sin(0)²|| a +0".
Moreover, we assume o² so we have var(e') = sin (
Using the rewritten y, we can treat the new private regression problem as estimating 3 Ba". Thus we will instantiate (112) with the linear regression bound from Theorem B.3 with dimensions and variance x² = =inn (in (0)?? || +1.
Now we bound the second half of the expression,
(a. (ā, 2) (t)
=E (la.3" 2) -0) BB" x) ((a" x) - v) - (1a" B x) w))
(a)) xx))² [B Ba Ba 2 =E ((Ba- (Bà-Ba' Ba"
Finally this leaves us to bound| |ßā "Ba-Ba" Ba Baallz lلg. We will make use of the following lemma:
2
Lemma D.1. Let à be the (public) linear regression estimate of the task on the projected data B
Then |ßa [ 11 (Bin BBa*
Proof.
Bâ-Ba" Ba
- xx B] ) Birx Ba " Ba*
BEEBB xr B
ra Ba
Ba*
-B xxx ( E rx
[BB Ba + Ba") --Ba Ba*
-B(EIBBT xx B Birx |B
Ba* Ba*
- Ba" Baa
= =B Ba -BBT Ba*
=(BBT BB BBo
Then
LBa-Ba"I 1
T -BB Ba
< <(sin(BB. Ba
Then have E
(aa.BTa) 2) 5)
E ((at,BTz) -B s {
Putting these together gives
(aprio _Bxx) x)) |
E
(la"" (la (B,) =y)
+ +h(n))
19

---

For the generic result with subspace error, we substitute h(n1)= = or Theorem 5.2 to instantiate the bound with the method of moments estimator (28). Substituting for g (22) and taking a union bound over failure probabilities gives the result.
D.2 Proof of Theorem 5.5
Here, we prove our result lower bounding the lowest possible transfer error achieved by any (e algorithm in our single-task transfer setting. We denote the class of two-stage algorithms of interest as M2sag (c, Each algorithm in our class satisfies the following:
1. The algorithm takes as input private dataset S of m2 i.i.d. samples from task 0++ I, along with accurate public estimate B,
2. Projects the input data point ø TT, for any (x.y) in the private dataset S.
3. Algorithm outputs an estimate in the column space B.
Similarly, we can define class of misspecified linear regression problem instances. We use Patgg (d, to denote the following class of problem instances for the private task 1++1:
1. The input product distribution over (z:.y) is given by the model in and additionally the noise ~)~N((
2. Let the true regression vector be Bat+1. A subspace B is known such that: sind Sy
Also, 0++1 TR
3. The i.i.d. sampled dataset S from the above model satisifies: ||0||2 for every x: S.
In the above, both B and B are d k matrices with orthonormal columns, i.e., B, B.B.Gr (R), where, (B)) is the Grassmann manifold [66] and consists of the set k-dimensional subspaces within an (d) we omit the dependence on B, which is fixed. We note here that our proof works for any fixed B.
underlying d-dimensional space. Also, for both M2stg
Now, we are ready to restate our Theorem 5.5
Theorem 5.5 (Two-stage single-task private transfer lower bound). Let M be an (E,6)-DP private algorithm where E (0,1), 8 <I/,"*, w O, that takes as imput: (i) publicly estimated subspace Brub from an oracle that only uses public samples; and (ūi) a dataset of private samples. For any such M, there exists private problem instance given by the pair (B,01+1) where B E aI (R), 0141 E RR sin O B, Byub) %, and |Ba44 S 1, such that for S sampled i...d. from this instance using the model in (2). we have:
EM ESI.,O R(niiBB.o (S,BBubb) 2x
(9)
1-2
00
(0" (02+g2)+
(10)
n3e22
122
Proof. Given the estimate B, the goal is to lower bound the following minimax risk:
inf MEMana(edr)) B.0c+1 E (d.ke))
sup
EMEsS Bie.)) (y- 1 (S
(20)
Let us begin by defining the class of regression vectors constiuting the set of all possible Boo+ that can be realized by problem instance in (d,k,)) Given some rank k subspace defined by the matrix Ó E (IR), we define the following set of d-dimensional l2 norm bounded vectors that are 7 < close to given B:
0(Br)) =: (0ERR :0 -= Bot+1 for (Pnn(())))
(21)
From the definition of the principal angles and P2stg (d, it follows that for any 0 (B.(B)))
[B0l22 VI-2² + 11(La- BB )0122
20

---

We can break the above set (B,)) into disjoint sets: @(B,)) = Dely 1= 1 where O defined as:
O₂(B) = == ||00|2==]]
(22)
The above subclass of regression vectors results in convenient subclass of problem instances class (pstg(p). Just as we did for Paseg (d. we can define the following minimax risk for P2stg (p).
inf M E Mana (+,6,)
sup
Es B(x,)) (y- (VM((S x)
(23)
Based on the above definitions we get:
inf
sup E (d.k.))
(s (y-e:iity---MM-M- (V-
(24)
M (MMa(((td)))
inf
sup pElv 1- 1
sup
Es B,) BB -V-M(s) M(S)T²)) x)²
(25)
M (2nggg(t..r)))
inf pElv felvi-93,1 (MM
sup
sup Esp==Bact B(ry))l=Barr - (8) (26)
(E),(
inf M (eMang(e.rr) PEl IlVV-~3,11
sup
sup (E),
(y (()) x)²
(27)
where the final inequality uses inf sup 2 supinf
We can do this because inf and sup are defined non-empty sets and the loss function remains bounded over the product space M2stg( (c,ō,n) x O,(B) (B). The loss function is bounded because the norm of the regression vector and the input covariates is bounded. Further, the linearly independent noise in y has finite variance.
67
For the next part of the proof, we focus on lower bounding the minimax risk (23) when the adversary searching over the set (B). The lower bound for the minimax risk over this subclass is given by two parts: (i) statistical error rate that is suffered by any non non-private algorithm for which we lower bound hypothesis testing lower bounds; and (ii) the risk suffered by any (e,5)-DP private estimator which we lower bound by constructing a tracing attack. We will begin the proof for the latter part and then plug in standard statistical risk lower bounds.
The following Lemma D.2 (proven later) states lower bound over the class (p)
Lemma D.2 (Lower bound for Paag (p)). For any fixed B and any (E,8)-DP private algorithm M
(where < < 1, 8 < 1/n-11 1 for some w > 0) that belongs to class M2sag (e,8,7), there exists a problem instance for the transfer lask in the class Pasg (p) such thut for the B, B,04+1 given by the problem instance:
E Esj Ba (M. -0(0000(
122
k
11 12)+1- 1o=+1-1))+1-ppp
100
(28)
n332
12
We can now come back to (27 and compute the supremum over p after plugging in the lower bound in Lemma D.2 Since p> /1 1- plugging in this value for in Lemma D.2 and from the minimax risk
lower bound on Pastg (d,k, r,) in (27,. we obtain the result in Theorem 5.3
D.2.1 Proof of Lemma
Proof. The proof for the subclass lower bound relies upon re-parameterizing the problem instance as k-dimensional linear regression in the problem, but now in the column span of B.
Let the worst case in instance in (p) be 0 Bat+1, where || 0 p. We shall derive low-dimensional linear regression problem posed by the the unknown worst case instance 0, and the projected inputs: z B" x. Recall that the joint data distribution for private samples is given by:
I NOO.Ta) I2),
(29)
y ~ N(x T0,02)
(30)
21

---

Additionally, we also recall that the learning algorithm is given n2 private i.i.d samples from the above distribution S = (=,,:)))=_ In addition, is also given a rank matrix with orthonormal columns: B E ( that close to the unknown low rank subspace B, i.e., sind (B,B) S = I1( - BB ) 2 S y Next, we write each sample in S in terms of the projection of the regression vector Be to the column span of B, when the input is also projected in the following way: x Bz:
~ (0,0²²
x Ba+2;
=xT((B'' =LBB""Ba+tl B BBa 1Ba)+2) +
=x Ba)+x 3 (Ia-BB BB )
=(=" (la-BB") Bax+z
(31)
where a: BT Ba is dideenssionall vector in the column span of the given subspace B.
For any output (S, BB) for an algorithm in (e,67)) from the independence of the two projec-
tions: BBT and (Ia -BBBB BB
argued in Lemma C.1
ESIBaa B(,,)) a (y M(S
= =Es B.a B. F,, BB ++x ( x)²
=0² = - BB 0012 +E
B 0- x)
(32)
Since, the norm of in the nullspace of B can be chosen without affecting the hardness of the above rejection problem, the worst case problem instance will maximize the additive error by picking any component along the null space (note that direction along null space does not impact the regression error) that has the maximum norm of 1 1- (recall that for any 0 E Patg (p), ||0||2 2 S 1). Now, from (3T) follows that the i.i.d. samples in S for the worst case instance are drawn from the following low-dimensional linear regression model:
x ~ N(OTTa) BB = T
8| (( where, 0² =0=
(33)
From the above model in (33 and equivalence in (3) we have the following equality for the minimax risk over (p).
inf M E (ane(t
sup B,aEPaac(e)
B (y ((, x))²
inf
sup A= B Ba, BoePaallo)
EsjaErja (ā b -M, +o
MeMuu((d8))
inf
sup 8=B Bo, BaEPaal())
Esja Esalla-M(Ss,)33-32211-
(34)
M Mana((())))
where S= ((pf) ,1)) and y = xf +4, where N(0.0
Our main technique for proving lower bounds for the minimax risk (34) is based on the tracing adversary technique proposed by [59]. Next, we prove that there exists prior the effective regression vector ó, and a tracing attack that is successful recovering an element of the i.i.d. sampled
dataset S, in expectation over the prior and the dataset. Consider the following tracing attack:
Aa((aB vy) (v- (x
(M (M(
(35)
Similar to the tracing attacks for mean estimation problems [88 Lemma proves that the attack A (S)) takes large value when (I .y) belongs to S and small value otherwise. We
22

---

compare the attack success with estimation error, and show that whenever the estimation error is small, the attack has be fairly successful. Since, we are only searching over private algorithms where the attack takes small values, this yields lower bound on the estimation error.
The minimax lower bound stated in Lemma D.2 i.e., the lower bound for the minimax risk subclass Pastg (p) (for fixed p), is given by the summation over two terms: the statistical lower bound and second term implied by the tracing attack sucess lower bound stated in Lemma D.3 (proven later).
Lemma D.3. For any fixed
o. V1->² s S 1, (B, satisfying sin ((B. S % and
|| = =I Balla=p$ let (r" .y) be an i...d. sample (and S a dataset ny i..d samples) drawn from the distribution defined in (33). Then, for every (z.5) differreennl private estimator M that takes as input B and satisfies (SBBaaaaMMM - - for every &, the following are true:
1. For each i E [m], let S denote the data set obtained by replacing (x ,3/) in S with an independent copy from the distribution in 31 then IEAA ((=r .9.), M))) =0 and
E [AAa(( (S)
(I|M (B -
2. There exists prior distribution of supported
o e R such that à p. and
Baan 2 p0²+1-
From Lemma C.2 and from the first part of Lemma D.3
Σ
Esja As (( (S, S 2ngevo²+1-p" -
+ +2n28T +122 /
For the tail probability term,
P P(IAa((ef ((IAa((eF..) (())((/).)) =P
11: al
Σ (((
j=1
SP [Iu - ||ا|||==
-12
SP [ - al / 220X S2XXp
2k(o2+1-pp)
By choosing //oosingg7///2/2271133660///// we obtain
Σ
(Bgg x 2nz 02 p2 3tt. 2013
sE[n]
Now plugging the second part of Lemma D.3 D.3givvee giieeseu us
(o 1- (0'+1-p²) <
Ex
Easx
Esja Aa ((x) W. M..
VE(N)
2n 02+1
(ooloo/)))))))
Since 8< xn for w 0, for every (s, (d.f)-diiferrentially private M we have
A
Es all ( ž (o (o
(36)
nsg2
Adding the statistical lower bound of M(a"+1-0" to the lower bound from (36), and from (34). we complete the proof Lemma
u2
23

---

D.2.2 Proof of Lemma D.3
Proof. Let us begin by looking at EAa (4. independent xf and Ely, (z,") =0:
(.)),(())B)),
where we the fact that yi (x is
((r 3i ((,,
E
(yl (T, à)
R
(vs (T= ())
BIM - 1[ -
/=1
R-1
0.
B[M(S,
This proves the first claim about the expected value of the attack when the datapoint is not part of the training set, i.e., ((x (S,)) = Next, we look the expected magnitude of the same random variable and upper bound it with term that scales with the estimation error.
E El ( M(,, V ((Aa M(S)))
(Jensen's 's inequality)
E
(y ( (x- a) Σ
(M B)
EE
/EK
(independence)
[(V
(since BTB B= Tx.)
E M +
- (independence)
where the last inequality uses the following derivation:
E ((v. - (2 [a))) = -0 -1( BB BBB)BBBBB )all
=02+1-p²
This completes the proof for the first part of the Lemma. For the second part will begin by constructing a convenient prior for å.
Note that â can take any value in the column span of B the adversary has complete control over B and a. Thus, defining a prior over á would involve defining prior over the column span of B such that ||o||2 = We define sample from the prior as a multi step procedure:
1. For E [k-
11, sample w; from the truncated Gaussian, with mean 0, variance n² 0²/(x-1), and truncation at points -P//R-1 and P/Vk-
2. Set we = Eiejk
uu 1
with equal probability for either sign.
24

---

3. Now, set Ba - where is the ith column of B. Consequently, á=B Ba= (i "W)2).,)u???
For the second part of the claim need to lower bound Eiew) Ba~a 13 A A,((Hnx)), which we can decompose over co-ordinates the following way:
Σ :E(u]
E
An (a (.
Σ
E
E
M(S.B),
Σ
(ys â
(af)at la
jE(R-11
iE(m)
2
Σ
B
E
M(S.B);
[log((S] [,
0ā,
jE(K-1 1)
E B E M(S,B), i Jā,
= =002+1-p²)
[log((S) |
jE(n-1
E E[M(S,)),
=
E &ewn
(37)
Dix
JE[R-1] 1]
where the final equation uses the log-derivative trick.
Next, we focus on EanT a B[M(S,B),] for any j E [k- - 1]. Recall, that for any dimen- aa
sion& j [K 1], the prior draws sample from the Gaussian N/O, -²/x-)), truncated at P/ k I, P\ -- independently. We will now apply Stein's Lemma (see Lemma C,3 for the term Ea-t
E [))
86y
Denoting j-j as the ( 1/5 \ { and n as the marginal prior over jth dimension of &j, can lower bound Ea-T a 06,
[S)] in the following way:
i E|M(S,B),
Ba-T
= F.a
Ea Dā
Es (((MB)) [ي-)
0āj
E&
Es Es(M(S,
(āg))
Ea
Es Es(M(S, (S +
- Ea [ -a;1] (38)
2 Ea
(a,)
Next, we use the density of the truncated Normal:
exp (k-1) 11 -3
20F
TT(a,)=
22Te/ (E-1/ (--)))
where ((-) is the CDF function for a standard Normal distribution. Thus, [(,]
(k-1) ag.
25

---

Substituting the above and applying Cauchy-Schwarzfollwww followed by Jensen't 's inequality we get,
E&
- ..r Eis
layl- Byy-Es(|M(S,BB), - الرة
I (K-11//2-
E&
B&
Es
((S.
JEL IK
s (k-1)//2.
E3 | 122
E&Es (M, -āll
(39)
From directly applying the density of the truncated Normal distribution get,
E
-1-11/2-BB
Σ à
(40)
jE(k-11
Plugging 40).
39 into 38 and using 37
we get,
Σ Bä-z E
iE(n)
(0² 2 +1 -1-²)
1-1
a3
Ra -al Ba- |
(41)
11/(1-1)
Note that Ei-x Σ 2j=1 03 p² by construction of the prior π and Ea -dl o(1) by assumption. Thus, Eng(n) Bnr B, B. B. Aa( (B, 2 (o=+1-13)-(k-1) which completes the proof of the second claim in Lemma D.3
k
26

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
The paper claims to provide empirical evidence and a theoretical model for the benefits of public representations in improving private training in out-of-distribution settings.
Justification:
These claims are clearly outlined in the abstract and introduction and reflect the results the paper.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or NA answer this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation as long as it is clear that these goals not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: See Section offor discussion of limitations.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set assumptions and complete (and correct) proof?
27

---

Answer: [Yes]
Justification: The assumptions for the theorems are provided in Section 5.1 and the proofs are provided in the Appendix.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proof' can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced
4. Experimental Result Reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
Justification:
The experimental setup is provided in Section 4.2
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access a hosted model (e.g.. in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm.
(b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. case of closed-source models, it may be that access to the model is limited in some way (e-g., .. to registered users), but it should possible for other researchers have some path to reproducing verifying the results.
the
5. Open access to data and code
28

---

Question: Does the paper provide open access to the data and code, with sufficient instructions faithfully reproduce the main experimental results, as described in supplemental material?
submission is published. All the packages and models we use are Answer: [No]
We do not include our experimental code with the submission at this time but plan to release
Justification:
open-source.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips... cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https
/ httpppscccpbllccccgcccccee
/ooeeee///ooedemiic ssi onPolicy) for more details.
The authors should provide instructions data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: Details of the training and test setup are provided in Section 4.2
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.
The full details can provided either with the code, in appendix, or as supplemental material.
7. Experiment Statistical Significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [No]
Justification: Because our experiments involved full training or fine-tuning of models, it was too computationally expensive to generate error bars over multiple runs.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
29

---

The method for calculating the error bars should be explained (closed form formula, call library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors). should be clear whether the error bar is the standard deviation or the standard error of the mean.
It is OK to report 1-sigma enror bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).
If error bars reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments Compute Resources
Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Answer:
[Yes]
Justification: Details are provided in Section 4.2
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (c.g., preliminary failed experiments that didn 't make it into the paper).
9. Code Of Ethics
Does the research conducted in the paper conform, in every respect, with the NeurlPS Code athiicshttpss:/eerripss http///rrw.p.pe/oaaaa//ss/ssaaar http:///aaaatiiiiaee//////////aa
Question:
Answer: [Yes]
Justification:
The pretrained models and datasets we use are currently publicly available and human subjects have been involved our experiments. Nevertheless, we acknowledge the blurry line between private and public data on the open web, and the potential for harmful content reside in publicly available datasets.
Guidelines:
The answer NA means that the authors have not reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.
The authors should make sure preserve anonymity (e.g., if there is special considera- tion due to laws or regulations their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: Our work contributes to the theory of differentially private learning, which is essential to preserving user privacy under increasing amounts of data collection. Our work has the potential benefits of making private learning more practical by incorporating public data. However, we echo cautions pointed out in recent work [14] that care must be taken to ensure that publicly available data does not contain sensitive information for real deployments, and applying our techniques without such measures could inadvertently leak sensitive information.
30

---

Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no societal impact or why the paper does address societal impact.
Examples negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deploymeet of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g.. pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: The method we evaluate is itself a form of safeguard (differentially private training). Above that, the methods do not suggest obvious new safeguards necessary.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do require this, but encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: We use the OpenCLIP [52) library for models and the Opacus [54] library for private training. We provide citations to these as well as the datasets we use in the paper.
Guidelines:
The answer NA means that the paper does
existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version the asset used and, possible, include URL.
31

---

The name of the license (e.g., CC-BY 4.0) should be included for each asset. For scraped data from particular source (c.g.. website), the copyright and terms of service that source should be provided.
If assets are released, the license, copyright information, and terms of use in the pack- age should be provided. For popular datasets, pawperewts.pp.wwwwii... thcode. com datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license derived asset (if it has changed) should be provided.
If this information is not available online, the authors are encouraged reach the asset's s creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification:
We have not (yet) released any new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/mooell as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Question:
Answer: [NA]
Justification:
We do not perform any human subjects experiments.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contri- bution of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurlPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Question:
Answer: [NA]
Justification:
We do not perform any human subjects experiments.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
32

---

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
33


Reviews:
[{"review_id": "yhQnOgUyRi", "reviewer": "Reviewer_S2j2", "summary": "This paper studies the role of using public training data for fine-tuning private tasks. The paper begins by showing, empirically, on three datasets, that fine-tuning significantly outperforms training privately from scratch. The experiments are conducted on several image classification datasets. Then, the paper provides a stylized theoretical model to explain the findings, based on models of nonprivate transfer learning. This model focuses on linear regression with isotropic noise corruption. The results are complemented by simulations matching the linear regression setup.", "questions": "See Weaknesses above.", "limitations": "The authors discussed the limitations of their work in Section 6.", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "- The paper provides both theoretical and empirical evidence to show that public training data are useful for private fine-tuning, even when the two sources come from different data distributions.\n\n- In the theoretical model, both upper and lower bounds are provided to justify the optimality of the rates.", "weaknesses": "- The theoretical setup may be too simplistic. For instance, the resulting algorithm 1 is different from what was implemented in the experiments.\n\n- The experiments only focus on image classification. Additional data modality could be utilized to broaden the interest of these results.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their time and feedback on our work.\n\n> **The theoretical setup may be too simplistic. For instance, the resulting algorithm 1 is different from what was implemented in the experiments.**\n\nWe understand the reviewers' concerns regarding the stylized theoretical model. We aimed to analyze the simplest model that would capture the main characteristics of public pretraining as well as the key property of concept shift. As we note in the common response, this model is commonly used in the non-private meta-learning literature to model similar experiments in the non-private setting. Nevertheless, we agree that building off our work to explore a more complex model capturing the performance of public-to-private transfer would be an interesting direction of future work. \n\n> **The experiments only focus on image classification. Additional data modality could be utilized to broaden the interest of these results.**\n\nThanks for this suggestion. We agree that experiments in another modality, such as language, could be interesting future work. Works such as [f, g] have shown that private training from scratch is ineffective for language models, while privately finetuning a public model can reach state-of-the-art (private) performance on standard NLP benchmarks. However, these works have not strictly explored the distribution shift setting. We expect that the benefits of a shared representation would extend across different language domains (due to the shared low-level features of language domains), but verifying this experimentally would be valuable. We will include a discussion of this in our revision.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "gStG615Jc8", "reviewer": "Reviewer_t1VG", "summary": "The paper proposes a theory for the benefit of transfer learning in DP ML even when public data and private data differ significantly from each other. This theory relies on a subspace shared between public and private data. They empirically verify the algorithm in their theory.", "questions": "Can you verify that there is limited overlap between the benchmark datasets evaluated and the DataComp dataset? I think even some sort of textual overlap is sufficient if not exhaustive.", "limitations": "There is a fair discussion of limitations.", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The paper is very clearly written and structured. It demonstrates a phenomenon, develops a theory to explain it, and verifies their theoretical results experimentally.\n\nThe paper makes a connection to theoretical work from metalearning.\n\nThe lower bound in Theorem 5.5 is interesting.", "weaknesses": "The advantage over prior work is mild for linear regression tasks. In linear regression, gradients, input data, or models being constrained to a linear subspace are all roughly the same thing.\n\nThe paper does not attempt to reconnect the theory back to the original experiments. This explanation does not seem refutable.\n\nThe paper does not quantitatively argue that there is limited overlap between the datasets considered and the base pretraining data.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for taking the time to read and provide feedback on our work.\n\n> **The advantage over prior work is mild for linear regression tasks. In linear regression, gradients, input data, or models being constrained to a linear subspace are all roughly the same thing.**\n\nWe acknowledge that these three settings (gradient, input, or model/task subspaces) lead to similar analyses in the linear regression setting. That said, we focus on the shared subspace of the linear models in public and private data distributions because it cleanly models concept shift and shared representation. Such a model also has not been explicitly stated or studied in prior work on public-to-private transfer.\n\n> **The paper does not attempt to reconnect the theory back to the original experiments. This explanation does not seem refutable.**\n\nWe understand the reviewer's concern regarding the disconnect between theory and practice. As we mention in the common response, our theory provides one simple model in which public data can indeed help under significant distribution shift (and one which has been validated to be useful empirically, in the nonprivate setting). That said, there is significant current research interest in understanding whether this \"shared representation\" model is accurate in practice.\n\n> **The paper does not quantitatively argue that there is limited overlap between the datasets considered and the base pretraining data [...] Can you verify that there is limited overlap between the benchmark datasets evaluated and the DataComp dataset? I think even some sort of textual overlap is sufficient if not exhaustive.**\n\nIn our paper, we point to the low zero-shot performance of the base CLIP models on each dataset as a natural proxy for the absence of sufficient training data in the CLIP dataset. (In contrast, for example, CLIP has high zero-shot performance on ImageNet, which is contained in the training data, as well as on ImageNet variants [c].)\n\nFor RESISC45 and fMoW, which are remote-sensing datasets, we also refer the reviewer to [b], which states: \"We obtain a remote sensing subset of [LAION-2B] by applying a binary classification model to determine whether an image in LAION-2B is a remote sensing image (see details in Appendix A.5). This subset, denoted as LAION-RS, contains 726K remote sensing image-text pairs—only **0.03% of all samples.** This shows that web crawling cannot efficiently collect remote sensing image-text pairs at scale.\"\n\nWe would be happy to replicate this analysis for PCam in an updated version of the paper (for Datacomp rather than LAION as LAION data is no longer publicly available). Unfortunately, due to the size and download time of the Datacomp dataset we were not able to perform this analysis during the rebuttal period.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "qhZdE3NfGm", "reviewer": "Reviewer_PptC", "summary": "There are many contexts where deep learning models trained to preserve differential privacy have much worse performance than models trained without the differential privacy constraint. One common way to improve model quality in these cases is to pre-train a model on publicly available data then fine-tune with the private data. These pre-trained models often demonstrate much better empirical performance than models trained exclusively on private data, even when the public data used for pre-training is very different from the private data. This phenomenon is not well understood theoretically so it is unclear what to expect from out-of-distribution pre-training.\n\nThis work seeks to understand public pre-training under distribution shift. The authors start by demonstrating the utility of public pre-training in the case of extreme distribution shift. Having demonstrated the phenomenon, the authors develop a theoretical model to explain the effectiveness of public pre-training. This model poses public data as coming from a sequence of linear regression tasks where the features of each task lie in a low-dimensional subspace. Working in this model, the authors develop a two stage algorithm for public-private linear regression that first approximates the subspace $B$ then uses DP-SGD to estimate the regression parameters $\\alpha$. The authors analyze this algorithm to prove an upper bound on the error and they provide a matching lower bound among algorithms that solve the regression problem on a subspace. To support the theoretical results, the authors provide empirical results on simulated data.", "questions": "- The authors discuss public data assisted query answering in the background section, are you aware of [this](https://proceedings.mlr.press/v238/fuentes24a.html) more recent work in the area? \n- An alternative to linear probing for parameter efficient fine tuning is [LoRA](https://arxiv.org/pdf/2106.09685), would you expect this method to perform similarly to linear probing in your transfer learning experiments?\n- How does the bound for your two stage algorithm compare to the error you would attain if you simply ignored the public data? It would be useful to say exactly what you gain from the subspace estimation step and if the naive approach would ever be better (like in a situation where the embedding dimension is almost as large as the latent dimension).", "limitations": "- The authors have adequately addressed the main limitation of the work (the fact that the theoretical model is limited and does not totally account for the neural representations used in practice). They accurately describe their model as being stylized and acknowledge the impacts of this limitation in the discussion section.", "rating": 7, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- Analyzing the impact of public data can be difficult without making strong assumptions about the public data distribution, but it is frequently observed that pre-training/transfer learning with out of distribution data is beneficial. Bringing this distribution shift model over from the meta-learning literature seems like a promising approach for understanding this phenomenon.\n-  The experiments on the vision datasets do a good job of demonstrating an clear example of OOD transfer learning being effective. \n- The theoretical model is introduced and motivated well, the results are clearly stated and, while full proofs are in the appendix, the authors describe the proof strategy in the main text to give helpful intuition to the readers.\n- The simulated result are good to see because they highlight the two sources of error (subspace estimation and DP-SGD error)", "weaknesses": "- The disconnect between the setting for the experiments and the stylized theoretical model is very apparent. It is difficult to say how much of the empirical results from section 4 can be explained by the theoretical results in section 5. That being said, the authors provide citations showing that this theoretical model is common in the meta-learning literature. I am not familiar enough with that literature to know if there are other alternate models that are common in that literature or how they would compare to the model used in this work. It would improve the paper if the authors compared to other potential models, if they exist, or stated that they do not.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their thoughtful feedback and comments on our work.\n\n> **The disconnect between the setting for the experiments and the stylized theoretical model is very apparent. It is difficult to say how much of the empirical results from section 4 can be explained by the theoretical results in section 5. [...] It would improve the paper if the authors compared to other potential models, if they exist, or stated that they do not.**\n\nIndeed, some other models and theoretical analysis techniques do exist in the non-private meta-learning literature. For example, [d] analyzes the gradient dynamics of model-agnostic meta-learning (MAML), and [e] proposes and analyzes a hierarchical clustering approach that uses multiple different MAML initializations to adapt well to new tasks. \n\nNevertheless, as evidenced by the works cited in our paper (lines 97-101, 226-227), due to the tractable and simple nature of the linear subspace model, it has attracted more attention than other analyses even in the nonprivate meta-learning literature. As we mention in the common response, this model is particularly well-suited for understanding transfer learning under concept shift. We will include a more thorough discussion of other theoretical models from the non-private setting in our revision.\n\n> **The authors discuss public data assisted query answering in the background section, are you aware of this more recent work in the area?**\n\nThank you for this reference; we were not aware of this work and will add it to our related work section.\n\n> **An alternative to linear probing for parameter efficient fine tuning is LoRA, would you expect this method to perform similarly to linear probing in your transfer learning experiments?**\n\nThis is an interesting direction for future exploration. In our results, linear probing already achieves performance much higher than standard fine-tuning. LoRA would still involve training more parameters than linear probing, though fewer than full fine-tuning. In our experiments, we observe that full private fine-tuning performs worse than private linear probing, and one explanation for this is the larger number of parameters involved in full fine-tuning. Thus we would expect LoRA to be another point on the tradeoff between the degradation from increased dimension vs. the improvement from accuracy.\n\n> **How does the bound for your two stage algorithm compare to the error you would attain if you simply ignored the public data? It would be useful to say exactly what you gain from the subspace estimation step and if the naive approach would ever be better (like in a situation where the embedding dimension is almost as large as the latent dimension).**\n\nWe point to the bound of Tripuraneni et al. (stated as Theorem 5.2 in our paper) as a starting point for this discussion. The error $\\gamma$ of the subspace estimation algorithm given in that paper grows as $O(\\sqrt{dk^2/n_1})$. As the reviewer suggested, this indicates that when the latent dimension is nearly as large as the embedding dimension, the subspace estimation error will outweigh the benefits of the dimension reduction. We would be happy to add a discussion about this in an updated version.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "f8iobvvesP", "reviewer": "Reviewer_iSuQ", "summary": "The work aims to bridge an important gap in literature, looking at using public pretraining to improve differentially private model training, even when there is a distribution shift, particularly concept shift, in the public and private data. They report impressive improvements in accuracy. Further, they provide a justification for these observations, citing the representation sharing nature of the private and public data.", "questions": "- [Q1] Referencing W1 it might be interesting to look at the naturally learnt representations, when the public data and private data is used independently to train different models. It would help verify the assumption that a common representation space is even learnable. What kind of representations are learnt? What is the difference between the representations learnt in a disjoint way and those learnt by forcing a common latent space?", "limitations": "All the limitations, if any, have been adequately discussed and addressed.", "rating": 7, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 4, "strengths": "- [S1] Highlighting the limitations of current public pre-training techniques which have the same distribution as the private data, which as the authors rightly point out, is often not the case. There is a reason why the data is private and one would suspect, more often than not, there would be differences in the nature of the public and private data.\n\n- [S2] Looks at the most general type of distribution shift, namely concept shift over covariate shift or label shift.\n\n- [S3] Highlight on cases where models when trained from scratch privately collapse, and where there is an actual need for additional ways to supply information. Line 155-156 clearly distinguishes between situations where zero-shot private performance translates quite well, and thus the task itself is of lesser interest to study from a research perspective as the headroom is not as significant.\n\n- [S4] I personally like the writing style of the paper quite a bit, it establishes clearly the gaps in the current research and even the setups people use, and goes on to very thoroughly present an alternative direction that should attract more work.\n\n- [S5] I further like the presentation of theorems that is supplemented with Proof Sketches that help grasp the intuition of the proof.", "weaknesses": "- [W1] Strong assumption that the shared, low-dimensional representation is even learnable across the private and public data.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their feedback and for the kind comments regarding the strengths of the work.\n\n> **[W1] Strong assumption that the shared, low-dimensional representation is even learnable across the private and public data.**\n\n> **[Q1] Referencing W1 it might be interesting to look at the naturally learnt representations, when the public data and private data is used independently to train different models. It would help verify the assumption that a common representation space is even learnable. What kind of representations are learnt? What is the difference between the representations learnt in a disjoint way and those learnt by forcing a common latent space?**\n\nThe reviewer raises an interesting point regarding how realistic the shared low-rank subspace assumption is in practice, for deep models and real data. Answering these questions in the most rigorous way may be an interesting paper in itself; this would require defining a metric to measure similarity between trained models (or containment in a subspace) to distinguish the pretrained representation from the trained-from-scratch representation. Another subtlety is that the private trained-from-scratch representations in our setting are simply unusable, and are likely not representative of the \"true\" representation of the data—this raises a question of whether we should compare to a private or non-privately trained representation in order to measure containment in a subspace.\n\nThat said, as a preliminary exploration, we plotted the eigenspectrum of the feature covariance matrix computed after extracting features of PCam images from the ViT-B-32 pretrained model. *This result is attached in the PDF in the global rebuttal.*\n\nFrom these results, we see that the pretrained features are approximately low-rank for the out-of-distribution task PCam, yet a linear probe over these features achieves good (83.5%) performance (Table 1). The fact that the representation still gives good performance when only a linear layer is trained on top suggests that the data does fundamentally lie in or near the low-rank space that is identified by the pretrained model. We would be happy to also provide this analysis for RESISC45 and fMoW in our revision.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
## Common response to all reviewers

We thank all of the reviewers for taking the time to read and give helpful feedback on our paper. 

As noted by the reviewers, our work provides both theoretical and empirical evidence to show that public pretraining is useful for private finetuning under distribution shift, and we appreciate the reviewers' acknowledgement of the significance of this work in the context of practical private machine learning in the realistic setting of concept shift.

In addition to providing individual responses to each reviewer, we would like to address here a common concern across reviewers regarding the relevance of the stylized theoretical model and the potential disconnect between the theoretical model and experimental setup.

We first note that the model we present is sufficient to answer the main research question we set out to answer: **Can public data help private learning under significant distribution shift?** Our model illustrates a clean, simple setting in which public data does in fact help, even when the private task can differ significantly. In this sense, we see the simplicity of the model as an advantage in that it highlights what we believe is a key benefit of using public data: learning the shared representation -- and answers our research question in the affirmative.

However, as we have touched on in our paper (lines 97-101, 226-227) we also note that analysis even in the non-private meta-learning setting is largely restricted to this linear subspace model. Out of the many examples we cite in our paper, we point to [a] as one example of work in the non-private setting studying a phenomenon similar to the one we study where the model is also restricted to a similar, linear subspace assumption to make the theoretical analysis tractable. Nevertheless, this work also finds that the simplified model is a good predictor for real experiments on fine-tuning tasks with deep networks. This again gives us confidence that the model is valuable to study even while being simple.

Finally, we point out that we in fact observe the biggest empirical improvement in the linear probing setting. As such, the linear regression model is particularly relevant because we learn a lower-dimensional linear model on top of the pretrained features, although those features may or may not be accurately represented by a low-rank linear subspace.



## Citations referenced in rebuttal (common to all responses)

[a] Kumar, A., Raghunathan, A., Jones, R., Ma, T., & Liang, P. (2022). Fine-tuning can distort pretrained features and underperform out-of-distribution. arXiv preprint arXiv:2202.10054.

[b] Wang, Z., Prabha, R., Huang, T., Wu, J., & Rajagopal, R. (2024, March). Skyscript: A large and semantically diverse vision-language dataset for remote sensing. In Proceedings of the AAAI Conference on Artificial Intelligence (Vol. 38, No. 6, pp. 5805-5813).

[c] Wortsman, M., Ilharco, G., Kim, J. W., Li, M., Kornblith, S., Roelofs, R., ... & Schmidt, L. (2022). Robust fine-tuning of zero-shot models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (pp. 7959-7971).

[d] Nichol, A., Achiam, J., & Schulman, J. (2018). On first-order meta-learning algorithms. arXiv preprint arXiv:1803.02999.

[e] Yao, H., Wei, Y., Huang, J., & Li, Z. (2019, May). Hierarchically structured meta-learning. In International conference on machine learning (pp. 7045-7054). PMLR.

[f] Yu, D., Naik, S., Backurs, A., Gopi, S., Inan, H. A., Kamath, G., ... & Zhang, H. (2021). Differentially private fine-tuning of language models. arXiv preprint arXiv:2110.06500.

[g] Li, X., Tramer, F., Liang, P., & Hashimoto, T. (2021). Large language models can be strong differentially private learners. arXiv preprint arXiv:2110.05679.

Meta-review: The reviewers were overall positive about the paper. However, there were a lingering concern regarding the disconnect between the theory and the experiments. We recommend the authors to discuss this aspect in more detail in the paper, in order, to ensure a better readability of the paper. We do acknowledge the authors address this point a fair bit in the rebuttal. It would be great if some of the discussion from the rebuttal can be ported to the paper.
Decision: Accept (poster)