Adversarial Training Helps Transfer Learning via Better Representations
Authors: Zhun Deng
Linjun Zhang
Kailas Vodrahalli
Kenji Kawaguchi
James Zou
Venue: NeurIPS 2021 Poster (2021)
Keywords: Transfer learning
adversarial training
semi-supervised learning
statistical learning theory

Abstract: Transfer learning aims to leverage models pre-trained on source data to efficiently adapt to target setting, where only limited data are available for model fine-tuning. Recent works empirically demonstrate that adversarial training in the source data can improve the ability of models to transfer to new domains. However, why this happens is not known. In this paper, we provide a theoretical model to rigorously analyze how adversarial training helps transfer learning. We show that adversarial training in the source data generates provably better representations, so fine-tuning on top of this representation leads to a more accurate predictor of the target data.  We further demonstrate both theoretically and empirically that semi-supervised learning in the source data can also improve transfer learning by similarly improving the representation. Moreover, performing adversarial training on top of semi-supervised learning can further improve transferability, suggesting that the two approaches have complementary benefits on representations.  We support our theories with experiments on popular data sets and deep learning architectures. 

Adversarial Training Helps Transfer Learning via
Better Representations
Zhun Deng"
Linjun Zhang
Kailas Vodrahalli
Harvard University
Rutgers University Piscataway, NJ 08854
Stanford University Stanford, CA 94025 kailasv@stanf edu
Cambridge, MA 02138 zhundengōg hwr.ard. edu
linjun www.naauannnraaa.... edu
Kenji Kawaguchi
James Zou Stanford University Stanford, CA 94025
Harvard University
Cambridge, MA 02138
Stawaguchi@fas hwrvarrra.... edu
jamssestanf ord. edu
Abstract
Transfer learning aims to leverage models pre-trained on source data to efficiently adapt to target setting, where only limited data available for model fine-tuning. Recent works empirically demonstrate that adversarial training in the source data can improve the ability of models to transfer this happens is not known. In this paper, we provide theoretical model to rigorously analyze how adversarial training helps transfer learning. We show that adversarial training in the source data generates provably better representations, fine-tuning on top this representation leads more accurate predictor of the target data. We further demonstrate both theoretically and empirically that semi-supervised learning the source data can also improve transfer learning by similarly improving the representation. Moreover, performing adversarial training on top of semi-supers cupe-suped learning can further improve transferability, suggesting that the two approaches have complementary benefits on representations. We support our theories with experiments on popular data sets and deep learning architectures.
domains. However, why
1 Introduction
Transfer learning popular methodology obtain well-perforning machine learning models in settings where high-quality labeled data is scarce [20, 48]. The general idea of transfer learning to take pre-trained model from source domain-whert labeled data abundant-ann adapt new target domain. Because the target data distribution often differs from the source setting, standard transfer learning fine-tunes the model using small-amount of labeled data from the target domain. In many applications, the fine-tuning is performed only on the last few layers of the network the amount target data is limited or if the one only has access representation (i.e. intermediate layers) produced by the model instead of the full model.
Transfer learning has demonstrated substantial empirical success and there is exciting literature investigating different approaches to making transfer learning more effective [26, 27]. Recent experiments empirically demonstrated intriguing phenomenon that models that trained using adversarial-robust optimization on the source data transfer better to target data compared to non- adversarially trained models. We illustrate this phenomenon in Figure 1, which replicates the findings
"Equal contribution.
35th Conference on Neural Information Processing Systems (NeurIPS 2021).

---

in [46]. Here two models are trained on the full ImageNet and 10% of ImageNet using different levels of adversarial training-é is the le magnitude of the adversarial attack. Following [46], we fine-tuned the last layer of the models using data from CIFAR-I0 and plot the final accuracy the target CIFAR-10. Adversarial training (E > 0) significantly improves the transfer performance compared to model without adversarial training = 0). Additional experiments demonstrating this effect are provided in [46, 55], however is still an open question how adversarial training in source helps transfer learning.
As our first contribution, we initialize the study of how adversarial training helps fixed-feature transfer learning from theoretical perspective. Our analysis shows how that adversarial training the source learns better repre- sentation such that fine-tuning on this representation leads to better performance on the target. Interestingly, we show that the robust representation can help transfer learning even when the source performance declines due to adver- sarial training. To the best our knowledge, this is the first rigorous analysis of the effect of adversarial training on transfer learning.
06.73
Accuracc (56)
$0.2%
25.7h
Hodel Trainss with 10%4 imagghet
102% imagekee
Epsilon
As our second contribution, extend our analysis to show that semi-supervised learning using pseudo-labeling can similarly lead to better representations for transfer learning. We support our theory with empirical experi- ments. Moreover our experiments demonstrate for the first time that performing adversarial training on top of pseudo- labeling in the source can further boost transfer learning performance. This suggests that the two data augmenta- tion techniques of adversarial training and sseudo-labeling have complementary benefits on learned representations.
Figure 1: Transfer accuracy improves with adversarial training on source task. We plot target task (CIFAR-10) accuracy across different levels of fz-adversarial training on the source task (ImageNet). The value of corresponds to the size of the adversarial attack; i.e., E = in- dicates no adversarial training. The two curves correspond training the source model using all of ImageNet and a 10% subsample of ImageNet.
As a third technical contribution, we generalize the tech- niques in prior papers for analyzing transfer learning regressions to classification settings, where adversarial training and pseudo-labeling are more commonly used. Together, our results provide a useful and tractable framework to understand factors that improve transfer learning.
Related Work Adversarial robust optimization has been major focus in machine learning security [7, 16, 33, 23, 10, 39]. A serious of works has been proposed to increase the adversarial robustness both empirically [34, 37, 1] and theoretically [14, 30, 44, 32, 13, 29, 24, 18, 62]. Meanwhile, other works demonstrate how to quantify the trade-off between adversarial robustness and standard accuracy [61, 47, 11, 50, 17]. Recently [56, 46) empirically studied the transfer performance of adversarially robust networks, but it is still not clear yet why adversarial training leads to better transfer from theoretical perspective.
Transfer learning has been used in a variety of applications, ranging from medical imaging [43], natural language processing [25, 15), to object detection [31, 49]. On the theoretical side, some prior works [3. 5, 36, 58] studied the test accuracy of the target task in the multi-task learning setting. More recent work [53. 52, 21, 59] focused more on the representation learning and provide a theoretical framework to study linear representation in the regression setting. In this work, we provide a counterpart to theirs and studies the classification setting. Prior works in semi-supervised learning largely focus on improving the prediction accuracy with unlabeled data [66, 65, 6]. Works have also shown that superviid learning can improve adversarial robustness [11, 19]. Several works have identified that using unlabeled data can empirically improve transfer learning [64, 63, 38), but a rigorous theoretical understanding of why this happens is lacking.
2 Preliminaries and model setup
Notation. We use [m]] for {1, 21.,.... . for any m E N+ and for any set S, let [S] to denote the cardinality of S. For matrix M, we denote o% (M) as the k-th singular value of matrix M. We
2

---

use Omxi to denote the space of matrices of dimension X l whose columns are orthonormal and use SP-1 to denote the unit sphere of dimension p. For two real matrices E. E 0mxi. we denote the subspaces spanned by the column vectors of E and F by & and F correspondingly, The subspace distance between & and F is defined as sin ( F) [60], where ((E, diag(cos σr (ETF), F),... cos ox(E' ()). For a vector v, we use ||e||y to denote the l norm. Let X and Z denote "less than" and "greater than" up to a universal constant respectively, < b to denote b 2 C' a for sufficiently large universal constant C. Our use of (()),(()))))))) (()) follows the standard literature of computer science. With some abuse of notation, we also write = ((6) if a (b) and a = (b(6) for a, b E IR.
Data generating processes. We assume there are T source tasks. For each task L E [7]. we have corresponding training data set of size nhe i.e. S = {= y()), (2 ym: }}, where z4 (1) X _ RR" and yt E [-1,1] are i.i.d. drawn from a joint distribution p!.) 2,9- We further denote n = mi nt- In other words, n is the smallest size of source data sets. The goal of transfer learning is to learn from multiple source tasks in the hope of learning a common representation such that for a target task with distribution P!T+1), we only need few data points to learn extra structures beyond the common representation and the learned model still achieves good prediction performance.
With this spirit, assume that for / E [T + 1], ((x2 (1) Mi ))+
(4)
are i.i.d. drawn from P(() Z.g> such that
10) = 1/ + Di (8)
(4)
J44,
(1)
for i.i.d. noise v/ that is independent of y where μr Bo, E R", a E R and B E R is an orthonormal matrix representing the projection onto subspace, i.e. BJB B = I, Here, B is the common structure shared among all the source tasks and the target task, and o₂ 's are task-specific parameters. Although this model is simple, the analysis is already highly nontrivial, and it captures the essense of the problem in transfer learning. In fact, similar models haves been considered in [52, 21]. Specifically, we consider the case where r & p. It can be viewed in way that the data is generated by mapping low dimensional data signal to the high-dimension, which coincides with the fact that commonly used real image data sets lie in the lower dimensional manifolds. In addition, we assume
i.e. E[exp(Au" Ti )]] I exp(>2 p//2) for all E SP-1 and X R. Throughout this paper, we consider Pt = ((1) for all E [T].
(t) is of zero- meen and is pf susssuubuussiie the noise term 17,
(t)
Remark 1. (i). The sub-gaussian assumption is quite flexible since many commonly used data sets such as image sets are all bounded, which implies sub- wub--aussianity. (ii). Different from the regression settings considered in previous theoretical work on transfer learning[21, 52], we focus on classification settings, in which adversarial training is more commonly studied.
Loss functions. The loss functions considered in this paper take the following form: for each task 1 E [T +1].
(z, y, W2 W1) = (x),
(2)
where (z) is a two-layer linear neural network parametrized by W1 and w² i.e. (x)
wa (() WTT xx with W1 E Opxr, (t) E Rx and || (2) < 1. Here, we mainly consider the case W1 is well-specified, i.e. with the same dimension of B. Our argument can be further extended to
the case where r is unknown by first estimating and details are left to the appendix. We put norm
constraint on wz since otherwise the minimizer is always of norm infinity. The loss function in (2) along with its variants have been commonly used in the theoretical machine learning community [47, 19]. Although in its simple form, has been consistently useful shed light upon complex phenomena. Meanwhile, even under this natural setting, it is highly non- non-trivial to demonstrate the effect of adversarial training in transfer learning.
Roughly speaking, like most settings in transfer learning, W1 is assumed to be the common weights shared among the models for all source tasks
learn "good" common repre- sentation. For each individual task t, parameter w/ aims to perform task-specific linear clas- sification. We leave the detailed discussions about how to take advantage of combining all the source tasks and obtaining a "good" W2 to Section 3. Further, we denote the empirical loss for task 1. as i(S,, wo(" Wi) = Em (1) (Wiwh") (0) ( ..(")/ms The expected loss for task / is L(PS!), w/ Wi) = -B (mn)) ( xx).
3

---

Problem Setup In rixeeerepreeentattinn transfer learning, the first step is learn the common representation in the model architectures using data from source tasks. The representation (e.g. the penultimate layer of a neural network) is then fixed. Finally, the target data is used to train or fine-tune a small model on top of the representation. Following this popular practice, our model setting, we use the data of source tasks { to obtain an estimator Wi. Then, we use the data of target (T+1) of the task parameter. Our evaluation criteria is the task S1+1 obtain estimator 2 excess risk:
R(W,, W2 ) I(PPT) (T+1) Wi)
min ||
I(P(T+1) W2, W1)-
(3)
3 Adversarial Training Help Representation Learning
In this section, we demonstrate our results about how adversarial training can learn better represen- tation, and therefore leads to smaller excess risks. We first describe our algorithm, and demonstrate the near-optimality of our algorithm in representation learning by a minimax lower bound. We then demonstrate for the settings where data has varying noise-signal ratios or sparsity structures, how l or loo adversarial training can help improve the representation learning.
3.1 Representation learning algorithm
Recall that the loss function for each task is {( y; w2 Wi) = --yw,"" WT x, where W1 E Opxr is common structure in model architectures shared among all the source tasks and the target set. In the spirit of transfer learning, the goal is to jointly learn W, from source tasks and then use the data from the target task to learn its task-specife parameter w/ (T+1) Here, W, essentially aims to recover the common structure in the data generating processes (1) (or more rigorously, recover the column space of B), such that the obtained estimator W1 satisfies || sin @(W,, B)]l +
(t)
Note that in our two-layer linear neural network structure, optimizing w2 and Wi simultancously for a single task has the issue of non-identifiability the loss value will not change if we multiply an orthonormal matrix A E RTxr to W1 and A to w2. However, we still can jointly learn a good estimator Wi to recover B following a similar method in [52] via singular value decomposition (SVD). In particular, we first simultaneously optimize w/² and W1 for each individual task for [T]. which is equivalent to optimizing a single parameter 3r = W wi matrix, the norm of 3g is still upper bounded by 1). Then, we apply SVD to the matrix consisting of
(t) (since W1 is an orthonormal
the optimizers 3, 's to obtain W,. In the final step, we use S to learn wz (1+1)
Algorithm 1 Learning for Linear Representations
Input: {
Step 1: Optimize the loss function on each individual source task t E [T] and obtain
3, = arrminn 1A
(3r.
Step 2: W, +- top-r SVD of 10,, B2, ...
37].
Step 3: 2/ (T+1) + argmin |!
Σ Eert' -
(++))
(1 (1) (T+1)).
(T+))
( 11
|| mr+1
Return W1, 02
(T+1).
Next, we provide lemma about the representation learning in the two-layer linear neural network model under the assumption below. Combining this lemma with a minimax lower bound, we will show that adversarial training cannot have any gain in representation or transfer learning without extra special data structures, which motivates our subsequent theories. To facilitate the presentation, let us define M [a, ||| Il. a2/1|a2ll, , ar/ rrolllll
Assumption 1 (Task normalization and diversity). For all the tasks, ||a.]| for all t te [1] and (M M/T) 2(1rr).
4

---

Remark 2. Throughout the paper, consider the low-rank where r is smaller than I and Meanwhile, notice that | Ei=1 (M). this assumption implies the condition number 01 (iM)/a,(MM) O(), which roughly means {a:/ [a:/||a:Ilii= cover the directions of IR' evenly.
Loosely speaking, if we denote fT++1 tions, with high probability
Eist
(T+1) (T+1) /nT+1, under some regularity condi-
(T+1), Wi) || S1,W1E0prr R(W,, 22 L(PIT+t), 122
L(P W2, Wi)
min
sin (W,, B)I +
B frrt B pr+1lll
(4)
representation error
task-spocific emorr
The task-specific error is easy to deal with given Eq. (4), we mainly focus on providing lemma to characterize the representation error.
Lemma 1. Under Assumption if 01 max /T, log (og for some universal constant C1 > and 2r min , T, for all [T]. For W, obtained in Algorithm 1, with probability at least 1 - )
sin W(W, B)lle Sr
V
P RiT
log nf
Application of Lemma gives us the following corollary about the excess risk R(W1, 1, th ((T+))). Corollary 1. Under Assumption if n ei max /T, log(1/6) r², rnr++1 r universal constant 6j > 0, 2r min T), for W1 obtained in Algorithm with probability at least - (n 100),
for some
(T+1)) R(W1, W2
$
logn
r2p
TT
Remark 3. Lemma and Corollary provide counterparts of the bound of subspace distance and excess risk studied in [52, 21] under the setting regression models. Since they squared losses, our bounds are different from theirs by square roots. Squaring our bounds provide results with similar rates those in previous work. we do not data from source tasks, will obtain excess risk bound of order / pp/nu41 instead, which will be significantly larger than the one in Corollary I + log < and pr² < nf, which happens in our low rank situation with abundant source task data.
Mcanwhile, provide the following minimax lower bound justify the near-optimality of our algorithm in learning the representation in general cases.
Proposition 1. Let us consider the parameter space E {A Rxr B RPxr (A (/T) B, B 1.). If w'l Z rp, we then have
inf sup El sin O(B, WillF Z nT
rp
W,
Remark For high dimensional data such that p is much larger than T and log n, the lower bound in Proposition matches the upper bound in Lemma 1 up to a factor /r. Since is considered small constant in our settings, we can see that in general cases when there is no additional structural assumptions, algorithm already obtains the near-optimal rate in representation learning, However, later sections, when we introduce some additional structural assumptions such as varying signal- to no-noise ratios and sparsity structures among tasks, which commonly happens in real applications. we will show that adversarial training can improve representation learning and further leads to smaller excess risks.
3.2 How -zadversarial training improves representation learning for transfer
this subsection, we consider the benefit adaaversaaial training. Specifically, the signal-to-noiss ratios varies among tasks in the sense that ||a,]' I's have different scales, fz-adversarial training can
5

---

lead to sharper representation estimation enror than standard training. In contrast. Lemma 1 and Proposition I demonstrate that under the case of uniform signal-to-noise ratios, adversarial training cannot have any gain over standard training, From high-level perspective, signal-to Sggnal-t-nooise ratios determiine the difficulties of classification. For those tasks with small signal-to-noise ratios, while adversarial attacks make them even harder to perform classification (increase bias), but also make these tasks less competitive (decrease variance). Thus, adversarial training will bias the model to focus on learning the representation out of those with large signal-to -onise ratios.
Assumption& 2 (Varying signal-tor -o-iie ratios), For the T source tasks, they can be divided into two disjoint sets. The first set S₁ = {t E [T] ||a.] || (1)), and the second set is S₂ {t [T] ||aell = (. where or + oc as T - oc, and S U S2 [T). In addition. |S2l/T ((1).
For the matrix M |a1/ / Il. az /la/ll............... ar/ |ar Ill. we further denote Ms, as the sub-matrix of MM whose columns consist of of az/1|a, || for E Sr. For instance, if S = (1,2,3}. then
Ms, [ai/||ail| Il. /2/ //2// az/ ////al]] We define Ms, similarly.
Assumption (Task diversity). For the T source tasks, minfo, (M3, o, M/T)3 = (11rr).
Remark 5. Assumption 2 indicates if we have more source tasks (larger T), more tasks with large signal-to-noise ratios would show up. Similar to Assumption 1, Assumption 3 requires both the columns in M and Ms, cover Iº evenly.
Now, we consider the adversarial training algorithm for aatatcck for q = 2, oc.
Algorithm 2 Adversarial Learning for Linear Features
Input: { 9
Step 1: Optimize the adversarial loss function on each individual source task / E [7] and obtain
Badv argmin
||||
max S6
(t) (3e, Yi
(1)
8.).
14
Step 2: Wrado top-r SVD of [óqde Aado .... Aado). (T+11 <- argmin Step 3: v2
1/3 (w1)
(T+1)
(1- Wardv
mi (T+1)).
S1 nT+1
Return Wadde, W22
(T+1)
The following theorem shows that even when the Bado's obtained by fz-adversarial training have large excess risk for each source task, the Waadv extracted from [ Aade ,.... [ady] can transfer knowledge from multiple source tasks better and result in smaller excess risk on the target task.
Theorem 1. Under Assumption and 3, for ||aT+al| = a = Q(1), in > e1 max(r2, max log T, log n/T,1} 1} and π. > (2( for universal constants C2, 2r < min(p, T}. There exists a universal constant C3. such that if we choose e E [maxtes ||alll + C3 c log T/n, minses, [|a,]| - ca plog T/n (this set will not be empty if n are large enough), for Wadd, 22
(T+)) obtained in Algorithm with q with probability at least 1 O(n 1100),
(V
22
2p2
2 log
sin O(Waddv BBIl $ (aT)-1
+
nf
nT
and the excess risk
R(Wado 10
(Tg)),(T+1) S vv
+ logn
+ (aT) -1
p2p
NT+1
nT
Similar to Assumption 2, here can also be function of the target task data size RT+1-
fzaadversarial training v.s. standard training: Under the exact same conditions in Theo- rem simple modification of Lemma leads to || sin () S Vr2/n /nr"/(n(T) / 2 n/ mn/n////1 (nTT) and R(W, +(((+))) $ aa logn) /T2p/ with high probability.

---

We can that the adversarial training would lead to better representation and an improved excess risk when at is growing Such a scenario happens when the source data consist of large diversity of tasks with varying difficulties of classification. Our proof indeed reveals that adversarial training would help the model to focus on learning the representation from easy-to- col-sssyysss tasks, and therefore improves the convergence rate of representation learning. The gain in representation learnigg further aade,(T++) (TT+))))))))))))))))))))))) compared with R(W,, 1i2 (((+
leads to smaller rates of (WWaa,, 1i:2
3.3 How foo adversarial training improves representation learning for transfer
In this subsection, we further consider the benefit of foe adversarial training. It is well-recognized that commonly used real data sets, such as MNIST and CIFAR-IO, actually lie in lower dimensional manifolds compared with their ambient dimensions. After certain transformations [2, 9], it is equivalent to having sparsity structure in the coordinates. We demonstrate that if there are some underlying sparsity structures in the mean parameters μ4 Ba, for t E [T], then & adversarial training leads to sharper bounds regarding the representation error and excess risk. To facilitate the discussion, let us Mt.j to denote the j-th coordinates of pe-
A Assumption 4 (Structural sparsity). For an integer such that 0 < 8 sign(pt, are i.i.d. and ((sign((tt.) = 0) n./2. We also refer as the sparsity level.
assume for all E [T]. Ts. P(sign((tt.) = 1) = P(sign((4))) = -1)
Assumption 4 guarantees that the sparsity of each column is upper bounded by O(s log 7) with high probability. Similar assumptions have been commonly used in the high-dimensional statistics literature [4, 51]. For adversaaraa training, we provide bounds obtained through adversarial training below.
Theorem 2. Under Assumptions 1 and 4, if > ei r2 max log" T/T, ruT+I, 1} for some universal constants ei > 0, 2r S min T}. There exists a universal constunt c2, such that if we choose & > 02 c2/llogp/n for and Wadv, i2
(T-1) obtained in Algorithm 2 with = co, with
probability least O( 100) 1 -(
(
s2
||sin O(Wadd B)lle S
+ n
log(T + +p),
nT
and the excess risk
R(Wado 202 202 (ogd).(++1)) S
+ log
+ rV
s2 nf
log(( + +p).
(5)
NT+1
adversssall training v.s. standard training: Under the exact same conditions in Theorem 2, again, a simple modification of Lemma I shows that without adversarial training, with high prob- ability, we have || sin B)Il S r(vI/n Vp/nTT + /logn/n(T) and the excess risk RRW,, 1, ",, -ifr-1)== S v(r+ log n)/m/+1 + tr/p/nT. Theorem 2 shows that adverrsaal training is able to learn significantly better representations when s² < p. This scenario is common in image classification where the label of an image only depends on small set of feature. Our proof reveals that to adversarial training would help remove the redundant features in the classification tasks and therefore improves the representation learning and the subsequent downstream prediction on target domain.
4 Pseudo-Labeling and Adversarial Training
In the previous section, we have shown that combining abundant data from source tasks with robust training can help learn good classifier for the target task. Sometimes, however, even the sources have limited labeled data. In that case, data augmentation by incorporating unlabeled source data, which are easier obtain, can be a powerful way to improve prediction accuracy. One of the most commonly used superviee learning algorithms is the pseudo-labeling algorithm [12]. In this section, we explore how using pseudo-labeling in the source data can improve transfer learning and how adversarial training can further boost that improvement, both empirically and theoretically.
7

---

Arqunaa Po)
Arcuncy (Na
100%
25% Imagelet
Inspotid
25% trapete 3E% PAAEEElaee
10% trugohtd 50% Patato sdub
380% teapelett
traeefid
Epsilon
Ession
(a) E2 norm training
(b) too norm training
Figure 2: Comparison of target task (CIFAR-10) accuracy for models trained on source task (Im- ageNet) using (i). a 10% sample of data from the source task, (ii), the 10% sample with ground truth labels, and the remaining 90% with pseudo-labels, and (iii). 100% of the source task data with ground truth labels. Models trained on the source task with (a). fz-adversarial training and (b). lx -dvvevrrrrial training both exhibit similar behavior. The x-axis refers to the magnitude, E,
used in adversarial traning-larree values indicate allowing more difficult adversarial examples; corresponds no adversarial training. The value in (b) scaled up by 255 so it corresponds to pixel difference on a [0,255] scale.
Experiments. We perform empirical study of image classification. Our source tasks are image classification ImageNet [45]; target tasks image classification CIFAR-10 [28]. To simulate the pseudo-labeling setup, we sample 10% of ImageNet, train a ResNet-18 model on this sample (without adversarial training), and generate pseudo-labels for the remaining 90%, We then train a new source model using all of the source labeled and pseudo-labeled data with and without adversarial training. We a public library for adversarial training [22]. The high-level approach for adversarial training is as follows: at each iteration, take small number of gradient steps to generate adversarial examples from input batch; then update network weights using the loss gradients from the adversarial batch.
In Figure 2, plot the target task accuracy of models trained our source task in 3 different settings, across different levels of adversarial training. Models in Figure (a) and (b) are trained on the source task with le and loc adversarial training respectively. We compare models trained on the source task using: 1) a fixed 10% sample of ImageNet,; 2) the 10% sample with ground truth labels and the remaining 90% sample using the generated pseudo-labele and 3) all of ImageNet with its ground truth labels. Adversarial training boosts transfer performance all settings. Pseudo-labels also boost transfer performance. In the
setting (i.e. adversarial training). the highest target accuracy is obtained by using labeled examples with pseudo-labels (green points). Moreover, adversarial training with pseudo-labels also increases performance; the optimal setting for adversarial training, the difference from using pseudo-labels instead of ground truth labels is only 1.5%.
Table 1: Effect of amount of pseudo-labels on transfer task performance (measured with accuracy). At 0%, we just use 10% of data from the source task; at 900%, we use all remaining 90% of data with pseudo-labels (this is 9 times the train set size). Adversarial training corresponds to using lg agaavaaralial training with e = 1 on the source task. Results on additional datasets in Appendix.
Source Task
Target Tusk +0% Pseudo lahels
+200% Psendo labels
+500% Pseudo lahels
+900% Pseudo labels
ImageNet
CIFAR-
73.0%
73.8%
77.1 1%
78.8
ImageNet (wanin.t)aining))
CIFAR-10
82.8%
85.7%
87.5 %
87.8
ImageNet
CIFAR-100
51.0%
$2.9%
55.3 %
58.4%
ImageNet (walivv.taanninn)
CIFAR-100
65.2%
In Table 1 we investigate how the amount of pseudo-labeled data affects performance. We train models settings: with adversarial and aoversrrraliall (standard) training on the source task. The adversarial training corresponds to & norm adversarial training with 1. Across all settings, we observe that robust training improves performance, and adding more pseudo-labeled data improves performance with diminishing returns.
8

---

Theoretical illustration. We further support the above experimental observations with theories.
We denote the unlabeled input data for each source task /. [7) as X {-, (x."(0) h=r. The algorithm we analyze is as the following:
Algorithm 3 Natural and Adversarial Learning for Linear Features with Pseudo-labeling
Input: {5 ( {XF)
Step I: Train initial classifier: wint argmin [ 1 E Mi (w. xx!))
Step 2: Obtain pseudo labels: V, y,:(0) sgn( son((wnnee (t) 2, ))
Step 3: Obtain augmented data sets Strang by combining S, and {(a :i
a.(t) )
Step 4: (W1,aug 20ng & Algorithm 1SSt,aug: Sr-1),
(Wiaug 2,ag (2dd)((+1) + Algorithm 2(St, Sr-1
Return Wi,aug (1) Wadd (T+1(
L,aug 2,agg
Theorem 3. Denote ñ min ni and assume ñ Ci /T
.n] for some constant 61 > 0. Assume a, (M M/T) = ((1/r) and n² 2 2 for some 02 > 1, if (T + d) and minkeyy ||a/|| Olog n) and f) (t) N0, 1 P @(1). Let W Wi,g
obtained in Algorithm 3, with probability 1 O 100),
(V
log n nT
|| sin O(W1,(Wus))))) B)llE S
+
NT
Comparing the results above with Lemma 1, we theoretically justify that by incorporating unlabeled data, we are able to learn a better representation when >> n. the following, we show that adversarial training, together with the pseudo-labeling, can further boost this improvement.
Theorem 4. Under the same conditions as those in Theorem 3,
(a). For lo atluck, under assumptions same to the those in Theorem 1, and additionally > e1 max(r?, (raa(rrr//T) max(plogT] logn n/T,1] for a universal constant es, and choose E [max4es, ||o/|| + ca / "plog T/ñ, minies, ||a.ll C3 /pplo T/ñl, we then have with probability at least 1 - Onn -000).
|| sin (WWrde I,aug: B) & (aT)-1 (V
2r T.
2
log(n) nT
+
(b). For foo attack, under assumptions same to the those in Theorem 2, and additionally ñ > C Cr-r² max(x" ( log" T/T, 1 for a universal constant Ci. There exists a universal constant C22 such that if we choose >> ca saylogg /l/ooo/// p/h, with probability at least O(n 100) O100))))
s2 7T
ssin @(Wadd 1,aug: B)IlF E r(y - +
log(T + +p).
Similar to the interpretations of Theorems 1 and 2, Theorem 4 suggests that adversarial training can boost the representation learning either (i). when the signal to noise ratio is varying (&z adversarial training helps in this case) and (ii). where there are many redundant features in classification (fxo adversarial training helps in this case. Same to the analysis before, we can obtain similar upper bounds on the excess risks as those Theorems and 2 by using Eq. (4).
5 Discussion
In this paper, we provide the first theoretical framework to explain how adversarial training on the source data improves transfer learning. We show that adversarial training helps learning a more robust representation, and therefore boosts the predictive performance on the target task. Additionally, we extend our analysis to semi-supervised setting and show that adversarial training, together with pseudo-labeling, have complementary benefits and can further improve the transfer.

---

Societal impacts and limitations Transfer learning helps learn well-performed machine learning model with only small amount of labeled data from the target task. Our work contributes to this field by providing insights into factors that improve transfer learning. A limitation of our work is that we have to make some standard assumptions on the data generative distribution when developing theories, which were also made in several other theory papers. While the model is simple, it captures the essence of the problem studied the paper and is the first tractable framework to study how adversarial training helps and are supported by our experiments.
transfer learning. The analysis here are already challenging
Acknowledgements
The research of Linjun Zhang is supported by NSF DMS-2015378. The research of James Zou is supported by NSF CCF 1763191, NSF CAREER 1942926 and grants from the Silicon Valley Foundation and the Chan-Zuckerberg Initiative. This work is also in part supported by NSF award 1763665.
References
[1] Yogesh Balaji. Tom Goldstein, and Judy Hoffman. Instance adaptive adversarial training: Improved accuracy tradeoffs in neural nets. arXiv preprint arXiv: 2019.
[2] Richard G Baraniuk. Compressive sensing [lecture notes). IEEE signal processing magazine, 24(4): 118-11,, 2007.
[3] Jonathan Baxter. A model of inductive bias learning. Journal of artificial intelligence research, 12:149-199 1989 2000.
[4] Mohsen Bayati and Andrea Montanari. The lasso risk for gaussian matrices. IEEE Transactions Information Theory, 58(4): 1997-2777 2011.
[5] Shai Ben-David and Reba Schuller. Exploiting task relatedness for multiple task learning. In Learning theory and kernel machines, pages 567-580. Springer, 2003.
[6] David Berthelot, Nicholas Carlini, Ian Goodfellow, Nicolas Papemot, Avital Oliver, and Colin Raffel. Mixmatch: A holistic approach to semi-supervised learning. arXiv preprinl arXiv:1905.022 0224999 2019.
[7] Battista Biggio and Fabio Roli. Wild patterns: Ten years after the rise of adversarial machine learning. Pattern Recognition, 84:317-331, 2018.
[8] Lukas Bossard, Matthicu Guillaumin, and Lue Van Gool. 1ond-i00 discriminative components with random forests. In European conference computer vision, pages 446-461. Springer, 2014.
[9] Emmanuel Candès et Compressive sampling. In Proceedings of the international congress of mathematicians, volume 3, pages 1433-1452. Madrid, Spain, 2006.
[10] Nicholas Carlini and David Wagner. Towards evaluating the robustness of neural networks. In 2017 ieee symposium on security and privacy (sp). pages 39-57. IEEE, 2017.
[11] Yair Carmon, Aditi Raghunathan, Ludwig Schmidt, John C Duchi, and Percy S Liang. Unlabeled data improves adversarial robustness. In Advances in Neural Information Processing Systems, pages 11 11190-1110 1201, 2019.
[12] Olivier Chapelle, Bernhard Scholkopf, and Alexander Zien. Semi-supervised learning (chapelle, o. et al., eds.; 2006)]book reviews]. IEEE Transactions on Neural Networks, 20(3):542-542,
2009.
[13] Ping- Yeh Chiang, Renkun Ni, Ahmed Abdelkader, Chen Zhu, Christoph Studor, and Tom Goldstein. Certified defenses for adversarial patches. arXiv preprint arXiv:2003.0660 2020.
[14] Jeremy M Cohen, Elan Rosenfeld, and Zico Kolter. Certified adversarial robustness via randomized smoothing. arXiv preprint arXiv: I 1902.02918, 2019.
10

---

[15] Alexis Conneau and Douwe Kicla. Senteval: An evaluation toolkit for universal sentence representations. arXiv preprint arXiv:1803.05444 2018.
[16] Nilesh Dalvi, Pedro Domingos, Sumit Sanghai, and Deepak Verma. Adversarial classification. In Proceedings of the tenth ACM SIGKDD international conference on Knowledge discovery and data mining, pages 99-108, 2004.
[17] Zhun Deng, Cynthia Dwork, Jialiang Wang, and Linjun Zhang. Interpreting robust optimization via adversarial influence functions. In International Conference on Machine Learning, pages 2464 2473. PMLR, 2020.
[18] Zhun Deng, Hangfeng He, Jiaoyang Huang, and Weijie Su. Towards understanding the dynamics of the first-order adversaries. In International Conference Machine Learning, pages 2484- 2493. PMLR, 2020.
[19] Zhun Deng. Linjun Zhang, Amirata Ghorbani, and James Zou. Improving adversarial robustness via unlabeled out-of-domain data. International Conference on Artificial Intelligence and Statistics, 2021.
[20] Jeff Donahue, Yangqing Jia, Oriol Vinyals, Judy Hoffman, Ning Zhang, Eric Tzeng, and Trevor Darrell. Decaf: A deep convolutional activation feature for generic visual recognition. In International conference on machine learning, pages 647-635. PMLR, 2014.
[21] Simon S Du, Wei Hu, Sham Kakade, Jason D Lee, and Qi Lei. Few-shot learning via learning the representation, provably. arXiv preprint arXiv:2002.0944 2020.
[22] Logan Engstrom, Andrew Hyas, Hadi Salman, Shibani Santurkar, and Dimitris Tsipras. Robust- ness (python library), 2019. URL https: /ttg//gtttuuu//.//////// cwm/ comm/Maadrylabbrobusutteeee
[23] Ian Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversar- ial examples. arXiv preprint urXiv:1412.00
2014.
[24] Dan Hendrycks, Kimin Lee, and Mantas Mazeika. Using pre-training can improve model robustness and uncertainty. arXiv preprint arXiv: 1901:::901.099000 099600 2019.
[25] Neil Houlsby, Andrei Giurgiu, Stanislaw Jastrzehski, Bruna Morrone, Quentin De Laroussilhe, Andrea Gesmundo. Mona Attariyan, and Sylvain Gelly. Patameter-efficient transfer learning for nlp. In International Conference on Machine Learning, pages 2790-2799, PMLR, 2019.
[26] Minyoung Huh, Pulkit Agrawal, and Alexei A Efros. What makes imagenet good for transfer learning? arXiv preprint 1rXiv:1608.08614 2016.
[27] Alexander Kolesnikov, Lucas Beyer, Xiaohua Zhai, Joan Puigcerver, Jessica Yung, Sylvain Gelly, and Neil Houlsby. Big transfer (bit): General visual representation learning. arXiv preprint arXiv: 1370, 6(2)88, 2019.
[28] Alex Krizhevsky and Geoffrey Hinton. Learning multiple layers of features from tiny images. Technical report, Citeseer, 2009.
[29] Alexey Kurakin, Ian Goodfellow, and Samy Bengio. Adversarial machine learning at scale. arXiv preprint arXiv: 1eee1611100220 2016.
[30] Mathias Lecuyer, Vaggelis Atlidakis, Roxana Geambasu, Daniel Hsu, and Suman Jana. Certified robustness adversarial examples with differential privacy. In 2019 IEEE Symposium on Security and Privacy (SP), pages 656-672. IEEE, 2019.
[31] Joseph Jaewhan Lim. Transfer learning by borrowing examples for multiclass object detection. PhD thesis, Massachusetts Institute of Technology, 2012.
[32] Chizhou Liu, Yunzhen Feng, Ranran Wang, and Bin Dong. Enhaneing certified robustness of smoothed classifiers via weighted model ensembling. arXiv preprint vrXiv:2005.093366 2020.
[33] Daniel Lowd and Christopher Meek. Adversarial learning. In Proceedings of the eleventh ACM SIGKDD international conference on Knowledge discovery in data mining, pages 641-647, 2005.
11

---

[34] Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. arXiv preprint 3Xiv:1706.0606) 2017.
[35] Subhransu Maji, Esa Rahtu, Juho Kannala, Matthew Blaschko, and Andrea Vedaldi. Fine- grained visual classification of aircraft. arXiv preprint arXiv:1306.5133 2013.
[36] Andreas Maurer, Massimiliano Pontil, and Bernardino Romera-Paredes. The benefit of multitask representation learning. Journal of Machine Learning Research, 17(81):1-32, 2016.
[37] Takeru Miyato, Shin-ichi Maeda, Masanori Koyama, and Shin Ishii. Virtual adversarial training: regularization method for supervised and semi-supervised learning. IEEE transactions pattern analysis and machine intelligence, 41(8):1979-1999 2018.
[38] lurii Mokrii, Leonid Boytsov, and Pavel Braslavski. A systematic evaluation of transfer learning and pseudo-labeling with bert-based ranking models. arXiv preprint arXiv:2103.03335 2021.
[39] Anh Nguyen, Jason Yosinski, and Jeff Clune. Deep neural networks are easily fooled: High confidence predictions for unrecognizable images. In Proceedings of the IEEE conference computer vision and pattern recognition, pages 427-436, 2015.
[40] Maria-Elena Nilsback and Andrew Zisserman. Automated flower classification over a large number of classes. In 2008 Sixth Indian Conference on Computer Vision, Graphics & Image Processing, pages 722-729. IEEE, 2008.
[41] Alain Pajor. Metric entropy of the grassmann manifold. Convex Geometric Analysis, 34: 181-188, 1998.
[42] Omkar M Parkhi, Andrea Vedaldi, Andrew Zisserman, and CV Jawahar. Cats and dogs. In 2012 IEEE conference computer vision and pattern recognition, pages -909-.505... IEEE, 2012.
[43] Maithra Raghu, Chiyuan Zhang, Jon Kleinberg, and Samy Bengio. Transfusion: Understanding transfer learning for medical imaging. arXiv preprint
2019.
[44] Aditi Raghunathan, Jacob Steinhardt, and Percy Liang. Certified defenses against adversarial examples. arXiv preprint arXiv: 8934449444 2018
[45] Olga Russakovsky, Jia Deng. Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, et al. Imagenet large scale visual recognition challenge. International journal of computer vision, 1 11533:211-252) 2015.
[46] Hadi Salman, Andrew Ilyas, Logan Engstrom, Ashish Kapoor, and Aleksander Madry, Do adversarially robust imagenet models transfer better? arXiv preprint arXiv:2007.0848) 2020.
[47] Ludwig Schmidt, Shibani Santurkar, Dimitris Tsipras, Kunal Talwar, and Aleksander Mądry. Adversarially robust generalization requires more data. arXiv preprint arXiv: /804. /285, 2018.
[48] Ali Sharif Razavian, Hossein Azizpour, Josephine Sullivan, and Stefan Carlsson. Can features off-the-shelf: an astounding baseline for recognition. Proceedings of the IEEE conference on computer vision and pattern recognition workshops, pages 806- 813, 2014.
[49] Hoo-Chang Shin, Holger R Roth, Mingchen Gao, Le Lu, Ziyue Xu, Isabella Nogues, Jian- hua Yao, Daniel Mollura, and Ronald M Summers. Deep convolutional neural networks for computer-aided detection: Cnn architectures, dataset characteristics and transfer learning. IEEE transactions on medical imaging, 35(5):1285-1299) 2016.
[50] Robert Stanforth, Alhussein Fawzi, Pushmeet Kohli, et al. Are labels required for improving adversarial robustness? arXiv preprint 17X2v:1905.13725, 2019.
[51] Weijie Su, Matgorzata Bogdan, Emmanuel Candes, et al. False discoveries occur early on the lasso path. Annals of Statistics, 45(5):2133-2150, 2017.
[52] Nilesh Tripuraneni, Chi Jin, and Michael Jordan. Provable meta-learning of linear representa- tions. urXiv preprint ar 2002 1184, 2020.
12

---

[53] Nilesh Tripuraneni, Michael Jordan, and Chi Jin. On the theory of transfer learning: The importance of task diversity. arXiv preprint arXiv:2006. 11650, 2020.
[54] Alexandre B Tsybakov. Introduction to nonparametric estimation. Springer Science & Business Media, 2008.
[55] Francisco Utrera, Evan Kravitz, N Benjamin Erichson, Rajiv Khanna, and Michael W Mahoney. Adversariallyytrained deep nets transfer better: Illustration on image classification.
[56] Francisco Utrera, Evan Kravitz, N Benjamin Erichson, Rajiv Khanna, and Michael W Mahoney. Adversarially-traiee trainee deep nets transfer better. arXiv preprint arXiv:2007.05866 2020.
[57] Martin Wainwright. High-dimensional statistics: non-asymptotic viewpoint, volume 48.
Cambridge University Press, 2019.
[58] Huaxiu Yao, Long-Ka Kai Huang, Linjun Zhang, Ying Wei, Li Tian, James Zou, Junzhou Huang. et al. Improving generalization meta-learning via task augmentation. In International Conference Machine Learning, pages 11887-11897. PMLR, 2021.
[59] Huaxiu Yao, Linjun Zhang, and Chelsea Finn. Meta-learning with fewer tasks through task interpolation. arXiv preprint Stti::220060269900 2021.
[60] Yi Yu, Tengyao Wang, and Richard Samworth. A useful variant of the Kavis-kahar theorem for statisticians. Biometrika, 102(2):315-323. 2015.
[61] Hongyang Zhang, Yaodong Yu, Jiantao Jiao, Eric Xing, Laurent El Ghaoui, and Michael Jordan. Theoretically principled trade-off between robustness and accuracy. In International Conference on Machine Learning, pages 7472-7482. PMLR, 2019.
[62] Linjun Zhang, Zhun Deng, Kenji Kawaguchi, Amirata Ghorbani, and James Zou. How does mixup help with robustness and generalization? arXiv preprint vrXiv:2010.04819, 2020.
[63] Ming Zhong, Jack LeBien, Marconi Campos-Cerqueira, Rahul Dodhia, Juan Lavista Ferres, Julian P Velev, and T Mitchell Aide. Multispecies bioacoustic classification using transfer learning of deep convolutional neural networks with pseudo-labeling. Applied Acoustics, 166: 107375, 2020.
[64] Hong- Yu Zhou, Avital Oliver, Jianxin Wu, and Yefeng Zheng. When semi-supervised learning meets transfer learning: Training strategies, models and datasets. arXiv preprint 1XXiv:81/2.0533: 2018.
[65] Xiaojin Zhu and Andrew B Goldberg. Introduction to semi-supervised learning. Synthesis lectures on artificial intelligence and machine learning, 111):1-130, 2009.
[66] Xiaojin Zhu, Zoubin Ghahramani, and John D Lafferty. Semi-supervised learning using gaussian fields and harmonic functions. In Proceedings of the 20th International conference on Machine learning (ICML-03), pages 912-919, 2003.
13


Reviews:
[{"review_id": "SQQeXJRwpV", "reviewer": "Reviewer_bcKd", "summary": "This paper reveals, both theoretically and empirically, that adversarial training is beneficial to transfer learning, which helps learning a more robust representation and therefore boosts the predictive performance on the target task. Additionally, pseudo-labeling is claimed to have complementary benefits and can further improve the transfer.", "questions": "", "limitations": "", "rating": 5, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "While attempting to tackle an important problem, this paper, at least in its current form, has several flaws. \n1. The experiments are not very solid. It would have been nice to see some experiments on another domain than image classification, to see how this method generalizes. \n2. And, I would like to know, when adversarial training is directly plugged to the existing transfer learning method, in theory, under what conditions will it be effective?", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Please refer to the section \"Main Review\".\n\n", "review_text": "", "extra_scores": {"ethical_concerns": "I think there are no ethical issues with this paper.\n\n", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "6"}, "extra_text": {}}, {"review_id": "7uL9g9hmaQJ", "reviewer": "Reviewer_ZMUB", "summary": "This work proposes a theoretical perspective on why adversarial training in the source domain improves the fixed-feature transfer learning ability. The authors further show both pseudo-labeling and adversarial training can lead to better representation.\n", "questions": "", "limitations": "", "rating": 6, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "- Originality: The proposed theoretical analysis is the first analysis on the effect of adversarial training on transfer learning. This is quite novel and non-trivial to establish theoretical analysis covering few common setups in $l_2$, $l_\n{\\infty}$, and pseudo-labeling scenarios. \n\n- Quality: The proposed technical framework seems general sounds (Note I didn’t examine the proof in details).\n\n- Clarity: \n  * Starting from page 4, I found it a bit hard to follow. Perhaps, it is due to the use of many symbols or without reading the proofs. From readers, it requires going back-and-forth to check the original definition and assumptions. It might be helpful to add some intuition behind the algorithm, theory, etc. I also found several descriptions are not easy to follow. \n  For example, \n    * (a) L167, “we consider the low-rank case” --> it is not immediately clear it is referring to M. \n    * (b) L172, \"the task-specific error is easy to deal with.\" --> it will be helpful to expand this in more detail. \n    * (c) In L202-203, it is not obvious to me for the conclusion drawing from uniform signal-to-noise ratios. \n    * (d) In L319-320, it is not clear to me how to lead to the conclusion in varying signal to noise ratio ($l_2$ adversarial training) or with many redundant features ($l_\\infty$ adversarial training). \n\n  * Few citation format could be improved. For example, in [43, 44, 56, 57], the transaction name should be Capitalized; [45] does not cover the full author; In [58] of supplementary material, davis–kahan -> Davis–Kahan. (only list a few of them here)\n  * L112, add a space between \"learning\" and \"[20, 47]\".\n \n\n\n- Significance: the proposed results seem sound and justify few previously empirical observations. I wonder whether it is possible to extend the framework to justify the improved adversarial robustness during transfer learning (even without adversarial training on the target domain), empirically shown in [1]. \n\n  [1] https://openreview.net/forum?id=bgQek2O63w\n\n\n\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "NA", "review_text": "", "extra_scores": {"ethical_concerns": "NA", "needs_ethics_review": "No", "ethics_review_area": ["I don’t know"], "time_spent_reviewing": "2"}, "extra_text": {}}, {"review_id": "-zjk6udUHki", "reviewer": "Reviewer_Wuja", "summary": "The authors attempt to provide a theoretical explanation on why adversarial training helps with transfer of learned representations sometimes. The main deficiency is the lack of explanation and verification of the underlying assumptions of their analysis, making it hard to connect what they proposed to what we observe in practice. \n", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "- The authors propose a theoretical explanation on why adversarial training help with transfer learning. It is an interesting attempt to explain why models trained on ImageNet transfer better to CIFAR with adversarial training. \n\n- The main problem I find with the theoretical explanation offered in this paper is the validity of the assumptions. Assumption 2 is very strong: the paper is assuming unbounded signal-to-noise ratio difference among the tasks in terms of their regression vector a_t. This is very unusual. Or if we take the view from regression or optimization, these are a set of very badly scaled features or badly conditioned optimization problems.  \n\n- The second problem with this set of assumptions is that we cannot (or the authors have not) empirically verify them on actual data. So we cannot really make the connection that this signal-to-noise ratio difference problem is what makes adversarial training works in practice. \n\n- There could be simpler explanations on why adversarial training helps transfer from Imagenet to CIFAR. CIFAR images are of much lower resolution compared to Imagenet, and adversarial training could help with making the model less reliant on low level filter features, which are dependent on resolutions. Of course this hypothesis also needs to be verified in some way (e.g., by checking the low level filters), but I am putting this up as an example that any explanation on why adversarial training helps transfer need to make a connection to the real data.   \n\n- The part on pseudo-labeling does not quite fit well with the rest of the paper. It is only introduced in the experiment section and was not considered as part of the analysis. It is nice to see the empirical results, but does not add to the coherence of the paper. \n\n- Overall I think the issues with establishing the assumptions used in the theoretical analysis in this paper prevents me from recommending this paper for publication. \n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Yes", "review_text": "", "extra_scores": {"ethical_concerns": "None", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "3"}, "extra_text": {}}]

Author rebuttal:

Meta-review: This is a solid paper that studies an important phenomenon that is not well understood: why adversarial training on the source data can improve the ability of models to transfer to new domains or targets? This is somewhat surprising as it might be easier to think of adversarial training as a task specific modification that can reduce performance on the source data, and the connection to transfer learning is not very intuitive on a first sight.

The paper replicates an adversarially trained ($\ell_2$ robustness) Imegenet model providing better accuracy on CIFAR10. Starting from fixed-feature transfer learning (Salman et.al.), an adversarial training step is plugged into the fixed-feature learning step.The authors also show that pseudo-labeling can lead to better representations. The authors analyse a hierarchical linear model for multitask learning and provide Alg-1 as the linear representation learning algorithm. This setup gives an upper bound to excess risk that can be decomposed in two terms: representation error (related to the sum of angles between the estimated and true factor matrices) and a task dependent term. The representation error can be further bounded using a Davis-Kahan style perturbation bound (Wedin's theorem). The strategy of the paper is to extend Alg-1 to the adversarial training setting (Alg-2) and finally also to include pseudo labelling (Alg-3). 

The structure and the arguments of the paper are easy to follow. During rebuttal, several concerns were raised, I summarize main ones succinctly and less nuanced below:

1. The validity of the assumptions. Assumption 2 is very strong. No verification of assumptions. (This assumption is related to the magnitude of task specific weights $\alpha_k$ where a large signal-to-noise (SNR) corresponds to a large $||\alpha||$, a proxy of task uncertainty)

2. Pseudolabel scenario is not much related

3. Observed robustness could be explained by different resolution on the source and target datasets, Analysis does not explain/has a big gap to experiment as it requires too many tasks

In my opinion, the authors very clearly addressed the concerns raised by the reviewers. In particular, the connection of variation in the
SNR's to explain why adversarial training might help seems to come out naturally from the analysis and seems to be quite interesting.

In particular, I feel the paper makes some relevant contributions.

It carefully builds and contrasts the various algorithms and provides a compelling explanation for the observed phenomenon. Adversarial training will bias the model to focus on learning the representation out of those with large signal-to-noise ratios and enhances  transferability.

In my opinion, one possible useful addition to address point 3 above would have been to verify the theory with a set of synthetic simulations. This would be fairly straightforward starting from the model and could potentially illustrate the assumptions much more clearly by explicitly controlling the task distribution. Especially, I can envision an experiment that directly controls the task diversity and SNR's to empirically show the regime where robust training is helping. Many related concerns (regarding non-asymptotic validity) could have been partially addressed through such a synthetic experiment. 

Overall, I am persuaded that the work would be a solid contribution and in the lack of strong counter arguments from the reviewers, I am using my own judgement and independent evaluation to suggest acceptance.

Decision: Accept (Poster)