A UTO -T RANSFER : L EARNING TO ROUTE TRANSFERABLE REPRESENTATIONS
Anonymous authors
Paper under double-blind review
A BSTRACT
Knowledge transfer between heterogeneous source and target networks and tasks has received a lot of attention in recent times as large amounts of quality labelled data can be difﬁcult to obtain in many applications.
Existing approaches typically constrain the target deep neural network (DNN) feature representations to be close to the source DNNs feature representations, which can be limiting.
We, in this paper, propose a novel adversarial multi-armed bandit approach which automatically learns to route source representations to appropriate target representations following which they are combined in meaningful ways to produce accurate target models.
We see upwards of 5% accuracy improvements compared with the stateof-the-art knowledge transfer methods on four benchmark (target) image datasets
CUB200, Stanford Dogs, MIT67 and Stanford40 where the source dataset is ImageNet.
We qualitatively analyze the goodness of our transfer scheme by showing individual examples of the important features our target network focuses on in different layers compared with the (closest) competitors.
We also observe that our improvement over other methods is higher for smaller target datasets making it an effective tool for small data applications that may beneﬁt from transfer learning.
I NTRODUCTION
Deep learning models have become increasingly good at learning from large amounts of labeled data.
However, it is often difﬁcult and expensive to collect sufﬁcient a amount of labeled data for training a deep neural network (DNN).
In such scenarios, transfer learning (Pan & Yang, 2009) has emerged as one of the promising learning paradigms that have demonstrated impressive gains in several domains such as vision, natural language, speech, etc., and tasks such as image classiﬁcation (Sun et al., 2017;
Mahajan et al., 2018), object detection (Girshick, 2015; Ren et al., 2015), segmentation (Long et al., 2015; He et al., 2017), question answering (Min et al., 2017; Chung et al., 2017), and machine translation (Zoph et al., 2016; Wang et al., 2018).
Transfer learning utilizes the knowledge from information-rich source tasks to learn a speciﬁc (often information-poor) target task.
There are several ways to transfer knowledge from source task to target task (Pan & Yang, 2009), but the most widely used approach is ﬁne-tuning (Sharif Razavian et al., 2014) where the target
DNN being trained is initialized with the weights/representations of a source (often large) DNN (e.g. ResNet (He et al., 2016)) that has been pre-trained on a large dataset (e.g. ImageNet (Deng et al., 2009)).
In spite of its popularity, ﬁne-tuning may not be ideal when the source and target tasks/networks are heterogeneous i.e. differing feature spaces or distributions (Ryu et al., 2020;
Tsai et al., 2020).
Additionally, the pretrained source network can get overwritten/forgotten which prevents its usage for multiple target tasks simultaneously.
Among the myriad of other transfer techniques, the most popular approach involves matching the features of the output (or gradient of the output) of the target model to that of the source model (Jang et al., 2019; Li et al., 2018; Zagoruyko & Komodakis, 2016).
In addition to the output features, a few methods attempt to match the features of intermediate states between the source and target models.
Here, in this paper, we focus on the latter by guiding the target model with the intermediate source knowledge representations.
Besides differing on what features to match, these methods also vary on how to match features, typically based on a regularization approach.
While common approaches allow knowledge transfer between heterogeneous tasks/networks, it is also important to recognize that constraining the target DNN representations to be close to certain source DNN representations may be sub-optimal.
For example, a source model, trained to classify cats vs dogs may be accessed at different levels to provide internal representations of tiger or wolf images to guide the target task in classifying tigers vs wolves.
Since the source model is trained with a large number of parameters and labeled examples of cats and dogs, it will have learned several patterns that distinguish cat images from dog images.
It is postulated that concepts or representations such as the shape of the tail, eyes, mouth, whiskers, fur, etc. are useful to differentiate them (Neyshabur et al., 2020), and it is further possible to reuse these learned patterns to generalize to new (related) tasks by accessing representations at the appropriate level.
This example raises three important questions related to knowledge transfer between the source-target models: 1) What knowledge to transfer? 2) Where to transfer
) How to transfer the source knowledge?
While the what and where have been considered in prior literature (Jang et al., 2019), our work takes a novel and principled approach to the questions of what , where and how to transfer knowledge in the transfer learning paradigm.
Speciﬁcally, and perhaps most importantly, we address the question of how to transfer knowledge, going beyond the standard matching techniques, and take the perspective that it might be best to let the target network decide what source knowledge is useful rather than overwriting one’s knowledge to match the source representations.
Figure 1 illustrates our approach to knowledge transfer where the question of what and where is addressed by an adversarial multiarmed bandit ( routing function) and the how is addressed by an aggregation operation detailed later.
In building towards these goals, we make the following contributions:
• We propose a transfer learning method that takes a novel and principled approach to automatically decide which source layers (if any) to receive knowledge from.
To achieve this, we propose an adversarial multi-armed bandit (AMAB) to learn the parameters of our routing function.
• We propose to meaningfully combine feature representations received from the source network with the target network-generated feature representations.
Among various aggregation operations that are considered, AMAB also plays a role in selecting the best one.
This is in contrast with existing methods that force the target representation to be similar to source representation.
• Beneﬁts of the proposed method are demonstrated on multiple datasets.
Signiﬁcant improvements are observed over seven existing benchmark transfer learning methods, particularly when the target dataset is small.
For example, in our experiment on ImageNet-based transfer learning on the target Stanford 40 Actions dataset, our auto-transfer learning method achieved more than 15% improvement in accuracy over the best competitor.
R ELATED W ORK
Transfer learning from a pretrained source model is a well-known approach to handle target tasks with a limited label setup.
A key aspect of our work is that we seek to transfer knowledge between heterogeneous DNNs and tasks.
Recent work focused on feature and network weight matching to address this problem where the target network is constrained to be near the source network weights and/or feature maps.
Network matching based on L 2 − SP regularization penalizes the (cid:96) 2 distance of the pretrained source network weights and weights of the target networks to restrict the search space of the target model and thereby hinder the generalization (Xuhong et al., 2018).
Recent work (Li et al., 2018) has shown that it is better to regularize feature maps of the outer layers than the network weights and reweighting the important feature via attention.
Furthermore, attention-based feature distillation and selection (AFDS) matches the features of the output of the convolutional layers between the source-target models and prunes the unimportant features for computational efﬁciency.
Similar matching can also be applied to match the Jacobians (change in output with respect to input rather than matching the output) between source and target networks (Srinivas & Fleuret, 2018).
Previous works (Dhurandhar et al., 2018; 2020) also suggested that rather than matching the output of a complex model, it could also be used to weight training examples of a smaller model.
Learning without forgetting (LwF) (Li & Hoiem, 2017) leverages the concept of distillation (Hinton et al., 2015) and takes it further by introducing the concept of stacking additional layers to the source network, retraining the new layers on the target task, and thus adapting to different source and target tasks.
SpotTune (Guo et al., 2019) introduced an adaptive ﬁne-tuning mechanism, where a policy network decides which parts of a network to freeze vs ﬁne-tune.
FitNet (Romero et al., 2014) introduced an alternative to ﬁne-tuning, where the internal feature representations of teacher networks were used as a guide to training the student network by using (cid:96) 2 matching loss between the two feature maps.
Attention Transfer (AT) (Zagoruyko & Komodakis, 2016) used a similar approach to FitNet, except the matching loss was based on attention maps.
The most relevant comparison to our work is that of Learning to Transfer (L2T-ww)
(Jang et al., 2019), which matches source and target feature maps but uses a meta-learning based approach to learn weights for useful pairs of source-target layers for feature transfer.
Unlike L2T-ww, our method uses a very different principled approach to combine the feature maps in a meaningful way (instead of feature matching) and let the target network decide what source knowledge is useful rather than overwriting one’s knowledge to match the source representations.
Finally, Ji et al.
(2021) uses knowledge distillation based approach to transfer knowledge between source and target networks.
A UTO -T RANSFER M ETHOD
In this section, we describe our main algorithm for Auto-Transfer learning and explain in detail the adversarial bandit approach that dynamically chooses the best way to combine source and target representations in an online manner when the training of the target proceeds.
What is the best way to train a target network such that it leverages pre-trained source representations speeding up training on the target task in terms of sample and time efﬁciency ?
We propose a routing framework to answer this:
At every target layer, we propose to route one of the source representations from different layers and combine it with a trainable operation (e.g. a weighted addition) such that the composite function can be trained together.
We propose to use a bandit algorithm to make the routing/combination choices in an online manner, i.e. which source layer’s representation to route to a given target layer and how to combine, while the training of the target network proceeds.
The bandit algorithm intervenes once every epoch of training to make choices using rewards from evaluation of the combined network on a hold out set, while the latest choice made by the bandit is used by the training algorithm to update the target network parameters on the target task.
We empirically show the beneﬁt of this approach with other baselines on standard benchmarks.
We now describe this framework of source-target representation transfer along with the online algorithm.
3. R OUTING R EPRESENTATIONS
For a given image x , let { f 1 S ( x ) , f 2 S ( x ) , · · · , f NS ( x ) } and { f 1 T ( x ) , f 2 T ( x ) , · · · , f MT ( x ) } be the intermediate feature representations for image x from the source and the target networks, respectively.
Let us assume the networks have trainable parameters
W S ∈ R d s and W T ∈ R d t where d s and d t are the total number of trainable parameters of the networks.
Clearly, the representations are a function of the trainable parameters of the respective networks.
We assume that the source network is pre-trained.
These representations could be the output of the convolutional or residual blocks of the source and target networks.
Our Key Technique: For the i -th target representation f iT , our proposed method a) maps i to one of the N intermediate source representations, f jS , or NULL (zero valued) representation; b) uses T j , a trainable transformation of the representation f jS , to get ˜ f jS , i.e. ˜ f jS ( x ) =
T j ( f jS ( x )) ; and c) combines transformed source ˜
f jS and the target representations f iT using another trainable operation (cid:76) chosen from a set of operations M .
Let W (cid:76) i,j be the set of trainable parameters associated with the operator chosen.
We describe the various possible operations below.
The target network uses the combined representation in place of the original i -th target representation:
[Equation]
In the above equation, the trainable parameters of the operator depend on the i and j (that dependence is hidden for convenience in notation).
The set of choices are discrete, that is, P = { [ N ] ∪ NULL }×M where [ N ] denotes set of N source representations.
Each choice has a set of trainable parameters T j , W (cid:76) i,j in addition to the trainable parameters W T of the target network.
3. L EARNING THE CHOICE THROUGH ADVERSARIAL BANDITS
To pick the source-target mapping and the operator choice, we propose an adversarial bandit-based online routing function (Auer et al., 2002) that picks one of the choices (with its own trainable parameters) containing information on what , where and how to transfer to the target representation i .
Brieﬂy, adversarial bandits choose actions a t from a discrete choice of actions at time t , and the environment presents an adversarial reward r t ( a t ) for that choice.
The bandit algorithm minimizes the regret with respect to the best action a ∗ in hindsight.
In our non-stationary problem setting, the knowledge transfer from the source model changes the best action (and the reward function) at every round as the target network adapts to this additional knowledge.
This is the key reason to use adversarial bandits for making choices as it is agnostic to an action dependent adversary. Bandit Update: We provide our main update Algorithm 1 for a given target representation i from layer ( (cid:96) ).
At each round t , the update algorithm maintains a probability vector π t over a set of all possible actions from routing choice space P .
The algorithm chooses a routing choice a t =
( j t → (cid:96), (cid:76) t ) randomly drawn according to the probability vector π t (in Line 7).
Here j t is the selected source representation to be transfered to the target layer l and combined with target representation i using the operator (cid:76) t .
Reward function: The reward r t for the selected routing choice is then computed by evaluating gain in the loss due to the chosen source-target combination as follows: the prediction gain is the difference between the target network’s losses on a hold out set D v with and without the routing choice a t i.e., L ( f MT ( x ))
− L ( ˜ f MT ( x )) for a given image x from the hold out data.
This is shown in the Algorithm 3 EVALUATE .
The reward function is used in Lines 4 and 5 to update the probability vector π p,t almost identical to the update in the classical EXP3.P algorithm of (Auer et al., 2002).
Note that if the current version of the trainable parameters is not available, then a random initialization is used.
In our experiments, this reward value is mapped to the [ − 1 , 1] range to feed as a reward to the bandit update algorithm.
Environment Update:
Given the choice j → i and the operator (cid:76) , the target network is trained for one epoch over all samples in the training data D T for the target task. Algorithm 2 TRAINTARGET updates the target network weights W T and other trainable parameters ( W (cid:76) i,j , T j ) of the routing choice a t for each epoch on the entire target training dataset.
Our main goal is to train the best target network that can effectively combine the best source representation chosen.
Here, L is the loss function which operates on the ﬁnal representation layer of the target network.
α t = 1 /t
and β is the exploration parameter.
We set β = 0 .
4 and γ = 10 − 3 .
1: Inputs: Learning rate α t , Exploration parameter β , Number of Epochs E .
Routing choice set
P
Initialize: w 0 ,p , ˜ r 0 ,p ← 0 .
2: for t ∈ [1 : E ] do 3: for p ∈ P do 4: w t,p ← log (cid:2) (1 − α t ) exp (cid:8)
w t − 1 ,p + γ ˜
r t − 1 ,p (cid:9) + α t K − 1 (cid:80) j (cid:54) = p exp (cid:8) w t − 1 ,j + γ ˜ r t − 1 ,j (cid:9)(cid:3) 5: π t,p ← (1 − β ) e w t,p (cid:80) Kj =1 e w t,j + β
K (2) 6: end for 7:
Choose action a t ∼ π t .
Let a t =
( j t → (cid:96), (cid:76) t ) .
Obtain current version of trainable parameters: (cid:16) W T , T j t , W (cid:76) t i,j (cid:17) .
Use the standard random initialization if not initialized.
9: r t,a t ← EVALUATE( a t , (cid:16) W T , T j t , W (cid:76) t i,j (cid:17) ) 10: (cid:16) W T , T j t , W (cid:76) t i,j (cid:17) ← TRAIN - TARGET( a t , (cid:16) W T , T j t , W (cid:76) t i,j (cid:17) ) 11: ˜ r t,p ← (cid:40) r t,p π t,p if p = a t , otherwise 12: end for
, (cid:76) ) .
Seed weight parameters: W T [0] , T j [0] , W (cid:76) i,j [0] .
2: Randomly shufﬂe D T .
3: for k ∈ [1 : | D T | ] do 4: x ← D 5: (cid:16)
T [
(cid:17) (cid:16) (cid:76)
i (cid:17) − η k ∇ (cid:16) W T ,T j , W (cid:76) i,j (cid:17) L ( f MT ( x 6: end for 7: Output: Last iterate
W
3. R OUTING C HOICES
The routing choice ( j → i, (cid:76) i,j ) can be seen as deciding where, what and how to transfer/combine the source representations with the target network.
Where to transfer?
The routing function j → i decides which one of the N intermediate source features is useful for a given target feature f iT .
In addition to these combinations, we allow the routing function to ignore the transfer using the NULL option.
[-Target task hold out set D v . 
-]
This allows the target network to potentially discard the source knowledge if it’s unrelated to the target task.
What to transfer?
Once a pair of source-task ( j → i ) combination is selected, the routing function decides what relevant information from the source feature f jS should be transferred to the target network using the transformation T j .
We use a Convolution-BatchNorm block to transfer useful features to the target network ˜ f
jS =
BN(Conv( f jS )) .
Here, T j = BN(Conv( · )) .
The convolution layer can select for relevant channels from the source representation and the batch normalization (Ioffe &
Szegedy, 2015) addresses the covariant-shift between the source and the target representations, we believe that this combination is sufﬁcient to ”match” the two representations.
This step also ensures that the source feature has a similar shape to that of the target feature.
How to transfer (i.e. combine the representations)?
Given a pair of source and target feature representations ( j → i ), the routing function chooses one of the following operations (i.e. (cid:76) ) to combine them.
We describe the class of operations M , i.e. the various ways (1) is implemented.
Identity (Iden) operation allows the target network just to use the target representation f iT after looking at the processed source representation ˜ f
jS from the previous Conv-BN step.
Simple Addition (sAdd) adds the source and target features:
˜ f iT = ˜
f jS + f iT .
Weighted Addition (wAdd) modiﬁes sAdd with weights for the source and target features.
These weights constitute W (cid:76) i,j . i.e. the trainable parameters of this operation choice: ˜ f
iT = w S,i,j ∗ ˜
f
jS + w T,i,j ∗ f iT .
Linear Combination (LinComb) uses the linear block (without bias term) along with the average pooling to weight the features: f iT =
Lin S,i,j ( ˜ f jS ) ∗ ˜
f jS + Lin T,i,j ( f iT ) ∗ f iT where Lin · ,i,j is a linear transformation with its own trainable parameters.
Feature Matching (FM) follows the earlier work and forces the target feature to be similar to the source feature.
This operation adds a regularization term w i,j (cid:107) ˜
f jS − f iT (cid:107) to the target objective L when we train.
Factorized Reduce (FactRed) use two convolution modules to reduce the number of channels c in the source and target features to c/ 2 and concat them together: f iT = concat(Conv c/ 2 S,i,j ( ˜ f jS ) , Conv c/ 2 T,i,j ( f iT )) .
An action a from the search space is given by [( j → i ) , (cid:76) i,j ] .
The total number of choice combinations is O (( N + 1) M ) .
Typically N and M are very small numbers, for instance, when Resnet is used as a source and target networks, we have N = 4 , M = 5 .
For large action search spaces, action pruning (Even-Dar et al., 2006) and greedy approaches (Bayati et al., 2020) can be used to efﬁciently learn the best combinations as demonstrated in our experiment section.
E XPERIMENTS
In this section, we present experimental results to validate our Auto-Transfer methods.
We ﬁrst show the improvements in model accuracy that can be achieved over various baselines on six different datasets and two network/task setups.
We then demonstrate superiority in limited sample size and limited training time usecases.
Finally, we use visual explanations to offer insight as to why performance is improved using our transfer method.
Experimental results on a toy example can be found in the supplement.
4. E XPERIMENTAL SETUP
Our transfer learning method is compared against existing baselines on two network/task setups.
In the ﬁrst setup, we transfer between similar architectures of different complexities; we use a 34-layer ResNet (He et al., 2016) as the source network pre-trained on ImageNet and an 18-layer ResNet as the target network.
In the second setup, we transfer between two very different architectures; we use an 32-layer ResNet as the source network pretrained on TinyImageNet and a 9-layer VGG (Simonyan & Zisserman, 2014) as the target network.
For ImageNet based transfer, we apply our method to four target tasks: Caltech-UCSD Bird 200 (Wah et al., 2011), MIT Indoor Scene Recognition (Quattoni & Torralba, 2009), Stanford 40 Actions (Yao et al., 2011) and Stanford Dogs (Khosla et al., 2011). For TinyImageNet based transfer, we apply our method on two target tasks:
CIFAR (Krizhevsky et al., 2009), STL-10 (Coates et al., 2011)
We investigate different conﬁgurations of transfer between source and target networks.
In the full conﬁguration, an adverserial multi-armed bandit (AMAB) based on Exponential-weight algorithm for Exploration and Exploitation (EXP3) selects (source, target) layer pairs as well as one of one of ﬁve aggregation operations to apply to each pair (operations are independently selected for each pair).
In the route conﬁguration, the AMAB selects (source, target) layer pairs but the aggregation operation is ﬁxed to be weighted addition.
In the ﬁxed conﬁguration, transfer is done between manually selected pairs of source and target layers with matching spatial size of intermediate representations.
Transfer can go between any layers, but the key is that the pairs are manually selected.
In each case, during training, the source network is passive and only shares the intermediate feature representation of input images hooked after each residual block.
After pairs are decided, the target network does aggregation of each pair of source-target representation in feed-forward fashion.
The weight parameters of aggregation are trained to act as a proxy to how much source representation is useful for the target network/task.
For aggregating features of different spatial sizes, we simply use a bilinear interpolation.
4. E XPERIMENTS ON T RANSFER B ETWEEN S IMILAR AND D IFFERENT A RCHITECTURES
In the ﬁrst setup, we evaluate all three Auto-Transfer conﬁgurations, full, ﬁxed, and route, on various visual classiﬁcation tasks, where transfer is from a Resenet-34 model to a Resnet-18 model.
Our ﬁndings are compared with an independently trained Resnet-18 model (Scratch), another Resnetmodel tuned for ImageNet and ﬁnetuned to respective tasks (Finetune), and the following existing baselines: Learning without forgetting (LwF) (Li & Hoiem, 2017), Attention Transfer (AT) (Zagoruyko & Komodakis, 2016), Feature Matching (FM) (Romero et al., 2014), Learning What and Where to Transfer (L2T-ww) (Jang et al., 2019) and Show, Attend and Distill (SAaD)
(Ji et al., 2021).
Results are shown in Table 6.
Each experiment is repeated 3 times.
First, note that the Auto-Transfer Fixed conﬁguration already improves performance on (almost) all tasks as compared to existing benchmarks.
The ﬁxed approach lets the target model decide how much source information is relevant when aggregating the representations.
This result supports our approach to feature combination and demonstrates that it is more effective than feature matching.
This even applies to the benchmark methods that go beyond and learn where to transfer to.
Next, note that the Auto-Transfer Route conﬁguration further improves the performance over the one-toone conﬁguration across all tasks.
For example, on the Stanford40 dataset, Auto-Transfer Route improves accuracy over the second best baseline by more than 15%.
Instead of manually choosing source and target layer pairs, we automatically learn the best pairs through our AMAB setup.
This result suggests that learning the best pairs through our AMAB setup to pick source-target pairs is a useful strategy over manual selection as done in the one-to-one conﬁguration.
Next, note that Auto-Transfer Full, which allows all aggregation operations, does well but does not outperform Auto-Transfer Route.
Indeed, the Auto-Transfer Full results showed that selected operations were all leaning to weighted addition, but other operations were still used as well.
We conjecture that weighted addition is best for aggregation, but the additional operations allowed in
Auto-Transfer Full introduce noise and make it harder to learn the best transfer procedure.
Additionally, we conducted experiments by ﬁxing aggregation to each of 5 operations and running
Auto-Transfer Route
and
found that weighted addition gave best performance Table 8.
In order to demonstrate that our transfer method does not rely on the source and target networks being similar architectures, we proceed to transfer knowledge from a Resnet-32 model to a VGGmodel.
Indeed, Table 6 in the appendix demonstrates that Auto-Transfer signiﬁcantly improves over other baselines for CIFAR100 and STL-10 datasets
4. E XPERIMENTS ON LIMITED AMOUNTS OF TRAINING SAMPLES
Transfer learning emerged as an effective method due to performance improvements on tasks with limited labelled training data.
To evaluate our Auto-Transfer method in such data constrained scenario, we train our Auto-Transfer Route method on all datasets by limiting the number of training samples.
We vary the samples per class from 10% to 100% at 10% intervals.
At 100%, Stanford has ∼ 100 images per class.
We compare the performance of our model against Scratch and L2T-ww for Stanford40 and report results in Figure 2 (top).
Auto-Transfer Route signiﬁcantly improves the performance over existing baselines.
For example, at 60% training set ( ∼ 60 images per class), our method achieves 77.90% whereas Scratch and L2T-ww achieve 29% and 46%, respectively.
To put this in perspective, Auto-Transfer Route requires only 10% images per class to achieve better accu-
[Table]
racy than achieved by L2T-ww with 100% of the images. We see similar performance with other three datasets: CUB200, MIT67, Stanford Dogs (Figure 9).
4. I MPROVEMENTS IN TRAINING AND INFERENCE TIMES
In order to assess training metrics and stability of learning, we visualize the test accuracy over training steps in
Figure 2 (bottom) for the Stanford40 dataset.
The results show that our method learns signiﬁcantly quicker relative to the second closest baseline.
For example, at epoch 25, our method achieves 74.55% accuracy whereas L2T-ww and Scratch achieve 25.55% and 21.69%, respectively.
In terms of training time, Auto-Transfer Route took ∼minutes to train 200 epochs on the Stanford40 dataset, whereas L2T-ww and Scratch models took 610 andminutes, respectively.
Taken together, our method signiﬁcantly improves performance over the second baseline with less than half the runtime.
We report additional experiments with training curves plotted against training time in appendix (Figure 7) and inference times plotted against test accuracy (Figure 8).
4. V ISUAL EXPLANATIONS
In order to qualitatively analyze what bandit AutoTransfer Route is learning, Grad-CAM (Selvaraju et al., 2017) based visual explanations are presented in Figure(additional explanations are in Figures 11, 12, and 13 in the appendix).
Grad-CAM highlights pixels that played an important role in correctly labelling the input image.
For each target task, we present a (random) example image that is correctly labelled by bandit Auto-Transfer but incorrectly classiﬁed by Scratch, along with layer-wise
Grad-CAM images that illustrate what each layer of the target model focuses on.
For each image, we report the incorrect label, correct label and class probability for correct ( p c ) and incorrect ( p i ) labels.
Overall, we observe that our method pays attention to relevant visual features in making correct decisions.
For example, in the ﬁrst image from MIT67 dataset, the Scratch model incorrectly labelled it as a gameroom while the correct class is bedroom ( p i = 0 .
67 , p c = 0 .
The Grad-CAM explanations show that layers 1-3 of the Scratch model pay attention to the green ﬂoor which is atypical to a bedroom and common in gamerooms (e.g. pool tables are typically green).
The last layer focuses on the surface below the window that looks like a monitor/tv that is typically found in gamerooms.
On the other hand, our model correctly identiﬁes the class as bedroom ( p c = 0 . 57 ) by paying attention to the bed and surrounding area at each layer.
To visualize an example from a harder task, consider the indigo bunting image from the CUBS dataset.
The Scratch model classiﬁes the image as a bluejay ( p i = 0 .
85 , p c = 0 . 09 )
, but our model correctly predicts it as a bunting ( p c = 0 . 99 ).
Indigo buntings and blue jays are strikingly similar, but blue jays have white faces and buntings have blue faces.
We clearly see this attribute picked up by the bandit Auto-Transfer model in layers 2 and 3.
We hypothesize that the source model, trained on millions of images, provides useful ﬁne-grained information useful for classifying similar classes.
C ONCLUSION
In this paper, we have put forth a novel perspective where we leverage and adapt an adversarial multi-armed bandit approach to transfer knowledge across heterogeneous tasks and architectures.
Rather than constraining target representations to be close to the source, we dynamically route source representations to appropriate target representations also combining them in novel and meaningful ways.
Our best combination strategy of weighted addition leads to signiﬁcant improvement over state-of-the-art approaches on four benchmark datasets.
We also observe that we produce accurate target models faster in terms of (training) sample size and number of epochs.
Further visualization based qualitative analysis reveals that our method produces robust target models that focus on salient features of the input more so than its competitors, justifying our superior performance.
E THICS S TATEMENT
We believe that one major ethical implication of our approach, which also applies to other transfer learning approaches, is the bias introduced by the source model to the target task when deployed.
We currently depend on the pretrained source models with having done ethical implications and having circumscribed their limitations and risks to society.
Another concern associated with the proposed approach is the choices made by the adversarial bandits based on the reward payoff which could be manipulated for intentional advantages.
One possibility to mitigate such concerns is to analyze the behavior of the target model on individual inputs.
To this end, we have visualized (in the appendix) regions in the input that the target model focuses on for both correct and incorrect predictions made by our model in comparison with the best performing baseline to facilitate such inquiries
R EPRODUCIBILITY D ISCUSSION
All experimental details (including the resources used for training and evaluating the models) are provided in Section 4 and the appendix A.3.
All datasets are public.
Code is provided in the supplementary material with instructions to run the models.
A A PPENDIX A. T OY E XAMPLE
In this section, we simulate our experiment on a toy example.
We compare our Auto-Transfer with the other baselines: L2T-ww and Scratch.
In this simulation, we consider Auto-Transfer with a ﬁxed (one-to-one) setup for simplicity in our experiment analysis.
We consider predicting a sine wave function ( y = sin ( x ) ) as our source task and a sinc function ( y = sin ( x ) x ) as our target task.
Clearly, the features from the pretrained source model will help the target task in predicting the sinc function.
Both the input data point x and the output value y are one-dimensional vectors ( d in = d out = 1 ).
We use a shallow linear network consists of 4 linear blocks: f 1 =
Lin ( d in ,h 1 ) ( x ) , f 2 =
Lin ( h 1 ,h 2 ) ( f 1 ) , f 3 =
Lin ( h 1 ,h 2 ) ( f 2 ) , out =
Lin ( h 3 ,d out ) ( f 3 ) for a datapoint x .
For source network, we set the hidden size to 64 (i.e., h 1 = h
= h 3 = 64 ) and 16 for the target network.
We sampled 30 , 000 data points to generate training set (x,y) and 10 ,test-set data points for the source network and (i.e., x is sampled from a Gaussian distribution and y = sin ( x ) ).
Similarly, we generated 1000 training examples and 800 test set examples for the target network.
Both the source and the target networks are trained for E = 50 epochs.
Figure 4 ( left ) shows the source model prediction for the test data.
Given the shallow linear network with 64 hidden dimensions and 30 , 000 training example, the source model perfectly predicts the sin ( x ) function.
Figure 4 ( right ) shows the predictions from the scratch target model, source model,
L2T-ww and Auto-Transfer for the target test data.
We report the Auto-Transfer with ﬁxed choice of [(0,0),(1,1),(2,2), wtAdd] for this experiment.
We can see that the Auto-Transfer accurately predicts the target task even when there is a limited amount of labeled examples.
Our results show the test set loss for the target data is relatively less compared to the other baselines ( 0 .
MSE loss for
Auto-Transfer vs 0 . 0033 and 0 .
125 MSE loss for the scratch and L2T-ww).
A. R EAL D ATASETS
We evaluate the performance of Auto-Transfer on six benchmarks with different tasks:
Stanford
Actions 40 dataset for action recognition, CUBS Birds 200 dataset for object recognition, Stanford
Dogs 120 for ﬁne-grained object recognition, MIT Indoors 67 for scene classiﬁcation, CIFARand STL-10 image recognition datasets.
Stanford Actions 40 .
Stanford Actions 40 dataset contains images of humans performing 40 actions.
There are about 180-300 images per class.
We do not use bounding box and other annotation information for training.
There are a total of 9,532 images, making it the smallest dataset in our benchmark experiments.
Caltech-UCSD Birds-200-2011 .
CUB-200-2011 is a bird classiﬁcation datset with 200 bird species.
Each species is associated with a wikipedia article and organized by scientiﬁc classiﬁ-
[Figure]
Stanford Dogs 120 .
The Stanford Dogs dataset contains images of 120 breeds of dogs from around the world.
There are exactly 100 examples per category in the training set.
It is used for the task of ﬁne-grained image categorization.
We do not use the bounding box annotations.
There are a total of 20,580 images.
MIT Indoors 67 .
MIT Indoors 67 is a scene classiﬁcation task containing 67 indoor scene categories, each of which consists of at most 80 images for training and 20 for testing.
Indoor scene recognition is challenging because spatial properties, background information and object characters are expected to be extracted.
There are 15,620 images in total.
CIFAR 100 .
CIFAR 100 is a image recognition task containing 100 different classes withimages in each class.
There are 500 training images and 100 testing images per class.
It is a subset of tiny images datastet.
STL 10 .
STL 10 is a image recognition task containing 10 classes. It is inspired by the CIFARdataset but with some modiﬁcations. In particular, each class has fewer labeled training examples than in CIFAR-10. A. E XPERIMENT D ETAILS
For our experimental analysis in the main paper, we set the number of epochs for training to
E = 200 .
The learning rate for SGD is set to 0 .
1 with momentum 0 .
9 and weight decay 0 .
The learning rate for the ADAM is set to 0 .
001 with and weight decay of 0 .
We use Cosine
Annealing learning rate scheduler for both optimizers.
The batch size for training is set to 64 .
Our target networks were randomly initialized before training.
The target models were trained in parallel on two machines with the speciﬁcations shown in Table 2.
[Table]
A. T RAINING AND T ESTING P ERFORMANCE
A. A DDITIONAL EXPERIMENTS ON LIMITED AMOUNTS OF DATA
To evaluate our Auto-Transfer method in data constrained scenario further, we train our AutoTransfer (route) method on the CUB200, Stanford Dogs and MIT67 datasets by limiting the number of training samples (Figure 9).
We vary the samples per class from 10% to 100% at 10% intervals.
[Table]
[Table] [Table] [Table] A. A BLATION STUDIES T RAINING THE NETWORK USING BANDIT SELECTED PAIRS
To evaluate the importance of training the target network with adversarial multi-armed bandit
[Table]
we retrained our target network with a ﬁxed source-layer conﬁguration selected at 200th epoch of previous best bandit based experiments.
For Eg.
in our best bandit based experiment for CUB
[Table]
the source [Table] target pairs were { (2,1), (3,2), (2,3), (2,4) } . As seen in Table
[Table] we ﬁnd that this experiment decreased performance in comparison to bandit based one in all target tasks.
This conﬁrms the need for bandit based decision maker, that learns combination weights and pairs over training steps.
[Table]
T RAINING THE NETWORK USING DIFFERENT AGGREGATION OPERATORS
To evaluate how different aggregation operators inﬂuence Auto-Transfer, we train Auto-Transfer
Route by ﬁxing aggregation to 5 different operations. Identity (iden), Simple Addition (sAdd), Weighted Addition (wtAdd), Linear Combination (LinComb) and Factored Reduction (FactRed). Results for Stanford40 dataset is found in Table 8.
We ﬁnd that weighted addition performs the best.
[Table]
[Figure]
A. A DDITIONAL E XPLANATIONS USING G RAD -CAM
We here offer more examples of visual explanations of what is being transferred using Auto-Transfer
Route.
The ﬁrst example in Figure 11 is an image of cooking from the Stanford40 dataset.
The Scratch model incorrectly classiﬁes the image as cutting ( p i = 0 .
88 , p c = 0 .
by paying attention to only the cooking surface that looks like a table and person sitting down (typical for someone cutting vegetables).
On the other hand, our model correctly labels the image ( p c = 0 . 99 ) by paying attention to the wok and cooking utensils such as water pot, etc.
We hypothesize that this surrounding information is provided by the source model which is useful in making the correct decision.
The second example in Figure 11 is from the Stanford Dogs dataset (Figure 11).
The scratch model fails to pay attention to relevant class information (dog) and labels a chihuahua as german sheperd ( p i = 0 .
23 , p c = 0 . 0002 )
by focusing on the ﬂower, while our method picks the correct label ( p c = 0 . 99 ).
Bandid Auto-Transfer gets knowledge about the ﬂower early on and then disregards this knowledge before attending to relevant class information.
Further examples of visual explanations comparing to L2T-ww (Figure 12) and counter-examples where our method identiﬁes the wrong label (Figure 13) follow below.
For these counter-examples we ﬁnd that the task is typically hard.
For eg.
playing violin vs playing guitar.
And, the class probability of incorrect label is closer to that of correct label, suggesting that our method was not conﬁdent in predicting wrong class.