Meta-learning Transferable Representations with a Single Target Domain

1 INTRODUCTION . Transfer learning—transferring knowledge learned from a large-scale source dataset to a small target dataset—is an important paradigm in machine learning ( Yosinski et al. , 2014 ) with wide applications in vision ( Donahue et al. , 2014 ) and natural language processing ( NLP ) ( Howard & Ruder , 2018 ; Devlin et al. , 2019 ) . Because the source and target tasks are often related , we expect to be able to learn features that are transferable to the target task from the source data . These features may help learn the target task with fewer examples ( Long et al. , 2015 ; Tamkin et al. , 2020 ) . Mainstream approaches for transfer learning are fine-tuning and joint training . Fine-tuning initializes from a model pre-trained on a large-scale source task ( e.g. , ImageNet ) and continues training on the target task with a potentially different set of labels ( e.g. , object recognition ( Wang et al. , 2017 ; Yang et al. , 2018 ; Kolesnikov et al. , 2019 ) , object detection ( Girshick et al. , 2014 ) , and segmentation ( Long et al. , 2015 ; He et al. , 2017 ) ) . Another enormously successful example of fine-tuning is in NLP : pre-training transformers and fine-tuning on downstream tasks leads to state-of-the-art results for many NLP tasks ( Devlin et al. , 2019 ; Yang et al. , 2019 ) . In contrast to the two-stage optimization process of fine-tuning , joint training optimizes a linear combination of the objectives of the source and the target tasks ( Kokkinos , 2017 ; Kendall et al. , 2017 ; Liu et al. , 2019b ) . Despite the pervasiveness of fine-tuning and joint training , recent works uncover that they are not always panaceas for transfer learning . Geirhos et al . ( 2019 ) found that the pre-trained models learn the texture of ImageNet , which is biased and not transferable to target tasks . ImageNet pre-training does not necessarily improve accuracy on COCO ( He et al. , 2018 ) , fine-grained classification ( Kornblith et al. , 2019 ) , and medical imaging tasks ( Raghu et al. , 2019 ) . Wu et al . ( 2020 ) observed that large model capacity and discrepancy between the source and target domain eclipse the effect of joint training . Nonetheless , we do not yet have a systematic understanding of what makes the successes of fine-tuning and joint training inconsistent . The goal of this paper is two-fold : ( 1 ) to understand more about when and why fine-tuning and joint training can be suboptimal or even harmful for transfer learning ; ( 2 ) to design algorithms that overcome the drawbacks of fine-tuning and joint training and consistently outperform them . To address the first question , we hypothesize that fine-tuning and joint training do not have incentives to prefer learning transferable features over source-specific features , and thus their capability of learning transferable features is rather accidental depending on the property of the datasets . To empirically analyze the hypothesis , we design a semi-synthetic dataset that contains artificiallyamplified transferable features and source-specific features simultaneously in the source data . Both the transferable and source-specific features can solve the source task , but only transferable features are useful for the target . We analyze what features fine-tuning and joint training will learn . See Figure 1 for an illustration of the semi-synthetic experiments . We observed following failure patterns of fine-tuning and joint training on the semi-synthetic dataset . • Pre-training may learn non-transferable features that don ’ t help the target when both transferable and source-specific features can solve the source task , since it ’ s oblivious to the target data . When the dataset contains source-specific features that are more convenient for neural nets to use , pretraining learns them ; as a result , fine-tuning starting from the source-specific features does not lead to improvement . • Joint training learns source-specific features and overfits on the target . A priori , it may appear that the joint training should prefer transferable features because the target data is present in the training loss . However , joint training easily overfits to the target especially when the target dataset is small . When the source-specific features are the most convenient for the source , joint training simultaneously learns the source-specific features and memorizes the target dataset . Toward overcoming the drawbacks of fine-tuning and joint training , we first note that any proposed algorithm , unlike fine-tuning , should use the source and the target simultaneously to encourage extracting shared structures . Second and more importantly , we recall that good representations should enable generalization : we should not only be able to fit a target head with the representations ( as joint training does ) , but the learned head should also generalize well to a held-out target dataset . With this intuition , we propose Meta Representation Learning ( MeRLin ) to encourage learning transferable and generalizable features : we meta-learn a feature extractor such that the head fit to a target training set performs well on a target validation set . In contrast to the standard model-agnostic meta-learning ( MAML ) ( Finn et al. , 2017 ) , which aims to learn prediction models that are adaptable to multiple target tasks from multiple source tasks , our method meta-learns transferable representations with only one source and one target domain . Empirically , we first verify that MeRLin learns transferable features on the semi-synthetic dataset . We then show that MeRLin outperforms state-of-the-art transfer learning baselines in real-world vision and NLP tasks such as ImageNet to fine-grained classification and language modeling to GLUE . Theoretically , we analyze the mechanism of the improvement brought by MeRLin . In a simple two-layer quadratic neural network setting , we prove that MeRLin recovers the target ground truth with only limited target examples whereas both fine-tuning and joint training fail to learn transferable features that can perform well on the target . In summary , our contributions are as follows . ( 1 ) Using a semi-synthetic dataset , we analyze and diagnose when and why fine-tuning and joint training fail to learn transferable representations . ( 2 ) We design a meta representation learning algorithm ( MeRLin ) which outperforms state-of-the-art transfer learning baselines . ( 3 ) We rigorously analyze the behavior of fine-tuning , joint training , and MeRLin on a special two-layer neural net setting . 2 SETUP AND PRELIMINARIES . In this paper , we study supervised transfer learning . Consider an input-label pair ( x , y ) ∈ Rd × R. We are provided with a source distributions Ds and a target distribution Dt over Rd × R. The source dataset D̂s = { xsi , ysi } ns i=1 and the target dataset D̂t = { xti , yti } nt i=1 consist of ns i.i.d . samples from Ds and nt i.i.d . samples from Dt respectively . Typically ns nt . We view a predictor as a composition of a feature extractor hφ : Rd → Rm parametrized by φ ∈ Φ , which is often a deep neural net , and a head classifier gθ : Rm → R parametrized by θ ∈ Θ , which is often linear . That is , the final prediction is fθ , φ ( x ) = gθ ( hφ ( x ) ) . Suppose the loss function is ` ( · , · ) , such as cross entropy loss for classification tasks . Our goal is to learn an accurate model on the target domain Dt . Since the label sets of the source and target tasks can be different , we usually learn two heads for the source task and the target task separately , denoted by θs and θt , with a shared feature extractor φ . Let LD̂ ( θ , φ ) be the empirical loss of model gθ ( hφ ( x ) ) on the empirical distribution D̂ , that is , LD̂ ( θ , φ ) : = E ( x , y ) ∈D̂ ` ( gθ ( hφ ( x ) ) , y ) where ( x , y ) ∈ D̂ means sampling uniformly from the dataset D̂ . Using this notation , the standard supervised loss on the source ( with the source head θs ) and loss on the target ( with the target head θt ) can be written as LD̂s ( θs , φ ) and LD̂t ( θt , φ ) respectively . We next review mainstream transfer learning baselines and describe them in our notations . Target-only is the trivial algorithm that only trains on the target data D̂t with the objective LD̂t ( θt , φ ) starting from random initialization . With insufficient target data , target-only is prone to overfitting . Pre-training starts with random initialization and pre-trains on the source dataset with objective function LD̂s ( θs , φ ) to obtain the pre-trained feature extractor φ̂pre and head θ̂s . Fine-tuning initializes the target head θt randomly and initializes the feature extractor φ by φ̂pre obtained in pre-training , and fine-tunes φ and θt on the target by optimizing LD̂t ( θt , φ ) over both θt and φ . Note that in this paper , fine-tuning refers to fine-tuning all layers by default . Joint training starts with random initialization , and trains on the source and target dataset jointly by optimizing a linear combination of their objectives over the heads θs , θt and the shared feature extractor φ : minθs , θt , φ Ljoint ( θs , θt , φ ) : = ( 1− α ) LD̂s ( θs , φ ) + αLD̂t ( θt , φ ) . The hyper-parameter α is used to balance source training and target training . We use cross-validation to select optimal α . 3 LIMITATIONS OF FINE-TUNING AND JOINT TRAINING : ANALYSIS ON SEMI-SYNTHETIC DATA . Previous works ( He et al. , 2018 ; Wu et al. , 2020 ) have observed cases when fine-tuning and joint training fail to improve over target-only . Our hypothesis is that both pre-training and joint training do not have incentives to prefer learning transferable features over source-specific features , and thus the performance of fine-tuning and joint training rely on whether the transferable features happen to be the best features for predicting the source labels . Validating this hypothesis on real datasets is challenging , if not intractable—it ’ s unclear what ’ s the precise definition or characterization of transferable features and source-specific features . Instead , we create a semi-synthetic dataset where transferable features and source-specific features are prominent and well defined . A semi-synthetic dataset . The target training dataset we use is a uniformly-sampled subset of the CIFAR-10 training set of size 500 . The target test dataset is the original CIFAR-10 test set . The source dataset of size 49500 , denoted by AB , is created as follows . The upper halves of the examples are the upper halves of the CIFAR-10 images ( excluding the 500 example used in target ) . The lower halves contain a signature pattern that strongly correlates with the class label : for class c , the pixels of the lower half are drawn i.i.d . from gaussian distribution N ( c/10 , 0.22 ) . Therefore , averaging the pixels in the lower half of the image can reveal the label because the noise will get averaged out . The benefit of this dataset is that any features related to the top half of the images can be defined as transferable features , whereas the features related to the bottom half are source-specific . Moreover , we can easily tell which features are used by a model by testing the performance on images with masked top or bottom half . For analysis and comparison , we define A to be the dataset that contains the top half of dataset AB and zeros out the bottom half , and B vice versa . See Figure 1 ( left ) for an illustration of the datasets . Further details are deferred to Section A.1 In Figure 1 ( right ) , we evaluate various algorithms ’ performance on target test data . In Figure 2 ( a ) ( left ) , we run algorithms with AB being the source dataset and visualize the learned features on the target training dataset and target test dataset to examine the generalizability of the features . In Figure 2 ( a ) ( right ) , we evaluate the algorithms on the held-out version of dataset A and B to examine what features the algorithms learn . ResNet-32 ( He et al. , 2016 ) is used for all settings . Analysis : First of all , target-only has low accuracy ( 38 % ) because the target training set is small . Except when explicitly mentioned , all the discussions below are about algorithms on the source AB . Fine-tuning fails because pre-training does not prefer to learn transferable features and fine-tuning overfits . Figure 2 ( b ) ( pre-training ) shows that the pre-trained model has near-trivial accuracy on heldout A but near-perfect accuracy on held-out B , indicating that it solely relies on the source-specific feature ( bottom half ) and does not learn transferable features . Figure 2 ( a ) ( pre-training ) shows that indeed pre-trained features do not have even correlation with target training and test sets . Figure 2 ( a ) ( fine-tuning ) shows that fine-tuning improves the features ’ correlation with the training target labels but it does not generalize to the target test because of overfitting . The performance of fine-tuning ( with source =AB ) in Figure 1 ( right ) also corroborates the lack of generalization . Joint training fails because it simultaneously learns mostly source-specific features and features that overfit to the target . Figure 2 ( b ) ( joint training ) shows that the joint training model performs much better on held-out B ( with 92 % accuracy ) than on the held-out A ( with 46 % accuracy ) , indicating it learns the source-specific feature very well but not the transferable features . The next question is what features joint training relies on to fit the target training labels . Figure 2 ( a ) shows strong correlation between joint training model ’ s features and labels on the target training set , but much less correlation on the target test set , suggesting that the joint training model ’ s feature extractor , applied on the target data ( which doesn ’ t have source-specific features ) , overfits to the target training set . This corroborates the poor accuracy of joint training on the target test set ( Figure 1 ) , which is similar to target-only ’ s.1 In Section 5 , we rigorously analyze the behavior of these algorithms on a more simplified settings and show that the phenomena above can theoretically occur .

Summary: The paper investigates failure cases for transfer learning (fine-tuning and joint training), specifically in the context where training on the source data may highlight features that are irrelevant for the target data. This is done through semi-synthetic data. Based on the insights, the authors present an approach called Meta Representation Learning (MeRLin) inspired by Meta Learning and Learning-to-learn approaches. This approach is evaluated on several real-world transfer-learning tasks from vision and NLP. The authors also derive theoretical results on constructed data distributions for which superiority of Merlin can be shown analytically.