SOSELETO: A Unified Approach to Transfer Learning and Training with Noisy Labels

1 INTRODUCTION . Deep learning has made possible many remarkable successes , leading to state of the art algorithms in computer vision , speech and audio , and natural language processing . A key ingredient in this success has been the availability of large datasets . While such datasets are common in certain settings , in other scenarios this is not true . Examples of the latter include “ specialist ” scenarios , for instance a dataset which is entirely composed of different species of tree ; and medical imaging , in which datasets on the order of hundreds to a thousand are common . A natural question is then how one may apply the techniques of deep learning within these relatively data-poor regimes . A standard approach involves the concept of transfer learning : one uses knowledge gleaned from the source ( data-rich regime ) , and transfers it over to the target ( data-poor regime ) . One of the most common versions of this approach involves a two-stage technique . In the first stage , a network is trained on the source classification task ; in the second stage , this network is adapted to the target classification task . There are two variants for this second stage . In feature extraction ( e.g . Donahue et al . ( 2014 ) ) , only the parameters of the last layer ( i.e . the classifier ) are allowed to adapt to the target classification task ; whereas in fine-tuning ( e.g . Girshick et al . ( 2014 ) ) , the parameters of all of the network layers ( i.e . both the features/representation and the classifier ) are allowed to adapt . The idea is that by pre-training the network on the source data , a useful feature representation may be learned , which may then be recycled – either partially or completely – for the target regime . This two-stage approach has been quite popular , and works reasonably well on a variety of applications . Despite this success , we claim that the two-stage approach misses an essential insight : some source examples are more informative than others for the target classification problem . For example , if the source is a large set of natural images and the target consists exclusively of cars , then we might expect that source images of cars , trucks , and motorcycles might be more relevant for the target task than , say , spoons . However , this example is merely illustrative ; in practice , the source and target datasets may have no overlapping classes at all . As a result , we don ’ t know a priori which source examples will be important . Thus , we propose to learn this source filtering as part of an end-to-end training process . The resulting algorithm is SOSELETO : SOurce SELEction for Target Optimization . Each training sample in the source dataset is given a weight , corresponding to how important it is . The shared source/target representation is then optimized by means of a bilevel optimization . In the interior level , the source minimizes its classification loss with respect to the representation parameters , for fixed values of the sample weights . In the exterior level , the target minimizes its classification loss with respect to both the source sample weights and its own classification layer . The sample weights implicitly control the representation through the interior level . The target therefore gets to choose the source samples which are most informative for its own classification task . Furthermore , the bilevel nature of the optimization acts as a kind of regularization on the target , mitigating overfitting , as the target does not directly control the representation parameters . Finally , note that the entire process – training of the shared representation , target classifier , and source weights – happens simultaneously . We pause here to note that the general philosophy behind SOSELETO is related to the literature on instance reweighting for domain adaptation , see for example Sugiyama et al . ( 2008 ) ; Yan et al . ( 2017 ) . However , there is a crucial difference between SOSELETO and this literature , which is related to the difference between domain adaptation and more general transfer learning . Domain adaptation is concerned with the situation in which there is either full overlap between the source and target label sets ; or in some more recent work Zhang et al . ( 2018 ) , partial but significant overlap . Transfer learning , by contrast , refers to the more general situation in which there may be zero overlap between label sets , or possibly very minimal overlap . ( For example , if the source consists of natural images and the target of medical images . ) The instance reweighting literature is concerned with domain adaptation ; the techniques are therefore relevant to the case in which source and target have the same labels . SOSELETO is quite different : it makes no such assumptions , and is therefore a more general approach which can be applied to both “ pure ” transfer learning , in which there is no overlap between source and target label sets , as well as domain adaptation . ( Note also a further distinction with domain adaptation : the target is often – though not always – taken to be unlabelled in domain adaptation . This is not the case for our setting of transfer learning . ) Above , we have illustrated how SOSELETO may be applied to the problem of transfer learning . However , the same algorithm can be applied to the problem of training with noisy labels . Concretely , we assume that there is a large noisy dataset , as well as a much smaller clean dataset ; the latter can be constructed cheaply through careful hand-labelling , given its small size . Then if we take the source to be the large noisy dataset , and the target to the small clean dataset , SOSELETO can be applied to the problem . The algorithm will assign high weights to samples with correct labels and low weights to those with incorrect labels , thereby implicitly denoising the source , and allowing for an accurate classifier to be trained . The remainder of the paper is organized as follows . Section 2 presents related work . Section 3 presents the SOSELETO algorithm , deriving descent equations as well as convergence properties of the bilevel optimization . Section 4 presents results of experiments on both transfer learning as well as training with noisy labels . Section 5 concludes . 2 RELATED WORK . Transfer learning As described in Section 1 , the most common techniques for transfer learning are feature extraction and fine-tuning , see for example Donahue et al . ( 2014 ) and Girshick et al . ( 2014 ) , respectively . An older survey of transfer learning techniques may be found in Pan & Yang ( 2010 ) . Domain adaptation Saenko et al . ( 2010 ) is concerned with transferring knowledge when the source and target classes are the same . Earlier techniques aligned the source and target via matching of feature space statistics Tzeng et al . ( 2014 ) ; Long et al . ( 2015 ) ; subsequent work used adversarial methods to improve the domain adaptation performance Ganin & Lempitsky ( 2015 ) ; Tzeng et al . ( 2015 ; 2017 ) ; Hoffman et al . ( 2017 ) . In this paper , we are more interested in transfer learning where the source and target classes are different . A series of recent papers Long et al . ( 2017 ) ; Pei et al . ( 2018 ) ; Cao et al . ( 2018a ; b ) address domain adaptation that is closer to our setting . In particular , Cao et al . ( 2018b ) examines “ partial transfer learning ” , the case in which there is partial overlap between source and target classes ( particularly when the target classes are a subset of the source ) . This setting is also dealt with in Busto & Gall ( 2017 ) . Ge & Yu ( 2017 ) examine the scenario where the source and target classes are completely different . Similar to SOSELETO , they propose selecting a portion of the source dataset . However , the selection is not performed in an end-to-end fashion , as in SOSELETO ; rather , selection is performed prior to training , by finding source examples which are similar to the target dataset , where similarity is measured by using filter bank descriptors . Another recent work of interest is Luo et al . ( 2017 ) , which focuses on a slightly different scenario : the target consists of a very small number of labelled examples ( i.e . the few-shot regime ) , but a very large number of unlabelled examples . Training is achieved via an adversarial loss to align the source and the target representations , and a special entropy-based loss for the unlabelled part of the data . Instance reweighting for domain adaptation is a well studied technique , demonstrated e.g . in Covariate Shift methods Shimodaira ( 2000 ) ; Sugiyama et al . ( 2007 ; 2008 ) . In these works , the source and target label spaces are the same . We , however , allow for different – even entirely nonoverlapping – classes in the source and target . Crucially , we do not make assumptions on the similarity of the distributions nor do we explicitly optimize for it . The same distinction applies for the recent work of Yan et al . ( 2017 ) , and for the partial overlap assumption of Zhang et al . ( 2018 ) . In addition , these two works propose an unsupervised approach , whereas our proposed method is completely supervised . Covariate shift determines the weighting for an instance as the ratio of its probability of being in the training set and being in the prediction set . Consequently , the feature vectors are used in re-weighting , regardless of their labels . This renders covariate shift unsuitable for handling noisy labels . Our re-weighing scheme is instead gradient-based and as we show next performs well in this task . Learning with noisy labels Classification with noisy labels is a longstanding problem in the machine learning literature , see the review paper Frénay & Verleysen ( 2014 ) and the references therein . Within the realm of deep learning , it has been observed that with sufficiently large data , learning with label noise – without modification to the learning algorithms – actually leads to reasonably high accuracy Krause et al . ( 2016 ) ; Sun et al . ( 2017 ) . The setting that is of greatest interest to us is when the large noisy dataset is accompanied by a small clean dataset . Sukhbaatar et al . ( 2014 ) introduce an additional noise layer into the CNN which attempts to adapt the output to align with the noisy label distribution ; the parameters of this layer are also learned . Xiao et al . ( 2015 ) use a more general noise model , in which the clean label , noisy label , noise type , and image are jointly specified by a probabilistic graphical model . Both the clean label and the type of noise must be inferred given the image , in this case by two separate CNNs . Li et al . ( 2017 ) consider the same setting , but with additional information in the form of a knowledge graph on labels . Other recent work on label noise includes Rolnick et al . ( 2017 ) , which shows that adding many copies of an image with noisy labels to a clean dataset barely dents performance ; Malach & ShalevShwartz ( 2017 ) , in which two separate networks are simultaneously trained , and a sample only contributes to the gradient descent step if there is disagreement between the networks ( if there is agreement , that probably means the label is wrong ) ; and Drory et al . ( 2018 ) , which analyzes theoretically the situations in which CNNs are more and less resistant to noise . A pair of papers Liu & Tao ( 2016 ) ; Yu et al . ( 2017 ) combine ideas of learning with label noise with instance reweighting . Bilevel optimization Bilevel optimization problems have a nested structure : the interior level ( sometimes called the lower level ) is a standard optimization problem ; and the exterior level ( sometimes called the upper level ) is an optimization problem where the objective is a function of the optimal arguments from the interior level . A branch of mathematical programming , bilevel optimization has been extensively studied within this community Colson et al . ( 2007 ) ; Bard ( 2013 ) . For recent developments , readers are referred to the review paper Sinha et al . ( 2018 ) . Bilevel optimization has been used in both machine learning , e.g . Bennett et al . ( 2006 ; 2008 ) and computer vision , e.g . Ochs et al . ( 2015 ) .

Summary: In this paper, the authors propose a SOSELETO (source selection for target optimization) framework to transfer learning and training with noisy labels. The intuition is some source instances are more informative than the others. Specifically, source instances are weighted and the weights are learned in a bilevel optimization scheme. Experimental studies on both training with noisy label problems and transfer learning problems demonstrate the effectiveness of the proposed SOSELETO.