REVISTING NEGATIVE TRANSFER USING ADVERSARIAL LEARNING

1 INTRODUCTION . Advances in machine learning have led to proficient supervised learning models with powerful representations in various prediction tasks . We now expect an ideal classification model to restrict itself to a pertinent set of evidences available to it from the input for prediction . Further , we expect the model to disregard any unrelated evidences in the data to enable better generalization . Let us consider the task of training an animal classifier “ cheetah vs. snow-leopards ” from a dataset of images of these animals , such as those illustrated in Figure 1 – a task which ideally should focus on the animal ’ s appearance features . However , a large portion of these images also contain various cues of the typical habitat of the animals in the background , i.e. , tall grass and snow ( see Figures 1 ( a ) and ( b ) ) which are , in principle , unrelated to the animal ’ s appearance . An archetypal model is deceived by the co-occurrence of such unrelated , yet easily detectable cues of habitat over the animal ’ s appearance features such as complex fur patterns . However , a proficient supervised learning model must identify relevant evidences for the label of interest and at the same time discard various unrelated evidences such as presence of snow , even though it tends to co-occur frequently with snow-leopard . Consequently , it would be more likely that such a model would perform better on rare-instances ( such as those in Figures 1 ( c ) and ( d ) ) and generalize better to unseen instances . This phenomenon of co-occurring but unrelated evidences being present in training data and thereby having a debilitating effect on model performance has been described in literature ( Jayaraman et al . ( 2014 ) ; Zhou et al . ( 2013 ; 2010 ) ; Romera-Paredes et al . ( 2012 ) ; Yoon & Hwang ( 2017 ) ; Zhou et al . ( 2011 ) ) . These techniques utilize the easy availability of labels for unrelated evidences ( e.g . background habitat labels above ) , called negative labels which constitutes an auxiliary task , and seek to mitigate its debilitating performance on the primary task ( e.g . animal classification above ) with techniques referred to as negative-sharing or negative-transfer . While all of these techniques have tackled this problem utilizing various forms of regularization , we describe several shortcomings of this class of approaches , most notable of which is their inapplicability to the popular paradigm of trainable features obtained via neural representation learning . Motivated by these limitations , in this paper we depart from the direction of regularization-based approaches and examine methods inspired from a domain-adaptation viewpoint to propose an adversarial training-based formulation . We uniquely view such a scenario as an instance of adversarial multi-task learning , where the classification tasks are either the primary task of interest ( i.e. , predicting the presence of fur pattern and color ) or the auxiliary negative tasks ( i.e. , characteristics of habitat ) to be avoided . Since the 2 tasks are unrelated , any label correlation between primary and auxiliary labels in the training data is only by chance and therefore from a domain-adaptation perspective , we envision a target-domain as possibly having a different correlation between the primary and auxiliary labels . The effects of negative transfer are hence mitigated when the classification task is trained in this domain . We discuss advantages of our proposed formulation , inspired from domain-adaptation , to alleviate the negative transfer over existing techniques , including ready applicability to neural networks in an end-to-end fashion . It must be noted that , while the formulation of the problem is motivated with multi-task learning , negative-transfer is a disposition of any supervised learning task from simple binary classification to recent popular supervised tasks such as image detection , captioning , or visual dialog . We present motivating literature that prelude this work next . 2 RELATED WORK ON TACKLING NEGATIVE SHARING . Image classification literature often characterize mid-level features as attribute predictors in multilabel learning . The inability of models to learn predictors with the correct semantic concept is attributed to negative transfer . To our knowledge , the predominent approach to tackle negative transfer in such setting was the use of specific regularizers ( Romera-Paredes et al. , 2012 ; Jayaraman et al. , 2014 ) . Specifically , the primary model avoid using features which are important for the auxiliary task , leading to models competing for features . Jayaraman et al . ( 2014 ) further extends this idea to attribute groups , where feature selection sparsity is induced across group , but encourage within them . We highlight three limitations of feature competition techniques below : • Repeated features : Consider the simple scenario where some features in the feature representations are repeated or dependent on others . Feature competition enforces tasks to pick unique features , however they are implicitly the same . Going back to the case of ‘ cheetah vs snow leopard ’ example mentioned earlier , when there are two copies of a feature which captures ‘ snow in the background ’ , then both primary and auxiliary classifiers would just pick different copies of that feature . Here , the idea of resisting cheetah/snow leopard classifier from picking features which capture snow is negated . • Trainable features : Neural representations have become extremely popular owing the power of learning trainable features . However , feature competition techniques fail with prediction models with trainable features . If there are features that are important for both primary and auxiliary task , models involving a trainable feature setup will lead to duplicating the same features , thereby resulting in the repeated feature scenario . • Easy auxiliary task with abundant features : Consider a scenario of an easy auxiliary task , that does not require a large number of features to be predicted correctly . Similar to the previous case , the spared features from the auxiliary task can be picked by the primary task resulting in negative transfer . Motivated by these shortcomings , we proceed to examine the negative transfer problem in a domain adaptation setting . 3 PROPOSED FRAMEWORK . In this section we first formalize and explain the problem setting of negative transfer . We present scenarios where earlier attempts ( regularization-based approaches ) to tackle this problem fails . We then explain our formulation for posing then negative transfer problem in a domain adaptation setting and derive adversarial learning algorithms to solve negative transfer . Lastly , we present a thorough analysis of the proposed formulation to address negative transfer by experimenting on a carefully designed synthetic dataset . 3.1 PROBLEM SETUP . In typical 2-label classification problem , we assume that training data and all future test examples are drawn i.i.d from a probability distribution D on instance space X and a labeling function fp : X → { +1 , −1 } for the primary task labels and fa : X → { +1 , −1 } for the auxiliary task labels . Every instance x ∈ X has primary and auxiliary labels : yp and ya respectively . The goal is to learn a classifier which performs well on future samples from D , which may have a different label correlation since the tasks are unrelated . Formally , we capture this label correlation via a joint label distribution P ( Y ) = P ( Ya , Yb ) , and we assume that P ( Y ) in training and test are different . This problem setup is different from earlier works on negative transfer in the way how unrelated tasks are defined . We define unrelated tasks as the ones which can have different correlation in training and test data . In ( Romera-Paredes et al. , 2012 ) unrelated tasks is referred as tasks which are defined over orthogonal sets of features . Zhou et al . ( 2010 ) uses the term negative correlation for unrelatedness among tasks . Two tasks are negatively correlated when one feature is deemed to be important to first task makes it unlikely to be important to the second task . 3.2 NEGATIVE TRANSFER AS A DOMAIN ADAPTATION PROBLEM . Let the instances in training data be drawn from source distribution DS . The aim is to train a classifier on this which performs well on instances drawn from target distribution DT . Instances in DS has primary and auxiliary labels correlated with joint label distribution PS ( Y ) = PS ( Yp , Ya ) whereas in DT the labels have a different correlation with different joint label distribution PT ( Y ) = PT ( Yp , Ya ) . We assume that the labelling function fp and fa are universal and common for both source and target domains . Typical unsupervised domain adaptation setting has labelled instances from source distribution and also unlabelled instances from target distribution . However , we have no information ( neither labels nor instances ) about the target distribution . The only information we have about the target domain is PT ( Y ) . We are either provided an estimate of PT ( Y ) from an oracle , or we make an assumption on PT ( Y ) ( for instance , a uniform distribution over the space of Y ) . Consider UT , the space of all distributions over X which has label distribution PT ( Y ) . It is extremely challenging to adapt from the source domain to all members of UT with given PT ( Y ) . However , we can settle for a particular DT . We pick such a DT ∈ UT as that distribution over X which is nearest to DS . We model this in terms of KL divergence as a constrained optimization as follows . D ? T = arg min D∈UT KL ( DS ||D ) ( 1 ) Before explaining the solution to the above optimization problem , we show the relationship between D and P ( Y ) . To articulate our intuition , we use Figure 3 ( a ) which illustrates a sample source distribution DS over 2D instance space ( green ) . The labelling functions ( in this case , fp and fa ) partitions the space of X into regions such that each region corresponds to a label combination ( y = { yp , ya } ) . We denote each of these regions as Ry . Notice that P ( Y = y ) becomes the integral of D over the region Ry . Let φS and φT be density functions of DS and DT respectively . Then , the above optimization problem becomes : φT = arg min φ∈∆ ∫ φS ( x ) log φS ( x ) φ ( x ) dx ( 2 ) s.t . ∫ Ry φ ( x ) dx = PT ( y ) ∀ y Where ∆ ⊃ UT is the set of all distributions overX and regionRy = { x : fp ( x ) = yp , fa ( x ) = ya } . The Lagrangian equivalent for the above problem can be stated as , L ( φ ) = ∫ φS ( x ) log φS ( x ) φ ( x ) dx+ ∑ y λy ∫ Ry φ ( x ) dx , where λy ∈ R. Using the Euler-Lagrange equation to solve above problem , φT ( x ) = φS ( x ) × PT ( y ) PS ( y ) ( 3 ) Intuitively we find that D ? T is a scaled version of DS with scaling factor as the ratio of PT ( Y ) and PS ( Y ) . Note that this scaling factor may vary for different x ∈ X depending on which region Ry it falls in . This is depicted in Figure . 3 ( b ) which is a target distribution derived from source distribution in 3 ( a ) . The regions R−1 , +1 and R+1 , −1 are scaled up whereas regions R+1 , +1 and R−1 , −1 are scaled down . Though the above derivation is for two labels , one can see that Eq . 3 extends to any number of labels in y . As mentioned earlier in this section , we have no instances from target domain . However φT ( x ) allows us to assign soft domain labels to the source domain instances , which indicates the probability of that instance belonging to the target domain . Specifically , let yD be the binary label indicating if an instance belongs to target domain . Then , P ( yD|x ) = φT ( x ) φT ( x ) + φS ( x ) = PT ( y ) PT ( y ) + PS ( y ) . ( 4 ) Two possible assumptions that could be made on PT ( Y ) ( if not provided ) are uniform distribution over space of y or uncorrelated ( independent labels ) . Next we present two methods that leverage these soft domain labels .

Summary: - The authors study the problem of negative transfer in representation learning, and propose to use the formulation proposed by Ganin & Lempitsky '15 for domain adaptation to reduce negative transfer. Instead of defining the domain classification as the adversarial task to learn a domain-independent representation, they collect a set of classification problems irrelevant to the main task as the adversarial tasks, and aim to learn a representation that focuses only on the primary task. There are very little changes compared to the proposal by Ganin & Lempitsky '15, but the application to solve the problem of negative transfer is interesting.