Regularized Learning for  Domain Adaptation under Label Shifts

1 INTRODUCTION . When machine learning models are employed “ in the wild ” , the distribution of the data of interest ( target distribution ) can be significantly shifted compared to the distribution of the data on which the model was trained ( source distribution ) . In many cases , the publicly available large-scale datasets with which the models are trained do not represent and reflect the statistics of a particular dataset of interest . This is for example relevant in managed services on cloud providers used by clients in different domains and regions , or medical diagnostic tools trained on data collected in a small number of hospitals and deployed on previously unobserved populations and time frames . Covariate Shift Label Shift p ( x ) 6= q ( x ) p ( y|x ) = q ( y|x ) p ( y ) 6= q ( y ) p ( x|y ) = q ( x|y ) There are various ways to approach distribution shifts between a source data distribution P and a target data distribution Q . If we denote input variables as x and output variables as y , we consider the two following settings : ( i ) Covariate shift , which assumes that the conditional output distribution is invariant : p ( y|x ) = q ( y|x ) between source and target distributions , but the source distribution p ( x ) changes . ( ii ) Label shift , where the conditional input distribution is invariant : p ( x|y ) = q ( x|y ) and p ( y ) changes from source to target . In the following , we assume that both input and output variables are observed in the source distribution whereas only input variables are available from the target distribution . While covariate shift has been the focus of the literature on distribution shifts to date , label-shift scenarios appear in a variety of practical machine learning problems and warrant a separate discussion as well . In one setting , suppliers of machine-learning models such as cloud providers have large resources of diverse data sets ( source set ) to train the models , while during deployment , they have no control over the proportion of label categories . In another setting of e.g . medical diagnostics , the disease distribution changes over locations and time . Consider the task of diagnosing a disease in a country with bad infrastructure and little data , based on reported symptoms . Can we use data from a different location with data abundance to diagnose the disease in the new target location in an efficient way ? How many labeled source and unlabeled target data samples do we need to obtain good performance on the target data ? Apart from being relevant in practice , label shift is a computationally more tractable scenario than covariate shift which can be mitigated . The reason is that the outputs y typically have a much lower dimension than the inputs x. Labels are usually either categorical variables with a finite number of categories or have simple well-defined structures . Despite being an intuitively natural scenario in many real-world application , even this simplified model has only been scarcely studied in the literature . Zhang et al . ( 2013 ) proposed a kernel mean matching method for label shift which is not computationally feasible for large-scale data . The approach in Lipton et al . ( 2018 ) is based on importance weights that are estimated using the confusion matrix ( also used in the procedures of Saerens et al . ( 2002 ) ; McLachlan ( 2004 ) ) and demonstrate promising performance on large-scale data . Using a black-box classifier which can be biased , uncalibrated and inaccurate , they first estimate importance weights q ( y ) /p ( y ) for the source samples and train a classifier on the weighted data . In the following we refer to the procedure as black box shift learning ( BBSL ) which the authors proved to be effective for large enough sample sizes . However , there are three relevant questions which remain unanswered by their work : How to estimate the importance weights in low sample setting , What are the generalization guarantees for the final predictor which uses the weighted samples ? How do we deal with the uncertainty of the weight estimation when only few samples are available ? This paper aims to fill the gap in terms of both theoretical understanding and practical methods for the label shift setting and thereby move a step closer towards having a more complete understanding on the general topic of supervised learning for distributionally shifted data . In particular , our goal is to find an efficient method which is applicable to large-scale data and to establish generalization guarantees . Our contribution in this work is trifold . Firstly , we propose an efficient weight estimator for which we can obtain good statistical guarantees without a requirement on the problem-dependent minimum sample complexity as necessary for BBSL . In the BBSL case , the estimation error can become arbitrarily large for small sample sizes . Secondly , we propose a novel regularization method to compensate for the high estimation error of the importance weights in low target sample settings . It explicitly controls the influence of our weight estimates when the target sample size is low ( in the following referred to as the low sample regime ) . Finally , we derive a dimension-independent generalization bound for the final Regularized Learning under Label Shift ( RLLS ) classifier based on our weight estimator . In particular , our method improves the weight estimation error and excess risk of the classifier on reweighted samples by a factor of k log ( k ) , where k is the number of classes , i.e . the cardinality of Y . In order to demonstrate the benefit of the proposed method for practical situations , we empirically study the performance of RLLS and show weight estimation as well as prediction accuracy comparison for a variety of shifts , sample sizes and regularization parameters on the CIFAR-10 and MNIST datasets . For large target sample sizes and large shifts , when applying the regularized weights fully , we achieve an order of magnitude smaller weight estimation error than baseline methods and enjoy at most 20 % higher accuracy and F-1 score in corresponding predictive tasks . For low target sample sizes , applying regularized weights partially also yields an accuracy improvement of at least 10 % over fully weighted and unweighted methods . 2 REGULARIZED LEARNING OF LABEL SHIFTS ( RLLS ) . Formally let us the short hand for the marginal probability mass functions of Y on finite Y with respect to P , Q as p , q : [ k ] → [ 0 , 1 ] with p ( i ) = P ( Y = i ) , and q ( i ) = Q ( Y = i ) for all i ∈ [ k ] , representable by vectors in Rk+ which sum to one . In the label shift setting , we define the importance weight vector w ∈ Rk between these two domains as w ( i ) = q ( i ) p ( i ) . We quantify the shift using the exponent of the infinite and second order Renyi divergence as follows d∞ ( q||p ) : = sup i q ( i ) p ( i ) , and d ( q||p ) : = EY∼Q [ w ( Y ) 2 ] = k∑ i q ( i ) q ( i ) p ( i ) . Given a hypothesis classH and a loss function ` : Y × Y → [ 0 , 1 ] , our aim is to find the hypothesis h ∈ H which minimizes L ( h ) = EX , Y∼Q [ ` ( Y , h ( X ) ) ] = EX , Y∼P [ w ( Y ) ` ( Y , h ( X ) ) ] In the usual finite sample setting however , L unknown and we observe samples { ( xj , yj ) } nj=1 from P instead . If we are given the vector of importance weights w we could then minimize the empirical loss with importance weighted samples defined as Ln ( h ) = 1 n n∑ j=1 w ( yj ) ` ( yj , h ( xj ) ) where n is the number of available observations drawn from P used to learn the classifier h. As w is unknown in practice , we have to find the minimizer of the empirical loss with estimated importance weights Ln ( h ; ŵ ) = 1 n n∑ j=1 ŵ ( yj ) ` ( yj , h ( xj ) ) ( 1 ) where ŵ are estimates of w. Given a set Dp of np samples from the source distribution P , we first divide it into two sets where we use ( 1 − β ) np samples in set Dweightp to compute the estimate ŵ and the remaining n = βnp in the set Dclassp to find the classifier which minimizes the loss ( 1 ) , i.e . ĥŵ = arg min h∈H Ln ( h ; ŵ ) . In the following , we describe how to estimate the weights ŵ and provide guarantees for the resulting estimator ĥŵ . Plug-in weight estimation The following simple correlation between the label distributions p , q was noted in Lipton et al . ( 2018 ) : for a fixed hypothesis h , if for all y ∈ Y it holds that q ( y ) ≥ 0 =⇒ p ( y ) ≥ 0 , we have qh ( i ) : = Q ( h ( X ) = i ) = k∑ j=1 Q ( h ( X ) = i|Y = j ) q ( j ) = k∑ j=1 P ( h ( X ) = i|Y = j ) q ( j ) = k∑ j=1 P ( h ( X ) = i , Y = j ) q ( j ) p ( j ) = k∑ j=1 P ( h ( X ) = i , Y = j ) wj for all i , j ∈ Y . This can equivalently be written in matrix vector notation as qh = Chw , ( 2 ) where Ch is the confusion matrix with [ Ch ] i , j = P ( h ( X ) = i , Y = j ) and qh is the vector which represents the probability mass function of h ( X ) under distribution Q . The requirement q ( y ) ≥ 0 =⇒ p ( y ) ≥ 0 is a reasonable condition since without any prior knowledge , there is no way to properly reason about a class in the target domain that is not represented in the source domain . In reality , both qh and Ch can only be estimated by the corresponding finite sample averages q̂h , Ĉh . Lipton et al . ( 2018 ) simply compute the inverse of the estimated confusion matrix Ĉh in order to estimate the importance weight , i.e . ŵ = Ĉ−1h q̂h . While C −1 h q̂h is a statistically efficient estimator , ŵ with estimated Ĉ−1h can be arbitrarily bad since Ĉ −1 h can be arbitrary close to a singular matrix especially for small sample sizes and small minimum singular value of the confusion matrix . Intuitively , when there are very few samples , the weight estimation will have high variance in which case it might be better to avoid importance weighting altogether . Furthermore , even when the sample complexity in Lipton et al . ( 2018 ) , unknown in practice , is met , the resulting error of this estimator is linear in k which is problematic for large k. We therefore aim to address these shortcomings by proposing the following two-step procedure to compute importance weights . In the case of no shift we have w = 1 so that we define the amount of weight shift as θ = w − 1 . Given a “ decent ” black box estimator which we denote by h0 , we make the final classifier less sensitive to the estimation performance of C ( i.e . regularize the weight estimate ) by 1. calculating the measurement error adjusted θ̂ ( described in Section 2.1 for h0 ) and 2. computing the regularized weight ŵ = 1+λθ̂ where λ depends on the sample size ( 1−β ) np . By `` decent '' we refer to a classifier h0 which yields a full rank confusion matrix Ch0 . A trivial example for a non- ” decent ” classifier h0 is one that always outputs a fixed class . As it does not capture any characteristics of the data , there is no hope to gain any statistical information without any prior information .

Summary: - The authors consider the problem of learning under label shifts, where the label proportions p(y) and q(y) of the training and test distributions differ, while the conditionals p(x|y) and q(x|y) are equal. They build upon the work by Lipton et al. 18 on estimating label proportion weights q(y)/p(y) using the confusion matrix, by proposing an improved estimator with regularization. They show that their estimator provides better weight estimates compared to the unregularized version, and it also gives better prediction accuracies under large label shift scenarios.