Discrepancy-Based Active Learning for Domain Adaptation

1 INTRODUCTION . Machine learning models trained on a labeled data set from a source domain may fail to generalize on new target domains of interest ( Saenko et al. , 2010 ) . This issue , which can be caused by domain shift , can be handled when no target labels are available through unsupervised domain adaptation methods ( Ganin et al. , 2016 ) . Using a small sample of labeled target data can , besides , greatly improve the model performances ( Motiian et al. , 2017 ) . Acquiring such new labels is often expensive ( Settles , 2010 ) and one seeks to query as few labels as possible . This explains why strategies of optimal labels acquisition , referred as active learning ( Cohn et al. , 1994 ) seem very promising for domain adaptation ( Su et al. , 2020 ) . Active learning is a challenging task and a broad literature exists . On the one hand , some active learning methods introduce heuristic approaches which provide the benefit of using practical algorithm based on simple criteria . For instance , the spatial coverage of the target domain ( Hu et al. , 2010 ; Bodó et al. , 2011 ) or the minimization of target uncertainties ( RayChaudhuri & Hamey , 1995 ; Gal et al. , 2017 ) are considered , as well as combination of these heuristics ( Wei et al. , 2015 ; Kaushal et al. , 2019 ) . However finding the proper heuristics is not straightforward and previous methods do not link their query strategy with the target risk ( Viering et al. , 2019 ) . On the other hand , active learning methods based on distribution matching aim at minimizing a distribution distance between the labeled set and the unlabeled one ( Balcan et al. , 2009 ; Wang & Ye , 2015 ; Viering et al. , 2019 ) . These methods provide theoretical guarantees on the target risk through generalization bounds . However the computation of the distances is either not scalable to large-scale data sets ( Balcan et al. , 2009 ; Wang & Ye , 2015 ; Viering et al. , 2019 ) or based on adversarial training ( Su et al. , 2020 ; Shui et al. , 2020 ) which involves complex hyper-parameter calibration ( Kurach et al. , 2019 ) . In this work , we propose to address the issue of active learning for general loss functions under domain shift through a distribution matching approach based on discrepancy minimization ( Mansour et al. , 2009 ) . In Section 2 , we derive theoretical results by adopting a localized discrepancy distance ( Zhang et al. , 2020 ) between the labeled and unlabeled empirical distributions . This localized discrepancy is defined as the plain discrepancy considered on a hypothesis space restricted to hypotheses close to the labeling function on the labeled data set . This distance has the benefit to focus only on relevant ∗Correspondence at antoine.de-mathelin-de-papigny @ michelin.com candidates for approximating the labeling function and thus provides tighter bound of the target risk under some given assumptions ( Zhang et al. , 2020 ) . Based on this distance , we provide a generalization bound of the target risk involving pairwise distances between sample points ( Theorem 1 ) . Inspired by this generalization error bound , we propose in Section 3 an accelerated K-medoids query algorithm which scales to large data set . In Section 4 , we present the related works and analytically show that our proposed approach displays tighter theoretical control of the target risk than the one provided by recent active learning methods . We finally present in Section 5 the benefit of the proposed approach on several empirical regression and classification active learning problems in the context of domain adaptation . 2 DISCREPANCY BASED ACTIVE LEARNING . Setup and theory develop in this section mainly focus on regression tasks . Section 4.3 presents how the algorithm derived from the theoretical results can be extended to classification tasks . 2.1 SETUP AND DEFINITIONS . Given two subsets X ⊂ Rp and Y ⊂ Rq and d : X ×X → R+ a distance on X , we denote the source data set S = { x1 , ... , xm } ∈ Xm and the target data set T = { x′1 , ... , x′n } ∈ Xn . We consider the domain shift setting where the respective data sets S and T are drawn according to two different distributions Q and P on X . We consider a loss function L : Y × Y → R+ and a hypothesis space H of k-Lipschitz functions from X to Y . We denote by LD ( h , h′ ) = Ex∼D [ L ( h ( x ) , h′ ( x ) ) ] the average loss ( or risk ) over any distribution D on X between two hypotheses h , h′ ∈ H . We also define the expected Rademacher complexity of H for the distribution P as : Rn ( H ) = E { x′i } i∼P [ E { σi } i∼U [ sup h∈H 1 n n∑ i=1 σih ( x ′ i ) ] ] , with σi drawn according to U the uniform distribution on { −1 , 1 } . We consider a labeling function for each distribution : fQ : X → Y and fP : X → Y . For adaptation to be possible , the two labeling functions are supposed to be close ( Mansour et al. , 2009 ) . We finally consider the single-shot batch active learning framework ( Viering et al. , 2019 ) where all queried data are picked in one single batch of fixed budget of K > 0 queries . In this framework , an active learning algorithm takes as inputs the source data set S along with its corresponding recorded labels as well as the target data set T . The algorithm then returns a batch of K queried target data denoted TK ⊂ T . The corresponding labels for TK are then recorded through an oracle and used along with the source labeled data to fit an hypothesis h ∈ H . We denote by LK = S ∪ TK the labeled data set . The goal is to select the K target data to label in order to minimize the target risk of h : LP ( h , fP ) . 2.2 LOCALIZED DISCREPANCY . To formulate the problem of active learning under domain shift as a distribution matching problem , one needs to consider a measure of divergence between distributions . Recent interest focuses on the discrepancy ( Mansour et al. , 2009 ) which proves to be useful for domain adaptation ( Cortes & Mohri , 2014 ; Zhang et al. , 2019 ) and is recently used on the active learning setting ( Viering et al. , 2019 ) . However this metric , defined as a maximal difference between domain losses over the whole hypothesis space , is relatively conservative as it includes hypotheses that the learner might not ever consider as candidates for the labeling function ( Cortes et al. , 2019c ; Zhang et al. , 2020 ) . Based on this consideration , we introduce a localized discrepancy ( Zhang et al. , 2020 ) to restrict the measure of divergence between domains on a set of relevant candidate hypotheses for approximating the labeling function : Definition 1 . Localized Discrepancy . Let K > 0 be the number of queries and ϵ ≥ 0 . Let TK be a queried batch of size K , the empirical distributions of LK = S ∪TK and T are respectively denoted Q̂K and P̂ . Let H be a hypothesis space and L a loss function . The localized discrepancy is defined as : discHKϵ ( Q̂K , P̂ ) = maxh , h′∈HKϵ |LQ̂K ( h , h ′ ) − LP̂ ( h , h ′ ) | , ( 1 ) with HKϵ = { h ∈ H ; L ( h ( x ) , fQ ( x ) ) ≤ ϵ ∀x ∈ LK } The localized space HKϵ includes hypotheses ” consistent ” with fQ on the labeled data , i.e . hypotheses fitting the labeled data with an error below ϵ . The parameter ϵ drives the size of HKϵ . Obviously , ϵ needs to be high enough to ensure that HKϵ is not an empty subset . Under appropriate assumptions on ϵ , a preliminary result is an empirical target risk bound for the localized discrepancy : Proposition 1 . Let K > 0 be the number of queries and H a hypothesis space . Let P̂ and Q̂K be the empirical distributions of the respective sets T and LK = S ∪ TK of respective size n and m+K . We assume that L is a symmetric , µ-Lipschitz and bounded loss function verifying the triangular inequality . We denote by M the bound of L. Let ηH be the ideal maximal error on S ∪ T : ηH ∆ = min h∈H max x∈S∪T [ L ( h ( x ) , fQ ( x ) ) + L ( h ( x ) , fP ( x ) ) ] ( 2 ) Then , for any ϵ ≥ ηH , any hypothesis h ∈ HKϵ and any δ > 0 , the following generalization bound holds with at least probability 1-δ : LP ( h , fP ) ≤ LQ̂K ( h , fQ ) + discHKϵ ( Q̂K , P̂ ) + ηH + 2µRn ( H ) +M √ log ( 1δ ) 2n . ( 3 ) This bound share a similar form than the one derived in Cortes et al . ( 2019c ) for the discrepancy . The ideal maximal error ηH characterizes the difficulty of the adaptation problem as well as the ability to learn the labeling functions with the considered hypothesis set H . If fP and fQ significantly differ on S ∪ T , ηH will be high and the proposed bound will lack of informativeness . We then follow the common domain adaptation assumption that the difference between the two functions is small ( Mansour et al. , 2009 ; Zhang et al. , 2020 ) . 2.3 MAIN RESULTS : GENERALIZATION BOUNDS FOR ACTIVE LEARNING . Considering the previous bound ( Proposition 1 ) it appears that a natural way of choosing the K queries in an active learning perspective is to pick the target data minimizing the localized discrepancy . Unfortunately this is a difficult problem for an arbitrary functional space H , since it leads to compute a maximum over the set space HKϵ . Our main idea is then to further bound the localized discrepancy with a computable criterion : Theorem 1 . Let K > 0 be the number of queries , H a hypothesis space of k-Lipschitz functions and ϵ ≥ ηH . Let LK = S ∪TK be the labeled set and T the target set drawn according to P . We assume that L is a symmetric , µ-Lipschitz and bounded loss function verifying the triangular inequality . We define M such that L ( y , y′ ) ≤M for any y , y′ ∈ Y . For any hypothesis h ∈ HKϵ and any δ > 0 , the following generalization bound holds with at least probability 1-δ : LP ( h , fP ) ≤ LQ̂K ( h , fQ ) + 2kµ n ∑ x′∈T d ( x′ , LK ) + 2ϵ+ ηH + 2µRn ( H ) + √ M2 log ( 1δ ) 2n ( 4 ) With d ( x′ , LK ) = minx∈LK d ( x ′ , x ) and d ( x′ , x ) the distance from x to x′ . Visual insights to understand Theorem 1 are presented in Figure 1 : the main idea is to approximate the maximal hypotheses h , h′ ∈ HKϵ returning the localized discrepancy by the k-Lipschitz envelope of the labeling function fQ , consistent with fQ on the labeled points , i.e . at most ϵ close to fQ on these points . Indeed , for any target point x′ ∈ T , the gap between h , h′ on x′ : L ( h′ ( x′ ) , h ( x′ ) ) is upper bounded by twice the distance from x′ to its closest labeled point times the Lipschitz constants of L , h and h′ plus the error on the labeled point : L ( h′ ( x′ ) , h ( x′ ) ) ≤ 2kµ d ( x′ , LK ) + 2ϵ . The generalization bound of Theorem 1 highlights the trade-off that exists between the Lipschitz constant k of the hypothesis space H and the parameters ηH and ϵ . To reduce the term 2ϵ + ηH in order to obtain tighter controls over the target risk , one needs to consider a more complex set of hypothesis and thus to increase the Lipschitz constant k. Then , to benefit from the theoretical guarantees of this bound , a careful choice of hypothesis set have to be made ( cf Section 4.3 ) . The important benefit of the derived bound is to bring out the bounding criterion ∑ x′∈T d ( x ′ , LK ) which is independent of the hypothesis complexity ( characterized by k ) and the loss function , it only involves pairwise distances between sample points . As this criterion is computable and depends on the queried batch TK , we can then propose an active learning strategy .

Summary: This paper proposes an active learning method to efficiently select data in the target domain that is suitable for learning hypotheses together with data in the source domain in domain adaptation problems.   The authors first propose to evaluate the dissimilarity between the source and target domains by localized discrepancy, where the hypothesis set is restricted to includes only hypothesithes with sufficiently small errors with the label function of the source domain on the training data (the union set of the input of the source domain and the input of the target domain to be labeled). Then an upper bound of the expected risk for the target domain based on this discrepancy. The main theoretical contribution is that the authors have shown that the discrepancy term in this upper bound is further bounded from above by the sum of the distances between the inputs of the training data and the inputs of the target domain, and this is the background for the design of the active learning algorithm.   In domain adaptation, the goal is to solve the problem of minimizing the discrepancy between the two domains. From the statements of this theorem, it can be replaced by the problem of selecting K data in the target domain so as to minimize the sum of the distances between the inputs of the training data and the inputs of the target domain. The latter problem is equivalent to solving the K-medoids problem for clustering, and the authors' technical contribution is that they have proposed an accelerated algorithm for this problem.   Finally, the authors conduct an evaluation experiment of the proposed method under various scenarios of domain shift using three types of benchmark data.