Implicit Task-Driven Probability Discrepancy Measure for Unsupervised Domain Adaptation

1 Introduction . Discrepancy measures on two distributions underpin a large variety of machine learning tasks , and have been studied extensively since the dawn of modern probability [ 1 ] . For example , in generative models , such a measure is applied to align the generated distribution with the empirical one , and prevalent examples include 1 ) the f -divergence that admits a convenient variational form hence can be effectively evaluated via sample-based adversarial optimization [ 2 , 3 ] ; 2 ) integral probability metric [ IPM , 4 ] that seeks the largest discrepancy in function expectation over a reproducing kernel Hilbert space ( RKHS ) [ MMD GAN , 5–7 ] , 1-Lipschitz continuous functions [ Wasserstein GAN , 8 , 9 ] , or unit L2 norm functions [ Fisher GAN , 10 ] , etc . In domain adaptation [ DA , 11 , 12 ] , probability discrepancy is also the key construct in the feature adaptation approach , where a feature extractor φ is sought to align the source and target distributions transformed by φ [ 13 , 14 ] . The aforementioned measures can be applied directly in this context . It has been long noted that the discrepancy should be tailored to the function class of interest , e.g. , those for which we would like to compute expectations . This principle has been applied to density estimation [ 15 ] amongst others , where the RKHS is selected to match the downstream task such as image categorization based on the compressed pixel distribution . Naturally this motivation can be easily implemented in IPMs by customizing the generating function space . However such tailoring remains oblivious to the loss and available labels of the end task . Intuitively , if the latent features in DA are to be used for classification , then whether the loss is AUC or F-score should ideally influence the probability discrepancy . The seminalH∆H-divergence is designed for classification accuracy [ 16 ] , with a few extensions to Bayesian and other losses [ 17–20 ] . Despite being data-dependent , however , they are unsupervised without accounting for the available labels . Likewise , if a generative model is used to augment data so as to improve segmentation accuracy [ 21 ] , then the adversarial network in GANs should not only be able to distinguish between real and synthetic , but also “ align '' , in an appropriate sense , with the segmentation labels at hand . 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . Warping probability discrepancies towards a task has been lightly touched in unsupervised DA ( UDA ) . [ 22 ] trains two classifiers that not only boost the source-domain accuracy , but also maximally disagree on the target domain . Unfortunately , it is only formulated as a procedure , not a probability discrepancy . Most relevant to our work is the margin disparity discrepancy [ MDD , 23 ] , which is based on the H∆H-divergence where two fictitious classifiers h and h′ are jointly optimized to maximally reveal the two distributions ’ difference . [ 23 ] took the key insight that h can be tied with the source-domain predictor , and can thus be optimized to simultaneously reduce the source-domain risk and the H∆H-divergence . However , in spite of its effectiveness in both theory and practice , we discover that the specific formulation conflicts with the H∆H-divergence — the latter tries to maximize over h so as to promote the divergence , while MDD tries to minimize it ( Section 3 ) . This undermines the power of MDD in distinguishing two distributions as illustrated in Figure 1 . Flipping the sign and min/max can not resolve the issue . Our first contribution , therefore , is to develop a new task-driven discrepancy framework that overcomes this obstacle . The key inspiration is that MDD relies on the pseudo-label in the target domain ( i.e. , speculation of their labels based , e.g. , on the source-domain head ) , and this is also the case for some other measures such as the contrastive domain discrepancy [ CDD , 24 ] , which promotes the proximity between the class mean of the two domains for each class , and pushes apart the mean of different classes . Such a commonality motivates us to generate the target-domain pseudo-label based on the optimal source-domain classifier h∗ . In MDD , this provides a natural substitute for the fictitious classifier h ( Section 3.2 ) which no longer needs to be optimized over , thereby solving the aforementioned problem . As our second contribution , we extend this strategy to CDD in Section 4 . The overall formulation becomes a bi-level optimization solvable by implicit differentiation ( hence the modifier “ implicit ” in the method ’ s name ) . We note in passing that pseudo-label is commonly used in self-training for UDA [ 25–28 ] . However , most methods require various refinements of it in order to mitigate its inaccuracy due to distributional shift [ 29 ] . Examples include label sharpening [ 30 ] , entropy reweighting [ 31 ] , cycle training [ 29 ] . We instead directly use the output of f∗ as the pseudo-label in probability discrepancy , outperforming state of the art on a range of datasets ( Section 6 ) . UDA has recently received considerable interest , and most algorithms rely on ad-hoc heuristics ; we will mention a few below . Many of them require perusing the code and configuration script . As such , our main goal is not to develop yet another highly engineered model that performs better , but to present a principled formulation solvable by off-the-shelf optimizers . Although our implicit task-driven discrepancy can be straightforwardly applied to generative models , we deem it a better use of space to fully demonstrate its power in UDA . Such a probability discrepancy can also be easily extended to measure ( conditional ) independence , which has witnessed immediate application in fair and disentangled representation learning [ 32–35 ] . 2 Preliminaries . In UDA , there is a source domain and a target domain , and they are respectively represented as a joint distribution S and T on an input-output space X ×Y . We will denote their marginal distributions via subscripts , e.g. , Sx and Ty . The Y domain can be multiclass with labels [ C ] : = { 1 , 2 , . . . , C } . We are provided with labeled examples in the source domain , denoted as an empirical distribution S̃ . On the target domain , however , we can only access unlabeled examples , i.e. , an empirical distribution T̃x which only encompasses the input part of an empirical distribution T̃ . In short , let the empirical distributions consist of { xsi , ysi } ns i=1 and { xtj } nt j=1 for the source and target domains respectively . The goal of UDA is to find a classifier that predicts well on the target domain T . This is often referred to as inductive learning , while , in contrast , transductive learning is only concerned with the prediction on the empirical distribution T̃ , whose feature component T̃x is available at training time . The classification model , shared by both domains , consists of a feature extractor ( e.g. , ResNet ) parameterized by φ and a head hθ parameterized by θ . Letting ` be the loss over the ground-truth label y and the prediction hθ ( φ ( x ) ) , we seek the φ and θ that minimize the target-domain risk E ( x , y ) ∼T ` ( y , hθ ( φ ( x ) ) ) , or its empirical counterpart E ( x , y ) ∼T̃ ` ( y , hθ ( φ ( x ) ) ) . ( 1 ) In order to leverage the labeled data from the source domain and the unlabeled target-domain data , the feature adaption approach enforces low empirical risk on the source domain ( thanks to the availability of labels there ) and encourages that the source domain distribution , after being transformed by the feature extractor φ , “ aligns ” well with that of the target domain [ 13 , 14 , 36 , 37 ] . This is achieved by min φ , θ E ( x , y ) ∼S̃ ` ( y , hθ ( φ ( x ) ) ) + d ( φ # S̃x , φ # T̃x ) , ( 2 ) where φ # S̃x is the pushforward distribution of S̃x , and d denotes some discrepancy measure between two distributions . The intuition is that by “ mixing ” the latent distributions across the two domains through φ , the favorable accuracy of hθ on the source domain can be transferred to the target domain . For simplicity , we will denote P : = φ # Sx and P̃ : = φ # S̃x , and explicitize its dependency on φ by writing Pφ whenever necessary . With z = φ ( x ) , we can derive a conditional distribution of y given z based on S , and we denote it as Sy|z . Similarly , let Q : = φ # Tx , Q̃ : = φ # T̃x , and define Ty|z analogously . 3 Implicit Task-Driven Margin Disparity Discrepancy . There has been a plethora of research on sample-based discrepancy measure between two distributions . Examples include maximum mean discrepancy [ MMD , 38 ] , and ( neural ) variational optimization [ 39 ] which effectively subsumes a number of adversarial learning based measures [ 2 , 14 ] . However , these methods are oblivious to the subsequent tasks that are based on P and Q . For example , UDA can be aimed to classify well on these distributions . In domain-adversarial neural networks [ DANN , 14 ] , the discrepancy between P and Q is measured via the Jensen-Shannon divergence , reformulated in an adversarial objective as in the generative adversarial network [ GAN , 40 ] . Moreover , MMD simply measures the largest possible difference in the function expectation over P and Q : MMD ( P , Q ) : = sup f∈H : ‖f‖H≤1 [ E x∼P f ( x ) − E x∼Q f ( x ) ] = ∣∣∣∣∣∣ E x∼P k ( x , · ) − E x∼Q k ( x , · ) ∣∣∣∣∣∣ H , ( 3 ) whereH is the reproducing kernel Hilbert space ( RKHS ) induced by a kernel k. Obviously , it does not take into account whether f is used for classification or regression . The celebratedH∆H-divergence addresses this problem by focusing on binary classification [ 16 , Lemma 3 ] : dH∆H ( P , Q ) : = max h∈H max h′∈H D ( h , h′ , P , Q ) , ( 4 ) where D ( h , h′ , P , Q ) : = |EP [ [ sign ◦h′ 6= sign ◦h ] ] − EQ [ [ sign ◦h′ 6= sign ◦h ] ] | . ( 5 ) Here sign ◦h applies the sign function on the output of h. H is a hypothesis space ( not necessarily an RKHS ) , and [ [ · ] ] is the Iverson bracket that evaluates to 1 if · is true , and 0 otherwise . However , it still does not concern the label of the data ( i.e. , align only in an unsupervised fashion ) . To warp the measure to the end-task in a data-dependent fashion , [ 23 ] proposed the margin disparity discrepancy ( MDD ) , which improved upon [ 22 ] by formulating a principled objective function instead of a heuristic procedure . According to Equation 24 of [ 23 ] , MDD employs dMDD ( P , Q ) = min h∈H { R ( h ; P ) + max h′∈H D ( h , h′ , P , Q ) } , ( 6 ) where R ( h ; P ) : = Ez∼P , y|z∼Sy|z ` ( h ( z ) , y ) + reg ( h ) is the regularized risk , ( 7 ) and the 0-1 loss in D can be replaced by smooth surrogates such as hinge loss or cross-entropy loss . Here reg is any standard regularizer applied in regularized risk minimization , e.g. , ` 2 norm . The underlying insight is that when comparing P and Q , one only needs to consider those h that predict well on the ( labeled ) source domain , while leaving h′ to reveal the maximum discrepancy between P and Q . Similar ideas have been leveraged in [ 22 , 41 ] .

Summary: In this paper, the authors propose a new bi-level optimization-based approach to handle the issue of [23], i.e., the proposed margin disparity discrepancy (MDD) measure conflicts with the H\DeltaH-divergence. The theoretical results are then applied to MDD and CDD, which gives the implicit task-driven discrepancy method, i-MDD and i-CDD. Experimental studies are done to verify the effectiveness of the proposed i-MDD and i-CDD.