Towards Stable and comprehensive Domain Alignment: Max-Margin Domain-Adversarial Training

1 INTRODUCTION . Deep neural networks have gained great success on a wide range of tasks such as visual recognition and machine translation ( LeCun et al. , 2015 ) . They usually require a large number of labeled data that can be prohibitively expensive to collect , and even with sufficient supervision their performance can still be poor when being generalized to a new environment . The problem of discrepancy between the training and testing data distribution is commonly referred to as domain shift ( Shimodaira , 2000 ) . To alleviate the effect of such shift , domain adaptation sets out to obtain a model trained in a label-rich source domain to generalize well in an unlabeled target domain . Domain adaptation has benefited various applications in many practical scenarios , including but not limited to object detection under challenging conditions ( Chen et al. , 2018 ) , cost-effective learning using only synthetic data to generalize to real-world imagery ( Vazquez et al. , 2013 ) , etc . Prevailing methods for unsupervised domain adaptation ( UDA ) are mostly based on domain alignment which aims to learn domain-invariant features by reducing the distribution discrepancy between the source and target domain using some pre-defined metrics such as maximum mean discrepancy ( Tzeng et al. , 2014 ) . Recently , Ganin & Lempitsky ( 2015 ) proposed to achieve domain alignment by domainadversarial training ( DAT ) that reverses the gradients of a domain classifier to maximize domain confusion . Having yielded remarkable performance gain , DAT was employed in many subsequent UDA methods ( Long et al. , 2018 ; Shu et al. , 2018 ) . Even so , there still exist three critical issues of DAT that hinder its performance : ( 1 ) as the domain classifier has high-capacity to discriminate two domains , the unbalanced adversarial training can not continuously provide effective gradients , which is usually overcome by manually adjusting the weights of adversarial training according to specific tasks ; ( 2 ) DAT-based methods can not deal with pixel-level domain shift ( Hoffman et al. , 2018 ) ; ( 3 ) the domain-invariant features learned by DAT are only based on intuition but difficult to interpret , which impedes the investigation of the underlying mechanism of adversarial domain adaptation . To overcome the aforementioned difficulties , we propose an innovative DAT approach , namely Max-margin Domain-Adversarial Training ( MDAT ) , to realize stable and comprehensive domain alignment . To demonstrate its effectiveness , we develop an Adversarial Reconstruction Network ( ARN ) that only utilizes MDAT for UDA . Specifically , ARN consists of a shared feature extractor , a label predictor , and a reconstruction network ( i.e . decoder ) that serves as a domain classifier . Supervised learning is conducted on source domain , and MDAT helps learn domain-invariant features . In MDAT , the decoder only focuses on reconstructing samples on source domain and pushing the target domain away from a margin , while the feature extractor aims to fool the decoder by learning to reconstruct samples on target domain . In this way , three critical issues can be solved by MDAT : ( 1 ) the max-margin loss reduces the discriminative capacity of domain classifier , leading to balanced and thus stable adversarial training ; ( 2 ) without involving new network structures , MDAT achieves both pixel-level and feature-level domain alignment ; ( 3 ) visualizing the reconstructed samples reveals how the source and target domains are aligned . We evaluate ARN with MDAT on five visual and non-visual UDA benchmarks . It achieves significant improvement to DAT on all tasks with pixel-level or higher-level domain shift . We also observe that it is insensitive to the choices of hyperparameters and as such is favorable for replication in practice . In principle , our approach is generic and can be used to enhance any UDA methods that leverage domain alignment as an ingredient . 2 RELATED WORK . Domain adaptation aims to transfer knowledge from one domain to another . Ben-David et al . ( 2010 ) provide an upper bound of the test error on the target domain in terms of the source error and the H4H-distance . As the source error is stationary for a fixed model , the goal of most UDA methods is to minimize theH4H-distance by reducing some metrics such as Maximum Mean Discrepancy ( MMD ) ( Tzeng et al. , 2014 ; Long et al. , 2015 ) and CORAL ( Sun & Saenko , 2016 ) . Inspired by Generative Adversarial Networks ( GAN ) ( Goodfellow et al. , 2014 ) , Ganin & Lempitsky ( 2015 ) proposed to learn domain-invariant features by adversarial training , which has inspired many UDA methods thereafter . Adversarial Discriminative Domain Adaptation ( ADDA ) tried to fool the label classifier by adversarial training but not in an end-to-end manner . CyCADA ( Hoffman et al. , 2018 ) and PixelDA ( Bousmalis et al. , 2017 ) leveraged GAN to conduct both feature-level and pixel-level domain adaptation , which yields significant improvement yet the network complexity is high . Another line of approaches that are relevant to our method is the reconstruction network ( i.e . the decoder network ) . The success of image-to-image translation corroborates that it helps learn pixellevel features in an unsupervised manner . In UDA , Ghifary et al . ( 2016 ) employed a decoder network for pixel-level adaptation , and Domain Separate Network ( DSN ) ( Bousmalis et al. , 2016 ) further leveraged multiple reconstruction networks to learn domain-specific features . These approaches treat the decoder network as an independent component that is irrelevant to domain alignment ( Glorot et al. , 2011 ) . In this paper , our approach proposes to utilize the decoder network as domain classifier in MDAT which enables both feature-level and pixel-level domain alignment in a stable and straightforward fashion . 3 PROBLEM FORMULATION . 3.1 PROBLEM DEFINITION AND NOTATIONS . In unsupervised domain adaptation , we assume that the model works with a labeled dataset XS and an unlabeled dataset XT . Let XS = { ( xsi , ysi ) } i∈ [ Ns ] denote the labeled dataset of Ns samples from the source domain , and the certain label ysi belongs to the label space Y that is a finite set ( Y = 1 , 2 , ... , K ) . The other dataset XT = { xti } i∈ [ Nt ] has Nt samples from the target domain but has no labels . We further assume that two domains have different distributions , i.e . xsi ∼ DS and xti ∼ DT . In other words , there exist some domain shift ( Ben-David et al. , 2010 ) between DS and DT . The ultimate goal is to learn a model that can predict the label yti given the target input xti . 3.2 IMBALANCED MINIMAX GAME IN DOMAIN-ADVERSARIAL TRAINING . To achieve domain alignment , Domain-Adversarial Training ( DAT ) is a minimax game between a shared feature extractor F for two domains and a domain classifier D. The domain classifier is trained to determine whether the input sample belongs to the source or the target domain while the feature extractor learns to deceive the domain classifier , which is formulated as : min F max D LDAT ( Ds , Dt ) = Ex∼Ds [ lnF ( x ) ] + Ex∼Dt [ ln ( 1−D ( F ( x ) ) ) ] . ( 1 ) In DAT , we usually utilize CNN as the feature extractor and fully connected layers ( FC ) as the domain classifier . DAT reduces the cross-domain discrepancy , achieving significant performance improvement for UDA . Nevertheless , the training of DAT is rather unstable . Without sophisticated tuning of the hyper-parameters , DAT can not reach the convergence . Through empirical experiments , we observe that such instability is due to the imbalanced minimax game . The binary domain classifier D can easily achieve convergence with very high accuracy at an early training epoch , while it is much harder for the feature extractor F to fool the domain classifier and to simultaneously perform well on the source domain . In this sense , the domain classifier dominates DAT , and the only solution is to palliate the training of D by tuning the hyper-parameters according to different tasks . In our method , we restrict the capacity of the domain classifier so as to form a minimax game in a harmonious manner . Inspired by the max-margin loss in Support Vector Machine ( SVM ) ( Cristianini et al. , 2000 ) ( i.e . hinge loss ) , if we push the source domain and the target domain away from a margin rather than as far as possible , then the training task of F to fool D becomes easier . For a binary domain classifier , we define the margin loss as Lmargin ( y ) = [ 0 , m− t · y ] + , ( 2 ) where y is the predicted domain label , [ · ] + : = max ( 0 , · ) , m is a positive margin and t is the ground truth label for two domains ( t = −1 for the source domain and t = 1 for the target domain ) . Then we introduce our MDAT scheme based on an innovative network architecture . 3.3 MAX-MARGIN DOMAIN-ADVERSARIAL TRAINING . Besides the training instability issue , DAT also suffers from restrictive feature-level alignment – lack of pixel-level alignment . To realize stable and comprehensive domain alignment together , we first propose an Adversarial Reconstruction Network ( ARN ) and then elaborate MDAT . As depicted in Figure 1 , our model consists of three parts including a shared feature extractor Ge for both domains , a label predictor Gy and a reconstruction network Gr . Let the feature extractor Ge ( x ; θe ) be a function parameterized by θe which maps an input sample x to a deep embedding z . Let the label predictor Gy ( z ; θy ) be a task-specific function parameterized by θy which maps an embedding z to a task-specific prediction ŷ . The reconstruction network Gr ( z ; θr ) is a decoding function parameterized by θr that maps an embedding z to its corresponding reconstruction x̂ . The first learning objective for the feature extractorGe and label predictorGy is to perform well in the source domain . For a supervised K-way classification problem , it is simply achieved by minimizing the negative log-likelihood of the ground truth class for each sample : Ltask = Ns∑ i=1 Ly ( xsi , ysi ) = − Ns∑ i=1 ysi · logGy ( Ge ( xsi ; θe ) ; θy ) , ( 3 ) where ysi is the one-hot encoding of the class label ysi and the logarithm operation is conducted on the softmax predictions of the model . The second objective is to render the feature learning to be domain-invariant . This is motivated by the covariate shift assumption ( Shimodaira , 2000 ) that indicates if the feature distributions S ( z ) = { Ge ( x ; θe ) |x ∼ DS } and T ( z ) = { Ge ( x ; θe ) |x ∼ DT } are similar , the source label predictor Gy can achieve a similar high accuracy in the target domain . To this end , we design a decoder network Gr that serves as a domain classifier , and then MDAT could be applied for stable training . Different from the normal binary domain classifier , MDAT lets the decoder network Gr only reconstruct the features in the source domain and push the features in the target domain away from a margin m. In this way , the decoder has the functionality of distinguishing the source domain from the target domain . The objective of training Gr is formulated as min θr Ns+Nt∑ i=1 Lmargin ( Lr ( xi ) ) = min θr Ns∑ i=1 Lr ( xsi ) + Nt∑ j=1 [ m− Lr ( xtj ) ] + , ( 4 ) where m is a positive margin and Lr ( · ) is the mean squared error ( MSE ) term for the reconstruction loss that is defined as Lr ( x ) = ||Gr ( Ge ( x ; θe ) ; θr ) − x||22 , ( 5 ) where || · ||22 denotes the squared L2-norm . Oppositely , to form a minimax game , the feature extractor Ge learns to deceive Gr such that the learned target features are indistinguishable to the source ones , which is formulated by : min θe Nt∑ j=1 Lr ( xtj ) . ( 6 ) Then the whole learning procedure of ARN with MDAT can be formulated by : min θe , θc Ns∑ i=1 Ly ( xsi , ysi ) + α Nt∑ j=1 Lr ( xtj ) , ( 7 ) min θr Ns∑ i=1 Lr ( xsi ) + Nt∑ j=1 [ m− Lr ( xtj ) ] + , ( 8 ) where Ly denotes the negative log-likelihood of the ground truth class for labeled sample ( xsi , ysi ) and α controls the interaction of the loss terms . In the following section , we provide theoretical justifications on how MDAT reduces the distribution discrepancy , and discuss why it is superior to the classic DAT .

Summary: This work proposes Adversarial Reconstruction Network (ARN), a network architecture, and Max-margin Domain-Adversarial Training (MDAT), an objective and training procedure for unsupervised domain adaptation. Similar to domain adversarial approaches, the generator aims at finding domain invariant representation while the discriminator now monitors the reconstruction loss of the source and target data using hinge-like lose. The method is very similar to some of the existing works in the literature. Experiment results on the standard digit datasets and the WiFi gesture recognition dataset show that the proposed method outperforms other alternatives.