A Closer Look at Smoothness in Domain Adversarial Training

1 INTRODUCTION . Domain Adversarial Training ( Ganin & Lempitsky , 2015 ) ( DAT ) refers to adversarial learning of neural network based feature representations that are invariant to the domain . For example , car images from the clipart domain have similar feature representations as car images from the web domain . DAT has been widely useful in diverse areas ( cited 3540 times ) such as fairness ( Adel et al. , 2019 ) , object detection ( Saito et al. , 2019 ) , domain generalization ( Li et al. , 2018 ) , imageto-image translation ( Liu et al. , 2017 ) etc . The prime driver of research on DAT is its application in unsupervised Domain Adaptation ( DA ) , which aims to learn a classifier using labeled source data and unlabeled target data , such that it generalizes well on target data . Various enhancements like superior objectives ( Acuna et al. , 2021 ; Zhang et al. , 2019 ) , architectures ( Long et al. , 2018 ) etc . have been proposed to improve its effectiveness . However , as DAT objective is combination of Generative Adversarial Network ( GAN ) ( Goodfellow et al. , 2014 ) and Empirical Risk Minimization ( ERM ) ( Vapnik , 2013 ) objectives , there has not been much focus on explicitly analyzing the nature of optimization in DAT . One direction of work aiming to improve generalization of ERM on unseen data focuses on developing algorithms that converge to a smooth ( or a flat ) minima ( Foret et al. , 2021 ; Keskar & Socher , 2017 ) . However , we find that these techniques , when directly applied for DAT , do not significantly improve the generalization on the target domain ( Sec . 4 and 7 ) . In this work , we analyze the loss landscape near the optimal point obtained by DAT , to gain insights into curvature . We first focus on the eigen-spectrum of Hessian of the task loss ( ERM term for classification ) where we find that using Stochastic Gradient Descent ( SGD ) as optimizer converges to a smoother minima in comparison to Adam ( Kingma & Ba , 2014 ) . Further we find that smoother minima w.r.t.task loss leads to better generalization on the target domain . Contrary to task loss , we find that smoothness enhancing formulation for adversarial components worsen performance , rendering ERM-based techniques which enhance smoothness for all loss components ineffective . Hence we introduce Smooth Domain Adversarial Training ( SDAT ) , which aims only to reach a smooth minima w.r.t . task loss , and helps in generalizing better on the target domain . SDAT requires an additional gradient computation step and can be combined with existing methods with a few lines of code . We show the soundness of the SDAT method theoretically by proving a generalization bound ( Sec . 4 ) on target error . We extensively verify the empirical efficacy of SDAT across various datasets for classification ( i.e. , DomainNet , VisDA-2017 and Office-Home ) , along with showing a prototypical application in DA for object detection , demonstrating it ’ s diverse applicability . In summary , we make the following contributions : • We analyze the optimization procedure of DAT , establishing the correlation between the smoothness near optima w.r.t . task loss and generalization on the target domain . • Contrary to ERM , we show through our theoretical and empirical analysis that smoothness enhancing adversarial formulation leads to sub-optimal performance . • For enhancing the smoothness w.r.t . task loss near optima in DAT , we propose a novel and theoretically motivated SDAT that improves the generalization on the target domain . SDAT effectively increases the average performance of even state-of-the-art adversarial adaptation methods . 2 RELATED WORK . Unsupervised Domain Adaptation : It refers to a class of methods that aim to adapt models to work in a target domain distinct from what it was trained on . One of the most prominent lines of work is based on DAT ( Ganin & Lempitsky , 2015 ) . This involves using an additional discriminator to distinguish between samples of source and target domain . The goal of the model is to learn features that can not be distinguished between source and target . The follow-up works have improved this basic idea by introducing a class information based discriminator ( CDAN ( Long et al. , 2018 ) ) , introducing a transferable normalization function ( Wang et al. , 2019 ) etc . In this work , we focus on analyzing and improving such methods . Another line of work involves DA by using self-training on target domain ( Kundu et al. , 2020b ; a ; Prabhu et al. , 2020 ) which will not be the focus of this work . Smoothness of Loss Landscape : As neural networks operate in the regime of over parameterized models , low error on training data does not always lead to better generalization ( Keskar et al. , 2017 ) . Often it has been stated ( He et al. , 2019 ; Dziugaite & Roy , 2017 ) that smoother minima does generalize better on unseen data . But until recently , this was practically expensive as smoothing required additional costly computations . Recently , a method called Sharpness Aware Minimization ( SAM ) ( Foret et al. , 2021 ) has been proposed to find a smoother minima with an additional gradient computation step . SAM also improves the ImageNet model performance ( Chen et al. , 2021 ) on ImageNet-C and ImageNet-R ( which are out of distribution ) . It has also been observed that smoothness w.r.t . input ( image ) is beneficial for domain adaptation ( Shu et al. , 2018 ; Cai et al. , 2021 ) , which motivates us to explore smoothness w.r.t weights ( W ) in case of DAT . However , the earlier work has focused on achieving a smoother minima w.r.t . W for ERM . Currently , no study has been done if the loss function is composed of both ERM and adversarial objectives ( as present in DAT ) . 3 BACKGROUND . 3.1 PRELIMINARIES . We will primarily focus on Unsupervised DA where we have labeled source data S = { ( xsi , ysi ) } and unlabeled target data T = { ( xti ) } . The source samples are assumed to be sampled i.i.d . from source 1Figures for the smooth minima and sharp minima are from ( Foret et al. , 2021 ) and used for illustration purposes only . distribution PS defined on input space X , similarly target samples are sampled i.i.d . from PT . Y is used for denoting the label set which is { 1 , 2 , . . . , k } in our case as we perform multiclass ( k ) classification . We denote y : X → Y a mapping from images to labels . Our task is to find a hypothesis function hθ that has a low risk on the target distribution . The source risk ( a.k.a expected error ) of the hypothesis hθ is defined with respect to loss function l as : RlS ( hθ ) = Ex∼PS [ l ( hθ ( x ) , y ( x ) ) ] . The target risk RlT ( hθ ) is defined analogously . The empirical versions of source and target risk will be denoted by R̂lS ( hθ ) and R̂ l T ( hθ ) . All notations used in paper are summarized in Table F. In this work we build on the domain adaption theory of ( Acuna et al. , 2021 ) which is a generalization of Ben-David et al . ( 2010 ) . We first define the discrepancy between the two domains . Definition 3.1 ( Dφhθ , H discrepancy ) . The discrepancy between two domains PS and PT is defined as following : Dφhθ , H ( PS ||PT ) : = sup h′∈H [ Ex∼PS [ l ( hθ ( x ) , h′ ( x ) ) ] ] − [ Ex∼PT [ φ∗ ( l ( hθ ( x ) , h′ ( x ) ) ) ] ] ( 1 ) Here φ∗ is a frenchel conjugate of a lower semi-continuous convex function φ that satisfies φ ( 1 ) = 0 , andH is the set of all possible hypothesis ( i.e . Hypothesis Space ) . This discrepancy distance Dφhθ , H is based on variational formulation of f-divergence ( Nguyen et al. , 2010 ) for the convex function φ. TheDφhθ , H is the lower bound estimate of the f-divergence function Dφ ( PS ||PT ) . See Lemma 4 in ( Acuna et al. , 2021 ) for additional details . We state a bound on target risk RlT ( hθ ) based on D φ hθ , H discrepancy ( Acuna et al. , 2021 ) : Theorem 1 ( Generalization bound ) . Suppose l : Y × Y → [ 0 , 1 ] ⊂ dom φ∗ . Let h∗ be the ideal joint classifier with least λ∗ = RlS ( h ∗ ) + RlT ( h ∗ ) ( i.e . joint risk ) in H. We have the following relation between source and target risk : RlT ( hθ ) ≤ RlS ( hθ ) +D φ hθ , H ( PS ||PT ) + λ ∗ ( 2 ) The above generalization bound shows that the target risk RlT ( hθ ) is upper bounded by the source risk RlS ( hθ ) and the discrepancy term D φ hθ , H along with an irreducible constant error λ ∗ . Hence , this infers that reducing source risk and discrepancy lead a to reduction in target risk . Based on this , we concretely define the unsupervised adversarial adaptation procedure in the next section . 3.2 UNSUPERVISED DOMAIN ADAPTATION . In this section we first define the components of the framework we use for our purpose : hθ = fΘ◦gψ where gψ is the feature extractor and fΘ is the classifier . The domain discriminator DΦ , used for estimating the discrepancy between PS and PT is a classifier whose goal is to distinguish between the features of two domains . For minimizing the target risk ( Th . 1 ) , the optimization problem can be written as : min θ Ex∼PS [ l ( hθ ( x ) , y ( x ) ) ] +D φ hθ , H ( PS ||PT ) ( 3 ) The discrepancy term under some assumptions ( refer App . B ) can be upper bounded by a tractable term : Dφhθ , H ( PS ||PT ) ≤ maxΦ d Φ S , T ( 4 ) where dΦS , T = Ex∼PS [ log ( DΦ ( gψ ( x ) ) ) ] + Ex∼PT log [ 1−DΦ ( gψ ( x ) ) ] . This leads to the final optimization objective of : min θ max Φ Ex∼PS [ l ( hθ ( x ) , y ( x ) ) ] + dΦS , T ( 5 ) The first term in practice is empirically approximated by using finite samples R̂lS ( hθ ) and used as task loss ( classification ) for minimization . The empirical estimate of the second term is adversarial loss which is optimized using a gradient reversal layer ( GRL ) as it has a min-max form . ( Overview in Fig . 1 ) The above procedure composes DAT , and we use CDAN ( Long et al. , 2018 ) as our default DAT method .

Summary: This paper introduces a smoothness penalty for domain adversarial training. The penalty encourages smoothness on the parameter space of a discriminative model. This smoothness penalty is motivated by results in Sharpness Aware Minimization. Results are included on the popular DA datasets such as Office-home and VisDA, where superior results are shown. Ablation experiments show how different smoothness penalties change the observed improvements.