A Closer Look at Smoothness in Domain Adversarial Training

1 INTRODUCTION . Domain Adversarial Training ( Ganin & Lempitsky , 2015 ) ( DAT ) refers to adversarial learning of neural network based feature representations that are invariant to the domain . For example , car images from the clipart domain have similar feature representations as car images from the web domain . DAT has been widely useful in diverse areas ( cited 3540 times ) such as fairness ( Adel et al. , 2019 ) , object detection ( Saito et al. , 2019 ) , domain generalization ( Li et al. , 2018 ) , imageto-image translation ( Liu et al. , 2017 ) etc . The prime driver of research on DAT is its application in unsupervised Domain Adaptation ( DA ) , which aims to learn a classifier using labeled source data and unlabeled target data , such that it generalizes well on target data . Various enhancements like superior objectives ( Acuna et al. , 2021 ; Zhang et al. , 2019 ) , architectures ( Long et al. , 2018 ) etc . have been proposed to improve its effectiveness . However , as DAT objective is combination of Generative Adversarial Network ( GAN ) ( Goodfellow et al. , 2014 ) and Empirical Risk Minimization ( ERM ) ( Vapnik , 2013 ) objectives , there has not been much focus on explicitly analyzing the nature of optimization in DAT . One direction of work aiming to improve generalization of ERM on unseen data focuses on developing algorithms that converge to a smooth ( or a flat ) minima ( Foret et al. , 2021 ; Keskar & Socher , 2017 ) . However , we find that these techniques , when directly applied for DAT , do not significantly improve the generalization on the target domain ( Sec . 4 and 7 ) . In this work , we analyze the loss landscape near the optimal point obtained by DAT , to gain insights into curvature . We first focus on the eigen-spectrum of Hessian of the task loss ( ERM term for classification ) where we find that using Stochastic Gradient Descent ( SGD ) as optimizer converges to a smoother minima in comparison to Adam ( Kingma & Ba , 2014 ) . Further we find that smoother minima w.r.t.task loss leads to better generalization on the target domain . Contrary to task loss , we find that smoothness enhancing formulation for adversarial components worsen performance , rendering ERM-based techniques which enhance smoothness for all loss components ineffective . Hence we introduce Smooth Domain Adversarial Training ( SDAT ) , which aims only to reach a smooth minima w.r.t . task loss , and helps in generalizing better on the target domain . SDAT requires an additional gradient computation step and can be combined with existing methods with a few lines of code . We show the soundness of the SDAT method theoretically by proving a generalization bound ( Sec . 4 ) on target error . We extensively verify the empirical efficacy of SDAT across various datasets for classification ( i.e. , DomainNet , VisDA-2017 and Office-Home ) , along with showing a prototypical application in DA for object detection , demonstrating it ’ s diverse applicability . In summary , we make the following contributions : • We analyze the optimization procedure of DAT , establishing the correlation between the smoothness near optima w.r.t . task loss and generalization on the target domain . • Contrary to ERM , we show through our theoretical and empirical analysis that smoothness enhancing adversarial formulation leads to sub-optimal performance . • For enhancing the smoothness w.r.t . task loss near optima in DAT , we propose a novel and theoretically motivated SDAT that improves the generalization on the target domain . SDAT effectively increases the average performance of even state-of-the-art adversarial adaptation methods . 2 RELATED WORK . Unsupervised Domain Adaptation : It refers to a class of methods that aim to adapt models to work in a target domain distinct from what it was trained on . One of the most prominent lines of work is based on DAT ( Ganin & Lempitsky , 2015 ) . This involves using an additional discriminator to distinguish between samples of source and target domain . The goal of the model is to learn features that can not be distinguished between source and target . The follow-up works have improved this basic idea by introducing a class information based discriminator ( CDAN ( Long et al. , 2018 ) ) , introducing a transferable normalization function ( Wang et al. , 2019 ) etc . In this work , we focus on analyzing and improving such methods . Another line of work involves DA by using self-training on target domain ( Kundu et al. , 2020b ; a ; Prabhu et al. , 2020 ) which will not be the focus of this work . Smoothness of Loss Landscape : As neural networks operate in the regime of over parameterized models , low error on training data does not always lead to better generalization ( Keskar et al. , 2017 ) . Often it has been stated ( He et al. , 2019 ; Dziugaite & Roy , 2017 ) that smoother minima does generalize better on unseen data . But until recently , this was practically expensive as smoothing required additional costly computations . Recently , a method called Sharpness Aware Minimization ( SAM ) ( Foret et al. , 2021 ) has been proposed to find a smoother minima with an additional gradient computation step . SAM also improves the ImageNet model performance ( Chen et al. , 2021 ) on ImageNet-C and ImageNet-R ( which are out of distribution ) . It has also been observed that smoothness w.r.t . input ( image ) is beneficial for domain adaptation ( Shu et al. , 2018 ; Cai et al. , 2021 ) , which motivates us to explore smoothness w.r.t weights ( W ) in case of DAT . However , the earlier work has focused on achieving a smoother minima w.r.t . W for ERM . Currently , no study has been done if the loss function is composed of both ERM and adversarial objectives ( as present in DAT ) . 3 BACKGROUND . 3.1 PRELIMINARIES . We will primarily focus on Unsupervised DA where we have labeled source data S = { ( xsi , ysi ) } and unlabeled target data T = { ( xti ) } . The source samples are assumed to be sampled i.i.d . from source 1Figures for the smooth minima and sharp minima are from ( Foret et al. , 2021 ) and used for illustration purposes only . distribution PS defined on input space X , similarly target samples are sampled i.i.d . from PT . Y is used for denoting the label set which is { 1 , 2 , . . . , k } in our case as we perform multiclass ( k ) classification . We denote y : X → Y a mapping from images to labels . Our task is to find a hypothesis function hθ that has a low risk on the target distribution . The source risk ( a.k.a expected error ) of the hypothesis hθ is defined with respect to loss function l as : RlS ( hθ ) = Ex∼PS [ l ( hθ ( x ) , y ( x ) ) ] . The target risk RlT ( hθ ) is defined analogously . The empirical versions of source and target risk will be denoted by R̂lS ( hθ ) and R̂ l T ( hθ ) . All notations used in paper are summarized in Table F. In this work we build on the domain adaption theory of ( Acuna et al. , 2021 ) which is a generalization of Ben-David et al . ( 2010 ) . We first define the discrepancy between the two domains . Definition 3.1 ( Dφhθ , H discrepancy ) . The discrepancy between two domains PS and PT is defined as following : Dφhθ , H ( PS ||PT ) : = sup h′∈H [ Ex∼PS [ l ( hθ ( x ) , h′ ( x ) ) ] ] − [ Ex∼PT [ φ∗ ( l ( hθ ( x ) , h′ ( x ) ) ) ] ] ( 1 ) Here φ∗ is a frenchel conjugate of a lower semi-continuous convex function φ that satisfies φ ( 1 ) = 0 , andH is the set of all possible hypothesis ( i.e . Hypothesis Space ) . This discrepancy distance Dφhθ , H is based on variational formulation of f-divergence ( Nguyen et al. , 2010 ) for the convex function φ. TheDφhθ , H is the lower bound estimate of the f-divergence function Dφ ( PS ||PT ) . See Lemma 4 in ( Acuna et al. , 2021 ) for additional details . We state a bound on target risk RlT ( hθ ) based on D φ hθ , H discrepancy ( Acuna et al. , 2021 ) : Theorem 1 ( Generalization bound ) . Suppose l : Y × Y → [ 0 , 1 ] ⊂ dom φ∗ . Let h∗ be the ideal joint classifier with least λ∗ = RlS ( h ∗ ) + RlT ( h ∗ ) ( i.e . joint risk ) in H. We have the following relation between source and target risk : RlT ( hθ ) ≤ RlS ( hθ ) +D φ hθ , H ( PS ||PT ) + λ ∗ ( 2 ) The above generalization bound shows that the target risk RlT ( hθ ) is upper bounded by the source risk RlS ( hθ ) and the discrepancy term D φ hθ , H along with an irreducible constant error λ ∗ . Hence , this infers that reducing source risk and discrepancy lead a to reduction in target risk . Based on this , we concretely define the unsupervised adversarial adaptation procedure in the next section . 3.2 UNSUPERVISED DOMAIN ADAPTATION . In this section we first define the components of the framework we use for our purpose : hθ = fΘ◦gψ where gψ is the feature extractor and fΘ is the classifier . The domain discriminator DΦ , used for estimating the discrepancy between PS and PT is a classifier whose goal is to distinguish between the features of two domains . For minimizing the target risk ( Th . 1 ) , the optimization problem can be written as : min θ Ex∼PS [ l ( hθ ( x ) , y ( x ) ) ] +D φ hθ , H ( PS ||PT ) ( 3 ) The discrepancy term under some assumptions ( refer App . B ) can be upper bounded by a tractable term : Dφhθ , H ( PS ||PT ) ≤ maxΦ d Φ S , T ( 4 ) where dΦS , T = Ex∼PS [ log ( DΦ ( gψ ( x ) ) ) ] + Ex∼PT log [ 1−DΦ ( gψ ( x ) ) ] . This leads to the final optimization objective of : min θ max Φ Ex∼PS [ l ( hθ ( x ) , y ( x ) ) ] + dΦS , T ( 5 ) The first term in practice is empirically approximated by using finite samples R̂lS ( hθ ) and used as task loss ( classification ) for minimization . The empirical estimate of the second term is adversarial loss which is optimized using a gradient reversal layer ( GRL ) as it has a min-max form . ( Overview in Fig . 1 ) The above procedure composes DAT , and we use CDAN ( Long et al. , 2018 ) as our default DAT method .

Summary: This work studies the loss landscape of domain adversarial neural networks for domain adaptation. The authors claim that similarly to the iid setting, smooth minima might yield better generalization on the unsupervised domain adaptation framework via domain adversarial training but in case smoothness is enforced only with respect to the task loss. Experiments on the OfficeHome dataset indicated this hypothesis might be true. The authors then proposed to apply a previously proposed approach to enforce smoothness (Sharpness Aware Minimization), so that only smoothness with respect to the task loss would be enforced. Moreover, a theorem showing that the non-smooth domain discrepancy estimation is better than the smoother counterpart was introduced, along with a generalization bound for the risk on the target domain in terms of the smooth risk on the source domain. Finally, the authors showed that the proposed approach, SDAT, improved the performance on target domains of three domain adaptation datasets in comparison to the considered baselines. Further experiments showed that SDAT performed better than other smoothing techniques, as well as yields improved robustness to label noise in comparison to not enforcing smooth minima.