Augmented Cyclic Adversarial Learning for Low Resource Domain Adaptation

1 INTRODUCTION . Domain adaptation ( Huang et al. , 2007 ; Xue et al. , 2008 ; Ben-David et al. , 2010 ) aims to generalize a model from source domain to a target domain . Typically , the source domain has a large amount of training data , whereas the data are scarce in the target domain . This challenge is typically addressed by learning a mapping between domains , which allows data from the source domain to enrich the available data for training in the target domain . One way of learning such mappings is through Generative Adversarial Networks ( GANs Goodfellow et al. , 2014 ) with cycle-consistency constraint ( CycleGAN Zhu et al. , 2017 ) , which enforces that mapping of an example from the source to the target and then back to the source domain would result in the same example ( and vice versa for a target example ) . Due to this constraint , CycleGAN learns to preserve the ‘ content ’ 1 from the source domain while only transferring the ‘ style ’ to match the distribution of the target domain . This is a powerful constraint , and various works ( Yi et al. , 2017 ; Liu et al. , 2017 ; Hoffman et al. , 2018 ) have demonstrated its effectiveness in learning cross domain mappings . 1Here the content refers to the invariant properties of the data with respect to a task . For example , in image classification the semantic information of an image would be its class . Thus , different task on the same data would result in different semantic information . In this paper we use content and semantic information interchangeably . Enforcing cycle-consistency is appealing as a technique for preserving semantic information of the data with respect to a task , but implementing it through reconstruction may be too restrictive when data are imbalanced across domains . This is because the reconstruction error encourages exact match of samples from the reverse mapping , which may in turn encourage the forward-mapping to keep the sample close to the original domain . Normally , the adversarial objectives would counter this effect ; however , when data from the target domain are scarce , it is very difficult to learn a powerful discriminator that can capture meaningful properties of the target distribution . Therefore , the resulting mappings learned is likely to be sub-optimal . Importantly , for the learned mapping to be meaningful , it is not necessary to have the exact reconstruction . As long as the ‘ semantic ’ information is preserved and the ‘ style ’ matches the corresponding distribution , it would be a valid mapping . To address this issue , we propose an augmented cyclic adversarial learning model ( ACAL ) for domain adaptation . In particular , we replace the reconstruction objective with a task specific model . The model learns to preserve the ‘ semantic ’ information from the data samples in a particular domain by minimizing the loss of the mapped samples for the task specific model . On the other hand , the task specific model also serves as an additional source of information for the corresponding domain and hence supplements the discriminator in that domain to facilitate better modeling of the distribution . The task specific model can also be viewed as an implicit way of disentangling the information essential to the task from the ‘ style ’ information that relates to the data distribution of different domain . We show that our approach improves the performance by 40 % as compared to the baseline on digit domain adaptation . We improve the phoneme error rate by ∼ 5 % on TIMIT dataset , when adapting the model trained on one speech from one gender to the other . 1.1 RELATED WORK . Our work is broadly related to domain adaptation using neural networks for both supervised and unsupervised domain adaptation . Supervised Domain Adaptation When labels are available in the target domain , a common approach is to utilize the label information in target domain to minimize the discrepancy between source and target domain ( Hu et al. , 2015 ; Tzeng et al. , 2015 ; Gebru et al. , 2017 ; Hoffman et al. , 2016 ; Gupta et al. , 2016 ; Ge and Yu , 2017 ) . For example , Hu et al . ( 2015 ) applies the marginal Fisher analysis criteria and Maximum Mean Discrepancy ( MMD ) to minimize the distribution difference between source and target domain . Tzeng et al . ( 2015 ) proposed to add a domain classifier that predicts domain label of the inputs , with a domain confusion loss . Gebru et al . ( 2017 ) leverages attributes by using attribute and class level classification loss with attribute consistent loss to fine-tune the target model . Our method also employs models from both domains , however , our models are used to assist adversarial learning for better learning of the target domain distribution . In addition , our final model for supervised domain adaptation is obtained by training on data from target domain as well as the transfered data from the source domain , rather than fine-tuning a source/target domain model . Unsupervised Domain Adaptation More recently , various work have taken advantage of the substantial generation capabilities of the GAN framework and applied them to domain adaptation ( Liu and Tuzel , 2016 ; Bousmalis et al. , 2017 ; Yi et al. , 2017 ; Tzeng et al. , 2017 ; Kim et al. , 2017 ; Hoffman et al. , 2018 ) . However , most of these works focus on high-resource unsupervised domain adaptation , which may be unsuitable for situations where the target domain data are limited . Bousmalis et al . ( 2017 ) uses a GAN to adapt data from the source to target domain while simultaneously training a classifier on both the source and adapted data . Our method also employs task specific models ; however , we use the models to augment the CycleGAN formulation . We show that having cycles in both directions ( i.e . from source to target and vice versa ) is important in the case where the target domain has limited data ( see sec . 4 ) . Tzeng et al . ( 2017 ) proposes adversarial discriminative domain adaptation ( ADDA ) , where adversarial learning is employed to match the representation learned from the source and target domain . Our method also utilizes pre-trained model from source domain , but we only implicitly match the representation distributions rather than explicitly enforcing representational similarity . Cycle-consistent adversarial domain adaptation ( CyCADA Hoffman et al. , 2018 ) is perhaps the most similar work to our own . This approach uses both ` 1 and semantic consistency to enforce cycle-consistency . An important difference in our work is that we also include another cycle that starts from the target domain . This is important because , if the target domain is of low resource , the adaptation from source to target may fail due to the difficulty in learning a good discriminator in the target domain . Almahairi et al . ( 2018 ) also suggests to improve CycleGAN by explicitly enforcing content consistency and style adaptation , by augmenting the cyclic adversarial learning to hidden representation of domains . Our model is different from recent cyclic adversarial learning , due to implicit learning of content and style representation through an auxiliary task , which is more suitable for low resource domains . Using classification to assist GAN training has also been explored previously ( Springenberg , 2015 ; Sricharan et al. , 2017 ; Kumar et al. , 2017 ) . Springenberg ( 2015 ) proposed CatGAN , where the discriminator is converted to a multi-class classifier . We extend this idea to any task specific model , including speech recognition task , and use this model to preserve task specific information regarding the data.We also propose that the definition of task model can be extended to unsupervised tasks , such as language or speech modeling in domains , meaning augmented unsupervised domain adaptation . 2 PRELIMINARIES . 2.1 GENERATIVE ADVERSARIAL NETWORK . To learn the true data distribution Pdata ( X ) in a nonparametric way , Goodfellow et al . ( 2014 ) proposed the generative adversarial network ( GAN ) . In this framework , a discriminator network D ( x ) learns to discriminate between the data produced by a generator network G ( z ) and the data sampled from the true data distribution Pdata ( X ) , whereas the generator models the true data distribution by learning to confuse the discriminator . Under certain assumptions ( Goodfellow et al. , 2014 ) , the generator would learn the true data distribution when the game reaches equilibrium . Training of GAN is in general done by alternately optimizing the following objective for D and G. min G max D V ( G , D ) = Ex∼Pdata ( X ) [ logD ( x ) ] + Ez∼Pz ( Z ) [ log ( 1−D ( G ( z ) ) ] ( 1 ) 2.2 CYCLEGAN . CycleGAN ( Zhu et al. , 2017 ) extends this framework to multiple domains , PS ( X ) and PT ( X ) , while learning to map samples back and forth between them . Adversarial learning is applied such that the result mapping from GS 7→T will match the target distribution PT ( X ) , and similarly for the reverse mapping from GT 7→S . This is accomplished by the following adversarial objectives : Ladv ( GS 7→T , DT ) = Ex∼PT ( X ) [ logDT ( x ) ] + Ex∼PS ( X ) [ log ( 1−DT ( GS 7→T ( x ) ) ] ( 2 ) Ladv ( GT 7→S , DS ) = Ex∼PS ( X ) [ logDS ( x ) ] + Ex∼PT ( X ) [ log ( 1−DS ( GT 7→S ( x ) ) ] ( 3 ) CycleGAN also introduces cycle-consistency , which enforces that each mapping is able to invert the other . In the original work , this is achieved by including the following reconstruction objective : Lcyc ( GS 7→T , GT 7→S ) = Ex∼PS ( X ) [ ‖GT 7→S ( GS 7→T ( x ) ) − x‖1 ] + Ex∼PT ( X ) [ ‖GS 7→T ( GT 7→S ( x ) ) − x‖1 ] ( 4 ) Learning the CycleGAN model involves optimizing a weighted combination of the above objectives 2 , 3 and 4 . 3 AUGMENTED CYCLIC ADVERSARIAL LEARNING ( ACAL ) . Enforcing cycle-consistency using a reconstruction objective ( e.g . eq . 4 ) may be too restrictive and potentially results in sub-optimal mapping functions . This is because the learning dynamics of CycleGAN balance the two contrastive forces . The adversarial objective encourages the mapping functions to generate samples that are close to the true distribution . At the same time , the reconstruction objective encourages identity mapping . Balancing these objectives may works well in the case where both domains have a relatively large number of training samples . However , problems may arise in case of domain adaptation , where data within the target domain are relatively sparse . Let PS ( X ) and PT ( X ) denote source and target domain distributions , respectively , and samples from PT ( X ) are limited . In this case , it will be difficult for the discriminator DT to model the actual distribution PT ( X ) . A discriminator model with sufficient capacity will quickly overfit and the resulting DT will act like delta function on the sample points from PT ( X ) . Attempts to prevent this by limiting the capacity or using regularization may easily induce over-smoothing and under-fitting such that the probability outputs of DT are only weakly sensitive to the mapped samples . In both cases , the influence of the reconstruction objective should begin to outweigh that of the adversarial objective , thereby encouraging an identity mapping . More generally , even if we are are able to obtain a reasonable discriminator DT , the support of the distribution learned through it would likely to be small due to limited data . Therefore , the learning signal GS 7→T receive from DT would be limited . To sum up , limited data within PT ( X ) would make it less likely that the discriminator will encourage meaningful cross domain mappings . The root of the above issue in domain adaptation is two fold . First , exact reconstruction is a too strong objective for enforcing cycle-consistency . Second , learning a mapping function to a particular domain which solely depends on the discriminator for that domain is not sufficient . To address these two problems , we propose to 1 ) use a task specific model to enforce the cycle-consistency constraint , and 2 ) use the same task specific model in addition to the discriminator to train more meaningful cross domain mappings . In more detail , let MS and MT be the task specific models trained on domains PS ( X , Y ) and PT ( X , Y ) , and Ltask denotes the task specific loss . Our cycle-consistent objective is then : LRCAL ( GS 7→T , GT 7→S , MS , MT ) = E ( x , y ) ∼PS ( X , Y ) [ Ltask ( MS ( GT 7→S ( GS 7→T ( x ) ) , y ) ] + E ( x , y ) ∼PT ( X , Y ) [ Ltask ( MT ( GS 7→T ( GT 7→S ( x ) ) , y ) ] ( 5 ) Here , Ltask enforces cycle-consistency by requiring that the reverse mappings preserve the semantic information of the original sample . Importantly , this constraint is less strict than when using reconstruction , because now as long as the content matches that of the original sample , the incurred loss will not increase . ( Some style consistency is implicitly enforced since each model M is trained on data within a particular domain . ) This is a much looser constraint than having consistency in the original data space , and thus we refer to this as the relaxed cycle-consistency objective . To address the second issue , we augment the adversarial objective with corresponding objective : LACAL−supervised ( GT 7→S , DS , MS ) = Ex∼PS ( X ) [ log ( DS ( x ) ) ] + Ex∼PT ( X ) [ log ( 1−DS ( GT 7→S ( x ) ) ) ] + E ( x , y ) ∼PS ( x , y ) [ Ltask ( MS ( x , y ) ) ] + E ( x , y ) ∼PT ( x , y ) [ Ltask ( MS ( GT 7→S ( x ) , y ) ) ] ( 6 ) LACAL−supervised ( GS 7→T , DT , MT ) = Ex∼PT ( X ) [ log ( DT ( x ) ) ] + Ex∼PS ( X ) [ log ( 1−DT ( GS 7→T ( x ) ) ) ] + E ( x , y ) ∼PT ( x , y ) [ Ltask ( MT ( x , y ) ) ] + E ( x , y ) ∼PS ( x , y ) [ Ltask ( MT ( GS 7→T ( x ) , y ) ) ] ( 7 ) Similar to adversarial training , we optimize the above objective by maximizing DS ( DT ) and minimizing GT 7→S ( GS 7→T ) and MS ( MT ) . With the new terms , learning of the mapping functions G get assists from both the discriminator and the task specific model . The task specific model learns to capture conditional probability distribution PS ( Y |X ) ( PT ( Y |X ) ) , that also preserves information regarding PS ( X ) ( PT ( X ) ) . This conditional information is different than the information captured through the discriminator DS ( DT ) . The difference is that the model is only required to preserve Algorithm 1 Augmented Cyclic Adversarial Learning ( ACAL ) Input : source domain data PS ( x , y ) , target domain data PT ( x , y ) , pretrained source task model MS Output : target task model MT while not converged do Sample from ( xs , ys ) from PS if yt in PT then % Supervised % Sample ( xt , yt ) from PT Finetune source model MS on ( xs , ys ) and ( GT 7→S ( xt ) , yt ) samples ( eq . 6 ) Train task model MT on ( xt , yt ) and ( GS 7→T ( xs ) , ys ) samples ( eq . 7 ) else % Un-supervised % Sample xt from PT Finetune source model MS on ( xs , ys ) samples ( eq . 8 ) Train task model MT ( GS 7→T ( xs ) , ys ) and ( xt , MS ( GT 7→S ( xt ) ) samples ( eq . 9 ) end end useful information regarding X respect to predicting Y , for modeling the conditional distribution , which makes learning the conditional model a much easier problem . In addition , the conditional model mediates the influence of data that the discriminator does not have access to ( Y ) , which should further assist learning of the mapping functions GT 7→S ( GS 7→T ) . In case of unsupervised domain adaptation , when there is no information of target conditional probability distribution PT ( Y |X ) , we propose to use source model MS to estimate PT ( Y |X ) through adversarial learning , i.e . PT ( Y |X ) ≈ Ex∼PT ( X ) [ MS ( GS 7→T ( x ) ) ] . Therefore , proposed model can be extended to unsupervised domain adaptation , with the corresponding modified objectives : LACAL−unsupervised ( GT 7→S , DS , MS ) = Ex∼PS ( X ) [ log ( DS ( x ) ) ] + Ex∼PT ( X ) [ log ( 1−DS ( GT 7→S ( x ) ) ) ] + E ( x , y ) ∼PS ( x , y ) [ Ltask ( MS ( x , y ) ) ] ( 8 ) LACAL−unsupervised ( GS 7→T , DT , MT ) = Ex∼PT ( X ) [ log ( DT ( x ) ) ] + Ex∼PS ( X ) [ log ( 1−DT ( GS 7→T ( x ) ) ) ] + E ( x , y ) ∼PT ( x , y ) [ Ltask ( MT ( x , MS ( GT 7→S ( x ) ) ) ) ] + E ( x , y ) ∼PS ( x , y ) [ Ltask ( MT ( GS 7→T ( x ) , y ) ) ] ( 9 ) To further extend this approach to semi-supervised domain adaptation , both supervised and unsupervised objectives for labeled and unlabeled target samples are used interchangeably , as explained in Algorithm 1 .

Summary: The authors propose an extension of cycle-consistent adversarial adaptation methods in order to tackle domain adaptation in settings where a limited amount of supervised target data is available (though they also validate their model in the standard unsupervised setting as well). The method appears to be a natural generalization/extension of CycleGAN/CyCADA. It uses the ideas of the semantic consistency loss and training on adapted data from CyCADA, but "fills out" the model by applying these techniques in both directions (whereas CyCADA only applied them in the source-to-target direction).