Robust Domain Adaptation By Augmented Cyclic Adversarial Learning
AfﬁliationAddress
Abstract
Training a model to perform a task typically requires a large amount of data fromthe domains in which the task will be applied.
However, it is often the case thatdata are abundant in some domains but scarce in others.
Domain adaptation dealswith the challenge of adapting a model trained from a data-rich source domainto perform well in a data-poor target domain.
In general, this requires learningplausible mappings between domains.
CycleGAN is a powerful framework thatefﬁciently learns to map inputs from one domain to another using adversarialtraining and a cycle-consistency constraint.
However, the conventional approach ofenforcing cycle-consistency via reconstruction may be overly restrictive in caseswhere one or more domains have limited training data.
In this paper, we proposean augmented cyclic adversarial learning model that enforces the cycle-consistencyconstraint via an external task speciﬁc model, which encourages the preservationof task-relevant content as opposed to exact reconstruction.
This task speciﬁcmodel both relaxes the cycle-consistency constraint and complements the role ofthe discriminator during training, serving as an augmented information sourcefor learning the mapping.
We explore adaptation in speech and visual domainsin low resource in supervised setting.
In speech domains, we adopt a speechrecognition model from each domain as the task speciﬁc model.
Our approachimproves absolute performance of speech recognition by 2% for female speakers inthe TIMIT dataset, where the majority of training samples are from male voices.
Inlow-resource visual domain adaptation
, the results show that our approach improvesabsolute performance by 14% and 4% when adapting SVHN to MNIST and viceversa, respectively, which outperforms unsupervised domain adaptation methodsthat require high-resource unlabeled target domain.
Introduction
Domain adaptation (Huang et al., 2007; Xue et al., 2008; Ben-David et al., 2010) aims to generalizea model from source domain to a target domain.
Typically, the source domain has a large amountof training data whereas the data are scarce in the target domain.
This challenge is typicallyaddressed by learning a mapping between domains, which allows data from the source domain toenrich the available data for training in the target domain.
One way of learning such mappings isthrough Generative Adversarial Networks (GANs Goodfellow et al., 2014) with cycle-consistencyconstraint (CycleGAN Zhu et al., 2017), which enforces that mapping of an example from the sourceto the target and then back to the source domain would result in the same example (and vice versa fora target example).
Due to this constraint, CycleGAN learns to preserve the ‘content’ 1 from the source domain while only transferring the ‘style’ to match the distribution of the target domain.
This is apowerful constraint, and various works (Yi et al., 2017;
Liu et al., 2017; Hoffman et al., 2018) havedemonstrated its effectiveness in learning cross domain mappings.
Enforcing cycle-consistency is appealing as a technique for preserving semantic information of thedata with respect to a task, but implementing it through reconstruction may be too restrictive whendata are imbalanced across domains.
This is because the reconstruction error encourages exact matchof samples from the reverse mapping, which may in turn encourage the forward-mapping to keepthe sample close to the original domain.
Normally, the adversarial objectives would counter thiseffect; however, when data from the target domain are scarce, it is very difﬁcult to learn a powerfuldiscriminator that can capture meaningful properties of the target distribution.
Therefore, the resultingmappings learned is likely to be sub-optimal.
Importantly, for the learned mapping to be meaningful,it is not necessary to have the exact reconstruction.
As long as the ‘semantic’ information is preservedand the ‘style’ matches the corresponding distribution, it would be a valid mapping.
To address this issue, we propose an augmented cyclic adversarial learning model for domainadaptation.
In particular, we replace the reconstruction objective with a task speciﬁc model.
Themodel learns to preserve the ‘semantic’ information from the data samples in a particular domain byminimizing the loss of the mapped samples for the task speciﬁc model.
On the other hand, the taskspeciﬁc model also serves as an additional source of information for the corresponding domain andhence supplements the discriminator in that domain to facilitate better modeling of the distribution.
The task speciﬁc model can also be viewed as an implicit way of disentangling the informationessential to the task from the ‘style’ information that relates to the data distribution of differentdomain.
We show that our approach improves the performance by 40% as compared to the baselineon digit domain adaptation.
We improve the phoneme error rate by ∼ 5% on TIMIT dataset, whenadapting the model trained on one speech from one gender to the other.
1.1 Related Work
Our work is broadly related to domain adaptation using neural networks for both supervised andunsupervised domain adaptation.
Supervised Domain Adaptation When labels are available in the target domain, a commonapproach is to utilize the label information in target domain to minimize the discrepancy betweensource and target domain (Hu et al., 2015;
Tzeng et al., 2015; Gebru et al., 2017; Hoffman et al.
,2016; Gupta et al., 2016; Ge and Yu, 2017).
For example, Hu et al.
(2015) applies the marginal Fisheranalysis criteria and Maximum Mean Discrepancy (MMD) to minimize the distribution differencebetween source and target domain.
Tzeng et al.
(2015) proposed to add a domain classiﬁer thatpredicts domain label of the inputs, with a domain confusion loss.
Gebru et al.
(2017) leveragesattributes by using attribute and class level classiﬁcation loss with attribute consistent loss to ﬁne-tunethe target model.
Our method also employs models from both domains, however, our models are usedto assist adversarial learning for better learning of the target domain distribution.
In addition, our ﬁnalmodel for supervised domain adaptation is obtained by training on data from target domain as well asthe transfered data from the source domain, rather than ﬁne-tuning a source/target domain model.
Unsupervised Domain Adaptation More recently, various work have taken advantage of thesubstantial generation capabilities of the GAN framework and applied them to domain adaptation (Liuand Tuzel, 2016;
Bousmalis et al., 2017;
Yi et al., 2017; Tzeng et al., 2017; Kim et al., 2017; Hoffmanet al., 2018).
However, most of these works focus on high-resource unsupervised domain adaptation,which may be unsuitable for situations where the target domain data are limited.
Bousmalis et al.(2017) uses a GAN to adapt data from the source to target domain while simultaneously traininga classiﬁer on both the source and adapted data.
Our method also employs task speciﬁc models;however, we use the models to augment the CycleGAN formulation.
We show that having cyclesin both directions ( i.e. from source to target and vice versa ) is important in the case where thetarget domain has limited data (see sec. 4).
Tzeng et al.
(2017) proposes adversarial discriminativedomain adaptation (ADDA), where adversarial learning is employed to match the representationlearned from the source and target domain.
Our method also utilizes pre-trained model from source domain, but we only implicitly match the representation distributions rather than explicitly enforcingrepresentational similarity.
Cycle-consistent adversarial domain adaptation (CyCADA Hoffmanet al., 2018) is perhaps the most similar work to our own.
This approach uses both (cid:96) 1 and semanticconsistency to enforce cycle-consistency.
An important difference in our work is that we also includeanother cycle that starts from the target domain.
This is important because, if the target domain is oflow resource, the adaptation from source to target may fail due to the difﬁculty in learning a gooddiscriminator in the target domain.
Almahairi et al.
(2018) also suggests to improve CycleGAN byexplicitly enforcing content consistency and style adaptation, by augmenting the cyclic adversariallearning to hidden representation of domains.
Our model is different from recent cyclic adversarial learning, due to implicit learning of contentand style representation through an auxiliary task, which is more suitable for low resource domains.
Using classiﬁcation to assist GAN training has also been explored previously (Springenberg, 2015;
Sricharan et al., 2017; Kumar et al., 2017).
Springenberg (2015) proposed CatGAN, where thediscriminator is converted to a multi-class classiﬁer.
We extend this idea to any task speciﬁc model,including speech recognition task, and use this model to preserve task speciﬁc information regardingthe data.
We also propose that the deﬁnition of task model can be extended to unsupervised tasks,such
[Figure]
as language or speech modeling in domains, meaning augmented unsupervised domain adaptation.
Preliminaries
2.1 Generative Adversarial Network
To learn the true data distribution P data ( X ) in a nonparametric way, Goodfellow et al.
(2014)proposed the generative adversarial network (GAN).
In this framework, a discriminator network D ( x )learns to discriminate between the data produced by a generator network G ( z ) and the data sampledfrom the true data distribution P data ( X ) , whereas the generator models the true data distributionby learning to confuse the discriminator.
Under certain assumptions (Goodfellow et al., 2014), thegenerator would learn the true data distribution when the game reaches equilibrium.
Training of GANis in general done by alternately optimizing the following objective for D and G .
[Equation]
2.2 CycleGAN
CycleGAN (Zhu et al., 2017) extends this framework to multiple domains, P S ( X ) and P T ( X ) , whilelearning to map samples back and forth between them.
Adversarial learning is applied such that theresult mapping from G S (cid:55)→ T will match the target distribution P T ( X ) , and similarly for the reversemapping from G T (cid:55)→ S .
This is accomplished by the following adversarial objectives:
[Equation]
CycleGAN also introduces cycle-consistency, which enforces that each mapping is able to invert theother.
In the original work, this is achieved by including the following reconstruction objective:
[Equation]
Learning the CycleGAN model involves optimizing a weighted combination of the above objectives2, 3 and 4.
Augmented Cyclic Adversarial Learning
Enforcing cycle-consistency using a reconstruction objective ( e.g. eq. 4) may be too restrictiveand potentially results in sub-optimal mapping functions.
This is because the learning dynamics of
CycleGAN balance the two contrastive forces.
The adversarial objective encourages the mappingfunctions to generate samples that are close to the true distribution.
At the same time, the reconstruction objective encourages identity mapping.
Balancing these objectives may works well in the casewhere both domains have a relatively large number of training samples.
However, problems mayarise in case of domain adaptation, where data within the target domain are relatively sparse.
Let P S ( X ) and P T ( X ) denote source and target domain distributions, respectively, and samplesfrom P T
( X ) are limited.
In this case, it will be difﬁcult for the discriminator D T to model the actualdistribution P T ( X ) .
A discriminator model with sufﬁcient capacity will quickly overﬁt and theresulting D T will act like delta function on the sample points from P T ( X ) .
Attempts to prevent thisby limiting the capacity or using regularization may easily induce over-smoothing and under-ﬁttingsuch that the probability outputs of D T are only weakly sensitive to the mapped samples.
In bothcases, the inﬂuence of the reconstruction objective should begin to outweigh that of the adversarialobjective, thereby encouraging an identity mapping.
More generally, even if we are are able to obtaina reasonable discriminator D T , the support of the distribution learned through it would likely to besmall due to limited data.
Therefore, the learning signal G S (cid:55)→ T gets from D T would be limited.
Tosum up, limited data within P T ( X ) would make it less likely that the discriminator will encouragemeaningful cross domain mappings.
The root of the above issue in domain adaptation is two fold.
First, exact reconstruction is a too strongobjective for enforcing cycle-consistency.
Second, learning a mapping function to a particular domainwhich solely depends on the discriminator for that domain is not sufﬁcient.
To address these twoproblems, we propose to 1) use a task speciﬁc model to enforce the cycle-consistency constraint, and2) use the same task speciﬁc model in addition to the discriminator to train more meaningful crossdomain mappings.
In more detail, let M S and M T be the task speciﬁc models trained on domains
P S ( X, Y ) and P T ( X, Y ) , and L task denotes the task speciﬁc loss.
Our cycle-consistent objective isthen:
[Equation]
Here, L task enforces cycle-consistency by requiring that the reverse mappings preserve the semanticinformation of the original sample.
Importantly, this constraint is less strict than when usingreconstruction, because now as long as the content matches that of the original sample, the incurredloss will not increase.
(Some style consistency is implicitly enforced since each model M is trainedon data within a particular domain.)
This is a much looser constraint than having consistency in theoriginal data space, and thus we refer to this as the relaxed cycle-consistency objective.
Input: source domain data P S ( x, y ) , target domain data P T ( x, y ) , pretrained source task model M S
Output: target task model M T while not converged do
Sample from ( x s , y s ) from P S if y t in P T then %Supervised%
Sample ( x t , y t ) from P T
Finetune source model M S on ( x s , y s ) and ( G T (cid:55)→ S ( x t ) , y t ) samples (eq. 6)
Train task model M T on ( x t , y t ) and ( G S (cid:55)→ T ( x s ) , y s ) samples (eq. 7) else %
Un-supervised%
Sample x t from P T
Finetune source model M S on ( x s , y s ) samples (eq.
Train task model M T ( G S (cid:55)→ T ( x s ) , y s ) and ( x t , M S ( G T (cid:55)→ S ( x t ))
samples (eq. 9) end end
To address the second issue, we augment the adversarial objective with corresponding objective:
[Equation]
Similar to adversarial training, we optimize the above objective by maximizing D S ( D T ) andminimizing G T (cid:55)→ S ( G S (cid:55)→ T ) and M S ( M T ).
With the new terms, the learning of mapping functions
G get assists from both the discriminator and the task speciﬁc model.
The task speciﬁc model learnsto capture conditional probability distribution P S
( Y | X ) ( P T ( Y
| X ) )
, that also preserves informationregarding P S ( X ) ( P T ( X ) ).
This conditional information is different than the information capturedthrough the discriminator D S ( D T ).
The difference is that the model is only required to preserveuseful information regarding X respect to predicting Y , for modeling the conditional distribution,which makes learning the conditional model a much easier problem.
In addition, the conditionalmodel mediates the inﬂuence of data that the discriminator does not have access to ( Y ), which shouldfurther assist learning of the mapping functions G T (cid:55)→ S ( G S (cid:55)→ T ).
In case of unsupervised domain adaptation, when there is no information of target conditionalprobability distribution P T ( Y | X ) , we propose to use source model M S to estimate P T ( Y | X ) throughadversarial learning, i.e. P T ( Y | X )
≈ E x ∼ P T ( X ) [ M S ( G S (cid:55)→ T ( x ))] .
Therefore, proposed model canbe extended to unsupervised domain adaptation, with the corresponding modiﬁed objectives:
[Equation]
x ))]
[Equation]
L aug − unsupervised (
[Equation]
(cid:55)→ T
[Equation]
T
[Equation]
[Equation] [Equation]
[Table]
To further extend this approach to semi-supervised domain adaptation
[Table]
both supervised and unsupervised objectives for labeled and unlabeled target samples are used interchangeably
[Table]
as explained in
Algorithm
[Table]
Experiments
In this section
[Table]
we evaluate our proposed model on domain adaptation for visual and speech recognition.
We continue the convention of referring to the data domains as ‘source’ and ‘target’, wheretarget denotes the domain with either limited or unlabeled training data.
Visual domain adaptation isevaluated using the MNIST dataset ( M ) Lecun et al.
Street View House Numbers (SVHN)datasets ( S ) Netzer et al.
(2011), USPS ( U ) (Hull, 1994), MNISTM ( MM ) and Synthetic Digits ( SD )
(Ganin and Lempitsky, 2014).
Adaptation on speech is evaluated on the domain of genderwithin the TIMIT dataset Garofolo et al.
(1993), which contains broadband 16 kHz recordings of 6300utterances ( 5 . 4 hours) of phonetically-balanced speech.
The male/female ratio of speakers acrosstrain/validation/test sets is approximately 70 % to 30 %.
Therefore, we treat male speech as the sourcedomain and female speech as the low resource target domain.
4.1 Model Ablations
To get an idea of the contribution from each component of our model, in this section we perform aseries of ablations and present the results in Table 1.
We perform these ablations by treating SVHNas the source domain and MNIST as the target domain.
We down sample the MNIST training dataso only 10 samples per class are available during training, which is only 0 .
17% of full training data.
The testing performance is calculated on the full MNIST test set.
We use a modiﬁed LeNet for allexperiments in this ablation.
The Modiﬁed LeNet consists of two convolutional layers with 20 and50 channels, followed by a dropout layer and two fully connected layers of 50 and 10 dimensionality.
There are various ways that one may utilize cycle-consistency or adversarial training to do domainadaptation from components of our model.
One way is to use adversarial training on the target domainto ensure matching of distribution of adapted data, and use the task speciﬁc model to ensure the‘content’ of the data from the source domain is preserved.
This is the model described in Bousmaliset al.
(2017), except their model is originally unsupervised.
This model is denoted as S → T in Table1.
It is also interesting to examine the importance of the double cycle, which is proposed in Zhu et al.(2017) and adopted in our work.
Theoretically, one cycle would be sufﬁcient to learn the mappingbetween domains; therefore, we also investigate the performance of one cycle only models, where onedirection would be from source to target and then back, and similarly for the other direction.
Thesemodels are denoted as (S → T → S)-One Cycle and (T → S → T)-One Cycle in Table 1, respectively.
To test the effectiveness of the relaxed cycle-consistency (eq. 5) and augmented adversarial loss (eq.
6and 7), we also test one cycle models while progressively adding these two losses.
Interestingly, theone cycle relaxed and one cycle augmented models are similar to the model proposed in Hoffmanet al.
(2018) when their model performs mapping from source to target domain and then back.
Thedifference is that their model is unsupervised and includes more losses at different levels.
As can be seen from Table 1, the simple conditional model performed surprisingly well as comparedto more complicated cyclic counterparts.
This may be attributed to the reduced complexity, sinceit only needs to learn one set of mapping.
As expected, the single cycle performance is poor whenthe target domain is of limited data due to inefﬁcient learning of discriminator in the target domain(see section 3).
When we change the cycle to the other direction, where there are abundant data inthe target domain, the performance improves, but is still worse than the simple one without cycle.
This is because the adaptation mapping ( i.e. G S (cid:55)→ T ) is only learned via the generated samples from
G T (cid:55)→ S , which likely deviate from the real examples in practice.
This observation also suggeststhat it would be beneﬁcial to have cycles in both directions when applying the cycle-consistencyconstraint, since then both mappings can be learned via real examples.
The trends get reversed whenwe are using relaxed implementation of cycle-consistency from the reconstruction error with the taskspeciﬁc losses.
This is because now the power of the task speciﬁc model is crucial to preserve thecontent of the data after the reverse mapping.
When the source domain dataset is sufﬁciently large,the cycle-consistency is preserved.
As such, the resulting learned mapping functions would preservemeaningful semantics of the data while transferring the styles to the target domain, and vice versa .
Inaddition, it is clear that augmenting the discriminator with task speciﬁc loss is helpful for learningadaptations.
Furthermore, the information added from the task speciﬁc model is clearly beneﬁcialfor improving the adaptation performance, without this none of the models outperform the baselinemodel, where no adaptation is performed.
Last but not least, it is also clear from the results that usingtask speciﬁc model improves the overall adaptation performance.
4.2 Visual Domain Adaptation
In this section
[Table]
we experiment on domain adaptation for the task of digit recognition.
In eachexperiment, we select one domain (MNIST, USPS, MNISTM, SVHN, Synthetic Digits) to be thetarget.
We conduct two type of domain adaptation.
First, low-resource supervised adaptation wherewe sub-sample the target to contain only a few examples per class, using the other full dataset asthe source domain.
Comparison with recent low resource domain adaptation, FADA (Motiian et al.,2017) for MNIST, USPS
[Table]
and SVHN adaptation is shown in Table 2.
[Table]
4.3 Speech Domain Adaptation
We also apply our proposed model to domain adaptation in speech recognition.
We use the TIMITdataset, where the male to female speaker ratio is about 7 : 3 and thus we choose the data subset frommale speakers as the source and the subset from female speakers as the target domain.
We evaluate thetest performance on the standard TIMIT test set and use phoneme error rate (PER) as the evaluation
[Table]
metric.
Spectrogram representation of audio is chosen for model evaluation.
As demonstrated by
Hosseini-Asl et al.
(2018), multi-discriminator training signiﬁcantly impacts adaptation performance.
Therefore, we used the multi-discriminator architecture as the discriminator for the adversarial loss inour evaluation.
Our task speciﬁc model is a pre-trained speech recognition model within each domainin this set of experiments.
The result are shown in Table 3.
We observe signiﬁcant performance improvements over the baselinemodel as well as comparable or better performance as compared to previous methods.
It is interestingto note that the performance of the proposed model on the adapted male almost matches the baselinemodel performance, where the model is trained on true female speech.
In addition, the performancegap in this case is signiﬁcant as compared to other methods, which suggests the adapted distributionis indeed close to the true target distribution.
In addition, when combined with more data, our modelfurther out performs the baseline by a noticeable margin.
Conclusion and Future Work
In this paper, we propose to use augmented cycle-consistency adversarial learning for domain adaptation and introduce a task speciﬁc model to facilitate learning domain related mappings.
We enforcecycle-consistency using a task speciﬁc loss instead of the conventional reconstruction objective.
Additionally, we use the task speciﬁc model as an additional source of information for the discriminator inthe corresponding domain.
We demonstrate the effectiveness of our proposed approach by evaluatingon two domain adaptation tasks, and in both cases we achieve signiﬁcant performance improvementas compared to the baseline.
By extending the deﬁnition of task-speciﬁc model to unsupervised learning, such as reconstructionloss using autoencoder, or self-supervision, our proposed method would work on all settings ofdomain adaptation.
Such unsupervised task can be speech modeling using wavenet (van den Oordet al., 2016), or language modeling using recurrent or transformer networks (Radford et al., 2018).