Original: Low Resource Style Transfer via Domain Adaptive Meta Learning
Revised: Low Resource Style Transfer via Domain Adaptive Meta Learning

Original: Anonymous ACL submission

Original: Abstract
Revised: Abstract

Original: Text style transfer (TST) without parallel datahas achieved some practical success.
Revised: Text style transfer (TST) without parallel data has achieved some practical success.

Original: However, most of the existing unsupervised textstyle transfer methods suffer from (i) requiringmassive amounts of nonparallel data to guidetransferring different text styles.
Revised: However, most of the existing unsupervised text style transfer methods suffer from (i) requiring massive amounts of non-parallel data to guide transferring different text styles.

Original: (ii) colossal performance degradation when ﬁne-tuningthe model in new domains.
Revised: (ii) colossal performance degradation when ﬁne-tuning the model in new domains.

Original: In this work, wepropose DAML-ATM(Domain Adaptive MetaLearning with Adversarial Transfer Model),which consists of two parts, DAML and ATM.
Revised: In this work, we propose DAML-ATM (Domain Adaptive MetaLearning with Adversarial Transfer Model), which consists of two parts: DAML and ATM.

Original: DAML is a domain adaptive metalearning approach to reﬁne general knowledgein multi-heterogeneous source domains, capable of adapting to new unseen domains witha small amount of data.
Revised: DAML is a domain adaptive meta-learning approach to learn general knowledge in multiple heterogeneous source domains, capable of adapting to new unseen domains with a small amount of data.

Original: Moreover, we propose a new unsupervised TST approach Adversarial Transfer Model (ATM), composed ofa sequence-to-sequence pre-trained languagemodel and uses adversarial style training forbetter content preservation and style transfer.
Revised: Moreover, we propose a new unsupervised TST approach Adversarial Transfer Model (ATM), composed of a sequence-to-sequence pre-trained language model and uses adversarial style training for better content preservation and style transfer.

Original: Results on multi-domain datasets demonstratethat our approach generalizes well on unseenlow-resource domains, achieving state-of-theart results against ten-strong baselines.
Revised: Results on multi-domain datasets demonstrate that our approach generalizes well on unseen low-resource domains, achieving state-of-theart results against ten strong baselines.

Original:  Introduction
Revised:  Introduction

Original: Text style transfer (TST) aims to change the styleof the input text and keep its content unchanged,which has been applied successfully to text formalization (Jain et al., 2019) , text rewriting (Nikolovand Hahnloser, 2018) , personalized dialogue generation (Niu and Bansal, 2018) and other stylizedtext generation tasks (Gao et al., 2019; Cao et al. ,2020; Syed et al., 2020).
Revised: Text style transfer (TST) aims to change the style of the input text and keep its content unchanged, which has been applied successfully to text formalization (Jain et al., 2019) , text rewriting (Nikolov and Hahnloser, 2018) , personalized dialogue generation (Niu and Bansal, 2018) and other stylized text generation tasks (Gao et al., 2019; Cao et al., 2020; Syed et al., 2020).

Original: Text style transfer has been explored as asequence-to-sequence learning task using paralleldatasets (Jhamtani et al., 2017; Wang et al., 2020b;
Revised: Text style transfer has been explored as a sequence-to-sequence learning task using parallel datasets (Jhamtani et al., 2017; Wang et al., 2020b;

Original: Pryzant et al., 2020).
Revised: Pryzant et al., 2020).

Original: However, parallel datasets aredifﬁcult to obtain due to expensive manual annotation.
Revised: However, parallel datasets are difﬁcult to obtain due to expensive manual annotation.

Original: The recent surge of deep generative methods(Hu et al., 2017a; Zhao et al., 2017; Li et al., 2018)has spurred progress in text style transfer without parallel data.
Revised: The recent surge of deep generative methods (Hu et al., 2017a; Zhao et al., 2017; Li et al., 2018) has spurred progress in text style transfer without parallel data.

Original: However, these methods typically require large amounts of nonparallel data and  not perform well in low-resource domain scenarios.
Revised: However, these methods typically require large amounts of non-parallel data and do not perform well in low-resource domain scenarios.

Original: One typical method is to resort to massive data from different domains, which has been studied as an effective solution to address the above data insufﬁciency issue (Glorot et al., 2011; Wang et al., 2017).
Revised: One typical method is to resort to massive data from different domains, which has been studied as an effective solution to address the above data insufﬁciency issue (Glorot et al., 2011; Wang et al., 2017; Li et al., 2021b).

Original: However, directly leveraging large amounts of data from other domains for the TST task is problematic due to the differences in data distribution over different domains, as different domains usually use their domain-speciﬁc lexica (Li et al., 2019a).
Revised: However, directly leveraging large amounts of data from other domains for the TST task is problematic due to the differences in data distribution over different domains, as different domains usually use their domain-speciﬁc lexica (Li et al., 2019a).

Original: For instance, ﬁne-tuning a TST model trained on a high-resource movie-related domain to a low-resource restaurant-related domain can get us unreasonable sentences like "the food is dramatic."
Revised: For instance, ﬁne-tuning a TST model trained on a high-resource movierelated domain to a low-resource restaurant-related domain can get us unreasonable sentences like "the food is dramatic."

Original: The sentiment word "dramatic" is  weird to comment on the food but suitable for a movie.
Revised: The sentiment word "dramatic" is suitable for commenting a movie but weird to comment on the food.

Original: In this work, we tackle the problem of domain adaptation in the scenarios where the target domain data is scarce and misaligned with the distribution in the source domain.
Revised: In this work, we tackle the problem of domain adaptation in the scenarios where the target domain data is scarce and misaligned with the distribution in the source domain.

Original: Recently, model-agnostic meta-learning (MAML) has received resurgence in the context of few-shot learning scenario (Lin et al., 2019; Gu et al., 2018; Nooralahzadeh et al., 2020).
Revised: Recently, model-agnostic meta-learning (MAML) has received resurgence in the context of few-shot learning scenario (Lin et al., 2019; Gu et al., 2018; Nooralahzadeh et al., 2020).

Original: Inspired by the essence of MAML (Finn et al., 2017), we propose a new meta-learning training strategy named domain adaptive meta-learning (DAML).
Revised: Inspired by the essence of MAML (Finn et al., 2017), we propose a new meta-learning training strategy named domain adaptive meta-learning (DAML).

Original: Unlike MAML, DAML adopts a domain adaptive approach to construct meta tasks that would be more suitable to learn a robust and generalized initialization for low-resource TST domain adaption.
Revised: Unlike MAML, DAML adopts a domain adaptive approach to construct meta tasks that would be more suitable to learn a robust and generalized initialization for low-resource TST domain adaption.

Original: With the DAML strategy, we design a TST model for each domain. Usually, if a TST model tries to decouple style information from the semantics of a text,  it tends to produce content loss during style transfer (Hu et al., 2017b; Dai et al., 2019; Carlson et al.,2018).
Revised: To well preserve content and transfer style, one typical strategy of a TST model is to decouple style information from the semantics of a text, and it tends to produce content loss during style transfer (Hu et al., 2017b; Dai et al., 2019; Carlson et al.,

Original: Thus, we  propose a new  style transfer model ATM, which is composed of a sequence-to-sequence pre-trained language modelcombined with adversarial style training for styletransfer.
Revised: Here, we do not try to decouple content and style, and propose a new Adversarial style Transfer model ATM, which is composed of a sequenceto-sequence pre-trained language model combined with adversarial style training for style transfer.

Original: In this way, ATM can better preserve thecontent information without disentangling contentand style in the latent space.
Revised: In this way, our model can better preserve the content information without disentangling content and style in the latent space.

Original: Combining DAML and ATM, in this paper, wepropose the method named DAML-ATM, whichextends traditional meta-learning to a domain adaptive method combined with a sequence-to-sequencestyle transfer model.
Revised: Combining DAML and ATM, in this paper, we propose the method named DAML-ATM, which extends traditional meta-learning to a domain adaptive method combined with a sequence-to-sequence style transfer model.

Original: DAML contains two alternating phases.
Revised: DAML contains two alternating phases.

Original: During the meta-training phase, aseries of meta-tasks are constructed from a largepool of source domains for balanced absorption ofgeneral knowledge, resulting in a domain-speciﬁctemporary model.
Revised: During the meta-training phase, a series of meta-tasks are constructed from a large pool of source domains for balanced absorption of general knowledge, resulting in a domain-speciﬁc temporary model.

Original: In the meta validation stage, thetemporary model is evaluated on the meta validation set to minimize domain differences and realizemeta knowledge transfer across different domains.
Revised: In the meta validation stage, the temporary model is evaluated on the meta validation set to minimize domain differences and realize meta knowledge transfer across different domains.

Original: In ATM, a pre-training language model based TSTmodel is used to improve text content retention.
Revised: In ATM, a pre-training language model based TST model is used to improve text content retention.

Original: Moreover, we propose a two-stage training algorithm to combine the DAML training method and
Revised: Moreover, we propose a two-stage training algorithm to combine the DAML training method and

Original: ATM model better.
Revised: ATM model better.

Original: In summary, the main contributions in this paperare three-fold: ( i ) We propose a new unsupervised
Revised: In summary, the main contributions in this paper are three-fold: ( i ) We propose a new unsupervised

Original: TST model, which achieves SOTA performancewithout disentangling content and style latent representations compared to other models.
Revised: TST model, which achieves SOTA performance without disentangling content and style latent representations compared to other models.

Original: ( ii ) Weextend the traditional meta-learning strategy to thedomain adaptive meta transfer method, effectivelyalleviating the domain adaption problem in TST.
Revised: ( ii ) We extend the traditional meta-learning strategy to the domain adaptive meta transfer method, effectively alleviating the domain adaption problem in TST.

Original: ( iii )
Revised: ( iii )

Original: We propose a two-stage training algorithm totrain DAML-ATM, achieving state-of-the-art performance against multiple strong baselines.
Revised: We propose a two-stage training algorithm to train DAML-ATM, achieving state-of-the-art performance against multiple strong baselines.

Original:  Related Work 
Revised:  Related Work 2. Text Style Transfer

Original: 2. Text Style Transfer

Original: Text style transfer based on deep learning has beenextensively studied in recent years.
Revised: Text style transfer based on deep learning has been extensively studied in recent years.

Original: A typical pattern is ﬁrst to separate the latent space as contentand style features, then adjust the style-related features and generate stylistic sentences through thedecoder.
Revised: A typical pattern is ﬁrst to separate the latent space as content and style features, then adjust the style-related features and generate stylistic sentences through the decoder.

Original: (Hu et al., 2017a; Fu et al., 2017;
Revised: (Hu et al., 2017a; Fu et al., 2017;

Original: Li et al. ,2019a)assume that appropriate style regularizationcan achieve the separation.
Revised: Li et al., 2019a)assume that appropriate style regularization can achieve the separation.

Original: Style regularizationmay be implemented as an adversarial discriminator or style classiﬁer in an automatic encodingprocess.
Revised: Style regularization may be implemented as an adversarial discriminator or style classiﬁer in an automatic encoding process.

Original: However, these style transfer paradigmsuse large amounts of annotation data to train models for speciﬁc tasks.
Revised: However, these style transfer paradigms use large amounts of annotation data to train models for speciﬁc tasks.

Original: If we already have a modelfor a similar task, it is unreasonable to need manydata still to train the model from scratch.
Revised: If we already have a model for a similar task, it is unreasonable to need many data still to train the model from scratch.

Original: On the other hand, some of the previous work learned to do TST without manipulating the style of the generated sentence based on this learned latent space.
Revised: On the other hand, some of the previous work learned to do TST without manipulating the style of the generated sentence based on this learned latent space.

Original: (Dai et al., 2019)use the transformer architecture language model to introduce attention mechanism, but they do not make full use of the prior knowledge of sequence to sequence pre-trained language model, such as Bart (Lewis et al., 2019) and T5 (Raffel et al., 2019), which have made signiﬁcant progress in text generation tasks.
Revised: (Dai et al., 2019)use the transformer architecture language model to introduce attention mechanism, but they do not make full use of the prior knowledge of sequence to sequence pre-trained language model, such as Bart (Lewis et al., 2019) and T5 (Raffel et al., 2019), which have made signiﬁcant progress in text generation tasks.

Original: In this paper, we proposed the DAML training method to solve the domain shift problem in TST and proposed a new TST model architecture named ATM, which makes no assumption about the latent representation of source sentence and takes the proven sequence-to-sequence pre-trained language model.
Revised: In this paper, we proposed the DAML training method to solve the domain shift problem in TST and proposed a new TST model architecture named ATM, which makes no assumption about the latent representation of source sentence and takes the proven sequence-to-sequence pre-trained language model.

Original: 2. Domain adaptation
Revised: 2. Domain adaptation

Original: Domain adaptation has been studied in various natural language processing tasks (Glorot et al., 2011;
Revised: Domain adaptation has been studied in various natural language processing tasks (Glorot et al., 2011;

Original: Qian and Yu, 2019; Wang et al., 2017).
Revised: Qian and Yu, 2019; Wang et al., 2017; Li et al., 2021a).

Original: However, there is no recent work about domain adaptation for a TST, except DAST (Li et al., 2019a).
Revised: However, there is no recent work about domain adaptation for a TST, except DAST (Li et al., 2019a).

Original: DAST is a semi-supervised learning method that adapts domain vectors to adapt models learned from multiple source domains to a new target domain via domain discriminator.
Revised: DAST is a semi-supervised learning method that adapts domain vectors to adapt models learned from multiple source domains to a new target domain via domain discriminator.

Original: Different From DAST, we propose to combine meta-learning and adversarial networks to achieve similar domain adaption ability, and our model exceeds the performance of DAST without domain discriminator.
Revised: Different From DAST, we propose to combine meta-learning and adversarial networks to achieve similar domain adaption ability, and our model exceeds the performance of DAST without domain discriminator.

Original: Although there are some methods perform well in few shot data transfer (Riley et al., 2021; Krishna et al., 2021), these methods discuss completely new text style transfer, while we focus on the domain adaptation issue.
Revised: Although there are some methods perform well in few shot data transfer (Riley et al., 2021; Krishna et al., 2021), these methods discuss completely new text style transfer, while we focus on the domain adaptation issue.

Original: 2. Model-Agnostic Meta-Learning
Revised: 2. Model-Agnostic Meta-Learning

Original: Model-agnostic meta-learning (MAML)
Revised: Model-agnostic meta-learning (MAML)

Original: (Finn et al., 2017) provides a general method to adapt to parameters in different domains.
Revised: (Finn et al., 2017) provides a general method to adapt to parameters in different domains.

Original: MAML solves few-shot learning problems by learning a good parameter initialization.
Revised: MAML solves few-shot learning problems by learning a good parameter initialization.

Original: During testing, such initialization can be ﬁne-tuned through a few gradient steps, using a limited number of training examples in the target domain.
Revised: During testing, such initialization can be ﬁne-tuned through a few gradient steps, using a limited number of training examples in the target domain.

Original: Although there have been some researches (Qian and Yu, 2019; Li et al., 2020;
Revised: Although there have been some researches (Qian and Yu, 2019; Li et al., 2020;

Original: Wu et al., 2020) on MAML in natural language processing, it is still scarce compared to computer vision.
Revised: Wu et al., 2020) on MAML in natural language processing, it is still scarce compared to computer vision.

Original: Unlike the above research on classiﬁcation under few-shot learning, our research focuses on text style transfer based on text generation.
Revised: Unlike the above research on classiﬁcation under few-shot learning, our research focuses on text style transfer based on text generation.

Original: In this paper, we seek a new meta-learning strategy combined with adversarial networks, which is moresuitable for encouraging robust domain representation.
Revised: In this paper, we seek a new meta-learning strategy combined with adversarial networks, which is more suitable for encouraging robust domain representation.

Original: As far as we know, we are the ﬁrst to adoptmeta-learning in the domain adaptation problem oftext style transfer tasks.
Revised: As far as we know, we are the ﬁrst to adopt meta-learning in the domain adaptation problem of text style transfer tasks.

Original:  Methodology
Revised:  Methodology

Original: In this section, we ﬁrst deﬁne the problem of domain adaptive learning for TST.
Revised: In this section, we ﬁrst deﬁne the problem of domain adaptive learning for TST.

Original: Then we describeour approach, DAML-ATM, in detail.
Revised: Then we describe our approach, DAML-ATM, in detail.

Original: 3. Task Deﬁnition
Revised: 3. Task Deﬁnition

Original: Let D S
Revised: Let D S

Original: = { D 1 , ..., D N } be N source domainsin the training phase, where D n (1 ≤ n ≤ N ) isthe n -th source domain containing style-labellednon-parallel data D n = { ( X i , l i ) } 
Revised: = { D 1 , ..., D N } be N source domains in the training phase, where D n (1 ≤ n ≤ N ) is the n -th source domain containing style-labelled non-parallel data D n = { ( X i , l i ) } L n i

Original: L n i =1 , where L n isthe total number of sentences, X i denotes the i thsource sentence, and l i denotes the correspondingstyle label, which belongs to a source style labelset: l i ∈ L S (e.g., positive/negative).
Revised: =1 , where L n is the total number of sentences, X i denotes the i th source sentence, and l i denotes the corresponding style label, which belongs to a source style label set: l i ∈ L S (e.g., positive/negative).

Original: Likewise,there are K target domains D T
Revised: Likewise, there are K target domains D T

Original: = { D 1 , ..., D K }which are unseen in D S .
Revised: = { D 1 , ..., D K } which are unseen in D S .

Original: Our task is to transfera sentence X i with style l i in the target domainto another sentence Y (cid:48)
Revised: Our task is to transfer a sentence X i with style l i in the target domain to another sentence Y (cid:48)

Original: i sharing the same contentwhile having a different style ˜ l i from l i and domainspeciﬁc characteristics of the target domain.
Revised: i sharing the same content while having a different style ˜ l i from l i and domainspeciﬁc characteristics of the target domain.

Original: We propose a two-stage algorithm for domainadaptation in TST: pre-training learning strategyand domain adaptive meta-learning strategy.
Revised: We propose a two-stage algorithm for domain adaptation in TST: pre-training learning strategy and domain adaptive meta-learning strategy.

Original: Inpre-training learning , our objective is to make themodel more able to preserve content informationand distinguish between different text styles.
Revised: In pre-training learning, our objective is to make the model more able to preserve content information and distinguish between different text styles.

Original: Indomain adaptive meta-learning, our objective is tolearn a meta-knowledge learner for the sequenceto-sequence model by leveraging sufﬁcient sourcedata D s .
Revised: In domain adaptive meta-learning, our objective is to learn a meta-knowledge learner for the sequenceto-sequence model by leveraging sufﬁcient source data D s .

Original: Given a new unseen domain from D new ,the new learning task of TST can be solved by ﬁnetuning the learned sequence-to-sequence model(domain-invariant parameters) with only a smallnumber of training samples.
Revised: Given a new unseen domain from D new , the new learning task of TST can be solved by ﬁnetuning the learned sequence-to-sequence model (domain-invariant parameters) with only a small number of training samples.

Original: 3. DAML-ATM Approach 
Revised: 3. DAML-ATM Approach 3.2. Overview of DAML

Original: Overview of DAML

Original: Model-agnostic meta-learning can utilize a fewtraining samples to train a model with good generalization ability.
Revised: Model-agnostic meta-learning can utilize a few training samples to train a model with good generalization ability.

Original: However, since it is based onthe assumption that the meta tasks are from thesame distribution (Figure 1, left), simply feedingall the sources data into it might get sub-optimal results (Chen and Zhu, 2020).
Revised: However, since it is based on the assumption that the meta tasks are from the same distribution (Figure 1, ), simply feeding all the sources data into it might get sub-optimal results (Chen and Zhu, 2020).

Original: Therefore, we propose a modiﬁed way to construct meta tasks (Figure 1, right).
Revised: Therefore, we propose a modiﬁed way to construct meta tasks (Figure 1, right).

Original: Different from MAML, for DAML, in one batch, the data in each meta task comes from the same source domain, and each meta task comes from a different domain.
Revised: Different from MAML, for DAML, in one batch, the data in each meta task comes from the same source domain, and each meta task comes from a different domain.

Original: In this way, we can guarantee that DAML can learn generic representations from different domains in a balanced way.
Revised: In this way, we can guarantee that DAML can learn generic representations from different domains in a balanced way.

Original: During each iteration, we randomly split all source domains into a meta-training set D tr and a metavalidation setD val , where D S = D tr ∪ D val and
Revised: During each iteration, we randomly split all source domains into a meta-training set D tr and a meta-validation set

Original: D  tr ∩ D val = ∅ .
Revised: D val , where D S = D tr ∪ D val and D tr ∩ D val = ∅ .

Original: A meta-training task T i is sampled from D tr and is composed of n instances from a speciﬁc domain.
Revised: A meta-training task T i is sampled from D tr and is composed of n instances from a speciﬁc domain.

Original: Likewise, a meta-validation task T j is sampled from D val .
Revised: Likewise, a meta-validation task T j is sampled from D val .

Original: The validation errors on D val should be considered to improve the robustness of the model.
Revised: The validation errors on D val should be considered to improve the robustness of the model.

Original: In short, with DAML, the parameters learned by the model in the parameter space are not biassed towards any one particular domains with as little data as possible during model updating as shown in Figure 1(right).
Revised: In short, with DAML, the parameters learned by the model in the parameter space are not biassed towards any one particular domain with as little data as possible during model updating as shown in Figure 1(right).

Original: In the ﬁnal evaluation phase, the metaknowledge learned by the sequence-to-sequence model can be applied to new domains.
Revised: In the ﬁnal evaluation phase, the metaknowledge learned by the sequence-to-sequence model can be applied to new domains.

Original: Given a new unseen domain D new =
Revised: Given a new unseen domain D new =

Original: ( T tr , T te ) , the learned sequence-to-sequence model and the discriminator are ﬁne-tuned on T tr and ﬁnally tested on T te .
Revised: ( T tr , T te ) , the learned sequence-to-sequence model and the discriminator are ﬁne-tuned on T tr and ﬁnally tested on T te .

Original: 3.2. ATM Model
Revised: 3.2. ATM Model

Original: In this section, we give a brief introduction to our proposed model: ATM , which combines sequenceto-sequence pre-trained model with adversarial training.
Revised: In this section, we give a brief introduction to our proposed model: ATM , which combines sequenceto-sequence pre-trained model with adversarial training.

Original: (1) For the content preservation, we train the sequence-to-sequence model θ to reconstruct the original input sentence X with the original style label l .
Revised: (1) For the content preservation, we train the sequence-to-sequence model θ to reconstruct the original input sentence X with the original style label l .

Original: For the style controlling, we train a discriminator network γ to assist the sequence-tosequence model network in better controlling the style of the generated sentence.
Revised: For the style controlling, we train a discriminator network γ to assist the sequence-tosequence model network in better controlling the style of the generated sentence.

Original: The structure ofthe model is shown in Figure 2.
Revised: The structure of the model is shown in Figure 2.

Original: S2S-model To ease the explanation, we startwith the sequence-to-sequence (S2S) modelhere.
Revised: S2S-model To ease the explanation, we start with the sequence-to-sequence (S2S) model here.

Original: Explicitly, for an input sentence X =( x 1 , x 2 , ..., x n ) of length n , X ∈
Revised: Explicitly, for an input sentence X = ( x 1 , x 2 , ..., x n ) of length n , X ∈

Original: D , the
Revised: D , the

Original: S2S encoder
Revised: S2S encoder

Original: Enc
Revised: Enc

Original: ( X ; θ E ) maps inputs to asequence of continuous hidden representations
Revised: ( X ; θ E ) maps inputs to a sequence of continuous hidden representations

Original: H =
Revised: H =

Original: ( h 1 , h 2 , ..., h n ) .
Revised: ( h 1 , h 2 , ..., h n ) .

Original: Then, the S2S decoder
Revised: Then, the S2S decoder

Original: Dec ( H ; θ D ) estimates the conditional probabilityfor the output sentence Y =
Revised: Dec ( H ; θ D ) estimates the conditional probability for the output sentence Y =

Original: ( y 1 , y 2 , ..., y n ) byauto-regressively factorized its as:
Revised: ( y 1 , y 2 , ..., y n ) by auto-regressively factorized its as:

Original: [Equation]
Revised: [Equation]

Original: At each time step t , the probability of the next tokenis computed by a softmax classiﬁer:
Revised: At each time step t , the probability of the next token is computed by a softmax classiﬁer:

Original: [Equation]
Revised: [Equation]

Original: where o t is logit vector outputted by decoder network.
Revised: where o t is logit vector outputted by decoder network.

Original: The standard S2S model without discriminator makes the output sequence Y the same as theinput sequence X .
Revised: The standard S2S model without discriminator makes the output sequence Y the same as the input sequence X .

Original: Discriminator Model By teacher forcing, S2Stends to ignore the style labels and collapses to areconstruction model, which might copy the inputsentence, hence failing to transfer the style.
Revised: Discriminator Model By teacher forcing, S2S tends to ignore the style labels and collapses to a reconstruction model, which might copy the input sentence, hence failing to transfer the style.

Original: Therefore, to make the model learn meaningful styleinformation, we apply a style discriminator γ forthe style regularization.
Revised: Therefore, to make the model learn meaningful style [Table] information, we apply a style discriminator γ for the style regularization.

Original: In summary, we use a stylediscriminator to provide the direction (gradient) for
Revised: In summary, we use a style discriminator to provide the direction (gradient) for

Original: TST to conform to the target style.
Revised: TST to conform to the target style.

Original: Our discriminator is a multi-layer perceptron with a sigmoid

Original: Input: sequence-to-sequence model f θ ,discriminator γ ,and a dataset D i with style l i belong to L s Output: well-trained parameter θ, γ 1: Sample a batch of m sentences X 1 , X 2 , ...

Original: X m from D i .

Original: 2: while in ﬁrst stage and not convergence do 3:

Original: Use f θ to generate new sentence 4:

Original: Y i =

Original: f θ ( X i , l i ) 5: Compute L cls ( γ ) for Y i by Eq. Compute L rec ( θ ) for Y i by Eq. (3) ; activation function to predict style labels or guide the direction of style transfer.
Revised: Our discriminator is a multi-layer perceptron with a sigmoid activation function to predict style labels or guide the direction of style transfer.

Original: Our model training involves a pre-training learning strategy and a domain adaptive meta-learning strategy.
Revised: Our model training involves a pre-training learning strategy and a domain adaptive meta-learning strategy.

Original: 3.2. First Stage: Pre-training Learning
Revised: 3.2. First Stage: Pre-training Learning

Original: In the ﬁrst stage, we train the discriminator model to distinguish different text styles.
Revised: In the ﬁrst stage, we train the discriminator model to distinguish different text styles.

Original: In this stage, the discriminator models are equivalent to a text classiﬁer.
Revised: In this stage, the discriminator models are equivalent to a text classiﬁer.

Original: Inspired by (Lewis et al., 2019), we feed the hidden states from the last layer of the decoder into the classiﬁer instead of the gumblesoftmax trick (Jang et al., 2017) for gradient backpropagation, which is more stable and better than gumble-softmax(See Table 5).
Revised: Inspired by (Lewis et al., 2019), we feed the hidden states from the last layer of the decoder into the classiﬁer instead of the gumblesoftmax trick (Jang et al., 2017) for gradient backpropagation, which is more stable and better than gumble-softmax(See Table 5).

Original: The loss function for the discriminator is simply the cross-entropy loss of the classiﬁcation problem:
Revised: The loss function for the discriminator is simply the cross-entropy loss of the classiﬁcation problem:

Original: [Equation]
Revised: [Equation]

Original: For the S2S model, we pre-train the S2S model to allow the generation model to learn to copy an input sentence X using teacher forcing.
Revised: For the S2S model, we pre-train the S2S model to allow the generation model to learn to copy an input sentence X using teacher forcing.

Original: The loss function of the sequence-to-sequence model minimizes the negative log-likelihood of the training data:
Revised: The loss function of the sequence-to-sequence model minimizes the negative log-likelihood of the training data:

Original: [Equation]
Revised: [Equation]

Original: In summary, we train the sequence model and the style classiﬁcation model separately on the source domain to learn content preservation and style discrimination in the ﬁrst stage.
Revised: In summary, we train the sequence model and the style classiﬁcation model separately on the source domain to learn content preservation and style discrimination in the ﬁrst stage.

Original: The ﬁrst stage training procedure of the ATM is summarized in Algorithm 1.
Revised: The ﬁrst stage training procedure of the ATM is summarized in Algorithm 1.

Original: 3.2. Second Stage: Domain Adaptive Meta

Original: Learning with Adversarial Training

Original: After the ﬁrst stage of training, the style classiﬁer has learned how to distinguish between different text styles.

Original: For style controlling, we adopt a method

Original: [Figure]
Revised: [Figure]

Original: Encoder
Revised: Encoder

Original: Decoder Domain
Revised: Decoder Domain

Original: [Figure]
Revised: [Figure]

Original: Domain N
Revised: Domain N

Original: Split
Revised: Split

Original: Meta-training
Revised: Meta-training

Original: Meta-validation
Revised: Meta-validation

Original: Style Discriminator
Revised: Style Discriminator

Original: Unseen Domain
Revised: Unseen Domain

Original: Encoder
Revised: Encoder

Original: Decoder …
Revised: Decoder …

Original: Style Discriminator
Revised: Style Discriminator

Original: Training Phase
Revised: Training Phase

Original: Final Evaluation Phase
Revised: Final Evaluation Phase

Original: Transfer Meta knowledge to new domain
Revised: Transfer Meta knowledge to new domain

Original: Final test
Revised: Final test

Original: Fine-tuning Few shot
Revised: Fine-tuning Few shot

Original: Y'
Revised: Y'

Original: Y' ), ( lX T i ), ( lX T j
Revised: Y' ), ( lX T i ), ( lX T j

Original: Adversarial
Revised: Adversarial

Original: [Figure]
Revised: [Figure]

Original: styleT L recT L recT
Revised: styleT L recT L recT

Original: tr L styleT tr L
Revised: tr L styleT tr L

Original: AdversarialTraining
Revised: Adversarial

Original: [Figure]
Revised: [Figure]

Original:  of adversarial training
Revised: After the ﬁrst stage of training, the style classiﬁer has learned how to distinguish between different text styles.

Original:  to avoid disentangling thecontent and style in the latent space.
Revised: For style controlling, we adopt a method of adversarial training to avoid disentangling the content and style in the latent space.

Original: The discriminator model aims to minimize the negative loglikelihood of opposite style
Revised: The discriminator model aims to minimize the negative loglikelihood of opposite style

Original: ˜ l i when feeding to thesequence model sentence
Revised: ˜ l i when feeding to the sequence model sentence

Original: X
Revised: X

Original: i with the style labell i .
Revised: i with the style label l i .

Original: In the second stage, we freeze the parameters ofthe discriminator.
Revised: In the second stage, we freeze the parameters of the discriminator.

Original: Therefore, style loss only workson the S2S model θ , which forces the S2S model θto generate opposite styles of sentences:
Revised: Therefore, style loss only works on the S2S model θ , which forces the S2S model θ to generate opposite styles of sentences:

Original: [Equation]
Revised: [Equation]

Original: In the second stage, we use the DAML algorithmfor domain adaptive TST, so the text reconstructionloss and the style discriminator loss are calculatedover the meta-training samples in task T i from D tr .
Revised: In the second stage, we use the DAML algorithm for domain adaptive TST, so the text reconstruction loss and the style discriminator loss are calculated over the meta-training samples in task T i from D tr .

Original: These two losses can be written as
Revised: These two losses can be written as

Original: [Equation]
Revised: [Equation]

Original: We add different preﬁxes to the input in the second stage, which allows the S2S model to perceivedifferent TST tasks.
Revised: We add different preﬁxes to the input in the second stage, which allows the S2S model to perceive different TST tasks.

Original: The second stage of the algorithm is called domain adaptive meta-strategy,which consists of two core phases: a meta-trainingphase and a meta-validation phase, as shown in
Revised: The second stage of the algorithm is called domain adaptive meta-strategy, which consists of two core phases: a meta-training phase and a meta-validation phase, as shown in

Original: Figure 3.
Revised: Figure 3.

Original: Domain Adaptive Meta-Training.
Revised: Domain Adaptive Meta-Training

Original: In the meta-training phase, our objective is to learn different domain-speciﬁc temporary models for each domain that are capable of learning the general knowledge of each domain.
Revised: In the meta-training phase, our objective is to learn different domain-speciﬁc temporary models for each domain that are capable of learning the general knowledge of each domain.

Original: Inspired by feature-critic networks (Li et al., 2019b), we use a similar manner to adapt the parameters of the domain-speciﬁc temporary model:
Revised: Inspired by feature-critic networks (Li et al., 2019b), we use a similar manner to adapt the parameters of the domain-speciﬁc temporary model:

Original: [Equation]
Revised: [Equation]

Original: where i is the adaptation step in the inner loop, and α is the learning rate of the internal optimization.
Revised: where i is the adaptation step in the inner loop, and α is the learning rate of the internal optimization.

Original: At each adaptation step, the gradients are calculated with respect to the parameters from the previous step.
Revised: At each adaptation step, the gradients are calculated with respect to the parameters from the previous step.

Original: For each domain of D tr , it has different θ old
Revised: For each domain of D tr , it has different θ old

Original: and θ new .
Revised: and θ new .

Original: The base model parameters θ 0 should not be changed in the inner loop. 
Revised: The base model parameters θ 0 should not be changed in the inner loop. Algorithm ATM

Original: [Table]
Revised: [Table]

Original: do //Outer loop 6: Sample a task T j from D val 7: for i in adaptation steps do //Inner loop 8:
Revised: D val 7: for i in adaptation steps do //Inner loop 8:

Original: Sample a task T i from D tr 9:
Revised: Sample a task T i from D tr 9:

Original: Compute meta-training rec loss L recT i 10:
Revised: Compute meta-training rec loss L recT i 10:

Original: Compute meta-training style loss L styleT j 11:
Revised: Compute meta-training style loss L style T j 11:

Original: Compute adapted parameters with gradient descent for θ
Revised: Compute adapted parameters with gradient descent for θ

Original: i
Revised: i

Original: −12: θ 
Revised: −12: θ oldi

Original: [Equation] [Equation]
Revised: = θ i − 1 − α ∇ θ i − 1 L trT i ( θ i − 1 , γ i − 1 ) 13: θ newi = θ oldi − 1 − α ∇ θ i − 1 L styleT

Original:  1 ) 13: θ [Equation] [Equation]
Revised: i ( θ i − 1 , γ i − 1 ) 14:

Original: Meta-validation: 15:
Revised: Meta-validation: 15:

Original: Compute meta-validation loss on T j : L valT j 16:
Revised: Compute meta-validation loss on T j : L valT j 16:

Original: Meta-optimization : 17:
Revised: Meta-optimization : 17:

Original: Perform gradient step w.r.t.
Revised: Perform gradient step w.r.t.

Original: θ 18: {+θ 
+}
Revised: θ 18: {+θ 
+}

Original: [Equation] [Equation] [Equation]
Revised: = θ 0 − β ∇ θ 0 E T j L valT j ( θ oldi , θ newi , γ )

Original: Domain Adaptive Meta-Validation
Revised: Domain Adaptive Meta-Validation

Original: After meta-training phase, DAML-ATM has already learned a temporary model( θ oldi , θ newi ) in the meta-training domains D tr .
Revised: After meta-training phase, DAML-ATM has already learned a temporary model( θ oldi , θ newi ) in the meta-training domains D tr .

Original: The meta-validation phase tries to minimize the distribution divergence between the source domains D tr and simulated target domains D val using the learned temporary model.
Revised: The meta-validation phase tries to minimize the distribution divergence between the source domains D tr and simulated target domains D val using the learned temporary model.

Original: In the meta-validation phase, each temporary model is calculated on the meta-validation domain D val to get meta validation losses.
Revised: In the meta-validation phase, each temporary model is calculated on the meta-validation domain D val to get meta validation losses.

Original: [Equation]
Revised: [Equation]

Original: Thus, the base model θ is updated by gradient descent
Revised: Thus, the base model θ is updated by gradient descent

Original: [Equation]
Revised: [Equation]

Original: where β is the meta-learning rate.
Revised: where β is the meta-learning rate.

Original: Unlike the ordinary gradient descent process, the update mechanism of Eq.
Revised: Unlike the ordinary gradient descent process, the update mechanism of Eq.

Original: (9) involves updating one gradient byanother gradient (w.r.t.
Revised: (9) involves updating one gradient by another gradient (w.r.t.

Original: the parameters of the temporary model).
Revised: the parameters of the temporary model).

Original: This process requires a second-orderoptimization partial derivative.
Revised: This process requires a second-order optimization partial derivative.

Original: 3.2. Final Evaluation Phase of DAML-ATM
Revised: 3.2. Final Evaluation Phase of DAML-ATM

Original: In the ﬁnal evaluation phase, we ﬁrst initialize themodel with the parameters learned during the abovealgorithm 2.
Revised: In the ﬁnal evaluation phase, we ﬁrst initialize the model with the parameters learned during the above algorithm 2.

Original: Then, the model takes input as a newadaptation task T , which consists of a small indomain data S tr for ﬁne-tuning the model and a testset S te for testing.
Revised: Then, the model takes input as a new adaptation task T , which consists of a small indomain data S tr for ﬁne-tuning the model and a test set S te for testing.

Original: The procedure is summarizedin Algorithm 3.
Revised: The procedure is summarized in Algorithm 3.

Original: (Note that the discriminator is notneeded for inference.)
Revised: (Note that the discriminator is not needed for inference.)

Original: Algorithm {+
+}
Revised: Algorithm {+
+}

Original: DAML-ATM
Revised: The Input: θ, γ learned from Algorithm 2, low resource training set S tr and test set S te of an unseen domain D new Output: Performance on S te 1: while not convergence do 2: Serialize a task T tr from the unseen domain S tr 3: Update θ = θ − β ∇ θ (cid:80) T tr

Original: [Table]
Revised: ( L recT tr ( θ ) + L styleT tr ( θ )) 4: return optimal θ ∗ for S te 5: Style accuracy, bleu, domain accuracy = f T te ( θ )

Original: [Table]
Revised: Dataset

Original: Domain
Revised: Domain

Original: Train
Revised: Train

Original: Dev
Revised: Dev

Original: Test
Revised: Test

Original: Human Reference
Revised: Human Reference

Original: Yelp
Revised: Yelp

Original: Restaurant 444k 4k 1k 1k
Revised: Restaurant 444k 4k 1k 1k

Original: Amazon
Revised: Amazon

Original: Product 554k 2k 1k 1k
Revised: Product 554k 2k 1k 1k

Original: IMDB
Revised: IMDB

Original: Movie 341k 2k 1k
Revised: Movie 341k 2k 1k

Original: No
Revised: No

Original: Yahoo!
Revised: Yahoo!

Original: Q & [Table] 5k 1k 1k
Revised: Q & A 5k 1k 1k

Original: No
Revised: No

Original:  Experiment
Revised:  Experiment

Original: [Figure]

Original: In this section, we ﬁrst detail the experimental setups.
Revised: In this section, we ﬁrst detail the experimental setups.

Original: Then, we present our experimental resultsover multiple target domains.
Revised: Then, we present our experimental results over multiple target domains.

Original: 4. Datasets and Experimental Setups
Revised: 4. Datasets and Experimental Setups

Original: In this experiment, we use the following fourdatasets from different domains: (i) IMDB moviereview corpus (Diao et al., 2014).
Revised: In this experiment, we use the following four datasets from different domains: (i) IMDB movie review corpus (Diao et al., 2014).

Original: (ii) Yelp restaurant review dataset (Li et al., 2018).
Revised: (ii) Yelp restaurant review dataset (Li et al., 2018).

Original: (iii) Amazon product review dataset (Li et al., 2018).
Revised: (iii) Amazon product review dataset (Li et al., 2018).

Original: (iv)
Revised: (iv)

Original: YAHOO!
Revised: YAHOO!

Original: Answers dataset (Li et al., 2019a), the amazon and yelp test sets each have 1k human annotations.
Revised: Answers dataset (Li et al., 2019a), the amazon and yelp test sets each have 1k human annotations.

Original: The statistics of these corpora are summarized in Table 1.
Revised: The statistics of these corpora are summarized in Table 1.

Original: For the S2S model, we take the T5 base model (Raffel et al., 2019)
Revised: For the S2S model, we take the T5 base model (Raffel et al., 2019)

Original: (220MB) for our experiments.
Revised: (220MB) for our experiments.

Original: For style discriminator, we use 4-layer fully connected neural networks.
Revised: For style discriminator, we use 4-layer fully connected neural networks.

Original: We train our framework using the Adam optimizer (Kingma and Ba, 2014)with the initial learning rate 1e-5.
Revised: We train our framework using the Adam optimizer (Kingma and Ba, 2014)with the initial learning rate 1e-5.

Original: The epoch is set to 50 for both stage 1 and stage 2.
Revised: The epoch is set to 50 for both stage 1 and stage 2.

Original: The inner learning rate α is 0.0001, and the outer learning rate β is 0.001.
Revised: The inner learning rate α is 0.0001, and the outer learning rate β is 0.001.

Original: Following (Shankar et al., 2018; Li et al., 2020), we use the leave-one-out evaluation method by picking a domain as the target domain
Revised: Following (Shankar et al., 2018; Li et al., 2020), we use the leave-one-out evaluation method by picking a domain as the target domain

Original: D new for the ﬁnal evaluation.
Revised: D new for the ﬁnal evaluation.

Original: For each iteration of the training phase, two source domains are randomly selected as the meta-training domain D tr and the remaining domains as the meta-validation domain D val .
Revised: For each iteration of the training phase, two source domains are randomly selected as the meta-training domain D tr and the remaining domains as the meta-validation domain D val .

Original: In order to evaluate the model performance, we use three famous and widely adopted automatic metrics following previous work (Li et al., 2019a;
Revised: In order to evaluate the model performance, we use three famous and widely adopted automatic metrics following previous work (Li et al., 2019a;

Original: Fu et al., 2017; Hu et al., 2017a) and a human metric.
Revised: Fu et al., 2017; Hu et al., 2017a) and a human metric.

Original: BLEU veriﬁes whether the generated sentences retain the original content (Papineni et al., 2002).
Revised: BLEU veriﬁes whether the generated sentences retain the original content (Papineni et al., 2002).

Original: While IMDB and Amazon have no manual references, we compute the BLEU scores w.r.t the input sentences.
Revised: While IMDB and Amazon have no manual references, we compute the BLEU scores w.r.t the input sentences.

Original: Style Control (S-Acc) measures the style accuracy of the transferred sentences with a style classiﬁer that is pre-trained on the datasets.
Revised: Style Control (S-Acc) measures the style accuracy of the transferred sentences with a style classiﬁer that is pre-trained on the datasets.

Original: Domain Control (D-Acc) veriﬁes whether the generated sentences have the characteristics of the target domain with a pre-trained domain classiﬁer to measure the percentage of generated sentences belonging to the target domain.
Revised: Domain Control (D-Acc) veriﬁes whether the generated sentences have the characteristics of the target domain with a pre-trained domain classiﬁer to measure the percentage of generated sentences belonging to the target domain.

Original: Human
Revised: Human

Original: Evaluation Following (Madotto et al., 2019), We randomly sampled 100 sentences generated on the target domain and distributed a questionnaire at
Revised: Evaluation Following (Madotto et al., 2019), We randomly sampled 100 sentences generated on the target domain and distributed a questionnaire at

Original: Amazon Mechanical Turk asking each worker to rank the content retention (0 to 5), style transfer( to 5 ) and ﬂuency(0 to 5): human score =
Revised: Amazon Mechanical Turk asking each worker to rank the content retention (0 to 5), style transfer( to 5 ) and ﬂuency(0 to 5): human score =

Original: Average ( (cid:80) score style + (cid:80) score content + 452 (cid:80) score fluency ) , human score ∈ [0 , 100] .
Revised: Average ( (cid:80) score style [Equation] score fluency ) , human score ∈ [0 , 100] .

Original: Five workers were recruited for human evaluation.
Revised: Five workers were recruited for human evaluation.

Revised: [Table]

Revised: [Table]

Original: The results of the other metrics are shown in the appendix.
Revised: The results of the other metrics are shown in the appendix.

Original: 4. Baselines
Revised: 4. Baselines

Original:  In our experiments, for ATM model, we adopt ﬁve state-of-the-art TST models for comparison:[Table]
Revised: [Table] In our experiments, for ATM model, we adopt ﬁve state-of-the-art TST models for comparison:

Original: [Table] [Table] CrossAlign (Shen et al., 2017), ControlGen (Huet al., 2017a), DAST (Li et al., 2019a), CatGen (Wang et al., 2020a) and FGIM (Wang et al.,2019).
Revised: CrossAlign (Shen et al., 2017), ControlGen (Hu et al., 2017a), DAST (Li et al., 2019a), CatGen (Wang et al., 2020a) and FGIM (Wang et al., 2019).

Original: They are jointly trained on the source domains and ﬁne-tuned on the target domain.
Revised: They are jointly trained on the source domains and ﬁne-tuned on the target domain.

Original: To well analyze our training method DAML, following (Li et al., 2020), we also use ﬁve simpleand effective domain adaptation settings with ControlGen (Hu et al., 2017a) structure as DAML: (1)
Revised: To well analyze our training method DAML, following (Li et al., 2020), we also use ﬁve simple and effective domain adaptation settings with ControlGen (Hu et al., 2017a) structure as DAML: (1)

Original: In-Domain method is trained on the training setof the target domain; (2) Joint-Training methodcombines all the training sets of the source andtarget domains and performs a joint-training on these datasets; (3) Fine-Tuning method is trained on the training sets of the source domains and then ﬁne-tuned on the training set of the target domain; (4) D-Shift This is trained on the combination of training sets from all source domains.
Revised: In-Domain method is trained on the training set of the target domain; (2) Joint-Training method combines all the training sets of the source and target domains and performs a joint-training on these datasets; (3) Fine-Tuning method is trained on the training sets of the source domains and then ﬁne-tuned on the training set of the target domain; (4) D-Shift This is trained on the combination of training sets from all source domains.

Original: Then, the evaluation is conducted on the test set of a target domain using the direct domain shift strategy; (5) MAML method uses classical model agnostic meta-learning algorithm
Revised: Then, the evaluation is conducted on the test set of a target domain using the direct domain shift strategy; (5) MAML method uses classical model agnostic meta-learning algorithm

Original: (Finn et al., 2017).
Revised: (Finn et al., 2017).

Original: 4. Results and Analysis
Revised: 4. Results and Analysis

Original: For DAML-ATM, we ﬁrst choose restaurant as the target domain and the other three as the source domains for observation.
Revised: For DAML-ATM, we ﬁrst choose restaurant as the target domain and the other three as the source domains for observation.

Original: Table 2 reports the results of different methods and models under both thefull-data and few-shot settings.
Revised: Table 2 reports the results of different methods and models under both the full-data and few-shot settings.

Original: From this table,we can see that DAML-ATM outperforms all baselines in terms of S-Acc , BLEU , D-Acc and humanevaluation.
Revised: From this table, we can see that DAML-ATM outperforms all baselines in terms of S-Acc , BLEU , D-Acc and human evaluation.

Original: We attribute this to the fact that DAMLATM explicitly simulates the domain shift duringtraining via DAML, which helps adapt to the newtarget domain.
Revised: We attribute this to the fact that DAMLATM explicitly simulates the domain shift during training via DAML, which helps adapt to the new target domain.

Original: We can also see that in the caseof a few-shot setting, the results of Fine-tuningand Joint training are even worse than In-domainand DAML.
Revised: We can also see that in the case of a few-shot setting, the results of Fine-tuning and Joint training are even worse than In-domain and DAML.

Original: The reason may be that the data sizeof the source domain is much larger than the target domain so that the model tends to rememberthe characteristics of the source domain.
Revised: The reason may be that the data size of the source domain is much larger than the target domain so that the model tends to remember the characteristics of the source domain.

Original: MAMLachieves good performance in most metrics.
Revised: MAML achieves good performance in most metrics.

Original: However, it does not balance meta-tasks across differentsource domains, performing poorly on D-acc.
Revised: However, it does not balance meta-tasks across different source domains, performing poorly on D-acc.

Original: Further, to verify the robustness of our methodunder the low-resource setting, we separately selectthe other three domains as the target domain.
Revised: Further, to verify the robustness of our method under the low-resource setting, we separately select the other three domains as the target domain.

Original: Asshown in Table 4, our approach has achieved goodperformance on different target domains.
Revised: As shown in Table 4, our approach has achieved good performance on different target domains.

Original: We also provide some examples in Table 3.
Revised: We also provide some examples in Table 3.

Original: Fromthe example, we can see intuitively that D-shiftand Fine-tuning will lead to the misuse of domainspeciﬁc words due to lack of target domain information.
Revised: From the example, we can see intuitively that D-shift and Fine-tuning will lead to the misuse of domainspeciﬁc words due to lack of target domain information.

Original: In addition, compared with Joint-training ,the sentences generated by DAML-ATM are moreconsistent with the human reference.
Revised: In addition, compared with Joint-training , the sentences generated by DAML-ATM are more consistent with the human reference.

Original: Compared to
Revised: Compared to

Original: MAML, DAML generates sentences that are morediverse and vivid due to the more balanced absorption of information from multiple domains.
Revised: MAML, DAML generates sentences that are more diverse and vivid due to the more balanced absorption of information from multiple domains.

Original: Figure 4 shows the system performance positively correlates with the amount of training data available in the target domain.
Revised: Figure 4 shows the system performance positively correlates with the amount of training data available in the target domain.

Original: To visualize how well DAMLATM performs on the new unseen domain, we use t-SNE (Van der Maaten and Hinton, 2008) plots to analyze the degree of separation between the source domain sentences and the generated target domain sentences.
Revised: To visualize how well DAMLATM performs on the new unseen domain, we use t-SNE (Van der Maaten and Hinton, 2008) plots to analyze the degree of separation between the source domain sentences and the generated target domain sentences.

Original: Figure 5 shows that as the training epoch increases, the sentences generated by DAML-ATM in the target domain are completely separated from the source domain in the latent space.
Revised: Figure 5 shows that as the training epoch increases, the sentences generated by DAML-ATM in the target domain are completely separated from the source domain in the latent space.

Original: 4. Ablation Study
Revised: 4. Ablation Study

Original: To study the impact of different components on the overall performance, we further did an ablation study for our model, and the results are shown in
Revised: To study the impact of different components on the overall performance, we further did an ablation study for our model, and the results are shown in

Original: Table 5.
Revised: Table 5.

Original: After we disabled the reconstruction loss, our model failed to learn meaningful outputs and only learned to generate a word for any combination of input sentences and styles.Then, when the
Revised: After we disabled the reconstruction loss, our model failed to learn meaningful outputs and only learned to generate a word for any combination of input sentences and styles.

Original: [Table] discriminator loss is not used, the model degrades[Table] rapidly, simply copying the original sentence without any style modiﬁcation.
Revised: Then, when the discriminator loss is not used, the model degrades rapidly, simply copying the original sentence without any style modiﬁcation.

Original: After not using the pre-training language model weights, the model’s performance is reduced in the metric of content preservation.
Revised: After not using the pre-training language model weights, the model’s performance is reduced in the metric of content preservation.

Original: When using gumble-softmax instead of hidden states for gradient descent, the model performs poorly in style accuracy because of the instability of gumble-softmax.
Revised: When using gumble-softmax instead of hidden states for gradient descent, the model performs poorly in style accuracy because of the instability of gumble-softmax.

Original: In summary, each factor plays an essential role in the DAML-ATM training stage.
Revised: In summary, each factor plays an essential role in the DAML-ATM training stage.

Original: [Table]
Revised: [Table]

Original: dataset.

Original: The size of adaptation training data is 1 % .
Revised: The size of adaptation training data is 1 % .

Original:  Conclusion
Revised:  Conclusion

Original: In this paper, we propose DAML-ATM, a novel training strategy combined with a new TST model for domain adaptation, which can be easily adapted to new domains with few shot data.
Revised: In this paper, we propose DAML-ATM, a novel training strategy combined with a new TST model for domain adaptation, which can be easily adapted to new domains with few shot data.

Original: On four popular TST benchmarks, we found signiﬁcant improvements against multiple baselines, verifying the effectiveness of our method.
Revised: On four popular TST benchmarks, we found signiﬁcant improvements against multiple baselines, verifying the effectiveness of our method.

Original: We explore extending this approach for other low resource NLP tasks in future work.
Revised: We explore extending this approach for other low resource NLP tasks in future work.

Original: A Appendix
Revised: Acknowledgements

Revised: This work was partially supported by National Key Research and Development Project (2019YFB1704002) and National Natural Science

Revised: Foundation of China (61876009)

Original:  A. More Details on Experiment Setups
Revised: A Appendix A. More Details on Experiment Setups

Original: Our model is initialized from T5 and Bart (Liu et al., 2020; Raffel et al., 2019).
Revised: Our model is initialized from T5 and Bart (Liu et al., 2020; Raffel et al., 2019).

Original: Speciﬁcally, the encoder and decoder are all 12-layer transformers with 16 attention heads, hidden size 1,024 and feedforward ﬁlter size 4,096, which amounts to 406M trainable parameters.
Revised: Speciﬁcally, the encoder and decoder are all 12-layer transformers with 16 attention heads, hidden size 1,024 and feedforward ﬁlter size 4,096, which amounts to 406M trainable parameters.

Original: We train our framework using the Adam optimizer (Kingma and Ba, 2014)with the initial learning rate 1e-5, and we employ a linear schedule for the learning rate, all models are trained on 8 RTX 3090 GPUs.
Revised: We train our framework using the Adam optimizer (Kingma and Ba, 2014)with the initial learning rate 1e-5, and we employ a linear schedule for the learning rate, all models are trained on 8 RTX 3090 GPUs.

Original: A. Details on Human Evaluation
Revised: A. Details on Human Evaluation

Original: For the results generated by each method, following (Krishna et al., 2020), we randomly selected 100 sentences to be placed in the Amazon Mechanical Turk 1 questionnaire.
Revised: For the results generated by each method, following (Krishna et al., 2020), we randomly selected 100 sentences to be placed in the Amazon Mechanical Turk 1 questionnaire.

Original: We pay our workers cents per sentence.
Revised: We pay our workerscents per sentence.

Original: As shown in Figure 6, the questionnaire asked to judge the generated sentences on three dimensions
Revised: As shown in Figure 6, the questionnaire asked to judge the generated sentences on three dimensions: strength of style transfer, degree of content retention, and text ﬂuency.

Original: of content retention, text minimize the impact of spamming, we require each worker to be a native English speaker with a % or  approval rate and a 
Revised: To minimize the impact of spamming, we require each worker to be a native English speaker with a 95% or higher approval rate and a minimum of 1,000 hits.

Original: A. More Ablation Study and Metrics
Revised: A. More Ablation Study and Metrics

Original: To verify that the general S2S models work wellwith our algorithm, we use bart (Lewis et al. , 2019)as the S2S base model.
Revised: To verify that the general S2S models work well with our algorithm, we use bart (Lewis et al., 2019) as the S2S base model.

Original: For the robustness of theexperiment, we add a new metric J-(a,c,f) (Krishnaet al., 2020) to measure our results, which is asentence-level aggregation strategy evaluate styletransfer models.
Revised: For the robustness of the experiment, we add a new metric J-(a,c,f) (Krishna et al., 2020) to measure our results, which is a sentence-level aggregation strategy evaluate style transfer models.

Original: [Table]
Revised: [Table]

Original: As can be seen from Table 6, our approach can becombined with other general pre-trained languagemodels and performs well, proving our method’sgenerality.
Revised: As can be seen from Table [Table] our approach can be combined with other general pre-trained language models and performs well, proving our method’s generality.

Original: Furthermore, as we can visually seefrom Table 7, our model also performs well onthe J-(a,c,f) metric, which indicates that our modelgenerates sentences in a speciﬁc style while havingthe right target style, preserving content, and beingﬂuent.
Revised: Furthermore, as we can visually see from Table [Table] our model also performs well on the J-(a,c,f) metric, which indicates that our model generates sentences in a speciﬁc style while having the right target style, preserving content

Original: A. Shakespeare Style Generation

Original: To demonstrate the robustness of our algorithm,we trained our algorithm on the Shakespearedataset (Xu et al., 2012).

Original: We choose three of theseplays, Hamlet, Macbeth, and Othello, as differentdomains.

Original: The results are shown in Table 8 and

Original: [Table]
Revised: [Table]

Original: As shown in Table 8, our approach achieves good results on the more difﬁcult Shakespearean style
Revised: and being ﬂuent

Original: [Table]
Revised: [Table]

Original: transfer, which indicates that our algorithm generalizes nicely to other complex style transfer tasks.

Original: A. More Generation Examples
Revised: A. More Generation Examples

Original: To demonstrate more examples of generation to verify the effectiveness of the model, we selected 10 generated sentences from amazon and yelp each, as shown in Table 9 and Table10.
Revised: To demonstrate more examples of generation to verify the effectiveness of the model [Table] we selected 10 generated sentences from amazon and yelp each, as shown in Table 8 and Table

Original: [Table]
Revised: [Table]

Revised: The restaurant domain is used as the target domain and the other three domains as the source domain.

Revised: G-score is the geometric mean of S-Acc and BLEU.The evaluation metrics is J-(a,c,f)

Original:  [Table]
Revised: [Figure] [Table] [Figure] [Table]

Original:  [Table]
Revised: [Figure] [Table] [Figure] [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Original: [Table] 
Revised: [Table] [Figure] [Table] [Table]