Stylized Dialogue Generation with Multi-Pass Dual Learning
Year: 2021

Abstract:
Stylized dialogue generation, which aims to generate a given-style response for an input context, plays a vital role in intelligent dialogue systems. Considering there is no parallel data between the contexts and the responses of target style S1, existing works mainly use back translation to generate stylized synthetic data for training, where the data about context, target style S1 and an intermediate style S0 is used. However, the interaction among these texts is not fully exploited, and the pseudo contexts are not adequately modeled. To overcome the above difficulties, we propose multi-pass dual learning (MPDL), which leverages the duality among the context, response of style S1 and response of style S0. MPDL builds mappings among the above three domains, where the context should be reconstructed by the MPDL framework, and the reconstruction error is used as the training signal. To evaluate the quality of synthetic data, we also introduce discriminators that effectively measure how a pseudo sequence matches the specific domain, and the evaluation result is used as the weight for that data. Evaluation results indicate that our method obtains significant improvement over previous baselines.

heading: 1 Introduction
text: In recent years, dialogue systems have attracted a surge of research interest and achieved great success [1, 2, 3, 4]. The dialogue generation task can be formulated as a sequence-to-sequence learning problem [5, 6], where the source sequence is the given context, and the target sequence is the response. Stylized dialogue generation, which automatically generates response following the desired style, obtains significant progress in open-domain dialogue systems. The styles could be formal/informal, modern/Shakespearean, serious/humor, etc.
A core challenge of this task is the lack of parallel training data between the contexts and responses of desired styles (denoted as style S1), which limits the performance of models. A natural solution is to establish the pseudo pairs between contexts and responses of style S1 using unsupervised or semi-supervised learning. The parallel data between contexts and responses of style S0 (denoted as style S0) is leveraged. Note the conversations corpus (context, style S0) are relatively easy to obtain. After that, the stylized dialogue model can be trained on the synthetic data. Specifically, Su et al. [7] propose a diversifying dialogue generation model based on iterative back translation [8]. Zheng et al. [9] propose a style routing approach with a joint training process, where an inverse model with style embeddings is used to generate pseudo pairs and train the stylized dialogue generation model.
∗corresponding authors: Rui Yan (ruiyan@ruc.edu.cn) and Dongyan Zhao (zhaody@pku.edu.cn).
35th Conference on Neural Information Processing Systems (NeurIPS 2021).
Although the above methods achieved success in the stylized dialogue generation, there still exists lots of room to improve. First, the interaction between context, style S0 and style S1 is not fully exploited. An example is that a pseudo context can be explicitly obtained by a two-hop process, S1 → S0 →pseudo context instead of using style embedding only. Second, in the aforementioned methods, the quality of the pseudo contexts generated through back translation is not properly modeled. By manually checking the generated data, we find that there are quite a few pseudo contexts that contain irrelevant contents of the response. Several common patterns in the pseudo contexts and responses include “I don’t know”, “I’m so happy”, etc. In previous methods, each pseudo data is treated equally regardless of their quality, which introduces bias to the training set and hurts the performances of model.
In this paper, we propose a new framework, multi-pass dual learning (MPDL), for stylized dialogue generation. Inspired by dual learning [10] that can automatically extract information through two dual tasks in an unsupervised manner, we extend it to a multi-pass version. In MPDL, there are three dual tasks in total, including context↔style S1, context↔style S0 and style S0 ↔style S1. The unlabeled texts in style S1 and contexts are leveraged. We also introduce two discriminators to evaluate the quality of the generated data. The discriminators are used to measure the similarity of generated data to the corresponding domains. The similarity is used as the weight of the synthetic data for training, by which we can adaptively use the unlabeled data. We conduct experiments with benchmarks of stylized response generation in formal English and Shakespearean English. We achieved state-of-the-art results in terms of both automatic evaluation and manual judgment.
Our contributions are summarized as follows:
(1) We propose multi-pass dual learning (MPDL) framework for stylized dialogue response generation, that can effectively leverage the unlabeled data; (2) Compared with standard dual learning, we introduce two discriminators to evaluate the quality of the pseudo parallel data. This is a new attempt for the general dual learning framework; (3) We provide a new dataset for this task and set several benchmarks using our method; (4) We empirically verify the effectiveness of MPDL on two datasets with formal and Shakespearean response generation.

heading: 2 Related work
text: Text style transfer aims to re-write a sentence with given stylistic properties. Some early works try supervised approaches, where a work is the transfer between modern English and Shakespearean style sentences [11, 12]. Nevertheless, more and more recent research begins to pay attention to the unsupervised approach due to the lack of parallel corpora [13, 14]. In general, there are three typical approaches. The first is to learn disentangled representations of content and style. Shen et al. [15] learned a shared representation of context across different styles and then leveraged the refined alignment of latent representations for style transfer. Fu et al. [16] utilize a multi-decoder with a style embedding model to learn separate content representations, where style representations are obtained via adversarial networks. Another is to generate stylized texts with back translation [17, 18], which produces pseudo parallel data to jointly train the forward and backward model. The third is based on the template, which tries to replace original stylistic words with the target stylistic words [19, 14].
Stylized dialogue generation refers to generate a response in the target style. Zhou et al. [20] propose an Emotional Chatting Machine (ECM) model for the emotional response. An intuitive solution generates a response with a pre-trained style language model [21], which builds a response generation model STYLEDGPT on top of a pre-trained language model DialoGPT [22] and devise both a word-level loss and a sentence-level loss to fine-tune the DialoGPT towards the target style. Niu and Bansal [23] propose three weakly supervised models that can generate diverse, polite (or rude) dialogue responses without parallel data. Some work bridge conversation modeling and non-parallel style transfer by sharing a structured latent space [24]. Recently, some work iteratively tuned to find the optimal mapping relation between conversational context and non-conversational utterances via iterative back translation [7, 9]. These works have achieved good performance, but we have achieved better diversity and efficiency through an auxiliary domain.
Back translation is to train a target-to-source seq2seq model for producing source sentences, and then construct pseudo parallel datasets. This approach is widely used in machine translation [8, 25], text style transfer [26, 27] and dialogue generation [7, 9]. Similar concurrent work is Dual learning [10], which is first proposed for neural machine translation, involves two tasks with an immediate reward.
Dual learning has proved to be very effective in neural machine translation and image translation [28]. Our work is also based on the idea of them. However, most of the recent works have focused on two domain mapping. We employ dual learning to work on a three-domain text related problem, then the contents of non-conversational text can be effectively utilized to enrich the dialogue generation.

heading: 3 Method
text: Let C, S0, S1 denote the domains of context, response in style S0 and response in style S1 respectively. Our task is to learn a mapping, that can map a sequence from context domain C to a stylized response domain S1. A core challenge is that there is no parallel data between C and S1, but there are some related parallel corpora: (i) A dialogue dataset Ddia = {(xi, yi)}ni=1 where xi ∈ C and yi ∈ S0; (ii) A style transfer dataset Dtra = {(yi, ỹi)}mi=1, where yi ∈ S0 and ỹi ∈ S1. The corpus sizes of the above two datasets are n and m. The sequences in style S0 are not the same between Ddia and Dtra, i.e., {y|(x, y) ∈ Ddia} ∩ {y|(y, ỹ) ∈ Dtra} = ∅. In addition to parallel data, there is another unlabeled stylized corpus Dsty = {ỹi}li=1 with size l, where ỹi ∈ S1.

heading: 3.1 Multi-pass dual learning framework
text: To fully utilize the parallel and unlabeled data, we propose multi-pass dual learning (briefly, MPDL), which is illustrated in Figure 1. There are three pairs of dual tasks involved in our framework: (i) Dialogue generation and the inversed task, fxy : C 7→ S0, fyx : S0 7→ C; (ii) Style transfer between S0 and S1, fyỹ : S0 7→ S1, fỹy : S1 7→ S0; (iii) Stylized dialogue response generation and the inversed task, fxỹ : C 7→ S1, fỹx : S1 7→ C. Our ultimate goal is to obtain the stylized dialogue response generation model fxỹ, while the remaining tasks are all auxiliary tasks. For multi-pass dual learning, given any ỹ ∈ Dsty, we could obtain reconstruction ỹ′ = fxỹ(fyx(fỹy(ỹ))) by sequentially applying fỹy, fyx and fxỹ. The reconstruction error between ỹ′ and ỹ determines the ability of model reconstruction. That is, we want to maximize the logarithmic probability:
logP (ỹ′ = ỹ|ỹ; fỹy, fyx, fxỹ). (1) Directly optimization the above probability is difficult since there are three models to be updated. Consequently, we derive a relaxed version of the above probability. We have that:
logP (ỹ′ = ỹ|ỹ; fỹy, fyx, fxỹ) = log ∑ x∈C ∑ y∈S0 P (ỹ′, y, x|ỹ; fỹy, fyx, fxỹ)
= log ∑ x∈C ∑ y∈S0 P (ỹ′|y, x, ỹ; fỹy, fyx, fxỹ)P (y, x|ỹ; fỹy, fyx, fxỹ). (2)
Considering that ỹ′ is only related to x and fxỹ, we have that: P (ỹ′|y, x, ỹ; fỹy, fyx, fxỹ) = P (ỹ′|x, fxỹ). Due to the concavity of log(. . . ), we can convert the Eqn.(2) to:
logP (ỹ′ = ỹ|ỹ; fỹy, fyx, fxỹ) ≥ ∑ x∈C ∑ y∈S0 P (y, x|ỹ; fỹy, fyx, fxỹ) logP (ỹ′|x, fxỹ)
= ∑ x∈C ∑ y∈S0 P (x|y; fyx)P (y|ỹ; fỹy) logP (ỹ′|x, fxỹ), (3)
where the last equation is obtained by the fact that x is only related to y and fyx, and y is only related to ỹ and fỹy .
Instead of maximizing logP (ỹ′ = ỹ| · · · ) directly, we maximize a lower bound as Eqn.(3) suggests. We use a sampling-based method to solve this problem: (i) Given a ỹ, we sample a y according to P (·|ỹ; fỹy); (ii) Based on y, we sample a x′ according to P (·|y; fyx); (iii) We maximize logP (ỹ|x′; fxỹ). We can use either beam search or random sampling to get the intermediate sequences. On the other hand, we can pick an x from the set {x|x ∈ Ddia}, sequentially use two sampling steps y ∼ P (·|x; fxy) and ỹ′ ∼ P (·|y; fyỹ) and then maximize logP (x|ỹ′; fỹx). In addition to the loss for unlabeled data, we propose loss functions on the parallel data. Therefore, there are four loss functions in total:
Ldia = E (x,y)∼Ddia − (logP (y|x; fxy) + logP (x|y; fyx)),
Ltra = E (y,ỹ)∼Dtra − (logP (ỹ|y; fyỹ) + logP (y|ỹ; fỹy)), Ldual,1(ỹ) = − logP (ỹ|x′; fxỹ), Ldual,2(x) = − logP (x|ỹ′; fỹx),
(4)
where x′ and ỹ′ are generated by the MPDL model as the above steps.

heading: 3.2 Network architecture
text: All the six models f ’s are built on Transformer-based architecture [29]. The encoder and decoder of f ’s are initialized using pre-trained DialoGPT weights trained on conversation corpus of Reddit community [22]. To reduce the memory cost, (i) the parameters of all forward models, including fxy , fyỹ and fxỹ are shared; (ii) the parameters of all backward models, including fyx, fỹy and fỹx are shared; (iii) the both encoder and decoder (excluding the encoder-decoder attention modules) share the same parameters too.
Taking the forward models as an example, to distinguish the models from each other, we introduce several domain embeddings T = {EC, Es0 , Es1} for context domatin (i.e., C), style S0 (i.e., S0) and style S1 (i.e., S1) domains respectively. Given an input x and the domain T , the encoder of the forward model works as follows:
Hf (x) = encf (EC +W(x)), (5) whereW(x) is the word embedding of input sequence x, and encf is encoder module of the forward model. Before feeding the representation Hf (x) into the decoder, according to the target domain (S0 or S1), Hf (x) is concatenated with another style embedding:
Hξf (x) = [Eξ, Hf (x)], (6)
where ξ ∈ {C, S0, S1}. In this way, the hidden representations are shifted according to the target domains. We can obtain HS0f (x) for dialogue response generation, H S1 f (x) for stylized dialogue generation and HS1f (y) for text style transfer. Similarly, for the backward model (marked by subscipt b), we can obtain the features as follows: HCb (y) for inverse dialogue response generation, H C b (ỹ) for inverse stylized dialogue generation and HS0b (ỹ) for inverse text style transfer. For ease of reference, we set that all the H’s are d-dimensional vectors.
3.3 Quality discriminators Input 𝒚" : I believe that Brazilian are very sexy.
Output𝒚"′ : I'm gonnato havea rest.
Output 𝒙′ : I'm so excited for Brazil to come out.
Input 𝒙 : Do youwant to goback to work right away?
[𝐷& → 0.45] Backword
Forword
[𝐷𝒚" → 0.13]
When analyzing the intermediate results, we find that previous work simply which use iteratively back translation cannot obtain proper context sequences because of the poor data quality. In many scenarios, examples of the synthetic context include:
based on which the model cannot generate meaningful response. The inadequate text modeling leads to a peak reduction in model performance. As shown in Figure 2. To evaluate the data quality of the intermediate data (i.e., the pseudo contexts x and responses in style S1), we employ two discriminators Dx : Rd 7→ [0, 1] and Dỹ : Rd 7→ [0, 1], which are used to classify whether an input text is a real one (e.g., written by people) or a synthetic one (e.g., generated by model).
The discriminators will give a weight to the pseudo sequence generated by model, which can be used to judge the degree to which the model parameters can be updated. Dx and Dỹ are implemented as 3-layer feed-forward networks with LeakyReLU activation function, whose input are Hf (x) and Hb(ỹ) obtained according to Eqn.(5). The loss function for discriminators is as follows:
`xdis = logDx(Hf (x)) + log(1−Dx(Hb(y))), `ỹdis = logDỹ(Hb(ỹ)) + log(1−Dỹ(Hf (y))), Ldis = E
(x,y)∼Ddia;(ỹ)∼Dsty − (`xdis + ` ỹ dis).
(7)
where Hb(y) is used to generate pseudo context x′ for inverse dialogue response generation task, Hf (y) is used to generate pseudo response ỹ′ for text style transfer task. Note that at inference time, the discriminator is not used, which means that MPDL will not introduce additional inference cost. The output of the discriminators could be regarded as a confidence score for the sample, which is used further to improve the dual loss in the previous section. Mathematically,
L∗dual,1 = E ỹ∼Dsty −Dx(Hf (x′)) logP (ỹ|x′; fxỹ), L∗dual,2 = E x∼Ddia −Dỹ(Hb(ỹ′)) logP (x|ỹ′; fỹx), (8)
In training stage, loss function of MPDL model consists of the standard maximum log likelihood loss and the multi-pass dual learning loss. The mixed learning objective function is defined as follows:
L = Ldia + Ltra + L∗dual,1 + L∗dual,2. (9)

heading: 4 Experiment

heading: 4.1 Experiment setup
text: Datasets. We train and evaluate our method on two benchmarks, TCFC [30], whose responses are of formal style, and Shakespearean Dialogue Generation Corpus (SDGC)2, whose responses are of Shakespearean style. TCFC is constructed by crawling text from Twitter. Following [9], we sample 217.2k (context, informal response) dialogue pairs as Ddia and 500k formal texts as Dsty from the TCFC dataset. The test set contains 978 manually-crafted dialogue pairs. For the Dtra, we sample 105.5k informal-formal parallel sentences from GYAFC [31]. SDGC is constructed by Twitter, Yahoo Answers and Shakespeare [11]. Shakespeare is a collection of Shakespearean plays with an aligned modern version on supervised style transfer task. For the test set, we manually craft 1,000 dialogue pairs and we will release them to the community. The Ddia is the same as that for TCFC. We extract 18.4k (modern English, Shakespearean English) pairs from [11] as Dtra. All Shakespearean English is used as Dsty. The statistic of datasets is shown in Table 5 of Appendix. Baselines. We compare our proposed model with six baselines, including standard Transformer-based dialogue model, multi-task learning method, language model method, hidden state fusion method, pipeline method and the latest end-to-end method. Specifically, (1) S2S: a Transformer-based model trained onDdia only, where the weights are initialized by DialoGPT [22]. (2) MTask: Luan et al. [32] propose a vanilla multi-task learning model, which is trained with both Ddia and Dsty. (3) SLM: Niu and Bansal [23] propose the fusion model by merging the decoder of a sequence-to-sequence model trained on Ddia and a language model trained on Dsty. (4) SFusion: a multi-task learning model proposed by [24], which builds a structured latent space to bridge the conversation modeling and style transfer through jointly learned with Ddia and Dsty. (5) S2S+BT: the responses with style S0 of pipeline approach is produced by the S2S baseline, and then text style transfer model proposed by [33] is used to get a response with style S1. (6) SRJT: a Style Routing approach by employing an
2Our code and dataset are publicly available at https://github.com/CodebaseLi/MPDL
inverse dialogue model with a Joint Training process proposed by [9], in which it is based on back translation. More baselines and evaluating metrics [34] are available in Appendix.
Training details. The DialoGPT (small-size 117M) [22] with 12 layers and 768 hidden sizes is used to initialize the multi-pass dual learning modules, which is released by Transformers3. Our model has a 50,257 vocabulary size and was trained on Nvidia GTX1080Ti machines with a batch size 10. The maximum input length and maximum output length are set as 45. We choose the Adam optimizer. The learning rate of generators is 2.25× 10−4 with warm-up steps 1× 103, while that for discriminators is 3× 10−4. We use the grid search to tune the hyper-parameters.

heading: 4.2 Main results
text: Automatic evaluation. In this work, we evaluate the generation quality of both style S1 and style S0, following the evaluation metrics in [9]. Content relevance, diversity and style consistency are our primary consideration. For the content relevance, we choose the BLEU score [35] to between the generated response and the ground-truth one. We focus on the unigram precision (BLEU-1) and bi-gram precision (BLEU-2). For diversity, we choose the distinct score [36], which is the ratio between the number of unique n-grams and the number of all n-grams in the generated responses. In this work, n is 2. For style consistency, we use two text style classifiers, a BERT-based model [37] and an SVM, to evaluate the style consistency, both of which are trained on the GYAFC and Shakespeare datasets. The accuracy of the BERT and SVM classifiers on the holdout test set is 93.98% and 89.57% respectively for the TCFC experiments, and 95.79% and 82.28% respectively for the SDGC dataset. Each setting is independently run five times and we report the mean value of each metric.
The automatic results are summarized in the left part of Table 1 (for TCFC) and Table 2 (for SDGC). In each table, the generation quality of both styles S1 and S0 are evaluated. Overall, our method achieves the highest BLEU-1 and BLEU-2 scores on the two tasks, which shows the superiority of our method. For formal response generation task on the TCFC dataset, compared with the previous state-of-the-art algorithm SRJT, MPDL improves the BLEU-1, BLEU-2 and distinct scores by 1.4, 0.36 and 7.9. The reasons are two-fold: (1) multi-pass dual learning framework significantly improves the learning ability since it bridges the relation between three domains; (2) the discriminator controls the quality of pseudo data. SFusion gets a better distinct score than MPDL, but has worse BLEU-1 and BLEU-2 scores. This is because the discriminator tends to maintain the consistency of the generated responses and the target style, at the cost of reducing diversity. The BERT and SVM scores of our method are also notable, which indicates that our model can generate target style response
3https://github.com/huggingface/transformers
better with the auxiliary text transfer task. On informal response generation, by using MPDL, the performances are also improved, where we also achieved the best BLEU-1 and BLEU-2 scores. We observe that compared to the formal case, the BERT score is similar to previous best result, and the SVM score is lower, which is caused by the lack of discriminator for the informal domain. We will explore it in the future.
For the SDGC dataset, MPDL model also shows great advantages in most metrics. We observed that S2S+BT achieved the highest BERT and SVM scores in Shakespearean style. This shows that the unsupervised pipeline method does help to migrate text to the target style, but at the same time, it will lose the conversation information since the style transfer model does not take the context into consideration. MPDL achieves the second achievement in the style classifier while other metrics are improved significantly. SFusion has a much higher Distinct score for the TCFC dataset than it does for SDGC. Because SFusion learns the patterns from the data of the style S1 domain, and more data is helpful to increase the diversity. TCFC contains 500k non-parallel data of style S1 domain while SDGC has only 18.4k. Therefore, MPDL has a better distinct score on the SDGC dataset. We further investigate the generalization ability of MPDL by evaluating modern responses, and find that the model generalizes well not only over different styles but also over different datasets.
Manual evaluation. Complementary to automatic evaluation, we also carry out the manual evaluation. We focus on three aspects: (1) fluency, which indicates how smooth the sentence is; (2) relevance, which shows the content of the response is consistent with the context; (3) style consistency, which evaluates how the response style matches the target domain. For a fair comparison, all generated responses and human responses are re-capitalized, de-tokenized and shuffled order. We sample 100 instances from the text set and employ three graduated annotators to rate each context-response pair. The score range of sample is from 0 to 1, where 1 indicates the best score. We provide the responses (generated by models and ground-truth by humans) for each example to our annotators. Every sample can obtain three human judgments and the final scores are averaged across different judges.
The manual evaluation scores are summarized in the right part of Table 1 and Table 2, where we can see that MPDL significantly outperforms previous baselines. Specifically, for formal response generation on TCFC, the scores of our method are close to the human score. On informal text generation, SFusion and SRJT achieved slightly better results on style consistency and relevance than our method. On SDGC, our method achieves the best human evaluation scores, which in general shows the effectiveness of our method. Compared to SFusion and SRJT in style S0, the relevance and style consistency scores of MPDL decrease on TCFC but increase on SDGC. The reason is that we use more unlabeled target domain data for TCFC than SDGC (500k v.s. 18.4k). Considering that we share the parameters of all tasks, the model for SDGC is less specialized to the target domain
and maintains the generalization ability across different domains. More details and the variance are available in the Appendix.

heading: 4.3 Discussions
text: Ablation studies. To understand the roles of discriminator, multi-pass dual learning (specifically, Ltra in Eqn.(4)), and pre-trained model in learning to generate stylized responses, we conduct ablation study by removing each component from our framework, and the results are in Table 3. We have the following observations: (1) Without the discriminator, all pseudo responses and context are equally treated. Both the BLEU and BERT scores drop, but the distinct scores increased. This is consistent with our previous explanation, since there is no discriminator to enforce that the styles of generated sequences should match the target domain. (2) Without the Ltra in Eqn.(4), the results of all metrics significantly drop due to the lack of a closed loop; (3) The pre-training is also helpful to the whole scheme, without which all metrics drop. The ablation study shows the importance of each component in our model. Moreover, we also provide some examples in Table 7 of Appendix to show how the discriminator weights the sequences. Specifically, in dialogue generation of Shakespearean style, the discriminator assigns a score 0.99 to the sentence “Tis a brave lady”, where “Tis” frequently appeared in Shakespearean play. In contrast, the response “I’m not sure what to do with it”, which is quite general, obtains a 0.14 weight.
Latent space visualization. To further study the effect of the discriminators, we visualize the hidden state on TCFC dataset, including the context, informal responses and formal responses. We use the output of the last layer in decoder as the representation of the sequences and utilize t-SNE algorithm [38] for visualization. We compare the models trained without the discriminator and with the discriminator. The results are in Figure 3, the remaining visualization results on SDGC dataset and more details are included in the Appendix. As shown in Figure 3(b), after using discriminators, the three types of sequences are properly separated. Without the discriminators (see Figure 3(a)), we can see that the boundary of context and the informal text is not quite clear, which verifies that the quality the synthetic data is low, and we need a module to re-weight them.
Parallel data size. To show the effect of the parallel conversation data Ddia size, we re-train MPDL, MPDL w/o discriminator and the SRJT baseline on TCFC dataset. The BLEU-1 is shown in Figure 4. The performances of all three methods increase w.r.t. to the labeled data size. In comparison, MPDL is less sensitive to the amount of labeled data compared to SRJT, and achieves the best results in BLEU-1, BLEU-2 and Distinct. Moreover, the MPDL with 50k labeled dialogue conversation data achieves comparable (even slightly better) results to the SRJT with 200k labeled data, demonstrating that our method can effectively utilize data. Experiment of parallel style transfer data Dtra with different sizes are provided in the Appendix.
Case study. We show some examples of the stylized response in Table 4, and make a comparison of the previous algorithms. For formal English generation (no parallel data between context and the formal response), the MTask model, whose weights are not initialized from the pre-trained model, tends to generate general and bland responses because of the lack of prior knowledge in the dialogue model. SFusion generates the wrong style, where there is an emoji “:)” at the end, since there is no discriminator. For SRJT, although it generates a formal sentence, the response is not consistent to the context, because we would like to see the response to “why would throw it away”. Overall, our model performance is the best, capturing more input information and the target style. For informal English generation where parallel data is available, by using our training scheme, we also observe some improvement over the baselines. MPDL model reduces the use of words like “I don’t ...” or “I’m sure ...” and tends to generate more meaningful words to make the response more interesting.

heading: 5 Conclusions and future work
text: In this paper, we propose a novel stylized dialogue generation model with multi-pass dual learning (MPDL), that can effectively leverage unlabeled data. Compared with standard dual learning, we employ a discriminator to ensure that the responses move towards the target style while the content is relevant. Experiments on two stylized response generation benchmark datasets demonstrate that our method has significant advantages over the recent works, and our model achieves state-of-the-art results in the formality and Shakespearean style response generation. For future work, first we can extend the stylized dialogue from single style to multiple styles without too much additional cost; second, we will increase the robustness of our model; third, we will further explore how to effectively use more unlabeled data.
Broader Impact
Generating stylized responses is an important step towards real artificial intelligence. It can be predicted that its achievements can significantly improve the experience of human-computer interaction, and can be applied in various fields, such as education, finance, and so on. This work has the potential positive impact on an intelligent and engaging dialogue system. At the same time, this work may have some negative consequences in social interaction. Besides, if the pre-training model is covered by malicious and vulgar information, it will bring bad information feedback to users. Therefore, we should be cautious of these advantages and disadvantages.

heading: Acknowledgments
text: This work was supported by the National Key Research and Development Program of China (No. 2020AAA0106600) and NSFC Grant (No. 62122089 and No. 61876196).

Review summaries:
This paper focuses on proposing multi-pass dual learning (MPDL) to build mappings among dialogue context and responses in two styles. The proposed model is reasonable and is the extension of the classical dual learning method, which usually maps between two domains. The idea is reasonable and novel, and the experiments show the improved performance in terms of both automatic and human evaluation for both datasets.  A main concern is about the paired style transfer data $D_{tra}$, which limits the flexibility of usage. Also, an intuitive baseline may be needed for fair comparison. The further suggestion is to remove the "paired style transfer data" constraint, considering that the proposed MPDL may still work.

This paper studies the task of stylized dialogue generation, where the goal is to generate dialogue responses in a particular target style (formal / informal / Shakespeare-like). The authors operate in a setting where they do not have access to any dialogue training data in the target style, but instead have access to dialogue data in a complementary style, along with parallel data to transfer between the two styles. For instance, the authors study dialogue generation in formal style, and assume access to dialogue data in informal style, parallel data between formal / informal sentences, and a corpus of unpaired formal sentences.  The authors model this task as a three-way dual learning problem, a technique which has proven useful for machine translation [1]. They jointly training their model to perform 1) style transfer in both directions; 2) dialogue / inverse-dialogue generation to the complementary style; 3) dialogue / inverse-dialogue generation to the target style. 1 / 2 act as auxiliary tasks to aid (3) via shared models. Since no parallel data is available for (3), the authors perform sampling (using current version of model for 1 / 2) to create pseudo parallel data for (3). Since sampling can be noisy, the authors additionally use discriminators to weight samples, which judge whether the samples are real/fake (akin to adversarial training).  The authors evaluate their approach on formal dialogue generation and Shakespearan dialogue generation. They compare their approach to 5-6 baselines, along with human upperbounds. Both automatic and human evaluation is conducted. The authors additionally conduct ablation studies, analyze their discriminator, and show the effect of the size of parallel data.  [1] - https://arxiv.org/abs/1611.00179

This paper studies the stylized dialog generation problem, in which the problem setup contains contexts C, responses with the original style S0, and responses with the desired style S1. The contributions of this paper includes: - A novel multiple-pass dual learning framework that explores the interaction between the pairwise relationship among C, S0 and S1. - The authors propose an additional discrimination to determine the text quality generated by the backward models, which has been shown to be helpful for the task. - Evaluations are compared to state-of-the-arts on two large text style transfer datasets, and show good amounts of improvements on both automatic and manual metrics. - Ablation studies further demonstrated the necessity of each component, especially convinced the capacity of discriminator (which is pretty clear from Figure 3).  - The authors also release their code and data.

The authors propose a method for learning stylized dialogue response generation: given a context, generate a response conditioned on a particular style domain (e.g. modern vs. shakespearean english). The method, Multi-Pass Dual Learning (MPDL), leverages large corpora of non-parallel data in tandem. Specifically, the framework involves several forward and backward models between the context and two styles (one which has labeled data from the context to the style, and one that does not); the models produce intermediate data that is used to train the desired style generation model. To make sure that the intermediate data is being used appropriately, the authors introduce two discriminators that evaluate the quality of the data and determine whether it falls in the respective style/domain. The scores of these discriminators weight the importance of the loss computed on that data to ensure that higher-quality pseudo tuples are more impactful in the learning process. The authors evaluate the framework on two style-transfer datasets, and show that the method outperforms 5 baselines, achieving state-of-the-art results in automated metrics and outperforming nearly all models in human evaluations. A case study and ablations are presented as well that highlight and support the authors’ design decisions.