Venue: NeurIPS 2021 Conference

Reviews:

final_score: rating: 7: Good paper, accept
review_content: summary: This paper studies the stylized dialog generation problem, in which the problem setup contains contexts C, responses with the original style S0, and responses with the desired style S1. The contributions of this paper includes:
- A novel multiple-pass dual learning framework that explores the interaction between the pairwise relationship among C, S0 and S1.
- The authors propose an additional discrimination to determine the text quality generated by the backward models, which has been shown to be helpful for the task.
- Evaluations are compared to state-of-the-arts on two large text style transfer datasets, and show good amounts of improvements on both automatic and manual metrics.
- Ablation studies further demonstrated the necessity of each component, especially convinced the capacity of discriminator (which is pretty clear from Figure 3). 
- The authors also release their code and data.
main_review: - Originality: There have been many work exploring the stylized dialog generation problem without parallel data. But it's new for this paper to use dual learning to explore the interactions between the pairwise relationship between context C, source style S0 and target style S1. The proposed discriminator component is also intuitive and effective (verified by the experiments), though I am wondering if pretraining such discriminator component might be more effective than optimizing it jointly with the dialog generation loss. Overall, the method is novel and technically sound.
- Quality & Clarity: In general, the paper's writing is clear and easy to follow. There are a few questions to me still: 1) In Equation 9, the optimization loss is defined as the sum of losses from the four components, which gave me the impression that the sum of losses is jointly optimized. However, from Algorithm 1 in the Appendix section, the different components are optimized sequentially. I would love to see the confusion could be resolved. 2) In Table 2, the style consistency scores by auto metrics (e.g., BERT and SVM) don't seem correlate well with the manual judgements, when you compare MPDL with S2S+BT. Is that because the small samples in manual testing, or inaccurate consistency prediction by BERT or SVM?
- Significance: Evaluations are compared to state-of-the-arts on two large text style transfer datasets, and demonstrate significant improvements over strong baselines on a subset of metrics. Human evaluation confirmed that the model generates more style-constrained responses. Ablation studies also provide insights about why this method work. 
limitations_and_societal_impact: There are a few flaws in the evaluation part:
- Instead of reporting BLEU-1 and BLEU-2, it's better to report BLEU-4, which is much widely adopted. 
- The author could consider to increase the number of samples for human eval to increase the evaluation significance. Also, it would be good to disclose the variance of manual's judgements in each metric.
review_title: null

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: The authors propose a method for learning stylized dialogue response generation: given a context, generate a response conditioned on a particular style domain (e.g. modern vs. shakespearean english). The method, Multi-Pass Dual Learning (MPDL), leverages large corpora of non-parallel data in tandem. Specifically, the framework involves several forward and backward models between the context and two styles (one which has labeled data from the context to the style, and one that does not); the models produce intermediate data that is used to train the desired style generation model. To make sure that the intermediate data is being used appropriately, the authors introduce two discriminators that evaluate the quality of the data and determine whether it falls in the respective style/domain. The scores of these discriminators weight the importance of the loss computed on that data to ensure that higher-quality pseudo tuples are more impactful in the learning process. The authors evaluate the framework on two style-transfer datasets, and show that the method outperforms 5 baselines, achieving state-of-the-art results in automated metrics and outperforming nearly all models in human evaluations. A case study and ablations are presented as well that highlight and support the authors’ design decisions.
main_review: *Originality*: The authors build upon the concept of dual learning and apply it to dialogue response generation; the framework is novel in the sense that the dual learning is required for three domain mappings, and is additionally novel in its use of discriminators to measure the quality of the intermediate data; related work is generally adequately cited, though there are a few cites that are not included (e.g. [1] https://arxiv.org/abs/2009.10855). In general, it is the first paper I have seen applying such a method to dialogue.

*Quality*: The claims are generally well supported, with the presented results clearly identifying MPDL as the preferred method amongst several interesting baselines. Relevant ablations and case studies are considered that highlight the impact of various components in the framework (e.g., the multi-pass and discriminator components are important indeed). Some of the results are not clearly justified or explained, however; see limitations below.

*Clarity*: The theoretical derivations are presented cleanly and concisely; although several pieces of notation are introduced, the proposed method and loss functions are all generally understandable. The results tables and figures are very clearly presented as well. However, I had several questions regarding the presentation of the overall method; see limitations below.

*Significance*: The method is understandable and reproducible and appears to bring strong gains to style generation tasks. What is especially interesting and important is the way that the method can leverage unlabeled data, which makes the method quite generalizable (which is supported by its use on two completely different style-transfer domains). 


[1] Smith et. al., "Controlling Style in Generated Dialogue", https://arxiv.org/abs/2009.10855
limitations_and_societal_impact: - I have several questions for the authors; in general, the presented equations all made sense, but required that I re-read them several times to resolve the notation. 
    - On line 56, the third contribution is stated that they provide a new dataset for this task; it is unclear what dataset they have provided (is it just the sampled set of data?)
    - The derivation of Pr(y\~|x, f_xy\~) in lines 112-113 is not super clear to me (why is y~ only related to x)? (similarly for lines 114-115)
- The scores presented in figure 2 do not seem to align with the quality of the input/output sentences (are they swapped?)
- I believe the first line of equation 7 has a typo, which confused me for quite a bit: should the last part read log(1-Dx(Hf(x)), rather than Hf(y)?
- Where does the informal response in “(context, informal response) dialogue pairs” (line 176) come from? 
- What does TCFC stand for?
- How does the model seem to have higher performance on the style classifiers than human responses for the formal response target in the TCFC task?
- Why does SFusion have a much higher Distinct score for the TCFC dataset than it does for SDGC (such that MPDL is now better)?
- I think a qualitative case study of the shakespeare data would be interesting to evaluate the qualitative performance of the model
review_title: null

final_score: rating: 7: Good paper, accept
review_content: summary: This paper focuses on proposing multi-pass dual learning (MPDL) to build mappings among dialogue context and responses in two styles. The proposed model is reasonable and is the extension of the classical dual learning method, which usually maps between two domains. The idea is reasonable and novel, and the experiments show the improved performance in terms of both automatic and human evaluation for both datasets.

A main concern is about the paired style transfer data $D_{tra}$, which limits the flexibility of usage. Also, an intuitive baseline may be needed for fair comparison. The further suggestion is to remove the "paired style transfer data" constraint, considering that the proposed MPDL may still work.
main_review: This paper proposes multi-pass dual learning (MPDL) to build mappings among three domains (dialogue context $C$, response of style $S_0$, response of style $S_1$) through duality. The target scenario is lack of $(C, S_1)$ parallel data for training, but there is unlabelled data with the style $S_1$. 

In the proposed MPDL, the main idea is to utilize the reconstruction property of the response with the style $S_1$, which is mapped to the response with the style $S_0$ via a style transfer model, then mapped to the context via an inverse dialogue generation model, and then finally mapped to the original one (the response with the style $S_0$) via a dialogue generation model. In addition to minimizing the reconstruction error through multiple passes, two discriminators are introduced to ensure the good quality of pseudo contexts and responses in style $S_1$. By jointly learning the mixed learning objective, the proposed method outperforms multiple baselines for two benchmark datasets and most metrics in the conducted experiments.

The idea about using multi-pass dual learning is reasonable and original, where the unlabelled data in style $S_1$ can be effectively utilized and the mappings among three domains can be obtained. However, there are some concerns to be addressed.

- In the experiments, three datasets are used: $D_{dia}$: $(C, S_0)$ paired data, $D_{tra}$: $(S_0, S_1)$ paired data, and $D_{sty}$: unlabelled $S_1$ data. An intuitive baseline can be a pipelined method, where the first model is to generate the response in style $S_0$ and the second model is to transfer it into the response in style $S_1$ in a supervised manner. However, this baseline is not compared in the experiments. Although the unlabelled data $D_{sty}$ is not used, it still needs to be shown in the table so that we can see how much improvement the unlabeled data can bring in. (The S2S+BT baseline uses "unsupervised" text style transfer model [33] instead of "supervised" learning, where the performance is expected worse than the supervised one.)

- In the proposed MPDL framework, it may be possible to use the unpaired $D_{tra}$ for learning $f_{\tilde{y}y}$ and $f_{y\tilde{y}}$ and add a discriminator to check the quality of $y$. Hence, the paired style transfer data may not be required, making the target scenarios more practical. Have the authors tried this setting? (Comparing the results of this setting with S2S+BT is fair.)

- In the discussions, the authors analyze the sensitivity of the size of the parallel data $D_{dial}$. The results are good, but it is also useful to show the impact of the size of $D_{tra}$. Because the main idea is to better utilize unlabelled $S_1$, so the paired style transfer data $D_{tra}$ may not be available/enough. Showing the results with different sizes of $D_{tra}$ may be more informative from my perspective.

- Are the numbers shown in Table 3 for the TCFC dev set? Because the numbers are quite different from ones in Table 1, can they be compared directly?

- For ablation study, removing discriminators shows the degraded performance in Table 3. Can the authors show the results of removing only one discriminator (either $D_x$ or $D_\tilde{y}$), so that we can see which one contributes more?

- The diversity (Distinct) is sometimes lower in the proposed method. Is it because $f_{yx}$ and $f_{x\tilde{y}}$ are not 1-1 mappings and the reconstruction $\tilde{y}'=f_{x\tilde{y}}(f_{yx}(f_{\tilde{y}y}(\tilde{y})))$ forces the mappings to be less diverse? Do the authors have any explanation about this observation?

In sum, the proposed MPDL is reasonable and seems useful in the conducted experiments. The paper is clear. The experiments can be improved in order to better convince the readers about the model's effectiveness. 
limitations_and_societal_impact: Yes.
review_title: null

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: This paper studies the task of stylized dialogue generation, where the goal is to generate dialogue responses in a particular target style (formal / informal / Shakespeare-like). The authors operate in a setting where they do not have access to any dialogue training data in the target style, but instead have access to dialogue data in a complementary style, along with parallel data to transfer between the two styles. For instance, the authors study dialogue generation in formal style, and assume access to dialogue data in informal style, parallel data between formal / informal sentences, and a corpus of unpaired formal sentences.

The authors model this task as a three-way dual learning problem, a technique which has proven useful for machine translation [1]. They jointly training their model to perform 1) style transfer in both directions; 2) dialogue / inverse-dialogue generation to the complementary style; 3) dialogue / inverse-dialogue generation to the target style. 1 / 2 act as auxiliary tasks to aid (3) via shared models. Since no parallel data is available for (3), the authors perform sampling (using current version of model for 1 / 2) to create pseudo parallel data for (3). Since sampling can be noisy, the authors additionally use discriminators to weight samples, which judge whether the samples are real/fake (akin to adversarial training).

The authors evaluate their approach on formal dialogue generation and Shakespearan dialogue generation. They compare their approach to 5-6 baselines, along with human upperbounds. Both automatic and human evaluation is conducted. The authors additionally conduct ablation studies, analyze their discriminator, and show the effect of the size of parallel data.

[1] - https://arxiv.org/abs/1611.00179
main_review: **Strengths**

1. Style-controlled text generation is an important problem in natural language generation technology, and is far from solved. This has several important applications in chatbots and coversation agents.

2. The authors use several interesting ideas in this paper, which were new to me. The idea to use dual learning to boost style-controlled generation as well as discriminators for weighing poor synthetic data were interesting and could be more widely applicable in style transfer literature.

3. The authors approach beats several strong baselines on automatic & human evaluation. Several good analysis experiments have been conducted in the paper including ablations, analysis of the effect of discriminators and the effect of the size of available parallel data.

**Weaknesses**

1. My biggest issue with the paper is the usage of parallel data. Most style transfer literature [1, 2, 3, 4] operates in an unsupervised / unpaired setting (with access to D_sty only), and recent work [5] is assuming access to only 5-10 exemplars from D_sty. Access to parallel data is a strong assumption --- besides formality, shakespeare, sentiment transfer, almost no parallel resources exist. Moreover, all these datasets are in English, and no parallel style transfer resources exist for other languages. Even assuming access to dialogue data in a single style (`D_dia`) is strong assumption in my opinion. Indeed, the authors show that their method's performance reduces significantly if you don't have access to this parallel data (Table 3 w/o Multi-Pass, Figure 4), which reduces the applicability of this work.

2. The evaluation in this paper could be improved. Here are some specific thoughts ---

* Since there are three orthogonal metrics [7] for both human / automatic evaluation (style accuracy, semantic correctness, fluency), it's hard to compare between systems when only one metric improves (and the other does not), for instance SFusion vs MPDL in Table 1; S2S+BT vs MPDL in Table 2; experiments in Table 3. I suggest using the aggregation strategy from [2] to combine the metrics into a single score, making it easy to compare between systems.

* The BLEU scores are quite low overall for all systems --- which is not surprising given the large space of possible outputs which makes evaluation challenging and easy to game [8]. I suggest using other automatic metrics too, such as those based on natural language inference [9] or the recent BEGIN benchmark [10].

* Since automatic metrics can be flawed, human evaluation is critical. While the authors conduct human evaluation, more details will be helpful. Specifically, i) Were the annotators familiar with the research goals? (ideally they should be unfamiliar to avoid bias) ii) Were they shown outputs from different systems in a blind manner (not aware which system produced which output) & shuffled order? (consecutive samples from different systems) iii) What's the agreement between annotators & variance / error-bars of their annotation? Are the gains statistically significant? iv) How much were the annotators paid for this experiment?

* More qualitative outputs from the system would be very helpful. Currently only two outputs are shown in Table 4. In both examples it's not really clear to me if the outputs are really formal / informal, which makes me doubt the quantiative evaluation in this work (In second example, "wouldn't" has an apostrophe which shouldn't be there in an informal style generation).

**Minor**: The notation in Section 3 is confusing, primarily due to lots of inverted commas and tildes. I suggest avoiding these and using subscripts with short natural language identifiers (like you did for `D_dia / D_tra / D_sty`). Also it will be good to mention `BERT / SVM` in Table 1 refer to classification accuracy (my initially incorrect thought reading this was a reference to BERTScore [6], which confused me).

**Overall**

In terms of the NeurIPS reviewer guidelines, I think the paper has decent originality (strength #2), average quality (strength #3, weakness #2), average clarity (minor #1), and average significance (strength #1, weakness #1). Overall, I liked the ideas in the paper, but I think weakness #1 is the biggest deal-breaker for me which will prevent me from giving a higher score. I encourage the authors to improve this in the next version of the paper.

[1] - https://arxiv.org/abs/2011.00416  
[2] - https://arxiv.org/abs/2010.05700  
[3] - https://arxiv.org/abs/1804.09000  
[4] - https://arxiv.org/abs/1811.00552  
[5] - https://arxiv.org/abs/2010.03802  
[6] - https://arxiv.org/abs/1904.09675  
[7] - https://arxiv.org/abs/1910.03747  
[8] - https://arxiv.org/abs/2103.06332  
[9] - https://arxiv.org/abs/1904.03371  
[10] - https://arxiv.org/abs/2105.00071

-------

**After author response**: Thanks for the excellent response. I've raised my score to 6.
limitations_and_societal_impact: Yes
review_title: null

Meta-review: This paper proposes a new method for stylized dialog generation. The task is that one learns a model to automatically generates a stylized response given a context in dialogue. The authors apply the technique of dual learning to address the problem by learning bi-directional transformations of texts between the contexts C, the original responses S0, and responses with the desired style S1.

Pros
* The paper is generally clearly written.
* A new approach has been proposed, which appears to be reasonable and technically sound.
* The proposed approach appears to be new.
* Experimental results have shown that the proposed method outperforms the baselines on two large datasets. An ablation study has also been conducted.
* The authors also plan to release their code and data.

Cons
* There are some issues with the presentation. The authors have addressed some of them in the rebuttal. They are encouraged to reflect the discussion results in the new version of the paper.
* The authors also conducted additional experiments during the rebuttal. They are asked to include the results in the new version of the paper.
* The proposed method appears to be complicated. One may doubt whether it can be really used in practice.* 
Decision: Accept (Poster)