Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: The paper proposed the use of Equal size hard EM algorithm to address the problem of diverse response generation in open domain dialog modeling. The paper modified the EM algorithm to overcome mode collapse issues with hard EM and synchronous training collapse issues with soft EM. Experimental results show improvement in diverse response generation on Weibo and Open subtitle datasets. Ablation studies also show that Equal size hard EM performs better than beam search, soft EM and pure hard EM. Qualitative analysis also shows that some of the modes learned unique attributes of the dialog data such as response length, question mark, and like/love. The paper also shows that the performance improves with increasing number of decoders, although the incremental improvement reduces. Overall, the paper is clearly written and well motivated.
strength_and_weaknesses: Strength:
The paper is clearly written and easy to follow. There are couple of minor typos to be fixed. The proposed method is novel and experimental results is thorough. The ablation study is also informative. 

Weakness:
The use of mixture of experts for dialog modeling is not new, but the implementation using adaptor is new. The evaluation did not include existing baseline that have used GAN or reinforcement learning to accomplish diverse response generation. Also, it is not clear if encode-decoder architecture is suitable for dialog response generation as the conditional probability modeling P(y/x) exacerbate the one-to-many problem. Would be useful if authors could compare the proposed method with finetuning vanilla GPT-2 modeling P(x,y) with the evaluated dataset as explored in [1].

[1] https://arxiv.org/pdf/1908.01841.pdf
clarity,_quality,_novelty_and_reproducibility: The paper is clearly written and easy to follow. The use of EM method for response generation which does not suffer from model collapse is novel. However, it's not clear how the approach compares to existing work that have attempted to solve similar problem of diverse response generation using GAN, reinforcement learning, or joint distribution modeling. 
summary_of_the_review: The proposed method of Equal-size hard EM is novel and well motivated and supported by experimental results. However, the paper needs to add other baselines to really evaluate the diverse response generation capability of the proposed approach.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: In order to solve the issue of generic responses generated by the neural dialogue systems because of one-to-many mapping phenomenon, the authors of the paper studied the state-of-the-art methods including the methods for training time and reference time, the application of the standard EM, Soft-EM, Hard EM and etc. In the paper, the analysis of the disadvantages of these methods was given and a new method EqHard-EM algorithm was proposed. 
In EqHard-EM, this paper proposed a multi-adaptor architecture with multi-decoder and a shared encoder, adopted hard assignments from Hard-EM but impose equal-assignment constraints to ensure all decoders are adequately trained. To evaluate the new method, the paper included the experiments on Weibo and Open Subtitles. The results and the comparison with the state-of-the-are methods and EM variants have been given in the paper. In the results, the EqHard-EM demonstrated its advantages again EM variants and a few state-of-the-art methods. 
strength_and_weaknesses: Strength: 
The study of the literature for the solutions to justify the  one-to-many mapping phenomenon was complete. The paper also analyzed the issues coming from standard EM, Soft-EM and Hard-EM. 
In the proposed new method, new neural architecture based on multi-adaptor architecture was created on top of and beyond a few state-of-the-art methods. Training methods and the theoretical analysis was given. 
Weakness or questions: 
It was not that clear how the Theorem 1 came up. Can the author elaborate more details of it? 
The organization of the paper can be improved. It maybe better to have related work section (Section 4) come earlier because now there are duplications among Section 1 (introduction), Section 2.2 and Section 4. 
clarity,_quality,_novelty_and_reproducibility: The proposed method in the paper is novel and the ideas came from a completed analysis of related methods and the innovations beyond them. 
As I said in the above weakness, the organization of the paper can be improved.  
summary_of_the_review: The method in the paper is new and developed based on a good study of related method and innovations/extensions beyond them. It justifies the one-to-many mapping phenomenon and the results of the two experiments demonstrates advantages comparing to others. 
This paper will benefit the readers with novel approaches.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This paper explores mixture models for diverse dialogue generation, following the same pipeline proposed by Shen et al., (2019). This method adopts a multi-decoder model to address the one-to-many mapping phenomenon, where a multi-adapter architecture is used to form the multi-decoder model. Based on that, the EM algorithm (soft or hard) is leveraged to learn the latent variable (i.e.,  the choice of the decoder). The authors further handle the non-training collapse of the Hard-EM algorithm by introducing equal-assignment constraints, which ensure that all decoders are adequately trained. Automatic evaluations on Weibo and OpenSubtitles datasets prove the effectiveness of introduced equal-assignment constraints, compared with the original EM algorithm.
strength_and_weaknesses: Strengths:
- This paper explores mixture models for dialogue generation tasks. 
- This paper proposes EqHard-EM to handle the non-training collapse of the Hard-EM algorithm.

Weaknesses:
- It is an incremental work of Shen et al., (2019) and the novelty of the proposed method is limited. 
- It is fine to apply and extend the method of Shen et al., (2019) in dialogue generation tasks, but the experiments are not good enough, including implementing all model variants in Shen et al., (2019) and full human evaluation. 
- The main contribution of this paper is to introduce equal-assignment constraints to handle the non-training collapse of the Hard-EM algorithm. However, the E-step of Hard-EM in this paper is different from Shen et al., (2019), which leverages learned prior (lp) or uniform prior (up) to ease this issue. It is difficult to judge that this method could lead to further improvements compared with Shen et al., (2019)'s version of Hard-EM.


Detailed Comments:

Overall, this paper is easy to follow and this idea is straightforward. I like this idea of introducing mixture models for diverse dialogue generation. But the experiments are not convincing enough to me. The E-step of Hard-EM in this paper is different from Shen et al., (2019), which leverages learned prior (lp) or uniform prior (up) to ease the non-training collapse of the Hard-EM algorithm. It is not sure that this method could lead to further improvements when combined with the version of the Hard-EM algorithm of Shen et al., (2019). It is necessary to reproduce the version of Shen et al., (2019) in the dialogue generation task.

While Shen et al., (2019) have explored four model variants and some crucial tricks (e.g., the effect of dropout) for diverse machine translation, it is better to revisit these models and details in dialogue generation. In addition, this paper lacks full human evaluations across two datasets and different methods, and automatic evaluations are not convincing enough in dialogue generation tasks. The authors only perform human evaluations on beam search and the proposed method, which should be improved.


Questions for the Author(s):
- As shown in Table 5, the example of the Hard-EM algorithm tends to repeat. Since we directly optimize the conditional language model in M-step, these examples are weird. What is your thought about it?
- Why is the E-step of the Hard-EM algorithm in this paper different from Shen et al., (2019)? ( i.e.,  argmax_{z} p (z| x,y; \theta) v.s.  argmax_{z} p (y| z,y; \theta)  )
- Have you evaluated the effect of the dropout during the EM algorithm? 
- How about training different decoders instead of introducing adapter layers?


clarity,_quality,_novelty_and_reproducibility: Please refer to Detailed Comments.

summary_of_the_review: This paper investigates introducing mixture models for diverse dialogue generation and proposes EqHard-EM to handle the non-training collapse of the Hard-EM algorithm. But the experiments are not convincing enough to me, including lacking more details of all model variants in Shen et al., (2019) and full human evaluation. The E-step of Hard-EM in this paper is also different from Shen et al., (2019), and it is uncertain whether this method will be further improved when combined with the version of Shen et al., (2019).
review_title: null

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: This paper addresses the issue of generic response generation by first using adapters for efficient multi-decoder frameworks, and next using a balanced EM algorithm to train the EM of the multi-decoders. The main contribution is to train the balanced EM by the classic Hungarian algorithm. Experiments compare the proposed model with other methods that address the same problem.

strength_and_weaknesses: Strengths: The idea is clear and the implementation of the idea should be easy.

Weakness:
The authors argue that the E-step in the proposed model contributes to less than 1% of the total training time. I want to know if the full experimental results can be provided, including the total training time, the number of EM steps, and the exact time of the E-step and M-step in the training time. Also, the time efficiency should be compared for all methods in your experiments.

The experimental results are not convincing.  First, only results on BLEU-2 of BLEU-n are given. How about BLEU-1? Also, human evaluation results of Table1 should be given.  Also, I think a pretrained T5 should not be proper to compare here, as it is not for dialog generation. Perhaps DialogGPT is better if the authors want to compare a pre-trained model. However, I found most of the compared methods except the proposed one perform even worse than T5, which is hard to believe.
clarity,_quality,_novelty_and_reproducibility: The authors should publicize their code, and testing results, which are not stated in the paper.
summary_of_the_review: Overall, I think the idea is reasonable but the experiments are a bit weak.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: This paper tackles the well-known generic response problem of neural response generation systems, due to the one-to-many issue. It provides a good summary and categorization of related work for generating more diverse responses and proposes a modified EM algorithm, EqHard-EM algorithm, to train a multi-decoder model for improved response diversity. The model used in this work includes multiple decoders with adaptors and a shared encoder. EqHard-EM assigns each decoder a sample and imposes an equal assignment constraint to ensure all decoders are well-trained.  Experimentation on Weibo and Open Subtitles datasets compares the proposed approach with state-of-the-art methods and EM variants (Soft-EM, Hard-EM, etc.), demonstrating the advantages of the proposed approach. 
Reviewers made a set of suggestions, including experimentation with GPT2, including results with BLEU variants, and including detailed run-time analysis, which are added during the author responses. There seem to be some misunderstandings on the first version of the paper, which the authors clarified in their response.
justification_for_why_not_higher_score: The paper is clear in general, but can be presented in a more self-contained way, with references to issues, such as decoder collapse problem.
justification_for_why_not_lower_score: Reviews include constructive suggestions and misunderstandings, which were clarified by the authors. The proposed approach is interesting and useful, and tackles an important problem, hence the paper is expected to attract interest from the response generation community.
Decision: Accept: poster