Paper:
Conventional methods model open domain dialogue generation as a black box through end-to-end learning from large scale conversation data. In this work, we make the first step to open the black box by introducing dialogue acts into open domain dialogue generation. The dialogue acts are generally designed and reveal how people engage in social chat. Inspired by analysis on real data, we propose jointly modeling dialogue act selection and response generation, and perform learning with human-human conversations tagged with a dialogue act classifier and a reinforcement approach to further optimizing the model for long-term conversation. With the dialogue acts, we not only achieve significant improvement over state-of-the-art methods on response quality for given contexts and long-term conversation in both machine-machine simulation and human-machine conversation, but also are capable of explaining why such achievements can be made.

Review:
The authors use a distant supervision technique to add dialogue act tags as a conditioning factor for generating responses in open-domain dialogues. In their evaluations, this approach, and one that additionally uses policy gradient RL with discourse-level objectives to fine-tune the dialogue act predictions, outperform past models for human-scored response quality and conversation engagement.
While this is a fairly straightforward idea with a long history, the authors claim to be the first to use dialogue act prediction for open-domain (rather than task-driven) dialogue. If that claim to originality is not contested, and the authors provide additional assurances to confirm the correctness of the implementations used for baseline models, this article fills an important gap in open-domain dialogue research and suggests a fruitful future for structured prediction in deep learning-based dialogue systems.
Some points:
1. The introduction uses "scalability" throughout to mean something closer to "ability to generalize." Consider revising the wording here.
2. The dialogue act tag set used in the paper is not original to Ivanovic (2005) but derives, with modifications, from the tag set constructed for the DAMSL project (Jurafsky et al., 1997; Stolcke et al., 2000). It's probably worth citing some of this early work that pioneered the use of dialogue acts in NLP, since they discuss motivations for building DA corpora.
3. In Section 2.1, the authors don't explicitly mention existing DA-annotated corpora or discuss specifically why they are not sufficient (is there e.g. a dataset that would be ideal for the purposes of this paper except that it isn't large enough?)
3. The authors appear to consider only one option (selecting the top predicted dialogue act, then conditioning the response generator on this DA) among many for inference-time search over the joint DA-response space. A more comprehensive search strategy (e.g. selecting the top K dialogue acts, then evaluating several responses for each DA) might lead to higher response diversity.
4. The description of the RL approach in Section 3.2 was fairly terse and included a number of ad-hoc choices. If these choices (like the dialogue termination conditions) are motivated by previous work, they should be cited. Examples (perhaps in the appendix) might also be helpful for the reader to understand that the chosen termination conditions or relevance metrics are reasonable.
5. The comparison against previous work is missing some assurances I'd like to see. While directly citing the codebases you used or built off of is fantastic, it's also important to give the reader confidence that the implementations you're comparing to are the same as those used in the original papers, such as by mentioning that you can replicate or confirm quantitative results from the papers you're comparing to. Without that there could always be the chance that something is missing from the implementation of e.g. RL-S2S that you're using for comparison.
6. Table 5 is not described in the main text, so it isn't clear what the different potential outputs of e.g. the RL-DAGM system result from (my guess: conditioning the response generation on the top 3 predicted dialogue acts?)
7. A simple way to improve the paper's clarity for readers would be to break up some of the very long paragraphs, especially in later sections. It's fine if that pushes the paper somewhat over the 8th page.
8. A consistent focus on human evaluation, as found in this paper, is probably the right approach for contemporary dialogue research.
9. The examples provided in the appendix are great. It would be helpful to have confirmation that they were selected randomly (rather than cherry-picked).