Venue: ICLR 2018 conference

Reviews:

final_score: rating: 7: Good paper, accept
review_content: The paper describes a technique to incorporate dialog acts into neural conversational agents.  This is very interesting work.  Existing techniques for neural conversational agents essentially mimic the data in large corpora of message-response pairs and therefore do not use any notion of dialog act.  A very important type of dialog act is "switching topic", often done to ensure that the conversation will continue.  The paper describes a classifier that predicts the dialog act of the next utterance.  The next utterance is then generated based on this dialog act.  The paper also describes how to increase the relevance of responses and the length of conversations by self reinforcement learning.  This is also very interesting.  The empirical evaluation demonstrates the effectiveness of the approach.  The paper is also well written.  I do not have any suggestion for improvement.  This is good work that should be published.
review_title: review

final_score: rating: 7: Good paper, accept
review_content: The authors use a distant supervision technique to add dialogue act tags as a conditioning factor for generating responses in open-domain dialogues. In their evaluations, this approach, and one that additionally uses policy gradient RL with discourse-level objectives to fine-tune the dialogue act predictions, outperform past models for human-scored response quality and conversation engagement.
While this is a fairly straightforward idea with a long history, the authors claim to be the first to use dialogue act prediction for open-domain (rather than task-driven) dialogue. If that claim to originality is not contested, and the authors provide additional assurances to confirm the correctness of the implementations used for baseline models, this article fills an important gap in open-domain dialogue research and suggests a fruitful future for structured prediction in deep learning-based dialogue systems.

Some points:
1. The introduction uses "scalability" throughout to mean something closer to "ability to generalize." Consider revising the wording here.
2. The dialogue act tag set used in the paper is not original to Ivanovic (2005) but derives, with modifications, from the tag set constructed for the DAMSL project (Jurafsky et al., 1997; Stolcke et al., 2000). It's probably worth citing some of this early work that pioneered the use of dialogue acts in NLP, since they discuss motivations for building DA corpora.
3. In Section 2.1, the authors don't explicitly mention existing DA-annotated corpora or discuss specifically why they are not sufficient (is there e.g. a dataset that would be ideal for the purposes of this paper except that it isn't large enough?)
3. The authors appear to consider only one option (selecting the top predicted dialogue act, then conditioning the response generator on this DA) among many for inference-time search over the joint DA-response space. A more comprehensive search strategy (e.g. selecting the top K dialogue acts, then evaluating several responses for each DA) might lead to higher response diversity.
4. The description of the RL approach in Section 3.2 was fairly terse and included a number of ad-hoc choices. If these choices (like the dialogue termination conditions) are motivated by previous work, they should be cited. Examples (perhaps in the appendix) might also be helpful for the reader to understand that the chosen termination conditions or relevance metrics are reasonable.
5. The comparison against previous work is missing some assurances I'd like to see. While directly citing the codebases you used or built off of is fantastic, it's also important to give the reader confidence that the implementations you're comparing to are the same as those used in the original papers, such as by mentioning that you can replicate or confirm quantitative results from the papers you're comparing to. Without that there could always be the chance that something is missing from the implementation of e.g. RL-S2S that you're using for comparison.
6. Table 5 is not described in the main text, so it isn't clear what the different potential outputs of e.g. the RL-DAGM system result from (my guess: conditioning the response generation on the top 3 predicted dialogue acts?)
7. A simple way to improve the paper's clarity for readers would be to break up some of the very long paragraphs, especially in later sections. It's fine if that pushes the paper somewhat over the 8th page.
8. A consistent focus on human evaluation, as found in this paper, is probably the right approach for contemporary dialogue research.
9. The examples provided in the appendix are great. It would be helpful to have confirmation that they were selected randomly (rather than cherry-picked).
review_title: Strong results from a simple idea

final_score: rating: 4: Ok but not good enough - rejection
review_content: The topic discussed in this paper is interesting. Dialogue acts (DAs; or some other semantic relations between utterances) are informative to increase the diversity of response generation. It is interesting to see how DAs are used for conversational modeling, however this paper is difficult for me to follow. For example:

1) the caption of section 3.1 is about supervised learning, however the way of describing the model in this section sounds like reinforcement learning. Not sure whether it is necessary to formulate the problem with a RL framework, since the data have everything that the model needs as for a supervised learning.
2) the formulation in equation 4 seems to be problematic
3) "simplify pr(ri|si,ai) as pr(ri|ai,ui−1,ui−2) since decoding natural language responses from long conversation history is challenging" to my understanding, the only difference between the original and simplified model is the encoder part not the decoder part. Did I miss something?
4) about section 3.2, again I didn't get whether the model needs RL for training.
5) "We train m(·, ·) with the 30 million crawled data through negative sampling." not sure I understand the connection between training $m(\cdot, \cdot)$ and the entire model.
6) the experiments are not convincing. At least, it should show the generation texts were affected about DAs in a systemic way. Only a single example in table 5 is not enough.
review_title: Interesting topic discussed, but the paper is not good enough

Meta-review: This work takes dialogue acts into account to generate responses in a human-machine conversation. However, incorporating dialogue acts into open-domain dialogue was already the focus of Zhao et al's ACL 2017 paper, Learning Discourse-level Diversity for Neural Dialog Models using Conditional Variational Autoencoders, and using dialogue acts in a policy for human-machine conversation was also an idea that already appeared in Serban et al 2017, A Deep Reinforcement Learning Chatbot. Despite the authors' response that tries to adjust their claims and incorporate a more thorough overview, I encourage the authors to re-work their research with a much more careful and reliable examination of previous work and how their effort should be understood in that more comprehensive context.
Decision: Reject