Venue: ICLR 2018 conference

Reviews:

final_score: rating: 5: Marginally below acceptance threshold
review_content: This paper argues that the objective of RNN is not expressive enough to capture the good generation quality. In order to address the problems of RNN in generating languages, this paper combines the RNN language model with several other discriminatively trained models, and the weight for each sub model is learned through beam search. 

I like the idea of using Grice’s Maxims of communication to improve the language generation. Human evaluation shows significant improvement over the baseline. I have some detailed comments as follows:

- The repetition model uses the samples from the base RNNs as negative examples. More analysis is needed to show it is a good negative sampling method.

- As Section 3.2.3 introduced, “the unwanted entailment cases include repetitions and paraphrasing”. Does it mean the entailment model also handles repetition problem? Do we still need a separate repetition model? How about a separate paraphrasing model?

- Equation 6 and the related text are not very clearly represented. It would be better to add more intuition and better explained. 

- In the Table 2, the automated bleu scores of L2W algorithm for Tripadvisor is very low (0.34 against 24.11). Is this normal? More explanation is needed here.

- For human judgement, how many scores does each example get? It would be better to get multiple workers on M-Turk to label the same example, and compute the mean and variance. One score per example may not be reliable. 

- It would be interesting to see deeper analysis about how each model in the objectives influence the actual language generation.
review_title: This paper combines RNN language model with several discriminatively trained models to improve the language generation. I like the idea of using Grice’s Maxims of communication to improve the language generation. However, some parts need to be further clarified and it would be nice to see more related analysis. 

final_score: rating: 6: Marginally above acceptance threshold
review_content: This paper proposes to bring together multiple inductive biases that hope to correct for inconsistencies in sequence decoding. Building on previous works that utilize modified objectives to generate sequences, this work proposes to optimize for the parameters of a pre-defined combination of various sub-objectives. The human evaluation is straight-forward and meaningful to compensate for the well-known inaccuracies of automatic evaluation. 

While the paper points out that they introduce multiple inductive biases that are useful to produce human-like sentences, it is not entirely correct that the objective is being learnt as claimed in portions of the paper. I would like this point to be clarified better in the paper. 

I think showing results on grounded generation tasks like machine translation or image-captioning would make a stronger case for evaluating relevance. I would like to see comparisons on these tasks. 

---- 
After reading the paper in detail again and the replies, I am downgrading my rating for this paper. While I really like the motivation and the evaluation proposed by this work, I believe that fixing the mismatch between the goals and the actual approach will make for a stronger work. 

As pointed out by other reviewers, while the goals and evaluation seem to be more aligned with Gricean maxims, some components of the objective are confusing. For instance, the length penalty encourages longer sentences violating quantity, manner (be brief) and potentially relevance. Further, the repetition model address the issue of RNNs failing to capture long-term contextual dependencies -- how much does such a modified objective affect models with attention / hierarchical models is not clear from the formulation. 

As pointed out in my initial review evaluation of relevance on the current task is not entirely convincing. A very wide variety of topics are feasible for a given context sentence. Grounded generation like MT / captioning would have been a more convincing evaluation. For example, Wu et al. (and other MT works) use a coverage term and this might be one of the indicators of relevance. 

Finally, I am not entirely convinced by the update regarding "learning the objective". While I agree with the authors that the objective function is being dynamically updated, the qualities of good language is encoded manually using a wide variety of additional objectives and only the relative importance of each of them is learnt. 
review_title: Neat contribution that integrates previous work

final_score: rating: 4: Ok but not good enough - rejection
review_content: This paper proposes to improve RNN language model generation using augmented objectives inspired by Grice's maxims of communication. The idea is to combine the standard word-by-word decoding objective with additional objectives that reward sentences following these maxims. The proposed decoding objective is not new; reseachers in machine translation 
 have worked on it referring to it as loss-augmented decoding: http://www.cs.cmu.edu/~nasmith/papers/gimpel+smith.naacl12.pdf
The use of RNNs in this context might be novel though.

Pros:
- Well-motivated and ambitious goals

- Human evaluation conducted on the outputs.

Cons:
- My main concern is that it is unclear whether the models introduced are indeed implementing the Gricean maxims. For eaxample, the repetition model would not only discourage the same word occurring twice, but also a similar word (according to the word vectors used) to follow another one. 

- Similary, for the entailment model, what is an "obvious" entailment"? Not sure we have training data for this in particular. Also, entailment suggests textual cohesion, which is conducive to the relation maxim. If this kind of model is what we need, why not take a state-of-the-art model?

- The results seem to be inconsistent. The working vocabulary doesn't help in the tripAdvior experiment, while the RNN seems to work very well on the ROCstory data. While there might be good reasons for these, the point for me is that we cannot trust that the models added to the objective do what they are supposed to do.

- Are the negative examples generated for the repetition model checked that they contain repetitions? Shouldn't be difficult to do. 

- Would be better to give the formula for the length model, the description is intuition but it is difficult to know exactly what the objective is

- In algorithm 1, it seems like we fix in advance the max length of the sentence (max-step).  Is this the case? If so why? Also, the proposed learning algorithm only learns how to mix pre-trained models, not sure I agree they learn the objective. It is more of an ensembling.

- As far as I can tell these ideas could have been more simply implemented by training a re-ranker to score the n-best outputs of the decoder. Why not try it? They are very popular in text generation tasks.
review_title: Well-motivated goals, but the methods don't achieve them.

Meta-review: I (and some of the reviewers) find the general motivation quite interesting (operationalizing the Gricean maxims in order to improve language generation). However, we are not  convinced that the actual model encodes these maxims in a natural and proper way.  Without this motivation, the approach can be regarded as a set of heuristics which happen to be relatively effective on a couple of datasets.  In other words, the work seems too preliminary to be accepted at the conference.

Pros:
-- Interesting motivation (and potential impact on follow-up work)
-- Good results on a number of datasets
Cons:
-- The actual approach can be regarded as a set of heuristics, not necessarily following from the maxims
-- More serious evaluation needed (e.g., image captioning or MT) and potential better ways of encoding the maxims

It is suitable for the workshop track, as it is likely to stimulate an interesting discussion and more convincing follow-up work.


Decision: Invite to Workshop Track