Title: Reference-Aware Language Models
Authors: Zichao Yang
Phil Blunsom
Chris Dyer
Wang Ling
Venue: ICLR 2017
Keywords: Natural language processing
Deep learning

Abstract: We propose a general class of language models that treat reference as an explicit stochastic latent variable. This architecture allows models to create mentions of entities and their attributes by accessing external databases (required by, e.g., dialogue generation and recipe generation) and internal state (required by, e.g. language models which are aware of coreference). This facilitates the incorporation of information that can be accessed in predictable locations in databases or dis- course context, even when the targets of the reference may be rare words. Experiments on three tasks show our model variants outperform models based on deterministic attention.

TL;DR: reference-aware language models

Review discussion:
Hi, I have a few (mostly clarification) questions:
(1) Does any of the models presented in Table 4, 5, 6 assume that the mention boundaries are given to the model during test?
(2) Do you provide gold co-reference chains for training for the coref-based LM?
(3) Do you provide gold co-reference chains for testing?
(4) Also, can you comment on how much "entity state update" in section 2 helps improve the performance for the coref-based LM?
Thanks.
Thanks for your question.
1) The mentions are given in the model.
But we time the probability altogether to get an upper bound on perplexity. An ideal solution is to marginalize them out in the decoding/test process.
2) Yes, we provide the co-reference chains for training and testing
3) We used an intuitive way to update the entity state and it performs well, so we didn't experiment other ways to update the entity state. There are other more sophisticated ways, like using a LSTM to encode each entity chain. It's harder to implement.
All the quantitative results are given in terms of perplexity and BLEU scores. It would have been more convincing to have some form of human evaluation.
Also, are there any empirical results showing that the BLEU metric can be applied to recipe modeling or similar tasks?
After all, it is known that BLEU is very sensitive to pronoun overlap, which could be negatively correlated with outputting correct reference words.
Thanks for the comment.
Although there can be some noise in BLEU, we think there is a large enough sample size in the val/test set to reduce the noise. Our main goal is to develop methods that can address references to existing structures.
Having a large validation/test set, won't reduce the bias in the evaluation method (e.g. idiosyncratic aspects/features BLEU focuses on). I agree it should reduce variance (or noise), but my concern was mainly related to the bias.
This paper introduces pointer-network neural networks, which are applied to referring expressions in three small-scale language modeling tasks: dialogue modeling, recipe modeling and news article modeling.
When conditioned on the co-reference chain, the proposed models outperform standard sequence-to-sequence models with attention.
The proposed models are essentially variants of pointer networks with copy mechanisms (Gulcehre et al., 2016; Gu et al., 2016; Ling et al., 2016), which have been modified to take into account reference chains.
As such, the main architectural novelty lies in 1) restricting the pointer mechanism to focus on co-referenced entities, 2) applying pointer mechanism to 2D arrays (tables), and 3) training with supervised alignments.
Although useful in practice, these are minor contributions from an architectural perspective.
The empirical contributions are centred around measuring perplexity on the three language modeling tasks. Measuring perplexity is typical for standard language modeling tasks, but is really an unreliable proxy for dialogue modeling and recipe generation performance.
In addition to this, both the dialogue and recipe tasks are tiny compared to standard language modeling tasks. This makes it difficult to evaluate the impact of the dialogue and recipe modeling results.
For example, if one was to bootstrap from a larger corpus, it seems likely that a standard sequence-to-sequence model with attention would yield performance comparable to the proposed models (with enough data, the attention mechanism could learn to align referring entities by itself). The language modeling task on news article (Gigaword) seems to yield the most conclusive results.
However, the dataset for this task is non-standard and results are provided for only a single baseline.
Overall, this limits the conclusions we can draw from the empirical experiments.
Finally, the paper itself contains many errors, including mathematical errors, grammatical errors and typos:
- Eq. (1) is missing a sum over $z_i$.
- "into the a decoder LSTM" -> "into the decoder LSTM"
- "denoted as his" -> "denoted as"
- "Surprising," -> "Surprisingly,"
- "torkens" -> "tokens"
- "if follows that the next token" -> "the next token"
- In the "COREFERENCE BASED LANGUAGE MODEL" sub-section, what does $M$ denote?
- In the sentence: "The attribute of each column is denoted as $s_c, where $c$ is the c-th attribute". For these definitions to be make sense, $s_c$ has to be a one-hot vector. If yes, please clarify this in the text.
- "the weighted sum is performed" -> "the weighted sum is computed"
- "a attribute" -> "an attribute"
- In the paragraph on Pointer Switch, change $p(z_{i,v} |s_{i,v}) = 1$ -> $p(z_{i,v} |s_{i,v}) = 0$.
- In the "Table Pointer" paragraph, I assume you mean outer-product instead of cross-product?
Otherwise, I don't see how the equations add up.
Other comments:
- For the "Attention based decoder", is the attention computed using the word embeddings themselves or the hidden states of the sentence encoder?
Also, it applied only to the previous turn of the dialogue or to the entire dialogue history? Please clarify this.
- What's the advantage of using an "Entity state update" rule, compared to a pointer network or copy network, which you used in the dialogue and recipe tasks? Please elaborate on this.
- In the Related Work section, the following sentence is not quite accurate: "For the task oriented dialogues, most of them embed the seq2seq model in traditional dialogue systems while our model queries the database directly.". There are task-oriented dialogue models which do query databases during natural language generation. See, for example, "A Network-based End-to-End Trainable Task-oriented Dialogue System" by Wen et al.
1) We thank the reviewer for the constructive review and feedback. We have rewritten our paper to make it more readable. Please refer to our new draft for the revision.
2) We want to point out that although the data sets (dialogues and recipes) we used are relatively small, they are new data sets that we build ourselves. For the coreference based language models, there is no standard data set either. Building the data sets is one of our contributions. We don’t agree with the reviewer’s comment that more data would decrease the necessity for model innovation.
Moreover, labeled data sets are hard and expensive to get.
3) Human evaluation would be ideal, but setting up the experiments is troublesome. For dialogue, since the table entries are quite rare in the sentences, so other evaluation metrics such as BLEU, METEOR are not feasible for this task. We report both ppl and BLEU for recipe generation, the BLEU score correlates highly with human evaluation for recipe task, according to Kiddon et al., 2016. Coreference based LM is a pure language model, it is hard to define the generation task under this context.
4) For “Attention based decoder”, we use attention mechanism over hidden states of the sentence encoder. We apply the attention only on previous turn of dialogue since it is mostly relevant.
5) We propose the  “Entity state update”, because there are multiple mentions for each entity. In coreference resolution, we only care about which entity the next mention refers to.
So we need to keep one state for each entity in the coreference resolution phrase. With this part, we can do coreference resolution together with language model. This is the main motivation with entity update process.
Although we did not report the coreference resolution result in the paper, the model design enables us to do it and it is one of the future directions to explore.
6) The work by Wen et al still has the table query part separate from the other neural network model, while our model is truly end-to-end.
7)In coreference LM, $M$ is the number of mentions for each entity.
Thank you for the rebuttal.
I appreciate the clarifications and revision you have made of the paper.
However, I do not find sufficient grounds to change my scores.
You stated that the BLEU metric was validated for recipe generation in "Globally coherent text generation with neural checklist" by Kiddon et al. I've looked through their paper and indeed the BLEU metric is one of the automated metrics used.
However, no study is carried out to correlate it with human evaluation, so I don't understand how you see that as supporting your argument.
In fact, on page 8 the authors write "... neither BLEU nor METEOR is suitable for evaluating generated text in terms of their adherence to the provided goal and the agenda ...".
Furthermore, in their paper a nearest-neighbour baseline outperforms all other models w.r.t. BLEU-4. This is worrisome, since your paper does not include this as a baseline.
You stated that the difference between your proposed model and the model proposed in "A Network-based End-to-End Trainable Task-oriented Dialogue System" by Wen et al., is that your model is trained end-to-end while Wen et al. needs an intermediate neural network to extract the table representation. I think this is mainly due to the difference between your tasks; for Wen et al.'s task, it is not possible to extract the table a priori and therefore it must be inferred online using a neural network trained with labelled data. Note, Wen et al.
also use a fixed database (restaurant listings) across dialogues, for which no intermediate neural network is necessary.
This paper explores 3 language modeling applications with an explicit modeling of reference expressions: dialog, receipt generation and coreferences.
While these are important tasks for NLP and the authors have done a number of experiments, the paper is limited for a few reasons:
1. This paper is not clearly written and is pretty hard to follow some details.
In particular,  there are many obvious math errors, such as missing the marginalization sum in Eq (1), and P(z_{i,v}...) = 1 (should be 0 here) on page 5, pointer switch section.
2. The major novelty seems to be the 2-dimensional attention from the table and the pointer to the 2-D table. These are more of a customization of existing work to a particular task with 2-D tables as a part of the input to seq2seq model with both attentions and pointer networks.
3. The empirical results are not very conclusive yet, limited by either the relatively small data size, or the lack of well-established baseline for some new applications (e.g., the recipe generation task).
Overall, this paper, as it is for now, is more suitable for a workshop rather than for the main conference.
1) We thank the reviewer for the constructive review and feedback. We have rewritten the paper to make it more readable. Please refer to our new draft for the revision.
2) Coupling neural network with database is an interesting direction and we are the first (to the best of our knowledge) to explore in this direction. There lacks good data sets for this type of research.
3) We want to point out that although the data sets (dialogues and recipes) we used are relatively small, they are new data sets that we build ourselves. For the coreference based language models, there is no standard data set either. Building the data sets is one of our contributions. We have provided the state of art baseline methods (seq2seq with attention and lstm language models) on these newly constructed data set and shown our model performs better than those baselines.
This paper presents a new type of language model that treats entity references as latent variables. The paper is structured as three specialized models for three applications: dialog generation with references to database entries, recipe generation with references to ingredients, and text generation with coreference mentions.
Despite some opaqueness in details that I will discuss later, the paper does a great job making the main idea coming through, which I think is quite interesting and definitely worth pursuing further.
But it seems the paper was rushed into the deadline, as there are a few major weaknesses.
The first major weakness is that the claimed latent variables are hardly latent in the actual empirical evaluation. As clarified by the authors via pre-review QAs, all mentions were assumed to be given to all model variants, and so, it would seem like an over-claim to call these variables as latent when they are in fact treated as observed variables. Is it because the models with latent variables were too difficult to train right?
A related problem is the use of perplexity as an evaluation measure when comparing reference-aware language models to vanilla language models. Essentially the authors are comparing two language models defined over different event space, which is not a fair comparison. Because mentions were assumed to be given for the reference-aware language models, and because of the fact that mention generators are designed similar to a pointer network, the probability scores over mentions will naturally be higher, compared to the regular language model that needs to consider a much bigger vocabulary set. The effect is analogous to comparing language models with aggressive UNK (and a small vocabulary set) to a language models with no UNK (and a much larger vocabulary set).
To mitigate this problem, the authors need to perform one of the following additional evaluations: either assuming no mention boundaries and marginalizing over all possibilities (treating latent variables as truly latent), or showing other types of evaluation beyond perplexity, for example, BLEU, METEOR, human evaluation etc on the corresponding generation task.
The other major weakness is writing in terms of technical accuracy and completeness. I found many details opaque and confusing even after QAs. I wonder if the main challenge that hinders the quality of writing has something to do with having three very specialized models in one paper, each having a lot of details to be worked out, which may have not been extremely important for the main story of the paper, but nonetheless not negligible in order to understand what is going on with the paper.
Perhaps the authors can restructure the paper so that the most important details are clearly worked out in the main body of the paper, especially in terms of latent variable handling — how to make mention detection and conference resolution truly latent, and if and when entity update helps, which in the current version is not elaborated at all, as it is mentioned only very briefly for the third application (coreference resolution) without any empirical comparisons to motivate the update operation.
1) We thank the reviewer for the constructive review and feedback. We have rewritten the paper to make it more readable. Please refer to our new draft for the revision.
2) By latent, we mainly mean the decision of copy is latent, so we marginalize the decision variable in the training process in dialogue and recipe task. For the coreference based language model, because there are so many decisions, marginalize them out is intractable, so we treat everything as given.
But we measure the ppl as the product of all probabilities together, which gives us an upper bound on the ppl.
3) The reviewer misunderstood the comparison between ordinary language models and reference-aware language models. The words refereed are not UNK, they actually appear in the context, and exit in the vocabulary. Our main argument is that explicitly considering the reference decision is helpful for language model and this is our main contribution.
4) We propose the  “Entity state update”, because there are multiple mentions for each entity. In coreference resolution, we only care about which entity the next mention refers to.
So we need to keep one state for each entity in the coreference resolution phrase. With this part, we can do coreference resolution together with language model. This is the main motivation with entity update process.
Although we did not report the coreference resolution result in the paper, the model design enables us to do it and it is one of the future directions to explore.

Meta-review:
role: PC
data: comment: All of the reviewers point out clarity problems; while these may have been resolved in an updated version, the reviewers have not expressed that the matter is resolved. There are several questions raised about the use of perplexity, both whether the comparison is fair, and whether it is a valid proxy for more standard measures in NLP. The former seems to be more of an issue for this area chair, and the discussion did not convince me that it was adequately resolved.

Decision: Reject