Title: REFERENCE-AWARE LANGUAGE MODELS

Abstract: We propose a general class of language models that treat reference as an explicit stochastic latent variable. This architecture allows models to create mentions of entities and their attributes by accessing external databases (required by, e.g., dialogue generation and recipe generation) and internal state (required by, e.g. language models which are aware of coreference). This facilitates the incorporation of information that can be accessed in predictable locations in databases or discourse context, even when the targets of the reference may be rare words. Experiments on three tasks show our model variants outperform models based on deterministic attention.

1. INTRODUCTION:

Referring expressions (REs) in natural language are noun phrases (proper nouns, common nouns, and pronouns) that identify objects, entities, and events in an environment. REs occur frequently and they play a key role in communicating information efficiently. While REs are common, previous works neglect to model REs explicitly, either treating REs as ordinary words in the model or replacing them with special tokens. Here we propose a language modeling framework that explicitly incorporates reference decisions.

In Figure 1 we list examples of REs in the context of the three tasks that we consider in this work. Firstly, reference to a database is crucial in many applications. One example is in task oriented dialogue where access to a database is necessary to answer a user's query (Young et al., 2013; Li et al., 2016; Vinyals & Le, 2015; Wen et al., 2015; Sordoni et al., 2015; Serban et al., 2016; Bordes & Weston, 2016; Williams & Zweig, 2016; Shang et al., 2015; Wen et al., 2016) . Here we consider the domain of restaurant recommendation where a system refers to restaurants (name) and their attributes (address, phone number etc) in its responses. When the system says "the nirala is a nice restaurant", it refers to the restaurant name the nirala from the database. Secondly, many models need to refer to a list of items (Kiddon et al., 2016; Wen et al., 2015) . In the task of recipe generation from a list of ingredients (Kiddon et al., 2016) , the generation of the recipe will frequently reference these items. As shown in Figure 1 , in the recipe "Blend soy milk and . . . ", soy milk refers to the ingredient summaries. Finally, we address references within a document (Mikolov et al., 2010; Ji et al., 2015; Wang & Cho, 2015) , as the generation of words will ofter refer to previously generated words. For instance the same entity will often be referred to throughout a document. In Figure 1 , the entity you refers to I in a previous utterance.

In this work we develop a language model that has a specific module for generating REs. A series of latent decisions (should I generate a RE? If yes, which entity in the context should I refer to? How should the RE be rendered?) augment a traditional recurrent neural network language model and the two components are combined as a mixture model. Selecting an entity in context is similar to familiar models of attention (Bahdanau et al., 2014) , but rather than being a deterministic function that reweights representations of elements in the context, it is treated as a distribution over contextual elements which are stochastically selected and then copied or, if the task warrants it, transformed (e.g., a pronoun rather than a proper name is produced as output). Two variants are possible for updating the RNN state: one that only looks at the generated output form; and a second that looks at values of the latent variables. The former admits trivial unsupervised learning, latent decisions are conditionally independent of each other given observed context, whereas the latter enables more expressive models that can extract information from the entity that is being referred to. In each of the three tasks, we demonstrate our reference aware model's efficacy in evaluations against models that do not explicitly include a reference operation.

Our contributions are as follows:

• We propose a general framework to model reference in language and instantiate it in the context of dialogue modeling, recipe generation and coreference based language models. • We build three data sets to test our models. There lack existing data sets that satisfy our need, so we build these data sets ourselves. These data sets are either built on top existing data set (we constructed the table for DSTC2 data set for dialogue evaluation), crawled from websites (we crawled all recipes in www.allrecipes.com) or annotated with NLP tools (we annotate the coreference with Gigaword corpus for our evaluation). • We perform comprehensive evaluation of our models on the three data sets and verify our models perform better than strong baselines.

2. REFERENCE-AWARE LANGUAGE MODELS:

Here we propose a general framework for reference-aware language models.

We denote each document as a series of tokens x 1 , . . . , x L , where L is the number of tokens in the document. Our goal is to maximize the probabilities p(x i | c i ), for each word in the document based on its previous context c i = x 1 , . . . , x i-1 . In contrast to traditional neural language models, we introduce a variable at each position z i , which controls the decision on which source x i is generated from. The token conditional probably is then obtained by:

EQUATION

In dialogue modeling and recipe generation, z i will simply taken on values in {0, 1}. Where z i = 1 denotes that x i is generated as a reference, either to a database entry or an item in a list. However, z i can also be defined as a distribution over previous entities, allowing the model to predict x i conditioned on its a previous mention word. This will be the focus of the coreference language model. When z i is not observed (which it generally will not be), we will train our model to maximize the marginal probability in Eq. 1 directly.

2.1. DIALOGUE MODEL WITH DATABASE SUPPORT:

We first apply our model on task-oriented dialogue systems in the domain of restaurant recommendations, and work on the data set from the second Dialogue State Tracking Challenge (DSTC2) (Henderson et al., 2014) . We can observe from this example, users get recommendations of restaurants based on queries that specify the area, price and food type of the restaurant. We can support the system's decisions by incorporating a mechanism that allows the model to query the database allowing the model to find restaurants that satisfy the users queries. Thus, we crawled TripAdvisor for restaurants in the M: Hello , welcome to the Cambridge restaurant system? You can ask for restaurants by area, price range or food type . How may I help you? U: i want a moderately priced restaurant in the north town M: the nirala is a nice restaurant in the north of town in the moderate price range U: what is the address M: Sure , the nirala is on 7 Milton Road Chesterton Cambridge area, where the dialog dataset was collected. Then, we remove restaurants that do not appear in the data set and create a database with 109 entries with restaurants and their attributes (e.g. food type). A sample of our database is shown in Table . 2. We can observe that each restaurant contains 6 attributes that are generally referred in the dialogue dataset. As such, if the user requests a restaurant that serves "indian" food, we wish to train a model that can search for entries whose "food" column contains "indian". Now, we describe how we deploy a model that fulfills these requirements. We build a model based on the hierarchical RNN model described in (Serban et al., 2016) , as in dialogues, the generation of the response is not only dependent on the previous sentence, but on all sentences leading to the response. We assume that a dialogue is alternated between a machine and a user. An illustration of the model is shown in Figure 2 .

Consider a dialogue with T turns, and the utterance from a user is denoted as X = {x i } T i=1 , where i is the i-th utterance, whereas the utterance from a machine is denoted as Y = {y i } T i=1 , where i is the i-th utterance. We define

x i = {x ij } |xi| j=1 , y i = {y iv } |yi| v=1

, where x ij denotes the j-th token in the i-th utterance from the user, whereas y iv denotes the v-th token in the i-th utterance from the machine. Finally, |x i | and |y i | denote the number of tokens in the user and machine utterances, respectively. The dialogue sequence starts with machine utterance {y 1 , x 1 , y 2 , x 2 , . . . , y T , x T }. We would like to model the utterances from the machine

p(y 1 , y 2 , . . . , y T |x 1 , x 2 , . . . , x T ) = ∏ i p(y i |y <i , x <i ) = ∏ i,v p(y i,v |y i,<v , y <i , x <i ),

where y <i denotes all the utterances before i and y i,<v denotes the first v -1 tokens in the i-th utterance of the machine. A neural model is employed to predict p(y i,v |y i,<v , y <i , x <i ), which operates as follows:

Sentence Encoder: We first encode previous utterances y <i and x <i into continuous space by generating employing a LSTM encoder. Thus, for a given utterance x i , and start with the initial LSTM state h x i,0 and apply the recursion

h x i,j = LSTM E (W E x i,j , h x i,j-1 ),

where W E x i,j denotes a word embedding lookup for the token x i,j , and LSTM E denotes the LSTM transition function described in Hochreiter & Schmidhuber (1997) . The representation of the user utterance is represented by the final LSTM state h x i = h x i,|xi| . The same process is applied to obtain the machine utterance representation h y i = h y i,|yi| . Turn Encoder: Then, combine all the representations of all the utterances with a second LSTM, which encodes the sequence {h y 1 , h x 1 , ..., h y i , h x i } into a continuous vector. Once again, we start with an initial state u 0 and feed each of the utterance representation to obtain the following LSTM state, until the final state is obtained. For simplicity, we shall refer to this as u i , which can be seen as the hierarchical encoding of the previous i utterances.

Seq2Seq Decoder: As for decoding, in order to generate each utterance y i , we can feed u i-1 into the decoder LSTM as the initial state s i,0 = u i-1 and decode each token in y i . Thus, we can express the decoder as:

s y i,v = LSTM D (W E y i,v-1 , s i,v-1 ), p y i,v = softmax(W s y i,v ),

where the desired probability p(y i,v |y i,<v , y <i , x <i ) is expressed by p y i,v . Attention based decoder: We can also incorporate the attention mechanism in our hierarchical model. An attention model builds a representation d by averaging over a set of vectors p. We define the attention function as a = ATTN(p, q), where a is a probability distribution over the set of vectors p, conditioned on any input representation q. A full description of this operation is described in (Bahdanau et al., 2014) . Thus, for each generated token y i,v , we compute the attentions a i,v , conditioned on the current decoder state s y i,v , obtaining the attentions over input tokens from previous turn (i-1). We denote the vector of all tokens in previous turn as

h x,y i-1 = [{h x i-1,j } |xi-1| j=1 , {h y i-1,v } |yi-1| v=1 ]. Let K = |h x,y

i-1 | be the number of tokens in previous turn. Thus, we obtain the attention probabilities over all previous tokens a i,v as ATTN(s y i,v , h x,y i-1 ). Then, the weighted sum is computed over these probabilities

d i,v = ∑ k∈K a i,v,k h x,y i-1,k

, where a i,v,k is the probability of aligning to the k-th token from previous turn. The resulting vector d i,v is used to obtain the probability of the following word p y i,v . Thus, we express the decoder as: 

s y i,v = LSTM D ([W E y i,v-1 , d i,v-1 ], s i,v-1 ), a i,v = ATTN(h x,y i-1 , s y i,v ), d i,v = ∑ k∈K a i,v,k h x,y i-1,k , p y i,v = softmax(W [s y i,v , d i,v ]).

2.1.2. INCORPORATING TABLE ATTENTION:



Table Pointer:

Step 1: attribute attn

Step 2: weighted column

Step 3: row attn

Step 4: weighted row

Step We now extend the attention model in order to allow the attention to be computed over a table, allowing the model to condition the generation on a database.

We denote a table with R rows and C columns as:

{f r,c }, r ∈ [1, R], c ∈ [1, C],

where f r,c is the cell in row r and column c. The attribute of each column is denoted as s c , where c is the c-th attribute. f r,c and s c are one-hot vector.

Table Encoding: To encode the table, we build an attribute vector g c for each column. For each cell f r,c of the table, we concatenate it with the corresponding attribute g c and then feed it through a one-layer MLP as follows: 3a . The attention over cells in the table is conditioned on a given vector q, similarly to the attention model for sequences ATTN(p, q) . However, rather than a sequence p, we now operate over a table f . Our attention model computes a attribute attention followed by row attention of the table. We first use the attention mechanism on the attributes to find out which attribute the user asks about. Suppose a user says cheap, then we should focus on the price attribute. After we get the attention probability p a = ATTN({g c }, q), over the attribute, we calculate the weighted representation for each row e r = ∑ c p a c e rc conditioned on p a . Then e r has the price information of each row. We further use attention mechanism on e r and get the probability p r = ATTN({e r }, q) over the rows. Then restaurants with cheap price will be picked. Then, using the probabilities p r , we compute the weighted average over the all rows e c = ∑ r p r r e r,c , which is used in the decoder. The detailed process is:

g c = W E s c and then e r,c = tanh(W [W E f r,c , g c ]).

EQUATION

EQUATION

)

EQUATION

)

This is embedded in the decoder by replacing the conditioned state q as the current decoder state s y i,0 and then at each step, conditioning the prediction of y i,v on {e c } by using attention mechanism at each step. The detailed diagram of table attention is shown in Figure 3a .

2.1.3. INCORPORATING TABLE POINTER NETWORKS:

We now describe the mechanism used to refer to specific database entries during decoding. At each timestep, the model needs to decide whether to generate the next token from an entry of the database or from the word softmax. This is performed as follows.

Pointer Switch: We use z i,v ∈ [0, 1] to denote the decision of whether to copy one cell from the table. We compute this probability as follows:

p(z i,v |s i,v ) = sigmoid(W [s i,v , d i,v ]

). Thus, if z i,v = 1, the next token y i,v will be generated from the database, whereas if z i,v = 0, then the following token is generated from a softmax. We shall now describe how we generate tokens from the database.

Table Pointer: If z i,v = 1,

the token is generated from the table. The detailed process of calculating the probability distribution over the table is shown in Figure 3b . This is similar to the attention mechanism, except that we perform a column attention to compute the probabilities of copying from each column after Equation. 5. More formally:

p c = ATTN({e c }, q), (6) p copy = p r ⊗ p c , (7) where p c is a probability distribution over columns, whereas p r is a probability distribution over rows. In order to compute a matrix with the probability of copying each cell, we simply compute the outer product p copy = p r ⊗ p c .

Objective: As we treat z i as a latent variable, we wish to maximize the marginal probability of the sequence y i over all possible values of z i . Thus, our objective function is defined as:

p(y i,v |s i,v ) = p vocab p(0|s i,v ) + p copy p(1|s i,v ) = p vocab (1 -p(1|s i,v )) + p copy p(1|s i,v ). (8)

The model can also be trained in a fully supervised fashion, if z i,v is observed. In such cases, we simply maximize the likelihood of p(z i,v |s i,v ), based on the observations, rather than using the marginal probability over z i,v .

2.2. RECIPE GENERATION:

ingredients recipe 1 cup plain soy milk

Blend soy milk and spinach leaves together in a blender until smooth. Add banana and pulse until thoroughly blended.

3/4 cup packed fresh spinach leaves 1 large banana, sliced Let the ingredients of a recipe be X = {x i } T i=1 and each ingredient contains L tokens

x i = {x ij } L j=1 . The corresponding recipe is y = {y v } K v=1 .

We first use a LSTM to encode each ingredient:

h i,j = LSTM E (W E x ij , h i,j-1 ) ∀i.

Then, we sum the resulting state of each ingredient to obtain the starting LSTM state of the decoder. Once again we use an attention based decoder:

s v = LSTM D (s v-1 , d v-1 , W E y v-1 ), p copy v = ATTN({{h i,j } T i=1 } L j=1 , s v ), d v = ∑ ij p v,i,j h i,j , p(z v |s v ) = sigmoid(W [s v , d v ]), p vocab v = softmax(W [s v , d v ]).

Similar to the previous task, the decision to copy from the ingredient list or generate a new word from the softmax is performed using a switch, denoted as p(z v |s v ). We can obtain a probability distribution of copying each of the words in the ingredients by computing p copy v = ATTN({{h i,j } T i=1 } L j=1 , s v ) in the attention mechanism. For training, we optimize the marginal likelihood function employed in the previous task.

2.3. COREFERENCE BASED LANGUAGE MODEL:

Finally, we build a language model that uses coreference links to point to previous words. Before generating a word, we first make the decision on whether it is an entity mention. If so, we decide which entity this mention belongs to, then we generate the word based on that entity. Denote the document as X = {x i } L i=1 , and the entities are E = {e i } N i=1 , each entity has M i mentions, e i = {m ij } Mi j=1 , such that {x mij } Mi j=1 refer to the same entity. We use a LSTM to model the document, the hidden state of each token is h i = LSTM(W E x i , h i-1 ). We use a set h e = {h e 0 , h e 1 , ..., h e M } to keep track of the entity states, where h e j is the state of entity j.

um and [I] 1 think that is whats -Go ahead [Linda] 2 . Well and thanks goes to Word generation: At each time step before generating the next word, we predict whether the word is an entity mention:

p coref (v i |h i-1 , h e ) = ATTN(h e , h i-1 ), d i = ∑ vi p(v i )h e vi p(z i |h i-1 ) = sigmoid(W [d i , h i-1 ]),

where z i denotes whether the next word is an entity and if yes v i denotes which entity the next word corefers to. If the next word is an entity mention, then p(

EQUATION

Entity state update: We update the entity state h e at each time step. In the beginning, h e = {h e 0 }, h e 0 denotes the state of an virtual empty entity and is a learnable variable. If z i = 1 and v i = 0, then it indicates the next word is a new entity mention, then in the next step, we append h i to h e , i.e., h e = {h e , h i }, if e i > 0, then we update the corresponding entity state with the new hidden state, h e [v i ] = h i . Another way to update the entity state is to use one LSTM to encode the mention states and get the new entity state. Here we use the latest entity mention state as the new entity state for simplicity. The detailed update process is shown in Figure 5 .

3. EXPERIMENTS 4 DATA SETS AND PREPROCESSING:

Dialogue: We use the DSTC2 data set. We only extracted the dialogue transcript from data set. There are about 3,200 dialogues in total. Since this is a small data set, we use 5-fold cross validation and report the average result over the 5 partitions. There may be multiple tokens in each table cell, for example in Table .2, the name, address, post code and phone number have multiple tokens, we replace them with one special token. For the name, address, post code and phone number of the j-th row, we replace the tokens in each cell with NAME j, ADDR j, POSTCODE j, PHONE j. If a table cell is empty, we replace it with an empty token EMPTY. We do a string match in the transcript and replace the corresponding tokens in transcripts from the table with the special tokens.

Each dialogue on average has 8 turns (16 sentences). We use a vocabulary size of 900, including about 400 table tokens and 500 words.

Recipes::

We crawl all recipes from www.allrecipes.com. There are about 31, 000 recipes in total, and every recipe has a ingredient list and a corresponding recipe. We exclude the recipes that have less than 10 tokens or more than 500 tokens, those recipes take about 0.1% of all data set. On average each recipe has 118 tokens and 9 ingredients. We random shuffle the whole data set and take 80% as training and 10% for validation and test. We use a vocabulary size of 10,000 in the model.

Coref LM::

We use the Xinhua News data set from Gigaword Fifth Edition and sample 100,000 documents from it that has length in range from 100 to 500. Each document has on average 234 tokens, so there are 23 million tokens in total. We use a tool to annotate all the entity mentions and use the annotation in the training. We take 80% as training and 10% as validation and test respectively. We ignore the entities that have only one mention and for the mentions that have multiple tokens, we take the token that is most frequent in the all the mentions for this entity. After the preprocessing, tokens that are entity mentions take about 10% of all tokens. We use a vocabulary size of 50,000 in the model.

4.1. MODEL TRAINING AND EVALUATION:

We train all models with simple stochastic gradient descent with clipping. We use a one-layer LSTM for all RNN components. Hyper-parameters are selected using grid search based on the validation set. We use dropout after the input embedding and LSTM output. The learning rate is selected from [0.1, 0.2, 0.5, 1], maximum gradient norm is selected from [1, 2, 5, 10] and drop ratio is selected from [0.2, 0.3, 0.5]. The batch size and LSTM dimension size is slightly different for different tasks so as to make the model fit into memory. The number of epochs to train are different for each task and we drop the learning rate after reaching a given number of epochs. We report the per-word perplexity for all tasks, specifically, we report the perplexity of all words, words that can be generated from reference and non-reference words. For recipe generation, we also generate the recipe using beam size of 10 and evaluate the generated recipe with BLEU. 

4.2. RESULTS AND ANALYSIS:

The results for dialogue, recipe generation and coref language model are shown in Table 4 , 5 and 6 respectively. We can see from Table 4 that models that condition on table performs better in predicting table tokens in general. Table pointer has the lowest perplexity for token in the table. Since the table token appears rarely in the dialogue, the overall perplexity does not differ much and the non-table tokens perplexity are similar. With attention mechanism over the table, the perplexity of table token improves over basic seq2seq model, but not as good as directly pointing to cells in the table. As expected, using sentence attention improves significantly over models without sentence attention. Surprisingly, table latent performs much worse than table pointer. We also measure the perplexity of table tokens that appear only in test set. For models other than table pointer, because the tokens never appear in training set, the perplexity is quite high, while table pointer can predict these tokens much more accurately. The recipe results in Table 5 in general follows that findings from the dialogue. But the latent model performs better than pointer model since that tokens in ingredients that match with recipe does not necessarily come from the ingredients. Imposing a supervised signal will give wrong information to the model and hence make the result worse. Hence with latent decision, the model learns to when to copy and when to generate it from the vocabulary. The coref LM results are shown in Table 6 . We find that coref based LM performs much better on the entities perplexities, but however is a little bit worse than for non-entity words. We found it is an optimization problem and perhaps the model is stuck in local optimum. So we initialize the pointer model with the weights learned from LM, the pointer model performs better than LM both for entity perplexity and non-entity words perplexity.

5. RELATED WORK:

Recently, there has been great progresses in modeling languages based on neural network, including language modeling (Mikolov et al., 2010; Jozefowicz et al., 2016) , machine translation (Sutskever et al., 2014; Bahdanau et al., 2014 ), question answering (Hermann et al., 2015) etc. Based on the success of seq2seq models, neural networks are applied in modeling chit-chat dialogue (Li et al., 2016; Vinyals & Le, 2015; Sordoni et al., 2015; Serban et al., 2016; Shang et al., 2015) and task oriented dialogue (Wen et al., 2015; Bordes & Weston, 2016; Williams & Zweig, 2016; Wen et al., 2016) . Most of the chit-chat neural dialogue models are simply applying the seq2seq models. For the task oriented dialogues, most of them embed the seq2seq model in traditional dialogue systems, in which the table query part is not differentiable. while our model queries the database directly. Recipe generation was proposed in (Kiddon et al., 2016) . Their model extents previous work on attention models (Allamanis et al., 2016) to checklists, whereas our work models explicit references to those checklists. Context dependent language models (Mikolov et al., 2010; Ji et al., 2015; Wang & Cho, 2015) are proposed to capture long term dependency of text. There are also lots of works on coreference resolution (Haghighi & Klein, 2010; Wiseman et al., 2016) . We are the first to combine coreference with language modeling, to the best of our knowledge. Much effort has been invested in embedding a copying mechanism for neural models (Gülc ¸ehre et al., 2016; Gu et al., 2016; Ling et al., 2016) . In general, a gating mechanism is employed to combine the softmax over observed words and a pointer network (Vinyals et al., 2015) . These gates can be trained either by marginalizing over both outcomes, or using heuristics (e.g. copy low frequency words). Our models are similar to models proposed in (Ahn et al., 2016; Merity et al., 2016) , where the generation of each word can be conditioned on a particular entry in knowledge lists and previous words. In our work, we describe a model with broader applications, allowing us to condition, on databases, lists and dynamic lists.

6. CONCLUSION:

We introduce reference-aware language models which explicitly model the decision of from where to generate the token at each step. Our model can also learns the decision by treating it as a latent variable. We demonstrate on three tasks, table based dialogue modeling, recipe generation and coref based LM, that our model performs better than attention based model, which does not incorporate this decision explicitly. There are several directions to explore further based on our framework. The current evaluation method is based on perplexity and BLEU. In task oriented dialogues, we can also try human evaluation to see if the model can reply users' query accurately. It is also interesting to use reinforcement learning to learn the actions in each step.

A VISUALIZATION M: Hello , welcome to the Cambridge restaurant system ? You can ask for restaurants by area , price range or food type . How may I help you ? U: and expensive restaurant that serves tuscan food M: Sorry there is no tuscan restaurant in the expensive price range U: french food M: cote is a great restaurant serving french food and it is in the expensive price range U: may i have the phone number M: The phone number of cote is 01223 311053 . U: thank you good bye (a) Dialogue script 

:



Figures and Tables:

Figure 1: Reference-aware language models.

Figure 2: Hierarchical RNN Seq2Seq model

Figure 3:Table based decoder.

Figure 4: Recipe pointer

Figure 5: Coreference based language model, example taken from Wiseman et al. (2016).

Figure 7: Recipe heat map example 2.

Example dialogue, M stands for Machine and U stands for User

Table content: <table><tr><td>NAME</td><td>PRICE RANGE</td><td>FOOD</td><td>AREA</td><td>ADDRESS</td><td>POST CODE</td><td>PHONE</td></tr><tr><td>ali baba</td><td>moderate</td><td colspan="3">lebanese centre 59 Hills Road City Cen-</td><td colspan="2">CB 2, 1 NT 01462 432565</td></tr><tr><td/><td/><td/><td/><td>tre</td><td/><td/></tr><tr><td>the</td><td>moderate</td><td>indian</td><td>north</td><td>7 Milton Road Chester-</td><td colspan="2">CB 4, 1 UY 01223 360966</td></tr><tr><td>nirala</td><td/><td/><td/><td>ton</td><td/><td/></tr></table>

Fragment of database for dialogue system.

Table content: <table/>

The diagram for table attention is shown in Figure

Table content: <table/>

Ingredients and recipe for Spinach and Banana Power Smoothie.Next, we consider the task of recipe generation conditioning on the ingredient lists. In this task, we must generate the recipe from a list of ingredients. Table.3 illustrates the ingredient list and recipe for Spinach and Banana Power Smoothie. We can see that the ingredients soy milk, spinach leaves, and banana occur in the recipe.

Table content: <table><tr><td/><td>soy</td></tr><tr><td>p copy p copy</td><td>p vocab p vocab</td></tr><tr><td>Yes</td><td>No</td></tr><tr><td>ingredients</td><td>z</td></tr><tr><td/><td>decoder</td></tr><tr><td>soy</td><td/></tr><tr><td colspan="2">Blend</td></tr><tr><td>encoder</td><td/></tr></table>

Dialogue perplexity results. (All means all tokens, table means tokens from table, table oov denotes table tokens that does not appear in the training set, word means non-table tokens). sentence attn denotes we use attention mechanism over tokens from past turn. Table pointer and table latent differs in that table pointer, we provide supervised signal on when to generate a table token, while in table latent it is a latent decision.

Table content: <table><tr><td>model</td><td/><td colspan="2">all</td><td>table</td><td/><td colspan="2">table oov</td><td>word</td></tr><tr><td>seq2seq</td><td/><td colspan="3">1.35±0.01 4.98±0.38</td><td colspan="4">1.99E7±7.75E6 1.23±0.01</td></tr><tr><td>table attn</td><td/><td colspan="3">1.37±0.01 5.09±0.64</td><td colspan="4">7.91E7±1.39E8 1.24±0.01</td></tr><tr><td colspan="2">table pointer</td><td colspan="3">1.33±0.01 3.99±0.36</td><td/><td colspan="3">1360 ± 2600 1.23±0.01</td></tr><tr><td>table latent</td><td/><td colspan="3">1.36±0.01 4.99±0.20</td><td colspan="4">3.78E7±6.08E7 1.24±0.01</td></tr><tr><td colspan="2">+ sentence attn</td><td/><td/><td/><td/><td/><td/></tr><tr><td>seq2seq</td><td/><td colspan="7">1.28±0.01 3.31±0.21 2.83E9 ± 4.69E9 1.19±0.01</td></tr><tr><td>table attn</td><td/><td colspan="3">1.28±0.01 3.17±0.21</td><td colspan="4">1.67E7±9.5E6 1.20±0.01</td></tr><tr><td colspan="2">table pointer</td><td colspan="3">1.27±0.01 2.99±0.19</td><td/><td colspan="3">82.86±110 1.20±0.01</td></tr><tr><td>table latent</td><td/><td colspan="3">1.28±0.01 3.26±0.25</td><td colspan="4">1.27E7±1.41E7 1.20±0.01</td></tr><tr><td/><td/><td>val</td><td/><td/><td/><td/><td>test</td></tr><tr><td>model</td><td>all</td><td colspan="2">ppl ing word</td><td>BLEU</td><td>all</td><td colspan="2">ppl ing word</td><td>BLEU</td></tr><tr><td colspan="3">seq2seq 5.60 11.26</td><td>5.00</td><td colspan="3">14.07 5.52 11.26</td><td>4.91</td><td>14.39</td></tr><tr><td>attn</td><td>5.25</td><td>6.86</td><td>5.03</td><td colspan="2">14.84 5.19</td><td>6.92</td><td>4.95</td><td>15.15</td></tr><tr><td>pointer</td><td>5.15</td><td>5.86</td><td>5.04</td><td colspan="2">15.06 5.11</td><td>6.04</td><td>4.98</td><td>15.29</td></tr><tr><td>latent</td><td>5.02</td><td>5.10</td><td>5.01</td><td colspan="2">14.87 4.97</td><td>5.19</td><td>4.94</td><td>15.41</td></tr></table>

Recipe result, evaluated in perplexity and BLEU score. ing denotes tokens from recipe that appear in ingredients.

Table content: <table><tr><td>model</td><td>val all entity</td><td>word</td><td>test all entity</td><td>word</td></tr><tr><td>lm</td><td colspan="4">33.08 44.52 32.04 33.08 43.86 32.10</td></tr><tr><td>pointer</td><td colspan="4">32.57 32.07 32.62 32.62 32.07 32.69</td></tr><tr><td colspan="5">pointer + init 30.43 28.56 30.63 30.42 28.56 30.66</td></tr></table>

Coreference based LM. pointer + init means we initialize the model with the LM weights.

Table content: <table/>

Attention heat map: cote is a great restaurant serving french food and it is in the expensive price range. Attention heap map: The phone number of cote is 01223 311053 .

Table content: <table><tr><td>NAME</td><td>PRICE RANGE</td><td>FOOD</td><td>AREA</td><td>ADDRESS</td><td>POST CODE</td><td>PHONE</td></tr><tr><td>charlie chan</td><td>cheap</td><td>chinese</td><td>east</td><td>Regent Street City Cen-</td><td colspan="2">C.B 2, 1 D.B 01223 361763</td></tr><tr><td/><td/><td/><td/><td>tre</td><td/><td/></tr><tr><td>chiquito restau-</td><td>expensive</td><td>mexican</td><td>south</td><td>2G Cambridge Leisure</td><td colspan="2">C.B 1, 7 D.Y 01223 400170</td></tr><tr><td>rant bar</td><td/><td/><td/><td>Park Cherry Hinton</td><td/><td/></tr><tr><td/><td/><td/><td/><td>Road Cherry Hinton</td><td/><td/></tr><tr><td>city stop</td><td>expensive</td><td>food</td><td>north</td><td>Cambridge City Foot-</td><td>EMPTY</td><td>01223 363270</td></tr><tr><td/><td/><td/><td/><td>ball Club Milton Road</td><td/><td/></tr><tr><td/><td/><td/><td/><td>Chesterton</td><td/><td/></tr><tr><td>clowns cafe</td><td>expensive</td><td>italian</td><td>centre</td><td>EMPTY</td><td colspan="2">C.B 1, 1 L.N 01223 355711</td></tr><tr><td>cocum</td><td>expensive</td><td>indian</td><td>west</td><td>71 Castle Street City</td><td colspan="2">C.B 3, 0 A.H 01223 366668</td></tr><tr><td/><td/><td/><td/><td>Centre</td><td/><td/></tr><tr><td>cote</td><td>expensive</td><td>french</td><td colspan="2">centre Bridge Street City Cen-</td><td colspan="2">C.B 2, 1 U.F 01223 311053</td></tr><tr><td/><td/><td/><td/><td>tre</td><td/><td/></tr><tr><td>curry garden</td><td>expensive</td><td>indian</td><td colspan="2">centre 106 Regent Street City</td><td>EMPTY</td><td>01223 302330</td></tr><tr><td/><td/><td/><td/><td>Centre</td><td/><td/></tr><tr><td>curry king</td><td>expensive</td><td>indian</td><td colspan="2">centre 5 Jordans Yard Bridge</td><td colspan="2">C.B 1, 2 B.D 01223 324351</td></tr><tr><td/><td/><td/><td/><td>Street City Centre</td><td/><td/></tr><tr><td>curry prince</td><td>moderate</td><td>indian</td><td>east</td><td>451 Newmarket Road</td><td>C.B 5, 8 J.J</td><td>01223 566388</td></tr><tr><td/><td/><td/><td/><td>Fen Ditton</td><td/><td/></tr><tr><td>(b) NAME</td><td>PRICE RANGE</td><td>FOOD</td><td>AREA</td><td>ADDRESS</td><td>POST CODE</td><td>PHONE</td></tr><tr><td>charlie chan</td><td>cheap</td><td>chinese</td><td>east</td><td>Regent Street City Cen-</td><td colspan="2">C.B 2, 1 D.B 01223 361763</td></tr><tr><td/><td/><td/><td/><td>tre</td><td/><td/></tr><tr><td>chiquito restau-</td><td>expensive</td><td>mexican</td><td>south</td><td>2G Cambridge Leisure</td><td colspan="2">C.B 1, 7 D.Y 01223 400170</td></tr><tr><td>rant bar</td><td/><td/><td/><td>Park Cherry Hinton</td><td/><td/></tr><tr><td/><td/><td/><td/><td>Road Cherry Hinton</td><td/><td/></tr><tr><td>city stop</td><td>expensive</td><td>food</td><td>north</td><td>Cambridge City Foot-</td><td>EMPTY</td><td>01223 363270</td></tr><tr><td/><td/><td/><td/><td>ball Club Milton Road</td><td/><td/></tr><tr><td/><td/><td/><td/><td>Chesterton</td><td/><td/></tr><tr><td>clowns cafe</td><td>expensive</td><td>italian</td><td>centre</td><td>EMPTY</td><td colspan="2">C.B 1, 1 L.N 01223 355711</td></tr><tr><td>cocum</td><td>expensive</td><td>indian</td><td>west</td><td>71 Castle Street City</td><td colspan="2">C.B 3, 0 A.H 01223 366668</td></tr><tr><td/><td/><td/><td/><td>Centre</td><td/><td/></tr><tr><td>cote</td><td>expensive</td><td>french</td><td colspan="2">centre Bridge Street City Cen-</td><td colspan="2">C.B 2, 1 U.F 01223 311053</td></tr><tr><td/><td/><td/><td/><td>tre</td><td/><td/></tr><tr><td>curry garden</td><td>expensive</td><td>indian</td><td colspan="2">centre 106 Regent Street City</td><td>EMPTY</td><td>01223 302330</td></tr><tr><td/><td/><td/><td/><td>Centre</td><td/><td/></tr><tr><td>curry king</td><td>expensive</td><td>indian</td><td colspan="2">centre 5 Jordans Yard Bridge</td><td colspan="2">C.B 1, 2 B.D 01223 324351</td></tr><tr><td/><td/><td/><td/><td>Street City Centre</td><td/><td/></tr><tr><td>curry prince</td><td>moderate</td><td>indian</td><td>east</td><td>451 Newmarket Road</td><td>C.B 5, 8 J.J</td><td>01223 566388</td></tr><tr><td/><td/><td/><td/><td>Fen Ditton</td><td/><td/></tr><tr><td/><td>(c)</td><td/><td/><td/><td/><td/></tr></table>

Dialogue visualization 1 M: Hello , welcome to the Cambridge restaurant system ? You can ask for restaurants by area , price range or food type . How may I help you ? U: cheap restaurant M: What kind of food would you like ? U: in the west part of town M: la margherita is a nice restaurant in the west of town in the cheap price range U: address M: Sure , la margherita is on 15 Magdalene Street City Centre U: thank you M: la margherita is a nice restaurant in the west of town in the cheap price range U: good bye Attention heat map: la margherita is a nice restaurant in the west of town in the cheap price range

Table content: <table><tr><td/><td/><td colspan="2">(a) Dialogue script</td><td/><td/><td/></tr><tr><td>NAME</td><td>PRICE RANGE</td><td>FOOD</td><td>AREA</td><td>ADDRESS</td><td>POST CODE</td><td>PHONE</td></tr><tr><td>india house</td><td>expensive</td><td>indian</td><td>west</td><td>31 Newnham Road</td><td>EMPTY</td><td>01223 461661</td></tr><tr><td/><td/><td/><td/><td>Newnham</td><td/><td/></tr><tr><td>j restaurant</td><td>cheap</td><td>oriental</td><td colspan="2">centre 86 Regent Street City</td><td colspan="2">C.B 2, 1 D.P 01223 307581</td></tr><tr><td/><td/><td/><td/><td>Centre</td><td/><td/></tr><tr><td>jinling noodle</td><td>moderate</td><td>chinese</td><td colspan="2">centre 11 Peas Hill City Cen-</td><td>C.B 2, 3 P.P</td><td>01223 566188</td></tr><tr><td>bar</td><td/><td/><td/><td>tre</td><td/><td/></tr><tr><td>kohinoor</td><td>cheap</td><td>indian</td><td colspan="2">centre 74 Mill Road City Cen-</td><td>EMPTY</td><td>01223 323639</td></tr><tr><td/><td/><td/><td/><td>tre</td><td/><td/></tr><tr><td>kymmoy</td><td>expensive</td><td>oriental</td><td colspan="2">centre 52 Mill Road City Cen-</td><td colspan="2">C.B 1, 2 A.S 01223 311911</td></tr><tr><td/><td/><td/><td/><td>tre</td><td/><td/></tr><tr><td>la margherita</td><td>cheap</td><td>italian</td><td>west</td><td>15 Magdalene Street</td><td colspan="2">C.B 3, 0 A.F 01223 315232</td></tr><tr><td/><td/><td/><td/><td>City Centre</td><td/><td/></tr><tr><td>la mimosa</td><td>expensive</td><td colspan="3">mediterranean centre Thompsons Lane Fen</td><td colspan="2">C.B 5, 8 A.Q 01223 362525</td></tr><tr><td/><td/><td/><td/><td>Ditton</td><td/><td/></tr><tr><td>la raza</td><td>cheap</td><td>spanish</td><td colspan="2">centre 4 -6 Rose Crescent</td><td colspan="2">C.B 2, 3 L.L 01223 464550</td></tr><tr><td>la tasca</td><td>moderate</td><td>spanish</td><td colspan="2">centre 14 -16 Bridge Street</td><td colspan="2">C.B 2, 1 U.F 01223 464630</td></tr><tr><td>lan hong house</td><td>moderate</td><td>chinese</td><td colspan="2">centre 12 Norfolk Street City</td><td>EMPTY</td><td>01223 350420</td></tr><tr><td/><td/><td/><td/><td>Centre</td><td/><td/></tr><tr><td>(b) NAME</td><td>PRICE RANGE</td><td>FOOD</td><td>AREA</td><td>ADDRESS</td><td>POST CODE</td><td>PHONE</td></tr><tr><td>india house</td><td>expensive</td><td>indian</td><td>west</td><td>31 Newnham Road</td><td>EMPTY</td><td>01223 461661</td></tr><tr><td/><td/><td/><td/><td>Newnham</td><td/><td/></tr><tr><td>j restaurant</td><td>cheap</td><td>oriental</td><td colspan="2">centre 86 Regent Street City</td><td colspan="2">C.B 2, 1 D.P 01223 307581</td></tr><tr><td/><td/><td/><td/><td>Centre</td><td/><td/></tr><tr><td>jinling noodle</td><td>moderate</td><td>chinese</td><td colspan="2">centre 11 Peas Hill City Cen-</td><td>C.B 2, 3 P.P</td><td>01223 566188</td></tr><tr><td>bar</td><td/><td/><td/><td>tre</td><td/><td/></tr><tr><td>kohinoor</td><td>cheap</td><td>indian</td><td colspan="2">centre 74 Mill Road City Cen-</td><td>EMPTY</td><td>01223 323639</td></tr><tr><td/><td/><td/><td/><td>tre</td><td/><td/></tr><tr><td>kymmoy</td><td>expensive</td><td>oriental</td><td colspan="2">centre 52 Mill Road City Cen-</td><td colspan="2">C.B 1, 2 A.S 01223 311911</td></tr><tr><td/><td/><td/><td/><td>tre</td><td/><td/></tr><tr><td>la margherita</td><td>cheap</td><td>italian</td><td>west</td><td>15 Magdalene Street</td><td colspan="2">C.B 3, 0 A.F 01223 315232</td></tr><tr><td/><td/><td/><td/><td>City Centre</td><td/><td/></tr><tr><td>la mimosa</td><td>expensive</td><td colspan="3">mediterranean centre Thompsons Lane Fen</td><td colspan="2">C.B 5, 8 A.Q 01223 362525</td></tr><tr><td/><td/><td/><td/><td>Ditton</td><td/><td/></tr><tr><td>la raza</td><td>cheap</td><td>spanish</td><td colspan="2">centre 4 -6 Rose Crescent</td><td colspan="2">C.B 2, 3 L.L 01223 464550</td></tr><tr><td>la tasca</td><td>moderate</td><td>spanish</td><td colspan="2">centre 14 -16 Bridge Street</td><td colspan="2">C.B 2, 1 U.F 01223 464630</td></tr><tr><td>lan hong house</td><td>moderate</td><td>chinese</td><td colspan="2">centre 12 Norfolk Street City</td><td>EMPTY</td><td>01223 350420</td></tr><tr><td/><td/><td/><td/><td>Centre</td><td/><td/></tr></table>

Dialogue visualization 2

Table content: <table/>

Peer review (iclr2017):
This paper presents a new type of language model that treats entity references as latent variables. The paper is structured as three specialized models for three applications: dialog generation with references to database entries, recipe generation with references to ingredients, and text generation with coreference mentions. Despite some opaqueness in details that I will discuss later, the paper does a great job making the main idea coming through, which I think is quite interesting and definitely worth pursuing further. But it seems the paper was rushed into the deadline, as there are a few major weaknesses. The first major weakness is that the claimed latent variables are hardly latent in the actual empirical evaluation. As clarified by the authors via pre-review QAs, all mentions were assumed to be given to all model variants, and so, it would seem like an over-claim to call these variables as latent when they are in fact treated as observed variables. Is it because the models with latent variables were too difficult to train right? A related problem is the use of perplexity as an evaluation measure when comparing reference-aware language models to vanilla language models. Essentially the authors are comparing two language models defined over different event space, which is not a fair comparison. Because mentions were assumed to be given for the reference-aware language models, and because of the fact that mention generators are designed similar to a pointer network, the probability scores over mentions will naturally be higher, compared to the regular language model that needs to consider a much bigger vocabulary set. The effect is analogous to comparing language models with aggressive UNK (and a small vocabulary set) to a language models with no UNK (and a much larger vocabulary set). To mitigate this problem, the authors need to perform one of the following additional evaluations: either assuming no mention boundaries and marginalizing over all possibilities (treating latent variables as truly latent), or showing other types of evaluation beyond perplexity, for example, BLEU, METEOR, human evaluation etc on the corresponding generation task. The other major weakness is writing in terms of technical accuracy and completeness. I found many details opaque and confusing even after QAs. I wonder if the main challenge that hinders the quality of writing has something to do with having three very specialized models in one paper, each having a lot of details to be worked out, which may have not been extremely important for the main story of the paper, but nonetheless not negligible in order to understand what is going on with the paper. Perhaps the authors can restructure the paper so that the most important details are clearly worked out in the main body of the paper, especially in terms of latent variable handling — how to make mention detection and conference resolution truly latent, and if and when entity update helps, which in the current version is not elaborated at all, as it is mentioned only very briefly for the third application (coreference resolution) without any empirical comparisons to motivate the update operation.