Original: OF W IKIPEDIA : K NOWLEDGE - POWERED C ONVERSATIONAL AGENTS
Revised: OF W IKIPEDIA : K NOWLEDGE - POWERED C ONVERSATIONAL AGENTS

Original: Anonymous authors

Original: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: In open-domain dialogue intelligent agents should exhibit the use of knowledge, however there are few convincing demonstrations of this to date.
Revised: In open-domain dialogue intelligent agents should exhibit the use of knowledge, however there are few convincing demonstrations of this to date.

Original: The most popular sequence to sequence models typically “generate and hope” generic utterances that can be memorized in the weights of the model when mapping from input utterance(s) to output, rather than employing recalled knowledge as context.
Revised: The most popular sequence to sequence models typically “generate and hope” generic utterances that can be memorized in the weights of the model when mapping from input utterance(s) to output, rather than employing recalled knowledge as context.

Original: Use of knowledge has so far proved difﬁcult, in part because of the lack of a supervised learning benchmark task which exhibits knowledgeable open dialogue with clear grounding.
Revised: Use of knowledge has so far proved difﬁcult, in part because of the lack of a supervised learning benchmark task which exhibits knowledgeable open dialogue with clear grounding.

Original: To that end we collect and release a large dataset with conversations directly grounded with knowledge retrieved from Wikipedia.
Revised: To that end we collect and release a large dataset with conversations directly grounded with knowledge retrieved from Wikipedia.

Original: We then design architectures capable of retrieving knowledge, reading and conditioning on it, and ﬁnally generating natural responses.
Revised: We then design architectures capable of retrieving knowledge, reading and conditioning on it, and ﬁnally generating natural responses.

Original: Our best performing dialogue models are able to conduct knowledgeable discussions on open-domain topics as evaluated by automatic metrics and human evaluations, while our new benchmark allows for measuring further improvements in this important research direction.
Revised: Our best performing dialogue models are able to conduct knowledgeable discussions on open-domain topics as evaluated by automatic metrics and human evaluations, while our new benchmark allows for measuring further improvements in this important research direction.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: Arguably, one of the key goals of AI, and the ultimate the goal of natural language research, is for humans to be able to talk to machines.
Revised: Arguably, one of the key goals of AI, and the ultimate the goal of natural language research, is for humans to be able to talk to machines.

Original: In order to get close to this goal, machines must master a number of skills: to be able to comprehend language, employ memory to retain and recall knowledge, to reason about these concepts together, and ﬁnally output a response that both fulﬁlls functional goals in the conversation while simultaneously being captivating to their human speaking partner.
Revised: In order to get close to this goal, machines must master a number of skills: to be able to comprehend language, employ memory to retain and recall knowledge, to reason about these concepts together, and ﬁnally output a response that both fulﬁlls functional goals in the conversation while simultaneously being captivating to their human speaking partner.

Original: The current state-of-the-art approaches, sequence to sequence models of various kinds (Sutskever et al., 2014; Vinyals & Le, 2015; Serban et al., 2016; Vaswani et al., 2017) attempt to address some of these skills, but generally suffer from an inability to bring memory and knowledge to bear; as indicated by their name, they involve encoding an input sequence, providing limited reasoning by transforming their hidden state given the input, and then decoding to an output.
Revised: The current state-of-the-art approaches, sequence to sequence models of various kinds (Sutskever et al., 2014; Vinyals & Le, 2015; Serban et al., 2016; Vaswani et al., 2017) attempt to address some of these skills, but generally suffer from an inability to bring memory and knowledge to bear; as indicated by their name, they involve encoding an input sequence, providing limited reasoning by transforming their hidden state given the input, and then decoding to an output.

Original: To converse intelligently on a given topic, a speaker clearly needs knowledge of that subject, and it is our contention here that more direct knowledge memory mechanisms need to be employed.
Revised: To converse intelligently on a given topic, a speaker clearly needs knowledge of that subject, and it is our contention here that more direct knowledge memory mechanisms need to be employed.

Original: In this work we consider setups where this can be naturally measured and built.
Revised: In this work we consider setups where this can be naturally measured and built.

Original: We consider the task of open-domain dialogue, where two speakers conduct open-ended chit-chat given an initial starting topic, and during the course of the conversation the topic can broaden or focus on related themes.
Revised: We consider the task of open-domain dialogue, where two speakers conduct open-ended chit-chat given an initial starting topic, and during the course of the conversation the topic can broaden or focus on related themes.

Original: During such conversations, an interlocutor can glean new information and personal points of view from their speaking partner, while providing similarly themselves.
Revised: During such conversations, an interlocutor can glean new information and personal points of view from their speaking partner, while providing similarly themselves.

Original: This is a challenging task as it requires several components not found in many standard models.
Revised: This is a challenging task as it requires several components not found in many standard models.

Original: We design a set of architectures speciﬁcally for this goal that combine elements of Memory Network architectures (Sukhbaatar et al., 2015) to retrieve knowledge and read and condition on it, and Transformer architectures (Vaswani et al., 2017) to provide state-of-the-art text representations and sequence models for generating outputs, which we term Transformer Memory Networks.
Revised: We design a set of architectures speciﬁcally for this goal that combine elements of Memory Network architectures (Sukhbaatar et al., 2015) to retrieve knowledge and read and condition on it, and Transformer architectures (Vaswani et al., 2017) to provide state-of-the-art text representations and sequence models for generating outputs, which we term Transformer Memory Networks.

Original: As, to our knowledge, no public domain dataset of requisite scale exists, we build a supervised dataset of human-human conversations using crowd-sourced workers, ﬁrst crowd-sourcingdiverse discussion topics and then conversations involving 201 , 999 utterances about them.
Revised: As, to our knowledge, no public domain dataset of requisite scale exists, we build a supervised dataset of human-human conversations using crowd-sourced workers, ﬁrst crowd-sourcingdiverse discussion topics and then conversations involving 201 , 999 utterances about them.

Original: Each topic is connected to Wikipedia, and one of the humans (the wizard ) is asked to link the knowledge they use to sentences from existing articles.
Revised: Each topic is connected to Wikipedia, and one of the humans (the wizard ) is asked to link the knowledge they use to sentences from existing articles.

Original: In this way, we have both a natural way to train a knowledgeable conversation agent, by employing a memory component that can recall and ground on this existing text, and a natural way to evaluate models that we build, by assessing their ability at locating and using such knowledge.
Revised: In this way, we have both a natural way to train a knowledgeable conversation agent, by employing a memory component that can recall and ground on this existing text, and a natural way to evaluate models that we build, by assessing their ability at locating and using such knowledge.

Original: Our Transformer Memory Network architectures, both in retrieval and generative versions, are tested in this setup using both automatic metrics and human evaluations.
Revised: Our Transformer Memory Network architectures, both in retrieval and generative versions, are tested in this setup using both automatic metrics and human evaluations.

Original: We show their ability to execute engaging knowledgeable conversations with humans, compared to a number of baselines such as standard Memory Networks or Transformers.
Revised: We show their ability to execute engaging knowledgeable conversations with humans, compared to a number of baselines such as standard Memory Networks or Transformers.

Original: Our new benchmark,which will be released publicly upon acceptance, aims to encourage and measure further improvements in this important research direction.
Revised: Our new benchmark, publicly in ParlAI ( http:// www.parl.ai/ ), aims to encourage and measure further improvements in this important research direction.

Original:  R ELATED W ORK
Revised:  R ELATED W ORK

Original: Many existing dialogue tasks do not study the use of knowledge explicitly.
Revised: Many existing dialogue tasks do not study the use of knowledge explicitly.

Original: For example, popular chit-chat datasets such as Open-Subtitles (Vinyals & Le, 2015), Persona-Chat (Zhang et al., 2018) and Twitter (Sordoni et al., 2015) have tested the ability of sequence-to-sequence models that attend over the recent dialogue history, but do not attempt to recall long-term knowledge beyond encoding it directly into the weights of the feed-forward network.
Revised: For example, popular chit-chat datasets such as Open-Subtitles (Vinyals & Le, 2015), Persona-Chat (Zhang et al., 2018) and Twitter (Sordoni et al., 2015) have tested the ability of sequence-to-sequence models that attend over the recent dialogue history, but do not attempt to recall long-term knowledge beyond encoding it directly into the weights of the feed-forward network.

Original: In the area of goal-directed dialogue, separate from open domain chit-chat, such as airline (El Asri et al., 2017) or restaurant booking (Henderson et al., 2014; Wen et al., 2016; Bordes et al., 2017), knowledge conditioning is typically employed by allowing access to a database through API calls or otherwise.
Revised: In the area of goal-directed dialogue, separate from open domain chit-chat, such as airline (El Asri et al., 2017) or restaurant booking (Henderson et al., 2014; Wen et al., 2016; Bordes et al., 2017), knowledge conditioning is typically employed by allowing access to a database through API calls or otherwise.

Original: In contrast, our work investigates unstructured knowledge across a large, diverse set of topics potentially spanning all of Wikipedia.
Revised: In contrast, our work investigates unstructured knowledge across a large, diverse set of topics potentially spanning all of Wikipedia.

Original: In question answering one does not produce a dialogue response based on a conversation history, but a factual answer based on a question.
Revised: In question answering one does not produce a dialogue response based on a conversation history, but a factual answer based on a question.

Original: In that case, it is clear that retrieving and conditioning knowledge is vital.
Revised: In that case, it is clear that retrieving and conditioning knowledge is vital.

Original: For example, in SQuAD neural models have been developed that attend to a given paragraph from Wikipedia to answer questions (Rajpurkar et al., 2016), or Open-SQuAD which extends this to answering without being given the paragraph, instead performing retrieval over the entirety of Wikipedia (Chen et al., 2017).
Revised: For example, in SQuAD neural models have been developed that attend to a given paragraph from Wikipedia to answer questions (Rajpurkar et al., 2016), or Open-SQuAD which extends this to answering without being given the paragraph, instead performing retrieval over the entirety of Wikipedia (Chen et al., 2017).

Original: Recently, the QuAC dataset investigates similar themes, but as a sequence of questions and answers in dialogue form instead (Choi et al., 2018).
Revised: Recently, the QuAC dataset investigates similar themes, but as a sequence of questions and answers in dialogue form instead (Choi et al., 2018).

Original: In this work we do not address question answering, but focus on natural human dialogues which contain a diverse set of utterances, not just questions and answers.
Revised: In this work we do not address question answering, but focus on natural human dialogues which contain a diverse set of utterances, not just questions and answers.

Original: The closest work to ours lies in the area of non-goal directed dialogue incorporating knowledge.
Revised: The closest work to ours lies in the area of non-goal directed dialogue incorporating knowledge.

Original: The work of Dodge et al.
Revised: The work of Dodge et al.

Original: (2016) employed Memory Networks to perform dialogue discussing movies in terms of recommendation and open-ended discussion from Reddit, conditioning on a structured knowledge base.
Revised: (2016) employed Memory Networks to perform dialogue discussing movies in terms of recommendation and open-ended discussion from Reddit, conditioning on a structured knowledge base.

Original: Zhou et al.
Revised: Zhou et al.

Original: (2018) also links Reddit to structured knowledge.
Revised: (2018) also links Reddit to structured knowledge.

Original: Both Parthasarathi & Pineau (2018) and Ghazvininejad et al.
Revised: Both Parthasarathi & Pineau (2018) and Ghazvininejad et al.

Original: (2018) use unstructured text instead, as we do: the former to discuss news articles using Wikipedia summaries as knowledge, and the latter to discuss local businesses in two-turn dialogues using Foursquare tips as knowledge.
Revised: (2018) use unstructured text instead, as we do: the former to discuss news articles using Wikipedia summaries as knowledge, and the latter to discuss local businesses in two-turn dialogues using Foursquare tips as knowledge.

Original: Ghazvininejad et al.
Revised: Ghazvininejad et al.

Original: (2018) uses an extended Encoder-Decoder where the decoder is provided with an encoding of the context along with the external knowledge encoding.
Revised: (2018) uses an extended Encoder-Decoder where the decoder is provided with an encoding of the context along with the external knowledge encoding.

Original: Neither involves dialogue authored with the given knowledge, so it is unclear when knowledge is useful or not.
Revised: Neither involves dialogue authored with the given knowledge, so it is unclear when knowledge is useful or not.

Original: In contrast, in our task, we know the
Revised: In contrast, in our task, we know the

Original: Wikipedia articles and sentences that ground crowdworkers dialogues.
Revised: Wikipedia articles and sentences that ground crowdworkers dialogues.

Original: Model-wise, Parthasarathi & Pineau (2018) uses a Bag-of-Words Memory Network type fact encoder and an RNN decoder.
Revised: Model-wise, Parthasarathi & Pineau (2018) uses a Bag-of-Words Memory Network type fact encoder and an RNN decoder.

Original: Our work compares Memory Networks (Sukhbaatar et al., 2015) and Transformers which have been shown to be on-par or superior to RNN encoder-decoders (Vaswani et al., 2017), and develops an architecture that combines these approaches.
Revised: Our work compares Memory Networks (Sukhbaatar et al., 2015) and Transformers which have been shown to be on-par or superior to RNN encoder-decoders (Vaswani et al., 2017), and develops an architecture that combines these approaches.

Original: Concurrently with our work Moghe et al.
Revised: Concurrently with our work Moghe et al.

Original: (2018) proposed a dataset based on the closed domain of movie chats.
Revised: (2018) proposed a dataset based on the closed domain of movie chats.

Original: Our paper shows models working on full multi-turn dialogue in an open-domain setting, which to our knowledge was not shown before.
Revised: Our paper shows models working on full multi-turn dialogue in an open-domain setting, which to our knowledge was not shown before.

Original:  W IZARD OF W IKIPEDIA
Revised:  W IZARD OF W IKIPEDIA

Original: We consider the following general open-domain dialogue setting: two participants engage in chitchat, with one of the participants selecting a beginning topic, and during the conversation the topic is allowed to naturally change.
Revised: We consider the following general open-domain dialogue setting: two participants engage in chitchat, with one of the participants selecting a beginning topic, and during the conversation the topic is allowed to naturally change.

Original: The two participants, however, are not quite symmetric: one will play the role of a knowledgeable expert (which we refer to as the wizard ) while the other is a curious learner (the apprentice ).
Revised: The two participants, however, are not quite symmetric: one will play the role of a knowledgeable expert (which we refer to as the wizard ) while the other is a curious learner (the apprentice ).

Original: Apprentice At each stage of the conversation the apprentice talks to the wizard freely, playing the role of a curious learner, eager to chat.
Revised: Apprentice At each stage of the conversation the apprentice talks to the wizard freely, playing the role of a curious learner, eager to chat.

Original: Their goal is to go into depth about a chosen topic that interests themselves or their partner, while keeping the conversation engaging and fun.
Revised: Their goal is to go into depth about a chosen topic that interests themselves or their partner, while keeping the conversation engaging and fun.

Original: Note that the instruction to delve deeply into a topic makes this different to more “shallow” chit-chat tasks.
Revised: Note that the instruction to delve deeply into a topic makes this different to more “shallow” chit-chat tasks.

Original: In this task the use of knowledge is emphasized more.
Revised: In this task the use of knowledge is emphasized more.

Original: Wizard The wizard is given the following instructions: “You have just met the other person, who seems quite curious, and you are eager to discuss a topic with them!”
Revised: Wizard The wizard is given the following instructions: “You have just met the other person, who seems quite curious, and you are eager to discuss a topic with them!”

Original: Their goal is to inform their conversation partner about a topic that one of them will choose.
Revised: Their goal is to inform their conversation partner about a topic that one of them will choose.

Original: Crucially, the wizard has access to an information retrieval system that shows them paragraphs from Wikipedia possibly relevant to the conversation, which are unobserved by the apprentice.
Revised: Crucially, the wizard has access to an information retrieval system that shows them paragraphs from Wikipedia possibly relevant to the conversation, which are unobserved by the apprentice.

Original: Before each conversation turn the wizard can read these paragraphs and then potentially base their next reply on that observed knowledge.
Revised: Before each conversation turn the wizard can read these paragraphs and then potentially base their next reply on that observed knowledge.

Original: Note, the wizard is particularly instructed not to simply parrot this knowledge, but to use it to craft a relevant reply, and to present any relevant knowledge in a fun and engaging way, if possible.
Revised: Note, the wizard is particularly instructed not to simply parrot this knowledge, but to use it to craft a relevant reply, and to present any relevant knowledge in a fun and engaging way, if possible.

Original: Conversation Flow
Revised: Conversation Flow

Original: The ﬂow of the conversation thus takes place as follows.
Revised: The ﬂow of the conversation thus takes place as follows.

Original: Either the wizard or apprentice is picked to choose the topic and speak ﬁrst.
Revised: Either the wizard or apprentice is picked to choose the topic and speak ﬁrst.

Original: The other player receives the topic information, and the conversation begins.
Revised: The other player receives the topic information, and the conversation begins.

Original: When the apprentice sends the wizard a message, the wizard is shown relevant knowledge (described below), and chooses a relevant sentence in order to construct a response, or else chooses the no sentence used option.
Revised: When the apprentice sends the wizard a message, the wizard is shown relevant knowledge (described below), and chooses a relevant sentence in order to construct a response, or else chooses the no sentence used option.

Original: The Wizard responds to the apprentice basing their response on their chosen sentence.
Revised: The Wizard responds to the apprentice basing their response on their chosen sentence.

Original: The conversation repeats until one of the conversation partners ends the chat (after a minimum of 4 or 5 turns each, randomly chosen beforehand).
Revised: The conversation repeats until one of the conversation partners ends the chat (after a minimum of 4 or 5 turns each, randomly chosen beforehand).

Original: After collecting data of such wizard-apprentice conversations between humans, the goal is to then replace the human wizard with a learned agent that will speak to a human apprentice instead, similar to the procedure in Wizard of Oz experiments (Bernsen et al., 2012).
Revised: After collecting data of such wizard-apprentice conversations between humans, the goal is to then replace the human wizard with a learned agent that will speak to a human apprentice instead, similar to the procedure in Wizard of Oz experiments (Bernsen et al., 2012).

Original: Topics We crowd-sourced a set of 1365 natural, open-domain dialogue topics, each linked to a
Revised: Topics We crowd-sourced a set of 1365 natural, open-domain dialogue topics, each linked to a

Original: Wikipedia article.
Revised: Wikipedia article.

Original: These include diverse topics such as commuting, Gouda cheese, music festivals, podcasts, bowling, and Arnold Schwarzenegger.
Revised: These include diverse topics such as commuting, Gouda cheese, music festivals, podcasts, bowling, and Arnold Schwarzenegger.

Original: Knowledge Retrieval
Revised: Knowledge Retrieval

Original: At each step of the dialogue the wizard has access to a set of passages of knowledge which may be relevant to the given dialogue context.
Revised: At each step of the dialogue the wizard has access to a set of passages of knowledge which may be relevant to the given dialogue context.

Original: While this is a potentially learnable part of the model, we required for this to be ﬁxed so that we could present the results to the annotator when collecting the dataset.
Revised: While this is a potentially learnable part of the model, we required for this to be ﬁxed so that we could present the results to the annotator when collecting the dataset.

Original: We thus used exactly the same retriever that is commonly used for the
Revised: We thus used exactly the same retriever that is commonly used for the

Original: Open-SQuAD dataset in Chen et al.
Revised: Open-SQuAD dataset in Chen et al.

Original: It uses a simple inverted index lookup followed by term vector model scoring.
Revised: It uses a simple inverted index lookup followed by term vector model scoring.

Original: Articles and queries are compared as TF-IDF weighted bag-of-word and n -gram vectors, using the hashing trick.
Revised: Articles and queries are compared as TF-IDF weighted bag-of-word and n -gram vectors, using the hashing trick.

Original: We retrieve the top 7 articles (ﬁrst paragraph only) for the last two turns of dialogue (by wizard and apprentice) and the article (ﬁrst 10 sentences only) for the original topic, and present these articles to the wizard as knowledge context, along with their titles.
Revised: We retrieve the top 7 articles (ﬁrst paragraph only) for the last two turns of dialogue (by wizard and apprentice) and the article (ﬁrst 10 sentences only) for the original topic, and present these articles to the wizard as knowledge context, along with their titles.

Original: Note that while this system is used to build the dataset, a superior method can in principle be learned and used by a model at test time.
Revised: Note that while this system is used to build the dataset, a superior method can in principle be learned and used by a model at test time.

Original: Knowledge Selection and Response Generation
Revised: Knowledge Selection and Response Generation

Original: During data collection, the wizard can click on any of the retrieved article titles in the dialogue UI to expand that article, at which point they can click on a sentence that is most relevant to the response they want to make (only one article, and one sentence can be selected on any turn, for simplicity).
Revised: During data collection, the wizard can click on any of the retrieved article titles in the dialogue UI to expand that article, at which point they can click on a sentence that is most relevant to the response they want to make (only one article, and one sentence can be selected on any turn, for simplicity).

Original: If they see no relevant article or sentence they can choose no sentence used instead.
Revised: If they see no relevant article or sentence they can choose no sentence used instead.

Original: The wizard then enters their response to the apprentice.
Revised: The wizard then enters their response to the apprentice.

Original: An image of the Wizard’s UI is shown in Appendix A.1.
Revised: An image of the Wizard’s UI is shown in Appendix A.1.

Original: [Table]
Revised: [Table]

Original: Final Dialogue Dataset The ﬁnal dialogue dataset we collect consists of 22,311 dialogues with 201,999 turns, which we divide into 166,787 for train, 17,715 for validation, and 17,497 for test.
Revised: Final Dialogue Dataset The ﬁnal dialogue dataset we collect consists of 22,311 dialogues with 201,999 turns, which we divide into 166,787 for train, 17,715 for validation, and 17,497 for test.

Original: The test set is split into two subsets, Test Seen and Test Unseen.
Revised: The test set is split into two subsets, Test Seen and Test Unseen.

Original: Test Seen contains 533 overlapping topics with the training set with new dialogues about those topics.
Revised: Test Seen contains 533 overlapping topics with the training set with new dialogues about those topics.

Original: Test Unseen consists oftopics never seen before in train or validation.
Revised: Test Unseen consists oftopics never seen before in train or validation.

Original: Overall data statistics can be found in Table 1, and further statistics and examples of collected conversations in Appendix A.2.
Revised: Overall data statistics can be found in Table 1, and further statistics and examples of collected conversations in Appendix A.2.

Original: We observe wizards and apprentices both asking and answering questions, and providing each other with a mixture of facts and personal feelings during their general discussion.
Revised: We observe wizards and apprentices both asking and answering questions, and providing each other with a mixture of facts and personal feelings during their general discussion.

Original:  M ODELS
Revised:  M ODELS

Original: In this work we consider learning dialogue models to replace the wizard in our learning tasks, i.e. the knowledgeable speaker.
Revised: In this work we consider learning dialogue models to replace the wizard in our learning tasks, i.e. the knowledgeable speaker.

Original: The dialogue model thus can have access to a knowledge source, in this case
Revised: The dialogue model thus can have access to a knowledge source, in this case

Original: Wikipedia, to ground the conversation with.
Revised: Wikipedia, to ground the conversation with.

Original: We thus develop extensions of the Memory Network (Sukhbaatar et al., 2015) and Transformer (Vaswani et al., 2017) models that can (i) retrieve from a large memory relevant information conditioned on the dialogue history, (ii) carefully read and attend over the retrieved set of knowledge, and then (iii) generate the next dialogue utterance.
Revised: We thus develop extensions of the Memory Network (Sukhbaatar et al., 2015) and Transformer (Vaswani et al., 2017) models that can (i) retrieve from a large memory relevant information conditioned on the dialogue history, (ii) carefully read and attend over the retrieved set of knowledge, and then (iii) generate the next dialogue utterance.

Original: This model is then used consecutively on each turn to form an entire dialogue with a user.
Revised: This model is then used consecutively on each turn to form an entire dialogue with a user.

Original: We develop two classes of models capable of leveraging knowledge: (i) retrieval models that produce an output among a set of candidate responses (the set of utterances from the training set); and (ii) generative models that generate word-by-word (using a beam).
Revised: We develop two classes of models capable of leveraging knowledge: (i) retrieval models that produce an output among a set of candidate responses (the set of utterances from the training set); and (ii) generative models that generate word-by-word (using a beam).

Original: The input to either model is the same: at each dialogue turn where the model is intended to make a response, it is given the current dialogue context x 1 , . . .
Revised: The input to either model is the same: at each dialogue turn where the model is intended to make a response, it is given the current dialogue context x 1 , . . .

Original: , x t of t dialogue turns, where x 1 is always the initial starting topic (e.g. “Kurt Cobain”), and the remaining turns swap between the two speakers.
Revised: , x t of t dialogue turns, where x 1 is always the initial starting topic (e.g. “Kurt Cobain”), and the remaining turns swap between the two speakers.

Original: The goal at each stage is to output the next utterance x t +1 .
Revised: The goal at each stage is to output the next utterance x t +1 .

Original: Knowledge Retrieval We assume a large knowledge base (memory) m 1 , . . .
Revised: Knowledge Retrieval We assume a large knowledge base (memory) m 1 , . . .

Original: , m N which is hierarchically organized into documents consisting of paragraphs and sentences.
Revised: , m N which is hierarchically organized into documents consisting of paragraphs and sentences.

Original: As it is infeasible for current neural attention techniques to operate on this scale, we use standard information retrieval (IR) techniques ( c =
Revised: As it is infeasible for current neural attention techniques to operate on this scale, we use standard information retrieval (IR) techniques ( c =

Original: IR ( x, m ) ) as a ﬁrst step to return a smaller set of candidates m c 1 , . . .
Revised: IR ( x, m ) ) as a ﬁrst step to return a smaller set of candidates m c 1 , . . .

Original: , m c K for ﬁne-grained selection.
Revised: , m c K for ﬁne-grained selection.

Original: In our experiments, we use the IR system provided to the human annotators during dataset creation, detailed in Section 3.
Revised: In our experiments, we use the IR system provided to the human annotators during dataset creation, detailed in Section 3.

Original: The retriever operates on the topic ( x 1 ) and the last two turns ( x t and x t − 1 ) if they exist, effectively calling the IR system three times with three different queries.
Revised: The retriever operates on the topic ( x 1 ) and the last two turns ( x t and x t − 1 ) if they exist, effectively calling the IR system three times with three different queries.

Original: Empirically, this provided better performance compared to merging into one query, likely because it can address quite different topics.
Revised: Empirically, this provided better performance compared to merging into one query, likely because it can address quite different topics.

Original: We retrieve the top 7 articles (ﬁrst paragraph only) for each lookup and then ﬂatten all the results into separate sentences (i.e. remove the organization of sentences belonging to articles), but prepend every sentence with its article title.
Revised: We retrieve the top 7 articles (ﬁrst paragraph only) for each lookup and then ﬂatten all the results into separate sentences (i.e. remove the organization of sentences belonging to articles), but prepend every sentence with its article title.

Original: In this way the candidates m c 1 , . . .
Revised: In this way the candidates m c 1 , . . .

Original: , m c K given to the neural model in the next stage can be attended to independently without having to deal with hierarchical issues.
Revised: , m c K given to the neural model in the next stage can be attended to independently without having to deal with hierarchical issues.

Original: Knowledge Attention We use an attention mechanism to perform ﬁne-grained selection of which knowledge sentences will be used to produce the next turn of dialogue.
Revised: Knowledge Attention We use an attention mechanism to perform ﬁne-grained selection of which knowledge sentences will be used to produce the next turn of dialogue.

Original: Each sentence in the memory is independently encoded with a Transformer encoder (Vaswani et al., 2017), and the same Trans-
Revised: Each sentence in the memory is independently encoded with a Transformer encoder (Vaswani et al., 2017), and the same Trans-

Original: [Figure]
Revised: [Figure]

Original: former is used to encode the dialogue context x .
Revised: former is used to encode the dialogue context x .

Original: We  perform dot the
Revised: We then perform standard dot-product attention the candidates and the

Original: Utterance Prediction Given the hidden state derived from the memory attention process described above, the ﬁnal stage is to predict the output utterance that will form the next dialogue turn.
Revised: Utterance Prediction Given the hidden state derived from the memory attention process described above, the ﬁnal stage is to predict the output utterance that will form the next dialogue turn.

Original: We consider different variants of the two stages above, knowledge attention and utterance prediction, when considering retrieval and generative variants of our models.
Revised: We consider different variants of the two stages above, knowledge attention and utterance prediction, when considering retrieval and generative variants of our models.

Original: We will now detail these in turn.
Revised: We will now detail these in turn.

Original: 4. R ETRIEVAL T RANSFORMER M EMORY N ETWORK
Revised: 4. R ETRIEVAL T RANSFORMER M EMORY N ETWORK

Original: This model encodes each knowledge sentence m c 1 , . . .
Revised: This model encodes each knowledge sentence m c 1 , . . .

Original: , m c K and the dialogue context x with a
Revised: , m c K and the dialogue context x with a

Original: Transformer, as described above.
Revised: Transformer, as described above.

Original: The ﬁnal input encoding is calculated by performing dot-product attention over enc ( m c 1 ) , . .
Revised: The ﬁnal input encoding is calculated by performing dot-product attention over enc ( m c 1 ) , . .

Original: , enc ( m c K ) and adding the resulting weighted sum of these vectors to enc ( x ) to get the representation rep LHS ( m c 1 , . . .
Revised: , enc ( m c K ) and adding the resulting weighted sum of these vectors to enc ( x ) to get the representation rep LHS ( m c 1 , . . .

Original: , m c K , x ) .
Revised: , m c K , x ) .

Original: The candidate responses r 1 , . . .
Revised: The candidate responses r 1 , . . .

Original: , r L are encoded with a separate Transformer to get rep RHS ( r i ) for each i .
Revised: , r L are encoded with a separate Transformer to get rep RHS ( r i ) for each i .

Original: We choose as a response r (cid:96) where
Revised: We choose as a response r (cid:96) where

Original: [Equation]
Revised: [Equation]

Original: The model is trained to minimize the cross-entropy loss, where the negative candidates for each example are the responses to the other examples in the batch (Henderson et al., 2017).
Revised: The model is trained to minimize the cross-entropy loss, where the negative candidates for each example are the responses to the other examples in the batch (Henderson et al., 2017).

Original: 4. G ENERATIVE T RANSFORMER M EMORY N ETWORK
Revised: 4. G ENERATIVE T RANSFORMER M EMORY N ETWORK

Original: We consider two versions: a Two-stage and an End-to-end version.
Revised: We consider two versions: a Two-stage and an End-to-end version.

Original: Both models ﬁnd the most relevant piece of knowledge m best , and then perform an encoding step by concatenating it with the dialogue context, allowing the decoder to attend over both the knowledge and dialogue when formulating its response.
Revised: Both models ﬁnd the most relevant piece of knowledge m best , and then perform an encoding step by concatenating it with the dialogue context, allowing the decoder to attend over both the knowledge and dialogue when formulating its response.

Original: We employ a beam search of 5 to select our best response.
Revised: We employ a beam search of 5 to select our best response.

Original: All generative models employ BPE encoding (Sennrich et al., 2016), which we found effective at enabling generators to copy rare words from Wikipedia sentences (Fan et al., 2018).
Revised: All generative models employ BPE encoding (Sennrich et al., 2016), which we found effective at enabling generators to copy rare words from Wikipedia sentences (Fan et al., 2018).

Original: In the End-to-end version, a shared Transformer encoder is used to encode all candidates m c
Revised: In the End-to-end version, a shared Transformer encoder is used to encode all candidates m c

Original: i and the dialogue history.
Revised: i and the dialogue history.

Original: The encoded candidates are ﬂattened into vectors using the normalization from
Revised: The encoded candidates are ﬂattened into vectors using the normalization from

Original: Cer et al.
Revised: Cer et al.

Original: (2018) (summing, and normalizing by the square root of the sentence length in order to balance short and long sentences) to produce an attention prediction over the memory.
Revised: (2018) (summing, and normalizing by the square root of the sentence length in order to balance short and long sentences) to produce an attention prediction over the memory.

Original: The full sequence encoding of the single highest selected knowledge m best is concatenated with the encoding of the dialogue, and passed into a Transformer decoder.
Revised: The full sequence encoding of the single highest selected knowledge m best is concatenated with the encoding of the dialogue, and passed into a Transformer decoder.

Original: An illustration of our End-to-end model is shown in Figure 1.
Revised: An illustration of our End-to-end model is shown in Figure 1.

Original: We train the model to minimize the negative log-likelihood of the response utterance.
Revised: We train the model to minimize the negative log-likelihood of the response utterance.

Original: We can add additional supervision by forcing the knowledge selection to correctly choose the same knowledge candidate as the human wizard in the training set by adding an additional crossentropy loss over the knowledge attention, modulated by a weight λ :
Revised: We can add additional supervision by forcing the knowledge selection to correctly choose the same knowledge candidate as the human wizard in the training set by adding an additional crossentropy loss over the knowledge attention, modulated by a weight λ :

Original: [Equation]
Revised: [Equation]

Original: In the Two-stage version, we employ two separately trained models for each of these two tasks, knowledge selection and utterance prediction.
Revised: In the Two-stage version, we employ two separately trained models for each of these two tasks, knowledge selection and utterance prediction.

Original: As the knowledge selection step creates a hard deci-
Revised: As the knowledge selection step creates a hard deci-

Original: [Table]
Revised: [Table]

Original: sion inﬂuencing the output of the generator, we ﬁnd maximizing the performance of this component to be vital.
Revised: sion inﬂuencing the output of the generator, we ﬁnd maximizing the performance of this component to be vital.

Original: We can also improve performance of the decoder by employing knowledge dropout (K.D.), wherein we artiﬁcially prevent the model from attending to knowledge a fraction of the time during training.
Revised: We can also improve performance of the decoder by employing knowledge dropout (K.D.), wherein we artiﬁcially prevent the model from attending to knowledge a fraction of the time during training.

Original: We ﬁnd this helps the generator be more resilient to errors at the knowledge selection stage, and makes training faster.
Revised: We ﬁnd this helps the generator be more resilient to errors at the knowledge selection stage, and makes training faster.

Original: K. D. is a novel technique we propose here, however it is similar to many other dropout techniques, e.g. feature dropout used in Wu et al.
Revised: K. D. is a novel technique we propose here, however it is similar to many other dropout techniques, e.g. feature dropout used in Wu et al.

Original:  E XPERIMENTS
Revised:  E XPERIMENTS

Original: We describe each of our experimental setups and results.
Revised: We describe each of our experimental setups and results.

Original: We ﬁrst investigate the ability of our models to select knowledge appropriately, and then consider the full task of dialogue with knowledge.
Revised: We ﬁrst investigate the ability of our models to select knowledge appropriately, and then consider the full task of dialogue with knowledge.

Original: 5. K NOWLEDGE S ELECTION T ASK
Revised: 5. K NOWLEDGE S ELECTION T ASK

Original: Before looking at the full Wizard dialogue task, we assess the ability of models to predict the knowledge selected by human wizards in the dataset given the dialogue history.
Revised: Before looking at the full Wizard dialogue task, we assess the ability of models to predict the knowledge selected by human wizards in the dataset given the dialogue history.

Original: This will inform us of the feasibility of this task and the best models to use in a two-stage architecture.
Revised: This will inform us of the feasibility of this task and the best models to use in a two-stage architecture.

Original: We compare Transformers against various baselines including a random baseline; an Information Retrieval (IR) baseline, which uses simple word overlap; and a Bag-of-Words Memory Network (Sukhbaatar et al., 2015).
Revised: We compare Transformers against various baselines including a random baseline; an Information Retrieval (IR) baseline, which uses simple word overlap; and a Bag-of-Words Memory Network (Sukhbaatar et al., 2015).

Original: Where noted, the Transformer is pretrained on Reddit data (Mazar ´ e et al., 2018), and ﬁne-tuned for our task.
Revised: Where noted, the Transformer is pretrained on Reddit data (Mazar ´ e et al., 2018), and ﬁne-tuned for our task.

Original: The results are shown in Table 2.
Revised: The results are shown in Table 2.

Original: Transformers work best, as long as they are pretrained on a large dataset (Reddit), while multi-tasking on SQuAD provides marginal impact.
Revised: Transformers work best, as long as they are pretrained on a large dataset (Reddit), while multi-tasking on SQuAD provides marginal impact.

Original: Further analysis of this task using other models is provided in Appendix B.1.
Revised: Further analysis of this task using other models is provided in Appendix B.1.

Original: We use the best performing Transformer model reported here for our two-stage generative Memory Network in the full dialogue task.
Revised: We use the best performing Transformer model reported here for our two-stage generative Memory Network in the full dialogue task.

Original: 5. F ULL T ASK : D IALOGUE WITH K NOWLEDGE
Revised: 5. F ULL T ASK : D IALOGUE WITH K NOWLEDGE

Original: We evaluate our models on the full task of dialogue generation given knowledge in two settings: given the gold knowledge sentence chosen by a human, or where the model needs to predict which knowledge to use.
Revised: We evaluate our models on the full task of dialogue generation given knowledge in two settings: given the gold knowledge sentence chosen by a human, or where the model needs to predict which knowledge to use.

Original: We separately describe experiments for retrieval and generative models.
Revised: We separately describe experiments for retrieval and generative models.

Original: Retrieval Experiments We use similar baselines as in the knowledge selection experiments, but now also apply Transformer Memory Networks, which attend over knowledge.
Revised: Retrieval Experiments We use similar baselines as in the knowledge selection experiments, but now also apply Transformer Memory Networks, which attend over knowledge.

Original: Models are evaluated measuring Recall@1 when ranking the gold response among 99 randomly chosen candidates, and unigram F1 of the model’s prediction with the gold response.
Revised: Models are evaluated measuring Recall@1 when ranking the gold response among 99 randomly chosen candidates, and unigram F1 of the model’s prediction with the gold response.

Original: The results are shown in Table 3.
Revised: The results are shown in Table 3.

Original: We ﬁnd that the addition of knowledge improves all models (improving Bow MemNet from 56 to
Revised: We ﬁnd that the addition of knowledge improves all models (improving Bow MemNet from 56 to

Original: R@1 and the Transformer MemNet from 79 to 87 R@1) for predicted knowledge.
Revised: R@1 and the Transformer MemNet from 79 to 87 R@1) for predicted knowledge.

Original: Performance improves dramatically when models are provided gold knowledge, but otherwise retain similar trends.
Revised: Performance improves dramatically when models are provided gold knowledge, but otherwise retain similar trends.

Original: Generative Experiments
Revised: Generative Experiments

Original: We compare our generative End-to-end and Two-stage Transformer
Revised: We compare our generative End-to-end and Two-stage Transformer

Original: Memory Network models to two more baselines: repeating the last utterance, and a generative
Revised: Memory Network models to two more baselines: repeating the last utterance, and a generative

Original: Transformer model trained to respond to dialogue but without access to knowledge.
Revised: Transformer model trained to respond to dialogue but without access to knowledge.

Original: Models are evaluated using perplexity (PPL) of the gold response and unigram F1.
Revised: Models are evaluated using perplexity (PPL) of the gold response and unigram F1.

Original: [Table] [Table]
Revised: [Table] [Table]

Original: [Table]
Revised: [Table]

Original: The results are given in Table 4.
Revised: The results are given in Table 4.

Original: Our experiments show that both the End-to-end and Two-stage models employ the knowledge in their response predictions, as they outperform their counterpart
Revised: Our experiments show that both the End-to-end and Two-stage models employ the knowledge in their response predictions, as they outperform their counterpart

Original: Transformer without knowledge
Revised: Transformer without knowledge

Original: [Table]
Revised: [Table]

Original: and demonstrate substantial improvements when provided the gold knowledge.
Revised: and demonstrate substantial improvements when provided the gold knowledge.

Original: While the Two-stage model produces signiﬁcantly stronger perplexity and F1 scores using the predicted knowledge, the End-to-end model outperforms the Two-stage model in the gold knowledge experiments.
Revised: While the Two-stage model produces signiﬁcantly stronger perplexity and F1 scores using the predicted knowledge, the End-to-end model outperforms the Two-stage model in the gold knowledge experiments.

Original: This suggests that the Two-stage model beneﬁts from the strong knowledge selection module (Section 5.1), but that the End-to-end model is better at employing the selected knowledge.
Revised: This suggests that the Two-stage model beneﬁts from the strong knowledge selection module (Section 5.1), but that the End-to-end model is better at employing the selected knowledge.

Original: Furthermore, we ﬁnd that the additional knowledge selection supervision (auxiliary loss) in the End-to-end model improves it on every metric, suggesting that tightly integrating these tasks is beneﬁcial.
Revised: Furthermore, we ﬁnd that the additional knowledge selection supervision (auxiliary loss) in the End-to-end model improves it on every metric, suggesting that tightly integrating these tasks is beneﬁcial.

Original: Knowledge dropout (K. D.) also helps (compare last two rows).
Revised: Knowledge dropout (K. D.) also helps (compare last two rows).

Original: More evidence
Revised: More evidence

Original: [Table]
Revised: [Table]

Original: for this is shown in Appendix B.1.
Revised: for this is shown in Appendix B.1.

Original: Lastly, we note that both Two-stage models give higher F1 scores than any of the retrieval models shown in Table 3.
Revised: Lastly, we note that both Two-stage models give higher F1 scores than any of the retrieval models shown in Table 3.

Original: 5. H UMAN E VALUATION
Revised: 5. H UMAN E VALUATION

Original: We perform human evaluation of our models using crowd-sourced workers.
Revised: We perform human evaluation of our models using crowd-sourced workers.

Original: Humans are paired with our models and asked to chat about a speciﬁc topic (given a choice of 2–3 topics) for 3–5 dialogue turns.
Revised: Humans are paired with our models and asked to chat about a speciﬁc topic (given a choice of 2–3 topics) for 3–5 dialogue turns.

Original: Following their conversation, the humans are asked to rate their dialogue partner on a scale of 1–5, with the rating indicating how much they “liked” the conversation (5 is best), which we refer to as the engagingness rating.
Revised: Following their conversation, the humans are asked to rate their dialogue partner on a scale of 1–5, with the rating indicating how much they “liked” the conversation (5 is best), which we refer to as the engagingness rating.

Original: Using the collected conversations, we also calculate a metric we call the Wiki F1 sore: the F1 overlap of the model’s utterances with the ﬁrst 10 sentences of the Wikipedia page for the chosen topic as a proxy for how much knowledge the model exhibits.
Revised: Using the collected conversations, we also calculate a metric we call the Wiki F1 sore: the F1 overlap of the model’s utterances with the ﬁrst 10 sentences of the Wikipedia page for the chosen topic as a proxy for how much knowledge the model exhibits.

Original: We seek a model that can be simultaneously engaging and knowledgeable, hence we would like to maximize both these metrics 1 .
Revised: We seek a model that can be simultaneously engaging and knowledgeable, hence we would like to maximize both these metrics 1 .

Original: For comparison, we also collect 100 human-human conversations, with only one human choosing the topic and performing evaluation.
Revised: For comparison, we also collect 100 human-human conversations, with only one human choosing the topic and performing evaluation.

Original: In total, we collect a total of 546 conversations with ratings from 464 distinct workers.
Revised: In total, we collect a total of 546 conversations with ratings from 464 distinct workers.

Original: These results are shown in Table 5.
Revised: These results are shown in Table 5.

Original: We ﬁnd that the retrieval models signiﬁcantly outperform the generative models on the human engagingness
Revised: We ﬁnd that the retrieval models signiﬁcantly outperform the generative models on the human engagingness

Original: evaluation(Student’s t-test, p < . 05 ).
Revised: evaluation(Student’s t-test, p < . 05 ).

Original: The human engagingness differences between retriever models with and without knowledge are not signiﬁcant, but note they both trend toward use of knowledge due to the candidate sentences retrieved, with the knowledgeable version obtaining signiﬁcantly higher Wiki F1 scores in both seen and unseen test sets.
Revised: The human engagingness differences between retriever models with and without knowledge are not signiﬁcant, but note they both trend toward use of knowledge due to the candidate sentences retrieved, with the knowledgeable version obtaining signiﬁcantly higher Wiki F1 scores in both seen and unseen test sets.

Original: For the generative models, we ﬁnd human engagingness ratings are signiﬁcantly improved by the use of knowledge ( p < . 01 ).
Revised: For the generative models, we ﬁnd human engagingness ratings are signiﬁcantly improved by the use of knowledge ( p < . 01 ).

Original: The signiﬁcantly higher Wiki F1 scores indicate that (i) these models convey more knowledge than their counterparts without knowledge conditioning; and (ii) on both seen and unseen sets they convey more knowledge than the retrieval models.
Revised: The signiﬁcantly higher Wiki F1 scores indicate that (i) these models convey more knowledge than their counterparts without knowledge conditioning; and (ii) on both seen and unseen sets they convey more knowledge than the retrieval models.

Original: In particular, on unseen data the gap between retrieval and generative models is larger.
Revised: In particular, on unseen data the gap between retrieval and generative models is larger.

Original: This is understandable, as retrieval models are limited to producing a response from the training set where the unseen topic did not appear.
Revised: This is understandable, as retrieval models are limited to producing a response from the training set where the unseen topic did not appear.

Original: There is still a considerable gap to human ratings of other humans compared to all our models (ﬁrst row of Table 5).
Revised: There is still a considerable gap to human ratings of other humans compared to all our models (ﬁrst row of Table 5).

Original: Figure 2 shows example conversations with the retrieval and generative models.
Revised: Figure 2 shows example conversations with the retrieval and generative models.

Original: Additional analysis and examples can be found in Appendix B.3 and C.
Revised: Additional analysis and examples can be found in Appendix B.3 and C.

Original:  C ONCLUSION
Revised:  C ONCLUSION

Original: In this work we build dialogue agents which are able to employ large memory systems containing encyclopedic knowledge about the world in order to conduct engaging open-domain conversations.
Revised: In this work we build dialogue agents which are able to employ large memory systems containing encyclopedic knowledge about the world in order to conduct engaging open-domain conversations.

Original: We develop a set of architectures, Transformer Memory Network models, that are capable of retrieving and attending to such knowledge and outputting a response, either in retrieval or generative modes.
Revised: We develop a set of architectures, Transformer Memory Network models, that are capable of retrieving and attending to such knowledge and outputting a response, either in retrieval or generative modes.

Original: To train and evaluate such models, we collect the Wizard of Wikipedia dataset, a large collection of open-domain dialogues grounded by Wikipedia knowledge, and demonstrated the effectiveness of our models in automatic and human experiments.
Revised: To train and evaluate such models, we collect the Wizard of Wikipedia dataset, a large collection of open-domain dialogues grounded by Wikipedia knowledge, and demonstrated the effectiveness of our models in automatic and human experiments.

Original: We will make our code and data publicly available  to encourage further model exploration, and  expect such efforts will result in signiﬁcant advances in this important research direction.
Revised: Our new publicly available benchmark aims to encourage further model exploration, and we expect such efforts will result in signiﬁcant advances in this important research direction.

Original: There is much future work to be explored using our task and dataset.
Revised: There is much future work to be explored using our task and dataset.

Original: Some of these include: (i) bridging the gap between the engagingness of retrieval responses versus the ability of generative models to work on new knowledge and topics, (ii) learning to retrieve and reason simultaneously rather than using a separate IR component; and (iii) investigating the relationship between knowledge-grounded dialogue and existing QA tasks which also employ such IR systems.
Revised: Some of these include: (i) bridging the gap between the engagingness of retrieval responses versus the ability of generative models to work on new knowledge and topics, (ii) learning to retrieve and reason simultaneously rather than using a separate IR component; and (iii) investigating the relationship between knowledge-grounded dialogue and existing QA tasks which also employ such IR systems.

Original: The aim is for those strands to come together to obtain an engaging and knowledgeable conversational agent.
Revised: The aim is for those strands to come together to obtain an engaging and knowledgeable conversational agent.

Original:  D ATASET 
Revised: A D ATASET C OLLECTION A. H UMAN A NNOTATION I NTERFACE (F OR W IZARD )

Original: C OLLECTION A. H UMAN A NNOTATION I NTERFACE (F OR W IZARD )

Original: A. W IZARD OF W IKIPEDIA E XAMPLES
Revised: A. W IZARD OF W IKIPEDIA E XAMPLES

Original: [Table]
Revised: [Table]

Original: So I am a lifeguard.
Revised: So I am a lifeguard.

Original: Know anything 
Revised: Know anything about saving lives in water

Original: [Table]
Revised: [Table]

Original: I’m impressed!
Revised: I’m impressed!

Original: It’s a big responsibility to supervise other people’s safety in the water!
Revised: It’s a big responsibility to supervise other people’s safety in the water!

Original: Tell [Table] more
Revised: Tell me more.

Original: [Table]
Revised: [Table]

Original: Well, I help make sure people do not drown or get injured while in or near the water
Revised: Well, I help make sure people do not drown or get injured while in or near the water

Original: [Table]
Revised: [Table]

Original: A lifeguard is
Revised: A lifeguard is

Original: [Table]

Original: who
Revised: a

Original: [Table]
Revised: rescuer who supervises the safety and rescue of swimmers, surfers Lifeguards are strong swimmers

Original: and
Revised: and

Original: [Table] of [Table] [Table]
Revised: trained

Original: [Table]

Original: are
Revised: in

Original: [Table]

Original: CPR
Revised: CPR

Original: AED ﬁrst
Revised: AED ﬁrst

Original: aid
Revised: aid

Original: [Table]

Original:  certiﬁed
Revised: , certiﬁed

Original: in water
Revised: in water

Original: [Table]

Original: [Table]
Revised: [Table]

Original: In some areas, the lifeguard service also carries out mountain rescues, or may function as
Revised: In some areas, the lifeguard service also carries out mountain rescues, or may function as

Original: the
Revised: the

Original: primary EMS provider
Revised: primary EMS provider

Original: [Table]
Revised: [Table]

Original: I’ve heard that in some places, lifeguards also help with other sorts of emergencies, like mountain rescues!
Revised: I’ve heard that in some places, lifeguards also help with other sorts of emergencies, like mountain rescues!

Original: Is that part of your job too
Revised: Is that part of your job too?

Original: [Table]
Revised: [Table]

Original: I have!
Revised: I have!

Original: I feel like you know much about this!
Revised: I feel like you know much about this!

Original: What brings you to know so much?
Revised: What brings you to know so much?

Original: [Table]
Revised: [Table]

Original: Oh, that’s about the extent of my knowledge.
Revised: Oh, that’s about the extent of my knowledge.

Original: I’ve just been around beaches
Revised: I’ve just been around beaches

Original: and I’ve always admired lifeguards.
Revised: and I’ve always admired lifeguards.

Original: I’m not a super strong swimmer myself.
Revised: I’m not a super strong swimmer myself.

Original: [Table]
Revised: [Table]

Original: I love animals and think armadillos are awesome with their leathery shell.
Revised: I love animals and think armadillos are awesome with their leathery shell.

Original: [Table]
Revised: [Table]

Original: I don’t think I’ve ever seen an armadillo in real life!
Revised: I don’t think I’ve ever seen an armadillo in real life!

Original: [Table]
Revised: [Table]

Original: I’ve seen them at the zoo.
Revised: I’ve seen them at the zoo.

Original: Armadillo means little armored one in Spanish.
Revised: Armadillo means little armored one in Spanish.

Original: [Table]
Revised: [Table]

Original: Are they native to a Spanish-speaking part of the world?
Revised: Are they native to a Spanish-speaking part of the world?

Original: [Table]
Revised: [Table]

Original: Armadillos are New World placental mammals in the order Cingulata ...
Revised: Armadillos are New World placental mammals in the order Cingulata ...

Original: The word “armadillo” means “little armoured one” in Spanish. · 
Revised: The word “armadillo” means “little armoured one” in Spanish. · ·

Original: [Table]
Revised: [Table]

Original: The nine-banded armadillo (“Dasypus novemcinctus”), or the nine-banded, long-nosed armadillo, is a medium-sized mammal found in North, Central, and South America.
Revised: The nine-banded armadillo (“Dasypus novemcinctus”), or the nine-banded, long-nosed armadillo, is a medium-sized mammal found in North, Central, and South America.

Original: [Table]
Revised: [Table]

Original: Yes, they are most commonly found in North, Central, and South America
Revised: Yes, they are most commonly found in North, Central, and South America

Original: [Table]
Revised: [Table]

Original: I just love ice cream.
Revised: I just love ice cream.

Original: I love the types with fruits and ﬂavours.
Revised: I love the types with fruits and ﬂavours.

Original: Do you like ice cream?
Revised: Do you like ice cream?

Original: Apprentice: I love Ice cream as much as any one.
Revised: Apprentice: I love Ice cream as much as any one.

Original: I especially like Gelato, foreign ice cream!
Revised: I especially like Gelato, foreign ice cream!

Original: [Table]
Revised: [Table]

Original: Ice cream is a sweetened frozen food typically eaten as a snack or dessert....
Revised: Ice cream is a sweetened frozen food typically eaten as a snack or dessert....

Original: It is usually made from dairy products, such as milk and cream, and ... · · · Bacon ice cream (or bacon-and-egg ice cream) is an ice cream generally created by adding bacon to egg custard and freezing the mixture.
Revised: It is usually made from dairy products, such as milk and cream, and ... · · · Bacon ice cream (or bacon-and-egg ice cream) is an ice cream generally created by adding bacon to egg custard and freezing the mixture.

Original: Wizard: Me too.
Revised: Wizard: Me too.

Original: There are some strange combinations though, have you heard of bacon ice cream?
Revised: There are some strange combinations though, have you heard of bacon ice cream?

Original: where they add bacon and even egg custard to the freezing mixture!
Revised: where they add bacon and even egg custard to the freezing mixture!

Original: Apprentice: Surprisingly bacon ice cream doesn’t surprise me.
Revised: Apprentice: Surprisingly bacon ice cream doesn’t surprise me.

Original: That doesn’t sound appealing to me, but perhaps it could be delicious...
Revised: That doesn’t sound appealing to me, but perhaps it could be delicious...

Original: where bothwizard and apprentice are humans.
Revised: both apprentice are humans access they statements

Original: The wizard has access to an information retrieval system over Wikipedia, so that they can ask and answer questions, and make statements relevant to the discussion.

Original: For each utterance, knowledge retrieval is performed based on dialogue history, giving ∼ 61 knowledge candidates per turn, with wizards clicking no sentence used 6.2% of the time.

Original: Assuming that a question contains a question mark or begins with ’how’, ’why’, ’who’, ’where’, ’what’ or ’when’, in the dataset Apprentices ask questions in 13.9% of training set utterances, and answer questions (i.e., the Wizard has asked a question) 39.5% of the time, while saying new or follow-on statements (neither asking nor answering a question) 49.3% of the time.
Revised: Assuming that a question contains a question mark or begins with ’how’, ’why’, ’who’, ’where’, ’what’ or ’when’, in the dataset Apprentices ask questions in 13.9% of training set utterances, and answer questions (i.e., the Wizard has asked a question) 39.5% of the time, while saying new or follow-on statements (neither asking nor answering a question) 49.3% of the time.

Original: Hence, the wizard and apprentice conduct conversations with a variety of dialogue acts.
Revised: Hence, the wizard and apprentice conduct conversations with a variety of dialogue acts.

Original: A. T OPIC S ELECTION
Revised: A. T OPIC S ELECTION

Original: To choose between topics that are natural we employed the existing Persona-Chat dataset (Zhang et al., 2018) where crowdworkers where asked to create personas of typical speakers.
Revised: To choose between topics that are natural we employed the existing Persona-Chat dataset (Zhang et al., 2018) where crowdworkers where asked to create personas of typical speakers.

Original: There are
Revised: There are

Original: ∼ 1000 personas, each of which consists of 4-5 sentences describing that person’s interests, e.g. “I love watching Game of Thrones”, “I like to eat cheetos” and “I recently got a cat”.
Revised: ∼ 1000 personas, each of which consists of 4-5 sentences describing that person’s interests, e.g. “I love watching Game of Thrones”, “I like to eat cheetos” and “I recently got a cat”.

Original: These can thus naturally be seen as topics of interest, and using another set of annotators we mapped each sentence to 1 or more relevant Wikipedia pages, if possible, e.g. “Ariel is my favorite princess” was labeled with the Wikipedia page for The Little Mermaid .
Revised: These can thus naturally be seen as topics of interest, and using another set of annotators we mapped each sentence to 1 or more relevant Wikipedia pages, if possible, e.g. “Ariel is my favorite princess” was labeled with the Wikipedia page for The Little Mermaid .

Original: As some sentences are harder to connect with
Revised: As some sentences are harder to connect with

Original: Wikipedia, e.g. “I am witty”, they are left unlabeled.
Revised: Wikipedia, e.g. “I am witty”, they are left unlabeled.

Original: We thus obtain 1,431 topics in total to use for our task.
Revised: We thus obtain 1,431 topics in total to use for our task.

Original: We retain the persona topic sets and thus present 2-3 related topic choices as conversation starters per dialogue during data collection.
Revised: We retain the persona topic sets and thus present 2-3 related topic choices as conversation starters per dialogue during data collection.

Original: B A DDITIONAL E XPERIMENTS B. K NOWLEDGE S ELECTION
Revised: B A DDITIONAL E XPERIMENTS B. K NOWLEDGE S ELECTION

Original: [Table]
Revised: [Table]

Original: B. F ULL D IALOGUE WITH R ETRIEVAL B. H UMAN E XPERIMENTS C E RROR A NALYSIS
Revised: B. F ULL D IALOGUE WITH R ETRIEVAL B. H UMAN E XPERIMENTS C E RROR A NALYSIS

Original: We perform an analysis of the dialogues produced from the human evaluation experiments detailed in Section 5.3.
Revised: We perform an analysis of the dialogues produced from the human evaluation experiments detailed in Section 5.3.

Original: We sample 20 conversations from each experimental setting, split between seen and unseen .
Revised: We sample 20 conversations from each experimental setting, split between seen and unseen .

Original: Conversations are re-tokenized and lowercased to reduce superﬁcial differences between models, and then analyzed in a single-blind setup.
Revised: Conversations are re-tokenized and lowercased to reduce superﬁcial differences between models, and then analyzed in a single-blind setup.

Original: We note of common errors and behaviors exhibited in each of the different conversations.
Revised: We note of common errors and behaviors exhibited in each of the different conversations.

Original: In general, the human-human conversations are starkly different than any of the bot conversations – humans tend to have more small talk, or use the topic of discussion as a mere icebreaker, with neither human behaving as a wizard.
Revised: In general, the human-human conversations are starkly different than any of the bot conversations – humans tend to have more small talk, or use the topic of discussion as a mere icebreaker, with neither human behaving as a wizard.

Original: This is in contrast to human-human conversations from the
Revised: This is in contrast to human-human conversations from the

Original: Wizard dataset itself, where one human has access to Wikipedia, and the conversation becomes more grounded in factual sentences.
Revised: Wizard dataset itself, where one human has access to Wikipedia, and the conversation becomes more grounded in factual sentences.

Original: Similarly, all models attempt to play the role of wizard and produce more factual sentences too.
Revised: Similarly, all models attempt to play the role of wizard and produce more factual sentences too.

Original: In some rounds, humans treat the bot as a sort of question-answer machine, suggesting that the models could be improved by additionally employing SQuAD-like training data.
Revised: In some rounds, humans treat the bot as a sort of question-answer machine, suggesting that the models could be improved by additionally employing SQuAD-like training data.

Original: The retriever without knowledge is particularly prone to non sequiturs, or rapidly changing the subject.
Revised: The retriever without knowledge is particularly prone to non sequiturs, or rapidly changing the subject.

Original: During unseen conversations, it is especially likely to discuss something other than the chosen topic.
Revised: During unseen conversations, it is especially likely to discuss something other than the chosen topic.

Original: In contrast, the retriever with knowledge tends to stick to the chosen topic strongly, but has difﬁculty if the human changes the subject.
Revised: In contrast, the retriever with knowledge tends to stick to the chosen topic strongly, but has difﬁculty if the human changes the subject.

Original: Frequently in unseen topics, the retriever with
Revised: Frequently in unseen topics, the retriever with

Original: [Table] [Table]
Revised: [Table] [Table]

Original: knowledge produces similar, but factually inaccurate answers to user queries.
Revised: knowledge produces similar, but factually inaccurate answers to user queries.

Original: For example, when one user asks about parts of Ireland to visit, the model enumerates a list of locations in Greece.
Revised: For example, when one user asks about parts of Ireland to visit, the model enumerates a list of locations in Greece.

Original: Nonetheless [Table] its repertoire of available responses often include inviting responses, allowing the bot to have a more natural conversational ﬂow.
Revised: Nonetheless, its repertoire of available responses often include inviting responses, allowing the bot to have a more natural conversational ﬂow.

Original: Selected conversations with the retriever with knowledge may be found in Figure 4, for both seen and unseen topics
Revised: Selected conversations with the retriever with knowledge may be found in Figure 4, for both seen and unseen topics

Original: [Table]
Revised: [Table]

Original: The generator without knowledge is particularly prone to many of the typical behaviors of seq2seq systems (Li et al., 2016; Vijayakumar et al., 2016), including local repetition (“cookies are made of ﬂour, oil, oil
Revised: The generator without knowledge is particularly prone to many of the typical behaviors of seq2seq systems (Li et al., 2016; Vijayakumar et al., 2016), including local repetition (“cookies are made of ﬂour, oil, oil

Original: [Table]
Revised: [Table]

Original: and oil”), global repetition (producing the near same utterance for multiple turns), or inconsistencies in its personality (saying it both likes and dislikes movies).
Revised: and oil”), global repetition (producing the near same utterance for multiple turns), or inconsistencies in its personality (saying it both likes and dislikes movies).

Original: The generator with knowledge has signiﬁcantly fewer issues with repetition, as it errs on the side of copying large fragments from the Wikipedia knowledge.
Revised: The generator with knowledge has signiﬁcantly fewer issues with repetition, as it errs on the side of copying large fragments from the Wikipedia knowledge.

Original: The generator with knowledge can also act as a selﬁsh conversationalist, choosing to respond or detail information without inviting a response.
Revised: The generator with knowledge can also act as a selﬁsh conversationalist, choosing to respond or detail information without inviting a response.

Original: Although it generally produces accurate statements, it sometimes produces statements using an incorrect date, name or word.
Revised: Although it generally produces accurate statements, it sometimes produces statements using an incorrect date, name or word.

Original: It also frequently produces formulaic responses, like “I don’t know, but I do know that [Wikipedia excerpt]”.
Revised: It also frequently produces formulaic responses, like “I don’t know, but I do know that [Wikipedia excerpt]”.

Original: Nonetheless, we ﬁnd the generator with knowledge is able to successfully generalize to unseen topics using the knowledge from Wikipedia.
Revised: Nonetheless, we ﬁnd the generator with knowledge is able to successfully generalize to unseen topics using the knowledge from Wikipedia.

Original: Selected conversations with the generator with knowledge may be found in Figure 5.
Revised: Selected conversations with the generator with knowledge may be found in Figure 5.

Original: [Table]
Revised: [Table]

Original: but would love to Model: It is the largest city in Northern Ireland.
Revised: but would love to Model: It is the largest city in Northern Ireland.

Original: (*) Human: I’ve always found Ireland to be fascinating and would love to visit sometime Model:
Revised: (*) Human: I’ve always found Ireland to be fascinating and would love to visit sometime Model:

Original: Same here!
Revised: Same here!

Original: It is the capital of Ireland too!
Revised: It is the capital of Ireland too!

Original: Human:
Revised: Human:

Original: What is the most popular place to visit in Ireland?
Revised: What is the most popular place to visit in Ireland?

Original: Model: I would recommend the Thrace, Crete, and the Ionian Islands as must see places.
Revised: Model: I would recommend the Thrace, Crete, and the Ionian Islands as must see places.

Original: (*) Human: I don’t know anything about the Ionian Islands Model: It was once referred to as the Hellenic Republic!
Revised: (*) Human: I don’t know anything about the Ionian Islands Model: It was once referred to as the Hellenic Republic!

Original: (*) Topic: American football ( unseen ) Model: American football can be traced back to Britain in the mid-19th century.
Revised: (*) Topic: American football ( unseen ) Model: American football can be traced back to Britain in the mid-19th century.

Original: The rules were very similar!
Revised: The rules were very similar!

Original: Human: I did not know that.
Revised: Human: I did not know that.

Original: Did they wear equipment back then?
Revised: Did they wear equipment back then?

Original: Model: Am not really certain about it but American football resulted from several major divergences from association football and rugby football Human: The Philadelphia eagles actually have a rugby player on their team this year.
Revised: Model: Am not really certain about it but American football resulted from several major divergences from association football and rugby football Human: The Philadelphia eagles actually have a rugby player on their team this year.

Original: he never played football until this year.
Revised: he never played football until this year.

Original: Model: Philadelphia is one of a few cities to have all four teams, but they love the Eagles more than anything else by a long shot!
Revised: Model: Philadelphia is one of a few cities to have all four teams, but they love the Eagles more than anything else by a long shot!

Original: Human:
Revised: Human:

Original: As a life long Philadelphia sports fan I have to agree with you.
Revised: As a life long Philadelphia sports fan I have to agree with you.

Original: Model: I’ve always been more of a fan of the American Football team from Pittsburgh, The Steelers!
Revised: Model: I’ve always been more of a fan of the American Football team from Pittsburgh, The Steelers!

Original: Human: Being from Philadelphia I am not much of a pittsburgh fan.
Revised: Human: Being from Philadelphia I am not much of a pittsburgh fan.

Original: Model: Are the Pittsburgh Steelers in the AFC?
Revised: Model: Are the Pittsburgh Steelers in the AFC?

Original: Human:
Revised: Human:

Original: Yes they are.
Revised: Yes they are.

Original: the AFC north to be exact.
Revised: the AFC north to be exact.

Original: Topic: List of national parks of the United States ( unseen ) Human: I have never visited a national park in the United States, but I would really like to!
Revised: Topic: List of national parks of the United States ( unseen ) Human: I have never visited a national park in the United States, but I would really like to!

Original: Model: Me too.
Revised: Model: Me too.

Original: I’d love to live in a national park Human: Which one do you think is the best to visit ﬁrst?
Revised: I’d love to live in a national park Human: Which one do you think is the best to visit ﬁrst?

Original: Model: The United Nations is an important place to visit, the Statue of Liberty, Central park is great also Human:
Revised: Model: The United Nations is an important place to visit, the Statue of Liberty, Central park is great also Human:

Original: Is the Statue of Liberty considered a National Park?
Revised: Is the Statue of Liberty considered a National Park?

Original: Model:
Revised: Model:

Original: Yes, the statue of Liberty is there on an island - The Liberty Island.
Revised: Yes, the statue of Liberty is there on an island - The Liberty Island.

Original: Human: I had no idea that that was a National Park.
Revised: Human: I had no idea that that was a National Park.

Original: I have heard Zion National Park is really nice!
Revised: I have heard Zion National Park is really nice!

Original: Model: Yes, it links the shenandoah national park and the great smoky mountains together!
Revised: Model: Yes, it links the shenandoah national park and the great smoky mountains together!

Original: [Table]
Revised: [Table]

Original: Human:
Revised: Human:

Original: Is Dano’s pizzaria open everyday?
Revised: Is Dano’s pizzaria open everyday?

Original: Model: yes it is.
Revised: Model: yes it is.

Original: it is an american style pizza chain headquartered in midtown brook, illinois.
Revised: it is an american style pizza chain headquartered in midtown brook, illinois.

Original: Topic: Animals in sport ( unseen ) Model: i love to watch animals.
Revised: Topic: Animals in sport ( unseen ) Model: i love to watch animals.

Original: it’s the most popular form of animal-related sport.
Revised: it’s the most popular form of animal-related sport.

Original: Human: horse racing is the most awesome animal sport!
Revised: Human: horse racing is the most awesome animal sport!

Original: Model: horse racing is an equestrian performance sport, typically involving two or more horses ridden by jockeys or driven over a set distance for competition.
Revised: Model: horse racing is an equestrian performance sport, typically involving two or more horses ridden by jockeys or driven over a set distance for competition.

Original: Human: yes, the Kentucky Derby and the triple crown is something I look forward to every year!
Revised: Human: yes, the Kentucky Derby and the triple crown is something I look forward to every year!

Original: Model: the horse has evolved over the past 45 to 55 million years from a small multi toed creature Human: that is very interesting, how did people come to domesticate horses for sport?
Revised: Model: the horse has evolved over the past 45 to 55 million years from a small multi toed creature Human: that is very interesting, how did people come to domesticate horses for sport?

Original: Model: i’m not sure but the domestication is believed to have been widespread by 3000 bc Human: very interesting!
Revised: Model: i’m not sure but the domestication is believed to have been widespread by 3000 bc Human: very interesting!

Original: I wonder how long people used horses to hunt before deciding to race them.
Revised: I wonder how long people used horses to hunt before deciding to race them.

Original: Model: i’m not sure
Revised: Model: i’m not sure

Original: but I do know that horses have been domesticated for thousands of years.
Revised: but I do know that horses have been domesticated for thousands of years.

Original: Topic: Dairy farming ( unseen ) Human: hi there i was wondering how you felt about farming Model: I do not like dairy farming
Revised: Topic: Dairy farming ( unseen ) Human: hi there i was wondering how you felt about farming Model: I do not like dairy farming

Original: but I do know that in developed countries, dairy farms typically consist of high dairy products.
Revised: but I do know that in developed countries, dairy farms typically consist of high dairy products.

Original: Human:
Revised: Human:

Original: oh yes they have alot of cows and push out alot of dairy every hour Model: other species used in commercial dairy farming include goats, sheep, and camels Human: i did not know you could get milk a camel Model: camel milk has supported bedouin Human:
Revised: oh yes they have alot of cows and push out alot of dairy every hour Model: other species used in commercial dairy farming include goats, sheep, and camels Human: i did not know you could get milk a camel Model: camel milk has supported bedouin Human:

Original: oh wow thats very interesting Model: in italy they are growing in popularity to produce an alternative milk source for humans
Revised: oh wow thats very interesting Model: in italy they are growing in popularity to produce an alternative milk source for humans

Original: Human: i have had goat milk before Model: the earliest evidence of using domesticated cows for dairy production is the seventh millennium bc-the neolithic era - Human: thats so neat.
Revised: Human: i have had goat milk before Model: the earliest evidence of using domesticated cows for dairy production is the seventh millennium bc-the neolithic era - Human: thats so neat.

Original: i would say cow milk is my favorite Model: I love cow milk, especially since the iphone is used in the iphone.
Revised: i would say cow milk is my favorite Model: I love cow milk, especially since the iphone is used in the iphone.

Original: Human: iphone and cow milk Model: the iphone was ﬁrst released in 2007.
Revised: Human: iphone and cow milk Model: the iphone was ﬁrst released in 2007.

Original: Human:
Revised: Human:

Original: oh ok well thanks for the talk have a great day Model: no problem!
Revised: oh ok well thanks for the talk have a great day Model: no problem!

Original: the iphone was ﬁrst released in 2007.
Revised: the iphone was ﬁrst released in 2007.