WikiChat: Stopping the Hallucination of Large Language Model Chatbots by Few-Shot Grounding on Wikipedia
Authors: Sina Semnani
Violet Yao
Heidi Chenyu Zhang
Monica Lam
Venue: EMNLP 2023 Findings (2023)
Keywords: large language models
hallucination
knowledge-grounded dialogue
TL;DR: We build and evaluate a fully few-shot LLM-based chatbot that almost never hallucinates, then distill it into a 7B-parameter LLaMA model.

Abstract: This paper presents the first few-shot LLM-based chatbot that almost never hallucinates and has high conversationality and low latency. WikiChat is grounded on the English Wikipedia, the largest curated free-text corpus.

WikiChat generates a response from an LLM, retains only the grounded facts, and combines them with additional information it retrieves
from the corpus to form factual and engaging responses. We distill WikiChat based on GPT-4 into a 7B-parameter LLaMA model with minimal loss of quality, to significantly improve its latency, cost and privacy, and facilitate research and deployment.

Using a novel hybrid human-and-LLM evaluation methodology, we show that our best system achieves 97.3% factual accuracy in simulated
conversations. It significantly outperforms all retrieval-based and LLM-based baselines, and by 3.9%, 38.6% and 51.0% on head, tail and recent knowledge compared to GPT-4. Compared to previous state-of-the-art retrieval-based chatbots, WikiChat is also significantly more informative and engaging, just like an LLM.

WikiChat achieves 97.9% factual accuracy in conversations with human users about recent topics, 55.0% better than GPT-4, while receiving significantly higher user ratings and more favorable comments.

WikiChat:
Stopping the Hallucination of Large Language Model Chatbots by Few-Shot Grounding on Wikipedia
Sina J. Semnani
Violet Z. Yao* Heidi C. Zhang"
Monica S. Lam
Computer Science Department
Stanford University
Stanford, CA
{sinaj, vyao, chenyuz, lam}@cs. stanford. edu
Abstract
after their pre-training, and are far less knowledge- able about less popular, or tail, topics (Mallen al., 2022; Sun et al., 2023). Therefore, for knowledge- intensive tasks (Lewis et al., 2020), users need to painstakingly verify any information they receive with external sources lest they be misled.
This paper presents the first few-shot LLM- based chatbot that almost never hallucinates and has high conversationality and low la- lency. WikiChat is grounded on the English Wikipedia, the largest curated free-text corpus.
WikiChat generates response from LLM, retains only the grounded facts, and combines them with additional information it retrieves from the corpus form factual and engaging responses. We distill WikiChat based on GPT-4 into 7B-parameter LLaMA model with mini- mal loss of quality, to significantly improve its latency, cost and privacy, and facilitate research and deployment',
This paper focuses on three metrics for knowledge-intensivee dialogues: factuality, conver- sationality, and latency. knowledge-based chat- bot needs to be first and foremost factual. We assume access to source of trusted text corpus; here the English Wikipedia is assumed to be fac- tual. While LLMs tend to hallucinate, they carry out natural and engaging conversations rather than giving dry answers to users' questions. We refer to the ability to give relevant, informational, natural, non-repetitive and temporally accurate an- swers collectively as conversationality. We single out latency as the third metric of focus because so- lutions addressing factuality like Gao et al. (2023); Jiang et al. (2023); Trivedi et al. (2023); Zhao et al. (2023) tend incur high latency that degrades experience and hinders adoption.
Using a novel hybrid human-and-LLLM evalua- tion methodology, we show that our best system achieves 97.3% factual accuracy in simulated conversations. It significantly outperforms all retrieval-based and LLM-baser baselines, and by 3.9%, 38.6 38.6% and 51.0% on head, tail re- cent knowledge compared to GPT-44 Compared to previous state-of-the-an retrieval- chat- bots, WikiChat is also significantly more infor- mative and engaging, just like an LLM.
WikiChat achieves 97.9% factual accuracy in conversations with human users about recent topies, 55.0% better than GPT-4, while receiv- ing significantly higher user ratings and more favorable comments.
1.1 Current Approaches
The basis of factuality in this work is informa- tion retrieval (IR), which bases the chatbot's s re- sponses retrieved information from trusted corpus (Zhang al., 2023; Li et al., 2022: Shuster et al., 2021). The retrieve- -ennnntee approach generates a response from data retrieved with the user query (Lewis et al., 2020; Trivedi et al., 2023; Izacard et al., 2022: Shuster et al., 2022b: Chen et al., 2021). Previous work is either not evaluated on conversational tasks (Lewis et al., 2020; Trivedi et al., 2023; Shi al., 2023), this paper, is likely to generate irrelevant and un- natural outputs when properly evaluated (Izacard et al., 2022). More importantly, chatbots based on retrieve- seaiveethenneeeee then-geenertte pipelines may still halluci- nate. popular academic datasets like Wizard
1 Introduction
Recent dramatic advances in LLM chatbots have made them indispensable tools for millions of peo- ple (Hu, 2023) who have come rely their broad skill set. Yet, LLM chatbots prone to hallucina- providing misleading information, tion (Bang al., 2023), often using a convineing and confident language. Notably, LLMs do not speak accurately about recent events that occurred
show in
"Equal contribution
'Code and model available at httpps://giteee com/ stanford-oval/Hiii ovaa//iiiihhat.

---

User: Have you heard about the upcoming film Oppenheimer? WikiChat: Yea! Oppenheimer biopic...
Chrietopher Nolan has Cillian
Murphy titulor role Oppenhelmer. keeping his tradition meticullous choices, I'm will be excellent film
Refine
User: What da you think about the casting?
1. Query
Title: Christopher Nolen
Ciliian M. Murphy has been cast play
Robirrt Opperbamme in Or Critstppeer
Query:
Naken's moilth dineturid verdum will be "Opprrhsimer" (2025)...
Notan's "Oppenteemmt mer move
Oppennslmer
2. Summarize & Filter
stopher Nickn is lenoon for modouuous casting
Oppenthelmer (flmm)
Oppenhalmmr signed
the signad betwoen Septembor Apri
agg September AndiApri
April 2022.
2012
Titl:: Cillian Murphty ... Murphy was contimmed star as J. Roberl Oppernetmer Nolan's upooming film Ogponholmer...
Clian Mumpy in play as Rober Oppenhsémer Oppenhenimer (2023) movie.
Retrivve
6. Draff
Chrislopher Molan known for his meliulous conding cholces, He often codt well-lenomm aeloon his films, this time has selected Tom Hanks and Michael Carne
mesiculous
Christopher Notan often woll actors in hä Sims.
Fact-
Christacher Nolan known Chrstaghir Henww his fllms.
Tor Hanks stars the Oppanhsines (20055) movis. Michael Caine stars in the Oppenteimer (2003) movie.
Check
Generate
4, Extract Claims
Retrieve
All WikiChat components, and sample conversation about an upcoming movie, edited for brevity. The steps taken to generate a response include (1) generating query to retrieve from Wikipedia, (2) summarizing and filtering the retrieved passages, (3) generating a response from an I.I.M (4) extracting claims from the L.MM response (5) fact-checking the claims in the LLM response using retrieved evidence, (6) drafting response, and (7)
Figure I:
refining the response.
Wikipedia (Dinan et al., 2019) and Wizard of Inter- net (Komeili et al., 2022), which are widely used to train retrieve-then- srrreve-tten-generaae chatbots, crowdwork- ers are free to add ungrounded information to their responses; in GPT-4-based commercial system like Bing Chat, only 58.7% of the facts generated are grounded in what it retrieves (Liu al., 2023a). Another IR approach is to fact-check system's s's outputs and remove errors (Gao et al., 2023; Jiang et al., 2023). When applied to LLM chatbots, the responses are conversational, but as shown in this paper, are lacking in content for recent or tail topics. Other IR approaches require expensive changes to the pre- pre-training process of language models (Lewis al., 2020; Guu al., 2020).
latency.
Few-Shot Knowledge-Groundel Chatbots. Our 7-stage pipeline (Figure 1) combines the best of IR approaches: We (1) use the user utterance retrieve information that LLMs may not be aware of, and (2) leverage the generative power of LLMs by asking them for responses and fact-checking them. All this curated information is used to draft and refine the final response.
It is not easy stop LLMs from hallucinating. In serreve-thenngeneraat pipelines, when IR does not retrieve any relevant information or when no rel- evant information is available in the knowledge cor- pus, LLMs hallucinate to pick up the slack. Thus, WikiChat summarizes and filters the retrieved in- formation instead of generating response directly. We fact-check every claim generated by LLMs sep- arately and teach the system to say "I don't know" when necessary. We teach it to understand the time context; e.g. a future tense an article may refer to a past event at the time of the conversation. Most importantly, we do not prematurely optimize for speed by forgoing these needed steps, but rely on model distillation to reduce the latency only once high quality is achieved.
Complementary IR, Knowledge Editing up- dates model weights to include recent knowledge as becomes available (De Cao et al., 2021; Meng et al., 2022; Mitchell et al., 2022). Similarly, Con- tinual Learning can be used to add new knowl- edge to LLMs (Jang et al., 2022). While these ap- proaches improve factuality on recent knowledge, they cannot address the tail knowledge problem.
1.2 WikiChat Overview
This paper presents WikiChat, the first few-shot chatbot that provides up-to-date and fact-checked information with high conversationality and low
The resulting pipeline is not specific to any cor- pus. While this paper applies this pipeline to

---

Wikipedia, the largest corpus of curated knowl- edge, to create WikiChat, applicable to any free-text corpus, including personal and corporate confidential information. The pipeline is not spe- cific to any LLM either, and we apply it to three different LLMs in this paper.
of distilling a multi-part system built with in- context learning (Brown et al., 2020) into a smaller yet effective model. Our distillation of WikiChat into a 7B- B--parameter LLaMA achieves a factual accuracy of 91.1% 1,, outperforming much larger baselines, while having 3.2 times lower end- to-end latency than its teacher model. We introduce an efficient and effective human-and-LLM methodology for evaluating knowledge-gronndee chatbots in settings beyond what is possible with just crowdsourcing.
Distillation for improved latency, affordabil- ity, and privacy. Not only is our 7-stage LLM- based pipeline slow and expensive, sending user data to LLM APIs does not provide the privacy and confidentiality needed by many applications. The simplicity in each of our stages makes it possible to effectively distill our best system into smaller multi-tasking model, which is responsive and af- fordable, and can be deployed locally for privacy. We release this model to aid further research and reproducibility of our results.
2 Related Work
Sewlledg-Grronded Chatbots. Information trieval is commonly used develop knowledge- grounded chatbots (Shuster et al., 2021). Blender- Bot 2 (Chen et al., 2021) incorporates Internet search. SceKeR (Shuster et al., 2022a) outperforms BlenderBot (Chen et al., 2021) by utilizing a sin- gle language model for three modular tasks: gen- crating search queries, producing relevant knowl- edge from retrieved documents, and generating the final response. BlenderBot 3 (Shuster et al., 2022b) fine mun-tunes 175B- 175B-parameter OPT (Zhang et al., 2022) on the combination of 20 question answering and dialogue datasets. Atlas (Izacard et al., 2022) is a state-of-the-arr model on the KILT benchmark (Petroni et al., 2021), which consists of 11 showledge-oriented tasks including Wizard of Wikipedia (Dinan et al., 2019).
Evaluation of LLM-based agents. We find that LLM-based chatbots have surpassed the quality of systems that conventional static crowdsourced benchmarks (Dinan et al., 2019; Komeili et al.. 2022) were meant to evaluate. For example, these benchmarks mainly evaluate the ability to chat about the head knowledge, which LLM chatbots are already very good at. We devise human-and- LLM hybrid evaluation methodology that can ade- quately analyze all chatbots, regardless of whether they knowledge-grounded grounded or LLM-based.
1.3 Contributions
We create a factual and engaging open-domain chatbot with 7-stage pipeline using a few-shot prompted LLM, as shown Figure 1. We vali- date the concept with three GPT-4, GPT-3.5, and LLaMA (Touvron et al., 2023) based chatbots grounded in Wikipedia.
Evaluating Factuality. FEVER (Thorne et al., 2018) a popular crowdsourced dataset that compares claims against evidence retrieved from Wikipedia, and was extended to dialogues by Gupta et al. (2022). The state-of-the-art system on this dataset (Krishna et al., 2022) has an accuracy of 81% when compared against human labels. Q² (Honovich et al., 2021) uses question answering and natural language inference models to evaluate the factuality of dialogue agents. Dziri al. (2022) compare this and several other automatic metrics to human evaluation and find that automatic met- rics fall significantly behind human performance, and rely on spurious correlations. Concurrently and similarly to our work, Min et al. (2023) break down long system outputs to claims and evaluate their fac- tuality using retrieval. Their best model has a sig- nificant 13% error rate on individual claims. Given the high error rates for these automatic metrics, our evaluation methodology (Section 5) incorpo- rates human judgement where needed. Approaches
Our experiments with simulated users show that the GPT-4-based WikiChat (WikiChat (4) achieves a 97.3% factual accuracy of its claims in simulated conversations. Each version of WikiChat more accurate than the LLM it is based on by an aver- age of 31.2%, 27.8% and 50.2% for GPT-4, GPT- 3.5 and LLLMAA respectively. It also outperforms the fine-tuned SOTA vitrieval-basee Atlas (Izacard al., 2022) in factuality and, unlike Atlas, matches the conversationality of LLMs.
Our real user study shows that WikiChat achieves 97.9% in factuality conversations of recent topics, 2.3 times more factual than GPT-4, while receiving higher user ratings.
We are the first to demonstrate the feasibility

---

like Qian al. (2023) and TripleScore (Goodrich al., 2019) only consider simple entity-relations between retrieved documents and system outputs, while we generalize that notion to all forms of facts.
The query is sent an information retrieval sys- tem to obtain relevant passages from the corpus, and the top results are re-ranked based on the tem- poral information to get NIR passages.
Stage 2. Since these passages may contain a mix- ture of relevant and irrelevant sections, WikiChat extracts relevant sections of the retrieved passages and summarizes them into bullet points while filter- ing out the irrelevant parts (Table 18).
3 Design of WikiChat
Given the history of conversation with user, WikiChat generates its next utterance by (1) cu- rating information that is correct and relevant to the conversation context, and (2) using the gath- ered information to form the response. Each stage of the pipeline in Figure is implemented using in-context learning. All prompts are included in Appendix F.
3.1.2 LLM Generation and Fact-Checking
Stage 3. We prompt the LLM to generate a re- sponse to the history of the conversation (Table 19), This response often contains interesting and rele- vant knowledge, but is inherently unreliable.
3.1 Curating Information
Stage 4. The LLM response is broken down multiple claims (Chen et al., 2022) (Table 20). This stage resolves co-references to reduce ambiguity. and resolves relative time information like "current" and "last year". to make all claims self-contained.
LLMs have the ability to interpret the user utter- ance, formulate responses that are out of the reach of retrieval or require aggregating information from many sources or drawing conclusions, as shown by the following example:
We use IR to retrieve Nevidence passages from the knowledge corpus for each claim serve evidence. We use the same time-based re-ranking as in Section 3.1.1 better handle time-sensitive topics.
User: Do you think Apple will continue to be big player in the technology industry?
GPT-4: Yes... ... Apple has strong track record of innovation, a loyal customer base, and a robust financial position.... the most valuable company globally, with market capitalization of over $2 trillion... continues to explore technologies such augmented reality, artificial intelligence, and autonomous vehieles. This will help them stay ahead of the competition.
Stage 5. The verification prompt (Table 21) uses chain-of-the tf-ttuugtt prompting ( (Wei et al., 2022) to assign each claim to one of three classes: whether the retrieved evidence supports the claim, refutes the claim, or if there is not enough information in the evidence to make this decision. Only claims that supported by the evidence are kept.
Unfortumately, LLMs cannot be trusted to be factual, thus we need fact-check their outputs. Furthermore, LLMs are unaware of recent events. Thus, we use both LLM generation and IR.
3.2 Forming the Response
The next step is to use the curated information to form an appealing response. Our experiments show that writing the final response in one go while sat- isfying all conversationality criteria is challenging with in-context learning, especially that the limited context length makes it difficult provide enough multi-turn conversations as few-shot examples cover all the necessary aspects. Thus, we use a two-step approach:
3.1.1 Retrieval from Corpus
During a conversation with a user, WikiChat identi- fies when accessing external information is needed. This could be because the last user utterance con- tains direct question (e.g. "Who is Stephen Curry?") or otherwise requires additional infor- mation for a comprehensive response (e.g. "I really like Stephen Curry.").
Stage 1. WikiChat generates search query that captures the user's 's interest with a prompt (Table 17). We discovered that existing systems especially struggle with the temporal context. WikiChat gen- erates the inferred time of the user's need alongside the query. The query time can be one of recent, year=yyyy, or none for when the retrieved informa- tion should be as recent as possible, for specific year, or the time not important, respectively.
Stage 6. WikiChat generates a draft response from the given list of bullet points and the history of the conversation (Table 22).
Stage 7. It then generates feedback and refines the response based on relevance, naturalness, non- repetitiveness, and temporal correctness (Table 23). The feedback contains the model's reasoning on each criterion and a score between and 100 for each. Refinement is conditioned on this feedback

---

and the scores as a chain of thought. Concurrent this work, Madaan et al. (2023) have explored the idea of prompting LLMs to refine their own generations in other settings.
they ... have enough knowledge of so that they can conduct a reasonable conversation." Since LLMs are already good conversants about familiar topics. testing them on these topics would lead to the false conclusion that no innovation is necessary.
As discussed in Section 1.2, is hard for LLMs to say "I don't 't know". In the special case where the curation stages return no relevant information, the draft prompt is skipped and instead "Sorry, I'm not sure" is sent to the refinement prompt. which dresses up match the conversation.
Furthermore, static benchmarks quickly lose their effectiveness in evaluating chatbots' use of up-to-date information whenever new LLM is released. For example, Wizard of Wikipedia does not contain any topies that are not seen by GPT-3, GPT-4 or LLaMA during their pre-training.
4 Model Distillation
Here we propose novel combination of simu- lated and real user conversations, as well as human and LLM-based evaluations, to understand the fac- tuality and conversationality of modern chatbots.
To improve latency, cost and privacy, we distill WikiChat based on a teacher LLM into a smaller student model. We use WikiChat based on GPT-4 (i.e. WikiChat G4) as the teacher, and the publicly available LLaMA (Touvron et al., 2023) model as the student to obtain WikiChat
5.1 Collecting Dialogues
Conversation Topics In our experiment, we pick an article from the knowledge corpus Wikipedia as starter topic. We choose a diverse set of topics covering the space of head, tail, and recent knowl- edge. Similar Mallen al. (2022), use the total number of views of a Wikipedia article as a proxy for how frequently that topic is likely be discussed on the Web and therefore the pre-training data of LLMs, given that views in Wikipedia are to a large degree generated from other online sources, Head: These are articles with the highest total view count, up to the end of 2020, which is before the cut-off date of the pre-training data of all LLMs we evaluate. Example article titles include "Sting (musician)". "Barack Obama". and "Gmail". The view count ranges from 68M to 16M for the head topics.
Each few-shot prompt consists of instruc- tion I and several examples. We use user sim- ulator (described in Section 5.1) to talk the teacher WikiChat about topies from Wikipedia, while recording the inputs the underlying teacher LLM sees, and the outputs it generates for those inputs. We use these input-output pairs and the instruction I (but not the few-shot examples) to fine-tune the student LLM. We distill all 7 sub- tasks of WikiChat into the same student model in a multi-task setting. The LLaMA- WikiChat calls LLaMA in each of the pipeline stages by spec- ifying instruction I and the input.
Distillation lowers the latency because the stu- dent LLM is many times smaller than the teacher LLM, and has shorter input length as it sees no few-shot examples, similar to context distilla- tion (Snell et al., 2022).
Tail: These are the least viewed articles, with less than 1000 views. Examples are "Amelia Geth- ing". "Last Tycoon Stakes", and "2008 CONCA- CAF Women's Olympic Qualifying Tournament". Recent: These are the most edited Wikipedia articles in the first four months of 2023, which is after the cut-off date of LLMs. Examples include big news stories of 2023 like "2023 Speaker of the United States House of Representatives election" and "Yeti Airlines Flight 691"
Furthermore, we remove chains of thought from the outputs of stages 5 and 7 (verification and re- finement), and merge stages 6 and 7. No drop in our metrics of factuality and conversationality is observed, suggesting that chain-of of-thuhf-thought prompt- ing and refinement may only be necessary for in- context learning. Fine-tuned models can learn these tasks from just inputs and outputs given big enough training set.
We manually remove topics that might be un- comfortable to talk about due to violence or explicit content, and ensure a diverse set of domains.
5 A Novel Evaluation Methodology
Most existing conversational benchmarks are crowdsourced and static. As Komeili et al. (2022) says about their use of crowdsoureing, "The in- tent... is that [erowdworkers] can choose a topic
Dialogues. For cost-effectiveness, we use sim- ulated conversations and validate them against a smaller real user study. Rule-based and neural user simulators have long been used to build and eval-

---

uate task-oriented dialogue systems (Schatzmann al., 2005; Zhu et al., 2020; Wan et al., 2022), and to generate training data for chatbots (Bao et al., 2023; Zheng et al., 2023). We use LLMs to sim- ulate users in order to evaluate knowledge-baree based chatbots. LLMs are good, fast, and cost-effective at simulating users. Via prompting, we can control the personality and specify the conversation topic. We also make sure the simulator continue the conversation by making relevant comments or ask- ing interesting questions, and handle the case where the chatbot under evaluation gives inconsistent re- sponses.
LLMs have been shown be effective in eval- uating soft qualities (Chiang and yi Lee, 2023; He al., 2023; Koemi and Federmann, 2023; Liu et al., 2023b; Finch et al., 2023), consistently bet- ter aligned with expert human evaluation than any other automatic metric. Thus, we use GPT-4 to evaluate these qualities in chatbot responses. The LLM is instructed to, given conversation history and chatbot response, "think out loud" (Wei al., 2022) about its reasoning for each criterion and provide a score. For each turn, all metrics are rated from to 5, except for temporal correctness which is converted to a binary score. We report the aver- age of each metric over all simulated conversation turns. We find that the inter-annotator agreement between GPT-4 ratings and one of the authors is about the same as the agreement between two au- thors (Appendix B).
5.2 Evaluation
Factuality Evaluated Manually. To evaluate fac- tuality, we use a hybrid human-LLM approach. We first use a GPT-4 prompt to break down each chat- bot response into small self-contained claims, and retrieve evidence for it using IR. We need to manu- ally determine whether an extracted claim is backed by the retrieved evidence because LLMs underper- form humans this task (Section 2). We ask crowdworkers to determine whether each claim is supported, refuted, or there is not enough informa- tion the retrieved paragraphs. See Appendix for more details about human evaluation and our quality control.
Implementation
WikiChat Using GPT. We create two versions of WikiChat based on GPT models: WikiChat G3.5 is based on GPT-3.5 (fcxt-davinci-003), WikiChat is based on GPT-4 (gpt-4-0314)
Distilling WikiChat to LLaMA. We use LLaMA as the target of distillation as it currently has the highest quality-to-size ratio among the pub- licly available language models. We generate train- ing data from 750 Wikipedia articles covering head, tail and recent topies; these are disjoint from the set of topics we use for evaluation. Simulating a 10-turn conversation between the user simulator and WikiChat G4 for each topic results in total of 37,499 (instruction, input, output) tuples. We hold out 5% of these conversations for validation, and fine-tune a 7B-parameter LLaMA on the rest.
When the crowdsource workers classify claim "not enough information", we need en- sure that the information is truly not available in Wikipedia, and not because IR has not retrieved the right paragraphs. The authors of this paper double check these rare cases against the entire Wikipedia. Conversationality Evaluated Automatically. Based on prior work on how chatbots should be human-like, natural, knowledgeable (Li et al., 2019) and non-repetitive (Roller et al., 2021), and our own observations of chatbots' weaknesses, we propose five response-level metrics to measure con- versationality:
Information Retrieval System. We use COIBERT v2 (Santhanam et al., 2022b) and PLAID (Santhanam et al., 2022a) over Wikipedia as our IR system. We use the WikiExtractor tool to extract the clean text from the English Wikipedia dump obtained on 4/28/2023. Like ColBERT, we divide each article (ignoring tables and information boxes) into disjoint text blocks referred to as pas- sages and prepend them with their article title. We limit the combined length of the passage and title to 120 words. In WikiChat, we set Novidence = 2 and NIR = 3. These are chosen empirically to obtain a high recall on our development set.
1. Relevant: On-topic and directly addresses the user's question.
2. Informational: Provides a suitable amount of information (whether or not it is factual).
3. Natural: Uses appropriate and engaging lan- guage to create an enjoyable experience.
4. Non-Repetitive: Does not repeat previously mentioned information.
5. Temporally Correct: Provides up-to-date in- formation and uses the appropriate tense.
2 "Accessdd via the Microsoft Azure OpenAl API Phttps://e //gi thub, com/attarda /wikiextractor

---

Factual
Relevant
Informational
Natural
Non-Repetitive
Temporal
Wi
98.8
5.0 0.0
0.2
5.0
5.0 0.0
99.0
GPT-4
94.9
5.0 + 0.4
4.6
5.2
5.0=02 0.2
99.0
WikiChat
97.1
4.5
4.6
5.2
4.90.5
94.0
Head
GPT-3.5
91.9
5.0
4.7
5.0
5.0= 0.2
96.0
WikiChat1.
95.2
4.9
4.6
4.4
49
96.0
LLaMA
83.9
4.7
1.0
4.6
=.0=02
96.0
Atlas
90.6
1.3
2.
3.6
4.5
94.0
Wi
94.6
4,8
1.0
4.4
5.0= 0.2
99.0
GPT-4
56.0
5.0
4.4
5.0
5.0 0.0
100.0
WikiChat
82.2
4.7
1.0
4.5
4.7 = 0.8
98.0
Tail
GPT-3.5
50.0
4.4
4.5
5.2
5.0 0.1
98.0
WikiChat
87.1
4.6
1.1
4.6
4.8 0.6
98.0
LLaMA
21.2
1.1
3.
4.+1.1
4.8 0.6
93.0
Atlas
86.5
32
3.
3.81111
4.5 = 1.2
99.0
WikiChat
4.5
4.9
4.3
5,0 0.1
100.0
GPT-4
47.5
4.5
0.7
5.0
5.0 0.1
97.0
WikiChat
88.2
4.
4.4
4.4
4.7 0.8
94.0
Recent
GPT-3.5
42.3
4.9
1.3
5.
4.9 0.3
91.0
WikiChat
4.
4.
4.6
4.9 0.4
LLaMA
17.7
4.7
4.1
4.8
4.9 0.4
92,0
Atlas
89.4
3.
28
3.2
4.5 1.2
81.0
WikiChat
97.3
4.4
4.648.8
4.3
5.0 0.2
99.3
GPT-4
5.0
48406
5.0
5.0 0.1
WikiChat
89.2
0.7
4.6499
4.9
4.8 0.7
95.3
GPT-3.5
4.9
4.4008
5.2
5.0 0.2
WikiChal
91.1
4.7
4.
0.5
4.9 0.5
97.0
LLaMA
0.9
4.0
0.9
4.9 0.4
Atlas
88.8
3,6 1
1.3
2.8 4.8410 1.2
3.8 1 1.1
4.5 1.2
91.3
Table 1: Evaluation results of WikiChat and baselines on simulated conversations. Factual and Temporal accuracy are percentages. Other metries are averages integers between and 5 (inclusive) and we report their mean and standard deviation. Factual accuracy is from human evaluation, other metrics are from few-shot GPT-4. Higher is better for all metrics.
All section, values that are better than their comparable model (c.g. WikiChat G4 vs. GPT-4) in statistically significant way with 0.05 underscored.
7 Simulated Dialogues Experiment
late 10 turns (5 user turns and 5 chatbot turns) with the user starting the conversation. This is compara- ble with other datasets: Wizard-of-V Wizard-of-Wikipediaa and Wizard-offInteene have and turns per dialogue on average, respectively. Examples of simulated conversations can be found in Appendix A.
Our first experiment analyzes how our systems per- form under various scenarios using simulated users.
7.1 Experiment Setup
We compare WikiChat to several LLM baselines: a 7B- 18-parameter LLaMA, GPT-3.5, and GPT-4, all prompted to act as Enowledge-orienter chatbots (See Table 19 for the prompt). We also com- pare with Atlas (Izacard et al., 2022), fine-tuned retrieval-based model, which has the state-of-the- art results on Wizard of Wikipedia and several other datasets KILT (Petroni et al., 2021). We update its knowledge source and fine-tune its model to use the same Wikipedia dump as WikiChat (Ap- pendix B).
To mimic how a curious user with limited knowl- edge may explore a topic, only the title and the first sentence of a Wikipedia article on topic shown to the user simulator (prompt in Table 16), but it is free to explore related topics from its own general knowledge. The chatbot's response is not limited to what is in the article.
main results for Wiki- Chat and all baselines on simulated conversations.
Table 1 summarizes
7.2 Factuality
We simulate users using GPT-4 due to its higher quality. Each chatbot carries out conversations on 20 topics in each of the head, tail, and recent knowl- edge categories. For each conversation, we simu-
We define the fhcctull accuracy of chatbot be the percentage of claims the bot makes in a given dialogue set, that are supported by the knowledge corpus. As mentioned in Section 5.2, this is done by

---

obtaining per-claim judgments of factuality from crowdworkers. We oblain 3 judgements for each of the 5974 claims our chatbots output in total.
Cost per
Time per
Time per
Claim (e)
Claim
Turn (s)
hat
19.6
7.4
26.6
GPT-4
0.7
2.3
5.8
The first column of Table shows the re- sults of our human evaluation of factual accuracy. WikiChat G4 achieves an impressive factual accu- racy of 08,8% 94.6%, and 98.5% for head, tail, and recent topics, respectively. WikiChat G4 scores average 8.1% and 6.2% higher than WikiChat G3.5 and WikiChat WikiChat's GPT-4, GPT-3.5, and LLaMA versions outperform their base LLMs with an average of 31.2%, 27.8%, and 50.2% respec- tively, with this gap increasing significantly in re- cent and tail knowledge. These results suggest that our pipeline is able to effectively mitigate the hal- lucination of LLMs.
ass
9.9
4.1
14.4
GPT-3.5
0.1
0.5
1.1
WikiChat
2.3
7.6
LLaMA
0.8
1.6
Table 2: The average cost (in cents) and latency (in seconds) each chatbot. The LLaMA models are run on a local GPU at negligible cost.
outputs. All API calls are done in parallel when possible, e.g. the stages 1-2 and 3-4 are indepen- dent and therefore done in parallel. LLaMA models served on single local NVIDIA A100 GPU using HuggingFace's TGI library (HuggingFace. 2023).
Note that GPT-4 scores lower in tail and recent knowledge compared head, by 38.9% and 47.4%, respectively, and the score for recent knowledge would be much lower had the simulated user not occasionally talked about older background infor- mation in the conversations. This illustrates that the common practice of solely evaluating on head knowledge would not have properly shown this weakness of LLMs.
Table shows the average cost and latency of various chatbots, LLaMA costs are negligible compared to the cost of other LLMs. Distilling WikiChat into WikiChat lowers its per-claim latency 3.2 times, bringing it in line with GPT-4. This makes WikiChat viable alternative the baselines: has similarly low latency, costs less and significantly more factual.
We note that naive implementation WikiChat took about 15 seconds per output claim. We reduced the latency 6.5 times by (1) fusing stages 6-7, (2) removing chains of thought, (3) us- ing TGI with FlashAttention (Dao et al., 2022), and other optimizations.
All three versions of WikiChat outperform the SOTA fine-tuned model Atlas on average in factu- ality.
7.3 Conversationality
Each version of WikiChat improves factuality over its base model without sacrificing conversationality. Even WikiChat i. is as good or better than LLaMA in all metrics, while scoring within 0.1 to 0.3 points of its teacher WikiChat G4-
7.5 Analysis of Results
WikiChat makes more claims than all base- lines in all subsets (Table 13). WikiChat G4+ WikiChat G3.5 and WikiChat l make an average of 3.6, 3.5 and 3.3 claims per turn, compared to 2.5, 2.2, and 2.0 claims of their base LLMs. and only 1.4 claims of Atlas. Our chatbots make more claims in the head subset as more information is available. Both information retrieval (Stages 1 and 2) and the underlying LLM (Stages 3 to 5) con- tribute to WikiChat (Table 12.) 27.0%, 32.2% and 24.5% of the claims the final response WikiChat G4. WikiChat G3.s and WikiChat 1. come from fact-checked LLM responses and the rest are from IR. This is the content that retrieve-then- generate systems cannot produce.
Atlas sacrifices substantial conversationality for factuality. Our analysis of 100 sampled dialogue turns reveals how Atlas underperforms in each metric. Informationality: it often gives short one- sentence answers when a detailed description is warranted. Relevance: it is less likely to address the user's questions. Naturalness: Atlas mainly copies from the retrieved passages instead of matching the tone of the conversation. Non-repetition: it some- times generates repetitive, near duplicate responses. Temporal accuracy: it confuses different dates.
7.4 Latency
On average, about one-third of the claims in LLM responses do not pass WikiChat's fact- checking (Table 12). The number of rejections is much higher in tail and recent subsets. This matches our expectation on where LLMs make
We measure the end-to-end latency and cost of our various chatbots. Retrieval from Wikipedia only accounts for about 0.2 seconds per turn and is neg- ligible compared to the time spent waiting for LLM

---

more factual errors. Removing these errors during the fact-checking stage is WikiChat's main weapon against ballucination. WikiChat has the highest rejection rate on tail (54.0%) and recent (64.4%) subsets because the underlying LLaMA model hal- lucinates lot more.
Factual
User Rating
WikiChat Ge GPT4
97.9
3.8
42.9
3,4
Table 3: Results from the user study. User rating difference is statistically significant with I 0.05 2.18, 0.03).
WikiChat says "I don't know" when neces- sary (Table 11). Our pipeline is carefully designed to prevent hallucination when no relevant informa- tion available. This is more prevalent for tail and recent knowledge, where the simulated user is likely to ask about the information that is not yet available in Wikipedia.
complained that the chatbot did not give a di- rect answer to some of their requests. However, it turns out that WikiChat G4 was right in giving no information, because Wikipedia truly lacked the information they sought. We did not find single valid complaint of hallucination.
GPT-4 received some favorable comments from 10 participants, noting that it is informational and that that it gave "reasonable" answers and seemed " "educated". However, it received 10 complaints: 6 on lack of specificity: "did not include key piece of information", or that "its responses were vague", "not nuanced" and "was generic and regurgitated surface level information"; on how it completely misunderstood them (due to obvious hallueination in its responses); 2 on wrong or outdated responses. More concerning, however, was that conversa- tions that received favorable comments also con- tain numerous plausible but factually incorrect sponses, except that the users did not realize that. This shows how easy it is be mislead by LLM- based chatbots.
WikiChat's refinement stage improves conver- sationality, especially in tail and recent topies (Table 15). Comparing the BLEU scores (Papineni et al., 2002) of the draft and the final responses (Ta- ble 14), we notice that WikiChat makes the most changes on tail and recent topics. Refinement im- proves naturalness, informationality and relevance of all versions of WikiChat by 0.1 to 0.4 points, and temporal correctness by 2.3% to 3.3%.
8 A Real User Study
We conduct study where participants are asked to chat about recent topic of their choice, a scenario that is particularly challenging for LLMs. We use Prolific (Prolific, 2023) to recruit 40 participants (20 female) for our user study. Each person is randomly assigned to either WikiChat G4 or GPT-4 without telling them which, and chats for 5 user turns. After each turn, they are asked to rate the response a scale of to 5. Afterwards, we ask each participant to comment their experience. More details in Appendix D.
In summary, our user study suggests that WikiChat is successful as an engaging and fac- tual chatbot while GPT-4 frequently hallucinates.
Conclusion
This paper shows how we can create a conversa- tional, factual, open-domain chatbot out of LLMs. The key insight is to properly combine retrieved data with the generative content from LLMs, with meticulous claim-by-claim fact-checking. We vali- date this methodology by creating WikiChat, which is grounded in Wikipedia, the largest hand-curated public text corpus.
Table shows the average factual accuracy (hu- man evaluated) and real user ratings. WikiChat achieves an accuracy of 97.9%, which is similar to that of simulated conversations. It outperforms GPT-4 in factuality by 55.0%, and achieves sta- tistically significant higher rating of 3.8 vs 3.4. Most participants who talked to WikiChat G4 re- ported a positive experience. Multiple participants praised its ability to provide "accurate and "up- to-date information" and multiple said that it is "natural", "conversational", "detailed" and "direct". They commented that it is "fun to play with" and "impressive" in finding information.
Our best system achieves 97.3% and 97.9% fac- tual accuracy on simulated and real conversations respectively, when GPT-4 only achieves 66.1 1.11% % and 42.9%. WikiChat resembles LLMs in conversation- ality and is preferred over GPT-4.
We also show that distilled LLaMA model with just 7B parameters can perform like 175B- parameter WikiChat G3.5 model and be as fast, cheaper and more factual than GPT-4. This ex- pands the applicability of this technology.
participants complained about the latency of the system caused mainly by fact-checking, as the study was conducted using the slower WikiChat G4-

---

Limitations
the authors for questions. The only dataset we collect in this paper is the conversations from the user study. The simulated conversations used for fine-tuning LLaMA and test- ing chatbots are based on topics from Wikipedia articles, and topics involving violence, sexual con- tent or other potentially disturbing matters were excluded.
Applications of today's s LLM-based chatbots are constantly expanding. This work focuses only on knowledge-intense dialogue. As such, other set- tings where chatbots are used to perform tasks (e.g. "write email for me") perform personalized chitchat (e.g. companion chathots) might not bene- fit from WikiChat's ' pipeline. More targeted study and evaluation of these settings is required to prop- erly balance the generative power of LLMs and the factual knowledge injected from an external corpus for these applications. Settings where a chatbot needs to exhibit initiatives (e.g. try to be persua- sive) are also outside the scope of this paper. We also only consider one-hop information retrieval, because it is the most natural and prevalent form of users' information need. Multi-hop retrieval could improve information access for more com- plex queries.
We release the code and the fine-tuned LLaMA- 7B model in accordance with its original license agreement. We believe this will encourage research and deployment of more reliable and trustworthy chatbots, to the benefit of end-users. We do not anticipate harm resulting from the approaches and modifications proposed in this paper.
Overall, the computation done for the paper in- cludes about 120 GPU-hours on an NVIDIA A100 GPU for fine-tuning and running inference on a LLaMA-7B model. Additionally, we have made several thousand API calls to OpenAI for infer- ence of GPT-3.5 and GPT-4, but it is not feasible to estimate the environmental cost of using these models due to the lack of public information on their hardware, architecture etc.
This work has only been tested on open-domain conversations. Generalizability of WikiChat to spe- cialized domains like medical or legal has not been evaluated.
This work has only been evaluated English conversations. Extending this work more lan- guages will be limited by the quality of available LLMs and retrieval systems in those languages. Incorporating recent work on multi-lingual mod- els (Scao et al., 2022) and information retrieval systems (Nair et al., 2022) is promising future direction.
Acknowledgements
This work is supported in part by the National Sci- ence Foundation, the Alfred P. Sloan Foundation, the Verdant Foundation, Microsoft Azure Al credit, KDDI, JPMorgan Chase, and the Stanford Human- Centered Artificial Intelligence (HAI) Institute. We also thank the reviewers for their valuable com- ments and suggestions.
Ethical Considerations
This study was approved by our institution's 's IRB. When conducting crowdsourcing for factuality, we compensate each worker for $0.2 per task, and we have one task per (system, dialogue turn, claim). This means that since there are more claims in longer chatbot responses, workers are compensated more for longer responses. In the end, each worker receives about $12 per hour of work. For our user study on the Prolific platform, each participant is compensated $2 to participate in a 10-minute user study, for compensation rate of S12 per hour of work. Prolific recommends a rate of $8-$16.
References
Yejin Bang, Samuel Cahyawijaya, Naycon Lee, Wen- liang Dai, Dan Su, Bryan Wilie, Holy Lovenia, Ziwei Ji, Tiezheng Yu, Willy Chung, Quyet V. Do, Yan Xu, and Pascale Fung. 2023, multitask, multilin- gual, multimodal evaluation of chatgpt on reasoning. hallucination, and interactivity.
Jianzhu Bao, Rui Wang, Yasheng Wang, Aixin Sun, Yitong Li, Fei Mi, and Ruifeng Xu. 2023. A synthetic data generation framework for grounded dialogues. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume I: Long Papers), pages 10866-10888 10882, Toronto, Canada. Association for Computational Linguistics.
When conducting the user study, participants provided with information on what the study en- tails and what information is collected (only their conversations, ratings, and the comments they pro- vide). They are asked not to share any personally identifiable information, and were able to contact
Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss

---

Gretchen Krueger, Tom Henighan. Rewon Child. Aditya Ramesh, Daniel M. Ziegler. Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Fric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. 2020. Language models are few-shot learners. In Ad- vances in Neural Information Processing Systems 33: Annual Conference on Neural Information Process- ing Systems 2020, NeurIPS 2020, December 6-12, 2020, virtual.
Luyu Gao, Zhuyun Dai, Panupong Pasupat, Anthony Chen, Arun Tejasvi Chaganty, Yicheng Fan, Vincent Zhao, Ni Lao, Hongrae Lee, Da-Cheng Juan, and Kelvin Guu. 2023. RARR: Researching and revising what language models say, using language models. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume I: Long Papers), pages (6477-16508, Toronto, Canada. Association for Computational Linguisties.
Ben Goodrich, Vinay Rao, Peter J. Liu, and Mohammad Salch. 2019. Assessing the factual accuracy of gen- crated text. Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining.
Jifan Chen, Aniruddh Sriram, Eunsol Choi, and Greg Durrett. 2022. Generating literal and implied sub- questions to fact-check complex claims. In Proceed- ings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 3495-3516, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.
Prakhar Gupta, Chien-Sheng Wu, Wenhao Liu, and Caiming Xiong. 2022. DialFact: A benchmark for fact-checking in dialogue. 60th Annual Meeting of the Association for Compu- tational Linguistics (Volume 1: Long Papers), pages 3785-3801, Dublin, Ireland. Association for Compu- tational Linguistics.
Proceedings of the
Moya Chen, Douwe Kiela, Mojtaba Komeili, Spencer Poff, Stephen Roller, Kurt Shuster, Arthur Szlam, Jason Weston, and Jing Xu. 2021. Blenderbot 2.0: An open source chatbot that builds long-term mem- ory and searches the internet. https://parl... ai/ projects/blenderoon May-2023.
Kelvin Guu, Kenton Lee, Zora Tung, Panupong Pasu- pat, and Ming- Wei Chang. 2020. Realm: Retrieval- augmented language model pre-training. In Proceed- ings of the 37th International Conference on Machine Learning, ICML'20. JMLR.org.
Online; accessed 20-
Cheng-Han Chiang and Hung yi Lee. 2023. Can large language models be an alternative to human evalua- tions?
Xingwei Hc, Zhenghao Lin, Yeyun Gong, A-Long Jin, Hang Zhang, Chen Lin, Jian Jiao, Siu Ming Yiu, Nan Duan, and Weizhu Chen. 2023. Annollm: Making large language models to be better crowdsourced annotators.
Tri Dao, Daniel Y. Fu, Stefano Ermon, Atri Rudra, and Christopher Ré. 2022. FlashAttention: Fast and memory-efficient exact attention with IO-awareness. In Advances in Neural Information Processing Sys- tems.
Ari Holtzman, Jan Buys, Li Du, Maxwell Forbes, and Yejin Choi. 2020. The curious case neural text degeneration. In 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020. OpenReview.net.
Nicola De Cao. Wilker Aziz, and Ivan Titov, 2021. Edit- ing factual knowledge in language models. In Pro- ceedings of the 2021 Conference on Empirical Meth-
ods in Natural Language Processing, pages 6491 6506. Online and Punta Cana. Dominican Republic. Association for Computational Linguisties.
Or Honovich, Leshem Choshen, Roee Aharoni, Filla Neeman, Idan Sypektor, and Omri Abend. 2021. knowledge-
q²: Evaluating factual consistency grounded dialogues via question generation and ques- tion answering. In Proceedings of the 2021 Confer- ence on Empirical Methods in Natural Language Pro- cessing, pages 7856-7870, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
Emily Dinan, Stephen Roller, Kurt Shuster, Angela Fan, Michael Auli, and Jason Weston. 2019. Wizard of wikipedia: Knowledge-powered conversational agents. In 7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019. OpenReview.net.
Krystal Hu. 2023. Chatgpt sets record for fastest- growing user base analyst note. Accessed: 10- 19-2023.
Nouha Dziri, Hannah Rashkin, Tal Linzen, and David Reitter. 2022. Evaluating attribution in dialogue sys- tems: The BEGIN benchmark. Transactions of the Association for Computational Linguistics, 10:1066. 1083.
HuggingFace. 2023. Text generation infer- ence. wwtgggeerraiionnnnneeeeeee
Sarah E. Finch, Ellie S. Pack, and Jinho D. Choi. 2023, Leveraging large language models for automated dia- logue analysis. In Proceedings of the 24th Meeting of the Special Interest Group Discourse and Dia- logue, pages 205, Prague, Czechia, Association for Computational Linguistics.
Gautier Izacard, Patrick Lewis, Maria Lomeli, Lueas Hosseini, Fabio Petroni, Timo Schick, Jane Dwivedi- Yu, Armand Joulin, Sebastian Riedel, and Edouard Grave. 2022. Atlas: Few-shot learning with retrieval augmented language models.

---

Joel Jang, Seonghyeon Ye. Changho Lee, Sohee Yang. Joongbo Shin, Janghoon Han, Gyeonghun Kim, and Minjoon Sco. 2022. TemporalWiki: A lifelong benchmark for training and evaluating ever-evolving language models. In Proceedings of the 2022 Con- ference Empirical Methody in Natural Language Processing, pages 6237-6250, Abu Dhabi, United Arab Emirates. Association for Computational Lin- guistics.
Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah Wiegreffe, Uri Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang, Scan Welleck, Bodhisatiwa Prasad Majumder, Shashank Gupta, Amir Yazdanbakhsh, and Peter Clark. 2023. Self-refine: Iterative refinement with self-feedhack.
Alex Mallen, Akari Asai, Victor Zhong, Rajarshi Das. Hannaneh Hajishirzi, and Daniel Khashabi. 2022. When not trust language models: Investigating effectiveness and limitations of parametric and non- parametric memories.
Zhengbao Jiang, Frank F. Xu, Luyu Gao, Zhiqing Sun, Qian Liu, Jane Dwivedi- Yu, Yiming Yang. Jamie Callan, and Graham Neubig. 2023. Active retrieval augmented generation.
Kevin Meng, David Bau, Alex Andonian, and Yonatan Belinkov. 2022. Locating and editing factual asso- ciations in GPT. Advances in Neural Information Processing Systems. 36.
Tom Koemi and Christian Federmann. 2023. Large language models are state-of-the-art evaluators of translation quality.
Sewon Min. Kalpesh Krishna, Xinxi Lyu, Mike Lewis, Wen tau Yih, Pang Wei Koh, Mohit lyyer. Luke Zettlemoyer, and Hannanch Hajishirzi. 2023. Factscore: rine-grained atomic evaluation of factual precision in long form text generation.
Mojtaba Komeili, Kurt Shuster, and Jason Weston. 2022. Internet-augmente au-gmettedd dialogue generation. In Proceed- ings of the 60th Annual Meeting of the Association for Computational Linguistics ( (Volume Long Pa- pers), pages 8460-8478, Dublin, Ireland. Association for Computational Linguistics.
Eric Mitchell, Charles Lin, Antoine Bosselut, Christo- pher D. Manning, and Chelsea Finn. 2022. Memory- based model editing at scale. In International Con- ference on Machine Learning. ICML 2022, 17-23 July 2022, Baltimore, Maryland, USA, volume 162 of Proceedings of Machine Learning Research, pages 15817-15831, PMLR.
Amrith Krishna, Sehastian Riedel, and Andreas Vlachos. 2022. ProoFVer: Natural logic theorem proving for fact verification. Transactions of the Association for Computational Linguistics, 10:1013-1030.
Patrick S. H. Lewis. Ethan Perez, Aleksandra Pik- tus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis. Wen-tau Yih, Tim Rocktäschel, Sebastian Riedel, and Douwe generation for inowledgg-intenniie NI.P tasks. In Advances in Neu- ral Information Processing Systems 33: Annual Con- ference on Neural Information Processing Systems 2020, NeurlPS 2020, December 6-12, 2020, virtual.
Suraj Nair, Eugene Yang, Dawn Lawrie, Kevin Duh, Paul McNamee, Kenton Murray, James Mayfield, and Douglas W. Oard. 2022. Transfer learning ap- proaches for building cross-language dense retrieval models. Advances in Information Retrieval, page 382-396.
Kiela. 2020.
Kishore Papineni, Salim Roukos, Todd Ward, and Wei- Jing Zhu. 2002. Bleu: a method for automatic evalu- ation of machine translation. In Proceedings of the 40th Annual Meeting of the Association for Compu- tational Linguistics, pages 311-318, Philadelphia. Pennsylvania, USA. Association for Computational Linguisties.
Huayang Li, Yixuan Su, Deng Cai, Yan Wang, and Lemao Liu. 2022. A survey on retrieval-augmented text generation.
Margaret Li, Jason Weston, and Stephen Roller. 2019. Acute-eval: Improved dialogue evaluation with opti- mized questions and multi-turn comparisons.
Fabio Petroni, Aleksandra Piktus. Angela Fan, Patrick Lewis, Majid Yazdani, Nicola De Cao, James Thorne, Yacine Jernite, Vladimir Karpukhin, Jean Maillard, Vassilis Plachouras, Tim Rocktäschel, and Sebastian Riedel. 2021. KILT: a benchmark for knowledge intensive language tasks. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 5523-2544 Online. Association for Computational Linguisties.
Nelson F. Liu, Tianyi Zhang, and Percy Liang. 2023a. Evaluating verifiahility in generative search engines. ArXiv:2304.09848.
Yang Liu, Dan Iter, Yichong Xu, Shuohang Wang, Ruochen Xu, and Chenguang Zhu. 2023b. G-eval: Nig evaluation using gpt-4 with better human align- ment.
Zhenghao Liu, Chenyan Xiong, Maosong Sun, and Zhiyuan Liu. 2020. Fine-grained fact verification with kernel graph attention network. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 7342-7351, On- line. Association for Computational Linguistics.
Prolific. 2023.
https://www.prolfffit
Acessed: June 2023.
Hongjin Qian, Zhicheng Dou, Jiejun Tan, Haonan Chen, Haoqi Gu, Ruofei Lai, Xinyu Zhang. Zhao Cao, and Ji-Rong Wen. 2023. Optimizing factual accuracy in

---

text generation through dynamic knowledge selec- tion.
Kurt Shuster, Jing Xu, Mojtaba Komeili, Da Ju. Eric Michael Smith, Stephen Roller, Megan Ung. Moya Chen, Kushal Arora, Joshua Lane, Morteza Behrooz, William Ngan, Spencer Poff, Naman Goyal, Arthur Szlam, Y-Lan Boureau, Melanie Kambadur, and Jason Weston. 2022h. Blenderbot 3: deployed conversational agent that continually learns respon- sibly engage.
Stephen Roller, Emily Dinan, Naman Goyal, Da Ju, Mary Williamson, Yinhan Liu, Jing Xu, Myle Out, Eric Michael Smith, Y-Lan Boureau, and Jason We- ston. 2021. Recipes for building an open-domain chatbot. In Proceedings of the 16th Conference of the European Chapter of the Association for Compu- tational Linguistics: Main Volume, pages 300-325, Online. Association for Computational Linguisties.
Charlie Snell. Dan Klein, and Ruiqi Zhong, 2022. Learning by distilling context.
Kai Sun, Yifan Ethan Xu, Hanwen Zha, Yue Liu, and Xin Luna Dong. 2023. Hcad-to-tail: How knowl- edgeable are large language models (llm)? a.k.a will Ilms replace knowledge graphs?
Keshav Santhanam, Omar Khattab, Christopher Potts, and Matei Zaharia. 2022a. Plaid: An efficient en- gine for late interaction retrieval. Proceedings of the st ACM International Conference on Information Knowledge Management.
Rohan Taori, Ishaan Gulrajani, Tianyi Zhang, Yann Dubois, Xuechen Li, Carlos Guestrin, Percy Liang, and Tatsunori B. Hashimoto. 2023, Stanford alpaca: An instruction followiinn llama model. https: // gi thub. cow ttatttttaa/attaaorrrarrr
Keshav Santhanam, Omar Khattab, Jon Saad-Falcon, Christopher Potts, and Matei Zaharia. 2022b. Col-
BERTv2: Effective and efficient retrieval via lightweight late interaction. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Hu- man Language Technologies, pages 3715-3734 Seat- tle. United States. Association for Computational Linguistics.
James Thorne, Andreas Vlachos, Christos Christoddoulooollss, and Arpit Mittal. 2018.
FEVER: large-scale dataset for fact extraction and VERification. In Proceedings of the 2018
Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume (Long Papers), pages 809-819, New Orleans, Louisiana. Association for Computational Linguistics.
Teven Le Scao, Angela Fan. Christopher Akiki, EI- lie Pavlick, Suzana Ilié, Daniel Hesslow, Roman Castagné, Alexandra Sasha Luccioni, François Yvon, Matthias Gallé, et al. 2022. Bloom: A 176b- parameter open-aceess multilingual language model. ArXiv preprint, abs/221 1.05 100.
Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, Aurelien Rodriguez, Armand Joulin, Edouard Grave, and Guillaume Lample. 2023. Llama: Open and efficient foundation language models.
Jost Schatzmann, Kallirroi Georgila, and Steve Young. 2005. Quantitative evaluation of user simulation tech- niques for spoken dialogue systems. Proceedings of the 6th SIGdial Workshop on Discourse and Dia- logue, pages 45-54, Lisbon, Portugal. Special Inter- est Group on Discourse and Dialogue (SIGdial).
Harsh Trivedi, Niranjan Balasubramanian, Tushar Khot, and Ashish Sabharwal. 2023. Interleaving retrieval reasoning for knowledge-
with chain-of-thouge intensive multi-step questions. In Proceedings of the 61 Annual Meeting of the Association for Com- putational Linguistics (Volume 1: Long Papers), pages 10014-10037, Toronto, Canada, Association for Computational Linguistics.
Weijia Shi, Sewon Min, Michihiro Yasunaga, Minjoon Seo, Rich James, Mike Lewis, Luke Zettlemoyer, and Wen tau Yih. 2023. Replug: Retrieval -ugmented black-box language models.
Kurt Shuster, Mojtaba Komeili, Leonard Adolphs, Stephen Roller, Arthur Szlam, and Jason Weston. 2022a. Language models that seek for knowl- edge: Modular search generation for dialogue and prompt completion. In Findings of the Association for Computational Linguistics: EMNLP 2022, pages 373-393, Abu Dhabi, United Arab Emirates. Associ- ation for Computational Linguistics.
Dazhen Wan, Zheng Zhang, Qi Zhu, Lizi Liao, and Min- lie Huang. 2022. A unified dialogue user simulator for few-shot data augmentation. In Findings of the Association for Computational Linguistics: EMNLP 2022, pages 3788-3799, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Ed H. Chi, Quoc Le, and Denny Zhou, 2022. Chain of thought prompting elicits reasoning in large language models. ArXiv preprint, abs/2201.119093
Kurt Shuster, Spencer Poff. Moya Chen, Douwe Kiela, and Jason Weston. 2021. Retrieval augmentation reduces hallucination in conversation. In Findings of the Association for Computational Linguistics: EMNLP 2021, pages 3784-3803, Punta Cana, Do- minican Republic. Association for Computational Linguistics.
Susan Zhang, Stephen Roller, Naman Goyal, Mikel Artetxe, Moya Chen, Shuohui Chen, Christopher De- wan, Mona Diab, Xian Li, Xi Victoria Lin, Todor Mi- haylov, Myle Ott, Sam Shleifer, Kurt Shuster, Daniel

---

Simig, Punit Singh Koura, Anjali Sridhar, Tianlu Wang, and Luke Zettlemoyer. 2022. Opt: Open pre- trained transformer language models.
Zihan Zhang, Meng Fang, Ling Chen, Mohammad-Res Namazi-Ran and Jun Wang. 2023, How do large language models capture the ever-changing world knowledge? review of recent advances.
Ruochen Zhao, Xingxuan Li, Shafiq Joty. Chengwei Qin, and Lidong Bing. 2023. Verify-and-edite knowledge-enhanced chain-of-thought framework.
Chujic Zheng, Sahand Sabour, Jiaxin Wen, Zheng Zhang, and Minlie Huang. 2023. AugESC: Dialogue augmentation with large language models for emo- tional support conversation. In Findings of the As- sociation for Computational Linguistics: ACL 2023, pages 1552-1568, Toronto, Canada. Association for Computational Linguistics.
Qi Zhu, Zheng Zhang, Yan Fang, Xiang Li, Ryuichi Takanobu, Jinchao Li, Baolin Peng. Jianfeng Gao, Xiaoyan Zhu, and Minlie Huang. 2020. ConvLaab-2: An open-source toolkit for building, evaluating, and diagnosing dialogue systems. In Proceedings of the 58th Annual Meeting of the Association for Compu- tational Linguistics: System Demonstrations, pages 142-149, 149, Online. Association for Computational Lin- guistics.
Example Conversations
Tables 4, 5, 6, and provide examples of conver- sations between bots, the same topic: the 2023 Australian Open Men's singles 4. Tables and show two con- versations between human users and GPT-4 and WikiChat G4, from our user study. The simulations were done using Wikipedia dump on 4/28/2023. The user study was conducted on 6/17/2023 using a Wikipedia dump of the same date.
simulator and four chat-
'https://en http://wweeeaa..r/iii//////////e

---

Can you tell me about the 2023 Australian Open Men's singles?
User:
User: Can you tell me about the 2023 Australian Open Men's singles final?
WikiChat the 2023 Australian Open Men's singles, Novak Djokovic won the title, defeating Stefanos Tsitsipas with a score of 6-3, 7-6(4), 7-6(5). This victory marked Djokovic's record- extending tenth Australian Open title and ma- jor title overall. He became the second man to win ten titles single major, after Rafael Nadal the French Open.
WikiChat o4: In the thrilling 2023 Australian Open Men's singles final, Novak Djokovic defeated Stefanos Tsitsipas with score of 6-3, 7-6(4), 7- 6(5). This amazing victory marked Djokovic's record-extending tenth Australian Open title and tied Rufael Nadal's all-time record of 22 men's sin- gles titles. Djokovic also became the second man to win len titles at single major, joining Nadal at the l'rench Open. With this win, he surpassed Nadal's tally for the fourth-most ATP Tour-Ievel singles titles.
User: That's impressive! How did Djokovic per- form throughout the tournament?
WikiChat,. Throughout the 2023 Australian Open, Novak Djokovic performed exceptionally well. He defeated opponents in straight sets and only dropped one set throughout the entire tourna- ment. the top seed, he defeated Sebastian Korda in the 2023 Adelaide International.
User: That's amazing! How did Djokovic perform throughout the toumament leading up to the final? WikiChat c4: Novak Djokovic had impres- sive performance throughout the 2023 Australian Open. Leading up to the final, he dropped only set, showcasing his dominance and skill the court. His exceptional performance culminated his straight-sets sets victory over Stefanos Tsitsipas in the final.
User: Wow, Djokovic's performance was indeed remarkable. What about Stefanos Tsitsipus, how did fare tournament before final? WikiChat L: Stefanos Tsitsipas had an impressive run the 2023 Australian Open before the final. defeated Quentin Halys, Rinky Hijikata, Tallon Griekspoor, Jannik Sinner, Jiri Lehecka, and Karen Khachanov his way to the final, This was his first Australian Open men's singles final.
User: Wow, that's quite achievement! On the other hand, how Tsstsipas perform during the tournament before facing Djokovic in the final? WikiChat c4: Stefanos Tsitsipas had strong per- formance during 2023 Australian Open before facing Djokovic in the final. He defeated Quentin Halys, Rinky Hijikata, Tallon Griekspoor, Jannik Sinner, Jiri Lehecka, and Karen Khachanov his way to the final. This was second Grand Slam final, following the 2021 French Open. In both finals, Tsitsipas lost Novak Djokovic.
User: It's great to see young players like Tsitsipas making it to the finals of major tournaments. How has Tsitsipas's 's overall career been far? WikiChat, e Stefanos Tsitsipus has had an impres- sive career so far. He has won nine AIP singles titles including two Masters 1000 championships. and has contested
major finals the 2021 French Open and 2023 Australian Open, finish- ing rumner-up Novak Djokovic buth times. His career-high doubles ranking is No. 64, achieved on August 2022.
User: It's great see young player like Tsitsipas reaching the finals of Grand Slam tournaments. Are there any memorable moments highlights from their match the final stood out? WikiChat c4: The final was a thrilling match. Some memorable moments from the match in- cluded Djokovic reclaiming the world No. rank- ing from Carlos Alcaruz.
User: It seems like Tsitsipas has a bright future ahead of him. Can you tell me more ahout his playing style and strengths the court?
WikiChat,, Stefamos Tsitsipas is an aggressive baseliner with strong all-court player style. He aims hit powerful groundstroke winners, with a particularly strong forehand using an Fastern grip. His forehand is particularly useful in finish- ing points off of the first serve, combined with his big serve, uses his high topspin rate pull opponents off the court for winners, or go to the net more often than typical baseline players.
User: That's awesome! Speaking of Carlos Al- caraz, how did he perform in the 2023 Australian Open? Did make far the tournament?
WikiChat ca: Carlos Alcaraz did not participate in the 2023 Australian Open. He withdrew from the tournament due hamstring injury in his right leg that suffered while training.
Table 4: A conversation between WikiChat G4 and the simulated user. There are no factual errors in this con- versation.
Table A conversation between WikiChat,, and the simulated user. There sation. However, the second chatbot utterance contains an irrelevant piece of information.
factual errors in this conver-

---

User: Can you tell about the 2023 Australian Open Men's singles final?
GPT-4: The 2023 Australian Open Men's singles final was an exciting match between Player A and Player B, Player emerging as the champion after a hard-four fought battle.
User: Oh, think the 2023 Australian Open Men's singles final between Novak Djokovic and Ste- fanos Tsstsipas.
User: How excited should he ahout indiana jones and the dial of destiny
GPT-4: Ah, see! The 2023 Australian Open Men's singles final featured Novak Djokovic de- feating Stefanos Tsitsipas thrilling match claim the title.
WikiChat "Indiana Jones and the Dial of Des- tiny" world premiere the 76th Cannes Film Festival on May 18, 2023, and received a five- minute standing ovation. The film is scheduled theatrically released the United States on June 30, 2023, Given the positive reception at its premiere, you have guod reason to be excited about this final installment the Indiana Jones series. User: hear that has more a political agenda than the other movies
User: What was the final score of the match be-
tween Djokovic and Tsitsipas?
GPT-4: The final score the match hetween Djokovic and Tsitsipas was 6-,, 6-2, 6-3, with Djokovic securing the victory.
User: Actually, the final score was 6-3, 7-6(4), 7- 6(5). was a thrilling match, wasn'l it? GPT-4: Yes, my apologies for the emror! The fi- nal score 6-3, 7-6(4). 7-6(5) indeed made it thrilling match between Djokovic and Tsitsipas. User: No problem! How many Australian Open titles has Djokovic
WikiChat haven't 't heard about that aspect, it's possible. It's always interesting to see how films incorporate diffterent themes and messages. User: Will there be another indiana jones after this WikiChat c4: "Indiana Jones and the Dial of Des- tiny" is the final installment in the series, so there are plans for another Indiana Jones movie after this one.
with this victory? GPT-4: With this victory. Novak Djokovic has won total of 11 Australian Open titles
Table 6: A conversation between GPT-4 and the simu- lated user. Factual errors are highlighted in yellow, Note that user simulator has access first sentence of the Wikipedia article, and here it tries correct the chatbot twice.
User: Will harrison ing movies in general
doing any more incom-
WikiChat G4: Yes, Harrison Ford will be doing more He
selected by Marvel dios to replace the late William Hurt as Thaddeus "Thunderbult" Ross in the upcoming 2024 films "Captain America: Brave New World" and "Thun- derbolts" set in the Marvel Cinematic Universe. User: Any other big names coming to the MCU in the future?
User: just heard about the 2023 Australian Open Men's singles final. Can you tell more Atlas: It was held in Melhourne, Australia.
WikiChat Gat
hig actors joining the MCU in the future include Shea Couleé, Zoe Terakes, Regan Aliyah, Shakira Barrrr. Rashida "Sheedz" Olayiwola, Sonia Denis, Paul Calderón, and Cree Summer who have joined the cast "Tronheart" in undisclosed roles. Sacha Baron Cohen has also joined the MCU, with role that would potentially first appear the later episodes "Ironheart" fol- lowed by appearances in other MCU projects.
User: That's great! Can you tell me more about match between Novak Djokovic and Stefanos Tsitsipas?
Atlas: Novak Djokovic won his tenth Australian
Open title and 22nd major title overall.
User: Wow, that's impressive! How did the match play out? What were the scores?
Atlas: It was a three-set match.
Table 8: A conversation between WikiChat of and human user. There are no factual errors in this conversa- tion.
User: Yes, heard was three-set match. Can
you tell me the scores of each set?
Atlas: The first set was 6-3, 7-6(4), 7-6(5).
User: That's interesting! seems like the last
sets were quite close. How did Djokovic manage to come out on top in those tight sets?
Atlas: He the third tiebreak.
Table 7: conversation between Atlas and the simu- lated user. There factual errors in this conversa- tion, but the chatbot scores low in conversationality

---

User: Hi there. I'm interested learning about the 2023 movie Reality HBO.
GPT-4: "Realit" is thrilling 2023 movie
HBO, exploring blurred lines between virtual and real worlds, featuring an immersive plot and a talented cast.
much about it, but do like the lead Sydney Sweeney. GPT-4: The plot follows Sydney Sweeney's char- she navigates virtual reality game, only to discover sinister secrets dangerous conspir- acy that threatens both the virtual and real worlds, User: Interesting. thought was about her being whistleblower some kind of political conspir- auy? Or is that all part of the game?
User: What's the plot about?
GPT-4: Yes, that's correct! As whistleblower the game, she uncovers political conspiracy, which adds depth and intrigue to the story, blending both virtual and woal-world elements. User: That dues sound exciting. What about the rest the cast, who are the other leads?
GPT-4: The other leads include Tom Holland, Tessa Thompson, and Rami Malek. who de- liver captivating performances alongside Sydney Sweeney, making for dynamic and engaging en- semble.
User: Oh cool, I love Tom Holland and Rami Malek. What else has Rami Malek been in re- cently? feel like haven't seen him in anything in little while.
GPT-4: Rami Malek recently starred the 2021 film "No Time to Die" as the villain Safin, and he's also known for his award-winning role as Freddie Mercury "Bohemian Rhapsody" (2018).
Table 9; A conversation between GPT-4 and human user. Factual errors are highlighted in yellow.

---

B Experiment Details
Distillation to LLaMA. When fine-tuning LLaMA-7B in the distillation experiments, we use hyperparameters from (Taori et al., 2023), namely learning rate of 2 10 cosine learning rate sched- ule, batch size of I 128, and training for epochs. Initial experiments with LLaMA-30D showed no significant improvements. Training is done on 4 NVIDIA A100 (80 GB) GPUs.
Statistical significance tests For statistical sig- nificance tests throughout the paper, we use inde- pendent two-sample t-test and consider the differ- ence significant if < 0.05.
Simulation Topics. We obtain the number of vis- its and edits of each Wikipedia article using the Wikimedia S.
Automatic Evaluation. In order to verify that using GPT-4 for conversationality metrics (Section 5.2) is indeed reasonable, we compare its scores against two authors of this paper. Table 10 shows inter-annotator agreement for ratings 50 ran- domly sampled conversation turns from all chat- bots, measured by caleulating Cohen's Kappa. The annotators are given the same instructions that is given to GPT-4 as prompt.
As mentioned in the paper, to select the recent topics, we look at the most edited Wikipedia ar- ticles in the first four months of 2023. Filtering based on the creation date did not lead to mean- ingful articles as there are many articles about old topics that just received articles in Wikipedia. In- stead, in our experience, most of the highly edited Wikipedia articles are about actual new topics.
An Alternative to WikiChat's Verification Stage. For the verification stage (Stage 5), initially perimented with two approaches for verification: the Kernel Graph Attention Network (KGAT) veri- fier (Liu et al., 2020) and a few-shot prompt-based verifier with chain-of-though prompting (Wei et al., 2022). KGAT is a model specifically designed for fact-checking and fine-tuned on the FEVER dataset (Thorne et al., 2018).
perraaarssesss. We use temperature of 0 and greedy decoding for all experiments. except the user simulator which has temparature of 1.0 and nucleus sampling (Holtzman et al., 2020) with p=0.5. We use no repetition penalty, except for a repetition penalty of 1 for the baseline L.LaMA model, because we find that repetition penalty of 1.0 (i.e. no penalty) results in the model frequently degenerating (Holtzman et al., 2020) repetitions.
While KGAT performs effectively for FEVER- style fact verification tasks, we found its perfor- mance lacking in our setting. FEVER claims are de- rived from edited Wikipedia sentences, leading to spurious correlations that do not exist when claims come from chatbots. In addition, we were able to incorporate user utterances and conversation his- tory as context in the few-shot verifier, while KGAT only looks at the claim and the evidence. Hence, decided to conduct our experiments using the prompt-basce verifier.
In most prompts of WikiChat, include at most the last five turns of the dialogue history reduce the chance of causing confusion for the few-shot models in longer conversations.
Baseline Atlas. We use the 3B 18-parameter Atlas- XL and update its index the same Wikipedia index as WikiChat for a fair comparison. We re- produce their best model on Wizard of Wikipedia, which is obtained by fine-tuning the Atlas pre- trained model and its retriever using the train set, except that we update their index to the same Wikipedia dump WikiChat. For this, use the fine-tuning code in the Atlas code repository and set learning rate 4e-5, dropout to 0, weight decay to 0.01, and retriever number of contexts to 40. We use a target maximum length of 64 in- stead of 16, to accommodate longer outputs. After this, the resulting model matches the Wizard of Wikipedia validation score reported in Izacard et al. (2022).
C Analysis of WikiChat
C.1 Saying "I don't know".
As mentioned carlier, WikiChat concedes that it does not know when none of the LLM-generated passes fact-checking and no relevant bullet points are retrieved. Table 11 contains the data on how fre- quently this happens. In this case, the draft prompt is skipped and instead "Sorry, I'm not sure" is sent to the refinement prompt, which dresses it up to match the conversation. For example:
'Accessed via httpt///////////aaarr/r///////// httpr///////ttttttooo/////eeeeer bttb://ahheeaaaple..eiiisss////a
User: "Are there any specific ethical dilemmas the characters face in the [M3GAN] film?"

---

Relevant
Inform.
Natural
Non-Rep.
k between author and author 2
0.41
0.61
0.28
0.48
K. between author and GPT-4
0.45
0.35
0.11
0.38
K. between author and GPT-4
0.67
0.60
0.49
0.41
Table 10: Cohen's ' Kappa between different annotators for conversationality metries.
WikiChat: "Yes, the movie raises ethical ques- tions about AI, but I don't want to spoil the plot by revealing specific dilemmas. You'll 'Il have to watch the film to find out!"
Head
Tail
Recent
All
WikiChat G4
4.4
3.1
3.4
3.6
GPT-4
2.8
2.6
2.2
2.5
WikiChat G3.5
4.2
3.2
3.2
3.5
GPT-3.5
2.6
2.1
1.9
2.2
Head
Tail
Recent
All
WikiChat
4.0
3.0
3.1
3.3
WikiChat G4
1.1
19.0
18.0
12.7
LLaMA
2.1
2.0
2.0
2.0
WikiChat Ga.s
0.0
13.0
8.0
7.0
Atlas
1.4
1,3
1.5
1.4
WikiChat
1.0
22.0
20.0
14.3
Table 13: The average number claims per turn for each subset and chatbot.
Table Il: Pencentage of turns in which WikiChat does not find relevant information Wikipedia retrieve or fact-check.
Head
Tail
Recent
All
WikiChat G4
84.1
66.0
69.7
73.3
C.2 Number of claims per turn
WikiChat G3.5
88.9
76.9
80.7
82.2
WikiChat 77.1
66.1
66.4
69.9
IR
L.M
Verified
Table 14: Analysis of WikiChat's response refinement. BLEU score with the refined response the prediction and the response before refinement as the target.
Head
4.3
2.8
80.1%
Tail
3.6
2.2
55.4%
WikiChat
Recent
4.0
2.1
43.8%
All
4.0
2.4
61.7%
Head
4.3
87.6%
fluent in English. Each participant paid with the rate of $12 per hour. Figure shows the user interface.
Tail
2.6
2.2
57.7%
WikiChat
Recent
3.1
2.0
54.0%
All
3.3
2.3
68.0%
Head
4.7
84.7%
Figure 3 shows the distribution of the user ratings from Table 3.
Tail
3.3
2.2
46.0%
WikiChat
Recent
3.8
1,9
35.6%
All
3.9
2.2
57.5%
E Human Evaluation
Table The average number of relevant bullet points that WikiChat obtains from information retrieval and LLM-generated responses, and the percentage of claims that pass the fact-checking stage.
We conduct human evaluation for part of our evalu- ation of factual accuracy (as described Section 5.2). We use the Scale Rapid pplaffom. Figure 4 shows the instruction and one of the examples we provide. Figure shows the user interface for each annotation task. We present the human annotator with the last user utterance, the chatbot's response. and a claim extracted from the chatbot's response using GPT-4. The annotator is then tasked with reading the evidence passages and determining whether the claim is correct, incorrect, or if there is insufficient information to verify the claim. We use three-way way consensus pipeline, where each claim is assessed by three graders independently. and the final label is determined based on the majority vote.
Table 12 contains the raw data on the contribu- tion of IR LLM, and Table 13 contains the raw data on the number of claims, both mentioned in Section 7.5.
C.3 Refinement improvements
Tables 14 and 15 have the raw data the refine- ment stage, used in Section 7.5.
D User Study
We use Prolific to conduct our user study. We select 40 participants (20 female) from the US who are
www. www.scale. scale.com

---

Relevant
Inform.
Natural
Non-Rep.
Temporal
Head
0.2
0.3
0.2
0.0
0.0 %
Tail
0.3
0.4
0.4
0.2
1.0 %
WikiChat G4
Recent
0.2
0.3
0.4
0.1
6.0
All
0.2
0.3
0.3
0.1
2.3 %
Head
0.0
0.0
0.1
0.0
2.0 %
Tail
0.1
0.3
0.3
0.0
5.0 %
WikiChat G3.5
Recent
0.1
0.1
0.2
0.1
3.0 %
All
0.1
0.1
0.2
0.1
3.3 %
Head
0.0
0.0
0.0
0.0
0.0 %
Tail
0.2
0.2
0.3
0.1
3.0
WikiChat L
Recent
0.2
0.3
0.4
0.0
6.0 %
All
0.1
0.2
0.2
0.1
3.0 %
Table 15: Improvements automatic conversationality metrics made by refinement.
One author periodically audits their work, provid- ing feedback and adding examples and tests for crowdworkers needed.
ieeee cereette takee
We provide annotators with detailed instructions on the task, and 8 examples covering special cases. We provide 7 training tasks used for onboarding, and 22 evaluation tasks. Only crowdworkers who receive score of 90% in this evaluation can move to the main task. We compensate each worker for $0.2 per task, and we have task per (system, dialogue turn, claim). This means that since there are more claims longer chatbot responses, work- ers are compensated more for longer responses. the end, each worker receives about $12 per hour of work.
Elle was lare, you
the
2913 Apsrellle Eore?
The 580 custractas dpen took clace an helssume fare
awe vogges sirt, tyre
terruse frem provies
Free
99.00 this reale your oplaton?
and 25
Eatrtt
Figure 2: Screenshot of the interface shown to partici- pants in the user study, after one turn of conversation.
F All Prompts
We provide the prompts mentioned in this paper. For brevity, we only show the few-shot exam- ples used in each prompt. The full text of prompts can be obtained from our code repository. The syntax used is the Jinja2 template language, which supports Python- loops ([% for xX% endfor x}), conditions ((% if x)(% endif %)), vari- ables (El var }}) and comments ((# #3). In all prompts, dlg is python list, today string like 4/28/2023, current_yea is string like 2023, and location set to U.S.
5
4.5
4
3.5
3
2.5
2
1.5
1
WikiChat GPT-4
Figure 3: User ratings for WikiChat and GPT-4.

---

Summary
We want fact-check the sutpurs chatbot, when tarking to user
Your task is to fact-chock the sentence high lahsed in blue claam evaluate") made the chalber.
Fact- chockin should
paragrashs response your reference on and should NOT be fact
Wiklpodio.
user quastion
Workflow
the saked [User]] the chatbot reptied (Chatbot:).
Carefully highighhed
ep title and content Whippeia paregraphe providedt rwattd the avaluate" and evaluate). sente sce you
some may be unrebeled.
Find specific evicence from one more of these peregraphs determine whether the ccain unter is factusiy correct paragraphs,
enough information" option.
Fact Check The Claim
options:
Select "This CORRECT according paragraphs." readiig
paragraphs potentially
plecing together informetion on from multipee paragraphs, you fiud information that sur catts at this stam
Select "This claim NOT CORRECT according to these parographs," by reading the provided paragraphs potentia ty find information contradicts
togother information
abeva options are appropriste. This means you could not findt information adiresssin all or part of the clsim in the providbed enregraaiis
Selaect "There ENOUGH INFORMATION these paragraphs to verity claim.
EXAMPLES
Sometimen mes it necessary to combinne information from multiple passages.
Australian
"Metboume
that TMelboume Park located in "Melboure,, au So togetherr, this incicates that the claim complete y correct.
corringg
cutting
wou thete to about
eould obeut the
toee suorcie ynu have
temeou che van dey you toree
pooote 2023
Djsicuic
ainglas
rnvsummment meternaninn the una
Connect
O Incomrect
Figure 4: Screenshot of the instructions and one of the examples we provide to crowdworkers for evaluation of factuality.
Fach-check the chile
would the
Ronadoo Coe you more
Polas 1aa canilaty
on chamss Criesano Ronalo dos Sentos Avelro a Portag vere professions footaaee who play as fonward for coptains graneet Mkayers of at having five Bullon awards and Eurapean dueeen Stoees, by Europeor playsr He Citampll plon and the USAA nattono Loogue, Hei the all time leadng soorer in the USR, Chomplons Looque ond goste.
prodded Konn
Laague
and Porugar
widel reganded
is CORRECT acconding these
leegue
Champioes
acoording
Tione funo ENUUUCH na DEBANT withese seneg sutte to venty cams.
Crlestono Ronalioo moophie
titees USFA Champtons
Languee, European Championahip.
Lmaguei
Wilipecle paosages for tack checidng:
Content: Pes.agaoe ton Boor 7905) Drecleno Hor Morelto dos Santos dearo boon's Fnterary 7305) a Porlggeee
Figure 5: Screenshot of each task crowdworkers see. For each claim, we provide them with 5 paragraphs from Wikipedia.

---

The current year is {f current_year }.. The following is a conversation be bween
and chatbot the topic of "([ title )}" (({ passage }))
Do not assume that the chatbot is able to have physical experiences, like watching movie.
Ask interesting follow-up questions when needed, and expand on the chatbot
responses using your life experiences:
Never volunteer information, and never correct chatbot's mistakes.
Continue the conversation for 15 turns. (n This is set to 15, even though simulations end after 10 turns. If set this to 10 turns, the simulator will start saying goodbye too early. #)
The first two turns don't 't have any content and 't be sent the Chatbot. They just meant specify the format. #}
You: Hi!
Chatbot: Hi, how
assist you today?
{% for dug-turn in dlg x}
You: {f alg_turn, user.uuer_uuee user_utteraccee Chatbot: {( dlg_turn. agent_utterance }}
{% endfor x)
You:
Table 16: User simulator prompt. This prompt is zero-shot. title the title of the Wikipedia page used for simulation, and passage is the first sentence of that article
You are chatting with a user. Use Google search to form response. You are both located in [[ location }}. Today 's date is [( today }). what do you type in the search box?
what date do you want the search results to be? Enter "recent" if you are looking for the newest results. Enter "none" if the date is not important.
(# Few-shot example 1 #)
You: Do you want to talk about sports?
User: Sure! Who is your favorite basketball player?
[Search needed? Yes. You Google "popular basketball players". The year of the results is "none".]
You: It has to be Lebron James.
User: Did he play well in his last game?
[Search needed? Yes. You Google "how did Lebron James do in his most recent game". The year of the results is "recent".]
(# More few-shot examples #}
(# The current dialogue it} t for alg_turn in dlg %)
(x if dlg_turn. user_utterance not x) User:
}}
endif x)
(x if inn.-tuunnnniiialssee [Search needed? Yes. You Google alg.ttrn.iiitial_ceae }}". }}"_]
none
The
year of the results "[( dlg_turn.
(x endif %}
if Elg_turn.co agent_utterance is none You: {( dlg_turn. agent_utterance }}
(x endif x} (8 endfor x) User: [( new_user_utterane }} [Search needed?
Table 17: Query generation prompt in WikiChat (Stage 1). This prompt has 6 few-shot examples.

---

You Google different search queries and then Break down the relevant parts of the articles you find. Today's date is {( today }}.
(# Few-shot example a)
Query: "worst earthquake ever"
Title: January 1934 earthquake India and Nepal
Article: The 1934 Nepal\u2013Indiz earthquake or 1934 Bihar\u2013Nepta ear was one of the worst earthquakes in India's history. The towns of Munger and Muzaffarpur were completely destroyed. This 8.0 magnitude earthquake occurred on 15 January 1934 at around 2:13\00eaepm IST (08:43 UTC) and caused widespread damage in northern Bihar and in Nepal. Earthquake. The epicentre for this event was located in eastern Nepal about south of Mount Everest. The areas where the most damage to life and property occurred extended from Purnea in the east to Champaran in the west (a distance of nearly ) and from Kathmandu in the north to Munger in the south (a distance of nearly )
Break down verbatum (s)t(s) of this article that are related to the search query "worst earthquake ever" or say None if the article is unrelated:
The 1934 Nepal-India earthquake, also known as the 1934 Bihar-Nepal earthquake, was one of the worst earthquakes in India's history.
The 1934 Nepal-India earthquake had magnitude of 8.0 and occurred on 15 January 1934.
As result of the 1934 Nepal-Indin earthquake, the towns of Munger and Muzaffarpur were completely destroyed. As result of the 1934 Nepal-India earthquake, widespread damage occurred in northern Bihar and Nepal, with the most damage extending from Purnea in the to Champaran in the west, and from Kathmandu the north to Munger in the south.
Few-shot example Query: "age Bruce Willis" Title: Matt Willis Article: April 2005, aged 21, Willis stayed three weeks London 's Priory Hospital for the treatment of alcoholism. admitted again for a few days cannabis from
July 2006, aged 23, he was drug abuse, because he was addicted age 13. He began to have problems from the drug-taking including physiological and memory problems. In June 2008, aged 25, Willis entered a rehab centre reported that a night out with close friend Amy Winehouse pushed Willis too far. Willis took the full five week course in drugs and alcohol.
Bournemouth after a marriage ultimatum
was
Break down verbatum (s) this article that are related to the search query 'age of Bruce willis" or say None if the article is unrelated:
None
(# More few-shot examples #)
(# The current dialogue #)
Query:
"[( query }}"
Title: {[ title }}
Article: {[ article }}
Break down verbatum (s) of this article that are related to the search query "(I query }}" or say None if the article is unrelated:
Table 18: Summarize and filter prompt of WikiChat (Stage 2). This prompt has 7 few-shot examples.

---

You are a friendly, knowledgeable and truthfull chatbot, talking to a user. Respond in at most one paragraph.
Today's 's date is (f today }}.
{N for dlg_ turn in dlg %)
User: [{ dlg_turn. user_utterance }} You: {( wew.uumm.gent.uttraannc.... }}
{8 endfor X)
User: {( new_user_utterance }}
You:
Table 19: The prompt for baseline chathots, also used for Stage 3 of WikiChat. This prompt zero-shot.
are chatting with a user. Rewrite the facts mentioned the last response into self-contained sentences. Exclude opinions, or subjective statements.
Today't date is [{ today }}.
(# Few-shot example #)
User: When was chinese new year this year?
You: It was on February 5th.
Rewrite the facts mentioned
the last response into self-contained sentences:
Chinese New Year in 2022 was on February 5th. The year of the results is "2022".
(# Few-shot example 2 4)
User: How are you doing today?
You: As an artificial intelligence, I don't have feelings or personal experiences, so I don't 't have good days or bad days. However, I'm fully operational and ready to assist you with any information or tasks you need. What can I help you with today?
Rewrite the facts mentioned in the last response into self sentences: Nothing.
(# Few-shot example 4)
User: Do you know about Queen Elizabeth II? I was just reading about her. You: Queen Elizabeth II is the current monarch of the United Kingdom and the Commonwealth realms She was born in 1926 and became queen in 1952.
Rewrite the facts mentianed in the last response into self-contained sentences:
Queen Elizabeth II is the current monarch of the United Kingdom and the Commonwealth realms of {{ today }}. The year of the results is "recent".
Queen Elizabeth II born in 1926. The year of the results is "1926".
Queen Elizabeth II became queen in 1952. The year of the results is "1952".
More few-shot examples #)
The current dialogue #)
{ for dlg_turn in dlg[-2:] conversations confuse LLM, and are not needed fact-checking. ex if dlg_turn. user -stterneee not none %}
Only last few
given, because longer
User: {f ull.turm.user.uuteraan (x endif x}
}}
(x alg_turn. agent_utterance is not none x)
You: {( dig_turn. agent_ut }}
(x endif x}
(8 endfor x)
User: {( new_user_utterance }}
You: {f perrent_agen_uttee
}}
Rewrite the facts mentianed the last response into self-contained sentences:
Table 20: Claim extraction prompt of WikiChat (Stage 4), This prompt has few-shot examples.

---

The following is conversation between user and chatbot. For each claim that the chatbot makes, search the internet to obtain ar articles that would support refute that claim, and output one of "SUPPORTS", "REFUTES", "NOT ENOUGH INFO".
Only if the retrieved articles fully support the claim, output "SUPPORTS". Today 's date is { today }}.
(# Few-shot example #) Chatbot: How your trip to Hawali? User: It great! In fact, witnessed the eruption of the largest volcano it, but sounds kinda dangerous. Is it the Mauna Yes, it is! Do you know when it started erupting?
earth.
Chatbot: Wow, hope could Loa?
User:
Chatbot: Yes,
started erupting March 25, 1984. [You search the internet
fact-check the claim "The last eruption of Mauna started on March 25, 1984"]
[You get these articles:
Title: 2022 eruption
Mauna Loa
Article: when active, Mauna Loa tends produce "voluminous, fast-moving lava flows" of the Hawaiian or effusive eruption type rather than more explosive phreatic or Plinian eruptions, though has produced explosive eruptions between 300 and 1,000 years ago. Before Nov 27, 2022, Mauna Loa had last erupted March 1984, in a 22-day event similarly concentrated in the volcano Northeast Rift Zone. The 2022 eruption was the volcano 's 34th eruption since 1843, when volcanic activity at Mauna Loa began to be continuousl, recorded, but only the third eruption since 1950. The 38-year span between the 1984 and 2022 eruptions was Mauna Loa's 's longest period of quiescence on record.
Title: 1984 eruption of Mauna Loa
Article: The 1984 eruption of Mauna Loa was Hawalian eruption in the U.S. state of Hawaii that lasted from March 25 to April 15, 1984. It ended a 9-year period of quiescence at the volcano and continued for 22 days, during which time lava flows and lava fountains issued from the summit caldera and fissures along the northeas, and southwest rift zones. Although the lava threatened Hilo, the flow stopped before reaching the outskirts of town.
]
Fact-check the claim "The last eruption of Mauna Loa started March 25, 1984". You think step by step: Mauna Loa had an eruption Nov 27, 2022, which is later than the claimed last eruption of March 25, 1984. So the last eruption of Mauna Loa was not March 25, 1984. So the fact-checking result is "REFUTES".
More few-shot examples
(# The current dialogue #)
Chatbot: original_reply }}
[You search the internet to fact-check the claim "C[ claim }}"]
[You get these articles:
{% for title in evidence_titles x)
Title: C[ title }}
Article: {( wwidence_texts[IIe index -1]}3
{% endfor x)
Fact-check the claim "([ claim } think step by step:
Table 21: Verification prompt of WikiChat (Stage 5). This prompt has few-shot examples.

---

The Following is a conversation between a friendly, knowledgeable and truthful chatbot, called WikiChat, and a user.
WikiChat can use search to get external knowledge when needed. In responding to the user, WikiChat carefully aggregates search results in at most one paragraph. Today's 's date is ({ today }}.
(# Few-shot example #)
User: What new discoveries from James Webb Space telescope can I tell my 9 year old about?
WikiChat: Your 9-year old would be fascinated to hear that the James Webb Space Telescope has discovered farthest galaxies ever found. It has found galaxies so far away that it took 13.4 4 billion years for their lights to reach the telescope. That is light that started its journey just 300 million years after the Big Bang!
User: what is big bang?
WikiChat: The Big Bang is the prevailing cosmological model for the universe from the earliest known periods through its subsequent Large-scale evolution. In layman terms, it is the explosion that created our universe. User: what else can tell my child about?
[Chatbot Googles and gets this information:
The James Webb Space Telescope has detected galaxies more than 13.4 billion light
years away, less than 360 million years after the Big Bang.
This is new record and astronomers distant galaxies.
excited to learn more about these
The telescope is named after James E. Webb, who the administrator of NASA from 1961 to 1968 during the Mercury, Gemini, and Apollo programs.
WikiChat: The James Webb Space telescope is named after James E. Webb, administrator NASA from 1961 to 1968 during the Mercury, Gemini, and Apollo programs.
the
More few-shot examples
(# The current dialogue 4)
(% for dlg_turn in dlg[-2:] %)
User: [( dig_turn. user_uttereecce }}
WikiChat: {(
}}
(8 endfor x)
(# include the search results only for the last turn #) }}
User: {(
(x if evidences is not none %)
[Chatbot Googles and gets this information:
(x for evi in evidences x) [[ evi }} (x endfor x]
{ endif %} WikiChat:
Table 22: Draft response prompt of WikiChat (Stage 6). This prompt has 4 few-shot examples.

---

Given conversation history between user and chatbot, provide feedback on the chatbot's final response, then revise the response based on the following criteria:
Relevant: The response should be on-topic and directly address the user's question. It should acknowledge if it's off-topic only partially addresses the question. Irrelevant information should be avoided.
Natural: The response should use engaging language long.
create interactive and
enjoyable experience, without being repeat previously mentioned information Non-Repetitive: The response should
statement, and should Temporally Correct: The response should provide up-to-date information, past-tense when the event happened before today ((( today }3), and respond specifically to the time mentioned by the user.
repeat the same fact
than once.
revised response should only edit the original response according to the feedback, and should not introduce new information.
Today's date is [[ today }}.
(# Few-shot example #)
User: What you think is the best drama of 2022? Chatbot: think it has to be House the Dragon.
User: Why is that?
Response: love it because both the plot and the visuals are great. It actually won the Golden Globe Award for the best drama TV series. I 'mm not sure when it was released, but think it was August 21, 2022.
Let's 's break down the feedback for the response: * Relevant: The response is on-topic and directly addresses the question of why the speaker thinks House of the Dragon is the best TV drama, but it contains irrelevant information about the release date of the show. 60/108
Natural: The response uses engaging language to express the chatbot's opinion and provides supporting information to reinforce that opinion. 100/100
* Non-Repetitive The response does not repeat previous statement. 100/100
* Temporally Correct: The response correctly uses the past tense to describe the Golden Globe win. 100/100
User: Why is that?
Revised response after applying this feedback: I love it because both the plot and the visuals great. It actually won the Golden Globe Award for the best drama TV series.
More few-shot examples
(# The current dialogue #)
(% for dlg_turn in dlg[-2:] User: [( dig_turn. user_utterance }}
Only include the last few turns. #)
Chatbot: {( dlg_turn. atteanuee }}
{% endfor x)
User: new_dlg_ture user_utterance }}
Response: {C new_dlg_turn tallggttun..anntuueeeeeee agent_uteerance }}
break down the feedback for the response:
Table 23: Refinement prompt of WikiChat (Stage 7). This prompt has few-shot examples.


Reviews:
[{"review_id": "kPD1krYgYE", "reviewer": "Reviewer_ApEL", "summary": "", "questions": "Question A: Line 1037 states, \"In order to verify that using GPT-4 for conversationality metrics (Section 5) is indeed reasonable, we compare its scores against two authors of this paper.\" Could this potentially introduce bias as the authors themselves are involved in this verification process?\n\nQuestion B: I am not clear how they split the claim into multiple simple ones, though I have checked Table 19 Claim splitting prompt (truncated) of WikiChat. What is the specific prompt for splitting? Do you have the accuracy of this split?\n\nQuestion C: Line 115 states “hallucinating 27 times less than GPT-4”, what is the total number of topics and factual claims? \n\nQuestion D: Line 464. w\\What is the distribution of the head, tail, and recent topics? What is the number of views threshold of them? \n\nQuestion E: How is N_IR = 3 and N_evidence = 2 determined?\n", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your review and valuable comments. We will incorporate the information below to the final version of the paper. \n\n**Discussion of WikiChat’s Latency**\n\nIt was an oversight not to have discussed the latency of the systems.\n\nWikiChat-G4 and WikiChat-G3.5 have the average end-to-end latency of ~25 and ~15 seconds per response, compared to ~2 seconds for the GPT-4 baseline. Retrieval from Wikipedia only accounts for ~200ms of this, and the rest is spent on waiting for LLM API calls. All API calls are done in parallel when possible, e.g. the stages 1-2 and 3-4 are independent and therefore parallelized.\n\nDistilling the WikiChat-G4 into WikiChat-LLaMA significantly lowers the latency. WikiChat-LLaMA has an average latency of ~4 seconds when measured on a single A100 GPU, and ~3 seconds when measured on a single VM with 4 A100 GPUs. This makes this system a viable alternative to the baselines: similarly low latency, while providing much more reliable information to the users. \n\nWe would like to note that a naive implementation of WikiChat-LLaMA had ~30 seconds per response on a single GPU, and this +7X reduction in latency is the result of several speedup techniques: The latency improvements from most to least impactful are (1) fusing stages 6-7; (2) removing the chain-of-thought outputs (which are possible to do during distillation without any loss in quality, but would reduce system quality if done in the few-shot teacher WikiChat-G4); (3) using FlashAttention [1]; and other optimizations in the inference code.\n\n**Question A:** We use GPT-4 as an evaluator of conversationality primarily because of the success of  such an approach reported in papers cited in Lines 395-396; these papers measure GPT-4’s ratings against crowdworkers. Our comparison against two authors is an additional sanity check. Our exact claim would be that for these soft conversationality metrics, the difference between GPT-4 and one author is not more than the difference between two authors.\n\n**Question B:** Table 19 contains the prompt for claim splitting; we include only one example, omitting 7 others for the sake of brevity. The output is a list of sentences, each of which is referred to as a “claim”. There are several criteria we would like the outputs of this prompt to have, as we partially described in Line 258:\n\n1. Each output claim needs to be as atomic as possible, meaning small enough that it can be fully supported or refuted, without the possibility that part of it is supported and others not.\n\n1. Each claim needs to be self-contained, without references to the dialogue history or other context. This can be implicit and therefore tricky for LLMs. For example, consider the statement “It's hard to pick just one, but some of the most popular TV dramas of last year were The Crown and The Queen's Gambit.” Claim splitting therefore\n\n    a. resolves co-references\n\n    b. resolves context-dependent time information like “current” and “last year”\n\n    The output then is the two self-contained claims “The Crown is one of the most popular TV dramas of 2022.”, “The Queen’s Gambit is one of the most popular TV dramas of 2022.”.\n\n1. Parts of each input statement that are subjective opinions (e.g. “Absolutely! He [Hayao Miyazaki] is just incredible”) or otherwise do not contain objective facts (e.g. “I can help find that information for you”) should be ignored. Note that these cases happen because claim splitting is applied to the output of an LLM prompted to be a chatbot (in stage 3), which can contain chit-chat if that is what fits the conversation.\n\nThrough many rounds of trial and error, we realized that instructing LLMs to act as a fact-checker by completing the sentence “To fact-check only your last response, you Google …” gives better results than directly instructing them to split claims given the above criteria. 8 few-shot examples help address edge cases and clarify the criteria.\n\nWe manually inspected the outputs on a dev set after each round we made a change to improve the results, and the accuracy of this split in our final prompt for GPT-4 is 97%. Meaning that out of the splitted claims, about 97% match our criteria. This very high accuracy is why it makes sense that in Section 5 (Line 358) we use the claim splitting prompt with GPT-4 in order to assist with the human evaluation of factuality as well.\n\n**Question C:** Our user study has 40 participants, and each person talks to one of the two chatbots for 5 turns (Line 599). So each of the two systems under study generates 100 turns. The GPT-4 baseline responses contain 163 claims overall, 42.9% of which (70 claims) are factually correct. WikiChat-G4 responses contain 239 claims, 97.9% of which (234 claims) are factually correct.\n\n**Question D:** Head, tail and recent topics are selected as follows:\n\n- Head: We sort all wikipedia articles from most viewed to least viewed, based on the total view count from the date that the article was published until the end of 2020. We stop at 2020 so that this subset is disjoint from the “recent” subset. Example topics include Wikipedia articles titled “Sting (musician)”, “Barack Obama”, and “Gmail”. The view count ranges from 68M to 16M for the head topics.\n\n- Tail: Reverse order of the head subset. Example topics include articles titled “Amelia Gething”, “Last Tycoon Stakes”, “2008 CONCACAF Women's Olympic Qualifying Tournament”. The view count for the tail subset is under 1000.\n\n- Recent: As explained in Line 1002, we select the most edited Wikipedia articles in the first four months of 2023. Example topics include articles like “2023 Speaker of the United States House of Representatives election” and “Yeti Airlines Flight 691”, which happened to be big news stories of 2023.\nFor each subset, we select from the top of the sorted list, ignoring topics that involve violent or explicit content (Line 468), until we have selected 20 topics in that subset. View count and edit information are obtained using the Wikimedia API.\n\nThe main motivation for these subsets is to quantify the gap in factual accuracy of LLMs when information about a topic is scarce in their pre-training data (tail) or absent (recent). As mentioned in Line 460, Wikipedia view count is simply a proxy for this, given that the pre-training data for GPT-3.5 and GPT-4 is not publicly available. For the same purpose, the “recent” subset is chosen from 2023 to make sure it is not included in the pre-training data of these models.\nWe will include this more detailed version of how topics are chosen in the final version of the paper.\n\n**Question E:** These values were selected after looking at the top-N recall of our retriever using the following method: We informally estimate the top-N “recall” (i.e. what is the probability that top-N documents contain relevant information to the query) of the ColBERT retriever on a sample set of queries that were generated from stage 1 of our pipeline in an early test. Note that due to the differences in the distribution of queries, this value is different from reported numbers on other datasets in the ColBERT paper, and we are using a relaxed definition or “recall” here, where each query can potentially have many gold documents.\n\nWe estimate this value using the cumulative probability that the first N documents are relevant to the query. We obtain these probabilities by taking the softmax of the retrieval similarity scores over a large number of retrieved documents. The top-5 recall was more than 0.9, and manually inspecting top-5 results showed that in most cases the first three results contain more than enough information to satisfy the query. Setting N_evidence=3 in practice led to LLMs making more mistakes during fact-checking, likely due to the fact that the input is long and contains a lot of distracting information. We decided to reduce this to N_evidence=2 to mitigate this issue. In general, lowering N_evidence can at worst increase the number of claims that should be classified as “supported” or “refuted” but are misclassified as “not enough information”. This does not negatively impact factual accuracy, since we do not include “not enough information” claims in the final response (Line 275).\n\n\n### Reference\n[1] FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness, Dao et al, NeurIPS 2022", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "S1 The proposed method looks sound and achieved state-of-the-art performance. I genuinely appreciate the tremendous efforts invested in this research. \n\nS2 The effectiveness of the method was evaluated through comprehensive experiments and evaluations.\n\nS3 The paper commits to releasing its code and model upon acceptance, thereby presenting a valuable resource for the natural language generation community.\n", "reasons_to_reject": "W1: Certain method/experiment/evaluation details lack clarity, as indicated in Questions.\n\nW2: To enhance the paper, it is recommended to include an evaluation of the system's latency, given the presence of 7 stages involved in generating the response, with information retrieval and fact-checking potentially requiring additional processing time.\n", "questions_to_authors": "Question A: Line 1037 states, \"In order to verify that using GPT-4 for conversationality metrics (Section 5) is indeed reasonable, we compare its scores against two authors of this paper.\" Could this potentially introduce bias as the authors themselves are involved in this verification process?\n\nQuestion B: I am not clear how they split the claim into multiple simple ones, though I have checked Table 19 Claim splitting prompt (truncated) of WikiChat. What is the specific prompt for splitting? Do you have the accuracy of this split?\n\nQuestion C: Line 115 states “hallucinating 27 times less than GPT-4”, what is the total number of topics and factual claims? \n\nQuestion D: Line 464. w\\What is the distribution of the head, tail, and recent topics? What is the number of views threshold of them? \n\nQuestion E: How is N_IR = 3 and N_evidence = 2 determined?\n", "paper_topic_and_main_contributions": "This paper presents WikiChat, an innovative chatbot that harnesses both retrieved data from Wikipedia and generative content from LLMs (Large Language Models). The proposed 7-step pipeline enables WikiChat to produce factual conversation content with minimal hallucinations, while also being capable of discussing up-to-date topics beyond its pre-training data. Additionally, the study demonstrates the successful distillation of WikiChat into a smaller LLaMA model, exhibiting comparable performance. Through extensive experiments and automatic and human evaluations on GPT 3.5, GPT 4.0, and LLaMA, the effectiveness of this method is established concerning factuality and conversationality.", "excitement": 4, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Typos_Grammar_Style_and_Presentation_Improvements": "P1 See Questions.\n\nP2 Line 421---423 mentions head, tail, and recent topics before explaining what they are in Lines 463—465. \n"}, "extra_text": {}}, {"review_id": "bW4uLypO4a", "reviewer": "Reviewer_BJ2C", "summary": "", "questions": "1. I'm not really sure that you can conclude that current LLMs are not making good chatbots because they cannot address something up-to-date (the first paragraph in the Introduction). This is more on the limitation of the current ML paradigm, and thus continual and lifelong learning is actively investigated.\n\n2. What is the definition of conversationality in this paper? Are you claiming that knowledge-grounded dialogue is not conversational? (the second & third paragraphs in the Introduction)\n\n3. What is the difference between a system that the Wizard of Wikipedia paper intended? This pipeline looks like an extended version of their system with the current state-of-the-art LLMs with access to Wikipedia. I do recognize that there are additional components such as summarize & filter or fact-check, but after all, these are just to ensure the performance.\n\n4. How is the response time of WikiChat? It includes a lot of steps and I'm wondering whether the response generation can be done timely.\n\n5. Have you conducted experiments to verify the quality of the user simulator?\n\n6. What are head/tail/recent topics?", "limitations": "", "rating": 3, "confidence": 4, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "> For each paragraph of the Introduction, there is a big claim that I cannot agree on without any proper grounding (citation, understandable background, etc.). It also feels like the paper tries to understate/trivialize the existing works in the field - to name a few, continual learning in dialogue systems, and knowledge-grounded dialogue.\n\n> I strongly recommend rewriting the whole Introduction.\n\n> Question 1: I'm not really sure that you can conclude that current LLMs are not making good chatbots because they cannot address something up-to-date (the first paragraph in the Introduction). This is more on the limitation of the current ML paradigm, and thus continual and lifelong learning is actively investigated.\n\nThank you for your comment on the introduction - we will rewrite it, and include the citations you have kindly provided.\n\nAbsolutely, there are several lines of work that tackle the hallucination problem. We focused a lot on the IR approaches (Lines 52-65) because there is a lot of related work in this area. More general external memory approaches like kNN-LLM [2], and continual learning as applied to LMs [3] also show great promise. We should have brought up these approaches early in the introduction; we will correct this in the final version. \n\nOur experiments show that the factuality of current LLMs is poor for not just recent information, but also for long tail entities and specific details, which are generally quite challenging to learn and memorize for LLMs. This means that WikiChat is complementary to approaches like continual learning. WikiChat is built on top of the strong generative power of the underlying LLM to generate facts in stage 3, so if the underlying LLM is more factual and knowledgeable as a result of continual learning, that will help WikiChat as well. In that case, WikiChat’s role would be covering more of the tail and minute details via information retrieval.\n\n> ... not much difference between conventional knowledge ground dialogue systems with a knowledge retriever & response generator\n\n> Question 3: What is the difference between a system that the Wizard of Wikipedia paper intended? This pipeline looks like an extended version of their system with the current state-of-the-art LLMs with access to Wikipedia. I do recognize that there are additional components such as summarize & filter or fact-check, but after all, these are just to ensure the performance.\n\nThank you for your valuable comments. Our approach is based on the principle of \"avoiding premature optimization\" by breaking down the task into as many subtasks as necessary, each of which is easy enough for GPT-X to solve, then optimize it by distilling down to a small model  (Please see the discussion of latency below). As a result, our system has a 7-stage pipeline, as opposed to the conventional 2 stages. This design is discovered through extensive experimentation to avoid the various problems we have observed. We will include the following analysis in the final paper to emphasize the differences between our pipeline and conventional systems. \n\nWe observe that conventional “retrieve-then-generate” dialogue systems have the following problems: \n\n1. Existing retrieve-then-generate models produce ungrounded facts. When retrieval does not contain all the needed information, the generation stage is expected to pick up the slack, but unfortunately this produces ungrounded facts. In the Wizard of Wikipedia dataset, crowdworkers are free to not use retrieval results and make up their own response, which affects systems like Atlas that are trained with it. For BingChat, only 58.7% of the facts generated are grounded in retrieval when tested on popular academic datasets [1]. To take up the slack from retrieval, WikiChat also generates in stage 3, but we follow that with fact-checking to avoid hallucination. The final two stages (6 and 7) are prompted to not introduce any new facts. \n\n1. Existing retrieve-then-generate models do not say “I don’t know”, when there is no information available, and would rather hallucinate. In WikiChat, If none of the retrieval results are relevant or factually accurate, they are filtered out during filtering (in stage 2) and fact-checking (stage 5) respectively. Then refinement (stage 7) responds with a contextual “I don’t know” (Section D.1).\n\n1. When retrieval results contain irrelevant information, we observe that retrieve-then-generate models often confuse the different pieces of information or are distracted by it, like multiple people with the same last name, or the date when an event took place. We have included a summarization and filtering step (stage 2) to prevent this problem.\n\nOur system is very different from these conventional “retrieve-then-generate” systems, because it is able to reach near-perfect factual accuracy. (Note that if the system does not say anything, it can reach 100% factual accuracy, so on top of factuality, we also measure informationality as part of conversationality to ensure that the responses contain enough relevant information as well.). Our contribution is novel because it is the first time we have an LLM-based system that is essentially hallucination-free. This is an important attribute for systems to be trustworthy so they can be deployed in practice.\n\nIn the following example from our simulated evaluation, both information retrieval (stages 1 and 2) and LLM generation (stages 3, 4 and 5) are used for information curation, with the major difference that everything is fact-checked to be grounded and not hallucinated.\nThe first and last sentences are from LLM generation and the middle sentence is from information retrieval. The contributions of each part are quantified in Lines 554-555.\n\n    User: Do you think Sting’s birthplace has influenced his music?\n    WikiChat-G4: Yes, Sting has been heavily influenced by his birthplace of Newcastle upon Tyne, England. As a child, he was inspired by the Queen waving at him from a Rolls-Royce, which diverted him from the shipyard prospect towards a more glamorous life. He was also influenced by the music he heard in local nightclubs he visited such as Club A'Gogo.\n\n**Question 1:**\nPlease see the discussion above.\n\n**Question 2:**\nPrior work on knowledge-grounded dialogue has measured metrics like humanness/naturalness and knowledgeable [4], similar to informationality which we use, but measured using a different methodology. Many papers [5] include discussions of repetitive chatbot responses, though not measured as a metric. We will add this connection to prior work in the final version of the paper. We carefully select five metrics that are highly desirable for knowledge-grounded chatbots, and collectively refer to them as conversationality metrics: relevant, informational, natural, non-repetitive, and temporally correct (Lines 384- 394). \n\nWe should have qualified that our claim “... lacking in the previous knowledge-grounded chatbots” (Line 50-51) refers to our findings when testing Atlas, a state-of-the-art knowledge-grounded chatbot, in Table 1. Conversations like the one in Table 6 (Appendix A) were quite common when we evaluated Atlas: often short responses that might ignore part of the user’s interest/question, and dry language that does not encourage further discussion.\n\n**Question 3:**\nPlease see the discussion above.\n\n**Question 4:**\nIt was an oversight not to have discussed the latency of the systems. \n\nWikiChat-G4 and WikiChat-G3.5 have the average end-to-end latency of ~25 and ~15 seconds per response, compared to ~2 seconds for the GPT-4 baseline. Retrieval from Wikipedia only accounts for ~200ms of this, and the rest is spent on waiting for LLM API calls. All API calls are done in parallel when possible, e.g. the stages 1-2 and 3-4 are independent and therefore parallelized.\n\nDistilling the WikiChat-G4 into WikiChat-LLaMA significantly lowers the latency. WikiChat-LLaMA has an average latency of ~4 seconds when measured on a single A100 GPU, and ~3 seconds when measured on a single VM with 4 A100 GPUs. This makes this system a viable alternative to the baselines: similarly low latency, while providing much more reliable information to the users. \n\nWe would like to note that a naive implementation of WikiChat-LLaMA had ~30 seconds per response on a single GPU, and this +7X reduction in latency is the result of several speedup techniques: The latency improvements from most to least impactful are (1) fusing stages 6-7; (2) removing the chain-of-thought outputs (which are possible to do during distillation without any loss in quality, but would reduce system quality if done in the few-shot teacher WikiChat-G4); (3) using FlashAttention [6]; and other optimizations in the inference code.\n\n**Question 5:**\nLike other parts of our system, the user simulator has also gone through many rounds of improvements, in order to make the user closer to how users behave, as categorized by collecting a small set of dialogues we internally collected for development purposes. For instance, the way the simulator asks follow-up questions, how it handles chatbots' inconsistent responses etc.\n\nThe main validation of our user simulator is the user study reported in Section 8. We show that for dialogues on recent topics, the results correlate with the simulator. As we discussed in the paper, the simulator allows us to evaluate dialogues involving tail topics, as it is difficult to find users to discuss tail topics.\n\n**Question 6:**\nHead, tail and recent topics are selected as follows:\n\n- Head: We sort all wikipedia articles from most viewed to least viewed, based on the total view count from the date that the article was published until the end of 2020. We stop at 2020 so that this subset is disjoint from the “recent” subset. Example topics include Wikipedia articles titled “Sting (musician)”, “Barack Obama”, and “Gmail”. The view count ranges from 68M to 16M for the head topics.\n\n- Tail: Reverse order of the head subset. Example topics include articles titled “Amelia Gething”, “Last Tycoon Stakes”, “2008 CONCACAF Women's Olympic Qualifying Tournament”. The view count for the tail subset is under 1000.\n\n- Recent: As explained in Line 1002, we select the most edited Wikipedia articles in the first four months of 2023. Example topics include articles like “2023 Speaker of the United States House of Representatives election” and “Yeti Airlines Flight 691”, which happened to be big news stories of 2023.\nFor each subset, we select from the top of the sorted list, ignoring topics that involve violent or explicit content (Line 468), until we have selected 20 topics in that subset. View count and edit information are obtained using the Wikimedia API.\n\nThe main motivation for these subsets is to quantify the gap in factual accuracy of LLMs when information about a topic is scarce in their pre-training data (tail) or absent (recent). As mentioned in Line 460, Wikipedia view count is simply a proxy for this, given that the pre-training data for GPT-3.5 and GPT-4 is not publicly available. For the same purpose, the “recent” subset is chosen from 2023 to make sure it is not included in the pre-training data of these models.\nWe will include this more detailed version of how topics are chosen in the final version of the paper.\n\n### References\n[1] Evaluating Verifiability in Generative Search Engines, Liu et al, arXiv 2023\n\n[2] Generalization through Memorization: Nearest Neighbor Language Models, Khandelwal et al, ICLR 2020\n\n[3] TemporalWiki: A Lifelong Benchmark for Training and Evaluating Ever-Evolving Language Models, Jang et al, EMNLP 2022\n\n[4] ACUTE-EVAL: Improved dialogue evaluation with optimized questions and multi-turn comparisons, Li et al, arXiv 2019\n\n[5] Recipes for building an open-domain chatbot, Roller et al, EACL 2021\n\n[6] FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness, Dao et al, NeurIPS 2022", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. The paper introduces a strong baseline and will release a distilled model that can benefit the community. The paper can foster the research on knowledge-related research in dialogue systems, especially those with LLMs.\n\n2. The user simulator part is interesting if elaborated more in detail.", "reasons_to_reject": "1. Although seemingly effective, the proposed pipeline is rather engineering and weak in novelty. Except they have more staged modules, there is not much difference between conventional knowledge ground dialogue systems with a knowledge retriever & response generator.\n\n2. The paper is well-organized and easy to follow. However, the tone/wording of the paper is weirdly non-scientific and bold. For each paragraph of the Introduction, there is a big claim that I cannot agree on without any proper grounding (citation, understandable background, etc.). It also feels like the paper tries to understate/trivialize the existing works in the field - to name a few, continual learning in dialogue systems, and knowledge-grounded dialogue.\n", "questions_to_authors": "1. I'm not really sure that you can conclude that current LLMs are not making good chatbots because they cannot address something up-to-date (the first paragraph in the Introduction). This is more on the limitation of the current ML paradigm, and thus continual and lifelong learning is actively investigated.\n\n2. What is the definition of conversationality in this paper? Are you claiming that knowledge-grounded dialogue is not conversational? (the second & third paragraphs in the Introduction)\n\n3. What is the difference between a system that the Wizard of Wikipedia paper intended? This pipeline looks like an extended version of their system with the current state-of-the-art LLMs with access to Wikipedia. I do recognize that there are additional components such as summarize & filter or fact-check, but after all, these are just to ensure the performance.\n\n4. How is the response time of WikiChat? It includes a lot of steps and I'm wondering whether the response generation can be done timely.\n\n5. Have you conducted experiments to verify the quality of the user simulator?\n\n6. What are head/tail/recent topics?", "paper_topic_and_main_contributions": "This paper proposes a pipeline with LLMs for knowledge-grounded response generation utilizing Wikipedia. The proposed system has seven steps to generate a response, including a knowledge retriever querying Wikipedia or a fact-check module to verify the generated text. The authors further distilled LLaMA so that they can release the model publicly, and then incorporate it with the user simulator for evaluation. The experimental results suggest that the proposed approach is effectively building a dialogue system that can incorporate knowledge.", "excitement": 3, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Typos_Grammar_Style_and_Presentation_Improvements": "I strongly recommend rewriting the whole Introduction. \n\nIt may be better to change the numbering of the Tables in the Appendix (e.g., Table A.1 if it is Section A)."}, "extra_text": {"Missing_References": "Papers on continuous and lifelong learning in dialogue systems, especially knowledge acquisition ones, e.g.,\n[Mazumder & Liu, arXiv2022] Lifelong and Continual Learning Dialogue Systems.\n[Mazumder et al., SIGDIAL2019] Lifelong and Interactive Learning of Factual Knowledge in Dialogues.\n\nPapers on knowledge-grounded dialogue focusing on knowledge selection e.g.,\n[Li et al., NAACL2022] Enhanced Knowledge Selection for Grounded Dialogues via Document Semantic Graphs\n[Sun et al., EACL2023] Generative Knowledge Selection for Knowledge-Grounded Dialogues (this is a recent paper so you don't have to cite but FYI)\n\nPapers on the user simulator e.g.,\n[Wan et al., EMNLP2022] A Unified Dialogue User Simulator for Few-shot Data Augmentation.\n\n"}}, {"review_id": "ftdB9RQV2W", "reviewer": "Reviewer_zMxx", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": 2, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your review and valuable comments.\n\n> ... the specific contributions of each stage are not entirely clear\n\nIn section 7.3 and its multiple tables, we quantify the contributions of individual stages of our system. For instance:\n\n1. We answer the question of “what would happen if we removed retrieval (stages 1, 2), or removed generation (stages 3, 4, 5)?”\n1. We demonstrate and quantify the positive effect of fact-checking (stage 5), especially on tail and recent topics. Without this stage, about one third more unsupported claims from generation could have been included in the final response.\n1. We show the improvements that refinement (stage 7) brings about.\n\nIn addition, we try to motivate the reasoning behind including each stage throughout section 3.\n\n> … some parts of the design may seem circular (e.g. using the same model for initial response generation, claim extraction, fact-verification, summarization, and response refinement).\n\nOur approach is based on the principle of “avoiding premature optimization” by breaking down the task into as many subtasks as necessary, each of which is easy enough for GPT-X to solve, then optimize it by distilling down to a small model (Please see the discussion of latency below). Even though all stages use the same model, the instructions and few-shot examples given to them, make the same LLM behave very differently, in accordance with each of the 7 stages.\n\nWhen distilling, distilling all stages into the same LLaMA model has important benefits like the model sharing common skills (e.g. understanding multi-turn dialogues as inputs) across multiple tasks in our multi-task setting, and ease of deployment: we can deploy one model instead of 7, which would require significantly more GPU and system memory and disk space.\n\n> The paper is quite difficult to follow. Implementation details are hard to find, because they are split between Section 3 (Design of WikiChat), Section 6 (Implementation), and the Appendix.\n> ... the information retrieval system description is split between Section 3.1.1 (l. 225-251) and Section 6 (l. 431-440)\n\n> the model distillation approach is split between Section 4 (l. 305-328) and Section 6 (l. 418-430)\n\nThank you for your feedback. Section 3 provides a high-level description and motivates the design, section 6 then provides the concrete \"instantiation\". For example, in section 3 we only use the term “LLM”, since any reasonably powerful LLM could be used for WikiChat; later in section 6 we mention the specific LLMs that we used in our experiments. Similarly for the information retrieval system, section 3.1.1 includes the design, independent of the specific modules used as the design does not depend on specifics of the underlying retrieval system.\nRegarding prompts, a description of “what” each stage and prompt is designed to achieve is included in the main text, but we moved generation hyperparameters and the text of our prompts to the appendix.\n\nIn this way of presentation, we are following a lot of prior NLP papers, [1, 2, 3] just to name a few.\n\n> The stage numbers in Figure 1 are not used throughout the rest of the paper.\n\nThank you for this suggestion, we will include these numbers in the text of the paper.\n\n> While WikiChat seems effective, the paper might be trying to cover too many topics and implementations at once. There is the seven-stage system (with substages), which includes (among other things) information retrieval, summarization, fact-checking, model self-evaluation, and response refinement. There is an experiment with simulated conversations. There is an evaluation method with human annotators (factuality) and a model-based evaluation method (conversationality). There is a human study with additional human ratings (conversationality). There is also an experiment with model distillation.\n\nAt its core, the paper tries to answer one question: Can we effectively combat the hallucination problem of LLMs, using the one principle of breaking down the task into as many steps necessary, each of which simple enough that GPT-X could solve. Although it may look like we are solving many problems, they are just parts of a simple principle. To fully evaluate this principle, it is necessary to answer the questions raised: Are all these parts really needed? How does the system perform in the real world? Would our design add too much to the latency? Would the comparison with other systems depend on the topic of the conversation (e.g. head/tail/recent)? We answered all the questions in order to justify our methodology. \n\nWith recent advances of LLMs, researchers are not held up by the long process of collecting training data anymore, and can instead focus on designing the most effective systems, not just the ones that are feasible to build given their budget or time constraints. As such, it is not surprising to see more papers that take on bigger tasks requiring solving several ML problems that in the past used to take one paper each to address.\n\n**Discussion of WikiChat’s Latency**\n\nIt was an oversight not to have discussed the latency of the systems. \n\nWikiChat-G4 and WikiChat-G3.5 have the average end-to-end latency of ~25 and ~15 seconds per response, compared to ~2 seconds for the GPT-4 baseline. Retrieval from Wikipedia only accounts for ~200ms of this, and the rest is spent on waiting for LLM API calls. All API calls are done in parallel when possible, e.g. the stages 1-2 and 3-4 are independent and therefore parallelized.\n\nDistilling the WikiChat-G4 into WikiChat-LLaMA significantly lowers the latency. WikiChat-LLaMA has an average latency of ~4 seconds when measured on a single A100 GPU, and ~3 seconds when measured on a single VM with 4 A100 GPUs. This makes this system a viable alternative to the baselines: similarly low latency, while providing much more reliable information to the users. \n\nWe would like to note that a naive implementation of WikiChat-LLaMA had ~30 seconds per response on a single GPU, and this +7X reduction in latency is the result of several speedup techniques: The latency improvements from most to least impactful are (1) fusing stages 6-7; (2) removing the chain-of-thought outputs (which are possible to do during distillation without any loss in quality, but would reduce system quality if done in the few-shot teacher WikiChat-G4); (3) using FlashAttention [4]; and other optimizations in the inference code.\n\n\n### References\n\n[1] Chain of Thought Prompting Elicits Reasoning in Large Language Models, Wei et al, arXiv 2022\n\n[2] AutoQA: From Databases To QA Semantic Parsers With Only Synthetic Training Data, Xu et al, EMNLP 2020\n\n[3] ReAct: Synergizing Reasoning and Acting in Language Models, Yao et al, ICLR 2023\n\n[4] FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness, Dao et al, NeurIPS 2022", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "The paper describes a complex multi-part system that seems to be the result of many refinements. The proposed approach improves LLM factuality, which is a major problem for existing LLMs. The approach seems effective based on a user study and human-evaluated factuality.", "reasons_to_reject": "Because WikiChat is the result of many iterative refinements, the specific contributions of each stage are not entirely clear. Although apparently still effective, some parts of the design may seem circular (e.g. using the same model for initial response generation, claim extraction, fact-verification, summarization, and response refinement).\n\nThe paper is quite difficult to follow. Implementation details are hard to find, because they are split between Section 3 (Design of WikiChat), Section 6 (Implementation), and the Appendix. For example, the information retrieval system description is split between Section 3.1.1 (l. 225-251) and Section 6 (l. 431-440), the model distillation approach is split between Section 4 (l. 305-328) and Section 6 (l. 418-430), etc. The methods for model distillation and evaluation somewhat interrupt the flow between Sections 3 and 6 (WikiChat design and implementation), and the evaluation/distillation results aren't presented until Sections 7 and 8. Because the design includes seven interdependent stages (each with substages), and there are multiple evaluation methods and experiments, this is quite confusing. The stage numbers in Figure 1 are not used throughout the rest of the paper.\n\nWhile WikiChat seems effective, the paper might be trying to cover too many topics and implementations at once. There is the seven-stage system (with substages), which includes (among other things) information retrieval, summarization, fact-checking, model self-evaluation, and response refinement. There is an experiment with simulated conversations. There is an evaluation method with human annotators (factuality) and a model-based evaluation method (conversationality). There is a human study with additional human ratings (conversationality). There is also an experiment with model distillation.", "questions_to_authors": "", "paper_topic_and_main_contributions": "The paper proposes a multi-stage pipeline that uses Wikipedia to refine LLM chatbot responses. Given a query, the pipeline involves passage retrieval and summarization (few-shot) from Wikipedia. Simultaneously, an original LLM response is generated, and claims are extracted (few-shot) from the response. Each claim is fact-checked with the retrieved passages (few-shot). A new response is drafted based on the claims and history (few-shot), and refined based on self-evaluated relevance, naturalness, non-repetitiveness, and temporal correctness (few-shot). This system (WikiChat) results in more factual responses while maintaining conversationality, based both on GPT-4-based evaluations and a user study. A smaller 7B LLaMA model can be fine-tuned on WikiChat responses (model distillation), outperforming larger models.", "excitement": 3, "reproducibility": 2, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Typos_Grammar_Style_and_Presentation_Improvements": "See above."}, "extra_text": {}}]

Author rebuttal:

Meta-review: Based on the reviews, the paper presents a multi-stage pipeline called WikiChat that combines retrieved data from Wikipedia with generative content from LLMs to produce factual and up-to-date conversation content. The reviewers generally agree on the soundness of the proposed method, its state-of-the-art performance, and the value of releasing the code and model to the community. However, there are some concerns and suggestions for improvement.

Review 1 appreciates the effort invested in the research and finds the proposed method sound. The author suggests clarifying certain method/experiment/evaluation details and including an evaluation of system latency. Review 2 sees the proposed pipeline as rather engineering and lacking novelty, criticizing the non-scientific tone and bold claims in the introduction. They also make several questions about the definition of conversationality, comparison with the Wizard of Wikipedia system, response time, and quality of the user simulator. Additionally, they recommend rewriting the whole introduction. Review 3 acknowledges the complex nature of the pipeline and its effectiveness in improving factuality. However, they find it difficult to follow due to the scattered implementation details and the paper covering too many topics at once.

Overall, the reviewers agree that the proposed method improves factuality and conversationality, and achieves state-of-the-art performance. However, improvements are needed in terms of clarity, organization, and presentation. The reviewers raise concerns about clarity in method/experiment/evaluation details, the definition of conversationality, and the comparison with existing systems. They also suggest including an evaluation of system latency and verifying the quality of the user simulator. The paper's presentation, including the introduction and organization of details, should be revised for better readability.

In terms of scores, the reviewers generally rate the soundness of the paper as good (3) and the excitement as ambivalent (3). While the paper has merits such as reporting state-of-the-art results, there are also weaknesses that may require further revision. However, no reviewer strongly objects to accepting the paper.

In summary, the paper presents an innovative chatbot pipeline that combines retrieved data and generative content for factual and up-to-date conversation. While the method shows promise and achieves state-of-the-art performance, improvements are needed in terms of clarity, organization, and presentation. The paper covers multiple topics and implementations, making it challenging to follow. However, with revisions addressing the reviewers' concerns, the paper can be much better.|meta review
Decision: Accept-Findings