RefGPT: Dialogue Generation of GPT, by GPT, and for GPT
Authors: Dongjie Yang
Ruifeng Yuan
YuanTao Fan
Yifei Yang
Zili Wang
Shusen Wang
hai zhao
Venue: EMNLP 2023 Findings (2023)
Keywords: Dialogue Generation
GPT
Model Hallucination
LLM
TL;DR: Use GPT to generate truthful and customized dialogues by leveraging the references.

Abstract: Large Language Models (LLMs) have attained the impressive capability to resolve a wide range of NLP tasks by fine-tuning high-quality instruction data. However, collecting human-written data of high quality, especially multi-turn dialogues, is expensive and unattainable for most people. Though previous studies have used powerful LLMs to generate the dialogues automatically, they all suffer from generating untruthful dialogues because of the model hallucination. Therefore, we propose a method called RefGPT to generate enormous truthful and customized dialogues without worrying about factual errors caused by the model hallucination. RefGPT solves the model hallucination in dialogue generation by restricting the LLMs to leverage the given reference instead of reciting their own knowledge to generate dialogues. Additionally, RefGPT adds detailed controls on every utterance to enable high customization capability, which previous studies have ignored. On the basis of RefGPT, we also propose two high-quality dialogue datasets generated by GPT-4, namely **RefGPT-Fact** and **RefGPT-Code**. RefGPT-Fact is a dataset with 100k multi-turn dialogues based on factual knowledge and RefGPT-Code has 76k multi-turn dialogues covering a wide range of coding scenarios.  Our code and datasets are released in https://github.com/mutonix/RefGPT.

RefGPT: Dialogue Generation of GPT, by GPT, and for GPT
Dongjie Yang!,, Ruifeng Yuan², Yuantao Fan', Yifei Yang",, Zili Wang', Shusen Wang', Hai 5hao
Shanghai Jiao Tong University, Hong Kong Polytechnic University,
Beijing University of Posts and Telecommunications
[[djyang. tony, yiny, www.eiiaggesssjtuuu edu edu.edu.co yuuneeegyyaaeeeccmeett www.yyynnnnnct.poo.... hk, Syuantaof an@bupt. funnnaaoannebptt edu. cn, "(ziliwang. do, wssatzzuu@gmaiil..... com, Zzhohaaicss sjtu ju. edu. cn
Abstract
to intention recognition and complete variety of NLP tasks only via fine-tuning with small amount of high-quality instruction data (Taori et al., 2023; Chiang et al., 2023; Xu et al., 2023a). However, such high-quality instruction datasets, especially multi-turn dialogues with instructions in vertical domains, requires enormous crowdsource workers with extensive professional knowledge collect (Ouyang et al., 2022), where the cost is unaffordable for most people.
Large Language Models (LLMs) have attained the impressive capability resolve wide range of NLP tasks by fine-tuning with high-
quality instruction data. However, collecting luman-written data of high quality, especially multi-turn dialogues, is expensive and unattain- able for most people. Though previous studies have used powerful LLMs to generate the dialogues automatically, they all suffer from generating untruthful dialogues because of the model ballucination. Therefore, we propose method called RefGPT to generate enormous truthful and customized dialogues without worrying about factual errors caused by the model hallucination. RefGPT solves the model hallucination in dialogue generation by restrict- ing the L.MS to leverage the given reference instead of reciting their own knowledge to generate dialogues. Additionally, RefGPT adds detailed controls on every utterance to enable high customization capability, which previous studies have ignored. On the basis of RefGPT, we also propose two high-quality dialogue datasets generated by GPT-4, namely RefGPT- Fact and RefGPT-Code. ReiGPT-Fact is dataset with 100k multi-turn dialogues based factual knowledge and RefGPT-Code has 76k multi-turn dialogues covering a wide range of coding scenarios. Our code and datasets are released in https:// thub. com/ wwt./ooommm/aeeeeeeeee mutonix/ RefGPT.
Previous studies (Peng et al., 2023; Xu et al., 20236; Ding et al., 2023) have shown the effective- ness of prompting LLMs like GPT-3 (Brown al., 2020) to generate enormous instructions (single- turn dialogues) or multi-turn dialogues with given Humann-ritterd instructions or conversation topics as seeds. However, such one-shot or few-shot methods have a common deficiency that they have the risk of generating untruthful and misleading content due to the language model hallucination (OpenAI, 2023; Ji al., 2023). The reason why the issue of untruthfulness happens is obvious. This is because the quantity of information in seed prompts like numan--ritter instructions or topics is not enough for being converted to the dialogue on a new topic so LLMs have to recite their own knowledge to complete such new dialogue which may lead to the model hallucination of generating untruthful facts.
Therefore, we introduce RefGPT, method for generating truthful and customized multi-turn dialogues utilizing the ability of powerful LLMs like GPT-3.5/GPT-4, RefGPT first provides plain text or a document as the reference and guides the LLMs to leverage the references to generate dialogues. By providing enough information on a new topic as context, LLMs will be prompted not to rely on their own knowledge to generate the dialogues, thus resolving the hallucination issue.
1 Introduction
General chat models (OpenAI, 2022, 2023; An- thropic, 2023) based on Large Language Models (LLMs) have shown the impressive capability
*Dongjic Yang. Yifei Yang and Hai Zhao with the Department of Computer Science and Engincering, Shanghai Jiao Tong University, and also with Key Laboratory of Shanghai Education Commission for Intelligent Interaction and Cognitive Engineering. Shanghai Jiao Tong University. 'Corresponding author: This paper was partially supported by Joint Research Project of Yangtze River Delta Science and Technology Innovation Community (No. 2022CSJGG1400).
After ensuring the authenticity of the dialogue, we further develop an effective prompting process

---

for RefGPT to guide the LLMs to generate highly controllable dialogues in specified uniform for- mat which is easy for training. Previous studies (Xu et al., 2023b; Wang et al., 2022) for automatically generating dialogues have very little control over the generated dialogues. For comparison, RefGPT enables LLMs to generate customized multi-turn dialogues with detailed controls the structure, style, and content, which further gives diversity to the generated dialogues.
2 Related Work
2.1 LLM based Dialogue Generation
The high-quality dialogue dataset is considered crucial for the success of current general chat models (Chiang et al., 2023; Köpf et al., 2023). Due to the high cost of human annotation, previous studies have explored the effectiveness of using LLMs for dialogue generation. Self-Instruch (Wang et al., 2022) presents framework that facilitates the automatic generation of instruction data (single- turn dialogues) by leveraging existing LLMs. The procedure commences with a set of human-written seed tasks and progressively generates new instruc- tions and responses by iteratively bootstrapping both the initial seeds and the newly produced data. Baize (Xu al., 2023b) generates multi- turn dialogues by leveraging LLMs to engage in a conversation with itself as both user and assistant based on the given seed topics. UltraChat (Ding et al., 2023) follows similar idea to Baize and adopts two separate LLM APIs in the generation, where acts the user and the other acts the assistant. However, the dialogues produced by these methods are susceptible to hallucination problems and are uncontrollable. Therefore, we present RefGPT as solution to generate dialogues with truthfulness and customization.
Based on the RefGPT, we also propose two new multi-turn dialogue datasets, namely RefGPT-Fact and RefGPT-Code. Both datasets have English and Chinese versions. RefGPT-Fact and RefGPT- Code consist of 100k and 76k high-quality multi- turn dialogues generated from GPT-4 separately, using the online encyclopedia websites and Github repositories as the references. As long as the con- tent on the online encyclopedia website and Github codes is truthful and reliable, the authenticity of the generated dialogues can be maximally ensured. Besides the topics in RefGPT-Fact and RefGPT- Code, ReiGPT has the potential to generate truthful dialogues any topics or vertical domains if we give it relevant references. RefGPT enables such people working in specific domain, e.g., the nuclear industry, have high-quality multi-turn dialogues dataset to train a chatbot specializing in such domain using their own knowledge base as the reference.
2.2 Reference Based Dialogue Generation
To sum up, our contributions are stated as follows:
QA pair and dialogue generation based on refer- ences have also been widely used. One important requirement for these methods is to ensure the truthfulness of the generated QA pairs and dia- logues. Previous studies (Ma al., 2020; Lewis et al., 2021) generate millions of high-quality QA pairs based on corpus documents using special- purpose question generation models. Dialogue inpainting (Dai et al., 2022) extends this line of work to dialogues by transforming passages from Wikipedia into multi-turn dialogues using masked conversational language model. In this work, we adopt a similar strategy using the LLMs that we take high nigh-quality documents as references ensure the truthfulness of the generated dialogee..
We propose RefGPT, method of generating truthful and customized dialogues using pow- erful LLMs. Given the reliable reference, Re- tGPT resolves LLM hallucination in dialogue generation to the greatest extent. RefGPT can also enable detailed customization in the structure, style and content of the dialogues.
With RefGPT, we construct two new multi- turn dialogue datasets using GPT-4, called RefGPT-Fact and RefGPT-Code. To our best knowledge, RefGPT-Fact is one of the largest multi-turn dialogue datasets based on factual knowledge. And RefGPT-Code is the first and largest synthetic multi-turn dialogue dataset covering nearly all aspects of code scenarios. These have shown the capability of applying RefGPT to generate dialogues in any vertical domain by utilizing corresponding domain -omain--pecific documents.
Generation Process
In this section, we present the whole process of RefGPT, which generates truthful and eustomized multi-turn dialogues by prompting the Large Lan- guage Models (LLMs) to effectively utilize the

---

Basic Prompt
Dialogue Template
Reference Selection
on the provided information
RefSPT-fact
above, expand it into multi-turn conversation...
Wikipedia
cuser t> [word count: 50 words) childliee
Natural processing (NLP) interdiscillinary subfield linguisties, co mpputer science, and artific ial intelligence concerned with the interactions between computers human languege, in particular how to computers analyze large smounts natural language data.
Dialogue Settings
cassistant I> (word count: 300 words] answers in way that a child understand
Task Description
cuser 2 (word count: 100 words) further gives specific instructions the assisant cassistant count: words) answer (rdetalled explanation]
and
multiple various
questions//rqqest to the assistant about factual factual knewledge
</chat>
Code User the above code and wants to solve with a computer propram...
Github Code Fibonacctinn
RefGPT-Code-
<chat>
(n en return it Fibonaceiln 2) Fibonacciin-
suser 1> (word count: 100 words]
Local Customization
asks young person's tone abour writing the code 500 words)
Structure
cassistant (word answer (+detalled explanation)
Count
Word Count Sampling
and give detailed code examples
Other documents the vertical domains (e-g, thopping)
Style
cuser 2> (word count: 150 words) specific instructions the assistant sbout forther explaining the code cassistant 2> (word count: 300 words] indetalied explanation)
childlike
Content
writing the code
</chat>
Figure 1:
Overview of the whole RefGPT generation process, which mainly consists of three steps: reference selection, basic prompt and dialogue settings.
reference information. As illustrated in Figure I, the RefGPT process is comprised of three main steps: Reference Selection (pertaining to truthfulness), Basic Prompt, and Dialogue Settings (pertaining customization).
contingent upon the existence of text-based knowl- edge repositories within those domains. These repositories include broad spectrum of subjects, including, but not limited to, general domains like factual knowledge with encyclopedias, program codes, and vertical domains like shopping applica- tions or the nuclear industry.
3.1 Reference Selection
RefGPT guides the LLMs to leverage the given external documents or plain texts as references, instead of reciting their own knowledge, to gen- erate truthful dialogues without worrying about hallucination.
3.2 Basic Prompt
To facilitate the generation of multi-turn dialogues that adhere to our basic requirements, we have devised a set of basic prompts:
The quality of generated dialogues in RefGPT relies on the selection of appropriate references, prioritizing quality and thematic relevance.
Prompt the LLMs to generate multi-turn dialogues based on the provided reference.
A reference in RefGPT can range from a piece of unrefined plain text a high-quality and depend- able document in a specific domain, whose credi- bility determines the upper limit of the truthfulness of the generated dialogues. On the premise that the reference has contained enough information, it is imperative to opt for high-quality references, such as authoritative knowledge-based websites like Wikipedia.
2. Specify the desired language for the dialogue generation. It is preferable for the language of the reference to be consistent with the dialogue to be generated.
Instruct the LLMs to reject unrcasonable user requests, such as illegal or inappropriate in- structions while providing appropriate advice to discourage such actions. This prompt aids in generating dialogues that align with human preferences certain extent.
Furthermore, the chosen reference profoundly influences the thematic direction of the generated dialogues. Consequently, RefGPT exhibits the potential to generate dialogues in diverse domains,
4. LLMs like GPT-3.5-turbo and GPT-4 offer an option of writing a "system" role prompt

---

to exert precise control over their behav- iors in responses. This capability enables customization of the chatbot's identity by providing relevant background information. For instance, in a vertical domain like a shopping app, RefGPT can generate dialogues that conform to the persona of a shopping assistant, even if the reference has no explicit association with shopping (but may have an
(I) Number of Turns LLMs like GPT-3.5/GPT- 4 often fail with counting the number of the turns of dialogues if we directly require a certain number. But we find that GPT-3.5/GPT-4 are good at following the given format and replacing the placeholders with their own generated content. Therefore, if we want to generate turns of dialogues, we explicitly give the <user> and <assistant> pairs to let LLMs follow the output format. We have also added numerical markers indicate the 7d turn of the dialogue, e-g.. <user i> and kassistant i>, allowing the LLMs to better identify the progress of the current generated turn.
implicit association).
3.3 Dialogue Settings
Rather than generating dialogues uncontrollably, RefGPT uses dialogue settings to convert the ref- erence to specific dialogue format and customize every utterance, as shown in the middle part of the Figure In dialogue settings, we first specify the task description to tell LLMs how to use the reference. We then customize the structure, style, and content of the dialogue, which can be collectively called local customization.
(2) Length of Utterance Generating whole dialogue at one-time, c.g., Self-Instruct (Wang et al., 2022), often leads to much shorter responses than the general chat models like GPT-3.5 do, shown in Table 1. However, in RefGPT, we control the lengths of only the responses of the assistant but also the questions raised by the user at every turn of the dialogue.
3.3.1 Task Description
We observe that specifying word count the prompt is useful for influencing the length of generated utterances. Following the auto- regressive (left-to-right) order, we first illustrate the requirement of word count like <user> (word count: x words) or cossistaat>(word count: X words) before our customization on style and content. Therefore, RefGPT can generate shorter or much longer question/response depending on the specified word count. Though this prompt also be used make the generated utterances longer with other methods like Self-Instruct, generating longer utterances always leads to more severe hallucination problem. RefGPT filters out the reference whose length is shorter than 80% of the required dialogue length to ensure truthfulness. Thus the LLMs have no necessity of reciting their own knowledge, as the reference length is similar and even longer than the dialogue length.
We begin by defining the task of dialogue genera- tion concerning the utilization of references, it relies on the specific aspect of the reference that we aim to initiate the dialogue. For instance, a given piece of program code can lead to multiple scenarios (tasks), such as explaining, creating, or debugging.
3.3.2 Local Customization
As per the task description, the local customization specifies the settings regarding the dialogue's structure, style, and content. These settings then incorporated into dialogue template for generating the final dialogue.
Dialogue Structure To define the dialogue struc- ture, we start the dialogue with the marker <chat> and end it with the marker </chat>. These two markers specify the range of the whole dialogue. Between the start and the end, we use <user> for the user giving instructions and <assistant> for the chatbot. A unified output format a dialogue template avoids most of the weird generations of LLMs and is easier for post-processing. What is more, we will show more merits of using such format to control the number of turns and length per turn.
Dialogue Style Staying organized around the same reference, the style of dialogue can vary in the style of asking and answering. For example, a dialogue start between user who is child and an assistant who answers in way that child understand. RefGPT enables this customization for every utterance of <user> and <assistant> in the dialogues by adding the style requirements before the content customization.

---

Dialogue Content After specifying the style, we can customize the content of each utterance about what to ask and what to answer.
as the reference and the Chinese version uses the frequenty-used Chinese online encyclopedia website, Baidu Baike. We use various dialogue settings mentioned in Sec 3.3 to increase the dialogue diversity.
For the task like factual knowledge, the user can be set to ask more about the entity or numbers in the reference. For the task of coding, the user can ask from different perspectives on writing, revising, and using the code and the assistant can choose to give an example not.
RefGPT-Code RefGPT-Code is a dataset con- taining 76k multi-turn dialogues about program- ming with 37k English and 39k Chinese, which has covered most aspects of code usage scenarios and multiple types of programming languages. Both the English version and Chinese version use the public Github dataset on Google BiqQuery with overlap in these two languages. RefGPT-Code has derived various ways of leveraging the program code as the reference enable different scenarios. We consider three perspectives code discussion, code creation and bug fixing in RefGPT-Code.
Dialogue Template We aggregate the local cus- tomizations into a dialogue template to transfer the reference to the dialogue. To enable diversity, we sample different local customization settings for each utterance in the dialogue, as shown in the right-most part in Figure 1. In practice, RefGPT work well even without style and content pools. These additional settings only need small amount of manual work for further customization and can be reused to generate diverse dialogues based on different references.
1. In RefGPT-Code-ds about code discussion, want the LLMs to generate dialogues about asking questions about the given reference code, including explaining. discussing, revis- ing, rewriting. and using the code. After the generation, we will concatenate the reference code as the context to the first question of the user to form the complete version of the dialogue, because we often give the code first before asking questions about it. Thus. the whole dialogue has much longer user utterances, as shown Table 1.
1. For the dialogue structure, we will set the number of turns by weighted sampling. And we sample the word count for both user and assistant in each utterance from a Gaussian distribution.
2. For the dialogue style, we construct a conver- sational style pool to sample the style settings.
3. For the dialogue content, we construct a content pool according to the task (factual knowledge, code, etc) to sample the content settings.
RefGPT-Code-ct about code creation, though we provide the program code as the reference, we assume that the user has an idea/reques//troubleuuk relevant to the given code but does not know such code exists, thus he/she wants the assistant to help with writing the code. And the assistant is required write the code according to the reference code instead of generating a new code ensure the reliability of the generated code.
4 RefGPT Dialogue Datasets
In this section, we present two multi-turn dialogue datasets, denoted as RefGPT-Fact and RefGPT- Code, which generated utilizing the GPT-4 API in conjunction with RefGPT, More information about these two datasets can be found Appendix A, and examples are provided in Appendix B.
3. In RefGPT-Code-bg about bug fixing, the user first writes a piece of code with bugs based on the given reference code, which is realized by asking the L.Mss to rewrite the code to a buggy one in the first utterance of the user. Then the assistant is required tell the user where the bugs are and how fix them according to the reference code. In this scenario, we assume the reference code is reliable and has no bugs.
4.1 Dataset Generation Process
RefGPT-Fact RefGPT-Fact a dataset con- taining 100k multi-turn dialogues about factual knowledge with 50k English and 50k Chinese. The English version uses the English Wikipedia
'In Table 1, we calculate the number of tokens with the OpenAl tokenizer library of tiktoken in https: thub. con/opent aopena//tiktkken We use cl cl1e0k_r base in tiktoken to tokenize.

---

Table 1: Comparsions on different dialogue datasets that contain instructions. Al means whether it is generated by AL Truthful indicates whether the truthfulness of the dialogues is guaranteed. QLen means the average number of tokens' of user utterance. RLen means the average number of tokens of assistant utterance. Turn means whether the number of dialogue turns can be specified. Lang indicates the languages the dataset supports. For fair comparison. only the English parts are selected in all the datasets.
Dataset
A1
Truthful
QL
RI.en
Turn
Lang
Dolly (Databricks, 2023)
N/A
16.3
78.2
en
Oasstl (Köpf et al., 2023)
N/A
28.0
169.5
I~5
multi multi
ShareGPT (Dom Eccleston, 2023)
75.6
268.8
1~5
Alpaca (Wang et al., 2022)
17.2
55.3
en
Baize Quora (Xu et al., 2023b)
15.7
43.2
3-55
en
UltraChat World (Ding et al., 2023)
28.6
207.9
3-/7
en
RefGPT-Fact
28.1
269.5
3-4
en, cn
RefGPT-Code-ds
281.7
374.6
3-4
en, cm
RefGPT-CCoeeer
36.9
395.0
3rd
en, cn
RefGPT-Code--g
155.7
380.8
2-44
en, cn
4.2 Dataset Collection Setup
length of ChatGPT response in ShareGPT. RefGPT-Code series implements various cus- tomizations to be adapted specific scenarios and have longer user and assistant utterances because we have not only the utterances but also the code attached to the dialogues.
We use the RefGPT with GPT-4 API to generate these two datasets. The length of every utterance is decided by sampling the Gaussian distribution of (M,, r), where accounts for the average word count (e.g., 300 words) of the utterance and is the standard variance (e.g., 50 words). The number of turns is decided by weighted sampling, where the weights determine the ratio of dialogues with a specific number turns in the dataset.
Experiment
5.1 Truthfulness Evaluation
In order to verify the reliability of RefGPT, We evaluate the truthfulness of the RefGPT dataset using both human evaluation for small sample and automatic evaluation with GPT-4 for large range of verificaiton. For automatic evaluation with GPT-4, though existing methods (Chiang et al., 2023; Liu et al., 2023) have leveraged the GPT-4 to evaluate the performance of other LLMs. However such evaluation is not reliable factual error checking because GPT-4 has the issue of model hallucination. Inspired by RefGPT, we design pipeline to evaluate the truthfulness of generated dialogues from our reference datasets. e.g., Wikipedia, by using the GPT-4 to evaluate but with the additional help of reference.
4.3 Dataset Statistics
As shown in Table 1, we compare our datasets to other high-quality dialogue datasets. ShareGPT (Dom Eccleston, 2023) collects the dialogues from the real users and ChatGPT, which have much longer user utterances and assistant utterances. we choose the responses of ChatGPT as a baseline, (Wang
methods with one API, c.g., et al., 2022) and Baize (Xu et al., 2023b), always lead to shorter assistant responses. UltraChat (Ding et al., 2023) with two independent APIs chatting to each other maintains the length of generated responses close to ChatGPT. However, shown in Table 2, such methods call the model API one utterance at a time with significantly increasing cost and time, as UltraChat has to attach the conversation history multiple times. By contrast, RefGPT generates the whole dialogue with one API call but can adjust the length of generated utterance flexibly according to the requirement.
Reference
Question
Selfinstruct Baloe
Wikipedia
hameworb. peography tviel ienouutio the Fart??
RerGin
Dialogue
LLM
RefGPT-Fact inherits the diversity of the refer- ences like Wikipedia and Baidu Baike. Besides that, RefGPT-Fact has an average response length of 269.5 in English which very similar to the
GPT-4
susero in Fanth,
Figure 2: Illustration of the process of truthfulness evaluation.

---

Table 2: Comparsions on different methods automatically generating dialogues via LLMs. Multi-turn means whether it multi-turn dialogue generation. Human Tru. and GPT-4 Tru. evaluate the truthfulness with accuracy by humans and GPT-4 model. Len uses ChatGPT's response length as the standard for long responses. Turn means whether the number of dialogue turns can be specified. Custo. depends on whether it can control the dialogue structure and content. Call is the number of model or model API calls needed for generating an instruction n-turn dialogue every time.
Method
Multi-turn
Human Tru.
GPT-4 Tru.
Len short short
Turn
Custo. Call
Self- Instructt (Wang et al., 2022)
x
54.0
50.2
limited
1
Baize Self-Chat (Xu al., 2023b)
50.0
47.2
random
limited
1
UltraChat (Ding al., 2023)
long adjustable
limited 2n
Re[GPT
98.0
97.5
adjustable
adjustable
highly 1
5.1.1 Evaluation Process
Therefore, shown in Figure 2, we let GPT-4 check the generated dialogue accords with the reference. If the generated dialogue does not align with the reference, indicates the presence of factual errors.
We compare RefGPT two popular automatic methods as the baselines, namely Self-Instruct (Wang et al., 2022) and Baize Self-Chat (Xu et al., 2023b). For a fair comparison, we want the generated dialogues of different methods to talk about the same things Thus we do an additional work that we let GPT-4 generate (question, answer) pairs from the selected references and restrict the answers the questions to be found inferred from the references. Given selected reference, for Self-Instruct, we follow the Alpaca (Taori al., 2023) that we randomly select three {question, answer) pairs (from other references) as few-shot examples and add the final question from the selected reference at the end of the model input. And we let the model respond to the final question. For Baize, use the question generated from the selected reference as the seed following the way that Baize uses questions in Quora as seeds. For RefGPT, we directly use the selected reference to generate. In practice, we select 1000 passages from Wikipedia as the references to generate 1000 seed (question, answer) pairs using the GPT-4. And we generate the dialogues using these three methods with GPT-3.5-turbo for the experiment. In Table 3, give examples of evaluating the truthfulness of the dialogues generated by RefGPT. And more examples of Self-Instruct and Baize can be seen in Appendix C.
5.1.2 Result
We accuracy measure the truthfulness in the evaluation process, which is the proportion of the number of dialogues without factual errors in the total of 1000 generated dialogues. In Table 2, to our surprise, we can see that Ielf-Inst and Baize Self-Chat have striking number of factual errors in the generated dialogues on both human and GPT-4 evaluations. As the dialogues generated by Baize are multi-turn, they are more likely to contain factual errors and thus have lower truthfulness score of 47.2. By contrast, RefGPT has a truthfulness score of 97.5 with merely no factual errors. This also implicitly indicates that a model like GPT-3.5-turbe already has the ability to generate the dialogues strictly conforming to the references rather than modifying with hallucination. Another method called UltraChat (Ding et al., 2023) in Table 2 is not included, as the code has not been open-source at the time write this paper.
5.2 Further Analysis
In this section, we explore the potential influence of the reference and customization on the generated dialogues by RefGPT. For each setting in the following experiments, we generate 1000 dialogues using GPT-3.5-turbo.
For human evaluation for small sample, we randomly sample 50 English dialogues each for Alpaca, Baize, and RefGPT about factual knowl- edge. And 2 humans evaluate the truthfulness of the dialogues according to the references.
5.2.1 Dialogue Quality
For automatic evaluation for large range. in order to let GPT-4 check the factual errors without suffering from the model hallucination, we need a reference for GPT-44 to refer like RefGPT.
As RefGPT generates the dialogue according to the reference, the reference has significant impact on the quality of generated dialogues. We use the evaluation method mentioned in Sec 5.1 to evaluate

---

Table 3: Examples truthfulness evaluation on the generated dialogues by RefGPT. We prompt GPT-4 to give the explanation first and then output the judgment.
Reference
Dialogue
Explanation
Result
Hampton ... announced that he transferring the nationally basketball pro- gram of Montverde Academy Montverde. Florida, before deciding to return to Little Elm
<user> heard that Hampton is one the talented baskethall players the NBA. Can you tell me more about his career and background?
Hampton did not trans- for Montverde Academy. He had announeed inten- tion transfer ultimately decided return Little Elm High School
x
<assistant>... he transferred to Montverde Academy, where he continued to excel and multiple honors.... ...
Star Wars: Rehel Assault rail shooter video game developed and published LucasArts...
<user> What is Star Wars: Rebel Assault? <assistant> Star Wars: Rebel Assault is famous rail shooter video game, which was developed and published by LucasArts
Assistant's answers accurately reflect the information pro- vided, covering the game's sig- nificance, storyline, gameplay mode
the influence of the dialogue quality (truthfulness) in the following validations.
as noise is that many references may come from the crawled data on the websites and contain many HTML labels as noise if we do not clean the data carefully. We experiment with adding 10% and 20% nonsense HTML labels as the noise.
Table 4: The truthfulness scores of 3-turn dialogues with different ratios of reference length and noise levels.
see in Table 4, the truthfulness of the generated dialogues only slightly decreases because of the additional noise. This indicates the good robustness of generating truthful dialogues even with GPT-3.5-turb..
As
Ref. Ratio
Truthfulness
Noise Level
Truthfulness
100%
96.5
0%
96.5
50%
96.2
10%
96.2
25%
97.3
20%
94.8
5.2.2 Dialogue Structure
Reference Length As length is proportional to the amount information the reference contains, we want to find out how the reference length will influence the truthfulness of the generated dialogues. We use the dialogue template of a 3- turn dialogue, where each utterance word count of the assistant is required to be 300 words. We experiment on different lengths of reference by the proportions: 100%, 50%, and 25% of the original required length x 300 900 words).
During post-procsssing of the generated dialogues of RefGPT, we find that the input length (related to reference length) and output length (related to the required word count) will influence the success rate of obeying the dialogue template. In order to evaluate the customization ability of RefGPT, we do experiments on generating 3-turn and 5-turn dialogues. As the input length (reference length) is also determined by the required word count, we experiment with different word counts of 100, 300, and 600 for each assistant utterance to verify the success rate of obeying the dialogue template.
As shown in Table 4, surprising to see that the truthfulness scores do not decrease much as the reference lengths are greatly reduced. We find that the GPT-3.5-turke chooses to reduce the length of the generated utterances to obey reference despite violating the length requirement.
Table 5: The success rates (%) of obeying the dialogue templates with different word count settings for 3-turn and 5-turn dialogues.
Reference Quality The reference in RefGPT can vary from plain texts to cleaned documents in the vertical domain.
Word Count
Turn
w/ </chat>
w/o </chat>
100
3/5
94,4
93.5 1..4
300
3/5
94.6
91.3 88.5
In order to quantify the influence of reference quality on dialogue quality, we experiment with different qualities of references by adding addi- tional noise. To be specific, we use the original reference as the baseline. We use HTML labels
600
3/5
93.2 86.5
/
From Table 5, we can see that dialogues with fewer tokens to generate (fewer words assistant utterances and fewer turns) will lead to better

---

control over the dialogue structure with a higher success rate. We further observe that if the ending mark </chat> is successfully generated, the dialogues are more likely to obey the dialogue template with the correct number of turns.
Wei-Lin Chiang, Zhuohan Li, Zi Lin, Ying Sheng, Zhanghao Wu, Hao Zhang, Lianmin Zheng, Siyuan Zhuang, Yonghao Zhuang. Joseph E. Gonzalez, lon Stoica, and Eric P. Xing. 2023. Vicuna: An open- source chatbot impressing gpt-4 with 90%* chatgpt quality.
6 Conclusion
Zhuyun Dai, Arun Tejasvi Chaganty, Vincent Zhao, Aida Amini, Qazi Mamunur Rashid, Mike Green, and Kelvin Guu. 2022. Dialog inpainting: Turning documents into dialogs.
We present RefGPT, a new method that generates truthful and customized multi-turn dialogues using LLMs like GPT-3.5/GPT-4. Incorporating reli- able reference, RefGPT minimizes hallucination and untruthful content generation. RefGPT also allows for dialogue customization in structure, style, and content, making it flexible to generate dialogues with diversity. On the basis of RefGPT, we also use GPT-4 construct two new multi-turn dialogue datasets, RefGPT-Fact and RefGPT-Code, based on the online encyclopedia websites and Github repositories. These datasets also showcase ReiGPT's significant potential for developing de- pendable. domain-specific dialogue data required by specialized chatbots and other natural language processing applications.
Databricks, 2023. Free dolly: Introducing the world's first truly open instruction- I1m. https:// gi thub. com da wttp//abbrrkssaab/ooollooooooooo
Ning Ding. Yulin Chen, Bokai Xu, Shengding Hu. Yujia Qin, Zhiyuan Liu, Maosong Sun, and Bowen Zhou. 2023. Ultrachat: large-scale generreedd multi-roune dialogue data. https::/gitte com/ mhunlp/ultrachal
Steven Tey Dom Fecleston. 2023. Share your wildest chatgpt conversations with one click. https:// gi github.co cwt. wtt.c///occccccccc//// dom//cleeeesn wtapreeeeetoossrepp......
Ziwei Ji. Nayeon Lee, Rita Frieske, Tiezheng Yu, Dan Su, Yan Xu, Etsuko Ishii, Ye Jin Bang. Andrea Madotto, and Pascale Fung. 2023. Survey of hallucination in natural language generation. ACM Computing Surveys, 55(12):1-38.
Limitations
Andreas Köpf. Yannic Kilcher, Dimitri von Rütte. Sotiris Anagnostidis, Zhi-Rui Tam, Keith Stevens. Abdullah Barhoum, Nguyen Minh Duc, Oliver Stanley, Richárd Nagyfi, Shahul ES, Sameer Suri, David Glushkov, Arnv Dantuluri, Andrew Maguire, Christoph Schuhmann, Huu Nguyen, and Alexander Mattick. 2023. Openassistant conversations democratizing large language model alignment.
RefGPT can only strictly generate the dialogues conforming to the references even though the ref- erence itself may have factual errors. Furthermore, the generated dialogues can not be avoided to be influenced by the biases from the references. Thus the datasets RefGPT-Fact and RefGPT-Code may have factual errors and typos from Wikipedia, or bugs and malicious program codes from Github repositories.
Patrick Lewis, Yuxiang Wu, Linqing Liu, Pasquale Minervini, Heinrich Küttler, Aleksandra Piktus, Pontus Stenetorp, and Sebastian Riedel. 2021. Paq: 65 million probably-asked questions and what you can do with them. Transactions of the Association for Computational Linguistics, 9:1098-1115.
LLMs like GPT-3.5/GPT-4 have their own biases, which will also have reflections in the dialogues generated by RefGPT.
Yang Liu, Dan Iter, Yichong Xu, Shuohang Wang. Ruochen Xu, and Chenguang Zhu. 2023, G-eval: Nig evaluation using gpt-4 with better human alignment.
References
Anthropic. 2023. Introducing claude. https://ww/ anthropic.com/indevee ctmmminde/intrrdduiing----- claude.
Ji Ma, Ivan Korotkov, Yinfei Yang. Keith Hall, and Ryan McDonald. 2020. Zero-shot neural passage retrieval via ttrain- synthetic question generation. arXiv preprimt arXiv:2004.1 14503.
Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert- Voss. Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scout Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. 2020. Language models are few-shot learners.
OpenAI. 2022. Introducing chatgpt. https://openai. homm/blog/chattgge
OpenAL. 2023. Gpt-4 technical report.
Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton,

---

Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul Christiano, Jan Leike, and Ryan Lowe. 2022. Training language models to follow instructions with human feedback.
Baolin Peng, Chumyuan Li, Pengcheng He, Michel Galley, and Jianfeng Gao. 2023. Instruction tuning with gpt-4.
Rohan Taori, Ishaan Guirajani, Tianyi Zhang, Yann Dubois, Xuechen Li, Carlos Guestrin, Percy Liang, and Tatsunori B. Hashimoto. 2023. Stanford alpaca: An instruction-followii llama model. https:// watpssssaab/sttoood._alls..
gi thub.
Yizhong Wang, Yeganeh Kordi, Swaroop Mishra, Alísa Liu, Noah A Smith, Damiel Khashabi, and Hannaneh Hajishirzi. 2022. model with self generated instructions. arXiv preprint 10550.212..15660
Aligning language
Can Xu, Qingfeng Sun, Kai Zheng, Xiubo Geng, Pu Zhao, Jiazhan Feng. Chongyang Tao, and Daxin Jiang. 2023a. WizardIm: Empowering large language models to follow complex instructions.
Canwen Xu, Daya Guo, Nan Duan, and Julian McAuley. 2023b. Baize: An open-source chat model with parameter-efficient tuning on self-chat data, arXiv preprint arXiv:2304.01990

---

A Dataset Card
A.1 RefGPT-Fact
RefGPT-Fact is a dataset comprising 100k multi-turn dialogues focusing on factual knowledge. There are two versions, with the English version containing 50k dialogues based on the English Wikipedia, while the Chinese version consists of 50k dialogues sourced from the widely-used Chinese online encyclopedia, Baidu Baike.
Since most of the passages in the English Wikipedia and Baidu Baike are written by individuals or unofficial organizations, many of the passages are not commonly seen in everyday life. We use GPT- 3.5-turbo API to quickly filter out the uncommon passages by asking it "Do you know xxx? If yes, return <yes>. If no, return <no>." where xxx is the title of the passage".
A.2 RefGPT-Code
RefGPT-Code is comprehensive dataset that consists of 76k multi-turn dialogues on programming, including 37k English and 39k Chinese dialogues. As illustrated in Figure 3, it encompasses a wide range of coding scenarios about discussion, creation, and bug fixing using various programming languages. The dataset utilizes the public Github dataset available on Google BigQuery, with overlapping data between the languages.
11%
17%
Ruby 4%
C# 6%
24%
discussion 37%
Python
12%
7%
creation 39%
JavaScript 10%
Markdown 7%
PHP 10%
HTML 10%
Figure 3: Composition of RefGPT-Code Dataset including English and Chinese.
"This method is hased on knowledge of GPT-3.5-turbo, where recall rate is limited, A more recommended way is using the access rate to filter out the uncommon passages.

---

Table example of the prompt for generating the English RefGPT-Fact data. (dialogue templltte is replaced by the dialogue template mentioned in Sec 3.3.
#MProvided Informationtit (reference) Based the iNfrovided Informationitt above and its relevant topic, expand into multi-round conversation. The conver sation requires you to act the chathot Assistant and interact with a human, helping to solve the requests raised by the human. The human will ask multiple various questions/requests to the Assistant based on the information above (but the conversation should not include expressions like "according to the above information") and the subsequent questions/requests will be follow-up based on the previous conversation history. For reasonable question/request posed by Human, Assistant should provide possible, offering further explanations examples. For unreasonable requests from (those that harmful society, immoral, illegal), Assistant will refuse answer and explain the reason to avoid such actions.
detailed
answering, while also providing reasonable advice
#Conversation Plantl Example: "<chat><Human 1>: (Word count requirement: x words)xXX <Assistant 1>: (Word count requirement: x words) XXX <Human 2>: (Word count requirement: x words)XXX <Assistant 2>: (Word count requirement: x words) XXX </chat>", "XXX" is the requirement for the current conversation content of that role, and "(Nord count requirement: words)" specifies the minimum word count requirement for utterance of Human Assistant. It must be noted: the conversation starts with <chat> the beginning of the multi-round conversation and ends with </chat> the end of the multi-round conversation. The following conversation follows this #Conversation Plan# word count requirements: "[dialoque_template"" a total (nunber_of_turns) turns of conversation.
siialogue_template <chat><Human (word count: words)asks question <Assistant 1>: (word count: 200 aosdess [+detailed explanation] <Human 2>: (word count: 150 words) asks
from the perspective real life <Assistant 2>: (word count: 100 words)answers [+detailed explanation] <Human 3>: (word count: 50 furthe asks question <Assistant 3>: (word count:
150 words)answers [+detailed explanation] </chat>

---

Table 7: An example of the prompt for generating the English RefGPT-Code-ds data.
#MProvided Informationtit (reference) Based the itfrovvded Informationitt and its relevant topic, expand into a multi-round conversation. The conversation requires you to act as the chatbot Assistant and interact with a human, helping to solve the requests raised by the human. The human will ask multiple various questions/requests to the Assistant based on the information above (but the conversation should not include expressions like "according to the above information"), and the subsequent questions/requests will be follow-up based on the previous conversation history. For detailed
reasonable question/request pased by Human, Assistant should provide as possible, offering further explanations examples. unreasonable requests from Human (those that harmful society, immoral, illegal), Assistant will refuse to answer and explain the reason not answering, while also providing reasonable advice to avoid such actions.
#Conversation Plantt Example: "<chat><Human 1>: (Nord count requirement: x words)xXX <Assistant 1>: (Word count rement: x words) XXX <Human 2>: (Word count requirement: x words) XXx <Assistant 2>: (Word count requirement: x words) XXX </chat>", "XXX" is the requirement for the current conversation content of that role, and "(Word count requirement: words)" specifies the minimum word count requirement for utterance of Human Assistant. It must be noted: the conversation starts with <chat> the beginning of the multi-round conversation and ends with </chat> end of the multi-round conversation. The following conversation follows this #Conversation Plant word count requirements: "[dialogue_template)" a total (nunber_of_turns) turns of conversation.
ddialogue_templatee <chat><Human 1>: (word count: words)makes a request about writing the code <Assistant 1>: (word count:
250 words) [+detailed explanation] and give code examples <Human 2>: (word count: 100 words)asks in a young person's tone about further modifying the
300 words)answers [+detailed explanation] and give code examples <Human 3>: (word count: 20 words)asks from the perspective of real life about further how to
code <Assistant 2>: (word count:
the code <Assistant 3>: (word count: 250 words)answers [+detailed explanation] and give code
examples </chat>

---

Table 8: An example of the prompt for generating the English RefGPT-Code-cr data.
##Provided Informationtt (reference) Based the ##Provided Information#t and its relevant topic, expand it into multi-round conversation.
assignment problem difficulty related to the above has an idea / requirement / task wants to solve it with a computer program, but doesn't know how to do it. But Human doesn't t know that the above code exists, so it can't be mentioned in conversation. Assistant needs to organize the above code into answers (which cannot be found by Human) according to Human's ideas, write specific program code for Human and explain it in detail so that Human's ideas can be realized. Based on this idea, Human would ask multiple questions and requests for specific code written by the Assistant, which will be follow-ups based the previous conversation history. For unreasonable requests from Human (those that harmful society, immoral, illegal), Assistant will refuse explain the reason for not answering, while also providing reasonable advice to avoid such actions.
#Conversation Plantt Example: "<chat><Human 1>: (Nord count requirement: x words)xXX <Assistant 1>: (Word count rement: x words) XXX <Human 2>: (Word count requirement: x words) xXx <Assistant 2>: (Word count requirement: x words) XXX </chat>", "XXX" is the requirement for the current conversation content of that role, and "(Word count requirement: words)" specifies the minimum word count requirement for utterance of Human Assistant. It must be noted: the conversation starts with <chat> the beginning of the multi-round conversation and ends with </chat> end of the multi-round conversation. The following conversation follows this #Conversation Plant word count requirements: "[dialogue_template) a total (nunber_of_turns) turns of conversation.
(dialogue_templatte) <chat><Human 1>: (word count: 50 words)asks with curiosity about creating code <Assistant 1>: (word count: 300 words)answers [+detailed explanation] and give code examples <Human 2>: (word count: 180 words)asks a question about further using the code <Assistant
2>: (word count: 250 words)answers [<detailed explanation] and give code examples <Human 3>: (word count: 150 words)asks a question about further explaining the code <Assistant 3>: (word count: 300 words) answess [+detailed explanation] and give code examples <Human 4>: (word count: 50
expresexs his/her needs and asks the Assistant for help about further using the code <Assistant 4>: (word count: 200 words)answers [+detailed explanation]<<ccht>

---

Table 9: An example of the prompt for generating the English RefGPT-Code-bg data.
#MProvided Informationtit (reference) Based the ##Prrvvied Informationitt above and its relevant topic, expand it into multi-round conversation. Human will write piece code with bugs based on the given code above (however, Human needs to hide the presence of the given code in the conversation, and it cannot be mentioned). They will then ask Assistant for help in fixing the bugs. Assistant needs to identify the mistakes in Human's ' code based on the given code above (but given code cannot be discovered by Human, and it cannot be mentioned in the conversation) and provide detailed explanations on how fix the bugs, along with more explanations examples if necessary. Afterward, Human and Assistant will continue the conversation around this code. For unreasonable requests from Human (those that harmful society, immoral, illegal), Assistant will refuse to answer explain the reason for not answering, while also providing reasonable advice to avoid such actions.
#Conversation Plantt Example: "<chat><Human 1>: (Nord count requirement: x words)xXX <Assistant 1>: (Word count requit: x words) XXX <Human 2>: (Word count requirement: x words)xxx <Assistant 2>: (Word count requirement: x words) XXX </chat>", "XXX" is the requirement for the current conversation content of that role, and "(Word count requirement: words)" specifies the minimum word count requirement for utterance of Human Assistant. It must be noted: the conversation starts with <chat> the beginning of the multi-round conversation and ends with </chat> end of the multi-round conversation. The following conversation follows this #Conversation Plant word count requirements: "[dialogue_template)" a total (nunber_of_turns) turns of conversation.
(dialogue_template) <chat><Human 1>: (word count: words)asks from the perspective of real life about writing a piece code with bugs and show the detailed code <Assistant 1>: (Word count: 250 answers [+detailed explanation] and tell Human about the error location in the code, then provide correct piece of code <Human 2>: (word count: 100 words)makes a request about turr using the code <Assistant 2>: (Word count: 200 words)answers [+detailed explanation] and give code examples <Human 3>: (word count: 50 words)asks with curiosity about further explaining the code <Assistant 3>: (Word count: 250 words) [+detailed explanation]<<chat>

---

B Dataset Examples
Table 10: An example of Chinese ReiGPT-Fact.
Reference
R. 2 -
4 ST
HE ME F.
Mitil
4 #1
WALDR IR (Beijing) M.K.R. *
J H4T 11 E B0 (EX 4
ABA 2. 4. HORD #. X/ 2
tu
MOMT 100 22 /IKE. R22 Ai
10455 11 14 2 C4 1E
1. TX
2015 403 7A2277077
M..
A. 1E B00
20154
112014 K6.9% 2015477 R, 11
BE HE INF LIE (Shnnghai """""""""""""""""""""""""""""""" #
TE AEA KAL BB W 1 #
<user> / EE 33 2ww.
<assistant> -ル B
14 ti
#ill - . SEE
<user> 1 1
<assistant> 1,
A FE 2E
E #
# TI 1A000 Jt BATTERY
A AAA.S 3EXX B85Y5
<user> 13
<assistant> Ho 201 63 13 17 2.
A2 H40 6.9% प
* 16 2521705555
#1 X,
<user> / 15,
<assistant> 1 " B 3 1 #
: SEX. 11
114 0 440
# BA

---

Table 11: An example of English Re/GPT-Fact.
Reference
Joomla O. also spelled Joomla! (with an exclamation mark) and sometimes abbreviated as J1, is free and spencs content management system (CMS) for publishing web content websites. Web content applications include discussion forums, photo galleries, Coommerce and user communities and numerous other web-based applications. Joomla is developed by a community of volunteers supported with the legal, organisational and financial resources of Open Source Matters, Inc. Joomla written PHP, uses object-oriented programming techniques and software design pattems, and stores data in MySQI. database. has a software dependency on the Symfony PHP framework. Joomla includes features such as page caching. RSS foeds, blogs, search, and support for language internationalisation. is built a model view- controotrrller web application framework that can be used independently of the CMS. Around 6,000 extensions are available from the Joomla website, and more are available from other sources. of 2021, was estimated to the fifth most used CMS on the Internet, after WordPress, Shopify, Wix and Squarespace, Overview Joomla has a web template system using a template processor. Its architecture is front controller, routing all requests for non-static URIs via PHP which parses the URI and identifies the target page. This allows support for more human-readable permalinks. The controller manages both the frontend. public-facing view. and a backend (GUI-driven) administration interface. The administration interface (a) stores management and content information within a database, and (b) maintains a configuration file (, usually located the file system root of the Joomla installation). The configuration file provides the connection between the server, datahase and file system and facilitates migrating the website from one server to another. The backend interface allows website operators to manage users, menus, extensions and web content ... (About 900 words)
<user> recently came across joomla and i was wondering if you could tell system (ems). how does it compare to other ems like wordpress, shopify, wix, and squarespace? what type of users does it cuter to, and what kind of applications can be built using joomla?
about this content management
<assistant> Joomla is a free and open-source content management system (ems) that is widely used for creating web content on websites. it is designed cater users who have basic website creation skills and ideal for building various reb-based applications such as discussion forums, photo galleries, e-commerce sites, user communities and much more. joomla developed and maintained by a community of volunteers, with open source matters, inc. offering legal, organizational, and financial resources support the project.
Joomla built using php, object-oriented programming techniques, and software design patterns. it stores data a mysql database and relies on the symfony php framework. some of its features include page caching. rss feeds, blogs, search capabilities, and support for language internationalization.. it also comes with a veweotolie web application framework that can be used independently of the cms.
As of 2021. 2021laoomla is estimated to be the fifth most used ems on the internet, coming after wordpress, shopify, wix, and squarespace. with around 6,000 extensions available on the joomla website and more from other sources, it provides ficxibility and extensibility to users.
Comparing it to other cms, joomla's main advantage is its open-source nature and extensive customization options through its wide range of extensions. while wordpress is more user-friendly and has a larger community, joomla strikes a balance between ease of use and flexibility. shopify, wix, and squarespace, on the other hand, are more geared towards non-technical users and have limitations in extensibility when compared to joomla. In summary, joomla is flexible and customizable option for users who have some website creation skills and require a cms to build diverse web applications.
<user> Thanks for explaining that. I've heard that Joomla has a unique web template system and a front-controller architecture. Can you explain how these features work together and how they benefit the user? Also, what kind of risk munagement, buckup, and recovery options are available with Joomla?
<assistant> Joomla's 's web template system uses a template processor, and its architecture is based on front-controller model. this means that all requests for non-static uris are routed through php, which parses the uri and identifies the target page. this approach allows for more readaleee permalinks, improving the user experience and search engine friendliness of website.
Joomla's 's front-controller architecture manages both the frontend, public-facing view. and backend, gui-driven administration interface. the backend allows website operators to manage users. menus, extensions, and web content. the system stores management and content information within database and maintains configuration file to manage the connection between the server, database, and file system.
As for risk management, backup, and recovery. joomla itself does not have built-in features for these. however, third-party extensions and standalone products are available to provide backup and recovery functionality.. it is the website operator's responsibility to implement and manage these tools to ensure the integrity and security of their website.
To summarize, joomla's web template system and front-controller architecture offer improved user experience. search engine optimization. and ease of administration. although the core ems doesn't provide backup and recovery features, third-party options fill this gap and deliver comprehensive risk management and data protection capabilities.
(Two more turs are not shown because of the limited space. The whole dialogue contains about 1055 words.)

---

Table 12: An example of Chinese RefGPT-Code-ds.
Reference
describe Remover: Team
titggghhhuuucciieeette double Github Client
Jet(:github_team { double Githhb Team id: 1, name: 'Owners') let(:tean) Remover: Team. new (ewgthuub_ccleent github_team) }
describe 'tname'
'returns Github team name" expect (team. name) eq tean name)
end
#H TAAALIAN 22 T AnHLOL 12 H (म1
describe Remover Team
let client) { double Github Client :)
deithhububb_tee team) { d Githhb Team id: 1, name: 'Owners') }
(et(:team) Remover: Team. new (eithhhuubcleen githbbttemm }
describe "#name' do it 'returns Github team name do expect ( (team. name) to eq(github_teen team name)
end end
<assistant> S Be (t Remover: B8 #
d fit "uused" 7 B1 2
fu tu
[i] 'false. BE.. 2r a
Remover::Team Team ST.
G BBB
github_cliem
('Github Client')
github_team double double('Githul Githhb Team" id: 1, name: 'Owners' github_team)
team Remover: :eam. new wwwthhgbcclleeeeeeeeeeee
26. T D "team used i d Be
MA TH A false"
A 'eeam.delete team o T ATBA
if not used delete_
end
<user> N TH
A 3
An #memmerms_aamm aa fncpossoii amount'. 7 io
MRRE
14
<assistant> 'eemover:Team" fi #b bi St मे. in ग
11 N #members amount" AĀ #repositories amount 7 Xæ म 1 # M1 S% AS # N $
A è
#
E
G
P
My team members' AI team repositories
(Two
shown because of the limited space. The whole dialogue contains about 1400 words.)

---

Table 13: An example of English RefGPT-Coodeds.
Reference
-x- coding: utf-8
import datetime
from south. db import db
from south. import SchemaMigration from django.db import models
class Migration (SchemaMigration)
def Corwards(self. orm):
Adding field 'Question. order''
abd (u' survey--qusttion 'order',
self ('django. db models. fields. IntegerField (default=8 )
deep_default=False
<user>
-+- coding: utf-8
import datetime
from south. db import db
from south. import SchemaMigration from django. db import models
class Migration(SchemaM//in (SchemaMigration))
the context of provided code, see there's a Django migration for adding a color field to an Idea model. Can you explain how a migration works in Django and how to apply this specific migration add the 'color' field to the 'Idea' model in an existing Django project?
cassistant> Sure! In Django, migrations are a way managing and propagating changes made to your models (which affect the database schema) over time. A migration is essentially a set of instructions that the Django migration system uses to modify your database schema to match the current state of your model.
Applying migrations to your Django project is a two-step process:
1. Creating the migration (migration file generation): When you make changes to your model, you need to inform Django ahout the changes. You do this by running the makemigrrttoon"' command, which creates new migration file hased on the alterations in the model.
2. Applying the migration (migration execution): After generating the migration tile, you need to apply it to your database to update the schema. You do this by running the 'migrate' command, which processes the migration files sequentially and updates the database schema accordingly.
In the provided code, migration is created to add a 'color' field to the 'Idea' model. To apply this migration in your Django project, follow these steps:
1. First, ensure that the migration is in the correct location within your Django project. Migration files are typically placed within a folder named 'migrations' inside application folder. If the Django app containing your 'Idea' model is named "brainstorming" the migration file should be located at traainsoominggmiirratonnsss
2. Next. check if the migration has been generated by running 'python manage.p py makemigrations This will generate new migration tiles hased on the changes your models. Since the code provided already includes migration file, this step may not be necessary. However, it's recommended run this command ensure all migrations accounted for.
3. Now, apply the migration to update your database schema by running 'python manage.py migrate'. This command will process all pending migration files and apply them the database.
By following these steps, you'll successfully apply the migration to add the 'color' field the 'Idea' model in your existing Django project. Note that this migration reversible, meaning you undo the change by executing your
the 'python manage.py py migrate <app names <migration
before, your migration command. In this case, the '<app name>' would be 'brainstorming' and '<migration. name before your migration>> would be the name of the migration immediately preceding the adding the 'color' field.
(Two
shown because the limited space. The whole dialogue contains about 1250 words.)

---

Table 14: An example of Chinese RefGPT-Coodecr.
Reference
#include "ScriptPCH. #include "zulaman. h²
#define BELLL..RMOVE_AAANE #define SPELL_PUSH_ MOJO 43923
43732
#define ENTRY_FOREST__ROO 24396
class npc_foresttfrog public CreatureScript
<user> - -
H BRA AA MIN ME 1
8. # J #
<assistant> ti 51 sR, I0 Creaaa
"j'npe forest dii3.. 18 11
E'nne forest frog F 321
19 A 'pe forest frogAl, 1 1E3 MER 19, X
* SSriptedAA # "ScriptedAl" 222 AL
5 Resee" "EnterCombat"
T
#l BI LE RO #o
'pellliit BS 1
E
PI "
ID By SPELL REMOVE AMANI CURSE" (1TT) EE 18 (MYEE) Hn &
#
E
"ScriptPCH. h" #include "zulaman.hr
#include
#define OPELL_REMOVS REMOVE_AMANT AMRSE 43732
#define SPELL.PUSH_MON MOJO 43923
#define ENTRY_FOREST_FRO 24396
class npc_forest_frog public CreatureScript
public: foc_ CreatureScript (oroicrrtttnnnccfofeeetteeeee O
struct noc_foorst_ public ScriptedAI
<user> H A
<assistant> JH 1ET th 3 3 Frando % 100 M 12
N # 21,, SR BOREE 10%.
RJ (
if (rand () 100 10) 10%
DoCast(caster. PEELL_PUSH_MOJO, true);
else DoSpawnRandam()
(One shown because of the limited space. The whole dialogue contains about 1 150 words.)

---

Table 13: An example of English RefGPT-Coddecr.
Reference
package ch. zhaw. mapreduce plugins wocket.icmi.......e.
import javax. inject. Inject;
import ch. zhaw. mapreduce CombinerIIntructiion;
import ch. zhaw. mapreduce iapInstructions
import ch. zhaw. mapreduce. Reduce Instruction;
<user> am working project that involves different tasks like mapping, reducing, and combining. need task runners that execute these tasks dynamically based on the received instructions. also need to differentiate between mapping and reducing tasks execule them correctly. Can help <assistant> Certainly! Let's start by creating interface called "unskRunnerFactory with single method "craateTasRRunner', that takes 'Agen(Task' object input and returns 'TaskRunner' object. Later, we'll implement the logic for differentiating between the mapping and reducing tasks. Here's the interface:
Java code that achieve this?
import javax. inject. Inject;
public final class TaskRumeerFactory.np implements TaskRunnerFactory {
private final ByteArrayClassLooder classLoader new ByteArrayClassLoader
()
private
final MapTaskRunnerFacttry mtrFactory;
private
final ReduceTTaskuuneefacctory rtrFactory;
@Inject
public
(askuumerraattrryynggttottttaain ReduceeTaskRumeeFFace rtrFactory)
mtrFactory,
this .wt.Facttrryyyoo = mtrFactory;
this .rtraactory = rtrFactory;
}
@0verride
public TaskRunner createTaskRunner (Agenttass task) throws
InvelldAgenTasskeeeppiinn
if (task instanceof MapAgentTask)
// Handle MapAgentTask
else if (task instanceof ReduceAgentTask) {
// Handle ReduceagentTask
else throw new wew.iidggentaakExeeeeeeeee ("Cannot Handle: task)
In this implementation, we first check if the 'AgentTaakk' is an instance of 'MapAgentTask" 'ReduccAgen'TTssk Based that, we'll execute the respective logic. We also use custom BByteArraCllssLoode to load the classes dynamically and call different getters based on the task type. Inside the 'ereate TaskRunner()) method, "RentccAAgenTaask instances:
now implement the logic to handle 'MapAgentTask" and
// For MapAgentTask MapAgentTask mt (MapAgentTask) task; MapInstruction mapInstr loadClass (mt
www.eeetnsstruuttioneeeee (), mt
getMapInstrucctinn]] MapInstruction. class);
(Two shown because the limited space. The whole dialogue contains about 800 words.)

---

Table 16: example of Chinese RefGPT-Code-bg.
Reference
class BTree:
def init__ (self, value): self root Node (value)
<uscr>
Adssisstatt # T M8. BOC BEEETTEE. SEW
I F SI??? F:
class BTree:
def isser value):
def helper(cur_node value): if value cur_node. value: if cur cur_node.rrghtt cur_node cur_node right helper (cur_node, value)
else cur_node right Node (value)
else
if cur_node left:
cur_node cur_node left
helper (cur_node, value)
else cur_node left Node (value) helper (self. root, value)
# # 14 B E T
!
<assistant> iR m x HE FV Bb REE 12
Fi'neert' 7i (2: "helper" 40
100 Tf W 1e
* return helper(cur _odee value) "M hheper(cur__ooe _ooee value)', 33 SE 1
class BTree:
def insert value): def helper value):
if value cor_node.. value: if cur_node.right: cur_node cur_node right return helper (cur_nncu__ value)
else
cur_node right (value)
else
if cur_node left:
cur_node cur cur_node, left return helper (cur_node, value)
cur_node left (value)
return
helper (self. f. root, value)
WIDERT 1
(Three more turns are not shown because of the limited space. The whole dialogue contains about 1300 words.)

---

Table 17: An example English RefGPT-Codd--g.
Reference
(function($)
var methods =
init function (options) {
var defaults
element
selector
shadowSize "30"
shadowColor "#222D3A",
shadowFade "0.3",
shadowAlpha "0.03",
<user> Hey, I was trying to implement long shadow effect on text element using, JQuery, but cannot seem get the desired effect. I've written this code, but it seems to have some bugs. Can you help me fix it? Here's 's the code:
function applyLongshadow (o)
for (var i <= textElement textileeeettssacoowiieee i++) { o.textElement. shadowSize 1; var y o. textElement. shadowSize i; shadow
"px
"px ";
if (o. textElement, shadowFade Fade shadow xcolor gradientlevel (darkerxx, txc, fadeText-1, fadeText)
fade) {
<assistant> see that you would like apply long shadow effect issues your code that might be causing the undesired effect. Firstly, you using gradentgra in your code, but you haven't included the 'xcolor" library defined the variahles 'darkerTX', 'txc", and 'fadcText'. You instead use the "avoolor.go wtttorrgrrdieeteeeeeeee methoo library, and sure include library your project. Here's corrected version your code:
element using jQuery. I noticed couple of
methodf romthee xcolor"
function applyLongShadow (o) {
for (var 8; i <= o. textElement. www.eleeeett.saadwwaeeeeeeeeeeee shadowSize; i++) {
var x = o. texetlleeeent shadowSize 1;
var = o. textElement. wttpeeeeett....siiieeeee shadowSize i; var shadow
"px
"px ";
if (o. textElement. shadowFadd & < fade) { var tweenedTX = x.oool gradientlevel (darkerTX, txc, fade i, fade );
Please make sure to include the "S.xcolor" library your project, and let me know you need further assistance.
(One more turns are not shown because of the limited space. The whole dialogue contains about 550 words.)

---

C Truthfulness Evaluation Examples
Table 18: An Example of truthfulness evaluation on the generated dialogues by Self-Instruct (Wang et al., 2022).
Reference
Dialogue
Explanation
Result
The fifth series of British talent competition programme Britain's Got Talent was broad- As Piers Morgan had departed from the programme the previous year, and the schedule of Simon Cowell made him unable to attend auditions, the producers arranged for Amanda Holden to be joined by David Hasselhoff and Michael McIntyre on the judging panel. Due to Hassel- hoff's i schedule during filming of the auditions, Louis Walsh stepped in as guest judge for the sessions he could not attend
<user> Who were the judges for the Sth series of Britain's 's Got Talent, and what changes occurred in the judging panel that year? <assistant> The judges for the 5th series Britain's ' Got Talent were Simon Cowell, Amanda Holden, Piers Morgan, and Kelly Brook, with Ant and Dec as hosts. That year, Kelly Brook was replaced by ro- tating panel of guest judges due to her absence after just six days of filming.
<|> The Assistant incorrectly states that the judges for the 5th series of Britain's 's Got Talent were Simon Cowell, Amanda Holden, Piers Mor- gan, and Kelly Brook. The correct judges were Amanda Holden, David Hasselhoff, and Michael McIntyre, with Simon Cowell only present for the live episodes. Piers Morgan had departed from the program the previous year, and Kelly Brook was not involved at all.
x
cast on ITV
<2> The answer claims that Kelly Brook was replaced by rotating panel of guest judges, but this is incorrect. In the 5th series, there were no guest judges other than Louis Walsh, who temporarily replaced David Hasselhoff when he could not attend the London auditions.
A consumer economy describes cconomy driven consumer spending percent of its gross domestic product, opposed to the other major components of GDP (gross private domestic in- vestment, govermment spending. and imports netted against ports). In the U.S., it usually said by economists, including Henry Hazlitt's Economics One Lessonihat 70% of spending consumer-based, but this number is disputed by economists like Businessweek columnist Michael Mandel
<user> What are the main components of gdp and how does a consumer economy fit into this?
The Assistant's answer correctly identifies the main components of GDP, which are consumption, investment, government spending, and net exports. The answer also accurately explains how consumer economy fits into this, stating that consumption is the largest component of GDP in most countries and thus the behavior of consumers drives economic growth.
<assistant> The main com- ponents of GDP are con- sumption, investment, gov- ermment spending, and net exports. consumer econ- omy is economy that driven by the spending and saving behavior of in- dividuals. Consumption the largest component GDP most countries, including the United States. Therefore, the behavior consumers a major driver economic growth.

---

Table 19; An Example of truthfulness evaluation on the generated dialogues by Baize (Xu et al., 2023b).
Reference
Dialogue
Explanation
Result
Scoat Fell is a fell in the western part of the Finglish Lake District. It stands at the head of the Mosedale Horseshoe with its back to Ennerdale. Paths lead Scoat Fell from Ennerdale over Steeple, from Wasdale over Red Pike, and along the ridge from Pillar. Topography The Westem Fells occupy a triangular sector of the Lake District, bordcred the River Cocker to the north cast and Wasdale the south east. Westwards the hills diminish toward the coastal plain Cumberland, At the central hub of the high country are Great Gable and its satellites, while two principal ridges fan out on either flank of Ennerdale, the western fells in effect being a great horseshne around this long wild valley, Scoat Fell is on the southern arm...
<user> Can you tell me the location of Scoat Fell in the English Lake District? <assistant> Sure! Scoat Fell is located in the western part of the Finglish Lake District, specifically in the Wasdale area, It one of the highest peaks in the Lake District, with height of 841 metres 2.759 feet.
<I> The location of Scoat Fell is mentioned inaccuratcly as being in the Wasdale area, but it is actually in the westem part of the English Lake District the head of the Mosedale Horseshoe with its back to Ennerdale.
x
<2>
The provided access routes to Scoat Fell incor- rect. According to the pro- vided information, the paths Scoat Fell are from Fineerdale over Steeple, from Wasdale over Red Pike, and along the ridge from Pillar.
<user> How access Scoat Fell? <assistant> There are several ways access Scoat Fell. One of the most popular routes start from the Wasdale Head village and hike up via the Brown Tongue path. Another option Lake District Youth Hostel in Eskdale and hike up via the path that leads to Burmmoor Tarn.
start from the
<user> Are there any other peaks near Scoat Fell that can visit?
<assistant> Yes, there are several peaks near Scoat Fell that are worth visiting. One of them is Scafell Pike, which is the highest mountain in England and is located only a few miles to the east of Scoat Fell. Other peaks nearby include Pillar, Haycock, and Red Pike.
Morchella esculenta, (commonly known as common morel, morel, yellow morel, true morel, morel mushroom. and sponge morel) is a species of fungus in the family Morchellaceae of the Ascomycota. It is one of the most readily recognized of all the edihle mushrooms and highly sought after. Each fruit body begins as a tightly compressed, grayish sponge with lighter ridges, and expands to form a large yellowish sponge with large pits and ridges raised on large white stem. The pitted yellow- brown caps measure broad by tall, and are fused to the stem at its lower margin, forming continuous hollow. The pits are rounded and irregularly arranged, The hollow stem is typically long by thick, and white yellow. The fungus fruits under hardwoods and conifers during short period in the spring. depending on the weather, and also assnciated with old orchards, woods and disturbed grounds... ...
<user> Can you tell me about morchella esculenta and its main features? <assistant> Morchella esculenta is a species of edible mushroom, also known as the morel mushroom. Its main features include a distinctive honeycomb appearance and a meaty texture.
The dialogue correctly de- seribes the main features of Morchella esculenta, its habi- tat, alternative types of morel mushrooms, and the flavor profile of Morchella esculenta. The answers provided by the Al are consistent with the pro- vided information, and there are no factual errors.
<user> Where can morchella esculenta be found?
<assistant> Morchella esculenta can typi- cally be found in areas with rich soil, such as forests and grasslands. It is commonly found in North America and Europe during the spring season.
<user> Are there any other types of morel mushrooms besides morchella esculenta? <assistant> Yes, there several other types of morel mushrooms, including Morchella conica and Morchella deliciosa. However, Morchella esculenta is the most commonly consumed morel mushroom.
<user> Can you describe the flavor of morchella esculenta? <assistant> Morchella esculenta has complex umami flavor that is often deseribed as nutty, earthy, and slightly smoky. pairs well with butter and herbs, making a popular ingredient many recipes.


Reviews:
[{"review_id": "xyvZdHGkCG", "reviewer": "Reviewer_7RQX", "summary": "", "questions": "- A.) Is the number of turns limited to 4? It appears so from Table 1. And are those 4 turns per user and assitant each, or 4 overall turns?\n- B.) I found Section 5.1.1 very hard to understand the way it is currently written, especially the description of the evaluation setup for Self-Instruct and Baize. What do you mean by \"we follow the Alpaca that we randomly select three {question, answer} pairs (from other references)...\"? And what is a seed? What are the {question, answer} pairs for? They are nowhere to be seen in the examples in Table 3 and Appendix C. Are the {question, answer} pairs used instead of references when generating dialogues? I'm sorry, I just find it impossible to follow what is going on in this whole evaluation.\n- C.) For the reference quality experiments in Section 5.2.1, did you try using some linguistic noise (e.g., replacing certain words with random words, shuffling sequences of words, removing negations, etc.) instead of adding random HTML labels? The model might simply ignore the HTML code for the most part, since it's typically not there in references.\n- D.) From what noise level did the truthfulness become really bad, such as below 50%? If 20% of noise only reduced it from 98% to 96%, it makes me wonder if even 50% of noise would not have that big of an impact. Or otherwise I don't understand what the purpose of adding this type of noise really was in this experiment.\n\nEDIT: Thank you for answering my questions and providing further clarifications. I have increased my scores accordingly.", "limitations": "", "rating": 3, "confidence": 3, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your valuable and insightful feedback!\n\nW1: The creation of conversational style and content pools could require a substantial amount of manual work and domain knowledge to generate 100k dialogues.\n\nA1: RefGPT can work well even without style and content pools. These additional settings only need a small amount of manual work for further customization. For example, in the generation of the RefGPT-Code dataset, the style pool, e.g. “asking like a young man”, has a size of about 10. And content pool, e.g. “writing、discussing the code”, has a size of about 20. We will supplement these details in our paper later.\n\nThe domain knowledge is only needed for designing the task description in the dialogue setting and can be reused to generate a lot of dialogues. For example, in Figure 1, the code creation scenario has the task description: The user has a task related to the above code and wants to solve it with a computer program ... We generate 30k dialogues of code creation using only this description.\n\n**In RefGPT, the diversity of the dialogues mainly comes from the references rather than different designs in the content pool and style pool.**\n\n\n\nW2: Some sections are not written very clearly (see an example in the questions section).\n\nA2: See the explanation in Q2.\n\n \n\nW3: Truthfulness evaluation based on automatic evaluations performed by GPT-4 is not trustful. Using human annotators would have been helpful.\n\nA3: Thanks for pointing this out. In the evaluation, we use the GPT-4 to compare if the assistant's responses are contradictory to the references, instead of letting GPT-4 judge the truthfulness of the responses with its own knowledge base. \n\n**We have supplemented the human evaluation following a similar pattern.** We randomly sample 50 English dialogues each for Alpaca, Baize, and RefGPT about factual knowledge. And 2 humans evaluate the truthfulness of the dialogues according to the references. The accuracies of Alpaca, Baize, and RefGPT are 54.0, 50.0, and 98.0. The results by humans have a similar trend as the GPT-4 evaluation (50.2, 47.2, 97.5).\n\n \n\nW4: Reproductability: hard pressed to reproduce the results.\n\nA4: We have provided our prompts of RefGPT in Appendix A and the dataset examples in Appendix B. We will release our code and datasets later and they can easily be reproduced.\n\n \n\nQ1: Is the number of turns limited to 4? Are those 4 turns per user and assistant each, or 4 overall turns?\n\nA5: Yes, we limit the number of turns in the RefGPT datasets to 4. However, the RefGPT method can generate dialogues of more than 4 turns. And one turn contains one user question and one assistant response.\n\n\n\nQ2: The description of the evaluation setup for Self-Instruct (Alpaca) and Baize in Section 5.1.1 is hard to understand.\n\nA6: Alpaca uses three {instruction, output} pairs as the seeds to generate a new instruction, as we call {question, answer} pairs in the paper. Different from Alpaca using human-written instructions as the seed pool, we use the GPT-4 to generate instructions from selected references and constrain the answers (outputs) to be included in the references. The answers are double-checked by humans. The truthfulness evaluation process of Alpaca is as follows:\n\n```\n#Seed Pool:\n{question1, answer1, reference1}, {question2, answer2, reference2}, ...\n\n#Generation:\nInput:\n{question1, answer1}\n{question2, answer2}\n{question3, answer3}\nquestion 4\nOutput: the response to question 4. (response 4)\n\n#Evaluation:\nGPT-4/Human judges if response 4 to question 4 is correct (according to reference 4).\n```\n\nBaize only uses the questions as seeds, which are generated in a similar pattern as Alpaca.\n\nWe will add more examples to better clarify the evaluation process according to your advice.\n\n \n\nQ3: For the reference quality experiments in Section 5.2.1, did you try using some linguistic noise (e.g., replacing or shuffling words, removing negations, etc.) instead of adding random HTML labels? \n\nA7: We do not use linguistic noise because such noise changes the meaning of the references and leads to untruthful dialogues. We want the generated dialogues to strictly conform with the references instead of revising the references even if the references are incorrect. **The motivation for using HTML labels as noise is that many references may come from the crawled data on the websites and contain many HTML labels as noise if we do not clean the data carefully.** Due to the reliability of GPT-4, RefGPT can generate truthful dialogues despite the dirty reference data.\n\n\n\nQ4: From what noise level did the truthfulness become really bad, such as below 50%? I don't understand the purpose of adding this type of noise.\n\nA8: The motivation we add the HTML labels as noise can be checked in Q3A7 above. We choose 10% or 20% because many references crawled from websites may not be carefully cleaned and remain a few HTML labels. We also supplement more experiments on more noise levels shown below (0 ~ 20 results are different from the paper, please see Reviewer7XKU Q2). \n\nWe can see GPT-4 can mostly generate truthful data from dirty references. \n\n| Noise Level | Truthfulness |\n| ----------- | ------------ |\n| 0           | 96.5         |\n| 10          | 96.2         |\n| 20          | 94.8         |\n| 30          | 93.3         |\n| 50          | 90.1         |\n| 70          | 86.2         |", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- Emphasis on truthfulness of the generated dialogues and eliminating hallucination.\n- Generating a whole dialogue in a single LLM query, which makes the dataset creation significantly faster (and presumably cheaper if using a paid API) than generating a dialogue turn by turn.\n- Customizability of the dialogue templates for greater diversity of the dialogues produced using this method.\n- Creation of two large multi-turn dialogue datasets that can be used to train perhaps much smaller models.", "reasons_to_reject": "- The creation of the conversational style and dialogue content pools could require a substantial amount of manual work and possibly a certain amount of domain knowledge (depending on the task) in order for the pools to support diverse conversations when generating 100K of them. There are no details provided on the size of these pools and how they were constructed, so unfortunately it's hard to judge.\n- Some sections are not written very clearly, hampering the understanding of the methods used and experiments performed (see an example in the questions section below).\n- Most claims about truthfulness in Section 5 are based on automatic evaluations performed by GPT-4. How do we know we can trust GPT-4 for this task? At least a small study using human annotators would have been helpful to convince the audience of GPT-4's suitability for this type of evaluation.", "questions_to_authors": "- A.) Is the number of turns limited to 4? It appears so from Table 1. And are those 4 turns per user and assitant each, or 4 overall turns?\n- B.) I found Section 5.1.1 very hard to understand the way it is currently written, especially the description of the evaluation setup for Self-Instruct and Baize. What do you mean by \"we follow the Alpaca that we randomly select three {question, answer} pairs (from other references)...\"? And what is a seed? What are the {question, answer} pairs for? They are nowhere to be seen in the examples in Table 3 and Appendix C. Are the {question, answer} pairs used instead of references when generating dialogues? I'm sorry, I just find it impossible to follow what is going on in this whole evaluation.\n- C.) For the reference quality experiments in Section 5.2.1, did you try using some linguistic noise (e.g., replacing certain words with random words, shuffling sequences of words, removing negations, etc.) instead of adding random HTML labels? The model might simply ignore the HTML code for the most part, since it's typically not there in references.\n- D.) From what noise level did the truthfulness become really bad, such as below 50%? If 20% of noise only reduced it from 98% to 96%, it makes me wonder if even 50% of noise would not have that big of an impact. Or otherwise I don't understand what the purpose of adding this type of noise really was in this experiment.\n\nEDIT: Thank you for answering my questions and providing further clarifications. I have increased my scores accordingly.", "paper_topic_and_main_contributions": "This paper introduces RefGPT, a method for generating dialogues using LLMs (e.g., GPT-4) without hallucinations, which are common in LLMs' outputs. To achieve that, the dialogues are generated from reference texts that can be trusted, such as Wikipedia articles. The method also involves an automatic construction of the dialogue template, including all the turns, and parameters for the individual turns to customize the conversation (e.g., the desired length of the response, tone used, etc.). The authors create two large datasets, with up to 100K dialogues each, using this method. In their automatic evaluation using GPT-4, they show the truthfulness of dialogues produced by their method is significantly higher than that of dialogues produced by other methods (97% vs. 50%).", "excitement": 3, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Typos_Grammar_Style_and_Presentation_Improvements": "- L262: \"ends\" -> \"end\"\n- L395: \"can\" -> \"to\"?\n- L475: extra \"are\""}, "extra_text": {}}, {"review_id": "0Ll7Inn07m", "reviewer": "Reviewer_8x7N", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 4, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks so much for your valuable comments!\n\nW1: GPT-4 is not reliable for evaluating the data quality and manual evaluation is needed.\n\nA1: Thanks for pointing this out. In the evaluation, we use the GPT-4 to compare if the assistant's responses are contradictory to the references, instead of letting GPT-4 judge the truthfulness of the responses with its own knowledge base. \n\n**We have supplemented the human evaluation following a similar pattern.** We randomly sample 50 English dialogues each for Alpaca, Baize, and RefGPT about factual knowledge. And 2 humans evaluate the truthfulness of the dialogues according to the references. The accuracies of Alpaca, Baize, and RefGPT are 54.0, 50.0, and 98.0. The results by humans have a similar trend as the GPT-4 evaluation (50.2, 47.2, 97.5).", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "This is an interesting and meaningful work that presents a simple and effective method for generating reliable datasets. The datasets provided by the authors are of great help to the development of the NLP community.", "reasons_to_reject": "The author uses GPT-4 to evaluate the data quality. I still feel that manual evaluation based on random sampling is necessary, because GPT-4 still faces the problem of model hallucinations.", "questions_to_authors": "", "paper_topic_and_main_contributions": "This manuscript proposes RefGPT to generate enormous truthful and customized dialogues without worrying about factual errors caused by the model hallucination.", "excitement": 4, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "ZMYQUseZbf", "reviewer": "Reviewer_7XKU", "summary": "", "questions": "1. There are some overclaims in the paper. line 13 \"without worrying about factual errors caused by the model hallucination\"  line70 \"LLMs will be prompted not to rely on their own knowledge to generate the dialogues, thus resolving the hallucination issue.\". Even given the reference as input, it still can not guarantee LLMs to generate perfect answers. Just as the paper mentioned in 5.2.2, even the </chat> marker can not be guaranteed to be generated. \n\n2. In Table 4. why 25% Ref. Ratio has the largest truthfulness? Also, why the 100% ref.Ratio does not have the same number with 0% Noise Level. It seems that the first row should be 100% Ref.Ratio and 0% Noise Level.\n", "limitations": "", "rating": 3, "confidence": 4, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks so much for your insightful comments!\n\nW1: Whether the proposed datasets can boost the performance is unclear. The comparison of tuning open-source LLMs is lacking. Better quality instruction data does not guarantee that tuned LLMs can avoid hallucination. A stronger baseline, that a LLM is fine-tuned on self-instruct during training and performs BM25-retrieved conditioned generation from the same online source as RefGPT during testing, is needed.\n\nA1: We agree that less hallucinated data can not guarantee the tuned LLM can avoid hallucination. However, the high-quality RefGPT data alleviates the harm of tuning with hallucinated data to some extent.\n\n**We supplement the strong baseline as you mention.** We fine-tune the llama-13b with RefGPT-Fact and with Alpaca separately. The strong baseline is the llama-13b model tuned with Alpaca and using the BM25 to find the reference as context. 2 humans evaluate 50 English questions in the RefGPT-Fact dataset on these two models. These two models both use a temperature of 0.1, top_p of 0.75, and top_k of 40 for generation. The Alpaca model with BM25 retrieval has an acc of 76.0 and RefGPT has an acc of 82.0. We observe that the retrieval-based method sometimes retrieves the wrong references so the model has to recite its own knowledge. And RefGPT model uses its memory in the fine-tuning state to answer the question but sometimes generates with hallucination. The results show that RefGPT can alleviate the hallucination on factual knowledge and is slightly better than the naive retrieval-based method.\n\n \n\nW2: Lacking some human evaluation for the RefGPT.\n\nA2: Thanks for pointing this out. In the evaluation, we use the GPT-4 to compare if the assistant's responses are contradictory to the references, instead of letting GPT-4 judge the truthfulness of the responses with its own knowledge base. \n\n**We have supplemented the human evaluation following a similar pattern.** We randomly sample 50 English dialogues each for Alpaca, Baize, and RefGPT about factual knowledge. And 2 humans evaluate the truthfulness of the dialogues according to the references. The accuracies of Alpaca, Baize, and RefGPT are 54.0, 50.0, and 98.0. The results by humans have a similar trend as the GPT-4 evaluation (50.2, 47.2, 97.5).\n\n \n\nW3: Using the external text to generate less hallucinated data is a common practice. The main contribution is the dataset, but the authors need more experiments to show its efficacy. \n\nA3: RefGPT is not only for generating less hallucinated data but also has the potential to generate dialogues of any vertical domain, e.g. coding scenario or a shopping assistant, by utilizing domain-specific documents as references. Besides utilizing the external text, RefGPT enables customization to generate different types of dialogues even with the same reference. For a piece of code, RefGPT can derive different scenarios of code creation, debugging, discussion, and so on.\n\nAs collecting code-related dialogues needs a large amount of domain knowledge, the RefGPT-Code dataset from real-world code data has intuitively better quality than simply collecting the synthetic code data from GPT-4. We will also train a model for the code scenario using the RefGPT-Code later to show more efficacy.\n\n \n\nQ1: Overclaims of resolving the model hallucination in dialogue generation.\n\nA4: We agree that GPT-4 can not always follow the references to generate the dialogues correctly thus our claims will be revised according to your advice.\n\n \n\nQ2: In Table 4. why 25% Ref. Ratio has the largest truthfulness? Why the 100% Ref.Ratio does not have the same number with a 0% Noise Level. It seems that the first row should be 100% Ref.Ratio and 0% Noise Level.\n\nA5: A smaller Ref ratio (25%) has a shorter context and GPT tends to generate a shorter dialogue, which makes it easier for GPT to follow the reference, thus having the largest truthfulness. \n\nThe noise level experiments are done separately on different settings (150 words each turn) in the paper from the Ref ratio experimental settings (300 words), thus there is a difference between 100% Ref.Ratio and 0% Noise Level. We supplement the experiments with the same settings (300 words) and the results are shown below:\n\n| Noise Level | Truthfulness |\n| ----------- | ------------ |\n| 0           | 96.5         |\n| 10          | 96.2         |\n| 20          | 94.8         |\n| 30          | 93.3         |\n| 50          | 90.1         |\n| 70          | 86.2         |\n\n\n\nQ3: Typos Grammar Style And Presentation Improvements\n\nA6: Thanks a lot for pointing these out. We will revise our paper according to your advice to make the paper express more clearly.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. Contributing two dialogue instruction tuning datasets (total 176k sessions) to the NLP community for potentially better LLM fine-tuning. The dataset is generated by GPT-4, using the online encyclopedia websites and Github repositories as the input references to generate more truthful dialogues.\n\n2. The prompt design experience about more fine-grained control in dialogue structure, number of turns, utterance length, and dialogue styles may bring some insights into the problem of locally customized LLM generation. \n\n3. Proposing an automatic evaluation framework of using GPT-4 as the referee to evaluate instruction data.  Given the ground-truth answer source (i.e., snippets of reference), GPT-4 can determine which dialogue is better and scale up to much larger benchmarks. \n", "reasons_to_reject": "1. The biggest problem is that whether the proposed dialogue instruction data can really boost the performance of the fine-tuned LLMs is unclear.  The comparison experiment of tuning open-source LLMs like LLaMa on self-instruct and RefGPT is lacking. Better quality instruction data does not guarantee the tuned LLMs can avoid hallucination since the LLM still relies on itself to generate during the inference. A stronger baseline would be an LLM that is fine-tuned on self-instruct during training and performs BM25-retrieved conditioned generation from the same online source as RefGPT during testing. If the author can verify that an LLM that is fine-tuned on RefGPT datasets is on-par with or even better than that strong baseline, this paper would be much more convincing and complete.  \n\n2. Lacking some human evaluation for the RefGPT. Although it is intuitive that RefGPT is more faithful and reliable than self-instruct and Baize, but the quality of the dataset is only proved good by GPT-4, another model. There could be some bias in using a model to judge; for example, it tends to select an answer that has a short repetition of the source reference, which may still not be the true answer to the question or hurt the answer variety. There still needs more experiments, esp. some human studies, to show the quality of RefGPT. \n\n3. Adding more external textual information as conditional input to generate less hallucinated data for LLMs is a common practice, so the methodology for this paper is not novel. The main contribution of the paper is actually the dataset, but the authors need more experiments to show its efficacy.", "questions_to_authors": "1. There are some overclaims in the paper. line 13 \"without worrying about factual errors caused by the model hallucination\"  line70 \"LLMs will be prompted not to rely on their own knowledge to generate the dialogues, thus resolving the hallucination issue.\". Even given the reference as input, it still can not guarantee LLMs to generate perfect answers. Just as the paper mentioned in 5.2.2, even the </chat> marker can not be guaranteed to be generated. \n\n2. In Table 4. why 25% Ref. Ratio has the largest truthfulness? Also, why the 100% ref.Ratio does not have the same number with 0% Noise Level. It seems that the first row should be 100% Ref.Ratio and 0% Noise Level.\n", "paper_topic_and_main_contributions": "This paper focuses on the instruction data construction problem. Similar to the previous method, like Self-Instruct, it utilized powerful LLMs (GPT3.5 and GPT-4) to generate dialogue data in a bootstrapping manner. The main difference is that it adds some reference messages as an additional information source for the input prompt to ensure the LLM can generate more authentic and truthful dialogue data.  Experiments show that the proposed RefGPT method can gather high truthfulness data and also more customized data via detailed prompt design than baselines. In the end, two relatively large instruction datasets RefGPT-Fact and RefGPT-Code are proposed for the LLM instruction tuning.", "excitement": 3, "reproducibility": 5, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {"Typos_Grammar_Style_and_Presentation_Improvements": "1. Line3 fine-tuning with\n2. Line 40 crowd -> crowdsource\n3. Paragraph 4 in Introduction. How to perform detailed control is a little bit too vague. Better to explicitly say that we use prompt design and give a simple example. \n4. Table 3 can add the result of other baselines for comparison with the same reference and question."}}]

Author rebuttal:

Meta-review: This paper introduces RefGPT, a method for generating truthful dialogue datasets using LLMs. Two reviewers pointed out that the truthfulness evaluation reported in the paper is based on automatic model-based evaluation with GPT-4, which cannot substitute for true human evaluation. The author response supplemented with additional human evaluation results and addressed the concern. One concern regarding the novelty -- improving LLM generation and reducing hallucination by providing additional textual information as conditional input is a common practice -- was not sufficiently addressed.
Decision: Accept-Findings