AutoConv: Automatically Generating Information-seeking Conversations with Large Language Models
Year: 2023

Abstract:
Information-seeking conversation, which aims to help users gather information through conversation, has achieved great progress in recent years. However, the research is still stymied by the scarcity of training data. To alleviate this problem, we propose AutoConv for synthetic conversation generation, which takes advantage of the few-shot learning ability and generation capacity of large language models (LLM). Specifically, we formulate the conversation generation problem as a language modeling task, then finetune an LLM with a few human conversations to capture the characteristics of the information-seeking process and use it for generating synthetic conversations with high quality. Experimental results on two frequently-used datasets verify that AutoConv has substantial improvements over strong baselines and alleviates the dependence on human annotation. In addition, we also provide several analysis studies to promote future research.

heading: 1 Introduction
text: In information-seeking conversations, users repeatedly ask questions based on their interests, and the dialogue system provides answers to fulfill their information needs (Stede and Schlangen, 2004; Choi et al., 2018; Reddy et al., 2019). This scenario is important for addressing real-world open-ended questions, which requires discussions to explore in depth (Dai et al., 2022), e.g., How to learn more efficiently? Though great progress has been achieved in recent years, most existing researches depend on abundant human annotation, which can be highly costly and limited in knowledge coverage.
A promising way to alleviate this problem is data augmentation (Chen et al., 2021). Traditional methods, including token-level manipulation (Kobayashi, 2018; Wei and Zou, 2019)
∗ This work is done when Siheng Li is an intern at Huawei Noah’s Ark Lab.
† Equal contribution. ‡ Corresponding author.
and sentence-level paraphrasing (Sennrich et al., 2016), improve the linguistic diversity of training data. However, they cannot create conversations grounded on new documents, which are indispensable for dealing with out-of-domain scenarios. Another line of research focuses on simulation-based methods (Wu et al., 2021; Kim et al., 2022). Specifically, they can iteratively generate conversations grounded on new documents based on a span extractor and an utterance generator. Nevertheless, both the training of the extractor and the generator still require abundant human dialogues. Besides the above ways, Dai et al. (2022) propose Dialog Inpainting, which creates information-seeking dialogues by inserting utterances between neighboring sentences in documents. One potential risk is the gap between the structure of documents and that of conversations. Documents are tighter, while realworld conversations are more open-ended.
To alleviate the above issues, we propose a simple yet effective method AutoConv for Automatically generating information-seeking Conversations, which takes advantage of the fewshot learning ability and generation capacity of large language models (LLM) (Brown et al., 2020). Specifically, we formulate conversation generation as a language modeling task and utilize an LLM for generating synthetic conversations grounded on external documents. Surprisingly, finetuning with a few human dialogues can help LLM capture the characteristics of the information-seeking process
1751
(e.g., grounding, question answering) and generate high-quality synthetic conversations. Then, we can train a small task model with these dialogues. The differences between AutoConv and others are shown in Table 1.
We conduct comprehensive experiments on two frequently-used datasets QuAC (Choi et al., 2018) and CoQA (Reddy et al., 2019) in the low-resource setting, where only dozens of human dialogues are available. The results show that AutoConv has substantial improvements over several strong baselines. When scaling up the synthetic dialogues, AutoConv has the improvement of up to 5.06 F1 gain compared with directly finetuning, and thus largely reduces the labor force for annotation. In addition, we find that the small task model trained with synthetic dialogues can even surpass finetuned LLM with only 1.7% parameters. Moreover, we also investigate the impact of decoding strategy and scaling laws for AutoConv.

heading: 2 Method

heading: 2.1 Task Formulation
text: Our goal is automatically generating informationseeking conversations. Specifically, each conversation is grounded on a document d and consists of a series of user questions and system answers.

heading: 2.2 Conversation Generation
text: Training. We formulate conversation generation as a language modeling task and finetune1 an LLM with a few human dialogues (e.g., 50 from QuAC (Choi et al., 2018)) to capture the characteristics of information-seeking conversations (e.g., grounding, question answering). The objective is the negative log-likelihood of each utterance:
L = − T∑
t=1
L∑
l=1
logP (utl |ut<l,h<t, d),
1In our preliminary experiments, we try to prompt LLM without training. However, we find that the performance is poor and LLM fails to generate conversations grounded on the documents, similar to the observation in Zheng et al. (2022).
where u represents a user question or a system answer, h is the dialogue history, L and T are the number of tokens and turns respectively.
Generating. Based on the finetuned LLM, we can generate synthetic dialogues with unlabeled documents, as in Figure 1. In information-seeking scenarios, user questions are typically open-ended. Thus we choose nucleus sampling (Holtzman et al., 2020) for generating user questions, which has shown great performance in various open-ended generation tasks (Su et al., 2022). However, when applying a sampling decoding strategy for system answer generation, we find it results in the “hallucination” problem (Shuster et al., 2021), where the generation is plausible but factually incorrect based on the document. To this end, we utilize greedy search for answer generation. Neural language models often generate the same sentences repetitively (Xu et al., 2022). To alleviate this problem, we first compute the diversity score of each synthetic dialogue as in Su et al. (2022), which considers the repetition at different n-gram levels. Then, we filter out dialogues based on this score.
After that, a two-stage training strategy is adopted (Xie et al., 2020b) for training a small task model. Specifically, we first pre-train it on the synthetic dialogues, then finetune it on the human dialogues used for finetuning the LLM. More training details are given in Appendix B.

heading: 3 Experiments
text: We conduct experiments on QuAC (Choi et al., 2018) and CoQA (Reddy et al., 2019), more details about them are shown in Appendix A.

heading: 3.1 Implementation
text: We focus on the low-resource setting, where human dialogues are scarce. To simulate this setting, we randomly sample a few human dialogues from the training set of QuAC or CoQA, and use them for finetuning the LLM. We use OPT-13B (Zhang et al., 2022) as the LLM and UnifiedQA-V2-base (222M) (Khashabi et al., 2022) as the small task model. All data augmentation methods use the same training strategy and small task model. More implementation details are shown in Appendix B.

heading: 3.2 Comparison with Baselines
text: We compare AutoConv with a series of baselines, and the details of them are given in Appendix C. As
shown in Table 2, AutoConv achieves better performance than GPT-3 prompting on QuAC with only 0.13% parameters and 50 human dialogues, but is less competitive on CoQA. We conjecture the reason stems from the intrinsic difference between the two datasets. CoQA contains more factoid questions, and the answers are named entities or short noun phrases like those in SQuAD (Rajpurkar et al., 2016). By training on large-scale text corpus from a web forum, GPT-3 might implicitly learn the format and structure of question answering (Sanh et al., 2022), and thus gets excellent performance on CoQA. On the other side, QuAC has more openended and exploratory questions as in natural conversations, and 86% questions are contextual (Choi et al., 2018). Therefore, it brings more difficulties for GPT-3 inference with few demonstrations, while our method learns better from both human dialogues and synthetic dialogues.
Compared with data augmentation methods, AutoConv achieves the best performance on both datasets and mitigates the gap between synthetic dialogues and human upper bounds. We find that the token-level augmentation method EDA and the sentence-level augmentation method BackTranslation even hurt the performance, which is
similar to the observation in Chen et al. (2021). One possible reason is that they bring too much noise. Dialog Inpainting (Dai et al., 2022) gets ordinary performance, and the reason possibly derives from the gap between the structure of natural conversations and that of the documents used for constructing synthetic dialogues.

heading: 3.3 Scaling up Human Dialogues and Synthetic Dialogues
text: In this part, we further analyze the performance of AutoConv when scaling up the human dialogues and synthetic dialogues. As shown in Figure 2, the
performance boosts when more human dialogues or synthetic dialogues are used. With 50 human dialogues, AutoConv outperforms the results of finetuning with 200 human dialogues. With 500 human dialogues, AutoConv gets competitive performance compared with finetuning with 2000 human dialogues. These results verify the high quality of synthetic dialogues, and our AutoConv can largely alleviate the labor force for annotation.

heading: 3.4 Comparison with Finetuned Large Language Model
text: AutoConv is a kind of symbolic knowledge distillation (West et al., 2022), where the finetuned large language model (LLM) transfers its knowledge to the small task model (STM) by generating synthetic dialogues for the training of STM. Here, we further investigate the effectiveness of AutoConv from the aspect of knowledge distillation. As shown in Table 3, finetuned LLM has substantial improvements over finetuned STM. However, it brings large memory and computation cost. On the other side, our AutoConv not only keeps the efficiency of STM, but also boosts the performance. Surprisingly, AutoConv even outperforms its teacher model in the 200 human dialogues setting. Similar observations are found in West et al. (2022); Ye et al. (2022), while they focus on different tasks. We leave the analysis of this novel observation for future work.

heading: 3.5 Impact of Decoding Strategy
text: During our preliminary experiments, we find that the decoding strategy is important for system answer generation. More precisely, we evaluate the answer generation performance of LLM with different decoding strategies on QuAC, and the results are shown in Table 4. Though nucleus sampling (Holtzman et al., 2020) has shown great performance in various generation tasks (Su et al., 2022), it performs less competitively than maximization-
based decoding strategies for answer generation. Compared with beam search, greedy search shows competitive performance and is more efficient. Thus we use greedy search by default in this paper.

heading: 3.6 Scaling Laws
text: We further analyze how the benefit of AutoConv is affected by the scale of LLM. As shown in Figure 3, the performance gets better with a larger model across a various number of synthetic dialogues. In addition, when the LM is small (350M) and with limited generation ability, the synthetic dialogues can even hurt the performance when the available human dialogues are scarce. Due to the limitation of computational resources, we limit our investigation to 13B parameters and leave larger models for future work.

heading: 3.7 Case Study
text: In Table 5, we present an example of our synthetic conversation for the case study. The original document describes the singer Ciara’s second studio album and her acting debut. The conversation consists of seven user questions and seven system answers, covering the title and sales of the album, the duration of the tour, etc. As we can see from this
Title 2006-2007: Ciara: The Evolution and acting debut
Document On December 5, 2006, Ciara released her second studio album, Ciara: The Evolution. According to the singer, the title of the album is \"about so much more than just my personal growth - it’s about the evolution of music, the evolution of dance, the evolution of fashion \" The source of the albumś creativity such as the sound and edge comes from Ciara in general. Ciara: The Evolution became Ciaraś first and only number one album on the U.S. Billboard 200, and her second number one on the Top R&B/Hip-Hop Albums charts with sales of 338,000 in the first week . . .
Dialogue Usr: What was the evolution? Sys: On December 5, 2006, Ciara released her second studio album, Ciara: The Evolution. Usr: What was the name of the album? Sys: The title of the album is \"about so much more than just my personal growth - it’s about the evolution of music, the evolution of dance, the evolution of fashion.\" Usr: Did the album do well? Sys: The album became Ciara’s first number one album on the U.S. Billboard 200 and second on the Top R&B/Hip-Hop Albums charts with sales of more than 338,000. Usr: What awards did she win if any? Sys: The album went on to be certified platinum by the RIAA in the United States, within only five weeks of its release, and has sold 1.3 million copies according to Nielsen SoundScan. Usr: What else did she accomplish during that time? Sys: In support of the album, Ciara went on her first headlining tour in October 2006. Usr: How long was the tour? Sys: The tour went to seventeen different clubs in cities throughout the United States. Usr: Did she have success with any songs? Sys: The lead single, \"Get Up\", which features Chamillionaire, reached number seven in the United States and gained a platinum accreditation.
Table 5: An example of the synthetic conversation generated by AutoConv, the LLM is finetuned with 50 human dialogues from QuAC (Choi et al., 2018).
example, the user questions are diverse (e.g. what, how, did, etc.) and the conversation is informative and conversational. For example, when the system mentions “tour” (the fifth system utterance), the user follows by asking “How long was the tour?”.

heading: 3.8 Error Analysis
text: To further analyze the limitation of our method, we conduct an error analysis by manually investigating 50 synthetic conversations generated by AutoConv, which is finetuned with 50 human conversations from QuAC (Choi et al., 2018). Particularly, we find that only 5% generated questions are not suitable (e.g., misspelled names). The reason stems from the open-ended characteristic of natural conversation that many kinds of user questions are possible under the same context. However, nearly 40% of system answers are not perfect, and we summarize the wrong answers into four major classes: (1) Irrelevant: 75% of them are totally irrelevant to user questions. (2) Related but not Accurate: 14% of them contain related knowledge from the grounded documents, but the answers are not accurate. Take an example in Table 5, the second user question asks for the name of the album, which is Ciara: The Evolution according to the document. While the LLM generates the interpretation of the album name by mistake. (3) Missing: 4% of them belong to the missing error that the system answers are “No Answer”, while the questions actually can be answered based on the documents. (4) Hallucination: 3% of them mention hallucination knowledge, which cannot be found in the documents. In addition, we also notice that AutoConv is more likely to generate wrong answers when grounding on longer and more complex documents.

heading: 4 Conclusion
text: In this paper, we propose a simple yet effective method, AutoConv, which formulates the conversation generation problem as a language modeling task. Then, based on a large language model and a few human dialogues, AutoConv can generate synthetic dialogues with high quality. Experimental results on both QuAC and CoQA verify the effectiveness of AutoConv, which alleviates the human efforts for annotation largely. Furthermore, we also provide case study and error analysis to prompt future research.
Limitations
In this paper, we propose a method named AutoConv, which means automatically generating information-seeking conversations with large language models (LLM). Though it has achieved great performance on both QuAC (Choi et al., 2018) and CoQA (Reddy et al., 2019), there are still some limitations that should be noticed.
Limitation of LLM. In our experiments, we use OPT-13B (Zhang et al., 2022) as the LLM for generating synthetic conversations due to the limited computational resources. Larger models should be considered to further understand the potential ability of AutoConv, e.g., GPT-3 (Brown et al., 2020), OPT-175B (Zhang et al., 2022), BLOOM-176B (Scao et al., 2022), and GLM-130B (Zeng et al., 2022) etc.
Limitation of Implementation. As mentioned in Section 2.2 and Appendix B, our method needs to finetune LLM and generate massive synthetic conversations based on the finetuned LLM, which has a high cost for implementation.
Limitation of Synthetic Dialogues. As shown in Table 2 and Section 3.8, there is still a gap between our synthetic dialogues and human dialogues. It is important to improve the quality of synthetic dialogues so that we can further alleviate the dependence on human annotation.
Ethics Statement
AutoConv is based on large language models (LLM), while LLM has some potential risks, e.g., social bias (Liang et al., 2021), offensive content (Ganguli et al., 2022) etc. Fortunately, we finetune the LLM to capture the characteristics of the information-seeking process, and the generated conversations are mostly grounded on the provided documents (take an example in Table 5). Therefore, our method alleviates the potential risks of directly using LLM. According to our manual check in error analysis (Section 3.8), we do not find any harmful content in the synthetic conversations. In addition, we also encourage considering more safety methods (Xu et al., 2020; Sun et al., 2022) to guarantee the quality of synthetic conversations.

heading: Acknowledgements
text: This work was partly supported by the National Key Research and Development Program of China
(No. 2020YFB1708200) , the "Graph Neural Network Project" of Ping An Technology (Shenzhen) Co., Ltd. and AMiner.Shenzhen SciBrain fund.

heading: A Datasets
text: QuAC. QuAC (Choi et al., 2018) is a leading conversational question answering dataset, consists of 14K information-seeking dialogues. Different from the factoid questions in most existing QA datasets, the questions in QuAC are more open-ended and exploratory. In addition, 86% of questions are contextual, and the model needs to understand the dialogue context to resolve coreference. As the test set is only available in the QuAC challenge2, we evaluate the performance on the development set.
CoQA. CoQA (Reddy et al., 2019) consists of 127K conversational QA pairs across seven domains. Different from QuAC, CoQA focus more on factoid questions, and the answers are mostly named entities or short phrases as in SQuAD (Rajpurkar et al., 2016). The test set of CoQA is only available in the CoQA challenge3, therefore we evaluate the performance on the development set.
B Implementation Details
General Setting. All experiments are based on Transformers4 (Wolf et al., 2020), DeepSpeed5 (Rasley et al., 2020) and Pytorch Lightning6. We use UnifiedQA-V2-base7 (Khashabi et al., 2020, 2022) as the small task model, which is based on T5 architecture with 222M parameters and pre-trained on many QA tasks (the tasks in our experiments are not included in). The training of the small task model follows the original paper (Khashabi et al., 2020) in a Text-to-Text framework (Raffel et al., 2020). The input is Dialogue History \n Document and the output is System Answer.
For the training hyperparameters, we set the learning rate as 3e − 4, batch size as 32, and use Adam optimizer (Kingma and Ba, 2015) with warmup learning rate schedule, the warmup ratio is 0.1. When comparing with baseline methods as in Section 3.2, all methods use the same small task model, the same two-stage training strategy (Xie et al., 2020b; Chen and Yang, 2021), the same human dialogues and the same number of synthetic dialogues for fairness (5 times the number of hu-
2https://quac.ai/ 3https://stanfordnlp.github.io/coqa/ 4https://huggingface.co/docs/transformers/ index 5https://github.com/microsoft/DeepSpeed 6https://github.com/Lightning-AI/lightning 7https://huggingface.co/allenai/ unifiedqa-v2-t5-base-1363200
man dialogues). For the 50 human dialogues setting, we train each model for 1K gradient steps in the pre-training stage and 200 gradient steps in the fintuning stage. For the 100 human dialogues setting, the steps are 2K and 400 respectively. When scaling up the number of synthetic dialogues as in Section 3.3 and Section 3.6, the numbers of pretraining steps scale up, which are 2K, 4K, 8K, 20K and 40K for 1K, 2K, 4K, 10K and 20K synthetic dialogues respectively, and the finetuning steps are 200, 400, 800 and 2K for 50, 100, 200 and 500 human dialogues respectively. For all experiments, we randomly sample 20% dialogues as the validation set, and others as the training set. The model is validated every epoch, and we choose the checkpoint with the best F1 score on the validation set for evaluation.
Ours. We use OPT-13B8 (Zhang et al., 2022) as the LLM for generating synthetic dialogues, which is a decoder-only pre-trained language model with 13B parameters. The learning rate and batch size are set as 1e-5 and 32. Adam optimizer (Kingma and Ba, 2015) with warmup learning rate schedule is utilized for optimization and the warmup ratio is 0.1. The max training steps of LLM are 200, 400, 800 and 2K for 50, 100, 200 and 500 human dialogues respectively. According to the performance of AutoConv on the validation set of human dialogues, we find that training LLM for 4 epochs is the most suitable. We randomly sample 5K documents from the training sets of QuAC and CoQA, and generate 8 synthetic dialogues for each document. The number of turn is set as 14 for QuAC and 30 for CoQA. Then, we filter a quarter of the synthetic dialogues based on the diversity score of each dialogue as in Su et al. (2022), which takes into account the repetition at different n-gram levels. It takes around 5 hours for training LLM and 18 hours for generating synthetic dialogues with 8 Tesla V100 32GB GPUs.
Evaluation. To evaluate the quality of synthetic conversations, we evaluate the conversational question answering performance of the small task model, which is trained on both synthetic conversations and a few human conversations. The metrics are Exact Match and word-level F1 as in Choi et al. (2018).
8https://huggingface.co/facebook/opt-13b

heading: C Baselines
text: Prompting. Prompting is a promising method for many NLP tasks. It aims to elicit the ability of large language models learned from pre-training with text demonstrations (e.g., task instruction and few-shot examples etc). In Table 2, we report the results from Brown et al. (2020).
Finetuning. Train the small task model with only human annotations.
EDA. Easy Data Augmentation (EDA) is a simple but effective method for text classification (Wei and Zou, 2019). Given an input text, including both the knowledge paragraph and dialogue history in our experiments, four operations are applied to create new examples, including synonym replacement, random insertion, random swap and random deletion. We use their open source code9 for implementation.
Back-Translation. Back-Translation is one of the most popular augmentation method for NLP tasks (Sennrich et al., 2016; Xie et al., 2020a). Specifically, we first translate the input text to a target language, then translate it back to the source language, thus we can get a paraphrased example. To get various augmentations for each sample, we use five target languages, including Chinese, French, German, Arabic, and Korean. Huawei Translate10 is used for the translation process.
Utterance Manipulation. Chen and Yang (2021) propose utterance-level manipulation to perturb the discourse relations in the conversation. Two simple operations are used: (1) random swapping, which randomly swaps two utterances to mess up the logic chain of the conversation, and (2) random deletion, which means randomly deleting an utterance to improve the discourse diversity. We randomly select one operation for each augmentation.
Dialog Inpainting. The state-of-the-art data augmentation method for conversational question answering. Given a document, they iteratively insert generated utterances between the consecutive sentences in the document, then the utterances and sentences can form an informative conversation (Dai et al., 2022). We randomly sample generated
9https://github.com/jasonwei20/eda_nlp 10https://www.huaweicloud.com/product/nlpmt.
html
dialogues from their open source data11.
11https://github.com/google-research/ dialog-inpainting
ACL 2023 Responsible NLP Checklist

heading: A For every submission:

heading: 3 A1. Did you describe the limitations of your work?
text: Limitations

heading: 3 A2. Did you discuss any potential risks of your work?
text: Ethics Statement

heading: 3 A3. Do the abstract and introduction summarize the paper’s main claims?
text: Abstract and Introduction
7 A4. Have you used AI writing assistants when working on this paper? Left blank.
B 3 Did you use or create scientific artifacts?

heading: Appendix A and Appendix B

heading: 3 B1. Did you cite the creators of artifacts you used?
text: Appendix A
B2. Did you discuss the license or terms for use and / or distribution of any artifacts? Not applicable. Left blank.

heading: 3 B3. Did you discuss if your use of existing artifact(s) was consistent with their intended use, provided
text: that it was specified? For the artifacts you create, do you specify intended use and whether that is compatible with the original access conditions (in particular, derivatives of data accessed for research purposes should not be used outside of research contexts)? Ethics Statement and Appendix A
B4. Did you discuss the steps taken to check whether the data that was collected / used contains any information that names or uniquely identifies individual people or offensive content, and the steps taken to protect / anonymize it? Not applicable. Left blank.

heading: 3 B5. Did you provide documentation of the artifacts, e.g., coverage of domains, languages, and
text: linguistic phenomena, demographic groups represented, etc.? Appendix A
3 B6. Did you report relevant statistics like the number of examples, details of train / test / dev splits, etc. for the data that you used / created? Even for commonly-used benchmark datasets, include the number of examples in train / validation / test splits, as these provide necessary context for a reader to understand experimental results. For example, small differences in accuracy on large test sets may be significant, while on small test sets they may not be. Appendix A and Appendix B
C 3 Did you run computational experiments? Section 3

heading: 3 C1. Did you report the number of parameters in the models used, the total computational budget
text: (e.g., GPU hours), and computing infrastructure used? Appendix B
The Responsible NLP Checklist used at ACL 2023 is adopted from NAACL 2022, with the addition of a question on AI writing assistance.

heading: 3 C2. Did you discuss the experimental setup, including hyperparameter search and best-found
text: hyperparameter values? Appendix B

heading: 3 C3. Did you report descriptive statistics about your results (e.g., error bars around results, summary
text: statistics from sets of experiments), and is it transparent whether you are reporting the max, mean, etc. or just a single run? Section 3

heading: 3 C4. If you used existing packages (e.g., for preprocessing, for normalization, or for evaluation), did
text: you report the implementation, model, and parameter settings used (e.g., NLTK, Spacy, ROUGE, etc.)? Appendix B
D 7 Did you use human annotators (e.g., crowdworkers) or research with human participants? Left blank.
D1. Did you report the full text of instructions given to participants, including e.g., screenshots, disclaimers of any risks to participants or annotators, etc.? Not applicable. Left blank.
D2. Did you report information about how you recruited (e.g., crowdsourcing platform, students) and paid participants, and discuss if such payment is adequate given the participants’ demographic (e.g., country of residence)? Not applicable. Left blank.
D3. Did you discuss whether and how consent was obtained from people whose data you’re using/curating? For example, if you collected data via crowdsourcing, did your instructions to crowdworkers explain how the data would be used? Not applicable. Left blank.
D4. Was the data collection protocol approved (or determined exempt) by an ethics review board? Not applicable. Left blank.
D5. Did you report the basic demographic and geographic characteristics of the annotator population that is the source of the data? Not applicable. Left blank.

heading: Future Work
text: Furthermore, we also provide case study and error analysis to prompt future research. Limitations
In this paper, we propose a method named AutoConv, which means automatically generating information-seeking conversations with large language models (LLM). Though it has achieved great performance on both QuAC (Choi et al. , 2018) and CoQA (Reddy et al. , 2019), there are still some limitations that should be noticed. Limitation of LLM. In our experiments, we use OPT-13B (Zhang et al. , 2022) as the LLM for generating synthetic conversations due to the limited computational resources. Larger models should be considered to further understand the potential ability of AutoConv, e. g. , GPT-3 (Brown et al. , 2020), OPT-175B (Zhang et al. , 2022), BLOOM-176B (Scao et al. , 2022), and GLM-130B (Zeng et al. , 2022) etc. Limitation of Implementation. As mentioned in Section 2. 2 and Appendix B, our method needs to finetune LLM and generate massive synthetic conversations based on the finetuned LLM, which has a high cost for implementation. Limitation of Synthetic Dialogues. As shown in Table 2 and Section 3. 8, there is still a gap between our synthetic dialogues and human dialogues. It is important to improve the quality of synthetic dialogues so that we can further alleviate the dependence on human annotation