UHGEval: Benchmarking the Hallucination of Chinese Large Language Models via Unconstrained Generation (2024)

Abstract: Large language models (LLMs) produce hallucinated text, compromising their practical utility in professional contexts. To assess the reliability of LLMs, numerous initiatives have developed benchmark evaluations for hallucination phenomena. However, they often employ constrained generation techniques to produce the evaluation dataset due to cost and time limitations. For instance, this may involve employing directed hallucination induction or deliberately modifying authentic text to generate hallucinations. These are not congruent with the unrestricted text generation demanded by real-world applications. Furthermore, a wellestablished Chinese-language dataset dedicated to the evaluation of hallucinations is presently lacking. Consequently, we have developed an Unconstrained Hallucination Generation Evaluation (UHGEval) benchmark, containing hallucinations generated by LLMs with minimal restrictions1. Concurrently, we have established a comprehensive benchmark evaluation framework to aid subsequent researchers in undertaking scalable and reproducible experiments. We have also evaluated prominent Chinese LLMs and the GPT series models to derive insights regarding hallucination.

heading: 1 Introduction
text: Large language models (LLMs) have unparalleled proficiency in language generation, knowledge application, and intricate reasoning (Zhao et al., 2023). However, they invariably manifest hallucination (Rawte et al., 2023; Yu et al., 2024c), as they often generate content that is incongruent with user input, the model’s output context, or factual information. Real-world hallucination examples from our UHGEval dataset can be observed in Fig. 1. * These authors contribute equally Corresponding author: lizy@iaar.ac.cn 1Framework, dataset, and results on our project webpage: https://iaar-shanghai.github.io/UHGEval/.
The fabricated news content depicted in Fig. 1 offers NO utility to journalists; on the contrary, the verification and rectification of such content exacts a toll on the valuable time of journalists. To this concern, it is crucial to first formulate a comprehensive, stringent, and demanding benchmark for the assessment of hallucination in language generation (Zhang et al., 2023; Wang et al., 2023b).
While there have been a bunch of efforts to develop benchmarks for hallucination assessment, they always employ restricted techniques to produce particular kinds of hallucinated utterances. This approach is at odds with real-world scenarios where hallucinations arise in unrestricted, spontaneously generated content. For example, HaluEval specifies the type of hallucination in the prompt when generating hallucinated text: “You are trying to answer a question but misunderstand the question context and intention” (Li et al., 2023). Additionally, benchmarks such as HaDes annotate hallucinations at a finer granularity by generating token-level hallucinations based on text perturbations (Liu et al., 2022), but the text perturbation method is still constrained.
Hallucinations must be generated in an unconstrained setting; otherwise, it’s difficult to determine whether the hallucinated texts in many
5266
datasets are indeed errors that language models will make on their own. This point carries profound implications. For example, with a dataset containing freely generated hallucinations, researchers can explore the differences in model hidden states (logits, hidden layers, etc.) between hallucinated text spans and unhallucinated text spans. Such in-depth analysis would not be possible with datasets generated under constrained settings. Appendix A provides a detailed comparison with three other datasets, TruthfulQA (Lin et al., 2022), HaluEval (Li et al., 2023), and HaDes (Liu et al., 2022).
Besides, many benchmarks are centered on the evaluation in English, neglecting the assessment of hallucination in Chinese. The extensive lexicon of Chinese characters, combined with the complexities introduced by Chinese word segmentation, renders the Chinese hallucination evaluation particularly arduous and deserving of focused scrutiny.
To address the aforementioned challenges, we introduce a novel benchmark for hallucination assessment, as depicted in Fig. 2. The benchmark dataset is composed of raw Chinese news articles and continuations of those articles freely generated by LLMs but annotated with hallucinations.
Furthermore, selecting texts from the news domain is intentional, given that news requires utmost precision in conveying factual information and exhibits minimal tolerance for hallucinations, presenting a considerable challenge for the majority of LLMs. Moreover, news data encompasses a wide range of topics, including medicine, tech-
nology, finance, sports, etc., incorporating features found in texts from other domains. Lastly, news articles are readily available and frequently employed as training corpora by a large number of LLMs, guaranteeing impartiality in the evaluation of many LLMs (Zhao et al., 2023).
Our contributions: (1) The development of an unconstrained hallucination evaluation dataset, comprising over 5000 items. Existing methods for constructing datasets often yield biases towards predefined directions, thereby hindering the full simulation of real-world hallucinations. (2) The establishment of a unified and diverse evaluation framework, UHGEval, that encompasses discriminative, selective, and generative evaluations. Current benchmark methods for hallucination evaluation often exhibit a singular approach and lack task specificity. (3) A comprehensive empirical analysis. We evaluated eight prominent Chinese LLMs and three classic GPT series models to explore the credibility of various LLMs.

heading: 2 Related Work
text: This section outlines hallucination evaluation benchmarks, their characteristics, and evaluation methodologies. A summary of these benchmarks is presented in Table 1. For related works on LLMs and hallucinations, please refer to Appendix B.

heading: 2.1 Benchmark Dataset Construction
text: Dataset construction usually involves three steps. Firstly, real-world texts for hallucination genera-
tion are collected, and most benchmarks directly use existing datasets, such as Wiki (Muhlgay et al., 2024), Alpaca (Li et al., 2023), PubMed (Pal et al., 2023), etc. Secondly, hallucinations are generated usually by LLMs such as GPT3.5-Turbo, and most works use a constrained hallucination generation (CHG) paradigm. STSN (Varshney et al., 2023) and XSum Hallu (Maynez et al., 2020) are the only two benchmarks that use UHG as we do. Thirdly, it is not certain that the content generated by the LLMs actually contains hallucinations, and often requires annotation, which is mostly done by human involvement. There are also works using automatic machine labeling (Muhlgay et al., 2024; Lee et al., 2022; Cheng et al., 2023). These are the basic methods for constructing datasets, but there are also some other paradigms, such as constructing the dataset purely using manual labor, e.g. ChineseFactEval (Wang et al., 2023a), HaDes (Liu et al., 2022), TruthfulQA (Lin et al., 2022), etc.

heading: 2.2 Benchmark Dataset Characteristics
text: Regarding the granularity of hallucinations labeled in the datasets, most studies assess hallucinations at the sentence and document levels, while a few examine them at the word (or keyword, concept) level. Concerning language, most evaluation datasets are in English. To our knowledge, the only two Chinese benchmarks, ChineseFactEval (Wang et al., 2023a) and HalluQA (Cheng et al., 2023) contain only 125 and 450 questions, respectively. Given the notably limited size of these datasets, our work
significantly enhances the pool of data available for Chinese hallucination evaluation.

heading: 2.3 Evaluation Schemes
text: Currently, building automatic metrics for evaluation is still dominant, and a small proportion of works use human evaluation (Min et al., 2023; Lin et al., 2022; Maynez et al., 2020). In terms of specific evaluation metrics, most works adopt common classification metrics, e.g., F1, accuracy, precision, and recall. Some other works construct their calculation methods, e.g., FACTOR (Muhlgay et al., 2024), FActScore (Min et al., 2023), HaLoCheck (Elaraby et al., 2023), etc. However, the above metrics are rule-based and can only evaluate the ability of LLMs to classify hallucinations, but not the ability of LLMs to generate content without hallucinations. Thus, some benchmarks explore further in generative evaluation. For example, KoLA (Yu et al., 2024a) evaluates knowledge creation (KC) using BLEU and ROUGE, and TruthfulQA (Lin et al., 2022) evaluates hallucinations using a specially trained classifier, GPT-judge.

heading: 3 The UHGEval Dataset

heading: 3.1 Data Collection and Pre-processing
text: We amassed tens of thousands of historical news articles from leading Chinese news websites, covering the period from January 2015 to January 2017, to serve as the foundation for constructing the dataset. It is worth noting that the decision to eschew more recent news articles (e.g., from
2024) was made to better assess the model’s understanding of existing knowledge. Indeed, the knowledge embedded within the training data of existing Chinese LLMs typically encompasses information about significant news between 2015 and 2017 (Zhao et al., 2023).
The collected news spans various topics, such as sports, education, science, society, finance, and more. This diversity underscores the advantage of choosing news texts for our dataset, as it enables the incorporation of a wide array of text genres. We hypothesize that the occurrence of hallucinations will vary as LLMs generate news across different categories. As a result, we have classified these diverse categories into four main types: documentintensive, number-intensive, knowledge-intensive, and general news, with details provided in Table 2.
In the data pre-processing stage, we divide a complete news article into three parts: the beginning text, the following text, and the reference information. The beginning text serves to guide the model in generating the continuation and is typically the opening portion of the news. During evaluation, the LLM is required to generate content following the beginning text. The following text comprises the subsequent sentences in the news article and serves as the ground truth for the continuation task. Finally, all the remaining text, after the beginning text is excluded, serves as a source of reference information. This section provides reference information for labeling and also acts as the reference text for the reference-based evaluation.

heading: 3.2 Unconstrained Hallucination Generation
text: Unlike directed hallucination generation (Li et al., 2023) or perturbation-based generation (Liu et al., 2022), we have adopted an unconstrained generation methodology for the continuation of natural language content, though it poses difficulties for subsequent annotations. This generation’s fashion
entails directly inputting the text to be continued into the model without any restrictive prompt instructions, thereby obtaining organic results.
Furthermore, current benchmarks for evaluating hallucination have predominantly relied on a single LLM to produce a hallucinated dataset. Notable examples include HaluEval (Li et al., 2023) and PHD (Yang et al., 2023b), which exclusively utilize ChatGPT, and FActScore (Min et al., 2023) and FACTOR (Muhlgay et al., 2024), which solely employ InstructGPT (Ouyang et al., 2022). In contrast, our methodology incorporates a suite of five distinct Chinese LLMs to generate hallucinated content. These models include ChatGLM2-6B (Du et al., 2022), Baichuan2-13B (Yang et al., 2023a), Qwen-14B (Bai et al., 2023), InternLM-20B (InternLM, 2023), and Xinyu-7B. For additional information about the Xinyu series models, please refer to the Appendix D.1.
For each input news article, we concurrently generate five candidate continuations using five different LLMs without constraint. Overall, our approach engenders a more unconstrained and heterogeneous generation of hallucinations, mitigating the bias that may arise from the use of a single model or constrained prompting.

heading: 3.3 Hallucination Ranking
text: Given the unconstrained nature of our paradigm, the task of discerning whether the generated content is indeed hallucinated presents a significant challenge. Upon generating the continuations, an exclusive dependence on human annotation would incur substantial costs, whereas a purely machinebased approach, such as utilizing GPT4, could potentially yield less accurate results.
To navigate these complexities, we have adopted a two-stage annotation. This approach begins with an initial stage of hallucination ranking (Section 3.3), designed to sort the generated content based on the likelihood of hallucination. The ranking is then followed by the second stage of automatic labeling and human rechecking (Section 3.4).
Hallucination ranking is a crucial step in selecting the most appropriate continuation from the five candidates generated by the five LLMs. This process relies on two critical metrics: fluency, ensuring that the continuation does not become too nonsensical, and likelihood, which stands for the likelihood of hallucination occurrence, ensuring that the continuation includes a detectable level of hallucinations. They are computed as follows.
Fluency This refers to the coherence and readability of the text (Liang et al., 2024). A fluent text should read smoothly and be grammatically correct in the context of the continuation. To assess fluency, a reward model developed by the Institute for Advanced Algorithms Research (IAAR) is employed, trained to score text fluency. The model is fine-tuned using a dataset annotated with news on an open-source reward model, Ziya model2.
Likelihood of Hallucination Occurrence This dimension evaluates the extent to which the continuation may contain hallucinated content. To estimate the probability, we evaluate the lexical correlation between the generated continuation and the reference information. The lower the correlation, the more likely hallucinations are to occur. Despite existing metrics, such as BLEU (Papineni et al., 2002) and ROUGE (Lin, 2004), we believe that these rule-based methods may not effectively discover hallucinations. Therefore, we propose the keyword precision (kwPrec) metric.
kwPrec uses an LLM (e.g., GPT3.5-Turbo) to extract keywords from the continuation and then determine whether these keywords have exact matches in the reference information. The ratio of all matches to the total keywords is then calculated. Since LLMs often extract appropriate keywords more effectively, kwPrec focuses more on factual relevance rather than expressional relevance. Fig. 3 illustrates the tokens segmented by kwPrec compared to those obtained by BLEU-4 and ROUGE-L. The prompt template utilized for extracting keywords is depicted in Fig. 13 within Appendix F.
With fluency and kwPrec, our task in the hallucination ranking step is to select one out of five candidate continuations that appears to be correct (highest in fluency) but is likely to contain hallucinations (lowest in kwPrec).
The specific steps are as follows (also shown in Algorithm 1). Step 1: Rank the five candidate con2https://huggingface.co/IDEA-CCNL/ Ziya-LLaMA-7B-Reward
Algorithm 1 Hallucination Ranking Require: candidate : list[str] Ensure: final : str candidate.sort(descend, by = fluency) picked← candidate[: 3] ▷ More fluency
picked.sort(ascend, by = kwPrec) final← picked[0] ▷ More Hallucination
tinuations in descending order by fluency. Step 2: Select the top three continuations with the highest fluency. Step 3: Rank these three continuations in ascending order by kwPrec. Step 4: Choose the continuation with the lowest kwPrec score. Following these steps, the continuation selected in Step 4 is the final choice. By employing such a ranking, it is guaranteed that, in the worst-case scenario, the final candidate ranks at least third in fluency and third in the likelihood of hallucination occurrence, achieving a balanced level.

heading: 3.4 Automatic Labeling and Human Rechecking
text: Through hallucination ranking, we can identify continuations that are both articulately expressed and likely to contain hallucinations. To detect continuations with confirmed hallucinations, we propose an annotation scheme that utilizes keywords, which includes automatic labeling and subsequent human verification, as shown in Fig. 4.
Automatic labeling We utilize the keywords identified by GPT3.5-Turbo from the candidate continuations, similarly to the process used in the computation of kwPrec previously. These keywords act as the focal points for subsequent verification. Thereafter, we employ GPT4-0613 (OpenAI, 2023) to perform annotation on these keywords. It evaluates the validity of the keywords in the continuations by conducting a cross-reference with the provided original news and provides explanations for any detected unreasonable keywords.
Human rechecking We undertake a manual, oneto-one verification process by analyzing the annotated results and explanations provided by GPT40613 against the original news. This step ensures the accuracy of the machine-generated annotations. In the end, instances verified as accurate by annotators comprise the final UHGEval dataset. For details on manual annotation, refer to Appendix C.1.

heading: 3.5 Dataset Statistics
text: Starting with 17,714 candidate hallucinated continuations, we curated a dataset of 5,141 hallucinated continuations, as detailed in the basic statistics in Table 3. For further analysis, the data volume of each step in the dataset creation pipeline, and an example of the dataset, please refer to Appendix C.2, Appendix C.3 and Appendix C.4, respectively.

heading: 4 Experiments

heading: 4.1 Models
text: Given that our dataset is tailored for the Chinese language generation domain, we selected eight widely used Chinese LLMs and three LLMs from OpenAI. These LLMs are from eight base models: Aquila2 (BAAI, 2023), Baichuan2 (Yang
et al., 2023a), GLM (Du et al., 2022), GPT3, InternLM (InternLM, 2023), Qwen (Bai et al., 2023), BLOOMZ (Muennighoff et al., 2023), and LLaMA2 (Touvron et al., 2023). Refer to the Appendix D.1 for a detailed overview of the LLMs used in the experiments.

heading: 4.2 Evaluation Forms
text: In this study, we conducted a detailed analysis of evaluation methods across three dimensions: form, metric, and granularity. A more comprehensive report can be found in the Appendix D.2. Here, we introduce the three forms of evaluation.
Firstly, there is the discriminative evaluation, which involves having the model determine whether a continuation contains hallucinations. Secondly, similar to discriminative evaluation, selective evaluation allows LLMs to choose the continuation without hallucinations from options with and without such content. Lastly, we have generative evaluation. Specifically, the LLM under evaluation is provided with a beginning text and is then tasked with generating a continuation. Subsequently, various reference-based techniques are employed to assess whether the generated continuation includes hallucinations.

heading: 4.3 Evaluation Framework
text: To accommodate different forms of evaluation methods, we have developed a data-secure, easyto-extend, and easy-to-use evaluation framework, as illustrated in Fig. 5. Refer to Appendix D.3 for a more detailed understanding of the various layers of the framework. UHGEval is both intuitive and secure for users, offering efficient usage while concurrently ensuring the integrity of experimental results through robust resistance to exceptions and support for resuming evaluations post unexpected interruptions. For developers and researchers, the modules within
3https://openai.com
the Dependency and Evaluator layers are fully interchangeable, thereby affording considerable flexibility for expansion.

heading: 4.4 Experimental Setup
text: Prompt Engineering We apply the technique of “intent + instruction + 3-shot (explainable) prompting.” Intent delineates the role, instruction outlines the task, and the prompt incorporates three examples to aid the few-shot learning (Chen et al., 2024; Yu et al., 2024b). Furthermore, political content in examples is prohibited to adhere to content policies from model providers. Explainable prompting entails not merely acquiring results, but also eliciting the model’s rationale behind its responses. Refer to Appendix F to view the complete templates.
Example Balancing To guarantee the reliability of experimental outcomes for all LLMs, we meticulously balance examples in discriminative and also in selective evaluations. Specifically, the LLM under evaluation will encounter an equal number of examples with and without hallucinations.
Hyperparameter Settings Managing parameters for heterogeneous LLMs is a multifaceted endeavor, as different LLMs feature unique interface designs, and the same parameters can have varying implications across LLMs. Despite these challenges, we commit to the principle of “guaranteeing overall output determinism while allowing for slight randomness, and aiming for consistent parameter settings across models.” Consequently, we set the temperature to 0.1, the top_p to 0.9, the top_k to 5, and the random seed to 22.
Metrics For discriminative and selective evaluation, accuracy serves as the metric. For generative evaluation, metrics consist of 4-gram BLEU (BLEU-4), the longest common subsequence-based ROUGE (ROUGE-L), kwPrec, and BERTScore.

heading: 4.5 Results and Analysis
text: Results are presented in Table 4 and Table 5.
Discriminative Evaluation Initially, the GPT series models’ performance is notably superior in discriminative evaluation, showcasing their formidable foundational capabilities in knowledge recall, utilization, and judgment. Moreover, a comparison of experimental outcomes at the keyword and sentence levels reveals that accuracy is generally superior at the keyword level. This could stem from the fact that the hallucinated continuations in
our dataset exhibit sufficient fluency, aligning with the fluency distribution of LLM outputs. This can potentially confuse the evaluated LLM, complicating the judgment of the continuation’s authenticity. Conversely, keywords bypass fluency concerns, rendering keyword-level evaluation more amenable to LLMs. This observation implies that detecting hallucinations could be more dependable at the keyword level compared to the sentence level.
Selective Evaluation Firstly, GPT4-1106 clinches the top spot, reaffirming the formidable foundational capabilities of the GPT series models. Concurrently, Xinyu2-70B attains second place, excelling as a model trained on the Chinese news corpus. This achievement, to a degree, confirms the merit of domain-specific LLMs. Secondly, when comparing the outcomes of the selective evaluation with those of the discriminative evaluation at the sentence level, most LLMs exhibit improved accuracy. We think, furnishing LLMs with more contrasting information alleviates the demand for the model’s fact recall, thus diminishing the challenge of selective evaluation. Therefore, we posit that selective evaluation is comparatively simpler for LLMs. Thirdly, a decline is observed in discriminative evaluation outcomes from GPT4-0613 to GPT4-1106, whereas selective evaluation outcomes register a notable increase of around 5%. This substantiates the “seesaw phenomenon,” wherein certain capabilities are enhanced while others may regress, in tandem with the model’s upgrade (Zheng et al., 2023). This suggests that the decision to either enhance a single capability individually or to balance multiple capabilities is critical.
Generative Evaluation Overall, InternLM-20B, Xinyu2-70B, and Aquila-34B have achieved commendable results, but the performance of Aquila34B could be attributed to its comparatively shorter average generation length. Additionally, the GPT series exhibits subpar performance, possibly due to the insubstantial amount of Chinese data in its training corpus. After all, the Chinese data incorporated into GPT’s training from the Common Crawl corpus comprises less than 5%4.
Evaluations by Type We focus on selective evaluation results and perform a comprehensive breakdown analysis of these across the four types, as 4https://commoncrawl.github.io/ cc-crawl-statistics/plots/languages.html
illustrated in Table 5. Initially, most LLMs demonstrate enhanced accuracy for knowledge-intensive and document-intensive news. This may be because the training datasets for LLMs typically include substantial human knowledge and official documentation of major historical events. Furthermore, the majority of LLMs show reduced accuracy in general and number-intensive news. General news often contains societal minutiae, which are not the focus of LLM training. Regarding numberintensive news, it poses a considerable challenge
for LLMs, given that encoding identical numbers with varied historical meanings is complex. However, GPT4-1106 attains especially high scores in the demanding number-intensive news.

heading: 4.6 Further Discussion
text: Each of the three evaluation forms possesses distinct advantages and drawbacks. Discriminative evaluation is often the method of choice for a range of standard benchmarks (Li et al., 2023; Cheng et al., 2023). This approach is intuitive, and the construction of evaluation prompts is straightforward. Selective evaluation resembles discriminative evaluation but is marginally less demanding because it includes a reference option for contrast. In both discriminative and selective evaluations, certain models might be suspected of conjecturing answers from a few shots due to inadequate reasoning skills, which can undermine the reliability of the outcomes. Consequently, the use of explainable prompting becomes essential. Generative evaluation most closely mirrors real-world applications. However, the generated content is unrestricted, which poses challenges for even the most dependable reference-based evaluation techniques.
Therefore, employing a combination of metrics simultaneously, including lexical evaluation based on token coverage and semantic evaluation based on textual similarity, is imperative.
The foundational capabilities required of LLMs can be arrayed on a spectrum from simple to complex: generative, selective, and discriminative evaluation. Generative evaluation entails the direct invocation of parameters for continuation, bypassing the need for an extensive grasp of instructions. Selective evaluation necessitates a degree of inferential reasoning but offers comparative choices, rendering the level of difficulty moderate. Conversely, discriminative evaluation demands the precise retrieval of facts, thereby increasing the challenge.
Moreover, various evaluations cater to different application contexts. Should the objective be to solely improve the model’s capacity for reliable continuation, generative evaluation would suffice. In the training of a dependable chatbot, selective and discriminative evaluations prove suitable. When aiming to train a reward model, selective evaluation is beneficial, offering evaluation for positive and negative instances. If the goal is to enhance the model’s ability to recall and apply knowledge, discriminative evaluation emerges as the demanding option.

heading: 5 Conclusion
text: LLMs are rapidly evolving, heralding a new era of potential applications within the realm of professional content generation. The progression of LLMs in this domain necessitates the establishment of robust benchmarks to steer their development effectively. In this work, we introduce a novel hallucination benchmark dataset using an unconstrained fashion, encompassing more than 5,000 instances annotated at the keyword level. Additionally, we propose a secure, scalable, and user-friendly evaluation framework to facilitate comprehensive assessments. Through meticulous experimentation on eleven prominent LLMs, our study has unearthed a series of enlightening findings. Looking ahead, our research endeavors will persist in exploring the intricacies of hallucination phenomena within professional content generation, aiming to further understand and enhance LLM capabilities.

heading: Limitations
text: Dataset Firstly, although we have utilized hallucination ranking, automatic labeling, human
rechecking, and various other techniques mentioned in Appendix C to ensure the quality of data annotation, with over 5,000 data entries, there is still a possibility of labeling errors. We have mobilized the power of the open-source community to collectively improve our dataset. Secondly, the dataset creation process is flexible, allowing for dataset expansion into English and broader domains, such as mathematical reasoning and programming codes. Thirdly, a minor error in the dataset creation process has resulted in a relatively unbalanced distribution of the dataset across the five different LLMs used for generation. A detailed analysis of this issue can be found in Appendix G.
Framework Although our framework simplifies the integration of LLMs through APIs or vLLM5, users seeking to utilize custom or diverse HuggingFace models may face initial hurdles. We need to further enhance the usability of our framework.
Constrained v.s. Unconstrained We have determined that constrained generation cannot fully reflect real-world applications, but empirical analysis is required to prove this point. This may involve constructing a text classifier to determine the type of hallucination, followed by comparing the distribution of hallucinations in our dataset with those in other benchmark datasets to observe any significant deviations. We leave this for future work.

heading: Acknowledgments
text: This work was supported by the National Natural Science Foundation of China (Grants No. 62072463, 71531012), the National Social Science Foundation of China (Grants No. 18ZDA309), the Research Seed Funds of the School of Interdisciplinary Studies at Renmin University of China, and the Opening Project of the State Key Laboratory of Digital Publishing Technology of the Founder Group.

heading: Appendices

heading: A Comparisons with Other Datasets 14
text: A.1 TruthfulQA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 A.2 HaluEval . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 A.3 HaDes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 A.4 Why Is Unconstrained Generation Important? . . . . . . . . . . . . . . . . . . . . . . . 14

heading: B More Related Works 15
text: B.1 Large Language Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 B.2 Hallucinations in LLM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

heading: C The UHGEval Dataset 16
text: C.1 Dive into Human Rechecking Process . . . . . . . . . . . . . . . . . . . . . . . . . . . 16 C.2 Analysis of the Final Dataset . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16 C.3 Data Volume for Each Step . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 C.4 An example from the UHGEval Dataset . . . . . . . . . . . . . . . . . . . . . . . . . . 18

heading: D Experiments 19
text: D.1 LLMs Employed in This Research . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 D.2 Evaluation Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 D.3 UHGEval Framework in Detail . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20

heading: E Figures in Chinese 21

heading: F Prompt Templates 22

heading: G Clarification of Imbalance of the Dataset in Term of Models 27
text: G.1 Imbalance Phenomenon . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27 G.2 Does the Imbalance Lead to Unreliable Outcomes? . . . . . . . . . . . . . . . . . . . . 27

heading: A Comparisons with Other Datasets
text: Below are specific comparisons with other datasets and the significance of unconstrained generation.

heading: A.1 TruthfulQA
text: The TruthfulQA dataset encompasses three modes of evaluation, with the primary mode being generative. In this mode, a problem is presented to the model, which then freely generates content that is assessed by humans or a fine-tuned GPT-judge. The other two modes are single- / multiple-choice questions. In these modes, a problem along with reference options is provided, the model makes a selection, and accuracy is calculated.
Figure 1 in the TruthfulQA paper includes statements indicating that some content is freely generated by GPT-3. This might be somewhat misleading. The content is used solely to evaluate the performance of the GPT-3 model in generative evaluation and is not part of the dataset. The actual free generation pertains to the "reference options" in the single- / multiple-choice questions. These reference options are manually crafted in TruthfulQA.
Appendix C of the paper details the method used to create the reference options:
Reference answers for each question in TruthfulQA are constructed as follows:
We take a set of true answers directly from Wikipedia (or the listed source). We then try to provide coverage of common variations on this answer...
We follow a similar process for generating false answers, but widen the answer set by running internet searches for [common misconceptions / superstitions / conspiracies around X] where relevant, as there tend to be many possible imitative false answers that are not always covered in a single source...

heading: A.2 HaluEval
text: The problem types within this benchmark are all judgment questions, tasked with determining whether an option contains hallucinations. Accordingly, they also provide reference options. However, their method of generating these options is targeted. An example of how they generate options is: "You are trying to answer a question but misunderstand the question context and intention." They then take such generated texts and real texts,
placing them together for downstream models to evaluate for the presence of hallucinations.

heading: A.3 HaDes
text: HaDes evaluates a model’s ability to identify hallucinated words within a given text. However, the method used to generate these hallucinations involves randomly altering correct text, thereby transforming some words into hallucinations. This approach to generating errors leads to a distributional bias compared to the hallucinations that arise from the model’s free output.
In summary, most existing datasets related to hallucinations are purposefully and manually generated with constraints. They do not represent the hallucinations that might be collected while the model is addressing user queries or responding to users in real-world scenarios. This raises the question: Are the errors generated in this manner truly reflective of the mistakes a model would make? Hence, in creating our dataset, we allowed the model to output freely, collecting only those portions where hallucinations occurred. This represents one of the major challenges in our work.

heading: A.4 Why Is Unconstrained Generation Important?
text: In datasets like TruthfulQA, HaluEval, and HaDes, it’s challenging to pinpoint exactly why a model might produce hallucinations. These texts, potentially containing inaccuracies, are designed to assess whether a downstream model can identify errors within a text. However, our dataset enables a genuine evaluation of model hallucinations, even tracing their origins. For instance, in our dataset, the entry with ID doc_000002 features hallucinations generated by the Baichuan2-13B model. The terms related to "economic development" and others, totaling five words, are involved in these hallucinations, while words like "China" and another set of five words are not. This distinction allows us to investigate whether there are differences in the token logits, the states of hidden layers, etc., between the words associated with hallucinations and those without, in the context of the Baichuan model. Theoretically analyzing the causes of hallucinations within the Baichuan model is part of our ongoing work. This approach is something that other benchmarks cannot offer, as their hallucinations are not freely produced by the model, and in some cases, not even generated by models.

heading: B More Related Works

heading: B.1 Large Language Models
text: Language models are pivotal in computer science, evolving from statistical language models to neural language models, to pre-trained language models (PLMs), and now to the current generation of LLMs. The advent of models such as ChatGPT has seen contemporary LLMs exhibit new capabilities in handling complex tasks. These models can manage few-shot tasks via in-context learning and tackle mixed tasks by following instructions (Zhao et al., 2023).
LLMs can be classified according to two dimensions. The first dimension concerns the openness of the model weights. For example, open-source models include Meta’s LLaMA (Touvron et al., 2023), Tsinghua University’s GLM (Du et al., 2022), and Alibaba’s Qwen (Bai et al., 2023), while closedsource models feature OpenAI’s GPT (OpenAI, 2023), Baidu’s ERNIE Bot (Sun et al., 2021), and Anthropic’s Claude6, among others. The second dimension differentiates between the use of a PLM or a supervised fine-tuned (SFT) model for specific inferences (Zhu et al., 2024). A PLM is a language model trained on extensive unlabeled textual data to discern underlying patterns, structures, and semantic knowledge within the corpus. Conversely, an SFT model involves further training a PLM with labeled datasets tailored to a specific task, to improve performance in that area. Many open-source models, including LLaMA, GLM, and Qwen, have made their PLM weights publicly available. For SFT models, users can access the chat variants of open-source models or the API services provided by closed-source models. In our research, we focus primarily on evaluating closed-source GPT series models and open-source Chinese SFT models.

heading: B.2 Hallucinations in LLM
text: Despite remarkable advancements in LLMs, they continue to encounter challenges, with hallucination being one of the most notable. Hallucination in language models refers to generating content that strays from factual accuracy, leading to unreliable outputs. Hallucinations occur when the generated content is not aligned with user input, deviates from the model’s previous outputs, or is at odds with established real-world knowledge (Zhang et al., 2023). 6https://www.anthropic.com/index/ introducing-claude
Specific examples include inaccuracies in age, currency, scores, and other numerical values; citing fictional statements; inventing non-existent characters; and muddling timelines by merging events from different periods (Rawte et al., 2023).
Regarding the causes of hallucinations, several factors can be responsible (Zhang et al., 2023). One contributing factor is the use of inaccurate or incomplete training data. During training, LLMs finetune their parameters with vast quantities of text data. However, this data may be flawed, harboring errors, inaccuracies, or gaps in information. Another factor involves inconsistencies in contextual information. While LLMs typically consider previously generated context when producing content, challenges in managing long-term dependencies or understanding complex contexts can result in inconsistencies. Additionally, hallucinations can arise from lacking or erroneous world knowledge. Although LLMs gain considerable world knowledge via training data, they may be deficient in specific domain knowledge or misinterpret certain facts, leading to hallucinations. Furthermore, model limitations, including generation strategies and alignment methods, can also play a role in hallucinations during content creation.

heading: C The UHGEval Dataset

heading: C.1 Dive into Human Rechecking Process
text: Least Hallucination Principle The keywordbased labeling scheme has inherent limitations. Languages exhibit a dependency structure (de Marneffe and Nivre, 2019). For instance, in the phrase “The rainbow is black,” the words “rainbow” and “black” exhibit interdependence. One could contend that “black” is incorrect, while another could maintain that “rainbow” is erroneous, given that “night” is typically described as black. To address the challenges stemming from language dependency structures, we have adopted the Least Hallucination Principle. If a set of words can be selected, and their replacement with contextually appropriate words yields a reasonable sentence, then such a set of words is designated as a hallucinated word group. The words selected for annotation must meet the condition of comprising the minimal number of words in the group, as illustrated in Equation 1. In the equation, W is the set of keywords in a sentence, w is the hallucinated word group, correct(·) is the correction function that modifies hallucinated words to non-hallucinated words, and hallucinated(·) assesses whether a sentence composed of keywords hallucinated.
min |w| s.t. w ⊂W
w′ = correct(w) false = hallucinated(W −w +w′)
(1)
By this principle, within the phrase “Journey to the West is an American novel and one of the Four Great Classics,” the word “American” would be marked for annotation, as altering this single keyword to “Chinese” dispels the hallucination throughout the sentence.
Engagement of Annotators Additionally, we acknowledge that hallucination annotation may become somewhat tedious. Consequently, annotators are integrated throughout the entire process, participating in discussions instead of solely evaluating the accuracy of machine annotations. This approach also yields benefits for our work. For example, an annotator with a journalism background offered valuable professional insights into pinpointing news-related hallucinations, emphasizing that fact increment is a critical aspect of news writing.
Annotation Team Our annotators are all Chinese nationals with Chinese as their native language, each holding at least a Master’s degree in Journalism. We collaborated with a well-known, sizable news organization in China, Xinhua News Agency. Some of their staff joined our research team and participated in data annotation for this project. There were a total of 9 annotators involved in this project, with a gender ratio of 1:2 (male to female). Regarding their compensation, they first received a standard employee salary. Additionally, they were paid an extra 3 RMB for each data item annotated, with each item taking about 40 seconds to annotate. Besides the annotators, our engineering team and experts from the journalism industry at Xinhua News Agency participated in the data review process, totaling 3 people. Our main responsibility was to supervise and review the quality of the annotations. The entire annotation process lasted for 22 days.

heading: C.2 Analysis of the Final Dataset
text: We developed a conversion rate chart to depict the transition from candidate hallucinations to the final dataset, as depicted in Fig. 6. The conversion rate can be interpreted as the likelihood of hallucinations occurring across various categories. Our observations indicate a higher likelihood of hallucinations in number-intensive and general news, whereas this likelihood is reduced in knowledgeintensive and document-intensive news.
By analyzing the hallucinated word cloud depicted in Fig. 7 for each news category, we can draw the following conclusions: Number-intensive news often includes numeric values that are challenging to remember, like 0.09% and 6:3, which pose difficulties for both LLMs and humans. General news encompasses a diverse vocabulary, featuring terms such as “social media” and “friendship,” which are often deemed less critical and thus challenging to incorporate into the training corpora of many LLMs. Knowledge-intensive news frequently fea-
tures terms such as “according to incomplete statistics” and “key technology,” which are prevalent in technical literature. However, LLMs may not always use these terms appropriately. Documentintensive news often contains terms associated with official statements, such as “representation,” “president,” and “spokesperson.” This suggests that LLMs are susceptible to introducing unauthorized alterations to the content of documents.

heading: C.3 Data Volume for Each Step
text: In this section, we present the data volume at various stages of our dataset creation process for reference and transparency.
• Data volume of original news dataset: 737,766
• Data volume after preprocessing: 25,005 (filtering out outliers in dimensions such as length and news type)
• Data volume after generating candidate hallucination text: 17,503 (filtering out data items that did not generate appropriate continuations, such as those with excessively short length or insufficient extracted keywords)
• Data volume with hallucination in machinelabeled text: 8,314 (filtering out texts deemed by the machine to lack hallucination)
• Data volume after human annotator labeling: 5,141 (filtering out instances not verified as hallucination upon manual review, or deemed inappropriate, such as those repeating previous content or generating text no longer of news type but rather comprehension questions)
C.4 An example from the UHGEval Dataset
{ "id": "num_000432", "headLine": "(Society) Jiangsu's First Selection of the Top 100 Green Foods Most Loved by Consumers", "broadcastDate": "2015-02-11 19:46:49", "type": "num", "newsBeginning": "Xinhua News Agency, Nanjing, February 11 (Reporter Li Xiang) 'Food is the paramount necessity of the people, and safety is the top priority of food.' On February 11, Jiangsu announced the results of the 'First Consumers' Favorite Green Foods' selection, with Lao Shan honey and 100 other foods receiving the title of 'Consumers' Favorite Green Food'.", "hallucinatedContinuation": "Jiangsu is one of the most developed provinces in the country in terms of green food production.", "generatedBy": "InternLM_20B_Chat", "appearedKeywords": ["Jiangsu", "national", "green food production"], "allKeywords": { "Jiangsu": "reasonable", "national": "reasonable", "green food production": "reasonable", "developed": "unreasonable, there is no factual evidence to prove that Jiangsu is one of the provinces with developed green food production in the country, but what can be confirmed is that Jiangsu has active practices and promotions in green food production", "province": "reasonable", "one of": "unreasonable, there is no specific factual evidence to show that Jiangsu is one of the developed provinces in terms of green food production in the country" }, "realContinuation": "61 award-winning production enterprises jointly signed a integrity pact, jointly building a green food integrity alliance.", "newsRemainder": "61 award-winning production enterprises jointly signed an integrity pact, jointly building a green food integrity alliance. This is an important measure for Jiangsu to ensure food safety and promote green food production.\n..." }
Figure 8: An example from the UHGEval dataset. (In English)
{ "id": "num_000432", "headLine": "（社会）江苏首次评选消费者最喜爱的百种绿色食品", "broadcastDate": "2015-02-11 19:46:49", "type": "num", "newsBeginning": " 新华社南京2月11日电（记者李响）“民以食为天，食以安为先”。江苏11日发布“首届消费者最喜爱的绿色食品”评选结果， 老山蜂蜜等100种食品获得消费者“最喜爱的绿色食品”称号。", "hallucinatedContinuation": "江苏是全国绿色食品生产最发达的省份之一。", "generatedBy": "InternLM_20B_Chat", "appearedKeywords": [ "江苏", "全国", "绿色食品生产"], "allKeywords": { "江苏": "合理", "全国": "合理", "绿色食品生产": "合理", "发达": "不合理，没有事实证明江苏是全国绿色食品生产发达的省份，但可以确定的是，江苏在绿色食品生产上有积极的实践和推动", "省份": "合理", "之一": "不合理，没有具体的事实证据表明江苏是全国绿色食品生产发达的省份之一" }, "realContinuation": "61家获奖生产企业共同签署诚信公约，共建绿色食品诚信联盟。", "newsRemainder": "61家获奖生产企业共同签署诚信公约，共建绿色食品诚信联盟。这是江苏保障食品安全、推动绿色食品生产的重要举措。 \n 此次评选由江苏省绿色食品协会等部门主办，并得到江苏省农委、省委农工办、省工商局、省地税局、省信用办、省消协等单位大力支持。评 选历时4个多月，经企业报名、组委会初筛、消费者投票等层层选拔，最终出炉的百强食品榜单由消费者亲自票选得出，网络、短信、报纸及现场投 票共310多万份票数，充分说明了评选结果的含金量。\n 食品安全一直是社会关注的热点。此次评选过程中，组委会工作人员走街头、进超市， 邀请媒体、消费者、专家深入产地开展绿色食品基地行，除了超市选购外，还搭建“诚信购微信商城”“中国移动MO生活绿色有机馆”等线上销售平台， 开创江苏绿色食品“评展销”结合新局面……" }
Figure 9: An example from the UHGEval dataset. (In Chinese)

heading: D Experiments

heading: D.1 LLMs Employed in This Research
text: All LLMs used in this study are detailed in Table 6.
GPT represents a series of LLMs developed by OpenAI (OpenAI, 2023). In this study, GPT3.5Turbo, GPT4-0613, and GPT4-1106 are utilized. GLM constitutes a pre-training framework proposed by Tsinghua University (Du et al., 2022), and the ChatGLM2-6B chat model is employed. InternLM serves as an open-source, lightweight training framework, with its development team releasing a spectrum of models utilizing this framework (InternLM, 2023); the InternLM-20B opensource chat model is utilized in the present work. Baichuan2 comprises a series of expansive, multilingual base language models (Yang et al., 2023a), with both the open-source Baichuan2-7B chat model and the closed-source Baichuan2-53B chat model being employed in this investigation. Qwen encompasses a language model series characterized by distinct models with varying parameter counts (Bai et al., 2023), and the Qwen-14B opensource chat model is utilized in the current study. Aquila2 represents a language model series devised by BAAI, noted for surpassing comparable models in terms of performance (BAAI, 2023), and the Aquila2-34B chat model is employed in this research.
Besides, the Xinyu series models are the results of a collaborative research and development effort between the Institute for Advanced Algorithms Research, Shanghai (IAAR, SH), and the State Key Laboratory of Media Convergence Production Technology and Systems of the Xinhua News Agency. Xinyu-7B is an augmented large-scale lan-
guage model derived from the foundational model, BloomZ-7B (Muennighoff et al., 2023) through continued pre-training, news-specific fine-tuning, and alignment optimization. And, Xinyu2-70B is developed based on the open-source LLaMA270B (Touvron et al., 2023) framework, incorporating expansions to the Chinese lexicon, ongoing pre-training, and news-specific fine-tuning, thereby endowing it with a robust foundational capability in the news domain.

heading: D.2 Evaluation Method
text: The evaluation of hallucinations can be decomposed into three principal dimensions: form, metric, and granularity. Form concerns how the model interacts with the evaluation dataset; metric refers to the precise computational approach utilized for performance assessment; and granularity signifies the depth of detail considered in the evaluation of hallucinations.
Form This encompasses human evaluation, discriminative evaluation, selective evaluation, and generative evaluation, among others. Human evaluation entails the direct application of human judgment to determine if the model’s output contains hallucinations, representing a critical evaluation form (Chang et al., 2024). However, the drawbacks of this approach are evident: evaluating too many data points is tantamount to annotating a new dataset, with the associated time and financial expenditures proving prohibitive.
Discriminative evaluation enables LLMs to respond with binary answers of “yes” or “no” (Li et al., 2023; Cheng et al., 2023). Specifically, this evaluation modality involves presenting the LLM under scrutiny with an initial text followed by a continuation that may or may not include hallucinations. The LLM is tasked with producing a verdict as to the presence of hallucinations. Owing to the efficacy of few-shot prompting, this evaluation paradigm is relatively uncomplicated for LLMs to administer, as it facilitates the elicitation of the requisite responses. However, this method depends solely on the LLM’s ability to draw upon the knowledge encoded within its parameters, necessitating the concurrent application of knowledge and reasoning, and thus requiring a robust foundational model capacity.
Selective evaluation allows LLMs to tackle multiple-choice questions by choosing between option A or B, as exemplified by PandaLM (Wang
et al., 2024). Specifically, in selective evaluation, the LLM under evaluation is presented with an initial text followed by two continuations: one that includes hallucinations and another that does not. The LLM’s objective is to identify which of the two is hallucinated. This assessment method offers the LLM more contextual information than discriminative evaluation, thereby alleviating the burden of fact-checking and lessening the dependence on retrieving facts from its parameters. Consequently, this reduces the level of difficulty for the LLM.
However, both discriminative and selective evaluations encounter a substantial challenge. They are predicated on the assumption that “LLMs’s capacity to produce reliable text is contingent upon their discernment between hallucinated and nonhallucinated content.” These methods do not simulate the evaluation of the model’s output for hallucinations. Consequently, generative evaluation is crucial as it directly evaluates the presence of hallucinations in the text generated by the LLM under evaluation. However, the challenge arises from the fact that it is not feasible to automatically and accurately ascertain if the newly generated text is hallucinated; if it were, annotated datasets would be redundant. In scenarios of unrestrained text generation, this issue becomes increasingly complex. This complexity stems from the fact that text generated without constraints may introduce a multitude of entities and facts absent in the reference material, complicating the verification of their accuracy. Despite these hurdles, generative evaluation continues to be a predominant strategy in Natural Language Generation (NLG) tasks (Novikova et al., 2017).
Metric Metrics include classification metrics such as accuracy, precision, recall, and others, which are applicable to human evaluation, discriminative evaluation, and selective evaluation. Generative evaluation, on the other hand, encompasses both lexical and semantic metrics. Lexical metrics evaluate the extent of token overlap between the generated text and the reference information, including metrics such as BLEU (Papineni et al., 2002), ROUGE (Lin, 2004), and the newly proposed metric by us, kwPrec. Semantic metrics gauge the similarity in meaning between sentences, with examples including BERTScore (Zhang et al., 2020), GPT-judge (Lin et al., 2022), and GPTScore (Fu et al., 2023), among others.
Granularity Evaluations can be conducted at both the sentence and keyword levels. Owing to our annotation methodology, our dataset is marked at the keyword level to signify instances of hallucinations. This approach affords a broader spectrum of possibilities for configuring the evaluation task, enabling the evaluated model to address the presence of hallucinations at either the keyword level, the sentence level, or even the document level.

heading: D.3 UHGEval Framework in Detail
text: The framework comprises four ascending layers: the dependency layer, the evaluator layer, the core layer, and the interface layer.
The dependency layer defines the essential foundational components needed for the evaluation framework, including datasets, LLM hubs, and various metrics. Importantly, each component is designed for extensibility: datasets can be replaced with custom ones, LLMs can be integrated via APIs or platforms like Hugging Face7, and metrics can be customized to fit specific needs.
The evaluator layer, constituting the second layer, centers on an abstract class, Evaluator, and its various implementations. Within this layer, three distinct types are implemented: GenerativeEvaluator, DiscriminativeEvaluator, and SelectiveEvaluator. Users may also engineer custom evaluators, contingent upon adherence to the interface specifications of the abstract class, necessitating merely three function overloads.
The core layer, representing the third stratum, comprises two principal modules: experiment.py and analyst.py. The former facilitates experiments involving multiple LLMs, evaluators, and processes, whereas the latter is tasked with the statistical analysis of experimental outcomes.
The interface layer, serving as the final layer, orchestrates the user’s interaction with UHGEval. To streamline the initiation process, a succinct 20- line demonstration is offered, alongside a run.py script for launching experiments through the command line.
7https://huggingface.co/models
E Figures in Chinese

heading: F Prompt Templates
text: In these templates, the orange text represents intent and instruction, the green text represents demonstrations, and the black text represents specific questions. The template may be very long, and we may use ellipses to omit some content in the middle. The original templates are in Chinese, and we also provide English translations.

heading: G Clarification of Imbalance of the Dataset in Term of Models
text: G.1 Imbalance Phenomenon
The distribution of the final hallucination dataset in terms of five different LLMs used in the generation is shown in Table 7.
You may notice a disproportionately high representation of the Baichuan model, a discrepancy linked to an oversight at the outset. Initially, the generation of candidates did not fully leverage the available models, primarily because the Baichuan models exhibited superior instruction-following capabilities for data generation. Consequently, we solely utilized five instances of the Baichuan model for ranking to generate candidate data items. It was only later that four additional models were incorporated, employing a total of five distinct model instances for ranking to generate candidate data items. This approach resulted in a final dataset that lacks a relatively balanced distribution among the different models.

heading: G.2 Does the Imbalance Lead to Unreliable Outcomes?
text: A quick answer is: No. Here are the justifications. We conducted a simple empirical study. Since the generated hallucinated texts are only used in selective and discriminative evaluations, if the imbalance significantly affects the experimental outcomes, it would only impact these two types of assessments. Fortunately, we have saved every piece of intermediate experimental results, allowing us to uniformly sample those results across models and re-aggregate the results to observe changes in key metrics like average accuracy. Specifically, we uniformly and randomly sampled 327 data points for each model and used a combined dataset of 327*5=1635 data points to re-aggregate the results. Below is a comparison of the experimental outcomes. Table 8 and Table 9 present the original
results and the new results, respectively, while Table 10 displays the differences between them.
In analyzing the differences, we calculated three metrics to illustrate the magnitude of change. The average change in the three columns of accuracy metrics is approximately -0.0147, with a standard deviation of 0.0150. Furthermore, the average absolute change in their rankings is 0.606 (because only six pairs of closely ranked models undergo internal swaps). Lastly, upon separately examining the Baichuan model, which was anticipated to be most affected, we found its change to be not the greatest. Therefore, we can assert that the variance in model proportions has an insignificant impact on the outcomes.
Moreover, intuitively, since all these models are Chinese text generation models and the content they generate was also reviewed by our manual annotators, we eliminated some data items that significantly deviated the text type from the mean during annotation (for example, repetitions of previous content, or continuations that diverged from news to generating a reading comprehension question, etc.). This enhanced the overall consistency of the final dataset, making the task of distinguishing hallucinations in the experiment unrelated to the source of those hallucinations.

heading: Future Work
text: We leave this for future work