CARE-MI: Chinese Benchmark for Misinformation Evaluation in Maternity and Infant Care
Venue: NeurIPS 2023 Datasets and Benchmarks Poster (2023)
Keywords: Benchmark
NLP
Large Language Model
Evaluation
Medical Domain
Text Generation
TL;DR: Propose a benchmark for the evaluation of misinformation of large language models on the topic of maternity and infant in Chinese

Abstract: The recent advances in natural language processing (NLP), have led to a new trend of applying large language models (LLMs) to real-world scenarios. While the latest LLMs are astonishingly fluent when interacting with humans, they suffer from the misinformation problem by unintentionally generating factually false statements. This can lead to harmful consequences, especially when produced within sensitive contexts, such as healthcare. Yet few previous works have focused on evaluating misinformation in the long-form (LF) generation of LLMs, especially for knowledge-intensive topics. Moreover, although LLMs have been shown to perform well in different languages, misinformation evaluation has been mostly conducted in English. To this end, we present a benchmark, CARE-MI, for evaluating LLM misinformation in: 1) a sensitive topic, specifically the maternity and infant care domain; and 2) a language other than English, namely Chinese. Most importantly, we provide an innovative paradigm for building LF generation evaluation benchmarks that can be transferred to other knowledge-intensive domains and low-resourced languages. Our proposed benchmark fills the gap between the extensive usage of LLMs and the lack of datasets for assessing the misinformation generated by these models. It contains 1,612 expert-checked questions, accompanied with human-selected references. Using our benchmark, we conduct extensive experiments and found that current Chinese LLMs are far from perfect in the topic of maternity and infant care. In an effort to minimize the reliance on human resources for performance evaluation, we offer off-the-shelf judgment models for automatically assessing the LF output of LLMs given benchmark questions. Moreover, we compare potential solutions for LF generation evaluation and provide insights for building better automated metrics.

CARE-MI: Chinese Benchmark for Misinformation Evaluation in Maternity and Infant Care
Tong Xiang Liangzhi Li Wangyue L Mingbai Bai Lu Wei
Bowen Wang* Noa Garcia
Meetyon AI Lab, Xiamen Key Laboratory of Women's 's Internet Health Management,
Southwest University of Finance and Economics, * Osaka University
(xiangtong, liliangzhi, liwangyue, baimingbai, weilu (alllu/xxayyouui..comm com, {wang, noagar cidda. osaka-u.ac.jp
Abstract
The recent advances in natural language processing (NLP), have led to new trend applying large language models (LLMs) to real-world scenarios. While the latest LLMs are astonishingly fluent when interacting with humans, they suffer from the misinformation problem by unintentionally generating factually false statements. This lead harmful consequences, especially when produced within sensitive contexts, such as healthcare. Yet few previous works have fo- cused on evaluating misinformation in the long-form (LF) generation of LLMs, especially for knowledge inteesiee topics. Moreover, although LLMs have been shown perform well in different languages, misinformation evaluation has been mostly conducted in English. To this end, we present a benchmark, CARE-MI, for evaluating LLM misinformation in: 1) a sensitive topic, specifically the maternity and infant care domain; and 2) language other than English, namely Chinese. Most importantly, we provide an innovative paradigm for building LF generation evaluation benchmarks that can be transferred to other knowledge-intent inteeeniiiiiieve do- mains and low-resourced languages. Our proposed benchmark fills the gap between the extensive usage of LLMs and the lack of datasets for assessing the misinfor- mation generated by these models. contains 1,612 expert-checked questions, accompanied with human-selected references. Using our benchmark, we conduct extensive experiments and found that current Chinese LLMs are far from perfect in the topic of maternity and infant care. In effort to minimize the reliance on human resources for performance evaluation, we offer off-the-shelf judgment models for automatically assessing the LF output of L.MMs given benchmark ques- tions. Moreover, we compare potential solutions for LF generation evaluation and provide insights for building better automated metries. Code and models are available at https: //gi thub. com/ cwwccoom//////eeeeeee Meettoe-etoooooo Baaeroou-----ae Lat-LLLERRREEE- MI.
Introduction
Over the past few years, the community witnesses the rise of pretrained autoregressive large language models (LLMs) [Brown et al., 2020a, Lieber et al., 2021, Rae et al., 2021, Smith et al., 2022]. These mega models have shown transcendent power [Singhal al., 2022], achieving and even surpassing human performance in breadth of tasks [Clark et al., 2019, Rajpurkar et al., 2018, 2016, Wang et al., 2019]. and cover various possible application scenarios such as code generation [Bommasani et al., 2021], healthcare chatbots [Nuance, 2023] and keyword matching [OpenAI, 2022].
*Corresponding author.
37th Conference on Neural Information Processing Systems (NeurIPS 2023) Track on Datasets and Benchmarks.

---

As the size of LLMs continues to grow, the potential harms caused by sodel-geeerated text are of increasing concern [Bender al., 2021, Bommasani et al., 2021, Dinan et al., 2021, Kenton et al., 2021]. A comprehensive taxonomy of ethical risks associated with recent L.MMs is introduced in [Weidinger et al., 2022]. One of its most pressing concerns the risk of misinformation, stemming from the generation of erroneous, deceptive, irrational, or substandard information, defined as LLM outputting false, misleading, nonsensical or poor quality information, without malicious intent of the users.² Potential harms of misinformation range from deceiving a person, causing physical damage, and amplifying the society's 's distrust on the usage of -LM--inorpooraeed systems [Weidinger et al., 2022]. For example, Galactica [Taylor al., 2022]. an LLM trained using a wide range of scientific sources including papers, reference materials, knowledge bases, etc., was reported to generate a fake study about the benefits of eating crushed glass. The aforementioned instance highlights the limitations of current L.LMs in consistently delivering factually correct responses to user queries, a deficiency that potentially yield hazardous outcomes.
Despite the existence of several prior analyses, concerns are still far from being fully resolved. First of all, the evaluation of misinformation harm in existing LLMs has been conducted only using relatively simple formats such as multiple-choice (MC) question-answering (QA) task [Hendrycks et al., 20]1a] or cloze task [Petroni et al., 2019a]. These tasks are usually either formalized as a completion task, where LLMs are required to predict only a single token instead of generating full sentences, or using relatively simple evaluation metrics, e.g., accuracy, perplexity [Dinan et al., 2019], ROUGE score [Lin, 2004] and BLEU score [Khashabi et al., 202 for the case of evaluation [Hendrycks et al., 2021b]. In the context of long-form (LF)* generation, Zhang et al. [2023] found that an incorrectly generated token at the outset will often be followed by a subsequent erroneous explanation; more generally, LLMs are reported to have the tendency generate false statements, ranging from subtle inaccuracies to blatantly inauthentic claims [Lin et al., 2022]. These phenomena highlight the importance of conducting LF generation evaluations for LLMs. Yet, there are no sufficient datasets available for such evaluations, especially in snowledge-intensvee domains. Furthermore, due to the unbalanced distribution in available language-related resources [Zeng et al., 2023], substantial number of existing datasets and benchmarks only focus on measuring the misinformation in English, which impedes similar evaluations from being performed in other languages. A brief summary of previous datasets on misinformation evaluation is shown in Table I; even though some of them are not initially designed for misinformation evaluation, we found that they can be casily used for it.
To motivate the misinformation evaluation on LLMs, we propose Chinese benchmARK Rk for misinfor- mation Evaluation in Maternity and Infant care (CARE-MI), benchmark test medical- misinformation of LLMs in Chinese. CARE-MI specifically focuses on the sub-domain of maternity and infant care, a topic in which users are prone to generate questions, especially expectant and first-time parents, about multiple issues, e.g., pregnancy and/or baby-related illnesses, symptoms, milestones. etc., and in which LLMs should respond factually and without errors. Although there are already existing datasets focusing on the medical domain, until now, there is no benchmark suitable for evaluating misinformation on such an important and sensitive topic as the maternity and infant care, neither in English nor in Chinese The closest dataset to CARE-MI is MATINF [Xu et al., 2020a], which, differently from us, focuses on community QA and contains neither expert-level annotations nor supporting evidence documents, making it not suitable for misinformation evaluation. Our benchmark, however, is not designed for directly evaluating user-LLM interactions, as most of our questions require expert-level knowledge and contain medical norms, but it is a necessary prerequisite for LLMs to perform well in those cases.
Additionally, we conduct an extensive evaluation on recent Chinese LLMs The results indicate that all current models are not able to maintain an acceptable performance while answering Nomain-related questions. We further explore the paradigm for automatic evaluation on LF generation of LLMs
This undesired phenomenon of LLMS sometimes is called hallucination by some previous work [Ji et al., 2023a], a term that is initially mentioned in the context of psychology, defined as "percept, experienced by waking individual, in the absence of an appropriate stimulus from the extracorporeal world" by Blom [2010]. However, we stick to the description of misinformation throughout our paper for consistency.
hhttps:/ //news. yahoo. com/meta-aeeee aott-t------------------- contr bubeddffakeee -itee0212209911 .html
'LF generation is widely used concept [Fan et al., 2019], referred as generation of text that "span multiple sentences" or seragraph-lennthh We found no stringent standard that can be utilized to distinguish LF generation against shorter ones; however, we put nearly no limitation on models' generation, i.c., we let the models generate sentences will until maximum length is reached. See Appendix B.6 for more details.

---

Table 1: A brief summary of datasets in misinformation evaluation. LF: whether the dataset is designed for LF generation task; Supervised: whether the dataset construction involves human supervision. "We refer the selected samples that have been annotated by annotators. & We are only referring to the expert annotated part of PubMedQA. "For Chinese mentioned here, we are referring both traditional and simplified Chinese.
Dataset
Language
#Question
LF
Supervised
Multiple
COMMONSENSEQA [Talmor al., 2019]
English
12,247
X
Wizard of Wikipedia [Dinan et al., 2019]
English
201,999
V
LAMA [Petroni et al., 2019b]
English
X
X
NQ Ikwitkoowkii al., 2019]
English
323,045
V
ELI5 [Fan et al., 2019]
English
R=272,000
MMLU [Hendrycks al., 2021a]
English
15,908
X
COM2SENSE, [Singh et al., 2021]
English
=240000
X
GOOAQ [Khashabi et al., 2021h]
English
33,000,000
V
X
KMIR [Gao al., 2022]
English
16,000
X
TruthfulQA [Lin al., 2022]
English
817
ScienceQA [Lu et al., 2022]
English
21,208
X
M3KE [Liu ct al., 2023]
Chinese
20,477
X
Legal
JEC-QA [Zhong et al., 2019]
Chinese
26,365 53,137
X
CaseHOLD [Zheng al., 2021]
English
X
Medical
cMedQA2 [Zhang al., 2018]
Chinese
108,000
V
PubMedQA [Jin et al., 2019]
English
=1,000
X
MATINF [Xu et al., 2020a]
Chinese
~1,070,000
X
X
MEDQA [Jin al., 2020]
Chinesc/English?
61,097
X
CMelE [Guan et al., 2020]
Chinese
22,406
X
BioLAMA [Sung et al., 2021]
English
R549,000
X
MLEC-QA [Li et al., 2021a]
Chinese 136,236
X
CARE-MI (ours)
Chinese
1,612
as a completion to the benchmark dataset for the purpose of efficient and accurate evaluation. We test multiple judgment models trained the same set of questions in the benchmark, with 1) synthetically generated positive and negative answers, and 2) expert-level annotations on the models' outputs, and 3) expert-annotated knowledge. The whole pipeline, encompassing the development of the benchmark and the training of the judgment model, can also serve as an innovative paradigm for creating similar benchmarks in other Knowledge-intensiee domains or low-resourced languages.
CARE-MI data acquisition
With existing datasets the topic (maternity and infant care), language (Chinese), and task (misinformation evaluation in LF generation) of interest, we construct CARE-MI from multiple data sources. We utilize two knowledge graph (KG) datasets and MCQA datasets our data sources. The collected data is then filtered to align with our focused topic.
KG datasets We rely on two medical-based KG datasets in Chinese: Bio-Medical Informatics Ontology System (BIOS) [Yu et al., 2022] and CPubMed [CPu, 2021], both of which come with <head, relation, tail> triplets. BIOS a machine-generated bio-medical KG built on top of the abstracts and central articles from PubMed, which is a search engine for bio-medica articles. Both head and tail BIOS are concepts representing the nodes in the KG; each concept includes multiple terms that are considered synonymous. In total, it contains 4.1 million concepts, 7.4 million terms, and 7.3 million relations. As the data is gathered from PubMed, BIOS triplets are collected in English and later translated into Chinese [Luo al.. 2021]: translation quality is ensured by applying back-trand transsaannl and filtering out samples with low confidence. On the other hand, CPubMed an
Sht :ttttps://ppbddd..mm ncbi nlm. nih, gov/
3

---

Table QA pair examples in MCQA datasets. Q. stands for question and A. for answer. The correct answers are underlined. English translations are shown for reference.
Source
Example
M 2 h. 4 40 i HAJ $2 + T
1 T EL (H
1 # . 12
Q. A 2-month-oid male infant, who started experiencing vomiting 20 days after birth. The vomiting progressively worsened, sometimes with projectile vomiting, mostly occurring within half an hour after breastfeeding. The vomit often contains milk curds but not bile. After vomiting, the baby has an excellent appctite, but his weight does not increase. The considered diagnosis is
MLEC-QA
A. Æ HI up MAA TE
Congenital hypertrophic pyloric stenosis
B. デ B
B. Congenital intestinal malrotation
A.
C. k
C. Gastroesophageal reflux discase
D. 1
D. Duodenal ulcer
E. #
E. Intussusception
A 1 E (
Q.
The only immunoglobulia that can pass through the placenta and enter thefetus is
MEDQA
A.
A. IgM
B. IgG
Ig
D. IgE
open-source KG dataset constructed by the full-text Chinese periodical data of the Chinese Medical Association. contains more than 1.7 million entities and around 4.4 millions of triplets. Examples of BIOS and CPubMed triplets can be found in Appendix B.2.
MCQA datasets Bio-medical KGs inherently entail factual information about the medical domain, yet the information limited due to the intrinsic format of triadic representation. To collect more complex samples, we further include two Chinese med-cal MCQA datasets: MLEC-QA [Li al., 2021b and MEDQA [Jin et al., 2020]. Both datasets are collected from the National Medical Licensing Examination in China (NMLEC), which is a Chinese official exam to evaluate professional knowledge and skills for medical and clinical practitioners [Li et al., 2021b]. Samples from these datascts can be categorized into several bio- medical aub-domains. Specifically, MLEC-QA covers Clinic, Stomatology, Public Health, Traditional Chinese Medicine, and Chinese Western Medicine domains, while MEDQA only covers the sub-field of Clinic. For MLEC-QA, we select samples from Clinic, Public Health, and Stomatology and exclude the rest. Examples can be found in Table 2.
Topic filtering To ensure that the collected data is related to the domain of maternity and infant care, filter the samples according to their topics. We rely four existing word-lists determine the topic of each sample; the details can be found in Appendix B.3. We first translate the English word-lists into Chinese and then aggregate all the word-lists into single one. To ensure relevancy, we manually check each word and only keep those ones that are exclusively related to the topic. In total, the aggregated word-list contains 238 Chinese domain-related words after deduplication. We then use the aggregated word-list to perform the domain filtering. For KG datasets, we only keep triplets where both head and tail are included in the aggregated word-list. For MCQA datasets, retain only the samples that contain at least one domain-related word from the aggregated word-list, either in the question or in the candidate answers. We further carry out deduplication for the samples from MEDQA and MLEC-QA as they originate from the same source; here we only remove the duplicated samples from MLEC-QA and leave MEDQA untouched. In total, we obtain 79 samples from BIOS, 497 samples from CPubMed, 1,634 samples from MEDQA and 1,349 from MLEC-QA; these samples serve as the initial set for the subsequent benchmark construction.
Phttps://en cma. org.on// cn/

---

3 CARE-MI benchmark
We construct the CARE-MI benchmark on top of the samples acquired in Section 2. The benchmark based on two parts: synthetic data generation process and set of judgment models. With the syn- thetic data generation process, we create samples the desired format for misinformation evaluation on LLMs. Then, empowered by the judgment model (see Section 4.3), offer an automated metric for efficient and accurate LF generation evaluation, aiming to simplify human- based evaluation which not only expensive but time-conee consuminn Note that both the synthetic data generation process and the judgment model construction are -omain-agnostic and can be casily applied to other misinfor- mation evaluation topics. In total, CARE-MI benchmark contains 1,612 samples; it is intended for IF generation evaluation under zero-shot [Wei et al., 2022, Brown et al., 2020b] setting. Statistics regarding the question length can be found in Figure 1. More details be found in Appendix B.
KG
64.64
600
Avg. length
MOQA
$6.67
"Sample
400
200
15.95
CPubMed
MLEC-QA
MEDQA
50
50-75
10:100
125125
150150
Source
length
Figure 1:
Statisties of the questions in CARE-MI. Left: average question lengths for each source (average over all questions is shown in gray). Right: question length distribution.
The synthetic data generation process is summarized in Figure 2. It consists of five components: 1) true statement generation, 2) false statement generation, 3) question generation, 4) knowledge retrieval, and 5) expert annotation.
True statement generation We define true statements as sentences that are evidence-based and factually correct, for example, Placenta previa may potentially lead to preterm birth. We generate true statements from the samples collected in Section For samples from KG datasets, we build the true statements heuristically from the triplets with rule-based methods. For samples from MCQA datasets, we formalize the generation true statements as a QA2D task where the model is required to perform sentence transformations to combine the question and answer into a declarative sentence [Demszky et al., 2018). 8J. The generation is done by using the combination of the rule-based method and an off- the-shelf model for simple of tokens such as " (referred as is in English), we directly concatenate the question with the answer the synthetic 2023a]. Details regarding the implementation of the rule-based system can be found in our code repository; details about the prompts for statement generation in Appendix B.S.
such that the question from the QA pair ends with a pre-defined set
statement; otherwise, the generation done using the GPT-3.5-turbo [OpenAI,
False statement generation Similar to the definition of true statement, define false statements as sentences that are factually incorrect, for example, progesterone is not a hormone. We approach the construction in two different ways: negation and replacement, corresponding to two types of false statements. For negated false statements, the generation is done on all available generated true statements where we generate the false statements by directly performing negation corresponding true ones; this construction procedure is also done by combining a rule-bacc based method with applying the GPT-3.5-turke and the details can be found in our code repository. We only generate false statements with replacement for samples that originally come from MCQA datasets; we generate the false statements by replacing the correct answers in the generated true statements with randomly selected wrong answers from the corresponding MC options. Our prompts utilized for generating negated statements can be found in Appendix B.7.
the
Question generation We generate questions based on the true statements. We rely on LLMs as their cettucttion-ollloinng ability allows to generate questions efficiently instead of using resource- consuming tine-tuning methods. To select the best LLM for question generation, experimentally
5

---

generation
False statement generation
Sources
True statement generation model
True statements
False statement generation model
False statements
3. Question generation
Synthetic data
Questions
Question generation model
TreeFaase statements
Questions
Filtered knowledge
Knowledge retrieval
Expert annotation
Retrieval model
Knowledge base
Knowledge
Expert annotation
Filtered knowledge
Figure 2: CARE-MI construction process. Data generation components are shown on green back- ground, whereas the final benchmark samples are shown orange background.
compare three LLMs available in Chinese: GPT-3.5 1PPT-3.-tturbe ChatGLM-6B [Du et al., 2022a, Zeng et al., 2022] and Yuat [ClueAL, 2023]. For this experiment, we evaluate the model performance on the BIOS dataset (See Appendix B.4 for more details) and select ChatYuan as the final choice.
True statements
#=:: 88 12 Wer [8
18
principle adoleseent anovulation type working blood stop bleeding and adjust cycle.
2H: # #0 tt 22 iiu i7 Ml At di Al #6 10)a??
questions
EN: the treatment principle of adolescent anovulation type working blood stop blooding and adjust the eycle?
OE questions
ZH: #1 98 if
di hf MM N?
EN: What is the principle of treatment of addoessent anvvulation type working blood?
Figure 3: An example of generated questions in MLEC-QA datasets. ZH and EN stands for Chinese and English. English sentences are translated for reference.
We generate two types of questions: True/False (TF) questions and open-ended (OE) questions. TF questions only admit binary answers, either agreeing or disagrecing with the statement the question, while OE questions allow variety of response styles. More specifically, we use YuatYuan along with a rule-based method to generate both TF and OE questions for MCQA samples while we only generate TF questions for KG samples (See Appendix B.4 for details). Information about the generated questions is shown in Table 3. In total, we generate 2,240 and 2,963 questions for MLEC- QA and MEDQA datasets, and 79 and 497 questions for BIOS and CPubMed datasets, respectively. Figure shows an example of generated questions in the MLEC-QA dataset.
Table 3: Number of generated questions for CARE-MA.
Source
TF
OE
BIOS
79
CPubMed
MIEC-QA
1,333
907
MEDQA
1,617
1,346
Total
3,526
2,253
Knowledge retrieval Maternity and infant care is critical subject matter wherein the dissemination of misinformation could endanger lives. our benchmark, we include external knowledge each sample to provide auxiliary information, not only for models but also for humans, allowing suitable

---

inferences on the veracity of the statements to be made. To achieve this, we apply BM25 [Robertson and Zaragoza, 2009] to obtain relevant knowledge from given knowledge sources based on the queries. We use two sources for knowledge bases: the Chinese Wikipedia" and Medical books collected by Jin et al. [2020]. We conduct topic filtering for Chinese Wikipedia as it originally contains huge amount of pages that might not be relevant to the topic of maternity and infant care. During the knowledge retrieval, queries are the concatenation of the questions and the corresponding true statements. We conduct the retrieval paragraph level; to account for differences between the two knowledge sources, we retrieve the top three most relevant paragraphs from both sources, respectively.
Expert annotation We hire two medical-domaie experts annotators to review the generated samples and a third meta-annotator to arbitrate their disagreement. The annotators are instructed to answer series of guiding questions. The guideline including rules and procedures that annotators should follow as well as the inter-annotator agreements are detailed in Appendix I. We ask them to evaluate 5,779 synthetic samples, and we discard those that receive two or more negative evaluations. After the expert annotation, the benchmark is downsized to 1,612 samples.
4 Experiments
We evaluate Chinese LLMs with respect to misinformation in LF generation using our proposed benchmark. Experiments are done in the form of single-round conversation where we feed models with questions and collect the direct output from the corresponding model. We perform the evaluation under the true zero-shot [Lin et al., 2022] setting: for each model, the input is question from our benchmark, without any additional instruction and examples; prompt and hyperparameters are not tuned in any way. All questions in CARE-MI are used in the evaluation.
4.1 Experimental details
Models As our goal is to evaluate the LLM misinformation in LF generation, we focus on the evaluation of autoregressive language models that are trained for generation tasks, as opposed to masked language models like BERT [Devlin al., 2019] RoBERTa [Liu et al., 2019]. We evaluate Chinese LLMs that have been specifically tuned for conversation scenarios, either by supervised fine-tuning (SFT) reinforcement learning with human feedback (RLHF) as we aim at measuring the misinformation occurring when the models directly interact with humans. We include ChatGLM- 6B [Zeng et al., 2022, Du et al., 2022a]. MOSS-16B 16--SF [Sun and Qiu, 2023], two variants of the BELLE [Ji et al., 2023b,c] series, a.k.a BELLE-7B-O.. the GPT family, GPT-3.5-turbe [OpenAI, 2023a] and GPT-4 [OpenAI, 2023b]. We also include a L.LMM [Touvron et al., 2023] model which we perform further pretraining on Chinese corpus including CLUFECorpus2020 [Xu et al., 2020b), Chinese Scientific Literature Dataset [Li al., 2022], Chinese Wikipedia', RedPajama--aata [Computer, 2023], etc., and fine-tuning on translated ShareGPT [RyokoAI, 2023], Aplaca-GPT4 [Peng et al., 2023] and WizardLM [Xu al., 2023]. We denote this version as ALaMA-13B-1 More details regarding the parameter settings can be found in Appendix B.5. Finally, we recruit a domain expert act as human baseline; we randomly select 200 questions from the benchmark and let the expert answer them correspondingly; the selected samples are collected strictly following their original distribution in the benchmark regarding their sources. The expert is allowed to check any resource that is necessary and is suggested to finish each question within 2 minutes.
and BELLE-7B- 7B-MM, and two models from
Evaluation metrics Unlike Lin et al. [2022] who consider answer to be truthful if and only if avoids asserting a false statement and thus allows non-committal answers such as No comments follow the human evaluation framework similar to
and I don't 't know as legal truthful answers. what has been explored in [Lu et al., 2022]. For each model- generrtee answer, we recruit three medical-domai expert-level annotators to evaluate the following two aspects:
1. Correctness: given a question, whether the answer is factually correct and relevant.
2. Interpretability: given question, whether the answer contains detailed and concise explanation that demonstrates how the conclusion was reached.
"https://dunge vikimedia. org/zhwiki/
7

---

Table 4: Average conrectness scores for all models and human baseline. Deeper color indicates better performance. "We only randomly select 200 questions for human evaluation.
Model
All
BIOS
CPu
MIFEC-QA
MEDQA
MOSS-T6B-SFI (2023)
0.671 10.221
0,930
0.925 0,116
0.644
0.639 10.316
ChatGLM-6B (2022h)
0.610
0.928 10.116
0.748
0.579 10,346
0.599
BELLE-7B-2M (2023b)
0.647 +0211
0.843
0.928
0.63 +-.314
0.605
BELLE-7B-0.2M (2023b)
0.670
0.947
0.942 141
0.624
0.646
GPT-4 (2023b)
0,867 215
0,958 125
0,967 =000
0.85 20,253
0.858
GPT-3.5-turbo (2023a)
0.824-10:63
0.973 108
0,948 0. 1610
0. 0.7999..270
0.81540226
LLaMA-1388 (2023)
0.709 40.301
0.87 +0.235
0.922-#0.778
0.678 +0.311
0.689
Human Baseline
0.938 #02213
4.000
1.000
0.945
0,908 00222
Table 5: Average interpretability scores for all models. Deeper color indicates better performance.
Model
All
BIOS
CPubMed
MLEC-QA
MEDQA
MOSS-I6B-SPPI (2023)
0.746
0,920+0.115
0,883 00114
0.726
0.731 4-0222
CharGLM-6B (2022b)
0.730
0,929
0.779 -0.248
0,705 +0.263
0.734-40242
BELLE-7B-2M (2023b)
0.728 4a235
0.839 40.251
0.930
0.723 4-0.216
0.694
BELLE-7B-0.2M (2023b)
0.645
0.716
0.746
0.609
0.650
GPT-4 (2023b)
0.928
0.973
0.981
0,060
0.92 10.14
0.922::0:33
GPT-3.5-turbo (2023a)
17718
0.977:100073
0,960 =0:0 0.094
0. 0.864:10.001
0.880
LLaMA-13B-T (2023)
0.8 0.816400200
0.836+4226
0.935
0.797 +0.244
0.808
We require the annotators to make their judgments independently during the evaluation; for each aspect, they are asked to give a scalar score between 0 and reflect their decisions for each sample (the higher the better). The final evaluation results are averaged over the three annotators. We refer the readers Appendix A.2 for more details about this human evaluation.
4.2 Results
Overview
Evaluation results on correctness and interpretability are shown in Table 4 and Table 5, respectively. Among all evaluated Chinese models, models from the GPT family perform the best by a large margin in both correctness (2 0.158) and interpretability (2 02). In general, all models samples generated from KG datasets than from MCQA datasets, as
exhibit better performance they generally have longer context and necessitate more reasoning ability from the models to produce the correct answer. Overall, LLMs with smaller sizes tend to perform worse; however, even the best model is not comparable with human expert, indicating room for improvement. Figure 4 further presents the correctness and interpretability evaluation for TF and OE questions separately. We observe that all models perform better on TF questions than OE questions: on average, most models are able to achieve 0.8 of correctness as well as interpretability for TF questions while only GPT models can achieve correctness of 0.6 for OE questions. This highlights the weakness of current models, where L.MSs still struggle with complex reasoning.
Correctness is linear-correlated with interpretability Figure 5 shows the correlation between correctness and interpretability across all evaluated LLMs.. general, models exhibit a similar pattern, where the interpretability linear correlates with the corresponding correctness. Exceptions include BELLE-6B-0.2M which shows abnormally low interpretability. On the contrary, ChatGLM- 6B and I.LMA- 13B-T present above-average interpretability; however, this is not always good. Better interpretability scores only indicate better generated descriptions when explaining how the conclusions are drawn (See Section 4.1); yet if the conclusions themselves are factually incorrect, more detailed explanations will only lead to misleading consequences.
More data is not always better HELLE-7B-0.2M. present slightly better performance (10.023) but lower interpretability (40.083) in comparison with its twin model BELLE-7B-2M. Both BELLE-7B- 0.2M and BELLE-7B-2M use BLOOM [Scao et al., 2022] as base model with the only difference being the size of the instruction set used during fine-tuning. As mentioned, better interpretability not always desired property, especially when it is paired with much lower correctness: more instruction-
8

---

1.00
TF
OE
Correctr
0.50
0.25
0.00
ChatGLM-6B
GPT4
GPT-3.5-uubo
1.00
TF
OE
Integgrdabiinn
0.75
HHH
0.50
0.25
MOSS S6T
ChaiGLM-68 BELLE-7B-2M BELLE-7B-02M
GPT-4
GPT-3.5-urbe
LLaMA-13B-T
Figure 4: Evaluation results for TF and OE questions.
tuning yields better language ability, yet it has no effect on improving knowledge correctness. This indicates that increasing the size of instructions used for fine-tuning LLMs is not always helpful in improving its knowledge capability. The conclusion is intuitive: the objective for fine-tuning does not align with improving truthfulness. Consequently, increasing the size of instruction-followii samples will not assist the model in providing more truthful answers.
4.3 Automated metries
Human evaluation, especially in knowledge- intensive domains, is costly and difficult to re- produce. On the other hand, traditional auto- mated evaluation metrics, such as perplexity [Di- nan et al., 2019], BLEU score [Khashabi et al.,
CharGLM-6B
G
LLaMA-13B-T
GPT-3.5-turo
BBLLE-7B-2M
2021a] and ROUGE score [Lin, 2004], suffer from misalignment problem in which the met- rics fail to capture the actual performance of the models. Lin et al. [2022] propose to train judgment model using human labels to serve as proxy of human annotators. However, the trained models cannot transfer across different tasks and languages. To enable efficient and ac- curate automated evaluation for our proposed benchmark, explore using different architectures as backbones of the judgment models and fine-tune them mimic human judgment.
MOSS-16B-SFT
0.7
BELLE-78-002M
0.50
0.65
0.70
0.75
0.80
0.85
Conrectnces
Figure 5: Correctness and interpretability metrics show linear relationship with the Ra being 0.834.
Table 6: Pearson correlation scores (Pear.) and accuracy (Acc.) are reported for each trained judgment model. We highlight the best Pearson correlation scores in bold.
Random
BERT- Large
GPT-3-350M
GPT-3-6.7B
LLaMA-13B-T
Metric
Pear.
Acc.
Pear:
Acc.
Pear.
Acc.
Pear.
Acc.
Pear.
Acc.
Correctness
0.560
0.606
0.560
0.783
0.835
0.803
0.858
0.868
0.898
Interpretability
0.800
0.013
0.794
0.565
0,822
0.634
0.828
0.683
0.835
We use 1) both generated true and false statements as synthetic positive and negative answers, and 2) the human evaluation results, train all judgment models. We also feed models with all available retrieved knowledge. To evaluate the performance of each judgment model, we select the answer outputs from MOSS-16B-S these two models received are closer to 0.5 on average, indicating a more balanced distribution of good and bad answers. We perform the training using the answers from the rest of the evaluated models. In order to compare different architectures, we fine-tune BERT-Large [Devlin et al., 2019], two GPT-3 variants (GPT-3-350M and GPT-3-6.7B) and LLaMA-13B-T (See Section 4.1 for details). For training BERT-Large, we the [SEP] token the separator between fields and directly
and ChatGLM-6R 6B the validation set. This is because the scores

---

concatenate all fields together (e.g., question, answer and retrieved knowledge) as the input. Prompt used for fine-tuning LLMs can be found in Appendix B.9. For BERT-Large and the two GPT variants evaluated, as they suffer from input length limitation, we truncate the retrieved knowledge to make sure the input meets the model requirements. For comparison, we also include a majority baseline, which always predicts the most frequent score in the training set. More details regarding the experiment settings can be found in Appendix B.5. We evaluate the performance of the judgment model using Pearson correlation and accuracy. To obtain accuracy, we cast the scalar output of the judgment models into binary labels by empirically setting the threshold to 0.5. Judgment models' results are shown in Table 6.
We observe that BERT- Large does not learn any- thing useful. This might be due to the strin- gent constraints on the input size as well as the model's 's inherent inability in complicated reason- ing tasks. Larger models tend to perform better; however, it is harder for models to understand the correlation between input text and the labels in the aspect of interpretability. For the final judgment model, we select L.LMM--1B--T as the backbone, it obtains the best results. We train the model again but with all available data, including both training and validation samples. We make the judgment models for both aspects publicty available.
Table 7: Evaluations are done on the validation set. w/o K. and w/ K. represents fine-tuning the models without and with knowledge. We highlight the best Pearson correlation scores in bold.
w/o K.
w/ K.
Metric
Pear.
Acc.
Pear.
Acc.
Correctness Interpretability
0.779
0.806
0.868 0.683
0.898 0.835
0.639
0.867
We further conduct an ablation study to assess whether incorporating retrieved knowledge enhances the performance of judgment models. Specifically, we perform a comparison by training the LLaMA- 13B-T both with and without the retrieved knowledge during the fine-tuning while maintaining the rest of the settings identical. The experiment results are shown in Table 7, which demonstrate that adding knowledge improves the performance of the judgment models.
5 Conclusion and Limitation
In this paper, we proposed CARE-MI, a Chinese benchmark for LLM misinformation evaluation in LF generation in the topic of maternity and infant care. This is the first Chinese benchmark that aims to quantify the misinformation on the maternity and infant care domain, and with a construction pipeline that can be easily transferred to other domains and languages. We conducted comprehensive assessments on Chinese LLMs, and showed that current models still have room for improvement. Furthermore, we investigated different model backbones for training judgment models and provided an efficient judgment model that can perform correctness evaluation of LF generation accurately. We believe that our proposed benchmark not only paves the way for easier benchmark construction for the whole research community, but also contributes to promoting better Chinese LLM applications the maternity and infant care domain.
LF generation cases, the only dataset with clear expert- -npert-annotations
Limitation CARE-MI aims solely at evaluating the misinformation in long-form generation tasks for Chinese LLMS on the topic of maternity and infant care. It is not designed for any other scenarios, e.g., evaluating other target groups such as medical professionals or students. The misuse of CARE- MI might lead unpredictable consequences. Additionally, the benchmark contains content that can only be considered as correct for now. With the development of modern clinical techniques, we expect the accuracy of the information provided the benchmark to decrease over time, and thus our proposed benchmark might not be suitable for misinformation evaluation at later stage, e.g., 10 years from now. Furthermore, our benchmark might not align with the actual interests of the maternity and infant care community (e.g., pregnant women) in real-life situations, as the benchmark not constructed by collecting the most frequently asked questions in the community. Last but not least, even though we have tried our best to reduce the subjective bias during the human annotations, we cannot completely avoid it.
Acknowledgement
This work was partly supported by JSPS KAKENHI No. 1222K12091.
10

---

References
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss,, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey Wu, Clemens Winter, Chris Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Ben- jamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, llya Sutskever, and Dario Amodei. Language Models are Few-Shot Learners. In Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin, editors, Advances in Neural Information Processing Systems, volume 33, pages 1877-1901. Curran Associates, Inc., 2020a. URL https: /ttppe///rooeeedniggg neurips. cc/ Paper pdf.
paper_ filesrfiles//eeeeeeeeeeee
Opher Lieber, Or Sharir, Barak Lenz, and Yoav Shoham. Jurassic-1: Technical details and evaluation. White Paper. AI21 Labs, 1, 2021.
Jack W. Rae, Sebastian Borgeaud, Trevor Cai, Katie Millican, Jordan Hoffmaann, H. Francis Song, John Aslanides, Sarah Henderson, Roman Ring, Susannah Young, Eliza Rutherford, Tom Hennigan, Jacob Menick, Albin Cassirer, Richard Powell, George van den Driessche, Lisa Anne Hendricks, Maribeth Rauh, Po-Sen Huang, Amelia Glaese, Johannes Welbl, Sumanth Dathathri, Saffron Huang. Jonathan Uesato, John Mellor. Irina Higgins, Antonia Creswell, Nat McAleese, Amy Wu, Erich Elsen, Siddhant M. Jayakumar, Elena Buchatskaya, David Budden, Esme Sutherland, Karen Simonyan, Michela Paganini, Laurent Sifre, Lena Martens, Xiang Lorraine Li, Adhiguna Kuncoro, Aida Nematzadeh, Elena Gribovskaya, Domenic Donato, Angeliki Lazaridou, Arthur Mensch, Jean-Baptiste Lespiau, Maria Tsimpoukelli, Nikolai Grigorev, Doug Fritz, Thibault Sottiaux, Mantas Pajarskas, Toby Pohlen, Zhitao Gong, Daniel Toyama, Cyprien de Masson d' Autume, Yujia Li, Tayfun Terzi, Vladimir Mikulik, Igor Babuschkin, Aidan Clark, Diego de Las Casas, Aurelia Guy, Chris Jones, James Bradbury, Matthew J. Johnson, Blake A. Hechtman, Laura Weidinger, lason Gabriel, William Isaac, Edward Lockhart, Simon Osindero, Laura Rimell, Chris Dyer, Oriol Vinyals, Kareem Ayoub, Jeff Stanway, Lorrayne Bennett, Demis Hassabis, Koray Kavukcuoglu, and Geoffrey Irving. Scaling Language Models: Methods, Analysis & Insights from Training Gopher. CoRR, abs/2 1446, 2021. URL
www.i.oog//..........ooooo org/ arrg/abs/2a /tt/1/1b/211.... 11446.
Shaden Smith, Mostofa Patwary, Brandon Norick, Patrick LeGresley, Samyam Rajbhandari, Jared Casper, Zhun Liu, Shrimai Prabhumoye, George Zerveas, Vijay Korthikanti, Elton Zheng, Rewon Child, Reza Yazdani Aminabadi, Julie Bernauer, Xia Song, Mohammad Shoeybi, Yuxiong He, Michael Houston, Saurabh Tiwary, and Bryan Catanzaro. Using DeepSpeed and Megatron to Train Megatron- Turing NLG 530B, A Large-Scale Scale Generative Language Model. CoRR, abs/2201.11990. arxiv. org/ abs abbs/200 2201. 11990.
2022. URL
Karan Singhal, Shekoofch Azizi, Tao Tu, S. Sara Mahdavi, Jason Wei, Hyung Won Chung, Nathan Scales, Ajay Kumar Tanwani, Heather Cole-Lewis, Stephen Pfohl, Perry Payne. Martin Seneviratne, Paul Gamble, Chris Kelly, Nathaneal Schärli, Aakanksha Chowdhery, Philip Andrew Mansfield, Blaise Agüera y Arcas, Dale R. Webster, Gregory Corrado, Yossi Matias, Katherine Chou, Juraj Gottweis, Nenad Tomasev, Yun Liu, Alvin Rajkomar, Joelle K. Barral, Christopher Semturs, Alan Karthikesalingam. and Vivek Natarajan. Large Language Models Encode Clinical Knowledge. CoRR, abs/2212. 13138, 2022. doi: 00.48550/rxii..212..555 48550/ arxiv. 2212. 13138.
URL https://doos org/10. 10.
Christopher Clark, Kenton Lee, Ming-Wo Wei Chang, Tom Kwiatkowski. Michael Collins, and Kristina Toutanova. BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questions. In Pro- ceedings of the 2019 Conference of the North American Chapter of the Association for Compu- tational Linguistics: Human Language Technologies, Volume (Long and Short Papers), pages 2924 2936, Minneapolis, Minnesota, 2019. Association for Computational Linguisties. doi: 10.18653/v 1/N13/111N11110000 URL htttps:/aalaantholll
Ntt/NN1911000
Pranav Rajpurkar, Robin Jia, and Percy Liang. Know What You Don't Know: Unanswer- able Questions for SQuAD. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 784-789, Melbourne, Aus- tralia, 2018. Association for Computational Linguistics. doi: 10. 10.11///111118-2224 URL https:/ / htt///acaannhhooooo otg/P18-2124

---

Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. SQuAD: 100,000+ Questions for Machine Comprehension of Text. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 2383-2392 Austin, Texas, 2016. Association for org/
Computational Linguistics. doi: 10. (0.18653/v1//1666 1264. URL
D16-1264.
Alex Wang, Yada Pruksachatkun, Nikita Nangia, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel R. Bowman. SuperGI.UF: A Stickier Benchmark for General- Purpose Language Understanding Systems. In Hanna M. Wallach, Hugo Larochelle, Alina Beygelzimer, Florence d'Alché-Buc, Emily B. Fox, and Roman Garnett, editors, Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, NeurIPS 2019, December 8-14, 2019, Vancouver, BC, Canada, pages 3261-3275, 2019. URL https://ppp http:///poeeeees.eeeeeece www..nuiis...ppapre///...... wwtpcccppe//////aeeeeeeeee h0th//ah/////////////// #496bf24afe7fab6rr 400.aa20ee000444222ddd22eee----e
Rishi Bommasani, Drew A. Hudson, Ehsan Adeli, Russ B. Altman, Simran Arora, Sydney von Arx, Michael S. Bernstein, Jeannette Bohg, Antoine Bosselut, Emma Brunskill, Erik Brynjolfsson, Shyamal Buch, Dallas Card, Rodrigo Castellon, Niladri S. Chatterji, Annie S. Chen, Kathleen Creel, Jared Quincy Davis, Dorotyaa Demszky, Chris Donahue, Moussa Doumbouya, Esin Durmus, Stefano Ermon, John Etchemendy, Kawin Ethayarajh, Li Fei-Fei, Chelsea Finn, Trevor Gale, Lauren Gillespie, Karan Goel, Noah D. Goodman, Shelby Grossman, Neel Guha Tatsunori Hashimoto, Peter Henderson, John Hewitt, Daniel E. Ho. Jenny Hong, Kyle Hsu, Jing Huang, Thomas Icard, Saahil Jain, Dan Jurafsky, Pratyusha Kalturi, Siddharth Karamcheti, Geoff Keeling, Fereshte Khani, Omar Khattab, Pang Wei Koh, Mark S. Krass, Ranjay Krishna, Rohith Kuditipudi, and et al. On the Opportunities and Risks of Foundation Models. CoRR, abs/2108.07255. 2021. URL https: //tx//aavvvvvva . org/ abs /2080 2128807258.
Nuance, Automatically document care with the Dragon Ambient eXperience, 2023. URL https: www.//haaahaareeeeeeeeoaeaa wwwatrhhaeeaaaeeeeeea hamp/mmmmnteaeeeeeeeeeeeeeeea -tteeaaainaal-nnellleeeeeeeee htmml. html.
wwwww.muanee wwwemncc..oo//aaee cww/heal
OpenAL. New and improved embedding model, 2022. URL https://openai.com
new new-and- and- and-improved- embedding model.
Emily M. Bender, Timnit Gebru, Angelina McMillan-Major, and Shmargaret Shmitchell. On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? In Madeleine Clare Elish, William Isaac, and Richard S. Zemel, editors, FAceT '21: 2021 ACM Conference on Fairness, Accountability, and Transparency, Virtual Event / Toronto, Canada, March 3-10, 2021, pages 610-623, ACM, 2021. doi: 10. 10.114573422188.3445922 URL attps://doo org/ /01 1145/ 0142188.344599
Emily Dinan, Gavin Abercrombie, A. Stevie Bergman, Shannon L. Spruit, Dirk Hovy, Y-Lan Boureau, and Verena Rieser. Anticipating Safety Issues in E2E Conversational Al: Framework and Tooling. CoRR, abs/2107.03451,, 2021. URL https: /tttpps///xxii...// org/ /aa/s//210... 03451.
Zachary Kenton, Tom Everitt, Laura Weidinger, lason Gabriel, Vladimir Mikulik, and Geoffrey Irving. Alignment of Language Agents. CoRR, abs/2103.14665, 2021. URL https://aree htrgaass21033 14659.
Laura Weidinger, Jonathan Uesato, Maribeth Rauh, Conor Griffin, Po-Sen Huang, John Mellor, Amelia Glaese, Myra Cheng, Borja Balle, Atoosa Kasirzadeh, Courtney Biles, Sasha Brown, Zac Kenton, Will Hawkins, Tom Stepleton, Abeba Birhane, Lisa Anne Hendricks, Laura Rimell, William Isaac, Julia Haas, Sean Legassick, Geoffrey Irving, and Iason Gabriel. Taxonomy of Risks posed by Language Models. In FAccT '2: 2022 ACM Conference
Fairness, Accountability, and Transparency, Seoul, Republic of Korea, June 21 24, 2022, pages 214-229. ACM, 2022. doi: 00.1445//5311463333000 URL https: //t::://dooiio org/ 10. 1145/3531146.... 3533088.
Ziwei Ji, Nayeon Lee, Rita Frieske, Tiezheng Yu. Dan Su. Yan Xu, Etsuko Ishii, Yejin Bang, Andrea Madotto, and Pascale Fung. Survey of Hallucination in Natural Language Generation. ACM Comput. Surv. ((11)):248:12248:3. 2023a. doi: 10.1145/3571730. URL https://doi. org/ 10. 1145/3571730.
Jan Dirk Blom. A dictionary of hallucinations. Springer, 2010.
12

---

Ross Taylor, Marcin Kardas, Guillem Cucurull, Thomas Scialom, Anthony Hartshorn, Elvis Saravia, Andrew Poulton, Viktor Kerkez, and Robert Stojnic. Galactica: A Large Language Model for Science. CoRR, abs/2211 .00085, 2022. doi: 10.48350/arXvv2211 112222111.90885 1.09085. URL. https:
//doi.org/10 /10.1/.855/8880 arviv. 09085.
Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. Measuring Massive Multitask Language Understanding. In International Conference on Learning Reprexentations, a. URL beattps://ppenreeeee wwtp://.eeeeeeeeeeeeeeeeeeeeee www/ffoummmee jm
Fabio Petroni, Tim Rocktäschel, Sebastian Riedel, Patrick Lewis, Anton Bakhtin, Yuxiang Wu, and Alexander Miller. Language Models as Knowledge Bases? In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 2463-2473 Hong Kong, China, 2019a. Association for Computational Linguisties. doi: 10.1 N0.18653/v1/D199 1250. URL https: //acc acl aclanthology wwg//ogg 1250..
Emily Dinan, Stephen Roller, Kurt Shuster, Angela Fan, Michael Auli, and Jason Weston. Wizard of Wikipedia: Knowledge-Powee Powered Conversational Agents, In 7th International Conference on Learning Representuations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019. OpenReview.net, 2019. URL
Chin-Yew Lin. ROUGE: A Package for Automatic Evaluation of Summaries. In Text Summarization Branches Out, pages 74 81, Barcelona, Spain, 2004. Association for Computational Linguistics. URL https://acl http///caalahhoooggygooooo wraelloooyooggo//.. org/w04- 1013.
Daniel Khashabi, Amos Ng. Tushar Khot, Ashish Sabharwal, Hannanch Hajishirzi, and Chris Kallison-Burchh GooAQ: Open Question Answering with Diverse Answer Types. In Findings of the Association for Computational Linguistics: EMNLP 2021, pages 421-433, Punta Cana, Dominican Republic, 2021a. Association for Computational Linguistics. doi: 10.18653/v 1/2021. findings-emnlp.36 URL htttps:/accaantholloo
owgooog//////// findings- wwwriinnnsemnll... emnlp.38. 38.
Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Ja- cob Steinhardt. Measuring Massive Multitask Language Understanding. In 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021. OpenReview.net, 2021b. URL httptt/t///ppereeiiee httprriiwwwwneefrreeeeeeeeee wwt//neeeeeidddoeooeo Jm
Angela Fan, Yacine Jernite, Ethan Perez, David Grangier, Jason Weston, and Michael Auli. ELI5: Long Form Question Answering. CoRR, abs/1907 0099900,..99. 2019. URL. httt::///a//ate org/ abs/ 1907.09190
Muru Zhang, Ofir Press, William Merrill, Alisa Liu, and Noah A Smith. How Language Model Hallucinations Can Snowball. arXiv preprint arXiv:2305. 135344 2023.
Stephanie Lin, Jacob Hilton, and Owain Evans. TruthfulQA: Measuring How Models Mimic Human Falsehoods. In Proceedings of the 60th Anmual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). pages 3214 3214-3252, Dublin, Ireland, 2022. Association for Computational Linguistics. doi: 10. won.://////22....llooog.... org/2022 wol.-ong..... 229.
URL https: / aaclaaaaaannooyyyyoo
Qingcheng Zeng, Lucas Garay, Peilin Zhou, Dading Chong, Yining Hua, Jiageng Wu, Yikang Pan, Han Zhou, Rob Voigt, and Jie Yang. GreenPLM: Cross-Lingual Transfer of Monolingual Pre- the
Trained Language Models at Almost No Cost. Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence, IJCAI 2023, 19th-25th August 2023, Macao, SAR, China, pages 6290-6000 6298. ijcai.org, 2023. doi: 10. 2229963/jcai22223//999 URL https://doi. org/10 10. 24963/ i cai. 2023/698.
Alon Talmor, Jonathan Herzig, Nicholas Lourie, and Jonathan Berant. CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledge. In Jill Burstein, Christy Doran, and Thamar Solorio, editors, Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2019, Minneapolis, MN, USA, June 2-7, 2019, Volume 1 (Long and Short Pupers), pages 4149- 4158. Association for Computational Linguistics, 2019. doi: 10. 10.118633/v1/1ll 1421. URL https:/ //doii. org/10 10. 18653/v1/n19/ 1421.
13

---

Fabio Petroni, Tim Rocktäschel, Sebastian Riedel, Patrick S. H. Lewis, Anton Bakhtin, Yuxiang Wu, and Alexander H. Miller. Language Models as Knowledge Bases? In Kentaro Inui, Jing Jiang, Vincent Ng. and Xiaojun Wan, editors, Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, EMNLP-IJCNLP 2019, Hong Kong, China, November 3-7, 2019, pages 2463-2473. Association for Computational Linguistics, 2019b. doi: 10.18653/ URL bttps://don org/10. 10.
Tom Kwiatkowski, Jennimaria Palomaki, Olivia Redfield, Michael Collins, Ankur Parikh, Chris Alberti, Danielle Epstein, Illia Polosukhin, Jacob Devlin, Kenton Lee, Kristina Toutanova, Llion Jones, Matthew Kelcey, Ming-Wee Wei Chang, Andrew M. Dai, Jakob Uszkoreit, Quoc Le, and Slav Petrov. Natural Questions: A Benchmark for Question Answering Research. Transactions of the Association for Computational Linguistics, 7:452-466, 2019. doi: 10.1162/tacl_a La_taaace___0266666_ URL org/ (ttg/QQ9-110 1026.
Shikhar Singh, Nuan Wen, Yu Hou, Pegah Alipoormolabashi, Te-Lin Wu, Xuezhe Ma, and Nanyun Peng. COM2SENSE: A Commonsense Reasoning Benchmark with Complementary Sentences. CoRR, 0bbs/2106.0996) 2021. URL https: //axivv....//i/////a///a orgo aaaaab//211000 00969.
Daniel Khashabi, Amos Ng, Tushar Khot, Ashish Sabharwal, Hannaneh Hajishirzi, and Chris Callison- Burch. GooAQ: Open Question Answering with Diverse Answer Types. CoRR, Abs/2104.08827,
2021b. URL https: / aww.i...o..................ea..e www.//g////////////////.//.../et at////bs/2// /204.. 08727.
Daniel Gao, Yantao Jia, Lei Li, Chengzhen Fu, Zhicheng Dou, Hao Jiang, Xinyu Zhang, Lei Chen, and Zhao Cao. KMIR: A Benchmark for Evaluating Knowledge Memorization, Identification URL https:
and Reasoning Abilities of Language Models. CoRR, abs/2202. 13522,, 2022.
//arxiv. org/ abs/2202. 13529.
Pan Lu, Swaroop Mishra, Tanglin Xia, Liang Qiu, Kai-Wei Chang, Song-Chun Zhu, Oyvind Tafjord, Peter Clark, and Ashwin Kalyan. Learn to Explain: Mul- timodal Reasoning via Thought Chains for Science Question Answering. In NeurIPS, 2022. URL http://papers http:/pppprs..ss...ooo ctaappscc/pppppa papppprrrllee___peeeepparaeeeeee wwlles/paper/22// www.aaaaaaaaaaaaata-------
html.
Chuang Liu, Renren Jin, Yuqi Ren, Linhao Yu, Tianyu Dong, Xiaohan Peng. Shuting Zhang, Jianxiang Peng, Peiyi Zhang, Qingqing Lyu, Xiaowen Su, Qun Liu, and Deyi Xiong. M3KE: A Massive Multi-Level Multi-Subject Knowledge Evaluation Benchmark for Chinese Large Language Models. CORR, abs/2305. 10263, 2023. doi: 10445500axXi..2355.0222 URL https: //doi.org/10 10. 48550/ 48550/arXiv.....////aa 2305 10263.
Haoxi Zhong, Chaojun Xiao, Cunchao Tu, Tianyang Zhang, Zhiyuan Liu, and Maosong Sun. JEC- QA: A Legal-Domain Question Answering Dataset. CoRR, abs/1911.12011, 2019. URLhttp // arxiv. org/ htbb//9111. 12011.
Lucia Zheng, Neel Guha, Brandon R. Anderson, Peter Henderson, and Daniel E. Ho. When does pretraining help?: assessing self-supervised learning for law and the CaseHOLD dataset of 53, 000+ legal holdings. In Juliano Maranhão and Adam Zachary Wyner, editors, ICAIL '21: Eighteenth International Conference for Artificial Intelligence and Law, São Paulo Brazil, June 21- 25, 2021, pages 159-168. ACM, 2021. doi: (0.114553462757.3466606 URL https: //doi.org// 10. 114111444444 /36777467777777 3466088.
Sheng Zhang, Xin Zhang, Hui Wang, Lixiang Guo, and Shanshan Liu. Multi-Scale Attentive Interaction Networks for Chinese Medical Question Answer Selection. IEEE Access, 6:74061- 74071, 2018. doi: 10. 10.1109//CESS 20122883377. URL htlhttss://doo http:////oooooo//.../ 2018. 2883637.
iiao Jin, Bhuwan Dhingra, Zhengping Liu, William Cohen, and Xinghua Lu. PubMedQA: A Dataset for Biomedical Research Question Answering. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference Hong Kong, China, URL https:
on Natural Language Processing (EMNLP-IJCNLP) pages 2019. Association for Computational Linguistics. doi: N..18653/v11DD1/ //aclanthhole org D2s//D999.2.
14

---

Canwen Xu, Jiaxin Pei. Hongtao WU, Yiyu Liu, and Chenliang Li. MATINF: A Jointly Labeled Large-Scale Dataset for Classification, Question Answering and Summarization. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguisties, pages 5586-3596, Online, 2020a. Association for Computational Linguistics. doi: 10.1 10.18665/111/020..cc.mmmmm URL https: htt///ccaannologoooo wtg/ acl-main. 330.
Di Jin, Eileen Pan, Nassim Oufattole, Wei-Hung Weng, Hanyi Fang, and Peter Szolovits. What Disease does this Patient Have? A Large-scale Open Domain Question Answering Dataset from Medical Exams. CoRR, 0tss22009.3088, 2020. URL https://arxivv htta//aiiv..orabaa/a//ra/aeaaeee wrggabs///aae 13081.
Tongfeng Guan, Hongying Zan, Xiabing Zhou, Hongfei Xu, and Kunli Zhang. CMelE: Construction and Evaluation of Chinese Medical Information Extraction Dataset. In Xiaodan Zhu, Min Zhang, Yu Hong, and Ruifang He, editors, Natural Language Processing and Chinese Computing- 9th CCF International Conference, NLPCC 2020, Zhengzhou, China, October 14- 18, 2020, Proceedings, Part I, volume 12430 of Lecture Notes in Computer Science, pages 270 282. Springer, 2020. doi: 001777883-330-005555
22. URL attps://doi org/ 1007 /978/ 330-60000---9
22.
Mujeen Sung. Jinhyuk Lee, Sean Yi, Minji Jeon, Sungdong Kim, and Jaewoo Kang. Can Language Models be Biomedical Knowledge Bases? In Proceedings of the 202/ Conference on Empirical Methods in Natural Language Processing, pages 4734,, Online and Punta Cana, Dominican Republic, 2021. Association for Computational Linguistics. doi: 388. URL //acl:::///aleee htttps://aclanttoloog/ org 2021. emn emmlp-main.380
1/253//1/022.emm////
Jing Li, Shangping Zhong. and Kaizhi Chen. MLEC-QA: A Chinese Multi-Choice Biomedical Question Answering Dataset. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 8862-8874, Online and Punta Cana, Dominican Republic, URL
2021a. Association for Computational Linguistics. doi: 10. http:////222...mlmmmmmma. https: hathoooocyanntoggyyyyoooo org 202g emnlp-main...........mmmmmmmm 698.
Sheng Yu, Zheng Yuan, Jun Xia, Shengxuan Luo, Huaiyuan Ying, Sihang Zeng, Jingyi Ren, Hongyi Yuan, Zhengyun Zhao, Yucong Lin, Keming Lu, Jing Wang, Yutao Xie, and Heung- Yeung Shum. BIOS: An Algorithmically Generated Biomedical Knowledge Graph. CoRR, abs/2203.09977 2022. 09975..
doi: 10.448550ar2iv.2223..00
URL https: //te::////o/.// 100/10.0/// wtw.g//.....///aae// 488550/arxivv
CPubMed-kG, 2021. URL https: //ttuppe///pppmmdd openi . org. cn/ graph/wiki.
Shengxuan Luo, Huaiyuan Ying. Jiao Li, and Sheng Yu. Sentence Alignment with Parallel Documents Facilitates Biomedical Machine Translation. arXiv preprint arXiv:2104.08588 2021.
Jing Li, Shangping Zhong, and Kaizhi Chen. MLEC-QA: A Chinese Multi-Choice Biomedical Question Answering Dataset. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 8862-8874 Online and Punta Cana, Dominican Republic, 2021b. Association for Computational Linguistics. doi: 10.18653/v wtt.:022..mmmmmmmaaaa. https: / //laaaaaannnoooooo//ooo
URL
2221/2211.22 mmnlp-main... 698.
Jason Wei, Maarten Bosma, Vincent Y. Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M. Dai, and Quoe Le. Finetuned Language Models are Zero-Shot Learners. In The Tenth International Conference on Learning Representattions, ICLR 2022, Virtual Event, April 25-29, 2022. OpenReview.net, 2022. URL https: http///pppreveieeeeee/ee/ee net fow.mniddgg//ttooolae
Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhari- wal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agar- wal, Ariel Herbert-Voss Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCan- dlish, Alec Radford, Ilya Sutskever, and Dario Amodei. Language Models are Few-Shot Learners. In Hugo Larochelle, Marc' Aurelio Ranzato, Raia Hadsell, Maria-Florina Balcan, and Hsuan-Tien Lin, editors, Advances in Neural Information Processing Systems 33: An- nual Conference
Neural Information Processing Systems 2020, NeurIPS 2020, December 6-12, 2020, viruual 2020b. URL. https: /ttpps:/prooceedingg 1457 cd49674 www.o44667188bbaaacccaec. -aw.rbBBacc2226daaaaattrcc Abstract html.. AA477004bb
wwwesnnerspp... wwwrips.cc/papeeeeeee wwtppppe//02////////ccc/ /hash/
15

---

Dorottya Demszky, Kelvin Guu, and Percy Liang. Transforming Question Answering Datasets Into Natural Language Inference Datasets. CoRR, abs/1809.02922, 2018. URL http://arxxii..o org/ abs/1809. 0Ds/1899.00922. 02922.
OpenAl. Introducing ChatGPT and Whisper APIs, 2023a. URL http:://openai..omm
introducing- chatgggtthhh-tt--eee and- www-wwwissaapppppe
Zhengxiao Du, Yujie Qian, Xiao Liu, Ming Ding, Jiezhong Qiu, Zhilin Yang, and Jie Tang. GLM: General Language Model Pretraining with Autoregressive Blank Infilling. In Proceedings of the 60th Anmual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 320-335, Dublin, Ireland, 2022a. Association for Computational Linguistics. doi: 10. 2ttp53//11/0222aalooooogg
URL https: /tttlps:://ccie h/taaaattaaaloyyy//ao//a hwwpooooogyyogg/////e org/2022. acl- 1oo--oonn--.. 26.
Aohan Zeng, Xiao Liu, Zhengxiao Du, Zihan Wang, Hanyu Lai, Ming Ding, Zhuoyi Yang, Yifan Xu, Wendi Zheng, Xiao Xia, Weng Lam Tam, Zixuan Ma, Yufei Xue, Jidong Zhai, Wenguang Chen, Peng Zhang, Yuxiao Dong, and Jie Tang. GLM-130B: An Open Bilingual Pre-trained Model. CoRR, hts/2210.024144 2022. doi: 10448500/r2xi.:22100044 URL https://doot org/10/ 10. 48550/arbb arr/v 2210.2222222222 02414.
ClueAI. Yhat Large Language Model for Dialogue in Chinese and English, 2023. URL https://gitae: com/ clue- Aalue-ai/Can ai/ChatYuan/.
Stephen E. Robertson and Hugo Zaragoza. The Probabilistic Relevance Framework: BM25 and Beyond. Found. Trends Inf. Retr, 3(4):333-389, 2009. doi: 10. 10.1561/1500000019 URL https:
1561/1500000013
Jacob Devlin, Ming- Wei Chang, Kenton Lee, and Kristina Toutanova. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume I (Long and Short Papers), pages 4171-4186, Minneapolis, Minnesota, 2019. Association for Computational Linguistics. doi: 10.18653/v1/N19-111 URL sttps://ace acl anthology org/N19-- NN1-1199110 1423.
Yinhan Liu, Myle Oit, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. RoBERTa: A Robustly Optimized BERT Pretraining Approach. CoRR, abs/1907.1 1692, 2019. URL http: http://awaaawa.aaaa/////a/s/////
11692.
Tianxiang Sun and Xipeng Qiu. MOSS, 2023. URL.tttps::/githubbccom com http:tthhc...ceceeeeeaaaaaaa/a/a
Yunjie Ji, Yong Deng, Yan Gong, Yiping Peng. Qiang Niu, Baochang Ma, and Xiangang Li. BELLE: Be Everyone's Large Language model Engine, 2023b. URL https://gi com/ SianjiaTech/BELLE
Yunjic Ji, Yong Deng, Yan Gong, Yiping Peng, Qiang Niu, Lei Zhang, Baochang Ma, and Xiangang Li. Exploring the Impact of Instruction Data Scaling Large Language Models: An Empirical Study on Real- World Use Cases. arXiv preprint arXiv:2303. 2023c.
OpenAI. GPT-4 Technical Report. CoRR, abt.a303..087744 2023b. doi: 10.48550/arxi.22333..7777 URL https::/t/o:/ org/10 48550/arXiv 01.303.08774
Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, Aurélien Rodriguez, Armand Joulin, Edouard Grave, and Guillaume Lample. LLaMA: Open and Efficient Foundation Language Models. CoRR, abs/2302.13971, 2023. doi: 10.485500axXi..222..3377 URL https:///ol //doi. org/10. arviv. 2302. 13971.
Liang Xu, Xuanwei Zhang, and Qianqian Dong. CLUFECrpuu2000: A Large-scale Chinese Corpus for Pre-training Language Model. CoRR, abs/2003.01355, 2020b. https/////// org/ abs/2003. 01355.
Yudong Li, Yuqing Zhang, Zhe Zhao, Linlin Shen, Weijie Liu, Weiquan Mao, and Hui Zhang. CSL: Large-scale Chinese Scientific Literature Dataset. In Proceedings of the 29th International Conference Computational Linguistics, pages 3917-3923, Gyeongju, Republic of Korea, 2022. International Committee on Computational Linguistics. URL https://aclanthollee org/ 2022. cwwino-niii.o.. 1. 3.44
16

---

Together Computer. Redpajama: An open source recipe to reproduce Ilama training dataset, 2023.
URL https://github. http:///oogggeeeeecoptotoe///eea j ama- Data.
RyokoAL. ShareGPT52K, 2023.
URL https://huggingfaan co..atastttt////kkoolllll
ShareGPT52K.
Baolin Peng. Chunyuan Li, Pengcheng He, Michel Galley, and Jianfeng Gao. Instruction Tuning with GPT-4. CoRR, abs/2304.03211 0337777 2023. doi: 10.48500/aXi..200403777. URL https: //doi. org/10..0/.. 48550//aXivv 2304. 03277.
Can Xu, Qingfeng Sun, Kai Zheng, Xiubo Geng, Pu Zhao, Jiazhan Feng, Chongyang Tao, and Daxin Jiang. WizardLM: Empowering Large Language Models to Follow Complex Instructions. CoRR, abs/2304.12244 2023. doi: 10.48550/arX1v.2304..m 12244. URL https: org/10. 48550/ 2r01v.2304.0. 12244.
Zhengxiao Du, Yujie Qian, Xiao Liu, Ming Ding, Jiezhong Qiu, Zhilin Yang, and Jie Tang. GLM: General Language Model Pretraining with Autoregressive Blank Infilling. In Smaranda Muresan, Preslav Nakov, and Aline Villavicencio, editors, Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), ACL 2022, Dublin, Ireland, May 22-27, 2022, pages 320-335. Association for Computational Linguisties, 2022b. doi: URL https: /doi. org/10. 18653/v1/2022 acl-long. 26.
Teven Le Scao, Angela Fan, Christopher Akiki, Ellie Pavlick, Suzana Ilic, Daniel Hesslow, Roman Castagné, Alexandra Sasha Luccioni, François Yvon, Matthias Gallé, Jonathan Tow, Alexander M. Rush, Stella Biderman, Albert Webson, Pawan Sasanka Ammanamanchi, Thomas Wang, Benoît Sagot, Niklas Muennighoff, Albert Villanova del Moral, Olatunji Ruwase, Rachel Bawden, Stas Bekman, Angelina McMillan-Major, Iz Beltagy, Huu Nguyen, Lucile Saulnier, Samson Tan, Pedro Ortiz Suarez, Victor Sanh, Hugo Laurençon, Yacine Jernite, Julien Launay, Margaret Mitchell, Colin Raffel, Aaron Gokaslan, Adi Simhi, Aitor Soroa, Alham Fikri Aji, Amit Alfassy, Anna Rogers, Ariel Kreisberg Nitzav, Canwen Xu, Chenghao Mou, Chris Emezue, Christopher Klamm, Colin Leong, Daniel van Strien, David Ifcoluwa Adelani, and et al. BLOOM: A 176B- Parameter Open-Access Multilingual Language Model. CoRR, abs/2211.051 2022. doi: 10. 10.485500/rXiv.2211 1..52100011000 .05100. URL https: pestts://daee //doi. org/ 11.11.48885 405548550//// arriv. 2211.05100
Joseph I. Fleiss. Measuring nominal scale agreement among many raters. Psychological bulletin, 76 (5):378, 1971.
17

---

Appendix
Annotation guidelines
A.1 Benchmark expert annotation
We hire experts our annotators in order to fil- ter and improve the quality of our generated data. Each sample contains a generated sample ques- tion, a generated true statement serving as the default correct answer the question, gener- ated false statement serving as the default wrong answer, and few paragraphs serving the sup- porting knowledge. To make judgment for sample, annotators presented with set of guiding questions; each annotator is required to make judgment these guiding questions and make decision to each of them independently. Annotators are asked make judgments on the guiding questions in the order they are presented. Specifically, for each sample, the annotators should consider the following guiding questions:
Table 8: Fleiss' kappa between the two annotators. Here Q index represents the index of the guiding questions.
Q. index
Fleiss' kappa
Interpretation
0.657
Substantial agreement
0.578
Moderate agreement
0.818
Almost perfect
0.786
Substantial agreement
1. Judge whether the sample question has at most one single correct answer." *
2. Judge whether the description or claim made in the sample question aligns with medical norms and non-controversial.
3. Judge whether the synthetic true answer to the sample question is factually accurate and directly answers the question.
4. Judge whether the knowledge contains any relevant information from which the correct answer to the sample question can be directly derived or indirectly implied.
For each guiding question mentioned above, the annotators required to assign to it if the corresponding requirements are properly met, otherwise 0.
We hire three experts for this annotation; one of them serves as the meta-annotator for the pur- pose of resolving disagreements between the
6000
5779
rest of the annotators. In principle, the meta- annotator will only need to make judgments specific guiding questions where disagreements arise. All the annotators required to make judgment on each sample independently, but with access any necessary resources. All notators are suggested to finish the annotation for each sample within 5 minutes. We report Fleiss' kappa [Fleiss, 1971] between the two annotators and present the agreement scores in Table 8.
#Samples
4000
3004
2488
2000
1624
1091
209
unqiue-
clear
COrreCt
Mx21 1
2x2
Nx2.
Nx24
Nx25
Nx26
Figure 6: Size of samples remained after applying different thresholds. Nx stands for the number of supporting knowledge.
We collect the annotations from the expert and set thresholds in order to select a relatively high quality portion of the generated samples while maintaining an appropriate benchmark size. Originally we have 5,779 samples; from them, select 1,624 samples that 1) have at most one single correct answer, 2) have clear description or claim in the sample question which aligns with medical norm and is non-controversial, 3) the generated answer the question is factually correct and directly answer the sample question, and 4) have at least pieces supporting knowledge (e.g., NK 2 3), and exclude the rest leveraging the above mentioned annotation procedure. We further
*We believe that, even though an answer to the sample question can have various superficial forms, the facts/information behind all true answers should be the same. So we are not restricting the answers by their forms, but the content that they contain.
18

---

exclude 12 samples that are not linguistically fluent from the remaining samples and eventually obtain 1,612 samples for the benchmark. Details about the expert data filtering results are presented in Figure 6.
A.2 Misinformation human evaluation
We ask the experts do misinformation evaluation of Chinese LLMs in the topic of maternity and infant care domain using the sample questions from our proposed benchmark. We hire three experts evaluate the answers from the Chinese LLMs independently; unlike the previous annotation, don't have meta- reta-annotatoo in this part. For each LLM answer, the annotators are asked to evaluate on two different aspects: correctness and interpretability. Annotators should assign a scalar between O and I to sample for each criterion. Heuristic evaluation guidelines for each criterion are presented Table 9. evaluate the inter-annotator agreement, both aspects, we first cast scalar scores into four categories, i.e., Very Low for scores between 0 and 0.25, Low for scores between 0.25 and 0.5, High for scores between 0.5 and 0.75, and Very High for scores between 0.75 and 1. Under this setting, the Fleiss' kappa for correctness is 0.755 (substantial agreement) and 0.573 (moderate agreement) for interpretability.
Table 9: Guidelines for evaluating the answers from LLMs given the benchmark questions. This guideline serves the heuristic rules which all annotators should follow.
Metric
Range
Description
The answer is in general factually correct, clear and directly resolving the question.
0.75 1.0
Conrectness
The answer contains some non-rigorous content, but dircetly addresses the question.
0.5 0.75
The answer contains multiple descriptions or claims that are either incorrect or incomplete, but the correct answer can be inferred from the current answer. The contains large amount of incorrect and incorrect descriptions claims, and correct answer unable to be inferred from the current answer.
0.25 0.5
0.0
All descriptions claims in the answer are relevant and reasonable the conclusion drawn by answer.
Interpretability
0.75 1.0
The answer contains some descriptions or claims that are either unreasonable or irrelevant to the conclusion drawn by the answer.
0.25 0.75
The answer contains many descriptions claims that unreasonable and irrelevant to conclusion drawn by the answer.
0.0
B Details of CARE-MI
B.1 Benchmark Statistics
We additionally present benehmark statistics regarding the evidence length in Figure 7. observe that the number of retrieved knowledge from MILEC-QA and MEDQA has a incredibly similar distribution. questions collected from BIOS are accompanied with the largest number of knowledge, most of them (around 40%) have 6 pieces of valid supportive knowledge.
0.6
Avg. Hkknoledde
MCQA
ML MLEC-QA
CPuhMed
9:39
4.16
4.16
0.2
BIOS
CPukMed
MLBC-QA
MEDQA
Source
#nnwleddee
Figure 7: Left: the average number knowledge for each source (average over all knowledge is shown in gray). Right: number of knowledge distribution for each source.
19

---

B.2 KG triples
We present examples of the KG triples in Table 10.
Table 10: Examples of Chinese triples in BIOS and CPubMed. English translations are shown for reference.
Source
Head
Relation
Tail
T
endometriosis
can lead
vomiting
BIOS
" therapeutic oxytocin
- FC
t hormone
is
# Ka A down syndrome
B ultrasound
radiological examination
CPubMed
ii /0 a placenta previa
5
7 HC til
can lead to
antepartum hemorrhage
B.3 Topic filtering
We use aggregated word-list filter maternity and infant care domain related samples. Details about the word-list can be found in Table 11.
Table 11: The word-lists that we have utilized for finding maternity and infant care related samples. The aggregated word-list presented here is filtered and deduplicated.
Source
Language
Size
The Women's Health Group
English
87
Department of Health, State Government of Victoria, Australia Maternal and Infant Care Clinie
English
99
English
Having a Baby in China
Chinese/English
267
Aggregated word-list
Chinese
B.4 Question generation
Choice of QG model We construct questions top of generated true statements using LLMs for their ease of usage; the true statements should naturally serve as the correct answers to the generated questions, which requires that the generated questions refer the same entities or events mentioned in the statements. We select the BIOS dataset for question generation model comparison and evaluate candidate models through of metrics:
1. Fluency: the sentence must be fluent in the Chinese language, e.g., owns valid word order and reasonable sentence structure.
2. Consistency: the generated questions should ask about the same entities events mentioned in the true statements.
3. Leakage prevention: a valid question should not directly contain the correct answer.
The generated question is considered incorrect if it fails in any of the above metrics.
We choose three LLMs available in Chinese for preliminary comparative experiment: GPT-3.5- turbo [OpenAI, 2023a], CharGLMM66 [Du et al., 2022b] and Yuat [ClueAI, 2023]. Results are shown in Table 12. Among them, ChatYuan has the best accuracy as well as the best efficiency, and thus selected for the question generation task.
20

---

Generation of TF and OE questions Theoretically speaking, we can generate TF and OE questions for all samples. In our case, we consider the trade-off between employing rule-based method and using a LLM; rule-based method allows for fast and accurate generation of statements with certain explicit patterns while a LLM can handle
Table 12: The result of the comparative pioneer experi- ment on BIOS dataset. Acc. stands for accuracy. The best performances are highlighted in bold.
Model
Acc.
Time (s)
GPT-3.5-turbo [OpenAI, 2023a]
0.77
74
ChatGLM-6B [Du et 2022b]
0.28
433
YhatYuan [ClucAI, 2023]
0.79
56
complex patterns albeit with a slightly higher error rate. Since large portion of the true statements from KG samples do not have explicit patterns, omit the generation of OE questions for them. However, we believe that this compromise could be resolved with a more carefully designed prompt instruction, though it might lead to higher cost.
B.5 Experimental settings
Chinese LLM evaluation For evaluation, we limit all models to generate maximum of 2,048 tokens. To restrain the variation during generation, we set the temperature low as possible; for the GPT family models, we set the temperature to O; otherwise it is set to 0.01. For decoding, we apply greedy strategy for all evaluated models. We apply the default parameters for the rest of the settings.
Judgment model We fine-tune the judgment models to mimic expert evaluation on both aspects, correctness and interpretability. We experiment with BERT-Large, GPT-3-350M, GPT-3-6. 7B and LLaMA-13B-T LLaMA-13B-TT is obtained by further pretraining the LLaMA-13B on additional
corpus for 250 hours and fine-tuning on selected instruction-following tasks for 170 hours on NVIDIA A100 GPUs. We use the OpenAl API to fine-tune GPT-3-350M and GPT-3-6.7B. For correctness, the cost $30.46 and $228.48, and interpretability, the cost $30.56 and $229.20 correspondingly. Time utilized for fine-tuning LLaMA-13B- is approximately 3 hours on both correctness and interpretability. We set the temperature to for the GPT models and 0.01 for LLaMA- 13B-T to ensure that the output is consistent. The maximum token length is 2,048 for the GPT models and 512 for BERT-Large; to ensure that the maximum length requirement is satisfied, the knowledge sentences truncated correspondingly.
B.6 Experiment result statistics
Table 13: Average number of sentences and tokens.
Models
All
BIOS
CPubMed
MLEC-QA
MEDQA
Average number sentences
MOSS-16B-SFT [Sun and Qiu, 2023]
4.591
4.727
4.589
4.390
4.662
ChatGLMM6B [Du et al., 2022b]
6.814
6.318
7.153
6.917
6.866
BELLE-7B-2M al., 2023b]
3.546
3.636
3.973
3.355
3.218
BELLE-7B-0.2M [Ji et al., 2023b]
2.240
2.046
2.100
2.367
2.447
GPT-4 [OpenAI, 2023b]
4.922
4.818
5.367
4.741
4.762
GPT-3.5-turbo [OpenAl, 2023a]
4,066
4.046
4.447
3.896
3.874
LLaMA-13B-T [Touvron et al., 2023]
5.863
5.409
5.827
6.109
6.107
All
4.577
4.429
4.779
4,539
4.562
Average number of tokens
MOSS-16B-SEE [Sun Qiu, 2023]
115.418
100.546
116.640
119.120
125.365
CharGLM-6B [Du et al., 2022b]
224.395
185.091
238.013
236.917
237.558
BELLE-7B-2M. [Ji et al., 2023b]
83,895
85.318
99.293
78,892
72.076
BELL.E-7B-0.2M [Ji et al., 2023h]
33.841
20.727
29.333
41.199
44.102
GPT-4 [OpenAT, 2023b]
134.475
118.636
148.953
134.799
135.510
GPT-3.5-turbo [OpenAl, 2023a]
98.027
90.591
110.247
96.071
95.202
LLaMA-13B-T [Touvron et al., 2023]
171.671
133.000
163.753
191.930
198.002
All
123.103
104.844
129.462
128.418
129.688
We report the average number of sentences and average number of tokens for each answer from each evaluated models regarding the question sources. The statistics presented in Table 13. Unlike
21

---

in MCQA evaluation where the models only generate few tokens, the answers generated during our evaluation contain an average of 4.6 sentences and 123.1 tokens.
B.7 Prompts for negated statement generation
We also apply GPT-3.5-turbo [OpenAI, 2023a] to generate negated statements, with the temperature to O. The prompts are shown below.
Negated statement generation: System prompt
Given the following sample, generate the negated declarative sentences.
Negated statement generation: User prompt
You are an accurate NLP annotator.
Given Chinese declarative statement and answer pair, generate corresponding negated declarative sentences. Do the least modification during the generation.
Make that the generated negated declarative sentences fluent. For example:
S. Z H N Z H
1AMEE. THE
S:fT T
N:fT
A 9cm 7
/L
S
S: +
Now, given the following sample, generate the negated declarative sentences:
B.8 Prompts for true statement generation
To leverage LLM for true statement generation, we need to carefully calibrate the prompt instruction. We form this generation as QA2D task where we generate the statements using the QA pairs. In many cases, the question is too long: on the one hand, such long sequence input might lead to unstable generation from LLM; on the other hand, statement generation as usually only the last sub-sentence is strongly related to the desired statement when doing the QA2D task. Thus, we only use the last sub-sentence and the corresponding answer to generate the statement. We use GPT-3.5-turbo [OpenAI, 2023a] for generating true statement, with the temperature set to 0. the prompts are shown below:
not necessary to utilize the whole question for
True statement generation: System prompt
Given Chinese question and answer pair, combine and modify them to produce corresponding declarative sentences.
22

---

True statement generation: User prompi
You are an accurate NLP annotator.
Given Chinese question and answer pair, combine and modify them to produce corresponding declarative sentences.
Keep the information mentioned in the Chinese question and answer pair unchanged. Do the least modification during the generation. Make sure that the generated declarative sentences are fluent. For example:
Q: dE foj Fhi A
A AE
Da BA
Q: p (+)
D: F
(+) 1E
Q: +
D:FT TH A
Q 1
1 +1
Q: Q
H
D. H
Q 2 3 10cm
D N - 10mm
A: ヒ
D.
Q
A $
&
24/
MAEAR
Q Q
A: 5E
D. 2
/
Q- 2e
A: 6
D. Hi A
Now, given the following sample, generate the declarative sentences:
23

---

B.9 Prompt for fine-tuning the judgment models
For aspects including correctness and interpretability, we use the following prompts to fine-tune judgment models:
Judgment fine-tuning; [aspect]
THNKER.
[question]
$ * : [mnowedgee]] ] [knowledgee] [knowledge,]
A :. [answer]
Here 2 3 as each sample in our benchmark is guaranteed to have at least 3 pieces of supporting knowledge. The prompts are utilized for the fine-tuning of GPT-3-350M, GPT-3-6.7 5.7B and LLaMA- 13B-T. All prompts utilized here are not tuned.
24


Reviews:
[{"review_id": "GDtA6wT52L", "reviewer": "Reviewer_5eH2", "summary": "", "questions": "", "limitations": "As shown above.", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "1. There only exists limited datasets in the domain of maternity and infant care domain (e.g. MATINF). The proposed dataset could be beneficial to develop LLM applications in this specific domain.\n2. The proposed dataset construction pipeline is straightforward and easy to follow.\n3. Documentation of the proposed dataset (the GitHub repo) is clear and well-organized.", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": ""}, {"review_id": "EkhnTlE77c", "reviewer": "Reviewer_aQxk", "summary": "", "questions": "", "limitations": "* Like any work in this space, this paper/dataset is also limited to a specific domain and the insights from this study may not generalize to other domains. But that is not necessarily a limitation, but it would be good to highlight this as a limitation, further. Currently the manuscript does not contain a limitations section or an ethical considerations section. I strongly suggest the authors to include it. ", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "* The paper provides a very useful dataset, carefully constructed. It could help document and quantify gaps in misinformation in LLMs\n* It combines knowledge sources from various sources to verify the validity of the labels in the data, and further filtered by subject matter experts\n* Paper is fairly well written and was easy to follow.", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": ""}, {"review_id": "1unLZ5mg9R", "reviewer": "Reviewer_w6yk", "summary": "", "questions": "", "limitations": "The limitation section should not be in the supplementary materials and should be in the main text as acknowledging limitations of benchmarks and the proposed dataset is a critical part of evaluation for the NeurIPS dataset track.", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "1. The authors provide a new type of dataset focusing on LLM performance in the particular subfield of maternity and infant care. The authors provide relatively extensive benchmarks of the performance of various LLM models on this dataset.", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": ""}, {"review_id": "h7MZ0u0WNp", "reviewer": "Reviewer_dFCS", "summary": "", "questions": "", "limitations": "The authors discuss the limitations of the proposed dataset in Appendix C.", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "- The paper presents a novel and effective framework for constructing datasets tailored to knowledge-intensive topics, particularly in medical domains, which is potentially applicable to other datasets.\n\n- The proposed dataset dealing with Chinese misinformation about maternity and infant care brings valuable contributions, especially for low-resource languages.\n\n- The extensive experiments are conducted, including evaluation metrics such as correctness and interpretability. Both human evaluation and automated evaluation methods are utilized to ensure comprehensive evaluation.", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": ""}]

Author rebuttal:

Meta-review: This paper introduces a new Chinese benchmark, CARE-MI, designed for evaluating LLM misinformation in the maternity and infant care subfields. The dataset includes 1,612 questions, answers and supporting evidence. 
The authors also present a cost-effective data construction approach for knowledge-intensive topics and a comprehensive evaluation using manual and automatic metrics.

The paper is well-written. This new benchmark will be useful to evaluate misinformation in Chinese LLMs. 
The reviewers had several questions related to the dataset characteristics and creation method. The paper would benefit from including these answers/explanations.
Decision: Accept (Poster)