FELM: Benchmarking Factuality Evaluation of Large Language Models
Authors: Shiqi Chen
Yiran Zhao
Jinghan Zhang
I-Chun Chern
Siyang Gao
Pengfei Liu
Junxian He
Venue: NeurIPS 2023 Datasets and Benchmarks Poster (2023)
Keywords: factuality
LLM
TL;DR: A benchmark for factuality evaluation of large language models

Abstract: Assessing factuality of text generated by large language models (LLMs) is an emerging yet crucial research area, aimed at alerting users to potential errors and guiding the development of more reliable LLMs. Nonetheless, the evaluators assessing factuality necessitate suitable evaluation themselves to gauge progress and foster advancements. This direction remains under-explored, resulting in substantial impediments to the progress of factuality evaluators. To mitigate this issue, we introduce a benchmark for Factuality Evaluation of large Language Models, referred to as FELM. In this benchmark, we collect responses generated from LLMs and annotate factuality labels in a fine-grained manner. Contrary to previous studies that primarily concentrate on the factuality of world knowledge (e.g. information from Wikipedia), FELM focuses on factuality across diverse domains, spanning from world knowledge to math and reasoning. Our annotation is based on text segments, which can help pinpoint specific factual errors. The factuality annotations are further supplemented by predefined error types and reference links that either support or contradict the statement. In our experiments, we investigate the performance of several LLM-based factuality evaluators on FELM, including both vanilla LLMs and those augmented with retrieval mechanisms and chain-of-thought processes. Our findings reveal that while retrieval aids factuality evaluation, current LLMs are far from satisfactory to faithfully detect factual errors.

FELM: Benchmarking Factuality Evaluation of Large Language Models
Shiqi Chen
Yiran Zhao Jinghan Zhang²
I-Chun Chern
Siyang Gao
Pengfei Liu Junxian He
City University of Hong Kong The Hong Kong University of Science and Technology
National University of Singapore 'Carnegie Mellon University SShangaii Jiao Tong University
schen438-cêmy (tt.:/8--@mm.ccttttc ciityu.edu.hk, junxianhecse.us ust.
Abstract
Assessing factuality of text generated by large language models (LLMs) emerging yet crucial research area, aimed at alerting users to potential errors and guiding the development of more reliable LLMs. Nonetheless, the evaluators assessing factuality necessitate suitable evaluation themselves to gauge progress and foster advancements. This direction remains under-explored, resulting in sub- stantial impediments to the progress of factuality evaluators. To mitigate this issue, we introduce benchmark for Factuality Evaluation of large Language Models, referred to as FELM. In this benchmark, we collect responses generated from LLMs and annotate factuality labels a fine-grained manner. Contrary to previous studies that primarily concentrate on the factuality of world knowledge (e.g. in- formation from Wikipedia), FELM focuses on factuality across diverse domains, spanning from world knowledge math and reasoning. Our annotation is based on text segments, which can help pinpoint specific factual errors. The factuality annotations further supplemented by predefined eгror types and reference links that either support or contradict the statement. In our experiments, we investigate the performance of several LLM- I-baeee factuality evaluators FELM, including both vanilla LLMs and those augmented with retrieval mechanisms and chain-of- thought processes. Our findings reveal that while retrieval aids factuality evaluation, current LLMs are far from satisfactory to faithfully detect factual errors)
Introduction
Large language models (LLMs) have achieved stunning success, resulting a paradigm shift towards generative Al based prompting (OpenAl, 2022 Chowdhery al. 2022 Touvron et al. 2023 OpenAl 2023). However, a known issue of LLMs is their tendency to generate falsehoods or hallucinate contents, posing significant hurdle to broader applications. Even saate-of-the-an LLMs such as ChatGPT (OpenAl 2022) are susceptible to this issue as shown in Borji (2023): Zhuo et al. (2023): Min et al. (023),, which raises concerns about the practical utility of these models. Consequently, factuality evaluators that could detect factual errors in LLM's 's responses are urgently needed alert users to potential risks and drive the development of more reliable LLMs. For example, an ideal factuality evaluation system, as demonstrated in Figure D. should be able to segment the LLM responses into fine-grained textual spans, assess the factual correctness of each segment, and highlight any errors for the users. To facilitate interpretability. the factuality evaluator may also categorize the error type, provide an explanation, and offer reference links to justify its assessment.
"Work done during visiting HKUST
'The FELM dataset is available at htlugs///ggiuubb.coo/ouuusaaappp
37th Conference on Neural Information Processing Systems (NeurIPS 2023) Track on Datasets and Benchmarks.

---

World Knowledge
Reasoning
(Wite the malerr conan in the worsh?
Levor Information
Prompt
Toun Srakkrdaarrrt.co gets 6 reseraatinsa right They rermaly order medi that each and
ChatGPP I' Respunse & Segment Judgrmmat
The smalisst ooein the world's is ihe Artte Ocsen. // locsed theacrti acthemmont part of the barth
knewiddgs ere.
CharGPT Response & Segment Judgement
Resnwn:
They get 2oood 2ogg2 meals right That Meoy
Durope
Winh on aina atabout milloon opar miles
12 2appst222222ae 22222222 moal X
// miloosaan litendaas
Mioo /nsoometeeee ros www.fettleeaaaaeeneoeea weweoooeeeeeeeensrr.to Lite!
Ryfirence Boik
Eeror Information
Erw
Reascring erter
FELM
Resson:
That mere they make 12"6:5o01225 shool on meal.
Science
Writing/Rcommmeddaiion
Presagt
Math
Promutt (Derommmend come onliee murees iorm Pyther
Beckowerds sptperoti inhonttn tamoe phoo mancn
ChatCiPT I' Response Segment Judgemuent
Promut
(-2 Se D what the salue at
thch
ChatGiT Resgonse Segment Judgement
zene momm
instead
ChutGPT Response & Segment Judgemeett
Pytter for bvenbody Univenity of Michhgin on counsrr Pyther Crach Coume by toe Nouthet X
tersa
Verian euantvtic coustee at Hreeforr
Information
Error Information
sosttin
ope lirkesan
Froor oyer
Froor Information
Reaso:: i Look
Reawe:
Ereor oper Reasciing
sincape
Refermace Bal:
Reasse:
t/ paato Cruch Courne Pradee-
Demonstration examples of factuality evaluation system it could highlight the text spans from LLMs' responses with factual errors, explain the error, and provide references to justify the decision. Our proposed benchmark. FELM, annotates all the information following this scheme, aiming to drive the development of such factuality evaluators.
Figure I:
While factuality evaluation of generated text has been extensively explored (Thorne et al. 2018
Wang et al.) 2020; Pagnoni et al. 2021 Fabbri et al. [02 Honovich et al. 2022). existing literature primarily focuses
specific task (e.g. summarization), a particular domain (c.g. Wikipedia), and text generated from less capable models such as BART (Lewis et al. 2020). Therefore, factuality evaluation of long-form text generated by LLMs in diverse settings emerges research direction. This area is becoming increasingly important as LLMs secure a dominant role as the foundation of the generative AI paradigm. To further this direction, we require new factuality evaluation methodologies and meta-evaluation benchmarks. This paper primarily addresses the latter, proposing a meta-evaluation benchmark to gauge the progress of factuality evaluators. We believe that appropriate evaluation is the prerequisite of facilitating future advancements.
novel yet challenging
Specifically, we broaden the conventional understanding of factuality within the world knowledge domain to encompass five diverse domains- - world knowledge, science and technology, math, writing and recommendation, and reasoning to align with LLMs capabilities of performing tasks in varied settings. For each domain, we undertake a four-step process to construct the benchmark, we (1) gather prompts from various sources, (2) collect the corresponding responses from ChatGPT, (3) segment the responses into fine-grained text spans, and (4) ask human annotators to annotate the factuality label, error type, еттоr reason as well references links that are used to make the judgment. The resulting benchmark, referred to as FELM (Factual Evaluation of large Language Models), embodies the data scheme displayed in Figure [ In the experiments, powerful L.LMs, ChatGPT and GPT-4 OOpenAl 2023). as factuality evaluators on our benchmark, augmented with different techniques such as external evidence and thain-of-thougee reasoning (Wei) let al.) 2022). Our findings show that factual error detection remains a challenging task for LLMs, and we highlight the need for external tools to improve the performance.
examine the abilities of two most
2 Related Work
Prior benchmarks for factuality detection mainly focus on specific tasks like summarization (Kryscin ski et al. 20200) Wang et al., [2020] Maynez et al.) 2020) Pagnoni et al.] [02T] Fabbri et al. [021 Tang et al. 2002), or particular domains like world knowledge (Thorne et al. 2018 Schuster et al. 2021 Kamoi et al., 2023), where the knowledge could verified by evidence from Wikipedia. In these works, factuality evaluation is to determine whether the given text could be entailed from relevant evidence. For example, summarization factuality detection aims to examine whether the generated summary is consistent with the given document, while other benchmarks often require
2

---

Response
Granularity
Evidence
Scenario
Datasets
Length
Generated by
Provided
Domain
FEVER
7.3
Human
Claim
Wikipedia
FacICC
20.8
Synthetic
Sentence
Newswire
QAGS
16.1
Model
Summary
Newswire
WICE
24.2
Human
Claim
Wikipedia
HaluEval
36.9
ChatGPT
Response
X
QA/Newswire
FELM 89.1
ChatGPT
Segment
Five domains
Table I: A comparison of published factuality henchmarks w.r.t model generated responses to be verified based on collected evidence. We explain the definition of "segment" and "claim" in 3.1
the factuality methods to have an external retrieval module that finds relevant evidence to succeed. Benchmarks presented by Thorne al.] (2018) and Kamoi al. (2023) only focus on factuality errors made by humans when addressing world knowledge. However, these benchmarks alone do not meet our specific requirements for evaluating LLM's factuality. recent work Li al. (2023a) introduces a factuality benchmark HaluEval which focuses on three tasks: knowledge-based dialogue, summarization and world knowledge QA. They construct HaluEval by deliberately inducing LLMs to produce errors, while we instead collect LLM's errors cases under real scenarios. There is another line of factuality benchmarks focus on knowledge-based dialogue. Dziri al. (2022) and Rashkin et al. (223)) specifically focus on factuality of dialogue systems that incorporate pre-injected back- ground knowledge. However, our study diverges by focusing on an open-domain context setting. This implies that the responses in FELM are generated directly without referencing any external knowledge sources. In this paper, we are concerned bout how factual errors in a long-form response generated by LLMs (e.g., ChatGPT) in different task scenarios under 0-shot setting can be identified in more granular manner.
3 FELM
3.1 Design Principles
The design of FELM first requires delineating the scope or definition of factuality. Factuality in text generation systems generally refers whether the synthetic text contains any factual errors or not. These errors can take various forms, such as an incorrect entity, a fabricated paper reference, a misleading scientific claim, unlogical reasoning, and incorrect ematical calculations. Despite the breadth of this definition, existing benchmarks, as indicated in Table [ typically focus on single domain. Most commonly, they target the world knowledge domain, wherein the factual knowledge is mostly about some entities such as celebrities and places. However, as LLMs have demonstrated strong generalization performance across wide range of scenarios (Chen et al., [02T]] Taylor et al. [022] OpenAl 2023 Li et al. 2023b Lightman et al. (2023), the user prompt queries can be highly diverse, leveraging LLMs perform nearly all the NLP tasks. In light of this, we argue that factuality evaluators should account for diverse factual errors, and the first high-level principle of
Factuality:
FELM is to cover multiple distinct domains as we will detail in $3.
What level of granularity should we adopt for the data samples? Should it be at the response, segment, or claim level? Previous work has adopted different granularities when creating data, as shown in Table [T] The data format of benchmarks like FELM is crucial as necessitates similar output format from factuality evaluators for assessment, indirectly guiding the development of factuality evaluators towards the defined outputs. Therefore, in FELM, we adopt user-oriented perspective and ask: which output format from factuality evaluators is more helpful, friendly, and interpretable for the users? Comparing different granularities, we find that segment-level annotation is the closest to our end goal, highlighting factual correctness of seg- ments directly from the response. This approach is not only intuitive and user-friendly but also aligns with widely adopted methods of providing references for text, as seen Wikipedia and Microsoft's 's Bing Search (in chat mode). Such fine-grained annotation allows factuality evalua- tors to examine the segments individually, a process considerably simpler than justifying an entire response directly. While finer- finer-grained annotations at the claim level-the extract atomic factual
Data formats:
3

---

claims have been adopted previously to simplify factuality evaluation (Min et al. 2023), the ex- tracted claims do not directly correspond text spans and may be less user-triendly as the final output. However, in the experiments ( we will demonstrate that claim-based factu- ality evaluators are the most effective, and the extracted atomic facts could serve as intermediate outputs that can ultimately be mapped back to segments. Beyond the basic factuality labels, we also aim to provide detailed error infor- mation, such as error type, reason for the error, and reference links supporting the label. We believe these additional meta information are vital outputs that users would value.
Prosuptt aboutisc Happiness
Segments and Claims
CharGPT Responded Segmonts
Segmose
World Hopptness Report an arnual report published by the Unieed Nations Sustairable
Devilopment Solutions Neltwort that rarios couninus by their level of happinass or subgectvve well bo Segmows
Extracted Claims
Segweor I:
Claim 1, The World tappiness Repport is annual report.
Claim 2. The (Wolld Huppinsss Report is publisited the United Nationo Susteinabre Development
Solutions Network.
tappiness Report
by tappiness srbjecttee
Segmont 2......
Figure 2: Segments and Claims
3.2 Factuality on Diverse Domains
In line with our design principles, FELM emphasizes a comprehensive concept of factuality, encom- passing five diverse and realistic domains as illustrated in Figure and detailed below.
This is one of the most widely-employe domains in factuality detection, which generally represents knowledge regarding specific entities such as movies, countries, dates, places, and people - for example, factual errors on the Arctic Ocean as shown Figure
World Knowledge:
Science and Technology: LLMs may hallucinate more often in terms of scientific claims and knowledge which occur relatively sparse on the web compared to world knowledge. For example, common observation is the tendency of LLMs to generate fabricated research papers and citations. In FELM, we encompass factual errors related to scientific claims and paper citations, which span various academic disciplines such as ematics, physics, chemistry, and biology.
Recommendation and Writing: Recommendation and writing are likely among the most com- monly used applications of LLMs nowadays. Examples include asking LLMs to recommend movies or draft an email. In these situations, users often pose broad and open-ended questions such as "How to learn Python?". In response, LLMs generate content in more unconstrained fashion. Factual errors in these instances pertain to the details generated about entities, such as a book and a movie.
Reasoning: Reasoning is one of the most important abilities of LLMs since it relates to L.MMs' potential in complex environments. In multi-step reasoning, chain-of -fhthuggtt prompting (Wei et al.) 2022) has become a standard for LLMs to first generate a trace of reasoning steps and then obtain the final answers. This task challenging for LLMs, and the reasoning traces often contain errors (Jung ct al. 2022) that have rarely been studied before.
Math: Mathematical problem solving is another challenge for LLMs. It requires LLMs to think logically and apply ematical principles to find the correct solution to problems. Some prior researches have shown concerns for LLMs' ability (Azaria 2022 Frieder et al. 2023).
3.3 Overview of FELM
Before diving into the specific construction steps of FELM, we first overview the overall statistics of FELM consists of total of 817 samples and 3948 segments, each domain has at least 100 samples and 500 segments. The responses are generally long with an average of 81.6 tokens. The overall error rate the response-level is 31.8%.
FELM in Table 2
3.4 Construction Process: Prompt Collection
The first step of constructing FELM to gather a variety of prompts. Specifically, we source prompts from online platforms like Quora, Twitter, standard benchmarks such as MMLU (Hendrycks et al. 2020) and TruthfulQA (Lin et al. [202)), and from self-instructed ChatGPT generations (Wang et al. 2022b). Additionally, manually draft minor fraction prompts. Representative examples in FELM

---

Staties
All
WK
Reasoning
Math
Science
W / /
#Sample
847
184
208
194
125
136
Error rate (%)
33.3
46.2
22.6
33.0
31.2
34.6
#Segment
4425
532
1025
599
683
1586
#Positive
3640
877
477
582
1319
#Negative
785
147
148
122
101
267
Avg. R Length
89.1
50.6
75.1
44,9
104.8
210.9
Avg. S Length
17.1
17.5
15.2
14.6
19.2
18.4
Agree rate (%)
91.3
81.5
94.5
94.2
87.7
96.6
Table 2: Statistics of the FELM henchmark. Here "WK" stands for "World Knowledge" and "W / R" stands for "Writing Reccommendation" "Error rate" is ratio the responses containing factual errors. "#Posi- tive" #"egatiivee"" denotes the numher of segments labeled as correct and incorrect respectively. "Avg. S Len." and "Avg. R len." are the average length for all the segments and responses. Agree rate is the agreement rate of annotators during annotation.
MMLU/cc 2.2%
Authors 2,0%
GSMBK 24.7%
sources 9.5%
World
83%
Knowledge
Cunra 10.6%
Science/Tech
86%
889 236-99
Truthfu/iA
Writing/Rec
66%
34%
Math
100%
21.2%
MATH 6%
Reasoning
100%
25%
50%
75%
Figure 3: Prompt Source in FELM
Figure 4: Distribution of different error types
are shown in Figure ! We utilize different sources to collect prompts for the five domains, and the overall distribution of prompt sources is illustrated in Figure
In detail, for world knowledge domain, we involve questions related with history, society, common sense and news from TruthfulQA (Lin et al. 2022), Quora (from the History and Society subjects), online sources Charces/hhh3 he3 (Guo et al. 2023), and MMLU (Hendrycks et al. (020) (only the US Foreign questions drafted by ChatGPT and the authors. For
Policy subject is used). There are also the science and technology domain, we curate scientific questions from Quora (we use Scientific Research, Science of everyday life, Technology, and Physics subjects), MMLU (we use College Chemistry, Computer Security, and Econometrics subjects), and online sources. We also draft a small fraction of queries ourselves. The recommendation and writing domain is constructed using questions generated by ChatGPT and manually crafted by the authors. As for reasoning, our dataset includes queries from GSM8K (Cobbe al. (021), supplemented by online sources. For the math domain, our question pool draws from MATH (Frieder et al. 2023), online sources and authors. We detail the prompt collection process of each domain in Appendix
3.5 Construction Process: Response Generation Segmentation
Following the prompt collection, we employ ChatGPT to generate responses for the collected prompts zero-shot setting. In accordance with the data format discussion $3.1 we segment each response into a list of text segments in the next step. We note that segmentation in FELM is mainly for enhancing interpretability which quite subjective there no definitive "optimal" segmentation
"The online blog, github repository. twitter thread and documented archive we take as reference
wtt.:////..ieee///ttttttt-aaaaa www.suuutt........ httpr/rrrs...stttaaccoooc//aa/ea 01987271454167992281mmmo 5=468t=7b5KyE 5=468t=7b6kyEEBBwdBoyyyot
https://github. cow/
httpp:://wwitterrcom/////
https://twi wtnttterrccm///// zhou htmmm/hoouyuuaaaaa00
ht http:///ech china, com. httpcc//ai/222222213942333 shtml and Borji
2023).
We use "online sources" to refer to them throughout the paper consistently unless otherwise specified.
5

---

Prompts for Math /
Recommendation Domains
You are asked to separate given text/code / text by segments using separator: **..e."
Here are some requirements:
1. The separation is conducted according to the meaning and each segment should be self-contained.
2. Adding all segments up should exactly he the original given text/code / text.
3. The segment may be full sentence or piece of code snippet with its description
procedure for solving a problem or so an item with its description or so.
4. The final return should be segments separated with separator:
Like this: (segment 1) (segmen2) t (ereeeennetttseeeeeeeeneeee (segment3) ***t
Table Prompts to request ChatGPT to segment responses for and recommendation domains. The brown texts are for domain, and the green texts are for recommentdation domain.
to ensure the best interpretability, as this largely depends on the individual user. Moreover, the segmentation does not necessarily impact the prediction process of factuality evaluators, which can always perform at their preferred granularity levels as the intermediate stage, as we will show in $4 how we benchmark a claim based factuality evaluator in FELM. Therefore, we decided to adopt simple and heuristic segmentation methods in FELM, which provide reasonably good results. Specifically, we adopt two different methods for the involved domains. The first approach is segmenting by sentence boundary, which is used for domains with standard rext-paragraph responses, such as world knowledge, science and technology. freestyle writing, and reasoning. We use NLTK's sentence tokenizer (Bird et al.] (009) achieve a consistent, heuristic segmentation. The second approach is segmenting with ChurGPT, which is used for and recommendation samples. Responses these domains often contain numbers, lists, or markdown symbols that are challenging for heuristic segmentation tools, thus we use ChatGPT perform the task. We use the prompts provided in Table 3 which works very well in practice. After separating the responses to segments, we could feed these segments to annotators to conduct the next step.
3.6 Construction Process: Human Annotation
Annotation for FELM is a highly challenging task. The difficulty arises three Firstly, annotators should find external supportive contradictory evidence themselves because the responses do not contain citation information. Therefore, the annotators must possess strong skills in using external tools such as Google Search and be able to filter out unreliable information. Secondly, the responses can be quite lengthy in certain tasks like freestyle writing and question answering, requiring good reading comprehension ability and patience. Finally, certain domains such as science and technology. and reasoning require the ability to solve complex reasoning problems and understand scientific concepts, adding another layer of difficulty the process. After taking the factors mentioned above into consideration and conducting several preliminary trials, including hiring crowd-sourced workers handle the task, it became evident that acquiring high- quality annotations from crowd-sourced workers presented significant challenge. Consequently, we decided to find expert annotators annotate the dataset. Specifically, the annotation involves expert annotators including some of the authors. The annotation interface for annotators is shown in $3. our annotations cover the following four dimensions.
Annotation:
aspects:
Appendix B As discussed
Factuality labels. For each given prompt and corresponding segmented response, annotators would annotate whether each segment contains factual errors or not.
Error reasons. For the segments which contain factual errors, annotators are asked to comment on the details of these errors. These are annotators' comments, mainly about what exactly is the error, why a certain error happens, and what the correct answer is or so.
Error types. We predefine four types of factual errors make it easier to identify the errors, and annotators are required to assign one error type to each segment with errors. The four types are (1) "Knowledge error" that is the most common error, occurring when the model produces hallucinated or inaccurate information in a segment. (2) "Reasoning error" that arises when claim employs flawed reasoning or faulty logic. In FELM, errors math and reasoning domains all belong to the reasoning error category. (3) "Irrelevant" that denotes that the content is unrelated to the prompt. For example, if the prompt is "What's 's a country where most people love playing rugby?", response like "New Zealand is country where

---

maimun nurmer of tacker
Vanilla
Cot
Link-augmented
Doc-augmented
Sunner the lightst namber oadhur maduu other coustries sperating sigetticant nereber ruciear power plants Soorn Koras
Laniiatone queition
will thour you queetion
[wil yreoquesiion a of aegranns
Jill a list bescufeer sagneers
sadlo lut signents
fransa, Dims, Rantt,
[clamn] All Uncaegment [claies van be coe steuaeed
(ctarmo, and rnternnc
[datnal and retermnze
(chlmed)
concitecated to a
(ckarss) can
(ilains) th
complet asswerto tuik
insess whetlerr cack ice:
Lass Your seil
conceorated torma
concaioand Bera
the highot namber muudea:
hay
The United Staioe tax the
paass whather ead bot
beguneet (ckarz) cestatt
pegreent (cain)
question Your talk is
ascess whether exch
asoess whether each
queatios, Rour task to
openatin teattrn
world, with 94
hagnest sembher cocker
Hoar grnerste sang
boknndag
gracustoviing
the berleppo the relienoo
factrall emors nut will
secua ereers ast with
the kelp of refirmece
Segrant
The Unhed bus
Though ressoring
planse Irchade Catea.
Aigttiinnt sumber
cueriio with
Sagment Fiacc thira Raars
emars (eperaedity camman)
Anneen Las the
(dasss) with
agmenta (clarna)
witt errorn at the
Plase grnerste exing the Inticuinggorme
Please genernte using the
Seath
significnt number of
rachor powsr planss
Auuwer Zie the Ltc arthe eepmente (chere))ett conrress)
Acswen Les iés of signeets
Arawee Lest the aás the oegeoees (clene)) comnens)).
[espurated consmna).
factually correct or not
Figure 5: We employ four evaluation schemes in our experiments: Vanilla, Chain-of-thought Reference-link augmented, and Reference-doc augmented evaluators (Prompts in the figure only for demonstration purpose, and the exact prompts we use are Appendix D).
rugby is considered a national passion and is deeply ingrained in the culture..." would be labeled as irrelevant. (ooeeo error" that occurs when the model fails recognize the falsehoods or jokes inherent in the prompt and provides an inaccurate or inappropriate response. For example, if ChatGPT is asked "Is true that new year's day 2023 falls on a Friday the 13th?", it replies "Yes, it is true..."" This type of error is often the result of lack context or understanding of the intent behind the prompt. The error type distribution on each domain is shown in Figure
References. Annotators conduct the annotation process mainly with the help of external tools, especially for noowlede-intensive domains such as world knowledge and science/tech. We ask annotators to indicate the website links that they take reference. The content in the reference link contains information that entails or contradicts the segments.
Every response is annotated by two expert annotators and report their segment-level agree rate in Table where they agree with each other 90.7% of the time on average.
After the first round of annotation, the annotation of each sample is reviewed by one author to ensure the quality. If the two annotators provide different annotations for a sample, we hold discussion between the annotators and the reviewer reach final decision. In the last stage, a super reviewer reviews all the data for quality assurance. At this point, we obtain the FELM dataset. Then, we perform further verification examine two important aspects: reference reliability whether the given reference itself contains incorrect knowledge, and safety - whether the examples contain toxic content. For each aspect, specifically, and the authors or crowd-source workers are asked to annotate reference reliability or safety. Results demonstrate that all the examined samples are safe and provided with reliable references. We detail these human verification experiments in Appendix C
Verification:
randomly select 100 samples from FELM,
4 Experiment
Our experiment below aims to assess several factuality detectors on FELM. We analyze their perfor- mance and point out possible usage of FELM.
4.1
Experimental Setup
Factuality evaluators: We consider LLMs like Vicuna, ChatGPT and GPT4 as the backbone models for factuality evaluators, and study various factuality evaluation methods on top of L.LMS. Specifically, we first cluster the methods as segment-based evaluators and claimbbsce evaluators.
7

---

In segment-based based methods, we directly require the models to assess the factuality of the segments in FELM. In claim-based methods, we first extract a list of atomic fact claims from each segment, and use L.LMs to examine these claims- we label a segment factually correct if all claims associated with the segment is correct, and factually incorrect otherwise. We note that this claim-based method is similar to (Min et al., (023), except that they do not assign segment-level labels. We prompt
LLMs to extract claims from a given segment following Min et al. (2023) (details in Appendix DD. For both segment-based and claim-based evaluators, we further examine four variants for each of them: (1) vanilla: LLMs make the judgement based on the question and segments (or claims in claim-based methods), (2) chain-of-thoute
(cot): LLMs are asked to first generate thought process (Wei et al., 2022) and then make the prediction, (3) reference link: we provide the reference links in FELM for the LLMs to help the assessment, we find this generally helpful since the links themselves often contain helpful information, and (4) reference doc: we access the text corresponding to the reference links and then use the BM25 algorithm (Robertson al. (0099) to retrieve the most relevant text chunks as additional input to the LLMs. We demonstrate the evalution setting in Figure 5 Note that there is reference for math and reasoning domains, and we do not report (laim-bases performance on these two domains either since the responses often involve multi-step reasoning where strong dependence is present between sentences self-contained, short atomic fact claims cannot be extracted in these cases, We test three powerful LLMs as the backbone for factuality evaluators:
Vicuna-33B (vi -vicuna-33----- and GPT4ggtt4-031 4, (OpenAl 2023)). The evaluation prompts in different settings along with other setup details are Appendix D
(Chiang al., 2023 ) ChatGPT (gpt3.5-88 5-0381. OpenAI (2022))
We compute two metrics: the F1 score (along with precision and recall scores) of detecting factual errors and balanced classification accuracy (Brodersen et al.) 2010) that balances the positive and negative examples during computing the accuracy. We measure both segment-level and response- level performance. We report Fl scores only in the main content for ease of space, while include the balanced accuracy numbers in Appendix B
Metries:
4.2 Experiment Results
FELM is a challenging benchmark: Segment-level and response-level results shown in Table 4 We observe that the majority of detectors performed unsatisfactorily on FELM, with only the GPT-4 evaluators achieving an overall average of score greater than 40 in some settings. Most ChatGPT detectors did not demonstrate any fact verification ability on FELM without external tools. In addition to attributing to challenges of the benchmark in general, ChaiGPT's failure on FELM may be due to the fact that all the errors in FELM are collected from ChatGPT's own generations - it is typically harder for a model to detect factual errors made by itself. Notably, the Vicuma-33B evaluators exhibit commendable F1 performance, outperforming ChatGPT significantly. However, upon closer examination of the balanced accuracy in Table TO it becomes evident that the Vicuna-33B evaluators still struggle on this task with balanced accuracy around a random level. Also, we briefly draw comparison with CharGPT/GPT-49 performance on previous factuality detection benchmarks to better understand the difficulty of FELM. For example, ChatGPT (zero-shot) shows around 60%-70% balanced accuracy in diverse summarization factual error detection datasets (Chen et al. 2023). On simpler datasets like SummEval (Fabbri et al. 202I). ChatGPT and GPT-4 are able to make over 80% balanced accuracy as demonstrated in Chen et al. (2023). These numbers are generally higher than the ones on FELM as indicated in Table 10] which implies that open-ended factual error detection as in FELM is harder than detecting factual errors from summaries as in previous benchmarks.
Retrieval-augnne augmented methods help: Both the augmentation approaches with reference links and reference document are effective detecting factual errors. For example, ChatGPT's retrieval- augmented reference document method achieves average increase of 6.4 points in Fl at the segment level, compared to the Vanilla method. Similarly, GPT-4's ettrevallauugennee reference document method achieves 5.5 point increase in Fl at the segment level. Moreover, the retrieval-augmented content method outperforms all other methods across all domains we tested. Therefore, we can conclude that the retrieval-augee augmented method is highly beneficial in detecting factual emrors.
tf-ho-of-thought (Cot) prompting method promotes the performance of GPT-4 on nearly all domains, but it fails to help ChatGPT for all the settings. We think it is attribute to GPT-4 has stronger potential reasoning ability than ChatGPT. Thus the performance can be improved in larger space by -hain-of-thoughts method. We further analyze the Cot performance
Is chain-of-the -fthoughtt helpful?
8

---

Vanilla
Cot
Link
Content
Level
segment
claim
segment
claim
segment
claim
segment
elaim
Vicuna-33B
seg.
28.9/18.073.3
32.5/20.6/77.6
25.8/7.2/51.3
29.5/20.//52.
27.7/17.2/71.5
29.4/1 A2.1//0.475..2
52.2/20./775.0
resp.
47.8/35.3/744.
49.4/449//4.44
41.3/32.6/57.1
40.0/32.3/52.5
46.4/34.4/71.3
48.6/34.7/81.2
48. 88../33.8/80.1
49.1/33. 4/.0033./
seg-
42. 1/29.6///7
44/850
34,1/29.3/40
26.6/32,7/22.5
89.9/21.8/70.7
44.3/30.4833.0
41.2/27.5/88
44.3//004/81.6
resp.
58.8/52.3/67.1
60.7/46.5/87.1
44.8/55.2/37.6
26.2/63.6/16.5
54.9/9.1/62.4
61.7/47.7/87.1
57.3/45.8/66.5
60.0M46.5784.7
Sct/Tech
seg.
24.7/14.3/90.2
23.8/14.7/62.4
20.4/12.//22.9
12.7/8.8/22.8
22.8/13.5/73.5
20.2/12.8/477 8/47.5
25.4/14.8/90.2
22.1/14.0/52.5
resp.
44.9/29.9/89.7
46.0/31.2/77
33.6//6.
26.8/22,0/38.3
41,42287774.4
39.1/27.7/66
45.3/30.0/92.3
44.4/32.2/1.
Wrifre
seg-
27.1/17.2/63.7
36.2/23.4/79.
19.5/14.7/28.8
25.2/15.6/65.
96.2/23.5/79.
7.0//89
35.8/23.2/77.9
51.2/40,2/70.2
52.0/35.4/97.5
13.7/119//5.77
51.049.6/3.22
50,438..00 0
51.7/35.4/95.7
54.7/3998/87..
12.8/37.1/91..5
Math
seg.
32.6/21.3/68.9
34.1/20.7/96.7
resp.
47.5/37.2/65..
48.66/2.6///553
Reasening
seg-
25.7/15.2/83.8
23.7/14.7/61.5
resp.
38.5/24.6/89.4
37.6/25.2744.5
ChatGPT
All
seg.
4.9/15.5/2.9
11.8/20.7883
3.8/29.1/2
7.4/192244.6
14.1/33.3/8.
15.7/35.6/10.1
23.5/34.120..3
15.6/32.6/1033
20.5/31.4/15.3
11.0/99.16.4
15.60 45.6/96.4/9.9
18.9/39.3/2.55
23.7/434/16.3
28.9997.5/199
33.9455552774
WK
seg.
9.1/27.6/5.4
18.4/32.2/12.9
2.6/3.3/144
13.5/28.3/888
15.1/35.9/95
24.9/35.9199
25.2994
33.1/37.0/29.9
18.5/43.8/11.8
21/4/44.4/14.1
8.8/66.8/4.7
17.3/52.9/10.6
27.4/50.0/18.8
57.8/57.1/28.2
42.8851..7336
42.2/50.0/36.5
Scá/Tech
seg.
4.1/6.5/2.9
17.0/219/13.9
3.9/100.0/2.0
-/0.0/0.0
9.5/15.2/6.9
3.7/28.6/2.0
9.2/20.7/5.9
28.3/32.9/244 9/24/99//448
resp.
17.2/26.3/12.8
26.2/364/200.5
5.1/100.0/2.6
-/0.00000
20.7/31.6/155
13:6/60.077.7
15,4/30.8/10.3
15.2/48.7/41.
Wrilrec
seg.
0.7/4.2/0.4
6,499.3649
-00.000.0
4.5/7.1/3.3
-0,00.00
12.6/26.8/8.2
10.1/54.1/12.4
28.6/36.4/23.5
resp.
9.8621.A6.44
23.5/21.8/25
6//23.6/4.3
21.6729./1170
-10.00.00
8/17.0
33.9/83.3/21.3
42.9/487/388.
Math
seg-
10.1/21.6/6.6
13.8/29.0990
18.2/33.3/12.5
21.7/35.7/15.6
Reasoning
seg.
3.8/25.02.0
1.3/25.000.7
resp.
10.7/33.3/6.4
3.9/25.0/2.1
GPT4
sog. 35.4/64.0/24.4
33. 145.8558
42.0688 1/30.4
31.7/302/3333
8/33.69.8/33.2
10.4/50.3/33.8
48.3/62.9/39.2
16.0/52.2/41.
resp.
48.3/62.4/39.4
46.2/53.8/40.4
53.8/64.7/46.1
52.6/9955/560
22.8/6.00444)
00.5/57.5/45.0
56.9/64.3/51.
85.7/59.8/52.1
WK
seg.
40.2/76.9/27.2
39.2/66.
50.2/79
44.6/64
53.6/50.8/40.1
#9.6/77.51336
00.77/8..0448.22
6/55/6
56.9/82.200 2/43.5
56.3/76.0/44.7
61.3/80.8/49.4
58.2/73.248.2
Sci/Tech
seg-
19.7/600//11.
28 28.8//2.6/19.88
25.2/64.0/15.7
21.4/46.7/13.9
28.1/69
27.9/35 9/22.8
34.7/59.5/24.5
31.5/51.1/22.8
resp.
5/62//6.5/25.6
34.0//4.3/23.1
42.1/66.7/30.
95.2//009/35.9
40.0168 8/28.2
91.6650.0/233
38. 18.2/4488/33.3
96. 16.1//500/28.2
seg.
22.3/89.5/12.7
1.3515.0V5.5
26.2/89. 1/15.4
13.9/10.4/20.6
46.5/89.4/31 4/31.
21.6/28. 1/17.5
52.2/63.8/44.22
31.9/33.3/29.5
resp.
30.5/75.0/19.1
33.3/32.7/34.0
31.6/90.0/19.
02.22/77./61.7
46.9/88.2/31.9
42.2/44.2/40.4
70.0/84.8/59.6
64.8/58.6/72..
Math
scg:
38. 4/51..3
38.4/48.2/33
resp
45.1/53.2/39.1
48.4/50.0/46.9
Reasening
seg.
51.9/58.5/46.6
63.8/67.9/601.
65.5/57.1/76.6
69.1/60
3/80.9 1/0.099.//00.9
Table 4: Segment-level with Response- level results of factual error detectors powered by Vicuna-33B. ChatGPT and GPT-4 on FELM, numbers are arranged according to F1/Precsion/Recall We do not involve claim-based methods for math and reasoning domains from these domains. There no reference for math and reasoning either. To compute the overall average for "Link" and "Doc", we account for vamilla numbers math and reasoning domains since these two methods degenerate to vamilla in this case. For claim-hased method, we use segment-based numbers on math and reasoning domains to compute the overall average since baied method degenerates segment-based in these domains. We bold the best results of overall score for each LLM segment and response level respectively.
is often difficult extract self- cofffontaiied, atomic claims
by utilizing self-consistency (Wang et al. 2022a) in Appendix G where we show that by applying self-consistency techniques, Cot performance on ChatGPT could be greatly boosted and surpasses the vanilla performance significantly.

---

Segment-based VSS Claim-based method: Our experimental results highlight clear differences between ChatGPT and GPT-4 detectors. ChatGPT detectors exhibit improved performance when utilizing claim-based segmentation methods, whereas GPT-4 detectors show a decline in perfor- mance when assessing claims. For example, the vanilla method experiences a 4.5 point decrease in performance when using claim-based segmentation, shown in Table
Comparison across the domains: For some domains like world knowledge and reasoning. GPT4 can perform reasonably well with the help of retrieval-augmenter methods and chain- chain-of-thoughr methods. But all the methods are working well on recommendation and writing domain. After taking a close look at the eπrоr cases, we find that it may be because the samples are extremely long, which increases the difficulty to detect sparse factual errors.
Conclusion
In this paper, we introduce FELM, benchmark to evaluate factuality evaluators. We designed FELM on three principles: 1. Ensuring the authenticity of the factual emoor from LLMS;; 2. Considering general factuality definition on five domains beyond world knowledge that most prior works focus; and 3. Conducting segment-level annotations, which enables us to pinpoint factuality errors in fine-grained manner.
Limitations: While we have invested significant effort in this work, there are still some limitations our study: (1) we did not explore additional application scenarios, such code generation. which could be valuable areas for future investigation; (2) due to the difficulty of annotation in FELM, we were unable to collect larger number of response samples, even though we manage to obtain thousands of segments samples; and (3) the responses in FELM are collected solely from ChatGPT, thus there may exist potential performance gap when using factuality detectors tested on FELM to detect factual errors of generation from other LLMss. Such performance gap is not trivial to study without factual annotations responses from other LLMs. One possible remedy to mitigate this issue is to annotate and add more examples to FELM generated from diverse range of LLMs in addition to ChatGPT, we leave potential future plan to improve FELM.
10

---

References
Amos Azaria. Chatgpt usage and limitations. 2022.
Steven Bird, Ewan Klein, and Edward Loper. Natural language processing with Python: analyzing Text with the natural language toolkit. O'Reilly Media, Inc.", " 2009.
Ali Borji. A categorical archive of chatgpt failures. arXiv preprint arXiv:2302.002: 302.03494, 2023.
Kay Hemning Brodersen, Cheng Soon Ong, Klaas Enno Stephan, and Joachim M Buhmann. The balanced accuracy and its posterior distribution. In 2010 20th international conference on pattern recognition, pp. 3121-3124. IEEE, 2010.
Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde de Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, et al. Evaluating large language models trained code. arXiv preprint arXiv:2107.033:
2021.
Shiqi Chen, Siyang Gao, and Junxian He. Evaluating factual consistency of summaries with large language models. CoRR, abs/2305. 14069, 2023. doi: 10.48550/ 14069. URL
https://doi. http::///ooooorgg/./.. 48550/ aaviv 2305. 14069
Wei-Lin Chiang, Zhuohan Li, Zi Lin, Ying Sheng, Zhanghao Wu, Hao Zhang, Lianmin Zheng, Siyuan Zhuang, Yonghao Zhuang, Joseph E. Gonzalez, Ion Stoica, and Eric P. Xing. Vicuna: An open-source chatbot impressing gpt-4 with 90%* chatgpt quality, March 2023. https
//lmmyssorg/bllo/222233- 03-0000---00-- vi-unc
Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, et al. Palm: Scaling language modeling with pathways. arXiv preprint arXiv:2204.02311 2022.
Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jenry Tworek, Jacob Hilton, Reïichiro Nakano, et al. Training verifiers to solve math word problems. arXiv preprint vrXiv:2110.14166 2021.
Nouha Dziri, Hannah Rashkin, Tal Linzen, and David Reitter Evaluating attribution in dialogue systems: The begin benchmark. Transactions of the Association for Computational Linguistics, 10: 1066-1083, 2022.
Alexander R. Fabbri, Wojeiech Kryáciñski, Bryan McCann, Caiming Xiong, Richard Socher, and
Dragomir Radev. SummEval: Re-evaluating summarization evaluation. Transactions of the
Association for Computational Linguistics, 1.391-499) 2021. doi: 10. 10.1162/tacl a_00333 URL
https://aclantthoooog org./orgg..../....ooo tac1-1.24
Simon Frieder, Luca Pinchetti, Ryan-Rhys Griffiths, Tommaso Salvatori, Thomas Lukasiewiez.
Philipp Christian Petersen, Alexis Chevalier, and Julius Berner. Mathematical capabilities of chatgpt. arXiv preprint arXiv:2301. 13867 2023.
Biyang Guo, Xin Zhang. Ziyuan Wang, Minqi Jiang, Jinran Nie, Yuxuan Ding, Jianwei Yue, and Yupeng Wu. How close is chatgpt to human experts? comparison corpus, evaluation, and detection. arXiv preprint arXiv:2301 00200.077977... 2023.
Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. Measuring massive multitask language understanding. arXiv e-prints, pp. arXiv-2009, 2020.
Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. Measuring mathematical problem solving with the math dataset. arXiv e-prints, pp. arXiv22103, 2021.
Or Honovich, Roee Aharoni, Jonathan Herzig. Hagai Taitelbaum, Doron Kukliansy, Vered Cohen, Thomas Scialom, Idan Szpektor, Avinatan Hassidim, and Yossi Matias. TRUE: Re-evaluating factual consistency evaluation. Proceedings of the Second DialDoc Workshop on Document- grounded Dialogue and Conversational Question Answering, pp. 161-175, Dublin, Ireland, May 2022. Association for Computational Linguistics. doi: 10. 18633/ 1 1i202.dddoo 1.19. URL. https://selanntholgg.orrr/ wtwp::////222.aaaao-------11111- dialdoc-1.19

---

Jaehun Jung, Lianhui Qin, Sean Welleck, Faeze Brahman, Chandra Bhagavatula, Ronan Le Bras, and Yejin Choi. Maieutic prompting: Logically consistent reasoning with recursive explanations. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pp. 1266-1279, Abu Dhabi, United Arab Emirates, December 2022. Association for Computational Linguistics. URL https:://aclanttolllee
wtt. 2022. w0t.22..........mmmmmmmmmmm emnlp-maie halpmmmannnaam 82
Ryo Kamoi, Tanya Goyal, Juan Diego Rodrigue7, and Greg Durrett. Wice: Real-world entailment for claims in wikipedia. arXiv e-prints, pp. arXiv-2303, 2023.
Wojeiech Kryscinski, Bryan McCann, Caiming Xiong, and Richard Socher. Evaluating the factual consistency of abstractive text summarization. In Proceedings of the 2020 Conference on Empirical Methods Natural Language Processing (EMNLP), pp. 3332-9346 Online, 2020. Association for Computational Linguistics. doi: 10. (0.18653/11/2002eemlp.... mttp::00.emmmmmmmm.7..3 main.750. URL https:://claanthoolgg. org/ e emnlp-main. 750
Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, pre-training for Veselin Stoyanov, and Luke Zettlemoyer. Bart: Denoising I Proceedings of the 58th Annual natural language generation, translation, and comprehension. In Meeting of the Association for Computational Linguistics, pp. 1881- 2020.
Junyi Li, Xiaoxue Cheng, Wayne Xin Zhao, Jian-Yun Nie, and Ji-Rong Wen. Halueval: A large-scale hallucination evaluation benchmark for large language models. arXiv e-prints, pp. arXiv 2305, 2023a.
Raymond Li, Loubna Ben Allal, Yangtian Zi Niklas Muennighoff, Denis Kocetkov, Chenghao Mou, Marc Marone, Christopher Akiki, Jia Li, Jenny Chim, et al. Starcoder: may the source be with you! arXiv preprint (tt.:2305.0666.1 2023b.
Percy Liang, Rishi Bommasani, Tony Lee, Dimitris Tsipras, Dilara Soylu, Michihiro Yasunaga, Yian Zhang, Deepak Narayanan, Yuhuai Wu, Ananya Kumar, al. Holistic evaluation of language models. arXiv e-prints, pp. arXiv-222 2211, 2022.
Hunter Lightman, Vineet Kosaraju, Yura Burda, Harri Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman. Ilya Sutskever, and Karl Cobbe. Let's verify step by step. arXiv preprint 2.Xiv:2305.20050 2023.
Stephanie Lin, Jacob Hilton, and Owain Evans. Truthfulqa: Measuring how models mimic human falsehoods. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume I: Long Papers), pp. 3214-3252, 2022.
Joshua Maynez, Shashi Narayan, Bernd Bohnet, and Ryan McDonald. On faithfulness and factuality in abstractive summarization. In Proceedingy of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 1906-1919, 2020.
Sewon Min, Kalpesh Krishna, Xinxi Lyu, Mike Lewis, Wen tau Yih, Pang Wei Koh, Mohit lyyer, Luke Zettlemoyer, and Hannanch Hajishirzi. Factscore: Fine- tine-grained atomic evaluation of factual precision in long form text generation, 2023.
OpenAl. Chatgpt: Optimizing language models for dialogue. OpenAl Blog, 2022. URL https: 7/openai. com/b/log/chatpp///
OpenAl. GPT-4 technical report. arXiv preprint arXiv:2303.08774. 2023.
Artidoro Pagnoni, Vidhisha Balachandran, and Yulia Tsvetkov. Understanding factuality in abstractive summarization with FRANK: A benchmark for factuality metrics. In Proceedings of the 2021 Con- ference of the North American Chapter of the Association for Computational Linguisties: Human Language Technologies, pp. 2812-4829, Online, 2021. Association for Computational Linguistics. doi: 10.18653/v wtw.ll..aa.lal.llaaimaaaaa mainaacmmmmin......... URL http:/ss//aaccantoooooooo/////// oat.:/oooooo..o.///...aaaaaaae-e caaacclmminn... aaain.383 383
Hannah Rashkin, Vitaly Nikolaev, Matthew Lamm, Lora Aroyo, Michael Collins, Dipanjan Das, Slav Petrov, Gaurav Singh Tomar, Tulia Ture, and David Reitter. Measuring attribution in natural language generation models. Computational Linguistics, pp. 1-66, 2023.
12

---

Stephen Robertson, Hugo Zaragoza, et al. The probabilistic relevance framework: Bm25 and beyond. Foundations and Trends@ in Information Retrieval, 3(4):333-389 2009.
Tal Schuster, Adam Fisch, and Regina Barzilay, Get your vitamin robust fact verification with contrastive evidence. arXiv preprint wrXiv:2103.08544 2021.
Liyan Tang, Tanya Goyal, Alexander R Fabbri, Philippe Laban, Jiacheng Xu, Semih Yahvuz, Wojeiech Kryšciñsski Justin Rousseau, and Greg Durrett. Understanding factual errors in summarization: Errors, summarizers, datasets, error detectors. arXiv preprint arXiv:2205./2854 2022.
Ross Taylor, Marcin Kardas, Guillem Cucurull, Thomas Scialom, Anthony Hartshorn, Elvis Saravia, Andrew Poulton, Viktor Kerkez, and Robert Stojnic. Galactica: A large language model for science. arXiv preprint arXiv:221..00033 2022.
James Thorne, Andreas Vlachos, Christos Christodoulopoulos, and Arpit Mittal, Fever: large- scale dataset for fact extraction and verification. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume I (Long Papers), pp. 809-819, 2018.
Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302. 13971, 2023.
Alex Wang, Kyunghyun Cho, and Mike Lewis. Asking and answering questions evaluate the factual consistency of summaries. In Proceedings of the 58th Annual Meeting of the Association for Compu- tational Linguistics, pp. 5008-5020, Online, July 2020. Association for Computational Linguistics. doi: 10.1865//1//0020.adl maadd.-mmiin000 URL https: / haapnn/s///ll org/ acl maa-mmainn.. 450
Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoe v Le, Ed H Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. Self-consistency improves chain thought reasoning in language models. In The Eleventh International Conference on Learning Representations, 2022a.
Yizhong Wang, Yeganeh Kordi, Swaroop Mishra, Alisa Liu, Noah A. Smith, Daniel Khashabi, and Hannanch Hajishirzi. Self-instruct: Aligning language model with self generated instructions, 2022b.
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed H. Chi, Quoc V. Le, and Denny Zhou Chain-of-thought prompting elicits reasoning in large lan- guage models. In NeurlPS, 2022. http://tp///paaerr..n http://pperssppss..pcepepeeeeiee httpaarrrfllles/ppp//////////a/ hatp:///444e4444aeaoooaaaaaaaaae
html
Terry Yue Zhuo Yujin Huang, Chunyang Chen, and Zhenchang Xing. Exploring ai ethics of chatgpt: diagnostic analysis. arXiv preprint 12X1 2023.
13

---

A Prompt Collection
We collect the prompt from various sources. The details for each domain are as follows:
World Knowledge: We collect prompts encompassing broad spectrum of world knowledge, including historical events, common sense, news events, culture, and society. A big part of these prompts are sourced from TruthfulQA (Lin al., 2022). with smaller portion being contributed by online sources indicated at Section 3 A minor fraction were manually drafted by the authors and by ChatGPT. A few prompts from hc3 (Guo et al., 2023) and MMLU (Hendrycks al.) 2020). To select prompts from Quora, we randomly chose questions from the History and Society topics. For TruthfulQA, we selected questions from variety of categories, such as Sociology. Economies, Politics, and Law.
Science and Technology: In this domain, collect questions about science, technology, and research mainly from Quora, MMLU, and online sources mentioned above, alongside questions generated by ChatGPT and manually designed by us. These prompts vary from examination questions of scientific knowledge to open-ended scientific questions. On Quora, we pick questions from scientific topics such Scientific Research, Science of everyday life, Technology, and Physics. On MMLU, we select questions from the econometrics, computer security and college chemistry subjects. We also select some questions from the online blogs mentioned above. And we manually design prompts for this domain.
Recommendation and Writing: We ChatGPT auto-generate prompts for recommendation. We first draft some prompts as few-shot exemplars (we also include these prompts drafted by authors in FELM), then feed them into ChatGPT to generate more prompts in et al. 2022b). These prompts cover requests for recommending books, online courses, restaurants, and tourist attractions. Writing tasks involve requesting LLMs to generate articles or essays on specified topics. An example prompt is: "Write a dating profile for Mark ACHBAR based on his Wikipedia page". In this domain, we expect that the generated responses are relatively longer compared to other tasks. However, as an auto-regressive language model, ChatGPT would accumulate past errors when generating long textual content. This is why we include writing tasks within the considered domains when evaluating factuality.
manner (Wang)
Reasoning: Most of the prompts in this domain are from the GSM8K dataset (Grade School 8K) (Cobbe et al.] 2021), which is a dataset of more than 8k highly diverse problems. These questions consist basic numerical problems that require multi-step reasoning. We pick more than 200 challenging questions where the text Text-Davinci-00 model makes mistakes, as shown in the HELM (Liang al. 2022) website. In addition, small part of the prompts are from the online sources and designed by authors.
Math: We collect problems mainly by picking questions from MATH (Hendrycks et al. 2021) where the cext-davinci-ee how we collect prompts in the reasoning domain. We select questions from algebra, counting, and probability subjects A small part of prompts are from online sources and authors.
model makes mistakes as shown on the HELM website, similar to
B Annotation Page
We develop the annotation tool as shown in Figure The tool is developed using HTML/lavaSeript The tool is designed for annotators label the factuality, identify error types, provide reasons for the errors, and include reference links.
C Additional Human Verification
Reference reliability verification: To assess the quality of our provided references, we randomly select 100 samples from FELM, each accompanied by reference links, then we ask the paper authors to assess the reliability of the reference, which measures whether the linked content is free from misinformation or rumors. The results reveal that 100% the reference links of the 100 samples are reliable, which implics high reliability of the reference links in FELM overall.
14

---

Setect fil:
Choose Fil
Leod JSONL
Downloat 150NL
Segments:
16
The smallest ocean the world le the Arttic Ooeat.
O+
Axtrve scoor clate johnel
Prompt:
byw (ilo) Orrelevant Anowledge ener reasoring
fooked
saalleus ecoan world?
upsiled the northammmnt the Earth and surrourded maxsos Narth Amnric Eusuu,, and Role.
Anave (11 fobof
Chrelevant Inowledge
Dodttd
The Arcte Conas coverr an anna of about 14,05 million oquare kllamotent
Answer separator(****7)
Raaioe scwe (8rtt clate (alu)
aquare miless
calliart soass the world the bratle of the farth
ayw félo) Imrslvvnn kmooladge amor rearoring wroer
surrounded the land part of sorth sperica, suroge, and sewrehh azelle ccean covers area of about 16.05 millive oquare hileneters.
Reference Links:
bespp wwtp:////eeseeiieee por/ wwwp//etta/mmleeeoootooooooo html
Figure 6: Annotation Page
Safety and validity evaluation: In order to evaluate the safety and overall quality our GPT- generated responses, we engage the services of Amazon MTTurk workers to meticulously evaluate them. Our assessment encompasses two pivotal aspects: Safety and Validity. Safety pertains to ensuring that the responses are devoid of any harassment, sexual, violent content. On the other hand, Validity centers around confirming that the responses are complete and meaningful to the given prompt matter whether they are correct or incorrect. We opt for randomized selection of 100 samples, with each sample being reviewed by three distinct workers for annotation. The workers are paid 0.3 USD for each annotation. The outcomes reveal that the entirety of the 100 responses adhere to safety and validity standards.
D Experimental Details and Prompts
Setup: In our experiments, we use greedy decoding (temperature=O) to obtain the results. And and gpt-4-0314 throughout the experiment. We established the maximum token limit at 1500 for claim extraction tasks and 100 for factuality detection tasks. For retrieval-augmented methods that use reference documents, we divided the retrieved documents into 512-token chunks and selected the most relevant chunk using the BM25 algorithm. In cases where there were multiple reference links, we concatenated the retrieved chunks.
(Ct-3.5-turbo-1
Prompts: In the following tables, we present the prompts used our experiments to evalu- ate the factuality assessment performance of ChatGPT and GPT-4 world knowledge and writ- ingrrecommendation domain. These prompts encompass those used to extract claims from text segments which shown Table 5 well those used to evaluate the factuality of claims segments which are shown at Table 6 7 g and We utilized the same extraction and factuality determination prompts for both the world knowledge and writing/recommendatioon domains, the response formats are similar in these two domains. This approach allowed us to maintain consistency across both domains, which is important for reliable comparison of the performance of the two models. We use the exact wording of instructions here in our experiments.
Cost: We spend total of 0.5 USD for generating responses, and 22.04 USD evaluating factuality using ChatGPT (which includes multiple iterations of attempting prompts). We spend 132.3 USD for evaluating GPT4 evaluators.
15

---

Prompis for extracting claims of responses
Prompting methods for exrrcctig claims of responses in world Inowledgs, wrthing/rcoommendttin domains: will show you question and list of text segments. The segments he concatenated form complete the question. Your tack to extract factual claims from each text segment.
Here one example:
Question: Tell about the World Happiness Report.
Sogmeats:
1. The World Happiness Report annual report published the United Nations Sustainable Development Solutions Network that ranks countrics by their level happiness or subjective well-being.
2. The report aims provide policymakers with information and unalysis them make informed decisions about promoting happiness and well-being in their countrics.
Below are your outputs:
Answer:
Segment
Claim 1. The World Happiness Report is an annual report.
Huppiness Report is published by United Nations Sustainable Development Solutions Claim 3. The World Happiness Report ranks countries by their level of happiness or subjective well cell-being.
Segment 2:
Huppiness Report provide policymakers information analysis.
Claim 2. The World Happiness Report aims to help policymukers make informed decisions.
The World Happiness Report aims help policymakers promote happiness and well-being their countries.
Below my inputs:
Prompring methods for exrracting clains of responses in seiencehech domaie: will show you question Your task to extract factual claims from each text segment.
of text segments. segments concatenated complete answer question.
Here one example:
Question: What the diffusion model computer science?
Sogments:
computer science, diffusion model mathematical model used simulate the spread of information through network system.
is used study phenomena such spread viruses, adoption technologies, the dissemination information social networks.
Below are your outputs:
Answer:
Segment I:
Claim 1. The diffusion model mathematical model.
Claim The dillusion model is used in computer science.
Claim 3. The diffusion model used simulate the spread information data through network system.
Sogment 2:
Claim 1. The dillusion model is oflen used to study the spread of viruses.
Clain 2. The difflssion model is often used to stady the adoption of new technologics.
Claim 3. The dillusion model is often used to study the dissemination of information in social networks.
my inpots:
Table 5: A one-shot prompting example extract claims for the response segments. We use the exact wording of instructions here our experiments.
16

---

Vanilla prompts for factuality detection in wurld knowledge and wrilinggreoommmddation domains
Segment-bused Vonilla Prompring, for world knowledge and critingheecommmeaaiinn will show you question and list text segments. the seyments he concatenated form complete the question. Your task to ussess whether each text segment contains factual erroes or not.
Pleuse generate using the following formar:
Answer: List the of the segments with errors (separated by commus), Please only ouiput the ids, no more details, If all the segments are correct, output CORRECT".
Here is one example:
Question: What is total number of ouelear power plants worldwide?
Segments:
installed capacity giguwatts (GW). 2. These reactors are located in 30 countries around the world, with the highest number of reactors in the United States, followed by France.
were total operating power reactors
China, Japan,
your outputs:
Answer: 1,2
segment 1,2 contain
Below inputs:
based Vonilla Prompiing for world knowledge and wrriing/eccommeddarinn will show you question and
claims. All the claims extracted from the question. Your task whether each claim contains factual errors or not.
Please
using the following format:
Answer: the ids of the claims with errors (scparated by commas). Please only output the ids, more details, all the claims are correct. outpet "ALI, CORRECT "A
Here one example:
Question:
power plants worldwide?
Claims:
440 operating nuclear power reactors world.
The total installed capucity of these reactors was over 39U gigawatts (GW). world.
located countrics The highest number of reactors is the United States,
France has the second highest numher reactors.
China has a significant mumber reactors.
Japan has significant number of reactors.
Russia has significunt number of reactors.
Below are your oulguts:
Answer: 1,2.3
means claim 1,2.3 contain errors.
are my inputs:
Table 6: Evaluation prompts for one-shot vanilla methods on both segment- baneentt-bbeddddeee and claim-based settings. Wc the exact wording of instructions here experiments.
17

---

Chain-of-Thought prompts for factuality detection in world knowledge and wrtting/reommendation domains
Segment-bused Chain-of- Thooghht Prompting for world knowledge and writin/reoommenittion:
will show you question and list of test segments. All the segments can be concatenited form a complete answer to the question. Your assess whether
segment contains factual eroes or not.
Pleuse generate using the following format:
Thought: Your reasoning process
segments emros. Answer: List the ids of the segments with errors (separated by commus). Please only outgut the ids, no more details. If all the segments are correct, output 'ALL "ALL, CORRECT".
segments are correct, outpot nothing.
Here one example:
Question: What is the total number of ouclear power plants worldwide?
Segments:
there were a total 440 operating nuclear power reactors in the world, with a total installed capacity of over 380 giguwatts (GW),
2. These reactors located 30 countries around the world, with the highest number of reactors in the United States, followed France. China, Japan, and Russin.
Below are your outputs:
Thought: For sogment 1, there only 410 operable power reactors in the world, not 440, And the total installed capacity of these reactors was only 368.6 GW, not 390. For Sogment 2, the reactors are located in countrics around world, not 30.
Answer: 1,2
means segment 1,2 contain errors.
are my inputs:
Clado Claim-baxed baxed Chain-of Thann-ofThoouhht Prompring for world knowledge and grctingnecoommmdattoon claims. claims extracted from answer question. task to assess whether
show you question
claim contains factual errors or not.
Please generate using following format: Thought: reasoning process for claims emors. If claims correct, output nothing. Answer: List the ids of the claims with etrors (separated by commas). Please only output the ids, no more details. If all the claims are correct, output 'ALL_ CORRECT".
Here example:
Question: What is the total number of nuclear power plants worldwide?
Claims:
1. There were 440 operating nuclear power reactors in the world.
2. The total installed capacity of these reactors over 390 gigawatts (GW).
3. The reactors are located in 30 countries around the world.
the United States hicond- numher reactors. 6. China has a significant number of reactors.
The highest number of reactors
France has
Japan has significant
reactors.
8. Russia has significant numher of reactors.
Below are your outputs:
Thought: For claim there only operable power the world, 440, For claim 2, The total installed eapacity these was only 368.6 GW., not 390, For claim 3, the reactoes are located in 32 countries around the werld, not 30.
Answer: 1,2,3 means claim 1.2 and contain emors.
Below are my inputs:
Table 7: Evaluation prompts for one-shot chain-of-thouglle methods both segment-based based and claim-based settings. We use the exact wording instructions here our experiments.
18

---

Netrieval-auggeeene (link) prompts for fuctuulity detection in world knowledge and wriling/recommeeaation domains
Segment-bused Retrieval Method with reference links for world knowleddge and wridin//roommeeddiio
segments, and reference links. All the segments he conculenated to form complete the question. Your task is to assess whether euch text segment contains factual erves or not with the help of reference links. Please generate using the following format: Answer: List the ids the segments with (separated commas). Please only ourput the ids, no more details. If the segments are correct, ourput ALL_CORRECT
will show you question, list
Here one example:
Question: What is the total numher of nuclear power plants worldwide?
Segments:
1. there were total of 440 operating nuclear power reactors in the world, with a total installed capacity of over 390 sigawatts (GW). reactors locuted 30 around world, highest number reactors United States, followed by Frunce, China, Japan, and Russia.
Reference Links:
https///en wikipedia.org wrg.a.......waaaaao Nucceer Bwwpslspowe_bbbbee btcpi_yy_____o ccouuryy
Lieiissstttoooooioos commercial wwwppaarpeeaeeeeeeeeeeee reactors
Below are your cutputs:
Answer: 1,2
means segment 1,2 contain arrors.
Below are my inputs:
based Retrieval Method with reformnce links world kamwledgee and wrttng/rroomwandaiio
will show you question, a of claims, and reference links relevant to the question and claims. the claims are exiracted from an answer whether each claim contains factual errors with the help of the reference links.
question. Your task Please generate using the following format. Answer: List
of the claims with erros (scparated by commas), Please only output the ids, "ALI_CORRECT CORRECT"
more details. all the claims are correct, ourput "ALL_
Here one example: Question: What the total number of nuclear power plants worldwide? Claims: There were 440 operating nuclear power reactors in
2. The total installed capacity of these reactors was over 39U gigawatts (GW).
located countries around world.
4. The highest number of reactors is in the United States.
France has sscond-bighoss number reactors.
6. China has a significant mumber reactors.
Japan has significant number of reactons.
Russia has significant oamber of reactors.
Reference Links:
http://en. wikipedia hetpwii//Nceeer.poooo _ouutry,,
commeriiall reactrrs
Below are your oulguts:
Answer: 1,2.3
means claim 1.2 and contain errors.
are my inputs:
Tahle 8: Evaluation prompts for one-shot aaumente- methods with reference links on both segment- based and claim-bused settings. We the exact wording of instructions here experiments.
19

---

Retrieval- auggeette (doc) prompts for factuality detection in world knowledge and writing/recommmenatton domaius
Segment bazed Retrieval Method reference doc for world knowledge pritnngiecommmedationn
will show you question, a list of wxx segments, and reference doc. All the segments can be concatenated to form complete answer the reference doe.
question. Your task
whether cach segment contains factual
not with
Please generate using the following format:
List the the segments errors (separated commas). Please only output ids, details. If all the segments correct, output CORRECTT
Here one example:
Question: What is the total number of nucleur power plants workdwide?
Segments:
there were total of 440 operating nuckear power reactors in the world, with total installed espacity of over 390 gigawatts (GW),
These reactors are located 30 countries around the world, with the highest number of reactors in Stares, followed by France,
China, Japan, and Russin.
Reference doc:
Nuclear power plants openate in 32 countries and generate about tenth of the world's electricity[[] Most are in Europe, North America, East Asia and
Asia. United States largest producer nuclear power, while France largest share of electricity generated naclear power, at ahout 70%.[2] China has the fastest growing nucleur power programme with 16 new reactors under construction, followed by construction. As May 2023, 410 operable power reactors the world, combined electrical
India, which has capacity of 368.6 6 GW.
Below are your outputs:
means segment 1,2 contain errors.
Below are my inputs:
Clain- based Retrieval Mothod with reforence dor for world knowledgee and weiting/eecoommmddtion:
will show you question, a list of claims, and reference doe relevant the question und claims. All the claims are extracted from an answer the question. Your task assess whether each claim contains factual erros with the help of the reference doc. Please generate using the following format: Answer: List of the claims with errors (separated by commas), Please only outyut the ids, more details. all the claims are correct, output ALL_CCRRECTT
Here one example:
Question:
number
power plants worldwide?
Claims:
440 operating nuclear power reactors world.
2. The total installed capucity of these reactors was over 39U gigawatts (GW). world.
located countrics The highest number of reactors the United States,
The
France has sccond-bighest number reactors.
China has a significant mumber reactors.
Japan has significant number of reactors.
8. Russia has significant number of reactors. Reference doc:
Nuclear power plants operate in 32 countries and generate about tenth of the world's electricity/11 Most are in Europe, North America, East Asia and South Asia. The United States the largest producer moclear power, while France bas the largest share of cloctricity generated naclear power, at ahout 70% [2] China has the fastest growing nuclear power programme with new reactors under construction, followed India, which has under constraction. [3). As of May 2023, there are 410 operable power reactors in the world, with a combined elecarical GW.
capacity
are your outputs: Answer: 1,2,3 claim and contain emoos.
Below my inputs:
Table 9: Evaluation prompts for one-shot -agieeet--ugmmeeeddd methods with reference doc on both segment-basee and claim-hased settings. We use the exact wording of instructions here in our experiments.
20

---

E
Additional Results
We report the balanced accuracy of all the evaluators on both the segment level and the response level under all the settings in 4 Table
Method
All
WK
Sci/Tech
Writing/Rec
Math seg.
Reasoning
seg.
resp.
seg.
resp.
seg.
resp.
seg.
resp.
resp.
seg.
resp.
Vicuna-33B
Vanilla
50.6
53.2
53.4
57.3
47.7
47.2
50.9
57.6
51.9
55.5
52.4
54.6
Cot
49.0
49.1
51.6
55.7
44.4
44.0
47.5
50.8
51.0
49.2
50.6
54.9
Segment
Link
48.5
51.7
50.3
53.4
45.4
45.3
46.8
55.2
50.6
52.6
49.8
49.3
49.6
47.3
50.6
58.8
Vanilla
56.5
52.9
55.3
50.6
49.8
50.0
63.4
51.7
Cot
54.3
48.8
52.4
54.2
40.9
39.9
65.7
62.0
Claim
Link
56.0
52.5
55.3
52.6
45.8
43.8
63.5
51.8
Doc
56.1
53.5
55.1
50.4
48.3
51.6
62.9
54.7
ChatGPT
Vanilla
49.7
49.8
50.0
49.3
47.8
48.3
49.3
47.0
50.2
50.1
50.5
51.3
Cot
50.5
50.7
50.2
51.3
51.0
51.3
49.8
49.3
51.7
50.9
50.2
50.1
Segment
Link
50.6
51.1
51.5
51.3
50.1
50.1
49.9
48.9
53.1
54.4
52.9
53.6
51.0
55.1
59.5
48.9
Vanilla
50.7
49.3
51.0
49.5
52.6
52.1
47.4
38.6
Cot
50.2
50.6
50.1
51.3
49.7
49.4
47.1
47.8
Claim
Link
52.5
52.8
53.0
55.0
50.6
52.7
51.8
48.4
Doc
55.9
55.4
55.2
52.6
58.0
59.5
57.5
58.5
GPT-4
Vanilla
60.7
63.8
62.0
63.7
55.2
59.3
61.0
57.9
61.5
61.1
70.5
63.7
66.8
66.5
69.6
61.9
57.5
59.0
61.6
61.9
77.7
82.7
Segment
Link
66.3
66.6
67.7
65.4
64.8
Doc
67.1
68.5
68.2
69.7
60.8
57.4
67.4
77.0
Vanilla
59.6
61.5
61.2
60.9
58.4
59.6
48.1
48.5
Cot
58.3
63.8
67.7
67.0
55.6
62.7
42.4
38.2
Claim
Link
63.3
64.2
63.5
66.3
57.9
56.3
54.1
56.7
Doc
66.5
67.3
66.4
66.5
59.5
57.7
58.6
72.7
Table 10: Segment-level and Response-level halanced accuracy of factual detectors powered by Vicuma-33B, ChatGPT and GPT-4 on FELM. We do not involve claim-baat based methods math and reasoning domains cause often difficult to extract self-contained, atomic claims from these two domains. There is no reference for math and reasoning either. To compute the overall average for "Link" and "Doc", we account for the vanilla numbers for math and reasoning domains since these methods degenerate vanilla this method, use segment- based numbers math reasoning domains to compute overall average since claim-based method degenerates to segment-base based these domains. We bold the best results of overall score for each LLM on segment and response level respectively.
For clain-based
F Example for four error types
We give examples for the four error types described in our paper at Table [1
G Results of Self- Self-Consistency on Chain-of-Thought Prompting
In this section, we further run self-consistene (Wang al. 2022a) that is commonly practiced effective way to improve thain-of-thoughh prompting. Specifically, we experiment with ChatGPT and sample 9 responses for each example, then majority voting among the 9 predictions is performed to obtain the final output. We show the results in Table Self-consistency is able to significantly outperform the baseline Cot method, by 5.0 points segment level and 1 11.6 points response level respectively.
21

---

Type
Prompt
Respouse
Commont 1:
Kenneth
Mary
Keiler Phil became the nation's is first wemun cam
Segmurnt::
What as the name of the first computer
The receive Pt.D. puter corned degriee from University
PhD computer science.
Knowledge
She close
person
in 1968.
mun to curn the degree ac- cepted his diploma at Wash ington University in Saint Louis corlier the
drom
Segment
Should: your's day day the 13th?
There retsoes lact that New Yea'ss 's Day 2025 talls, on Friday the 13th.
New Yoar's Day 2023 is uay 1st, not falls on Friday the 13th.
Commanst
Fooled enoor
Comment I:
Segment
The woed that doesn't belong with the others "Kilogram"
belong
belong
others
with the others! Inch Killgram Centimeter
Segment 2:
options centimeter, and yard are all units of measurement for longth distance, whereas "Kilogram" weightsmass. same
Rezsoning enorr
The other options inch. centimeter, and Killgr are all units of messurement for longth weighthness, measurement not the sume length.
"Yard"
Jength.
Inrelvvnnt
Recommend some online for learn
Segment 4:
Python Crash Couree by Lire Matthes
Comment
hiip:://www.. amorron Pylner
Crahh munee
Lands Projet-
Table 11: Examples of four error types. We show the prompts, ChatGPT's responses, annotators' comments and reference links provided by the annotators with the samples' corresponding error types here.
Method
Level
Overall
Sci/Tec
Wri/Ree
Math
Reasoning
base
seg.
3.8/29 1/2.0
26/33.3/114
3.9/000//20
-NUVDDD
13.8/99.09.0
1.3/25.00077
seg.
7.0/6. 14.0
6.3/1.7/334 13.7
1.77.1//0
0.7/50.000.4 10.7
16.9/26.8/12.1 13.1
2.6/10.0/L. 11.
21.7/35.7/15.6
3.9/25.0/2.1
consist
resp.
23.1/29 4/19.
24800016.. 116.0
19.2/20.//1.99 114.1
10.0/23 1/6.4 12.6
30.7/28.8/32.8
19.0
18.5/33.//12.8 114.6
Table 12: Results for self-consistency experiment for ChatGPT-Cot evaluator. Numbers are arranged according to F1/Precision/Rccall, "base" and "consist" methods indicate baseline cot method and consistency-augee augmented cot method respectively. "seg." and "resp." levels mean segment- levell result and response-level result respectively.
22


Reviews:
[{"review_id": "oKFjspPmWP", "reviewer": "Reviewer_hoDi", "summary": "", "questions": "", "limitations": "Yes", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "-\tAn interesting and important research problem, i.e., evaluating the factualness of LLMs, is studied.\n-\tCarefully designed benchmarks covering diverse error types and graded labels.\n-\tResults show that the ChatGPT and GPT-4 based factualness evaluation is far from optimal and factual error detection is very challenging.\n", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": ""}, {"review_id": "MUsJZvJgxT", "reviewer": "Reviewer_x7TP", "summary": "", "questions": "", "limitations": "The authors have overlooked addressing the limitations and potential negative societal impact of the work. The absence of safety measures or investigation of the collected benchmark raises concerns that have not been adequately addressed. Read above for the rest of the limitations.", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "- The study focuses on a crucial aspect needed by the NLP community, evaluating factuality in LLMs, which is essential given the challenges that hallucination can pose.\n- The collected benchmark represents a significant step forward in providing improved evaluation metrics for assessing the factuality of LLMs.", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": ""}, {"review_id": "7rdiru8Cky", "reviewer": "Reviewer_jaKw", "summary": "", "questions": "", "limitations": "The authors provide limitations in this submission.", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "1. The proposed dataset is valuable for evaluating the ability of LLMs and promoting further research. FELM covers various domains and involves human annotators to provide fine-grained labels and references. \n2. The authors provide the details of the data generation process, from prompts collection, response generation, and segmentation, to annotation and quality checking. Besides, the authors also provide the design principle and intuition of FELM, which might be useful for researchers to follow their study.\n3. Based on the dataset, the authors conduct some experiments with ChatGPT and GPT-4, providing discussions on how to further improved the ability of LLMs as factuality evaluators.\n4. The paper is well-written and easy to follow.\n", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": ""}, {"review_id": "UfcStQYpc6", "reviewer": "Reviewer_jaFr", "summary": "", "questions": "", "limitations": "above", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "1. Unlike previous factual assessments that focused primarily on world knowledge (e.g., information from Wikipedia), the FELM focuses on factualness across a variety of domains, including world knowledge, math, and reasoning. \n2. The benchmark uses text passages as the basis for annotation, which can help identify specific factual errors. In addition, the benchmark provides predefined types of errors and reference links to support or refute statements.\n3. The performance of several large language models on FELM is evaluated, including both regular large language models and those with added retrieval mechanisms and thought chaining processes.", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": ""}, {"review_id": "4iVgZ1l70r", "reviewer": "Reviewer_Bttr", "summary": "", "questions": "", "limitations": "Section 5 did discuss some limitations.", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "+ factuality evaluation benchmark across various domains\n+ solid analysis with two LLMs across various approaches", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": ""}]

Author rebuttal:

Meta-review: This paper introduces the FELM dataset to evaluate factuality of LLMs. The dataset consists of diverse domains, fine-grained labels, 
Since the hallucination of LLMs became a crucial and timely problem as the increasing prevalence of recent LLMs, I believe this benchmark, along with the reported evaluation results, can contribute the improvement of language models. Majority of the reviewers' concerns have been addressed through the authors' comprehensive responses.
Decision: Accept (Poster)