FacTool: Factuality Detection in Generative AI -- A Tool Augmented Framework for Multi-Task and Multi-Domain Scenarios
Authors: Ethan Chern
Steffi Chern
Shiqi Chen
Weizhe Yuan
Kehua Feng
Chunting Zhou
Junxian He
Graham Neubig
Pengfei Liu
Venue: COLM 2025 (2025)
Keywords: factuality
llm
hallucination
TL;DR: We introduce FacTool, a tool augmented factuality detection framework that can effectively detect diverse factual errors generated by LLMs.

Abstract: The emergence of generative pre-trained models has facilitated the synthesis of high-quality text but has also posed challenges in identifying factual errors in the generated text. In particular: (1) A wider range of tasks now face an increasing risk of containing factual errors when handled by generative models. (2) The content generated by these models tends to be lengthy and lacks clearly defined granularity for individual facts. (3) There is a scarcity of explicit evidence available during the process of fact checking. With the above challenges in mind, in this paper, we propose FacTool, a tool augmented multi-task and multi-domain framework for detecting factual errors of texts generated by large language models (e.g., ChatGPT). Experiments on four different tasks (knowledge-based QA, code generation, mathematical reasoning, and scientific literature review) show the efficacy of the proposed method. We release the code of FacTool with ChatGPT plugin in https://github.com/GAIR-NLP/factool.

Published as a conference paper at COLM 2025
FACTOOL: Factuality Detection in Generative AI A Tool Augmented Framework for Multi-Task and Multi- Domain Scenarios
I-Chun Chern' 1.2, Steffi Chern', Shiqi Chen', Weizhe Yuan", 4 Kehua Feng",
Chunting Zhou², Junxian Her, Graham Neubig', Pengfei Liu 'Shanghai Jiao Tong University ?Generative AI Research Lab (GAIR)
OCity University of Hong Kong "New York University Meta Al
'hhe Hong Kong University of Science and Technology
CCarnegie Mellon University
Abstract
The emergence of generative pre-trained models has facilitated the syn- thesis of high-quality text but has also posed challenges in identifying factual errors in the generated text. In particular: (1) wider range of tasks now face an increasing risk of containing factual errors when handled by generative models. (2) The content generated by these models tends to be lengthy and lacks clearly defined granularity for individual facts. (3) There is a scarcity of explicit evidence available during the process fact checking. With the above challenges in mind, in this paper, we propose FACTOOL, tool augmented multi-task and multi-domain framework for detecting factual errors of texts generated by large language models (e-g., ChatGP'T). Experiments on four different tasks (knowledge-basee QA, code generation, mathematical reasoning, and scientific literature review) show the efficacy of the proposed method. We release the code of FACTOOL with NLP/factool.
ChatGPT plugin here: https: //gi thub.
1 Introduction
Generative artificial intelligence (AI) technology (OpenAI, 2023) consolidates various tasks in natural language processing (NLP) into a single sequence generation problem. This uni- fied architecture enables users to complete multiple tasks (e-g., question answering (Thoppi- lan et al., 2022), code generation (Chen et al., 2021), math problem solving (Lewkowycz et al., 2022), and scientific literature generation (Taylor et al., 2022)) through natural language interface (Liu et al., 2023) with both unprecedented performance (Bubeck et al., 2023) and interactivity.
However, at the same time, such generative paradigm also introduces unique challenges. Content that is au- tomatically generated can often exhibit inaccuracies or deviations from the truth due the limited capac- ity of large language models (L..Ms) (īi et al., 2023; Schulman, 2023). LLMs are susceptible to producing content that appears credible but factually incorrect or imprecise. This limitation restricts the application of generative AI in some high-stakes areas, such healthcare, finance, and law. Therefore, it is crucial to identify these errors systematically to improve the usefulness and reliability of the generated content.
Claims
Tool Queries
Quety
Generation
Claim
Tool
Prompt
Extraction
Querying
CharGiT
Response
Facrool
Evidence
Tools
Collection
Verification
100
5
Factuality
Evidence
Current literature on detecting and mitigating fac- tual errors generated by Al models focuses predomi- nantly on a single specific task, for example, retrieval-
Tool augmented frame- work for factuality detection.
Figure
* Corresponding author

---

Published as a conference paper at COLM 2025
Prostt
Knowlooggs-ooad Q4
Ratann sing oelaneg vnooddiied nanderr curteg frem up iadadie
Code Generation
Chian
The Ctd Teaterat De tie
Clin
[I) lock Davsry is CTO of Thiiee
(7) lack hunnly Soodie "witee in
LIM Repuee
Chisn
6di deiatng JasEEoo
Deney lk codtande Toitst
Ecradico
Esidence
reil
for raqale
Euratoe
fort nagojnil
toual to dadl
of ITTakte lack Bency cociaad
2) Whe be conner (80
Donny
Sance foot Oidee
lice (1.23 hurre SuC Univonal inleetiiing
List Yoecrmi will bocane...
Nive allen
Test Caues
(2) Oiiring sepunsett U) [ixig__sepeee
atag (equsce44)
ream rewiisitg
QAe
Exre Keutks
[1P171 Eeggeee 1234
(]0capend:
111013 2 23
Taitor
Clire lo Surnality
Repseas love Pacsualgg 6
Repesseleed Factally
Math Problem Soluing
Factool
Scientific Literature Review Writing
Prosutt
cack, packs Mark poad casl s45, Eso meay bosis of (hare Mart
Pronaee
Math Claase
Dicas the aplicaeea and inthiens of quaren conpating cring aee paper plane indak tike the aulith pablictice your
(T) ttne Ouazan Compusin Do
Chise
EJ M Respome
Mare onterd packs of tur cosr the our hous stook no
Eoractoo
Chise
Querkes
pipaaagaadd 12)
(Trs5- 15- 99)
(%) 0330033-10 -0 Io
Quary
LIM Requese
raper.
Savadie
(1) Quetter Compriing
Oueriee
Poskil pakicatass year 28
beyven
Quaay Cmoostion
(7t protoroosdys is, Tym-1))
corpaing pabetal fold
the teal areun puadity by Mari 45-135 15-- $30
(3 paaaaeeesa/ Ty 100
revelatintar
copeagepsr to gpiniatee and sreiskm
Conpuning the S150 er and bepond, aabes Jonn
divits cananing unoaad the
Scares
Teee (2) (P) Tiue
for one comoitee. Oez ful heve contibeted to the fold in
each hec
Rospaoooo
Houts
Soere
Diackerr ondicdd undeval boous of proza
boved john Prokill 12015
Recpass lovel lovel Factallty
Figure 2: Our proposed framework for factuality detection in four domains: KB-QA, code, math, and scientific.
augmented verification models for QA (Lewis et al., 2020), hallucination detection models for text summarization (Fabbri et al., 2022), and execution-based evaluation for code (Shi et al,, 2022). While these methods have proven successful within their respective areas, given the remarkable versafility of tasks and domains handled by LLMs, we argue that it is also important to have a more comprehensive factuality detection and verification framework that is similarly versatile.
Moreover, in existing literature, factuality detection is usually defined either (i) verifying the factuality of given claim, or (ii) checking if the provided evidence supports the claim. This definition is overly simplistic as it does not encompass interactions with LLMs like ChatGPT, where we often need to verify the factuality of long-form generation without explicit claims and evidence.
In this paper, we propose multi-task and multi-domain factuality detection framework, FACTOOL, aiming to detect factual errors in LLM-generated texts. We illustrate our frame- work in Fig. 1, where we connect the concept of "tool use" (Thoppilan et al., 2022; Gao et al., 2022b; Schick al., 2023) with "faciuality delection" and demonstrate that the ability to use tools in LLMs is crucial for factuality detection. Specifically, FACTOOL leverages various tools, including Google Search, Google Scholar, code interpreters, Python, to gather evidence about the factuality of the generated content. Moreover, FACTOOL employs the reasoning abilities of LLMs to assess the factuality of the content, given the gathered evidence. We develop benchmark and perform experiments across four tasks: knowledge-baad QA (KB-QA), code generation (code), math problem solving (math), and scientific literature review writing (scientific). In summary, our contributions are:
We revisit and extend the task of factuality detection to diverse domain and tasks beyond KB-QA, allowing better audit of modern LLMs.
We are the first to propose and connect the concept of "tool use" with "factuality de- tection", developing a unified and versatile framework for factuality detection across domains and tasks.
We use FACTOOL to evaluate the factuality of chatbots and find GPT-4 to exhibit the best factuality in most scenarios. Vicuna-13B (supervised fine-tuned chatbot) shows decent factuality in KB-QA but underperforms in more challenging scenarios including math, code, and scientific.
2

---

Published as a conference paper at COLM 2025
Response
Claim
Evidence
Scenario
Methods
Length
Generated by
Granalarity
Provided
Provided
Domain
Task
REVER-haeed
Human
Fact
Wikipedia
Fact Verification
FactCC
20.83
Synthetic
Sentence
Newswite
Summ. Factualityy
QACS-based
16.11
Model
Summary
Newswire
Summ. Factuality
WICE-based
Wikipedia
RARR
PaLM/LaMDDA
Fact
Wikipedia
QA
FActScore
ChutGPT/PPrrleeni
Fact
X
X
Wikipedin
QA (people biographies)
41.80
ChaiGPT
Fact
Wikipedia
QA
FACTOOL
67:13
30.37
ChatGPT
ChatCPT
Statement
Snippet
X
X
Math
Python
Math Problems
Code generation
ChaIGPT
Tuple
text
Revlew
Table 1: A comparison published approaches for factuality detection in terms generated responses and claims to be verified based collected evidence. "Scenario" represents which task and domain the corresponding approach has been justified. "Sci." represents "Scientific".
Related Works
Factuality Detection in NLP Factuality detection was topic of rigorous study even before the advent of generative AI. Existing works can be organized by their differences in terms of the "response" to be verified, the "claim" extracted from the response, and supporting "evidence". As illustrated in Tab. 1, the creation of the FEVER dataset (Thorne et al., 2018b) spawned models (Zhong et al., 2020; Krishna et al., 2022) that determine whether a given fine-grained claim made based on Wikipedia articles is correct. In this task setting, both the claim and related evidence are given. FactCC (Kryscinski et al., 2020) and QAGS-based models (Wang et al., 2020) adopted different task formulations to detect fuctual consistency, iee, given the evidence text, and the goal is to determine the generated summaries or summary sentences factually consistent with the given text.
Recent works adopt LLM-based methods for factuality detection. WICE-based meth- ods (Kamoi et al., 2023) verify if Wikipedia sentence is supported by provided evidence. RARR (Gao et al., 2022a) introduces an approach by directly prompting LLMs to generate queries, retrieve evidence, and determine factuality. FActScore (Min et al., 2023) offers fine-grained factuality evaluation for LLM-generated text. However, all WICE, RARR, and FActScore are limited to the KB-QA task and the Wikipedia domain. WICE-based methods require provided evidence, limiting their generality. RARR does not support fine-grained factuality and focuses on "text editing", different from our focus on "factuality detection." Additionally, RARR's complicated verification process, involving webpage parsing and relevant matching algorithms, limits its practical usage. FACTOOL offers a more intuitive agreement verification process by leveraging the strong reasoning capabilities of LLMs to directly feed LLMs with the collected evidence. FActScore is confined to its specific benchmark (people biographies), whereas FACTOOL supports diverse KB-QA benchmarks. FActScore also lacks a query generation module that generates relevant queries from atomic facts to query external databases, affecting evidence collection performance. In this work, we focus on factuality detection without explicit claims or evidence. We propose a unified framework capable of addressing the escalating challenge of hallucination in a variety of scenarios, including KB-QA, math, code, and scientific.
Tool use LLMs LLMs store limited knowledge within their parameters. To overcome this limitation, various tools have been introduced assist LLMs to further expand their capabilities. For example, Press et al. (2022); Komeili et al. (2022) gathered information from the Internet to enhance QA and dialog systems, respectively. Schick et al. (2023) trained model capable of interacting with five tools including calculator, a translation system, etc. Shen et al. (2023) introduced a framework that employs LLMs to connect various AI models from the ML communities to tackle AI tasks. Liang et al. (2023) proposed new Al ecosystem that connects LLMs with millions of existing APIs to accomplish tasks. In this work, explore tool use in LLMs for the task of factuality detection.
3

---

Published as a conference paper at COLM 2025
3 Revisiting Factuality in Generative Al
Tasks
Prompt (p)
Response (r)
Claim
Evidence
KB-QA
Question
I Long-form answer
Atomic component
searched results
Code Generation
Code Query
Executable code
Code snippet
Python library
Math Problems
Math problems
Math solution
Math calculation
Calculator
Sci. Lit Review
Scientific question
Long form review
Tuple (paper title, year, authors)
Google scholar
Table Factuality definition in different tasks.
"Sci. Lit Review" represents scientific
literature review.
3.1 Definition
Versatile Factuality In previous works, factuality has been defined as whether claim in text can be supported by evidence from a trustworthy knowledge base, such as Wikipedia for fact-checking (Thorne et al., 2018a), input documents for summarization (Kryscinski et al., 2020). In this paper, we extend this definition to whether the claims in generated signals (which could be text, code, mathematical expressions and so on) can be supported by evidence under specific rules. These rules may range from consistency with a Wikipedia- derived knowledge base to verification rules in Python library or operational rules in mathematics. By adopting this broader definition, we establish a unified framework for addressing factuality issues in generative Al beyond the textual domain.
Fine-grained Factuality Typically, one can ascertain the factuality of a generated signal (e-g., text) at various levels of granularity, including sentence and document level. A more granular assessment can be especially valuable as not only (1) enable users to pinpoint where inaccuracies occur (Liu et al., 2021) but also (2) functions as a reward model for developers to refine their generative systems (Lightman et al., 2023). Nevertheless, implementing fine-grained factuality detection challenging for two reasons: (1) specifying the desired granularity level unambiguously, and (2) extracting claims that accord with the predetermined granularity level. In this paper, we argue that the robust instruction- following ability and natural language interface of LLMs can be effectively utilized to address the challenge of defining and extracting fine-grained claims via claim definition-ban -based few-shot prompting. Additional details are in $4.1.
Structurally speaking, given a prompt (e.g., a query or instruction) and the corresponding model-generated response, the fine-grained factuality detection task involves the following concepts:
Prompt (p): query or instruction that users provide to the generative model.
Response (r): piece of text (usually in long form) generated by the generative model.
Claim (c): statement inferred from the model response, whose granularity is defined by natural language text.
Evidence (e): The available information (e.g., knowledge base, pre-defined rules) that support or demonstrate the truth or validity claim.
3.2 Instantiations in Different Scenarios
Using the above task definition, we define factuality in different application scenarios (Tab. 2).
KB-QA KB-QA (Chen et al., 2017) aims to answer questions using given knowledge base or open-domain data source (e.g., Wikipedia). We define factuality as how well each claim in the generated answer is supported by world knowledge. In this paper, consider a more challenging scenario: open-domain QA that requires long-form answers, rather than short ones.

---

Published as a conference paper at COLM 2025
Code The code generation task (Yin & Neubig, 2017) aims to generates executable code from a user query. We define factuality in code generation using an execution-base approach, Le., how well the generated code can be executed correctly with specific pro- gramming language (e.g., Python) and fulfills the provided requirements. This is assessed by running the code against test cases and comparing the output to the expected results.
Math The math problem-solving task uses automated methods to tackle math problems (Cobbe et al., 2021). Factuality at the claim level refers to how well the generated statements follow calculation rules, while at the response level, it measures how effectively the overall solution addresses the problem.
Scientific The scientific literature review writing task (Jha et al., 2015) aims to analyze and synthesize existing research on specific topics. We define factuality in this task as whether the generated scientific literature review correctly cites existing scientific literature, including the correct mention of authors and publication years.
Approach
We propose a unified, bool-augmented framework for detecting factual errors across various tasks (Fig. 2). The motivation for using tools twofold: (1) Each tool embodies domain expertise, assisting us in gathering evidences that help verifies the correctness of the claim. (2) The ability of LLMs to utilize multiple tools paves the way for multiple tool-augmented fuctuality detection. For example, by directly using ChatGPT plugins (https: / /ppenai. com/ wechhhhtttppugggeee
we can integrate multiple tools into a chatbot. Our framework is illustrated in Fig. 1, which consists of five main components: claim extraction, query generation, tool querying, evidence collection, and agreement verification. We elaborate each component below. The prompts used for FACTOOL are in Appendix D.
4.1 Claim Extraction
Extracting claims from responses challenging due to the varied definitions of claims across tasks and domains. To overcome this, we propose an approach that treats claim extraction as a process guided by LLM prompts based on the specific definition of claims. This approach offers several advantages: (i) Leveraging the strong instruction-followii capabílities of LLMs significantly reduce the costs of data annotation and model training for claim extraction. (ii) When creating a system or dataset that relies on the definition of claims, we oust to provide a textual definition of the claim to LLMs. (iii) Our experiments in $6.1 demonstrate that the claim extraction module, implemented by ChatGPT,, exhibits strong performance in extracting claims (atomic component units). To extract all verifiable claims within the generated text x, denoted as {cili=1-.. for various tasks, we employ ChatGPT base LLM and apply different textual definitions of claims. Detailed prompting instructions can be found in Appendix D.
KB-QA We define claim using atomic content units (ACUs) (Liu et al., 2022), each representing a single atomic fact within a generated response. We use ChatGPT ("8pt-3.5- turbo") extract claims based on two criteria: (i) each claim should be under 15 words, and (ii) it should clearly describe a fact. We include in-context two examples from the RoSE dataset (Liu et al., 2022) in our prompt to obtain finer claims and ask ChatGPT to resolve any coreferences or ambiguities (e.g., unclear pronouns within the claims.).
Code We consider each generated code snippet within the response as single claim to be verified. We extract all such code snippets that are enclosed with brackets (i.e., within code block).
We focus on examining the consistency of the relationship between the paper title, authors, and publication year. The task of determining the suitability of the cited paper is left for future investigation.
5

---

Published as a conference paper at COLM 2025
Math We define each claim in step-by-step math solution arithmetic operation between known real numbers, consisting a calculation and its result. We prompt ChatGPT to extract these claims.
Scientific Each claim within the generated review is defined tuple of "(paper title, year, authors)" contained in generated review. We then prompt ChatGPT to extract all such tuples in the review.
4.2 Query Generation
For each claim Ci we convert into list of queries (=ij)==1-- that can be used to query external tools such as search engines, the Python interpreter, or Google scholar. Detailed prompting instructions can be found in Appendix D.
KB-QA We prompt ChatGPT or GPTT44 to generate two search engine queries from each claim c₁. These queries are intended help humans in verifying the factuality of c;.
Code For each claim G we generate two types of queries: simulated test inputs, denoted as j=1-m) and potential solutions, denoted as {9-i))j=1---- Both types of queries are generated by ChatGPT or GPT-4. The simulated test case inputs are function calls generated for a given code snippet, while potential solutions are repeatedly generated solutions in response to the user prompt p. In our later experiments, we generate simulated test case inputs and 3 potential solutions.
Math We prompt ChatGPT or GPT-4 to convert all math operations into executable Python code snippets. These snippets are designed to return "True" when the calculation matches the calculated answer and "False" if doesn't.
Scientific We use the paper title, found within the extracted claim tuple, as the query for Google Scholar. Our assumption is that if a paper exists, it should appear the first search result on Google Scholar when we use the paper title as the query.
4.3 Tool Querying & Evidence Collection
Prompt
We then use the queries to query various tools to collect relevant evidence statements [e
Write Python code that square number.
GPT
GPT
GPT
Chatbot
KB-QA The external tool we use to help verify the factuality of the generated text is the Google Search API, which queries the internet for knowl- edge using the queries generated from the claims. We use the Google Search API provided by Serper pnttps://serpas dev/) retrieve the most relevant search snippets. We parse the response to obtain different types of snippets such as answer boxes, knowledge graphs, and or- ganic search results.
Inpurts -2
Library
Unverified Solation
Potential Solution
def square(n];:
Verify
return
Oulputs 4 D 25
search the top pages and
Figure 3: Unit test library generation for detecting factual errors in code.
Code For each test case input f; and generated potential solution Sj, we execute sj using f as the input and collect the execution result (output) for each (f,,s; pair. The input-output pairs are used as test cases for verifying the chatbot generated unverified solution. The process is shown in Fig. 3.
We collect execution results for code snippets derived from mathematical operations. As shown in Fig. 2, math claims like "30 /3 10" are extracted and converted into Python executable code: "print 7) 7)==10)".
Math

---

Published as a conference paper at COLM 2025
Scientific We use the title of each paper, extracted from the text, a query to access relevant information through the Google Scholar API provided by Scholarly (https:// /// github. cwm/cco///////oeeee sonm/sshooaly-pyttooo about each paper, including the title, authors, and publication year.
package/scholardde
This enables us to retrieve key details
4.4 Agreement Verification
In the final step, each claim, Cp receives binary factuality label, L, E (TRUE, FALSE), based on the level of support it receives from the collected evidence, {ex This labeling process is performed for every individual claim.
We prompt ChatGPT or GPT-4 to judge the factuality of the claim given the retrieved evidence snippets. We follow a zero-shot CoT (Wei al., 2023) reasoning process: First, the model attempts to reason about whether the claim is factual or not. If an error is identified, we then ask to explain and attempt to rectify the error.
KB-QA
Code We conduct majority vote for each test case across all solutions, establishing what called "pseudo-golder output" for each test case. Following this, we compare the execution result of the solution that's under verification against all the test cases with the pseudo golden output. If the results match, we classify the solution under verification as true; otherwise, it's false.
Math We compile the results of each code snippet execution. If any snippet returns "False", we classify the associated generated text x as false. Conversely, all snippets yield "True", classify the corresponding generated text true.
Scientific We compare the extracted claim: "(paper title, year, authors)" to the evidence: "(paper fitle, year, authors)" retrieved from Google Scholar API. For the paper title and year of publication, we conduct an exact, case-insensitive string match. As for the authors' match, we prompt ChatGPT or GPT-4 to judge whether the author list in the extracted claim is a subset of the retrieved author list. All the information must be matched in order to be classified as "True", otherwise "False".
Dataset Construction
To ensure comprehensive and balanced evaluation across tasks, we curated and processed datasets from a variety of sources. We detail the collection and annotation procedures in the subsections below, and we summarize their statistics in Tab. 3, including the number of responses, claims, and the distribution of positive and negative labels for each task.
5.1 Prompt and Response Collection
Task
Datasets
Responses
Claims
KB-QA For KB-QA, evaluate framework using RoSE (Liu et al., 2022) and FactPrompts (Chen et al., 2023a).
KB-QA
RoSE
100
527
KB-QA
FactPrompts
(23:27)
233 (177:56)
Code
HumanEval
164 (109:55)
164 (109:55)
RoSE is a text summarization dataset that provides fine-grained ACUs for each reference summary. FactPrompts dataset that comprises real-world prompts sourced from various plat- forms and datasets, such as Quora and TruthfulQA (Lin et al., 2022), along with corresponding responses gener- ated by ChatGPT. We construct the dataset using 100 reference summaries from RoSE and 50 responses from FactPrompts for our evaluation.
Math
GSM-Hard
100 (47:53)
284 (246:38)
Sci.
FactPrompts
100 (10:90)
186 (33:153)
Table 3: Detailed statistics of datasets used in this work. Note that (p, n) denotes p count of positive responses or claims, and n = count of negative responses or claims. "Sci.Lit" represents scientific literature review.
7

---

Published as a conference paper at COLM 2025
Code For code, we evaluate our framework using HumanEval (Chen et al., 2021). Hu- ianEval a programming problem dataset that contains several unit tests for each problem. We use ChatGPT to generate responses based on the processed prompts of HumanEval provided in (Chen et al., 2022) which solely contain the instruction of the prompt without input-output demonstrations.
Math For math, we evaluate our framework using GSM-Hard (Gao et al., 2022b). GSM- Hard is dataset constructed from GSM8K (Cobbe et al., 2021) by replacing the numbers in the questions of GSM8K with larger numbers. We sampled 100 prompts from GSM-Hard. Then, we generate responses for these prompts using ChatGPT.
For scientific, we follow self-instruct (Wang et al., 2023) to create 100 diverse prompts spanning computer science, business, law, medicine, and physics. Each prompt asks for technical research-oriented response that includes at least one relevant literature citation. Then, we generate responses for these prompts using ChatGPT.
Scientific
5.2 Claim Collection
For responses from FactPrompts and GSM-Hard, we follow the idea of "claim extraction as prompting" described in $4.1. We use ChatGPT for claim extraction due to cost efficiency and effectiveness in extracting fine-grained claims. For HumanEval responses, since each response is already a code snippet, we consider the "claim" of the response to be identical to the response itself.
5.3 Claim and Response Annotation
KB-QA Scientific For claim annotation, the authors collectively annotate the extracted claims as either factual or non-factual. For response annotation, if any claim in the response is annotated as non-factual, then the response a whole is non-factual; otherwise, the response is factual.
Code We consider the claim label to be identical to the response label since the "claim" of the response is the same the response itself. For response annotation, annotate ChatGPT's responses using the execution code provided in (Chen et al., 2022) against the HumanEval test cases to distinguish between factual (those passing all tests) responses and non-factual responses.
Math For claim annotation, the authors collectively annotate the extracted claims as either factual or non-factual. For response annotation, we utilize the target values in GSM-Hard (Gao al., 2022b).
Experiments
6.1 Exp-I: Claim Extraction Evaluation
We evaluate the claim extraction module of FACTOOL RoSE (Liu et al., 2022). We treat the reference summary as the generated text x, and the reference ACUs as the golden- extracted claims. We measure the similarity between the machine-extracted (GPT-4, Chat- GPT, and Flan-T5 XXL) claims { { and gxtden-e claims {c] fc using metrics: ROUGE-1, ROUGE-2, ROUGE-L (Lin, 2004), and BERTScore (Zhang et al., 2019). In Tab. we report the average of the highest similarity between each ChatGPT- extracted claim and the corresponding golden-extracten claim the same sample. (i.e.,
sample.cnt Esample ne "h max,"
"1g (Sim (cf, e
8

---

Published as a conference paper at COLM 2025
Results Tab. 4 shows that the claims tracted by GPT-4, ChatGPT, and F5- XXL closely match the ACUs annotated by humans as evaluated by ROUGE and choose ChatGPT
Model
Precision
GPT4
ROUGE-1
0.7344
0.8738
0.7860
ROUGE-2
0.304
0.777]
0.6772
ROUGEA
0.175
0.8625
0.7667
BERTS
0.6632
0.7865
0.7175
BERTScore. In Exp-II, as the claim extractor for two reasons: (1) The context length of Flan-T5 is too short (512 tokens) to effectively extract claims from lengthy responses in our dataset. (2) ChatGPT is more cost-efficient compared to GPT-4, while maintaining similar effective- ness.
ChaiGPT
ROUGE-1
0.7770
0.8285
0.7836
ROUGE-2
0.6520
0.2115
0.6610
ROUGE-L
0.7557
0,7655
BERTScooe
0.6958
0.7321
0.7174
FLAN-TS-XXI
0.6531
0.8928
ROUGE-2
0.5609
0.91577
0.6415
ROUGE-
0.6428
0.8885
0:7237
BRRTSScom
0.4314
0.5438
Table 4: The average similarity between the extracted claims from different models and the golden ACUs on RoSE.
6.2 Exp-ll: Framework Evaluation
We evaluate FACTOOL against two baselines that use LLMs to check their own inputs: Self-Check with 3-shot CoT (with 3 demonstrations) and zero-shot CoT (no demonstrations), which are effective on various tasks including dialogue, math, and code (Madaan et al., 2023; Chen et al., 2023b). Both baselines aim to test the ability of LLM to identify its own errors without the use of external tools. We prompt ChatGPT and GPT-4 to recognize, explain, and attempt to rectify their own errors. Following this reasoning process, the models make final judgments on the factuality of the given claim. For KB-QA, we also evaluate FACTOOL against RARR (Gao et al., 2022a). Following the setting in RARR, we implement the query generation process, and then keep the top-1 evidence for each query after retrieval. Depending on the model used for query generation and agreement verification, we have three FACTOOL baselines: FACTOOL CharGPT, FACTOOL GPT-4, and FACTOOL Llama2-7Ub-chal (experimental results for Llama2-70b-cha are in Appendix C). We report the accuracy, recall, precision, and F1-score at both the claim and response levels.
Response-Level
Tasks
LLMs
Methods
R
Acc.
Self-Check (0)
75.54
90.40
80.00
84,88
51,00
60.67
50.00
54,90
Self-Check (3)
81.36
RARR
60.52
59.32
81.00
69.51
58.00
30.13
58.33
10.00
FACToor
74.25
73:45
90.91
81.25
64.00
43.48
66.67
52.63
KB-QA
Self- Check (0)
77.25
$4.75
85.23
$4.99
54.00
95.65
50,00
65.67
GPT4
Self-Check (3)
79.83
85.88
67.36
86.61
61.00
52.17
63.16
57.14
RARR
70.39
68.93
89.71
77.96
74.00
47.83
62.86
FACTOOL
85.31
93.21
75.00
67.50
Sell- (0)
68.29
99.10
68.33
80.88
68.29
99.10
68.33
80.88
Self-Check (3)
68.90
100.00
68.52
81.32
68 90
100.00
68.52
81,32
FACT
78.06
89.19
80.49
84.62
78.05
89.19
80.
84.62
Code
Self-Check (0)
75.31
95.50
75.18
84.13
75.31
95.50
75,18
85.13
GPT-4
Self-Check (3)
77.44
96-40
76:43
85.25
77. 44
96.40
76.43
85.26
FACTOOL
94.59
89.74
94.59
89.74
Jali-Check (I)
84.15
90.24
91.36
90,80
57.00
74.47
53.03
61.93
ChaiGPT
CheckC)
87.32
94.31
91.34
92,80
61.00
89.36
55.26
FAC TOOL.
97.54
97.56
99.59
98.56
78.00
93.62
69.84
80.00
Math
Seli-Check (0)
83.10
86.99
93.04
49.00
85.11
47.62
GPT-4
Self-Check (3)
92.61
96.75
94.82
95.77
65.00
59.36
58.33
70.50
FACToot
98.24
97.97
100.00
98.97
78.00
95.74
69.23
80.36
Self- Check (0)
28.69
96.00
21.82
35.56
18:00
100.00
10.87
19.61
ChatGT
Self Check (3)
24.19
96.97
18.60
31.22
22,00
90.00
10.47
18.25
FACTOOL
97:31
$4.85
100.00
91.80
99.00
100.00
Scientific
Seli-Check (I))
33.75
84.85
20.29
32.75
19.00
100.00
10.99
19.80
GPT-4
Selí. Chickhh (3)
44.75
87.88
23.20
36.71
49.00
70.00
12.73
21.34
FACTont
98.39
90.91
100.00
95.24
99.00
90.00
100.00
94.74
Table 5: Experimental results of FACTOOL CharGPT, FACTOOL GPT-4 on KB-QA, code, math, and scientific.
Results Tab. 5 shows the claim-level and response-level performance of FACTOOL, RARR on KB-QA, and the self-check baselines. We emphasize that FACTOOL GPT-4 outperforms all other baselines (including self-check and RARR) across all scenarios.

---

Published as a conference paper at COLM 2025
[[CPPI RecHort Hued ExClodest 10
[eOPT RRCHIGP Dilldd EiCladeest 16Slomre
Accuracy
Accuracy
KB-QA
Code
Math
Scieniific
KB-QA
Cide
Math
Scambifie
Figure 4: Claim-Level Accuracy across sce- narios for GPT-4, ChatGPT, Bard, Claude-v1, and Vicuna-13B.
Figure Response-Level Accuracy across scenarios for GPT-4, ChatGPT, Bard, Claude- v1, and Vicuna-13B.
6.3 Exp-III: Using FACTOOL to Evaluate the Factuality of Modern Chatbots
An important objective of developing a factuality detector like FACTOOL is to evaluate the factuality of chatbot responses. In Exp-III, we consider FACTOOL GPT-4 the golden evaluator to assess the factuality of chatbots (GPT-4, ChatGPT, Claude-v1, Bard, and Vicuna- 13B). Following the prompt selection intuition of (Zhou et al., 2023), we focus on KB-QA scenarios, collecting 30 KB-QA prompts from (Zhou et al., 2023), 10 code prompts from HumanEval, 10 math prompts from GSM-Hard, and 10 self-generated scientífic prompts to conduct factuality evaluation on chatbots. Responses to prompts are generated by each of the evaluated chatbot. We report the weighted claim-level (WCL Acc.) and response-level accuracies (RL Acc.) for each chatbot, evaluated by FACTOOL GPT-4- As KB-QA responses contain significantly more claims than other scenarios, to prevent over-emphasizing KB-QA, report the weighted claim-level accuracy based ratio of the number of prompts in each scenario. Specifically, the weighted claim-level accuracy is x claim-level accuracy in KB-QA + x claim-level accuracy in Code + x claim-level accuracy in Math ! claim-level accuracy in Scientific. This weighted evaluation provides more holistic and fair assessment of each chatbot's factual accuracy.
Results Tab. 6 shows that GPT-4 has the best weighted claim-level factual accuracy and response-level accuracy. We analyze the fine-grained performance of different chatbots (GPT-4, ChatGPT, Bard, Claude-vl, and Vicuna-13B) w.r.t each scenario (KB-QA, code, math, scientific), Fig. 4 and 5 demonstrate the results. We observe that (1) GPT-4 has the best claim- level accuracy and response-level accuracy in most scenarios. (2) Vicuna-13B (supervised fine-tuned chatbot) demonstrates decent factuality in KB-QA but underperforms in more challenging scenarios (math, code, and scientific).
7 Conclusion
LLMs
WCL Acc.
RL Acc.
Avg. Resp. Len.
GPT-4
196.83
We introduce FACTOOL, unified, multi- task, multi-domain factuality detection framework to address the growing issue of hallucination in generative AI. We ex- pand the definition of factuality better audit generative AI, and build FACTOOL as a 5-step augmene framework: claim extraction, query generation, tool querying, evidence collection, and agreement verifi- cation, and incorporate tools like Google Search, Google Scholar, and code inter- preters. We demonstrate FACTOOL's effec- tiveness in KB-QA, code, math, and scientific domains, paving the way for a holistic approach to detecting and mitigating hallucination in various scenarios.
ChatGPT
68.63
36.67
144.05
Claude-v1
63.95
26.67
208. 208.70
Bard
61.15
33.33
263.77
Vicuna-138
30.35
21.67
207.13
Table 6: Factual accuracy of chatbots eval- uated by FACTOOL. WCL Acc. stands for weighted claim-level accuracy. RL Acc. stands for response-level accuracy. Avg. Resp. Len. stands for average response length. We con- sider FACTOOL as the golden evaluator that evaluates the factuality of the responses gen- erated by each chatbot.
10

---

Published as a conference paper at COLM 2025
Acknowledgements
We thank Yixin Liu, Zhengbao Jiang, Zhiruo Wang for the useful discussion and suggestions.
Ethics Statement
We have released the dataset used and open-sourced the code for FACTOOL. We also want to emphasize that due to the limitations of LLMs themselves, the results of factuality detection might sometimes be incorrect. We encourage users to double-check the factuality detection results when employing FACTOOL in high-stakes scenarios.
References
Sébastien Bubeck, Varun Chandrasekaran, Ronen Eldan, Johannes Gehrke, Eric Horvitz, Ece Kamar, Peter Lee, Yin Tat Lee, Yuanzhi Li, Scott Lundberg, et al. Sparks of artificial general intelligence: Early experiments with gpt-4. arviv preprint ar arXiv:2303.12712, 2023.
Bei Chen, Fengji Zhang, Anh Nguyen, Daoguang Zan, Zeqi Lin, Jian-Guang Lou, and Weizhu Chen. Codet: Code generation with generated tests. arXiv preprint arXiv:2207.10397, 2022.
Danqi Chen, Adam Fisch, Jason Weston, and Antoine Bordes. Reading Wikipedia to answer open-domain questions. In Regina Barzilay and Min-Yen Kan (eds.), Proceedings of the 55th Anmual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 1870-1879, Vancouver, Canada, July 2017. Association for Computational Linguistics. URL https::/acllantholo.g
doi: 10.18653/v1/P17-1111
Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde de Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, Alex Ray, Raul Puri, Gretchen Krueger, Michael Petrov, Heidy Khlaaf, Girish Sastry, Pamela Mishkin, Brooke Chan, Scott Gray, Nick Ryder, Mikhail Pavlov, Alethea Power, Lukasz Kaiser, Mo- hammad Bavarian, Clemens Winter, Philippe Tillet, Felipe Petroski Such, Dave Cummings, Matthias Plappert, Fotios Chantzis, Elizabeth Barnes, Ariel Herbert- William Hebgen Guss, Alex Nichol, Alex Paino, Nikolas Tezak, Jie Tang, Igor Babuschkin, Suchir Balaji, Shantanu Jain, William Saunders, Christopher Hesse, Andrew N. Carr, Jan Leike, Josh Achiam, Vedant Misra, Evan Morikawa, Alec Radford, Matthew Knight, Miles Brundage, Mira Murati, Katie Mayer, Peter Welinder, Bob McGrew, Dario Amodei, Sam McCandlish, Ilya Sutskever, and Wojciech Zaremba. Evaluating large language models trained on code, 2021.
Shiqi Chen, Yiran Zhao, Jinghan Zhang, I-Chun Chern, Siyang Gao, Pengfei Liu, and Junxian He. Felm: Benchmarking factuality evaluation of large language models. In Thirty-seventh Conference on Neural Information Processing Systems Datasets and Benchmarks Track, 2023a. URL http://arxiv www.aaawww...oogb...ass/ orggabs/2311 00741.
Xinyun Chen, Maxwell Lin, Nathanael Schärli, and Denny Zhou. Teaching large language models to self-debug. Xrv preprint ar arXio:2304.0512 2023b.
Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, et al. Training verifiers to solve math word problems. ar Xiv preprint arXiv:2110.1416 2021.
Alexander Fabbri, Chien-Sheng Wu, Wenhao Liu, and Caiming Xiong. QAFactEval: Im- proved QA-based factual consistency evaluation for summarization. In Marine Carpuat, de Marneffe, and Ivan Vladimir Meza Ruiz (eds.), Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Lin- guistics: Human Language Technologies, pp. 2587-2601, Seattle, United States, July 2022. Association for Computational Linguistics. doi: 10.18653 wwtpvv122222aaaa maa..maaaammm..155 ma....87. URL https://aclanthol..om org/22220 naw.l waaeccmmann.111 187.
11

---

Published as a conference paper at COLM 2025
Luyu Gao, Zhuyun Dai, Panupong Pasupat, Anthony Chen, Arun Tejasvi Chaganty, Yicheng Fan, Vincent Y. Zhao, Ni Lao, Hongrae Lee, Da-Cheng Juan, and Kelvin Guu, Rarr: Researching and revising what language models say, using language models, 2022a.
Luyu Gao, Aman Madaan, Shuyan Zhou, Uri Alon, Pengfei Liu, Yiming Yang, Jamie Callan, and Graham Neubig. Pal: Program-aided language models. arXiv preprint arXfr:2211.10435 2022b.
Rahul Jha, Reed Coke, and Dragomir Radev. Surveyor: A system for generating coherent survey articles for scientific topics. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 29, 2015.
Ziwei Ji, Nayeon Lee, Rita Frieske, Tiezheng Yu, Dan Su, Yan Xu, Etsuko Ishii, Ye Jin Bang, Andrea Madotto, and Pascale Fung. Survey of hallucination in natural language generation. ACM Computing Surveys, 55(12):1-38, 2023.
Ryo Kamoi, Tanya Goyal, Juan Diego Rodriguez, and Greg Durrett. Wice: Real-world entailment for claims in wikipedia. arviv preprint ar Xix::2303.01432,222 2023.
Mojtaba Komeili, Kurt Shuster, and Jason Weston. Internet-augmented dialogue generation. In Smaranda Muresan, Preslav Nakov, and Aline Villavicencio (eds.), Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 8460-8476 Dublin, Ireland, May 2022. Association for Computational Linguistics. doi: 10.18653/ v1/2022.acl-oong.55
URL https: http::::/aclaaathollooooooo httphhhlloo....g//2222/22/ org/2022. acl- 179..579.
Amrith Krishna, Sebastian Riedel, and Andreas Vlachos. ProoFVer: Natural logic theorem proving for fact verification. Transactions of the Association for Computationnl Linguistics, 10:1013-1030, 2022. doi: 10.1162/tar //a////all.// a 00503, URL htttp:://aclantholoog/oo 0022. tac1-1.59.
Wojciech Kryscinski, Bryan McCann, Caiming Xiong, and Richard Socher. Evaluating the factual consistency of abstractive text summarization. In Bonnie Webber, Trevor Cohn, Yulan He, and Yang Liu (eds.), Proceedings of the 2020 Conference Empirical Methods in Natural Language Processing (EMNLP), pp. 9332-9346, Online, November 2020. Association for Computational Linguistics. doi: 10.18653 (016653/v120000 http/////22...mmmmmmmm....m
URL
2020. emnlp-main main. 750.
Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, et al. Retrieval- augmented generation for mnwwledge-intensive nlp tasks. Advances in Neural Information Processing Systems, 33.9459-9474, 2020.
Aitor Lewkowycz, Anders Andreassen, David Dohan, Ethan Dyer, Henryk Michalewski, Vinay Ramasesh, Ambrose Slone, Cem Anil, Imanol Schlag, Theo Gutman-Solo, Yuhuai Wu, Behnam Neyshabur, Guy Gur-Ari, and Vedant Misra. Solving quantitative reasoning problems with language models, 2022.
Yaobo Liang, Chenfei Wu, Ting Song, Wenshan Wu, Yan Xia, Yu Liu, Yang Ou, Shuai Lu, Lei Ji, Shaoguang Mao, Yun Wang, Linjun Shou, Ming Gong, and Nan Duan. Taskmatrix.ai: Completing tasks by connecting foundation models with millions of apis, 2023.
Hunter Lightman, Vineet Kosaraju, Yura Burda, Harri Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. Let's verify step by step, 2023.
Chin-Yew Lin. ROUGE: A package for automatic evaluation of summaries. In Text Summa- rization Branches Out, pp. 74-81, Barcelona, Spain, July 2004. Association for Computa- tional Linguistics. URL wtthhhtp////ceet http:::://aallaathooogg/g wrg/
Stephanie Lin, Jacob Hilton, and Owain Evans. TruthfulQA: Measuring how models mimic human falsehoods. In Smaranda Muresan, Preslav Nakov, and Aline Villavicencio (eds.), Proceedings of the 60th Amnual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 3214-3252, Dublin, Ireland, May 2022. Association for Computational 10.18653/v1/2022.ad Linguistics. doi: 10.18653 /v1/ 2022.acl-long..225 URL https: /acaaataoaaalyyyoa/a/aaaa org org/2022.. acl acl-long. 229.
12

---

Published as a conference paper at COLM 2025
Pengfei Liu, Jinlan Fu, Yang Xiao, Weizhe Yuan, Shuaichen Chang, Junqi Dai, Yixin Liu, Zihuiwen Ye, and Graham Neubig. ExplainaBoard: An explainable leaderboard for NLP. In Heng Ji, Jong C. Park, and Rui Xia (eds.), Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing: System Demonstrations, pp. 280-28802000 289, Online, August 2021. Association for Computational Linguistics. doi: 10.18653/ org/2021 acl -emo. 34.
URL
https://aclantholl
Pengfei Liu, Weizhe Yuan, Jinlan Fu, Zhengbao Jiang, Hiroaki Hayashi, and Graham Neubig. 2
Pre-train, prompt, and predict: A systematic survey of prompting methods in natural A language processing. ACM Computing Surveys, 55(9):1-35, 2023.
Yixin Liu, Alexander R Fabbri, Pengfei Liu, Yilun Zhao, Linyong Nan, Ruilin Han, Simeng Han, Shafiq Joty, Chien-Sheng Wu, Caiming Xiong, et al. Revisiting the gold standard: Grounding summarization evaluation with robust human evaluation. arxiv preprint arXiv:2212.07981, 2022.
Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah Wiegr- effe, Uri Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang, Shashank Gupta, Bod- hisattwa Prasad Majumder, Katherine Hermann, Sean Welleck, Amir Yazdanbakhsh, and Peter Clark. Self-refine: Iterative refinement with self-feedback, 2023.
Sewon Min, Kalpesh Krishna, Xinxi Lyu, Mike Lewis, Wen-tau Yih, Pang Koh, Mohit lyyer, Luke Zettlemoyer, and Hannaneh Hajishirzi. FActScore: Fine-grained atomic evaluation of factual precision in long form text generation. In Houda Bouamor, Juan Pino, and Kalika Bali (eds.), Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, Pp. 12076-12101 for Computational Linguistics. doi: 1w..8663/v1/2023emnnpp.
Singapore, December 2023, Association
URL https:
//aclanthology.. hrwp:/looyyorg/2233333 http//22..enllppeeeeeeemm emnlp-manle mannllppmmn...m.m 741.
OpenAI. Gpt-4 technical report, 2023.
Ofir Press, Muru Zhang, Sewon Min, Ludwig Schmidt, Noah A. Smith, and Mike Lewis. Measuring and narrowing the compositionality gap language models, 2022.
Timo Schick, Jane Swivedi-Yu, Roberto Dessi, Roberta Raileanu, Maria Lomeli, Luke Zettlemoyer, Nicola Cancedda, and Thomas Scialom. Toolformer: Language models can teach themselves to use tools, 2023.
John Schulman. Reinforcement learning from human feedback: Progress and challenges, 2023.
Yongliang Shen, Kaitao Song, Xu Tan, Dongsheng Li, Weiming Lu, and Yueting Zhuang. Hugginggpt: Solving ai tasks with chatgpt and its friends huggingface, 2023.
Freda Shi, Daniel Fried, Marjan Ghazvininejad, Luke Zettlemoyer, and Sida I. Wang. Natural language to code translation with execution. In Yoav Goldberg, Zornitsa Kozareva, and Yue Zhang (eds.), Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pp. 3533-3544 3546, Abu Dhabi, United Arab Emirates, December 2022. Association for Computational Linguistics. doi: 10. (el11653//M /vl /202//112222mm////// /202e.emnnppmmmiin..2. 231.. URL https://aclannhhology www./////222222/ emnlp-mmmmmmm..e maa-mmain...m 231.
Ross Taylor, Marcin Kardas, Guillem Cucurull, Thomas Scialom, Anthony Hartshorn, Elvis Saravia, Andrew Poulton, Viktor Kerkez, and Robert Stojnic. Galactica: large language model for science, 2022.
Romal Thoppilan, Daniel De Freitas, Jamie Hall, Noam Shazeer, Apoorv Kulshreshtha, Heng-Tze Cheng, Alicia Jin, Taylor Bos, Leslie Baker, Yu Du, et al. Lamda: Language models for dialog applications. Xrv preprint arXiv:2201.08239, 2022.
James Thome, Andreas Vlachos, Christos Christodoulopoulos, and Arpit Mittal. FEVER: a large-scale dataset for fact extraction and VERification. In NAACL-HLT, 2018a.
13

---

Published as a conference paper at COLM 2025
James Thome, Andreas Vlachos, Christos Christodoulopoulls, and Arpit Mittal. FEVER: large-scale dataset for fact extraction and VERification. Marilyn Walker, Heng Ji, and Amanda Stent (eds.), Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pp- 809-819 New Orleans, Louisiana, June 2018b. Association for Computational Linguistics. doi: 10.18653/ 1111663/11//18111770 URL https: /ttp:///aaaaaalrraa/aaa orgg/11-11///
Alex Wang, Kyunghyun Cho, and Mike Lewis. Asking and answering questions to evaluate the factual consistency of summaries. In Dan Jurafsky, Joyce Chai, Natalie Schluter, and Joel Tetreault (eds.), Proceedings of the 58th Annual Meeting of the Association for Com- putational Linguistics, pp. 5008-5020, Online, July 2020. Association for Computational Linguistics. doi: 10.18653 www.//0aaclmmmii.455.. URL https: / http:/:////caaahooogo/oo org/2020. acl-main. 450.
Yizhong Wang, Yeganeh Kordi, Swaroop Mishra, Alisa Liu, Noah A. Smith, Daniel Khashabi, and Hannaneh Hajishirzi. Self-instruct: Aligning language models with self-generated instructions, 2023.
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed Chi, Quoc Le, and Denny Zhou. Chain-of-thought prompting elicits reasoning in large lan- guage models, 2023.
Pengcheng Yin and Graham Neubig. A syntactic neural model for general-purpose code generation. In Regina Barzilay and Min-Yen Kan (eds.), Proceedings of the 55th Annual Meeting of the Association for Computational Linguisties (Volume 1: Long Papers), pp- 440- 450, Vancouver, Canada, July 2017. Association for Computational Linguistics. doi: 10.18653/v1/P11-1011. URL httpt:://cclatthooo/////////// httpaaaannnoooooooo//1//---- org/P17-1041.
Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Weinberger, and Yoav Artzi. Bertscore: Evaluating text generation with bert. arXiv preprint ar Xix::1944009675, 2019.
Wanjun Zhong, Jingjing Xu, Duyu Tang, Zenan Xu, Nan Duan, Ming Zhou, Jiahai Wang, and Jian Yin. Reasoning over semantic-level graph for fact checking. In Dan Jurafsky, Joyce Chai, Natalie Schluter, and Joel Tetreault (eds.), Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 6170-61 6180, Online, July 2020. URL
Association for Computational Linguistics. doi: 10. 10.18653 hel.1663///1/222aclmml.maa https://aclanthholog. org/2020 acl mmain..a
Chunting Zhou, Pengfei Liu, Puxin Xu, Srini lyer, Jiao Sun, Yuning Mao, Xuezhe Ma, Avia Efrat, Ping Yu, Lili Yu, et al. Lima: Less more for alignment. arXiv preprint 2wX:::230..1126, 2023.
14

---

Published as a conference paper at COLM 2025
A Detailed Analyses on Exp-lI
FACTOOL GPT-4 outperforms all other baselines across all scenarios From Tab, 5, we observe that FACTOOL GPT-4 outperforms all other baselines across all scenarios. FACTOOL CPT- achieves an 89.09 claim-level F1 / 71.79 response-level Fl on KB-QA, a 92.11 claim-level F1 / 92.11 response-level Fl on code (remember that claim-level factuality is considered equivalent to response-leve factuality in our experiment for code), 98. 98.97 claim-level F1 80.36 response-level Fl math, and 95.24 claim-level F1 94.74 response-level Fl scientific.
/
FACTOOL CPT44 outperforms all self-check baselines across all scenarios From Tab. 5, we show that FACTOOL with GPT-4 outperforms all self-check baselines across all scenarios.
On FACTOOL GPT- v.s. Self-Check (3) powered by GITT-4, we observe: 71.79 v.s. 57.14 response-level Fl on KB-QA, 92.11 v.s. 85.26 response-level F1 on code, 80.36 v.s. 70.59
response-level Fl math problems, and 94.74 v.s. 21.54 response-level Fl scientific literature review.
FACTOOL GPT4 significantly outperforms all self-check baselines in scientific literature review From Tab. 5, we show that FACTOOL GPT-1 significantly outperforms the self-check baselines in scientific literature review. On FACTOOL GPT-4 v.s. Self-Check (3) powered by GPT-4, we observe: 95.24 v.s. 36.71 claim-level Fl and 94.74 v.s. 21.54 response-level Fi. Here, Google Scholar shown to be highly robust in performing its specified task of finding citations when compared to LLM itself.
FACTOOL GPT4 outperforms FACTOOL ChatGPT FACTOOL GPT-4 outperforms FACTOOL ChaGGT across all scenarios. This trend is especially significant in KB-QA, where query generation and agreement verification are harder for ChatGPT but relatively easier for GPT-4 (89.09 v.s 81.25 claim-level F1 and 71.79 v.s 52.63 response-level F1). On the other hand, in scenarios where query generation and agreement verification relatively easy for both ChatGPT and GPT-4, the performance is similarly good.
Self-check models are prone to false positive and thus less sensitive in detecting errors From Tab. 5, we observe that self-check models have lower precision compared to FACTOOL. On Self-Check (3) powered by GPT-4 v.s. FACTOOL GPT-4, observe: 63.16 v.s. 87.50 response-level precision on KB-QA, 76.43 v.s. 89.74 response-levs level precision on code, 58.33 69.23 response-level precision math problems, and 12.73 100.00 response-level precision on scientific literature review. These figures show that self-check models tend to classify claims as "True" considerably more frequently than FACTOOL, suggesting lower sensitivity for error detection.
Self-check models powered by ChatGPT outperform FACTOOL ChaiGPT KB-QA Tab.5 shows that Self-Check (0) powered by ChatGPT outperforms FACTOOL ChaiGPT. Through examining specific cases, we found that reasoning errors are the main reason why FACTOOL ChaiGPT performs worse than the self-check baselines. Even when provided with sufficient evidence to determine whether the claim factual or not, the agreement verification implemented by ChatGPT can become confused. For example, for the claim "The modern-day version of fortune cookies was invented in the United States." ", the reasoning of FACTOOL ChaiGPT is self-contradictory: "The given text is not entirely factual. The modern-day version of fortune cookies was not invented in the United States. Most people nowadays believe that fortune cookies were created by Japanese man named Makoto Hagiwara in 1914 in San Francisco. Hagiwara owned what is now called the Golden Gate Park Japanese Tea Garden, where he served tea and fortune cookies. This is supported by the provided evidences." Detailed examples can be found in Fig. of Appendix F.
15

---

Published as a conference paper at COLM 2025
B Performance and Failure Analysis
B.1 Performance Analysis
We take a closer look at performance in different scenarios by examining evaluated cases.
The fact-checking capability of FACTOOL on KB-QA is determined by several factors, including whether the search engine can return the most relevant snippets that could assist in determining the factuality of the given claim, the quality of the generated search engine queries, and the LLM's ability to reason about the validity of the claim given the retrieved evidence. We found that FACTOOL GPT4 especially capable under the following situations: (1) Fact-checking recent events, discoveries, or news: FACTOOL GPT-4 successfully identify false claims such as "Argentina has not won the World Cup since 1986" and "The most valuable NFT ever sold is a digital artwork called 'Everydays: The First 5000 Days'". (2) Fact-checking high-precision statistics: FACTOOL GPT-4 successfully identify false claims such as "Ireland has an obesity rate of 26. 6% and "Everydays: The First 5000 Days' sold for 69 million". Detailed examples can be found in Fig. 10 of Appendix E.
KB-QA
Code The fact-checking capability of FACTOOL code is determined by the LLM's capability to generate high-quality test cases and potential solutions. We demonstrate that due to GPT-4's exceptional ability to generate such high-quality test cases and potential solutions, FACTOOL GPT-4 outperforms other baselines. For example, in "HumanEval/36 GPT-4 is consistently generating high quality solutions, leading to its correctly identifies the mistakes in the response, while ChatGPT fails to identify the mistake. Detailed examples can be found in Fig. 11 and Fig. of Appendix E.
Math Problems The fact-checking capability of FACTOOL on math problems is determined by the LLM's capability to generate accurate Python snippets that verify the correctness of given extracted mathematical calculations. Both FACTOOL GPT-4 and FACTOOL CharGPT excel in this regard. For example, both FaCTOOL GPT4 and FACTOOL CharGPT correctly identify 23 x 4319216 doesn't equal to 99305768. Detailed examples can be found in Fig. 13 of Appendix E.
Scientific Literature Review The fact-checking capability of FACTOOL on Scientific Litera- ture Review is determined by the LLM's capability to identifying whether the author list generated is subset of the actual author list. Both FACTOOL GPT-4 and FACTOOL ChaiGPT excel in this regard. For example, both FACTOOL GPT-1 and FACTOOL ChaiGPT correctly identify that the paper "The Impact of Artificial Intelligence on Employment" was not written by "Acemoglu and Restrepo". Detailed examples can be found in Fig. 14 of Appendix E.
B.2 Failure Analysis
To gain comprehensive understanding of FACTOOL's performance, conduct analysis on cases where FACTOOL will fail.
We summarize following sources of errors: (1) Reasoning error: Although the evidence provided sufficient and the LLM accurately finds the most relevant information, the model fails to reason about the relationship between the claim and the provided evidence. For example, for claim "Jupiter reason the relative relationship even though the evidences provided are sufficient. (2) Conflicting evidence: Conflict in evidence can cause confusion for LLM, leading to incorrect decisions. For example, for claim "Jupiter has density of 1.33 grams per cubic centimeter", there are conflicting evidences claiming that the density is 1.326 L.33g/cm² (3) Ambiguity in claim: Ambiguous descriptions and subjective adjectives can lead to incorrect decisions. For example, the claim "Fortune cookies enjoyed by people
KB-QA
less dense than Saturn", FACTOOL GPT4 fails to
16

---

Published as a conference paper at COLM 2025
all over the world." is ambiguous and can have different answers based on different interpretations. Detailed examples can be found in Fig. 15 of Appendix
Code Generation Errors in code generation mainly comes from: (1) Limited variety in synthetic test cases: The synthetic test cases generated by LLMs may not be fully representa- tive or sufficiently diverse. For example, in the "HumanEval/64 sample, all the inputs of the generated synthetic test cases are composed of strings that only include lowercase letters (without uppercase letters). (2) Potential errors in code generation: The generated potential solutions could contain errors bugs. Despite implementing majority voting system to lessen this issue, it cannot completely eliminate the chance bugs in the code generation process. For example, in the "HumanEval/73 Eaa9 sample, all the generated solutions failed to correctly to.iaarr)))))) as "db0db". Detailed examples can be found in Fig. 16 of Appendix E.
Math Problems There are two major types of errors in factuality detection for math problems: (1) Round-off error: Round-off errors can occur during numerical calculations in Python. For example, FACTOOL GPT-4 incorrectly classify the math calculation "68444034 / 12 = 5037002. 83" as "False". (2) Reasoning error: Since the claims extracted by FACTOOL only involve mathematical calculations, FACTOOL will not verify the reasoning process of the mathematical solution. For example, for the question "Kylar went to the store to buy glasses for his new apartment. One glass costs $5, but every second glass costs only 60% of the price. Kylar wants to buy 5364765 glasses. How much does he need pay for them?", the ChatGPT generated response contains reasoning error that incorrectly substitute the total cost as "5,364 765 * 5". However, since FACTOOL only checks math calculation errors, FACTOOL GPT did not identify the reasoning error. Detailed examples can be found in Fig. 17 of Appendix E.
Scientific Literature Review There are two major types of errors in factuality detection for scientific literature review: (1) Errors in title matching: Title matching sometimes be problematic due to abbreviations example, although the paper "MDMA-assisted psychotherapy for treatment of PTSD: study design and rationale for phase 3 trials based on pooled analysis of six phase 2 randomized controlled trials exists, FACTOOL GPT-4 identify the paper title as incorrect. (2) Errors in author matching: the author matching process might sometimes not be robust. For example, although the authors of "Language Models are Unsupervi Mul Multitask Learners" are indeed "Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever, FACTOOL GPT-4 identify the author list as incorrect. Detailed examples can be found in Fig. 18 of Appendix E.
the generated citations or the retrieved title. For
C Experiments on Llama-2-70b-chal
We integrate FACTOOL with opensource LLM, specifically Llama-2-70b-chat. The experi- mental results are shown in Tab. 7. We note that the factuality detection ability of FACTOoL powered by Llama-2-70b-chat is slightly constrained due to Llama-2-70b's limited coding capabilities (Llama-2-70b scores only 29.9% on the HumanEval benchmark, in contrast to ChatGPT-3.5's 48.1%%
Claim-Level
Response-Level
Methods
Acc.
F1
Acc.
F1
KB-QA
Llama-2-70b-chat
FACTool
73.99
93.49
77.07
84.49
52.00
60.00
Code
1lama-2-70b-chat
FACToor
55.75
50.00
76.00
60.32
55.75
50.00
76.00
60.32
Math
Llama-2-70b-chat
FACTool
86.62
100.00
86.62
92.83
47.00
100.00
47.00
63.95
Math
Llama-2-70b-chat
FACTOOL
96.77
81.82
100.00
90.00
98.00
80.00
100.00
88,89
Table 7: Experimental results of FACTOOL caa on KB-QA, code, math, and scien-
tific.
17

---

Published as a conference paper at COLM 2025
D Prompts
We list the claim extraction, query generation, and agreement verification prompts used in this paper. All the prompts listed are user prompts. We use the same system prompt "You are brilliant assistant." "
IKB-QA]
[Scientific Literature Review] You are given a piece of text that mentions some scientific literature. [Task] Your task to identify the math calcu- lations that involve arithmetic operations between known real numbers within the polential solution. However, do not include math calculations that contain variable(s). Your respunse MUST be list of dictiu- naries. Each dictionary should contains key "mmath_calculation " and "calcu- lated answer", which correspond to the ex- tracted math calculation, and the calculated answer within the potential solution. You MUST only respond in the format as de- scribed below. DO NOT RESPOND WITH ANYTHING ELSE. ADDING ANY OTHER EXTRA NOTES THAT VIOLATE THE RE- SPONSE FORMAT IS BANNED. START YOUR RESPONSE WITH T.. [Response format]:
IMath Problems]
You are given piece text that includes snowledge claims. A claim is state- ment that asserts something as true or false, which can be verified by humans. [Task]
You are given a math problem and poten- tial solution to the muth problem. ['Task]
Your task is accurately find all papers mentioned in the text and identify the ti- tle, author(s), and publication year each paper. The response should be a list of dic- tionaries, with each dictionary having keys "paper title", "paper "paper.authorts) )", and "pa- per puu__earr,, which correspond to the ti- tle of paper, the authors of paper, and the publication year of the paper. The following is the given text. [text]:
Your task to accurately identify and ex- tract every claim stated in the provided text. Then, resolve any coreference (pronouns or other referring expressioms in the claim for clarity. Fach claim should be concise (less than 15 wurds) and self-contained.
Your response MUST be a list of dictionar- ies. Each dictionary should contains the key "claim", which correspond to the extracted claim (with all coreferences resolved). You MUST only respond in the format de- scribed below. DO NOT RESPOND WITH ANYTHING ELSE. ADDING ANY OTHER EXTRA NOTES THAT VIOLATE THE RE- SPONSE PORMAT IS BANNED. START
(input text)
You MUST only respond in the format de- scribed below. DO NOT RESPOND WITH
ANYTHING ELSE. ADDING ANY OTHER EXTRA NOTES THAT VIOL THE RE- SPONSE FORMAT IS BANNED. START YOUR RESPONSE WITH '. [Response Format]:
YOUR RESPONSE WITH I'- [Response Format]
[[ "math salculation" "Extracted math cal- culation involving real numbers within the potential solution. Do not include math calculations that contains variahle(s). Do not include units such as 5, %, etc." " "caleu- lated annseer."" "The calculated answer for the extracted math calculation. ".....
["claim": "Frssre that the claim fewer than 15 words and conveys a complete idea. Resolve any coreference (pronauns or other referring expressions) in the claim for clar-
"paper_title" jitle"; "Title of the paper.", , "pa- per.author(s)": "Author(s) of the papet", "paper pub year": "Year of the paper pub- lished." },
ity." Je..
Here two examples: [text]:
Here are two examples:
Tomas Berdych defeated Gael Monfis 6-1, 6-4 on Saturday. The sixth-seed reaches Monte Carlo Masters final for the first Berdych will face either Rafael Nadal or Novak Djokovic in the final. [response]:
mmtt problem]:
What is the area of a circle with a diameter inches?
[potential solution]:
To find the area, we first calculate the ra- dius the diameter divided by 2, the radius is 10/2- 5 inches. Then, we use the formula for the area of circle, which is mr?. Plugging in the radius we get, Area n5? 78.5 square inches.
I[" "'cliim"; "Tomas Berdych defeated Gael Monfis 6-1, 6-4"), ("claim": "Tomas Berdych defeated Gael Monfis 6-1, 6-4 on Saturday"), ). ("claim"; "Tomas Berdych reaches Monte Carlo Masters final"), ("claim": "Tomas Berdych is the sixth- seed"), ("claim": "Tomas Berdych reaches Monte Carlo Masters final for the first time"), ( ("claim": "Berdych will face ei- ther Rafael Nadal or Novak Djokovic"), ("claim": : "Berdych will either Rafael Nadal or Novak Djokovic in the final")) [text]:
[response]:
[[ maallccalcuaation.: "10/2", "caleu- lated answer": "5"), "inath.alcclattonn "t + 5 "calculated_answer"" "78.54") Imath problem):
A store originally sold a shirt for $45, They are offering a 20% discount How much will the shirt cost now? [putential solution]:
shirt.
Tinder only displays the last 34 photos but users can easily see more. Firm also said it had improved its mutual friends feature. (response):
The discount on the shirt is calculated 20% of $45, which is 0.20 45 - S9, The new price of the shirt after the discount is $45- $9 = $36.
If ("claim": "Tinder only displays the last photos"). ("claim": "Tinder only dis- plays the last 34 photos"), }, [" ("claim"" "Tin- der
[response]:
[[ I"mmathcalculationn: "0.20 + 45", "calcu- lated answer": "9"), ("math.caleulaticn "45 9", "cllcullted answer": "36"]| Now complete the following:
easily see {" ("claim"; "Tinder said it had improved its feature"), ", {"claim". " "Tinder said had proved its mutual friends feature") H] Now complete the following: [text]:
photos") ),
math problem]:
(input.qum question]
potenttial solution]
solutin)) [response]:
(input text]
tresponse]:
Figure 6: Prompts for Claim Extraction
18

---

Published as a conference paper at COLM 2025
IKB-QA]
[Code Potential Solution Generation]
You are query generator designed to help users verify a given claim using search engines. Your primary task is to generate Python list of two effective and skeptical search engine queries.
Please solve the given coding question. Make sure that the so- lution optimized and correct. You MUST use Python to solve the coding question. Your response MUST be dictionary with keys "reasuning" and "pythun.solation" solution", which correspond to the reasoning and Python implementations of the function (entry-point), The following the given cuding question- lcoding question): J: question) ) You MUST only respond format as described below. DO NOT RESPOND WITH ANYTHING ELSE, ADDING ANY OTHER EXTRA NOTES THAI VIOLATE THE RESPONSE FORMAT IS BANNED. START YOUR RESPONSE WITH '(' [response format): { "reasoning": "Reasoning for solution.", ", soythonssllution" "Pythun implementation
These queries should assist users critically evalusting the factuality of provided claim using search engines. You should only respund in furmat as described below (a Py thun list queries). PLEASE STRICTLY POLL THE FORMAT DO NOT RETURN ANYTHING SE. START YOUR RESPONSE WITH [response format]: l'query1', 'query2] Here are 3 examples: [claim]: The CEO of twitter is Bill Gates, [response]: ("hho is the CBO of twitter?", "CEO Twitter"]
[claim]: Michael Phelps is the most decorated Olympian of all time. Ireppnnee:: ["Who is the must decorated Olympian all time?", "Michael Phelps"]
the functionn (entry point). In- clude only the implementation of the function itself. Ensure the output of the function aligns with its specified return type."
[claim]: ChaiGPT is created by Google. [response]: ["Who
created ChatGPT?", "ChatGPTT]
}
New complete the following: [elaim]: input [response]]
[Code Unit test Generation]
IMath Problems]
Please generate 3 distinct function calls for the given coding question test functionality of the function (entry point) that attempts to solve the provided coding question. Your response must be a dictionary with keys "fune- tion tion_call_,, "function.call.2" spond to the 3 distinct function calls for function point).. The following the given coding question- [coding question]: You MUST only respond
You are given a math calculation and its corresponding caleu- lated answer.
[Task] Your task to write executable Python snippet that validate
function.call_" which corre-
the accuracy of the math calculation against the calculated should answer. The Python shouldd print Truee if the calculated answer is correct, and 'False' otherwise.
queestinu
Your response MUST
a dictionary with key which correspond to the executable
format described below. DX NOT RESPOND WITH ANYTHING ELSE, ADDING ANY OTHER EXTRA NOTES THAT VIOLATE THE RESPONSE FORMAT IS BANNED. START YOUR RESPONSE WITH 'f' fresponse format]: { Franctionccal___ 1": "First function call for (entry- entrr-pointt) Do not include anything else." "func- tion call 2": "Second function call for function fentry point). Do not include anything else.", "function ca". "Third func- tion call for function eentry.point). Do not include anything else." }
"python snisppt",,, "python.snippet", python snippet.
[math calculation]: {muthh.caccuationn)
Icalculated answer]: cralculate_,ansver
You MUST only respond in the format described below. DO NOT RESPOND WITH ANYTHING ELSE. ADDING ANY OTHER EXTRA NOTES THAT VIOLATE THE RESPONSE FORMAT IS BANNED. START YOUR RESPONSE WITH '(. [Response format]:
{ ("python snippet: "An executable Python snippet that vali- dates the accuracy of the math calculation against the caleulated answer. The Python snippet should print 'True' the calcu- lated answer is correct, and Talse' otherwise." }
Figure 7: Prompts for Query Generation
IKB-QAI
[Scientific Literature Review]
You are given a piece of text. Your task to Identify whether within the text. When you are judging the factuality of the given text, you could reference the provided evidences if needed. The provided evidences be helpful. Some evidences may contradict to each other. You must he careful when using evidences to judge the factuality of the given text. When The response should be a dic- tionary with four keys - "reasoning", , "factuality", "error", and "currection" which correspond to the reasoning, whether the given text is factual or not (Boolean- True or False), the factual errue present the text, and currected text. The following is the given text [text]: claim The following is the provided evidences levidences]: evidence You should only respond format described below. DO NOT RETURN AN ANYTHING ELSE. START YOUR RESPONSE WITH f [resporsee format]: {{ "reasoning": "Why is the given text factual non-fase factual? Be careful when you said something is non-factual When you said something is non-factua dences to support your decision." ", "error": "None if the text is factual, otherwise, describe the error. ", ";orrection"; "The cor- rected text if there is error." "factuality": True if the given text is factual, False otherwise. }}
Please generate distinct function calls for the given coding question test the You provided with two inputs, string (stringl) containing several names, and a list (listl) also con- taining names. Your task is assess whether all the last names mentioned in string] are included in listl.
there any factual
You should only respond in format as described below. DO NOT RETURN ANYTHING ELSE START YOUR RESPONSE WITH (1'' [response format]: {f "reasoning": "Explanation whether all the last names stringl are found within listl", "factuality": This will be True if all last names from stringl are
present in listl, and False otherwise. }}
Example I: [string]]: I: "J. Devlin and M. Chang" [list]]: ("De- vlin", "M Chang", " "Kristina Toutanova" II [response]: "reason- ing": "stringl contains last names 'Devlin' and 'Chang.. Both of these last names are present in listl.", "factualityy"; True Example 2: [string]]: "Tom Brown et. al" [lstt]] ("Y, Lecun", "G. Hinton"] [response]: "reasoning": "string" 1 contains 1 last name 'Brown'. Brown not present listl.", "factuality": False
you must provide mulitple evi-
Complete the following: [string]]: stringl [ist]]: list2 [re- sponse]:
Figure& 8: Prompts for Agreement Verification
19

---

Published as a conference paper at COLM 2025
E
Example cases of FACTOOL
We list the example cases of FACTOOL in each scenario.
"The modern-day version fortune cookies was invented the United States." "label": troe, "with tool classification" false,
"claim":
"with wwith wot.ttoolleasse reasoning": "The given text is not entirely factual. modern-day version fortune cookies was not invented the United States, Most people nowadays believe that fortune cookies were created by Japanese man named Makoto Hagiwara 1914 in San Francisco. Hagiwara owned what is new called the Golden Gate Park Japanese Tea Garden, where he served tea and fortune cookies. This supported by the provided evidences." "
"queries": ("Where were fortune cookies invented?", "Histury of fortune cookies in the United States"]. "evidences": ("Most people nowadays believe that fortune cookies were created by a Japanese man in 1914 in San Francisco. Hagiwara owned what now called the Golden Gate Park Japanese Tea Garden, where he served tea and fortune cookies. Nov 19, 2019", "Most people nowadays believe that fortune cookies were created by a Japanese man named Makoto Hagiwara 1914 San Francisco. Hagiwara owned what is called the Golden Gate Park Japanese Tea Garden, where he served tea and fortune cookies. Nov 19, 2019"]
Hagiwara
"claim": "Sonia Sotomayor female Supreme Court justice", "label": true,
": false, wwith_tool_rrssoning "The given text partially factual. While Sunia Subomayor indevd female US Supreme Court justice, the text fails mention that she was also the first Latina to serve on the Supreme Court. This information is provided in both of the provided evidences."
"queries": ["Is Sonia Sotomayor a US Supreme Court justice?", "Sonia Sotomayor gender US Supreme Court"), "evidences": I'In 2009, Sonia Sotomayor broke boundaries by becoming the Latina,
third woman, serve on
Supreme Court of the United States, the highest court in the country. Sotemayor was born in New York City Puerto Rican
parents.", "Supreme Court of the United States. Sonia Sotomayor 2013 the fearless federal trial court judge who saved Major League Baseball from ruinous 1995 strike 2013 entered the record book as the first Hispanic and the third woman to serve on the High Court.")
Figure 9; Reasoning errors of FACTOOL CharGPT on KB-QA
20

---

Published as a conference paper at COLM 2025
[Fact-checking recent events, discoveries, or news]
[Fact-checking high-precision statisties]
"claim": "Argentina has not won the World Cup since 1986",
"claim": "Ireland an obesity rate of 26.9%" "lahel": false,
"label": false,
"with_tool.c wow...ccassiiccatioon"" false,
"with_tool.ch too..ccassiiicaiinn.. false,
"with tool reasoning": "The given text states that Argentina has not won the World Cup since 1986, However, multiple pieces of evidence suggest that Argentina won the World Cup
"with tool reasoning": "The given text states that Ireland has an obesity rate of 26.9%, but the provided evidences show different numbers. The first evidence doean't provide clear percentage for obusity, but the second evidence states that the prevalence obesity Irish adults currently 18%, with men at 20% and women at 16%. This contradicts the given text." ", "queries": I'Ireland obesity rate statistics", "Current obesity rate in Ireland"].
in 2022.",
"queries": I"Argentina World Cup wins since 1986", "Last time Argentina won World Cup"l
"evidences", ": [" ["Argentina is one of the most successful leams in the tournament's history, having won three World Cups: in 1978, 1986, 2022. Argentina has also been runner up three times: in 1930, 1990 and 2014. In World Cup tournaments, Argentina has 47 victories in 88 matches." "Argentinu is one of the most successful teams
"evidences": ["Just under four in ten (37%) of people have a normal weight, six out of (37% overweight and a further 23% obese) uverweight or obese. A smaller difference exists between men and women aged under than these older this." ", "The prevalence of obesity in Irish adults is currently 18%, with men at 20% and women at 16%. further 47% of and 33% of women uverweight (BMI 25.0-29 kg/m2). Since 1990, obesity has more than doubled in men from 8% to 20%, increased from to 16% women." "]
tournament's history, having won three World Cups: in 1978, 1986, 2022, Argentina has also been runner up .... "Previously, the last time Argentina won 1986, when it defeated Germany to win its second title in three tourmaments." " "Argentina have won the FIFA World Cup three times 2013 1978, 1986 and 2022. They have reached the final six times, including second-place "The Argentines have won two FIFA World Cups, in 1978 and 1986. They have placed second three times before, 1930, 1990, and 2014." " Argentina last won the FIFA World Cup in 1986, defeating West Germany in the championship round, 3-2. Argentina World Cup Wins. Argentina's..." "
the World Cup
"claim": " "Everydays: The First 5000 Days' sold for $69 mil- lonn",
"label": false,
"with_tool_die tool_classification" false,
"with.tuu. nollrressonng"" "The given text almost factual, but the exact amount for which the artwork was sold is slightly off. Multiple evidences mention the correct amount.", "queries": ["Everydays: The First 5000 Days sale price", " $69 million Everydays artwork"].
"claim" "The most valuable NFT ever sold is digital artwork called 'Everydays: The First 5000 Days". "label": false,
"with_too writh.tool.cco dool.classificatoon"" false,
"evidences": ['The work collage of 5000 digital images created by Winkelmann for his Everydays series. Its associ- million at
"with tool reasoning": "The given text states that the most valuable NFT ever sold is 'Everydays: The First 5000 Days', provided evidence, The First 5000 Days' the second most expensive NFT ever sold, with 'Merge' being the most expensive one." "
non nonnfuigible token (NFT) was sold for Christie's 's in 2021, making it the most expensive non-fungible token ever., "201cNow there is a way to collect digital art. The piece was sold, Everydays: The First 5000 Days, is collage of Winkelmann's work starting ...", "That day, Winkel- mann, a digital artist more commonly known by the handle Beeple, sold 2verrvyyyyyy:: The First 5000 Days,201d an NET- linked ...", everydayyy 2014 The First 5000 201d by the artist known as Beeple, set record for digital artwork in sale at Christie's. ".", "Artist Beeple, a North Fond du Lac native, sold digital artwork for $69 million. New you the same art in a book for a lot less.", "Beeple, aka artist Mike Winkelmann, appeared on The the]."
According
"queries": | ["Most valuable NFT ever sold", "Bverydays: The First 5000 Days"].
"evidences": ['The Most Expensive NFTs Ever Sold Merge 2014 US 3918M. Pak - Merge. ...2. The First 5000 Days 2014 US$69 3M. Beeple- 2014 The First
Clock 2014 US$52.7M. Julian Assange & Pak- 2014 Clock. ...4.
Days.
HUMAN ONE- 2014 USS29M. Beeple- - 2014 HUMAN ONE. 5..5. CryptoPunk #5822 2014 US$23.7M CryptoPunk #5822.", ", "Everydays: the First 5000 Days: Work of art "Everydays: the First 5000 Days digital work of art created by Mike Winkelmann, known professionally as Beeple. The work is collage of 5000 digital images created by Winkelmann for his Everydays series." "Fverydays: the First 5000 Days Artist: Mike Winkelmann. "'veryddys: the First 5000 Days Owner: Vignesh Sundaresan (MetaKovan)." ", "Everydays: the First 5000 Days Medium: Digital (PEG)."
Figure 10: Examples of KB-QA claims verified by FACTOOL GFT-4. The queries generated, evidences collected, the reasoning process, and the final decision of FACTOOL GPT-4 is reported.
21

---

Published as a conference paper at COLM 2025
taskk id": "HumanEval/2",
"tssk id": "HumanEval/57"
"prompt":
"prompt":
def truncate_number (number: float) float: Given a positive floating point number, it decomposed into and integer part (largest integer smaller than given namber) and decimals
def monotonic(1: :: list):
Return True if list elements are monotonically increasing decreasing.
"entry point": "monotonic",
(leftover part always smaller than 1). Return the decimal part of the namber." ...
"completion":
def monotonic(1: list): len(l) <
"entry point": "truncate number", completion":
return True
increasing all <= [[i+1] for range (1)
def councate_number (number: float) float: integer-part number // decimal-part number integer-part
decreasing all([[i] [[i+]] for range (1)-1))
return increasing or decreasing
return decimal part
"result": "passed",
"passed": true,
"result"; "passed",
"with woo.ttoo..call classiffffcct.innn true,
"passed"; true,
"testcases.inpue input" ["monotonic[[1, 2, 2,3,4,5))", "monotonic([5,
colilttoooollssiiiitttooo""" true, cestccases_input""
4,3, 2, 1]]" "mom "multi solutions":
"trun-
cate cate_number(0.122 123)", "rrmmate_ummee(((99999)
"multi "multi solutions::
def monotonic(1: list): if len(1) <
def truncate number(uumber. float) -> float:
return True increasing decreasing for i in range(len(1))
number int (number))
1):
def number (number: return number intunue
(nmber float) >> float:
[[i] < 1[i + 1]:
decreasing False 1lil > 11i 1]:
increasing False if not increasing and not decreasing: False
def truncate number (number: float) >> float: number inumber))
return True
def truncate number (number: float) -> float:
integer part number // decimal.part number integer-part return decimal.part
monotonic list): if len(1) < 2:
return True direction [[]] 1[0]
for i in range len(1))
"xxec result": 599999999999999 9.9999999999996
direction 0:
0.5 0.99999999999999 0.559999999991999]] 10.123, 0.123,
direction [[i] [[i-1]
0.123, 0.123), [0.9 91999999999999999999999999999999 0,9999999999999988
continue
.9889,888,889 0198999988888831]
if (I[i] [[i-1]) direction 0:
return False
return True
def monotonic(1: list):
if len(1) 2:
return True direction 1[1] 1[0]
far i in range len(1)):
direction 0:
direction I[1] 1[1 1]
continue
(I[i] Ifi 11) direction 0:
return False
True
def monotonic(1: list)::
if len(l) return True increasing all <= [[i+1] for range (len(())1))) (1)-1))
decreasing (1l >= [[i+]] for range(Ien(1)
return increasing or decreasing
exxccresull" [[true, true, true, true), [true, true, true, true), [fals,, false, false, false]]
Figure 11: Examples of code claims verified by FACTOOL GPT4- The queries generated, evidences collected, the reasoning process, and the final decision of FACTOOL GPT-4 is reported.
22

---

Published as a conference paper at COLM 2025
[ChaiGPT]
[GPT-4]
"d": "HumanEval/36", "prompt":
"dask.id" "HumanEval/3",, "prompt":
def fizz.buzz (n: int):
bizz.buzz (n: int):
""" Return the usmber of times the digit appears in integers less than n
""* Return the wumber of times the digit appears in integers less than n which divisible by 11 13. ...
which divisible by 11 13.
enttry pointt:: "completion":
"entry-ponn poittt" "fizz.bzzz" "completion":
fizz. (nzz int): count
buzz.buzz(n: int): count for in range (n):
for in range(n): (n):
if % 11 = 0 i 13 0:
if 11 = or %% if str(i count > 0:
if str(i) count (7') > 0:
count
return count
return count
"result": "failed: AssertionError:"
"result": "failed: AssertionEriue:
"passed": false,
"passed": false,
innptt"
("fizxbuzzu(00), "fizz buzz(()
col.itiitiisssiiiiiiiin false,
"fizz buzz))000"),,
"testeases input":
"fizz (1000"
"multi solutions":
"fizz. buzz(150)"|,
"multi_solution""
def fizz.buzz (n: int):
count in ra)ge(n):
def fizz.buzz (n: int):
count
if % 11 = 0 or 13 == 0: 7 in str(i):
for in range(1, n):
or
count +=
count str count('7) return count
return count
def fizz.buzz (n: int):
def fizz. (n: int):
count for in range(n):
count
for in range n): i count str(i). countt) '7')
1% 13 = 0:
or
0:
if '7' in str
count
return count
def buzz.uuzzle (n: int):
def fizz buzz int):
count
count 0 for in (n): if % 11 = 0 1 13 count + str(i). count))
for in range n):
0: count str(i). count ' return count
or
def fizz. (izz.buzzn: int): count for in ra)ge(n):
def fizz buzz(n: int):
count for in (n): if % 11 = 0 i = 0: if str(i) count ('7)) > 0: count
11 = or % 13 0: if str(i) count (' 0: count
return count
result": [00,0,0,0] 13 (.44333)
resuult [[0, 0, 01 2, 5, wothhhooooddssssfcaaaatio true
Figure 12: Examples of code claims verified by FACTOOL ChatGPT and FACTOOL GPT-4- The queries generated, evidences collected, the reasoning process, and the final decisions of FACTOOL ChaiGPT and FACTOOL GPT- are reported.
23

---

Published as a conference paper at COLM 2025
[ChalGPT]
[GPT-4]
"math "aaahhcclculation::: "23" 4319216",
"math calculation" "23 4319216", "calculated.answer.. "99305768",
"calcelatedaasseer. "99305768", "label": fabs,
"label": false,
"python.snippee"" snippet": "print(23 4319216 99305768)", "exec_result": "False",
"query": "python
"query": ("python snippet": "calculation 4319216, an- 99305768; printicalculation answer)"), eexec result": "False",
"with. classffciica"""""""" false
"with classification:: false
"matt_.alculaaionn. "20/100 $10884297.00""
"calculated "2176859.40",
nnahh.caccllttoon" "20/100 $10884297.00",
"lahel": true,
"calculated answer": "2176839.40", "label": true,
"query": ("python_snippet" "print(abs(200100) 10884297.00 2176859-40) (176859.40)000111 0.01)"),
"query": { ("python snippet": "calculation = (20/100)
result": "True",
10884297.00; answer 2176859.66 printtabs/calsuaatioon swer) (.01)"),
"with wwthh.tool_glasifiicooo s trae
"esull" "True"
"with tool www.saiiicctiion...... true
Figure 13: Examples of math claims verified by FACTOOL ChaiGPT and GPT-4. The full response of FACTOOL GPT-1 is reported, including the queries generated, evidences collected, the reasoning process, and the final decision.
JChaIGPTI
IGPT-41
"poper title": "BERT: Pre-training of Deep Bidirectional Trans- formers for Language Understanding". paaper.authrrss" "Jacob Devlin, Ming-Wei Chang, Kenton Lee, Kristina Toutanova", "paper-puh.year" "2018", "label": true,
"paper title": "BERT: Pre-training of Deep Bidirectional Trans- formers for Language Understanding", coperr.uthorrss".. "Jacob Devlin, Wing-Wei Chang, Kenton Lee, Kristina Toutanova", "paper_pub_year"" "2018", "label": true. "with tool classification": "erorr: II
"with lool tool classification"
"eroor: II
"paper.tille": "The Impact Artificial Intelligence on Emplay-
titl"" "The Impact of Artificial Intelligence on Emplay- ment".
ment",
"paper.author(s)" "Acemoglu and Restrepo", "paper pub year": "2019", "label"" false.
"paperauthor(ss": "Acemoglu and Restrepo", 'aaper pub year": "2019", "label": false,
wwith.tooo.chsssfiaatoooo false,
"with.tool.caasiiicttton""" false, "crrur": wwrrong-aper.uuthrriy pwr.ng.ppp
"errur:: : ("wrong- puprraaoop... wwrong-pppe..gg wwwppppppp.ppprrrrr...p.
Examples scientific claims verified by FACTOOL ChaiGPT and GPT-4. The full response of FACTOOL ChatGPT and GPT-4 is reported, including the queries generated, evidences collected, the reasoning process, and the final decision.
Figure 14:
24

---

false, Published as a conference paper at COLM 2025
[Reasoning Error]
[Conflicting Evidences]
[Ambiguity in descriptions
"claim": "Jupiter is less dense than Satur",, "label": false,
"claim": "Jupiter has density of 1.33 grams per cubic centimeter", laabll: true,
"claim": "Fortune cookies are enjoyed by people all over world.", "label": false.
cll.soooccclssscaatiio""" : true, tooth ressoninn "The given text is factual because both provided evidences support the claim that jupiter is less dense than Saturn. The first evidence provides numerical data on the densities of Jupiter and Saturn, and the second evidence con- firms that Saturn the least dense planet the solar system.", "queries": I"Jupiter Saturn density", "Which planet is less dense Jupiter or Sat- um??".
"with.fuul wuw..classfficatio "with, tool real.ress.nng" "The given text is mostly in line with the provided evidences. There minor discrepancy between the two evidences, with one stating the density of Jupiter as 1.326 g/em00b3 and the other as 33 g/ s/cm00b3. However, the difference minimal and could be due rounding." " "queries", ["Jupiter density grams per cubic centimeter", "Density of Jupiter" I, "evidences": ["Densify of Jupiter: However, 1. 1,326 g/cm3, it is also the recond-moct dense the gas giants, Feb 17, 2016", "1.33 cm00b3")
"with too. clss.caasiiict truee "with_roo hachhhoolluorro real.resssn "The given text states that fortume cookies are enjoyed by people
the world. While the evi- dence provided does not explicitly mention the global enjoyment fortune cookies, it does mention that 3 billion fortune cookies are made each year globally, with the ma- jority consumed in the US. This suggests that fortune cookies are indeed enjoyed by people in various parts of the world." " "queries": I"Origin of fortune cookies", "Popularity of fortune cookies world- wide").
"evidences": ['The density of Jupiter is 1, 1,33 g/ s/mm0003 whereas the density of Sat- urn is only 0,687 g/cm00b3. If Saturn were same density Jupiter, would have radius of 30.079 km.", "Jupiter is the largest planet the solar system, but it's Satur2201tth solar system's second largest planet2014that least dense. Aug 1, 1,10"
"evidences": ["Japan", "There are approxi- mately billion fortune cookies made each year globally, the majority of them con- sumed the US. The largest manufacturer of the cookies is Wonton Food, Inc, head- quartered Brooklyn, New York. They make over 4.5 million fortune cookies per day:"I
prize
Figure 15: Some error cases of FACTOOL on KB-QA. The full response FACTOOL GPT-4 is reported, including the queries generated, evidences collected, the reasoning process, and the final decision is also reported.
25

---

Published as a conference paper at COLM 2025
[Limited variety in synthetic testcases]
[Potential errors in code generation]
"da"k.id": "HumanEval/64",
"task.id"; "HumanEval/79", "prompt":
"prompt": "Write function vomels.countt) which takes a string representing a word as input and returns the number vowels in the string. Vowels in this case are '', 'c', Y, 'o', 'u'. Here, y also vowel, but only when at the end the given word.", "
def decimal_to. -toarr (decimal): """You will be given number in decimal form consert
your
hinary format.
entry point"; "vowels count", "completion":
The function should refarn string
with each character representing binary number. Each character in the string will be "D" or 'I'
def covetsscount(s):
There will an extra couple of characters "db
vowels count
ee', '1", 'o', 'I'I
the beginning
and at the end of the string- extra characters are there to help with the format.
for i range (ange(len(s)) if s[i] in vowels:
count += if [s] and s[i] count ==
"entry-pointt:
"entry point": "decimal to tiibarr,"" "completion".
"result": "falled: AssertionError: Test
def decimal_to toniiarrrt.bbiarry...aaaaa (decimal) binary-list []
"passed"; false,
"with totooooooolsssit Lelassification". true,
while decimal
"testcases input": I'yowels.counff "'uwee-ouunftttyyloooppa"aa encycoopedcc"""""" "vow-
remainder decimal %
els count( (attttttttt""" "vowels count| count[/heautifeeei beautiful"""
binary. (tr..liis..pend((s(((((( (trrmmidttrr
"multi solutions":
decimál decimal // hinary list. rist.reverse
def vowels.count(s): vowels = aeiou
(iionnniiaar...... (binary.list) binary.striin binary-string 'db"
''joinn
return db
count 0 for i, c in enumerate(s):
"result": "failed: AssertionErroor",
lower () in vowels:
"passed"; false,
count +=
"with classification":: trse,
elif c.lower == 'y' and len count
"esttcase_.inutt"
["decimal torcimmlLo.bbiaar((1))
"deci-
mal.to bii.atobbia "decimal (tecimal.o.biiarr(0""
return count
"multi solutions":
def decimal to Sinary(deciman
def cowels)count(s)::
binary_num
vowels "eeoou" count
while decimal > 0:
remainder decimal
for char s:
char. lower() vowels:
decimal decimal //
binary .uum str (remainder) binary-num
count +=
return 'db" binary
ddb'
s[=]].lower() yy
count +- return count
def decimal.to.binary (decimal):
binary
while decimal
def cowels_count(():
remainder decimal %
vowels 'aeiou" count = char if char. char.lower() in vowels: count
decimal = decimal //
binary (temainind))) binary
return dbb + binary * dbb"
def decimal to binary (decimal):
if s[=r].lower () 'y': count +=
binary
while decimal 0:
remainder decimal
decimal decimal binary = stm(aiiindd + binary
vowels (ovelss.ount(s):
vowels = l'a', 'e', '1', 'o', 'u'] count =
db binary dbb
for in range if slil in vowels: count if i == (sn and slil "y' count return count
def decimal _to to__inary(ddeimm...
binary list = I] while decimal > 0:
remainder decimal % appddtt.
decimal decimal
binary binary_list. riserse)
exec result": 5[5,5,5,5, 0, 55,5,5,5]
binary.string ".join (binaairryy_ return "db binary-string 'db'
1.
[["b1010db", "bb1010b"" "db101оdb" ["abi1111iab", "ah111111b""" "b11111114b" "αbлииииa" p'abdb", "dbdb". "dbdb", "dbdb"]]
"exee result":
"db1010ab"],
Figure 16: Some error of FACTOOL on code. The full response of FACTOOL GPT-1 is reported, including the queries generated, evidences collected, the reasoning process, and the final decision also reported.
26

---

Published as a conference paper at COLM 2025
[Round-off error]
"muth.calculation" "60440034 12",
"calculated answer": "5037000.83", "label"; true, "query": :nithotnsniippt... "calculation 60444034 12; answer 3037002.8) pranaacciitttaallllclllltttnnnn answer) 1e-9)" "exec_reuul"" "False",
"with writh_tool_clase ttw.toolllssfficatiio"" false
[Reasoning error]
"input": "Kylar went to the store to buy glasses for his new apartment. One glass costs $5, but every second glass costs only 60% of the price. Kylar wants buy 5364765 glasses. How much does he need to pay for them?",
"target": 21459061.0,
price $5 each. Therefore, he buy (5364765 glasses at discounted price of 60% of S3, which is $3, Now, can set the equation:
"response":
assume
Kylar buys glasses
5x + 3(5364765- -X) total cost
Simplifying the equation, 5x + 16094295- -3 total cost
get:
16094295 total
Therefore, the total cost of all the glasses will be 2x + 16094295 dollars. Now, we just need to find the value of We know that
Kylar wants to buy 5364765 glasses, so we can substitute this value the equation:
2x + 16094295 total cost
16094295 5,364,765
16094295- 26823825
2x = 10729530
x 5364765
Therefore, Kylar needs to pay 2x + 16094295 dollars, which is: 2(5364765) 16094295 26823825. Hence, Kylar needs pay $26823825 the glasses."
"answer": "26823825",
"annotation": false,
"claims": [[" ('"math_cacclaatoon:: "5364765-x" calcullted.ansserr "5364765-x" "label": true). ("mmath.clculatton" "0.60 5", Caleulated.answer "3", "label": true). I"maahhaaacclation"" "5x", "calculatedlansser"""" "3x", "label": true), caluuaatlcl.
"5x 16094295 3xx,, "caluu- lated answen":" "16094295 2x", "label": true), ("math calculation": "2x 16094295", "caleulated answer": "total cost", "label": "null"), " ("math.calculation "5364765* "calculated answerr" "26823825", "label": true), ("malh mmathhcalculation "2(5364765),, "calculated answer": "10729530", "label": true). ("math calculation". ": "2x", "calculated answer": "10/29530", "label": "null")] "math, calculation": "5364765 x", "calculated answer": "5364765 x", "label": true, "query": : "python printmatth.aalculation
"3(5364765 x)", "calculated answer"; "16094295 3x", "label": true), "mmaathcalculation"
"x calculated ansserated.msswer""" wow.hooocclsssfcattiin true "matt.calculation... "0.60 5", "caleulated answer": "3", "label" true, "query": "python snippet": "calculation
D: math-calculation 3364765
caleulated.answer 5364765
"exec_result"; "True",
5; answer 3; printicaleulation answer)", , "exer result": "True", "with tool classification" true "math calculation" "5x", "calculated answerr" "5x", "label": true, "query": "python "python.snippet" snippet": "def validate calculation(ckculation answer): return calculation == answer, print(validate calculation! 5xx" '5x")" "exec result": "True", "with tool classification,. true "math calculation" "3(5344765 x)", "calculated answer": "16094295 3x". "label": true. "query"; apython_snipper "x= resultl =3 (5364765 x); result2 16094295 33 3", priitfrrrssultt == result2y", "exer_resule" "True", wot.h
"16094995 2x", "la- == (16094295 + +2* x))", "exer result": "True", "with tool classification": true "math calculation": "2x 16094295", "caleulated answer": "total cost", "label": "null", "query": "python snippet") "x calullaann =2 x 16094295; calculated annserr 16094305, printimath htttmaatthaalllaaati lated Lanswer)", , "eseclresult": "True", "wittoooo.classiiitttto""" true "aluuaaal"llation "5364765 5", "calculated.ansseer"" "26823825", "label": true, "query": "pythun snippet: "calculation = 5364765 answer = 26823825; printicaleulation answer)", result""" "True", woith_too__lasssficattee true "math maath_aalclatioon" "2(3364765)", "calculated "calculatee_pnswer"" answer": "10729530", "label": true, "query": pythonnsnippets "calculation =2 5364765; answer 10729530; print(ealculation
"math.calculation. "5x 16294295
calculated
bel": true, "query": : "python.snippet" "x= 1; print((5 prin(([
16094295
calcu-
answer)". "esec_result":
"True", "with tool classification". true "math calculation": "2x", "calculated answerr:: "10729630", "label": "null", "query": snippet"; "python.stipppt";
"python
5364765; print(2 5364765)print(2 x 10729530)", result:" "True", "with tool classificattion" true
Figure 17: Some eттor cases of FACTOOL on math. The full response of FACTOOL GPT-4 is reported, including the queries generated, evidences collected, the reasoning process, and the final decision is also reported.
27

---

Published as a conference paper at COLM 2025
[Errors in title matching]
"paper_title": "MDMA -assisted psychotherapy for treatment of PTSD: shidy design and rationale for phase trials based pooled analysis of six phase 2 randomized controlled trials", "paper author(s)": "Mithoefer al.", paaeer.ub.yyee. puar,ear": "2019",
"label": true,
withh.ooolede cla.sooacccsssccatiiio"" : false,
"ervor": ("wrong paciggg-ppp..ttee title")
[Errors in author matching]
ppaper.ttlee". "Language Models are Unsupervised Multitask Learners", "paper_authorss) "Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Darlo Amodel, Ilya Sutskever", paper.pubbyyerr' "2019",
"label": true,
witthtooo_clessffiattion" false,
"error": ["wrong paper author(s)"]
Figure 18: Some cases of FACTOOL scientific. The full response of FACTOOL GFT-4 is reported, including the queries generated, evidences collected, the reasoning process, and the final decision is also reported.
28


Reviews:
[{"review_id": "vDxzO9NLZc", "reviewer": "Reviewer_jo5j", "summary": "The paper introduces a framework, FacTool, to identify the factual errors in the generations from LLMs on a variety of tasks. FacTool consists of five steps and is able to perform on long-form generations without fixed evidence provided, utilizing the instruction-following ability of the LLM along with a set of external tools (e.g., Google Search API).The paper evaluates a set of modern models using this framework.", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. The paper is well-organized and easy to follow.\n1. The experimental setup is clear with a rich number of details.\n1. The paper identifies the existing challenges and explains how the proposed framework addresses them.", "reasons_to_reject": "1. It is valuable to have an integrated framework to identify factual errors in LLM’s generations in a wide range of domains, but the novelty of individual components in FacTool seems to be limited.\n1. In Section 6.1, there is a lack of baseline to compare against the proposed claim extraction method (e.g., [1]).\n1. In Section 6.2, for KB-QA and Scientific, a baseline using RAG with a local datastore might be needed to showcase the benefits of using this agentic framework. For evaluating code and math, adding a stronger baseline would make the results more convincing as well (e.g., a minimal pipeline to execute generated code).\n1. Showing the transferability of FacTool on other datasets is important especially because math and coding generations can be dependent on the prompts and vary in format and style. However, there are no such experiments shown. Even in Section 6.3, the prompts were taken from the same set of datasets.\n1. The numbers of prompts evaluated in Section 6.3 seem to be too small especially for domains other than KB-QA.", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "nD2jHSK57s", "reviewer": "Reviewer_BTty", "summary": "This paper proposed a tool augmented factuality check method for AI generated contents. By leveraging the specific tools for different domains, the proposed method can effectively check the factuality of in different domains for different tasks. To evaluate the proposed method, the authors also built a benchmark. Finally, the authors evaluated a couple sota LLMs with the proposed method, across different scenarios. \n\n### quality\nThe paper looks sound to me. The way this proposed method evaluates the generated output is very similar to how human would be evaluating them, so it's not surprising that it would work better than simple self-check.\n\n### clarity\nThis paper is overall clearly written and easy to follow.\n\n### originality \nI'm not over familiar with literature on this topic, but according to the authors this is the first work that integrates \"tool use\" for factuality detection.\n\n### significance\nI'd imagine that the ultimate goal of such method is to be a standalone factuality check tool, or integrated into LLM applications to control the generation quality. But the cost would be fairly high either way, which may limit the practical usefulness of the proposed work.", "questions": "", "limitations": "", "rating": 6, "confidence": 1, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. paper is clearly written and easy to follow\n2. proposed method is reasonable and works well", "reasons_to_reject": "1. the proposed method seems costly with unsure practical usability", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "EnfgmFccTk", "reviewer": "Reviewer_qwzF", "summary": "The paper introduces FactTool, a task- and domain-agnostic framework designed to detect factual errors in texts generated by LLMs. The framework leverages tools such as Google Search, Google Scholar, code interpreters, and even LLMs themselves to gather evidence about the factuality of the generated content. It consists of five main components: claim extraction, query generation, tool querying, evidence collection, and agreement verification. The authors evaluate FactTool across four different tasks from knowledge-based QA, code generation, mathematical reasoning, and scientific literature review, and compare its performance with self-check baselines. The results demonstrate that FactTool outperforms the baselines in detecting factual errors across all scenarios.", "questions": "", "limitations": "", "rating": 4, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- FactTool is a relatively general framework\n- Wide exploration of tools", "reasons_to_reject": "- Limited novelty in individual components: Though the integration of tools into a unified framework is somewhat innovative, each individual component of FactTool lacks sufficient novelty and relies on methods akin to existing approaches. This makes the paper seem like a mere engineering assembly rather than a work that proposes or solves a novel research problem. And the framework seems simply like 1+1=2, which means the readers can not obtain any insights from the \"unified\" framework.\n- Bias in tool outputs not discussed: The use of external tools introduces the possibility of bias in the collected evidence. For example, search engine results can be influenced by various factors, which may compromise the objectivity of the factuality assessment.\n- Computational and resource constraints: The tool-augmented approach may demand substantial computational resources and time, especially when dealing with large volumes of generated text or complex queries. This could undermine the practicality and scalability of the framework in real-world applications.\n- Insufficient experiments: The experiments appear inadequate as the models used seem somewhat outdated. There is no comparison with the latest models, such as DeepSeek-R1 and other reasoning models.\n- Prompt dependency: FactTool is based on prompt implementation, which heavily relies on the model's ability to follow instructions and is highly sensitive to the prompt itself.\n- Overclaim: The paper claims to be “the first to propose and connect the concept of ‘tool use’ with ‘factuality detection’”. However, numerous prior works have already employed tools to perform a variety of tasks, including factuality detection (see 1).\n\nTypo: In line 70, “scientific” should be corrected to “science”.\n\nMissing references:\n[1] Vykopal et al., Generative Large Language Models in Automated Fact-Checking: A Survey.", "questions_to_authors": "N/A", "extra_scores": {}}, {"review_id": "oFacjJA56T", "reviewer": "Reviewer_7gdn", "summary": "The paper proposes FacTool, a multi-task and multi-domain framework for detecting factual errors in LLM-based generated texts. The core idea of FactTool is to provide a set of tools that can be used for factuality detection across different domains and tasks proposing a uniform framework. In this paper, FacTool is used to evaluate the factuality of a diverse set chatbots. FacTools consists of five components: 1) claim extraction, 2) query generation, 3) tool querying, 4) evidence collection, and 5) agreement verification which are applied in this order (see figure 1.) for the different domains and tasks under consideration.  The components 1) and 2) are using ChatGPT with specific prompt instructions. Component 3) is based on a Google Search API for candidate evidence retrieval. The task of component 4) is to label each extracted claim as true or false taking into account the extracted evidence candidates. All these steps have to be fine-grained for the specific tasks under consideration. Four tasks are considered: knowledge-based QA, code generation, mathematical reasoning, and scientific literature review. The prompts used for the components are developed and evaluated using a number of task-relevant third-party data sets. The evaluation results suggest that the approach is effective.", "questions": "", "limitations": "", "rating": 7, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- interesting approach for factuality extraction using a zero-shot/few-shot approach\n\n- good results achieved", "reasons_to_reject": "- claim extraction and framework only evaluated on small set of LLMs (basically GPT-4, ChatGPT and Flan-T5-XXL). \n\n- it sounds unfair to use ChatGPT also as part of the FactTools because it might cause some positive bias for ChatGPT compared to non-OpenAI models in Table 5. On the other hand, why is GPT-4 so much better?\n\n- Whole approach seems to be a pipeline but concrete interaction is unclear. Are there any problems with error propagation?\n\n- missing qualitative discussion ion the main part of the paper", "questions_to_authors": "- see above\n\n- can you explained the extreme differences in results for self-check and FactTool for scientific task in Table 4?", "extra_scores": {}}]

Author rebuttal:

Meta-review: The paper presents FacTool, a framework for factuality evaluation of LLM outputs using tools. 
The framework includes claim extraction, query generation, tool execution and verification.
The work presents the framework, together with experiments, results, data and code.

Reasons to accept: 
- The work is sound and has already showed significant impact since it was published on arxiv.
- The open source software included with the work is also impactful and used by many.
- Long-form factuality is still an important problem for LLMs today, and this work is useful to drive progress on this.

Reasons to reject: 
- The work is not cutting edge research: early versions appeared two years ago (but maybe not published in a conference before).
Decision: Accept