Explainable Claim Verification via Knowledge-Grounded Reasoning with Large Language Models
Authors: Haoran Wang
Kai Shu
Venue: EMNLP 2023 Findings (2023)
Keywords: Claim Verification
Natural Language Reasoning
Large Language Model

Abstract: Claim verification plays a crucial role in combating misinformation. While existing works on claim verification have shown promising results, a crucial piece of the puzzle that remains unsolved is to understand how to verify claims without relying on human-annotated data, which is expensive to create at a large scale. Additionally, it is important for models to provide comprehensive explanations that can justify their decisions and assist human fact-checkers. This paper presents First-Order-Logic-Guided Knowledge-Grounded (FOLK) Reasoning that can verify complex claims and generate explanations without the need for annotated evidence using Large Language Models (LLMs). FOLK leverages the in-context learning ability of LLMs to translate the claim into a First-Order-Logic (FOL) clause consisting of predicates, each corresponding to a sub-claim that needs to be verified. Then, FOLK performs FOL-Guided reasoning over a set of knowledge-grounded question-and-answer pairs to make veracity predictions and generate explanations to justify its decision-making process. This process makes our model highly explanatory, providing clear explanations of its reasoning process in human-readable form. Our experiment results indicate that FOLK outperforms strong baselines on three datasets encompassing various claim verification challenges. Our code and data are available.

Explainable Claim Verification via Knowledge-rrounde Reasoning with Large Language Models
Haoran Wang Kai Shu Illinois Institute of Technology, Chicago, IL, USA hwang21 9@hawk. iitiaaww.iii.................... kshu@iit. edu
Abstract
Claim: Luthohslo wen silver medal 2012 SportAccord World Mind Garnes inmugurated July 2011 in Reijing.
Claim verification plays crucial role in com- bating misinformation. While existing works claim verification have shown promising results, a crucial piece of the puzzle that re- mains unsolved is to understand how to ver- ify claims without relying human-annotated data, which is expensive to create at a large scale. Additionally, is important for mod- els provide comprehensive explanations that can justify their decisions and assist human fact-checkers. This paper presents Frrst-rrder- Guide- Knowledge- Grounder (FOLK) Reasoning that can verify complex claims and generate explanations without the need for notated evidence using Large Language Mod- els (L.LMs). FOLK leverages the in-context learning ability of LLMs to translate the claim into First-Order-Logie (FOL) clause consist- ing of predicates, each corresponding a sub- claim that needs to be verified. Then, FOLK performs FOL-Guided reasoning over a set of snowledge-groundeer questton-and-answer pairs to make veracity predictions and gener- ate explanations to justify its decision-making process. This process makes model highly explanatory, providing clear explanations of its reasoning process numan-readable form. Our experiment results indicate that FOLK out- performs strong baselines on three datasets encompassing various claim verification chal- lenges. Our code and data are available.
Label: (NOP SUPPORTED]
Predicates:
Wont Kondlo, silver medal)) Verify Lubohako Kondlo won silver medal
Inaggratedcche SportAccord World Games, July Beijing) :: Vendy the 2012 SportAccord World Mind Games was inzaguraate in July Boijing.
Follow-up Question: What did Lubabalo Kondlo win in the 2012 SportAccord World Gumes!
Grounded Answer: In 2012 be won the silver medall in Beijing, China.
Follow-up Question: When where was SportAcoond World Mind Games inuogurated?
Grounded Answer: Insernational Mind Sports Association (IMSA)
gurated the Apoord World Mind Games Devember 201 in Beijing
WannLababalo Kondlo, a silver medal) is True because In 2012 he won the silver medal SportAccord World Mind
Beijing, Chins.
Inauguratedihhe 2012 Aportd World Mind Games, July 2011, Heijing) False heaause International Mind Sports Associatio (IMSA) inmugurated the Aporrd World Mind Games Desember 2011 in Reijing Labaaaaaa Kondlo, silver medil) & Inauguratedittbe 2012
SportAccord World Mind Games, July 2011, Beijing) False. The claim is |NOT SUPPORTED).
Explanatio:
Lahabalo Kondlo wes a silver medal in the 2012 Sportccard World Mind Games, However. the event inaugurated in December 2012. July 2011. in Beijing.
Table 1: An example from FOLK with GPT-3.5 on HOVER, multi-hop claim verification dataset. We first use LLM to translate the claim into a First- Order- Logic clause (highlighted in orange), consisting of two predicates (highlighted in blue and purple). The LLM then perform knowledge- grounded reasoning to predict label and generate explanation.
knowledge-rrounded
of claim detection, evidence retrieval, verdict pre- diction, and justification production. Despite some early promising results, they rely on the availabil- ity of large-scale human-annotated datasets, which pose challenges due to labor-intensie intensive annotation efforts and the need for annotators with specialized domain knowledge. To address the issue of creat- ing large-scale datasets, recent works (Pan et al., 2021; Wright et al., 2022; Lee et al., 2021) focus on claim verification zero-shot and few-shot scenar- ios. However, these methods follow the traditional claim verification pipeline, requiring both claim
1 Introduction
Claim verification (Guo al., 2022) has become increasingly important due to widespread online misinformation (Tian et al., 2023; Jin et al., 2023). Most of the existing claim verification models (Zhou et al., 2019; Jin et al., 2022; Yang et al., 2022; Wadden et al., 2022b; Liu et al., 2020; Zhong et al., 2020)
automated pipeline that consists
hhttps: //gl:::////a www.tt...w...aaaeeeeeee.eee..

---

and annotated evidence for veracity prediction. Ad- ditionally, these models often lack proper justifi- cations for their predictions, which are important for human fact-checkers to make the final verdicts. Therefore, we ask the following question: Can we develop model capable of performing claim verifi- cation without relying annotated evidence, while generating natural language justifications for its decision-making process?
experiment results demonstrate that FOLK can ver- ify complex claims while generating explanations to justify its decision-making process. Additionally, we show the effectiveness of FOL-guided claim de- composition and knowledge gegouddddde reasoning for claim verification.
In summary, contributions are:
We introduce a new method to verify claims without the need for annotated evidence.
To this end, we propose novel framework showledge-Groundee (FOLK) to perform explainable claim verification by leveraging the reasoning capabilities of Large Language Models (LLMs) (Brown et al., 2020; Touvron et al., 2023; Zhang et al., 2022; Chowdh- cry et al., 2022). Figure illustrates a real-world example from FOLK where it can provide veracity prediction based on logical reasoning and generate explanation of its decision-making process in short paragraph. To ensure accurate prediction and provide high- qughltualtty explanations, FOLK first translates the input claim into a Lirst- (FOL) (Enderton, 2001) clause consisting of a set of conjunctive predicates. Each predicate repre- sents part of the claim that needs to be verified. Next, the generated FOL predicates guide LLMs to generate a set of questions and corresponding answers. Although the generated answers may ap- pear coherent and plausible, they often lack factual accuracy due to LLM's hallucination problem (Ji et al., 2023; Ouyang et al., 2022). To address this problem, FOLK controls the knowledge source of the LLMs by grounding the generated answers in real-world truth via retrieving accurate informa- tion from trustworthy external knowledge sources (c.g. Google Wikipedia). Finally, FOLK lever- ages the reasoning ability of LLMs evaluate the boolean value of the FOL. clause and make a ve- racity prediction. Given the high stakes involved in claim verification, FOLK prompts the LLMs to generate justifications for their decision-making process in natural language. These justifications are intended to aid human fact-checkers in making the final verdict, enhancing the transparency and interpretability of the model's predictions.
We demonstrate the importance of using sym- bolic language to help claim decomposition and provide knowledge knowledge-rrounding grounding for LLM perform reasoning.
We show that FOLK can generate high-quality explanations to assist human fact-checkers.
Background
Claim Verification. The task of claim verifica- tion aims to predict the veracity of claim by trieving related evidence documents, selecting the most salient evidence sentences, and predicting the veracity of the claim as SUPPORTS or RE- FUTES. Claim verification falls into the broader task of Fact-checking, which includes all the steps described in claim verification with the addition of claim detection, step to determine the check- worthiness of a claim.
While steady progress has been made in this field, recent research focus has shifted to 1) deal- ing with insufficient evidence (Atanasova et al., 2022) and 2) using explainable fact-checking mod- els to support decision- maasiionmmainng (Kotonya and Toni, 2020a). In the line of explainable fact-checking, (Popat et al., 2018) and (Shu et al., 2019) use visual- ization of neural attention weights as explanations. Although iitention-based explanation can provide insights into the deep learning model's decision process, it does not generate numan-readable expla- nations and cannot be interpreted without any prior machine learning knowledge. (Atanasova et al., 2020; Kotonya and Toni, 2020b) formulate the task of generating explanations as extractive summa- rizing of the ruling comments provided by profes- sional fact-checkers. While their work can generate high-quality explanations based on training data from professional fact-checkers, annotating such datasets expensive and not feasible at a large scale. Our work explores using reasoning steps as explanations of the model's decision-making
We evaluate our proposed methods three fact- checking datasets (Jiang et al., 2020; Aly al., 2021; Wadden et al., 2022a) with the following dis- tinct challenges: multi-hop reasoning, numerical reasoning, combining text and table for reasoning, and open-domain scientific claim verification. Our

---

process while generating explanations in natural language.
logue and question answering. In contrast their approach, our primary focus on providing LLMs with knowledge-grounde
Large Language Models for Reasoning. Large language models have demonstrated strong rea- soning abilities through chain-of-though of-thought (CoT) prompting, wherein LLM is prompted to gener- ate its answer following a step-by-step explanation by using just a few examples prompts. Recent works have shown that CoT prompting im- prove performance on reasoning-hess tasks such multi-hop question answering, multi-step com- putation, and common sense reasoning (Nye et al., 2021; Zhou et al., 2022; Kojima et al., 2022).
facts to enable FOL- Guided reasoning for claim verification, rather than solely concentrating on enhancing the factual accuracy of LLMs' responses. ProgramFC (Pan et al., 2023) leverages LLMs to generate computer- program-like functions to guide the reasoning pro- cess. In contrast to their approach, which only uses LLMs for claim decomposition, we use LLMs for both claim decomposition and veracity predic- tion. By using LLMs for veracity prediction, we can not only obtain comprehensive understand- ing of LLMs' decision process but also generate explanations for their predictions. Furthermore, ProgramFC is limited to closed-domain, as it needs to first retrieve evidence from a large textual corpus like Wikipedia. FOLK on the other hand, can per- form open momn-domain claim verification since it does not require pre-defined evidence source.
Verifying complex claims often requires multi- step (multi-hop) reasoning (Mavi et al., 2022), which requires combining information from multi- ple pieces of evidence to predict the veracity of a claim. Multi-step reasoning can be categorized into forward-reasoning and backward-reasoning (Yu al., 2023). Forward-reasoning (Creswell al., 2022: Sanyal al., 2022; Wei et al., 2022) em- ploys bottom-up approach that starts with existing knowledge and obtains new knowledge with infer- ence until the goal is met. Backward-reasoning (Min et al., 2019; Press et al., 2022) on the other hand, is goal-driven, which starts from the goal and breaks it down into sub-goal until all of them solved. Compared to forward reasoning, backward reasoning more efficient, the wwvide-and-conquee search scheme can effectively reduce the problem search space. We propose FOLK, a FOL-guided backward reasoning method for claim verification.
Method
Our objective is to predict the veracity of a claim without the need for annotated evidence while generating explanations to elucidate the decision- making process of LLMs. As shown in Figure 1, our framework contains three stages. In the FOL- Guided Claim Decomposition stage, we first trans- late the input claim into a FOL clause P, then we use P guide LLM generate set of intermedi- ale question-answer pairs (q:, a.). Each intermedi- ate question 4 represents specific reasoning step required to verify the claim. In the Knowledge- Grounding stage, each ag represents the answer generated by LLMs that has been verified against ground truth obtained from an external knowledge source. Finally, in the Veracity Prediction and Ex- planation Generation stage, we employ P guide the reasoning process of LLMs over the knowledge- grounded question ceesssioo-nnd-aaa and-answer pairs. This allows us to make veracity predictions and generate justi- fications for its underlying reasoning process.
Despite the recent progress in using LLMs for reasoning tasks, their capability in verifying claims has not been extensively explored. (Yao et al., 2022) evaluate using LLMs to generate reasoning traces and task-specific actions fact verification tasks. Their reasoning and action steps are more complex than simple CoT and rely on prompting much larger models (PaLM-540B). Additionally, they test their model's performance on the FEVER dataset (Thorne et al., 2018), 8), which lacks many- hop relations and specialized domain claims. In contrast to their approach, our proposed method demonstrates effectiveness significantly smaller LLMs without requiring any training, and we test our method on scientific claims.
3.1 FOL-Guided Claim Decomposition
Although LLMs have displayed decent perfor- mance in natural language reasoning tasks, they fall short when asked to directly solve complex reasoning problems. This limitation arises from the lack of systematic generalization capabilities in language models (Valmeckam et al., 2022; Elazar et al., 2021). Recent works have discovered that
Contemporaneous to our work, (Peng et al.. 2023) propose set of plug-and-play modules that augment with LLMs to improve the factuality of LLM--gnnrated responses for task-oriented dia-

---

Multi-Hop Reasoning:
Numerical Reasoning:
Open-Domain Scientific:
Tomás Smid and Fabrice
Michihira had two wives and three children during his short lifetime of 47 years.
Hypertension can be accurately diagnosed ambulatory blood pressure monitoring.
Santoro were both American tennis players.
CLAMM
K
First-Order-Logic (FOL)
2. Retrieve Knowledge-Grounded Answers
Veracity Prediction and Explanation Generation
Guided Claim Decomposition
PREPITIOON
REFUTE
SUPPORT
SUPPORT
EXPLANATION:
Tomáé Smid former from Ezsahoabuslis, Fabrica fannis player. Therefore, they
Morosada doduhtaa Kinakl
Njo Michihire doughint Njo
Ambulstory Blood Pressure Montorinng
(ABPM) anuacoaate diagnosa
and dlcuurttt another who adopted was from Fatruary 27, 2338 which years.
American players.
Tomnokouj family
hypertension
blood pressure readings.
Figure 1: Overview of our FOLK framework, which consists of three steps: (i) FOLK translates input claim into FOL clause and uses it to guide LLMs to generate set of question-and-ansee answer pairs; (ii) FOLK then retrieves mnowledge-grounded answers from external knowledge-sonee souree; and (iii) FOLK performs FOL-Guided reasoning over knowledge- inowiedge-grounned grounded answers to make veracity prediction and generate explanations. FOLK can perform variety of reasoning tasks for claim verification, such as multi-hop reasoning, numerical reasoning, and open-domain scientific claim verification.
LLMs are capable of understanding and converting textual input into symbolic languages, such as for- mal language (Kim, 2021), mathematical equations (He-Yueyy et al., 2023), Python codes (Gao et al., 2022). Inspired by these recent works, we harness the ability of LLMs to translate textual claims into FOL clauses. This allows us to guide LLMs in breaking down claims into various sub-claims.
3.2 Retrieve Knowledge-Grrunned Answers
Although LLMs exhibit the ability to generate co- herent and well-written text, worth noting that they can sometimes hallucinate (Ji et al., 2023), and produce text that fails to be grounded in real-world truth. To provide wwwwlede--groudder answers for the generated intermediate questions, we employ retriever based on Google Search, via the Ser- PAPI service. Specifically, we return the top-1 search result returned by Google. While is im- portant to acknowledge that Google search results may occasionally include inaccurate information, it generally serves as a more reliable source of knowl- edge compared to the internal knowledge of LLMs. Additionally, in real-world scenarios, when human fact-checkers come across unfamiliar information, they often rely Google for assistance. There- fore, consider the answers provided by Google search as mhowledge-grounder answers.
At this stage, given the input claim C, the LLM
first generates a set of predicates P = [pi.... Pn. that correspond to the sub-elaims C = [ct, .... Cn]
Each predicate P; E P is First-Orddr Logic (FOL) predicate that guides LLMss prompt question- and-answer pair that represents sub-claim c.. The claim C can be represented as a conjunction of the predicates = P1 A p2 A ... Pa. To classify the claim C as SUPPORTED, all predicates must eval- uate to True. If any of the predicates are Faleaaase False, the claim is classified as REFUTED. By providing the LLMS with symbolic languages such as pred- icates, alongside few in-context examples, observe that LLMs can effectively identify the cru- cial entities, relations, and facts within the claim. Consequently, L.MSs are capable of generating rel- evant question-ane and-wnswer pairs that align with the identified elements.
3.3 Veracity Prediction and Explanation Generation
At this stage, the LLM is asked to make a verdict prediction V c (SUPPORT, REFUTE) and pro- vide explanation & justify its decision. Veracity Prediction Given the input claim C, the
Phttps://serpapi.

---

predicates [pi,,..,Phl and knowledge-grounded question-and-answe pairs, FOLK first checks the veracity of each predicate against corresponding knowledge-groune gerouddddg answers while giving reasons behind its predictions. Once all predicates have been evaluated, FOLK makes final veracity pre- diction for the entire clause. In contrast to solely providing LLMs with generated questions and their corresponding grounded answers, we found that the inclusion of predicates assists LLMs in identifying the specific components that require verification, allowing them to offer more targeted explanations. Explanation Generation We leverage LLMs' ca- pability to generate coherent language and prompt LLMs to generate paragraph of human-readable explanation. We evaluate the explanation gener- ated by LLMs with manual evaluation. Further- more, since claim verification is high-stake task, it should involve human fact-checkers to make the final decision. Therefore, we provide URL links to the relevant facts, allowing human fact-checkers to reference and validate the information.
hop reasoning, and combining tables and text, SeiFact-Open (Wadden et al., 2022a) is a test- ing dataset for scientific claim verification. This dataset aims to test existing models' claim verifica- tion performance in an open-domain setting. Since the claims in SciFact-Open do not have a global label, we select claims with complete evidence that either support or refute the claim and utilize them as the global label. This dataset tests our model's s performance on specialized domains that require domain knowledge to verify.
4.2 Baselines
We compare our proposed method against the fol- lowing four baselines.
Direct This baseline simulates using L.LM as standalone fact-checkers. We directly ask LLMs to give us veracity predictions and explanations given an input claim, relying solely on LLMs' internal knowledge. It is important to note that we have no control over LLM's knowledge source, and it is possible that LLMs may hallucinate.
4 Experiments
Chain-of-Thougee (Wei et al., 2022) is a popu- lar approach that demonstrates chains of inference to LLMs within an in-context prompt. We decom- pose the claims by asking LLMs to generate the necessary questions needed to verify the claim. We then prompt LLMs to verify the claims step-by-step given the claims and chowledge-groundee answers.
We compare FOLK to existing methods on claim verification challenges from three datasets. Our experiment setting is described in Sections 4.1 & 4.2 and we discuss our main results in Section 4.4.
4.1 Datasets
We experiment with the challenging datasets listed below. Following existing works (Yoran et al., 2023; Kazemi et al., 2022; Trivedi al., 2022), limit the overall experiment costs, we use strat- ified sampling to select 100 examples from each dataset to ensure a balanced label distribution.
Self-Ask (Press et al., 2022) is a structured prompting approach, where the prompt asks LLMs to decompose complex questions into casier sub- questions that it answers before answering the main question. It is shown to improve the perfor- mance of Mhain-of-Thougee answering tasks. We use the same decomposition and knowledge -rrouddiiggg processes as in COT. For veracity prediction, we provide both questions and mnwwledge-groundee answers to LLMs to reason, instead of just the knowledge-grounded answers.
multi-hop question-
HORER (Jiang et al., 2020) is multi-hop fact verification dataset created to challenge models to verify complex claims against multiple informa- tion sources, or "hop". We use the validation set for evaluation since the test sets are not released publicly. We divide the claims in the validation set based on the number of hops: two-hop claims, hope claims, and four-hop claims.
ProgramFC (Pan et al., 2023) is a recently pro- posed baseline for verifying complex claims using LLMs. It contains three settings for knowledge- source: gold-evidence, open-book, and closed- book. To ensure that ProgramFC has the same prob- lem setting as FOLK, we use the open-book setting for ProgramIC. Since we only use one reasoning chain, we select N=1 for ProgramFC. Since Pro- gramFC cannot perform open open-domain claim verifi- cation, we exclude it from SciFact-Open dataset.
FEVEROUS (Aly et al., 2021) is a benchmark dataset for complex claim verification over struc- tured and unstructured data. Each claim is anno- tated with evidence from sentences and forms in Wikipedia. We selected claims in the validation set with the following challenges to test the effective- ness of our framework: numerical reasoning, multi-

---

HoVER
FEVEROUS
SciFact-Open
2-Hop
3-Hop
4-Hop
Numerical
Multi-hop
Text and Table
Direct
57.11
44.95
55.91
48.52
50.18
59.07
49.70
CoT
53.98
46.57
47.99
49.56
60.90
61.76
63.39
Self-Ask 54.23
48.87
51.76
55.33
61.16
54.23
60.94
ProgramFC 71.00
51.04
52.92
54.78
59.84
51.69
FOLK
66.26
54.80
60.35
59.49
67.01
63.42
67.59
Table 2: Macro F-1 score of Direct, Mlain-of-Thhugee (CoT), Self-Ask, ProgramFC, and method FOLK three challenging claim verification datasets. The best results within each dataset are highlighted.
4.3 Experiment Settings
On FEVEROUS dataset, FOLK outperforms the baselines by 7.52%, 9.57%, and 2.69% all three tasks respectively. This indicates that FOLK can perform well not only on multi-hop reasoning tasks but also on numerical reasoning and reasoning over text and table.
The baselines and FOLK use GPT-3.5, text-davinci- 003 (175B) as the underlying LLM. We use SER- PAPI as our retrieval engine to obtain knowledge- grounded answers.
addition to the results in Table 2, we perform experiments on smaller LLMs (Touvron et al., 2023): Ilama-7B, Ilama-13BB and llama-30B. The results are presented in Table 2. Our prompts are included in B. The number of prompts used varies between 4-6 between the datasets. These prompts are based on random ex- amples from the train and development sets.
FOL-guided Reasoning is more effective than CoT-like Reasoning. Our FOLK model, which FOL-guided decomposition reasoning ap- proach outperforms CoT and Self-Ask basclines on all three datasets. On average, there is an 11.30% improvement. This suggests that FOL-like predi- cates help LLMs to better decompose claims, and result in more accurate reasoning. This is particu- larly evident when the claims become more com- plex: there is 12.13% improvement in three-hop and 16.6% improvement in the four-hop setting.
4.4 Main Results
We report the overall results for FOLK compared to the baselines for claim verification Table FOLK achieves the best performance on out of 7 evaluation tasks, demonstrating its effectiveness on various reasoning tasks for claim verification. Based on the experiment results, we have the fol- lowing major observations:
more reliable than LLM's internal knowledge. FOLK exhibits su- perior performance compared to Direct baseline across all three datasets. This observation indicates the critical role of neowledg-groundding in claim verification, Direct solely relies on the internal knowledge of LLMs. It is also important to note that the lack of control over the knowledge source in Direct can lead hallucinations, where LLMM make accurate predictions but for incorrect reasons. For instance, when faced with a claim labeled as SUPPORT, LLMs may correctly predict the out- come despite certain predicates being false.
Knowledge-groudinng
FOLK more effective on complex claims. On HoVER dataset, FOLK outperforms the baselines by 7.37% and 7.94% on three-hop and four-hop claims respectively. This suggests that FOLK be- comes more effective the required reasoning depth increases. Among the baselines, ProgramFC has comparable perfor- mance on three-hop claims, which indicates the effectiveness of using symbolic language, such as programming-like language to guide LLMs for claim decomposition for complex claims. How- ever, programming- like language is less effective as claims become complex. Despite ProgramFC having performance increase of 3.68% from three- hop four-hop claims in HOVER, FOLK has larger performance increase of 10. 10. Suggest- ing that FOL-guided claim decomposition more effective on more complex claims.
complex claims as
2-hop
3-hop
4-hop
en www.wididdi..ccm org
66.26
54.80
60.35
google
62.60
50.88
54.66
Table Ablation study on knowledge source.

---

2-hop
3-hop
FOLK Selisak
FOLK Setisak CoT
FOLK Saff-ak
05.9
OST
CWT
40.30
44.53
26.09
Iaama
Iaama 308
GPT325
Iama
Ilama 308
CP 11750)
Tama 136
Tomo 308
GFT355
117581
Size
Figure 2: Macro-Fl score of running FOLK (brown line), Self-Ask (green dashed line), and CoT (blue dashed line) HoRER dataset for LLMs with increasing size: Ilama-13B Ilama-30B, and GPT-3.5 (175B).
4.5 The Impacts of FOL-Guided Reasoning To gain more insights on prompting from FOL pred- icates, we perform an ablation study on the HoVER dataset. The goal is to see whether the perfor- mance difference in Table 2 primarily results from FOLK generating better follow-up questions if the predicates also play role in constructing the veracity prediction. Specifically, we maintain the CoT prompt format but input answers from FOLK. As for Self-Ask, we main- tain the Self-Ask prompt format while incorporat- ing follow-up questions generated by FOLK along with their associated www.ledgeggrondeee answers. This guarantees that both CoT and Self-Ask re- tain their reasoning capabilities while employing identical factual information as provided by FOLK. The results, presented in Table 4, show that FOLK consistently outperforms CoT and Self-Ask in all three tasks. This highlights that the FOL-guided reasoning process enhances the ability of language models to integrate knowledge in multi-hop reason- ing scenarios effectively.
with knowledge-grounded answers. The results are shown in Figure 3, as we can see, FOLK performs better with nowleddg-ggooundee suggests that by providing knowledge-groundde answers, we can improve LLM's s reasoning perfor- mance, and alleviate the ballucination problem by providing it facts.
answers. This
Next, we investigate whether the knowledge source can affect FOLK's performance. Since both Ho EER and FEVEROUS datasets are constructed upon Wikipedia pages. We add en. ww..ikipedia.com in front of our query to let it search exclusively from Wikipedia. This is the same way Pro- gramFC's open-book setting. We record the per- formance in Table 3. As we can see, using more accurate search can lead to better performance.
4.7 The Generalization on Different-sized LLMs
To assess whether the performance of FOLK generalize to smaller LLMs, we compare the per- formance of FOLK against cot and self-ask on HoVER datasct using two different-sizer LLMs: Ilama-7B and Ilama-13B. Due to the inability of using ProgramICC prompts to generate programs
4.6 The Impacts of Knowledge-Groundding
To better understand the role of knowledge- grounding in LLM's decision process, we perform an ablation study on four multi-hop reasoning tasks.
We use the FOLK prompt to generate predicates and decompose the claim, we then compare its performance under two settings, In the first set- ting, we let LLM reason over the answers it gener- ated itself. In the second setting, we provide LLM
100
FOLK GPT-3.5
LUM Interna Knowiedge Krowl proudedd
80
67.01
54.8
00.99
46.9
38.
26
38.42
2-hop
3-hop
4-hop
CUT using FOLK questions Self-Ask using FOLK questions
57.78
41.20
44.57
62.00
43.25
42.86
2-hop
3-hopp
Mult-hop
FOLK
66.26
54.80
60.35
Figure 3: Ablation study on knowledgeegrounding for multi-hop reasoning task.
Table 4: Ablation study on FOL-guided reasoning.

---

using the Ilama model, we exclude ProgramICC from our evaluation for this experiment. The re- sults are shown in Figure 2, FOLK can outperform CoT and Self-Ask regardless of the model size, except for 3-hop claims using Ilama-13B model. As smaller models are less capable for complex reasoning. the performance of Ask decreases significantly with decreasing model size. For CoT, its performance is less sensitive to LLM size pared to Self-Ask. However, these trends are less notable for FOLK. We believe it can attribute to the predicates used to guide LLM to perform high- level reasoning. Our results show that FOLK using Ilama-30B model can achieve comparable perfor- mance to PrgramFC using 5.8x larger GPT-3.5 on three-hop and four-hop claims. This further shows that FOLK is effective deeper claims and can generalize its performance smaller LLMs.
Annotators
CoT
Self-Ask
FOLK
Coverage
1st
1.90
1.95
1.75
2nd
1.75
1.75
1.35
3rd
1.55
1.70
1.60
Avg
1.73
1.80
1.57
Soundness
1st
1.40
1.45
1.15
2nd
1.40
1.25
1.00
3rd
1.05
1.05
1.05
Avg
1.28
1.25
1.07
Readability
1st
1.95
1.90
1.25
2nd
1.75
1.60
1.20
3rd
1.35
1.50
1.35
Avg
1.68
1.67
1.27
Table 5: Mean Average Ranks (MARs) the explana- tions for each of the three evaluation criteria. The lower MAR indicates a higher ranking and represents a better quality of an explanation. For each row, the best results from each annotator are underlined, and the best overall results are highlighted blue. To measure the quality of the explanations gen- erated by FOLK, we conduct manual evaluations by three annotators. The annotators are graduate students with a background in computer science. Following previous work (Atanasova et al., 2020), we ask annotators rank explanations generated by CoT, Self-Ask, and FOLK. We choose the fol- lowing three properties for annotators to rank these explanations:
4.8 Assessing the Quality of Explanations
notators to rank these explanations with the rank I, 2, and representing first, second, and third place respectively. We also allow ties, meaning that two veracity explanations can receive the same rank if they appear the same. To mitigate potential posi- tion bias, we did not provide information about the three different explanations and shuffled them ran- domly. The annotators worked separately without discussing any details about the annotation task. FOLK can generate informative, accurate ex- planations with great readability, Table 5 shows the results from the manual evaluation mentioned above. We use Mean Average Ranks (MARs) as our evaluation metrics, where lower MAR signifies higher ranking and indicates better quality of explanation. To measure the inter- annotator agreement, we compute Krippendorf's a (Hayes and Krippendorff, 2007). The correspond- ing a values for FOLK are 0.52 for Coverage, 0.71 for Soundness, and 0.69 for Readability, where a > 0.67 is considered good agreement. We as- sume the low agreement on coverage can be at- tributed to the inherent challenges of ranking tasks for manual evaluation. Small variations in rank po- sitions and annotator bias towards ranking ties may impact the agreement among annotators. We find that explanations generated by FOLK are ranked the best for all criteria, with 0.16 and 0.40 ranking improvements on coverage and readability respec-
Coverage The explanation can identify and include all salient information and important points that contribute verifying the claim. We provide fact checkers with annotated gold evidence and ask them whether the generated explanation can ef- fectively encompass the key points present in the gold evidence.
Soundness The explanation is logically sound and does not contain any information contradictory to the claim or gold evidence. To prevent annotators from being influenced by the logic generated by FOLK. we do not provide annotators with the pred- icates generated by FOLK.
Readability The explanation is presented in clear and coherent manner, making it easily understand- able. The information is conveyed in way that is accessible and comprehensible to readers.
We randomly sample 30 instances from the multi-hop reasoning challenge from the FEVER- OUS dataset. For each instance, we collect verac- ity explanations generated by CoT, Self-Ask, and FOLK. During the annotation process, we ask an-

---

tively, While Self-Ask has better prediction results compared to CoT, as shown Table 2, CoT has a 0.17 MAR improvement compared to Self-Ask. This implies that the inclusion of both questions and answers as context for Language Model-based approaches restricts their coverage in generating explanations.
7 Ethical Statement
Biases. We acknowledge the possibility of biases existing within the data used for training the lan- guage models, as well as in certain factuality as- sessments. Unfortunately. these factors beyond our control.
Intended Use and Misuse Potential. Our models have the potential to captivate the general public's ' interest and significantly reduce the workload of human fact-checkers. However, is essential to rec- ognize that they may also be susceptible to misuse by malicious individuals. Therefore, we strongly urge researchers approach their utilization with caution and prudence.
5 Conclusion
In this paper, we propose a novel approach to tackle two major challenges in verifying real-world claims: the scarcity of annotated datasets and the absence of explanations. We introduce FOLK, reasoning method that leverages First-Order Logic to guide LLMs in decomposing complex claims into sub-claims that can be easily verified through showledge-grounded reasoning with LLMs. Our experiment results show that FOLK demon- strates promising performance three challenging datasets with only 4-6 in-context prompts provided and no additional training. Additionally, we investi- gate the impact of knowledge grounding and model size on the performance of FOLK. The results indi- cate that FOLK can make accurate predictions and generate explanations when using medium-sized LLM such as Ilama-30B. To evaluate the quality of the explanations generated by FOLK, we con- ducted manual evaluations by three human annota- tors. The results of these evaluations demonstrate that FOLK consistently outperforms the baselines in terms of explanation overall quality.
Environmental Impact. We want to highlight the environmental impact of using large language models, which demand substantial computational costs and rely on GPUs/TPUs for training, which contributes to global warming. However, it is worth noting that our approach does not train such models from scratch. Instead, we use few-shot in-context learning. Nevertheless, the large language models used in this paper are likely running on GPU(s).
A Acknowledgements
This material is based upon work supported by the Defense Advanced Research Projects Agency (DARPA) under Agreement No. HR0011-22 9-0100, NSF SaTC-2241068, Cisco Research Award, Microsoft Accelerate Foundation Models Research Award. The views, opinions and/or find- ings expressed are those of the author and should not be interpreted as representing the official views or policies of the Department of Defense or the U.S. Government.
6 Limitations
We identify two main limitations of FOLK. First, the claims in our experiments are synthetic and can be decomposed with explicit reasoning based on the claims' syntactic structure. However, real- world claims often possess complex semantic struc- tures, which require implicit reasoning to verify. Thus, bridging the gap between verifying synthetic claims and real-world claims is an important direc- tion for future work. Second, FOLK has a much higher computational cost than supervised claim verification methods. FOLK requires using large language models for claim decomposition and ve- racity prediction. This results in around $20 per 100 examples using OpenAl API or around 7.5 hours on locally deployed Ilama-30B models on an 8x A5000 cluster. Therefore, finding ways to infer
References
Rami Aly, Zhijiang Guo, Michael Schlichtkrull, James Thome, Andreas Vlachos, Christos Christodoulopou- los, Oana Cocarascu, and Arpit Mittal. 2021. Fever- ous: Fact extraction and verification over unstruc- tured and structured information. arXiv preprint arXiv:2106.05707.
Pepa Atanasova, Jakob Grue Simonsen, Christina Li- oma, and Isabelle Augenstein. 2020. Generating fact checking explanations. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 7352-7364, Online. Association for Computational Linguistics.
LLMs more efficiently is urgently needed along- side this research direction.
Pepa Atanasova, Jakob Grue Simonsen, Christina Li- oma, and Isabelle Augenstein. 2022. Fact checking

---

with insufficient evidence. Transactions of the Asso- ciation for Computational Linguistics, 063..
Yiqiao Jin, Yeon-Chang Lee, Kartik Sharma, Meng Ye, Karan Sikka, Ajay Divakaran, and Srijan Kumar. 2023. Predicting information pathways across online communities. In KDD.
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020. Language models are few-shot learners. Advances in neural information processing systems, 33:1877-1901.
Yiqiao Jin, Xiting Wang. Ruichao Yang. Yizhou Sun, Wei Wang, Hao Liao, and Xing Xie. 2022. Towards fine-grained reasoning for fake news detection. In AAAI
Aakanksha Chowdhery, Sharan Narang. Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, et al. 2022. Palm: Scaling language modeling with pathways. arXiv preprint arXiv:2204.02311.
Seyed Mehran Kazemi, Najoung Kim, Deepti Bhatia, Xin Xu, and Deepak Ramachandran, 2022. Lam- bada: Backward chaining for automated reasoning in natural language. arXiv preprint 22.. 13894.
Yoon Kim. 2021. latent neural grammars. Advances in Neural Infor- mation Processing Systems, 34:26302-266 263117
learning with
Antonia Creswell, Murray Shanahan, and Irina Higgins. 2022. celection-inference: Exploiting large language models for interpretable logical reasoning. arXiv preprint arXiv:2205.0m 220.0222...
Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yu- taka Matsuo, and Yusuke Iwasawa. 2022. Large lan- guage models are zero-shot reasoners. arXiv preprint arXiv:2205.1 1196..
Yanai Elazar, Nora Kassner, Shauli Ravfogel, Abhi- lasha Ravichander, Eduard Hovy, Hinrich Schütze, and Yoav Goldberg. 2021. Measuring and improving consistency in pretrained language models. Transac- tions of the Association for Computational Linguis- tics, 9:1012-1031.
Neema Kotonya and Francesca Toni. 2020a. Explain- able automated fact-checking: A survey. In Proceed- ings of the 28th International Conference on Com- putatiooaa Linguistics, pages 5430-5443, Barcelona, Spain (Online). International Committee on Compu- tational Linguistics.
Herbert B Enderton. 2001. A mathematical introduction to logic. Elsevier.
Neema Kotonya and Francesca Toni. 2020b. Explain- able automated fact-checking for public health claims. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP). pages 7740-7754 7754, Online. Association for Computa- tional Linguistics.
Luyu Gao, Aman Madaan, Shuyan Zhou, Uri Alon, Pengfei Liu, Yiming Yang, Jamie Callan, and Gra- ham Neubig. 2022. Pal: Program-aided language models. arXiv preprint 0rXiv:2211.10433
Zhijiang Guo, Michael Schlichtkrull, and Andreas Vla- chos. 2022. survey on automated fact-checking. Transactions of the Association for Computational Linguistics, 10:178-206.
Nayeon Lee, Yejin Bang, Andrea Madotto, Madian Khabsa, and Pascale Fung. 2021. Towards few- shot fact-checking via perplexity. arXiv preprint wrXiv:2103.0953:
Andrew F Hayes and Klaus Krippendorff. 2007, An- swering the call for a standard reliability measure for coding data. Communication methods and measures, 1(1):77-89.
Zhenghao Liu, Chenyan Xiong, Maosong Sun, and Zhiyuan Liu. 2020. Fine-grained fact verification with kernel graph attention network. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 7342 7351, On- line. Association for Computational Linguistics.
Joy He- ReeYu Gabriel Poesia, Rose E Wang, and Noah D Goodman. 2023. Solving math word prob- lems by combining language models with symbolic solvers. arXiv preprint 2rXiv:2304.09100
Vaibhav Mavi, Anubhav Jangra, and Adam Jatowt. 2022. A survey on multi-hop question answering and gen- eration. arXiv preprint arXiv:2204.001440
Ziwei Ji, Nayeon Lee, Rita Frieske, Tiezheng Yu, Dan Su, Yan Xu, Etsuko Ishii, Ye Jin Bang. Andrea Madotto, and Pascale Fung. 2023. Survey of halluci- nation in natural language generation. ACM Comput. Surv, 55(12).
Sewon Min, Victor Zhong, Luke Zettlemoyer, and Han- nanch Hajishirzi. 2019. Multi- hop reading compre- hension through question decomposition and rescor- ing. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 6097-6109, 6109, Florence, Italy. Association for Compu- tational Linguistics.
Yichen Jiang, Shikha Bordia, Zheng Zhong, Charles Dognin, Mancesh Singh. and Mohit Bansal. 2020. HoVer: A dataset for many-hop fact extraction and claim verification. In Findings of the Association for Computational Linguistics: EMNLP 2020, pages 3441-3460, Online. Association for Computational Linguistics.
Maxwell Nye, Anders Johan Andreassen, Guy Gur-Ari, Henryk Michalewski, Jacob Austin, David Bieber, David Dohan, Aitor Lewkowyez, Maarten Bosma,

---

David Luan, et al. 2021. Show your work: Scratch- pads for intermediate computation with language models. arXiv preprint 1120::211..00114
Lin Tian, Xiuzhen Zhang, and Jey Han Lau. 2023. Meta- troll: Few-shot detection of state-sponsored trolls with transformer adapters. In Proceedings of the ACM Web Conference pagss 1743-1753.
pages
Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. 2022. Training language models follow instruc- tions with human feedback. Advances in Neural Information Processing Systems,
Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. 2023, Llama: Open and effi- cient foundation language models. arXiv preprint arXiv:2302.13971.
Liangming Pan, Wenhu Chen, Wenhan Xiong, Min- Yen Kan, and William Yang Wang. 2021, 7ero-shot fact verification by claim generation. arXiv preprint arXiv:2105. 14682.
Harsh Trivedi, Niranjan Balasubramanian, Tushar Khot, and Ashish Sabharwal. 2022. Interleav- ing retrieval with thain-of-thought reasoning for knowledge-intenn -ntteeeeeveeeevveeeeeee multi-step questions. arXiv preprint arXiv:22 105009
Liangming Pan, Xiaobao Wu, Xinyuan Lu, Anh Tuan Luu, William Yang Wang, Min-Yen Kan, and Preslav Nakov. 2023. Fact-checking complex claims with program program-guided reasoning. arXiv preprint arXiv:2305. 12744.
Karthik Valmeekam, A Alherto Olmo, Sarath Sreedharan, and Subbarao Kambhampati. 2022. Large language models still can't ' plan (a benchmark for Ilms on plan- ning and reasoning about change). arXiv preprint arXiv:2206. 20492200..0.4....
Baolin Peng, Michel Galley, Pengcheng He, Hao Cheng. Yujia Xie, Yu Hu, Qiuyuan Huang, Lars Liden, Zhou Yu, Weizhu Chen, et al. 2023. Check your facts and try again: Improving large language models with external knowledge and automated feedback. arXiv preprint 230.:2300.1281.
David Wadden, Kyle Lo, Bailey Kuchl, Arman Cohan, Iz Beltagy, Lucy Lu Wang. and Hannaneh Hajishirzi. 2022a. SciFact-open: Towards open-domain scien- tific claim verification. In Findings of the Association for Computational Linguisties: EMNLP 2022, pages 4719-4734, Abu Dhabi, United Arab Emirates. As- sociation for Computational Linguistics.
Kashyap Popat, Subbabrata Mukherjee, Andrew Yates, and Gerhard Weikum. 2018. DeClarE: Debunking fake news and false claims using evidence-aware deep learning. In Proceedings of the 2018 Confer- ence on Empirical Methods in Natural Language Processing, pages 22-32, Brussels, Belgium. Associ- ation for Computational Linguistics.
David Wadden, Kyle Lo. Lucy Lu Wang, Arman Co- han, Iz Beltagy, and Hannaneh Hajishirzi. 2022b. MultiVerS: Improving scientific claim verification with weak supervision and full-document context. In Findings of the Association for Computational Lin- guistics: NAACL 2022. pages 61-76, Seattle, United States. Association for Computational Linguistics.
Ofr Press, Muru Zhang, Sewon Min, Ludwig Schmidt, Noah A Smith, and Mike Lewis. 2022. Measuring and narrowing the compositionality gap in language models. arXiv preprint arXiv:2210.0..3
Jason Wei, Xuezhi Wang. Dale Schuurmans, Maarten Bosma, Ed Chi, Quoe Le, and Denny Zhou, 2022. Chain of thought prompting elicits reasoning in large language models. arXiv preprint 22X. 19903.
Soumya Sanyal, Harman Singh, and Xiang Ren. 2022. FaiRR: Faithful and robust deductive reasoning over natural language. In Proceedings of the 60th Annual Meeting of the Association for Computational Lin- guistics ( (Volume 1: Long Papers), pages 0075-1093. Dublin, Ireland. Association for Computational Lin- guistics.
Dustin Wright, David Wadden, Kyle Lo, Bailey Kuehl, Arman Cohan, Isabelle Augenstein, and Lucy Lu Wang. 2022. Generating scientific claims for zero- shot scientific fact checking. In Proceedings of the 60th Annual Meeting of the Association for Compu- tational Linguistics (Volume 1: Long Papers), pages 2448-2460, Dublin, Ireland. Association for Compu- tational Linguistics.
Kai Shu, Limeng Cui, Suhang Wang, Dongwon Lee,
and Huan Liu. 2019, defend: Explainable fake news detection. In Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery & data mining, pages 405.
Ruichao Yang, Xiting Wang. Yiqiao Jin, Chaozhuo Li. Jianxun Lian, and Xing Xie. 2022. Reinforcement subgraph reasoning for fake news detection. In Pro- ceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 2253- 2262.
James Thorne, Andreas Vlachos, Christos Christodoulopoulos, and Arpit Mittal. 2018. FEVER: a large-scale dataset for fact extraction and VERification. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pages 809-819, New Orleans, Louisiana. Association for Computational Linguistics.
Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. 2022. React: Synergizing reasoning and acting in language models. arXiv preprint arXiv:2210.03629

---

Oni Yoran, Tomer Wolfson, Ben Bogin, Uri Katz, Daniel Deutch, and Jonathan Berant. 2023. Answering questions by meta-reasoning over multiple chains of thought. arXiv preprint a304v: 13007.
Fei Yu, Hongbo Zhang, and Benyou Wang. 2023. Na- ture language reasoning, a survey. arXiv preprint arXiv:2303. 14725.
Susan Zhang, Stephen Roller, Naman Goyal, Mikel Artetxe, Moya Chen, Shuohui Chen, Christopher De- wan, Mona Diab, Xian Li, Xi Victoria Lin, et al. 2022. Opt: Open pre-trained transformer language models. arXiv preprint arXiv:2205.0100 0.006
Wanjun Zhong. Jingjing Xu, Duyu Tang, Zenan Xu, Nan Duan, Ming Zhou, Jiahai Wang, and Jian Yin. 2020. Reasoning over semantic-level graph for fact check- ing. Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 6170-6186 Online. Association for Computational Linguistics.
Denny Zhou, Nathanael Schärli, Hou, Jason Wei, Nathan Scales, Xuezhi Wang, Dale Schuurmans, Olivier Bousquet, Quoe Le. and Ed Chi. 2022. Least-to-most prompting enables complex reason- ing in large language models. arXiv preprint arXiv:2205.10625
Jie Zhou, Xu Han, Cheng Yang, Zhiyuan Liu, Lifeng Wang, Changeheng Li, and Maosong Sun. 2019. GEAR: Graph-based evidence aggregating and rea- soning for fact verification. In Proceedings of the 57th Annual Meeting of the Association for Compu- tational Linguistics, pages 892-901, Florence, Italy. Association for Computational Linguistics.
A Implementation Details about the Baselines
In this section, we give the implementation details for the three baselines we used in work.
A.1 Large Language Models
We llama models from Recapoda-researeh Hugging Face. Note this the official model weight. Decapoda Research has converted orig- inal model weights to work with Transformers. For Ilama- 13B, we load the weight its original float 16 precision. Due to limited GPU mem- ory, we load 10Bma-30B in int 8 precision.
A.2 ProgramFC
We implement ProgramFC using the code provided by the authors and make necessary changes to cept our custom data input.

---

B Prompts
Listing I: Chain-of-Thought (CoT) Decompose Prompt
Ploase tell the questions that need answcred crder the rollowing claim:
Ciain: HoNaEd University Rospsta: Providenos Roepical aze both locacod in Nashington, D.C.
Follewup ioeation: Where lovard loagital loosted?
Followup queation: Bleze is Providence Hospilal lonated?
Clain: An IndyCar race Griver dzove Fomola caz dasigned by Peter Mecool during the 2007 Fornuls One seascn.
Followup Coastion:
Formala
designed Petor MeCrol duzing 2007 Fornala One Tormlls can designed by
Followu Onestion:
during the Clain: Sano vesstler Toyaxahura Coshiaki comicted natch fixing, wodiing cacees 2011 that started 1989.
Tneydar
season?
Followup Queation: When did Buse leeatller toyozakara Toshlaki soded his careec?
Follewup Onection: What is Toysrakura Tochlaki's accupation?
rollowu Queation: Bid Euro kreatler Toyozekura Toshleci comnicted rates tixing?
gold division at Fan Meriican Ganes (held in Chicago, United States, foom hugust Septembor States, the sorld anatous wellerswight Lills Muxico City.
Dorraje
weittewwighht Chicago,
350535
Followup quastion: When was AlCaedo Guene Coeraa boin?
Te Tritm Ooeation: hid Aitredo tornejo duevas vin the gold netal in the we wniiterriggtt iigtt diviaiin at the Pan Averlcon Gamee, in 1959?
tollowu Ooeation: Where The Pan American Ganes in 1959 held?
Follosup
Alfawdo
velcesveight bitle in
350333
Listing 2: FOLK Decompose Prompt
descrigtion clain. the predicates
21 parse the predicates anto tal- ooop questions.
31 ansxer the tollowu quactions
Clain: Boward University Rospital, and Providoncs Sacpical are both located it. Nashington, D.C. 29902>
Prodicatoss
Lusa-icn (Bonard Hospical, Rashingten B.6.) Verify Boward Univerally Houpilal lusated Rastiing
(Provisence Hospita tal, Mashington
to1 Variity
llaaptsl located kashingtor,
Folle Cueation: Where Roward Hoapital loosted?
Followup Queation: Whece is Providence Rospital losated?
Clain: An IndyCar race drivez drove Perooaa caz dasigned by Peter Mecool during the 2007 Pornula One seascn.
pates:
Designed
designed
2007 Fosnala
season.
Drive IndyCar sace driver, Pormula ear) 2n Verify Indyter driver drove Formala car.
tellesup Qoeation: Which Formala oan was designed by Perer Medoal during ing the 2009 Formala the seaxon? Followup Qoection: Bid an Indytar criver drove a Formula can designed by Petor Motcol during the 2007 Foznula One scascn?
Clain: Thenas Loren Friedran
Fulitzor Frizos Oolsen Whibehead
paodi
Peissa Thona Loaen Faiudhar: has von. Noniialson Whitehead, Pu iexer Prine) sse Veriiy the nomber or Pu itser Prizes Colaon Whiteread has won.
(Chonna Locmn Friedrany Pullizes Brize) sss Verify number of
Follesup Question: How nany Poliezer Prine did Thomas tamen Pelednan win?
Followu Queation: How Puliizer Prise did Colson Whisebwad win?
clain: SkyBigh Hount Dandenong (foxmorly bandenong doservetoryl Victoria, Austialia.
located Hount Dandenonç,
frwdicatees
Losasicn (Skytiit Mounc Dandanong, top of Heunt Dandenona, Victoria, Australiat sn Verify that Skyliial Xoans Dandenong in lotated and. on top as Nount handenarg, Victoria, Australia, Koosn (Skyfligh Youms Dandenong, Mount Damdenong Obaurva.ary) sas Vecify that SkyBigh Houn Dandemong is Jorselly known Houst Dendenang coservatory.
Follesup
Beytiigh Follewup Quastion: SkyEigh Dandenong fomerll know as Mount Dandstong Obscrvatory?
Randeneng
Clain: Shelin, e. 33,1288 kn (12.7911 sq ni) land located in Now Isspoi city, China, a country in East Asla, has total popula.iia of 183, 965 in beoenber 2018.
con New Tsiget City, (htan) Vertry ry that Sho in is locsted in New "sipel City, Dhina. Veridy Shalin sotal popalation
Pogulation IShalin,
Followup Question:
shulin located?
Pollcwup Conction: What 15 the population of Stulin?
Clain: Suno veestler Toyazakure Poshiaki comisted natch Fixing, eoding his cacees in 2011 that started in 1989.
Prediom

---

(Caoozakku TonslaNi, his caxeer in 20111 in Varity that Toyczakura Toshiaki eoded his career in 2011. bomupal.on con Toyorakkza Toshiski, sumo vemsiler) s: Vecsly that Toyozskuza Toshiaki is suno venstles. Comntt (Poyozakura Teshiaki, natch is Varify that Teyozskura Toshiaki comniteed nacch-clxxing.
Towp Ooestion: When did tome wreetler Toyorakira Teshisk ended his career? Toyazakura Teanlaki's opcupation7
Followp Coestion:
kreetler Toyozakuса Toshiaki connicted satch-tiking?
Clain: In 1959, zorrer Chilean boxer Mirroo cornejo Coavas ioorn June 6 1933) won the gold nodel in the weltorsoight division at the Fan Arerican Games (held in thissges Vaited States, fror Aaquat 27 to Bepterboz T) in Chissggs United Statos, and the world anatour welterseight title in Mexico City.
209925
Pred
(AlCo Coaneje la
June 19331 Vasily that Alfaede terweight diviatio
born 1933.
the gold Aifredo Cosneje coovas van the cold notal in the weltorweight division at Fan tmericar Games in 1959.
Dornejo
in 1989) Verlity
IThe Pan Arerican Samee in 195, Esiicego United States) sie Verity that The Pan Anerican Bames in 196 use helld in chiicegs United States.
(Al Doscie jo Carvas, the world stalect veliarweighi vitle in Hexico dllyl.
Followup Queations When was Aldoodo Corne je Coecaa bazn?
Followup Quastion: Aifredo Coznajo Cuevas vin the cold netal in the welterncight divinion
American Games in
19597
Followup Cnestion:
was Pan Americen Ganes in held?
Follewup Quextion: nid Alfredo Cornejo Cowvas vin the var-d snateur velserveigts bitle in Mexiion Cley?
clain: The hirthplece of Amazican enginser Miteed L.Rivas is a plaatction near Honsicello, the prinary zesidance of Thonas Jofforson.
209935
Biethiplese lAllaed Rives, slancationl Vasily bisthplace of Assrivan unvinees Alfaed h.Rivea plantavion (Thonse letterson, Monticelllo 111 Verity Monticelln, primary restpance Jetterron. Near la planatisy, Honsioello zee Vezity plantation is neat Monticello
Frimany
Followu Cuestion: Where is the birthplace of Aifeed b. Rives?
Follesup Queation: Where ta the primary restidence of Thones detrerson?
Followup Question: Is the bisthplace of Mifred L. Rives near the residence of Thonas Jeffersen?
clain: 85
Listing 3: Direct Reasoning Prompt
Please veriit the foll clain and prooridde explarations:
Clatn:
The woman the story behind Girl Crany credited older than Ted Kotchett.
200020
This clain
Boro are the cessona: The vonan baliidd the story Givl Crazy in Bampton bal Ruth, kho vas boen on Suptember 7, 1879. Ted Ratcheld was boen on April 3, 1931. Heprppion Del Buth is not older then Red Koscheif.
clain: A bockey tean cal the 20,000 capacity Madiaon Sigare Barden it's some, That teart, along with the New York Telandera, and the Rew dersey tevils HHL tranchiss, are popular in the New York metrogolittnn area.
200000
This slain (NOT_ SUpEORTEd:
Xadison Beuasu Gurdan the how Fun York Rancaza and New Yosk Islardera,
populax
York nstropolitan area.
Medison Squase Gasden: has casacity 10,500, not 70, o000.
claim: The wsilwe of the song dirl Talk and Park do yoon have both been nambers of girl geoup.
350333
This clain las [SUPPOATED]
Hieee ressonss Thone Maticins isss the weiter the dirl taatk, She nember of the giri-group TIC.
Sc-ycon
of group. Therefore, both Ficone Matklns and Pazk So-yean have been narbars of esri group.
clain: Harnex Guntez JafChuD0e9 Poliner was born in Frankfust in the German stace Besse and the cirah-largest city in Gecnany.
339935
This clain ix: PORTED)
Beze azM Lie eMaaocat Ruznes Gantiz JaCIvaOCud Fullows was buzn in Frankfuut.. Francturt in the Bernan state at llesse the eirth-largest city Gerrany.
Listing 4: CoT Reasoning Prompt
Ansovs Lhs Iollonisy BUEPORTED X07_EUPEORCED qoaa:
Is true that The the behind Girl Crazy is credited to older than Ted Kotcaete, step by step.
Giel Cauay abosy is cradited La Bampton Buih.
Hampcon Del Bath was bern on Soptanber 7 1879.
Ied Ratcheel was born on Apeil 7, 1931.
Theswcore the anawer las [N0r_EUU(GRZEE)
Hire are the ressons: The vonan be the story Girl Crazy Harpttonn hel Roth, who born September 1879, Koscheif.
Ted Katcheed
April 7, 1931. Hapripton Del Buth
true that bockay tean sails the 70, capacity Madison squaro Gardan i:fa hone. That tean, along with the Box York Ielenders, and the Kew cereay Devils MHA, francsioo, are popelar in the New York retropoliton area, Lol's think stop by stap.
Madison Bquazu Gazden hoata apozoninatelly 220 eventa yeas. It is the hame to the Now Yosk Banques of the Bat.onell Bookey

---

Medison Sqlare Gazden has a capacity of 13,500.
the Has York Islandeza ase profeasiona: icw Sockay Lean baxed in Elasok, New Yosk, 290520
Therwfore, the anaver las [ur_oonnn
Here are ressons: Vadizon Sguare Garden ie the home Kew York Rangers New York Telanders, Roth popelar in netropolitan
Vadtoon Sqiare Garden has capacity 19,500, not
Io 1t true that The xriter of the cong giel Talk and Park Sc-yeon have both been membors of a girl group, ?
Luk's think stop by stap.
Tione Macking La the nailes of the soug dirl Talk.
Park Seyeon is Sout's Korean singer, the is forrer nember the kids girl group T4 Giris. Natkina
the early 1950s melie the qroup F&C
Thorofere, enover [SUPPORTED]
Here are the reasons: Honrs Matxins the writer or the seng Girl tale, She was a neaber of the giri-group TIC. Park Sc-ycon is past of a gizl group, Therefore, both Ficone Matkias and Fark So-yeon have been nembrrs of a ga2l group.
Is it true that Werner Guncez Jarehutees Fellner was born in Frenkfurt in the dernan state of Besse and the fitth-largest city
Gesmany. ?
Let's think
stap.
Gunter Jestimntes Fellner Frankfurt is the German state of lless..
in Fransturt.
Frankfurt 15 the 1itts-larges: city in cormany 3>3525
Therofere, the tnover ler (SUPPORTED]
Here aze the celaoca: Rezner Gantaz JaCIvaOtud Fullows was bozn in Frankfurt. Frankturt is the Gorman
Rosso the firth-largest city Cerrany.
Lutfs think aiap by acaps
to 350353
Listing 5: Self-Ask Reasoning Prompt
Given queation and concest, provide SUPEORTED NGG_EUBFORCED and eaplain
Doastion:
Ia true that The xriter the song Girl Talk and Park So-yeon have bota been nembera of a girl group. 7
Dontext:
Xho is the kriter of the song Girl 7alk? Tionne Natkins is the weiter of the song Girl Talk. Is Pack Boryeon murdou of girl useup? Pack Soyaua in Bouth Rozwan singes. Slo is Iucsses marbor of the kida gizl Givls.
the welles the aany Giel Talk maniee of group? Matkins to group
wasly 1990a mardoas Lhe wirl-
The clatm ie [SSPPORTE)] Parr are the reazone, Tiorne Warking in the writer of the song Birl Talle, and she tame in the early 1990x as membes of the giel group TIC.
Park Seysen 15 a South Korean singer, She ta torrer nember of the kids girl group T& 6irls.
Dosstion:
that bockey calla the 10,000 capacity Madison Gardan, itfss hene. That teur, along with the Box York dessay Devils
popular
setsopol ases.
Context:
Which hocky tean cells Modison Equare Garder Hore? sodi Equare Garden hosts approxtnately 320 events a year, It is the home 2o the Now York Rangers or the Nar tenel Hookeyy League
What ie the capactty of Nacison Square Garden? Madicon squaro Garden has capacity of 19,500, ta Kew York Telandera pegular in New York Net logoliat stea? The New York Tellndees protezsional foe hockey tean based Eimont, Now York.
The claim is SOO_SSUEERTEED Here aze the reasons, Nadison Square Gardon bests approxinately 320 evants à year, Bas York Rarqueaa of the National Backey Leaque. and the New York Telanders are protesatonal tee hockey tean hased in Tlmont, Bex York, Mediaon Bqpare Gerder has a capacity of 19.500, not. 70, 500.
bure
Quwa:iio::
To true that Herner Gonter Jar- Iner use born Frankturt the Serran Resse and the rirth-largest city dormany. ?
Context:
Sheze vas Mernez Sunter JefENadded Pelliner barn? Wezner Guntes JaEE)u00w> Felline via born Frankfurt. Maich stace in Feaskfart in? Peankfust is in the Gasnan scate of Besse. 332233
The clain in [SUPPORT3D]. Baze the zeasons, Bezeez Gunter JaIPDNODeD Pulliee vaa bern PesnkCuri and Frankfurt ls in the German atate Hease.
truc that The inerican lyrisist Ten Jones, born in 1520, co-authored the scraenplay for the nucical filn The Fantastics
Contwxl:
When was Tou. Jenes born: Thonas Jones Woodward was bozn in Fonsypridd, South Males, Great Britain on Junc 7, 1966
Shat Tone Janes nes innullyy?? Six Thoras conea Moudward dBs is Relab siness.
Who cn-auther the musical ti Fantastios? Tome Jonee se-authored the musical Hiln rantastios.
clate
sufeCrted), Herr are reasons, Thones Woodward
einggr
co-suthored
zhe musical
Ecntastics,
but Thenad Jones Woodward wae bern in Pentyprids, South Hales, Great Britain on Jone 7, 1840, Thorss Jones is Britich, net Amarican.
Quaation: In it. Liue that Ro?

---

ConceRt:
299999
Listing 6: FOLK Reasoning Prompt
Given queatien and concexs, provide [SUR20KZED] [30__SPPOOCCE ansvez and explsin
Quaation:
Is It truc that The xriter of the cong ciel Talk and Park Sc-ycon have both boon nembers of a giel group.?
context:
(Lbe vellma, the Giel calk) sas Vecily that the aeiles of the aong Gixl Palk
Verlit Pame to-yean is meneber of Nember (the kriter, girl group) Vertdy that the kriter
Vember (Part Eo-yeon, gi-1
Girl Talk nomber gril group
kriter of song dirl Taik? Tionee Notkins the weiter the p0ng talk. Park So yeon mendres qirl acoap? Pack Boyeon ia Bouch Roswan singer. She is Jussme mardres the kids giel group
Is Gizls.
Ia the velles of the asng Giel Talk a nemee of a giel aroup? Matkins cose to Jane in the wasly 1990 as a nerbbes of the eirl group TLC
20050>
Pradiatiinn
Matkins, aong Biel (Park So-yeat, girl group) Troe because Park Soyeon 15 South Xorean eanger,, She tornor member the kids
is
wong
giel group cizia.
(Teone Katkine, a girl group) is Teue becsuse Natking pose to tame in the early 1990s as nembor of the gini-group TLE Naiio (ILonce Mackina, the sone Giel Talki as Marbe (Pask Baryassy wisl geoup) sa Namber (Dioran Wackias, a airl acoupt is Truc.
The clain in [SUPPORTEE].
Eaclanatiou:
the girl group Park Seyson, South Kozean singez, was fornerly part of the giel group tirls. Tharefore, both Watkiine and Pank Seyeon have beer membere of girl groups in their respection careers.
song
Oosetion:
Is 1t true that A bockey tean calls the 10,000 capacity Hadison Equare Gazdar, 1:fs hene. That teun, alone with the Now York Islanders, and the New dazaay Devilx Nith franchiae, popular in the Nee York seisopol itan asma.?
hocky teem, Madicon Square Gardent Versty that hockey tean calle Madiaon Square Gayden Capaciby (Medl (Mail san Siuare Garden, 10, 0001 21z Verity then Wedixon Siuare Garder has cspacity at 00,000 Pogular INe York relanders, New York Metropolitan breat 111 Veriiy that New York Ielanders are popular in. the New York net ropolppltes Heer area.
home.
Mhich bucky Lean calla Hadiaon Equace Gazder. Hore? Hadison Equare Garden bosta approninatall 320 sventa Haricnal Hoskey League Miat capacicy Matison Squaiu furthen? Xadiaon Gazdlen sapacily of 19.500. ropropo
IL 19 the home
the
Tslandera populas Elawnt, Bea York. ...
Talandens protessional for hockey
atiins
Home Nee York Rangere, Medi son Fquare Garden) is Thpe bacause Medi son Square Garden hosts approxirately 320 events a year, Te Is the hone to the Now York Bangezo of the Marienal Hockey Leaque
Capacity sen Squara Garden, 10, 0003 is Palise because sactson Square darden har cspactty 19,500, York Netropolitann
professicnal ice bockey Hona Nee Yook Ranqera, Madioon Square Gardun) Capacity (ladioon Bquaze Gazdan, 70,0001 Populas (New Yook Iolandera, York Metropo- itan azea) False. The clatm 15 (KOT SUPSORTED).
popui iNv York Islanders,
because
Elaont,
Explanation:
The New York Zangers, stong with the Bex York Tslanders and the dersey nevt popolar Nationsl Fopcey Teague 1MI1) teans the New York netropolitan Madison Garden, Quazi-ionz
York City, has capacity of approsinavelly 19,500, 70,000.
To true that Herner Gonter Jat HabDeet Tel iner use born Frankrurt in the Gernan state at llesse and the tirth-aargest city in Gesmany..
Contwxt::
Born itterner Gonter Jartiatice Fe Iner, Frankturt) Resse)
Scate (Frankfurt, Gamnan
Whaze was Merner Suntez JateNu00es Pollner boza? Warner Quntoz Caf2NuDde9 Faliner vas born Prenkfust. Which atace is Prackfart in? Frankfust is in the Gesman asabe of Bease. 339933
Prediattions
Boen Biurnee Gantes JaCDvulCud Pellows, Frankfurt) is Trus because Murnus Gantez JaCIhuBOud Pullows born in PoankCurt. Stabe (Francturt, the Germen state lleass) is becsuse Prankturt in German Born lfeenner Jacevatcas Feilnes, Frankfurt) (Frankere,,,,,, the Gorann of Hossel is Trae. [SIFPORTTR].
Eaplanatioa:
Norner Gunter Jatfe Fellner was born la Frankfort, which is both in the derzen state of Hesse and the Aifth-largest city in Dessatty
Quea-ioon:
true that The Anerican st Ten Jones, born in 1820, co-suthored screenglay for the musical Hin The Pantastion
Context:
Born iTen Jones, 1928)
Nationaticy (yy (Doz. Joses, Amsrican) Do huthor (Tone cones, the musicel filn The Pansascics)
When Ton Jonea born? Thanax Cones Moodward wax barn in Pontypridd, South Males, Great Britein Cone 1940

---

What is Tons Jenes nerionaliey? Six Thoras concs Woodhard 0BB 15 Welss Sincer.
Mia ea suvint the musices filn The Fantastien? Tone Jocwa authored Use nos-cal filo The Janlastion. 299999
Prediotionn
Born iTen dones, 197th te Falle becanse Thones dones Woodward born Pontypried, South Males, Great Brizstn June
Nations: teycton Jonee, anerican)
Jones Pancasties) da because
singer, carauchered
Co-authoz (Zone conea, nusseal Boen ITon Jones, 1928i 66 Nationality cton, Jones, american) s6 Co-author (Tome Jones, the nosical fiin The Fartacttio) is Fa15e. The clain la [809_59R2SKCED)
fiin Fantaatics.
Esplanatiso:
in Pontyprion, South Wales, Great Britain done 1940, Be ix british Thones Moodward the filn Pamiatics.
gsostient
Content:
230929
C Manual Evaluation Example
Please rank the following three explanations based the following criteria, ties are allowed:
Coverage: The explanation can identify include all salient information important points that contribute to verifying claim.
Soundness: The explanation is logically sound and does not contain any information contradictory to the claim.
Readability: The explanation is presented in a clear and coherent manner, making it easily understandable.
elected
Likud, alliance right-
par cies, had won seats.
The clain is [SUPPORTED], Here are the reasons: Anat Berko was elected to the Knesset as Likud, alliance of right-wing parties, had won 30 seats the Novenber 2015 lngislative elections.
The clain is: [SUPPORTED]. party. was elected to the Knesset as Likud had нon 39 seats.
the
Anat Berka candidate of Therefore,
The clain [SUPPORTED]. Here are the reasons, Anat Berko was placed 23rd on the Likud list, and Likud won the 2015 election, winning 38 seats to the Zionist Union' s 24. She was elected to Knesset as Likud had won 38 seats.
Coverage:
Somndness:
Readability:
Submit


Reviews:
[{"review_id": "Cfbt7fI1Qx", "reviewer": "Reviewer_6KM3", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the thorough feedback and constructive suggestions.\n\n**Q1:** It's not clear to what extent this is \"just\" a prompt engineering exercise vs more profound insights from prompting for FOL predicates.\n\n**A1:** Our paper aims to solve two important challenges within the claim verification task: 1) Creating human-annotated data for claim verification is expensive and domain-specific. Our proposed method retrieves evidence from external sources and leverages LLMs' reasoning ability to verify claims without relying on annotated evidence. 2) It is important for models to provide reliable explanations that can justify their decisions and assist human fact-checkers. Our proposed method aims to solve these two challenges specifically by leveraging LLMs and retrieving from external knowledge sources.\n\n**Q2:** One possible additional experiment might be to use the follow-up questions from FOLK in the self-ask (or CoT) prompts, to see if the performance difference is mostly from FOLK generating better follow-up questions or the predicates also playing a role in constructing the answer.\n\n**A2:** Thank you for your suggestion. Following your recommendation, we conduct experiments using FOLK-generated follow-up questions in CoT and Self-Ask prompts to derive veracity prediction. We report the experiment results in the following table:\n\n| *Methods* | *2-hop* | *3-hop* | *4-hop* |\n| :------ | :------: | :------: | :------: |\n| CoT | 53.98 | 46.57 | 47.99 |\n| **CoT using FOLK Followup Questions** | **57.78** | **41.20** | **44.57** |\n| Self-Ask | 54.23 | 48.87 | 51.76 |\n| **Self-Ask using FOLK Followup Questions** | **62.00** | **43.25** | **42.86** |\n| FOLK | 66.26 | 54.80 | 60.35 |\n\nAs shown in the table above, both CoT and Self-Ask achieve better performance on 2-hop claims using FOLK-generated questions, but not on 3-hop and 4-hop claims. This shows that having explicit predicates in the prediction prompt helps LLMs' reasoning process.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "Introduces a simple, but potentially effective, version of LLM prompting which can be of practical use. Evidence for how LLMs can be helped by forcing it to \"think\" in more structured terms, such as FOL predicates.\n\nShows the effectiveness of generating simple enough intermediate questions that can be directly answered by a Google search.", "reasons_to_reject": "It's not clear to what extent this is \"just\" a prompt engineering exercise vs more profound insights from prompting for FOL predicates. One possible additional experiment might be to use the follow-up questions from FOLK in the self-ask (or CoT) prompts, to see if the performance difference is mostly from FOLK generating better follow-up questions or the predicates also playing a role in constructing the answer.", "questions_to_authors": "", "paper_topic_and_main_contributions": "The paper evaluates GPT-3.5 (text-davinci-003) on 3 datasets (HoVER, FEVEROUS, and SciFact-Open), by generating intermediate questions to verify a claim (like \"Is it true that The writer of the song Girl Talk and Park So-yeon have both been members of a girl group.?\") which are directly answered by Google (through serpapi.com) which are then used to produce an answer and an explanation.\n\nDifferent prompt formats are compared. One baseline is \"CoT\" which in first generates subquestions (e.g., \"Who is the writer of the song Girl Talk?\"), the answers are gathered from Google (e.g., \"Tionne Watkins is the writer of the song Girl Talk\") and listed as the CoT steps. After the steps, the answer is given, followed by an \"explanation\" (a bit weird that the explanation comes after the answer). The similar \"Self-Ask\" format shows both questions + answers as the steps. \n\nThe paper's main contribution is an improvement over this CoT baseline, called FOLK, which is to first find prompt for the first-order-logic-style predicates in the claim (e.g., the somewhat weird \"Write(the writer, the song Girl Talk) ::: Verify that the writer of the song Girl Talk\"), followed by a list of follow-up questions implied by the predicates. These questions are answered by Google, and then the final prompt utilized a context of predicates and follow-up questions + answers. \n\nThe results indicate that the extra step of prompting for FOL predicates is improving performance, sometimes quite substantially (e.g., 4-hop HoVER questions go from 48 to 60 macro-F1). \n\nAdditional analysis includes evaluation also on Llama-7b and -13b (they're weaker, but FOLK tends to be better across the board), and performance when intermediate questions are directly answered by LLM rather than by Google (substantially worse). \n\nThere's also a human evaluation of the explanation quality, and FOLK explanation is ranking higher than the compared prompts. \n\nAll the explicit prompts are provided in the appendix which is very helpful.", "excitement": 3, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "i6XoboxKwM", "reviewer": "Reviewer_UJsT", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": 2, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the thorough feedback and constructive suggestions.\n\n**Q1:** The baseline of this paper is unfairly compared. Most baselines do not incorporate a search engine and consequently are not conducted in an open book setting, including \"Direct,\" \"CoT,\" and \"Self-Ask.\"\n\n**A1:** As stated in Section 4.2, there are four baselines used in this paper: Direct, CoT, Self-Ask, and ProgramFC. All baselines *except for Direct* retrieve knowledge from external sources, instead of purely relying on LLMs' internal knowledge. The decomposed sub-claims by all methods are saved as JSON files in our GitHub repository, which will be made public once the reviewing period ends. The table below shows the knowledge source for each baseline:\n\n| *Baseline* | *Knowledge Source* |\n| ------ | ------ |\n| Direct | LLMs' Internal Knowledge |\n| **CoT** | **Retrieved from Serpapi (same as FOLK)** |\n| **Self-Ask** | **Retrieved from Serpapi (same as FOLK)** |\n| ProgramFC | Wikipedia corpus (Open-book setting in ProgramFC) |\n\nWe select Direct baseline as a vanilla method for directly using LLMs for claim verification tasks. As stated in section 4.2, all the other baselines retrieve external knowledge. We will make it more clear on the knowledge source used by CoT and Self-Ask in the next version.\n\n**Q2:** The motivations in the Abstract and Introduction don't make sense. There are two motivations why the authors designed this method: 1) how to verify claims without relying on human-annotated data is still unresolved 2) previous models cannot provide comprehensive explanations that can justify their decisions. I don't see how the prompt method solves these two problems. Authors should notice that it is the LLMs themselves who solve these two questions, not the prompt method in this paper. The in-context learning or zero-shot ability is emergent when we scale up LMs. The ability to generate fluent and logically correct explanations emerges when we scale up the LMs.\n\n**A2:** Our experiments have shown that LLMs themselves cannot solve the two questions: 1) Table 2 shows that when testing on the same backbone LLMs, our proposed method can achieve better performance than CoT and Self-Ask, which shows the effectiveness of our proposed method. 2) LLMs themselves can generate fluent and logically correct text. However, they can suffer from hallucination problems and produce fluent text with non-factual information. Figure 3 shows the importance of retrieving knowledge from reliable external sources can help with this issue.\n\n**Q3:** Moreover, the authors should split the main result in Table 2 into more tables with different settings, such as one table for models in the open book setting and one for the closed setting.\n\n**A3:** Thank you for your suggestions. Due to the 8-page limit, we cannot split Table 2 into more tables. However, we will consider doing so in our next version.\n\n**Q4:** The ablation study is weak. Authors only show the performance when Knowledge Grounding (search engine) is removed.\n\n**A4:** Figure 3 shows FOLK's performance with and without knowledge-grounding. Additionally, Table 3 shows the performance of using knowledge from different sources.\n\n**Q5:** However, I am curious why authors must use the first-order-logic form of atomic claims that must be checked. Can you use a free text form, like what ToolFormer[1] does, to call API? Can using the first-order logic give more performance improvement?\n\n**A5:** We use FOL predicates to decompose better sub-claims, which is crucial when the input claims are complex or require explicit reasoning abilities, such as numerical reasoning. As shown in Table 2, FOLK achieves better performance than methods without using FOL predicates for claim decomposition. We can use ToolFormer's free-form style to call APIs. However, it will still need FOL predicates to help decompose complex claims.\n\n**Q6:** A writing tip: you need to explain what \"Followup Question\" and \"Grounded Answer\" are in Figure 1. It is better to be self-contained.\n\n**A6:** Thank you for your suggestion. We will add these two terms in the caption of Figure 1 to make it self-contained.\n\n**Q7:** For the \"Self-Ask\" baseline, I think you also need to read this paper \"LEAST-TO-MOST PROMPTING ENABLES COMPLEX REASONING IN LARGE LANGUAGE MODELS\"\n\n**A7:** Thank you for your suggestion. The key idea of Least-to-Most prompting strategy is to let LLM answer subproblems sequentially. However, for fact-checking, the sub-claims are often not related in a sequential manner and cannot be answered by using sub-answers from previous sub-questions.\n\n**Q8:** What about the efficiency of this method? Does calling a search engine cost much more time?\n\n**A8:** Calling Serpapi costs around 0.5 seconds per request. Additionally, this service allows for high throughput. This method has been adopted as an efficient way to retrieve knowledge from external sources by other works [1,2,3].\n\n[1] Yoran, Ori, et al. \"Answering questions by meta-reasoning over multiple chains of thought.\" arXiv preprint arXiv:2304.13007 (2023).\n\n[2] Press, Ofir, et al. \"Measuring and narrowing the compositionality gap in language models.\" arXiv preprint arXiv:2210.03350 (2022).\n\n[3] Lazaridou, Angeliki, et al. \"Internet-augmented language models through few-shot prompting for open-domain question answering.\" arXiv preprint arXiv:2203.05115 (2022).", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. The method proposed by the authors is interesting.\n2. Experiments on three datasets (HoVER, FEVEROUS, and SciFact-Open) shows the effectiveness of this paper.", "reasons_to_reject": "1. What about the efficiency of this method? Does calling a search engine cost much more time?\n2. The baseline of this paper is unfairly compared. Most baselines do not incorporate a search engine and consequently are not conducted in an open book setting, including \"Direct,\" \"CoT,\" and \"Self-Ask.\" In contrast, the prompt method proposed by the authors uses a search engine to get nearly perfect knowledge. Such a comparison makes the experimental results highly doubtful.  In my understanding, tool-using works such as Toolformer[1] are compatible with CoT and Self-Ask. The authors should add retrieved knowledge to those methods too. Moreover, the authors should split the main result in Table 2 into more tables with different settings, such as one table for models in the open book setting and one for the closed setting.\n3. The ablation study is weak. Authors only show the performance when Knowledge Grounding (search engine) is removed. However, I am curious why authors must use the first-order-logic form of atomic claims that must be checked. Can you use a free text form, like what ToolFormer[1] does, to call API? Can using the first-order logic give more performance improvement?\n4. The motivations in the Abstract and Introduction don't make sense. There are two motivations why the authors design this method: 1) how to verify claims without relying on human-annotated data is still unresolved 2) previous models cannot provide comprehensive explanations that can justify their decisions. I don't see how the prompt method solves these two problems. Authors should notice that it is the LLMs themselves who solve these two questions, not the prompt method in this paper. The in-context learning or zero-shot ability is emergent when we scale up LMs. The ability to generate fluent and logically correct explanations emerges when we scale up the LMs.\n5. A writing tip: you need to explain what \"Followup Question\" and \"Grounded Answer\" are in Figure 1. It is better to be self-contained.\n\nRef:\n[1] Schick, Timo, et al. \"Toolformer: Language models can teach themselves to use tools.\" arXiv preprint arXiv:2302.04761 (2023).", "questions_to_authors": "", "paper_topic_and_main_contributions": "The article discusses the importance of claim verification in combating misinformation and the challenge of verifying claims without relying on expensive human-annotated data. The article proposes a new approach called First-Order-Logic-Guided Knowledge-Grounded (FOLK) Reasoning. This method can verify complex claims and generate explanations without annotated evidence using Large Language Models (LLMs). FOLK translates the claim into a First-Order-Logic (FOL) clause and performs FOL-guided reasoning over knowledge-grounded question-and-answer pairs to make veracity predictions and generate explanations to justify its decision-making process. The results of experiments indicate that FOLK outperforms strong baselines on three datasets encompassing various claim verification challenges. The code and data are available for others to use.", "excitement": 3, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Missing_References": "For the \"Self-Ask\" baseline, I think you also need to read this paper \"LEAST-TO-MOST PROMPTING ENABLES COMPLEX REASONING IN LARGE LANGUAGE MODELS\""}, "extra_text": {}}, {"review_id": "MH3IXQsgmj", "reviewer": "Reviewer_ajs8", "summary": "", "questions": "Not relying on human-annotated data is an advantage on which a a lot of emphasis has been given. However, majority of the recent methods (the ones using LLMs) focus on this setting only. I think a bit extra emphasis has been given to this point.\n", "limitations": "", "rating": 3, "confidence": 4, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the thorough feedback and constructive suggestions.\n\n**Q1:** The reason behind first creating a FOL predicate for each sub-claim is not very convincing. Since they are not using any logical reasoning tools like CLINGO for solving the predicate. I don’t see a justification for doing this. Instead, directly instructing the LLMs with a prompt as simple as “Generate questions verifying the sub-claims of the above claim” does very well in directly generating sub-questions that can be used for verifying the claims.\n\n**A1:** We appreciate you bringing up concern regarding the motivation for using FOL predicate to decompose the claims. We use CoT as one of the baselines, which directly instructs the LLMs with a prompt to generate sub-questions that can be used for verifying the claims. The explicit prompt can be found in *Appendix B*, \"Please tell me the necessary questions that need to be answered in order to verify the following claim:\". Our experiment shows that FOLK can outperform CoT across all three datasets, which shows the importance of using FOL to guide LLMs for claim decomposition. Additionally, we observe that for more complex claims, FOLK has an improvement of 17.67% and 25.76% over CoT on 3-hop and 4-hop claims respectively. This further shows the importance of using FOL can better decompose more complex claims.\n\n**Q2:** For Chain-of-Thought and Self-Ask baselines, do you provide the retrieved knowledge to verify the sub-claims as done with FOLK? If yes, then how would you justify the better performance of your approach since both these approaches also result in accurate questions for verifying the subclaims. In no, then verifying each sub-claim (decomposed by existing methods) leveraging the retrieved knowledge should be a fair baseline in comparison to the proposed approach that uses the retrieved knowledge.\n\n**A2:** (1) We retrieved knowledge for Chain-of-Thought and Self-Ask baselines, which use the same Serpapi calls as used in FOLK. (2) Although both baselines and FOLK retrieve knowledge from an external source, they each generate decomposed sub-claims differently. FOLK can generate better sub-claims especially when the claim becomes more complex, such as more hops or require numerical reasoning. The decomposed sub-claims by all methods are saved as JSON files in our GitHub repository, which will be made public once the reviewing period ends.\n\n**Q3:** Furthermore, since creating FOL predicates is not necessarily a trivial task (at least more complicated than prompting it to decompose the given claim), an error in creating FOL predicates can propagate in generating the followup questions. On the other hand, directly generating the decomposed sub-claims has been shown to work really well in prior works.\n\n**A3:** Thank you for bringing up the concern of error propagation of creating FOL predicates. This is a valid concern that could happen. Since checking the correctness of generated FOL predicates is a non-trivial task, we would like to explore this research direction further.\nOur experiment shows that FOLK can outperform methods that directly generate the decomposed sub-claims (CoT and Self-Ask).\n\n**Q4:** Discrepancies in the presented results of ProgramFC approach. As reported in the ProgramFC paper, it achieves 60.63 on HoVER 3-hop (as compared to 54.80 of FOLK) and similarly for the FEVEROUS dataset. However, this paper reports 51.04 on the HoVER 3-hop. Please provide a justification for this.\n\n**A4:** As stated in Section 4.1, we use stratified sampling to select 100 examples for each sub-dataset (e.g. 3-hop, 4-hop, numerical) in HoVER and FEVEROUS datasets. This is to limit the overall API call cost, which is a common practice used by other works [1,2] that require OpenAI API calls. On the other hand, ProgramFC performs experiments on the entire validation set of HoVER and FEVEROUS. Therefore, the performance of ProgramFC is different from our paper.\n\n[1] Yoran, Ori, et al. \"Answering questions by meta-reasoning over multiple chains of thought.\" arXiv preprint arXiv:2304.13007 (2023).\n\n[2] Trivedi, Harsh, et al. \"Interleaving retrieval with chain-of-thought reasoning for knowledge-intensive multi-step questions.\" arXiv preprint arXiv:2212.10509 (2022).\n\n**Q5:** A human study on how well the verification process leveraging only the top-1 search result is required. Also, why not 5 or maybe more search results are used as the knowledge because the first result may not always contain sufficient information to verify the sub-claim.\n\n**A5:** (1) We appreciate your suggestion on evaluating the quality of search results. We will add its study results in the next version of the paper. (2) Adding the top 5 search results could potentially add more information. However, it also adds to the API cost. Additionally, we found that the source of knowledge has a significant impact on the performance, as shown in Table 3.\n\n**Q6:** Analyzing the relation between correct claim verification decision and the correct generated explanation can be added in the analysis section.\n\n**A6:** We appreciate your suggestion on analyzing the relation between correct verification decision and correct generated explanation. Since performing human evaluation takes time to facilitate, we will add the results in the next version of our paper.\n\n**Q7:** A lot of emphasis in the paper has been on generating explanations; however, only 30 examples have been evaluated which may not provide enough evidence to support the claims of the paper.\n\n**A7:** We follow the practice from similar works [1,2] on human evaluation of generated explanations. These two works use 40 instances for human evaluation.\n\n[1] Atanasova, Pepa, et al. \"Generating fact checking explanations.\" ACL (2020).\n\n[2] Jolly, Shailza, Pepa Atanasova, and Isabelle Augenstein. \"Generating fluent fact checking explanations with unsupervised post-editing.\" arXiv (2021).\n\n**Q8:** Not relying on human-annotated data is an advantage on which a a lot of emphasis has been given. However, majority of the recent methods (the ones using LLMs) focus on this setting only. I think a bit extra emphasis has been given to this point.\n\n**A8:** While methods that use LLMs can verify claims without human-annotated data, our method adds better accuracy and interpretability, which are two important areas to consider for the claim verification task.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "Claim verification is an important topic with practical significance.\n\nThe proposed approach is intuitive, easy to implement, and effective. The experiments show that it leads to some performance improvements. \n\nFurthermore, the paper is well written and provides a good background for the problem statement.  \n", "reasons_to_reject": "Weaknesses: \n\nThe reason behind first creating an FOL predicate for each sub-claim is not very convincing. Since they are not using any logical reasoning tools like CLINGO for solving the predicate. I don’t see a justification for doing this. \nInstead, directly instructing the LLMs with a prompt as simple as “Generate questions verifying the sub-claims of the above claim” does very well in directly generating sub-questions that can be used for verifying the claims. \n\nFor Chain-of-Thought and Self-Ask baselines, do you provide the retrieved knowledge to verify the sub-claims as done with FOLK? If yes, then how would you justify the better performance of your approach since both these approaches also result in accurate questions for verifying the subclaims.\nIn no, then verifying each sub-claim (decomposed by existing methods) leveraging the retrieved knowledge should be a fair baseline in comparison to the proposed approach that uses the retrieved knowledge. \n\nFurthermore, since creating FOL predicates is not necessarily a trivial task (atleast more complicated than prompting it to decompose the given claim), an error in creating FOL predicates can propagate in generating the followup questions. On the other hand, directly generating the decomposed sub-claims has been shown to work really well in prior works. \n\n\nDiscrepancies in the presented results of ProgramFC approach. As reported in the ProgramFC paper, it achieves 60.63 on HoVER 3-hop (as compared to 54.80 of FOLK) and similarly for the FEVEROUS dataset. However, this paper reports 51.04 on the HoVER 3-hop. Please provide a justification for this.\n\nA human study on how well the verification process leveraging only the top-1 search result is required. Also, why not 5 or maybe more search results are used as the knowledge because the first result may not always contain sufficient information to verify the sub-claim.\n\nAnalyzing the relation between correct claim verification decision and the correct generated explanation can be added in the analysis section. \n\nA lot of emphasis in the paper has been on generating explanations; however, only 30 examples have been evaluated which may not provide enough evidence to support the claims of the paper.", "questions_to_authors": "Not relying on human-annotated data is an advantage on which a a lot of emphasis has been given. However, majority of the recent methods (the ones using LLMs) focus on this setting only. I think a bit extra emphasis has been given to this point.\n", "paper_topic_and_main_contributions": "This paper proposes a method to verify claims using the ability of LLMs to first decompose a given claim into several sub-claims and then verify each sub-claim by retrieving relevant knowledge.\n\nSpecifically, they first translate a given claim into a First-Order-Logic clause consisting of several predicates, each corresponding to a part of the claim that needs to be verified.\n\nThen, for each predicate, they create a followup question, retrieve relevant knowledge, and generate an answer for the followup question leveraging the retrieved knowledge. \n\nThen, they evaluate the boolean value of the FOL clause and make a prediction as one of SUPPORTS or REFUTES. \n\nThey also prompt the LLM to generate an explanation for its decision-making process.\n\nThey evaluate the proposed method on 3 fact-checking datasets and demonstrate its effectiveness. \n", "excitement": 3, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: Summary: This paper proposes to verify claims by generating intermediate questions. Google's search results are used to answer these questions, and the answers are then used to produce an answer and an explanation. The paper compares different prompts, such as CoT and Self-Ask. The main contribution of the paper is an improved prompt format called FOLK, which involves finding prompts for first-order-logic-style predicates in the claim and generating follow-up questions based on those predicates. The results show that the FOLK approach improves performance, especially for more complex questions. 

Strength: This paper presents a straightforward yet promising prompting method for fact-checking. The paper is well-written and easy to follow. Additional experimental results in the author's response further indicate that the presence of the FOL forms helps LLMs combine knowledge in multi-hop cases. Most of the reviewers agree that this is a solid contribution but the excitement is relatively limited. 

Weaknesses: I don't think there are any major weaknesses -- most of the weaknesses have been addressed during the discussion phase. The authors should take the experimental results during rebuttal into account when preparing for the final version of the paper.
Decision: Accept-Findings