Paper: FacTool: Factuality Detection in Generative AI - A Tool Augmented Framework for Multi-Task and Multi-Domain Scenarios (ICLR-2024)

Reviewer comment (Reproducibility):
Although the described framework makes sense to me, the authors highly rely on ChatGPT to implement their framework (even if the function is just to check whether list A is a subset of list B).

Author response:
Thanks for the comment!

> Although the described framework makes sense to me, the authors highly rely on ChatGPT to implement their framework (even if the function is just to check whether list A is a subset of list B); this may hinder other researchers from reproducing their results and additional costs.

We are currently working on integrating FacTool with Llama2 and various other open-source models. Detailed results are:

| Methods | Tasks        | Foundation Model | Claim-Level |      |      |      | Response-Level |      |      |      |
|---------|--------------|------------------|-------------|------|------|------|----------------|------|------|------|
|         |              |                  | Accuracy    | Recall | Precision | F1   | Accuracy    | Recall | Precision | F1   |
| FACTOOL | Knowledge-QA | llama2-70b-chat  | 73.99       | 93.49 | 77.07      | 84.49| 52.00       | 78.26 | 48.65      | 60.00|
| FACTOOL | Code         | llama2-70b-chat  | 55.75       | 50.00 | 76.00      | 60.32| 55.75       | 50.00 | 76.00      | 60.32|
| FACTOOL | Math         | llama2-70b-chat  | 86.62       | 100.00| 86.62      | 92.83| 47.00       | 100.00| 47.00      | 63.95|
| FACTOOL | Scientific   | llama2-70b-chat  | 96.77       | 81.82 | 100.00     | 90.00| 98.00       | 80.00 | 100.00     | 88.89|

We note that the error detection ability of FacTool powered by Llama-2-70b-chat is slightly constrained due to Llama-2-70b's limited coding capabilities (Llama-2-70b scores only 29.9% on the HumanEval benchmark, in contrast to ChatGPT-3.5's 48.1%).

Reviewer comment (Reproducibility):
This high reliance on ChatGPT may hinder other researchers from reproducing their results and may incur additional costs.

Author response:
Thanks for the comment!

> Although the described framework makes sense to me, the authors highly rely on ChatGPT to implement their framework (even if the function is just to check whether list A is a subset of list B); this may hinder other researchers from reproducing their results and additional costs.

We are currently working on integrating FacTool with Llama2 and various other open-source models. Detailed results are:

| Methods | Tasks        | Foundation Model | Claim-Level |      |      |      | Response-Level |      |      |      |
|---------|--------------|------------------|-------------|------|------|------|----------------|------|------|------|
|         |              |                  | Accuracy    | Recall | Precision | F1   | Accuracy    | Recall | Precision | F1   |
| FACTOOL | Knowledge-QA | llama2-70b-chat  | 73.99       | 93.49 | 77.07      | 84.49| 52.00       | 78.26 | 48.65      | 60.00|
| FACTOOL | Code         | llama2-70b-chat  | 55.75       | 50.00 | 76.00      | 60.32| 55.75       | 50.00 | 76.00      | 60.32|
| FACTOOL | Math         | llama2-70b-chat  | 86.62       | 100.00| 86.62      | 92.83| 47.00       | 100.00| 47.00      | 63.95|
| FACTOOL | Scientific   | llama2-70b-chat  | 96.77       | 81.82 | 100.00     | 90.00| 98.00       | 80.00 | 100.00     | 88.89|

We note that the error detection ability of FacTool powered by Llama-2-70b-chat is slightly constrained due to Llama-2-70b's limited coding capabilities (Llama-2-70b scores only 29.9% on the HumanEval benchmark, in contrast to ChatGPT-3.5's 48.1%).

Reviewer comment (Novelty):
The technical contributions are somewhat limited and mostly incremental.

Author response:
Thanks for the comment!

> The technical contributions are somewhat limited and mostly incremental. 

We want to emphasize the novelty of our paper:

(i) **Motivation**: Given that a wider range of tasks now face an increasing risk of containing factual errors when handled by generative AI. We aim to provide a unified framework for detecting factual errors across various tasks and domains. 

(ii) **A Unified Framework for Multi-Task and Multi-Domain Scenarios**: FacTool presents a unified tool-augmented framework for detecting factual errors across diverse tasks and domains, including KBQA, coding, math, and scientific. 

(iii) **An automatic factuality evaluator for modern chatbots across different scenarios**: FacTool is essentially a automate factuality evaluator that can be used to evaluate the factuality of modern chatbots across various scenarios. By utilizing factool, we maintain a factuality leaderboard for chatbots. We believe factool can facilitate the development of modern LLMs by helping developers detect potential factual errors generated by their LLMs.

(iv) **An user-friendly API interface and supports ChatGPT plugin**: We have made significant efforts to ensure that FacTool is user-friendly and supports ChatGPT plugin, enabling both technical and general users to benefit from our research.

Reviewer comment (Novelty):
The intermediate question generation module to validate any claims made by the LLM against an external evidence has been proposed in other works (https://arxiv.org/pdf/2210.08726.pdf, https://arxiv.org/pdf/2309.11495.pdf). The novelty as far as I understand is mostly in claim generation which is different for each task because the inputs are no longer just declarative statements but also things like code snippets, questions, etc.

Author response:
Thanks for the comment!

> The intermediate question generation module to validate any claims made by the LLM against an external evidence has been proposed in other works (https://arxiv.org/pdf/2210.08726.pdf, https://arxiv.org/pdf/2309.11495.pdf). The novelty as far as I understand is mostly in claim generation which is different for each tasks because the inputs are no longer just declarative statements but also things like code snippets, questions, etc. 

We would like to emphasize three main difference between FacTool and RARR:

(i) FacTool provides versatile claim extraction, enabling not only the provision of fine-grained factuality information to help users more easily detect potential factual errors but also the integration of factuality detection across various domains and scenarios. In contrast, RARR neither supports fine-grained factuality nor multi-task factuality.

(ii) FacTool offers a more intuitive and straightforward agreement verification process. After evidence collection, we leverage the strong reasoning capabilities of LLMs to directly feed LLMs with the collected evidence, eliminating the need for extra steps in relevance matching. RARR requires an extra step to parse over webpages and uses a relevant matching algorithm to find the most relevant snippets, which is more complicated compared to FacTool.

(iii) FacTool focuses on “factuality detection”, i.e., detecting factual errors from the generated text from LLMs, while RARR focuses on “text editing”, i.e., editing text that may contain hallucinations.

As for https://arxiv.org/pdf/2309.11495.pdf , this work was completed after FacTool (and actually cites a pre-print of the FacTool paper).

Reviewer comment (Theory):
The authors stretch the definition of factuality to new tasks like code generation which are not necessarily considered "factual". For instance, there can be multiple ways to write a logical code snippet and not necessarily a universal way for it to be deemed as a fact.

Author response:
Thanks for the comment!

> The authors stretch the definition of factuality to new tasks like code generation which are not necessarily considered "factual". For instance, there can multiple ways to write a logical code snippet and not necessarily a universal way for it to be deemed as a fact. 

We agree that there can be multiple ways to write a logical code snippet. In fact, this argument is consistent with our framework since we are adopting an **execution-based approach** to identify whether a code snippet is factual or not. As defined in our paper, the factuality of a code snippet is grounded in an execution-based approach to code evaluation. This approach measures the correctness of generated code by executing it against test case inputs and comparing its output to the expected (or golden) output.

Reviewer comment (Experiments):
The evaluations mostly focus on different language models and not existing methods for factual error detection like RARR or SELF-REFINE which could have been used for at least KBQA.

Author response:
Thanks for the comment!

> The evaluations mostly focus on different language models and not existing methods for factual error detection like RARR or SELF-REFINE which could have been used for at least KBQA.

As mentioned in the response of Weakness 2, RARR focuses on text editing, which is different from our focus on factuality detection. Also, the workflow of RARR doesn’t include a claim extraction process, which is very different from our focus to provide fine-grained claim-level information to users. Thus, we did not include RARR in our baseline.

On the other hand, we included 'Self-Check', essentially the first step of 'Self-Refine', as our baseline. It's important to note that our focus is on 'detection' rather than 'refinement'. Therefore, we use the first step of 'Self-Refine' as our baseline and refer to it as 'Self-Check'.

Reviewer comment (Reproducibility):
Could you please describe how the baseline method Self-Check(*) was setup? Does it use some existing code or were the prompts developed by the authors?

Author response:
Thanks for the comment!

> Could you please describe how the baseline method Self-Check(*) was setup? Does it use some existing code or were the prompts developed by the authors? 

The prompts are developed by the authors, based on the intuition of Self-Refine. Self-Refine has been acknowledged as a strong baseline for LLM to refine their own response. Recall that the first step of SELF-REFINE begins by generating an output, which is then fed back into the same LLM to obtain feedback. This is coherent with our focus, factuality detection. Therefore, we utilize the first step of Self-Refine as our baseline and named it as “Self-Check”.

Reviewer comment (Experiments):
In question generation module for code generation task, how will the model know what are the right corner case inputs for unit testing the piece of code?

Author response:
Thanks for the comment!

> In the question generation module for code generation task, how will the model know what are the right corner case inputs for unit testing the piece of code? 

The test case inputs are generated automatically by LLMs through prompt-based learning. We acknowledge that, at times, the variety of generated test cases can be limited, as mentioned in Figure 16 of Appendix D. However, we believe that further prompt engineering and advancements in more sophisticated LLMs can mitigate this issue.

Reviewer comment (Experiments):
In Appendix C (Fig 6) some characters in the prompt are all-caps, was there any specific reason for that. Out of curiosity, does standard casing yield bad performance?

Author response:
Thanks for the comment!

> In Appendix C (Fig 6) some characters in the prompt are all-caps, was there any specific reason for that. Out of curiosity, does standard casing yield bad performance?

We follow the same intuition as in https://github.com/openai/evals, which uses capitalization or brackets to emphasize certain parts of the prompt.

Reviewer comment (Novelty):
The novelty of the approach appears limited. While the paper employs LLMs to generate search queries and refers to search engines as tools, this methodology isn't drastically different from prior evidence-based fact-checking methods that use search engines for evidence retrieval.

Author response:
Thank you for the comment!

> The novelty of the approach appears limited. While the paper employs LLMs to generate search queries and refers to search engines as tools, this methodology isn't drastically different from prior evidence-based fact-checking methods that use search engines for evidence retrieval.

The fact checking methodology of FacTool differs from prior works that use search engines for evidence retrieval in evidence-based fact checking, with its **versatile claim extraction** module. This module effectively provides **fine-grained factuality information for long-form text across various domains and scenarios** that enables a **unified framework for factuality detection across different scenarios**. 

Classical evidence-based fact-checking methods [1, 2, 3, 4] typically follow a three-step pipeline: document retrieval, sentence retrieval, and claim verification. This pipeline is designed to address scenarios where the **claims are already given**. In contrast, FacTool is capable of fact-checking **long-form texts in which claims are not explicitly given**.

More recent prior works such as RARR [5] utilize search engines for evidence retrieval but also do not employ a claim extraction process to collect detailed factuality information. Other concurrent works like FactScore [6] employ a claim extraction module to extract atomic facts from long-form texts but are limited to the Knowledge-Based Question Answering (KBQA) task. Our work expands the definition of factuality beyond the KBQA task and does not solely focus on using search engines for evidence retrieval. FacTool is designed to be versatile and capable of handling multiple tasks, integrating various tools such as search engines, Python executors, and Google Scholar, as demonstrated in our paper. This comprehensive integration enables FacTool to tackle a wider spectrum of factuality issues in generative AI (GAI), overcoming the limitations of existing works that are focused on single tasks or specific scenarios.

References:

[1] Bekoulis et al.. A review on fact extraction and verification

[2] Thorne et al. FEVER: a large-scale dataset for fact extraction and VERification

[3] Zhong et al. Reasoning Over Semantic-Level Graph for Fact Checking

[4] Zhou et al. GEAR: Graphbased Evidence Aggregating and Reasoning for Fact Verification

[5] Gao et al. Rarr: Researching and revising what language models say, using language models

[6] Min et al. FACTSCORE: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generation

Reviewer comment (Experiments):
Despite the paper's emphasis on fact-checking, notable fact-checking datasets like FEVER and SciFact are absent from the evaluation.

Author response:
Thank you for the comment!

> Despite the paper's emphasis on fact-checking, notable fact-checking datasets like FEVER and SciFact are absent from the evaluation.

In our paper, we focus on the growing challenges of factual errors generated by models in generative AI like ChatGPT. To this end, we have constructed a dataset based on the natural distribution of errors generated by these models. Fact-checking datasets like FEVER and SciFact are not natural distribution of errors generated by LLMs, so we didn’t focus on these datasets. Additionally, both FEVER and SciFact provide claims, which do not align with our task setting: offering an end-to-end system capable of delivering detailed, claim-level factuality information from long-form context.

Reviewer comment (Experiments):
Were any experiments conducted using preivous LLMs or non-LLM-based fact-checking systems within the framework of your experimental settings?

Author response:
> Were any experiments conducted using preivous LLMs or non-LLM-based fact-checking systems within the framework of your experimental settings?

**Please refer to the response of Weakness 2.**