O pen F act C heck: A Unified Framework for Factuality Evaluation of LLM s (2024)

Abstract: The increased use of large language models (LLMs) across a variety of real-world applications calls for automatic tools to check the factual accuracy of their outputs, as LLMs often hallucinate. This is difficult as it requires assessing the factuality of free-form open-domain responses. While there has been a lot of research on this topic, different papers use different evaluation benchmarks and measures,which makes them hard to compare and hampers future progress. To mitigate these issues, we developed OpenFactCheck, a unified framework, with three modules: (i) RESPONSEEVAL, which allows users to easily customize an automatic fact-checking system and to assess the factuality of all claims in an input document using that system, (ii) LLMEVAL, which assesses the overall factuality of an LLM, and (iii) CHECKEREVAL, a module to evaluate automatic fact-checking systems. OpenFactCheck is open-sourced (https://github.com/mbzuai-nlp/openfactcheck) and publicly released as a Python library (https://pypi.org/project/openfactcheck/) and also as a web service (http://app.openfactcheck.com). A video describing the system is available at https://youtu.be/-i9VKL0HleI.

## introduction large language models (llms) have demonstrated impressive capabilities in generating naturallysounding answers over a broad range of human inquiries. however, @xcite and other text generation models still produce content that deviates from real-world facts @xcite @xcite . this degrades the performance of llms and undermines their reliability, which is a significant bottleneck for their deployment @xcite , especially for high-stake applications, e.g., clinical, legal, and financial settings. figure 1 : overview of the openfactcheck demo system for llm factuality evaluation and its modules. green responseeval: a customized fact-checker to identify factual errors given text inputs. orange llmeval: an llm factuality evaluator to assess the llm factual ability from different aspects and then to produce a report to illustrate its weaknesses and strengths. purple check-ereval: a fact-checker evaluator and leaderboard to encourage the development of advanced checkers in terms of performance, latency and costs. many studies have explored evaluating the factuality of llms @xcite @xcite . two challenges have been identified: (i) it is difficult to assess the factuality of open-domain free-form responses, and (ii) different papers use different evaluation datasets and measures, which makes it hard to compare them, thus hampering future progress @xcite . to mitigate these issues, we introduce openfactcheck. openfactcheck is an open-source factuality evaluation framework for llms and it comprises the following three core modules (see figure 1 ): the modules are designed for seamless integration, each contributing to and enhancing the capabilities of the others. the results of human verification derived from llmeval can be used as the benchmark for evaluating the accuracy of automated fact-checkers. simultaneously, the most effective checker identified in checkereval can be deployed for automated fact-checking tasks. each fact-checker in checkereval can be an implementation in responseeval. complex user inquiries may be considered as potential candidates of the factuality assessment dataset utilized in llmeval. general users can tailor their checkers according to their specific needs, such as domain specialization, cost-effectiveness, or rapid processing, and identify factual errors for both human-written text (a claim or document) and the outputs of llms. llm researchers and practitioners can directly submit their llm responses to the llmeval by downloading our question set. subsequently, we conduct evaluations to assess the model's factual accuracy and to generate a report analyzing the model performance from multiple aspects. similarly, developers who seek to evaluate and to fairly compare the efficacy of their fact-checking systems to other ones can upload their checker's verification outcomes to checkereval. then, our system will show the ranking information in the leaderboard after evaluating under the same measurements. to sum, three modules of openfactcheck respectively address the following: we have launched an open-source initiative that includes the development of a python library and a web interface tailored to support three major functionalities. this foundation is expected to act as a catalyst for future advancements in factuality evaluation for llms. we encourage extensive implementation of unique, effective, and robust claim processors, retrievers and verifiers within fact-checking pipelines, collections of challenging questions that llms tend to make factual errors, and human-annotated fine-grained verification examples. we believe that this will help to promote and to advance future research on llm factuality. ## related work while numerous automatic fact-checking systems have developed, such as rarr, factscore, factool, factcheck-gpt, longform safe and fire @xcite @xcite @xcite , they are often inaccessible to general users who lack a python environment to compile code and run verification. although these systems can function as the backend of a service, a user-friendly web interface is necessary to allow general users to verify text inputs by simply typing or copying text and clicking a check button. openfactcheck addresses this by providing an accessible web interface. in addition, various fact-checking systems have distinct advantages. for instance, factcheck-gpt offers a fine-grained framework to involve all possible subtasks that could improve the factchecking system, factool uses a low-latency evidence retriever through asynchronous processing, and factscore introduces a scoring metric that calculates the percentage of true claims in a given text, thereby quantitatively assessing the credibility of the input. openfactcheck integrates these components into a unified system @xcite . recent open-sourced demo system loki @xcite also aims to leverage strength of various automatic fact-checkers, while it emphasizes optimization a single fact-checking system in terms of accuracy, latency, robustness, cost-efficiency, and extensive support for multiple languages and llms. in contrast, openfactcheck is a unified framework to cover three major functionalities for factuality evalaution of llms, including customizing a fact-checker by combining modules of different checkers, assessing llm factuality from various perspectives, and evaluating the accuracy of automatic fact-checkers @xcite . ## system architecture the design of openfactcheck emphasizes two principles: (i) customizability and extensibility for both users and developers, and (ii) compatibility with existing methods and datasets. it consists of three modules: responseeval, llmeval, and checkereval. we detail the design and implementation of each components below. ## access and deployment openfactcheck is accessible via a user-friendly web interface and features an integrated database that maintains a user leaderboard. it is also available as a standalone open-source python library. ## conclusion and future work we implemented a unified, easy-to-use and extensible framework openfactcheck. it is accessible by both python library and web service, supporting the customization and evaluation of automatic fact-checking systems and llm factuality evaluation. specifically, openfactcheck allows general users to check whether a claim and a document are factual or not by clicking check, and also facilitate llm practitioners and developers to effectively and efficiently evaluate the factuality of their llms from various perspectives, and to assess the accuracy of automatic fact-checking systems. in the future, we will continue to integrate new techniques, features, and evaluation benchmarks to openfactcheck to facilitate the research progress of llm fact-checking.