FactBench: A Dynamic Benchmark for In-the-Wild Language Model Factuality Evaluation

## Abstract

Abstract Language models (LMs) are widely used by an increasing number of users, underscoring the challenge of maintaining factuality across a broad range of topics. We first present VERIFY ( V erification and E vidence R etr I eval for F actualit Y evaluation), a pipeline to evaluate LMs’ factuality in real-world user interactions. VERIFY considers the verifiability of LM-generated content and categorizes content units as supported , unsupported , or undecidable based on the retrieved evidence from the Web.
Importantly, factuality judgment by VERIFY correlates better with human evaluations than existing methods.
Using VERIFY , we identify “hallucination prompts” across diverse topics, i.e., those eliciting the highest rates of incorrect and inconclusive LM responses. These prompts form FactBench , a dataset of 1K prompts across 150 fine-grained topics. Our dataset captures emerging factuality challenges in real-world LM interactions and can be regularly updated with new prompts.
We benchmark widely-used LMs from GPT, Gemini, and Llama3.1 family on FactBench , yielding the following key findings: (i) Proprietary models exhibit better factuality, with performance declining from Easy to Hard hallucination prompts. (ii) Llama3.1-405B-Instruct shows comparable or lower factual accuracy than Llama3.1-70B-Instruct across all evaluation methods due to its higher subjectivity that leads to more content labeled as undecidable . (iii) Gemini1.5-Pro shows a significantly higher refusal rate, with over-refusal in 25% of cases. Our code and data are publicly available at https://huggingface.co/spaces/launch/factbench .

## 1 Introduction

Despite ongoing efforts to enhance their factuality, language models (LMs) continue to generate false or irrelevant content, often known as hallucination (Huang et al., ; Liu et al., ). The widespread use of LMs and the evolving nature of information demand a dynamic factuality evaluation benchmark to identify the challenges LMs face in real-world applications.
Current long-form factuality evaluation benchmarks (Min et al., ; Wei et al., ; Malaviya et al., ) are constrained by being static and have a narrow coverage of usage scenarios. The static design makes these benchmarks susceptible to data leakage (Magar & Schwartz, ), rendering them unsuitable for capturing factuality challenges in real-world LM usages.
Moreover, existing benchmarks often focus on a narrow subset of tasks. For instance, data used in developing FactScore (Min et al., ) primarily addresses biographical queries, while ExpertQA (Malaviya et al., ) recruits human experts to curate domain-specific questions. Other benchmarks (Chen et al., ; Wei et al., ) cover queries that are either LM-generated or human-curated, limiting their applicability to in-the-wild scenarios.

In this work, we make two primary contributions: (i) a dynamic and diverse factuality evaluation benchmark curated from real-world LM usage and (ii) a factuality evaluation framework that measures the appropriateness of in-the-wild prompts for inclusion in our benchmark by estimating how frequent strong LMs generate incorrect and inconclusive responses.
Concretely, we introduce FactBench, an updatable benchmark grounded in the real-world usage of LMs. FactBench comprises 1,000
diverse information-seeking prompts across 150 topics (examples in Figure 2),
that can cause LMs to produce false and inconclusive responses. Using clustering methods, we begin by identifying 382 unique tasks within the LMSYS-Chat-1M dataset (Zheng et al., ). Prompts within each task cluster are then labeled for verifiability, indicating whether the prompt’s response can be verified using Google Search results. We then assess the usefulness of prompts by considering factors such as clarity, interest, and relevance to a broad audience. Finally, verifiable prompts that meet a specified usefulness threshold become candidates for inclusion in FactBench.

We further define the “hallucination prompts” as the ones that can elicit incorrect or inconclusive content from a selected group of strong LMs (e.g., GPT-4 and Claude-2). To identify such prompts, we design VERIFY, a Verification and Evidence RetrIeval for FactualitY evaluation pipeline to detect nonfactual content in LM responses. VERIFY first extracts content units from model responses and identifies their type (e.g., facts, instructions, disclaimers, etc.).
It then evaluates only the verifiable units against Web-based evidence using an interactive query generation and evidence retrieval technique. Finally, VERIFY categorizes units as supported, unsupported, or undecidable based on the evidence.

LMs should ideally generate factual outputs that can be verified against reliable sources. To quantify the degree of hallucination in model responses, we propose a hallucination score that penalizes both incorrect claims (unsupported) and claims that cannot be verified due to ambiguity, partial correctness, or missing context (undecidable).
This score measures the appropriateness of the corresponding user prompts for our final benchmark.
We categorize prompts into three tiers (Hard, Moderate, and Easy) based on the responding models’ strength and select those with the highest hallucination scores within each tier. This results in a final benchmark of 1,000 prompts after manual inspection.

To assess the performance of widely-used LMs on FactBench, we evaluate two proprietary models, i.e., GPT4-o (OpenAI, ) and Gemini1.5-Pro (Team et al., ), and two open-weight models from Llama3.1 family, i.e., Llama3.1-70B-Instruct and Llama3.1-405B-Instruct (Meta, ).
The results show that LM performance significantly increased across tiers, aligning with our curation strategy. To compare the effectiveness of different factuality evaluation methods, we use VERIFY units as a common basis and feed them into factuality evaluation baselines for verification. Our results reveal that VERIFY achieves the highest correlation with human judgments compared to state-of-the-art methods. This finding underscores the effectiveness of our approach in benchmark creation and factuality assessment.

In summary, our contributions are as follows:

- •
We introduce FactBench, a new benchmark grounded in the real-world usage of LMs. FactBench is designed to be updatable by continuously incorporating newly collected hallucination prompts that cause LMs to generate incorrect content. This dynamic approach ensures that the benchmark remains relevant, addressing the evolving challenges in factual generation.
- •
We design VERIFY, a factuality evaluation pipeline that considers the verifiability of generated content and categorizes units into supported, unsupported, or undecidable according to retrieval results. VERIFY addresses the limitations of prior work that only makes binary decisions on supportedness and achieves the highest average correlation with human evaluations.
- •
We release factuality annotations by humans on 4,467 content units, with each unit independently judged by two annotators. Each annotator evaluates the independence of extracted units and their factuality using Google Search.
This human-annotated data provides quantifiable evaluation resources for assessing future factuality evaluation techniques.

## 2 Related Work

### 2.1 Factuality Evaluation Benchmarks

The widespread adoption of LMs, coupled with their tendency to hallucinate, demands new benchmarks that can effectively identify their factual weaknesses across diverse scenarios.
Prior factuality evaluation benchmarks mainly focus on short-form and human-curated question-answering (QA) tasks.
For instance, TruthfulQA (Lin et al., ), HaluEval (Li et al., ), and FELM (Chen et al., ) mostly focus on short-form knowledge-based QA with questions with human-selected topics, despite LMs typically engaging in long-form conversations.
The data used in developing FactScore (Min et al., ), while long-form, is limited to a single, relatively easy task of biographical QA. LongFact (Wei et al., ) expands to 38 human-selected topics, but the prompts are LM-generated rather than user-driven. FactCheck-Bench (Wang et al., ) collects ChatGPT hallucinations from Twitter, but its scope is narrow (94 prompts) and focuses on a specific and rather obsolete model. Moreover, all these datasets are static and prone to the data contamination issues (Magar & Schwartz, ). We fill these gaps by offering a benchmark that systematically mines hallucination prompts from in-the-wild user-model chat logs in LMSYS-chat-1M Zheng et al. (), spanning diverse topics.
FactBench is designed to be regularly updated as new prompts are gathered from real-world interactions, overcoming the constraints of fixed time frames and ensuring ongoing relevance to evolving LM capabilities and usages.

### 2.2 Factuality Evaluation Methods and Verifiability

The challenge of distinguishing verifiable from non-verifiable claims is central to fact-checking. AFaCTA (Ni et al., ) stresses that claims are verifiable when they provide sufficient specificity for evidence retrieval. The subjective nature of check-worthiness, shaped by political and social contexts (Konstantinovskiy et al., ; Nakov et al., ), complicates this, particularly in LM-generated content where fact-opinion lines blur (Vosoughi et al., ). To address this, VERIFY introduces an undecidable label for claims with ambiguous factuality, accommodating both objective and context-dependent claims (see details in Section 4.1).

Long-form content evaluation presents unique challenges due to its complexity and the numerous claims it typically contains. To address these challenges, SAFE (Wei et al., ) and FactScore (Wang et al., ) decompose content into individual facts for granular verification. Our method, VERIFY, builds upon this approach by not only decomposing LM-generated content into units but also distinguishing between verifiable and non-verifiable elements that appear in user-model interactions.
While VeriScore (Song et al., ) similarly recognizes that not all LM-generated content is verifiable, its single-step approach to extracting and decontextualizing verifiable claims may limit its effectiveness in complex scenarios. Moreover, VeriScore’s fixed-size context window could fail to capture long-range dependencies in real-world responses, potentially missing crucial context for accurate fact-checking. We address these limitations through a multi-step process that leverages the complete LM response to carefully identify independent verifiable content units. Our pipeline then evaluates these verifiable units by classifying them as supported or unsupported only when confident evidence is found, labeling them undecidable otherwise. This approach introduces a more robust framework for evaluating the factual precision of LM-generated content. In contrast, Factcheck-GPT (Wang et al., ) extracts coarser-grained units and heavily relies on parametric knowledge, limiting its reliability for factual evaluation.

Figure: Figure 1: This figure outlines the two-step process we use to evaluate LM responses. Step 1 (left) involves cleaning, clustering, and evaluating prompts for verifiability and usefulness. Step 2 (right) decomposes responses into units, retrieves external evidence, and generates factuality labels (supported, unsupported, undecidable) with a hallucination score to flag inaccuracies. This process involves the collection and appropriateness assessment of hallucination prompts.
Refer to caption: x1.png

## 3 Harvesting Hallucination Prompts From LM Interactions

Our current understanding of LM performance on verifiable tasks is limited, and existing factuality evaluation benchmarks cover only a narrow range of verifiable use cases.
To address this gap, we collect English prompts from the first turn of conversations in the LMSYS-Chat-1M dataset (Zheng et al., ), which is a large-scale, in-the-wild LM conversations dataset. Our objective is to identify prompts that are verifiable, diverse, and relevant for evaluating model responses through a multi-step process, which is described below. Figure 1 (left) outlines our collection process.

- •
Data Clustering: After cleaning the data (see details in Appendix 12.1), we get 294,333 distinct prompts and cluster them into various topics. We use BERTopic (Grootendorst, ), a dynamic topic-modeling pipeline that (1) embeds prompts using OpenAI’s text-embedding-3-small model (OpenAI, ), (2) applies UMAP (McInnes et al., ) for dimensionality reduction, and (3) employs HDBSCAN (Malzer & Baum, ), a hierarchical density-based clustering algorithm. HDBSCAN identifies 142,702 (48.5%) of the prompts as outliers, which we exclude to remove overly-specific prompts referring to niche user requests. Finally, we use a class-based TF-IDF method to select the top 100 most representative prompts from each cluster and summarize them into concise topics (up to 10 words) using GPT-4 Turbo (OpenAI, ). This results in 382 clusters with verbalized topic examples demonstrated in Appendix Figure 9.
- •
Verifiability Classification: We focus on prompts that ask for varying degrees of verifiable responses. To identify these, we employ Llama3-70B-Instruct (AI@Meta, ) to distinguish between verifiable and non-verifiable prompts (see classification prompt and the portion of verifiable prompts within each cluster in Appendix 12.10.2 and Figure 9). Overall, verifiable prompts constitute 45.8% of total prompts from the previous step.
- •
Usefulness Evaluation. The remaining collection contains around 70K prompts, too large for manual or automated fact-checking to identify hallucination prompt as described in Section 4.
Randomly selecting a subset for evaluation would be suboptimal, as it may include unclear or over-specific requests. Therefore, we propose a set of criteria to identify useful prompts. A useful prompt needs to be (i) clear and understandable, (ii) generalizable to various users or scenarios, (iii) potentially interesting or valuable to a broader audience, and (iv) is within the capabilities of LMs (e.g., excludes real-time data). To reduce single-model bias, we use two language models, GPT-4-Turbo and Llama3-70B-Instruct, as annotators. Each model scores prompts on a scale of 1 (low) to 5 (high) for each criterion. The scoring prompt is provided in Appendix 12.10.3.
The final usefulness score for each prompt is calculated as
the average score across all criteria, summing the score from two models. The usefulness scores are then used to select an initial set of prompts, as described in Section 5.

## 4 VERIFY : Verification and Evidence Retrieval for Factuality Evaluation

In this section, we design an automatic factuality evaluation pipeline called VERIFY to measure the appropriateness of these prompts, i.e., based on whether they tend to elicit nonfactual responses from LMs, for inclusion into the FactBench. Our goal is to verify model responses in natural user-LM settings, where responses may include a mixture of verifiable and non-verifiable statements. To evaluate model responses efficiently, we first establish the key criteria for determining the verifiability of statements.

### 4.1 Verifiability-driven Factual Evaluation

A statement is verifiable if it provides sufficient information to guide fact-checkers in verification (Ni et al., ).
We classify verifiable statements into two categories:

Context-independent Statements: These are objective assertions that can be directly verified against knowledge sources. For example, “RTX 3060 has a memory bandwidth of 360 Gbps.”

Context-dependent Statements: These statements require additional information for verification. For instance, to verify the statement
“The difference in memory bandwidth between the RTX 3060 and RTX 3060 Ti is relatively small”,
one needs knowledge of both GPUs’ bandwidths and an understanding of what qualifies as relatively small in this context.

Focusing only on verifiable statements during user-LM interactions allows us to assess the factuality of the response more efficiently and accurately, as we describe in the following
section.

### 4.2 Unit Extraction and Labeling

User requests span a wide range of domains (examples provided in Figure 2), and model responses contain a variety of content types.
To evaluate verifiable statements, we first decompose the model response into independent content units. A content unit can represent a Fact, a Claim, an Instruction, a Disclaimer, a Question, or other relevant content type that appears during user-model interaction. A taxonomy of content types is detailed in Appendix 12.10.4 as part of the prompt description.
Many of these content units are not verifiable, meaning they cannot be fact-checked. Examples include questions and disclaimers, which often pertain to the context of the conversation or the capabilities of the model itself rather than presenting factual information.

We label each unit by its type to identify those suitable for verification. We use the Llama3-70B-Instruct model, which serves as the backbone LM for all tasks in this pipeline, to extract and label content units. A carefully crafted prompt with examples (see Appendix 12.10.4) guides the model in this task. We classify objective statements as Fact
and potentially subjective, context-dependent statements as Claim. Only units labeled as Fact or Claim are passed to the next step.

### 4.3 Unit Decontextualization

Gunjal & Durrett () highlights the importance of “molecular units”—units that contain sufficient information to be uniquely identifiable in factuality assessment. Inspired by that, we implement a unit decontextualization component in our pipeline to minimally revise verifiable units and make them self-contained. The prompt is provided in Appendix 12.10.5.

### 4.4 Query Generation and Evidence Retrieval

In order to verify the self-contained units, we need to retrieve relevant evidence from knowledge sources. We utilize SerperAPI(^1^11https://serper.dev/) for Google Search
and Web-evidence retrieval.

To enhance search quality and ensure the retrieval of relevant evidence that best assists in verification, we implement an interactive query refinement technique. The query generator operates iteratively within an interactive feedback loop. It first produces a query for the target unit, which is then used in Google Search to return relevant snippets. In subsequent rounds, the query generator evaluates the relevance of the retrieved snippets for verifying the target unit and refines the query accordingly. This iterative process continues for up to five rounds, progressively improving the query’s quality and relevance. The final set of queries and associated search results are then passed to the next step for final judgment. The prompt is provided in Appendix 12.10.6.

### 4.5 Final Answer Generation

In this step, the model (Llama3-70B-Instruct) is tasked with making a final decision on units’ accuracy by evaluating multiple rounds of retrieved evidence using Chain-of-Thought prompting (Wei et al., ). For each unit, the model (i) summarizes the relevant knowledge points, (ii) assesses their relationship to the unit, and (iii) determines whether the evidence supports (supported), refutes (unsupported), or is inconclusive (undecidable). The prompt is provided in Appendix 12.10.7. This results in annotation labels for all verifiable units in the original model response.
An overview of our evaluation pipeline is provided in the right part of Figure 1. VERIFY facilitates affordable factuality checking by leveraging open-weight LMs and is instantiated with Llama3-70B-Instruct in this work.

### 4.6 Hallucination Score

After annotating the individual content units, we propose a hallucination metric to quantify the incorrect and inconclusive contents within a model’s response. Let $U$ represent the set of undecidable units, $C$ the set of unsupported units, and $V$ the total set of extracted verifiable units (Claims and Facts). The final Hallucination Score is computed as follows:

$$ $\displaystyle\vspace{-1pt}H(R)=\frac{|C|+\alpha|U|}{\sqrt{|V|}}\vspace{-1pt}$ (1) $$

Here, $\alpha\in(0,1)$ is a weighting factor that controls the relative importance of undecidable units compared to unsupported ones. This adjustment is necessary because LMs are expected to generate content based on their training data, including web data, though some factual content may not appear in current web searches. Additionally, undecidable units may be correct but lack sufficient evidence in search results for verification.
For experiments, we set $\alpha=0.5$. The denominator, $\sqrt{|V|}$ ensures that the weight of errors (unsupported and undecidable units) grows more significantly relative to the total number of verifiable units, placing stronger emphasis on even a few errors without allowing large sets of units to mask their importance. Using this score, we employ a tiered-based ranking strategy to curate a benchmark, as described next.

## 5 FactBench Dataset

The hallucination score helps us assess prompt appropriateness for our final dataset. To ensure balanced representation across model capabilities, we categorize prompts into three tiers: Hard, Moderate, and Easy, based on the overall performance of responding models(^2^22Model performance rankings are based on the [Chatbot Arena Leaderboard](https://huggingface.co/spaces/lmsys/chatbot-arena-leaderboard), which ranks LMs through pairwise human comparisons). The Hard tier consists of prompts queried to the strongest models in LMSYS-Chat-1M (e.g., GPT-4, Claude-2), preventing weaker models’ higher hallucination rates from dominating our dataset. The full list of models for each tier is provided in Appendix Table 4.

For prompt selection, we apply tier-specific usefulness thresholds: $4$ or higher for Hard, $4.5$ or higher for Moderate, and exactly $5$ for Easy. This approach reflects our assumption that responses from stronger models better indicate prompt appropriateness, hence allowing a lower threshold for the Hard tier. From the 70K prompts collected in Section 3, this filtering yields 4.2K prompts distributed as 53% Hard, 34% Moderate, and 13% Easy.

Figure: Figure 2: Statistics of different factuality benchmarks. FactBench is the first dynamic and in-the-wild factuality evaluation benchmark with diverse topic coverage.
Refer to caption: x2.png

We then evaluate these prompts using the hallucination score (Equation 1) of their corresponding LM responses. We select 1K prompts with the highest scores while maintaining the original tier distribution (532 Hard, 332 Moderate, 136 Easy), then iteratively filter out-of-scope prompts through manual inspection as detailed in Appendix 12.2. Table 2 compares our benchmark statistics with other long-form factuality evaluation benchmarks.
Our work introduces the first real-world factuality evaluation benchmark comprising hallucination prompts across diverse topics. We intend to regularly update FactBench as more prompts are added to the LMSYS-chat-1M data.

## 6 Experimental Setup

Language Models(^3^33We continue to benchmark more LMs on FactBench. Our up-to-date leaderboard is available at [https://huggingface.co/spaces/launch/factbench](https://huggingface.co/spaces/launch/factbench).): We benchmark LMs against FactBench to evaluate their performance on this dataset using different factuality evaluation methods. We evaluate the most recent and powerful models (available via APIs) from proprietary and open-source categories. From the proprietary models, we benchmark GPT-4o (omni) (OpenAI, ) and Gemini1.5-Pro (Team et al., ). For open-source models, we evaluate the highest-ranked open-source models on Chatbot Arena Leaderboard(^4^44[https://huggingface.co/spaces/lmsys/chatbot-arena-leaderboard](https://huggingface.co/spaces/lmsys/chatbot-arena-leaderboard)), Llama3.1-70B-Instruct and Llama3.1-405B-Instruct (Meta, ).

Baselines:
For comparison, we consider three reference-dependent factuality evaluation techniques: FactScore (Min et al., ), Search-Augmented Factuality Evaluator (SAFE) (Wei et al., ), and Factcheck-GPT (Wang et al., ). A detailed description of these methods and their experimental setup is provided in Appendix 12.3.

**Table 1: Results of VERIFY and baselines across three tiers of FactBench and 4 widely-used LMs (*Instruct version) using the factual precision score (Equation 3), as in prior work (Min et al., ; Wei et al., ). For each evaluation method and within each tier, the best and second-best factuality scores are highlighted in blue and green, respectively. Proprietary models are more accurate, with factuality declining from Easy to Hard prompts.**
| FactBench | Model | FactScore | SAFE | Factcheck-GPT | VERIFY |
| --- | --- | --- | --- | --- | --- |
| Tier 1: Hard | GPT4-o | 57.09 | 67.42 | 87.43 | 75.65 |
| Gemini1.5-Pro | 55.51 | 64.74 | 84.08 | 73.78 |  |
| Llama3.1-70B^∗ | 57.27 | 65.82 | 82.82 | 70.07 |  |
| Llama3.1-405B^∗ | 56.81 | 64.81 | 83.77 | 68.59 |  |
| Tier 2: Moderate | GPT4-o | 59.27 | 70.25 | 90.85 | 80.72 |
| Gemini1.5-Pro | 56.59 | 67.01 | 87.99 | 78.02 |  |
| Llama3.1-70B^∗ | 58.44 | 68.74 | 86.38 | 75.76 |  |
| Llama3.1-405B^∗ | 57.92 | 67.82 | 86.60 | 75.05 |  |
| Tier 3: Easy | GPT4-o | 73.01 | 79.27 | 94.57 | 91.63 |
| Gemini1.5-Pro | 68.66 | 77.64 | 91.97 | 89.86 |  |
| Llama3.1-70B^∗ | 73.05 | 80.01 | 94.08 | 89.30 |  |
| Llama3.1-405B^∗ | 73.34 | 78.87 | 93.66 | 86.57 |  |

## 7 Experimental Results and Further Analyses

In this section, we measure LMs’ factuality on FactBench using different factuality evaluation methods (Section 7.1). These methods extract content units at different granularities, and thus, we establish a unified evaluation framework with human labeling as described in Section 7.2.

### 7.1 Factuality Improves with Proprietary LMs and on Easier Prompts

To compare model performance on FactBench, we use the factual precision metric proposed by Min et al. (). This metric quantifies an LM’s factuality by calculating the proportion of supported units among all extracted units in response, averaged across all responses (detailed in Appendix 12.4). Table 1 compares the factual precision of LMs on FactBench as measured by different evaluation methods. Despite all baselines verifying responses at their finest granularity (standalone units), we observe varying factual precision ranges across different pipelines. We observe that VERIFY consistently maintains the same ranking of models across all three tiers, unlike other evaluation methods. VERIFY ranks GPT4-o as having the highest factual precision across all tiers, followed by Gemini1.5-Pro and the two open-source Llama3.1 models. This consistency suggests that VERIFY may provide a more stable and reliable evaluation of model performance across varying difficulty levels.

Surprisingly, Llama3.1-405B-Instruct demonstrates comparable or inferior factuality to Llama3.1-70B-Instruct according to VERIFY. Further investigation (Figure 3) reveals that Llama3.1-405B-Instruct has a lower proportion of unsupported units but the highest proportion of undecidable units across all LMs. This primarily stems from stronger subjectivity in Llama3.1-405B-Instruct, with more frequent use of subjective adjectives such as “solid,” “exclusive,” and “well-known.” The reasoning logic enforced in our pipeline typically classifies these subjective elements as undecidable, thereby reducing the factual precision. Detailed analysis and examples are provided in Appendix 12.9.

Another significant observation is the consistent improvement in factuality precision across LMs from the Hard to Easy tiers, as shown by all evaluation methods.
This aligns with our tiered benchmark design, where prompts are categorized based on the responding model’s strength. Easy prompts are less likely to trigger hallucinations in strong models, as their appropriateness is measured based on weaker LM hallucinations.

Figure: Figure 3: Average percentage of unsupported (UnS) and undecidable (UnD) labels in different LMs (*Instruct version) evaluated by VERIFY. Llama3.1-405B-Instruct has the highest proportion of undecidable units across all LMs.
Refer to caption: x3.png

### 7.2 Factuality Evaluation by VERIFY Strongly Correlates with Human

The factuality of a model, measure by a factuality evaluation method, depends on the granularity of the extracted units and the method’s verification capabilities.
FactScore extracts units with a finer granularity compared to VERIFY due to its focus on biographical text evaluation, which typically consists of objective and easily separable units. On the other hand, Factcheck-GPT’s claim-level decomposition (finest-level) often results in sentence-level containing multiple claims.
To establish a unified evaluation framework for these methods, we selected 40 FactBench prompts from 40 randomly sampled topics (one prompt per topic) and generated responses from each model. We then applied our unit extraction (Section 4.2) and decontextualization approach (Section 4.3) to decompose generated LM responses into self-contained and verifiable units. This method was chosen for its ability to handle user-model conversations (with careful instructions and in-the-wild demonstrations), extract moderately granular units, and filter them based on verifiability.

Three fluent English speakers are hired to annotate a total of 160 LM responses for 4 models on the same set of 40 prompts. VERIFY breaks LM responses into 4,467 units, with each response annotated by two annotators.
Each annotator evaluated the independence and factuality of each unit. A unit is considered Independent if it is verifiable and self-contained. A Dependent unit, on the other hand, is either an unverifiable piece of information, e.g., “I can provide you with some examples.” or under-specified content. For example, “She won the best actress award” is underspecified as it contains a pronoun, and it is unclear what specific award is referenced.
Overall, 82.6% units are considered Independent by both annotators with an inter-annotator agreement of 0.53 in terms of Cohen’s Kappa score.
Additionally, each annotator evaluates the factuality of a unit by using two labels: (1) Factual when supporting evidence can be found using Web search and (2) Other when refuting evidence is found, or the factuality cannot be determined.
Annotators agree on the factuality of 85.9% of units, with a Cohen’s Kappa agreement of 0.57. These binary factuality labels are compatible with VERIFY and other baselines.
A unit is Independent if both annotators agree; otherwise, it is labeled Dependent. Factuality is decided in the same manner.

#### 7.2.1 Accuracy Overlooks Decision Quality

Figure: Figure 4: Average accuracy of factuality evaluation methods compared to human annotations across LMs.
Refer to caption: x4.png

We fed only Independent units, as agreed by both annotators, to the factuality evaluation methods for verification. Figure 4 compares the accuracy of different methods against human labels. While VERIFY demonstrates on-par or superior performance on LLaMA3.1 models compared to other baselines (detailed accuracy results in Appendix Figure 7), it is generally outperformed by Factcheck-GPT.
Further analysis reveals that Factcheck-GPT achieves the highest average precision in predicting factual units (84.6% compared to VERIFY’s 75.1%) across models. This performance gap is attributed to the distinct design choices of the two methods. Factcheck-GPT leverages its internal knowledge when no external evidence is found, whereas VERIFY heavily relies on external knowledge to make decisions and labels units without evidence as undecidable.
Although VERIFY’s approach may result in lower factuality, it maintains high confidence and trustworthiness in its evaluations, particularly in cases where conservative, evidence-based decisions are preferable.
Therefore, to better capture the human-method decision alignment, we calculate correlation following previous work (Wei et al., ; Min et al., ). As shown in Figure 5, VERIFY consistently outperforms other methods when being compared with human labels. Notably, VERIFY achieves a significantly higher correlation with human annotation in the Other category. This reflects our method’s nuanced handling of undecidable cases and demonstrates how VERIFY’s judgments mirror human reasoning in scenarios where information cannot be definitively assessed.

Figure: Figure 5: Response-level correlation between factuality evaluation methods and human annotations of 40 prompts across 4 LMs (averaged via z-score). F refers to Factual labels, and O refers to Other. VERIFY achieves the highest correlation with human judgments compared to baselines.
Refer to caption: x5.png

### 7.3 Gemini1.5-Pro’s Refusal Rate Impacts Its Factuality

Current factual precision metrics (detailed in 12.4) do not evaluate LM factuality when models refrain from answering. In this section, we investigate LMs’ refusal rate and its significance. Previous work (Min et al., ) relied on simple heuristics or keywords to identify refusal responses, but our initial investigation finds these methods unreliable.
To address this, we prompt GPT-4 Turbo to detect refusals and categorize them into different types, including clarification requests, lack of knowledge, safety concerns, and misinformation risks.
Figure 6 shows the refusal rates for each model across different FactBench tiers (see Appendix 12.6 for the task prompt and distribution of refusal categories for different LMs). Notably, Gemini1.5-Pro exhibits a significantly higher refusal rate than other models, approaching 10% on the Hard portion of FactBench. As shown, it refuses to answer prompts in the Hard tier more frequently, as it contains the most challenging hallucination prompts.
Moreover, while Gemini1.5-Pro’s refusal strategies can help prevent hallucinations (see example LABEL:tab:model_responses), the high rate of refusals impacts its overall factuality precision. Additionally, our manual inspection reveals that in 25% refusal cases, Gemini1.5-Pro’s reason for not answering is not valid. For example, the model interpreted “give me studies on the recommended interval between COVID vaccines” as a request for medical advice and refused to answer. These findings highlight the need for more nuanced factuality evaluation metrics to account for refusals as an important direction for future research.

Figure: Figure 6: Refusal rate of different LMs across Hard, Moderate, and Easy tiers of FactBench. Gemini1.5-Pro shows a significantly higher refusal rate than other LMs.
Refer to caption: x6.png

## 8 Conclusion

We presented VERIFY, a factuality evaluation pipeline that automatically annotates the factuality of LM responses in real-world settings. VERIFY breaks down a response into content units, identifies verifiable ones, evaluates them against retrieved Web evidence, and labels each unit as supported, unsupported, or undecidable. Our method correlates strongly with human evaluations compared to existing state-of-the-art methods and is more robust in handling flawed units. Using VERIFY, we curated FactBench, a benchmark of 1,000 prompts across 150 fine-grained topics that are grouped into three tiers: Hard, Moderate, and Easy. Prompts in each tier are sorted by appropriateness, i.e. their tendency to elicit hallucinated content from LMs. FactBench is the first real-world and diverse factuality evaluation benchmark designed to be regularly updatable to capture evolving factuality challenges faced by language models. Finally, we used our dataset to evaluate frontier LMs in proprietary and open-source categories. Our findings show that the proprietary model exhibits better factually, with the performance declining factuality from Easy to Hard tiers. Interestingly, Llama3.1-405B-Instruct shows lower factuality mainly due to its frequent use of subjective terms that are hard to verify.

Current factuality-evaluating metrics oversimplify the evaluation and do not consider model refusals and low factual recall, which is an important direction for future research in this field. Additionally, future work can explore more sophisticated evaluation pipelines that account for individual factual support as well as logical inter-unit connections. Such approaches would verify not only the factual precision of each content unit but also the logical coherence within the overall response, further improving the reliability of LM evaluation methods.

## 9 Acknowledgement

We thank the Serper ([https://serper.dev/](https://serper.dev/)) team for providing access to the Google Search API, which significantly facilitated the curation of FactBench and accelerated the evaluation of hallucination prompts.

## 10 Ethics Statement

This research adheres to strict ethical standards, especially considering the potential impact of releasing datasets and methodologies designed for evaluating the factual precision of language models. All crowdsourced data used in this work was collected in accordance with relevant ethical guidelines, ensuring that participant privacy is respected and no personally identifiable information is included in the dataset. Specifically, the user-LM interaction dataset, LMSYS-Chat-1M, on which this research is based, has had all personally identifiable information removed, and we further filtered conversations potentially related to personal information during data cleaning. The potential risks associated with misusing our findings have been carefully considered. We aim to mitigate these risks by clearly delineating the scope and limitations of our models, ensuring that our dataset and methodology are used responsibly. Moreover, we acknowledge the possibility of inherent biases in crowdsourced data and have employed techniques to identify and minimize these biases in our models and evaluations. There are no conflicts of interest or external sponsorships that have influenced the outcomes of this research. All findings, including any potential model limitations, have been reported transparently to promote fairness, accuracy, and integrity in factuality evaluation tasks.

## 11 Reproducibility Statement

We have made significant efforts to ensure the reproducibility of the results presented in this paper. All experimental details, including the dataset collection and processing steps, are provided in Section 3 and Section 5 and further elaborated in Appendix 12.1. The proposed factuality evaluation framework, VERIFY, and its implementation are thoroughly described in Section 4, with additional details, including the prompts used and evaluation pipeline, in Appendices 12.10.4 through 12.10.7.
To facilitate the reproducibility of our results, we will release the FactBench and provide an anonymous
[link](https://anonymous.4open.science/r/FactBench-381B/README.md) to the source code in the supplementary materials, enabling other researchers to replicate our benchmarks and use the evaluation metrics described in Section 7.
Furthermore, we include all theoretical proofs and detailed statistical methods in the appendices, ensuring that all claims can be validated through the provided materials.

## 12 Appendix

### 12.1 Data Cleaning

We begin by collecting prompts from the first turn of conversations in the LMSYS-Chat-1M dataset, which is a large-scale, in-the-wild LM conversations dataset. Since the existing language labels are unreliable, we employ the Llama3-70B-Instruct model (AI@Meta, ) to identify the language of each conversation using the prompt in Appendix 12.10.1. This gives us 516,771 distinct English prompts with at least 32 characters. Next, we remove anonymized (30.9%) and duplicated (12.1%) prompts. Meanwhile, we observed that some users queried LMs with thousands of identical prompts. To mitigate this issue’s impact on subsequent clusters, we filter out prompts with a Jaccard similarity score greater than $0.9$. Our cleaned data contains 294,333 distinct prompts.

### 12.2 Manual Check on Prompt Verifiability

In order to ensure the verifiability specified in Section 3, three authors have conducted multiple rounds of human inspection and validation to exclude all non-verifiable prompts like problem-solving (e.g., “A suit manufacturer has 14 suits for men and 4 suits for women. How many suits are available overall?”) and faithfulness-related (e.g., “Translate the given text”) tasks. More unverifiable examples are available in our prompt at Appendix 12.10.2.

### 12.3 Baselines Description

We use gpt-3.5-turbo-0613 (Brown et al., ) as a backbone LM when running all baselines.

- •
FactScore (Min et al., ):
FactScore evaluates the factual precision of LMs by breaking text into atomic facts and assessing the percentage of facts supported by Wikipedia articles. The original FactScore method is provided with Wikipedia pages with relevant information. However, the extracted units from in-the-wild requests are not associated with a Wikipedia page and might not even be found in Wikipedia articles. To make a fair comparison, we use the Wikipedia API (Goldsmith, ) to map these atomic units to the 5 closest Wikipedia topics in the Wiki database for retrieval.
- •
Search-Augmented Factuality Evaluator (SAFE) (Wei et al., ):
SAFE evaluates long-form factuality by decomposing text into atomic facts, adopting the same FactScore fact extraction component, and checking each fact’s relevancy to the original query. For relevant facts, SAFE queries the Google search engine for evidence retrieval and labels each fact as either supported or refuted accordingly.
- •
Factcheck-GPT (Wang et al., ):
Factcheck-GPT is a hallucination detection and mitigation framework. In the annotation phase, it assesses the factuality of LM-generated content using a multi-step annotation pipeline that includes the decomposition of claims, decontextualization, evidence retrieval through Google Search, evidence snippets generation, final factuality decision, and revision of non-factual elements. For this study, the final revision step is excluded from the baseline methodology.

### 12.4 Factual Precision Metric

We adopt the factual precision utilized by FactScore (Min et al., ) to compare the performance of different models on FactBench.
Given the set of prompts $P$ and knowledge source $K$, we first obtain model $M$ responses $\{R_{M}=M(p)\text{ for }p\in P$}. All baselines decompose each response into atomic units (facts). Therefore, we denote $U$ to be the set of units in $R_{M}$. We calculate the factual precision of $R_{M}$ as:

$$ $\displaystyle f(R_{M})=\frac{1}{|U|}\sum_{u\in U}\mathds{I}[u\text{ is supported by }K]$ (2) $$

The overall factuality precision of each model on $P$ prompts is calculated as:

$$ $\displaystyle F(M)=\mathds{E}_{p\in P}[f(M_{p}|M_{p}\text{ responds}]$ (3) $$

We do not evaluate factual recall, such as cases where models abstain from answering or generate responses with minimal factual content. Wei et al. () suggests that there is a fixed number of content units users care about and that this number can be tuned. However, we did not find this metric compelling, as different models show different verbosity levels, making it difficult to establish a unified threshold. Moreover, the number of units is not a reliable quality indicator, as the content may still lack relevance or usefulness.

### 12.5 Accuracy of Factuality Evaluation Methods across LMs

Figure 7 (left) evaluates the ability of different methods to verify Independent units per LM. However, factuality evaluation methods often generate both dependent and independent units, requiring mechanisms to handle ambiguous units that are rather difficult to verify using existing online knowledge.
Figure 7 (right) evaluates the ability of different methods to verify all kinds of units (including both Independent and Dependent units). By comparing the right figure with the left one, we can evaluate the robustness of factuality evaluation methods when handling ambiguous and contextually dependent units. As shown in the right figure, the gap between VERIFY and the best-performing baseline, Factcheck-GPT, narrows to an average accuracy difference of 0.6%. This suggests VERIFY ​’s improved accuracy relative to other baselines when dealing with dependent units.

Figure: Figure 7: Accuracy of factuality evaluation labeling compared to human annotations across LMs. Independent Units are independent units identified by human annotators, and All Units include both dependent and independent units. VERIFY is robust to dependent units, as demonstrated by the noticeable accuracy improvement in the right figure.
Refer to caption: x7.png

### 12.6 Refusal Prompt and Refusal Type Distributions

The refusal categories explain various reasons for declining to answer queries. “No Refusal” indicates a complete response, while categories like “Safety Concerns” and “Misinformation Risks” reflect avoidance of harmful or misleading information. Refusals may also stem from requests for “Sensitive or Private Information,” where personal data is involved, or a ”Clarification Request,” where the model seeks further details. Other reasons include “Ethical and Legal Advice,” “Hate Speech or Discrimination,” and “Lack of Knowledge/Capability,” which acknowledge the model’s limitations. The “Other” category covers refusals that don’t fit these reasons.

Figure: Figure 8: Distribution of LM’s answer refusal categories on FactBench
Refer to caption: x8.png

Figure 8 shows the distribution of refusal categories across LMs, with Gemini1.5-Pro having the highest number of refusals across a broad range of categories. We also show an example in Table LABEL:tab:model_responses where Gemini1.5-Pro refrained from answering by requesting clarifications while GPT4-o produced a fully hallucinated response.

**Table 2: Shows the varying behavior of LMs when confronted with knowledge limitations. GPT4-o, Llama3.1-70B, and Llama3.1-405B produce hallucinated or inaccurate content (marked in red), while Gemini1.5Pro either refrains from generating a response or asks for clarifications to better understand the query: “Recommend some restaurants in Fuzhou City”.**
| Model | Model Response |
| --- | --- |
| GPT-4-o | Sure! Fuzhou, the capital of China’s Fujian province, is known for its rich culinary culture. Here are some highly recommended restaurants where you can try a variety of delicious dishes:<br>1. Ding Feng Zhuang:<br>- Cuisine: Traditional Fujian …<br>2. Huangxiang Shousibao:<br>- Cuisine: Seafood …<br>3. Yu Xiangyan:<br>- Cuisine: Fujian and Cantonese…<br>4. Juchunyuan :<br>- Cuisine: Fujian…<br>5. Shawanrong:<br>- Cuisine: Modern Fusion… |
| Gemini | I can definitely recommend some restaurants in Fuzhou City! To give you the best suggestions, please tell me:<br>- What kind of cuisine are you interested in? (e.g., Fuzhou specialties, Cantonese, Sichuan, Western, etc.)<br>- What’s your budget like? (e.g., fine dining, mid-range, budget-friendly)<br>- What’s your dining style? (e.g., street food, casual, formal, romantic)<br>- What area of Fuzhou are you interested in? (Specific neighborhoods or landmarks will help!)<br>Once I have this information, I can provide you with a more tailored and helpful list of restaurant recommendations. |
| Llama 3.1 (70B) | Fuzhou, the capital of Fujian Province in China, is known for its rich culinary heritage and delicious local dishes. Here are some recommended restaurants in Fuzhou City:<br>1. Buddha Jumps Over the Wall: This restaurant is famous for its signature dish, ”Buddha Jumps Over the Wall,” a traditional Fujianese soup …<br>2. Fuzhou Fish Ball Restaurant: This casual eatery is a local favorite for its delicious fish balls, which are made from fresh fish and served in a light broth.<br>3. Gongdelin Vegetarian Restaurant: This vegetarian restaurant offers a wide range of plant-based dishes, including traditional Fujianese options like stir-fried vegetables and tofu… |
| Llama 3.1 (405B) | Fuzhou, the capital of Fujian Province in China, is renowned for its rich culinary heritage… Here are some recommended restaurants in Fuzhou that showcase the best of Fujian cuisine and more:<br>1. Dongbai Lou - Known for its traditional Fujian dishes, including the famous Dongbai chicken and a variety of seafood options. …<br>2. Wu Ming Lou - Offers a mix of Fujian and Cantonese dishes …<br>3. Fuzhou Wanshou Road Food Street - While not a single restaurant, this food street is a must-visit for any food enthusiast…<br>4. Min Jiang Lou - Specializes in Fujian cuisine with a focus on river and sea delicacies…<br>5. Huangcheng Hotel Restaurant - Offers a range of Fujian dishes in a comfortable setting… |

Below is the prompt used for refusal detection and categorization:

### 12.7 Factuality Evaluation Methods Statistics

**Table 3: Detailed data for Figure 3. UnS stands for unsupported, and UnD stands for undecidable. Llama3.1-405B-Instruct contains a lower number of unsupported units compared to Llama3.1-70B-Instruct, while it has the highest number of undecidable units across all LMs (*Instruct version)**
| Model | Tier 1: Hard | Tier 2: Moderate | Tier 3: Easy |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
|  | # UnS | # UnD | # Units | # UnS | # UnD | # Units | # UnS | # UnD | # Units |
| GPT4-o | 5.18 | 23.98 | 22.91 | 3.87 | 21.22 | 22.83 | 1.65 | 7.88 | 27.73 |
| Gemini1.5-Pro | 5.72 | 25.08 | 21.59 | 4.83 | 22.03 | 20.94 | 3.98 | 8.89 | 24.09 |
| Llama3.1-70B^∗ | 7.84 | 25.2 | 25.75 | 6.87 | 22.20 | 23.17 | 4.15 | 8.56 | 28.77 |
| Llama3.1-405B^∗ | 7.04 | 28.43 | 25.23 | 6.73 | 23.45 | 23.44 | 2.91 | 10.65 | 28.05 |

### 12.8 Benchmark Topic Distribution

Figure: Figure 9: Percentage of verifiable prompts in 50 most dense clusters
Refer to caption: x9.png

**Table 4: Prompt statistics of LMs in each Tier (Hard, Moderate, Easy).**
|  | Models | # Prompts | # Selected Prompts | Total Prompts | Total Selected Prompts |
| --- | --- | --- | --- | --- | --- |
| Hard | gpt-4 | 3431 | 500 | 15499 | 2205 |
| claude-2 | 1074 | 181 |  |  |  |
| gpt-3.5-turbo | 3607 | 524 |  |  |  |
| claude-1 | 7387 | 1000 |  |  |  |
| Moderate | claude-instant-1 | 2422 | 171 | 30613 | 1435 |
| vicuna-33b | 10548 | 434 |  |  |  |
| llama-2-13b-chat | 12160 | 628 |  |  |  |
| wizardlm-13b | 5483 | 202 |  |  |  |
| Easy | mpt-30b-chat | 3150 | 11 | 195641 | 542 |
| vicuna-13b | 183117 | 500 |  |  |  |
| palm-2 | 2463 | 8 |  |  |  |
| guanaco-33b | 5282 | 20 |  |  |  |
| llama-2-7b-chat | 1629 | 3 |  |  |  |

### 12.9 Case Study on LLaMA3.1-405B-Instruct and LLaMA3.1-70B-Instruct performance gap

As mentioned in the main text, the performance of Llama3.1-405B-Instruct on VERIFY is close to or even lower than that of Llama3.1-70B-Instruct. We conduct an in-depth investigation into the possible reasons. Based on results in Figure 3, we find that Llama3.1-405B-Instruct has the highest proportion of undecidable labels among all the models, directly resulting in its lower factual precision. After manually reviewing 30 cases with the largest factual precision gap between the 405B and 70B models, we observed a common phenomenon: the 405B model tends to use more subjective modifiers such as “solid”, “exclusive”, and “well-known”, while the 70B model usually prefer plain descriptions. For example, consider the following case when both 405B and 70B models respond with the same Debian guidebook:

Llama3.1-405B-Instruct

Atomic Unit: “Debian GNU/Linux: Guide to Installation and Usage” by John Goerzen and Ossama Othman is a solid foundation for understanding Debian’s basics.

Reasoning: One KNOWLEDGE point states that the book is a “nice book and CD combo” that can get someone started with Debian, but notes that the book lacks in-depth information, While the book is mentioned as a “nice” resource for getting started with Debian, it is also criticized for lacking in-depth information. Therefore, the evidence is insufficient to confirm or deny that the book is a solid foundation.

Label: undecidable

Llama3.1-70B-Instruct:

Atomic Unit: “Debian GNU/Linux: Guide to Installation and Usage” by John Goerzen and Ossama Othman is a guidebook of Debian basics.

Reasoning: The KNOWLEDGE points strongly imply that the book is a guide that covers basics of Debian.

Label: suppported

Based on the case above, we find that due to introducing a detailed CoT reasoning mechanism and the undecidable label, VERIFY is very rigorous in verifying atomic units, especially when subjective adjectives are involved. We use a pre-trained subjectivity classification model GroNLP/mdebertav3-subjectivity-multilingual (Ruggeri et al., ) to classify the output units of both models. The classification results show that 9.4% of the 405B model’s response units are classified as subjective, while only 7.5% of the 70B model’s response units are subjective. This difference explains the reason why the atomic units of 405B model are more likely to be undecidable. Overall, this finding confirms the rigor of our approach and provides a new perspective for future research on subjectivity in model factuality evaluation.

### 12.10 Prompts

In this section, we show the prompts we used throughout the experiments.

#### 12.10.1 Language Detection

#### 12.10.2 Factual Prompt Labeling

#### 12.10.3 Prompt Usefulness Scoring

#### 12.10.4 Unit Extraction Prompt

#### 12.10.5 Decontextualization Prompt

#### 12.10.6 Query Generator Prompt

#### 12.10.7 Final Accuracy Decision Prompt

## Peer Reviews

confidence: 4
contribution: 2
initial_rating: 5
presentation: 3
questions: N/A
soundness: 3
strengths: 1. FACTBENCH is a novel dynamic benchmark that captures evolving factuality challenges in real-world LM interactions, addressing the limitations of existing static benchmarks.

2. VERIFY, the factuality evaluation pipeline, considers the verifiability of generated content and introduces an "undecidable" label for ambiguous cases, providing a more robust framework for assessing factuality.

3. The authors release human-annotated factuality data on 5,519 content units, which can serve as a valuable resource for future research on factuality evaluation.
summary: This paper presents FACTBENCH, a dynamic benchmark dataset for evaluating the factuality of language model (LM) responses in real-world user interactions. The authors introduce a two-step process to curate the benchmark: (1) collecting verifiable and useful prompts from an in-the-wild LM conversation dataset, and (2) using VERIFY, a factuality evaluation pipeline, to measure the appropriateness of these prompts based on whether they elicit unfactual responses from strong LMs. The resulting FACTBENCH contains 985 hallucination prompts across 213 topics. The authors also benchmark several widely-used LMs on FACTBENCH and find that proprietary models exhibit better factuality than open-weight models, and VERIFY achieves the highest correlation with human judgments compared to other factuality evaluation methods.
weaknesses: 1. The usefulness evaluation criteria and scoring process are not well-justified. More details are needed on how these criteria were determined and how the scores from two LMs were combined.

2. The VERIFY pipeline uses a single LM (Llama3-70B-Instruct) for key tasks such as unit extraction, labeling, and decontextualization. The potential bias introduced by relying on a single model is not adequately addressed.

3. The evaluation of FACTBENCH is limited to a small set of LMs. A more comprehensive evaluation involving a wider range of models would strengthen the paper's claims.

4. The weighting factor α in the Hallucination Score is set to 0.5 without much explanation. A sensitivity analysis or ablation study on this hyperparameter would be informative.

---

confidence: 3
contribution: 3
initial_rating: 5
presentation: 3
questions: 1.	What factors influenced the decision to categorize prompts into three tiers?
2.	Why was a topic model-based approach (BERTopic) chosen over a general clustering method? Given that BERTopic parameters can affect clustering quality, how were these parameters tuned, and might they influence the final benchmark results? 
3.  See the weakness.
soundness: 2
strengths: 1.	This study addresses a growing concern in the factuality of LM-generated content, especially in the context of hallucinations. 
2.	FACTBENCH is an innovative, dynamic benchmark that adapts to new factuality challenges. And VERIFY is an innovative factuality evaluation approach that achieves more precise, human-aligned assessments. 
3.	Extensive experiments across multiple models show that VERIFY aligns closely with human judgments.
summary: The authors propose FACTBENCH, a dynamic benchmark for evaluating language model factuality in real-world scenarios by using prompts that often provoke hallucinations. They also present VERIFY, a pipeline that assesses factuality by categorizing responses as supported, unsupported, or undecidable, based on retrieved web evidence. The experiments show that VERIFY can achieve the highest correlation with human judgments.
weaknesses: 1.	Only three speakers were hired for annotation, with relatively low inter-annotator agreement (Cohen’s Kappa scores of 0.52 and 0.55). The difference in factuality labeling between VERIFY and Factcheck-GPT is minor in Table 2 (≤ 0.01 for Factual labels), and VERIFY's performance is noticeably lower in Tables 1 and 3, suggesting potential limitations in annotation reliability and robustness.
2.	While FACTBENCH is described as dynamic, the paper lacks specifics on its update process, such as the conditions under which prompts are added or removed, the frequency of updates, and criteria for integrating new prompts.
3.	The paper provides limited discussion on key hyperparameters, such as α used in the hallucination score and the number of evidence retrieval rounds in VERIFY.

---

confidence: 3
contribution: 3
initial_rating: 6
presentation: 3
questions: ● Could you explain in detail how Llama3-70B was used to determine whether the data was verifiable?
● Why did you choose 0.5 as the weighting factor for the hallucination score? Might this choice impact the correlation of VERIFY with human preferences?

Comments:

Overall, this paper makes meaningful contributions to the development of factuality evaluation methods for LMs, particularly in establishing a dynamic benchmark that could adapt to future model advancements. However, improvements in experimental design, verification transparency, and difficulty categorization would enhance the robustness and generalizability of the findings.
soundness: 3
strengths: ● The paper proposes a benchmark across multiple topics with varying difficulty levels.
● It offers an interesting categorization of hallucinations, distinguishing between context-independent and context-dependent statements, which could facilitate finer-grained hallucination detection in future studies.
● A new weighting factor is introduced to account for unsupported and undecidable units, adding robustness to hallucination scoring.
● The VERIFY framework shows good alignment with human judgment, particularly in handling nuanced cases.
summary: This paper introduces FactBench, a dynamic benchmark designed to evaluate the factual accuracy of large language models (LMs) in real-world contexts. FactBench continuously updates its dataset of prompts to capture scenarios where LMs are likely to generate hallucinations, addressing the limitations of existing static benchmarks. Additionally, the VERIFY framework is presented as a factuality assessment pipeline that categorizes LM responses based on evidence-supported verification categories, demonstrating higher alignment with human evaluations.
weaknesses: ● The experimental design could be improved; using VERIFY to classify data and then evaluate it may introduce circularity in the results.
● The VERIFY method lacks innovation in hallucination detection, particularly in terms of recall, which is essential in high-stakes fields like medical and encyclopedic contexts where hallucinations must be minimized.
● Difficulty ratings for prompts based solely on scores from multiple large models are unconvincing; a broader and more comprehensive classification method is needed.

---

confidence: 4
contribution: 3
initial_rating: 5
presentation: 2
questions: (1) There is a typo on line 305: it references Table 2 when it should be Figure 2.

(2) The high refusal rate observed in Gemini1.5-Pro, especially in the Hard tier, is mentioned briefly in the paper. Could the authors provide more details for these refusals and any insights into how different refusal categories might impact the overall factuality evaluation? Also, in cases where refusals were misclassified or unjustified, did the authors investigate how these instances were distributed across prompts or tiers?

(3) Did you verify whether calculating the score using the Hallucination Score as defined in Eq. 1 is indeed meaningful?
soundness: 2
strengths: The task of factuality evaluation is extremely important in large language models (LLMs), as ensuring the factual accuracy of model outputs is crucial for their reliability in real-world applications. The proposed approach of categorizing LM-generated content into "supported," "unsupported," or "undecidable" represents a novel and rigorous method, distinguishing this work from previous studies that often lack this level of granularity. This categorization allows for a more nuanced understanding of model limitations and strengths in factual reasoning. Furthermore, the authors carefully selected datasets and established comprehensive baselines, ensuring that their comparisons across methods are both fair and robust.
summary: This paper introduces VERIFY, an automatic factuality evaluation pipeline that assesses language model (LM) responses by verifying and categorizing content units based on web evidence. Using VERIFY, the authors built FACTBENCH, a benchmark of 985 prompts across various topics and difficulty levels to evaluate LMs' accuracy in real-world settings. Results show that VERIFY aligns closely with human judgment, establishing it as a reliable method for assessing factuality in LM outputs.
weaknesses: (1) Building FACTBENCH requires multiple steps, yet each step relies on relatively simple, heuristic approaches, which may limit the novelty and robustness of the benchmark in capturing nuanced factuality challenges.

(2) The lack of qualitative analysis for ambiguous samples or Tier 1 (Hard) prompts leaves gaps in interpreting how VERIFY handles especially difficult cases. Providing a more detailed examination of these prompts and their handling would enrich the analysis.

(3) While VERIFY’s methodology is strong in categorizing factuality, it is relatively limited in interpretability when applied to complex or context-dependent responses. Expanding on how VERIFY’s outputs could be used to provide actionable feedback for model improvement or on how it handles responses with interdependent factual claims would increase its utility and depth.

Meta-review: This paper introduces FACTBENCH, a dynamic benchmark aimed at evaluating the factuality of language model responses by identifying and categorizing hallucinations. The research addresses an important challenge in LLM development by proposing VERIFY, a pipeline that assesses response factuality across 985 prompts spanning 213 topics. Key strengths include its interesting approach to capturing real-world interaction challenges, the introduction of an "undecidable" label for ambiguous cases, and the potential to provide a valuable resource for future factuality research. The authors demonstrate that proprietary models generally perform better in factuality tests, and their methodology shows promising alignment with human judgments across multiple evaluation dimensions.

However, the paper suffers from significant limitations that finally lead to its rejection. Reviewers consistently pointed out critical weaknesses, including a poorly justified usefulness evaluation process, over-reliance on a single model (Llama3-70B-Instruct) for key tasks, which introduces potential bias (xqv7, mXTb). The experimental design shows limited comprehensiveness, with a small set of evaluated LMs and unconvincing difficulty ratings (BPdM, 42L8). Methodological concerns are further compounded by low inter-annotator agreement, with Cohen's Kappa scores of only 0.52 and 0.55, raising questions about annotation reliability (mXTb). Additionally, the paper lacks transparency in critical areas such as the update process for the benchmark, justification for key hyperparameters (particularly the weighting factor \alpha), and a comprehensive analysis of the methodology's limitations (xqv7, mXTb, BPdM). These shortcomings, coupled with the incremental nature of the contribution, led reviewers to rate the paper marginally below the acceptance threshold.

Overall, this is a good paper. But it should be significantly polished before accepted.

Decision: Reject