Truth or Mirage? Towards End-To-End Factuality Evaluation with LLM-Oasis
Authors: Alessandro Scirè, Andrei Stefan Bejgu, Simone Tedeschi, Karim Ghonim, Federico Martelli, Roberto Navigli


Reviewer paper summaries: The author introduced LLM-OASIS, a large-scale, general-purpose dataset for factuality evaluation, and provided a comprehensive benchmark for testing state-of-the-art LLMs. The authors used an LLM to generate 81k pairs of factual (paraphrased) and unfactual texts from Wikipedia paragraphs. The unfactual texts were generated in three main steps: (i) extracting a list of atomic claims from a Wikipedia page, (ii) falsifying one of the extracted claims, and (iii) producing an unfactual text that includes the falsified statement. The generated dataset was evaluated via human annotation, resulting in a gold standard subset of LLM-OASIS. The authors demonstrated that this benchmark poses significant challenges to state-of-the-art LLMs, with smaller models trained on LLM-OASIS achieving competitive performance.
Summary of strengths: 1. Unlike previous task-specific or domain-specific datasets, this resource is designed to be more general and larger in scale, potentially enabling broader applicability
2. The LLM-generated benchmark is evaluated by human experts, producing a "gold-standard" test set that should be reliable for benchmarking
3. The finding this benchmark presents significant challenges to SOTA LLM models is intriguing
Summary of weaknesses: 1. The benchmark is generated from an LLM, which can inherent potential biases or predictable patterns in the model and or induce the particular prompts used for the generation. For instance, the type of false information produced by an LLM may be just a subset of or deviate significantly from the kind of natural false information in the downstream application. There is no evidence in the paper to preclude the potential issues such as (i) the generated falsehood is limited in diversity or (ii) the generated falsehood follows predictable patterns that are unlike real misinformation; (iii) there may be prompt-induced patterns in the dataset

2. Concerns in 1 are very likely to be why a small model trained on LLM-OASIS can rival the performance of SOTA LLMs. The result of this experiment raises concerns that the falsification process may contain predictable patterns that a smaller model if trained on LLM-OASIS itself, may capture and utilize to outperform SOTA LLM on the same dataset. 

3. The evaluation methodology used by human experts verifies that the falsified text is actually false but does not assess whether the kind of falsification it provides does not contain biases or predictable patterns. 

4. The misinformation in the "gold standard test set" is also LLM-generated and may contain the same biases and predictable patterns.
Comments, suggestions and typos: No experiments have been conducted on how the evaluation system the authors trained on LLM-OASIS provides strong performance on other benchmarks. If the authors can show that their model, trained on LLM-OASIS, can perform similarly to SOTA LLMs on some external benchmarks, it could relieve some of the concerns raised in the previous section.
Overall assessment scores: 3