FactPrompt: LLM Factuality Prompting for Open-Ended Text Generation
Authors: Jingyi Zhang, Gerard de Melo


Reviewer paper summaries: This paper studies the factuality of open-ended text generation. More specifically, it investigates the impact of adding a phrase designed to enhance factuality (e.g., appending the original prompt with “only tell me things that are correct”) referred to as FactPrompt. The hypothesis is that LLMs can understand its uncertainty, thus, when prompted to be more cautious about its generation, it could refrain from generating uncertain statements.

The paper conducts experiments using biography generation (where the entities were sourced from Wikipedia, so the ground-truth information is available for assessment), using 12 different fact prompts (categorised into 3 groups) on 4 models (mistral, llama3, GPT-3.5, GPT-4). The results show that fact prompts can increase FactScore (an automatic measure for faithfulness score) while decreasing the number of generated tokens.
Summary of strengths: - The method is simple, yet improving factuality scores.
- The paper provides empirical findings of FactPrompt on FactScore
Summary of weaknesses: Since this paper is largely empirical, where it studies the impact of prompting on factuality, one would expect to show more ablations to support the findings (i.e., to demonstrate the generality of the findings). The current limitations are:

- One scenario investigated (only biography generation): Since the findings and conclusion are based on experimental results, only studying “biography generation” means that the findings may not be as general as “open-ended generation”. 

- The number of test examples is small (i.e., 183 in total and even smaller for each split in Table 4). The evaluation in this paper is entirely automatic, so I would see no issue in scaling up the experiments.

- One variant of FactPrompt for each style: I believe that in practice there could be multiple valid fact prompts; however, this study doesn’t offer any insight into the sensitivity of the prompt and whether the improvement/degradation is statistically significant.

- Since FactScore correlates with the length (e.g., shown in Fig2), generating shorter (i.e., more concise) responses could also increase FactScore. This study does not compare and contrast the impact on FactScore of other prompts (those that are not strictly about factuality but could reduce the length).

- This study only has automatic evaluation: As the work is mainly empirical, I would expect to see some human evaluation (at least on a small scale) to verify that the generated responses are factual and won’t be biased by FactScore.
Comments, suggestions and typos: no additional comments and no obvious typo.
Overall assessment scores: 2.5