Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: This papers proposes FACTUALITYPROMPTS, a new benchmark for open-ended text generation factuality evaluation. The dataset used for the benchmark is selected from FEVER. Based on the document-level and sentence-level factuality, the authors adapt two evaluation metrics for factuality evaluation focusing on named entities hallucination and textual entailment. Using the proposed benchmarks, the authors test language models with different sizes and also compare greedy decoding and top-p sampling. Informed by their observation that greedy decoding leads to more factually consistent generation and that tokens appearing later in the sentences are more important, the authors proposed a modified top-p decoding with decaying p within sentences (factual-nucleus sampling). Lastly, the authors presents a modified training method to enhance factuality, which mask off training signals from first few tokens in the sentence (tokens to the left of a “sentence pivot” token).
strengths_and_weaknesses: Strengths:

1. The authors conduct extensive experiments to quantify the level of factuality in text generation, and how their proposed inference and training method improve factuality. 

2. The paper is innovative as most previous works target factuality for specific downstream tasks (such as text summarization) but this work focus on general purpose open-domain text generation

3. The paper is well-written.


Weaknesses:

1. From Section 6, I am not sure if the proposed training method "topic-prefix" and "sentence completion loss" individually could bring significant improvement in smaller-size language models.

2. The proposed factuality-based training requires “topic prefix” which may not be available in other domains like "arXiv papers, medical reports". It would be helpful if the authors could address that limitation / discuss how to obtain such topic prefix from domains outside of wikipedia. 

3. It is not clear how the "range of language models" with different sizes are trained and tested, e.g. the data (and domains) used to pre-train these language models. Can we decouple the factuality from the pre-training data from factuality that the model learns from the authors' fine-tuning data?


questions: Please see weaknesses 1 - 3.
limitations: The authors should address if the proposed benchmark and the trained models may have any unfair biases or other potential negative impact.
review_title: null

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: This paper addresses the tendency of open generation systems to produce factually incorrect sequences. In order to quantify and measure factual correctness of text generative models, a dataset comprising factual and wrong prompts is created. This dataset is created from a fact checking dataset FEVER that has true and fake documents with their corresponding Wikipedia sources. The proposed dataset uses FEVER to obtain the linked Wikipedia article for each prompt and defines metrics mainly focused on named entities and entailment of generation by the corresponding sources. This paper finds that larger LMs are more factually correct, but in general tend to mix and match the named entities with wrong facts or fabricate entirely new named entities, especially with nucleus sampling. The authors propose to vary the nucleus parameter during nucleus sampling as the decoding proceeds so that there is more randomness at the start of a sentence and less randomness as it ends, so that nucleus sampling generates factually correct sequences.

The paper also proposes fine tuning objectives to improve factual correctness. One of the strategies is to prepend sequences with topic titles (obtained via Wikipedia titles), and the other approach is to use the latter part of sentences for loss computation with the idea being that facts are generated toward the end which have to be consistent with the information generated in prior steps. Empirical analysis is performed to test the proposed approaches.
strengths_and_weaknesses: The identified problem is important and the associated dataset and metrics will be a useful resource for factuality assessment of generative models. That said, the scope of the dataset is limited to Wikipedia articles and hence the metrics might be biased toward Wikipedia-like language. Extending to other kinds of knowledge bases does not seem very straightforward as this approach depends on a pre-exisitng dataset called FEVER which in turn heavily depends on Wikipedia.

The findings that large LMs are more factual is interesting and the proposed fixes also show that the larger LMs show greater improvement.

Nucleus sampling is compared with greedy decoding which is a bit unfair. It should be compared with other sampling approaches like ancestral sampling, sampling with a range of temperatures etc. Also, the proposed fix seems ad-hoc as it encourages deterministic output toward latter parts of the generation. However, the prompt itself sets up some background so following the proposed motivation, there should be less stochasticity in the generations immediately following the prompt as well. It should be noticed that barring diversity, greedy decoding is superior on other metrics. Comparison with temperature based sampling will help -- especially if the same scheme is applied to the evolution of temperature as is applied to the nucleus parameter. This would shed more light on the effect of reducing stochasticity gradually while generating.

The proposed training fixes are simple heuristics as well but the results do seem to suggest that they are very effective at improving factuality generation. However, it should be noticed that fine tuning is done on Wikipedia as well which favors the proposed dataset and metrics. Experiments with fine tuning on factual data from sources other than Wikipedia would be helpful. Also, these approaches do require finetuning of a large model though which can be expensive.

Overall, the paper has important contributions but they come with some caveats.
questions: Please find my comments in the section above.
limitations: N/A
review_title: null

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: In this paper, the authors highlight the factuality problem in the generation of language modeling. And they proposes a two-fold solution to improve factuality. The first part includes several sampling adaptations to avoid unnecessary logits sacrifice according to the growing sequence length. The second part includes a topic prefix method and sentence completion as the additional loss. The results show that their methods reduce the entity and entailment error in a factual-based perspective.
strengths_and_weaknesses: Strength
- The entity error and entail error measurements are reasonable criteria for factuality measurement.
- The observation of randomness in nucleus sampling adds value to better understand the factuality in language generation.

Weaknesses
- It seems that the entity ratio is not the whole world of factuality, organization and representation of sentences, is important as well apart from the entity mentions (e.g. negations). It would better if the experiments are conducted on some factual intensive tasks or having some human evaluations to help better interpret the results.
questions: It would be helpful to elaborate on line 303.
limitations:
review_title: null

final_score: rating: 4: Borderline reject: Technically solid paper where reasons to reject, e.g., limited evaluation, outweigh reasons to accept, e.g., good evaluation. Please use sparingly.
review_content: summary: This paper examines the factual incorrectness issue in text generation and proposed some decoding and training method to address these. Experimental results showed some insights regarding the effect of decoding on factuality, and the proposed methods showed improved factuality based on automatic evaluation metrics. 

strengths_and_weaknesses: The paper addresses an important problem in text generation using large language models. 

The analysis results regarding greedy and nucleus sampling decoding are intuitive. The proposed decoding method is a reasonable tradeoff between greedy and nucleus sampling, and it’s good to see that empirical results show it’s effective. Authors also provided results when only wikipedia is used, i.e., domain adaptive pertaining, there is less effect on factuality, supporting the improvement comes from their proposed approach. 
Overall the paper has some interesting results, however, it leaves some questions in its current form, and is not ready for publication yet. 

About the results:
Automatic metrics can give some information about the model quality. But human evaluation is an important aspect in generation evaluation and providing those results would strengthen the paper. Such results are not needed for every setup.   

The proposed method lacks some clarify and justification. Specifically the factuality enhanced pretaining  is not well justified.  

Unclear parts about the additional pertaining:
Authors motivated the TopicPrefix using the pronoun issues in the document. However, I don’t see why adding that prefix address the issue. Even though it’s telling the model this part is related to factual information, it can still connect with the wrong entities.
Another question is how is this prepending used during testing text generation?  Does it know for what prompts this needs to be prepended?  If so, is this a reasonable assumption? 

Similarly for the sentence completion loss, it’s not well explained or justified how or why applying zero-masking for the token prediction loss helps factuality.  For different pivot it’s only used during training, during testing, nothing is needed? 




questions: 
Additional suggestions: 
Providing more examples would help better understand the system output. 

limitations: Authors didn't provide this in the paper, I think factuality is an important topic and worth adding some discussion for the model's limitations and societal impact of the technology.
review_title: null

Meta-review: metareview: This paper proposed a new dataset and a new benchmark for factuality of open-ended text generation.  Based on an analysis on the factuality of language models with different sizes using this benchmark, this paper proposed a modified top-k sampling strategy and a modified training method to improve the factuality of text generation.  The work is solid.  Three of four reviewers give positive ratings while reviewer 4pXB negative.  The author(s) addressed the concerns adequately but the reviewer 4pXB did not reply.
confidence: Certain
Decision: Accept