Collaborative decoding of critical tokens for boosting factuality of large language models
Authors: Lifeng Jin, Baolin Peng, Linfeng Song, Haitao Mi, Dong Yu


Reviewer paper summaries: This paper presents a novel method called Collaborative Decoding Strategy (CDS) to enhance the factuality of large language models. The strategy hinges on the concept of 'critical tokens' which guide the decoding process using a specially designed critical token classifier. This classifier determines which model should be used for generating the next token. Essentially, the Model CDS leverages the high factuality inherent in pretrained models while capitalizing on the instruction-following capabilities of the aligned model. 

The performance of the Model CDS is demonstrated as superior when compared to various baselines. It achieves improved factuality without compromising on the diversity of responses. Importantly, the Model CDS exhibits compatibility with other approaches involving updates to parameterized knowledge within models, thereby pushing the performance ceiling higher than that of the pretrained model.
Summary of strengths: 1. The idea of a unique, collaborative decoding approach to increase factuality in language models is well-motivated.
2. Versatility of the Model CDS, allowing it to be compatible with other methodologies, implying broad applicability.
Summary of weaknesses: 1. The core assumption is that pre-trained models inherently perform better, which has only been verified on limited benchmarks, which may be biased.
2. Holding two models for decoding is also expensive.
Comments, suggestions and typos: 1. Is it limited to QA tasks? How do we understand and define critical token predictors in a more general scenario?
2. How well does the critical token prediction model generalize?
Overall assessment scores: 2.5
Metareview score: 3
Metareview: The paper introduces a collaborative decoding framework to harness the high factuality within pretrained models through the concept of critical tokens. This strategy leverages the high factuality of pretrained models while benefiting from the instruction-following capabilities of aligned models.
Summary of reasons to publish: Introduces a unique, collaborative decoding approach, Model CDS, to enhance factuality in language models.

Introduces the concept of critical tokens and develops a model-based approach to predict their location.

Model-agnostic method.