When to Speak, When to Abstain: Contrastive Decoding with Abstention
Authors: name: Hyuhng Joon Kim; name: Youna Kim; name: Sang-goo Lee; name: Taeuk Kim
Published: 2024-12-17 04:38:08

Abstract: Large Language Models (LLMs) demonstrate exceptional performance across
diverse tasks by leveraging both pre-trained knowledge (i.e., parametric
knowledge) and external knowledge (i.e., contextual knowledge). While
substantial efforts have been made to leverage both forms of knowledge,
scenarios in which the model lacks any relevant knowledge remain underexplored.
Such limitations can result in issues like hallucination, causing reduced
reliability and potential risks in high-stakes applications. To address such
limitations, this paper extends the task scope to encompass cases where the
user's request cannot be fulfilled due to the lack of relevant knowledge. To
this end, we introduce Contrastive Decoding with Abstention (CDA), a
training-free decoding method that empowers LLMs to generate responses when
relevant knowledge is available and to abstain otherwise. CDA evaluates the
relevance of each knowledge for a given query, adaptively determining which
knowledge to prioritize or which to completely ignore. Extensive experiments
with four LLMs on three question-answering datasets demonstrate that CDA can
effectively perform accurate generation and abstention simultaneously. These
findings highlight CDA's potential to broaden the applicability of LLMs,
enhancing reliability and preserving user trust.