When to Trust Context: Self-Reflective Debates for Context Reliability
Authors: name: Zeqi Zhou; name: Fang Wu; name: Shayan Talaei; name: Haokai Zhao; name: Cheng Meixin; name: Tinson Xu; name: Amin Saberi; name: Yejin Choi
Published: 2025-06-06 12:09:34

Abstract: Large language models frequently encounter conflicts between their parametric
knowledge and contextual input, often resulting in factual inconsistencies or
hallucinations. We propose Self-Reflective Debate for Contextual Reliability
(SR-DCR), a lightweight framework that integrates token-level self-confidence
with an asymmetric multi-agent debate to adjudicate such conflicts. A critic,
deprived of context, challenges a defender who argues from the given passage; a
judge model evaluates the debate and determines the context's reliability. The
final answer is selected by combining the verdict with model confidence.
Experiments on the ClashEval benchmark demonstrate that SR-DCR consistently
enhances robustness to misleading context while maintaining accuracy on
trustworthy inputs, outperforming both classical debate and confidence-only
baselines with minimal computational overhead. The code is available at
https://github.com/smiles724/Self-Reflective-Debates.