Synchronous Faithfulness Monitoring for Trustworthy Retrieval-Augmented Generation (2024)

Abstract: Retrieval-augmented language models (RALMs) have shown strong performance and wide applicability in knowledge-intensive tasks. However, there are significant trustworthiness concerns as RALMs are prone to generating unfaithful outputs, including baseless information or contradictions with the retrieved context. This paper proposes SynCheck, a lightweight monitor that leverages fine-grained decoding dynamics including sequence likelihood, uncertainty quantification, context influence, and semantic alignment to synchronously detect unfaithful sentences. By integrating efficiently measurable and complementary signals, SynCheck enables accurate and immediate feedback and intervention. Experiments show that SynCheck significantly outperforms existing faithfulness detection baselines, achieving over 0.85 AUROC across a suite of six long-form retrieval-augmented generation tasks. Leveraging SynCheck, we further introduce FOD, a faithfulness-oriented decoding algorithm guided by beam search for long-form retrieval-augmented generation. Empirical results demonstrate that FOD outperforms traditional strategies such as abstention, reranking, or contrastive decoding significantly in terms of faithfulness, achieving over 10% improvement across six datasets.

## introduction retrieval-augmented language models (ralms) synergize large language models (llms) with external knowledge sources such as wikipedia, web search, or tool invocations @xcite @xcite . recent studies show that directly providing free-formed retrieved evidence in the context of llms can correct inaccurate or outdated parametric knowledge @xcite , leading to strong performance on knowledge-intensive tasks such as open-domain question answering @xcite and longform generation @xcite . despite the promising performance, the trustworthiness of ralms' generation has become a concern: human evaluations reveal a substantial number of claims generated by ralms contradicting with the provided context or cannot be grounded to any evidence @xcite . such unfaithful use of knowledge by llms renders it difficult to trust the output even if the knowledge source is proven trustworthy. towards a fully faithful and transparent use of knowledge, one line of work proposes post-hoc attribution or revision @xcite ), yet they are computationally expensive and could only be triggered after decoding. on the other hand, several synchronous decoding interventions have been proposed for critiquing and correcting ralms' outputs on-the-fly, such as dynamic retrieval @xcite , reranking with fine-tuned critique tokens @xcite , and contrastive decoding for amplifying the influence of the knowledge @xcite . however, these methods are mainly accuracy-oriented and it is unclear how well these signals are able to distinguish faithful samples from unfaithful ones. more importantly, these algorithms cannot provide any ways to guarantee or control the level of faithfulness of ralms' final output. to bridge these gaps, this paper undertakes a principled approach to faithfulness-oriented detection and decoding for long-form generation of ralms. to start with, we compile a comprehensive benchmark to thoroughly evaluate faithfulness detectors at sentence-level, with the tasks covering biography generation, question answering, summarization, and data-to-text. surprisingly, existing quality control methods in ralm systems including likelihood-based filtering @xcite and instruction-tuned critique tokens @xcite exhibit serious deficiency, only achieving approximately 0.6 auroc across all the tasks ( §5.1). instruction retrieved contexts (a) synchronous faithfulness monitoring (b) faithfulness-oriented decoding autoregressive decoding real-time dynamics 𝑷 𝒇𝒂𝒊𝒕𝒉 = 𝟎. 𝟖 what are some psychological effects of hugging? syncheck syncheck syncheck syncheck backtrack faithfulness-guided beam search hugging has a positive impact on the body's physiology, resulting in feelings of relaxation [1]. according to [2], hugging can be seen as a form of nonverbal communication that conveys empathy. in addition, hugging and touch can significantly reduce the anxietyinduced mortality rate [3]. syncheck syncheck 𝑷 𝒇𝒂𝒊𝒕𝒉 = 𝟎. 𝟗 𝑷 𝒇𝒂𝒊𝒕𝒉 = 𝟎. 𝟐 𝑷 𝒇𝒂𝒊𝒕𝒉 = 𝟎. 𝟕 𝑷 𝒇𝒂𝒊𝒕𝒉 = 𝟎. 𝟑 ralm the physical contact in hugging can release oxytocin, which promotes emotional healing. finally, a psychological science paper mentions that hugging increases the sense of belonging and social connection [3]. 3 1 2 3-1 1 2 4-1 5-1 4-2 3-2 5-2 likelihood uncertainty semantic alignment context influence 𝑷 𝒇𝒂𝒊𝒕𝒉 3-3 4-3 5-3 4-4 3-4 5-4 6-1 𝜙 6-3 𝜙 faithful sentences unfaithful sentences beam size prune kept beams threshold prune in response, we propose syncheck, a simple, efficient, yet accurate faithfulness error detector. to accurately characterize different types of faithfulness errors, syncheck computes and monitors multiple complementary signals during ralm decoding: sequence likelihood, uncertainty quantification, context influence, and semantic alignment (figure 1a ). the signals are efficiently measured synchronously in the decoding process, and ensembled via a lightweight aggregator that imposes minimal overhead. despite the efficient design, syncheck achieves a strong faithfulness judgment performance, scoring on average over 0.85 auroc across six datasets and two llms, outperforming five traditional baselines by ranging from 4% to 35% ( §5.1). furthermore, we find taskspecific or model-specific training is not strictly required for training syncheck's aggregator, which significantly improves its usability ( §5.3). with syncheck, is it possible to design an intelligent and efficient algorithm to improve the generation's faithfulness with a guarantee? traditional methods struggle to strike a balance between informativeness and faithfulness. for instance, abstention @xcite @xcite can be overly conservative, wasting the high quality part of the response. by contrast, contrastive decoding @xcite fails to enforce hard constraints to guarantee a basic level of faithfulness. we propose faithfulness-oriented decoding (fod), a novel decoding algorithm that leverages syncheck to synchronously monitor the generation faithfulness and guide the decoding process towards producing more faithful outputs. specifically, fod entails (1) backtracking at a lowquality sentence and (2) initiating a beam search that uses the faithfulness score to prune samples and guide the search direction (figure 1b ). experiments show that fod significantly improves the generation's faithfulness over greedy search (12%), abstention (10%), reranking (13%), and context-aware decoding (cad, shi et al. ( 2024a )) (19%) across six datasets. compared to abstention, fod improves in both faithfulness and informativeness. compared to cad, fod generates more faithful samples at the same number of sentences ( §5.2). we will publicly release the benchmark and our code at https://github.com/xiaowu0162/ sync-ralm-faithfulness . advanced decoding for ralms recently, there has been a research interest in improving the decoding of black-box ralms. @xcite and @xcite consider iterative retrieval for refining the generation. @xcite use contrastive decoding to emphasize the influence of the retrieved context. flare @xcite actively updates the context by setting a threshold on the likelihood of the least confident token in the generated sentence. toolformer @xcite learns to actively invoke tools at knowledge-intensive locations in generation. self-rag @xcite learns critique tokens to score and rerank hypotheses generated based on different retrieved documents. compared to prior work, our decoding algorithm utilizes syncheck to give a fine-grained accurate guarantee of output quality, striking a better balance between faithfulness and informativeness of the output. ## conclusion in this paper, we introduced syncheck, a realtime lightweight detector designed to enhance the faithfulness of ralms in long-form generation by leveraging fine-grained decoding dynamics to detect unfaithful generated segments. our empirical results demonstrate that syncheck significantly outperforms existing faithfulness detection methods, achieving high auroc scores across various tasks. additionally, we proposed faithfulness-oriented decoding, which improves the faithfulness and informativeness of ralm outputs compared to traditional faithfulness intervention methods. this work underscores the importance of realtime monitoring and targeted interventions in advancing ralms as reliable tools for knowledgeintensive generation, paving the way for more trustworthy and interpretable ralm systems.