Multi-Granularity Guided Fusion-in-Decoder (2024)

Abstract: In Open-domain Question Answering (ODQA), it is essential to discern relevant contexts as evidence and avoid spurious ones among retrieved results. The model architecture that uses concatenated multiple contexts in the decoding phase, *i.e.*, Fusion-in-Decoder, demonstrates promising performance but generates incorrect outputs from seemingly plausible contexts. To address this problem, we propose the ***M**ulti-**G**ranularity guided **F**usion-**i**n-**D**ecoder (**MGFiD**)*, discerning evidence across multiple levels of granularity. Based on multi-task learning, MGFiD harmonizes passage re-ranking with sentence classification. It aggregates evident sentences into an *anchor vector* that instructs the decoder. Additionally, it improves decoding efficiency by reusing the results of passage re-ranking for *passage pruning*. Through our experiments, MGFiD outperforms existing models on the Natural Questions (NQ) and TriviaQA (TQA) datasets, highlighting the benefits of its multi-granularity solution.

## introduction open-domain question answering (odqa) @xcite is a challenging task that requires deriving factual responses from a vast knowledge corpus without relying on explicit evidence, i.e., the evidence context is not given. recently, retrievalaugmented generation (rag) @xcite has emerged to combine the retrieval of relevant information with response generation. exemplified by the retriever-reader architecture @xcite @xcite , rag effectively addresses odqa. the retriever first pinpoints the most relevant passages using the question as a query. subsequently, the reader extracts or generates a response using the question and the relevant passages. it generally allows us to perform a decoupled optimization for * corresponding author the retriever or the reader. in this paper, we mainly focus on optimizing the reader. to improve the reader, existing studies @xcite @xcite focus on addressing two questions: (i) how to effectively use the evidence in multiple passages, and (ii) how to improve the discrimination in dealing with spurious passages. multi-passage reader. as the representative model, fusion-in-decoder (fid) @xcite ) using a generative text-to-text model @xcite ) is an effective multipassage reader to aggregate evidence across multiple passages. it first encodes multiple pairs of a question and a relevant passage at the encoder. then, it generates an answer using a cross-attention mechanism over concatenated embeddings at the decoder. one limitation of the fid architecture is inefficiency due to the intensive cross-attention op-erations performed on the concatenated matrix. to mitigate this, some studies have proposed either shortening the input length @xcite or omitting some layers in the @xcite . more importantly, the standard fid model often struggles with handling spurious passages that degrade the accuracy of generating the answer @xcite . multi-task reader. several studies @xcite @xcite @xcite have attempted to address handling spurious passages by employing multi-task learning. it aims to improve the reader's discernment regarding the evidentiality of the retrieved passages, thereby achieving robustness against spurious ones. @xcite and @xcite proposed to incorporate information from factual triplets contained in the knowledge graph. another solution is to employ passage labels to discern spurious passages in the fid architecture. @xcite ; @xcite ; @xcite determined the rationality of passages based on whether they contain an answer span. although learning signals from answer span inclusions has been proven effective, it may lead to false positive passages, producing sub-optimal results. furthermore, existing multi-task readers face challenges in identifying the key sentences within the passage. we argue that relying solely on answer spans or identifying evident passages is insufficient to determine the evidence. figure 1 illustrates two plausible scenarios, highlighting the limitations of existing methods using either the answer spans or passage-level evidentiality. in figure 1(a) , the mere presence of the answer span in the passage does not guarantee relevance for the question. more importantly, figure 1(b) shows that a model trained primarily on aggregating evidence across passages generates an incorrect answer, and there is a need to distinguish complex and confusing sentences. this paper aims to discern evidence in coarseand fine-grained textual information, i.e., passages and sentences, and utilize the byproduct from multitask learning to enhance the model's performance. to this end, we propose a novel model called multi-granularity guided fusion-in-decoder (mgfid). specifically, the key idea behind mgfid is twofold. (i) we train the fid to distinguish evidentiality using multi-task learning to minimize the influence of false contexts during answer generation. in this process, we employ both passage-and sentence-level contexts to account for evidentiality in multi-granularity contexts. since it is expensive to label gold passages, we use the ranking abilities of language models @xcite to filter out irrelevant contexts for the question. (ii) we reuse auxiliary information from multi-task learning to improve accuracy and efficiency. we generate an anchor vector derived from sentence-level classification and infuse it into the [bos] token used in the decoder. since the anchor vector indicates a significant feature for relevant sentences, it helps the decoder generate the correct answer. furthermore, we employ passage-level re-ranking results to prune less supportive passages, improving the efficiency in the decoding phase. to summarize, the key contributions of this paper are as follows. (i) we introduce the evidentiality of the fid using multi-granularity contexts. (ii) we utilize llms to generate pseudo-labels for supportive passages in odqa task. (iii) we reuse multi-granularity contexts to improve accuracy and efficiency further using an anchor vector in the decoder and thresholding-based passage pruning. (iv) through our experiments on two benchmark datasets, we show that mgfid improves the original fid by more than 3.5% and 1.0% in exact match on natural questions, and triviaqa, outperforming the other baselines. ## related work we briefly review existing studies for improving the fid @xcite architecture in two key aspects: accuracy and efficiency. ## proposed method in this section, we first outline our method for multi-task learning, which integrates generating answers and determining their evidence at different levels of granularity, i.e., passages and sentences (section 3.1). second, leveraging sentence-level predictions, we introduce an anchor vector to provide a rationale signal to the decoder (section 3.2). we then present threshold-based pruning using passage-level scores to enable efficient decoding (section 3.3). lastly, we describe the process of generating pseudo-labels for supportive passages (section 3.4). ## conclusion this paper presents the multi-granularity guided fusion-in-decoder (mgfid), a novel reader for managing evidence across multiple granularities. addressing the prevalent challenges of misleading passages and sentences, mgfid synergies coarselevel passage re-ranking with fine-level sentence classification. we also incorporate llms to enhance the quality of heuristic labels. moreover, mgfid capitalizes on its multi-granularity evidence by constructing an anchor vector that guides the decoder toward significant evidence and employs passage pruning to enhance decoding efficiency. our empirical results demonstrate that mg-fid using multi-granularity contexts achieves significant advancements over baseline models.