RF i D : Towards Rational Fusion-in-Decoder for Open-Domain Question Answering (2023)

Abstract: Open-Domain Question Answering (ODQA) systems necessitate a reader model capable of generating answers by simultaneously referring to multiple passages. Although representative models like Fusion-in-Decoder (FiD) have been proposed to address this challenge, these systems can inadvertently rely on spurious features instead of genuine causal relationships between the question and the passages to generate answers. To counter this problem, we introduce the Rational Fusion-in-Decoder (RFiD) model. Our model leverages the encoders of FiD to differentiate between causal relationships and spurious features, subsequently guiding the decoder to generate answers informed by this discernment. Experimental results on two ODQA datasets, Natural Questions (NQ) and TriviaQA (TQ), demonstrate that our model surpasses previous methods, achieving improvements of up to 1.5 and 0.7 in Exact Match scores on NQ, and exhibits an enhanced ability to identify causal relationships.

## introduction open-domain question answering (odqa) has garnered significant attention @xcite @xcite , leading to the development of various systems designed to retrieve relevant passages @xcite from large databases and generate corresponding answers @xcite . we utilize the fusion-in-decoder (fid) @xcite model as our baseline model, a sequence-tosequence paradigm based on the t5 model @xcite . given a question, the fid model encodes k retrieved passages using k respective t5 encoders, concatenates these k encoder hidden states, and then feeds the result into a t5 decoder to generate the answer. fid treats all passages equally in its encoders, relying exclusively on the cross-attention mechanism to establish correlations between the decoder and encoders. however, the cross-attention mechanism lacks an explicit mechanism for distinguishing differences among passages, which can result in the detection of spurious patterns @xcite . consequently, it becomes challenging for the model to identify crucial passages. an example of such spurious patterns observed in our experiment is depicted in figure 1 , where the model confuses "disney art of animation resort" with "art of disney animation" due to the prevalence of passages about the latter, resulting in an incorrect answer. to address this issue, we propose a conceptually straightforward strategy by introducing a rationalization process to explicitly determine which retrieved passages contain the answer before conducting answer generation. this process assigns different embeddings to rationale passages and irrelevant passages. these embeddings then guide the cross-attention during the answer generation phase. we dub this new model the rational fusionin-decoder (rfid). we evaluate the effectiveness of our proposed rfid model through experiments on the natural questions (nq) @xcite and triviaqa (tq) @xcite datasets. our results demonstrate that our methods can help models overcome spurious patterns and enhance their reasoning abilities, leading to an improvement of up to 1.5/0.7 exact match points on the nq/tq datasets respectively. further analysis reveals that our methods effectively direct models to focus more on correct causal features and less on spurious features. for instance, as seen in the rightmost column of figure 1 , our model has increased its attention on the relevant passage. to the best of our knowledge, we are the first to incorporate rationalization into odqa models, thus underscoring the importance of passage rationalization. ## method in this section, we explain the baseline fusion-in-decoder (fid) model and our rational-fid (rfid) model. the rfid model uses a passage-level classifier on top of each fid encoder to determine whether the corresponding passage is a rationale for the question. it guides the decoder with a rationale embedding concatenated to the encoder hidden states, as well as a multi-task framework to blend fid training with rationale training. ## conclusion in this study, we sought to rationalize the reader component of open question answering by introducing an explicit mechanism to identify potential answer-containing passages among the retrieved candidates. experimental results show that our rfid model effectively improves the rationale ability of the fusion-in-decoder model, leading to enhanced performance on odqa tasks. additionally, it outperforms the baseline and other concurrent models on both the natural questions and trivi-aqa datasets, with only a minimal increase in the number of parameters. we just use only @xmath0.