An Encoder Attribution Analysis for Dense Passage Retriever in Open-Domain Question Answering (2022)

Abstract: The bi-encoder design of dense passage retriever (DPR) is a key factor to its success in open-domain question answering (QA), yet it is unclear how DPR’s question encoder and passage encoder individually contributes to overall performance, which we refer to as the encoder attribution problem. The problem is important as it helps us identify the factors that affect individual encoders to further improve overall performance. In this paper, we formulate our analysis under a probabilistic framework called encoder marginalization, where we quantify the contribution of a single encoder by marginalizing other variables. First, we find that the passage encoder contributes more than the question encoder to in-domain retrieval accuracy. Second, we demonstrate how to find the affecting factors for each encoder, where we train DPR with different amounts of data and use encoder marginalization to analyze the results. We find that positive passage overlap and corpus coverage of training data have big impacts on the passage encoder, while the question encoder is mainly affected by training sample complexity under this setting. Based on this framework, we can devise data-efficient training regimes: for example, we manage to train a passage encoder on SQuAD using 60% less training data without loss of accuracy.

## introduction attribution analysis, or credit assignment, concerns how individual components of a system contribute to its overall performance @xcite . in this paper, we are interested in the encoder attribution problem of dense passage retrievers (dpr) @xcite for open-domain question answering @xcite . dpr leverages a bi-encoder structure that encodes questions and passages into low dimensional vectors separately. figure 1 : encoder marginalization. here, "*" denotes the target encoder we want to evaluate, where we use the q-encoder of dpr trained on nq as an example. the q-encoder is evaluated on nq-test data and paired with different p-encoders, and the final contribution is determined by averaging across the scores of different encoder pairings. follow-up work has proposed various methods to further improve and analyze dpr @xcite @xcite . however, most of these methods only test the bi-encoder model in tandem, leaving two questions unanswered: the first problem, which we refer to as encoder attribution, is important as it helps us understand which part of the dpr model might go wrong and identify possible sources of error in the data for the second problem. therefore, it is important to separately inspect individual encoders of dpr. in this paper, we perform an encoder attribution analysis of dpr under a probabilistic framework, where we model the evaluation function for dpr's predictions as a probabilistic distribution. the core component of our method is called encoder marginalization, where we target one encoder and marginalize over the other variables. we then use the expectation under the marginalized distribution as the encoder's contribution to the evaluation score. the marginalization can be approximated using monte-carlo, as illustrated in fig. 1 , where encoders trained from different domains are used as empirical samples, which will be discussed in section 3.2. for question (1), we introduce a technique we call encoder marginalization to compare the question encoder and passage encoder of the same dpr (section 5.2). we find that in general, the passage encoder plays a more important role than the question encoder in terms of retrieval accuracy, as replacing the passage encoder generally causes a larger accuracy drop. for question (2), we perform a case study where we analyze dpr's individual encoders under a data efficiency setting. we evaluate different dpr models trained with different amounts of data. under this setting, we find that positive passage overlap and corpus coverage of the training data might be the affecting factors for the passage encoder, while the question encoder seems to be affected by the sample complexity of training data. based on the discovery of these affecting factors, we develop a data-efficient training regime, where we manage to train a passage encoder on squad using 60% less training data with very little drop in accuracy. our work makes the following four main contributions: ## background and related work attribution analysis it is also known as credit assignment and has long been discussed in various areas and applications. in reinforcement learning @xcite , the accumulated re-ward from the environment needs to be distributed to the agent's historical decisions @xcite @xcite . in investment @xcite , it is used to explain why a portfolio's performance differed from the benchmark. attribution analysis has also been used in nlp @xcite and cv @xcite to interpret models' decisions. therefore, attribution analysis is an important topic for understanding a system's behavior, especially for black-box models like deep neural networks @xcite . retrieval for qa first-stage retrieval aims to efficiently find a set of candidate documents from a large corpus. term-matching methods such as bm25 @xcite have established strong baselines in the firststage retrieval of various qa tasks @xcite @xcite . recently, retrievers based on pre-trained language models @xcite also make great advancements @xcite @xcite . particularly, dense passage retrievers (dpr) @xcite set a milestone by encoding questions and passages separately with a bi-encoder design. based on dpr, multiple works on compression @xcite @xcite , hardnegative mining @xcite , multi-vector encoding @xcite , and qa pre-training @xcite expand the boundary of dense retrieval. other analyses of dpr beir investigates dpr's transferability to multiple domains and retrieval tasks @xcite , while mr.tydi evaluates dpr pre-trained on english for retrieval in a multi-lingual setting @xcite . @xcite find that most of the test answers also occur somewhere in the training data for most qa datasets. @xcite observe that neural retrievers fail to generalize to compositional questions and novel entities. @xcite also find that dense models can only generalize to common question patterns. ## experimental setup we follow the dpr paper @xcite to train and evaluate our dense retrievers. we reproduce their results on five benchmark datasets using tevatron foot_1 @xcite , a toolkit for efficiently training dense retrievers with deep neural language models. our reproduced results have only a maximum difference of ∼2% compared to their numbers. we report the top-20 and top-100 accuracy for evaluation. ## conclusions we propose an encoder attribution analysis of dpr using encoder marginalization to individually evaluate each encoder of dpr. we quantify the contribution of each encoder of dpr by marginalizing the other random variables under a probabilistic framework. we find that the passage encoder plays a more important role compared to the question encoder in terms of top-k retrieval accuracy. we also perform a case study under the data efficiency setting to demonstrate how to find possible affecting factors in the qa datasets for individual encoders. we identify that passage encoders are affected by positive passage overlap and corpus coverage of the training data, while question encoders are sensitive to the training sample complexity. our framework is also very general and can be applied to other methods based on bi-encoders for encoder attribution analysis, but one needs to pay attention to the choice of the encoder prior distribution to ensure the marginalization is appropriate.