Detrimental Contexts in Open-Domain Question Answering
Authors: Philhoon Oh
James Thorne
Venue: EMNLP 2023 Findings (2023)
Keywords: Retrieval
Reasoning
Question Answering
DPR
KILT
Fusion In Decoder
TL;DR: Why does more evidence harm model performance for open domain QA

Abstract: For knowledge intensive NLP tasks, it has been widely accepted that accessing more information is a contributing factor to improvements in the model's end-to-end performance. However, counter-intuitively, too much context can have a negative impact on the model when evaluated on common question answering (QA) datasets.  In this paper, we analyze how passages can have a detrimental effect on retrieve-then-read architectures used in question answering. Our empirical evidence indicates that the current read architecture does not fully leverage the retrieved passages and significantly degrades its performance when using the whole passages compared to utilizing subsets of them.  
Our findings demonstrate that model accuracy can be improved by 10\% on two popular QA datasets by filtering out detrimental passages. Additionally, these outcomes are attained by utilizing existing retrieval methods without further training or data. We further highlight the challenges associated with identifying the detrimental passages. First, even with the correct context, the model can make an incorrect prediction, posing a challenge in determining which passages are most influential. Second, evaluation typically considers lexical matching, which is not robust to variations of correct answers. Despite these limitations, our experimental results underscore the pivotal role of identifying and removing these detrimental passages for the context-efficient retrieve-then-read pipeline.

Detrimental Contexts in Open-Domain Question Answering
Philhoon Oh
James Thorne KAIST AI
KAIST AI
philhoonohekaist.aacuu
thorne@kaist.ac.
Abstract
Reader Model Input
EM Score
AcEM Score
For knowledge intensive NL. tasks, has been widely accepted that accessing more informa- tion contributing factor improvements in the model's end-to-end performance. However, counter-intuitively, too much context can have negative impact the model when evaluated on common question answering (QA) datasets. this paper, we analyze how passages can have detrimental effect reaeeeve-ten-rrad architectures used in question answering. Our empirical evidence indicates that the current read architecture does not fully leverage the retrieved passages and significantly degrades its performance when using the whole passages compared to utilizing subsets of them. Our find- ings demonstrate that model accuracy can be improved by 10% on popular QA datasets by filtering out detrimental passages. Addition- ally, these outcomes are attained by utilizing existing retrieval methods without further train- ing or data. We further highlight the challenges associated with identifying the detrimental pas- sages. First, even with the correct context, the model can make an incorrect prediction, pos- ing challenge in determining which passages are most influential. Second, evaluation typi- cally considers lexical matching. which not robust variations of correct answers. De- spite these limitations, our experimental results underscore the pivotal role identifying and removing these detrimental passages for the context-efficient surieve--heen-read pipeline.
P1
P1
EM Scons Regression
Figure I: Even when provided with correct passage (P2) addition of further information (P4) causes the reader model to change the prediction, highlighting is- when using recall-optimized retrievers. Differences between Exact Match (EM) score and Accumulated EM (AcEM) indicate knowledge issues.
common architecture for modelling (Lewis et al., 2020; Petroni et al., 2021; Thakur et al., 2021; Izac- ard et al., 2022; Lyu et al., 2023) where a two-stage architecture of a retriever model finds contextual passages and passes these to a reader module that uses this context when it generates answer.
In this two-step pipeline, expected performance (such answer exact match score) increases mono- tonically more information is provided to the reader model at test time. Therefore, it is com- monly conceived that providing the reader with more context improves overall performance (Izac- ard and Grave, 2020; Liu et al., 2023). However, empirical studies (Sauchuk et al., 2022) contradict this general belief: even when true positive pas- sages are provided to the model, the presence of relevant but non-evidential documents in phenomena
1 Introduction
Knowledge-intensing NLP tasks such open- domain question answering (Rajpurkar al., 2016; Yang al., 2018) and evidence-base based fact veri- fication (Thorne et al., 2018; Jiang et al., 2020) require models to use external sources of textual information to condition answer generation in re- sponse to an input. This family of tasks shares
change
correct outputs to incorrect called damaging retrieval. While this phenomena was not thoroughly explored in the context of ques- tion answering, it could suggest that the current reader may be influenced by damaging retrieval, potentially leading to decline in performance.
Coode and data are available on httts://giitul hetillabbmmml2222222 hamaging-retrieve retrieval
In this paper, we propose an approach to address-

---

ing this problem by identifying and removing con- texts that are detrimental the performance of the model. As described in Figure 1, we treat the reader model, FiD (Izacard and Grave, 2020), as a black- box oracle and identify which passages are dam- aging by evaluating how different subsets of the retrieved context cause the answer to change. Our experiments are conducted on two question answer- ing tasks: Natural Questions (Kwiatkowski al., 2019) and TriviaQA (Joshi et al., 2017), evaluating the effect of three IR systems: DPR (Karpukhin et al., 2020), SEAL (Bevilacqua et al., 2022), and Contriever (Izacard et al., 2021).
contrast to the conventional T5-Model (Raffel al., 2019), each (query, passage) pair is independently encoded, and the encoded vectors are concatenated when input into the decoder model. This allows more documents to be encoded overcoming the high memory complexity of self-attention in the en- coder. Experimental results indicate that this archi- tecture also supports recall-oriented optimization of the upstream information retrieval systems. For question-anse answering tasks, the answer exact match scores increased monotonically with the number of context passages used (Izacard and Grave, 2020; Liu et al., 2023).
Our findings challenge the conventional assump- tion that more passages lead to higher performance. In fact, introducing additional relevant contexts can actually worsen the performance of the model in question answering. By excluding passages that have a detrimental effect, we observe up to 10% improvement in the exact match score under ideal conditions without requiring any architectural mod- ifications. Notably, these gains surpass the perfor- mance improvements seen in recent works that pro- pose FiD extensions like FiD-ligh((Hofssääteer et al.. 2022) and FiE (Kedia al., 2022). In addition, we analyze why automatically identifying damag- ing contexts is challenging, even with attention- based and model-based proxies for filtering. De- spite these inherent limitations, considering effec- tiveness and operational efficiency, our empirical findings could potentially direct the focus of open- domain NLP models and yield performance im- provements when using fewer context passages.
Extensions of this architecture, such as FiD- Light (Hofstätter al., 2022), FiDO (de Jong al., 2022) are introduced to further increase the runtime efficiency. FiD-Liiht, for example, compresses passage encoder embedding into first-K vectors to overcome a performance bottleneck in the decoder. In the case of FiDO, it only keeps cross-attention every K-th decoder layer well applics multi- query attention to mitigate the inference costs and memory usage. Unlike these architectures, FiE (Kedia al., 2022), an encoder-only model, adds global encoder layers to fuse the information across multiple evidence passages. While these models significantly reduce inference time, they only result in minor increases in the answer exact match score.
Recent advances in LLMs demonstrate remark- able capabilities in natural language generation tasks (Liang et al.. 2022). Within the context of question answering, Lyu et al. 2023 utilize an LLM as reader in retrieval- manner (Lewis et al., 2020). However, recent stud- ies (Zheng et al., 2023) demonstrate that answers generated from LLMs vary depending on the order of given contexts/answer choices and are some- times lost in the middle (Liu et al., 2023). While Zheng et al. 2023 proposes a method for alleviat- ing this order-variant issue by permutating given choices, this approach is not practical for 100 can- didate contexts. Although investigating this issue falls outside the scope of research, further search is needed address the issue of the answer variability in L.LMs, which arises from the order of provided contexts/answer choices.
generation
2 Background
2.1 Knowledge Intensive NI.P
To integrate external knowledge for NLP tasks such question answering (Kwiatkowski et al., 2019; Joshi al., 2017; Yang et al., 2018), slot filling (Levy et al., 2017), and fact verification (Thorne et al., 2018), systems employ a sttrieve-then-ree pipeline of two models (Chen al., 2017). A re- triever first selects the context passages over large corpus such as Wikipedia and then feeds informa- tion to the reader to derive the answer. Conse- quently, improving the recall of the retriever leads to improvement of the downstream reader.
2.2 Improving Retrieval
Large-scale information fusion architectures such as Fusion in Decoder (FiD) (Izacard and Grave, 2020) are able to combine information in multiple passages when decoding the answer. In
Two different approaches have been studied for re- trieving relevant contexts: one approach employs a bi-encoder architecture, where the inner product of

---

query embeddings and context embeddings used a proxy for the relevance score. ORQA(Lee et al., 2019), DPR (Karpukhin et al.. 2020) are both based on BERT-based bi-encoder architec- tures (Devlin et al., 2018). Contriever (Izacard et al., 2021) trains a single encoder in an unsuper- vised manner to represent both query and context embeddings. These encodings capture the relation between query and passages where most similar results can be identified through ranking the top-N articles through inner-product search between em- bedded passages and the embedding of the query. Alternatively, generatieve retrieval offers an al- ternative mechanism where the relationship be- tween the query and an entity is encoded in the parametric space within a model. GENRE (Cao et al., 2020) utilizes an autoregressive sequence-to- sequence model to generate canonical entity titles for query with constrained decoding using a pre- fix tree. Extending this, SEAL (Bevilacqua et al., 2022) direetly predicts substrings in the corpus us- ing an FM-index. GenRead (Yu al., 2023) does not rely on any indexing system: LLMs are used to generate synthetic contexts. These approaches demonstrate favorable precision-recall trade-offs while requiring smaller index storage footprints.
such as FiD that treat retrieved evidence as a set, filtering the damaging passages is more appropriate than re-ranking the retrieval results.
Motivating Pilot Study
We demonstrate the effect of damaging passages extending Sauchuk et al. (2022) for the FiD model demonstrating that the model performance is sen- sitive to irrelevant context information. To select damaging context passages, we employ two meth- ods: random sampling and negative sampling. For random sampling, we select passages from the cor- pus, C, uniformly at random, while for negative sampling, we use BM25 to select passages that do not contain the correct answer but have high lexical overlaps. Results are reported in Figure 2 an Table 1. We use a context of up to 5 passages on 2539 instances from the NQ dev dataset and employ FiD-large? trained on NQ train set. We add between 0-4 sampled passages in addition the gold passage from the dataset. FiD invari- ant to passage ordering: we derive this analytically from lack of positional encoding between the en- coded passages and validate it empirically through random permutation of the context set.
3.1 Effects of Simulated Damaging Passages
2.3 Damaging Retrieval
We compute Stability Error Ratio (Krishna et al., 2021, SER): the ratio of instances with modified outputs to additional samples, given the instance has correct predictions using one gold passage.
Conventionally, retrieval systems are optimized to find maximally relevant documents to support a given user query (Ferrante et al., 2018). However, Terra and Warren (2005) demonstrates that the re- triever can extract relevant but harmful documents and Sauchuk et al. (2022) also show that irrelevant incorrect documents added to the input of the model can negatively affect its performance. Identifying these damaging passages in a two- stage pipeline has not been extensively explored. However, one can consider reranking (lyer al., 2021: Kongyoung al., 2022: Glass et al., 2022) as process for selecting an optimal subset of re- trieved passages and removing detrimental ones. However, FiD architectures scale well with many passages, enabling higher exact match scores sim- ply by increasing the number of passages provided to the model. Given a large enough budget for passages, re-ranking this set of passages may not necessarily exclude them from the answer set. Fur- thermore, as further discussed in Section 1, the FiD model is order invariant and the ranking of pas- sages is ignored during encoding. In architectures
Random Sampling Random passages should not contain information related to the query.. Where models are resilient to noise, we expect that ran- domly sampled additional do not drastically affect the reader model. Because of the large seman- tic distance between the randomly sampled texts and the gold passage, irrelevant information is not damaging. Injecting these random passages had negligable increase on answer EM (Figure 2(a)) and SER is under 3% across all the cases Ta- ble 1. This, in fact, indicates that FiD is robust to additional random passages.
Negative Sampling Passages sampled by BM25 have high lexical overlap and are semantically simi- lar meaning to the query. We expected this result in confusion in the model as the model uses related but insufficient information to condition answer
Pht //gi thub. http:///aaaaooooeeeeara////aai

---

# Negative Samples 0.75%
2
3
with retrieval models to confirm the presence of damaging effects in the tehreveethen--rad pipeline. While retrievers predict a set of passages that are relevant to the model, it does not have perfect precision, and we do not have knowledge of which passages from the retriever are beneficial or detri- mental. In order to identify wheter passuge is damaging not, we utilize the reader model black box oracle to distinguish damaging con- texts from positive contexts. Starting with the top-1 passage, we incrementally add passages from the retrieved list and evaluate how the answer changes. If the inference is correct with the top-1, it is clas- sified as a positive context. We then proceed to evaluate the top-1 and top-2 passages together. If those passages generate incorrect answer, the second evidence passage will be considered damag- ing. This is possible because of the order invariance property of FiD. Therefore, the process can be re- peated iteratively for up to the top-N passages. We then use this sequence of predictions to yield the exact match pattern. The exact match pattern is binary sequence where 1 represents that the k- th prediction matches the answer, and 0 otherwise (Figure 1). This string represents an exact match at k (EM @k) when using up to top-k passages as input. From this, we also compute accumulated ex- act match (AcEM), AcEM@k k is the maximum of all EM @k values up to k. For instance, ACEM @k equals 1 if at least one of the values in (EM @ 1, EM @ 2, .... EM@k) equals 1. Thereby, the discrep- ancy between AcEM @k and EM @k can be used to measure the damaging effects retrievers.
SER w/ random
1.4%
1.7%
2.0%
SER w/ BM25
8.3%
12.5%
14.8%
15.9%
Table 1: Stability Error Ratio on Random/BM25 nega- tive passages. An increase in SER implies the model's 's instability on negative samples.
2.622
0.70
Eaxr Parrs
0.28
0.55
0.4
3.679
Marber Content
Nuybe LCoontoss
(a) Random passages
(b) BM25 negative passages
Figure 2: Exact Match scores of FiD on Natural Ques- tions with a different number of contexts (X-axis). As the Exact Match score remains unchanged regardless of the position of the gold context, we represent this relationship using single blue line in the plot.
generation. Adding additional negative samples re- sults in monotonic decrease in EM score, which is represented as a blue line in Figure 2(b) and an increase in SER, reported in Table 1. A rising number of instances where the model changes the answer from predicting the correct answer to an incorrect highlights the damaging effects.
In general, negative passages tend to have nega- tive impact on the model's 's inference. However, have discovered that certain negative passages can actually enhance the model's performance when paired with gold evidence. To measure this, we calculate an extension of EM score called Accu- mulated Exact Match(AcEM) This checks the ex- istence of Exact Match on up to top-K passages. Surprisingly, AcEM monotonically improves with additional negative passages, yielding higher re- sults than when using only gold contexts. This demonstrates that the model can also benefit from negative samples to make correct predictions and implies that damaging passages should be differen- tiated from negative contexts. We report compar- isons of AcEM and EM with respect to the number of context sizes in Figure 2(b).
We utilize three different retrievers for two ques- tion answering validation datasets, Natural Ques- tions (K Kwiatkowski et al., 2019, NQ) and TriviaQA (Joshi et al., 2017, TQA), which were evaluated on FiD (Izacard and Grave, 2020). Following pre- vious work, we use a corpus of non-overlapping 100 word chunks for retrieval. We use the 100-100 candidate passages retrieved by DPR (Karpukhin et al., 2020), SEAL (Bevilacqua et al., 2022), and Contriever (Izacard al.. 2021)$. For DPR, we utilize the published retrieval result, while for the other retrievers, we run the code using the default parameters. We use two published models of FiD- large that were trained on DPR retrieved contexts from NQ/TriviaQA for inference.
4 Damaging Passages from Retrievers
Section 3 simulates the impact of damaging pas- sages. However, these negative passages are sam- pled from a distribution different from what the reader is trained on. Therefore, we experiment
hctoos//ggttuub..co/aaacooorrrre htmm//aacebookreeaarcc//rrr
'https: htwp://///tttubbcoomo httptabb..c/aaacccooorreeaarr contriever

---

on Questions,
DPR
Costrieer
Netural Questions
0.73
0.73
EM
AcEM
Figure 3:
Incremental inference for 3 retrievers NQ and TriviaQA dev set. This implies the existence of damaging passages in retrieved contexts and higher performance can be attained using subset of context
Results are illustrated in Figure All models show higher AcEM@k on all values. For NQ, the EM 100 scores from DPR, SEAL, and Con- triever are 52.5%, 50.0%, and 50.7% while attain- ing 62.3%, 59.4% and 60.8% in AcEM@ 100. This indicates the existence of damaging effects in re- trieved contexts, and the model performance could be at least 9% points higher if the retrieved pas- sages can be refined remove damaging passages.
versely, transition from suggests that the additional passage negatively impacts the predic- tion. Thus, we classify passages into five types based on transitions in the EM patterns: IZ(initial zeros), DP(definite positives), DN(definite nega- SP(semii-ositive) positive), and EN(semi-negative tives), SP(semi
Let EM(ps) denote EM@kk. Then, for each y) given P {pI P2, .... PN] ), we define Type(p:):
IZ where EM(pp) 0 and EM(1,) for all j = 1,, 1. Consecutive zeros appcaring at the start of the EM pattern correspond to possibly relevant or uninformative passages.
5 Analysis Methodology
5.1 Identifying Passage Types
In order identify which passages are responsible for damaging effects the retrieved list, we cat- egorize passages based on the Exact Match(EM) pattern as an indicator to determine if passage is positive not. If the EM@K k is equal to 1, it can be inferred that the kth passage is positive. However, this does not necessarily imply that the passage contains relevant information for the outcome. could be false positive due to the presence of pre- ceding positive passages in the input. Similarly, 0 in the EM pattern may suggest that the passage is negative. However, this could also be a false negative resulting from FiD's 's failure to predict the answer despite the presence of correct passage.
DP where EM(p:) = and EM(p; 1) 0 or EM(p,) = when i = 1. Definite positive passages cause EM to transition from 0 to 1 or the first passage enabling the correct answer.
DN where EM(p;) 0 and EM(p;-1) 1. Def- inite negative passages cause transition from a correct prediction to an incorrect one.
SP and SN are semi-posittive/negative pas- sages do not cause transition(EM((p)) EM(p: 1)). Incremental inference with these passages does not reveal their utility.
5.2 Passage Type Selection
What can be inferred from the EM pattern is when there is change in value. If there is a tran- the EM pattern, we can infer that the passage corresponding to contains least some positive information for the outcome. Con-
To assess the impacts of DN contexts along with different context types, we generate six probe pat- terss to feed into the model. These patterns consist of different combinations of the DP, SP, and SN.
sition from

---

aeatie
spired by this idea, we train three different encoders to train binary classification on DP and DN con- texts. This enables us to check whether the models are capable of distinguishing between DP and DN.
Maxt Patan
2) Iakkousepuu put
Experiments
Paraaee
6.1 Probe-based Selection Inference
Probe
We evaluate all 6 probing strategies from Sec- tion 5.2. Initially we add padding passages and fix the context size to 100 during the inference. For probe 3, showing the highest EM score, we further experiment with varying context sizes of 5, 10, 20 and 40 on each DPR-retrieved test set. The remaining experimental setup follows Section
Prbe?
F
Ps
P2
Figure 4: Context types and Probe Methods. Definite positive/negative occur transitions of the EM pattern.
6.2 Attention Analysis
We also compare the effect of the IZ in our probe patterns too. In total, there are six probe patterns, illustrated in Figure 4. The most relevant approach ours (Asai et al., 2021) only performs leave- one-out generation: a passage is considered to be positive if the model fails to generate the correct inference without the passage, This approach aids identification of positive passages but does not con- sider which information can be detrimental to model. Further elaboration on the distinctions with methodology is provided in Appendix A.2.
Following the approach by Izacard and Grave 2022, we calculate the attsnttttnttonn score by averaging across heads and layers of input tokens on the first output token. We conduct the experiment on top- 20 DPR-retrieved contexts on NQ dev set. Next, visualize the distribution of DP, DN, and other passage types using density plot. We perform inference on the same dataset using passages with attention scores at various threshold values: 0.025, 0.05, 0.075, 0.1, and 0.2. We only consider con- texts with an attention score higher than the thresh- old. If no context meets this criterion, we use the entire candidate passages instead.
5.3 Model Analysis
Section 5.2, we classify context types to assess their influence on the outcomes. In this section, we analyze DP and DN using two existing methods to check for distinct signals or patterns.
6.3 Binary Classification
We train binary classifier on the DP and DN contexts. To prepare the data, we split the DPR- retrieved NQ dev set into 4:1 ratio, resulting in 7005 instances for training and 1752 instances for evaluation. From these instances, we extract 5205 DPs and 1516 DNs for training, while 1308 DPs and 396 DNs are used for evaluation. We make pre- diction using the question, title and context compar- ing RoBERTa-large and T5-large encoder initial- ized with the huggingface repository checkpoints. We additionally compare using FiD-large encoder, which was pre-trained DPR-retrieved NQ train datasets. For hyperparameters and settings, we use a batch size of 64, learning rate of 5e-5, 3 epochs, and implement the fine-tuning process using the Huggingface transformer library
Attention Score Analysis Attention scores have been extensively studied to enhance the perfor- mance of both readers and retrievers. For instance, Xu et al. 2021 employ cross-attention mechanisms to improve the performance of extractive readers. While Izacard and Grave 2022 use attention scores measure of passage contri-
from the reader bution. Building on this insight, we compare the attention scores of DP and DN passages from the reader model.
Binary Classification Re-ranking is commonly used for enhancing the performance of the reader in the retrieve-then-read pipeline (lyer al., 2021; Kongyoung et al., 2022). Additionally, Glass et al. 2022 demonstrates that a pacsaage re- re-ranking (classification) can improve the final outcome. In-
"https:/hhugiingaace http:///gg..c.eeooooo/etttnnnron

---

Datasel Retriever
EM@ 100
AcEM@ 100
Probe 3
Natural Questions
DPR
52.5
62.3
61.8
0.65
NQ
SEAL
50.0
59.4
$2.9
Contriever
50.7
53.0
0.60
DPR
72.3
77.7
77.6
TQA
SEAL
67.1
72.3
72.5
Contriever
69.7
75.5
72.5
Table 2: Exact Match scores using different retrievers on Natural Questions and TriviaQA development sets.
AcEM
Probe
0.35
7 Results and Analysis
20
40
60
80
100
Number of Context
7.1 Probe-based Selection Inference
Figure 5: Probe with varying size of input passages compared to incremental inference DPR retrieved passages on Natural Questions test set.
Using the probing patterns established in Sec- tion 5.2, achieve significantly higher EM scores with fewer passages by only using the positive- leaning passages (Table 2). Removing DN and SN clearly demonstrates redudancy of these pas- sages where E 100 for probe 3 approaches AcEM@ 100. Scores for all other probing patterns are reported in Appendix A. Comparison be- tween the other probe types indicates that the model is typically not confident with IZ, and it is helpful retain this information rather than discard it.
TrvaQA
0.75
0.70
AcEM
Few-sentence prediction We apply Probe vary- ing the number of passages from 5, 10, 20, 40 and 100 to evaluate whether we can attain higher accu- racy while using much smaller budget of passages for the reader model. With 5 passages, the model attains more than 12% increase in EM@5 for NQ and 8% over EM 5 for TriviaQA the test set. Results reported in Figure 5 and Figure 6.
20
40
60
80
100
Number of Contectt
Figure 6: Probe with varying size of input passages compared to incremental inference on DPR retrieved passages on TriviaQA test set.
Deraily plot of atlantion
While models can attain higher performance by adding more passages to their input, this comes at considerable computational costs, leading to in- creased memory usage and latency in prediction. Our results indicate that with just using 5 passages, state of the art EM scores can be exceeded with judi- cious filtering of the retrieved passages. is crucial that retrieval models not only consider relevance for the end-user but also consider how irrelevant passages can cause a catastrophic interaction with the downstream reader model.
Figure 7: Attention density plot of top-20 passages in NQ development set. DP contexts present relatively high attention scores but cannot easily be disentangled.
7.2 Attention Inference
Figure 7 visualizes the distribution of DP, DN, and other passage types, showing more dispersed and higher attention score distribution of DP, which aligns with the previous finding (Izacard and Grave, 2022). Interestingly, however, focusing on high at- tention scores results in a monotonic decline in performance, as shown in Table 3. This implies
that passages with high attention scores do not al- ways guarantee the correct answer. In addition, we investigate the highest attention score in the pres- ence of DN. Table 4 illustrates DP accounts for 40 of the highest attention while DN only represents 26%. However, 67 % of transitioned answers are lo- cated within DN, while only 13 % are attributed to

---

Attention threshold
Exact Match
Model
Acc
Pre
Re
F-1
0.025
49.5
RoBERTAa (Large)
0.76
0.76
1.0
0.76
0.05
49.11
FiD Ficceder (Lagge
0.74
0.77
0.94
0.85
0.075
48.30
Lncoder(Large)
0.1
47.55
0,2
46.59
Table 5: Binary classification results of different models on the development All three models are highly likey to predict DPs.
2c
58,11
EM
50.21
Table 3; EM score of various threshold values top-20 passages. In cases where none contexts met the threshold values, all top-20 passages were used.
ods exhibited either lower or equal performance compared to EM@ 100, falling significantly below Probe 3. Both ablations underscore the challenge in models for predicting EM patterns and passage types, reflecting the difficulty in distinguishing DP and DN within the retrieved contexts.
Feature
Context Types
Percentage
DP
40.6
Highest Atlention Score
DN
26.3
DP
13.6
Transformed Prediction
DN
67.7
Table 4: Out of DN 1091 cases, 40.6 % DP still exhibits the highest attentions score, but 67.7 % of transitioned predictions comes from DN.
8.2 Qualitative Analysis on Definite Negative
Results from Section 7.1 indicate that removing DN and SN passage types can significantly enhance the FM score. However, this approach relies on the availability of known answers to identify the damaging passage types. To discern patterns in the transition of EM patterns, we manually exam- ine instances containing DN passages. We sample 100 examples for NQ development set and discover that out these instances, cases identified as correct answers, misled by dataset issues such as Equivalent answers, where answers that are se- mantically equivalent but have variations on sur- face form (c.g. alternative spellings), Alternative answer where only one answer out of set is la- belled in the dataset (e.g. multiple actors play character), and Temporality where the correct an- swer depends on the current context (e.g. questions asking about latest events).
DP, further highlighting an ambiguous relationship between attention and answers.
7.3 Binary Inference
We observe that it is challenging for models to differentiate between damaging contexts and pos- itive ones. Table 5 presents the results binary classifications. No model trained well: all models exhibit a high recall rate at the expense of preci- sion, suggesting strong tendency to erroncously classify most passages as DPs. We conjecture that the retrieved passagess already demonstrate a high relevance score over the query, model cannot differentiate them properly.
8 Ablations
This finding supports Bulian et al. 2022 that the EM may not fully capture the impact of issues like Equivalent answer. The remaining 49 cases are indeed affected by DN. This analysis highlights the limitations of inferring damaging contexts without answers. For more details and examples of qualita- tive analysis on DN, please refer to Appendix A.3.
8.1 Passage Type Classification
Binary classification of passage type DP or DN (Section 7.3) was not infomrative. We further con- sider passage classification fication on all context types and evaluate its end- to-end performance. We use the same models and datasets as in Section 6.3, resulting in 700,500 and 175,200 instances for training and evaluation, re- spectively. For the end-to-end evaluation, we em- ploy the NQ test set. All training is performed with a batch size of 64, learning rate of 5e-5, 11,000 steps (upproximatcly 1 epoch). Multi-class classification results indicate that all models strug- gle with distinguishing between DP and DN, fa- voring predictions of SP, which constitutes 50% the instances. Detailed results are reported in Appendix A.4.1 which show that all probe meth-
multi-class classi-
8.3 Semantically Equivalent Answers
Contemporaneous work from Kamalloo et al. 2023 asserts the limitation of a lexical matching sys- tem (EM score) that leads to the underestima- tion of Reader's performance. They demonstrate that prompting LLM assess the final outcome yields similar results to manual evaluation. Inspired by this, we re-evaluate our results via zero-shot prompting, following the methodology outlined in Kamalloo et al. 2023. We calculate adjusted

---

Scores with EM variations
with Veryinn
varying
0.7
0.52
0.6
0.05
0.50
0.10
0.50
0.55
0.72
0.2
0.45
0.50
6.65
Seb
0.4
S0zM
5654
0.25
20
Number of Contest
40
23
40
Orniat
60
00
100
humber Contest
Number Conteet
(a) gpt-3.5-turbo
(b) Claude2
(a) gpt-3,5-turbo
(b) Claude2
Figure 8: Comparison of different metrics NQ dev subsets, comprising 1752 instances. SeEM and AcEM are adjusted versions of EMM, correcting semantically equivalent answers via LLM prompting.
Figure 9: Probe3 evaluations with varying sizes(5, 10. 20, 40, 100) on NQ dev subsets. Probe3 can achieve a performance close to the SeAcEM@ @ 100 with 20 times fewer contexts.
EM and AcEM, which refer to as SeEM(Semanticc Equivalent EM) and SeenaiMs Equivalent AcEM), to assess the damaging effects. Our exper- iments are conducted on subset of NQ dev set (1752 instances) using Claude2 and gpt-3.5-turbo (detailed in Appendíx A.5).
shortcomings in evaluating QA with exact match, we demonstrate that by filtering passages, models can achieve 10% higher EM scores using subsets of context that 20X times smaller.
10 Limitations
Figure 8 reports the results of SeEM and SeAcEM in comparison to EM and AcEM. In- crease performance were observed for both Claude2 and gpt-3.5-turbo to evaluate semantic equivalence. There was a 25.6% increase in the gpt- 3.5-turbo setting (EM @ 100: 0.520 to SeEM@ 100: 0.776) and an 8.9% increase in the Claude2 setting (EM@ @ 100: 0.520 to SeEM @ 100: 0.609). How- ever, discrepancies of 8.8% and 9.1 1% persist be- tween SeAcEM@ 100 and SeEM @ 100 in both set- tings, indicating that damaging effects still remain despite the semantic equivalent adjustments. To evaluate the effectiveness of our Probe3, we apply the same procedure outlined in Section 7.1. The results reported in Figure 9 demonstrate a 5.8% and 5.7% increase Probe3@* @5 (0.834, 0.666) compared the conventional approach SeEM@ @ 100 (0.776, 0.609). Notably, this improvement is achieved us- ing 1/20th of the context. This result illustrates the persistent presence of damaging passages even after adjusting for semantic equivalence, emphasiz- ing the need for filtering out damaging passages.
Identifying the behavior of black-box models is challenging. While we identify different subsets of evidence that the model reacts well to when gen- erating correct answer, there is no guarantee that these subsets of information correspond to what humans users would consider useful. Furthermore, some of the reasons the model changed its predic- tion, such as generating more specific answer, would be correct multiple references were avail- able for evaluating the models. However, these alternative answers are not available in the datasets, which means we are optimizing the models for limited subset of truly valid answers. Lastly, our approach may not be practical for decoder-only LLMs where the order of context/answer choices varies the outcome, To assess the answerability of the given n candidate contexts, (n!) inferences are required for LLMs, while FiD only needs one inference due to its inder-invarrance property.
We report a limitation, evaluation and position of established modelling techniques that can help guide the community for future research. If models can effectively leverage external information, they should be capable of using text as interpretable source of information rather than relying solely on knowledge that is stored within inaccessible model parameters. This approach may contribute to a future with NLP models that are more interpretable and controllable.
9 Conclusions
The reader models in retrieve-then-ree pipelines are sensitive the retrieved contexts when gen- erating answers. Damaging passages in this set can lead to incorrect responses. Filtering damaging passages results in increases in EM scores without the need for architectural modifications. Despite

---

11 Acknowledgements
Michael Glass, Gactano Rossiello, Md Faisal Mahbub Chowdhury, Ankita Naik, Pengshan Cai, and Alfio Gliozzo. 2022. Re2G: Retrieve, rerank, generate. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computa- tional Linguistics: Human Language Technologies, pages 2701-2715, 15, Seattle, United States. Association for Computational Linguistics.
This work supported by Institute of Informa- tion & communications Technology Planning & Evaluation (ITT) grant funded by the Korea gov- ernment (MSIT) (No. 2019-0-00075, Artificial In- telligence Graduate School Program (KAIST)) and Artificial intelligence industrial convergence clus- ter development project funded by the Ministry of Science and ICT(MSIT, Korea) & Gwangju Metropolitan City.
Sebastian Hofstätter, Jiecao Chen, Karthik Raman, and Hamed Zamani. 2022. Fid-light: Efficient and ef- fective retrieval-augmeente text generation. arXiv preprint arXiv:2209, 220..990..
Srinivasan Iyer, Sewon Min, Yashar Mehdad, and Wen- tau Yih. 2021. Reconsider: Re-ranking using span- focused cross-attention for open domain question answering. In NAACL.
References
Akari Asai, Matt Gardner, and Hannaneh Ha- jishirzi. 2021. Evidentiality-guiee generation for knowledge-inteniive NLP tasks. CoRR, abs/2112.08688..
Gautier Izacard, Mathilde Caron, Lucas Hosseini, Se- bastian Riedel, Piotr Bojanowski, Armand Joulin, and Edouard Grave 2021. Unsupervised dense infor- mation retrieval with contrastive learning.
Michele Bevilacqua, Giuseppe Ottaviano, Patrick Lewis, Wen tau Yih, Sebastian Riedel, and Fabio Petroni. 2022. Autoregressive search engines: Generating substrings as document identifiers. In arXiv pre-print 2204. 10628.
Gautier Izacard and Edouard Grave. 2020. Leveraging passage retrieval with generative models for open domain question answering.
Gautier Izacard and Edouard Grave. 2022. Distilling knowledge from reader to retriever for question swering.
Jannis Bulian, Christian Buck, Wojciech Gajewski, Ben- jamin Börschinger, and Tal Schuster. 2022. Tomayto, tomahto. beyond token-level answer equivalence for question answering evaluation. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 291-305, Abu Dhabi, United Arab Emirates. Association for Computa- tional Linguistics.
Gautier Izacard, Patrick Lewis, Maria Lomeli, Lu- cas Hosseini, Fabio Petroni, Timo Schick, Jane Dwivedi- Yu. Armand Joulin, Sebastian Riedel, and Edouard Grave. 2022. Few-shot learning with re- trieval augmented language models. arXiv preprint 0rX1v:2208.03299.
Nicola De Cao, Gautier Izacard, Sebastian Riedel, and Fabio Petroni. 2020. Autoregressive entity retrieval. CoRR, abs/2010.009044
Yichen Jiang, Shikha Bordia, Zheng Zhong, Charles Dognin, Maneesh Singh, and Mohit Bansal. 2020. Ho A dataset for many-hop fact extraction and claim verification. In Findings of the Association for Computational Linguistics: EMNLP 2020, pages 3441-3460, Online. Association for Computational Linguistics.
Danqi Chen, Adam Fisch, Jason Weston, and Antoine Bordes. 2017. Reading Wikipedia answer open- domain questions. In Proceedings of the 55th Annual Meeting of the Association for Computational Lin- guistics (Volume 1: Long Papers), pages 1870-1879, Vancouver, Canada. Association for Computational Linguistics.
Mandar Joshi, Eunsol Choi, Daniel Weld, and Luke Zettlemoyer. 2017. TriviaQA: A large scale distantly supervised challenge dataset for reading comprehen- sion. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Vol- une I: Long Papers), pages 1601-1611, Vancouver, Canada. Association for Computational Linguistics.
Michiel de Jong, Yury Zemlyanskiy, Joshua Ainslie, Nicholas FitzGerald, Sumit Sanghai, Fei Sha, and William Cohen. 2022. Fido: Fusion-in-decoder opti- mized for stronger performance and faster inference.
Ehsan Kamalloo, Nouha Dziri, Charles Clarke, and Davood Rafiei. 2023. Evaluating open-domain ques- tion answering in the era of large language models. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguisties (Volume I: Long Papers). pages 5591-5606, Toronto, Canada. Association for Computational Linguisties.
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. BERT: pre-training of deep bidirectional transformers for language under- standing. CoRR, Abs/1880000800
Marco Ferrante, Nicola Ferro, and Silvia Pontarollo. 2018. A general theory of ir evaluation measures. IEEE Transactions on Knowledge and Data Engi- neering, 42224409-42..
Vladimir Karpukhin, Barlas Oguz, Sewon Min, Patrick Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, and

---

Wen-tau Yih. 2020. Dense passage retrieval for open- domain question answering. In Proceedings of the 2020 Conference Empirical Methods in Natural Language Processing (EMNLP), pages 6769-6781, Online. Association for Computational Linguistics.
Henderson, Qian Huang, Ryan Chi, Sang Michael Xie. Shibani Santurkar, Surya Ganguli. Tatsunori Hashimoto, Thomas Icard, Tianyi Zhang, Vishrav Chaudhary, William Wang. Xuechen Li, Yifan Mai, Yuhui Zhang, and Yuta Koreeda. 2022. Holistic eval- uation of language models.
Akhil Kedia, Mohd Abbas Zaidi, and Haejun Lee. 2022. Fie: Building global probability space by leverag- ing early fusion in encoder for open-domain question answering.
Nelson F. Liu, Kevin Lin, John Hewitt, Ashwin Paran- jape, Michele Bevilacqua, Fabio Petroni, and Percy Liang. 2023, Lost in the middle: How language models use long contexts.
Sarawoot Kongyoung, Craig Macdonald, and Iadh Ou- nis. 2022. monoQA: Multi-task learning of reranking and answer extraction for open-retrieval conversa- tional question answering. In Proceedings of the 2022 Conference on Empirical Methods Natu- ral Language Processing, pages 7207-7218, Abu Dhabi, United Arab Emirates. Association for Com- putational Linguistics.
Xiaozhong Lyu, Stefan Grafberger, Samantha Biegel, Shaopeng Wei, Meng Cao, Scbastian Schelter, and Ce Zhang. 2023. Improving large language models via data importance learning.
Fabio Petroni, Aleksandra Piktus, Angela Fan, Patrick Lewis, Majid Yazdani, Nicola De Cao, James Thorne, Yacine Jernite, Vladimir Karpukhin, Jean Maillard, Vassilis Plachouras, Tim Rocktäschel, and Sebastian Riedel. 2021. KILT: benchmark for knowledge intensive language tasks. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 2523-2544 Online. Association for Computational Linguistics.
Amrith Krishna, Sebastian Riedel, and Andreas Vlachos. 2021. Proofver: Natural logic theorem proving for fact verification. CoRR, abs/2108. 11357.
Tom Kwiatkowski, Jennimaria Palomaki, Olivia Red- field, Michael Collins, Ankur Parikh, Chris Alberti, Danielle Epstein, Illia Polosukhin, Jacob Devlin, Ken- ton Lee, Kristina Toutanova, Llion Jones, Matthew Kelcey, Ming-Wi Ming-Wei Wei Chang, Andrew M. Dai, Jakob Uszkoreit, Quoc Le. and Slav Petrov, 2019. Natu- ral questions: A benchmark for question answering research. Transactions of the Association for Compu- tational Linguistics, 7:452-466.
Colin Raffel, Noam Shazeer. Adam Roberts. Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. 2019, Exploring the limits of transfer learning with a unified text-to-text trans- former. CORR, abs/19 1910..060....
Kenton Lee, Ming-Wei Chang, and Kristina Toutanova. 2019. Latent retrieval for weakly supervised open domain question answering. CoRR, Abs/1906.00300
Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. 2016. SQuAD: 100,000+ questions for machine comprehension of text. In Proceedings of the 2016 Conference on Empirical Methods in Natu- ral Language Processing, pages 2383-2392, Austin, Texas, Association for Computational Linguistics.
Omer Levy, Minjoon Sco, Eunsol Choi, and Luke Zetlemoyer. 2017. Zero-shot relation extraction via reading comprehension. In Proceedings of the s1 Conference Computational Natural Language Learning (CoNLL 2017). pages 333-342, Vancouver, Canada. Association for Computational Linguistics.
Artsiom Sauchuk. James Thore,, Alon Halevy, Nicola Tonellotto, and Fabrizio Silvestri. 2022. On the role of relevance in natural language processing tasks. In Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Infor- mation Retrieval, SIGIR '22, page 1785-1789, New York, NY, USA. Association for Computing Machin- ery.
Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Hein- rich Küttler, Mike Lewis, Wen-tau Yih, Tim Rock- täschel, Sebastian Riedel, and Douwe Kiela, 2020. Retrieval-augmented generation for knowledge- intensive nlp tasks. In Advances in Neural Infor- mation P Processing Systems. volume 33, pages 9459 9474. Curran Associates, Inc.
Egidio Terra and Robert Warren. 2005. Poison pills: Harmful relevant documents in feedback. In Pro- ceedings of the 14th ACM International Conference on Information and Knowledge Management, CIKM *05, page 319-320, New York, NY, USA. Association for Computing Machinery.
Percy Liang, Rishi Bommasani, Tony Lee, Dimitris Tsipras, Dilara Soylu, Michihiro Yasunaga, Yian Zhang. Deepak Narayanan. Yuhuai Wu, Ananya Ku- mar, Benjamin Newman, Binhang Yuan, Bobby Yan, Ce Zhang, Christian Cosgrove, Christopher D. Man- ning, Christopher Ré, Diana Acosta-Navas Drew A. Hudson, Eric Zelikman, Esin Durmus, Faisal Lad- hak, Frieda Rong, Hongyu Ren, Huaxiu Yao, Jue Wang, Keshav Santhanam, Laurel Oir, Lucia Zheng, Mert Yuksekgonul, Mirac Suzgun, Nathan Kim, Neel Guha, Niladri Chatterji, Omar Khattab, Peter
Nandan Thakur, Nils Reimers, Andreas Rücklé, Ab- hishek Srivastava, and Iryna Gurevych. 2021. Beir: A heterogenous benchmark for zero-shot evalua- tion of information retrieval models. arXiv preprint arXiv:2104.00 00.00663.
James Thorne, Andreas Vlachos, Christos Christodoolopoulls, and Arpit Mittal, 2018.

---

FEVER: a large-scale dataset for fact extraction and VERification. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Volume I (Long Pupers), pages 809-819, New Orleans, Louisiana. Association for Computational Linguistics.
Peng Xu, Davis Liang, Zhiheng Huang, and Bing Xi- ang. 2021. Attention-guicl guided generative models for extractive question answering.
Zhilin Yang, Peng Qi, Saizheng Zhang. Yoshua Bengio, William Cohen, Ruslan Salakhutdinov, and Christo- pher D. Manning. 2018. HotpotQA: A dataset for diverse, explainable multi-hop question answering. Proceedings of the 2018 Conference on Empiri- cal Methods in Natural Language Processing, pages Brussels, Belgium. Association for Com- putational Linguistics.
Wenhao Yu, Dan Iter, Shuohang Wang, Yichong Xu. Mingxuan Ju, Soumya Sanyal, Chenguang Zhu, Michael Zeng, and Meng Jiang, 2023. Generate rather than retrieve: Large language models are strong context generators. In International Confer- for Learning Representation (ICLR).
Chujie Zheng. Hao Zhou, Fandong Meng, Jie Zhou, and Minlie Huang. 2023. On large language models' selection bias in multi-choice questions.

---

A Appendix
A.1 Probe pattern performance
We report the performance of all probe patterns from Section 5.1. Probe pattern 3 gives EM scores that are closest to the established AcEM @ 100 upper bound.
Dataset
Retriever
EM@ 100
AcEM 100
Probe
Probe 2
Probe 3
Probe
Probe 5
Probe 6
DPR
52.5
62.3
58.1
56.3
61.8
61.7
55.2
56.0
NQ
SEAL
50.0
59.4
43.9
30.0
52.9
44,9
$2.5
44,6
Contriever
50.7
60.8
44.3
29.0
53.0
45.1
52.6
44.4
DPR
72.3
77.7
76.2
75.1
77.6
77.4
73.8
74.0
TQA
SEAL
67.1
72.3
67.9
51.5
72.5
63.7
72.3
63.9
Contriever
69.7
75.5
68.4
51.7
72.5
62.9
72.3
62.9
Table Exact Match scores using different retrievers on Natural Questions and TriviaQA development sets, The datasets retrieved by DPR exhibit consistently high EM score across various Probes. This observation can be explained by the fact that the FiD models used for evaluation are trained on DPR retrieved datasets.
A.2 Difference between leave-one-out masking and EM pattern
There are two key distinctions between Asai et al. 2021 and our apporach. Firstly, while leave-one- out masking focuses on finding positive contexts, ours metsaddeassage Type Selection) aims to detect detrimental passages that can negatively impact the inference and it can also distinguish positive contexts effectively. Secondly, the time taken for inference to determine whether passage is positive negative is faster in our approach compared leave-one-out masking. For instance, let N be the number of contexts, G denote the reader model, and P (p1,P2. ... Pv- denote the set of retrieved passages for a given query. When new context, PNv is added the passages lists, requires N inferences over N-1 passages in the case of leave-one-out masking, whereas ours requires a single inference over N passages.
A.3 Qualitative Analysis
In this section, we delve further into the qualitative analysis of DN. We focus on the top-20 passages as examining the top-100 passages would require more extensive investigation. Out of the 8,757 cases in the NQ development set, 1,091 instances exhibit the presence of DN, accounting for approximately 12% of the dataset. We selected sample of the first 100 cases and discovered that out of these instances, 51 DN cases were attributed limitations within the dataset, which can be regarded false negative cases. Other 49 cases are true-negative cases and occurred due the presence of DN. Regarding the limitations within the dataset, we classify them into three distinct types: Equivalent Answer Example, Alternative Answer, and Temporality.
A.3.1 Dataset Issue
Dataset issues exemplify situations where the predicted answers are accurate, yet the evaluation is insufficient. Prediction within the context is highlighted in red, while the supporting information is marked in blue.
Equivalent Answer Example
There are 34 cases where predictions have different formats or are supersets/subsets of gold answers.
Query: when did the king kong ride burn down EM pattern: 010000000000000000000
Gold Answers: ['2008']
DN index:
Prediction June 1, 2008
Predicted answer in DN context: Yes
DN context

---

('id": "4215890", "title": "King Kong", 'text': "and Universal Orlando Resort in Orlando, Florida. The first King Kong attraction was called King Kong Encounter and was part of the Studio Tour at Universal Studios Hollywood. Based upon the 1976 film "King Kong", the tour took the guests in the world 1976 New York City, where Kong was seen wreaking havoc on the city. was opened on June 14, 1986 and was destroyed on June 2008 in major fire. Universal opened replacement 3D King Kong ride called "" that opened on July 1, 2010, based upon Peter Jacksons 2005 film "King Kong".)
Alternative Answer Example
There are 13 cases in which the predictions can serve as alternative answers for the given query
Query: who introduced the system of civil services in india
EM pattern: 10011111111100000
Gold Answers: ('Charles Cornwallis"]
DN index: 12
Prediction Warren Hastings
Predicted answer in DN context: No
Context index including prediction: Context
('id': '14394957', 'title': 'Civil Services India', 'text': "administer them. The civil service system in India is rank-based and does not follow the tenets of the position-based civil services. In 2015, the Government of India approved the formation of Indian Skill Development Service. Further, in 2016, 6, the Government of India approved the formation of Indian Enterprise Development Service. Warren Hastings laid the foundation of civil service and Charles Cornwallis reformed, modernised and rationalised it. Hence, Charles Cornwallis is known as the 'Father of Civil Service in India'. He introduced Covenanted Civil Services (Higher Civil Services) and Uncovenanted Civil Services (Lower Civil Services). The present civil services India")
Temporality Example
There are cases where queries ask for the up-to-date context but generate the answer on the outdated context
Query: who plays cat in beauty and the beast
EM pattern: 011111111111
Gold Answers: ['Kristin Kreuk']
DN index: 19
Prediction Linda Carroll Hamilton
Predicted answer in DN context: No
Context index including prediction: 6
Context
('id': '728625', 'title": 'Linda Hamilton', 'text': 'Linda Hamilton Linda Carroll Hamilton (born September 26, 1956) is an American actress best known for her portrayal of Sarah Connor in "The Terminator" film series and Catherine Chandler in the television series "Beauty and the Beast" (1987-1990), for which she was nominated for two Golden Globe

---

Awards and Emmy Award. She also starred Vicky in the horror film "Children of the Corn". Hamilton had recurring role as Mary Elizabeth Bartowski on NBC$ "Chuck". Hamilton was born Salisbury, Maryland. Hamiltons father, Carroll Stanford Hamilton, physician, died when she was five, and her mother later married')
A.3.2 Definite Negative Cases
The presence of DN contexts destabilize the reader, leading to the conversion of correct gold answer. The prediction present within the context highlighted in red.
Predictions in DN context
In 43 cases, the predictions (transitioned from gold answers) are found within the DF passage.
Query: real name of gwen stacy in amazing spiderman
EM pattern: 11100111111111111
Gold Answers: ('Emma Stone"]
DN index: 4
Prediction Mary Jane Watson
Predicted answer in DN context: Yes
DN context
('id': '1283490', 'title": 'Gwen Stacy', 'text': 'relationship with chemical weapon devel- oper Norman Osborn. Mary Jane Watson, popular actress in this reality, played Gwen Stacy in the film adaptation of Apider-Man's 's life story. Gwen and her father read textual accounts of their deaths in the main universe, though they believe this simply to be the morbid imaginings of Peter Parker, who is suffering from mental health issues. Gwen Stacy first appeared in "Marvel Adventures Spider-Man" #53 as a new student of Midtown High. She had transferred from her previous school after the Torino Gang, powerful New York mob, began harassing her in an attempt
Predictions in previous context
There are 4 cases where the prediction appears in the previous contexts.
Query: who will get relegated from the premier league 2016/17 EM pattern: 0000010001 11 101111
Gold Answers: ("Middlesbrough', 'Sunderland', 'Hull City')
DN index: 15
Prediction Norwich City
Predicted answer in DN context: No
Context index including prediction: 7
Context
('id': '19245453', 'title": '2016-17 Premier League", "text': "the league the top sev- enteen teams from the previous scason, well as the three teams promoted from the Championship. The promoted teams were Burnley, Middlesbrough and play-off winners Hull City, who replaced Aston Villa, Norwich City and Newcastle United. West Ham United played for the first time at the London Stadium, formerly known as the Olympic Stadium. Although having a capacity of 60,010, for the first Premier League game this was limited to 57,000 due to safety fears following persistent standing by fans at West Ham's

---

Europa League game played in early August. Stoke City announced that from")
Predictions not in contexts
There are 2 cases where the prediction does exist in the candidate passages.
Query: how many episodes does riverdale season one have
EM pattern: 111100001111111110
Gold Answers: ['13']
DN index: 4
Prediction 21
Predicted answer in DN context: No
Context index including prediction: None
A.4 Passage Type Classification
A.4.1 Mutli-classffictioo
Model
Metric precision
DP
DN
SP
SN
0
0.54
0.09
0.5
FiD-Encoder
recall
0
0.72
0.04
0.37
precision
0
0.55
0.1
0.5
T5. -n-onnrder
recall
0
0.72
0.05
0.37
precision
0
0.5
0
0
RoBERTa
recall
0
0
Total Instances
1308
396
86936
15438
71122
Table The table shows the results on 175,200 evaluations instances, which are 20% of original DPR- retrieeee NQ devset. All models struggle with identifying DN and DP, favoring predictions of SP, which constitutes 50% of instances. Notably, three models can't differentiate context types properly. Among 175,200 total instances, T5 predicts I 11,6606 (65.4%) to SP, FiD labels 15,522 (65.9%) SP, and RoBERTa classifies all SP. This highlights multi-classifccation's 's challenge in capturing passage types and inability to identify DN and DP.
A.4.2 End-to-End Results
Model
Probe
Probe 2
Probe 3
Probe
Probe 5
Probe
FiD-Encoder
48.59
37.51
53.77
42.74
54.71
43.66
T5--Enoder
48.73
37.45
53.46
42.3
54.24
43.19
RoBERTa
54.43
35.84
54.43
35.84
54.43
35.84
Table 8: End-to-End result multi-classifcation on NQ test set. Among all classification models, RoBERTa achieves the highest EM scores across Probel, Probe3, and Probe5. This is due the fact that RoBERTa inferences are all SPs, resulting in all contexts being used as inputs for the reader. Consequently, these scores should match EM@ 10054.43)), which uses all contexts for inference. All Prohe results exhibit either lower equal performance compared to EM@ 100, notably falling significantly below Probe 3's EM score(65. 12)). This underscores the classification models' struggle in identifying EM patterns and passage types, reflecting the challenge in distinguishing DN and DP within the provided retrieved context.

---

A.5 Semantically Equivalent Answers
To calculate the SeEM and SeAcE, candidate answers are collected from the incremental answers. Then, we prompt spt-3.5-turbo-061. and Claude2 by iterating the candidate answer list over the gold answer list. Suppose we have an instance like this:
query "who sings does love with reba" gold_ans_lst = ['Linda Davis'] cand__st C'Linda Kaye Davis', 'Linda Davis']
In this case, EM fails to capture "Linda Kaye Davis" as the correct answer because of "Kaye" in the middle. We perform zero-shot-prompiing by iterating over gold ans 1st and cand 1st. Here is an example:
Question: who sings does he love me with reba
Answer: Linda Davis Candidate:
Linda Kaye Davis
Is candidate correct?
Although each API generates different response formats to the given prompt, we only consider cases where the response starts with explicit tokens (Yes/No); otherwise, we discard the candidate.
response: Yes, the candidate correct.
Linda Kaye Davis is the singer who performed the duet "Does He Love You" with Reba McEntire.
Finally, we construct an adjusted gold answer list based the response and calculate the SeAm and SeAcEm.
A.6 Attention Score of top-20 passges
1.0
Query: who will get relegated from the premier league 2016/17
LG0
N 0.39
0.51
m
0.8
8.10
608 6.17 E,20
0.35
0.24
8.00
5.05
0.55
0.27
0.24
Nubmer of passages
5.04
6.03
0.0
0.05
0.000
0.10
8.08
6.03
6.07
0.06
0.7
6.22 5.04
DLEW
13
0.00
9.30
8.08
6.05
0.07
0J88
0.00
9.21
DLEY
UIZZ
11
0.4
14
8.03
6.03
0.08
0.10
0.04
8:08
0.2
0.05
0.24
16
0.05
8.09
8.03
6.02
6.06
0:07
0.05
0.37
0.00
0.08
0.15
002
0.04
0.23
0.02
5.00
6.07
6.06
0.61
0:07
0127
1U.4
0.23
0.09
0.
0.00
R.03
6.0
0.3
0.04
014
5.03
0.09
002
0.4
0.2
0.09 008 002
West Bromwich Albion
Soltull Moors
sotthul Moars
solitull Moors
Solthull Moors
Hull City
Stoke City
Norwich City
Hull City
Hudi Hull City
City
Hull City
Hull City
Hull City
Narwich City
Middlesbrough h
Hull City
Hutl City
Hult City
Norwich
EM match: : answer ('Middlesbrought, "Sunderland;; 'Hull City']
Figure 10: The attention of the top 20 retrieved passages depicted in this example. It can be observed that the 7th and 16th passages are DN. Interestingly, the passage with the highest attention score(8th) is identified as SN


Reviews:
[{"review_id": "RtZaumPKLU", "reviewer": "Reviewer_RWj7", "summary": "", "questions": "Question A: Which retriever is used for the experiments presented in Section 6.2 (and Table 3)?\nQuestion B: What would be compression rate (i.e. the average percentage drop with respect to the number of used passages) that is achieved by using the five passages of probe 3 in comparison to the number of passages that are used by the upper-bound solution of Table 2?", "limitations": "", "rating": 3, "confidence": 3, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Dear Reviewer RWj7,\n\nWe sincerely appreciate your thoughtful review. We are glad that you find our attempt to filter out damaging passages laying the groundwork for better understanding the detrimental effect of some passages on FiD-based architecture.\n\n\nBelow, we have tried our best to address your questions and concerns. We have addressed each \"Reason to Reject\" and \"Question for the Authors\" as denoted by R1, R2, Q1, Q2, and so forth. If any subsections or questions are present within these, they will be referenced as R1-1, R1-2, and so on. Regarding \"Typos Grammar Style And Presentation Improvements,\" we will use T1 and T2 for reference. Please take a look at our response and let us know if further clarification is needed.\n\n**R1) I think the paper could benefit from another round of proof-reading. There are several sections that are difficult to follow.**\n\nWe value your suggestion. We are currently undergoing a proof-reading to enhance the organization based on feedback from reviewers. Our goal is to provide readers with a clearer understanding of our work.  Additionally, as suggested in the presentation improvement section, we have rephrased specific portions of the paper. These revisions can be found in T3). \n\n**R2) My understanding is that t`he solution for identifying the passage type hypothesises the existence of gold-answer data. The paper goes one-step further and highlights the challenges associated with seeking to identify harmful passages, when gold-answer data is not available. However, no concrete solutions to this problem are provided.**\n\nWe want to point out that the absence of a specific solution to the problem does not diminish the value of our work. Our objective is to emphasize this limitation and identify a problem that will drive future research. Detecting damaging passages is a challenge that demands future efforts when building more resilient open-domain NLP systems. From this, we can understand the behaviors and limitations and make improvements to the models that use retrieved evidence. \n\nOur contributions are sound and clear.\n- Contrary to conventional assumption, we introduced extra relevant passages that can degrade the performance of the model in question answering tasks.\n- We identified the existence of subsets from the retrieved passages with higher EM scores without any architectural modifications.\n- Also demonstrated the complexity of distinguishing between DP and DN, which requires further research.\n\nTo conclude, we denote to a recently published paper, [Lost in the Middle: How Language Models Use Long Contexts](https://arxiv.org/abs/2307.03172)  (Liu, Nelson F., et al.,2023), also investigating the identification of relevant passage within long inputs. This paper contains a section, ‘Is More Context Always Better?’. (This paper was published after the EMNLP submission deadline). In this section, they said NO for the question, referring to the saturation point in the Reader's accuracy with respect to the number of passages. They suggest that the reordering of retrieved documents and truncation of ranked lists are promising directions for future work. However, our research can provide different insights, filtering out damaging passages for higher inference results as well as run-time efficiency.\n\n**Q-A) Question A: Which retriever is used for the experiments presented in Section 6.2 (and Table 3)?**\n\nWe use DPR(Dense Passage Retriever) provided by the authors for the experiments presented in Section 6.2 (and Table 3). We will add this information in Section 6.2 as well as Table 3.\n\n**Q-B) What would be compression rate (i.e. the average percentage drop with respect to the number of used passages) that is achieved by using the five passages of probe 3 in comparison to the number of passages that are used by the upper-bound solution of Table 2?**\n\nWe introduce three additional columns (Probe3@5, EM@5, Compression rate) to Table 2. To enhance clarity, we replace “Lower” and “Upper” with “EM@100” and “AcEM100”, respectively. We explain each column component as follows:  \n- EM@100 refers to the highest Exact Match Score of the Reader utilizing top-100 passages \n- AcEM@100 denotes incremental inference results on the same top-100 passages. \n- Probe 3 presents the inference outcome of the selection method outlined in Figure 5. \n- Probe 3@5 denotes the inference outcome of utilizing top-5 passages in Probe 3. \n- Compression rate is derived by subtracting Probe 3@5 from AcEM100 and dividing by the number of passages used (5).\n- EM@5 represents the inference outcome using top-5 passages from the retrieved list\n\n| Dataset | Retreiver  | EM@100 | AcEM100 | Probe 3 | Probe 3@5 | EM@5  | Compression rate |   |   |\n|---------|------------|--------|---------|---------|-----------|-------|------------------|---|---|\n| NQ      | DPR        | 52.5   | 62.3    | 61.8    | 57.55     | 45.8  | 0.95             |   |   |\n|         | SEAL       | 50     | 59.4    | 52.9    | 46.21     | 38.47 | 2.63             |   |   |\n|         | Contriever | 50.7   | 60.8    | 53      | 46.09     | 40.02 | 2.94            |   |   |\n| TQA     | DPR        | 72.3   | 77.7    | 77.6    | 75.77     | 66.45 | 0.38            |   |   |\n|         | SEAL       | 67.1   | 72.3    | 72.5    | 66.48     | 55.26 | 1.16            |   |   |\n|         | Contriever | 69.7   | 75.5    | 75.5    | 66.56     | 60.14 | 2.78             |   |   |\n\nWhen evaluating the NQ dataset, we utilize the FiD-large reader that was trained on the DPR-retrieved NQ training dataset. Similarly, for the TQA dataset, we employ the FiD-large reader trained on the DPR-retrieved TQA training dataset. \n\n**T1), T4), T5), T6) Typos**\n\n- T1) \"put\" in line 59 should be replaced with \"but\". \n    -  Right.\n- T4)  Something (maybe \"passages\") is missing from line 295 of Section 3.2, after \"top-K\". \n    - Right. \"Passages\" is missing.\n- T5) The legend of Figure 4 refers to a non-existent type of passage (i.e. \"DF\"). Should it be replaced by \"DP\"? \n    - Yes, it should be replaced by \"DP\".\n- T6) Some captions, such as those of Figure 3 and 4, are also lacking proper punctuation. \n    - We will keep proper punctuation.\n\n\nWe really appreciate your pointing out typos. \n\n**T2) The order of the figures is not consistent with the train-of-thought presented in the main body of the manuscript. For instance, Figure 1 (located at page 1) is first referred to, in the main body, at the end of page 4, while Figure 2 (located at page 4) at the middle of page 3.** \n\nT2) Figure 1 is located on page 1 to give readers a general overview of our paper. We will reference Figure 1 in the introduction section on page 1 to enhance its clarity. As for the remaining figures and tables, we will ensure they are appropriately aligned with their respective reference points. We appreciate your valuable suggestions.\n\n**T3) There are parts of the paper that could benefit from rephrasing. For instance: lines 76--85 and lines 86--87 from introduction and lines 234--240 from Section 3.1.**\n\n**T3-1) We have rephrased lines 76 ~ 85 to emphasize our contributions and findings**\n\n[Preivous] Identifying context passages that are detrimental to the performance of the model, and removing them illustrates performance bounds for the models under ideal conditions showing how different subsets of the retrieved evidence result in up to a 10% improvement in exact match score: this exceeds the performance gains from contemporaneous works proposing extensions to FiD such as FiD-light and fusion in encoder \n\n[Rephrased] In summary, our findings challenge the conventional assumption that more passages lead to higher performance. In fact, introducing additional relevant context can actually worsen the performance of the model in question answering. By excluding passages that have a detrimental effect, we observe up to 10% improvement in the exact match score under ideal conditions. This enhancement is achieved by identifying subsets within the retrieved passages that yield higher EM scores without requiring any architectural modifications. Notably, these gains surpass the performance improvements seen in recent works that propose FiD extensions like FiD-light and fusion in encoder.\n\n\n**T3-2) We have rephrased lines 86~87 to emphasize our findings**\n\n[Preivous] Even though models have access to a sufficient set of information to generate the answer, the a superset of this information is detrimental.\n\n[Rephrased] Our contributions reveal how to leverage the retrieved passages. This involves not only identifying pertinent passages but also excluding harmful ones, highlighting the importance of filtering out damaging passages.\n\n**T3-3) We have rephrased lines 234--240 from Section 3.1 to emphasize our contributions in a more concise manner down below**\n\n[Preivous] Given this perspective, our primary focus and contribution for this paper is on removing passages that may harm the overall accuracy of the system, but rather than simulating these from a different retrieval model, our goal is to identify these from the same retrieval model used for the full pipeline rather than simulating them.\n\n[Reprhased] One of the contributions in this paper is the removal of damaging passages, which can undermine the accuracy of the system in open-domain question answering tasks where damaging effects have not been thoroughly investigated. Ultimately, our objective is to validate the presence of detrimental effects within the complete open-domain pipeline rather than relying solely on simulations.\n\nWe will rephrase parts of the paper if needed to give a clear understanding and explanation to readers.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- Interesting methodology for identifying different passage types along with the relevant probe patterns (i.e. combinations of different passage types that will be used as input to the reader model).\n- The domain of the work is interesting. The paper lays the groundwork for understanding better the detrimental effect that some passages can have on the performance of FiD-based architectures.\n- Different insights and solutions are explored for seeking to identify automatically passages whose inclusion in the reader model's input would be harmful for the final answer.", "reasons_to_reject": "- I think the paper could benefit from another round of proof-reading. There are several sections that are difficult to follow.\n- My understanding is that the solution for identifying the passage type hypothesises the existence of gold-answer data. The paper goes one-step further and highlights the challenges associated with seeking to identify harmful passages, when gold-answer data is not available. However, no concrete solutions to this problem are provided.", "questions_to_authors": "Question A: Which retriever is used for the experiments presented in Section 6.2 (and Table 3)?\nQuestion B: What would be compression rate (i.e. the average percentage drop with respect to the number of used passages) that is achieved by using the five passages of probe 3 in comparison to the number of passages that are used by the upper-bound solution of Table 2?", "paper_topic_and_main_contributions": "The paper focuses on the open-domain QA task. In particular, the extent to which the performance of architectures based on the Fusion-in-Decoder (FiD) is impacted by the relevance and the number of the input passages, which are provided by a retriever model. The authors provide an analysis about the conditions under which the resulting answer from an FiD-based architecture can be improved should any harmful passages (i.e. passages whose inclusion would be harmful for the Exact Match score) are omitted from the reader model's input. Furthermore, they explore different approaches for identifying harmful passages dynamically (i.e. by using either a binary classifier or heuristics based on the reader model's attention scores), without requiring hypothesising the existence of gold-answer data.", "excitement": 3, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {"Typos_Grammar_Style_and_Presentation_Improvements": "- \"put\" in line 59 should be replaced with \"but\".\n- The order of the figures is not consistent with the train-of-thought presented in the main body of the manuscript. For instance, Figure 1 (located at page 1) is first referred to, in the main body, at the end of page 4, while Figure 2 (located at page 4) at the middle of page 3. \n- There are parts of the paper that could benefit from rephrasing. For instance: lines 76--85 and lines 86--87 from introduction and lines 234--240 from Section 3.1.\n- Something (maybe \"passages\") is missing from line 295 of Section 3.2, after \"top-K\".\n- The legend of Figure 4 refers to a non-existent type of passage (i.e. \"DF\"). Should it be replaced by \"DP\"?\n- Some captions, such as those of Figure 3 and 4, are also lacking proper punctuation."}}, {"review_id": "CdGe8KEG1f", "reviewer": "Reviewer_5NS3", "summary": "", "questions": "Please see Reasons to Reject", "limitations": "", "rating": 3, "confidence": 4, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Dear Reviewer, 5NS3\n\nThank you for the time to review our paper thoroughly. We deeply appreciate how you found our approach to identifying relevant evidence significant, which currently does not fully leverage the retrieved passages.\n\nBelow, we have tried our best to address your questions and concerns. In our rebuttal, we have addressed each \"Reason to Reject\" and \"Question for the Authors\" as denoted by R1, R2, Q1, Q2, and so forth. If any subsections or questions are present within these, they will be referenced as R1-1, R1-2, and so on. Please take a look at our response and let us know if further clarification is needed.\n\n**R1) The authors propose a simple approach for evidence selection from the retrieved passages and the novel contributions are rather limited. While some analysis in the paper is interesting, in my opinion the main points could de describe in a short paper.\n.**\n\nThe contribution of the paper isn't just the methodology. We introduce a novel concept, the notion of damaging passages in the context of the question-answering task that has yet to be thoroughly explored. Thus needs space for analysis. Here is the overview of our work:\n- Introducing the knowledge-intensive NLP, Retrieval, Damaging Retrieval\n    - We introduce the domains of knowledge-intensive NLP, retrieval methodologies, and the concept of damaging retrieval.\n- Simulating damaging passages in question answering \n    - Our study simulates damaging passages within the context of question answering.\n- Demonstration of Order-Invariant FiD\n    - We underscore the order-invariant property of FiD\n- Incremental Inference using FiD\n    - Building upon the aforementioned insights, we conduct incremental inference utilizing FiD as the reader, unveiling the presence of detrimental effects.\n- Propose Selection Inference\n     - We introduce the concept of selection inference and confirm the existence of subsets within retrieved passages that exhibit higher Exact Match (EM) scores than conventional EM computed over the entire retrieved list.\n- Analysis of DP and DN\n    - Our study employs two distinct methodologies to analyze transitions triggered by DP and DN passages.\n    - Attention Score Analysis\n        - We delve into attention score analysis, a technique previously employed to enhance reader performance and elucidate that it fails to reveal transition signals effectively.\n    - Evaluation of Binary Classification for Transition Pattern\n        - We assess the efficacy of binary classification in capturing transition patterns.\n\nWe can be a bit more terse with the explanation, making more space for justifying our method, adding further analyses, and improving the organization of the paper.\n\n**R2) The approach has serious limitations, making it hard to apply to real problems. Relying on the order of retrieved passages raises questions regarding the upper bound performance as well as concerns regarding the generalization of the approach to other domains. The use of the NQ and TQ datasets is also limiting due to the fact that the answers are concise and short. It unclear how the context selection applies for long form QA.**\n\nWe address R2 in two different responses R2-1), and R2-2).\n\n**R2-1) Relying on the order of retrieved passages raises questions regarding the upper bound performance as well as concerns regarding the generalization of the approach to other domains.**\n\nYes. Our findings rely on the order of retrieved passages. There are two main reasons for utilizing the order from the retriever: 1) Top passages extracted by these retrievers are highly likely to contain information relevant to the answer by its nature, and 2) True optimal upper bound for retrieved passages is intractable. Here, we illustrate each reason in detail.\n\n\n- Recall-optimized retrievers\n\n    Retrievers are optimized for recall. This implies that passages ranked higher contain more pertinent information to the answer than those ranked lower. This aligns with the results depicted in Figures 6,7, and 8. All retrievers present a steady decrease in the increment rate of AcEM and EM with respect to the number of passages. This saturation in AcEM and EM implies that lower-ranked passages offer little information about the answer. \n\n- Intractability of the true upper bound\n\n    Determining the true upper bound is not feasible in practice. Consider a scenario with 100 retrieved passages. Let's denote k as the number of selected passages used for inference. For a single query and corresponding 100 passages, it would require 100! times k inferences for k ranging from 1 to 100  to establish the optimal upper bound. Given that our paper demonstrates the order invariant property of FiD, this could be further reduced to 100Ck for k ranging from 1 to 100 where 100Ck pertains to the selection of k passages. However, despite these considerations, this still remains practically intractable. \n    \n    On the other hand, calculating AcEM based on the retrieved order is achievable, requiring 100 inferences for a single question and retrieved passages. For our research, the computation of AcEM sets requires approximately 100 times more effort than that for the vanilla model, accounting for the subsets of evidence.\n\nBased on our findings, we demonstrate that a higher upper bound can be established given a retrieved passage through AcEM, which illustrates the limitation of the previous upper bound formed by  EM match score.\n\n**R2-2) The use of the NQ and TQ datasets is also limiting due to the fact that the answers are concise and short. It unclear how the context selection applies for long form QA.**\n\nWe disagree with the assertion that the use of NQ and TQ datasets in our findings undermines the potential future research in long form QA. Instead, our work establishes the foundation for investigating the influence of damaging passages in long-form QA, for the following reasons:\n\n- Prior works in damaging passages revolve around simple tasks, such as fact verification and query operations, where responses are specific and confined. Despite the simplicity of these tasks, the presence of relevant passages has been shown to have negative consequences.\n\n- Employment of the NQ and TQ datasets aims to identify the presence of detrimental passages in question answering tasks. Through the application of EM patterns, we substantiate the existence of damaging passages within question answering. Building upon this observation, we apply selection inference, showing the model's performance can be enhanced.\n\n- Direct application of the selection method to long-form QA might be challenging because answers are in the form of sentences, and there is no deterministic metric such as EM. However, this does not mean that the damaging passages cannot be captured. If we confine the number of contexts and their input sizes, we may perform human evaluations to verify the existence of damaging effects in long-form QA. Further, we can conduct incremental inference and qualitative analysis, as we conducted in the paper, to confirm the existence of damaging passages in long-form QA. \n\n**R3) The approach relies on using the gold answer for passage selection. This is not possible in real tasks. The use of the binary classifier to identify DP and SP passage types is not effective since it tends to classify all passages as useful (DP).**\n\nYes. The approach relies on employing the gold answer for passage selection. Nonetheless, it is worth noting that our method isn't intended as a SOTA model for benchmarking purposes. Instead, we aim to underscore this limitation and identify a problem that will inspire future research. Detecting damaging passages is a challenge that demands future efforts when building more resilient open-domain NLP systems. From this, we can understand the behaviors and limitations and improve the models that use retrieved evidence.\n\nOur contributions are sound and clear.\n\n- Contrary to conventional assumption, we verify that extra relevant passages can degrade the model's performance in question answering in a real-world setting.\n- We identified the existence of subsets from the retrieved passages that can generate higher EM scores without any architectural modifications.\n- We demonstrated the complexity of distinguishing between DP and DN, which requires further research.\n\nTo conclude, we denote to a recently published paper, [Lost in the Middle: How Language Models Use Long Contexts](https://arxiv.org/abs/2307.03172)  (Liu, Nelson F., et al.,2023), also investigating the identification of relevant passage within long inputs. This paper contains a section, ‘Is More Context Always Better?’. (This paper was published after the EMNLP submission deadline). In this section, they said NO, referring to the saturation point in the Reader's accuracy with respect to the number of passages. They suggest that reordering of retrieved documents and truncation of ranked lists are promising directions for future work. However, our research can provide different insights, filtering out damaging passages for higher inference results as well as run-time efficiency.\n\n**R4) The experimental section could be made more clear. It now reads as a sequence of findings and it is not immediately clear why some experiments are important. I had difficulty relating the passage attention as well as the binary classifier experiments to the main points of the paper. It would also be useful to specify more clearly what the lower and upper bounds are in section 6..**\n\nWe address R4 in two different responses.\n\n**R4-1) The experimental section could be made more clear. It now reads as a sequence of findings and it is not immediately clear why some experiments are important. I had difficulty relating the passage attention as well as the binary classifier experiments to the main points of the paper.**\n\nWe appreciate the feedback. To enhance clarity, we will explain the motivations and context of attention analysis and binary inference in section 4 and offer more detailed settings in section 5. Furthermore, we will clearly separate the experimental findings and further analyses. We will implement as follows:\n\n- Explain previous works and how they relate to the main points of the paper for attention analysis and binary classification\n- For the experiment section, we will articulate the experimental settings in more details\n\n**R4-2) It would also be useful to specify more clearly what the lower and upper bounds are in section 6**\n\nWe will replace the lower bounds and upper bounds with EM@100, AcEM@100. We will also articulate that EM@100 refers to the highest Exact Match Score of the Reader utilizing top-100 passages, and AcEM@100 denotes incremental inference results on the same top-100 passages in Table 2 as well.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "Finding the informative context for the open QA task is an important problem. \nThe authors show that simple approaches for identifying the evidence in the retrieved passages can be effective.", "reasons_to_reject": "1. The authors propose a simple approach for evidence selection from the retrieved passages and the novel contributions are rather limited. While some analysis in the paper is interesting, in my opinion the main points could de describe in a short paper. \n2. The approach has serious limitations, making it hard to apply to real problems. Relying on the order of retrieved passages raises questions regarding the upper bound performance as well as concerns regarding the generalization of the approach to other domains. The use of the NQ and TQ datasets is also limiting due to the fact that the answers are concise and short. It unclear how the context selection applies for long form QA. \n3. The approach relies on using the gold answer for passage selection. This is not possible in real tasks. The use of the binary classifier to identify DP and SP passage types is not effective since it tends to classify all passages as useful (DP).\n4. The experimental section could be made more clear. It now reads as a sequence of findings and it is not immediately clear why some experiments are important. I had difficulty relating the passage attention as well as the binary classifier experiments to the main points of the paper.  It would also be useful to specify more clearly what the lower and upper bounds are in section 6.1 .  ", "questions_to_authors": "Please see Reasons to Reject", "paper_topic_and_main_contributions": "This paper addresses the problem of damaging retrieval for open domain question answering. The authors propose a method for identifying the passages that are detrimental for the QA task by evaluating how different subsets of the retrieved passages change the answer. The experiments use an FiD model and the method is evaluated on the NaturalQuestions and  the TriviaQA datasets. \n\nThe authors rely on the Exact Match pattern where  increasingly longer sequences are given as a context to a generative model. The interesting passages produce changes in the EM score of the generated answers from 0 to 1 or from 1 to 0. Based on the EM pattern the passages can be classified  in 5 categories: IZ (Initial Zero), DP (Definite positive), DN(definite negatives), SP(semi-positive) and SN(semi negative).  Once the type of the passages are identified, there are 6 probing strategies that can be used to construct the context from the retrieved passages.\nThe experiments show that using the DP and SP passage types in the context improved the EM of the generation.", "excitement": 3, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "DGzaWK59vn", "reviewer": "Reviewer_LMuo", "summary": "", "questions": "1. Figure 4: DF -> DP in the legend\n2. Line 569: This finding supports ? that\n3. Why binary classification is considered instead of multi-class classification on the 5 types? Why not apply the classification results to the passage selection to see the end-to-end result? ", "limitations": "", "rating": 4, "confidence": 3, "soundness": 2, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Dear Reviewer, LMuo\n\nThank you for your appreciation of our work and insightful review. We are grateful that you find our problem, identifying damaging passages, crucial. We are also encouraged that you see the potential of our proposed insights to have an impact on future research. We hope that the final version of our paper will more clearly communicate our important findings.\n\nBelow, we have tried our best to address your questions and concerns. We have addressed each \"Reason to Reject\" and \"Question for the Authors\" as denoted by R1, R2, Q1, Q2, and so forth. If any subsections or questions are present within these, they will be referenced as R1-1, R1-2, and so on. Please take a look at our response and let us know if further clarification is needed.\n\n**R1) The writing of this paper needs improvement. First, the organization is poor. There is no introduction on why we need Selection Inference, Attention Analysis, and Binary Classification. Second, the content is not properly prioritized. A whole section (section 3 Damaging Retrieval) with more than one page is used to discuss the effect of damaging passages, which is a known phenomenon in previous works and contributes little to the introduction of the main contribution of the paper. Third, it lacks the necessary details. For example, how the binary classification works? Does it consider the question as input? Are the models initialized with the trained reader on the QA dataset?** \n\nWe address R1) into 3 responses.\n\n**R1-1) First, the organization is poor. There is no introduction on why we need Selection Inference, Attention Analysis, and Binary Classification.**\n\nWe appreciate the feedback. We'll re-organize each section and incorporate introductions and motivations behind selection inference, attention analysis, and binary classification. We will dedicate additional space in section 4 to describe these methods and provide a clearer description of why these methods were used. Furthermore, we will clearly separate the experimental findings and further analyses and provide the end-to-end evaluation.  \n\n**R1-2) Second, the content is not properly prioritized. A whole section (section 3 Damaging Retrieval) with more than one page is used to discuss the effect of damaging passages, which is a known phenomenon in previous works and contributes little to the introduction of the main contribution of the paper.** \n\nDamaging passages have not been thoroughly explored within the context of open-domain question answering. It is a fairly new concept, and we provide novel results in addition to (Sauchuk et al. 2022). \n\nThe limitation of previous work is that all sentences are jointly encoded in a single model and are not representative of open-domain question answering using FiD architecture. Our findings show that the issue is beyond joint encoding of passages, which was also reported in the [Lost in the Middle: How Language Models Use Long Contexts](https://arxiv.org/abs/2307.03172)  (Liu, Nelson F., et al.,2023) but also extends to independent encoding with FiD. \n\nThis allows us to affirm the invariance of FiD on the set of contexts as well as the existence of damaging passages in Reader, which allows us to perform incremental inference. This has not been addressed in previous works.\n\nTo aid with the readability and prioritization of the paper’s content, we will merge The knowledge-intensive NLP and damaging passage sections as “previous work” and separate our simulating damaging passages experiments (previously in 3.1 and 3.2) as an experimental finding to make it clearer that this is a key contribution. \n\n**R1-3)  Third, it lacks the necessary details. For example, how the binary classification works? Does it consider the question as input? Are the models initialized with the trained reader on the QA dataset?**\n\nWe will add necessary information in selection inference, attention analysis, and binary classification (5.1, 5.2, 5.4) in the experiment section. Regarding questions in R1-3), here are the details:\n\n- Binary classification is trained on DP and DN as described in section 5.3. We will add motivation and explanation in section 4. Method for clarification.\n\n- Input format is the same as that of FiD’s, “question : {question} title: {title} context : {passage}” for all classification methods. \n\n- RoBERTa(Large) and T5 Encoder(Large) are initialized with the huggingface repository checkpoints, and we employ FiD Encoder (Large) embeddings available on its source repository, which was pre-trained on DPR retrieved NQ train datasets. \n\n\n**R2) The paper still does not answer the question: why more information can be detrimental to a model. Instead, the paper simply presents some unexpected observations without explaining them with a hypothesis.**\n\nOur findings suggest that while additional information can have a detrimental effect, they also highlight the challenge of identifying such harmful information. The difficulty in identifying damaging passages primarily arises from the opaque nature of LLMs, which act as black boxes. To the best of my knowledge, no research completely elaborates how language models amalgamate long contextual knowledge with their parametric knowledge. Right now, the community lacks the tools to understand the exact reasons behind these detrimental effects. While we cannot answer the exact reason for why this happens at this moment in time, we hope that our analyses will help show the challenges available and help rally the community to study this important problem. We do answer the question in our title, and importantly, we show that if the problem can be resolved, answer accuracy can be greatly improved for many open-domain knowledge-intensive models while using fewer contextual passages. \n\n**R3) The 10% performance gain is achieved in an ideal scenario. Identifying the definite negative and semi-negatives is not a trivial problem.**\n\nWe divided R3 into two sub-questions.\n\n**R3-1) 10% performance gain is achieved in an ideal scenario**\n\nOur findings present the challenges of evaluating this very common architecture. Even with a superset of the evidence, models create mistakes. This is a negative feature that must be studied in future work when building more resilient open-domain NLP systems. It is critical to understand the limitations of the model. From this, we can understand the behaviors and limitations and make improvements to the models that use retrieved evidence. Our method isn't a SOTA model for benchmarking; our goal is to demonstrate this limitation and identify a problem that will motivate future research. We focus on identifying passages that detrimentally affect the final inferences made by the Reader model. \n\n**R3-2) Identifying the definite negative and semi-negatives is not a trivial problem**\nYes. Differentiating between these definite negative passages and semi-negatives is a challenge and definitely not trivial. One of our research goals is to determine the feasibility of capturing the pattern or signal within Reader that can be learned by a neural network when transition occurs. So, we put more emphasis on DP and DN; however, this does not mean that DN and SN are trivial. We conducted additional experiments to evaluate other methods for identifying the other passage types as well. Details and experiments are reported in Q3-2)\n\n**Q1) Figure 4: DF -> DP in the legend**\n\nWe corrected this issue. Thank you.\n\n**Q2) Line 569: This finding supports ? that**\n\nWe intended to refer to this paper; [Tomayto, Tomahto. Beyond Token-level Answer Equivalence for Question Answering Evaluation](https://aclanthology.org/2022.emnlp-main.20) (Bulian et al., EMNLP 2022) regarding answer equivalence. Sorry that this link was broken.\n\n**Q3) Why binary classification is considered instead of multi-class classification on the 5 types? Why not apply the classification results to the passage selection to see the end-to-end result?**\n\nWe Address Q3 in two different responses.\n\n**Q3-1) Why binary classification is considered instead of multi-class classification on the 5 types?**\n\nWe focus on Definite positives (DP) and Definite negative (DN) as these are transitions for the model where it changes the answer. We believe these to be the most significant for classification. If DP and DN can be distinguished through binary classification, this could serve as a foundational step toward implementing the suggested multi-class classification approach. However, we also agree with your opinion that multi-class classification can be applied. We lay out multi-classification on the 5 types with its end-to-end result in Q3-2).\n\n**Q3-2) Why not apply the classification results to the passage selection to see the end-to-end result?**\n\nIn the paper, we conclude that DPs and DNs are not easy to distinguish based on their binary classification and attention analysis. Also, as described in Table 5, all the models tend to classify DN as DP. Therefore, directly applying this classification cannot properly perform a selection method since DNs are considered to be DPs. However, based on the feedback, we experiment with two classification methods: binary classification on the whole EM pattern and multi-classification on 5 types (IZ, DP, DN, SP, SN). Also, for each classification method, we evaluate intermediate results (accuracy, f1, recall, precision) and end-to-end performances. \n\n**Binary/Multi Classification**\n\nFor all classification methods, the input format is the same as that of FiD’s, “question : {question} title: {title} context : {passage}”. The output is EM pattern, 0 or 1 for binary classification, and context type (IZ, DP, DN, SP, SN) for multi-classification. In contrast to the previous binary classification, the new binary classification is trained on the entire context.\n\nWe train three models: RoBERTa(Large), FiD Encoder(Large), and T5 Encoder(Large). RoBERTa(Large) and T5 Encoder(Large) are initialized from the huggingface checkpoints. We employ FiD Encoder(Large) available on FiD's Git repository, pre-trained on NQ datasets. We use the same datasets as for DP/DN binary classification, resulting in 700,500 and 175,200 instances, respectively. For end-to-end evaluation, we use the NQ test set. All training uses a batch size of 64, a learning rate of 5e-5, 11,000 steps (about 1 epoch), and other huggingface default parameters.\n\nThe table below describes the intermediate results of binary classification models on the development set.\n| Model | Accuracy | Recall | Precision | F-1   |   |   |   |   |   |\n|------------|----------|--------|-----------|-------|---|---|---|---|---|\n| FiD        | 0.564    | 0.791  | 0.547     | 0.646 |   |   |   |   |   |\n| T5         | 0.578    | 0.638  | 0.573     | 0.604 |   |   |   |   |   |\n| RoBERTa    | 0.503    | 1      | 0.503     | 0.669 |   |   |   |   |   |\n\nThe high recall and low precision indicate models often predict 1s even when true labels are 0s. RoBERTa consistently predicts 1s aligns with DP/DN classification. These results show binary classification inadequacy in identifying EM patterns, and performance demonstrates significant variation depending on the model.\n\n|                |           | DN  | SN    | SP    | IZ    | DP   |   |   |   |\n|----------------|-----------|-----|-------|-------|-------|------|---|---|---|\n| FiD            | precision | 0   | 0.09  | 0.54  | 0.5   | 0    |   |   |   |\n|                | recall    | 0   | 0.04  | 0.72  | 0.37  | 0    |   |   |   |\n|                |           | DN  | SN    | SP    | IZ    | DP   |   |   |   |\n| T5            | precision | 0   | 0.1   | 0.55  | 0.5   | 0    |   |   |   |\n|                | recall    | 0   | 0.05  | 0.72  | 0.37  | 0    |   |   |   |\n|                |           | DN  | SN    | SP    | IZ    | DP   |   |   |   |\n| RoBERTa        | precision | 0   | 0     | 0.5   | 0     | 0    |   |   |   |\n|                | recall    | 0   | 0     | 1     | 0     | 0    |   |   |   |\n| Total instance |           | 396 | 15438 | 86936 | 71122 | 1308 |   |   |   |\n|                |           |     |       |       |       |      |   |   |   |\n\nThe table shows that models struggle with identifying DN & DP, favoring predictions of SP, which constitutes 50% of instances. Notably, all three models can't differentiate DNs and DPs at all. Among 175,200 instances, T-5 assigns 114,606 (65.4%) to SP, FiD labels 115,522 (65.9%) as SP, and RoBERTa classifies all as SP. This highlights multi-classification's challenge in capturing passage types and inability to identify DN and DP.\n\n**End-to-End Result**\n\nFor the end-to-end evaluation, we construct two test sets from the NQ test dataset for each classification method. Models use these test sets to generate either EM patterns or passage types. For EM patterns, the selection method can be directly applied. For passage types, they are first converted to EM patterns before applying the selection method.\n\nThe first table presents the Exact Match score for the end-to-end result on the NQ test set. For comparison, the second table displays EM@100 and Probe3 on the same NQ test set.\n\n|        |         | Probe1 | Probe2 | Probe3 | Probe4 | Probe5 | Probe6 |   |   |\n|--------|---------|--------|--------|--------|--------|--------|--------|---|---|\n| Binary | FiD     | 46.18  | 35.37  | 52.69  | 42.27  | 53.66  | 43.3   |   |   |\n|        | T5      | 46.12  | 36.98  | 53.57  | 44.68  | 54.27  | 45.29  |   |   |\n|        | RoBERTa | 54.43  | 35.84  | 54.43  | 35.84  | 54.43  | 35.84  |   |   |\n| Multi  | FiD     | 48.59  | 37.51  | 53.77  | 42.74  | 54.71  | 43.66  |   |   |\n|        | T5      | 48.73  | 37.45  | 53.46  | 42.3   | 54.24  | 43.19  |   |   |\n|        | RoBERTa | 54.43  | 35.84  | 54.43  | 35.84  | 54.43  | 35.84  |   |   |\n\n| EM@100 | Probe 3 |   |   |   |   |   |   |   |   |\n|--------|---------|---|---|---|---|---|---|---|---|\n| 54.43  | 65.12   |   |   |   |   |   |   |   |   |\n\nAmong all classification models, both binary RoBERTa and multi RoBERTa achieve the highest EM scores across Probe1, Probe3, and Probe5. This is due to all RoBERTa inferences being 1s or SPs for each classification, resulting in all contexts being used as inputs for the reader in Probe1, 3, and 5. Consequently, these scores should match EM@100, which uses all contexts for inference.\n\nAll Probe results exhibit either lower or equal performance compared to EM@100, notably falling significantly below Probe 3's EM score. This underscores the classification models' struggle in identifying EM patterns and passage types, reflecting the challenge in distinguishing DN and DP within the provided retrieved context, as observed in the end-to-end evaluation.\n\nWe will integrate these findings into the Ablation studies following the results and analysis", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "There are many interesting insights observed from the experimental results. For example,  passages with low attention scores can contribute to the correct predictions, while passages with high attention scores do not always guarantee the correct answer. For another example, retrieval models should not optimize for the recall but also consider how irrelevant passages can cause a catastrophic interaction with the downstream reader model. These insights can be helpful for future research. \n", "reasons_to_reject": "1. The writing of this paper needs improvement. First, the organization is poor. There is no introduction on why we need Selection Inference, Attention Analysis, and Binary Classification. Second, the content is not properly prioritized. A whole section (section 3 Damaging Retrieval) with more than one page is used to discuss the effect of damaging passages, which is a known phenomenon in previous works and contributes little to the introduction of the main contribution of the paper. Third, it lacks the necessary details. For example, how the binary classification works? Does it consider the question as input? Are the models initialized with the trained reader on the QA dataset?\n2. The paper still does not answer the question: why more information can be detrimental to a model. Instead, the paper simply presents some unexpected observations without explaining them with a hypothesis. \n3. The 10% performance gain is achieved in an ideal scenario. Identifying the definite negative and semi-negatives is not a trivial problem. ", "questions_to_authors": "1. Figure 4: DF -> DP in the legend\n2. Line 569: This finding supports ? that\n3. Why binary classification is considered instead of multi-class classification on the 5 types? Why not apply the classification results to the passage selection to see the end-to-end result? ", "paper_topic_and_main_contributions": "This paper proposes a method to identify and filter damaging passages in the retrieve-then-read architectures, which improves at least 10% on accuracy for two popular QA tasks. The method is motivated by the fact that the model performance could be hurt by the incremental inference on BM25 negative samples. Thus, the paper defines 5 types of passages based on their EM results compared to previous retrieved passages. It is observed that a 10% performance gain can be achieved by removing definite negative and semi-negative passages. ", "excitement": 4, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: The reviewers are in disagreement about the soundness and exciting qualities of this work, giving scores between ambivalent and strong. 

Questions about soundness centered on the goals of the study. Reviewers critiqued the practical applicability of the method (e.g., when gold data is not available), but the authors clarified that their goal was not to build a universal tool for model improvement but a higher level goal of problem illumination. I believe the authors adequately responded to these concerns, which were the strongest of the soundness concerns. 

The simplicity of their method is a pro, and and the various insights and tests seeking harmful passages are useful in themselves and can inspire future research. These insights contribute to some reviewers finding the work strongly exciting.

The reviewers suggested clarifying the organization and writing in some sections of the paper, and these change would greatly improve a final version of the paper. I agree with this request and believe the authors can make the paper clearer (as one example: by labeling axes on plots and providing more intuitive explanations in figure captions).
Decision: Accept-Findings