Merging Generated and Retrieved Knowledge for Open-Domain QA
Authors: Yunxiang Zhang
Muhammad Khalifa
Lajanugen Logeswaran
Moontae Lee
Honglak Lee
Lu Wang
Venue: EMNLP 2023 Main (2023)
Keywords: large language model
retrieval-augmented language model
open-domain question answering
TL;DR: We propose a compatibility-oriented framework to better combine LLM-generated knowledge and retrieval knowledge for answering open-domain questions.

Abstract: Open-domain question answering (QA) systems are often built with retrieval modules. However, retrieving passages from a given source is known to suffer from insufficient knowledge coverage. Alternatively, prompting large language models (LLMs) to generate contextual passages based on their parametric knowledge has been shown to improve QA performance. Yet, LLMs tend to "hallucinate" content that conflicts with the retrieved knowledge. Based on the intuition that answers supported by both sources are more likely to be correct, we propose COMBO, a Compatibility-Oriented knowledge Merging for Better Open-domain QA framework, to effectively leverage the two sources of information. Concretely, we match LLM-generated passages with retrieved counterparts into compatible pairs, based on discriminators trained with silver compatibility labels. Then a Fusion-in-Decoder-based reader model handles passage pairs to arrive at the final answer. Experiments show that COMBO outperforms competitive baselines on three out of four tested open-domain QA benchmarks. Further analysis reveals that our proposed framework demonstrates greater efficacy in scenarios with a higher degree of knowledge conflicts.

Merging Generated and Retrieved Knowledge for Open-Domain QA
Yunxiang Zhang,, Muhammad Khalifa", Lajanugen Logeswaran', Moontae Leeft, Honglak Lee"t, Lu Wang" University of Michigan', LG Al Rescarch", University of Illinois at Chicago!
Abstract
NaturalQuestions (Single-hop QA)
Open-domain question answering (QA) sys- tems are often built with retrieval modules. However, retrieving passages from given source is known to suffer from insufficient knowledge coverage. Alternatively, prompt- ing large language models (LLMs) to gener- ate contextual passages based on their para- metric knowledge has been shown to improve QA performance. Yet, LLMs tend to "hal- lucinate" content that conflicts with the re- trieved knowledge. Based on the intuition that answers supported by both sources are more likely to be correct, we propose COMBO, Compatibilltty Ormiattbiliyy-OOienteedd knowledge Merging for Better Open-domain QA framework, to effec- tively leverage the two sources of informa- tion. Concretely, we match L.LM-genrraeed passages with retrieved counterparts into com- patible pairs, based on discriminators trained with silver compatibility labels. Then Fusion- in- Decoder-based (Izacard and Grave, 202 reader model handles passage pairs to arrive at the final answer. Experiments show that COMBO outperforms competitive baselines three out of four tested open-domain QA benchmarks. Further analysis reveals that our proposed framework demonstrates greater effi- cacy in scenarios with higher degree of know]- edge conflicts.'
Question: Who plays Charlotte in "The Strain" season 4?
Correct Answer: Rhona Mitra V
Prediction: Alexandra Breckenridge
Retrieved Passage
(Rhona Mitra) ... she played Charlotte in the fourth season of "The Strain" TV Series
LuM-generated Passage
The Strain an American horror drama television series Charlotte is played Alexandra Breckenridge season
Figure Fusion-in-Deccder-basse (Izacard and Grave, 202 QA model is misled by the knowledge conflict between retrieved passage and hallucinating I.MM-generated passage on example from Natu- ralQuestions (Kwiatkowski et al., 2019). This is because QA models tend to favor LLM passages when conflict exists. More detailed analyses Appendix A.I.
sources, e.g., Wikipedia, enjoys the benefits of be- ing factual, but may suffer from incomplete knowl- edge coverage and contain irrelevant information.
Large Language Models (LLMs) have been shown to store wide range of knowledge in their parameters, which can serve as an alternative in- formation source (Brown et al., 2020; Chowdhery et al., 2022; OpenAI, 2023; Ouyang al., 2022). Capitalizing on the success of leveraging LLMs' parametric knowledge for natural language under- standing tasks, a new paradigm known as generate- then-read (Yu et al., 2023) has emerged. It prompts an L.M to generate contextual passages for a ques- tion in lieu of retrieval from static corpora. Com- pared to retrieved passages, LLM LLM-gennrrted texts more relevant to the question, as its generative nature implicitly optimizes for content relevance. Yet, they frequently contain factual errors due hallucinations (Ji al., 2023; Peng et al., 2023).
1 Introduction
Open-domain question answering (QA) typically requires models consolidate and reason over in- formation from external knowledge sources (Chen al., 2017; Petroni et al., 2021). A common retrieve-rhen-readd framework (Izacard and Grave, 2021a,b; Izacard et al., 2022; Karpukhin et al., 2020) would first fetch relevant passages from an external corpus and pass them to a reader model to arrive at an answer. Retrieving from reliable
This work aims address how to combine re- trieved and parametric knowledge to get the best of both worlds for open-domain QA. Specifically,
*Correspondence to yunxiang@umich. edu OOur code is publicly available at https: http///ttttuucccee yunx-z/COMBO.

---

we want maintain the factuality of retrieved knowledge while leveraging the relevance of LLM knowledge. Yu et al. (2023) have explored simple merging approach by separately encoding the two types of passages into a Fusion-in-Decoder (Izac- ard and Grave, 2021b) reader (Figure 2a). While this setting outperforms using either source alone. the direct merging approach ignores inconsistent facts between the sources, which easily con- fuse the reader model. Figure 1 shows an exam- ple of knowledge conflicts where LLL--generated passage contains fabrication, contradicting the re- trieved knowledge.
We introduce COMBO, a novel compatibility- oriented framework for merging LLM knowl- edge and retrieved knowledge in open-domain QA.
We automatically mine silver labels for train- ing compatibility scorers without human an- notations.
We demonstrate the effectiveness of our frame- work with extensive experiments and analyses over four open-domain QA datasets.
Related Work
To address this challenge, we propose novel Cumpatibility-Orientee knowledge Merging for Better Open-domain (COMBO) QA framework. Intuitively, an answer tends to be correct if sup- ported by information from both sources. There- fore, we promote the model's i usage of factual and relevant information by combining retrieved and generated knowledge into compatible pairs, which are then fed into reader module.
Parametric and Retrieved Knowledge for QA. QA models commonly have access to two knowl- edge sources: parametric knowledge stored in lan- guage models and retrieved knowledge from ex- ternal corpora. Previous work (Chen et al., 2022; Longpre et al., 2021; Pan et al., 2021) focuses on analyzing knowledge conflicts in simulated settings by perturbing retrieved contexts, e.g., replacing en- tities with incorrect counterparts. These studies reveal that reader models overly rely on the para- metric knowledge and demonstrate limited abilitics of resolving conficting information. In contrast, propose solutions to repress models from using generated knowledge that contradicts with retrieved contexts. Other works (Li al., 2022a; Mallen et al., 2022; Neeman et al., 2022) leverage prede- fined rules to guide the models to choose between parametric vs. retrieved knowledge sources un- der conflicts. Instead of relying on dataset-specific heuristics, we introduce a generalizable approach that teaches QA models to prioritize compatible information over confficting ones.
To date, there has been no gold-standard annota- tion for training compatibility scorer for passage pairs To this end, we introduce a novel method that automatically mines silver labels of compatibility scale, without the need for human annotation dptaset- heuristics. Specifically, we esti- mate the silver compatibility by checking whether the prediction correctness of a QA model would flip one or both passages from a target pair were to be removed from the input. Afterward, we train discriminators on silver labels to compute passage pairs' compatibility scores.
Lastly, we benchmark COMBO in a fully- supervised setting on four popular open-domain QA datasets, including both single-hop (Natu- ralQuestions (Kwiatkowski et al., 2019), Triv- iaQA (Joshi et al.. 2017), WebQuestion (Be- rant et al., 2013)) and multi-hop questions (Hot- potQA (Yang et al., 2018)). Using state-of-the- art retrievers, DPR (Karpukhin et al., 2020) and MDR (Xiong et al., 2021), as well as performant LLMs, e.g., InstructGPT (Ouyang et al., 2022) and ChatGPT, COMBO outperforms competitive base- lines on all testbeds except HotpotQA by up to +1,9 exact match score.
Augmentation with Large Language Model Out- puts. LLMs, such InstructGPT (Ouyang al., 2022) or PaLM (Chowdhery al., 2022), store rich world knowledge in their model weights and demonstrate impressive performance on open- domain QA tasks with prompting techniques. Re- cent studies further boost their performance by elic- iting supporting evidence before answer prediction. The types of supporting evidence include encyclo- pedia knowledge (Li et al., 2022b; Sun et al., 2022; Yu et al., 2023), commonsense knowledge (Liu et al., 2022a,b; Shwartz et al., 2020; Wang et al., 2022; West et al., 2022) and chain-of-thought rea- soning steps (Magister et al., 2022; Trivedi et al., 2022; Wei et al., 2022; Zhang et al., 2022). Most
To summarize, the main contributions of our work are three-fold:
Fiiuure 8 in Appendix A shows another example of knowl- edge conflict on HotpotQA (Yang et al., 2018).

---

EP
0.83
0.34
0.90
0.72
Consistency Discrimiiator
LLM Psg
LP
LP
0.87
91
0.35
0.88
0.80
Dc
Compatibility Scoring
06
RP
LP3
0.85
8.89
0.13
0.95
0.74
D
FID
4
Evidentiality Discriminslor
Retrieved Pog
E.P.4
0.00
-0
2
LLM Pag
Retneved Psg
: Retrieved
E38
Psg
Pag
FiD
LLM Retrieved Psg
2. Pairwise Compatibility Matching
3. Predicting with Passage Pairs
(a) Direct Merging (Yu et al., 2023)
(h) COMBO (Ours)
Figure 2: Overview of COMBO framework (right) that uses both LLM- LLM-generated passages and retrieved passages for open-domain QA. We compute pairwise compatibility scores, which quantify the extent to which the two passages support each other regarding evidence related the question. 2. We then match them into pairs by maximizing their overall compatibility (matched pairs are highlighted in red boxes). 3. Finally, passage pairs are sorted by their compatibility and a FiD-based (Izacard and Grave, 2021b) reader model to produce an answer.
relevant to our work, Yu et al. (2023) directly aug- ment retrieval passages with LLM-generated texts, but they do not take knowledge conflicts into count, which could mislead the model and degrade the prediction. Our work promotes the model's usage of factual evidence over hallucinating infor- mation LLM outputs (Creswell and Shanahan, 2022; ApenAI, 2023; Peng et al., 2023; Zhao et al., 2023) with the help of retrieval knowledge.
read and generate-then-read counterparts." 3 The core idea is to feed paired passages one from LLM, one from retriever- into a FiD-based reader model. The passages are paired in such way that they provide consistent evidence the question for the reader to identify both the fac- tual and relevant information. Specifically, for each question, N retrieved passages and M LLM- generated passages are given to COMBO. The an- swer is produced in three steps shown in Figure 2b.
Bridentiality-guiide QA. Traditionally, the field of Open-Domain QA has been dominated by retrieve-then-reaa models (Izacard et al., 2021; Izacard and Grave, 2021a; Karpukhin et al., 2020; Xiong et al., 2021). The performance of the readers largely depends on the evidentiality of the retrieved passages-wa whetthr they contain the correct evi- dence to support the answer. Recent work (Asai al., 2022; Fajeik et al., 2021; Lee al., 2021) augments QA performance by adding a (silver) ev- identiality signal of each retrieved passage to the training of reader models. However, their method insufficient to work with L.M--geeeated passages that contain hallucination errors. We further incor- porate the evidentiality of -Mgeeerraeed passages and leverage both sources to highlight the correct
The compatibility scores of all possible pas- sage pairs are computed according to an evi- dentiality discriminator and a consistency dis- criminator ($3.1), trained using silver labeled data ($3.2).
A compatibility -oreented matching strategy se- lects passage pairs to maximize overall com- putibility while balancing the usage of all pas- sages ($3.3).
3. A FiD-based reader handles passage pairs, sorted by compatibility scores, to generate the final answer.
3.1 Defining Compatibility
evidence for the reader.
We first describe two assumptions, based on pre- liminary analysis provided in Appendix A.4, which lay the basis for the compatibility formulation."
3 Method
In this section, we present COMBO, which com- bines both the retrieved passages and the LLM'' parametric knowledge to improve open-domain QA performance over the existing retrieved-then
"We provide a brief introduction to the retrieved then-reead generate-then-rer QA frameworks Appendix A.2. "For simplicity, we illustrate our method under the single- hop QA setting in the main paper. Appendix A,3 shows how to adapt it to the multi-hop setting with minimal modifications.

---

Assumption 1 Retrieved passages are faithful to real-world facts, i.e,, being factual, regardless of its relevance to the question.
Question: When Italy FIFA World Cup?
Answer: 1934
Retrieved Passage 1
(FIFA World Cup hosts] the only remaining candidate Italy to take hosting job for the
Cup.
was retified...
Assumption 2 LLM-generated passages contain relevant evidence to the question, i.e., being plausi- ble, irrespective of its factuality.
Retrieved Passage
(Italy national football team) ... Italy ane most successful national teams the history of the World Cup, having won four titles (1934...
Retrieved Passage 3
Figure 1 shows an example of Assumption where a plausible answer ("Alexandra Breckenridge") is supported by the LLM-generated text despite its inconsistency with the retrieved passage.
(1990 FIFA World Cup) The vote choose the hosts the 1990
fournament was held on May 1984 zürich, Switzerfand...
LLM teeeggeeettedd Passage
The 1934 FIFA World Cup was the second FIFA World Cup, and was held Italy from JDne 1934.
With these two assumptions, when an answer can be evinced by both the retrieved passage and the LLM-generated passage. it is likely that the retrieved knowledge contains the relevant evidence whereas the generated knowledge is factual. More- over, the answer tends be correct in such situa- tions. Therefore, we define the compatibility of pair passages as follows.
1934 V
II.
RP2
RP3
Finetured
Base Reader
III.
RP1
RP2
RP
I.P
1934
IV.
1934 V
Consistent
I.P.
Figure Example for demonstrating the construction of silver consistent labels to train the consistency dis- criminator De-
Definition 1 A passage pair is COMPATIBLE if both passages contain the proper evidence to sup- port answering the question correctly.
ity). P(Ip: + Q rP; F Q) characterizes the con- sistency between the retrieved and LLM-generated knowledge.
With this concept, we hope to match two passages, one from each source, into compatible pairs. The pairs will facilitate reader model by promoting the correct evidence when knowledge conflicts exist.
Based on the decomposition in Equation 1, we train two binary discriminators accordingly: an evidentiality discriminator D3 for modeling P(rP; F Q) and consistency discriminator De for modeling P(lp, F Q rpj F Q). Con- cretely, DE takes question Q and the j-th re- trieved passage rpj as input, and predicts whether rPj is evidential (positive) or non-evidential (neg- ative). If rpj is evidential, De will take Q, the i-th -L.e-geeertted passage (pt, and rp, as input, and predicts whether (4pt, rPj) is consistent (pos- itive) or conflicting (negative), i.c., the retrieved passage contains the correct evidence while the L.Le--geerrtee passage does not. As result, passage pair is considered compatible if it is both evidential and consistent.
Compatibility Formulation. To illustrate how we compute passage compatibility, we formulate the concept of compatibility with mathematical notations. Given a question Q, use lp; to denote passage for Q, and rp,
the i-th L.LM-generate for the j-th retrieved passage. Ip; Q means lp; contains the correct evidence for Q, and similarly for rPj F Q. Following Definition 1, given Q, for a pair of LLM-generated and retrieved passages lp, and rpj, we formulate the compatibility score d.. P(lp: F Q rp; F Q), which could be further factorized into two components:
compatibility score
P(lp; F Q.rp; QQ= P(rp, F Q) P(pp: +
rpj (rrrp/ Q)-
3.2 Mining Silver Labels
evidentiality score
consistency score
We do not have human-annotated evidentiality and consistency labels in most existing datasets. To avoid costly manual annotation, we propose an approach to automatically mine silver labels for training the two discriminators. To collect silver evidentiality labels for training De, we w largely fol- low the leave-one-out generation approach by Asai et al. (2022). Briefly, a passage is deemed EVI- DENTIAL if the prediction by a given QA model
P(rp; + Q) measures whether the retrieved pas-
sage contains the correct evidence (i.e., evidential-
'Notice that we do not factorize the compatibility score the other way round (i.e.. P(ly; F Q) P(rp; F Q ly: F= Q)). This is because lp, always contains plausible evidence to the question (Assumption 2) so the discriminator cannot measure the evidentiality of lp. on its own (P (P(Ip. Q)). Instead, we use a retrieved factual passage (A (Assumption 1) to determine whether the lp, is hallucinating or not.

---

changes from correct to incorrect after removing from a pool of retrieved passages. We explain the details in Appendix C.I.
trains the model rely on the factual evidence that always located in the latter passage when facing confficting information.
We then extend this idea to mine consistency labels of passage pairs for training De, as demon- strated in Figures 3 and (for negative sample, in Appendix C.2). Intuitively, if the prediction changes from correct to incorrect after removing either passage from the target pair (c.g., IV=II for an LLM-ggnnrated passage and I->II for re- trieved passage, as in Figure 3), both passages supposedly contain the correct evidence and they are therefore a CONSISTENT pair. Concretely, for each dataset, we first finetune a base reader model initialized from musion-in-Decodee (Izacard and Grave, 2021b) with top-N retrieved passages Pa- {rp1; .... rpx} } input. We then la- bel the consistency of (lp;, rp;) based on the cor- rectness of predictions with four types of inputs: I. all the retrieved passages (Q, PR) II. all the re- trieved passages except the target retrieved passage (Q,PR) PRAPARRP (r III. all the retrieved passages plus the target LLM-gencrated passage (Q, Pn {(p:)) IV. all the retrieved passages without the target re- trieved passage and with the target LLL-ggeneraed passage (Q,Pr U (Upr) (rp;)) We consider (lpt, rpy) as consistent if the model's prediction is incorrect with II and all correct with I, II1, and IV, Conversely, (lp., rpy) is labeled as CONFLICTING if the prediction is correct with I and all incorrect with II, III, IV?
To create passage pairs with a balance of fac- tuality and coverage of supporting evidence, we design compatibility-guddee optimal matching strategy with the following two goals. First, for sim- plicity, we include all passages during the matching process, leaving
future work to filter incom- patible passage pairs. Second, we aim to maximize the compatibility of matched passage pairs, to min- imize the adverse impact of knowledge conflicts on the reader model.
To do so, we first use the two discriminators together compute scores for evaluating the com- patibility of all possible pairwise combinations of the retrieved and LLM--generate passages of a question (i.e., {(pp,, rPy) é E [1, 2,., N)).. This results in 2- 2-dimensional ma- trix shown in Figure 2b, where each element rep- resents the compatibility score for passage pair. A subset of all possible M x N pairs (red boxes highlighted in the matrix of Figure 2b) are then selected as input to the reader model. The selection is solved as a bipartite graph maximum muximtu-weeighted matching problem, with details in below paragraph.
M,,,,, c
The algorithm requires that compatible pairs score higher than conficting and non-evidential ones, it can achieve the maximum overall compatibil- ity of matched passage pairs while balancing the usage of all passages. To this end, devise a simple heuristic for compatibility scoring, dubbed as evidentialiy-cuttff:
3.3
Matching Retrieved and LLM -LL--eenerated Passages into Pairs
To signal the reader model with the connections between the two knowledge sources (compatible vs. conflicting), we adapt FiD to encode passage pairs as input. Each pair of passages is concatenated with the question, and processed independently from other pairs by the encoder. We add special to- kens: "question:"" "generated passage:" and "retrieved passage:" before the question and the passages, respectively. In this way, the encoder could perform self-attention (Vaswani et al., 2017) across the retrieved and LLM- -eMeerenettedd passages. We keep the relative order of LLM- --M--eeerrtedd and retrieved passage fixed across all input pairs, by always putting lp; in front of rp, (Figure 2b). This
+J = P(lp; = Q rp, QQ) (P(())=Q))0.5)
(2)
where 1 is an indicator function. It binarizes the decision from DE to score non-evidential pairs and prioritize compatible pairs over conflicting counterparts.
Compatibility- guided Optimal Matching We view the compatibility-uudde matching process as a maximum-weighht matching problem on a bipartite graph. Concretely, we treat each pas- sage as a node and there are edges connecting LaM- with retrieved-paaeee passsppassssosoe nodes, whose weights are the corresponding compatibility score. This results in complete bipartite graph G (Pz, PRi F)) where E. = {(1p, rP;) e {1, 2,., M) M). E (1,2, N)). The weight func- tion w. E > [0, 1] assigns compatibility score of
our experiments, we set = 10 across all datasets. "To speed up the consistency label mining process, we only need to obtain model predictions for III and IV if is correct and Il is incorrect (i.c., rp, is evidential).

---

Equation 2 edge weight: w(lpp,, rP;)) q3 We want find a perfect matching M of maxi- mum weight where the weight of matching M is given by w(M) = - w(e). This problem is typically solved by the Hungarian algorithm (Kuhn, 1955, 1956) in polynomial time. This matching is optimal in the sense that it covers all passages while maximizing the sum of their compatibility scores.
IQQA all Q dev
HQA bridge Q
NQ test
TQA
WebQ test
Methods
test
Single Knowledge Source
Retrieved Psg. Only 46.7
61.9
48.1
59.9
55.4
LLM Psg. Only
40.3
67.8
51.5
42.6
35.9
Two Knowledge Sources
Direct Merging
52.7
74.2
51.1
61.6
57.8
Random Matching
53.3
74.2
51.6
61.5
57.7
COMBO (ours)
54.2
74.6
53.0
61.6
58.0
4 Experiments
Table 1: Exact Match (EM) results on NaturalQues- tions (Kwiatkowski et al., 2019), TriviaQA (Joshi et al., 2017), WebQuestion (Berant et al., 2013), and Hot- poIQA (Yang al., 2018). For experiments under Two Knowledge Sources, we conduct three runs with differ- ent random seeds and report the average.
4.1 Experimental Setup
We experiment with four open-domain QA datasets:
NaturalQuestions Open (Kwiatkowski et al., 2019), TAiviaQA unfiltered (Joshi et al., 2017), WebQuestion (Berant et al., 2013) for single-hop QA and HotpotQA full wiki setting (Yang et al., 2018) for multi-hop QA. We provide brief introduc- tions and statistics for each dataset in Appendix B. We use Exact Match (EM) (Rajpurkar et al., 2016) as our major evaluation metric across all datasets. We run each experiment three times with different random seeds and report the average. 8
the training of its discriminators with data from the other two single-hop QA datasets. Note that the additional warm-up data is only used for train- ing the discriminators but not the reader model, so as ensure fair comparison with the baselines. More details of the prompts for LLMs, model im- plementation and hyperparameters are included in Appendix C.2.
We focus the fully-supervised setting and obtain retrieved and -.L--geeerted passages for each question in the dataset. For retrieved passages, we use retrieval results by DPR (Karpukhin al., 2020) for single-hop QA and MDR (Xiong et al., 2021) for multi-hop QA, where no gold-standane passages are used in this study. We do not further finetune the retriever. For LLM-generated passages, we use the ones provided by Yu et al. (2023) for the three single-hop QA datasets, which are based on InstructGPT, Since there are no available re- sources for LLM-gencrated passages on multi-hop QA, we obtain them by calling ChatGPT's API (gpt-35-turbo). We choose ChatGPT because is a performant LLM with reasonable costs.
4.2 Results and Analysis
Comparison with Baselines. Table shows ex- perimental results on the four open-domain QA datasets. All methods use 10 retrieved and/or 10 LLM--generate passages for each question Q and FiD-large (770M) as the reader model. Yu et al. (2023) presents strong baseline of directly merg- ing two knowledge sources. With the improved coverage of world knowledge, it beats the retrieval- only model by an average 7.1 EM over the three single-hop QA datasets. With the same reader model and the same set of input passages, our COMBO framework further improves over Direct Merging by an average of 1.3 EM scores across the single-hop QA datasets.
We employ FiD (Izacard and Grave, 2021b) our reader model. For the discriminators 'Dg and De, for single-hop QA and DeBeRTa-large (He al..
RoBERTa-large (Liu et al., 2019)
We also compare our proposed algorithm to a Random Matching baseline, which randomly di- vides passages into pairs. Our improvement over Random Matching indicates that compatible pairs of LLM and retrieved passages are essential for model to make better predictions. Appendix D.1 also shows the results of an oracle setting assuming access ground-truth answers for matching and the state-of-the-an results reported in existing liter- ature. Since our approach does not modify retriever or reader architecture, it can be easily extended to
2021) for multi-hop QA. DeBERTa features rela- tive positional embedding, making it possible to adapt to longer input for passages of multi-hop QA.
In our experiments, we train separate discrimina- tors De and De for each dataset. However, given the limited amount of silver labels mined from the small-scale dataset WebQuestions, we warm up
'For results of standard deviation under EM and Fl scores, please see Table 6 Appendix D.I.
"https: / tttbb/ttttttuttte/eee//eteeeeebt honm/wyy97/Genne Geneead. git

---

LLM Psg
Retrieved Pag
LLM Pag
Retreved Pog
Retrieved
LLM Psg
11M
Retrieved Pog
FiD
Q
Rerrieved Peg.
FI
LLM Psg Retrieved
:
Retrieved Pig
FiD
LLM Psg
Retrieved Pag
LLM
Pog 4
Retrieved Psg.
COMBO (ours)
w/o pairwise input
w/o fixed rPpr order
Figure 4: Illustrations of the input formats for two of the ablation experiments in Table 2.
capitalize on larger models, more retrieved pas- sages and additional knowledge sources (c.g., la- bles and KBs), as already leveraged in state-of-the- art systems, We also note that any potential per- formance improvements are somewhat constrained by the percentage of the hallucinated (generated) and evidential (retrieved) passages provided. In the extreme scenario where all generated passages are devoid of hallucinations or all retrieved passages lack correct evidence, method would default to Random Matching, as the compatibility matching score would be uniform.
NQ dev
TQA
Method
dev
COMBO (ours)
52.3
73.9
w/o evidentiality discriminator
51.8
73.5
w/o pairwise input
51.5
73.4
w/o sorting pairs
51.7
73.7
w/o fixed (lpt, rp,) order
50.8
73.3
w/o evidentiality-cutoff optimal matching
51.7
73.6
51.6
73.6
Table 2: Ablation study results on NaturalQuestions (NQ) and TriviaQA (TQA).
Different from single-hop QA, we do not ob- serve improvements on HotpotQA. However, mi- nor improvement is observed on the subset (73%) of bridge questions that require the retriever or LLM to find bridge entity that connects two pieces of evidence together to reveal the answer. It shows that our method is not directly generaliz- able to the more challenging multi-hop setting, and thus calls for a more fine-grained modeling of the compatibility between hops of evidence. We note that for the rest of comparison questions in Hot- potQA (e.g., "who is younger, Keith Bostic Jerry Glanville?") models could make the right predic- tion even if the LLM-generated passages contain hallucinated evidence (see Appendix D.2 for an example). This explains why our compatibility- oriented framework appears to be less effective on this subset."" 10
consistency discriminator De as the compatibility score for ranking passage pairs (w/o evidentiality discriminator). The performance drop highlights the importance of having an evidentiality discrim- inator to filter out irrelevant (i.c., non-evidential) retrieved passages before computing the passage consistency and thus echoes our compatibility de- composition motivated by Equation 1.
Second, we remove the pairwise formulation by linearizing the matched pairs into sequence of single passages input (w/o pairwise input). See Figure in Appendix C.2 for illustration of the linearized inputs. The performance loss vali- dates the usefulness of the encoder's 's self-attention between tokens of -LL--reertted and retrieved passages, which allows the reader module to jointly reason over passage pairs as aggregated evidence.
Third, we randomly shuffle the order input passage pairs instead of ranking by their compati- bility scores (w/o sorting pairs). The score worsens, as this model variant fails to learn to prioritize com- patible information. Further analysis of model's 's behavior in Appendix D.4 verifies that model could attribute higher attention scores compati- ble pairs than others when making predictions.
Ablation Study. We create several variants to val- idate the effectiveness of different components of our COMBO framework, by removing or replacing each component individually. Results are shown in Table 2. First, we remove the evidentiality discrim- inator De and only use the probability given by the
We ure unable to show results of multiple methods on the hidden test set of HotpotQA, as the official leaderboard of Hot- potQA (https: /not:o//otttta....mm tit. io/) allows submitting predictions only once. Therefore, we show results on the dev set instcad.
For the fourth ablation experiment (w/o fixed (lps, rPş) order), we randomly shuffle the order of Ip: and rp, within each passage pair, instead always putting lp, in front of rp; (Figure 4, Ap-

---

Conflicting
Subser5i
Retrieved Psg. Only
Direct Merging
COMBO
NaturalQuestions
Rate
49.5
D 0.1
56.2%
41.6
47.7
48.5 (+0.8)
0.2
22.7%
45.1
53.1
53.3 (40.2)
49.0
0.2-0.3
15.5%
$2.5
59.0
59,9 (+0.9)
Exact Exact Match
0.4
1.8%
62.5
65.0
67.5 (+2.5)
48.5
0.5
2.2%
57.4
64.0
66.0 (+2.0)
0.5 1.0
1.6%
61.8
56.6
61.0 (+4.4)
48.0
Direct merging
Table 3: The performance of our method compared to others on NaturalQuestions dev set w.r.t. conflicting rate (percentage of conflicting passage pairs, Equation 3). Larger improvements of COMBO over Direct Merging are shaded with darker orange. Overall, COMBO's improvement over Direct Merging is greater when the conflicting rate higher, suggesting the robustness of our method to knowledge conflicts.
47.5
Random matching
COMBO
(10, 10)
(10, 20)
(20, 20)
(20, 50)
(M, N)
Figure 5: The performance on NaturalQuestions dev set when scaling the number of LLM -gnneratd passages (M) and retrieved passages (N) for each question. We report the average number and standard deviation (error bar) over three runs with different random seeds.
pendix C.2). We find that fixing their order allows the model to learn rely on the factual evidence in the latter retrieved passage when facing confficting information.
conflicting knowledge. We measure conflict rate as follows:
We also explore other ablations show the con- tributions of several heuristics employed in our system, including evidentialdy-cuttof (Equation 2) and optimal matching (Section 3.3). We replace Equation with Equation 1 for producing compati- bility scores (w/o evidentiality-cutonn directly multiplies the two probabilities given by the two discriminators together. demonstrates that models benefit from binarized decision by De and we hypothesize that this is because the raw predicted probability is often not well calibrated (i.e., predicted probabilities do not correlate well with the probabilities of correctness (Guo et al., 2017: Jiang al., 2021)).
conflicting_rate NA (M MA)
(3)
N M
NA refers to the number of retrieved passages that contain the gold answer string A and M MA means the number LLM-ggnerated passages that do not contain the gold answer string. The con- flicting rate indicates the percentage of conflicting pairs (i.e., rP; contains the answer while (p, does not) over all possible pairs. Table 3 suggests that when there is minimal confficts between the two sources, both Direct Merging and COMBO can significantly improve over the Retrieved -pasaag---- only model. However, when the conflict rate is high, only COMBO can maintain a consistent im- provement over the Retrieved Retrieved-passage-one base- line, suggesting its robustness.
Equation 1
Additionally, we replace the optimal matching (maximum weighted matching) with a greedy stral- egy (w/o optimal matching). Specifically, we first match compatible passage pairs, followed by con- flicting and non-evidential pairs. When finding the k-th pair to match, we only consider passages not appearing in previous k - pairs. The performance drop shows the optimal matching could bring to- gether more compatible pairs and thus better guide the model's predictions.
Scaling with Number of Passages. We further evaluate the performance of COMBO with respect to different numbers of LLM- --M-eeertteed passages (M) and retrieved passages (N), Figure shows the results for NaturalQuestions. Given a larger number of passages, we switch our reader model from FiD-large to FiD-base due to GPU memory limits. When M is smaller than N, we simply duplicate the LLM-generated passages to match the number of retrieved passages, that every passage is included in the matching results (con- sistent with Section 3.3). We observe that given more passages from both sources, our framework generally achieves greater performance gain over
Impact of Conflicting Contexts on the Reader Model. Here aim to answer question: How well can the reader model (i.e., FiD) pinpoint the correct answer when varying degrees of knowledge conflicts exist between -.ee-aeeertted texts and retrieved passages? Table 3 shows the performance different methods when facing different rates of

---

Question: what NPL coach has the most wins ever?
Answer: Don Shula
Predictions: Direct Merging + George Halas COMBO + Don Shula
L.M-generated Passage 3
Retrieved Passage
The NFL coach with the most wins over Don Shula, who coached the Mami
[Don Shula] He turrently holds the NIL record for mest career wins head souch, with 347.
Dalphins from 1970 1995...
LLM LlM-genertted Passage 10
Retrieved Passage
National Footbal Lergue (NF1), the head coach with the most regular season wins is George Halas, who complied 518 victories 40 seasons with the
[Guy Chumberlin] -. complled 58-15 recond in six years as head coach in National Football League (NFL), the best percentage of coach in NEL history..
Figure 6: An example of QA pair and the passage matching results by COMBO. Passage pairs are sorted by their compatibility scores. shows COMBO rectifies prediction baseline methad under knowledge conflicts by prioritizing compatible pairs (green connecting line) over incompatible pairs (red connecting line).
the direct merging approach. Additional analysis in Appendix D.3 shows that more knowledge conflicts arise from the increased number of input passages, again highlighting the importance of compatibility- guided knowledge merging. Because train discriminators only the top 10 passuges, they may not generalize well when applied the top 50 passages. This possibly why our method seems less effective when provided with 50 retrieved pas- sages as input.
40
35
Compatible
40
1
9
30
Predicted Label
25
Conflicting
37
20
15
10
Non-avidertiall
5
Compatible Conflicting evveentiiiall Annotated Label
Case Study of Matching Results. Figure shows example from NaturalQuestionn where the COMBO matching results of passages help rec- tify the prediction of the Direct Merging model. In the first compatible pair, both passages contain cor- rect evidence that supports the ground-truth answer "Don Shula" The last one is an incompatible pair where the LLM-ggnerated passages frequently con- tain hallucinating error ("George Halas") that mis- leads the Direct Merging model to make wrong prediction. Since we prioritize compatible pairs over incompatible pairs, the reader model could leverage this inductive bias by paying more atten- tion to the higher-ranked passages.
Figure 7: Confusion matrix of human annotations vs. predicted labels by our discriminators 150 random samples from Natural QueestoQQsssoons dev set. The overall accuracy is 78%.
5 Conclusion and Future Work
In this work, we study the problem of merging retrieved and LLM-generated knowledge for open- domain QA. We tackle the challenge of knowledge conflicts caused by LLM's hallucination with a compatibility-orieni
knowledge merging frame- work (COMBO), Specifically, we match LLM- generated and retrieved passages for given ques- tion into pairs based on their compatibility and per- form information fusion the encoder side of the FiD- based reader by feeding matched pairs in- put. Experiments on four open-domain QA datasets demonstrate the empirical success of COMBO.
Human Evaluation of Compatibility Labels. Figure shows the confusion matrix of the com- patibly labels predicted by our discriminators on 150 examples from dev set NaturalQuestions. Specifically, we randomly select 25 questions and sample 2 compatible pairs, conflicting pairs and 2 non-evidential pairs (if applicable) for each ques- tion. The authors manually analyze whether the passages actually contain the correct evidence. We find that discriminators yield an overall accu- racy of 78% on this 3-way classification task. shows that our discriminators are capable provid- ing signals of passage compatibility to the reader model for making well-informed decisions.
In the future, we plan to extend our framework to few-shot QA settings by employing an LLM with in-context learning to compute the compat- ibility scores instead of training a discriminator from scratch, thereby eliminating the necessity for (weakly) supervised training. But it would also quire carefully examining whether and which LLM is able to discern the knowledge conflicts between its own parametric memory and retrieved evidence.

---

Limitations
Ethics Statement
1. We only evaluate our knowledge merging framework on the tasks of open-domain QA. It would be interesting to apply our method to other knowledge-inttnnsive NLP tasks such as fact checking (Thorne et al., 2018) and knowledge-enhamee text generation (Dinan et al., 2019; Yu et al., 2022).
Large language models are known to have racial and gender biases and may generate harmful content such as fabricated facts and toxic re- sponse (OpenAl, 2023). Since we leverage gen- erated contextual passages from LLMs to enhance QA performance,
models may inherit these biases during generation. Although we aim to pro- mote the usage of factual information by combin- ing generated knowledge with retrieved knowledge from an external trustworthy corpus, we do not fully eliminate hallucinations from LLMs. Inter- ested practitioners should carefully address these risks before deploying our framework in certain domains such as politics, finance, and healtheare.
2. Under the conditions of our experiment, we did not manipulate the distribution of ques- tions input passages in any way. However, we anticipate that in more controlled settings, such PopQA (Mallen et al., 2023) where questions about infrequent entities are exam- ined in more detail, our method could yield more dramatic performance improvements. At present, LLMs appear more susceptible to hallucinating knowledge pertaining to less frequent entities. We eagerly anticipate inves- tigating this aspect further in our future work.
A Acknowledgements
This work is supported by LG Al Research and computational resources and services provided by Advanced Research Computing (ARC), a division of Information and Technology Services (ITS) at the University of Michigan, Ann Arbor. Addition- ally, we would like to thank Wenhao Yu for his invaluable assistance in clarifying aspects of their published work and generously sharing code and data to facilitate the reproducibility of the results presented in this paper. We also appreciate the anonymous reviewers for their valuable sugges- tions. We thank the members of the LAUNCH group at the University of Michigan for their dis- cussions and suggestions.
3. For each dataset, only conduct experi- ments with generated passages from single LLM (either InstructGPT or ChatGPT) and re- trieved passages from a single retriever (either DPR or MDR). Our experiments are limited to generative reader model which is based on the widely-used Fusion-in in----odee (Izacard and Grave, 2021b) architecture.
4. Regarding computational overheads of COMBO, we leverage the same set of input passages and same reader model as direct merging. During the training of the FiD-based reader model, according to our empirical observations on NaturalQuestions, it only introduces minimal computational overheads of approximately 23% more GPU memory and 27% more training time, compared to direct merging. Practitioners who can afford to train the previous direct merging model should also be able to train our framework.
References
Rohan Anil, Andrew M. Dai, Orhan Firat, Melvin John- son, Dmitry Lepikhin, Alexandre Passos, Siamak Shakeri, Emanuel Taropa, Paige Bailey, Zhifeng Chen, Eric Chu, Jonathan H. Clark, Laurent Shafey, Yamping Huang, Kathy Meicr-Hellstern, Gau- rav Mishra, Erica Moreira, Mark Omernick, Kevin Robinson, Sebastian Ruder, Tay, Kefan Xiao, Yuanzhong Xu, Yujing Zhang, Gustavo Hernández Ábrego, Junwhan Ahn, Jacob Austin, Paul Barham, Jan A. Botha, James Bradbury, Siddhartha Brahma, Kevin Brooks, Michele Catasta, Yong Cheng, Colin Cherry, Christopher A. Choquette- -hho, Aakanksha Chowdhery, Clément Crepy, Shachi Dave, Mostafa Dehghani, Sunipa Dev, Jacob Devlin. Mark Díaz. Nan Du. Ethan Dyer, Vladimir Feinberg, Fangxi- aoyu Feng, Vlad Fienber, Markus Freitag, Xavier Garcia, Sebastian Gehrmann, Lucas Gonzalez, and et al. 2023. Palm 2 technical report. CoRR, abs/2305. 10403.
5. Our proposed approach for silver label mining could result in limited amount of labels small dataset, which could not provide suf- ficient data train a dataset-specific discrim- inator. It thus calls for the need for training unified discriminator that works for various datasets and tasks.

---

Akari Asai, Matt Gardner, and Hannaneh Ha- jishirzi. 2022. Evidentiality- guided generation for knowwedge-ittennive NI.P tasks. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguisties: Hu- man Language Technologies, NAACL 2022, Seautle, WA, United States, July 10-15, 2022, pages 2226- 2243. Association for Computational Linguistics.
David Dohan, Shivami Agrawal, Mark Omernick, An- drew M. Dai, Thanumalayan Sankaranarayana Pil- lai, Marie Pellat, Aitor Lewkowyez, Erica Moreira, Rewon Child, Oleksandr Polozov, Katherine Lee, Zongwei Zhou, Xuezhi Wang. Brennan Saeta, Mark Diaz. Orhan Firat, Michele Catasta, Jason Wei, Kathy Meier-Hellstem, Douglas Eick, Jeff Dean, Slav Petrov, and Noah Fiedel. 2022. Palm: Scaling language mod- eling with pathways. CoRR, abs/2204.02311.
Jonathan Berant, Andrew Chou, Roy Frostig, and Perey Liang. 2013. Semantic parsing on freebase from question-answer pairs. In Proceedings of the 2013 Conference on Empirical Methods Natural Lan- guage Processing, EMNLP 2013, 18-21 October 2013, Grand Hyall Seattle, Seattle, Washington, USA, A meeting of SIGDAT, a Special Interest Group of the ACL, pages 1544,1544, ACL.
Antonia Creswell and Murray Shanahan. 2022. Faith- ful reasoning using large language models. CoRR, 1es/2208.14271.
Emily Dinan, Stephen Roller, Kurt Shuster, Angela Fan, Michael Auli, and Jason Weston. 2019. Wizard of wikipedia: Knowledge- -oweeedd conversational agents. In 7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019. OpenReview.net.
Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan. Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert- Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler. Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Fric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ily Sutskever, and Dario Amodei. 2020. Language models are few-shot learners. In Ad- vances in Neural Information Processing Systems 33: Anmual Conference on Neural Information Process- ing Systems 2020, NeurlPS 2020, December 6- 2020, virtual.
Martin Fajcik, Martin Docekal, Karel Ondrej, and Pavel Smrz. 2021. R2-D2: A modular baseline for open- domain question answering. In Findings of the Asso- ciation Computational Linguisties: EMNLP 2021, Virtual Event / Punta Cana, Dominican Republic, 16- 20 November, 2021, pages 854-870. Association for Computational Linguistics.
Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q. Wein- berger. 2017. On calibration of modern neural net- works. In Proceedings of the 34th International Con- ference on Machine Learning, ICML 2017, Sydney, NSW, Australia, 6-11 August 2017, volume 70 of Proceedings of Machine Learning Research, pages 1321-1330. PMLR.
Danqi Chen, Adam Fisch, Jason Weston, and Antoine Bordes. 2017. Reading wikipedia to answer open- domain questions. In Proceedings of the 55th Annual Meeting of the Association for Computational Lin- guistics, ACL 2017, Vancouver, Canada, July 30- August 4, Volume 1: Long Papers, pages 1870-1879. Association for Computational Linguisties.
Pengcheng He, Xiaodong Liu. Jianfeng Gao, and Weizhu Chen. 2021. Deberta: decoding-enhanced bert with disentangled attention. In 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021. OpenRe- view.net.
Hung-Ting Chen, Michael J. Q. Zhang, and Eunsol Choi. 2022. Rich knowledge sources bring complex knowledge conflicts: Recalibrating models to reflect conflicting evidence. In Proceedings the 2022 Con- ference on Empirical Methods in Natural Language Processing, EMNLP 2022, Abu Dhabi, United Arab Emirates, December 7-11, 2022, pages 2292-2307. Association for Computational I. Linguisties.
Gautier Izacard, Mathilde Caron, Lucas Hosseini, Se- bastian Riedel, Piotr Bojanowski, Armand Joulin, and Edouard Grave 2021. Towards unsupervised dense information retrieval with contrastive learning. CoRR, abs/2112.091118.
Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, Parker Schuh, Kensen Shi, Sasha Tsvyashchenko, Joshua Maynez, Abhishek Rao, Parker Barnes, Yi Tay, Noam Shazeer, Vin- odkumar Prabhakaran, Emily Reif, Nan Du, Ben Hutchinson, Reiner Pope. James Bradbury, Jacob Austin, Michael Isard, Guy Gur-Ari, Pengcheng Yin, Toju Duke, Anselm Levskaya, Sanjay Ghemawat, Sunipa Dev, Henryk Michalewski, Xavier Garcia, Vedant Misra, Kevin Robinson, Liam Fedus, Denny Zhou, Daphne Ippolito, David Luan, Hyeontaek Lim, Barret Zoph, Alexander Spiridonov, Ryan Sepassi,
Gautier Izacard and Edouard Grave 2021a. Distilling knowledge from reader to retriever for question an- swering. In 9th International Conference on Learn- ing Representations, ICLR 2021, Virtual Event, Aus- tria, May 3-7, 2021. OpenReview.net.
Gautier Izacard and Edouard Grave. 2021b. Leveraging passage retrieval with generative models for open do- main question answering. In Proceedings of the 16th Conference of the European Chapter of the Associ- ation for Computational Linguisties: Main Volume, EACL 2021, Online, April 19 23, 2021, pages 874- 880. Association for Computational Linguistics.

---

Gautier Izacard, Patrick Lewis, Maria Lomeli, Lucas Hosseini, Fabio Petroni, Timo Schick, Jane Dwivedi- Yu, Armand Joulin, Sebastian Riedel, and Edouard Grave. 2022. Few-shot learning with retrieval augmented language models. CoRR, abs/2208.03299.
through eseudo-evidentiality training. In Proceed- ings the 59th Annual Meeting of the Association for Computational Linguistics and the 1 11th International Joint Conference on Natural Language Processing, ACL/UCNLPP 2021. (Volume I: Long Papers), Virtual Event, August 1-6, 2021, pages 61 10-61 19. Associa- tion for Computational Linguistics.
Ziwei Ji, Nayeon Lee, Rita Frieske, Tiezheng Yu, Dan Su, Yan Xu, Etsuko Ishii, Ye Jin Bang. Andrea Madotto, and Pascale Fung. 2023. Survey of halluci- nation in natural language generation. ACM Comput. Surv. 55(12).
Nayeon Lee, Wei Ping, Peng Xu, Mostofa Patwary. Mohammad Shoeybi, and Bryan Catanzaro. 2022, Factuality enhanced language models for open-ended text generation. CoRR, 0bs/2206.046244
Zhengbao Jiang, Jun Araki, Haibo Ding, and Graham Neubig. 2021. How can we know When language models know? on the calibration of language mod- els for question answering. Trans. Assoc. Comput. Linguistics,
Daliang Li, Ankit Singh Rawat, Manzil Zaheer, Xin Wang, Michal Lukasik, Andreas Veit, Felix X. Yu. and Sanjiv Kumar. 2022a. Large language mod- els with controllable working memory. CoRR, abs/2211.05110.
Mandar Joshi, Eunsol Choi, Daniel S. Weld, and Luke Zettlemoyer. 2017. Triviaqa: A large scale distantly supervised challenge dataset for reading comprehen- sion. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, ACL 2017, Vancouver, Canada, July 30 August 4, Volume Long Papers. pages 1601-1611. Association for Computational Linguistics.
Junlong Li, Zhuosheng Zhang, and Hai Zhao. 2022b. Seof-promp large language models for open- domain QA. CoRR, abs/2212.08635.
Jiacheng Liu, Skyler Hallinan, Ximing Lu, Pengfei He, Sean Welleck, Hannaneh Hajishirzi, and Yejin Choi. 2022a. Rainier: Reinforced knowledge introspector for commonsense question answering. In Proceed- ings of the 2022 Conference on Empirical Methods in Natural Language Processing. EMNIP 2022, Abu Dhabi, United Arab Emirates, December 7-11, 2022, pages 9938-8958. Association for Computational Linguistics.
Vladimir Karpukhin, Barlas Oguz, Sewon Min, Patrick S. H. Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen-tau Yih. 2020. Dense passage retrieval for open- gpen-domain question answering. In Proceedings of the 2020 Conference on Empirical Methods in Nat- ural Language Processing, EMNLP 2020, Online, November 16-20, 2020. pages 7769-6781. Associa- tion for Computational Linguistics.
Jiacheng Liu, Alisa Liu, Ximing Lu, Sean Welleck, Pe- ter West, Ronan Le Bras, Yejin Choi, and Hannaneh Hajishirzi. 2022b. Generated knowledge prompting for commonsense reasoning. In Proceedings of the 60th Annual Meeting of the Association for Compu- tational Linguistics (Volume 1: Long Pupers), ACL 2022, Dublin, Ireland, May 22-27, 2022, pages 3154- 3169. Association for Computational Linguistics.
Harold W Kuhn. 1955. The hungarian method for the assignment problem. Naval research logistics quar- terly. 2(1-2):83-97
Harold W Kuhn. 1956. Variants of the hungarian method for assignment problems. Naval research logistics quarterly, 3(45-253-258.
Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Man- dar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Roberta: robustly optimized BERT pretraining approach. CoRR, abs/1907. 11692.
Tom Kwiatkowski, Jennimaria Palomaki, Olivia Red- field, Michael Collins, Ankur P. Parikh, Chris Alberti, Danielle Epstein, Illia Polosukhin. Jacob Devlin, Ken- ton Lee, Kristina Toutanova, Llion Jones, Matthew Kelcey, Ming- Wei Chang, Andrew M. Dai, Jakob Uszkoreit, Quoe Le, and Slav Petrov. 2019. Natu- ral questions: a benchmark for question answering research. Trans. Assoc. Comput. Linguistics, 7:452- 466.
Shayne Longpre Kartik Perisetla, Anthony Chen, Nikhil Ramesh, Chris DuBois, and Sameer Singh. 2021. Entity-based knowledge conflicts in question answering. In Proceedings of the 2021 Conference Empirical Methods in Natural Language Process- ing, EMNLP 2021, Virtual Event Punta Cana, Do- minican Republic, 7-11 November, 2021, pages 7052- 7063. Association for Computational I Linguistics.
Kenton Lee, Ming- Wei Chang, and Kristina Toutanova. 2019. Latent retrieval for weakly supervised open do- main question answering. In Proceedings of the 57th Conference of the Association for Computational Lin- guistics, ACI. 2019, Florence, Italy, July 28- August 2, 2019, Volume 1: Long Papers, pages 6086-6096. Association for Computational Linguistics.
Kaixin Ma, Hao Cheng, Yu Zhang. Xiaodong Liu, Eric Nyberg, and Jianfeng Gao. 2023. Chain-of-skills: A configurable model for open-domain question an- swering. CoRR, 0os/2305.03130.
Lucie Charlotte Magister, Jonathan Mallinson, Jakub Adámek, Eric Malmi, and Aliaksei Severyn. 2022. Teaching small language models to reason. CoRR. abs/2212.08410
Kyungjae Lee, Seung-won Hwang, Sang-eun Han, and Dohyeon Lee. 2021. Robustifying multi-hop QA

---

Alex Mallen, Akari Asai, Victor Zhong, Rajarshi Das. Hannaneh Hajishirzi, and Daniel Khashabi. 2022. When not to trust language models: Investigating effectiveness and limitations of parametric and non- parametric memories. CoRR, abs/2212.105111.
Human Language Technologies. NAACL-HLT 2021, Online, June 6-11, 2021, pages 2523-2544, Associa- tion for Computational Linguistics.
Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. 2020. Exploring the limits of transfer learning with a unified text-to-text trans- former. J. Mach. Learn. Res., 21:140:1-140:6.
Alex Mallen, Akari Asai. Victor Zhong, Rajarshi Das, Daniel Khashabi, and Hannaneh Hajishirzi. 2023. When not to trust language models: Investigating effectiveness of parametric and non- parametric mem- ories. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Vol- ume I: Long Papers), ACL 2023, Toronto, Canada, July 9414, 2023. pages 9802 9822. Association for Computational I Linguistics.
Pranav Rajpurkar, Jian Zhang. Konstantin Lopyrev, and Percy Liang. 2016. Squad: 100, 000+ questions for machine comprehension of text. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, EMNLP 2016, Austin, Texas, USA, November 1-4, 2016, pages 2383-2392. The Association for Computational Linguisties.
Ella Neeman, Roee Aharoni, Or Honovich, Leshem Choshen, Idan Szpektor, and Omri Abend. 2022. Disentqa: Disentangling parametric and contextual knowledge with counterfactual question answering. CoRR, abs/221 1.06555.
Vered Shwartz, Peter West, Ronan Le Bras, Chandra Bhagavatula, and Yejin Choi. 2020. Unsupervised commonsense question answering with self-talk, In Proceedings of the 2020 Conference on Empirical Methods Natural Language Processing. EMNLP 2020, Online, November 16-20, 2020, pages 4615- 4629. Association for Computational Linguistics.
Barlas Oguz, Xilun Chen, Vladimir Karpukhin, Stan Peshterliev, Dmytro Okhonko, Michael Sejr Schlichtkrull, Sonal Gupta. Yashar Mehdad, and Scott Yih. 2022. Unik-qa: Unified representations of structured and unstructured knowledge for open- domain question answering. In Findings of the Asso- ciation for Computational Linguistics: NAACL 2022, Seattle, WA, United States, July 10-15, 2022, pages 1535-1546. Association for Computational Linguis- tics.
Zhiqing Sun, Xuezhi Wang, Yi Tay, Yiming Yang, and Denny Zhou. 2022. Recitation-augmenteed language models. CoRR, abs/2210.01226.
James Thorne, Andreas Vlachos, Christos Christodoulopoulos, and Arpit Mittal. 2018. FEVER: a large-scale dataset for fact extraction and verification. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACI-HIT 2018, New Orleans, Louisiana, USA, June 1-6, 2018, Volume (Long Pupers). pages 809-819. Association for Computational Linguistics.
OpenAl. 2023. GPT-4 technical report. CoRR, abs/2303.08774.
Long Ouyang. Jeff Wu, Xu Jiang, Diogo Almeida, Car- roll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman. Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul F. Christiano, Jan Leike, and Ryan Lowe. 2022. Training language models to follow instructions with human feedback. CoRR, abs/2203.02155.
Harsh Trivedi, Niranjan Balasubramanian, Tushar Khot, and Ashish Sabharwal. 2022. Interleav- ing retrieval with chain-of-thought reasoning for Bhowledge-intensse multi-step questions. CORR. abs/2212.10509
Liangming Pan, Wenhu Chen, Min-Yen Kan, and William Yang Wang. 2021. Contraqa: Question answering under contradicting contexts. CoRR, abs/2110.07803.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. Advances in Neural Information Pro- cessing Systems 30: Annual Conference on Neural Information Processing Systems 2017, December 4-9, 2017, Long Beach, CA, USA, pages 5998-6008
Baolin Peng. Michel Galley, Pengeheng Hc, Hao Cheng, Yujia Xie, Yu Hu, Qiuyuan Huang, Lars Liden, Zhou Yu, Weizhu Chen, and Jianfeng Gao. 2023. Check your facts and try again: Improving large language models with external knowledge and automated feed- back. CoRR, abs/2302.128115
Wenya Wang, Vivek Srikumar, Hanna Hajishirzi, and Noah A. Smith. 2022. Ellaooaaioonggeeeaae com- monsense question answering at scale. CoRR, abs/2209.01232..
Fabio Petroni, Aleksandra Piktus, Angela Fan, Patrick S. H. Lewis, Majid Yazdani, Nicola De Cao, James Thorne, Yacine Jemnit., Vladimir Karpukhin, Jean Maillard, Vassilis Plachouras, Tim Rocktäschel, and Sebastian Riedel. 2021. KILT: a benchmark for knowledge intensive language tasks. In Proceedings of the 2021 Conference of the North American Chap- ter of the Association for Computational Linguistics:
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Ed H. Chi, Quoc Le, and Denny Zhou. 2022. Chain of thought prompting elicits reasoning in large language models. CoRR, 1

---

Peter West, Chandra Bhagavatula, Jack Hessel, Jena Hwang. Liwei Jiang. Ronan Le Bras, Ximing Lu, Sean Welleck, and Yejin Choi. 2022. Symbolic knowledge distillation: from general language mod- els to commonsense models. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Hu- man Language Technologies, NAACL 2022, Seattle, WA, United States, July 10-15, 2022, pages 4602- 4625. Association for Computational Linguistics.
Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pier- ric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, and Jamie Brew. 2019. Huggingface' 's transformers: State-of-the-art natural language processing. CoRR, abs/1910.03771.
Wenhan Xiong, Xiang Lorraine Li, Srini Iyer, Jingfei Du. Patrick S. H. Lewis, William Yang Wang, Yashar Mehdad, Scott Yih, Sehastian Riedel, Douwe Kiela, and Barlas Oguz. 2021. Answering complex open- domain questions with multi-hop dense retrieval. In 9th International Conference on Learning Represen- tations, ICIR 2021, Virtual Event, Austria, May 3-7, 2021. OpenReview.net.
Zhilin Yang. Peng Qi. Saizheng Zhang, Yoshua Ben- gio, William W. Cohen, Ruslan Salakhutdinov, and Christopher D. Manning. 2018. Hotpotqa: A dataset for diverse, explainable multi-hop hop question answer- ing. In Proceedings of the 2018 Conference on Em- pirical Methods in Natural Language Processing, Brussels. Belgium, October November 4, 2018, pages 2369-2380. Association for Computational Linguistics.
Wenhao Yu, Dan Iter. Shuohang Wang, Yichong Xu, Mingxuan Ju, Soumya Sanyal, Chenguang Zhu, Michael Zeng, and Meng Jiang. 2023. Generate rather than retrieve: Large language models I1th International Con- ference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023. Oee
strong context generators.
Wenhao Yu, Chenguang Zhu, Zaitang Li, Zhiting Hu, Qingyun Wang, Heng and Meng Jiang. 2022. survey of wwwledge-enhaneee text generation. ACM Comput. Swre. 52:11)))227:112211
Zhuosheng Zhang, Aston Zhang, Mu Li, and Alex Smola. 2022. Automatic chain of thought prompting in large language models. CoRR, abs/2210.0349))
Wayne Xin Zhao, Kun Zhou, Junyi Li. Tianyi Tang. Xiaolei Wang, Yupeng Hou, Yingqian Min, Be- ichen Zhang, Junjie Zhang, Zican Dong. Yifan Du. Chen Yang, Yushuo Chen. Zhipeng Chen, Jinhao Jiang, Ruiyang Ren, Yifan Li, Xinyu Tang, Zikang Liu, Peiyu Liu, Jian- Yun Nie, and Ji-Rong Wen. 2023. A survey of large language models. CoRR, 10//3333.18223

---

A Preliminaries
HotpotQA (Multi-hop QA)
Question: Gail Matthius co-anchored the Weekend Update segment "Saturday Night Live" with the actor who played the Nicholas Andre what movie?
A.1 Analysis of Reader's Predictions under Knowledge Conflicts
Answer: Dumb and Dumber
We are interested in the question: does FiD FiD-based reader model prefer LLM- generated passages over retrieved ones for making predictions under knowl- edge conflicts? We compare the outputs of two models: one is fed with retrieved passages only, and the other takes the concatenation of retrieved passage and generated passage as input (i.e., Direct Merging shown in Figure 2a). We focus the examples from dev set of NaturalQuestions where knowledge conflicts probably exist. Here we se- lect questions whose retrieved passages contain correct answer while LLM-generated passages do not. Specifically, we obtain a subset of questions with more then 20% of conflicting passage pairs (i.e., conflicting rate defined in Equation 3). On this subset, the Retrieved-per Retrieved-passage-nnl model is not affected by knowledge conflicts since it does not have access to LLM LLM- generated passages.
Prediction: The Goonies
Retrieved Passage
1. (Gail Matthius). co-enchored the Weekend Update segment with Charles Rocket 1981.
2. [Charles Rocket) He was best known for his role as the villain Nicholas Andre the film "Dumb and Dumber"...
gatergeeereddd Passage
1. Gail Matthius an American actress ... She also co anchored the
Weekend Update Robert Davi
of the show "Saturday Night Live".
perhaps known for 'The Goonies" where he played the character Nicholas Andre.
movies such
(Izacard and Grave, 2021b) QA model is misled by the knowl- edge conflict between retrieved passage and halluci- nating LLM-generater passage example from HotpotQA (Yang et al., 2018).
Figure 8: A
as A (R). We use Fusion-in in-Decoder (FiD) (Izacard and Grave, 2021b), state-of-the- art cttreval-auugmente generation model, as our reader model. FiD a sequence-to-sequees chitecture based on the pretrained T5 models (Raf- fel et al., 2020). It first encodes separately each passage appended to the question, resulting in representation P = T5-eencoer((,,1 rp;). Then the decoder attends to the concatenation of rep- resentations of all passages and generates the an- swer: A T5-decoder(pp P2..... ... P.v). The cross- attention mechanism of the decoder enables FiD to perform evidence aggregation, i.e., assigning different attention scores to each passage.
We find that on the examples initially predicted method,
ar-
as correct by the detrieved-passage-onee models are fooled by adding LLM--gnnrated pas- sages on 16% of them (i.e., Direct Merging method gives the wrong predictions). The performance drop shows that there is still room for improve- ment for FiD-based reader to resolve conflicts on its own, thus calling for the need of our compatibility- oriented QA framework.
Additionally, we observe that 65% of predic- tions are sub-spans at least one of the generated passages. This shows that the reader favors LLM passages more than retrieved passages when con- flict exists. LLM passages often appear to be more relevant to the question and describe the plausible evidence in way more specific to question. We hypothesize that the reader model find it easier to extract answer from LLM-generated passages so tend to rely on them more for predictions.
The recently proposed generate-then-reat pipeline (Yu et al., 2023) simply substitutes the retriever R with an LLM L that generates top-M contextual passages Pz (lp1, [pee....MMM)) ex- pressing knowledge stored in its parameters. Then these LLL--geerrteed passages are handled by the reader give the final answer: A = G(Q, Pt)). Yu et al. (2023) also explore a simple approach (Figure 2a) of directly merging passages from both sources together: A = G(Q,Pz UPR), regardless of knowledge conflicts between P, and Pi--
A.2
RetrréveeGGeneratt--hee--rdd Open-domain QA Framework
The task of open-domain QA is typically solved by a retrieve-then-reant framework (Karpukhin et al., 2020) consisting of two modules: 1) retriever R that fetches top-N relevant passages PR - (rPi, (rp...p....., rP,,.., }pN] to the question Q from a large corpus UR such as Wikipedia and 2) a generative reader & that generates the answer A conditioned on retrieved passages PRt denoted
A.3 Extension of Compatibility Definition from Single-hop to Multi-hop QA
Different from the single-hop QA setting, our trieved item under the multi-hop QA setting is ac- tually passage chain rej (rp).rpf.... rp?....,,, rp))) instead of a single passage rPj. following the com- mon practice proposed by Xiong et al. (2021).

---

Settings
Single-hop QA
Multi-hop QA
Assumptions
Retrieved passages factual.
Retrieved passage chains are factual.
LLM- --geerrrtedd passages contain relevant evi-
LLM-generate passage chains contain rele-
dence to the question.
vant reasoning path to the question.
Definitions
Given a question Q, LLM--generated passage Ips
Given question Q I.L.-gennrated passage chain
and retrieved passage Tp; are
le; and retrieved passage chain re; are...
COMPATIBLE if both ip; and rp; contain the
COMPATIBLE if both le, and re; contain the
correct evidence to support ground-trude answers of the question Q
correct reasoning path to support the ground-truth
answers of the question Q
CONFLICTING if rp; contains correct evidence
CONFLICTING if rc; contains correct reasoning
while Ip; contains incorrect evidence
path while le, contains incorrect reasoning path
CON-EVIDENTIAL if rp, does not contain
NON-EVIDENTIAN if rey does not contain correct reasoning path
correct evidence
Table 4: Comparision of definitions and assumptions for compatibility between single-hop and multi-hop QA settings.
Specifically, in HotpotQA (Yang et al., 2018), all questions are supposed to 2-hop, so a passage chain ray is essentially an ordered pair of passages (rp}. rp}) that provides sufficient evidence for an- swering Q. An example of the retrieved passage chain is shown in Figure 8. We modify the as- sumptions and definitions for the multi-hop setting accordingly, shown in Table 4. Specifically, one could simply replace rPj with rcj and Ip; with le; throughout this paper to obtain the same conclu- sion. The modifications of definitions from the single-hop to the multi-hop setting are minimal yet reasonable, allowing for a unified concept formu- lation and system implementation to handle both single-hop and multi-hop QA.
incorrect, meaning that these passages suffer from the hallucinations of LLM and pose risk to the reliability of QA readers. This highlights the chal- lenge of knowledge merging and the importance of our compatibiliy--rrinneedd matching framework COMBO.
B Dataset Details
We use three single-hop QA datasets (NaturalQues- tions (Kwiatkowski et al., 2019), TriviaQA (Joshi et al., 2017), WebQuestions (Berant et al., 2013)) and one multi-hop (HotpotQA (Yang et al., 2018)) QA dataset the testbeds for evaluating our method. NaturalQuestions consists of real-world information seemmatton--eekingg queries issued to the Google search engine and their corresponding long answers (gold evidence passage) and short answers (one or more entities). We use the open-domain version created by Lee et al. (2019) which only keeps ques- tions and short answers with less than five tokens. TriviaQA includes questions from trivia and quiz- league websites. WebQuestions contains questions from Google Suggest API that queries entities in Freebase. HotpotQA contains questions requiring reasoning over multiple Wikipedia documents to answer. We focus on the fullwiki setting where QA systems need to retrieve relevant evidence from the whole Wikipedia corpus. For train / dev / test splits, we use the same setting Karpukhin et al. (2020) for NaturalQuestions, TriviaQA, and We- buuestions, and the official leaderboard version!! for HotpotQA.
A.4
Validating Assumptions behind the Compatibility Definition
Scction 3.1, we make two assumptions for com- puting passage compatibility. Assumption ("re- trieved passages are factual") is widely adopted in the existing literature (Ji et al., 2023; Lee et al., 2022; Thorne et al., 2018) when retriev- ing from Wikipedia corpus. For Assumption 2 passages contain relevant evi- dence to the question"), the authors manually anno- tate 100 random examples from the NaturalQues- tions (Kwiatkowski et al., 2019) dataset to verify the plausibility of LLM's generations. For each example, we first determine whether it contains a plausible answer If yes, we further compare it to the ground-truth answer search Wikipedia label it as hallucinating or not. We show ex- amples of annotations in Table 5. We find that 94% of them contain plausible answers, thus sup- porting our assumption. Of all the passages that contain plausible answers, 51 % of the answers are
hhttps://hotpootgg.itttt io/

---

Type
Question
Gold An- swer
LLM-generaled Passage
Plausible
Plau.
Hallu,
Pere.
Answer
Who played Floyd the Andy Griflith show?
Howard Terbell McNear
Floyd Lawson recurring charac- The Andy Griflthh Show. He played by Walter Brennen.
Walter
Y
Y
48%
Brennen
11 When did the new 3DS XL come out?
Fehruary 13, 2015
The New Nintendo 3DS XI. was leased on February 13, 2015, in Japan and North America.
Fehruary 13, 2015
N
46%
III Who shifted the capital from Calcutta to Delhi?
Government of British India
In 1911. the British imperial capital was shifted from Calcutta to Delhi. This was done as strategic move to make administration more efficient. The move was also seen as an effort co-opt and acculturate the growing Indian middle class into the colonial project.
N
6%
Table 5: Examples of annotations for plausibility (Plau.) and hallucinations (Hallu.) of L.M-generatee passages. Analysis is performed on 100 random samples from the dev set of NaturalQuestion.
C Implementation Details
C.1 Silver Evidentiality Label Mining
We use the same reader G, for consistency label mining in Section 3.2. Inspired by Asai et al. (2022), we consider rPj as evidential for Q if I) the prediction based on top-N retrieved passages (Q, PR) is correct and 2) Go(Q, PR (rp;)) the prediction based top-N retrieved passages except rPj, is incorrect. rpj thus believed to contain the correct evidence since removing itself makes the predictions change from right to wrong. Sim- ilarly, we consider TPj as non-evidential for Q is
Question: who plays Charlotte in "The Strain" season
Answer: Rhona Miltr
Retrieved Passage 1
[Rhona Mitra) the played Charlatte in the fourth season of "The Strain" TV television
2018,
Retrieved Passage
(Lauren Coilins) 2013, she appeared recurring guest role in the opcoming fourth the veries Strain"
Retrieved Passage
Strain (TV The Strain
harror television
that aired from July 13, to September 2017
LLM JJM-generated Passage 7
horror television in the series ... She is played by Alexandra Breckennide season 4.
Strain
character
if PR) is incorrect and correct, which indicates adding rpj to the rest of input passages misleads the model.
RP
RP
RP3
Rhona Mitra
RP
RP
Finctuncd
Louren Collinn
Base
RP1
RP
RP3
LPP
Reader
Alexandra Breekenridge
C.2 Model Implementations and Hyperparameters
RP
RP3
IP7
Alexandra Breekenridge
Conflicting
RP
LP7
For LLM-generated passages, use the ones pro- vided by Yu al. (2023)12 for the three single- hop QA datasets. They prompt InstructGPT (text-davinci-ee)) (Ouyang et al., 2022) with the prompt "Provide a background document from Wikipedia to answer the given question. \n'n [question) \n\n" and use sampling to generate 20 documents for each question. Additionally, we prompt ChatGPT Sept-3.5-turbon to generate contextual passages for HotpotQA. We use the prompt "You are assistant designed to provide chain of two 100-word documents from Wikipedia
Figure Overview and examples our proposed ap- proach for mining silver conflicting labels. For pair of target retrieved and LLM-ggnerated passages (c.g., s, RP 1 and LP in this Figure), create four types of input by I. using all the retrieved passages, II. masking target retrieved passage, LLM-generated passage. IV. doing both II and III. We obtain predictions by a finetuned reader model when using different types of input. If the prediction cor- rect with I but all incorrect with II, II, IV, we label it as a conflicting pair. See Figure in Section 3.2 for an example of consistent pair.
appending the target

---

HQA Q
HQA bridge Q
NQ
TQA
WebQ
Methods
test
test
test
dev
Direct Merging
52.70.60
61.20.58
74.2a.
80.5p.
51.10.30
58.5o.
61.60.50 77.11.224
(5..8..0 00 /7..0..0
Random Matching
53.30.35
15.96.15
74.2a.
80.7a.10
51.61.55
58.61.57
57.7a.00 60.0o. 61.60.17 74.2 58.00.00 /72.30.00
61.5o.37
1o.0
/ n.00
COMBO (ours)
54.2o.26 62.70.21
74.60.00 09
880.9.05
53.0p.10
Table 6: Exact Match (EM) FI scores on NaturalQuestions (Kwiatkowski et al., 2019), TriviaQA (Joshi al.,
2017), WebQuestion (Berant al., 2013), and HotpotQA (Yang et al., 2018). We conduct three runs with different random seeds and report the average and standard deviation.
that can be combined together to answer the user's question. Here's an example of your output format: Document 1: ""An\n Document 2: ""\n'n {question}". We then parse the output to get the generated passage chains. It costs around 400 US dollars to generate 10 pas- sage chains per question with ChatGPT for Hot- potQA.
HQQ
HQA bridge Q dev
NQ test
TQA
WebQ
Methods
all Q dev
test
test
COMBO (ours)
54.2
74.6
53.0
61.6
58.0
Same-answer matching
55.6
74.8
53.5
62.4
58.5
SOTA
64.0''
86.12
57.8
66.22*
Table Exact Match (FMM results on NaturalQues- tions (Kwiatkowski et al., 2019), TriviaQA (Joshi et al.. 2017), WebQuestion (Berant et al., 2013) and Hot- potQA (Yang al., 2018) for an oracle setting with access to ground-truth answers during inference. State- of-the-art (SOTA) systems - ATLAS (Izacard et al.. 2022), PPALM-2 (Anil et al., 2023), UNIK-QA (Oguz et al., 2022), 'COS (Ma et al., 2023) - leverage larger models (up to 340B) additional knowledge.
For both the evidentiality discriminator D3 and consistency discriminator De. we initialize the model from the pretrained RoBERTa-large 13 for single-hop QA and DeBERTa-large" for multi- hop QA from huggingface (Wolf 2019). We set the sequence length of RoBERTa as 512 and DeBERTa as 1024 since each passage chain HotpotQA contains two passages. The training is performed on two A40 GPUs and the batch size is 16 per GPU. Peak learning rate set to le-5 for RoBERTa and 5e-6 for DeBERTa. We use the Adam optimizer and línear schedule of learning rate. We train the model for epochs and select the best epoch checkpoint based on the Fl score of the silver dev set, which is mined from the dev set of the original corresponding dataset. We apply sample weights in the cross-entropy loss function to handle imbalanced classes in the silver training data.
accumulation step to 16 to imitate large batch size. The learning rate is lc-4 with 2000 warmup steps and linear scheduler. Adam is the optimizer and the dropout probability is 0.1.
D Additional Experimental Results and Analysis
D.1 More Results
We also compare our proposed compatibility- guided optimal matching algorithm to an oracle setup for passage matching, in addition to the Ran- dom Matching baseline mentioned in Table 1. We first match passages into pairs if both of them tain the ground truth answer string and randomly pair the rest. We call same-answer matching. Although an answer-containe unswer-containing passage does not necessarily contain the correct evidence (Asai et al., 2022), this setting still gives us a rough estimate of the upper bound of the model's performance. Results shown in Table 7. We show that our method is close to the upper bound, indicating that our compatibility discriminators are good at identi- fying passages that contain the correct evidence.
We employ FiD as the backbone architecture for our reader model in this paper. We train the models for 100k steps using four A40 GPUs with 46 GB memory and save checkpoints every 10k steps. The best checkpoint is selected based on the EM score on the dev set. For pairwise input, we set the max input length per passage pair as 400 for single-hop QA and 1000 for multi-hop QA. For input, we the max input length per passage as 200 for single-hop QA and 500 for multi-hop hop QA. Per GPU batch size is and we set the gradient
passage
1tt "https://huggingffas http://neaacee..ooorrrtttaaaaa 19 "https: http://hggggaaaaa deberta-v3- -a
Table 7 also includes toitee-f--hh-att results on these datasets. It should be noted that ot

---

NaturalQuestions
Type Nor-evidential Compatbbe Confricting
Compatible Conflicting
Non-evidential
Attemtion Score
[10, 20)
M)
(0)
(20,50)
-6
20
40
60 Frequency
80
100
(10. 101
(10.20) (20,20)
(M.M)
(20,50)
Figure 10: The relative frequeney of each type of pair on the test set of NaturalQuestions w.r. the number of LLM-generated passages (M) and retrieved passages (N) for each question. Complement to Figure 5, it shows that more knowledge conflicts arise from the increased number of input passages.
Figure 11: The attention score distribution of differ- ent types of pairs on the test set of NaturalQuestions w.t.t the number of LLM-generated passages (M) and retrieved passages (N), Our model assigns higher atten- tion scores compatible pairs than incompatible ones.
D.3 Distribution of Pairwise Relationships
the-art systems for these datasets leverage readers with much larger size (up 340B) (Izacard al., 2022: Anil et al., 2023) or incorporate more in- put retrieved passages (Ma et al., 2023) and ad- ditional knowledge in different formats (e.g., Ta- bles, KBs) (Oguz et al., 2022). Therefore, it is not apple-to-apple comparison between our models and theirs. However, knowledge merging in principle benefit wide range of open-domain QA system so our contributions should be complemen- tary other modules.
In Figure 10, we show the distribution of the rela- tionships of passage pairs (compatible vs. conflict- ing vs. non-evidential) determined by our eviden- tiality and consistency discriminators. We observe trend that as we increase the number of input LLM passages and retrieved passages, the percent- age of compatible pairs decreases, which shows that more knowledge conflicts arise. This could count for the larger improvements of our COMBO framework over direct merging with more passages as input, which shown Figure 5.
retrievers and reader
D.4 Analyzing Attention Distributions Over Different Types of Passage Pairs
D.2 Comparison Questions from HotpotQA are Less Sensitive to Hallucinations in I.LM-generated Passages
In Figure 11, we show the distribution of attention scores for each type of passage pair across differ- ent numbers of input LLM-generated and retrieved passages. Following Izacard and Grave (2021a). the attention score for passage pair is obtained by averaging the pre-normalized attention score of all tokens in the passage pair, all layers and all heads of the decoder. We find that COMBO model gen- crally assigns higher attention scores to compatible pairs than incompatible ones. This indicates that it learns to prioritize compatible information when making its predictions, which mitigates the harmful impact of hallucination errors.
A comparison question is one of the two types of question in HotporQA, testing models' ability shared proper-
compare entities ties (Yang et al., 2018). This type of question is less sensitive to hallucinations in LLM--generaed pas- sages than the bridge question. For example, for comparison question "who is younger, Keith Bostic or Jerry Glanville?", the CharGPT-generated pas- sage chain is "Document 1: Keith Bostic was born on January 8, 1956 'n Vi Document 2: Jerry Glanville was born October 14, 1941 It con- tains hallucination errors since Keith Bostic was actually born January 17, 1961. Nevertheless. the reader model could still make the correct pre- diction ("Keith Bostic") based on the fabricated evidence.


Reviews:
[{"review_id": "x0Rwcqut0N", "reviewer": "Reviewer_KaNa", "summary": "", "questions": "Question A: Are the discriminators transferable across different dataset?   \nQuestion B: Why not include results of filtering pairs with zero evidentiality?  \nQuestion C: Is the improvement of COMBO over “Random Matching” statistically significant?  \nQuestion D: The definition of conflict rate is a bit confusing. Passages not containing the answer are not necessarily contradicting the gold passages; they might just not include the information needed for answering the question.   \nQuestion E: What’s the percentage of predictions that are sub-spans of at least one of the retrieved passages? Some predictions could potentially be sub-spans of both generated and retrieved passages, so 65% does not necessarily mean the reader favors generated passages.  \nQuestion F: The authors describe a subset of examples where the initial predictions are correct using Retrieved-passage-only method, but fooled by adding LLM-generated passages. I wonder how much of these fooled examples could be un-fooled using the proposed method? (Meaning “Retrieved-passage-only” is correct, “Direct Merging” is incorrect, and “COMBO” is correct again).   \n", "limitations": "", "rating": 3, "confidence": 4, "soundness": 5, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for the helpful comments. Please see our responses below. \n\n***Re: Incremental results***\n\nThe experiment results in Table 1 of the submitted paper are based on a single run of models. Considering the potential for random seeds to influence these outcomes, to the best of our efforts, we try to mitigate the uncertainty by repeating each experiment for 3 runs. Here, we offer the average and standard deviation (std.) of Exact Match (EM) metrics, over three single-hop QA datasets. \n\n**Table I**: Exact Match results on NaturalQuestions, TriviaQA, and WebQuestion.\n| Methods    | NQ | TQA | WebQ |\n| -------- | ------- | ------- | ------- | \n| Direct Merging  |  52.4±0.80  |  74.2±0.08 |  51.1±0.30 |\n| Random Matching |   53.3±0.32   |  74.3±0.21 |  51.6±1.55 |\n| COMBO    |  54.2±0.20  | 74.6±0.07  | 52.3±0.60 |\n\nWe observe a similar trend that our proposed method, COMBO, significantly outperforms both baselines. The improvement of Random Matching over Direct Merging accentuates the value of pairwise input formulation in aiding the reader model to encode knowledge from dual sources. Then we take a further step to enhance performance by composing compatible pairs.\n\nWe also note that any potential performance improvements are somewhat constrained by the percentage of the hallucinated (generated) and evidential (retrieved) passages provided. In the extreme scenario where all generated passages are devoid of hallucinations or all retrieved passages lack correct evidence, our method would default to Random Matching, as the compatibility matching score would be uniform. \n\nIn the conditions of our experiment, we did not manipulate the distribution of questions or input passages in any way. However, we anticipate that in more controlled settings, such as PopQA (Mallen et al., 2023) where questions about infrequent entities are examined in more detail, our method could yield more dramatic performance improvements. At present, LLMs appear more susceptible to hallucinating knowledge pertaining to less frequent entities. We eagerly anticipate investigating this aspect further in our future work.\n\n\n***Re: Requires extra training***\n\nTo reduce the effort of training a separate discriminator for each dataset, we additionally investigate the performance of a unified discriminator. Specifically, we train a discriminator on silver labels mined from both NQ and TQA and report its results on NQ and TQA separately (row ‘COMBO’ in Table I). We also demonstrate a use case for smaller datasets where fewer labels could be mined. For results on WebQ of Table I, we further finetune the unified discriminator with WebQ’s data. The performance gains over baselines show the effectiveness of training a generalizable discriminator that works for diverse datasets. \n\nRegarding the efficiency of our pipeline, we argue that discriminator training essentially requires one initial setup prior to its deployment across various datasets. The discriminator model can be any encoder-only model in principle and easily fits into most GPUs available in school labs. We would also make our checkpoints publicly accessible so future NLP practitioners don’t have to train from scratch again. \n\nOne future direction is to employ an LLM with in-context learning to compute the compatibility scores instead of training a discriminator from scratch, thereby eliminating the necessity for (weakly) supervised training. This makes our approach generalizable to a few-shot setting. But it would also require carefully examining whether and which LLM is able to discern the knowledge conflicts between its own parametric memory and retrieved evidence. \n\n\n***Re: Discriminators might not work out-of-domain***\n\nIn “***Re: Requires extra training***”, we demonstrate our discriminators’ generalizability under in-domain setups for datasets of diverse domains. We will add the results of discriminators trained on dataset X and tested on dataset Y in camera-ready. We do anticipate that the out-of-domain performance will not be as good as the in-domain one shown in Table I. \n\nRegarding your concern that “*it is difficult to quantify the discriminators’ performances*”, we encourage you to check out Appendix D.4, where we conduct a human evaluation of the discriminators’ performance. We find that our discriminators yield an overall accuracy of 78% on their predicted labels. It shows that our discriminators are capable of providing signals of passage compatibility to the reader model for making well-informed decisions.\n\nIn Table I, we show that “*for datasets with fewer silver labels*”, e.g. WebQ, we can warmup the discriminator training with silver labels mined from other similar datasets. \n\n***Re: Question A***\n\nWe plan to incorporate the results of the transferred discriminator (trained on NQ and tested on TQA) in camera-ready in case of acceptance. Please also see “***Re: Requires extra training***” for a unified discriminator experiment.\n\n\n***Re: Question B***\n\nIn the early stage of this project, we experimented with some simple strategies for filtering passages based on the evidentiality and consistency scores, but they did not work well empirically due to the noise of discriminators trained on silver labels. For simplicity, we include all passages in this project, leaving it to future work to filter passage pairs. We note that previous work (Asai et al., 2022) also does not filter non-evidential passages but instead includes all of them during prediction.\n\n\n***Re: Question C***\n\nFor the paper results based on a single run of the model, the improvement of COMBO over “Random Matching” is not statistically significant. However, randomness of model training may play a role here so we encourage you to check out Table I for more reliable results based on 3 runs. We highlight that the average performance of COMBO is still higher than Random Matching.\n\n\n***Re: Question D***\n\nWe would like to clarify the assumptions behind the definition of the conflicting rate. According to Assumption 2 (lines 215-217) and our empirical observations in Appendix A.4, the LLM-generated passage almost always contains a plausible answer to the question, be it factual or hallucinated. Based on this assumption, LLM passages not containing the gold answer ($M-M_{A}$) are likely to contain a hallucinated answer, which contradicts the evidential retrieved passages. We believe you try to argue this point for the *retrieved* passages, which “*may not include the information needed for answering the question*”. Therefore, we still think our definition is conceptually valid. However, our assumptions may not always hold true so we agree that the conflicting rate is only a proxy for the actual situation.\n\n\n***Re: Question E***\n\n26% of predictions are sub-spans of at least one of the retrieved passages on NQ. This indicates a certain level of resilience to knowledge conflicts within the base reader model, however, it could be further improved through our proposed approach.\n\n\n***Re: Question F***\n\nOur proposed method successfully corrects the answers on 41.0% of the examples that fool the model.\n\n\n**References**\n\n[1] Mallen, Alex, et al. \"When not to trust language models: Investigating effectiveness of parametric and non-parametric memories.\" *Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)*. 2023.\n\n[2] Asai, Akari, Matt Gardner, and Hannaneh Hajishirzi. \"Evidentiality-guided Generation for Knowledge-Intensive NLP Tasks.\" *Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies*. 2022.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. The idea is sound: The mathematical derivation of compatibility, evidentiality and consistency is clear and sound. The intuition behind each design is carefully described. \n2. Solid experiments: The authors provided meaningful baselines to the experiments, and report the SOTA results in the appendix. The results come with statistical significance. There are extensive ablation studies and results of using different numbers of passages. The conflicting rate experiment (Table 3) also provides a possible explanation of the performance improvement.\n", "reasons_to_reject": "There are three points in total, ranked from major to minor:\n\n1. Incremental results: The performance improvement over “Direct Merging” is already incremental, and yet the boost over “Random Matching” is even smaller. This indicates that most improvement comes from matching retrieved and LLM-generated passages. Also, Table 6 shows the upper bound is not too far from the proposed approach, meaning little room for improvement. The bottleneck might be generating better related passages. \n2. Requires extra training: Both the evidentiality and consistency discriminator are separately trained on each dataset. This implies that an extra training step is required each time the model is tested on a new dataset. Given the limited performance gain, it is difficult to justify the training cost. \n3. Discriminators might not work out-of-domain: There is no gold label for the discriminators, and thus it is difficult to quantify their performances. However, it is conceivable that using the discriminator trained on NQ might not transfer to TQA, for example. This limits the use case of the framework, and it may not work with datasets with fewer silver labels, as mentioned in the limitation section. \n", "questions_to_authors": "Question A: Are the discriminators transferable across different dataset?   \nQuestion B: Why not include results of filtering pairs with zero evidentiality?  \nQuestion C: Is the improvement of COMBO over “Random Matching” statistically significant?  \nQuestion D: The definition of conflict rate is a bit confusing. Passages not containing the answer are not necessarily contradicting the gold passages; they might just not include the information needed for answering the question.   \nQuestion E: What’s the percentage of predictions that are sub-spans of at least one of the retrieved passages? Some predictions could potentially be sub-spans of both generated and retrieved passages, so 65% does not necessarily mean the reader favors generated passages.  \nQuestion F: The authors describe a subset of examples where the initial predictions are correct using Retrieved-passage-only method, but fooled by adding LLM-generated passages. I wonder how much of these fooled examples could be un-fooled using the proposed method? (Meaning “Retrieved-passage-only” is correct, “Direct Merging” is incorrect, and “COMBO” is correct again).   \n", "paper_topic_and_main_contributions": "The paper proposed a compatibility-based approach for open-domain QA that merges both retrieved and generated knowledge. A framework that follows previous work (Asai et al., 2022) for mining silver labels is used to train a compatibility scorer, which contains two components, evidentiality and consistency. The experiments show that the proposed method improves upon simply merging retrieved and generated knowledge, and extensive ablation studies reveal the usefulness of each component. ", "excitement": 3, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "tEVrruUILz", "reviewer": "Reviewer_mizo", "summary": "", "questions": "1. In lines 490—494, did you try the fixing the order to (rp_j, lp_i)?\n2. How did you generate different passages for the same question using LLMs? Include the details of sampling in the paper.\n3. (This is a suggestion, not a question:) F1 score is a more stable and reliable QA metric than EM. Consider also reporting F1 scores.", "limitations": "", "rating": 3, "confidence": 4, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for the helpful comments. Please see our responses below. \n\n***Re: The performance gain over random matching is quite small***\n\nPlease see our response under “***Re: Incremental results***” to Reviewer KaNa.\n\n***Re: Given the considerable amount of additional effort COMBO requires and also its higher parameter count (due to the two discriminators), such small improvements in quality make me question the practicality of the approach***\n\nPlease see our response under “***Re: Requires extra training***” to Reviewer KaNa.\n\n\n***Re: Question 1***\n\nWe do not experiment with flipping the order of generated and retrieved passages. However, given the total input length is relatively short (less than 400 tokens per pair), we would expect similar results as long as the relative order is fixed.\n\n\n***Re: Question 2***\n\nWe use the same prompt and generate passages by multiple decoding passes with nucleus sampling. We will include detailed hyperparameters of the sampling process in camera-ready. We also refer our readers to the GenRead (Yu et al., 2023) paper for details since we use their provided passages directly.\n\n\n***Re: Question 3***\n\nPer your suggestion, we report the F1 metric as follows. Basically, the trend is similar to EM results.\n\n**Table II**: F1 results on NaturalQuestions, TriviaQA, and WebQuestion.\n| Methods    | NQ | TQA | WebQ |\n| -------- | ------- | ------- | ------- | \n| Direct Merging  |  61.2±0.27  |  80.5±0.11 |  58.5±0.11 |\n| Random Matching |   61.9±0.12   |  80.7±0.08 |  58.6±1.28 |\n| COMBO    |  62.7±0.17  | 80.9±0.04  | 59.3±0.48 |\n\n\n**References**\n\n[1] Wenhao Yu, Dan Iter, Shuohang Wang, Yichong Xu, Mingxuan Ju, Soumya Sanyal, Chenguang Zhu, Michael Zeng, and Meng Jiang. 2023. Generate rather than retrieve: Large language models are strong context generators. 11th International Conference on Learning Representations. 2023", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. The paper proposes a novel and interesting approach to merging retrieved and LLM-generated content for downstream QA.\n2. The experimental protocol is sound and thorough.\n3. The paper is very clearly written — I enjoyed reading it.", "reasons_to_reject": "The performance gain over random matching is quite small (and also apparently not statistically significant). Given the considerable amount of additional effort COMBO requires and also its higher parameter count (due to the two discriminators), such small improvements in quality make me question the practicality of the approach.", "questions_to_authors": "1. In lines 490—494, did you try the fixing the order to (rp_j, lp_i)?\n2. How did you generate different passages for the same question using LLMs? Include the details of sampling in the paper.\n3. (This is a suggestion, not a question:) F1 score is a more stable and reliable QA metric than EM. Consider also reporting F1 scores.", "paper_topic_and_main_contributions": "This paper proposes a new method named COMBO for merging content from retrieved and LLM-generated passages for open-domain QA. A primary goal of the work is to leverage the parametric knowledge of LLMs while making sure they do not contradict with the content of retrieved passages. COMBO uses two discriminators to identify “compatible” pairs of retrieved and LLM-generated passages, and then feeds them to a fusion-in-decoder (FiD) reader to produce the final answer. The authors also present approaches to construct silver training data for the two discriminators. Evaluated on a number of test sets, COMBO outperforms various relevant baselines, including other methods for merging retrieved and LLM-generated content. The authors also present several ablation and qualitative results.\n", "excitement": 3, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Typos_Grammar_Style_and_Presentation_Improvements": "Lines 196—197: “all possible passage pairs” -> “all possible (retrieved, LLM-generated) passage pairs”"}, "extra_text": {}}, {"review_id": "jlQTCHC2TU", "reviewer": "Reviewer_rVZR", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 4, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for the helpful comments. Please see our responses below. \n\n***Re: The paper could further discuss the performance gain compared to the \"Random Matching\" method***\n\nPlease see our response under “***Re: Incremental results***” to Reviewer KaNa.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. The paper is well-written and presents the concepts in a clear and accessible manner. The use of visually appealing illustrations aids in understanding the proposed framework.\n2. The experimental evaluation shows that the proposed COMBO method achieves consistently strong results across various datasets, including both single-hop and multi-hop scenarios.\n3. Through detailed analysis, the paper demonstrates that the COMBO method effectively addresses conflicts between retrieved and generated passages, leading to improved performance compared to direct merging.\n\n", "reasons_to_reject": "While the COMBO method exhibits significant improvements over the direct merging baseline, the paper could further discuss the performance gain compared to the \"Random Matching\" method. Clarifying the margin of improvement in this aspect would provide a more comprehensive perspective on the method's efficacy.", "questions_to_authors": "", "paper_topic_and_main_contributions": "The paper introduces a compatibility-oriented knowledge merging framework (COMBO) that addresses the challenge of merging retrieved and LLM-generated knowledge in open-domain QA [1]. Unlike previous methods that tend to favor generated passages over retrieved ones when conflicts arise, COMBO combines both types of information to promote the usage of factual and relevant content. The authors propose 'compatible scores' based on consistency discriminator and evidentiality discriminator, enabling the identification of compatible pairs of retrieved and generated passages. The experimental results demonstrate that the proposed approach outperforms the direct merging baseline, especially in scenarios with a high conflicting rate.\n\n[1] Generate rather than Retrieve: Large Language Models are Strong Context Generators (Yu et al., ICLR'23)", "excitement": 4, "reproducibility": 5, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper presents an approach for combining LM generated and retrieved support passages in open-domain QA to balance coverage and faithfulness. The paper solves an important problem, and the reviewers all agreed that the approach is sound and that the paper is well-written. The reviewers were moderately excited about this work because the improvements over baselines were small particularly given that the approach requires additional training.

I think this is solid work, and while the improvements are small, it does provide an interesting solution for the coverage-faithfulness tradeoff by combining retrieval and generation.
Decision: Accept-Main