SlowGuess/NAACL_2022_OCR huggingface dataset:
Akari Asai*
Matt Gardner*
Microsoft Semantic Machines
{akari,hannaneh}@cs.washington.edu
mattgardner@microsoft.com
Retrieval-augmented generation models have shown state-of-the-art performance across many knowledge-intensive NLP tasks such as open-domain question answering and fact verification. These models are trained to generate a final output given retrieved passages that can be irrelevant to an input query, leading to learning spurious cues or memorization. This work introduces a method to incorporate evidentiality of passages—whether a passage contains correct evidence to support the output into training the generator. We introduce a multi-task learning framework to jointly generate the final output and predict the evidentiality of each passage. Furthermore, we introduce a new task-agnostic method for obtaining high-quality silver evidentiality labels, addressing the issues of gold evidentiality labels being unavailable in most domains. Our experiments on five datasets across three knowledge-intensive tasks show that our new evidentiality-guided generator significantly outperforms its direct counterpart on all of them, and advances the state of the art on three of them. Our analysis shows that the multi-task learning and silver evidentiality mining play key roles. $^{1}$
Knowledge-intensive tasks, including open-domain Question Answering (QA) and fact verification, require evidence passages related to an input query to be retrieved from a large collection of passages (e.g., Wikipedia). Recently, most successful methods use retrieval-augmented generation (Lewis et al., 2020c; Izacard and Grave, 2021b), which is a pipeline approach of first training a retriever model (Karpukhin et al., 2020) for retrieving passages and then independently training a generator model (Lewis et al., 2020a; Raffel et al., 2020) given the passages.
Figure 1: Examples where a trained generator ignores the evidential passages (evidentiality-positive passages; green rounded rectangles) and makes incorrect predictions from passages that do not provide sufficient evidence (evidentiality-negative passages; red rounded rectangles). The highlighted part indicates the supporting evidence.
Ideally, a model should generate a correct answer given the information presented in evidential passages (Lee et al., 2021) that correctly support the answer and should not be distracted by other passages, even when they happen to contain a string close to the gold answer. However, the disjoint training process in the prior work disregards the evidentiality of passages, leading to generation models that ignore retrieved passages, leverage spurious cues, and generate hallucinations when the context is not evident (Longpre et al., 2021; Xu et al., 2021). In particular, incorrectly-retrieved passages with high lexical overlap to the query can mislead the answer generator (the first example in Figure 1). Adopting heuristics such as answer string matching (Chen et al., 2017) to train a QA model with passages containing the target strings can partially
*Work done in part while at the Allen Institute for AI
1Our code is available at https://github.com/AkariAsai/evidenceuality_qa.
Human Language Technologies, pages 2226 - 2243
solve this problem for some QA tasks. Still, these passages with answer strings might lack evidence (the second example in Figure 1). What is more, such heuristics cannot be applied for open-ended generation or classification tasks (the third example in Figure 1).
In this paper, we introduce a multi-task training framework of answer generation and evidentiality prediction, which is an auxiliary task to predict if a passage provides evidence relevant to the task (evidentiality-positive passages; green passages in Figure 1) or not (evidentiality-negative passage; red passages in Figure 1). Since most existing datasets do not provide evidentiality labels, we introduce a new task-agnostic approach for mining silver evidentiality annotations.
Specifically, we train an evidentiality labeling model that takes an input query, a gold output and a single passage and predicts if the passage supports the gold output or not. After training, the evidentiality labeling model predicts the silver evidentiality labels of all of the passages used for the multi-task training. To supervise this evidentiality labeling, we use a combination of partially available gold passage annotations and data collected by a novel leave-one-out generation approach. This leverages a trained generator model and evaluates the relevance of each passage to a query through the correctness of the generated output when the passage is removed from the pool of retrieved passages. Unlike prior multi-task learning work in QA relying on available annotated data (Lee et al., 2021; Nishida et al., 2019) or heuristics such as answer string matching to label pseudo evidentiality (Fajcik et al., 2021), our approach is applicable to diverse downstream tasks, where we cannot use additional annotations or heuristics. Our evidentiality mining approach for high-quality silver labels can be applied to diverse NLP tasks, and our auxiliary task has a new purpose of evaluating passage evidentiality suitable for the open-retrieval.
We run experiments across representative knowledge-intensive tasks: open-domain QA (Natural Questions Open; Kwiatkowski et al., 2019, TriviaQA unfiltered; Joshi et al., 2017), fact verification (FaVIQ Ambig; Park et al., 2021, FEVER; Thorne et al., 2018) and knowledge-enhanced dialogue (Wizard of Wikipedia; Dinan et al., 2019). Our experiments show large performance improvements across all datasets over the direct counterpart, FiD (Izacard and Grave, 2021b).
Moreover, on the latter two tasks, our model outperforms all previously published models, advancing state of the art on FaVIQ-Ambig, FEVER and Wizard of Wikipedia. Further human evaluations find that the evidentiality labeling model yields  $95\%$  accuracy, and often correctly identifies negative passages spuriously containing answer strings. Our analysis shows that both multi-task learning and silver evidentiality mining contribute to the improvement, helping the generator learn to focus on the more relevant passages.
Problem. Knowledge-intensive tasks (e.g., open-domain QA, fact checking) are designed to retrieve evidence passages related to an input query  $x$  given a large collection of passages such as Wikipedia. Most successful previous work in this domain uses a retrieval-augmented generation framework such as Fusion-in-Decoder (FiD; Izacard and Grave, 2021b) that consists of two components: a retriever model  $\mathcal{R}$  and a generator model  $\mathcal{G}$ . The retriever model  $\mathcal{R}$  is trained to retrieve a set of passages  $\mathbf{P} = \{p_1, p_2, \dots, p_i, \dots, p_N\}$  with the highest top  $N$  relevance score for each training query  $x$ :  $\mathbf{P} = \mathcal{R}(x)$ . The base generator model  $\mathcal{G}$  (Section 2.2) is then trained to generate the final output  $y$  given an input query and the top retrieved passages:  $y = \mathcal{G}(x, \mathbf{P})$ .
Our analysis (Appendix in Section A.1) shows that a base generator  $\mathcal{G}$  trained in this manner often generates the answers from passages ranked high by the retriever, which are not necessarily the correct evidence passages. Our goal is to build a model that recognizes the evidentiality of each passage and generates answers based only on passages that contain relevant evidence. We define passages with evidence relevant to the task as positive and passages without evidence as negative, even if they happen to include some spurious cues a model can exploit (e.g., a gold answer string for QA).
Method overview. Our method extends the retrieval-augmented generation paradigm by improving the generator  $\mathcal{G}$  to generate answers from passages with correct evidence. We train our new evidentiality-guided generator  $\mathcal{G}^+$  using a multi-task learning framework, sketched in Figure 2. Specifically, given an input query  $x$ , we combine the generation of the correct answer  $\hat{y}$
Figure 2: Overview of our proposed framework. The components inside the blue rectangle is a base generator  $\mathcal{G}$  and our evidentiality-guided generator is the area inside the yellow rectangle. The straight arrows represent the input-output flow, and the dashed arrows indicate the losses.
with the prediction of binary evidentiality labels for each passage in  $\mathbf{P}$  used for training:  $\hat{\mathbf{E}} = \{\hat{e}_1,\hat{e}_2,\dots ,\hat{e}_i,\dots ,\hat{e}_N\}$
It is challenging to obtain gold evidentiality labels  $\hat{\mathbf{E}}$  for many tasks. Most datasets are curated with only query-answer annotations  $(x, \hat{y})$ , or cover subsets of gold passages existing in the large collection of passages, and considering those original gold passages as only positive passages may result in many false negative passages with correct evidence. Therefore, we heuristically obtain silver evidentiality data  $\mathbf{E}^{silver}$  (§2.3) by training an evidentiality labeling model  $\mathcal{M}$  that assigns a silver evidentiality label  $e_i^{silver}$  to each passage  $p_i$  given the query  $x$  and the gold output  $\hat{y}$ . In order to find gold evidence passages to train  $\mathcal{M}$ , we introduce a new approach to evaluate the relevance of passages in generating the correct answer by leaving one passage at a time in answer generation (called leave-one-out generation, sketched in Figure 3). We mine new gold passages for the target task, and train  $\mathcal{M}$  using the mixture of partially available gold evidence passage data and newly mined data. After training, we run  $\mathcal{M}$  on all the training data  $(x, \mathbf{P}, \hat{y})$  to obtain  $\mathbf{E}^{silver}$ .
Finally, we describe auxiliary multi-task learning (sketched in Figure 2) using  $(x,\hat{y})$  and the newly mined silver evidentiality data  $\mathbf{E}^{\text {silver }}$  in Section 2.4. Our evidentiality-guided generator  $\mathcal{G}^{+}$  learns to simultaneously predict the probabilities of output sequences  $y$  and evidentiality for all of the input passages  $\mathbf{E}$ .
2.2 Base Generator  $\mathcal{G}$
We use FiD (Izacard and Grave, 2021b), a state-of-the-art retrieval-augmented generation model, as our base generator model  $\mathcal{G}$ . We include a high-level summary of the model for clarity, referring
the reader to Izacard and Grave (2021b) for more details.
Encoder. We first encode the input query and passages using a pre-trained T5 (Raffel et al., 2020) encoder. The input query  $x$  is prepended to each passage, and the encoder encodes each of  $N$  passages independently. Formally, we transform passage  $p_i$  into  $\mathbf{p}_i \in \mathbb{R}^{L \times h}$ , where  $L$  is the input text length and  $h$  is a hidden size.
Answer generator.  $\tilde{\mathbf{P}}$  is an input summary representation, formed by concatenating  $\mathbf{p}_1,\dots ,\mathbf{p}_N$ . The answer generator takes  $\tilde{\mathbf{P}}$  and outputs the final answer autoregressively. Specifically, it outputs the sequence probability for  $y$  as follows:
$$
P (y | x, \tilde {\mathbf {P}}) = \prod_ {j = 1} ^ {T} p (y _ {j} | y _ {<   j}, x, \tilde {\mathbf {P}}).
$$
where  $y_{j}$  denotes the  $j$ th token of the generated output  $y$  and  $T$  is the length of the final output. The generator is based on the T5 architecture and uses cross attentions to model the interactions between retrieved passages.
2.3 Mining Silver Evidentiality  $\mathbf{E}^{\text {silver }}$
As discussed above, evidentiality labels are unavailable in most of the datasets, and even in some datasets with gold evidence annotations such as Natural Questions (Kwiatkowski et al., 2019), it only covers subsets of gold passages from certain articles. To overcome these limitations, we introduce an evidentiality labeling model  $\mathcal{M}$  which computes the probability that a paragraph  $p_i$  contains evidence for an input  $x$ , given the correct answer  $\hat{y}$ :  $p(e_i^{\text{silver}}|x,p_i,\hat{y})$ . We use a RoBERTa (Liu et al., 2019)-based binary classification model for  $\mathcal{M}$ . This model is trained using
gold evidentiality annotations when those are partially available, or using labels obtained from a new heuristic mining approach described below. Finally, we use the trained evidentiality labeling model to generate silver evidentiality labels for all of the passages included in the training data.
Leave-one-out generation. To precisely identify gold passages with correct evidence when a target dataset only has input-output annotations, our leave-one-out generation approach (sketched in Figure 3) leverages a trained base generator model and uses its predictions to estimate the relevance to the query of the passage. Specifically, we feed an input query  $x$  and retrieved passages  $\mathbf{P}$  to our trained base generator for  $N$  times, where we mask the  $i$ th passage in the  $i$ th iteration to evaluate if the model can still generate the correct answer without the information presented in  $i$ th passage. We consider  $i$ th passage positive if the model fails to generate  $\hat{y}$  when and only when  $i$ th passage is masked. We also consider  $i$ th passage negative if the model succeeds in generating  $\hat{y}$  when and only when  $i$ th passage is masked—this means that the  $i$ th passage confuses the model. This approach may not find all of the gold evidence passages when there are multiple gold passages in  $\mathbf{P}$  or the answers are memorized during fine-tuning of the base generator. Yet, we found that we can mine a sufficient number of high-quality gold passages using our approach to quickly adapt the evidentiality labeling model  $\mathcal{M}$  to a new task. In our experiments, we combine the gold evidentiality data (i.e., long answers) from Natural Questions with task-specific leave-one-out data to train a separate evidentiality model  $\mathcal{M}$  for each task. See the details of the data mining for each task in Appendix.
2.4 Multi-task Learning with  $\mathbf{E}^{silver}$
Our generator  $\mathcal{G}^+$  shares a similar, T5-based encoder-decoder architecture as the base generator, but we have an additional decoder that is used for the evidentiality prediction. We train  $\mathcal{G}^+$  with a multi-task objective given the originally available data  $(x,\mathbf{P},\hat{y})$  and newly mined  $\mathbf{E}^{\text{silver}}$ .
Evidentiality predictor. The evidentiality predictor predicts the evidentiality of each passage. Similarly to the answer generator, we use the T5 decoder architecture for the classifier. Our evidentiality predictor generates the evidentiality  $e_i$  given encoded passage representation  $\mathbf{p_i}$ :  $p(e_i|q,\mathbf{p_i})$ . The
Figure 3: Overview and examples of our leave-one-out generation to find new positive and negative examples. We mask (remove) one passage at each iteration.
evidentiality predictor in  $\mathcal{G}^+$  has a much harder problem than the evidentiality model  $\mathcal{M}$  from the previous section:  $\mathcal{M}$  has access to the gold answer  $\hat{y}$ , while  $\mathcal{G}^+$  does not. Intuitively, we can get reasonably accurate evidentiality labels from  $\mathcal{M}$  using the gold answer, then force  $\mathcal{G}^+$  to predict those labels without access to the gold answer, in order to teach the encoder of  $\mathcal{G}^+$  to better determine the relationship between  $x$  and  $p_i$ .
Multi-task training. We conduct multi-task training of generation and evidentiality prediction. In particular, our framework minimizes a multi-task objective below:
$$
\mathcal {L} = \mathcal {L} _ {\text {g e n}} + \lambda \mathcal {L} _ {\text {c l a s s}}, \tag {1}
$$
where  $\lambda$  is a weighting parameter to balance the two objectives and would be tuned. In Eq. (1),  $\mathcal{L}_{gen}$  is formulated as follows:
$$
\mathcal {L} _ {g e n} = - \sum_ {j} ^ {T} \log p (\hat {y} _ {j} | y _ {<   j}, q, \tilde {\mathbf {P}}), \tag {2}
$$
where  $\hat{y}_j$  denotes the  $j$ th token of the annotated gold answer  $\hat{y}$ . Similarly, evidentiality prediction objective  $\mathcal{L}_{class}$  can be written as follows:
$$
\mathcal {L} _ {\text {c l a s s}} = - \sum_ {i} ^ {N} \log p (e _ {i} ^ {\text {s i l v e r}} | q, p _ {i}). \tag {3}
$$
Note that this probability is computed by a T5 decoder as a common practice (Raffel et al., 2020); even though  $e_i^{\text{silver}} \in \{\text{positive}, \text{negative}\}$ , the probability is normalized over T5's entire output vocabulary.
2We also tried to fine-tune a simple binary classification model using additional output layer on the top of T5 encoder. We found that this model performs much worse than T5-decoder-based classification model.
Dataset & Task	# of examples	evaluation metric	Top-20 recall (%)
train	dev	test
1. Open-domain QA
Natural Questions Open (Kwiatkowski et al., 2019)	79,168	8,757	3,610	EM	82.1
TriviaQA unfiltered (Joshi et al., 2017)	78,785	8,837	11,313	EM	75.2
2. Fact Verification
FEVER (Thorne et al., 2018)	104,966	10,444	10,100	Accuracy	98.1
FaVIQ-Ambig (A) (Park et al., 2021)	17,008	4,260	4,688	Accuracy	100.0
3. Knowledge-enhanced Dialogue
Wizard of Wikipedia (Dinan et al., 2019)	63,734	3,054	2,944	F1	96.2
Table 1: Dataset statistics. We experiment with three diverse knowledge-intensive NLP tasks across six datasets. "Top 20 recall" calculates if any of the top 20 passages include the answer strings (for open-domain QA datasets and FaVIQ-A) or comes from the provenance article (for FEVER and Wizard of Wikipedia) in the development set. FEVER and Wizard of Wikipedia are based on the KILT (Petroni et al., 2021) version.
3 Experimental Setups
We experiment on three knowledge-intensive tasks: open-domain QA, fact verification, and knowledge-enhanced dialogue. Statistics for each dataset are provided in Table 1.
3.1 Tasks, Datasets, and Metrics
Open-domain QA. We use Natural Questions Open (Kwiatkowski et al., 2019) and TriviaQA-unfiltered (Joshi et al., 2017) to evaluate our method on open-domain QA. Natural Questions consists of questions, long answers (e.g., gold evidence passages) and short answers (e.g., spans in the long answers), and the open-domain QA version is created by discarding questions that only have long answers or short answers whose length is longer than five tokens (Lee et al., 2019). TriviaQA-unfiltered (Joshi et al., 2017) includes unfiltered 110K Trivia question and answer pairs. For both of the datasets, we use publicly available DPR retrieval results for training and inference data, and do not further fine-tune retrievers. Only the Natural Questions dataset has gold passage annotations and we use the gold passage annotations to train the evidentiality labeling model  $\mathcal{M}$  only. Following prior work (Lee et al., 2019), we use Exact Match (EM) as our primary metric.
Fact verification. We use FaVIQ Ambig (FaVIQ-A; Park et al. 2021) and FEVER (Thorne et al., 2018) via the KILT benchmark (Petroni et al., 2021) to evaluate our method on fact verification. FaVIQ-A is created from an information-seeking QA dataset, AmbigQA (Min et al., 2020) to pose realistic fact verification queries. We use the baseline code provided by the authors of the
FaVIQ dataset and KILT. We use accuracy as our evaluation metric.
Knowledge-enhanced dialogue. We use Wizard of Wikipedia (WoW; Dinan et al. 2019) to evaluate our method on knowledge-enhanced dialogue. We use the officially available KILT DPR baseline codes (Petroni et al., 2021)<sup>4</sup> to obtain passages and evaluate downstream F1 score.
We use FiD (Izacard and Grave, 2021b) as our primary baseline using their official implementation. In addition, we report results from the best published, publicly available generator models for each dataset including RAG (Lewis et al., 2020b) and DPR + BART (Petroni et al., 2021). For FEVER and WoW, we also compare our method with the published models on the KILT leaderboard.
3.3 Hyper parameters
Due to the computational budget, we use T5's base-size models throughout our experiments for our evidentiality-guided generator. For our evidentiality labeling model  $\mathcal{M}$ , we use a RoBERTa (Liu et al., 2019)-base binary classification model. If not specified, we use the top 20 passages during training and inference, which also reduces the computational times from the original FiD model that uses top 100 passages. We train the models for 120k steps using 8 GPUs with 24 GB memory and take the checkpoint that achieves the highest score on the development set. The batch size is set to 1
4 github.com/facebookresearch/KILT/ blob/main/kilt/retrievers/README.md 5 github.com/facebookresearch/FiD ai.facebook.com/tools/kilt/
$^{3}$ github.com/facebookresearch/DPR
Models	NQ EM	TQA EM
dev	test	dev	test
RAG (large)	-	44.5	-	56.8
FiD (base)	46.9	48.3	67.1	67.2
Ours(base)	47.8	49.8	67.7	67.8
R2D2 (large*)	-	55.0	-	69.9
(a)
Models	FaVIQ-A	FEVER
dev	test	dev	test
DPR+BART (large)	66.9	64.9	88.1	86.7
DPR+BART (base)	60.2	-	-	-
RAG (large)	-	-	87.7	86.3
FiD (base)	67.8	64.3	89.5	-
Ours (base)	69.6	65.7	89.8	88.5
(b)
Models	WoW
dev	test
DPR+BART (large)	15.5	15.2
RAG (large)	13.8	13.1
FiD (base)	16.9	-
Ours (base)	17.9	17.3
Table 2: Main Results. "base" and "large" denote the base generator model sizes (e.g., T5-large, BART-base). (a) Performance on Natural Questions Open and TriviaQA unfiltered. "NQ" denotes Natural Questions Open, "TQA" denotes TriviaQA unfiltered. The state-of-the-art model is R2D2 from Fajcik et al. (2021), which has 1.29 billion parameters (more than twice more parameters than our model), consisting of a ranker and two reader models with ELECTRA (Clark et al., 2020)-large and T5-large. (b) Performance on FaVIQ-A and FEVER. Previous best model is DPR+BART (large) from Park et al. (2021) and Petroni et al. (2021) on FaVIQ-A and FEVER, respectively. (c) Performance on Wizard of Wikipedia (WoW). The best published model on the development set is DPR+BART (large) from Petroni et al. (2021). Test set results of WoW and FeVER are based on the leaderboard results at the time of the paper submission (January, 2022).
and to imitate the larger batch size, we set the gradient accumulation step to be 4. The learning rate is set to  $10^{-5}$  and the number of warm-up steps is 1000. We set  $\lambda$  to be 0.5 for open-domain QA and dialogue, and 0.1 for fact verification. See more details in Appendix.
Our approach significantly improves over its direct counterpart on all datasets, and outperforms all prior published results on FaVIQ-A, FEVER and WoW, advancing their state-of-the-art performance.
4.1 Task Results
Open-domain QA. Table 2a shows experimental results on the two open-domain QA datasets. On Natural Questions Open, we improve the performance over FiD by 1.5 EM score. We observe performance improvements over FiD on TriviaQA as well. It should be noted that on open-domain QA, most of the recent models (e.g., Fajcik et al., 2021) contain a few times more parameters than our model or use improved retrievers (Izacard and Grave, 2021a), both of which are beyond our computational budgets. Our results represent state-of-the-art performance for models with access to similar computational resources, and our contributions should be complementary to work focusing on improving retrieval components.
Fact verification. Table 2b shows the experimental results on FaVIQ-A and FEVER. In addition to the original paper's baseline, we have fine-tuned a BART-base baseline using their original public codebase (DPR+BART (base)) for a fair compar-
son. Our model significantly outperforms the prior best model, DPR+BART (large), on FaVIQ-A by a large margin. Our model also significantly outperforms FiD on FaVIQ by  $1.8\%$  on the development set and  $1.4\%$  on the test set, yielding state-of-the-art performance on this dataset. Our evidentiality-guided generator also outperforms other models on FEVER. On the FEVER hidden test set, our model yields  $88.5\%$  down-stream accuracy and ranks second among all submissions, outperforming all of prior published work (Maillard et al., 2021; Petroni et al., 2021; Lewis et al., 2020b).