Probabilistic Assumptions Matter: Improved Models for Distantly-Supervised Document-Level Question Answering (2020)

Abstract: We address the problem of extractive question answering using document-level distant super-vision, pairing questions and relevant documents with answer strings. We compare previously used probability space and distant supervision assumptions (assumptions on the correspondence between the weak answer string labels and possible answer mention spans). We show that these assumptions interact, and that different configurations provide complementary benefits. We demonstrate that a multi-objective model can efficiently combine the advantages of multiple assumptions and outperform the best individual formulation. Our approach outperforms previous state-of-the-art models by 4.3 points in F1 on TriviaQA-Wiki and 1.7 points in Rouge-L on NarrativeQA summaries.

## introduction distant supervision assumptions have enabled the creation of large-scale datasets that can be used to train fine-grained extractive short answer question answering (qa) systems. one example is triviaqa @xcite . there the authors utilized a pre-existing set of trivia questionanswer string pairs and coupled them with relevant documents, such that, with high likelihood, the documents support answering the questions (see fig. 1 for an illustration). another example is the narrativeqa dataset @xcite , where crowd-sourced abstractive answer strings were used to weakly supervise answer mentions in the text of movie scripts or their summaries. in this work, we focus on the setting of documentlevel extractive qa, where distant supervision is specified as a set a of answer strings for an input question-document pair. iaqa example, there are three occurrences of the original answer string "joan rivers" (blue), and one alternate but incorrect alias "diary of a mad diva" (purple). only two "joan rivers" mentions (shown in blue boxes) support answering the question. in the narrativeqa example, there are two answer stings in a: "in the spring at mount helicon" (blue) and "mount helicon" (orange), with the latter being a substring of the former. both mentions in p2 are correct answer spans. depending on the data generation process, the properties of the resulting supervision from the sets a may differ. for example, the provided answer sets in triviaqa include aliases of original trivia question answers, aimed at capturing semantically equivalent answers but liable to introducing semantic drift. in fig. 1 , the possible answer string "diary of a mad diva" is related to "joan rivers", but is not a valid answer for the given question. on the other hand, the sets of answer strings in narrativeqa are mostly valid since they have high overlap with human-generated answers for the given question/document pair. as shown in fig. 1 , "in the spring at mount helicon" and "mount helicon" are both valid answers with relevant mentions. in this case, the annotators chose answers that appear verbatim in the text but in the more general case, noise may come from partial phrases and irrelevant mentions. while distant supervision reduces the annotation cost, increased coverage often comes with increased noise (e.g., expanding entity answer strings with aliases improves coverage but also increases noise). even for fixed document-level distant supervision in the form of a set of answers a, different interpretations of the partial supervision lead to different points in the coverage/noise space and their relative performance is not well understood. this work systematically studies methods for learning and inference with document-level distantly supervised extractive qa models. using a bert @xcite joint question-passage encoder, we study the compound impact of: we show that the choice of probability space puts constraints on the distant supervision assumptions that can be captured, and that all three choices interact, leading to large differences in performance. specifically, we provide a framework for understanding different distant supervision assumptions and the corresponding trade-off among the coverage, quality and strength of distant supervision signal. the best configuration depends on the properties of the possible annotations a and is thus data-dependent. compared with recent work also using bert representations, our study show that the model with most suitable probabilistic treatment achieves large improvements of 4.6 f1 on triviaqa and 1.7 rouge-l on narra-tiveqa respectively. additionally, we design an efficient multi-loss objective that can combine the benefits of different formulations, leading to significant improvements in accuracy, surpassing the best previously reported results on the two studied tasks. results are further strengthened by transfer learning from fully labeled short-answer extraction data in squad 2.0 @xcite , leading to a final state-of-the-art performance of 76.3 f1 on triviaqa-wiki and 62.9 on the narra-tiveqa summaries task. foot_2 ## probability space here, we first formalize both paragraph-level and document-level models, which have been previously used for document-level extractive qa. typically, paragraph-level models consider each paragraph in the document independently, whereas document models integrate some dependencies among paragraphs. to define the model, we need to specify the probability space, consisting of a set of possible outcomes and a way to assign probabilities to individual outcomes. for extractive qa, the probability space outcomes consist of token positions of answer mention spans. the overall model architecture is shown in fig. 2 . we use bert @xcite to derive representations of document tokens. as is standard in state-of-the-art extractive qa models @xcite @xcite , the bert model is used to encode a pair of a given question with one paragraph from a given document into neural text representations. these representations are then used to define scores/probabilities of possible answer begin and end positions, which are in turn used to define probabilities over possible answer spans. then the answer string probabilities can be defined as the aggregation over all possible answer spans/mentions. in the following, we show that paragraph-level and document-level models differ only in the space of possible outcomes and the way of computing answer span probabilities from answer position begin and end scores. scoring answer begin and end positions given a question q and a document d consisting of k paragraphs p 1 , . . . , p k , the bert encoder produces contextualized representations for each question-paragraph pair (q, p k ). specifically, for each token position i k in p k , the final hidden vector h (i,k) ∈ r d is used as the contextualized token embedding, where d is the vector dimension. the span-begin score is computed as s b (i k ) = w t b h (i,k) using a weight vector w @xmath0. the span-end score s e (j k ) is defined in the same way. the probabilities for a start position i k and an end position j k are the partition functions z b and z e depend on whether we use a paragraph-level or documentlevel probability space. paragraph-level model in paragraph level models, we assume that for a given question against a document d, each of its paragraphs p 1 , . . . , p k independently selects a pair of answer positions (i k , j k ), which are the begin and end of the answer from paragraph p k . in the case that p k does not support answering the question q, special null positions are selected (following the squad 2.0 bert implementation 3 ). thus, the set of possible outcomes ω in the paragraph-level probability space is the set of lists of begin/end position pairs, one from each paragraph: {[(i 1 , j 1 ), . . . , (i k , j k )]}, where i k 3 https://github.com/google-research/bert and j k range over positions in the respective paragraphs. the answer positions in different paragraphs are independent, and the probability of each paragraph's answer begin and end is computed by normalizing over all possible positions in that paragraph, i.e., and the probability of an end at j k is defined analogously. the probability of a possible answer position assignment for the document d is then defined as as we can see from the above definition, due to the independence assumption, models using paragraph-level normalization do not learn to directly calibrate candidate answers from different paragraphs against each other. ## distant supervision assumptions there are multiple ways to interpret the distant supervision signal from a as possible outcomes in our paragraph-level and document-level probability spaces, leading to corresponding training loss functions. although several different paragraphlevel and document-level losses @xcite @xcite @xcite have been studied in the literature, we want to point out that when interpreting the distant supervision signal, there is a tradeoff among multiple desiderata: we begin with setting up additional useful notation. given a document-question pair (d, q) and a set of answer strings a, we define the set of a-consistent token spans y a in d as follows: for each paragraph p k , span (i k , j k ) ∈ y k a if and only if the string spanning these positions in the paragraph is in a. for paragraph-level models, if for paragraph p k the set y k a is empty, we redefine y k a to be {null}. similarly, we define the set of aconsistent begin positions y k b,a as the start positions of consistent spans: y k e,a for a-consistent end positions is defined analogously. in addition, we term an answer span (i, j) correct for question q, if its corresponding answer string is a correct answer to q, and the context of the specific mention of that answer string from positions i to j entails this answer. similarly, we term an answer begin/end position correct if there exists a correct answer span starting/ending at that position. h1: all a-consistent answer spans are correct. while this assumption is evidently often incorrect (low on the quality dimension ), especially for triviaqa, as seen from fig. 1 , it provides a large number of positive examples and a strong supervision signal (high on coverage and strength ). we include this in our study for completeness. h1 translates differently into possible outcomes for corresponding models depending on the probability space (paragraph or document). paragraphlevel models select multiple answer spans, one for each paragraph, to form a possible outcome. thus, multiple a-consistent answer spans can occur in a single outcome, as long as they are in different paragraphs. for multiple a-consistent answer spans in the same paragraph, these can be seen as mentions that can be selected with equal probability (e.g., by different annotators). document-level models select a single answer span in the document and therefore multiple a-consistent answer spans can be seen as occurring in separate annotation events. h2 has two variations: correct span, assuming for paragraph-level and document-level models, h2 corresponds differently to possible outcomes. for paragraph models, one outcome can select answer spans in all positive paragraphs and null in negative ones. for document-level models, we view answers in different paragraphs as outcomes of multiple draws from the distribution. the identity of the particular correct span or begin/end position is unknown, but we can compute the probability of the event comprising the consistent outcomes. on the other hand, h3 has lower coverage ( ) than h1 and h2, because it provides a single positive example for the whole document, rather than one for each positive paragraph. it also reduces the strength of the supervision signal ( ), as the model now needs to figure out which mention to select from the larger document-level set y a . note that we can only use h3 coupled with a document-level model, because a paragraph-level model cannot directly tradeoff answers from different paragraphs against each other, to select a single answer span from the document. as with the other distant supervision hypotheses, spanbased and position-based definitions of the possible consistent outcomes can be formulated. the log-probabilities of these events are defined in row three of the probability-space (paragraph vs. documentlevel) and the distant supervision assumption (h1, h2, and h3, each position or span-based) together define our interpretation of the distant supervision signal resulting in definitions of probability space outcomes consistent with the supervision. next, we define corresponding optimization objectives to train a model based on this supervision and describe the inference methods to make predictions with a trained model. ## optimization and inference methods for each distant supervision hypothesis, we maximize either the marginal log-likelihood of a-consistent outcomes (mml) or the log-likelihood of the most likely outcome (hardem). the latter was found effective for weakly supervised tasks including qa and semantic parsing by @xcite . inference: since the task is to predict an answer string rather than a particular mention for a given question, it is potentially beneficial to aggregate information across answer spans corresponding to the same string during inference. the score of a candidate answer string can be obtained as p a (x) = ξ (i,j)∈x p s (i, j), where x is the set of spans corresponding to the answer string x, and ξ can be either or max. foot_4 it is usually beneficial to match the training objective with the corresponding inference method, i.e. mml with marginal inference ξ = , and hardem with max (viterbi) inference ξ = max. @xcite showed hardem optimization was useful when using an h2 span-level distant supervision assumption coupled with max inference, but it is unclear whether this trend holds when inference is useful or other distant supervision assumptions perform better. we therefore study exhaustive combinations of probability space, distant supervision assumption, and training and inference methods. ## related work distant supervision has been successfully used for decades for information extraction tasks such as entity tagging and relation extraction @xcite . several ways have been proposed to learn with ds, e.g., multi-label multi-instance learning @xcite , assuming at least one supporting evidence @xcite , integration of label-specific priors @xcite , and adaption to shifted label distributions @xcite . recent work has started to explore distant supervision to scale up qa systems, particularly for open-domain qa where the evidence has to be retrieved rather than given as input. reading comprehension (rc) with evidence retrieved from information retrieval systems establishes a weakly-supervised qa setting due to the noise in the heuristics-based span labels @xcite @xcite . one line of work jointly learns rc and evidence ranking using either a pipeline system @xcite @xcite or an end-to-end model @xcite . another line of work focuses on improving distantly-supervised rc models by developing learning methods and model architectures that can better use noisy labels. @xcite propose a paragraph-pair ranking objective, which has components of both our h2-p and h3-d position-based formulations. they don't explore multiple inference methods or combinations of objectives and use less powerful representations. in @xcite , a coarse-to-fine model is proposed to handle label noise by aggregating information from relevant paragraphs and then extracting answers from selected ones. @xcite propose a hard em learning scheme which we included in our experimental evaluation. our work focuses on examining probabilistic assumptions for document-level extractive qa. we provide a unified view of multiple methods in terms of their probability space and distant supervision assumptions and evaluate the impact of their components in combination with optimization and inference methods. to the best of our knowledge, the three ds hypotheses along with position and span-based interpretations have not been formalized and experimentally compared on multiple datasets. in addition, the multi-objective formulation is new. ## conclusions in this paper, we demonstrated that the choice of probability space and interpretation of the distant supervision signal for document-level qa have a large impact, and that they interact. depending on the properties of the data, different configurations are best, and a combined multi-objective formulation can reap the benefits of its constituents. a future direction is to extend this work to question answering tasks that require reasoning over multiple documents, e.g., open-domain qa. in addition, the findings may generalize to other tasks, e.g., corpus-level distantly-supervised relation extraction.