Bridging Anaphora Resolution as Question Answering (2020)

Abstract: Most previous studies on bridging anaphora resolution (Poesio et al., 2004; Hou et al., 2013b; Hou, 2018a) use the pairwise model to tackle the problem and assume that the gold mention information is given. In this paper, we cast bridging anaphora resolution as question answering based on context. This allows us to find the antecedent for a given anaphor without knowing any gold mention information (except the anaphor itself). We present a question answering framework (BARQA) for this task, which leverages the power of transfer learning. Furthermore, we propose a novel method to generate a large amount of “quasi-bridging” training data. We show that our model pre-trained on this dataset and fine-tuned on a small amount of in-domain dataset achieves new state-of-the-art results for bridging anaphora resolution on two bridging corpora (ISNotes (Markert et al., 2012) and BASHI (Ro ̈siger, 2018)).

## introduction anaphora accounts for text cohesion and is crucial for text understanding. an anaphor is a noun phrase (np) that usually refers back to the same or a different entity (the antecedent) in text. anaphora resolution is the task to determine the antecedent for a given anaphor. while direct anaphora resolution attracts a lot of attention in the nlp community recently, such as winograd schema challenge @xcite @xcite , indirect anaphora resolution or bridging anaphora resolution is less well studied. in this paper, we focus on bridging anaphora resolution where bridging anaphors and their antecedents are linked via various lexico-semantic, frame or encyclopedic relations. following @xcite and @xcite , we mainly consider "referential bridging" in which bridging anaphors are truly anaphoric and bridging relations are context-dependent. in example 1 foot_0 , both "her building" and "buildings with substantial damage" are plausible antecedent candidates for the bridging anaphor "residents" based on lexical semantics. in order to find the antecedent (buildings with substantial damage), we have to take the meaning of the broader discourse context into account. most previous studies on bridging anaphora resolution @xcite @xcite tackle the problem using the pairwise model and assume that the gold mention information is given. most work @xcite @xcite uses syntactic patterns to measure semantic relatedness between the head nouns of an anaphor and its antecedent. @xcite proposes a simple deterministic algorithm that also considers the semantics of modifications for head nouns. these approaches, however, do not take the broader context outside of noun phrases (i.e., anaphors and antecedent candidates) into account and often fail to resolve context-dependent bridging anaphors as demonstrated in example 1. resolving bridging anaphors requires contextdependent text understanding. recently, @xcite argue that question answering (qa) is a natural format to model tasks that require question understanding. in this paper, we cast bridging anaphora resolution as question answering based on context. we develop a qa system (barqa) for the task based on bert @xcite . given a context as shown in example 1, we first rephrase every anaphor as a question, such as "residents of what?". by answering the question, the system then identifies the span of the antecedent from the context. compared to the pairwise model, our qa system does not require the gold or system mention information as the antecedent candidates. in addition, this framework allows us to integrate context outside of nps when choosing antecedents for bridging anaphors. for instance, "green" and "damage were color-coded" are among the top predicted answers for the above question. different from coreference resolution, there are no large-scale corpora available for referential bridging resolution due to its complexity. in this paper we propose a new method to generate a large amount of "quasi-bridging" training data from the automatically parsed gigaword corpus @xcite . we demonstrate that our "quasi-bridging" training data is a better pre-training choice for bridging anaphora resolution compared to the squad corpus @xcite . moreover, we show that our model pre-trained on this dataset and fine-tuned on a small amount of in-domain dataset achieves new state-ofthe-art results for bridging anaphora resolution on two bridging corpora (i.e., isnotes @xcite and bashi @xcite ). to summarize, the main contributions of our work are: (1) we formalize bridging anaphora resolution as a question answering problem and propose a qa model to solve the task; (2) we explore a new method to generate a large amount of "quasi-bridging" training dataset and demonstrate its value for bridging anaphora resolution; and (3) we carefully carry out a series of experiments on two referential bridging corpora and provide some error analysis to verify the effectiveness of our qa model to resolve the context-dependent bridging anaphors in isnotes. we release the code and all experimental datasets at https://github. com/ibm/bridging-resolution . it is worth noting that the bridging definition in the arrau corpus is different from the one used in the other two datasets. @xcite pointed out that isnotes and bashi contain "referential bridging" where bridging anaphors are truly anaphoric and bridging relations are contextdependent, while in arrau, most bridging links are purely lexical bridging pairs which are not context-dependent (e.g., europe -spain or tokyo -japan). in this paper, we focus on resolving referential bridging anaphors. regarding the algorithm for bridging anaphora resolution, most previous work uses the pairwise model for the task. the model assumes gold or system mention information (nps) is given beforehand. it creates (positive/negative) training instances by pairing every anaphor a with its preceding mention m. usually, m is from a set of antecedent candidates which is formed using a fixed window size. @xcite and @xcite trained such pairwise models to resolve mereological bridging anaphors in the english gnome corpus foot_2 and the french dede corpus @xcite , respectively. one exception is @xcite , which proposed a joint inference framework to resolve bridging anaphors in isnotes. the framework is built upon the pairwise model and predicts all semantically related bridging anaphors in one document together. recently, hou (2018a) generated a word representation resource for bridging (i.e., embeddings bridging) and proposed a simple deterministic algorithm to find antecedents for bridging anaphors in isnotes and bashi. the word representation resource is learned from a large corpus input text barqa ## barqa: a qa system for bridging anaphora resolution in this section, we describe our qa system (called barqa) for bridging anaphora resolution in detail. figure 1 illustrates how barqa predicts antecedents for bridging anaphors in example 1. ## error analysis in order to better understand our model, we automatically label bridging anaphors in isnotes as either "referential bridging/world-knowledge" or "referential bridging/context-dependent". we then analyze the performance of barqa and the best model from hou (2018a) on these two categories. @xcite pointed out that although lexical and referential bridging are two different concepts, sometimes they can co-occur within the same pair of expressions. in example 2, "employees" is an anaphoric expression. at the same time, the relation between the antecedent entity "{mobil corp./the company's}" and the bridging anaphor "employees" corresponds to the common-sense world knowledge which is true without any specific context. we call such cases as "referential bridging/world-knowledge". differently, we call a bridging anaphor as "referential bridging/context-dependent" if it has multiple equally plausible antecedent candidates according to the common-sense world knowledge about the np pairs and we have to analyze the context to choose the antecedent argue that "{the exploration and production division -employees}" in example 2 is also a valid common-sense knowledge fact, however, we consider that it is less prominent than "{the company's -employees}". for a bridging anaphor a, the deterministic algorithm (embeddings bridging) from hou (2018a) uses a word representation resource learned from a large corpus to predict the most semantically related np among all np candidates as the antecedent. the predictions from this system reflect the common-sense world knowledge about the np pairs. we thus use this algorithm to label bridging anaphors in isnotes: if a bridging anaphor is correctly resolved by embeddings bridging, we label it as "referential bridging/world-knowledge", otherwise the label is "referential bridging/contextdependent". ## conclusions in this paper, we model bridging anaphora resolution as a question answering problem and propose a qa system (barqa) to solve the task. we also propose a new method to automatically generate a large scale of "quasi-bridging" training data. we show that our qa system, when trained on this "quasi-bridging" training dataset and fine-tuned on a small amount of in-domain dataset, achieves the new state-of-the-art results on two bridging corpora. compared to previous systems, our model is simple and more realistic in practice: it does not require any gold annotations to construct the list of antecedent candidates. moreover, under the proposed qa formulation, our model can be easily strengthened by adding other span-based text understanding qa corpora as pre-training datasets. finally, we will release our experimental qa datasets (in the squad json format) for bridging anaphora resolution on isnotes and bashi. they can be used to test a qa model's ability to understand a text in terms of bridging inference.