Constrained Multi-Task Learning for Bridging Resolution (2022)

Abstract: We examine the extent to which supervised bridging resolvers can be improved without employing additional labeled bridging data by proposing a novel constrained multi-task learning framework for bridging resolution, within which we (1) design cross-task consistency constraints to guide the learning process; (2) pre-train the entity coreference model in the multi-task framework on the large amount of publicly available coreference data; and (3) integrating prior knowledge encoded in rule-based resolvers. Our approach achieves state-of-the-art results on three standard evaluation corpora.

## introduction bridging @xcite plays an important role in establishing entity coherence in a text. in contrast to direct anaphors, which indicate the coreference relation between a nominal expression and its antecedent, bridging anaphors or associative anaphors link to their antecedents via non-identical relations. bridging resolution is the task of recognizing and resolving bridging anaphors in a text. bridging resolution and coreference resolution are closely related to information status (is henceforth) classification, the goal of which is to assign an is to each discourse entity that indicates how these entities are referred to in a text @xcite @xcite . in general, an entity is old if it is coreferent with an entity that has been mentioned before (e.g., "[the business]" and "[its]" in figure 1 ). bridging anaphors are discourse-new but hearer-old. they have not been introduced in the discourse directly, but are inferrable from previously mentioned entities (e.g., "[the customers]" in figure 1 ). new entities are introduced into the discourse for the first time and are not known to the hearer before (e.g. "[the bakersfield supermarket]" in figure 1 ). progress on bridging resolution research is limited in part by the scarcity of annotated training s1: [the bakersfield supermarket] _new went bankrupt last may. s2: [the business] _old closed when @xcite _old old owner was murdered by robbers. s3: [the murder] _old saddened [the customers] _bridging . ## related work bridging resolution. bridging resolution is composed two sub-tasks: bridging anaphora recognition and antecedent selection. most previous work tackles them separately. one line of research models bridging recognition as part of is classification @xcite @xcite @xcite , while others have focused on antecedent selection based on gold bridging anaphors @xcite @xcite . there are a few studies tackling the challenging task of full bridging resolution (i.e., bridging anaphor recognition and resolution). @xcite and @xcite develop rules to identify bridging links based on syntactic and semantic constraints. @xcite propose a pipeline system built on top of complex manually designed features. @xcite design a mtl neural model for bridging resolution that uses coreference resolution as an auxiliary task. recently, @xcite show the effectiveness of a hybrid rule-based and mtl approach for bridging resolution. for a detailed overview of these approaches, we refer the reader to a recent survey by @xcite . constrained multi-task learning with deep neural networks. multi-task learning has been widely adopted in various nlp applications to improve the performance of individual tasks @xcite . recently, several studies have demonstrated that multi-task training in neural networks can be further improved by integrating logical constraints to enforce a coherent output across different tasks @xcite @xcite . however, for a complex task like bridging resolution, it is non-trivial to choose auxiliary tasks and model the relationships between these tasks in deep neural networks. in this work, we (1) jointly train three tasks (i.e., bridging resolution, coreference resolution, and is classification); (2) design five soft cross-task consistency constraints to guide the training process; and (3) integrate prior knowledge about bridging into our mtl model. ## model in this section, we present our constrained mtl framework for bridging resolution. inspired by @xcite span-based model for bridging resolution, which employs an unconstrained mtl framework that jointly learns bridging and coreference, our model takes as input a document d represented as a sequence of word tokens and gold mentions m , from which we create span representations. our model simultaneously learns three tasks, namely is classification, bridging, and coreference, as defined below. the is classification task aims to assign each span i an is y is taken from an is inventory. the model predicts the is of i to be y * @xmath0, y is ), where s is is a function suggesting i's likelihood of having y is as its is. the bridging resolution task involves determining an antecedent for each bridging anaphor. formally, it assigns span i an antecedent y b , where y @xmath1, ..., i -1, ϵ}. in other words, the value of each y b is the id of its antecedent, which can be one of the preceding spans or a dummy antecedent ϵ (if the mention underlying i is not a bridging anaphor) in the associated document. we define the following scoring function: the entity coreference resolution task involves determining an antecedent for each identity anaphor. formally, it aims to assign span i an antecedent y c based on a scoring function s c that can be defined in an analogous manner as the s b function in the bridging resolution task. ## conclusion we proposed the first neural model for full bridging resolution that (1) exploits the connection between information status classification, entity coreference resolution, and bridging resolution in a multi-task learning framework, (2) employs soft cross-task consistency constraints to guide the learning process, (3) pre-trains the entity coreference model, and (4) integrates prior knowledge encoded in handcrafted bridging resolution rules into the learning framework. our model outperformed several strong baselines and achieved state-of-the-art results on three evaluation datasets. ablation results provided suggestive evidence that each component of our model contributed positively to bridging resolution performance.