Evaluating the Impact of a Hierarchical Discourse Representation on Entity Coreference Resolution Performance (2021)

Abstract: Recent work on entity coreference resolution (CR) follows current trends in Deep Learning applied to embeddings and relatively simple task-related features. SOTA models do not make use of hierarchical representations of discourse structure. In this work, we leverage automatically constructed discourse parse trees within a neural approach and demonstrate a significant improvement on two benchmark entity coreference-resolution datasets. We explore how the impact varies depending upon the type of mention.

## introduction historically, theories of discourse coherence @xcite @xcite have offered elaborate expositions on how the patterns of anaphoric references in discourse are constrained by limitations in human capacity to manage attention and resolve ambiguity. @xcite acknowledges that these human limitations have meant that coreference resolution in natural text can be achieved with relatively high accuracy using a combination of recency and simple semantic constraints. state-of-the-art neural approaches for coreference resolution @xcite @xcite have therefore not surprisingly shown strong performance relying on surface-level features and local-context (i.e., extracted from a small text window around the mention). traditional approaches, on the other hand, make an attempt to formally model the process of managing attention, for example, the stack in @xcite 's model. their stack-based model suggests specific places where recency might fail while a more explicit model of discourse structure might make a correct prediction, for example, where an anaphor and a nearby potential (but incorrect) antecedent are in adjacent but separate discourse segments. because of the potential existence of such cases, we hypothesize that formally incorporating a representation of discourse structure would have a small but non-random positive impact on the ability to correctly resolve anaphoric references. this effect might vary depending upon the semantic informativeness of alternative types of anaphoric expressions, since they impose different constraints on where their antecedent can be located within a hierarchical discourse structure. there is also a danger that the level of accuracy with which the hierarchical structure of discourse can be obtained in practice might reduce the positive impact still further. the contribution of this paper is an empirical investigation of the impact of including a representation of the hierarchical structure of discourse within a neural entity coreference approach. to this end, we leverage a state-of-the-art rst discourse-parser to convert a flat document into a tree-like structure from which we can derive features that model the structural constraints. we embed this representation within an architecture that is enabled to learn to use this information deferentially depending upon the type of mention. the results demonstrate that this level of nuance enables a small but significant improvement in coreference accuracy, even with automatically constructed rst trees. ## related work though recency is the strongest predictor for coreference resolution (cr), prior work in cr has benefited from the inclusion of semantic features such as type-information on top of the surface and syntaxlevel features. @xcite ; @xcite used dictionaries like wordnet to extract the semantic class for a noun. more recently, @xcite showed that adding ner style type-information to @xcite substantially improves performance across multiple datasets. discourse-level features have been successfully employed in multiple downstream nlp tasks like summarization @xcite , sentiment analysis @xcite , and student writing evaluation @xcite . for coreference resolution, @xcite showed that the potential of natural language systems to correctly determine co-referential links, can be increased by exploiting the hierarchical structure of texts. their discourse model was informed by vein theory @xcite , which identifies chains of elementary discourse units, over discourse structure trees that are built according to the rst @xcite requirements. @xcite proposed an entity-centered model that leveraged discourse features like dependencyparse tree distance, sentence distance, and the syntactic positions (subject, object, and oblique) of the mention and antecedent to perform coreference. in this work, we use @xcite 's rst parser to convert documents into rst discoursestructure trees @xcite . from these trees, we derive distance and coverage-based features to model the discourse-level structural constraints, which are passed as input to a neural-network based coreference resolver. to our knowledge, ours is the first work that tries to explicitly incorporate discourselevel constraints for coreference resolution in a neural setting. ## model in this section, we explain how we introduce discourse-level features into a neural cr system. ## experimental setup in this section, we describe the datasets and evaluation metrics we use in our experiments. ## conclusion in this paper, we show that a representation of hierarchical discourse structure is beneficial for entity coreference resolution. our proposed discourseinformed model observes small but statistically significant improvements over a state-of-the-art neural baseline on two coreference resolution datasets. our analysis shows that the impact of the representation on performance is related to the cognitive load imposed by the type of anaphoric mention. while the model proposed in this work could serve as a useful baseline for the benefits of including discourse structure-based features in neural coreference resolution models, we realize that there is potential for achieving additional improvements by including more complex constraints (e.g. right frontier constraint @xcite ). we plan to study the affect of such features in future work.