Word-Level Coreference Resolution (2021)

Abstract: Recent coreference resolution models rely heavily on span representations to find coreference links between word spans. As the number of spans is O(n^2) in the length of text and the number of potential links is O(n^4) , various pruning techniques are necessary to make this approach computationally feasible. We propose instead to consider coreference links between individual words rather than word spans and then reconstruct the word spans. This reduces the complexity of the coreference model to O(n^2) and allows it to consider all potential mentions without pruning any of them out. We also demonstrate that, with these changes, SpanBERT for coreference resolution will be significantly outperformed by RoBERTa. While being highly efficient, our model performs competitively with recent coreference resolution systems on the OntoNotes benchmark.

## introduction coreference resolution is used in various natural language processing pipelines, such as machine translation @xcite , question answering @xcite ) and text simplification @xcite . as a building block of larger systems itself, it is crucial that coreference resolution models aim to not only achieve higher benchmark scores but be time and memory efficient as well. most recent coreference resolution models for english use the mention ranking approach: the highest ranking antecedent of each span is predicted to be coreferent to that span. as this approach presents a serious computational challenge of o(n 4 ) complexity in the document length, @xcite keep only m top-scoring spans, while @xcite extend this idea by keeping only top k antecedents for each span based on an easily computable score. their c2f-coref model, which showed 72.6 f1 on the ontonotes 5.0 shared task @xcite , was later improved by @xcite , who introduced spanbert to obtain better span representations (79.6 f1), and by @xcite , who proposed a novel way of higher-order coreference resolution (80.2 f1). the current state-of-the-art system built by @xcite uses a different approach, formulating the task as a machine reading comprehension problem. while they were able to achieve an impressive 83.1 f1 on the ontonotes 5.0 shared task, their model is particularly computationally expensive as it requires a full transformer pass to score each span's antecedents. to make coreference resolution models more compact and allow for their easier incorporation into larger pipelines, we propose to separate the task of coreference resolution from span extraction and to solve it on the word-level, lowering the complexity of the model to o(n 2 ). the span extraction is to be performed separately only for those words that are found to be coreferent to some other words. an additional benefit of this approach is that there is no need for a complex span representation, which is usually concatenation-based. overall this allows us to build a lightweight model that performs competitively with other mention ranking approaches and achieves 81.0 f1 on the ontonotes benchmark. foot_0 2 related work ## experiments we use the ontonotes 5.0 test dataset to evaluate our model and compare its performance with the results reported by authors of other mention ranking models. the development portion is used to reason about the importance of the decisions in our model's architecture. ## conclusion we introduce a word-level coreference model that, while being more efficient, performs competitively with recent systems. this allows for easier incorporation of coreference models into larger natural language processing pipelines. in addition, the separation of coreference resolution and span extraction tasks makes the model's results more interpretable. if necessary, the span prediction module can be replaced by a syntax-aware system to account for non-continuous spans. such task separation also makes the coreference module independent from the way coreferent spans are marked up in the training dataset. this will potentially simplify using various data sources to build robust coreference resolution systems.