Dual Cache for Long Document Neural Coreference Resolution (2023)

Abstract: Recent works show the effectiveness of cache-based neural coreference resolution models on long documents. These models incrementally process a long document from left to right and extract relations between mentions and entities in a cache, resulting in much lower memory and computation cost compared to computing all mentions in parallel. However, they do not handle cache misses when high-quality entities are purged from the cache, which causes wrong assignments and leads to prediction errors. We propose a new hybrid cache that integrates two eviction policies to capture global and local entities separately, and effectively reduces the aggregated cache misses up to half as before, while improving F1 score of coreference by 0.7 5.7pt. As such, the hybrid policy can accelerate existing cache-based models and offer a new long document coreference resolution solution. Results show that our method outperforms existing methods on four benchmarks while saving up to 83% of inference time against non-cache-based models. Further, we achieve a new state-of-the-art on a long document coreference benchmark, LitBank.

## introduction coreference resolution (cr) is fundamental in document-level natural language processing. its goal is to identify mentions belonging to the same entity. these mentions often scatter throughout the document, but they can be linked together through coreference resolution. cr is a building block for common sense understanding @xcite @xcite , reading comprehension @xcite , information extraction @xcite @xcite , and text summarization @xcite @xcite . conventional cr models enumerate every pair of mentions in a document in parallel, so the com-maria skłodowska was born in warsaw, in congress poland in the russian empire, on 7 november 1867 ... ## related work conventional cr methods @xcite @xcite are designed for short documents with length less than 1,000 words. they typically encode the input document and propose candidate mentions with a pretrained model. their model enumerates all pairs of mentions in parallel to identify coreference relations, resulting in memory and computation cost quadratic to document length. these models also require a post-processing to gather connected mentions as entities and need to compute a large tensor of o(n × n × d), where n is the number of mentions, and d is the dimension of hidden states in the relation classifier. this is often impractical for long documents with thousands of mentions. another direction is to reformulate cr, @xcite treats cr as a question answering problem and achieves higher performance on some benchmarks but costs more computation since the model needs to do question answering repeatedly. ## method a common workflow of neural cr models is to detect candidate mentions, encode them into vector representations, and identify coreference relations between each mention and past entities by feeding their representations to an mlp classifier. we denote candidate mentions as {m 1 , m 2 , • • • , m n }, and an entity is defined as a set of mentions, we use a special operator h(•) to represent vector representations, for example, h(m 1 ) is the representation of the mention m 1 . ## experiments we conduct experiments on four cr benchmarks and compare with prior cache-based cr methods. ## conclusion we proposed a new cache structure, dual cache, to tackle long document coreference resolution. it contains two caches governed by different eviction polices. the local cache follows lru policy to deal with local, clustered mentions, whereas the global cache follows lfu policy to target global entities within a long span. empirical results show that the dual cache lowers the cache miss up to half as before and improves the f1 score consistently on four benchmarks by offering an average gain of 5.7 f1 score for small caches while taking the same inference time. we also achieve a new stateof-the-art, 79.5 f1 score, on litbank.