F -coref: Fast, Accurate and Easy to Use Coreference Resolution (2022)

Abstract: We introduce fastcoref, a python package for fast, accurate, and easy-to-use English coreference resolution. The package is pip-installable, and allows two modes: an accurate mode based on the LingMess architecture, providing state-of-the-art coreference accuracy, and a substantially faster model, F-coref, which is the focus of this work. F-coref allows to process 2.8K OntoNotes documents in 25 seconds on a V100 GPU (compared to 6 minutes for the LingMess model, and to 12 minutes of the popular AllenNLP coreference model) with only a modest drop in accuracy. The fast speed is achieved through a combination of distillation of a compact model from the LingMess model, and an efficient batching implementation using a technique we call leftover batching. https://github.com/shon-otmazgin/fastcoref

## introduction coreference resolution consists of identifying textual mentions that refer to the same entity in a given text @xcite . this fundamental nlp task can benefit various applications such as information extraction @xcite @xcite , question answering @xcite , machine translation @xcite , and summarization @xcite @xcite . however, compared to other core tasks such as pos tagging, named-entity recognition or syntactic parsing, existing packages and state-of-the-art models for coreference resolution are challenging to apply: there are few easy-to-use packages implementing state-of-the-art models, and the available packages consume a lot of gpu memory, and take very long to process each document. for example, the coreference model in the popular allennlp 1 https://github.com/shon-otmazgin/ fastcoref package @xcite , implementing the model of @xcite , requires 27gb of gpu memory and takes 12 minutes to process the 2.8k documents of the ontonotes corpus, on a v100 gpu. in this work, we introduce f-coref, a new open source python package for simply running an efficient coreference model using a few lines of code. f-coref predicts coreference clusters 29 times faster than the allennlp model (processing the ontonotes corpus in 25 seconds) and requires only 15% of its gpu memory use, with only a small drop in performance (78.5 vs 79.6 average f1). the package also includes lingmess @xcite , a state-of-the-art coreference model, which is almost twice as fast as the allennlp model, while being more accurate (81.4 average f1), under the same api. to achieve f-coref's speed, we use two additive techniques: model distillation of the strong-butslow lingmess model using large unlabeled data, and an effective batching technique that reduces the number of padded tokens in a batch. ## the f-coref api the fastcoref python package is pip installable (pip install fastcoref) and provides an easy and fast api for coreference information with only few lines of code without any prepossessing steps. the f-coref constructor initializes our pretrained model on a single device: the main functionally of the package is the predict function, which accepts a list of texts. the return value of the predict function is a list of corefresult objects, from which one can extract the coreference clusters (either as strings or as character indices over the original texts), as well as the logits for each corefering entity pair: processing can be applied to a collection of texts of any length in a batched and parallel fashion: @xmath0, 'text 2',.., 'text n'] # control the batch size # with max_tokens_in_batch parameter @xmath1.predict( @xmath2, @xmath3 the max_tokens_in_batch parameter can be used to control the speed vs. memory consumption tradeoff, and can be tuned to maximize the utilization of the associated hardware. to control speed vs. accuracy tradeoff, use the larger but more accurate lingmess model, simply import lingmesscoref instead of fcoref: from fastcoref import lingmesscoref @xmath4:0') on top of the provided models, the package also provides the ability to train and distill coreference models on your own data, opening the possibility for fast and accurate coreference models for additional languages and domains. to summarize, the package provides a simple api that makes predicting coreference entities straightforward and easy-to-use. the package supports any text length as input, and performs efficient batching. the package's f-coref model is 29 times faster and 4 times smaller than the popular coreference model in the allennlp package, while the provided lingmess mode is twice as fast the allennlp implementation, and more accurate. ## method we employ two complementary directions in order to obtain a fast and efficient coreference model. first, we substantially reduce the size of the s2e model using knowledge distillation ( §4.1) from the lingmess model. second, our implementation aims to maximize parallelism via batching while limiting the number of unnecessary computations such as padded tokens ( §4.2). ## conclusions we introduce the fastcoref python package for coreference resolution, and hope its speed an ease of use will facilitate work that utilizes coreference resolution at scale.