Since the introduction of deep pretrained language models , most task-specific NLP models were reduced to simple lightweight layers. An exception to this trend is the challenging task of coreference resolution, where a sophisticated end-to-end model is appended to a pretrained transformer encoder. While highly effective, the model has a very large memory footprint -- primarily due to dynamically-constructed span and span-pair representations -- which hinders the processing of complete documents and the ability to train on multiple instances in a single batch. We introduce a lightweight  coreference model that removes the dependency on span representations, handcrafted features, and heuristics. Our model performs competitively with the current end-to-end model, while being simpler and more efficient.
The introduction of pretrained language models has reduced many complex task-specific NLP models to simple lightweight layers. An exception to this trend is coreference resolution, where a sophisticated task-specific model is appended to a pretrained transformer encoder. While highly effective, the model has a very large memory footprint -- primarily due to dynamically-constructed span and span-pair representations -- which hinders the processing of complete documents and the ability to train on multiple instances in a single batch. We introduce a lightweight end-to-end coreference model that removes the dependency on span representations, handcrafted features, and heuristics. Our model performs competitively with the current standard model, while being simpler and more efficient.
R
Since the introduction of deep
The introduction of
R
, most
has reduced many complex
D
were reduced
D
the challenging task of
R
end-to-end
task-specific
A
end-to-end
R
end-to-end
standard