  Abstractive summarization is the task of compressing a long document into a
coherent short document while retaining salient information. Modern abstractive
summarization methods are based on deep neural networks which often require
large training datasets. Since collecting summarization datasets is an
expensive and time-consuming task, practical industrial settings are usually
low-resource. In this paper, we study a challenging low-resource setting of
summarizing long legal briefs with an average source document length of 4268
words and only 120 available (document, summary) pairs. To account for data
scarcity, we used a modern pretrained abstractive summarizer BART (Lewis et
al., 2020), which only achieves 17.9 ROUGE-L as it struggles with long
documents. We thus attempt to compress these long documents by identifying
salient sentences in the source which best ground the summary, using a novel
algorithm based on GPT-2 (Radford et al., 2019) language model perplexity
scores, that operates within the low resource regime. On feeding the compressed
documents to BART, we observe a 6.0 ROUGE-L improvement. Our method also beats
several competitive salience detection baselines. Furthermore, the identified
salient sentences tend to agree with an independent human labeling by domain
experts.
