A rg L egal S umm: Improving Abstractive Summarization of Legal Documents with Argument Mining (2022)

Abstract: A challenging task when generating summaries of legal documents is the ability to address their argumentative nature. We introduce a simple technique to capture the argumentative structure of legal documents by integrating argument role labeling into the summarization process. Experiments with pretrained language models show that our proposed approach improves performance over strong baselines.

## introduction abstractive summarization has made great progress by leveraging large pretrained language models such as bart @xcite , t5 @xcite , pegasus @xcite , and longformer @xcite . these models leverage large scale datasets such as @xcite , pubmed @xcite , and new york times @xcite . unlike news and scientific texts, which contain specific formatting such as topic sentences and abstracts, legal cases contain implicit argument structure spreading across long texts @xcite . current abstractive summarization models do not take into account the argumentative structure of the text, which poses a challenge towards effective abstractive summarization of legal documents. in this work, we bridge the gap between prior research focusing on summarizing legal documents through extracting argument roles of legal text @xcite @xcite , and prior research focused on producing abstractive summaries of legal text @xcite . our work proposes a technique that blends argument role mining and abstractive summarization, which hasn't been explored extensively in the literature. figure 1 describes the main flow of our approach, which decomposes the summarization process into two tasks. first, each sentence in the document is assigned an argument role by using an independent model. then, the predicted roles are blended with the original document's sentences and fed into a sequence to sequence-based abstractive summarizer. our contributions are as follows: 2 related work legal document summarization. prior research has mainly focused on extractive techniques @xcite @xcite , exploiting features such as the document structure and prior knowledge of legal terms to extract salient sentences that represent the summary of the legal text. recent research has also shifted gears to abstractive techniques due to their superiority to extractive methods on automatic measures such as rouge @xcite . these abstractive techniques benefited from neural models such as pointer generator networks @xcite (utilized in legal public opinion summarization @xcite ) and transformer-based se-quence to sequence encoder-decoder architectures such as bart @xcite and longformer @xcite (employed to summarize long legal documents @xcite ). however, the current abstractive approaches ignore the argumentative structure of the legal text. in our work, we combine both the rich argumentative structure of legal documents and state-of-the-art abstractive summarization models. argument mining. argument mining aims to represent the argumentative structure of a text in a graph structure that contains the argument roles and their relationship to each other. constructing the graphs usually involves several steps: extracting argument units, classifying the argument roles of the units, and detecting the relationship between different argument roles. recently, contextualized embeddings were employed to improve argument role labeling @xcite . in many domains, argument roles are classified into claims, major claims, and premises as proposed in @xcite . alternatively, @xcite proposed to classify the argument roles in legal documents to issues, reasons, and conclusions which fits the legal text structure. we use the same set of legal argument role labels in our work, and use contextualized embeddings to automatically predict them. argument mining and summarization. prior research integrating argument mining and summarization has mainly focused on extracting chunks of text that contain key argument units @xcite @xcite . combining argument mining and abstractive summarization has received less attention in the literature. @xcite proposed a dialogue summarization dataset with argument information. in their work, the authors included argument information in abstractive summarization by linearizing the argument graph to a textual format, which is used to train an encoder-decoder summarization model. however, their proposed approach didn't improve over encoder-decoder baselines. we propose an alternative method that relies on argument roles only, which shows higher improvements over encoder-decoder baselines. ## experiments and results our experiments are conducted in two settings: assuming argument roles are manually labeled (which we refer to as oracle) versus predicting argument role labels (referred to as predicted). ## conclusion and future work we proposed to utilize argument roles in the abstractive summarization of legal documents to accommodate their argumentative structure. our experiments with state-of-the-art encoder-decoder models showed that including argument role information can improve the rouge scores of summarization models capable of handling long documents. specifically, improved results were achieved using longformer with input documents augmented with argument roles (highlighted using special marker tokens), and this finding was robust across two special token schemes. we also showed that using predicted argument roles showed consistent improvements to using the manually labeled ones. in future work, we plan to explore methods to unify argument mining and summarization to reduce the computational resources necessary to host two models.