Text Simplification for Legal Domain: I nsights and Challenges (2022)

Abstract: Legal documents such as contracts contain complex and domain-specific jargons, long and nested sentences, and often present with several details that may be difficult to understand for laypeople without domain expertise. In this paper, we explore the problem of text simplification (TS) in legal domain. The main challenge to this is the lack of availability of complex-simple parallel datasets for the legal domain. We investigate some of the existing datasets, methods, and metrics in the TS literature for simplifying legal texts, and perform human evaluation to analyze the gaps. We present some of the challenges involved, and outline a few open questions that need to be addressed for future research in this direction.

## introduction contracts are legal documents used in several business workflows. they consist of paragraphs of text (clauses) outlining the terms and conditions for the involved parties. prior to signing a contract, the parties need to understand the clauses, to ensure that they are aware of what they are agreeing to. contract clauses are usually very long, domainspecific, and contain several complex phrases ( we believe natural language processing (nlp) techniques for text simplification (ts) can be of particular utility to aid legal document understanding for laypeople without much legal knowledge, who are the target readers for this work. the objective of ts is to provide simpler translations for complex input texts. ts is performed at different levels. lexical simplification aims to replace complex words in a given text with simpler alternatives with equivalent meaning @xcite . syntactic simplification typically involves splitting long sentences in shorter ones @xcite . @xcite @xcite use seq2seq-based supervised methods for ts, owing to the availability of parallel datasets @xcite . there have also been recent advancements in unsupervised ts without the need for parallel datasets @xcite . however, we believe they may not be readily suited to legal ts, due to the extremely complex nature of legal text as op-posed to the complex text seen in general news or wikipedia-like datasets. while prior works on challenges in ts @xcite focus on the quality of the ts datasets and evaluation metrics, we focus on the generalizability of existing ts systems to legal domain, and challenges in using existing evaluation metrics for legal ts. in this paper, we aim to address two main research questions. (1) how do existing simplification methods perform (in the absence of legal parallel datasets) on the task of legal ts? we specifically examine three types of simplification, namely lexical ts, sentence splitting, and end-to-end ts (split-and-rephrase). (2) what are the challenges, if any, in using existing automatic evaluation metrics for legal ts? to this end, we investigate three stateof-the-art (sota) unsupervised ts methods in the legal domain ( §2.1): (a) a bert-based method for lexical simplification @xcite , (b) a rulebased discourse-aware sentence splitting framework @xcite , and (c) a rewardbased simplification method that learns to balance fluency, salience, and simplicity of output translations @xcite . we also investigate sequence-to-sequence-based supervised methods @xcite trained on three recently released parallel datasets for ts ( §2.2). to address the second question, we use several reference-free automatic metrics in the ts literature for simplicity, meaning preservation, and fluency on the model outputs, and conduct human studies to analyze their effectiveness. finally, we outline some of the challenges in adapting existing methods and metrics to the legal domain, and present a few preliminary research questions that need to be addressed for furthering the research in the space of legal ts. ## text simplification for legal domain we use several unsupervised and supervised methods. we briefly describe them below (please refer to appendix b for further details). ## experiments we train the kis model on 67k legal text sentences selected randomly from ledgar dataset that do not occur in the test data (further implementation details in appendix b). for evaluation, we use the ledgar dataset @xcite consisting of securities and exchange commission (sec) contracts. we use 5k sentences randomly sampled from 100 most frequently occurring legal clauses in ledgar. details on the types of clauses and sentence statistics are provided in appendix a. metrics. we evaluate the legal sentences and model outputs on meaning preservation, syntactic simplicity, fluency, hallucination, and readability measures. for readability, we use flesch kincaid (fk) @xcite , smog @xcite , and automatic readability index (ari) @xcite to estimate the minimum age required to understand the given text. we compute syntactic simplicity as the average depth of dependency parse trees of the sentences. for meaning preservation, we use bertscore (bs) @xcite which is a similarity score for each token in the input sentence with each token in the simplified sentence, coverage (cov) @xcite which is the accuracy of filling-in the masked tokens in the masked input sentence using the simplified sentence, and blanc @xcite . we measure hallucination as: (1) % of outputs entailed by the input (entail) computed using sota roberta-based @xcite textual entailment model trained on mnli @xcite , and (2) % of entities (found using spacy library) in the output not present in the input (%unseen) @xcite . we compute fluency (ppl) using perplexity score from gpt-2. ## results results are shown in ## conclusions while legal text is complex and domain-specific, thus making it a very interesting domain for ts, it is still in a nascent stage in nlp literature. we investigate and compare some of sota methods for lexical simplification, sentence splitting, and seq2seq sentence simplification, either unsupervised, or trained on closely related parallel datasets, using automatic metrics and human ratings. we conclude that lexical simplification methods will benefit from having a legal lexicon as they still sometimes generate replacements that do not fit the legal context. seq2seq methods perform only surfacelevel transformations by either directly copying input sentences, or deleting a few phrases to make the sentences shorter, without much paraphrasing. while sentence splitting methods make the long nested sentences much shorter, they do so by sacrificing fluency. reward-based generation method achieves transformations to an extent, but does so at the cost of meaning preservation. legal ts can be particularly challenging, as even expert annotators have varied views of how to simplify legal sentences for laypeople. understanding whether every detail is needed to be conveyed or providing a high-level overview suffices can aid in curating parallel datasets for furthering research in this space.