SC on E :Contextual Relevance based S ignificant C ompo N ent E xtraction from Contracts (2022)

Abstract: Automatic extraction of “significant” components of a legal contract, has the potential to simplify the end user’s comprehension. In essence, “significant” pieces of information have 1) information pertaining to material/practical details about a specific contract and 2) information that is novel or comes as a “surprise” for a specific type of contract. It indicates that the significance of a component may be defined at an individual contract level and at a contract-type level. A component, sentence, or paragraph, may be considered significant at a contract level if it contains contract-specific information (CSI), like names, dates, or currency terms. At a contract-type level, components that deviate significantly from the norm for the type may be considered significant (type-specific information (TSI)). In this paper, we present approaches to extract “significant” components from a contract at both these levels. We attempt to do this by identifying patterns in a pool of documents of the same kind. Consequently, in our approach, the solution is formulated in two parts: identifying CSI using a BERT-based contract-specific information extractor and identifying TSI by scoring sentences in a contract for their likelihood. In this paper, we even describe the annotated corpus of contract documents that we created as a first step toward the development of such a language-processing system. We also release a dataset of contract samples containing sentences belonging to CSI and TSI.

## introduction contracts are agreements, between two or more parties, that govern what each party can or cannot do and are usually dense in information. extracting contract elements and locating novel clauses and assignments from a legal contract is a desired feature by many as it will greatly simplify and accelerate user comprehension. traditionally, it requires a domain expert as there are parts of a contract that can only be noticed by a reader experienced in reviewing contracts. for an untrained eye, it is often difficult and time consuming to identify rare and unique sentences. to reduce dependency on experts and to lessen the human effort required, in this paper we introduce approaches for automatic identification and extraction of significant components of the contract. when compared with the corpora on which most pre-trained deep models are based, the structure and vocabulary of texts in contracts differ significantly. contracts frequently take constrained forms, sometimes even "template-like" for the sake of ensuring legal unambiguity. on carefully examining the semantics and structure of diverse legal contracts sourced from sec edgar foot_0 (employment, software license, purchase, severance), we observe that i) within contracts of same category, although the wording and sentence structure differ between individual contracts, the information conveyed remains almost the same, ii) within an individual contract, within an individual contract, we have compo-ii) within an individual contract, we have components, sentences or paragraphs that are remarkably distinct with little redundancy components in an individual contract can be broadly classified as: templatised sentences are sentences that follow a template. a phrase or only a part of the sentence may vary and the rest of the content is semantically same across contracts. examples include contract elements @xcite like title of the contract, parties involved in the contract, dates, governing law. as observed in boilerplate sentences foot_1 sentences that are standard formulations, and uniformly found in all contracts of a type. they are huge in number and constitute a large portion of contract. in this paper, we extend the definition of boilerplate sentences to include sentences which contain the same semantic content across contracts but differ lexically and structurally. as we can see in rare sentences in a contract include content not commonly found in contracts of that type and hence are conspicuous by their presence in the current contract. in rare sentences to achieve balance, your current tax withholdings may cease and a hypothetical rate of tax may be calculated and withheld from your wages. no additional stock units granted as part of the award may be earned following the change in control. the contract because they bring in novelty. rare sentences in a contract are identified on the basis of contract type to which a contract belongs. in summary, a contract hasi) template sentences, which contain contract specific information (csi) and are generic across contract types. this approach of extracting significant components does not really qualify as a standard summarization task because there is no merit in summarizing boilerplate sentences which are well understood. abstractive summarization @xcite techniques would inadvertently change the semantics of the contract. even when compared to an extractive setting @xcite , in this study our main focus is to accentuate rare and templatised sentences as significant components in comparison to boilerplate sentences. unfortunately, we are not aware of any large, open corpora of contracts for running comparable experiments. the outcome of our approach is presented in two formats a) highlighted input document figure 1-where sections of interest are highlighted within the overall contract. this helps in vizualizing the significant components of the contract. b) a cover-page -a consolidated page containing the extracted significant components. the effectiveness of the automated significant components identification model was further evaluated by conducting an experimental study that compares the performance between human and machine for the task. the contribution of our work in addition to identifying "significant" components is to understand how much fine-tuned data is required for achieving a moderately reasonable accuracy. this becomes important as contract types can vary considerably, and organizations would be burdened with huge annotation efforts for every document type. ## related work language being the core of law and legal contracts, an increased interest in applying natural language processing techniques to a wide range of problems ranging from information extraction to sentence prediction in law @xcite @xcite has been observed. considerable amount of work has been done in contract analysis and information extraction from contracts @xcite @xcite . the most obvious approach to automatic contract element extraction is to model it as sequence labeling task. statistical methods like conditional random fields @xcite were popular for sequence labeling prior to neural networks. @xcite involved hand written rules along with hand crafted features to uniquely identify and extract the contract elements. recently, neural networks @xcite @xcite and bert @xcite based approaches @xcite were developed for sequence labeling and slot with joint intent classification. our work for csi extraction closely resembles @xcite where the contract elements are extracted from regulatory filings and property lease agreements using the standard bio tagging scheme for the contract elements of interest. we include more categories of contracts (employment, incentive, purchase, severance, software-license) and the contract elements are majorly kept consistent. scope identification is another popular area of research in legal domain as it is tedious to read legal documents. contracts or legal documents contain many key sentences. it often becomes necessary to have domain knowledge regarding contracts to avoid missing any important or key information. summarization @xcite is a reliable approach and summarizing legal contracts was attempted @xcite by taking the document features and ordering the sentences according to their importance. classification and hand crafted rules @xcite was another recent approach to precisely identify the scope and was applied to construction contracts to identify requirements automatically. these techniques do not differentiate between boilerplate sentences which forms the bulk of the contract and the other sentences of the contract. instead of summarizing well understood and accepted clauses, our study intends to focus on contract specific information (csi) and contract type specific information (tsi). regression @xcite is another technique where the sentences are scored on their importance and the model learns to include sentences in a summary based on the scores it predicts. based on our observations that legal contracts of same category have repetitive information, we devised an approach to calculate sentence likelihood with respect to the contract type and use these scores to identify tsi. the likelihood scores calculated using labse @xcite while bert @xcite was adapted to learn and predict these likelihood scores. ## approach significant component extraction is accomplished in two stages: the overall architecture is shown in figure 2 . the input to the model is a document d containing a set of sentences s. the output is a set of sentences p, that effectively highlight information unique and specific to the document d, such that @xmath0. ## identifying csi identifying contract elements is similar in approach to identifying named entities but is not directly extendable without retraining them on contracts. ner systems typically identify persons, organizations, dates, locations, currency terms etc,. contract elements would carry more features attributed to it along with being a named entity. for example, a ner system can identify dates and persons but will not be able to differentiate if the date is an effective start date or termination date. similarly not all instances of persons, organization or location in a contract would be contract parties or governing law elements. the sentences that contain these csi are almost in a template like schema, therefore training a sequence labeling model to understand the sentence semantics and to extract sentences which contain contract elements, yields better results. we sampled 500 legal documents (100 documents of each category mentioned in ## identifying tsi contract type specific information (tsi) extraction problem has not been studied extensively and is the main focus of our study. we identify unique or novel details concerning the contract by looking at structural and semantic similarities among a pool of contracts belonging to a specific type. a clause that is rare for an employment type contract may not be rare for a stock options awards type contract. figure 3 highlights few clauses that may seem ordinary but are different from their usual construction in contracts. based on our observations, legal contracts of same category have repetitive information (boilerplate). this requires ranking the sentences based on a metric for rarity. scoring sentences @xcite based on both importance and redundancy among sentences was attempted for summarization @xcite tasks. the approach however, does not guarantee inclusion of rare and unique sentences as sentences scored based on their importance are most likely to pick boilerplate sentences since they are the core of any contract. redundancy is almost negligible for business documents like contracts. textrank (mihalcea and tarau, 2004) is a popular graph-based unsupervised ranking model for text processing. it identifies text units that best define the task at hand and links them with sub units of text by identifying relations among them. the local outlier factor (lof) algorithm @xcite is an unsupervised anomaly detection method which computes deviation of a given data point with respect to its neighbors. we applied both textrank and lof algorithm on our sampled data as baselines. since our aim is to capture the rare sentences, we sorted textrank scores in ascending order and considered the top sentences as rare. though the model works well in capturing rare information, deciding on the threshold or cut-off is often difficult as it would differ from contract to contract and contract type to contract type. we devised an unsupervised approach to calculate tsi score with respect to the contract type and use these scores to identify tsi. tsi score of a sentence here indicates the confidence with which a given sentence is a part of a specific contract type. identifying rare components of a contract type is often limited by the presence of named entities in templatised sentences. these templatised sentences, though common across contract types, would be counted as rare by the virtue of having named entities in them. the information contained in such sentences is often extracted using contract element extraction approaches @xcite . in order to ignore these sentences and to make sentences more comparable across contracts we mask all the named-entities in contracts using spacy foot_2 to replace named-entities by their type (people's names to person, organization names to org). masking sentences that contain namedentities increases its tsi score. ## human evaluation to assess the effectiveness of the tsi model we conducted an experimental study that compares the performance of the model against a human annotated corpus. two annotators were asked to read the contract set provided to them and then label the sentences as rare or familiar. we chose to make this a binary classification task for the humans in order to reduce cognitive load. model generated scores in the test set were converted to labels based on their likelihood scores thresholded by the knee-point value for each class in figure 4 . if the sentence score is below the threshold set for rare sentences, then the sentences are labeled rare (0) . if the sentence score is above the threshold (set for rare sentences), then it is labeled familiar (1) . the ## conclusion and future work our work is an attempt to study the structure of contracts and harness the semantic "strictness" of these contracts in order to extract "significant" pieces of information contained therein. here, significance is defined by two distinct ideas: rarity in a type of contract and commonality across types. we find that this view of contracts removes a need to review elements which are boilerplate and would, in turn, reduce the effort required to find critical content in a given contract. we show that our models can achieve reasonable accuracy with relatively low training data. this work can be extended in future to a query based model by taking input from the users in the form of a query and highlight text most relevant to a given query. since the task is novel and there exists no parallel corpora, we wish to release sentences, that are rare and sentences that contain contract specific elements from the sampled contracts.