Legal Case Winning Party Prediction With Domain Specific Auxiliary Models (2022)

Abstract: Sifting through hundreds of old case documents to obtain information pertinent to the case in hand has been a major part of the legal profession for centuries. However, with the expansion of court systems and the compounding nature of case law, this task has become more and more intractable with time and resource constraints. Thus automation by Natural Language Processing presents itself as a viable solution. In this paper, we discuss a novel approach for predicting the winning party of a current court case by training an analytical model on a corpus of prior court cases which is then run on the prepared text on the current court case. This will allow legal professionals to efficiently and precisely prepare their cases to maximize the chance of victory. The model is built with and experimented using legal domain specific sub-models to provide more visibility to the final model, along with other variations. We show that our model with critical sentence annotation with a transformer encoder using RoBERTa based sentence embedding is able to obtain an accuracy of 75.75%, outperforming other models.

## introduction natural language processing (nlp) is undergoing rapid development and has proven to be practically useful across many text rich domains. with the proper utilization of tools and technologies, effective methodologies can be derived to tackle various problems that are repetitive, cognitively demanding and time consuming otherwise. legal domain is such a text rich domain with a growing need for task automation. legal domain corpora consists of statutes, regulations, constitutions and case law documents among many others which have to be repeatedly and constantly sifted through by legal professionals to obtain information pertinent to their current case. this research is primarily carried on case law documents where a model is train on a corpus of existing case law documents so that a prediction of the winning party in a current case law document can be obtained. ## related work in the work by shaikha et al. (2020), they have categorized the past approaches to predict the outcome of a legal case into three categories. three approaches are distinguished by the use of 1) political or social science based, 2) linguistics based or 3) legal domain based features as the descriptors for the machine learning algorithms they use. 19 features have been formalized with respect to the legal domain, that has the potential to impact the decision of a criminal court case. it is important to note that feature extraction is manually done by going through court cases, and therefore it requires experts to identify the features. after feature extraction and preprocessing, researchers have conducted classification under 8 different algorithms such as regression trees, bagging and random forests, support vector machines and k-nearest neighbours. classification and regression trees have been found to be the best performing. in the research by @xcite , they have conducted their research fundamentally on german tax law cases. the research is conducted on features extracted using mostly regular expressions and manual annotations. a naive bayes classifier have been chosen as the best performing machine learning model. they have achieved 0.57 precision, 0.58 f1 score and 0.60 recall for positive outcomes. research done by @xcite on predicting the decision of the european court of human rights, is identified as the first systematic approach to predicting winning parties by using nlp, as per the authors. they have modeled the problem as a binary classification problem, while using support vector machines and n-grams and topics as features for the model. @xcite also proposes a classification approach for identifying the winning party of a court case. the process consists of two phases. in the 1st phase, an article classification model extracts top k articles that are cited in the case document. in the 2nd phase, the judgement classification model tries to predict the judgement of the court case. they have considered domain specific aspects such as punishment, cited statutes and features derived using nlp such as sentiment, as features for their model. a tree based approach which uses new feature engineering techniques is proposed in the research conducted by @xcite . the dataset used in this research consist of cases from the united states supreme court. researchers have considered the impact from political biases for the decisions as well. they have used data ranging over multiple presidential terms to generalize the model more. features already present with there chosen dataset have been used and some has been introduced by them. with the 7700 cases used, they have succeeded in getting 69.7% accuracy and individual judge votes with 70.9% accuracy. @xcite have proposed a machine learning approach to develop a system that predicts brazilian court decisions. researchers have suggested for it to be used as a supporting tool or a benchmark for legal professionals. the approach to calculate both the decision class and the unanimity of decisions have been designed. they have achieved good accuracy for some of the many model variations. ## methodology in this section, the approach used for dataset preparation and the methodology for deriving the architecture used in this research are be discussed. ## experiments and results experiments are performed by varying the document encoder model configurations and application of additional details to case sentences using the critical sentence identification model @xcite . document encoder is experimented using different rnn configurations and transformer encoder configurations. to identify the number of layers best suitable for the transformer encoder, it was experimented with layers 6,3,2, and 1. as seen in the fig 4 , the best number of layers for the transformer encoder was found to be 1 in this case. rnn and transformer encoder components are used to encode the case documents. rnn models are experimented with both gru and lstm variations. pre-trained sentence-bert by @xcite , based on bert @xcite and distilbert by @xcite ,a distilled version of the roberta-base @xcite , models are used for sentence embedding. model building, training and evaluation are done using tensorflow v2.8. the following configurations were used for the transformer encoder: we summarize out findings in ## conclusion and future work legal domain corpora carries its own complexities due to the domain nature. therefore applying nlp in the legal domain requires domain specific approaches. in this study, we showed that our model with critical sentence annotation with a transformer encoder using roberta based sentence embedding is able to obtain an accuracy of 75.75%, outperforming other models. the need for domain-specific models can also be seen by the increase in accuracy when the critical sentence annotation is used. this system can be horizontally extended by adding more sub models to provide features to the final model. while the results obtained by distilbert @xcite sentence embeddings are impressive, extending the conclusions drawn by @xcite for word embeddings, it can be postulated that legal-domain specific sentence embeddings would potentially reap better results. also as future work, the impact of having models trained with supervised approaches and unsupervised approaches should be experimented, as legal domain has a deficit of labeled data compared to its large corpora.