Exploiting Contrastive Learning and Numerical Evidence for Confusing Legal Judgment Prediction (2023)

Abstract: Given the fact description text of a legal case, legal judgment prediction (LJP) aims to predict the case’s charge, applicable law article, and term of penalty. A core problem of LJP is distinguishing confusing legal cases where only subtle text differences exist. Previous studies fail to distinguish different classification errors with a standard cross-entropy classification loss and ignore the numbers in the fact description for predicting the term of penalty. To tackle these issues, in this work, first, in order to exploit the numbers in legal cases for predicting the term of penalty of certain charges, we enhance the representation of the fact description with extracted crime amounts which are encoded by a pre-trained numeracy model. Second, we propose a moco-based supervised contrastive learning to learn distinguishable representations and explore the best strategy to construct positive example pairs to benefit all three subtasks of LJP simultaneously. Extensive experiments on real-world datasets show that the proposed method achieves new state-of-the-art results, particularly for confusing legal cases. Ablation studies also demonstrate the effectiveness of each component.

## introduction legal judgment prediction (ljp) is one of the most attractive research topics among legal artificial intelligence @xcite @xcite . given the fact description text of a legal case, ljp aims to predict its charge, applicable law article, and term of penalty @xcite @xcite @xcite @xcite . one core problem hindering the performance of ljp from being satisfying is confusing legal cases, which have subtle text or number differences, but with totally different charges, applicable law articles, or terms of penalty. figure 1 shows two examples of confusing legal cases. figure 1 (a) shows an example whose golden charge label is crime of picking a quarrel, which is easily classified into its confusing charge crime of intention injury. figure 1 ( @xcite shows an erroneous prediction of the term of penalty, which does not exploit the amounts related to the crime. a series of studies on this problem has been conducted, including manually discriminative legal attributes annotation @xcite , distinguishing representations learning via graph neural networks @xcite , and separating the fact description into different circumstances for different subtasks @xcite . however, we argue that there exist two drawbacks to these studies. firstly, all of these studies use a standard cross-entropy classification loss, which cannot distinguish different mistaken classification errors. for example, the error of classifying the charge crime of picking a quarrel into its confusing charge crime of intention injury is the same as classifying crime of picking a quarrel into a not confusing charge crime of rape. the model should be punished more if it classifies a charge into its corresponding confusing charges. secondly, the crime amounts in the fact description are crucial evidence for predicting the penalty terms of certain types of cases, such as financial legal cases. however, the crime amounts are distributed randomly throughout the fact description. thus it is difficult for the model to directly deduce the precise total crime amount and predict correct penalty terms based on the scattered numbers. to tackle these issues, we present a framework that leverages numerical evidence and moment contrast-based supervised contrastive learning for confusing ljp. firstly, the framework proposes to extract numerical evidence (the total crime amount) from the fact description as a named entity recognition (ner) task for predicting the term of the penalty, where the recognized numbers make up the total crime amount. this formulation is able to address the difficulty of directly deducing the precise total crime amount from the fact description where the numbers are scattered randomly throughout and only some of them are part of the crime amount, while others are not. then, the extracted numerical evidence is infused into the term of penalty prediction model while preserving its numeracy, which is achieved by a pre-trained number encoder model @xcite . secondly, in order to pull fact representations from the same class closer and push apart fact representations from confusing charges, the framework introduces the moment contrast-based supervised contrastive learning(scl; @xcite @xcite ) and explores the best strategy to construct positive example pairs to benefit all three subtasks of ljp simultaneously. the proposed moment contrastbased scl addresses two challenges to applying the original in-batch scl to ljp. the first challenge is that the number of charge classes is significantly greater than the number studied in previous studies @xcite ) (e.g., 119 classes for charge prediction), which increases the difficulty of finding sufficient negative examples in the minibatches. to address it, we introduce a momentum update queue with a large size for scl, which allows for providing sufficient negative examples. the second challenge is when applying the original single-task scl to the multi-task ljp there exists a contradictory phenomenon. contradictory phenomenon means that instances with the same charge label may have different applicable law or penalty term labels. if we pair the training instances with the same charge label as positive examples, the resulting learned shared features will benefit the charge prediction task but will degrade the performance of the other two tasks. to tackle this challenge, we explore the best way to construct positive examples, which can benefit all three subtasks of ljp simultaneously. the proposed framework provides the following merits for predicting confusing ljp: 1) the framework is model-agnostic, which can be used to improve any existing ljp models; 2) the extracted numerical evidence makes the predictions of the term of penalty more interpretable, which is critical for legal judgment prediction; 3) compared with previous studies @xcite , the use of supervised contrastive learning does not require additional manual annotation. we conduct extensive experiments on two realworld datasets (i.e., cail-small and cail-big). the experimental results demonstrate that the proposed framework achieves new state-of-the-art results, obtaining up to a 1.6 f1 score improvement for confusing legal cases and a 3.73 f1 score improvement for numerically sensitive legal cases. ablation studies also demonstrate the effectiveness of each component of the framework. ## background in this section, we formalize the ljp task and its multi-task learning framework. ## case studies figure 4 shows two cases to qualitatively demonstrate the effect of the proposed framework. in the first case, ladan mtl incorrectly predicts the case's charge into its confusing charge crime of intentional injury, which should be crime of picking a quarrel. however, with the proposed framework, this error is corrected. the second case demonstrates the effect of numerical evidence. ladan mtl incorrectly predicts the case's term of penalty as label 9, meaning a sentence of fewer than 6 months. given the accurately extracted crime amount of $10,400, which is a relatively large crime amount, the model correctly predicts the term of penalty as label 7, meaning a sentence of more than 9 months but less than 12 months. ## conclusion in this paper, we present a framework that introduces moco-based supervised contrastive learning and weakly supervised numerical evidence for confusing legal judgment prediction. the framework is capable of automatically extracting numerical evidence for predicting number-sensitive cases and learning distinguishable representations to benefit all three subtasks of ljp simultaneously. extensive experiments validate the effect of the framework.