Precedent-Enhanced Legal Judgment Prediction with LLM and Domain-Model Collaboration (2023)

Abstract: Legal Judgment Prediction (LJP) has become an increasingly crucial task in Legal AI, i.e., predicting the judgment of the case in terms of case fact description. Precedents are the previous legal cases with similar facts, which are the basis for the judgment of the subsequent case in national legal systems. Thus, it is worthwhile to explore the utilization of precedents in the LJP. Recent advances in deep learning have enabled a variety of techniques to be used to solve the LJP task. These can be broken down into two categories: large language models (LLMs) and domain-specific models. LLMs are capable of interpreting and generating complex natural language, while domain models are efficient in learning task-specific information. In this paper, we propose the precedent-enhanced LJP framework (PLJP) – a system that leverages the strength of both LLM and domain models in the context of precedents. Specifically, the domain models are designed to provide candidate labels and find the proper precedents efficiently, and the large models will make the final prediction with an in-context precedents comprehension. Experiments on the real-world dataset demonstrate the effectiveness of our PLJP. Moreover, our work shows a promising direction for LLM and domain-model collaboration that can be generalized to other vertical domains.

## introduction legal ai has been the subject of research for several decades, with the aim of assisting individuals in various legal tasks, including legal qa @xcite , court view generation @xcite , legal entity recognition @xcite , and so on. as one of the most important legal tasks, legal judgment prediction (ljp) aims to predict the legal judgment of the case based on the case fact description. the legal judgment typically includes the law article, charge and prison term. precedents, which refer to previous cases with similar fact descriptions, hold a crucial position within national legal systems @xcite . on a more macro level, precedents are known as the collective body of judge-made laws in a nation @xcite . they serve the purpose of ensuring consistency in judicial decisions, providing greater legal guidance to judges and facilitating legal progress and evolution to meet dynamic legal demands. in the common law system, the precedents are the mandatory basis of the judgment of the subsequent case @xcite . in the civil law system, judge-made laws are perceived as secondary legal sources while written laws are the basic legal sources @xcite . in the contemporary era, there is also a growing trend to treat the precedents as a source of "soft" law @xcite , and judges are expected to take them into account when reaching a decision @xcite . thus, it is worthwhile to explore the utilization of precedents in the legal judgment prediction. with the development of deep learning, many technologies have been adopted in the ljp task, which can be split into two categories: large lan-guage models (llms) and domain-specific models @xcite . owing to extensive training, llms are good at understanding and generating complex natural language, as well as in-context learning. on the other hand, domain-specific models are designed to cater to specific tasks and offer cost-effective solutions. however, when it comes to incorporating precedents into the ljp task, both categories of models face certain limitations. llms, constrained by their prompt length, struggle to grasp the meaning of numerous abstract labels and accurately select the appropriate one. for domain models, though trained with label annotations, the drawback is the limited ability to comprehend and distinguish the similarities and differences between the precedents and the given case. in this paper, as fig. 1 shows, we try to collaborate the llms with the domain-specific models and propose a novel precedent-enhanced legal judgment prediction framework (pljp). specifically, domain models contribute by providing candidate labels and finding the proper precedents from the case database effectively; the llms will decide the final prediction through an in-context precedent comprehension. following the previous ljp works @xcite @xcite , our experiments are conducted on the publicly available real-world legal dataset. to prevent any potential data leakage during the training of the llms, where the model may have already encountered the test cases, we create a new test set comprising cases that occurred after 2022. this is necessary because the llms we utilize have been trained on a corpus collected only until september 2021. by doing so, we ensure a fair evaluation of the pljp framework. remarkably, our proposed pljp framework achieves state-of-the-art (sota) performance on both the original test set and the additional test set. to sum up, our main contributions are as follows: ## problem formulation in this work, we focus on the problem of legal judgment prediction. we first clarify the definition of the terms as follows. then the problem can be defined as: problem 1 (legal judgment prediction). given the fact description f , our task is to get and comprehend the precedents p , then predict the judgment @xmath0, c, t). ## precedent-enhanced ljp (pljp) in this section, we describe our precedent-enhanced legal judgment prediction framework (pljp), fig. 2 shows the overall framework. ## conclusion and future work in this paper, we address the important task of legal judgment prediction (ljp) by taking precedents into consideration. we propose a novel framework called precedent-enhanced legal judgment prediction (pljp), which combines the strength of both llms and domain models to better utilize (e.g., retrieve and comprehend) the precedents. experiments on the real-world dataset prove the effectiveness of the pljp. based on the pljp, in the future, we can explore the following directions: 1) develop methods to identify and mitigate any biases that could affect the predictions and ensure fair and equitable outcomes. 2) validate the effectiveness of llm and domain collaboration in other vertical domains such as medicine and education.