Unleashing the Power of LLM s in Court View Generation by Stimulating Internal Knowledge and Incorporating External Knowledge (2024)

Abstract: Court View Generation (CVG) plays a vital role in the realm of legal artificial intelligence, which aims to support judges in crafting legal judgment documents. The court view consists of three essential judgment parts: the charge-related, law article-related, and prison term-related parts, each requiring specialized legal knowledge, rendering CVG a challenging task.Although Large Language Models (LLMs) have made remarkable strides in language generation, they encounter difficulties in the knowledge-intensive legal domain.Actually, there can be two types of knowledge: internal knowledge stored within LLMs’ parameters and external knowledge sourced from legal documents outside the models.In this paper, we decompose court views into different parts, stimulate internal knowledge, and incorporate external information to unleash the power of LLMs in the CVG task.To validate our method, we conduct a series of experiment results on two real-world datasets LAIC2021 and CJO2022. The experiments demonstrate that our method is capable of generating more accurate and reliable court views.

## introduction in legal artificial intelligence (legal ai), the task of court view generation (cvg) has been studied for years @xcite @xcite , aiming to generate the judgment document based on the fact description of a legal case to assist judges in writing legal documents. the court view is mainly composed of three parts: the charge-related part, the law article-related part, and the prison term of the defendant. each part is formed by the judgment result and rationale, as shown in the fig. 1 . also, each part exhibits distinctive legal characteristics, requiring different legal knowledge, which makes cvg a challenging task. the previous cvg works mostly focus on generating more fluent court views while ignoring the figure 1 : an example case. the court view consists of three judgment parts. the text highlighted in color represents the judgment results, while the rest indicates the judgment rationale. the internal and external legal knowledge can be utilized by llms for enhanced generation. however, the llms perform poorly in knowledge-intensive legal domain tasks. how to effectively augment llms with legal knowledge bases to improve performance in the cvg task remains a challenge. in order to provide the judges more accurate and reliable court views, we unleash the power of llms by leveraging multiple legal knowledge bases. more specifically, we stimulate internal knowledge and incorporate external knowledge to construct multiple legal knowledge bases as shown in the tab. 1, and then prompt the llm to interact with these knowledge bases in the different judgment parts. first, in the task of the charge-related part, the problem of confusing charges is a quite common issue in real judgment scenarios @xcite . we consider that many lawyers have written numerous clarifications about confusing charges on the website, and we assume that this kind of knowledge has been crawled as part of the pre-training corpus of the llms. so we stimulate the internal knowledge of llms by prompting the llms to generate clarifications for all the charge pairs. secondly, unlike charges, the label of law articles are meaningless index numbers (e.g. article 347), which makes it difficult for llms to generate a sequence of numbers. to address this, given that the definitions of law articles are readily accessible, we collect these definitions as external knowledge. subsequently, we retrieve the relevant law article definitions and append them to the input text, thereby incorporating external legal knowledge. thirdly, in terms of the judgment of the prison term, many penalty principles have been released by the supreme court but often remain underutilized. taking drug trafficking as an example, the prison term can be calculated precisely based on factors such as the amount of drugs, type of drugs, and so on. as a result, the prison term can be calculated in a symbolic way instead of learning a deep neural network with a large amount of data. here we gather penalty principles corresponding to each charge as external knowledge. we utilize the internal ability of llms to generate python code based on these penalty principles. in a given case, llms can extract relevant elements from the fact description and utilize them as parameters for calculating the prison term. overall, we decompose the cvg task into three distinct parts, employing distinct strategies for generation based on specific legal knowledge. eventually, these three judgment parts are integrated to form a complete court view document. due to the fact that the pretraining corpus of llms primarily entails data up until 2021, we create a new dataset named cjo2022. this dataset is composed of criminal cases sourced from china judgements online in the year 2022. we conduct a series of experiments on the laic2021 dataset and the newly constructed cjo2022 dataset to validate our method. the experiments demonstrate that our method can generate more accurate and reliable court views and has competitive or even better performance against fully supervised state-of-the-art (sota) methods with only a few in-context examples. in summary, our contributions can be outlined as follows: ## method in this section, we decompose the court view generation (cvg) task into three parts and describe our method for each part. we stimulate the internal knowledge stored in the llms and incorporate external legal knowledge to generate the different judgment parts, as shown in the tab. 1. ## conclusion and future work in this paper, we explore the court view generation (cvg) task, leveraging the powerful generation capabilities of llms. to apply llms in the knowledge-intensive cvg task, we decompose the cvg into three different parts and construct multiple legal knowledge bases by stimulating internal knowledge and incorporating external knowledge. the llms are enhanced by interacting with different legal knowledge bases in different subprocesses to generate more accurate and reliable court views. the experiments on two real-world datasets validate the effectiveness of our method. in the future, we will explore applying llms to other legal tasks (e.g., legal elements extraction, etc.) and combining llms with a broader range of legal knowledge to adapt to different legal tasks.