Enhancing Legal Case Retrieval via Scaling High-quality Synthetic Query-Candidate Pairs (2024)

Abstract: Legal case retrieval (LCR) aims to provide similar cases as references for a given fact description. This task is crucial for promoting consistent judgments in similar cases, effectively enhancing judicial fairness and improving work efficiency for judges. However, existing works face two main challenges for real-world applications: existing works mainly focus on case-to-case retrieval using lengthy queries, which does not match real-world scenarios; and the limited data scale, with current datasets containing only hundreds of queries, is insufficient to satisfy the training requirements of existing data-hungry neural models. To address these issues, we introduce an automated method to construct synthetic query-candidate pairs and build the largest LCR dataset to date, LEAD, which is hundreds of times larger than existing datasets. This data construction method can provide ample training signals for LCR models. Experimental results demonstrate that model training with our constructed data can achieve state-of-the-art results on two widely-used LCR benchmarks. Besides, the construction method can also be applied to civil cases and achieve promising results. The data and codes can be found in https://github.com/thunlp/LEAD.

## introduction legal case retrieval (lcr) aims to search for historically relevant cases based on a given fact description @xcite @xcite @xcite . this task can help legal professionals, such as judges and lawyers, improve work efficiency by providing past cases as references for current judgments. thus, it plays a crucial role in promoting judicial fairness by facilitating similar cases receiving similar judgments. different from open-domain retrieval, lcr demands a complex understanding of case details and necessitates models equipped with legal knowledge to generate knowledge-rich case representations @xcite . as shown in figure 1 , models are required to recognize that the severity of injury rather than the location of injury is the key factor in assessing the relevance of given candidates to the query. recent years have seen significant efforts by scholars to improve the performance of lcr, including introducing additional knowledge features @xcite @xcite and designing lcr-oriented pre-training objectives @xcite . however, despite these advancements, the realworld application of lcr still faces the following challenges: (1) asymmetric retrieval. existing methods mostly focus on symmetric retrieval settings with lengthy fact descriptions for both queries and candidates. in contrast, real-world user queries often consist of only a few sentences describing key details. this inconsistency between application and training scenarios results in sub-optimal performance. (2) limited data. another challenge is the limited data scale, as legal data annotation requires highly skilled and experienced annotators, making it time-consuming and labor-intensive. existing lcr datasets contain only a few hundred queries @xcite , compared to tens of thousands in open-domain retrieval datasets @xcite @xcite . besides, most retrieval methods rely heavily on data-hungry neural models, making the construction of large-scale, high-quality legal retrieval data a key to enhancing lcr performance. to address these issues, this paper proposes a method for automatically constructing high-quality, synthetic legal retrieval datasets for model training. specifically, given a case candidate, we employ a large-scale generative language model to first extract key facts, and omit entities, including names and places. then, based on the anonymous key fact, we require the model to generate a brief and coherent description of the case, which is regarded as the search query. in this way, the generated query is short and contains only a few sentences. additionally, to improve data diversity and enable the model to retrieve relevant cases even when key facts are not entirely consistent, we employ a knowledge-driven data augmentation strategy. for each query, we select the case that is most similar from the perspective of charges, related legal articles, and prison term, from the entire corpus as the augmented positive candidate. this approach enables us to rapidly build the largest lcr dataset, lead, with over 100k querycandidate pairs and without any manual annotation, surpassing existing lcr datasets by a hundredfold. to verify the effectiveness of our method, we train dense passage retrieval models with lead and compare the model with several competitive baseline models, on two widely-used criminal lcr benchmarks. the experimental results demonstrate that models trained with our enriched high-quality case retrieval data can achieve state-of-the-art performance in lcr tasks. besides, the proposed framework for data generation can be easily applied to civil case retrieval, and achieve satisfying performance. the code and data in our paper will be released to promote the development of lcr. earliest work for lcr attempt to employ traditional retrieval models, including, bm25 @xcite and tf-idf @xcite , for legal retrieval @xcite . with the development of deep learning, many efforts have been devoted into designing neural architectures to enhance long textual representation @xcite , interpretability @xcite , legal knowledge enriched representation @xcite @xcite @xcite . due to the lack of a largescale lcr dataset, these researches mainly focus on the re-ranking phrase, overlooking the significance of dense passage retrieval (dpr) for high recall rate @xcite . to elevate the data scarcity issues, some researchers explore the self-supervised pre-training for legal dpr. for instance, sailer @xcite ) adopts an asymmetric encoder-decoder architecture, integrating various pre-training objectives to encode rich semantic information across tasks. caseencoder @xcite leverages fine-grained legal provisions to select relevant and irrelevant cases for each query, thus improving the quality of training data. in this paper, we find that our data construction methods can further facilitate the lcr performance by scaling the high-quality instances for lcr. ## data construction to address the challenges of asymmetric retrieval, queries in the training dataset should align with real-world user queries, which are often characterized by brevity and conciseness. as shown in figure 2 , we propose an automatic method to generate queries based on case facts. we will introduce the details about the data generation in this section. ## conclusion in this paper, we propose a method for automatically constructing high-quality, asymmetric legal case retrieval datasets. we construct the largest lcr dataset to date, with over one hundred thousand query-candidate pairs, surpassing existing datasets by a hundredfold. we conduct experiments on two widely-used datasets, achieving state-ofthe-art performance in lcr tasks. moreover, our method is highly versatile, showing superior performance in civil case retrieval as well.