Information Extraction for Planning Court Cases (2024)

Abstract: Legal documents are often long and unstructured, making them challenging and time-consuming to apprehend. An automatic system that can identify relevant entities and labels within legal documents, would significantly reduce the legal research time. We developed a system to streamline legal case analysis from planning courts by extracting key information from XML files using Named Entity Recognition (NER) and multi-label classification models to convert them into structured form. This research contributes three novel datasets for the Planning Court cases: a NER dataset, a multi-label dataset fully annotated by humans, and newly re-annotated multi-label datasets partially annotated using LLMs. We experimented with various general-purpose and legal domain-specific models with different maximum sequence lengths. It was noted that incorporating paragraph position information improved the performance of models for the multi-label classification task. Our research highlighted the importance of domain-specific models, with LegalRoBERTa and LexLM demonstrating the best performance.

## introduction the use of artificial intelligence (ai) techniques within the legal domain has been rapidly growing, transforming the way legal professionals handle their complex tasks @xcite . the advancement in natural language processing (nlp) in legal informatics @xcite has significantly enhanced tasks such as question-answering, judgment predictions, and information extraction from legal text @xcite @xcite . for countries with common-law jurisdictions like the uk and the us, legal research needs to be consistent with referencing similar past cases @xcite . however, legal research is extremely time-consuming due to the extensive length of legal texts @xcite and the need for domain expertise to navigate the specialised vocabulary and legal jargon @xcite . additionally, the unstructured nature of legal documents, such as court hearings, adds to the complexity @xcite . to address these challenges, an nlp-based technique that can automatically extract relevant information from unstructured legal cases into a structured format would be highly beneficial. the primary task is twofold: (1) structuring the raw document formats (pdf and xml) of these planning court cases, and (2) curating a novel dataset to support future research efforts. to achieve this, we apply named entity recognition (ner) and multilabel classification techniques, which are effective at organizing and categorizing legal information. large language models (llms) have demonstrated strong capabilities in the legal sector @xcite ), but they require large amounts of domain-specific, accurate data @xcite .we choose to use traditional extractive methods, such as named-entity recognition, which are well suited to the need for precision in the legal workflow and do not yield hallucinations. we opt to build upon those methods that are widely used by legal search softwares and propose to improve them using llms. this research focuses on cases from the planning court, part of the administrative court of england and wales foot_0 , provided by the find case law service of the national archives uk. it addresses two main issues: structuring the initial document format (pdf and xml) of planning court cases using llms, and curating a novel dataset for future research. we have employed ner and multilabel classification to bring structure to it. the project will benefit: the primary research questions guiding the project are as follows: to address these research questions, this study investigated the utility of language models in extracting information from legal documents specific to the planning court. our contributions are as follows: (1) we create a novel dataset of planning court cases specifically curated for ner and multi-label classification, (2) we propose an endto-end pipeline to extract and structure this data using ner and multi-label classification to analyse those cases automatically by extracting legally relevant entities and paragraphs, (3) lastly we create a structured database from our results, allowing for a quick and efficient search based on the extracted entities. ## ethics statement the curated dataset contains sensitive information, including the names of claimants and appellants. our research utilizes data that is already publicly available and not anonymous. we have obtained permission to use this data under the open justice licence provided by the find case law service, which allows us to copy, publish, distribute, and transmit the information. our primary task is to transform this semi-structured data into a structured format. while we acknowledge the potential concerns regarding dual use, we focus on streamlining the analysis of legal cases, making the likelihood of such concerns minimal.