HLDC : H indi Legal Documents Corpus (2022)

Abstract: Many populous countries including India are burdened with a considerable backlog of legal cases. Development of automated systems that could process legal documents and augment legal practitioners can mitigate this. However, there is a dearth of high-quality corpora that is needed to develop such data-driven systems. The problem gets even more pronounced in the case of low resource languages such as Hindi. In this resource paper, we introduce the Hindi Legal Documents Corpus (HLDC), a corpus of more than 900K legal documents in Hindi. Documents are cleaned and structured to enable the development of downstream applications. Further, as a use-case for the corpus, we introduce the task of bail prediction. We experiment with a battery of models and propose a Multi-Task Learning (MTL) based model for the same. MTL models use summarization as an auxiliary task along with bail prediction as the main task. Experiments with different models are indicative of the need for further research in this area.

## introduction in recent times, the legal system in many populous countries (e.g., india) has been inundated with a large number of legal documents and pending cases @xcite . there is an imminent need for automated systems to process legal documents and help augment the legal procedures. for example, if a system could readily extract the required information from a legal document for a legal practitioner, then it would help expedite the legal process. however, the processing of legal documents is challenging and is quite different from conventional text processing tasks. for example, legal documents are typically quite long (tens of pages), * corresponding author highly unstructured and noisy (spelling and grammar mistakes since these are typed), use domainspecific language and jargon; consequently, pretrained language models do not perform well on these @xcite . thus, to develop legal text processing systems and address the challenges associated with the legal domain, there is a need for creating specialized legal domain corpora. in recent times, there have been efforts to develop such corpora. for example, @xcite have developed an english corpus of european court of justice documents, while @xcite have developed an english corpus of indian supreme court documents. @xcite have developed chinese legal document corpus. however, to the best of our knowledge, there does not exist any legal document corpus for the hindi language (a language belonging to the indo-european family and pre-dominantly spoken in india). hindi uses devanagari script (wikipedia contributors, 2021) for the writing system. hindi is spoken by approximately 567 million people in the world @xcite . most of the lower (district) courts in northern india use hindi as the official language. however, most of the legal nlp systems that currently exist in india have been developed on english, and these do not work on hindi legal documents @xcite . to address this problem, in this paper, we release a large corpus of hindi legal documents (hindi legal documents corpus or hldc) that can be used for developing nlp systems that could augment the legal practitioners by automating some of the legal processes. further, we show a use case for the proposed corpus via a new task of bail prediction. india follows a common law system and has a three-tiered court system with district courts (along with subordinate courts) at the lowest level (districts), followed by high courts at the state level, and the supreme court of india at the high-est level. in terms of number of cases, district courts handle the majority. according to india's national judicial data grid, as of november 2021, there are approximately 40 million cases pending in district courts @xcite as opposed to 5 million cases pending in high courts. these statistics show an immediate need for developing models that could address the problems at the grass-root levels of the indian legal system. out of the 40 million pending cases, approximately 20 million are from courts where the official language is hindi (national judicial data grid, 2021). in this resource paper, we create a large corpus of 912,568 hindi legal documents. in particular, we collect documents from the state of uttar pradesh, the most populous state of india with a population of approximately 237 million @xcite . the hindi legal documents corpus (hldc) can be used for a number of legal applications, and as a use case, in this paper, we propose the task of bail prediction. given a legal document with facts of the case, the task of bail prediction requires an automated system to predict if the accused should be granted bail or not. the motivation behind the task is not to replace a human judge but rather augment them in the judicial process. given the volume of cases, if a system could present an initial analysis of the case, it would expedite the process. as told to us by legal experts and practitioners, given the economies of scale, even a small improvement in efficiency would result in a large impact. we develop baseline models for addressing the task of bail prediction. in a nutshell, we make the following main contributions in this resource paper: ## related work in recent years there has been active interest in the application of nlp techniques to the legal domain @xcite . a number of tasks and models have been proposed, inter alia, legal judgment prediction @xcite , legal summarization @xcite , prior case retrieval @xcite , legal question answering @xcite , catchphrase extraction @xcite , semantic segmentation @xcite . legal judgement prediction (ljp) involves predicting the final decision from the facts and arguments of the case. chalkidis et al. (2019) released 11,478 cases from the european court of human rights (echr). it contains facts, articles violated (if any), and the importance scores. malik et al. (2021b) provided 34,816 case documents from the supreme court of india for the prediction task. strickson and de la iglesia (2020) published 4,959 documents from the u.k.'s supreme court (the highest court of appeal). majority of corpora for legal-nlp tasks have been in english; recently, there have been efforts to address other languages as well, for example, @xcite , have created a large-scale chinese criminal judgment prediction dataset with over 2.68 million legal documents. work on legal-nlp in languages other than english is still in its incipient stages. our paper contributes towards these efforts by releasing corpus in hindi. majority of the work in the legal domain has focused on the higher court @xcite @xcite ; however, the lower courts handle the maximum number of cases. we try to address this gap by releasing a large corpus of district court level legal documents. some of the recent work has explored other legal-nlp tasks in languages other than english. chalkidis et al. (2021 ) released a multilingual dataset of 65k european union (e.u.) laws for topic classification of legal documents. the data was translated into the 23 official e.u. languages and annotated with labels from the multilingual thesaurus, eurovoc. luz de araujo et al. (2018) ecourts website extract raw documents raw pdf court documents apply ocr raw text documents prior case retrieval bail judgement prediction legal summarisation hindi legal documents corpus (hldc) final cleaning and collation legal nlp applications document segmentation segmented documents into header and body figure 1: hldc corpus creation pipeline have released 70 documents in portuguese for legal named entity recognition. the dataset contains specific tags for law and legal cases entities in addition to the normal tags for named entities like person, locations, organisation and time-entities. coliee (competition on legal information extraction/entailment) tasks @xcite have published legal data in japanese, along with their english translation. the competition has two sub-tasks, a legal information retrieval task and an entailment identification task between law articles and queries. multiple datasets in chinese have been released for different tasks, namely reading comprehension @xcite , similar case matching @xcite , question answering @xcite . duan et al. (2019) proposed chinese judicial reading comprehension (cjrc) dataset with about 10k documents and almost 50k questions with answers. zhong et al. (2020b) presented jec-qa, a legal question answering dataset collected from the national judicial examination of china with about 26k multiple-choice questions. they augment the dataset with a database containing the legal knowledge required to answer the questions and also assign meta information to each of the questions for in-depth analysis. xiao et al. (2019) proposed cail2019-scm, a dataset containing 8,964 triplets of the case document, with the objective to identify which two cases are more similar in the triplets. similar case matching has a crucial application as it helps to identify comparable historical cases. a historical case with similar facts often serves as a legal precedent and influences the judgement. such historical information can be used to make the legal judgement prediction models more robust. kleinberg et al. (2017) proposed bail decision prediction as a good proxy to gauge if machine learning can improve human decision making. a large number of bail documents along with the binary decision (granted or denied) makes it an ideal task for automation. in this paper, we also propose the bail prediction task using the hldc corpus. ## hindi legal documents corpus hindi legal documents corpus (hldc) is a corpus of 912,568 indian legal case documents in the hindi language. the corpus is created by downloading data from the e-courts website (a publicly available website: https:// districts.ecourts.gov.in/ ). all the legal documents we consider are in the public domain. we download case documents pertaining to the district courts located in the indian northern state of uttar pradesh (u.p.). we focus mainly on the state of u.p. as it is the most populous state of india, resulting in the filing of a large number of cases in district courts. u.p. has 71 districts and about 161 district courts. u.p. is a predominantly hindi speaking state, and consequently, the official language used in district courts is hindi. we crawled case documents from all districts of u.p. corresponding to cases filed over two years, from may 01, 2019 to may 01, 2021. figure 2 shows the map of u.p. and district wise variation in the number of cases. as can be seen in the plot, the western side of the state has more cases; this is possibly due to the high population and more urbanization in the western part. legal documents, particularly in lower courts, are highly unstructured and lack standardization with respect to format and sometimes even the terms used. we converted the unstructured documents to semi-structured documents. we segmented each document into a header and a body. the header contains the meta-information related to the case, for example, case number, court identifier, and applicable sections of the law. the body contains the facts of the case, arguments, judge's summary, case decision and other information related to the final decision. the documents were segmented using regex and rule based approaches as described in appendix d. case type identification: hldc documents were processed to obtain different case types (e.g., bail applications, criminal cases). the case type was identified via the meta-data that comes with each document. however, different districts use a variation of the same case type name (e.g., bail application vs bail app.). we resolved these standardization issues via manual inspection and regex-based patterns, resulting in a final list of 300 unique case types. ## hldc: ethical aspects we create hldc to promote research and automation in the legal domain dealing with underresearched and low-resource languages like hindi. the documents that are part of hldc are in the public domain and hence accessible to all. given the volume of pending cases in the lower courts, our efforts are aimed towards improving the legal system, which in turn would be beneficial for millions of people. our work is in line with some of the previous work on legal nlp, e.g., legal corpora creation and legal judgement prediction (section 2). nevertheless, we are aware that if not handled correctly, legal ai systems developed on legal corpora can negatively impact an individual and society at large. consequently, we took all possible steps to remove any personal information and biases in the corpus. we anonymized the corpus (section 3) with respect to names, gender information, titles, locations, times, judge's name, petitioners and appellant's name. as observed in previous work @xcite , anonymization of a judge's name is important as there is a correlation between a case outcome and a judge name. along with the hldc, we also introduce the task of bail prediction. bail applications constitute the bulk of the cases ( §3), augmentation by an ai system can help in this case. the bail prediction task aims not to promote the development of systems that replace humans but rather the development of systems that augment humans. the bail prediction task provides only the facts of the case to predict the final decision and avoids any biases that may affect the final decision. moreover, the bail corpus and corresponding bail prediction systems can promote the development of explainable systems @xcite , we leave research on such explainable systems for future work. the legal domain is a relatively new area in nlp research, and more research and investigations are required in this area, especially concerning biases and societal impacts; for this to happen, there is a need for corpora, and in this paper, we make initial steps towards these goals. ## bail prediction task to demonstrate a possible applicability for hldc, we propose the bail prediction task, where given the facts of the case, the goal is to predict whether the bail would be granted or denied. formally, consider a corpus of bail documents @xmath3, b 2 , • • • , b i , where each bail document is segmented as b @xmath4, f i , j i , y i ). here, h i , f i , j i and y i represent the header, facts, judge's summary and bail decision of the document respectively. additionally, the facts of every document contain k sentences, more formally, f @xmath5, s 2 i , • • • , s k i ), where s k i represents the k th sentence of the i th bail document. we formulate the bail prediction task as a binary classification problem. we are interested in modelling p θ (y i |f i ), which is the probability of the outcome y i given the facts of a case f i . here, y @xmath6, 1}, i.e., 0 if bail is denied or 1 if bail is granted. ## bail prediction models we initially experimented with off-the-shelf pretrained models trained on general-purpose texts. however, as outlined earlier ( §1), the legal domain comes with its own challenges, viz. specialized legal lexicon, long documents, unstructured and noisy texts. moreover, our corpus is from an under-resourced language (hindi). nevertheless, we experimented with existing fine-tuned (pre-trained) models and finally propose a multi-task model for the bail prediction task. ## future work and conclusion in this paper, we introduced a large corpus of legal documents for the under-resourced language hindi: hindi legal documents corpus (hldc). we semi-structure the documents to make them amenable for further use in downstream applications. as a use-case for hldc, we introduce the task of bail prediction. we experimented with several models and proposed a multi-task learning based model that predicts salient sentences as an auxiliary task and bail prediction as the main task. results show scope for improvement that we plan to explore in future. we also plan to expand hldc by covering other indian hindi speaking states. furthermore, as a future direction, we plan to collect legal documents in other indian languages. india has 22 official languages, but for the majority of languages, there are no legal corpora. another interesting future direction that we would like to explore is the development of deep models infused with legal knowledge so that model is able to capture legal nuances. we plan to use the hldc corpus for other legal tasks such as summarization and prior case retrieval.