Named Entity Recognition in I ndian court judgments (2022)

Abstract: Identification of named entities from legal texts is an essential building block for developing other legal Artificial Intelligence applications. Named Entities in legal texts are slightly different and more fine-grained than commonly used named entities like Person, Organization, Location etc. In this paper, we introduce a new corpus of 46545 annotated legal named entities mapped to 14 legal entity types. The Baseline model for extracting legal named entities from judgment text is also developed.

## introduction artificial intelligence has the potential to increase access to justice and make various legal processes more efficient @xcite . populous countries such as india have a problem with high case pendency. as of march 2022, over 47 million cases are pending in indian courts 1 . hence, it becomes imperative to use ai to reduce the strain on the judicial system and reduce pendency. for developing legal ai applications, it is essential to have access to judicial data and open-source foundational ai building blocks like named entity recognition (ner). a lot of indian legal data is publicly available thanks to open data initiatives like national judicial data grid (njdg) and the crime and criminal tracking @xcite . njdg provides non-exhaustive metadata of indian court judgements like the names of petitioners, respondents, lawyers, judges, date, court etc. extracting these entities from judgment text makes the information extraction exhaustive and reduces errors like misspellings compared to njdg metadata. helpful information like precedents and statutes are also not written in the njdg metadata. hence it is essential to extract from court judgment texts rather than just relying on the published njdg metadata. extracting named entities from the text also paves the foundation for more tasks like relation extraction, coreference resolution, knowledge graph creation etc. in this paper, we have created a corpus of annotated judgment texts with 14 legal entities (details in §3). an example of annotated entities is shown in figure 1 . we make the following contributions in this paper • we create a corpus of 14444 indian court judgment sentences and 2126 judgment preambles annotated with 14 legal named entities. supreme court of india criminal appeal jurisdiction [arising out of special leave petition (crl.) no. 7999/2010 state of kerala court petitioner ... appellant -versus-raneef respondent ... respondent judgement markandey katju judge 1. leave granted 2. heard learned counsel for the parties 3. the appellant has filled this appeal challenging the impugned order of the kerala high court dated 17.09.2010 granting bail to the respondent dr. raneef , who is a medical practitioner (dentist) in ernakulam district in kerala , and is accused in crime no. 704 of 2010 of p.s. muvattupuzha for offences under various provisions of the i.p.c. ,the explosive substances act , and the unlawful activities (prevention) act . the raneef other_person kerala gpe p.s. muvattupuzha org i.p.c. statute explosive substances act statute unlawful activities (prevention) act statute kerala high court ernakulam gpe court 17.09.2010 date preamble judgement text figure 1: legal named entities in a court judgment transition-based parsing for ner was proposed by @xcite using stacked lstm. ner task can be treated as graph-based dependency parsing @xcite to provide a global view of the input using biaffine model. recent advances in span representation have shown promising results for named entity recognition @xcite . span pretraining methods @xcite improve the span representation for pre-trained language models via span-level pretraining tasks. infusing external knowledge for entity representation and linking @xcite , @xcite helps to better represent the knowledge in legal texts. @xcite considered interrelation between spans by considering the neighbouring spans integrally to better model the entity boundary information. recently a lot of work has been done in the legal ai field in the indian context. structuring the court judgments @xcite , legal statute identification @xcite , judgment outcome prediction @xcite , judgment summarization @xcite provide ai building blocks. @xcite ) created in-legalbert and incaselawbert, which are further pre-trained versions of legalbert @xcite and caselawbert @xcite respectively on indian legal text. ## ner baseline model the end goal behind this work is to enable the development of other legal ai applications that consume automatically detected legal named entities from judgment texts. towards this goal, we experimented with some famous ner model architectures. a single model was trained to predict entities from both judgment sentences and the preamble. as transformer-based architectures have shown a lot of success in ner tasks @xcite , we mainly experimented with them. we compared the performance of 2 ner architecture types when trained on our legal ner dataset. the first architecture type uses a transition-based dependency parser @xcite on top of the transformer model. the second architecture type uses a fine-tuning based approach which adds a single linear layer to the transformer model and fine-tunes the entire architecture on the ner task. figure 2 shows the 2 ner architecture types. we experimented with multiple transformer models for each of the architecture types. for transition-based parser architecture we experimented with the roberta-base model @xcite , inlegalbert @xcite using spacy library. for the fine-tuning approach we experimented with roberta-base, inlegalbert, legalbert @xcite using tner library @xcite . the models are evaluated by using recall, precision and strict f1 scores on combined preamble and judgment sentences. the named entity is considered correct when both boundary and entity class are predicted correctly. performance of the best performing model (roberta+ transition-based parser) on each of the entity classes on test data along with average character length is shown in the trained roberta+transition-based parser is made available as a spacy pipeline in our git repository and hugging face model repository foot_3 . ## post-processing of named entities since the annotators were asked to annotate individual sentences without document-level context, any trained ner model on this data will also focus only on the sentence-level information. while inferring the ner model on a complete judgment text, it is important to perform post-processing of extracted entities to capture document-level context. in particular, we create rules to perform the following tasks ## conclusion & future directions in this paper, we proposed a new corpus of legal named entities using 14 legal entity types. we also proposed baseline models trained using this corpus along with post-processing of the extracted entities to capture document-level information. we have also released a representative sample of indian court judgments which could be used in further studies. we believe this corpus will lay the foundation for further nlp tasks like relationship extraction, knowledge graph population etc. using indian court judgments.