NITS _ L egal at S em E val-2023 Task 6: Rhetorical Roles Prediction of I ndian Legal Documents via Sentence Sequence Labeling Approach (2023)

Abstract: Legal documents are notorious for their complexity and domain-specific language, making them challenging for legal practitioners as well as non-experts to comprehend. To address this issue, the LegalEval 2023 track proposed several shared tasks, including the task of Rhetorical Roles Prediction (Task A). We participated as NITS_Legal team in Task A and conducted exploratory experiments to improve our understanding of the task. Our results suggest that sequence context is crucial in performing rhetorical roles prediction. Given the lengthy nature of legal documents, we propose a BiLSTM-based sentence sequence labeling approach that uses a local context-incorporated dataset created from the original dataset. To better represent the sentences during training, we extract legal domain-specific sentence embeddings from a Legal BERT model. Our experimental findings emphasize the importance of considering local context instead of treating each sentence independently to achieve better performance in this task. Our approach has the potential to improve the accessibility and usability of legal documents.

## introduction legal case documents are typically quite lengthy, often spanning many pages, which can make it time-consuming for legal practitioners and academics to read them in their entirety. in many cases, these professionals may only need to access specific portions of a document, such as the facts of the case or the arguments put forward by the parties involved @xcite . however, legal case documents are often unstructured and lack clear section headings, unlike research papers or books. this can make it difficult for readers to quickly and efficiently locate the information they need. the lack of structure in legal case documents can be particularly problematic for legal practitioners who are trying to build a strong legal argument. without the ability to easily navigate and access relevant information, lawyers may struggle to build a coherent case that is based on sound legal principles and precedents. to address this challenge, researchers and practitioners are exploring a range of approaches to structuring legal case documents and making them more accessible to readers. this includes the development of new tools foot_0 and technologies that can automatically extract key information from legal documents, such as the parties involved, the legal issues at stake, and the arguments put forward by each side @xcite . by leveraging such tools, legal practitioners and academics can more quickly and efficiently access the information they need to build strong legal arguments and advance the field of law. rhetorical role labeling of sentences is a technique that can help legal practitioners quickly comprehend the structure and specific components of a legal case document @xcite . this method involves identifying the semantic function associated with each sentence in the document. formally, rhetorical role labeling refers to the process of classifying each sentence in a legal document based on its role in the overall document @xcite . by understanding the specific function of each sentence, legal practitioners can more easily identify the relevant portions of the document and extract the information they need to build a strong case. such kind of upstream tasks are also helpful for performing downstream tasks such as summarization @xcite . this can save valuable time and improve the efficiency of legal research and analysis. to facilitate research in this area, the organizers of legaleval 2023 have proposed a task of rhetorical labeling. the dataset provided for this task includes 247 training document-summary pairs, 30 development document-summary pairs, and 50 docfor more details about the task, please refer to the overview paper @xcite . in this work, a detailed experimental study is conducted to solve the problem of rhetorical role labeling, by considering both sentence level as well as sentence sequence level classification approaches. moreover, the utilization of legal domainspecific sentence embeddings is also considered in this work so that better model training is possible. from the experimental study it has been identified that domain-specific embeddings along with local context of sentence sequences are important for achieving improved performances in this task. such an approach can significantly improve the comprehension of lengthy legal documents for legal practitioners as well as other readers. our code is publicly available foot_1 . the rest of the paper is organized as follows: section 2 presents the related works for the rhetorical roles prediction task. section 3 describes our method used for performing rhetorical roles prediction. section 4 presents the experimental results along with a detailed discussion. finally, section 5 concludes the paper with future research directions. ## related works legal documents play a critical role in our society, as they provide the foundation for laws and regulations that govern our behavior. however, these documents can be difficult to read and understand, even for legal experts. in the recent years, there has been growing interest in the legal specific tasks such as rhetorical labeling @xcite , legal document summarization @xcite @xcite , court judgment prediction @xcite @xcite ) and so on. there have been several comparative analysis works performed using the legal documents to understand the behavior of these lengthy documents @xcite @xcite . due to the progress of deep learning techniques, researchers have started employing these methods to analyze indian court judgments for rhetorical labeling tasks as well. farzindar and lapalme @xcite as well as hachey and grover @xcite have used the concept of rhetorical roles to generate summaries of legal texts. this approach involves identifying the various roles played by different segments of the text. by understanding these roles, the researchers were able to create condensed summaries that captured the main points of the text while maintaining its overall structure and coherence @xcite . recently, bhattacharya et al. @xcite have proposed bilstm-crf model to automatically assign rhetorical roles to the sentences of an indian case judgement document. in another work, @xcite have constructed a corpus of rhetorical roles (rr) and annotated it with 13 different detailed roles. furthermore, they have proposed a multitask learning pipeline for identifying rhetorical roles. in another work done by @xcite , authors have created a larger rhetorical role dataset as compared to the dataset created by @xcite . the authors created a baseline system using the scibert-hsln architecture @xcite . in this work also, we utilize indian legal bert @xcite to create the embeddings of sentences in a sequence preserving the local context, followed by feeding them to a bilstm-based model for identifying the rhetorical roles. ## system description the task of rhetorical role labeling can be modeled in several different ways. keeping in mind the specific characteristics of legal documents, we set out in this work to find the most appropriate approach for solving the rhetorical role labeling problem. this section describes the several different methods explored in this work, along with the specific implementation details. ## conclusion the organizers of legal eval 2023 have introduced a rhetorical roles prediction task (task a) as part of their competition, due to the reputation of indian case judgments for being lengthy and unstructured. our team participated in this task and achieved a 71.43% f1 score on the testing data, as reported on the codalab foot_2 leaderboard. we also conducted exploratory experiments and discovered that context is a significant factor in accurately identifying the rhetorical roles. also when it comes to roles that are very rare across such documents, oversampling based model training can actually be quite helpful. as part of the future work, an ensembling based approach can be explored that combines the predictive power of both oversampling as well as nonoversampling based approaches. such an approach has the potential to achieve even higher quality rhetorical role labeling.