Knowledge Graph-based Thematic Similarity for I ndian Legal Judgement Documents using Rhetorical Roles (2022)

Abstract: Automation in the legal domain is promising to be vital to help solve the backlog that currently affects the Indian judiciary. For any system that is developed to aid such a task, it is imperative that it is informed by choices that legal professionals often take in the real world in order to achieve the same task while also ensuring that biases are eliminated. The task of legal case similarity is accomplished in this paper by extracting the thematic similarity of the documents based on their rhetorical roles. The similarity scores between the documents are calculated, keeping in mind the different amount of influence each of these rhetorical roles have in real life practices over determining the similarity between two documents. Knowledge graphs are used to capture this information in order to facilitate the use of this method for applications like information retrieval and recommendation systems.

## introduction the indian legal system is currently facing the problem of legal pendency owing to the large volume of cases that are filed each day. this is exacerbated by the lack of trained legal professionals and absence of good resources to aid legal experts. thus, creating sound legal tech systems, especially ones aided with the advancements that have been seen by the field of artificial intelligence, is imperative for remedying the current situation. identifying similarity between documents isn't a new task, but identifying the similarity between legal text documents is challenging, understudied and quite essential. a mechanism that can identify the similarity of legal cases quite clearly would make a great backbone for a powerful information retrieval engine or a recommendation system that would greatly boost any legal expert's research and preparations. presently, popular legal information retrieval systems like https://indiankanoon.org/ of-ten use plain full-text similarity that doesn't make use of any kind of context, semantic or otherwise. through this work, the task of legal case similarity is explored using knowledge graphs. to accomplish this, the use of semantic segments present in case documents, identified using deep learning, and the jurisdiction of the case itself are implemented. this information is then used to calculate similarity scores for case documents. a knowledge graph is used to store the extracted information because of its ability to capture the relationships between the documents. the intent behind creating such a system is to provide a sturdy and reliable backbone to information retrieval engines that can be used by legal experts and laymen alike for research, preparation, study, etc. the paper is structured as follows: the subsection 2 illustrates the related work of researchers, section 3 shows the framework. section 4 explains the methodology, that includes, description of the dataset, data-preparation steps, similarity metrics and methods used and details of how the case documents are stored in a knowledge graph. section 5 discusses the result and application of this work in the real world. finally, section 6, concludes the paper with a brief view about future scope. ## proposed methodology the intent with this work was to create a framework that is robust, reliable and focused on equity without bias . therefore, the proposed solution introduces a framework that operates on an input case judgment document and generates as output the most similar case documents. the database consists of documents from the ildc (indian legal document corpus) dataset. rhetorical roles are then identified from all documents, similarity scores computed between the segments and represented as a knowledge graph as depicted in figure 1 . ## results on comparing expert-anointed rankings for case documents based on their full-text similarity, tf-idf with cosine similarity metric was observed to capture it most effectively with the highest micro f1 score of 0.71, amongst other methods such as bert and doc2vec. rhetorical roles were identified using the scibert-hsln approach which achieved a micro f1 of 77.7 on hidden test data. similarity scores were further enhanced to achieve a micro f1 score of 0.73, by integrating the aspect of thematic similarity using rhetorical roles. it was found to model expert-scores more accurately than the full-text methods alone. this information when represented in the form of a knowledge graph through the neo4j database yielded an average query response time of 223 ms, showing an improvement nearly 10 times faster than retrieving the documents saved in a table. in a use case such as ours, it is of essence to note that the relationship between different documents is important. only capturing course-grained details like document text and leaving similarity calculations to application logic leads to added computational costs incurred at every query, which can be avoided if this information is stored in a knowledge graph at the time that a new document is added to the database. ## conclusion this paper introduces a new method to compute and store similarity between judicial case documents. it identifies the rhetorical roles in case documents and leverages it to find the thematic similarity between the documents. the similarity scores thus obtained are stored in a knowledge graph along with the documents. in this representation the documents and their semantic segments are captured in the nodes and the similarity scores are represented as the edge weights. it is shown that this method can be used for building reliable information retrieval systems and recommendation systems. for future work, using this data-structure for other legal tasks such as judgment prediction can be explored. it also remains to be determined if the task of explainability can be enhanced by the context provided by the identification of rhetorical roles. the weights in this work are determined using expert-rule methods. as an extension, different deep learning methods can be used to ascertain the optimal weights associated with the edges.