H i C u LR : Hierarchical Curriculum Learning for Rhetorical Role Labeling of Legal Documents (2024)

Abstract: Rhetorical Role Labeling (RRL) of legal documents is pivotal for various downstream tasks such as summarization, semantic case search and argument mining. Existing approaches often overlook the varying difficulty levels inherent in legal document discourse styles and rhetorical roles. In this work, we propose HiCuLR, a hierarchical curriculum learning framework for RRL. It nests two curricula: Rhetorical Role-level Curriculum (RC) on the outer layer and Document-level Curriculum (DC) on the inner layer. DC categorizes documents based on their difficulty, utilizing metrics like deviation from a standard discourse structure and exposes the model to them in an easy-to-difficult fashion. RC progressively strengthens the model to discern coarse-to-fine-grained distinctions between rhetorical roles. Our experiments on four RRL datasets demonstrate the efficacy of HiCuLR, highlighting the complementary nature of DC and RC.

## introduction rhetorical role labeling (rrl) of legal documents involves assigning the functional role played by each sentence of the document in the legal discourse (e.g., preamble, fact, evidence, reasoning). rrl for long legal case documents is a precursor task for several downstream tasks, such as case summarization @xcite @xcite , semantic case search @xcite , case law analysis @xcite and argument mining @xcite . initial works on rrl of legal judgements have regarded the task as straightforward classification of sentences without modeling any contextual dependency between them @xcite which later got to be viewed as sequence labeling @xcite . initial works @xcite @xcite ) performed rrl using hand-crafted features as part of a summarization pipeline. further works @xcite used conditional random fields on these hand-crafted features. recently, deep learning-based methods have been applied to this task on japanese documents @xcite , indian documents @xcite @xcite . these methods adopt a hierarchical approach to account for the sequential sentence classification nature of the task, drawing context from surrounding sentences. this architecture, initially based on word embeddings @xcite , has transitioned to bert-based contextualized embeddings @xcite , becoming the standard approach for rrl tasks. recently, @xcite reformulated the task as span-level sequential classification that segment the document into sets of contiguous sequence of sentences (spans) and assign them labels. further @xcite proposed contrastive and prototypical learning to effectively leverage knowledge from semantically similar instances (neighbours). all these current approaches typically perform fine-tuning by presenting all training examples in a completely random order during training. however, the difficulty levels of discourse structures in legal documents can vary significantly, with some following a standard format identifiable by simple lexical clues, while others require deeper analysis due to unconventional writing styles. similarly, identifying rhetorical roles such as preamble, which encompass broad definitions, is easier compared to finer-grained roles like the ratio of the decision, which is often confused with analysis or ruling by the present court. in this work, we leverage these observations to employ curriculum learning (cl) @xcite , where the model's training process mimics a meaningful learning order in-spired by human cognition. by excluding the negative impact of difficult examples in the early training stages, cl can guide learning towards a better local minima in the parameter space @xcite . cl has demonstrated success in various nlp tasks such as machine translation @xcite @xcite , nlu @xcite , amr parsing @xcite , summarization @xcite @xcite , datato-text generation @xcite , information retrieval @xcite @xcite , pre-training @xcite and even in legal tasks such as statutory retrieval for mining negatives @xcite . in this work, we propose a hierarchical curriculum learning approach that nests two complementary curricula: (i) document-level curriculum, which orders input documents based on their difficulty. we explore various difficulty metrics, such as deviations from standard discourse structure, to order these documents. we employ a baby-step scheduler @xcite , gradually exposing the model to more difficult samples over time. (ii) rhetorical role-level curriculum, which utilizes similarity-based scheduling @xcite . ordering rhetorical roles in a sequential easy-to-hard sequence, as in the document-level curriculum, is infeasible since we cannot disentangle sentences from a document, which is our input. hence we use a similarity-based curriculum where the model is initially allowed to belong to similar rhetorical roles to a lesser extent, instead of solely belonging to its ground-truth role and it gets refined over training. that is, in the beginning small mistakes of similar classes are less corrected than bigger mistakes, resembling a teaching process in which broad concepts are explained before subtle differences are addressed later. our experimental results on four rrl datasets demonstrate the effectiveness of our various difficulty metrics for both dc and rc, as well as our model-agnostic hiculr framework, which incorporates both curricula. ## preliminaries task : given a judgment document @xmath0, x 2 , . . . , x m } with m sentences as the input, where x @xmath1, x i2 , . . . , x in } represents the i th sentence containing n tokens, the task of rrl is to predict sequence of @xmath2, l 2 , . . . , l m } where l i is the rhetorical role corresponding to sentence x i and l @xmath3, set of predefined rhetorical roles. baseline : we use hierarchical sequential labeling network from prior works @xcite and demonstrate effectiveness of our model-agnostic hiculr framework in conjunction with it. initially, each sentence x i is encoded independently using a bert model @xcite to derive token-level representations. these token-representations are passed through a bi-lstm layer @xcite , followed by an attention pooling layer @xcite , to yield sentence representations. these are passed through bi-lstm layer to obtain contextualized sentence representations to encode information from surrounding sentences which are passed through a crf layer @xcite ) that predicts the best sequence of labels. the model is trained end-to-end with standard cross entropy loss. ## hiculr framework a curriculum learning framework consists of two main components -difficulty scoring function which quantifies the data based on relative easiness and a pacing function which arranges the transitioning of data from easy to difficult examples for training @xcite . in this section, we describe our document-level difficulty estimators along with the baby-step scheduler (3.1) and rhetorical role-level curriculum with similarity based scheduling (3.2). further, we describe combining both of these complementary curricula in a nested hierarchical fashion (3.3). ## experiments & results we experiment on four rrl datasets: (i) build @xcite , (ii) paheli @xcite (iii) m-cl and (iv) m-it @xcite , derived from indian legal judgments. build has 13 rhetorical roles annotated, while the others have 7 each. detailed dataset descriptions and implementation details are in app. a and b. ## conclusion in this work, we proposed a novel hierarchical curriculum learning framework, hiculr, for rrl of legal documents. hiculr integrates complementary curricula, with rhetorical role-level curriculum (rc) on the outer layer and document-level curriculum (dc) on the inner layer we investigate different difficulty scoring metrics in dc and similarity based strategies in rc, observing notable improvements in performance. particularly, the data-based probabilistic method within dc and the confusion matrix approach within rc stood out as effective strategies. our experiments on four rrl datasets, verify the effectiveness of hicurl.