Leveraging Task Dependency and Contrastive Learning for Case Outcome Classification on E uropean Court of Human Rights Cases (2023)

Abstract: We report on an experiment in case outcome classification on European Court of Human Rights cases where our model first learns to identify the convention articles allegedly violated by the state from case facts descriptions, and subsequently uses that information to classify whether the court finds a violation of those articles. We assess the dependency between these two tasks at the feature and outcome level. Furthermore, we leverage a hierarchical contrastive loss to pull together article-specific representations of cases at the higher level, leading to distinctive article clusters. The cases in each article cluster are further pulled closer based on their outcome, leading to sub-clusters of cases with similar outcomes. Our experiment results demonstrate that, given a static pre-trained encoder, our models produce a small but consistent improvement in classification performance over single-task and joint models without contrastive loss.

## introduction the nlp task of classifying case outcome information from a textual statement of case facts is generally referred to as legal judgement prediction (ljp)(e.g., @xcite . it has been studied using corpora from different jurisdictions, such as the european court of human rights (ecthr) @xcite @xcite @xcite @xcite , chinese criminal courts @xcite @xcite @xcite , us supreme court @xcite , indian supreme court @xcite , the french court of cassation @xcite , brazilian courts (bertalan and ruiz, 2020), the federal supreme court of switzerland @xcite , the turkish constitutional court @xcite , uk courts @xcite , german courts @xcite , the philippine supreme court @xcite , and the thailand supreme court @xcite . in this work, we experiment with classifying case outcomes in the ecthr a and b benchmark tasks introduced by lexglue @xcite . task b is to identify the set of articles of the european convention of human rights (echr) that the claimant alleges to have been violated, while task a's goal is to classify which of the convention's articles has been deemed violated by the court. the input for both tasks is the case's fact description that has been extracted from the published judgement document. it should be noted that, despite these tasks being typically referred to as instances of 'legal judgement prediction', the fact statements are typically not finalized until the decision outcome is known, making the task effectively one of retrospective classification rather than prediction @xcite . while this does lead to distracting and confounding phenomena (see our prior work in @xcite , the dataset remains a useful resource for the development of nlp models that analyze these fact statements for text patterns that correspond to specific convention articles as drafted by the court. consequently, we speak of case outcome classification (coc). positive instances for task a are cases in which an article was deemed violated by the court. negative instances, however, usually encompass not only cases in which that particular article was alleged and considered not violated, but also the cases in which the particular article was not alleged in the first place. given that the conditional probability of a positive task a label (violation) given its task b label (allegation) can be very high in the lexglue dataset, we posit that models can fall into this pitfall of identifying dominant articles with high conditional violation probability, missing information specific to violation classification. thus, we experiment with multi-task architectures to decouple these reasoning steps involved in task a . this is similar to work in chinese criminal cases judgement prediction carried out in @xcite @xcite that coordinated multiple outcome variables (law articles, criminal charges, and penalty terms). in the ecthr context, most of the previous works @xcite @xcite @xcite employed independent models for allegation identification and violation classification. concurrent work by @xcite has recently explored multi-task joint models that learn task a and task b simultaneously. their claim-outcome model decomposes them into two independent classifiers with separate encoders for each allegation and violation classification given allegation information. our work differs in three ways: (i) we learn the joint representation for both tasks through a shared encoder following our intuition that the shared representation is beneficial as both tasks require similar features at the lower level; (ii) we model dependency from allegation to violation classification at both the allegation outcome and feature levels. conditioning violation classification on the allegation outcome leads to higher performance, but we find that passing along feature level information yields additional improvement. this way, the violation branch will focus on identifying specific information required for determining violation than falling prey to only allegation level features. (iii) we also model correlations among different articles that tend to be concurrently alleged. inspired by recent advances in contrastive learning to learn effective representations @xcite , we further devise a two-level hierarchical contrastive loss for coc. first, we strive to maximize the latent space distance between different article representations, which assists the model in learning distinct article-specific views of case facts. second, we apply contrastive learning within each article latent space to form distinct sub-clusters of similar outcome cases. unlike our two-level hierarchical contrastive learning, a single-level one has been explored for ljp in the concurrent work by @xcite our experiments demonstrate that, given a static pre-trained encoder, our models outperform singletask and joint models without contrastive loss by a small but consistent margin, with larger improvements for sparse classes in classification performance. ## method our model takes as input the case fact description as token encodings @xmath0, x 2 , . . . , x m } where x @xmath1, x i2 , . . . , x in } and outputs the set of articles claimed to be alleged by the applicant (task b) and set of articles deemed to be violated by the court (task a) as multi-hot vectors. x i and x ip denote i th sentence and p th token of i th sentence of the case facts respectively. m and n denote the number of sentences and tokens in the i th fact sentence, respectively. our model is built on top of a hierarchical attention network @xcite , which we chose to be able to process the very long fact description inputs. our model contains three main components: (1) encoding layer, (2) article-specific case representation and (3) task dependency learning. see fig. 1 for an overview of our model. encoding layer: we encode each sentence in the case facts description with legalbert @xcite to obtain the token level representations {z i1 , z i2 , . . . , z in }. sentence level representations are then computed using a token-level atten-tion mechanism as follows: then we utilize the task b allegation information to enhance task a violation classification to capture the task dependency. we concatenate the obtained article-interaction-aware allegation representations c ′ b and allegation label probability logits o ′ b foot_0 with article-specific case representations c to obtain the enhanced article-specific representation for task a as we again em-ploy multi-head self-attention mechanism on these enhanced article-aware representations c a for violation prediction to capture the correlations that exist between violation of different articles. finally, the obtained representations are passed through k article-specific task a classification layers to obtain the binary outcome o a corresponding to each article's violation output. to be able to evaluate the effectiveness of our interaction architecture in a clean way, we freeze the legalbert encoder weights in all our experiments. ## conclusion we improve ecthr article violation classification from fact statements by leveraging feature and label information from an allegation classification model. we also leveraged hierarchical contrastive loss to contrast between different article representations and case representations based on outcome with respect to a specific article. given a static pretrained encoder, our models outperform a straightforward multi-task architecture by a small but consistent margin, with larger improvements for sparse classes. these results suggest that the tasks of allegation and violation classification on ecthr fact statements interrelate in a way that may not be optimally captured using straightforward multi-task architectures, and motivate further research on dependency modeling between related legal classification tasks.