What About the Precedent: An Information-Theoretic Analysis of Common Law (2021)

Abstract: In common law, the outcome of a new case is determined mostly by precedent cases, rather than by existing statutes. However, how exactly does the precedent influence the outcome of a new case? Answering this question is crucial for guaranteeing fair and consistent judicial decision-making. We are the first to approach this question computationally by comparing two longstanding jurisprudential views; Halsbury’s, who believes that the arguments of the precedent are the main determinant of the outcome, and Goodhart’s, who believes that what matters most is the precedent’s facts. We base our study on the corpus of legal cases from the European Court of Human Rights (ECtHR), which allows us to access not only the case itself, but also cases cited in the judges’ arguments (i.e. the precedent cases). Taking an information-theoretic view, and modelling the question as a case out-come classification task, we find that the precedent’s arguments share 0.38 nats of information with the case’s outcome, whereas precedent’s facts only share 0.18 nats of information (i.e.,58% less); suggesting Halsbury’s view may be more accurate in this specific court. We found however in a qualitative analysis that there are specific statues where Goodhart’s view dominates, and present some evidence these are the ones where the legal concept at hand is less straightforward.

## introduction legal systems around the world can be divided into two major categories @xcite : civil law systems, which rely predominantly on the rules written down in statutes, and common law systems, which rely predominantly on past judicial decisions, known as the precedent. within common law systems, jurisprudential scholars have pondered over the nature of precedent in law for at least a century @xcite . is it the judges' argumentation in the precedent, or is it the claimants' specific individual circumstances that are the deciding factor in what becomes the law? here, we present a new information-theoretical methodology that helps answer this question. in common law countries, statutes establish the general idea of the law, but the actual scope of the law is determined by the courts during a trial. to keep case outcomes consistent and predictable in subsequent cases, judges are forced to apply the reasoning developed in prior cases with similar facts (precedent), to the facts of the new case under the doctrine of stare decisis @xcite @xcite . this is done by identifying the ratio decidendi (the reasons for the decision) as opposed to the obiter dicta (that which is said in passing). the distinction between ratio and obiter is an important one, since ratio is binding, whereas obiter is not. this means that courts will only strive to remain consistent in upholding ratio, but can freely depart from the obiter. but what does the ratio consist of? there is no accepted overarching theory of precedent @xcite , but there are two tests of ratio. on the one hand, lord @xcite claims that what is binding is the judge's reasoning and arguments. for instance, by using a high degree of abstraction, judges can analogise physical and psychological pain. a different view has been put forward by @xcite , who argues it is the analogy of the facts of the precedent and the case at hand, without the need for reasoning (e.g. comparing the pain caused by a knife to that caused by another instrument, requiring a far lower degree of abstraction). these give rise to the two wellknown legal tests for ratio: halsbury's test and goodhart's test. in this paper, we are the first to approach this problem from a data-driven perspective, using the european court of human rights (ecthr) 1 case law; see figure 1 . we build a citation network over this corpus in order to have access to many precedents' full text. training our model on either the facts or the arguments of the precedent, we can put halsbury's and goodhart's views to the test. we cast this problem as an information-theoretic study by measuring the mutual information @xcite between the case outcome and either the precedent facts or arguments. we find that precedent arguments and case outcome share information to the degree of 0.38 nats, whereas facts and case outcome only share information to the degree of 0.18 nats (i.e., 58% less). we therefore observe that-at least for ecthr-halsbury's view of the precedent is more accurate than that of goodhart. ## legal background despite the importance of the precedent in civil law, its operationalization remains shrouded in philosophical debate centred around how the precedent actually forms the binding law. jurisprudentially, we can think of this as searching for the ratio decidendi in the judgement, i.e. separating the ratio decidendi from the obiter dicta, or binding law from merely circumstantial statements. it is the nature of ratio that distinguishes halsbury's view from goodhart's. ## an information-theoretic approach notation. we denote the set of cases as c, writing each of its element as c. the set of cases that form the precedent for case c are denoted p c ⊂ c. we will consider three main random variables in this work. first, we consider o, a random variable that ranges over a binary outcome space @xmath0, 1} k , where k is the number of articles. an instance @xmath1. since o is a vector of binary outcomes for all articles, we can index it as o k to get the outcome of a specific k th article and we analogously index the random variable o k . we will denote o c the outcome of a specific case c. foot_1 next, we consider f , a random variable that ranges over the space of facts. we denote the space of all facts as @xmath2, where σ is a set of sub-word units and σ * is its kleene closure. we denote an instance of f as f . we will further denote the facts of a specific case c as f c . finally, we consider a, a random variable that ranges over the space of arguments. analogously to facts, the space of all arguments is @xmath3. an element of a is denoted as a, which we again term a c when referring to a specific case. ## experimental setup we choose to work with the ecthr corpus for three reasons. first, it can be treated as operating under precedential law, in the vein of common law countries. this is not a given, as the ecthr is an international court of highest appeal without a formal doctrine of stare decisis @xcite , but there is nevertheless strong evidence that it is precedential. this evidence comes from the court's own guidelines @xcite , but can also be found in the writings of a former judge of the ecthr @xcite and of legal scholars @xcite . second, there is existing research on the neural modeling of ecthr case law we can build upon @xcite @xcite . third, the documents of the ecthr 3 nats are computed with ln, while bits use log 2 . for our experiment, we need a sub-corpus where each case has at least one outgoing citation where the full text is contained in our corpus. in practice, there will be other outgoing citations we cannot resolve, for instance because the document is not in english or hudoc happens not to contain them. we also need our citations to be de-duplicated. we create such a sub-corpus, which contains 9,585 documents (i.e., citing documents), with 94,167 in-corpus links (tokens) to 7,113 cases (types) and 65,495 out-of-corpus links to 22,328 types (cited documents). we start from the original ecthr split of 9,000 training, 1,000 validation and 1,000 test cases, and after citation filtering arrive at 7,627 training, 976 validation and 982 test cases. for every citation, we extract the text under headings with regular expressions such as "the facts" and "the law", labelling it as facts and arguments, respectively. ## results our experimental results are contained in we first note that both our mutual information estimates are statistically larger than zero, i.e. goodhart's and halsbury's cross-entropies are statistically smaller than that of the facts. 7 the question we asked ourselves at the outset, though, concerns whether the data supports halsbury's or goodhart's view. we find that our estimate of mi(o; h | f ) is significantly larger at 0.31 nats than our estimate of mi(o; g | f ) at 0.18 nats. these results suggest that the information contributed by the precedent arguments give us nearly 58% more information about the outcome of the case than the information contained in the facts of the precedent. in terms of the uncertainty coefficient, the outcome entropy is reduced by 6% for facts and by 10% for arguments. we therefore observe that halsbury's view is more widely used in the domain of ecthr than goodhart's. ## discussion & analysis a more nuanced story can be told if we inspect the individual articles even though the small number of cases per article does not allow for conclusive significance tests. the core rights of the convention are contained in (articles 2-18). 8 figure 3 shows that for some of the core articles, we see the opposite effect from what we observed 7 we measure significance using the two tailed paired permutation tests with @xmath4.05 after @xcite correction. 8 the convention section 1 is the first section of the echr and elevates some of the universal declaration of human rights principles into actionable rights of european citizens @xcite . we hypothesise that the reason for this is either that the judges have not yet developed a functional legal method for these articles, that the relevant precedent has been placed late in the list of precedents (and thus was truncated away by our methodology), or that the complexity of the arguments requires a reasoning ability our models are simply not capable of. we consider each hypothesis separately below. ## related work in this section, we contextualise our work with relation to the related research on legal ai. computational approaches to solving legal problems go back at least as far as the late 1950's @xcite . early research has focused on crafting rule-based systems for case outcome prediction, achieving human-like performance by the early 2000's @xcite . these systems however proved too brittle to keep up with the everchanging legal landscape and never transitioned from research into industry. more recently, a new wave of deep learning methods has reinvigorated the research interest in legal ai. the majority of this new work has been conducted on statutory legal systems which do not rely on the doctrine of precedent to nearly the same extent as their common law counterparts. for instance, in chinese law the use of neural models for case outcome classification has already been investigated extensively @xcite @xcite . in the precedential legal domain, smaller corpora of annotated cases have been investigated over the years @xcite . however, large-scale corpora necessary for deep learning architectures have become available only recently. the caselaw access project foot_6 introduced a large dataset of american case law in 2018. @xcite have introduced the ecthr corpus, and @xcite have run deep neural networks on it in order to predict outcome. similarly, the canadian supreme court case corpus has been used in infor-mation retrieval for the first time by @xcite . this improved access to a high quality common law datasets has opened up a potential for new work in the field of legal ai. particularly similar to our work is the study done by @xcite , who have considered the influence of petitioners and responders (amicus) briefs on the us supreme court decision and opinions. ## conclusion in this paper, we have shifted the focus of legal ai research from practical tasks such as precedent retrieval or outcome prediction, to a theoretical question: which aspect of the precedent is most important in forming the law? to this end, we trained a similar neural modeling approach as @xcite to predict the outcome of a case on the ecthr dataset, and inspected the difference in the mutual information between our operationalisations of halsbury's and goodhart's view. we have used a method inspired by @xcite to approximate the mi. we observe that out of the two archetypal views on precedent, that of halsbury and goodhart, the former has a better empirical support in the domain of ecthr case law. this study has demonstrated a novel method of approaching jurisprudential questions using the information-theoretic toolkit. we hope that future work can leverage our methodology towards answering other questions of legal philosophy. however, our results are not only of an interest in the context of legal theory, but they can also inform a development of better legal models in practice. since most precedential reasoning is conducted using the arguments in the precedent, outcome prediction models should take advantage of the case arguments, instead of relying solely on the facts.