The Ethics of Automating Legal Actors (2024)

Abstract: The introduction of large public legal datasets has brought about a renaissance in legal NLP. Many of these datasets are composed of legal judgments—the product of judges deciding cases. Since ML algorithms learn to model the data they are trained on, several legal NLP models are models of judges. While some have argued for the automation of judges, in this position piece, we argue that automating the role of the judge raises difficult ethical challenges, in particular for common law legal systems. Our argument follows from the social role of the judge in actively shaping the law, rather than merely applying it. Since current NLP models are too far away from having the facilities necessary for this task, they should not be used to automate judges. Furthermore, even in the case that the models could achieve human-level capabilities, there would still be remaining ethical concerns inherent in the automation of the legal process.

## introduction this paper discusses the ethical aspects of using natural language processing (nlp) research to augment or replace the work of legal experts, with particular emphasis on common law legal systems. although we agree that there are potential benefits to the practical application of nlp to the legal domain, these applications face several ethical challenges. some of these challenges are resolvable with technical advances; others, however, appear to be intrinsic to using any kind of automation. we consider the two main legal actors, the judge and the lawyer, and find that while automation of either can be beneficial, lawyer automation presents fewer challenges. however, current work in legal nlp and the legal domain are motivated by modeling judges directly: many current proposals for automation in the legal domain concentrate on the judiciary, and similarly, a number of popular legal datasets represent text produced by judges. we begin by giving a brief legal background in §2, where we explore the role of judges and lawyers in common law legal systems and their distinct functions. we then introduce the ideas of the rule of law and substantive justice-the pillars on which the judicial system is built and which connect law and morality. in §3, we explore practical automation proposals discussed in legal nlp research. these proposals can be broken into two groups: the first of these advocate a complete replacement of legal professionals with technology, whereas the second group advocates a mere augmentation of judges or lawyers by partial automation or by supplementation of legal tasks with nlp. concluding our background sections, in §4, we turn to legal nlp and discuss the impact of the shift from symbolic to sub-symbolic ai on the field. we then proceed in three stages. in the first stage, §5, we outline the risks and benefits of legal nlp. we suggest that current proposals for implementing legal nlp face three technical challenges, namely, (1) the lack of contextual and social intuition at the trial stage; (2) the inability to make controversial moral and political decisions to develop the law; and, (3) the inability to justify the decisions to the public. further, even if these technical challenges are resolved, existing proposals pose unavoidable ethical risks. they have the potential to (1) centralize power; in the second stage ( §6), we turn to the main legal actors, the judge and the lawyer, to evaluate their exposure to the risks and benefits. we show that the various concerns affect lawyers and judges differently, and we come to the conclusion that the more promising domain for legal nlp is the role of the lawyer. in a nutshell, lawyers are not burdened with the task of being lawmakers. instead, the success of their work can be measured more easily, namely, whether or not they can convince judges and other lawyers of the quality of their arguments. in the third stage ( §7), we turn to the role of the judge in contemporary legal nlp research. many existing legal nlp datasets comprise collections of cases, in other words, text produced by judges. we demonstrate how a lack of data about lawyers and their arguments can undermine the modeling task based on these datasets. to mitigate this issue, we recommend that legal nlp researchers should focus more on the role of the lawyer in the legal system. in the conclusion ( §8), we contextualize our work in the wider nlp ethics discussion. our proposals do not call for a wholesale reorganization of the field. if we want to further our understanding of law and language, judges remain an important subject to study. some datasets already implicitly contain the voice of a lawyer, in the form of legal claims, although these need to be disentangled from the voice of the judge. for example, @xcite and @xcite predict both lawyers' claims and judicial outcomes, using the same legal dataset, an early step in the right direction, and a demonstration that the shift to the lawyer's perspective can sometimes be achieved by a simple re-purposing of existing datasets. we therefore believe that focusing on the lawyer and their interactions with the judge can supplement existing approaches, and inspire new, more robust legal nlp research. we focus on the common law system for two reasons. first, common law has a longstanding philosophical debate surrounding it. second, practical proposals for nlp have mostly focused on common law @xcite , and many of the datasets discussed in §4 and §7 involve common law material. however, we acknowledge there is much work in legal nlp applied to civil law countries. chinese law, in particular, has been studied extensively through the lens of nlp tools over the years @xcite @xcite . ## automation from a policy viewpoint a number of proposals exist on the future role of legal nlp @xcite @xcite @xcite . we taxonomize these proposals into two groups: (1) those that advocate for the replacement of legal actors with technology, and (2) those that advocate for mere augmentation. the former, more radical view suggests nlp can be used to replace all legal tasks. the latter, on the other hand, proposes legal nlp can only supplement the work of traditional human lawyers and judges, without completely replacing them. ## legal nlp legal nlp can trace its origins all the way back to the late 1950 @xcite @xcite . 5 hypo @xcite , one of the earliest symbolic ai systems, explicitly encoded the principles of case-based reasoning in terms of analogy and difference, often based on hand-extracted features. many others were inspired by it (aleven and @xcite @xcite . aleven ( 2003 ) originally introduced the task of predicting an outcome of a case, which has nowadays become the modern benchmark of legal nlp. for a long time, issue based prediction @xcite , one of the successors of the hypo model, held the state of the art for this task. however, the wider adaption of the symbolic systems was hindered by their reliance on humans for both processing the input and encoding the changing rules of law. this became a problem for the deployment of these models since the law can change constantly with every new court decision. recently, the popularity of machine learning, combined with a desire for more robust models of law, has rejuvenated interest in developing applications for the legal domain. this shift from symbolic to sub-symbolic legal ai has implications for contemporary legal nlp research. the aim is now to approximate legal reasoning from legal data, rather than encoding it by hand. since the architectures powering many sota approaches in legal nlp are fairly homogeneous-they are all pre-trained transformer-based llms-the deciding factor is the choice of data the ml models are fine-tuned on and the number of parameters. legal ml tasks include question answering @xcite , legal entity recognition @xcite ), text summarization @xcite , outcome prediction @xcite @xcite , majority opinion prediction @xcite , legal topic classification @xcite , court opinion generation @xcite , case citation resolution @xcite , study of legal precedent @xcite , or applications in legal consulting @xcite . for a comprehensive overview of legal nlp, see @xcite . yet another strain of work has emerged where the goal is to model individual judges using sociodemographic and similar features, instead of legal text. an example for this line of work is that of @xcite , who predict the outcome of the us supreme court case law. the features used in this work are meta-data of the court, such as date, court position in the court hierarchy, judge names and lower court outcome; with these, a simple nearest neighbour classifier achieved 70% accuracy. ## ethical dimensions of legal nlp we believe there are both benefits and risks in the use of nlp in the legal domain. the benefits are large enough that refusing to use nlp could present a moral failure akin to not using new medical treatments to treat patients. on the other hand, the ethical risks of nlp are also significant and need to be weighed up against these potential benefits. ## related work there is a growing field of research on nlp data privacy @xcite . concerns over privacy naturally arise from training nlp models on sensitive data, such as medical records @xcite . these concerns are relevant to legal nlp in particular, since the content of legal documents is often confidential and might additionally contain medical details. since training data can be reconstructed from large language models @xcite , which underpin much of the current research in nlp and legal nlp, data privacy is a pressing issue. one approach for privacy-preserving techniques for legal nlp is the differential privacy approach, a cryptography-based approach using transformer language models developed by @xcite . going beyond privacy concerns, machine learning approaches to nlp open the systems up to ethical questions by training on human-generated data @xcite . data from the internet, and from social media in particular, may contain biases which, if left unchecked, could adversely affect users on nlp systems trained on such polluted data. as far as legal nlp is concerned, llms harbor biases that can affect legal nlp models built on top of them. the legal data itself also contains biases which we would not want to be replicated in our models @xcite . some researchers have conducted studies on uncovering and quantifying different types of bias @xcite @xcite @xcite . others have started to develop tools to de-bias the models @xcite @xcite . with the scale of datasets growing over time, these issues are likely to grow @xcite . finally, while the law can be viewed as a codification of moral reasoning to some extent, very little has been written about the ethics of automating the legal system. there has been an ongoing critique of nlp research aimed at automating legal sentencing and even calls not to publish this kind of research @xcite . in response to this criticism, others have warned against the threat of moralism impinging on academic freedoms @xcite . in contrast to the previous work, we focus on the roles of the legal actors and the ethical implications of their automation. ## conclusion in this work, we have evaluated the ethical and practical feasibility of different proposals for nlp research in law. by surveying a number of popular legal nlp tasks, we identify that the datasets they are built on favour the voice of the judge. however, the criteria for what makes a good judge include moral discretion and political accountability, and we therefore believe that practical applications of these kinds of nlp models face difficult challenges. for a machine judge to be successful, it would need to have capabilities far exceeding what is currently available. these capabilities include moral and social intuition; the sophisticated ability to give explanations; and, in turn, the ability to receive feedback from members of the public. even if these technical challenges were met, there are further inherent risks to using legal nlp in the real world. we have discussed three of these: the centralization of power, brittleness and lack of accountability. comparing the judge and a lawyer, we find the role of a lawyer less susceptible to these challenges. our appeal is, therefore, to focus on the voice of the lawyer in future legal nlp datasets. we believe that overlooking the role of the lawyer hinders current neural approaches for modeling law. for the academic pursuit of furthering legal nlp research, too much focus on the text produced by judges hides the fruitful and interesting interplay between the lawyers that fuels the legal discourse.