The Law and NLP : Bridging Disciplinary Disconnects (2023)

Abstract: Legal practice is intrinsically rooted in the fabric of language, yet legal practitioners and scholars have been slow to adopt tools from natural language processing (NLP). At the same time, the legal system is experiencing an access to justice crisis, which could be partially alleviated with NLP. In this position paper, we argue that the slow uptake of NLP in legal practice is exacerbated by a disconnect between the needs of the legal community and the focus of NLP researchers. In a review of recent trends in the legal NLP literature, we find limited overlap between the legal NLP community and legal academia. Our interpretation is that some of the most popular legal NLP tasks fail to address the needs of legal practitioners. We discuss examples of legal NLP tasks that promise to bridge disciplinary disconnects and highlight interesting areas for legal NLP research that remain underexplored.

## introduction rapid advances in nlp technology are already promising to transform society and the economy (see e.g., @xcite @xcite , not least by their impact on many professions. given that legal practice is embedded in written language, it stands to gain immensely from the application of nlp techniques. this promise has attracted research on nlp applications related to a wide array of legal tasks including legal research @xcite , legal reasoning @xcite , contract review @xcite , statutory interpretation @xcite , document review @xcite , and legal question answering @xcite @xcite . the current model of legal services is failing to address legal needs in several important con-texts. in the united states, around 92% of civil legal problems experienced by low-income americans receive no or inadequate legal help @xcite . in u.s. criminal cases, where criminal defendants generally have a right to legal counsel, public defenders are systematically overworked and under-resourced @xcite . access to legal services is also limited for u.s. small businesses @xcite . the combination of unequal access to and high costs of legal services results in a troubling access to justice issue. internationally, there is tremendous variation in legal systems and challenges, but many opportunities where technology could help address legal inefficiencies and under-served communities have been identified globally (see e.g., @xcite @xcite . against this backdrop, numerous scholars have focused on legal technology as a path towards reducing the access to justice gap by extending the abilities of legal professionals and helping non-lawyers navigate legal processes @xcite @xcite @xcite . of course, technology is no panacea and many of the inequities and inefficiencies in jurisprudence are related to deeply rooted social and cultural phenomena. nonetheless, we view the responsible deployment of legal technology as a crucial step towards improving the legal profession and broadening access to legal services. despite their potential to improve the provision of legal services, the legal industry has been slow to adopt computational techniques. the majority of legal services continue to rely heavily on manual work performed by highly trained human lawyers. this slow adoption may be partially attributed to risk aversion, misaligned incentives, and a lack of expertise within the legal community @xcite . we argue that there is another factor to blame, rooted not in legal practice but rather in legal nlp research: in short, legal nlp is failing to develop many applications that would be useful for lawyers. instead, legal nlp research tends to focus on generic nlp tasks and applies widely-used nlp methodologies to legal data, rather than developing new nlp tools and approaches that solve problems unique to the legal context. for example, nlp research might apply text classification to predict the direction of a u.s. supreme court judgment based on portions of the judicial opinion. these types of models tend to be of limited practical utility: first, the vast majority of lawyers and legal disputes will never reach the supreme court. second, the legal reasoning applied by the supreme court is unlikely to be representative of lower courts. and lastly, classifiers trained on published judgments may emulate judicial idiosyncrasies rather than modeling optimal legal reasoning. meanwhile, there has been less research on systems to help lawyers identify relevant precedent for trial, on exploring automated summarization and generation of legal documents, or leveraging nlp for online dispute resolution. the work done by @xcite ; @xcite ; rhode (2013) and others takes an important step toward bridging disciplinary disconnects by providing overviews of nlp and related methods to legal and multidisciplinary communities. we hope to build on this work by encouraging the legal nlp community to understand the needs of legal practitioners. our paper offers some initial starting points for nlp research that are informed by practical needs. we base our argument on a review of recent legal nlp research, which identifies key themes in this literature (see figure 1 ). we find that a large portion of this research focuses on tasks which we believe are disconnected from the needs of legal practitioners. we further observe that only a small fraction of citations to legal nlp publications stem from legal publications, providing evidence that nlp publications have not managed to rise to the attention of the legal community (see left panel of figure 2 ). grounded in this review, we segment legal nlp tasks into three categories: applications that could aid the provision of legal services; widespread nlp applications that have limited impact on practical legal issues; and areas of legal nlp research that could have significant impact on legal practice but which remain underexplored. our work adds to a growing number of recent position papers discussing the intersection of law and nlp (see e.g., @xcite @xcite @xcite . the number of survey papers in this domain might suggest some confusion about the state of legal nlp. in addition to offering descriptive findings, we aim to provide a normative argument. in light of the access to justice issues highlighted above, we encourage legal nlp researchers to pragmatically focus on work that promises to broaden access to justice. this objective helps advance a shared normative goal that does not neglect the 'legal' dimension of legal nlp. to summarize, we make the following contributions and recommendations. ## literature review we conduct a rapid literature review via forward citation chasing over 171 papers, following recommendations made by @xcite . literature reviews have been employed in several position papers investigating the intersection of nlp and other disciplines (see e.g., @xcite . the starting point of our rapid review is zhong et al. (2020), an overview paper about how legal nlp benefits the legal system. we selected this work as our starting point because it is a recent contribution in a top nlp conference that has received a reasonably high number of citations for us to review. moreover, it provides (to the best of our knowledge) a fairly accurate overview about the current state of legal nlp. our review includes all papers citing zhong et al. (2020) on google scholar. zhong et al. ( 2020 ) aim to identify popular legal nlp tasks and discuss these tasks from the perspectives of legal professionals. tasks outlined in the paper include legal judgment prediction, question answering, and similar case matching. we read and manually annotate all papers citing @xcite , resulting in 171 annotated papers. 1 for each paper, we determine: we follow a hybrid inductive-deductive content analysis methodology, employed in @xcite , and aim to follow best practices and recommendations from the qualitative content analysis literature (see e.g., @xcite @xcite . we start with a set of ten objectives described in @xcite , such as relation extraction, 1 at the time of writing, the paper had 182 citations in total. we discarded nine citations for which google scholar did not provide a link. legal judgment prediction and question answering. we list all of these in appendix a. if a paper describes an objective that cannot be classified into one of these categories, we define a new category. after every 20 annotations, we review all categories assigned so far and decide whether to merge existing categories (e.g., named entity recognition and semantic role labeling merge into information extraction, while similar case matching and semantic search merge into information retrieval) -this represents the deductive step in our approach. after annotating all 171 papers, we review all categories again and perform a final merge. finally, for each paper, we determine whether it is a legal, nlp, or interdisciplinary publication. this categorization is based on the publication venue and the author's affiliations. we display our main findings in figure 2 . we find that despite the inherently interdisciplinary nature of legal nlp, most work we reviewed is produced and consumed by the nlp community. only 10% of the papers citing @xcite are primarily interdisciplinary. perhaps more strikingly, only a single law review article cites @xcite . in terms of paper objectives, we find that legal judgment prediction appears to be the most popular objective, with 20% of reviewed papers focusing on this task. ## categorizing legal nlp applications based on our review, we categorize popular legal nlp objectives in terms of their ability to impact the practice of law or broaden access to justice. subsequently, we identify potentially impactful areas of research that remain underexplored. ## discussion the disconnect between ai research on applications and specific disciplines is not limited to law (see e.g. @xcite . law, however, is unique among disciplines in that it is a field built on language. given the current state of legal practice, there is a need for innovation to make legal services more affordable and to address the access to justice crisis. as such, law presents a valuable opportunity for the nlp community to conduct research on applications that could aid legal practitioners and that expand access to justice. impactful legal nlp research must be grounded in the needs of the legal community. the observed lack of cross-disciplinary citations in our review suggests that legal nlp researchers are largely disconnected from the legal community. we encourage legal nlp researchers to identify tasks that are performed frequently by legal practitioners and that lend themselves to the application of nlp techniques. to aid nlp researchers in identifying these tasks, we urge them to consider closer interdisciplinary collaborations with the legal community or at least to address legal issues identified in the legal literature. ## conclusion by leveraging a literature review, we find that the legal nlp community is largely disconnected from legal academia. we emphasize that certain popular legal nlp tasks are only of limited utility to legal practitioners. we thus urge legal nlp researchers to focus on access to justice as a shared normative objective, to ground their work in the needs of the legal community, and to consider collaborating with lawyers to ensure that their research has applications in practice. nlp has the potential to positively transform the practice of law and by extension society. however, this is impossible without cross-disciplinary understanding and collaboration.