Automatic Linking of Judgements to UK S upreme C ourt Hearings (2023)

Abstract: One the most important archived legal material in the UK is the Supreme Court published judgements and video recordings of court sittings for the decided cases. The impact of Supreme Court published material extends far beyond the parties involved in any given case as it provides landmark rulings on arguable points of law of the greatest public and constitutional importance. However, the recordings of a case are usually very long which makes it both time and effort consuming for legal professionals to study the critical arguments in the legal deliberations. In this research, we summarise the second part of a combined research-industrial project for building an automated tool designed specifically to link segments in the text judgement to semantically relevant timespans in the videos of the hearings. The tool is employed as a User-Interface (UI) platform that provides a better access to justice by bookmarking the timespans in the videos which contributed to the final judgement of the case. We explain how we employ AI generative technology to retrieve the relevant links and show that the customisation of the GPT text embeddings to our dataset achieves the best accuracy for our automatic linking system.

## introduction in the uk, hm courts & tribunals service (hmcts) publishes both live and recorded videos of court hearing sessions across different jurisdictions. this has been a tradition going on as back as 2008 for different types of courts in the uk. the main objective for this is to improve public access to, and understanding of, the work of the courts. the supreme court sits above all the uk separate jurisdictions and as the final court of appeal, its decisions contribute to the development of united kingdom law. moreover, the court hearings crucially aid in new case preparation, provide guidance for court appeals, help in legal training and even guide future policy. however, there are two main obstacles to making use of this rich material to learn more about the judicial system and have a better access to justice. first, the audio/video material for a case typically spans over several hours on several days, which makes it both time and effort consuming for legal professionals to extract important information relevant to their needs. second, currently, the existing need for legal transcriptions, covering 449k cases p.a in the uk across all court tribunals, is largely met by human transcribers @xcite . this, of course, makes the recorded material, which is rich of legal arguments relevant to how a judgement is reached, difficult to navigate either in text format or in its original audio-visual format. in this research, we present a combined researchindustrial effort to construct an integrated system for the automatic navigation of segments in the media data of uk supreme court hearings based on their semantic relevance to particular paragraph(s) in the text of the judgement issued following the hearing. based on the timing metadata of the court hearing transcription segments, we manage to assign bookmarks on the video sessions and link them to their semantically relevant paragraphs in the judgement text. the main objective of the video bookmarking is to provide legal professionals, as well as the general public an automatic navigation tool that pins down the arguments and legal precedents presented in the long hearing sessions and which are of particular importance to how the judges made their decision on the case. we call our system the judgementto-hearing automatic linking (j-hal) and we deployed it as a user-interface platform that can be used by legal professionals, academics and the public. figure 1 shows a snapshot of the ui we created. on the left side of the screen, the paragraphs of the written judgement are displayed. the user can use the scroll down button to choose a specific paragraph in the judgement. on the right side, the timespan in the court hearing video that is semantically relevant to the legal point mentioned in the selected judgement paragraph is displayed along with temporal metadata (session number, day and time). the user can play the particular timespan and go back and forth around it as well as read our tool's transcription of the speech. major stakeholders from legal domain have shown interest in employing our ui as a bookmarking tool for identifying legally critical minutes in video sessions of supreme court hearings to be used by legal academics and professionals. also, our ui is currently in the application process to be published as an innovative patent by the uk intellectual property office. in this paper, we explain how we establish the automatic linking between judgement paragraphs and video session bookmarks through information retrieval (ir) models. first, in section 2, we briefly summarise relevant work for ir in the legal domain. then, in section 3, we explain our system's pipeline and its objectives. in section 4, we show how we compiled and preprocessed our dataset. then, in section 5, we summarise our experiments on this dataset using a zero-shot ir approach as a prefetching stage and how we customise the gpt text embeddings for optimising our system. finally, in section 6, we present our conclusion on the experiments conducted as well as our future plans for improving the linking system. ## related work recently there has been an increased interest in employing nlp techniques to aid the textual processing in the legal domain @xcite @xcite . the main focus has been on legal document summarisation @xcite , predicting judgements @xcite and contract preprocessing and generation @xcite . moreover, nlp methods for information extraction and textual entailment have been extensively used in the domain of legal nlp to either find an answer to a legal query in legal documents @xcite or to connect textual data @xcite . for example, @xcite experiment with different ir models to extract relevant eu and uk legislative acts that are important for organisations' regulatory compliance which they need to ensure it complies with the relevant laws. their experiments show that fine-tuning a bert model on an in-domain classification task is the best pre-fetcher for their dataset. similarly, @xcite use textual semantic annotation to extract government regulations in different countries which companies and software developers are required to comply with. they show that ai-based ir tools are effective in reducing the human effort to derive requirements from regulations. although there has recently been significant increase in the legal ir research, the processing and deployment of spoken court hearings for legal ir has not received the same attention as understanding and extracting information from textual legal data. in this research, we introduce an industrial product that employs ir tools to automatically connect judgements and videos of court hearings. ## linking judgements and case hearings the pipeline for automatically connecting judgements of decided cases to relevant bookmarks in the court hearing videos consists of two stages. first, we build a customised speech-to-text language model and employ nlp methods to improve the quality of the court hearing video transcriptions. the objective of this stage is to obtain a high quality transcript for our automatic retrieval model in the second stage. the second stage consists of building an ir system capable of extracting the best n-links between a paragraph(s) of the judgement text and the timespans of the transcribed video sessions of that particular case. the links are then translated into timestamp bookmarks in the long videos of each case to be used in constructing our ui. the first stage of the pipeline is beyond the scope of the current paper and is described in @xcite . in the following sections, we explain the methodology we adopt for conducting the second stage in the system pipeline where the automatic linking is established between judgement text and video bookmarks. in the second stage, we treat the linking of a judgement paragraph to the relevant timespan transcripts of a video session as a text retrieval task. in nlp, text retrieval is usually divided into two sub tasks according to the length of the text: query-to-passage retrieval and query-to-document retrieval @xcite . for our use case, it is a document-to-document retrieval task where we first transcribe the video sessions by a custom speech-totext language model that we developed in stage one, and then segment the judgement into paragraphs where a paragraph(s) is treated as a query and the transcript of the case is the corpus in which we search for an answer to that query. it is important to note that the judgement text and the timespans of a transcript usually consist of several lines of text which makes the task more challenging than the traditional query-to-document ir where the query is typically short. another challenge specific of our data is that we attempt to establish a link between texts that belong to two different language registers: written and spoken @xcite . linguistically, the complexity of speech is "choreographic" (halliday, 2007) where meta-linguistic elements such as intonation, loudness or quietness, pausing, stress, pitch range and gestures communicate semantic connotations. moreover, spoken language is characterised by complex sentence structures with low lexical density (fewer high content words per clause), whereas written language typically contains simple sentence structures with high lexical density (more high content words per clause) (halliday, 2007). thus, the retrieval task in our case is nontraditional as it needs a careful preprocessing and segmentation of the spoken and the written datasets to obtain accurate results. in the following sections, we explain how we compiled and preprocessed our training dataset. ## data compilation and processing for training our ir models, we extracted 7 case judgements consisting of 1.4m tokens scraped from the official site of the uk supreme court foot_0 . as for the transcription data, it consisted of 53 hours of video material for the selected cases obtained from the uk national archive foot_1 . the video sessions were transcribed by our custom speech-to-text model which we trained in stage one of j-hal system pipeline. we then ran a number of preprocessing steps to obtain the best linking accuracy between a judgement segment and the relevant timespans in the transcripts. the main challenge in preprocessing the dataset was how to segment the judgement text into semantically cohesive sections that would be treated as queries in our ir method. we noticed that typically the supreme court judgement is structured manually into sections such as: "introduction", "the context", "facts of the case", "the outcome of the case", etc. however, after we carefully scrutinised the dataset, we found that the naming of sections is not consistent. on the other hand, the judgement texts are consistently divided into enumerated paragraphs (typically a digit(s) followed by a dot). we opted, therefore, for segmenting the judgement text into windows of enumerated paragraphs. after experimenting with different window sizes, the optimum window size consisted of three enumerated paragraphs with an average length of 389 tokens per segment. as for the preprocessing of the transcription, it consisted mainly of excluding very short timespans since they were mostly either interjections (e.g. "yes, sorry, i'm not following", "i beg your pardon.", etc.) or reference to logistics of the hearing (e.g. "this is your paper, isn't it?", "please turn to the next page. ", etc.). we chose to exclude transcription spans less than 50 tokens as an empirical threshold for semantically significant conversation units. for both the judgement and transcript data, we cleaned empty lines and extra spaces but kept punctuation intact as it is essential in identifying names of cases and legal provisions foot_2 . ## zero-shot information retrieval the ability of an ir system to retrieve the top-n most relevant results is usually assessed by comparing its performance with human-generated similarity labels on a sentence-to-sentence or queryto-document similarity dataset(s) (e.g. @xcite @xcite . in order to create a human-generated evaluation dataset, we needed to assign human annotators to manually check the correct links between judgement segments and the timespans of video hearing transcripts for each of our chosen cases. however, in our use case, this is not feasible since to annotate one supreme court case with, for example, 50 judgement segments and 300 timespans of video transcript, the annotators will need to read 50 x 300 judgement-timespan link, which amounts to 15,000 doc-to-doc link per case. to overcome this problem, we adopted a zero-shot ir approach. thus to create a dataset for annotation, we first experimented with different ways to encode the judgement segments and transcription timespans as numeric vectors for a single case in our dataset. we used the cosine similarity as our semantic distance metric to extract the top closest 20 transcript timespans per judgement segment in vector space. then, we assigned a human annotator, postgraduate law student, to evaluate the first 20 links produced by the different models. the annotator compared each judgement segment against each timespan to choose either 'yes' there is a semantic link or 'no' there is not. the ir models used for our experiments are the following: @xcite : bm25 is a traditional keyword search based on a bag-ofwords scoring function estimating the relevance of a document d to a query q, based on the query terms appearing in d. it is a modified version of the tf-idf function where the ranking scores change based on the length of the document d in words, and the average d length in the corpus from which documents are drawn. ## conclusion this research presented the second stage of our pipeline which employs generative ai to automatically link a judgement text of a decided case in the uk supreme court to its video hearings. the ir system we provide assists users in extracting the arguments and information they may find useful in understanding the particular case they are studying. the system does not, however, explicitly return answers to questions legal professionals may have on a legal precedent. the ui we provide supports the users in browsing or filtering the lengthy videos of the court hearing sessions by searching through hundreds of video timespans and then provides a set of need-to-watch bookmarks that are crucial in understanding the judgement decided for the case. the implications of this tool extends beyond providing practical aid for legal professionals and academics to expanding the public's ability to access information on court proceedings and justice in general. in the future stages of the project, we aim to expand our annotated linking dataset and explore the effectiveness of coupling judgements and video hearings according to common legal entities such as articles, legal provisions and names of similar cases. we also aim to adopt a similar methodology for constructing similar ui tools for other purpose domains such as bookmarking recorded lecture spans to educational text books or linking written documents to recorded meetings in the business sector.