Joint Learning for Legal Text Retrieval and Textual Entailment: Leveraging the Relationship between Relevancy and Affirmation (2023)

Abstract: In legal text processing and reasoning, one normally performs information retrieval to find relevant documents of an input question, and then performs textual entailment to answer the question. The former is about relevancy whereas the latter is about affirmation (or conclusion). While relevancy and affirmation are two different concepts, there is obviously a connection between them. That is why performing retrieval and textual entailment sequentially and independently may not make the most of this mutually supportive relationship. This paper, therefore, propose a multi–task learning model for these two tasks to improve their performance. Technically, in the COLIEE dataset, we use the information of Task 4 (conclusions) to improve the performance of Task 3 (searching for legal provisions related to the question). Our empirical findings indicate that this supportive relationship truly exists. This important insight sheds light on how leveraging relationship between tasks can significantly enhance the effectiveness of our multi-task learning approach for legal text processing.

## introduction in legal text processing and reasoning, legal document retrieval and textual entailment are two important tasks. given an input legal question, the former helps to narrow down and locates a subset of most relevant documents while the latter attempts to give a yes/no answer to the question by analyzing those relevant documents. legal document retrieval and textual entailment have several challenges, such as intricate legal language structures @xcite , scarcity of annotated legal data @xcite , rich vocabulary with multiple meanings @xcite , and high dependency and the interrelationship between legal statutes @xcite . in this paper, we focus on these two tasks within the context of the competition on legal information extraction/entailment (coliee) @xcite . in coliee, the retrieval task (a.k.a task 3) involves the retrieval of relevant statutes from a database of japanese civil code statutes regarding an input yes/no legal question. the textual entailment task (a.k.a task 4) aims to confirm the entailment of a yes/no answer based on the anlsysis of the retrieved civil code statutes. traditionally, these two tasks have been addressed independently as two separate steps without considering their interdependence. this formulation approach should ignore a mutually supportive relationship between them. in this work, our main idea is that the joint and simultaneous learning and inference of these two tasks can make the most of the mutual relationship between relevancy and affirmation. in some cases, truly relevant statutes that confirm an input question may not relevant in terms of having common vocabulary in traditional information retrieval. our hypothesis is that this joint and "bi-directional" learning will make more accurate decisions in each task. especially, legal entailment information (task 4) can help to improve the performance of the retrieval task (task 3). technically, we propose a multi-task learning framework that integrate and learn task 3 and task 4 simultaneously. this should allow the exploitation of shared features and relevant information that may be ignored in the independent and separate training of these tasks. our method aims to leverage the strengths of the mentioned techniques to deal with the challenges of legal text processing as well as make use of the relationship between relevancy and affirmation to improve the performance of legal text retrieval and entailment tasks. the remaining of the paper is organized as follows. section 2 mentions related work. section 3 addresses the problem. section 4 presents our multi-task learning method. the experiments, results, and anslysis are described in section 5. finally, section 6 draws our conclusions. ## related work various methods have been proposed to tackle legal text retrieval and textual entailment in the coliee competition. researchers have identified bm25 as a useful baseline for statute retrieval, and several studies have focused on combining lexical and neural ranking models @xcite . with the coliee 2020 dataset, @xcite an equivalence between the entailment relation and the relation between the topic sentence and other sentences in a passage. consequently, the research team employed a weaklabeling method to generate additional data and trained a supporting model. using this supporting model as a pre-trained model helped the research team achieve the highest performance in tasks 1 and 2 of coliee 2020 @xcite . a graph-augmented dense statue retriever (g-dsr) which leverages legislation's structural aspects using a graph neural network surpassed robust retrieval benchmarks when evaluated on a real-world sar dataset expertly annotated by professionals @xcite . for improving the retrieval performance in task 1 of coliee 2021, abolghasemi et al. used multi-task learning which combined document-level representation learning and ranking objective @xcite . for legal textual entailment, approaches involve ir-based systems, bert ensemble methods, and transformer-based techniques combined with textual similarity @xcite . yoshioka et al. employed an systematic method for creating training data for syntatic structure understanding @xcite . multitask learning has been employed to overcome data scarcity in the domain, addressing tasks such as translation, summarization, multi-label classification, and legal case retrieval and entailment @xcite . ## conclusions in this study, we proposed a multi-task learning framework for legal document retrieval and textual entailment within the coliee dataset, introducing legal outcome-based retrieval (lor) to enhance traditional legal retrieval. the approach decomposes textual entailment relationships and transforms retrieval into an outcome-based problem, leveraging the relationship between relevancy and affirmation. the experimental results highlight the effectiveness of multi-task learning and reveal potential future research directions, such as expanding to other legal domains, languages, and utilizing advanced deep learning models to improve legal document retrieval and textual entailment tasks.