Everything Has a Cause: Leveraging Causal Inference in Legal Text Analysis (2021)

Abstract: Causal inference is the process of capturing cause-effect relationship among variables. Most existing works focus on dealing with structured data, while mining causal relationship among factors from unstructured data, like text, has been less examined, but is of great importance, especially in the legal domain. In this paper, we propose a novel Graph-based Causal Inference (GCI) framework, which builds causal graphs from fact descriptions without much human involvement and enables causal inference to facilitate legal practitioners to make proper decisions. We evaluate the framework on a challenging similar charge disambiguation task. Experimental results show that GCI can capture the nuance from fact descriptions among multiple confusing charges and provide explainable discrimination, especially in few-shot settings. We also observe that the causal knowledge contained in GCI can be effectively injected into powerful neural networks for better performance and interpretability.

## introduction causal inference is the process of exploring how changes on variable t affect another variable y . here we call t and y as treatment and outcome, respectively, and the changes on t are called intervention. in other words, the process of drawing a conclusion about whether and how y changes when intervening on t is called causal inference. most research in causal inference is devoted to analyzing structured data. take the research question how smoking causes lung cancer @xcite as an example. smoking, lung cancer, together with distractors like age are extracted from structured data, like electronic health records, and considered as factors. usually, such defendant a rented a car from b on april 22, 2015, and forged an agreement to buy the car from b. on the 26th, a lied that he purchased the car, and used it as a guarantee to borrow money from victim c, defrauding c of 19,000 yuan. studies properly organize those factors into humandesigned structures, e.g., a causal directed acyclic graph @xcite with factors {smoking, age, lung cancer} as nodes and causal relations {@xmath0, @xmath1, and perform inference on such structures. recent works attempt to integrate text information into causal inference @xcite @xcite , but they mainly treat the text as a single node in the causal graph, which is relatively coarsegrained. for instance, @xcite investigate how the complaints from consumers affect the company's responses (admission or denial). they regard the entire text of complaint as a treatment, without looking into different aspects of the text, like the events that consumers complained about and the compensation that consumers requested. actually, discovering causal relationship inside unstructured and high-dimensional data, like text, is also beneficial or even crucial for scenarios involving reading comprehensive text and making decisions accordingly. for instance, when a legal ai system assists judges to deal with complicated cases that involve multiple parties and complex events, causal inference could help to figure out the exact distinguishable elements that are crucial for fair and impartial judgements. as shown in figure 1 , if the system can automatically spot two essential key points, 1) the deceitful acts of the defendant, and 2) obtaining properties from the victim, from the unstructured fact descriptions, then the prediction fraud can be more convincing and helpful, rather than a label from a black box. in practice, we would expect a legal ai system to provide human-readable and sound explanations to help the court make the right decisions. it is worthwhile especially for underdeveloped areas, where such techniques could help the judges of rural areas with more trustworthy references from previous judgements. this would further help to maintain the principle of treating like cases alike in many continental law systems. a main challenge in judicial practice is to distinguish between similar charges, we thus propose a new task similar charge disambiguation as a testbed. cases of similar charges often share similar context, and a system is expected to mine the nuance of reasoning process from the text. however, performing causal inference on fact descriptions of criminal cases is not trivial at all. it poses the following challenges. 1) without expert involvement, it is not easy to extract factors that are key to prediction, and organize them in a reasonable form that can both facilitate the inference process and tolerate noise. for example, automatically extracted elements may not cover all the key points in the textual descriptions, and automatically built graphs may contain unreliable edges. 2) it is not easy to benefit from both traditional causal inference models and modern neural architectures. in this paper, we propose a novel graph-based causal inference (gci) framework, which could effectively apply causal inference to legal text analysis. gci first recognizes key factors by extracting keywords from the fact descriptions and clustering similar ones into groups as individual nodes. then we build causal graphs on these nodes with a causal discovery algorithm that can tolerate unobserved variables, reducing the impact of missing elements. we further estimate the causal strength of each edge, to weaken unreliable edges as much as possible, and apply the refined graph to help decision making. experimental results show that our gci framework can induce reasonable causal graphs and capture the nuance from plain text for legal applications, especially with few training data. we also explore the potential of gci by integrating the captured causal knowledge into neu-ral network (nn) models. we propose two approaches: 1) imposing the causal strength constraints to the nn's attention weights; 2) applying recurrent neural networks on the causal chains extracted from our causal graphs. experiments indicate that our methods can successfully inject the extracted causal knowledge and thus enhance the nn models. we also show that integrating gci helps to mitigate the underlying bias in data towards the final prediction. our main contributions are as follows: 1) we propose a novel graph-based causal inference (gci) framework to apply causal inference to unstructured text, without much human involvement. 2) we explore to equip popular neural network models with our gci, by encouraging neural models to learn from causal knowledge derived from gci. 3) we evaluate our methods on a legal text analysis task, similar charge disambiguation, and experimental results show that our gci can capture the nuance from plain fact descriptions, and further help improve neural models with interpretability. ## background two types of questions are typically related to causality. the first is whether there is causal relationship between a set of variables, and the second question is when two variables t and y are causally related, how much would y change if we change the value of t . both of them are discussed in our gci framework, and we first briefly introduce the key concepts. ## graph-based causal inference framework our graph-based causal inference (gci) framework consists of three parts, constructing the causal graph, estimating causal strength on it, and making decisions. figure 2 shows the overall architecture. ## integration of causal analysis and neural networks neural networks (nn) are considered to be good at exploring large volumes of textual data. this motivates us to integrate the causal framework with nn, to benefit each other. here we propose two integration methods as shown in figure 3 . ## analysis 6.1 quality of causal graphs to analyze the robustness of the causal discovery process, we apply sensitivity analysis on the causal graphs. in detail, we make disturbance to the original causal relations, and examine the sensitivity of causal effect towards the violations. following kiciman and sharma (2018), we use three re-futers for examination: 1) random confounder, a new confounder with random value is added to the graph, and ideally, the causal strength should remain the same as before. 2) placebo treatment, the value of a treatment is replaced by a random value, so the treatment becomes a placebo, and the strength should be zero. 3) subset of data, we use a subset of cases to recalculate the strength. ideally, the strength estimation will not vary significantly. we take a sampled causal graph of f&e (fraud & extortion) as an example, and exhibit the refuters on the treatments of charge extortion in figure 4 . causal strength is almost the same as before after random confounder and subset of data refutation; and turns to nearly zero after placebo treatment. the results show that our graph construction method is robust against disturbance. ## discussion although gci is effective on its own and when working with powerful neural network models, there is still room for further improvement. more precise causal inference models. the causal estimation results of gci is based on the constructed causal graphs in former stages, and the automated construction process may bring imprecise factors and even omissions. as clustering algorithms try to summarize the general characteristics of text, descriptions with subtle differences may be clustered into one factor, but the differences matter in legal judgement. for example, in personal injury's graph, different ways of killing are summarized as the factor kill, therefore lose valuable information. specifically, beaten to death might occur in cases of involuntary manslaughter, while shooting cases are more likely to be associated with murder. also, factors with low frequency may be omitted in clustering, but are actually useful for discrimination. overall, under the circumstance without much expert effort, it is worthwhile to explore how to construct a more reliable causal graph. deep understanding on legal documents. although gci to some extent tackles the challenges of incapability of causal inference methods on unstructured text, it may make mistakes when facing complex fact descriptions. negation semantics is a typical example. it is occasional to see negation word usage in fact descriptions, which usually indicates that someone did not have a certain behaviour. however, gci has not considered this aspect, and may be awry in cases containing negation usage. besides, pronoun resolution is also an important aspect that may confuse models. for example, certain behaviour is done by the victim and the subject of the behaviour is a pronoun. if the model was unaware of the subject of the behaviour, it would be counted as criminal's behaviour and introduces noise to later inference stage. moreover, intent could be a decisive factor when discriminating charges, for example, murder and involuntary manslaughter. but it may not be mentioned explicitly in the fact descriptions. it should be better to recover the complete course of fact and recognize the implicit intents between the lines with deep understanding of the context and relevant common sense knowledge. ## conclusions we propose gci, a graph-based causal inference framework to discover causal information in text, and design approaches to integrate causal models and neural networks. in the similar charge disambiguation task, we show our approaches capture important evidence for judgement and nuance among charges. further analysis demonstrates the quality of causal graphs, value of causal chains, and interpretability of computed causal strength.