Learning Causal B ayesian Networks from Text (2020)

Abstract: Causal relationships form the basis for reasoning and decision-making in Artificial Intelligence systems. To exploit the large volume of textual data available today, the automatic discovery of causal relationships from text has emerged as a significant challenge in recent years. Existing approaches in this realm are limited to the extraction of low-level relations among individual events. To overcome the limitations of the existing approaches, in this paper, we propose a method for automatic inference of causal relationships from human written language at conceptual level. To this end, we leverage the characteristics of hierarchy of concepts and linguistic variables created from text, and represent the extracted causal relationships in the form of a Causal Bayesian Network. Our experiments demonstrate superiority of our approach over the existing approaches in inferring complex causal reasoning from the text.

## introduction causation is a powerful psychological tool for human to choreograph his surrounding environment into a mental model, and use it for reasoning and decision-making. however, inability to identify causality is known to be one of the drawbacks of current artificial intelligence (ai) systems @xcite . extraction of causal relations from text is necessity in many nlp tasks such as question answering and textual inference, and has attracted a considerable research in recent years @xcite @xcite @xcite . however, the state-of-the-art methods are limited to the identification of causal relations between low-level individual events @xcite @xcite and fail to capture such relationships at conceptual level. furthermore, relying on linguistic features limits the identification of causal relations to those whose cause and effect are located in the same sentence or in consecutive sentences. in this paper, we propose a method for extracting concepts and their underlying causal relations from written language. furthermore, to leverage the extracted causal information, we represent the causal knowledge in the form of a causal bayesian network (cbn). having this tool enables answering complex causal and counter-factual questions, such as: how psychotherapy can affect the patient's emotion?, or what would happen if instead of medicine x, medicine y was prescribed? the contribution of this paper is three-fold. firstly, we focus on identifying causal relation between concepts (e.g. physical activity and health). secondly, we propose a novel method to represent the extracted causal knowledge in the form of a causal bayesian network, enabling easy incorporation of this invaluable knowledge into downstream nlp tasks. thirdly, we release psycaus dataset which can be used to evaluate causal relation extraction models in the domain of psychology foot_0 . in addition, our proposed method identifies causality between concepts independent of their locations in text, and is able to identify bi-directional causal relations between concepts, where two concepts have causal effect on each other. by aggregating linguistic variable, we construct a hierarchy where each variable, e.g. delusional disorder, lies under its related concept, e.g. disorder. this hierarchical and inheritance structure allows for the inference of causal relations between concepts that are not directly discussed in the text. in order to evaluate our proposed method, we gathered a corpus of psychological articles the experimental results shows that the proposed method performs significantly better than the state-of-the-art methods. ## related works identification of causality in nlp is not trivial as a result of language ambiguity. hence, most current approaches focus on verb-verb, verb-noun, and noun-noun relations. the explicit relations are often captured with narrow syntactic and semantic constructions @xcite @xcite which limits their recall. to go beyond surface form constructions few works have proposed neural models @xcite @xcite covering wider causal constructions. however, most works don't go beyond extracting causality between adjacent events, and so lack the ability to capture causality in non-adjacent concept level, e.g. genetics and hallucination. therefore, in this paper we propose a model for identifying causality between concepts, independent of their location, and represent the causal knowledge in form of a causal bayesian network. ## methodology given the input, in form of human written language, we aim to extract the causal relation between concepts and represent the output in form of a causal bayesian network. hence, we split this task into three sub-tasks: extracting linguistic variables and values, identifying causality between extracted variables, and creating conditional probability table for each variable. in the following sub-sections each of these sub-tasks are explained. ## experimental results in this section, we evaluate the effectiveness of our method and compare it with that of the state-ofthe-art methods on a collection of wikipedia articles related to psychology. each article within this collection is selected based on the terms in apa dictionary of psychology @xcite . this collection contains a total number of 3,763,859 sentences. among all possible relation between concepts, we studied 300 relationships between concepts, which were annotated by 5 graduate student from school of psychology. each tuple of relationship in form of (a, b), were labelled as -1, 0, or 1, where 1 indicates a cause ---→ b, -1 shows that b cause ---→ a, and 0 implies no causal relations. with the overlap tuples (25%) we measured the degree of agreement between annotators using fleiss' kappa measure @xcite , which was around 0.67. this indicates the reliability of our test setup. we compare our model to the feature-based and distribution-based methods proposed by @xcite , with different proxy projection functions, including { w2vii, w2vio, w2voi, counts, prec-counts, pmi, prec-pmi}. furthermore, we compare our model to heuristic models, figure 1 shows the accuracy of different methods for identifying causal relationships. we observe that our method (the red bar) outperforms all other approaches with an accuracy of 0.694. this indicates an improvement of 13% over the stateof-the-art feature-based methods (the blue bars), 17% over the distribution-based approaches (the orange bars), and 20% over the baseline methods (green bars). the baseline methods represented the worst performance, however, the accuracy achieved by precedence suggests that most of our corpus is written in form of active rather than passive voice, resulting in consequential connection between concepts. to analyse the sensitivity of our method to the threshold µ in equation 1, we trained the model on psycaus's training set(d tr ), and analysed the development set performance in terms of macroaveraged f1 with a range of values [0, 1) for µ. as shown in figure 2 , f1 score reaches the maximum of 0.66 with µ = 0.05, well above a random classifier. during the annotation process, we noticed that some concepts, e.g. eating disorder and emotion, may have bi-directional causal relations, depending on the context (eating disorder cause ←-→ emotion). we ran our model against these examples and found out that our approach is interestingly capable of identifying these relations as well. in equation 1, approximation of absolute values of both operands in the negation to one indicates bidirectional causality. while a bi-directional causal relation cannot be presented in a cbn, as it is a directed graphical model, a decision tree can contain these types of information. in addition, some concepts, e.g. delusional disorder and displeasure, that have not been connected with any type of causality connectives were also accurately identified as causal relations. this is due to the hierarchical design of variable-values in our model. ## conclusion in this paper we have presented a novel approach for identifying causal relationship between concepts. this approach enables machines to extract causality even between non-adjacent concepts. hence, a significant improvement was delivered comparing to naive baselines. furthermore, we represented the causal knowledge extracted from human-written language in form of causal bayesian network. to the best of our knowledge this representation is novel. having a causal bayesian network can empower many downstream applications, including question-answering and rea-soning. among all applications, causal and counterfactual reasoning, which can be build on top of the outcome of this paper, may address some current hallmarks of artificial intelligence systems.