Domain Adaptative Causality Encoder (2020)

Abstract: Automated discovery of causal relationships from text is a challenging task. Current approaches which are mainly based on the extraction of low-level relations among individual events are limited by the shortage of publicly available labelled data. Therefore, the resulting models perform poorly when applied to a distributionally different domain for which labelled data did not exist at the time of training. To overcome this limitation, in this paper, we leverage the characteristics of dependency trees and adversarial learning to address the tasks of adaptive causality identification and localisation. The term adaptive is used since the training and test data come from two distributionally different datasets, which to the best of our knowledge, this work is the first to address. Moreover, we present a new causality dataset, namely MedCaus, which integrates all types of causality in the text. Our experiments on four different benchmark causality datasets demonstrate the superiority of our approach over the existing baselines, by up to 7% improvement, on the tasks of identification and localisation of the causal relations from the text.

## introduction causality is the basis for reasoning and decision making. while human-beings use this psychological tool to choreograph their environment into a mental model to act accordingly @xcite , the inability to identify causal relationships is one of the drawbacks of current artificial intelligence systems @xcite . the projection of causal relations in natural language enables machines to develop a better understanding of the surrounding context and helps downstream tasks such as question answering @xcite ), text summarisation @xcite , and natural language inference @xcite . the task of textual causality extraction can be divided into two main subtasks, causality identification and causality localisation. the former subtask focuses on identifying whether a sentence carries any causal information or not, which can be seen as classification problem. the objective of the latter subtask is to extract text spans related to cause and effect, subject to existence. the automatic identification and localisation of causal relations in textual data is considered a nontrivial task @xcite . causal relations in text can be categorised as marked/unmarked and explicit/implicit @xcite . marked causality refers to the case where a causal linguistic feature, such as "because of", is stated in the sentence. for example, in "his ocd is because of genetic factors.", because of is a causal marker, whereas in unmarked causality there is no such indicator. for instance, in "don't take these medications before driving. you might feel sleepy." the cause and effect relationship is spread between two sentences without a marker. on the other hand, explicit causality refers to the case where both cause and effect are mentioned in text. however, in implicit causality, either cause or effect are directly mentioned in the text. in a more complex case called nested causality, multiple causal relations may exist in one sentence (e.g., "procaine can also cause allergic reactions causing individuals to have problems with breathing"). all of these ambiguities contribute to the challenging nature of this task. traditional approaches to address the problem of causality extraction mainly relied on predefined linguistic patterns and rules to identify the existence of causal relations in a sentence @xcite . more advanced approaches combined pattern-based methods with machine learning techniques @xcite , and as such they require heavy manual feature engineering to perform reasonably. to overcome this problem, the recent approaches have adopted deep learning techniques to extract meaningful features from the text @xcite . however, all the aformentioned approaches suffer from the problem of domain shift, where there is a distribution difference between the training and the test data. more specifically, the existing approaches perform poorly on the data from a new test domain (e.g. financial) which is contextually different from the training domain (e.g. medical). to overcome the limitations of existing approaches on the tasks of causality identification and localisation, we propose a novel approach for domain-adaptive causality encoding which performs equally well when applied on the out-ofdomain sentences. our contribution is three-fold: ## related works the projection of causal relation in textual data can be in various forms, depending on the type of causality. the categorisation mentioned in section 1 can indicate the relation between pairs of events, phrases, concepts, named entities or a mixture of the aforementioned text spans @xcite . some works in the area have endeavoured to extract and present the textual information between concepts or events. causal relations are a component of semeval task @xcite , but it involves a limited set of causal relations between pairs of nominals. @xcite developed a framework based on combining semantic association and supervised causal discourse classification in order to identify causal relations between pairs of events. they expand the patterns in pdtb @xcite using a self-training approach. other methods @xcite leveraged linguistic features such as part-of-speech information, alongside with discourse markers, for identifying causal relations between events. an et al. (2019) used the syntactic patterns and word vectors to develop an unsupervised method for constructing causal graphs. to expand the repository of causal syntactic patterns, hidey and mckeown (2016) built a parallel corpus between english wikipedia and simple wikipedia, where the same causal relation might be in different syntactic markers in two parallel sentences. a supervised method was adapted by mirza and tonelli (2016) using lexical, semantic, and syntactic features within a sentence to address this task. using hidey and mckeown (2016)'s method, martínez-cámara et al. (2017) created a set of labelled sentences, assuming all of the sentences include a causal relation, and presented a neural model based on lstm to identify causality. dasgupta et al. (2018) collected a dataset and developed a model using bilstm for extracting causal relation within a sentence foot_1 . other approaches in event prediction applied granger causality @xcite to identify causal relations in time series of events @xcite . @xcite defined a proxy variable, which may carry some information about cause and effect, to identify causal relationship between static entities. @xcite al. (2017) 2019 also deployed a self-attentive neural model to address the task of causality identification, however, the extraction of causal information is not addressed by their model. in more recent works, a dataset of counterfactual sentences was released, as a part of semeval2020 task5 @xcite . the aim of this task is to identify and tag the existing counterfactual part of the sentence. some works have attended to address this task using different deep learning architectures @xcite . while counterfactuals are usually represented in form of a causal relation, this dataset does not cover different forms of textual causality. as opposed to the aforementioned models, we propose a unified neural model for addressing both tasks of identifying and localising causality from a sentence. our method leverages both syntactic and semantic relations within a sentence, and adapts to out-of-domain sentences. ## our approach in this section, we first describe the architecture of causality extractor, which uses graph convolutional networks (gcn) at its core. we then present how we make use of the adversarial learning strategy for adapting the model to new domains. figure 1 illustrates the high level overview of our approach. ## experiments in this section, we first describe the datasets that have been used for the evaluation of our models, including our collected dataset. then we present results of our proposed models on both (adaptive) causality identification and causality localisation. ## conclusion in this work, we propose a new dataset for the task of causal identification and causal extraction from natural language text. we further propose a neuralbased model for textual causality identification and localisation, which makes use of dependency trees. we then make use of adversarial training to adapt the causality identification and localisation models to new domains. empirical results show that our method outperforms state-of-the-art models and their adapted versions.