Exploring Causal Directions through Word Occurrences: Semi-supervised B ayesian Classification Framework (2023)

Abstract: Determining causal directions in sentences plays a critical role into understanding a cause-and-effect relationship between entities. In this paper, we show empirically that word occurrences from several Internet domains resemble the characteristics of causal directions. Our research contributes to the knowledge of the underlying data generation process behind causal directions. We propose a two-phase method: 1. Bayesian framework, which generates synthetic data from posteriors by incorporating word occurrences from the Internet domains. 2. Pre-trained BERT, which utilises semantics of words based on the context to perform classification. The proposed method achieves an improvement in performance for the Cause-Effect relations of the SemEval-2010 dataset, when compared with random guessing.

## introduction understanding causality is critical for various tasks including question answering. @xcite provide a great example: dorothy poured water on the fire. the fire went out. subsequently, if it is followed by the question did she put out the fire?, the answer is yes because poured water on implies that the two sentences are causally linked. when provided with two entities, namely e1 and e2, in the sentence that are known to have a causal relation, the causal direction tells us which one is a cause and which one is an effect. in the previous example, poured water on is the cause whereas fire went out is the effect. therefore, the causal direction in this case is poured water @xmath0. in this study, we show that word occurrences resemble the characteristics of causal directions. our research contributes to the knowledge of the underlying data generation process behind causal directions. to achieve this, we propose a semi-supervised classification method foot_0 for determining a causal direction if its causal relation is known to exist in the sentence. the github page foot_1 is available for reference purposes. ## related work in this section, we provide a brief overview of two approaches for identifying causal relations. the first approach, pointwise mutual information, is designed to eliminate the need for corpus creation. the second approach, data augmentation, clearly involves the need of creating a corpus. ## dataset for our study, we use the semeval-2010 (task 8) dataset @xcite . this dataset focuses on a multi-class classification task. however, for the purpose of our study, we narrow our attention to the specific category labelled as cause-effect in the dataset. a sentence is considered as cause-effect if two entities, which are marked @xmath1, show a causal relation. the cause-effect category comprises a total of 1,331 instances, divided between the training and test data. in the training data, there are 1,003 instances labelled as cause-effect, with 659 of them demonstrating the relationship @xmath2. in the test data, out of the 328 cause-effect instances, 134 exhibit the @xmath3. there are no bidi- ## method in order to gain insights into the similarity between word occurrences and causal directions, we simulate a semi-supervised classification setup and exclude the training data from our analysis. the motivation behind examining word occurrences is that if two words frequently collocate, this linguistic clue can be used to infer a causal direction. for instance, if the words smoking and lung cancer frequently collocate, this pair suggests a potential causal direction, the direction of which we need to determine. our method consists of two phases -bayesian framework, and pre-trained bert. ## experiments to evaluate our method, we have two experimental set-ups: (a) random and (b) bayesian + pre-trained bert. to the best of our knowledge, there are no existing semi-supervised models for detecting causal directions. thus, the random approach serves as the baseline, which blindly guesses causal directions. while one might argue that a baseline should always predict @xmath4, it is important to note that the proposed method does not leverage such information. hence, the random approach is more appropriate for our evaluation. given the semeval-2010 (task 8) dataset is well known, it might be tempting to consider using an established supervised model as a baseline. using a supervised model as a baseline in a semi-supervised classification scenario is not recommended for several reasons. firstly, supervised models are trained on labelled data whereas semi-supervised models lack annotated labels. this difference renders any experimental results incomparable: using a supervised model as a baseline can have unrealistic expectations for the performance of a semisupervised model. lastly, the primary objective of our study is to demonstrate the resemblance between word occurrences and the characteristics of causal directions. using a supervised model as a baseline may distract from this objective. we conducted the experiment under two distinct settings in the bayesian framework. in the first setting, we examined whether the priors were satisfied with ppc (referred to as ppc+), resulting in predictions for 7 out of 328 cases. in the second setting, we did not apply any prior checks (referred to as ppc-), and this yielded predictions for 281 out of 328 cases. this allows us to gain insights into the quality of the data generated by the bayesian framework. because there were not enough predicted directions generated in the ppc+ setting, primarily due to a substantial number of the priors being rejected by ppc (for a detailed ## results ## discussion although the bayesian framework is inherently statistically sound, it is not immune to failure when confronted with certain word occurrences used in constructing the priors. in this section, we explore the bayesian framework more comprehensively, aiming to understand the rationale behind the specific predictions made by the bayesian framework, especially two cases from the test data where the predictions were incorrect. ## further work there are many areas we can explore to improve the study further. in this section, we present three of them: earth mover's distance, mixture models, and bayesian network. ## conclusion in this paper, we have shown empirically that word occurrences resemble the characteristics of causal directions. this finding provides significant implications and contributes significantly to our understanding of the data generation process underpinning causal directions.