ISIKUN at the F in C ausal 2020: Linguistically informed Machine-learning Approach for Causality Identification in Financial Documents (2020)

Abstract: This paper presents our participation to the FinCausal-2020 Shared Task whose ultimate aim is to extract cause-effect relations from a given financial text. Our participation includes two systems for the two sub-tasks of the FinCausal-2020 Shared Task. The first sub-task (Task-1) consists of the binary classification of the given sentences as causal meaningful (1) or causal meaningless (0). Our approach for the Task-1 includes applying linear support vector machines after transforming the input sentences into vector representations using term frequency-inverse document frequency scheme with 3-grams. The second sub-task (Task-2) consists of the identification of the cause-effect relations in the sentences, which are detected as causal meaningful. Our approach for the Task-2 is a CRF-based model which uses linguistically informed features. For the Task-1, the obtained results show that there is a small difference between the proposed approach based on linear support vector machines (F-score 94%) , which requires less time compared to the BERT-based baseline (F-score 95%). For the Task-2, although a minor modifications such as the learning algorithm type and the feature representations are made in the conditional random fields based baseline (F-score 52%), we have obtained better results (F-score 60%). The source codes for the both tasks are available online ( https://github.com/ozenirgokberk/FinCausal2020.git/ ).

## introduction the causality detection, which is a well-known task in text mining, has a goal of the extraction of the cause-effect relations from a given text. although the causality detection task has been applied to various types of texts in various domains such as biomedical @xcite @xcite , clinical @xcite , and financial @xcite @xcite until now, there is still room for the improvement of these approaches. the fincausal-2020 shared task is a community-wide effort, which presents an opportunity for the researchers to test their financial causality detection systems on a common platform @xcite . the fincausal-2020 shared task consists of two sub-tasks, which are called task-1 (sentence classification) and task-2 (relation extraction). task-1 is a binary classification task in which the participants are intended to classify the given financial sentences as 1 if the text section is considered containing a causal relation, and 0 otherwise. task-2 is a relation extraction task in which the participants are intended to extract the text sections as a cause or an effect from the given text in which it is known that there is at least one causality relation. in this paper, the two systems that are developed for the two sub-tasks of the fincausal-2020 shared task are explained. for the task-1, the problem is handled as a binary classification problem. we have used two different linguistically informed machine learning classifiers which are naive bayes @xcite and linear support vector machines. for the task-2, which aims the automatic extraction of cause-effect relations from a given financial text, conditional random fields @xcite are used. ## data set the data set, which is provided by the organizers, is created from 2019 financial news. the training data set for the task-1 has entries which are the index of the sentence, the sentence, and the class label which the corresponding sentence belongs to, while the test data set entries consist of the index of the sentence and the sentence only without the class labels. the test data set that is related with the task-2 consists of the index of the sentence and the sentence entries, while the training data set has entries which are the index of the sentence, the sentence, as well as the cause mention, and the effect mention of the corresponding sentence. ## task-1: sentence classification the workflow of our system for the task-1 begins with the preprocessing phase of the data, which is followed by the feature representation of the sentences. after obtaining the sentences as feature vectors, different supervised machine learning classifiers are used to predict the given text as containing a causal relation or not-containing a causal relation. in the following subsections, a detailed explanation of the system that we have developed for the task-1 is provided. ## results the results for the task-1 of the fincausal 2020 shared task, which are obtained on the training data set with 5-fold cross-validation, are shown at the results of the crf-based basedline model with the base features and the l-bfgs learning algorithm for the task-2 are shown at on the other hand, the results of the crf-based system with the linguistically informed features and the average perceptron learning algorithm for the task-2 are shown at the official evaluation results for the task-1 of the fincausal 2020 shared task, which are obtained on the test data set, are shown in ## conclusion in this study, we have presented two systems that are implemented in the scope of the fincausal 2020 shared task. the aim of the first system is the binary classification of the sentences in a financial text as carrying causality relation or not-carrying causality relation, whereas the goal of the second system is the extraction of the cause-effect relations in the financial text. in conclusion, compared to the baseline, for the task-1, there is a small difference (1%) between the results of the proposed approach based on linear support vector machines which requires less time compared to the bert-based baseline. for the task-2, on the other hand, better results are obtained compared to the baseline.