CR - COPEC : Causal Rationale of Corporate Performance Changes to learn from Financial Reports (2023)

Abstract: In this paper, we introduce CR-COPEC called Causal Rationale of Corporate Performance Changes from financial reports. This is a comprehensive large-scale domain-adaptation causal sentence dataset to detect financial performance changes of corporate. CR-COPEC contributes to two major achievements. First, it detects causal rationale from 10-K annual reports of the U.S. companies, which contain experts’ causal analysis following accounting standards in a formal manner. This dataset can be widely used by both individual investors and analysts as material information resources for investing and decision-making without tremendous effort to read through all the documents. Second, it carefully considers different characteristics which affect the financial performance of companies in twelve industries. As a result, CR-COPEC can distinguish causal sentences in various industries by taking unique narratives in each industry into consideration. We also provide an extensive analysis of how well CR-COPEC dataset is constructed and suited for classifying target sentences as causal ones with respect to industry characteristics.

## introduction many critical decisions on events may require appropriate explanations of decisions based on accurate causal rationale. justifying the root statements is directly related to identifying the causes of the events. when one could observe an event that is an apparent cause of a desired outcome, one can make a proper decision with confidence on events. there has been extensive research in extracting causes of events in numerical data. as an example, granger causality finds linear temporal dependence between two (or more) temporal sequences @xcite . shapley values derive the contributions of individual input attributes when a decision is made by a complex function @xcite . these methods can explain numerical causes of decisions made by automated systems, e.g. robo-advisers in financial services @xcite @xcite . however, human uses various types of information including numerical and textual inputs when making an important decision. as an example, analysts write summarized reports by extracting related causal information from multiple textual sources in conference calls, annual reports, earning statements and markets reports. our research goal is to extract causal rationales from financial reports. in general, when investing firms predict a certain financial performance, they provide analysts' reports to support their predictions. thus, we want to generate appropriate explanations for performance changes of a corporate from official documents. our algorithm classifies causal sentences from documents and provides binary classification results. we consider fine-tuning of pre-trained neural language models (nlms) for the causality modeling tasks. pre-trained nlms have been state-of-thearts for many natural language processing (nlp) tasks. for example, nlms such as bert @xcite and albert @xcite demonstrate outstanding performance in some tasks such as answering questions @xcite and computing conditional probabilities of masked words in a sentence @xcite . nonetheless, recent research indicates that the size of human-annotated data continues to be a significant factor influencing the performance of models @xcite . there are previous works that introduce dataset for causality detection in the financial domain @xcite . however, existing studies lacked consideration of industryspecific characteristics. thus, it can be beneficial to consider it because the items in the financial statements that greatly affect financial performance are different for each company's primary business. our main contribution is to collect sufficient annotations to achieve reasonable causality detection performance with nlms. we achieved this by collecting over 283k sentences from 1,584 10-k annual reports, that give a detailed summary of the financial status and business operations of each company, along with audited financial statements. then, we manually label individual sentences whether the sentences explain the cause of certain financial performance changes. we name the 283k pairs of a sentence and a corresponding label as causal rationale of corporate performance changes (cr-copec). cr-copec is built on a large scale with guides of experts in the financial domain. trained with our dataset, bert can distinguish sentences containing main causes of its financial events, from annual reports written officially from most u.s. public companies. thus, individual investors can save efforts to read a huge amount of reports by themselves. however, we find collecting the dataset does not solve the problem as itself. one challenge we observe in the process of annotation is diverse causality in industries. that is, we find diverse causalities over different sectors. another challenge is imbalanced training, where the number of data for each sector varies. thus, we need to build a model carefully, as applying a common model to all sectors does not work in our problem setting. we provide extensive analysis on our cr-copec dataset to overcome these issues. ## related work the causal rationale is "the true sufficient rationales to fully predict and explain the outcome without supurious information." @xcite and extracting rationale is invaluable when a decision has to be made. research on extracting rationale from text has been tried with various types of text documents @xcite @xcite . a model detecting and identifying rationale from chat messages was suggested in @xcite . bug reports from chrome web browser were used as main sources to extract rationale @xcite . moreover, patent documents were utilized to discover design rationale @xcite . to extract causal textual structures, one may consider a rule-based system where specific words such as 'due to', 'owing to' and 'affects' are listed to identify sentences including causal information for prediction @xcite @xcite . diverse studies have explored causality extraction datasets @xcite @xcite . some widely recognized datasets include semeval-2007 task 4 @xcite , semeval-2010 task 8 @xcite , pdtb 2.0 @xcite , bioinfer @xcite , and ade @xcite . among these, three datasets stem from general domain, while two originate from the biomedical field (bioinfer and ade). the maximum number of causality sizes of these datasets is less than 10k (pdtb 2.0, 9,190 causal examples). in contrast, cr-copec contains a larger substantial data size of 105,861 sentences, including 11,132 sentences with causal rationales. in addition, co-copec targets the financial domain especially focused on corporate performance changes from financial reports. these characteristics contribute to the uniqueness of cr-copec and potentially helpful for the decision of financial experts. there are previous studies that introduce causal rationale corpus in the financial domain. to identify causal sentences from uk preliminary earning announcements (peas), thirteen performance keywords including 'sales', 'revenue' and 'turnover' are used and selected sentences are annotated by human @xcite . @xcite 's study is the most related our research. herein, the authors built fincausal corpus collected from financial news and websites which labeled with tags indicating the presence of causality and causal chunk as quantitative or non-quantitative. the corpus consists of two subtasks. the first subtask (fincausal task 1) is a binary classification task that targets to extract text having causality. the second subtask (fincausal task 2) is a relation extraction task identifying the substrings that indicate cause and effect. the main difference between cr-copec and fin-causal is that texts are written with a formal tone in 10-k reports since they should comply with regulatory rules. in addition, cr-copec solely concentrates on causal rationales of accounting items considering unique characteristics of various industries. meanwhile, fincausal dataset itself is written our goal is to collect causal rationale of sentences that contain causal rationales in predicting changes in corporate performance based on key accounting items within the target industrial sector. for this purpose, we target management's discussion and analysis (md&a) section of 10-k reports since it provides the company's perspective on its operations and financial results of the prior fiscal year. we gather md&a reports filed in 1997 and 2017. md&a of 1997 is gathered from the existing md&a data repository @xcite ) and 2017's is downloaded directly from the securities and exchange commission (sec) system. then, we construct cr-copec dataset through keywordbased filtering (section 3.1) and human annotation process (section 3.2) illustrated in figure 1 . the validity of the dataset is verified in section 3.3. in addition, we provide protocols of the dataset to train nlms in section 3.4. ## conclusion we introduce a novel large scale dataset to extract causal rationales from financial reports in various sector. the dataset was annotated by considering accounting factors according to the industry. we further validate the process of building our cr-copec dataset. finally, through the qualitative and quantitative analysis, we observed that the model trained with cr-copec recognize the clue of causal sentences. we hope that our work would promote the study of causality detection in the financial text domain. license is not required to access the database. in addition, according to information presented on www.sec.gov is considered public information and may be copied or further distributed by users of the web site without the sec's permission. secondly, to all annotators, we paid them by obeying the minimum wage standard of national law. besides, the general annotators were full-time employees.