It’s Time to Reason: Annotating Argumentation Structures in Financial Earnings Calls: The F in A rg Dataset (2022)

Abstract: With the goal of reasoning on the financial textual data, we present in this paper, a novel approach for annotating arguments, their components and relations in the transcripts of earnings conference calls (ECCs). The proposed scheme is driven from the argumentation theory at the micro-structure level of discourse. We further conduct a manual annotation study with four annotators on 136 documents. We obtained inter-annotator agreement of lpha_{U} = 0.70 for argument components and lpha = 0.81 for argument relations. The final created corpus, with the size of 804 documents, as well as the annotation guidelines are publicly available for researchers in the domains of computational argumentation, finance and FinNLP.

## introduction the rise of data and the development of machine learning have led to the interdisciplinary financial technology (fintech) that aims at supporting the financial industry with digital innovations and technology-enabled business models @xcite . different applications have been explored such as fraud detection, digital payment, blockchain and trading systems. in terms of the latter, several factors impact its movement and it is hard, in reality, to get a very accurate prediction of the future stock prices. the efficient market hypothesis theory @xcite states that it is impossible to "beat the market" consistently since current prices incorporate all available information and expectations. nevertheless, the current view of the market comes from behavioural economics which see humans as irrational beings who are influenced by biases and experience when making investment decisions. in our previous work @xcite , we analysed the impact of stocktwits 1 and online news using a hybrid approach which consists of sentiment and event-based features as well as the price information for different observation and prediction time windows. @xcite aimed at capturing expert-like rationales from social media platforms without the requirement of the annotated data. similarly, @xcite hit the question: what makes some forecasters better than others? by exploring connections between the language people use to describe their predictions and their forecasting skills. on the other hand, @xcite targeted the prediction of professional analysts recommendations who influence the decisions of many investors towards buying or selling in particular markets. their findings confirm that earnings calls are moderately predictive of analysts' decisions even though theses decisions are function of different parameters including private communication with company executives and market conditions. moreover, while different works considered the sentiment and semantic analysis of text, we are looking towards a deeper understanding and interpretation of the language by the means of argument mining. according to @xcite , argument mining can be applied to understand the public's expectations for the market, providing valuable information for investment and other close applications. while they mostly studied the investors' posts on social platforms, we aim to particularly study the impact of arguments on the professional analysts themselves during the earnings conference calls (eccs). therefore, we present in this paper the first step of that methodology by discovering and annotating argumentation structures in eccs. eccs are generally held in every fiscal quarter and consist of three main parts: a safe harbor statement, a presentation and the question answering (q&a) session. in the presentation, executives give the statements about the performance of company in the last quarter and exhibit their expectations for the next one. during the q&a session, professional analysts ask their questions and demand clarifications from the company's representatives. different studies found that the discussion during the question answering session is the most informative and influencing part on the market @xcite . therefore, we focus in our study on these sessions, and more specially on annotating the arguments, their components and relations in the given answers of the company executives, where they try to justify their opinions and convince the other party to believe in them, which is indeed the essence of argumentation @xcite . to the best of our knowledge, no prior work has been carried out to annotate arguments in earnings calls transcripts. therefore, the contributions of this paper are the following: this paper is organized as follows: in section 2, we explore a conceptual background of argumentation modeling, and we highlight related works on argumentation in finance and on eccs in particular. in section 3, we present our proposed annotation scheme to model the argument components and relations in the executives' answers stated during the earning call. we further illustrate in section 4 the whole process of corpus creation. we move to distilbert results on our dataset in section 5. we finally move back to the big picture of the financial application and discuss the future directions in section 6. ## argumentation structure in earnings calls in this section, we discuss our proposed annotation scheme to model the argument components as well as the argumentative relations that constitute the argumentative discourse structure of earnings calls. we have first to point that the answers do not exhibit any common structure among all of them, to be hence structured as a connected tree or graph with circular relations. rather, the answers are full of arguments that may or may not be directly related. this could be justified by the fact that those answers are part of an oral argumentation limited by time. therefore, the company representatives tend to basically enumerate their evidences (premises) that support their claims. they may make the link between different claims and reasons they mentioned (or reformulate the same claim as well), whereas in most cases, they move to the next question. hence and to simplify the task enough, we did not ask the annotators to define the relations between the arguments (macro-structure level) or to follow more fine-grained annotation scheme that will differentiate the major claim from other claims as in @xcite . instead, we are interested in detecting the arguments themselves as independent units. in particular, we model the structure of each argument using one-claim-approach proposed by @xcite . this approach considers only the root node of an argument as a claim and the remaining nodes in the structure as premises. the arrow from the premise to the claim composites the relation which can be either a support or an attack relation. figure 1 represents a sample of our annotation scheme which implies that we can have different types of micro-structure arguments (e.g., basic, convergent and serial) in one answer. moreover, you can see a real example of the data in figure 2 . as we have mentioned, we are in particular interested in annotating the arguments stated by the company representatives. therefore, in the answer of luca maestri, we see first some general information that is not argumentative (stated in italic face), then the speaker start to argue about his claim (c 1 ) by stating different premises. the annotator marked every sentence (p 1 to p 4 ) as a premise since they all emphasize the stance of the speaker. in this example, all those premises belong to the same claim and they are all marked with a support relation type. ## corpus creation the motivation for creating a new corpus is threefold: ## evaluation as a base-line model, we fine-tuned distilbert @xcite with our dataset on the argument identification task (i.e., argument/ non-argument classification) at the sentencelevel. we aim in the future work to employ this data and train models to the end of prediction of analysts' post-call recommendations. this opens up different research questions like the required granularity of the data, the emission time of the recommendation's announcement, the analyst's questions (topic and sentiment) during the earning call (if applicable) and others. however, we believe that it's time to reason on financial textual data and to move from basic linguistic features, semantics and sentiment analysis to the reasons behind it and the quality of it with the help of argument mining and argument quality assessment which we will address in our future work as well. as a conclusion, we claim that our dataset presented in this paper will foster the research in fintech domain in parallel with computational argumentation as an nlp task itself.