Predicting Modality in Financial Dialogue (2020)

Abstract: In this paper, we perform modality prediction in financial dialogue. To this end, we introduce a new dataset and develop a binary classifier to detect strong or weak modal answers depending on surface, lexical, and semantic representations of the preceding question and financial features. To do so, we contrast different algorithms, feature categories, and fusion methods. Perhaps counter-intuitively, our results indicate that the strongest features for the given task are financial uncertainty measures such as market and individual firm risk.

## introduction in this paper, we predict the modality of answers depending on their preceding question and other features in financial dialogue. modality is an important concept in principal-agent settings of asymmetric information such as the stock market, since it can be used as a strategic tool by company executives: using modality markers such as "probably" or "certainly," investor expectations can be managed or the effect of negative news can be mitigated without having to commit to false statements. @xcite suggest to examine the hypothesis that larger shares of modal words in conference calls might worsen stock or operating performance. subsequently, @xcite found that executive modality is indeed predictive of stock price as well as analyst's earnings forecasts and firm valuations @xcite . although different to past work, we explore causes, not effects, of modality in the financial domain, this shows that modality prediction has potential down-stream uses in return, risk, and analyst forecast prediction. specifically, modality prediction models could be employed for intra-day return prediction. ## related work in the financial domain, the task of modality or vagueness detection is closely related to risk and return prediction. @xcite handcrafted a set of sentiment lexica based on frequent terms in a sample of 60k 10-ks. these lexica (from now on: lm) span the categories positive, negative, uncertain, litigious, strong modal, and weak modal and have been shown to possess predictive power of risk. subsequent work in the nlp community automatically expands said lexica by adding semantically similar terms according to word embedding models for predictions of risk in form of return volatility @xcite or correlations with it @xcite . @xcite perform speculation detection in the monetary policy domain as a binary sentence classification task. they use a list a list of uncertainty triggers extracted from the conll-2010 shared task's training set @xcite , the lm uncertain lexicon, and an own list of speculation triggers tailored to the task. theil et al. (2018a) train a binary sentence classifier predicting the linguistic uncertainty of 1k sentences randomly sampled from a dataset of earnings calls. they use lemmatized bag-of-words (bow) vectors, part-of-speech tags, a set of handcrafted syntactic rules, the conll-2010 list of uncertainty triggers @xcite , and the lm uncertain lexicon. their results indicate that bow vectors and the lm lexicon are the strongest features, which is why we include them in our classifier, too. note that different to these works, we do not aim to predict the uncertainty of a sentence given its content, but rather the uncertainty of an answer given the content of the preceding question. furthermore, we explore additional feature categories, such as semantic or financial features. using a set of 120k earnings calls, @xcite find that the modality of executive utterances is correlated with post-call stock price, analyst's earnings forecasts, and firm valuation. @xcite gather 12k earnings call transcripts and find that pragmatic and semantic features are moderately predictive of analysts' price forecast targets following the call dates. their pragmatic feature set contains a dictionary of uni-and n-gram hedges @xcite as well as the lm dictionary @xcite ; however, they find the influence of semantic features (bow and doc2vec @xcite as past research suggests a correlation between linguistic modality and financial risk. different to these works, we do not predict external financial measures based on linguistic features. instead, we aim to predict a linguistic variable (modality) as we are interested in uncovering its determinants in financial q&a settings. ## methodology we begin by introducing a new dataset for modality prediction in financial dialogue (cf. section 3.1), proceed by defining different features sets (cf. section 3.2), and finally introduce a classifier for our binary classification task (cf. section 3.3). ## conclusion in this paper, we present a new dataset for modality prediction in financial dialogue and introduce a binary classifier to address this task. in our experiments, we contrast the performance of various algorithms, feature sets, and fusion methods. interestingly, we reach a counter-intuitive result indicating that financial features (most prominently market and firm risk) possess a higher predictive power for answer modality than linguistic features (such as bags-of-words, topic models, or word embeddings) of the preceding question. in future work, it would be interesting to explore whether this effect persists when using a larger context window for the textual representations.