Incorporating Stock Market Signals for T witter Stance Detection (2022)

Abstract: Research in stance detection has so far focused on models which leverage purely textual input. In this paper, we investigate the integration of textual and financial signals for stance detection in the financial domain. Specifically, we propose a robust multi-task neural architecture that combines textual input with high-frequency intra-day time series from stock market prices. Moreover, we extend wt–wt, an existing stance detection dataset which collects tweets discussing Mergers and Acquisitions operations, with the relevant financial signal. Importantly, the obtained dataset aligns with Stander, an existing news stance detection dataset, thus resulting in a unique multimodal, multi-genre stance detection resource. We show experimentally and through detailed result analysis that our stance detection system benefits from financial information, and achieves state-of-the-art results on the wt–wt dataset: this demonstrates that the combination of multiple input signals is effective for cross-target stance detection, and opens interesting research directions for future work.

## introduction stance detection (sd) is the task of automatically classifying the writer's opinion expressed in a text towards a particular target @xcite . starting from @xcite 's seminal work, research on twitter sd gained increasing popularity @xcite , embracing new topics @xcite @xcite and languages @xcite @xcite . in recent years, research on sd has mainly focused on cross-target generalization, in which an sd system is tested on targets unseen during training @xcite . cross-target generalization constitutes one of the biggest challenges in twitter sd @xcite : in this context, researchers investigated a wide range of techniques, including adversarial training @xcite , cross-lingual transfer @xcite , knowledge transfer using semantic and emotion lexicons @xcite , weak supervision through synthetic samples @xcite , and various types of cross-domain transfer @xcite . in this paper, we study multimodality as a means to enhance cross-target generalization in twitter sd. multimodal machine learning studies the integration and modeling of multiple modalities @xcite , where a modality refers to the way in which something happens @xcite . our contributions are as follows: despite being an established research area in nlp @xcite , sd in a multimodal context is still understudied. ## problem formulation we study sd in the financial domain and consider tweets discussing m&a operations, i.e. financial transactions in which the ownership of a company (the target) is transferred to another company (the buyer, @xcite ). an m&a process usually comprises many stages, ranging from informal talks between the companies' boards to acquisition planning, negotiations, and external approvals, up to the closing of the deal (or its rejection, e.g. by antitrust bodies). m&as account for billions of dollars of investment globally and have been widely studied under many aspects @xcite . they are well known in nlp @xcite @xcite and constitute an important application in other ai fields, with a strong focus on automatic prediction of the m&a outcome @xcite @xcite . in our task, a model receives a tweet and a target merger, and has to predict the stance expressed by the tweet's author with respect to the likelihood of the merger to succeed: in this paper, we use a variation of the stock market prices from the n days prior to a tweet's posting as a means to provide a model with such context. according to the efficient market hypothesis @xcite , stock market prices reflect all publicly known information. even though the efficient market hypothesis is controversial @xcite , stock market prices still reflect a consider-able amount of publicly known information. therefore, we argue that they can be used as a proxy for the available knowledge about the merger at a given time. the relationship between rumors about an m&a operation and their effect on the involved companies' stocks is mutual and has been widely studied in finance @xcite @xcite @xcite , but never investigated in nlp. to our knowledge, the integration of textual and financial data signals has been studied for financial forecasting @xcite @xcite @xcite , but has yet to be investigated for sd. ## extending the wt-wt dataset text signal. as our text signal, we use will-they-won't-they @xcite 4 , which collects english tweets discussing four m&as between us companies ( analysis. figure 1 shows an example of the integration of the two signals. on the day the antitrust complaint was made to the department of justice regarding the m&a operation, antm's price increased while ci's decreased. such movements testify that the event changed the world's view: people believe that the merger is less likely to happen, and this is reflected by their investment decisions. the direction of the price variation reflects standard m&a theory @xcite : the buyer will not buy the target's shares at a premium, thus the owners of target's stocks will not profit from the acquisition. the price variation is useful for classifying a tweet on that day, as it implies that the likelihood of a refute label is higher. this is reflected in the tweet distribution in the lower part of the figure : the distribution of tweets on that day shows that most of them were indeed refuting. we report one more example in appendix a. ## models as shown in figure 2 , our multitask sdtf model is composed of a textual, a financial and a multimodal component. ## experimental setting preprocessing. we perform minimal preprocessing on the textual signal. concerning the financial signal, we consider a window of 30 timepoints in the past, and price variations every 30 minutes: depending on the tweet's posting time, this accounts for the previous ∼2.5 days foot_8 . for ff, we predict ups or downs in the considered company's closing price 2 hours after the tweet foot_9 (see appendix b.1 for details). training setup and evaluation. details on the training setup and (hyper-)parameter settings are reported in appendix b.2 for replication. following @xcite al. (2018); conforti et al. (2020b) , we consider macro-averaged precision, recall and f 1 score. to account for performance fluctuations @xcite , we average three runs for each model (standard deviation is reported in appendix b.2). baselines. we consider six published baseline models, including the four best models of @xcite ## results and discussion adding the financial signal. adding our financial component proves to be effective over all considered targets, with improvements in f 1 scores up to +5.8 (aet_hum). single-label performance seems to suggest that price variations encode very useful information for all labels, resulting in notable improvements not only on the unrelated (+3.7), but also on the refute and support samples (resp. +2.1 and +5.4 in accuracy): this is important because those labels, apart from being the minority classes, arguably constitute the most relevant information for downstream tasks @xcite . adding multi-task objectives and ablation experiments. results of ablation experiments ( moving to the other ancillary tasks, ff results are encouraging 12 , even if we considered a considerably shorter time window of historical pricing than architectures specifically designed for ff @xcite @xcite . this suggest that the learned multimodal textual and financial vectors constitute an informative input for the ff predictors. ## conclusions in this paper, we studied the well-established task of twitter sd in a multitask scenario, focusing on the financial domain. we proposed sdtf, a novel model which integrates two modalities, text and financial time series data. we extended wt-wt, a large dataset for financial sd, with financial signals from stock market prices. our detailed analysis of models' results demonstrated that financial sd on tweets benefits from such signals: models which include textual and financial features showed better cross-target generalization capabilities, and obtained better results on all stance labels. finally, we proposed a simple but effective setting to leverage useful signals encoded in synthetic samples, reaching state-of-the-art results on wt-wt. we release the financial signal collected to complement wt-wt: together with the stander corpus of news sd, which discusses the same mergers, it constitutes an invaluable and unique resource to foster research on multi-modal, multi-genre sd, and to model the integration and mutual influences between stock market variations, tweets, and authoritative news sources.