Event-Driven Learning of Systematic Behaviours in Stock Markets (2020)

Abstract: It is reported that financial news, especially financial events expressed in news, provide information to investors’ long/short decisions and influence the movements of stock markets. Motivated by this, we leverage financial event streams to train a classification neural network that detects latent event-stock linkages and stock markets’ systematic behaviours in the U.S. stock market. Our proposed pipeline includes (1) a combined event extraction method that utilizes Open Information Extraction and neural co-reference resolution, (2) a BERT/ALBERT enhanced representation of events, and (3) an extended hierarchical attention network that includes attentions on event, news and temporal levels. Our pipeline achieves significantly better accuracies and higher simulated annualized returns than state-of-the-art models when being applied to predicting Standard&Poor 500, Dow Jones, Nasdaq indices and 10 individual stocks.

## introduction it is widely reported that financial news, especially financial events expressed in the news, influence the movements of stock markets @xcite @xcite @xcite @xcite . for example, one company's releasing of new products brings novel time-sensitive factors to the stock movement of that company and further to the whole market; changing of interest rates from the central bank of the country bring changes of currency fluidity invested in the stock market. frequently, good news brings increased estimation of the future value of the target company and consequently a higher stock price tendency. warren buffett said in his interview 1 that he fre-1 https://www.youtube.com/watch?@xmath0. regarding the rich information included in daily published news articles, we are encouraged to read them not by ourselves but by employing deep learning algorithms to guide our investments. one solution is to automatically collect thousands of news published every day, extract financial events from the news, and rank the importance of the events to predict market behaviours. in particular, we are aiming at quantifying the latent relevance of between financial event streams and target stocks' price volatilities. however, this is not a trivial task and there are a number of challenges. first, it is ambiguous to define good or bad news. for example, bad news for one company could be worse news to its downstream supply-chain partner companies yet good news to its competitors. it is time-consuming and untrackable to annotate news manually and train a sentiment analysis model on it, considering that the generalization of the model to novel financial events is fragile. second, how shall we express the impact of the news published in different days? generally, news articles have their individual and accumulated influences to the investors. for example, billion-dollar mergers and acquisitions frequently bring bigger and longer impact than adding a new member to the board. third, regarding that news articles are too long to be generalized for comparison, how to extract shorter, highlevel summarized and complete financial events from news? there are factual and opinion-level events expressed in news and their appearance order matters for concluding the news. finally, how to measure the similarities among financial events guided by historical stock movements? the latentspace representation of events empowers the generalization ability of a machine learning model for predicting event-volatility based on representing and projecting novel events into existing event rep-resentation space. in order to tackle these challenges, we propose a classification network for stock movement prediction. we first describe a combined event extraction method basing on open information extraction @xcite and neural co-reference resolution @xcite to keep the mined events to be compact and complete with meaning. then, we learn the event representations and similarities among events under bert/albert @xcite pretrained contextualized language models. finally, we construct a hierarchical attention network (han) @xcite that employs events, news, and historical days to organize the granularity of information for final multi-category stock movement prediction. ## bert-enhanced hans with three-level attentions we propose two updates of the original hans used in @xcite and @xcite : inserting an event-layer between words and news levels, and representing events by pretrained contextualized language models such as bert @xcite instead of bidirectional gated recurrent unit (bi-gru) @xcite plus attention networks. the encoding of financial information in the units of events is inspired by @xcite . the difference is that, we replace the neural tensor network @xcite by bert. @xcite directly used selected words in news as the initial layer in their han for stock movement prediction. the drawbacks that we find are (1) one news include hundreds to thousands words and it is difficult to select representative words from them to fit the final stock movement prediction task, and in the event-level network block, our target is to construct a recurrent representation for the news by taking all its m events into consideration. we first apply a bi-gru to the vectors of events: the result h i incorporates the contextual information of m events. through this way, we encode the event sequence of each news. considering that different events contribute unequally to the final representation of one news, we adopt the attention mechanism @xcite to aggregate the events weighted by an assigned attention value, in order to reward the event(s) that offer critical information: we first estimate attention values by feeding h i through a one-layer full-connection linear neural network followed by a sigmoid function to obtain the event-level attention value u i . then, we calculate its normalized attention weight β i through a softmax function. finally, we calculate the news vector n i as a weighted sum of each event vector. a parameter θ i is attached to each event in the softmax layer, indicating in general which event is more representative. through this way, we construct the event-level representation and attention for each news. we reuse these recurrent and attention layers to news-level block. suppose that there are maximum l news in one day's financial news collection. we construct news-level recurrent and attention computing for one day's news. here, bi-gru is employed to capture the "contextual" relations among news sorted by their published timestamps. also, attention mechanism is employed to capture which news contributes more to the representation of that day's vector d i . again, d i is an attention weighted sum of the news vectors. in the temporal level, we suppose that there are k historical days for tomorrow's stock movement prediction. the news published at different dates contribute differently to the final stock trend. we the third time use bi-gru to capture the latent dependencies of day vectors and attention mechanism to weight the day vectors. the final vector v is a weighted sum of d 1 to d k . the final discriminative network is a standard multi-layer perceptron (mlp) that takes v as the input and produces the multi-category classification of the stock movement. generally, the prediction is explainable by listing the most valuable events in the most trustable news published in those important historical days ( ## related work employing event representations for stock movement prediction has been proposed in @xcite @xcite for index and individual stock prediction. for example, external commonsense knowledge, such as the intents and emotions of the event participants, was utilized in @xcite to enhance event representation learning. we follow the usage of events dynamically extracted from financial news. the differences are that we additionally perform a neural coreference resolution module to keep the events being independent and we did not perform any manual filtering (refer to figure 1 and in addition to event representation learning followed by shallow neural networks, deep hans @xcite ) that embeds various granularities of market document-style information are another tendency. hierarchical graph attention networks @xcite made use of existing corporate relational data from wikidata. examples of these relational data are alike triplets of [apple, founded by, steve jobs], which align with the commonsense knowledge used in @xcite . graph neural networks by incorporating company knowledge graphs which express inter-market and inter-company relations were proposed in @xcite for japanese stock market prediction. in stead of using existing relational data, our pipeline keeps updating itself by extracting updated events from lastly published news. in our eventhans, the final predictions are explainable in terms of which events expressed in which news published in which day ( on the other hand, pretrained contextualized language models, such elmo @xcite , bert @xcite , gpt @xcite and their consequent variants @xcite are leading the research in numerous nlp applications. however, most existing pretraining models can only take limited length of sequences (such as 512 tokens) as inputs while there are thousands of news containing million-level words appearing everyday. each news expresses limited information and investors are required to accumulate news together to predict their influence to the future market. in this paper, we combine three elements together: event streams represented by bert/albert and their integration in hans to capture super long event sequences for better stock movement prediction. the strengths of our pipeline include: (1) significantly large-scale syntactically independent event sequences are extracted, (2) extremely long event sequences are leveraged, and (3) explainable predictions with accuracies significantly better than state-of-the-art baselines are achieved. ## conclusion in this paper, we investigate answers to the question if textual information such as financial events can qualitatively and quantitatively influence the stock market's movements. our contributions to this field include: a neural co-reference enhanced openie pipeline for event extraction from financial news, bert/albert enhanced event representations, an event-enhanced han that utilizes event, news and temporal attentions, and significantly better accuracies and simulated annualized returns than four state-of-the-art baselines on 3 indices and 10 stocks. we observe that quantitative prediction is feasible in a sense that the strength or importance of news is successfully understood and absorbed by the market. this aligns with the efficient-market hypothesis that asset prices reflect all available information 14 @xcite . for sure, there are a lot of future work: (1) modelling and predicting of global scale markets such as nikkei 225, topix in japan, hsi index in hong kong, and (2) integrate textual information with financial asset price models, such as capital asset pricing model (capm) @xcite , arbitrage pricing theory (apt) @xcite , and multifactor models @xcite will be one interesting direction that combines nlp techniques and finance theory through deep neural networks for a same target of future asset pricing: investors read both textual finance information and digital finance indicators.