From Stock Prediction to Financial Relevance: Repurposing Attention Weights to Assess News Relevance Without Manual Annotations (2021)

Abstract: We present a method to automatically identify financially relevant news using stock price movements and news headlines as input. The method repurposes the attention weights of a neural network initially trained to predict stock prices to assign a relevance score to each headline, eliminating the need for manually labeled training data. Our experiments on the four most relevant US stock indices and 1.5M news headlines show that the method ranks relevant news highly, positively correlated with the accuracy of the initial stock price prediction task.

## introduction events such as lawsuits, the unveiling of a newly discovered technology, the introduction of new legislation, or previous market movements can have a significant impact on stock prices. a quick and informed reaction to such an event is crucial for financial analysts. information overload is pervasive in the financial industry, hindering the analysts' ability to incorporate the most relevant events into their decision process. one of the main natural language understanding challenges across many industries is to prioritize incoming information, reducing the risk of missing important events. in this paper, we propose a novel method to identify relevant financial news. the key insight is that this can be achieved without relying on manually created relevance judgments, instead leveraging the correlation of news events and stock prices. the core idea is to train an attention-based neural network on the stock prediction task, using the price movement as label. the input of the network is a set of events in the form of news headlines (embedded using bert @xcite ), and the output is the price movement of a specific stock index with respect to the previous day (i.e., down, * equal contribution stay, up), mediated by an attention layer @xcite . the layer acts as an input selector, computing the weight for each headline on a given day. these weights are repurposed to score and thus rank news headlines according to financial relevance. as each weight solely depends on the headline itself, we can use it to compare headlines across the entire dataset. we evaluated our method on the most prominent us stock indices (s&p500, dow jones, russell 1000, and nasdaq) and 1.5 million headlines @xcite @xcite @xcite @xcite @xcite @xcite @xcite @xcite @xcite from gigaword @xcite . a first automatic evaluation confirmed a positive correlation between stock prediction accuracy and relevance scores (via the attention weight). in a second, manual evaluation, we labeled 1000 headlines and found that the method ranks relevant events highly: a network trained on the dow jones stock index prices, for example, resulted in 89% relevant events in the top 200 ranks, compared to only 19% relevant events in a uniform sample of headlines. ## training the attention layer for event scoring the idea is to make use of universal key attention mechanism as in @xcite to learn a headline relevance weight by predicting the stock price movement. once the network is trained we can use the unnormalized weights of the attention layer as a global relevance score for the news headlines. as input we use all daily headlines and their categories. the output is down, stay, up with respect to the next trading session open price. the full network is displayed in figure 1 . each headline hl 1 , . . . , hl k consisting of a (padded) sequence of n tokens {w i } @xmath0,...,n , is encoded into vectors {hhl i } @xmath1,...,n of length 768 via the bert-base-uncased model pooled output: each headline comes with a category label hc 1 , hc 2 , . . . (details in section 4) embedded into a randomly initialized vector of length 30 and projected to a vector h p i of length 100 by a fully connected feed forward with elu activation following @xcite , an attention layer computes normalized weights for each headline in the input day via a universal key, h p d := {h p i } @xmath2,...,k , and aggregates them according to those weights. the final label l i (down, stay, up) is computed using a feed forward layer with softmax activation every input layer is normalized, and weights are initialized using he. the dropout rate is 0.25. all weights are fine-tuned on the task. we ran on 3 tesla v100 with a total batch size of 15. the model has a total of ∼110m parameters. ## evaluation ## conclusion and future work we presented an exploratory analysis to rank financially relevant events without manually labeled data. we showed that when a simple neural network is able to extract informative signals from news, the attention layer was able to score higher the most relevant news. future work needs to focus on a more fine-grained analysis of the data and understanding the stability of the scores.