News-Driven Stock Prediction Using Noisy Equity State Representation
Authors: Xiao Liu
Heyan Huang
Yue Zhang
Venue:  (2021)
Keywords: news-driven stock prediction
equity state representation
recurrent state transition

Abstract: News-driven stock prediction investigates the correlation between news events and stock price movements.
Previous work has considered effective ways for representing news events and their sequences, but rarely exploited the representation of underlying equity states.
We address this issue by making use of a recurrent neural network to represent an equity state transition sequence, integrating news representation using contextualized embeddings as inputs to the state transition mechanism.
Thanks to the separation of news and equity representations, our model can accommodate additional input factors.
We design a novel random noise factor for modeling influencing factors beyond news events, and a future event factor to address the delay of news information (e.g., insider trading).
Results show that the proposed model outperforms strong baselines in the literature.

Under review as a conference paper at ICLR 2021
NEWS-DRIVEN STOCK PREDICTION USING NOISY EQUITY STATE REPRESENTATION
Anonymous authors Paper under double-blind review
ABSTRACT
News-driven stock prediction investigates the correlation between news events and stock price movements. Previous work has considered effective ways for repre- senting news events and their sequences, but rarely exploited the representation of underlying equity states. We address this issue by making use of a recurrent neural network to represent an equity state transition sequence, integrating news repre- sentation using contextualized embeddings as inputs to the state transition mech- anism. Thanks to the separation of news and equity representations, our model can accommodate additional input factors. We design a novel random noise factor for modeling influencing factors beyond news events, and a future event factor to address the delay of news information (e.g., insider trading). Results show that the proposed model outperforms strong baselines the literature.
INTRODUCTION
Stock movement prediction (Ding et al. 2014 2015) is central task in computational quantitative finance. With recent advances in deep learning and natural language processing (NLP), event-driven stock prediction has received increasing research attention (Xie et al. 2013 Ding et al. 2015). The goal is to predict the movement of stock prices according financial news. Previous work adopts relatively simple model on the stock movement process, casting price change as response a set of news. The prediction model can therefore be viewed as variation of a classifier that takes news as input and yields movement direction output. Investigations have focused on news representation, where bag-of-wordd (Kogan et al. (2009), named entities (Schumaker & Chen 2009). event structures (Ding et al. 2014) or neural representation features (Ding et al. 2015 Xu & Cohen 2018 are considered.
Intuitively, news events carry information on important changes of company management, market, revenue and other factors, which can affect the fundamental values of equities, and thereby can consequently impact the stock price, shown in Figure Properly representing news events is key to modeling such impact on the market. However, the stock market movement can also be influenced by accumulated effects of fundamental changes over time, the overall market sentiment, and other volatile factors, which can be considered as noise to analytical models. These factors have been relatively less modeled by existing work on event-driven stock prediction. For example, although there has been work modeling long-term event impacts by representing event sequences (Ding et al., 2015), little work has considered representing fundamental values directly.
To address these issues, we consider representing the equity state directly using a recurrent neural network over time and propose the stock movement prediction network using Noisy Equity State representation (NES). At each time step. the equity state reflects the current stock price trend, and can used directly for predicting the next movement. The advantage of separating news repre- sentation from equity state representation is that factors beyond news can be modeled as additional input in the recurrent state transition process. Although such factors can be calculated using external tools such sentiment classification over tweet data, we simply treat them as random noise factor. The reason two-fold. First, in practice, noise is inevitable in stock prediction and no single math- ematical model can perfectly fit the stock price movement distribution. Second, for fair comparison with existing work on news-driven stock prediction, no additional input should be used on top of standard benchmark input settings.

---

Under review as a conference paper at ICLR 2021
Stock Price Volatility (%)
News Impact
posts Nigher quarterly earnings.
3M selis drug unit.
3M ta buy Braziian orthodantic products muker.
1.5
stashes 800 jobs
0.5
200///8/21
200//10/21
-0.
2005/10/24
2006/11/0L
-1.
2008/12/30
2008/12/15
Figure I: The stock price volatility and the news impact 3M Company. Over the first and the second periods, there was only one event. And in the third period, there were two events affecting the stock price movements simultancously.
The input to each recurrent equity state transition consists of a news factor and noise factor. The news factor consists of three components, namely past news within thrity days, present news within trading day, and future news within seven trading days. We use real historical news for the future news component for training, and predicted future news according to the current equity state during testing. The motivation for modeling future news to address the negative effect of delayed news release and insider trading on the prediction accuracy. We represent each news event using contex- tualized representation
the news title, and aggregate news representations by using the current equity state query to conduct attention. The noise factor is integrated into the model by using the current equity state to derive a normal noise distribution specific to the trading day, and then sampling noise vector.
Compared with existing work, our model has three main potential advantages. First, the relative importance of news be easily visualized using attention. Second, insider trading effect is plicitly handled. Third, noise effect is integrated into the model. All three benefit result directly from the direct representation of equity state. Experiments over the benchmark of Ding et al. (2015) show that our method outperforms strong baselines, giving the best reported results in the literature. To our knowledge, we are the first explicitly model both events and fundamental equity states for news-driven stock movement prediction, and the first to consider noise factors using neural random sample module.
2 RELATED WORK
Modeling Price Movements Correlation Most existing work on event-driven stock prediction treats the representation of news events independently using bag-of-words (Kogan al. 2009).
named entities (Schumaker & Chen 2009), semantie frames (Xie et al.] 2013), event structures
DDig et al. 2014), event embeddings (Ding et al. 2015) knowledge bases (Ding et al.) 2016).
In contrast, work on time-series based stock prediction (Levine Zervos 1996] Amihud 22022
& Cohen, 2018 Zhang al.) 2018a) aims to capture continuous movements of prices themselves.
There has also been work modeling the correlations between samples by sparse matrix factorization
(Wong et al. 2014), hidden Markov model (Zhang al.,, 201 and Bi-RNNs (Xu & Cohen 2018
Yang et al. 2019) using both news and historical price data. Some work models the correlations among different stocks by pre-defined correlation graph (Peng & Jiang, 2016) and tensor factoriza- tion (Zhang et al. 2018b). Our work is different in that we use only news events as inputs, and our recurrent states are additionally designed accommodate noise.
Explainable Prediction Rationalization is an important problem for news-driven stock price move- ment prediction, which is find the most important news event along with the model's 's prediction. Factorization, such as sparse matrix factorization (Wong et al. (0144) and tensor factorization (Zhang) ct al. 2018b), is a popular method where results can be traced back upon the input features. Our attention-based module achieves similar goal yet has linear time complexity on feature size. Chang et al. (0016) use attention for characterizing the influence of individual news within a trading day for predicting the cumulative abnormal return in three-day window. Yang et al. (2019) apply dual- layer attention to predict the stock movement by using news published in the previous six days. Each day's news embeddings and seven days' embeddings are summed by the layer. Our work is
2

---

Under review as a conference paper at ICLR 2021
Current Prediction
LSTM-based Recurrent State Transition
+
Previous
Modeling Noise
Current
State
State
Modeling News
Modeling Future
News
News
News
Figure 2:
The model framework NES representing a trading day the trading sequence. The solid lines used both in the training and the evaluating procedures, while the dotted line is only used in the training procedure.
different from Yang et al. (2019) in that our news events attention is query-based, which more strongly related noisy recurrent states. In contrast, their attention is not query-based and tends to output the same result for each day even if the previous day's decision changed.
METHOD
Following previous work (Ding et al. 2014 2013). the task is defined as a binary classification task. Formally, given trading day x, the input history news set about targeted stock index and the output is a label E {+1, - indicating whether the adjusted closing price") Py will be greater than P=-1 (y +1) not (y -1).
The structure of our model is shown in Figure 2 The equity state is modeled using LSTM (Hochre- iter & Schmidhuber 1997). which serves as basis for making prediction. News events are sep- arately represented as inputs to the state transition process. For each trading day. we consider the news events in the day well as the past news events using neural attention (Vaswani et al. (017). Considering the impacts of insider trading, we also involve future news in the training procedure. To model the high stochasticity of stock markets, we sample an additive noise using a neural module.
Considering the general principle of sample independence, building temporal connections between individual trading days in training set can harm training (Xu Cohen 2018) and we find easy to overfit. To address this issue, extended the time span of one sample to T previous tinuous trading days (t - T + 1,t T 2.... t 1, t), which we call a trading sequence. Trading sequences used as the basic training elements in this paper.
3.1 LSTM-BASED RECURRENT STATE TRANSITION
Formally, we denote the equity state using z. In each trading day, the recurrent state transition can be written
2, = f(2)), = LSTM( 21-1)
(1)
where 9, represents the news event on the trading day and f is function for the noise effect. Non-linear compositional effeets of multiple events can also be captured in a time window because of the LSTM representation. To predict the current day stock price movement, we use the sequential state 2, to make binary classification
p softmax W 2t),
argmax pe(0e ilt) ie 11(+
where is the estimated probabilities for up (+1) and down (-1) movements, De the predicted label, W is the classification parameter and f is the input trading day.
https://yhooe it/3:e...t///11/

---

Under review as a conference paper at ICLR 2021
3.2 MODELING NEWS EVENTS
We separately represent long-term and short-term impact of news events for each day / in a trading sequence. For short-term impact, and before the trading day as the set of present news. Similarly, for long-term impact, we use news articles within thirty calendar days as the set of past news.
articles published after the previous trading day /
Each news article is simply regarded as a separate news event. We extract the headline and use V-dim hidden state by concatenating the bidirectional output hidden states of the first and the last words as the basic representation of news event. By stacking the vector representations of different news articles, we obtaín two embedding matrices Ca and B for the present and past news events, respectively,
ELMo (Peters et al., 2018) to transform
e= |ELM h) -1 (h)
(3)
C = le o 0 e B [ej Φ es e.... Beeil
(4)
where I, is the vector concatenation operation, + is the vector stacking operation, le is the size of present news set and 4, is the size of past news set.
To make the model more numerically stable and avoid overfitting, we apply the over-parameterized method of Merity| (2019) to the news event embedding matrices
C o (W @ tanh(W B, = oW B)) @ (Wnh
(5)
where is element-wise multiplication, (+) is the sigmoid function, and W/ and W model parameters.
Due to the unequal importance fact news events with regard to the stock price movement day f, we use scaled dot-product attention (Vaswani et al. (017)) to capture the influence of news C; and B, the current day stock movement. Formally, we first transform the last trading day's equity state zt-1 to a query vector q:, and then calculate two attention score vectors 7 and B, for the present and past news events, respectively as
vV C ) B. = softmax
B
qi = tanh Mann(W"' (W Y = softmax
(6)
VV
We sum the news event embedding matrices to obtain two final news representation vectors e and b. on the trading day t according to the weights 7/t and B., respectively, as
C2 tanh MC)) b, tanh B.B:)
3.3 MODELING FUTURE NEWS
We find that news events exert influence the stock price movement before being released, which can be attributed to news delay or insider trading (Seyhun 1992) factors. To this end, we propose novel future event module represent backward news influence. In this paper, we define future news events as those that are published within seven calendar days after the trading day t.
Similarly the past and present news events, stack the ELMo embeddings (Peters et 2018) of future news event headlines to an embedding matrix A and then adapt the over-parameterized method and sum the stacked embedding vectors by scaled dot dot-product attention. Formally, the future news events impact vector on the trading day t is calculated as
Agg: VV
A, (oW tanh W W'),,, a softmax
a, tanh Σ ofA:)
(8)
where 1,, is the size of future news set.
The above steps can be used to reduce overfitting in the training procedure, where the future event module is trained over gold "future" data over historical events. At test time, future events
4

---

Under review as a conference paper at ICLR 2021
not accessible. To address this issue, events impact vector a with the past and present news events impact vectors b, and C₁ as
non-linear transformation to estimate future news
a = tanh W" [c,, be])
where : is the vector concatenation operation and W is model parameter. We concatenate the mentinent three types of news event vectors at b, and C2 to obtain a final news event input 2t for the recurrent state transition on trading day t.
3.4 MODELING NOISE
All factors beyond imput news articles, such as sentiments, expectations and noise are explicitly modeled as noise using random factor. For each trading day, sample random factor from normal distribution (0, σx) parameterized by 2/ as
o, = /exp(ttah((W*2
(10)
In addition, to facilitate back-propagation in training, we use re-parameterivation for normal distri- butions (Srivastava & Sutton 2017), drawing sample random factor from parameter-free distri- bution to obtain the noisy recurrent state as
tanh(z; +,6)), 4 ~ 1))
(11)
3.5 TRAINING OBJECTIVE
For training, we consider two main terms for defining the loss function. The first term is a cross entropy loss for the stock movement probabilities P. and gold labels yr. and the second term is the mean squared error between the estimated future impact vector â, and the true future impact vector a, stated in Section 3.3 Formally, the loss for trading sequence containing T trading days with standard Le regularization is calculated as
Lee
log(1 (e(())()) Limac V
Σ
(a) ai2²
(12)
Liotal Loe +- Lmms + 1|
(13)
where 0 is a hyper- parameter which indicates the relative importance of Lmoe to Lees & is the set of trainable parameters the model and the regularization weight.
EXPERIMENTS
We use the public financial dataset released by Ding et al. (2014), which is crawled from Reuters and Bloomberg over the period from October 2006 to November 2013, and follow their method for splitting the dataset. Experiments are conducted on predicting the Standard & Poor's 500 stock (S&P 500) index and set of selected individual stocks, obtaining prices from Yahoo Finance 2
We report the final results after tuning paraaeparameters in the development experiments.
4.1 SETTINGS
The hyper-parameters are shown in Table 1 We use mini-batch and SGD with momentum to update the parameters. Most of the syper-parameters are chosen according to development experiments, while the dropout rate and SGD momentum μ set according to common values. Following previous work (Xie et al. 2013; Ding et al. 2014: Xu Cohen 2018), we adopt accuracy and Matthews Correlation Coefficient (MCC) evaluate model performances. MCC is applied because avoids bias due to data skew.
4.2 DEVELOPMENT EXPERIMENTS
4.2.1
INITIALIZING NOISY RECURRENT STATES
httts:://iinnneee httr:://Fiaance.yyaoo..omm com

---

Under review as a conference paper at ICLR 2021
Table 1: hyper-parameters setting.
Figure 3: Results of different T.
Name
Value
71
Accuracy/MCc (%)
Mins/Epoch
batch size
66
learning rate Ir
0,005
61
SGD momentum 1 dropout rate
0.9
56
0.3
MSE loss weight 0 regularization weight A
0.4
51
news embedding dimension V
0.0005
256
41
46
Accuracy
Time
MCC
5
recurrent state dimension D
100
36
1
5
9
13
15
Longth
As the first set of development experiments, compare dif- ferent ways to initialize the noisy equity states NES. For each trading day, we compare the results whether states tran- sitions are modeled or not as well as the initialization methods of the equity states. NES_Sin S_nn_Snng R is randomly initializing the states for each single trading day. NES Sing Z is initializing the states as zeros for each single trading day. NES_Seq R is randomly initializing the first states for each trading sequence only. NES NES_Seq Z is initializing the first states zeros for each trading sequence only.
Table 2: Results initializing the noisy equity states.
Accuracy
MCC
NES Luns_Sing_h R
62.91 %
0.3704
NES JES_Sing Z
63.63%
0.3672
NES BES_Seq.RR
67.94%
0.5141
NES_Seq.Z
68.51 %
0.5392
The random initialization method returns tensor filled with random numbers from the standard normal distribution (N0, 1). The results on the S&P 500 index are shown in Table We can see that modeling recurrent sequences (NES Seq R and NES See__Sq Z) gives better result than treating each trading day separately (NES Sing R and NES Sing Z), which shows that modeling equity states can capture long-term and non-linear compositional effects multiple events. From another perspec- tive, comparision with the models NES_Sing and NES_Sing Z also demonstrates the strengths of our basic representations of news events in isolation. In particular, we can also see that using only the basic news events representations is not sufficient for index prediction, while combination with the equity state transition can give better results. This consistent with previous reports on news- driven stock prediction (Ding et al., 2015). By comparing the results of NES Seq R and NES Seq_See-22 Z. we decide to use zero initialization in the remaining experiments.
4.2.2 STUDY ON TRADING SEQUENCE LENGTH
We use the development set to find a suitable length T for a trading sequence, searched from (1, 5, 7, 11, 13, 15). The S&P 500 index prediction results and time per training epoch on the development set are shown in Figure 3 We can see that the accuracy and MCC are positively correlated with 7' When T Z the growth of MCC becomes slower, while the running time per training epoch is nearly linear with respect to T. We therefore choose the prpee-pprameter T = for the remaining experiments.
4.3 MAIN RESULTS
We compare our approach with five strong baselines on predicting the S&P 500 in-
Table Test results on predicting S&P 500 index.
dex. Luss & D'Aspremont] (2015) use bag-
of-words to represent news documents and
Accuracy
MCC
adopt Support Vector Machines. Ding et al. (2015) use event embeddings as input and convolutional neural network to model a se- quence of events. Ding al. (2016) em- power event embeddings with knowledge bases like YAGO and also adopt convolu- tional neural networks as the event sequence model. dos Santos Pinheiro & Dras (2017)
Luss D' D'Aspremont (2015)
56.38%
0.0711
Ding el al.
2015)
64.21%
0.4035
Ding et al
2016)
66.93%
0.5072
dos Santos Pinheiro & Dras 2017
63.34%
et
64.55%
NES
67.34 % 0.5475

---

Under review as a conference paper at ICLR 2021
Table 5: Test set results of individual stock price movement prediction.
Company News
Sector News
All News
Stock
Sector
#docs
Accuracy
MCC
#does
Accuracy
MCC
Accuracy
MCC
Apple
2.398
69.21%
0.5632
12.812
64.35%
0.3861
56.14%
0.2355
Citigroup
Financials
2,058
63.57%
0.5193
117,659
56.29%
0.3021
55.15%
0.1852
Bocing Company
Industrials
1,870
66.25%
0,4423
17,969
61.35%
0.2719
57.23%
0.1824
Google
Communication
1,762
66.13%
0.3717
13.344
60.47%
0.2644
58.41%
0.1387
Wells Fargo
Financials
845
61.64%
0.3944
117,659 57.34%
0.1294
54.64%
0.0823
use fully connected model and character-level embedding with LSTM to encode news sequences. Lin et al. (017)) use recurrent neural networks with skip-thought vectors to represent news text.
Table 3 shows the test set results. All the models make prediction based on news input only. From the table we can see that NES achieves the best results on the test set, which demonstrates the advantage of separately representing equity state sequences and integrating noise features. By com- paring with Lush & Aspremont (2015), we can find that using news event embeddings and equity state representations can be more effective and also flexible when dealing with high-dimension fea- tures. When comparing with Ding et al.] (2015) and the knowledge-enhancee
method (Ding et al.] 2016), we find that modeling the correlations between trading days in trading sequences can bet- ter capture the compositional effects of multiple news events. In addition, by comparing with dos Santos Pinheiro & Dras (017) and [in al. (2017), also find that modeling the noise by using a state-related random factor is very beneficial and effective to meet the aim to deal with the high market stochasticity.
4.4 ABLATION STUDY ON NEWS AND NOISE
We explore the effects news events and random noise with ablation the test set. By ablating the past news, the present news, future news and the noise factor, respectively, the S&P 500 index prediction results of the ablated models are shown in Table[4] First, without using the past news events as input, the result becomes the lowest. This shows that history news
Table 4: Results of ablation study.
Accuracy
MCC
NES
67.34%
0.5475
Past News
62.17%
0.4421
articles contain the largest amount of relevant news events.
Present News
64.73%
0.4823
In addition, since we set the trading sequence length 7'
Future News
64.58%
0.4781
and we only use the news between two trading days as the
Noise
63.90%
0,4608
present news, those ablated news which was published before the date range will not be involved in our model, while the ablated present or past news will be a part the input on adjacent trading days.
Second, we observe that using future news events is more effective than using the present news events. On the one hand, confirms the importances to involve the future news in NES, which can deal with insider trading factors to some extent. On the other hand, the results also benefits from the recurrent state transition nature of the model, as the future news impact on the (t - 1)-th day can be carried forward to the t-th day in the equity state to compensate absence of the present news events.
The effect of the noise factor is lower only to modeling the past news events, but higher than the other ablated models, which demonstrates the effectiveness of the noise factor module. This shows the advantage of considering factors beyond news events. It also directly benefits from our repre- sentation of equity states, which gives basis for calculating noise distribution conditioned on accumulated history.
4.5 PREDICTING INDIVIDUAL STOCK MOVEMENTS
Other than predicting the S&P 500 index, we also investigate the effectiveness of our approach on individual stock prediction using the test set. We count the amounts of individual company related news events for each company by name matching, and select five well known companies from four different sectors with the most news, Apple, Citigroup, Boeing Company, Google and Wells Fargo. The sectors are decided by the Global Industry Classification Standard. For each company,
7

---

Under review as a conference paper at ICLR 2021
Date
Events
2013-05-17 17
Apple Joins Facebook, Microsoft in Outining Cata Requests
0.04
0:652
2013-06-21
Apple Suit Against Samsung Japan
0.04
2013-05-24
Apple Falls Below $400 Amid iPhone Slump, Worker Exits
04344
0.033
0876
6:003
O1B4
2013-05-25
Samsungg Beats Apple japanese Patent Suit on Synding
2013-07-02
Apple Nevado solar farm cleon energy push for data
5014
6.779
0.03
0
0134
2013-07-10
Applo Faces Damages Tril Over B- Boook Antitrast vialstion
200
5.00
2004
004
2013-07-17
Apple May Delay Introduction [Phone
Times Says
2013-07-19
Apple Said to Buy HopStop, Pushing Deepar into Maps
0C41
000
2013-07-21
High End Smartphone Boom Ending Price Drop Hits Apple
2013-07-22
Apple Developer Website Taken Down After Hecker Attack
2014
Gold
1
1
4
Pred
11
1
Figure 4: Attention visualization and test set results comparison of the trading sequence (07/15/2013,
07/23/2013] when predicting Apple Inc.'s stock price movements using only company news.
prepare not only news events about itself, but also news events about the whole sector. We use company news, sector news and all financial news to predict individual stock price movements, respectively. The experimental results and news statistics are listed in Table
Individual stock prediction by only using company news dramatically outperforms that using sector news and all news, which demonstrates negative correlation between total used amounts of news events and model performance. The main reason may be that company-relace more directly affect the volatility of company shares, while sector news and all news contain many irrelevant news events, which obstruct NES's s learning the underlaying stock price movement trends.
news events can
4.6
CASE STUDY
To look into what news event contributes the most to our prediction result, we further analyze the test set results of the model for Apple Inc. 's stock price movements only using company news, which achieves the best results among the five selected companies mentioned before. As shown in Figure we take the trading sequence from 07/15/2013 to 07/23/2013 for illustration. The table on the left shows the top-ten news events. while attention visualization and results are shown on the right chart. The news events listed on the left table are ranked by the attention scores from the past news events, which are the most effective news according to the ablation study. There are some zeros in the attention heat map because these news do not belong to the corresponding trading days.
We can find that the News Event 1 has been correlated with the stock price rises on 07/15/2013, but for the next two trading days, its impact fades out. On 07/18/2011 8/2013, the News Event 7 begins to show its impact. However, NES pays more attention to it compared with other events, which leads to the incorrect prediction that the stock price decreases. On the next trading day, our model infers that the impact of the News Event 2 is bigger than that of the News Event 7, which leads to an incorrect prediction again. From these findings, we can that NES tends to pay more attention new event when it first occurs, which offers us a potential improving direction in the future.
CONCLUSION
We investigated explicit modeling of equity state sequences in news-driven stock prediction by us- ing an LSTM to model the recurrent state transition, adding news impact and noise impact by using attention and noise sampling, respectively. Results show that our method is highly effective, giv- standard benchmark for stock index movement prediction. To our
ing the best performance knowledge, we are the first to explicitly model both events and noise factors for news-driven stock movement prediction.
8

---

Under review as a conference paper at ICLR 2021
REFERENCES
Yakov Amihud. Illiquidity and stock returns: cross-section and time-series effects. Journal of Financial Markets, 5(1):31-56, 2002. ISSN 1386-4181.
Ching- Yun Chang, Yue Zhang, Zhiyang Teng, Zahn Bozanic, and Bin Ke. Measuring the informa- tion content of financial news. In Proceedings of the 26th International Conference on Computa- fional Linguistics, pp. 3216-3225, 2016.
Xiao Ding, Yue Zhang, Ting Liu, and Junwen Duan. Using structured events to predict stock price movement: An empirical investigation. In Proceedings of the 2014 Conference Empirical Methods in Natural Language Processing, pp. 1415-1425, 2014.
Xiao Ding, Yue Zhang, Ting Liu, and Junwen Duan. Deep learning for event-driven stock prediction. In Proceedings of the 24th International Joint Conference on Artificial Intelligence, pp. 2327- 2333, 2015.
Xiao Ding, Yue Zhang, Ting Liu, and Junwen Duan. Knowledge-driven event embedding for stock prediction. In Proceedings of the 26th International Conference on Computational Linguistics, pp. 2133-2142, 2016.
Leonardo dos Santos Pinheiro and Mark Dras. Stock market prediction with deep learning: A Character-based neural language model for event-based trading. In Proceedings of the Aus- tralasian Language Technology Association Workshop. pp. 15, 2017.
Sepp Hochreiter and Jürgen Schmidhuber. Long short-term memory. Neural Computation, 9(8):
1735-1780, 1997.
Shimon Kogan, Dimitry Levin, Bryan R. Routledge, Jacob S. Sagi, and Noah A. Smith. Predicting risk from financial reports with regression. In Proceedings of the 2009 Conference of the North American Chapter of the Association of Computational Linguistics: Human Language Technolo- gies, pp. 272-280, 2009.
Ross Levine and Sara Zervos. Stock market development and long-run growth. The World Bank Economic Review, 10(2):323-339, 1996. ISSN 0258-6770.
Peikang Lin, Xianjie Mo, Guidong Lin, Liwen Ling, Tingting Wei, and Wei Luo. A news-driven current neural network for market volatility prediction. In 4th IAPR Asian Conference on Pattern Recognition, pp. 776-781, 2017.
Ronny Luss and Alexandre D' Aspremont. Predicting abnormal returns from news using text classi- fication. Quantitative Finance, 15(6):999-102. 2015.
Stephen Merity. Single headed attention RNN: stop thinking with your head. CoRR, abs/1911.11422,, 2019.
Yangtuo Peng and Hui Jiang. Leverage financial news to predict stock price movements using word embeddings and deep neural networks. Proceedings of the 2016 Conference of the North Amer- ican Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 374 379, 2016.
Matthew E. Peters, Mark Neumann, Mohit lyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. Deep contextualized word representations. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 2227-2233 2018.
Robert P. Schumaker and Hsinchun Chen. Textual analysis of stock market prediction using breaking financial news: The azfin text system. ACM Transactions on Information Systems, 27:2):12:1 12:19, 2009.
H. Nejat Seyhun. Why does aggregate insider trading predict future stock returns. The Quarterly Journal of Economics, 107(4):103--1331 1992. ISSN 00335533, 15314650.

---

Under review as a conference paper at ICLR 2021
Akash Srivastava and Charles A. Sutton. Autoencoding variational inference for topic models. In Proceedings of the Sth International Conference on Learning Representations. 2017.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Advances Neural Infor- mation Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, pp. 5998-6008, 2017.
Felix Ming Fai Wong. Zhenming Liu, and Mung Chiang. Stock market prediction from WSJ: text mining via sparse matrix factorization. In Proceedings of the 2014 IEEE International Conference on Data Mining, pp. 430-439, 2014.
Boyi Xie, Rebecca J. Passonneau, Leon Wu, and Germán Creamer. Semantic frames to predict stock price movement. In Proceedings the st Anmual Meeting of the Association for Computational Linguistics, pp. 873-883, 2013.
Yumo Xu and Shay B. Cohen. Stock movement prediction from tweets and historical prices. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguisties, pp- 1979- 2018.
Linyi Yang, Zheng Zhang, Su Xiong, Lirui Wei, James Ng. Lina Xu, and Ruihai Dong. Explainable text-driven neural network for stock prediction. CoRR, abs/1902.04994, 2019.
Xi Zhang, Yixuan Li, Senzhang Wang, Binxing Fang, and Philip S. Yu. Enhancing stock market prediction with extended coupled hidden markov model over multi-sourced data. Knowledge and Information Systems, 1-20, 2018a.
Xi Zhang, Yunjia Zhang, Senzhang Wang, Yuntao Yao, Binxing Fang, and Philip S. Yu. Improving stock market prediction via heterogeneous information fusion. Knowledge-Baud Based Systems, 143: 201 8b.
10


Reviews:
[{"review_id": "cfdWYXainHb", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "- Summary: A stock return prediction algorithm using news events is proposed. The authors claim that separately representing the news and equity states is a novel part of this paper. Also, the random noise factor and future event factors are proposed as novel modules.\n- Quality: The paper is somewhat unclearly written and the performance improvement compared with conventional methods is marginal. The novelty is marginal or the authors should explain the novel contents in detail. \n- Clarity: The paper contains some unclear explanations and inappropriate notations but could be improved.\n  - Regarding notation, the authors need to be more strict. \n  - Why the trading day is denoted by $x$ in Sec. 3 and denoted by $t$ in other text? It should be consistent. \n   - Eq. (2). Why there is an 'i' on the upper side of argmax? If it is a typo, then please fix it.\n   - Eq. (2). Is the t in the right hand side is necessary? (i.e. the $| t$ in the $\\hat{p_t}(\\hat{y_t} = i | t)$).\n   - In the Equation (2) and (3), the $\\overleftarrow{}$ and $\\overrightarrow{}$ seems redundant representation. The authors could simplify the notation.\n  - What is $h$ in Eq (3)? I can only guess but the authors should clarify. \n   - There is no explicit definition of the equity state $z$. I guess that most of the ML people would easily figure out that it is an implicit state handled by LSTM, however, the people in the finance field may be confused that it could be an explanatory state (factor) such as momentum, volatility, or mean reversion. Also, the original news events are represented in natural language but it is represented as a news event vector $v_t$ in this paper. I suggest that the authors add an explicit explanation of the definition for a more self-contained paper.\n   - Through subsections 3.2, 3.3, and 3.4, the submodules are explained with many notations. But it is difficult to figure out what the inputs and outputs are. I suggest the authors modify Figure 2 to include associated variables.\n  - Equation (3)~(8) can be rearranged by operations since the past, present, and future news are computed in a similar way. \n   - Sec. 3.2 The long-term articles may include short-term articles, and it seems redundant. Why not merge them into one module? (i.e. weighting scheme based on recent time). \n  - Is the $v_t=[a_t, b_t, c_t]$ for training and $v_t=[\\hat{a}_t, b_t, c_t]$ for test? It should be clarified in Sec. 3.3. \n- Originality: I barely see originality in this paper.\n- Significance: I barely see significant modules proposed in this paper.\n- Pros\n- Cons\n   - Random noise factor. \n      - One of the two novel features in this paper is the random noise factor. The authors claimed that it could model unknown factors such as sentiments, however, what I understand is that it is a simple data augmentation function. If it is an important module, the authors could assign more pages to explain and showing experimental results associated with the module.\n   - Future news\n    - What does it mean by predicting future news $\\hat{a}_t$ using the past news $c_t$ and present news $b_t$ in Equation (9)? I don't make sense to me that future news can be predicted. \n     - Throughout the paper, it should be very careful and clear about explaining future news. It may confuse the readers that the algorithm employs future data even in the test. Only careful readers would find out the \"At test time, future news events are not accessible\". It should be emphasized.\n  - Ablation study\n     - Excluding only one module significantly drops the predicted performance and could not improve than Ding et al.(2016). It means that the surviving modules may not be novel and effective in terms of performance. \n- Experiments\n   - I recommend the authors try other evaluation metrics such as the Sharpe ratio, to evaluate the realistic wealth curve. (i.e. if the model makes the right prediction for small movement and makes the wrong prediction for large price movement, then the wealth will eventually be reduced).\n   - Ablation study is also performed properly.", "extra_scores": {}, "extra_text": {}}, {"review_id": "XoOODuWQsWc", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Summary:\n \nThis work extends the line of research for the problem of news-driven stock price movement prediction. In doing so,  the authors presents a Noisy Equity State (NES) representation model, \nwhere they explicitly model equity state sequences by a recurrent state transition in an LSTM model. The proposed modeling approach also incorporates news impact and \nnoise impact employing attention and noise sampling techniques. \n\nThe proposed modeling approach offer the following advantages:\n- incorporating additional inputs, such as past and present news, in equity state transition process\n- addressing the effects of delayed news/insider trading and unknown factors by incorporating future news and random noise\n- offers to filter and visualize the critical news based on the accumulated equity knowledge\n \n\nStrengths:\n- the paper is well motivated and clear\n- extends news-driven stock price movement prediction task by introducing an LSMT based model, explicitly modeling equity state sequences and introducing additional signals/information. Also, recurrent equity representation based attention scores allow to filter out the non-critical news events.\n- sound evaluation and analysis, including ablation and case studies \n- extensive analysis to show the importance of past, present, future news and noise\n- reports gains with the proposed approach \n\n\nWeaknesses:\n- the paper lacks novelty in terms of methodological contribution and is incremental. The work combines existing approaches and introduces additional signal/information  \n- unclear experimental setup of baselines: Do all the baselines models also use the same (additional) information as the proposed approach does.\n- Table 3: marginal gains in accuracy (Compared to Ding et al 2016). Does the baseline Ding et al 2016 also use the same set of information for a fair comparison?  \n- For a fair comparison, the baselines (particularly Ding 2016) be trained on the same event embeddings as the proposed approach does, instead of using knowledge base embeddings. Additional investigation for fair comparison is required.\n\nQuestions: \n- Why not BERT-based contextualized embeddings employed instead of ELMo?\n- Table 3: Are the scores reported  after re-run of the baselines ?", "extra_scores": {}, "extra_text": {}}, {"review_id": "4UXwQZdrT3r", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "In this paper, the authors propose a model for integrating news representations for stock predictions. The authors claim that their proposed model outperforms the previous baselines. \n\nHere are my comments: \n\n1. Presentation needs improvement. Some statements are difficult to follow, or more explanations are needed. For example, \"little work has considered representing fundamental values directly\". What does \"fundamental values\" mean? This phase only occurs once in the introduction part. The authors do not elaborate on how to model fundamental values in their method. The benefits of modeling fundamental values should be illustrated.\n\n2. Following the above point, I do not see a clear motivation for this work. Also, I do not find the challenges of the problem addressed in this paper. Without knowing this, it is difficult to evaluate the contributions of this paper.\n\n3. For the framework in Figure 2, I like the modeling Future component. This design is interesting and reasonable. However, I do not understand the modeling noise component. I fail to find its rationality in the paper. Figure 2 is the key to this paper. There is no description of that. The second paragraph of Section 3 unfortunately is not its description.\n\n4. I lost the connection between Section 3.5 and Section 3.1 - 3.4. Since there are many variables, a notation table is suggested.\n \n5. Stock prediction is a very interesting topic. However, I find the benchmark dataset used in this paper is out-of-the-date. This is an application-oriented paper, which should demonstrate its practicability. It is encouraged to see the performance of the proposed method on the recent stock prediction. \n", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: In this paper, the authors propose a model for integrating news representations for stock predictions. While the research direction has good value in real applications, it seems that this particular paper has not done a sufficiently good job in pushing the frontier of this direction. The reviewers have raised quite a few concerns, for example:
1)	Paper writing needs significant improvement (e.g., confusion regarding future news).
2)	Limited technical novelty as compared to previous works
3)	Benchmark datasets are out of date, baselines are a little weak and not well explained, more evaluation measures (Such as Sharpe value) are needed
4)	Marginal improvements over the baselines

The authors have not submitted their rebuttals. Therefore the concerns are still there and we do not think the paper is ready for publication at ICLR.

Decision: Reject