Causality-Guided Multi-Memory Interaction Network for Multivariate Stock Price Movement Prediction
Year: 2023

Abstract:
Over the past few years, we’ve witnessed an enormous interest in stock price movement prediction using AI techniques. In recent literature, auxiliary data has been used to improve prediction accuracy, such as textual news. When predicting a particular stock, we assume that information from other stocks should also be utilized as auxiliary data to enhance performance. In this paper, we propose the Causality-guided Multi-memory Interaction Network (CMIN), a novel end-to-end deep neural network for stock movement prediction which, for the first time, models the multi-modality between financial text data and causality-enhanced stock correlations to achieve higher prediction accuracy. CMIN transforms the basic attention mechanism into Causal Attention by calculating transfer entropy between multivariate stocks in order to avoid attention on spurious correlations. Furthermore, we introduce a fusion mechanism to model the multi-directional interactions through which CMIN learns not only the self-influence but also the interactive influence in information flows representing the interrelationship between text and stock correlations. The effectiveness of the proposed approach is demonstrated by experiments on three real-world datasets collected from the U.S. and Chinese markets, where CMIN outperforms existing models to establish a new state-of-the-art prediction accuracy.

heading: 1 Introduction
text: Financial services, known for their competitiveness, have always been at the forefront of adopting data science techniques to drive investment decisions. Quantitative trading, a specific field within it, has drawn immense interest from both academia and industry over the last few decades. With the rapid advancements in deep learning recently, computer scientists and quantitative researchers have
*Equal contribution. †Corresponding author: Rui Yan (ruiyan@ruc.edu.cn).
joined forces to apply AI techniques to tackle the challenges within this domain.
Among various tasks, one of the most prominent is stock price movement prediction (Bhardwaj, 2021). The reason for its popularity is selfevident: once a model is able to predict future movement with considerable accuracy, numerous trading strategies can be easily built around it.
Recent studies have shown that deep neural networks are ideal candidates for such prediction models (Yoo et al., 2021; Gunduz, 2021). Supporters of the efficient-market hypothesis (EMH), which posits that asset prices reflect all available information, tackle the task with price information alone (Zhang et al., 2017; Stoean et al., 2019; Sezer and Özbayoglu, 2020). However, an alternative perspective suggests that additional insights can be gained from analyzing news articles and social media posts, which may hold valuable clues about the future (Hu et al., 2018; Xu and Cohen, 2018; Wang et al., 2019b; Tang et al., 2020).
Another intriguing approach analyzes the relationships between different stocks. Clearly positive and negative correlations, or even non-correlations can be immensely useful in constructing a diversified stock portfolio (Borodin et al., 2003). Several studies even empirically demonstrate that exploiting correlations can improve the accuracy of stock price movement prediction (Long et al., 2020; Yoo et al., 2021). However, their correlations are often realized by acquiring industry sector and calculating correlation matrices or attention scores, which are bidirectional and symmetrical, leading to excessive attention on spurious correlations. Due to the lag problem widely existed between two time series, we are more concerned about the dominance of information flow between stocks, specifically, the direction of causality.
Additionally, we have observed that the situation can significantly change when incorporating text information. Let’s consider two highly correlated
12164
companies (A and B) and there is promising news specifically about company A. In such a scenario, it’s fairly easy to infer that the current news might still have a substantial impact on company B, despite there being no direct connection between the two companies on paper. However, it’s impossible to reach this conclusion by just examining the news about company A or the correlation between A and B alone, which highlights the limitations of relying solely on individual pieces of textual information or traditional correlations between stocks.
Inspired by observations above, we propose the Causality-guided Multi-memory Interaction Network (CMIN), a novel end-to-end deep neural network which captures both financial news as well as the causality-enhanced correlations between stocks for better stock price movement prediction.
To achieve this goal, CMIN incorporates two key components: the Text Memory Network and the Stock Correlation Memory Network. Both networks utilize a recurrent neural network with nonlinear combination of memory attentions to generate a global memory abstraction. And we introduce a global causality matrix according to the transfer entropy between stock price time series to guide the abstraction process, forming a Causal Attention mechanism to capture the asymmetric correlations. By considering causality, CMIN goes beyond traditional symmetric correlations and captures the true inter-dependencies between stocks. Furthermore, we employ an attention-based fusion mechanism between the two networks, introducing multi-directional interactions through which CMIN learns not only the self-influence within each network but also the interactive influence between them. It captures the interrelationship between textual information and correlations, enhancing the overall predictive power of CMIN.
We further demonstrate the effectiveness of CMIN with experiments conducted on 3 real-world datasets collected from both the U.S. and Chinese markets, where CMIN achieves state-of-the-art prediction accuracy, surpassing existing models in terms of performance.
To summarize, our main contributions are:
• Proposal of a causality-guided multi-memory interaction network for stock movement prediction which is to our best knowledge the first attempt to simultaneously consider causalityenhanced correlations and textual information to achieve higher prediction accuracy;
• Introduction of the attention-based multidirectional interactions, so that CMIN captures not only the self-influence of temporal movements and textual information but also the interactions between these two types of information flows;
• Collection and release of two new datasets: one for the U.S. market and another for the Chinese market. Both datasets include comprehensive financial texts and stock price time series data, which are publicly available at https://github.com/ BigRoddy/CMIN-Dataset, facilitating further research and benchmarking in the field.

heading: 2 Related Work

heading: 2.1 Stock Movement Prediction
text: In traditional trading practices, two main frameworks are commonly used to make predictions on future stock prices (Ferreira et al., 2021). The first is fundamental analysis, which aims to assess the intrinsic value of a stock by considering various factors related to it as a whole, such as financial statements, industry trends and economic conditions. The other is technical analysis, which operates under the assumption that the market is efficient (i.e., the Efficient Market Hypothesis holds true) and focuses on analyzing only historical and current price patterns in order to predict future movements.
Although both frameworks have been widely adopted by top hedge funds and investment firms, technical analysis has gained more popularity among AI practitioners, many of whom focus on employing long short-term memory networks and other innovative architectures to model stock price history alongside technical analysis indicators (Nelson et al., 2017; Zhang et al., 2017; Stoean et al., 2019; Sezer and Özbayoglu, 2020). This is primarily because processing a single stream of price data is relatively simpler than analyzing and synthesizing a range of diverse data sources with varying frequencies and characteristics.

heading: 2.2 Predicting with the Help of Text Data
text: The recent advancement of natural language processing (NLP) techniques has opened up new possibilities for analyzing large volumes of text data in the context of stock movement prediction. Many researchers have recognized the potential value of incorporating news articles, analysis, commentaries
and even social media posts (Xu and Cohen, 2018), which are believed to provide valuable insights about the future. Some studies focus solely on textual information. For example, (Hu et al., 2018) leverages attention mechanism at multiple levels within a deep structure to identify the most important news articles and predict price trends. Others adopt a two-step approach. First, they extract features (e.g. investor sentiment) from financial texts. Then they fuse these features with price information to make predictions such as (Li et al., 2017) and (Jin et al., 2020). This integration of text analysis with quantitative techniques holds promise for enhancing the accuracy and effectiveness of stock movement prediction models.

heading: 2.3 Exploiting the Relations between Stocks
text: Another important trading framework takes advantage of the correlations between different stocks. Portfolio selection, particularly pairs trading, is a well-known and successful trading strategy that exploits the correlated nature of stocks, whether positive or negative. In fact, as early as (Borodin et al., 2003) pointed out that stock correlations based portfolio selection could beat any strategy that relied on predicting trends or specific targets.
The incorporation of correlations in stock movement prediction has gained attention in recent years, drawing inspiration from several existing works. For example, (Yoo et al., 2021) utilizes transformer to learn dynamic correlations between stocks in an end-to-end manner. (Long et al., 2020) employs knowledge graphs and graph embedding techniques to model the relationships between stocks. These studies have achieved admirable results, potentially due to effective feature engineering however, because the direct benefit of stock correlations in predicting future prices lacks fundamental logic.
In this paper, we propose constructing a single model to handle both textual data and stock correlations simultaneously, aiming to shed light on the success of correlation-based approaches with the help of financial texts. We also introduce a novel causal attention mechanism to interpret the underlying logic behind stock correlations, leveraging transfer entropy to provide insights. We further model the multi-directional interactions between texts and correlations so that we could uncover not only relevant texts for prediction through correlations, but also the hidden stock correlations through texts. By integrating text data and stock correla-
tions within a unified model, we aim to provide a comprehensive understanding of the relationship between the two and discover valuable insights for stock movement prediction.

heading: 3 Problem Formulation
text: This paper is dedicated to predict the price movement of a target stock. To this end, we leverage both the correlations between stocks and textual information to make prediction.
Consider a target stock with numerical features denoted as Ptarget ∈ Rk×d, where k represents the number of time steps in the monitoring window and d represents the dimension of price features, such as the highest and the closing prices. The prices of n other relevant stocks are denoted as: P = {P1, P2, · · · , Pn} ∈ Rn×k×d.
Besides, we have financial documents associated with the target stock, which are represented as M = {M1,M2, · · · ,Mk} ∈ Rk×l×w, where l denotes the number of documents in a time step and w is the maximum number of words in a document. In cases where a specific stock has fewer than l documents at a given time step, zero padding values are added to align the lengths. Similarly, if a document contains fewer than w words, zero padding is applied to ensure uniform length across all documents (Ang and Lim, 2022).
We formulate the task as a binary classification problem whose goal is to predict the movement of the target stock at the next time step, denoted as ŷtarget. Here, ŷtarget = 1 indicates a rise in the price while ŷtarget = 0 indicates a fall.

heading: 4 Proposed Method

heading: 4.1 Model Overview
text: Figure 1 presents an overview of the Causalityguided Multi-Memory Interaction Network (CMIN). It is consisted of three main modules: feature embedding module, multi-memory networks and multi-directional interaction module.
(1) The feature embedding module includes two encoders, one for embedding the textual information and another for embedding the price time series. Additionally, a global causality matrix is introduced to capture the asymmetric correlations using transfer entropy, which then guides the calculation of attention weights in the multi-memory networks.
(2) The multi-memory networks consist of the Text Memory Network and Stock Correlation Memory Network, which are designed to select and re-
tain the most relevant and influential information (textual and correlational) for the target stock.
(3) The multi-directional interaction module facilitates the interaction between the textual and correlational information. This interaction allows the two types of information to reinforce each other and leverage the advantages of different information flows for better prediction performance, enhancing the predictive capabilities of the CMIN.

heading: 4.2 Feature Embedding
text: Self-attention mechanisms have proven to be effective in capturing long-term dependencies and modeling complex sequential patterns, particularly in the Transformer architecture (Vaswani et al., 2017). Given the significance of historical information in financial documents and stock prices for stock price movement prediction, we employ attention mechanisms to summarize this information.

heading: 4.2.1 Text Encoder
text: The Text Encoder focuses on processing the financial documents M to extract meaningful representations for stock movement prediction. We firstly use a popular word representation tool Glove (Li et al., 2018) to generate the word embedding tensor Mword ∈ Rk×l×w×dw , where dw is the size of word embeddings. Each word in the financial documents is represented as a dw-dimensional vector.
Then the word embeddings are passed through a text embedding layer. Here we adopt the bidirectional Gated Recurrent Unit (Bi-GRU) (Li et al., 2022) to capture both preceding and succeeding contexts within each document. The average of the last hidden vectors is taken as the text embeddings Mtext ∈ Rk×l×dm , or equivalently Mtext ∈ Rs×dm , where s is the total number of documents in the monitoring window.
After that, the text attention mechanism is applied to summarize all historical documents across time steps. The text embedding of the last time step Mtext,−1 ∈ Rl×dm , serves as the query matrix, while the entire text embeddings Mtext ∈ Rs×dm acts as both the key and value matrices. Soft scaled dot-product attention is used to compute the attention weights, which are then applied to the text embedding to obtain a representation Etext ∈ Rl×dm enhanced by the history state attention:
Etext = softmax( Mtext,−1MTtext√
dm )Mtext. (1)
The resulting Etext is the textual embedding that contains highly concentrated information from the stock’s related texts. This embedding serves as a summary of the historical text data and is used for further processing in the multi-memory networks and multi-directional interaction module of CMIN.

heading: 4.2.2 Price Encoder
text: The Price Encoder is introduced to utilize multivariate features from historical prices and capture their temporal interrelationships. Firstly we employ a feature mapping layer to project them into a latent space of dimension dp, aiming to improve the learning capacity (Yoo et al., 2021). For target stock price Ptarget ∈ Rk×d, the historical price embeddings P̃target ∈ Rk×dp can be formulated as:
P̃target = ReLU(PtargetWt + bt), (2)
where Wt ∈ Rd×dp , bt ∈ Rdp are parameters. Moreover, recognizing that historical patterns can repeat themselves sometimes, we incorporate a multi-head price attention layer to capture each stock’s distinctive changing patterns. The price embedding of the target stock at the last time step is donated as P̃−1target ∈ Rdp . Then we employ the multi-head attention mechanism with the query P̃−1target and the key/value P̃target as follows:
vtarget = MultiheadAtt(P̃target, P̃−1target) (3)
vtarget is a key vector that serves as the initial hidden state for the two memory networks, playing a crucial role in the final prediction. Similarly, we process the remaining stocks and obtain the correlational embedding Ecorr ∈ Rn×dp . Notably, the shared parameters across all stocks ensure the stability and generality of the extracted features (Wang et al., 2019a).

heading: 4.2.3 Causality Matrix
text: When it comes to detecting causal relationships and conducting predictive analysis, transfer entropy, a non-linear generalization of Granger causality (Seth, 2007), serves as a conceptually neat and mathematically rigorous method. It has been considered as an important tool for causality analysis and successfully applied in diverse domains including financial markets (Sandoval Junior et al., 2015).
Transfer entropy is derived from Shannon Entropy: H = −∑Ni=1 pi log pi. In this context, considering the time series of a stock, we can partition the possible values into different bins and calculate the probabilities at each time step. Transfer entropy from series X to another series Y can be defined as the average amount of information contained in the source X but not contained in Y’s past:
TEX→Y =H(Yfuture|Ypast) −H(Yfuture|Xpast, Ypast) (4)
Based on this principle, for each monitoring window, we calculate the transfer entropy between all stocks using their historical closing prices and generate a transfer entropy matrix, referred to as the Causality Matrix C ∈ Rn×n, which illustrates the asymmetric flow of information from one stock to another. Specifically, C[i, j] represents the transfer entropy from stock i to stock j, and C[i, j] > C[j, i] indicates that stock i provides more predictive information about the movement of stock j than j to i. This Causality Matrix will next serve as a guide for the memory networks, enabling the identification of causal dependencies between multivariate stocks.

heading: 4.3 Multi-memory Networks
text: We introduce a Text Memory Network and a Stock Correlation Memory Network (Sukhbaatar et al., 2015) to manage the textual and correlational information separately. They each maintain a continuous representation and update it iteratively using multiple computational steps (hops), ultimately producing a global memory abstraction.
As shown in Figure 1, each layer of the memory network comprises an attention unit and a GRU unit, which receive textual or correlational embeddings as inputs and are supervised by the continuous representation generated in the previous layer. To initialize the continuous representations of each network, we use the target stock vector vtarget (generated from Eq.3):
v (0) text = v (0) corr = vtarget. (5)

heading: 4.3.1 Text Memory Network
text: In each layer h ∈ [1, H] of the Text Memory Network, we input the textual embeddings Etext (Eq.1) and the continuous representation from the previous layer v(h−1)text . We utilize an attention unit (Eq.3) to identify important information within the textual embeddings. Subsequently, a non-linear GRU cell unit (Xu et al., 2019) acts as an information aggregator, determining the amount of text information to retain:
v Att(h) text = MultiheadAtt(Etext, v (h−1) text ), (6)
where v(h−1)text is the query matrix and Etext represents the raw form of the key and value matrices.
Then the GRU cell unit updates the current hidden state into the next hidden state and outputs it to the next layer as the new continuous representation:
v (h) text = GRU(v Att(h) text , v (h−1) text ). (7)

heading: 4.3.2 Stock Correlation Memory Network
text: The Stock Correlation Memory Network is employed to dynamically identify stock relationships and update the continuous representation of stock correlations in an intuitive and asymmetric manner.
However, the use of unsupervised attention weights in previous models can be problematic as they may be inevitably misled by the dataset bias, resulting in excessive attention on spurious stock correlations. To address this, we introduce extra knowledge in the form of Transfer Entropybased causality to guide the attention weights and mitigate potential confounding effects.
For each target stock, we extract a causal vector vcausal = C[:, target] from the pre-calculated causality matrix, which quantifies the degree of information flow from other stocks to it. Then we modify the traditional attention mechanism into Causal Attention by incorporating causal guidance:
S = softmax(QK T
√ d ), S̃ = f(S, vcausal). (8)
Here, f is a function that aggregates the attention weight S and the causal vector vcausal to produce a causality-guided attention weight S̃. We use the average aggregation method for simplicity (i.e., f(S, vcausal) = (S + vcausal)/2). To better balance them, one can introduce a hyperparameter λ ∈ [0, 1]. Then f() updates to f(S, vcausal) = λS + (1− λ)vcausal. We believe that different degrees of causal attention can impact the model’s performance, and leave it for future exploration.
The continuous representation is gradually updated through the Causal Attention, indicating the influence of causal relationships on movement prediction and the self-influence on the flow of correlation information:
v Att(h) corr = CausalAtt(Ecorr, v (h−1) corr ) (9)
v (h) corr = GRU(v Att(h) corr , v (h−1) corr ) (10)
It is important to note that although we design multiple layers within each memory network to learn deep representations, different layers of the same memory network share the same unit. This enables the network to focus on crucial information that affects the movement of the target stock, thereby enhancing the continuous representation.

heading: 4.4 Multi-directional Interactions
text: In reality, textual information and correlations have an impact on each other when it comes to stock
price movement prediction. For instance, news about a technological breakthrough in the new energy sector may uplift the prices of most stocks in that industry, thereby affecting the correlations among those stocks.
To simulate this phenomenon and enhance the synergy between textual and correlational information, we introduce a multi-directional interaction module. This module allows textual and correlational information to reinforce each other and amplify the advantages of different information flows for better prediction performance.
Take the Text Memory Network as an example, in each layer we firstly calculate the self-influence by using v(h−1)text as the query:
v Att(h) text−>text = MultiheadAtt(Etext, v (h−1) text ) (11)
Next we consider the interactive influences from correlations to texts using v(h−1)corr as the query:
v Att(h) corr−>text = MultiheadAtt(Etext, v (h−1) corr ) (12)
Finally, we produce a new attentional continuous representation by averaging these two influences:
v Att(h) text =
v Att(h) text−>text + v Att(h) corr−>text
2 , (13)
which means that we replace Eqs. 6 with Eqs. 11- 13 to obtain the new attention-aggregated vector.
The workings of Stock Correlation Memory Network are quite similar.
Consequently, the fusion of different information flows is promoted due to the multi-directional interaction mechanism in which CMIN learns not only the influences from text/correlation to movement prediction within each information flow but also the interactive influences between different information flows, representing the interrelationship between text and correlations.

heading: 4.5 Learning Objective
text: With the continuous representations v(H)text and v (H) corr from the last layer of each memory network, along with the target stock representation vtarget, we concatenate them and apply a softmax function to generate the final prediction vector ŷ:
ŷ = softmax(Wy[v (H) text , vtarget, v (H) corr ] + by). (14)
The objective is to minimize the cross entropy loss:
L(y, ŷ)=− n∑
i=1
(yi log (ŷi) + (1−yi) log (1−ŷi)) (15)
where n is the size of the training set.

heading: 5 Experiments
text: In this section, we empirically evaluate our CMIN model with three real-world datasets collected from the U.S. and Chinese stock markets.

heading: 5.1 Experimental Settings

heading: 5.1.1 Datasets
text: In our experiments we have used three datasets, namely ACL18, CMIN-US and CMIN-CN, spanning different time periods to evaluate our proposed model CMIN against other baselines.
ACL18 (Xu and Cohen, 2018) is a classic dataset with tweets from Twitter as financial texts in the task of text-enhanced stock movement prediction. As there are few existing high-quality datasets containing both texts and price, we are also making available two new benchmark datasets along with this paper from 2018-01-01 to 2021- 12-31 in the U.S. and Chinese market named CMIN-US and CMIN-CN. These two datasets are available at https://github.com/BigRoddy/ CMIN-Dataset to facilitate further research and enable reproducibility. More details and statistics of those three datasets are in Appendix A.

heading: 5.1.2 Baselines
text: We compare CMIN against the following four baselines, all of which are high-performing stock movement prediction models proposed by recent studies: •ALSTM(Qin et al., 2017) is a dual-stage attention-based recurrent neural network, which selects relevant time series across all time steps. •Adv-LSTM(Feng et al., 2019) uses adversarial training to improve the generalization of ALSTM. •Stocknet(Xu and Cohen, 2018) introduces recurrent continuous latent variables and uses variational inference to address the posterior inference. •DTML(Yoo et al., 2021) is a newly published attention-based model that exploits the correlations between stocks to improve the prediction accuracy.

heading: 5.1.3 Evaluation metrics
text: As we have formulated stock price movement prediction as a classification problem, we choose two classic metrics: Accuracy (Acc.) and Matthews Correlation Coefficient (MCC), similar to the previous work (Xu and Cohen, 2018; Yoo et al., 2021).
Acc. = tp+ tn
tp+ tn+ fp+ gn (16)
MCC= tp× tn− fp× tn√
(tp+ fp)(fn+ tp)(fn+ tn)(fp+ tn) (17)

heading: 5.1.4 Implementation details
text: We set our model for daily price prediction, with a history market window size k = 5 and the number of price features dp = d = 3, namely the highest, the lowest and the closing prices. We limit the maximum number of financial texts in one single day to be l = 20 , and the maximum length of a text document w = 30. Within the Text Encoder, we set the size of word embedding vector dw = 50 and the hidden state of Bi-GRU network dm = 50.
We implement the CMIN with Pytorch on a NVIDIA Tesla V100 and train it with an Adam optimizer (Kingma and Ba, 2015). All parameters of our model are initialized with Xavier Initialization (Glorot and Bengio, 2010). We search the hyperparameters of CMIN as follows: number of layers of each memory network H in {1, 2, 3, 4, 5}, dropout rate in {0.1, 0.2, 0.3}, number of epochs in {10, 20, 50}, and size of the price hidden state dp in {3, 10, 50}. For baselines, we use their default parameters and fine-tune them to fit our data.

heading: 5.2 Performance Analysis
text: The results are summarized in Table 1. Among all models, ALSTM and Adv-LSTM performed poorly with little improvement over random prediction. This could be attributed to the fact that these models rely solely on stock prices as the basis for decision-making. The Stocknet and DTML incorporate additional information beyond stock prices, demonstrated significant improvements over ALSTM and Adv-LSTM, which highlights the importance of utilizing financial texts and stock correlations for this challenging task. CMIN outperformed all baselines and achieved state-of-the-art performance on both two metrics across all datasets, showing its excellent capabilities to leverage both financial texts and stock correlations, as well as capture their interrelationship.

heading: 5.3 Ablation Studies
text: To evaluate the contribution of CMIN’s different components, we compare against several variants: •CMIN-TE: CMIN without the Text (TE), which makes decisions just based on stock prices. •CMIN-PR: CMIN without the Price (PR), which makes decisions just based on related texts. •CMIN-CM: CMIN without the guide of causality matrix (CM). •CMIN-MI: CMIN without multi-directional interactions (MI) between memory networks. The results are summarized in Table 2. CMINTE only achieves a level of prediction accuracy on par with ALSTM and Adv-LSTM, and is worst among all the variants, again indicating the importance of text data. Similar to the performance of Stocknet, CMIN-PR has a relatively high Acc. but a low MCC, suggesting texts are particularly helpful to predict on one side of the binary classification. By modeling both text data and stock relationships, CMIN-CM reaches a good result. Finally, better performance achieved when causality matrix and multi-directional interactions are introduced into the network. Overall, the ablation studies show that every component makes an important contribution to CMIN, and as a result the full model with all components achieves the best performance.

heading: 5.4 Analysis of Memory Network Depth
text: As introduced before, we propose two memory networks to retain vital features of texts and correlations with multiple computational layers. And we want to understand what would be the ideal number of depths to achieve the best prediction results.
We change the number of layers H of each memory network to find out how the performance fluctuates with it. The results are summarized in Figure 2. When we only have one memory layer, there is no multi-directional information flows between the two memory networks and as a result they only try
to identify the vital information in the embeddings related to or having an impact on the movement of the target stock under the supervision of vtarget. As the number of memory layers increases, the interactions between two memory networks also intensifies. It is intuitive that the performance of CMIN reaches its peak when it has three memory layers. With further increase the number of memory layers, CMIN is prone to overfit.

heading: 5.5 Case Study
text: Here we present an example to illustrate how CMIN considers both financial texts and stock correlations to avoid random noises in time series.
We visualized the causality matrix of ACL18 using a heat map as shown in Figure 3. Stocks are sequenced by their industry sector. The black box on the left shows weak causality, representing weak information flow from Utilities to Materials. On the other hand, the yellow box on the right indicates the relative strong information flow from Materials to Finance and within the Finance industry.
The target stock is Bank Of America (BAC) with a monitor window spanning from 13/11/2015 to 19/11/2015. We employ CMIN to predict BAC’s next movement direction on the day of 20/11/2015 and then output the attention scores of texts and causality-guided correlation. The most focused stock by CMIN is Berkshire Hathaway Inc. (BRKA). It’s interesting to note that both are in the same industry sector: Finance, and they do appear to follow a very similar movement pattern in the trading days leading to 20/11/2015, which demonstrates the ability of CMIN to find the dynamic stock correlations with the guidance of Causality Matrix.
The financial text of BAC that obtains the highest attention score is "Beer, Credit Card Debt And
Other Positives For Bank Of America", the title of an news article1 which reports the rapidlyimproving banking landscape in the U.S.. This text is clearly highly relevant to BAC’s subsequent stock performance, which demonstrates that CMIN is able to identify highly relevant texts having a impact on the target stock movement.
Furthermore, it also illustrates the underlying interrelationship between financial texts and stock correlations. Except expressing an optimistic sentiment towards BAC, the news also shows a rapidly improving state of affairs for the wider financial industry. Therefore, through the Multi-directional Interactions mechanism, the text strengthens the model’s attention to stocks in the same sector. These two aspects mutually reinforce and complement each other to help the model make the best judgment that BAC’s stock price will rise on the next day.

heading: 6 Conclusions
text: In this paper, we proposed CMIN, a causalityguided multi-memory interaction network that simultaneously models financial documents, causality-enhanced stock correlations and the interactions between the two, and recurrently learns a global memory representation for movement prediction. This multi-modality network was designed to enable the concurrent discovery of texts and stock correlations relevant to future price change and we demonstrated, through experiments on three datasets across two distinct markets, that each component of the proposed architecture made significant contributions to the model, leading CMIN to achieve state-of-the-art accuracy.
1https://seekingalpha.com/article/3692516
Limitations
We discuss the limitations of our model as follows:
1. Due to the natural uncertainty of financial forecast, although we have taken many methods to improve the generalization performance of the model (such as limiting the depth of memory layers and with the assistance of auxiliary data), creating a trustworthy application requires considering many other factors beyond the algorithmic level. We advise that users monitor the model’s performance over time and regularly update it to adapt to everchanging market conditions.
2. This paper uses Granger causality based on transfer entropy to make a preliminary attempt to introduce causality between time series to model the similarity between stocks more accurately. But this description is junior and classical, and there are lots of more modern methods to measure precise causality in mathematics (like PC algorithm), which we believe would further improve the performance.
3. We only experiment the performance of model on the task of binary classification, leaving more complex tasks (such as regression task and returns prediction) and simulating actual investment to evaluate the capability and potential of the model comprehensively.

heading: Acknowledgements
text: We would like to express our sincere gratitude to the reviewers for their valuable efforts and insightful contributions. This work was supported by National Natural Science Foundation of China (NSFC Grant No. 62122089), Beijing Outstanding Young Scientist Program NO. BJJWZYJH012019100020098, and Intelligent Social Governance Platform, Major Innovation & Planning Interdisciplinary Platform for the ”Double-First Class” Initiative, Renmin University of China. Rui Yan is the corresponding author.

heading: A Dataset Details
text: The statistics of datasets are summarized in Table 3. ACL18 (Xu and Cohen, 2018) consists of 87 stocks of 9 industries from the U.S. stock market. It also includes two types of data: tweets from Twitter and historical stock prices from Yahoo finance. We have processed in the same way as described in (Xu and Cohen, 2018).
As there are few existing high-quality datasets containing both texts and prices, we are also making available two new benchmark datasets along with this paper from 2018-01-01 to 2021-12-31: CMIN-US includes the top 110 stocks from US by market capitalisation; CMIN-CN consists of all 300 constituents of CSI300, a major Chinese stock market index. Similar to ACL18, both CMIN-US and CMIN-CN include financial texts as well as historical stock prices data. The historical price data in both datasets comes from Yahoo Finance. The text data of CMIN-US is collected from Yahoo finance2 and CMIN-CN from Wind 3. In our experiments, we have used news headlines instead of the entire texts for efficiency and noise reduction.
2https://finance.yahoo.com/ 3https://www.wind.com.cn/
ACL 2023 Responsible NLP Checklist

heading: A For every submission:

heading: 3 A1. Did you describe the limitations of your work?
text: the last section

heading: 3 A2. Did you discuss any potential risks of your work?
text: the last section

heading: 3 A3. Do the abstract and introduction summarize the paper’s main claims?
text: the first section
7 A4. Have you used AI writing assistants when working on this paper? Left blank.
B 3 Did you use or create scientific artifacts? Section 5

heading: 3 B1. Did you cite the creators of artifacts you used?
text: Section 5.1
B2. Did you discuss the license or terms for use and / or distribution of any artifacts? Not applicable. Left blank.

heading: 3 B3. Did you discuss if your use of existing artifact(s) was consistent with their intended use, provided
text: that it was specified? For the artifacts you create, do you specify intended use and whether that is compatible with the original access conditions (in particular, derivatives of data accessed for research purposes should not be used outside of research contexts)? Section 5.1
B4. Did you discuss the steps taken to check whether the data that was collected / used contains any information that names or uniquely identifies individual people or offensive content, and the steps taken to protect / anonymize it? Not applicable. Left blank.

heading: 3 B5. Did you provide documentation of the artifacts, e.g., coverage of domains, languages, and
text: linguistic phenomena, demographic groups represented, etc.? Section 5.1
3 B6. Did you report relevant statistics like the number of examples, details of train / test / dev splits, etc. for the data that you used / created? Even for commonly-used benchmark datasets, include the number of examples in train / validation / test splits, as these provide necessary context for a reader to understand experimental results. For example, small differences in accuracy on large test sets may be significant, while on small test sets they may not be. Section 5.1.1 and Appendix A
C 3 Did you run computational experiments? Section 5

heading: 3 C1. Did you report the number of parameters in the models used, the total computational budget
text: (e.g., GPU hours), and computing infrastructure used? Section 5.1.4
The Responsible NLP Checklist used at ACL 2023 is adopted from NAACL 2022, with the addition of a question on AI writing assistance.

heading: 3 C2. Did you discuss the experimental setup, including hyperparameter search and best-found
text: hyperparameter values? Section 5.1.4

heading: 3 C3. Did you report descriptive statistics about your results (e.g., error bars around results, summary
text: statistics from sets of experiments), and is it transparent whether you are reporting the max, mean, etc. or just a single run? Section 5.2

heading: 3 C4. If you used existing packages (e.g., for preprocessing, for normalization, or for evaluation), did
text: you report the implementation, model, and parameter settings used (e.g., NLTK, Spacy, ROUGE, etc.)? Section 5.1
D 7 Did you use human annotators (e.g., crowdworkers) or research with human participants? Left blank.
D1. Did you report the full text of instructions given to participants, including e.g., screenshots, disclaimers of any risks to participants or annotators, etc.? Not applicable. Left blank.
D2. Did you report information about how you recruited (e.g., crowdsourcing platform, students) and paid participants, and discuss if such payment is adequate given the participants’ demographic (e.g., country of residence)? Not applicable. Left blank.
D3. Did you discuss whether and how consent was obtained from people whose data you’re using/curating? For example, if you collected data via crowdsourcing, did your instructions to crowdworkers explain how the data would be used? Not applicable. Left blank.
D4. Was the data collection protocol approved (or determined exempt) by an ethics review board? Not applicable. Left blank.
D5. Did you report the basic demographic and geographic characteristics of the annotator population that is the source of the data? Not applicable. Left blank.

heading: Future Work
text: This multi-modality network was designed to enable the concurrent discovery of texts and stock correlations relevant to future price change and we demonstrated, through experiments on three datasets across two distinct markets, that each component of the proposed architecture made significant contributions to the model, leading CMIN to achieve state-of-the-art accuracy. 1https://seekingalpha. com/article/3692516
Limitations
We discuss the limitations of our model as follows:
1. Due to the natural uncertainty of financial forecast, although we have taken many methods to improve the generalization performance of the model (such as limiting the depth of memory layers and with the assistance of auxiliary data), creating a trustworthy application requires considering many other factors beyond the algorithmic level. We advise that users monitor the model’s performance over time and regularly update it to adapt to everchanging market conditions. 2. This paper uses Granger causality based on transfer entropy to make a preliminary attempt to introduce causality between time series to model the similarity between stocks more accurately. But this description is junior and classical, and there are lots of more modern methods to measure precise causality in mathematics (like PC algorithm), which we believe would further improve the performance. 3. We only experiment the performance of model on the task of binary classification, leaving more complex tasks (such as regression task and returns prediction) and simulating actual investment to evaluate the capability and potential of the model comprehensively