MCASP : Multi-Modal Cross Attention Network for Stock Market Prediction (2023)

Abstract: Stock market prediction is considered a complex task due to the non-stationary and volatile nature of the stock markets. With the increasing amount of online data, various information sources have been analyzed to understand the underlying patterns of the price movements. However, most existing works in the literature mostly focus on either the intra-modality information within each input data type, or the inter-modal relationships among the input modalities. Different from these, in this research, we propose a novel Multi-Modal Cross Attention Network for Stock Market Prediction (MCASP) by capturing both modality-specific features and the joint influence of each modality in a unified framework. We utilize financial news, historical market data and technical indicators to predict the movement direction of the market prices. After processing the input modalities with three separate deep networks, we first construct a self-attention network that utilizes multiple Transformer models to capture the intra-modal information. Then we design a novel cross-attention network that processes the inputs in pairs to exploit the cross-modal and joint information of the modalities. Experiments with real world datasets for S&P500 index forecast and the prediction of five individual stocks, demonstrate the effectiveness of the proposed multi-modal design over several state-of-the-art baseline models.

## introduction stock market movements are inherently affected by a multitude of data sources, encompassing historical price data, technical indicators @xcite , financial news @xcite , social media @xcite , and official announcements @xcite . it has been established that analyzing these multiple data modalities together enables the capture of underlying patterns in stock movements, rendering stock market prediction a multi-modal learning task @xcite . the efficacy of employing effective multi-modal representation and learning techniques to uncover the joint influence of these data modalities is pivotal for model performance @xcite . simultaneously, it is important to extract the intra-modal information within each data source. early information fusion techniques combine raw input features initially and then construct a prediction model, which aids in capturing the combined influence of modalities but neglects intra-modal information. late fusion techniques, conversely, analyze input features separately and subsequently employ a fusion layer for prediction. while this approach facilitates a focus on modalityspecific features, it may overlook inter-modal information. balancing the capture of intra-modal and inter-modal information from input modalities is essential. researchers have identified that pairs of data modalities, such as financial news and market prices, as well as market prices and technical indicators @xcite , both impact price movements. however, existing models, while striving to capture the joint influence of all modalities together, may overlook the underlying bi-modal relationships between various data inputs. therefore, in addition to capturing their collective influence, it is also crucial to understand the bi-modal relationships among pairs of input modalities. to address these challenges, various methods have been developed, primarily categorized as inter-modality and intra-modality-based techniques. inter-modality methods aim to capture the underlying relationships among input modalities but may miss the connections within each modality. conversely, intra-modality techniques focus on uncovering modality-specific relations but tend to disregard the inter-modal connections across input modalities. combining modality-specific features with inter-modal connections can synergize and enhance overall analysis. hence, exploring a uni-fied framework capable of capturing both intermodality and intra-modality relations within the input data is imperative. motivated by these challenges, we present a novel multi-modal cross-attention network for stock market prediction (mcasp). mcasp forecasts the direction of price movements by jointly modeling inter-modality and intra-modality relationships within the input data (i.e., financial news, market data, and technical indicators) within a unified deep learning framework. to achieve this, we construct two distinct attention networks: a selfattention network and a cross-attention network, designed to capture intra-modal and inter-modal relationships, respectively. the self-attention module focuses on extracting modality-specific features from the input modalities. we first employ two separate long short-term memory (lstm) networks to extract latent features from market data and technical indicators. simultaneously, we leverage finbert @xcite to encode textual data (i.e., financial news). within the self-attention network, the lstm network outputs are processed by two transformer @xcite units, while the encoded textual data undergo analysis via a convolutional neural network (cnn). the cross-attention module involves creating three pairs by concatenating representations of news and market data, news and technical indicators, and market data and technical indicators. these pairs are then fed into three separate transformer units. the outputs from the self-attention and cross-attention modules converge in the fusion layer to generate a combined feature vector. finally, we employ a fully-connected layer to predict the direction of price movements. ## related work in this section, we review related work in stock market prediction, multimodal machine learning and the attention mechanism. ## model design in this section, we provide a detailed description of the architecture of the proposed mcasp model. the design of our mcasp model is demonstrated in figure1. ## experimental settings in our experiments, we utilized real-world datasets encompassing financial news, market data, and technical indicators spanning from january 1, 2010, to december 31, 2019, encompassing a 10-year period. the financial news was sourced from reuters foot_0 , with each article containing a title, body, and publication date. the publication date was employed to align the articles with the daily market data. we specifically focused on the headlines from the financial news, as research has demonstrated that using news titles can yield superior prediction results compared to using the entire article body @xcite . the number of news titles per trading day varied; hence, we aggregated all the titles for a given day into a single extended sentence and employed finbert to encode the textual data into feature vectors. consequently, we obtained a single sentence embedding vector for each trading day. we utilize historical market data for s&p index and individual stocks from yahoo finance foot_1 for the corresponding dates. these five companies included google, tesla, amazon, apple, and microsoft and the data includes open, high, low, close prices, and volume. we normalize the market data to be within the range of [0, 1]. we initially employ an 80-20% split for training and testing for index price prediction. we also evaluate the yearly performances of the models by utilizing the first 10 months of each year for training and the last 2 months for testing. we utilize the 80-20% split again for training and testing purposes for individual stock prediction. based on the literature @xcite , we computed seven technical indicators for each trading day using the market data over the preceding five days. we employ accuracy (acc) and matthews correlation coefficent (mcc) to evaluate the performance of different models. mcc is generalyy employed when the sizes of classes @xmath0. ## results and analysis in order to test the effectiveness of our model, we run experiments using real-world dataset including financial news data, historical market data and technical indicators. ## conclusion we have proposed a novel multi-modal cross attention network for stock market prediction that models the intra-modal and inter-modal information from the input modalities in a unified framework. we first analyze the input modalities via three separate deep networks to extract the salient features. we then process these features with the proposed self-attention and cross-attention modules to jointly model the intra-modal and inter-modal information. we analyze financial news, historical market data and technical indicators to predict the movement direction of s&p500 index prices and the prices of five individual stocks. we test the effectiveness of the proposed multi-modal design using real-world dataset from reuters and yahoo! finance and compare its performance against multiple state-of-theart baseline models. experimental results show that our model achieves improved performance in stock market prediction.