Guided Attention Multimodal Multitask Financial Forecasting with Inter-Company Relationships and Global and Local News (2022)

Abstract: Most works on financial forecasting use information directly associated with individual companies (e.g., stock prices, news on the company) to predict stock returns for trading. We refer to such company-specific information as local information. Stock returns may also be influenced by global information (e.g., news on the economy in general), and inter-company relationships. Capturing such diverse information is challenging due to the low signal-to-noise ratios, different time-scales, sparsity and distributions of global and local information from different modalities. In this paper, we propose a model that captures both global and local multimodal information for investment and risk management-related forecasting tasks. Our proposed Guided Attention Multimodal Multitask Network (GAME) model addresses these challenges by using novel attention modules to guide learning with global and local information from different modalities and dynamic inter-company relationship networks. Our extensive experiments show that GAME outperforms other state-of-the-art models in several forecasting tasks and important real-world application case studies.

## introduction forecasting stock prices or returns is an important task in trading. such forecasts can also be used in investment and risk management applications such as portfolio allocation and risk forecasting. stock returns in financial markets are influenced by large volumes of textual information from diverse sources, e.g., news, blogs, social media. such textual information can be directly associated with a specific company (local), e.g, a company's ceo stepping down; or relevant to multiple companies (global), e.g., disruptions in supply chains due to export curbs in key countries, airline industry bankruptcies. in this paper, articles with company tags are treated as local information. all articles are treated as global information as any article could be potentially relevant to a company. direct and indirect relationships between companies also serve as channels through which the effects of information from both global and local textual and numerical information propagate and influence stock returns, e.g., a disruption in company a could affect all its suppliers; a scandal involving company a's ceo may affect company b if the ceo is a member of company b's board. we illustrate such diverse information and effects in figure 1 . apart from low signal-to-noise ratios in financial time-series due to market forces, there are other challenges in modeling such diverse information. time scales of information from different modalities are of different granularity, e.g., numerical financial information may be available daily, while publication of financial text happens at irregular times. companies' local financial news are typically sparse and long-tailed, e.g., a company may not be in the news for an extended period of time, but suddenly becomes the focus of many news reports in a short period due to a scandal. local textual information may also be noisy with regards to its relevance to the company's stock returns, e.g., a news article on a company's hr practices may have little effect on its stock returns, whereas a news article on a sector's outlook can have a significant effect on the company's stock returns even without any mention of the company. more research on financial forecasting is required to address such challenges. most existing works model financial information of a single modality @xcite @xcite , and do not model the effects of inter-company relationships. some works @xcite @xcite ) model both unimodal financial information and the effects of intercompany relationships. there are however few works capturing multimodal financial information and inter-company relationships @xcite . ang and ee-peng (2021) utilizes both numerical and global textual information, as well as inter-company relationships but does not address challenges related to capturing global and local multimodal information. most works also focus on a single task -forecasting stock returns for trading. another equally important set of forecasting tasks which has many investment and risk management applications involves similar challenges. it involves a multivariate multitask setting, where there is a need to manage the returns and risks of financial portfolios that comprise many stocks (multivariate), and make investment and risk decisions based on multiple forecasts (multitask): forecast stock i) mean returns and ii) risks (volatilities) over a future horizon to balance potential returns and risks when making investment decisions, as well as forecast iii) correlations between stocks in portfolios over a future horizon. to address financial data challenges in multitask settings, we propose the guided attention multimodal multitask network (game) model. our key idea is to use attention to guide learning between information from different sources and modalities. game incorporates several important components: i) guided latent cross-attention learning between modalities of different time-scales and sparsity; ii) graph-guided representation learning based on inter-company relationships with dynamic weights learnt from multimodal information; and iii) guided cross-attention learning between global and local information. game is trained on multiple tasks -forecasting means, volatilities and correlations over a future horizon, which could be used for portfolio allocation and risk management. while existing works for financial forecasting capture either local or global information, or network information with either global or local information, game jointly captures global, local and network information. compared to existing works that utilize transformers for time-series forecasting @xcite , game proposes novel cross-attention mechanisms that enable i) more effective modelling of local information of different lengths and granularity from different modalities by first encoding such information to a common latent representation; and ii) the extraction of global information by leveraging such local information. hence, our key contributions are as follows: ## related work as this work involves time-series forecasting and network learning, we review key related works in these areas. time series forecasting. classical methods @xcite are commonly applied to time-series forecasting. however, they are designed for numerical data but not unstructured financial text. deep learning models have been increasingly applied to time-series forecasting. they include feed-forward networks @xcite @xcite , convolutional neural networks @xcite @xcite , recurrent neural networks @xcite @xcite , and transformers @xcite . a detailed review of these works can be found in @xcite ; jiang (2021); torres et al. @xcite . time-series transformer (tst) @xcite is a recent model based on the transformer encoder architecture designed for numerical inputs. stockembed (se) @xcite ) is designed for global textual features, while financial news and tweet based time aware network (fast) @xcite is designed for local textual features. to encode sequences of textual features, se utilizes bidirectional grus, while fast utilizes time-aware lstms @xcite . these works are designed for information from a single modality, do not model the effects of company-to-company relationships, and do not address the challenges of capturing global and local multimodal information. network learning. graph neural networks (gnn) compose messages based on network features, and propagate them to update the embeddings of nodes and/or edges over multiple neural network layers @xcite . in particular, graph convolutional network (gcn) @xcite aggregates features of neighboring nodes and normalizes aggregated representations by node degrees. graph attention network (gat) @xcite assigns neighboring nodes with different importance weights during aggregation. such gnns are designed for static networks with static node attributes and cannot be directly applied to networks where attributes are evolving time series. a few recent works extend gnns to prediction tasks on financial time-series data @xcite @xcite . relational stock ranking (rsr) @xcite uses lstm to generate output embeddings for numerical time-series data of companies before feeding the latter to learn company embeddings in a network using a gcnbased model, but does not consider textual information. knowledge enriched company embedding (kece) (ang and ee-peng, 2021) captures numerical and global textual information and uses a gat-based model to capture inter-company relationships but does not address the challenges of capturing global and local multimodal information. rsr and kece also do not learn the dynamic importance of inter-company relationships. ## guided attention multimodal multitask network model game represents companies in a network @xmath0, e, x), where v represents a set of company nodes, e represents relationships between companies, x represents sequences of multimodal attributes. given a time step t, we define numerical features x num j (t) = [x num j (t -k), ..., x num j (t -1)] to be the sequence of numerical price-related data associated with company v j over a window of k time steps up to t -1. textual news features include local and global textual features, i.e. x @xmath1,loc , x txt,glo }. the pre-encoded local news textual features directly associated with a company v j within the same window are denoted as x txt,loc as shown in figure 2 , game first encodes both x num (t) ∈ r |v |×k×d num and x txt,loc (t) ∈ r |v |×s×d txt , where d num , d txt are embedding dimension sizes, to a common latent sequence length l and dimension d by using latent attention-based encoders inspired by jaegle et al. (2021a), where l ≪ k as part of the guided latent cross-attention learning step. we introduce guided cross-attention to enable information from one modality to guide the attention learning of another. in the next dynamic graph-guided attention learning step, representations of both modalities are used to discover and update importance weights of inter-company relationships before applying dynamic graph convolutions. a latent decoder inspired by jaegle et al. (2021b) then decodes the numerical and local textual representations to the original sequence lengths k and s. in the guided global-local attention learning step, we use the decoded local representations to guide the attention extraction of the sequence of global textual features relevant to each company v j . the resultant representations are then combined and sequentially encoded with a transformer, followed by attention-based temporal and multimodal fusion. finally, game generates forecasts of means, volatilities and correlations of financial returns over a selected future horizon of k ′ timesteps, i.e. the means, volatilities and correlations of y returns (t) = [y returns (t), ..., y returns (t + k ′ )], where y returns (t) = (price(t) -price(t -1))/price(t -1) and price(t) denote the percent-age return and stock price at time step t respectively. we further elaborate on game modules below. guided latent cross-attention learning. this step addresses the challenge of learning information from modalities of different sequence lengths, degrees of sparsity and distributions, specifically x num (t) ∈ r |v |×k×d num and x txt,loc (t) ∈ r |v |×s×d txt . for x num (t), we first project the inputs to common dimension d and add a learnt time vector @xcite . the time vector is learned from the time-stamps t (t) corresponding to the inputs. in this paper, we use day of week, week and month of year for t num (t), and further include seconds of day for t txt,loc (t) as these are most relevant to the respective inputs. the time vector p num (t) ∈ r |v |×k×d is learned by combining functional forms and learnable weights and could be viewed as a time-sensitive version of positional encodings used in transformers @xcite . for game, the empirically chosen components used to generate the time vectors are φ num 1 = sigmoid(linear(t num (t))) and φ num 2 = cos(linear(t num (t))), which enable the model to extract non-linear and seasonalitybased temporal patterns. we then concatenate these components and project them: the same set of steps is repeated for x txt,loc (t) to obtain htxt,loc (t). hence, after the latent encoding step, both hnum (t) and htxt,loc (t) have the same sequence length l and dimension d, i.e. hnum (t), htxt,loc (t) ∈ r |v |×l×d , and share a common latent space due to the common l. in the next guided cross-attention step, information from each of the modalities guide attention learning of the other. sharing a common latent space facilitates mutually guided learning between the modalities and is more efficient as l ≪ k ≪ s. for this step, we generate queries, keys, and values from the numerical and local text representations: queries of one modality are used to guide the learning of the other modality as follows: dynamic graph-guided attention learning. we then utilize inter-company relationships e to guide learning. while these relationships do not frequently change (e.g., common sector relationships), their importances vary across time. hence, we discover dynamic relationship weights with the dynamic attentionbased edge weights discovery (dw) module. we concatenate and project hnum-txt (t) and htxt-num (t) with a linear layer to obtain: h(t) = linear[ hnum-txt (t)|| htxt-num (t)]. we then generate: to learn the importance of inter-company relationships in a dynamic manner, we compute attention weights: this results in the weighted adjacency tensor ã(t) ∈ r |v |×|v |×l with ãij (t) ∈ r l representing the weighted relational edges between asset i and j across latent dimension l. next, in the dynamic network convolution step, we utilize the encoded company representations h(t) and the weighted adjacency tensor ã(t) as inputs to a weighted dynamic graph convolution step to encode network representations of companies. for company v i , we compute its network representations z i (t) ∈ r l×d across l dimension by aggregating representations from its neighbors n (i, t) based on ãi,j (t), @xmath2: across all assets, we obtain z(t) ∈ r |v |×l×d . we adopt this approach instead of other gnns for computational efficiency as it allows us to apply graph convolution across multiple dimensions in parallel. ## ablation studies