Financial Forecasting from Textual and Tabular Time Series (2024)

Abstract: There is a variety of multimodal data pertinent to public companies, spanning from accounting statements, macroeconomic statistics, earnings conference calls, and financial reports. These diverse modalities capture the state of firms from a variety of different perspectives but requires complex interactions to reconcile in the formation of accurate financial predictions. The commonality between these different modalities is that they all represent a time series, typically observed for a particular firm at a quarterly horizon, providing the ability to model trends and variations of company data over time. However, the time series of these diverse modalities contains varying temporal and cross-channel patterns that are challenging to model without the appropriate inductive biases. In this work, we design a novel multimodal time series prediction task that includes numerical financial results, macroeconomic states, and long financial documents to predict next quarter’s company earnings relative to analyst expectations. We explore a variety of approaches for this novel setting, establish strong unimodal baselines, and propose a multimodal model that exhibits state-of-the-art performance on this unique task. We demonstrate that each modality contains unique information and that the best performing model requires careful fusion of the different modalities in a multi-stage training approach. To better understand model behavior, we conduct a variety of probing experiments, reveal insights into the value of different modalities, and demonstrate the practical utility of our proposed method in a simulated trading setting.

## introduction investors are faced with the consumption of a myriad of diverse datasets relevant to public companies, spanning modalities, genres, and sources. in general, this data is released on a quarterly basis, and includes accounting statements, financial reports, qfr ect fin mac ... ... concatenate ... ... and earnings calls that comprehensively detail the current operations of the firm, recent financial performance, and discuss future business prospects and risks. while the accounting statements typically receive the most attention from investors, the textual content provided in earnings conference calls and financial reports is of equal importance as it reflects a direct communication between company executives and shareholders @xcite . this textual content provides two important qualitative sources of information relative to the financial metrics. first, it provides management context about how to interpret the current and historical financial results. second, it also allows management to express their views about the future prospects of the company, providing the opportunity to capture the tone and sentiment from the most well-informed stakeholders. for instance, during periods of macroeconomic shocks, such as covid-19, financial variables tend to capture backward-looking performance that may not apply in a new economic regime, while the textual commentary continues to provide forward-looking content ( §6.2). however, most work in financial prediction focuses on the most recently reported financial results and textual documents in isolation, without consideration of the temporal context of their historical patterns. while the most recent data point may be the most important, the historical context provides the ability to contextualize the current value and measure trends over time that help better predict future performance, as evidenced in both financial metrics and executive language patterns @xcite @xcite . while financial analysts consume this data to make their quarterly earnings forecasts, it is difficult to quantitatively reconcile this complex, diverse information across sources and modalities to arrive at accurate estimates. while financial analyst estimates of company earnings are generally regarded as market expectations, they have been shown to exhibit predictable biases that can be exploited by machine-based methods @xcite . for instance, financial analysts often do not fully incorporate the subtle signals in macroeconomic shocks (ball and ghysels, 2018) or long financial documents @xcite into their earnings estimates. therefore, we hypothesize that it is possible to use machine learning to effectively analyze this multimodal data and learn complex interactions between disparate sources of information that can be used to forecast earnings surprises months in advance of the report date. in this work, we consider a multimodal time series forecasting problem in which we investigate the value of financial text, time series, and their interaction in predicting next quarter's company performance relative to market expectations. in doing so, we introduce a new multimodal time series dataset and challenging financial prediction task, and propose a novel method that learns rich temporal and cross-channel features from the numerical and textual content of noisy financial time series. in summary, we make the following contributions: ## methods we systematically explore a comprehensive set of unimodal baselines and propose a novel multimodal method that significantly outperforms the best unimodal models. ## conclusion in conclusion, we introduce a new multimodal financial time series prediction task. we propose a novel textual feature extraction process and mul-statistic ar +fin+mac +ect+qfr net return 4.51 7.08 9.74 volatility 9.64 9.68 9.47 net sharpe ratio 0.47 0.73 1.03