Enhancing Financial Forecasting through Time Series Prediction Updates with Event-Driven Insights from Large Language Models
Authors: Litton Jose Kurisinkel, Pruthwik Mishra, Yue Zhang


Reviewer paper summaries: The paper proposes to model stock price time-series within the context of financial events. It fine-tunes LLM to predict the price change of a stock for next n days, which are fed as sequence input to GRU to obtain the sequence of temporal stock state embeddings. The stock state at each future day is read out into a probability vector of "increase, decrease, stay" which is used to compute the expected price amplification/attenuation. The price amplification/attenuation values and predicted stock prices for next n days are then concatenated and converted to the updated prices for the next n days by a dense layer.

This paper introduces a multimodal framework called TimeS for ﬁnancial forecasting that integrates time series prediction with event-driven insights from large language models (LLMs). The framework leverages the intuition of LLMs about future changes to update time series predictions. The key components include:

1. Training multivariate time series models to predict individual stock prices

2. Fine-tuning LLMs to predict discrete labels of price change directions over the next n days following an event

3. Computing stock state transitions based on the LLM-predicted price change labels to estimate event-induced price ﬂuctuations, which are then used to update the time series predictions.

The eﬀectiveness of this approach was evaluated on ﬁnancial market data. Experiments across small-cap, mid-cap and large-cap stocks conﬁrmed that incorporating LLM insights about events can improve the accuracy of stock price forecasting.
Summary of strengths: S1. The overall method based on the 3 steps listed in Lines 216-219 looks reasonable.
S2. Experiments are extensive and show that the method works better than baseline methods.

1. The authors propose a novel multimodal framework called TimeS that integrates time series prediction with eventdriven insights from large language models (LLMs) for ﬁnancial forecasting. This framework leverages the powerful intuition of LLMs about future changes to update and enhance the predictions made by time series models.

2. The authors propose a method to compute stock state transitions based on the price change labels predicted by the ﬁne-tuned LLMs. These stock state transitions are then used to estimate the event-induced price ﬂuctuations, which capture the impact of events on stock prices over time. The estimated price ﬂuctuations are subsequently incorporated into the time series predictions to update and reﬁne them. 

3. The authors propose an end-to-end training scheme for the TimeS framework, which allows for the seamless optimization of the entire system.
Summary of weaknesses: W1. The model is straightforward and there is not much novelty in the approach. In fact, some design choices do not look proper and look quite naive, like the dense layer in Eq (4) which totally ignores the time step. A reasonable solution should at least align A_s[t] with P_s[t] to create transformed feature at time t (e.g., using plus).

W2. The results do not look very promising. For example, Table 2 shows that the F1-score for change type prediction is merely 0.6-0.7, which is problematic since that's just the middle step to compute price amplification/attenuation values, and the error will propagation forward along the model!

W3. The writing is poor to the extent that a lot of technical contents are not clear. For example,
- Line 284, Eq(2) has a stock embedding S which is never defined or explained. Do you mean you have a stock embedding matrix to be learned?
- Line 328 P^a_s[t] is never defined and its meaning is totally unclear. Also, if you use word, at least use equation editor so that "a" and "s" are vertically aligned.

W4. The writing is very careless. For example,
- Line 229 "time- steps"
- Line 259 and Line 273 miss a space after colon, and it is unclear why U in Line 315 is in different font format than F in Line 273
- Line 280 begins with "predict stock predict stock"
- Line 298 "We use the stock 298 states compute", there are two predicates "use" and "compute"
- Notation inconsistency: Line 308 uses As_t, but Line 319 uses A_s[t]
- Line 314 should be \in\mathbb{R}^n, but you even use epsilon to mean \in...
- Line 413 "fine tune" should be fine-tune

W5. Some related works are not reviewed. For example, the following work also uses BERT to extract news features to improve stock price prediction but was not reviewed or mentioned at all.
- Yingzhe Dong, Da Yan, Abdullateef Ibrahim Almudaifer, Sibo Yan, Zhe Jiang, Yang Zhou. "BELT: A Pipeline for Stock Price Prediction Using News". IEEE BigData 2020: 1137-1146

1. While the paper demonstrates the eﬀectiveness of the proposed TimeS framework on ﬁnancial market data, the experiments are limited to a speciﬁc dataset (EDT) and a particular time period. The authors could have strengthened their findings by evaluating the framework on additional datasets from different financial markets or time periods.

2. The paper compares the performance of the TimeS framework with baseline time series models and a simpliﬁed version called TimeL. However, it would have been beneﬁcial to include comparisons with other state-of-the-art methods that combine time series forecasting with textual information or event-driven insights. 

3. The TimeS framework consists of several components, including multivariate time series models, ﬁne-tuned LLMs, and stock state transition modeling. While the paper presents ablation studies to analyze the performance under different magnitudes of change, it could have provided a more detailed analysis of the individual contributions and interactions of these components.
Comments, suggestions and typos: Typos and writing issues are basically W3 and W4 above:

W3. The writing is poor to the extent that a lot of technical contents are not clear. For example,
- Line 284, Eq(2) has a stock embedding S which is never defined or explained. Do you mean you have a stock embedding matrix to be learned?
- Line 328 P^a_s[t] is never defined and its meaning is totally unclear. Also, if you use word, at least use equation editor so that "a" and "s" are vertically aligned.

W4. The writing is very careless. For example,
- Line 229 "time- steps"
- Line 259 and Line 273 miss a space after colon, and it is unclear why U in Line 315 is in different font format than F in Line 273
- Line 280 begins with "predict stock predict stock"
- Line 298 "We use the stock 298 states compute", there are two predicates "use" and "compute"
- Notation inconsistency: Line 308 uses As_t, but Line 319 uses A_s[t]
- Line 314 should be \in\mathbb{R}^n, but you even use epsilon to mean \in...
- Line 413 "fine tune" should be fine-tune

Weaknesses.
Overall assessment scores: 1.5

2