Beyond Trend and Periodicity: Guide Time Series Forecasting with Textual Cues

Authors: Zhijian Xu, Yuxuan Bian, Jianyuan Zhong, Xiangyu Wen, Qiang Xu
Primary area: learning on time series and dynamical systems
Keywords: Time Series Forecasting, Multi-modal Model

Abstract:
This work introduces a novel Text-Guided Time Series Forecasting (TGTSF) task. By integrating textual cues, such as channel descriptions and dynamic news, TGTSF addresses the critical limitations of traditional methods that rely purely on historical data. To support this task, we propose TGForecaster, a robust baseline model that fuses textual cues and time series data using cross-attention mechanisms. We then present four meticulously curated benchmark datasets to validate the proposed framework, ranging from simple periodic data to complex, event-driven fluctuations. Our comprehensive evaluations demonstrate that TGForecaster consistently achieves state-of-the-art performance, highlighting the transformative potential of incorporating textual information into time series forecasting. This work not only pioneers a novel forecasting task but also establishes a new benchmark for future research, driving advancements in multimodal data integration for time series models.

Decision: reject
Reviewer scores: 5, 6, 3, 6

Reviews and discussion:
<thread>
creation date:
1729048895017

thread_type:
official_review

<thread_object>
creator: reviewer_ghyy

<content>
summary:
This paper introduces a novel Text-Guided Time Series Forecasting (TGTSF) task, integrating textual cues, such as channel descriptions and dynamic news, to enhance traditional forecasting methods that rely solely on historical data. The authors propose TGForecaster, a robust baseline model employing cross-attention mechanisms to fuse textual information with time series data. They present four meticulously curated benchmark datasets, ranging from simple periodic patterns to complex, event-driven fluctuations, to validate this task. Comprehensive evaluations demonstrate that TGForecaster consistently outperforms existing methods, highlighting the significant potential of incorporating textual information into time series forecasting.

strengths:
1. This paper introduces a novel task for multimodal time series analysis.
2. This paper presents four comprehensive multimodal time series datasets. 
3. The extensive experiments look hopeful.

weaknesses:
While this paper explores the potential of multimodal time series analysis, I have the following concerns:

1. **Information leakage**: Although the paper repeatedly emphasizes that their text data is known prior to prediction and that no information leakage occurs, I remain unconvinced. Among the three text forms, I believe only common knowledge is likely secure. System-level limited predictions and hypothesized or controlled events both present potential risks of information leakage.

   - Equation 1 indicates that TGForecaster relies on textual information that not only pertains to the future but is also precisely aligned with the point in time being predicted.
   - For system-level limited predictions, as exemplified by Figure 6, we must know not only that it will rain, but also the exact time the rainfall will cease in order to accurately capture rainfall peaks.
   - For hypothesized or controlled events, such as the rainfall prediction in row 220, we must not only anticipate artificial rainfall but also the duration of its impact.

   It seems challenging to provide this level of information using non-leaking text, which significantly limits the practical applicability and scope of the paper.

2. **Dataset contributions**: The authors claim to have contributed four datasets. Setting aside the synthesized dataset, we are left with three:

   - **Electricity**: Only holiday information is introduced as external text, and there seems to be no clear advantage over previous approaches utilizing timestamps [1].
   - **Weather**: From Appendix N.3, it appears they employed GPT-4 to convert accurate numerical predictions into textual predictions. Since precise numerical forecasts are already available, a direct approach using exogenous variables seems more efficient.
   - **Steam**: This dataset is not publicly available due to intellectual property restrictions.

   In summary, these datasets fail to convincingly demonstrate the utility of the paper.

3. **Task novelty**: The authors assert that they have introduced a text-guided time series forecasting task, yet previous works like Time-MMD [3] exist. While I do not expect the authors to conduct a quantitative comparison, given that both works may belong to the same period, I encourage them to discuss the distinctions between the two as they deem appropriate. To my knowledge, Time-MMD only utilizes historical textual information and does not face the issue of information leakage. Furthermore, its dataset construction appears to be more rigorous.

**References**

[1] 2024, Rethinking the Power of Timestamps for Robust Time Series Forecasting: A Global-Local Fusion Perspective

[2] 2024, TimeXer: Empowering Transformers for Time Series Forecasting with Exogenous Variables

[3] 2024, Time-MMD: A New Multi-Domain Multimodal Dataset for Time Series Analysis

questions:
The authors can refer to the weakness listed above.
</content>
</thread_object>

</thread>
<thread>
creation date:
1729049017885

thread_type:
official_review

<thread_object>
creator: reviewer_xiwt

<content>
summary:
The paper addresses a common challenge in time series forecasting (TSF): information insufficiency. It proposes an approach, TGForecaster with open-source word embedding, transformer block and cross-attention mechanism, which integrates textual cues to enhance forecasting models with external knowledge. The authors developed four datasets designed to validate TGTSF task. The proposed TGForecaster model demonstrates improvements in toy dataset and perform as good as previous small-size work in other dataset.

strengths:
Thanks for your work! The following are some strengths:
1. The paper is well organized and the motivation is clear
2. The paper shows a good result especially on toy dataset, which proves that it at least work in some way.
3. The author give ablation study in their paper which compare open-source embedding and w/o some texts, which is direct and clear

weaknesses:
1. TGForecaster itself is not innovative, which use open-source embedding and attention to do information fusion.  As far as I know, numbers derive from sampling, words come from natural description, so it's important to decide how to fuse the information with a more well-designed transformer or better alignment strategy, otherwise I would say this good result is based on the raise of parameters number. Also, for channeling modelling, graph-related work also give some insights.
2. The ablation study is just too simple, which leaves people thinking about \textit{what is the role of these words in TGForecaster ?}. This question is not about with or without words, but what will happen if I give random words. Your work involve long sequence of texts, which give many high-dimension variables and regression from there variables and give some ordinary result is simple.
3. It's really new for multi-modal time-series tasks, but the texts are not well-designed and are simply collect together, so I suggest doing more jobs on  a. how to fairly compare  b. how to align  c. how to evaluate the importance of texts and select  d. dig some real theory behind, eg. how to extract the essential information in high-dimension or how to compress unfixed texts
4. Limited evaluation dataset.
5. weak generalization ability.

questions:
1. How did you test Time-LLM(Jin et al., 2023), did you train it with your designed texts, can you give some samples how you evaluate the performance of Time-LLM and which language model did you use.
2. I wonder if there is some information leakage. For example, in the prediction of weather, words may indicate the time-series trend or related property
3. Are TGForecaster really able to understand the text? How do you study this.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730608624139

thread_type:
official_review

<thread_object>
creator: reviewer_gvc9

<content>
summary:
This article addresses the limitations of insufficient information in time series forecasting. It introduces TGForecaster, a Transformer-based multimodal model designed to integrate channel descriptions and dynamic news texts for time series forecasting, which includes two key innovations: Time-Synchronized Text Embedding and Text-Guided Channel Independent (TGCI). To validate the effectiveness of Text-Guided Time Series Forecasting (TGTSF) task, this article proposes four types of multimodal datasets to test the model's capabilities. Experimental results show that TGForecaster consistently outperforms all SOTA models on four datasets and underscore the significance of textual data on performance enhancement.

strengths:
1. The article has a clear structure, relatively sufficient experiments, and clear presentation of experimental results, and the writing is relatively smooth.
2. The TGForecaster framework proposed in this article is novel and its effectiveness has been verified through experiments.

weaknesses:
1. The article mentions two key innovations of Time-Synchronized Text Embedding and Text-Guided Channel Independent, but the experimental part lacks ablation experiments on these two innovative points. Suggest the author to supplement experiments to better highlight these two innovative points.

2. For the comparative experiment part, this article lacks a comparison with the naïve model. It is recommended that you can refer to the MM-TFSlib framework mentioned in Time-MMD (https://arxiv.org/pdf/2406.08627v2), which uses existing time series forecasting models and LLMs to process time series and textual data separately, and then combines them for forecasting. Suggest the author to supplement this experiment to better illustrate the necessity and rationality of TGForecaster in the aspect of architecture design.

3. In Table 2 and Table 7 (in Appendix I), the experimental results of the electricity dataset (LBW=120 and Pred. Len.=720) are inconsistent (0.200/0.209). Please confirm and make the necessary modifications.

4. In the “Ablation Study” part on page 9, the author evaluates the impact of three different embedding models on the performance of TGForecaster based on the Weather-Captured-Medium dataset. According to the comparison with the experimental results in Table 3, the author ultimately chooses the OpenAI embedding. So why do we choose MiniLM as the embedding model when conducting comparative experiments on whether adding channel descriptions and news texts or not? Suggest the author to supplement corresponding experiments or provide explanations.

5. Regarding the confusion of the subplots "wv (m/s)" and "max. wv (m/s)" in Figure 7 (in Appendix F): according to the time series historical ground truth, theoretically they may be two easy forecasting cases and the forecasts given by PatchTST (Green line) should be close to a horizontal line, but the actual forecasts differ significantly from it. please explain the reason.

6. The color bar is missing from Figures 9 and 10 in the Appendix H. Please add it. In addition, there are duplicate words in the caption of the two figures, such as "on on" and "onon" in lines 1078 and 1114. Suggest the author carefully checks the wording in this article.

questions:
Refer to "Strengths And Weaknesses".
</content>
</thread_object>

</thread>
<thread>
creation date:
1730708202004

thread_type:
official_review

<thread_object>
creator: reviewer_4blg

<content>
summary:
The paper introduces a new task, Text-Guided Time Series Forecasting (TGTSF), which addresses limitations in traditional time series forecasting by integrating textual information as additional inputs. The authors propose TGForecaster, a model that uses cross-attention mechanisms to combine time series data with text, such as channel descriptions and news, enhancing forecasting accuracy. The paper presents four benchmark datasets, each designed to test different facets of this task, from periodic patterns to complex, event-driven fluctuations. Experimental results show that TGForecaster consistently outperforms baseline models, demonstrating the value of incorporating external text data in time series forecasting.

strengths:
1. The paper is clearly written, with thorough explanations of model components and datasets.
2. TGForecaster is well-designed, with cross-attention mechanisms validated through comprehensive experiments across diverse datasets.
3. TGTSF demonstrates that textual context enhances forecasting accuracy, establishing a valuable foundation for future multimodal research in forecasting applications.

weaknesses:
1. Limited novelty: The approach of using textual embeddings for time series forecasting, especially in financial domains, is not new. Prior work in stock forecasting, such as [1][2], has already explored combining news and other textual data with time series data to improve forecasting.
2. Compositional methodology: Since the channel descriptions are generated by large language models (LLMs), it would be logical and consistent to utilize embeddings directly from the LLMs used for text generation, rather than relying on a separate text embedding model.
3. Missing reference: The paper ignores the existing attempts in using textual embeddings for time series forecasting, such as [1][2]


[1] Sawhney, Ramit, Arnav Wadhwa and Shivam Agarwal. “FAST: Financial News and Tweet Based Time Aware Network for Stock Trading.” Conference of the European Chapter of the Association for Computational Linguistics (2021).
[2] Liu, Mengpu, Mengying Zhu, Xiuyuan Wang, Guofang Ma, Jianwei Yin and Xiaolin Zheng. “ECHO-GL: Earnings Calls-Driven Heterogeneous Graph Learning for Stock Movement Prediction.” AAAI Conference on Artificial Intelligence (2024).

questions:
1. Could the authors clarify what unique insights or capabilities TGTSF adds to this field? Specifically, how does TGTSF extend beyond the use of composite news and channel descriptions?

2. Is there a specific reason why embeddings from the LLMs used for generation were not directly employed, instead of relying on an external text embedding model?

3.  Would the authors consider expanding the evaluation to include comparisons with other advanced multimodal models from financial or weather forecasting?
</content>
</thread_object>

</thread>


Metareview: