From News to Forecast: Integrating Event Analysis in LLM-Based Time Series Forecasting with Reflection

Authors: xinlei  wang
maike  feng
jing  qiu
jinjin  gu
junhua  zhao
Venue: NeurIPS_cc_2024_Conference
Published: 2024-05-12 15:38:52

Abstract: This paper introduces a novel approach that leverages Large Language Models (LLMs) and Generative Agents to enhance time series forecasting by reasoning across both text and time series data. With language as a medium, our method adaptively integrates social events into forecasting models, aligning news content with time series fluctuations to provide richer insights. Specifically, we utilize LLM-based agents to iteratively filter out irrelevant news and employ human-like reasoning to evaluate predictions. This enables the model to analyze complex events, such as unexpected incidents and shifts in social behavior, and continuously refine the selection logic of news and the robustness of the agent's output. By integrating selected news events with time series data, we fine-tune a pre-trained LLM to predict sequences of digits in time series. The results demonstrate significant improvements in forecasting accuracy, suggesting a potential paradigm shift in time series forecasting through the effective utilization of unstructured news data.

Main paper: 
 introduction: 
 Time series forecasting [18, 21] serves as an essential foundation for decision-making across a wide range of economic, infrastructural, and social domains [2, 13, 14, 16, 56] . The purpose of analyzing time series data is to decode the evolving relationships within complex, dynamic realworld systems. Traditional forecasting methods, while effective at identifying patterns in historical data, perform well when time series distributions remain consistent over time. However, they have limitations in addressing sudden disruptions or anomalies caused by external random events, and do not systematically connect complex social events with fluctuations in time series data. Integrating insights from real-world events and their effects on social and economic behavior is crucial for improving the reliability and accuracy of time series forecasting.News articles provide crucial insights into unexpected incidents, policy changes, technological developments, and public sentiment shifts-factors that numerical data alone may not capture. Integrating news into forecasting enriches its inputs with context that closely mirrors the complexities of human behavior and societal changes. On the one hand, news offers a real-time snapshot of events, enabling the model to adjust predictions based on updated information. On the other hand, qualitative data from news sources enables the model to account for non-linear and non-numeric influences. By combining both quantitative and qualitative insights, the model can improve forecast accuracy, especially in rapidly changing environments, making it more reflective of real-world dynamics.In this work, we propose a unified approach that embeds news and supplementary information into time series data using textual prompts. By fine-tuning large language models (LLMs) [54, 55, 66] , we transform time series forecasting into the prediction of the next token in text, taking into account relevant contextual information. The inductive reasoning capabilities of pre-trained LLMs, along with their ability to model multi-modal distributions, enable few-shot predictions in time series [17] . The potential of language models in time series forecasting has also been proven [23, 71] . After further training on our dataset, which includes time series, news, and supplementary information, language models can generate forecasts that consider the textual context provided in the input prompts.Effective news filtering is a key issue for enhancing time series forecasting as input diversity increases. This task requires more than simple keyword extraction; it demands a deep understanding of how news elements interact with forecast variables, extending beyond linear reasoning to incorporate intelligent analytical methods. We employ LLM agents [43, 57] with advanced human-like reasoning capabilities for dynamic and effective news selection. These agents use few-shot learning to adapt their strategies based on logical scenarios that mimic human reasoning about factors affecting time series fluctuations. This enables them to identify relevant news, which is then paired with corresponding time series data to create a context-aware dataset that improves prediction accuracy. Additionally, LLM agents also play a crucial role in model evaluation, continuously refining their selection logic by comparing forecast errors with all relevant events. This iterative self-evaluation helps identify and integrate critical news items previously overlooked. By automating the analysis of unstructured text and applying chain-of-thought prompting, the agents effectively uncover patterns linking news events to forecasting discrepancies, revealing the nuanced effects of external factors on predictions.Our contribution can be summarized as follows:• The paper introduces a novel time series forecasting framework that integrates unstructured news data with numerical time series inputs, providing deeper contextual understanding and improving the model's responsiveness to social events and real-world dynamics.• The research highlights the use of LLM agents for dynamic news selection and analysis. We leverage the reasoning ability of LLM to automate the effective understanding and filtering of news content. The agents iteratively refine their news selection process based on forecasting results, improving model accuracy and reliability. • We propose a data construction method that integrates time series data with news information, and build a dataset spanning multiple domains to support our research. The dataset includes taskspecific time series data and verified public news reports, which facilitates further exploration in time series research. • Our experiment results demonstrate the effective integration of news data, achieving superior prediction accuracy across diverse domains such as energy, exchange rate, traffic, and bitcoin domains. Our findings demonstrate that incorporating news is highly adept at navigating complexities, especially in energy demand patterns. 
related work: 
 Time series forecasting. The traditional method of time series forecasting relies on analyzing historical data and utilizing statistical models to predict future trends, with an assumption that past patterns will persist in the future [8, 12, 20, 27, 42] . However, these methods were limited to smallscale datasets. The advent of deep learning [30] has introduced a range of time series forecasting networks [33, 34, 36, 40, 53, 59, 60, 67, 69] that excel in managing larger, more complex datasets by capturing non-linearities and dependencies directly from historical data. Recent advancements include pre-training on diverse, large-scale datasets, allowing models to be fine-tuned on specific tasks with fewer data and resources [6, 24, 58, 63] . While these methods continue to evolve and improve performance benchmarks, they often neglect the influence of external and contextual factors.Attempts to incorporate textual information (e.g., Twitter feeds, news articles, public reports) into time series forecasting have been made across various domains, such as finance [7, 48, 49] , energy [3, 41] , entertainment [26] , pandemics [65] , and tourism [47] . Traditional methods often simplify text analysis to counting keyword frequencies [41] or using dummy variables, which do not capture nuanced meanings. Advanced efforts include extracting richer textual features like word frequencies and sentiments using traditional NLP [9] and machine learning methods, as demonstrated by Bai et al. [3] . However, these approaches require labor-intensive feature engineering, struggle with long-text dependencies, and lack a deep contextual understanding. In contrast, LLMs excel in processing complex textual data and understanding contextual relationships, which can improve prediction accuracy and efficiency through automated feature extraction and enhanced scalability across multiple tasks. Despite their potential, no study has yet fully exploited LLM to enhance forecasting with their capabilities in understanding unstructured textual data.Language models for time series forecasting. LLMs such as the GPT series [1, 4, 44, 45] and LLaMa [54] have excelled in a variety of natural language processing tasks. With their vast parameter sets, LLMs acquire extensive general knowledge and reasoning capabilities during pre-training, which is crucial for building intelligent systems equipped with common sense. LLMs architectures are increasingly applied to time series processing and forecasting [23, 25, 35, 37, 51] . For instance, TEMPO [6] adapts GPT architectures for dynamic temporal representation learning, while TIME-LLM [23] utilizes LLMs for time series forecasting by reprogramming input data and applying Prompt-as-Prefix techniques. Similarly, FPT [71] demonstrates that even frozen LLMs can perform effectively in time series tasks, leveraging the universality of self-attention mechanisms. Lag-LLaMa [46] uses a decoder-only transformer for univariate probabilistic forecasting, and Gruver et al. [17] show that by framing time series forecasting as next-token prediction, LLMs can surpass traditional models through effective tokenization and adaptation. However, existing studies mainly utilize the mapping capabilities of LLMs for numerical regression, without incorporating external textual inputs or leveraging the reasoning abilities of LLMs in understanding language.Reasoning with language models. LLMs can automate tasks with human-like reasoning through "Chain of Thought" (CoT) prompting [57] , enhancing reasoning by step-by-step emulation of human thinking [28, 32] . CoT prompting is useful for transforming complex questions into answers by introducing intermediate steps. The "Tree of Thoughts" (ToT) approach [61] refines this by mimicking trial-and-error methods, enhancing auto-regressive LLMs with a prompter agent, checker module, memory module, and ToT controller for multi-round dialogues. LLM-based agents can solve tasks by reflecting feedback signals in text and retaining them in a memory buffer for better decisions [50] . Cai et al. [5] proposed the LLMs As Tool Makers (LATM) framework, where LLMs create reusable tools for problem-solving, interleaving reasoning and actions to aid task completion and interaction [62] . These agents can debate their responses and reasoning to arrive at final actions [11] . Example:• News: Train services were suspended in Melbourne after a train hit a car at a level crossing near Parkdale. VIC, 2019-06-12 3:41:00 PM. • Rationality: The suspension of train services may directly affect electricity load as alternative transportation and emergency services require increased power consumption following the incident.Example:• News: Optus has activated early parts of its 5G network, with more areas to follow; General Australia, 2019-01-30 14:05:00; • Rationality: 5G network expansion requires extensive infrastructure, increasing long-term electricity demands due to additional facilities and greater reliance on 5G technologies. Event Occurring Time Series-Embed live updates and projected timelines concerning construction activities, road shutdowns, and significant infrastructure ventures. Utilize adaptive algorithms to dynamically forecast alternative route usage and resultant traffic flow variations. 
method: 
 In this work, we aim to integrate news insights into time series forecasting. The development of such a system faces several challenges. Firstly, the forecasting method must handle unstructured, non-numerical news inputs flexibly and adjust predictions based on the context of the news events. Secondly, constructing this model involves filtering news and establishing connections between the news and the time series data. This requires sifting through vast amounts of internet data to find relevant information, demanding deep societal understanding and sophisticated reasoning skills. Therefore, an intelligent agent is designed to manage this complexity. Moreover, potential inaccuracies in news selection or inferential errors may still affect the forecast accuracy, requiring further refinement of news selection and reasoning based on the predictions. Our approach includes three main modules: a language model-based forecasting module (Sec 3.1), a reasoning agent for news filtering and inference, and an evaluation agent to assess and refine the forecasting model (Sec 3.2). The core workflow of our method and the interrelations between these modules are shown in Figure 1 . The subsequent sections will discuss these three modules in detail.Time series forecasting can be considered as a conditional generation problem of sequences [17] . This aligns with the general paradigm of natural language processing represented by LLMs. Taking the LLaMa language model as an example, assuming a number series {123,456}, LLaMa's tokenizer will regard this number as a sequence of digit tokens, i.e., {"1","2","3",",","4","5","6"}. Given the input series "123", the probability of predicting "456" can be represented as a probabilistic forecasting process in an autoregressive manner: P ("456"|"123") = P ("4"|"123") • P ("5"|"4", "123") • P ("6"|"45", "123"). Generally, denoting the time series tokens at time t as x t , the LLM predict the next token in the series x t+1 using the conditional probability distribution P (x t+1 |x 0:t ). During pre-training, LLMs optimize its internal parameters to maximize this conditional probability over a wide range of natural language corpora. Though counterintuitive, Gruver et al. [17] have shown that pre-trained language models exhibit a significant few-shot capability for time series forecasting. This shows the potential of language models in understanding input digital tokens, and also inspires us to use the language model as a reasonable platform to study how to introduce the information contained in textual prompt into time series prediction.News context offers critical insights into complex social events that traditional numerical data often overlook, and it also reflects sudden shifts in time series due to random events. In fact, the time series we collected can already be seen as being influenced by the aforementioned events. Assume an event E and a time series x 0:t , its impact on the future sequence can be expressed also as a conditional probability P (x t+1 |x 0:t , E). However, when information about event E is not provided, we can only predict through historical time series. Although time series data itself can show patterns and trends, it lacks the ability to indicate the causality behind events. The event information E offers the context needed to understand why certain spikes or drops occur. We show in Figure 2 some news that are closely related to time series forecasting.In language models, news events can also be represented as text tokens. Consider a set of news text tokens {e 0 , e 1 , . . . , e u }, which represent event E. LLMs treat this news information as a condition input and perform conditional probability predictions P (x t+1 |x 0:t , e 0:u ). Including e 0:u provides crucial contextual information that influences the prediction of future values. This process aligns with the standard approach used by language models to interpret text, allowing for the incorporation of information about multiple events through various news contexts simultaneously to enhance prediction accuracy. In practice, we only need to integrate news text with historical time series data in a prompt engineering manner.Other supplementary information also provide contextual information for the forecasting model. For example, weather and climatic factors may affect energy demand and industrial output; financial indicators and economic metrics influence consumer behavior and business operations. Including this diverse range of information allows models to adjust for environmental, economic, and seasonal variations, improving prediction accuracy. The supplementary information can also be understood as conditions and integrated into the above conditional probability forecasting framework. Our approach incorporates this information into language for flexible integration with language models. For example, we use the text "Weather on historical dates: the lowest temperature is 292.01; the highest temperature is 298.07; the humidity is 94." to express weather conditions. This enriches the input to cover various factors affecting the time series. Part (c) of Figure 1 illustrates the method of prompt integration for time series forecasting and the corresponding responses.Fine-tuning LLMs for time series forecasting. Integrating the above information, we can construct inputs for LLMs to perform time series forecasting. Although pre-trained LLMs are already capable of generating time series predictions to some extent, relying on these pre-trained models for few-shot predictions in such a context-rich environment poses several challenges. Firstly, controlling the output of time series is difficult; predicting long sequences of numerical digit tokens is uncommon for LLMs. Secondly, the connections between the news and supplementary information and the time series typically need to be derived from historical data, which goes beyond the usual scope of using LLMs for few-shot time series predictions. To enable language models to more effectively forecast time series while considering the conditions imposed by news and supplementary information, we propose fine-tuning the language models to predict conditional probabilities. We employ a supervised instruction tuning method to train LLMs on historical time series data paired with corresponding news and supplementary information, formatted into text input-output pairs (shown in Appendix A.3).The same loss function used during pre-training is applied here. To fine-tune the LLM, we use the Low-Rank Adaptation (LoRA) method [19] , which updates only a small subset of parameters, reducing computational demands while retaining most of the pre-trained knowledge. This strategy allows the model to efficiently adapt to new forecasting tasks without losing its foundational strengths.Next, we construct a dataset to train the above model. While obtaining time series data is relatively straightforward, matching it with appropriate news and supplementary information is not trivial. The internet is flooded with news, most of which are irrelevant to the time series we aim to forecast. Introducing irrelevant news can disrupt forecasting. Therefore, it is crucial to analyze the relevance and causality between the time series forecasting task and the select news accordingly. However, gaining such an understanding is complex, requiring knowledge of human societal mechanisms and logical reasoning skills. In our work, we utilize LLMs for filtering and reasoning about news content. We also recognize that even the most advanced language models cannot complete all reasoning and judgment in a single generation process. We explain how to use a combination of multiple LLM generations to create an intelligent agent that fulfills the complex requirements of news filtering.Time series and news pre-pairing. For the initial stage of data preparation, news is retrieved to align with time series data based on matching time frequencies, horizons, and geographical areas. This synchronization ensures that insights from textual information are timely and regionally relevant. For example, to understand state-level electricity demand in Australia from 2019 to 2021, we gather local news from various Australian states and international news occurring in the same period that might directly or indirectly affect demand. In this way, potentially relevant candidate information can be roughly selected first, and such filtering can be easily completed through crawler means. Reasoning agent for news selection. We employ an LLM-based reasoning agent capable of sophisticated tasks such as conversation, reasoning, and semi-autonomous action. This agent is programmed using detailed prompts that define roles, instructions, permissions, and context, enabling it to interpret human commands and perform complex tasks. This approach condenses extensive news datasets into a refined selection of pertinent articles. It leverages its reasoning capabilities to effectively screen, categorize, and interpret news texts. We employ few-shot prompting [4] and the CoT [57] method to develop an agent that understands the context of news for forecasting. This technique enhances the model's ability to handle multi-step, commonsense reasoning tasks essential for accurate forecasting. The agent uses multi-step prompts to break down complex problems into simpler, manageable parts. Our three-phase prompting method, illustrated in Figure 3 Part I, includes:1. We develop an understanding of time series influencers, sorting them by impact (positive/negative) and duration (short/long-term), considering economic, policy, seasonal, and technological factors;2. We direct the agent to filter and categorize news based on either automatically generated logic or a given reasoning logic, focusing on relevance to time series and classifying the impact (e.g., long-term and short-term) along with the rationale;3. We specify the output format for the agent to organize the selected news into JSON, detailing aspects like summary, affected region, reporting time, and rationale. More details about our prompting method can be found in Appendix A.6.The LLM agent can automatically develop an understanding of time series influencers, with the option to provide predefined reasoning logic in our models. In the automated process, the agent forms its logic through prompts designed to guide it in determining how different types of news impact a specific domain. For instance, we use open-ended questions to allow the agent to independently summarize and create its own filtering logic. User knowledge can also be incorporated into these prompts as additional reasoning, enabling the agent to generate more comprehensive logic. The agent then filters news based on the generated logic, whether fully automated or with user-provided input.Evaluation agent for reasoning updating. We also design an evaluation agent to assess and improve the effectiveness of the aforementioned news filtering. Relying solely on the reasoning agent for news selection is not optimal, as the interaction between news and time series is complex. The reasoning agent can only analyze the potential impact of different news from the perspective of news content, without knowing whether the trained time series forecasting model based on them can make accurate forecasts. The evaluation agent is deployed after the time series prediction model has been trained. The evaluation agent extends beyond simple numerical assessments of prediction accuracy by incorporating human-like logical reasoning to refine the news selection logic chain. We focus our evaluation on identifying inaccuracies potentially caused by missing news, such as unusual events or illogical reports. It observes the model's predictive outcomes to determine if any crucial news has been overlooked and adjusts the news filtering strategy for the training data based on these results. Figure 4 : Overall pipeline iteratively combines news reasoning agents, fine-tuning, and evaluation agents.For the evaluation agent, we structured the prompt design into three phases, as illustrated in Figure 3 Part II. In the first phase, we input the forecasting task type, the time horizon, and background information, which the agent uses to generate the steps for evaluating the prediction outcomes. In the second phase, we provide the ground truth, the discrepancies between predicted and actual series, as well as selected and historical news. The agent analyzes these inputs to identify overlooked news based on the distribution of prediction errors over time. In the third phase, the agent generates updated logic based on its analysis, guiding future news selection. After processing all validation set predictions, the reasoning agent consolidates the updated logic into a cohesive final strategy.We integrate the news reasoning and evaluation agents with the fine-tuning of the LLM forecasting model to enhance the quality of training data, as illustrated in Figure 4 . In the first iteration, we use the LLM agent to establish news selection logic based on the domain and timing of the time series task. This logic directs the reasoning agent to filter relevant news, align it with time series data, and input it into the model for initial fine-tuning. After validating the model's predictions with a validation set, which is randomly extracted from the training data for each iteration, the evaluation agent checks for any missing news that may have influenced the prediction. This feedback helps the reasoning agent refine the filtering logic in subsequent iterations. The cycle continues until the final iteration, where the reasoning agent consolidates all updates to create the definitive news filter for training the final model. We use the GPT-4 Turbo model as the LLM for the agents described above.-Harness data from local event schedules and public announcements to anticipate the traffic implications of major events such as sports games, concerts, or festivals. Predict substantial increases in local traffic volumes as participants commute to and from locations. 
data preparation: 
 Time series data. We selected time series data from domains influenced by human activities and social events to test our method's ability to capture complex human-driven dynamics during forecasting. These domains include Traffic [39] (traffic volume), Exchange [29] (exchange rate), Bitcoin [15] (Bitcoin price), and Electricity [15] (Australian electricity demand). To avoid bias from pre-trained language models, we updated the Exchange and Electricity datasets up to 2022. We use half-hourly electricity demand data from the Australian Energy Market Operator (AEMO) (aemo.com.au) and daily exchange rate data from the Exchange Rates API (exchangeratesapi.io). These datasets vary in frequency, including daily, hourly, and half-hourly updates, allowing us to evaluate the algorithms' effectiveness across different temporal resolutions. More details are in Appendix A.1. News collection. Since there are no public datasets that pair time series data with news events, we have collect news specifically for the above time series to facilitate our research. Some of the news content is collected from the GDELT dataset [31] , a database tracking news from nearly every country in over 100 languages. GDELT provides real-time insights into societal, political, and economic events, enabling detailed analysis of global trends and their effects. We incorporate GDELT's event information into our forecasting models to enhance predictive accuracy. For domains needing the latest information, we collect real-time news from sources like News Corp Australia (news.com.au) and Yahoo Finance (yahoo.com), focusing on region-specific and task-specific activities.Supplementary information. We enhance our forecasting models with open-source tools to grasp additional data for better accuracy and context. Weather information from OpenWeatherMap [10] provides daily temperatures, atmospheric pressure, wind speed, and humidity, crucial for load forecasting. Calendar dates, obtained using the Python packages datetime and holidays, account for seasonal and cyclical effects. Economic indicators are integrated using the pandas_datareader library, accessing data like GDP, inflation rates, and employment statistics from sources such as the Federal Reserve, World Bank, and international financial markets.Effectiveness of news intergration. In our approach, we incorporate news and supplementary information into time series forecasting by fine-tuning language models. Firstly, we assess whether this additional information can enhance time series forecasting. We conducted experiments to verify the necessity and effectiveness of integrating news data into our forecasting model. We compared four different scenarios as detailed in Appendix A.2 and Appendix A.3:1. Pure numerical tokens: Uses numerical tokens, encompassing all variables without news. Except for region names or date information, it excludes other textual tokens as a baseline for comparison. 2. Textual descriptive sentence tokens: Evaluates whether using sentence-form descriptions instead of only raw digital numbers can enhance accuracy, with no news integration included. 3. Unfiltered news with textual descriptive sentence tokens: Assesses how integrating descriptive sentences of time series with unfiltered news data affects the model's performance. 4. Filtered news with textual descriptive sentence tokens: Shows the effects of integrating descriptive sentences with news that has been specifically filtered for relevance by the proposed agents. The performance of different prompt designs is presented in Table 1 . It can be seen that the fine-tuned LLM can be used to forecast time series even when using only digital tokens as prompts. The introduction of proper news and other supplementary information leads to significant performance improvements across all four domains. Nevertheless, such improvements are not easily achieved. If the introduced news information is not carefully selected, it can severely impair the results. There are two main reasons for this: first, the influx of a large number of news items introduces too many tokens, which can decrease the performance of the LLM as the number of tokens increases. Second, irrelevant news can introduce noise and incorrect causal information, leading to misleading predictions.Effectiveness of the evaluation agent. To make our news filtering and reasoning processes more effective and comprehensive, we introduce an evaluation agent to reflect on and improve the effects of news selection based on prediction outcomes. As shown in Table 2 , our evaluation agent refines news filtering through an iterative process, which is reflected in the progressively improved time series prediction results. The result corresponding to each case is the prediction outcome of the model after pairing the news selected according to the logic obtained from the corresponding iteration. Our findings suggest that, in most cases, two iterations are sufficient to achieve significant improvements, with multiple iterations consistently yielding better results than a single iteration due to the reflection mechanisms. We also found that this iterative filtering process reveals interesting insights into human society from the language model agent. These examples are presented in the Appendix A.7.Compare to other forecasting methods. We also compare our method with existing time series forecasting techniques, detailed in Appendix A.1. We show a quantitative comparison against these baseline methods in Table 3 . While the baseline methods use inverse normalization to revert predictions to their original scale, our model operates without normalization. This approach retains Auto. [59] In. [67] Dlin. [64] iTrans. [38] FiLM [68] Times. [58] Pyra. [36] PatchTST [40] FED. [ the physical meaning of data, such as electricity demand or economic indicators, ensuring that our outputs remain interpretable. Normalization could obscure the impact of news events due to the nonlinear and scale-dependent relationship between these events and the original data values.Our approach significantly outperforms traditional methods that rely solely on historical time series data in domains like electricity demand, exchange rates, and the bitcoin market, where events embodied in the news have substantial impacts. This demonstrates the potential of our method. However, the improvement from integrating news into the traffic sector is notably modest. The performance of our traffic forecasting model, which covers all roads in California, is hampered by the coarse granularity of publicly available news data, lacking the local details necessary for precise predictions. Traffic data primarily reflects specific road traffic flows, whereas our news sources are mostly regional or global, failing to capture localized traffic conditions adequately. This limitation is evident in the model's Mean Squared Error (MSE), which is sensitive to outliers and tends to exaggerate errors from traffic spikes that state-level news often does not report. Our model achieves good results with the Mean Absolute Error (MAE), indicating reliable average accuracy. Incorporating more localized road information could potentially improve these issues.Figure 5 takes the electricity domain as an example and compares the ground truth with predictions by cases with or without news data, demonstrating the effect of incorporating news into forecasting models. The "With News" predictions are closer to the actual values than the "No News" predictions, particularly at critical timestamps where abrupt events significantly influence electricity demand.-Evaluate economic influences such as openings of new businesses, significant layoffs, or industry closures to determine shifts in commuting patterns, focusing on movements toward or away from economic centers. 
conclusion and discussion: 
 In conclusion, our study demonstrates the benefits of integrating news into time series forecasting using LLM-based forecasting method and LLM-based agents. These agents enhance model intelligence by autonomously identifying and addressing missed news, refining their logic, and assessing the impact of events on predictions. Our findings advocate for incorporating extensive domain knowledge, encouraging a shift towards more nuanced and context-aware forecasting. This approach enriches time series forecasting for adaptive, comprehensive forecasting aligned with real-world dynamics.Limitations of our approach. While our approach demonstrates that LLMs like LLaMa 2 [55] can enhance time series forecasting by integrating news, there are limitations to its applicability. The effectiveness of news integration is primarily evident in domains where human and market activities significantly influence trends. Our framework is less suitable for domains requiring precise meteorological modeling or where human activities have minimal impact, such as in meteorological or physical data. Additionally, the model is constrained by the maximum token length of pretrained LLMs, complicating the simultaneous processing of large amounts of time series or multiple sequences, which can lead to data truncation and affect the accuracy of long-term predictions. Finally, our strategy enhances rather than completely replaces traditional time series tasks like classification or interpolation across all fields. Our aim is to demonstrate that by leveraging language models, it is possible to incorporate useful textual information to enhance time series prediction tasks.Future work. Future enhancements will focus on extending the current forecasting model's scope in several key areas. Firstly, attribution analyses of news content used in the model will pinpoint which factors most significantly impact forecasting accuracy, facilitating an optimized news integration process. Advanced analytical toolkits can also be provided to reasoning agents, enabling sophisticated data processing and real-time application of complex analytical techniques. These developments will enhance the precision and relevance of the time series prediction model, contributing deeper contextual insights and expanding its applicability in the predictive analytics field.Broader impact. Ethically, it is crucial that we conduct thorough reviews to ensure that our use of news content does not inadvertently perpetuate biases or negatively influence public opinion. This involves implementing rigorous checks for accuracy and balance to avoid the risks associated with misinformation, ensuring that our data sources are credible and that the content is factually correct. Furthermore, the potential misuse of news, especially the spread of "fake news", highlights the need for our models to incorporate sophisticated mechanisms to verify information reliability before integration. Beyond the discussed sectors, this approach has the capability to extend into forecasting GDP trends, analyzing carbon emissions, or predicting public health outcomes, each carrying significant implications for policymaking and public welfare. Thus, while our research offers substantial benefits in enhancing predictive analytics, it also obligates us to handle these capabilities responsibly, ensuring our contributions positively impact economic planning, environmental strategy, and informed decision-making across various domains.-Incorporate latest modifications in traffic rules or public transit changes into the models to predict changes in travel behaviors, timings, and route preferences. 
emergency and incident response:: 
 -Employ real-time monitoring to swiftly adjust traffic volume predictions in response to accidents, disasters, or other unforeseen emergencies that could result in road obstructions or necessitate evacuations. 
political and social influences:: 
 -Factor in the effects of political rallies, demonstrations, and changes in government policies on road occupancy and public transit, tailoring predictions to the anticipated influence and size of such events. 
safeguards: 
 Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?Answer: [Yes]Justification: The paper ensures responsible usage and mitigating risks associated with potential misuse.Guidelines:• The answer NA means that the paper poses no such risks.• Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters. • Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images. • We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.12. Licenses for existing assets Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?Answer: [Yes]Justification: The paper credits the creators of any used assets.Guidelines:• The answer NA means that the paper does not use existing assets.• The authors should cite the original paper that produced the code package or dataset.• The authors should state which version of the asset is used and, if possible, include a URL. • The name of the license (e.g., CC-BY 4.0) should be included for each asset.• For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided. • If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset. • For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.• If this information is not available online, the authors are encouraged to reach out to the asset's creators. 13. New Assets Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets? Answer: [Yes] Justification: The paper introduces new assets which are well documented alongside the paper.Guidelines:• The answer NA means that the paper does not release new assets.• Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc. • The paper should discuss whether and how consent was obtained from people whose asset is used. • We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution. • For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review. 
 
 Appendix: A.1 Experimental Setting / Details Baselines. To ensure a thorough evaluation of our model, we selected a diverse set of baselines that represent the latest advancements in time series forecasting, covering a broad spectrum of empirical studies. Our selection includes transformer-based models, such as transformer-based model: Informer [67] , Autoformer [59] , FEDformer [69] , Pyraformer [36] , PatchTST [40] , iTransformer [38] , FiLM [68] . Additionally, we included the CNN-based Timesnet [58] and MLP-based Dlinear [64] . Finally, we incorporated the LLM-based GPT4TS [70] , leveraging GPT2's generative capabilities. For Informer [67] , Autoformer [59] , FEDformer [69] , Pyraformer [36] , PatchTST [40] , iTransformer [38] , FiLM [68] , and Timesnet [58] , we use the code from the Time-Series-Library (https://github.com/thuml/Time-Series-Library/tree/main). For the GPT4TS method, we use the official code from https://github.com/DAMO-DI-ML/NeurIPS2023-One-Fits-All.When conducting comparative experiments, all the information utilized in our method, except for the news content, is also provided to the baseline methods. While our LLM fine-tuning approach maintains the original scale of the numbers to preserve the physical meaning of the time series data, we normalize the variables to the range [0, 1] for training the baseline methods. For non-numeric variables, we represent them using dummy variables for these baseline methods. To ensure optimal performance, we adhere to the official architecture settings for these methods and experiment with learning rates of 0.001, 0.0005, 0.0001, and 0.00005. Each configuration is tested three times with different initializations, and the best-performing setup is selected from all trials.Our method. When fine-tuning, we employ the LoRa [19] method to fine-tune the LLama 2 large language model [55] . The rank in LoRa is set to either 8 or 16, depending on the required token length to be processed. The alpha value in LoRa is set to 16, and the learning rate is set at 0.0001. When formatting numeric tokens, we uniformly maintain three significant figures. Retaining more significant figures offers minimal benefits and introduces an excessive number of tokens. The finetuning is conducted on a single NVIDIA A100 40G GPU, with the large language model undergoing several hundred to 1,000 training iterations on our curated data, requiring several hours.Evaluation. We choose five metrics to compare the prediction performances of different forecasting models. Mean squared error (MSE) is used to verify outliers, focusing on larger prediction errors. Root Mean Square Error (RMSE) measures error magnitude by averaging squared errors and taking the square root, aligning the error scale with the original data. Mean absolute error (MAE) is more balanced for different magnitude errors. Mean Absolute Percentage Error (MAPE) expresses the error as a percentage of the actual values, that is, on average how far off the predictions are from the actual values in percentage terms. The accuracy rate is defined as the proportion of true results among the total number of cases examined. These metrics are calculated by:EQUATIONwhere y i represents the unnormalized true value of load, and ŷi denotes the unnormalized result of the prediction model. n denotes the number of prediction timestamps.A. We analyzed the classical time series forecasting datasets, such as the Monash-TSF dataset, to identify the specific time periods covered by the time series data. For the traffic domain, data from 2015 to 2016 was used. For the Bitcoin domain, data from 2019 to 2021 was utilized. Relevant news information was filtered from the GDELT dataset using domain-specific keywords corresponding to these periods. We conducted web crawling and intelligent parsing of the associated news web pages. The traffic data was sourced from Yahoo, while the Bitcoin data was gathered from various websites. Regular expressions were employed to extract essential information from the parsed text, including news titles, URLs, publication dates, and the main content of the articles. From the GDELT dataset, we initially filtered 14,543 traffic-related articles and 19,392 Bitcoin-related articles. After removing invalid and redirected links, we retained 5,867 traffic articles and 5,906 Bitcoin articles. The traffic-related news data primarily covered the region of California, USA. The extracted data was formatted into JSON files to facilitate seamless integration into our model training. For the region-specific tasks, such as AU electricity demand and AU exchange rate, we collected news articles spanning from 2015 to 2023 primarily sourced from news.com.au. In total, we gathered 380,560 articles covering a diverse range of topics pertinent to electricity demand and exchange rates in Australia. The news articles were processed using a similar methodology as described for the traffic and Bitcoin domains. We performed web crawling and intelligent parsing to extract key information from each article, including titles, summaries, categories, URLs, publication dates, and the full article contents. Regular expressions were utilized to ensure the accurate extraction of essential data points. The structured data was then formatted into JSON files, ensuring compatibility and ease of integration into our model training processes. By maintaining a consistent data collection and processing methodology across different domains, we ensured the reliability and utility of the gathered information. Random events are unpredictable and unplanned, such as natural disasters, accidents, health crises, and criminal acts. Normal events are planned or anticipated based on patterns, like political activities, sports & cultural events, economic reports, and public holidays. We used the LLM agent to categorize and detect all random and normal events from our raw news dataset spanning January 1st to August 5th, 2019. The analysis revealed that, on average, 27.7% of all events are random. Figure 7 and Figure 8 show the daily distribution of these random events. In our framework, the agent analyzes and selects the most relevant news, which mainly consists of five categories: economic or political events, health crises, natural disasters, technology development, and social sentiment. Additionally, the reflection agent, which analyzes prediction errors in the training dataset and missed news, helps identify unexpected and counterintuitive events buried in the raw news.Table 6 : We tested the effects of removing all or partial supplementary information, including Case 3, which includes only Filtered News, and Case 4, which includes Filtered News and Partial Supplementary Information. Removing all supplementary information and keeping only the news led to slight improvements compared to using only Supplementary Information in a numerical/textual prompt. However, this approach did not perform as well as Case 4. The results indicate that supplementary information remains essential. Overall, "Case 6: Textual Prompt with Filtered News and all Supplementary Information" which includes all supplementary information and filtered news, yields the best results. We tried other language models using our proposed method. Mistral v0.1 [22] , a 7B model, produced similar results as Llama 2 (7B). The Gemma 2B model [52] had slightly worse results, which may be due to its limited number of parameters. It is necessary to adjust the training for small models to achieve better results. Nonetheless, the results demonstrate the potential of language models to achieve good performance in our proposed methods. A.5 More ResultsFigure 13 , Figure 14 , Figure 15 , and Figure 16 provide showcases of some forecasting tasks results.It can be seen that our method has better results in predicting some sudden events and cases with distribution changes. We also tested the effectiveness of fine-tuned LLMs on traditional time series forecasting datasets. Given that long-term forecasting involves a significant number of digits and long sequences occupy many input tokens, we selected the Ill dataset for testing our method, which features prediction sequences of lengths 24, 36, 48, and 60. We also evaluated the performance of univariate forecasts and trained univariate versions of other baseline methods for comparison. The results, displayed in Table 9 , show that, although counterintuitive, the fine-tuned LLMs can achieve performance comparable to baseline methods, providing a solid foundation for our research. Table 6 shows additional comparison experiments of different types of data incorporation, and Table 7 shows the comparison of using different pre-train language models. The Daily Unemployment rate (unit: %) of the United States during the last week was 6.4,6.4,6.4,6.4,6.4,6.4,6.4 . The Daily Interest rate (unit:%) of the United States during the last week was 0.1,0.1,0.1,0.1,0.1,0.1,0.1. The date of prediction starts on 2021-1-10, which is the Weekend, and it is not a public holiday. The Daily GDP of Australia on the prediction date is 568773. The Daily Unemployment rate (unit: %) of Australia on the prediction date is 6.2556. The Daily Cash Rate Target (unit: %) of Australia on the prediction date is 0.1. The Daily GDP of the United States on the prediction date is 22600.2. The Daily Unemployment rate (unit: %) of the United States on the prediction date is 6.4. The Daily Interest rate (unit: %) of the United States on the prediction date is: 0.1."Prompt 3: This is the news we used for the prediction: <selected_news>; Here are all the news, including the day-ahead news and today's news in JSON format: <all_news>. Please determine whether there was any news that should have been considered in the prediction but was not included. The actual value is <actual_values>. Predicted values minus actual values are <errors>. The prediction date is <predictions_time>. According to the news and errors on this day, determine if any news has been missed. The output format should be: The missed news is xxx, occurred at xxxx, the possible reasoning is xxxx.Answer: Examining the provided news on 2021-01-09 for potentially unconsidered factors in the AUD/USD exchange rate prediction, one significant event stands out that could resonate with economic implications, specifically concerning Australia as a significant commodity exporter: • Missed News Summary: "Brisbane's streets have been empty in eerie scenes as the city experiences its first day of lockdown to stop the spread of coronavirus." • Publication Time: 2021-01-09 12:59:00 • Category: Breaking News • Possible Reasoning: This news shows a potential shift in economic activity due to new COVID-19 lockdowns in a major Australian city, which could affect the AUD due to changes in both domestic consumption and possibly a short-term impact on business conditions. Lockdowns can create short-term economic disruptions influencing investor sentiment and risk perception towards the AUD, potentially missed in the prediction model. These items suggest an underestimation of contemporaneous socio-economic events that might indirectly influence the AUD/USD exchange rate through shifts in economic expectations and risk sentiment related to the management of COVID-19, which would have been highly relevant for predictions made for 2021-01-10. 1. Increased Risk Aversion: The lockdown in Brisbane may heighten risk aversion among investors, leading to a potential decline in the AUD as investors flock to safe-haven currencies like the USD. This can be anticipated as a prevalent reaction whenever there are renewed concerns about COVID-19 impacts. 2. Domestic Consumption Impact: With the Brisbane lockdown, domestic consumption could be considerably reduced. A decrease in consumer spending within Australia might result in lower economic output forecasts, potentially weakening the AUD. 3. Market Sentiment and Speculation: News of the lockdown and its impact on one of Australia's significant cities can drive market sentiments and speculative trades, which often have immediate effects on currency exchange rates. Negative news could prompt a bearish sentiment towards the AUD. 4. Impact on Commodity Prices: As a major commodity exporter, any perceived slowdown in Australia's economy due to lockdowns can affect global commodity prices, influencing the AUD. With a decrease in economic activities, demands for commodities might decline, causing a domino effect on the currency. 5. Investor's Perception of Economic Management: Prolonged or frequent lockdowns might affect international perceptions of Australia's management of economic challenges during the pandemic. This perception can influence foreign direct investment flows and currency strength.Prompt 1: Improve and polish this paragraph to reduce repeated content and summarize the news selection logic that affects the Australian dollar exchange rate:<all_updated_logic> Prompt 2: According to the given updated logic, please directly rephrase the current prediction logic and output the adjusted new logic. This is the current prediction logic that you need to adjust and improve:<selection_news_logic_current>By analyzing the relationship between prediction errors and news through the agent, we can identify missing news (such as events with indirect impacts) and adjust the agent's expectations of how news events affect time series. For example, the agent might discover that a certain type of incident has a relatively small impact on a specific region. These intelligent and flexible discoveries are detailed in the missed news summary. Here are some examples of missed news during the iterative analysis: Possible Reasoning: Although this did not happen in Australia, a deeper analysis of its impact on the AUD could enhance the prediction framework. This shift towards sustainable energy from a major oil producer could influence global oil prices and economic stability. As a significant exporter of commodities, Australia's economy and the AUD could be affected. The zero deviation between predicted and actual values suggests either accurate forecasting by other models or a delayed effect on exchange rates. Further analysis may be needed to understand the long-term impacts on currency values.3. Missed news: " US Senator Bernie Sanders, one of the most vocal critics of Donald Trump, has launched his presidential bid to a crowd of thousands of supporters." Occurred at 2019-03-03 04:30:00. The possible reasoning is that political announcements in a country like the US can significantly impact global financial markets, including forex. Bernie Sanders's candidacy news could have influenced market speculation and investor sentiment, affecting USD values and other major currencies. This detection suggests that incorporating global political news from major economies like the US might enhance the model's predictive capabilities. Significant US political events, like presidential candidacies, can influence global markets and forex currency fluctuations. Thus, overlooking major political news can leave a model vulnerable to missing market reaction surprises.(a) Australian Housing Market News: Missed News: "Despite economic uncertainty throughout 2020, Australians confidently embraced the housing market pushing sales volumes beyond even 2019 levels." Occurred at: "2021-01-17 10:00:00". Possible Reasoning: Although this news might seem domestically focused, the robust activity in the housing market can reflect economic confidence and potentially boost investor sentiment towards the AUD. Healthy real estate market metrics often attract foreign investment and can have favorable ripple effects on currency strength. (b) Australian Government Seeking Pfizer Vaccine Information: Missed News: "The Australian government is seeking 'immediate' advice and information after Norway reported 29 deaths related to the Pfizer vaccine." Occurred at: "2021-01-17 10:10:00". Possible Reasoning: News related to vaccine concerns could create shortterm caution among investors, especially in contexts where the vaccine rollout impacts economic recovery prospects. If international observers view the vaccine issues as a slowdown to Australia's reopening or economic normalization, it could temper enthusiasm for the AUD.The news is finally filtered out by the agent based on the prediction task, and the rationale is explained. As shown in Table8, each forecasting domain has a corresponding example of selected news. Predictive Considerations: Predicting Bitcoin's price movements based on news requires not only understanding the news itself but also how these events fit into the broader economic, technological, and regulatory landscape. Analysts typically monitor a wide range of sources and use both quantitative and qualitative data to gauge market sentiment and potential price movements. Cultural perception and adoption levels in different regions also play significant roles. 1. Weather Impact Analysis:-Integrate real-time and anticipated weather conditions, particularly severe phenomena like wildfires, storms, and snow. Forecast immediate and potential effects on traffic, including road closures and decreased travel speeds due to hazardous conditions.-Scope and Magnitude: Assess local versus statewide impact by determining the scale of occurrences to better predict scope -Timing Considerations: Analyze event timing relative to traffic peaks to anticipate their immediate impact on traffic. -Event Predictability: Differentiate between scheduled, routine events and unexpected emergencies to enhance predictive precision. -Geographic Considerations: Evaluate the geographic location of events to discern if impacts are localized or widespread, assisting in targeted traffic management. -By refining these components, the predictive system is boosted in its capability to efficiently manage, respond to, and adapt to evolving traffic conditions across California, leading to improved traffic circulation and less congestion. Justification: The paper includes a separate "Limitations" section. Guidelines:• The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper. • The authors are encouraged to create a separate "Limitations" section in their paper.• The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be. • The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated. • The authors should reflect on the factors that influence the performance of the approach.For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon. • The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size. • If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness. • While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations. • If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics. • The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction). 10. Broader Impacts Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed? Answer: [Yes] Justification: The paper provides a balanced discussion of the potential societal impacts. Guidelines:• The answer NA means that there is no societal impact of the work performed.• If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact. • Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.• The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster. • The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology. • If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML). 


Decision: This paper introduces a novel framework to integrate unstructured news with time series data for time series forecasting by using Large Language Models (LLMs). Empirical experiments demonstrate the effectiveness of the proposed framework by a noticeable improvement over baselines.

All the reviewers are positive about this submission. They also agree with the significant contribution of the paper such as novelty, strong empirical results. Concerns and issues raised by reviewers were properly addressed during the discussions with authors. They also promise to incorporate this improvement in the camera-ready version which we’re looking forward to.

Overall, this paper presents a solid and interesting work, I would love to recommend its acceptance .

Reviews: review: paper_summary: paper_summary: The paper proposes a novel method to integrate event news as external information into the time series forecasting system.
strength_weakness: strength_weakness: 1. Some questions regarding the experiments need clarification.
questions: questions: 1. Though the idea that relevant event news may positively benefit time series forecasting is intuitively correct, this is not well demonstrated in the paper regarding the datasets used. Specifically, for many datasets tested, it’s hard to imagine what kind of news could dramatically affect them. Of course, it’s impractical to manually evaluate the news considered relevant. Besides the examples already provided, the authors could also test the average number of relevant news items per time window. This could give a rough idea of the distribution of relevant news, which can be used to approximate if the LLM works as expected.
2. Conflicting news could exist within a time window. For example, Elon Musk may praise or criticize a cryptocurrency within a short time frame. It seems that the model doesn’t consider this situation. Can the authors elaborate more on this issue?
3. The iterative analysis results show that the performance after each iteration is somewhat random. Though, in general, the final results are better than the initial iteration, it’s actually hard to predict if one iteration will be better or worse between two adjacent iterations. This raises a concern since the huge computational cost may seem unnecessary.
4. Another concern is that only LLaMA 2’s behavior is tested. Therefore, the sample set of LLMs tested is quite small, making it difficult to predict the behavior of other LLMs using the proposed method.
limitations: limitations: N/A
paper_summary: paper_summary: This paper introduces a novel approach to enhance time series forecasting using Large Language Models (LLMs) and Generative Agents. By integrating news content with time series data, the method aims to align social events with fluctuations in time series to provide enriched insights. The approach involves filtering irrelevant news and employing human-like reasoning to evaluate predictions, continuously refining the logic of news selection and the robustness of the model's output. The results show significant improvements in forecasting accuracy by effectively harnessing unstructured news data.
strength_weakness: strength_weakness: - The performance heavily relies on the relevance of the news data selected. I'm worried that Inaccurate or irrelevant news can degrade the forecasting accuracy.
- The method may not perform as well in domains requiring highly localized or specific news data that is not available in general news sources.
questions: questions: - How does the model ensure the relevance of the selected news items? Could the authors provide more details on the filtering criteria and logic used by the LLM agents?
- How does the model mitigate the impact of irrelevant or misleading news on the forecasting results? Are there any mechanisms in place to detect and exclude such news?
limitations: limitations: The authors have addressed some limitations, such as the dependency on the relevance and quality of news data and the complexity of integrating textual and numerical data.
paper_summary: paper_summary: This paper proposes a new framework for time series forecasting. This framework fine-tunes a generative large language model (LLM) to improve forecasting accuracy by integrating news and supplementary information with numerical data and introducing iterative self-evaluation through LLM-based agents.
strength_weakness: strength_weakness: Weakness 1: There could be some statistical analysis on random events compared with normal events which represent a universal knowledge distribution with time going by. 

 

Weakness 2: It seems a bit redundant and contradictory for methods 1) and 2). The description of the three-phase prompting could be better organized. 

 

Weakness 3: Ablation studies and sensitivity analyses are encouraged. Based on the description of the four scenarios from line 290 to line 297, the news and the supplementary information are always integrated throughout the experiments. 

 

Weakness 4: There could be more detailed analysis for Table 2, especially regarding the roles of the evaluation agent.
questions: questions: Question 1: Apart from prediction accuracy, how to demonstrate the improved reliability? From line 144 to line 145, do sudden shifts embedded in random events from news context help the framework improve prediction reliability?  

 

Question 2: From line 216 to line 217, is the understanding of time series influencers, or the sorting based on impact and duration, developed manually by people or automatedly by the LLM agent? What is the difference between such an understanding and a given reasoning logic mentioned from line 218 to line 219?  

 

Question 3: From line 277 to line 278, apart from news articles, are there any ablation studies studying the accuracy differences between keeping and removing partial components of or the entire supplementary information? 

 

Question 4: According to Table 2, why would introducing more rounds of reasoning selection decrease the forecasting accuracy?
limitations: limitations: The authors have adequately addressed the limitations and the potential negative societal impact of their work.
paper_summary: paper_summary: This paper introduces a time series forecasting framework, where LLM-based agents are employed to sift out relevant news to time series of interests and the news are utilized to enhance the accuracy of time series forecasting models.
strength_weakness: strength_weakness: 1. Time consumption of the framework should be discussed.
2. The components in the framework are inherited from existing works, making the model itself not as innovative as the idea of the paper.
3. Collecting-then-filtering mechanism of the proposed framework might have negative effects on the real-time performance of the model.
questions: questions: For real-world application, we may have to collect up-to-date news related to time series of our interests, do the authors have any suggestion of the amount of news?
limitations: limitations: Limitations are well discussed in the manuscript.

Comments: title: Thanks to the ACs and the Reviewers.
comment: We sincerely thank the Area Chairs and the Reviewers for their time and effort during the discussion and review process. We are pleased to see that this paper has received consistently positive feedback from the reviewers.

In this work, we integrate news events into time series forecasting by fine-tuning a large language model. Additionally, we leverage LLM-based agents to iteratively filter out irrelevant news and apply human-like reasoning and reflection to evaluate predictions. We hope that this paper offers valuable insights into time series forecasting tasks and aids in the application of language models to real-world scenarios.

Bests,

The Authors
While I appreciate the thorough responses provided in your rebuttal, I have no further questions at this time and will maintain my positive rating.
The response has addressed my concerns well and I decide to raise my score to 6.
Thank you for providing more detailed explanations on prediction reliability and conducting more experiments as ablation studies. I acknowledge that I have read the rebuttal and have no further questions.