From News to Forecast: Integrating Event Analysis in LLM-Based Time Series Forecasting with Reflection
Authors: Xinlei Wang
Maike Feng
Jing Qiu
Jinjin Gu
Junhua Zhao
Venue: NeurIPS 2024 poster (2024)
Keywords: Large Language Model
Time Series Forecasting
AI Agent
Primary area: machine_learning_for_other_sciences_and_fields

Abstract: This paper introduces a novel approach that leverages Large Language Models (LLMs) and Generative Agents to enhance time series forecasting by reasoning across both text and time series data. With language as a medium, our method adaptively integrates social events into forecasting models, aligning news content with time series fluctuations to provide richer insights. Specifically, we utilize LLM-based agents to iteratively filter out irrelevant news and employ human-like reasoning to evaluate predictions. This enables the model to analyze complex events, such as unexpected incidents and shifts in social behavior, and continuously refine the selection logic of news and the robustness of the agent's output. By integrating selected news events with time series data, we fine-tune a pre-trained LLM to predict sequences of digits in time series. The results demonstrate significant improvements in forecasting accuracy, suggesting a potential paradigm shift in time series forecasting through the effective utilization of unstructured news data.

From News to Forecast: Integrating Event Analysis in LLM-Based Time Series Forecasting with Reflection
Xinlei Wang', Maike Feng", Jing Qiu', Jinjin Gu Junhua 2
'School of Electrical and Computer Engineering, The University of Sydney
Sccooo of Science and Engineering, The Chinese University of Hong Kong, Shenzhen *Shenzhen Institute of Artificial Intelligence and Robotics for Society (xinlei. wang, jeremy qiu, jinjin.gul guydeyd edu. au, zuaouunnuae edu,
Abstract
This paper introduces novel approach that leverages Large Language Models (LLMs) and Generative Agents to enhance time series forecasting by reasoning across both text. and time series data. With language adaptively integrates social events into forecasting models, aligning news content with time series fluctuations provide richer insights. Specifically, LLM-based agents to iteratively filter out irrelevant news and employ human-like reasoning to evaluate predictions. This enables the model to analyze complex events, such as unexpected incidents and shifts in social behavior, and continuously refine the selection logic of news and the robustness the agent's output. By inte- grating selected news events with time series data, we fine-tune a pre-trained LLM to predict sequences of digits time series. The results demonstrate significant improvements in forecasting accuracy, suggesting a potential paradigm shift in time series forecasting through the effective utilization of unstructured news data.
medium, our method
utilize
1 Introduction
Time series forecasting 18 serves as an essential foundation for decision-making across a wide range of economic, infrastructural, and social domains 2 K 16 56 The purpose of analyzing time series data is to decode the evolving relationships within complex, dynamic real- world systems. Traditional forecasting methods, while effective at identifying patterns in historical data, perform well when time series distributions remain consistent over time. However, they have limitations in addressing sudden disruptions or anomalies caused by external random events, and do not systematically connect complex social events with fluctuations in time series data. Integrating insights from real-world events and their effects on social and economic behavior crucial for improving the reliability and accuracy of time series forecasting.
News articles provide crucial insights into unexpected incidents, policy changes, technological developments, and public sentiment shifts-factors that numerical data alone may not capture. Integrating news into forecasting enriches its inputs with context that closely mirrors the complexities of human behavior and societal changes. On the one hand, news offers a real-time snapshot of events, enabling the model adjust predictions based on updated information. On the other hand, qualitative data from news sources enables the model to account for non- lon-lincar and non-numeric influences. By combining both quantitative and qualitative insights, the model can improve forecast accuracy, especially in rapidly changing environments, making it more reflective of real-world dynamics.
In this work, propose unified approach that embeds news and supplementary information into time series data using textual prompts. By fine-tuning large language models (LLMs) S4 55 66
Junhua Zhao and Jinjin Gu are the corresponding authors. Code and data available at htttps://gittuub.comm/ http:///t.....ooooooooo/ooo/o///
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

Contextual Information Database
Website
Supplementan Information
Category: Finance, Politics, Weather, Social, Entertainment...
Calendar Dete; Holiday rnformatonn Geographic
News
Information; Weather; Economic tndicators...
Project
Forecasting Tasks
Aj Information Retrieval
Supplementary Information
Time
Requirements
Retrieval Keywords
Series
Region
Frequency
Time Horizon
Domalnn
Raw News
Reasoning Agent
*oor trainnn the default settings reasoning
Initial Reasoning Logic Setting
Reasoning Logic Update
News Filtering
Long: Term Effect News
Rationality
Proppt...ummmriia
Prompt: According to (iiised and flast logic), prediction logic...
Prompt:
selecting news logic for foorecn task) in (required J5ON format)...
(updated logic] and (nctrieval keesss news
Shart- Effect News
Filtered News
Prompt Integration for Data Preparation
friisooriaa time series). Predict (forecosting toskj the (forecosting harizon]: Historical data covers (time length) with (the frequency). [Suppiementory information, weather, calendor date, region); (publication time), (news summary) with (Natioonlityy. (Actual volues of time series the prediction perioa].
Input:
Output:
D) LLM Finetune
E) Evaluation Agent
# Large
Language
Time Series
Prediction
Training
Prompt: Accordiingtoo (the news ond errors), determineit been missed...
Result Analysis Reflection
Missed
M
Model
Actual
Values
Prediction Errors
Figure 1: Integrating textual information in time series forecasting. (A) We retrieve relevant original news supplementary information from our comprehensive database based on information such the geographic location and time frame of the prediction task. (B) LLM-based agents analyze and select relevant news for different forecasting horizons. (C & D) The selected news and contextual information are combined with time series data for fine-tuning the LLM forecasting model. (E) Discrepancies between predictions and ground truth trigger review of historical news and data reprocess missed information and refine reasoning logic.
transform time series forccasting into the prediction of the next token in text, taking into account relevant contextual information. The inductive reasoning capabilities of pre-trained LLMs, along with their ability model multi-modal distributions, enable few-shot predictions in time series [17]. The potential of language models in time series forecasting has also been proven [23] 71). After further dataset, which includes time series, news, and supplementary information, language models can generate forecasts that consider the textual context provided in the input prompts.
training
Effective news filtering is a key issue for enhancing time series forecasting as input diversity increases. This task requires more than simple keyword extraction; it demands a deep understanding of how news elements interact with forecast variables, extending beyond linear reasoning incorporate intelligent analytical methods. We employ LLM agents [43] 57 with advanced human-like reasoning capabilities for dynamic and effective news selection. These agents use few-shot learning to adapt their strategies based on logical scenarios that mimic human reasoning about factors affecting time series fluctuations. This enables them to identify relevant news, which is then paired with corresponding time series data to create a context-award dataset that improves prediction accuracy. Additionally. LLM agents also play crucial role in model evaluation, continuously refining their selection logic by comparing forecast errors with all relevant events. This iterative self-evaluation helps identify and integrate critical news items previously overlooked. By automating the analysis of unstructured text and applying chain-of-thoug. prompting, the agents effectively uncover patterns linking news events to forecasting discrepancies, revealing the nuanced effects of external factors on predictions.
Our contribution can summarized as follows:
The paper introduces a novel time series forecasting framework that integrates unstructured news data with numerical time series inputs, providing deeper contextual understanding and improving the model's 's responsiveness to social events and real-world dynamics.
The research highlights the use of L.M agents for dynamic news selection and analysis. We leverage the reasoning ability of LLM to automate the effective understanding and filtering of

---

news content. The agents iteratively refine their news selection process based on forecasting results, improving model accuracy and reliability.
We propose a data construction method that integrates time series data with news information, and build dataset spanning multiple domains support our research. The dataset includes task- specific time series data and verified public news reports, which facilitates further exploration in time series research.
Our experiment results demonstrate the effective integration of news data, achieving superior prediction accuracy across diverse domains such as energy, exchange rate, traffic, and bitcoin domains. Our findings demonstrate that incorporating news is highly adept at navigating com- plexities, especially in energy demand patterns.
2 Related Work
Time series forecasting. The traditional method of time series forecasting relies on analyzing historical data and utilizing statistical models to predict future trends, with an assumption that past patterns will persist in the future [8] 12 20 27, 42). However, these methods were limited to small- scale datasets. The vevent of deep learning 30 has introduced range of time series forecasting networks [33 34 36, 40 [53] [59 60 67 69 that excel in managing larger, more complex datasets by capturing non- coneriieesssies and dependencies directly from historical data. Recent advancements include pre-training on diverse, lcrge-scale datasets, allowing models to be fine-tuned on specific tasks with fewer data and resources 66] [4 58 63]. While these methods continue to evolve and improve performance benchmarks, they often neglect the influence of external and contextual factors.
Attempts incorporate textual information (e.g.. Twitter feeds, news articles, public reports) into time series forecasting have been made across various domains, such as finance [8 49) energy B 4T) entertainment [26), pandemics [65), and tourism [47. Traditional methods often simplify text analysis to counting keyword frequencies [41] or using dummy variables, which do not capture nuanced meanings. Advanced efforts include extracting richer textual features like word frequencies and sentiments using traditional NLP [9] and machine learning methods, as demonstrated by Bai et al. 3 However, these approaches require labor-intensive feature engineering. struggle with long-text dependencies, and lack a deep contextual understanding. In contrast, LLMs excel in processing complex textual data and understanding contextual relationships, which can improve prediction accuracy and efficiency through automated feature extraction and enhanced scalability across multiple tasks. Despite their potential, no study has yet fully exploited L.M to enhance forecasting with their capabilities in understanding unstructured textual data.
Language models for time series forecasting. LLMs such as the GPT series [
and LLaMa (54] have excelled in variety of natural language processing tasks. With their vast parameter sets, LLMS acquire extensive general knowledge and reasoning capabilities during pre-training,
which is crucial for building intelligent systems equipped with common sense. LLMs architectures
5T) For instance, TEMPO 66] adapts GPT architectures for dynamic temporal representation learning, while TIME- LLM 23 utilizes LLMS for time series forecasting by reprogramming input data and applying Prompt-as- Prefix techniques. Similarly, FPT [71] demonstrates that even frozen I.I.s can perform effectively in time series tasks, leveraging the universality of self-attention mechanisms. Lag-ILLaMa (66) uses decoder-only transformer for univariate probabilistic forecasting, and Gruver et al. [I7] show that by framing time series forecasting as next-token prediction, LLMs can surpass traditional models through effective tokenization and adaptation. However, existing studies mainly utilize the mapping capabilities of LLMs for numerical regression, without incorporating external textual inputs or leveraging the reasoning abilities of LLMs in understanding language.
are increasingly applied time series processing and forecasting 23 25
35
37
Reasoning with language models. LLMs can automate tasks with human- like reasoning through "Chain of Thought" (CoT) prompting 57 enhancing reasoning by step-by-step emulation of human thinking 28 32. CoT prompting is useful for transforming complex questions into answers by
introducing intermediate steps. The "Tree of Thoughts" (ToT) approach (61] refines this by mimicking tril--nnd-rror methods, enhancing auto-regressive LLMs with a prompter agent, checker module, memory module, and ToT controller for multi-round dialogues. LLMdbbsed agents can solve tasks by reflecting feedback signals in text and retaining them in a memory buffer for better decisions [50]. Cai et al. [5] proposed the LLMs As Tool Makers (LATM) framework, where LLMs create reusable tools for problem-see solving, interleaving reasoning and actions to aid task completion and interaction 662. These agents can debate their responses and reasoning to arrive at final actions [
3

---

Short-Term Effect News
Short-Ter Term Effect News
REWS
Example:
Nows: delaysfer Meloummc with majofficwwyy dlooed ofter lousersss, MC22019
Exampp::
News: deluge approaches southmm states, Various southem AM.
AM
srates
Rationality. traffic eventicn immed influenceeleetttnn
Rationality Westher wor nings nd weether
consumption, incressediuse tranperrrtttttntoonnnn systems, atorrd business aperinghour,, traff monngemmnntessoooonnn
adjustments dletrrity
incressediuse heater..
proparing for potontia utages, and emeogenooyssspenssss
Exent
NEWS
Time
Nown
2019-06-10
2019-06-11
2019-06-12
Newx Conpaay
60ori s00
60M/9 kooe
00M1 3040
CALDEN
S1ARB 12:0
stana 1808
Kauna 060
GR279 1200
Chote 3006
Long-Tere Term Effect News
Short-Term Effect News
Example:
Doample:
News: Opttus hatastieededd sorly parta of Ss network with more
News: Tralnservices were suspen ded Melbourne after train hita
NEWS
areas to Geneeay Austral, 2015-01- 3411:300
oar ata lovel crossingnenn car Farisale VIC, 2013-05 12 3:41 00 PM.
nationality: 5G network exparsion equ ros entersiis infrattuutuure increasinght pareasinghnng-eee glangetrmmeeee peeriitty demands dua to vndirnnal and greater relance on SC technologies.
Rationality: The suspension ofriinnnnttiiiieiieiie may directly
electricity load acalemative tos inpoppptationn emergercy services require increseed power conumption fulliwning the neident.
Figure 2: Relationship between news and time series. This figure illustrates the news filtered by the reasoning agents, using the example of Australia's 's state-level electricity demand. It features load data Victoria state and selected news from June 10 to 12, 2019. The black arrow indicates time-specific events, the blue curve shows load fluctuations. The x-axis represents time, and the y-axis displays load values in kilowatts. The blue box displays the short-term impact news and long-term impact news selected by the reasoning agent (e.g.. traffic incidents or new construction projects).
3 Method
In this work, we aim integrate news insights into time series forecasting. The development of such a system faces several challenges. Firstly, the forecasting method must handle unstructured, non-numenn non-numerical news inputs flexibly and adjust predictions based on the context of the news events. Secondly, constructing this model involves filtering news and establishing connections between the news and the time series data. This requires sifting through vast amounts of internet data to find relevant information, demanding deep societal understanding and sophisticated reasoning skills. Therefore, an intelligent agent is designed to manage this complexity. Moreover, potential inaccuracies in news selection or inferential errors may still affect the forecast accuracy, requiring further refinement of news selection and reasoning based on the predictions. Our approach includes three main modules: a language model-based forecasting module (Sec (..c reasoning agent for news filtering and inference, and an evaluation agent to assess and refine the forecasting model (Sec 3.2). The core workflow of our method and the interrelations between these modules are shown The subsequent sections will discuss these three modules in detail.
in Figure I
3.1
Rethinking Time Series Forecasting Problem and Elements.
Time series forecasting can be considered as a conditional generation problem of sequences [17] This aligns with the general paradigm of natural language processing represented by LLMs. Taking the LLaMa language model as an example, assuming a number series (123,456), LLaMa's tokenizer will regard this number as a sequence of digit tokens, i.e., 1"1", "2", """",""""""" " ",,,"", " "5"", " "6"). Given the input series "123", the probability of predicting "456" can be represented as a probabilistic forecasting process in an autoregressive manner: P( P("456"|"123" = P( "4"" "123") P("5"| "4", "123") P("6"|"44" "45", "123"). ). Generally, denoting the time series tokens at time as F2, the LLM predict the next token in the series #4+1 using the conditional probability distribution P( (mn=)). During pre-training, LLMs optimize its internal parameters to maximize this conditional probability over a wide range of natural language corpora. Though counterintuitive, Gruver et al. [17] have shown that pre-trained language models exhibit a significant few-shot capability for time series forecasting. This shows the potential of language models in understanding input digital tokens, and also inspires to use the language model as a reasonable platform to study how to introduce the information contained in textual prompt into time series prediction.
News context offers critical insights into complex social events that traditional numerical data often overlook, and it also reflects sudden shifts in time series due to random events. In fact, the time series we collected can already be seen as being influenced by the aforementioned events. Assume an event E and a time series Toa, its impact on the future sequence can be expressed also as a conditional probability P(1+41 | E). However, when information about event not provided, we can only predict through historical time series. Although time series data itself can show patterns and trends,

---

lacks the ability to indicate the causality behind events. The event information & offers the context needed to understand why certain spikes or drops occur. We show in Figure 2 some news that are closely related to time series forecasting.
In language models, news events can also be represented as text tokens. Consider a set of news text tokens {eo, ei...., , ). which represent event E. LLMs treat this news information as condition input and perform conditional probability predictions P 60ra). Including 60.a provides crucial contextual information that influences the prediction of future values. This process aligns with the standard approach used by language models to interpret text, allowing for the incorporation of information about multiple events through various news contexts simultaneously to enhance prediction accuracy. In practice, we only need to integrate news text with historical time series data in prompt engineering manner.
Other supplementary information also provide contextual information for the forecasting model. For example, weather and climatic factors may affect energy demand and industrial output; financial indicators and economic metrics influence consumer behavior and business operations. Including this diverse range of information allows models to adjust for environmental, economic, and seasonal variations, improving prediction accuracy. The supplementary information can also be understood as conditions and integrated into the above conditional probability forecasting framework. Our approach incorporates this information into language for flexible integration with language models. For example, we use the text "Weather on historical dates: the lowest temperature is 292.01; the highest temperature is 298. 07: the humidity 94." to express weather conditions. This enriches the iillustrates the method of
input to cover various factors affecting the time series. Part (c) of Figure [ prompt integration for time series forecasting and the corresponding responses.
Fine-tuning LLMs for time series forecasting. Integrating the above information, we can construct inputs for LLMs perform time series forecasting. Although pre-trained LLMs are already capable of generating time series predictions to some extent, relying on these pre-trained models for few-shot predictions in such context-rich environment poses several challenges. Firstly, controlling the output of time series is difficult; predicting long sequences of numerical digit tokens is uncommon for LLMs. Secondly, the connections between the news and supplementary information and the time series typically need to be derived from historical data, which goes beyond the usual scope of using LLMS for few-shot time series predictions. To enable language models to more effectively forecast time series while considering the conditions imposed by news and supplementary information, we propose fine-tuning the language models to predict conditional probabilities. We employ supervised instruction tuning method to train LLMs on historical time series data paired with corresponding news and supplementary information, formatted into text input-output pairs (shown in Appendix (A.3). The same loss function used during pre-training is applied here. To fine-tune the LLM, we use the Low-Rank Adaptation A) method [19], which updates only a small subset of parameters, reducing computational demands while retaining most of the pre- trainedid knowledge. This strategy allows the model to efficiently adapt to new forecasting tasks without losing its foundational strengths.
3.2 Analytical Agent for Aggregation and Reasoning of Contextual News Information.
Next, we construct a dataset to train the above model. While obtaining time series data is relatively straightforward, matching it with appropriate news and supplementary information not trivial. The internet is flooded with news, most of which are irrelevant to the time series we aim to forecast. Introducing irrelevant news can disrupt forecasting. Therefore, it is crucial to analyze the relevance and causality between the time series forecasting task and the select news accordingly. However, gaining such an understanding is complex, requiring knowledge of human societal mechanisms and logical reasoning skills. In our work, we utilize LLMSs for filtering and reasoning about news content. We also recognize that even the most advanced language models cannot complete all reasoning and judgment in a single generation process. We explain how to use a combination of multiple LLM generations to create an intelligent agent that fulfills the complex requirements of news filtering.
Time series and news pre-pairing. For the initial stage of data preparation, news is retrieved to align with time series data based on matching time frequencies, horizons, and geographical areas. This synchronization ensures that insights from textual information are timely and regionally relevant. For example, to understand state-level electricity demand in Australia from 2019 to 2021, we gather local news from various Australian states and international news occurring in the same period that might directly or indirectly affect demand. In this way, potentially relevant candidate information can be roughly selected first, and such filtering can be casily completed through crawler means.
5

---

Reasoning logic
Record in
Auto- Setting
Validation
selection thet will change the itorecastng took in specttic /output format)
on [foreanding for Grven
and relevant
predicted future fforccactng
weiss
values,
A: (The initall news solection logic).
newe references, assess prediction sccuracy and analyce whether any important nows has been overlooked. & (Background information for s...
Al: AWi fallw these stecs, inckuting camparing predicted vs. wctuol values, analyring vorations, revitwing News artdes, correistig ng news with traffic anomalles, and reocssing News relevence.
Part Each Record ny in the Data Sut
Q: I1 give you al hews, based on the ahove postive & negalive iscuss analyis of (news sp lection lagis),
Q2. Given (welected nesel (all newe, lifettal values] and (prediction wron), deteemine been missed. Output the missed (th requiredd format).
1) please choose the news that may have a long-teem affect on (forccasting task toplc):
(Vissed for flrrcasting
please choese the that may have short-iam effect Hoorecattng task topic)
According the outtooked news (missed news] for forcccating perfoe for please drwcily condade veverall new predistion lagic of (Jowcasiing time sar damain AB: (Upoated logit for rab.
Requinements: includle
time information
these news prediction
(predictton date). The happened before and on the prediction include:(ell newel.
Only give the Json output including all relevant news and make it (the weidd hon furmat)
Bioluotion Agent
2
Q4. improve and polish this panagraph and reduce repeated content, and summarize the news selection lngie that affrets the (fonecasting ing time write domein ): (updated lngin for updoted logic for updated oggic for rz
Reasoning Agant
A4::The final news selection logi)
Figure 3: Example of prompt designs for each iteration during fine-tuning, Step I involves the reasoning agent selecting news using default logic. Step 2 evaluates predictions based on validation sets to refine the logic. In step 3, the updated logic directs data pairing for the next iteration. Full prompts are shown in Appendix AA66]
Reasoning agent for news selection. We employ an LLM-based reasoning agent capable of sophisticated tasks such as conversation, reasoning, and semi-autonomous action. This agent is programmed using detailed prompts that define roles, instructions, permissions, and context, enabling interpret human commands and perform complex tasks. This approach condenses extensive news datasets into a refined selection of pertinent articles. It leverages its reasoning capabilities to effectively screen, categorize, and interpret news texts. We employ few-shot prompting 4 and the CoT [57] method to develop an agent that understands the context of news for forecasting. This technique enhances the model's ability to handle multi-step, commonsense reasoning tasks essential for accurate forecasting. The agent uses multi-step prompts to break down complex problems into simpler, manageable parts. Our three-phase prompting method, illustrated in Figure Part I, includes:
1. We develop an understanding of time series influencers, sorting them by impact (positiveenegaative) and duration (short/ong--temi) considering economic, policy, seasonal, and technological factors;
2. We direct the agent to filter and categorize news based on either automatically generated logic given reasoning logic, focusing relevance to time series and classifying the impact (e.g.. long-term and short-term) along with the rationale;
3. We specify the output format for the agent to organize the selected news into ISON, detailing aspects like summary, affected region, reporting time, and rationale. More details about our prompting method can be found in Appendix [A.6]
The LLM agent can automatically develop an understanding of time series influencers, with the option to provide predefined reasoning logic in our models. In the automated process, the agent forms its logic through prompts designed to guide it in determining how different types of news impact a specific domain. For instance, we use open- open-ended questions to allow the agent to independently summarize and create its own filtering logic. User knowledge can also be incorporated into these prompts as additional reasoning, enabling the agent to generate more comprehensive logic. The agent then filters news based on the generated logic, whether fully automated or with user-provided input.
Evaluation agent for reasoning updating. We also design an evaluation agent to assess and improve the effectiveness of the aforementioned news filtering. Relying solely on the reasoning agent for news selection is not optimal, as the interaction between news and time series is complex. The reasoning agent can only analyze the potential impact of different news from the perspective of news content, without knowing whether the trained time series forecasting model based on them can make accurate forecasts. The evaluation agent is deployed after the time series prediction model has been trained. The evaluation agent extends beyond simple numerical assessments of prediction accuracy by incorporating human-like logical reasoning to refine the news selection logic chain. We focus our evaluation on identifying inaccuracies potentially caused by missing news, such as unusual events or illogical reports. It observes the model's predictive outcomes to determine if any crucial news has been overlooked and adjusts the news filtering strategy for the training data based on these results.

---

Reasoning Agent
Logic
Initial Reasoning
Updated Reasoning Logic for Iteration
Updated Reasoning for Iteration N
Updated Reasoning Logic For Final Finetuning
for Iteration
Iteration 1
Iteration
Iteration N
Final
Reasoning Agent
Ressoning Agent
Reasoning Agent
Reasoning Agent
Selected News
Selected News
Selected News
Selected News
LLM Finetuning
LLM Finetuning
LLM Finetuning
LUM Finetuning
Validation Results
Validation Results
Validation Results
Test Results
Evaluation Agent
Evaluation Agent
Evaluation Agent
Supplementary Information
Missed News
Missed News
Missee News
Figure 4: Overall pipeline iteratively combines news reasoning agents, fine-tuning, and evaluation agents.
For the evaluation agent, we structured the prompt design into three phases, as illustrated in Figure Part II. In the first phase, we input the forecasting task type, the time horizon, and background information, which the agent uses to generate the steps for evaluating the prediction outcomes. In the second phase, we provide the ground truth, the discrepancies between predicted and actual series, as well as selected and historical news. The agent analyzes these inputs to identify overlooked news based on the distribution of prediction errors over time. In the third phase, the agent generates updated logic based on its analysis, guiding future news selection. After processing all validation set predictions, the reasoning agent consolidates the updated logic into cohesive final strategy.
3.3 Overall Pipeline
We integrate the news reasoning and evaluation agents with the fine-tuning of the LLM forecasting model to enhance the quality of training data, illustrated in Figure 4 In the first iteration, use the LLM agent to establish news selection logic based on the domain and timing of the time series task. This logic directs the reasoning agent to filter relevant news, align it with time series data, and input it into the model for initial fine-tuning. After validating the model's predictions with a validation set, which is randomly extracted from the training data for each iteration, the evaluation agent checks for any missing news that may have influenced the prediction. This feedback helps the reasoning agent refine the filtering logic in subsequent iterations. The cycle continues until the final iteration, where the reasoning agent consolidates all updates to create the definitive news filter for training the final model. We the GPT-4 Turbo model as the LLM for the agents described above.
Experiments
4.1 Data preparation
Time series data. We selected time series data from domains influenced by human activities and social events test our method's s ability capture complex human-driven dynamics during forecast- ing. These domains include Traffic (39 (traffic volume), Exchange (29] (exchange rate), Bitcoin [[5]
(Bitcoin price), and Electricity [13] (Australian electricity demand). To avoid bias from pre-trained language models, we updated the Exchange and Electricity datasets up to 2022. We use half-hourly electricity demand data from the Australian Energy Market Operator (AEMO) (aemo. com. au) and daily exchange rate data from the Exchange Rates API (exchangoratesapp. io). These datasets vary frequency, including daily, hourly, and half-hourly updates, allowing us to evaluate the algorithms' effectiveness across different temporal resolutions. More details are in Appendix [.I]
News collection. Since there are no public datasets that pair time series data with news events, we have collect news specifically for the above time series to facilitate our research. Some of the news content collected from the GDELT dataset [31. database tracking news from nearly every country in over 100 languages. GDELT provides real-time insights into societal, political, and economic events, enabling detailed analysis of global trends and their effects. We incorporate GDELT's event information into our forecasting models to enhance predictive accuracy. For domains needing the latest information, we collect real-time news from sources like News Corp Australia (news... com. and Yahoo Finance (yahoo. com), focusing region-specifie and task-specific activities.
7

---

Supplementary information. We enhance our forecasting models with open-source tools to grasp additional data for better accuracy and context. Weather information from Open MeatherMaa [To] provides daily temperatures, atmospheric pressure, wind speed, and humidity, crucial for load forecasting. Calendar dates, obtained using the Python packages datetime and holidays, account for seasonal and cyclical effects. Economic indicators are integrated using the pandas_ landas_dttareee datareader library, accessing data like GDP, inflation rates, and employment statistics from sources such as the Federal Reserve, World Bank, and international financial markets.
4.2 Results
Effectiveness of news intergration. In our approach, we incorporate news and supplementary information into time series forecasting by fine-tuning language models. Firstly, we assess whether this additional information can enhance time series forecasting. We conducted experiments to verify the necessity and effectiveness of integrating news data into our forecasting model. We compared four different scenarios as detailed in Appendix A.. and Appendix A.3
1. Pure numerical tokens: Uses numerical tokens, encompassing all variables without news. Except for region names or date information, it excludes other textual tokens as baseline for comparison.
2. Textual descriptive sentence tokens: Evaluates whether using sentence-form descriptions instead of only raw digital numbers
enhance accuracy, with no news integration included.
3. Unfiltered news with textual descriptive sentence tokens: Assesses how integrating descriptive sentences of time series with unfiltered news data affects the model's 's performance.
4. Filtered news with textual descriptive sentence tokens: Shows the effects of integrating descriptive sentences with news that has been specifically filtered for relevance by the proposed agents.
Table 1: Performance comparison of different prompt designs. Red font indicates the best.
Electricity
Exchange
RMSE
MSE
MAF
MAPF
RMSE
MSF
MAB
MAPF
Only Numeric Prompt
337.10
113.64
204.89
5.27%
7.80
6.10
5.74
0.77%
Textual Prompt w without News
336.41
113,17
206.08
5.29%
7.41
5,49
5.44
0.73%
Textual Prompt with Non-Filtee Filtered
166.35
250.75
6.84%
0.85%
Textual Prompt with Filtered News
280.39
78.62
180.96
5.15%
6.46
4.17
4,83
0.63%
Traffic
Bitcoin
RMSE.117
MSE
MAE
RMSE
MSE 3u
MAPE
Only Numeric Prompt
4.55
2.07
1.66
4.46
19.94
3.07
5.72%
Textual Prompt without News Textual Prompt with Non-Filieree News
1,97
1.54
3.87
14,97
2.76
5.08%
4.89
2.39
1,89
4.02
16.13
2.88
5,35%
Textual Prompt with Filtered News
4,22
1.78
1.43
3.67
13.41
2.68
4.95%
The performance of different prompt designs is presented Table T can be seen that the fine-tuned LLM can be used to forecast time series even when using only digital tokens as prompts. The introduction of proper news and other supplementary information leads to significant performance improvements across all four domains. Nevertheless, such improvements are not easily achieved. If the introduced news information is not carefully selected, it can severely impair the results. There are two main reasons for this: first, the influx of a large number of news items introduces too many tokens, which can decrease the performance of the LLM as the number tokens increases. Second, irrelevant news can introduce noise and incorrect causal information, leading misleading predictions.
Effectiveness of the evaluation agent. To make our news filtering and reasoning processes more effective and comprehensive, we introduce an evaluation agent to reflect on and improve the effects of news selection based on prediction outcomes. As shown in Table our evaluation agent refines news filtering through an iterative process, which is reflected in the progressively improved time series prediction results. The result corresponding to each case is the prediction outcome of the model after pairing the news selected according to the logic obtained from the corresponding iteration. Our findings suggest that, in most cases, two iterations are sufficient to achieve significant improvements, with multiple iterations consistently yielding better results than a single iteration due to the reflection mechanisms. We also found that this iterative filtering process reveals interesting insights into human society from the language model agent. These examples are presented in the Appendix A.77
Compare to other forecasting methods. We also compare our method with existing time series forecasting techniques, detailed in Appendix A. We show a quantitative comparison against these baseline methods in Table 3 While the baseline methods use inverse normalization to revert predictions to their original scale, our model operates without normalization. This approach retains
8

---

Table 2: Comparison of Iterative Analysis. The baseline case is the initial selection. The arrow means the
comparison of each case with baseline cases, red downward arrow indicates an improvement, blue upward arrow indicates performance degradation.
Electricity
Exchange
RMSE
MSE,
MAE
MAPE
RMSEx10a
MSE
MAEx10a
MАPE
1. Initial selection
313.89
98.53
190.79
5.36%
6.61
4.37
4.83
0.65%
2. The second selection
4287.35
482.57
4180.49
44.93%
46.46
44.17
44.83
40.65%
3. The third selection
4.303.03
491.83
+192.30
+5.38%
+7.69
+5.92
15.63
+0.75%
The fourth selection
4280.39
478.62
4180.96
45.15%
46.60
44.36
44.82
40.65%
Traffic
Bitcoin
RMSEx16²
MSE;
MAEx10
RMSE
MSEx10-:
MAE,
MАРE
1. Initial selection
4.36
1.90
1.45
4.12
16.98
2.97
5,50%
second selection
43.67
44.95%
3. The third selection
+4,22
+1.78
+1.43
43.75
+14.08
42,83
45.18%
Table 3: Comparison baselines for time series forecasting on different metrics. A lower value indicates better performance. Red: the best. Blue: the second best.
Domains
Metries
Onns
Aatoo
In:
[Min.
[d]]
ITrans.
HILM [05
[imes
Pyrm
16
Patch1ST
GPT4TS
Electeickty
MSE
KMSSU
280.39
78.62
251.79
501
407.52
165.07
153.90
291.3
366.64
134.42
133.53
135.96
160
377.02
142.60
MAPE
5.15%
10.63%
8.949
7.29%
6.36%
7.36%
6.81%
6.87%
6.50%
6.75%
6.61%
MAF.
4.83
0.27
1.25
6.55
5.12
6.44
5.24
14.6
6.73
8.96
15.05
Exchange
MSSE
0.42
1.87
1.36
4.76
0.91
0.671
0.675
0.27
0.673
3.55
0.875
0.77
1.13
1:24
3.00
4.01
MAE
2.49
1.70
1.65
1.51
1.74
1.64
Traffi::
MSE
1.78
2.19
5.27
1.67
1.54
1.21
1.49
1.98
1.54
1.43
1.45
RMSE,
4.22
4.68
7.26
4.09
3.93
4.14
3.86
3.13
3.92
3.79
3.81
MAE
2.68
4.25
12.27
5.74
3.20
3.26
3.17
0.22
2.85
3.96
2.84
13-4
13.41
163.47
16:31
17.65
5.26
12:75
11.12
3.70
7.61%
5.64%
the physical meaning of data, such as electricity demand or economic indicators, ensuring that our outputs remain interpretable. Normalization could obscure the impact of news events due to the nonlinear and dependettn relationship between these events and the original data values.
Our approach significantly outperforms traditional methods that rely solely on historical time series data in domains like electricity demand, exchange rates, and the bitcoin market, where events embodied in the news have substantial impacts. This demonstrates the potential of our method. However, the improvement from integrating news into the traffic sector is notably modest. The performance of our traffic forecasting model, which covers all roads California, is hampered by the coarse granularity of publicly available news data, lacking the local details necessary for precise predictions. Traffic data primarily reflects specific road traffic flows, whereas our news sources are mostly regional or global, failing to capture localized traffic conditions adequately. This limitation is evident in the model's Mean Squared Error (MSE), which is sensitive to outliers and tends exaggerate errors from traffic spikes that state-level news often does not report. Our model achieves good results with the Mean Absolute Error (MAE), indicating reliable average accuracy. Incorporating more localized road information could potentially improve these issues.
Figure5]takes, takes the electricity domain as an example and compares the ground truth with predictions by cases with or without news data, demonstrating the effect of incorporating news into forecasting models. The "With News" predictions are closer to the actual values than the "No News" predictions, particularly at critical timestamps where abrupt events significantly influence electricity demand.
5 Conclusion and Discussion
In conclusion, our study demonstrates the benefits of integrating news into time series forecasting us- ing LLM-based forecasting method and LLM-based agents. These agents enhance model intelligence by autonomously identifying and addressing missed news, refining their logic, and assessing the impact of events on predictions. Our findings advocate for incorporating extensive domain knowledge, encouraging a shift towards more nuanced and context-award forecasting. This approach enriches time series forecasting for adaptive, comprehensive forecasting aligned with real-world dynamics.
Limitations of our approach. While our approach demonstrates that LLMS like L.LMM (55) can enhance time series forecasting by integrating news, there are limitations its applicability.

---

Lool
News:
E.nley's eipected to deveuee
Toad (193
-.teornningg coriiletiaal elechissty loading
Load (1W9
peneee ontage egeetied offict eleswiciiy
(890)
exegyy siigh..
consuggaos..
12000
11003
10000
3080
10000
4550
9000
5000
9000
7000
4000
7000
Figure 5: Day-ahead Australia electricity demand forecasting with/without news. The horizontal axis is the time index (half hour). Actual load demands are in solid blue, predictions with news in solid red, and predictions without news in dashed green. (a) Sydney's 's lockdown news effects; (b) Residential electricity consumption behavior news effects; (c) Anticipated power outage news effects.
The effectiveness of news integration is primarily evident in domains where human and market activities significantly influence trends. Our framework is less suitable for domains requiring precise meteorological modeling or where human activities have minimal impact, such as in meteorological or physical data. Additionally, the model is constrained by the maximum token length of pre- trained LLMs, complicating the simultaneous processing of large amounts of time series multiple sequences, which can lead to data truncation and affect the accuracy of long-term predictions. Finally, our strategy enhances rather than completely replaces traditional time series tasks like classification or interpolation across all fields. Our aim is to demonstrate that by leveraging language models, it is possible to incorporate useful textual information to enhance time series prediction tasks.
Future work. Future enhancements will focus on extending the current forecasting model's scope in several key areas. Firstly, attribution analyses of news content used in the model will pinpoint which factors most significantly impact forecasting accuracy, facilitating an optimized news integration process. Advanced analytical toolkits also be provided to reasoning agents, enabling sophisticated data processing and real-time application of complex analytical techmiques. These developments will enhance the precision and relevance of the time series prediction model, contributing deeper contextual insights and expanding its applicability in the predictive analytics field.
Broader impact. Ethically, it is crucial that we conduct thorough reviews to ensure that our use of news content does not inadvertently perpetuate biases or negatively influence public opinion. This involves implementing rigorous checks for accuracy and balance to avoid the risks associated with misinformation, ensuring that our data sources are credible and that the content is factually correct. Furthermore, the potential misuse of news, especially the spread of "fake news", highlights the need for our models to incorporate sophisticated mechanisms to verify information reliability before integration. Beyond the discussed sectors, this approach has the capability to extend into forecasting GDP trends, analyzing carbon emissions, or predicting public health outcomes, each carrying significant implications for policymaking and public welfare. Thus, while our research offers substantial benefits in enhancing predictive analytics, it also obligates us to handle these capabilities responsibly, ensuring our contributions positively impact economic planning, environmental strategy, and informed decision-making across various domains.
Acknowledgment
This work was supported in part by the Shenzhen Key Lab of Crowd Intelligence Empowered Low-Carbon Energy Network (No. SBSSSS222200660066000 Foundation Grant of China (7233 1009); in part by the Australian Research Council (ARC) Research Hub under Grant 1H18010008 in part by the ARC Training Centre under Grant IC200100023; in part by the ARC Linkage Project under Grant LP200100056 and Grant ARC 10220103881.
in part by the National Science
References
[1] Josh Achiam. Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08 087744 2023.
[2] Taghreed Alghamdi, Khalid Elgazzar. Magdi Bayoumi, Taysseer Sharaf, and Sumit Shah. Forecasting traffic congestion using arima modeling. In 2019 15th international wireless communications & mobile computing conference (IWCMC), pages 1227-1232 IEEE, 2019.
10

---

[3] Yun Bai, Simon Camal, and Andrea Michiorri. News and load: A quantitative exploration of natural language processing applications for forecasting day-ahead electricity system demand. IEEE Transactions on Power Systems, 2024.
[4] Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. Advances in neural information processing systems, 33:1887-19001 2020.
[5] Tianle Cai, Xuezhi Wang. Tengyu Ma, Xinyun Chen, and Denny Zhou. Large language models as tool makers. arXiv preprint arXiv:2305. 17126, 2023.
[6] Defu Cao, Furong Jia, Sercan O Arik. Tomas Pfister, Yixiang Zheng. Wen Ye, and Yan Liu. Tempo: Prompt- based generative pre-trained transformer for time series forecasting. arXiv preprint arXiv:23 2023.
[7] Mark Cccchini, Haldun Aytug, Gary J Kochler, and Praveen Pathak. Making words work: Using financial text as predictor of financial events. Decision support systems, 50(1):164-17 2010.
[8] Bo-Juen Chen, Ming-Wey Wei Chang, et al. Load forecasting using support vector machines: A study on eunite competition 2001. JEEE transactions on power systems, 19(4):1221-1800 2004.
[9] KR Chowdhary and KR Chowdhary. Natural language processing. Fundamentals of artificial intelligence, pages 603-649, 2020.
[10] Christine Dewi and Rung-Ching Chen. Integrating real-time weather forecasts data using openweathermap and twitter. International Journal of Information Technology and Business, 12):48-52, 2019.
[11] Yilun Du, Shuang Li, Antonio Torralba, Joshua B Tenenbaum, and Igor Mordatch. Improving factuality and reasoning in language models through multiagent debate. arXiv preprint arXiv:2305. 14325, 2023.
[12] Grzegorz Dudek. Short-term load forecasting using random forests. In Intelligent Systems' 2014: Pro- ceedings of the 7th IEEE International Conference Intelligent Systems 20' September 24-26, 2014, Warsaw, Poland, Volume 2: Tools, Architectures, Systems, Applications, pages 821 828. Springer, 2015.
[13] Robert Fildes, Shaohui Ma, and Stephan Kolassa. Retail forecasting: Research and practice. International Journal of Forecasting, 38(4): 1243))1283-318, 2022.
[14] Brian S Freeman, Graham Taylor, Bahram Gharabaghi, and Jesse Thé. Forecasting air quality time series using deep learning. Journal of the Air & Waste Management Association, 68(8):666-886, 2018
[15] Rakshitha Godahewa, Christoph Bergmeir, Geoffrey Webb, Rob Hyndman, and Pablo Montero-Manso. Monash time series forecasting archive. arXiv preprint arXiv: 2tti::21050000 (664000666334 2021.
[16] George Gross and Francisco D Galiana. Short-term load forecasting. Proceedings of the IEEE, 75(1)):5558 1573, 1987.
[17] Nate Gruver, Mare Finzi, Shikai Qiu, and Andrew G Wilson. Large language models are zero-shot time series forecasters. Advances in Neurul Information Processing Systems, 36, 2024.
[18] James D Hamilton. Time series analysis. Princeton university press, 2020.
[19] Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang. Lu Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models. arXiv preprint arXiv:2/0m 09680, 2021.
[20] Shyh-Jier Huang and Kuang- Shih. Short-term load forecasting via arma model identification including non -aggaaasii process considerations. IEEE Transactions on power systems, 18(2):673 679, 2003.
[21] RobJ J Hyndman and George Athanasopoulos. Forecasting: principles and practice. OTexts, 2018
[22] Albert Q Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, et al. Mistral 7b. arXiv preprint arXiv:2310.0668
2023.
[23] Ming Jin, Shiyu Wang, Lintao Ma, Zhixuan Chu, James Y Zhang, Xiaoming Shi, Pin- Yu Chen, Yuxuan Liang, Yuan-Fang Li, Shirui Pan, et al. Time-Ilm: Time series forecasting by reprogramming large language models. arXiv preprint arXiv:2310.011
2023.
[24] Ming Jin, Qingsong Wen, Yuxuan Liang, Chaoli Zhang. Siqiao Xue, Xue Wang, James Zhang, Yi Wang. Haifeng Chen, Xiaoli Li, et al. Large models for time series and spatio-temporal data: A survey and outlook. arXiv preprint arXiv:2310. 10.19996, 2023.
[25] Ming Jin, Yifan Zhang, Wei Chen, Kexin Zhang, Yuxuan Liang, Bin Yang, Jindong Wang, Shirui Pan, and Qingsong Wen. Position paper: What can large language models tell us about time series analysis. arXiv preprint arXiv:2402.0 02713, 2024.
[26] Mahesh Joshi, Dipanjan Das, Kevin Gimpel, and Noah A Smith. Movie reviews and revenues: An experiment in text regression. In Human language technologies: The 2010 annual conference of the North American chapter of the Association for Computational Linguistics, pages 293-296, 2010.
11

---

[27] Prajakta S Kalekar et al. Time series forecasting using holt-winters exponential smoothing. Kanwal Rekhi school of information Technology, 4329008(13):1-13, 2004.
[28] Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa. Large language models are zero-shot reasoners. Advances in neural information processing systems, 35:22199-22213, 2022.
[29] Guokun Lai, Wei-Cheng Chang. Yiming Yang. and Hanxiao Liu. Modeling long-and short-term temporal patterns with deep neural networks. In The st international ACM SIGIR conference on research & development in information retrieval, pages 10-104,22 2018.
[30] Yann LeCun, Yoshua Bengio, and Geoffrey Hinton. Deep learning. nature, 521(7553):436 2015.
[31] Kalev Leetaru and Philip A Schrodt. Gdelt: Global data on events, location, and tone, 1979-2012. In ISA anmual convention, volume 2, pages 1-49, Citeseer, 2013.
[32] Aitor Lewkowyez, Anders Andreassen, David Dohan, Ethan Dyer, Henryk Michalewski, Vinay Ramasesh, Ambrose Slone, Cem Anil, Imanol Schlag, Theo Gutman-Solo, et al. Solving quantitative reasoning problems with language models. Advances in Neural Information Processing Systems, 35:3843-3857, 2022.
[33] Shiyang Li, Xiaoyong Jin, Yao Xuan, Xiyou Zhou, Wenhu Chen, Yu-Xiang Wang, and Xifeng Yan. Enhancing the locality and breaking the memory bottleneck of transformer on time series forecasting. Advances in neural information processing systems, 32, 2019.
[34] Chang Liu, Zhijian Jin, Jie Gu, and Caiming Qiu. Short-term load forecasting using long short- term memory network. In 2017 IEEE PES innovative smart grid technologies conference Europe (ISGT Suroope pages 1-6, IEFE, 2017.
[35] Chenxi Liu, Sun Yang. Qianxiong Xu, Zhishuai Li, Cheng Long, Ziyue Li, and Rui Zhao. Spatial-temporal large language model for traffic prediction. arXiv preprint arXiv:2401.
2024.
[36] Shizhan Liu, Hang Yu, Cong Liao, Jianguo Li, Weiyao Lin, Alex X Liu, and Schahram Dustdar. Pyraformer: Low-complexity pyramidal attention for long-range time series modeling and forecasting. In International conference on learning representations, 2021.
[37] Xu Liu, Junieng Hu, Yuan Li. Shizhe Diao, Yuxuan Liang, Bryan Hooi, and Roger Zimmermann. Unitime: A language-enpowered unified model for cross-domain time series forecasting. arXiv preprint arXiv:2310.095// 2023.
[38] Yong Liu, Tengge Hu, Haoran Zhang, Haixu Wu, Shiyu Wang. Lintao Ma, and Mingsheng Long. itrans- former: Inverted transformers are effective for time series forecasting. arXiv preprint arXiv:2310.06625, 2023.
[39] Vitaly Kuznetsov Will Cukierski Maggie, Oren Anava. Web traffic time series forecasting, 2017.
[40] Yuqi Nie, Nam H Nguyen, Phanwadee Sinthong, and Jayant Kalagnanam. A time series worth 64 words: Long-term forecasting with transformers. arXiv preprint arXiv:2211.14733 147730 2022.
[41] David Obst, Joseph De Vilmarest, and Yannig Goude. Adaptive methods for short-term electricity load forecasting during covid- 19 lockdown in france. IEEE transactions on power systems, 36(5):475444763,
2021.
[42] Alex D Papalexopoulas and Timothy C Hesterberg. A regression-based approach to short-term system load forecasting. IEEE Transactions on power systems, 5(4): 153551533- 1990.
[43] Joon Sung Park, Joseph O' O'Brinn, Carrie Jun Cai, Meredith Ringel Morris, Percy Liang, and Michael Bernstein, Generative agents: Interactive simulacra of human hehavior. In Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology, pages 1-22, 2023.
[44] Alce Radford, Karthik Narasimhan, Tim Salimans, Ilya Sutskever, al. Improving language understanding
by generative pre-training. 2018.
[45] Alec Radford, Jeffrey Wu, Rewon Child, David I Luan, Dario Amodei, Ilya Sutskever, et al. Language models are unsupervised multitask learners. OpenAl blog. 1(8):9, 2019.
[46] Kashif Rasul, Arjun Ashok, Andrew Robert Williams, Arian Khorasani, George Adamopoulos, Rishika Bhagwatkar, Marin Biloš, Hena Ghonia, Nadhir Vincent Hassen, Anderson Schneider, et al. Lag-Ilama: Towards foundation models for time series forecasting. arXiv preprint arXiv:2310.08277,, 2023.
[47] Filipe Rodrigues, Ioulia Markou, and Francisco C Pereira. Combining time- mene-series and textual data for taxi demand prediction in event areas: A deep learning approach. Information Fusion, 49:120-129, 12290,0,..0000 2019.
[48] Robert P Schumaker and Hsinchun Chen. Textual analysis of stock market prediction using breaking financial news: The azfin text system. ACM Transactions on Information Systems (TOIS), 27(2): 2009.
[49] Robert P Schumaker and Hsinchun Chen. A discrete stock price prediction engine based on financial news. Computer, 43(1):51-56, 2010.
12

---

[50] Noah Shian, Beck Labash, and Ashwin Gopinath. Reflexion: an autonomous agent with dynamic memory and self-reflection. arXiv preprint arXiv:2303. 2023.
[51] Chenxi Sun. Yaliang Li, Hongyan Li, and Shenda Hong. Test: Text prototype aligned embedding to activate I1m's ability for time series. arXiv preprint arXiv:2308. 08224,, 2023.
[52] Gemma Team, Thomas Mesnard, Cassidy Hardin, Robert Dadashi, Surya Bhupatiraju. Shreya Pathak, Laurent Sifre, Morgane Rívière, Mihir Sanjay Kale, Juliette Love, al. Gemma: Open models based on gemini research and technology. arXiv preprint arXiv:2403. 08295, 2024.
[53]
José F Torres, Dalil Hadjout, Abderrazak Sebaa, Francisco Martinez- Álvarez, and Alicia Troncoso. Deep learning for time series forecasting: survey. Big Data, 9(1::3 21 2021.
[54] Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothéc Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, ct al. Llama: Open and efficient foundation language models. arXiv preprint arXiv: 11022.19970 13971, 2023.
[55] Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaci, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. Ilama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307,092885 2023.
[56] Ruey S Tsay. Analysis of financial time series. John wiley & sons, 2005.
[57] Jason Wei, Xuezhi Wang. Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, al. Chain-of-thouglle prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824-24888 24437,, 2022.
[58] Haixu Wu, Tengge Hu, Yong Liu, Hang Zhou, Jianmin Wang, and Mingsheng Long. Timesnet: Temporal 2d éd-variation modeling for general time series analysis. In The eleventh international conference on learning representations, 2022.
[59] Haixu Wu, Jichui Xu, Jianmin Wang, and Mingsheng Long. Autoformer: Decomposition transformers with auto-correlation for long-term series forecasting. Advances in neural information processing systems.
34:22419-22430, 2021.
[60] Min Xia, Haidong Shao, Xiandong Ma. and Clarence W de Silva. A stacked gru- ren---nneeedd approach for predicting renewable energy and electricity load for smart grid operation. IEEE Transactions on Industrial 2021.
Informatics, 17(10):7000-7
[61] Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Tom Griffiths, Yuan Cao, and Karthik Narasimhan. Tree of thoughts: Deliberate problem solving with large language models. Advances in Neural Information Processing Systems, 36, 2024.
[62] Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. React: Synergizing reasoning and acting in language models. arXiv preprint arXiv:22/0.03669, 2022.
[63] Chin-Chia Michael Yeh, Xin Dai, Huiyuan Chen, Yan Zheng, Yujie Fan, Audrey Der, Vivian Lai, Zhongfang Zhuang. Junpeng Wang, Liang Wang, et al. Toward foundation model for time series data, In Proceedings of the 32nd ACM International Conference on Information and Knowledge Management, pages 4400-4404, 2023.
[64] Ailing Zeng, Muxi Chen, Lei Zhang, and Qiang Xu. Are transformers effective for time series forecasting? In Proceedings of the AAAI conference on artificial intelligence, volume 37, pages 11121-11128, 2023.
[65] Xiongwei Zhang, Hager Saleh, Eman MG Younis. Radhya Sahal, and Abdelmgeid A Ali. Predicting coronavirus pandemic in real-time using machine learning and big data streaming system. Complexity, 2020:1-10, 2020.
[66] Wayne Xin Zhao, Kun Zhou, Junyi Li, Tianyi Tang, Xiaolci Wang, Yupeng Hou, Yingqian Min, Be- ichen Zhang, Junjie Zhang, Zican Dong, et al. A survey of large language models. arXiv preprint wxXi:23303118225 2023.
[67] Haoyi Zhou, Shanghang Zhang, Jieqi Peng, Shuai Zhang, Jianxin Li. Hui Xiong, and Wancai Zhang. Informer: Beyond efficient transformer for long sequence time-series forecasting. In Proceedings of the AAAI conference artificial intelligence, volume 35, pages 11106-11115, 2021.
[68] Tian Zhou, Ziqing Ma. Qingsong Wen, Liang Sun, Tao Yao, Wotao Yin, Rong Jin, al. Film: Frequeney improved legendre memory model for long-term time series forecasting. Advances in Neural Information Processing Systems, 15:76677- 2022.
[69] Tian Zhou, Ziqing Ma, Qingsong Wen, Xue Wang, Liang Sun, and Rong Jin. Fedformer: Frequency enhanced decomposed transformer for long-term series forecasting, In International conference machine learning, pages 27268-27286. PMLR, 2022.
[70] Tian Zhou, Peisong Niu, Liang Sun, Rong Jin, et al. One fits all: Power general time series analysis by pretrained Im. Advances in neural information processing systems, 36:43322-43333 2023.
[71] Tian Zhou, Peisong Niu, Liang Sun, Rong Jin, al. One fits all: Power general time series analysis by pretrained Im. Advances in neural information processing systems, 36, 2024.
13

---

A Appendix Supplemental Material
A.1 Experimental Setting / Details
Table 4: Details of forecasting task designs and datasets. The electricity data represents half-hourly
state-level electricity demand. The exchange rate data represents the daily exchange rate of the Australian dollar. Traffic data denotes the hourly traffic volume in California. Bitcoin data denotes the daily Bitcoin price.
Datasets
lilness
Electicity
Exchange- Ratte Rate
Traffic
Bitcoin
Time Horizon
Not specified
2019.01-2022
2018.01 -02222122
2015.01-22
22.10112016.12
2019.01-2021 -0210000066
Variates
19
7
862
18
Timesteps
966
52,560
1,825
17,544
858
Granularity
week
30 minutes
1 day
1 hour
I day
Input length
24
48
7
24
Prediction length
(4.36,48.60)
48
7
24
7
Baselines. To ensure a thorough evaluation of our model, we selected diverse set of baselines that represent the latest advancements in time series forecasting, covering a broad spectrum of em- pirical studies. Our selection includes transformer-based models, such as transformer-bant
model: Informer (67), Autoformer [59]. FEDformer [69]. Pyraformer [36]. PatchTST [40). iTransformer
38
FiLM (68). Additionally, included the CNN-based Timesnet (58) and MLP-base Dlinear Finally, we incorporated the LLM--baed GPT4TS leveraging GPT2's generative capa-
64
bilities. For Informer [67]. Autoformer [59]. FEDformer (69), Pyraformer [6), PatchTST [40), iTransformer 38 FiLM [68] and Timesnet [58], we use the code from the -ame---rris--ibbaay
http://eeeneiissiiiiieeeeeeeeeee wwwpbbrrrrrrreeeeeeeeeeeeeeeeeee taaan//nn/e//////...mm.. For the GPT4TS method, we use the official code from httpps://giti
(https://githe) . com
When conducting comparative experiments, all the information utilized in our method, except for the news content, is also provided to the baseline methods. While our LLM fine-tuning approach maintains the original scale of the numbers to preserve the physical meaning of the time series data, we normalize the variables to the range [0, 1] for training the baseline methods. For non-n non-numeric variables, represent them using dummy variables for these baseline methods. To ensure optimal performance, we adhere to the official architecture settings for these methods and experiment with learning rates of 0.001, 0.0005, 0.0001, and 0.00005. Each configuration is tested three times with different initializations, and the best-performing setup is selected from all trials.
employ the LoRa 19 method to fine-tune the LLama 2 large language model [55] The rank in LoRa is set to either 8 or 16, depending on the required token length to be processed. The alpha value in LoRa is set 16, and the learning rate is set at 0.0001. When formatting numeric tokens, we uniformly maintain three significant figures. Retaining more significant figures offers minimal benefits and introduces an excessive number of tokens. The fine- tuning is conducted on a single NVIDIA A100 40G GPU, with the large language model undergoing several hundred to 1,000 training iterations on our curated data, requiring several hours.
Our method. When fine- tine-tuning,
Evaluation. We choose five metrics to compare the prediction performances of different forecasting models. Mean squared error (MSE) is used to verify outliers, focusing larger prediction errors. Root Mean Square Error (RMSE) measures eтror magnitude by averaging squared errors and taking the square root, aligning the error scale with the original data. Mean absolute error (MAE) is more balanced for different magnitude errors. Mean Absolute Percentage Error (MAPE) expresses the error as a percentage of the actual values, that is, average how far off the predictions are from the actual values in percentage terms. The accuracy rate is defined as the proportion of true results among the total number of cases examined. These metrics are calculated by:
MSE
Σ
(3
9:)²
MAE Σ
n
1A
n
(1)
1
n
MAPE 100
y/3
0:
RMSE
(U)
D1)2
n
72
3/4
where Yi represents the unnormalized true value of load, and ÿJ denotes the unnormalized result of the prediction model. n denotes the number of prediction timestamps.
14

---

A.2 Example of Numerical Input for Fine-tuning LLM
1. Instruction: "
2. Input: "NSW: 2019-11-9; Weekend; not a public holiday; 2019-11-10; Weekend; not
public holiday:286.5:2977.88 34.0:1012.0; 284.92; 301,0;; 46.0; 1016.0."
3. Output: ".
A.3 Example of Textual Input for Fine-tuning LLM
1. Instruction:
"The historical load data is: 77771577.677..666666644446..333.
2. Input:
"Based on the historical load data, please predict the load consumption in the next day. The region for prediction is NSW. The start date of historical data was on 2019- 1-9 that is Weekend, and it is not a public holiday. The data frequency is 30 minutes per point, Historical data covers 1 day. The date of prediction is on 2019-11-10 that is Weekend, and it is not public holiday. Weather of the start date: the minimum temperature is 286.5; the maximum temperature is 297.96; the humidity is 34.0; the pressure is 1012.0. Weather forecast of the prediction date: the minimum temperature is 284.92; the maximum temperature is 301,04; the humidity is 46.0; the pressure 1016.0. On 2019-1 1-09 08:51:00, the news can change the time series fluctuation that The ongoing fires lead to an immediate and direct effect on today's ' load consumption mostly due to loss of infrastructure, increased demand from firefighting efforts, and the need for emergency communications. On 2019-11- 09 20:20:00, the news can change the time series fluctuation that The devastating bushfires in NSW lead to increased short-term electricity consumption due to emergency services' operations, resident evacuations, and heightened communication needs.
3. Output: " 112.6666666..063312..666666
A.4 News Sources and Details
We analyzed the classical time series forecasting datasets, such as the Monash- TSF dataset, to identify the specific time periods covered by the time series data. For the traffic domain, data from 2015 to 2016 was used. For the Bitcoin domain, data from 2019 to 2021 was utilized. Relevant news information was filtered from the GDELT dataset using domain-specific keywords corresponding to these periods. We conducted web crawling and intelligent parsing of the associated news web pages. The traffic data was sourced from Yahoo, while the Bitcoin data was gathered from various websites. Regular expressions were employed to extract essential information from the parsed text, including news titles, URLs, publication dates, and the main content of the articles. From the GDELT dataset, initially filtered 14.543 traffic-related articles and 19,392 Bitcoin-related articles. After removing invalid and redirected links, we retained 5,867 traffic articles and 5,906 Bitcoin articles. The traffic-related news data primarily covered the region of California. USA. The extracted data was formatted into ISON files to facilitate seamless integration into our model training.
For the region-specific tasks, such as AU elec-
Adelaide Poli t
ICSLife
destturantt
fe
Australia
tricity demand and AU exchange rate, we col- lected news articles spanning from 2015 to 2023
Sereer Faliana
Sport
wwwoorttssreiii Faashhon bourne
primarily sourced from news. com. au In total, we gathered 380,560 articles covering a diverse range of topies pertinent to electricity demand and exchange rates in Australia. The news arti- cles were processed using a similar methodology as described for the traffic and Bitcoin domains. We performed web crawling and intelligent pars-
Fanoey
Food
WI
Hocey
Advics
WO
W
Cor
Schal
iime
Hot eatti
@
Breaking
Busines's ROat Racingg
ing to extract key information from each article, including titles, summaries, categories, URLs, publication dates, and the full article contents. Regular expressions were utilized to ensure the accurate extraction of essential data points. The struc- tured data was then formatted into JSON files, ensuring compatibility and ease of integration into our model training processes. By maintaining a consistent data collection and processing methodology across different domains, ensured the reliability and utility of the gathered information.
CelebrityHed
tn
Werll
Election
Figure 6: Word-Cloud of News Categories.
15

---

Table 5: Statistical Information of Selected Event for Different Domains. In this table, analyze the content and keywords of relevant news selected by the agent for training the model. We performed word frequency analysis, with the second column showing the keywords and their frequencies in parentheses, and the third column representing the total number of selected news articles.
Domains
weyss Selected News Their Frequency
Number Selected News
emergency (339), weather (324), infrastructure (283), commercial events (181), residensial (174),
Electricity
coronavirus (168), economic (160), heating (136), temgerature (134), health (123), bushfires (115),
3,655
constraction (106), global (91), government (84), lockdoown (76). traffii (71)
economic (3033), international (2861), investor (1755), sentiment (1256), currency (1077),
Exchange
Chisa (1058), trade (10+9), USA (991), stability (886). goopolätical (786), financial (689).
4,383
covid (652), health (440), political (328), energy (267), Trump (223), rha (186), tourism (172)
California (1823), USA (1284), road (380), emergency (279), closures (275), commutinn (233),
police (253), disruptions (219), infrastructure (191). media (157) protests (149), congesiion (138),
Traffic
incident (136). transportation (136), security (126), fire (124), wildiire (113), travel (91).
2,109
shooting (85), vehicles (80), university (57), regulations (54)
investor (1 (138)), trading (557), seatinent (519), mining (449), global (262), interest rate (246).
acceptance (240). Satoshi (212),
(209), eoonoiic (196), bi (177), nakahoto (150),
Bitcoin
institutional (140), Elon (132), tochnology (129), Ethercum (119).
2,616
NVidia (114), geopelitical (108), bullish (100), (100), legitimacy (97)
Nen- Kandom Randov Swants
Non- Random Events
160
140
100
Random Events
Figure 8: Pie Chart of the Pro- portions of Random and Non- random Events.
Dasa
Figure 7: Daily Distribution of Random and Non-random Events.
Random events are unpredictable and unplanned, such as natural disasters, accidents, health crises, and criminal acts Normal events are planned or anticipated based on patterns, like political activities, sports & cultural events, economic reports, and public holidays. We used the LLM agent categorize and detect all random and normal events from our raw news dataset spanning January 1st to August 5th, 2019. The analysis revealed that, average, 27.7% of all events random. Figure 7 and Figure show the daily distribution of these random events. In our framework, the agent analyzes and selects the most relevant news, which mainly consists of five categories: economic political events, health crises, natural disasters, technology development, and social sentiment. Additionally, the reflection agent, which analyzes prediction errors in the training dataset and missed news, helps identify unexpected and counterintuitive events buried in the raw news
Table 6: We tested the effects of removing all or partial supplementary information, including Case 3, which includes only Filtered News, and Case which includes Filtered News and Partial Supplementary Information. Removing all supplementary information and keeping only the news led to slight improvements compared to using only Supplementary Information in numerical/textual prompt. However, this approach did not perform as well as Case 4. The results indicate that supplementary information remains essential, Overall, "Case 6: Textual Prompt with Filtered News and all Supplementary Information" which includes all supplementary information and filtered news, yields the best results.
Electricity
RMSE
MSE
MAE
MAPE
Case 1:
Only Numeric Prompt with all Supplementary Information
337.10
113.64
204.89
5.27%
Case 2:
Texuaa Prompe with Supplementary Information (No News)
336.41
113.17
206.08
5.29%
Case 3:
Textual Prompe with Filtered News (No Supplementary Information)
330.79
109.42
201.33
5.23%.
Textual Prompt with Filtered News and Partial Supplementary Information
310.29
192,98
5.19%.
Case 5:
Textual Prompt with Non-Faltered News and all Supplementary Information
407.86
166.35
250.75
6.841%
Case Textua Prompt with Filtered News and all Supplementary Information
280.39
78.62
5.15%
16

---

Table 7:
We tried other language models using our proposed method. Mistral v0.1 [22 similar results Llama (7B). Gemma 2B model [52] had slightly worse results, which may due to its limited number of parameters. It is necessary to adjust the training for small models to achieve better results. Nonetheless, the results demonstrate the potential of language models to achieve good performance in our proposed methods.
7B model, produced
Electricity
RMSE
10-3
MAE
MAPE
Llama 2 (7B)
280.39
78.62
180.96
5.15%
Mistral v0.1 (7B)
291.66
85.07
206.08
5.35%
Gemma 2 (2B)
350.08
122.56
216.72
5.63%
A.5
More Results
Figure Figure Figure 15 and Figure 16 provide showcases of some forecasting tasks results. can seen that our method better results predicting some sudden events and cases with distribution changes. We also tested the effectiveness of fine-tuned LLMs on traditional time series forecasting datasets. Given that long-term forecasting involves a significant number of digits and long sequences occupy many input tokens, we selected the IIl dataset for testing our method, which features prediction sequences of lengths 24, 36, 48, and 60. We also evaluated the performance of univariate forecasts and trained univariate versions of other baseline methods for comparison. The results, displayed in Table D show that, although counterintuitive, the fine-tuned LLMs can achieve performance comparable to baseline methods, providing solid foundation for our research. Table 6 shows additional comparison experiments of different types of data incorporation, and Table 7 shows the comparison of using different pre-train language models.
A.6 Full Prompt Design
A.6.1 Prompt Example Reselecting News through the Reasoning Agent:
The reasoning logic is """Predicting Australia's region-level electricity demand every 30 minutes involves various factors:
Prompt 1:
1. Positive Issues Increasing Load:
(a) Short-Term:
i. Economic Growth: Boosts energy consumption.
ii. Tech Advancements: New technologies spike demand.
iil. Seasonal Factors: Extreme weather increases AC use.
iv. Social Events: Large events boost energy use.
(b) Long- -eng-Term:
i. Population Growth: Raises residential energy use.
ii Industrial Development: Increases energy demand.
iil. Urbanization: Expanding cities raise energy use.
iv. Energy Transition: Shift to electric tech.
2. Negative Issues Decreasing Load:
(a) Short-Term:
Economic Downturns: Reduce industrial activity and energy use.
ii. Efficiency Improvements: Lower consumption.
iii. Weather Patterns: Mild weather reduces heating/cooling needs.
iv. Public Health Crises: Reduce industrial and commercial activity.
(b) Long-Teri -ong-Term:
Energy Efficiency: Better insulation and appliances.
ii. Demographic Changes: Aging populations lower birth rates.
iii. Policy and Regulation: Promote conservation and sustainability.
iv. Tech Innovations: More efficient technologies.
3. Other Factors:
17

---

(a) Political Stability: Affects energy policies and investments.
(b) Global Market Dynamics: Affect local energy prices and use.
(c) Environmental Consciousness: Changes in behavior and renewable adoption. -**
Prompt 2: The prediction date is "2020-06-06". If give you all news before the prediction, based on the above positive & negative effect analysis rewpronnngrraoniingg...o
1. please choose all news that may have long-term affect on future load consumption
2. please choose all news that may have short-term effect on future load consumption.
3. please choose all news that may have real-time direct effect on today's load consumption. if there is no suitable news, please say no.
Also, please include the region (NSW/IICTSS/QLLDSSS Q/DS//WW/ and the time information of this there are multiple relevant news, please ensure that you include all relevant news. Remember to only give the JSON output, including all relevant news, and make the valid JSON format.
Prompt 3: The news happened before the prediction include: <all news>. The selected news is organized in JSON format. The Json output format is
"Long- Term Effect on Future Electricity Demand":
"news": "Another major renewable energy project initiated in WA, expected to supply significant power by 2022."
"region": "MA",
"time": "2019-03-15 11:30:00",
"rationality": "Long- electricity load will be impacted by the integration of renewable energy sources, which expected to offset dependence on traditional fossil fuels."
].
"Short Term Effect Future Electricity Demand":
"news": "SA just sweltered through a very warm night, after day of extreme heat where some regional areas reached nearly 48C.",
"region": "SA"
"time": "2019-01-03 17:57:00",
"rationality"; "Extreme weather conditions, particularly the intense heat, will lead to higher electricity consumption in the short term residents and businesses increase the air conditioning and cooling systems manage temperatures."
}.
{
"news": "A sudden cold snap in Victoria leads to spike in electric heating usage.".
"region": "VIC"
"time" "2019-01-04 05:22:00",
"rationality": "Short-ten electricity load spikes are often caused by unexpected weather events that drive up heating cooling demand."
].
"Real Time Direct Effect Today's 's Electricity Demand": [
"news": "An unseasonal dounpour has wreaked havoc on Perth's electricity network this morning.", "region": "WA",
"time": "2019-01-03 10:11:00",
"rationality"; "The sudden weather event causing disruptions the electricity network can have an immediate impact on load consumption due to power outages, infrastructure damage, emergency response measures."
18

---

}.
"news": "Lightning strike at a major substation causes sidespread outages in Sydney.", "region": "NSW",
"time": 22019-01-03 19:45:00",
"rationality": "Direct effects load consumption include
sudden drops power supply, triggering emergency measures to restore stability in the network."
A.6.2 Prompt Example Evaluating Predictions through the Evaluation Agent:
Based on historical data and relevant news from the last week, I have predicted the future exchange rate of Australia the next day. The base USD. will provide you with our predicted values and actual values, as well as the news references. Please assess the accuracy of the predictions and analyze whether any important news has been overlooked.
Prompt 1:
Prompt 2: This is the background information: <hackground>
Examples of Background Information
" "The start date historical data was 2021-1-3, which a Weekend, and not a public hol- iday. The data frequency is hour per point. Historical data covers 7 days. The Daily GDP of Australia during the last week was 568773,568773,55 5,556773.6677335877..6666
The Daily Unemployment rate (unit: %) of Australia during the last week was Australia during the last week 6,155666.25566.2555555 The Daily Cash Rate Target (unit: %) of The Daily GDP of the United States during the last week was 2200002.266002222222222220202200 222.222...22222222.222202..
255566622266,6,2566666655666
The Daily Unemployment rate (unit: %) of the United States during the last week The Daily Interest rate (unit:%) the United States during the The date of prediction starts 2021-1-10, which is
2.4.6.4.6.466.446666.3
last week the Weekend, and it is not a public holiday. The Daily GDP of Australia on the prediction date 568773. The Daily Unemployment rate (unit: %) of Australia on the prediction date is 6.2556. The Daily Cash Rate Target (unit: %) of Australia on the prediction date is 0.1. The Daily GDP of the United States on the prediction date is 22600.2. The Daily Unemployment rate (unit: %) of the United States on the prediction date is 6.4. The Daily Interest rate (unit: %) of the United States on the prediction date is: 0.1."
This is the news we used for the prediction: <selected news>; Here are all the news, including the day-ahead news and today's 's news in JSON format: <all news>. Please determine whether there was any news that should have been considered in the prediction but was not included. The actual value <actual valuess. <acuual_values>, Predicted values minus actual values are <errors>. The prediction date is <predictions_ting time>. According to the news and errors on this day, determine if any news has been missed. The output format should be: The missed news is xxX, occurred xxxx, the possible reasoning is xxxx.
Prompt 3:
19

---

Examples of Missed News Outputs
Answer: Examining the provided news 2021-01-09 for potentially unconsidered factors in the AUD/USD exchange rate prediction, one significant event stands out that could resonate with economic implications, specifically concerning Australia significant commodity exporter:
Missed News Summary: "Brisbane's 's streets have been empty in eerie scenes as the city experiences its first day of lockdown to stop the spread of coronavirus."
Publication Time: 2021-01-09 12:59:00
Category: Breaking News
This news shows a potential shift in economic activity due to new COVID-19 lockdowns in major Australian city, which could affect the AUD due to changes both domestic consumption and possibly a short-term impact on business conditions. Lockdowns create short-term economic disruptions influencing investor sentiment and risk perception towards the AUD, potentially missed in the prediction model.
Possible Reasoning:
These items suggest an underestimation of contemporaneous socio-economic events that might indirectly influence the AUD/USD exchange rate through shifts in economic expecta- tions and risk sentiment related to the management of COVID-19, which would have been highly relevant for predictions made for 2021-01-10.
Prompt 4: According to coutlooked news>, please directly conclude several new prediction logic of the daily AUD exchange rate. (Remember not output the detailed implementation plan)
Example Answers of News Reasoning Logic Updation
Answer:
1. Increased Risk Aversion: The lockdown in Brisbane may heighten risk aversion among investors, leading to a potential decline in the AUD as investors flock safe-haven currencies like the USD. This can be anticipated as prevalent reaction whenever there are renewed concerns about COVID-19 impacts.
2. Domestic Consumption Impact:
With the Brisbane lockdown, domestic consumption could be considerably reduced. A decrease in consumer spending within Australia might result in lower economic output forecasts, potentially weakening the AUD.
3. Market Sentiment and Speculation: News of the lockdown and its impact Australia's s significant cities can drive market sentiments and speculative trades, which often have immediate effects currency exchange rates. Negative news could prompt bearish sentiment towards the AUD.
of
Impact on Commodity Prices:
As a major commodity exporter, any perceived slowdown in Australia's economy due to lockdowns the AUD. With a decrease in economic activities, demands for commodities might decline, causing domino effect on the currency.
affect global commodity prices, influencing
5. Investor's Perception of Economic Management: Prolonged or frequent lockdowns might affect international perceptions of Australia's management of economic challenges during the pandemic. This perception can influence foreign direct investment flows and currency strength.
A.6.3 Prompt Example Refining the Final Logic
Improve and polish this paragraph reduce repeated content and summarize the news selection logic that affects the Australian dollar exchange rate:<all _ap____p___deeeee
Prompt 1:
According to the given updated logic, please directly rephrase the current prediction logic and output the adjusted new logic. This is the current prediction logic that you need adjust and improve:sselection news logic. current>
Prompt 2:
20

---

A.7 Missed News Examples
By analyzing the relationship between prediction errors and news through the agent, we can identify missing news (such events with indirect impacts) and adjust the agent's expectations of how news events affect time series. For example, the agent might discover that a certain type of incident has a relatively small impact on a specific region. These intelligent and flexible discoveries are detailed in the missed news summary. Here are some examples of missed news during the iterative analysis:
1. Missed News: A coalition of 62 countries is backing Australia's ' call for an inquiry into the origins of the new coronavirus. It comes as Trade Minister Simon Birmingham's 's request for meeting with China went unanswered." Occurred at: 2020-05-16 16 21:15:00 Possible reasoning: While the news about Australia pushing for an inquiry into COVID- 19's origins and facing unresponsive behavior from China was considered, the geopolitical tension this implies may have broader implications. This nature of international relations potentially has consequences for the AUD, particularly in terms of trade volumes and investor sentiment, given China's role as major trading partner. Given the prediction results, this suggests that any immediate effect of this news the exchange rate by the prediction date (2020-05-17) 17) was minimal or offset by other factors considered in the model.
2. Missed News: "Saudi Arabia, the world's top oil exporter, aims to achieve net zero car- bon emissions by 2060, as stated by Crown Prince Mohammed bin Salman." Occurred at 2021-10-25 1 11:1:200. Possible Reasoning: Although this did not happen in Australia, deeper analysis of its impact on the AUD could enhance the prediction framework. This shift towards sustainable energy from major oil producer could influence global oil prices and economic stability. As a significant exporter of commodities, Australia's 's economy and the AUD could be affected. The zero deviation between predicted and actual values suggests either accurate forecasting by other models or delayed effect on exchange rates. Further analysis may be needed understand the long-term impacts on currency values.
3. Missed news: US Senator Bernie Sanders, one of the most vocal crities of Donald Trump, has launched his presidential bid to a crowd of thousands of supporters." Occurred at 2019-03-03 04:30:00. The possible reasoning is that political announcements in a country like the US can significantly impact global financial markets, including forex. Bernie Sanders's candidacy news could have influenced market speculation and investor sentiment, affecting USD values and other major currencies. This detection suggests that incorporating global political news from major economies like the US might enhance the model's predictive capabilities. Significant US political events, like presidential candidacies, can influence global markets and forex currency fluctuations. Thus, overlooking major political news can leave a model vulnerable to missing market reaction surprises.
(a) Australian Housing Market News: Missed News:
"Despite economic uncertainty throughout 2020, Australians confidently embraced the housing market pushing sales volumes beyond even 2019 levels." " Occurred at: "2021-01-17 10:00:00". Pos- sible Reasoning: Although this news might seem domestically focused, the robust activity in the housing market can reflect economic confidence and potentially boost investor sentiment towards the AUD. Healthy real estate market metrics often attract foreign investment and can have favorable ripple effects on currency strength.
(b) Australian Government Seeking Pfizer Vaceine Information: Missed News: "The Australian government is seeking 'immediate' advice and information after Nor- way reported 29 deaths related to the Pfizer vaccine." Occurred at: "2021-01-17 10:10:00* Possible Reasoning: News related to vaccine concerns could create short- term caution among investors, especially in contexts where the vaccine rollout impacts cconomic recovery prospects. If international observers view the vaccine issues as slowdown to Australia's reopening or economic normalization, it could temper enthusiasm for the AUD.
A.8 Selected News by Agents
The news is finally filtered out by the agent based on the prediction task, and the rationale is explained. As shown in Table8 each forecasting domain has corresponding example of selected news.
21

---

Table 8: Examples of Selected News for Different Domains
Publication
News Tide
Rationale
Domati
Region
Time
2/20/2019
Elon Musk Pruises Biicoin:
Publicty negative statements by national banks about Biscoin immediate but trunsieni drops shock reaction investor
Bitcoin
Incerationnal
18:59:21
PPpper Money Soon Going Away'
News
price sentiment.
Angeles police officer sought in fatal bur shooting
real-time enforeessent incident
LA,
22:24:33
immediate rond closures, incressed police activity, and media presence in the srea, significuntly impacting traffic flow consing delays
vicinity the incideat.
N/26/2020 Hundreds of Victorian health workers and lold isolate after outbeeak of CUVID:19 at
This does direcity affect today's load cousumpp- boo signtiicatly imppiie a potential the reduction in operational cupacity and energy usage the hospital.
Electricity
Melhoume,
down
Melbourne
3/13/2019
Interest Rote Decisions: De- by
Immediate resetions to interest rote decisions can cause signiffiant
Exchange
AUl
Bank
AUD/USD exchinge rate global investors adjust portfolics accoedingly.
Australia
interest cates can enhance the AUD's attracting higher yields, particuhely in global low-imieree mon-imeerest--iie environment.
global capital
Table 9: Comparisons of different forecasting methods on the IIl dataset.
Metries
Finetuming
Autoformer
Crossformer
Transformer
PatehTST
TimeNet
24
MAE MSE
0.6341
0.6593
1.4327
0.7150
0.7088 1.0646
0.6495
0.5927
0.5946
0.6143
0.7239
2.9286
0.8164
0.8201
0.5424
0.5626
0.6301
1,2915
0.6875
0,5610
0.6968
0.6602
36
MSE
0.6478
0.5917
2,3478
1,3918
0.6686
0.4971
0.6885
0.8109
MAE
0.7006
0.7246
1,2873
0.7331
0,9404
0.6186
0.6602
0.6996
MSE
0.7033
0.7376
2.2908
0,8099
1,0886
0.6366
0.7237
MAE MSE
0.7673
0.7948
1.3335
1.0400
1.1852
0.7102
0.6976
0.6656
60
0.8358
0.8811
2,4814
1.4858
1.6650
0.7017
0.6991
0.6835
A.9
Reasoning Logic Examples
Figure
Figure 10 Figure 11
and Figure
show some Reasoning Logic Examples.
22

---

Predicting each state's region-level load consumption data Australia every minutes involves understanding diverse factors:
Positive Issues Leading to Increased Load Consumption:
Short-Term:
1.Extreme Weather: Heatwaves cold snaps increase hesting cooling usage, surging electricity demand.
2.Public Events and Holidays: Celebrations an days (like Australia Day or Vivid Sydney Festival, Christmas, or the sudden announcement public holidays) electricity usage,
domestic electricity usage gatherings,
lighting, and celebration preparations
3.Special Shopping Days: High retall activity on days like Black Friday and Boxing Day Sales causes increased electricity consumption.
merrggnncs and News Events: Significant lead higher usage of lighting and electronic devices.
5, S.Power Outages and Public Gatherings: Blackouts and large social events like concerts cause sudden and significant surges in electricity load.
6. 6.Media-Cowe Cover-d Events and Sports: National emergencies, significant broadcasts, and sports drive higher electricity consumption through increased public engagement and operational needs transportation and hospitality sectors. 7.nncreased commercial and public space activity related to the easing of social distancing restrictions. Long-Term:
1. Urban and Demographic Growth: Expanding populations and urbanization elevate base electricity demand.
2. Electric Vehicles and High-Energy Technologies: Adoption electric vehicles and technologies like data centers boost
electricity
3. Construction and Infrastructure Projects: New developments require substantial energy. increasing long-term demand.
4. Renewable Energy and High-Tech Industries: Transitioning to green technologies and integrating automation and robotics industries also electricity consumption.
Negative Issues Leading to Decreased Load Consumption:
Short-Term:
1.Energy Conservation Initiatives: Power-saving programs efficient management
demand.
2.Event Cancellations and COVID-19 Restrictions: Major event cancellations and lockdowns lead to significant reductions in commercial and industrial power usage, partially offset by increased residential consumption. 3.Behavioral Changes
Grid Management: Public energy smooth out demand curves and decrease overall consumption.
efforts and advanced grid technologies
Lorg-Term:
Technologies
Resaurces:
appliances increased
energy
reduce grid dependency.
2. Sustainable Building Designs and Regulatory Frameworks: Net-zero building designs and enhanced energy policies
decrease long-term energy needs.
3. Economic Shifts: Changes from manufacturing service lower overall energy intensity.
Other Factors:
1. Socio-Economic and Regulatory Changes: Lifestyle changes, new policies, and incentives renewable energy
significantly influence energy consumption patterns.
2. Technological and Societal Shifts: Evolving technologies and social norms continually reshape how and when energy used, necessitating dynamic adjustments in energy forecasting. Special
Major events as Sydney's lightning strike, severe weather impacts, Start of Vivid Sydney Festivel, flood events, and public responses to social issues have historically been overlooked load forecasts, affecting their accuracy.
Figure 9: News Selection Logic for AU Electricity Domain
23

---

Predicting the exchange movements AUD/USD requires understanding myriad of factors.
Positive Factors Leading to AUD Appreciation:
Short-Term
1.Rising Commodity Prices: Australia's economy heavily relies on commodity exports; higher global prices can strengthen the AUD.
2.Interest Rate Decisions: Higher interest rates set by the Reserve Bank of Australia [RBA] can attract foreign capital, boosting the AUD.
P.Poitveve Trade Balances: An increase in export volumes relative imports generally strengthens the local currency.
4.Investor Sentiment: Positive economic data political stability enhance investor confidence in AUD.
Long-Term:
L.conomicc Sustained periods robust
performance enhance currency strength. Foreign Direct Investment (FDI): Increased FDI into Australia bolsters economic capacity and currency value,
G.Global Positioning: Strengthening of Australia's strategic economic agreements with other countries.
4.Policy Stability: Consistent and favorable economic policies enhance investor confidence and currency value.
Negative Factors Leading to AUD Depreciation:
Short-Term:
1.Falling Commodity Prices: Decinnes in global commodity prices can reduce the AUD's value.
2.Interest Rate Cuts: Lower interest rates by the RBA make AUD less attractive foreign investors.
N.Neativee Trade Balances: Higher imports than exports can weaken the AUD.
M.Maket Volatility: Uncertain global financial conditions lead to AUD depreciation.
Long-Term:
Ecooooiicc Slowdown: Long-term declines economic performance can weaken the AUD.
Decreased FDI: Lower levels foreign investment can negatively impact the economy the currency.
Policy Instability: Erratic economic policies can reduce investor confidence in the AUD.
A.Goopoiitical Issues: International disputes or regional instability can negatively affect the currency.
Other Factors:
1. U.S. Dollar Strength: As AUD/USD direct quote of the Australian dollar against the U.S. dollar,
strengthening of the USD inherently weakens the AUD/USD rate.
2. Global Economic Trends: Shifts in global economic health, such international recessions booms, heavily influence AUD/USD.
Reserve Bank of Australia Policies: RBA's monetary policy and currency intervention practices significantly impact valuation.
Figure 10: News Selection Logic for Exchange Domain
24

---

predict news will affect Bitcoin's price, it's essential to consider types news events that can cause fluctuations. Bitcoin's s price can also be swayed by range of short-term and long-term events, investor sentiment, and broader economic indicators. Here's summary of key types of events that could affect Bitcoin's price:
Positive Influencers Bitcoin Price:
Adoption by Companies and Countries: News about large companies or countries adopting or investing in Bitcoin can lead to price increases.
Regulatory Approval: Positive regulatory news, example, the approval Bitcoin ETFs or favorable legislation around cryptocurrency, can increase investor confidence and drive up the price.
Technological Advances: Improvements in blockchain technology. security enhancements, and innovations that increase the utility of Bitcoin can also contribute price appreciation.
Negative Influencers Bitcoin Price:
Regulatory Crackdowns: Announcements governmental crackdowns or negative regulations affecting Bitcoin or cryptocurrencies general can lead to price drops.
Security Issues: News about hacks, security breaches, thefts involving Bitcoin exchanges wallets can decrease confidence and negatively impact price.
Market Manipulation Accusations: Reports rumors of price manipulation in the crypto markets can lead significant price volatility uncertainty.
Other Factors Impacting Bitcoin:
Macro-Economic Indicators: Similar to traditional currencies, global economic trends can affect Bitcoin. Economic instabilities, inflation rates, monetary policy changes in major economies, and significant shifts stock markets influence Bitcoin's market.
Public Sentiment and Media Coverage: General public sentiment driven by media coverage can greatly influence Bitcoin's price. Positive news can lead to price increases, while negative news can cause declines.
Market Trends of Other Cryptocurrencies: Often, Bitcoin's performance is linked with that of other major cryptocurrencies. surge drop other crypto coins similarly affect Bitcoin's price.
Predicting Bitcoin's price movements based on news requires not only understanding the itself but also how these events fit into the broader economic, technological, and regulatory landscape. Analysts typically monitor wide range of sources and use both quantitative and qualitative data to gauge market sentiment and potential price movements. Cultural perception and adoption levels different regions also play significant roles.
Predictive Considerations:
Figure 11: News Selection Logic for Bitcoin Domain
25

---

Adjusted Prediction Logic for Assessing Traffic Volume Changes in California:
Weather impact Analysis:
Integrate real-time and anticipated weather conditions, particularly severe phenomena like wildfirre, storms, and snow. Forecast immediate and potential effects on traffic, including road closures and decreased travel speeds due to hazardous conditions.
Infrastructure Dynamics:
Embed updates projected timelines concerning construction activities, road shutdowns, significant infrastructure ventures. Utilize adaptive algorithms to dynamically forecast alternative route usage and resultant traffic flow variations.
3. Event Driven Traffic:
Harness data from local event schedules and public announcements to anticipate the traffic implications of major events such sports games, concerts, or festivals. Predict substantial increases in local traffic volumes participants commute and from locations. Economic Indicators:
Evaluate economic influences such as openings of new businesses, significant layoffs, or industry closures determine shifts in commuting patterns, focusing on movements toward away from economic centers.
5. Transportation Policy Adjustments:
Incorporate latest modifications in traffic rules public transit changes into the models to predict changes in travel behaviors, timings, and route preferences.
Emergency and Incident Response:
Employ real-time monitoring swiftly adjust traffic volume predictions in response accidents, disasters, or other unforeseen emergencies that could result in road obstructions or necessitate evacuations.
7. Political and Social Influences:
Factor in the effects of political rallies, demonstrations, and changes in government policies on road occupancy and public transit, tailoring predictions to the anticipated influence and size of such events.
Summary for Impact Prediction:
Scope and Magnitude: Assess local statewide impact by determining the scale of occurrences better
predict scope
Timing Considerations: Analyze event timing relative traffic peaks anticipate their immediate impact on traffic.
Event Predictability: Differentiate between scheduled, routine events and unexpected emergencies to enhance predictive precision.
Geographic Considerations: Evaluate the geographic location of events to discern if impacts localized widespread, assisting targeted traffic management.
By refining these components, predictive system is boosted in capability efficiently manage, respond to, and adapt to evolving traffic conditions across California, leading to improved traffic circulation and less congestion.
Figure 12: News Selection Logic for Traffic Domain
26

---

Informer
DLinear
Broudd Twil
5000
5000
2000
26
25
FEDfommr
0000
ortund Truth
6000
Iround Troh
caud vilue
Prodited Value
0000
4200
2000
Time
Thme Pyraformer
6000
uetund Trush
6505
fround Wredcssd taku
Predi cted value
5000
5300
4000
4000
Timesh
ITaasformer
Vilue
Fredicedd lake
5000
5000
2060
GPT4TS
Ours
Oreand Truth
Ground Thii
5000
5000
2000
2000
26
22
Dunear
0000
decund Trith
2000
Bir from
Predctsi bokue
mn-simlllkke
0000
6000
Time
Thm
0000
oiound Truth
8005
Brrund wndicoelloee
Pediccted value
3000
6000
3 caco
PatchTST
Pyraformer
Fredicedd laku
7000
7100
Times/
ITaasformer
8060
Oreand Tulb
Ground This
1200
26
22
GPT4TS
0000
8000
Oreane Trth Predicesd volue
Ground Trii Fredicsd Value
6000
6cce
Time
Time
Figure 13: Visualization of electricity demand forecasting results from different methods.
27

---

Informer
Creure arddaed wake
Grandd fech aedizee Wike
10008
10000
Time
Thme
FEDformer
Fredided Yake
6M1
hedecwe Veku
0000
Time
Tin
PatchTST
Pyrafomer
and lake
300
Wedicted Yalu
Time
1inn
TimesNet
iTransformer
Predited Wakes
hedicie Wels
Time
GPMATS
Cuts
taki
200
50000
Wedicaed
Informer
DLinarr
2000
Grand
2008
Ground Tron
1500
J5ce
1000
330e
20
20
60
LDformer
Ground Trubh stad valus
Ground Thuc
Valus
1540
1000
5:00
20
60
500
20
Pyraformer
Ground Truth
Ground Prodiced lakus
fred cied valus
TimesN
irransformer
2000
Predicted Value
2000
Predicced Valus
15:00
310e
1000
2000
GPT4TS
Ours
2000
Ground Trush
2000
Iround Trum
15/00
1100
1000
3100
20
25
Figure 14: Visualization of electricity demand forecasting results from different methods.
28

---

Informer
Sccre Tuhh
Couune
0.32
Neokited Vake
Vake
0.30
Prosided Vike
0.05
8.05
36
Time
1ime
FEDfommer
0.32
Preiites Vake
0.30
Presidted Wike
0.55
0.65
0.00
0.00
PatchTSST
Pyraformer
Ground
Courd
0.60
6.00
tn
16
34
Timeshitt
iTranstrmer
Bround Truh
oe hrth Preditedd Vakee
0.32
Prodsted Vilan
030
8.00
33
Time
Time
GPTATS
Preticuu Value
Prsiided Vilee
0.65
8.00
Informer
DLinear
Ground Buth
Ground
00
0..
06.
Le
5
Time
FEDformer
0.-
Ground Pruth
Ground Prodicted Vaku
Prod ded Valus
02-
001
003
PatchTST
Pyraformer
Groure Puth
Groune
00-
0.-
0..
32
Time iTranstormer
TimesNot
0.-
Gooun Puth Pood ded Volue
03.
Goxed huth Prodictes Vike
Wan 2006
02-
00
Thm GPT4TS
Time Ours
0.-
46
12
Figure 15: Visualization of traffic volume forecasting results from different methods.
29

---

Informer
DLineer
04-
dicund Buth
Gvond Puth Prodicts Vilue
Poodi del Volue
00-
35
Time
Time
F'Dfformer
FILM
Reedictdd
02-
02
16
20
de
10
PatchTST
Pyraformer
0.-
decend Irum
dreen hum Reedcttd Vakue
eodidted Value
0.3-
0.6
06-
Vine
Time
TimesNet
iTransformer
inadided Value
Prodictdd Vako
02-
02-
Lo
20
46
L6
30
GPT4TS
Oufs
Ground Pun and Value
04
Ground Tun irudicted Vaku
0.2
0.2-
0.0
0.-
3b
Time Informer
Time
DLinear
0.22
6:0
0.35
Prekkted Valee
2000
0.3-
frsidde Wake
0.6
0.00
0.00
14
FEDformer
FLMM
0.2
Serand Tub
620
Grundd
0:35-
635
0.32-
6:30-
300
0.05
8:0
10
16
30
30
45
PatchTTST
Pyraformer
0.2
Tut
030
cound Path Proside Wike
Reokted Wike
0:3
030
866
10
30
Time
Time
TimessNot
iTanstomer
0.20
620
0.3
Neekited Vakee
1006
0.33
Preiide like
0.05
8.05
0.8-
6:0
GPTATS
Ours
0.2-
Smare
620
Gourd
g 9035-
$ 635-
033
030
0.8
0.0
10
10
32
Figure 16: Visualization of traffic volume forecasting results from different methods.
30

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The abstract and introduction clearly state the main contributions and scope of the paper.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
The paper includes separate "Limitations" section.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
31

---

The paper does not include theoretical results or proofs, focusing instead on empirical and experimental contributions.
Justification:
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
The paper provides detailed descriptions of the datasets, experimental setups, hyperparameters, and evaluation metrics, which are sufficient for reproducing the main results.
Justification:
Guidelines:
The answer NA means that the paper does not include experiments. If the paper includes experiments, a No answer this question will not perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
specific model and empirical evaluation, may
While NeurIPS does not require releasing code, the conference does require submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm.
(b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way access this model for reproducing the results way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reprodueibility may be tricky in some cases, in which authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path reproducing verifying the results.
5. Open access to data and code
32

---

The authors should provide scripts to reproduce all experimental results for the new reproducible, they Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: The authors have stated their commitment to releasing the code and model upon acceptance, aligning with the NeurlPS code and data submission guidelines.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.c cc/
public/guides/
for more details.
While we encourage the release of code and data, we understand that this might be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https
/ httpppscccppbllccguuudooeo Coaee///ddeSSbbmmcma ssi onPolicy) for more details.
The authors should provide instructions data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
proposed method and baselines. only subset of experiments should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code is permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: The paper thoroughly describes the training and testing details.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can provided either with the code, in appendix, or as supplemental material.
7. Experiment Statistical Significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification: The paper providing clear explanations of the statistical methods used to compute metrics and ensuring the robustness of the reported findings.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
33

---

The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
should be clear whether the error bar is the standard deviation or the standard error of the mean.
It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables figures symmetric error bars that would yield results that are out range (c.g. negative emror rates).
If error bars reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes)
Justification:
The paper provide sufficient information on the computer resources.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't 't make it into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics
Answer: [Yes]
Justification:
The research aligns with the NeurlPS Code of Ethics.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., if there is special consid- eration due to laws or regulations in their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification:
The paper provides a balanced discussion of the potential societal impacts.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
34

---

The conference expects that many papers will be foundational research and not tied particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, scraped datasets)?
Question:
Answer: [Yes]
Justification: The paper ensures responsible usage and mitigating risks associated with potential misuse.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g.. code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: The paper credits the creators of any used assets.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the Datasets.papersstt paperswi thcode. cww/ datasets package should be provided. For popular datasets,
has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
35

---

If this information is not available online, the authors are encouraged to reach out to the asset's 's creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification:
The paper introduces assets which well documented alongside the
paper.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification: The research does not involve crowdsourcing or experiments with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: As the study does not involve human subjects, IRB approvals or equivalent reviews are not required.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
recognize that the procedures for this may vary significantly between institutions and locations, and expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if' applicable), such the institution conducting the review.
36


Reviews:
[{"review_id": "fxVBEnYLdc", "reviewer": "Reviewer_QJWf", "summary": "The paper proposes a novel method to integrate event news as external information into the time series forecasting system.", "questions": "1. Though the idea that relevant event news may positively benefit time series forecasting is intuitively correct, this is not well demonstrated in the paper regarding the datasets used. Specifically, for many datasets tested, it’s hard to imagine what kind of news could dramatically affect them. Of course, it’s impractical to manually evaluate the news considered relevant. Besides the examples already provided, the authors could also test the average number of relevant news items per time window. This could give a rough idea of the distribution of relevant news, which can be used to approximate if the LLM works as expected.\n2. Conflicting news could exist within a time window. For example, Elon Musk may praise or criticize a cryptocurrency within a short time frame. It seems that the model doesn’t consider this situation. Can the authors elaborate more on this issue?\n3. The iterative analysis results show that the performance after each iteration is somewhat random. Though, in general, the final results are better than the initial iteration, it’s actually hard to predict if one iteration will be better or worse between two adjacent iterations. This raises a concern since the huge computational cost may seem unnecessary.\n4. Another concern is that only LLaMA 2’s behavior is tested. Therefore, the sample set of LLMs tested is quite small, making it difficult to predict the behavior of other LLMs using the proposed method.", "limitations": "N/A", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. An important problem is studied in this paper.\n2. An innovative idea of an automatic relevant news extraction mechanism is proposed.\n3. Overall, the presentation is clear and good.", "weaknesses": "1. Some questions regarding the experiments need clarification.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We are grateful for the reviewer's insightful comments and for recognizing the novelty in our work.\n\n```Q1```: **\"Demonstrate the datasets used\"**\n\n```A1```: Thanks for your question. In Appendix A.4, we presented the source details of datasets, and they may answer part of the question. Our news datasets are collected from the GDELT Dataset [30], Yahoo Finance, and News AU. GDELT monitors a wide range of real-time worldwide web news. Specifically, (1) to predict electricity demand and exchange rates in Australia, we gathered 380,560 raw articles covering diverse national and international topics over five years; (2) to predict California's traffic volume, we obtained 14,543 raw articles covering two years in California; (3) For Bitcoin price predictions, we collected 19,392 worldwide articles covering three years. This collection of raw news data provides a foundation for analyzing the impact of various events.\n\n```Q2```: **\"What kind of news could dramatically affect\"**\n\n```A2```: In our framework, the agent analyzes and selects the most relevant news. According to our results, the selected news mainly consists of five categories: economic or political events, health crises, natural disasters, technology development, and social sentiment. For example, fiscal policy changes impact exchange rates, health crises like COVID-19 influence traffic volume and electricity load, AI breakthroughs can affect Bitcoin prices, and political events like elections or new legislation impact exchange rates and electricity demand.\n\nAdditionally, the reflection agent, which analyzes prediction errors in the training dataset and missed news, helps identify unexpected and counterintuitive events buried in the raw news. These might include local incidents causing significant impacts or various events with indirect effects on time series. For example, news about Saudi Arabia’s net zero carbon emissions goal could impact global oil prices, indirectly affecting Australia’s economy and exchange rate. Although this news is not directly related to economic policy or typical keywords monitored, it indirectly reduces oil production for carbon mitigation, which can influence oil prices and exchange rates. Some analysis and examples can be found in Appendix A.7 and A.8. The reviewer can refer to these sections for more details.\n\n```Q3```: **\"Distribution of relevant news\"**\n\n```A3```: Thanks for the question. In the rebuttal file, we provide more statistical distribution details of selected news and their keywords in Table 1, Figure 1, and Figure 2. In our settings (mostly day-ahead prediction), the average number of relevant news influencing a specific domain generally includes 2 to 8 critical events per day. Several examples of selected news contents are provided in Appendix A.8.\n\n```Q4```: **\"Conflicting news could exist within a time window\"**\n\n```A4```: Thanks for raising this critical point. We respond to this question from the following aspects:\n1. Our model does not simply understand individual events in isolation; instead, it considers the aggregation of various events within a specific timeframe. The model can synthesize conflicting information to better understand the overall trend. Different events cause immediate reactions in their respective timeframes. For example, different statements by Elon Musk on Bitcoin will have different effects on its price over time. Our model considers these factors (which are part of our input).\n2. Our reflection agent identifies and adjusts conflicting news by analyzing prediction errors and all the raw news. The agent refines the model’s understanding of how conflicting news affects predictions, ensuring better adaptation to future events and improved accuracy.\n3. When contradictory news appears, we may also need to judge its source and information authenticity. Our agent can flexibly integrate the analysis and reasoning process to determine the authenticity of new news. Reliable sources of news include official media and newspapers.\n4. In the future, we plan to assign different weights to news based on source credibility, relevance, and context. For instance, statements from highly influential individuals will be weighted appropriately. If such conflicts appear, the reflection agent will adjust the weights accordingly.\n\n```Q5```: **\"It's hard to predict if one iteration will be better\"**\n\n```A5```: Thanks for the question. Our findings suggest that, generally, two iterations are sufficient to see significant improvements. Multiple iterations consistently provide better results than only one iteration due to the reflection mechanisms. In our study, we performed multiple iterations primarily to explore the potential for further enhancements and to understand the impact of iterative refinement. This process is crucial during the model training for refining the filtering logic. However, it is not required during the testing phase. For practical applications, the maximum number of iterations should be adjusted based on the prediction timeframe and the types of news being analyzed. For instance, if more categories of news or new trends emerge globally within the timeframe, additional iterations (three or four times) may be needed. Our paper demonstrates one possible approach for better prediction performance. In practical applications or engineering practices, the optimal iteration number can be determined by balancing performance gains against computational costs.\n\n```Q6```: **\"Difficult to predict the behavior of other LLMs using the proposed method\"**\n\n```A6```: Thanks for the question. We add more experiments with other LLMs, such as mistral 7b [R1] and gemma 2b [R2]. The results are shown in Table 3 of the rebuttal file, demonstrating the potential of these LLMs.\n\n[R1] Jiang, Albert Q., et al. \"Mistral 7B.\" arXiv preprint arXiv:2310.06825 (2023).\n\n[R2] Team, Gemma, et al. \"Gemma: Open models based on gemini research and technology.\" arXiv preprint arXiv:2403.08295 (2024).", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "iVcdCRDjOL", "reviewer": "Reviewer_MLnY", "summary": "This paper introduces a novel approach to enhance time series forecasting using Large Language Models (LLMs) and Generative Agents. By integrating news content with time series data, the method aims to align social events with fluctuations in time series to provide enriched insights. The approach involves filtering irrelevant news and employing human-like reasoning to evaluate predictions, continuously refining the logic of news selection and the robustness of the model's output. The results show significant improvements in forecasting accuracy by effectively harnessing unstructured news data.", "questions": "- How does the model ensure the relevance of the selected news items? Could the authors provide more details on the filtering criteria and logic used by the LLM agents?\n- How does the model mitigate the impact of irrelevant or misleading news on the forecasting results? Are there any mechanisms in place to detect and exclude such news?", "limitations": "The authors have addressed some limitations, such as the dependency on the relevance and quality of news data and the complexity of integrating textual and numerical data.", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The proposed framework integrates unstructured news data with numerical time series inputs, enhancing the contextual understanding and responsiveness to real-world events.\n- The use of LLM-based agents for dynamic news selection and analysis is interesting. The agents effectively filter and analyze news content, continuously improving their logic based on forecasting results.\n- The incorporation of news data significantly improves prediction accuracy across various domains such as finance, energy, traffic, and bitcoin, demonstrating the model's ability to navigate complex real-world dynamics.", "weaknesses": "- The performance heavily relies on the relevance of the news data selected. I'm worried that Inaccurate or irrelevant news can degrade the forecasting accuracy.\n- The method may not perform as well in domains requiring highly localized or specific news data that is not available in general news sources.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the insightful comments and for recognizing the potential of our work.\n\n```Q1```: **''Inaccurate or irrelevant news can degrade the accuracy.''**\n\n```A1```: Thanks for the question. Inaccurate or irrelevant news does reduce prediction accuracy, as demonstrated in our paper. Reviewers can check Table 1 of our paper, where _the Case (Textual Prompt Non-Filtered News section)_ contains a lot of irrelevant news, significantly reducing the prediction effect and highlighting the importance of introducing the LLM Agent. Ensuring the relevance of news items is crucial for the effectiveness of our model. Our paper proposes LLM agent workflows (Section 3.2) to select and filter the most relevant news for predictions, thereby improving overall accuracy.\n\n```Q2```: **\"How does the model mitigate the impact of irrelevant or misleading news\"**\n\n```A2```: Thank you for your question. There are three ways to mitigate the impact of irrelevant or misleading news:\n1. **News pre-process**: Before the agents analyze the news, we enhance relevance by ensuring source credibility and the temporal and spatial consistency between the selected news and the time series to be predicted. During the raw news collection process, we prioritize reliable and authoritative sources over less reliable ones. Additionally, relevant news is filtered from the open-source news dataset, considering spatial and temporal connections. For example, for traffic domain predictions, we primarily gather local news from California. We align news data with time series data by matching time frequencies, horizons, and geographical areas. This process selects the first round of raw news specific to each domain, ensuring the general relevance of the news to the time series being forecasted.\n2. **LLM Agents**: We use LLM-based agents to filter and analyze news content, refining raw news into relevant events. Detailed methods are in Section 3.2, with specific prompts in Appendix A.6. Our agents autonomously filter and categorize news based on its impact (positive/negative) and duration (short-term/long-term). The selected news is formatted into structured JSON, detailing the affected area, time, and rationale. An evaluation agent continuously assesses and improves the filtering process by analyzing prediction errors, refining the model’s news selection, and excluding irrelevant or misleading news.\n3. **News Verification**: Our extensible agent workflow can integrate fake news detection techniques to enhance reliability and accuracy. For instance, the agent can use fake news detection algorithms to assess the consistency and credibility of news content, effectively filtering out misleading information.\n\n```Q3```: **\"Filtering criteria and logic\"**\n\n```A3```: Thanks for your question. Our filtering logic for selecting news involves a multi-step reasoning process. First, our agent uses several Chain of Thought (CoT) prompts to understand the details of forecasting tasks and automatically identify time series influencers within the specific domain. This forms the initial filtering logic, which instructs the agent to sort news by impact (positive/negative) and duration (short/long-term), considering factors such as economic, policy, seasonal, and technological elements. The reasoning agent then filters and categorizes news based on this logic, focusing on its relevance to the time series and classifying the impact as long-term, short-term, or real-time. Additionally, the prediction time span is considered for further filtering of news. For instance, if day-ahead or real-time predictions are needed, long-term influencers will be filtered out.\n\nTo refine this logic, we also deploy an evaluation agent that assesses prediction accuracy and uses logical reasoning to identify and correct inaccuracies from missing or irrelevant news. The evaluation agent works in three phases:\n1. Input forecasting task type, time horizon, and background information to generate evaluation steps.\n2. Analyze ground truth data, prediction errors, and selected news to identify overlooked news.\n3. Update the logic based on its findings, refining it for future news selection.\n\nThe refined filtering logic ensures the news data selected for time series forecasting is relevant and reliable, enhancing the model's predictive accuracy. For more details on the filtering logic examples and prompts, please refer to Section 3.2, Appendix A.6 and A.9.\n\n\n```Q4```: **''Not perform as well in domains requiring highly localized or specific news data''**\n\n```A4```: Thank you for your question. Relying solely on open-source datasets may not always be optimal for domains requiring highly localized or specific, unpublished news data. However, our paper demonstrates that including more relevant news can significantly enhance prediction accuracy, potentially improving traditional methods. While localized news may be hard to obtain, public events can still impact time series data related to human activities. Our results show that as the quality and relevance of news data increase, so does prediction accuracy. The proposed framework can also be adapted to incorporate localized and specific news sources, enhancing performance in specialized domains. Our paper demonstrates this potential of incorporating textual news into time series forecasting.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "jG4EnKksiO", "reviewer": "Reviewer_UwQn", "summary": "This paper proposes a new framework for time series forecasting. This framework fine-tunes a generative large language model (LLM) to improve forecasting accuracy by integrating news and supplementary information with numerical data and introducing iterative self-evaluation through LLM-based agents.", "questions": "Question 1: Apart from prediction accuracy, how to demonstrate the improved reliability? From line 144 to line 145, do sudden shifts embedded in random events from news context help the framework improve prediction reliability?  \n\n \n\nQuestion 2: From line 216 to line 217, is the understanding of time series influencers, or the sorting based on impact and duration, developed manually by people or automatedly by the LLM agent? What is the difference between such an understanding and a given reasoning logic mentioned from line 218 to line 219?  \n\n \n\nQuestion 3: From line 277 to line 278, apart from news articles, are there any ablation studies studying the accuracy differences between keeping and removing partial components of or the entire supplementary information? \n\n \n\nQuestion 4: According to Table 2, why would introducing more rounds of reasoning selection decrease the forecasting accuracy?", "limitations": "The authors have adequately addressed the limitations and the potential negative societal impact of their work.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "Strength 1: This paper has good originality to identify a unique challenge in time series forecasting task, which is the lack of effective modeling to address the distortions induced by additional random events with time going by.  \n\n \n\nStrength 2: Through Figures 1, 2, 3, and 4, this paper has good clarity to describe the proposed framework and the detailed procedures to complete the time series forecasting task.  \n\n \n\nStrength 3: This paper conducts comprehensive experiments, using time series datasets across multiple domains, to demonstrate the effectiveness of the proposed framework.", "weaknesses": "Weakness 1: There could be some statistical analysis on random events compared with normal events which represent a universal knowledge distribution with time going by. \n\n \n\nWeakness 2: It seems a bit redundant and contradictory for methods 1) and 2). The description of the three-phase prompting could be better organized. \n\n \n\nWeakness 3: Ablation studies and sensitivity analyses are encouraged. Based on the description of the four scenarios from line 290 to line 297, the news and the supplementary information are always integrated throughout the experiments. \n\n \n\nWeakness 4: There could be more detailed analysis for Table 2, especially regarding the roles of the evaluation agent.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks to the reviewer for the insightful feedback and for highlighting the originality of our work.\n\n```Q1```: **\"Statistical analysis on random events compared with normal events\"**\n\n```A1```: Thanks for your question. To address it, let me first define random and normal events. Random events are unpredictable and unplanned, such as natural disasters, accidents, health crises, and criminal acts. Normal events are planned or anticipated based on patterns, like political activities, sports & cultural events, economic reports, and public holidays. We used the LLM agent to categorize and detect all random and normal events from our raw news dataset spanning January 1st to August 5th, 2019. The analysis revealed that, on average, 27.7% of all events are random. Figure 1 and Figure 2, presented in the rebuttal PDF, show the daily distribution of these random events.\n\n```Q2```: **\"The description of the three-phase prompting could be better organized\"**\n\n```A2```: Thank you for the feedback. We’ll clarify the distinction between Method 1, which covers the theory and mathematical modeling of how news affects time series predictions, and Method 2, which outlines practical steps for implementation, including dataset preparation, fine-tuning language models, and agent design. To improve clarity, we’ll present the three-phase prompting more clearly and concisely, moving from the conceptual idea to practical implementation, showing how these elements work together to enhance predictions. This reorganization will eliminate redundancy and improve the presentation.\n\n```Q3```: **About ''prediction reliability''**\n\n```A3```: Thank you for your insightful question. While sudden shifts in random events introduce volatility, our integration of agent reasoning helps the model effectively manage these changes. The agent does not simply translate individual events; it intelligently understands and processes news into concise summaries and rationales in standardized formats, focusing on the impact direction, impact duration, and impact scale of different events. This approach ensures consistent categorization of various random events as positive or negative, maintaining the model’s reliability. Additionally, we ensure prediction consistency through iterative reflections, demonstrating that the model produces stable results. Correlating prediction errors with missed news helps mitigate unexpected prediction deviations, further highlighting the model’s reliability.\n\n```Q4```: **About line 216 -- line 219: \"...difference between an understanding and a given reasoning logic...\"**\n\n```A4```: Thank you for your question. The LLM agent can automatically form the understanding of time series influencers, and providing a given reasoning logic in our models is optional. In the automated process, the agent forms its logic through prompts designed to help it determine how different types of news affect a specific domain. For example, we use open-ended questions to allow the agent to independently summarize and develop filtering logic. User knowledge can also be incorporated into these prompts as a given reasoning to help the agent generate more comprehensive logic. The agent then filters news based on the generated logic, either fully automatic or including user-provided input.  We will revise the paper to clarify these points.\n\n```Q5```: **\"Removing partial components of or the entire supplementary information\"**\n\n```A5```: Thanks for the question. We add more experiments with different training datasets (1. removing partial supplementary information; 2. removing the entire supplementary information). The results are shown in Table 2 in the rebuttal file.\n\n```Q6```: **\"Why more rounds decrease the accuracy?\"**\n\n```A6```: Thanks for raising this important question. Optimizing the number of iteration rounds is crucial for achieving the best prediction accuracy. Our findings in Table 2 of the paper show that multiple rounds generally enhance logic and provide more comprehensive insights compared to a single round, without inherently reducing accuracy. Typically, two or three rounds are sufficient enough for significant improvements. However, beyond this, additional rounds may introduce more complexity and noise, potentially affecting accuracy. In our study, we performed multiple iterations primarily to explore the potential for further enhancements and to understand the impact of iterative refinement. While we demonstrated the effectiveness of multiple rounds, determining the optimal number remains an ongoing challenge. This involves refining the evaluation agent’s workflow to balance reasoning depth with the risk of noise accumulation. Our future work will focus on finding this balance to ensure optimal predictive performance.\n\n\n```Q7```: **\"More detailed analysis regarding the evaluation agent\"**\n\n```A7```: Thanks for the question. The evaluation agent enhances news filtering by analyzing and relating prediction errors with potential missed news during training. It examines ground truth data, prediction errors, selected news, all raw news, and the forecasting task type to identify overlooked news. For example, if there is a significant discrepancy between the predictions and the actual values during a certain time window, it is necessary to closely examine the recent news from that time frame and look for any relevant events that may have been missed. This refines the model’s understanding of how missing news affects predictions. Insights from this analysis are then used to generate updated logic for subsequent news selection rounds, which is consolidated into a final version after processing all iterations for validation sets. Through this iterative evaluation, the model continuously improves its understanding of relevance. More details are in Section 3.2 and 3.3, Appendix A.6 and A.7.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "TrCxdi5OU0", "reviewer": "Reviewer_pDM7", "summary": "This paper introduces a time series forecasting framework, where LLM-based agents are employed to sift out relevant news to time series of interests and the news are utilized to enhance the accuracy of time series forecasting models.", "questions": "For real-world application, we may have to collect up-to-date news related to time series of our interests, do the authors have any suggestion of the amount of news?", "limitations": "Limitations are well discussed in the manuscript.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The idea of filtering and utilizing news to enhance time series forecasting is innovative and interesting.\n2. The whole framework is well designed. Each component is with reasonable motivation.\n3. The experimental result is impressive, showing great superiority of the proposed framework on time series forecasting.", "weaknesses": "1. Time consumption of the framework should be discussed.\n2. The components in the framework are inherited from existing works, making the model itself not as innovative as the idea of the paper.\n3. Collecting-then-filtering mechanism of the proposed framework might have negative effects on the real-time performance of the model.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the insightful comments and for recognizing the innovation of our idea.\n\n```Q1```: **''Time consumption''**\n\n```A1```: Thanks for the question. The time cost of our method can be divided into training time and inference time. For a dataset with 3,500 time series, training requires approximately 10-15 A100 GPU hours, translating to 1-3 hours in a typical multi-GPU setup. Inference follows the normal language model’s reasoning time, with each instance taking from a few seconds to over ten seconds, depending on the output sequence length. With efficient information integration channels, obtaining real-time news can be very rapid, making real-time prediction feasible. We will include this information in the revised paper.\n\n```Q2```: **''Model itself not as innovative as the idea of the paper''**\n\n```A2```: Thanks for the question and for recognizing the innovation of our idea. We want to emphasize that our method is equally innovative. Our approach goes beyond model training to include **data construction methods, LLM agent construction methods, and a framework for integrating the agent with the model**. We innovatively combine these elements to utilize LLMs for time series forecasting, incorporating supplementary information and news. We introduced an LLM agent for reasoning and assistance within the LLM fine-tuning framework for time series prediction. We also proposed a Reflection mechanism to evaluate the LLM output of time series forecasting. These practices are novel in the literature and are crucial to our work. We believe these innovations will provide valuable insights and contribute to other projects leveraging LLMs to address domain-specific challenges.\n\n```Q3```: **''Real-world application''**\n\n```A3```: Thanks for your question. The amount of recent news required for practical applications varies based on the frequency, duration, domain, and geographical coverage of the specific time series forecasting tasks. Here are some general suggestions:\n\n**Amount of News**: For national-level events, we typically use an average of 100,000 news articles covering one year for model training. For week-ahead forecasting, about 1,000 articles covering one week provide a robust dataset, while day-ahead forecasting requires around 100 articles per day to cover a broad range of events. Our news sources include the GDELT [30], Yahoo Finance, and News AU.\n- Australia’s Half-Hourly Electricity Demand and Hourly Exchange Rate: We collected 380,560 raw articles/events over five years.\n- California’s Hourly Traffic Volume: We gathered 14,543 raw articles/events over two years.\n- Daily Bitcoin Price Prediction: We collected 19,392 raw articles/events from around the world over three years.\n\n**Real-Time News Collection**: We recommend using automated tools and APIs for real-time news collection. For example, GDELT 2.0 [30] updates every 15 minutes, capturing global news events in 65 languages, providing insights into various themes, emotions, and activities. This approach ensures an up-to-date dataset without manual intervention. It’s crucial to ensure the reliability of news sources by selecting official media channels and maintaining a sufficient number of high-quality, relevant, and diverse news items to improve the accuracy and reliability of time series forecasting.\n\n```Q4```: **''Collecting-then-filtering mechanism might have negative effects on the real-time performance''**\n\n```A4```: Thank you for your valuable feedback. The collecting-then-filtering mechanism is essential for ensuring the accuracy and reliability of the model's predictions. By initially collecting a broad range of news data to train the model, we use this mechanism to remove irrelevant news, thereby enhancing the overall quality of the input data for time series forecasting.\n\nWhile the training phase may require considerable time to analyze all raw news covering long periods in the datasets, the data collection process during the real-world testing phase can be automated, thanks to the availability of real-time news datasets, such as the GDELT Dataset [30] and Yahoo Finance API. On average, analyzing and filtering daily news requires 30 seconds to 1 minute, with each model's prediction instance taking from a few seconds to over ten seconds. This overall processing time is reasonable.\n\nIn practical applications, there is often a trade-off between speed and accuracy. Although the collecting-then-filtering mechanism may introduce a slight delay, it significantly improves the model's accuracy and reliability. We believe that this trade-off is justified, especially in scenarios where precise and dependable predictions are critical. Our framework is also extensible and can be adjusted to improve real-time capabilities, and we will continuously explore further improvements to enhance both performance and accuracy.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
We thank all reviewers and area chairs for their valuable comments. We are pleased that all reviewers have responded positively to our paper. They acknowledge that our work addresses an important problem (Reviewers QJWf, UwQn), introduces an innovative idea (Reviewers QJWf, UwQn), and demonstrates good results (Reviewer UwQn, pDM7). Additionally, they appreciate the clear presentation (Reviewer QJWf) and find the work both interesting and meaningful (Reviewers MLnY, UwQn, pDM7).

Reviewer QJWf (denoted as Reviewer 1), Reviewer MLnY (denoted as Reviewer 2), Reviewer UwQn (denoted as Reviewer 3), and Reviewer pDM7 (denoted as Reviewer 4) all give insightful comments. To answer their questions, we provide corresponding clarifications and analysis. Besides, we provide more experimental results. We summarize this rebuttal as follows:

1. According to Reviewer 1 and Reviewer 3’s questions, we add a statistical analysis of both selected news distribution and random event distribution during a specific time window. The results are illustrated in Table 1, Figure 1, and Figure 2 in the rebuttal PDF file.

2. According to Reviewer 1’s concerns, we add new experiments which use other language models with our proposed method. The results are in Table 3 of the rebuttal PDF file.

3. According to Reviewer 2 and Reviewer 3’s questions, we add more descriptions of our proposed method. We give a more detailed analysis of agent workflow for filtering irrelevant news.

4. According to Reviewer 3’s concerns, we add new experiments about the particular case of removing all supplementary information and partial supplementary information. The results are in Table 2 of the rebuttal PDF file.

5. Inspired by Reviewer 4’s comments, we discuss the time consumption and suggestions for the real-world application of our work in more detail.

In the final version, we will improve other minor points of Reviewer 1, Reviewer 2, Reviewer 3, and Reviewer 4. Thank you all for the valuable suggestions.

Best,

Authors

Meta-review: This paper introduces a novel framework to integrate unstructured news with time series data for time series forecasting by using Large Language Models (LLMs). Empirical experiments demonstrate the effectiveness of the proposed framework by a noticeable improvement over baselines.

All the reviewers are positive about this submission. They also agree with the significant contribution of the paper such as novelty, strong empirical results. Concerns and issues raised by reviewers were properly addressed during the discussions with authors. They also promise to incorporate this improvement in the camera-ready version which we’re looking forward to.

Overall, this paper presents a solid and interesting work, I would love to recommend its acceptance .
Decision: Accept (poster)