Semantic-Enhanced Time-Series Forecasting via Large Language Models
Authors: Hao Liu
Zhang xiaoxing
Chun Yang
Xiaobin Zhu
Venue: ICLR 2026 Poster (2026)
Keywords: Large Language Models; Time Series Forecasting; Semantic Ehanced; Time-Adapter
Primary area: learning on time series and dynamical systems

Abstract: Time series forecasting plays a significant role in finance, energy, meteorology, and IoT applications. Recent studies have leveraged the generalization capabilities of large language models (LLMs) to adapt to time series forecasting, achieving promising performance. However, existing studies focus on token-level modal alignment, instead of bridging the intrinsic modality gap between linguistic knowledge structures and time series data patterns, greatly limiting the semantic representation. To address this issue, we propose a novel Semantic-Enhanced LLM (SE-LLM) that explores the inherent periodicity and anomalous characteristics of time series to embed into the semantic space to enhance the token embedding. This process enhances the interpretability of tokens for LLMs, thereby activating the potential of LLMs for temporal sequence analysis. Moreover, existing Transformer-based LLMs excel at capturing long-range dependencies but are weak at modeling short-term anomalies in time-series data. Hence, we propose a plugin module embedded within self-attention that models long-term and short-term dependencies to effectively adapt LLMs to time-series analysis. Our approach freezes the LLM and reduces the sequence dimensionality of tokens, greatly reducing computational consumption. Experiments demonstrate the superiority performance of our SE-LLM against the state-of-the-art (SOTA) methods.

Published as a conference paper at ICLR 2026
SEMANTIC-ENHANCEI TIME-SERIES FORECASTING VIA LARGE LANGUAGE MODELS
Hao Liu', Xiaoxing Zhang", Chun Xinobiii Zhu' Universiity of Science and Technology Beijing, Yizhi, China Telecom d2024104410xs ustb. edu. cn, 2hangxx7@chinater cn, chunyangeustb... edu. en, whuxiacbin@ustb.com www.eeuuuuuueeeuee........eeee.e
ABSTRACT
Time series forecasting plays a significant role in finance, energy, meteorology. and IoT applications. Recent studies have leveraged the generalization capa- bilities of large language models (LLMs) to adapt to time series forecasting, achieving promising performance. However, cxisting studies focus on token-level modal alignment, instead of bridging the intrinsic modality gap between linguistic knowledge structures and time series data patterns, greatly limiting the semantic representation. To address this issue, we propose a novel Semantic- LLM (SE-LLM) that explores the inherent periodicity and anomalous character- istics of time series to embed into the semantic space to enhance the token em- bedding. This process enhances the interpretability of tokens for LLMs, thereby activating the potential of LLMs for temporal sequence analysis. Moreover, ex- isting Transformer-bast bassed LLMs excel at capturing long-range dependencies but are weak at modeling short-term anomalies in time-series data. Hence, we pro- pose plugin module embedded within self-attention that models long-term and short-term dependencies to effectively adapt LLMs to time-series analysis. Our approach freezes the LLM and reduces the sequence dimensionality of tokens, greatly reducing computational consumption. Experiments demonstrate the supe- riority performance of our SE-LLM against the ofate-of-the--at (SOTA) methods.
INTRODUCTION
Time series forecasting has widespread applications in various tasks, such finance, energy, meteo- rology, and industrial IoT (Liu et al. 2016 Arslan 2022 Sherstinsky 20118). Recently, Transformer-
based LLMs (Radford et al. 2019: Devlin al. [2119] Yang et al. 2024a] Touvron al. (023) are
prevalent in time series forecasting (Jin et al. [2024] Liu et al. 2024c] 2025| Hu et al. (0255 by in-
tegrating multi-domain knowledge to model temporal dependencies, thereby enhancing time series analysis effectively.
Time series data fundamentally differs from linguistic structures in nature. Existing methods often convert temporal information into textual prompts activate the potential of LLMss. GPT4mTS (Tia ctt al. 2024a) introduces a prompt-based framework that jointly processes temporal data and textual information. Zhou et al. (Zhou et al. 2023) processed time-series data by randomly initializing the embedding layer and directly performing serialized amalysis through frozen LLMs. However, the inherent modality gap between temporal and linguistic data resulted in low-quality embeddings. In this regard, LLM4'S (Chang et al. 2023) introduces a unified framework that fuses token, posi- tional, and temporal embeddings to maintain modality consistency. Time-LLM (Jin al.) 2024) converts time-series data into text prototype representations and integrates textual prompts to en- hance time series analysis. Meanwhile, S2IP-LLM (Pan et al., (2224) incorporates time-series data into LLM's semantic space as prompts, compensating for the lack of textual prompts. However, incorporating textual information may introduce noise (Liu et 2025), and the text descriptions incur additional computational cost (Lin et al. 2024 Ma et al. 2023 Shang et al. 2024).
Most existing LLM LLM-based methods mainly focus on token-level alignment (Hu al. 2025). ) as
shown in Fig. (a) (Pan et al. 20244a Liu et al. 2025). The joint space serves as an implicit prompt,
Corresponding author. The code is available hups://gitub.ccom// http://uhh..ooom////////////L..
H335//FFLLI.MM

---

Published as a conference paper at ICLR 2026
Word Embedding
LLM
Semantic Association Dynamic Evolution
Semantic Space
LLM
Joint Space
Word Embedding
Alignment
Semantic Fhanced
Time-Adapter
Semantic Space
IS Emiedding
Semantic Ehanced
Embodding
(b)
Figure (a) The input time series is mapped to obtain time series (TS) Embeddings. Next, feature space is aligned with the semantic space derived from the pre- trained word token embed- ding. (b) Temporal patterns are injected into the joint space and use implicit prompts to LLM for forecasting. (c) Emhedding Time Adapter enhances learning of temporal patterns.
guiding the LLM in conducting time series forecasting. Semantic tokenization eliminates the de- pendency on auxiliary textual descriptions. However, this token-level alignment overlooks temporal and channel dependencies, making difficult to capture dynamic temporal patterns. To address analysis identifies critical oversight in existing LLM-based approaches (Jin et al. 20024 Liu et al.) [024c] Chang et al.) 2025 Zhou et al., 2023). Regarding the semantic space composed of tokens and time steps, propose the Temporal -Semantic Cross-Correlation (TSCC) Module to obtain enhanced semantics that represent temporal patterns, emphasize anomaly and de- anomaly patterns, thereby enhancing the interpretability of the token embeddings, as shown in Fig, (b). Although we provide semantic guidance with high interpretability from LLMs, there exists an inherent difference between LLMs' understanding of linguistic knowledge and temporal pattern. We propose to freeze the LLM and introduce an adapter to transform the model's ' semantic information into the capability of modeling dynamic changes in time series, as shown Fig. (c).
these limitations,
Seemporal-Semnttc
In summary, we propose Semantic-Enhanced LLM, novel framework that enhances LLM for time series forecasting with two key components: the Temporal-Semantic cross-Correlation (TSCC) module and the TimeeAdapter architecture. The TSCC captures cross-correlation of temporal em- bedding and semantic space with temporal dependencies and channel-wiss relationships, which em- bed into token embedding to adapt LLM to temporal features. The Time-Adape enhances fore- casting capability by overcoming the limitation of LLMs in modeling both short-term and long-term dependencies based on historical time segments. This enables pretrained LLMs to forecast time series effectively. Our main contributions are four-fold:
We introduce a novel SE-LLM bridge the inherent modal differences between time ries and language data, activating the generalization capability of LLMs for time-series analysis.
We propose novel Temporal-Senen Serrantic Cooss-Correlation (TSCC) module to enhance to- ken embedding by endowing semantic information with temporal patterns, which improves temporal analysis.
We propose Time- Time-Adapter to bridge the modality gap between LLMs and time- series data, effectively modeling both long-term dependencies and short-term patterns.
Extensive experiments verify the superior performance and efficiency of our SE-LLM.
2 RELATED WORKS
2.1 TIME SERIES FORECASTING
Traditional time series forecasting methods, such as ARIMA (Liu et al. 2016) and Prophet (Arslan
2022). often relied statistical and classical machine learning techniques. The rise of deep learning
brought Recurrent Neural Networks (RNNs) (Sherstinsky) 2018 Qin et al., 2017; Yeo et al. 2018
Khaldi et al. 2023), which excelled at modeling temporal dependencies. Long Short-Term Memory
2

---

Published as a conference paper at ICLR 2026
2023 improved RNNs by addressing vanishing gradients and Long-range dependencies. Advances PGN (Jia et al. 2024b) further enhanced long- term temporal relationship modeling, and improved computational efficiency. Concurrently, Graph Neural Networks (GNNs) (Cao et al. 2020; Pan et al. 2024b Deng et al. 2020) excelled at cap- turing spatial and structural dependencies in spatiotemporal contexts. Recently, Transformer-based architectures (Chen et al. [202]] Zhang & Yan 2023 Wang et al. (2024) have dominated due to their ability to capture global temporal patterns. However, these models struggle with generalizability and flexibility in highly dynamic, woaldworld time series analysis.
(LSTM) networks( et al. 2025 Shahzad et al.
2.2 LLM-BASED TIME SERIES FORECASTING
Recently, LLMs Radford et al. 2019 Touvron et al. 2023 Zhang et al. 2022
DeepSeek-Al
2024 Yang al. 2024a) have developed rapidly, and demonstrated significant potential in advanc-
ing time-series forecasting. Gruver et al. (Gruver et al. 2023) and Xue et al. (Xue & Salim 2023)
reformulated forecasting as sereennce-t--eneenee translation task to bridge the modality gap. Time-
LLM (Jin et al., 2024) leverages token-level embedding as prompt to enhance the textual descrip-
tion to enhance LLM's understanding of temporal concepts. Pan et al. (Pan al. (2024a) integrate
semantic space with embedding prompt to guide LLM. Liu el al. (Liu et al.] (025) indicate that the embedding of textual information weak, and they proposed an entangled reliable embedding. Hu et al. (Hu et al. 2025) introduced FSCA, which utilizes Graph Neural Networks (GNNs) to ensure structural and logical alignment. AutoTimes (Liu et al. (024c) explored autoregressive fore- casting with timestamp embedding to improve LLM's performance. Notably, the aforementioned studies and most research (Jia et al. 2024a) employ a frozen LLM approach for time series learning, indicating that LLMs have already acquired rich general semantic recognition capabilities during the pre-training weight. If all parameters are directly fine-tuned, these general capabilities may be com- promised, leading to unstable performance in time series tasks. Current research merely leverages the generalization ability of LLMs for downstream tasks, suggesting that the inherent architecture of LLMs is not well-suited for time series forecasting.
2.3 ADAPTER-BASEL LLM FINE-TUNE
Existing methods show that fine-tuning LLMs etal 2024 Zhou et al.
2023
Gruver et al.
2023)
not only consumes computational resources but also degrades forecasting performance across data from different domains. Adapter-based fine-tuning techniques (Hu al.
2022 Guo et al. 2024
Zhang et al. 2023 Lester et al., 2021: Zhang et al. 2025) have demonstrated efficient adaptation
to task-specific distributions in various task (Liu al.] 2024a| Chen et al., 2024 Mu et al. 2025).
Empirical studies, e.g., Gupta ef al al. (Gupta et al., 2024), validate the effectiveness of LoRA-style fine-tuning in boosting time-series forecasting performance. AutoTimes(L.in Liu et al., (202c)) incorpo- rates LoRA (Hu et al., (002)) into GPT-2 (Radford et al. 2019), achieving substantial improvements in long-range prediction scenarios. However, the success of these fine-tuning techniques is highly contingent upon algorithmic design choices (Pan et al., 2024a; Zhou et al., 2023 Jin et al.) 2024), and the transferability of these methods across diverse time-series domains remains unclear. We contend that current adapters, while enhancing LLMs' understanding of semantic structures, not designed to capture temporal patterns effectively. In contrast, the Adater is specifically de- signed to model short- and long-term dependencies, boosting LLMS adapt temporal forecasting tasks.
3 METHODS
3.1 THE OVERVIEW OF SE-LLM
The proposed SE-LLM model consists of two primary components: the TSCC and the Time- Adapter, as depicted in Fig. 2 The process begins with the input sequence, time series data matrix with batch size B and sequence length L. A sliding window operation is applied partition the temporal dimension into smaller segments, restructuring the representation and reducing com- putational complexity from O(L?) to O(N2K), where T + T, T E PBxXXX This design reduces self-attention and forward network complexities, improving the model's efficiency.
3

---

Published as a conference paper at ICLR 2026
Semaoral-Semannic
Concat
Multi-lead
Tayer
Semantic Space
Gated Fusion
Decader
TS Embeediig
Fusion
Lange Language Model
TSCC
Word Embedding
Scaled Pro-Producct
Doaneraly
Anceadly Muci
TSCC
L Lincer
Lince
Linear
Semantic Space
TS Embedding
VAB
V
Wond Encoder
Time Encoder
Cross Attention
Time- Adapter
(n)
(b)
(c)
Figure 2: (a) An overview of SE-LLM. Red indicates trainable, while blue indicates frozen. (b) The overall framework of TSCC, with detailed processes referring to Fig. B (c) For the Transformer- based LLM, the Time-Adapter embedded within the key and value vectors of the multi-head attention mechanism. JS is an abbreviation for Joint Space.
TS Embedding. The temporal representation is passed through the Time Encoder module, which projects the data into a high-dimensional TS Embeddings, which can be formulated as:
= F2 (a (Fi(T)))
(1)
where H E (BXNXCCFFLC F2, is linear model, a is activation. This projection captures more ex- pressive features of the temporal patterns within the data.
Semantic Space. The semantic space is the semantic embedding obtained by mapping the word em- bedding of large language model through a linear layer, which collectively refer the seman- tic space. Specifically, we map the Word Embedding layer from W E IR VxC to S E R (Fang al. 2025). The pre-trained word embeddings from LLM are used as general features, and linear transformation is applied to transfer the features new semantic space. allowing the model to match the time series of the TS Embeddings.
We input both the semantic space and temporal features into the TSCC to obtain enhanced seman- ties, as shown in Fig. bb). Following this, the LLM embeds Time-Adapter into the Multi-Head Attention to further refine the model's ability to understand temporal patterns in the time series data, as shown in Fig. cc). This component enhances the overall temporal pattern recognition by integrat- ing additional contextual information, enabling better forecasting performance. Finally, the Decoder component decodes the output from the L.M to produce the final prediction as:
O F2 (a (Fi(Y)))
where Y is the LLM's output. The test processes are provided in Appendix A
Gated Pasion
Linear model
TS
Guted Mechatisn
Fealane
Cincalcnale
Spece
Fibhoddng
Joknt
DA
DC
Space
Joint Space
Cross Anturmon
VAE
DC
Semantic Spuce
M
TS Embetking
TS Embedding
multipicaation
Hlmeen--ase
Figure 3: Architecture of TSCC module, where DA and DC correspond to "de-anomaly semantic" and "anomaly semantic".
4

---

Published as a conference paper at ICLR 2026
3.2 SIMPPORL-SEMANTEE CROSS- COSSEEOORRNNNIONN
Fig. iis the architecture of the TSCC module. The core of this module is to infuse temporal patterns into the joint space. Firstly, we align the temporal embeddings with the semantic space through
cross-attention mechanism (Jin et al., 2024 Liu et al. 2025 Chang et al. (025)..
Cross-Modality Alignment. The Cross-Domain Attention Mechanism is capable of dynamically capturing complex associations between different modalities, and achieving efficient fusion through adaptive weight allocation. Hence, employ Cross Attention align temporal embeddings with the semantic space, thereby obtaining Joint Space (JS) C E IR BxXxC This can be formulated as:
C = CrossAAttnS, H).
Anomaly Pattern Modeling. In time series analysis, the anomaly noise non-stationarity infor- mation lead to inaccurate model parameter estimation and increased prediction bias. We employ a Anomaly Modeling Variational Autoencoder (AM-VAE) (Cemgil et al. 2020) to reconstruct the mean and variance of anomaly noise in temporal data, as described in Algorithm |
joint space, comprehensively capturing the underlying stochastic process
The reconstruction process elimi- nates the reconstruction loss function, enabling the model allow the latent variable distribution to dynamically adapt over time, thereby capturing temporal pattern.
Algorithm Anomaly Noise Modeling via AM-VAE Module
Require:
Cross-attention output: Joint Space C E R BBNxC
Ensure:
Decomposed anomaly semantic DC and de-anomalized semantic DA with temporal pat- tern
1: Latent Feature Projection: V + a (Fe where Fe: Linear mapping from RC + R"
2: Latent Distribution Estimation: μe, log ṭ² Fu,( (V), Fo.(), where Fa: Linear map-
ping from R" - I""
3: Reparameterization Trick: Sample from latent space, c (0.5 log σ²) +
и + O E (0,
4: Anomaly Sample: In time series forecasting, the true observations are modeled as: b f(az) n, where f(a:) is a prediction function based historical data, and is random noise term. We assume that future values follow a certain distribution, for example: b. ~ (44, a?). The prediction not single point estimate, but a probability distribution. To generate possible future trajectories, sampling from this distribution required: b, N(0,1). 1). The statistical characteristics of the noise may vary over time, which corresponds precisely to the per- timetep application of the Reparameterization Trick.
+ ot, where e ~
5: Anomaly Semantic Modeling: Decode latent variable to recover anomaly semantic, DC Fa(z). Fa IR - C
6: Anomaly Decomposition: DA C DC, DC BxNcC DA E BBNXC where DC represents the latent anomaly noise semantic in joint space, while DA represents the joint space where anomalous semantic are removed.
Temporal Pattern Infusion. The cross-correlation matrix M is computed via standardization over both temporal and semantic features, which can be formulated as:
H
PH
ST
JsS.
M
(4)
OH
ds
Followed by a top-K (Liu et al., 2024a) filtering mechanism to select the most strongly correlated positions and weighted to the output of AM-VAE to endow both anomaly and de-anomaly semantics with a temporal features that exhibits strong correlations. It can be formulated as:
DA DA Σ P(S,, mop.)) DC = DC Σ P(S,, mtop:):
where DC represents enhanced anomaly semantics reconstructed by the AM-VAE module from the joint space, DA represents enhanced de-anomaly semantics, P represents the matrix in the semantic space corresponding to the indices of the top-k positions in the cross-correlation matrix.
5

---

Published as a conference paper at ICLR 2026
Channel Dependency Enhancement. The aforementioned enhanced semantics are learned by in- tegrating the temporal features TS. To prevent the loss of feature information, we further acquire channel dependencies based the enhanced semanties modeled along the temporal dimension. Specifically, concatenate the TS Embeddings and the DA, DC, that share information across channel dimensions. We employ a Multi-Layer Perceptron (MLP) for attention extraction, perform channel enhancement the enhanced semantics in channel dimension, which can be formulated
Attn, (HP DAJ),
where DC has same processed get Att and will mainly focus on introducing DA.
The Attn incorporates region weights that model the TS Embeddings from both channel and se- quence dimensions, making it sensitive temporal patterns. We employ gating mechanism to integrate the semantic space and the TS Embeddings, thereby endowing the joint space with tempo- ral patterns, can be formulated as:
GA Film (Attn, H + Attn,)) @ DA),
where, Ful is a linear layer that maps the enhanced semantics to the token space of the LLM. The resulting embeddings are enriched with semantic information that reflects temporal patterns. Fi- nally, the de de-anomalization (GA E R BxNCC) and anomaly (GC R #xXXx)) representations mapped through a linear layer for alignment and fused to generate a unified semantic representation, which is more interpretable and suitable for LLMs analysis. It can be formulated as:
Y LLM(GA GC).
This represents the fusion of the enhanced de-anomalized (GA) and anomalous (GC) semantics enriched with temporal patterns. The resulting unified representation then processed through the L.M to analyze these enhanced semantics and generate more accurate prediction.
Figure 4: The architecture of Time-Adapter, a modular enhancement designed to improve the temporal modeling capabilities of LLMs. It comprises two linear layers and two parallel LSTM modules, enabling effective capture of both long-tern term and short-term temporal depen- dencies.
Pretrain = RM
V
Linear
LSTM
LSTM
Linear
Mg ege it"
Agj
3.3
Time-Adapter explicitly captures long-term and short-term dependencies, compensating for the Transformer's 's inherent weaknesses in temporal pattern extraction, as shown in Fig. 4 Specifically, building upon LoRA (Hu et al.) 2022), we replace the low-rank matrix with dual linear layers and introduce two parallel paths incorporating LSTM (Hochreiter & Schmidhuber 1997) units. These paths process temporal dependencies by leveraging long-range information from the Transformer's architecture through nonlinear transformations, thereby equipping the model with temporal data- handling capabilities during training. The Time-Adapter module explicitly captures long-term and short-term dependencies by processing the temporal patterns through parallel LSTM paths. These dependencies are integrated into the key and value matrices of the multi- attention mechanism. This enhancement enables the model to handle complex temporal patterns more effectively.
Our method consists of three key operations executed sequentially. First, a Low-rank Projection applied, where linear layer learns low-rank matrix to reduce the input dimensionality. Then the Long-term Dependency Modeling, in which an LSTM (Zhang et al., (022) maps the com- pressed features to high-dimensional space, effectively capturing long-term temporal patterns. Finally, Short-term Dependency Modeling is performed, where a second LSTM processes the high-dimensiona representation via a reverse projection (high-to-low mapping), isolating localized short-tene teort-term dynamics. Finally, linear layer integrates these refined temporal dependencies into the key (k E R ffx NxC) and value (v e R B xC) matrices the self- attentttontonn mechanism, enhancing their capacity to model temporal relationships.

---

Published as a conference paper at ICLR 2026
4
EXPERIMENTS
The evaluation metrics, datasets, and baseline methods are provided in the Appendix B.1
4.1 COMPARATIVE RESULTS
Long- Term Forecasting.
Long-term forecasting faces challenges such as trend drift, structural changes, periodic fluctuations, and noise accumulation. These difficulties make it important yet demanding task in time-series forecasting. The long-term forecasting results are provided in [able[] SE-LLM achieves superior accuracy on ETThI, Traffic, ECL, and Solar. On the Traffic dataset, which contains highly nonlinear patterns with strong variability, our approach obtains 4.4% reduc- tion in MSE compared with the best baseline. On ECL. SE-LLM also reaches the best performance and shows strong stability under periodic consumption patterns. For ETTh1 and Solar, where sea- sonal and trend components are pronounced, our method maintains competitive accuracy across all horizons. Some results are missing because several previous methods do not provide accessible code or cannot be applied to specific datasets due to design constraints. All reported comparisons follow identical training and evaluation settings. Overall, the results indicate that SE-LLM reduces Long-horizon degradation and preserves high accuracy across diverse data distributions, confirming its effectiveness in long-term forecasting.
Table 1: The input length of the SE-LLM 672. The forecast horizon is (96, 192, 336, 720). The best results are in red and the second best are blue. Full results are provided in Table
TimeMruut+ X015
TimMM6
Time-CMA 2025
TQNer 2025
L.M-TS 2025
AmsoTmmm--PP 3024
Time-1.1
iTrandFemmer
Models
SE-ILM
2074
Mecics
MSE
MAE
MAE
MSB
MAB
MSE
MAE
MAB
MAE
MSE
MAE
MSE
MAE
MSE
MAE
ETTh
0.281
0.415
0.429
0.443
0412
0.426
0.396
0.19
0,438
0.450
0.454
0.451
0.397
0.425
0.409
0.432
0.421
0.445
Weather
0.229
0.267
0.228
0.266
0.256
0.289
0.250
0.276
0.227
0,268
0.285
0,212
0.281
0.227
0.266
0.256
0.291
0.386
0.361
0.11
0.278
0.406
0.276
0-402
0.204
0.304
0.274
BCL
0.161
0.255
0.174
0.369
0.266
0.173
0.208
0.370
0.258
Sola
0.192
0.342
0.256
0.227
0.276
0,230
0.270
0.207
0.346
0.234
0.291
Short-Term Forecasting. refers to predicting target variables within a relatively brief future time horizon. We have meticulously selected algorithms suitable for short-term forecasting for compar- ison. SE-LLM was compared with the SOTA models in short-term forecasting the M4 dataset, as shown in Table 2 and the results demonstrated that we achieved the best performance. Our method achieved 0.26% reduction in Symmetric Mean Absolute Percentage Error (SMAPE), 0.94% decline in Mean Absolute Scaled Error (MASE), and a 0.47% decrease in Weiihted Absolute Error (QWA) relative the second best. This indicates that SE-LLM has the ability to capture short-term time-varying patterns through enhanced short-term dependencies.
Table 2: The M4 dataset includes yearly, quarterly, monthly, weekly, daily, and hourly data, the short-term forecasts averaged across all M4 subsets. Full results are provided in Table TO
TimeMixert+ 2025
Time-VLM 2025
AutoTimes 2024
S2IP-LLM 2024
Time-LMM 2024
iTunsformer
Models
SE-LIM
2025
2024
2024
SMAPE
11.778
11.884
11.828
11.894
11.831
12.021
11.983
11.991
12,142
Average
MASE
1.597
1.580
1.592
1.585
1.612
OWA
0.847
0.856
0.850
0.855
0.850
0.857
0.859
0.861
0.874
Zero-Shot forecasting. The TSCC module utilizes the AM-VAE to construct simulated anomaly data, facilitating the learning of latent distribution characteristics in time series data. This enables the model to better generalize to unseen temporal patterns, making it well-suited for zero-shot forecast- ing. The ability to generalize across different datasets is critical for this task, as the model must adapt to varying data distributions and temporal structures without prior training on those specific patterns. We conducted zero-shot prediction experiments the univariate M3 and M4 datasets, which exhibit various temporal variation patterns and different data distributions. Unlike many existing methods that struggle with such generalization, our model leverages the latent space learned by AM- VAE to capture underlying temporal structures that facilitate transferability across domains. Specifically, the M3 and M4 datasets present distinct temporal patterns (e.g., from quarterly to monthly or daily
7

---

Published as a conference paper at ICLR 2026
Table 3: The experimental results are based on the generalization of consistent temporal patterns. For non-matching patterns, we evaluate frequency transfers from Monthly to Weekly/Dail/Houu in the M3 >> M4 dataset, and from Quarterly other frequencies in the M4 - M3 dataset. Full result are shown in Table 11
Method
SE-LLM
AutoTimes
FPT
Dlincar
PatchTST
TimesNet
FEDformer
Informer
Reformer
sM4]
13.024
13,036
13.125
15.337
13.228
14.553
15.047
19.047
14,092
M4 +M3
12.560
12.750
13.060
14.030
13,390
14.170
13,530
15.820
13,370
frequencies), and our approach benefits from the generalization capability afforded by the AM-VAE to simulate realistic temporal patterns, even in the absence of direct training data for those patterns.
In the zero-shot experiments, we compared our approach (SE-LLM) with several state-of-the-ar arrt methods (see T and found that SE-LLM outperformed others in zero-shot generalization. No- tably, when transferring from M3 to M4, the SMAPE was reduced by 0.1% compared to Auto and by 1.4% when transferring from M4 to M3. These results underscore the model's robustness in handling unseen data and diverse temporal patterns. Additionally, we examine domain gener- alization between the M3 and M4 datasets, where the temporal frequencies shift from monthly to weekly, daily, or hourly in the M3 - M4 transfer, and from quarterly to other frequencies in the M4 - M3 transfer. This domain adaptation capability, driven by the model's s ability to learn shared latent structure across datasets, plays key role in improving performance in zero-shot scenarios. The results further verify the model's ability to generalize beyond the patterns seen during training.
4.2 ABLATION STUDY
Ablation on Long-Term Forecasting, Ablation experiments are conducted the ECL and Traffic datasets. To reduce computational costs, we selected different lightweight L.MSs for comparison, including GPT2 (Radford al. 2019). BERT (Devlin et al. 2019), Opt-125m (Zhang et al. 2022)) and Qwen2.5-0.5B (ang et al., 2024a), which are commonly used in current research (Zhou et al.,
2024). The experimental results are shown in Table Our baseline uses a cross domain attention mechanism for multi- modal alignment (Chang et al. 2025 Jin et al. 2024 Liu et al. 2025). After replacing with the TSCC framework and adding the Time- Adapter, the prediction accuracy of different LLMs has improved, demonstrating the effectiveness of the innovative methods in this paper. Comparative experiments show Qwen-0.5B yields lower prediction errors among LLMs. The ablation study on short-term forecasting are provided in the
2023
Liu et al. 2024c| 2025 Jin al.
Appendix B.3 and Table 1
Table 4: In the ablation experiments, we set the input length to 672 and the output length to 96, with forecast lengths from the set (96, 192, 336, 720}. Full results are provided in Table 12
IIM
GPT2
Ren
25m
Qwan2.5-0.18 5-0.5
Duteset
ECL
Traffcc
Traffic
BCL
Traffic
BCL
Traffic
Metrics
MSE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MSE
MAE
Hascline
0.179
0.270
0.422
0.284
0.184
0.276
0.472
0.321
0.263
0,405
0.279
+TSCC
0.172
0,280
0.177
0.431
0.294
0.170
+Time Adapter
0,166
0.279
0.167
0.360
0.412
0.282
0.163
0.257
0.161
0.386
Ablation on TSCC Module. The TSCC module features a sophisticated architecture. To better understand the contribution of each component, we perform a systematic ablation study, as summa- rized in Table 5 Specifically, we investigate the impact of removing or modifying key elements: (1) The AM-VAE module is removed to assess how the absence of simulated noisy data affects the modeling of temporal patterns.
(2) The cross-domain attention mechanism is replaced with a simple linear layer for feature concatenation, thereby altering the approach to dynamic alignment and fu- sion across domains. (3) The gated attention mechanism is eliminated to examine the importance of capturing channel-win wise dependencies. (4) Finally, the semantic space is substituted with a learnable parameter matrix, aiming to implicitly approximate the large language model's embedding with- out explicit semantic guidance. Experimental results show that the TSCC module provides reliable
8

---

Published as a conference paper at ICLR 2026
imput to the large language model by effectively mining temporal patterns, injecting them into the semantic space, and employing an implicit prompting strategy.
Table 5: Ablation study results the ETThl dataset, where checkmarks indicate the modules employed, showing the results for different prediction horizons. All the strategies together yield the best results marked in red.
Module Ablution
192
336
720
Avg
AM-YAE
Atos
Gaaed_Fuusion
Semantic Space
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAB
MSE
MAE
0.355
0.396
0.390
0.418
0.408
0.431
0.419
0.448
0.393
0.423
0.358
0.397
0.419
0.10
0.423
0.450
0.396
0.425
0.366
0.405
0.398
0.427
0412
0.438
0.421
0.456
0.399
0.432
4
0.364
0.401
0.397
0.426
0.414
0.439
0.438
0.458
0.402
0.431
0.352
0.393
0.411
0.391
0.420
0.402
0.437
0.38
0.415
Qualitative Analysis on TSCC Module. In Fig. 5 (a) we extract segment from the ETTh dataset and apply STL decomposition to obtain the trend, seasonal, and residual components. The local de- viations and abrupt fluctuations the residual component represent the anomalies that our AM- V-E is designed to capture. (b) STL decomposition reveals residual anomalies that are difficult to detect using traditional methods. The abrupt changes (marked by the red bar) are anomalous and challeng- ing to explain, but our AM-VAE module effectively approximates these segments. The anomaly information is then analyzed using LLM's generalization ability to uncover latent patterns in the semantic space. (c) The heatmap distribution of the noise semantics is shown. (d) demonstrates significant overlap between some channels and the anomalous segments the residuals, indicating effective channel relationship mining for anomaly patterns. In (e-f), the t-SNE analysis shows that the actual anomaly distribution and noise semantics form similar clusters. Seven tightly clustered points in the t-SNE projection reflect the alignment between the noise semantics and the identified anomalies, suggesting that the noise semantics closely resemble real anomalies, thus revealing la- tent patterns in the time series. (g). we show how noise semantics are regularized along the time dimension and compared with anomaly patterns. (h) The linear distribution in the correlation map confirms the close relationship between the two, indicating that the model captures the statistical properties of the anomaly patterns.
Segments
horma Lood Score
200
450
1036
2000
1000
1000
[bi STL Beedual
Soooooa go0g
Dompatioon
3.75-
230-
240
Channel
Rusidual
Figure 5: Qualitative analysis illustrating how the AM-VAF. effectively models anomaly patterns.
Ablation on Adapters. Long-term forecasting should capture long-term dependencies, making the task more challenging. Therefore, to demonstrate the performance of Time-Adapter, we conducted comparison with the LoRA (Hu et al. 2022 Liu al., 2024c), and the results are presented in Tabll 6 The experimental baseline is the Qwen2.5 with the TSCC module. Experiments findings clearly demonstrate that LoRA lacks generalization capability to enhance forecasting performance. In contrast, Time-Adapter endows LLM with the ability to adapt forecasting tasks.
Ablation on LLM-based SOTA methods. The modality gap between time-series and language data remains a challenging issue. To address this, we integrate TSCC and Time-Adapter as plug-in into Time-LLM (Jin et al., 2024). AutoTimes (Liu et al. 2024c), and TimeCMA (Liu et al. 2025), evaluating their impact on the ETTh and ETTm datasets using the original code's 's default parameters. All other settings remain unchanged, with GPT-2 consistently serving as the LLM backbone. Results in show consistent error reductions, with TimeCMA achieving notable drop in MSE Since

---

Published as a conference paper at ICLR 2026
Table 6: All reported results are averaged. Full results are provided in Table 14
Weather
Solar
Metrics
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
Bascline-TSC
0.381
0.415
0.242
0.287
0.389
0.264
0.166
0.262
0.201
0.251
LoRRA
0.425
0.390
Time-AAapter
0.389
0.420
0.229
0.267
0.386
0.261
0.161
0.255
0.192
0.243
TimeCMA uses LLMs only to generate outputs from textual time-series prompts without training the LLM, ablation studies Time-Adapter not feasible. These results demonstrate that our approach addresses a common limitation across existing frameworks.
Table The forecasting horizons are in {96, 192, 336, 720), and all experimental results presented averages. Full results provided in Table 15
Time-IIMM
-TSCC:
STime-Adapter
AutoTimes
+TSCC
-Time-Adipter
Time-CMA
+TSCX:
MAE
MAE
MSE
MAE
ETT22
0.370
0.444
0.368
0.401
0.364
0.399
0.367
0.410
0.363
0.406
0.357
0.407
0.416
0.429
0.379
0.403
ET1ml
0.358
0.388
0.360
0.390
0.357
0.389
0.361
0.389
0.354
0.384
0.351
0.382
0.399
0.413
0.383
0.396
ETTm2
0.262
0.324
0.267
0.328
0.266
0.325
0.274
0.329
0.259
0.325
0.268
0.323
0.294
0.337
0.285
0.347
4.3
EFFICIENCY ANALYSIS
We evaluate the computational efficiency of our approach. All experiments were conducted with batch size of 256 and sequence length of 672, under which the training and inference times for each method were recorded under the same hardware and software environment. The bar chart Fig. 6 compares the training and inference speeds of SE-LLM with classical algorithms,, all evaluated on the same dataset, highlighting the superior efficiency our method. addition, the bubble chart illustrates the relationship between training speed and MSE variations the ECL dataset when our approach is applied with different I.MMs, further demonstrating the trade-off's between computational speed and prediction accuracy.
Traning Gafter)
Inforence lafter
Flecvichy Datavel
Opi-125M
0.1725
SE:1MM
SE-UMM AutoTimes
AutoTimas
Bort
0.1700
1LM
Time LLVI
Mean Squaree Enner
GPT2
Qwen2
0.1925
6.7900
Owen2 5-1.50
0.24
DeegSeek-1100 1.58
0.1530
Traking Time (illee)
Figure 6: The size of the circles reflects the proportional relationship among the parameter counts of different LLMs.
5 CONCLUSION
This paper introduces SE-LLM, which addresses the inherent modality gap between LLMs and time series data. We explore the temporal and channel dependencies between temporal embeddings and the semantic space of LLMs. Token embeddings are enhanced with temporal-patter formation, improving interpretability and unlocking LLMs potential. Moreover, we incorporate plugin module tailored for time-series forecasting into LLMs, which activates their ability to model temporal dependencies. Experimental results demonstrate that SE-LLM surpasses SOTA perfor- mance. Furthermore, embedding TSCC and performance improvements, shedding light on common challenges leveraging LLMs for time- series forecasting. SE-LLM efficiently integrates temporal information and language models.
semantic in-
into existing frameworks also leads to
10

---

Published as a conference paper at ICLR 2026
6 ACKNOWLEDGEMENTS
This research is supported by National Science and Technology Major Project (2022DD0119002 National Science Fund for Distinguished Young Scholars (62125601), National Natural Science Foundation of China (62576031).
ETHICS STATEMENT
This work focuses on advancing the methodology time series forecasting by enhancing large lan- guage models (LLMs) with semantic structures derived from temporal data patterns. Our approach does not involve human subjects, personal data, or sensitive information, and all experiments are conducted on publicly available time series datasets. The proposed SE-LLM framework is designed to improve model interpretability and computational efficiency without altering the core parameters of the pre-trained LLM.. We do not foresee direct societal harms from this research.
REPRODUCIBILITY STATEMENT
Our code is available, and the specific parameter configurations provided within the code. All datasets used in the experiments are publicly available, and all compared baseline algorithms are from open-source projects. Our experimental results reproducible.
This paper did not conduct ablation experiments using larger language models (e.g.. Llama-7B above) primarily because such large models incur significantly higher resource consumption compared to traditional time series forecasting methods, which goes against the requirement for lightweight applications. Although we propose large language model-based algorithm for time series forecasting, we opted for smaller-scale model to balance efficiency and practicality. Fur- thermore, we have already performed ablation analysis on large-parameter language models in the sections on Table 13 and Efficiency Analysis 4.3
The discussion section of this paper Appendix C and the limitations section in Appendix
The usage of large language models is discussed in Appendix
REFERENCES
Serdar Arslan. A hybrid forecasting model using Istm and prophet for energy consumption with decomposition of time series data. PeerJ Computer Science, 8:1001, 2022.
Defu Cao, Yujing Wang, Juanyong Duan, Ce Zhang, Xia Zhu, Congrui Huang, Yunhai Tong, Bix- iong Xu, Jing Bai, Jie Tong, and Qi Zhang. Spectral temporal graph neural network for multivari- time-series forecasting. In Advances Neural Information Processing Systems, 2020.
A.
Taylan Cemgil, Sumedh Ghaisas, Krishnamurthy Dvijotham, Sven Gowal, and Pushmeet Kohli. The autoencoding variational autoencoder. In Advances in Neural Information Processing Sys- tems, 2020.
Ching Chang, Wei-Yao Wang, Wen-Chih Peng, and Tien-Fu Chen. LLM4TS: aligning pre-trained llms as data-efficient time-series forecasters. ACM Trans. Intell. Syst. Technol, (6(3):60:1-60:20 2025.
Can Chen, Gabriel L. Oliveira, Hossein Sharifi-Noghabi and Tristan Sylvain. LLM-TS integrator: Integrating LLM for enhanced time series modeling. Trans. Mach. Learn. Res., 2025, 2025.
Minghao Chen, Houwen Peng, Jianlong Fu, and Haibin Ling. Autoformer: Searching transformers for visual recognition. In International Conference on Computer Vision, pp. 12250-12260, 2021.
Shaoxiang Chen, Zequn Jie, and Lin Ma. Llava-mole: Sparse mixture of lora experts for mitigating data conflicts in instruction finetuning milms. CORR, Abs/2400.11616) 2024.
DeepSeek-AL. Deepseek-vi v2: A strong, economical, and efficient mixture-of-expprtt language
model. CoRR, 0al/2405.04443 2024.
11

---

Published as a conference paper at ICLR 2026
Songgaojun Deng, Shusen Wang. Huzefa Rangwala, Lijing Wang, and Yue Ning. Cola-gnn: Cross- location attention based graph neural networks for long-term ILI prediction. In International Conference on Information and Knowledge Management, pp. 245-254, 2020.
Jacob Devlin, Ming- Wei Chang, Kenton Lee, and Kristina Toutanova. BERT: pre-training of deep bidirectional transformers for language understanding. In North American Chapter of the Associ- ation.fo Computational Linguistics: Human Language Technologies, pp. 4171-4186, 2019.
Zhiyu Fang, Xiaobin Zhu, Chun Yang, Hongyang Zhou, Jingyan Qin, and Xu-Cheng Yin. Aligning enhanced feature representation for generalized zero-shot learning. Science China Information Sciences, 68(2):122108 2025.
Nate Gruver, Mare Finzi, Shikai Qiu, and Andrew Gordon Wilson Large language models are zero-shot time series forecasters. In Advances in Neural Information Processing Systems, 2023.
Han Guo, Philip Greengard, Eric P. Xing, and Yoon Kim. Lq-lora: Low-rank plus quantized matrix decomposition for efficient language model finetuning. In International Conference on Learning Representations, 2024.
Divij Gupta, Anubhav Bhatti, Suraj Parmar, Chen Dan, Yuwei Liu, Bingjie Shen, and San Lee. Low-rank adaptation of time series foundational models for out-of-domain modality forecasting. In International Conference on Multimodal Interaction, pp. 382-386, 2024.
Sepp Hochreiter and Jürgen Schmidhuber. Long short-term memory. Neural computation, 9:1735- 1780, 1997.
Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Ahu, Yuanzhi Li, Shean Wang. Lu Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models. In International Con- ference on Learning Representations, 2022.
Yuxiao Hu, Qian Li, Dongxiao Zhang, Jinyue Yan, and Yuntian Chen. Context-alignment Acti- vating and enhancing Ilms capabilities in time series. In International Conference on Learning Representations, 2025.
Furong Jia, Kevin Wang, Yixiang Zheng, Defu Cao, and Yan Liu. GPT4MTS: prompt-based large language model for multimodal time-series forecasting. In AAAI Conference on Artificial Intelli- gence, pp. 23343-23351, 2024a.
Yuxin Jia, Youfang Lin, Jing Yu, Shuo Wang, Tianhao Liu, and Huaiyu Wan. PGN: the rnn's new successor is effective for long-range time series forecasting. In Advances in Neural Information Processing Systems, 2024b.
Ming Jin, Shiyu Wang. Lintao Ma. Zhixuan Chu, James Y. Zhang, Xiaoming Shi, Pin-Yu Chen, Yuxuan Liang, Yuan-Fang Li, Shirui Pan, and Qingsong Wen. Time-llm: Time series forecasting by reprogramming large language models. In International Conference on Learning Representa- fions, 2024.
Rohaifa Khaldi, Abdellatif El Afia, Raddouanne Chiheb, and Siham Tabik. What is the best rnn-cell structure to forecast each time series behavior? Expert Systems with Applications, 215:119140 2023.
Nikita Kitaev, Lukasz Kaiser, and Anselm Levskaya. Reformer: The efficient transformer. In International Conference on Learning Representations, 2020.
Yaxuan Kong, Zepu Wang. Yuqi Nie, Tian Zhou, Stefan Zohren, Yuxuan Liang, Peng Sun, and Qing- song Wen. Unlocking the power of LSTM for long term time series forecasting. In Association for the Advancement of Artificial Intelligence, pp. 11968-11976. 2025.
Guokun Lai, Wei-Cheng Chang, Yiming Yang, and Hanxiao Liu. Modeling long- and short-term temporal patterns with deep neural networks. In SIGIR Conference on Research & Development in Information Retrieval, pp. 95-104, 2018.
Brian Lester, Rami Al-Rfou, and Noah Constant. The power of scale for parameter-efficient prompt tuning. In Empirical Methody in Natural Language Processing, pp. 3045-3059, 2021.
12

---

Published as a conference paper at ICLR 2026
Shengsheng Lin, Weiwei Lin, Xinyi Hu, Wentai Wu, Ruichao Mo, and Haocheng Zhong. Cyclenet: Enhancing time series forecasting through modeling periodic patterns. In Advances in Neural Information Processing Systems, 2024.
Shengsheng Lin, Haojun Chen, Haijie Wu, Chunyun Qiu, and Weiwei Lin. Temporal query net- work for efficient multivariate time series forecasting. In International Conference Machine Learning, volume 267, 2025.
Chenghao Liu, Steven C. H. Hoi, Peilin Zhao, and Jianling Sun. Online ARIMA algorithms for time series prediction. In AAAI Conference on Artificial Intelligence, pp. 1867-1873, 2016.
Chenxi Liu, Qianxiong Xu, Hao Miao, Sun Yang, Lingzheng Zhang, Cheng Long, Ziyue Li, and Rui Zhao. Timecma: Towards llm-eeppwered multivariate time series forecasting via cross moddllty alignment. In Association for the Advancement of Artificial Intelligence, pp. 18780-18788, 2025.
Shilong Liu, Zhaoyang Zeng, Tianhe Ren, Feng Li, Hao Zhang, Jie Yang. Qing Jiang, Chunyuan Li, Jianwei Yang, Hang Su, Jun Zhu, and Lei Zhang. Grounding dino: Marrying dino with grounded pre-training for open-set object detection. In European Conference on Computer Vision, volume 15105, pp. 38-55, 2024a.
Yong Liu, Chenyu Li, Jianmin Wang, and Mingsheng Long. Koopa: Learning non-stationary time series dynamics with koopman predictors. In Advances in Neural Information Processing Sys- tems, 2023.
Yong Liu, Tengge Hu, Haoran Zhang. Haixu Wu, Shiyu Wang, Lintao Ma, and Mingsheng Long. itransformer: Inverted transformers are effective for time series forecasting. In International Conference Learning Representations, 2024b.
Yong Liu, Guo Qin, Xiangdong Huang, Jianmin Wang, and Mingsheng Long. Autotimes: Autore- gressive time series forecasters via large language models. In Advances in Neural Information Processing Systems, 2024c.
Shusen Ma, Tianhao Zhang, Yun-Bo Zhao, Yu Kang, and Peng Bai. Teln: A transformer-based conv-lstm network for multivariate time series forecasting. Applied Intelligence, 53(23):28401- 28417, 2023.
Spyros Makridakis, Evangelos Spiliotis, and Vassilios Assimakopoulos. The m4 competition: 100,000 time series and 61 forecasting methods. International Journal of Forecasting, 36(1): 54-74, 2020.
Jie Mu, Wei Wang, Wenqi Liu, Tiantian Yan, and Guanglu Wang. Multimodal large language model with lora fine-tuning for multimodal sentiment analysis. ACM Trans. Intell. Syst. Technol., 16(6):
139:1-13: 2025.
Yuqi Nie, Nam H. Nguyen, Phanwadee Sinthong, and et al. A time series is worth 64 words: Long- term forecasting with transformers. In International Conference Learning Representations, 2023.
Zijie Pan, Yushan Jiang, Sahil Garg, Anderson Schneider, Yuriy Nevmyvaka, and Dongjin Song. S2IP-LLM: semantic space informed prompt learning with LLM for time series forecasting. In International Conference Machine Learning, 2024a.
Zijie Pan, Yushan Jiang, Dongjin Song, Sahil Garg, Kashif Rasul, Anderson Schneider, and Yuriy Nevmyvaka. Structural knowledge informed continual multivariate time series forecasting. CoRR, a22/2402.127222, 2024b.
Yao Qin, Dongjin Song, Haifeng Chen, Wei Cheng, Guofei Jiang, and Garrison W. Cottrell. A dual- stage attention- based recurrent neural network for time series prediction. In International Joint Conference on Artificial Intelligence, pp. 2627-2633, 2017.
Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, llya Sutskever, et al. Language models are unsupervised multitask learners. OpenAl blog, 188)99 2019.
13

---

Published as a conference paper at ICLR 2026
Zaheer Shahzad, Anjum Nadeem, Hussain Saddam, Abeer D Algarni, and Iqbal Jawaid. A multi parameter forecasting for stock time series data using Istm and deep learning model. Mathematics, 11(3), 2023.
Zongjiang Shang. Ling Chen, Binqing Wu, and Dongliang Cui. Ada-mshyper: Adaptive multi-scale hypergraph transformer for time series forecasting. Advances in Neural Information Processing Systems, 2024.
Alex Sherstinsky. Fundamentals recurrent neural network (RNN) and long short-term memory (LSTM) network. CoRR, 110118088333111 2018.
Xiaoming Shi, Shiyu Wang. Yuqi Nie, Dianqi Li, Zhou Ye, Qingsong Wen, and Ming Jin. Time-moe: Billion-scale time series foundation models with mixture of experts. In International Conference on Learning Representations, 2025.
Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, Aurélien Rodriguez, Ar- mand Joulin, Edouard Grave, and Guillaume Lample. Llama: Open and efficient foundation language models. CoRR, abs/2302. 13971, 2023.
Shiyu Wang, Jiawei Li, Xiaoming Shi, Zhou Ye, Baichuan Mo, Wenze Lin, Shengtong Ju, Zhixuan Chu, and Ming Jin. Timemixer++: general time series pattern machine for universal predictive analysis. In International Conference on Learning Representations, 2025.
Yuxuan Wang, Haixu Wu, Jiaxiang Dong, Guo Qin, Haoran Zhang, Yong Liu, Yunzhong Qiu, Jian- min Wang, and Mingsheng Long. Timexer: Empowering transformers for time series forecasting with exogenous variables. Advances in Neural Information Processing Systems, 2024.
Haixu Wu, Tengge Hu, Yong Liu, Hang Zhou, Jianmin Wang, and Mingsheng Long. Timesnet: Temporal 2d-variation modeling for general time series analysis. In International Conference on Learning Representations, 2023.
Hao Xue and Flora D Salim. Promptcast: A new prompt-based learning paradigm for time series forecasting, IEEE Transactions on Knowledge and Data Engineering, (66(1))68511 1-6644 2023.
An Yang. Beichen Zhang, Binyuan Hui, Bofei Gao, Bowen Yu, Chengpeng Li. Dayiheng Liu, Jianhong Tu, Jingren Zhou, Junyang Lin, Keming Lu, Mingfeng Xue, Runji Lin, Tianyu Liu, Xingzhang Ren, and Zhenru Zhang. Qwen2.5-math technical report: Toward mathematical pert model via self-immree improvemreett CoRR. abs/2409.12122. 2024a.
Runze Yang, Longbing Cao, Jie Yang, and et al. Rethinking fourier transform from basis functions perspective for long-term time series forecasting. In Advances in Neural Information Processing Systems, 2024b.
Kyongmin Yeo, Igor Melnyk, Nam Nguyen, and Eun Kyung Lee. DE-RNN: forecasting the proba- bility density function of nonlinear time series. In International Conference on Data Mining, pp. 697-706, 2018
Ailing Zeng, Muxi Chen, Lei Zhang, and Qiang Xu. Are transformers effective for time series forecasting? In AAAI Conference on Artificial Intelligence, pp. 11121-11128, 2023.
Juyuan Zhang, Wei Zhu, and Jiechao Gao. Adapting large language models for time series modeling via a novel parameter-efficient adaptation method. CoRR, abs/2502. 13725, 2025.
Longteng Zhang, Lin Zhang, Shaohuai Shi, Xiaowen Chu, and Bo Li. Lora-fa: Memory-efficient low-rank adaptation for large language models fine-tuning. CoRR, 0t//2308.03303. 2023.
Susan Zhang, Stephen Roller, Naman Goyal, Mikel Artetxe, Moya Chen, Shuohui Chen, Christo- pher Dewan, Mona T. Diab, Xian Li, Xi Victoria Lin, Todor Mihaylov, Myle Ott, Sam Shleifer, Kurt Shuster, Daniel Simig, Punit Singh Koura, Anjali Sridhar, Tianlu Wang, and Luke Zettle- moyer. OPT: open pre-trained transformer language models. CoRR, abs/2205.01063 2022.
14

---

Published as a conference paper at ICLR 2026
Yunhao Zhang and Junchi Yan. Crossformer: Transformer utilizing cross-dimension dependency for multivariate time series forecasting. In International Conference on Learning Representations, 2023.
Siru Zhong, Weilin Ruan, Ming Jin, Huan Li, Qingsong Wen, and Yuxuan Liang. Time-vlm: Explor- ing multimodal vision-language models for augmented time series forecasting. In International Conference on Machine Learning, 2025.
Haoyi Zhou, Shanghang Zhang. Jieqi Peng, Shuai Zhang, Jianxin Li, Hui Xiong, and Wancai Zhang. Informer: Beyond efficient transformer for long sequence time-series forecasting. In AAAI Con- ference on Artificial Intelligence, pp. 11 1110--1115, 2021.
Tian Zhou, Peisong Niu, Xue Wang, Liang Sun, and Rong Jin. One fits all: Power general time series analysis by pretrained LM. In Advances in Neural Information Processing Systems, 2023.
A APPENDIX FOR SE-LLM
A.1 IMPLEMENTATION DETAILS OF THE TSCC PIPELINE
The pipeline consists of several interdependent components such as cross-attention, AM-VAE, top-k correlation filtering, and gating fusion. Below, we provide detailed pseudocode for each component to improve readability and explain their roles within the pipeline.
Cross-Modality Alignment. The first step to align temporal embeddings with the semantic space using cross-attention mechanism, where semantic queries attend to temporal keys and values.
Algorithm 2 Cross-Modality Alignment
1: Input: Temporal sequence time_seq. Semantic embeddings word.
2: Output: Aligned temporal features aligned _time.
3: Compute query, key, and value matrices: {Project semantic tokens queries; temporal tokens as keys/values.]
4: Q Query
5: K < Key(time_seq)
6: V Value(time_seq]
7: Calculate attention scores: [Measure similarity between semantic queries and temporal keys.} }
8: attn scores
va model
9: Apply softmax to get attention weights: (Convert raw scores to normalized attention distribu- tion.)
10: hetn_weights +- softmax(atin_scoree
11: Compute aligned temporal features: (Weighted aggregation of temporal values guided by se- mantic queries.}
12:
aligned time + attn weights
Anomaly Pattern Modeling. The AM-VAE module models latent anomaly patterns by estimating the underlying distribution and disentangling anomaly versus non-anomaly semanties.
Temporal Pattern Infusion. We inject temporal correlations into semantics via cross-correlation analysis and top-k filtering.
15

---

Published as a conference paper at ICLR 2026
Algorithm 3 Anomaly Pattern Modeling via AM-VAE
1: Input:
Cross-attention output C (joint space).
2: Output: Decomposed anomaly semantic DC and de-anomalized semantic DA.
3: Project the input to latent space: {Encode joint space representation into latent variables.)
4: V < Encoder(C C)
5: Estimate latent distribution: (Predict mean and variance for variational inference.]
6: μ., log x² +- Latent Distribution(VV
7: Apply reparameterization trick: (Sampling latent code with differentiabilityy)) }
8: z t μ + c cxp(0.5 log o²)
9: where (O, I) is random noise.
10: Decode latent variable: (Generate anomaly semantics from latent representation.)
DC t Decoder(z)
12: Compute de-anomaly semantic: (Subtract anomaly component from original to obtain clean
semantics.) }
13: DA = C DC
Algorithm 4 Temporal Pattern Infuse via Top-k Filtering
1: Input: Anomaly feature DC, De-anomalized feature DA, Cross-correlation matrix M.
2: Output: Enhanced temporal and semantic features.
3: Compute cross-correlation matrix: (Normalize temporal and semantic representations and com- pute their correlation.)
4: M + H-μи SH-
08
5: Apply top-k filtering: {Select only the strongest temporal-semanee correlations.)
6: DA < DA
Σ i==
P(S,, mpp
7: DC < DC
Σn === P(S,, m(pp.)
Gated Fusion. A gate adaptively selects how much temporal vs. semantic information contributes to the fused representation.
Algorithm Gated Fusion of Temporal and Semantic Features
Input: Temporal feature time feat, Text feature text feat, Semantic embeddings word.
2: Output: Fused feature fused.
3: Compute similarity: {Match temporal features with the most relevant semantic tokens.) }
4: Sim-matrix < matmul cantmul(tina feat, word"')
5: Select top-k semantic tokens: (Keep only the most relevant semantic dimensions.}
6: topk idx + tspk(sim matrix, 32)
7: Enhance selected semantic features: {Aggregate top-k semantic cues.)
8: enhanced tcxt +- mean(sclected _extt, dim =
9: Improve semantic alignment: [Normalize and amplify discriminative semantics.)
10: enhanced text +- softmax (enhnnced text, dim 1)
Concatenate features: (Prepare inputs for gating network.)
12: combined + concat( time feat, enhanced text)
Compute gate values: (Predict adaptive weight between temporal and semantic paths.) )
14: gate <- Gate_Net( combined)
Fuse features: (Final adaptive fusion.)
16: fused +- gate time feat + (I gate) text_feat
Final Model Fusion. Combine anomaly and de-anomaly branches to produce the final enhanced representation. The final fused output is passed through linear layer or subsequent predictor for forecasting.
16

---

Published as a conference paper at ICLR 2026
Algorithm 6 Final Fusion of Anomaly and De-Anomaly Features
1: Input: Fused anomaly output, Fused de-anomaly output.
2: Output: Final fused output fused output.
3: Combine both outputs: (Residual fusion capturing full semantic spectrum.)
4: output -- fused mused.output.de + fused -oeeutuotp
A.2 AUTOREGRESSIVE FORECASTING
In time series forecasting, autoregressive models offer the advantage of making predictions over arbitrary time horizons while being trained solely on sequences of fixed length, thereby significantly reducing computational overhead. These models generate future values by iteratively conditioning on previously observed data points. This strictly sequential inference mechanism leverages the temporal continuity and intrinsic patterns present in historical observations. When these underlying patterns remain stable over time, autoregressive models typically produce reliable and consistent forecasts.
Algorithm Autoregressive Inference with Token Stride
Require: Input X BXLC stride horizon T. Assume Model (X) outputs O E BBLxXC
Ensure: Output Y RxTxC
1: N + [T/T]
2:
3: K + N N I[r # 0] {K [T/T]}
4: P - [] {store predicted chunks}
5: for <- 1 to K do
6:
if j then
7:
X + Cat (o, 1.10 Pj-1)
8:
if
9:
O MX)el(X)
10: Рj - O. L-T+1:L, flast steps}
11: append Pj to P
12: end for
13: Y -Ca((P) (B (K 7) C)
14: r then
15: Y YoT (truncate to T}
end if
17: return Y
SE-LLM adopts autoregressive inference strategy, inspired by the design of AutoTimes. Specif- ically, future values are generated by recursively feeding the model's prior outputs back into the input sequence context for subsequent predictions. This iterative process enables long-horizon forecasting while maintaining coherence with the temporal dynamics the series. The complete inference procedure is summarized in Algorithm 7
B EXPERIMENT
B. EXPERIMENTAL SETUP
Time series forecasting, quantitative evaluation is essential for assessing model performance. In this study, we adopt standard metrics tailored to different forecasting scenarios. For long-term forecasting, we Mean Squared Error (MSE) and Mean Absolute Error (MAE);
Evaluation Metrics.
N
1
MAE
Y,
Yn/,
(9)
n=1
MSE
Σ
(Yn
8n)2
(10)
N
n=1
17

---

Published as a conference paper at ICLR 2026
MAE reflects the average magnitude of forecast errors, while MSE penalizes larger deviations more heavily due to the squaring operation. For short-term and zero-shot forecasting on the M4 and M3 dataset, additionally employ the following metries:
Mean Absolute Percentage Error (MAPE):
100
Y,
Ynl
MAPE-
(II)
N
Ynl
==1
Symmetric Mean Absolute Percentage Error (SMAPE):
200 SMAPE
Y Y ml
(12)
N
Y YYal
==1
Mean Absolute Scaled Error (MASE):
Y Yul
MASE
(13)
N-m Ei=
jim
n=1
N
Overall Weighted Average (OWA):
SMAPE
MASE
OWA
(14)
SMAPFNaive?
MAS Naivee
MAPE and SMAPE provide scale-invariant percentage-basee error assessments, with SMAPE mit- igating issues related to small denominators. MASE offers a relative comparison against a naive seasonal benchmark. OWA combines MASE and SMAPE relative the Naïve2 baseline, serving as composite indicator of overall performance.
Table 8: The dataset includes detailed descriptions, with specifications such dimensionality, the distribution of total time points across training, validation, and test partitions, the target range of future time points for prediction, and the temporal sampling rate.
Tasks
Datasets
Dim
Forecast Herizon
Dataset Sizee
Frequency
Information
ETTai
195, 152, 336, 720)
18545, 2881. 2881]
Hourly
Electrivity
192,
105-20
10min
Long: Term Forecasting
192, 336, 720
112185, 1767, 3509)
Hoarty
Transiporation
ECL
321
196, 192, 236.720)
[18917, 2833, 5261]
Hourly
Eelecticity
Solar
137
196, 192, 336, 720)
(38001, 5161, 10c1r)
10min
Energy
M4. M4-Ycarly
[3300, 23000
Yeurly
Univariate
Mi-Quarteryy
22000,
Quarterly
M4-Mourhly
(80000 48000)
Monthly
Short: Temm Forecasting and Zero-Shot Forecasting
M4. Weekly
13
[350, 358)
Weekly
Univasiate
Mt Daily
14
(2227, 0, (227)
Daily
Univasiate
Mt-Bourl
48
(414.0. (14))
Hourly
Univariate
M3-Ycury
Yearly
QaaQttrteyyy
[156.
Quarterty
Zero Shot Forecasting
M3 Moathly
[1628, 0, 14288
Monthly
Univasiate
M3-OOteers
(174, 0, 174)
Others
Univasiate
Datasets. We adopted publicly available benchmarks in four research domains: long-term fore-
casting, short-term forecasting, zero-shot forecasting and in-context forecasting (Liu et al. 2024c). The experimental datasets comprised: ETTh (Zhou et al., 2021). ETTm (Zhou et al. 2021
Weather (Chen et al. 202T). Elcetricity (ECL) (Chen et al., al.] 2021). Traffic (Chen al. 2021), ), So-
lar (Lai et al. 2018). M3 (Wu et al. 2023). and M4 (Makridakis et al., 2020) datasets. The detailed introduction refers to the T Time series datasets consist of sequential data points collected or recorded at consistent time intervals (e.g.. hourly, daily, monthly). Each entry includes timestamp and one or more observed values. These datasets are used to train models that identify historical patterns to forecast future values. Common applications include predicting stock prices, energy demand, weather, sales, and website traffic.
The experiments TimeMixer+ (Wang et al. (2025) and AutoTimes (Liu et al. 2024c) as the perforerformance base- lines for evaluation. The algorithms involved in all the experiments are TimeMOR (Shi al. 2025)
Baseline.
conducted and compared with SOTA methods, such
18

---

Published as a conference paper at ICLR 2026
Table 9: The input length of the SE-LLM 672. The forecast horizon is (96, 192, 7203. The best results are in red and the second best are blue.
Tunehtoee
TineNOE
TONet
LLMITS
AusoTimes
Time
iTranfommes
Models
SI-11M
Dusnects
Mesic
MIE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAB
MSE
MAE
MSE
MAE
MSE
MEAE
MSE
MAE
96
01:532
0
0.375
0.-04
0.349
0.372
0.379
0.803
0.365
0.346
0,405
0.430
0.350
0.397
0.350
0,412
01247
0.415
0.47
400
0.442
0.457
0.433
0.434
0.450
730
0.482
0
0.505
0.510
0.505
0.424
0.450
0.430
0.459
0.647
0.475
0.381
0.415
0.429
0.44
0.412
0.436
0.438
0.450
0.395
0.49
0.454
0.451
0.397
0.425
0.409
0.432
0.421
0.445
96
01150
0.200
0.150
0.200
0.157
0.11
0.149
0.200
0.167
0.211
0.166
0.337
0.158
0.208
0.385
0.200
0.174
0.225
0.343
0.19m
0.356
0:201
0.249
0:339
0.207
0.364
0.227
0.2n8
Weather
0.348
0.277
0.262
0.350
0.374
0.369
0.229
0.267
0.228
0.256
0.256
0.289
0.227
0.268
0.250
0.276
0.257
0.255
0.242
0.251
0.227
0
0.266
0.291
96
01350
0.243
0.367
0.256
0.366
0.255
0.557
0.315
0.30
0.257
0.313
0.280
0.353
0.259
0.179
0:392
0.276
0.612
0.326
0.38%
0.668
0:300
0.388
0.83
Troffc
0.634
0.39
11275
720
0.453
0.442
0.330
ang
0.408
0.400
0.27%
0.118
0.25
0.38
0.214
95
0.130
0.225
0.132
0.225
0.131
0.225
01443
0.341
0.167
0.271
0.140
0.236
0.153
0.244
0.133
0.229
193
11:48
0.242
0.151
0.346
0.153
0.251
116
0.259
0.17%
0.260
0.159
0.255
0.162
0.71
0111
0.345
BCL
0.100
0:188
0.177
0,200
0.201
0.291
0.205
0.304
ang
0.257
0.259
0.766
0.175
0.268
0.164
0.258
95
0.165
0.220
0.172
0.235
0.197
0.244
0.202
0.223
0.179
0.321
0.234
0.259
0.183
0.265
193
18183
0.238
0.190
0.249
0:250
0.274
0216
0.301
0.186
0.236
0.244
0259
11.15
0.281
Solar
0.34
0.199
0.344
0.281
0.55
0.25
0.201
0.234
0.299
0ng
0.387
0.194
0.216
0.230
0.270
0
0.207
0.346
0.344
0.55
0.213
0.241
-Mne (Lu al.
2025). TQNet (Lin et al. 2025). Time-LLM (Jin et al.
2024 LLM-TS Chen
al. 2025), ) Transformer (Liu et al. (024b)) DLinear (Zeng et al. (023). Time- VLM
(Zhong et al.,
2025
S2IP-LLM (Pan et al. 2024a), PST (Nie et al. (023)). TimesNet (Wu et al.
2023). FED-
Former (Yang et al. 2024b), Informer (Zhou 2021). Reformer (Kitaev et al. 2020). Koopa (Liu
al.
)023), and FPT (Zhou et al. 2023). some of which serve as supplementary references.
B.2
COMPARATIVE RESULTS
Long Term Forecasting. The full results for long-term forecasting are shown in Table 9 On ETTh1, Weather, Traffic and Solar datasets, our method achieves the best performance. On Weather, our MAE ranks second to TimeMixer++ and Time-LLM. Overall, SOTA methods. We note that the source code of TimeMixer++ is not publicly available. To ensure fair comparison under an input sequence length of 672, we reproduced TimeMixer++ to the best of our ability based on the descriptions in the original paper and publicly available information. For Times, Time-LLM, and iTrrasformer, we directly adopt the results reported in AutoTimes under comparable experimental settings. For the remaining baselines, we re-ran the models with different input sequence lengths and report the best results. All comparisons were conducted under the same training and evaluation protocol.
approach outperforms existing
Short Term Forecasting. The full results for short-term series forecasting are shown in Table 10
The experimental results indicate that We achieved the best performance in Yearly and Monthly pat- terns. In the Others pattern, Time-Mixer++ achieved the lowest SMAPE, while Auto reached the lowest error in Quarterly pattern. From the lowest average error overall, SE-LLM achieve lower SMAPE, signifying the best forecasting performance.
Zero-Shot Forecasting. The full results for zero-shot forecasting are shown in Table I1 We adopt the zero-shot forecasting methodology from AutoTimes, where each experimental setup involves training a model on designated source dataset and subsequently deploying it to generate forecasts on a target dataset. These zero-shot scenarios are executed across subsets categorized by their sam- pling frequencies, each exhibiting distinct distributional properties. Overall, SE-LLM achieves the best performance.
B.3 ABLATION STUDY
Ablation on Long-Tere Term Forecasting, The full results of the innovation methods are shown in
Table 12 SE-LLM uses cross-domain attention alignment (Jin et al.) 2024 Chang al., 2025 Liu et al. 2025) as the baseline. After replacing it with TSCC, errors were reduced across dif- ferent L.LMs. To make LLMS more suitable for processing time-series data, incorporated
19

---

Published as a conference paper at ICLR 2026
Table 10: The M4 dataset includes yearly, quarterly, monthly, weekly, daily, and hourly data, the short-term forecasts averaged across all M4 subsets.
TimeMixar+
FSCA
Time-VLM 2025
AuteTTimes 2024
S2IP-LLM
Time-LLM
FPT 2004
iTranformer
Models
SE-LLM
2025
2025
2024
2024
2024
SMAPE
13.294
13.397
13:288
13.419
13.319
13,413
13.419
15,531
13,652
Yearly
MASE
2,970
2,990
2,974
3,005
3.021
3,024
3.005
3,015
3.095
0.780
0.786
0.789
0.792
0.807
SMAPE
10.079
10.206
10.037
10.110
10.020
10,352
10.110
10.177
10.153
Quarterly
MASE
1.177
1.201
1.174
1.178
1162
1.228
1.178
1.994
1.209
OWA
0.887
0.901
0.884
0.889
0.878
0.922
0.889
0.897
0.911
SMAPE
12.618
12,720
12.762
12,980
12.696
12,995
12.980
12.894
13.079
Monthly
0.931
0.943
0.947
0.970
0.974
OWA
0.875
0.884
0.897
0.903
0,880
0.910
0,903
0.897
0.911
SMAPE
4.896
4.593
4,761
4.795
4,916
8,80
4.795
4,940
4,780
Others
MASE
3,306
3,380
3,207
3.178
3,310
3.247
3.178
3,228
3,231
OWA
1.037
1,054
1,007
1.006
1,039
1,017
1.006
1,029
1,012
SMAPE
11.778
11,884
11.894
11,831
12,021
11,991
12,142
Averag
MASE
1.578
1.597
1,580
1.592
1.585
1.612
1.595
1,600
1.631
OWA
0.847
0.856
0.850
0.855
0,850
0.857
0.859
0,861
0.874
Table 11: The experimental results are based on the generalization of consistent temporal patterns. For non-matching patterns, evaluate frequency transfers from Monthly Weekly/Daily/Hourj) in the M3 + M4 dataset, and from Quarterly to other frequencies in the M4 - M3 dataset.
Method
SE:LLM
AutoTimes
FPT
Dlinear
PatchTST
TimesNet
FEDformer
Informer
Reformer
Yearly
13,706
13.708
13.740
14.193
13,966
15.655
13.887
18.542
15.662
Quarterly
10,718
10.742
10.787
18.856
10.929
11,877
11,513
16.907
11.051
M3-
Monthly
14,553
14.558
14.630
14.765
14.664
16.165
18.154
23.454
15.604
Others
6,269
6.259
7,081
9.194
7.087
6,863
7.529
7.348
7.001
Avg
13,024
13.036
13.125
15.337
13.228
14.553
15.047
19.047
14.092
Yearly
15.692
15.731
16.420
17.430
15.990
18.750
16,000
19.700
16.030
Quarterly
9.011
9.350
10.130
9.740
9.620
12:260
9.480
13.000
9.760
Monthly
13.978
14.060
14.100
15.650
14.710
14.010
15.120
15.910
14.800
M4-M3
Others 4.872
5.790
4,810
6,810
9.440
6,880
8.940
13.030
7,530
Avg
12,560
12,750
13.060
14.030
13.390
14.170
13.530
15.820
13.370
Time-Adapter, which further improved the model's predictive performance. The ablation results demonstrate the effectiveness of our innovations.
Table 12: In the ablation experiments, we set the input length to 672 and the output length to 96, with forecast lengths from the set (96, 192, 336, 720}. The values in bold represent the average results, while red indicates the best performance, followed by blue.
0.55
Traffie
Truflic
ECL
ECL
Trafiic
Metrics
MSC
MAE
MSB
MAG
MSE
MAE
MSC
MAB
MSE
MAC
MSB
MAB
MSC
MAB
MSC
MAE
96
0.145
0.240
0.387
0.367
0.150
0.245
0.440
0.304
0.140
0.237
0.368
0.258
0.135
0.233
0.373
0.263
192
0.165
0.257
0.410
0.277
0.168
0.262
0.459
0.314
0.158
0.254
0.390
0.268
0,152
0.249
0.394
0:272
Baseline
336
0.183
0.275
0.427
0.286
0.187
0.281
0.476
0.322
0.176
0.272
0.407
0.292
0.170
0.267
0.409
0.280
720
0.234
0.309
0.463
0.306
0.230
0,316
0.514
0.344
0.216
0.365
0.445
0.300
0.212
0.30
0.464
0.300
0.179
0.270
0,184
0.276
0.472
0.267
0.483
0,250
0.263
0.405
0.279
96
0.138
0.233
0.368
0.257
0.144
0.240
0.305
0.227
0.136
0.232
0.361
0.251
0.134
0.231
0.356
0:347
192
0.157
0.250
0.393
0.269
0.161
0.256
0.417
0. 0.786
0.154
0.349
0.583
0.263
0.153
0.245
0.377
0.357
+TsCC
335
0.176
0.269
0.251
0.181
0.274
0.436
0.295
0.175
0.267
0.401
0.159
0.266
0.393
0.266
720
0.218
0.305
0.312
0.225
0.309
0.474
0.316
0.215
0.305
0.439
0.294
0.210
0.301
0.431
0.287
0.172
0.264
0.409
0.250
0.177
0.270
0.431
0.294
0.170
0.263
0.386
0,270
0.166
0.262
0.389
0.264
95
0.133
0.227
0.363
0.255
0.125
0.230
0.373
0. 0.266
0.132
0.227
0.354
0.247
0.130
0.225
0.350
0.243
192
0.152
0.64
0.388
0.267
0.152
0.247
0.600
0.225
0.189
0.245
0.378
0.258
0.148
0.242
0.373
0.256
TTiee Adapter
336
0.189
0.262
0.411
0.281
0.170
0.265
0.417
0.23 0.263
166
0.761
0.366
0.367
0.164
0.259
0.390
0.263
720
0.211
0.299
0.314
0.209
0.295
0.293
0.429
0.285
0.166
0.358
0.279
0.167
0.360
0.413
0.163
0.257
0
0.266
0.161
0.255
0.386
0.261
Ablation on Short-Term Forecasting, The ablation experiments for short-term forecasting are
presented in the Table 13 We compared the performance by replacing different Large Language Models (LLMs) with SE-LLM on the M4 dataset. The findings indicate that LLama exhibits lower error,, followed by 5-en2.
20

---

Published as a conference paper at ICLR 2026
Table 13: The first column represents different LLMs replaced based on SE-LLM. "Others" is ob- tained by averaging the results from the weekly, daily, and hourly periods.
LLM
GPT2
Bert
Opt-125m
Qwca2.5
Liama
sMAPE
13.843
15.659
14.028
13.294
Year
MASE
3. 1.999
3.674
3.198
3.002
2.970
OWA
0.826
0,941
0.832
0.788
0.780
<APE
10.300
12.237
10,320
10.168
10.079
Quarterly
MASE
1.198
1.573
1.207
1.182
1.177
OWA
0.904
1.129
0.909
0.893
0.887
sMAPE
12,935
14.376
13.273
12.738
12.618
Monthly
MASE
0.957
1,118
1.021
0.938
0.931
OWA
0.898
0.883
0.875
sMAPE
5.061
6,358
5.191
4.998
4,896
Others
MASE
3.468
4.367
3,507
3.306
OWA
1.079
1.358
1.099
1.065
1.037
sMAPE
12.121
13.757
12.334
11.886
11.778
Average
MASE
1.977
1.578
OWA
0.8811
1.024
0.897
0.855
0.847
Ablation on Adapters. We use the best-performing Qwen2.5-0.5B model the baseline and eval- uate the proposed Timetedspeer's performance on time-series forecasting tasks. The results are compared to the LoRA adapter. The LoRA parameter configuration is kept consistent with the Time-Adapter, with a rank of 8 and the full ablation results are presented in Table [4] The com- parison protocol involves evaluating both adapters across several time-series datasets, specifically focusing on the performance metrics for each dataset. Our findings reveal that the LoRA adapter shows improvements only the weather and ECL datasets, whereas the Time-Adapter consistently enhances performance across all datasets. In particular, the error reduction is most notable in the weather dataset, where the Time-Adapter achieves 5.4% decrease in MSE compared to the base- line. This suggests that the Time-Adapter provides more generalized performance boost across diverse time-series data, as compared to LoRA.
Table 14: Time-Adapte compared with LoRA. The input length is 672 and forecasting horizons were set to (96, 192, 336, 720].
Datasets
ETThl
Weather
Traffic
ECL.
Solar
Metrics
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
96
0.352
0.393
0.162
0.245
0.356
0.247
0.134
0.231
0.175
0.230
192
0.378
0.411
0.209
0.259
0.377
0.257
0.152
0.248
0.194
0.245
TSCC
336
0.391
0.420
0.263
0.299
0.393
0.266
0.169
0.266
0.209
0.257
720
0.402
0.437
0.332
0.346
0.431
0.287
0.210
0.301
0.227
0.270
Avg
0.381
0.415
0.242
0.287
0.389
0.264
0.166
0.262
0.201
0.251
96
0.362
0.399
0.157
0.210
0.357
0.251
0.132
0.228
0.173
0.228
192
0.393
0.420
0.204
0.254
0.378
0.260
0.149
0.244
0.194
0.245
+LoRA
336
0.410
0.432
0.260
0.297
0.394
0.269
0.166
0.261
0.210
0.260
720
0.418
0.447
0.339
0.351
0.430
0.290
0.205
0.295
0.229
0.277
Avg
0.396
0.425
0.240
0.278
0.390
0.268
0.163
0.257
0.202
0.253
96
0.355
0.398
0.150
0,200
0.350
0.243
0.130
0.225
0.165
0.220
192
0.389
0.421
0.194
0.243
0.373
0.254
0.148
0.242
0.3
0.236
+Time-Adapter
336
0.409
0.435
0.247
0.285
0.390
0.263
0.164
0.259
0.199
0.249
720
0.418
0.448
0.323
0.339
0.429
0.285
0.203
0.293
0.219
0.266
Avg
0.393
0.426
0.229
0.267
0.386
0.261
0.161
0.255
0.192
0.243
Ablation on LLM-based SOTA methods. In this paper, we propose TSCC and Time-Adapter to enhance the interpretability of LLMs. Furthermore, we integrate our methods into Time-LLM, AutoTimes, and Time-CMA. However, due to inherent algorithmic constraints, Time-CMA relies solely on the LLM to extract textual descriptions of time-series data and thus cannot incorporate Time-Adapter. In Table I3] we present the performance enhancement achieved by our method in long-term forecasting compared to current research. It is worth noting that the original autoregres- sive forecasting paradigm of AutoTimes is not well suited for the ETT dataset. Therefore, we make several modifications in implementation. Specifically, the LLM component is instantiated with
21

---

Published as a conference paper at ICLR 2026
Table 15: The results were conducted in the ETThI, ETTh2, ETTml, and ETTm2 datasets. The Baseline represents our reproduced results, which differ from those in the original paper. Under the same parameters, we added TSCC and Time-Adapter. The forecasting horizons were set to (96, 192, 336, 720}.
Methods
Time-LLM
TSCC
Tise-AAapperr
AutoTimes
TSCC
Time-Adagter
Time-CMA
TSCC
Metric
MSF
MAE
MSF
MAE
MSE
MAE
MSE
MAR
MSE
MAF
MSE
MAF
MSE
MAF
MSE
MAF
96
0.364
0.411
0.375
0.401
0.368
0.400
0.366
0.307
0.354
0.394
0.352
0.395
0.395
0.414
0.577
0.506
192
0.414
0.430
0.405
0.425
0.401
0.421
0.391
0.419
0.385
0.415
0.383
0.414
0.435
0.437
0.429
0.424
ET1h]
336
01453
0.448
0.414
0.430
0.412
0.430
0
01432
11401
0.427
0.399
0.425
0.472
0.453
0.468
0.444
720
0.448
0:468
0.436
0.460
0.439
0.499
0.429
0.453
0.420
0.445
0.417
0.445
0.462
0.479
0.474
0.471
0.420
0.439
0.407
0.429
0.405
0.397
0.425
0.390
0,421
0.385
0.420
0.446
0.446
0.434
0,332
0.287
0.294
0.285
0.339
0.378
0.299
0.379
0.400
0.367
0.393
0.570
0.351
CTTh2
0
0.414
0,414
0.273
0,578
720
0.425
0.426
0.446
0.418
0.445
0440
02435
0,261
0,416
0.452
0.462
Aug
0.370
0.404
0.368
0.401
0.364
0.399
0,367
0.410
0,363
0.406
0,357
0.407
0.416
0.429
0.379
96
0.300
0.356
0.302
0.354
0.303
0.354
0.291
0.347
0.286
0,341
0.285
0,340
0.340
0.380
0.311
192
0,340
0.378
0.341
0.378
0.345
0.376
0.388
0.376
0.334
0,571
0,352
0,369
0.379
0.407
0.360
0.381
336
00.700
0.395
0.375
0.399
0.366
0.394
0.376
0.308
0.370
0.394
0.366
0.392
0.405
0.413
0.396
0.405
720
0.422
0.428
0.422
0.427
0.415
0.428
0.438
0.433
0.427
0.429
0.421
0:426
0.472
0.450
0.164
0.441
Aeg
0.358
0.389
0.360
0.390
0.357
0
0.361
0.389
0.335
0,384
0.351
0.382
0.399
0.413
0.383
0.396
96
0.172
0.265
0.188
0.270
0.178
0.268
0.176
0.264
0.174
0.261
0.173
0.262
0.191
0.274
0.173
0.256
192
0.252
0.306
0.247
0.314
0.229
0.563
0.238
0.307
0.236
0.303
0.254
0.302
0.257
0.316
0.342
0.301
ETTm2
336
0.281
0.339
0.276
0.335
0.422
0.346
0.341
0.291
0.313
0.350
0.307
0.323
720
0.362
0.387
0.389
Asg
0.262
GPT-2, multi-step forecasting strategy is adopted instead of autoregressive prediction, and tempo- ral positional encodings are integrated at the input level.
Statistical Stability. To ensure the reliability and reproducibility of results, all experiments are conducted under random seeds corresponding to 2024. 2025, and 2026. For each forecasting horizon, we report the mean and standard deviation of MSE and MAE across repeated runs under these seeds. This evaluation protocol mitigates randomness arising from parameter initialization,, data shuffling, and stochastic optimization dynamies. We note that, due to differences in environ- ment configurations and prrdware-depedeen behaviors, the LLM-basee modules may introduce slight numerical variability, leading to minor fluctuations in prediction performance within a range of approximately 0.01.. Such variations are expected and do not affect the overall conclusions. Ta- ble reporrss the forecasting results of SE-LLM under the three random seeds across five datasets and four forecasting horizons. The standard deviations of both MSE and MAE remain consistently small, indicating that the model is largely insensitive to random initialization and environmental perturbations. This stability holds across long-sequence datasets and different forecasting ranges, demonstrating that SE-LLM delivers robust and reproducible performance.
C DISCUSSION
The proposed TSCC and Time-Adapter modules are designed to explicitly bridge the gap between large language models and time series forecasting from modeling perspective. Rather than directly transferring language models to temporal data through prompt engineering or token-level alignment alone, TSCC focuses enriching semantic representations with structured temporal correlations, while the Lime-Adapter addresses the mismatch in temporal dependency modeling by explicitly cap- turing both long-term and short-term dynamics. Experimental results demonstrate that the general knowledge encoded in pre-trained language models insufficient for structured temporal reasoning unless it is coupled with mechanisms that explicitly encode temporal structure, dependencies, and variability. From this perspective, our work suggests a principled direction for future research on aligning foundation models with structured non-linguistic data, where adaptation should be guided by the intrinsic properties of the target domain rather than relying on superficial modality conversion.
D LIMITATION
We provide further clarification on the limitations of SE-LLM. Firstly, in short-term and zero-shot forecasting using the LLama model, Time-Adapter was not employed in all instances due to com- putational constraints, which contradicts the low computational cost objective of this paper. As a result, Time-Adapter was excluded from these experiments. In fact, incorporating Time-Adapter in
22

---

Published as a conference paper at ICLR 2026
Table 16: Performance stability under different random seeds (three runs). For each dataset and forecasting horizon, we report the results from three independent runs (2024, 2025, 2026 seeds), together with the averaged performance (mean) and variability (std). The consistently low standard deviations across all datasets indicate that SE-LLM is robust to randomness in initialization and training, demonstrating stable forecasting performance.
2014
2025
2026
mean
std
MSE
MAE
MSE
MSE
MAE
MSE
MAE
MSE
MAE
0.352
0.393
0.355
0.355
0.394
0.354000
0.393667
0.001414
0,000471
192
0.378
0.411
0.382
0.412
0.382
0.412
0.380667
0.411667
88008886
0.000471
ETTh
0.391
0.420
0.394
0.421
0.394
0.421
0.393000
0.420667
0.001414
0.000471
720
0.402
0.437
0.405
0.438
0.405
0.439
0,404000
0,438000
0.001414
0.0008 16
Avg
0.381
0.415
0.384
0.416
0.384
0.417
0.383000
0.416000
0.001414
0.0008 16
0.150
0,200
0.152
0.202
0.151
0,202
0,151000
0,201333
0.000816
0.000943
0.194
0.243
0.200
0.198
0.247
197333
0.246333
0.002494
0.002494
Weather
336
0.247
0.285
0,256
0.292
0.255
0.291
0.252000
0,289333
0.003742
0.003091
0.323
0.339
0.328
0.343
0.324
0.337
0.325000
0.339667
0.002160
0,002494
0.229
0.267
0.234
0.272
0.232
0.269
0.231667
0,269333
0.002055
0.002055
96
0.350
0.243
0,351
0.244
0
0.246
0.350667
0,244333
0.000471
0.001
0.373
0.254
0.373
0.254
0.375
0.256
0.373667
0.254667
0,000943
0.0009433
Traffic
336
0.390
0.263
0,389
0,263
0,390
0.264
0.389667
0.263333
0.000471
0.000471
720
0.429
0,285
0.427
0.284
0.428
0,285
0,428000
0,284667
0.000816
0.004471
Avg
0.386
0.261
0.385
0.261
0.386
0.263
0.385667
0.261667
0.000471
0.000943
96
0.130
0.225
0,133
0.228
0.134
0.231
0.132333
0.228000
0.001700
0.002449
192
0.148
0.242
0,151
0.244
0.152
0.247
0.150333
0,244333
0.001700
0.002055
ECL
0.164
0,262
0.166000
0.261000
0,001414
720
0.203
0.293
0,208
0,292
0,208
0.299
0. 0.206333
0.294667
0.002357
0.003091
Avg
0.161
0.255
0.165
0.257
0.165
0,260
163667
0,257333
8.0
0.002055
0.165
0.220
0.168
0,220
0.167
0.225
16666667
0.221667
0.001247
0.002357
192
0.183
0.236
0.188
0.236
0.186
0.241
0.185667
0.237667
0.002055
0.002357
Solar
336
0.199
0.249
0.206
0.251
0,202
0.255
0. 0.202333
0.251667
0.002867
0.002494
0.219
0.266
0.220
0.272
0.221000
0,268333
0.002160
0.002625
Avg
0.192
0.243
0.197
0.244
0.194
0.248
0. 0.493333
0.245000
0.002055
0.002160
specific layers would likely lead to performance improvements. However, due to concerns about computational costs, we did not pursue this avenue in the current study. Additionally, LLama was not considered for long-term forecasting due to its generally poor performance and high computa- tional demands.
E THE USE OF LLMS
This paper employs a large language model (LLM) to check for spelling errors, correct grammar, and refine the language.
23


Reviews:
[{"review_id": "QWQCpV4LVa", "reviewer": "Reviewer_tzsk", "summary": "The paper addresses adapting large language models (LLMs) to time series forecasting without fully fine-tuning the LLM backbone, targeting long-term trends, short-term dynamics, cross-channel dependencies, and anomalous behaviors. It proposes SE-LLM, which combines a Temporal-Semantic Cross-Correlation (TSCC) module that aligns time series embeddings with the LLM semantic space, separates anomalous and de-anomalized semantics, and performs gated multi-channel fusion, with a Time-Adapter module that injects lightweight temporal inductive bias into the attention key and value paths via parallel long- and short-horizon branches. The contribution is a modular architecture that augments a mostly frozen LLM through cross-modal temporal–semantic alignment and structured temporal adapters instead of full fine-tuning. The method is evaluated on long-horizon multivariate forecasting benchmarks, short-term forecasting benchmarks at different frequencies, and cross-domain transfer settings, reporting reductions in standard forecasting metrics (MSE, MAE, SMAPE, MASE, OWA) relative to recent LLM-based and transformer-style baselines, supported by ablations isolating TSCC and Time-Adapter.", "questions": "1. Can you provide statistical variation (for example, multiple random seeds, confidence intervals, or significance tests) for the main quantitative comparisons to recent LLM-based and transformer-style baselines in the primary result tables?  \n2. Can you include qualitative analyses or failure cases that visualize which time points and channels TSCC marks as anomalous or high-impact, and how this correlates with the model's predictions, in order to substantiate the interpretability claims?  \n3. Can you precisely describe the runtime and efficiency measurement setup (hardware, batch size, input length, prediction horizon, caching policy) used to generate the reported training and inference cost comparisons, so that the claimed efficiency of the proposed Time-Adapter can be reproduced?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "1. The paper proposes a structured framework (SE-LLM) intended to bridge the mismatch between raw temporal dynamics and the semantic token space of a largely frozen LLM by explicitly aligning time series signals with language representations and injecting temporal inductive bias through specialized modules.\n2. The architecture separates into TSCC and Time-Adapter. TSCC performs temporal–semantic cross-alignment, anomaly and non-anomalous semantic disentanglement, and gated multi-channel fusion, while Time-Adapter augments attention key and value paths using lightweight temporal branches for long-term and short-term dynamics instead of full backbone fine-tuning.", "weaknesses": "1 The paper attributes interpretability to anomaly and non-anomalous semantic separation and channel-gated fusion, but it does not present qualitative, per-sequence visual evidence (e.g., highlighted time steps or channels) that would allow readers to verify that the model is focusing on meaningful temporal events rather than incidental fluctuations.  \n2 Reported efficiency comparisons (e.g., training and inference cost versus error) do not specify the hardware environment, batch size, sequence lengths, or caching assumptions, which limits the reproducibility and interpretability of the efficiency and lightweight adaptation claims.  \n3 Some core components are only partially specified at the technical level. For example, the anomaly modeling and semantic disentanglement step (AM-VAE within TSCC) is described conceptually, but the precise probabilistic objective (e.g., reconstruction terms and regularization) is not fully detailed, so it is difficult to assess how the model enforces the claimed anomalous versus de-anomalous semantic separation.  \n4. The presentation quality is below typical conference standards, including missing or underspecified captions (Figure 1 does not explain subfigure (c), the efficiency plot lacks runtime conditions, and Table 6 does not clearly describe the comparison protocol).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "zRtKeTdEhX", "reviewer": "Reviewer_azcz", "summary": "This paper presents a framework that enhances the performance of using LLM for time series forecasting by bridging the modality gaps between time series numerical data and language word embeddings. Specifically, it designs two modules: Temporal-semantic cross-correlation, which aligns temporal embedding with LLM's semantic space using cross-attention and anomaly modeling; and the Time-Adapter, which is a plug-in module that augments the LLM attention mechanism through dual LSTM pathways. The evaluation on multiple datasets and in various settings demonstrates that the SE-LLM consistently outperforms baselines and ablation studies show the effectiveness of design modules.", "questions": "See weakness", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The designed TSCC module addresses the modality gaps between temporal embeddings and the semantic space of LLMs by employing cross-attention and a VAE-based anomaly decomposition. It presents a more interpretable semantic representation for time series forecasting.\n\n2. The low-rank style of using dual LSTM is suitable for freezing LLMs for time series forecasting. It compensates for the Transformer’s weaknesses in handling local and global temporal dependencies while maintaining parameter efficiency.\n\n3. The paper evaluates SE-LLM across a wide range of forecasting horizons, datasets, and various settings, showing the robustness of the designed model.", "weaknesses": "1. The pipeline presents several interdependent components such as cros-attention, AM-VAE, top-k correlation filtering and gating fusion, which make the pipeline highly complex. The implementation details are missing.\n\n2. Missing qualitative analysis showing how the semantic space affects LLM token representations for time series forecasting. Whether the model truly benefits from the semantic space is questioned, which is a claim of this paper. \n\n3. The designed component introduces significant model complexity; thus, it is important to compare the number of parameters or states training requirements.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "MvMgH8uyIe", "reviewer": "Reviewer_cETm", "summary": "This paper works on time series forecasting (TSF) and presents an LLM-based TSF framework SE-LLM to address the modality gap between language and time series. It mainly includes:  (1) TSCC which infuses temporal patterns (e.g., periodicity, anomalies) into semantic embeddings, and (2) Time-Adapter that can capture both long-term and short-term dependencies in time series. The model freezes the LLM backbone to preserve generalizability and reduces computational cost via sequence dimensionality reduction. Experiments on long-term, short-term, and zero-shot forecasting tasks demonstrate performance improvements over state-of-the-art methods, with ablation studies validating the effectiveness of the proposed modules.", "questions": "N.A.", "limitations": "", "rating": 4, "confidence": 5, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1. It's reasonable to study the modality gap between language and time series, and integrate temporal patterns to bridge the gap.\n2. Time-Adapter’s dual LSTM paths for long/short-term dependency modeling can complement Transformers’ weaknesses in temporal dynamics.\n3. Extensive experiments have been conducted to validate the effectiveness of modules.", "weaknesses": "1. While TSCC and Time-Adapter address known limitations, their individual components (cross attention, VAE for anomaly modeling, adapter-based fine-tuning) are not fundamentally new. The paper fails to sufficiently articulate how their combination offers a unique contribution beyond incremental improvements over existing modular designs.\n2. In Table 1, the performance gain over the best baseline is minor in almost all cases.\n3. The choice of lightweight LLMs (GPT2, Qwen2.5-0.5B) limits generalizability. No experiments on larger LLMs (e.g., Llama-7B) raise questions about scalability, even with the claim of prioritizing efficiency.\n4. Zero-shot forecasting results lack analysis of why the model generalizes across datasets.\n5. The impact of different temporal patterns (e.g., seasonal vs. irregular) on performance is unexplored.\n6. The exclusion of Time-Adapter from some short-term/zero-shot experiments due to computational constraints undermines the framework’s completeness.\n7. The description of the TSCC module’s channel dependency enhancement and the Time-Adapter’s integration into multi-head attention is vague. Key formulas (e.g., Eq 8) lack sufficient explanation.\n8. (Minor) The natbib package is not correctly used.\n9. (Minor) There are some typos, e.g., (1) in Figure 2(b), \"JS with Temporal-Semantc\" -> \"JS with Temporal-Semantic\", (2) in Table 1, 4, 6, 12, 14, \"Mertics\" -> \"Metrics\".\n10. (Minor) In Table 7, some column names are missing.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "d0zpGSOh7b", "reviewer": "Reviewer_9dMA", "summary": "Key contributions include:\n\n1. Enhances token embeddings by infusing temporal patterns (periodicity, anomalies) into LLM semantic spaces.\n\n2. A plugin for self-attention that models short/long-term dependencies, adapting frozen LLMs to time-series forecasting.\n\n3. Freezes LLM parameters and reduces token sequence dimensionality, lowering computational costs.\n\n4. Outperforms SOTA on benchmarks (ETTh1, Traffic, M4) in long/short-term and zero-shot forecasting.", "questions": "1. Could the authors estimate the performance trade-offs for larger LLMs (e.g., Llama-7B) without full training?\n\n2. Why choose VAE over other generative models (e.g., Diffusion Models) for anomaly simulation in TSCC?\n\n3. Have you explored comparing TSCC with graph-based temporal models (e.g., MTGNN) to capture spatial dependencies?\n\n4. How does TSCC handle datasets with non-stationary distributions (e.g., financial time-series)?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "1. Novel integration of temporal dynamics (anomalies, periodicity) into LLM semantic spaces via TSCC, addressing a critical modality gap.\n2. Time-Adapter innovatively combines LSTM with low-rank projections to capture multi-scale dependencies, unlike prior token-level alignment methods.\n3. TSCC’s AM-VAE for anomaly modeling and cross-correlation filtering is technically sound.\n4. Well-structured with clear figures and pseudocode.", "weaknesses": "1. Experiments limited to smaller LLMs. Larger models (e.g., Llama-7B) are untested due to computational constraints, raising questions about generalizability.\n\n2. AM-VAE’s reconstruction loss (Algorithm 1) lacks quantitative comparison to other anomaly detection methods\n\n3. Missing comparisons to hybrid methods (e.g., LLM + GNNs) in Table 1\n\n4. No discussion on latency or memory overhead for edge/IoT applications, despite claims of efficiency.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)