UniCA: Unified Covariate Adaptation for Time Series Foundation Model
Authors: Lu Han
Yu Liu
Lan Li
Qiwen Deng
Jian Jiang
Yinbo sun
Zhe Yu
Binfeng Wang
Xingyu Lu
Lintao Ma
Han-Jia Ye
De-Chuan Zhan
Venue: ICLR 2026 Poster (2026)
Keywords: time series foundation model
adaptation
covariate-aware forecasting
heterogeneous covariates
Primary area: learning on time series and dynamical systems

Abstract: Time Series Foundation Models (TSFMs) have achieved remarkable success through large-scale pretraining. However, their design primarily targets real-valued series, limiting their ability to handle general forecasting tasks involving diverse and often \emph{heterogeneous covariates}—such as categorical variables and multimodal data (e.g., images, text)—which are typically task-specific and difficult to leverage during pretraining. To address this gap, we propose Unified Covariate Adaptation (UniCA), a framework to bridge TSFMs with general covariate-aware forecasting. UniCA first performs covariate homogenization to transform heterogeneous covariates into high-level homogeneous series representations and then fuses them via a unified attention-based fusion mechanism. UniCA is compatible and universal for adaptation with both homogeneous and heterogeneous covariates, incorporating extra covariate information while preserving the generalization ability of TSFMs. Extensive experiments on multiple unimodal and multimodal covariate-aware forecasting benchmarks demonstrate the superiority of UniCA, highlighting the promise of covariate-aware TSFM adaptation in real-world forecasting scenarios. Code: https://github.com/hanlu-nju/UniCA.

Published as a conference paper at ICLR 2026
UNICA: UNIFIED COVARIATE ADAPTATION FOR TIME SERIES FOUNDATION MODEL
Lu Han" 1.2 Yu Liu" Lan Li Qiwen Deng". Jian Jiang', Yinbo Sun", Zhe Yu²; Binfeng Wang', Xingyu Lu Lintao Ma 33 Han-Jia Y, De-Chuan Zhan School of Artificial Intelligence, Nanjing University, China National Key Laboratory for Novel Software Technology, Nanjing University, China
3 Ant Group. Hangzhou, China
fhanlu, yehj, zhandc] e e lamda n juu cn, fnuoman.ly, lintao. mww.meaatttgoooooo com, yww.. 623 gma il. com
ABSTRACT
Time Series Foundation Models (TSFMs) have achieved remarkable success through large-scale pretraining. However, their design primarily targets real- valued series, limiting their ability handle general forecasting tasks involving di- verse and often heterogeneous covariates-such as categorical variables and mul- timodal data (e.g., images, text)-whieeh are typically task-specific and difficult to leverage during pretraining. To address this gap, we propose Unified Covariate Adaptation (UniCA), framework bridge TSFMs with general covariate-aware forecasting. UniCA first performs covariate homogenization to transform hetero- geneous covariates into high-level homogeneous series representations and then fuses them via unified attention- based fusion mechanism. UniCA is compatible and universal for adaptation with both homogeneous and heterogeneous covari- ates, incorporating extra covariate information while preserving the generalization ability of TSFMs. Extensive experiments on multiple unimodal and multimodal covariate-awand forecasting benchmarks demonstrate the superiority of UniCA, highlighting the promise of covariate-aware TSFM adaptation in real-world fore- casting scenarios. Code:
INTRODUCTION
Time series forecasting is essential in a wide range of domains. including environmental monitoring (rruca et al.] (2022). traffic man- agement Kadiyala & Kumar (2014), energy systems [ardakos et al. (2013), communica- tion networks Peng et al. (2013), and health- care Morid et al. (2023). Accurate forecasting supports critical decisions in planning, policy- making, and operations. Traditional statisti- cal models such as ARIMA and Exponential Smoothing Box et al. (2015) have been widely used for their simplicity and effectiveness in specific settings. With the advancement of deep learning, models based Recurrent Neural Networks (RNNs) Hochreiter Schmidhuber 1997 : Cho et al. (0044):Rangaapue et al. 2018): Franceschi et al. (2019) have enabled more expressive modeling of complex temporal
Covariates
Addptaation
Heterogeensuss
Generalized Pre/Train
General Covariste-Aware Adaptation
Figure 1: TSFMs are pretrained on time series from diverse domains. However, many tasks tain homo/heterogeneous covariates that are hard use in pre-training. Adaptation methods han- dle these covariates are important in these tasks.
al. (2018) and Convolutional Neural Networks (CNNs) Bai
"Equal Contribution
1 "Correpoondig Author

---

Published as a conference paper at ICLR 2026
dynamics. Transformer-base based architectures Zhou et al. (2021 Nie et al.
(2023); ; Liu et al. (002bb)
further advanced the field by capturing long-range dependencies and achieving strong performance, particularly in hong-horizon multivariate forecasting. Inspired by the success of foundation models
in NLP and vision Devlin et al.] (2019); [Brown et al. (2020); |Radford et al. (202T); Kirillov et al.
(2023) recent Time Series Foundation Models (TSFMs) Das al. (2024); Woo et al. (002)); Ansari et al. (2024): [ooswan et al.] (2024) have shown strong generalization capability by pretraining on large-scale time series. They learn transferable temporal representations and deliver impressive performance even in zero-shot scenarios.
However, most tht TSFMs (Ansari et al. 2024] Das et 2024 Goswami et al. 2024)
are fundamentally designed for univariate forecasting, processing each time series in isolation. This architectural choice renders them unable to leverage the critical relationships between a target se- ries and its exogenous covariates, limiting their applicability in many real-world scenarios. Some TSFMs adopt covariate-aware (Woo et al., (024) strategies during pretraining: yet, models trained in this manner often fail to achieve stable and superior performance across diverse tasks (Aksu al., 2024). More fundamentally, the standard TSFM pretraining paradigm imposes a key limitation: only effectively leverage homogeneous covariates (e.g., real-valued time series similar to the tar- get variable). This restricts their ability to handle heterogeneous covariates, which are increasingly common in practical scenarios. Heterogeneous covariates typically includes structured categorical variables (e.g.. item IDs, calendar features) and multimodal inputs (e.g.. images, texts) (Ma et al.)
2024 Liu et al., 2024a). The diversity and task-specific nature make their integration into existing TSFM pipelines non-trivial.
While prior work has addressed unimodal covariate-aware forecasting or multi-modal forecasting through specialized model architectures [Salinas et al. (2020); Lim et al. (2019); Das et al. (2023); Jin al. (2024): Ma et al. (0024); Liu al. (2024a), these methods are often biased to the task- specific data, lack the generalization ability, and underperform compared to large-scale pretrained
TSFM models (Aksu et al.) 2024). Therefore, a key challenge remains: How can we adapt powerful pretrained TSFMs to general covariate forecasting, including homogeneous and heteroge- neous covariates, without losing the generalization ability obtained pretraining?
In this paper, we address this challenge by proposing unified and effective adaptation method named Unified Covariate Adaptation (UniCA). The core idea is to perform covariate homoge- nization that transforms heterogeneous covariates into unified, homogeneous temporal representa- tion, representing the high-level feature changing over time. This transformation allows us to solve the general covariate forecasting with a unified framework in the time series modality. In addition, we design an attention-based fusion mechanism with pre-fusion and post-fusion components that incorporate covariate information before and after the TSFM backhone, respectively, with the pa- rameters of the TSFMs unchanged. The adaptation modules fully utilize the encoding and temporal extraction power of the TSFMs, incorporating the covariates' information while retaining the fore- casting ability obtained during their pretraining process. Extensive experiments across a wide range of benchmarks, including traditional single-modal modal covariate datasets and challenging multimodal datasets, demonstrate the effectiveness and flexibility of UniCA. Our results show that by properly adapting covariate information into the series space, TSFMs can significantly outperform specialized models, thus opening new possibilities for general-purpose time series forecasting in covariate-rich environments. Our main contributions are summarized as follows:
We formalize the problem of adapting Time Series Foundation Models (TSFMs) to general covariate-aware forecasting scenarios, where the heterogeneous covariates, like categorical or multi-modal covariates, can not be directly utilized by TSFMs.
We propose Unified Covariate Adaptation (UniCA), a novel framework featuring: (a) covariate homogenization to transform diverse covariates into unified temporal representation, and (b) dual secentiion-bsseee fusion mechanism to integrate covariate representation with frozen TSFM backbone.
We conduct comprehensive experiments across single-modal modal and multimodal covariate datasets, demonstrating that UniCA enables TSFMs to achieve superior performance compared to task- specific baselines. The effectiveness of covariate homogenization on TSFMs and specialized methods also proves that it is a simple way to integrate heterogeneous covariates.
2

---

Published as a conference paper at ICLR 2026
2
RELATED WORK
Time Series Foundation Models (TSFMs). Recent efforts (Ansari et al., 2024 Woo et al.
2024 Goswami et al. 2024 Das et al.) 2024 Ekambaram et al., 2024) have developed large- scale pretrained models for time series, enabling zero-shot forecasting fine-tuning across tasks. Moirai (Woo et al., 2024), MOMENT (Goswami et al.] 2024), and TimesFM (Das et al., 2024) adopt a patch-based transformer architecture, while TTM (Ekambaram et al.) 2024) builds on TSMixer (Ekambaram et al., (023), which uses MIPs to mix temporal and feature dimensions. In contrast, Chronos (Ansari al. 2024) tokenizes time series into discrete vocabularies and trains language models directly on these sequences. TSFMs are trained based on either channel- independence (Han et al.) 2024; Goswami et al. 2024; Das et al., 2024 Ansari al. 2024) that ignores covariates, or the equivariant attention mechanism (Woo et al. (2024) that requires covariates
be homogeneous. The adaptation heterogeneous covariates scenarios is an unsolved challenge.
Forecasting with Covariates. Covariates- help in the forecasting of the target series, but that we are not interested in forecasting-play play crucial role in capturing external signals in forecasting tasks. Classical models like ARIMA (Box & Jenkins 1968) incorporate them via extra coefficients, while deep models such as DeepAR (Salinas et al. 2020) and TFT (Lim et al., 2019) integrate them as inputs or through specialized encoders. NBEATSx (Olivares et al. 202 concatenates covariates with the main series for fixed-size input. TTM-CM (Ekambaram et al. 2024) introduces fine- tuning approach based on channel mixing. (Chen & Zhao 2024) introduces MiTSformer to handle mixed time series by recovering latent continuous representations from discrete variables to mit- igate heterogeneity. Among TSFMs, only Moirai (Woo et al. 2024) natively supports covariates by flattening series and covariates into a joint sequence, using variate IDs for differentiation. None of the existing methods can handle both homogeneous and heterogencous, especially multi-modal covariates, while our method meant solve this challenge.
Multimodal Time Series Forecasting. Most multimodal forecasting studies focus on textual en- hancement. A line of work seeks to utilize the powerful temporal encoding ability of LLM im- prove the forecaster (Zhou et al., 2023 Gruver et al., 2023). While these methods provide the pos- sibility for multimodal forecasting, they usually handle static textual data. Another line combines numerical time series with dynamic textual data, e.g. news (Dheenadayalan et al., 2022 Wang et al.) 2024a) or weather reports (Obst et al. 2019). Time-MMP (Liu et al., (0024a) introduces a multimodal dataset and model that processes time and text modalities independently and merges them via lin- car fusion. Towards image covariates, FusionSF (Ma et al. 2024) proposed the MMSP dataset and a method meant especially for the satellite scenario. Our approach proposed a new way that converts information from other modalities into series and handles them uniformly in time series modality.
Adapting TSFMs to handle Covariates. Standard foundation model adaptation, often employ- ing parameter-efffiient methods like adapters (Houlsby et al.) 2019) and LoRA (Hu et al. 2022). typically assumes consistency between pretraining and downstream task input/output structures. Adapting TSFMs to handle covariates is more complex. While methods like TimesFM (Das et al.; 2024). which uses auxiliary regressor for residual correction, and ChronosX (Pineda-Arango et al. 2025), which injects covariates through linear transformations but limits its application to only point-wise TSFM. All the current adaptation methods struggle with heterogeneous covariates, highlighting the need for more flexible adaptation strategies, which our proposed method provides.
PROBLEM FORMULATION
General Covariates-Aware Time Series Forecasting. In covariates-aware time series forecast- ing, the objective is to predict YイŞ1т4Н E H by utilizing both past observations of the target YET E RTx1 and the external covariates, as well as considering their temporal relationships. The model takes into account both static covariates S and dynamic covariates C1.T+ # make predic- tions about the future. Formally, we can express the prediction problem as:
YT+LT+H f(Y.r, Cr
For notational simplicity, we denote both future-known and future-unknown covariates as CI-T+ For future-unknown covariates, the values in the interval [T +++ 1 H] are unobserved at prediction time T.
3

---

Published as a conference paper at ICLR 2026
where the static covariates S remain unchanged within series. The dynamic covariates CLT++H may provide extra information about the past or future state. Both S and CIT+H may contain homogeneous and heterogeneous covariates.
Heterogeneous Covariates. In traditional time series analysis tasks, exogenous covariates typi- cally share the same form as the target series, often represented as real-valued numerical variables. This homogeneity allows exogenous covariates and targets to be processed under a unified model- ing framework without the need for moddalitysppecife designs (Woo et al.) [024] Liu et al. 2024b). Such covariates referred to as homogeneous covariates. However, with the advancement of data collection, covariates in modern forecasting scenarios exhibit increasingly diverse forms. In many practical applications, covariates are no longer restricted to simple real-valued signals but may in- volve a wide range of data types. Among these, a particularly significant challenge for TSFMs comes from heterogeneous covariates. These covariates can be broadly categorized into two major types: (1) Categorical covariates: Discrete attributes such as item identifiers, store locations, event types, temporal markers. These variables not inherently numerical and require embedding techniques or specialized handling to be incorporated into forecasting models. (2) Multimodal co- variates: nigh-dimensional, complex data modalities such as images, text descriptions. The emer- gence of heterogeneous covariates poses fundamental challenges to existing TSFM architectures. Unlike homogeneous covariates, which can be directly integrated, heterogeneous covariates demand modality-specific preprocessing, feature extraction, and fusion strategies.
Covariate-Aware Adaptation. Time Series Foundation Models (TSFMs) are designed to model the temporal dependencies within given series. These models are trained diverse time series from different domains, which makes it hard to incorporate covariates across different domains, co- variate adaptation involves modifying the model architecture to integrate the covariate information while fully utilizing the temporal encoding ability. The mathematical formulation of this task can be expressed as follows: Given TSFM f/m, the objective is construct new forecaster based on the trained foundation model:
(1) where Jada is the adaptation module, and Jada ff is the composition model after adaptation. In contrast training covariate-awes aware deep learning models from scratch, covariate adaptation involves three distinct challenges:
+++IT+ Jada o fi (Y CI.T+H. S),
Compatibility: The adaptation module should be compatible with pretrained TSFMs without requiring extensive full-model retraining or architecture redesign.
Universality: should be able to handle both homogeneous and heterogeneous covariates.
. Generalization Preservation: It should leverage the temporal encoding capabilities learned dur- ing pretraining while preserving the generalization ability of the foundation model.
In response to these challenges, we propose Unified Covariate Adaptation (UniCA).
4 METHODOLOGY
Unified Covariate Adaptation (UniCA) is general framework that enables Time Series Founda- tion Models (TSFMs) to effectively incorporate heterogeneous covariate information without dis- rupting their pretrained temporal modeling capabilities. At a high level, UniCA follows two key principles (1) Covariate Homogenization: We transform categorical and multimodal covariates into dense continuous series representations, thus reducing the heterogeneity gap between covariates and target time series. (2) Modular Fusion: We decompose the TSFM architecture into interpretable stages and insert an attention- fusion module to inject covariate information at appropriate locations without interfering with the model's pretrained dynamics.
4.1 COVARIATE HOMOGENIZATION
To address covariate heterogeneity, UniCA introduces a homogenization process that converts all co- variates into a unified homogeneous space. Specifically, categorical covariates are processed using embedding layers that map discrete tokens into continuous vectors. Multimodal covariates-such as images or texts-are initially fed through modality-speciffe encoders (e.g., convolutional neural net- works for images, pretrained transformers for text) to obtain dense feature representations (het).
4

---

Published as a conference paper at ICLR 2026
Past Target
Takenizer
Pre-Fusion
Temponii Encoder
Past-Fusion Module
Predictor
Conditional
softnax
Past Tarpet
Static Covariate
Tokenizer
Pre-Pusion Module
Homegeneous
Coveristes
Future Coveriate
Past Homogrneous Covariate
Fature Homogenepux Coveriate
Conditiona Self
Hidden
Tokenizen
Attention (CSA)
Cat Homegenizer
Homogenizer
Homoeonnee
Attention
Aal
Coveriates
Catsgorical Coveriate
Image Covarione
Text Covariate
(b) Postiuusion Module
Figure 2: Overview of Unified Covariate Adapter (UniCA). UniCA consists of two key pipelines (1) Covariate Homogenization: We use converter to transform heterogeneous covariates into dense continuous series representations, thus reducing the heterogeneity gap between covariates and target time series. (2) Modular Fusion: We decompose the TSFM architecture into interpretable stages and insert Pre-Fusion and Post-Fusion modules to inject covariate information at appropriate locations without interfering with the model's pretrained dynamics.
Similar to connectors used in multimodal learning (Liu et al. 2023 we use a Covariate Homog- enizer (CH), a simple linear layer, transform (hett) into latent homogeneous covariates (teet) These covariates encapsulate the temporal dynamics of high-level features derived from heteroge- neous covariates:
(het) = CH(HEr
where C1:TH E (RT++)))d'*: with hhat being tunable hyperparameter. Finally, all homoge- neous covariates-whether hidden or observed are aligned along the temporal dimension and concate- nated produce cohesive set of homogeneous series covariates C1.+H [CI:T CTT H enabling their integration into a unified covariate fusion framework. In the following part, we as- sume the unified covariates representation C1.T+H (R+++) HM where M the total number of homogeneous covariates, including the observed homogencous and the homogenized heterogeneous covariates. This homogenization process ensures the universality of UniCA.
(het))
(het))
4.2 COVARIATE FUSION MODULE
Decomposition of TSFM. To better incorporate covariate information into the TSFM and fully leverage capabilities, we first decompose the TSFM architecture according the functionality:
Tokenizer: Z T(Y ( This module transforms time series inputs Y into sequence of tokens Z IR Pxd where d is the token dimension, and P is the number of tokens along the temporal dimension and varies between patch-based (Das et al.] 2024 Liu et al. 2024c) and point-based (Ansari et al. 2024 Hoo et al.) 2025 Shi et al. 2004) methods. The tokenizer is responsible for generating suitable representations for the temporal encoder, acting as the connection between the representation and the main part of the model.
Temporal Encoder H E(Z): Subsequently, the encoder processes the tokenized sequence Z to extract high-level temporal patterns and dependencies. The most popular encoder is Trans- former. This stage leverages the pre-trained temporal encoding capabilities of the TSFM.
Predictor
YT+LT++ = -P(H): Finally, the predictor utilizes the encoded representations H to generate forecasts Y for the future horizon T to T + H. For decoder-only architec-
tures (Brown et al. 2020 Das et al. 2024), we regard the linear output layer as the predictor.
This modular decomposition is applicable to the vast majority of TSFMs, ensuring the compatibil- ity of UniCA and enabling clean separation of responsibilitiess and facilitating the integration of covariate information without disrupting the core temporal processing. Based on this, we propose attention-based pre/post fusion modules to incorporate past and future covariates into the TSFMs.
5

---

Published as a conference paper at ICLR 2026
Pre-Fusion Module. Prior to the encoding stage, the pre-fusion module integrates past covariate information with the historical target values. This module enriches the tokens with historical exter- nal factors, allowing the encoder to capture the joint dynamies between the time series and its past covariates. Inspired and simplified from Lim et al. (2019), we use a Conditional Attention Pool- ing(CAP) mechanism to fuse the past information while maintaining interpretability. Concretely, given past target YI,,, past covariates CL.T and the static feature S, C1.T and S are first converted embeddings by the tokenizer of the TSFM and newly initialized embedding layer p.
E = T(Cr.)) Es p(S),
(3)
where E E PPxxx Es E R (Es 0 if static covariates provided) the representa- tion of dynamic and static covariates. Then:
Zr = CAP(E Es) == (aattmax(AAVV.
(4)
where A = CRN(fat(Ee.rr) Es) and V GRN(ECt.r
GRN is Gated Residual Network (a residual MLP) used in Lim et al. (211)) flat(-) flattens the last two dimension of Ec A E R xx is the attention affinity on each feature, V E / Then, Gated Linear Unit (GLU) Dauphin et al. (017)) is used to further trade off the influence of covariates Ze:
ğ Z GLU(
(5)
This fused representation is then forwarded the temporal encoder to produce H PRd:
H E(Z)
Post-Fusion Module. The future-known covariates CT+1.T+ provide direct insight into future conditions, making them particularly valuable for forecasting. Therefore, we choose to use post- fusion module to incorporate future covariate information into the encoded representations H after the temporal extraction process. This step crucial when future exogenous factors are expected to influence the forecast. We first tokenize the future-known covariates:
Ecr 1:21 н = T(CT+I.++H)
where Ecr+1T4 represents the tokenzied future covariates. We then apply the conditional attention pooling mechanism to selectively aggregate the most relevant aspects these future covariates at each time step. Formally,
(8) Once the most relevant future information is selected and fused, integrate it with the past quence by feeding both into a self-attention layer. This step enables the model to learn contextual dependencies between past and future covariates, allowing for enriched representation that better captures the interplay between historical and horward-looking information. Mathematically, the final fused representation is obtained as:
ZCr-LT-H CAP
Es).
[,, Zo, n. SelfAttn([H. Zor P-rr-rr-a 1. ..r=))).
Then we predict the future target Y with the predictor P:
Yr+IT+ = P(H).
(10)
Similar adapters (Houlsby al. 2019 in LLM, our UniCA is plug-in module that keeps the pretrained model parameters unchanged, thus preserving generalization capabilities of TSFMs.
4.3 Loss FUNCTION
A key design principle of UniCA is its seamless compatibility with diverse TSFMs. We train the UniCA adaptation modules using the same loss function the foundation model was originally pre- trained with. This aligns the adaptation process with the TSFM's inherent objective. Specifically, we employ the quantile loss (Wen et al. 2017 Lim et al. 2019 for Chronos and TimesFM, the Huber loss (Huber) 1992) for TiEe-MoE, and Negative Log Likelihood (NLL) for Moirai. For training sta- bility across series of varying scales, we normalize each target instance by its historical mean and standard deviation, following the instance normalization approach in Kim al. (202))

---

Published as a conference paper at ICLR 2026
Thean back
27
1170
Creare Be
Tim
Chaniea Me
Tasule
"ell
Culle
More
More
(a) Unimodal Datasets (TS)
(b) MMSP (TS- (TS-Image)
(c) Time- Mime-MMDDR (TS- Text)
Figure 3: Forecasting performance on general covariate-aware forecasting datasets, including 12 unimodal datasets and multi-modal datasets MMSP and Time-MMD. Results are reported MAPE averaged over sub-datasets for both unimodal and Time-MMD datasets. For the MMSP dataset, MAE used instead, as near-zero target values render MAPE unstable.
5 EXPERIMENTS
Following the evaluation in (Aksu et al. 2024 Zhou al. 2021). we consider four met- ries to evaluate the performance of forecasters: Mean Absolute Percentage Error (MAPE), Mean Square Error (MSE), Mean Absolute Error (MAE) for point forecasting ability, and Continuous Ranked Probability Score (CRPS) for probabilistic forecasting, which is implemented as the mean Weighted Quantile Loss (WQL) (Park et al., 2022). In all experiments, the WQL is computed on quantile levels f0.1. 0.2, .... 0.9). For methods generating sample forecasts, we compute the quan- tiles based on 256 samples, whereas quantile forecasting methods are trained on the same quantile levels for evaluation. Following the practice in Woo al. (2024) to reduce the dataset bias, we normalize each result by dividing the result of the Naive method (Hyndman & Athanasopoulos 2018), where all forecasts have the value of the last observation.
Metrics.
Compared Methods. To comprehensively evaluate the effectiveness of our proposed UniCA framework, we compare it against broad set of baseline methods spanning four major categories: (a) Specialized Models: These models are trained from scratch for specific forecasting tasks. We include two representative subtypes: (i) univariate methods, which include PatchTST (Nie et al,
2023) and (ii) covariate-awesr awwre methods, which includes DeepAR (Salinas et al.] 2020). TFT (Lim
al. 2019). TiDE (Das et al. (023), N-BEATSx (Olivares et al. 2021). TimeXer (Wang al.,
2024b). (b) Pretrained TSFM (ZS). They are evaluated in a zero-shot manner without task-specific
Tune-tuning. We select three popular TSFMs Chronos-Bolt (Ansari al. 20244. TimesFM (Das)
et al. 2024), Time-MoE (Shi et al. 2024). (c) Fine-tuned TSFM (SFT): Full- parameter fine-tuning on downstream datasets. (d) Adapter-based Models: These methods introduce additional mod- ules attached to the TSFM to inject covariate information, allowing adaptation with fewer train- able parameters. We compare the Linear Regression (I.R) adaptation proposed in (Ansari et al. 2024 Das et al.] 2024), which regresses the ground truth against covariates and the residuals are fed to TSFMs, TTM-R2 [Ekambaram et al. 2024), ChronosX2
Sineda-Arange et al. 2025 ). For
multi-modal experiments, we include FusionSF (Ma et al. 2024) and MM-TSF(Chronos- Bolt as TS predictor) (Liu et 2024a) for TS- IS-image task (MMSP); Time-LLM (Jin et al. 2024). MM- TSF and ThatTime (Wang et al. 2025) for TS-text task (Time-MMD).
Implementation Details. We adopt default context length for each TSFM, e.g. 2048 for Chronos- Bolt and 4096 for Time-MoF, while prediction lengths are dataset-specific (Appendix A). All time series data is pre-processed such as normalization, as detailed in Appendix C33 In all experiments, learning rates were selected from {10-3, 10-4, 1003 10106 66 based on validation performance. The homogenizer CH is implemented as simple linear model. For each heterogeneous covariate, we select the number of projected hidden series d in {1, 2, 4, 8, 16). For image covariates. a simple l-yeeer CNN (Krizhevsky et al. 2012) because the satellite images with dimension 64 x 64 x 4 are not regular images. For text covariates, use GIST (Solatorio) 2024) the encoder. A comprehensive list of hyperparameters is presented in Appendix C.7
This method is especially designed for Chronos-Ts T5 model. Results obtained from our own implementation, official code for ChronosX was available the time of this work.
7

---

Published as a conference paper at ICLR 2026
5.1 UNI-MODAL COVARIATE AWARE FORECASTING
Datasets. We evaluate our method on 12 publicly available datasets commonly employed in covariate-aware forecasting research (Lim et al., 2019 Das al. 2023] Oreshkin et al.] 2019 Arando-Arango et al., 2025 Aksu et al. [024] Olivares et al., 2021). ). To create the test sets, we employ two distinct strategies based on the number of subseries in each dataset. For datasets with relatively large number of subseries, i.e., we reserve the final "prediction length" points of each subseries as the test set (Lim et al. 2019). For datasets with fewer subseries, we partition 10% the data as the test set and apply a sliding window approach for evaluation, with a step size of 1 (Zhou et al. 202I). We also spare validation sets with the same points as the test set. Detailed descriptions of the datasets can be found in Appendix A
Main results. The results in Figure 3a highlight the effectiveness of UniCA in uni-modal covariate- aware forecasting. UniCA consistently outperforms zero-shot TSFMs, achieving optimal performance among adapter methods (0.506 MAPE for Chronos-Bolty While standard finetuning shows minimal gains or degradation, UniCA delivers substantial improvements, confirming its abil- ity to preserve generalization. UniCA also surpasses specialized methods (0.596-0.738 MAPE), demonstrating universality across architectures. These results validate UniCA's design goals of com- patibility, universality, and generalization preservation.
5.2 MULTI-MODAL COVARIATE-AWARE FORECASTING
Datasets. We evaluated UniCA on tasks involving multi-modal covariates, specifically images and text. For image-basee based covariates, we utilized the Multimodal Solar Power (MMSP) dataset from (Mal jet al. 2024). For text- based covariates, we used the Time-MMD (Liu et al. 2024a) dataset. Details in the Appendix A Traditional covariate models use no multi-modal information.
Main results. On the Time-MMD dataset (figure 3c TimesFM (UniCA) ranks among the top performers, significantly outperforming most specialized and pretrained baselines. In the MMSP benchmark (figure 3b). TFT variants achieve the best results, while our
models show consistent improvements over their base versions. Notably, UniCA provides substantial gains for TimesFM (6.5% reduction in error) and Chronos-Bolk (5.9% reduction), confirming its effec- tiveness in multi-modal covariates modeling. This suggests that multi-modal covariates can hinder forecasting performance if not handled appropriately, but our UniCA framework can robustly con- fusion module and covariate homogenization,
trol the information flow through its leading to consistent performance improvements across different foundation models.
Homogenizer on Specialized Methods. To evaluate the generalizability of the Covariate Homog- enizer (CH), we integrate it into two representative covariate-based forecasting models support multi-modal forecasting: TFT and TiDE. The models augmented with CH consistently outperform their vanilla counterparts. For instance, TFT+CH achieves a notable 5% drop on MMSP and 13.0% on Time-MMD Similarly, TiDE+CH demonstrates a substantial improvement, with the MAE re- duced by 30.1% % on MMSP and 10.5% on Time-MMD. These results highlight that CH provides a simple yet effective way to integrate multimodal information.
5.3 ANALYSIS
Efficiency. The homogenizer of the UniCA uses linear layer. The pre-/post- fusion module com- putes the covariate weights and pooling with the weights, introducing complexity only linear to the number of covariates and the model's dimension. All the components are lightweight. Figure 5a shows that UniCA introduces little computation or storage burden for the TSFMs.
Effectiveness of Covariate Adaptation. Figure highlights the effectiveness of the adapter- based covariate integration strategy in leveraging the generalization capabilities of pre-trained TSFM models. In figure 4a| we observe that the Adapted variant-using our proposed UniCA conster-- outperforms the Pretrained zero-shot model (Moira(ZZS)) across all met- rics. This indicates that the diverse reliance of covariates different datasets is difficult to learn with a pretrained model. Adaptation with UniCA provides better performance. However, the pre- trained knowledge should not be fully discarded. In figure 4b) compared to fully finetuned back- bones, the Frozen + Adapter setup achieves better performance, particularly in terms of MAPE and CRPS. These findings validate our design intuition: adapter-based covariate incorporation serves as
8

---

Published as a conference paper at ICLR 2026
Pretrained Adastedd
Finstune Frozon
Linea
c.om
2.30
MAPS
MS
Hal HoM Ma CANS MAR MAPE MAX
COR4
(a) Pretrained vs, Adapted (b) Finetuned vs. Frozen
(c) Structure
(d) Dimension
Figure 4: Average relative MAPE unimodal datasets with model setups (a) Pretrained: Moirai(ZS), Adapted: Moirai (UniCA), (b) Finetumed: Chronos-Bolt (UniCA) with fine-tuned
backbone, Frozen:freeze backbone; Ablation on (c) structure of covariate homogenizer. (d) hid-
den dimension of covariate bomogenizer.
Covalit Crualeer
90% Homo
190
Time Stop
(a)
(b)
(c)
The adapter adds minimal overhead in inference time (left panel) and trainable parameters (right panel). (b) Covariate Homogenization on MMSP: Aligned heterogeneous covariates reveal meaningful patterns like seasonality and trends. (c) Attention Maps: The fusion module dynamically attends to different covariates over time for the sample in (b).
Figure 5: Analysis of UniCA. (a) Efficiency Time-MMD:
lightweight yet powerful mechanism to bridge the gap between general- purpose time series repre- sentations and task-specific covariate contexts, fully utilizing pretrained knowledge while enabling covariate-aware forecasting.
ceneral-purpose
Homogenizer Architecture. Our evaluation of the homogenizer architecture, detailed in figure 4c shows that simple Linear layer and Multi-Layer Perceptron (MLP) achieve similar performance. Notably, the Linear model slightly outperforms the MLP, indicating that more parsimonious design only sufficient but preferable. Therefore, we use the Linear homogenizer as the default.
Homogenizer Dimension. We conduct an ablation study on the homogenized dimension dbst which controls the projection space for diverse covariates. Varying hee from 128, we evaluate performance using MAE, MAPE, MSE, and CRPS. As shown in Figure 4d performance improves sharply from hh
4 (c.g., MSE drops from 0.15 to under 0.10), highlighting the benefit of more expressive projection. Increasing dhet beyond 8 yields diminishing returns and slight perfor- mance degradation, suggesting redundancy or overfitting. Metries remain stable in the range [4, 32). We set geet by default.
Visualization of Covariate Homogenization. To illustrate UniCA's behavior and the effect of covariate homogenization, we visualize examples from the MMSP dataset (figure 5b The homog- enized representations of satellite images reveal meaningful temporal patterns: Image Covariate captures periodicity, while Image Covariate also reflects trends aligned with target scale. This shows that homogenization effectively transforms heterogeneous covariates into task-relevant rep- resentations, validating our alignment design in UniCA.
Souure5csshww attention maps before and after fusion for the same sample in figure 5b The fusion module dynamically adjusts attention weights across time; notably, Covariate 13 consistently receives the highest weights, matching its rich temporal patterns and strong correlation with the target. In contrast, the target itself is not overly emphasized, suggest- ing the fusion module learns to complement, rather than duplicate, target signals demonstrating its ability identify and integrate informative covariates.
Attention-based Covariate Selection.

---

Published as a conference paper at ICLR 2026
6 CONCLUSION
In this work, we address a critical limitation of existing Time Series Foundation Models (TSFMs): their inability to incorporate homogencous and heterogeneous covariates in general forecasting ef- fectively. To overcome this, we propose UniCA, a unified covariate adaptation framework that ex- tends TSFMs general covariate-awar forecasting scenarios. UniCA achieves this by transforming diverse covariates into high-order homogeneous series and integrates them via an attention-based fusion module, preserving the integrity of pretrained temporal modeling. Extensive experiments both unimodal and multimodal datasets demonstrate UniCA's compatibility, universality, and ef- fectiveness across diverse forecasting tasks. A discussion of its limitations and future directions is provided in Appendix !
7 ETHICS STATEMENT
This work complies with the ICLR Code of Ethics Our study focuses on methodological contri- butions to time series forecasting with heterogeneous covariates. All experiments conducted on publicly available benchmark datasets (c.g., M5, Retail, MMSP, Time-MMD), which do not contain personally identifiable sensitive information. No human subjects private data and thus no additional ethical approval was required. We have taken care to ensure that our meth- ods and findings do not pose foreseeable risks of harm, discrimination, misuse. We believe this research aligns with the principles of responsible stewardship, fairness, transparency, and repro- ducibility.
involved,
REPRODUCIBILITY STATEMENT
We have made extensive efforts to ensure reproducibility of our results. Detailed descriptions of the model architecture, training procedures, hyperparameters, and evaluation protocols are provided in the main paper and Appendix. We also include ablation studies and additional results in the supple- mentary materials to validate robustness. All datasets used our experiments are publicly acces- sible, with preprocessing steps clearly documented. To further support reproducibility, provide anonymous source code and scripts, enabling verification and extension of our findings.
ACKNOWLEDGMENTS
This research was supported by Natural Science Foundation of Jiangsu Province of China under Grant (BK20250062), NSFC (62476123, 62522605), Collaborative Innovation Center of Novel Software Technology and Industrialization, Ant Group Research Intern Program.
REFERENCES
Taha Aksu, Gerald Woo, Juncheng Liu, Xu Liu, Chenghao Liu, Silvio Savarese, Caiming Xiong, and Doyen Sahoo. Gift-eval: A benchmark for general time series forecasting model evaluation. CoRR, abs/2410.10393, 2024.
Alexander Alexandrov. Konstantinos Benidis, Michael Bohlke-Schneine Valentin Flunkert, Jan Gasthaus, Tim Januschowski, Danielle C. Maddix, Syama Rangapuram, David Salinas, Jasper Schulz, Lorenzo Stella, Ali Caner Türkmen, and Yuyang Wang. GluonTS: Probabilistic and Neural Time Series Modeling in Python. Journal of Machine Learning Research, 21(116):1-6, 2020.
Abdul Fatir Ansari, Lorenzo Stella, Ali Caner Türkmen, Xiyuan Zhang, Pedro Mercado, Huibin Shen, Oleksandr Shchur, Syama Sundar Rangapuram, Sebastian Pineda-Arango. Shubham Kapoor, Jasper Zschiegner, Danielle C. Maddix, Michael W. Mahoney, Kari Torkkola, An- drew Gordon Wilson, Michael Bohlke-Schneidee and Yuyang Wang. Chronos: Learning the language of time series. CoRR, abs/2403.07815, 2024.
Shaojie Bai, J. Zico Kolter, and Vladlen Koltun. An empirical evaluation of generic convolutional and recurrent networks for sequence modeling. CoRR, abs/1 803.01271, 2018.
10

---

Published as a conference paper at ICLR 2026
G. E. P. Box and G. M. Jenkins. Some recent advances in forecasting and control. Journal of the Royal Statistical Sociery. Series C (Applied Statistics), 17(2):91-09,, 1968.
George EP Box, Gwilym M Jenkins, Gregory C Reinsel, and Greta M Ljung. Time series analysis: forecasting and control. John Wiley & Sons, 2015.
Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhari- wal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss. Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. Language models are few-shot learners. In NeurlPS, 2020.
Jiawei Chen and Chunhui Zhao. Addressing spatial-temporal heterogeneity: General mixed time series analysis via latent continuity recovery and alignment. In Advances in Neural Information Processing Systems, volume 37, 2024.
Kyunghyun Cho, Bart van Merrienboer, Dzmitry Bahdanau, and Yoshua Bengio. On the properties of neural machine translation: Encoder-decoder approaches. ESST@EMNLA, pp. 103-111. Association for Computational Linguisties, 2014.
Corporación Favorita. Corporación favorita grocery sales forecasting competition, 2018. URL
https: www.
Abhimanyu Das, Weihao Kong, Andrew Leach, Shaan Mathur, Rajat Sen, and Rose Yu. Long-term forecasting with tide: Time-series dense encoder. CoRR, abs/2304.08424, 2023.
Abhimanyu Das, Weihao Kong, Rajat Sen, and Yichen Zhou. A decoder-only foundation model for time-series forecasting. In ICML. OpenReview.net, 2024.
Yann N. Dauphin, Angela Fan, Michael Auli, and David Grangier. Language modeling with gated convolutional networks, In ICML,, volume 70 of Proceedings of Machine Learning Research, pp- 933-941. PMLR, 2017.
Jacob Devlin, Ming-Wes Wei Chang, Kenton Lee, and Kristina Toutanova. BERT: pre-training of deep bidirectional transformers for language understanding. In NAACI-HITT pp. 4171-4186, 2019.
Kumar Dheenadayalan, Nitesh Kumar, Suprabath Reddy, and Sumant Kulkarni. Multimodal neu- ral network for demand forecasting. In ICONIP, volume 13625 of Lecture Notes in Computer Science, pp. 409 421. Springer, 2022.
Vijay Ekambaram, Arindam Jati, Nam Nguyen, Phanwadee Sinthong, and Jayant Kalagnanam. Tsmixer: Lightweight mip-mixer model for multivariate time series forecasting. In KDD, pp- 469-469, 2023.
Vijay Ekambaram, Arindam Jati, Pankaj Dayama, Sumanta Mukherjee, Nam Nguyen, Wesley M. Gifford, Chandra Reddy, and Jayant Kalagnanam. Tiny time mixers (ttms): Fast pre-trained mod- els for enhanced zero/few-shot forecasting of multivariate time series. In 2024, 2024.
Mostafa Farrokhabadi, Jethro Browell, Yi Wang, Stephen Makonin, Wencong Su, and Hamidreza Zareipour. Day-ahead electricity demand forecasting competition: Post-covid paradigm. 1EEE Open Access Journal of Power and Energy, 9:185-191, 2022.
Jean-Yves Franceschi, Aymeric Dieuleveut, and Martin Jaggi. Unsupervised scalable representation learning for multivariate time series. In NeurlPS, pp. 4652-4666 2019.
Mononito Goswami, Konrad Szafer, Arjun Choudhry, Yifu Cai, Shuo Li, and Artur Dubrawski. MO- MENT: A family of open time-series foundation models. In Forty-first International Conference on Machine Learning, ICML 2024, Vienna, Austria, July 21-27, 2024. OpenReview.net, 2024.
Aleksandra Gruca, Federico Serva, Llorenç Lliso, Pilar Rípodas, Xavier Calbet, Pedro Herruzo, Jin Pihrt, Rudolf Raevskyi. Petr Šimánek, Matej Choma, et al. Weather4cast at neurips 2022: Super-resolution rain movie prediction under spatio-temporal shifts. In NeurIPS 2022 Competi- fion Track, pp. 292-313. PMLR, 2022.
11

---

Published as a conference paper at ICLR 2026
Nate Gruver, Mare Finzi, Shikai Qiu, and Andrew Gordon Wilson. zero-shot time series forecasters. In NeurIPS, 2023.
Large language models
Lu Han, Han-Jia Ye, and De-Chuan Zhan. The capacity and robustness trade-off: Revisiting the channel independent strategy for multivariate time series forecasting. IEEE Transactions on Knowledge and Data Engineering, 36(11):7129-714 2024.
Sepp Hochreiter and Jürgen Schmidhuber. Long short-term memory. Neural computation, 9(8):
1735-1780, 1997.
Noah Hollmann, Samuel Müller, Lennart Purucker, Arjun Krishnakumar, Max Körfer, Shi Bin Hoo, Robin Tibor Schirmeister, and Frank Hutter. Accurate predictions on small data with a tabular foundation model. Nature, 2025.
Tao Hong, Pierre Pinson, and Shu Fan. Global energy forecasting competition 2012. International Journal of Forecasting, 30:357-363, 2014.
Tao Hong, Pierre Pinson, Shu Fan, Hamidreza Zarcipour, Alberto Troccoli, and Rob J Hyndman. Probabilistic energy forecasting: Global energy forecasting competition 2014 and beyond. Inter- national Journal of Forecasting, 32:896-91 913666993,27 2016.
Tao Hong, Jingrui Xie, and Jonathan Black. Global energy forecasting competition 2017: Hier- archical probabilistic load forecasting. International Journal of Forecasting, 2019.
Shi Bin Hoo, Samuel Müller, David Salinas, and Frank Hutter. The tabular foundation model tabpfn outperforms specialized time series forecasting models based on simple features. CoRR, abs/2501. 02244,, 2025.
Neil Houlsby, Andrei Giurgiu, Stanislaw Jastrzebski, Bruna Morrone, Quentin de Laroussilhe, An- drea Gesmundo, Mona Attariyan, and Sylvain Gelly. Parameter-efficient transfer learning for NLP. In ICML, volume 97 of Proceedings of Machine Learning Research, pp. 2790-2799, PMLR, 2019.
Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models. In ICLR. OpenRe- view.net, 2022.
Peter Huber. Robust estimation of a location parameter. In Breakthroughs in statistics: Methodol- ogy and distribution, pp. 492-518. Springer, 1992.
Rob Hyndman and George Athanasopoulos. Forecasting: principles and practice. OTexts, 2018.
Nicholas
J.
Hourly https:
demand, generation, and weather www http:/wwwwwwww...c.ee http///aaastttt//enaaaae aiccholashaaaaa and weather
energy
data.
2019. Accessed:
2025-05-09.
Ming Jin, Shiyu Wang, Lintao Ma, Zhixuan Chu, James Y. Zhang, Xiaoming Shi, Pin-Yo Yu Chen, Yuxuan Liang, Yuan-Fang Li, Shirui Pan, and Qingsong Wen. Time-Ilm: Time series forecasting by reprogramming large language models. In ICLR. OpenReview.net, 2024.
Akhil Kadiyala and Ashok Kumar. Multivariate time series models for prediction of air quality inside a public transportation bus using available software. Environmental Progress Sustainable Energy, 33(2):337-311, 2014.
Evaggelos G Kardakos, Minas C Alexiadis, Stylianos I Vagropoulos, Christos K Simoglou, Pan- delis N Biskas, and Anastasios G Bakirtzis. Application of time series and artificial neural network models in short-term forecasting of pv power generation. 2013 48th International Universities' Power Engineering Conference (UPEC), pp. 1-6. IEEE, 2013.
Taesung Kim, Jinhee Kim, Yunwon Tae, Cheonbok Park. Jang-Ho Choi, and Jaegul Choo. Re- versible instance normalization for accurate time-series forecasting against distribution shift. In ICL.R 2021.
12

---

Published as a conference paper at ICLR 2026
Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloé Rolland, Laura Gustafson, Tete Xiao, Spencer Whitchead, Alexander C. Berg, Wan- Yen Lo, Piotr Dollár, and Ross B. Girshick. Segment anything. In IEEE/CVF International Conference Computer Vision, ICCV 2023, Paris, France, October 1-6, 2023, pp. 3992-4003. IEFE, 2023.
Alex Krizhevsky, Ilya Sutskever, and Geoffrey Hinton. Imagenet classification with deep convo- lutional neural networks. In NIPS, pp. 1106-1114, 2012.
Jesus Lago, Grzegorz Marcjasz, Bart De Schutter, and Rafal Weron. Forecasting day-ahead electric- ity prices: A review of state-of-the-ar algorithms, best practices and open-access benchmark. Applied Energy, 293:1 16983, 2021.
Bryan Lim, Sercan Ömer Arik, Nicolas Loeff, and Tomas Pfister. Temporal fusion transformers for interpretable multi-horizon time series forecasting. CoRR, abs/1912.09363, 2019.
Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning. NeurIPS, 2023.
Haoxin Liu, Shangqing Xu, Zhiyuan Zhao, Lingkai Kong, Harshavardhan Kamarthi, Aditya B. Sasa- nur, Megha Sharma, Jiaming Cui, Qingsong Wen, Chao Zhang, and B. Aditya Prakash. Time- mmd: Multi-domain multimodal dataset for time series analysis. In NeurlPS, 2024a.
Yong Liu, Tengge Hu, Haoran Zhang, Haixu Wu, Shiyu Wang. Lintao Ma, and Mingsheng Long- itransformer: Inverted transformers are effective for time series forecasting. In ICLR. OpenRe- view.net, 2024b.
Yong Liu, Guo Qin, Xiangdong Huang, Jianmin Wang, and Mingsheng Long- Timer-xl: Long- context transformers for unified time series forecasting. CoRR, abs/2410.04803, 2024c.
Ziging Ma. Wenwei Wang, Tian Zhou, Chao Chen, Bingqing Peng, Liang Sun, and Rong Jin. Fu- sionsf: Fuse heterogeneous modalities in vector quantized framework for robust solar power forecasting. In SIGKDD, pp. 5532-5543. ACM, 2024.
Spyros Makridakis, Evangelos Spiliotis, and Vassilios Assimakopoulos. The m5 competition: Back- ground, organization, and implementation. International Journal of Forecasting, 38(4):3325 1336, 2022.
Clayton Miller, Anjukan Kathirgamanathan, Bianca Picchetti, Pandarasamy Arjunan, June Young Park, Zoltán Nagy, Paul Raftery, Brodie W. Hobson, Zixiao Shi, Forrest Meggers National Uni- versity of Singapore, University College Dublin, Comision Nacional de Energia Atomica. Ar- gentina., Berkeley Education Alliance for Research in Singapore, The University of Texas at Austin, University of California Berkeley, C. Zhang Cardiff University, and Princeton University. The building data genome project 2, energy meter data from the ashrae great energy predictor iii competition. Scientific Data, 7, 2020.
Mohammad Amin Morid, Olivia R Liu Sheng, and Joseph Dunbar. Time series prediction using deep learning methods healtheare. ACM Transactions on Management Information Systems, 14(1):1-2) 29,,1129,222 2023.
Yuqi Nie, Nam H. Nguyen, Phanwadee Sinthong, and Jayant Kalagnanam. A time series is worth 64 words: Long-term forecasting with transformers. In ICLR, 2023.
David Obst, Badih Ghattas, Sandra Claudel, Jairo Cugliari, Yannig Goude, and Georges Oppenheim. Textual data for time series forecasting. CoRR, abs/1910.12618. 2019.
Kin G. Olivares, Cristian Challu, Grzegorz Marcjasz, Rafal Weron, and Artur Dubrawski. Neural basis expansion analysis with exogenous variables: Forecasting electricity prices with nbeatsx. CoRR, abs/2104.05522. 2021.
Boris N Oreshkin, Dmitri Carpov, Nicolas Chapados, and Yoshua Bengio. N-BEATS: Neural basis expansion analysis for interpretable time series forecasting. In ICLR, 2019.
13

---

Published as a conference paper at ICLR 2026
Youngsuk Park, Danielle C. Maddix, Lrançois-Xavien Aubet, Kelvin Kan, Jan Gasthaus, and Yuyang Wang. Learning quantile functions without quantile crossing for distribution free time series forecasting. In A/STATS, volume 151 of Proceedings of Machine Learning Research, pp. 8127- 8150. PMLR, 2022.
Yu Peng, Miao Lei, Jia Guo, and Xiyuan Peng. Multiresolution analysis and forecasting of mobile communication traffic. Chinese Journal of Electromics, 22(2):373-376, 2013.
Sebastian Pineda-Arango, Pedro Mercado, Shubham Kapoor, Abdul Fatir Ansari, Lorenzo Stella, Huibin Shen, Hugo Senetaire, Caner Turkmen, Oleksandr Shchur, Danielle C. Maddix, Michael Yuyang Wang, and Syama Sundar Rangapuram. Chronosx: Adapting pre- trained time series models with exogenous variables. CoRR, abs/2503.12107 2025.
Alec Radford, Jeff Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. Language models are unsupervised multitask learners. 2019.
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning transferable visual models from natural language supervision. In ICML, volume 139 of Proceedings of Machine Learning Research, pp. 8748-8763. PMLR, 2021.
Syama Sundar Rangapuram, Matthias W. Seeger, Jan Gasthaus, Lorenzo Stella, Yuyang Wang, and Tim Januschowski. Deep state space models for time series forecasting, In NeurlPS, pp. 7796- 7805, 2018.
David Salinas, Valentin Flunkert, Jan Gasthaus, and Tim Januschowski. DeepAR: Probabilistic forecasting with autoregressive recurrett networks. International Journal of Forecasting, 36(3): 1181-1191, 2020.
Xiaoming Shi, Shiyu Wang, Yuqi Nie, Dianqi Li, Zhou Ye, Qingsong Wen, and Ming Jin. Time- moe: Billion-scale time series foundation models with mixture of experts. CoRR, abs/2409.16040, 2024.
Aivin V. Solatorio. Gistembed: Guided sa-sae selection of training negatives for text embedding fine-tuning. CoRR, abs/2402.16829, 2024.
Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Niko- lay Bashlykov, Soumya Batra, Prajiwal Bhargava, Shruti Bhosale, Dan Bikel, Lukas Blecher, Cristian Canton-Ferrer, Moya Chen, Guillem Cucurull, David Esiobu, Jude Fernandes, Jeremy Fu, Wenyin Fu, Brian Fuller, Cynthia Gao, Vedanuj Goswami, Naman Goyal, Anthony Hartshorn Saghar Hosseini, Rui Hou, Hakan Inan, Marcin Kardas, Viktor Kerkez, Madian Khabsa, Isabel Kloumann, Artem Korenev, Punit Singh Koura, Marie-Anne Lachaux, Thibaut Lavril, Jenya Lee, Diana Liskovich, Yinghai Lu, Yuning Mao, Xavier Martinet, Todor Mihaylov, Pushkar Mishra, Igor Molybog, Yixin Nie, Andrew Poulton, Jeremy Reizenstein, Rashi Rungta, Kalyan Saladi, Alan Schelten, Ruan Silva, Eric Michael Smith, Ranjan Subramanian, Xiaoqing Ellen Tan, Binh Tang, Ross Taylor, Adina Williams, Jian Xiang Kuan, Puxin Xu, Zheng Yan, Iliyan Zarov, Yuchen Zhang, Angela Fan, Melanie Kambadur, Sharan Narang, Aurelien Rodriguez, Robert Stojnic, Sergey Edunov, and Thomas Scialom. Llama 2: Open foundation and fine-tuned chat models. CoRR, abs/2307.09 2023.
Chengsen Wang, Qi Qi, Jingyu Wang, Haifeng Sun, Zirui Zhuang, Jinming Wu, Lei Zhang, and Jianxin Liao. Chattime: A unified multimodal time series foundation model bridging numerical and textual data. In AAAL, pp.
AAAI Press, 2025.
Xinlei Wang, Maike Feng, Jing Qiu, Jinjin Gu, and Junhua Zhao. From news to forecast: Integrating event analysis in Ilm-based time series forecasting with reflection. In NeurIPS, 2024a.
Yuxuan Wang. Haixu Wu, Jiaxiang Dong, Guo Qin, Haoran Zhang, Yong Liu, Yunzhong Qiu, Jian- min Wang, and Mingsheng Long. Timexer: Empowering transformers for time series forecasting with exogenous variables. In NeurIPS, 2024b.
Zhixian Wang, Qingsong Wen, Chaoli Zhang, Liang Sun, Leandro Von Krannichfeldt, and Yi Wang. Benchmarks and custom package for electrical load forecasting. CoRR, abs/2307.07191, 2023.
14

---

Published as a conference paper at ICLR 2026
Ruofeng Wen, Kari Torkkola, Balakrishnan Narayanaswamy, and Dhruv Madeka. A multi-horize quantile recurrent forecaster. CoRR, abs/1711.11053, 2017.
Gerald Woo, Chenghao Liu, Akshat Kumar, Caiming Xiong, Silvio Savarese, and Doyen Sahoo. Unified training of universal time series forecasting transformers. In ICMI. OpenReview.net, 2024.
Ashfak Yeaf.. Pdb electric power load history. https://wwzz http://wwwwwwggwl.... wtt.:/taaee.....a. -twpaayyeaailppp--oo--------eeee
2021. Accessed: 2025-05-09.
Haoyi Zhou, Shanghang Zhang, Jieqi Peng, Shuai Zhang, Jianxin Li, Hui Xiong, and Wancai Zhang. Informer: Beyond efficient transformer for long sequence time-series forecasting, In AAAI, pp. 1110--1111,, 2021.
Tian Zhou, Peisong Niu, Xue Wang, Liang Sun, and Rong Jin. One fits all: Power general time series analysis by pretrained LM. In NeurlPS, 2023.
Appendix
Table of Contents
Appendices
15
A Datasets Descriptions
16
A Uni-modal Time Series Datasets
16
A.2 Muli--modal Datasets
16
B Compared Methods
18
B.1 Specialized Method
18
B.2 Pretrained Method
19
B. Multimodal Metehod
19
Finetuned Method
Adapter Method
Implementation Details
Code availability
20
Compute Resource Information
20
C33
Preprocessing
20
C44
Architecture TSFM
21
C.S
Algorithm of UniCA
22
C.6
Optimization
22
C.7
Hyperparamters
22
D Experiment Details
23
D.1 Train-Test Splitting,
23
D.2
Evaluation Metrics
24
E
Full Results
25
E.
Robustness Evaluation via Error Bars
25
E.2 Unimodal Forecasting
26
Modal Forecasting
28
15

---

Published as a conference paper at ICLR 2026
E.4
Imputation
29
Showcases
31
More Ablation
31
G.
Fusion Position.
31
G.2
Infueecee of Modality Encoder
36
G.3
Architecture of Covariate Homogemizer
36
G
Inffuence of Static Covariates
37
G.5
Imaac of Covariates
................................
38
6.6
Robustness to Noisy Covariates
38
7
Fusion Architecture
39
Correlation Analysis of Homogenized Covariate Embeddings
40
Discussion of Limitation
41
The Use of LLMS
41
A DATASETS DESCRIPTIONS
A.1 UNI-MODAL TIME SERIES DATASETS
Our study employed uni-modal datasets with covariates. Detailed descriptions of the targets, covariates, and data sources are provided in Table | while Table 2 outlines the dataset statis- ties. Specifically, a selection of electricity load forecasting datasets, including Covid19 Energy, GFC12, GFC17, PDB, Spain, BDG-2 Hog, BDG-2 Bull, and BDG-2 Cockatoo, was directly re- trieved from the Lotsa repository on Hugging Face: https: : /wwgggiggaacceeeeceoo//o co dataset s Saresforce/ lotsa_dattte An exception to this is the GFC14 dataset, which acquired from its original source (Hong et al. 2016) due to the absence of covariate data the version available on Hugging Face.
Table 1: Dataset Descriptions
Descriptions
EPF
Day aboud electiiitt pricss Don Sye majo murlze Nord poot. PIM, FR. BIF: and Di
Ieal forecass, wind ganematon
Lago ct 2011
competion using bicramhical
sore
year, pracs, work
duily
Rabil
Copprocia Fesecito Grocery Scles Firecosting
conpetitcon hosted Kaggle
week day, morth, year holidays events.
stote to, tem or procottoon of prices.
Orperasinn Foomits 201k
BDG-2 Heg
bailding--oe lovel collacted from 3055 elerticity makess which onenn 1556 building.
Hos open
includes residienial
wind dresion, speed
Miler N00 Warg
200
BDG-2 collested frvm Unis
Asaii.
leapenutee, speed
Miller
Wang
BDG-2 Cockatoo
BDG-2 duses coliectod fnon Concel Univenity.
lemperitue
Miler 2020 Wasg 2027 Who 2004
31 year load data foo the Day Alead Electiiity
Energy
Demand Furzsaeing Compeiiiion The purpose dudy
lomperitune
Rarehhabadh Wang
GPC12
Clubal Encaay Foressiint Cumpetiion 2012
20 aggrogated level load series data from
hacene theme is unc- tnser contopantiine beoween the tenyerauee and load data
Randoall selected secoad teapenatue data
Hong et 2014 2014
2023 Who
2024
GPC
Global
yorr
Compettions
trmporem
2016
GUOC1
Eight load data from you 2016 2017 snigtallyy fnm
Global Eive gy Poreaastngg Compettion 2014
sir somperatte
Borg al 2010 ca
2011 Winne
2014
Honny
Spais
Hunrly zrongy Semond gracnaionn woither tive aujer clses in Spein It's Kagle data compettion.
sir Jomperatue of Banzlonu
2010 Wang et 2003 Visi
2004
A.2 MULTTIMODAL DATASETS
To evaluate our approach, we utilized two distinct multi-mon modal datasets: Time-MMD (Liu et al. 2024a) and the Multimodal Solar Power (MMSP) dataset (Ma al. 2024
16

---

Published as a conference paper at ICLR 2026
Table 2: Dataset Statistics. Dynamic covariates and past dynamic covariates are covariates that are observed and unobserved in the forecasting horizon, respectively.
Num. Series
Categorical Cov.
Real Cov.
Num.
Pred.
Dataset Name
Domain
Freq.
Static
Dynamic
Dynamic
Past Dynamic
Obs.
Len.
EPF M5-daily
Energy/Price
5
H
0
0
2
218,280
48
Sales
30,490
5
0
59,181,090
28
119,048
140,246,203
BDG-2 Bull
Energy/l oad
41
H
O
0
0
3
719,304
48
BDG-2 Cockatoo
Energy/Load
1
H
O
0
0
17.544
48
1ovid Energy
Energy/Load
Q
31,912
48
GFC12
Energy/Load
20
O
0
788,280
48
GFC14
Enengy/Load
1
H
Q
0
0
60,600
48
GFC17
Energy/Load
8
H
0
0
1
140.352
48
BDG-2 Hog
Lodd
421,056
PDB
Energy/Load
0
17,520
48
Spain
Energy/Lood
H
0
35,064
48
Table 3: Multi-modal Dataset Descriptions
Dutaset
Domain: Target
Num. Series
Freq-
Multi Modal Covariates
Num.
Pred.
Name
Text
Image
Time Series
Obs.
Len.
Agriculture: retail broiler
1
W
Market News Report:
Daily Nationa Broiler
N/A
N/A
17,520
12
compusite
Market Gilanoe,
Climate:
Index
Precipitation
National Climate Report
Drought Report
Economy:
U.S. International Trade in
US unde goods Woeld
Goods and Services
N/A
N/A
423
12
Time-MMD
Indicutors Report
Energy: gosoline price
Anmul Energy Outlook ElA:
Weekly Permleumm Status Report
N/A
N/A
1479
12
Social Good: unemploymen statistics
montlly employment situations;
armual labor force churacteristies by race othuicity
NIA
N/A
900
12
Public Health:
Weekly U.S. Influmzza Surveilluce Report Annull Flu Senson Key Studies and News Reports
Influenzal lkk Illness
w
N/A
N/A
1389
12
statisties
Environment:
quality
Daily
Traflic:
Trallic Volume Trendss
Weekly Traflic Volome Repart
N/A
N/A
531
12
Security:
Billion-Dolas Weather
Disaster and Emergency
Climate
N/A
emergency
Energy:
Numerical Weather
MMSP
Solar Power
88
H
NA
Satellite Images
Predictions, (Latitude, Longitude)
1,129,920
24
Generation
The Time-MMD dataset is multi-domain resource encompassing nine diverse areas such as Agri- culture, Climate, Health, and Traffic. It features paired textual and time series data, where the textual information derived from curated reports and web search results, as detailed in (Liu et al., (024a). Among the original 9 subsets, we exclude 2 sets (Agriculture and Economy) because we find no specialized method can outperform the Naive method, thus they may be unpredictable. We reserved 20% of each dataset the test set. MMD allows for the investigation of models capable of integrating information across different modalities and domains.
The MMSP dataset comprises one and half years of solar power generation records collected from 88 individual plants. We select the first 10 series as in (Ma et al. 2024). Crucially, it includes tempo- rally aligned heterogencous covariates for each plant, consisting of satellite imagery and numerical weather predictions. This dataset provides a challenging real-wor real-world scenario for multi-modal learn- ing, requiring the fusion of visual and numerical information predict power output.
17

---

Published as a conference paper at ICLR 2026
B COMPARED METHODS
In this section, we provide an overview of the baseline methods employed in our experiments, with focus on their methodological frameworks and covariate handling strategies. Each approach is an- alyzed in terms of its integration of homogeneous covariates, highlighting strengths and limitations in modeling external dependencies.
B.1 SPECIALIZED METHOD
PatchTST (Ni et al., 2023) PatchTST is model that transforms time series into patches, which are then encoded using a Transformer to produce forecasts. involves two main components: Patch- ing and Channel Independence. Patching divides time series into subseries-leve patches, serving as input tokens to the Transformer. This preserves local temporal patterns while minimizing compu- tational complexity for attention maps, enabling longer history modeling. Channel Independence ensures that each channel uses the same embedding and Transformer weights, treating multivariate inputs separate but parallel sequences. PatchTST does not incorporate covariate information in forecasting.
NBEATS (Neural Basis Expansion Analysis for Time Series Forecasting) deep learning model designed for univariate time series forecasting. Its core idea decompose the time series into interpretable components, typically trend and seasonality, using stack-based architecture where each stack consists of multiple blocks. Each block learns to forecast portion of the input series using basis expansion functions (approximated by fully connected layers) and subtracts its forecast from the input, passing the residual to the next block stack. This iterative residual learning allows NBEATS to model complex patterns and achieve strong forecasting perfor- mance, often outperforming statistical ans hybrid methods, while also offering some interpretability through its decomposition into basis function.
NBEATS (Orehkknn al., 2019)
NBEATSx (Olivares et al., 2021 An extension of the purely univariate NBEATS model, NBEATSx incorporates covariates by appending them to the backcast and forecast layers in each neural block. It uses a dual-stack architecture (generic interpretable) model both nonlinear dependencies and explicit covariate effects. This makes it effective for scenarios where future co- variates (e.g.. planned events) are known.
DeepAR (alinas et al. 2020) Developed by Amazon, DeepAR is probabilistic RNN-based model that explicitly integrates covariates. Each time step's s dynamic covariates (e.g.. temperature, price) are concatenated with the target variable and fed into the RNN. It assumes covariates known for both training and prediction, making it ideal for applications like demand forecasting where external factors (e.g.. marketing campaigns) drive outcomes.
TFT (Lim et al., 2019) Developed by Google, the Temporal Fusion Transformer (TFT) uses modular design to integrate static covariates (e.g., store IDs), known future inputs (e.g., holidays), and observed variables (e.g., sales). It processes past covariates in the encoder and future covari- ates the decoder via gated recurrent networks and variable selection networks. This hierarchical approach ensures robustness to missing noisy covariates while maintaining interpretability.
TiDE (Das et al. 2023) TiDE (Time-series Dense Encoder) a deep learning model for multi- variate time series forecasting, distinguished by its efficient, MLP-only architecture. It processes the historical lookback window of the target series and any available past covariates through a dense en- coder learn a latent representation. separate MLP-based decoder then uses this representation, along with linearly projected future covariate information, to generate multi-step forecasts.
TimeXer (Wang et al., 2024b) TimeXer a Transformer-based time series model that processes time series as sequences of patches. It uses hierarchical structure with patch embedding, tem- poral encoding, and attention mechanisms to capture both short-term and long-term dependencies. TimeXer handles the covariates by employing the "variate-level embedding". External covariates are embedded and then integrated directly into the patch representations of the target time series. This allows the model to learn how these external factors influence the internal time series dynamics
18

---

Published as a conference paper at ICLR 2026
at the patch level, enabling the model to account for the impact of these exogenous variables in its predictions.
B.2 PRETRAINED METHOD
Moirai (Woo et al., 2024) Moirai, time series foundation model from Salesforce, is engineered for universal forecasting across diverse time series data. its core, Moirai utilizes a Transformer- based architecture and is pre-trained on massive and varied dataset called LOTSA. A key architec- tural component its ability to handle any number and type covariates, both those known in the future and those that are not. Moirai achieves this by conditioning its probabilistic forecast gener- ation, which uses flexible output distribution, on these covariates, allowing the model to produce forecasts that are informed by the provided exogenous variables.
TabPFN-TS, regression variant of the TabPFN(Hollman) al. 2025) model to time series, is foundation model pre-trained on pure artificial datasets, enabling few-shot time series forecasting. When incorporating covariate data, TabPFN-TS typically treats these exogenous variables as additional features. These covariate features are then concatenated with the temporal information before being processed by its Transformer-base is adapted for tabular data. The model's pre-training on tabular data allows it to potentially learn complex relationships between all input features, including the covariates, even with limited time series-specific training.
TabPFN-TS (Hoo et al. 2025
architecture, which
Time-MoE
See section C.4 for the model details.
Chronos-Bolt
See section C.4 for the model details.
TimesFM See section C44 for the model details.
B.3 MULTIMODAL METEHOD
Time-LLM Time-LLM (Jin et al. 2024 is a foundation model that adapts a pre-trained Large Language Model (LLM) for general-purpose time series analysis. Instead of full fine-tuming, it employs a lightweight "reprogramming" layer. This layer transforms input time series into a text- prototype format that the frozen LLM can process. By aligning the LLM'' inherent sequence model- ing capabilities with statistical time series patterns, Time- LLM can perform diverse tasks like fore- casting, classification, and anomaly detection through simple text prompts, leveraging the LLM's reasoning abilities while remaining computationally efficient.
ChatTime ChatTime (Wang et al. 2025) is a multimodal time series foundation model designed single, end-to-end language model that directly processes interleaved sequences of numerical time series data and natural language. It introduces key innovations, including a unified time series tokenizer that represents time series patches discrete tokens and temporal-aware attention mech- anism to effectively capture complex temporal dependencies. By training this mixed- -oddlltty input, Chat'Time can perform diverse analysis tasks like forecasting and classification through con- versational interface, directly interpreting and responding to queries about the provided data. We used the checkpoint ChengsenWang/CCCC Wangg/ChaTTime-117B 1-aee-1-----hheeeeeeee by the authors our experi- ment.
B.4 FINETUNED METHOD
Supervised Fine-Tuning (SFT) the process of adapting a pre-trained Time Series Foundation Model (TSFM) to specific downstream task or dataset by further training it on target-specific labeled data. This allows the model to leverage its general time series understanding learned during
'buts:/hhuggingfacecoo htup:///tngiiggaacc.ooossssooooo http:///www..eeeeeeeeeooninaa...
19

---

Published as a conference paper at ICLR 2026
pre-training and specialize its parameters for improved performance the new, specific time ries. In our implementation, we utilized the target time series without the covariates and adopted the same hyperparameters (c.g., learning rate) and data split UniCA for consistency. During training, the model adjusts its pre-learned weights to better align with the characteristics of the target series, enhancing specialization for the task hand.
B.5 ADAPTER METHOD
Linear Regression (LR) Adapter. In our experiments, we leveraged exogenous variables using linear regression methodology inspired by the approaches of the Chronos Ansari et al. 2024) and TimeFM(Das et al. 2024) time series foundation models. This regressor approach involves decom- posing the target series into two components: contributions from the covariates and the target itself. Initially, we perform a regression the target variable against the known covariates. Subsequently, we subtract the predicted target values from the actual target values to compute the residuals. These residuals serve as the context for the time series foundation models, which forecast future residuals. The ultimate forecasts are obtained by summing the predicted residuals with the target forecasts derived from the covariates.
A notable limitation of the covariate regressor approach is its reliance covariates that are known over the forecasting horizon, such as dynamic categorical and dynamic real covariates. Past covari- ates, including past dynamic categorical and past dynamic real covariates, only available for the context window. To address this limitation, we employ the corresponding TSFM to forecast these past covariates into the horizon, thus extending their utility beyond the context window.
TTM Ekambaram et al. 2024 Tiny Time Mixers (TTM) compact models for multivari- ate time series forecasting. featuring only million parameters. Built on the efficient TSMixer ar- chitecture, TTMs MLPMixer blocks with simple gated attention, offering faster alternative to traditional Transformer self-attention mechanisms. TTMs are pre-trained on diverse, large-scale datasets from Monash and LibCity, encompassing various domains and temporal scales. TTM's ar- chitecture addresses data heterogeneity through innovations such as Adaptive Patching for adjusting patch lengths, Diverse Resolution Sampling for enhancing generalization across resolutions, and Resolution Prefix Tuning for embedding resolution info in training. This approach allows TTMs to excel resource-limited settings by initially training models channel-independently followed by fine-tuning to integrate target and exogenous channel correlations.
C IMPLEMENTATION DETAILS
C.1 CODE AVAILABILITY
Our code has been made anonymous and is available at Inttps://anonymous. 4oper. wcw.eccer//cccccoo http:///nnnn//SII/
C.2 COMPUTE RESOURCE INFORMATION
For all the experiments, we use 4 GeForce RTX 3090. For baselines, we used cpu instances with 40 virtual cpus and 384 GiB of memory. The library requirement for reproducing the results is available on the above repository.
C.3 PREPROCESSING
We mainly follow the series preprocessing pipeline proposed in TFT (Lim et al. 2019). We impute missing values in both the target and covariate series using forward filling and add correspond- ing binary indicator to mark the imputed timesteps. The time features are generated based on the time series frequency (e.g., hour, weekday, month as periodic features). Then, the time features are vertically stacked with known dynamic features to form unified feature matrix. Missing static fea- tures are filled with a default value of zero. Finally, each time series is assigned unique identifier
20

---

Published as a conference paper at ICLR 2026
Table Overview of the baseline models, grouped by type and implementation source. We utilized implementations from the popular time series libraries Gluon'TSS and NeuralForecast, or the official author repository (marked as "Reference)). All experiments were conducted using the default hy- perparameters provided by the respective implementation.
Model
Type
Implementation
PatchTST
Specialized Specialized
GluonTS
NBEATS
GluonTS
DeepAR
Specialized
GluonTS
TFT
Specialized
GluonTS
TiDE
Specialized
GluonTS
NBEATSx
Specialized
NeuralFonecast
TimeXer
Specialized
NeuralForecast
Moirai
Pretrained
Reference
TimesFM
Pretrained
Reference
TabPFN-TS
Pretrained
Reference
TTM-R2
peetranedd/Aaapte
Reference
FusionSF
Multimodal
Reference
Time-LLM
Multimodal
Reference
ChatTime
Multimodal
Reference
distinguish it multi-series forecasting. This ensures that the resulting data format meets the input requirements of deep learning models, providing normalized representation for time series forecasting.
C.4 ARCHITECTURE OF TSFM
In this section, we detail the tokenization, encoding, and decoding procedures of two time series foundation models: Chronos and TimesFM. These decomposition steps provide a clearer under- standing of their internal mechanisms and differences.
Chronos-Bolt (Ansari et al. 2024). Chronos-Boli adopts an -ecoddrr architecture based on T5. During tokenization, the input time series undergoes instance normalization. It is then seg- mented into patches along with mask, and the two streams are concatenated before embedding. An optional [REG] token can be added to support regression-style outputs. The encoder transforms the tokenized inputs via a stack T5 encoders, generating contextualized hidden states. These are fed to the decoder, which performs sequence generation conditioned on attention masks, and yields multiple quantile forecasts. Extended prediction lengths are handled through decoding extrapolation. All outputs rescaled back using stored normalization parameters.
TimesFM utilizes Transformer decoder-only architecture. The tokenization stage includes preprocessing, fixed-length patching, and normalization via mean and standard deviation. Patches are augmented with mask features and projected into embedding space, optionally with positional encodings. The encoder applies multi-layer self-attention to obtain con- textual representations. The decoder operates in an auto-regressive manner, iteratively generating future values. Outputs include both mean and quantile predictions, which are de-normalized to store the original scale. TimesFM also supports frequency-ban based conditioning and modeling for improved multi-scale forecasting.
TimesFM (Das al. 2024..
Time-MoE (Shi et al. 2024 Time-MoE the first billion-scale time-series foundation model that marries decoder-only Transformer with sparse mixture-of-experts (MoE) backbone to boost
'httss//hugginnfaccccom http:///wooooliinnni///000000000 https:///gttuh.cmmimmmmmeeeiiiee
21

---

Published as a conference paper at ICLR 2026
capacity without proportional inference cost. Each Transformer block replaces the dense feed- forward layer with a shared pool of eight experts, and a learned router sparsely activates just two experts per token, while rotary positional embeddings and RMSNorm enhance stability. The authors pre-train three variants- -oie-TTm------aaee large (200 M 453 M) and Time-MoF-ultra (1.1 1.1 B activated 2.4 B total)-all all support channel- independent forecasting for arbitrary horizons via a multi-resolution head. Training uses Huber loss and the newly curated Time-300B corpus: se 48 M sequences and 300 billion time points drawn from nine domains.
(50 M activated / 113 M total parameters), Time-Mofs
Table 5: Comparison between Chronos- Bolt, TimesFM and Time-MoB terms of model architec- ture and processing steps.
Component
Chronos-Bolt
TimesFM
Time-MoE
Architecture
TS-based encoder-decoder
Decoder-only
Decoder-only
Tokenizer
Patch-based Residual MLP
Patch-based Residual MLP
Point-based Gated Linear
Encoder
encoder stack
Custom Transformer
Custom Transformer
Predictor
TS decoder
Residual MLP
Linear
Prediction Output
Multiple quantile predictions
Mean and quantile predictions
Point predictions
C.5 ALGORITHM OF UNICA
We summarize the entire forecasting process of UniCA in Algorithm 1 The procedure begins with Covariate Homogenization, where heterogeneous covariates such as categories, images, or text are first encoded into dense features and then mapped into latent homogeneous space via the Co- variate Homogenizer (CH). These transformed covariates are then temporally aligned and concate- nated with any observed homogeneous covariates form the unified covariate sequence. Next, in the Pre-Fite Fre-Fusion Module, we integrate historical covariate information into the tokenized target se- quence using a conditional global attention mechanism followed by a gating unit. This enriched quence is passed to the pretrained encoder of the TSFM to extract temporal patterns, After encoding, the Post-Fusion Module incorporates future-known covariates using another attention-based fusion mechanism, allowing the model to dynamically select complementary covariate signals. Finally, the predictor the TSFM generates workflow enables UniCA to flexibly and effectively adapt general-purpose TSFMs to covariate-rich forecasting scenarios, while preserving the pretrained temporal modeling capabilities.
future forecasts from the fused representation. This modular
C.6 OPTIMIZATION
We train our models using the Adam optimizer. The initial learning rate is set to a fixed value and subject to adjustment via ReduceLROnP lateau learning rate scheduler. The scheduler monitors validation loss (if available) or training loss and reduces the learning rate by a factor of 0.5 when the monitored metric has not improved for specified number of epochs (patience). No weight decay is applied to regularize the model.
To prevent overfitting, early stopping is implemented based validation loss. Training proceeds in mini-batches, with each epoch comprising 50 gradient steps. Model checkpoints are saved corre- sponding to the epoch that yields the best validation performance.
C.7 HYPERPARAMTERS
For all experiments, we search the hyperparameters listed in table
22

---

Published as a conference paper at ICLR 2026
Algorithm UniCA: Unified Covariate Adaptation for TSFM
Require: Target series YI.T, static covariates S, dynamic covariates
CITTH
(Ci.T,C+4..T CT+1.T+H1 pretrained TSFM (T, P)
Ensure: Forecast Yr+I:T+
1: Covariate Homogenization:
2: for each heterogeneous covariate do
3: Encode modality to dense feature (hett)
4: Convert to homogeneous covariate via covariate homogenizer C(hee) CH(H)(heet)
5: end for
6: Concatenate all homogeneous covariates: C+ +C,
--C,C(hat) C(het))
7: Pre-Fusion Module:
8: Tokenize past target: Z -
9: Tokenize past covariates: ECEr T(Ci.))
10: Embed static covariates: Es = =SS
Compute conditional attention: Z CondAttnPool(Ee, (Errr Es)
Fuse covariates with GLU: Z Z + GLU(Zc,r)
13: Temporal Encoding:
14: Encode fused sequence: H = E())
Post-Fusion Module:
16: Tokenize future covariates: ECry1744 T ((CT+11++)
17: Compute conditional attention: ZCTIIT+* = CondAttnooll (Err-Ir- Es)
18: Fuse via self-attention: [H 2Cr SelfAttn((H)) HH, Zor ir nl=
Forecasting:
20: Predict target: YT+1T+H = P(H)
Table 6: Key hyperparameters, their search spaces, fixed values used in training UniCA across all datasets.
Hyperparameter
Value / Range
Description
Learning rate
(1e-3,1e-4,ee-5,1666
Initial learning rate for Adam opti- mizer
Weight decay
(le-2,1e-..lc-6)
L2 regularization weight Epochs to wait before reducing LR
Scheduler patience
Scheduler factor
0.5
Multiplicative factor for LR reduction
Batch size
{8,1,,3,
Number of samples per training batch
Max epochs
100
Maximum number of training epochs
Early stopping patience
10
Epochs to wait for improvement be- fore stopping
Context length
TSFM-specific
Length of input window for encoder
Prediction length
Dataset-specific
Length of prediction window
Embedding dimension
TSFM-specific
Dimension of input embeddings
Homogenizatoin dimension
(1.2.4.8,16)
Dimension of homogenized series of each heterogeneous covariate
D EXPERIMENT DETAILS
D.1 TRAIN-TEST SPLITTING.
The train-tes follows the setups in (Aksu et al. 2024). For datasets with large number of series, Le. M5 Makridakis et al. 2022) and Retail (Lim al. 2019), spare the last "prediction length"
23

---

Published as a conference paper at ICLR 2026
of each series for test and all the observed points before the test points used for training. For other datasets, we partition 10% of the data as the test set and apply a sliding window approach for evaluation, with step size of (Zhou et al. 2021). Among the training points, also split the validation points.
D.2 EVALUATION METRICS
We use four metrics evaluate performance of forecasters; Mean Absolute Error (MAE), Mean Square Error (MSE), Mcan Absolute Percentage Error (MAPE) for point forecasting ability, and Continuous Ranked Probability Score (CRPS) for probabilistic forecasting. For all metrics, we use Gluon'TS library implementation to calculate final values (Alexandrov et al. 2020).
The Mean Absolute Error (MAB) is a commonly used evaluation metric time series fore- casting that measures the average magnitude of errors between predicted and actual values, without considcring their direction. It is defined as:
MAE
1 MAE
Σ t=1
Yr
Prl.
where:
Yg is the ground truth value at time step f,
Y, is the predicted value time step is the total number of observations.
MAE is scale-dependent and expresses the error in the same units as the data, making directly interpretable. It is robust outliers compared to squared-error metrics, but does penalize large errors as heavily as MSE.
MSE
The Mean Squared Error (MSE) quantifies the average of the squared differences between predicted and actual values. It is defined as:
1 (Y, -2)², MSE
[ml
where:
Y, is the actual value at time
Y, is the forecasted value at time t,
72 is the number of observations.
MSE penalizes larger errors more severely due to the squaring operation, which makes it particularly sensitive to outliers. Like MAE, MSE is also scale-dependent, and it is widely used in regression and forecasting tasks due to mathematical properties that facilitate optimization.
МАРE MAPE is an evaluation metric used to measure the accuracy of forecasts in time series analysis. defined as the mean the absolute percentage differences between the actual values Y, and the predicted values Y, The formula for MAPE is:
Y
Y
MAPE
Y
t=1
where:
Y, is the actual value at time
24

---

Published as a conference paper at ICLR 2026
Y2 is the forecasted value time t, 7t is the number of observations.
This metric expresses the forecast error as percentage of the actual values, making it scale- independent and easy interpret. However, it is sensitive to values of Y, that are zero or close to zero, as this can lead to division by zero inflated error percentages.
CRPS The Continuous Ranked Probability Score (CRPS) is metric used in probabilistic forecast- ing evaluate the accuracy of predicted cumulative distribution functions (CDFs) against observed values. Given predicted distribution with CDF F and ground truth value the CRPS is defined as:
CRPS(F. y) = 2Fa da,
where the quantile loss Aa(1. y) is defined as:
(g = (a (v) g).
In practice, computing the CRPS integral can be computationally intensive. To address this, we finite set of quantile levels. This approximation,
approximate the CRPS using discrete often referred to as the mean weighted quantile loss (Park et al. 2022), given by:
K
CRPS RJ
w
QL/aa),
K k=1
where K is the number of quantile levels, and {a), ... (e.g., dg = 0.1k for = 1, 2,2..... when = 9).
.ak are the selected quantile levels
The weighted quantile loss wQL[ai for each quantile level is calculated as:
wQL = Ee Aa (ác(a).
E, 1V6
where:
T (a) is the predicted a-quantile at time step
Ve the actual observed value at time
Ao (āc(a), yr) is the quantile loss at time for quantile level a.
E FULL RESULTS
E. ROBUSTNESS EVALUATION VIA ERROR BARS.
To evaluate the robustness of UniCA under different random seeds, we conduct experiments with seed values (41, 42, 43, 44, 45) on all uni-modal datasets and report the average performance along with 1-sigma standard deviation (mean std), as shown in Table 7 Specifically, "C. (UniCA)" and "T. (UniCA)" denote the proposed UniCA framework built on top of Chronos Bolt and TimesFM, respectively.
The results demonstrate that UniCA consistently achieves strong performance with low variance across all metrics, indicating its robustness against random initialization. For example, on the GFC17 dataset, the MSE is 0.096 0.003 for Chronos-bais based UniCA and 0.094 0.004 for TimesFM--base UniCA, showcasing both accuracy and stability. This pattern holds across other datasets, supporting the statistical reliability and generalization ability of UniCA regardless of the underlying backbone.
25

---

Published as a conference paper at ICLR 2026
Table 7: Error bar results of UniCA on uni-modal datasets. "C." and "T." indicate UniCA instantiated with Chronos-Boo Bolt and TimesFM, respectively. All results are averaged over five runs with random seeds (41, 42, 43, 44, 45) and are reported as mean standard deviation (1-sigma).
Averaa
Bell
Cadiitee
CONIDe
EFF
GECI
GFC14
GFCI
XF
pub
Ritall
Sysls
Avcrage
01477232011
0:144250000
BAS4SRURG
DASPEROTO
6:44 2002200
CHIBSEURC
8071 -009
6.79
MAPE
655--09
033840H00
0555-2917
0.33446663
MSSE
033220000
036520IRG
0 639225380
Avvrage
000 0001
012.000
060310000
1.50
6512-n00
030610000
053610000
10100
0.23
8
MAPS
(1314221/002
0511-30011
066351000
036422000
C4023000
0:3220000
032855001
033935000
B 848355021
CIRPS
(1415-00:30
0600--099
0070 -0311
0177-0999
03906-0001
0351-00011
0:587:0005
07164000
0:15--000
116-
0.3330006
60000
0 (85500000
Chronos-Bolt (UNCCA)
3.08
(UNICA)
ClerrowBull (SF))
[ireeFM (25)
4/2
6.75
Prrmaine
retuned
Time (UHCAA)
Choomssfice (6)
5.133
6,000
Finatamac
Choomo-Bol (25)
4.00
Cleiron-fad (FT
6417
ChronssFoll Rol (10)
7.50
Chonoy-uul Bull 1U0
8.003
Cheoran
7.43
Times (SFT)
0.367
TimesFM
0.83
Model
11,417
PatchTST
NBEAT5X
TepPFN-75
12:42
Chroneck
L1-750
TFT
12.75
12:007
Naim
TasPFN-75
11.250
DeepAR
15.17
DecphR
14
15:47
TDD
15.250
15.333
Time-Mtt (JNCCA)
Timpkar
Time McE (25)
22.25
Time HoE (25)
17:000
MAE Rank
MSE Rank
Chrumos- Berrss-Boitta (UnCA)
Spec abrec F Fawetnedd
ChronssBook
Cleuros-Bul (SF))
6,077
Tima FM (UnCCA
Probial not Finetuned
Chroncs- (25)
[i (25)
5.00
5,500
Aduçto
TN
550077
Rol [1R)
Chronos (D0
ChronosX
9.300
Chronos-Boll Rot (SFT)
8.75
TimeeFMM
9.003
Model
PatchTST
66667
'FT
11.563
Time-Mof Moll
11,08313
11.033
TFT
11.5
12,003
11.99667
PatchiST
12:347
Timpkor Timm-Mot (UnICA)
14.31133
DocpAR
13.585
14.56533
mimker
14397
TIDE
53555
Thme (25)
TimesFM
Mime (UNICA)
17:809
C Chronos Soft (Un(CA)
16.600
MAPE Rank
CRPS Rank
Figure 6: Metrics rank on uni-modal covariate-awart datasets.
E.2 UNIMODAL FORECASTING
Main results. To provide a comprehensive comparison across all baseline and proposed methods, we report the detailed forecasting results on the 12 unimodal covariate-aware datasets in Table g This includes performance across four common evaluation metries: MAE, MAPE, MSE, and CRPS. The models compared include traditional baselines (e.g., NBEATS, DeepAR, TFT), pretrained foun- dation models (e.g., TimesFM, Chronos), and our proposed adaptation strategies (UniCA, SFT, LR, 7S). We average each metric across all datasets and report both the raw values and metric-wise ranks.
To further illustrate the relative performance of each method, Figure C shows the average rank of each model across all datasets under each metric. Lower rank indicates better performance. As shown, our method Chronos-Bolt (UniCA) consistently outperforms all others in all four evalua- tion metrics. Notably, both Chronos and TimesFM, when adapted using UniCA, achieve significant improvements over their zero-shot and fine-tuned variants.
EPF subsets. The Electricity Price Forecasting (EPF) dataset is often treated as a collection of distinct sub-datasets representing different markets: BE, DE, FR. NP, and PJM. These subsets exhibit diverse distributional characteristics and covariate dynamics. Table figure/7presce the average performance as well the relative performance, including MAE, MAPE, MSE, and CRPS.
26

---

Published as a conference paper at ICLR 2026
Table 8: Forecasting results on 12 unimodal covariate datasets.
PaidiTST
DeqAR
1F7
10E
NELATS:
Tineker
TTM
Mesicl
TAOPEN-TS
Chreeeo
Chessns (Bek)
TiaesEM
Time-Mell
Arg. MAPK
0.56
0.2m
(1343
01:31
(.58)
0.527
0.53
0.54
0.510
0.587
0.5%
0.556
(199)
B6-9
01774
11943
0U500
0.511
0.56
0.966
0.53
0.54
0,902
R916
3.918
CRPS
0.50
53223
(149)
IUST
0L325
0.835
0355
01495
0445
0646
[641
0.40
0426
6,42
0421
0.305
01465
0643
0387
0353
0.75%
0.166
0.70
0.79
0.57
01301
0.77)
0.50
0773
0.39
0.70
0.766
0.13
0.714
0.79
0300
0348
MAA
[U6]
0.866
01:48
0.60
0.87
0.35
0.81
0.81
0.37
0.51
1006
MISE
2236
(U99)
1076
0277
01732
0.301
01725
0U581
85233
0344
6713
670
0/20
0.25
1126
0333
$183
COVIDW KodkaN
Avergge
23:66
0777
0.66
0098
1.330
0.533
01.35
0.80
0301
0353
055
1:83
122
1980
5.455
MARE
5353
0.00
0.61
0006
1.400
0.11
01311
0036
0.6
5300
0363
0225
0800
1354
001
5300
3331
CRPS
5414
0.79
0.90
1.456
1.155
0.337
0417
0451
0351
1.255
030
6.365
0304
180
1186
1060
1.03
0,77
110L
MAP
5211
0.5
0.38
0.29
0.31
0.33
01377
0315
035
6113
8.71
0200
011
0337
1307
MD
D304
3334
MISE
6091
0090
0.03
0182
0054
0022
6081
0561
0020
1364
$354
CRPS
2228
0125
0.325
0.255
0.20
029
626
5186
0.27
926
611
9211
025
2303
0.826
0.33
11574
037
5464
0200
D206
0948
5700
EYP
MAPE
6.50
0.897
6.71
037
033
000
6755
0397
085
0801
0555
11000
0603
1060
100
027
CRPS
IL5-5
0299
0.366
0.439
1.215
130
0350
0.36
0300
0331
0311
1300
1394
5.56
GRCIL
MAP
0360
0.96
0.9
0.655
0.61
055)
0.85
015
013
056
6.66
6.31
0511
03W
600
554
MIS
0.545
0,61
(1340
0.3
8348
0100
0323
0311
0354
1317
5670
5351
CRPS
0.551
5.04
0.5-
1/96
0U5T
0.65
0715
01545
0U56
531
0.92
0.56
6-87
6401
0117
(12
1,96
D379
5382
839
1318
0.31
0.76
0.368
0.14
033
0.31
635
0131
030
6306
035
013
135
0331
5:10
5:33
OFOH
MAPE
0449
048
(.48)
0440)
631
B411
0481
0425
0201
8431
1225
1463
1.455
MSE
0181
0.96
0212
0.06
0.35
0229
0.335
0322
0001
5.177
0.89
0130
016e
0119
1311
0/66
2164
134
GECP
MAE
0,30
0339
0.355
0154
8,50
8393
0500
620
025
035
1012
0211
MARE
R1N
0.76
619
0125
0.331
0131
0.27
1.54
0114
027
011
110
0313
0313
CRPS
3.98
0.70
0365
01345
0.51
03355
0.00
0:91
0345
635
(.9)
0754
0714
0554
5574
Aneras
0L700
0,59
0551
0,50
1,732
0.55
0.575
0.55
0.90
6:71
0395
0530
671e
020
0355
0350
1241
196
MAR
1151
[.91
0.53
0.87
0.72
035
0206
034)
1180
1.50
5.0%
MSE.
0.0
0.8
0463
(15)
n.b
130
0311
0917
0211
0311
1121
1.5
1541
0014
1127
062
01540
0AD
063
5.6h
6002
1229
133
0311
0914
5531
MAPE:
2334
10155
0075
1000
01722
8.54
01955
0.011
1051
0351
0251
1764
0231
011
1119
1051
514)
CRPS
5.44
0.18
0.6
0.590
0.503
0.18
8/5
046
6/54
0.155
0.55
(/81
011
1056
005
5606
5161
1.5M
5.41
03722
0.5%
0.5m
11:38
0.35)
6184
0329
052
0000
0355
1359
118
5126
5342
MAP
5246
0.30
0036
0056
0.377
0.382
11355
0221
0387
0911
0224
1155
6251
1:81
5.13
MSSI
8115
51100
0130
0122
000
01554
0164
047
0285
666
0010
0081
076
0440
5762
Aeg
1000
2:15
0.85
0,00
0715
0.756
0.54
(1390
0.06
177
D161
0164
0,96
5663
0.12
1771
1.26
5355
MAPE
5195
0.76
0251
0176
0369
0150)
6256
5/84
0131
0636
D/81
0341
0.76
164
1160
3171
CRPS
0.299
0.22
0.314
0,20
0.590
0.54
0.291
0.39
0481
0.40
0.25
0,50
6,514
0354
0,366
191
0.00
(1391
Span
MAA
2404
0432
0,255
0453
0413
0411
0.36
031
0356
8313
026
025
0.64
0111
0.36
1336
5340
3335
MIEE
2:18
0315
016
0.35
1199
0266
0196
0201
000
8.89
0124
000
0.15
0.89
0133
01c1
5965
5301
CRPS
2418
0.41
0415
0.36
0.66
0346
2311
0223
038)
0.23
0.50
6809
0355
27

---

Published as a conference paper at ICLR 2026
Chronos- Bolt (UNICA)
TimeFMM
10
24
HAb Fank
Chronos- Bolt (UNCC)
Chemnosflltt (SFT)
22
M
S 2nd
Chemrenshutt55 (5F))
12
(UnCAM(UHHAA)
Finduuios
ChronosBol (25)
Choons-fic (25
4
TimsoFF (25)
54
TiresFM (U))
TimesFFt [LR]
60
Chronas Bok (U0
52
ChronssBoot [UR)
TimesfM (25)
Model
Times
MREATS
7A
Mode
Times 14 (SFT)
102
PatchTS
1122
PatchTST
11.0
Maral
NBEATS
NBE4T5
mekeer
DecpAR
DeepAR
Timeker
MAE
MSE
Chronos Minos-Bolt/Unne (UniCA)
TimeFM (UniCA)
Soeciaion and
Chronos diolt (UnICA)
TimesFM (UNICA)
26
Sapcilood ool
Chrones (SF()FF))
4.6
Frietnec Adouter
TimesfMt (25)
26
Freeuned
Chronos Bok (25)
Chronss-Soc Solk (25)
4.4
Adogttr
TimesFM (25)
5.0
Chroris (Srroc-bt((FF)
4.6
Chrones Bok [LR]
Chrones Bot: (LR)
TimesFkl (LR]
6-5
TimesFM (LR)
64
Model
TM
NBEATSX
Model
TMmesFM (SFT)
30.0
TapPFN-TS
NBEATS
PatchTST
PatchTST
TTM
Tineker
Mairal
TimeKer DeepAR
16.6
DespAR
NRTATS
MAPE Rank
CRPS
Figure 7: Metrics rank on EPF subsets.
We compare traditional deep forecasting models, pretrained time series foundation models (e.g.. TimesFM and Chronos), and their adapted variants using zero-shot (ZS), supervised fine-tuning (SFT), linear adaptation (LR), and our unified covariate adapter (UniCA).
Across all EPF subsets, the UniCA adaptation strategy consistently delivers competitive or leading performance, especially when applied to pretrained models like Chronos and TimesFM. Notably, TimesFM (UniCA) and Chronos (UniCA) show particularly strong results on BE, PJM, and DE, indicating effective covariate adaptation and transferability across markets.
Table 9: Forecasting results on subsets of EPF.
PuccTST
NBEATS
TiacKer
Moirai
TTM
TAIPEN-TS
75
SFT
TR
UBICA
75
STT
1.R
UsICA
NEAK
(L:59
11771
11355
11.96
0.61
11596
11358
11355
0.46)
11586
1447
0.480
1446
0.30%
11671
11458
MAPE::
0,573
1L366
1U3001
11363
2850
0339
0404
01394
01391
0.50)
0.96
0408
6.76
0,50
0,312
01400
11487
MSL.
D4333
01335
01422
5415
01356
0.881
0.644
0.407
04427
0337
IL348
11348
01382
01358
ANsTagc
D415
0.570
0.625
04422
0444
0.421
0.565
0.45
0.440
0.40
0:351
0.37
0355
0.356
03333
0.407
0.36
0.356
0.56
0.54n
0.530
0.497
035
0.400
0
MSE
B413
0.35
0.558
0.475
0.475
0.46
0.511
0.472
0.42
0.400
0.363
0.3
0355
0.36
0.355
0.45
0.36
0.5%
CRPS
0.342
0.401
0.300
0.25
0.293
0.257
0.323
0.267
0.29
0.242
0.214
0.16
0211
0214
0315
0.25
0217
0.10
0.30
0.58
0.500
0.544
0.51
0.500
0.666
0.5
0555
0541
0355
0.51
0.58
05%
SAAE
DA50
11%
0.65
0669
5,500
0.500
0.77
0.60
0.500
D.679
0/01
0.50
0590
0366
D567
0.50
0.57
053
MAPE
D361
6226
0360
0856
0776
07766
0448
0754
0875
0.11
015
0724
0205
8710
075
0.33
070
CRINE
1.305
0351
0.566
[361
0517
Anveage
01725
0.475
244
0.414
0.964
0.:05
11456
8.34
L150
IL164
0.30
03314
MAPE:
BA50
0126
1148
2.54
(164)
0654
0642
1466
0.55
036
0352
1156
0.3%
033
0362
MSE
01880
01555
2475
01477
0.521
01882
1.426
04100
8388
0380
840
0.41
0.6
CHPS
03325
01481
0155
01327
0.481
0.81
0.405
0.27
021
6115
0121
61mh
0:544
022
0.73
0.304
0.50
0,500
01543
0652
0.69%
0,641
0.70
0.54.
0632
0.05
0.07
0529
6.4%
0511
0.554
0551
0.50
MAE
0.755
0.588
0.912
0.581
0650
0.58
0.82
0750
0.60
0.73
033
055
0ST2
0.586
0546
0,04
0.55
0.28
0.781
0.71
0716
0.14
036
CRPS
B655
0.622
0.792
0.50
2.556
0.474
0.577
0.25
0.68
0.50
0.45
0.72
0462
0.445
0460
0313
0462
045
Average
0.46L
0.00
0.685
0412
0.401
0.40
0353
0354
0.30
03
036
MAPE
03363
0.001
0.80
0371
3.397
0364
0.459
0.333
01385
0.313
0.313
1300
0337
6331
0333
0.361
033
MIS::
0181
0335
0:57
0708
0/300
1.179
2.05
0310
191
0315
0166
015
6157
6330
8163
01175
196
E.3 MULTI-MODAL FORECASTING
Time-MMD (TS-Text). Table 10 reports the full forecasting results on the Time-MMD bench- mark across seven domains: Climate, Energy, Environment, Health, Security, SocialGood, and Traf-
28

---

Published as a conference paper at ICLR 2026
fic. We compare UniCA with strong baselines, including classic models (e.g., DeepAR, TFT), cent pretrained models (e.g., Chronos-Boll TimesFM), and domain-specific models (e.g. Time- LLM, TTM, Moirai). Metries include Mean Absolute Error (MAE), Mean Absolute Percentage Error (MAPE), Mean Squared Error (MSE), and Continuous Ranked Probability Score (CRPS).
UniCA consistently achieves the best competitive performance across most domains and met- rics, demonstrating its versatility and scalability in handling heterogeneous covariates under diverse multimodal forecasting scenarios.
Table 10: Full forecasting results on TiMe-MMD.
NBCCTT
Degah
TDC
Tae-LM
ITM
Nozal
ADUISE
Clutkea
Cleom (444
Time XOXK
14T
391
UsEUN
113.31
13383
1071
0151
2:413
1321
3.05
E35
1021
1311
1387
E11
1131
13113
11351
(1313
(4711
03776
3385
1,42
1834
0355
076
1461
E711
11:9
8.45
1351
0.54
(111
0636
1.56
0.77
0.681
1.44
C315
10990
03855
1.300
3881
0866
8.07
8.23
5205
1567
1.20
44733
8.70
8.11
0.81
(791
0345
1.20
(144
0.45
0.57
MAR
1799
3386
1345
5.65
3103
0.00
1005
0668
130
8431
0631
0.11
0.15
2.11
0114
01681
1575
3180
0915
0.80
0.88
0.86
1184
1.88
01277
3.51
157
0.87
0.61
0121
1.80
0.83
0111
MAREE
1.000
0.333
1.01
3005
0881
0.03
0.87
1/80
1334
1520
0070
8.80
3553
0.6
016
0006
1.00
1330
0.35
CRPS
1/:00
17585
7.60
1.05
140
1190
0.00
0.30
0609
1.04
1271
0994
110
MAN Tam Innieeed
MAKK
0165
0771
0650
0626
0544
0.80
143
1.5
1467
8.35
0.15
300
055
1.45
031
5.11
0315
E35
8.50
1365
1961
06.1
630
1997
146
0981
2315
1108
1114
1154
[00
1991
110%
8716
6.59
6981
637
(19%
171
4791
197
1865
CRPS
1089
1.31
3099
066
1319
1001
100
100
1.56
008
$200
636
0.86
6415
0.71
1855
180
1480
1336
180
1:40
119
E31
06.
3.10
390
033
0.63
1955
1100
1.00
1800
1800
1004
196
8.10
4333
6104
1081
196
013
0.51
1.6
3.97
1118
0.75
0701
1231
3.55
1.8
1110
1947
00-1
635
0136
2.04
1314
0912
630
K716
03/3
LIPS
1434
1.55
1.50
1.41
4956
1781
194
1.55
0.40
0351
133
106
186
6.48
MAPS
05.
3.00
03.5
5.08
3007
1000
0030
8.45
8.0
65%
0.61
133
0.00
1815
0186
C3P%
1180
1171
6163
0.95
1850
175
0.60
6.79
0911
640
0669
1.01
046
0.24
09771
E.4 IMPUTATION
UniCA is general covariate adaptation framework for time-series foundation models (TSFMs). Its applicability is therefore not limited to Vorecasting-Uniie -nnCCA can be attached to any downstream task that the underlying TSFM supports. provided the task is covariate-aware.
To demonstrate this, we evaluate UniCA on the imputation setting of the MOMENT TSFM Gossami al. (2024), which natively supports forecasting, classification, anomaly detec- tion, and imputation. Among these tasks, only imputation naturally involves multivariate inputs or covariates, making the most appropriate benchmark for UniCA.
For each dataset (ETTh1, ETTh2, ETTml, ETTm2, Electricity, Weather), we treat the OT column the target variable and use all remaining variables as past dynamic real covariates. Each dataset is split with a 6:2:2 ratio. Following the MOMENT imputation setup, on the test split we extract sliding windows of length 512 and randomly mask patches of length 8 using MOMENT's patch-based masking module at mask ratios 12.5% 25%, 37. 37,5% 50%. MOMENT is loaded in "reconstruction" mode and is never munet its RevIN normalizer and reconstruction head remain fixed throughout.
UniCA operates on top of the frozen MOMENT encoder: the target context is tokenized by MO- MENT; all covariates are homogenized and fused through UniCA's gated residual and attention modules; and the fused representation is passed directly into MOMENT's frozen reconstruction head. During training, we optimize only UniCA's parameters to minimize squared error on the masked target positions. MOMENT's weights remain completely frozen. At evaluation time, compare zero-shot MOMENT and MOMENT+UniCA using the same patch masks and report MSE and MAE over the masked entries only. Results are shown in Table T2
Across all six datasets and all four mask ratios, UniCA consistently improves imputation over zero-shot MOMENT model. Averaged over mask ratios, UniCA reduces MSE by roughly 33-35% on ETTh//EThh,, about 60-85% on ETTm//ETTmb 45% on electricity, and 40% on weather, with corresponding MAE reductions of about 20-25% (ETTh), 40-65% (ETTm), 25% (electric-
29

---

Published as a conference paper at ICLR 2026
Table Forecasting performance on MMSP (TS-Image multimodal) dataset.
I3INT
1FT
(-CTE)
Mastral
3MM-T3P
Chrense (Bal)
Tlaws/M
Time-MOL
UTICA
75
SFT
USICA
Average
8.177
0:112
0.390
0.345
0.29
NIAII:
16533
04531
0104
16311
1402
8306
18899
0:31
Data
0.831
14019
0360
08:83
I699
0775
0.337
U3RP5
0.370
0.214
3.18
0.337
3:30
0.20
1.333
0.482
0335
0.001
0.27
0.233
0225
Table 12: Imputation performance of zero-shot MOMENT and UniCA-adapted MOMENT on six ETT/clecriciiy//eaatee benchmarks under four random patch-masking ratios (12.5%, 25%, 37.5%, 50%). Reported metrics are MSE and MAE on the masked target entries, together with percentage- point improvements of UniCA over the MOMENT baseline (negative values indicate lower eтrоr with UniCA).
MSE
MAE
Dataset
Mask Ratio
MOMENT
UniCA
A(%)
MOMENT
UniCA
A A(%)
12.5%
0.016
0.012
-23.9%
0.093
0.079
-14.5%
25.0%
0.022
0.016
-27.9%
0.090
-18.4%
ETTh1
37.5%
0,030
0.019
-35.4%
0.128
0.100
-21.6%
0.025
Averago
0.027
0.018
-33.3%
0.121
0.096
-20.4%
12.5%
0.042
0.026
-37.8%
0.150
0.108
-27.6%
0.039
-27.3%
ETN2
37.5%
0.086
0.059
31.8%
0.212
0.154
-27.0%
Average
0.079
0.052
-34.5%
0.202
0.145
-28.2%
12.5%
0.007
0.003
48.44%
0.057
0.039
-31.3%
-37.5%
ETTml
37.5%
0.015
0.005
62.9%
0.083
0.048
42.1%
0.007
-65.4%
Average
0.013
0.005
61.1%
0.077
0.046
40.2%
12.5%
0.015
0.002
83.11%
0.080
0.032
-59.6%
ETTm2
37.5%
0.051
0.007
86.2%%
0,148
0.051
-65.6 66%
50.0%
0.082
0.014
-83.3%
0.068
Average
0.044
0.007
84.4%
0.133
0.048
-63.9%
12,5%
0.124
0.098
20.6%%
0.258
0.234
99.3%
0.159
electricity
37.5%
0.232
0.127
-45.2%
0.351
0,266
-24.2%
50.0%
0.376
0.151
59.9%
0.458
0.288
-37.0%
Average
0.223
0.122
-45.4%
0.339
0.259
-23.6%
12,5%
0.000
0.000
-1955%
0.008
0.007
-10.6%
25.0%
0.000
0.000
-35.0%
-19.4%
weather
37.5%
0.000
0.000
-43.9%
0.012
0.008
-26.8%
50.0%
0.000
0.000
-5.22%
0.014
0.010
-30.2%
Average
0.000
0.000
-39.5%
0.011
0.008
-23.3%
30

---

Published as a conference paper at ICLR 2026
ity), and 23% (weather). The gains monotonic in the mask ratio: UniCA's relative improvement smallest at 12.5% masking and largest at 50% masking on every dataset, showing that covariate- aware adaptation becomes increasingly beneficial as the imputation problem becomes harder.
F SHOWCASES
In this section, we present a detailed case study to demonstrate the effectiveness of our proposed UniCA framework. We analyze the feature attention affinity, important covariates identified by our model, and compare the prediction performance with and without UniCA adaptation.
illustrate our analysis on two time series samples for each dataset, labeled (a) and (b). The visualization is organized into three components for each sample: Important Features Visualization (top), Prediction Comparison (middle), and Feature Weights Visualization (bottom). The Important Features Visualization reveals how our model identifies and leverages key covari- ates during prediction. figure X (a), covariate 18 demonstrates high importance (0.8323) while covariate 7 shows minimal contribution (0.0125). Similarly, in sample (b), covariate 18 maintains high importance (0.8586) while covariate 24 has low importance (0.0077). This selective attention mechanism enables UniCA to focus the most relevant covariates for each specific forecasting task, effectively filtering out noise from less informative features.
Figure 8 TO
The Prediction Comparison clearly demonstrates the superior performance of UniCA- mod- els compared to their non-adapted counterparts. The middle rows show predictions without UniCA adaptation, while the lower rows display predictions with UniCA adaptation. Both models generate prediction intervals (10%-90%), but the UniCA-adapted model produces forecasts that align more closely with the ground truth values. Notably, the predictions with UniCA show better alignment with the temporal patterns and magnitude of the ground truth, particularly in the forecast horizon (the shaded area after the vertical dotted line).
The Feature Weights Visualization at the bottom provides insight into how attention is distributed across different feature dimensions and sequence positions. The heatmaps reveal that certain feature dimensions consistently receive higher attention weights (shown in brighter yellow), indicating their greater influence on the final prediction. These patterns vary between samples (a) and (b), highlight- ing UniCA's ability to adapt dynamically to different time series characteristics.
Our case study demonstrates that UniCA effectively identifies important covariates through its tention mechanism and significantly improves prediction accuracy by incorporating this covariate information. The comparison between adapted and non-adapted models confirms that UniCA suc- cessfully bridges TSFMs with covariate-aware forecasting while preserving the foundation model's generalization capabilities. Furthermore, the feature weight visualizations provide interpretability insights, showing which dimensions and temporal positions are most influential for specific fore- casting tasks.
G MORE ABLATION
G.1 FUSION POSITION.
To further understand the impact of fusion positions in integrating covariate information, we con- duct an ablation study by varying where the past and future covariates are fused within the TSFM pipeline. Specifically, we compare four fusion strategies: Pre-Pre, Post-Pre, Post-Post, and Pre-Post (our default setting). These denote whether past and future covariates are fused before (Pre) or after (Past) the temporal encoder.
As shown in Figure 12 the results indicate that the choice of fusion position has a relatively minor impact on the overall forecasting performance. On TimesFM, all variants achieve similar perfor- mance, with the aggregated error ranging from 0.472 to 0.476. Interestingly, although Post -Post slightly underperforms the others, the differences remain marginal. On Chronos Bollt all configura- tions perform nearly identically, with Post-Post achieving the lowest error (0.455), reinforcing the robustness of our fusion design. These findings suggest that while the timing of fusion can affect the model's attention mechanism and how contextualizes covariate information, UniCA
31

---

Published as a conference paper at ICLR 2026
Prediction Comparison
Prediction Comparison
Value
ground truth prediction UnicA)
Value
past ground truth prediction
interval (1 (0%))00%)
interval (10% 90%)
Time Step
Time Step
Value
Value
ground truth
ground truth prediction Ged UniCA)
prediction (we UniCA) 90%))
Time Step
Time Step
Important Features Visualization
Important Features Visualization
Importance Covoilat: Umpprtance: 3.0125)
(imprrence limprraacc: 5.0277)
Value
Value
Time Step
Time Step
Feature Weights Visualization
Feature Weights Visualization
Sequence Position
Sequence Position
Feature Dimension
Feature Dimension
(a)
(b)
Figure 8: Visualization of prediction comparison, important covariates and attention map on MMSP
32

---

Published as a conference paper at ICLR 2026
Important Features Visualization
Important Features Visualization
past
past
ground
ground cildnition
prodicton
Covorats dimportanc: 0,2673)
Covarfate (imprraance 0 2406
Covar uts Amportance: 0.1573)
Covuriate [imporaaice:: 0.3045).
Value
Value
129
100
Time Step
Step
Prediction Comparison
Prediction Comparison
Value
ground truth
Value
ground truth
prediction (wa/ UniCA) interval (10%-9000
prediction UniCA
interval (10%5-90%)
8
Time Step
Time Step
past ground prediction tw UnICA) interval 120% 90%
200
past
Value
Value
ground
prediction (w UnICA)
Interval (10% 90%)
Time Step
Time Step
Feature Weights Visualization
Feature Weights Visualization
@ Postbil
COODOO 8 Posttion
Feature Dimension
Feature Dimension
(a)
(b)
Figure 9: Visualization of prediction comparison, important covariates and attention map GFC14
33

---

Published as a conference paper at ICLR 2026
Important Features Visualization
Important Features Visualization
past
Dast gmund prediction
gmonn
prediciion Pla DA
Covar ott (moortt 64051
favoriate 6 seppottonce: 0.6284)
Covar alle (mporta scie 0.17401
Covariate 5 [empertance: 0.1870)
Value
Value
Time Step
Step
Prediction Comparison
Prediction Comparison
MM
Value
past ground truth prediction (wo// UniCA)
Value
ground truth
prediction Eveg/ UniCA)
interval (10%-908)
Interval (10%-90%%
Time Step
Time Step
Value s
MMMM
past
past ground truth
Value
ground
prediction tw UnICA)
prediction (ww UniCA)
Interval 110% 90%)
interval (10% 90%5)
Time Step
Time Step
Feature Weights Visualization
Feature Weights Visualization
Sequence Postbion
Sequence Position
Feature Dimension
Feature Dimension
(a)
(b)
Figure 10: Visualization of prediction comparison, important covariates and attention map on EPF
34

---

Published as a conference paper at ICLR 2026
Important Features Visualization
Important Features Visualization
past
Dast gmund prediction
ground
prodictin
Covar ate amportance: 0.40233
Cotoriate [emportonce: 0.44511
10:00
Covar uta Ompertance: 0.1535)
Caveriate 5 [imprrtance: 0.1478)
Value
Value
19:00
1100
Time Step
Time Step
Prediction Comparison
Prediction Comparison
M M
Value
past
Value
ground truth prediction twoi UniCA) interval 90%9990%)
ground truth
prediction (wol UniCA! Interval (10 (10%-90%%
6900
Time Step
Time Step
M
past truth prediction (w/ UniCA) Interval (10% 90%)
Value
13900
past
1509
1000
ground truth
prediction UniCA)
interval (10% 90%)
Time Step
Time Step
Feature Weights Visualization
Feature Weights Visualization
Sequence Posttion
Sequence Position
Feature Dimension
Feature Dimension
(a)
Figure 11: Visualization of prediction comparison, important covariates and attention map on Hog
35

---

Published as a conference paper at ICLR 2026
0.50
Pro-Pre
Post Pro
0.48
Pre-Post
Aggnegeted Error
0.47
0.433
0476
0.47
0.46
0.655
0.45
048
TimesFM
Chronos Bolt
Model
Figure 12: Ablation study fusion position. Pre-Pre, Post-Pre, Post-Post, and Pre-Post indicate the fusion positions past and future covariates (before or after the temporal encoder). Results shown on TimesFM and Chronos Ahronos-Boot. Performance remains stable across all configurations, demonstrating the fusion position does not affect the performance much.
mains stable and effective regardless of specific fusion positions. This reflects the flexibility of our attention-based fusion modules and their adaptability across model architectures.
G.2 INFLUENCE OF MODALITY ENCODER
To investigate the effect of different text modality encoders forecasting performance, duct a comprehensive evaluation on the Time-MMD benchmark across six domains: Climate, En- ergy, Environment, Health, Security, and SocialGood. We embed the same textual covariates
ing four representative pretrained language models GIST (Solatorio 2024 (a text embedding
model). BERT Devlin al. 2019 GPT-2 (Radford et al. 2019 and LLAMA-2 TTovron
et al. 2023 13 and report forecasting results under two forecasting backbones: Chronos- Bont and TimesFM, both implemented via the UniCA framework. The results are summarized in Table [3] Chronos consistently benefits from text embeddings, with minor variation across encoder types. In contrast, TimesFM exhibits significantly larger fluctuations in performance depending on the en- coder. For instance, under the Environment domain, the MAE of TimesFM ranges from 0.756 (GIST) 1.181 (BERT), whereas Chronos maintains stable performance across all encoders (MAE 0.738 for all). Among all text encoders, GIST demonstrates the most consistent performance across both Chronos and TimesFM, achieving the lowest average forecasting errors in domains such as Security (MAE = 0.70 (0707) and SocialGood (MAE 0.860) for TimesF.M. On the other hand, large language models such as LLAMA do not necessarily outperform smaller encoders. In several cases, GPT and BERT lead to degraded performance in TimesFM, especially for domains with noisier text (c.g., Environment and Security). The optimal encoder choice appears domain-dependent. For example, in the Health domain, GPT yields the best MAE (0.609) under Chronos, while GIST performs best under TimesFM (MAE = 0.692). In the Security domain, TimesFM benefits the most from GIST (MAE 0.707), which is substantially better than other encoders.
These findings suggest that the selection of the modality encoder impacts model performance, de- pending on the dataset and base model. However, we also warn that the number of observed points in Time-MMD may not be enough to draw consistent conclusion. This experiment only shows preliminary results.
G.3 ARCHITECTURE COVARIATE HOMOGENIZER
We examine how the architecture of the covariate homogenizer affects forecasting performance in our unified framework. Specifically, we compare two designs: simple Linear layer and two-layer
10 "htps:/huuginggfcce
" http::/huugiigggaccoooggggeeeeee co/ggooglebeerbbeettbbb--eeeeeee uncased 12 hhtps://huggingface. www.eeaeeccceeteeeeeeeaeeeaaeeee www.eeta-lammmlllaaaaa
36

---

Published as a conference paper at ICLR 2026
Table 13: Forecasting error on Time-MMD subsets with different text encoder.
Chronos-Bol (UniCA)
T (UniCA)
GIST
Bert
GPT
I.AMM
GIST
Bert
GPT
LLAMA
Averuge
0.507
0.507
0.507
0.507
0.533
0.541
0.542
0.533
Climate
MAE
0.628
0.628
0.628
0.628
0.635
0.630
0.631
0.635
МAРE
0.421
0.423
0.421
0.543
0.577
0.578
0.543
MSE
0.411
0.411
0.411
0.411
0.402
0.401
0.402
0,402
CRPS
0.567
0.567
0.552
0.558
0.556
0.552
Averuge
0.879
0.890
0.893
0.903
0.904
0.915
0.907
0.954
Energyy
MAE
0.921
0.929
0.928
0.937
0.957
0.972
0.934
0.968
MAPE
0.831
0.850
0.866
0.876
0.795
0.811
0.925
0.971
MSE
0,934
0.935
CRPS
0.899
0.907
0.906
0.916
0.928
0.944
0.903
0.944
Eaviromment
Average
0.625
0.625
0.625
0.625
0.642
1.103
0.661
1.024
MAE
0.738
0.738
0.738
0.738
0.756
1.181
0.754
1.095
МAРE
0.666
0.667
0.666
0.666
0.656
1.189
0.716
1.145
MSE
0.560
0.561
0,560
0,560
0.599
1.114
0.620
0.982
0.536
0.556
0.930
Averuge
0.612
0.609
0.609
0.617
0.692
0.693
0.710
0.709
Health
MAR
0.697
0.694
0.702
0.753
0.754
0.769
0.764
МАРE
0.565
0.569
0.565
0.573
0.684
0.679
0.698
0.718
MSE
0.600
0.604
CRPS
0.699
0.696
0.695
0.703
0.733
0.735
0.750
0.742
Average
0.705
0.706
0.702
0.698
0.593
0.663
0.879
0.642
Security
MAE
0.860
0.862
0.858
0.854
0.707
0.785
1.009
0.762
МАРE
0.526
0.467
0.553
0.854
0.526
MSE
0.660
0.660
0,658
0.656
0.578
0.624
0.747
0.613
0.757
SocialGood
Average
0.790
0.790
0.781
0.790
0.664
0.656
0.661
0.654
MAR
0.860
0,860
0.834
0,860
0.699
0.689
0.695
0.687
МАPE
0.666
0.666
0.673
0.666
0.547
0.530
0.541
0.529
MSE
0.784
0.784
0.791
0.784
0.717
0.711
0.717
CRPS
0.850
0.850
0.827
0.850
0.694
0.692
0.690
0.683
MLP, implemented under two different model Beabbones-Chrooo---e and TimesFM, both with
UniCA. Results are reported the MMSP and MMSP benchmarks, as shown in Table 14
From the results, we observe that for Chronos, the homogenizer design has negligible impact on per- formance: both Linear and MLP yield nearly identical errors across all metries. In contrast, TimesFM shows a consistent preference for the Linear homogenizer. For instance, in MMSP, the Average er- ror increases from 0. 147 to 154 when switching from Linear to MI.P, with similar degradation observed in MAE, MSE, and CRPS. This trend holds across both MMSP and MMSPt
These results suggest that more complex homogenizer structures like MLP may introduce unneces- sary parameterization and lead to overfitting in already expressive models such as TimesFM, while simpler designs suffice for both backbones. Therefore, we adopt the Linear homogenizer as the default in all main experiments.
Table 14: Forecasting error MMSP with different designs of covariate homogenizer.
Chronos-Boil (UniCA)
TimesF'M (UniCA)
Linear
MLP
Linear
MLP
Average
0.121
0.120
0.147
0.154
MISP
0.193
0.229
0.236
MAPE
0.019
0.019
0.046
0.050
MSE
0.090
0.090
0.098
0.106
CRPS
0.180
0.180
0.215
0.222
G.4 INFLUENCE OF STATIC COVARIATES
Our UniCA model incorporates the static covariate that most of the datasets do not include. To isolate and quantify the benefit of this feature, conduct an ablation study two datasets with static
37

---

Published as a conference paper at ICLR 2026
covariates: M5 and Retail' The results, presented in table 15 indicate that incorporating static covariates generally improves performance. The performance gains are most pronounced on the highly diverse Retail dataset, where UniCA improves all metrics for both base models (e.g.. reducing the Average metric for TimesFM from 0.672 to 0.655). While on the M5 dataset, the improvements modest but still noticeable in several metrics (e.g.. Chronos-Boo Bolt's MAPE drops from 0.779 to 0.764, MSE from 0.581 to 0.566). These findings suggest that static variables provide useful categorical and contextual information that enhances model generalization, especially in datasets with diverse item-level characteristics like Retail.
Table 15: Ablation study on the impact of static covariates, conducted on the M5 and Retail datasets. Results show that adding static variables generally improves performance, especially on the Retail dataset, indicating their importance for capturing categorical and contextual information.
Chronoo-Bolt
TimesFM
Dataset
Metric
wol static
w/ static
wal static
w/ static
Avernge
0.625
0.613
0.613
MAE
0.706
0.699
0.699
0,701
M5
MAPE
0.779
0.764
0.731
0.737
MSE
0.581
0.566
0,581
0.587
CRPS
0.432
0.424
0.425
0.426
Averuge
0.680
0.656
0.672
0.655
MAE
0.712
0.712
0.709
Retail
MAPE
0,702
0.655
0,707
0.648
MSE
0.799
0.769
0.781
0.776
CRPS
0.499
0.489
0.486
0.485
G.5 IMPACT OF COVARIATES
To see whether UniCA's performance gains simply stem from the introduction of additional trainable parameters rather than the effective integration of covariates, we designed a crucial ablation study. We created an ablated model, denoted as UniCA wo/ Cov, which retains the exact same model structure and the same number of trainable parameters as the full UniCA model. However, in this ablated version, we intentionally set the gating mechanism for all covariates to zero throughout the training and inference processes. This ensures that while the extra parameters are present and trained, the adapted covariate features prevented from influencing the TSFM backbone. We compare the full UniCA model against this UniCA wo/ Cov variant using both Chronos-Boo and TimesFM backbones across all metrics.
The results of this ablation study are presented in Figure 13 For nearly all metrics and both back- bones, the full UniCA model (red bars) significantly outperforms the ablated UniCA wo/ Cov model (blue bars). For instance, with the Chronos- Bolt backbone, UniCA achieves 0.509 MAE and 0.383 MSE, substantially better than the wo/ Cov model's 0.517 MAE and 0.396 MSE. A similar pattern holds for TimesFM, particularly for the MAE and MAPE metrics. The consistent degrada- tion in performance for the wo/ Cov configuration clearly demonstrates that the performance gain is not merely due to the additional parameters introduced by the adaptation layer. Instead, the gains are primarily attributable to the effective and meaningful integration of external covariate infor- mation enabled by the learned adaptation and gating mechanism, thereby validating the core design of UniCA.
G.6 ROBUSTNESS TO NOISY COVARIATES
To verify UniCA's stability against irrelevant external information, we conducted a robustness ex- periment by introducing purely noisy covariate to all benchmark datasets. This synthetic feature was generated as white noise, sampled from (O, 1), and is entirely uninfornative and future- unknowable, simulating low-quality input that robust model should ignore. The UniCA model in its w/ N configuration processes this noise alongside any existing covariates. We com- pared the performance of UniCA (w/ Noise) against the Zero-Shot (ZS) baseline and the original
"The M5 dataset includes static covariates 'item id", "depeartment id", "category id", "store id", and "state id". The Retail dataset includes "city", "state", "type", "cluster", "family", "class", and "perishable"
38

---

Published as a conference paper at ICLR 2026
0.7
UniCA wo/ Cov
0.6
0.922
0,548
value
0,509
0.517
0.332 0.528
0.5
0.455
0.445
0.429
0.4
0.396
0.403 0
MAE
MAPE
MSE
CRPS
MAE
MAPE
MSE
CRPS
Chronos-Bad Bolt
TimesFM
Figure 13: Comparison between the full UniCA model (red) and an ablated version (UniCA wo/ Cov, blue) where the covariate adaptation parameters are introduced but the covariate influence is intentionally gated zero. A lower score better.
UniCA configuration, testing both Chronos-Bole and TimesFM backbones, with the goal of ensuring the adaptation mechanism does not catastrophically integrate non-predictive features.
Table Performance comparison of UniCA and Zero-Shot (ZS) baseline when UniCA is pro- vided with purely random, future-unknowable noisy covariate (w/ Noise). Results are averaged across all benchmark datasets, demonstrating UniCA's resilience to irrelevant noise inputs.
Chornos Bolt
TimesFM
ZS
UniCA
w/ Noise
ZS
UniCA
w/ Noise
Avg
0.472
0.457
0.468
0.473
0.472
0.475
MAE
0.521
0.509
0.518
0.530
0.526
0.526
MAPE
0.522
0.506
0.516
0,523
0.514
0.523
MSE
0.403
0.383
0.397
0,402
0.403
0.403
CRPS
0.441
0.429
0.441
0.437
0.445
0.446
The results, summarized in Table [6. confirm UniCA's strong resilience to noisy covariates. With the Chronos-Bolt backbone, UniCA (w/ Noise) shows only marginal performance degradation (Avg Metric 0.457 - 0.468) and still outperforms the baseline (0.468 vs 0.472). This suggests the model's feature integration module effectively suppresses the disturbance from the pure noise. The stability is even clearer with the TimesFM backbone, where the performance nearly identical (0.472 + 0.475). This high degree of stability validates the design of the UniCA architecture: its adaptation layer is highly effective at identifying and mitigating the impact of uninformative or noisy input features, ensuring that the integration of external data does not compromise the model's forecasting accuracy.
G.7 FUSION ARCHITECTURE
To justify the choice of Gated Residual Network (GRN) and Gated Linear Unit (GLU) for covariate adaptation and fusion, we conducted an ablation study against simpler alternative. The reviewer questioned the necessity of these specific non-linear structures. fusion mechanism, termed Weight Fusion, where the complex GRN/GLU adaptation module is linear combination (a single fully connected layer followed by
thus implemented a baseline
placed by simple, weight-learnate linear output) before fusion with the TSEM backbone's representations. This alternative maintains a similar parameter count but removes the complex gating and non- non-linear residual structure. We compare the performance of the full UniCA model against this simpler Weight Fusion approach,
39

---

Published as a conference paper at ICLR 2026
Table 17: Performance comparison between the full UniCA model (using GRN/GLU for adaptation) and an alternative "Weight Fusion" mechanism where the complex adaptation module is replaced by a simple trainable weighted summation. A lower score is better, validating the superiority of the non-linear GRN/GLU structure.
Chomos-Bolt
TimesFM
ZS
UniCA
Weight Fusion
ZS
UniCA
Weight Fusion
Avg
0,472
0.457
0,471
0.473
0.472
0.478
MAE
0,521
0.509
0,519
0,530
0.526
0,532
MAPE
0.522
0.506
0.529
0.523
0.514
0.523
MSE
0,403
0.383
0,396
0,402
0.403
0.408
CRPS
0.441
0.429
0.439
0.437
0.445
0.450
as well as the Zero-Shot (ZS) baseline, across all metrics and both Chronos-Bov and TimesFM backbones.
The results of the fusion mechanism ablation are presented in the table. Across all metrics and both backbones, the full UniCA model consistently achieves the best performance. Specifically, the simple Weight Fusion mechanism performs noticeably worse than the full UniCA model and, in many cases (e.g., Chronos-Bolt Avg: 0.471, TimesFM Avg: 0.478), its performance degrades close to or even below the Zero-Shot (7S) baseline (TimesFM Avg: 0.473). For example, with Chronos- Bolt, UniCA's MAE 0.509, while Weight Fusion's is 0.519. This significant performance gap strongly validates our architectural choice. The GRN and GLU structures are critical because their gated, non-linear, and residual design allows the model to selectively adapt and dynamically weigh the contribution of each covariate feature, which is essential for effective fusion with the general- purpose TSFM representations. Simple linear fusion, contrast, fails to capture the necessary com- plexities for optimal TSFM adaptation.
H CORRELATION ANALYSIS OF HOMOGENIZED COVARIATE EMBEDDINGS
To address the reviewer's concern regarding the interpretability and meaningfulness of the homog- enized covariate embeddings, we conducted an analysis of the feature space. The core idea of ho- mogenization is to transform the diverse covariate inputs into unified, rich representation space suitable for fusion with the TSFM's temporal embeddings. To demonstrate that these features cap- ture meaningful temporal structure, we calculated the Pearson Correlation Coefficient between the first four dimensions of the final homogenized covariate embedding (just before it enters the TSFM backbone) and the target time series for diverse set of benchmark datasets.
Table 18: Pearson Correlation of Homogenized Embedding Dimensions with the Target Series. The table presents the Pearson correlation coefficient between the first four dimensions of the homoge- nized embeddings (before fusion) and the corresponding target time series across various datasets. The results show that different embedding dimensions capture distinct and meaningful temporal structure related the target.
MMSP
Climate
Energy
Environment
Health
Security
SocialGool
Traffic
Feature 1
-0.149
-0,058
0.068
-0.055
0.033
0.052
0.434
0.271
Feature 2
0.415
0.058
-0.419
-0.001
-0.114
0.063
-0.114
0.127
Feature
0.310
-0.117
0.113
0.042
0.087
0.105
0.459
0.289
Feature 4
0.222
0.024
0.224
0.037
-0.023
-0.084
-0.371
-0.104
The correlation results presented in the table. While the absolute value of the correlation varies significantly by dataset and feature dimension, the patterns strongly support our claim that the ho- mogenized embeddings capture meaningful, yet diverse, temporal structures. For instance, in the SocialGodd dataset, Feature exhibits a strong positive correlation (0.4593), whereas Feature shows moderate negative correlation (-0.3713), indicating that different dimensions of the embed-
40

---

Published as a conference paper at ICLR 2026
ding are learning to capture distinct aspects of the underlying temporal dynamics of the target series. The general non-zero correlations across datasets (c.g., strong correlation MMSP and Traffic confirms that the adaptation process successfully transforms the raw, disparate covariate informa- tion into a fixed-length embedding that temporally structured and relevant to the forecasting task, which is necessary condition for effective fusion.
DISCUSSION OF LIMITATION
UniCA assumes temporal alignment between covariates and the target series, which approximate using imputation and missing-value indicators. However, more effective alignment strategies may exist. Additionally, noisy conflicting covariates can degrade performance. Future work may corporate uncertainty-awar fusion, handle non non-aligned or partially observed covariates, and embed task-specific inductive biases to enhance the robustness and generalizability of TSFM adaptation.
THE USE OF LLMS
We utilized a Large Language Model (LLM) to assist in the writing process of this paper. The pri- mary use of the LLM was for improving the language, style, and readability of the text. This included refining sentence structure, correcting grammatical errors, and ensuring consistency in terminology. All intellectual contributions, including the research ideas, methodology. and conclusions, are solely the work of the human authors. The authors have reviewed and take full responsibility for the entire content of this paper, ensuring its originality and scientific accuracy.
41


Reviews:
[{"review_id": "eNn1UzdQG0", "reviewer": "Reviewer_Ruus", "summary": "The authors propose UniCA, Unified Covariate Adaptation, an adaptation method for Time Series Foundation Models (TSFMs) to support covariate-aware forecasting. This adaptation is achieved through addition of linear layers to adapt covariates into the FM embedding space and use of Conditional Attention Pooling (CAP) to fuse covariates signals into the FM univariate predictions.\n\nAuthors have conducted thorough evaluation using datasets containing time series covariates, as well as text and image covariates, and demonstrated their method outperforms prior work.\n\nThe main contribution of the work is the adaptation framework that supports any modality covariate-aware forecasting through training of the adapter.", "questions": "1. How does UniCA perform when provided with noisy covariates?\n2. Can UniCA achieve 100% accuracy when provided with the oracle label (i.e. the target time series)?\n3. Why does UniCA perform better with Chronos-bolt in some cases and with TimesFM in others?\n4. Does your ChronosX implementation achieve the performance reported in the ChronosX Arango et al. paper?", "limitations": "", "rating": 4, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. Problem formulation and motivation are explained clearly.\n2. Authors have conducted a thorough evaluation spanning various prior work, datasets and covariates.\n3. Proposed method has minimal overhead during inference time.\n4. Training one adapter per dataset can work on any forecast horizon. (No requirement of unique adapter for each forecast horizon)", "weaknesses": "1. Although TSFM parameters are frozen when training the adapter, training cost is still incurred to perform adaptation. This takes away the ability of TSFMs to forecast zero-shot. \n2. The adapter proposed in ChronosX is also based on linear layers. The contribution of this work therefore appears mostly incremental. The authors add additional parameters and attention modules before and after the backbone transformer FM. It is unclear whether the performance gain arises simply from these additional parameters.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "hWtbUWARV5", "reviewer": "Reviewer_h1TS", "summary": "This paper proposes UniCA, a unified covariate adaptation framework that enables pretrained Time Series Foundation Models (TSFMs) to handle heterogeneous covariates such as categorical, image, and text data. By combining covariate homogenization and attention-based fusion, UniCA integrates diverse covariates without modifying pretrained TSFM parameters.", "questions": "See the weakness", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The specially designed plug-in fusion modules preserve pretrained generalization while efficiently leveraging covariate information.\n- The two-stage design (homogenization + fusion) is intuitive and broadly compatible with multiple TSFM architectures\n- Strong empirical results demonstrating superior performance on 12 unimodal and multimodal datasets with minimal added cost.\n- Comprehensive analysis and ablations confirming the framework’s universality, interpretability, and computational efficiency.", "weaknesses": "1. **Unclear embedding alignment across heterogeneous modalities.**  \n  The paper claims that the *Covariate Homogenization* module transforms heterogeneous covariates (e.g., categorical, image, and text) into a unified latent space, yet no explicit *alignment loss* or *architectural constraint* ensures such consistency. Given the complexity of learned representations, the claim that all embeddings could be easily projected into the same latent space by only using one linear layer is not very convincing.\n\n2. **Limited justification for architectural choices in CAP.**  \n  It is unclear why the authors chose **Gated Residual Networks (GRN)** and **Gated Linear Units (GLU)** in the CAP instead of other modules. The paper should either explain the reasons why GRN and GLU are particularly suited for covariate fusion or provide ablation studies comparing them with simpler or alternative fusion mechanisms.\n\n3. **Insufficient qualitative evidence for homogenization effectiveness.**  \n  The visualization in Figure 5(b–c) shows only a single example, which may be cherry-picked. To convincingly demonstrate that the homogenized embeddings capture meaningful temporal structure, more examples or aggregate statistics should be included in the appendix.\n\n4. **Lack of discussion on zero-shot forecasting capability.**  \n  One of the defining advantages of TSFMs is their **zero-shot generalization**, yet the paper does not analyze whether UniCA preserves or degrades this property. Since UniCA introduces additional learned modules, a dedicated experiment or discussion on its impact on zero-shot forecasting is essential to establish its compatibility with the TSFM paradigm.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "umKWQyRdFw", "reviewer": "Reviewer_87eP", "summary": "This paper formulates a novel problem setting, i.e., adapting pre-trained channel-independent time series foundation model to covariate-aware pratical tasks. The covariate may involve categorical variables,images or text modalities. The motivation is clear and significant.  Accordingly, a  Unified Covariate Adaptation (UniCA) framework is proposed, which bridges the gap between the pre-trained model and real practice. Extensive experiments are carried on to verify the effectiveness of the proposed approach.", "questions": "1. the code repo seems to be expired, could the author fix it?\n\n2. The time series tasks addressed in this paper seem to be limited to forecastingonly. Can this framework be extended to other tasks, such as anomaly detection and classification?\n\n3. Covariate-aware prediction is a very important scenario in practical time series applications. Moreover, actual time series data may also have issues such as data missingness. How does this framework handle these problems?", "limitations": "", "rating": 6, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The motivation is clear and sufficient. Adapting pre-trained channel-independent time series foundation model to covariate-aware pratical tasks makes sense.\n\n2. This is the first work to formalize the problem of adapting Time Series Foundation Models (TSFMs) to general covariate-aware forecasting scenarios.\n\n3. Technically sound and experiments are comprehensive.", "weaknesses": "1. When dealing with multi-modal covariate, the tokenization methods for different modality is critical, which need more discussion and explanation.\n\n2. According to Fig.3 (a), the improvements for Chron os-Bolt and TimesFM seem to be marginal.\n\n3. For modeling time series with discrete variables, there are existing works should be discussed:\n\n[1]. General Mixed Time Series Analysis via Latent Continuity Recovery and Alignment. NeurIPS 2024.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Edv54CbNIF", "reviewer": "Reviewer_KAgW", "summary": "The paper proposed UniCA, which focuses on facilitating time series foundation models to be able to handle heterogeneous covariates, such as categorical variables and covariates from other modalities. This is achieved by projecting the covariates into the representation space. The representations are then fused with an attention mechanism.", "questions": "**Q1:** Following W1, it would be useful to see the performance influence when the categorical information is mixed up / randomly initialised.\n\n**Q2:** Following W2, it would be beneficial to also show how the performance would be influenced when the information is weighted differently w.r.t. their information level.\n\n**Q3:** Do the two fusion methods, GRN (in pre-fusion) and attention (in post-fusion), work exchangeable or is there a special concern in this modelling choice?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "**S1:** The motivation is clear. The research question highlighted by the paper is valid and addresses an important problem in time series forecasting for real-life scenarios, i.e., rather than relying on the target covariate itself, external covariates should be taken into account to better analyze the dynamics.\n\n**S2:** The experiments are rather extensive. The way the authors try to present the results is straightforward to convey the information using visualized graphs.", "weaknesses": "**W1:** It makes sense to use a frozen pretrained model to transform the image and text into the representation space. However, when it comes to categorical data, the authors mentioned that they are converted using embedding layers, which have not been pretrained \\& frozen. However, this embedding layer has no prior knowledge about the given categories. Rather, it serves as a lookup table of random vectors. This led to concern about whether the added categorical information is useful in training. See Q1 for follow-up.\n\n**W2:** The exogenous covariates have different levels of information (e.g., the categorical is sparse while the image and text are dense). In this method, the inclusion of these external covariates has simply been stacked together. While this naive method is reasonable, it raises concerns that there might be cases where the dense information is down-weighted, limiting the model’s ability to fully leverage the high-dimensional semantic content. See Q2 for follow-up.\n\n**W3:** Another concern would be that the covariates, after mapping with the embedding layer / model, do not contain temporal information that can be strictly aligned with the time series. From the modelling design perspective, the proposed architecture does not support the claim that the model captures the feature changes over time, as the linear layer is a mixing over time. \n\n**W4:** It would, in general, be beneficial if the authors could give more justification for the model design. Apart from the concern mentioned in W3. See Q3.\n\nMinor: \n\n- Code in the anonymised link has expired.\n\n- The notation system is confusing. The future-known and future-unknown covariates have been simplified without an explicit definition.\n\n- The definition clarity can be improved. The term \"Covariate\" includes both the target and the exogenous variables (or as exogenous covariates, as the authors denoted in the paper). However, in some cases (e.g., lines 154-156), this paper simply uses covariates in place of exogenous covariates, which can lead to confusion.\n\n- Some results can be hard to read due to an over small font, e.g., Fig 4 (c).\n\n- The expression can be improved, e.g., a \"-6.5\\% improvement\" can be \"reduced the MAE by 6.5\\%\", otherwise it appears to be a performance degradation at first sight.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)