AdaPTS: Adapting Univariate Foundation Models to Probabilistic Multivariate Time Series Forecasting
Authors: Abdelhakim Benechehab
Vasilii Feofanov
Giuseppe Paolo
Albert Thomas
Maurizio Filippone
Balázs Kégl
Venue: ICML 2025 poster (2025)
Keywords: Multivariate time series forecasting
Foundation models
Probabilistic inference
Primary area: applications->time_series
TL;DR: We introduce adapters to enhance pre-trained univariate time-series foundation models (FMs) for multivariate tasks, addressing feature dependencies and uncertainty quantification.

Abstract: Pre-trained foundation models (FMs) have shown exceptional performance in univariate time series forecasting tasks. However, several practical challenges persist, including managing intricate dependencies among features and quantifying uncertainty in predictions. This study aims to tackle these critical limitations by introducing **adapters**—feature-space transformations that facilitate the effective use of pre-trained univariate time series FMs for multivariate tasks. Adapters operate by projecting multivariate inputs into a suitable latent space and applying the FM independently to each dimension. Inspired by the literature on representation learning and partially stochastic Bayesian neural networks, we present a range of adapters and optimization/inference strategies. Experiments conducted on both synthetic and real-world datasets confirm the efficacy of adapters, demonstrating substantial enhancements in forecasting accuracy and uncertainty quantification compared to baseline methods. Our framework, **AdaPTS**, positions adapters as a modular, scalable, and effective solution for leveraging time series FMs in multivariate contexts, thereby promoting their wider adoption in real-world applications. We release the code at https://github.com/abenechehab/AdaPTS.

AdaPTS: Adapting Univariate Foundation Models to Probabilistic Multivariate
Time Series Forecasting
Abdelhakim Benechehab 1 Vasilii Feofanoy
Giuseppe Paolo Albert Thomas
Maurizio Filippone
Balázs Kégl'
Abstract
ETTh1 (H = 96)
Pre-trained foundation models (FMs) have shown exceptional performance in univariate time se- ries forecasting tasks. However, several practi- cal challenges persist, including managing intri- cate dependencies among features and quantify- ing uncertainty in predictions. This study aims to tackle these critical limitations by introducing adapters-featureeyee feature-space transformations that fa- cilitate the effective use of pre-trained univariate time series FMs for multivariate tasks. Adapters operate by projecting multivariate inputs into a suitable latent space and applying the FM inde- pendently to each dimension. Inspired by the literature on representation learning and partially stochastic Bayesian neural networks, we present a range of adapters and optimization/infernnce strategies. Experiments conducted on both syn- thetic and real-world datasets confirm the effi- cacy of adapters, demonstrating substantial en- hancements in forecasting accuracy and uncer- tainty quantification compared to baseline meth- ods. Our framework, AdaPTS, positions adapters as a modular, scalable, and effective solution for leveraging time series FMs in multivariate con- texts, thereby promoting their wider adoption in real-world applications. We release the code https://githu..com/baeee http://hwwbb..ooob/mc/ccccc/////
460
480
500
520
540
560
580
600
Steps
Momant
Moment AdaPTS
3,5
foreach component
[X]
x
FM
PM-XX.FM)
RixD
*
R
(b)
Figure 1: (a) Augmenting Moment time series foundation model with the AdaPTS framework provides probubilistic and more accurate predictions.
(b) The AdaPTS frame- work: The input time series transformed through feature space transformation y that maps into stochastic latent space, where the FM now operates. The fire symbol indicate trainable weights while the snowflake implicates that the parameters of the FM are kept frozen.
often multivariate, incorporating multiple descriptive fea- tures (Wei, 2019), and (b) estimating the uncertainty of forecast is equally important, requiring probabilistic model outputs (Gneiting & Katzfuss, 2014). These challenges particularly relevant in real-world applications where risk assessment depends on reliable forecasts, such as health- care (Jones & Spiegelhalter, 2012), finance (Groen et al., 2013), energy management (Zhang et al., 2014; Nowotarski & Weron, 2018), and weather prediction (Palmer, 2012; Bi et al., 2023).
1. Introduction
Time series forecasting is well-established machine learn- ing task that involves analyzing sequential data to predict future trends based historical patterns. Two key chal- lenges frequently arise in this context: (a) time series are
'Huawei Noah's Ark Lab, Paris 'Department of Data Science, EURECOM 'Statisties Program, KAUST. Correspondence to: Ab- delhakim Benechchab <abdelhakim. www.kiww..ncccccheeee.... gmail.com>.
Existing foundation models (FMs) for time series forecast- ing, such as Chronos (Ansari et al., 2024), are typically trained for univariate forecasting tasks due to tractability constraints, as a wide range of real-world time series prob- lems often involve different numbers of features. Neverthe- less, handling multivariate time series directly within these
Proceedings of the ndnd International Conference Machine Learning, Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

AdaPTS: Adapting Univariate Foundation Models to Probabilistic Multivariate Time Series Forecasting
models has been attempted (Liu al., 2024), yet remains computationally challenging due to the high-dimensional dependencies among features. This limitation raises a funda- mental question: how can we leverage existing pre-trained univariate FMs to enable probabilistic forecasting for multi- variate time series?
2.1. Problem setup
Consider multivariate long-term time series forecasting task, represented by: data matrix X R where L is the context window size and D is the multivariate time series dimensionality, and a target matrix Y E R#*D where H is the forecasting horizon. We denote by X E R-x1 (respectively Ya E R [X1) the d-th component of the imput (respectively target) multivariate time series.
To address this, we introduce AdaPTS, a novel framework designed to augment FMs with probabilistic adapters. As illustrated in Figure 1, AdaPTS applies stochastic feature transformation that maps the input time series into latent space, where predictions are made using the frozen FM. Our framework sets itself apart from existing literature by fitting an inverse transformation as part of the adapter, al- lowing predictions to be transformed back into the original feature space. Beyond enhaneing forecasting accuracy, the integration of stochasticity into the adapter's latent represen- tation ensures that the model captures uncertainty, thereby improving both calibration and robustness.
Our goal is leverage frozen pre-trained univariate time series foundation model denoted by fFM RRAX1 - Rf (Fig, lb). In the case of a multivariate input X. for the sake of simplicity, we denote by (XM the application of frM to each channel independently. We aim to achieve hest forecasting quality by minimizing the mean squared error (MSE) loss defined as follows:
L= -PX - = E Iya-- (x (I)
Our approach leads to several novel insights and contribu- tions, which we summarize as follows:
We now formally define an adapter, tool by means of which we aim to best use the foundation model fFM for multivariate forecasting:
1. Multivariate FM adaptation. We introduce a prin- cipled methodology for adapting existing pre-trained univariate FMs to multivariate probabilistic forecasting, resulting in the AdaPTS framework.
Definition 2.1 (adapter). An adapter is a feature-space trans- formation Pp DR - RDD that is applied to the data prior to the foundation model'. The forecast is then obtained by transforming the predictions back to the original feature space:
2. Theoretical foundations of adapters. We provide theoretical analysis to support the necessity of adapters, starting with the analytically tractable case of linear adapters and linear FMs. We then build on the literature on partially stochastic Bayesian neural networks to introduce probabilistic adapters.
Y(: = x (fraa((((((()))
According to this definition, an adapter is valid only if the inverse transformation - 1: :R' + R D such that Yx e Rº. -P. o p(x) = x, is well-defined on RD', In the rest of the paper, we relax this condition by implementing the direct transformation as encoder (ip - encc) and respectively, the inverse transformation as decoder (y- 1 - dec). In this case, the predictions obtained after the application of the adapter become: YX: enc, dec) = dec (frM (enc(X))).
3. Empirical validation. We conduct experiments on multivariate time series forecasting benchmarks, demonstrating that our approach improves forecasting accuracy over baseline methods. We then analyze the interpretability of the learned latent representation and show that adapters enable cost-efficient adaptation by reducing the dimensionality of the feature space.
We note that in the literature, there exist alternatives to adapt to the multivariate setting (Zhang & Yan, 2023; Tian et al., 2023), but we have chosen this family of adapters due to their high flexibility as: (a) any foundation model can be plugged-in, (b) no requirement of fine-tuning due to feature- level transformations (Feofanov et al., 2024), (c) adaptation to the computation budget by defining the number of en- coded channels.
The rest of this paper is organized as follows: Section 2 states the problem setup and related work, Section 3 details the theoretical analysis on linear adapters. Section 4 extends our framework to probabilistic adapters, while Section 5 showcases experimental results. Finally, we conclude with the accompanying Python package in Section 6, and future directions in Section 7.
Optimality of an adapter. In order for an adapter to be useful, it has to achieve forecasting error lower compared to an adapter that represents the identity matrix I of size D and does not perform any transformation of the feature
2. Background knowledge
In this section, we first define rigorously the problem, and then discuss the related work.
II practice oo is applied on matrices X in LXD This denotes the application of yp on each row X.
2

---

AdaPTS:
Adapting Univariate Foundation Models to Probabilistic Multivariate Time Series Forecasting
space. Therefore, we define the optimality of the adapter based on improving the forecasting error of the identity baseline:
3. Theoretical analysis
The purpose of this section is to study the optimization problem that the adapter yp is aiming to solve:
(2)(p) -
10----7 xx--y-'(Jexii (Fxx(((((
where C is defined by Eq. (1).
min J| (Jxx(x((( (2)
2.2. Related work
Under mild assumptions on the adapter function class and the backbone foundation model fFM, we aim at character- izing the optimal solution xp and prove that it realizes the optimality condition: [(p²*) C.
Time Series Foundational Models. Over the past two years, a plethora of foundation models have been pro- posed with a particular focus on time series forecasting. Some of these models like GPT4TS (Tian al., 2023) and Time-LLM (Jin et al., 2024) "reprogram" Large Lan- guage Model to the forecasting setting by freezing most of its layers and fine-tuning additional time series-specific modules to a new downstream task. The majority of time series FMs including Lag-LL ama ama (Rasul et al., 2024), Chronos (Ansari et al., 2024), Moirai (Liu et al., 2024), Time sFM (Das et al., 2024) and Moment (Goswami al., 2024) trained from scratch on large volume of time se- ries data. Beyond the transformer architecture, Ekambaram et al. (2024) present TTM, an all MLP foundation model.
We first consider the linear case where we constrain the adapter &p to the class of linear transformations, parametrized by matrix W. E RDDD; ((X) = XW p.
Assumption 3.1. W.p y has full rank: rank p) D, in- suring its invertibility.
Assumption 3.2. For ease of derivation, we consider a similar linear parametrization for the foundation model: (X) = W + bPMl T where WFM E R bFM E R", and 1 a vector of ones of dimension D.
Proposition 3.3 (Optimal linear adapter). Under Assump- tion 3.1 and Assumption 3.2, the closed- form solution of the problem
Adapters. The multivariate setting presents a significant challenge for time series FMs, as different tasks involve varying numbers of channels². To the best of our knowledge, the only model that naturally accommodates any number of channels is Moir rai (Liu et al., 2024), which, however, suf- fers from high computational demand due to processing all channels flattened in the transformer, leading to a quadratic memory complexity w.r.t. to the number of channels. Most foundation models, instead, treat each one of these inde- pendently, which, as noted by Feofanov et al. (2024; 2025), remains computationally expensive when full fine-tuning is required. For classification tasks with hundreds thou- sands of features, they demonstrated that simple adapters like the rotation matrix obtained through Principal Com- ponents Analysis (PCA) mitigate this issue. At the same time, Benechehab et al. (2025) showed that PCA preserves channel interactions by learning new disentangled compo- nents. However, in both cases, PCA provided little improve- ment over independent processing, leaving room for further enhancements. In the context of tabular regression, foun- dation models such as TabDFT T (Ma et al., 2024) also use PCA to adapt to a variable number of features.
L(W p) = - - (W (WEM XW + bbyl T
W
(3)
writes as:
W 4 B BBA
(4)
where W = arg minw. C(W A W B = bya and (B A)+ denoting the pseudo-inverse operator.
Proof. The result follows by differentiating L(W,)) p) with respect to W., and solving the Euler equation: Vw, L = 0. The detailed proof is deferred to Appendix A.
Remark 3.4. In this case, the fact that the matrix B by 1 have identical columns renders the matrix B A degenerate (with rank [ A) = 1). In practice, we add positive constant to the diagonal in order to numerically stabilize the matrix inversion: W (B A+ B. with À > O. In Section 3.1, we show that
able to reach an optimal solution regardless of this added regular- ization.
Less related to our work, Li & Marlin (2016) use a Gaus- sian process adapter in the context of irregular time series classification. In other domains, adapters have been used for multimodal (text-time series) representation learning (Zhang al., 2024) and computer vision (Li al., 2025; Yin et al., 2023; Pan et al., 2022).
Proposition 3.3 establishes the existence of a solution the optimal adapter problem defined in Eq. (2) and demonstrates that this solution does not necessarily correspond to the identity matrix. This analysis in the linear case provides the foundational motivation for our gradient descent based optimization approach to determine the optimal adapter configuration.
Throughout the paper, the words features, chonnels, and com- ponents are used interchangeably to refer to the number of variates a multivariate time series, represented as Section 2.1.
3

---

AdaPTS: Adapting Univariate Foundation Models to Probabilistic Multivariate Time Series Forecasting
3.1. Working example
4. AdaPTS: Adapters for Probabilistic Multivariate Time Series Forecasting
Synthetic data. Our synthetic dataset comprises two mul- tivariate time series, one with several independent compo- nents and the other with linearly correlated ones (Fig. 2), designed to evaluate linear feature-space transformation. The data generation process creates five uncorrelated base signals (sinusoids with distinct frequencies, amplitudes, and iid noise) and derives eight additional channels through linear combinations of these bases with additive Gaussian noise of different magnitude E (0.1, 0.5).
In this section, we introduce families of adapters that verify the conditions stated in Definition 2.1. Furthermore, we extend this definition to include probabilistic variants of adapters, useful for uncertainty quantification.
4.1. Families of adapters
Our framework accommodates various families of trans- formations that can serve as adapters. Initially, we define linear AutoEncoders and subsequently extend them to their deep non-linear counterparts. Ultimately, we introduce the probabilistic adapter framework, encompassing Vari- ational AutoEncoders (VAE) and azopout-bases families of adapters.
This construction provides a controlled environment where the ground truth relationship between channels is known: the underlying data manifold is effectively five-dimensional, but in the correlated case the observed night-dimensional multivariate time series includes varying levels of noise and
linear mixing.
Linear AutoEncoders. In addition to the linear setup in- troduced in Section 3, we extend Linear AutoEncoders to provide a simple yet effective method for dimension- ality reduction while preserving the temporal relationships within time series data. In this more general case, the en- coder compresses the multivariate time series X into a po- tentially lower-dimensional representation Z = XW enc, where Wene E DDD' is the linear transformation matrix, and D' S D. The decoder reconstructs the forecast to the original feature space as Y = SFM (Z) Weee- Finally, the parameters of the encoder Wene and the decoder W dec are jointly optimized to minimize the objective in Eq. (2).
Randomly generated linear FMs. The experimental setup in Fig. 2 consists in randomly sampling the linear parame- ters of a toy foundation model: WFM and bFM. To simulate a realistic scenario, we use Glorot-uniform initialization dis- tribution as would be the case in neural network- based ar- chitectures. We then compute the closed-form solution W Eq. (4) on raw data X, and compare the resulting loss value with the baseline (using the identity matrix I as adapter) and the PCA-only adapter.
+
Independent
Correlated
Deep non-linear AutoEncoders, Deep non-linear AutoEn- coders extend their linear counterparts by employing multi- ple layers of non-linear transformations. The encoder maps the input X to a latent space Z = enc(X: e. where enc is parameterized by a deep neural network. Similarly, the de- coder reconstructs the predictions of the foundation model in the latent space: Y = dec( dlecfFM[ (Z); Odec).
lel
4
MSE
3
Besides AutoEncoders, Normalizing Flows (Kobyzev et al., 2021) such as RealNVP (Dinh et al., 2017) are a valid choice in the context of adapters, thanks to their inherently invertible nature. However, their training proved challeng- ing due to the transformation being tied to its inverse leading to poor performance in practice. We defer a discussion on Normalizing Flows as adapters to Appendix B.
WiCA
W
WPCA
W;
Figure 2: Optimality of W Comparing the MSE obtained with W against the baseline, for 1000 randomly generated linear FM.
4.2. Probabilistic Adapters
Fig. 2 shows that in the case of uncorrelated data (left col- umn) PCA is equivalent the identity matrix, while the solution W to the problem L(W,) reaches an order of magnitude better forecasting loss. In the correlated case, we observe that PCA has a similar performance to the op- timal solution. This example motivates the adapter idea through the existence of better linear transformations than the identity matrix in the case of linear foundation models.
We now discuss an alternative to the optimization of adapters, which is based a Bayesian treatment of their parameters. There are many options on how to carry out inference over these parameters, and we can draw from the literature on Bayesian inference for neural networks (Papamarkou et al., 2024).
Considering a FM which yields point predictions, the appeal

---

AdaPTS: Adapting Univariate Foundation Models to Probabilistic Multivariate Time Series Forecasting
of Bayesian adapters is that they enable probabilistic pre- dictions, which can be used for uncertainty quantification. Note that this is the case for models such as Chronos and Moirai, which ouiput distribution over the time series continuous values". For deterministic FMs such as Moment (Goswami et al., 2024), a Bayesian treatment of adapters yields an ensemble of such predictions, which is key for accounting for the predictive uncertainty.
the training objective of the VAE, which brings together the forecasting loss and regularization term, similarly the evidence lower bound (ELBO) (Kingma & Welling, 2013) objective, in the following proposition.
Proposition 4.1 (VAE adapter training objective). The train- ing objective for the VAE adapter is the maximization of an ELBO-like lower bound on the marginal likelihood of the target Y:
Inference. Recalling that f represents the set of parameters of encoder (ency) and decoder (deca), we can attempt to ob- tain the posterior distribution over these parameters through Bayes theorem (Gelman et al., 2013):
log Pa ( (Y,,, 2B.,(2/x) [log ( fFM
- KgL p())),
where KI, denotes the Kullback-Leible divergence.
p X) x PY ())
The derivation of this lower bound and a discussion on the implications of each term of the loss are deferred to Ap- pendix A.2.
where p(0) is the prior distribution over the parameters and p(Y|X, 8) the likelihood, with Y, X representing a training dataset in this context. Alternatively, we can treat the latent representation Z as stochastic, where the interest is now to characterize the following posterior:
Remark 4.2. In practice. we use the Gaussian likeli- hood our likelihood model: po [FM(()) N(Y: o²)). with Y deco(ffm(Z)) In this case, the forecasting loss term boils down the MSE ob- jective in Eq. (2) up multiplicative and additive noise-related constants: log.V(Y: o'o -
X)) x Z)(,
In these two formulations, the posterior distribution over the parameters is instrumental in obtaining predictive distribu- tions useful for uncertainty quantification. For instance, in the case of inference over 0, for new test data Y X* we obtain:
Y HD log(2mo²).
Notice that one can also learn a model of the noise where (cco(frx(Z)) (no Y XX, fFM (Z)), os (Yx,,
2
Remark 4.3. The KL divergence regularization term can be multiplied by a scaling factor ,3 to control the disentangle- ment independence of the latent representation compo- nents. This results in 3-VAE (Higgins et al., 2017), that we refer to the VAE adapter throughout the paper.
PY XX, - 1XX'IX0,0000 Y, X)d0.
Characterizing the posterior analytically, however, is in- tractable and we need to resort to approximations. The liter- ature on Bayesian inference offers various strategies, which can be adapted to neural networks (Papamarkou et al., 2024), including variational inference (Graves, 1), Laplace ap- proximations (Yang al., 2024), and Markov Chain Monte Carlo (MCMC) (Chen et al., 2014; Tran et al., 2022).
Dropout as approximate VI. Dropout (Srivastava et al., 2014) can be interpreted form of variational inference, where a variational distribution is imposed over the weights of a neural network (Gal & Ghahramani, 2016). Specifically, applying dropout during training corresponds to approxi- mating a posterior over the weights using a Bernoulli dis- tribution. This perspective allows the deterministic models to be transformed into probabilistic models by introducing stochasticity through dropout.
Within the AdaPTs framework, we focus in particular on variational inference (VI) for VAE adapters and on Monte Carlo dropout (Gal& & Ghahramani, 2016) as an approximate form of VI for earrying out inference over 0.
Adapters and partially stochastic Bayesian neural net- works. Treating adapters in Bayesian manner while keeping the FM fixed aligns with the concept of partially stochastic Bayesian neural networks, which provides theo- retical guarantees on universal conditional density estima- tion (Sharma al., 2023). This framework ensures that the model can approximate any conditional density, provided that stochasticity is introduced early enough in the archi- tecture and that the number of stochastic units matches or exceeds the output dimension. Using probabilistic adapters, we comply with these conditions by making the encoder stochastic, allowing the learned latent space to capture un- certainty while leveraging the FM's fixed parameters.
Variational AutoEne.ders. Following the Bayesian per- spective adapters, VAE assume a prior distribution over the latent representation Z, typically (N( I). The en- coder then outputs parameters of the posterior distribution (Z( and the decoder generates reconstructions of pre- dictions Y ~ Po (Y]X, fFM ()) where 0 parametrize a likelihood model p. In the context of adapters, we define
II the case of Chronos, this distribution is obtained through categorical distribution (with softmax probabilities) over tok- enized space of the time series values.
5

---

designs preva- AdaPTS: Adapting Univariate Foundation Models to Probabilistic Multivariate Time Series Forecasting
No adapter
with adapter
Dataset
H
Moment
PCA
LinearAE
dropout
LinearVAE
VAE
96
0.411 10.012
0.433
0.40210.002
0.395
0.400
0.404_o.001
ETTh1
192
0.431+0.000
0.440
0.452+0.002
0.44640.001
0.44840.000
0.431
24
10.023
2.98
2.624
2.7611.061
2.542
2.461 10.008
Illness
60
3.000
3.079
3.11040.122
2..04
2.75210.040
2.960=o.092
96
0.10700.010
0. 0.17640.000
0.16910.000
0.156_10001
1.0.01
0.187
Weather
192
0.202=0.000
0.208 001
0.198=0.001
0.200+0.001
0.20440.000
0.226=0.000
96
0.13040.011
0.147 0.222
00.011
0.13040.011
0.243 0.39
0.455=0.010
ExchangeRate
192
0.210+0.00z
0.30440.005
0.305=0.013
0.457=#0.020
0.607=0.021
Table I: Performance comparison between the baseline Moment model without adapters against different adapter archi- LinearVAE, and VAE). for multivariate long-term forecasting with
tectures (PCA, Li LinearAE, dropout different horizons H. We display the average test MSE + standard error obtained on runs with different seeds. Best results are in bold, with lower values indicating better performance.
5. Experiments & Results
been used in the literature for model-based reinforcement learning (Benechchab et al., 2025) and time series classifi- cation (Feofanov et al., 2024; 2025).
In this section, we empirically demonstrate quantitative and qualitative superiority of AdaPTS for multivariate long- term time series forecasting on common benchmarks. We show that the proposed adapter approach yields an improve- ment to Moment, a commonly used time series forecasting foundation model, over different datasets and prediction horizons. We then extend the analysis to other foundation models such TTM and TimesFM.
AdaPTS improves the performance of Moment. We present the forecasting error measured by the Mean Squared Error (MSE) in Table and the Mean Absolute Error (MAE) in Appendix D. On the ETTh1 dataset with prediction horizon H = 96, all adapter-based variants outperform the baseline Moment model, with Proopuutineerrhe achieving the best performance, showing an 8% improve- ment. Similar results are observed for the Illness dataset, where all adapters improve over the baseline. Notably, the VAE achieves a significant 15% improvement, reducing the MSE from 2.902 to 2.461 at H 24. In the Weather dataset, the dropout www.eiaaaaaa adapter shows the best improvement across all adapter architectures for H = 96, while its deterministic counterpart, LinearAE, takes the lead at H = 192. The results on the ExchangeRate dataset are mixed, with some adapters matching the baseline per- formance (dropout rear at H 96) while others show degraded performance, particularly at longer pre- diction horizon (H- = 192), which is also observed for the ETThl dataset. Overall, AdaPTS improves the forecasting accuracy of Moment in 5 out of the 8 considered tasks, matches its performance in 2, and degrades performance in task.
5.1. Time series forecasting
Datasets. Our experiments are conducted on four publicly available real-world multivariate time series datasets, com- monly used for long-term forecasting (Ilbert et al., 2024; Wu et al., 2021; Chen et al., 2023; Nie et al., 2023; Zeng et al., 2023). These datasets include the Electricity Trans- former Temperature dataset (ETTh!) (Zhou et al., 2021), ExchangeRate (Lai et al., 2018), Weather (Max Planck In- stitute, 2021), and Influenza- liie Illness (U.S. Centers for Disease Control and Prevention, 2024). All time series are segmented with an input length of L = 512, prediction horizons H E [96, 192] and H E [24, 60) for the Illness dataset, and a stride of 1, meaning each subsequent win- dow is shifted by one step. These datasets (detailed in Ap- pendix I. originate from various application domains, en- abling a comprehensive evaluation of our framework across diverse real-world scenarios.
Beyond Moment. To validate our approach at a broader scope, tested AdaPTS against other TSFMs with distinct architectural characteristies: (1) TTM, an all-MLP architec- ture that contrasts with the transformer-bast lent in other foundation models, (2) TimesFM, a decoder- only transformer model (in contrast to Moment's 's encoder- only architecture). Table 3 summarizes the characteristics
Baseline. We compare our method against the vanilla ap- plication of Moment, (where s stands for small) from the Moment family of models (Goswami et al., 2024). This means that for each dataset, we apply Moment, in- dependently to each feature. Additionally, we compare our Bearning-basee adapters against PCA, an adapter that has
6

---

AdaPTS: Adapting Univariate Foundation Models to Probabilistic Multivariate Time Series Forecasting
(Explained 6%)
InearAE
LineariA
VAF
Labent dm
Latent
-10
-15
-5
Latent
Training
Test
Figure 3: Visualization the latent representation obtained by different adapters (with number of components equal to 2) on Illness(H = 24). Shaded colors indicate the time dimension, with lighter colors representing earlier timesteps.
of these models, all of which have been integrated into our package (see Section 6). We evaluated these foundation models on the Illness( H 24) benchmark, measuring the relative percentage improvement achieved through optimal adapter selection and corresponding hyperparameters.
ETTh1 (K-96)
2 8.45
5.40
4.
Table 2:
MSE relative improvement on the Illness(H = 24)
35
task (# the standard error).
3.0
25
FM
Optimal adapter
MSE Imp (%)
Number of Components
TTM
VAE =)
12.35+2.51
PCB
IneartE
drepouttineeAc
Innervil
Wi
baselnn
TimesFM
dropout
(p 0.1)
3.64
Figure 4: Impact of the number of components on the model performance. The horizontal dashed line indicates the per- formance of Moment without adapters. The shaded area represents its standard deviation, and the vertical line indi- cates the number of original features.
Table 2 reveals varying performance improvements across different foundation models, with TTM exhibiting substan- tial gains while T me showing more modest enhance- ments. However, these findings should be interpreted as preliminary as evaluation across additional datasets and benchmarks necessary to fully characterize the perfor- mance of different foundation models, a direction for ongo- ing research. This observation might also suggest that the effectiveness of the adapter-based approach depends on the underlying foundation model.
only components, underscoring the potential of our ap- proach for cost-effective adaptation of time series founda- tion models. Ultimately, we find that expanding dimension- ality beyond the original feature count does not yield further improvements, as no adapter shows notable enhancements past this point.
5.2. Dimensionality Reduction
5.3. Interpretability of the latent representations
Fig. 4 illustrates the impact of varying the adapter's s la- tent space dimension D' on the forecasting performance across different adapters. For the ETTh1 dataset with a 96-step horizon, all adapter architectures achieve optimal performance components (matching the original fea- ture count), with MSE values consistently lower than the baseline. Notably, at just components, all adapters (ex- cept the PCA baseline) match the baseline score, demon- strating the suitability of our framework for low-resource setups through dimensionality reduction. The Illness dataset (H = 24) presents more compelling results, as the VAE adapter achieves significantly optimal performance with
Fig. 3 compares the representation learning capabilities of different adapters on the Illness (H = 24) dataset, focusing on their ability to distinguish between training and test data. To visualize the raw dataset, we employ PCA for dimension- ality reduction, retaining only two principal components, which is justified by the 95.6% explained variance. When representing the training and test datasets in the space of the first two principal components, we observe a clear distribu- tion shift, potentially complicating the forecasting task for the baseline foundational model. In contrast, using AdaPTr TS results in well-overlapping Gaussian distributions for the

---

AdaPTS: Adapting Univariate Foundation Models to Probabilistic Multivariate Time Series Forecasting
training and test data in the latent space. This demonstrates our framework's ability enforce structured, isotropic rep- resentation that mitigates distribution shift. This effect is par- ticularly pronounced with the VAE adapter and, to a lesser extent, with L, nea rVAE and dropout L nea rAE.
forecasting accuracy, possibly due to the disentangling ef- fect of regularization towards a prior distribution with sta- tistically independent components. Conversely, the ECE heatmap (right) shows that higher | and log o² values result in lower calibration error, with optimal results at B = 4.0 and log o² 3.0. This outcome is anticipated, as larger val- ues of 33 and a mitigate overfitting, where the model tends to exhibit overconfidence in its predictions. Additionally, it is observed that maintaining fixed during training generally outperforms including it in the optimization loop, configuration denoted as auto in Fig. 6.
The findings emphasize the advantages of VAE in managing distribution shift, a critical challenge in time series repre- sentation learning. By modeling uncertainty and enfore- ing a continuous latent space, VAE enhance generalization, making them especially valuable for real-worke world applications where test distributions differ from training data. This aligns with the objective of utilizing adapters in foundational mod- els to optimize zero-shot performance, ensuring robustness across various tasks without extensive fine-tuning.
MSE
6:228
6
2.46
2.43
2:45
2:43
2.58
0.28
0:28
0.28
0:27
0:29
-2556
0.27
0.28
0.20
5.4. On the calibration of the probabilistic adapters
2.42
2.44
2.41
2:40
2.39
2.56
0.28
0.28
0.27
0.27
0.27
0.28
0.27
-2.48
To evaluate the calibration of our adapter-based probabilis- tic forecasters, we use quan- tile calibration (Gneiting al., 2007) as depicted in the re- liability diagram in Fig. 5. In an ideal scenario, a well- calibrated probabilistic fore- cast should align with the red dashed diagonal, indicat- ing that the empirical pro- portion of observations falls within the predicted quantiles at the expected rate. The over- all conclusion is that we ob- serve gradual deviation from ideal calibration as the predic- tion horizon increases (darker shades). While early prediction horizons display reason- ably well-calibrated predictions, longer-horizon forecasts systematically underestimate uncertainty, as shown by the curve falling below the diagonal. This indicates that ob- served values exceed predicted quantiles more frequently than expected, suggesting that the predictive distribution becomes too narrow, resulting in overconfident forecasts.
2.44
2.39
2:42
2:43
2.41
2.68
0.27
0.27 0.27 0.27 0.26 0.28
0026
2.44
2.5
1.0
1.5
2.0
3.0
0.5
1.5
1.5
2.0
3.0
log
0
log 0"
0.8
Figure 6: 3 and log 0² VAE hyperparameters ablation on the Illness(H = 24) dataset. For reference, the Moment bascline score this task is 2.902 10.033
0.6
0.4
0.2
0.0
0.0
0.5 quantile
1.0
LinearAE components. The ablation study presented in Fig. 7 examines the performance of different com- ponents of the linear autoencoder adapter (LinearAE) across three datasets: ETTh1, Weather, and ExchangeR- ate. The figure compares the full linear autoencoder with its encoder-only (LinearEncoder) and decoder-only [LinearDecoder variants. Overall, the results reveal that the decoder component of the linear autoencoder plays the most important role. The encoder-only variant's contri- bution varies, being more impactful in the Weather dataset compared to ETTh1 and ExchangeRate. These findings highlight the significance of the decoder in the AA adapter and suggest that, in the deterministic case, a decoder might be sufficient to capture feature dependencies.
2'5 50
75
Prediction Horizon
Figure 5: Reliability diagram for the first feature of the ETTh1 (H = 96) dataset using LinearVAE.
0.5
MSE
0.0
5.5. Ablation studies
ETTh1
Weather
ExchangeRate
linearEncoder
linearAE
linearDecoder
Influence of a and 3 in the VAE Adapter. Fig. 6 illus- trates an ablation study examining the 3 parameter in 3- VAE and the noise scale of the likelihood model applied to the prediction Y. assessing their effects on MSE and Expected Calibration Error (ECE) (Guo et al., 2017). The MSE heatmap (left) demonstrates that increasing 3 gener- ally diminishes MSE, with the lowest values observed at = 2.0 and 8 = 4.0, particularly for higher log o². This indicates that stronger regularization through @ can enhance
Figure 7: I LinearAE components ablation.
Nevertheless, as shown in our previous experiments, particu- larly Table 1, probabilistic adapters generally outperformed the deterministic ones. This underscores the importance of the encoder as well, which is responsible for approximating the posterior distribution in the latent space- mechanism inherent to our probabilistic framework.
8

---

AdaPTS: Adapting Univariate Foundation Models to Probabilistic Multivariate Time Series Forecasting
6. The AdaPTS Python package
model TUM
darisse IiT filees
adippee TAat Lawwat
trarkig siperiae halvnt
other Sadlos comme loggh
To facilitate reproducibility and promote widespread adop- tion of our framework, we have developed AdaPTS as Python package that provides unified interface for adapting time series foundation models multivariate probabilistic forecasting tasks. The package is designed with modularity, extensibility, and case of use in mind, enabling researchers and practitioners to seamlessly integrate our adapter-based approach into their workflows.
run
-ioddl "riosenn"
-cillad Thnsso"
-adappar "VAE"
-bakiing "superviind"
-other
Figure 8: Command-ling interface for configuring and run- ning experiments within AdaPTS.
a reconstruction loss, where the encoder-decoder pair learns to reconstruct the input time series. This ap- proach is useful for representation learning and can serve as a pretraining step before supervised fine- tuning.
6.1. Unified Foundation Model Interface
key feature of the AdaPTS package is the development of a consistent interface across different time series foundation models. The package currently supports the foundation model families listed in Table 3.
Fine-tuning support. This training mode permits to fine-tune the foundation model parameters alongside adapter training. This is particularly relevant for mod- els like Moment, where the lincar forecasting head (denoted as LP for linear probing in (Goswami et al., 2024)) is not pre-trained. Additionally, this can be implemented through bi-level approach, where the foundation model undergoes secondary fine-tuning af- ter completing adapter training.
Table 3: Time Series Foundation Models supported in the adapts s Python package. MV: Multivariate support; P: Probabilistic forecasting.
Model
MV
P
Link (HF)
Moment
autonlab/moment
TTM
ibm-granite/ttm
TimesFM X
X
google/timesfm
Fig. illustrates the command-line interface for running the AdaPTS approach, where users can configure their experi- ments by specifying the foundation model, dataset, adapter type, and training method through simple script parameters.
Chronos X V
amazon/chronos
Moirai
salesforce/moirai
Moirai uses flatten + attention bias approach to MV.
7. Conclusion
The package abstracts the complexities inherent in working with different foundation models through the ICLTrainer class hierarchy. Each foundation model family is imple- mented as a specialized child class that handles model- specific initialization, preprocessing, and inference proce- dures while maintaining a unified API. This architectural design enables users effortlessly switch between differ- ent foundation models and implement custom ICLTrainer classes for newly-released or proprietary foundation models.
In this paper, we investigate how pre-trained univariate time series foundation models can be adapted for probabilistic multivariate forecasting. Our method, AdaPTS, offers a novel approach to training feature space transformations that facilitate uncertainty quantification and enhance the perfor- mance of baseline foundation models. Through a series of experiments, we demonstrate that our framework improves forecasting accuracy, provides reasonably well-calibrated uncertainty estimates, reduces inference cost through dimen- sionality reduction, and offers interpretable feature space latent representations.
6.2. Training modes
Our framework supports multiple adapter training paradigms to handle different experimental scenarios:
Limitations & Future directions. While our primary anal- ysis focuses on Moment, our approach generalizes to other foundation models, several of which have been integrated into our publicly available package. Extensive experimen- tation across different foundation models and benchmark datasets remains necessary to fully characterize our ap- proach's potential. Furthermore, while inference for computational efficiency, exploring alternative methods such as MCMC could enhance uncertainty esti- mation and calibration, albeit with increased computational overhead.
Supervised Training. This constitutes the primary training methodology analyzed throughout the paper and showcased in Fig. 1b. The adapter components (encoder and decoder) are optimized to minimize the forecasting MSE loss (Eq. (2)) while the frozen foun- dation model generates predictions within the learned latent space.
variational
Unsupervised Training. Adapters are trained using

---

AdaPTS: Adapting Univariate Foundation Models to Probabilistic Multivariate Time Series Forecasting
Impact statement
Ekambaram, V., Jati, A., Dayama, P., Mukherjee, S., Nguyen, N. H., Gifford, W. M., Reddy, C.,, and Kalagnanam, J. Tiny time mixers (ttms); Fast pre-trained models for enhanced zero/few-shot forecasting of multi- variate time series, 2024.
This paper presents work whose goal to advance the field of Machine Learning. There are many potential societal consequences of our work, none which we feel must be specifically highlighted here.
Feofanov, V., Ilbert, R., Tiomoko, M., Palpanas, T., and Redko, I. User-friendly foundation model adapters for multivariate time series classification. arXiv preprint arXiv: 1491222409.22264, 2024.
Reproducibility Statement
In order to ensure reproducibility we release the code at https//gifuub...omm/hennnhhhhaah
The accompany- ing Python package is detailed in Section 6 while more implementation details and hyperparameters are listed in Appendix C.2.
Feofanov, V., Wen, S., Alonso, M. Ilbert, R., Guo, H., Tiomoko, M.. Pan, L., Zhang, J., and Redko, I. Mantis: Lightweight calibrated foundation model for user-friendly time series classification. arXiv preprint arXiv:250.11637,, 2025.
References
Gal, Y. and Ghahramani, Z. Dropout As a Bayesian Ap- proximation: Representing Model Uncertainty in Deep Learning. In Proceedings of the 33rd International Con- ference on International Conference on Machine Learn- ing Volume 48, ICML' 16, pp. 1050-1059. JMLR.org. 2016.
Ansari, A. F., Stella, L., Turkmen, C., Zhang, X., Mercado, P., Shen, H., Shchur, O., Rangapuram, S. S., Arango, S. P., Kapoor, S., et al. Chronos: Learning the language of time series. arXiv preprint arXiv:2403.078/1 2024.
Benechehab, A., Hili, Y. A. E., Odonnat, A., Zekri, O., Thomas, A., Paolo, G., Filippone, M., Redko, I., and Kégl, B. Zero-shot model- based reinforcement learning using large language models. In The Thirteenth International Conference on Learning Representations, 2025.
Gelman, A., Carlin, J., Stern, H., Dunson, D., Vehtari, A., and Rubin, D. Bayesian Data Analysis. Chapman and Hall/CRC, United States, 3rd ed edition, 2013. ISBN 9781439840955.
Bi, K., Xie, I.., Zhang, H., Chen, X., Gu, X., and Tian, Q- Accurate medium-range global weather forecasting with 3d neural networks. Nature, 619(7970):533--555 538, 2023.
Gneiting, T. and Katzfuss, M. Probabilistic forecasting. Annual Review of Statistics and Its Application, 1(1):125- 151, 2014.
Chen, S.-A., Li, C.-L,, Arik, S. O., Yoder, N. C., and Pfis- ter, T. TSMixer: An all-MLP architecture for time se- ries forecast-ing. Transactions Machine Learning Research, 2023. ISSN 2835-8856.
Gneiting, T., Balabdaoui, F., and Raftery, A. E. Probabilis- tic forecasts, calibration and sharpness. Journal of the Royal Statistical Society: Series B (Statistical Methodol- ogy), 69(2):243-268 2007. doi: 1467-9868. 20w.
Chen, T., Fox, E., and Guestrin, C. Stochastic Gradient Hamiltonian Monte Carlo. In Xing, E. and Jebara, T. (eds.), Proceedings of the 31st International Conference Machine Learning, volume 32 of Proceedings of Ma- chine Learning Research, pp. 1683-1691, Bejing, China, 22-24 Jun 2014. PMLR.
Goswami. M., Szafer, K. Choudhry, A.. Cai, Y., Li, S., and Dubrawski, A. Moment: A family of open time- series foundation models. In International Conference on Machine Learning, 2024.
Graves, A. Practical variational inference for neural net- works. In Shawe- Taylor, J., Zemel, R., Bartlett, P., Pereira, F., and Weinberger, K. (eds.), Advances in Neural Infor- mation Processing Systems, volume 24. Curran Asso- ciates, Inc., 2011.
Cowen-Rivers, A., Lyu, W., Tutunov, R., Wang, Z., Grosnit, A., Griffiths, R.-R., Maravel, A., Hao, J., Wang, J., Peters, J., and Bou Ammar, H. Hebo: Pushing the limits of sample-efficient hyperparameter optimisation. Journal of Artificial Intelligence Research, 74, 07 2022.
Groen, J. J., Paap, R., and Ravazzolo, F. Real-time inflation forecasting in a changing world. Journal of Business & Economic Statistics, 31(1):29-44, 2013.
Das, A., Kong, W., Sen, R., and Zhou, decoder- only foundation model for time-series forecasting. In Forty-first International Conference on Machine Learn- ing, 2024.
Guo, C., Pleiss, G., Sun, Y., and Weinberger, K. Q. On calibration of modern neural networks. In Proceedings of the 34th International Conference on Machine Learning Volume 70, ICML'17, pp. 1321-1330. JMLR.org. 2017.
Dinh, L., Sohl- Dack-iikstein, J., and Bengio, S. Density estima- tion using real nvp. 2017.
10

---

AdaPTS:
Adapting Univariate Foundation Models to Probabilistic Multivariate Time Series Forecasting
Higgins, I., Matthey, L., Pal, A., Burgess, C., Glorot, X., Botvinick, M., Mohamed, S., and Lerchner, A. beta- VAE: Learning basic visual concepts with constrained variational framework. In International Conference on Learning Representations, 2017.
Liaw, R., Liang, E., Nishihara, R., Moritz, P., Gonzalez, J. E., and Stoica, I. Tune: A research platform for dis- tributed model selection and training. arXiv preprint arXiv: 08,18, 2018.
Liu, X., Liu, J., Woo, G., Aksu, T., Liang, Y., Zimmer- mann, R., Liu, C., Savarese, S., Xiong, C., and Sahoo, D. Moirai-man moe: Empowering time series foundation models with sparse mixture of experts. arXiv preprint arXiv:2410. 10469, 2024.
Ilbert, R., Odonnat, A., Feofanov, V., Virmaux, A., Paolo, G., Palpanas. T., and Redko, I. Samformer: Unlocking the potential of transformers in time series forecasting with sharpness- aware minimization and channel-wist atten- tion. In Proceedings of the s International Conference on Machine Learning, volume 235. PMLR, 2024.
aware
Ma, J., Thomas, V., Hosseinzadeh, R., Kamkari, H., Labach, A., Cresswell, C., Golestan, K., Yu, G., Volkovs, M., and Caterini, A. L. Tabdpt: Scaling tabular foundation models, 2024.
Jin, M., Wang, S., Ma, L.,, Chu, Z., Zhang, J. Y., Shi, X., Chen, P.-Y., Liang, Y., Li, Y.-F,, Pan, S., and Wen, Q. Time-LLM: Time series forecasting by reprogramming large language models. In International Conference on Learning Representations (ICLR), 2024.
Max Planck Institute. Weather dataset, 2021.
Nie, Y., Nguyen, N. H., Sinthong, P., and Kalagnanam, J. A time series is worth 64 words: Long-term forecasting with transformers. In The Eleventh International Conference Learning Representations, 2023.
Jones, H. F. and Spiegelhalter, D. J. Improved probabilistic prediction of healthcare performance indicators using bidirectional smoothing models. Journal of the Royal Statistical Society Series A: Statistics in Society, 175(3): 729-747,20 2012.
Nowotarski, J. and Weron, R. Recent advances in electricity price forecasting: A review of probabilistic forecasting. Renewable and Sustainable Energy Reviews, 81:1548- 1568, 2018
Kim, T., Kim, J., Tae, Y., Park, C.,, Choi, J.-H., and Choo, Reversible instance normalization for accurate time-series forecasting against distribution shift. In International Conference on Learning Representations, 2022.
Palmer, T. Towards the probabilistic earth-system simulator: A vision for the future of climate and weather prediction. Quarterly Journal of the Royal Meteorological Society, 138(665) 2012.
Kingma, D. P. and Ba, J. Adam: A method for stochastic optimization, 2017.
Pan, J., Lin, Z., Zhu, X., Shao, J., and Li, H. St-adapter: Parameter-efficient image-to-video transfer learning. In Koyejo, S., Mohamed, S., Agarwal, A., Belgrave, D., Cho, K., and Oh, A. (eds.), Advances in Neural Informa- tion Processing Systems, volume 35, pp. 26462-26477. Curran Associates, 2022.
Kingma, D. and Welling, M. Auto-encoding variational bayes. CoRR, abs/1312.6114, 2013.
Kobyzev, I., Prince, S. J., and Brubaker, M. A. Normalizing flows: An introduction and review of current methods. IEEE Transactions Pattern Analysis and Machine In- telligence, 43(11) (1::1)))3643377) November 2021. ISSN 1939-3539. doi: 10.1109/tpmii.220 20t.ami...200.29922..2
Papamarkou, T., Skoularidou, M., Palla, K., Aitchison, L., Arbel, J., Dunson, D., Filippone, M., Fortuin, V., Hennig, P., Hermández-Lobato. J. M., Hubin, A., Immer, A., Kar- aletsos. T., Khan, M. E., Kristiadi, A., Li, Y., Mandt, S., Nemeth, C., Osborne, M. A., Rudner, T. G. J., Rügamer, D., Teh, Y. W., Welling, M., Wilson, A. G., and Zhang, R. Position: Bayesian deep learning needed the age of large-scale ai. In Proceedings of the 41st International Conference on Machine Learning, ICML'24. JMLR.org, 2024.
Lai, G., Chang, W.-C., Yang. Y., and Liu, H. Modeling long- and short-term temporal patterns with deep neural networks, 2018.
Li, C.-X and Marlin, M. A scalable end-to-end gaus- sian process adapter for irregularly sampled time series classification. In Lee, D., Sugiyama, M., Luxburg, U., Guyon, I., and Gameett, R. (eds.), Advances in Neural Information Processing Systems, volume 29, Curran As- sociates, Inc., 2016.
Rasul, K., Ashok, A., Williams, A. R., Ghonia, H., Bhag- watkar, R., Khorasani, A., Bayazi, M. J. D., Adamopou- los, G., Riachi, R., Hassen, N., Biloš, M., Garg, S., Schneider, A., Chapados, N., Drouin, A., Zantedeschi, V., Nevmyvaka, Y., and Rish, I. Lag-Ilama: Towards foun- dation models for probabilistic time series forecasting, 2024.
Li, Y., Ye, J., Wen, X., Xu, G., Wang, J., and Liu, X. Padapter: Adapter combined with prompt for image and video classification. Image and Vision Comput- ing, 154:1,, 2025. ISSN 0262-8856. doi: https: //doi.org/10.1011 101 1016///0...1//..aaaai... imavis 20.ii...22..1000 105395.

---

AdaPTS: Adapting Univariate Foundation Models to Probabilistic Multivariate Time Series Forecasting
Sharma, M., Farquhar, S., Nalisnick, E., and Rainforth, T. Do Bayesian neural networks need be fully stochas- tic? In Ruiz, F., Dy, J., and van de Meent, J.-W. (eds.), Proceedings of The 26th International Conference on Artificial Intelligence and Statistics, volume 206 of Pro- ceedings of Machine Learning Research, pp. 7694-7722.
Zhou, H., Zhang, S., Peng, J., Zhang, S., Li, Xiong, H., and Zhang, W. Informer: Beyond efficient transformer for long sequence time-series forecasting. In The Thirty- Fifth AAAI Conference on Artificial Intelligence, AAAI 2021, Virtual Conference, volume 35, pp. 11106-11115. AAAI Press, 2021.
PMLR, 25-27 Apr 2023.
Srivastava, N., Hinton, G., Krizhevsky, A., Sutskever, I., and Salakhutdinov, R. Dropout: A simple way to prevent neural networks from overfitting. Journal of Machine Learning Research,
2014.
Tian, Z., Peisong, N., Xue, W., Liang, S., and Rong, One Fits All: Power general time series analysis by pretrained Im. In NeurlPS, 2023.
Tran, B.-H., Rossi, S., Milios, D., and Filippone, M. All you need is good functional prior for Bayesian deep learning. 23(1), 2022. ISSN 1532-4435.
U.S. Centers for Disease Control and Prevention. Fluview: Flu activity & surveillance, 2024.
Wei, W. W. Multivariate time series analysis and applica- fions. John Wiley & Sons, 2019.
Wu, H., Xu, J., Wang, J., and Long, M. Autoformer: De- composition transformers with auto-correlation for long- term series forecasting. In Beygelzimer, A., Dauphin, Y., Liang, P., and Vaughan, J. W. (eds.), Advances in Neural Information Processing Systems, 2021.
Yang. A. X., Robeyns, M., Wang, X., and Aitchison, L. Bayesian low-rank adaptation for large language models. In The Twelfth International Conference on Learning Representations, 2024.
Yin, D., Hu, L., Li, B., and Zhang, Y. Adapter is all you need for tuning visual tasks, 2023.
Zeng, A., Chen, M., Zhang, L., and Xu, Q. Are transformers effective for time series forecasting? 2023.
Zhang, W., Ye, J., Li, Z., Li, and Tsung, F. Dualtime: A dual-adapter multimodal language model for time series representation, 2024.
Zhang, Y. and Yan, J. Crossformer: Transformer utilizing cross-dimension dependency for multivariate time series forecasting. In The eleventh international conference on learning representations, 2023.
Zhang, Y., Wang, J., and Wang, X. Review on probabilistic forecasting of wind power generation. Renewable and Sustainable Energy Reviews, 32:255-2700 2014.
12

---

AdaPTS: Adapting Univariate Foundation Models to Probabilistic Multivariate Time Series Forecasting
Appendix
Outline. In Appendix A, we provide the proofs and a discussion on Proposition 3.3 and Proposition 4.1. We then provide perspective on Normalizing Flows as adapters in Appendix B. The experimental setup is presented in Appendix C, including all the implementation details in Appendix C.2. Finally we showcase some additional results in Appendix D.
Table of Contents
Theoretical analysis
14
A.I Proof of Proposition 3.3
14
A.2 Proof of Proposition 4.1
15
B Normalizing Flows
16
Experimental setup
16
C.1 Datasets
16
C.2 Implementation details
16
Additional results
17
D.I Moment applied to synthetic data.
17
D.2 Mean Absolute Error
18
13

---

AdaPTS: Adapting Univariate Foundation Models to Probabilistic Multivariate Time Series Forecasting
A. Theoretical analysis
A.I. Proof of Proposition 3.3
We begin by restating the proposition, and its underlying assumptions:
Assumption A.I. W has full rank: rank( p) D, insuring its invertibility.
Assumption A.2. For ease of derivation, we consider a similar linear parametrization for the foundation model: fFM(X)
W FM X bFM 1 where WIM RLXH bFM E R and vector of ones of dimension D.
Proposition A.3 (Optimal linear adapter). Under Assumption 3.1 and Assumption 3.2, the closed-form solution of the
problem:
((W = Y (W + bay b W
(5)
writes as:
W = (B A
(6)
where W arg mi L(W,) v) A A - w FM B= bFM1 T, and (BT A)+ denoting the pseudo-inverse operator.
Proof.
We begin by expanding the loss function:
L(W = Y (W FM XWW,, + bau (
|A BW
where = WM and B = bbM 1 Expanding the Frobenius norm:
L(W ()) Tr ((A (A BW
Taking the gradient with respect to W-1 yields:
OC
-B A +B 2BBBBW:
OW
Knowing that W. is invertible, We have that:
-W -T dc W aw, ow,
DC
hence
0C 2W T AB W.
aw
ac Setting 7W
and multiplying both sides by WT p. we obtain:
B B
Multiplying both sides by W
BTAW. 4 BB.
Finally applying the pseudo-inverse solve for W. gives our final result:
W BBB
Given the convexity of L(W p) (which follows from the convexity the Frobenius norm ||. the inverse operation, and an affine transformation), we conclude that W is global solution for Eq. (3).
14

---

AdaPTS: Adapting Univariate Foundation Models to Probabilistic Multivariate Time Series Forecasting
Remark A.4. We make of the pseudo-inverse due the current construction of the matrix B (with identical rows) which implies that the product B' A is degenerate. To bypass this limitation and further ensure the invertibility of W gp we revisit the definition of the foundation model in Assumption 3.2 to include channel dependent biases and ensure a full rank matrix B.
A.2. Proof of Proposition 4.1
To derive the evidence lower bound (ELBO) used in the training objective of the VAE adapter, we start from the marginal likelihood of the observed data given the inputs X and foundation model fFM. The marginal likelihood is expressed as:
log po (YX, fFM) log / (rm)))) dz,
where Z is the latent variable, P (Z)) the likelihood model parameterized by 0, and p(Z) the prior distribution over the latent variable Z.
Direct optimization of this marginal likelihood is generally intractable due to the integration optimization feasible, introduce variational distribution qs (Z(X), parameterized by o, as approximation to the true posterior (2 X, Y, fFM). Using (Z(X), we can reformulate the oog-magginal likelihood as follows:
Z. To make this
Pa X|,, /FM (Z)) dz
log Po (Y frM) = log
4s XZ
(8)
90 (Z
pe Y X, FFM (Z))
= log (2(x)
(9)
4s (
Using Jensen's 's inequality, we can derive a lower bound on this log-marginal likelihood:
Po DY(,,, fFM (Z))
lo Y X,M, 2 1g0(2/x) log
(10)
(Z)
La (Z
(2)()) Po (Y), frM (Z))]
(z)())
log
(11)
p(Z)
The second term can rewritten as the Kullback-Leibler (KI.) divergence between the variational posterior do (Z and the prior (Z):
log [x(
KL (Z p(Z)) = E (2(x)
(Z|x)
p(Z)
(12)
Substituting this into the inequality, we obtain the evidence lower bound (ELBO):
log (Y 2 (g0(2(x) (Y (FM(()) KL (4s (Z[)) || p(Z))
(13)
The ELBO consists of two terms:
The forecasting term, Eyg (2(X) the latent variable Z.
which measures how well the model can reconstruct Y given
The regularization term, KI (g+ p()), which encourages the variational posterior to stay close to the prior distribution pZZ).
Thus, maximizing the ELBO provides a tractable way to train the parameters and ø by optimizing the balance between forecasting accuracy and latent space regularization.
15

---

AdaPTS: Adapting Univariate Foundation Models to Probabilistic Multivariate Time Series Forecasting
B. Normalizing Flows
Normalizing Flows make use invertible transformations to map a simple base distribution (e.g. Gaussian) to a complex data distribution. Each transformation T designed maintain invertibility and efficient Jacobian computation. The transformation is applied iteratively: Z Ti o - ..... Ti (X). Current Normalizing Flow instantiations (e.g. RealNVE) make use of generie invertible transformations such coupling flows; the latters can be parametrized using neural network leading to powerful non-linear generative models that are trained to maximize the data log-likelihood:
log det (T:(;;0)
log (X) log p(Z) +
OZ,-1
where 0 denote the parameters of the non-linear parametrization of the invertible transformations T; and Zi-1 is the output of the transformation T;-1. In the context of time series adapters, we directly optimize the parameters of the transformations based on their direct and inverse application on the time series forecasting problem:
Laow = - o T2 o.... o...o.... To' frM (Th Th-10 ( (i(x:))):
where the encoder is represented by the series of direct transformations: enc(.) - Th TR-10-.... Ti(:0), and respectively the decoder by the series of inverse transformations dec(.) =T> T2² ....o (::0)
As defined here, Normalizing Flows suffer from the constraint of keeping the same dimension in both original and learned representation space. For this purpose, we investigate coupling a normalizing flow with a linear encoder-decoder type of architecture to enable dimensionality reduction prior to applying the transformation T. The parameters of the additional encoder and decoder are then jointly trained to optimize the learning objective Lnow.
Given that the parameters of the encoder and the decoder are shared in Normalizing Flows, the pradient-based optimization within our framework receives conflicting directions due to gradient flow from both the direct and inverse transformations simultaneously. We discovered that this adapter construction was challenging to optimize in practice, and we defer the exploration of this direction to future research endeavors.
C. Experimental setup
C.1. Datasets
Table 4: Characteristics of the multivariate time series datasets used in our experiments with various sizes and dimensions.
Dataset
ETThl
Illness
ExchangeRate
Weather
features # time steps
7
7
8
21
13663
169
6791
51899 10 minutes
Granularity
hour
I week
1 day
(Train, Val, Test)
(8033, 2785, 2785)
(69, 2, 98)
(4704, 665, 1422)
(36280, 5175, 10444)
C.2. Implementation details
In this section, we describe the full AdaPTs framework, starting from the data preprocessing, the training algorithm, and the hyperparameters optimization.
Preprocessing. Given that the adapter defined in Definition 2.1 is feature space transformation, we start by rescaling (StandardScaler and MinMaxScaler) the data where all the timesteps are regarded as data points. To account for the temporal specificities in each batch, we use Reversible Instance Normalization (RevIn) (Kim et al., 2022) that has been proven to mitigate riletre distribution shifts in time series problems. Finally, and following the observation that PCA when composed with a linear adapter showed the best result in the case of correlated data (Fig. 2), we include the possibility of applying full-component PCA as part of our data pre- propescnnsn pipeline.
16

---

AdaPTS; Adapting Univariate Foundation Models to Probabilistic Multivariate Time Series Forecasting
Mean Squared Eroor
16-1
Mean Squared Error
7.00
6.75
5.4-
6.50
Wya
6r'
Waas
Waee
W
Adapter Type
Adapter Type
(a) Independent
(b) Correlated
Figure 9; Moment on simulated independent data.
Training parameters. After the pre- pe-processing phase, we proceed to split the data into train wata--aaaaiiiiteeeeeeeeeee sets, where the validation set serves as tool to select the best hyperparameters with the optimal hyperparameters then tested against the unseen test dataset. For all of our experiments, we first train the linear forecasting head of Moment (referred as Linear Probing in (Goswami al., 2024)) with the Adam optimizer (Kingma & Ba, 2017), batch size of 32, cycle scheduler starting with 0.001 learning rate. Once the forecasting linear head is trained, we freeze its parameters and proceed to training the adapter. This is done using the Adam optimizer, batch size of 32, a reduce on plateau scheduler starting with 0.001 as learning rate.
the adapter. The resulting adapter that is instantiated
Hypeeparrameer optimization. In order to select the best hyperparameters for the adapter architecture we use Ray tune (Liaw al., 2018) with the Heteroscedastic and Evolutionary Bayesian Optimisation solver (HEBO) fowen-Riiers et al., 2022) engine, reporting the average mean squred error (MSE) from k-fold cross validation. Table 5 shows the default hyperparameters for each considered adapter.
Table 5: Adapters hyperparameters.
adapter
LinearÄE
DrepoutLinesrAE
LinearVAE
p dropout
0.1
Number of layers
2
dimension
0.5
0.5
1.0
1.0
D. Additional results
D.I. Moment applied to synthetic data.
To validate the adapter optimality condition with large non-linear foundation models, Moment (Goswami et al., 2024). The optimal linear adapter in this case minimizes the following intractable objective:
L(W () Y fttmmnt XWW W 1
(14)
To approximately solve this optimization problem, we instantiate W as a single-linear-layer encoder denoted encs, and respectively the inverse transformation W as single-linear-layer decoder denoted deca. then use gradient-based optimization of the parameters using the Adam optimizer, aiming at solving the following optimization problem:
0- = arg min Y - -deo (Jrommnnt (encg (X)) 1
(15)
Fig. 9 shows the performance gain obtained by optimizing a linear adapter on Moment-small foundation model. Unlike the tractable case, we observe that both data modalities (independent and correlated data), PCA has little to no improvement over the identity baseline, while Pg- reaches an order of magnitude better solution. This confirms our intuition about the existence of better solution than the identity matrix, even in the case of real-worll complex foundation models.
17

---

AdaPTS: Adapting Univariate Foundation Models to Probabilistic Multivariate Time Series Forecasting
D.2. Mean Absolute Error
No adapter
with adapter
Dataset
H
Momentamall
pea
linear
dropout
linear VAB
VAE
96
0. 00000
00.40
0.423
0.415 00002
0.420
0.42640.001
ETTh1
192
0.436
0.445
0.44910.000
0.450
0.451
0.444
24
1.140
1..63
2.62440.003
1.156
1.074
1.057 :0.012
Illness
60
1.149 10.001
1.161 10.001
1.227100.30
1.173
1.112
1.105
96
0.202
0.235 0.00
0. 0.226+0.000
0.212+0.000
0.218+0.00
0.243++0.001
Weather
192
0. 0.251 10.001
0.260
0.251
0.251
0.255
0.27440.000
96
0.252:10 0100
0.264
0.308
0.269
0.376
0.488
ExchangeRate
192
0.329 10.001
0.335 10.000
0.415
0.419
0.513 10.010
0.585
Table 6: Performance comparison between the baseline Moment model without adapters against different adapter archi-
tectures (PCA, Li nearAE, dropout Li LinaaVE,,, and VAE), for multivariate long-term forecasting with different horizons H. We display the average test MAE + standard error obtained on runs with different seeds. Best results are in bold, with lower values indicating better performance.
18


Reviews:
[{"review_id": "Zwg7CX2JND", "reviewer": "Reviewer_267f", "summary": "The paper proposed AdaPTS, an adapter for univariate time series foundation models, which makes them both multivariate and produce probabilistic predictions. The authors first provide a theoretical framework for adapters for time series foundation models, and discuss many adapters (encoder-decoder combinations) which satisfy these properties. The authors demonstrate the performance of their adapters on a few multivariate time series forecasting datasets using the MOMENT time series foundation model.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate the thoughtful feedback provided by Reviewer 267f and would like to address the concerns raised in their review.\n\n> Multivariate Baselines: Beyond PCA, the paper does not compare to some other existing ways of imbuing multivariate context to time series foundation models.\n\nWe acknowledge the reviewer's suggestion to include additional baselines beyond PCA. We chose PCA as it aligns closely with our framework's design. We have also experimented with other non-learning-based adapters such as **SVD** decomposition and **Random Projections**, which yielded similar or worse performance compared to PCA. We thank the reviewer for the provided references and will incorporate these suggested baselines in the updated version of the paper.\n\n> More TSFMs: The authors do mention this as a limitation. I believe that demonstrating their approach on another TSFM, different in design from Moment would make the experiments stronger. I would recommend TTMs (only TSFM not based on the Transformer architecture), or TimesFM (decoder-only, while Moment is encoder-only).\n\nWe thank the reviewer for suggesting additional TSFMs to test within our framework as we recognize the importance of demonstrating the benefit of our approach on diverse TSFMs. We have already integrated **MOIRAI** and tested it with our adapters. In addition, we plan to include more foundation models in the future, including TTMs and TimesFM suggested by the reviewer, to better validate our methodology.\n\n> Datasets: (1) I would encourage the authors to compare their methods on more datasets. The long horizon forecasting benchmark that the authors used, has more datasets, for example ETTh2, ETTm1, ETTm2, Traffic and Electricity. But even beyond these datasets, the GIFT-Eval benchmark has a lot of multivariate time series useful for forecasting. (2) Also, these existing datasets have known issues. For example, a very strong baseline in the Exhange Rate dataset is predicting the last time step. Also, some studies including [1] have shown that these time series datasets do not significantly benefit from modeling cross-channel dependencies.\n\nWe thank the reviewer for the suggested additional datasets, and we plan to use them to strengthen our experimental evidence. Generally speaking, we believe that selecting the appropriate time series benchmark has been a broader issue that affects the whole time series forecasting community. In the multivariate case, it is even more challenging as no study has been performed to indentify datasets with reasonable channel interdependence. In addition, we have experimentally found (Section 3.3) that adapters still can be useful **even in the case when channels are independent**, since it adds an additional level of complexity to the architecture. Identifying an appropriate benchmark for the channel-interdepence studies is an important direction for future work.\n\n> I wonder how flexible is the parametric distribution of the probabilistic predictions. The authors use a Gaussian Likelihood, but I wonder if you can use mixtures of multiple distributions just like MOIRAI and/or LagLlama do.\n\nThe reviewer raised an important point regarding the expressivity of the fitted probability distributions. In theory, previous work has established the universal approximation property for any conditional density, given sufficient stochasticity introduced early in non-linear models. We aim to mimic this setup in our adapters by incorporating stochastic units in the encoder through Variational Inference (VI) or Dropout as approximate VI. In practice, however, we agree with the reviewer that the Gaussian likelihood may not capture complex, multimodal distributions. Alternatives such as Flow matching or parametric mixtures could be more suitable solutions, and we will explore these in future work.\n\n> How does your method compare with conformal prediction to generate prediction intervals?\n\nWe haven't yet investigated conformal prediction for AdaPTS, though we see its potential as a parallel approach to our Bayesian adapters for probabilistic forecasting. We consider it to be an interesting avenue for future research.\n\n> How efficient are the adapters in terms of parameters and runtime? I want to understand how computationally feasible is this adaptation process.\n\nTo address the reviewer's query about computational efficiency, we provide the following details:\n\n- **Parameter Counts**: Our adapters introduce a minimal number of additional parameters, for instance, in the ETTh1 dataset, the optimal VAE adapter has 2,659 parameters (1 hidden layer with 64 units). This is significantly less than the full 37.9M params of the Moment small foundation model.\n- **Training Time**: As an example, the training time for this VAE adapter on the ETTh1 dataset (13k timesteps, 7 features) with a single V100 (32GB VRAM) GPU is **25min**.\n\nWe appreciate the reviewer's insights and will incorporate the suggested improvements to strengthen our paper.", "claims_and_evidence": "The paper has two claims. The authors aim to leverage **existing pre-trained univariate FMs** to enable **probabilistic** forecasting for **multivariate time series**.\n\nOverall, I believe that the authors present some evidence to support their claims. They evaluate their methods on 4 multivariate long-horizon forecasting datasets, and augment Moment to produce multivariate probabilistic forecasts.\n\nI strong suggest that the authors demonstrate the following, to improve the :\n1. **Multivariate Baselines:** Beyond PCA (which is a excellent choice for a strong baseline), the paper does not compare to some other existing ways of imbuing multivariate context to time series foundation models. Please see [1, 2, 3] for some ways to imbue multivariate context to TSFMs. Some of these methods may be used as baselines. \n2. **More TSFMs**: The authors do mention this as a limitation. I believe that demonstrating their approach on another TSFM, different in design from Moment would make the experiments stronger. I would recommend TTMs (only TSFM not based on the Transformer architecture), or TimesFM (decoder-only, while Moment is encoder-only). \n3. **Datasets:** (1) I would encourage the authors to compare their methods on more datasets. The long horizon forecasting benchmark that the authors used, has more datasets, for example ETTh2, ETTm1, ETTm2, Traffic and Electricity. But even beyond these datasets, the GIFT-Eval benchmark has a lot of multivariate time series useful for forecasting. (2) Also, these existing datasets have known issues. For example, a very strong baseline in the Exhange Rate dataset is predicting the last time step. Also, some studies including [1] have shown that these time series datasets do not significantly benefit from modeling cross-channel dependencies. \n\n### References\n1. Żukowska, Nina, et al. \"Towards Long-Context Time Series Foundation Models With A Handful Of Additional Parameters.\" NeurIPS 2024 Workshop on Fine-Tuning in Modern Machine Learning: Principles and Scalability.\n2. Liu, Mingzhu, Angela H. Chen, and George H. Chen. \"Generalized Prompt Tuning: Adapting Frozen Univariate Time Series Foundation Models for Multivariate Healthcare Time Series.\" arXiv preprint arXiv:2411.12824 (2024).\n3. Lee, Seunghan, Taeyoung Park, and Kibok Lee. \"Partial Channel Dependence with Channel Masks for Time Series Foundation Models.\" arXiv preprint arXiv:2410.23222 (2024).", "methods_and_evaluation_criteria": "The proposed methods and the evaluation criteria make sense. Please see comments in the previous section.", "theoretical_claims": "I did not rigorously verify the proofs or correctness of the claims, but they look reasonable and correct to me.", "experimental_designs_or_analyses": "The experimental design and analysis is sound. Please", "relation_to_broader_scientific_literature": "The paper uses univariate foundation models, and adapts them to model multivariate data and produce probabilistic forecasts. The contributions are simple and easy to understand.", "essential_references_not_discussed": "I have mentioned some key references in the Claims and Evidence section.", "other_strengths_and_weaknesses": "Strengths: The paper solves an important problem. It is well motivated, well-written, and backed with theoretical insights.\nWeaknesses: I think the paper needs a few more experiments to highlight the impact of their contributions.", "other_comments_or_suggestions": "I do not have any other comments or suggestions.", "questions_for_authors": "1. I wonder how flexible is the parametric distribution of the probabilistic predictions. The authors use a Gaussian Likelihood, but I wonder if you can use mixtures of multiple distributions just like MOIRAI and/or LagLlama do. \n2. How does your method compare with conformal prediction to generate prediction intervals?\n3. How efficient are the adapters in terms of parameters and runtime? I want to understand how computationally feasible is this adaptation process.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "eAVOU9k4Xu", "reviewer": "Reviewer_AXte", "summary": "The paper introduces a variation-autoencoder style encoder and decoder around a foundational model to enable it to perform forecasting for probabilistic and multivariate settings.", "questions": "", "limitations": "", "rating": 1, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank Reviewer AXte for their feedback. We appreciate the acknowledgment of our method’s simplicity and validity, as well as the soundness of our theoretical justification.\n\nWe would like to address the concerns raised in the review:\n\n> Lack of enough baselines\n\nOur primary objective is to enhance the forecasting capabilities of a vanilla foundational model. Therefore, it serves as the most relevant baseline for comparison. Additionally, we have explored non-learning-based adaptation approaches such as PCA that is included in the paper, and SVD decomposition and random projections that we did not include as they have comparable results with PCA. \n\n> Lack of base foundational models used (only MOMENT is studied)\n\nWe acknowledge the limitation of evaluating our method on a single foundational model (MOMENT), which we explicitly state in the limitations section of the paper. In response to this concern, we have since experimented with **Moirai** and observed promising results. If the paper is accepted, we will include these findings in the camera-ready version, along with potentially other foundation models as suggested by the other Reviewers (267f).\n\n> Many popular forecasting benchmarks are not used.\n\nOur paper introduces a framework that enables (i) probabilistic forecasting, (ii) channel mixing, and (iii) dimensionality reduction. We believe that there exists no benchmark so far that can clearly highlight each of these aspects, so we tried to choose the datasets from the perspective of diversity application (electicity, medicine, weather and finance). We plan to include more datasets in our study, but identifying the best benchmark for the multivariate setting is an important direction for future work. Meanwhile, on the considered datasets, we validated our claims demonstrating a methodology to maintain or surpass the original foundational model’s performance while using a reduced number of channels and providing uncertainty estimates.\n\nIn summary, we believe our method provides a meaningful contribution to the field by demonstrating how any univariate time-series foundational model can be extended to tackle more complex multivariate probabilistic forecasting tasks. Based on our clarifications, we hope the reviewer acknowledges the value of our contributions and revises their evaluation accordingly.", "claims_and_evidence": "The claim fo the paper is that any time-series univariate foundational model can be adapted to perform much harder problem of multivariate probabilistic forecasting. The paper show some improvements of few benchmarks on the MOMENT model which doesn;t fully validate the claim.", "methods_and_evaluation_criteria": "The evaluation setup is limited to only one foundational model. The datasets used are also very limited.", "theoretical_claims": "The theoretical justification to add the constant diagonla matrix to encoder weight matrix is, while trivial, valid and sound.", "experimental_designs_or_analyses": "The forecasting setup makes sense but the authors only use single foundational model for forecasting accuracy evaluation. The benchmarks are very limited", "relation_to_broader_scientific_literature": "Due to lack of enough experimental validation of the method, the significance of this work on furthering research on foundational time-series models is unclear.", "essential_references_not_discussed": "Relevant papers are discussed.", "other_strengths_and_weaknesses": "Weaknesses:\n1. Lack of enough baselines\n2. Lack of base foundational models used (only MOMENT is studied)\n3. Many popular forecasting benchmarks are not used.\n\nStrengths\n1. The method is simple and looks valid", "other_comments_or_suggestions": "See weaknesses and other comments above", "questions_for_authors": "See weaknesses above", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "HLeHTuuN5T", "reviewer": "Reviewer_NRzf", "summary": "The paper introduces AdaPTS, a framework designed to adapt pretrained univariate time series Foundation Models (FMs) to multivariate probabilistic forecasting tasks. The core challenge addressed is the inherent limitation of existing FMs (e.g., Moment, Chronos), which are typically trained on univariate data and struggle with multivariate dependencies and uncertainty quantification. AdaPTS proposes probabilistic adapters—learned feature-space transformations—that project multivariate inputs into a latent space compatible with univariate FMs, process each dimension independently via the frozen FM, and invert the transformation to produce probabilistic forecasts in the original feature space. \n\n---\nKey conceptual contributions include: \\\n(1) Adapter Framework that is defined as invertible transformations $(φ:R^{D} → R^{D′})$ that map multivariate time series into a latent space where univariate FMs operate. The framework enforces invertibility to ensure predictions can be mapped back to the original space. Two adapter families are also explored, including Deterministic adapters (e.g., linear autoencoders, deep nonlinear autoencoders), and Probabilistic adapters (e.g., $β$-VAE, dropout as approximate variational inference), which introduce stochasticity into the latent space to capture uncertainty.\n\n---\n(2) Methodological Insights, including: (2.1) Decoder Dominance: In deterministic adapters, the decoder contributes more critically to performance than the encoder (ablation study in Fig. 7), (2.2) Hyperparameter Trade-offs: For $β$-VAE, higher $β$ values improve disentanglement and calibration but require careful tuning of the likelihood noise scale ($σ$), and (2.3) Scalability: The framework supports zero-shot adaptation (no FM fine-tuning) and generalizes to variable feature dimensions, making it practical for real-world deployment.\n\n---\nAdaPTS establishes a principled, modular framework for adapting univariate FMs to multivariate settings while enabling uncertainty quantification. By decoupling feature-space transformations from FM parameters, it offers a scalable solution for real-world applications requiring probabilistic forecasts. The integration of Bayesian principles with deep learning architectures positions the work at the intersection of representation learning and time series analysis, with empirical validation across diverse domains underscoring its practical utility.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We deeply appreciate the reviewer’s thoughtful and constructive feedback. The recognition of the theoretical novelty and empirical strengths of our approach is greatly appreciated. \n\nIn this rebuttal, we address the reviewer’s specific concerns and questions in detail to further clarify our methodology, experimental choices, and future directions.\n\n> (1) Foundation Model Generalizability\n\nWe agree that testing AdaPTS across diverse FMs is important for validating generalizability. While our experiments currently focus on Moment, we have already integrated **Moirai** into our framework, which we plan to include in the camera-ready version. AdaPTS is designed to be \"plug-and-play\" with different FMs, and we anticipate similar success with tokenization-based models like Chronos (besides some technical details, like differentiability of the foundation model with respect to its input, which is not guanranteed given the discrete nature of the tokenization operation). Further experimental results with additional FMs will be included to strengthen this claim.\n\n> (2) ExchangeRate Performance Degradation\n\nThe degradation on the ExchangeRate dataset, as mentioned by Reviewer 267f, stems from simple tricks being enough to get a strong baseline, and the potential uselessness of modeling cross-channel dependencies in such datasets. We acknowledge this limitation and will explore more suitable datasets, and benchmarks, where the strengths of our framework will be more easily perceivable.\n\n> (3) Computational Efficiency Quantification\n\nWe appreciate your comment regarding computational efficiency. While we demonstrate dimensionality reduction benefits, we have not exactly quantified inference times or FLOPs. However, as the inference complexity is linear in the number of channels, the inference time gain is a multiplicative factor given the reduced number of channels. Furthermore, in the response to reviewer 267f, we provide the order of magnitudes of the adapter parameter count, and the time it takes to train on the ETTh1 dataset as an example.\n\n> (4) Theoretical Guarantees for Nonlinear Adapters\n\nWhile Proposition 3.4 provides a closed-form solution for linear adapters, we acknowledge that the extension to nonlinear settings has been primarily based on empirical validation. Exploring approximation error bounds or Lipschitz continuity for nonlinear adapters is an exciting future direction, as it would offer a more rigorous understanding of their behavior. However, we believe the linear case analysis provides enough insights to motivate our framework, proving that an optimal solution to the adapter optimization problem exists, beyond the identity baseline.\n\n> (5) Multivariate Architecture Comparisons\n\nWe agree that comparing AdaPTS with other specialized multivariate architectures, like Crossformer and TSMixer, would provide valuable context. However, these baselines result from a different task-specific forecasting paradigm, which we believe is not relevant for our raised problematic, that of adapting foundation models.\n\n> (6) Calibration Remediation Strategies\n\nWe appreciate the reviewer’s attention to calibration degradation for longer forecasting horizons. Indeed, calibration is crucial for the reliability of probabilistic forecasting, and we observed that AdaPTS may experience a degradation in calibration for longer horizons, as shown in Figure 5. To address this issue, several strategies exist in the literature, including the application of temperature scaling or isotonic regression on a held-out calibration set, which are commonly used to improve calibration in such settings.\n\nWe thank the reviewer again for their valuable feedback, and we hope these clarifications answer their questions.", "claims_and_evidence": "The work presents clear, convincing evidence for its primary claims, with rigorously designed experiments and theoretically grounded adapter designs. The few underperforming cases (e.g., ExchangeRate) are contextualized but merit deeper investigation. I am providing detailed assessments of the evidence quality for each major contribution as follows: \n\n---\n(1) Multivariate Foundation Models (FMs) Adaptation via Adapters: The paper provides strong empirical evidence for the core claim that adapters improve multivariate forecasting accuracy. Experiments across four real-world datasets demonstrate consistent MSE improvements over the baseline Moment model in 5/8 tasks (Table 1), with gains of up to 15% on Illness ($H=24$). The synthetic linear FM experiment (Fig. 2) and the nonlinear FM validation (Fig. 8) further corroborate the framework’s ability to learn superior transformations compared to identity/PCA baselines. However, the underperformance on ExchangeRate ($H=192$) remains unexplained beyond dataset-specific characteristics, raising questions about generalizability to high-volatility financial time series. While the authors attribute this to domain shifts, deeper analysis (e.g., feature correlation patterns and noise profiles) would strengthen causal interpretation.\n\n---\n(2) Theoretical Foundations: The closed-form derivation for linear adapters (Proposition 3.4) is mathematically sound under stated assumptions, and the synthetic linear FM experiment (Fig. 2) validates the theory’s practical relevance. However, the analysis does not extend to nonlinear FMs like Moment, where adapters are optimized via gradient descent rather than closed-form solutions. The authors implicitly assume that the linear case provides sufficient intuition for nonlinear regimes, but formal approximation guarantees or Lipschitz continuity analyses would bridge this gap. The Bayesian treatment of probabilistic adapters (Proposition 4.1) follows standard variational inference principles, though the absence of posterior contraction rates or PAC-Bayes bounds limits theoretical novelty.\n\n---\n(3) Uncertainty Quantification: Calibration results (Fig. 5) show that probabilistic adapters produce reasonably calibrated forecasts for short horizons but exhibit overconfidence as $H$ increases. While this aligns with known challenges in long-horizon forecasting, the evaluation is restricted to LinearVAE on ETTh1. A broader analysis across adapter types and datasets (e.g., Weather, Illness) would better substantiate the claim. Furthermore, the lack of comparative baselines (e.g., MC dropout applied directly to Moment) makes it unclear whether the calibration improvements stem from the adapter architecture or merely the introduction of stochasticity.\n\n---\n(4) Dimensionality Reduction: The claim that adapters enable cost-effective inference is supported by experiments showing that VAE achieves optimal performance with $2$ latent dimensions on Illness (Fig. 3), retaining $95.6%$ explained variance. However, the paper does not quantify computational savings (e.g., FLOPs reduction vs. accuracy trade-offs) or compare against alternative compression techniques (e.g., pruning, quantization). This weakens the practical impact argument for resource-constrained deployments.\n\n---\n(5) Latent Space Interpretability: The visualization of latent representations (Fig. 4) provides qualitative evidence that VAE adapters mitigate distribution shifts by enforcing isotropic Gaussian latent spaces. However, the absence of quantitative metrics (e.g., Maximum Mean Discrepancy between train/test embeddings) limits the strength of this claim. Additionally, the analysis does not explore whether the structured latent space improves robustness to adversarial perturbations or out-of-distribution inputs.\n\n---\n(6) Ablation Studies: The hyperparameter analysis (Fig. 6) and component ablation (Fig. 7) are methodical, revealing key insights, such as Decoder dominance in deterministic adapters suggests that feature recombination is more critical than encoding for forecasting, and Higher $β$ in $β$-VAE improves disentanglement and calibration but requires careful tuning of $σ$. While informative, the ablation scope is narrow. For instance, the study does not explore the impact of adapter depth/width in nonlinear architectures or the role of pretraining data diversity in FM compatibility.\n\n---\nIt is also worth mentioning that the authors transparently acknowledge limitations, including (1) Restriction to Moment (The framework’s compatibility with other FMs (e.g., Chronos, Moirai) remains unverified), (2) Calibration gaps (No post-hoc recalibration or adaptive noise scaling is attempted), and (3) Normalizing Flows (Optimization challenges with invertible flows are noted but not resolved).", "methods_and_evaluation_criteria": "The methodological and evaluation framework of AdaPTS demonstrates substantial technical merit in addressing multivariate probabilistic forecasting via univariate foundation models (FMs), though certain design choices invite deeper scrutiny. I am providing a critical analysis of its alignment with the problem's requirements. \n\n---\nMethodological Appropriateness: The core innovation - probabilistic adapters as invertible latent-space transformations - directly tackles the dimensionality mismatch between univariate FMs and multivariate inputs. By decoupling feature-space projections ($φ$) from FM inference, AdaPTS achieves three critical objectives: (1) Zero-shot compatibility: Avoids FM fine-tuning, preserving pretrained representations while adapting to new tasks - a necessity given the computational impracticality of retraining large FMs. (2) Uncertainty propagation: Stochastic adapters (e.g., LinearVAE) inject Bayesian principles into deterministic FMs like Moment, enabling probabilistic forecasts without architectural changes. (3) Dimensionality reduction: Learned latent spaces (e.g., VAE with $D' = 2$ on Illness) reduce inference costs while preserving performance, addressing real-world deployment constraints. \nThe theoretical analysis for linear adapters (Proposition 3.4) is mathematically rigorous under stated assumptions, with synthetic experiments (Fig. 2) validating closed-form solutions outperforming identity/PCA baselines. However, the extension to nonlinear FMs relies on gradient-based optimization without formal guarantees (e.g., Lipschitz continuity or approximation bounds), leaving a gap between linear theory and nonlinear practice. While common in deep learning, explicit analysis of how nonlinear adapter architectures interact with FM inductive biases (e.g., Moment’s transformer backbone) would strengthen claims of generalizability.\n\nThe Bayesian treatment of adapters via variational inference (Proposition 4.1) aligns with best practices but introduces limitations such as $β$-VAE’s isotropic priors oversimplifying latent dependencies, potentially limiting cross-channel interaction modelling (evidenced by degraded performance on ExchangeRate), and Calibration gaps for long horizons (Fig. 5) suggest overconfidence, yet the absence of comparisons against ensemble methods or deep kernel learning obscures whether improvements stem from adapter architecture or stochasticity alone.\n\n---\nEvaluation Criteria Strengths and Gaps: The benchmark selection (ETTh1, Illness, Weather, ExchangeRate) spans energy, healthcare, and finance, ensuring domain diversity. However, key limitations emerge: (1) Temporal distribution shifts: While latent-space visualizations (Fig. 4) qualitatively demonstrate mitigated shift, quantitative metrics (e.g., Maximum Mean Discrepancy) are absent, weakening robustness claims. (2) Volatility analysis: ExchangeRate’s underperformance (Table 1) is attributed to volatility but lacks analysis of heteroskedasticity or leverage effects—critical in financial data. A volatility-aware adapter ablation would clarify failure modes. (3) Metric limitations: MSE/MAE focuses on point forecasts, neglecting probabilistic metrics like CRPS or sharpness. Reliability diagrams (Fig. 5) assess calibration but omit quantile coverage statistics, limiting uncertainty quantification depth. \nBaseline comparisons against PCA and identity adapters are informative but incomplete. State-of-the-art multivariate transformers (e.g., Crossformer) and FM-specific adaptations (e.g., Moirai) are noted but not benchmarked, leaving open questions about relative performance. For instance, Crossformer’s explicit cross-channel attention could outperform AdaPTS’s latent-space projections on highly interdependent features.\n\n---\nEmpirical Validation and Practical Impact: Experiments demonstrate consistent improvements over Moment in 5/8 tasks (Table 1), with VAE adapters reducing Illness MSE by $15%$ ($2.902 → 2.461$). However, the framework’s computational overhead is unquantified: while dimensionality reduction (Fig. 3) suggests efficiency gains (e.g., optimal performance with 2 latent dimensions on Illness), wall-clock time or FLOPs comparisons against native multivariate FMs are omitted. This weakens claims about deployability, as adapters introduce additional training/inference costs despite latent compression.\n\n---\nTheoretical and Practical Trade-offs: The linear adapter analysis assumes full-rank $W_φ$ and linear FMs, but real-world FMs (e.g., Moment’s transformer) are nonlinear. A perturbation analysis (e.g., Lipschitz constants for $φ$/FM compositions) or approximation error bounds would bridge this gap. Similarly, the decoder’s dominance in deterministic adapters (Fig. 7) highlights feature recombination as critical, but the absence of adversarial robustness tests (e.g., input perturbation sensitivity) limits insights into representation stability.", "theoretical_claims": "From my understanding, the theoretical analysis in AdaPTS focuses on two primary proportions: (1) Proportion 3.4 (optimal linear adapter derivation) and (2) Proposition 4.1 (VAE adapter training objective). Now, I am assessing their correctness, assumptions, and limitations, which are aligned with ICML's standard guidelines. \n\nProportion 3.4: Optimal Linear Adapter: The authors claim that for linear foundation models and linear adapters under full-rank assumptions, the closed-form optimal solution is $ W_{φ}^{∗} = (B^{⊤}A)^{+}B^{⊤}B $, where $ A = Y - W_{FM} ^ {⊤}X $ and $ B = b_{FM}1^{⊤} $. After checking the derivation steps and assumptions provided by the authors, I found them reasonable, but I still think this claim has some limitations. One of them is the numerical stability. The regularization term $ λI $ added to $ B^{⊤} A $ (Remark 3.5) is not explicitly justified in the proof but is standard practice to avoid singular matrices. The next limitation is generalizability. The closed-form solution assumes a linear Foundation Model, but real-world foundation models (e.g., Moment's transformer) are nonlinear. The authors acknowledge this gap but do not provide approximate bounds or Lipschitz continuity arguments for nonlinear extensions. In conclusion, I think the proof is mathematically correct under stated assumptions but lacks theoretical guarantees for nonlinear Foundation models. It'd be nice to hear back from the authors about these limitations. \n\n---\nProposition 4.1: VAR Adapter Training Objective: The authors claim that the training objective for VAE adapters maximizes an ELBO-like lower bound. While they are providing easy-to-understand derivation steps and assumptions, I think this claim suffers from two limitations, including Identifiability. Their proof does not address whether the latent representation $ Z $ uniquely identifies the input $ X $, a known challenge in VAEs. Another limit would be approximation quality. The variational posterior $ q_{ϕ} (Z ∣ X) $ is assumed to be flexible enough to approximate the true posterior, but no PAC-Bayes or approximation error bounds are provided. To conclude, The ELBO derivation is technically correct but lacks novelty and fails to address critical Bayesian challenges (e.g., posterior contraction rates). Please note that my concern here is not novelty ( While it is essential ), but I'd like to hear back from the authors about this limitation, and it would be nice to address it further.", "experimental_designs_or_analyses": "The experimental design and analysis in AdaPTS exhibit careful construction with notable strengths. I would like to provide a couple of the methodological issues that I think should be addressed further. \n\n---\nFirst, the evaluation framework lacks comprehensive uncertainty quantification metrics, which undermines the paper's probabilistic claims. While reliability diagrams (Fig. 5) provide qualitative insights into calibration, the analysis is limited to LinearVAE on ETTh1 with no quantitative metrics like CRPS or proper scoring rules. For a paper emphasizing probabilistic forecasting, this represents a significant gap—particularly as calibration deteriorates for longer horizons without proposed remediation strategies. An outstanding experimental design would systematically evaluate calibration across all adapter variants and datasets, benchmark against alternative uncertainty quantification methods (e.g., ensemble approaches), and quantify miscalibration using established metrics.\n\n---\nSecond, the baseline comparison framework insufficiently contextualizes AdaPTS within the multivariate forecasting landscape. The experimental design evaluates against vanilla Moment and PCA adapters but omits comparisons with state-of-the-art multivariate models (e.g., Crossformer, TSMixer) that explicitly model cross-channel dependencies. This limitation is particularly notable for ExchangeRate, where adapters underperform the baseline—suggesting that certain multivariate dependencies require specialized architectures. Without these comparisons, it remains unclear whether adapter-based approaches offer advantages over dedicated multivariate architectures beyond computational convenience.\n\n---\nThird, despite claims of \"cost-effective inference through dimensionality reduction\" (Fig. 3), the experimental design lacks rigorous efficiency analysis. While results demonstrate performance preservation with reduced dimensions (e.g., VAE achieving optimal performance with 2 latent dimensions on Illness), no quantitative measurements of computational savings (FLOPs, wall-clock time) or adapter overhead are provided. This omission prevents objective assessment of the practical utility claims, especially since adapter training introduces additional computational costs that may offset inference savings. A more complete analysis would quantify these trade-offs across different adapter architectures and deployment scenarios.", "relation_to_broader_scientific_literature": "First, the framework's approach to time series foundation model adaptation through invertible feature-space transformations builds upon but significantly extends prior adapter methodologies from adjacent fields. While previous work demonstrated the utility of simple transformations like PCA for classification tasks and model-based reinforcement learning, these approaches yielded limited improvements for forecasting tasks. AdaPTS diverges from these precedents by enforcing an invertibility constraint that enables bidirectional transformation between input and latent spaces, a crucial innovation for forecasting. This represents a fundamentally different architectural approach compared to alternatives like Moirai, which handles multivariate inputs through flattened channels but suffers from quadratic memory complexity as dimensionality increases. The paper's systematic comparison between learned transformations and static ones (e.g., PCA) demonstrates the limitations of prior art in capturing the complex, task-dependent relationships in multivariate forecasting.\n\n---\nSecond, the theoretical foundations developed for adapters connect previously disparate research threads in representation learning and uncertainty quantification. The closed-form solution for linear adapters (Proposition 3.4) provides a mathematical foundation that explains when and why adapters can outperform identity mappings—addressing a key gap in the empirical findings. More significantly, the paper's Bayesian treatment of adapters extends the partially stochastic Bayesian neural network framework to the specific constraints of foundation model adaptation. While prior work established dropout as an approximate variational inference, AdaPTS uniquely applies this principle to the adapter component specifically, enabling uncertainty quantification without modifying the underlying foundation model architecture. This creates a new bridge between deterministic foundation models like Moment and the probabilistic forecasting literature that traditionally required specialized probabilistic architectures.\n\n---\nThird, the empirical validation offers insights that challenge assumptions in dimensionality reduction for time series. While the standard approach in multivariate time series processing often preserves original dimensionality, AdaPTS demonstrates that performance can be maintained or even improved with dramatically reduced latent dimensions (e.g., VAE achieving optimal performance with just 2 latent dimensions on Illness data that originally had 7 features). This finding substantially extends prior work on time series representation learning by showing that learned nonlinear projections can capture cross-channel dependencies more efficiently than traditional methods. Furthermore, the observed decoder dominance in deterministic adapters (Fig. 7) challenges the symmetrical encoder-decoder paradigm prevalent in representation learning literature, suggesting that feature recombination plays a disproportionately important role in multivariate forecasting—a phenomenon not previously documented in time series foundation model research.", "essential_references_not_discussed": "N/A", "other_strengths_and_weaknesses": "Regarding originality, the paper makes a genuinely novel contribution by reframing the multivariate forecasting problem through the lens of feature-space transformations. While adapter architectures have been explored in natural language processing and computer vision, the paper's invertibility constraint introduces a fundamentally different approach tailored to time series data. Most remarkably, the theoretical formulation of adapter optimality conditions (Proposition 3.4) extends beyond empirical validation to provide mathematical intuition for why and when adapters outperform the direct application of foundation models. This theoretical-empirical synergy distinguishes the work from purely engineering-driven approaches in the time series literature. The reconceptualization of dropout as approximate variational inference, specifically within the adapter framework, further demonstrates the creative adaptation of existing techniques to the unique constraints of time series foundation models.\n\n---\nThe paper's significance, while substantial, is somewhat constrained by the experimental scope. The focus on Moment as the sole foundation model leaves open questions about generalizability across the rapidly evolving landscape of time series foundation models. This limitation is particularly relevant given the architectural diversity among recent models like Chronos (tokenization-based) and Moirai (mixture-of-experts). Additionally, while the paper convincingly demonstrates improved forecasting accuracy on standard benchmarks, it misses an opportunity to explore challenging real-world scenarios where distributional shifts are more severe or where domain-specific characteristics (such as financial volatility clustering in ExchangeRate) dramatically affect performance. Such extensions would significantly strengthen the practical impact claims.\n\n---\nClarity represents both a strength and weakness. The mathematical formalism is precise and well-structured, with propositions clearly stated and adequately proven. The visualization of latent representations (Figure 4) effectively communicates the distribution shift mitigation benefits of probabilistic adapters. However, the paper occasionally suffers from terminology inconsistencies—notably in the interchangeable use of \"channels,\" \"features,\" and \"components\"—which, while footnoted, creates unnecessary cognitive load. The experimental section would benefit from a clearer exposition of hyperparameter sensitivity, particularly regarding how adapter dimensionality affects computational savings. A quantitative analysis of inference time or FLOP reductions would provide concrete evidence for the \"cost-effective adaptation\" claims. Finally, the discussion of failure cases (especially on ExchangeRate) remains somewhat superficial, missing an opportunity for deeper insights into adapter limitations.\n\n---\nIn conclusion, I think this work represents a significant conceptual advancement in time series foundation model adaptation, with strong theoretical underpinnings and promising empirical results that could substantially impact how practitioners deploy these models in real-world multivariate forecasting tasks.", "other_comments_or_suggestions": "N/A", "questions_for_authors": "I have a couple of questions from the authors and I am eager to hear back from them during the rebuttals. I would be happy to change my score if the answers are convincing. Here are the questions:\n\n---\n(1) Foundation Model Generalizability: Your experiments exclusively use Moment as the base FM. How would AdaPTS perform with architectures like Chronos (tokenization-based) or Moirai (mixture-of-experts)? Maybe an empirical validation across diverse FM architectures would strengthen the generalizability claims, while evidence of architecture-specific limitations. I'd like to see the authors' thoughts on it in general. \n\n---\n(2) ExchangeRate Performance Degradation: The VAE adapter shows significantly worse performance than the baseline Moment on the ExchangeRate dataset (0.455 vs. 0.130 MSE). Could you elaborate on the specific characteristics of financial time series that challenge your method, particularly regarding heteroskedasticity or leverage effects? \n\n---\n(3) Computational Efficiency Quantification: While you demonstrate dimensionality reduction benefits (e.g., optimal performance with 2 latent dimensions on Illness), no wall-clock time or FLOPs comparisons are provided. Could you quantify the actual computational savings, including adapter overhead? Concrete efficiency metrics would transform the \"cost-effective inference\" claim from theoretical to practical, substantially strengthening the real-world applicability argument.\n\n---\n(4) Theoretical Guarantees for Nonlinear Adapters: Proposition 3.4 provides a closed-form solution for linear adapters with linear FMs, but the extension to nonlinear settings relies solely on empirical validation. Have you explored approximation error bounds or Lipschitz continuity properties for nonlinear adapter compositions with nonlinear FMs? \n\n---\n(5) Multivariate Architecture Comparisons: Your baselines focus on adapter variants rather than comparing against dedicated multivariate forecasting architectures (e.g., Crossformer, TSMixer). How does AdaPTS compare to state-of-the-art multivariate models that explicitly model cross-channel dependencies? Such comparisons would clarify whether adapter-based approaches offer advantages beyond computational convenience—particularly for datasets like ExchangeRate where adapters underperform.\n\n---\n(6) Calibration Remediation Strategies: Figure 5 demonstrates increasing calibration degradation for longer horizons. What post-hoc recalibration methods or architectural modifications did you consider to address this systematic overconfidence?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {"ethical_review_concerns": "N/A"}}, {"review_id": "RZWv4inSQM", "reviewer": "Reviewer_Qhdv", "summary": "The paper presents AdaPTS, a novel framework for adapting pre-trained univariate foundation models (FMs) to probabilistic multivariate time series forecasting. AdaPTS introduces adapters—feature-space transformations that project multivariate series into latent spaces, where predictions are made independently by the frozen FM. Results are inverted back via decoders. This approach improves forecasting accuracy, uncertainty quantification, and robustness across benchmarks. The paper demonstrates strong empirical results, showing AdaPTS consistently outperforms baseline methods, provides meaningful uncertainty estimates, and effectively reduces dimensionality. Conceptually, AdaPTS bridges representation learning with Bayesian inference, enhancing FM adaptability and interpretability.", "questions": "", "limitations": "", "rating": 1, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate the reviewer’s detailed and insightful feedback. We are particularly grateful for the recognition of our work’s strong empirical results, the effectiveness of AdaPTS in improving forecasting accuracy and uncertainty quantification.\n\nWe would like to clarify and address the raised concerns:\n\n> However, there are not many details on the reproducibility of the results, where the provided link to the code does not work.\n\nAs stated in the reproducibility section of our paper, the code will be made publicly available upon acceptance. We used the placeholder URL (\"URL hidden for review\") as a mean to maintain anonymity during the review process, and we apologize for any confusion due to this practice. However, we have provided relevant implementation details in Appendix C.2 to ensure transparency and facilitate reproducibility.\n\n> (1) the authors didn't specify how the feature interactions can be handled when adapting the univariate forecasting model to a multivariate model\n\nAs defined in our paper (Definition 3.1), feature-space transformations ($\\varphi$) play a crucial role in capturing channel dependencies. Applying $\\varphi$ to a multivariate time series projects the data into a latent space where each component is a nonlinear function of the original features (when using a nonlinear encoder). For instance, PCA—a baseline in our study—transforms data into a space of linearly uncorrelated components, demonstrating how our approach inherently manages multivariate dependencies. This mechanism is central to AdaPTS and a key aspect of our contribution to multivariate time-series forecasting.\n\n> (2) there is not much technical innovation in the proposed method: it basically added an encoder and decoder before and after frozen foundation models followed by applying techniques such as variational inference on obtained embeddings for uncertainty intervals.\n\nWhile we acknowledge that autoencoders and variational inference are established techniques, our contribution is methodological and practical. The novelty of AdaPTS lies in the formalization of the probabilistic multivariate time-series adaptation problem, the design choices behind our framework, and the comprehensive analysis of its effectiveness. Multivariate and probabilistic forecasting remain central challenges in time-series research, and our work provides a principled and practical approach that can be applied to real-world forecasting tasks. We believe this adds significant value to the time series community.\n\nGiven our clarifications, we hope the reviewer recognizes the merit of our contributions and considers adjusting their evaluation accordingly. We appreciate the constructive feedback and welcome further discussion on how AdaPTS enhances the field of foundation models for time-series forecasting.", "claims_and_evidence": "The paper provides clear evidence supporting its claims about AdaPTS’s improved forecasting accuracy and uncertainty quantification, demonstrated across multiple datasets and adapter configurations. Calibration results indicate room for improvement in uncertainty estimation at longer horizons, weakening the robustness claim.", "methods_and_evaluation_criteria": "The proposed AdaPTS methods and evaluation criteria are well-aligned with the problem of adapting univariate foundation models for probabilistic multivariate time series forecasting. The set of real-world datasets is the standard benchmark in the forecasting literature.", "theoretical_claims": "Yes. The authors have provided proofs for prop 3.4 and 4.1 for linear and VAE adapters. They both followed standard linear algebra and variational inference principles.", "experimental_designs_or_analyses": "Yes, the experimental designs are sound. However, there are not many details on the reproducibility of the results, where the provided link to the code does not work.", "relation_to_broader_scientific_literature": "The proposed method directly builds upon recent advancements in foundation models like Moment and Chronos, designed primarily for univariate forecasting. It extends ideas from the literature on adapters used in other domains, such as PCA-based adapters (Feofanov et al., 2024; Benechehab et al., 2025), to enable multivariate forecasting. It also integrates concepts from probabilistic representation learning, leveraging Bayesian neural network ideas (Gal & Ghahramani, 2016) and variational autoencoders (Kingma & Welling, 2013) to quantify uncertainty.", "essential_references_not_discussed": "None", "other_strengths_and_weaknesses": "Weaknesses: (1) the authors didn't specify how the feature interactions can be handled when adapting the univariate forecasting model to a multivariate model; (2) there is not much technical innovation in the proposed method: it basically added an encoder and decoder before and after frozen foundation models followed by applying techniques such as variational inference on obtained embeddings for uncertainty intervals.", "other_comments_or_suggestions": "Please address the comments in the weakness section.", "questions_for_authors": "None", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "GyDHnpSkAQ", "reviewer": "Reviewer_4t4s", "summary": "The paper introduces AdaPTS, a framework to adapt pre-trained time series models (univariate) for probabilistic multivariate forecasting. The authors use adapters to project multivariate inputs into a latent space where a frozen pre-trained model is applied independently to each channel. To enforce the invertibility, the authors adopt several types of autoencoders. They provide theoretical analysis of linear adapters and extend the framework for probabilistic forecasting using Bayesian inference. Empirical results on synthetic and real-world datasets demonstrate that AdaPTS improves the accuracy and uncertainty quantification based on a pre-trained model Moment.", "questions": "", "limitations": "", "rating": 1, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We would like to thank Reviewer 4t4s for their detailed feedback and constructive comments. We now address the concerns raised in their review:\n\n> Claim in L147: no requirement of fine-tuning due to feature-level transformations\n\nThe claim in line 147 regarding \"no requirement of fine-tuning\" pertains specifically to the *pre-trained weights* of the foundation model. As shown in Figure 1b, the weights of the foundational model are kept frozen, while only the lightweight adapter (which is significantly smaller in parameter count compared to the FM) is trained.\n\n> Evaluations in Table 1 are not comprehensive:\n\nWe agree with the reviewer that the evaluation in Table 1 could be expanded. Our choice of the Moment model for validation was motivated by its widespread use in the literature. However, we are actively expanding the range of foundation models considered, with **Moirai** already incorporated into our framework. Additionally, the datasets and forecasting horizons chosen for the evaluation are not omitted but rather reflect a careful selection that highlights the most relevant aspects of our framework. We do recognize the importance of a more comprehensive evaluation and will consider incorporating additional datasets and horizons in future work.\n\n> Lack of probabilistic metrics\n\nOur current evaluation focuses on comparing AdaPTS to the vanilla Moment model, which provides point forecasts. As probabilistic metrics are ill-defined for deterministic predictors, we chose to report the MSE for Table 1. Neverthless, we use calibration metrics such as the ECE and the reliability diagram (Figures 5 and 6), to evaluate the probabilistic aspect of our approach. We plan to include more probabilistic foundation models, like **Moirai**, and will incorporate the reviewer's suggestion to report probabilistic metrics for these models.\n\n> UP2ME\n\nWe thank the reviewer for bringing up UP2ME. While we agree that UP2ME is a relevant work, we believe there are significant differences in our approaches. UP2ME focuses on pre-training a model on univariate time series from a given task, then fine-tuning on multivariate data from the same task. AdaPTS in contrast, aims to provide a plug-and-play adapter that can be applied to any univariate foundational model without fine-tuning.\n\n> Assumption 3.3\n\nIn the linear case analysis, the invertibility condition (Assumption 3.2) is imposed on the adapter matrix $W_\\varphi$, while Assumption 3.3 pertains to the linear parameterization of the foundation model’s predictor, which does not need to be invertible. For autoencoders, the inverse transformation is learned through gradient-based optimization rather than being explicitly imposed. We discuss potential extensions, such as Normalizing Flows, which could be explored to create inherently non-linear and invertible adapters. The primary goal of the linear analysis was to demonstrate that our approach offers solutions better than the identity baseline (the vanilla FM), as evidenced by Proposition 3.4 and Figure 2.\n\n> (1) The proposed method may lack novelty\n\nWe refer the reviewer to our reponse to reviewer Qhdv.\n\n> LoRA can also enhance existing pre-trained models\n\nWe would like to notice that there is probably confusion between us and the reviewer in terms of terminology, because by an adapter, we call a block that operates along the feature dimension and preceeds the frozen foundation model with the goal to model channel interdependence and potentially reduce the dimension. In contrast, LoRA fine-tunes the **Foundation Model’s weights**, so it doesn't solve the problem of adapting the model to the multivariate setting. Thus, AdaPTS and LoRA are rather complementary to each other and can be used together, if necessary. In our case, we focus on the frozen foundation model, but performing experiments where FM's model weights are fine-tuned together with an adapter is a good direction for future work.\n\n> some adapters degrade performance\n\nThe differences in performance between adapters are due to the variations in their architecture (linear vs non-linear, deep vs shallow) and stochastic strategy (VI vs Dropout Vs deterministic). This makes their performance sensitive to the specific task, data characteristics, and convergence of the optimization algorithm during training.\n\n> longer-horizon forecasts tend to underestimate uncertainty\n\nCalibration is a critical aspect of forecasting, especially for longer horizons. Techniques like temperature scaling on a held-out calibration set can be effective in improving calibration and mitigating underestimation of uncertainty at longer horizons.\n\n> applying AdaPTS to other univariate FMs\n\nWe have integrated **Moirai** with our framework and are considering additional foundation models upon acceptance of the paper.\n\nWe believe these clarifications address the reviewer’s concerns, and we would be grateful if the reviewer could reconsider their score in light of the additional insights provided.", "claims_and_evidence": "> Claim in L147: no requirement of fine-tuning due to feature-level transformations\n\nFrom Figure 1, AdaPTS still needs to train learnable transformations respectively on time series with different variate numbers.", "methods_and_evaluation_criteria": "* Evaluations in Table 1 are not comprehensive: (1) while the author claimed that the proposed method \"can be plugged in any foundation model\", only one type of pre-trained model is adapted in Table 1. (2) Some baseline datasets are omittet, such as ECL, Traffic, and other ETT subsets. (3) Prediction lengths of {96, 196, 336, 720} are commonly evaluated in previous works, but the forecasting horizon in this paper only includes {96, 196}, \n\n* Lack of probabilistic metrics (e.g., MASE and WQL) in evaluation. I'm confused about whether AdaPTS has empowered the pre-trained model with the capability of probabilistic forecasting. How about the performance?\n \n* The evaluation does not explicitly demonstrate that the pre-trained model can benefit from the multivariate modeling. A support comes from the mutable performance when applying different autoencoders in Table 1.\n\n* Lack of comparison with related works like UP2ME: Univariate Pre-training to Multivariate Fine-tuning as a General-purpose Framework for Multivariate Time Series Analysis.", "theoretical_claims": "While the authors aim to enforce the invertibility in adapters (Definition 3.1). Assumption 3.3 uses the linear parametrization with the bias term that has no explicit inverse. Also, these derivations based on linear weighting do not contribute much to the paper because the proposed method does not adopt a simple linear layer. Does the proposed method explicitly maintain the invertibility in the learned autoencoders?", "experimental_designs_or_analyses": "See Methods And Evaluation Criteria.", "relation_to_broader_scientific_literature": "The paper focuses on adapting pre-trained time series models. The authors position their contributions in the usage of adapters to leverage pre-trained FMs for multivariate and probabilistic tasks.", "essential_references_not_discussed": "The authors have adequately discussed related works.", "other_strengths_and_weaknesses": "* Strength: The paper discussed an important problem in adapting univariate pre-trained models for multivariate probabilistic forecasting.\n\n* Weakness: (1) The proposed method may lack novelty, since the authors adopt existing AEs wihtout further adaptations. (2) I am not clear about the advantage of the proposed method compared to previous works. For example, UP2ME and LoRA can also enhance existing pre-trained models and both of them and AdaPTS still requires task-specific fine-tuning. The outcome model may not be applicable for zero-shot forecasting. (3) No evaluation is provided to demonstrate the claimed efficiency of the proposed methods. (4) The performance in Table 1 is mutable when using different AEs.", "other_comments_or_suggestions": "See above.", "questions_for_authors": "* Results in Table 1 show that some adapters degrade performance. Could the authors provide a more detailed analysis of why this happens?\n\n* The calibration results indicate that longer-horizon forecasts tend to underestimate uncertainty. Are there any strategies the authors could suggest to improve calibration for longer horizons?\n\n* The paper focuses on Moment as the foundation model. Have the authors considered applying AdaPTS to other univariate FMs?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: This paper has been assessed by five knowledgeable reviewers. Two of them supported its acceptance (score of 4), the other three opted for straight rejection (score of 1). The authors provided a rebuttal and engaged the reviewers who offered positive scores in a discussion which led to increases of the initial scores. Effectively, the key concerns of those reviewers have been addressed by the authors. However, the reviewers who recommended a rejection did not respond to the rebuttal, even though they have acknowledged seeing it. I read the paper and carefully analyzed the reviews and the discussion between the authors and those two reviewers who were engaged. I found the paper to be valuable and confirmed that the main concerns voiced by any of the reviewers were addressed by the authors quite satisfiably via rebuttal and discussion. The negative reviews were curt and appeared to have missed the key messages of the paper (which, by the way, should signal to the authors the need to improve the way they lay down their story so that it could be accessible and understandable to a broader audience).The positive reviews were very thorough and comprehensive, demonstrating high levels of competence of these reviewers and their understanding of the material at hand. Considering all factors at play, I recommend acceptance of this paper. It does bring a new and pragmatic perspective on handling multivariate data with natively univariate time series foundation models that I believe should be presented at ICML.
Decision: Accept (poster)