$K^2$VAE: A Koopman-Kalman Enhanced Variational AutoEncoder for Probabilistic Time Series Forecasting
Authors: Xingjian Wu
Xiangfei Qiu
Hongfan Gao
Jilin Hu
Bin Yang
Chenjuan Guo
Venue: ICML 2025 spotlightposter (2025)
Keywords: Time Series Probabilistic Forecasting
Primary area: deep_learning->sequential_models_time_series

Abstract: Probabilistic Time Series Forecasting (PTSF) plays a crucial role in decision-making across various fields, including economics, energy, and transportation. Most existing methods excell at short-term forecasting, while overlooking the hurdles of Long-term Probabilistic Time Series Forecasting (LPTSF). As the forecast horizon extends, the inherent nonlinear dynamics have a significant adverse effect on prediction accuracy, and make generative models inefficient by increasing the cost of each iteration. To overcome these limitations, we introduce $K^2$VAE, an efficient VAE-based generative model that leverages a KoopmanNet to transform nonlinear time series into a linear dynamical system, and devises a KalmanNet to refine predictions and model uncertainty in such linear system, which reduces error accumulation in long-term forecasting. Extensive experiments demonstrate that $K^2$VAE outperforms state-of-the-art methods in both short- and long-term PTSF, providing a more efficient and accurate solution.

K²VAE: A Koopman-Kalman Enhanced Variational AutoEncoder for Probabilistic Time Series Forecasting
Xingjian Wu "1
Xiangfei Qiu
Hongfan Gao
Jilin Hu
Bin Yang
Chenjuan Guo
Abstract
0.6
Probabilistic Time Series Forecasting (PTSF) plays a crucial role in decision-making across vari- ous fields, including economics, energy, and trans- portation. Most existing methods excell at short- term forecasting, while overlooking the hurdles of Long-term Probabilistic Time Series Forecast- ing (LPTSF). As the forecast horizon extends, the inherent nonlinear dynamics have significant ad- verse effect prediction accuracy, and make gen- erative models inefficient by increasing the cost of each iteration. To overcome these limitations, we introduce VAE, an efficient VAE-based gener- ative model that leverages a KoopmanNet to trans- form nonlinear time series into a linear dynamical system, and devises KalmanNet refine predic- tions and model uncertainty in such linear system, which reduces еттоr accumulation in long-term forecasting. Extensive experiments demonstrate that VAE outperforms state-of-the-aa methods in both short- and long-term PTSF, providing a more efficient and accurate solution.
0.5
CRPS
0.4
0.2
12
24
48
96
192
336
720
Forecasting Horizon
FITS
PatchTST
iTrnnformer
GRU MAF
TImeGrad
CSDI
Figure I. We compare three native probabilistic forecasting mod- els including GRU MAF, TimeGrad. and CSDI with three point forecasting models equipped with distributional heads including FITS, PSt and ITransformer ETTh1. Longer forecast- ing horizons lead to rapid collapse CRPS metric (lower better) probabilistic forecasting models, even worse than point forecasting models.
provides significant support for decision-making in various fields such as economics (Sezer et al., 2020; Huang et al., 2022b), traffic (Wu et al., 2024b; 2025d;
1. Introduction
Pan et al., 2023a; Cirstea et al., 2022b; Fang et al., 2021; Yang et al., 2021; 2022), energy (Wang et al., 2024d; Guo et al., 2015; Sun et al., 2022), and AIOps (Lin et al., 2024a; Campos et al., 2022; Chen et al., 2023; Pan al., 2023c: Lin et al., 2025). In these practical applications, an urgent need extend the prediction time the distant future, known Long- term Probabilistic Time Series Forecasting (LPTSF), which is highly meaningful for long-term planning and carly warn- ing. Most existing methods excell at short-term problem settings, such as predicting up to 48 steps or fewer (Rasul et al., 2021as a; Kollovieh et al., 2023; Rasul al., 2021b), while directly applying these methods long-term forecast- ing tasks often results in poor performance Figure 1.
In recent years, time series analysis has seen remarkable progress, with key tasks such anomaly detection (Wang et al., 2023; Liu & Paparrizos, 2024; Miao et al., 2025; Hu et al., 2024; Wu et al., 2025c), classification (Yao et al., 2024; Campos et al., 2023), and imputation (Gao et al., 2025; Wang et al., 2024a;c; Yu et al., 2025a), among others (Wang et al., 2024b; Miao et al., 2024a; Liu et al., 2025a; Huang al.. 2023: Yao al., 2023), gaining attention. Among these, Probabilistic Time Series Forecasting (PTSF) is crucial and widely studied task. By quantifying the stochas- tic temporal evolutions of multiple continuous variables, it
Eguual contribution 'School of Data Science and Engineering, East China Normal University, Shanghai, China. Correspondence to: Bin Yang <byang @ www..ccn...c.o... senu.edu.cn.
However, probabilistic forecasting faces numerous chal- lenges in long-term forecasting tasks. First, the inherent nonlinearity of time series challenges probabilistic mod- els in modeling dynamic evolution. Due to factors such as consstationarity and complex interdependencies between
Proceedings of the ndnd International Conference Machine Learning. Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

A Koopman-Kalmn Enhanced Variational AutoEncoder for Probabilistic Time Series Forccasting
variables, time series typically exhibit nonlinear characteris- ties, complicating the construction of probabilistic models. Specifically, the nonlinearity makes it difficull for these models to derive simple equation that precisely describes the state transition process. As a result, the uncertainties within the models are also hard to quantify, particularly in long-term forecasting tasks. Second, as the forecasting horizon extends, the accuracy and efficiency become major bottlenecks. Longer foreacasting horizons lead to more in- tricate target distributions, which causes remarkable error accumulation. It also makes the diffusion-baser (K (Kollovieh et al., 2023; Rasul et al., a)) or flow-based models (Rasul et al., 2021b) difficult to find a clear probabilistic transition path and inefficient to perform each iteration, which results in more computational consumption but worse performance.
function with clear semanties. Compared to diffusion-based models flow-based models with longer generation pro- cesses, which cause more computational consumption and memory overhead, VA adopts a VAE-base AAE-based structure composed of lightweight but effective KoopmanNet and KalmanNet, which contributes to fast one-step generation and lower memory occupation. The contributions are sum- marized as follows:
To address PTSF, propose efficient framework called VAE. It transforms nonlinear time series into a linear dynamical system. Through predicting and re- fining the process uncertainty of the system, VAE demonstrates strong generative capability and excells in both the short- and long-term probabilistic forecasting.
Since the nonlinearity in time series leads to the dynamic evolution of complex patterns, probabilistic models struggle to effectively capture these changes and accurately model their evolution. To tackle this thorny issue, Koopman The- ory (Koopman, 1931) provides a lincarization approach to transform the nonlinear time series into the space of measurement function, which is theoretically infinite- dimeansional space characterizing all measurements of the dynamical system at each moment, and the transition pro- cess of these measurements can be captured by a linear Koopman Operator (Lan & Mezić, 2013). On the other hand, in order to accurately and efficiently model the process un- certainty and mitigate the error accumulation phenomenon in long-term forecasting, the Kalman Filter (Welch, 1995) provides a solution, which fuses observations from multiple sensors to extract the Kalman gains, to refine the prediction and process uncertainty. This inspires us to transform the probabilistic time series forecasting into modeling the pro- cess uncertainty of linear dynamical system in the space of the measurement function.
To distengle the complex nonlinearity in the time series, we design KoopmanNet fully exploit the underlying linear dynamical characteristics in the space of measure- ment function, simplify the modeling, and thus contribut- ing to high model efficiency.
To mitigate the error accumulation in LPTSF, we devise a KalmanNet to model, and refine the prediction and uncertainty iteratively.
Comprehensive experiments on both short- and long- term PTSF show that V²VAE outperforms state- of-the-art baselines. Additionally, all datasets and code are avaliable at https: /ettps//ggite //t::/giiitt com/ beeisionintelligenee
2. Preliminaries
Koopman Theory. Koopman Theory (Koopman, 1931;
Lan & Mezić, 2013) is a widely used mathematical tool for dynamic system analysis. providing way to linearize the nonlinear systems. For nonlinear system xg+11 f(xx) ). where I& denotes system state and is nonlinear function, it assumes that the system's 's state can be mapped into the space of measurement function tb where it can be modeled by an infinite-dimension linear Koopman Operator K::
In this study, we propose VAE,E, a generative probabilistic forecasting model tailored for LPTSF see Figure 2. First, to handle the nonlinearity and capture the underlying dynamics in time series, we patchify the time series into tokens and model them through the KoopmanNet. The KoopmanNet provides a way to simulate the Koopman Theory, which transforms the nonlinear time series into latent measure- ments, and fit the Koopman Operator to construct "biased" linear dynamical system easy to describe and model. Sec- ond, to achieve accurate long-tern forecasting performance, we design a KalmanNet in a data-driven manner based on the principle of Kalman Filter. Through integrating the resid- ual nonlinear information as control inputs, while treating the biased linear dynamical system as the observation, the KalmanNet predicts and updates to model and refine the uncertainty with Kalman gain. This effectively mitigates the error accumulation of the linear system and helps construct the variational distribution in the space of the measurement
((x++1) = (+((x))) = KC o v((x))
(1)
Koopman Theory helps understand the underlying dynamics of complex nonlinear systems and serves as a powerful tool to linearize them for ease of process.
Kalman Filter. Kalman Filter (Welch, 1995; Simon, 2001) is a recursive algorithm used for estimating the state of a linear dynamic system. It works in two steps: first, it pre- dicts the current state 2ı and uncertainty covariance matrix Ps based on the system's 's state transition equation; then, it updates the estimation by incorporating the difference be- tween the measurement and prediction, known as Kalman gain Kg. The Kalman Filter effectively fuses information
2

---

A Koopman- Kalman Enhanced Variational AutoEncoder for Probabilistic Time Series Forecasting
Nonlinear System
Biased Linear System
Quantitative Uncertainty
Quantitative Uncertainly
Legend
Over the Horizon
Time Sertes
Koopmanneet
KalmanNet
Decoder
Variable
han
M
Predicsod
Target Harican
Contest
Predicted Horizen
Contest
Predicte Herizen
Hortonn
Confidenc Intervo)
Figure 2.
The data flow K VAE. models time series through the KoopmanNet, which constructs a biased linear system. Then the linear system is refined through the KalmanNet while the uncertainty is modeled. Finally, the target distributions over the horzion predicted through the Decoder.
from multiple sensors to enhance estimation accuracy while modeling the uncertainty of the system.
objective, which means the series generated by the Koop- man Operator is biased. We then design the KalmanNet to refine such biased linear system and model the uncertainty by outputting the covariance matrix of multi-dimensional state vector, which assigns the variational posterior distri- bution [Z[ ) in the space of measurement function with clear semantics. The Decoder works as the inverse measure- ment function & to map the samples from Q(2 ( to the original space, which also serves as the decoder of VAE and models the target distribution ZX of the forecasting horizon.
VAE for Probabilistic Time Series Forecasting. PTSF can be treated conditional generative task, i.c., generating forecasting horizon y [#T+1, Fr42,
.t IR N given context series X = [21 '22 ... .rl E RRXI, where N denotes the number of variables, T denotes the context length, and L denotes the forecasting horizon. The objec- tive is to model the conditional distribution P(Y|X) and sample from it to obtain Y. When using Variational Au- toEncoder (Higgins al., 2017; Pu et al., 2016), the log- likehood objective is optimized through the Evidence Lower Bound (2) which is obtained by Jensen Inequality:
3.1.1. I. INPUT TOKEN EMBEDDING
Since Triformer (Cirstea et al., 2022a) first proposes the Patching technique, existing works (Nie et al., 2023; Wu et al., 2025c) demonstrate that considering a patch as the "token" retains most semantic information and helps estab- lish meaningful state transition procedure for autoregres- sive models. Our proposed V2 also works like an autoregressive dynamic system model the state transi- tion procedure. Different from those Channel-Indepente models which divides patches for each channel and projects them independently, we consider multivariate patches as tokens to implicitly model the cross-variable interaction during state transition. We divide the context series X [r1, (2,,"22 ... [T]] E R-N into non-overlapping patches:
CELBO
- -ooo X)] D (x.
In our proposed VAE, we meticulously construct the variational distribution QZ] aligning with the uncer- tainty of the dynamical system. This endows the latent space in VAAE with clear semantics, enhancing its generative capabilities in PTSF.
3. Methodology
3.1. VAE Architecture
As demonstrated in Figure 3, V2VAE consists of four main components: Input Token Embedding, KoopmanNet, KalmanNet, and Decoder. The KoopmanNet and Kalman- Net consitute the Encoder of VAE. To facilitate compre- hension, we present the Data Flow-see Figure 2.
X = p , II
R Nxaxn
where s = T/n denotes the patch size, n denotes the patch number, and xf E -RN denotes patch. Then X are embeded into high- digh-dimensional hidden space:
Overall, K VAE employs a meticulously designed pipeline to model the time series at the perspective of dynamic sys- tem. First, the Input Token Embedding module patchifys the time series into tokens. Then the KoopmanNet projects them into the space of measurement function, where the inherent nonlinearity and intricate joint distribution between vari- ables are reconsidered for ease. Sequentially, the Koopman Operator is fitted and iterates over the first token to delineate a linear system. Obviously, the perfect measurement func- tion which constructs an absolute linear system is the ideal
P' Projection(Flatten X ²)),
(4)
where patches are first flattened into {(Nx)) and then mapped into embeddings X E Rdxx through linear projection to fuse the variable information.
3.1.2. K2VAE ENCODER
Linearizing with the KoopmanNet. Since there ex- ists variable-wise periodic misalignment or temporal non- stationarity in realistic multivariate time series, yielding
3

---

A Koopman- Kalman Enhanced Variational AutoEncoder for Probabilistic Time Series Forecasting
Legend
k
Times
Element-wise Plus
Resample
KalmanNet
Iterative part
Output of KalmanNet
L
Input Token
KoopmanNet
~ N'O. R0)
KalmanNet
Embedding
eDMD
Observation Model
Og
Time Series
Learnable
Kiyta
K.
Decoder
X x
Nio.
Patching
Measurement Function V
Koopman
Operator
IC
Process Model
Updator
X &. R Nxxx
Ex
Embedding
L
Z
Pa
XP E Rddon
Residual
Integrator
Z'
QZ((X)
U
Figure 3. The architecture of VAE. Input Token Embedding Module patchifys the time series into tokens and applies Embedding. Encoder Module consists of KoopmanNet and KalmanNet, transforming the tokens into linear system in the space of measurement function, refining it and modeling the process uncertainty as the variational distribution. After resampling from the variational distribution. Decoder module constructs the likehood distribution about the forecasting horizon.
non-linearity, VAE applies Koopman Theory (Koopman, 1931) to construct the measurement function to project the system states into measurements which can be modeled as a linear system. Practically, we use a learnable MLP-based network to serve as the measurement function yे:
Modeling the Uncertainty with the KalmanNet. Since we adopt a data-driven paradigm to model the measure- ment function &a and Koopman Operator K,, it exists bias between the generated X and X during optimization, known as a biased linear system. Inspired by Kalman Fil- ter (Welch, 1995; Simon, 2001) which is born to refine such biased linear sytem, we devise a KalmanNet to model and refine the uncertainty adaptively, aligning it with the variational distribution xX) in the latent measurement space. Specifically, first fully reuse the nonlinear resid- ual through the Integrator based on an Encoder-Only Vanilla Transformer (Vaswani et al., 2017):
X ((x")
where X E Rdxn denotes the projected tokens in the measurement space. capture the transition rule, we utilize the one-step eDMD (Schmid, 2010; Liu et al., 2023) over XP" to efficiently find the best fitted Kloci
X
back
=
1
2
n
(6)
X = XP" - 8C,
(12)
X Xfore
fore
122
, 2g
p
n
(7)
U = Integrator X Res) = [u1, U2 um]
(13)
Kloc = fore ( book))
(8)
where U E Rd denotes the output integrated by the Inte- grator. We then construct the Process Model of KalmanNet, which describes the state transition process:
where (xct denotes the Moore-Penrose inverse of X Kloc effectively captures the local transition rule in the space of current measurement function. However, when vb is underfitted, the low quality of the space may cause numerical instability or guide the model to converge in a wrong direction. To mitigate this issue and capture the global-shared dynamies, we introduce learnable part Kgio- Then we delineate the system through the Koopman Operator K- Kloc + Kglo:
2k A + Bug + wk:
(14)
p"
(15)
where A E Raxd is the state transition matrix, B E Rd is the control input matrix, and w% ~ N0, Q) is the pro- cess noise and Q is its covariance matrix. Sequentially, we construct the Observation Model:
20 xc [
: 22
_
(9)
Ok H + Uka
(16)
x [ AH
.441
(10)
where H E Rddd is the observation matrix and we treat the prediction X H as the prior observation in Update Step (20). Us (O, R) is the observation noise and R is its co- variance matrix. Our goal is to reuse the information from the nonlinear residual, and integrate it into the linear sys- tem constructed by KoopmanNet, thus obtaining a more accurate linear system and modeling the uncertainty. In the KalmanNet, all the matrices are learnable. Additionally, we
:9 = (K) of 24 (KC)tt-110
(11)
where XC E dx denotes the reconstruction context gen- crated by Koopman Operator K E 1dd and X e Ædxm is the predicted horizon, m = L/s means that predicting L steps in the original space is equivalent to predicting m steps in the space of measurement function.
4

---

A Koopman- Kalman Enhanced Variational AutoEncoder for Probabilistic Time Series Forecasting
initialize the covariance matrices Q and R as identity matri- ces and use lower triangular matrices T and In to keep the positive definiteness: Q = ا and R = L
that the (( -N a) is modeled. We also map back the Koopman reconstruction X from the measure- ment space to optimize the Re (27), which helps measure- ment function & to build a linear system.
Then we conduct the Prediction Step and Update Step itera- tively, the Prediction Step can be formulated as:
1.4.4. OVERALL LEARNING OBJECTIVE
2x = Azk-1 1 + Buk:
(17)
The overall learning objective is weightsumed by CELBO and LRee:
P^ = AP + Q
(18)
CELBO Blog P(Y Z, x)]+
where 2 is the predicted state and Plc is the predicted co- variance matrix of the process uncertainty. Then the Update Step measures the weight between observation and predic- tion through Kalman gain Kş to refine the system:
D ((221
(26)
LR ||X X 2.
(27)
where the LELBO ensures the fundmental mechanism of
K2 = P H + +R) R
(19) VAE. The prior distribution is P(ZIX) N I),
5k 2x + Kx H2x),
(20)
where we hope the linear system in measurement space converge to a stable state. Lnee facilitates the linearization of the measurement space.
Pk P K.H)
(21)
where 2 and Pkk is the refined state vector and covari- ance matrix. We then obtain the refined predictions Z = [21, 22, ... _mm] and covariance matrices of each token [PI, P P2, ... P.... which describes the temporal pro- cess uncertainty in the dynamical system. We show that the process also obeys the basic assumptions of Koopman Theory in Section 3.2. To fully utilize the ability of the Integrator, we make skip connection:
3.2. Theoretical Analysis
3.2.1. THE STABILITY OF KALMANNET
Since the proposed KalmanNet works in a data-driven man- ner, the Hlotting-poite operation error may cause the co- variance matrix P losing positive definiteness, which often occurs in the step (21). To mitigate this, we utilize numer- ically stable form for this step.
z' Z + U
(22)
Theorem 3.1. The vositive-defiiiteess of covariance ma- trix Pa during the update step P& (I- H2 H2 can be retained through a numerically stable form:
During the training process, the model leverages the Integra- tor to integrate nonlinear information and gradually adjust the topological structure of the mcasurement space. Op- timized by Lee (27), the deviation of the linear system constructed by the KoopmanNet gradually decreases, caus- ing U + O. This facilitates a linear dynamical system in the measurement space and gradually reduces dependence on the Integrator.
1 P&
2 _Px + PP)),
(28)
pdual = (AL K.HL) + K&RAKI
(29)
where (28) ensures the symmetry, (29) stabilizes the positive- definiteness by decomposing the formula into the sum of two positive definite terms, which better ensures positive definiteness during floating operation.
3.1.3. VA DECODER
After obtaining the prediction Z', and the covariance ma- trix P of process uncertainty, the variational distribution is formulated as Q(2]X)= = =N(Z',)) P). During training, we conduct reparameterizztion sampling from it to keep the ensure the propagation of the gradient. Finally, we utilize the Decoder to map the samples back to the original space and model the IPY| Z) with an isotropic Gaussian distribu- tion. Specifically, the Decoder consists of two same MLP structures as the inverse of the Koopman Encoder v we formalize them as t/ and v
3.2.2. THE CONVERGENCE OF K2VAE
Since K2VAE models a linear dynamical system in the measurement space where the Koopman Operator serves as the state transition equation, we hope that the convergence state of the KalmanNet does not violate the assumptions of Koopman Theory. In k we meticulously design the KalmanNet by making it gradually converge to the Koop- man Operator in the forecasting horizon.
Theorem 3.2. When U -> 0, the state transition equation of the KalmanNet in K22 VAE gradually converges to the
7 sample Resample Resample(Q((/
(23)
# = 1p 1 (2 sample)) ),0 = -/- (2 (goomlle) Reec_ = (80),
(24) Koopman Operator.
(25) We provide the proof of Theorem 3.1-3.2 in Appendix A.
5

---

A Koopman-Kalman Enhanced Variational AutoEncoder for Probabilistic Time Series Forecasting
Table I. Statistical information of the datasets.
Horizon
Dataset
#var.
range
freq.
timesteps
Description
ETThi/h2-L
7
R+
H
17,420
Electricity transformer temperature per hour
ETTml/m2-I.
7
R+
15min
69,680
Flectricity transformer temperature every 15 min
Electricity-I.
321
R+
H
26,304
Electricity consumption (Kwh)
Long-ferm
Traffic-L
862
(0.1)
H
17,544
Road occupancy rates
Exchange-L
8
R+
Busi. Day
7.588
Daily exchange rates of 8 countries
II.I-I
(0,1)
W
966
Ratio of patients seen with influenza-like illness
Weather-1.
21
R+
10min
52,696
Local climatological data
ETTh1/h2-S
R+
H
17,420
Electricity transformer temperature per hour
ETTm1/m2-S
7
R+
15min
69,680
Electricity transformer temperature every 15 min
Exchange-S
8
R+
Busi. Day
6.071
Daily exchange rates of 8 countries
Short-term
Solar-S
137
B+
H
7,009
Solar power production records
Electricity-S
370
R+
5,833
Electricity consumption
T
963
(0.1)
H
4,001
Road occupancy rates
Table Comparison on short-term probabilistic forecasting scenarios across eight real-world datasets. Lower CRPS or NMAE values indicate better predictions. The means and standard errors are based on 5 independent runs of retraining and evaluation. Red: the best, Blue: the 2nd best.
Model
Metric
Exchango-S
Solar-S
Electricity-S
Traffic-S
ETThI-S
ETTh2-S
ETTmI-S
ETTm2--
CRPS
0.01210.002 0,51610.011
0.06810.003
0.32010.017 0.2082
0.21210.012
0.19910 0,19310.00
FITS
NMAE
0.017=0.008 0.701
0.092+0.004
0.392=0.028
0.278+0.009 0.423+0.033
0.260 0.249
CRPS
0.052+0.016 0.000
0.063=0.005
0.278=00.18
0.31 42.022
0. 0.207=00.006
0.23420.011
0.212 018
PatchTST
NMAF
0.069+0.013 0.66320.010
0.08520.006
0.363=20022
2.007
0. 0.26020.00m 000
0.257 00.00
011
CRPS
0.059 0.504
0.066200004
0.244
0.317
0.2 0.219920.00
0.254200012
0.201 2: 018
iTransforner
NMAE
+0.022 0.695=0.017
0.087=0.006
0.319=0.019
0.408 028
0 0.276=0.017
0,242 0.291
ong
CRPS
0.01210.001 0.545110:016
0.0852:0004
0.253:2.0118
0.32610.013
0.211
0.220 0.288
Koopa
NMAE
0.015+0.002 0.742+0.022
0.112=0.019
0.330=0.019
0.423+0.617
0.266+0.022
0.329+0.0a6
0.278+0 022
TSDiff
CRPS
0.077+0.019
0.568
0.111=0.013
0. 0.189
0.009
0.304
0. 0.204420.00
0.124 0.209 008
NMAE
0.09610.024
0.63510.012
0.11510.018
0,206 Lo.01
0.272.1.015 1..00
0.27610.00
0. 0.162.10 one
D*VAE
CRPS
0.001
0. 7..699002
1.070000000
1.1443=0.0m 008
0.324+0.019
0.216=00.015
0. 0.198+0.015
0.303 024
NMAE
0.01210.002
0.09220.013 1.909
0. 0..78
0.41010.016
0 0.26710.018
0.378 0.250 001
CRPS
0.01940.006 0.530
0.062=0.003
U. 0,168
0.39810.034
0 a..00
0.276.0 0.455
GRU NVP
NMAE
0.024+0.007
0.670=0.011
0,081#0.005
0, 00.00
0.477
0.375+40.024
0.584
0.349 0.08
CRPS
0.012+0.00
0.0
0.056=0.002
0,144=0.022
+.258
O. 0..0
0.151
0.146.0.00 011
GRU MAF
NMAE
0.01610.002 002
0. 2.00000000
0.07320.004
0. 0.182 28220.029
0.326200016
0.208.0.003
0.108
0.193 o0e
CRPS
0.012#0.001
0.442+0.011
0.054=0.002
0. 1.133=0..0m 004
0.309+0.009
0.200+0.012
0. +33900.00
0. 1800000000
Trans MAF
NMAE
0.01610.001
0.57710.014
0.0011..000
0. 1,160 10.008
0.4001..011
0. 0.20610..00
0.162 0,16210.000 Lo.006
0. 0.224 000
CRPS
1.00910.001 0,465110.016
0.05710.002
0,130 000
0.27310.007
0. 0.100
0,186
0.14810..0
TimeGrad
NMAE
0.012+0.0r 0.609+0.016
0.073=0.004
0.155=0.00m 007
0.356+0.013
0.224+0.014
0.246=00.07
0. 0.189 006
CRPS
0.00910.001
0.392.1.000
0.051
0,147 10.014
0.26210.012
U. 10.3000000
0.144 0.14010.012
CSDI
NMAF
0.013+0.001
0.533+0.007
0.06620.001
0. 1712200033
0.339=0.000
0. 1100011
0. 20.00221
0.181 20 024
KVVAE
CRPS
0.009 2M0920.000 001
0.367:0.005
0.053=0.002
0.129=00.0m 0os
0.256
0. 0.128=20.00
0. 135
0.122220.008
NMAE
0.00920.000 001000
0.480
0.068
0. 0.157=0.000 000
0.31220.008
0.1 0.140:20.07
0.152:0.007
0.146:20.000
4. Experiments
sive benchmark used to evaluate probabilistic forecasting tasks. Specifically, we use the datasets ETTh1-S, ETTh2-S, ETTmI-S, ETTm2-S, Electricity-S, Solar-S, Traffic-S, and Exchange-S for short-term forecasting, of which the con- text length T equivalent to forecasting horizon L with T L = 30 for Exchange-S and T L = 24 for the oth- ers. For long-term forecasting, we use the datasets ETTh1- L, ETTh2-L, ETTmI-LL ETTm2-L, Electricity-L,, Traffic-L, Exchange-L., Weather-L, and IIII-I with forecasting horizon L E (24, 36, 48, 60) for ILI-L and L E {96, 192, 336, 720} for the others. Note that we fix the context length of all the models with T = 36 for II.I-I. and T = 96 for the others to
In this section, we provide empirical results to show the strong performance of VAB against of- baselines on both short- and long-term probabilistic forecasting tasks.
We also analyze the model efficiency and the key parameters of K VAB as the proof of architectural superiority.
4.1. Experimental Setup
We conduct experiments on 8 datasets of short- term forecasting and 9 datasets of long-term forecasting
Datasets.
based on ProbTS (Zhang et al., 2024a), a comprehen-
6

---

A Koopman-Kalmn Kalman Enhanced Variational AutoEncoder for Probabilistic Time Series Forecasting
Table Comparison on long-term prohabilistic forecasting (forecasting horizon 1=720) scenarios across nine real-world datasets. Lower CRPS or NMAE values indicate better predictions. The means and standard errors are based on 5 independent runs of retraining and evaluation. Red: the best, Blue: the 2nd best. The full results of all four horizons 96, 192, 336, 720 are listed in Table 9, 10 in Appendix C.5.
Model
Metric
ETTmI-L
ETTm2-L
ETThI-L
ETT12-L
Electricity-L
Traffic-L
Weather-L
Exchange-L
ILI-L
CRPS
0.305 024
0.449
1.348
0.11519.024 0.314
0.374
0.074 0.267-0.003
0.211 20.011
FITS
NMAE
0. 0.70
0 0.540
0.401 0 0.4
0.149
0.453 oge
0.317
0.097
0.245
PatcTTST
CRPS
0.30420.022
0 00.00
0. 0.323
0.004
0.127
0.214:00001
0.142.0.00s
0.097 3:0
0.233
NMAR
0.382
0 0,288 10
0.428 govea
2000m 20
0.16410.021
0.15220.00 00258
0.126 oo1
0.287
CRPS
0.45520.021
0.011
0 0.350
0.542s0.015
0.100
0.284::0008
0.133=0.004
0.087
0 0.222:0.ca0
Transformer
NMAB
0 0.490
0 0.385 042
s.607
0.1 0.100000.000
0.361
0.113 0.147=0.01
0.278
Koopa
CRPS
0 2.255:00002
0 0.233:00.05
:0.00
0. 1.293 20.00
0.11310.018
0.338
0.140
0.091
0.228
NMAR
0.377 uS7
0 0.2
0 0.16
0. 0.2
0.432 o02
0. 0.16220.00m
0.116 o22
0. 0.288 c010
CRPS
0.4 0.478
0.51620.027 0.344:0.040
0. 0.66000.050
0.478
0.391=0.00 002
0,152=0.008
0.082
0.263=6.c22
TSDiff
NMAE
0 0.41630.055 0.62220.045
2000s 2.005
0.657
0.48220.022
0.622
0.478 006
0.141 26.02
0.272 0.142
CRPS
0.5
0 0.561+0.27a
0 0.50220.009
0 0.339=0.090
0.114
0,211 004
0.110=0.004
0.079
0.307=00.0m
GRU NVP
NMAE
0.707
0.740
0.643
1.688
0.144g0.017
0.264 006
0.135 00s
0.103 0ce
0.333 00s
CRPS
0.272:0.029 0.53620.0s3
0 0..33
0.900
0.106
0. 0.122:0.006
0.100
0.172::0.084
GRU MAF
NMAE
0.71120.081
0.355
1.092:0.019
0.1 0.13610.006
0.149=6.c3c
0.182:0.010
0.216:0.014
CRPS
0.658
0 0.335:00.03
0. 20.06a
=0.07s
0.113=0..04
0.14820.017
0 131
Trans MAF
NMAE
0. 0.822
0 0.475:0.c29
0. 45.502
0.412:0.020
0, 0..48:0.040
0.191 0.0.
0. 18.0010
TimeGrad
CRPS
1621=0.007
0. 0.170=0.00s 054
0. 0.523=0.c27
0. 0.415=0.006
0.108
0.220 002
0. 0.113:0.011
0.090 0000
0,295=a.066 063
NMAE
0.793=0.054
0 0.561+0.044
0.672#0.016
0,350=
0.1344.004
0.263=0.001
0.136=0.020
0.113#0.016
0.325
CRPS
0. 0.23920.035 0.448=0.058
0.328:0.012
0.302=0.040
0.087=c.00s
0.1433=0.02
0,283=s.012
CSDI
NMAE
0.578
0 0.30610.040
0.382s0.000
0.102-0.005
0.17320.00
0,299
KPVAE
CRPS
0..00
0.221 02a
0.314=0.011
0.280
0.007
0.200=0.00
0.084=0.003
0.069
0.142
NMAE
0.373
0.396 0.275
0.278
0.117
0.248=0.010
0.099
0.084:0.007
0.167=0.007
Duc to the excessive time and memory consumption, some results are unavailable and denoted as-
ensure a fair comparison. Details are shown in Table 1.
best baseline, CSDL. In long-term scenarios, it surpasses PatchTST with improvements of 20. 9% and 19.9%.
Baselines. We compare VAE with 11 strong baselines, including 4 point forecasting models: FITS (Xu et al., 2024), PatchTST (Nie al., 2023), iTransformer (Liu et al., 2024), and Koopa (Liu et al., 2023), well as 7 generative mod- els: TSDiff (Kollovieh et al., 2023), V&VAE (Li et al., 2022), GRU NVP, GRU MAF, Trans MAF (Rasul et al., 2021b), TimeGrad (Rasul et al., 2021a), and CSDI (Tashiro et al., 1), in both short-term and long-term probabilis- tic forecasting scenarios. The point forecasting models are equipped with gaussian heads to predict the distributions. Detailed descriptions of these models can be found in Ap- pendix C.2.
2) VAE shows significant advantage on nonstationary time seris datasets such as Exchange-S and Exchange-L. There exists distribution drift phenomenon in these datasets, which causes non-linearity and hinders the prediction and uncertainty modeling. Though diffusion- -based and flow- based models are theoretically capable of fitting any distribu- tions, they struggle to construct explict probability transition paths to reach such complex destinations. While V2VAE simplifys this difficulty through modeling the time series in linear dynamical system, where the uncertainty is more explicit and casier to be modeled.
Evaluation Metrics. We use two commonly-used metrics CPRS (Continuous Ranked Probability Score) and NMAE (Normalized Mean Absolute Error) to evaluate the proba- bilistic forecasts. Detailed descriptions of these metrics can be found in Appendix C.3.
3) K VAE also shows stable and strong performance with respect to the varying forecasting horizons-see Table 9 and Table 10 in Appendix C.5. The performance of most base- lines drops significantly as the forecasting horizon extends, while VAE maintains superiority. One potential reason that VAE utilizes the KoopmanNet to effectively handle the inherent nonlinearity in long-term forecasting. Another reason is that the KalmanNet mitigates the error accumula- tion by integrating diverse information.
4.2. Main Results
Comprehensive probabilistic forecasting results are listed in Table 2 and Table 3 with the best in red and the second in blue. We have the following observations:
4.3. Ablation Studies
1) VAV outperforms tfate-of-the-art baselines, show- ing notable improvements in predictive performance. In short-term scenarios, it achieves a 7.3% reduction in CRPS and 14.5% reduction in NMAE compared to the second-
4.3.1. 1. VARIANTS OF KOOPMAN OPERATOR
We design the local Koopman Operator Kioc obtained by one-step eDMD and a learnable part Kyto- As one-step
7

---

A
Enhanced Variational AutoEncoder for Probabilistic Time Series Forccasting
eDMD estimates the Kloe through matrix calculation, which relys on the local quality of the space of measurement func- tion. Once the initilization leads to an ill-conditioned topo- logical structure, the one-step eDMD suffers from the nu- merical calculation error or guides the model to converge in a wrong direction, which often occurs in long-term proba- bilistic forecasting scenarios and hinders the performance. To enhance the robustness, we adopt global learnable part Kig to mitigate this phenomenon while capturing the global-shared dynamics. As shown in Table 4, mixed Koop- man Operator K = Kine + Kgto demonstrates better per- formance on both short- and long-term tasks while pro- viding roubustness to avoid calculation error if Kloe fails. Complete experimental results are provided in Table 12 in Appendix C.6.
Table 5. Comparison on different connections in KalmanNet. Lower values indicate better performance. Red: the best. L: the forecasting horizon. The full results can be found Table 13 in Appendix C.6.
Connectios in KKllanNer
Metrics
Dlectrictty
ETTml L (L 720)
Exchange (L 30)
7201
CL - 24)
wlo Iniegrance
CRPS
0.350 0.082ami01
0.015ar.a0
0.308
NMAR
88
0.4424002
0.022
0.631
wio skip connection
NMAE
CRPS
0.131a2a15
0.0834300
0.315ana
0.4002
0.01
0.01344.004
0.888
0.511
wro contrll ingut
CRPS
0.322
0.01Bac.a05
0. 0:423
NMAE
0.14245aus
0.418400012
0.017a6.903
0.5000...00
CRPS
0.294
0.009
0.367
Mixed
NMAE
0,117
0.373:22.12
0.009...0011
0,450 -3.03
input" gives too little support for KalmanNet to adaptively refine the prediction and process uncertainty. Complete ex- perimental results are provided in Table 13 in Appendix C.5.
Table 4. Comparison different Koopman Operators. Lower val- ues indicate better performance. Red: the best. L: the forecasting horizon. The full results can be found in Table 12 in Appendix C.6.
4.3.3. A ABLATIONS OF KOOPMANNET & KALMANNET
As the most important modules, the KoopmanNet and KalmanNet jointly contribute to state-of-the-art perfor- mance of VAE. To evaluate their impactment, we con- duct ablation studies and the results are shown in Table 6.
Koopman
Electricity-L aL= 720)
ETTaI-L
Exchange-S (L=
Solan-S
Metrics
Operator
(L= 720)
(L. 24)
CRPS
0.012=00or
Niw
NMAD
0.014gnome
0.56815.05
Kytt
CRPS
0.085
0.311
=.011
4.374
It is observed that both the KoopmanNet and KalmanNet show indispensability in probabilistic forecasting. Since the KoopmanNet ensures the lincarization of the modeling, it shows greater impact in forecasting performance. An- other reason is that KalmanNet does not excell at non-linear modeling because it is based on the linear kalman filter.
NMAE
0.395 0,1301oms
0.013
1.48815.00
Kioe
Nigto
CRPS
a.057 0.11722.019
0.009.0.0m 0.294.0.00
1.367
NMAR
0.373
0.009
1.000
Due the numerical instability. some results are unavaslable and denoted as
4.3.2. CONNECTIONS IN KALMANNET
We adopt an Integrator to assist the KalmanNet for faster convergence, which potentially helps tune the topological structure of the space of measurement function into the lin- ear dynamical system. Specifically, the Integrator integrates the non-line linear residual into the control input of KalmanNet, which is proved not to affect the prior of Koopman Theory in Section 3.2. We also make skip connection between Integrator and the KalmanNet for the final prediction, this constraints Integrator predicting residuals from residuals. Since the space of the measurement space is optimized to converge a linear system, the Integrator serves an as- sistant and gradually stop helping the model. We showcase the different variants in Table 5, to which only some of the features mentioned above are applied.
Table 6. Ablations on KoopmanNet and KalmanNet. Lower values indicate better performance. Red: the best. L: the forecasting
horizon. The full results can he found in Table 14 in Appendix C.6.
Elecnicity-1.
FTTml-1.
Eschinge-S
Sola-SS
Variants
Metriss
(L 720)
(L= 720>
(L=
(L=20)
CRPS
0.074 0:483
0.385anne
wil KoogasanNet
NMAE
0,162samus
0.001:5.05
0.016
0.528
KolmanNet
CRPS
0.089 0.011 0.308
0.575aauuc
NMAE
0.102:0.00
0.500
0.01244.04
0.400
V'V
CRPS
0005
0.294
0.009
-0.00
0.367-00.55
NMAE
0.117
0.373
0.009-4.00
0.450
4.4. Model Efficiency
We evaluate the model efficiency from three aspects: prob- abilistic forecasting performance (CRPS), inference time (sec/sample), and max gpu memory (GB). Figure 4 show- cases a common scenario on Electricity-L (96-96), which re- flects the overall relative relationships on above-mentionel three aspects. &²VAEE achieves best forecasting perfor- mance while occupying the minimum gpu memory and hay- ing the fastest inference speed. One reason is that VAE applies KoopmanNet and KalmanNet, composed of sev- eral lightweight MLPs or linear layers, to efficiently build the variational distribution as process uncertainty, thus en- hancing generation capability of 22VAE, Another reason
We observe that our adopted Mixed variant outperforms others, because it provides gains for the KalmanNet, which integrates non-linear information for adaption, and provides constraints for the Integrator, which makes full use of it without destroying the assumptions of Koopman Theory. The variant "w/o skip connection" completely depends on the linear fitting ability of the KalmanNet, which is hard to disentangle the nonlinear components in the early stages of training, thus potentially hindering the modeling of process uncertainty. On the other hand, the variant "w/o control
8

---

A Koopman- Koopman-Kalman Enhanced Variational AutoEncoder for Probabilistic Time Series Forecasting
memory occupation, and better performance.
TSDiff 0.355GB
5.2. VAE for Time Series
CRPS
Variational Autoencoders (VAEs) (Kingma & Welling, 2014) have found wide applicability across various time series tasks. In time series generation, VAEs synthesize time series by encoding the data into lower-dimensional latent space and then decoding it to recreate similar se- quences, which helps preserve the statistical properties of the original data, making VAEs valuable for data augmen- tation (Li et al., 2023a; Desai et al., 2021). In time series imputation, VAEs recover missing values by learning the underlying latent structure of the data (Boquet al., 2019; Li et al., 2021). By capturing temporal dependencies and relationships, they help restore incomplete time series with high accuracy. In time series anomaly detection, VAEs are used to learn the expected patterns within time series data and flag deviations that indicate anomalous behavior (Huang et al., 2022a; Wang et al., 2024f). In time series forecast- ing, Time VAE (Desai et al., 2021) and D&VAE (Li et al., 2022) tailored for short-term probabilistic foercasting tasks. Koopa (L.iu et al., 2023), as a strong baseline based on Koopman Theory, is tailored for long-term deterministic forecasting by adopting multi-scale MLP structures, which also falls short in probabilistie forecasting. Compared to these methods, VAE is tailored for LPTSF, which con- siders the inherent nonlinearity of time series through KoopmanNet and tackles the eπтоr accumulation through KalmanNet, thus enhancing the ability to predict long-term future distributions. This facilitates better decision-makingg in dynamic and uncertain environments.
CSDI LJIIG
0.13
GRU NVP 0.145B
MAF 0.14GGB
TimeGrad (LEESG
GRU MAP 0.128GB
AEVAE 0
0.04
61 Inference Time (see/sample)
Figure 4. Model efficiency comparison. All the statistical data is obtained on the Electricity-L (T' = L = 96). Sample-wist wise inference time and max gpu memory is obtained with batch size equals 1. Lower values of CRPS indicate better performance.
is that VAE utilizes the VAE architecture and obeys the mee-tee--gnnratioon paradigm, while diffusion- based or flow-based models have longer probabilistic transition paths, which produces more intermediate results and con- sumes longer duration. More evidence of model efficiency provided in Table 11 in Appendix C.5.
5. Related Works
5.1. Probabilistic Time Series Forecasting
Probabilistic forecasting aims to provide the predictive dis- tribution of the target variable. With the rapid development of deep learning, new methods are continually emerging. DeepAR (Salinas et al., 2020) uses recurrent neural net- works (RNNs) to model the transitions of hidden states and generates Gaussian distribution for predictions. Following the autoregressive paradigm, DeepState (Rangapuram et al., 2018) and DSSMF (Li et al., 2019) combine state space models with deep learning to improve forecasting accuracy. MANF (Feng et al., 2024) and ProTran (Tang & Matteson, 2021) introduced attention-based methods that enhance the model's ability to capture long-range dependencies, fur- ther improving forecasting accuracy. Diffusion models, such those proposed by TimeGrad (Rasul et al., 202 la), TSDiff (Kollovieh et al., 2023), and CSDI (Tashiro et al., 2021), approach the forecasting task as denoising pro- cess, excelling in handling high-dimensional data. Another approach involves using more complex distribution forms, such as normalizing flows (Rasul et al., 2021b), to further enhance forecasting performance. Compared with RNN- based or State Space models, VAE also autoregressively models the time series in a linear dynamical system, but mitigates the error accumulation through KalmanNet. Com- pared with generative diffusion- iiffusion-based or fow-based models, K2 adopts the VAE structure and follows single-step- generation principle, achieves faster inference speed, lower
6. Conclusion
In this work, we propose a VAB-based probabilistic forecast- ing model called &² to solve PTSF. By leveraging the KoopmanNet, VAE transforms nonlinear time series into linear dynamical system, which allows for more effec- tive representation of state transitions and the inherent pro- cess uncertainties. Furthermore, the KalmanNet provides solution to model the uncertainty in the linear dynami- cal system, mitigating the error accumuation in long-term forecasting tasks. Through comprehensive experiments, we demonstrate that KR not only outperforms exist- ing state-of-the-att methods in both short- and long-term probabilistic forecasting tasks, but also achieves fascinating model efficiency.
In the future, we hope to continuously study the one-step generation paradigm in time series probabilistie modeling to further improve the model performance and efficiency. An- other primary direction is to pionner the exploration of foun- dation probabilistic time series forecasting models, which can work effectively in zero-shot scenarios.

---

A Koopman- Kalman Enhanced Variational AutoEncoder for Probabilistic Time Series Forecasting
Impact Statement
Cirstea, R.-G., Yang, B., Guo, C., Kieu, T., and Pan, S. Towards spatio-temporal aware traffic time series fore- casting, In ICDE, pp. 2900-2913, 2022b.
This paper presents work whose goal to advance the field of Machine Learning. There are many potential societal consequences of our work, none which feel must be specifically highlighted here.
Cui, K., Liu, S., Feng, W., Deng, X., Gao, I.,, Cheng, M., Lu, H., and Yang. tention network for fashion compatibility modeling in uge systems. ACM Transactions Multimedia Computing. Communications and Applications, 2024a.
Correlation aware cross- cross-modal at-
Acknowledgements
This work was partially supported by National Natural Sci- ence Foundation of China (62472174, 62372179). Bin Yang the corresponding author of the work.
Cui, K., Tang, W., Zhu, R., Wang, M., Larsen, D., Pauca, V. P., Alqahtani, S., Yang, F,, Segurado, D., Fine, P., et al. Real-time localization and bimodal point pattern analysis of palms using uav imagery. arXiv preprint 2024b.
References
Boquet, G., Vicario, J. L., Morell, A., and Serrano, J. Miss- ing data in traffic estimation: variational autoencoder imputation method. In ICASSP, pp. 2882-28888 2019.
Cui, K., Zhu, R., Wang, M., Tang, W., Larsen, G. D., Pauca, V. P., Alqahtani, S., Yang. F., Segurado, D., Lutz, D., et al. Detection and geographic localization of natural objects in the wild: A case study on palms. arXiv preprint a5Xiv:2502. 13023, 2025.
Box, G. E. and Pierce, D. A. Distribution of residual auto- correlations in innpegresssiv-intgrratee moving average time series models. Journal of the American statistical Association, 45(332):1509-1520 1970.
Dai, T. Wu, B., Liu, P., Li, N., Bao, J., Jiang, Y., and Xia, S.-T. Periodicity decoupling framework for long-term series forecasting. In ICLR, 2024.
Breiman, L. Random forests. Machine learning, 45:5-32, 2001.
Desai, A., Freeman, C., Wang, Z., and Beaver, I. Timevae: A variational auto-encoder for multivariate time series generation. arXiv preprint arXiv:21 SXXv:2111.00099, 2021.
Campos, D., Kieu, T., Guo, C., Huang, F., Zheng, K., Yang. B., and Jensen, C. S. Unsupervised time series outlier detection with diversity- driven convolutional ensembles. Proc. VLDB Endow., 15(3):61 1223, 2022.
Fang, Z., Pan, L., Chen, L., Du, Y., and Gao, Y. MDTP: A multi-source deep traffic prediction framework over spatio-temporal trajectory data. Proc. VLDB Endow, 14 12)9 1297, 2021.
Campos, D., Zhang, M., Yang, B., Kieu, T., Guo, C., and Jensen, C. S. Lightts: Lightweight time series classifi- cation with adaptive ensemble distillation. Proc. ACM Manag. Data, 1()::77:1-171:27 2023.
Feng, S., Miao, C., Xu, K., Wu, J., Wu, P., Zhang, Y., and Zhao, P. Multi-scale attention flow for probabilistic time series forecasting. IEEE Trans. Knowl. Data Eng., 36(5): 2056-2068, 2024.
Chen, T. and Guestrin, C. Xgboost: A scalable tree boosting system. In SIGKDD, pp. 785-794, 2016.
Chen, Y., Huang, X., Zhang, Q., Li, W., Zhu, M., Yan, Q., Li, S., Chen, H.. Hu, H.. Yang, J., et al. Gim: A million-scale benchmark for generative image manipulation detection and localization. arXiv preprint arXiv:2406. 16531, 2024.
Gao, H., Shen, W., Qiu, Xu, R., Yang, B., and Hu, J. Ssd- Exploring the potential of linear state space models for diffusion models in time series imputation. In SIGKDD, 2025.
Chen, Y., Huang, S., Cheng, Y., Chen, P., Rao, Z., Shu, Y., Yang, B., Pan, L., and Guo, C. AimTS: Augmented series and image contrastive learning for time series classifica- tion. In ICDE, 2025.
Godahewa, R., Bergmeir, C., Webb, G. L., Hyndman, R. J., and Montero-Manso, Monash time series forecasting archive. arXiv preprint arXiv:2105.000 06643, 2021.
Chen, Z., Ding, L., Chu, Z., Qi, Y., Huang, J., and Wang, H. Monotonic neural ordinary differential equation: Time- series forecasting for cumulative data. In CIKM, pp. 4523- 4529, 2023.
Guo, C., Yang, B., Andersen, O., Jensen, C. S., and Torp, K. Ecomark 2.0: empowering eco-routing with vehicular en- vironmental models and actual vehicle fuel consumption data. Geolnformatica, 19:567-599. 2015.
Cirstea, R., Guo, C., Yang, B., Kieu, T., Dong, X., and Pan, S. Triformer: Triangular, variable-speciffc attentions for long sequence multivariate time series forecasting. In IJCAI, pp. 1994-2001, 2022a.
Higgins, I., Matthey, L., Pal, A., Burgess, C. P., Glorot, X., Botvinick, M. M., Mohamed, S., and Lerchner, A. beta- vae: Learning basic visual concepts with a constrained variational framework. ICLR (Poster), 3, 2017.
10

---

A Koopman- Kalman Enhanced Variational AutoEncoder for Probabilistic Time Series Forecasting
Hu, S., Zhao, K., Qiu, X., Shu, Y., Hu, J., Yang, B., and Guo, C. Multire: Joint learning for time series anomaly prediction and detection with multi-scale reconstructive contrast. arXiv preprint arXiv:2410. 15997 2024.
Lan, Y. and Mezić, Linearization in the large of nonlinear systems and koopman operator spectrum. Physica D: Nonlinear Phenomena, 242(():42-53, 2013.
Li, H., Yu, S., and Principe, J. Causal recurrent variational autoencoder for medical time series generation. In AAAI, volume 37, pp. 8562 8570, 2023a.
Hu, Y., Liu, P. Zhu, P., Cheng, D., and Dai, T. Adap- tive multi-scale decomposition framework for time series forecasting. In AAAI, 2025a.
Li, J., Ren, W., and Han, M. Variational auto-encoders based on the shift correction for imputation of specific missing in multivariate time series. Measurement, 186: 110055, 2021.
Hu, Y., Zhang, G., Liu, P., Lan, D., Li, N., Cheng, D., Dai, T., Xia, S.-T., and Pan, S. Timefitter Patch-specific spatial-temporal graph filtration for time series forecast- ing. ICML, 2025b.
Huang, Q.. Shen, L., Zhang, R., Ding, S., Wang, B., Zhou, Z., and Wang. Y. Crossgnn: Confronting noisy multi- variate time series via cross interaction refinement, In NeurIPS, pp. 46885-46902, 2023.
Li, L., Yan, J., Yang, X., and Jin, Y. Learning interpretable deep state space model for probabilistic time series fore- casting, In IJCAI, pp. 2901-2908, 2019.
Li, Y. Lu, X., Wang, Y., and Dou, D. Generative time series forecasting with diffusion, denoise, and disentanglement. Advances Neural Information Processing Systems, 35: 23009-23022, 2022.
Huang, T. Chen, P., and Li, R. A semi-supervised vae based active anomaly detection framework in multivariate time series for online systems. In WWW, pp. 1797-1806, 2022a.
Li, Y., Wang, H. Li, Z., Wang. S., Dev, S., and Zuo, G. Daanet: Dual attention aggregating network for salient object detection. In 2023 IEEE International Conference on Robotics and Biomimetics (ROBIO), pp. 1-7, 2023b.
Huang, X., Yang, Y., Wang, Y., Wang, C.,, Zhang, Z., Xu, J., Chen, I.,, and Vazirgiannis, M. Dgraph: large-scale financial dataset for graph anomaly detection. In NearIPS, pp. 22765 22777, 2022b.
Li, Y., Wang, H., Xu, J., Ma, Z.,, Wu, P., Wang, S., and Dev, S. CP2M: Clustered- -eteredd-Pach--Mixee Mosaic Augmen- tation for Aerial Image Segmentation. arXiv preprint 2.Xi::5501113389 2025a.
Hyndman, R., Koehler, A. B., Ord, J. K., and Snyder, R. D. Forecasting with exponential smoothing: the state space approach. 2008.
Jing, P.. Cui, K., Guan, W., Nie, L., and Su, Y. Category- aware multimodal attention network for fashion compati- bility modeling. JEEE Transactions on Multimedia, 25: 9120-9131, 2023.
Li, Y., Wang, H., Xu, J., Wu, P., Xiao, Y., Wang, S., and Dev, S. DDUNet: Dual Dynamic U-Net for Highly-Efficient Cloud Segmentation. arXiv preprint arXiv:2501.15338 2025b.
Jing, P., Cui, K., Zhang, J., Li, Y., and Su, Y. Multimodal high-order relationship inference network for fashion compatibility modeling in internet of multimedia things. IEEE Internet of Things Journal, 1 1111:353-3355 2024.
Li, Z., Qiu, X., Chen, P., Wang, Y., Cheng, H., Shu, Y., Hu, J., Guo, C., Zhou, A., Jensen, S., and Yang, B. TSMF-Bench: Comprehensive and unified benchmark- ing of foundation models for time series forecasting. In SIGKDD, 2025c.
Ke, G., Meng, Q. Finley, T., Wang, T., Chen, W., Ma, W., Ye, Q., and Liu, T.-Y. Lightgbm: A highly efficient gradient boosting decision tree. Advances in Neural In- formation Processing Systems, 30, 2017.
Lin, S., Lin, W., Wu, W., Zhao F., Mo, R., and Zhang, H. Segrnn: Segment recurrent neural network for long-term time series forecasting. arXiv preprinl urXiv:2308.1 11200, 2023.
Kingma, D. P. and Welling, M. Auto-encoding variational bayes. In ICLR, 2014.
Kollovieh, M., Ansari, A. F., Johlke-Schneider M., Zschiegner, J., Wang, H., and Wang, Y. Predict, refine, synthesize: Self-guiding diffusion models for probabilis- tic time series forecasting. In NeurlPS, 2023.
Lin, S., Lin, W., Wu, K., Wang, S., Xu, M., and Wang, J. Z. Cocv: A compression algorithm for time-series data with continuous constant values in iot-based monitoring systems. Internet of Things, 25:101049, 2024a.
Koopman, B. O. Hamiltonian systems and transformation in hilbert space. Proceedings of the National Academy of Sciences, 1555):315-318, 1931.
Lin, S., Lin, W., Wu, W., Chen, H., and Yang. J. Sparse Modeling long-term time series forecasting with 1k pa- rameters. ICML. pp. 30211 1021223 2024b.

---

A Koopman- Kalman Enhanced Variational AutoEncoder for Probabilistic Time Series Forecasting
Lin, S., Lin, W., Xinyi, H., Wu, W., Mo, R., and Zhong, H. Cyclenet: Enhancing time series forecasting through modeling periodic patterns. In NeurIPS, 2024c.
Pan, Z., Sharma, A., Hu, J. Y.-C., Liu, Z., Li, A., Liu, H., Huang, M., and Geng, T. Ising-trafiin Using ising machine learning to predict traffic congestion under un- certainty, In AAAI, volume 37, pp. 9354-9363, 2023a.
Lin, S., Lin, W. Zhao, F., and Chen, H. Benchmarking and revisiting time series forecasting methods in cloud workload prediction. Cluster Computing, 28(1):71, 2025.
Pan, Z.., Wang, Y., Zhang, Y., Yang, S. B., Cheng, Y., Chen, P., Guo, C., Wen, Q. Tian, X., Dou, Y., et al. Magicscaler: Uncertainty-awan predictive autoscaling, In Proc. VLDB Endow,, 2023b.
Liu, C., Xu, Q. Miao, H., Yang, S., Zhang, L., Long, C., Li, Z., and Zhao, Timecma: Towards llm-empowered multivariate time series forecasting via cross-modality alignment. In AAAI, volume 39, pp. 18780-18788, 2025a.
Pan, Z., Wang, Y., Zhang, Y., Yang, S. B.. Cheng, Y., Chen, P., Guo, C., Wen, Q. Tian, X., Dou, Y., et al. Magicscaler: Uncertainty-awars predictive autoscaling. Proc. VLDB Endow., 16(12))3808-336 2023c..
Liu, P., Guo, H., Dai, T., Li, N., Bao, J., Ren, X., Jiang, Y., and Xia, S.-T. Calf: Aligning lims for time series forecasting via cross-modal fine-tuning, AAAI, 39(1)) 18915-18923, 2025b.
Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., Antiga, L., Desmaison, A., Köpf, A., Yang, E. Z., DeVito, Z., Raison, M., Tejani, A., Chilamkurthy, S., Steiner, B., Fang. I.., Bai, J., and Chintala, Pytorch: An imperative style. high-performance deep learning library. In NeurlPS, pp. 8024-8035, 2019.
Liu, P., Wu, B., Hu, Y., Li, N., Dai, T., Bao, J., and Xia, S.-T. Timebridge: Non-stationarity matters for long-term time series forecasting. In ICMI., 2025c.
Liu, Q. and Paparrizos, J. The elephant in the room: Towards reliable time-series anomaly detection benchmark. In NeurlPS, 2024.
Pu, Y., Gan, Z., Henao, R., Yuan, X., Li, C., Stevens, A., and Carin, L. Variational autoencoder for deep learn- ing of images, labels and captions. Advances in neural information processing systems, 29, 2016.
Liu, Y., Li, C., Wang, J., and Long, M. Koopa: Learning non- stationary time series dynamics with koopman predictors. Advances in neural information processing systems, 36: 12271-12290, 2023.
Qiu, X., Hu, J., Zhou, L.,, Wu, X., Du J., Zhang, B., Guo, C., Zhou, A., Jensen, C. S., Sheng, Z., and Yang, B. TFB: towards comprchensive and fair benchmarking of time series forecasting methods. Proc. VLDB Endow, 17(9): 2024.
Liu, Y., Hu, T., Zhang, H., Wu, H., Wang, S.. Ma, L., and Long, M. itransfomer: Inverted transformers are effective for time series forecasting. In ICLR, 2024.
Qiu, X., Cheng, H., Wu, X., Hu, J., and Guo, C. A com- prehensive survey of deep learning for multivariate time series forecasting: A channel strategy perspective. arXiv preprint arXiv:2502. 10721, 2025a.
Matheson, J. E. and Winkler, R. L. Scoring rules for contin- uous probability distributions. Management science, 22 110):1087-1066, 1976.
Qiu, X., Li, X., Pang, R., Pan, Z., Wu, X., Yang, L., Hu, J., Shu, Y., Lu, X., Yang, C., Guo, C., Zhou, A., Jensen, C. S., and Yang, B. Easytime: Time series forecasting made easy. In ICDE, 2025b.
Miao, H., Liu, Z., Zhao, Y., Guo, C., Yang, B., Zheng, K., and Jensen, C. S. Less is more: Efficient time ries dataset condensation via two-fold modal matching. PVLDB, 6(2):226-238) 2024a.
Qiu, X., Li, Z., Qiu, W., Hu, S., Zhou, L., Wu, X., Li, Z. Guo, C., Zhou, A., Sheng, Z., Hu, J., Jensen, C. S., and Yang, B. TAB: Unified benchmarking of time series anomaly detection methods. In Proc. VLDB Endow., 2025c.
Miao, H., Zhao, Y., Guo, C., Yang. B., Zheng, K., Huang, F, Xie, J., and Jensen, C. S. A unified replay-based contin- uous learning framework for spatio-temporal prediction on streaming data. In ICDE, pp. 1050-1062, 2024b.
Miao, H., Xu, R., Zhao, Y., Wang, S., Wang, J., Yu, P. S., and Jensen, S. A parameter-efficien federated frame- work for streaming time series anomaly detection via lightweight adaptation. TMC, 2025.
Qiu, X., Wu, X., Lin, Y., Guo, C., Hu, J., and Yang, B. DUET: Dual clustering enhanced multivariate time series forecasting. In SIGKDD, pp. 1185-1196, 2025d.
Rangapuram, S. S., Seeger, M. W., Gasthaus, J., Stella, L., Wang, Y., and Januschowski, T. Deep state space models for time series forecasting. NeurIPS, pp. 7796-7805, 2018.
Nie, Y., Nguyen, N. H., Sinthong, P., and Kalagnanam, J. A time series is worth 64 words: Long-term forecasting with transformers. In ICI.R 2023.
12

---

A Haapman-KKlman Enhanced Variational AutoEncoder for Probabilistic Time Series Forecasting
Rasul, K., Seward, C., Schuster, I., and Vollgraf, R. Au- toregressive denoising diffusion models for multivariate probabilistic time series forecasting, In ICML, volume 139, pp. 8857-8868, 2021a.
Wang, H., Chen, Z., Liu, Z., Pan, L., Xu, H., Liao, Y., Li, H., and Liu, X. Spot-i: Similarity preserved optimal transport for industrial iot data imputation. IEEE Transactions on Industrial Informaties, 2024c.
Rasul, K. Sheikh, A.. Schuster, I., Bergmann, U. M., and Vollgraf, R. Multivariate probabilistic time series forecast- ing via conditioned normalizing flows. In ICLR, 2021b.
Wang. H., Wang. Z., Niu, Y., Liu, Z., Li, H., Liao, Y., Huang, Y., and Liu, X. An accurate and interpretable framework for trustworthy process monitoring. IEEE Trans. Artif. Intell., 5(5):2241-2252, 2024d.
Salinas, D., Flunkert, V., Gasthaus, J., and Januschowski, T. Deepar: Probabilistic forecasting with autoregressive recurrent networks. International journal of forecasting, 36(3):1181-1191, 2020. 202..
Wang, H., Chen, Z., Zhang, H., Li. Z., Pan, L., Li, H., and Gong, M. Debiased recommendation via wasserstein causal balancing, ACM Transactions on Information Sys- tems, 2025a.
Schmid, P. J. Dynamic mode decomposition of numerical and experimental data. Journal fluid mechanics, 656: 5-28, 2010.
Wang. J., Zhang, 2., He, Y., Song, Y., Shi, T., Li, Y., Xu, H., Wu, K., Qian, G., Chen, Q., et al. Enhancing code llms with reinforcement learning in code generation. arXiv preprint arXiv:2412.20553 2024c.
Sezer, O. B., Gudelek, M. U., and Özbayoglu, A. M. Fi- nancial time series forecasting with deep learning A systematic literature review: 2005-2019. Appl. Soft Com- put., 90: 106181. 2020.
Wang, Y., Qiu, Y., Shu, Y., Rao, Z., Pan, L., Yang, B., and Chenjuan, G. Lightgts: A lightweight general time series forecasting model. In ICML, 2025b.
Simon, D. Kalman filtering. Embedded systems program- ming, 14(6):72-79, 2001.
Wang. Y., Qiu, Y., Zhao, k., Shu. Y., Rao, Z., Pan, L., Yang, B., and Chenjuan, G. Towards a general time series forecasting model with unified representation and adaptive transfer. In ICML, 2025c.
Sun, Y., Xie, Z., Wang, H., Huang. X., and Hu, Q. Solar wind speed prediction via graph attention network. Space Weather, 20(7), 2022.
Wang, Z., Pei, C., Ma, M., Wang. X., Li, Z., Pei, D., Rajmo- han, S., Zhang, D., Lin, Q., Zhang, H., et al. Revisiting vae for unsupervised time series anomaly detection: A frequency perspective. In WWW, pp. 3096-3105, 2024f.
Tang, B. and Matteson, D. Probabilistic transformer for time series analysis. In NeurlPS, pp. 23592-23608, 2021.
Tashiro, Y., Song, J., Song, Y., and Ermon, S. CSDI: condi- tional basee diffusion models for probabilistic time series imputation. In NewrIPS, pp. 14804-24816 16, 2021.
Welch, G. An introduction the kalman filter. 1995.
Wu, H., Hu, T., Liu, Y., Zhou, H., Wang, J., and Long, M. Timesnet: Temporal 2d-variation modeling for general time series analysis. In ICLR, 2023a.
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., and Polosukhin, I. Attention is all you need. In NeurlPS, pp. 5998-6008, 2017.
Wu, W., Qiu, X., Song, S., Huang, X., Ma, F,, and Xiao, J. Prompt categories cluster for weakly supervised semantic segmentation. arXiv preprint 2oXix:2412.:35
2024a.
Wang, C., Zhuang, Z., Qi, Q. Wang, J., Wang, X., Sun, H., and Liao, J. Drift doesn't matter: Dynamic decomposition with diffusion reconstruction for unstable multivariate time series anomaly detection. In NeurlPS, pp. 10758- 10774, 2023.
Wu, W., Dai, T., Chen, Z., Huang, X., Ma, F,, and Xiao, J. Generative prompt controlled diffusion for weakly supervised semantic segmentation. Neurocomputing, pp. 130103, 2025a.
Wang, H., Chen, Z., Liu, Z., Li, H., Yang, D., Liu, X., and Li, H. Entire space counterfactual learning for reliable content recommendations. IEEE Trans. Autom, Sci. Eng., pp. 12 2024a.
Wu, W., Song, S., Qiu, X., Huang, X., Ma, F., and Xiao, J. Image fusion for cross-domain sequential recommen- dation. In Companion Proceedings of the ACM Web Conference 2025, 2025b.
Wang, H., Chen, Z., Liu, Z., Li, H., Yang, D., Liu, X., and Li, H. Entire space counterfactual learning for reli- able content recommendations. IEEE Transactions on Information Forensics and Security, 2024b.
Wu, X., Zhang, D., Zhang, M., Guo, C., Yang, B., and Jensen, C. S. AutoCTS+: Joint neural architecture and hyperparameter search for correlated time series forecast- ing. Proc. ACM Manag. Data, 1(1):97:1-977 97:26, 2023b.
13

---

A Kropman-Kalman Enhanced Variational AutoEncoder for Probabilistic Time Series Forecasting
Wu, X., Wu, X., Yang, B., Zhou, L., Guo. C., Qiu, X., Hu, J., Sheng, Z.., and Jensen, C. S. AutoCTS++: zero-shot joint neural architecture and hyperparameter search for correlated time series forecasting. The VLDB Journal, 33 (5):1743-177) 1770, 2024b.
Yu, X., Elazab, A., Ge, R., Jin, H., Jiang, X., Jia, G., Wu, Q., Shi, Q., and Wang. C. Ich-scnet: Intracerebral hemor- rhage segmentation and prognosis classification network using clip-guided sam mechanism. In 2024 IEEE Inter- national Conference on Bioinformatics and Biomedicine (BIBM), pp. 2795-2800, 2024b.
Wu, X., Qiu, X., Li, Z., Wang, Y., Hu, J.. Guo, C., Xiong, H., and Yang. B. CATCH: Channel-aware multivariate time series anomaly detection via frequency patching. In ICLR, 2025c.
Yu, X., Li, X., Ge, R., Wu, S., Elazab, A., Zhu, J., Zhang, L., Jia, G., Xu, T., Wan, X., et al. Ichpro: Intracerebral hemorrhage prognosis classification via joint-attention fusion based 3d cross-modal network. In 2024 IEEE International Symposium on Biomedical Imaging (ISBI), pp. 1-5, 2024c.
Wu, X., Wu, X., Zhang, D., Zhang, M., Guo, C., Yang, B., and Jensen, C. S. Fully automated correlated time series forecasting in minutes. In Proc. VLDB Endow.,
Yu, X., Elazab, A., Ge, R., Zhu, J., Zhang, L., Jia, G., Wu, Q., Wan. X., Li, L.. and Wang, C. Ich-prnet: cross- modal intracerebral haemorrhage prognostic prediction method using joint-attention interaction mechanism. Neu- ral Networks, 184:107096, 2025b.
volume 18, pp. 144-157, 2025d.
Xu, Z., Zeng, A., and Xu, Q. FITS: modeling time series with 10k parameters. In ICLR, 2024.
Yang. S. B., Guo, C., Hu, J., Tang, J., and Yang, B. Un- supervised path representation learning with curriculum negative sampling. In IJCAI, pp. 2286-3292, 2021.
Zeng, A., Chen, M., Zhang, L., and Xu, Q. Are transformers effective for time series forecasting? In AAAI, volume 37, pp. 11121-11128, 128, 2023.
Yang. B., Guo, C., and Yang, B. Context-aware path ranking in road networks. IEEE Trans. Knowl. Data Eng., 34(7):3133-3168, 2022.
Zhang. J.. Wen, X., Zhang. Z.. Zheng. S., Li, J., and Bian, J. ProhTS: Benchmarking point and distributional fore- casting across diverse prediction horizons. In NeurIPS, 2024a.
Yao, Y., Li, D., Jie, H., Chen, L., Li, T., Chen, J., Wang, J., Li, F., and Gao, Y. Simplets: An efficient and universal model selection framework for time series forecasting. Proc. VLDBB Endow., 16(12):3741-3753 2023.
Zhang, Q. and Qi. Y. Can milms guide weakly-supervised temporal action localization tasks? arXiv preprint arXiv:2411.08460 2024.
Yao, Y., Jie, H., Chen, L., Li, T., Gao, Y., and Wen, S. Tsec: An efficient and effective framework for time series classification. In ICDE, pp. 1394-1406, 2024.
Zhang, Q., Liu, X., Li, W., Chen, H., Liu, J., Hu, J., Xiong, Z. Yuan, C., and Wang, Y. Distilling semantic priors from sam to efficient image restoration models. In Proceedings of the IEEF/CVF Conference Computer Vision and Pattern Recognition, pp. 25409-25419, 2024b.
Yi, Q. He, Y., Wang, J., Song, X., Qian, S., Zhang, M., Sun, L., and Shi, T. Score: Story coherence and re- trieval enhancement for ai narratives. arXiv preprint arXiv:2503.23512.. 2025.
Zhang, Q. Qi, Y., Tang, X., Fang. J., Lin, X., Zhang, K., and Yuan, C. Imdprompter: Adapting sam to image manipulation detection by cross-view automated prompt learning. arXiv preprint arXiv:2502.02454, 2025a.
Yu, C., Wang, F., Shao, Z., Sun, T., Wu, L., and Xu, Y. Ds- former: double sampling transformer for multivariate time series long-term prediction. In CIKM, pp. 3062- 3072, 2023.
Zhang, Q. Qi, Y., Tang, X., Yuan, R., Lin, X., Zhang, K., and Yuan, C. Rethinking pseudo-label guided learn- ing for weakly supervised temporal action localization from the perspective of noise correction. arXiv preprint arXiv:2501. /24, 2025b.
Yu, C., Wang, F., Shao, Z., Qian, T., Zhang, Z., Wei, W., and Xu, Y. Ginar: An end-to-end multivariate time se- ries forecasting model suitable for variable missing. In SIGKDD, pp. 3989-4000, 2024a.
Zhao, K., Guo, C., Cheng, Y., Han, P., Zhang, M., and Yang, B. Multiple time series forecasting with dynamic graph modeling. Proc. VLDB Endow, 17(4)7553-765 2023.
Yu, C., Wang, F. Shao, Z., Qian, T., Zhang, Z., Wei, W., An, Z., Wang, Q. and Xu, Y. Ginar+: A robust end-to-end framework for multivariate time series forecasting with missing values. IEEE Transactions on Knowledge and Data Engineering, pp. 1-14, 2025a.
Zhou, H., Zhang, S., Peng, J., Zhang, S., Li, J., Xiong, H., and Zhang, W. Informer: Beyond efficient transformer for long sequence time-series forecasting. In AAAI, vol- ume 35, pp. 11106-11115, 2021.
14

---

A Koopman- Kalman Enhanced Variational AutoEncoder for Probabilistic Time Series Forecasting
A. Theoretical Analyses
A.1. The Stability of KalmanNet
Since the proposed KalmanNet works in a data-driven manner, the seating-point operation error may cause the covariance matrix P losing positive definiteness, which often occurs in the step (21). To mitigate this, we utilize a numerically stable form for this step.
Theorem A.1. The
of covariance matrix Pk during the update step Pa = (I- - KxH can be retained through numerically stable form:
Pk = 2 (P& + P)), (I K2 Hx) PA(T - K&HL)* + KgRKKT
(30)
pdual
(31)
Proof.
The goal is to demonstrate the equivalence of the numerically stable form and original form: Pdual Pk-
Pdual
(I KxHH)PP(I K&H,)² + K
-I-KKKTH LI-KHHe)PE (I-K,
KRR,KE - T-K - P. K.H.P + KARAKT
(I K.H&)PP P. + KH R
K2 PA (HaP + R)-
pduol (1 K.HH))P P PAHE
+P.IEKE
[I-KKHHa)P =
Pdual is numerically equivalent to the original PE-
where (30) ensures the symmetry, (31) stabilizes the pesitive-definiteness by decomposing the formula into the sum of two positive definite terms, which better ensures positive definiteness during floating operation.
A.2. The Convergence of VAV
Since VAE models a linear dynamical system in the measurement space where the Koopman Operator serves as the state transition equation, we hope that the convergence state the KalmanNet does not violate the assumptions Koopman Theory. In VAE, we meticulously design the KalmanNet by making it gradually converge to the Koopman Operator in the forecasting horizon.
Theorem A.2. When U + 0, the state transition equation of the KalmanNet gradually converges to the Koopman Operator.
Proof. Under the assumptions of Koopman Theory, "y + 0 means the linear system constructed by Koopman Operator has little bias in the current measurement space, which leads to high performance in prediction. Meanwhile, the Predict and Update Steps of 2, are converted to:
Predict:
E4 = Azk-1
(32)
Update: K₁. Px (HPWHT + +R)
(33)
Zn -e Ka H
(34)
In this basic case, the state transition equation obeys the basic assumptions of Koopman Theory and A can be treated as "fine-tuned" Koopman Operator IC which is enhanced by the Kalman gain and has stronger generalization ability.
We then consider the special case that KalmanNet fully relys on the observation a from the linear system constructed by Koopman Operator XC, thus H + A, + 0, R. + 0, the Predict and Update Steps are converted to:
Predict: &
(35)
Update: 2k: = a
(36)
The system constructed by KalmanNet can be treated as 2, K2,-1 equivalent to the original Koopman Operator.
15

---

A Ghopman-Kalman Enhanced Variational AutoEncoder for Probabilistic Time Series Forecasting
B. Related Works
B.1. Time Series Forecasting
Time series forecasting (TSF) predicts future observations based on historical observations. TSF methods are mainly categorized into three distinct approaches: (1) statistical learning- methods, (2) machine learning- -based methods, and (3) deep learning-based methods. Early TSF methods primarily rely on statistical learning approaches such as ARIMA (Box & Pierce, 1970), ETS (Hyndman al., 2008), and VAR (Godahewa al., 2021). With advancements in machine learning, methods like XGBoost (Chen & Guestrin, 2016), Random Forests (Breiman, 2001), and LightGBM (Ke et al., 2017) gain popularity for handling nonlinear patterns. However, these methods still require manual feature engineering and model design. Recently, deep learning has made impressive progress in natural language processing (Chen et al., 2024; Zhang & Qi, 2024; Wang et al., 2024e; Wu et al., 2024a; 2025a), computer vision (Zhang et al., 2025b; 2024b; Wu et al., 2025b; Cui et al., 2024b; Yu et al., 2024b; Li et al., 2025a;b; Yu et al., 2024c), multimodal (Zhang et al., 2025a; Cui et al., 2024a; Jing et al., 2023; 2024), and other aspects (Wang et al., 2025a; Yu al., 2025b; Cui et al., 2025; Yi et al., 2025; Li al.. 2023b; Miao et al., 2024b; Wu al., 2023b; Zhao et al., 2023; Chen al., 2025). Studies have shown that learned features may perform better than human-designed features (Qiu et al., 2025b;a; Liu et al., 2025b; Yu et al., 2024a). Leveraging the representation learning of deep neural networks (DNNs), many deep learning-based methods emerge. TimesNet (Wu et al., 2023a) and SegRNN (Lin et al., 2023) model time series as vector sequences, using CNNs or RNNs to capture temporal dependencies. Additionally, Transformer architectures, including Informer (Zhou et al., 2021), Dsformer (Yu et al., 2023), TimeFilter (Hu et al., 2025b), TimeBridge (Liu et al., 2025c), PDF (Dai et al., 2024), Triformer (Cirstea et al., 2022a), PatchTST (Nie et al., 2023), ROSE (Wang et al., 2025c), LightGTS (Wang et al., 2025b), and MagicScaler (Pan al.. 2023b) capture complex relationships between time points more accurately, significantly improving forecasting performance. MLP-based methods, including DUET (Qiu et al., 2025d), AMD (Hu et al., 2025a), SparseTSF (Lin et al., 2024b), CycleNet (Lin et al., 2024c), NL.near (Zeng et al., 2023), and DLiiner (Zeng et al., 2023), adopt simpler architectures with fewer parameters but still achieve highly competitive forecasting accuracy.
C. Experimental Details
C.1. Datasets
In order to comprehensively evaluate the performance of VAV,E, we conduct experiments on 8 datasets of short-term forecasting and 9 datasets of long-teri forecasting under the framework of ProbTS (Zhang et al., 2024a), comprehensive benchmark used to evaluate probabilistic forecasting tasks. Specifically, use the datasets ETTh1-S, ETTh2-S, S, ETTm1-S, ETTm2-S, Electricity-S,, Solar-S, Traffic-S, and Exchange-S for short-term forecasting, of which the context length is equivalent to forecasting horizon with T = L 30 for Exchange-S and T = L = 24 for the others. For long-term forecasting, we use the datasets ETThI-L, ETTh2-L, ETTmI-L, ETTm2-L, Electricity-L, Traffic-L, Exchange-L, Weather-L, and III--I with prediction length I. E (24, 36, 48, 60) for II.I-I and E (96, 192, 336, 720] for the others. Note that we fix the context length of all the models with T 36 for IIII and 96 for the others to ensure a fair comparison. Pleuse note that although datasets with the same prefix may appear similar, they are not necessarily the same. For example, Electricity-L and Electricity-S S are not the same dataset, despite both having the prefix "Electricity" The datasets use are all derived from the authoritative probabilistic forecasting benchmark, ProbTS. Furthermore, due to the differences in long-term and short-term tasks, the datasets used for long- term and short-term forecasting in ProbTS and K2 VAV are also different. Table 7 lists statistics of the multivariate time series datasets.
C.2. Baselines
In the realm of probabilistic time series forecasting, numerous models have surfaced in recent years. Following the experimental setting in ProbTS, we compare VAVAE with 11 strong baselines including 4 point forecasting models: FITS, PatchTST, iTransformer, Koopa, and 7 generative models: TSDiff, V GRU NVP, GRU MAF, Trans MAF, TimeGrad, CSDI on both short-term and long-term probabilistic forecasting scenarios. The specific code repositories for each of these models-see Table 8.
C.3. Evaluation Metrics
We use two commonly-used metrics NMAE (Normalized Mean Absolute Error) and CPRS (Continuous Ranked Probability Score) in ProbTS (Zhang et al., 2024a) evaluate the probabilistic forecasts.
16

---

A Koopman-Kalman Enhanced Variational AutoEncoder for Probabilistic Time Series Forecasting
Table Dataset Summary.
Horizon
Dataset
#var.
range
freq.
timesteps
Description
ETTh1/h2-L
R+
H
17,420
Electricity transformer temperature per hour
ETTml/m2-L
7
R
15min
69,680
Electricity transformer temperature every 15 min
Electricity-L.
321
R+
26,304
Electricity consumption (Kwh)
Long-term
Traffic-L
862
(0,1)
H
17,544
Road occupancy rates
Exchange-L
R+
Busi. Day
7,588
Daily exchange rates of 8 countries
ILI-L
(0,1)
966
Ratio of patients seen with influenza-like illness
Weather-I.
21
R+
10min
52,696
Local climatological data
ETTh1/h2-S
R+
17,420
Electricity transformer temperature per hour
ETTmI/m2-S
7
R+
15min
69,680
Electricity transformer temperature every 15 min
Exchange-S
R+
Busi. Day
6,071
Daily exchange rates of 8 countries
Short-term
Solar-S
137
R+
7,009
Solar power production records
Electricity-S
370
R+
H
5,833
Electricity consumption
Traffic-S
963
(0,1)
H
4,001
Road occupancy rates
Table 8. Code repositories for baselines.
Baselines
Code repositories
Koopa
http:://ithub.commhun/kkog
iTransformer
http://gttaabccoommmmm////r/ssem
FITS
http://ggghabbccom/EEEwxXXXCCCC
PatchTST
http:///ghhabbcco////ugggeee//fr
TSDifT
D
GRU NVP
http://gtth...oom/aaddddceccccct
GRU MAF
Trans MAF
TimeGrad
http:///gttuabccoom/yyyggeeeeest
CSDI
http://ggthbbbocom//ccmmoggggggo
VAE (ours)
Normalized Mean Absolute Error (NMAE) The Normalized Mean Absolute Error (NMAE) is normalized version of the MAE, which is dimensionless and facilitates the comparability of the error magnitude across different datasets or scales. The mathematical representation of NMAE is given by:
K NMAE
21=
(37)
ΣT
|
Continuous Ranked Probability Score (CRPS) 1976) quantifies the agreement between a cumulative distribution function (CDF) F and an observation x, represented as:
The Continuous Ranked Probability Score (CRPS) (Matheson & Winkler,
CRPS
(F(2) 1(x S d2)
(38)
17

---

A Koopman-Kalman Kalman Enhanced Variational AutoEncoder for Probabilistic Time Series Forecasting
where (2x S 2} denotes the indicator function, equating if x and otherwise.
Being a proper scoring function, CRPS reaches its minimum when the predictive distribution F' coincides with the data distribution. When using the empirical CDF of F, denoted as F(z) = : > I[x; S 2), where represents the number of samples X; ~ F, CRPS can be precisely calculated from the simulated samples of the conditional distribution pe (x, (h)). In our practice, 100 samples are employed to estimate the empirical CDF.
C.4. Implementation Details
For each method, we adhere to the parameter searches across multiple sets, with a limit of sets. The optimal result is then selected from these evaluations, contributing to a comprehensive and unbiased assessment of each method's performance.
as specified in their original papers. Additionally, we perform hyper-
The "Drop Last" issue is reported by several researchers (Qiu al., 2024; 2025c; Li et al., 2025c). That is, in some previous works evaluating the model on test with drop-last=True setting may cause additional errors related to test batch size. In our experiment, to ensure fair comparison in the future, set the drop last to False for all baselines to avoid this issue.
All experiments are conducted using Py Porch (Paszke et al., 2019) in Python 3.10 and execute on an NVIDIA Tesla-A800 GPU. The training process is guided by the LELBO and Leee, employing the ADAM optimizer. Initially, the batch size is set to 32, with the option to reduce it by half (to a minimum of 8) in case of an Out-Of-Memory (OOM) situation. To ensure repro- ducibility and facilitate experimentation, datasets and code are available at:
18

---

A Koopman- Kalman Enhanced Variational AutoEncoder for Probabilistic Time Series Forecasting
C.5. Full Results
We provide all the main results of LPTSF in Table and Table 10, covering all four horizons (L E (96, 192, 336, 720)) on 9 real world datasets. The results show that VAE achieves a comprehensive lead in long-term prediction tasks, not only outperforming generative models specialized for probabilistic prediction but also demonstrating significant advantages compared to long-term boint-baeed prediction models.
We provide the complete results of ablation studies in Table 12-13. We compare the different variants under various tasks across different horizons, empirical results demonstrate that VAE adopts the most appropriate design.
We also provide the complete efficiency analyses under different forecasting scenarios, which demonstrates that our proposed VAE exhibits low memory overhead, fast inference speed, and high accuracy across various tasks. Compared to generative models such as those diffusion-based or flow-based models, VAV is both more precise and lightweight.
Table 9. Results of CRPS (mean.x) on long-term forecasting scenarios, each containing five independent runs with different seeds. The context length is set to 36 for the ILI-L dataset and 96 for the others. Lower CRPS values indicate better predictions. The means and standard errors are hased on 5 independent runs of retraining and evaluation. Red: the best, Blue: the 2nd best.
Dataset
Horizon|
Koopa
TTansfoomer
FITS
PaccTTST
GRU MAF
Trans MAF
TSDaff
CSDI
TimeGrnd
GRU NVP
X=VAE
96
0.301 00.
oss
00.26:20.023
0.2995 0.261
0.313 045
0.344
0.236
ood
0.522 100
0.3833
0.232 010
ETTmI-L
192
0.28010.021 021
0.3141
023
0.261 10111000028 n22
0.2761. 030
0.38910.00 603
0.424 029
0.345.10.0 eas
0.291 10 055
0.603119 099
0.396100000
0.259 013
336
0,286=0.00
0.3114
ous
0.285 0.27540.000
oze
0:42940.02
0.48140.0.9
0.46240.044
0.32220.009
0.601
0.48620.082
0.262.0.00 oso
720
0.295++.20 027
0.455+o.c.
021
0.305+0.024
+.300+
0.536+a.c0
0.688+o.c43
0.478++0.o27
0.448+0.oas
1.621+9.c07
0.546++.cap
0,294+o.08 022
00.17++0.02
0,181+0 os:
0.1662+0.oss
0..177+0.00 0.142+0.054
0.227+0.023
0.175++0.029
0.115-0.009
0.427+0.co4
0.319+0.044
0.126++0.c0
ETTm2-1.
192
1854000014
0.190
010
0.172 0.185=20.oss
ozs
0.411#0.00 oze
os? 0.253
0.255 029
0.147 oos
0.4244.9.cmm oas
0.326=0.022
0.1 co9
336
0.198++0.03
0.206++o
oss
0.218+0..03
0.195++o.00 042
0.377+0.023
0.253++.0:3
0.328+t.047
0.190-o.028
0.469
0.449-0.345
0,164 .10
720
0.23311.022
0.31110.00
021
0.44990.004
0.220
0.27210.00
0.355.1..00
0.34410.016
0.23010.005
0.47010.com
1.661
96
10.
0.202
1.o o3s
[0.29410.oo3
0.312 o3e
[0.29310.00
037
0.333 045
0.395:10.00
0.437
0.455110.04
0.379...cc0
0,264 020
ETTh1-L
192
0.301
0.298
0.304+0.02s
0.313+*..04
0.348++..07
0.351+t..0
0.467+0.0044
0,496+0.001
0.016+0.com
0.425+0.019
0.290=0.016
336
0.312.0
0.327 1. 013
0.31810.00
0.3191. 6a5
0.377.10
0.371 031
0.4501. 027
car 0.5121. 1.454
0.458100004
0.308 021
720
0.318 ooy
0,350 019
0.31820.022
0.323 cre
0.393 o4s
0,363 osa
0.51620.c27
0.528
0.523
0.502200.00
0.31420.00 011
96
10.199 012
0.1852. 01s
00188720.01
0.197 os1
[0:23910 o19
0.263 0s0
0.33641 oz:
0.164....00 ora
0.35811 coe
0.4432.0 141
0.162.0 000
ETT12-L
192
0. 0.198++0.0 022
0.199+0.00 010
0.195+0.oz2
0.313+0..0 0.2044+0.0s
ose
054
0.273+
024
0.205++..o45
0.226+a.oLs
0.457+0.os1
0. 0.625+0..70
0,186+=0
018
336
0.26220.019
0,271
oss
0.246=0.044
0,277
0.376
0,265
0.350+2..o0
0.274
0.481
0.793
0,257 29.7490.22
720
0.293
0.542+ 018
0.314+0.022
0.304+a.022
0.990++..o2
02.
0.327+0.o33
0.406++..00
0.302+0.04:
0.445++n.ose
0.539+0.000
0,250 .14
96
00.110=0.00
0.102++.00 004
001105+0.006
0.126++..ox 0o0
0.083+0.000
0.344+0.co0 0.058+0.0:4
0.1 357
0.096+9.coz
000911=0.005
0,073
Electricht-1.
0.109
0.10411..00 014
0.11220.304
0.12310.00
632
0.083
0.097.10.0 0o0
0.345
0.200.n.064
0.1001...
0.097:10.003
0.080 .04
336
0.121++0.011
0.104+
010
0.111+0.004
0.131+0.004
0.095++..00
0.462+0.054
0.102++..com
0.099-0.001
0.054 .011
720
00113310:08
0.10010.00 014
00.11510.024
0.127
0.106 10.00.00000
0.478110.00
0.10811..cm
0.114_o.00a
0.057 005
96
10.29710.019
0.26611
0o4
[0.25810.004
0.119410.000
[0.21510.00m 0.0
0.208 1. 004
0.294:10.om
0.20211. coo
D.187 10.002
0.086 001
Traffic-L
336
192
0.334100017
0.30820.00
0.26110
0.2501.0 009
0.327.10.00
(.225520.00 000
0.198100 004
0.204
0.3061. 004
0.31710.cc
0.21310
0.20811
0.201
0.192220.00
0JI8N 002
0.19510.00
720
0.358 oz2
0,284 0044
0.37442.004
0.214:20.00
0.301 +1000000022
0.220 c022
0.211 20.00
0.200 001
96
0.13220.000
0.1311:
[0:21020.01a
0.1111.0
0.1399...00
0.105
0.704..0.co
0.068
0.1301:0.00
0.11620.018
0.08011..cm
Weather-1.
192
01s 0,132+a.0r 0.133+0.017
0.205++0.029
0.131+o.o:4
0.143++..o
0.142+0.co 022
0.134+0..02
0.068
006
ous 0.127++0.co
0.123+0.021
0.079
336
0.136.00.02
0.132 e2o
0.137 i oos 0.22111..00
0.129:10.00 012
0.13329.0:4
0.137:1.000
0.043
n0002
000
0.130 ooo
0.128::0.011
0.082
720
0.100+0.007
0,133+
0.267+0.003
0.142++o.o0
0.122+0.000
0.113+0.004
0.152+0.008
0.087
0.113+n.011
0.110+0.004
0.084-0.005
96
10.063+0.006
0,061++.00 oos
[0.088+0.004
0.063 ooe
0.026++0.00o
0.028+0.ooz
0.079+n.0o7
1.028-o..00
0.068++n.cos
0.071++0.ood
0.031 +0.002
Exchange-L
192
0.065110.00
0.0621..000
0.067 0.04911.00:
0.034:10.00
0.04610.037
0.093
0.087 0.046110.003
0.068
0.032
336
0.072210 008
0.067.0 oos
0.052210.00 o1a
0.071 017
0.05810
0.045100010
0.081 1.0 0o7
0.0602.0.0 004
0.074 coe
0.072200000
0.048.10 004
720
00.09116.00
0.087.1
023
0.074
0.097
007 007
0.16010.019
0.14810.037
0.08210.00
oxo
0.143 090
0.
0.09010..01
0.0791o..oe
0.069 005
24
10:24510.002
0.2121
o33
[0:23310.01:
0.3121a.0:
[0.09710..0 030
0.092 Lo o:s
0.228
0.2500n.o::a
0.275 047
0.257.10.003
0.087.0 .00
ILL.L
36
0.21422000 0o8
0.1822
016
0.21720 020
0.241 021
0.11720.00 017
0.115
011
0.235 one
0.286
01.0
0.2721. os7
0.281 20:01
0.113:: 005
18
0.271=0.02
0,213+0.00
0.185#0.026
0.212+0.0.e
0.128+0.029
0.133+0.00
0.265+0.com
0,285
0.295++..cm
0.288=0.005
0.124=0.010
00.22810.023
0.222 020
0.21110.001
0.23310.033
0.17221..044
D.150
0.263310.022
0.28330.013
0.295
csa
0.307:20.005
0.142::0000 .03
Due excessive time and memory consumption, some resalls are unsvallable our implementaaioon denoted 35
19

---

A Koopman- Koopman-Kalman Enhanced Variational AutoEncoder for Probabilistic Time Series Forecasting
Table 10. Results of NMAE (meanga) on long-term forecasting scenarios, each containing five independent runs with different seeds. The context length is set to 36 for the ILI-L dataset and 96 for the others. Lower NMAE values indicate better predictions. The means and standard errors are based on independent runs of retraining and evaluation. Red: best, Blue: the 2nd best.
Hurioxe
Koopa
TTaanfommer
FITS
PaccTST
GRU
TSD
CSDI
TineGrd
NVP
K²VAE
96
0.362++..02
0.309 029
0.349
0.329 300
0.402+0.007
0.456+0.042
0.441+n.02:
0.308 ocd
0.645+0.12
0.284-o.00 0.438+0.00s
ETTm1-1
336
192
0.364:2.320 000
0.365+0.00 052
0.38440.00 041
0.380
020
0.35610.022
0.341+0.002
0.338+0.00
0.34410.003 00.
02.
0.476+0.046
0.590 0.522100.00
0.553+0.022
0.07110.c03
0.411+0.019
0.41920.04:
0.748+0.co
0.75911.005
0.514+0.042
0.630.0.c20
0.330 014
020
730
(0.37720
0.4901. oss
0.4066..000 072
0.382 ose
0.71110:06
0.82210 034
0.62210 045
0.67829.00 051
0.70319 ca4
0.70720.000
0.373. 052
96
(0.22511
1 009
0.221.10 oss
0.211010:000
0.2161..00m oas
[..21210 062
0.270.0 oz1
0.224 caa
1.146
0.625.10.com 6:17
0.41310.009
0.144_0.011
ETTm2-L
192
0.233310 0..33.0..0000 020
0.2291.
oas
0.2151 0.234410.036
022
0.2921...0 0.63511.002 029
041
0.31610.040
0.150
1.1901..00 10.01
0.03011.c0m ooo
0.427:1.003
0.1701:0 009
0.267
0:24511
0:276100019
0:234410
0.40710.00
0.3091.
0.307
0:248
0.66610
0.68020.10
720
0.290=e..os
0,385+0.04 042
0.540=0.002
0,288+0.0s
0.385+0.044
0.475+0.c29
0.416+0.o6
0.306=0.040
0.561+9.044
0.749==.3ea
0.275e0.0st
96
10,407=0.002
0.386 ouz
0039930.1442
0.407
0.377
0.423
0.510=0.com
0.557=o.002
004881
0.336 041
ETTb1-
336
192
0.406+0.02
0.396220.00
0.415++0 oz2
0.3888. 04)
0.4062.0.00 0m9
0.410=0.06s
0.412+0.oz 0224
0.405.20.00 oss
0.430g0 0z2
0.462+0.040
0.481+0..0
0.451
041
0.581+n.ose
0.586110
0.574==..22
0.625..0.00 oe5
0. 0.686++0.c4
0.680110 osse
0.372.0 2.631
0.580=n.0e4
0 022
025
720
0.412
0.449
0.468:10.032
0.428
0.49610.029
0.45520.
0.617:100017
0.657:20.044
0.67210.015
0.643:0.046
0.0921
96
0.234+0.00
022
[0.243+0.000
0.247 o2s
(0.292+0.012
1.345+o.o42
0.421+n.003
D.214+n.028
0.448
0.5488+o.1ms
0,189 0.0
ETT22-1.
192
0.249
10-002
0.247
0.25210.092
0.265.10.0
0.37610-112
0.34310.00
0.339
0.294.1
0.67510.co
0.76610.220
N.21310..00
336
0.274 10.00.0.000
0.297:10
029
0.291110.00
0.314:10.00 045
0.454410.00 007
0.3331..00
0.427 044
0.35332.002
0.00G1..
0.942:10408
0.263:: 039
720
0.28622 012
0.66710 019
[0.401 099
0.371 o21
1.09210 o19
0.41220 090
0.482.. 099
0.38220 000
0.6501..co one
0.68820 161
0.278 020
96
00.16610.00
0.13410 oos
[D.137.0.00 002
0.16810.00 ons
D.1088
0.114.. ond
0..41
0.203
188
0.11101
0.1181...00
0.093 002
Electricity-L. -
192
0. 0.148=0.00 ozs
0.137 ozz
0. 0.143=20.1 132
0.163 os2
0.120 000
0.131#0.00 ove
0.441 2.000s
0.264 129
0.124 coo
0:
0.121 1:21=0.000 000
0.102:: 010
336
0.151 20177
0.1361.0 o02
0.139200000 002
0.16810.00 010
0.1234..00 018
0.671 1:0
0.126.10
0.123g...00
0.107 002
0.119=0.00
0.100+0 ooe
0.1199=0.012
0.16440.024
0.136+0.000
0.622++0.44
0.134+
0.144=0.017
0.117=0.02
96
00.377
0.332 ode
0.33220.007
0.228
002
0,265
0.31240.04z
0.2344
00.00
2.230
Traffie-L
336
0.368+0.511
0.416616.00 os8
0.326++ eoe
0.
010
0.400.10.011
0.350+0..00
0.242
0.225+0.012
10.002
0.39210..c 0.00
0.35:4+n.012
0.246110 coa
0.209
0.236+0.002
0.24810.006
0.204++.c0m
0.242..0 007
720
0..432:0.032
0.36110.00
eso
0.445320.022
1.2536n.022
0.478:10.006
0.26631n.co
0.26420.006
0.248 00.000
96
0.144+0.00 00.
017
0..29++0.027
0.1145+0.oce
00.176+0.011
0.139 oco
0.113+t..oz
0.087
+0.002
0,164 caa
0.1455+0.o17
0.086
Weather-L L
192
0.148110.0
022
0.14510.00
015
0.26410.003
0.14410..0
0.16610..00 022
0.16010 037
0.144410.00 000
0.0%G
LoNGGL..00 007
0.1.5810 0o4
0.147
0.08310 0.1
0.15222.000
0.1:662: 011
0.149:...00 0:28320.02
ou2
0.16820.014
21772...00 o07
0.138:0.com
0.0088
0.1624:..00
0.160:0.012
730
0.1622 009
0.1471. 019
0.15220 ox9
0:14011
0.148 040
0:141 1:0
0.10220 oos
0.13611 coe
0.135g00000
0.099 000
96
0.07710 10007900.005
0.070 10.06910.007
0.03311..0
0.0361
0.0901..00
0.0361n.0od
0.079
0.091 10.000
0.032
Exchange-1.
336
192
0.083+ 0.066+0.203
0.078 0.08120.015
oue
oos
0.071+0.005
0.081 0.069:20.007
0.085+a.020
002
0.074+0.017
0.044:0.00 004
0.058+ oos
0.058 our
0.106++n.010
0.106:20.co0
0.076+n.ood
0.058
ocs
0.1004:..co cae
0.086++n.co
0.091+t.oo2
0. 0.087=0.008
0.054-o 001
0.040:0 005
720
0.116=0.0zz
0,113+0..0
0.097+0.01:
0.126+0..0
0.182+0.020
0.191+0.oor
0.142+4..opp
0.173=e0.oz
0.113+t.cce
0.103=0..co
0.084=0.017
24
0.2651 10.30312..23
02r
0.271100.00
0.382 o:s
0.124:10.00
D.118 o33
0.242 ooo
0.296 0.2633n.0:2
.44
0.283:20.001
0.116 011
ILI-L
36
0.262+0.013
0.222++.00
0.258+0.05s
0.256++.035
0.144+0.00:
0.143+0
0.246+0.117
0.298+0.011
0.298 09988++..c4
0.307++..00
0.142+0.005
0..34410.33
0.26211. 023
(.225510.00
0.291 633
0.150
0.160
0.275
0014
0.301
0.320
0.31410.000
0.15210.00 017
60
10:28820.00 0s3
0.278 017
00:24520.01
0.287
o23
0. 0:21620.014
0.18320.0.0
0.272.0.co0 0.0
0. 0.325 0.29920.003
coe
0. 0.33320.005
0.16720 007
Die to the excessive time and memory consummpton, some resalis are unavallable in our implementation and denoted 25
20

---

A
Enhanced Variational AutoEncoder for Probabilistic Time Series Forecasting
C.6. Model Analysis
Table 11. Comparison on model efficiency. Lower values of Inference Speed (sec/sample) or Memory (GB) indicate higher model
efficiency. L: the forccasting horizon. The results are obtained with batch size equals 1. Red: the best, Blue; the 2nd best.
Model
Metric
Heorootyy U. PM0
ETTm1-L.
0. 120)
Bachonge-8
(1.s
Selan-5 (1:224)
Excmietty 960
ETTel--
0..90
Elcetiiity L
(1 1913
0.. 197)
Hlemmcny- 0. N96
BTT 336
TSCHIf
Mennar
0.866
0.352
12339
0.122
1314
20382
0.479
CRU NVP
Informaz Spood
Memarry
26,296
0447
6.823
1400
0.428
0.34
0.040
0.461
0.345
1,500
0011
0.191
0.173
7,444
0005
1815
12.41
0.344
0,018
1.356
GRU MAF
Informnee Speod
Menorry
125
18615
0.008
0317
0001
0.040
9.130
08.422
0.120
0.023
1651
177.56
0.175
0.034
4355
700000
0.568
0,055
9,888
Trans MAP
Inferonce Speed
Mennry
$32.151
0,368
19,401
0.081
0.011
0.883
9,375
0.087
453367
0.147
0.073
1.500
150 568
0.201
0
5.336
311.054
0.272
0,075
10.130
TinaGrad
Infermne Speed
Menorry
34.866
0.06
19:441
0.041
0.128
94388
0.066
0.301
104
155.013
0.038
284.551
0.044
CSD
Inference Speed
Memorry
0,133
0.183
358315
30004
478
25.836
0.033
0,011
X2VAE
Specil
534
0.029
0.015
0.374
Doe to the dxcesiiee tine and mecuocy cossaratios, some results are unuaalable in our amplenectation and dearted 6s
Table 12. Comparison on different Koopman Operators. Lower CRPS or NMAE values indicate better performance. Red: the best. L: the
forecasting horizon.
Koopman Operator
Electricity-L L (L= 720)
ETTmI-L
Exchange-S S
Solar-S (L 24)
Electricity--. II
ETTmI-L.
Electricity-I.
ET I-I.
Electricity-I.
ETTmI-L
Metrics
(L= 720)
(L=30) 30)
(L=
(L= 96)
(L= 192)
(L= 192)
(L= 336)
(L= 336)
CRPS
0.012=0.002
0. 0.07720.305 0.130020.02
0.114=0.00 0,20840.022
Kine
NMAR
0.014.0.002
0.50810005
0.10120:001
0.3871am
0.134:0::11
Kytt
CRPS
1.005
0 1002
0.01 00.000
1.304
0.070_n.ow
0.248_am
0.082.0.001
0.263
0.057
0.268
NMAE
0.130g0.00
0.395go 087
0.013:0.00
0.488-20.0ce
0. 1.10000..m 005
0.304ga.02
0. 0.10020.0 D13
0.329
0.11420.00 003
0.341
CRPS
0.057 20.00s
0.009
0.367
0.073
0.232
0,080 00:00
0.259
0.054.0.00
0.262
Kooo + Kegto
NMAE
0.117
0.009 o.
0.093 0.480:
0.284:cum
0.102 00.00
0.323
0.107
0.330
Due to the numerical instability, some results are unavailable in our implementation and denoted as.
Table 13. Comparison on different connections of KalmanNet. Lower CRPS or NMAE values indicate better performance. Red: the best.
L; the forecasting horizon.
Connections
Flectricity-I.
RTTm1-1. (L- (L-720)
Exchange-S (L.30)
Solar-8 (L.
Electricity--. (L- 96)
ETTm1---.
FllctricityyI. (L- 192)
ETTml-11.
Electricity-I.
RTTmi-1.
in KalmanNet
Metrics
(L - 720)
(L- 96)
aL- 192)
(L 336)
(L- 336)
w/o AuxiliaryNer
CRPS
0.08210.00 m1
0.3301
0.0152000
0.008_00055
0.26811 0:08310.co
0.0F410.001 0.2032001 0.10F200s
0.2671000
NMAB
0.18840.com cos
0.4 0.442
0.022=0000
0.531
0. 200
0.31880.01
0.1 0.14220012
0.330:400m
0.1 0.14770.003
0.346550.0s
CRPS
0.063
0.315:ams
0.011
0.388
0,243 0.092:0.co4
0.087
0.058 0.277ama1
0.266:am11
w/o skip connection
NMAB
0,131
0. 102e0us
0.013=0004
10511=000s
0. 0.11600..00
0. 0,292e0.010
0.11
0,370=00222
0. 0,119:0,000
0.319
w/o control ingut
CRPS
1.08910.005
0.3221am7
0.013L0000
0.423_000s
0.2421c.011 1.07910.c
1.084
0.28010011
0.06410.cm
1.27
NMAH
0.1 0.144250.014
0.41826.02
0.01720005
0.000000000
0.10420.c01
0.285a6..0 013
0.10820007
0.341:0010
0. 1112830.0m 002
0.356550.05
Mixed
CRPS
0.057.00.o eoG
0.2941a.0s
0.00900000
0.36720000
0.23220.000 000
0.080
0.259.0001
0.05410.00
0.262:a.00
NMAB
0,117#0.01
0.37360.02
0,009
0.450=00005
0,093
0,284:0.011
0,102
0,323=0020
0.107
0.330
Table 14. Ablations on KoopmanNet and KalmanNet. Lower CRPS or NMAE values indicate better performance. Red: the best. L: the
forecasting horizon.
Electricity-I. I.
ETTmm--1.
Exchange-S (L=
Solar-S (L= 24)
Electricity-L. (L= 96)
ETTml--. (L= 96)
Electricity-I.
ETTml--
Electricity-I.
ETTml-L
Variants
Metries
[L= 720)
(L= 720)
(L= 192)
(L
(L= 336)
(L 336)
w/o KoopmanNet
CRPS
100.0000
D. 0.113=0004
0.01isaa02
0.265=0005 0.0799:0.s 0.385
0.087=0000
0.288=0.010
0.11
0.291+0.007
NMAE
0. 0.162so.0u
0.801=0008
0.0162ame1
0.112:00006 0.528
0.112:00:12 0.
0.382s0.c2
0.372e0.027 0.5
wlo NalmanNet
CRPS
0.0112000
0.37
0.088
0.278_0003
0.001 20000
0.375
0. 0.29810.0m 003
0.301 10 nas
NMAE
0.192ac.00 025
0. 045
0.01240001
0.499 000
0.133=0000 o08
0.338-0012
0.122=0001
0.443a0.ca
0.350+0.017
0.30
A"
CRPS
0.057
0.294
2.009
0.367
0.073
0.232=00:0
0.080=0004
0.259:0.01
0.262:10.cm 2.054
NMAE
0.117
0.373
0.009
0.480 000
0.093
0.284=000:1
0.102-0000
0.323=n.220
0,107=0,002
0.330++.0144
21

---

A Koopman-Kalmnn Enhanced Variational AutoEncoder for Probabilistic Time Series Forecasting
C.7. Showcases
We provide some showcases of VAV in Figure and 7, which demonstrates the strong interval estimation capabilities of VAE. We observe that VAE achieves good performance in 95% confidence interval, which means the forecasting horizon of the time series is well modeled and estimated.
44
04000
43
Figure 5. Visualization input- ayput-24-put 2ww---ddiprr------ results on Solar-S dataset.
0.-
400
Figure Visualization of 9rpptt-6--redict-96 results on the ETTm1-L dataset.
Figure 7, Visualization 96-rreeict-96 results the Electricity-L dataset.
22


Reviews:
[{"review_id": "Df8wdM2Gau", "reviewer": "Reviewer_iEFP", "summary": "This study introduces $K^2$VAE , a VAE-based probabilistic forecasting model designed to address PTSF. By leveraging the KoopmanNet, $K^2$VAE converts nonlinear time series into a linear dynamical system, enabling a more effective representation of state transitions and inherent process uncertainties. Additionally, the KalmanNet models uncertainty within this linear dynamical system, reducing error accumulation in long-term forecasting tasks.", "questions": "", "limitations": "", "rating": 5, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**Reply to W1. The experimental design of this paper is very comprehensive, evaluating multiple datasets on both long and short step tasks. However, in Table 5, some datasets have the same name but different actual lengths. What is the reason for this?**\n\nThank you for your valuable comments.  We follow the evaluation protocol in ProbTS, a well known benchmark for probabilistic forecasting tasks.  Some datasets with same names but different suffixes, e.g., Electricity-S and Electricity-L, are different datasets with different lengths and channels. The ETT datasets share the same in short- and long-term probabilistic forecasting tasks. We have summarized the details of datasets in our appendix, we list the table here:\n\nhttps://anonymous.4open.science/r/K2VAE-D957/datasets_info.md\n\n\n\n**Reply to W2. The related work section discusses the application of VAE in time series data but seems to omit some classic algorithms, such as D3VAE. It is suggested to include the discussion and experiments of these algorithms.**\n\nThanks for the suggestions. Although $D^3$VAE is closer to a Diffusion model, it does indeed follow the paradigm of VAE. Specifically, $D^3$VAE is a dual-directional variational auto-encoder that combines diffusion, denoising, and factorization. By coupling diffusion probability models, it expands time series data, reduces data uncertainty, and simplifies the inference process. Furthermore, it treats latent variables as multivariate and minimizes total correlation to separate them, thereby enhancing the interpretability and stability of predictions. Both $D^3$VAE and $K^2$VAE aim to better model the uncertainty in the target window through decoupling methods, and we will discuss the similarities and differences in the Related Works section. In terms of experiments, we also provide a detailed comparison with $D^3$VAE, assessing $D^3$VAE in both short- and long-term PTSF tasks:\n\n\n  https://anonymous.4open.science/r/K2VAE-D957/compare_with_d3vae.md\n\nWe keep the contexutal length equals to the horizon length to meet the setting of $D^3$VAE. The results show that $K^2$VAE outperforms $D^3$VAE in both short and long step scenarios, and $D^3$VAE also seems less suitable for long-term PTSF tasks.\n\n\n\n**Reply to W3. Although the ablation studies of key components in KoopmanNet and KalmanNet are discussed, the complete removal of these two modules to analyze their impacts has not been considered.**\n\nFor KoopmanNet and KalmanNet, they act as two important parts of $K^2$VAE. We further analyze the impact of these two key modules on PTSF tasks, and supplement the ablation experiments using the same datasets in the paper: \n\nhttps://anonymous.4open.science/r/K2VAE-D957/ablations.md\n\nThe results demonstrate that KalmanNet is more critical for LPTSF tasks. When KalmanNet is removed, the arrcuracy breaks down sharply, indicating that the importance of KalmanNet to effectively eliminate the cumulative errors in LPTSF tasks. When removing KoopmanNet, on the other hand, would lead to a performance decline across all the tasks. Without KoopmanNet,  the nonlinear time series is hard to model, and the uncertainties in such nonlinear system is also difficult to be captured through KalmanNet, which furtherly declines the model's modeling capabilities. The experiment shows that both KoopmanNet and KalmanNet are critical and indispensable in our design.\n\n\n\n**Reply to W4. The conclusion of the article needs to be adjusted to meet the final format requirements of ICML, as it currently exceeds two lines.**\n\n Thanks for your reminder! We will fix it.\n\n\n\n**Thanks again for the Reviewer iEFP's valuable opinion! We deal with W1-W4, and provide analysis and evidence from the perspective of model design and experiment. Do our reply solve your question? If there are any other questions, we can discuss them further!**", "claims_and_evidence": "This work is dedicated to addressing the challenges of nonlinear phenomena in time series probabilistic forecasting and the cumulative errors in long-step predictions. Theoretically, it proposes corresponding modules for improvement based on Koopman theory and the Kalman Filter, respectively. Experimentally, it has been proven effective through evaluations under numerous settings and also demonstrated that the model is lighter compared to generative models based on Diffusion and Flow, suggesting a promising application outlook.", "methods_and_evaluation_criteria": "The design of $K^2$VAE appears to be quite sophisticated, with the proposed KoopmanNet and KalmanNet having logically reasonable connections. The KoopmanNet models nonlinear time series as a linear transition process between measurements, while the KalmanNet is suitable for handling error accumulation issues in linear dynamic systems. Extensive experimental evidence has also demonstrated the outstanding performance of $K^2$VAE in both short-term and long-term probabilistic prediction scenarios, while maintaining a lightweight structure.", "theoretical_claims": "I have checked the proofs of Theorems 3.1-3.2 in the appendix, and they are all correct and consistent with the purpose of the article.", "experimental_designs_or_analyses": "Although the ablation studies of key components in KoopmanNet and KalmanNet are discussed, the complete removal of these two modules to analyze their impacts has not been considered.", "relation_to_broader_scientific_literature": "This work has inspired the design of more lightweight probabilistic forecasting models in the field of time series, for full-scenario temporal probabilistic forecasting tasks. Previous Diffusion-based works generally caused a large amount of computational resource overhead and seemed unable to model probability distributions well over longer forecasting windows. The proposal of $K^2$VAE has inspired researchers to shift their focus to the design of VAE, spending more effort on designing proper structures that better conform to the inductive biases of time series.", "essential_references_not_discussed": "The related work section discusses the application of VAE in time series data but seems to omit some classic algorithms, such as $D^3$VAE. It is suggested to include the discussion and experiments of these algorithms.", "other_strengths_and_weaknesses": "Strength.\n\nS1. The paper introduce an efficient frameworkcalled $K^2$VAE. It transforms nonlinear time series into a linear dynamical system. Through predicting and refining the process uncertainty of the system.\n\nS2. $K^2$VAE demonstrates strong generative capability and excells in both the short- and long-term probabilistic forecasting.\n\nWeakness.\n\nW1. The experimental design of this paper is very comprehensive, evaluating multiple datasets on both long and short step tasks. However, in Table 5, some datasets have the same name but different actual lengths. What is the reason for this?\n\nW2. The related work section discusses the application of VAE in time series data but seems to omit some classic algorithms, such as $D^3$VAE. It is suggested to include the discussion and experiments of these algorithms.\n\nW3.  Although the ablation studies of key components in KoopmanNet and KalmanNet are discussed, the complete removal of these two modules to analyze their impacts has not been considered.\n\nW4. The conclusion of the article needs to be adjusted to meet the final format requirements of ICML, as it currently exceeds two lines.", "other_comments_or_suggestions": "The conclusion of the article needs to be adjusted to meet the final format requirements of ICML, as it currently exceeds two lines.", "questions_for_authors": "The experimental design of this paper is very comprehensive, evaluating multiple datasets on both long and short step tasks. However, in Table 5, some datasets have the same name but different actual lengths. What is the reason for this?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "oTf4dbZzqo", "reviewer": "Reviewer_rXKc", "summary": "This study presents an efficient framework named K2VAE, which transforms nonlinear time series into a linear dynamical system. By predicting and refining the process uncertainty within the system, K2VAE showcases powerful generative capabilities and excels in both short- and long-term probabilistic forecasting.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**Reply to W1. Koopman theory requires that the measurement function maps inputs to several observable variables to construct a linear dynamical process in that space. Traditional methods typically choose fixed basis functions to meet this requirement, but this paper adopts a learnable measurement function. How can we ensure that the constructed space is within a linear dynamical system?**\n\nThank you for your valuable comments. Koopman Theory requires that the measurement function possess sufficient projection capabilities to map nonlinear inputs into a high-dimensional measurement space and model them using a linear dynamical system. Traditional methods would use some fixed functions, such as polynomial bases, because these functions have strong high-order fitting capabilities. Similarly, in deep learning, the universal approximation theorem guarantees that multi-layer perceptron models (MLP) have strong fitting capabilities, theoretically able to approximate any continuous function to arbitrary precision. $K^2$VAE, as a deep learning model, also adopts multi-layer perceptron fitting for measurement functions. KoopmanNet can construct a linear dynamic system because it designs the Measurement Function $\\psi$ and Decoder $\\psi^{-1}$ in a fully symmetric manner and controls the outputs of the linear system constructed by the Koopman Operator $\\mathcal{K}$ close to the contextual time series in the original space by using reconstruction loss $\\mathcal{L} _{rec}$. Additionally, the Koopman Operator is also linear, which aligns with the assumptions of Koopman Theory.\n\n\n\n**Reply to W2. In the ablation experiments for the components of KoopmanNet in Table 3, why does using the local Koopman operator lead to numerical instability?**\n\nThis is because we use one-step eDMD to accelerate the efficient computation of $\\mathcal{K}_{loc}$, as shown in the following steps.\n\n$$X^{P^\\ast} _{back} = \\left[x^{P^\\ast} _1,x^{P^\\ast} _2,\\cdots,x^{P^\\ast} _{n-1}\\right]$$\n$$X^{P^\\ast} _{fore} = \\left[x^{P^\\ast} _2,x^{P^\\ast} _3,\\cdots,x^{P^\\ast} _{n}\\right]$$\n$$\\mathcal{K} _{loc} = X^{P^\\ast} _{fore} (X^{P^\\ast} _{back})^{\\dagger}$$\n\nBy using a one step offset and then calculating the pseudo-inverse, we estimate $\\mathcal{K} _{loc}$. This process is prone to matrix computation errors when the difference between the contextual length and the horizon length is significant (e.g., 96 and 720). And this phenomenon often occurs when the model is just beginning to be optimized, and the space constructed by the measurement function is not good enough. Theoretically, though the Moore-Penrose Pseudo Inverse always exists, the corresponding computational method relies on Singular Value Decomposition (SVD), which can be numerically unstable and lead to computation failures. Therefore, we need to use a learnable $\\mathcal{K} _{glo}$ to ensure the computational stability at the beginning of training.\n\n\n\n**Reply to W3. There is a lack of sensitivity analysis regarding the parameters, which obscures understanding the behavior of the proposed method in different hyperparameter settings.**\n\nThanks for your valuable comments. Following your suggestions, we systematically evaluated three critical hyerparameters: Patch Size $p$,  and the dimensionality $d$ of hidden layers in the Measurement Function, and the number of hidden layers $l$ in the Measurement Function. Specific experimental results are presented in the following Tables :\n\nhttps://anonymous.4open.science/r/K2VAE-D957/sensitivity.md\n\nThese empirical findings substantiate that $K^2$VAE maintains robust performance across different hyperparameter configurations. To achieve best performance, we recommand a group of stable hyperparameters for both short-term probabilistic forecasting and long-term probabilistic forecasting:\n\n1. **Patch Size**:\n   - Short-horizon forecasting tasks achieve peak performance with patch_size $p$: 8\n   - Long-horizon forecasting benefits from extended context capture with patch_size $p$: 24\n2. **Network Architecture**:\n   - Number of hidden layers $l$: 2-3 layers yield optimal performance-efficiency balance\n   - Dimensionality of hidden layers $d$: 256 provides sufficient representational capacity\n\n\n\n**Reply to Other Problems**\n\nWe will carefully examine and correct the normativity of the citation, thank you for the correction!\n\n\n\n**Thanks again for the Reviewer rXKc7's valuable opinion! We deal with W1-W3, and provide analysis and evidence from the perspective of model design and experiment. Do our reply solve your question? If there are any other questions, we can discuss them further!**", "claims_and_evidence": "Claim 1: This study asserts that KoopmanNet can fully leverage the inherent linear dynamical properties within the measurement function space, streamline modeling processes, and thereby enhance model efficiency.\n\nClaim 2: This study contends that KalmanNet can effectively mitigate error accumulation in long-term predictive state forecasting.\n\nBoth claims have been substantiated through ablation experiments.", "methods_and_evaluation_criteria": "This paper compares with the state-of-the-art (SOTA) in probabilistic forecasting, utilizing common evaluation metrics.", "theoretical_claims": "Most equations and derivations can be followed.", "experimental_designs_or_analyses": "Although the author has provided a detailed analysis of the overall performance, there is a lack of sensitivity analysis regarding the parameters.", "relation_to_broader_scientific_literature": "The author's methodology is related to time series analysis.", "essential_references_not_discussed": "Related works seem to be covered.", "other_strengths_and_weaknesses": "Strength.\n\nS1. Multivariate time series probabilistic forecasting is important to time-series analysis.\nS2. This work focuses on an important problem that could have real-world applications.\nS3. The tables and figures used in this work are clear and easy to read.\n\nWeakness.\n\nW1. Koopman theory requires that the measurement function maps inputs to several observable variables to construct a linear dynamical process in that space. Traditional methods typically choose fixed basis functions to meet this requirement, but this paper adopts a learnable measurement function. How can we ensure that the constructed space is within a linear dynamical system?\n\nW2. In the ablation experiments for the components of KoopmanNet in Table 3, why does using the local Koopman operator lead to numerical instability? \n\nW3. There is a lack of sensitivity analysis regarding the parameters, which obscures understanding the behavior of the proposed method in different hyperparameter settings.", "other_comments_or_suggestions": "Some citations should be updated to published versions instead of preprints.", "questions_for_authors": "Koopman theory requires that the measurement function maps inputs to several observable variables to construct a linear dynamical process in that space. Traditional methods typically choose fixed basis functions to meet this requirement, but this paper adopts a learnable measurement function. How can we ensure that the constructed space is within a linear dynamical system?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "0ZvISzgaVL", "reviewer": "Reviewer_Se2E", "summary": "This paper points out that traditional probabilistic methods in predicting the collapse of long-term series uncertainty estimate provide a new perspective. Then, to overcome these limitations, this paper introduces K2VAE, an efficient VAE-based generative model that leverages a KoopmanNet to transform nonlinear time series into a linear dynamical system, and devises a KalmanNet to refine predictions and model uncertainty in such a linear system, which reduces error accumulation in long-term forecasting.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**Reply to Q1: Why can the proposed method solve the problem of error accumulation well?**\n\nThank you for your valuable comments.  In $K^2$VAE, the KoopmanNet models time series in linear dynamical systems, where uncertainties are represented as deviations from the linear system’s predictions. Then KalmanNet works in a two-phase operation:\n\n1. **Prediction Phase**: Outputs state predictions and covariance matrices of uncertainty distributions for the linear system.\n   $$ \\hat{z} _{k} = A z _{k-1} + B u _k $$\n   $$\\hat{\\mathrm{P}} _k = A\\mathrm{P} _{k-1}A^T + Q $$\n\n1. **Update Phase**: Integrates observations to compute the Kalman gain $K_k$, refining predictions and covariance via:\n   $$K _k = \\hat{\\mathrm{P}} _k H^T(H\\hat{\\mathrm{P}} _k H^T + R)^{-1}$$\n   $$z _k =  \\hat{z} _{k} + K _k(\\hat{x} _k^H - H \\hat{z} _{k}) $$\n   $$\\mathrm{P} _k = (I - K _k H) \\hat{\\mathrm{P}} _k$$\n\nThe prediction and the update phase fuse the  information from Integrator ($u_k$) and KoopmanNet  ($\\hat{x}^H _k$) to effectively eliminate error accumulation in the prediction $\\hat{z} _k \\to z _k$ and the covariance matrix $\\hat{\\mathrm{P}} _k \\to \\mathrm{P} _k$. To demonstrate KalmanNet’s effectiveness, we include an ablation study of it in https://anonymous.4open.science/r/K2VAE-D957/ablations.md. The experiments demonstrate the importance of the KalmanNet in multiple PTSF tasks,  particularly in tasks of L=336 and 720. \n\n\n**Reply to Q2: According to my understanding, this paper constructs a probabilistic time series model based on Koopas. This should further clarify. Why is the experiment not compared with Koopas?**\n\nWe first explain the differences between $K^2$VAE and Koopa in the following  points:\n\n1. Koopa applies Koopman Theory to address the non-stationarity issue in point forecasting tasks.  $K^2$VAE utilizes the KoopmanNet to make It easier to model the uncertainties.  Koopa is a model completely based on the Koopman Theory and our proposed $K^2$VAE benefits from the Koopman Theory, Kalman Filter, and VAE.\n2. Koopa is a deterministic model while $K^2$VAE is a non-deterministic model. Koopa is a MLP based model while $K^2$VAE adopts the generative architecture VAE, and excels at modeling the distributions over the target horizon.\n3. Koopa assumes that the time series can be accurately modeled based on the pure Koopman Structures, so that it utilizes multi-scale MLP to  simulate the Koopman process, trying to model an unbiased linear dynamical system in the measurement space.  While $K^2$VAE is a model tailored for PTSF tasks, which starts from analyzing the ``bias'' in the linear dynamical system modeled by KoopmanNet, using a KalmanNet to model these uncertainties.\n4. A key design in Koopa's is that it utilizes multiple MLP layers to continuously models the residual parts. In our asssumptions, such residual parts may be uncertainties which are hard to model. Therefore, $K^2$VAE not only designs a KoopmanNet to linearize the time series, but also models the uncertainties through the subsequent KalmanNet.\n\nWe include Koopa in our baselines. Specifically,  we equip it with a Gaussian distribution, which is consistent with other point-forecasting baselines such as FITS, iTransformer and PatchTST.  The experimental results are shown in https://anonymous.4open.science/r/K2VAE-D957/comare_with_koopa.md . In both long-term and short-term prediction tasks, Koopa lags behind $K^2$VAE. \n\n\n\n**Reply to Q3: Is the proposed method for more accurate prediction of long time series or for effective uncertainty modeling over longer periods of time? if the former, you can add any probabilistic model with koopa.**\n\n$K^2$VAE focuses on  probabilistic forecasting, which is in line with ``effective uncertainty modeling over longer periods of time.''  Combining with the answers to Q1 and Q2, $K^2$VAE employs KalmanNet to achieve better uncertainty modeling. Based on the generative model VAE, $K^2$VAE accurately describes the variational distribution in the measurement space constructed by KoopmanNet, thus enhancing the ability to construct the target distribution of the forecasting horizon. On the other hand, Koopa is constructed for the deterministic long-term point forecasting tasks. Its characteristic lies in constructing a multi-scale MLP structure and modeling the time series from the perspective of decomposition. And $K^2$VAE does not directly apply the structure of Koopa. Instead, it has a greater inclination towards the application of the original Koopman Theory. For the part of uncertainties, $K^2$VAE  models it with the KalmanNet. We will discuss the differences from Koopa in the Related Works and will include Koopa as an additional baseline (as elaborated in Q2) in the paper.  \n\n**Reply to Other Problems**\n\n Figure 3 is redrawn in https://anonymous.4open.science/r/K2VAE-D957/README.md .**Thanks again for the valuable comments!  May I ask if our responses have resolved your questions? Or we can have further discussions!**", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "Yes", "theoretical_claims": "Yes", "experimental_designs_or_analyses": "Yes", "relation_to_broader_scientific_literature": "N/A", "essential_references_not_discussed": "The connection and difference between proposed method and [1] should be  further clarified.\n\n[1]Liu, Yong, Chenyu Li, Jianmin Wang, and Mingsheng Long. \"Koopa: Learning non-stationary time series dynamics with koopman predictors.\" Advances in neural information processing systems 36 (2023): 12271-12290.", "other_strengths_and_weaknesses": "Strongthness:\n\n1. The experimental results show the advantages of the proposed model in terms of inference time and computing resources.\n\n2. The experimental results show that the proposed method is effective.\n\n3. This paper points out that the traditional probability method in forecasting long time series uncertainty estimation collapse problem, provides a new point of view\n\n4. K2VAE outperforms state-of-the-art baselines, showing notable improvements in predictive performance.\n\n\n\nWeakness:\n\n1.", "other_comments_or_suggestions": "1.  Figure 3 is complicated and difficult to understand.", "questions_for_authors": "1. Why can the proposed method solve the problem of error accumulation well?\n\n2. According to my understanding, this paper constructs a probabilistic time series model based on Koopas [1]. This should further clarify. Why is the experiment not compared with Koopas? \n\n3. Is the proposed method for more accurate prediction of long time series or for effective uncertainty modeling over longer periods of time? if the former, you can add any probabilistic model with koopa. \n\n\n\n[1]Liu, Yong, Chenyu Li, Jianmin Wang, and Mingsheng Long. \"Koopa: Learning non-stationary time series dynamics with koopman predictors.\" Advances in neural information processing systems 36 (2023): 12271-12290.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "hJL7f9CaXy", "reviewer": "Reviewer_auu7", "summary": "This study presents $K^2VAE$, an efficient variational autoencoder (VAE)-based generative model. It utilizes a KoopmanNet to convert nonlinear time series into a linear dynamical system. Furthermore, it designs a KalmanNet to enhance predictions and model uncertainty in this linear system, thereby reducing error accumulation in long-term forecasting.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**Reply to W1. Why were CRPS and (NMAE) chosen as the main evaluation metrics? Why not use other evaluation criteria?**\n\nThank you for your valuable comments. In probabilistic forecasting, evaluation metrics including CRPS (Continuous Ranked Probability Score), CPRS_sum, NMAE (Normalized Mean Absolute Error), NMAE_sum, NRMSE (Normalized Root Mean Square Error), and NRMSE_sum are widely adopted as standard performance metrics [1] [2] [3]. From a comprehensive evaluation perspective, CRPS and its aggregated variant CRPS_sum quantify the distributional approximation quality, while NMAE/NMAE_sum or NRMSE/NRMSE_sum assess point estimation accuracy. Typically, one metric from each category is selected for evaluation. The ProbTS benchmark [3], recognized as a comprehensive framework for probabilistic forecasting models, employs CRPS and NMAE as primary evaluation criteria. This benchmark configuration, having undergone extensive empirical validation across numerous algorithms, was adopted in our study to ensure methodological consistency and equitable comparison with baseline models.\n\n[1] Kollovieh, Marcel, et al. \"Predict, refine, synthesize: Self-guiding diffusion models for probabilistic time series forecasting.\" *Advances in Neural Information Processing Systems* 36 (2023): 28341-28364.\n\n[2] Rasul, K., Seward, C., Schuster, I., and Vollgraf, R. Autoregressive denoising diffusion models for multivariateprobabilistic time series forecasting. In *ICML*, volume139, pp. 8857–8868, 2021a.\n\n[3] Zhang, Jiawen, et al. \"ProbTS: Benchmarking point and distributional forecasting across diverse prediction horizons.\" *Advances in Neural Information Processing Systems* 37 (2024): 48045-48082.\n\n**Reply to W2. The conclusion of the paper exceeds two lines and needs to be adjusted to meet the final format requirements of ICML.**\n\nThanks! We will modify this issue.\n\n**Reply to W3. For input token embeddings, the conventional patch partitioning method was not used, and no detailed explanation was provided, which is confusing.**\n\n1. **The conventional patching strategy**, as introduced in PatchTST [4], aims to uniformly partition multivariate time series across channels to facilitate subsequent channel-wise independent modeling. The dimensional transformation process operates as: $X \\in \\mathbb{R}^{B \\times N \\times T} \\to X^\\prime \\in \\mathbb{R}^{B \\times N \\times n \\times p} \\to X^{patch} \\in \\mathbb{R}^{(B \\times N) \\times n \\times p} \\to X^{embedding} \\in \\mathbb{R}^{(B \\times N) \\times n \\times d}$, where $n$ denotes the number of patches, $p$ denotes the patch size. This architecture collapses the batch dimension $B$ and variate dimension $N$ into a single axis, effectively enforcing univariate modeling at the patch level. \n2. In contrast, our adopted patching mechanism maps multivariate subsequences to high-dimensional representations compatible with Koopman operator theoretic frameworks, enabling systematic state transition modeling. The modified transformation pipeline is formalized as: $X \\in \\mathbb{R}^{B \\times N \\times T} \\to X^\\prime \\in \\mathbb{R}^{B \\times N \\times n \\times p} \\to X^{patch} \\in \\mathbb{R}^{B \\times n \\times (N \\times p)} \\to X^{embedding}\\in \\mathbb{R}^{B \\times n \\times d}$, where $n$ denotes the number of patches, $p$ denotes the patch size. Our architecture collapses the patch dimension $p$ and variate dimension $N$ into a \tsingle axis.\n\n[4] Nie, Yuqi, et al. \"A Time Series is Worth 64 Words: Long-term Forecasting with Transformers.\" *The Eleventh International Conference on Learning Representations*.\n\n\n**Replay to W4. Although the authors analyzed the overall performance, key components, and efficiency of K2VAE , there is a lack of analysis on parameter sensitivity. It is hoped that the authors can add an analysis of key parameters.**\n\nThanks for your valuable comments. Following your suggestions, we systematically evaluated three critical hyerparameters: Patch Size $p$,  and the dimensionality $d$ of hidden layers in the Measurement Function, and the number of hidden layers $l$ in the Measurement Function. Specific experimental results are presented in the following Tables :\n\nhttps://anonymous.4open.science/r/K2VAE-D957/sensitivity.md\n\nThese empirical findings substantiate that $K^2$VAE maintains robust performance across different hyperparameter configurations. To achieve best performance, we recommand a group of stable hyperparameters for both short-term probabilistic forecasting and long-term probabilistic forecasting:\n\n1. **Patch Size**:\n   - Short-horizon forecasting tasks achieve peak performance with patch_size $p$: 8\n   - Long-horizon forecasting benefits from extended context capture with patch_size $p$: 24\n2. **Network Architecture**:\n   - Number of hidden layers $l$: 2-3 layers yield optimal performance-efficiency balance\n   - Dimensionality of hidden layers $d$ : 256 provides sufficient representational capacity\n\n\n**Thanks again for the Reviewer auu7's valuable opinion!**", "claims_and_evidence": "The statement that \"The model exhibits robust generative capability and performs well in both short- and long-term probabilistic forecasting\" is strongly supported by experimental findings on real-world datasets.", "methods_and_evaluation_criteria": "There are detailed running examples and complexity comparison covering 9 baselines. 8 datasets. 2 metrics. The selection of baselines aligns with domain of time series forecasting. \n\nHowever, why were Continuous Ranked Probability Score (CRPS) and Normalized Mean Absolute Error (NMAE) chosen as the main evaluation metrics?", "theoretical_claims": "The equations are correct and align with the code submitted.", "experimental_designs_or_analyses": "The design of the experiments is clear and comprehensive, and support $K^2VAE$ claims. \n\nAlthough the authors analyzed the overall performance, key components, and efficiency of $K^2VAE$ , there is a lack of analysis on parameter sensitivity. It is hoped that the authors can add an analysis of key parameters.", "relation_to_broader_scientific_literature": "Transferring data from a nonlinear space to a linear space can resolve non-stationarity issues, thereby effectively alleviating the predominantly non-stationary nature of real-world datasets. This approach is highly recommended for broad application.", "essential_references_not_discussed": "I think there are no essential references that have not been discussed.", "other_strengths_and_weaknesses": "Strengths：\n\nS1. The paper is well-motivated, as the challenge of effectively modeling both short and long-term multivariate time series probabilistic forecasting.\n\nS2. This paper is well written. The notations are clear.\n\nS3. Experimental results demonstrate that $K^2VAE$  consistently outperforms state-of-the-art baselines across multiple real-world datasets.\n\nWeaknesses：\n\nW1. Why were CRPS and  (NMAE) chosen as the main evaluation metrics? Why not use other evaluation criteria?\n\nW2. The conclusion of the paper exceeds two lines and needs to be adjusted to meet the final format requirements of ICML.\n\nW3. For input token embeddings, the conventional patch partitioning method was not used, and no detailed explanation was provided, which is confusing.\n\nW4. Although the authors analyzed the overall performance, key components, and efficiency of $K^2VAE$ , there is a lack of analysis on parameter sensitivity. It is hoped that the authors can add an analysis of key parameters.", "other_comments_or_suggestions": "It appears that there is an inconsistency between Equation 12 and Figure 3. The more accurate representation should be using Equation 12 to express \\( Res \\). Please correct this, and unify the representations in these two places.", "questions_for_authors": "None", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: This paper presents $K^2$VAE, a novel variational autoencoder framework that combines Koopman operator theory and Kalman filtering for probabilistic time series forecasting. The proposed model effectively transforms nonlinear dynamics into a linear latent space via KoopmanNet and refines uncertainty through KalmanNet, addressing long-standing challenges in long-term forecasting such as error accumulation and inefficiency in generative models. The method is theoretically sound, and the implementation is carefully designed and well-motivated. Extensive experiments across diverse datasets show that $K^2$VAE consistently outperforms strong baselines, including recent diffusion-based and transformer-based approaches, while remaining lightweight and efficient. The authors provide thorough ablations and comparative studies, including with $D^3$VAE and Koopa, and respond convincingly to reviewer concerns. Overall, this is a strong, well-executed contribution to the field, with both theoretical depth and practical relevance. I recommend acceptance.
Decision: Accept (spotlight poster)