KooNPro: A Variance-Aware Koopman Probabilistic Model Enhanced by Neural Process for Time Series Forecasting
Authors: Ronghua Zheng
Hanru Bai
Weiyang Ding
Venue: ICLR 2025 Poster (2025)
Keywords: Probabilistic time series prediction; Neural Process; Deep Koopman model
Primary area: learning on time series and dynamical systems

Abstract: The probabilistic forecasting of time series is a well-recognized challenge, particularly in disentangling correlations among interacting time series and addressing the complexities of distribution modeling. By treating time series as temporal dynamics, we introduce **KooNPro**, a novel probabilistic time series forecasting model that combines variance-aware deep **Koo**pman model with **N**eural **Pro**cess. KooNPro introduces a variance-aware continuous spectrum using Gaussian distributions to capture complex temporal dynamics with improved stability. It further integrates the Neural Process to capture fine dynamics, enabling enhanced dynamics capture and prediction. Extensive experiments on nine real-world datasets demonstrate that KooNPro consistently outperforms state-of-the-art baselines. Ablation studies highlight the importance of the Neural Process component and explore the impact of key hyperparameters. Overall, KooNPro presents a promising novel approach for probabilistic time series forecasting.

Published as a conference paper at ICLR 2025
KOONPRO: A VARIANCE-AWARE KOOPMAN PROBABILISTIC MODEL ENHANCED BY NEURAL PROCESS FOR TIME SERIES FORECASTING
Ronghua Zheng, Hanru Bai & Weiyang Ding Institute of Science and Technology for Brain-inspired Intelligence
Fudan University, Shanghai 200433, China rhzheng23 @m. fudan. edu. en, hrbai23@m. fudan, edu, en, dingwy edu en
ABSTRACT
The probabilistic forecasting of time series is a well-recognized challenge, particu- larly in disentangling correlations among interacting time series and addressing the complexities of distribution modeling. By treating time series as temporal dynam- ics, we introduce KooNPro, a novel probabilistic time series forecasting model that combines variance-aware deep Koopman model with Neural Process. KooNPro introduces a variance-aware continuous spectrum using Gaussian distributions capture complex temporal dynamics with improved stability. It further integrates the Neural Process to capture fine dynamics, enabling enhanced dynamics capture and prediction. Extensive experiments on nine real-world datasets demonstrate that KooNPro consistently outperforms state-of-the-art baselines. Ablation stud- ies highlight the importance of the Neural Process component and explore the impact of key hyperparameters. Overall, KooNPro presents a promising novel ap- proach for probabilistic time series forecasting. Code is available at this repository: https://github.ccm htwpghhhwww.ooo///hhhheeeeee Sher-Zhhnn/KKooprrr
INTRODUCTION
Time series forecasting is essential in supply chain management, finance, energy, and healthcare. Probabilistic forecasting quantifies uncertainty, supporting robust decision-making in tasks like inventory optimization and risk management. For instance, supply chains, helps businesses anticipate demand fluctuations, optimize planning, and improve efficiency. Traditional methods like AegooReeressive Integrated Moving Average (ARIMA) (Said & Dickey] 1984) and the Kalman filter family (Auger et al. 2013) struggle with the complexities of real-world time series. Deep learning approaches for time series forecasting have evolved from Recurrent Neural Networks (RNNs) (Salehinejad et al.) 2017) to Long Short-Term Memory (LSTM) networks (Yu et al.. 2019). The introduction of the attention mechanism (Vaswani et al.)
2017) further advanced the field, leading
to Transformer-based models such as Zhou
(202
Wu al. (202T). and Zhou al. (2022)
More recently, diffusion probabilistic models (Ho et al. 2020; Rasul et al. 2021 Li et al. 2022
Fan al. (024) have emerged as promising paradigm for probabilistie forecasting, with works like Kollovich et al. (2024) demonstrating their potential. Moreover, foundation models for time series forecasting, such as Ansari et al. (2024), have introduced the concept of learning a universal representation for diverse time series tasks. Despite these advancements, many deep learning models focus primarily on point estimation, which limits their ability to fully capture the uncertainty and dynamics inherent in complex systems. State space models, such as Rangapuram et al. (2018a) 8a), [aagg et al. (2019) and Paria et al.] (202T) offer an alternative by modeling time series as low-dimensional latent decompositions. These methods excel representing temporal dynamics in reduced spaces but often rely on fixed assumptions about state transitions, which may fail to generalize to highly nonlinear and non-stationary scenarios.
Dynamic Mode Decomposition (DMD) powerful tool for modeling time series as dynamic systems (Kuttichira et al. 2017), aiming identify the underlying dynamics and subsequently
"ooresponding author

---

Published as a conference paper at ICLR 2025
utilize them for prediction (Kou & Zhang 2019 Yuan al. 2021). However, DMD's reliance discrete eigenvalues (point-spectra) restricts its ability to describe the nonlinear patterns of complex systems. The Koopman theory offers a powerful framework for analyzing nonlinear systems by representing them linear but infiniteddimensional space. The ability to decompose complex dynamies into Koopman eigenfunctions has inspired many works incorporate this theory into time series modeling. Combining the Koopman theory (Koopman] 1931) with deep learning (Lusch et al., 2018) enhances the ability model temporal dynamics with greater sophistication, leading more accurate predictions. For example, models such as Liu et al. (2023) and Wang et al. (2023) enhance predictive capabilities through deep Koopman theory by learning the continuous-spectr. of dynamics. However, these models are often affected by spectrum pollution, leading to unstable convergence and reduced accuracy, especially in highly nonlinear systems (Colbrook & Townsend] (024). To address this, the concept of pseudo-spectra has been introduced. Pseudo-spectra accounts for the latent spaces where cigenvalues may appear under different perturbation conditions, which makes them more robust to noise and better suited for capturing continuous variations.
Figure 1: To enhance the accuracy and robustness of probabilistic forecasting, model time series as dynamic evolutions in hidden space. In our setting, the dynamic system is constrained within unit circles. The point-spectra (purple), continuous-spectre (red), and pseudo-spectra (yellow) are represented within unit circles. Multiple unit circles can be deemed as different time points in the hidden space. Figure adapted from Elden Ring (@ FromSoftware Inc. and Bandai Namco Entertainment Inc.), modified and used under fair use for academic purposes.
The motivation of this work is to assemble various spectral components within unified framework, as illustrated in Fig. [ Point-spectra are incorporated via Neural Process (Garneoo et al.) (0188) to capture global temporal dynamics. Continuou-spectra are modeled using the Koopman theory to represent local temporal dynamics. For Pseudo-spectra, we propose variance-aware continuous- spectra modeled by Gaussian distributions, drawing on the idea that probability distributions can characterize pseudo-spectra within neighborhood by quantifying overall dispersion (Colbrook et al.) 2024).
By modeling time series dynamic systems grounded in multi-spectra representations, propose KooNPro, a novel probabilistic framework that combines global latent features from point-spectra via Neural Process with variance-aware modeling of continuous-spectra. We adopt variational inference and derive a tailored Evidence Lower Bound (ELBO) to optimize the unified model. KooNPro achieves high predictive accuracy and robustness in high-dimensional, non-stationary settings, demonstrating state-of-the fie-thh-rt art performance. Our contributions are summarized as follows:
We introduce KooNPro, a novel probabilistic prediction model that synergistically integrates the probabilistic Koopman model with Neural Process. Drawing inspiration from perturba-
2

---

Published as a conference paper at ICLR 2025
tions in pseudo-spectra, KooNPro incorporates variance-awan continuous-spectt to learn local temporal dynamics effectively. Additionally, utilizes Neural Process to capture the full point-spectra which represented the global temporal dynamics, facilitating improved predictive capabilities.
Extensive experiments conducted on diverse real-world datasets convincingly demonstrate the superiority of our proposed model. It consistently achieves hate-of-the-ee art performance, significantly outperforming existing methods across two metries.
We perform comprehensive ablation studies demonstrating the ability of Neural Process to capture temporal dynamics globally, thereby enhancing prediction performance. We also assess the impact of key hyperparameters detailed case study that visually illustrates the prediction performance of KooNPro, yielding intuitive and interpretable results.
model efficacy. Additionally, we provide
2 RELATED WORK
This work engages with three key areas: probabilistic time series prediction, the probabilistic Koopman model, and Neural Process. While each of these fields has received considerable attention, we limit our discussion to the most pertinent studies to ensure brevity. A more extensive version can be found in Appendix A
Probabilistic time series prediction: Recent developments probabilistic forecasting for time series integrate deep learning, statistical approaches, and diffusion models. |Ragaauuram al. (2018b), Salinas et al. (2020) and Li et al. (2021) combine state space model with deep learning. Feng et al. (2023) and Tang & Matteson (2021 I) developed attention-based mechanism that enhance long-range dependencies for improved forecast accuracy. Gaussian Process (GP), combined with temporal decomposition, was used by Yan al. (2021), Nguyen Quanz (202I). and (Salinas et al.) (2019) to better model uncertainty in multivariate settings. The data distribution-ue based generative model proposed by (Gouttes et al.] 202T) and the Kalman filter-based approach proposed by/de Bézenac et al.. (2020). blend probabilistic techniques for better scalability and robustness. Diffusion models, such as those proposed by Rasul et al. (2021). [i et al. (2022), and Fan et al. (2024), model forecasting as a denoising task, excelling high-dimensional settings.
Probabilistic Koopman model: The Koopman theory has been developed in lots of fields, the most related work is the probabilistic Koopman model, originally proposed by MMrton et al. (2019). Han et al. (022)) designed stochastic Koopman neural network for control, in which the latent observables are represented through Gaussian distribution. Colbrook al. (2024) integrates the concept of variance into the Koopman framework by introducing variance-aware pseudo-spectra, thereby ensuring convergence within the model. For time series tasks, Naiman al. (2023) utilized Koopman theory to represent the latent conditional prior dynamics via a linear map, while Mallen (tt al. 2024) introduced a framework enabling probabilistic forecasting for systems with periodically varying uncertainty.
2018a) as Conditional Neural Process, bridges neural networks' scalability and Gaussian Process's ability to model uncertainty. Garnelo al. (2018b) provide flexible, probabilistic function approximations, making them efficient and adaptable across tasks. Attentive Neural Process (Kim et al.) 2019). Convolutional Conditional Neural Process (Gordon et al.) (019b)). Gaussian Neural Process (Bruinsma et al., 202I) and Au- toregressive Neural Process (Bruinsma et al. 2023) deploy different methods to probe the relation between input-output pairs. Other works extend NP to meta-learning, such as Meta-Leam Learnnn Station- ary Stochastic Process (Foong al. 2020) and Meta-Learning Probabilistie Inference (Gordon et al.) 2019a). A comprehensive survey by Jha et al. (2022) has organized these developments, exploring their wide applications in uncertainty-aware learning.
Neural process: Neural Process (NP). first proposed by Garneoo
BACKGROUND
Due to space constraints, we discuss the background details in Appendix B and focus here on the core of Koopman theory and Neural Process, which form the foundation of KooNPro.
3

---

Published as a conference paper at ICLR 2025
The spectrum decomposition of the Koopman operator is crucial for understanding dynamies. The point-spectra reveal modal contributions via eigenvalues and corresponding cigenfunctions, which reveal how different modes contribute to the system's evolution. However, many complex systems exhibit continuous-spectra, reflecting chaotic or highly irregular behavior. In such cases, the Koopman operator's spectrum forms continuum rather than isolated eigenvalues and offers sophisticated framework for capturing both regular and chaotic components of system dynamics. Althrough the Koopman operator effective for predictive modeling but prone to spectrum pollution, which can hinder convergence and stability. Drawing inspiration from pseudo-spectra perturbations and the idea that probability distributions capture variable dispersion, we propose variance-aware continuous-spectrr modeled by Gaussian distributions.
Neural Process (NP) is a stochastic process p(f) that describes the predictive distribution over the target set (æD. == (m; Vi)teD given the context set (æc, yc) == (mx, Y: kec. Garnelo et al. (2018b) prompts use the distribution of high-dimensional random vector S to represent as
plyboo,, ac, UC) == f pLUD ED ()p(5 (on(Saccyyc)e yS.
NP consists of two components: an encoder that maps the input-output pairs of the context set (xc, 3cc) S for representing p(f), and a decoder that combines S with æp generate yp. Due the intractable log-likelihood, NP adopts amortized variational inference and maximizes the evidence lower bound (ELBO) of the log-likelihood as follows
log (yDl D. yo, xc) E5 B5-aas aD.DD
[logp (yplæd, S)] DE (S) yD) lpp (S oc. (e))y))):
(2)
4
METHOD
This section offers detailed overview of KooNPro, with the complete architecture illustrated in Fig KooNPro leverages the synergistic integration of Neural Process (NP) with probabilistic deep Koopman model to learn temporal dynamics for probabilistic future prediction. Initially, NP globally captures the cointssecctra of temporal dynamics to govern the full time series, indicated by the downward arrows in Fig Additionally, inspired by the concept of pseudo-spectra, we utilize probabilistic deep Koopman model refine continuous-spectra, yielding variance-awan spectra for a more nuanced representation of local temporal dynamics, as demonstrated by the shadowed box in Fig 2
4.1 CAPTURE TEMPORAL DYNAMICS BY NEURAL PROCESS
Dynamic Mode Decomposition (DMD) a classical method that elucidates the relationship between time series data and corresponding dynamics. However, DMD's reliance on point-spectra limits its capacity to capture the nonlinear patterns inherent in complex systems, thereby constraining its predictive power. To address these limitations, KooNPro integrates Neural Process (NP) with DMD to generate a stochastic process that captures the global dynamics of the whole time series more effectively. Specifically, KooNPro identifies the distribution of the latent variable E R which integrates the underlying dynamics of time series in Eq as shown in the left part of Fig
We utilize Takens' theory like Yuan et al. 2021 ) define a time series 21:7 E TTxxxx where T denotes the time length, d denotes the feature dimension and l denotes the delay-embedded length, and let x = -1.7-1 y = 22.7. To estimate the dynamics governing time series in the context set, KooNPro employs a neural network T to embed E ((f-1)xxx1 and f E (T (1)xx)) as the initial components of delay embedding like
TC = -(EC. 2bb (1
Σ vec (6 (t
1=1
where vp denotes a learnable Multi-Layer Perceptron (MLP), denotes the number of items in the context set, and the eigenvalues of क' can approximate the point-spectra of the corresponding time series dynamics. We model the distribution of S by factorized Gaussian distribution parametrized by TC, i.e.,
p(S]xc, ((())
4

---

Published as a conference paper at ICLR 2025
Context
Time segment
Time segment
Time segment
Time segment
T Target
Tar get,
cont -ar cargetz:e+
Contextxx47 -Tar get
Embedding t
h++
Decoder Φ
((2etTlRe47)
Ze
(
((STF))
Complex Multiplication
ht
Auxiliary
p(u.li,.S)
8
Encoder c
K
Varianco Continuous
T-1
Resample
Spectrum Distribution
Koopman Operator
Leamable Part
The complete architecture of KooNPro: p(S|TD) and ((SiTc) are the latent representa- tions of DMD mean over the target set and the context set respectively. Given the first time step in the target set, the encoder generates the latent variable h1- We apply the Koopman operator A to h1 for T - steps then generate the distribution of h2.r. Finally, the decoder Φ maps h2.r back to the original space to generate the distribution of 22.7.
Figure 2:
Denote p(S(rc) p(S xC, UC) for any set C, thus p(S) represents the distribution of underlying dynamics according to different time series sets. The ELBO in Eq[[car can be reformulated as follows
(5) The second term the right of Eq captures the difference in the distribution of S, reflecting the diversity of temporal dynamics across different time series. Although we employ т to embed the point-spectra that characterize the global temporal dynamics, it provides only coarse approximation. For the first term on the right side of 5 we maximize using probabilistic deep Koopman model, which provides more nuanced representation of the local temporal dynamics through the lens of the pseudo-spectra.
log(yD pDp yc; xc) 2 Eg Dogr(VD|>> m. S)] DxL (
4.2 PROBABILISTIC DEEP KOOPMAN MODEL
The proposed probabilistic deep Koopman model shown in the shadowed box of Fig concentrates on explaining local characteristics of time series with variance-aw continuous-spectra. This approach offers deeper understanding of temporal dynamics behavior over time, leading improved predictive capabilities. In essence, estimating the true temporal dynamics is equivalent to maximizing the expectation LE ()), (lorsr [logp(yp|æp. S.
For simplicity, we diseuss only the data in the target set and omit the index D. Following Lusch et al. (2018) we compose KooNPro by three MIPs: an encoder Φ Rdxk - RD and decoder Baxk identify an appropriate linear space, along with an auxiliary x: IR=+ _ T
to learn the continauus-spectra of the corresponding temporal dynamics in linear space. Firstly, KooNPro vectorized the hemay-embedded time series æ 11:T-1 from size (T' 1) x d x & to (T 1) (d k) 1, then applies the encoder as h1:7-1 (fxx). We hypothesize the latent space created by @ possesses linear characteristics, thus the dynamics in such space can be described as follows
h A.Al h,
(6)
We assume h, represents the eigenvalue of the dynamics, consisting of pair of conjugate numbers, and X, to be pure imaginary like Lange et al. (202I). let X, = jw, thus we have
Re(h++1 = Re(h:) @ (t)(w))t) Im o sin At), Im 1m(h+1)) Ro(h,) Atn( Im(he) @ cos (w) At);
(7)
(8)

---

Published as a conference paper at ICLR 2025
where @ is the element-wise product. We can simplify Eq and 8 to
h++1 A(hi,
In order to learn the Koopman operator A, which is conditioned on the global temporal dynamics representation S, KooNPro utilizes the auxiliary R to project the concatenation of h, and S to Ki,
h
(10)
Drawing on the theory of pseudo-spectra, KooNPro aims learn dynamic information not only from the eigenvalues but also from their neighborhoods. To facilitate this, KooNPro employs the auxiliary network to generate the corresponding Gaussian distribution of wi parameterized by Ni as
p(w,|,, S) NW(w; (u(Nee, ((x))
(11)
To simplify KooNPro, we assume the independence of h1:7-1 on the linear space. Consequently, we descrihe the uncertain dynamics
T-1
p( 1ITT- S) = II ((s)
(12)
Since h, = A(h 11 it yields that cantinuous-spectrr in the linear space can be characterized within the framework of the Gaussian distribution like
h1:T-11 S) Thus, the variance-aware
(haTThuTT-1: p(lor WIT hIT -ht.--hhhhh---SSS----l-ll.. S/pWW.---lbir--1...)
(13)
T-1
II ((s))
(14)
Utilizing the decoder KooNPro maps the temporal dynamic evolution from the linear space back to the original space, as follows
p(22r1 h2:1)
(15)
tm2
The likelihood plyl ær 5) in Eq
where x = 21.T-1 x S) ( S) = p(22. ( h2.7 h 11,
and y = Z2.r, can derived as
(16)
given that the encoder & is deterministic function. Insert Eq 14 and Eq the expectation as
15 to Eq 16
we can draw
Es Es-a.srrp [ogg(yp|eD, = Bs-agsiip) Log(p(z,)h))
(hg S))] (17)
The first term can be interpreted as the prediction loss, while the second term represents the linear loss, as described in Lusch et al. (2018). By maximizing the ELBO in Eq we derive that Sc approximates the dynamics for each time series present during the training phase. Consequently, our predictions in the test phase commence with the initial ø1 E IR flxxxk like
= - (0(=))
(18)
It is crucial to note that the time length T during the training stage is distinct from the prediction horizon T,, with the input history length solely determined by the delay-embedded length k. This enables train KooNPro once and then forecast future values of arbitrary length.
EXPERIMENTS
In this section, we conduct comprehensive experiments on nine real-world datasets to evaluate the performance of KooNPro against state-of-the-ar baselines. We demonstrate Neural Process's 's ability to enhance KooNPro's predictive performance and investigate the impact of key hyperparameters through ablation studies. Finally, we visualize the prediction results for the Solar dataset and analyze the relationship between prediction emrr and variance.

---

Published as a conference paper at ICLR 2025
5.1 SETTINGS
Datasets, We consider nine real-world datasets characterized by a range of temporal dynamics, namely ETTs, Solar, Electricity, Traffic, Taxi, and cuD- The data is recorded intervals of 15 minutes, 30 minutes, 1 hour, or day frequencies. Refer to Appendix C for details. All datasets are split chronologically and adopt the same train/validationn/ess ratios, i.e., 7:1:2.
Evaluation Metrics. Following previous work (Fan et al. 2024), we assess our model and all baselines using CRPS (Continuous Ranked Probability Score), widely used metrie for proba- bilistic time series forecasting, as well as NRMSE (Normalized Root Mean Squared Error). The details of metrics are shown in Appendix D
Baselines. We assess the predictive performance of KooNPro in comparison with multivariate time
Transformer-Min MAF (Rasul et series forecasting models, including GP-Copula (Salinas al. 2019),
2021 Grae (Rasul et al. 2021). TACTiS (Ashok et al.
2023), D (Li et al. 2023). DPK
(Mallen et al. 2024), and MG- NGGTTSDIA (Fan et al. 2024). The more details of baselines can be found in
Appendix B
Implementation details. The training process early stopped within epochs using the Adam optimizer with fixed learning rate of 10-5, The mini-bateh size is set to 128. Additional hyper- parameters, such as time length T. delay-embeddedd length k. and layers of MLPs are detailed in All models are trained and tested on single NVIDIA RTX4070Ti 12GB GPU.
Appendix F
5.2 RESULTS
In this section, we compare the performance of KooNPro with baselines using CRPS and NRMSEaum. More comprehensive results, including long-term predictions (Appendix GG) per- formance in different noisy scenarios (Appendix H). and the relationship between performance and memory consumption (Appendix [}. can be found in the appendix.
Table 1:
Comparison of CRPS (denoted as C-s, smaller is better) and NRMSE (denoted as N-s, smaller is better) across nine real-world datasets. The means and standard errors are based on 10 independent runs of retraining and evaluation. The best performances are highlighted in red and the second are in blue.
Model
Music
BTTHI
ETT12
BTTal
ETTx2
Soli
Elctricit
Trflic
Taxi
Oup
0.587
0.264 o2a
0.241 043
0.147 010 0.80610.024 0.078 oas 0.199 ...000
0.286 $o.0ooooo 000 0.217 ooe
GP-Cipula
Na
0.836 oag 0.368 oa2
0.424:0.008
0.1724 o 0a7 0.671.0.024 0.122 oss 0 0.208g..00m 01s 0. 0.412go.116 0.346 010
Cs
0.800
0, 0,223go0
0.370
0.22810 040 o. 0.16410.013
0,144
O 0.477g0 0.403
0,207 20
Tras MAF
1.288.0.188
0. 0.91540.024 0.67740.082
0.306 0.00 oon 1.00540.022 6.240.0.007 o 0.001 0. 0.308 0.308 020020
+0.00
Cs
0.547+0.022
0.24140.000
0.28740.029 0.21240.016
0.60440.011
0.00440.010 o 0.455+0.020 0 3..8700005
0.27140.087
Tiegsi
0.889:0.030
0.32540..00
0.3 0.29140.012 0010 1.01100.015 0.07240.014
0.560
0.498 0.34140.007
0.60140.004 004
0.208 002
0.68440.009 009
0.142 0.11 01n
1.87120.022
0.26440.013 012 o, 0.45020.000 000 0.981
0.270
TACTIS
No
0.907+0.010
0.920
1011840.010
0.320
ood 20809 0.391
1.871
1.17040.013
0.403
0.446 0.26610.010
2211200000
0.177 20.000000 017 0.912g0.0a5
0.198 020 0.265g0.027 oar 0.257
0.248
Na
o. 0.002:0.020 0.4
0.25740.ons
0,263
O. 0.642.0.070 0.268
o. 0.926go.105 0.391 go.0ss 0.5014a.000
Cs
0.71840.011 0.47140.024
0.656+0.018
0.341
0.708
0.784 nos 0.827+0.007 0.84340.000 0.728+0.020
BPK
1.064000122
0.725
0.687+0.023
0.391+0.157
1.15040.040 1.002000000 oos 1. 1.0000000r 10.00 1.147+0.050
Co
0.43040.038 0.1744g.000
0.25440.064
0.129 000 0.29840.025
0.107 o.. o 0.82840.057 0. 0.250
0.32840.015
MG-1SD
0.60320.083
0.220g0.017
0.594g0.076 0.29210 2 oen 0.028gn.026 0.156
o63 0.710go.0se 20.07 0.83810.000
0.32840.037 0.149
0.10640.007
0.081 o. 02n 0.231 6.067+0.000 nos o. 0.18440.022 0 40.00 0.20440.017
KooPbo
No
0.520g0.045 0.224400 0.226g0.02s 0.122 ose 0.918go.os4 0.096 o. 0.280 0.830g0.075 0,308 ea ose
Tab presents the CRPS and NRMSE values, averaged over 10 independent runs. In comparison, traditional baseline models (c.g., GP-Copula and Timegrid) have higher CRPS values most datasets, which indicates their diffieulty in handling complex time series data.
7

---

Published as a conference paper at ICLR 2025
Compared to modern deep generative models (e.g., VAE and TACTIS), KooNPro achieves better balance between predictive stability and accuracy. Although MG-TSD shows a slight advantage in NRMSE certain datasets (e.g., ETTs), its CRPS fluctuates significantly, suggesting an inadequate characterization of distributional uncertainty. KooNPro introduces the variance- aware spectrum to enhance global pattern capturing, avoiding spectrum instability caused by local perturbations, thus achieving higher stability and accuracy in handling complex nonlinear patterns. This demonstrates its effectiveness in high-dimensional time series forecasting.
5.3 ABLATION STUDY
In this section, conduct ablation study to identify the factors contributing to KooNPro's 's success in prediction tasks. We first examine the role the Neural Process (NP), which primarily learns and represents the underlying dynamics across whole time series within S. We track the KL divergence between Sp and Sc during the training phase (Fig. 3 upper) on the ETTh1 dataset. It decreases over time, converging with the validation set around the 70th epoch, indicating effective learning of time series dynamics. To assess generalization, we compute the KL divergence between Se and 150 test time series (Fig. 3 lower). The low divergence suggests Sc reliably captures temporal dynamics, even for previously unseen data.
KL- divieergence
Valicote sct
1019
100
Train epochs
Count
20
KL-divergence
x10
Figure 3: The upper section illustrates the KI KL-divergence between Sp and Sc during the training phase, with training epochs on the x-axis and an exponential scale on the y-axis. The lower section analyzes 150 test dataset time segments, displaying their KL-divergence with Sc, with the x-axis using an exponential scale and the y-axis showing time series counts.
Table Prediction performance when replacing Neural Process (NP) with Attention Neural Process (ANP), Gaussian Process (GP), or removing NP version (denoted as with-ANP, with-GP, and without- NP, respectively).
Menic
ETTal
ETT22
Cap
C+
0.328 0.149
g0.077
0.081 20.020
0,211 to
0.087-0.000
0.154
0, 0.226 0.204g0.017
0.52020.045 0.22440.06s
22522.022
0.513 0.122200.04
D4s 0.095_0.012
20.05 0.330g0.078 0.308300.000
0.88420.081
0:194go.06s oes
0.100g0.020
0.107:00.019
0:201 tolllooo.o 062 0. 2.067_0.022
0.197 +0.016 0.213 0.806:0.000
Na
0.84140.054 0.300
0.35040.001
0.182240.035
0.577
0.127-00088
0.308
0.314440.00 0.499740.008
0.91240.007
0.61340.088 o5e 47M44.022 0.407 0.30440.002 0.125-0.095
+0.025 0.349
0.419.0. 160
wib-Op
8a
1.2 1.21020.051
0.791
0.861 ozn 0 0.02220.075 0.487
0 0.205_0.000 0.388 10.8810.016 o. 0.501
0.658
Ca
0.30020.093 0.218840 120 0.332 2o.....00 oow 0.156 10.561
0.343 0.093_0.027 0.313
0.262g0 ooe
0.674g0.224
withouNN
N+
o. 10120.117 0.318 1ea 0.51 10010 o p.saggo.0s 0.533 zo oes o 1000000006 o 0.580.0.000
o. 0.38020.002 092 1000820.311
To evaluate the contribution of NP further, we compare different methods for generating Sc, including cases without it. For clarity, label the methods follows: With-ANP (using Attention Neural
8

---

Published as a conference paper at ICLR 2025
Time longth
12 14 16 18
20
22
24
26
20 34
35
40 42
45
50
CRP5-eam abouttre lingh
2.5
CRPS-sum sbo.tetddinng
VANIS--uum abod-beediing langt
2.
NRMS MM-E-uuu -c-m
CRPS
1.
Embodding longth
Figure 4: The top axis shows the increase time length T, while the bottom axis indicates the increase in delay-embedded length k. The orange and transparent orange curves represent the CRPS and NRMSE for increasing T. The blue and transparent blue curves correspond to these metrics for increasing k. Error bars reflect the standard error from 10 independent retraining and evaluation runs.
Process), With-GP (using GP), and Without-NP (no NP). Implementation details are provided in Appendix As shown in Tab the overall performance of With-ANP is comparable to KooNPro, with predictive performance improving as data dimensionality increases. Notably, on the taxi dataset, the highest-dimensione
dataset, With-ANP outperforms KooNPro on both metrics. However, ANP incurs higher computational complexity, increasing from O(n+m) for NP to O(n(n+m)) (Kim) et al. 2019). With-GP demonstrates significantly degraded performance, even worse than Without- NP. This phenomenon may be attributed to the critical influence of prior knowledge in defining the kernel function. Furthermore, the non-learnable parameters may impede Sp from adequately capturing the temporal dynamics governing the entire time series.
Next, we investigate the impact of the hyperparameter time length T and delay gelay-embedded length on the prediction performance. In Fig 4 we analyze the impact of hyperparameters on the ETTh1 dataset. The orange and transparent orange curves illustrate changes in CRPS and NRMSE as increases. We observe that KooNPro's prediction accuracy improves significantly with 7 until T' = 30, beyond which further increases yield no additional benefits despite higher computational costs. A similar trend is seen when varying k, as shown by the blue and transparent blue curves. This suggests that temporal dynamics in time series cannot be fully captured simply by extending the learning time segment. Additional ablation studies across various datasets are presented in Appendix
K
5.4
CASE STUDY
The predictive capability of KooNPro arises from its ability capture the underlying dynamics of time series data effectively, we illustrate this using the Solar dataset, which comprises hourly measurements from 137 solar plants located in Alabama state. The dataset exhibits clear diurnal pattern: values are non-zero from 6:00 to 18:00 and zero at night. In probabilistic prediction, large errors often correspond high variance, indicating uncertainty. We assess this by analyzing the correlation between prediction variance and accuracy, measured by the mean absolute error (MAE) of the predicted mean relative to the true value. A scatter plot of MAE versus prediction variance (5ig(5) left) shows a correlation of 0.94 with significance level near zero, confirming that KooNPro effectively captures temporal dynamics and ensures reliable predictions. The right panel visualizes the ground truth and predictions for the first eight plants over 24 hours. The results demonstrate that KooNPro accurately captures the temporal fluctuations in Solar energy generation: The prediction mean approaches zero during the transition from nighttime to daylight and vice versa for each plant. Furthermore, the prediction intervals expand during sunlight and night periods, aligning with the peaks and troughs of Sol fluctuations, indicating the reliability of the predictions.

---

Published as a conference paper at ICLR 2025
Commicion: 1.55, Sigiiteant lave: 6.76-13
Dimension
Dimension
Dimension
Dimension 4
USC
20.
Dimension 5
Dimensioné
Dimension
Dimension 8
^
Vian Absoluto Error
Observations
Prediction mean
50% pro diction interval
90% prediction interva
Figure 5: The left panel illustrates the correlation between the MAE the prediction means relative to the true values and the prediction variance. The title indicates that the correlation achieves 0.95 and the significant level is 6.74 10-13, The right panel visualizes the changes in the first eight dimensions of the Solar dataset over 24 hours.
CONCLUSION
This paper presents KooNPro, a novel probabilistic forecasting model that treats time series as temporal dynamics. KooNPro integrates Neural Process for global temporal dynamics modeling and variance-aware continuous-spectra inspired by pseudo-spectra: for local temporal dynamics, enhancing learning capabilities. Extensive experiments on nine real-world datasets demonstrate KooNPro's superior performance compared state-of-the-art methods. Comprehensive ablation studies explore the origins of KooNPro's predictive power, while visualizations of Solar dataset predictions showcase its accuracy and reliability.
ACKNOWLEDGEMENTS
This work was partially supported by the National Natural Science Foundation of China (No. 12471481, U24A2001), the Science and Technology Commission of Shanghai Municipality (No. 23ZR 1403000, and the Open Foundation of Key Laboratory Advanced Manufacturing for Optical Systems, CAS (No. KLMSKF202403).
10

---

Published as a conference paper at ICLR 2025
REFERENCES
Ian Abraham, Gerardo De La Torre, and Todd D Murphey. Model-based control using koopman operators. arXiv preprint arXiv:1709.00 01568, 2017.
Alexander Alexandrov, Konstantinos Benidis, Michael Bohlke-Schneider, Valentin Flunkert, Jan Gasthaus, Tim Januschowski, Danielle C Maddix, Syama Rangapuram, David Salinas, Jasper Schulz, et al. Gluonts: Probabilistic and neural time series modeling in python. Journal of Machine Learning Research, 21 21(116):1-6, 2020.
Abdul Fatir Ansari, Lorenzo Stella, Caner Turkmen, Xiyuan Zhang, Pedro Mercado, Huibin Shen, Oleksandr Shchur, Syama Sundar Rangapuram, Sebastian Pineda Arango, Shubham Kapoor, et al. Chronos: Learning the language of time series. arXiv preprint axXiv2403.078/55, 2024.
Arjun Ashok, Étienne Marcotte, Valentina Zantedeschi, Nicolas Chapados, and Alexandre Drouin. Tactis-2: Better, faster, simpler attentional copulas for multivariate time series. arXiv preprini arXiv:2310.01327, 2023.
François Auger, Mickael Hilairet, Josep M Guerrero, Eric Monmasson, Teresa Orlowska- Kowalska, and Seiichir Katsura. Industrial applications of the kalman filter: review. IEEE Transactions on Industrial Electronics, 60(12):5458-54) 5471, 2013.
Omri Azencot, Wotao Yin, and Andrea Bertozzi. Consistent dynamic mode decomposition. SIAM Journal on Applied Dynamical Systems, 18(3):1565-1855 2019.
Omri Azencot, N Benjamin Erichson, Vanessa Lin, and Michael Mahoney. Forecasting sequential data using consistent koopman autoencoders. In International Conference Machine Learning, pp. 475-485, PMLR, 2020.
Wessel P. Bruinsma, James Requeima, Andrew Y. K. Foong, Jonathan Gordon, and Richard E. Turner. The Gaussian Neural Process, January 2021.
Wessel P Bruinsma, Stratis Markou, James Requiema, Andrew YK Foong, Tom Andersson, Anna Vaughan, Anthony Buonomo, J Scott Hosking, and Richard E Turner. Autoregressive conditional neural processes. arXiv preprint arXiv:2303. /4468, 2023.
Steven L. Brunton, Bingni W. Brunton, Joshua L. Proctor, and J. Nathan Kutz. Koopman Invariant Subspaces and Finite Linear Representations of Nonlinear Dynamical Systems for Control. PLOS ONE, 11(2)=0110111, February 2016. ISSN 1932-6203. doi: 1 500.00
015011..
Steven L Brunton, Marko Budišić, Eurika Kaiser, and Nathan Kutz. Modern koopman theory for dynamical systems. arXiv preprint 1xX. 2021.
Matthew J Colbrook and Alex Townsend. Rigorous data- driven computation of spectral properties of koopman operators for dynamical systems. Communications on Pure and Applied Mathematics, (7(1))221-298) 2024.
Matthew Colbrook, Qin Li, Ryan Raut, and Alex Townsend. Beyond expectations: residual dynamic mode decomposition and variance for stochastic dynamical systems. Nonlinear Dynamics, 112(3):2037-206 2024.
Emmanuel de Bézenac, Syama Sundar Rangapuram, Konstantinos Benidis, Michael Bohlke- Schneider, Richard Kurle, Lorenzo Stella, Hilaf Hasson, Patrick Gallinari, and Tim Januschowski. Normalizing Kalman Filters for Multivariate Time Series Analysis. In Advances in Neural Infor- mation Processing Systems, volume 33, pp. 2995-30900 Curran Associates, Inc., 2020.
Akshunna S Dogra and William Redman. Optimizing neural networks via koopman operator theory. Advances in Neural Information Processing Systems, 33:2087-2099 2020.
Xinyao Fan, Yueying Wu, Chang Xu, Yuhao Huang, Weiqing Liu, and Jiang Bian. MG-TSD: Granulrra Time Series Diffusion Models with Guided Learning Process, March 2024.
Shibo Feng, Chunyan Miao, Ke Xu, Jiaxiang Wu, Pengcheng Wu, Yang Zhang, and Peilin Zhao. Multi-scale Attention Flow for Probabilistic Time Series Forecasting, July 2023.

---

Published as a conference paper at ICLR 2025
Andrew Foong, Wessel Bruinsma, Jonathan Gordon, Yann Dubois, James Requeima, and Richard Tumne. Meta-learning stationary stochastic process prediction with convolutional neural processes. Advances in Neural Information Processing Systems, 33:8284-822) 8295, 2020.
Marta Garnelo, Dan Rosenbaum, Christopher Maddison, Tiago Ramalho, David Saxton, Murray Shanahan, Yee Whye Teh, Danilo Rezende, and S. M. Ali Eslami. Conditional Neural Processes. In Proceedings of the 35th International Conference Machine Learning, pp. 1704-1113 PMLR,
July 2018a.
Marta Garnelo, Jonathan Schwarz, Dan Rosenbaum, Fabio Viola, Danilo J. Rezende, S. M. Ali Eslami, and Yee Whye Teh. Neural Processes, July 2018b.
Jonathan Gordon, John Bronskill, Matthias Bauer, Sebastian Nowozin, and Richard E. Turner. Meta-Learning Probabilistic Inference For Prediction, August 2019a.
Jonathan Gordon, Wessel P Bruinsma, Andrew YK Foong, James Requeima, Yann Dubois, and Richard E Turner. Convolutional conditional neural processes. arXiv preprint arXiv: 13556, 2019b.
Adèle Gouttes, Kashif Rasul, Mateusz Koren, Johannes Stephan, and Tofigh Naghibi. Probabilistic Time Series Forecasting with Implicit Quantile Networks, July 2021.
Zongyu Guo, Cuiling Lan, Zhizheng Zhang, Yan Lu, and Zhibo Chen. Versatile neural processes for learning implicit neural representations. arXiv preprint arXiv:2301.08883, 2023.
Minghao Han, Jacob Euler-Rolle, and Robert K Katzschmann. DESKO: STABILITY-ASSUMT ROBUST CONTROL OF NONLINEAR SYSTEMS WITH A DEEP STOCHASTIC KOOPMAN OPERATOR. 2022.
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33:6840-6851, 2020.
Saurav Jha, Dong Gong, Xuesong Wang, Richard E Turner, and Lina Yao. The neural process family:
Survey, applications and perspectives. arXiv preprint arXiv:22090005/7 2022.
Eurika Kaiser, J Nathan Kutz, and Steven Brunton. Data-driven discovery of koopman eigenfunc- tions for control. Machine Learning: Science and Technology, 2(3)30350 2021.
Makoto Kawano, Wataru Kumagai, Akiyoshi Sannai, Yusuke Iwasawa, and Yutaka Matsuo. Group equivariant conditional neural processes. arXiv preprint arXiv:2102.08777 2021.
Hyunjik Kim, Andriy Mnih, Jonathan Schwarz, Marta Garnelo, Ali Eslami, Dan Rosenbaum, Oriol Vinyals, and Yee Whye Teh. Attentive Neural Processes, July 2019.
Diederik P. Kingma and Max Welling, Auto-encoding variational bayes. In Yoshua Bengio and Yann LeCun (eds.), 2nd International Conference on Learning Representations, ICLR 2014, Banff. AB, Canada, April 14-16, 2014, Conference Track Proceedings, 2014. URL http:
f/arxiv.org/ waap///orr/aa///11....111 6114
Marcel Kollovich, Abdul Fatir Ansari, Michael Sohlke-Schheiiee Jasper Zschiegner, Hao Wang. and Yuyang Bernie Wang. Predict, refine, synthesize: Self-guiding diffusion models for probabilistic time series forecasting. Advances in Neural Information Processing Systems, 36, 2024.
Bernard O Koopman. Hamiltonian systems and transformation in hilbert space. Proceedings of the National Academy of Sciences, 1755):31-3318, 1931.
Milan Korda and Igor Mezić. Linear predictors for nonlinear dynamical systems: Koopman operator meets model predictive control. Automatica, 93:149- 160, 2018.
Jiaqing Kou and Weiwei Zhang. Dynamic mode decomposition with exogenous input for data-driven modeling of unsteady flows. Physics fluids, 31()), 2019.
Deepthi Praveenlal Kuttichira, EA Gopalakrishnan, Vijay Krishna Menon, and KP Soman. Stock price prediction using dynamic mode decomposition. In 2017 International Conference on Advances in Computing, Communications and Informatics (ICACCI), pp. 55-60. IEFF, 2017.
12

---

Published as a conference paper at ICLR 2025
Henning Lange, Steven L Brunton, and J Nathan Kutz. From fourier to koopman: Spectral methods for long-term time series prediction. Journal of Machine Learning Research, 22(41):1-38, 2021.
Hyungi Lee, Eunggu Yun, Giung Nam, Edwin Fong, and Juho Lee. Martingale posterior neural processes, arXiv preprint arXiv:2304. 09431, 2023.
Longyuan Li. Junchi Yan, Xiaokang Yang, and Yaohui Jin. Learning Interpretable Deep State Space Model for Probabilistic Time Series Forecasting, January 2021.
Yan Li, Xinjiang Lu, Yaqing Wang, and Dejing Dou. Generative time series forecasting with diffusion, denoise, and disentanglement. Advances Neural Information Processing Systems, 35: 23009-23022 2022.
Yanhong Li. Jack Xu, and David C. Anastasiu. An Extreme-Adaptive Time Series Prediction Model Based on Wrobability-Fnhanne LSTM Neural Networks. Proceedings of the AAAI Conference on Artificial Intelligence, 37(7):684-8691, 8691, June 2023. ISSN 2374-3468, 2159-5399, doi: 10. 10.1160/aaai.v333 wttp///ai...3....... 2.005..
Yong Liu, Chenyu Li, Jianmin Wang, and Mingsheng Long. Koopa: Learning Non-stationary Time Series Dynamics with Koopman Predictors.
Yong Liu, Chenyu Li, Jianmin Wang, and Mingsheng Long. Koopa: Learning con-stationary time series dynamics with koopman predictors, Advances in neural information processing systems, 36: 12271-12290, 2023.
Bethany Lusch, Nathan Kutz, and Steven L. Brunton. Deep learning for universal linear embeddings of nonlinear dynamics. Nature Communications, 9(1):4950, November 2018. ISSN 2041-1723.
doi: 10. 10.1008//41467-018807700000
Alex T Mallen, Henning Lange, and J Nathan Kutz. Deep probabilistic koopman: long-term time-series forecasting under periodic uncertainties. International Journal of Forecasting, 40(3):
859-800 009-866,222 2024.
Iva Manojlović, Maria Fonoberova, Ryan Mohr, Aleksandr Andrejčuk, Zlatko Drmač, Yannis Kevrekidis, and Igor Mezić, Applications of koopman mode analysis to neural networks. arXiv preprint arXiv:2006. 11765, 2020.
Peiman Mohseni and Nick Duffield. Spectral convolutional conditional neural processes. arXiv preprint arXiv:2404. 13182, 2024.
Jeremy Morton, Freddie D Witherden, and Mykel Kochenderfer. Deep variational koopman models: Inferring koopman observations for uncertainty-awar dynamics modeling and control. arXiv preprint 1rXiv:1902.09742, 2019.
Ilan Naiman and Omri Azencot. A koopman approach to understanding sequence neural models. arXiv preprint arXiv:2 1rXiv:2102.07824 2021.
Iln Naiman, Benjamin Erichson, Pu Ren, Michael W Mahoney, and Omri Azencot. Gen- erative modeling of regular and irregular time series data via koopman vaes. arXiv preprint arXiv:2310.026/99 2023.
Abhinav Narasingam, Sang Hwan Son, and Joseph Sang-ll Kwon. Data-driven feedback stabilisation of nonlinear systems: Koopman-based model predictive control. International Journal of Control, 96(3):77--78, 2023.
Nam Nguyen and Brian Quanz. Temporal Latent Auto Iuto-Encoder: A Method for Probabilistic Multi- variate Time Series Forecasting. Proceedings of the AAAI Conference on Artificial Intelligence, 35 (10):9117-1125, May 2021. ISSN 2374-3468, 1399. doi: 10. wwt.///aa...35ii.aaa. 17100..
Tung Nguyen and Aditya Grover. Transformer neural processes: Uncertainty-wwar meta learning via sequence modeling. arXiv preprint arXiv:2207.04177 0417,, 2022.
Bernt Oksendal. Stochastic differential equations: an introduction with applications. Springer Science & Business Media, 2013.
13

---

Published as a conference paper at ICLR 2025
Biswajit Paria, Rajat Sen, Amr Ahmed, and Abhimanyu Das. Hierarchically regularized deep forecasting. arXiv preprint arXiv:2106.076. 2021.
Syama Sundar Rangapuram, Matthias W Seeger, Jan Gasthaus, Lorenzo Stella, Yuyang Wang, and Tim Januschowski. Deep state space models for time series forecasting. Advances in neural information processing systems, 31, 2018a.
Syama Sundar Rangapuram, Matthias W Seeger, Jan Gasthaus, Lorenzo Stella, Yuyang Wang, and Tim Januschowski. Deep State Space Models for Time Series Forecasting. In Advances in Neural Information Processing Systems, volume 31. Curran Associates, Inc., 2018b.
Kashif Rasul, Calvin Seward, Ingmar Schuster, and Roland Vollgraf. Autoregressive Denoising Diffusion Models for Multivariate Probabilistic Time Series Forecasting. In Proceedings of the 38th International Conference on Machine Learning, pp. 8857-8868. PMIR, July 2021.
William T Redman, Maria Fonoberova, Ryan Mohr, Ioannis G Kevrekidis, and Igor Mezic. An
operator theoretic view on pruning deep neural networks. arXiv preprint arXiv:2110. 14856, 2021.
Said E Said and David A Dickey, Testing for unit roots in autoregressivv- average models of unknown order. Biometrika, 71(3):599-607, 1984.
Hojjat Salehinejad, Sharan Sankar, Joseph Barfett, Errol Colak, and Shahrokh Valaee. Recent advances in recurrent neural networks. arXiv preprint arXiv: 1801.01078, 2017.
David Salinas, Michael Sealke-Schneider, Laurent Callot, Roberto Medico, and Jan Gasthaus. High- dimensional multivariate forecasting with low-rank Gaussian Copula Processes. In Advances in Neural Information Processing Systems, volume 32. Curran Associates, Inc., 2019.
David Salinas, Valentin Flunkert, Jan Gasthaus, and Tim Januschowski. DeepAR: Probabilistic forecasting with autoregressive recurrent networks. International Journal of Forecasting, 36(3): 1181-1191, July 2020. ISSN 0169-2070. doi: 10. (0066/.jforrcaas.201..00
Lifeng Shen and James Kwok. Non-autoregressive Conditional Diffusion Models for Time Series Prediction. In Proceedings of the 40th International Conference on Machine Learning, pp. 31016- 31029, PMLR, July 2023.
Naoya Takeishi, Yoshinobu Kawahara, and Takehisa Yairi. Learning koopman invariant subspaces for dynamic mode decomposition. Advances in neural information processing systems, 30, 2017.
Binh Tang and David S Matteson. Probabilistic Transformer For Time Series Analysis. In Advances in Neural Information Processing Systems, volume 34, pp. 23592 236608 Curran Associates, Inc.,
2021.
Kshitij Tayal, Arvind Renganathan, Rahul Ghosh, Xiaowei Jia, and Vipin Kumar. Koopman invertible autoencoder: Leveraging forward and backward dynamics for temporal modeling. In 2023 IEEE International Conference on Data Mining (ICDM), pp. 588-597 IEEE, 2023.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones. Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention all you need. Advances in neural information processing systems, 30, 2017.
Rui Wang, Yihe Dong, Sercan Ö Arik, and Rose Yu. Koopman Neural Forecaster for Time Series with Temporal Distribution Shifts, February 2023.
Yuyang Wang, Alex Smola, Danielle Maddix, Jan Gasthaus, Dean Foster, and Tim Januschowski. Deep factors for forecasting, In International conference on machine learning, pp. 6607-6617,
PMLR, 2019.
Haomin Wen, Youfang Lin, Yutong Xia, Huaiyu Wan, Qingsong Wen, Roger Zimmermann, and Yux- uan Liang. DiffSTG: Probabilistic Opatio-Tempone Graph Forecasting with Denoising Diffusion Models, March 2024.
Haixu Wu, Jiehui Xu, Jianmin Wang, and Mingsheng Long. Autoformer: Decomposition Transform- ers with Auto-Correlation for Long- Term Series Forecasting. In Advances in Neural Information Processing Systems, volume 34, pp. 22419-22430. Curran Associates, Inc., 2021.
14

---

Published as a conference paper at ICLR 2025
Tijin Yan, Hongwei Zhang, Tong Zhou, Yufeng Zhan, and Yuanqing Xia. ScoreGrad: Multivariate Probabilistic Time Series Forecasting with Continuous Energy-based Generative Models, June 2021.
Yong Yu, Xiaosheng Si, Changhua Hu, and Jianxun Zhang. A review of recurrent neural networks: Lstm cells and network architectures. Neural computation, 31(7):1235-1270, 2019.
Yuan Yuan, Kaiwen Zhou, Wenwu Zhou, Xin Wen, and Yingzheng Liu. Flow prediction using dynamic mode decomposition with time-delay embedding based local measurement. Physics of Fluids, 33(9). 2021.
Haoyi Zhou, Shanghang Zhang, Jieqi Peng, Shuai Zhang, Jianxin Li, Hui Xiong, and Wancai Zhang. Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting. Proceedings of the AAAI Conference Artificial Intelligence, 35(12)11100-11115,, May 2021.
ISSN 2374-3468, 2159-5399. doi: 10. 2..1609aaiivv35522.177.3. v35.52.
Tian Zhou, Ziqing Ma, Qingsong Wen, Xue Wang, Liang Sun, and Rong Jin. FEDformer: Frequency Enhanced Decomposed Transformer for Long-term Series Forecasting. In Proceedings of the 39th
International Conference on Machine Learning, pp. 27268 272866. PMLR, June 2022.
15

---

Published as a conference paper at ICLR 2025
A APPENDIX: RELATED WORK
This work engages with three key areas: probabilistic time series prediction, the probabilistic Koopman model, and Neural Process. While each of these fields has received considerable attention in recent literature. Here we talk only about the most related works:
Recent developments in probabilistic forecasting for time series integrate deep learning, statistical upproaches, and diffusion models. State space models have been significantly enhanced by deep learning, as seen in (Rangapuram et al. (20T8b)). (Salinas et al. (2020)) and (Li et al. (202T)), where they effectively capture temporal dynamies and handle missing data. (Feng et al. (2023) and (Tang & Matteson| (202I) developed attention-bascl mechanism that enhance long-range dependencies for improved forecast accuracy. Gaussian Process (GP), combined with temporal decomposition, were used by (Yan et al. (2021) and (Nguyen & Quanz (202I)) to better model uncertainty in multivariate settings. (Salinas et al. (219)) proposed a Gaussian copula process, and (Ashok al. (2023) combines Gaussian copula and the attention mechanism for capturing dependencies in high-dimensional multivariate forecasting. The data distribution-bast generative model proposed by (Gouttes et al. (202T)) and the Kalman filter-based approach of (de) Bézenac et al. ), blend probabilistic technique for better scalability and robustness. Diffusion models, such as those proposed by (Rasul al. (202T), eLi al. (2022). and (Fan et al. (2024)) settings. For faster training and prediction, (Shen Kwok (2023) introduced con-autoregressive diffusion model. (Wen et al.) (2024)) introduced spatio-temporal diffusion model, extending these methods to spatial dependencies.
Probabilistic time series prediction:
model forecasting as denoising task, excelling in high-dimensionn
Probabilistic Koopman model: Over the past two decades, Koopman techniques have garnered substantial attention, with applications spanning analysis (Brunton et al. (2016). (Takeishi et al.
(0177). ) (Lusch et al. (2018), (Azencot et al. (2019)); control Abbrahmm et al. (2017), Korda Mezić
(2018) Kaiser et al. 2021p. (Narasingam et al. (2023)); optimization (Dogra & Redman (2020))
(Manojlović et al. (2020D) (Naiman & Azencol (202T)) (Redman et al. (2021)) and forecasting
Azencot et al. (2020), Lange al. (2021), (Wang et al. (2023)) (Liu al.), (Tayal et al. (2023)
(Brunton et al. (202T)) comprehensively discusses these advances and highlights future research
directions. The most closely related work to ours is the probabilistic Koopman model, originally proposed by (Morton et al. (2019). (Han et al. (2022) designed a stochastic Koopman neural network for control, in which the latent observables are represented through Gaussian distribution. (Colbrook et (024) integrates the concept of variance into the Koopman framework by introducing pecitee-pseuoosspppe
thereby ensuring convergence within the model. For time series tasks, (Naiman et al.] (2023) utilized Koopman theory to represent the latent conditional prior dynamics via a linear map, while (Mallen et al. (2024)) introduced framework enabling probabilistic forecasting for systems with periodically varying uncertainty.
Neural process: Neural Process (NP), first proposed by (Garnelo et al. (2018a)) as Conditional Neural Process, bridges neural networks' scalability and Gaussian Process's 's ability to model uncer- tainty. Garnelo et al./(22 (2018b)) provide flexible, probabilistic function approximation, making them efficient and adaptable across tasks, but struggled with long-range dependencies and flexible function distributions. Various models have since emerged to address these limitations. Attentive Neural Process (Kim et al. (2019) introduced the attention mechanism to handle long-range dependencies better. Convolutional Conditional Neural Process (Gordon et al. (2019b) adapted convolutional layers to process images and time series data more effectively. Gaussian Neural Process (Bruinsma et al. (20TT) combined NP with Gaussian inference, improving performance in regression tasks by leverag- ing Gaussian uncertainty. (Lee et al./ (2023)) refined uncertainty quantification of NP, providing better posterior estimation. Other works extend NP to meta- meta-learning, such as Meta-Learning Stationary Stochastic Process (Foong et al. (2020)) and Meta-Learning Probabilistic Inference (Gordon et al.) (2019a)), both addressing generalization to unseen tasks. Recent works focus on expanding NP applications to complex data structures. Group Equivariant Conditional Neural Process (Kawano al. (2021)) and Versatile Neural Process (Guo et al. (2023)) improved the handling of symmetry and representation, enhancing NP's capacity to model equivariant and implicit functions. To improve generation ability further, (Nguyen & Grover (2022) and (Mohseni & Duffield (2024) combine NP with Transformer and Neural Operator respectively. A comprehensive survey by (Jha et al. (2022) has organized these developments, exploring their wide applications in uncertainty- learning.
16

---

Published as a conference paper at ICLR 2025
B BACKGROUND
B.I KOOPMAN THEORY AND SPEUTO-SPECTRA
The spectrum decomposition of the Koopman operator is a powerful tool for understanding the underlying dynamics. If the operator has a discrete spectrum, can be expressed in terms of its eigenvalues Xj and eigenfunctions éj. where K0₂ = ンyoj. This decomposition reveals how different modes contribute to the system's 's evolution. However, many complex systems exhibit continuous- spectra, indicating chaotic or highly irregular behavior. spectrum is not composed of isolated cigenvalues but rather continuum, requiring a more nuanced analysis. The operator can be represented through a spectrum measure and an integral over the continoous-ppectr:: Kg(x)) Jo e (F(()) where E(w) is projection-valee vallued measure. This spectrum approach provides a comprehensive framework for capturing both regular and chaotic components of a system's s dynamics. While the Koopman operator effectively models dynamics for predictive purposes, is prone to spectrum pollution, which can impede convergence and stability. Inspired by perturbations in pseudo-spectra and the notion that probability distributions characterize variables through overall dispersion, we propose variance-aware continuous- using Gaussian distribution. This method ensures that eigenvalue perturbations are influenced not only by local disturbances but also by the Gaussian distribution over the dynamics.
such cases, the Koopman operator's
and model
B.2 NEURAL PROCESS
A stochastic process can be viewed as random function F A -> where inputs can be regarded as indexing the output random variables. With relaxed use notation, we employ p(f) in denoting stochastic process, where map inputs E X' to E y. When fulfilling exchangeability and consistency as stated by Kolmogorov' 's extension theorem (Oksendal 2013). Neural Process (NP) cited by Garnelo et al. (2018)) is stochastic process that describes the predictive distribution the target set (æ yn = (2 Yi ten given the context set (æc. yc) == (æ;; yi) Ji)e Garnelo et al. (2018b) prompts use the distribution of high-dimensional random vector S to represent p(J) as
plyplep, Ic, BC) = pLuD zp.
(19)
NP consists of two components: an encoder that maps the input-output pairs the context set (xc, BC) S for representing p(f), and decoder that combines S with æp to generate yo. Due to the intractable log-likelihood, NPs adopt amortized variational inference as Kingma & Welling (2014) and maximize the evidence lower bound (ELBO) of the log-liieldoood follows
log (yplan l UC, (c)) 2 Es =p.dd [logr (ynlap, S)] DEL (1 (S) yn) l|l (S (S]ac (<)y))))
(20)
C BENCHMARK DATASETS
For our experiments, we use ETTs, Sclar, Electricity, Traffic, Taxi, and open-source datasets, with their properties listed in Tab[C The dataset can be obtained through the links below.
(i) ETTs: https//ghhha..com///ooooy///laas
(ii) Solar:
(iii) Dat.ch/cw....i/iicoooooeaueeeeee
(iv) Traffic: httpss:/peemssddotr..ggg
(v) Taxi: http://wwwwwygywoouuudddduutttlr
(vi) KDD-cup: http:///wwwww....///////////dd/d
17

---

Published as a conference paper at ICLR 2025
Table 3: Datasets detail
Name
Frequency
Dimensions
Input length
Prediction length
ETTh1 1 hour
10
24
ETTh2
1 hour
10
24
ETTml
15 min
10
24
ETTm2
min
10
24
Solar 1 hour
137
15
24
Electricity
hour
370
10
24
Traffic 1 hour
862
10
24
Taxi 30 min
1214
15
24
D APPENDIX: EVALUATION METRIC
We consider metries: CRPS and NRMSE the first one describe the predictive distribution, and the second can describe the distance between truth value and prediction mean, more details can be found in Gluonts documentation (Alexandrov al. (2020)).
CRPS CRPS is a univariate, strictly proper scoring rule that quantifies the compatibility of cumulative distribution function F with an observed value x: E IR as:
CRPS (F(y) 1((x S yy)
(21)
where 1((x < y) denotes the indicator function. The CRPS achieves the minimum value when predictive prediction F' same as the data distribution. CRPS can be extend to CPRS.um to evaluate multivariate distribution:
CRPS R2, (CPRS(FFF..) F Σ at))
(22)
where Fm11 is computed by aggregating samples across dimensions and subsequently sorting them obtain quantiles. A smaller CRPS indicates more accurate predictions.
NRMSEsum: NRMSEsum is an adaptation of the Root Mean Squared Error (RMSE) that accounts for the scale of the target values. It is defined follows:
me 1UY--)) Y)²) mcan([Y])
NRMSE
(23)
where Y represents the predicted time series, and Y represents the true target time series. NRMSE quantifies the average squared difference between predictions and true values across all dimensions, normalized by the mean absolute magnitude of the target values. A smaller NRMSE indicates more accurate predictions.
E DETAILS ON TRAINING BASELINES
We train baselines by open code which is reported in corresponding papers, and follow the default setting. The code for the baseline methods is obtained from the following sources.
TACTIS: hutps://gtthbb.com///rrrccoo//cs
D http://githubbccmm//mmmeee/////s
18

---

Published as a conference paper at ICLR 2025
DPK: http:///ttttcoooo////e//llllleoe
MG-TSD: http:///gtttbb.comm//dddddldl/d0
G.GP-oopula (2019): method combining Gaussian Process with Copulas model complex dependencies between variables in multivariate time series, providing uncertainty estimates and capturing nonlinear correlations.
2. Transformer-MAY (2021) Combines Transformers with Masked Autoregressive Flow (MAF) to model long-ferm dependencies and the conditional probability distribution of time series data effectively.
TimeGrad (2021) diffusion model-based approach for time series forecasting, which progres- sively generates samples to capture complex dynamics and uncertainty in the data.
TACTiS (2023) probabilistic autoregressive model leveraging Transformers to handle non- stationary time series, focusing on dynamic structures and probabilistic predictions.
D3VAE (2023) A deep variational autoencoder (VAE))based model designed for time series, featuring dynamic decoder to effectively capture and predict complex temporal structures.
DPK (2024) Dynamic Probabilistic Kernel (DPK) models probabilistic dependencies in time series using dynamic kernel-based approach, balancing flexibility and efficiency for multivariate data.
MG-TSD (2024) Munta-Granularity Time Series Decomposition (MG-TSD) decomposes time series into components of varying frequencies or trends, modeling each with probabilistic framework to capture multi-scale patterns.
F APPENDIX: IMPLEMENTATION DETAILS
In Tab we show the hyperparameters of KooNPro include time length T' delay-embedded length k layers the encoder Φ layers the decoder & --1, layers of the auxiliary k, layers of shown in
Eq Note that the choice of T and l is based on the ablation study showcased in Appendix K
Table 4: Hyperparameters of KooNPro
Name
Φ-1
K
xbb
ETTh1
30
10
4
4
4
2
ETTh2
20
10
4
4
4
2
ETTml
20
10
4
4
4
2
ETTml
20
10
4
2
Solar
15
8
3
Electricity
30
4
2
Traffic
20
10
4
4
4
2
Taxi
30
15
6
6
3
KDD-cup
10
10
4
4
4
2
G APPENDIX: LONG-TERM PREDICTION
To assess KooNPro's predictive capability in capturing hidden temporal dynamics within time series, we evaluate its performance under extended prediction length, as detailed in Tab According to the results in Ta KooNPro exhibits the least degradation across both metrics compared baseline methods. This outcome highlights the effectiveness of KooNPro learning and leveraging the temporal dynamics of time series for accurate prediction.
19

---

Published as a conference paper at ICLR 2025
Table Datasets detail
Name
Frequency
Dimensions
Context length
Prediction length
ETTh1
hour
7
10
48
ETTh2
hour
7
10
48
ETTml
15 min
7
10
48
ETTm2
15 min
7
10
48
Solar hour
137
15
48
Electricity
hour
370
10
48
Traffic
hour
862
10
48
Taxi
30 min
1214
15
48
KDD-cup
hour
270
10
72
Table Comparison CRPS (denoted as C-s, smaller is better) and NRMSE (denoted as N-s, smaller is better) across nine real-world datasets. The means and standard errors based on 10 independent runs of retraining and evaluation. The best performances are highlighted in red and the second are in blue. A block marked with "_" denotes numerical issue encountered during model training with longer prediction lengths.
Mided
Menc
BTTHI
RTTex
Selbr
Toffic
Thcl
Og
20.110.000 031 0 0. one 0.646.0. one 0.008 o 054 0000 0.284 047 0.52920.000 one
1.007.0.025
0.781
GP-Copola
0.909 033 0.810 000
0.78220
1as 0.84420 1 69a 0.700gn.oes
0.382 ro3 0.718go.011 1.404.0.057
1.07810.000
Co
1.271 0.507200004
1015520.085 0.27920 007
D. 0.188 o 3.592.0000
D. 0.769
0.410g0.073
Thas MAF
No
1.571+0.144
0. 0.7
1.79740.122
0.48540.127
0.897 10.127 o. 0.872g0.04s 0.036 20.000
0.518
0.79640.084
0.477+0.007
0.45840.069 0.346 0.010
0.006
0.283 n2s 0.728+0.05m
0.791-00022 0.42140.080
Tinggit
No.
0.953.0.102 0.69720.008 0.08820. 104 0.455.0 o1d 1.24320.050 0.423
0.902g0.0s5
1971.0.118 0.5222..000 098
Cs
0.76220.004 004 0.401g0:00 001
10131
0.261 02n 3. 1. 0.360 0o4 o 0.600gg..ooo 067
1.398 20.014
0,390
0.94840.004
0.822#0.002
1858-+0.000 0.422 o 023 5.815.2.108 0.498+0.002 002 o. 0.18100.028 +0.00
0.505+0.022
Co
0.916 0.828g0.oc4
0.59840.021 0.737 1. 0s1 0.726 0.408
o. 0.000
0.814.00.035
No
1.206.0.078 0.861g0:041
0.70840.007 0.961 031 0.919 0.001
1.120g0.139
1.808
+0.077
0.744
0.82440.040 0.510 a. 092 0o.on 6.907+0.012
1.181+0.002 0.969 +0.00
0.900+9.015
1.20220.032 032
138 134820.070 0.50240 o 202 1.301 20.004 1.96340.013
1.47820.000 1.213 1.30910.021
Co 0.619
o. 0.
0.371 40.140008
083
0.209 10000
0.174 0.617
0.409
0.58010.001
MG-TSD
No
0.967+0.071
0.627 0.688 109 0.318 050 1.630 0.28840.nas
0. 0.882+0.053 0.621
+0.0ss
0.767
0.48820.025
0.378gp.025 0.386.0.01s
1. 002 0.417
20.021
0.186.00001 0.401gn.011 0.396 20.02
0.327 20.011
KooPoo
0.71640.044 141120.005 0.08140.023 0.343 034
20.00 0.28740.013 017 o. 0.715go.00
0.65840.042 0.45740.021
20

---

Published as a conference paper at ICLR 2025
H APPENDIX: ROBUSTNESS OF PERFORMANCE
To assess the robustness of KooNPro's predictive performance, we evaluate it under varying Signal- to-Noise Ratio (SNR) conditions (20dB, 40dB, and 60dB). During training, KooNPro is trained with ground truth data. the testing phase, Gaussian noise is added the imput data, and predictions are compared to the ground truth. As shown in Tab(7] the performance degradation with decreasing SNR remains within acceptable limits, demonstrating KooNPro's robust predictive capability across different noise levels.
Table 7: KooNPro represents robust performance across various noisy-add scenarios, with SNR ranging from 20dB to 60dB. Note that KooNPro is trained once noise-free data and then tested on different noisy-add data.
Midel
Max
BTTM
ETT1Z
BTTal
BTTx2
Solar
Eles
Tuffle
Taxi
Cuy
0.326 +0.037 0.0051 0.165
+0.020
0.201 0.057+0.000 0.184=0.022 0.226 +0,044 0.204
0.520 045 0.228g0.0m 0.0
0.928-20.025
0.19220.031 0.31320.004
0-088100.122
288220022
20.0020.078 0.308 o5o
Co 0.307
023
0.19$
0.18310.001
0.112.0
0.22810.002
0.074go 0os
0.19220.0z
0:208
0.217
MAII
No
0.38640.023 0.28440.032
0
0.58040.021
0.192+0.015
0.92840.042
0.11840.014
0.353=0.022
0.337 +0.02
0.339+0.106
0.386
022
0.221
0.271
0.16220.001
0.248.0.000
0.19210.002
0.213_0.019
0.204.2.015
0.229
AMB
No.
0.608 049
2.007
0.060100045
0.162 0og
20.3020.035
0.283
061
0.346g0.00
0.380
0.352 081
Co 0.43240.021
O 0.311 0.20140.010
0. 0.10440.017
D. 40.04
0.287 O 268
0.296 40.000 O 0.249 + 020
3MB
0.08720.031 0.981g0.028 0.49120.026
0.245g0.018 0.38520.056 0.41820.049
0.30020.021 0.884.0.012 0.305
I APPENDIX: THE PARAMETERS OF LEARNED MODELS
The backhone of KooNPro is based on MIPs, with the number of parameters determined by the input data dimensions, as well as the depth and width of the MLP. Tab. [ compares the memory consumption of KooNPro to baseline methods. Furthermore, illustttes the relationship between memory consumption and predictive performance for each model on the ETTm2 dataset. The results indicate that KooNPro achieves the best performance with the fewest memory consumption.
Table Comparison of memory consumption for different models' parameters across nine datasets (in MB).
Model
ETTh!
ETTh2
ETTml
ETTm2
Solar
Elec.
Traffic
Taxi
Cup
Trans-MAF
9.69
9.69
9.69
9.69
12.21
16.73
26.26
33.08
14.79
Timegrid
4.22
4.22
4.22
4.22
25.93
29.32
37.83
45.06
27.82
TACTIS
7.48
7.48
7.48
7.48
7.49
7.49
7.50
7.51
7.49
DOVAE
58.41
58.41
58.41
58.41
58.72
59.29
60.50
61.36
59.05
DPK
0.17
0.17
0.17
0.17
0.25
0.40
0.70
0.92
0.34
MG-TSD
2.41
2.41
2.41
2.41
3.05
6.33
14.60
21.65
5.87
Noo
1.73
1.73
1.73
1.73
10.23
7.18
14.42
17.50
5.60
21

---

Published as a conference paper at ICLR 2025
0.8
0.7
0.6
CRPS-sum
0.5
Trans-MAF
Timegrid
0.4
TACTIS
DÂVAE
DPK
0.3
MG-TSD
ro
0.2
20
30
40
60
Memory consumption (in MB)
Figure 6: The plot illustrates the relationship between memory consumption and prediction perfor- mance of each model on ETTm2. The x-axis indicates memory consumption (in MB), and the y-axis represents CRPS Optimal performance is achieved as the marker approaches the bottom-left corner.
IMPLEMENTATION OF NEURAL PROCESS THROUGH DIFFERENT WAYS
This section details the implementation of various methods for generating Sp. represented the point- spectra globally and learned by Neural Process. We introduce three KooNPro variants: Without-NP, With-ANP, and With-GP.
Without-NP closely follows the KooNPro procedure. To isolate the impact of Sp. the training process adheres to Sec. 4.2 omitting S generated in S.c The test procedure is outlined as follows
2q - (+(21))).
(24)
With-ANP, following the Attention Neural Process (ANP) (Km et al. 2019) and integrating it with our work, employs the attention mechanism to generate Sp. which governs the temporal dynamics of the entire time series. In this process, we the Key as æc, the Value V as Sc. and the Query Q as pp then compute Sp.
With-GP employs the Gaussian process to predict Sp the target set æ using context set se and their corresponding outputs Sc by modeling a joint Gaussian distribution. The procedure can summarized follows
Kc = k(zc, zc) + o
(25)
Kp k(zD, =p);
(26)
KCD = k(zc; zD);
(27)
where k we choose the radial basis function kernel. Consequently, the distribution of Sp ~ N(u, Σ) can be calculated by
# = Ko Sc.
(28)
Σ = KD KonK KCD,
(29)
where z is mean and E is covariance.
22

---

Published as a conference paper at ICLR 2025
K APPENDIX: ABLATION STUDY
We conduct ablation studies on the time length and delay-embedded length across several datasets. The results indicate that merely extending the learning time segment is insufficient to reveal the temporal dynamics underlying the time series, as discussed in Sec 5.3 The choices for I and presented in Tab are guided by the results of this ablation study.
Time length
0.9
09
MRMST -um abeddiing
0.7
0.7
NRMSE sum
0.4
104
003
0.1
0.1
9
10
16
70
Embedding length
Figure Ablation study on ETTh2 dataset.
Time length
1.4
eboutimee
CAPS-sUm
NRMSE sum
0.6
0.4
10.4
8
10
13
34
35
18
27
18
19
Embedding length
Figure 8: Ablation study on ETTml dataset.
23

---

Published as a conference paper at ICLR 2025
Time length
26
CRPS sun boouttime ength
X aboutim
0.8
08
0.7
07
NRISE sum
CRPS-SUm
0.5
0.5
0.4
0.4
0.2
02
10
15
16
17
18
19
Embedding length
Figure 9: Ablation study on ETTm2 dataset.
Time length
26
1.0
MRMSE CRPS-ssm wloust abeedinn
MRMSE cum cbedsing longth
1.2
NR MSE sum
0.0
0.8
9.6
9
10
15
16
Embedding length
Figure 10: Ablation study on Solar datasct.
Time length 20 22 24 26 30
48
CRPS-uum Wbout time engh
MEMST tim leegh
CRPS sun cbosdin lorgth
NRMSE cun sbediig leagth
0.8
NRMSE sum
CAPS-sum
0.6
0.6
0
Embedding Inngth
Figure 11: Ablation study Electricity dataset.
24

---

Published as a conference paper at ICLR 2025
Time length
24
26
44
1.2
MRMSS
CRPS-am aloust wlsedding
MKMM5--uum stadddig langt)
NRMSE sum
CRPS-SUm
0.4
0.4
0.2
16
18
Embedding length
Figure 12: Ablation study on Taxi dataset.
Time length 26
12
22
24
28
30
44
46
48
50
4.5
MRMSE
45
CRPS-sum about ebesting
NKMSE dbutd lange)
3.5
35
NRMSE sum
1.5
0.5
0.5
9
15
16
Embedding length
Figure 13: Ablation study on KDD- dataset.
25


Reviews:
[{"review_id": "M172snzEgE", "reviewer": "Reviewer_ahsz", "summary": "- KooNPro is a novel time series forecasting method based on Koopman theory.\n- It effectively estimates the process dynamics in a latent space where state transitions are linear.\n- It eventually provides context-dependent variance estimates.", "questions": "Note: The most important questions are listed first.\n\n1. Regarding Sec. 5.2: The presented results are very impressive. However, a large chunk of the time series forecasting literature performs point estimation instead of full distribution modeling and, therefore, commonly evaluates using MAE/MSE (see, for instance, the overview [here](https://github.com/thuml/Time-Series-Library)). Additionally providing these metrics (possibly in the appendix) would help contextualize the findings in the broader body of work. These methods can also provide probabilistic forecasts, e.g., by learning an output that parameterizes a simple distribution to be sampled from or by methods such as Monte Carlo dropout.\n2. Does the method need to be run multiple times to obtain the variance estimation samples?\n3. The case study in 5.4 is interesting, yet it makes me question the ability of KooNPro to model the variance of the data truthfully. While I agree that many dimensions are modeled appropriately, the results indicate serious limitations of the method.\n\t1. Dimensions #3 and #6 show values far outside the 90% interval. Are they within something like the 99% interval since the model learns a possibly appropriate long tail of the distribution? Or does it show some of the method's limitations (which would be fine if acknowledged as such)?\n\t2. Ll. 446f explains the data characteristics of the Solar dataset. However, the model fails to appropriately model the very low nightly variance of the power production around zero and instead shows a significant variance estimate at, for instance, midnight. Why does that occur?\n\t3. Combining these two observations, the overall variance estimate appears rather *uniform* and not well-adapted to the dataset.\n4. How large are the learned models measured in the number of parameters?\n5. Could the method be straightforwardly used to forecast fractional steps into the future?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "- Time series tasks, especially probabilistic forecasting, are relevant in many domains and far from being solved.\n- The method is presented well and provides many benefits (variance estimation, flexible forecast horizon, nice theory, and practical performance).\n- I am not aware of these ideas being presented before.", "weaknesses": "1. See Questions below.\n2. The definition of target and context sets (ll. 153ff) and the splitting of $z$ (ll. 208ff) were not familiar to me, as they are not common in point forecasting. I would have personally benefitted from more exposition.\n\n#### Minor Comments\n- References should not be used as nouns (e.g., ll. 037f, 049f).\n- Notation: Eq. (2)/(5) is missing a $p$ on the left-hand side and has odd typesetting of the expectation subscript. Some parens/brackets should be adjusted in height, e.g., in Eq. (3) and (17). In l. 219, $x_C$ and $y_C$ are not bold. L. 473 is the $e$  referring to \"scientific notation\" common in programming languages (in base-10) or Euler's number?\n- One could rename Sec. 5 to Experiment**s**.\n- Re. l. 306: Are the conditioning lookbacks of the test and the forecast horizon of validation (etc.) overlapping or not?\n- Language: The senctence in l. 821f is odd.\n- The combination in Fig. 3 is clever. However, given two separate plots would easily fit side-by-side, it is not worth the extra time needed to decipher the legend/axis labels.\n- The abstract in OpenReview should be formatted with Markdown syntax, instead of LaTeX.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "bnVRvrWXRk", "reviewer": "Reviewer_VVQE", "summary": "A method for multi-variate probabilistic time series forecasting is presented which combines the methods for temporal dynamics modeling using deep Koopman model with Neural Processes. The model consists of an encoder computing the hidden state h_t, a learned Koopman operator 'A' predicting the future hidden states and a decoder used to predict the final values using the hidden state h_t. In addition a neural process is used to model the latent dynamics S of the time series, which is used as an input to the Koopman operator to compute the future states.", "questions": "- What is the utility of the neural process latent variable S at a high level? Does it help model the correlations between the different time series? Can it be thought of as an encoding of each time series which also model the inter-correlations between the time series?\n- It seems that S is an s-dimensional latent variable meaning that it encodes the whole time series in a low dimensional subspace. Many papers have modeled time series as state space models involving low-dimensional latent decomposition. What is the advantage of the proposed approach over these approaches.\nExample papers modeling time series into low dimensional sub-spaces:\n  - Wang, Yuyang, et al. \"Deep factors for forecasting.\" International conference on machine learning. PMLR, 2019.\n  - Rangapuram, Syama Sundar, et al. \"Deep state space models for time series forecasting.\" Advances in neural information processing systems 31 (2018).\n  - Paria, Biswajit, et al. \"Hierarchically regularized deep forecasting.\" arXiv preprint arXiv:2106.07630 (2021).\n- What is the utility of the delay embedding dimension k?\n- Do context and target sets refer to training and testing time periods? can the context and target sets be interspersed, meaning can this method be used to predict missing values of irregularly sampled time series?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "Strengths\n- This paper proposes a new way for probabilistic modeling of multi-variate time series by representing the time series into a low dimensional space using neural processes and using them to model the temporal dynamics of the time series.\n- The idea presented in the paper is an intuitive approach for multi-variate forecasting. Modeling the latent dynamics of the time series is an important aspect that is missed by most state-of-the-art modeling approaches. As such, the method presented in this paper is interesting to the time series community.\n- The paper presents extensive experimental evaluation and ablation studies demonstrating the effectiveness of the approach.", "weaknesses": "Weaknesses\n- The description of the model is quite cryptic and not written in a clear manner. See questions below for parts that are unclear.\n- While the paper describes everything using mathematical equations, the utility or advantage of such equations/models is unclear. How do each of the components of the model help with a better forecast? (i am looking for a high level motivation for each of the components of the model)\n- Several works have introduced Koopman theory into time series modeling, however, there is no clear discussion of existing methods and their comparison with this work. The novelty of the paper is unclear without a clear distinction with existing work.\n- Other probabilistic forecasting methods modeling latent factors in time series have not been discussed of compared (for example deep state space models).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "DxN73DS4yB", "reviewer": "Reviewer_xASk", "summary": "The authors study the practical problem of probabilistic time series forecasting and address the challenges. They propose KooNPro, which combines two methods, i.e., the Koopman model and Neural Processes.", "questions": "1. How can the method generalize past Gaussian distributions to modeling arbitrary distributions?\n2. Have the authors tested with the Attentive Neural Process (ANP) Kim et. al, 2019, which shoes better performance than Neural Processes?\n3. Could the model also be run with Gaussian Processes? I think a comparison to the simpler GP would be nice to add to motivate the benefit of the NP.\n4. What is it about the electricity dataset that TimeGrid shows improved performance?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "- Addresses important problem of probabilistic time series forecasting\n- Interesting idea to use the latent representations from the Neural Processes, typically used in computer vision\n- Extensive experiments on 9 time series datasets\n- Multivariate forecasting problem is challenging\n- Good selection of multivariate probabilistic baselines to compare to including GP-Copula\n- Good probabilistic metric CRPS is used in the benchmarking\n- KooNPro shows state-of-the-art performance on 8/9 datasets", "weaknesses": "- I think more background on the importance of probabilistic forecasting and its use in practical downstream task, e.g., supply chain in the introduction would be helpful\n- Overall the writing quality could be improved.\n- When discussing LSTMs in the introduction, DeepAR should be cited\n- For probabilistic diffusion models, Kollovieh et. al, \"Predict, refine, synthesize: Self-guiding diffusion models for probabilistic time series forecasting\", NeurIPS 2024 should also be cited.\n- There has also been an abundance of foundation models for time series forecasting, e.g., Ansari et. al, \"Chronos: Learning the language of time series\", 2024 that is not discussed int he introduction.\n- DMD is more typically used in the dynamical systems and scientific computing communities\n- The motivation for choosing Neural Processes as the probabilistic models rather than more current state-of-the-art probabilistic models, i.e., diffusion models is not clear.\n- For application of NPs to spatio-temporal time series (PDEs), see Hansen et. al, \"Learning Physical Models that Can Respect Conservation Laws\", ICML, 2023\n- Some of the background in Section 3 could be moved to an appendix to allow for more novelty of the method presentation and results in the main body.\n- The architecture in Figure 1 is also taking a large amount of space.\n- I have some concerns on the novelty since the proposed method is just combining two previously proposed approaches.\n- May be good to include multivariate time series forecasting problem definition.\n- Comparisons to other baselines, e.g., DeepAR (Salinas et. al, 2019) and MQ-CNN (Wen et. al, https://arxiv.org/abs/1711.11053, https://proceedings.mlr.press/v151/park22a/park22a.pdf), MQTransformer (https://arxiv.org/pdf/2009.14799, ) are missing, which could be run on electricity and traffic\n- Bold the best in Table 2 or use same convention as Table 1 with color in red but I think bolding would be best for both.\n- The method seems off in the case study in Dimension 6 and 8", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Z9lIMUVImE", "reviewer": "Reviewer_piba", "summary": "The paper presents KooNPro, a novel approach for probabilistic time series forecasting that integrates a variance-aware deep Koopman model with Neural Processes (NPs). By employing a variance-aware continuous spectrum modeled with Gaussian distributions, KooNPro effectively captures complex temporal dynamics with enhanced stability. It leverages NPs to capture global patterns across time series, improving prediction accuracy. Extensive evaluations on nine real-world datasets show that KooNPro outperforms state-of-the-art models, with ablation studies confirming the importance of its components and hyperparameters.", "questions": "Can you explain more intuitively why complex temporal dynamics can be captured using the proposed probabilistic deep Koopman model? In addition, I'm curious about the validity of the assumptions that formula (6) relies on: \"hypothesize the latent space created by $\\phi$ possesses linear characteristics\", since formula (6) seems to be an important foundation of the subsequent derivations.", "limitations": "", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The proposed variance-aware continuous spectrum modeling address the complex nonlinear temporal dynamics, which is novel and has a solid theoretical foundation.\n2. The empirical improvements are significant, outperforming other methods in complex, high-dimensional forecasting tasks.\n3. The ablation study demonstrates the effectiveness of Neural Process and help to understanding the role of this key component. The case study shows that KooNPro's predictive capability aligns well with real-world observations, capturing diurnal patterns and demonstrating reliable predictive intervals.", "weaknesses": "1. The integration of Neural Processes and the probabilistic deep Koopman model requires a sophisticated training approach, including variational inference for optimizing the ELBO. This could pose challenges in implementation, and could increase the computational complexity. \n2. The effectiveness of KooNPro relies heavily on the design of the encoder and decoder networks for the linear space transformation. My concern is that poorly designed architectures could lead to suboptimal representations and thus hinder the model’s ability to learn the underlying temporal dynamics accurately.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "8NDdrYeW9Z", "reviewer": "Reviewer_MwLy", "summary": "The authors of the paper introduce a novel probabilistic time series forecasting model called KooNPro. KooNPro utilizes a variance-aware continuous spectrum, modeled using Gaussian distributions, to capture complex temporal dynamics and improve stability. By incorporating Neural Processes, the model captures fine dynamics and enhances global modeling capabilities. The authors evaluate KooNPro on nine real-world datasets and find that it consistently outperforms other state-of-the-art models in terms of accuracy and stability.", "questions": "* It is not clear what the term \"spectral pollution\" means in line 143. Consider explaining it for a reader who may not know about Koopman operators.\n\n* Why are all context lengths 10 in Table 4 (Page 15)? Why wasn't the model tried with a different context lengths? This is also concerning as much larger context lengths are used in practice.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "* Deep Koopman models had been used for time series forecasting before but not in conjunction with Neural Processes, hence the proposed model is non-trivial and new. \n* The authors enhance their model by a variance-aware continuous spectrum, which to my knowledge has not been done before.", "weaknesses": "* Clarity: The authors could improve how they present their work. The math is clunky and too detailed, and can cause the reader to lose attention. For example, in section 4, the authors never refer to Fig.1 more than once (in line 169), so there is a disconnect between the math and the figure. Maybe explain the figure first and detail the model at a high-level, then go in the inner workings and foundations. Also in the caption of Figure 1, consider adding pointers to different parts of the figure to help the reader understand where to look in the Figure for each reference in the caption.\n\n* Better analysis of each component of the model: The authors can provide a more comprehensive and insightful analysis of the Neural Process contribution. They may investigate how different NP architectures (e.g., Attentive Neural Processes, Convolutional Conditional Neural Processes) affect KooNPro's performance.\n\n* Limited empirical discussion on the exact benefits of this model: The authors can better demonstrate For example, conduct experiments to specifically evaluate KooNPro's performance on, say, non-stationary time series or time series tasks with a much larger prediction length etc., comparing it to baselines that do not incorporate NPs. Compared to other models, if there were more convincing and broader experiments demonstrating the benefits of the proposed model, it would lead to wider adoption. \n\n* More empirical evidence for robustness and stability: The paper claims robustness and stability as advantages of the proposed model, but these claims are well empirically validated. For example, the authors can add experiments evaluating the model's performance under different noise levels\n\n* Details on training baselines: Since the baselines were trained for each dataset, the authors should provide details on how the baselines were trained (how hyperparameter selection was done, what protocol was used etc.) in the appendix. It is important to check if the evaluation compares all models in a fair manner.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)