HopCast: Calibration of Autoregressive Dynamics Models
Authors: Muhammad Bilal Shahid
Cody Fleming
Venue: Accepted by TMLR ()

Abstract: Deep learning models are often trained to approximate dynamical systems that can be modeled using differential equations. Many of these models are optimized to predict one step ahead; such approaches produce calibrated one-step predictions if the predictive model can quantify uncertainty, such as Deep Ensembles. At inference time, multi-step predictions are generated via autoregression, which needs a sound uncertainty propagation method to produce calibrated multi-step predictions. This work introduces an alternative Predictor-Corrector approach named HopCast that uses Modern Hopfield Networks (MHN) to learn the errors of a deterministic Predictor that approximates the dynamical system. The Corrector predicts a set of errors for the Predictor's output based on a context state at any timestep during autoregression. The set of errors creates sharper and well-calibrated prediction intervals with higher predictive accuracy compared to baselines without uncertainty propagation. The calibration and prediction performances are evaluated across a set of dynamical systems. This work is also the first to benchmark existing uncertainty propagation methods based on calibration errors.

Published in Transactions on Machine Learning Research (12/2025)
HopCast: Calibration of Autoregressive Dynamics Models
Muhammad Bilal Shahid
belal edu
Department of Mechanical Engineering
lowa State University
Cody Fleming
Reminge edu
Deportment of Mechanical Engineering lowa State University
Reviewed on OpenReview: https: opppr/vveeer........ net/ per/formm? id= f
Abstract
Deep learning models often trained approximate dynamical systems that be mod- eled using differential equations. Many of these models are optimized to predict one-step ahead; such approaches produce calibrated one-step predictions if the predictive model can quantify uncertainty, such as Deep Ensembles. At inference time, multi-step predictions are generated via autoregrossion, which needs a sound uncortainty propagation method to produce calibrated multi-slep predictions. This work introduces an alternative Predictor- Corrector approach named HOPCAST that uses Modern Hopfield Networks (MHN) to learn the errors of deterministic Predictor that approximates the dynamical system. The Cor- rector predicts set of errors for the Predictor's output based timestep during autoregression. The set of errors creates sharper and well-calibrated pre- diction intervals with higher predictive accuracy compared to baselines without uncertainty propagation. The calibration and prediction performances are evaluated across a set of dy- namical systems. This work is also the first to benchmark existing uncertainty propagation methods based on calibration errors. We also evaluate HOPCAST as a substitute for Deep Ensembles within model-based reinforcement learning planner, demonstrating improved performance across multiple control tasks.
context state at any
1 Introduction
Approximating dynamical systems that can be modeled using differential equations has many applications in systems biology (Wang al.] 2021 Brauer & Kribs 2016), control (Moerland et al. 2023 Luu et al. 2021), economics (Tu 2012), and cyber-physical systems (Biaal Shahid Fleming 2025 Shahid et al. 2024
Robison et al.] 2024 Many deep learning approaches to approximate dynamical systems optimize the
model to predict one-step ahead during training (Khansari-Zadeh & Billard 2011 Coates et al.] 2008 Ko
et al. 2007). The multi-step predictions are generated via autoregression at inference time (Lu et al.) 2021) In doing so, the model error accumulates over time, resulting in multi-step predictions diverging from the ground truth (Venkatraman et al. 2015 Janner et al. 2021). Hence, is imperative to develop methods to generate accurate and calibrated multi-step predictions.
Stale-of-theart methods for generating accurate and calibrated predictions include Probabilistic Ensembles (Deep Ensembles) (Lakshminarayanan et al. 2017). To approximate the dynamics typically associated with differential equations, these models are optimized to predict one-step ahead during training and then autore- gressively generate multi-step predictions at inference time. To generate calibrated multi-step predictions, Probabilistic Ensembles need sound uncertainty propagation methods such as Trajectory Sampling (Chual
et al. 2018a).
In contrast, we introduce a Prodictor-Corrector mechanism to produce accurate and calibrated multi-step predictions for dynamical systems without the need for uncertainty propagation. The Predictor is optimized

---

Published in Transactions on Machine Learning Research (12/2025)
to predict one-step ahead point estimate of the next state of the system during training and produce multi- step predictions via autoregression at inference time. Our Corrector is based on the idea that similar states of a system lead to similar errors. This idea was leveraged by Auer et al. (2023) to build calibrated prediction intervals for onc-step prediction utilizing Modern Hopfield Networks (MHN). However, the same state may not lead to the same error for multi-slep predictions for dynamical systems. In such situations, the error also depends on how the system arrives at that state, i.c., where the system started and when it reached the state. To that end, we introduce the concept of contert state, assuming that similar context states lead to similar errors. The Corrector, utilizing MHN, learns similarity between context states in terms of errors. At any timestep during autoregression, the Predictor predicts, and the Corrector generates prediction intervals for the prediction. To control the width of prediction intervals for calibration, we introduce the concept of Atlention Span. We name our approach HOPCAST.
Our contributions are:
the introduction of the Predictor-Correcta mechanism, where the Predictor autoregrossivelly gen- erates the next state for dynamical system, and the Corrector predicts a set of errors for the full forecasting horizon. The expected error corrects the prediction, while the set of errors generates the calibrated prediction intervals.
a method HOPCAST - that has no assumptions about the Predictor except that it prodicts a point estimate of the next stale of the system. It is therefore a general framework that can provide multi-step calibrated predictions irrespective of the form or capability of the underlying Predictor.
lower calibration and prediction error across several benchmarks, without the use of complex uncer- tainty propagation techniques. Unlike uncertainty propagation methods, HOPCAST creates sharper prediction intervals based on context state instead of accumulating uncertainty over time during au- toregression. We also test HOPCAST as replacement for Probabilistic Ensembles in model-based reinforcement learning planner, achieving strong performance on standard control tasks.
2
Related Work
Since we propose a mechanism to quantify the predictive uncertainty and generate calibrated prediction intervals, the related works include those from the uncertainty quantification (UQ) and model calibration. In UQ, there are two lines of work: Bayesian and Ensemble methods.
Bayesian Methods, In Bayesian models, distribution over the parameters of the underlying model is learned, called posterior distribution. At inforence time, multiple samples of paramcters can be taken from the posterior distribution using an accurate sampling method, such as Hamiltonian Monte Carlo (HMC) [Betancourt] (017).. These samples serve as multiple predictive models and give us the desired diversity in the predictive variable of interest. A notable method from this domain is Bayes by Backprop (Blundell et al. 2015). which learns approximate posterior distribution over the weights of feedforward model via Variational Inference (VT) by optimizing the evidence lower bound (ELBO) (Kingma et al. 2015).
Ensemble Methods. In classical ensemble methods, an ensemble of models is trained with variations in the data for each model in the ensemble to train robust predictors (Davison & Hinkley] 1997). Deep Ensembles, however, are constructed using modern deep learning models with random initializations in the parameter space. These wver-paramctrized models, when initialized randomly, converge to different local minima, giving us the desired diversity in the predictive variable (Fort et al. 2019) These models were first presented by [akshminarayanan et al. (2017) in the context of predictive uncertainty estimation. A notable extension of that is Neural Ensemble Search (Zaidi et al. 2021), which uses different architectures for models in an ensemble and improves upon deep ensembles. The Deep Ensemble is a scalable way of quantifying uncertainty compared to Bayesian methods (Wilson & Izmailov] 2020). Hence, these are used as baseline models with different uncertainty propagation methods in the current work.
Model Calibration. Calibration has been studied widely for classification models. In binary classification, Platt Scaling (Platt et al. 1999) and isotonic regression Siculescu-Mizil & Caruana, 2005) have been used
2

---

Published in Transactions on Machine Learning Research (12/2025)
successfully for recalibration. There are extensions of such works to multi-class classification problems (Zadrozny & Elkan 2002). In the context of regression, Gneeiing & Raftery (2007) proposed several proper scoring rules to evaluate the calibration of a probabilistic model for continuous variables. Those scoring rules have boen used the literature as loss functions, for example, continuous ranked probability score (CRPS) Gasthaus el. al. 2019.. Calibration has also been discussed in the literature on probabilistic lorecasting, mainly the context of mectooologyy (Gneiting Raftery 2005). resulting in specialized calibration systems (Raftery et al.) 2005). An approach called calibrated regression was proposed by Kuleshov et al. (2018), which used isotonic regression to recalibrate Bayesian models. In contrast, our work turns deterministic Predictor into a calibrated Predictor via Predictor-Corrector mechanism. Also, we focus Predictors that autoregressively generate the next state of dynamical system, a setting rarely addressed in the model calibration literature to the best of our knowledge.
3 Problem Description
Consider a dynamical system described by multivariate ordinary differential equation (ODE).
dx(t) f(x(1)) di
x(1)
(1)
where x(1) E R.D denotes the state of D-dimensional system at time t and x(t) E TR. is its first order time derivative. The f(x(t)) specifies the vector-valued time derivative function. The state of a dynamical system at time l can be obtained by integrating the ODE as:
x(t) xu + J. f(x(T))dr
xu+
(2)
The ODE is integrated forward in time starting from the initial condition x(0) = Xo. We assume that the vector field is unknown, but can be learnod basod on observed data. The goal of predictive model is to predict x(I) as accurately as possible, for as long as possible, that is, to minimize the distance bet the prediction (t), and ground-truth trajectory, x(t). Additionally, in this work, for D predictions of each variable in x(),, we wish to create set of models that correct such predictions, M,(t) for i E DJ.
The following exposition describes the data generation process and nomenclature for Predictors, Correctors, trajectory types, errors, and so forth. The method is general and can be used for any D-dimensional problem as in equation [1. For ease of exposition and without loss of generality, we select a two-dimensional ODE to describe the inner workings and preliminaries. The Lotka- Volterra (LV) system is used here and in section for illustrative purposes; subsequent sections show experimental results for variety of systems. TV dynamics can be written as:
dx
az Bra dt
(3)
dy bzy YY dt
(4)
The state of this system can be defined as: (I, y). We assume a set of ground-truth N trajectories of T timesteps IN = [N where Tn (( 2x 1)) and s (xf, v²") This dataset can be generated by solving the equations with numerical ODE solver such solve ivp from scipy with random initial conditions. To simulate measurement noise, we perturb each state post-integration with additive Gaussian noise, i.., s & + e, where e <~ NN Σ) and = diag(az, The a is the scaling factor. We name IN an Integrated datasct. We assume that there is a Predictor B (a learned representation of vector-valued function f), trained on IN, that predicts 5,+1 given 5,, autoregressively generating predicted set of trajectories starting from the same initial conditions. Precisely, AN 1.N } where F, = {[5] ()))),))))) and S ("). We call Ay Autoregressive datasct. The crror (c.)) is the difference between tt
The overhead bar over variable shows that belongs Autoregressive dataset.
3

---

Published in Transactions on Machine Learning Research (12/2025)
timestep of nh trajectory from Integrated and Autoregressive, that is, oj (exy, Cxr) where of ss S Jn
Therefore, the set of errors of N trajectories is FN {[er)==
For the LV example, Predictor B. M2 and My will predict set of errors F ( and Fy = b Jizz associated with
assume two correction models, Mg and My, corresponding to each output of the
th timestep of a trajectory inference time, respectively. The 2% prediction interval, i.c., PIž and PT for I++1 and Me+1 at tth timestep can be generated as follows:
PI S11 Fr+1 + (Q. Q1-
(5)
PI #e+1
= M1+1 + (g) Q1-a
(6)
where a = 1_z For example, a 90% prediction interval corresponds to a = 0.05. The Qa denotes the a- quantile. We generate prediction intervals for the outputs of Predictor B t timestep during autoregression solely based on the set of errors predicted by correction models (M, and My), thereby avoiding uncertainty propagation.
4 HopCast
The proposed methodology, HOPCAST, consists a Predictor-Corrector mechanism shown in Fig. [ At any timestep (t) during autoregrossion, the Predictor B produces a point forocast of the next state of the system (T++1, 1) given previous state (īe, Wt). The Corrector retrieves conlext-dependent errors (E, and Eg) to correct the forecast and quantify uncertainty. In particular, the Corrector consists of correction models (Ma and My), one for each output of the Predictor B. Each correction model (M) consists of an Encoder (m) and an MHN (Ramsauer et al. 2020; Auer et al. 2023). The Encoder (m) is a fully connected leedforward model. Architectural details are included in Appendix E.1 We frame correction as a pattern retrieval task (Ramsauer et al. 2020). Precisely, M, and My take the context state (Ø1, M1, Zt. Vt. t) as a pattern and outputs Eg and E,, respectively. Eg and E,, contain errors associated with similar context states from the past. The context state the input of Predictor (Z1, 51) and time step (t)- The context state captures both local and trajectory-level structure, enabling the Corrector to retrieve errors specific to the current dynamical regime. The Eg & Ey are used to derive prediction intervals.
(ãt, augmented with the initial condition
x1+1
Predictor
Di+1
Context
PI 8211 1Ra (Ex), (1-a
Pl Pi-1 + [ea (Ez). ).Q.... Qr-a [Ey)]
X1
Corrector
1
Encoder,
MHN
>
>Eg
fd E
1=1
> Ey ( {Et 1
Encoder,
MHN,
t
Figure 1: Predictor- Corttettrr mechanism
The remainder of this section consists three subsections, Section 4.1 discusses the shuffling of IN, An, Fy datascts to train correction models for pattern retrieval tasks. Section 4.2 then discusses the training of the correction model (M₂), and section 4.3 describes the inference with M, where we generate the set of errors (Fiz) given the context state.
4

---

Published in Transactions on Machine Learning Research (12/2025)
4.1 Shuffled Data
The MHN in HOPCAST is trained a pallern-rellieuw relrieval model rather than a sequence model. To this end, we break every trajectory into independent context-error pairs and shuffle them, so the model learns to retrieve errors based on the similarity of context, rather than the temporal ordering of the data. We explain that below.
Shuffled Queries (Sq). Before training the correction model (M₂), the states of the system from Au- toregressive (Av) dataset are augmented with context information, that is, initial condition and time step. Appendix E.2 emphasizes the importance of adding initial condition as context. Specifically, the tth Wh . after adding context information. The tuple (#", DI') is the initial condition of trajectory n The C denotes the context state associated with the 1th timestep of ntth Irajectory. The set of all context states constructed from dataset AN is {[ 17"
timestep of n trajectory in from Ax, i.e., D)), will be o. (#), ...
1 This nested set of context states is flattened and shuffled randomly to form a new set
N S = Shuffle UleT)E))
Appendix E.7 highlights the importance of shuffling the data.
n==1
|i Shuffled Keys (Sk). Likewise, we construct the context states 1T for Integrated (Iv) dataset as well, where cf = (2]. yi, xft vi 1). Similar to SQ: we flatten the nested set ((c))==)) JN and randomly
U
shuffle it to form SK = Shuffle
n=1
Shuffled Values (Sv, & Sv,). Given errors FN = [T iN where ox = (Cpp,Oyp) EN can be split into two sets of errors F {eer M and F = {leg Y Ie=lIn=1- IN= Recall that we train separate correction models (M₂ and My) for each output of Predictor B. F and F% are flattened and shuffled to
NNN ( and Sv, = Shufll
Uley n=1
form Sv, = Shufle (1=)
Once we have Sq, Sk. Sv,, Sv,, they are split into train/test with an 80/20 split. To train M,, we need dataset D3 ((Q, Vo)i} comprising triplets (Q, K, Va). Each triplet of Q, K, and V, are constructed by sampling Sy (Sequence Length) number of elements from sets St STrain Train and SV. respectively. For instance, one triplet with S, = 5 is: Q = (c), c of c)) K (c) C, of 3 (T, and V. (e2g. Cag.Caq, C2g. The error c = x2 #2 denotes the error of Predictor B at 4th timestep of 3rd
Shrain The 20% split with trajectory. Likewise, construct D,- {QQ K, Vy)i using S SK TTrain and SV, inference data is used for final evaluation.
4.2 Training
Figure 2 illustrates how the correction model (M²) retrieves context-dependent errors using an Encoder (m,) and Modern Hopfield Network (MHN,): (i) the m, maps each context state into d-dimensional embedding, and (ii) the MHN, forms associations between embedded queries and keys and retricves the corresponding error values. Specifically, the m. takes in as input the context state, such cy, and outputs d-dimensional embedding. We choose d = 4 in our experiments, though other values of d are expected to perform comparably [Appendix [.6]. We use the same m, to construct embedded queries Qg and keys K2 from Q and K, respectively. These appear as row/column labels in the Training block of Fig. 2 The MHN, uses an attention mechanism (Vaswani 2017) roct an association matrix (As xs,) based on the similarity of elements in Qº with elements in Kg. Mathematically,
Q
x = softmax
(7)
Vd
V 2 = Agxxs .
(8)
1
L
VV a - V2112
(9)
St.
5

---

Published in Transactions on Machine Learning Research (12/2025)
The association matrix (Asixst) is shown in the Training block of Fig. with St 5. We mask the association from Autoregressive context state to its Integrated context state. This masking is reflected in the zeroed diagonal entries of the association matrix. The (i,j)th entry of As shows the similarity of t element of Q2 with jth element K We use softmax over As, xs. [i. to get association weights for ith element of Qu- Therefore, each row of Asxss, sums up to one. The loss function is used to learn the parameters of the my via backpropagation, where the V, denotes the predicted errors.
Training (M,)
Keys (KyJ) [Integrated]
Values (V.)
Prediction (2))
me,
mn((c5)
m.(el)
Queries (2p) [Autoregrattion)
(c.)
0.2
0.1
0.6
0.1
ex
2x1
0.3
0
0.2
0.1
0.4
2x]
2x
0.4
0.4
0.1
0.1
2x
0x4
(
0.5
0.1
0.1
0
0.3
2x
0.7
0.1
0.1
0.1
0
ex]
8x]
Inf Frencc (M,)
Association memory
<x
ex5
ex]
c
e2g
(x))
my(e])
m_(c])
(y(c])
m,(c3)
(ef
(=(e))
Association weights (af)
00 m,
0.02
0.2
0.1
0.13
0.07
0.24
0.15
0.3
Figure 2: Training and Inference with Correction Model (Encoder, + MHN
4.3 Inference
At inference time, the correction model (M²) retrieves past er- rors based on the similarity between the encoded query and the encoded keys stored in the MHN,, Association memory. To build the Association memory, set of K keys (Kmem) with the corresponding values (Ve (Venm) is sampled from STrain and STrain, respectively. For instance, a randomly sampled set of keys and values would be Kmem = (c], cf,, ... and V V.nem = (ezg, Cag, Exg, ,egg), respectively. The trained Encoder m, used to encode the Kmem as Ky mem. The K mem and V mem con- stitute the Association memory of MHN It is shown in the Inference block in Fig. 2 We loaded 2000 keys in memory for evaluation [Appendix E. Once the memory is set up, the cor- rection model (M₂) takes in the context state (Tr, W1, Te, Nt, 1) as query Qi 4t time step during autoregression and encodes Q The association weights (a))) at. thh timestep are,
Timestep 106
0.00150
0.00125
0.00100
0.00075
0.00050
0.00025
8..
-20
20
40
Errors
Figure An example set of erroors (E,) sampled from Association Memory at Timestep 106 during autoregression for the x-output of the Predictor for the Lorenz system, where the red dotted line denotes the ground truth error.
Q K mem
a softmax
(10)
Vd
6

---

Published in Transactions on Machine Learning Research (12/2025)
where a (p1.... PK), E P: = and K denotes the number of keys in memory. The weights a show how strongly the query Q matches each entry K mem- A set of categorics is sampled as: {m:li-1 ~ Categorical(af). We select 1000 in our experiments [Appendix E.3
Pk))
set of errors (Ez) is selected from Vm mem based sampled categories {be)i=1> i.e,, F (emm [b i=1- A sampled set of errors Eg is shown in Fig. for the x-output of the Predictor B for the Lorenz system. An alternative approach to sampling from a is to retrieve top-k probabilitics. We provide an ablation study on this in Appendix E.4
We have discussed the training of M, and how to utilize it to obtain a set of errors (Fa) for the x-output of the Predictor B at the inference time. To obtain Ey by training My, we repeat the same procedure stated in soction 4.2 and section 4.3 with dataset Dy: and this can be done for an arbitrary number of dimensions depending on the system dynamics.
5 Experiments
5.1 Evaluation Metrics
We propose three metrics to evaluate the quality of HOPCAST against the benchmarks.
Calibration Error (Kuleshov et al., 2018) (CE), A calibrated model has observed fractions (p) of its predicted random variable match with the expected fractions (p)- The difference between these two is calibration error. To evaluate CE, w(= 9) equally spaced prediction intervals are chosen from 10% to 90%. For each prediction interval, we count the number of times the observed variable falls between intervals. Mathematically, CE = DE.1 (f. -p:)².
Prediction Interval Width (Auer et al. 2023 (PI-Width). A calibrated model should also be sharp. Sometimes, it is trivial to reduce the CE by always predicting the expected value of random variable (Kuleshov al.) 2018). A good predictor should be calibrated and sharp. Hence, propose to use PI- Width measure of sharpness. Mathematically,
PI-Width = W
(11)
where w is for equally spaced PT, and denotes the number of validation data points; i.e., v |S
|SInference
Mean Squared Error (MSE). MSE is used to evaluate the predictive accuracy of the proposed approach against the baselines.
5.2 Datasets
We have discussed one dynamical system, i.e. LV, in section 3 Other dynamical systems include Lorenz, FitzHugh-Nagumo (FHN), Lorenz95, and the Glycolytic Oscillator. To generate datasets, we randomly sample initial conditions from within a specified range for the state variables of each system. The solve ivp method from scipy is used to integrate the dynamics with the adaptive-step RK45 solver. To produce uniformly sampled trajectories, system states are extracted fixed time intervals At as mentioned in Table 4 For Lorenz and IV, the dynamics of both system states and their derivatives modcled, whereas the dynamics of states are modeled for the rest of the systems. The mathematical forms of each dynamical system, ranges of initial conditions, and parameter values are given in Appendix B
5.3 Baselines
The baseline methods include three uncertainty propagation approaches, i.e., Expectation, Moment Match- ing, and Trajectory Sampling. These approaches are used to propagate uncertainty with Probabílistic En- sembles (Chua al.] (2018a). The details of Probabilistic Ensembles' training and uncertainty propagation methods presented in Appendix A.

---

Published in Transactions on Machine Learning Research (12/2025)
6 Results
Overall Comparison As shown in Table 1 HOPCAST outperforms the bascline approaches in terms of CE and MSE in the vast majority of cases. It achieves the lowest average CE of 0.046 among all, with the second- best approach being the Trajectory Sampling with CE of 0.075. In terms of MSE, HOPCAST outperforms in 11 out of 15 cases, showing the effectiveness of our Prodictor-Corrocto mechanism. The Expectation and Moment Matching do not perform well in terms of CE and MSE. These two approaches generate conservative intervals with average PI-Widths of 7.56 and 3.63, respectively, resulting in overconfident. models. The overconfidence is evident in the calibration curves of Expectation and Moment Matching in Fig, 6 The average CE of these two approaches is 0.21 and 0.68, respectively, showing high miscalibration compared to Trajectory Sampling and HOPCAST. PI-Width is only useful when discussed in conjunction with CE as it is trivial to reduce the Wi-th while being overconfident and miscalibrated.
Table 1: Prediction Interval Widths (PI-Widths) and Calibration Error (CE) metrics of HOPCAST and baseline approaches with different scaling factor (a) across various dynamical systems. The results are averaged across 3 runs of the same experiment. All of the CEs within 5% of the best one in a row are highlighted. The PI-Width is only used as tic-breaker if there are different models with CEs within 5% of the best. In that case, the berf-perf model based on CE is denoted with an asterisk and PI-Width is highlighted. For MSE, we simply highlight anything within 5% of the best one in row.
Probabilbttc Enserable (PE)
Model
HopCass
Expectation
Momeui Mutching
Sumpling
Motricaa
MSE
PL-Witth
CE
MSE
FL-Wiath
CE
MSE
PL Pi-Width
CE
MSE
PL-Wath
CE
0.05
2.00 L0.17
0,022
40:
1.000
0.12
0.41
0.22
Volterrs
Lotka
0.1
6.10
0.0000 L85
0.002
0.15
0.17
28.0
0.96:0004
0.14
2100
6.7
0.18 0.10
9.09 #: 0.28
0.0011 0.10
0.004
25.60000 0.00
201
0.21 015
00.
2.96
0000 = 0.02
28.3 0.05
6.19
0:30 0:07
0.05
0.11
8.000
1.38
Lecons
0.3
1248
38.00
0.009 0.00
1413.1 38.33
35:40 à
0.111
0.015
287
6.78:0001
1.21 0.08
1914/60 2:
45.28 0.69
0.08
1681 1: 46.36
40.44
0.91
0.010 0.004
3848.5 4 16.8
25.0
0.18
200.5 =:
1808
2111 0.00
0.88 0.30
1788.6 = 10.9
48.02 = 0.30
0.018 0.110
0.05
0001
FHI
0.1:1:000
1.0
0.52 1.2.2.0.5
0.17
0.66
0.18
2.13
0.00
1:
0.83 C.000
1,35 0.005
0.51
0.04
1.27 0.0
0.31 0.02
LG3 0.09
1000
0.10
1.25 0:000
1.22 0.000
0.108 =2 0,010
C.10
1.50 0.004
0.001
& 0.11
0.05
00:44:0008
0.1
0.007
= 0.003
10.11 021
0.13 0.05
0.017
14111:06 0.68
184 0.000
0.62 0.04
9:90 0.051
0.30
0.007
Loccud6
2.11
5778
0018
00025:
0.007
13.31
3.29 0.06
0.27 1 5:05
17:38 1.15
3:21 0.05
0.55
11.63 0.11
5:111001
0.001
6:000
044420000
6.06
0.0
0.0015
0.001
0.46.220.11
4.6 0.04
0.13 3.029
16:77
A: 0.25
6.01 0.05
0.0036 0.03
15.15 0.075
6,49
5:40:00007
0.000
2: 0.000
0.05
0.0010000
0.20
0.033
0.018
0.10 0.000
021:1:000
0.37
0L13 1:000
0.066
0.022
1.58
0.32
0.00
0.36
0.00
0.06 0225
Glyeolytic
Oscillator
0.022 0.008
0.25
0.007
0018
0.006
012.4 8/00
0.21 0.00
0.10 0.03
8115 6/6000
0.88
5.001
0.15
0.10 C.000
0:41
0.00
6.05
0:111:0001
0.57 2: 0.008
0.016 0.000
0.18 0.00
0.23 0.01
0.52 0:28
(126 & 00000
0.21
0.000
0.54 c2 0.07
(6 C.000
0.66 0.005
0015
0.01
Average
9:03
(L0%
7:56
0.21
3.55
0.56
11.48
0.075
Attention Span HOPCAST lets us control the confidence of the model with precise control over the width of PT based on a concept. we introduce called Atlention Span. The Allention Span can be increased aeaasedd/eeeassed by pncreasing/deccrassing the S, of Q and K in As To demonstrate that, we collect dataset H [12z, y:)1 using the sine function with heteroscedastic noise (Chua et al. 2018a) shown in equation 12
and Fig. Aaa. The x4 's are uniformly sampled from the following domain
2 U
[TT. 5x 2
(1r v) + x. (. +N (0.0.2 sin
3
1)
T 8
(12)
We use the same correction model (M₂) consisting of Encoder (m,) and MHN, with the same training and inference procedures described in section 4.2 and [.3] respectively. The only exception is that the set of triplets {(Q K, V):} are constructed from dataset H. For instance, triplet with S. = 4 is: Q
8

---

Published in Transactions on Machine Learning Research (12/2025)
(210, 213, 214, 21), K (æ10; #13, T14; 21), and V (V10. V13, M14: 111. We trained two M, models with Sp of 3 & 8. At the inference time, the M2 is given data point = 3.28 the keys in memory based on similarity in teriss of y. The keys from memory are sampled based on their association weights. The frequency plots of sampled keys for S1 of 3 & 8 are in Fig. 4 & (c), respectively. For S 3, the sampled keys are clustered around 3.28 Fig. [b), which are the most similar keys the query x 3.28. As we increase St. to 8, the M, expands its Attention Spon by picking other keys from memory around æ -3.2 & 6.5 that have nearly similar y values as shown in Fig. 4c).
query to be associated with
The function sinc
Nolay Data
-10
10
Frequency
Frequency
200
100
100
-10
-5
0
5
10
-10
-5
0
5
10
X
Figure 4: (a) Upper center: Sine function with heteroscedastic noise (b) Lower left: Attention Span for
x = 3.28 as query with S, 3 (c) Lower right: Attention Span for x = 3.28 as query with S =
Calibration Performance The reason HOPCAST performs well in terms of CE compared to basclines is owing to the St that can be prreasedd/eeceaseeen to increase/decrent the diversity of sampled errors Fg from MHN memory. As increase the St, the M, starts to expand its Attention Span by associating other keys from memory (K%..) to the query (Q.") based their similarity in terms of errors, effectively controlling the width of prediction intervals. We train separate correction models (e.g., M2 and M,) for each output of the Predictor and tune their Sr separately. The Sr. hyperparameter and number of models in a Probabilistic Ensemble for each setting are shown in Appendix [ For the baseline approaches, we tune the number of models in an ensemble, In general, we observed that the Probabilistic Ensembles tend increase the uncertainty (or widen their prediction intervals) as we increase the number of models, and the uncertainty platcaus at some point. Therefore, if the uncertainty plateaus and the model is overconfident, adding more models to the ensemble will not make it a calibrated model. This was observed in the calibration curves of Expectation and Moment Matching in Fig. 6 Trajectory Sampling, in contrast, provided calibrated uncertainty with few models in the ensemble (i.c., or 4) in most cases. It is owing to its nature of taking predicted uncertainty into account during propagation.
Predictive Performance HOPCAST outperforms in most cases in terms of MSE because we proposed a Corrector that predicts a set of errors, and prediction intervals are a side effect of the approach. Also, our model generates a smaller PI-width in most cases because we generate prediction intervals based on the context state (zo, No: It, Nt. propagation approaches generate intervals based on uncertainty accumulated due to propagation up certain time step (t = t') starting from = 1. A manifestation of that is shown in Fig. 5 (b) & (d). For Fig, a),, the output of Predictor B is shown in the dark, and the Mcan indicates the prediction after adding the expected error (Ei=1 a [bx] Ex] from the Corrector. For Fig. 5kb), (c), & (d), the Mean indicates the expected prediction from all models in a Probabilistic ensemble. Fig. 5(a) shows that HOPCAST reduces PI-Width around time step 270 due to smaller error, resulting in sharper intervals. On the other hand, Expectation and Trajectory Sampling have wider PI due to accumulated uncertainty up that time step, even though the Mean is close to the ground truth.
any timestep during autoregression. On the other hand, uncertainty

---

Published in Transactions on Machine Learning Research (12/2025)
Ground Truth
Mean
Upper Pl
Lower Pl
Basc Prodictor
20
-10
-20
-20
150
200
30
100
150
200
250
300
20
200
20
300
Figure 5: (a). Upper left: HOPCAST (b). Upper right: Expectation (c). Lower left: Moment Matching (d). Lower right: Trajectory Sampling. A comparison of prediction intervals generated by HOPCAST and three uncertainty propagation approaches. The x-axis denotes the time steps, and the y-axis shows the output of the Lorenz system. The Upper PI and Lower PI show the m equally spaced prediction intervals from 10% to 90%.
HopCast
Expectation
Outpue
Dutpe
0utpot
Observed Fraction
Outpor
Dotpxt
O-
DUPpuE
Oitpxt
Dupput
0.2
0.2
0.0
0.0
0.2
0.6
1.0
0.2
0.6
1.0
Expected Fraction
Expected Fraction
Moment Matching
Trajectory Sampling
Uotpxt.
Dupput
Observed Fraction
Dutput
Observed Fraction
0.8
DMps11
Oxpput
OLtput
0.6
Oi-pxi.1
Ou
0.6
0.4
0.4
0.2
0.0
Expected Fraction
0.6
1.0
0.0
Expected Fraction
0.4
0.6
Figure 6: The calibration curves of HOPCAST and three uncertainty propagation methods for Glycolytic Oscillator at = 0.3. The dark dotted line corresponds to the perfect calibration. The calibration curves are shown separately for seven states of the system.
Trajectory Sampling vs. Counterparts On average, Trajectory Sampling performs better than its counterpart Deelines-Exxpctattion and Moment Matching-in in terms of CE, owing to how it propagates certainty. It takes into account the predicted uncertainty of the model while propagating uncertainty from time step t to 1. It samples the state for the next step from the predicted Multivariate Normal distribution rather than propagating just the predicted mean like the Expectation. Moment Matching also samples the next state from the predicted Multivariate Normal distribution, just like Trajectory Sampling, However, it recasts prediction from all models in an ensemble every timestop (t) to a Multivariato Normal distribution and then resamples next states before propagation. This results overly conservative PI, shown in Fig. Eic), and evidenced by the PI-Width of 3.63 in Table
10

---

Published in Transactions on Machine Learning Research (12/2025)
7 Hyperparameter Tuning
St. (Sequence Length) is a hyperparameter that must tuned for calibrated uncertainty. We propose Algorithm basod on the intuition that uncertainty typically increases with larger St. (Section 6 Fig. A small value (e.g,, 10) produces overconfident estimates, while large value (e.g., 1000) leads to underconfidence. Intermediate values such as S, 500 yield near-perfect calibration, whereas St = 400 and Sy = 600 show overconfidence and cndderconiieene,, respectively. We iteratively refine St with the calibration curve and typically converge to an optimal sctting within a few trials. Appendix E.9 provides addi- tional experiments.
| to tune
HopCast
1.0
Observed Fraction
0.8
688
1080
0.6
0.4
0.2
The number of models in ensemble is hyperparameter. As discussed in the section P the ensembles tend to build up the
0.0
uncertainty until it plateaus [Appendix E.8 We keep adding models to the ensemble until provides calibrated uncertainty. some cases, the uncertainty saturates while the model over- confident. In those cases, the number of optimal models very high, e.g., or 7. Appendix contains additional details about hyperparameters.
0.0
0.2
0.4
0.6
0.8
1.0
Expected Fraction
Figure Calibration curves the y- output of Lorenz for different St.
A Algorithm Sy. Tuning for Calibration
1: Initialize OptimalSI- + True
2: L Pick large & small S,
3: D Pick an between large and small
4: while OptinalSt do
5: Three possibiliitis Colibration Curve
6: if Calbrratd Uncertointy then
7: OptimalSL False
8: else Underconfidence then
9: Decrease
else Overconfidence then
11: Increase St.
12: end if
13:
end while
8
Model-Based Reinforcement Learning
We compare the performance of HOPCAST and Probabilistic Ensembles as the uncertainty-aware dynamics models within a model-based reinforcement learning (RL) algorithm. PETS (Probabilistic Ensembles with Trajectory Sampling) is a widely used model-based RL algorithm that relies on Probabilistic Ensembles to model the transition dynamics and propagate uncertainty for planning via Model Predictive Control (MPC) CCha el. al., 2018b). In PETS, the planning quality is heavily influenced by the accuracy and calibration of the dynamics model, model bias compounds rapidly during long-horizon rollouts, and miscalibrated dynamics models lead the planner to choose overly optimistic overly conservative action sequences.
Our earlier experiments in Table demonstrate that HOPCAST produces more accurate and multi-stop predictions than Probabilistic Ensembles, which utilize different uncertainty propagation methods, i.e., Expectation (E) A.1 Moment Matching (MM) A.3 and Trajectory Sampling (TS) A.2 Table 2 we show the performance of HOPCAST on four standard control tasks when it used as a drop-in replacement for the Probabilistic Ensembles in the PETS algorithm. The results for baselines, i.e., Probabilistie Ensembles & Deterministic Ensembles, with three uncertainty propagation methods, i.c., Expectation, Moment Matching, & Trajectory Sampling, are taken from Chua et al. (0088)) b
The HOPCAST yields comparable performance
2ode available httpps:/ http://gghhuuboom.m of

---

Published in Transactions on Machine Learning Research (12/2025)
to Probabilistic Ensembles on Cartpole and 7-DOF Reacher, while it outperforms 7-DOF Pusher and Half- Cheetah tasks. The Probabilistic Ensembles, which utilize uncertainty propagation methods, outperform on Cartpole and 7-DOF Reacher, showing their competitiveness against HOPCAST. However, the Deterministic Ensembles cxhibit competitive performance a relatively low-dimensional setting, such as Cartpole, but show performance degradation on high-diimnnsionl lasks. These experiments demonstrate the efficacy of our proposed methodology HOPCAST within a model-based RL. algorithm on various control tasks.
Table 2: The performance of different tasks in terms of rewards with different dynamics models, iee, Proba- bilistic Ensembles (PE) & Deterministic Ensembles (DE), and various uncertainty propagation methods, i.c., Expectation (F), Moment Matching (MM), & Trajectory Sampling (TS). The HOPCAST shows the results with our proposed mothodology as a drop-in replacement of PE within the PETS planning algorithm. The best results are shown bold.
Model
Task
HOPCAST
PE-E
PE-MM
PE-IS
DE-E
DE-MM
DE-TS
Cartpole
181 2.3
180
183
179
177
181
7-DOF Pusher
45 4.3
7-DOF Reacher
4.4344
Half-cheetah
7170 40
5700
200
7100
3800
190
3950
9
Conclusions
We proposed Predictor-Correcte mechanism for autoregrossive dynamics models to correct the Predictors' predictions and generate calibrated prediction intervals for it at any timestep during autoregression. HoP- CAST performs competitively well against the alternative approach (i.e., uncertainty propagation), giving accurate and calibrated autoregressive dynamics models. Out of three uncertainty propagation approaches, i.c., Trajectory Sampling, Moment Matching, and Expectation, Trajectory Sampling performs competitively well against ours. This is due its nature of taking predicted uncertainty into account during propagation. HOPCAST offers lower calibration and prediction error with sharper prediction intervals. The sharper prediction intervals result directly from modeling context-specific errors, and the lower prediction error is the consequence of modeling errors in the form of a Corrector. In addition, we introduce a concept called Attention Spon that gives precise control over the width of prediction intervals with a hyperparameter we in- troduce called Sequence Length (St). We also deploy HOPCAST within a model-based reinforcement learning planner as an alternative Probabilistic Ensembles, demonstrating its competitive performance on diverse control tasks.
Limitations and Future Directions One of the limitations of our work is due to the use of explicit timestep IDs (i.e., t =1,2,3 etc.) in the context state (cf). At inference time, the Corrector will only generate calibrated prediction intervals for a trajectory length equal to or less than what was trained on. Morcover, the Corrector was trained on trajectories sampled a fixed time interval At and does not generalize other sampling intervals. A modified version of the Corrector could be made discretizationniivvriaal We hand-tuned the S₁ hyperparameter, training multiple models to get lower calibration error. An alternative approach to obtaining calibrated uncertainty with HOPCAST without tuning the S₁ hyperparameter is an interesting area for future work.
10 Acknowledgements
This work was partially supported by the National Acronautics and Space Administration, USA, under grant 80NSSC24CA037.
12

---

Published in Transactions on Machine Learning Research (12/2025)
References
Andreas Auer, Martin Gauch, Daniel Klotz, and Sepp Hochreiter. Conformal prediction for time series with modern hopfield networks. Advances in Neural Information Processing Systems, 36:56027-50074, 2023.
Michael Betancourt. A conceptual introduction
hamiltonian monte carlo.
ar preprint
arXiv:1701.02233 2017.
Muhammad Bilal Shahid and Cody Fleming. Toward robust car-following dynamics modeling via blackbox models: Methodology, analysis, and recommendations. Transportation Research Record, 2679(4):114-22, 2025.
Charles Blundell, Julien Cornebise, Koray Kavukcuoghu, and Daan Wierstra. Weight uncertainty in neural network. In International conference on machine learning, pp. 1613 1622. PMLR, 2015.
Fred Brauer and Christopher Kribs. Dynamical systems for biological modeling. CRC press, 2016.
Steven Brunton, Joshua Proctor, and J Nathan Kutz. Discovering governing equations from data by sparse identification of nonlinear dynamical systems. Proccedings of the national academy of sciences, 113 (15):3932 3937, 2016.
Kurtland Chua, Roberto Calandra, Rowan McAllister, and Sergey Levine. Deep reinforcement learning in a handful of trials using probabilistic dynamics models. Advances in neural informalion processing systems, 31, 2018a.
Kurtland Chua, Roberto Calandra, Rowan McAllister, and Sergey Levine. Deep reinforcemenn learning in a handful of trials using probabilistic dynamics models, 2018b. URL http::://aaaxww.og/bbb////////// 12114
Adam Coates, Pieter Abbeel, and Andrew Y Ng. Learning for control from multiple demonstrations. In Proceedings of the 25th international conference on Machine learning, pp. 144-151, 2008.
Bryan C Daniels and Lya Nemenman. Efficient inference of parsimonious phenomenological models of cellular dynamics using s-systems and alternating regression. PloS one, 10(3):c0119821, 2015.
Anthouy Christopher Davison and David Victor Hinkley, Bootstrop methods and their application. Number 1. Cambridge university press, 1997.
Stanislav Fort, Huiyi Hu, and Balaji Lakshminarayanan. Deep ensembles: A loss landscape perspective. arXiv preprint ar (tt::91120200 2019.
Jan Gasthaus, Konstantinos Benidis, Yuyang Wang, Syama Sundar Rangapuram, David Salinas, Valentin Flunkert, and Tim Januschowski. Probabilistic forecasting with spline quantile function rmms. In The 22nd international conference on artificial intelligence and statistics, pp.
PMLR, 2019.
Tilmann Gneiting and Adrian E. Raftery. Weather forecasting with ensemble methods. Science, 310(5746): 248-249, 2005.
Tilmann Gneiting and Adrian Raftery. Strictly proper scoring rules, prediction, and estimation. Journal of the American statistical Association, 102(477):359-3777 2007.
M. Izhikevich and R. FitzHugh. Naaguuo model. Scholarpedia, 1(9):1349, 2006. doi: 10.4249/ scholarpedia.1341 revision # 123664.
Michael Janner, Qiyang Li, and Sergey Levine. Offline reinforcement learning as one big sequence modeling problem. Advances in neural information processing systems, 34:1273-12 1286, 2021.
S Mohammad Khansari-Zadeh and Aude Billard. Learning stable nonlinear dynamical systems with gaussian mixture models. IEEE Transactions on Robolics, 27(5):943 957, 2011.
Diederik Kingma and Jimmy Ba. Adam: method for stochastic optimization, 2017. URL https:
7/axxiv.oo org. http/abb/11555 1988
13

---

Published in Transactions on Machine Learning Research (12/2025)
Durk P Kingma, Tim Salimans, and Max Welling. Variational dropout and the local reparameterizatiion trick. Advances in neural informalion processing systems, 28, 2015.
Jonathan Ko, Daniel Klein, Dieter Fox, and Dirk Haehnel. Gp-ukf: Unscented kalman filters with gaussian process prediction and observation models. In 2007 IEEE/RSJ international conference on intelligent robots and systems, pp. 1901-1907. IEEE, 2007.
Volodymyr Kuleshov, Nathan Fenner, and Stefano Ermon. Accurate uncertainties for deep learning using calibrated regression. In International conference machine learning, pp. 2796-2804. PMLR, 2018.
Balaji Lakshminarayanan, Alexander Pritzel, and Charles Blundell. Simple and scalable predictive uncer- tainty estimation using deep ensembles. Advances in neural information processing systems, 30, 2017.
Edward N Lorenz. Predictability: A problem partly solved. In Proc. Seminar on predictability, volume Reading, 1996.
llya Loshchilov and Frank Hutter. Decoupled weight decay regularization, 2019. URL https://arxiv.orrs abs/1711.0510
Cong Lu, Philip J Ball, Jack Parkor-Holder, Michael A Osborne, and Stephen J Roberts. Revisiting design choices in oflline model-based reinforcement learning, arXiv preprinl arXiv:2110.04 04155 2021.
Thomas M Mocrland, Joost Brockens, Aske Plaat, Catholijn M Jonker, et al. Model-based reinforcement learning: A survey. Foundations and Trends@ in Machine Learning, 16(1):1-118, 2023.
Alexandru Niculescu-Mizil and Rich Caruana. Predicting good probabilities with supervised learning. In Proceedings of the 22nd international conference on Machine learning, pp. 625-632 2005.
John Platt et al. Probabilistic outputs for support vector machines and comparisons to regularized likelihood methods. Advances in large margin classifiers, 10(3):61-74, 1999.
Adrian E. Raftery, Tilmann Gneiting, Fadoua Balabdaoui, and Michael Polakowski. Using bayesian model averaging to calibrate forecast ensembles. Mouthly Weather Review, 133( 2005. doi: 10.1175/ MWR2906.1. URL https://journalls hwtpppurnnaaaaatttooooooo aww.amc. http:///aaaaaaa.ooo/oaaaaaaaaaaa
11xml
Hubert Ramsauer, Bernhard Schäfl, Johannes Lehner, Philipp Seidl, Michael Widrich, Thomas Adler, Lukas Gruber, Markus Holzleitner, Milena Pavlović, Geir Kjetil Sandve, et al. Hopfield networks is all you need. arXiv preprint arXix:2008. 022177, 2020.
Robert Robison, Tom Shafor, Victor Dilorcto, Natalia M Alexandrov, Muhammad Bilal Shahid, and Cody H Fleming. Differential equation approximation using gradient-boosted quantile regression. In ATAA SCITECH 2024 Forum, pp. 0107, 2024.
Muhammad Bilal Shahid, Robert Robison, Tom Shafer, Victor Diloreto, Natalia M Alexandrov, and Cody H Fleming, Uncertainty quantification using deep ensembles for decision making in cyber cyber-physical systems. In AIAA SCITECH 2024 Forum, pp. 0108, 2024.
Pierre NV Tu. Dynamical systems: an introduction with applications in economics and biology. Springer Science Business Media, 2012.
A Vaswani. Attention is all you need. Advances in Neural Information Processing Systems, 2017.
Arun Venkatraman, Martial Hebert, and J Bagnell. Improving multi-step prediction of learned time series models. In Proceedings of the AAAI Conference Artificial Intelligence, volume 29, 2015.
Ying Wang, Bo Hu, Yuxue Zhao, Guofang Kuang, Yaling Zhao, Qingwei Liu, and Xiuli Zhu. Applications of system dynamics models in chronic discase prevention: systematic review. Preventing Chronác Disease, 18:E103, 2021.
Peter J Wangersky, Lotka-volterra population models. Annual Review of Ecology and Systematics, 9:189-21 218. 1978.
14

---

Published in Transactions on Machine Learning Research (12/2025)
Andrew G Wilson and Pavel Izmailov. Bayesian deep learning and probabilistic perspective of generaliza- tion. Advances in neural information processing systems, 33:4697-4708, 2020.
Yuan Yao, Lorenzo Rosasco, and Andrea Caponnetto. On early stopping in gradient descent learning, Constructive approrimation, 26(2):289-31) 315, 2007.
Bianca Zadrozny and Charles Elkan. Transforming classifier scores into accurate multiclass probability estimates. In Proceedings of the eighth ACM SIGKDD international conference on Knowledge discovery and data mining, pp. 694-699, 2002.
Sheheryar Zaidi, Arber Zela, Thomas Elsken, Chris C Holmes, Frank Hutter, and Yee Teh. Neural ensemble search for uncertainly estimation and dataset shift. Advances in Newral Information Processing Systems, 14:7898-7911 2021.
15

---

Published in Transactions on Machine Learning Research (12/2025)
A
Baseline Methods
This section will discuss how propagate uncertainty with Probabilistic Ensembles using three uncertainty propagation methods, i.c., Expectation. Moment Matching and Trajectory Sampling. These will be discussed with reference to the Volta-Volterra (LV) system, whose state at time step (t 1++ 1) is 8t+1 = (Iz+1 V(+1). We will see how to generate 2% prodiction intervals associated with S1+1 (PI at any timestep during autoregression with these approaches.
Probabilistic Ensembles To construct Probabilistic Ensembles, we train a population of fully connected feedforward models, i..., Ω (f where fi represents the 7Mh model within the population. Each model is trained with Gaussian negative log-likelihood (NLL) loss as demonstrated in Lakshminarayanan et al. (2017). Each model predicts Multivariate Normal (MVN) over the next state St+1 given the previous state of the system S,. Formally, Sc+1 MVN( E441) For the LV system, pe+1 (1Et1, 1441) and E(+1 =
Uncertainty Propagation At inference time, sample of M models (Jm) can be laken from the population (92 to form Probabilistic Ensemble. All three propagation approaches use a particle-based approach, where the core idea is to generate multiple trajectories via autorogression starting from the same inilial condition (Chua el al. 2018a). Let sy." denote the state of the system associaled with 4th particle from mh model at tth time step. Each model fm receives P copies of the same initial condition 1 where P denotes the number of particles assigned to model fm. The propagation of uncertainty starts with P M copics of the same initial conditions across {f M models and results in P * M mumber of trajectories. The diversity in these trajectories will come from two sources, i.e., M different models and their probabilistic predictions. These diverse trajectories will be used to derive prediction intervals. In contrast, HOPCAST used a set of errors (e.g., Eg E,)) to derive prediction intervals.
The rest of the section discusses the algorithmic details of three propagation approaches and the construction of prediction intervals using the outputs of three algorithms.
A.1 Expectation
The Expectation method always uses P= = 1 particle for each model fm- This is because the predicted uncertainty (E/tTt) is ignored, and the predicted mean (u(:m) is propagated to the next time step. Therefore, assigning more than one particle to a model fn will not generate diverse trajectories from that model. Even though this approach ignores predicted uncertainty (EF+1) during propagation, it will be used to generate prediction intervals explained in section A.3 The overall procedure of this approach is in Algorithm 2
Algorithm 2 Expectation
1: Input: Initial states
(s 1 models fi...., JM, timesteps T
2: for t = 0 to - do
for m 1 to M do 1.m Predict p I.im + fm fs! "m) S141 1.m
5:
1441
I.m
D Propagale mean
6: end for
7: {s m=1
D inputs to the next timestep
end for
return {5 1==
D the 1 M number of trajectories of length T
A.2 Trajectory Sampling
This method uses P particles for each model fm since it considers predicted uncertainty (E while propagating uncertainty the next step. The overall procedure of this algorithm is summarized Algorithm
16

---

7: end for end for Published in Transactions on Machine Learning Research (12/2025)
Algorithm 3 Trajectory Sampling
1M models f timesteps T, particles P Input: Initial states (p=1--pp=mm=
for t 0 to T do for = to M do
for p -= 1 to P do
5:
Predict p D..m < (s 1+1
6: Sample p.m.
pm 1+1 MVN 1+1 D(4))
considers predicted uncertainty
(( (1941))=11 LM
inputs to the next timestep
10: end for
11: return (( LP LM
jp=lm=m
D
the P M namber of trajectories of length T
A.3 Moment Matching
This method uses P particles for each model m since it considers predicted uncertainty (E)"") while prop- agating uncertainty to the next step. This is different from Trajectory Sampling in that it fits a Gaussian distribution to P M predictions (f )
4th timestep, that is, MVN(U141 E(11). We assume 1.M independence between all P M predictions. A set of P M samples (1s )p= p==1 p==m)= is taken from that distribution which becomes input at the next time step. The overall procedure is summarized in Algorithm
Algorithm Moment Matching
1: Input: Initial states {[sD:")) 1M
Ip=1/m=lt models fi,..., timesteps T, particles
2: for t = 0 to
do
for m 1 M do for p 1 to do Predict u.- 241 + (s Sample $441 MVN(MNFFF 411 D.my
p.m
7: end for
8: end for
9: 111111 p=1 m=1
P.MP 1M
> fit a Gaussian distribution
10: MLt P.M m== 1=
S41
Mean
11: Σ141 + P.M. 2m
M
p
(st+1
Me4 (s 12+1)² Variance
12: (( 18141pp11mm m= MVN E,+1)
p T.
D sample inputs for the next timestep
13: end for
14: return (1 LM (ps
D the P M number of trajectories of length T
Prediction intervals using uncertainty propagation All three approaches output P*M number of tra- jectorics length T {(s P 1 At any time step during propagation, the (PI,, are generated using
(
((s P } Specifically, we construct a MVN whose parameters are: pnet -1 MM Ep= spi and Enet 1 P.M Em=1 Ep= Ep.m + (sp.m Mmet) (sp:' (met) For IV system, the pnne (Maet: 1%
and Enet diag(o ). The prediction intervals for x and y are derived as follows:
Pl x M + x - a net
(13)
PI Yisi = waat + - a x
(14)
where a = for 2% prediction intervals. The - denotes the inverse cumulative distribution function of the Standard Normal distribution. In contrast, equations were used to generate prediction intervals with HOPCAST.
17

---

Published in Transactions on Machine Learning Research (12/2025)
B Dynamical Systems
There are five dynamical systems studied in this paper. One those is Lotka-Volteryy (LV) equations, which are already discussed section along with its closed form, parameters, and ranges of initial conditions. Here, we repeat. the description of the IV system and then describe the rest. of the systems. We add zero- mean additive Gaussian noise each differential equation post-integration as discussed in section 3 with the variance equal to the standard deviation of the variable modelled by the differential equation scaled by factor a.
B.1 Vatka-Volterra (Wangersky 1978
dr
0. fzy
(15)
dy
8ay 7M
(16)
Parameters: a= 1.1; 0.4; 0.4; 8 = 0.1
Initial Condition Ranges: x E [5, 20]; E [5, 10]
B.2 Lorenz (Brunton et al., 2016)
dz o (y x))
(17)
dt
dy x(p 2)
(18)
dt
dz
zy Bz
(19)
dt
Paramcters: 10; p= 28; 8
Initial Condition Ranges: x E [-20, 20}; E [-20, 20}; E [0,
B.3
FitzHugh-Nagumo (FHN) (Izhikevich & FitzHugh, 2006
dv dt dw
3
v
w +
(20)
3
bw)
(21)
Paramcters: 0.7:b = 0.8;c 0.08:7 = 0.5
Initial Condition Ranges: v [-1.5, 1.5] E [-1.5,1.3
B.4 Lorenz95 (Lorenz 1996
dXi
(X2 Xa)X, X1 F
(22)
dt
dX, (Xg X1)Xs - X2 +F
(23)
dl
dX3 (X4 X2xX - X3 F
(24)
dt
dX4 (Xs X3)X, - FF
(25)
dt
dXs (X1 X,)Xs - X5 F
(26)
dt
Parameters: F=8 Initial Condition Ranges: X; [-10.5, 10.5) where i E (1,2,3,4.5)
18

---

Published in Transactions on Machine Learning Research (12/2025)
B.5 Glycolytic Oscillator (Daniels & Nemenman, 2015
dSi
k1S10 S6
J0 + (Sa/ K2
(27)
df
dS²
by S1
2
12 Ss) Sg
(28)
df
1+ (Sb/ K2)
dSg
S - 8b) K3 S6)
(29)
dt
4Sp
( (A- Sb) - k4S45s - x($ ST)
(30)
1p
SSp 22 S (N- - 8s) - -4S4 K652Ss
(31)
dt
SSp
S1
1 + (So/K) K) + 2A3 S6) - Kg
(32)
di
dSp Ux((s - S7) -KS7
(33)
di
Paramcters: Jo = k. 100; 1: = K3 16; h44 100; k.
1.28; kb 12; 1.8; 13:9 4;
0.52; v 0.1; = A:
Initial Condition Ranges: S1 E 00.15, 1.60]: S2 E [0.19, 2.16]; S3 E [0.04,0.20)::5 E [0.10, 0.35|: Ss E
[0.08, 0.30): Sa E [0.14, 2.67]: S; E [0.05, 0.10)
C Hyperparameters
In Table we provide the Sequence Length (Sr. for each output of the Predictor and a number of models in Probabilistic Ensembles for each experimental setting in Table
Data Generation
Table 4: Details about data generation
Model
At
Timesteps
Trajectorios
Lotka Volterra
0.1
300
500
Lorenz
0.01
300
1000
FHN
0.5
400
350
Lorenz95
0.01
300
666
Glycolytic
0.01
400
750
E
Ablation Studies
E.1 Encoder Architecture
As discussed in section 4 each correction model utilizes an Encoder with MHN. The architecture of the En- coder (e.g., mz) is a fully connected feedforward network with one layer and 100 neurons for all experiments. Here, we show the impact of adding more layers of fully connected neurons the Encoder on the proposod metrics. Table shows that adding more layers doesn't significantly impact the MSE and CE. However, the MSE and CE increase considerably when the Eacoder is removed from the correction models.
19

---

Published in Transactions on Machine Learning Research (12/2025)
Table 3: Sequence lengths (St) for all outputs of Predictor and number of models in Probabilistic ensemble
across various dynamical systems and noise levels
Probabilistic Ensemble
Model
HopCast
Moment Matching
Trajectory
Expectation
Sampling
Hyperparameter
Sequence lengths
Models
Models
Models
0.05
30,30,50,60
5
Lotka Volterra
0.1
70,30,70,40
5
5
0.3
70,20,60,70
6
5
30,35,35,38 10,50,,50,1000000
6
8
Lorenz
0.1
500,500,500,3000 30,,000000,000000000
σ
0.3
25,25,25,,00,500,m
8
7
3
0.05
35,55
5
5
3
FHN
15,15
4
5
0.3
15,15
0.05
2200 for all
6
7
3
Lorenz95
2000 for all
6
5
0.3
2000 all
0.05
35,50,35, 10,,3,5,0
Glycolytic
0.1
6
7
3
Oscillator
50 for all
7 8
Table 5: Effect of different Encoding network architectures on the proposed metrics. The results without the Encoder also included. The FC((00), means the fully connected feodforward model with layer of 100 neurons. The results are reported over three runs of the same experiment random seeds.
Dynamical System
Lotka Volterra
Lorenz
Noise Level
0.3
0.3
Metrics
MSE
PI-
CE
MSE
P1-Width
CE
Withont Encoder
1907.07 + 9.33
38.69 + 0.42
0.06 + 0.005
16.12 + 0.36
3.37 + 0.23
0.025 + 0.012
FC(100);
1681 + 46.36
40.44 + 0.91
0.019 0.004
9.69 0.23
3.19 0.12
0.0051 0.004
FC(100)2
1717.362 + 30.3
41.84 + 0.68
0.014 + 0.002
9.43 0.23
3.08 0.09
0.006 + 0.002
FC(100)s
1710.18 + 46.19
42.45 + 0.55
0.018 + 0.002
9.37 + 0.007
2.98 + 0.07
0.0035 0.001
E.2 Initial condition and context state (c
As discussed section I our context state (cf') comprises the initial condition, the current state of the system, and the timestep ID. Here, we show the results on two systems without the initial condition a part of the context state in Table 0 The MSE increases significantly for both cases, while the CE exhibits minor changes. This shows the significance of making the initial condition part of the context state.
20

---

Published in Transactions on Machine Learning Research (12/2025)
Table 6: Comparison of proposed metrics with and without initial condition included the context state.
The results are averaged over three runs.
Dynamical System
Glycolytic Oscillator
Lotka Volterra
Noise Level
0.1
0.1
Metrics
MSE
PI-Width
CE
MSE
PI-Width
CE
With IC
0.072 + 0.003
0.25 + 0.007
0.018 + 0.006
6.10 + 0.20
1.88 + 0.05
0.0070 =: 0.002
Without IC
0.176 + 0.003
0.416 + 0.003
0.027 + 0.008
43.90 + 1.09
5.17 + 0.14
0.011 + 0.002
E.3 Number of Retrieved Samples (s)
To generate calibrated prediction intervals, set of samples is drawn from a! described in section 4.3 Here, we analyse the sensitivity of proposed metrics with varying values of {100, 500, 800, 1000) for two settings from Table [ The results everywhere else in the paper are reported with 1000. In Table it can be seen that the results are largely unchanged as the number of samples varies from 1000 to 500. However, at s = 100, we see a drop in performance in terms of MSE and CE, suggesting that too few samples limit the diversity of error samples, negatively impacting the performance.
Table 7: Comparison of proposed metrics with varying number of retrieved residual samples (s) from MHN memory. The results are averaged over three runs.
Dynamical System
Glycolytic Oscillator
Lotka Volterra
Noise Level
a= ==0.1 0.1
a = 0.1
Metrics
MSE
PI- Fi-ti
CE
MSE
PI- Widtt
CE
1000
0.072 + 0.003
0.25 + 0.007
0.018 + 0.006
6.10 + 0.20
1.88 + 0.05
0.0070 + 0.002
800
0.070 + 0.005
0.24 + 0.009
0.018 + 0.008
6.10 + 0.2
1.89 + 0.03
0.007 + 0.003
500
0.071 + 0.005
0.26 + 0.003
0.015 = 0.008
6.13 + 0.20
1.87 + 0.04
0.006 + 0.005 0.005
100
0.096 + 0.005
0.18 0.006
0.025 + 0.008
7.9 0.17
0.69 + 0.04
+ 0.019
E.4
Top-k retrieval
As discussed in section 4.3 the samples are drawn from at generate calibrated prediction intervals. Here, we analyze an alternative approach to sampling, i.c., using top-k probabilities from aς. Table & shows the results on proposed metrics with varying values of k {100, 500, 800, 1000). The results remain largely unchanged for k 2 500. At 100, the CE shows an increase for both settings, indicating that too few samples lead to decreased diversity in sampled errors.
E.5 Size of Association Memory
In section we discussed the inference with the correction model (M2). A set of K keys (Kmem) with the corresponding values (Vmem) is randomly sampled from S and s'rain respectively and loaded into the Association memory. Here, the impact of the size Association memory on the proposed metrics will be discussed. Table shows the variation in proposed metrics as the number of keys memory varies from 2000. For all systems, the MSE and CE improve significantly as the keys increase from 10 50. The MSE and CE that showed considerable change the number of keys increases shown in bold. As the
21

---

Published in Transactions on Machine Learning Research (12/2025)
Table 8: Comparison of proposed metrics based on top-k probabilities from af instead sampling. The results are reported for k {100, 500, 800, 1000). The results are averaged over three runs.
Dynamical
Glycolytic Oscillator
Lotka Volterra
System
Noise Level
σ 0.1
σ 0.1
Metrics
MSE
PI-Width
CE
MSE
PI-Width
CE
1000
0.67 + 0.002
0.24 + 0.007
0.011 + 0.005
7.31 + 0.99
2.38 + 0.33
0.016 + 0.003
800
0.067 + 0.002
0.24 + 0.01
0.011 + 0.005
7.52 + 1.18
2.41 + 0.38
0.014 + 0.004
500
0.069 + 0.003
0.24 + 0.009
0.017 = 0.005
8.39 + 2.50
2.53 + 0.64
0.017 0.008
100
0.07 + 0.004
0.20 + 0.005
0.06 + 0.03
6.36 + 0.22
1.24 + 0.11
0.25 + 0.036
keys increase beyond 50, not all systems' MSE and CE showed a considerable increase. The MSE and CE show insignificant changes as the number of keys increases beyond 100 until 2000. Thus, we fix the number of keys to 2000 for our final evaluation of results.
Table 9: The results on proposed metrics with varying numbers of keys in memory. The MSE and CE that showed considerable improvement with the increase in the number of keys in memory shown in bold. The results are averaged over three runs.
Glycolytic Oscillator
Lotka Volterra
Keys in Memory
Lorenz
FHN
Lorenz95
Metric
a 0.1
a 0.3
0.05
0.05
a
MSE
1765.48 1: 170.73
0.15 1 0.009
26.86 + 4.80
0.15 1 0.03
24.81 1: 3.11
PI-Width
34.59 5.69
0.37 + 0.006
3.61 1.06
0.24 + 0.03
6.59 0.69
CE
0.11 = 0.03
0.080 + 0.009
0.10 + 0.04
0.24 + 0.19
0.039 + 0.05
MSE
1419.29 +: 103.78
0.13 + 0.001
9.66 1.51
0.072 + 0.001
18.10 + 1.75
50
PI-Width
37.64 + 0.35
0.36 + 0.008
2.90 + 0.03
0.21 + 0.012
5.98 = 0.11
CE
0.025 + 0.009
0.012 + 0.002
0.038
0.003
0.12 + 0.02
0.016 0.003
MSE
1285.90 +: 30.9
0.12 + 0.002
7.76 0.67
0.076 0.001
18.37 = 0.51
100
PI-Width
36.83 + 0.39
0.36 + 0.015
2.99 + 0.06
0.21 + 0.01
6.09 + 0.15
CE
0.015 = 0.007
0.018 + 0.007
0.026
0.006
0.111 + 0.003
0.006 0.005
MSE
1249 + 11.03
0.12 + 0.001
6.89 + 0.30
0.076 + 0.002
17.06 + 0.86
500
PI-Width
38.19 + 1.39
0.36 + 0.001
2.85 + 0.08
0.21 = 0.006
6.02 = 0.10
CE
0.020 0.004
0.013 + 0.003
0.022 0.003
0.093 0.03
0.0021 + 0.003
MSE
1211.72 + 7.55
0.12 + 0.003
7.08 + 0.01
0.077 + 0.001
16.91 + 0.74
1000
PI-Width
38.37 1.91
0.37 + 0.005
2.91 0.093
0.21 + 0.001
6.13
CE
0.018 = 0.001
0.017 + 0.004
0.021 + 0.004
0.092 + 0.01
0.0020 = 0.002
MSE
1248.17 15
0.11 + 0.001
6.87 +: 0.32
0.076 1 0.002
16.44 + 0.66
2000
PI- Width
38.66 + 0.38
0.37 + 0.008
2.93 + 0.17
0.20 + 0.005
6.08 - 0.06
CE
0.019 + 0.03
0.016 + 0.003
0.022 + 0.004
0.091 + 0.004
0.0015 + 0.001
22

---

Published in Transactions on Machine Learning Research (12/2025)
E.6 Embedding dimension
As discussed in section 4.2 the Encoder outputs d-dimensional embedding given context state (cf), where d = 4. Here, want to show that the performance of correction models is insensitive to the embedding dimension provided that the S, is retuned accordingly. In Table 10] the results on metrics and S, for LV at o = 0.1 in the first row are copied from Tables | & 3 respectively. Table 10 shows that the CE shows a considerable increase from 0.0070 to 0.065 as we change the to 5 from 4, while the MSE shows minor changes. The last row shows the results at d = 5 when the St. is retuned, where the CE (i.e., 0.0098) is close to what we had earlier at d (i.e., 0.0070).
Table 10: The impact of change in embedding dimension on the proposed metrics. The last row shows the results when St is retuned for the new embedding dimension (d = 5). The results reported over three runs.
Lotka Volterra
Embedding Dimension (d)
S, of all outputs
0.1
MSE
PI-Width
CE
d 4
700,30,70,40
6.10 + 0.20
1.88 + 0.05
0.0070 0.002
d 5
770,30,70,40)
6.75 + 0.065
1.94 + 0.02
0.065 + 0.005
d = 5
55,,70,55
6.98 = 0.115
1.69 + 0.01
0.0098 + 0.003
E.7 Impact of shuffled data
As discussed in section 4.1 the dataset is shuffled before forming queries, keys, and values for training. Here, we show that this shuffling brings two major benefits. One, the MSE and CE show significant improvement. Second, CE and MSE show changes with respect to the number of keys in memory that are consistent across different systems.
Table 11 shows the results on the proposed metrics for FHN and LV with and without shuffling the data. The MSE and CE increase considerably for both systems when the data is not shuffled, irrespective of the number of keys in memory. For I.V, the MSE starts to decrease and CE starts to increase as we reduce the number of keys in memory from 2000 to 10. For FHN, in contrast, MSE and CE begin to grow as the number of keys goes down to 10 from 2000. On the other hand, the MSE and CE remain largely unchanged for both systems as we reduce the number of keys from 2000 to 500 when the data is shuflled. For keys less than 500, the MSE and CE show considerable change for both systems.
When the data shuffled, the changes in MSE and CE as we reduce the number of keys show a generalizable paitern for both systems. Hence, the guidelines regarding the number of keys in the memory at the inference time are generalizable.
E.8 Number of models in Probabilistic Ensembles
As discussed in section 7 the Probabilistic Ensembles tend to saturate uncertainty after certain number of models in the ensemble. Here, demonstrate that for all propagation approaches via calibration curves. As shown in Fig. M(a) for Expectation, the uncertainty goes up from overconfidence to underconfidence we increase the number of models in the Probabilistic Ensemble. The last. three increments in the number of models (13 to 15) don't impact uncertainty much. For Moment Matching (Fig. Bb)), the uncertainty increases with the number of models and saturates later while the model is still overconfident. For Trajectory Sampling (Fig. [(c)), increasing the number of models leads marginal change uncertainty and saturates after four models.
23

---

Published in Transactions on Machine Learning Research (12/2025)
Table 11: Performance on proposed metrics at different numbers of keys in memory, with and without data
shuffling. The results are averaged over three runs.
Lotka-Volterrs
FHN
Keys in Memory
Metric
0.05
0.05
Shuffled Data
Non-Shuffed Data
Shuffled Data
Non-Shuffled Data
MSE
26.86 + 4.80
33.09 + 0.009
0.15 + 0.03
0.16 0.01
10
PI-Width
3.61 1.06
0.54 0.001
0.24 1 0.03
0.17 0.005
CE
0.10 0.04
0.98 0.007
0.24 0.19
1.15 0.05
MSE
9.66 + 1.51
32.91 + 0.03
0.072 0.001
0.12 + 0.02
50
PI-Width
2.90 T 0.03
0.49 1 5.42
0.21 1 0 0.012
0.18 0.01
CE
0.038 0.03
0.74 0.005
0.12 0.02
0.38 0.06
MSE
7.76 0.67
32.63 1 0.09
0.076 - 0.001
0.10 T 0.02
100
PI-Width
2.99 0.06
0.51 0.001
0.21 0.01
0.16 0.01
CE
0.026 0.006
0.72 0.001
0.111 + 0.003
0.46 + 0.08
MSE
6.89 T 0.30
42.14 1 22.03
0.076 TT 0.002
0.12 T 0.03
500
PI-Width
2.85 0.08
3.32 0.09
0.21 0.006
0.18 0.009
CE
0.022 0.003
0.12 + 0.06
0.093 = 0.03
0.41 + 0.09
MSE
7.08 + 0.01
34.80 + 16.27
0.077 = 0.001
0.092 + 0.01
1000
PI-Width
2.91 + 0.093
3.36 0.07
0.21 + 0.001
0.27 0.006
CE
0.021 0.004
0.083 + 0.053
0.092 0.01
0.12 + 0.12
MSE
6.87 1 0.32
60.84 1: 11.17
0.076 1 0.002
0.091 0.013
2000
PI-Width
2.93 0.17
4.77
= 0.005
0.37 0.005
CE
0.022 + 0.004
0.063 + 0.015
0.091 + 0.004
0.16 + 0.042
Callbration Curve (Expectation)
Calibration Curve (Moment Matching)
Callbration Curve (Trajectory Sampling)
1.0
1.0
1.0
Observed Fraction
0.6
Observed
0.4
0.4
0.4
0.2
0.2
0.8
1.0
0.8
1.0
0.4
0.6
Fraction (b)
Expected Fraction
(c)
Figure The calibration curves of three propagation approaches for x-oulput of Lorenz (a 0.05) to delineate the saturation of uncertainty the number of models in ensemble goes up from 2 to 15.
E.9 Sequence length (s.) & Calibration
The section discusses the impact of varying S.. on the model's confidence for the y-oulput of Lorenz. Here, we show the same for all outputs of the two systems, iee, Lorenz and Glycolytic Oscillator. The calibrated confidence of both systems with their optimal S, shown the middle plot of Fig. Da & 9b To show
underconfidence and overconfidence, we pick small s, 10 and large St 1000. At small St 10. we expect all outputs of both systems to be overconfident as shown in the leftmost plots of Fig. 9a & 96 At
24

---

Published in Transactions on Machine Learning Research (12/2025)
large S1 1000, expect all outputs of both systems to be underconfident shown in the rightmost plots of Fig. 9a & 9
Tt shows that the concept of the increase in uncertainty with the increase in St. generalizes well across different outputs of different systems.
(HopCasti
(HopCast)
Calibration (HopCast)
1.0
10-
1.0
Diseevee www.eedeeeeettoeeooo
Obsupadd Fraccion
Obrerved
0.4
0.4
0.4
0.6
1.0
[10,10110100001100]
150,550,550,,300330,,
[1000;1000,1000 TO00,
(a) Lorenz (a= 0.1). The leftmost plot shows the overconfidence of all six outputs of Lorenz at small Sz. The middle plot shows the calibrated confidence for all outpuls at optimal Sy. The rightmost plot shows the underconsidence of all outputs at large St.
Calbrratin
2.0
1.0
Orrjct
Obsarved Frection
Observic Fraectar
Oirgent
0.6
0.6
Obzoneed
0.2
0.0
0.2
0.6
0.2
0.4
0.8
0.0
0.4
0.6
1.5
Expeccodd Fraction 110110100100,001100
Expected Praction
Expocted Froczion
[50,,00,50,505500
[1000,1000, 100000,000,1000000 1000, 1000,1000)
(b) Glycolytic Oscillator (a = 0.3). The leftmost plot shows the overconfidence of all seven outputs at small St. The middle plot shows the calibrated confidence for all outputs at optimal St. The rightmost plot shows the underconfidence of all outputs at large St.
Figure 9: Calibration curves of HOPCAST for two dynamical systems to demonstrate the impact of St on calibration. At the bottom of each plot, S₁ of all outputs are shown in ascending order such that the first value corresponds to the first output, sccond to the second output, and so on.
F
Implementation Details
This section contains details regarding the implementation of baselines and HOPCAST. All experiments were run on NVIDIA A100-SXM4-80GB. PyTorch used to implement baselines and HOPCAST.
F.1 Baselines Implementation
We train population of 15 models for the Probabilistic Ensembles. Regarding architecture, two layers of fully connected feedforward model with 400 neurons each were used for LV and FHN, and three layers with 400 neurons for Lorenz, Loren295, and Glycolytic Oscillator. The train/test datasets were prepared following an 80/20 split. The batch size, learning rale, optimizer, and epochs were kept the same for all experiments, and are 128, 0.001, Adam (Kingma & Ba 2017), and 1000, respectively. The early stopping was used as a criterion to train the Probabilistic Ensembles (Yao et al.] (007). The number of particles P for each model in Probabilistic Ensembles was 1 for Expoctation, and 20 for Moment Matching & Trajectory Sampling (Chua et al. 2018a).
25

---

Published in Transactions on Machine Learning Research (12/2025)
F.2 HopCast Implementation
The Encoder was fully connected leedforward model with one layer and 100 neurons. The section i contains details about tuning an important hyperparameter, i.e., St, of HOPCAST. Table 3 has S,, for each output of all systems at various noise scaling factors a. The learning rate of 0.001 and the optimizer AdamM Loshchilov & Hutter 2019) were kept the same for all experiments. The batch size and epochs were different for each experiment, and are provided the form of yml files as supplementary material along with other hyperparameterss for each system and noise scaling factor (o).
The deterministic Predictor was fully connected feedforward model of two layers with 400 neurons each for TV and FHN, and three layers with 400 neurons for Lorenz, Lorenz95, and Glyoolytic Oscillator.
26


Reviews:
[{"review_id": "v1tY7HP9cp", "reviewer": "Reviewer_hNjw", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper proposes an algorithm for calibrating confidence intervals in autoregressive dynamics models with many trajectories. The key to the proposed algorithm is the Modern Hopfield Networks (MHN), which provides samples of the error distribution of these dynamic models. Intensive experiments were conducted to validate the calibration of the proposed algorithm. Various aspects, such as the coverage rate and hyperparameter tuning.  The results seem promising, compared with the basic baselines.", "claims_and_evidence_yn": "Yes", "claims_explanation": "The formulation and simulation setups are solid and sufficiently validated.", "audience_yn": "Yes", "audience_explanation": "The use of MHN is interesting, and it can provide novel aspects of DNN.", "requested_changes": "- The role of MHN is a key factor in this paper. Therefore, the motivation, role, and detailed analysis of MHN should be required.\n- The prediction of $m$ is too short, which can affect the calibration performance. It requires a more intensive analysis of these aspects. \n- The effects of $ \\sigma$ and $L$ appear to be non-negligible. Can you provide any suggestions based on the data-driven approaches?", "broader_impact_concerns": "This study can affect the advancement of the explainable and accountable AI, which is crucial of the safety and trust AI.", "additional_comments": "I have some questions as follows:\n\n- The authors used only $(\\bar{x}_1^n, \\bar{y}_1^n , \\bar{x}_t^n , \\bar{y}_t^n , t).$ However, more timestamps can be used. Did you consider this or have any reason for your choice?\n- In Fig. 7, the fraction value is limited in the algorithm’s results. Can you provide a finer grid in the $x$-axis, especially near 0 or 1?\n- Do you have any idea for the time-series with one trajectory, common in the real world?", "extra_scores": {}, "extra_text": {}}, {"review_id": "6Ynu8oce0Q", "reviewer": "Reviewer_h5Ri", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper presents a method to calibrate multi-step learned dynamical systems. The original motivation comes from the fact that typical dynamical systems are trained with one step transitions, leading to multi-step errors; the paper proposes a method that given a trajectory dataset, trains multi-step calibration models that adjust for multi-step uncertainties. The paper improves over alternative methods that train dynamical models given a fixed dataset.", "claims_and_evidence_yn": "Yes", "claims_explanation": "The paper seems to present interesting evidence suggesting that the new method  leads to more calibrated dynamical systems compared to baselines. However, I think the solidity of the technical contributions can be made more solid and detailed.", "audience_yn": "Yes", "audience_explanation": "Since the paper concerns methods to calibrate for uncertainties in training world models, I am sure some TMLR audience will be interested in this work.", "requested_changes": "=== **problem description and dynamical systems 1-2**===\n\nI am slightly confused about the presentation of dynamical 1-2 as the main examples used in the paper. It is not made explicitly clear that the HopCast method is generally applicable to generic dynamical systems, or to systems specific to the form of 1-2. It does seem to me that the method is more general than motivated in Section 3. I'd suggest that the authors present the message in the more clear way, indicating explicitly how general the method is and what algorithmic assumptions and data assumptions are made in the first place. Dynamical system in the form of 1-2 are quite restrictive and I am not sure how interesting they are to general ML audience.\n\n=== **section 4**===\n\nIt seems to me that the paper can benefit from a more clear exposition of the content in Section 4, where to me the algorithmic idea and architectural designs (transformer specific) are blended together in a confusing way. Specifically the training procedure in Sec 4.2 does not seem to be directly dependent on the assumption of training a transformer model, instead one can also leverage RNN style state space models. It is not clear to me how much of the algorithmic developments depend on the assumption of using transformers and why is it necessary. Do you think transformer is somehow more advantageous than RNN? On the set of tasks presented in the paper, I am not sure significant difference will be incurred if you repeat the experiments with RNN.\n\n=== **comparison to baselines**===\n\nThough it is definitely interesting to see promising results such as in Fig 5 where true trajectories are within the CI of the prediction, as well as the fact that in Fig 6 the proposed method is more calibrated. Overall I find the paper to lack technical solidity in its comparison against reasonable baselines. After all, calibrated prediction through posterior prediction is well studied in the probabilistic inference literature, and there does not seem to be properly developed baselines compared against in this paper. This puts into question how much gains come from just applying basic posterior inference techniques to the prediction problem itself, compared to HopCast which is designed specifically for this setup (in combination with transformer, a specific architecture).\n\nIt is also not clear to me that the method is scalable to higher dimensional more complex setups. More experiments in the context of control-based deep RL environments are very appreciated in this case, rather than low-d ODE simulations.\n\nThe paper also lacks ablations in the designs: e.g., why transformer? why not other architectures, etc.", "broader_impact_concerns": "NA", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "TrXCvaR0ep", "reviewer": "Reviewer_BLxi", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper proposes HopCast, a predictor–corrector framework for multi-step forecasting in dynamical systems. A standard autoregressive predictor produces point forecasts, while a learned corrector retrieves context-conditioned historical residuals using an encoder with a Modern Hopfield Network (MHN). The retrieved residuals both (i) correct the point prediction and (ii) yield calibrated prediction intervals (PIs) without explicit uncertainty propagation. A single hyperparameter (“Sequence Length”, SL) controls an effective attention span that trades off sharpness vs. coverage. Experiments on several synthetic ODE systems show improved calibration error with competitive MSE versus ensemble-based baselines and alternative uncertainty propagation schemes. Ablations study memory size, encoder depth, embedding dimension, initial-condition context, and SL.", "claims_and_evidence_yn": "Yes", "claims_explanation": "- Clear, modular idea: using an associative memory (MHN) to retrieve residuals conditioned on state, initial condition, and time is simple and broadly applicable to autoregressive models.\n- Strong focus on calibration: reports calibration error across multiple PI levels, PI width (sharpness), and reliability diagrams; demonstrates a practical knob (SL) to tune the sharpness–coverage tradeoff.\n- Useful ablations: thoughtful analysis of architectural/memory choices and the role of initial-condition context; results are interpretable and generally consistent.\n- Reasonable path to practice: can be attached to existing predictors without modifying the forecasting backbone.", "audience_yn": "Yes", "audience_explanation": "The paper is related to dynamical systems and sequential learning, which has a broad audience in TMLR.", "requested_changes": "*1) Questions for the authors*\n- Can the corrector be made multivariate to capture cross-state error covariance? Any preliminary results?\n- How sensitive are calibration and MSE to the number of retrieved residual samples and to using top-k or temperature-scaled association weights?\n- Could SL (or a temperature on the association distribution) be learned/tuned automatically by minimizing validation calibration error?\n- How does HopCast perform under irregular time steps or when tested at Δt different from training?\n- What happens under OOD initial conditions or perturbed system parameters (e.g., Lorenz ρ)? Any robustness curves?\n- What is the runtime and memory footprint vs. ensemble trajectory sampling, for matched accuracy/calibration?\n\n*2) Suggestions for improvement / additional experiments*\n- Implement a joint multivariate corrector (vector-valued residuals) and report CE/MSE and reliability curves vs. the per-dimension approach.\n- Add multi-step conformal baselines (rolling residual, naive + recalibration, residual-based with covariates) and a quantile/CRPS-trained sequence model; include lightweight Bayesian last-layer or Laplace/SWAG baselines.\n- Automate SL selection (e.g., grid/Bayesian search minimizing validation CE, or learn a softmax temperature as a differentiable knob).\n- Analyze sensitivity to residual sampling policy (s, top-k, temperature).\n- Test irregular/variable-step scenarios and generalization across Δt.\n\n*3) Minor comments / typos*\n- Standardize capitalization (e.g., underconfidence/overconfidence) and hyphenation.\n- Ensure consistent notation for vectors/scalars (boldface or arrow).\n- Consider moving some algorithmic details and full configs to the supplement to tighten the main text.", "broader_impact_concerns": "No", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Decision: Accept (poster)