SlowGuess/NeurIPS_2023_OCR huggingface dataset:
François Rozet
University of Liege
francois.rozet@uliege.be
Gilles Loupe
g.louppe@uliege.be
Data assimilation, in its most comprehensive form, addresses the Bayesian inverse problem of identifying plausible state trajectories that explain noisy or incomplete observations of stochastic dynamical systems. Various approaches have been proposed to solve this problem, including particle-based and variational methods. However, most algorithms depend on the transition dynamics for inference, which becomes intractable for long time horizons or for high-dimensional systems with complex dynamics, such as oceans or atmospheres. In this work, we introduce score-based data assimilation for trajectory inference. We learn a score-based generative model of state trajectories based on the key insight that the score of an arbitrarily long trajectory can be decomposed into a series of scores over short segments. After training, inference is carried out using the score model, in a non-autoregressive manner by generating all states simultaneously. Quite distinctively, we decouple the observation model from the training procedure and use it only at inference to guide the generative process, which enables a wide range of zero-shot observation scenarios. We present theoretical and empirical evidence supporting the effectiveness of our method.
Data assimilation (DA) [1-9] is at the core of many scientific domains concerned with the study of complex dynamical systems such as atmospheres, oceans or climates. The purpose of DA is to infer the state of a system evolving over time based on various sources of imperfect information, including sparse, intermittent, and noisy observations.
Formally, let  $x_{1:L} = (x_1,x_2,\ldots ,x_L)\in \mathbb{R}^{L\times D}$  denote a trajectory of states in a discrete-time stochastic dynamical system and  $p(x_{i + 1}\mid x_i)$  be the transition dynamics from state  $x_{i}$  to state  $x_{i + 1}$ . An observation  $y\in \mathbb{R}^M$  of the state trajectory  $x_{1:L}$  follows an observation process  $p(y\mid x_{1:L})$ , generally formulated as  $y = \mathcal{A}(x_{1:L}) + \eta$ , where the measurement function  $\mathcal{A}:\mathbb{R}^{L\times D}\mapsto \mathbb{R}^{M}$  is often non-linear and the observational error  $\eta \in \mathbb{R}^{M}$  is a stochastic additive term that accounts for instrumental noise and systematic uncertainties. In this framework, the goal of DA is to solve the inverse problem of inferring plausible trajectories  $x_{1:L}$  given an observation  $y$ , that is, to estimate the trajectory posterior
$$
p \left(x _ {1: L} \mid y\right) = \frac {p (y \mid x _ {1 : L})}{p (y)} p \left(x _ {1}\right) \prod_ {i = 1} ^ {L - 1} p \left(x _ {i + 1} \mid x _ {i}\right) \tag {1}
$$
where the initial state prior  $p(x_{1})$  is commonly referred to as background [5-9]. In geosciences, the amount of data available is generally insufficient to recover the full state of the system from the observation alone [8]. For this reason, the physical model underlying the transition dynamics is of paramount importance to fill in spatial and temporal gaps in the observation.
State-of-the-art approaches to data assimilation are based on variational assimilation [1, 2, 5-7]. Many of these approaches formulate the task as a maximum-a-posteriori (MAP) estimation problem and solve it by maximizing the log-posterior density  $\log p(x_{1:L} \mid y)$  via gradient ascent. Although this
approach only produces a point estimate of the trajectory posterior, its cost can already be substantial for problems of the size and complexity of geophysical systems, since it requires differentiating through the physical model. The amount of data that can be assimilated is therefore restricted because of computational limitations. For example, only a small volume of the available satellite data is exploited for operational forecasts and yet, even with these restrictions, data assimilation accounts for a significant fraction of the computational cost for modern numerical weather prediction [10, 11]. Recent work has shown that deep learning can be used in a variety of ways to improve the computational efficiency of data assimilation, increase the reconstruction performance by estimating unresolved scales after data assimilation, or integrate multiple sources of observations [12-19].
Contributions In this work, we propose a novel approach to data assimilation based on score-based generative models. Leveraging the Markovian structure of dynamical systems, we train a score network from short segments of trajectories which is then capable of generating physically consistent and arbitrarily-long state trajectories. The observation model is decoupled from the score network and used only during assimilation to guide the generative process, which allows for a wide range of zero-shot observation scenarios. Our approach provides an accurate approximation of the whole trajectory posterior - it is not limited to point estimates - without simulating or differentiating through the physical model. The code for all experiments is made available at https://github.com/francois-rozet/sda.
Score-based generative models have recently shown remarkable capabilities, powering many of the latest advances in image, video or audio generation [20-27]. In this section, we review score-based generative models and outline how they can be used for solving inverse problems.
Continuous-time score-based generative models Adapting the formulation of Song et al. [28], samples  $x \in \mathbb{R}^D$  from a distribution  $p(x)$  are progressively perturbed through a continuous-time diffusion process expressed as a linear stochastic differential equation (SDE)
$$
\mathrm {d} x (t) = f (t) x (t) \mathrm {d} t + g (t) \mathrm {d} w (t) \tag {2}
$$
where  $f(t) \in \mathbb{R}$  is the drift coefficient,  $g(t) \in \mathbb{R}$  is the diffusion coefficient,  $w(t) \in \mathbb{R}^D$  denotes a Wiener process (standard Brownian motion) and  $x(t) \in \mathbb{R}^D$  is the perturbed sample at time  $t \in [0,1]$ . Because the SDE is linear with respect to  $x(t)$ , the perturbation kernel from  $x$  to  $x(t)$  is Gaussian and takes the form
$$
p (x (t) \mid x) = \mathcal {N} (x (t) \mid \mu (t) x, \Sigma (t)) \tag {3}
$$
where  $\mu (t)$  and  $\Sigma (t) = \sigma (t)^{2}I$  can be derived analytically from  $f(t)$  and  $g(t)$  [29, 30]. Denoting  $p(x(t))$  the marginal distribution of  $x(t)$ , we impose that  $\mu (0) = 1$  and  $\sigma (0)\ll 1$ , such that  $p(x(0))\approx p(x)$ , and we chose the coefficients  $f(t)$  and  $g(t)$  such that the influence of the initial sample  $x$  on the final perturbed sample  $x(1)$  is negligible with respect to the noise level - that is,  $p(x(1))\approx \mathcal{N}(0,\Sigma (1))$ . The variance exploding (VE) and variance preserving (VP) SDEs [28, 31, 32] are widespread examples satisfying these constraints.
Crucially, the time reversal of the forward SDE (2) is given by a reverse SDE [28, 33]
$$
\mathrm {d} x (t) = \left[ f (t) x (t) - g (t) ^ {2} \nabla_ {x (t)} \log p (x (t)) \right] \mathrm {d} t + g (t) \mathrm {d} w (t). \tag {4}
$$
That is, we can draw noise samples  $x(1) \sim \mathcal{N}(0, \Sigma(1))$  and gradually remove the noise therein to obtain data samples  $x(0) \sim p(x(0))$  by simulating the reverse SDE from  $t = 1$  to 0. This requires access to the quantity  $\nabla_{x(t)} \log p(x(t))$  known as the score of  $p(x(t))$ .
Denoising score matching In practice, the score  $\nabla_{x(t)}\log p(x(t))$  is approximated by a neural network  $s_\phi (x(t),t)$ , named the score network, which is trained to solve the denoising score matching objective [28, 34, 35]
$$
\arg \min  _ {\phi} \mathbb {E} _ {p (x) p (t) p (x (t) | x)} \left[ \sigma (t) ^ {2} \| s _ {\phi} (x (t), t) - \nabla_ {x (t)} \log p (x (t) \mid x) \| _ {2} ^ {2} \right] \tag {5}
$$
where  $p(t) = \mathcal{U}(0,1)$ . The theory of denoising score matching ensures that  $s_{\phi}(x(t),t)\approx \nabla_{x(t)}\log p(x(t))$  for a sufficiently expressive score network. After training, the score network
is plugged into the reverse SDE (4), which is then simulated using an appropriate discretization scheme [28, 30, 36, 37].
In practice, the high variance of  $\nabla_{x(t)}\log p(x(t)\mid x)$  near  $t = 0$  makes the optimization of (5) unstable [30]. To mitigate this issue, a slightly different parameterization  $\epsilon_{\phi}(x(t),t) = -\sigma (t)s_{\phi}(x(t),t)$  of the score network is often used, which leads to the otherwise equivalent objective [30, 32, 36]
$$
\arg \min  _ {\phi} \mathbb {E} _ {p (x) p (t) p (\epsilon)} \left[ \| \epsilon_ {\phi} (\mu (t) x + \sigma (t) \epsilon , t) - \epsilon \| _ {2} ^ {2} \right] \tag {6}
$$
where  $p(\epsilon) = \mathcal{N}(0,I)$ . In the following, we keep the score network notation  $s_{\phi}(x(t),t)$  for convenience, even though we adopt the parameterization  $\epsilon_{\phi}(x(t),t)$  and its objective for our experiments.
Zero-shot inverse problems With score-based generative models, we can generate samples from the unconditional distribution  $p(x(0))\approx p(x)$ . To solve inverse problems, however, we need to sample from the posterior distribution  $p(x\mid y)$ . This could be accomplished by training a conditional score network  $s_{\phi}(x(t),t\mid y)$  to approximate the posterior score  $\nabla_{x(t)}\log p(x(t)\mid y)$  and plugging it into the reverse SDE (4). However, this would require data pairs  $(x,y)$  during training and one would need to retrain a new score network each time the observation process  $p(y\mid x)$  changes. Instead, many have observed [28, 38-41] that the posterior score can be decomposed into two terms thanks to Bayes' rule
$$
\nabla_ {x (t)} \log p (x (t) \mid y) = \nabla_ {x (t)} \log p (x (t)) + \nabla_ {x (t)} \log p (y \mid x (t)). \tag {7}
$$
Since the prior score  $\nabla_{x(t)}\log p(x(t))$  can be approximated with a single score network, the remaining task is to estimate the likelihood score  $\nabla_{x(t)}\log p(y\mid x(t))$ . Assuming a differentiable measurement function  $\mathcal{A}$  and a Gaussian observation process  $p(y\mid x) = \mathcal{N}(y\mid \mathcal{A}(x),\Sigma_y)$ , Chung et al. [41] propose the approximation
$$
p (y \mid x (t)) = \int p (y \mid x) p (x \mid x (t)) d x \approx \mathcal {N} (y \mid \mathcal {A} (\hat {x} (x (t))), \Sigma_ {y}) \tag {8}
$$
where the mean  $\hat{x} (x(t)) = \mathbb{E}_{p(x|x(t))}[x]$  is given by Tweedie's formula [42, 43]
$$
\begin{array}{l} \mathbb {E} _ {p (x \mid x (t))} [ x ] = \frac {x (t) + \sigma (t) ^ {2} \nabla_ {x (t)} \log p (x (t))}{\mu (t)} (9) \\ \approx \frac {x (t) + \sigma (t) ^ {2} s _ {\phi} (x (t) , t)}{\mu (t)}. (10) \\ \end{array}
$$
As the log-likelihood of a multivariate Gaussian is known analytically and  $s_{\phi}(x(t), t)$  is differentiable, we can compute the likelihood score  $\nabla_{x(t)} \log p(y \mid x(t))$  with this approximation in zero-shot, that is, without training any other network than  $s_{\phi}(x(t), t)$ .
3 Score-based data assimilation
Coming back to our initial inference problem, we want to approximate the trajectory posterior  $p(x_{1:L} \mid y)$  of a dynamical system. To do so with score-based generative modeling, we need to estimate the posterior score  $\nabla_{x_{1:L}(t)} \log p(x_{1:L}(t) \mid y)$ , which we choose to decompose into prior and likelihood terms, as in (7), to enable a wide range of zero-shot observation scenarios.
In typical data assimilation settings, the high-dimensionality of each state  $x_{i}$  (e.g. the state of atmospheres or oceans) combined with potentially long trajectories would require an impractically large score network  $s_{\phi}(x_{1:L}(t), t)$  to estimate the prior score  $\nabla_{x_{1:L}(t)} \log p(x_{1:L}(t))$  and a proportional amount of data for training, which could be prohibitive if data is scarce or if the physical model is expensive to simulate. To overcome this challenge, we leverage the Markovian structure of dynamical systems to approximate the prior score with a series of local scores, which are easier to learn, as explained in Section 3.1. In Section 3.2, we build upon diffusion posterior sampling (DPS) [41] to propose a new approximation for the likelihood score  $\nabla_{x_{1:L}(t)} \log p(y \mid x_{1:L}(t))$ , which we find more appropriate for posterior inference. Finally, in Section 3.3, we describe our sampling procedure inspired from predictor-corrector sampling [28]. Our main contribution, named score-based data assimilation (SDA), is the combination of these three components.
Figure 1: Trajectories  $x_{1:L}$  of a dynamical system are transformed to noise via a diffusion process. Reversing this process generates new trajectories, but requires the score of  $p(x_{1:L}(t))$ . We approximate it by combining the outputs of a score network over subsegments of  $x_{1:L}(t)$ .
3.1 How is your blanket?
Given a set of random variables  $x_{1:L} = \{x_1, x_2, \ldots, x_L\}$ , it is sometimes possible to find a small Markov blanket  $x_{b_i} \subseteq x_{\neq i}$  such that  $p(x_i \mid x_{\neq i}) = p(x_i \mid x_{b_i})$  for each element  $x_i$  using our knowledge of the set's structure. It follows that each element  $\nabla_{x_i} \log p(x_{1:L})$  of the full score  $\nabla_{x_{1:L}} \log p(x_{1:L})$  can be determined locally, that is, only using its blanket;
$$
\begin{array}{l} \nabla_ {x _ {i}} \log p \left(x _ {1: L}\right) = \nabla_ {x _ {i}} \log p \left(x _ {i} \mid x _ {\neq i}\right) + \nabla_ {x _ {i}} \log p \left(x _ {\neq i}\right) (11) \\ = \nabla_ {x _ {i}} \log p \left(x _ {i} \mid x _ {b _ {i}}\right) + \nabla_ {x _ {i}} \log p \left(x _ {b _ {i}}\right) = \nabla_ {x _ {i}} \log p \left(x _ {i}, x _ {b _ {i}}\right). (12) \\ \end{array}
$$
This property generally does not hold for the diffusion-perturbed set  $x_{1:L}(t)$  as there is no guarantee that  $x_{b_i}(t)$  is a Markov blanket of the element  $x_i(t)$ . However, there exists a set of indices  $\bar{b}_i \supseteq b_i$  such that
$$
\nabla_ {x _ {i} (t)} \log p \left(x _ {1: L} (t)\right) \approx \nabla_ {x _ {i} (t)} \log p \left(x _ {i} (t), x _ {\bar {b} _ {i}} (t)\right) \tag {13}
$$
is a good approximation for all  $t \in [0,1]$ . That is,  $x_{\bar{b}_i}(t)$  is a "pseudo" Markov blanket of  $x_i(t)$ . In the worst case,  $\bar{b}_i$  contains all indices except  $i$ , but we argue that, for some structures, there is a set  $\bar{b}_i$  not much larger than  $b_i$  that satisfies (13). Our rationale is that, since we impose the initial noise to be negligible, we know that  $x_{b_i}(t)$  becomes indistinguishable from  $x_{b_i}$  as  $t$  approaches 0. Furthermore, as  $t$  grows and noise accumulates, the mutual information between elements  $x_i(t)$  and  $x_j(t)$  decreases to finally reach 0 when  $t = 1$ . Hence, even if  $\bar{b}_i = b_i$ , the pseudo-blanket approximation (13) already holds near  $t = 0$  and  $t = 1$ . In between, even though the approximation remains unbiased (see Appendix A), the structure of the set becomes decisive. If it is known and present enough regularities/symmetries, (13) could and should be exploited within the architecture of the score network  $s_\phi(x_{1:L}(t), t)$ .
In the case of dynamical systems, the set  $x_{1:L}$  is by definition a first-order Markov chain and the minimal Markov blanket of an element  $x_i$  is  $x_{b_i} = \{x_{i-1}, x_{i+1}\}$ . For the perturbed element  $x_i(t)$ , the pseudo-blanket  $x_{\bar{b}_i}(t)$  can take the form of a window surrounding  $x_i(t)$ , that is  $\bar{b}_i = \{i-k,\dots,i+k\} \setminus \{i\}$  with  $k \geq 1$ . The value of  $k$  is dependent on the problem, but we argue, supported by our experiments, that it is generally much smaller than the chain's length  $L$ . Hence, a fully convolutional neural network (FCNN) with a narrow receptive field is well suited to the task, and any long-range capabilities would be wasted resources. Importantly, if the receptive field is  $2k+1$ , the network can be trained on segments  $x_{i-k:i+k}$  instead of the full chain  $x_{1:L}$ , thereby drastically reducing training costs. More generally, we can train a local score network (see Algorithm 1)
$$
s _ {\phi} \left(x _ {i - k: i + k} (t), t\right) \approx \nabla_ {x _ {i - k: i + k} (t)} \log p \left(x _ {i - k: i + k} (t)\right) \tag {14}
$$
such that its  $k + 1$ -th element approximates the score of the  $i$ -th state  $\nabla_{x_i(t)}\log p(x_{1:L}(t))$ . We also have that the  $k$  first elements of  $s_{\phi}(x_{1:2k + 1}(t),t)$  approximate the score of the  $k$  first states  $\nabla_{x_{1:k}(t)}\log p(x_{1:L}(t))$  and the  $k$  last elements of  $s_{\phi}(x_{L - 2k:L}(t),t)$  approximate the score of the  $k$  last states  $\nabla_{x_{L - k:L}(t)}\log p(x_{1:L}(t))$ . Hence, we can apply the local score network on all sub-segments  $x_{i - k:i + k}(t)$  of  $x_{1:L}(t)$ , similar to a convolution kernel, and combine the outputs (see Algorithm 2) to get an approximation of the full score  $\nabla_{x_{1:L}(t)}\log p(x_{1:L}(t))$ . Note that we can either condition the score network with  $i$  or assume the statistical stationarity of the chain, that is  $p(x_i) = p(x_{i + 1})$ .
Algorithm 1 Training  $\epsilon_{\phi}(x_{i - k:i + k}(t),t)$
1 for  $i = 1$  to  $N$  do
2  $x_{1:L}\sim p(x_{1:L})$
3  $i\sim \mathcal{U}(\{k + 1,\ldots ,L - k\})$
4  $t\sim \mathcal{U}(0,1),\epsilon \sim \mathcal{N}(0,I)$
5  $x_{i - k:i + k}(t)\gets \mu (t)x_{i - k:i + k} + \sigma (t)\epsilon$
6  $\ell \gets \| \epsilon_{\phi}(x_{i - k:i + k}(t),t) - \epsilon \| _2^2$
7  $\phi \gets \mathrm{GRADIENTDESCENT}(\phi, \nabla_{\phi}\ell)$
Algorithm 2 Composing  $s_{\phi}(x_{i - k:i + k}(t),t)$
1 function  $s_{\phi}(x_{1:L}(t), t)$
2  $s_{1:k + 1}\gets s_{\phi}(x_{1:2k + 1}(t),t)[:k + 1]$
3 for  $i = k + 2$  to  $L - k - 1$  do
4  $s_i\gets s_\phi (x_{i - k:i + k}(t),t)[k + 1]$
5  $s_{L - k:L}\gets s_{\phi}(x_{L - 2k:L}(t),t)[k + 1:]$
6 return  $s_{1:L}$
3.2 Stable likelihood score
Due to approximation and numerical errors in  $\hat{x} (x(t))$ , computing the score  $\nabla_{x(t)}\log p(y\mid x(t))$  with the likelihood approximation (8) is very unstable, especially in the low signal-to-noise regime, that is when  $\sigma (t)\gg \mu (t)$ . This incites Chung et al. [41] to replace the covariance  $\Sigma_{y}$  by the identity  $I$  and rescale the likelihood score with respect to  $\| y - \mathcal{A}(\hat{x} (x(t)))\|$  to stabilize the sampling process. These modifications introduce a significant error in the approximation as they greatly affect the norm of the likelihood score.
We argue that the instability is due to (8) being only exact if the variance of  $p(x \mid x(t))$  is null or negligible, which is not the case when  $t > 0$ . Instead, Adam et al. [40] and Meng et al. [44] approximate the covariance of  $p(x \mid x(t))$  with  $\Sigma(t) / \mu(t)^2$ , which is valid as long as the prior  $p(x)$  is Gaussian with a large diagonal covariance  $\Sigma_x$ . We motivate in Appendix B the more general covariance approximation  $\sigma(t)^2 / \mu(t)^2\Gamma$ , where the matrix  $\Gamma$  depends on the eigendecomposition of  $\Sigma_x$ . Then, taking inspiration from the extended Kalman filter, we approximate the perturbed likelihood as
$$
p (y \mid x (t)) \approx \mathcal {N} \left(y \mid \mathcal {A} (\hat {x} (x (t))), \Sigma_ {y} + \frac {\sigma (t) ^ {2}}{\mu (t) ^ {2}} A \Gamma A ^ {T}\right) \tag {15}
$$
where  $A = \partial_x\mathcal{A}\mid_{\hat{x} (x(t))}$  is the Jacobian of  $\mathcal{A}$ . In practice, to simplify the approximation, the term  $A\Gamma A^T$  can often be replaced by a constant (diagonal) matrix. We find that computing the likelihood score  $\nabla_{x(t)}\log p(y\mid x(t))$  with this new approximation (see Algorithm 3) is stable enough that rescaling it or ignoring  $\Sigma_y$  is unnecessary.
3.3 Predictor-Corrector sampling
To simulate the reverse SDE, we adopt the exponential integrator (EI) discretization scheme introduced by Zhang et al. [30]
$$
x (t - \Delta t) \leftarrow \frac {\mu (t - \Delta t)}{\mu (t)} x (t) + \left(\frac {\mu (t - \Delta t)}{\mu (t)} - \frac {\sigma (t - \Delta t)}{\sigma (t)}\right) \sigma (t) ^ {2} s _ {\phi} (x (t), t) \tag {16}
$$
which coincides with the deterministic DDIM [36] sampling algorithm when the variance preserving SDE [32] is used. However, as we approximate both the prior and likelihood scores, errors accumulate along the simulation and cause it to diverge, leading to low-quality samples. To prevent errors from accumulating, we perform (see Algorithm 4) a few steps of Langevin Monte Carlo (LMC) [45, 46]
$$
x (t) \leftarrow x (t) + \delta s _ {\phi} (x (t), t) + \sqrt {2 \delta} \epsilon \tag {17}
$$
where  $\epsilon \sim \mathcal{N}(0, I)$ , between each step of the discretized reverse SDE (16). In the limit of an infinite number of LMC steps with a sufficiently small step size  $\delta \in \mathbb{R}_+$ , simulated samples are guaranteed to follow the distribution implicitly defined by our approximation of the posterior score at each time  $t$ , meaning that the errors introduced by the pseudo-blanket (13) and likelihood (15) approximations do not accumulate. In practice, we find that few LMC steps are necessary. Song et al. [28] introduced a similar strategy, named predictor-corrector (PC) sampling, to correct the errors introduced by the discretization of the reverse SDE.
We demonstrate the effectiveness of score-based data assimilation on two chaotic dynamical systems: the Lorenz 1963 [47] and Kolmogorov flow [48] systems. The former is a simplified mathematical model for atmospheric convection. Its low dimensionality enables posterior inference using classical sequential Monte Carlo methods [49, 50] such as the bootstrap particle filter [51]. This allows us to compare objectively our posterior approximations against the ground-truth posterior. The second system considers the state of a two-dimensional turbulent fluid subject to Kolmogorov forcing [48]. The evolution of the fluid is modeled by the Navier-Stokes equations, the same equations that underlie the models of oceans and atmospheres. This task provides a good understanding of how SDA would perform in typical data assimilation applications, although our analysis is primarily qualitative due to the unavailability of reliable assessment tools for systems of this scale.
For both systems, we employ as diffusion process the variance preserving SDE with a cosine schedule [52], that is  $\mu(t) = \cos(\omega t)^2$  with  $\omega = \arccos \sqrt{10^{-3}}$  and  $\sigma(t) = \sqrt{1 - \mu(t)^2}$ . The score networks are trained once and then evaluated under various observation scenarios. Unless specified otherwise, we estimate the posterior score according to Algorithm 3 with  $\Gamma = 10^{-2}I$  and simulate the reverse SDE (4) according to Algorithm 4 in 256 evenly spaced discretization steps.
4.1 Lorenz 1963
The state  $x = (a,b,c) \in \mathbb{R}^3$  of the Lorenz system evolves according to a system of ordinary differential equations
$$
\dot {a} = \sigma (b - a)
$$
$$
\dot {b} = a (\rho - c) - b \tag {18}
$$
$$
\dot {c} = a b - \beta c
$$
where  $\sigma = 10$ ,  $\rho = 28$  and  $\beta = \frac{8}{3}$  are parameters for which the system exhibits a chaotic behavior. We denote  $\tilde{a}$  and  $\tilde{c}$  the standardized (zero mean and unit variance) versions of  $a$  and  $c$ , respectively. As our approach assumes a discrete-time stochastic dynamical system, we consider a transition process of the form  $x_{i+1} = \mathcal{M}(x_i) + \eta$ , where  $\mathcal{M}: \mathbb{R}^3 \mapsto \mathbb{R}^3$  is the integration of the differential equations (18) for  $\Delta = 0.025$  time units and  $\eta \sim \mathcal{N}(0, \Delta I)$  represents Brownian noise.