SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Nathanael Bosch
Tübingen AI Center, University of Tübingen
nathanael.bosch@uni-tuebingen.de
Philipp Hennig
philipp.hennig@uni-tuebingen.de
Filip Tronarp
filip.tronarp@matstat.lu.se
Probabilistic solvers provide a flexible and efficient framework for simulation, uncertainty quantification, and inference in dynamical systems. However, like standard solvers, they suffer performance penalties for certain stiff systems, where small steps are required not for reasons of numerical accuracy but for the sake of stability. This issue is greatly alleviated in semi-linear problems by the probabilistic exponential integrators developed in this paper. By including the fast, linear dynamics in the prior, we arrive at a class of probabilistic integrators with favorable properties. Namely, they are proven to be L-stable, and in a certain case reduce to a classic exponential integrator—with the added benefit of providing a probabilistic account of the numerical error. The method is also generalized to arbitrary non-linear systems by imposing piece-wise semi-linearity on the prior via Jacobians of the vector field at the previous estimates, resulting in probabilistic exponential Rosenbrock methods. We evaluate the proposed methods on multiple stiff differential equations and demonstrate their improved stability and efficiency over established probabilistic solvers. The present contribution thus expands the range of problems that can be effectively tackled within probabilistic numerics.
Dynamical systems appear throughout science and engineering, and their accurate and efficient simulation is a key component in many scientific problems. There has also been a surge of interest in the intersection with machine learning, both regarding the usage of machine learning methods to model and solve differential equations [36, 18, 35], and in a dynamical systems perspective on machine learning methods themselves [8, 5]. This paper focuses on the numerical simulation of dynamical systems within the framework of probabilistic numerics, which treats the numerical solvers themselves as probabilistic inference methods [11, 12, 33]. In particular, we expand the range of problems that can be tackled within this framework and introduce a new class of stable probabilistic numerical methods for stiff ordinary differential equations (ODEs).
Stiff equations are problems for which certain implicit methods perform much better than explicit ones [10]. But implicit methods come with increased computational complexity per step, as they typically require solving a system of equations. Exponential integrators are an alternative class of methods for efficiently solving large stiff problems [48, 16, 7, 15]. They are based on the observation that, if the ODE has a semi-linear structure, the linear part can be solved exactly and only the non-linear part needs to be numerically approximated. The resulting methods are formulated in an explicit manner and do not require solving a system of equations, while achieving similar or better stability than implicit methods. However, such methods have not yet been formulated probabilistically.
Figure 1: Probabilistic numerical ODE solvers with different stability properties. Left: The explicit EK0 solver with a 3-times integrated Wiener process prior is unstable and diverges from the true solution. Center: The semi-implicit EK1 with the same prior does not diverge even though it uses a larger step size, due to it being A-stable, but it exhibits oscillations in the initial phase of the solution. Right: The proposed exponential integrator is L-stable and thus does not exhibit any oscillations.
In this paper we develop probabilistic exponential integrators, a new class of probabilistic numerical solvers for stiff semi-linear ODEs. We build on the ODE filters which have emerged as an efficient and flexible class of probabilistic numerical methods for general ODEs [40, 21, 45]. They have known convergence rates [21, 46], which have also been demonstrated empirically [2, 26, 24], they are applicable to a wide range of numerical differential equation problems [23, 25, 3], their probabilistic output can be integrated into larger inference problems [20, 39, 47], and they can be formulated parallel-in-time [4]. But while it has been shown that the choice of underlying Gauss-Markov prior does influence the resulting ODE solver [30, 45, 2], there has not yet been strong evidence for the utility of priors other than the well-established integrated Wiener process. Probabilistic exponential integrators provide this evidence: in the probabilistic numerics framework, "solving the linear part of the ODE exactly" corresponds to an appropriate choice of prior.
Contributions Our main contribution is the development of probabilistic exponential integrators, a new class of stable probabilistic solvers for stiff semi-linear ODEs. We demonstrate the close link of these methods to classic exponential integrators in Proposition 1, provide an equivalence result to a classic exponential integrator in Proposition 2, and prove their L-stability in Proposition 3. To enable a numerically stable implementation, we present a quadrature-based approach to directly compute square-roots of the process noise covariance in Section 3.2. Finally, in Section 3.6 we also propose probabilistic exponential Rosenbrock methods for problems in which semi-linearity is not known a priori. We evaluate all proposed methods on multiple stiff problems and demonstrate the improved stability and efficiency of the probabilistic exponential integrators over existing probabilistic solvers.
2 Numerical ODE solutions as Bayesian state estimation
Let us first consider an initial value problem with some general non-linear ODE, of the form
$$
\dot {y} (t) = f (y (t), t), \quad t \in [ 0, T ], \tag {1a}
$$
$$
y (0) = y _ {0}, \tag {1b}
$$
with vector field  $f: \mathbb{R}^d \times \mathbb{R} \to \mathbb{R}^d$ , initial value  $y_0 \in \mathbb{R}^d$ , and time span  $[0, T]$ . Probabilistic numerical ODE solvers aim to compute a posterior distribution over the ODE solution  $y(t)$  such that it satisfies the ODE on a discrete set of points  $\mathbb{T} = \{t_n\}_{n=0}^N \subset [0, T]$ , that is
$$
p \left(y (t) \mid y (0) = y _ {0}, \left\{\dot {y} \left(t _ {n}\right) = f \left(y \left(t _ {n}\right), t _ {n}\right) \right\} _ {n = 0} ^ {N}\right). \tag {2}
$$
We call this quantity, and approximations thereof, a probabilistic numerical ODE solution. Probabilistic numerical ODE solvers thus compute not just a single point estimate of the ODE solution, but a posterior distribution which provides a structured estimate of the numerical approximation error.
In the following, we briefly recapitulate the probabilistic ODE filter framework of Schober et al. [40] and Tronarp et al. [45] and define the prior, data model, and approximate inference scheme. In Section 3 we build on these foundations to derive the proposed probabilistic exponential integrator.
2.1 Gauss-Markov prior
A priori, we model  $y(t)$  with a Gauss-Markov process, defined by a stochastic differential equation
$$
\mathrm {d} Y (t) = A Y (t) \mathrm {d} t + \kappa B \mathrm {d} W (t), \quad Y (0) = Y _ {0}, \tag {3}
$$
with state  $Y(t) \in \mathbb{R}^{d(q + 1)}$ , model matrices  $A \in \mathbb{R}^{d(q + 1) \times d(q + 1)}$ ,  $B \in \mathbb{R}^{d(q + 1) \times d}$ , diffusion scaling  $\kappa \in \mathbb{R}$ , and smoothness  $q \in \mathbb{N}$ . More precisely,  $A$  and  $B$  are chosen such that the state is structured as  $Y(t) = [Y^{(0)}(t), \dots, Y^{(q)}(t)]$ , and then  $Y^{(i)}(t)$  models the  $i$ -th derivative of  $y(t)$ . The initial value  $Y_0 \in \mathbb{R}^{d(q + 1)}$  must be chosen such that it enforces the initial condition, that is,  $Y^{(0)}(0) = y_0$ .
One concrete example of such a Gauss-Markov process that is commonly used in the context of probabilistic numerical ODE solvers is the  $q$ -times Integrated Wiener process, with model matrices
$$
A _ {\mathrm {I W P} (d, q)} = \left[ \begin{array}{c c c c} 0 & I _ {d} & \dots & 0 \\ \vdots & \vdots & \ddots & \vdots \\ 0 & 0 & \dots & I _ {d} \\ 0 & 0 & \dots & 0 \end{array} \right], \quad B _ {\mathrm {I W P} (d, q)} = \left[ \begin{array}{c} 0 \\ \vdots \\ 0 \\ I _ {d} \end{array} \right]. \tag {4}
$$
Alternatives include the class of Matérn processes and the integrated Ornstein-Uhlenbeck process [46]—the latter plays a central role in this paper and will be discussed in detail later.
$Y(t)$  satisfies linear Gaussian transition densities of the form [44]
$$
Y (t + h) \mid Y (t) \sim \mathcal {N} (\Phi (h) Y (t), \kappa^ {2} Q (h)), \tag {5}
$$
with transition matrices  $\Phi (h)$  and  $Q(h)$  given by
$$
\Phi (h) = \exp (A h), \quad Q (h) = \int_ {0} ^ {h} \Phi (h - \tau) B B ^ {\top} \Phi^ {\top} (h - \tau) d \tau . \tag {6}
$$
These quantities can be computed with a matrix fraction decomposition [44]. For  $q$ -times integrated Wiener process priors, closed-form expressions for the transition matrices are available [21].
2.2 Information operator
The likelihood, or data model, of a probabilistic ODE solver relates the uninformed prior to the actual ODE solution of interest with an information operator  $\mathcal{I}$  [6], defined as
$$
\mathcal {I} [ Y ] (t) := E _ {1} Y (t) - f \left(E _ {0} Y (t), t\right), \tag {7}
$$
where  $E_{i} \in \mathbb{R}^{d \times d(q + 1)}$  are selection matrices such that  $E_{i}Y(t) = Y^{(i)}(t)$ .  $\mathcal{I}[Y]$  then captures how well  $Y$  solves the given ODE problem. In particular,  $\mathcal{I}$  maps the true ODE solution  $y$  to the zero function, i.e.  $\mathcal{I}[y] \equiv 0$ . Conversely, if  $\mathcal{I}[y](t) = 0$  holds for all  $t \in [0,T]$  then  $y$  solves the given ODE. Unfortunately, it is in general infeasible to solve an ODE exactly and enforce  $\mathcal{I}[Y](t) = 0$  everywhere, which is why numerical ODE solvers typically discretize the time interval and take discrete steps. This leads to the data model used in most probabilistic ODE solvers [45]:
$$
\mathcal {I} [ Y ] \left(t _ {n}\right) = E _ {1} Y \left(t _ {n}\right) - f \left(E _ {0} Y \left(t _ {n}\right), t _ {n}\right) = 0, \quad t _ {n} \in \mathbb {T} \subset [ 0, T ]. \tag {8}
$$
Note that this specific information operator is closely linked to the IVP considered in Eq. (1a). By defining a (slightly) different data model we can also formulate probabilistic numerical IVP solvers for higher-order ODEs or differential-algebraic equations, or encode additional information such as conservation laws or noisy trajectory observations [3, 39].
2.3 Approximate Gaussian inference
The resulting inference problem is described by a Gauss-Markov prior and a Dirac likelihood
$$
Y \left(t _ {n + 1}\right) \mid Y \left(t _ {n}\right) \sim \mathcal {N} \left(\Phi_ {n} Y \left(t _ {n}\right), \kappa^ {2} Q _ {n}\right), \tag {9a}
$$
$$
Z _ {n} \mid Y (t _ {n}) \sim \delta \left(E _ {1} Y (t _ {n}) - f \left(E _ {0} Y (t _ {n}), t _ {n}\right)\right), \tag {9b}
$$
with  $\Phi_n \coloneqq \Phi(t_{n+1} - t_n)$ ,  $Q_n \coloneqq Q(t_{n+1} - t_n)$ , initial value  $Y(0) = Y_0$ , discrete time grid  $\{t_n\}_{n=0}^N$  and zero-valued data  $Z_n = 0$  for all  $n$ . The solution of the resulting non-linear Gauss-Markov regression problem can then be efficiently approximated with Bayesian filtering and smoothing techniques [37]. Notable examples that have been used to construct probabilistic numerical ODE solvers include quadrature filters, the unscented Kalman filter, the iterated extended Kalman smoother, or particle filters [19, 45, 46]. Here, we focus on the well-established extended Kalman filter (EKF). We briefly discuss the EKF for the given state estimation problem in the following.
Prediction Given a Gaussian state estimate  $Y(t_{n - 1})\sim \mathcal{N}(\mu_{n - 1},\Sigma_{n - 1})$  and the linear conditional distribution as in Eq. (9a), the marginal distribution  $Y(t_{n})\sim \mathcal{N}(\mu_{n}^{-},\Sigma_{n}^{-})$  is also Gaussian, with
$$
\mu_ {n} ^ {-} = \Phi_ {n - 1} \mu_ {n - 1}, \tag {10a}
$$
$$
\Sigma_ {n} ^ {-} = \Phi_ {n - 1} \Sigma_ {n - 1} \Phi_ {n - 1} ^ {\top} + \kappa^ {2} Q _ {n - 1}. \tag {10b}
$$
Linearization To efficiently compute a tractable approximation of the true posterior, the EKF linearizes the information operator  $\mathcal{I}$  around the predicted mean  $\mu_n^{-}$ , i.e.  $\mathcal{I}[Y](t_n) \approx H_n Y(t_n) + b_n$ ,
$$
H _ {n} = E _ {1} - F _ {y} E _ {0}, \tag {11a}
$$
$$
b _ {n} = F _ {y} E _ {0} \mu_ {n} ^ {-} - f \left(E _ {0} \mu_ {n} ^ {-}, t _ {n}\right). \tag {11b}
$$
An exact linearization with Jacobian  $F_{y} = \partial_{y}f(E_{0}\mu_{n}^{-},t_{n})$  leads to a semi-implicit probabilistic ODE solver, which we call the EK1 [45]. Other choices include the zero matrix  $F_{y} = 0$ , which results in the explicit EK0 solver [40, 21], or a diagonal Jacobian approximation (the DiagonalEK1) which combines some stability benefits of the EK1 with the lower computational cost of the EK0 [24].
Correction step In the linearized observation model, the posterior distribution of  $Y(t_{n})$  given the datum  $Z_{n}$  is again Gaussian. Its posterior mean and covariance  $(\mu_n,\Sigma_n)$  are given by
$$
S _ {n} = H _ {n} \Sigma_ {n} ^ {-} H _ {n} ^ {\top}, \tag {12a}
$$
$$
K _ {n} = \Sigma_ {n} ^ {-} H _ {n} ^ {\top} S _ {n} ^ {- 1}, \tag {12b}
$$
$$
\mu_ {n} = \mu_ {n} ^ {-} - K _ {n} \left(E _ {1} \mu_ {n} ^ {-} - f \left(E _ {0} \mu_ {n} ^ {-}, t _ {n}\right)\right), \tag {12c}
$$
$$
\Sigma_ {n} = \left(I - K _ {n} H _ {n}\right) \Sigma_ {n} ^ {-}. \tag {12d}
$$
This is also known as the update step of the EKF.
Smoothing To condition the state estimates on all data, the EKF can be followed by a smoothing pass. Starting with  $\mu_N^S \coloneqq \mu_N$  and  $\Sigma_N^S \coloneqq \Sigma_n$ , it consists of the following backwards recursion:
$$
G _ {n} = \Sigma_ {n} \Phi_ {n} ^ {\top} \left(\Sigma_ {n + 1} ^ {-}\right) ^ {- 1}, \tag {13a}
$$
$$
\mu_ {n} ^ {S} = \mu_ {n} + G _ {n} \left(\mu_ {n + 1} ^ {S} - \mu_ {n + 1} ^ {-}\right), \tag {13b}
$$
$$
\Sigma_ {n} ^ {S} = \Sigma_ {n} + G _ {n} \left(\Sigma_ {n + 1} ^ {S} - \Sigma_ {n + 1} ^ {-}\right) G _ {n} ^ {\top}. \tag {13c}
$$
Result The above computations result in a probabilistic numerical ODE solution with marginals
$$
p \left(Y (t _ {i}) \mid \left\{E _ {1} Y (t _ {n}) - f \left(E _ {0} Y (t _ {n}), t _ {n}\right) = 0 \right\} _ {n = 0} ^ {N}\right) \approx \mathcal {N} \left(\mu_ {i} ^ {S}, \Sigma_ {i} ^ {S}\right), \tag {14}
$$
which, by construction of the state  $Y$ , also contains estimates for the ODE solution as  $y(t) = E_0Y(t)$ . Since the EKF-based probabilistic solver does not compute only the marginals in Eq. (14), but a full posterior distribution for the continuous object  $y(t)$ , it can be evaluated for times  $t \notin \mathbb{T}$  (also known as "dense output" in the context of ODE solvers); it can produce joint samples from this posterior; and it can be used as a Gauss-Markov prior for subsequent inference tasks [40, 2, 47].
2.4 Practical considerations and implementation details
To improve numerical stability and preserve positive-semidefiniteness of the computed covariance matrices, probabilistic ODE solvers typically operate on square-roots of covariance matrices, defined by a matrix decomposition of the form  $M = \sqrt{M} \sqrt{M}^{\top}$  [26]. For example, the Cholesky factor is one possible square-root of a positive definite matrix. But in general, the algorithm does not require the square-roots to be upper- or lower-triangular, or even square. Additionally, we compute the exact initial state  $Y_{0}$  from the IVP using Taylor-mode automatic differentiation [9, 26], we compute smoothing estimates with preconditioning [26], and we calibrate uncertainties globally with a quasi-maximum likelihood approach [45, 2].
3 Probabilistic exponential integrators
In the remainder of the paper, unless otherwise stated, we focus on IVPs with a semi-linear vector-field
$$
\dot {y} (t) = f (y (t), t) = L y (t) + N (y (t), t). \tag {15}
$$
Assuming  $N$  admits a Taylor series expansion around  $t$ , the variation of constants formula provides a formal expression of the solution at time  $t + h$ :
$$
y (t + h) = \exp (L h) y (t) + \sum_ {k = 0} ^ {\infty} h ^ {k + 1} \left(\int_ {0} ^ {1} \exp (L h (1 - \tau)) \frac {\tau^ {k}}{k !} \mathrm {d} \tau\right) \frac {\mathrm {d} ^ {k}}{\mathrm {d} t ^ {k}} N (y (t), t). \tag {16}
$$
This observation is the starting point for the development of exponential integrators [31, 15]. By further defining the so-called  $\varphi$ -functions
$$
\varphi_ {k} (z) = \int_ {0} ^ {1} \exp (z (1 - \tau)) \frac {\tau^ {k - 1}}{(k - 1) !} \mathrm {d} \tau , \tag {17}
$$
the above identity of the ODE solution simplifies to
$$
y (t + h) = \exp (L h) y (t) + \sum_ {k = 0} ^ {\infty} h ^ {k + 1} \varphi_ {k + 1} (L h) \frac {\mathrm {d} ^ {k}}{\mathrm {d} t ^ {k}} N (y (t), t). \tag {18}
$$
In this section we develop a class of probabilistic exponential integrators. This is achieved by defining an appropriate class of priors that absorbs the partial linearity, which leads to the integrated Ornstein-Uhlenbeck processes. Proposition 1 below directly relates this choice of prior to the classical exponential integrators. Proposition 2 demonstrates a direct equivalence between the predictor-corrector form exponential trapezoidal rule and the once integrated Ornstein-Uhlenbeck process. Furthermore, the favorable stability properties of classical exponential integrators is retained for the probabilistic counterparts as shown in Proposition 3.
3.1 The integrated Ornstein-Uhlenbeck process
In Section 2.1 we highlighted the choice of the  $q$ -times integrated Wiener process prior, which essentially corresponds to modeling the  $(q - 1)$ -th derivative of the right-hand side  $f$  with a Wiener process. Here we follow a similar motivation, but only for the non-linear part  $N$ . Differentiating both sides of Eq. (15)  $q - 1$  times with respect to  $t$  yields
$$
\frac {\mathrm {d} ^ {q - 1}}{\mathrm {d} t ^ {q - 1}} \dot {y} (t) = L \frac {\mathrm {d} ^ {q - 1}}{\mathrm {d} t ^ {q - 1}} y (t) + \frac {\mathrm {d} ^ {q - 1}}{\mathrm {d} t ^ {q - 1}} N (y (t), t). \tag {19}
$$
Then, modeling  $\frac{\mathrm{d}^{q - 1}}{\mathrm{d}t^{q - 1}} N(y(t),t)$  as a Wiener process and relating the result to  $y(t)$  gives
$$
\mathrm {d} y ^ {(i)} (t) = y ^ {(i + 1)} (t) \mathrm {d} t, \tag {20a}
$$
$$
\mathrm {d} y ^ {(q)} (t) = L y ^ {(q)} (t) \mathrm {d} t + \kappa I _ {d} \mathrm {d} W ^ {(q)} (t). \tag {20b}
$$
This process is also known as the  $q$ -times integrated Ornstein-Uhlenbeck process (IOUP), with rate parameter  $L$  and diffusion parameter  $\kappa$ . It can be equivalently stated with the previously introduced notation (Section 2.1), by defining a state  $Y(t)$ , as the solution of a linear time-invariant (LTI) SDE as in Eq. (3), with system matrices
$$
A _ {\mathrm {I O U P} (d, q)} = \left[ \begin{array}{c c c c} 0 & I _ {d} & \dots & 0 \\ \vdots & \vdots & \ddots & \vdots \\ 0 & 0 & \dots & I _ {d} \\ 0 & 0 & \dots & L \end{array} \right], \quad B _ {\mathrm {I O U P} (d, q)} = \left[ \begin{array}{c} 0 \\ \vdots \\ 0 \\ I _ {d} \end{array} \right]. \tag {21}
$$
Remark 1 (The mean of the IOUP process solves the linear part of the ODE exactly). By taking the expectation of Eq. (20b) and by linearity of integration, we can see that the mean of the IOUP satisfies
$$
\dot {\mu} ^ {(0)} (t) = L \mu^ {(0)} (t), \quad \mu^ {(0)} (0) = y _ {0}. \tag {22}
$$
This is in line with the motivation of exponential integrators: the linear part of the ODE is solved exactly, and we only need to approximate the non-linear part. Figure 2 visualizes this idea.
Figure 2: Damped oscillator dynamics and priors with different degrees of encoded information. Left: Once-integrated Wiener process, a popular prior for probabilistic ODE solvers. Center: Once-integrated Ornstein-Uhlenbeck process (IOUP) with rate parameter chosen to encode the known linearity of the ODE. Right: IOUP with both the ODE information and a specified initial value and derivative. This is the kind of prior used in the probabilistic exponential integrator.
3.2 The transition parameters of the integrated Ornstein-Uhlenbeck process
Since the process  $Y(t)$  is defined as the solution of a linear time-invariant SDE, it satisfies discrete transition densities  $p(Y(t + h) \mid Y(t)) = \mathcal{N}\left(\Phi(h)Y(t), \kappa^2 Q(h)\right)$ . The following result shows that the transition parameters are intimately connected with the  $\varphi$ -functions defined in Eq. (17).
Proposition 1. The transition matrix of a q-times integrated Ornstein-Uhlenbeck process satisfies
$$
\Phi (h) = \left[ \begin{array}{c c} \exp \left(A _ {\mathrm {I W P} (d, q - 1)} h\right) & \Phi_ {1 2} (h) \\ 0 & \exp (L h) \end{array} \right], \quad \text {w i t h} \quad \Phi_ {1 2} (h) := \left[ \begin{array}{c} h ^ {q} \varphi_ {q} (L h) \\ h ^ {q - 1} \varphi_ {q - 1} (L h) \\ \vdots \\ h \varphi_ {1} (L h) \end{array} \right]. \tag {23}
$$
Proof in Appendix A. Although Proposition 1 indicates that  $\Phi(h)$  may be computed more efficiently than by calling a matrix-exponential on a  $d(q + 1) \times d(q + 1)$  matrix, this is known to be numerically less stable [41]. We therefore compute  $\Phi(h)$  with the standard matrix-exponential formulation.
Directly computing square-roots of the process noise covariance Numerically stable probabilistic ODE solvers require a square-root,  $\sqrt{Q(h)}$ , of the process noise covariance rather than the full matrix,  $Q(h)$ . For IWP priors this can be computed from the closed-form representation of  $Q(h)$  via an appropriately preconditioned Cholesky factorization [26]. However, for IOUP priors we have not found an analogous method that works reliably. Therefore, we compute  $\sqrt{Q(h)}$  directly with numerical quadrature. More specifically, given a quadrature rule with nodes  $\tau_{i} \in [0,h]$  and positive weights  $w_{i} > 0$ , the integral for  $Q(h)$  given in Eq. (6) is approximated by
$$
Q (h) \approx \sum_ {i = 1} ^ {m} w _ {i} \exp \left(A \left(h - \tau_ {i}\right)\right) B B ^ {\top} \exp \left(A ^ {\top} \left(h - \tau_ {i}\right)\right) =: \sum_ {i = 1} ^ {m} M _ {i}, \tag {24}
$$
with square-roots  $\sqrt{M_i} = \sqrt{w_i}\exp (A(h - \tau_i))B$  of the summands, which is well-defined since  $w_{i} > 0$ . We can thus compute a square-root representation of the sum with a QR-decomposition
$$
X \cdot R = \operatorname {Q R} \left(\left[ \begin{array}{l l l} \sqrt {M _ {1}} & \dots & \sqrt {M _ {m}} \end{array} \right] ^ {\top}\right). \tag {25}
$$
We obtain  $Q(h) \approx R^{\top}R$ , and therefore an approximate square-root factor is given by  $\sqrt{Q(h)} \approx R^{\top}$ . Similar ideas have previously been employed for time integration of Riccati equations [42, 43]. We use this quadrature-trick for all IOUP methods, with Gauss-Legendre quadrature on  $m = q$  nodes.
3.3 Linearization and correction
The information operator of the probabilistic exponential integrator is defined exactly as in Section 2.2. But since we now assume a semi-linear vector-field  $f$ , we have an additional option for the linearization: instead of choosing the exact  $F_{y} = \partial_{y}f$  (EK1) or the zero-matrix  $F_{y} = 0$  (EK0), a cheap approximate Jacobian is given by the linear part  $F_{y} = L$ . We denote this approach by EKL. This is chosen as the default for the probabilistic exponential integrator. Note that the EKL approach can also be combined with an IWP prior, which will be serve as an additional baseline in the Section 4.
3.4 Equivalence to the classic exponential trapezoidal rule in predict-evaluate-correct mode
Now that the probabilistic exponential integrator has been defined, we can establish an equivalence result to a classic exponential integrator, similarly to the closely-related equivalence statement by Schober et al. [40, Proposition 1] for the non-exponential case.