SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Proposition 2 (Equivalence to the PEC exponential trapezoidal rule). The mean estimate of the probabilistic exponential integrator with a once-integrated Ornstein-Uhlenbeck prior with rate parameter  $L$  is equivalent to the classic exponential trapezoidal rule in predict-evaluate-correct mode, with the predictor being the exponential Euler method. That is, it is equivalent to the scheme
$$
\tilde {y} _ {n + 1} = \varphi_ {0} (L h) y _ {n} + h \varphi_ {1} (L h) N (\tilde {y} _ {n}), \tag {26a}
$$
$$
y _ {n + 1} = \varphi_ {0} (L h) y _ {n} + h \varphi_ {1} (L h) N (\tilde {y} _ {n}) + h ^ {2} \varphi_ {2} (L h) \frac {N \left(\tilde {y} _ {n + 1}\right) - N \left(\tilde {y} _ {n}\right)}{h}, \tag {26b}
$$
where Eq. (26a) corresponds to a prediction step with the exponential Euler method, and Eq. (26b) corresponds to a correction step with the exponential trapezoidal rule.
The proof is given in Appendix B. This equivalence result provides another theoretical justification for the proposed probabilistic exponential integrator. But note that the result only holds for the mean, while the probabilistic solver computes additional quantities in order to track the solution uncertainty, namely covariances. These are not provided by a classic exponential integrator.
3.5 L-stability of the probabilistic exponential integrator
When solving stiff ODEs, the actual efficiency of a numerical method often depends on its stability. One such property is  $A$ -stability: It guarantees that the numerical solution of a decaying ODE will also decay, independently of the chosen step size. In contrast, explicit methods typically only decay for sufficiently small steps. In the context of probabilistic ODE solvers, the EK0 is considered to be explicit, but the EK1 with IWP prior has been shown to be A-stable [45]. Here, we show that the probabilistic exponential integrator satisfies the stronger  $L$ -stability: the numerical solution not only decays, but it decays fast, i.e. it goes to zero as the step size goes to infinity. Figure 1 visualizes the different probabilistic solver stabilities. For formal definitions, see for example [27, Section 8.6].
Proposition 3 (L-stability). The probabilistic exponential integrator is  $L$ -stable.
The full proof is given in Appendix C. The property essentially follows from Remark 1 which stated that the IOUP solves linear ODEs exactly. This implies fast decay and gives L-stability.
3.6 Probabilistic exponential Rosenbrock-type methods
We conclude with a short excursion into exponential Rosenbrock methods [14, 17, 28]: Given a nonlinear ODE  $\dot{y}(t) = f(y(t), t)$ , exponential Rosenbrock methods perform a continuous linearization of the right-hand side  $f$  around the numerical ODE solution and essentially solve a sequence of IVPs
$$
\dot {y} (t) = J _ {n} y (t) + \left(f (y (t), t) - J _ {n} y (t)\right), \quad t \in \left[ t _ {n}, t _ {n + 1} \right], \tag {27a}
$$
$$
y \left(t _ {n}\right) = y _ {n}, \tag {27b}
$$
where  $J_{n}$  is the Jacobian of  $f$  at the numerical solution estimate  $\hat{y}(t_n)$ . This approach enables exponential integrators for problems where the right-hand side  $f$  is not semi-linear. Furthermore, by automatically linearizing along the numerical solution the linearization can be more accurate, the Lipschitz-constant of the non-linear remainder becomes smaller, and the resulting solvers can thus be more efficient than their globally linearized counterparts [17].
This can also be done in the probabilistic setting: By linearizing the ODE right-hand side  $f$  at each step of the solver around the filtering mean  $E_0\mu_n$ , we (locally) obtain a semi-linear problem. Then, updating the rate parameter of the integrated Ornstein-Uhlenbeck process at each step of the numerical solver results in probabilistic exponential Rosenbrock-type methods. As before, the linearization of the information operator can be done with any of the EK0, EK1, or EKL. But since here the prediction relies on exact local linearization, we will by default also use an exact EK1 linearization. The resulting solver and its stability and efficiency will be evaluated in the following experiments.
Figure 3: The IOUP prior is more beneficial with increasing linearity of the ODE. In all three examples, the IOUP-based exponential integrator achieves lower error while requiring fewer steps than the IWP-based solvers. This effect is more pronounced for the more linear ODEs.
In this section we investigate the utility and performance of the proposed probabilistic exponential integrators and compare it to standard non-exponential probabilistic solvers on multiple ODEs. All methods are implemented in the Julia programming language [1], with special care being taken to implement the solvers in a numerically stable manner, that is, with exact state initialization, preconditioned state transitions, and a square-root implementation [26]. Reference solutions are computed with the DifferentialEquations.jl package [34]. All experiments run on a single, consumer-level CPU. Code for the implementation and experiments is publicly available on GitHub.<sup>1</sup>
4.1 Logistic equation with varying degrees of non-linearity
We start with a simple one-dimensional initial value problem: a logistic model with negative growth rate parameter  $r = -1$  and carrying capacity  $K \in \mathbb{R}_{+}$ , of the form
$$
\dot {y} (t) = - y (t) + \frac {1}{K} y (t) ^ {2}, \quad t \in [ 0, 1 0 ], \tag {28a}
$$
$$
y (0) = 1. \tag {28b}
$$
The non-linearity of this problem can be directly controlled through the parameter  $K$ . Therefore, this test problem lets us investigate the IOUP's capability to leverage known linearity in the ODE.
We compare the proposed exponential integrator to all introduced IWP-based solvers, with different linearization strategies: EK0 approximates  $\partial_y f \approx 0$  (and is thus explicit), EKL approximates  $\partial_y f \approx -1$ , and EK1 linearizes with the correct Jacobian  $\partial_y f$ . The results for four different values of  $K$  are shown in Fig. 3. The explicit solver shows the largest error of all compared solvers, likely due to its lacking stability. On the other hand, the proposed exponential integrator behaves as expected: the IOUP prior is most beneficial for larger values of  $K$ , and as the non-linearity becomes more pronounced the performance of the IOUP approaches that of the IWP-based solver. Though for large step sizes, the IOUP outperforms the IWP prior even for the most non-linear case with  $K = 10$ .
4.2 Burger's equation
Here, we consider Burger's equation, which is a semi-linear partial differential equation (PDE)
$$
\partial_ {t} u (x, t) = D \partial_ {x} ^ {2} u (x, t) - u (x, t) \partial_ {x} u (x, t), \quad x \in [ 0, 1 ], \quad t \in [ 0, 1 ], \tag {29}
$$
with diffusion coefficient  $D \in \mathbb{R}_+$ . We transform the problem into a semi-linear ODE with the method of lines [29, 38], and discretize the spatial domain on 250 equidistant points and approximate the differential operators with finite differences. The full IVP specification, including all domains, initial and boundary conditions, and additional information on the discretization, is given in Appendix D.
The results shown in Fig. 4 demonstrate the different stability properties of the solvers: The explicit EK0 with IWP prior is unable to solve the IVP for any of the step sizes due to its insufficient stability, and even the A-stable EK1 and the more approximate EKL require small enough steps  $\Delta t < 10^{-1}$ . On the other hand, both exponential integrators are able to compute meaningful solutions for a larger range of step sizes. They both achieve lower errors for most settings than their non-exponential
<https://github.com/nathanaelbosch/probabilistic-exponential-integrators>
Figure 4: Benchmarking probabilistic ODE solvers on Burger's equation. Exponential and nonexponential probabilistic solvers are compared on Burger's equation (a) in two work-precision diagrams (b). Both exponential integrators with IOUP prior achieve lower errors than the existing IWP-based solvers, in particular for large steps. This indicates their stronger stability properties.
Figure 5: Benchmarking probabilistic ODE solvers on a reaction-diffusion model. Exponential and non-exponential probabilistic solvers are compared on a reaction-diffusion model (a) in two work-precision diagrams (b). The proposed exponential integrators with IOUP prior achieve lower errors per step size than the existing IWP-based methods. The runtime comparison shows the increased cost of the Rosenbrock-type (RB) method, while the non-Rosenbrock probabilistic exponential integrator performs best in this comparison.
counterparts. The second diagram in Fig. 4 compares the achieved error to the number of vector-field evaluations and points out a trade-off between both exponential methods: Since the Rosenbrock method additionally computes two Jacobians (with automatic differentiation) per step, it needs to evaluate the vector-field more often than the non-Rosenbrock method. Thus, for expensive-to-evaluate vector fields the standard probabilistic exponential integrator might be preferable.
4.3 Reaction-diffusion model
Finally, we consider a discretized reaction-diffusion model given by a semi-linear PDE
$$
\partial_ {t} u (x, t) = D \partial_ {x} ^ {2} u (x, t) + R (u (x, t)), \quad x \in [ 0, 1 ], \quad t \in [ 0, T ], \tag {30}
$$
where  $D \in \mathbb{R}_+$  is the diffusion coefficient and  $R(u) = u(1 - u)$  is a logistic reaction term [22]. A finite-difference discretization of the spatial domain transforms this PDE into an IVP with semi-linear ODE. The full problem specification is provided in Appendix D.
Figure 5 shows the results. We again observe the improved stability of the exponential integrator variants by their lower error for large step sizes, and they outperform the IWP-based methods on all settings. The runtime-evaluation in Fig. 5 also visualizes another drawback of the Rosenbrock-type method: Since the problem is re-linearized at each step, the IOUP also needs to be re-discretized and thus a matrix exponential needs to be computed. In comparison, the non-Rosenbrock method only discretizes the IOUP prior once at the start of the solve. This advantage makes the non-Rosenbrock probabilistic exponential integrator the most performant solver in this experiment.
The probabilistic exponential integrator shares many properties of both classic exponential integrators and of other filtering-based probabilistic solvers. This also brings some challenges.
Cost of computing matrix exponentials The IOUP prior is more expensive to discretize than the IWP as it requires computing a matrix exponential. This trade-off is well-known also in the context of classic exponential integrators. One approach to reduce computational cost is to compute the matrix exponential only approximately [32], for example with Krylov-subspace methods [13, 17]. Extending these techniques to the probabilistic solver setting thus poses an interesting direction for future work.
Cubic scaling in the ODE dimension The probabilistic exponential integrator shares the complexity most (semi-)implicit ODE solvers: while being linear in the number of time steps, it scales cubically in the ODE dimension. By exploiting structure in the Jacobian and in the prior, some filtering-based ODE solvers have been formulated with linear scaling in the ODE dimension [24]. But this approach does not directly extend to the IOUP-prior. Nevertheless, exploiting known structure could be particularly relevant to construct solvers for specific ODEs, such as certain discretized PDEs.
We have presented probabilistic exponential integrators, a new class of probabilistic solvers for stiff semi-linear ODEs. By incorporating the fast, linear dynamics directly into the prior of the solver, the method essentially solves the linear part exactly, in a similar manner as classic exponential integrators. We also extended the proposed method to general non-linear systems via iterative re-linearization and presented probabilistic exponential Rosenbrock-type methods. Both methods have been shown both theoretically and empirically to be more stable than their non-exponential probabilistic counterparts. This work further expands the toolbox of probabilistic numerics and opens up new possibilities for accurate and efficient probabilistic simulation and inference in stiff dynamical systems.
The authors gratefully acknowledge financial support by the German Federal Ministry of Education and Research (BMBF) through Project ADIMEM (FKZ 01IS18052B), and financial support by the European Research Council through ERC StG Action 757275 / PANAMA; the DFG Cluster of Excellence Machine Learning - New Perspectives for Science, EXC 2064/1, project number 390727645; the German Federal Ministry of Education and Research (BMBF) through the Tübingen AI Center (FKZ: 01IS18039A); and funds from the Ministry of Science, Research and Arts of the State of Baden-Württemberg. Filip Tronarp was partially supported by the Wallenberg AI, Autonomous Systems and Software Program (WASP) funded by the Knut and Alice Wallenberg Foundation. The authors thank the International Max Planck Research School for Intelligent Systems (IMPRS-IS) for supporting Nathanael Bosch. The authors also thank Jonathan Schmidt for many valuable discussions and for helpful feedback on the manuscript.
[1] J. Bezanson, A. Edelman, S. Karpinski, and V. B. Shah. Julia: A fresh approach to numerical computing. SIAM review, 59(1):65-98, 2017.
[2] N. Bosch, P. Hennig, and F. Tronarp. Calibrated adaptive probabilistic ODE solvers. In International Conference on Artificial Intelligence and Statistics. PMLR, 2021.
[3] N. Bosch, F. Tronarp, and P. Hennig. Pick-and-mix information operators for probabilistic ODE solvers. In International Conference on Artificial Intelligence and Statistics. PMLR, 2022.
[4] N. Bosch, A. Corenflos, F. Yaghoobi, F. Tronarp, P. Hennig, and S. Särkkä. Parallel-in-time probabilistic numerical ODE solvers, 2023.
[5] R. T. Q. Chen, Y. Rubanova, J. Bettencourt, and D. K. Duvenaud. Neural ordinary differential equations. In Advances in Neural Information Processing Systems. Curran Associates, Inc., 2018.
[6] J. Cockayne, C. Oates, T. Sullivan, and M. Girolami. Bayesian probabilistic numerical methods. SIAM Review, 61:756-789, 2019.
Probabilistic Exponential Integrators — Appendix
A Proof of Proposition 1: Structure of the transition matrix
Proof of Proposition 1. The drift-matrix  $A_{\mathrm{IOUP}(d,q)}$  as given in Eq. (21) has block structure
$$
A _ {\mathrm {I O U P} (d, q)} = \left[ \begin{array}{c c} A _ {\mathrm {I W P} (d, q - 1)} & E _ {q - 1} \\ 0 & L \end{array} \right], \tag {31}
$$
where  $E_{q - 1} \coloneqq [0 \quad \ldots \quad 0 \quad I_d]^\top \in \mathbb{R}^{dq \times d}$ . From Van Loan [48, Theorem 1], it follows
$$
\Phi (h) = \left[ \begin{array}{c c} \exp \left(A _ {\mathrm {I W P} (d, q - 1)} h\right) & \Phi_ {1 2} (h) \\ 0 & \exp (L h) \end{array} \right], \tag {32}
$$
which is precisely Eq. (23). The same theorem also gives  $\Phi_{12}(h)$  as
$$
\Phi_ {1 2} (h) = \int_ {0} ^ {h} \exp \left(A _ {\mathrm {I W P} (d, q - 1)} (h - \tau)\right) E _ {q - 1} ^ {(d - 1)} \exp (L \tau) \mathrm {d} \tau . \tag {33}
$$
Its  $i$ th  $d \times d$  block is readily given by
$$
\begin{array}{l} \left(\Phi_ {1 2} (h)\right) _ {i} = \int_ {0} ^ {h} E _ {i} ^ {\top} \exp \left(A _ {\mathrm {I W P} (d, q - 1)} (h - \tau)\right) E _ {q - 1} \exp (L \tau) \mathrm {d} \tau \\ = \int_ {0} ^ {h} \frac {(h - \tau) ^ {q - 1 - i}}{(q - 1 - i) !} \exp (L \tau) d \tau \tag {34} \\ = h ^ {q - i} \int_ {0} ^ {1} \frac {\tau^ {q - 1 - i}}{(q - 1 - i) !} \exp (L h (1 - \tau)) d \tau \\ = h ^ {q - i} \varphi_ {q - i} (L h), \\ \end{array}
$$
where the second last equality used the change of variables  $\tau = h(1 - u)$ , and the last line follows by definition.
B Proof of Proposition 2: Equivalence to a classic exponential integrator
We first briefly recapitulate the probabilistic exponential integrator setup for the case of the once integrated Ornstein-Uhlenbeck process, and then provide some auxiliary results. Then, we prove Proposition 2 in Appendix B.3.
B.1 The probabilistic exponential integrator with once-integrated Ornstein-Uhlenbeck prior
The integrated Ornstein-Uhlenbeck process prior with rate parameter  $L$  results in transition densities  $Y(t + h) \mid Y(t) \sim \mathcal{N}(Y(t + h); \Phi(h)Y(t), Q(h))$ , with transition matrices (from Proposition 1)
$$
\begin{array}{l} \Phi (h) = \exp (A h) = \left[ \begin{array}{c c} I & h \varphi_ {1} (L h) \\ 0 & \varphi_ {0} (L h) \end{array} \right], (35) \\ Q (h) = \int_ {0} ^ {h} \exp (A \tau) B B ^ {\top} \exp (A ^ {\top} \tau) d \tau (36) \\ = \int_ {0} ^ {h} \left[ \begin{array}{l l} I & \tau \varphi_ {1} (L \tau) \\ 0 & \varphi_ {0} (L \tau) \end{array} \right] \left[ \begin{array}{l l} 0 & 0 \\ 0 & I \end{array} \right] \left[ \begin{array}{l l} I & \tau \varphi_ {1} (L \tau) \\ 0 & \varphi_ {0} (L \tau) \end{array} \right] ^ {\top} d \tau (37) \\ = \int_ {0} ^ {h} \left[ \begin{array}{c c} \tau^ {2} \varphi_ {1} (L \tau) \varphi_ {1} (L \tau) ^ {\top} & \tau \varphi_ {1} (L \tau) \varphi_ {0} (L \tau) ^ {\top} \\ \tau \varphi_ {0} (L \tau) \varphi_ {1} (L \tau) ^ {\top} & \varphi_ {0} (L \tau) \varphi_ {0} (L \tau) ^ {\top} \end{array} \right] d \tau , (38) \\ \end{array}
$$
where we assume a unit diffusion  $\sigma^2 = 1$ . To simplify notation, we assume an equidistant time grid  $\mathbb{T} = \{t_n\}_{n=0}^{N}$  with  $t_n = n \cdot h$  for some step size  $h$ , and we denote the constant transition matrices simply by  $\Phi$  and  $Q$  and write  $Y_n = Y(t_n)$ .
Before getting to the actual proof, let us also briefly recapitulate the filtering formulas that are computed at each solver step. Given a Gaussian distribution  $Y_{n} \sim \mathcal{N}(Y_{n};\mu_{n},\Sigma_{n})$ , the prediction step computes
$$
\mu_ {n + 1} ^ {-} = \Phi \mu_ {n}, \tag {39}
$$
$$
\Sigma_ {n + 1} ^ {-} = \Phi (h) \Sigma_ {n} \Phi (h) ^ {\top} + Q (h). \tag {40}
$$
Then, the combined linearization and correction step compute
$$
\hat {z} _ {n + 1} = E _ {1} \mu_ {n + 1} ^ {-} - f \left(E _ {0} \mu_ {n + 1} ^ {-}\right), \tag {41}
$$
$$
S _ {n + 1} = H \Sigma_ {n + 1} ^ {-} H ^ {\top}, \tag {42}
$$
$$
K _ {n + 1} = \Sigma_ {n + 1} ^ {-} H ^ {\top} S _ {n + 1} ^ {- 1}, \tag {43}
$$
$$
\mu_ {n + 1} = \mu_ {n + 1} ^ {-} - K _ {n + 1} \hat {z} _ {n + 1}, \tag {44}
$$
$$
\Sigma_ {n + 1} = \Sigma_ {n + 1} ^ {-} - K _ {n + 1} S _ {n + 1} K _ {n + 1} ^ {\top}, \tag {45}
$$
with observation matrix  $H = E_{1} - LE_{0} = [-L \quad I]$ , since we perform the proposed EKL linearization.
B.2 Auxiliary results
In the following, we show some properties of the transition matrices and the covariances that will be needed in the proof of Proposition 2 later.
First, note that by defining  $\varphi_0(z) = \exp z$ , the  $\varphi$ -functions satisfy the following recurrence formula:
$$
z \varphi_ {k} (z) = \varphi_ {k - 1} (z) - \frac {1}{(k - 1) !}. \tag {46}
$$
See e.g. Hochbruck and Ostermann [15]. This property will be used throughout the remainder of the section.
Lemma B.1. The transition matrices  $\Phi(h), Q(h)$  of the once integrated Ornstein-Uhlenbeck process with rate parameter  $L$  satisfy
$$
H \Phi (h) = \left[ \begin{array}{l l} - L & I \end{array} \right], \tag {47}
$$
$$
Q (h) H ^ {\top} = \left[ \begin{array}{l} h ^ {2} \varphi_ {2} (L h) \\ h \varphi_ {1} (L h) \end{array} \right], \tag {48}
$$
$$
H Q (h) H ^ {\top} = h I, \tag {49}
$$
$$
H \Phi (h) = \left(E _ {1} - L E _ {0}\right) \left[ \begin{array}{c c} I & h \varphi_ {1} (L h) \\ 0 & \varphi_ {0} (L h) \end{array} \right] = \left[ \begin{array}{l l} 0 & \varphi_ {0} (L h) \end{array} \right] - L \left[ \begin{array}{l l} I & h \varphi_ {1} (L h) \end{array} \right] = \left[ \begin{array}{l l} - L & I \end{array} \right]. \tag {50}
$$
$$
\begin{array}{l} Q (h) H ^ {\top} = \int_ {0} ^ {h} \left[ \begin{array}{c c} \tau^ {2} \varphi_ {1} (L \tau) \varphi_ {1} (L \tau) ^ {\top} & \tau \varphi_ {1} (L \tau) \varphi_ {0} (L \tau) ^ {\top} \\ \tau \varphi_ {0} (L \tau) \varphi_ {1} (L \tau) ^ {\top} & \varphi_ {0} (L \tau) \varphi_ {0} (L \tau) ^ {\top} \end{array} \right] H ^ {\top} d \tau (51) \\ = \int_ {0} ^ {h} \left[ \begin{array}{c} \tau \varphi_ {1} (L \tau) \varphi_ {0} (L \tau) ^ {\top} - L \tau^ {2} \varphi_ {1} (L \tau) \varphi_ {1} (L \tau) ^ {\top} \\ \varphi_ {0} (L \tau) \varphi_ {0} (L \tau) ^ {\top} - L \tau \varphi_ {0} (L \tau) \varphi_ {1} (L \tau) ^ {\top} \end{array} \right] d \tau (52) \\ = \int_ {0} ^ {h} \left[ \begin{array}{l} \tau \varphi_ {1} (L \tau) \left(\varphi_ {0} (L \tau) ^ {\top} - L \tau \varphi_ {1} (L \tau) ^ {\top}\right) \\ \varphi_ {0} (L \tau) \left(\varphi_ {0} (L \tau) ^ {\top} - L \tau \varphi_ {1} (L \tau) ^ {\top}\right) \end{array} \right] d \tau (53) \\ = \int_ {0} ^ {h} \left[ \begin{array}{l} \tau \varphi_ {1} (L \tau) \\ \varphi_ {0} (L \tau) \end{array} \right] d \tau (54) \\ = \left[ \begin{array}{l} h ^ {2} \varphi_ {2} (L h) \\ h \varphi_ {1} (L h) \end{array} \right] (55) \\ \end{array}
$$
where we used  $L\tau \varphi_{1}(L\tau) = \varphi_{0}(L\tau) - I$ , and  $\partial_{\tau}\left[\tau^{k}\varphi_{k}(L\tau)\right] = \tau^{k - 1}\varphi_{k - 1}(L\tau)$ . It follows that
$$
H Q (h) H ^ {\top} = H \left[ \begin{array}{l} h ^ {2} \varphi_ {2} (L h) \\ h \varphi_ {1} (L h) \end{array} \right] = h \left(\varphi_ {1} (L h) - L h \varphi_ {2} (L h)\right) = h I, \tag {56}
$$
where we used  $L\tau \varphi_{2}(L\tau) = \varphi_{1}(L\tau) - I$
Lemma B.2. The prediction covariance  $\Sigma_{n + 1}^{-}$  satisfies
$$
\Sigma_ {n + 1} ^ {-} H ^ {\top} = Q (h) H ^ {\top}. \tag {57}
$$
Proof. First, since the observation model is noiseless, the filtering covariance  $\Sigma_{n}$  satisfies
$$
H \Sigma_ {n} = \left[ \begin{array}{l l} 0 & 0 \end{array} \right]. \tag {58}
$$
This can be shown directly from the correction step formula:
$$
\begin{array}{l} H \Sigma_ {n} = H \Sigma_ {n} ^ {-} - H K _ {n} S _ {n} K _ {n} ^ {\top} (59) \\ = H \Sigma_ {n} ^ {-} - H \left(\Sigma_ {n} ^ {-} H ^ {\top} S _ {n} ^ {- 1}\right) S _ {n} K _ {n} ^ {\top} (60) \\ = H \Sigma_ {n} ^ {-} - H \Sigma_ {n} ^ {-} H ^ {\top} \left(H \Sigma_ {n} ^ {-} H ^ {\top}\right) ^ {- 1} S _ {n} K _ {n} ^ {\top} (61) \\ = H \Sigma_ {n} ^ {-} - I S _ {n} K _ {n} ^ {\top} (62) \\ = H \Sigma_ {n} ^ {-} - S _ {n} \left(\Sigma_ {n} ^ {-} H ^ {\top} S _ {n} ^ {- 1}\right) ^ {\top} (63) \\ = H \Sigma_ {n} ^ {-} - S _ {n} S _ {n} ^ {- 1} H \Sigma_ {n} ^ {-} (64) \\ = \left[ \begin{array}{l l} 0 & 0 \end{array} \right]. (65) \\ \end{array}
$$
Next, since the observation matrix is  $H = \left[-L\quad I\right]$ , the filtering covariance  $\Sigma_{n}$  is structured as
$$
\Sigma_ {n} = \left[ \begin{array}{l} I \\ L \end{array} \right] \left[ \Sigma_ {n} \right] _ {0 0} \left[ \begin{array}{l l} I & L ^ {\top} \end{array} \right]. \tag {66}
$$
This can be shown directly from Eq. (58):
$$
\left[ \begin{array}{l l} 0 & 0 \end{array} \right] = H \Sigma = \left[ \begin{array}{l l} - L & I \end{array} \right] \left[ \begin{array}{l l} \Sigma_ {0 0} & \Sigma_ {0 1} \\ \Sigma_ {1 0} & \Sigma_ {1 1} \end{array} \right] = \left[ \begin{array}{l l} \Sigma_ {1 0} - L \Sigma_ {0 0} & \Sigma_ {1 1} - L \Sigma_ {0 1} \end{array} \right], \tag {67}
$$
$$
\Sigma_ {1 0} = L \Sigma_ {0 0}, \tag {68}
$$
$$
\Sigma_ {1 1} = L \Sigma_ {0 1} = L \Sigma_ {1 0} ^ {\top} = L \Sigma_ {0 0} L ^ {\top}. \tag {69}
$$
$$
\Sigma = \left[ \begin{array}{c c} \Sigma_ {0 0} & L \Sigma_ {0 0} \\ \Sigma_ {0 0} L ^ {\top} & L \Sigma_ {0 0} L ^ {\top} \end{array} \right] = \left[ \begin{array}{l} I \\ L \end{array} \right] \Sigma_ {0 0} \left[ \begin{array}{c c} I & L ^ {\top} \end{array} \right]. \tag {70}
$$
Finally, together with Lemma B.1 we can derive the result: