SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$$
\begin{array}{l} \Sigma_ {n + 1} ^ {-} H ^ {\top} = \Phi (h) \Sigma_ {n} \Phi (h) ^ {\top} H ^ {\top} + Q (h) H ^ {\top} (71) \\ = \Phi (h) \left[ \begin{array}{l} I \\ L \end{array} \right] \bar {\Sigma} _ {n} \left[ \begin{array}{l l} I & L ^ {\top} \end{array} \right] \left[ \begin{array}{l} - L ^ {\top} \\ I \end{array} \right] + Q (h) H ^ {\top} (72) \\ = \Phi (h) \left[ \begin{array}{l} I \\ L \end{array} \right] \bar {\Sigma} _ {n} \cdot 0 + Q (h) H ^ {\top} (73) \\ = Q (h) H ^ {\top}. (74) \\ \end{array}
$$
With these results, we can now prove Proposition 2.
Proof of Proposition 2. We prove the proposition by induction, showing that the filtering means are all of the form
$$
\mu_ {n} := \left[ \begin{array}{c} y _ {n} \\ L y _ {n} + N \left(\tilde {y} _ {n}\right) \end{array} \right], \tag {75}
$$
where  $y_{n},\tilde{y}_{n}$  are defined as
$$
\tilde {y} _ {0} := y _ {0}, \tag {76}
$$
$$
\tilde {y} _ {n + 1} := \varphi_ {0} (L h) y _ {n} + h \varphi_ {1} (L h) N (\tilde {y} _ {n}), \tag {77}
$$
$$
y _ {n + 1} := \varphi_ {0} (L h) y _ {n} + h \varphi_ {1} (L h) N (\tilde {y} _ {n}) - h \varphi_ {2} (L h) \left(N (\tilde {y} _ {n}) - N (\tilde {y} _ {n + 1})\right). \tag {78}
$$
This result includes the statement of Proposition 2.
Base case  $n = 0$  The initial distribution of the probabilistic solver is chosen as
$$
\mu_ {0} = \left[ \begin{array}{c} y _ {0} \\ L y _ {0} + N \left(\tilde {y} _ {0}\right) \end{array} \right], \Sigma_ {0} = 0. \tag {79}
$$
This proves the base case  $n = 0$ .
Induction step  $n\to n + 1$  Now, let
$$
\mu_ {n} = \left[ \begin{array}{c} y _ {n} \\ L y _ {n} + N (\tilde {y} _ {n}) \end{array} \right] \tag {80}
$$
be the filtering mean at step  $n$  and  $\Sigma_{n}$  be the filtering covariance. The prediction mean is of the form
$$
\mu_ {n + 1} ^ {-} = \Phi (h) \mu_ {n} = \left[ \begin{array}{c} y _ {n} + h \varphi_ {1} (L h) \left(L y _ {n} + N \left(\tilde {y} _ {n}\right)\right) \\ \varphi_ {0} (L h) \left(L y _ {n} + N \left(\tilde {y} _ {n}\right)\right) \end{array} \right] = \left[ \begin{array}{c} \varphi_ {0} (L h) y _ {n} + h \varphi_ {1} (L h) N \left(\tilde {y} _ {n}\right) \\ \varphi_ {0} (L h) \left(L y _ {n} + N \left(\tilde {y} _ {n}\right)\right) \end{array} \right]. \tag {81}
$$
The residual  $\hat{z}_{n + 1}$  is then of the form
$$
\begin{array}{l} \hat {z} _ {n + 1} = E _ {1} \mu_ {n + 1} ^ {-} - f \left(E _ {0} \mu_ {n + 1} ^ {-}\right) (82) \\ = \varphi_ {0} (L h) \left(L y _ {n} + N \left(\tilde {y} _ {n}\right)\right) - f \left(\varphi_ {0} (L h) y _ {n} + h \varphi_ {1} (L h) N \left(\tilde {y} _ {n}\right)\right) (83) \\ = \varphi_ {0} (L h) \left(L y _ {n} + N \left(\tilde {y} _ {n}\right)\right) - L \left(\varphi_ {0} (L h) y _ {n} + h \varphi_ {1} (L h) N \left(\tilde {y} _ {n}\right)\right) - N \left(\tilde {y} _ {n + 1}\right) (84) \\ = \varphi_ {0} (L h) L y _ {n} + \varphi_ {0} (L h) N (\tilde {y} _ {n}) - L \varphi_ {0} (L h) y _ {n} - L h \varphi_ {1} (L h) N (\tilde {y} _ {n}) - N (\tilde {y} _ {n + 1}) (85) \\ = \left(\varphi_ {0} (L h) - L h \varphi_ {1} (L h)\right) N \left(\tilde {y} _ {n}\right) - N \left(\tilde {y} _ {n + 1}\right) (86) \\ = N \left(\tilde {y} _ {n}\right) - N \left(\tilde {y} _ {n + 1}\right), (87) \\ \end{array}
$$
where we used properties of the  $\varphi$ -functions, namely  $Lh\varphi_{1}(Lh) = \varphi_{0}(Lh)$  and the commutativity  $\varphi_0(Lh)L = L\varphi_0(\bar{L} h)$ . With Lemma B.2, the residual covariance  $S_{n + 1}$  and Kalman gain  $K_{n + 1}$  are then of the form
$$
S _ {n + 1} = H \Sigma_ {n + 1} ^ {-} H ^ {\top} = H Q (h) H ^ {\top} = h I, \tag {89}
$$
$$
K _ {n + 1} = \Sigma_ {n + 1} ^ {-} H ^ {\top} S _ {n + 1} ^ {- 1} = Q (h) H ^ {\top} (h I) ^ {- 1} = \left[ \begin{array}{c} h \varphi_ {2} (L h) \\ \varphi_ {1} (L h) \end{array} \right]. \tag {90}
$$
This gives the updated mean
$$
\begin{array}{l} \mu_ {n + 1} = \mu_ {n + 1} ^ {-} - K _ {n + 1} \hat {z} _ {n + 1} (91) \\ = \left[ \begin{array}{c} \varphi_ {0} (L h) y _ {n} + h \varphi_ {1} (L h) N (\tilde {y} _ {n}) \\ \varphi_ {0} (L h) \left(L y _ {n} + N \left(\tilde {y} _ {n}\right)\right) \end{array} \right] - \left[ \begin{array}{c} h \varphi_ {2} (L h) \\ \varphi_ {1} (L h) \end{array} \right] \left(N \left(\tilde {y} _ {n}\right) - N \left(\tilde {y} _ {n + 1}\right)\right) (92) \\ = \left[ \begin{array}{c} \varphi_ {0} (L h) y _ {n} + h \varphi_ {1} (L h) N (\tilde {y} _ {n}) - h \varphi_ {2} (L h) \left(N (\tilde {y} _ {n}) - N (\tilde {y} _ {n + 1})\right) \\ \varphi_ {0} (L h) \left(L y _ {n} + N (\tilde {y} _ {n})\right) - \varphi_ {1} (L h) \left(N (\tilde {y} _ {n}) - N (\tilde {y} _ {n + 1})\right) \end{array} \right]. (93) \\ \end{array}
$$
This proves the first half of the mean recursion:
$$
E _ {0} \mu_ {n + 1} = \varphi_ {0} (L h) y _ {n} + h \varphi_ {1} (L h) N (\tilde {y} _ {n}) - h \varphi_ {2} (L h) \left(N (\tilde {y} _ {n}) - N (\tilde {y} _ {n + 1})\right) = y _ {n + 1}. \tag {94}
$$
It is left to show that
$$
E _ {1} \mu_ {n + 1} = L y _ {n + 1} - N \left(\tilde {y} _ {n + 1}\right). \tag {95}
$$
Starting from the right-hand side, we have
$$
L y _ {n + 1} + N \left(\tilde {y} _ {n + 1}\right) \tag {96}
$$
$$
= L \left(\varphi_ {0} (L h) y _ {n} + h \varphi_ {1} (L h) N (\tilde {y} _ {n}) - h \varphi_ {2} (L h) \left(N (\tilde {y} _ {n}) - N (\tilde {y} _ {n + 1})\right)\right) + N (\tilde {y} _ {n + 1}) \tag {97}
$$
$$
= \varphi_ {0} (L h) L y _ {n} + L h \varphi_ {1} (L h) N (\tilde {y} _ {n}) - L h \varphi_ {2} (L h) \left(N (\tilde {y} _ {n}) - N (\tilde {y} _ {n + 1})\right) N (\tilde {y} _ {n + 1}) \tag {98}
$$
$$
= \varphi_ {0} (L h) L y _ {n} + \left(\varphi_ {0} (L h) - I\right) N \left(\tilde {y} _ {n}\right) - \left(\varphi_ {1} (L h) - I\right) \left(N \left(\tilde {y} _ {n}\right) - N \left(\tilde {y} _ {n + 1}\right)\right) N \left(\tilde {y} _ {n + 1}\right) \tag {99}
$$
$$
= \varphi_ {0} (L h) \left(L y _ {n} + N \left(\tilde {y} _ {n}\right)\right) - \varphi_ {1} (L h) \left(N \left(\tilde {y} _ {n}\right) - N \left(\tilde {y} _ {n + 1}\right)\right) \tag {100}
$$
$$
= E _ {1} \mu_ {n + 1}. \tag {101}
$$
This concludes the proof of the mean recursion and thus shows the equivalence of the two recursions.
C Proof of Proposition 3: L-stability
We first provide definitions of L-stability and A-stability, following [27, Section 8.6].
Definition 1 (L-stability). A one-step method is said to be  $L$ -stable if it is  $A$ -stable and, in addition, when applied to the scalar test-equation  $\dot{y}(t) = \lambda y(t)$ ,  $\lambda \in \mathbb{C}$  a complex constant with  $\operatorname{Re}(\lambda) < 0$ , it yields  $y_{n+1} = R(h\lambda)y_n$ , and  $R(h\lambda) \to 0$  as  $\operatorname{Re}(h\lambda) \to -\infty$ .
Definition 2 (A-stability). A one-step method is said to be A-stable if its region of absolute stability contains the whole of the left complex half-plane. That is, when applied to the scalar test-equation  $\dot{y}(t) = \lambda y(t)$  with  $\lambda \in \mathbb{C}$  a complex constant with  $\operatorname{Re}(\lambda) < 0$ , the method yields  $y_{n+1} = R(h\lambda)y_n$  and  $\{z \in \mathbb{C} : \operatorname{Re}(z) < 0\} \subset \{z \in \mathbb{C} : R(z) < 1\}$ .
Proof of Proposition 3. Both L-stability and A-stability directly follow from Remark 1: Since the probabilistic exponential integrator solves linear ODEs exactly its stability function is the exponential function, i.e.  $R(z) = \exp (z)$ . A-stability and L-stability then follow: Since  $\mathbb{C}^{-}\subset \{z:|R(z)|\leq 1\}$  holds the method is A-stable. And since  $|R(z)|\to 0$  as  $\operatorname {Re}(z)\to -\infty$  the method is L-stable.
D.1 Burger's equation
Burger's equation is a semi-linear partial differential equation (PDE) of the form
$$
\partial_ {t} u (x, t) = - u (x, t) \partial_ {x} u (x, t) + D \partial_ {x} ^ {2} u (x, t), \quad x \in \Omega , \quad t \in [ 0, T ], \tag {102}
$$
with diffusion coefficient  $D \in \mathbb{R}_+$ . We discretize the spatial domain  $\Omega$  on a finite grid and approximate the spatial derivatives with finite differences to obtain a semi-linear ODE of the form
$$
\dot {y} (t) = D \cdot L \cdot y (t) + F (y (t)), \quad t \in [ 0, T ], \tag {103}
$$
with  $N$ -dimensional  $y(t) \in \mathbb{R}^N$ ,  $L \in \mathbb{R}^{N \times N}$  the finite difference approximation of the Laplace operator  $\partial_x^2$ , and a non-linear part  $F$ .
More specifically, we consider a domain  $\Omega = (0,1)$ , which we discretize with a grid of  $N = 250$  equidistant locations, thus we have  $\Delta x = 1 / N$ . We consider zero-Dirichlet boundary conditions, that is,  $u(0,t) = u(1,t) = 0$ . The discrete Laplacian is then
$$
[ L ] _ {i j} = \frac {1}{\Delta x ^ {2}} \cdot \left\{ \begin{array}{l l} - 2 & \text {i f} i = j, \\ 1 & \text {i f} i = j \pm 1, \\ 0 & \text {o t h e r w i s e .} \end{array} \right. \tag {104}
$$
The non-linear part of the discretized Burger's equation results from another finite-difference approximation of the term  $u \cdot \partial_x u$ , and is chosen as
$$
[ F (y) ] _ {i} = \frac {1}{4 \Delta x} \left\{ \begin{array}{l l} y _ {2} ^ {2} & \text {i f} i = 1, \\ y _ {d - 1} ^ {2} & \text {i f} i = d, \\ y _ {i + 1} ^ {2} - y _ {i - 1} ^ {2} & \text {e l s e .} \end{array} \right. \tag {105}
$$
The initial condition is chosen as
$$
u (x, 0) = \sin (3 \pi x) ^ {3} (1 - x) ^ {3 / 2}. \tag {106}
$$
We consider an integration time-span  $t \in [0,1]$ , and choose a diffusion coefficient  $D = 0.075$ .
D.2 Reaction-diffusion model
The reaction-diffusion model presented in the paper, with logistic reaction term, has been used to describe the growth and spread of biological populations [22]. It is given by a semi-linear PDE
$$
\partial_ {t} u (x, t) = D \partial_ {x} ^ {2} u (x, t) + R (u (x, t)), \quad x \in \Omega , \quad t \in [ 0, T ], \tag {107}
$$
where  $D \in \mathbb{R}_+$  is the diffusion coefficient and  $R(u) = u(1 - u)$  is a logistic reaction term. We discretize the spatial domain  $\Omega$  on a finite grid and approximate the spatial derivatives with finite differences, and obtain a semi-linear ODE of the form
$$
\dot {y} (t) = D \cdot L \cdot y (t) + R (y (t)), \quad t \in [ 0, T ], \tag {108}
$$
with  $N$ -dimensional  $y(t) \in \mathbb{R}^N$ ,  $L \in \mathbb{R}^{N \times N}$  the finite difference approximation of the Laplace operator, and the reaction term  $R$  is as before but applied element-wise.
We again consider a domain  $\Omega = (0,1)$ , which we discretize on a grid of  $N = 100$  points. This time we consider zero-Neumann conditions, that is,  $\partial_x u(0,t) = \partial_x u(1,t) = 0$ . Including these directly into the finite-difference discretization, the discrete Laplacian is then
$$
\left[ L \right] _ {i j} = \frac {1}{\Delta x ^ {2}} \cdot \left\{ \begin{array}{l l} - 1 & \text {i f} i = j = 1 \text {o r} i = j = d, \\ - 2 & \text {i f} i = j, \\ 1 & \text {i f} i = j \pm 1, \\ 0 & \text {o t h e r w i s e .} \end{array} \right. \tag {109}
$$
$$
u (x, 0) = \frac {1}{1 + e ^ {3 0 x - 1 0}}. \tag {110}
$$
The discrete ODE is then solved on a time-span  $t \in [0,2]$ , and we choose a diffusion coefficient  $D = 0.25$ .