SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Consider the general case of the numerical scheme as follows:
$$
\begin{array}{l} \boldsymbol {x} _ {i + 1} = \frac {\sigma_ {t _ {i + 1}}}{\sigma_ {t _ {i}}} e ^ {- \int_ {\lambda_ {t _ {i}}} ^ {\lambda_ {t _ {i + 1}}} \tau^ {2} (\lambda_ {u}) \mathrm {d} \lambda_ {u}} \boldsymbol {x} _ {i} + \sum_ {j = - 1} ^ {s - 1} b _ {i - j} \boldsymbol {x} _ {\boldsymbol {\theta}} \left(\boldsymbol {x} _ {i - j}, t _ {i - j}\right) \tag {55} \\ + \sigma_ {t _ {i + 1}} \int_ {t _ {i}} ^ {t _ {i + 1}} e ^ {- \int_ {u} ^ {t _ {i + 1}} \tau^ {2} (\lambda) \mathrm {d} \lambda} \tau (u) \sqrt {- 2 \frac {\mathrm {d} \lambda_ {u}}{\mathrm {d} u}} \mathrm {d} \bar {w} _ {u}. \\ \end{array}
$$
in which Eq. (17) and Eq (14) are the special case of this scheme. We will provide proof of the mean-square convergence order of the numerical scheme  $\max_{0\leq i\leq M}\| \pmb{x}_{t_i} - \pmb{x}_i\|_{L_2}$ . We define the local
error of the numerical scheme Eq. (55) for the approximation of the SDE Eq. (54) as
$$
\begin{array}{l} L _ {i + 1} = \boldsymbol {x} _ {t _ {i + 1}} - \boldsymbol {x} _ {i + 1} = \boldsymbol {x} _ {t _ {i + 1}} - \frac {\sigma_ {t _ {i + 1}}}{\sigma_ {t _ {i}}} e ^ {- \int_ {\lambda t _ {i}} ^ {\lambda t _ {i + 1}} \tau^ {2} (\lambda) \mathrm {d} \lambda} \boldsymbol {x} _ {t _ {i}} - \sum_ {j = - 1} ^ {s - 1} b _ {i - j} \boldsymbol {x} _ {\boldsymbol {\theta}} \left(\boldsymbol {x} _ {t _ {i - j}}, t _ {i - j}\right) \tag {56} \\ - \sigma_ {t _ {i + 1}} \int_ {t _ {i}} ^ {t _ {i + 1}} e ^ {- \int_ {u} ^ {t _ {i + 1}} \tau^ {2} (\lambda) \mathrm {d} \lambda} \tau (u) \sqrt {- 2 \frac {\mathrm {d} \lambda_ {u}}{\mathrm {d} u}} \mathrm {d} \bar {w} _ {u}. \\ \end{array}
$$
$L_{i + 1}$  can be decomposed into  $R_{i + 1}$  and  $S_{i + 1}$ . Then the mean-square convergence can be derived, which is summarized in the following theorem proved by [29]:
Theorem B.6 ([29], Theorem 1). The mean-square convergent of  $\pmb{x}_i$  is bounded by
$$
\left. \max  _ {0 \leq i \leq M} \| \boldsymbol {x} _ {t _ {i}} - \boldsymbol {x} _ {i} \| _ {L _ {2}} \leq S \left\{\max  _ {0 \leq i \leq s - 1} \| D _ {i} \| _ {L _ {2}} + \max  _ {s \leq i \leq M} \left(\frac {\| R _ {i} \| _ {L _ {2}}}{h} + \frac {\| S _ {i} \| _ {L _ {2}}}{h ^ {\frac {1}{2}}}\right) \right\}. \right. \tag {57}
$$
In Eq. (57),  $D_{i}, i = 0, \dots, s - 1$  are the initial error which we do not consider. Given Theorem B.6, to show the convergence order  $\mathcal{O}\left(\max_{0 \leq t \leq T} \tau(t) h + h^{s}\right)$  of our  $s$ -step SA-Predictor and the convergence order  $\mathcal{O}\left(\max_{0 \leq t \leq T} \tau(t) h + h^{s + 1}\right)$  of our  $s$ -step SA-Corrector, we just need to prove the following lemmas.
Lemma B.7 (Convergence rate of  $s$ -step SA-Predictor). For
$$
\boldsymbol {x} _ {t _ {i + 1}} = \frac {\sigma_ {t _ {i + 1}}}{\sigma_ {t _ {i}}} e ^ {- \int_ {\lambda_ {t _ {i}}} ^ {\lambda_ {t _ {i + 1}}} \tau^ {2} (\lambda_ {u}) \mathrm {d} \lambda_ {u}} \boldsymbol {x} _ {t _ {i}} + \sum_ {j = 0} ^ {s - 1} b _ {i - j} \boldsymbol {x} _ {\boldsymbol {\theta}} \left(\boldsymbol {x} _ {t _ {i - j}}, t _ {i - j}\right) + \tilde {\sigma} _ {i} \boldsymbol {\xi}, \quad \boldsymbol {\xi} \sim \mathcal {N} (\boldsymbol {0}, \boldsymbol {I}),
$$
$$
\tilde {\sigma} _ {i} = \sigma_ {t _ {i + 1}} \sqrt {1 - e ^ {- 2 \int_ {\lambda_ {t _ {i}}} ^ {\lambda_ {t _ {i + 1}}} \tau^ {2} (\lambda) \mathrm {d} \lambda}}
$$
$$
b _ {i - j} = \sigma_ {t _ {i + 1}} \int_ {\lambda_ {t _ {i}}} ^ {\lambda_ {t _ {i + 1}}} e ^ {- \int_ {\lambda_ {u}} ^ {\lambda_ {t _ {i + 1}}} \tau^ {2} (\lambda_ {v}) \mathrm {d} \lambda_ {v}} \left(1 + \tau^ {2} (\lambda_ {u})\right) e ^ {\lambda_ {u}} l _ {i - j} (\lambda_ {u}) \mathrm {d} \lambda_ {u}, \quad \forall 0 \leq j \leq s - 1 \tag {58}
$$
There exists a decomposition of local error  $L_{i}$  such that  $L_{i} = R_{i} + S_{i}$  and
$$
\left\| R _ {i} \right\| _ {L _ {2}} \leq h ^ {s + 1}, \left\| S _ {i} \right\| _ {L _ {2}} \leq \max  _ {0 \leq t \leq T} \tau (t) h ^ {\frac {3}{2}}, \tag {59}
$$
Lemma B.8 (Convergence rate of  $s$ -step SA-Corrector). For
$$
\pmb {x} _ {t _ {i + 1}} = \frac {\sigma_ {t _ {i + 1}}}{\sigma_ {t _ {i}}} e ^ {- \int_ {\lambda_ {t _ {i}}} ^ {\lambda_ {t _ {i + 1}}} \tau^ {2} (\lambda_ {u}) \mathrm {d} \lambda_ {u}} \pmb {x} _ {t _ {i}} + \hat {b} _ {i + 1} \pmb {x} _ {\pmb {\theta}} (\pmb {x} _ {t _ {i + 1}} ^ {p}, t _ {i + 1}) + \sum_ {j = 0} ^ {\hat {s} - 1} \hat {b} _ {i - j} \pmb {x} _ {\pmb {\theta}} (\pmb {x} _ {t _ {i - j}}, t _ {i - j}) + \tilde {\sigma} _ {i} \pmb {\xi},
$$
$$
\tilde {\sigma} _ {i} = \sigma_ {t _ {i + 1}} \sqrt {1 - e ^ {- 2 \int_ {\lambda t _ {i}} ^ {\lambda t _ {i + 1}} \tau^ {2} (\lambda) \mathrm {d} \lambda}}
$$
$$
\hat {b} _ {i - j} = \sigma_ {t _ {i + 1}} \int_ {\lambda_ {t _ {i}}} ^ {\lambda_ {t _ {i + 1}}} e ^ {- \int_ {\lambda_ {u}} ^ {\lambda_ {t _ {i + 1}}} \tau^ {2} (\lambda_ {v}) \mathrm {d} \lambda_ {v}} \left(1 + \tau^ {2} (\lambda_ {u})\right) e ^ {\lambda_ {u}} \hat {l} _ {i - j} (\lambda_ {u}) \mathrm {d} \lambda_ {u}, \quad \forall 0 \leq j \leq s - 1
$$
$$
\hat {b} _ {i + 1} = \sigma_ {t _ {i + 1}} \int_ {\lambda_ {t _ {i}}} ^ {\lambda_ {t _ {i + 1}}} e ^ {- \int_ {\lambda_ {u}} ^ {\lambda_ {t _ {i + 1}}} \tau^ {2} (\lambda_ {v}) \mathrm {d} \lambda_ {v}} \left(1 + \tau^ {2} (\lambda_ {u})\right) e ^ {\lambda_ {u}} \hat {l} _ {i + 1} (\lambda_ {u}) \mathrm {d} \lambda_ {u} \tag {60}
$$
$$
\left\| R _ {i} \right\| _ {L _ {2}} \leq h ^ {s + 2}, \left\| S _ {i} \right\| _ {L _ {2}} \leq \max  _ {0 \leq t \leq T} \tau (t) h ^ {\frac {3}{2}}, \tag {61}
$$
Lemma B.7 and B.8 will be proved in Sec. B.4.
B.3 Lemmas for the Proof
To better analyze the local error here, we state the following definitions and results from [45]. For a continuous function  $y:\mathbb{R}^d\times [0,T]\to \mathbb{R}^d$ , a general multiple Wiener integral over the subinterval  $[t,t + h]\subset [0,T]$  is given by
$$
I _ {r _ {1} r _ {2} \dots r _ {j}} ^ {t, t + h} (y) = \int_ {t} ^ {t + h} \int_ {t} ^ {s _ {1}} \dots \int_ {t} ^ {s _ {j - 1}} y \left(\boldsymbol {x} _ {s _ {j}}, s _ {j}\right) \mathrm {d} w _ {r _ {1}} \left(s _ {j}\right) \dots \mathrm {d} w _ {r _ {j}} \left(s _ {1}\right), \tag {62}
$$
where  $r_i \in \{0, 1, \dots, d\}$  and  $\mathrm{d}w_0(s) = \mathrm{d}s$ . Then we have the following lemma.
Lemma B.9 (Bound of Wiener Integral). For any function  $y:\mathbb{R}^d\times [0,T]\to \mathbb{R}^d$  that satisfies a growth condition in the form  $|y(\pmb {x},t)|\leq K(1 + |\pmb {x}|^{2})^{\frac{1}{2}}$ , for any  $\pmb {x}\in \mathbb{R}^d$ , and any  $t\in [0,T]$ ,  $h > 0$  such that  $t + h\in [0,T]$ , we have that
$$
\mathbb {E} \left[ I _ {r _ {1} r _ {2} \dots r _ {j}} ^ {t, t + h} (y) | \mathcal {F} _ {t} \right] = 0 \quad i f r _ {i} \neq 0 f o r s o m e i \in \{1, \dots , j \}, \tag {63}
$$
$$
\left\| I _ {r _ {1} r _ {2} \dots r _ {j}} ^ {t, t + h} (y) \right\| _ {L _ {2}} = \mathcal {O} \left(h ^ {l _ {1} + \frac {l _ {2}}{2}}\right), \tag {64}
$$
where  $l_{1}$  is the number of zero indices and  $l_{2}$  is the number of non-zero indices  $r_i$ .
Lemma B.10 (Property of Lagrange interpolation polynomial). For  $s + 1$  points  $(t_{i+1}, y_{i+1}), (t_i, y_i), \dots, (t_{i-(s-1)}, y_{i-(s-1)})$ , the Lagrange interpolation polynomial is
$$
L (t) = \sum_ {k = i - (s - 1)} ^ {i + 1} l _ {k} (t) y _ {k}. \tag {65}
$$
Then the following  $s + 1$  equalities hold
$$
\sum_ {k = i - (s - 1)} ^ {i + 1} l _ {k} (u) = 1,
$$
$$
\sum_ {k = i - (s - 1)} ^ {i + 1} l _ {k} (u) \int_ {t _ {i - (s - 1)}} ^ {t _ {k}} \mathrm {d} u _ {2} = \int_ {t _ {i - (s - 1)}} ^ {u} \mathrm {d} u _ {2},
$$
$$
\sum_ {k = i - (s - 1)} ^ {i + 1} l _ {k} (u) \int_ {t _ {i - (s - 1)}} ^ {t _ {k}} \int_ {t _ {i - (s - 1)}} ^ {u _ {2}} \dots \int_ {t _ {i - (s - 1)}} ^ {u _ {s}} \mathrm {d} u _ {s + 1} \dots \mathrm {d} u _ {3} \mathrm {d} u _ {2} =
$$
$$
\int_ {t _ {i - (s - 1)}} ^ {u} \int_ {t _ {i - (s - 1)}} ^ {u _ {2}} \dots \int_ {t _ {i - (s - 1)}} ^ {u _ {s}} d u _ {s + 1} \dots d u _ {3} d u _ {2} \tag {66}
$$
Proof. For the first equality, consider  $y_{k} \equiv 1$  for  $i - (s - 1) \leq k \leq i + 1$ . The Lagrange interpolation polynomial for these  $y_{k}$ 's is a constant function  $L(t) \equiv 1$ . We have  $L(u) = \sum_{k=i-(s-1)}^{i+1} l_{k}(u) = 1$ .
For the second equality, consider  $y_{k} = \int_{t_{i - (s - 1)}}^{t_{k}}\mathrm{d}u_{2}$ . The Lagrange interpolation polynomial for these  $y_{k}$ s is a polynomial of degree 1  $L(t) = t - t_{i - (s - 1)}$ . We have  $L(u) = \sum_{k = i - (s - 1)}^{i + 1}l_k(u)\int_{t_{i - (s - 1)}}^{t_k}\mathrm{d}u_2 = u - t_{i - (s - 1)} = \int_{t_{i - (s - 1)}}^u\mathrm{d}u_2$ .
For equalities from the third to the last, without loss of generality, we prove the  $p - th$  equality, where  $3 \leq p \leq s + 1$ . Consider  $y_k = \int_{t_{i-(s-1)}}^{t_k} \int_{t_{i-(s-1)}}^{u_2} \dots \int_{t_{i-(s-1)}}^{u_{p-1}} \mathrm{d}u_p \dots \mathrm{d}u_3 \mathrm{d}u_2$ . The Lagrange interpolation polynomial for these  $y_k$ s is a polynomial of degree  $p - 1$ .  $L(t) = \int_{t_{i-(s-1)}}^{t} \int_{t_{i-(s-1)}}^{u_2} \dots \int_{t_{i-(s-1)}}^{u_{p-1}} \mathrm{d}u_p \dots \mathrm{d}u_3 \mathrm{d}u_2$ .
We have  $L(u) = \int_{t_{i - (s - 1)}}^{u}\int_{t_{i - (s - 1)}}^{u_2}\dots \int_{t_{i - (s - 1)}}^{u_{p - 1}}\mathrm{d}u_p\dots \mathrm{d}u_3\mathrm{d}u_2 =$
$\int_{t_{i - (s - 1)}}^{u}\int_{t_{i - (s - 1)}}^{u_2}\dots \int_{t_{i - (s - 1)}}^{u_{p - 1}}\mathrm{d}u_p\dots \mathrm{d}u_3\mathrm{d}u_2.$
B.4 Proof of Lemma B.7 (for Theorem. 5.1) and Lemma B.8 (for Theorem. 5.2)
To simplify the notation, we will introduce two operators which will appear in the Ito formula. Suppose we have an  $I\hat{o}$  SDE  $\mathrm{d}\pmb{x}_t = f(\pmb{x}_t,t)\mathrm{d}t + g(\pmb{x}_t,t)\mathrm{d}\pmb{w}_t$  and  $h(\pmb{x},t)$  is a twice continuously differentiable function. Let  $\Gamma_0(\cdot) = \partial_t(\cdot) + \nabla_{\pmb{x}}(\cdot)f,\Gamma_1(\cdot) = \frac{g^2}{2}\Delta (\cdot)$  and  $\Gamma_2(\cdot) = \nabla_{\pmb{x}}(\cdot)g$  in which
$\nabla_{\pmb{x}}$  is the Jacobian matrix, and  $\Delta$  is the Laplacian operator. With the notation here, we can express the Itô formula for  $h(\pmb{x}, t)$  as
$$
h \left(\boldsymbol {x} _ {t}, t\right) = h \left(\boldsymbol {x} _ {s}, s\right) + \int_ {s} ^ {t} \left(\Gamma_ {0} (h) + \Gamma_ {1} (h)\right) \mathrm {d} t + \int_ {s} ^ {t} \Gamma_ {2} (h) \mathrm {d} \bar {\boldsymbol {w}} _ {t}. \tag {67}
$$
Given the above lemmas, we will analyze the local error  $L_{i + 1}$  step by step. Inspired by Theorem B.6, for data-prediction reparameterization model,  $L_{i + i}$  can be estimated by decomposing the terms step by step. The first step of decomposition is summarized as the following lemma:
Lemma B.11 (First step of estimating local error  $L_{i + 1}$  in data-prediction reparameterization model). Given the exact solution of data prediction model
$$
\boldsymbol {F} _ {\boldsymbol {\theta}} (s, t) = \int_ {\lambda_ {s}} ^ {\lambda_ {t}} e ^ {- \int_ {\lambda} ^ {\lambda_ {t}} \tau^ {2} (\tilde {\lambda}) \mathrm {d} \tilde {\lambda}} \left(1 + \tau^ {2} (\lambda)\right) e ^ {\lambda} \boldsymbol {x} _ {\boldsymbol {\theta}} \left(\boldsymbol {x} _ {\lambda}, \lambda\right) \mathrm {d} \lambda \tag {68}
$$
With proper  $b_{k}, k \in [i - (s - 1), i + 1]$ , The local error  $L_{i + 1}$  in Eq. (56) is
$$
L _ {i + 1} = R _ {i + 1} ^ {(1)} + S _ {i + 1} ^ {(1)} \tag {69}
$$
$$
\begin{array}{l} S_{i + 1}^{(1)} = \mathcal{O}\left(\max_{0\leq t\leq T}\tau (t)h^{\frac{3}{2}}\right) \\ R _ {i + 1} ^ {(1)} = \sum_ {k = i - (s - 1)} ^ {i - 1} \sigma_ {t _ {i + 1}} \left(\int_ {t _ {i}} ^ {t _ {i + 1}} e ^ {- \int_ {\lambda_ {u}} ^ {\lambda_ {t _ {i + 1}}} \tau^ {2} (\lambda) \mathrm {d} \lambda} (1 + \tau^ {2} (u)) e ^ {\lambda_ {u}} \frac {\mathrm {d} \lambda_ {u}}{\mathrm {d} u} \mathrm {d} u\right) \times \\ \left(\int_ {t _ {k}} ^ {t _ {k + 1}} \Gamma_ {0} (\boldsymbol {x} _ {\boldsymbol {\theta}}) \mathrm {d} t\right) \tag {70} \\ + \sigma_ {t _ {i + 1}} \int_ {t _ {i}} ^ {t _ {i + 1}} e ^ {- \int_ {\lambda_ {u}} ^ {\lambda_ {t _ {i + 1}}} \tau^ {2} (\lambda) \mathrm {d} \lambda} (1 + \tau^ {2} (u)) e ^ {\lambda_ {u}} \left(\int_ {t _ {i}} ^ {u} \Gamma_ {0} (\boldsymbol {x} _ {\boldsymbol {\theta}}) \mathrm {d} t\right) \frac {\mathrm {d} \lambda_ {u}}{\mathrm {d} u} \mathrm {d} u \\ - \sum_ {j = - 1} ^ {s - 1} b _ {i - j} \sum_ {k = i - (s - 1)} ^ {i - j - 1} \int_ {t _ {k}} ^ {t _ {k + 1}} \Gamma_ {0} (\boldsymbol {x} _ {\boldsymbol {\theta}}) \mathrm {d} t. \\ \end{array}
$$
Proof. The difference between Eq. (55) and Eq. (56) is that  $\boldsymbol{x}_j$  is our numerical approximation, while  $\boldsymbol{x}_{t_j}$  is the exact solution of SDE Eq. (54) at time  $t = t_j$ . Substitute the exact solution Eq. (31) of  $\boldsymbol{x}_{t_{i+1}}$ , we have
$$
\begin{array}{l} L _ {i + 1} = \frac {\sigma_ {t _ {i + 1}}}{\sigma_ {t _ {i}}} e ^ {- \int_ {\lambda_ {t _ {i}}} ^ {\lambda_ {t _ {i + 1}}} \tau^ {2} (\lambda) \mathrm {d} \lambda} \pmb {x} _ {t _ {i}} + \sigma_ {t _ {i + 1}} \int_ {t _ {i}} ^ {t _ {i + 1}} e ^ {- \int_ {u} ^ {t _ {i + 1}} \tau^ {2} (\lambda) \mathrm {d} \lambda} \tau (u) \sqrt {- 2 \frac {\mathrm {d} \lambda_ {u}}{\mathrm {d} u}} \mathrm {d} \bar {\pmb {w}} _ {u} \\ + \sigma_ {t _ {i + 1}} \int_ {\lambda_ {t _ {i}}} ^ {\lambda_ {t _ {i + 1}}} e ^ {- \int_ {\lambda_ {u}} ^ {\lambda_ {t _ {i + 1}}} \tau^ {2} (\lambda) \mathrm {d} \lambda} (1 + \tau^ {2} (\lambda_ {u})) e ^ {\lambda_ {u}} \boldsymbol {x} _ {\boldsymbol {\theta}} (\boldsymbol {x} _ {\lambda_ {u}}, \lambda_ {u}) \mathrm {d} \lambda_ {u} \\ - \sigma_ {t _ {i + 1}} \int_ {t _ {i}} ^ {t _ {i + 1}} e ^ {- \int_ {u} ^ {t _ {i + 1}} \tau^ {2} (\lambda) \mathrm {d} \lambda} \tau (u) \sqrt {- 2 \frac {\mathrm {d} \lambda_ {u}}{\mathrm {d} u}} \mathrm {d} \bar {w} _ {u} \\ - \frac {\sigma_ {t _ {i + 1}}}{\sigma_ {t _ {i}}} e ^ {- \int_ {\lambda_ {t _ {i}}} ^ {\lambda_ {t _ {i + 1}}} \tau^ {2} (\lambda_ {u}) \mathrm {d} \lambda_ {u}} \boldsymbol {x} _ {t _ {i}} - \sum_ {j = - 1} ^ {s - 1} b _ {i - j} \boldsymbol {x} _ {\boldsymbol {\theta}} \left(\boldsymbol {x} _ {t _ {i - j}}, t _ {i - j}\right) \tag {71} \\ = \sigma_ {t _ {i + 1}} \int_ {t _ {i}} ^ {t _ {i + 1}} e ^ {- \int_ {\lambda_ {u}} ^ {\lambda_ {t _ {i + 1}}} \tau^ {2} (\lambda) \mathrm {d} \lambda} (1 + \tau^ {2} (u)) e ^ {\lambda_ {u}} \boldsymbol {x} _ {\boldsymbol {\theta}} (\boldsymbol {x} _ {u}, u) \frac {\mathrm {d} \lambda_ {u}}{\mathrm {d} u} \mathrm {d} u \\ - \sum_ {j = - 1} ^ {s - 1} b _ {i - j} \boldsymbol {x} _ {\boldsymbol {\theta}} \left(\boldsymbol {x} _ {t _ {i - j}}, t _ {i - j}\right). \\ \end{array}
$$
Let  $f(\pmb{x}, t) = \left( \frac{\mathrm{d} \log \alpha_{t}}{\mathrm{d} t} - \left( 1 + \tau^{2}(t) \right) \frac{\mathrm{d} \lambda_{t}}{\mathrm{d} t} \right) \pmb{x} + \left( 1 + \tau^{2}(t) \right) \alpha_{t} \pmb{x}_{\pmb{\theta}}(\pmb{x}, t) \frac{\mathrm{d} \lambda_{t}}{\mathrm{d} t}, g(t) = \tau(t) \sigma_{t} \sqrt{-2 \frac{\mathrm{d} \lambda_{t}}{\mathrm{d} t}}.$
By Ito's formula [38], we have
$$
\boldsymbol {x} _ {\boldsymbol {\theta}} \left(\boldsymbol {x} _ {u}, u\right) = \boldsymbol {x} _ {\boldsymbol {\theta}} \left(\boldsymbol {x} _ {t _ {i - (s - 1)}}, t _ {i - (s - 1)}\right) + \sum_ {k = i - (s - 1)} ^ {i - 1} \int_ {t _ {k}} ^ {t _ {k + 1}} \left(\Gamma_ {0} \left(\boldsymbol {x} _ {\boldsymbol {\theta}}\right) + \Gamma_ {1} \left(\boldsymbol {x} _ {\boldsymbol {\theta}}\right)\right) d t \tag {72}
$$
$$
+ \int_ {t _ {i}} ^ {u} \left(\Gamma_ {0} (\boldsymbol {x} _ {\boldsymbol {\theta}}) + \Gamma_ {1} (\boldsymbol {x} _ {\boldsymbol {\theta}})\right) d t + \sum_ {k = i - (s - 1)} ^ {i - 1} \int_ {t _ {k}} ^ {t _ {k + 1}} \Gamma_ {2} (\boldsymbol {x} _ {\boldsymbol {\theta}}) d \bar {\boldsymbol {w}} _ {t} + \int_ {t _ {i}} ^ {u} \Gamma_ {2} (\boldsymbol {x} _ {\boldsymbol {\theta}}) d \bar {\boldsymbol {w}} _ {t},
$$
$$
\begin{array}{l} \boldsymbol {x} _ {\boldsymbol {\theta}} \left(\boldsymbol {x} _ {t _ {i - j}}, t _ {i - j}\right) = \boldsymbol {x} _ {\boldsymbol {\theta}} \left(\boldsymbol {x} _ {t _ {i - (s - 1)}}, t _ {i - (s - 1)}\right) + \sum_ {k = i - (s - 1)} ^ {i - j - 1} \int_ {t _ {k}} ^ {t _ {k + 1}} \left(\Gamma_ {0} (\boldsymbol {x} _ {\boldsymbol {\theta}}) + \Gamma_ {1} (\boldsymbol {x} _ {\boldsymbol {\theta}})\right) d t \tag {73} \\ + \sum_ {k = i - (s - 1)} ^ {i - j - 1} \int_ {t _ {k}} ^ {t _ {k + 1}} \Gamma_ {2} (\boldsymbol {x} _ {\boldsymbol {\theta}}) \mathrm {d} \bar {\boldsymbol {w}} _ {t}, \\ \end{array}
$$
Substituting Eq. (72) and Eq. (73) into Eq. (71), we have
$$
\begin{array}{l} L _ {i + 1} \\ = \left(\sigma_ {t _ {i + 1}} \int_ {t _ {i}} ^ {t _ {i + 1}} e ^ {- \int_ {\lambda_ {u}} ^ {\lambda_ {t _ {i + 1}}} \tau^ {2} (\lambda) \mathrm {d} \lambda} (1 + \tau^ {2} (u)) e ^ {\lambda_ {u}} \frac {\mathrm {d} \lambda_ {u}}{\mathrm {d} u} \mathrm {d} u - \sum_ {j = - 1} ^ {s - 1} b _ {i - j}\right) \boldsymbol {x} _ {\boldsymbol {\theta}} \left(\boldsymbol {x} _ {t _ {i - (s - 1)}}, t _ {i - (s - 1)}\right) \\ + \sum_ {k = i - (s - 1)} ^ {i - 1} \sigma_ {t _ {i + 1}} \left(\int_ {t _ {i}} ^ {t _ {i + 1}} e ^ {- \int_ {\lambda_ {u}} ^ {\lambda_ {t _ {i + 1}}} \tau^ {2} (\lambda) \mathrm {d} \lambda} \left(1 + \tau^ {2} (u)\right) e ^ {\lambda_ {u}} \frac {\mathrm {d} \lambda_ {u}}{\mathrm {d} u} \mathrm {d} u\right) \times \\ \left(\int_ {t _ {k}} ^ {t _ {k + 1}} \left(\Gamma_ {0} \left(\boldsymbol {x} _ {\boldsymbol {\theta}}\right) + \Gamma_ {1} \left(\boldsymbol {x} _ {\boldsymbol {\theta}}\right)\right) d t\right) \\ + \sum_ {k = i - (s - 1)} ^ {i - 1} \sigma_ {t _ {i + 1}} \left(\int_ {t _ {i}} ^ {t _ {i + 1}} e ^ {- \int_ {\lambda_ {u}} ^ {\lambda_ {t _ {i + 1}}} \tau^ {2} (\lambda) \mathrm {d} \lambda} (1 + \tau^ {2} (u)) e ^ {\lambda_ {u}} \frac {\mathrm {d} \lambda_ {u}}{\mathrm {d} u} \mathrm {d} u\right) \times \left(\int_ {t _ {k}} ^ {t _ {k + 1}} \Gamma_ {2} (\boldsymbol {x} _ {\boldsymbol {\theta}}) \mathrm {d} \bar {\boldsymbol {w}} _ {t}\right) \\ + \sigma_ {t _ {i + 1}} \int_ {t _ {i}} ^ {t _ {i + 1}} e ^ {- \int_ {\lambda_ {u}} ^ {\lambda_ {t _ {i + 1}}} \tau^ {2} (\lambda) \mathrm {d} \lambda} \left(1 + \tau^ {2} (u)\right) e ^ {\lambda_ {u}} \left(\int_ {t _ {i}} ^ {u} \left(\Gamma_ {0} (\boldsymbol {x} _ {\boldsymbol {\theta}}) + \Gamma_ {1} (\boldsymbol {x} _ {\boldsymbol {\theta}})\right) \mathrm {d} t\right) \frac {\mathrm {d} \lambda_ {u}}{\mathrm {d} u} \mathrm {d} u \\ + \sigma_ {t _ {i + 1}} \int_ {t _ {i}} ^ {t _ {i + 1}} e ^ {- \int_ {\lambda_ {u}} ^ {\lambda_ {t _ {i + 1}}} \tau^ {2} (\lambda) \mathrm {d} \lambda} (1 + \tau^ {2} (u)) e ^ {\lambda_ {u}} \left(\int_ {t _ {i}} ^ {u} \Gamma_ {2} (\boldsymbol {x} _ {\boldsymbol {\theta}}) \mathrm {d} \bar {\boldsymbol {w}} _ {t}\right) \frac {\mathrm {d} \lambda_ {u}}{\mathrm {d} u} \mathrm {d} u \\ - \sum_ {j = - 1} ^ {s - 1} b _ {i - j} \left(\sum_ {k = i - (s - 1)} ^ {i - j - 1} \int_ {t _ {k}} ^ {t _ {k + 1}} \left(\Gamma_ {0} (\boldsymbol {x} _ {\boldsymbol {\theta}}) + \Gamma_ {1} (\boldsymbol {x} _ {\boldsymbol {\theta}})\right) d t + \sum_ {k = i - (s - 1)} ^ {i - j - 1} \int_ {t _ {k}} ^ {t _ {k + 1}} \Gamma_ {2} (\boldsymbol {x} _ {\boldsymbol {\theta}}) d \bar {\boldsymbol {w}} _ {t}\right) \tag {74} \\ \end{array}
$$
We will divide the local error  $L_{i + 1}$  into distinct terms. The first term has a coefficient
$$
\sigma_ {t _ {i + 1}} \int_ {t _ {i}} ^ {t _ {i + 1}} e ^ {- \int_ {\lambda_ {u}} ^ {\lambda_ {t _ {i + 1}}} \tau^ {2} (\lambda) \mathrm {d} \lambda} (1 + \tau^ {2} (u)) e ^ {\lambda_ {u}} \frac {\mathrm {d} \lambda_ {u}}{\mathrm {d} u} \mathrm {d} u - \sum_ {j = - 1} ^ {s - 1} b _ {i - j}. \tag {75}
$$
By Lemma B.10,  $b_{k}$  constructed by the integral of Lagrange polynomial in Eq. (58) and Eq. (60) satisfies  $b_{k} = \mathcal{O}(h)$  and the coefficient (75) is zero. Furthermore, we have  $g(t) = \tau(t)\sigma_t\sqrt{-2\frac{\mathrm{d}\lambda_t}{\mathrm{d}t}} =$
$\mathcal{O}\left(\max_{0\leq t\leq T}\tau (t)\right)$ . By Lemma B.9, we have the following estimations
$$
\sum_ {k = i - (s - 1)} ^ {i - 1} \sigma_ {t _ {i + 1}} \left(\int_ {t _ {i}} ^ {t _ {i + 1}} e ^ {- \int_ {\lambda_ {u}} ^ {\lambda_ {t _ {i + 1}}} \tau^ {2} (\lambda) \mathrm {d} \lambda} (1 + \tau^ {2} (u)) e ^ {\lambda_ {u}} \frac {\mathrm {d} \lambda_ {u}}{\mathrm {d} u} \mathrm {d} u\right) \times \left(\int_ {t _ {k}} ^ {t _ {k + 1}} \Gamma_ {1} (\boldsymbol {x} _ {\boldsymbol {\theta}}) \mathrm {d} t\right)
$$
$$
= \mathcal{O}\left(\max_{0\leq t\leq T}\tau^{2}(t)h^{2}\right),
$$
$$
\sum_ {k = i - (s - 1)} ^ {i - 1} \sigma_ {t _ {i + 1}} \left(\int_ {t _ {i}} ^ {t _ {i + 1}} e ^ {- \int_ {\lambda_ {u}} ^ {\lambda_ {t _ {i + 1}}} \tau^ {2} (\lambda) \mathrm {d} \lambda} (1 + \tau^ {2} (u)) e ^ {\lambda_ {u}} \frac {\mathrm {d} \lambda_ {u}}{\mathrm {d} u} \mathrm {d} u\right) \times \left(\int_ {t _ {k}} ^ {t _ {k + 1}} \Gamma_ {2} (\boldsymbol {x} _ {\boldsymbol {\theta}}) \mathrm {d} \bar {\boldsymbol {w}} _ {t}\right)
$$
$$
= \mathcal {O} \left(\max  _ {0 \leq t \leq T} \tau (t) h ^ {\frac {3}{2}}\right),
$$
$$
\sigma_ {t _ {i + 1}} \int_ {t _ {i}} ^ {t _ {i + 1}} e ^ {- \int_ {\lambda_ {u}} ^ {\lambda_ {t _ {i + 1}}} \tau^ {2} (\lambda) \mathrm {d} \lambda} \left(1 + \tau^ {2} (u)\right) e ^ {\lambda_ {u}} \left(\int_ {t _ {i}} ^ {u} \Gamma_ {1} (\boldsymbol {x} _ {\boldsymbol {\theta}}) \mathrm {d} t\right) \frac {\mathrm {d} \lambda_ {u}}{\mathrm {d} u} \mathrm {d} u = \mathcal {O} \left(\max  _ {0 \leq t \leq T} \tau^ {2} (t) h ^ {2}\right),
$$
$$
\sigma_ {t _ {i + 1}} \int_ {t _ {i}} ^ {t _ {i + 1}} e ^ {- \int_ {\lambda_ {u}} ^ {\lambda_ {t _ {i + 1}}} \tau^ {2} (\lambda) \mathrm {d} \lambda} \left(1 + \tau^ {2} (u)\right) e ^ {\lambda_ {u}} \left(\int_ {t _ {i}} ^ {u} \Gamma_ {2} (\boldsymbol {x} _ {\boldsymbol {\theta}}) \mathrm {d} \bar {\boldsymbol {w}} _ {t}\right) \frac {\mathrm {d} \lambda_ {u}}{\mathrm {d} u} \mathrm {d} u = \mathcal {O} \left(\max  _ {0 \leq t \leq T} \tau (t) h ^ {\frac {3}{2}}\right),
$$
$$
\sum_ {j = - 1} ^ {s - 1} b _ {i - j} \sum_ {k = i - (s - 1)} ^ {i - j - 1} \int_ {t _ {k}} ^ {t _ {k + 1}} \Gamma_ {1} (\boldsymbol {x} _ {\boldsymbol {\theta}}) \mathrm {d} t = \mathcal {O} \left(\max  _ {0 \leq t \leq T} \tau^ {2} (t) h ^ {2}\right),
$$
$$
\sum_ {j = - 1} ^ {s - 1} b _ {i - j} \sum_ {k = i - (s - 1)} ^ {i - j - 1} \int_ {t _ {k}} ^ {t _ {k + 1}} \Gamma_ {2} (\boldsymbol {x} _ {\boldsymbol {\theta}}) \mathrm {d} \bar {\boldsymbol {w}} _ {t} = \mathcal {O} \left(\max  _ {0 \leq t \leq T} \tau (t) h ^ {\frac {3}{2}}\right), \tag {76}
$$
and the summation of the above terms is  $S_{i + 1}^{(1)} = \mathcal{O}\left(\max_{0\leq t\leq T}\tau (t)h^{\frac{3}{2}}\right)$ .
The remaining terms of local error are
$$
\begin{array}{l} R _ {i + 1} ^ {(1)} = \sum_ {k = i - (s - 1)} ^ {i - 1} \sigma_ {t _ {i + 1}} \left(\int_ {t _ {i}} ^ {t _ {i + 1}} e ^ {- \int_ {\lambda_ {u}} ^ {\lambda_ {t _ {i + 1}}} \tau^ {2} (\lambda) \mathrm {d} \lambda} (1 + \tau^ {2} (u)) e ^ {\lambda_ {u}} \frac {\mathrm {d} \lambda_ {u}}{\mathrm {d} u} \mathrm {d} u\right) \times \left(\int_ {t _ {k}} ^ {t _ {k + 1}} \Gamma_ {0} (\boldsymbol {x} _ {\boldsymbol {\theta}}) \mathrm {d} t\right) \\ + \sigma_ {t _ {i + 1}} \int_ {t _ {i}} ^ {t _ {i + 1}} e ^ {- \int_ {\lambda_ {u}} ^ {\lambda_ {t _ {i + 1}}} \tau^ {2} (\lambda) \mathrm {d} \lambda} (1 + \tau^ {2} (u)) e ^ {\lambda_ {u}} \left(\int_ {t _ {i}} ^ {u} \Gamma_ {0} (\boldsymbol {x} _ {\boldsymbol {\theta}}) \mathrm {d} t\right) \frac {\mathrm {d} \lambda_ {u}}{\mathrm {d} u} \mathrm {d} u \\ - \sum_ {j = - 1} ^ {s - 1} b _ {i - j} \sum_ {k = i - (s - 1)} ^ {i - j - 1} \int_ {t _ {k}} ^ {t _ {k + 1}} \Gamma_ {0} (\boldsymbol {x} _ {\boldsymbol {\theta}}) \mathrm {d} t, \tag {77} \\ \end{array}
$$
The remaining problem is to estimate the  $R_{i + 1}^{(1)}$  in Eq. (70). We can further expand the term  $\Gamma_0(\pmb{x}_{\pmb{\theta}})$  as following