SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$$
x _ {h} ^ {k} = \mathbb {J} _ {h \mathbf {A}} y _ {h} ^ {k - 1}
$$
$$
y _ {h} ^ {k} = \frac {k}{k + 1} (2 x _ {h} ^ {k} - y _ {h} ^ {k - 1}) + \frac {1}{k + 1} X _ {0}.
$$
We want to show for fixed  $T > 0$
$$
\lim_{h\to 0 + }\sup_{0\leq k <   \frac{T}{2h}}\left\| x^{k}_{h} - X(2hk)\right\| = 0.
$$
Equivalently we may show for fixed  $T > 0$ , for every  $\{h_n\}_{n \in \mathbb{N}}$  such that  $h_n > 0$  and converges to 0,
$$
\lim_{n\to \infty}\sup_{0\leq k <   \frac{T}{2h_{n}}}\left\| x^{k}_{h_{n}} - X(2h_{n}k)\right\| = 0.
$$
We will show this by considering inequality
$$
\begin{array}{l} \left\| x _ {h} ^ {k} - X (2 h k) \right\| \leq \| X (2 h k) - X _ {\lambda} (2 h k) \| + \| X _ {\lambda} (2 h k) - X _ {\lambda , \delta} (2 h k) \| \tag {24} \\ + \left\| X _ {\lambda , \delta} (2 h k) - X _ {\lambda , \delta} ^ {k} \right\| + \left\| X _ {\lambda , \delta} ^ {k} - x _ {\lambda} ^ {k} \right\| + \left\| x _ {h, \lambda} ^ {k} - x _ {h} ^ {k} \right\| =: S (h, \lambda , \delta , k). \\ \end{array}
$$
Our goal is to show, for every  $\{h_n\}_{n\in \mathbb{N}}$  such that  $h_n > 0$  and converges to 0, there is a sequence  $\{(\delta_n,\lambda_n)\}_{n\in \mathbb{N}}$  such that  $\lim_{n\to \infty}\sup_{0\leq k <   \frac{T}{2h_n}}S(h_n,\lambda_n,\delta_n,k) = 0$ , and thus
$$
\lim_{n\to \infty}\sup_{0\leq k <   \frac{T}{2h_{n}}}\left\| x^{k}_{h_{n}} - X(2h_{n}k)\right\| \leq \lim_{n\to \infty}\sup_{0\leq k <   \frac{T}{2h_{n}}}S\left(h_{n},\lambda_{n},\delta_{n},k\right) = 0.
$$
To clarify our goal, we need to find proper  $\{(\delta_n,\lambda_n)\}_{n\in \mathbb{N}}$  in terms of  $\{h_n\}_{n\in \mathbb{N}}$ . As  $\{h_n\}_{n\in \mathbb{N}}$  is determined,  $\left\| x_{h_n}^k -X(2h_nk)\right\|$  is fixed and doesn't change by the choice of  $\{(\delta_n,\lambda_n)\}_{n\in \mathbb{N}}$ . But if we find  $\{(\delta_n,\lambda_n)\}_{n\in \mathbb{N}}$  that makes  $S(h_{n},\lambda_{n},\delta_{n},k)$  small, since (24) holds for any choice of  $\delta ,\lambda$  right choice of  $\{(\delta_n,\lambda_n)\}_{n\in \mathbb{N}}$  can guaranteee  $\left\| x_{h_n}^k -X(2h_nk)\right\|$  is small. Thus to find such  $\{(\delta_n,\lambda_n)\}_{n\in \mathbb{N}}$ , we will observe each terms in  $S$  to find the required conditions.
Lemma D.3. For  $h, \lambda, \delta > 0$  following is true.
(i)  $\| X(2hk) - X_{\lambda}(2hk)\| = \mathcal{O}\left(\sqrt{\lambda}\right)$
(ii)  $\| X_{\lambda}(2hk) - X_{\lambda,\delta}(2hk)\| = \mathcal{O}(\delta)$
(iii)  $\left\| x_{h,\lambda}^{k} - x_{h}^{k}\right\| = O(\lambda)$ .
For  $L_{\lambda ,\delta} = \max \left\{\frac{1}{\lambda},\frac{\sqrt{2}}{\lambda}\left\| m(\mathbb{A}(X_0))\right\| ,\frac{1}{\delta},\frac{4\sqrt{2}}{\delta}\left\| m(\mathbb{A}(X_0))\right\|\right\}$
(iv)  $\left\| X_{\lambda ,\delta}\left(2hk\right) - X_{\lambda ,\delta}^{k}\right\| = \mathcal{O}\left(he^{2L_{\lambda ,\delta}T}\right).$
Further more if  $0 < \frac{h}{\lambda} < \frac{1}{2}$
(v)  $\left\| X_{\lambda ,\delta}^k -x_\lambda^k\right\| = \mathcal{O}\left(h\right) + \mathcal{O}\left(h^2 L_{\lambda ,\delta}e^{2L_{\lambda ,\delta}T}\right)$
$$
+ 3 ^ {\frac {T}{\lambda}} \left(\mathcal {O} \left(\frac {h}{\lambda}\right) + \mathcal {O} \left(\frac {h}{\lambda} e ^ {2 L _ {\lambda , \delta} T}\right) + \mathcal {O} (h) + \mathcal {O} \left(\frac {h ^ {2}}{\lambda} e ^ {2 L _ {\lambda , \delta} T}\right) + \mathcal {O} \left(\frac {e ^ {2 \delta L _ {\lambda , \delta}}}{L _ {\lambda , \delta}}\right) + \mathcal {O} (\delta)\right).
$$
We prove this lemma in next subsection, here we assume the lemma is true and prove Theorem 2.1. Suppose above lemma is true. The calculations are messy but the strategy is simple; balancing the speed of the terms  $h, \delta, \lambda$  going zero to make above terms reach zero. Above lemma motivate to take sequences as
$$
\begin{array}{l} \delta_ {n} = \min  \left\{\frac {h _ {n}}{2}, \frac {8 M T}{\log_ {3} \left(\frac {1}{h _ {n}}\right)} \right\} \\ \lambda_ {n} = \max  \left\{\delta_ {n}, \frac {\sqrt {2} \left\| m (\mathbb {A} (X _ {0})) \right\| \delta_ {n}}{M}, \frac {2 T}{\log_ {3} \left(\frac {1}{\delta_ {n}}\right)} \right\}. \\ \end{array}
$$
where  $M = \max \left\{1, 4\sqrt{2}\|m(\mathbb{A}(X_0))\|\right\}$ . When  $\lim_{n\to \infty}h_n \to 0$  with  $h_n > 0$ , we can easily check  $\lim_{n\to \infty}\delta_n = 0$  and  $\lim_{n\to \infty}\lambda_n = 0$ . So the cases (i), (ii), (iii) go to zero.
Now observe from the definition of  $\lambda_{n}$  we have,
$$
\lambda_ {n} \geq \max \left\{\frac {\sqrt {2} \| m (\mathbb {A} (X _ {0})) \| \delta_ {n}}{M}, \delta_ {n} \right\} \Rightarrow \frac {M}{\delta_ {n}} \geq \max \left\{\frac {\sqrt {2} \| m (\mathbb {A} (X _ {0})) \|}{\lambda_ {n}}, \frac {1}{\lambda_ {n}} \right\} \Rightarrow L _ {\lambda_ {n}, \delta_ {n}} = \frac {M}{\delta_ {n}}.
$$
$$
e ^ {2 L _ {\lambda , \delta} T} \leq e ^ {\frac {2 M T}{\delta n}} \leq 3 ^ {\frac {2 M T}{\delta n}} \leq 3 ^ {\frac {1}{4} \log_ {3} \left(\frac {1}{h _ {n}}\right)} = \frac {1}{h _ {n} ^ {1 / 4}}
$$
$$
3 ^ {\frac {T}{\lambda_ {n}}} \leq 3 ^ {\frac {M T}{\delta_ {n}}} \leq 3 ^ {M T \frac {\log_ {3} \left(\frac {1}{h _ {n}}\right)}{8 M T}} = \frac {1}{h _ {n} ^ {1 / 8}}
$$
$$
\frac {e ^ {2 \delta_ {n} L _ {\lambda_ {n} , \delta_ {n}}}}{L _ {\lambda_ {n} , \delta_ {n}}} = e ^ {2 M} \frac {\delta_ {n}}{M} = \mathcal {O} (\delta_ {n}).
$$
Therefore when  $\lim_{n\to \infty}h_n\to 0$
$$
h _ {n} e ^ {2 L _ {\lambda_ {n}, \delta_ {n}} T} \leq h _ {n} ^ {3 / 4} \rightarrow 0
$$
$$
h _ {n} ^ {2} L _ {\lambda_ {n}, \delta_ {n}} e ^ {2 L _ {\lambda_ {n}, \delta_ {n}} T} = \frac {h _ {n} ^ {2}}{2 T} (2 L _ {\lambda_ {n}, \delta_ {n}} T) e ^ {2 L _ {\lambda_ {n}, \delta_ {n}} T} \leq \frac {h _ {n} ^ {2}}{2 T} e ^ {4 L _ {\lambda_ {n}, \delta_ {n}} T} \leq \frac {h _ {n} ^ {3 / 2}}{2 T} \to 0
$$
$$
3 ^ {\frac {T}{\lambda_ {n}}} h _ {n} = h _ {n} ^ {7 / 8} \rightarrow 0
$$
$$
3 ^ {\frac {T}{\lambda_ {n}}} \frac {h _ {n}}{\lambda_ {n}} = 3 ^ {\frac {T}{\lambda_ {n}}} \frac {T}{\lambda_ {n}} \frac {h _ {n}}{T} \leq 3 ^ {\frac {2 T}{\lambda_ {n}}} \frac {h _ {n}}{T} \leq \frac {h _ {n} ^ {3 / 4}}{T} \rightarrow 0
$$
$$
3 ^ {\frac {T}{\lambda_ {n}}} \frac {h _ {n}}{\lambda_ {n}} e ^ {2 L _ {\lambda_ {n}, \delta_ {n}} T} \leq \frac {h _ {n} ^ {3 / 4}}{T} e ^ {2 L _ {\lambda_ {n}, \delta_ {n}} T} \leq \frac {h _ {n} ^ {1 / 2}}{T} \rightarrow 0
$$
$$
3 ^ {\frac {T}{\lambda_ {n}}} \frac {h _ {n} ^ {2}}{\lambda_ {n}} e ^ {2 L _ {\lambda_ {n}, \delta_ {n}} T} \leq \frac {h _ {n} ^ {3 / 2}}{T} \rightarrow 0
$$
$$
3 ^ {\frac {T}{\lambda_ {n}}} \delta_ {n} \leq 3 ^ {T \frac {\log_ {3} (\delta_ {n})}{2 T}} \delta_ {n} = \delta_ {n} ^ {1 / 2} \to 0.
$$
As  $\lim_{n\to \infty}h_n = 0$ , without loss of generality we may assume  $h_n < 1$ . Since  $h_n > 0$  we have  $\lambda_{n}$  is well-defined and satisfies the condition for Lemma D.3. Thus terms for the case (iv) and (v) go to zero as well. Therefore we have  $\lim_{n\to \infty}\sup_{0\leq k < \frac{T}{2h_n}}S(h_n,\lambda_n,\delta_n,k) = 0$ , as  $\{h_n\}_{n\in \mathbb{N}}$  is arbitrary, we get the desired result.
D.2.1 Proof for case (i), (ii), (iii), (iv) of Lemma D.3
As case proof for (v) need lot of work, we provide it in a different subsection and here we provide the proofs for the cases from (i) to (iv).
(i)  $\| X(2hk) - X_{\lambda}(2hk)\| = \mathcal{O}\left(\sqrt{\lambda}\right)$
This is result of Lemma B.14. Considering (20) with  $p = 1$ , taking limit  $\nu \to 0$  we know
$$
\sup  _ {t \in [ 0, T ]} \| X (t) - X _ {\lambda} (t) \| \leq \sqrt {\frac {\lambda T}{2}} \| m (\mathbb {A} (X _ {0})) \| = \mathcal {O} \left(\sqrt {\lambda}\right).
$$
(ii)  $\| X_{\lambda}(2hk) - X_{\lambda,\delta}(2hk)\| = \mathcal{O}(\delta)$
This is result of Proposition B.7. Consider (14) with  $\gamma = p = 1$  for Lemma B.6 and taking limit  $\nu \to 0$ . Then applying Lemma B.10 (iii) we have
$$
\sup  _ {t \in [ 0, T ]} \| X _ {\lambda} (t) - X _ {\lambda , \delta} (t) \| \leq 2 \sqrt {2} \delta \| \mathbb {A} _ {\lambda} (X _ {0}) \| \leq 2 \sqrt {2} \delta \| m (\mathbb {A} (X _ {0})) \| = \mathcal {O} (\delta).
$$
(iii)  $\left\| x_{h,\lambda}^{k} - x_{h}^{k}\right\| = O(\lambda)$
We first show show a general fact about Yosida approximation and resolvent. From [15, Proposition 23.7 (iv)] we have
$$
\begin{array}{l} \left\| \mathbb {J} _ {h \mathbf {A}} (x) - \mathbb {J} _ {h \mathbf {A} _ {\lambda}} (x) \right\| = \left\| \mathbb {J} _ {h \mathbf {A}} (x) - \left(\mathbb {I} + \frac {1}{1 + \lambda} \left(\mathbb {J} _ {(1 + \lambda) h \mathbf {A}} - \mathbb {I}\right)\right) (x) \right\| \\ = \left\| \mathbf {J} _ {h \mathbf {A}} (x) - \left(\frac {\lambda}{1 + \lambda} x + \frac {1}{1 + \lambda} \mathbf {J} _ {(1 + \lambda) h \mathbf {A}} (x)\right) \right\| \\ \leq \frac {1}{1 + \lambda} \left\| \mathbf {J} _ {h \mathbf {A}} (x) - \mathbf {J} _ {(1 + \lambda) h \mathbf {A}} (x) \right\| + \frac {\lambda}{1 + \lambda} \| x - \mathbf {J} _ {h \mathbf {A}} (x) \| \\ \end{array}
$$
From [15, Proposition 23.31 (iii)], we have
$$
\left\| \mathbb {J} _ {h \mathbf {A}} (x) - \mathbb {J} _ {(1 + \lambda) h \mathbf {A}} (x) \right\| \leq \lambda \| \mathbb {J} _ {h \mathbf {A}} (x) - x \|.
$$
Combining two facts we get
$$
\left\| \mathbb {J} _ {h \mathbb {A}} (x) - \mathbb {J} _ {h \mathbb {A} _ {\lambda}} (x) \right\| \leq \frac {2 \lambda}{1 + \lambda} \| x - \mathbb {J} _ {h \mathbb {A}} (x) \|.
$$
From Lemma D.2, we know the iteration of  $y^{k}$  sequence in (5) is equivalent to below sequence
$$
y ^ {k + 1} = \frac {1}{k + 1} X _ {0} + \frac {k}{k + 1} \left(2 \mathbb {J} _ {h \mathbf {A}} - \mathbb {I}\right) \left(y ^ {k}\right).
$$
Using this alternating form we have
$$
\begin{array}{l} \left\| y _ {h} ^ {k + 1} - y _ {h, \lambda} ^ {k + 1} \right\| = \frac {k}{k + 1} \left\| \left(2 \mathbb {J} _ {h \mathbb {A}} - \mathbb {I}\right) \left(y _ {h} ^ {k}\right) - \left(2 \mathbb {J} _ {h \mathbb {A} _ {\lambda}} - \mathbb {I}\right) \left(y _ {h, \lambda} ^ {k}\right) \right\| \\ \leq \frac {k}{k + 1} \left(\left\| \left(2 \mathbf {J} _ {h \mathbf {A}} - \mathbb {I}\right) \left(y _ {h} ^ {k}\right) - \left(2 \mathbf {J} _ {h \mathbf {A} _ {\lambda}} - \mathbb {I}\right) \left(y _ {h} ^ {k}\right) \right\| + \left\| \mathbb {R} _ {h \mathbf {A} _ {\lambda}} \left(y _ {h} ^ {k}\right) - \mathbb {R} _ {h \mathbf {A} _ {\lambda}} \left(y _ {h, \lambda} ^ {k}\right) \right\|\right) \\ \leq \frac {k}{k + 1} \left(2 \left\| \mathbb {J} _ {h \mathbb {A}} \left(y _ {h} ^ {k}\right) - \mathbb {J} _ {h \mathbb {A} _ {\lambda}} \left(y _ {h} ^ {k}\right) \right\| + \left\| y _ {h} ^ {k} - y _ {h, \lambda} ^ {k} \right\|\right) \\ \leq \frac {k}{k + 1} \left(\frac {4 \lambda}{1 + \lambda} \left\| y _ {h} ^ {k} - \mathbb {J} _ {h \mathbf {A}} \left(y _ {h} ^ {k}\right) \right\| + \left\| y _ {h} ^ {k} - y _ {h, \lambda} ^ {k} \right\|\right) \\ \leq \frac {k}{k + 1} \left(\frac {4 \lambda}{1 + \lambda} \frac {\| X _ {0} - X _ {\star} \|}{k} + \| y _ {h} ^ {k} - y _ {h , \lambda} ^ {k} \|\right). \\ \end{array}
$$
The first inequality comes from triangular inequality. The second inequality is from nonexpansiveness of reflected resolvent  $\mathbb{R}_{\mathrm{A}} = 2\mathbb{J}_{\mathrm{A}} - \mathbb{I}$ , [15, Corollary 23.11]. The third inequality is from the inequality shown previously. The last inequality comes from the convergence rate of APPM[35, Theorem 4.1],  $\left\| y_h^k - \mathbb{J}_{h\mathbb{A}}(y_h^k) \right\| \leq \frac{\|X_0 - X_\star\|}{k}$ .
Now multiplying both sides by  $k + 1$  and summing up from 0 to  $k$  we get
$$
(k + 1) \left\| y _ {h} ^ {k + 1} - y _ {h, \lambda} ^ {k + 1} \right\| \leq \sum_ {i = 0} ^ {k} \frac {4 \lambda}{1 + \lambda} \| X _ {0} - X _ {\star} \| = (k + 1) \frac {4 \lambda}{1 + \lambda} \| X _ {0} - X _ {\star} \|.
$$
Finally, from the relation between  $x^{k}$  and  $y^{k}$  in APPM we have
$$
\begin{array}{l} \left\| x _ {h} ^ {k + 1} - x _ {h, \lambda} ^ {k + 1} \right\| = \left\| \mathbb {J} _ {h \mathbb {A}} \left(y _ {h} ^ {k + 1}\right) - \mathbb {J} _ {h \mathbb {A} _ {\lambda}} \left(y _ {h, \lambda} ^ {k + 1}\right) \right\| \\ \leq \left\| \mathbb {J} _ {h \mathbb {A} _ {\lambda}} \left(y _ {h} ^ {k + 1}\right) - \mathbb {J} _ {h \mathbb {A} _ {\lambda}} \left(y _ {h, \lambda} ^ {k + 1}\right) \right\| + \left\| \mathbb {J} _ {h \mathbb {A}} \left(y _ {h} ^ {k + 1}\right) - \mathbb {J} _ {h \mathbb {A} _ {\lambda}} \left(y _ {h} ^ {k + 1}\right) \right\| \\ \leq \left\| y _ {h} ^ {k + 1} - y _ {h, \lambda} ^ {k + 1} \right\| + \frac {2 \lambda}{1 + \lambda} \left\| y _ {h} ^ {k + 1} - \mathbb {J} _ {h \mathbf {A}} \left(y _ {h} ^ {k + 1}\right) \right\| \\ \leq \frac {4 \lambda}{1 + \lambda} \| X _ {0} - X _ {\star} \| + \frac {2 \lambda}{1 + \lambda} \frac {\| X _ {0} - X _ {\star} \|}{k + 1} \\ = \left(2 + \frac {1}{k + 1}\right) \frac {2 \lambda}{1 + \lambda} \| X _ {0} - X _ {\star} \| = O (\lambda). \\ \end{array}
$$
(iv)  $\left\| X_{\lambda ,\delta}\left(2hk\right) - X_{\lambda ,\delta}^{k}\right\| = \mathcal{O}\left(he^{2L_{\lambda ,\delta}T}\right)$
From (23), we can consider  $\dot{X}_{\lambda, \delta}$  as of function  $F \colon \mathbb{R}^n \times [0, \infty) \to \mathbb{R}^n$  defined as below
$$
F (X, t) = \left\{ \begin{array}{l l} - \mathbb {A} _ {\lambda} (X) - \frac {1}{\delta} (X - X _ {0}) & 0 \leq t <   \delta \\ - \mathbb {A} _ {\lambda} (X) - \frac {1}{t} (X - X _ {0}) & t > \delta . \end{array} \right. \tag {25}
$$
Note  $F$  is 2 max  $\left\{\frac{1}{\lambda},\frac{1}{\delta}\right\}$  -Lipschitz with respect to  $X$
For convenience name  $\alpha = 2h$ . Define  $\epsilon_{k} \coloneqq X_{\lambda, \delta}(\alpha k) - X_{\lambda, \delta}^{k}$ . By definition of Euler discretization and from fundamental theorem of calculus, we have the following
$$
X _ {\lambda , \delta} ^ {k + 1} = X _ {\lambda , \delta} ^ {k} + \alpha F (X _ {\lambda , \delta} ^ {k}, \alpha k)
$$
$$
\begin{array}{l} X _ {\lambda , \delta} (\alpha (k + 1)) = X _ {\lambda , \delta} (\alpha k) + \int_ {\alpha k} ^ {\alpha (k + 1)} \dot {X} _ {\lambda , \delta} (t) d t \\ = X _ {\lambda , \delta} (\alpha k) + \int_ {\alpha k} ^ {\alpha (k + 1)} \left(\dot {X} _ {\lambda , \delta} (\alpha k)\right) + \int_ {\alpha k} ^ {t} \ddot {X} _ {\lambda , \delta} (s) d s) d t \\ = X _ {\lambda , \delta} (\alpha k) + \alpha F (X _ {\lambda , \delta} (\alpha k), \alpha k) + \int_ {\alpha k} ^ {\alpha (k + 1)} \int_ {\alpha k} ^ {t} \ddot {X} _ {\lambda , \delta} (s) d s d t. \\ \end{array}
$$
From Lemma B.4 we have  $F(X_{\lambda, \delta}, t) = \dot{X}_{\lambda, \delta}(t)$  is Lipschitz continuous respect to  $t$ , so  $\ddot{X}_{\lambda, \delta}$  is defined almost everywhere and fundamental theorem of calculus is valid. Therefore we have
$$
\begin{array}{l} \epsilon_ {k + 1} = X _ {\lambda , \delta} (\alpha (k + 1)) - X _ {\lambda , \delta} ^ {k + 1} \\ = X _ {\lambda , \delta} (\alpha k) - X _ {\lambda , \delta} ^ {k} + \alpha \left(F \left(X _ {\lambda , \delta} (\alpha k), \alpha k\right) - F \left(X _ {\lambda , \delta} ^ {k}, \alpha k\right)\right) + \int_ {\alpha k} ^ {\alpha (k + 1)} \int_ {\alpha k} ^ {t} \ddot {X} _ {\lambda , \delta} (s) d s d t \\ \end{array}
$$
As  $F$  is  $2\max \left\{\frac{1}{\lambda},\frac{1}{\delta}\right\}$  -Lipschitz with respect to first variable, we have
$$
\begin{array}{l} \| \epsilon_ {k + 1} \| \leq \left\| X _ {\lambda , \delta} (\alpha k) - X _ {\lambda , \delta} ^ {k} \right\| + \alpha \left\| F (X _ {\lambda , \delta} (\alpha k), \alpha k) - F (X _ {\lambda , \delta} ^ {k}, \alpha k) \right\| + \int_ {\alpha k} ^ {\alpha (k + 1)} \int_ {\alpha k} ^ {t} \left\| \ddot {X} _ {\lambda , \delta} (s) \right\| d s d t \\ \leq \left(1 + 2 \alpha \max  \left\{\frac {1}{\lambda}, \frac {1}{\delta} \right\}\right) \| \epsilon_ {k} \| + \int_ {\alpha k} ^ {\alpha (k + 1)} \int_ {\alpha k} ^ {\alpha (k + 1)} \left\| \ddot {X} _ {\lambda , \delta} (s) \right\| d s d t. \\ \end{array}
$$
Now we observe  $\left\| \ddot{X}_{\lambda ,\delta}\right\|$  is bounded. By differentiating  $\dot{X}_{\lambda ,\delta}$ , as
$$
\ddot {X} _ {\lambda , \delta} = - \frac {d}{d t} \mathbb {A} _ {\lambda} (X _ {\lambda , \delta}) + \frac {1}{t ^ {2}} \left(\dot {X} _ {\lambda , \delta} - X _ {0}\right) - \frac {1}{t} \dot {X} _ {\lambda , \delta} = - \frac {d}{d t} \mathbb {A} _ {\lambda} (X _ {\lambda , \delta}) + \frac {1}{t} \left(- \mathbb {A} _ {\lambda} (X _ {\lambda , \delta}) - \dot {X} _ {\lambda , \delta}\right) - \frac {1}{t} \dot {X} _ {\lambda , \delta}
$$
for  $t > \delta$ , we have for almost every  $t$
$$
\ddot {X} _ {\lambda , \delta} = \left\{ \begin{array}{l l} - \frac {d}{d t} \mathbb {A} _ {\lambda} (X _ {\lambda , \delta}) + \frac {1}{\delta} \dot {X} _ {\lambda , \delta} & 0 \leq t <   \delta \\ - \frac {d}{d t} \mathbb {A} _ {\lambda} (X _ {\lambda , \delta}) - \frac {1}{t} \mathbb {A} _ {\lambda} (X _ {\lambda , \delta}) - \frac {2}{t} \dot {X} _ {\lambda , \delta} & t \geq \delta \end{array} \right..
$$
Considering  $\gamma = 1$ ,  $p = 1$  to Lemma B.6 and from Lemma B.10 (iv) we have
$$
\left\| \dot {X} _ {\lambda , \delta} (t) \right\| \leq \sqrt {2} \| \mathbb {A} _ {\lambda} (X _ {0}) \| \leq \sqrt {2} \| m (\mathbb {A} (X _ {0})) \|,
$$
$$
\begin{array}{l} \| \mathbb {A} _ {\lambda} (X _ {\lambda , \delta} (t)) \| = \left\| \dot {X} _ {\lambda , \delta} (t) + \frac {1}{\max \{\delta , t \}} (X _ {\lambda , \delta} (t) - X _ {0}) \right\| \\ \leq \left\| \dot {X} _ {\lambda , \delta} (t) \right\| + \frac {1}{\max  \{\delta , t \}} \| X _ {\lambda , \delta} (t) - X _ {0} \| \\ \leq \left\| \dot {X} _ {\lambda , \delta} (t) \right\| + \frac {1}{\max \{\delta , t \}} \int_ {0} ^ {t} \left\| \dot {X} _ {\lambda , \delta} (s) \right\| d s \leq 2 \sqrt {2} \left\| m (\mathbb {A} (X _ {0})) \right\|. \\ \end{array}
$$
And since  $\mathbb{A}_{\lambda}$  is  $\frac{1}{\lambda}$ -Lipschitz, we know  $\mathbb{A}_{\lambda} \circ X_{\lambda, \delta}$  is  $\frac{1}{\lambda}\sqrt{2}\|m(\mathbb{A}(X_0))\|$ -Lipschitz, thus we have for almost all  $t$ ,
$$
\left\| \frac {d}{d t} \mathbb {A} _ {\lambda} (X _ {\lambda , \delta} (t)) \right\| \leq \frac {\sqrt {2} \| m (\mathbb {A} (X _ {0})) \|}{\lambda}.
$$
Applying these facts we have
$$
\begin{array}{l} \left\| \ddot {X} _ {\lambda , \delta} (t) \right\| \leq \left\| \frac {d}{d t} \mathbb {A} _ {\lambda} \left(X _ {\lambda , \delta} (t)\right) \right\| + \frac {1}{\delta} \| \mathbb {A} _ {\lambda} \left(X _ {\lambda , \delta} (t)\right) \| + \frac {2}{\delta} \| \dot {X} _ {\lambda , \delta} (t) \| \\ \leq \frac {\sqrt {2} \left\| m (\mathbb {A} (X _ {0})) \right\|}{\lambda} + \frac {4 \sqrt {2}}{\delta} \left\| m (\mathbb {A} (X _ {0})) \right\| \leq 2 \max \left\{\frac {\sqrt {2} \left\| m (\mathbb {A} (X _ {0})) \right\|}{\lambda}, \frac {4 \sqrt {2}}{\delta} \left\| m (\mathbb {A} (X _ {0})) \right\| \right\}. \\ \end{array}
$$
$$
\left\| \epsilon_ {k + 1} \right\| \leq \left(1 + 2 \alpha \max  \left\{\frac {1}{\lambda}, \frac {1}{\delta} \right\}\right) \| \epsilon_ {k} \| + 2 \max  \left\{\frac {\sqrt {2} \| m (\mathbb {A} (X _ {0})) \|}{\lambda}, \frac {4 \sqrt {2}}{\delta} \| m (\mathbb {A} (X _ {0})) \| \right\} \alpha^ {2} \tag {26}
$$
Now for  $L_{\delta, \lambda} = \max \left\{\frac{1}{\lambda}, \frac{\sqrt{2} \|m(\mathbb{A}(X_0))\|}{\lambda}, \frac{1}{\delta}, \frac{4\sqrt{2}}{\delta} \|m(\mathbb{A}(X_0))\|\right\}$ , we show
$$
\left\| \epsilon_ {k} \right\| \leq h e ^ {2 L _ {\lambda , \delta} T}.
$$
Multiplying  $(1 + 2\alpha L_{\lambda ,\delta})^{-(k + 1)}$  to (26) we have
$$
\left(1 + 2 \alpha L _ {\lambda , \delta}\right) ^ {- (k + 1)} \| \epsilon_ {k + 1} \| \leq \left(1 + 2 \alpha L _ {\lambda , \delta}\right) ^ {- k} \| \epsilon_ {k} \| + \left(1 + 2 \alpha L _ {\lambda , \delta}\right) ^ {- (k + 1)} L _ {\lambda , \delta} \alpha^ {2}
$$
As  $\| \epsilon_0\| = \| X_0 - X(0)\| = 0$ , summing up from 0 to  $k - 1$  we have
$$
\begin{array}{l} \left(1 + 2 \alpha L _ {\lambda , \delta}\right) ^ {- k} \| \epsilon_ {k} \| \leq \sum_ {i = 1} ^ {k} \left(1 + 2 \alpha L _ {\lambda , \delta}\right) ^ {- i} L _ {\lambda , \delta} \alpha^ {2} \\ = \frac {\left(1 + 2 \alpha L _ {\lambda , \delta}\right) ^ {- 1} \left(1 - \left(1 + 2 \alpha L _ {\lambda , \delta}\right) ^ {- k}\right)}{1 - \left(1 + 2 \alpha L _ {\lambda , \delta}\right) ^ {- 1}} L _ {\lambda , \delta} \alpha^ {2} \\ = \frac {1}{2} \left(1 - \left(1 + 2 \alpha L _ {\lambda , \delta}\right) ^ {- k}\right) \alpha . \\ \end{array}
$$
Multiplying  $(1 + 2\alpha L_{\lambda, \delta})^k$  to both sides and applying  $\alpha = 2h$  we have
$$
\left\| \epsilon_ {k} \right\| \leq \frac {\alpha}{2} \left(\left(1 + 2 \alpha L _ {\lambda , \delta}\right) ^ {k} - 1\right) = h \left(\left(1 + 4 h L _ {\lambda , \delta}\right) ^ {k} - 1\right). \tag {27}
$$
Now from
$$
(1 + 4 h L _ {\lambda , \delta}) ^ {k} \leq \left((1 + 4 h L _ {\lambda , \delta}) ^ {\frac {1}{4 h L _ {\lambda , \delta}}}\right) ^ {4 h L _ {\lambda , \delta} k} \leq e ^ {4 h L _ {\lambda , \delta} k},
$$
applying  $k\leq \frac{T}{2h}$
$$
\left\| \epsilon_ {k} \right\| \leq h \left(e ^ {4 h L _ {\lambda , \delta} k} - 1\right) \leq h e ^ {2 L _ {\lambda , \delta} T}.
$$
$$
\left\| X _ {\lambda , \delta} (2 h k) - X _ {\lambda , \delta} ^ {k} \right\| \leq h e ^ {2 L _ {\lambda , \delta} T} = \mathcal {O} \left(h e ^ {2 L _ {\lambda , \delta} T}\right). \tag {28}
$$
D.3 Proof for case (v) of Lemma D.3
As APPM has coefficient  $\frac{1}{k + 1}$ , we consider  $X_{\lambda, \delta}^{k + 1}$  instead of  $X_{\lambda, \delta}^k$  due to calculation simplicity. From triangular inequality, we have
$$
\left\| X _ {\lambda , \delta} ^ {k} - x _ {\lambda} ^ {k} \right\| \leq \left\| X _ {\lambda , \delta} ^ {k} - X _ {\lambda , \delta} ^ {k + 1} \right\| + \left\| X _ {\lambda , \delta} ^ {k + 1} - x _ {\lambda} ^ {k} \right\|.
$$
We will show
$$
\begin{array}{l} \left\| X _ {\lambda , \delta} ^ {k} - X _ {\lambda , \delta} ^ {k + 1} \right\| = \mathcal {O} (h) + \mathcal {O} \left(h ^ {2} L _ {\lambda , \delta} e ^ {2 L _ {\lambda , \delta} T}\right) \\ \left\| X _ {\lambda , \delta} ^ {k + 1} - x _ {\lambda} ^ {k} \right\| = 3 ^ {\frac {T}{\lambda}} \left(\mathcal {O} \left(\frac {h}{\lambda}\right) + \mathcal {O} \left(\frac {h}{\lambda} e ^ {2 L _ {\lambda , \delta} T}\right) + \mathcal {O} (h) + \mathcal {O} \left(\frac {h ^ {2}}{\lambda} e ^ {2 L _ {\lambda , \delta} T}\right) + \mathcal {O} \left(\frac {e ^ {2 \delta L _ {\lambda , \delta}}}{L _ {\lambda , \delta}}\right) + \mathcal {O} (\delta)\right). \\ \end{array}
$$
First one is simple. Since  $X_{\lambda, \delta}^{k+1} = X_{\lambda, \delta}^k + 2hF(X_{\lambda, \delta}, 2hk)$  and  $F$  is  $2\max \left\{\frac{1}{\lambda}, \frac{1}{\delta}\right\}$ -Lipschitz with respect to the first variable, we have
$$
\begin{array}{l} \left\| X _ {\lambda , \delta} ^ {k} - X _ {\lambda , \delta} ^ {k + 1} \right\| = 2 h \left\| F \left(X _ {\lambda , \delta} ^ {k}, 2 h k\right) \right\| \\ \leq 2 h \left\| F \left(X _ {\lambda , \delta} (2 h k), 2 h k\right) \right\| + 2 h \left\| F \left(X _ {\lambda , \delta} (2 h k), 2 h k\right) - F \left(X _ {\lambda , \delta} ^ {k}, 2 h k\right) \right\| \\ \leq 2 h \left\| \dot {X} _ {\lambda , \delta} (2 h k) \right\| + 4 h \max  \left\{\frac {1}{\lambda}, \frac {1}{\delta} \right\} \| X _ {\lambda , \delta} (2 h k) - X _ {\lambda , \delta} ^ {k} \| \\ \leq 2 \sqrt {2} h \| m (\mathbb {A} _ {\lambda} (X _ {0})) \| + 4 h ^ {2} L _ {\lambda , \delta} e ^ {2 L _ {\lambda , \delta} T} \\ = \mathcal {O} (h) + \mathcal {O} \left(h ^ {2} L _ {\lambda , \delta} e ^ {2 L _ {\lambda , \delta} T}\right). \\ \end{array}
$$
Second one is complicated, we present our proof with dividing steps to subsections.
D.3.1 Recursive inequality for  $\epsilon_{k} = \left\| X_{\lambda ,\delta}^{k + 1} - x_{h,\lambda}^{k}\right\|$
Define  $\epsilon_{k} = \left\| X_{\lambda ,\delta}^{k + 1} - x_{h,\lambda}^{k}\right\|$ . Recall,  $X_{\lambda ,\delta}$  was solution of approximated ODE
$$
\dot {X} _ {\lambda , \delta} (t) = F (X _ {\lambda , \delta}, t) = \left\{ \begin{array}{l l} - \mathbb {A} _ {\lambda} (X _ {\lambda , \delta}) (t) - \frac {1}{\delta} (X (t) - X _ {0}) & 0 \leq t <   \delta \\ - \mathbb {A} _ {\lambda} (X _ {\lambda , \delta}) (t) - \frac {1}{t} (X (t) - X _ {0}) & t \geq \delta \end{array} \right..
$$