SlowGuess/NeurIPS_2023_OCR huggingface dataset:
where  $L'$  is the Lipschitz constant of  $\eta^E$ . We have seen in the analysis of (B.42) that  $\| v^E(X_n^E) + U_n^E + b_n^E \| = \mathcal{O}(1)$  almost surely by Assumption 3 on the event  $T = \infty$ . Therefore,  $|Y_{n+1}| = \mathcal{O}(\gamma_n) = o(\sqrt{\alpha_n})$  which implies both Lemma B.1(i) and (iv).
- Verifying Lemma B.1(ii): Let  $k' = k \| v^{E} \| + \sigma$  where  $k$  is given by Proposition B.1(iii) and  $\| v^{E} \| \coloneqq \sup \{v^{E}(x^{E}) : x^{E} \in \mathcal{U}^{E}(\mathcal{S}^{E})\}$  and  $\sigma$  is the uniform bound of  $U_{n}$ . If  $n \leq T$ , using Proposition B.1(ii), (iii), (v) and (vi) we have
$$
\begin{array}{l} \eta^ {E} \left(X _ {n + 1} ^ {E}\right) - \eta^ {E} \left(X _ {n} ^ {E}\right) \geq \gamma_ {n} \mathrm {D} \eta^ {E} \left(X _ {n} ^ {E}\right) \left(v ^ {E} \left(X _ {n} ^ {E}\right) + U _ {n} ^ {E} + b _ {n} ^ {E}\right) \\ + \mathrm {D} \eta^ {E} \left(X _ {n} ^ {E}\right) \epsilon_ {n} ^ {E} - k \gamma_ {n} ^ {2} \left(\| v ^ {E} \| + \| U _ {n} ^ {E} \| + \| b _ {n} ^ {E} \|\right) ^ {2} \\ \geq \gamma_ {n} \beta \eta^ {E} \left(X _ {n} ^ {E}\right) + \gamma_ {n} D \eta^ {E} \left(X _ {n} ^ {E}\right) U _ {n} ^ {E} + \gamma_ {n} D \eta^ {E} \left(X _ {n} ^ {E}\right) b _ {n} ^ {E} \\ + \mathrm {D} \eta^ {E} \left(X _ {n} ^ {E}\right) \epsilon_ {n} ^ {E} - 2 k ^ {\prime} \gamma_ {n} ^ {2} - 2 k \gamma_ {n} ^ {2} \| b _ {n} ^ {E} \| ^ {2}. \tag {B.48} \\ \end{array}
$$
By Assumption 3, there exists a constant  $c' > 0$  such that  $-\|b_n^E\| \geq -c'\gamma_n$  (a.s.). Combining this with the Lipschitz continuity of  $\eta^E$  and (B.42), we can merge the last four terms in (B.48) as
$$
\eta^ {E} \left(X _ {n + 1} ^ {E}\right) - \eta^ {E} \left(X _ {n} ^ {E}\right) \geq \gamma_ {n} \beta \eta^ {E} \left(X _ {n} ^ {E}\right) + \gamma_ {n} \mathrm {D} \eta^ {E} \left(X _ {n} ^ {E}\right) U _ {n} ^ {E} - 2 k ^ {\prime \prime} \gamma_ {n} ^ {2} \tag {B.49}
$$
for some constant  $k'' > 0$ . We thus get
$$
\mathbb {1} _ {\{n \leq T \}} \mathbb {E} \left[ Y _ {n + 1} \mid \mathcal {F} _ {n} \right] \geq \mathbb {1} _ {\{n \leq T \}} \left[ \gamma_ {n} \beta \eta^ {E} \left(X _ {n} ^ {E}\right) - 2 k ^ {\prime \prime} \gamma_ {n} ^ {2} + \gamma_ {n} \mathbb {E} \left[ \mathrm {D} \eta^ {E} \left(X _ {n} ^ {E}\right) U _ {n} ^ {E} \mid \mathcal {F} _ {n} \right] \right]. \tag {B.50}
$$
By Proposition B.1(ii) again, we have
$$
\begin{array}{l} \mathbb {E} \left[ \mathrm {D} \eta^ {E} \left(X _ {n} ^ {E}\right) U _ {n} ^ {E} | \mathcal {F} _ {n} \right] \geq \mathrm {D} \eta^ {E} \left(X _ {n} ^ {E}\right) \mathbb {E} \left[ U _ {n} ^ {E} | \mathcal {F} _ {n} \right] = 0 \\ = \mathrm {D} \eta^ {E} \left(X _ {n} ^ {E}\right) \mathbb {E} \left[ \mathrm {D} \iota_ {X _ {n}} U _ {n} \mid \mathcal {F} _ {n} \right] = 0 \tag {B.51} \\ \end{array}
$$
since we have assumed the noise to be zero mean. Combining (B.50) and (B.51), we then get
$$
\mathbb {1} _ {\{n \leq T \}} \mathbb {E} \left[ Y _ {n + 1} \mid \mathcal {F} _ {n} \right] \geq \mathbb {1} _ {\{n \leq T \}} \left[ \gamma_ {n} \beta \eta^ {E} \left(X _ {n} ^ {E}\right) - 2 k ^ {\prime \prime} \gamma_ {n} ^ {2} \right]. \tag {B.52}
$$
If  $n > T$ ,  $Y_{n + 1} = \gamma_n$  so trivially
$$
\mathbb {1} _ {\{n \leq T \}} \mathbb {E} \left[ Y _ {n + 1} \mid \mathcal {F} _ {n} \right] \geq 0. \tag {B.53}
$$
Combining (B.52) with (B.53), we see that Lemma B.1(ii) is satisfied with  $\varepsilon_{n} = \frac{k^{\prime\prime}}{\beta}\gamma_{n}$ .
- Verifying Lemma B.1(iii): We begin by observing that
$$
\mathbb {E} \left[ S _ {n + 1} ^ {2} - S _ {n} ^ {2} \mid \mathcal {F} _ {n} \right] = \mathbb {E} \left[ Y _ {n + 1} ^ {2} \mid \mathcal {F} _ {n} \right] + 2 S _ {n} \mathbb {E} \left[ Y _ {n + 1} \mid \mathcal {F} _ {n} \right]. \tag {B.54}
$$
If  $S_{n}\geq \varepsilon_{n}$ , then the last term on the right-hand side of (B.54) is non-negative by Lemma B.1(ii) that we just verified above. If  $S_{n} < \varepsilon_{n}$ , (B.52) with (B.53) imply that  $S_{n}\mathbb{E}[Y_{n + 1}|\mathcal{F}_{n}]\geq -\varepsilon_{n}k^{\prime \prime}\gamma_{n}^{2} = -\mathcal{O}(\gamma_{n}^{3})$ . In other words, (B.54) can be rewritten as
$$
\mathbb {E} \left[ S _ {n + 1} ^ {2} - S _ {n} ^ {2} \mid \mathcal {F} _ {n} \right] \geq \mathbb {E} \left[ Y _ {n + 1} ^ {2} \mid \mathcal {F} _ {n} \right] - \mathcal {O} \left(\gamma_ {n} ^ {3}\right). \tag {B.55}
$$
Below, we shall prove that  $\mathbb{E}[Y_{n + 1}^2 |\mathcal{F}_n]\geq b_1\gamma_n^2$  for some  $b_{1} > 0$  and  $n$  large enough. Combining this with (B.55) proves Lemma B.1(iii).
From (B.49), we deduce
$$
\mathbb {1} _ {\{n \leq T \}} \left[ \mathbb {E} \left[ \left(Y _ {n + 1}\right) _ {+} \mid \mathcal {F} _ {n} \right] - \left(\gamma_ {n} \mathbb {E} \left[ \left(\mathrm {D} \eta^ {E} \left(X _ {n} ^ {E}\right) U _ {n} ^ {E}\right) _ {+} \mid \mathcal {F} _ {n} \right] - k ^ {\prime \prime} \gamma_ {n} ^ {2}\right) \right] \geq 0. \tag {B.56}
$$
We now claim that
$$
\mathbb {1} _ {\{n \leq T \} \cap \left\{X _ {n} ^ {E} \notin \mathcal {Q} ^ {E} \right\}} \left(\mathbb {E} \left[ (\mathrm {D} \eta^ {E} \left(X _ {n} ^ {E}\right) U _ {n} ^ {E}) _ {+} \mid \mathcal {F} _ {n} \right]\right) \geq c _ {1} \zeta . \tag {B.57}
$$
where  $c_{1}$  is given by Proposition B.1(iv) and  $\zeta$  is defined in Assumption 3. To see this, recall that  $\| U_n\|_{x_n} < \sigma$  by Assumption 3. Moreover, we have  $X_{n}^{E}\in \mathcal{U}^{E}(\mathcal{S}^{E})$  on the event  $T = \infty$ . Proposition B.1(i) then implies  $\eta^E$  is differentiable on  $\mathcal{U}^E (\mathcal{S}^E)\setminus \mathcal{Q}^E$ , and Proposition B.1(iv) further shows that
$$
\begin{array}{l} \mathbb {1} _ {\{n \leq T \} \cap \left\{X _ {n} ^ {E} \notin \mathcal {Q} ^ {E} \right\}} \left(\mathbb {E} \left[ (\mathrm {D} \eta^ {E} \left(X _ {n} ^ {E}\right) U _ {n} ^ {E}) _ {+} \mid \mathcal {F} _ {n} \right]\right) \\ = \mathbb {1} _ {\{n \leq T \} \cap \left\{X _ {n} ^ {E} \notin \mathcal {Q} ^ {E} \right\}} \left(\mathbb {E} \left[ \left[ \langle \eta^ {E} \left(X _ {n} ^ {E}\right), U _ {n} ^ {E} \rangle \right] _ {+} | \mathcal {F} _ {n} \right]\right) \\ = \mathbb {1} _ {\{n \leq T \} \cap \left\{X _ {n} ^ {E} \notin \mathcal {Q} ^ {E} \right\}} \left(\mathbb {E} \left[ \left[ \left\langle \Pi_ {\mathcal {T} _ {X _ {n} ^ {E}} \mathcal {M} ^ {E}} \left(\eta^ {E} \left(X _ {n} ^ {E}\right)\right), U _ {n} ^ {E} \right\rangle \right] _ {+} | \mathcal {F} _ {n} \right]\right) \\ \geq c _ {1} \zeta \tag {B.58} \\ \end{array}
$$
where we have used the fact that  $U_n^E \in \mathcal{T}_{X_n}\mathcal{M}^E$  and Assumption 3.
If  $X_{n}^{E}\in \mathcal{Q}^{E}$ , we can choose a unit vector  $v_{n}^{E}\in \ker (I - \mathrm{D}\Pi (X_{n}^{E}))^{\perp}\cap \mathcal{T}_{X_{n}^{E}}\mathcal{M}^{E}$  where  $\Pi$  denotes the projection operator onto  $\mathcal{Q}^E$ ; note that  $v_{n}^{E}\in \ker (I - \mathrm{D}\Pi (X_{n}^{E}))^{\perp}\cap \mathcal{T}_{X_{n}^{E}}\mathcal{M}^{E}\neq \emptyset$  since  $\mathcal{E}_{x^E}^u$  contains at least one direction in  $\mathcal{T}_{x^E}\mathcal{M}^E$  for all  $x^{E}\in \mathcal{U}^{E}(\mathcal{S}^{E})\cap \mathcal{M}^{E}$ . By the definition of  $v_{n}^{E}$ , we have  $\langle U_n^E,v_n^E\rangle = \langle U_n^E -\mathrm{D}\Pi (X_n^E)U_n^E,v_n^E\rangle$ . Let  $\mathcal{H} = \{n\leq T\} \cap \{X_n^E\in \mathcal{Q}^E\}$ . By Proposition B.1(iv), Cauchy-Schwartz, and Assumption 3, we get
$$
\begin{array}{l} \mathbb {1} _ {\mathcal {H}} \mathbb {E} \left[ \left[ \mathrm {D} \eta^ {E} \left(X _ {n} ^ {E}\right) U _ {n} ^ {E} \right] _ {+} | \mathcal {F} _ {n} \right] \geq c _ {1} \mathbb {1} _ {\mathcal {H}} \mathbb {E} \left[ \| U _ {n} ^ {E} - \mathrm {D} \Pi \left(X _ {n} ^ {E}\right) U _ {n} ^ {E} \| | \mathcal {F} _ {n} \right] \\ \geq c _ {1} \mathbb {1} _ {\mathcal {H}} \mathbb {E} \left[ \left[ \langle U _ {n} ^ {E} - \mathrm {D} \Pi (X _ {n} ^ {E}) U _ {n} ^ {E}, v _ {n} ^ {E} \rangle \right] _ {+} | \mathcal {F} _ {n} \right] \\ = c _ {1} \mathbb {1} _ {\mathcal {H}} \mathbb {E} \left[ \left[ \left\langle U _ {n} ^ {E}, v _ {n} ^ {E} \right\rangle \right] _ {+} \mid \mathcal {F} _ {n} \right] \\ = c _ {1} \mathbb {1} _ {\mathcal {H}} \mathbb {E} \left[ \left[ \left\langle U _ {n}, v _ {n} \right\rangle_ {X _ {n}} \right] _ {+} \mid \mathcal {F} _ {n} \right] \tag {B.59} \\ \end{array}
$$
where  $v_{n}$  is the pullback of  $v_{n}^{E}$  under  $\iota$ . Since  $\iota$  is an isometry, the pullback preserves the inner product, and therefore
$$
\mathbb {1} _ {\mathcal {H}} \mathbb {E} \left[ \left[ \mathrm {D} \eta^ {E} \left(X _ {n} ^ {E}\right) U _ {n} ^ {E} \right] _ {+} \mid \mathcal {F} _ {n} \right] \geq c _ {1} \zeta \mathbb {1} _ {\mathcal {H}} \tag {B.60}
$$
by Assumption 3. Combining Eqs. (B.53), (B.56) and (B.60) and Item Case 1. then gives
$$
\mathbb {E} \left[ \left[ Y _ {n + 1} \right] _ {+} \mid \mathcal {F} _ {n} \right] \geq c _ {1} \zeta \gamma_ {n} - k ^ {\prime \prime} \gamma_ {n} ^ {2}. \tag {B.61}
$$
On the other hand, we always have  $\mathbb{E}[Y_{n + 1}^2 |\mathcal{F}_n]\geq \mathbb{E}[[Y_{n + 1}]_+|\mathcal{F}_n]^2$  by Jensen. It then follows that  $\mathbb{E}[Y_{n + 1}^2 |\mathcal{F}_n]\geq b_1\gamma_n^2$  for some  $b_{1} > 0$  and large enough  $n$  as desired.
We have now verified conditions (i)-(iv) in Lemma B.1. Thus, Lemma B.1 concludes that
$$
\mathbb {P} \left(\lim  _ {n \rightarrow \infty} S _ {n} = 0\right) = 0 \tag {B.62}
$$
which finishes the proof for the case of  $\sum_{n}\gamma_{n}^{2} < \infty$ .
Case 2. When  $\Sigma_n\gamma_n^2 = \infty$ , the same proof above shows that  $\mathbb{E}[Y_{n + 1}^2 |\mathcal{F}_n]\geq b_1\gamma_n^2$  for some  $b_{1} > 0$  and large enough  $n$ . Combining this with (B.55) yields
$$
\mathbb {E} \left[ S _ {n + 1} ^ {2} - S _ {n} ^ {2} \mid \mathcal {F} _ {n} \right] \geq c \gamma_ {n} ^ {2} \tag {B.63}
$$
for some  $c > 0$ . Lemma B.2 then concludes:
$$
\mathbb {P} \left(\lim  _ {n \rightarrow \infty} S _ {n} = 0\right) = 0
$$
B.3. Proof of Proposition 1. We conclude this appendix with the application of Theorem 1 to Algorithms 1-5 under the explicit oracle assumptions of Proposition 1. For convenience, we restate the relevant result below:
for all  $z \in T_x\mathcal{M}$ ,  $\|z\|_x = 1$ , and for suitable functions  $\zeta, \sigma: \mathcal{M} \to \mathbb{R}_+$  with  $\sigma$  bounded on bounded subsets of  $\mathcal{M}$  and  $\inf_x \zeta(x) > 0$ . Then, under Assumptions 2 and 4, the conclusion of Theorem 1 holds, that is, Algorithms 1-5 avoid strict saddle manifolds of  $f$ .
Remark. In addition to the claimed Algorithms 1-5, we will further prove the same conclusion for the two algorithms considered in Appendix A.
Proof. By Theorem 1, it suffices to verify Assumption 3 under (17) and the event  $\mathrm{dist}(X_n, \mathcal{S}) \to 0$ . We proceed method by method.
Algorithm 1. Since  $b_{n} = 0$  in Algorithm 1, Assumption 3 holds trivially by (17).
Algorithms 2, 3 and 5. By definition,  $\mathcal{R}_x(z)$  is a smooth map and hence satisfies  $\lim_{z\to 0}\mathcal{R}_x(z) = x$ . On the event  $\mathrm{dist}(X_n,S)\to 0$ , we have  $v(X_{n}) + U_{n} + b_{n} = \mathcal{O}(1)$ , and therefore  $X_{n + 1}$  lies in the injectivity radius of  $X_{n}$  with probability 1 for  $n$  large enough. As a result, the mapping  $\log_{X_n}(X_{n + 1})$  is well-defined for all  $n$  large enough.
We first consider Algorithms 2 and 5 whose proofs are identical since they are both the form:
$$
X _ {n + 1} = \mathcal {R} _ {X _ {n}} \left(\gamma_ {n} V \left(X _ {n}; \theta_ {n}\right)\right). \tag {B.64}
$$
Let  $\tilde{v}_n\in \mathcal{T}_{X_n}\mathcal{M}$  be the vector such that  $\exp_{X_n}(\gamma_n\tilde{v}_n) = X_{n + 1}$ , i.e.,
$$
\gamma_ {n} \tilde {v} _ {n} = \log_ {X _ {n}} \left(\mathcal {R} _ {X _ {n}} \left(\gamma_ {n} V \left(X _ {n}; \theta_ {n}\right)\right)\right). \tag {B.65}
$$
Then (B.64) is an RRM scheme with  $W_{n} = \tilde{v}_{n} - v(X_{n})$  where  $\tilde{v}_n$  is defined in (B.65). Consider the curve  $c(t)\coloneqq \mathcal{R}_{X_n}(tV(X_n;\theta_n))$ . By (17), on the event  $\mathrm{dist}(X_n,\mathcal{S})\to 0$ , the curve  $c(t)$  lies in the injectivity radius of  $X_{n}$  almost surely for all  $t\in [0,\gamma_n]$  and all  $n$  large enough. Let  $\hat{c} (t)$  be the smooth curve of  $c(t)$  in the normal coordinate with base  $X_{n}$  and an arbitrary orthonormal frame, and let  $\hat{X}_{n + 1}$  be the normal coordinate of  $X_{n + 1}$ . Also, let  $\tilde{v}_n^{\mathrm{N}}$  be the (Euclidean) vector of  $\tilde{v}_n$  expanded in the chosen orthonormal basis, and define  $V^{\mathrm{N}}(X_n;\theta_n)$  and  $\mathrm{err}^{\mathrm{N}}(X_n;\theta_n)$  similarly. By definition,  $\hat{X}_{n + 1}$  is nothing but  $\gamma_{n}\tilde{v}_{n}^{\mathrm{N}}$ . In addition, by (17), we have
$$
\left\| \operatorname {e r r} ^ {\mathrm {N}} \left(X _ {n}; \theta_ {n}\right) \right\| = \left\| \operatorname {e r r} \left(X _ {n}; \theta_ {n}\right) \right\| _ {X _ {n}} \leq \sigma \tag {B.66}
$$
for some  $\sigma <  \infty$
Since  $X_{n} = c(0)$  and  $X_{n + 1} = c(\gamma_n)$ , by properties of a retraction map we must have
$$
\begin{array}{l} \gamma_ {n} \tilde {v} _ {n} ^ {\mathrm {N}} = \hat {c} (\gamma_ {n}) \\ = \hat {c} (0) + \gamma_ {n} \dot {\hat {c}} (0) + \mathcal {O} \left(\gamma_ {n} ^ {2} \| \dot {\hat {c}} (0) \| _ {2} ^ {2}\right) \\ = \gamma_ {n} V ^ {\mathrm {N}} \left(X _ {n}; \theta_ {n}\right) + \mathcal {O} \left(\gamma_ {n} ^ {2} \| V \left(X _ {n}; \theta_ {n}\right) \| _ {X _ {n}} ^ {2}\right) \\ =: \gamma_ {n} V ^ {\mathrm {N}} \left(X _ {n}; \theta_ {n}\right) + \gamma_ {n} \tilde {b} _ {n} \tag {B.67} \\ \end{array}
$$
where  $\tilde{b}_n = \mathcal{O}\big(\gamma_n\| V(X_n;\theta_n)\|_{X_n}^2\big) = \mathcal{O}(\gamma_n)$ . Therefore,
$$
\left\| b _ {n} \right\| _ {X _ {n}} = \left\| \mathbb {E} \left[ W _ {n} \mid \mathcal {F} _ {n} \right] \right\| _ {X _ {n}} = \left\| \mathbb {E} \left[ \tilde {b} _ {n} \mid \mathcal {F} _ {n} \right] \right\| = \mathcal {O} (\gamma_ {n}) \tag {B.68}
$$
which proves the condition for  $b_{n}$  in Assumption 3. On the other hand, (B.67) shows that
$$
\begin{array}{l} \left\| U _ {n} \right\| _ {X _ {n}} \leq \left\| V ^ {\mathrm {N}} \left(X _ {n}; \theta_ {n}\right) + \tilde {b} _ {n} \right\| + \left\| \mathbb {E} \left[ V ^ {\mathrm {N}} \left(X _ {n}; \theta_ {n}\right) + \tilde {b} _ {n} \right] \right\| \\ = \mathcal {O} (1) \\ \end{array}
$$
since  $\| V^{\mathrm{N}}(X_n;\theta_n)\| = \mathcal{O}(1)$  by (17) and  $\tilde{b}_n = \mathcal{O}(\gamma_n)$ . Finally, for any unit vector  $z\in \mathcal{T}_{X_n}\mathcal{M}$ , (B.67) implies
$$
\begin{array}{l} \mathbb {E} \left[ \left[ \langle z, U _ {n} \rangle_ {x _ {n}} \right] _ {+} \right] \geq \mathbb {E} \left[ \left[ \langle z, \operatorname {e r r} \left(X _ {n}; \theta_ {n}\right) \rangle_ {x _ {n}} \right] _ {+} \right] - \| \tilde {b} _ {n} \| \\ = \mathbb {E} \left[ \left[ \left\langle z, \operatorname {e r r} \left(X _ {n}; \theta_ {n}\right) \right\rangle_ {X _ {n}} \right] _ {+} \right] - \mathcal {O} (\gamma_ {n}). \tag {B.69} \\ \end{array}
$$
Since  $\gamma_{n}\rightarrow 0$ , this finishes the proof of Algorithms 2 and 5. For Algorithm 3, an Euclidean oracle of the form (17) translates to a Riemannian oracle with  $\mathrm{err}'(x;\theta)\coloneqq \nabla^2 h(x)^{-1}\mathrm{err}(x;\theta)$ . It then suffices to note that, on the event  $\mathrm{dist}(X_n,S)\to 0$ ,  $\nabla^2 h(X_n)$  is both upper and lower bounded.
Algorithms 4, 6 and 7. For (RSEG),  $U_{n} = \Gamma_{X_{n}^{+}\to X_{n}}(\mathrm{err}(X_{n}^{+};\theta_{n}^{+}))$  so
$$
\left\| U _ {n} \right\| _ {X _ {n}} = \left\| \operatorname {e r r} \left(X _ {n} ^ {+}; \theta_ {n} ^ {+}\right) \right\| _ {X _ {n} ^ {+}} \leq \sigma \tag {B.70}
$$
by (17) and the fact that the parallel transport map is a linear isometry. For the bias term, the definition of (RSEG) yields
$$
\left\| b _ {n} \right\| = \left\| \Gamma_ {X _ {n} ^ {+} \rightarrow X _ {n}} \left(v \left(X _ {n} ^ {+}\right)\right) - v (X _ {n}) \right\| _ {X _ {n}} \leq L \operatorname {d i s t} \left(X _ {n} ^ {+}, X _ {n}\right) = \gamma_ {n} L \| V \left(X _ {n}; \theta_ {n}\right) \| _ {X _ {n}} = \mathcal {O} (1) \tag {B.71}
$$
by the same argument as for Algorithms 2, 3 and 5.
For (ROG), we have  $U_{n} = \Gamma_{X_{n}^{+}\to X_{n}}(\mathrm{err}(X_{n};\theta_{n}^{+}))$  and  $b_{n} = \Gamma_{X_{n}^{+}\to X_{n}}(v(X_{n}^{+})) - v(X_{n})$ , so Assumption 3 can be checked exactly as in the case of Algorithm 7 above. The analysis for Algorithm 6 is similar so we omit the details.