SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$$
\begin{array}{l} \| \boldsymbol {X} ^ {k + 1} - \boldsymbol {X} ^ {\star} \| _ {F} ^ {2} \leq \left(1 - \mu_ {k} \cdot \frac {n p q}{8 \delta_ {k}}\right) \| \boldsymbol {X} ^ {k} - \boldsymbol {X} ^ {\star} \| _ {F} ^ {2} + \mu_ {k} ^ {2} \| \widetilde {\nabla} _ {\mathcal {R}} f (\boldsymbol {X} ^ {k}) \| _ {F} ^ {2} \\ \leq \left(1 - \frac {p q}{8}\right) \| \boldsymbol {X} ^ {k} - \boldsymbol {X} ^ {\star} \| _ {F} ^ {2} + (1 + 2 \epsilon) ^ {2} n ^ {3} p ^ {2} q ^ {2} \mu_ {k} ^ {2}, \\ \end{array}
$$
where the last inequality is due to (46). Since  $\mu_{k} = \mathcal{O}\left(\frac{\delta_{k}}{n}\right)$  and  $\xi_0 = \Theta (\sqrt{npq}\delta_0)$  (i.e.,  $\xi_{k} = \Theta (\sqrt{npq}\delta_{k})$ , we have  $n^3 p^2 q^2\mu_k^2 = \mathcal{O}(pq\xi_k^2)$ , which implies
$$
\left\| \boldsymbol {X} ^ {k + 1} - \boldsymbol {X} ^ {\star} \right\| _ {F} ^ {2} \leq \left(1 - \frac {p q}{8}\right) \xi_ {k} ^ {2} + \frac {p q}{1 6} \xi_ {k} ^ {2} = \left(1 - \frac {p q}{1 6}\right) \xi_ {k} ^ {2} = \xi_ {k + 1} ^ {2}.
$$
Lemma 9. With high probability, suppose that  $\pmb{X}^{k}\in \mathcal{N}_{F}^{k}\cap \mathcal{N}_{\infty}^{k},\mu_{k} = \mathcal{O}\left(\frac{\delta_{k}}{n}\right)$ , and
$$
\delta_ {0} = \mathcal {O} \left(p ^ {2}\right) \quad a n d \quad \xi_ {0} = \mathcal {O} (\sqrt {n p q} \delta_ {0}), \tag {48}
$$
then  $X^{k + 1}\in \mathcal{N}_{\infty}^{k + 1}$
Proof. As stated at the beginning of Appendix B, we can assume without loss of generality that  $\pmb{R}^{\star} = \pmb{I}$  and  $\pmb{X}_i^{\star} = \pmb{I}$  for all  $1\leq i\leq n$ . We divide the index set  $[n]$  into three sets
$$
\begin{array}{l} \mathcal {I} _ {1} = \left\{i \mid \| \boldsymbol {X} _ {i} ^ {k} - \boldsymbol {I} \| _ {F} \leq \frac {\delta_ {k}}{4} \right\}, \quad \mathcal {I} _ {2} = \left\{i \mid \frac {\delta_ {k}}{4} <   \| \boldsymbol {X} _ {i} ^ {k} - \boldsymbol {I} \| _ {F} \leq \frac {3 \delta_ {k}}{4} \right\}, \\ \text {a n d} \quad \mathcal {I} _ {3} = \left\{i \mid \frac {3 \delta_ {k}}{4} <   \| \boldsymbol {X} _ {i} ^ {k} - \boldsymbol {I} \| _ {F} \leq \delta_ {k} \right\}. \\ \end{array}
$$
For any  $i\in \mathcal{I}_1\bigcup \mathcal{I}_2$  , we have
$$
\left\| \boldsymbol {X} _ {i} ^ {k} - \mu_ {k} \widetilde {\nabla} _ {\mathcal {R}} f \left(\boldsymbol {X} _ {i} ^ {k}\right) - \boldsymbol {I} \right\| _ {F} \leq \left\| \boldsymbol {X} _ {i} ^ {k} - \boldsymbol {I} \right\| + \mu_ {k} \left\| \widetilde {\nabla} _ {\mathcal {R}} f \left(\boldsymbol {X} _ {i} ^ {k}\right) \right\| \leq \frac {3 \delta_ {k}}{4} + 2 \mu_ {k} n p q \leq \delta_ {k + 1}, \tag {49}
$$
where the last inequality holds because we choose  $\mu_{k} = \mathcal{O}\left(\frac{\delta_{k}}{n}\right) \leq \frac{\delta_{k}}{16n}$ .
It remains to consider the case  $i \in \mathcal{I}_3$ . Firstly, it is easy to see that
$$
\operatorname {d i s t} \left(\boldsymbol {X} ^ {k}, \boldsymbol {X} ^ {\star}\right) ^ {2} \geq \sum_ {i \in \mathcal {I} _ {2} \bigcup \mathcal {I} _ {3}} \| \boldsymbol {X} ^ {k} - \boldsymbol {X} ^ {\star} \left(\delta_ {k}\right) \| _ {F} ^ {2} \geq \frac {\delta_ {k} ^ {2}}{1 6} \left| \mathcal {I} _ {2} \bigcup \mathcal {I} _ {3} \right|.
$$
Note that we have  $|\mathcal{I}_2 \bigcup \mathcal{I}_3| \leq \frac{\mathrm{dist}(\pmb{X}^k, \pmb{X}^\star)^2}{\delta_k^2 / 16} \leq \frac{16\xi_k^2}{\delta_k^2} = \mathcal{O}(npq)$  according to the assumption  $\xi_0 = \mathcal{O}(\sqrt{npq}\delta_0)$ . Hence, for any  $i \in \mathcal{I}_3$ , we have
$$
\widetilde {\nabla} f \left(\boldsymbol {X} _ {i}\right) = \sum_ {j \in \mathcal {A} _ {i} \cap \mathcal {I} _ {1}} \frac {\boldsymbol {X} _ {i} - \boldsymbol {X} _ {j}}{\| \boldsymbol {X} _ {i} - \boldsymbol {X} _ {j} \| _ {F}} + \sum_ {j \in \mathcal {A} _ {i} \backslash \mathcal {I} _ {1}} \frac {\boldsymbol {X} _ {i} - \boldsymbol {X} _ {j}}{\| \boldsymbol {X} _ {i} - \boldsymbol {X} _ {j} \| _ {F}} + \widetilde {\nabla} h \left(\boldsymbol {X} _ {i}\right). \tag {50}
$$
Since  $|\mathcal{I}_2 \bigcup \mathcal{I}_3| = \mathcal{O}(npq)$ , we can choose  $\xi_0$  properly such that  $|\mathcal{I}_2 \bigcup \mathcal{I}_3| \leq \epsilon npq$ . Then, we have
$$
\left| \mathcal {A} _ {i} \setminus \mathcal {I} _ {1} \right| \leq \left| \mathcal {I} _ {2} \bigcup \mathcal {I} _ {3} \right| \leq \epsilon n p q \quad \text {a n d} \quad \left| \mathcal {A} _ {i} \cap \mathcal {I} _ {1} \right| \geq \left| \mathcal {A} _ {i} \right| - \left| \mathcal {A} _ {i} \setminus \mathcal {I} _ {1} \right| \geq (1 - 2 \epsilon) n p q.
$$
Let us use  $\widetilde{\nabla} g^{1}(X_{i})$  to denote the first term on the RHS of (50). We have
$$
\begin{array}{l} \widetilde {\nabla} g ^ {1} (\boldsymbol {X} _ {i}) = \sum_ {j \in \mathcal {A} _ {i} \cap \mathcal {I} _ {1}} \frac {\boldsymbol {X} _ {i} - \boldsymbol {X} _ {j}}{\| \boldsymbol {X} _ {i} - \boldsymbol {X} _ {j} \| _ {F}} = \sum_ {j \in \mathcal {A} _ {i} \cap \mathcal {I} _ {1}} \frac {\boldsymbol {X} _ {i} - \boldsymbol {I}}{\| \boldsymbol {X} _ {i} - \boldsymbol {X} _ {j} \| _ {F}} + \frac {\boldsymbol {I} - \boldsymbol {X} _ {j}}{\| \boldsymbol {X} _ {i} - \boldsymbol {X} _ {j} \| _ {F}} \\ = \sigma (\boldsymbol {X} _ {i} - \boldsymbol {I}) + \sum_ {j \in \mathcal {A} \cap \mathcal {I} _ {1}} \frac {\boldsymbol {I} - \boldsymbol {X} _ {j}}{\| \boldsymbol {X} _ {i} - \boldsymbol {X} _ {j} \| _ {F}}, \tag {51} \\ \end{array}
$$
where  $\sigma = \sum_{j\in \mathcal{A}_i\cap \mathcal{I}_1}\frac{1}{\|\pmb{X}_i - \pmb{X}_j\|_F}$ . For the last term in the above equation, we have
$$
\left\| \sum_ {j \in \mathcal {A} _ {i} \cap \mathcal {I} _ {1}} \frac {\boldsymbol {I} - \boldsymbol {X} _ {j}}{\| \boldsymbol {X} _ {i} - \boldsymbol {X} _ {j} \| _ {F}} \right\| _ {F} \leq \sum_ {j \in \mathcal {A} _ {i} \cap \mathcal {I} _ {1}} \frac {\| \boldsymbol {I} - \boldsymbol {X} _ {j} \| _ {F}}{\| \boldsymbol {X} _ {i} - \boldsymbol {X} _ {j} \| _ {F}} \leq \frac {\delta_ {k}}{4} \sum_ {j \in \mathcal {A} _ {i} \cap \mathcal {I} _ {1}} \frac {1}{\| \boldsymbol {X} _ {i} - \boldsymbol {X} _ {j} \| _ {F}} = \frac {\delta_ {k} \sigma}{4}.
$$
In addition, the projection of  $X_{i} - I$  onto the cotangent space can be bounded as
$$
\left\| \mathcal {P} _ {\mathrm {T} _ {\boldsymbol {X} _ {i}} ^ {\perp}} \left(\boldsymbol {X} _ {i} - \boldsymbol {I}\right) \right\| _ {F} = \left\| \left(\boldsymbol {X} _ {i} - \boldsymbol {I}\right) \right\| _ {F} ^ {2} / 2 \leq \delta_ {k} ^ {2} / 2,
$$
which implies that  $\widetilde{\nabla}_{\mathcal{R}}g^{1}(\boldsymbol{X}_{i}) = \mathcal{P}_{\mathrm{T}\boldsymbol{x}_{i}}(\widetilde{\nabla} g^{1}(\boldsymbol{X}_{i}))$  satisfies
$$
\left\| \widetilde {\nabla} _ {\mathcal {R}} g ^ {1} (\boldsymbol {X} _ {i}) - \sigma (\boldsymbol {X} _ {i} - \boldsymbol {I}) \right\| _ {F} \leq \frac {\delta_ {k} \sigma}{4} + \frac {\delta_ {k} ^ {2} \sigma}{2} \leq \frac {\delta_ {k} \sigma}{2}.
$$
Then, the fact  $\widetilde{\nabla}_{\mathcal{R}}f(\boldsymbol{X}_i) = \widetilde{\nabla}_{\mathcal{R}}g^1(\boldsymbol{X}_i) + \mathcal{P}_{\mathrm{T}_{\boldsymbol{X}_i}^\perp}\left(\sum_{j \in \mathcal{A}_i \setminus \mathcal{I}_1} \frac{\boldsymbol{X}_i - \boldsymbol{X}_j}{\|\boldsymbol{X}_i - \boldsymbol{X}_j\|_F}\right) + \widetilde{\nabla}_{\mathcal{R}}h(\boldsymbol{X}_i)$  implies
$$
\begin{array}{l} \| \widetilde {\nabla} _ {\mathcal {R}} f (\boldsymbol {X} _ {i}) - \sigma (\boldsymbol {X} _ {i} - \boldsymbol {I}) \| _ {F} \leq \| \widetilde {\nabla} _ {\mathcal {R}} g ^ {1} (\boldsymbol {X} _ {i}) - \sigma (\boldsymbol {X} _ {i} - \boldsymbol {I}) \| _ {F} + | \mathcal {A} _ {i} \setminus \mathcal {I} _ {1} | + \left\| \widetilde {\nabla} _ {\mathcal {R}} h (\boldsymbol {X} _ {i}) \right\| _ {F} \\ \leq \frac {\delta_ {k} \sigma}{2} + \epsilon n p q + 5 \sqrt {2 \delta_ {0}} n q. \\ \end{array}
$$
Next, motivated by the update of ReSync, we can construct
$$
\boldsymbol {X} _ {i} ^ {k} - \mu_ {k} \widetilde {\nabla} _ {\mathcal {R}} f (\boldsymbol {X} _ {i} ^ {k}) - \boldsymbol {I} = (1 - \mu_ {k} \sigma) (\boldsymbol {X} _ {i} ^ {k} - \boldsymbol {I}) + \mu_ {k} (\widetilde {\nabla} _ {\mathcal {R}} f (\boldsymbol {X} _ {i}) - \sigma (\boldsymbol {X} _ {i} - \boldsymbol {I})),
$$
$$
\begin{array}{l} \left\| \boldsymbol {X} _ {i} ^ {k} - \mu_ {k} \widetilde {\nabla} _ {\mathcal {R}} f (\boldsymbol {X} _ {i} ^ {k}) - \boldsymbol {I} \right\| _ {F} \leq (1 - \mu_ {k} \sigma) \left\| \boldsymbol {X} _ {i} ^ {k} - \boldsymbol {I} \right\| + \mu_ {k} \left(\frac {\delta_ {k} \sigma}{4} + \frac {\delta_ {k} ^ {2} \sigma}{2} + \epsilon n p q + 5 \sqrt {2 \delta_ {0}} n q\right) \\ \leq (1 - \mu_ {k} \sigma) \delta_ {k} + \mu_ {k} \left(\frac {\delta_ {k} \sigma}{2} + \epsilon n p q + 5 \sqrt {2 \delta_ {0}} n q\right) \\ = \delta_ {k} - \mu_ {k} \left(\frac {\delta_ {k} \sigma}{2} - \epsilon n p q - 5 \sqrt {2 \delta_ {0}} n q\right). \tag {52} \\ \end{array}
$$
In order to further upper bound the above inequality, we can compute
$$
\begin{array}{l} \sigma = \sum_ {j \in \mathcal {A} _ {i} \cap \mathcal {I} _ {1}} \frac {1}{\| \boldsymbol {X} _ {i} - \boldsymbol {X} _ {j} \| _ {F}} \geq \sum_ {j \in \mathcal {A} _ {i} \cap \mathcal {I} _ {1}} \frac {1}{\| \boldsymbol {X} _ {i} - \boldsymbol {I} \| _ {F} + \| \boldsymbol {X} _ {j} - \boldsymbol {I} \| _ {F}} \\ \geq \frac {4}{5 \delta_ {k}} \left| \mathcal {A} _ {i} \cap \mathcal {I} _ {1} \right| = \frac {4 (1 - 2 \epsilon) n p q}{5 \delta_ {k}}. \\ \end{array}
$$
where the second inequality holds because  $\| X_i - I\| _F + \| X_j - I\| _F\leq 5\delta_k / 4$  as  $j\in \mathcal{I}_1$ . It implies
$$
\frac {\delta_ {k} \sigma}{2} - \epsilon n p q - 5 \sqrt {2 \delta_ {0}} n q \geq \frac {2 (1 - 2 \epsilon) n p q}{5} - \epsilon n p q - 5 \sqrt {2 \delta_ {0}} n q \geq \frac {n p q}{4}.
$$
By plugging the above bound into (52) and ignoring the high-order error term in (45), we complete the proof.
Finally, we present Lemma 10 and its proof.
Lemma 10. With high probability, the following holds for all  $\pmb{X}^{0} \in \mathcal{N}_{\infty}^{0}$ :
$$
\left\| \widetilde {\nabla} _ {\mathcal {R}} h (\boldsymbol {X} _ {i}) \right\| _ {F} \leq 5 \sqrt {2 \delta_ {0}} n q \quad \forall i \in [ n ]. \tag {53}
$$
Proof of Lemma 10. Firstly, define  $\tilde{\boldsymbol{O}}_{ij} = \boldsymbol{O}_{ij}\boldsymbol{X}_j\boldsymbol{X}_i^\top$ , then
$$
\widetilde {\nabla} h \left(\boldsymbol {X} _ {i}\right) = \sum_ {j \in \mathcal {E} _ {i} / \mathcal {A} _ {i}} \frac {\boldsymbol {X} _ {i} - \boldsymbol {O} _ {i j} \boldsymbol {X} _ {j}}{\| \boldsymbol {X} _ {i} - \boldsymbol {O} _ {i j} \boldsymbol {X} _ {j} \| _ {F}} = \sum_ {j \in \mathcal {E} _ {i} / \mathcal {A} _ {i}} \frac {\boldsymbol {I} - \boldsymbol {O} _ {i j} \boldsymbol {X} _ {j} \boldsymbol {X} _ {i} ^ {\top}}{\| \boldsymbol {X} _ {i} - \boldsymbol {O} _ {i j} \boldsymbol {X} _ {j} \| _ {F}} \boldsymbol {X} _ {i} = \sum_ {j \in \mathcal {E} _ {i} / \mathcal {A} _ {i}} \frac {\boldsymbol {I} - \tilde {\boldsymbol {O}} _ {i j}}{\| \boldsymbol {I} - \tilde {\boldsymbol {O}} _ {i j} \| _ {F}} \boldsymbol {X} _ {i}
$$
The fact that  $\mathbf{X} \in \mathcal{N}_{\infty}^{0}$  implies that  $\| \mathbf{X}_i - \mathbf{X}_j \|_F \leq 2\delta_0$ , i.e.,  $\| \mathbf{I} - \mathbf{X}_j \mathbf{X}_i^\top \|_F \leq 2\delta_0$ . For any  $\| \mathbf{O}_{ij} - \mathbf{I} \| \geq \sqrt{2\delta_0}$ , we have
$$
\begin{array}{l} \left\| \frac {\boldsymbol {I} - \tilde {\boldsymbol {O}} _ {i j}}{\| \boldsymbol {I} - \tilde {\boldsymbol {O}} _ {i j} \| _ {F}} - \frac {\boldsymbol {I} - \boldsymbol {O} _ {i j}}{\| \boldsymbol {I} - \boldsymbol {O} _ {i j} \| _ {F}} \right\| _ {F} \leq \left\| \frac {\boldsymbol {I} - \tilde {\boldsymbol {O}} _ {i j}}{\| \boldsymbol {I} - \tilde {\boldsymbol {O}} _ {i j} \| _ {F}} - \frac {\boldsymbol {I} - \tilde {\boldsymbol {O}} _ {i j}}{\| \boldsymbol {I} - \boldsymbol {O} _ {i j} \| _ {F}} \right\| _ {F} \\ + \left\| \frac {\boldsymbol {I} - \tilde {\boldsymbol {O}} _ {i j}}{\| \boldsymbol {I} - \boldsymbol {O} _ {i j} \| _ {F}} - \frac {\boldsymbol {I} - \boldsymbol {O} _ {i j}}{\| \boldsymbol {I} - \boldsymbol {O} _ {i j} \| _ {F}} \right\| _ {F} \leq 2 \sqrt {2 \delta_ {0}} \\ \end{array}
$$
where the last inequality holds because
$$
\begin{array}{l} \left\| \frac {\boldsymbol {I} - \tilde {\boldsymbol {O}} _ {i j}}{\| \boldsymbol {I} - \tilde {\boldsymbol {O}} _ {i j} \| _ {F}} - \frac {\boldsymbol {I} - \tilde {\boldsymbol {O}} _ {i j}}{\| \boldsymbol {I} - \boldsymbol {O} _ {i j} \| _ {F}} \right\| _ {F} = \| \boldsymbol {I} - \tilde {\boldsymbol {O}} _ {i j} \| _ {F} \left| \frac {1}{\| \boldsymbol {I} - \tilde {\boldsymbol {O}} _ {i j} \| _ {F}} - \frac {1}{\| \boldsymbol {I} - \boldsymbol {O} _ {i j} \| _ {F}} \right| \\ = \left| 1 - \frac {\| \boldsymbol {I} - \tilde {\boldsymbol {O}} _ {i j} \| _ {F}}{\| \boldsymbol {I} - \boldsymbol {O} _ {i j} \| _ {F}} \right| \\ = \frac {\left| \| \boldsymbol {I} - \boldsymbol {O} _ {i j} \| _ {F} - \| \boldsymbol {I} - \tilde {\boldsymbol {O}} _ {i j} \| _ {F} \right|}{\| \boldsymbol {I} - \boldsymbol {O} _ {i j} \| _ {F}} \leq \frac {\| \boldsymbol {O} _ {i j} - \tilde {\boldsymbol {O}} _ {i j} \| _ {F}}{\| \boldsymbol {I} - \boldsymbol {O} _ {i j} \| _ {F}} \\ = \frac {\left\| \boldsymbol {I} - \boldsymbol {X} _ {j} \boldsymbol {X} _ {i} ^ {\top} \right\| _ {F}}{\left\| \boldsymbol {I} - \boldsymbol {O} _ {i j} \right\| _ {F}} \leq \sqrt {2 \delta_ {0}}, \\ \end{array}
$$
$$
\left\| \frac {\boldsymbol {I} - \tilde {\boldsymbol {O}} _ {i j}}{\| \boldsymbol {I} - \boldsymbol {O} _ {i j} \| _ {F}} - \frac {\boldsymbol {I} - \boldsymbol {O} _ {i j}}{\| \boldsymbol {I} - \boldsymbol {O} _ {i j} \| _ {F}} \right\| _ {F} = \frac {\| \boldsymbol {O} _ {i j} - \tilde {\boldsymbol {O}} _ {i j} \| _ {F}}{\| \boldsymbol {I} - \boldsymbol {O} _ {i j} \| _ {F}} = \frac {\| \boldsymbol {I} - \boldsymbol {X} _ {j} \boldsymbol {X} _ {i} ^ {\top} \| _ {F}}{\| \boldsymbol {I} - \boldsymbol {O} _ {i j} \| _ {F}} \leq \sqrt {2 \delta_ {0}}.
$$
Let  $\Phi_i = \{j \in \mathcal{E}_i / \mathcal{A}_i \mid \| \boldsymbol{O}_{ij} - \boldsymbol{I} \| \leq \sqrt{2\delta_0}\}$ . According to the fact that  $|\mathcal{E}_i / \mathcal{A}_i| \leq (1 + \epsilon)nq$  and the randomness of  $\boldsymbol{O}_{ij}$ , it is easy to show that  $|\Phi_i| \leq \sqrt{2\delta_0}nq$  hold for all  $1 \leq i \leq n$  with high probability. Thus, by splitting the sum  $\sum_{j \in \mathcal{E}_i / \mathcal{A}_i}$  in two parts:  $j \in \Phi_i$  and  $j \notin \Phi_i$ , we have
$$
\left\| \sum_ {j \in \mathcal {E} _ {i} / \mathcal {A} _ {i}} \frac {\boldsymbol {I} - \tilde {\boldsymbol {O}} _ {i j}}{\| \boldsymbol {I} - \tilde {\boldsymbol {O}} _ {i j} \| _ {F}} - \sum_ {j \in \mathcal {E} _ {i} / \Omega_ {i}} \frac {\boldsymbol {I} - \boldsymbol {O} _ {i j}}{\| \boldsymbol {I} - \boldsymbol {O} _ {i j} \| _ {F}} \right\| _ {F} \leq 4 \sqrt {2 \delta_ {0}} n q. \tag {54}
$$
Besides, since  $\mathbf{O}_{ij}$  is uniformly distributed on  $\mathrm{SO}(d)$ , according to Lemma A.1 in [39], we know that  $\mathbb{E}\left\{\frac{\mathbf{I} - \mathbf{O}_{ij}}{\|\mathbf{I} - \mathbf{O}_{ij}\|_F}\right\} = c(d)\mathbf{I}$ . Then, the matrix Bernstein's inequality [38] tells us that, with high probability,
$$
\left\| \sum_ {j \in \mathcal {E} _ {i} / \Omega_ {i}} \frac {\boldsymbol {I} - \boldsymbol {O} _ {i j}}{\| \boldsymbol {I} - \boldsymbol {O} _ {i j} \| _ {F}} - | \mathcal {E} _ {i} / \Omega_ {i} | \cdot c (d) \boldsymbol {I} \right\| _ {F} \leq \sqrt {2 \delta_ {0}} n q. \tag {55}
$$
This, together with (54), implies that
$$
\left| \left| \sum_ {j \in \mathcal {E} _ {i} / \mathcal {A} _ {i}} \frac {\boldsymbol {I} - \tilde {\boldsymbol {O}} _ {i j}}{\| \boldsymbol {I} - \tilde {\boldsymbol {O}} _ {i j} \| _ {F}} - | \mathcal {E} _ {i} / \Omega_ {i} | \cdot c (d) \boldsymbol {I} \right| \right| _ {F} \leq 5 \sqrt {2 \delta_ {0}} n q.
$$
The fact that  $\widetilde{\nabla} h(\mathbf{X}_i) = \sum_{j\in \mathcal{E}_i / \mathcal{A}_i}\frac{\mathbf{I} - \tilde{\mathcal{O}}_{ij}}{\|\mathbf{I} - \tilde{\mathcal{O}}_{ij}\|_F}\mathbf{X}_i$  implies that
$$
\left. \right.\left\| \right. \widetilde {\nabla} h \left(\boldsymbol {X} _ {i}\right) - \left| \mathcal {E} _ {i} / \Omega_ {i} \right| \cdot c (d) \boldsymbol {X} _ {i} \left. \right| _ {F} \leq 5 \sqrt {2 \delta_ {0}} n q. \tag {56}
$$
We complete the proof by taking the projection operator  $\widetilde{\nabla}_{\mathcal{R}}h(\boldsymbol{X}_i) = \mathcal{P}_{\mathrm{T}\boldsymbol{X}_i}(\widetilde{\nabla} h(\boldsymbol{X}_i))$  and the fact that  $\mathcal{P}_{\mathrm{T}\boldsymbol{X}_i}(\boldsymbol{X}_i) = 0$ .