SlowGuess/NeurIPS_2023_OCR huggingface dataset:
The proof proceeds by arguing that the term (32) is in  $O_{\mathbb{P}} \left( n^{2/q} / \sqrt{p} \right)$ , and the terms (33) and (34) are in  $O_{\mathbb{P}} \left( n^{1/q} / \sqrt{p} \right)$ , where we note that this asymptotic concerns the limit as  $n^{2/q} / \sqrt{p} \to 0$ .
In order to analyse the term (32), let  $\tilde{\mathbb{P}}$  denote the probability law of the additive model for  $\mathbf{Y}_1, \ldots, \mathbf{Y}_n$  in equation (1) in section 2.1, in the case that  $\mathbf{S}(v) = 0$  for all  $v \in \mathcal{Z}$ . Let  $\tilde{\mathbb{E}}$  denote the associated expectation. Then for any  $\delta > 0$  and  $i, j \in [n], i \neq j$ ,
$$
\begin{array}{l} \mathbb {P} \left(\left| \alpha (i, j) - \frac {1}{p} \langle \mathbf {X} (i), \mathbf {X} (j) \rangle \right| > \delta\right) \\ = \tilde {\mathbb {P}} \left(\left| \frac {1}{p} \langle \mathbf {Y} _ {i}, \mathbf {Y} _ {j} \rangle - \alpha (Z _ {i}, Z _ {j}) \right| > \delta \mid Z _ {1} = 1, \dots , Z _ {n} = n\right) \\ = \frac {\tilde {\mathbb {E}} \left[ \mathbb {I} \{Z _ {1} = 1 , \ldots , Z _ {n} = n \} \tilde {\mathbb {P}} \left(\left| \frac {1}{p} \langle \mathbf {Y} _ {i} , \mathbf {Y} _ {j} \rangle - \alpha (Z _ {i} , Z _ {j}) \right| > \delta \mid Z _ {1} , \ldots , Z _ {n}\right) \right]}{\tilde {\mathbb {P}} (\{Z _ {1} = 1 , \ldots , Z _ {n} = n \})}. \\ \end{array}
$$
The conditional probability on the r.h.s of the final equality can be upper bounded using the inequality (27) in the proof of proposition 2, and combined with the same union bound argument used immediately after (27), this establishes (32) is in  $O_{\mathbb{P}} \left( n^{2/q} / \sqrt{p} \right)$  as required.
To show that (33) and (34) are in  $O_{\mathbb{P}} \left( n^{1/q} / \sqrt{p} \right)$ , it suffices to show that  $\max_{i \in [n]} \left| p^{-1/2} \| \mathbf{X}(i) \| - 1 \right|$  is in  $O_{\mathbb{P}} \left( n^{1/q} / \sqrt{p} \right)$ . By re-arranging the equality:  $(|a| - 1)(|a| + 1) = |a|^2 - 1$ , we have
$$
\left| p ^ {- 1 / 2} \| \mathbf {X} (i) \| - 1 \right| \leq \left| p ^ {- 1} \| \mathbf {X} (i) \| ^ {2} - 1 \right| = \left| p ^ {- 1} \sum_ {j = 1} ^ {p} \Delta_ {j} (i) \right|, \tag {35}
$$
where  $\Delta_j(i) \coloneqq |X_j(i)|^2 - 1$ . Thus under the model from section E.3,  $p^{-1} \sum_{j=1}^{p} \Delta_j(i)$  is an average of  $p$  mean-zero random variables, and by the same arguments as in the proof of proposition 2 under the mixing assumption A3 and the assumption of the theorem that  $\sup_{j \geq 1} \max_{v \in \mathcal{Z}} \mathbb{E}[|X_j(v)|^{2q}] < \infty$ , combined with a union bound, we have
$$
\max _ {i \in [ n ]} \left| p ^ {- 1} \| \mathbf {X} (i) \| ^ {2} - 1 \right| \in O _ {\mathbb {P}} (n ^ {1 / q} / \sqrt {p}).
$$
Together with (35) this implies  $\max_{i\in [n]}\left|p^{-1 / 2}\| \mathbf{X}(i)\| -1\right|$  is in  $O_{\mathbb{P}}\left(n^{1 / q} / \sqrt{p}\right)$  as required, and that completes the proof.
E.4 Limitations of our modelling assumptions and failings of dot-product affinities
As noted in section 5, algorithm 1 is motivated and theoretically justified by our modelling assumptions, laid out in sections 2-3 and E.3. If these assumptions are not well matched to data in practice, then algorithm 1 may not perform well.
The proof of lemma 3 shows that, as a consequence of the conditional independence and martingale-like assumptions, A1 and A2,  $\alpha(u,v) \geq 0$  for all  $u,v$ . By theorems 2 and 3,  $\hat{\alpha}_{\mathrm{data}}$ ,  $\hat{\alpha}_{\mathrm{pca}}$  approximate  $\alpha$  (at the vertices  $\mathcal{Z}$ ) under the additive model from section 2.1, and  $\hat{\alpha}_{\mathrm{cos}}$  approximates  $\alpha$  under the multiplicative model from section E.3. Therefore if in practice  $\hat{\alpha}_{\mathrm{data}}(i,j)$ ,  $\hat{\alpha}_{\mathrm{pca}}(i,j)$  or  $\hat{\alpha}_{\mathrm{cos}}(i,j)$  are found to take non-negligible negative values for some pairs  $i,j$ , that is an indication that our modelling assumptions may not be appropriate.
Even if the values taken by  $\hat{\alpha}_{\mathrm{data}}(i,j)$ ,  $\hat{\alpha}_{\mathrm{pca}}(i,j)$  or  $\hat{\alpha}_{\mathrm{cos}}(i,j)$  are all positive in practice, there could be other failings of our modelling assumptions. As an academic but revealing example, if instead of (1) or (29), the data were to actually follow a combined additive and multiplicative noise model, i.e.,
$$
\mathbf {Y} _ {i} = \gamma_ {i} \mathbf {X} (Z _ {i}) + \mathbf {S} (Z _ {i}) \mathbf {E} _ {i},
$$
then in general neither  $\hat{\alpha}_{\mathrm{data}}$ ,  $\hat{\alpha}_{\mathrm{pca}}$  nor  $\hat{\alpha}_{\mathrm{cos}}$  would approximate  $\alpha$  as  $p \to \infty$ .