SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$$
\| C _ {\infty} \boldsymbol {S} _ {n} - \boldsymbol {S} _ {n} \boldsymbol {C} _ {\infty} \| \leq 2 \| \boldsymbol {S} _ {n} \| _ {F} \| \boldsymbol {C} _ {\infty} - \boldsymbol {C} _ {n} \| + \| \boldsymbol {C} _ {n} \boldsymbol {S} _ {n} - \boldsymbol {S} _ {n} \boldsymbol {C} _ {n} \| _ {F} \leq 2 \rho \| \boldsymbol {C} _ {\infty} - \boldsymbol {C} _ {n} \| + \delta_ {n}.
$$
Also, note that there exists  $\beta > 0$  such that  $(S_{n}\mathbf{1})_{i} \geq \beta$  for all  $i \in [m]$  as  $\bar{f}_{n} \leq \rho$  and  $\| S_{n}\|_{1,1} - \alpha \mathbf{1}^{\top} \log (S_{n}\mathbf{1}) \to \infty$  when  $S_{n} \to \mathbf{0}$ . Thus, applying Lemma E.4 to the linear system
$$
\tilde {S} := \left\{\boldsymbol {S} \in \mathbb {R} ^ {m \times m}: \boldsymbol {S C} _ {\infty} = \boldsymbol {C} _ {\infty} \boldsymbol {S}, \boldsymbol {S} \in \mathcal {S}, (\boldsymbol {S 1}) _ {i} \geq \beta , \forall i \in [ m ] \right\}
$$
yields that there exists  $\tilde{c} > 0$  such that
$$
\operatorname {d i s t} \left(\boldsymbol {S} _ {n}, \tilde {\boldsymbol {S}}\right) \leq \tilde {c} \cdot \left(2 \rho \| \boldsymbol {C} _ {\infty} - \boldsymbol {C} _ {n} \| + \delta_ {n}\right).
$$
Hence, there exists  $\tilde{S}$  in the domain of  $\bar{f}$  such that
$$
\| \boldsymbol {S} _ {n} - \tilde {\boldsymbol {S}} \| _ {F} \leq \tilde {c} \cdot (2 \rho \| \boldsymbol {C} _ {\infty} - \boldsymbol {C} _ {n} \| + \delta_ {n}) \quad \text {a n d} \quad (\tilde {\boldsymbol {S}} \mathbf {1}) _ {i} \geq \beta , \forall i \in [ m ].
$$
Since the function  $S \mapsto \|S\|_{1,1} - \alpha \mathbf{1}^\top \log(S \mathbf{1})$  is locally Lipschitz continuous when  $(S \mathbf{1})_i \geq \beta$ , there exists  $L > 0$  such that
$$
\begin{array}{l} \bar {f} (\tilde {\boldsymbol {S}}) = \| \tilde {\boldsymbol {S}} \| _ {1, 1} - \alpha \mathbf {1} ^ {\top} \log (\tilde {\boldsymbol {S}} \mathbf {1}) \leq \| \boldsymbol {S} _ {n} \| _ {1, 1} - \alpha \mathbf {1} ^ {\top} \log (\boldsymbol {S} _ {n} \mathbf {1}) + L \| \boldsymbol {S} _ {n} - \tilde {\boldsymbol {S}} \| _ {F} \\ = \bar {f} _ {n} (\boldsymbol {S} _ {n}) + L \| \boldsymbol {S} _ {n} - \tilde {\boldsymbol {S}} \| _ {F} \\ \leq \bar {f} _ {n} (\boldsymbol {S} _ {n}) + L \tilde {c} \cdot (2 \rho \| \boldsymbol {C} _ {\infty} - \boldsymbol {C} _ {n} \| + \delta_ {n}). \\ \end{array}
$$
Setting  $c_{1} \geq \max \{1, L\} \cdot \tilde{c}$ , one can obtain that for any  $\mathbf{S}_n \in [\bar{f}_n \leq \rho] \cap \mathbb{B}(\mathbf{0}, \rho)$
$$
\inf  _ {\mathbb {B} \left(\boldsymbol {S} _ {n}, \eta\right)} \bar {f} \leq \bar {f} (\tilde {\boldsymbol {S}}) \leq \bar {f} _ {n} \left(\boldsymbol {S} _ {n}\right) + c _ {1} \cdot \left(2 \rho \| \boldsymbol {C} _ {\infty} - \boldsymbol {C} _ {n} \| + \delta_ {n}\right) \leq \max  \left\{\bar {f} _ {n} \left(\boldsymbol {S} _ {n}\right), - \rho \right\} + \eta , \tag {20}
$$
where  $\eta \coloneqq c_{1} \cdot (2\rho \| C_{\infty} - C_{n} \| + \delta_{n})$ . Combining inequality (19) and (20), we can conclude that
$$
\hat {\mathrm {d}} _ {\rho} \left(\operatorname {e p i} \bar {f}, \operatorname {e p i} \bar {f} _ {n}\right) \leq c _ {1} \cdot \left(2 \rho \| C _ {\infty} - C _ {n} \| + \delta_ {n}\right). \tag {21}
$$
In order to derive the conclusion (i) and (ii), it remains to check the requirements in Lemma E.2. Since  $\rho \geq \alpha m$ , the first statement of Proposition D.1 shows that the optimal solutions to  $\inf \bar{f}$  and  $\inf \bar{f}_n$  lie in  $\mathbb{B}(\mathbf{0},\rho)$ . Since  $\rho \geq f^{*}$  and  $-\rho \leq \alpha m(1 - \log \alpha)$ , the second statement of the proposition shows that  $\inf \bar{f}, \inf \bar{f}_n \in [-\rho, \rho]$ . Hence, applying Lemma E.2 completes the proof of the first two statements.
To prove conclusion (iii), we first make the following two claims:
(a)  $S_0^* \mathbf{1}$  is a singleton, whose element is denoted by  $S^* \mathbf{1}$ ,
(b) For any  $\bar{\varepsilon} \in [0, \infty)$ , there exists a  $\delta(\bar{\varepsilon}) > 0$  such that for all  $0 \leq \varepsilon \leq \bar{\varepsilon}$  and  $S_{\varepsilon} \in S_{\varepsilon}^{*}$ , one has that
$$
\left\| \boldsymbol {S} _ {\varepsilon} \mathbf {1} - \boldsymbol {S} ^ {*} \mathbf {1} \right\| _ {2} \leq \delta (\bar {\varepsilon}) \cdot \sqrt {\varepsilon}. \tag {22}
$$
Granting these and with the help of Theorem 4.1 (ii), we can derive that for all  $S_{n}^{*}\in S^{n,*}$
$$
\begin{array}{l} \operatorname {d i s t} \left(\boldsymbol {S} _ {n} ^ {*} \mathbf {1}, \boldsymbol {S} _ {0} ^ {*} \mathbf {1}\right) = \| \boldsymbol {S} _ {n} ^ {*} \mathbf {1} - \boldsymbol {S} ^ {*} \mathbf {1} \| _ {2} \leq \| \boldsymbol {S} _ {n} ^ {*} \mathbf {1} - \boldsymbol {S} _ {2 \varepsilon_ {n}} \mathbf {1} \| _ {2} + \| \boldsymbol {S} _ {2 \varepsilon_ {n}} \mathbf {1} - \boldsymbol {S} ^ {*} \mathbf {1} \| _ {2} \\ \leq \sqrt {m} \operatorname {d i s t} \left(\boldsymbol {S} _ {n} ^ {*}, \boldsymbol {S} _ {2 \varepsilon_ {n}} ^ {*}\right) + \| \boldsymbol {S} _ {2 \varepsilon_ {n}} \mathbf {1} - \boldsymbol {S} ^ {*} \mathbf {1} \| _ {2} \\ \leq \tilde {c} _ {1} \varepsilon_ {n} + \tilde {c} _ {2} \sqrt {\varepsilon_ {n}}, \\ \end{array}
$$
where  $\tilde{c}_1, \tilde{c}_2$  are positive constants, and  $S_{2\varepsilon_n} \in S_{2\varepsilon_n}^*$  satisfies  $\| S_n^* - S_{2\varepsilon_n} \|_F = \mathrm{dist}(S_n^*, S_{2\varepsilon_n}^*)$  (whose existence is guaranteed since  $S_{\varepsilon}^*$  is convex and compact). Hence,
$$
\operatorname {d i s t} \left(\mathcal {S} ^ {n, *} \mathbf {1}, \mathcal {S} _ {0} ^ {*} \mathbf {1}\right) \leq \tilde {c} _ {1} \varepsilon_ {n} + \tilde {c} _ {2} \sqrt {\varepsilon_ {n}}.
$$
To proceed, it remains to prove the claims. Define an auxiliary function  $h: \mathbb{R}^m \to \mathbb{R}$  as  $h(\pmb{x}) = \sum_{i=1}^{m} x_i - \alpha \sum_{i=1}^{m} \log x_i$  for each  $\pmb{x} \in \mathbb{R}_+^m$ . Consider the following optimization problem:
$$
\min _ {\boldsymbol {x}} h (\boldsymbol {x})
$$
s.t.  $\pmb {x}\in \{\pmb {S1}\in \mathbb{R}^m\mid \pmb {S}$  that is feasible for LogSpecT}.
For the sake of brevity, denote the  $\varepsilon$ -suboptimal solution set of (23) as  $\mathcal{H}_{\varepsilon}^{*}$ . In the remaining part, we will first show that  $S_{\varepsilon}^{*}\mathbf{1} = \mathcal{H}_{\varepsilon}^{*}$  and then, by the strict convexity of  $h$ , the desired two claims hold.
The first step is to show that the optimal function value of the problem (23) satisfies  $h^* = f^*$ . Since it is obvious that  $\tilde{\pmb{x}} = \pmb{S}^*\mathbf{1}$  is feasible for (23),  $h^* \leq h(\tilde{\pmb{x}}) = f(\pmb{S}^*) = f^*$ . Suppose to the contrary that  $h^* < f^*$ , from the fact that the objective function is coercive and continuous and the feasible set is closed, there exists  $\tilde{\pmb{S}}$  such that it is feasible for LogSpecT and  $\pmb{x}^* = \tilde{\pmb{S}}\mathbf{1}$ , where  $\pmb{x}^*$  is an optimal solution to (23). Since  $h^* = h(\pmb{x}^*) = h(\tilde{\pmb{S}}\mathbf{1}) = f(\tilde{\pmb{S}})$ , this contradicts the fact that  $f(\tilde{\pmb{S}}) \geq f^*$ . Hence,  $h^* = f^*$ . Next, we will show that  $S_{\varepsilon}^*\mathbf{1} = \mathcal{H}_{\varepsilon}^*$ . Consider any  $\varepsilon$ -suboptimal solution  $\pmb{S} \in S_{\varepsilon}^*$ , i.e.,
$$
h (\boldsymbol {S 1}) = f (\boldsymbol {S}) \leq f ^ {*} + \varepsilon = h ^ {*} + \varepsilon .
$$
Hence,  $S\mathbf{1}\in \mathcal{H}_{\varepsilon}^{*}$  and it implies that  $S_{\varepsilon}^{*}\mathbf{1}\subseteq \mathcal{H}_{\varepsilon}^{*}$ . On the other hand, for any  $\varepsilon$ -suboptimal solution  $\pmb {x}\in \mathcal{H}_{\varepsilon}^{*}$ , there exists  $\pmb{S}$  that is feasible for LogSpecT such that  $\pmb {x} = \pmb{S}\mathbf{1}$ . Thus,
$$
f (\boldsymbol {S}) = h (\boldsymbol {x}) \leq h ^ {*} + \varepsilon = f ^ {*} + \varepsilon .
$$
This implies that  $S \in S_{\varepsilon}^{*}$  and consequently  $\mathcal{H}_{\varepsilon}^{*} \subseteq S_{\varepsilon}^{*}\mathbf{1}$ . Hence,  $\mathcal{H}_{\varepsilon}^{*} = S_{\varepsilon}^{*}\mathbf{1}$ .
Since  $h$  is strictly convex, its optimal solution set  $\mathcal{H}_0^*$  is a singleton. Then,  $S_0^* \mathbf{1} = \mathcal{H}_0^*$  is a singleton, which proves the first claim. For the second claim, we know that for any  $S_{\varepsilon} \in S_{\varepsilon}^{*}$  there exists  $x_{\varepsilon} \in \mathcal{H}_{\varepsilon}^{*}$  such that
$$
\left\| \boldsymbol {S} _ {\varepsilon} \mathbf {1} - \boldsymbol {S} ^ {*} \mathbf {1} \right\| _ {2} = \left\| \boldsymbol {x} _ {\varepsilon} - \boldsymbol {x} ^ {*} \right\| _ {2}, \tag {24}
$$
where  $\pmb{x}^{*}\in \mathcal{H}_{0}^{*}$ . The coerciveness of  $h$  asserts that  $\pmb{x}_{\varepsilon}$  and  $\pmb{x}^*$  are bounded. This together with the fact that  $h$  is strongly convex on any bounded set, illustrates that there exists  $\mu >0$  such that
$$
h \left(\boldsymbol {x} _ {\varepsilon}\right) \geq h \left(\boldsymbol {x} ^ {*}\right) + \left\langle \nabla h \left(\boldsymbol {x} ^ {*}\right), \boldsymbol {x} _ {\varepsilon} - \boldsymbol {x} ^ {*} \right\rangle + \frac {1}{\mu} \| \boldsymbol {x} _ {\varepsilon} - \boldsymbol {x} ^ {*} \| _ {2} ^ {2} \geq h \left(\boldsymbol {x} ^ {*}\right) + \frac {1}{\mu} \| \boldsymbol {x} _ {\varepsilon} - \boldsymbol {x} ^ {*} \| _ {2} ^ {2}, \tag {25}
$$
where the second inequality comes from the global optimality of  $\pmb{x}^*$ . Combining (24) and (25) gives that
$$
\| S _ {\varepsilon} \mathbf {1} - S ^ {*} \mathbf {1} \| _ {2} = \| \boldsymbol {x} _ {\varepsilon} - \boldsymbol {x} ^ {*} \| _ {2} \leq \sqrt {\mu (h (\boldsymbol {x} _ {\varepsilon}) - h (\boldsymbol {x} ^ {*}))} \leq \sqrt {\mu \varepsilon}.
$$
This completes the proof of the claims.
E.3 Proof of Corollary 4.4
Suppose to the contrary that there exists a sequence  $\{S_n^*\}_{n}$ , where the  $n$ th element is an optimal solution to rLogSpecT with sample size  $n$ , such that
$$
\operatorname {d i s t} \left(S _ {n} ^ {*}, S _ {0} ^ {*}\right) \not \to 0.
$$
From Proposition D.1, we know that  $\{S_n^*\}_{n}$  is bounded, and consequently, has a convergent subsequence. Without loss of generality, we may assume that the sequence itself is convergent and the limiting point is  $S^{*}$ . Note that
$$
\left\| \boldsymbol {C} _ {n} \boldsymbol {S} _ {n} ^ {*} - \boldsymbol {S} _ {n} ^ {*} \boldsymbol {C} _ {n} \right\| _ {F} \leq \delta_ {n}, \quad \boldsymbol {C} _ {n} \to \boldsymbol {C} _ {\infty} \text {a n d} \delta_ {n} \to 0.
$$
Hence,  $C_{\infty}S^{*} = S^{*}C_{\infty}$ . This indicates that  $S^{*}$  is feasible for LogSpecT. Then, from Theorem 4.1, we know that  $f(S_{n}^{*}) = f_{n}^{*}\rightarrow f^{*}$ , which leads to  $f(S^{*}) = f^{*}$  since  $f(\cdot) = \| \cdot \|_{1,1} - \alpha \mathbf{1}^{\top}\log (\cdot \mathbf{1})$  is continuous. Together with the fact that  $S^{*}$  is feasible, we conclude that  $S^{*}$  is an optimal solution to LogSpecT. This further implies that  $\mathrm{dist}(S_n^*,S_0^*)\to 0$ , which is a contradiction.
E.4 Proof of Lemma 4.7
Recall the generative model (1). Since  $\pmb{w}$  follows a sub-Gaussian distribution, it can be shown that for every  $t > 0$ ,
$$
\mathbb {P} (\| \boldsymbol {x} \| _ {2} > t) \leq \mathbb {P} \left(\| \boldsymbol {w} \| _ {2} > \frac {t}{\| \mathcal {H} (\boldsymbol {S}) \|}\right) \leq C e ^ {- v ^ {\prime} t ^ {2}},
$$
for some positive constant  $v'$ , which means that  $x$  also follows a sub-Gaussian distribution. Thus, due to the sub-Gaussian property,  $\| C_n - C_\infty \|$  can be explicitly bounded by the following lemma.
Lemma E.5 ([40, Proposition 2.1]). Consider sub-Gaussian, identical, independent random vectors  $\pmb{x}_1, \pmb{x}_2, \dots, \pmb{x}_n \in \mathbb{R}^m$  with  $n > m$ . Then for all  $\varepsilon > 0$ , it follows that
$$
\mathbb {P} \left(\left\| \frac {1}{n} \sum_ {i = 1} ^ {n} \boldsymbol {x} _ {i} \boldsymbol {x} _ {i} ^ {\top} - \mathbb {E} [ \boldsymbol {x} \boldsymbol {x} ^ {\top} ] \right\| _ {2} \leq \varepsilon\right) \geq 1 - 2 e ^ {2 m - l \varepsilon^ {2} n},
$$
for some constant  $l > 0$ .
Setting  $\varepsilon^2 = (4 / l)\log (2n)m / n$ , Lemma E.5 indicates that with high probability (lower bounded by  $1 - n^{-1}$ ),
$$
\left\| \boldsymbol {C} _ {n} - \boldsymbol {C} _ {\infty} \right\| \leq \mathcal {O} \left(\sqrt {\frac {\log n}{n}}\right).
$$
F Derivations of L-ADMM and Convergence Analysis
This section includes the details of L-ADMM for rLogSpecT.
F.1 Proof of Proposition 5.1
Note that the minimization problem (7) is separable for  $Z$  and  $q$ , and can be split into two subproblems:
$$
\min  _ {\mathbf {Z} \in \mathbb {B} (\mathbf {0}, \delta_ {n})} \| C _ {n} \mathbf {S} ^ {(k)} - \mathbf {S} ^ {(k)} C _ {n} + \mathbf {A} ^ {(k)} / \rho - \mathbf {Z} \| _ {F} ^ {2}, \tag {26}
$$
$$
\min  _ {\boldsymbol {q}} - \alpha \mathbf {1} ^ {\top} \log \boldsymbol {q} + \boldsymbol {\lambda} _ {2} ^ {(k) \top} (\boldsymbol {q} - \boldsymbol {S} ^ {(k)} \mathbf {1}) + \frac {\rho}{2} \| \boldsymbol {q} - \boldsymbol {S} ^ {(k)} \mathbf {1} \| _ {2} ^ {2}. \tag {27}
$$
For problem (26), the optimal solution is the projection of  $C_n S^{(k)} - S^{(k)} C_n + \Lambda^{(k)} / \rho$  onto  $\mathbb{B}(\mathbf{0}, \delta_n)$  which is given by
$$
\boldsymbol {Z} ^ {(k + 1)} = \min  \left\{1, \frac {\delta_ {n}}{\| \tilde {\boldsymbol {Z}} \| _ {F}} \right\} \tilde {\boldsymbol {Z}} \text {w i t h} \tilde {\boldsymbol {Z}} = \boldsymbol {C} _ {n} \boldsymbol {S} ^ {(k)} - \boldsymbol {S} ^ {(k)} \boldsymbol {C} _ {n} + \boldsymbol {\Lambda} ^ {(k)} / \rho .
$$
For problem (27), the first-order optimality condition gives
$$
- \alpha 1 / \boldsymbol {q} + \boldsymbol {\lambda} _ {2} ^ {(k)} + \rho (\boldsymbol {q} - \boldsymbol {S} ^ {(k)} \mathbf {1}) = 0.
$$
This together with the fact that the objective function is convex implies that
$$
\boldsymbol {q} ^ {(k + 1)} = \frac {\tilde {\boldsymbol {q}} + \sqrt {\tilde {\boldsymbol {q}} ^ {2} + 4 \alpha / \rho \mathbf {1}}}{2} \text {w i t h} \tilde {\boldsymbol {q}} = \frac {1}{\rho} (\rho \boldsymbol {S} ^ {(k)} \mathbf {1} - \boldsymbol {\lambda} _ {2} ^ {(k)}).
$$
F.2 Calculation of  $\Pi_S(\cdot)$
The projection of  $X$  to  $\mathcal{S}$  can be calculated via an optimization problem:
$$
\min  _ {\boldsymbol {S}} \| \boldsymbol {X} - \boldsymbol {S} \| _ {F} ^ {2}
$$
$$
\begin{array}{l} \text {s . t .} \quad \boldsymbol {S} ^ {\top} = \boldsymbol {S}, \end{array}
$$
$$
S _ {i i} = 0, i = 1, 2, \dots , m,
$$
$$
S _ {i j} \geq 0, \forall i, j,
$$
$$
\min  \sum_ {i <   j} \left(\left(X _ {i j} - S _ {i j}\right) ^ {2} + \left(X _ {j i} - S _ {i j}\right) ^ {2}\right)
$$
$$
\begin{array}{l l} \text {s . t .} & S _ {i j} \geq 0, \forall i <   j, \end{array}
$$
$$
S _ {i i} = 0, \forall i.
$$
$$
(\Pi_ {\mathcal {S}} (\boldsymbol {X})) _ {i j} = \left\{ \begin{array}{l l} \frac {1}{2} \max  \{0, X _ {i j} + X _ {j i} \}, & \quad i \neq j, \\ 0, & \quad i = j. \end{array} \right.
$$
F.3 Stopping criterion and updating rule of  $\rho$
We follow the procedures in [3] to update  $\rho$  in each iteration. Similarly, we define the primal residual and dual residual as follows:
$$
p _ {\mathrm {r e s}} ^ {(k + 1)} = \sqrt {\| \boldsymbol {Z} ^ {(k + 1)} - \boldsymbol {C} _ {n} \boldsymbol {S} ^ {(k + 1)} + \boldsymbol {S} ^ {(k + 1)} \boldsymbol {C} _ {n} \| _ {F} ^ {2} + \| \boldsymbol {q} ^ {(k + 1)} - \boldsymbol {S} ^ {(k + 1)} \boldsymbol {1} \| _ {2} ^ {2}},
$$
$$
d _ {\text {r e s}} ^ {(k + 1)} = \rho^ {(k)} \left(\boldsymbol {C} _ {n} (\boldsymbol {S} ^ {(k + 1)} - \boldsymbol {S} ^ {(k)}) - (\boldsymbol {S} ^ {(k + 1)} - \boldsymbol {S} ^ {(k)}) \boldsymbol {C} _ {n} + \mathbf {1} ^ {\top} (\boldsymbol {S} ^ {(k + 1)} - \boldsymbol {S} ^ {(k)}) \mathbf {1}\right).
$$
The aim of updating  $\rho$  is to control the decaying speed of  $p_{\mathrm{res}}$  and  $d_{\mathrm{res}}$  such that their difference is not too large. To this end, we update  $\rho$  adaptively following the scheme:
$$
\rho^ {(k + 1)} := \left\{ \begin{array}{l l} 2 \rho^ {(k)}, & \text {i f} p _ {\text {r e s}} ^ {(k + 1)} > 5 d _ {\text {r e s}} ^ {(k + 1)}, \\ \rho^ {(k)} / 2, & \text {i f} d _ {\text {r e s}} ^ {(k + 1)} > 5 p _ {\text {r e s}} ^ {(k + 1)}, \\ \rho^ {(k)}, & \text {o t h e r w i s e}. \end{array} \right.
$$
When  $p_{\mathrm{res}}$  and  $d_{\mathrm{res}}$  are both smaller than the threshold  $\varepsilon = 10^{-5}$ , we stop the algorithm.
F.4 Convergence analysis
Define  $D \coloneqq \operatorname{Diag}(\mathbf{1}_m^\top, \ldots, \mathbf{1}_m^\top) \in \mathbb{R}^{m \times m^2}$ . Then,  $D$  satisfies  $D\mathrm{vec}(S) = S1$  and  $\| D^\top D \| = m$ . Denote
$$
\boldsymbol {Q} := \tau \boldsymbol {I} - \boldsymbol {D} ^ {\top} \boldsymbol {D} - \boldsymbol {A} _ {n} ^ {\top} \boldsymbol {A} _ {n}.
$$
Then the linearized ADMM update (8) of  $S$  can be written as:
$$
\min  _ {\boldsymbol {S}} L (\boldsymbol {S}) + \frac {\rho}{2} \| \operatorname {v e c} (\boldsymbol {S}) - \operatorname {v e c} (\boldsymbol {S} ^ {(k)}) \| _ {\boldsymbol {Q}},
$$
where  $\| \pmb{x}\|_{\pmb{Q}}\coloneqq \pmb{x}^{\top}\pmb{Q}\pmb{x}$ . Since  $\tau >m + \| \pmb{A}_n\|^2$ , we know that  $\pmb{Q}$  is positively definite. Consequently, by treating  $(Z,\pmb {q})$  as one variable, we can apply Theorem 4.2 in [45] and directly obtain the result.
G More Experiments and Discussions on Synthetic and Real Data
G.1 Influence of graph filters
To make a fair comparison between rSpecT and rLogSpecT, we test rSpecT on BA graphs with the same graph filters and the results are reported in Figure 6. It is obvious that rSpecT fails in these cases and cannot benefit from the increase in sample size. This is reasonable since SpecT fails on BA graphs as indicated in Figure 3, let alone the approximation formulation rSpecT.
We further test rLogSpecT on ER graphs with different numbers of signals observed. The parameter  $\delta_{n}$  is set as  $20\sqrt{\log n / n}$  and the results are reported in Figure 7. The figure shows that for graph filters that are not high-pass, rLogSpecT can achieve nearly perfect recovery when the sample size is large enough. Also, compared with the performance on BA graphs, rLogSpecT works better on ER graphs. This observation is in accordance with the conclusion from Figure 3 that LogSpecT performs better on ER graphs than BA ones. We further notice that the difference between the low-pass graph filter and the high-pass one is huge. To check the conjecture that rLogSpecT generally performs better on low-pass graph filters, we choose different graph filters  $\exp (tS)$  with  $t$  ranging from  $-2$  to 2 and conduct the experiments on ER graphs. When the graph shifting operator is the adjacency matrix, the positive low-pass parameter  $t$  corresponds to low-pass graph filters and the negative  $t$  corresponds to the high-pass ones [26, 11]. We omit the case when  $t = 0$  since this filter does not contain any graph information (note that  $\exp (0S) = I$ ).
We then repeat the experiments for 50 times and report the average results in Figure 8. The comparison between the performance of low-pass graph filters and high-pass graph filters indicates that the low-pass graph filters generally outperforms the high-pass ones. A closer look at the results shows that the performance grows faster when the absolute value of  $t$  is smaller. And eventually, the graph filter with smaller absolute value of  $t$  prevails. This observation is interesting since Figure 3 indicates that the choice of graph filters has few impacts on the model performance. One explanation is that both low-pass graph filters and high-pass graph filters attenuate some frequencies of the graph and the larger absolute value of  $t$  leads to the more loss of information carried by finite signals.
G.2 Experiments on USPS dataset
USPS dataset is a handwritten digit dataset. As shown in [25], it is nearly stationary with respect to the nearest neighbour graph. This dataset collects images of handwritten digits. In the experiment, each pixel is viewed as a node and the value on the pixel is view as the graph signal. We follow the approach in [25] to construct the 20 nearest neighbour graph, on which the data is verified to be nearly stationary. More specifically, we pick the 1296 digit 1 images. The weights between two nodes are decided by the Gaussian radial basis function. It turns out that the stationarity measure  $s \coloneqq \| \mathrm{diag}(\boldsymbol{U}^\top \boldsymbol{C}_n\boldsymbol{U}) \|_2 / \| \boldsymbol{U}^\top \boldsymbol{C}_n\boldsymbol{U} \|_F$  equals 0.78 in this dataset. Here  $\boldsymbol{U}$  is the eigen-matrix of the constructed graph and  $\boldsymbol{C}_n$  is the covariance matrix of the observed graph signals. We view the 20 nearest neighbour graph as the groundtruth and compare the learned graph from rLogSpecT with it. We present the subgraphs consisting of the top 10 nodes with the largest degrees in the two graphs respectively in Figure 9. The F-measure of these two subgraphs is 0.96. This result corroborates the efficiency of our proposed rLogSpecT model.