SlowGuess/NeurIPS_2023_OCR huggingface dataset:
For calculating the relative error as defined in (20), there is no additional computational cost for PGD, APGD, PQN-LBFGS and FPN, since the objective function value is already evaluated within the backtracking line search during each iteration. In contrast, BCD, optGL, and GGL do not require evaluating the objective function, leading to an extra computational cost when computing the relative error in (20) for comparison purposes. However, this additional cost can be considered negligible, as these methods only compute the objective function value after completing a cycle. Moreover, the number of cycles needed for BCD, optGL, and GGL is substantially fewer than the number of iterations required by other methods.
A.2 Financial time-series data
We carry out numerical experiments on a financial time-series dataset to evaluate the performance of our method in recovering graph edges. The applicability of MTP2 models on financial time-series data is well-established, as market factors lead to positive dependencies among stocks [5].
The dataset consists of 201 stocks composing the S&P 500 index, spanning the period from January 1, 2017, to January 1, 2020, yielding 753 observations per stock, i.e.,  $p = 201$  and  $n = 753$ . We construct the log-returns data matrix as  $X_i, j = \log P_{i,j} - \log P_{i-1,j}$ , where  $P_{i,j}$  represents the closing price of the  $j$ -th stock on the  $i$ -th day. The stocks are categorized into five sectors based on the Global Industry Classification Standard (GICS) system: Consumer Staples, Utilities, Industrials, Information Technology, and Energy.
Directly assessing the correctness of the learned graph edges is not feasible in financial time-series data, as the underlying graph structure remains unknown. However, we expect stocks from the same sector to have interconnected edges. To measure the performance of edge recovery, we employ the modularity metric [48]. Given a graph  $\mathcal{G} = (V,E)$ , where  $V$  represents the vertex set and  $E$  denotes the edge set, the modularity is defined as:
$$
Q := \frac {1}{2 | E |} \sum_ {i, j \in V} \left(A _ {i j} - \frac {d _ {i} d _ {j}}{2 | E |}\right) \delta \left(c _ {i}, c _ {j}\right), \tag {24}
$$
where  $A_{ij} = 1$  if  $(i,j) \in E$ , and 0 otherwise.  $d_i$  represents the number of edges connected to node  $i$ ,  $c_i$  indicates the type of node  $i$ , and  $\delta(\cdot, \cdot)$  refers to the Kronecker delta function, with  $\delta(a,b) = 1$  if  $a = b$  and 0 otherwise. A stock graph with high modularity exhibits dense connections among stocks within the same sector and sparse connections between stocks in distinct sectors. A higher modularity value implies a more faithful representation of the underlying stock network.
(a) Glasso,  $Q = 0.47$
(b) FPN,  $Q = 0.65$
(c) FPN  $(\mathcal{E})$ $Q = 0.67$
Figure 6: Financial time-series graphs learned via (a) Glasso, (b) FPN, and (c)  $\mathrm{FPN}(\mathcal{E})$ .
Figure 6 demonstrates that the performances of our proposed FPN and  $\mathrm{FPN}(\mathcal{E})$  are better than that of Glasso [22], since the majority of connections in graphs learned through FPN and  $\mathrm{FPN}(\mathcal{E})$  occur between nodes within the same sector. In contrast, only a few connections (depicted as gray-colored edges) exist between nodes from different sectors. Both FPN and  $\mathrm{FPN}(\mathcal{E})$  achieve higher modularity values compared to Glasso, indicating that the former have a higher degree of interpretability than the latter. Furthermore, we observe that  $\mathrm{FPN}(\mathcal{E})$  moderately enhances the performance of FPN.
We fine-tune the sparsity regularization parameter for each method based on the modularity value, allowing only a limited number of isolated nodes. Note that increasing the regularization parameter for Glasso would result in numerous isolated nodes that cannot be grouped.  $\mathrm{FPN}(\mathcal{E})$  refers to the application of FPN for solving Problem (1) with a disconnectivity set  $\mathcal{E}$ . This set is obtained through hard thresholding on the MLE, which is also used in computing regularization weights in (23).
In this section, we provide the proofs for Propositions 3.1, 3.3, 3.4, and 3.5, as well as Theorems 4.1, 4.2, 4.4, and 4.5.
Proof. The Hessian matrix at  $X_{k}$  has the form:
$$
\boldsymbol {H} _ {k} = \boldsymbol {X} _ {k} ^ {- 1} \otimes \boldsymbol {X} _ {k} ^ {- 1}.
$$
$$
\boldsymbol {H} _ {k} ^ {- 1} = \boldsymbol {X} _ {k} \otimes \boldsymbol {X} _ {k}.
$$
Following from the property of the Kronecker product that  $\operatorname{vec}(ABC) = (C^\top \otimes A)\operatorname{vec}(B)$ , we can obtain
$$
\operatorname {v e c} \left(\boldsymbol {P} _ {k}\right) = \boldsymbol {H} _ {k} ^ {- 1} \operatorname {v e c} \left(\nabla f (\boldsymbol {X} _ {k})\right) = \operatorname {v e c} \left(\boldsymbol {X} _ {k} \nabla f (\boldsymbol {X} _ {k}) \boldsymbol {X} _ {k}\right).
$$
$$
\boldsymbol {P} _ {k} = \boldsymbol {X} _ {k} \nabla f (\boldsymbol {X} _ {k}) \boldsymbol {X} _ {k},
$$
B.2 Proof of Proposition 3.3
Proof. Following from (10) and (11), we obtain
$$
\left[ \boldsymbol {P} _ {k} \right] _ {\mathcal {I} _ {k} ^ {c}} = \left[ \boldsymbol {H} _ {k} ^ {- 1} \right] _ {\mathcal {I} _ {k} ^ {c} \mathcal {T} _ {k} ^ {c}} \left[ \nabla f (\boldsymbol {X} _ {k}) \right] _ {\mathcal {I} _ {k} ^ {c}} = \left[ \boldsymbol {H} _ {k} ^ {- 1} \operatorname {v e c} \left(\mathcal {P} _ {\mathcal {I} _ {k} ^ {c}} \left(\nabla f (\boldsymbol {X} _ {k})\right)\right) \right] _ {\mathcal {I} _ {k} ^ {c}},
$$
where projection  $\mathcal{P}_{\mathcal{I}_k^c}$  is defined in (12). For ease of presentation, by a slight abuse of notation, both  $[A]_{\mathcal{I}_k^c} \in \mathbb{R}^{|\mathcal{I}_k^c|}$  and  $[\operatorname{vec}(A)]_{\mathcal{I}_k^c} \in \mathbb{R}^{|\mathcal{I}_k^c|}$  represent a vector containing all elements of  $A$  in the set  $\mathcal{I}_k^c$ . Following from the fact that  $\operatorname{vec}(ABC) = (C^\top \otimes A)\operatorname{vec}(B)$  and  $H_k^{-1} = X_k \otimes X_k$ , we have
$$
\boldsymbol {H} _ {k} ^ {- 1} \operatorname {v e c} \left(\mathcal {P} _ {\mathcal {I} _ {k} ^ {c}} (\nabla f (\boldsymbol {X} _ {k}))\right) = \operatorname {v e c} \left(\boldsymbol {X} _ {k} \mathcal {P} _ {\mathcal {I} _ {k} ^ {c}} (\nabla f (\boldsymbol {X} _ {k})) \boldsymbol {X} _ {k}\right).
$$
By collecting the elements in the set  $\mathcal{I}_k^c$ , we obtain
$$
\left[ \boldsymbol {P} _ {k} \right] _ {\mathcal {I} _ {k} ^ {c}} = \left[ \boldsymbol {X} _ {k} \mathcal {P} _ {\mathcal {I} _ {k} ^ {c}} (\nabla f (\boldsymbol {X} _ {k})) \boldsymbol {X} _ {k} \right] _ {\mathcal {I} _ {k} ^ {c}},
$$
B.3 Proof of Proposition 3.4
To prove Proposition 3.4, we first establish Lemma B.1 below to show that the lower level set of the objective function is compact. We note that Lemma B.1 depends on the condition that the sample covariance matrix has strictly positive diagonal elements, which is assumed throughout the paper and holds with probability one.
Lemma B.1. The lower level set  $L_{f}$  defined in (17) is nonempty and compact, and for any  $\mathbf{X} \in L_{f}$ , we have
$$
m \boldsymbol {I} \preceq \boldsymbol {X} \preceq M \boldsymbol {I},
$$
where  $m$  and  $M$  are two positive scalars.
Proof. We first show that the largest eigenvalue of any  $X \in L_{f}$  can be upper bounded by  $M$ . The objective function of (1) can be written as
$$
f (\boldsymbol {X}) = - \log \det  (\boldsymbol {X}) + \operatorname {t r} \left(\boldsymbol {X} \boldsymbol {G}\right),
$$
where  $G = S - \lambda$  with  $\lambda$  defined by
$$
[ \boldsymbol {\lambda} ] _ {i j} = \left\{ \begin{array}{l l} \lambda_ {i j} & \text {i f} i \neq j, \\ 0 & \text {o t h e r w i s e .} \end{array} \right. \tag {25}
$$
Define two constants
$$
\mu := \min  _ {i} G _ {i i}, \quad \text {a n d} \quad v := \max  _ {i \neq j, (i, j) \notin \mathcal {E}} | G _ {i j} |.
$$
Note that  $\mu > 0$  because  $S_{ii} > 0$  for each  $i \in [p]$ . Then one has
$$
\operatorname {t r} (\boldsymbol {X} \boldsymbol {G}) = \sum_ {i} G _ {i i} X _ {i i} + \sum_ {i \neq j} G _ {i j} X _ {i j} \geq \mu \operatorname {t r} (\boldsymbol {X}) + v \sum_ {i \neq j} X _ {i j}, \tag {26}
$$
where the inequality follows from the fact that  $X_{ij} \leq 0$  for  $i \neq j$ , and  $X_{ij} = 0$  for  $(i,j) \in \mathcal{E}$ . We denote the largest eigenvalue of  $X$  by  $\lambda_{\max}(X)$ . Then we have
$$
\lambda_ {\max } (\boldsymbol {X}) \leq \operatorname {t r} (\boldsymbol {X}) \leq \frac {1}{\mu} \left(\operatorname {t r} \left(\boldsymbol {X} \boldsymbol {G}\right) + v \sum_ {i \neq j} | X _ {i j} |\right), \tag {27}
$$
where the second inequality follows from (26). In what follows, we bound the terms  $\operatorname{tr}(\mathbf{X}\mathbf{G})$  and  $\sum_{i\neq j}X_{ij}$ . For any  $\mathbf{X}\in L_f$ , one has  $f(\mathbf{X}^o)\geq -\log \det (\mathbf{X}) + \operatorname {tr}(\mathbf{X}\mathbf{G})$ . Therefore, the term  $\operatorname {tr}(\mathbf{X}\mathbf{G})$  can be bounded by
$$
\operatorname {t r} \left(\boldsymbol {X} \boldsymbol {G}\right) \leq f \left(\boldsymbol {X} ^ {o}\right) + \log \det  (\boldsymbol {X}) \leq f \left(\boldsymbol {X} ^ {o}\right) + p \log \left(\lambda_ {\max } (\boldsymbol {X})\right). \tag {28}
$$
Let  $\lambda^{\prime}:= \min_{i\neq j,(i,j)\notin \mathcal{E}}\lambda_{ij}$ . Then one has
$$
\sum_ {i \neq j} | X _ {i j} | \leq \frac {1}{\lambda^ {\prime}} \sum_ {i \neq j} \lambda_ {i j} | X _ {i j} | \leq \frac {1}{\lambda^ {\prime}} \left(f \left(\boldsymbol {X} ^ {o}\right) + p \log \left(\lambda_ {\max } (\boldsymbol {X})\right)\right), \tag {29}
$$
where the last inequality follows from  $\sum_{i\neq j}\lambda_{ij}|X_{ij}|\leq \operatorname {tr}(\boldsymbol {X}\boldsymbol {G})$ , because  $\operatorname {tr}(\boldsymbol {X}\boldsymbol {S})\geq 0$  since  $\pmb {X}\in \mathbb{S}_{++}^{p}$  and  $\pmb {S}\in \mathbb{S}_{+}^{p}$ . Together with (27), (28) and (29), we obtain
$$
\lambda_ {\max } (\boldsymbol {X}) \leq \frac {1}{\mu} \left(1 + \frac {v}{\lambda^ {\prime}}\right) \left(f \left(\boldsymbol {X} ^ {o}\right) + p \log \left(\lambda_ {\max } (\boldsymbol {X})\right)\right).
$$
Since  $\log (\lambda_{\max}(\pmb {X}))$  grows much slower than  $\lambda_{\max}(\pmb {X})$ ,  $\lambda_{\max}(\pmb {X})$  can be upper bounded by a constant  $M$ , which depends on  $f(\pmb{X}^o)$ ,  $\pmb{S}$ , and  $\pmb{\lambda}$ .
We denote the smallest eigenvalue of  $\mathbf{X}$  by  $\lambda_{\min}(\mathbf{X})$ . For any  $\mathbf{X} \in L_f$ , one has
$$
f \left(\boldsymbol {X} ^ {o}\right) \geq f (\boldsymbol {X}) \geq - \log \det  (\boldsymbol {X}) \geq - \log \lambda_ {\min } (\boldsymbol {X}) - (p - 1) \log M.
$$
As a result, we have  $\lambda_{\min}(\pmb{X}) \geq e^{-f(\pmb{X}^o)} M^{-(p-1)}$ , which shows that  $\lambda_{\min}(\pmb{X})$  can be lower bounded by a positive constant  $m = e^{-f(\pmb{X}^o)} M^{-(p-1)}$ . Finally, we show that the lower level set  $L_f$  is compact. First,  $L_f$  is closed because  $f$  is a continuous function. Second,  $L_f$  is also bounded because it is a subset of  $\{\pmb{X} \in \mathbb{R}^{p \times p} | m\pmb{I} \preceq \pmb{X} \preceq M\pmb{I}\}$ .
Proof of Proposition 3.4. We first show that  $\mathbf{X}_k(\gamma_k) \in \mathbb{S}_{++}$  holds for a small enough step size. The  $\mathbf{X}_k(\gamma_k)$  can be equivalently written in the form  $\mathbf{X}_k(\gamma_k) = \mathbf{Z}_k - \gamma_k \mathbf{G}_k$ , where  $\mathbf{Z}_k \in \mathbb{R}^{p \times p}$  defined by  $[\mathbf{Z}_k]_{ij} = [\mathbf{X}_k]_{ij}$  if  $(i,j) \in \mathcal{I}_k^c$ , and 0 otherwise.  $\mathbf{G}_k$  is some search direction only over  $\mathcal{I}_k^c$ , which is a symmetric matrix. It is known that a matrix  $\mathbf{A}$  is a nonsingular  $M$ -matrix if and only if  $\mathbf{A}$  is a  $Z$ -matrix and there exists a vector  $x > 0$  with  $A x > 0$  [49]. Following from the fact that  $\mathbf{X}_k$  is a nonsingular  $M$ -matrix, there exists  $x > 0$  such that
$$
\mathbf {Z} _ {k} \boldsymbol {x} \geq \mathbf {X} _ {k} \boldsymbol {x} > 0.
$$
Therefore,  $Z_{k}$  is also a nonsingular  $M$ -matrix, implying that  $Z_{k} \in \mathbb{S}_{++}$ . As a result, if  $\gamma_{k} < \lambda_{\min}(Z_{k}) / \rho(G_{k})$ , where  $\rho(G_{k})$  is the spectral radius of  $G_{k}$ , then  $X_{k}(\gamma_{k}) \in \mathbb{S}_{++}$ . The definition of  $X_{k}(\gamma_{k})$  further guarantees that  $X_{k}(\gamma_{k}) \in \mathcal{U}^{p}$ . We can verify that  $f(X) = +\infty$  for any  $X \in \operatorname{cl}(\mathcal{U}^{p}) \setminus \mathcal{U}^{p}$ , where  $X$  is positive semidefinite and singular. Thus, we consider an  $X^{o} \in \mathcal{U}^{p}$  with  $f(X^{o})$  is sufficiently large such that  $f(X_{k}(\gamma_{k})) \leq f(X^{o})$ . Therefore,  $X_{k}(\gamma_{k}) \in L_{f}$ .
Next we prove that the line search condition (15) holds for a small enough step size. Recall that  $\mathcal{I}_k^c$  is the complement of the set  $\mathcal{I}_k$  defined in (8). For any  $X_{k}\in L_{f}$ , the set  $\mathcal{I}_k^c$  can be represented as
$$
\mathcal {I} _ {k} ^ {c} = \mathcal {T} ^ {c} (\boldsymbol {X} _ {k}, \epsilon_ {k}) \cap \mathcal {E} ^ {c} = \bigcup_ {l = 1} ^ {5} \mathcal {B} _ {k} ^ {(l)},
$$
where  $\mathcal{B}_k^{(l)}, l = 1, \ldots, 5$ , are defined as
$$
\mathcal {B} _ {k} ^ {(1)} = \left\{(i, j) \in \mathcal {E} ^ {c} \mid - \epsilon_ {k} \leq \left[ \boldsymbol {X} _ {k} \right] _ {i j} \leq 0, \left[ \nabla f (\boldsymbol {X} _ {k}) \right] _ {i j} \geq 0, \left[ \boldsymbol {P} _ {k} \right] _ {i j} <   0 \right\},
$$
$$
\mathcal {B} _ {k} ^ {(2)} = \left\{(i, j) \in \mathcal {E} ^ {c} | - \epsilon_ {k} \leq \left[ \boldsymbol {X} _ {k} \right] _ {i j} \leq 0, \left[ \nabla f (\boldsymbol {X} _ {k}) \right] _ {i j} \geq 0, \left[ \boldsymbol {P} _ {k} \right] _ {i j} \geq 0 \right\},
$$
$$
\mathcal {B} _ {k} ^ {(3)} = \left\{(i, j) \in \mathcal {E} ^ {c} \mid [ \boldsymbol {X} _ {k} ] _ {i j} <   - \epsilon_ {k}, [ \boldsymbol {P} _ {k} ] _ {i j} <   0 \right\},
$$
$$
\mathcal {B} _ {k} ^ {(4)} = \left\{(i, j) \in \mathcal {E} ^ {c} \mid [ \boldsymbol {X} _ {k} ] _ {i j} <   - \epsilon_ {k}, [ \boldsymbol {P} _ {k} ] _ {i j} \geq 0 \right\},
$$
$$
\mathcal {B} _ {k} ^ {(5)} = \left\{(i, j) \in \mathcal {E} ^ {c} \mid \left[ \boldsymbol {X} _ {k} \right] _ {i j} > 0 \right\}.
$$
Each subset  $\mathcal{B}_k^{(l)}$  is disjoint with each other.  $\mathcal{B}_k^{(5)}$  only contains the indexes of the diagonal elements of  $X_{k}$ , while the other subsets include the indexes of the off-diagonal elements. Then one has, for any  $(i,j)\in \mathcal{B}_k^{(1)}$  and  $\gamma_k > 0$ ,
$$
0 \leq \left[ \boldsymbol {X} _ {k} \left(\gamma_ {k}\right) \right] _ {i j} - \left[ \boldsymbol {X} _ {k} \right] _ {i j} = \left[ \mathcal {P} _ {\Omega} \left(\boldsymbol {X} _ {k} - \gamma_ {k} \boldsymbol {P} _ {k}\right) \right] _ {i j} - \left[ \boldsymbol {X} _ {k} \right] _ {i j} \leq - \gamma_ {k} \left[ \boldsymbol {P} _ {k} \right] _ {i j}. \tag {30}
$$
For any  $(i,j)\in \mathcal{B}_k^{(2)}$  and  $\gamma_{k} > 0$ , one has  $\left[\boldsymbol {X}_k(\gamma_k)\right]_{ij} - \left[\boldsymbol {X}_k\right]_{ij} = -\gamma_k\left[\boldsymbol {P}_k\right]_{ij}$ . For any  $(i,j)\in \mathcal{B}_k^{(3)}$ , if the step size satisfies
$$
0 <   \gamma_ {k} \leq \min  _ {(i, j) \in \mathcal {B} _ {k} ^ {(3)}} \frac {\epsilon_ {k}}{\left| [ P _ {k} ] _ {i j} \right|}, \tag {31}
$$
then one has  $\left[\boldsymbol{X}_k(\gamma_k)\right]_{ij} - \left[\boldsymbol{X}_k\right]_{ij} = -\gamma_k\left[\boldsymbol{P}_k\right]_{ij}$ . Similarly, for any  $(i,j) \in \mathcal{B}_k^{(4)}$  and  $\gamma_k > 0$ ,  $\left[\boldsymbol{X}_k(\gamma_k)\right]_{ij} - \left[\boldsymbol{X}_k\right]_{ij} = -\gamma_k\left[\boldsymbol{P}_k\right]_{ij}$ . Finally, for any  $(i,j) \in \mathcal{B}_k^{(5)}$ ,  $\left[\boldsymbol{X}_k\right]_{ij}$  must be on the diagonal of  $\boldsymbol{X}_k$ . Therefore, we can directly remove the projection  $\mathcal{P}_{\Omega}$ , and obtain  $\left[\boldsymbol{X}_k(\gamma_k)\right]_{ij} - \left[\boldsymbol{X}_k\right]_{ij} = -\gamma_k\left[\boldsymbol{P}_k\right]_{ij}$ . Therefore, if (31) holds, one has
$$
\left\langle \left[ \nabla f (\boldsymbol {X} _ {k}) \right] _ {\mathcal {I} _ {k} ^ {c}}, \left[ \boldsymbol {X} _ {k} \left(\gamma_ {k}\right) \right] _ {\mathcal {I} _ {k} ^ {c}} - \left[ \boldsymbol {X} _ {k} \right] _ {\mathcal {I} _ {k} ^ {c}} \right\rangle \leq - \gamma_ {k} \left\langle \left[ \nabla f (\boldsymbol {X} _ {k}) \right] _ {\mathcal {I} _ {k} ^ {c}}, \left[ \boldsymbol {P} _ {k} \right] _ {\mathcal {I} _ {k} ^ {c}} \right\rangle , \tag {32}
$$
where the inequality follows from (30). Recall that  $\left[\mathbf{X}_k(\gamma_k)\right]_{\mathcal{I}_k} = \mathbf{0}$ , which can be equivalently expressed as the form of projected gradient descent:
$$
\left[ \boldsymbol {X} _ {k} \left(\gamma_ {k}\right) \right] _ {\mathcal {I} _ {k}} = \mathcal {P} _ {\Omega} \left(\left[ \boldsymbol {X} _ {k} \right] _ {\mathcal {I} _ {k}} - \gamma_ {k} \left[ \tilde {\boldsymbol {D}} _ {k} \odot \nabla f \left(\boldsymbol {X} _ {k}\right) \right] _ {\mathcal {I} _ {k}}\right), \tag {33}
$$
where we write  $\mathcal{P}_{\Omega}\big([A]_{\mathcal{I}_k}\big)$  for  $\left[\mathcal{P}_{\Omega}(\boldsymbol {A})\right]_{\mathcal{I}_k}$  with a slight abuse of notation, and  $\tilde{D}_k$  is defined as
$$
\left[ \tilde {\boldsymbol {D}} _ {k} \right] _ {i j} = \frac {\epsilon_ {k}}{\gamma_ {k} \left| \left[ \nabla f (\boldsymbol {X} _ {k}) \right] _ {i j} \right|}, \quad \forall (i, j) \in \mathcal {I} _ {k} \setminus \mathcal {E}.
$$
Then one has
$$
\begin{array}{l} \left\| \boldsymbol {X} _ {k} (\gamma_ {k}) - \boldsymbol {X} _ {k} \right\| _ {\mathrm {F}} ^ {2} \\ \leq \gamma_ {k} ^ {2} \left\langle \left[ \boldsymbol {P} _ {k} \right] _ {\mathcal {I} _ {k} ^ {c}}, \left[ \boldsymbol {P} _ {k} \right] _ {\mathcal {I} _ {k} ^ {c}} \right\rangle - \gamma_ {k} \left\langle \left[ \tilde {\boldsymbol {D}} _ {k} \odot \nabla f (\boldsymbol {X} _ {k}) \right] _ {\mathcal {I} _ {k}}, \left[ \boldsymbol {X} _ {k} (\gamma_ {k}) \right] _ {\mathcal {I} _ {k}} - \left[ \boldsymbol {X} _ {k} \right] _ {\mathcal {I} _ {k}} \right\rangle \tag {34} \\ \leq \gamma_ {k} ^ {2} \left\langle \left[ \boldsymbol {P} _ {k} \right] _ {\mathcal {I} _ {k} ^ {\mathrm {c}}}, \left[ \boldsymbol {P} _ {k} \right] _ {\mathcal {I} _ {k} ^ {\mathrm {c}}} \right\rangle + a _ {k} \left\langle \left[ \nabla f (\boldsymbol {X} _ {k}) \right] _ {\mathcal {I} _ {k}}, \left[ \boldsymbol {X} _ {k} \right] _ {\mathcal {I} _ {k}} \right\rangle , \\ \end{array}
$$
where  $a_{k} = \frac{\epsilon_{k}}{\left\|\left[\nabla f(\boldsymbol{X}_{k})\right]_{\mathcal{T}_{k}\setminus\mathcal{E}}\right\|_{\min}}$  and  $\mathcal{T}_k$  represents  $\mathcal{T}(\pmb {X}_k,\epsilon_k)$ . Note that  $\mathcal{T}_k = \mathcal{T}_k\cup \mathcal{E}$ , and  $[\pmb {X}_k]_{ij} = [\pmb {X}_k(\gamma_k)]_{ij} = 0$  for any  $(i,j)\in \mathcal{E}$
It is worth mentioning that we primarily focus on cases with non-empty  $\mathcal{T}_k$ . In the rare event that  $\mathcal{T}_k$  is empty, such as when  $\mathcal{T}_{\delta}$  is empty (in this situation we define  $\epsilon_k = \delta$ , leading to an empty  $\mathcal{T}_k$ ), we can simply ignore the terms related to  $\mathcal{I}_k$  in (34) and the following proof.
Furthermore, one has
$$
\begin{array}{l} \left\langle \left[ \boldsymbol {P} _ {k} \right] _ {\mathcal {I} _ {k} ^ {c}}, \left[ \boldsymbol {P} _ {k} \right] _ {\mathcal {I} _ {k} ^ {c}} \right\rangle \leq \left\| \left[ \boldsymbol {M} _ {k} ^ {- 1} \right] _ {\mathcal {I} _ {k} ^ {c} \mathcal {I} _ {k} ^ {c}} ^ {\frac {1}{2}} \right\| _ {2} ^ {2} \left\| \left[ \boldsymbol {M} _ {k} ^ {- 1} \right] _ {\mathcal {I} _ {k} ^ {c} \mathcal {I} _ {k} ^ {c}} ^ {\frac {1}{2}} \left[ \nabla f (\boldsymbol {X} _ {k}) \right] _ {\mathcal {I} _ {k} ^ {c}} \right\| ^ {2} \tag {35} \\ = \lambda_ {\max } \left(\left[ M _ {k} ^ {- 1} \right] _ {\mathcal {I} _ {k} ^ {c} \mathcal {I} _ {k} ^ {c}}\right) \left\langle \left[ \nabla f (\boldsymbol {X} _ {k}) \right] _ {\mathcal {I} _ {k} ^ {c}}, \left[ \boldsymbol {P} _ {k} \right] _ {\mathcal {I} _ {k} ^ {c}} \right\rangle . \\ \end{array}
$$
The largest eigenvalue of  $\left[M_k^{-1}\right]_{\mathcal{I}_k^c\mathcal{I}_k^c}$  can be bounded by
$$
\lambda_ {\max } \left(\left[ M _ {k} ^ {- 1} \right] _ {\mathcal {I} _ {k} ^ {c} \mathcal {I} _ {k} ^ {c}}\right) \leq \lambda_ {\max } \left(\boldsymbol {H} _ {k} ^ {- 1}\right) = \lambda_ {\max } \left(\boldsymbol {X} _ {k} \otimes \boldsymbol {X} _ {k}\right) \leq M ^ {2}, \tag {36}
$$
where the first inequality follows from the Eigenvalue Interlacing Theorem, and the second inequality follows from Lemma B.1. Combining (34), (35) and (36) yields
$$
\left\| \boldsymbol {X} _ {k} (\gamma_ {k}) - \boldsymbol {X} _ {k} \right\| _ {\mathrm {F}} ^ {2} \leq \gamma_ {k} ^ {2} M ^ {2} \big \langle [ \nabla f (\boldsymbol {X} _ {k}) ] _ {\mathcal {I} _ {k} ^ {c}}, [ \boldsymbol {P} _ {k} ] _ {\mathcal {I} _ {k} ^ {c}} \big \rangle + a _ {k} \big \langle [ \nabla f (\boldsymbol {X} _ {k}) ] _ {\mathcal {I} _ {k}}, [ \boldsymbol {X} _ {k} ] _ {\mathcal {I} _ {k}} \big \rangle .
$$
Since  $X_{k}(\gamma_{k})\in L_{f}$  , we obtain
$$
f \left(\boldsymbol {X} _ {k} \left(\gamma_ {k}\right)\right) - f \left(\boldsymbol {X} _ {k}\right) \leq \left\langle \nabla f \left(\boldsymbol {X} _ {k}\right), \boldsymbol {X} _ {k} \left(\gamma_ {k}\right) - \boldsymbol {X} _ {k} \right\rangle + \frac {1}{2 m ^ {2}} \left\| \boldsymbol {X} _ {k} \left(\gamma_ {k}\right) - \boldsymbol {X} _ {k} \right\| _ {\mathrm {F}} ^ {2}, \tag {37}
$$
where the inequality follows from
$$
\lambda_ {\max } \left(\nabla^ {2} f (\boldsymbol {X})\right) = \lambda_ {\max } \left(\boldsymbol {X} ^ {- 1} \otimes \boldsymbol {X} ^ {- 1}\right) = \lambda_ {\max } ^ {2} (\boldsymbol {X} ^ {- 1}) \leq \frac {1}{m ^ {2}}, \quad \forall \boldsymbol {X} \in L _ {f},
$$
where the inequality follows from Lemma B.1.
Let  $\bar{\gamma}_k = \min \left(\frac{2(1 - \alpha)m^2}{M^2},\min_{(i,j)\in \mathcal{B}_k^{(3)}}\frac{\epsilon_k}{\left|\left[\boldsymbol{P}_k\right]_{ij}\right|},\frac{\lambda_{\min}(\boldsymbol{Z}_k)}{\rho(\boldsymbol{G}_k)}\right)$ . Note that  $\bar{\gamma}_k$  is bounded away from zero, because each  $[\boldsymbol{P}_k]_{ij}$  is bounded following from Lemma B.1. To this end, for any  $\gamma_k\in (0,\bar{\gamma}_k)$ ,
$$
f \left(\boldsymbol {X} _ {k} (\gamma_ {k})\right) - f \left(\boldsymbol {X} _ {k}\right) \leq - \alpha \gamma_ {k} \left\langle \left[ \nabla f \left(\boldsymbol {X} _ {k}\right) \right] _ {\mathcal {I} _ {k} ^ {c}}, \left[ \boldsymbol {P} _ {k} \right] _ {\mathcal {I} _ {k} ^ {c}} \right\rangle - \alpha \left\langle \left[ \nabla f \left(\boldsymbol {X} _ {k}\right) \right] _ {\mathcal {I} _ {k}}, \left[ \boldsymbol {X} _ {k} \right] _ {\mathcal {I} _ {k}} \right\rangle , \tag {38}
$$
where the inequality follows from (32), the definition of  $\epsilon_{k}$  in (9), the inequality  $\left\| \left[\nabla f(\pmb{X}_k)\right]_{\mathcal{T}_{\delta} \setminus \mathcal{E}}\right\|_{\min} \leq \left\| \left[\nabla f(\pmb{X}_k)\right]_{\mathcal{T}_{k} \setminus \mathcal{E}}\right\|_{\min}$ , because  $\mathcal{T}_k \subseteq \mathcal{T}_{\delta}$ , and the inequality  $\langle [\nabla f(\pmb{X}_k)]_{\mathcal{T}_k^c}, [P_k]_{\mathcal{T}_k^c} \rangle \geq 0$ , because  $[M_k^{-1}]_{\mathcal{T}_k^c \mathcal{T}_k^c}$  is positive definite and
$$
\left\langle \left[ \nabla f \left(\boldsymbol {X} _ {k}\right) \right] _ {\mathcal {I} _ {k} ^ {c}}, \left[ \boldsymbol {P} _ {k} \right] _ {\mathcal {I} _ {k} ^ {c}} \right\rangle = \left\langle \left[ \nabla f \left(\boldsymbol {X} _ {k}\right) \right] _ {\mathcal {I} _ {k} ^ {c}}, \left[ \boldsymbol {M} _ {k} ^ {- 1} \right] _ {\mathcal {I} _ {k} ^ {c} \mathcal {I} _ {k} ^ {c}} \left[ \nabla f \left(\boldsymbol {X} _ {k}\right) \right] _ {\mathcal {I} _ {k} ^ {c}} \right\rangle , \tag {39}
$$
B.4 Proof of Proposition 3.5
Proof. The line search condition (15) leads to