SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$$
f \left(\boldsymbol {X} _ {k} \left(\gamma_ {k}\right)\right) - f \left(\boldsymbol {X} _ {k}\right) \leq - \alpha \gamma_ {k} \left\langle \left[ \nabla f \left(\boldsymbol {X} _ {k}\right) \right] _ {\mathcal {I} _ {k} ^ {c}}, \left[ \boldsymbol {M} _ {k} ^ {- 1} \right] _ {\mathcal {I} _ {k} ^ {c} \mathcal {T} _ {k} ^ {c}} \left[ \nabla f \left(\boldsymbol {X} _ {k}\right) \right] _ {\mathcal {I} _ {k} ^ {c}} \right\rangle , \tag {40}
$$
$$
\left\langle \left[ \nabla f (\boldsymbol {X} _ {k}) \right] _ {\mathcal {I} _ {k}}, \left[ \boldsymbol {X} _ {k} \right] _ {\mathcal {I} _ {k}} \right\rangle \geq 0.
$$
$$
\lambda_ {\min } \left(\left[ \boldsymbol {M} _ {k} ^ {- 1} \right] _ {\mathcal {I} _ {k} ^ {c} \mathcal {T} _ {k} ^ {c}}\right) \geq \lambda_ {\min } \left(\boldsymbol {H} _ {k} ^ {- 1}\right) = \lambda_ {\min } \left(\boldsymbol {X} _ {k} \otimes \boldsymbol {X} _ {k}\right) \geq m ^ {2}, \tag {41}
$$
where the first and second inequalities follow from Eigenvalue Interlacing Theorem and Lemma B.1, respectively. We complete the proof by combining (40) and (41).
Proof. We first prove that Problem (1) has at least one minimizer. It is known by the Weierstrass' extreme value theorem [50] that the set of minima is nonempty for any lower semicontinuous function with a nonempty compact lower level set. Therefore, the existence of the minimizers for Problem (1) can be guaranteed by Lemma B.1.
On the other hand, we show that Problem (1) has at most one minimizer by its strict convexity. We have  $\nabla^2 f(\pmb{X}) = \pmb{X}^{-1} \otimes \pmb{X}^{-1}$ . Thus  $\nabla^2 f(\pmb{X}) \succ 0$  for any  $\pmb{X}$  in the feasible region of Problem
(1) defined in (16). Therefore, Problem (1) is strictly convex, and thus has at most one minimizer. Together with the existence of the minimizers, we conclude that Problem (1) has an unique minimizer.
The Lagrangian of Problem (1) is
$$
\mathcal {L} (\boldsymbol {X}, \boldsymbol {Y}) = - \log \det  (\boldsymbol {X}) + \operatorname {t r} (\boldsymbol {X} \boldsymbol {S}) + \langle \boldsymbol {Y} - \boldsymbol {\lambda}, \boldsymbol {X} \rangle , \tag {42}
$$
where  $\pmb{Y}$  is a KKT multiplier with  $Y_{ii} = 0$  for  $i\in [p]$ , and  $\lambda$  is defined in (25).
For a convex optimization problem with Slater's condition holding, a pair is primal and dual optimal if and only if the KKT conditions hold. Thus,  $(X^{\star},Y^{\star})$  is primal and dual optimal if and only if it satisfies the KKT conditions of (1) as below,
$$
- \left(\boldsymbol {X} ^ {\star}\right) ^ {- 1} + \boldsymbol {S} - \boldsymbol {\lambda} + \boldsymbol {Y} ^ {\star} = \mathbf {0}; \tag {43}
$$
$$
X _ {i j} ^ {\star} = 0, \forall (i, j) \in \mathcal {E}; \tag {44}
$$
$$
X _ {i j} ^ {\star} Y _ {i j} ^ {\star} = 0, X _ {i j} ^ {\star} \leq 0, Y _ {i j} ^ {\star} \geq 0, \forall i \neq j \text {a n d} (i, j) \notin \mathcal {E}; \tag {45}
$$
$$
Y _ {i i} ^ {\star} = 0, \forall i \in [ p ]. \tag {46}
$$
We first prove that the minimizer  $X^{\star}$  must satisfy all conditions in (18). Note that the KKT conditions (43)-(46) must hold for the minimizer  $X^{\star}$ . Let  $\mathcal{V} = \{(i,j)\in [p]^2\mid X_{ij}^{\star} = 0\}$ . First,  $X_{ij}^{\star} = 0$  for any  $(i,j)\in \mathcal{E}$  following from (44). Second, for any  $(i,j)$  with  $i\neq j$  and  $(i,j)\in \mathcal{V}^{c}$ , we have  $X_{ij}^{\star}\neq 0$  and  $(i,j)\notin \mathcal{E}$ . Following from (45), we further obtain  $Y_{ij}^{\star} = 0$ . Together with (46), we conclude that  $Y_{ij}^{\star} = 0$  for any  $(i,j)\in \mathcal{V}^{c}$ . Following from (43),  $\left[\nabla f(X^{\star})\right]_{\mathcal{V}^c} = \mathbf{0}$ . Since  $X^{\star}$  is positive definite,  $(i,i)\in \mathcal{V}^{c}$  for any  $i\in [p]$ . Then, for any  $(i,j)\in \mathcal{V}\backslash \mathcal{E}$ , we have  $i\neq j$ , thus obtain  $Y_{ij}^{\star}\geq 0$  according to (45). Following from (43), we get  $\left[\nabla f(X^{\star})\right]_{\mathcal{V}\backslash \mathcal{E}}\leq \mathbf{0}$ .
Now we prove that any point  $X^{\star} \in \mathcal{M}^{p}$  satisfying the conditions in (18) must be the minimizer, i.e., the KKT conditions (43)-(46) hold for  $X^{\star}$ . We construct  $Y^{\star}$  by  $Y^{\star} = -\nabla f(X^{\star})$ . First, it is straightforward to check that the conditions (43) and (44) hold. Second, we have
$$
\left[ \boldsymbol {Y} ^ {\star} \right] _ {\mathcal {V} ^ {c}} = - \left[ \nabla f \left(\boldsymbol {X} ^ {\star}\right) \right] _ {\mathcal {V} ^ {c}} = \mathbf {0}. \tag {47}
$$
We know that  $(i,i)\in \mathcal{V}^c$  for any  $i\in [p]$ , since  $X^{\star}\in \mathcal{M}^{p}$ . Together with (47), we obtain that the condition (46) holds.
Finally, following from the fact that  $(i,i)\in \mathcal{V}^c$  for any  $i\in [p]$  and  $\mathcal{E}\subseteq \mathcal{V}$ , we obtain
$$
\left\{(i, j) \in [ p ] ^ {2} \mid i \neq j, (i, j) \notin \mathcal {E} \right\} = \mathcal {I} _ {1} \cup \mathcal {I} _ {2},
$$
where  $\mathcal{I}_1 = \{(i,j)\in [p]^2\mid (i,j)\in \mathcal{V}^c,i\neq j\}$ , and  $\mathcal{I}_2 = \{(i,j)\in [p]^2\mid (i,j)\in \mathcal{V},(i,j)\notin \mathcal{E}\}$ . For any  $(i,j)\in \mathcal{I}_1$ , we have  $Y_{ij}^{\star} = 0$  according to (47), and  $X_{ij}^{\star} < 0$  since  $X^{\star}\in \mathcal{M}^{p}$ . Thus the condition (45) holds for any  $(i,j)\in \mathcal{I}_1$ . For any  $(i,j)\in \mathcal{I}_2$ , we have  $X_{ij}^{\star} = 0$ , and  $Y_{ij}^{\star} = -[\nabla f(X^{\star})]_{ij}\geq 0$  according to the second condition in (18). Thus the condition (45) also holds for any  $(i,j)\in \mathcal{I}_2$ . Totally, the condition (45) holds. To sum up, all KKT conditions (43)-(46) hold for any  $X^{\star}\in \mathcal{M}^{p}$  satisfying the conditions in (18), and thus we conclude that  $X^{\star}$  is the minimizer of Problem (1).
Proof. Following from Proposition 3.4, for any iterate  $\mathbf{X}_k \in L_f$ , a small enough step size ensures that the line search condition (15) holds, which leads to a sufficient decrease of the objective function as shown in Theorem 3.5, and the next iterate  $\mathbf{X}_{k + 1} \in L_f$ . When the sequence starts with  $\mathbf{X}_0 \in L_f$ , each point of the sequence  $\{\mathbf{X}_k\}_{k \geq 0}$  admits  $\mathbf{X}_k \in L_f$ . Note that it is easy to construct an initial point  $\mathbf{X}_0 \in L_f$ , because we could consider an  $\mathbf{X}^o \in \mathcal{U}^p$  in (17), which is close to  $\operatorname{cl}(\mathcal{U}^p) \setminus \mathcal{U}^p$  such that  $f(\mathbf{X}^o)$  sufficiently large, following from the fact that  $f(\mathbf{X}) = +\infty$  for any  $\mathbf{X} \in \operatorname{cl}(\mathcal{U}^p) \setminus \mathcal{U}^p$ . Lemma B.1 shows that the lower level set  $L_f$  is compact, thus the sequence  $\{\mathbf{X}_k\}$  has at least one limit point. For every limit  $\mathbf{X}^\star$  of the sequence  $\{\mathbf{X}_k\}$ , we have  $\mathbf{X}^\star \in \mathcal{M}^p$ . Define  $\mathcal{T}^\star = \mathcal{T}(\mathbf{X}^\star, \epsilon^\star) \cup \mathcal{E}$ , where  $\mathcal{T}(\mathbf{X}^\star, \epsilon^\star)$  is equal to
$$
\mathcal {T} (\pmb {X} ^ {\star}, \epsilon^ {\star}) = \left\{(i, j) \in [ p ] ^ {2} \mid - \epsilon^ {\star} \leq \left[ \pmb {X} ^ {\star} \right] _ {i j} \leq 0, \left[ \nabla f (\pmb {X} ^ {\star}) \right] _ {i j} <   0 \right\},
$$
and  $\epsilon^{\star}$  is defined as  $\epsilon^{\star}:= \min \left(2(1 - \alpha)m^{2}\left\| \left[\nabla f(\pmb{X}^{\star})\right]_{\mathcal{T}_{\delta}^{\star}\setminus \mathcal{E}}\right\|_{\min},\delta\right)$ , where  $\mathcal{T}_{\delta}^{\star}$  denotes  $\mathcal{T}(\pmb{X}^{\star},\delta)$ . Since  $f$  is continuous and  $\{f(\pmb{X}_k)\}$  keeps decreasing and is bounded, it follows that  $\{f(\pmb{X}_k)\}$  converges and
$$
\lim  _ {k \rightarrow + \infty} f (\boldsymbol {X} _ {k}) - f (\boldsymbol {X} _ {k + 1}) = 0.
$$
Following from Proposition 3.4, the line search condition (15) ensures that each  $X_{k}\in \mathbb{S}_{++}^{p}$  and, for a strictly positive  $\gamma_{k}$ , it holds
$$
\begin{array}{l} f (\boldsymbol {X} _ {k}) - f (\boldsymbol {X} _ {k + 1}) \geq \alpha \gamma_ {k} \left\langle \left[ \nabla f (\boldsymbol {X} _ {k}) \right] _ {\mathcal {I} _ {k} ^ {c}}, \left[ \boldsymbol {P} _ {k} \right] _ {\mathcal {I} _ {k} ^ {c}} \right\rangle + \alpha \left\langle \left[ \nabla f (\boldsymbol {X} _ {k}) \right] _ {\mathcal {I} _ {k}}, \left[ \boldsymbol {X} _ {k} \right] _ {\mathcal {I} _ {k}} \right\rangle \tag {48} \\ = \alpha \gamma_ {k} \bigl \langle \left[ \nabla f (\boldsymbol {X} _ {k}) \right] _ {\mathcal {I} _ {k} ^ {c}}, \left[ \boldsymbol {M} _ {k} ^ {- 1} \right] _ {\mathcal {I} _ {k} ^ {c} \mathcal {T} _ {k} ^ {c}} \left[ \nabla f (\boldsymbol {X} _ {k}) \right] _ {\mathcal {I} _ {k} ^ {c}} \bigr \rangle + \alpha \bigl \langle [ \nabla f (\boldsymbol {X} _ {k}) ] _ {\mathcal {I} _ {k}}, [ \boldsymbol {X} _ {k} ] _ {\mathcal {I} _ {k}} \bigr \rangle . \\ \end{array}
$$
Recall that  $\mathcal{I}_k = \mathcal{T}_k\cup \mathcal{E}$ . We note that the following facts hold:  $\alpha \in (0,1)$  is a constant,  $\left[M_k^{-1}\right]_{\mathcal{I}_k^c\mathcal{I}_k^c}$  is positive definite,  $[X_k]_{ij} = 0$  for any  $(i,j)\in \mathcal{E}$ ,  $[X_k]_{ij}\leq 0$  for any  $(i,j)\in \mathcal{I}_k\backslash \mathcal{E}$ , and  $[\nabla f(\pmb {X}_k)]_{ij} < 0$  for any  $(i,j)\in \mathcal{I}_k\backslash \mathcal{E}$  due to the definition of  $\mathcal{T}(\pmb {X},\epsilon)$ . Thus, the two terms on the right-hand side of (48) are both nonnegative. Moreover, the right-hand side of (48) approaches zero if and only if both  $\left[\nabla f(\pmb {X}_k)\right]_{\mathcal{I}_k^c}$  and  $[X_k]_{\mathcal{I}_k}$  simultaneously go to zero. Therefore, we deduce that every limit point  $\pmb{X}^{\star}$  must satisfy:
$$
\left[ \nabla f \left(\boldsymbol {X} ^ {\star}\right) \right] _ {\{\mathcal {I} ^ {\star} \} ^ {c}} = \mathbf {0}, \quad \text {a n d} \quad \left[ \boldsymbol {X} ^ {\star} \right] _ {\mathcal {I} ^ {\star}} = \mathbf {0}. \tag {49}
$$
We show that every limit point  $\pmb{X}^{\star}$  is the minimizer of Problem (1) according to Theorem 4.1. Let  $\mathcal{V} = \{(i,j) \in [p]^2 \mid [\pmb{X}^{\star}]_{ij} = 0\}$ . First, for any  $(i,j) \in \mathcal{E}$ , we have  $[\pmb{X}^{\star}]_{ij} = 0$ , because of the projection  $\mathcal{P}_{\Omega}$  in each iteration. Note that  $[\pmb{X}^{\star}]_{ij} = 0$  for any  $(i,j) \in \mathcal{I}^{\star}$  according to (49). For any  $(i,j) \in \mathcal{V}^{c}$ , i.e.,  $[\pmb{X}^{\star}]_{ij} \neq 0$ , we must have  $(i,j) \in \{\mathcal{I}^{\star}\}^{c}$ . Together with (49),  $[\nabla f(\pmb{X}^{\star})]_{\mathcal{V}^{c}} = \mathbf{0}$  holds. For any  $(i,j) \in \mathcal{V} \backslash \mathcal{E}$ , we must have
$$
(i, j) \in \mathcal {T} \left(\boldsymbol {X} ^ {\star}, \epsilon^ {\star}\right) \cup \left\{\mathcal {I} ^ {\star} \right\} ^ {c}.
$$
Recall that  $\left[\nabla f\left(\boldsymbol{X}^{\star}\right)\right]_{ij} < 0$  for any  $(i,j) \in \mathcal{T}(\boldsymbol{X}^{\star},\epsilon^{\star})$ , and  $\left[\nabla f\left(\boldsymbol{X}^{\star}\right)\right]_{ij} = 0$  for any  $(i,j) \in \{\mathcal{I}^{\star}\}^{c}$ . Overall, we obtain
$$
\left[ \nabla f \left(\boldsymbol {X} ^ {\star}\right) \right] _ {\mathcal {V} \backslash \mathcal {E}} \leq \mathbf {0}.
$$
To sum up, all the conditions in Theorem 4.1 hold for every limit point  $X^{\star}$ , and thus every limit point is the minimizer of Problem (1).
Since the minimizer of Problem (1) is unique, we obtain that the limit point of the sequence  $\{\pmb{X}_k\}$  is also unique, and thus  $\{\pmb{X}_k\}$  is convergent. Therefore, we conclude that the sequence  $\{\pmb{X}_k\}$  converges to the unique minimizer of Problem (1). The monotone decreasing of the sequence  $\{f(\pmb{X}_k)\}$  can be established by Proposition 3.5.
B.7 Proof of Theorem 4.4
Proof. We prove that the support of  $\pmb{X}^{\star}$  is consistent with the set  $\mathcal{I}_k^c$  for a sufficiently large  $k$ . Without loss of generality, we specify the constant  $\delta$  in (9) as
$$
\delta = \omega \min  _ {(i, j) \in \operatorname {s u p p} \left(\boldsymbol {X} ^ {\star}\right)} | [ \boldsymbol {X} ^ {\star} ] _ {i j} |, \tag {50}
$$
where  $\omega \in (0,1)$  is a constant. Note that the sequence  $\{\pmb{X}_k\}$  converges to  $\pmb{X}^{\star}$ . Under Theorem 4.3 that  $\left[\nabla f(\pmb{X}^{\star})\right]_{ij}$  is strictly negative for any  $(i,j) \in \mathrm{supp}^{c}(\pmb{X}^{\star}) \setminus \mathcal{E}$ , there must exist some  $a > 0$  and  $K_{1} \in \mathbb{N}_{+}$  such that
$$
\left[ \nabla f \left(\boldsymbol {X} _ {k}\right) \right] _ {i j} <   - \frac {a}{2 (1 - \alpha) m ^ {2}}, \quad \forall (i, j) \in \operatorname {s u p p} ^ {c} \left(\boldsymbol {X} ^ {\star}\right) \backslash \mathcal {E} \tag {51}
$$
holds for any  $k \geq K_1$ , where  $\alpha \in (0,1)$  is a constant, and  $m$  is a positive constant defined in Lemma B.1. We consider a neighbourhood of  $X^{\star}$  defined by
$$
\mathcal {N} \left(\boldsymbol {X} ^ {\star}; r\right) := \left\{\boldsymbol {X} \in \mathbb {R} ^ {p \times p} \mid \| \boldsymbol {X} - \boldsymbol {X} ^ {\star} \| _ {\mathrm {F}} \leq r \right\}, \tag {52}
$$
where  $r$  is a positive constant defined as
$$
r := \min  \left(c \min  _ {(i, j) \in \operatorname {s u p p} \left(\boldsymbol {X} ^ {\star}\right)} | [ \boldsymbol {X} ^ {\star} ] _ {i j} |, a, \delta\right), \tag {53}
$$
where  $c < 1 - \omega$  is a positive constant. There must exist  $K_{2} \in \mathbb{N}_{+}$  such that  $\pmb{X}_k \in \mathcal{N}\big(\pmb{X}^\star; r\big)$  holds for any  $k \geq K_2$ . Take  $K_{o} = \max(K_{1}, K_{2})$ . For any  $k \geq K_{o}$  and  $(i, j) \in \operatorname{supp}(\pmb{X}^{\star})$ , one has
$$
\left| \left[ \boldsymbol {X} ^ {\star} \right] _ {i j} \right| - \left| \left[ \boldsymbol {X} _ {k} \right] _ {i j} \right| \leq \left| \left[ \boldsymbol {X} ^ {\star} \right] _ {i j} - \left[ \boldsymbol {X} _ {k} \right] _ {i j} \right| \leq \left\| \boldsymbol {X} _ {k} - \boldsymbol {X} ^ {\star} \right\| _ {\mathrm {F}} \leq r,
$$
Thus one can obtain, for any  $k \geq K_{o}$ ,
$$
\left| \left[ \boldsymbol {X} _ {k} \right] _ {i j} \right| \geq \min  _ {(i, j) \in \operatorname {s u p p} \left(\boldsymbol {X} ^ {\star}\right)} \left| \left[ \boldsymbol {X} ^ {\star} \right] _ {i j} \right| - r > \delta > 0, \quad \forall (i, j) \in \operatorname {s u p p} \left(\boldsymbol {X} ^ {\star}\right), \tag {54}
$$
where the last inequality follows from (50) and (53). Recall that for any  $(i,j)\in \mathcal{T}(\pmb{X}_k,\delta)$ ,  $|\left[\pmb{X}_k\right]_{ij}|\leq \delta$ . Then one has
$$
\mathcal {T} \left(\boldsymbol {X} _ {k}, \delta\right) \subseteq \operatorname {s u p p} ^ {c} \left(\boldsymbol {X} ^ {\star}\right). \tag {55}
$$
Then the  $\epsilon_{k}$  in (9) can be bounded by
$$
\epsilon_ {k} \geq \min  \left(2 (1 - \alpha) m ^ {2} \| [ \nabla f (\boldsymbol {X} _ {k}) ] _ {\operatorname {s u p p} ^ {c} (\boldsymbol {X} ^ {*}) \backslash \mathcal {E}} \| _ {\min }, \delta\right) \geq \min  (a, \delta), \tag {56}
$$
where the first and second inequalities follow from (55) and (51), respectively. For any  $k \geq K_{o}$  and  $(i,j) \in \mathcal{T}(\pmb{X}_k, \epsilon_k) \cup \mathcal{E}$ , one has
$$
\left| \left[ \boldsymbol {X} _ {k} \right] _ {i j} \right| \leq \epsilon_ {k} \leq \delta ,
$$
where the second inequality follows from the definition of  $\epsilon_{k}$ . Thus we obtain
$$
\mathcal {T} \left(\boldsymbol {X} _ {k}, \epsilon_ {k}\right) \cup \mathcal {E} \subseteq \operatorname {s u p p} ^ {c} \left(\boldsymbol {X} ^ {\star}\right). \tag {57}
$$
On the other hand, for any  $k \geq K_{o}$  and  $(i,j) \in \operatorname{supp}^{c}(\mathbf{X}^{\star})$ , one has
$$
\left| \left[ \boldsymbol {X} _ {k} \right] _ {i j} \right| = \left| \left[ \boldsymbol {X} _ {k} \right] _ {i j} - \left[ \boldsymbol {X} ^ {\star} \right] _ {i j} \right| \leq \left\| \boldsymbol {X} _ {k} - \boldsymbol {X} ^ {\star} \right\| _ {\mathrm {F}} \leq r \leq \epsilon_ {k}, \tag {58}
$$
where the last inequality follows from (53) and (56). Therefore, one has
$$
\operatorname {s u p p} ^ {c} \left(\boldsymbol {X} ^ {\star}\right) \subseteq \mathcal {T} \left(\boldsymbol {X} _ {k}, \epsilon_ {k}\right) \cup \mathcal {E} \cup \mathcal {B} _ {k} ^ {(1)} \cup \mathcal {B} _ {k} ^ {(2)}.
$$
Note that  $\mathcal{B}_k^{(5)}\cap \mathrm{supp}^c (\pmb {X}^\star) = \varnothing$  , because any  $(i,j)\in \mathcal{B}_k^{(5)}$  corresponds to the element on the diagonal which must be nonzero. Moreover, following from (51), one has
$$
\left(\operatorname {s u p p} ^ {c} \left(\boldsymbol {X} ^ {\star}\right) \setminus \mathcal {E}\right) \cap \mathcal {B} _ {k} ^ {(1)} = \varnothing , \quad \text {a n d} \quad \left(\operatorname {s u p p} ^ {c} \left(\boldsymbol {X} ^ {\star}\right) \setminus \mathcal {E}\right) \cap \mathcal {B} _ {k} ^ {(2)} = \varnothing .
$$
Therefore, we can obtain  $\mathrm{supp}^c (\pmb {X}^\star)\subseteq \mathcal{T}(\pmb {X}_k,\epsilon_k)\cup \mathcal{E}$ . Together with (57), we obtain
$$
\operatorname {s u p p} ^ {c} \left(\boldsymbol {X} ^ {\star}\right) = \mathcal {T} \left(\boldsymbol {X} _ {k}, \epsilon_ {k}\right) \cup \mathcal {E} = \mathcal {I} _ {k}. \tag {59}
$$
Equivalently, we have
$$
\operatorname {s u p p} \left(\boldsymbol {X} ^ {\star}\right) = \mathcal {I} _ {k} ^ {c}, \quad \forall k \geq K _ {o}. \tag {60}
$$
We can see that the sets  $\mathcal{I}_k$  and  $\mathcal{I}_k^c$  are fixed for any  $k\geq K_o$ . Therefore, following from the iterate of  $\pmb{X}_{k + 1}$
$$
\left[ \mathbf {X} _ {k + 1} \right] _ {\mathcal {I} _ {k + 1}} = \left[ \mathbf {X} _ {k + 1} \right] _ {\mathcal {I} _ {k}} = \mathbf {0}, \quad \forall k \geq K _ {o}.
$$
Moreover, together with (54) and (60), we obtain that for any  $k \geq K_{o}$ ,  $\left[\mathbf{X}_{k+1}\right]_{\mathcal{T}_{k+1}^{c}} \neq \mathbf{0}$ . Take  $k_{o} = K_{o} + 1$ . We obtain
$$
\operatorname {s u p p} \left(\boldsymbol {X} _ {k}\right) = \mathcal {I} _ {k} ^ {c}, \quad \forall k \geq k _ {o},
$$
B.8 Proof of Theorem 4.5
Proof. Theorem 4.5 is a direct extension of the result in [51]. Let  $g$  be twice continuously differentiable and consider the following iterate
$$
\boldsymbol {x} _ {x + 1} = \boldsymbol {x} _ {k} - \gamma_ {k} \boldsymbol {D} _ {k} \nabla g (\boldsymbol {x} _ {k}),
$$
where  $D_{k}$  is positive definite and symmetric. Then the sequence  $\{\pmb{x}_k\}$  admits the following convergence result [51],
$$
\lim  _ {k \rightarrow \infty} \frac {\left\| \boldsymbol {x} _ {k + 1} - \boldsymbol {x} ^ {\star} \right\| _ {\boldsymbol {D} _ {k} ^ {- 1}} ^ {2}}{\left\| \boldsymbol {x} _ {k} - \boldsymbol {x} ^ {\star} \right\| _ {\boldsymbol {D} _ {k} ^ {- 1}} ^ {2}} = \lim  _ {k \rightarrow \infty} \sup  \max  \left(| 1 - \gamma_ {k} m _ {k} ^ {\prime} | ^ {2}, | 1 - \gamma_ {k} M _ {k} ^ {\prime} | ^ {2}\right), \tag {61}
$$
where  $\pmb{x}^{\star}$  is the limit of the sequence  $\{\pmb{x}_k\}$ , which satisfies that  $\nabla g(\pmb{x}^{\star}) = \mathbf{0}$  and  $\nabla^2 g(\pmb{x}^{\star})$  is positive definite, and  $m_k^\prime$  and  $M_{k}^{\prime}$  are the smallest and largest eigenvalues of  $(D_k)^{1 / 2}\nabla^2 g(\pmb {x}_k)(D_k)^{1 / 2}$ , respectively. This conclusion is extended from the convergence result for the quadratic objective function. Conceptually, this makes sense because a twice continuously differentiable objective function is very close to a positive definite quadratic function in the neighborhood of a non-singular local minimum.
Following from Theorem 4.4, for any  $k \geq k_{o}$ , iterate (13) can be written as
$$
\left[ \boldsymbol {X} _ {k + 1} \right] _ {\mathcal {I} _ {k} ^ {c}} = \left[ \boldsymbol {X} _ {k} \right] _ {\mathcal {I} _ {k} ^ {c}} - \gamma_ {k} \boldsymbol {R} _ {k} ^ {- 1} \left[ \nabla f (\boldsymbol {X} _ {k}) \right] _ {\mathcal {I} _ {k} ^ {c}}, \tag {62}
$$
which reduces to an iterate of an unconstrained optimization algorithm on some subspace. Following from the results in (61), we obtain
$$
\lim  _ {k \rightarrow \infty} \frac {\left\|\left[ \boldsymbol {X} _ {k + 1} \right] _ {\mathcal {I} _ {k} ^ {c}} - \left[ \boldsymbol {X} ^ {\star} \right] _ {\mathcal {I} _ {k} ^ {c}} \right\| _ {\boldsymbol {R} _ {k}} ^ {2}}{\left\|\left[ \boldsymbol {X} _ {k} \right] _ {\mathcal {I} _ {k} ^ {c}} - \left[ \boldsymbol {X} ^ {\star} \right] _ {\mathcal {I} _ {k} ^ {c}} \right\| _ {\boldsymbol {R} _ {k}} ^ {2}} = \lim  _ {k \rightarrow \infty} \sup  \max  \left(| 1 - \gamma_ {k} m _ {k} | ^ {2}, | 1 - \gamma_ {k} M _ {k} | ^ {2}\right), \tag {63}
$$
where  $m_{k}$  and  $M_{k}$  are the smallest and largest eigenvalues of  $\pmb{R}_{k}^{-\frac{1}{2}}\left[\pmb{H}_{k}\right]_{\mathcal{I}_{k}^{c}\mathcal{T}_{k}^{c}}\pmb{R}_{k}^{-\frac{1}{2}}$ , respectively. Following from (11), Theorem 4.2, and  $\pmb{R}_{k}^{-1} = \left[\pmb{H}_{k}^{-1}\right]_{\mathcal{I}_{k}^{c}\mathcal{T}_{k}^{c}}$ , we obtain
$$
\left\| \left[ \boldsymbol {X} _ {k} \right] _ {\mathcal {I} _ {k} ^ {c}} - \left[ \boldsymbol {X} ^ {\star} \right] _ {\mathcal {I} _ {k} ^ {c}} \right\| _ {\boldsymbol {R} _ {k}} ^ {2} = \left\| \boldsymbol {X} _ {k} - \boldsymbol {X} ^ {\star} \right\| _ {\boldsymbol {M} _ {k}} ^ {2}, \quad \forall k \geq k _ {o}. \tag {64}
$$
When  $k \geq k_{o}$ , the line search condition reduces to
$$
f \left(\boldsymbol {X} _ {k + 1}\right) \leq f \left(\boldsymbol {X} _ {k}\right) - \alpha \gamma_ {k} \left\langle \left[ \nabla f (\boldsymbol {X} _ {k}) \right] _ {\mathcal {I} _ {k} ^ {c}}, \left[ \boldsymbol {P} _ {k} \right] _ {\mathcal {I} _ {k} ^ {c}} \right\rangle .
$$
Similar to the unconstrained case in [52], the step size must satisfy the line search condition if  $\gamma_k \geq \min \left(1, 2(1 - \alpha)\beta / M_k\right)$  as  $k \to \infty$ . Then together with (63) and (64), we complete the proof.