SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Graph-based semi-supervised learning (GSSL) serves as a powerful tool to model the underlying manifold structures of samples in high-dimensional spaces. It involves two phases: constructing an affinity graph from available data and inferring labels for unlabeled nodes on this graph. While numerous algorithms have been developed for label inference, the crucial graph construction phase has received comparatively less attention, despite its significant influence on the subsequent phase. In this paper, we present an optimal asymmetric graph structure for the label inference phase with theoretical motivations. Unlike existing graph construction methods, we differentiate the distinct roles that labeled nodes and unlabeled nodes could play. Accordingly, we design an efficient block-wise graph learning algorithm with a global convergence guarantee. Other benefits induced by our method, such as enhanced robustness to noisy node features, are explored as well. Finally, we perform extensive experiments on synthetic and real-world datasets to demonstrate its superiority to the state-of-the-art graph construction methods in GSSL.
Graph-based semi-supervised learning (GSSL) is a burgeoning research field [52, 9, 14, 63, 65, 62]. As a subclass of semi-supervised learning (SSL), GSSL exhibits promise since it encapsulates the smoothness or manifold assumption, where samples with similar features are likely to share the same label. GSSL methods begin by constructing an affinity graph, wherein nodes represent samples, and weighted edges denote similarity between pairs of nodes. This process aligns with the manifold assumption, implying that nodes connected by edges with large weights tend to have the same label. Upon obtaining the affinity graph, various label inference algorithms such as label propagation [85, 82, 25, 65] can be executed to predict labels for the unlabeled nodes.
Preliminary empirical studies [16] suggest that the quality of the affinity graph significantly influences label prediction performance. However, the graph construction phase in GSSL has received less scrutiny compared to the subsequent label inference phase. Constructing a high-quality graph presents a challenge as its quality can only be assessed indirectly through postmortem verification via label inference performance. Classic solutions such as the Radial Basis Function (RBF) Kernel [85], kNN graph [17], and  $b$ -matching [26], while simple, may exhibit low robustness against noise due to their simplicity. More recent and complex methods, typically framed as optimization problems to find
the optimal graph under the smoothness assumption, suffer from inefficient optimization techniques without fast convergence guarantees [18, 29, 63] or lack a solid theoretical foundation [58, 14, 23].
Consequently, a series of research questions arise: 1) What is the optimal graph structure for label inference algorithms in GSSL? 2) How can we optimize this optimal graph efficiently? 3) What kinds of benefits can this optimal graph bring?
Most existing GSSL graph construction methods treat all nodes equally, disregarding label information, which results in a symmetric, undirected graph. However, we contend that an asymmetric, directed graph structure might be more beneficial for subsequent label inference due to the distinct roles labeled and unlabeled nodes can play. Intuitively, edges from labeled to unlabeled nodes would naturally facilitate supervision information propagation, while edges from unlabeled to labeled nodes may introduce inconsistency, potentially undermining the prediction accuracy for the labeled nodes.
Motivated by this key observation, we revisit a unified optimization framework that encompasses most label inference algorithms, assuming the affinity graph is already constructed. We then fix the downstream label inference result and position the graph weight matrix (or adjacency matrix) as the optimization variable. This shift allows us to define optimality in the graph construction step of GSSL concisely, addressing our first research question. We subsequently present a tailored optimal solution featuring an asymmetric graph structure that aligns with our proposed intuition. In response, we introduce the Block-wise Affinity Graph Learning (BAGL) algorithm by leveraging duality and the fast proximal gradient method, addressing our second research question. Finally, we demonstrate that BAGL ensures a sub-linear global convergence rate of  $O(1 / k)$  and can alleviate issues of noisy node features, addressing our third research question.
In summary, this work offers four main contributions. First, we provide a succinct definition of the optimality of the affinity graph in GSSL, and through rigorous derivation, propose an ad-hoc solution with an asymmetric structure. Second, we design a block-wise graph learning framework, BAGL, to infer the weights in the optimal graph structure. Third, we prove that a global sub-linear convergence rate is guaranteed for BAGL and analyze other benefits. Fourth, we perform extensive experiments on synthetic and real-world datasets to demonstrate the effectiveness and efficiency of BAGL.
We present a formulation for the GSSL problem, comprising two steps: graph construction and label inference [52]. Our study primarily investigates the optimal construction of the affinity graph in the first phase to facilitate enhanced performance in the second label inference phase.
Given a set of data points  $\{\pmb{x}_i\}_{i=1}^n$ , where each  $\pmb{x}_i \in \mathbb{R}^d$  is sampled from a  $d$  dimensional feature space. In this paper, we interchangeably use the terms node, point, and sample to refer to  $\pmb{x}_i$ . Each sample  $\pmb{x}_i$  has a label  $y_i \in \mathbb{N}_c$ , where  $\mathbb{N}_c = \{i \in \mathbb{N}^+ \mid 1 \leq i \leq c\}$  with  $c$  being the number of classes. Given the labels of the  $l$  samples  $\{\pmb{x}_i\}_{i=1}^l$  as  $\{y_i\}_{i=1}^l$ , the ultimate goal of general transductive SSL is to infer the labels  $\{y_i\}_{i=l+1}^{l+u}$  for the remaining  $u$  unlabeled samples  $\{\pmb{x}_i\}_{i=l+1}^{l+u} (n = l + u)$ . As a subcategory of general SSL, GSSL methods first construct a graph  $\mathcal{G} = \{\mathcal{V}, \mathcal{E}, \mathcal{W}\}$  based on all the training samples  $\{\pmb{x}_i\}_{i=1}^n$  and partially given labels  $\{y_i\}_{i=1}^l$ . Here,  $\mathcal{V}$  is the node set with  $|\mathcal{V}| = n$ . Each node represents each sample  $\pmb{x}_i$ .  $\mathcal{E}$  is the edge set in which each edge  $(i,j)$  is assigned with a weight  $W_{ij}$  (the  $i$ -th row,  $j$ -th column entry in  $\mathcal{W} \in \mathbb{R}^{n \times n}$ ) to reflect the affinity or similarity between the sample pair  $(\pmb{x}_i, \pmb{x}_j)$ . Generally, a larger weight indicates a higher level of similarity.  $W_{ij} = 0$  indicates no edge between node  $i$  and node  $j$ . Therefore, the key challenge in the first graph construction step is to generate  $\mathcal{W}$  based on  $\{\pmb{x}_i\}_{i=1}^n$  and  $\{y_i\}_{i=1}^l$  so that the underlying manifold of the data is properly encoded. In the second step, various label inference algorithms can be performed on  $\mathcal{G}$  to propagate the given labels  $\{y_i\}_{i=1}^l$  and make predictions  $\{y_i\}_{i=l+1}^{l+u}$  for unlabeled nodes.
However, the performance of the label inference step is significantly contingent on the quality of the weight matrix  $W$  from the first graph construction step. This paper aims to address three critical research questions pertaining to the graph construction step in GSSL. First, what constitutes the overall structure of the optimal weight matrix,  $W^{*}$ ? (Sect. 3.1). We define the optimal weight matrix,  $W^{*}$ , as the one that gives the best prediction results when used with the same label inference method
across different graphs  $\mathbf{W}$ . Second, how to find an efficient method for optimizing the entries in  $\mathbf{W}^{*}$ ? (Sect. 3.2). Third, what kinds of benefits can  $\mathbf{W}^{*}$  bring from the theoretical perspective? (Sect. 3.3)
2.2 Recap on the Unified Framework for Label Inference Step in GSSL
Before we delve into the proposed structure for the optimal affinity graph, we first revisit the unified framework of the label inference step, allowing for a seamless definition of the optimal affinity graph.
If we assume the affinity graph has been constructed, then numerous influential label inference methods [85, 82, 83, 4, 5, 9] can be performed over this graph to infer the labels for the unlabeled nodes. However, the majority of them can be incorporated into the following framework.
We first define the node feature matrix  $\mathbf{X} \in \mathbb{R}^{n \times d}$  as  $\mathbf{X} = [\pmb{x}_1, \dots, \pmb{x}_n]^{\top}$  and the predicted soft label matrix  $\mathbf{F} \in \mathbb{R}^{n \times c}$  as  $\mathbf{F} = [\pmb{f}_1, \dots, \pmb{f}_n]^{\top}$  with  $\pmb{f}_i \in \mathbb{R}^c (1 \leq i \leq n)$ . Ground-truth label matrix is given as  $\mathbf{Y} = [\pmb{y}_1, \dots, \pmb{y}_n]^{\top} \in \{0, 1\}^{n \times c}$  with  $\pmb{y}_i \in \{0, 1\}^c (1 \leq i \leq n)$ . Here, for the labeled nodes  $\{\pmb{x}_i\}_{i=1}^l$ ,  $\pmb{y}_i$  is a one-hot vector in which  $Y_{ij} = 1$  if  $\pmb{x}_i$  belongs to class  $j$  ( $y_i = j$ ), and  $Y_{ij} = 0$ , otherwise. For the unlabeled nodes  $\{\pmb{x}_i\}_{i=l+1}^{l+u}$ ,  $\pmb{y}_i$  is an all-zero vector for initialization.
Driven by the geometry of the affinity graph, we can unify these label inference algorithms as a minimizer of the optimization problem as Problem (1).
$$
\boldsymbol {F} ^ {*} = \underset {\boldsymbol {F}} {\arg \min } Q (\boldsymbol {F}) = \underset {\boldsymbol {F}} {\arg \min } \left\{\operatorname {T r} \left(\boldsymbol {F} ^ {\intercal} \boldsymbol {S} \boldsymbol {F}\right) + \operatorname {T r} \left((\boldsymbol {F} - \boldsymbol {Y}) ^ {\intercal} \boldsymbol {\Lambda} (\boldsymbol {F} - \boldsymbol {Y})\right). \right. \tag {1}
$$
Note that the loss function  $Q(\pmb{F})$  consists of a quadratic variation term  $\mathrm{Tr}(F^{\top} S F)$  as the graph smoothness regularizer, and a quadratic Frobenius error norm  $\| \pmb{F} - \pmb{Y} \|_F^2 = \mathrm{Tr}((\pmb{F} - \pmb{Y})^{\top} (\pmb{F} - \pmb{Y}))$ , both of which should ideally be small subject to a trade-off parameter  $\Lambda$  between them. Here the smoothing matrix  $S = s(\pmb{W}) \in \mathbb{S}_+^n$  in the first term, with  $s \colon \mathbb{R}^{n \times n} \to \mathbb{S}_+^{n \times n}$ , is positive semidefinite and determined by the weight matrix  $W$  of the graph to ensure the adjacent nodes share similar predictions. The second term measures the distance between predicted results and initial assignments, and restricts the output for labeled nodes from deviating too much compared with the ground-truth.  $\Lambda$  is a diagonal matrix with  $\Lambda_{ii} \geq 0$  and  $S + \Lambda$  must be invertible to avoid trivial solutions.
By the first-order optimality condition, we can easily obtain the optimal solution for Problem (1).
$$
\boldsymbol {F} ^ {*} = (\boldsymbol {S} + \boldsymbol {\Lambda}) ^ {- 1} \boldsymbol {\Lambda} \boldsymbol {Y} = (s (\boldsymbol {W}) + \boldsymbol {\Lambda}) ^ {- 1} \boldsymbol {\Lambda} \boldsymbol {Y}. \tag {2}
$$
Based on the optimal soft label matrix  $\pmb{F}^{*}$ , the final predicted label for each node is given as  $\hat{y}_i = \hat{y}(\pmb{f}_i) = \arg \max_{1 \leq j \leq c} F_{ij}^*$ . It is worth noting that the unified optimization framework in Problem (1) can admit most of the mainstream GSSL methods. For instance, if we set the smoothing matrix as the normalized graph Laplacian matrix  $S = \mathcal{L} = s(W) = D^{-\frac{1}{2}}(D - W)D^{-\frac{1}{2}}$  and  $\Lambda = \lambda I$ , we can easily recover one of the most popular label inference methods for GSSL, Local and Global Consistency (LGC) [82], as  $\pmb{F}^{*} = \arg \min_{\pmb{F}} \left\{ \frac{1}{2} \sum_{i,j=1}^{n} \left\| \frac{\pmb{f}_i}{\sqrt{D_{ii}}} - \frac{\pmb{f}_j}{\sqrt{D_{jj}}} \right\|_2^2 W_{ij} + \lambda \sum_{i=1}^{n} \| \pmb{f}_i - \pmb{y}_i \|_2^2 \right\}$ . Here,  $\mathcal{L} \in \mathbb{S}_+^n$  is defined as  $\mathcal{L} = D^{-\frac{1}{2}}LD^{-\frac{1}{2}}$ , where the combinatorial Laplacian matrix  $L \in \mathbb{S}_+^n$  is given as  $L = D - W$ , and the degree matrix  $D \in \mathbb{R}^{n \times n}$  is a diagonal matrix defined as  $D_{ii} = \sum_{j=1}^{n} W_{ij}$ . We summarize several representative works under this unified framework in Table 5 with explicit forms of  $S$  (or  $s(W)$ ) and  $\Lambda$  in Appendix B.2.
3.1 Motivation: Optimal Affinity Graph Structure
3.1.1 Definition of the Optimality of the Affinity Graph
If we perform the same label inference algorithm from the above-mentioned generalized framework on all possible affinity graphs  $W$ , the optimal graph  $W^{*}$  will enable the label inference step to obtain the most accurate predictions. Under the above-mentioned unified framework for label inference (Problem (1)), the weight matrix  $W$  is fixed while the soft label matrix  $F$  is the optimization variable. This is due to our goal of executing label inference to attain the optimal  $F^{*}$  given the affinity graph  $W$ . Similarly, when we want to construct the optimal graph  $W^{*}$  given the label inference framework,
we consider the weight matrix  $\mathbf{W}$  as the optimization variable, keeping the soft label matrix  $\mathbf{F}$  fixed as the solution in Eq. (2). The form of  $Q(\cdot)$  remains unchanged as it is related to the generalization bound for GSSL that would be discussed in Appendix D.2. This approach aids in identifying the "optimal" affinity graph for the label inference algorithms under the unified framework.
Definition 1 (Optimality of the Affinity Graph). The affinity graph  $\mathcal{G}$  is optimal for label inference under the unified GSSL framework if its weight matrix  $\mathbf{W}$  is the minimizer of Problem (3).
$$
\min  _ {\boldsymbol {W} \in \mathbb {R} ^ {n \times n}} \left\{\operatorname {T r} \left(\boldsymbol {F} ^ {\intercal} s (\boldsymbol {W}) \boldsymbol {F}\right) + \operatorname {T r} \left((\boldsymbol {F} - \boldsymbol {Y}) ^ {\intercal} \boldsymbol {\Lambda} (\boldsymbol {F} - \boldsymbol {Y})\right) \right\} \quad s. t. \quad \boldsymbol {F} = (s (\boldsymbol {W}) + \boldsymbol {\Lambda}) ^ {- 1} \boldsymbol {\Lambda} \boldsymbol {Y}, \tag {3}
$$
Although the optimization Problem (3) is intractable in general due to the various forms of  $s(\mathbf{W})$  in the constraint, we can present an ad-hoc optimal structure of  $\mathbf{W}^*$  that is independent of  $s(\mathbf{W})$ , which motivates our proposed method in Sect. 3.2 from a theoretical perspective.
3.1.2 Structure of the Optimal Affinity Graph
We then present an equivalent proposition in Theorem 1, which provides a necessary and sufficient condition for the optimality of  $\mathbf{W}^*$  in Problem (3). Motivated by some classic graph sharpening techniques [47, 15, 46], Theorem 1 helps to circumvent the challenges of directly dealing with Problem (3) and holds regardless of the forms of  $s(\mathbf{W})$  (Appendix E.1).
Theorem 1.  $W^{*}$  obtained by Definition 1 is optimal if and only if  $\mathbf{Y}_l = \mathbf{F}_l$  holds.  $\mathbf{Y}_l\in \{0,1\}^{l\times c}$ ,  $\mathbf{F}_l\in \mathbb{R}^{l\times c}$  are the ground-truth label matrix, and the soft label matrix for labeled nodes.
To put it in a simpler way, Theorem 1 tells us that if the affinity graph is optimal, then after we perform the label inference algorithm for GSSL, the predicted soft label for the labeled nodes would coincide with the ground truth precisely. The converse of this observation also holds true. Unfortunately, it remains an open question to solve  $\mathbf{Y}_l = \mathbf{F}_l$  by listing all possible classes of solutions due to the complex interconnection of  $\mathbf{F}$  and  $\mathbf{W}$ . However, we can provide a simple ad-hoc solution for  $\mathbf{Y}_l = \mathbf{F}_l$  in Proposition 1. For one thing, this asymmetric graph structure, in the theoretical sense, conforms to the intuition of the better affinity graph we discussed earlier. For another, it also sheds some light on the proposed optimization framework to infer the weights in this graph structure later.
Proposition 1. If  $W^{*}$  can be expressed as (4),  $W^{*}$  is an optimal solution given by (3) in Definition 1.
$$
\boldsymbol {W} ^ {*} = \left( \begin{array}{c c} \boldsymbol {O} & \boldsymbol {O} \\ \boldsymbol {W} _ {u l} & \boldsymbol {W} _ {u u} \end{array} \right), \tag {4}
$$
where  $\pmb{W}_{ul} \in \mathbb{R}^{u \times l}$ , and  $\pmb{W}_{uu} \in \mathbb{R}^{u \times u}$  can be non-zero submatrices with arbitrary entries.
Proposition 1 provides an ad-hoc solution to Problem (3) (Appendix E.2). If the constructed graph is asymmetrical with edges from unlabeled nodes to labeled nodes only, then it is optimal by Definition 1. This asymmetrical optimal graph structure answers the first research question in Sect. 2.1.
The optimal asymmetric graph structure presented in Proposition 1 offers meaningful interpretations and notable advantages. It effectively eliminates the influence from unlabeled nodes to labeled nodes by enforcing  $\mathbf{W}_{ll}$  and  $\mathbf{W}_{lu}$  to be zero matrices. This aligns with the intuition that label information should be propagated from labeled nodes to unlabeled nodes in the GSSL methods, rather than the other way around. More technically, when applying the classic label inference algorithm LGC [82] on this optimal graph structure, the soft label matrix for unlabeled nodes now becomes  $\mathbf{F}_u = (\mathbf{I} - \mu \mathbf{W}_{uu})^{-1}\bar{\mathbf{W}}_{ul}\mathbf{Y}_L$  with some constant  $0\leq \mu \leq 1$ . This formulation indicates that the LGC algorithm spreads supervision information from labeled nodes to unlabeled nodes once through  $\mathbf{W}_{ul}\mathbf{Y}_l$ , followed by propagating this information solely among unlabeled nodes through  $(\mathbf{I} - \mu \mathbf{W}_{uu})^{-1} = \mathbf{I} + \mu \mathbf{W}_{uu} + \mu^2\mathbf{W}_{uu}^2 +\dots$ . By Theorem 1, this optimal asymmetric graph guarantees zero empirical risk on the labeled nodes. Moreover, many existing graph construction methods in GSSL primarily focus on node features while disregarding label information. Consequently, these methods may produce heterophilous edges that connect nodes with similar features but different labels. Such edges violate the manifold assumption in GSSL, where nodes with the same label tend to be linked. During the label inference step, the label information of these nodes connected by heterophilous edges confuses each other during propagation, resulting in misleading predictions. By setting  $\mathbf{W}_{ll} = \mathbf{O}$ , our method eliminates these heterophilous edges completely. As a result, it increases the edge homophily ratio of the constructed graph and enhances the robustness of subsequent label inference algorithms, as validated in Appendix D.1.
3.2 Implementation: Block-wise Graph Learning Algorithm
3.2.1 Framework
By differentiating the roles that labeled and unlabeled nodes would play, we arrive at an optimal structure for the affinity graph in Proposition 1. However, an efficient algorithm to infer the exact weights or entries for blocks in Eq. (4) is still in demand. Motivated by previous works [29, 30, 19], we make the following reasonable restrictions or assumptions on  $W$  to obtain a more meaningful graph for GSSL. First, the features for two directly connected nodes should not vary too much, no matter whether they are labeled or not. This agrees with the manifold assumption in GSSL, where links tend to form between similar nodes. Violation of this fundamental assumption usually causes a significant performance drop in the label inference step. Second, the constructed graph should be well-connected in terms of both  $W_{ul}$  and  $W_{uu}$  to ensure the supervision information could propagate freely among all nodes. Otherwise, some disconnected nodes may never receive any label information. Third, it is desirable to control the sparsity of the graph, avoiding an overly sparse or overly dense graph. We will empirically demonstrate the effects of the sparsity control later. We propose two similar optimization frameworks for  $W_{ul}$  and  $W_{uu}$  in Eq. (4) as Problem (5) and (6).
$$
\min  _ {\boldsymbol {W} _ {u l}} \| \boldsymbol {W} _ {u l} \odot \boldsymbol {Z} _ {u l} \| _ {1} - \alpha_ {1} \mathbf {1} ^ {\intercal} \log (\boldsymbol {W} _ {u l} \mathbf {1}) - \alpha_ {1} \mathbf {1} ^ {\intercal} \log (\boldsymbol {W} _ {u l} ^ {\intercal} \mathbf {1}) + \beta_ {1} \| \boldsymbol {W} _ {u l} \| _ {F} ^ {2}, \text {s . t .} \boldsymbol {W} _ {u l} \geq 0. \tag {5}
$$
$$
\min  _ {\boldsymbol {W} _ {u u}} \| \boldsymbol {W} _ {u u} \odot \boldsymbol {Z} _ {u u} \| _ {1} - \alpha_ {2} \mathbf {1} ^ {\intercal} \log (\boldsymbol {W} _ {u u} \mathbf {1}) - \alpha_ {2} \mathbf {1} ^ {\intercal} \log (\boldsymbol {W} _ {u u} ^ {\intercal} \mathbf {1}) + \beta_ {2} \| \boldsymbol {W} _ {u u} \| _ {F} ^ {2}, \text {s . t .} \boldsymbol {W} _ {u u} \geq 0. \tag {6}
$$
Here, we define the pairwise distance matrix  $\mathbf{Z} = \left( \begin{array}{cc}\mathbf{Z}_{ll} & \mathbf{Z}_{ul}^{\top}\\ \mathbf{Z}_{ul} & \mathbf{Z}_{uu} \end{array} \right)\in \mathbb{R}_{+}^{n\times n}$  with  $Z_{ij} = \| \pmb {x}_i - \pmb {x}_j\| _2^2$ .  $\odot$  is the Hadamard product and  $\log (\cdot)$  is the element-wise logarithm operator. Since Problem (5) and (6) share the same form, we will only focus on Problem (5) as an example. The first term  $\| \pmb {W}_{ul}\odot \pmb {Z}_{ul}\| _1 = \sum_{1\leq i\leq u,1\leq j\leq l}W_{ij}\| \pmb {x}_i - \pmb {x}_j\| _2^2$  encourages similar nodes to be connected with larger weights, meeting the manifold assumption. The second and third logarithmic barrier term act on the out-degree and in-degree vectors to make sure that each unlabeled node is connected by at least one labeled node and vice versa, improving the overall connectivity of the graph. The last Frobenius norm term measures the sparsity of the graph. The parameters  $\alpha_{1},\alpha_{2},\beta_{1},\beta_{2}$  are positive.
3.2.2 Optimization
For convenience of presentation, we view the entries in  $\mathbf{W}_{ul}(\mathbf{W}_{uu})$  as a new vector  $\mathbf{w} = \operatorname{vec}[W_{ul}] \in \mathbb{R}_+^{ul}$ . Similarly, we have  $\mathbf{z} = \operatorname{vec}[Z_{ul}] \in \mathbb{R}_+^{ul}$ . Accordingly, a linear mapping matrix  $T_1 \in \{0,1\}^{u \times ul}$  transforms the edge weights to the corresponding out-degree vector (i.e.  $T_1 \mathbf{w} = \mathbf{W}_{ul} \mathbf{1}$ ). Similarly, we have  $T_2 \mathbf{w} = \mathbf{W}_{ul}^{\top} \mathbf{1}$  for the in-degree vector. Further, if we let  $T^{\top} = (T_1^{\top}, T_2^{\top}) \in \{0,1\}^{n \times ul}$ ,  $\alpha = \alpha_1 = \alpha_2$ , and  $\beta = \beta_1$  we can easily transform Problem (5) into Problem (7) (primal) in a more compact way with an extra linear constraint.
$$
\min  _ {\boldsymbol {w}, \boldsymbol {v}} f (\boldsymbol {w}) + g (\boldsymbol {v}) \quad \text {s . t .} \quad \boldsymbol {v} = \boldsymbol {T} \boldsymbol {w}, \tag {7}
$$
with  $f(\pmb{w}) = \pmb{w}^{\mathsf{T}}\pmb{z} + \beta \| \pmb{w}\|_{2}^{2} + \mathbb{I}_{\{\pmb{w}\geq 0\}}$ $g(\pmb {v}) = -\alpha \mathbf{1}^{\intercal}\log (\pmb {v}).$
The state-of-the-art method [61] applies the linearized alternating direction method of multipliers (ADMM) algorithm [8] directly to the primal problem with a similar structure in Problem (7), which lacks the theoretical guarantee on its convergence rate since the objective function in Problem (7) has no Lipschitz gradient. Motivated by the recent work [45], we circumvent this critical issue by applying the FISTA algorithm [2], a proximal gradient method, to the dual problem of (7) instead. Motivated by recent advances [3, 61], we can now provide a better convergence rate with rigorous theoretical analysis so that our proposed graph construction method is much more efficient with guarantees.