SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Remark A.1. Clearly, we have  $R_{ij} = R_{ji}$  for any  $i$  and  $j$ .
Example: For instance, using Figure 7 as an example, we consider a path from node 1 to 10 as
$$
d _ {1, 1 0} = \{(1, 4), (4, 5), \underbrace {(5 , 6)} _ {\text {b r i d g e}}, (6, 7), (7, 8), (8, 9), \underbrace {(9 , 1 0)} _ {\text {b r i d g e}} \}, \tag {18}
$$
the bridge path is then given as
$$
\mathcal {B} _ {1, 1 0} = \{(5, 6), (9, 1 0) \}, \tag {19}
$$
Figure 7: An example of computing bridge path. For example, from node 1 to 10, all the paths will cover the bridges (5, 6) and (9, 10). Hence, we have  $\mathcal{B}_{1,10} = \{(5, 6), (9, 10)\}$ .
and  $R_{1,10}$  is computed as
$$
\begin{array}{l} R _ {1, 1 0} = \sqrt {S _ {1 , 1}} \cdot g _ {1, 1 0} (\boldsymbol {R}) \cdot \sqrt {S _ {1 0 , 1 0}} \\ = \sqrt {S _ {1 , 1}} \cdot \frac {R _ {1 , 5}}{\sqrt {S _ {1 , 1} S _ {5 , 5}}} \cdot \frac {R _ {5 , 6}}{\sqrt {S _ {5 , 5} S _ {6 , 6}}} \cdot \frac {R _ {6 , 9}}{\sqrt {S _ {6 , 6} S _ {9 , 9}}} \cdot \frac {R _ {9 , 1 0}}{\sqrt {S _ {9 , 9} S _ {1 0 , 1 0}}} \cdot \frac {R _ {1 0 , 1 0}}{\sqrt {S _ {1 0 , 1 0} S _ {1 0 , 1 0}}} \cdot \sqrt {S _ {1 0 , 1 0}}. \tag {20} \\ \end{array}
$$
Particularly, we note that
- The bridge path  $\mathcal{B}_{1,10}$  can not be written as  $\{(9,10), (5,6)\}$  as  $d_{1,10}$  would always visit the bridge  $(5,6)$  before visiting  $(9,10)$ .
- The bridge path  $\mathcal{B}_{1,10}$  can not be written as  $\{(6,5),(9,10)\}$  as  $d_{1,10}$  leave first cluster at node 5, then enter second cluster at node 6. Therefore, we have  $\psi(1) = \psi(5)$ .
- It is easy to observe that  $\mathcal{B}_{10,1} = \{(10,9), (6,5)\}$ . Based on definitions, we can obtain  $R_{10,1} = R_{1,10}$ .
- Specially, if the matrix  $S$  is given as a correlation matrix, i.e.,  $\mathrm{diag}(S) = 1$ , then  $R_{1,10} = R_{1,5} \cdot R_{5,6} \cdot R_{6,9} \cdot R_{9,10} \cdot R_{10,10}$ , which can be seen as a continued produce on a path, known as path product.
The matrix  $\pmb{R}$  shares some important properties introduced as follows.
Lemma A.2. For any  $i,j\in \mathcal{V}_k$ $\widehat{\pmb{R}}_k$  together with  $\widehat{\Theta}_k$  would satisfy the following equations
$$
\begin{array}{l} - \left[ \widehat {\boldsymbol {R}} _ {k} \right] _ {\pi (i), \pi (j)} + S _ {i, j} = 0 \quad i f i = j, (21a) \\ - \left[ \widehat {\boldsymbol {R}} _ {k} \right] _ {\pi (i), \pi (j)} + S _ {i, j} - \Lambda_ {i, j} = 0 \quad i f i \neq j, \left[ \widehat {\boldsymbol {\Theta}} _ {k} \right] _ {\pi (i), \pi (j)} \neq 0, (21b) \\ - \left[ \widehat {\boldsymbol {R}} _ {k} \right] _ {\pi (i), \pi (j)} + S _ {i, j} - \Lambda_ {i, j} \leq 0 \quad i f i \neq j, \left[ \widehat {\boldsymbol {\Theta}} _ {k} \right] _ {\pi (i), \pi (j)} = 0, (21c) \\ \end{array}
$$
Proof. Let's denote  $\Gamma_{\pi(i),\pi(j)}$  as the dual variables associated with the constraints  $[\widehat{\Theta}_k]_{\pi(i),\pi(j)} \leq 0$ . With  $\widehat{R}_k = \widehat{\Theta}_k^{-1}$ , the KKT conditions include
$$
\text {s t a t i o n a r i t y :} \quad - \widehat {\boldsymbol {R}} _ {k} + \boldsymbol {S} _ {\mathcal {V} _ {k}} - \boldsymbol {\Lambda} _ {\mathcal {V} _ {k}} + \boldsymbol {\Gamma} _ {\mathcal {V} _ {k}} = \boldsymbol {0}, \tag {22a}
$$
$$
\text {p r i m a l} \quad [ \widehat {\Theta} _ {k} ] _ {\pi (i), \pi (j)} \leq 0, \forall \pi (i) \neq \pi (j) \tag {22b}
$$
$$
\text {d u a l} \quad \Gamma_ {\pi (i), \pi (j)} \geq 0, \forall \pi (i) \neq \pi (j) \tag {22c}
$$
$$
\text {c o m p l e m e n t a r y} \quad [ \widehat {\Theta} _ {k} ] _ {\pi (i), \pi (j)} \cdot \Gamma_ {\pi (i), \pi (j)} = 0, \forall \pi (i) \neq \pi (j) \tag {22d}
$$
We can eliminate the dual variables as follows:
- For  $[\widehat{\Theta}_k]_{\pi(i),\pi(j)} < 0$ , the complementary slackness leads to  $\Gamma_{\pi(i),\pi(j)} = 0$ , which implies  $-[\widehat{R}_k]_{\pi(i),\pi(j)} + S_{i,j} - \Lambda_{i,j} = 0$ .
- For  $[\widehat{\Theta}_k]_{\pi(i), \pi(j)} = 0$ , we have  $\Gamma_{\pi(i), \pi(j)} \geq 0$ , which implies  $-[\widehat{\pmb{R}}_k]_{\pi(i), \pi(j)} + S_{i,j} - \Lambda_{i,j} \leq 0$ . Hence, Lemma A.2 is obtained.
In particular, we note that  $[\widehat{\pmb{R}}_k]_{\pi (i),\pi (i)} = S_{i,i}$  holds for any  $i\in \mathcal{V}_k$  according to (21a). As a result, the following property can be easily verified via the definitions.
Corollary A.3. Suppose that two nodes  $i$  and  $j$  are in different clusters, i.e.,  $\psi (i)\neq \psi (j)$ , and the bridge path is not empty and denoted by  $\mathcal{B}_{ij}$  with form of (14), then we have
$$
R _ {i j} = \frac {R _ {i , u _ {t}} \cdot R _ {u _ {t} , j}}{S _ {u _ {t} , u _ {t}}}, \quad \forall t = 0, \dots , 2 T + 1, \tag {23}
$$
Proof. Corollary A.3 can be easily verified via definitions. If  $t = 0$  or  $t = 2T + 1$ , Corollary A.3 holds as  $R_{i,i} = S_{i,i}$  and  $R_{j,j} = S_{j,j}$ . Let  $s$  be a number that satisfies  $0 \leq s \leq 2T$ . For any  $(u_s, u_{s+1}) \in \mathcal{B}_{ij}$ , we have
$$
\begin{array}{l} R _ {i, j} \cdot S _ {u _ {s}, u _ {s}} = \underbrace {\sqrt {S _ {i , i}} \cdot \frac {R _ {u _ {0} , u _ {1}}}{\sqrt {S _ {u _ {0} , u _ {0}} S _ {u _ {1} , u _ {1}}}} \cdots \cdots \frac {R _ {u _ {s - 1} , u _ {s}}}{\sqrt {S _ {u _ {s - 1} , u _ {s - 1}} S _ {u _ {s} , u _ {s}}}} \cdot \sqrt {S _ {u _ {s} , u _ {s}}}} _ {R _ {i, u _ {s}}} \\ \cdot \underbrace {\sqrt {S _ {u _ {s} , u _ {s}}} \cdot \frac {R _ {u _ {s} , u _ {s + 1}}}{\sqrt {S _ {u _ {s} , u _ {s}} S _ {u _ {s + 1} , u _ {s + 1}}}}} \dots \dots \frac {R _ {u _ {2 T - 1} , u _ {2 T}}}{\sqrt {S _ {u _ {2 T - 1} , u _ {2 T - 1}} S _ {u _ {2 T} , u _ {2 T}}}} \cdot \sqrt {S _ {j , j}}}. \tag {24} \\ \end{array}
$$
Clearly, Corollary A.3 is obtained.
The aforementioned properties of matrix  $\mathbf{R}$  are important for establishing the proof of Theorem 3.4 and Theorem 3.3.
A.2 Proof of Theorem 3.4
Proof. Let  $\pmb{F} = \Theta \pmb{R}$ , where  $\Theta$  and  $\pmb{R}$  have the forms of (7) and (8), respectively. The target is to show  $F_{ii} = 1, \forall i$  and  $F_{ij} = 0, \forall i \neq j$  using the following results we have derived.
(a) For any  $i,j\in \mathcal{V}_k$  , we have
$$
\boldsymbol {\Theta} _ {i j} = \left\{ \begin{array}{l l} \left[ \widehat {\boldsymbol {\Theta}} _ {k} \right] _ {\pi (i), \pi (j)} + \frac {1}{S _ {i i}} \cdot \sum_ {(i, m) \in \mathcal {B}} \frac {T _ {i m} ^ {2}}{S _ {i i} S _ {m m} - T _ {i m} ^ {2}} & \text {i f} i = j, \\ \left[ \widehat {\boldsymbol {\Theta}} _ {k} \right] _ {\pi (i), \pi (j)} & \text {i f} i \neq j, \end{array} \right. \tag {25}
$$
and  $R_{ij} = [\widehat{\pmb{R}}_k]_{\pi (i),\pi (j)}$  according to the definitions.
(b) For any nodes  $i$  and  $j$  in different clusters, suppose that  $\mathcal{B}_{ij}$  is not empty and edge  $(u,v)$  is a bridge in  $\mathcal{B}_{ij}$ , then  $R_{ij}S_{uu} = R_{iu}R_{uj}$  and  $R_{ij}S_{vv} = R_{iv}R_{vj}$  hold according to Corollary A.3.
(c) The matrix  $\widehat{\pmb{F}}_k$ , which is defined as  $\widehat{\pmb{F}}_k = \widehat{\pmb{R}}_k\widehat{\pmb{\Theta}}_k$  is equal to the identity matrix, i.e,  $\widehat{\pmb{F}}_k = \pmb{I}$ . In other words, we have
$$
\left[ \widehat {\boldsymbol {F}} _ {k} \right] _ {\pi (i), \pi (i)} = 1, \text {a n d} \left[ \widehat {\boldsymbol {F}} _ {k} \right] _ {\pi (i), \pi (j)} = 0, \quad \forall i \neq j \text {a n d} i, j \in \mathcal {V} _ {k}. \tag {26}
$$
Based on (a), (b), and (c), we first show that  $F_{ii} = 1$  holds for any  $i \in \{1, \dots, p\}$ . We suppose that node  $i$  is in  $k$ -the cluster, i.e.,  $k = \psi(i)$ . Then, by using  $\forall m \notin (\mathcal{V}_k \cup \mathcal{N}(i)) : \Theta_{im} = 0$  from definitions in (7), we have
$$
\begin{array}{l} F _ {i i} = \sum_ {m = 1} ^ {p} R _ {i m} \Theta_ {i m} = \sum_ {m \in \mathcal {V} _ {k} \backslash \{i \}} R _ {i m} \Theta_ {i m} + R _ {i i} \Theta_ {i i} + \sum_ {(i, m) \in \mathcal {B}} R _ {i m} \Theta_ {i m} \\ \stackrel {(a)} {=} \sum_ {m \in \mathcal {V} _ {k} \backslash \{i \}} \left[ \widehat {\boldsymbol {R}} _ {k} \right] _ {\pi (i), \pi (m)} \left[ \widehat {\boldsymbol {\Theta}} _ {k} \right] _ {\pi (i), \pi (m)} + \left[ \widehat {\boldsymbol {R}} _ {k} \right] _ {\pi (i), \pi (i)} \left[ \widehat {\boldsymbol {\Theta}} _ {k} \right] _ {\pi (i), \pi (i)} \\ + S _ {i i} \cdot \frac {1}{S _ {i i}} \sum_ {(i, m) \in \mathcal {B}} \frac {T _ {i m} ^ {2}}{S _ {i i} S _ {m m} - T _ {i m} ^ {2}} + \sum_ {(i, m) \in \mathcal {B}} T _ {i m} \left(- \frac {T _ {i m}}{S _ {i i} S _ {m m} - T _ {i m} ^ {2}}\right) \\ = \sum_ {m \in \mathcal {V} _ {k} \backslash \{i \}} [ \widehat {\boldsymbol {R}} _ {k} ] _ {\pi (i), \pi (m)} [ \widehat {\boldsymbol {\Theta}} _ {k} ] _ {\pi (i), \pi (m)} + [ \widehat {\boldsymbol {R}} _ {k} ] _ {\pi (i), \pi (i)} [ \widehat {\boldsymbol {\Theta}} _ {k} ] _ {\pi (i), \pi (i)} \stackrel {(c)} {=} [ \widehat {\boldsymbol {F}} _ {k} ] _ {\pi_ {k} (i), \pi_ {k} (i)} = 1. \tag {27} \\ \end{array}
$$
Therefore,  $T_{ii} = 1$  holds for any  $i \in \{1, \dots, p\}$ .
Let us proceed with the proof that  $F_{ij} = \sum_{m=1}^{p} R_{im} \Theta_{mj} = 0$  for any  $i \neq j$ . Without loss of generality, we suppose that node  $j$  is in the  $k$ -th cluster, i.e.,  $k = \psi(j)$ . According to the definitions, we have
$$
\forall m \notin \left(\mathcal {V} _ {k} \cup \mathcal {N} (j)\right): \Theta_ {m j} = 0. \tag {28}
$$
Here, there would be three cases to analyze.
The first case happens where  $i \in \mathcal{V}_k$ , which means that nodes  $i$  and  $j$  are in the same cluster. In this case, we have
$$
\begin{array}{l} F _ {i j} = \sum_ {m = 1} ^ {p} R _ {i m} \Theta_ {m j} = \sum_ {m \in \mathcal {V} _ {k} \backslash \{j \}} R _ {i m} \Theta_ {m j} + R _ {i j} \Theta_ {j j} + \sum_ {(m, j) \in \mathcal {B}} R _ {i m} \Theta_ {m j} \\ \stackrel {(a)} {=} \sum_ {m \in \mathcal {V} _ {k} \backslash \{j \}} \left[ \widehat {\boldsymbol {R}} _ {k} \right] _ {\pi (i), \pi (m)} \left[ \widehat {\boldsymbol {\Theta}} _ {k} \right] _ {\pi (m), \pi (j)} \\ + [ \widehat {\boldsymbol {R}} _ {k} ] _ {\pi (i), \pi (j)} \left([ \widehat {\boldsymbol {\Theta}} _ {k} ] _ {\pi (j), \pi (j)} + \frac {1}{S _ {j j}} \sum_ {(m, j) \in \mathcal {B}} \frac {T _ {m j} ^ {2}}{S _ {m m} S _ {j j} - T _ {m j} ^ {2}}\right) + \sum_ {(m, j) \in \mathcal {B}} R _ {i m} \Theta_ {m j} \\ \stackrel {(c)} {=} [ \widehat {\boldsymbol {F}} _ {k} ] _ {\pi_ {k} (i), \pi_ {k} (j)} + R _ {i j} \left(\frac {1}{S _ {j j}} \sum_ {(m, j) \in \mathcal {B}} \frac {T _ {m j} ^ {2}}{S _ {m m} S _ {j j} - T _ {m j} ^ {2}}\right) + \sum_ {(m, j) \in \mathcal {B}} R _ {i m} \Theta_ {m j} \\ \stackrel {(b, c)} {=} 0 + R _ {i j} \left(\frac {1}{S _ {j j}} \sum_ {(m, j) \in \mathcal {B}} \frac {T _ {m j} ^ {2}}{S _ {m m} S _ {j j} - T _ {m j} ^ {2}}\right) + \sum_ {(m, j) \in \mathcal {B}} \frac {R _ {i j} R _ {j m}}{S _ {j j}} \left(- \frac {T _ {m j}}{S _ {m m} S _ {j j} - T _ {m j} ^ {2}}\right) \\ = 0. \tag {29} \\ \end{array}
$$
The second case happens where  $i \notin \mathcal{V}_k$ , but there is no path between nodes  $i$  and  $j$ . In such a case, for any node  $m$  in  $\mathcal{V}_k \cup \mathcal{N}(j)$ , there is also no path from nodes  $i$  to  $m$ . Consequently, based on the definition of  $R$ , we have
$$
\forall m \in \mathcal {V} _ {k} \cup \mathcal {N} (j): \quad R _ {i m} = 0. \tag {30}
$$
$$
F _ {i j} = \sum_ {m = 1} ^ {p} R _ {i m} \Theta_ {m j} = \sum_ {m \in \mathcal {V} _ {k} \cup \mathcal {N} (j)} R _ {i m} \Theta_ {m j} = 0. \tag {31}
$$
The third case happens where  $i \notin \mathcal{V}_k$  and there exists a path from nodes  $i$  to  $j$ . Then we have  $\psi(i) \neq \psi(j)$  and the bridge path  $\mathcal{B}_{ij}$  is denoted as
$$
\mathcal {B} _ {i j} = \left\{\left(u _ {1}, u _ {2}\right), \left(u _ {3}, u _ {4}\right), \dots , \left(u _ {2 T - 1}, u _ {2 T}\right) \right\}, \tag {32}
$$
where  $\psi(u_1) = \psi(i)$  and  $\psi(u_{2T}) = \psi(j) = k$ . In particular, we emphasize that  $u_{2T} \in \mathcal{V}_k$  as  $(u_{2T-1}, u_{2T})$  is a bridge to enter the cluster that contains  $j$ . Therefore, we have
$$
F _ {i j} = \sum_ {m = 1} ^ {p} R _ {i m} \Theta_ {m j} \stackrel {(b)} {=} \frac {R _ {i , u _ {2 T}}}{S _ {u _ {2 T} , u _ {2 T}}} \cdot \sum_ {m = 1} ^ {p} R _ {u _ {2 T}, m} \Theta_ {m j} = 0, \tag {33}
$$
where the last equality is from (29) by using  $u_{2T} \in \mathcal{V}_k$ . In conclusion, via enumerating all the possible cases, we have  $\pmb{F} = \pmb{I}$ , which means that  $R\Theta = I$ .
Remark A.4. Theorem 3.4 generally holds for any  $T$ ,  $S$  and  $\{\widehat{\Theta}_k \in \mathbb{S}_{++}^{|\mathcal{V}_k|}\}_{k=1}^K$ . It does not utilize  $\mathrm{MTP}_2$  properties from  $\Theta$ . Hence, Theorem 3.4 itself may be extended to graphical models beyond  $\mathrm{MTP}_2$  assumption.
B Proof of Main Results
To prove Theorem 3.3, we require some preliminaries on  $\mathrm{MTP}_2$  properties.
Lemma B.1. [11, Theorem 1] A real symmetric matrix  $X \in \mathbb{S}^p$ , which admits the form:
$$
\forall i \neq j: \quad X _ {i j} \leq 0, \tag {34}
$$
is a non-singular  $M$ -matrix if and only if one of the following statements holds: (1) Every eigenvalue of  $\mathbf{X}$  is positive; (2)  $\mathbf{X}$  is inverse-positive. That is,  $\mathbf{X}^{-1}$  exists and  $\mathbf{X}^{-1} \geq \mathbf{0}$ .
A direct outcome of Lemma B.1 is that the matrix  $\pmb{R}$  is a non-negative matrix.
Corollary B.2. The matrix  $R$  given by (8) satisfies  $R \geq 0$ .
Corollary B.2 holds according to the fact that
$$
\forall k \in \{1, \dots , K \}: \quad \widehat {\boldsymbol {\Theta}} _ {k} \in \mathcal {M} ^ {p _ {k}} \quad \Rightarrow \quad \widehat {\boldsymbol {R}} _ {k} \geq \mathbf {0}. \tag {35}
$$
Now, we begin proving Theorem 3.3.
Proof of Theorem 3.3: First of all, we show that the matrix  $\Theta$ , given as
$$
\boldsymbol {\Theta} _ {i, j} = \left\{ \begin{array}{l l} {[ \widehat {\boldsymbol {\Theta}} _ {k} ] _ {\pi (i), \pi (i)} + \zeta_ {i}} & \text {i f} i = j \in \mathcal {V} _ {k}, \\ {[ \widehat {\boldsymbol {\Theta}} _ {k} ] _ {\pi (i), \pi (j)}} & \text {i f} i \neq j \text {a n d} i, j \in \mathcal {V} _ {k}, \\ - T _ {i j} / \left(S _ {i i} S _ {j j} - T _ {i j} ^ {2}\right) & \text {i f} (i, j) \in \mathcal {B}, \\ 0 & \text {o t h e r w i s e .} \end{array} \right. \tag {36}
$$
is primal feasible and positive definite. Based on Assumption 2.1, which states that  $S_{ij} < \sqrt{S_{ii}S_{jj}}$ , we know that  $T_{ij}^2 \leq (S_{ij} - \Lambda_{ij})^2 \leq S_{ij}^2 < S_{ii}S_{jj}$ . Hence, for any  $i \neq j$ , we have  $\Theta_{ij} \leq 0$  due to  $-T_{ij} / (S_{ii}S_{jj} - T_{ij}^2) \leq 0$  and the primal feasibility of each  $\widehat{\Theta}_k$ . Additionally, since we have  $R \geq 0$  according to Corollary B.2, we can conclude that  $\Theta$  is inverse-positive according to Theorem 3.4, which implies that it is a positive-definite matrix. In other words,  $\Theta$  is both primal feasible and inverse-positive, and we can express this as  $\Theta \succ 0$  based on Lemma B.1.
In second step, we show that  $\Theta = R^{-1}$  satisfies the optimality condition listed as follows, i.e.,
$$
- \left[ \Theta^ {- 1} \right] _ {i j} + S _ {i j} = 0 \quad \text {i f} i = j, \tag {37a}
$$
$$
- \left[ \Theta^ {- 1} \right] _ {i j} + S _ {i j} - \Lambda_ {i j} = 0 \quad \text {i f} i \neq j, \Theta_ {i j} \neq 0, \tag {37b}
$$
$$
- \left[ \boldsymbol {\Theta} ^ {- 1} \right] _ {i j} + S _ {i j} - \Lambda_ {i j} \leq 0 \quad \text {i f} i = j, \Theta_ {i j} = 0. \tag {37c}
$$
For (37a), we let  $k = \psi(i)$ . Then, according to (21a) in Lemma A.2, we have
$$
- \left[ \boldsymbol {\Theta} ^ {- 1} \right] _ {i j} + S _ {i j} = - \left[ \widehat {\boldsymbol {R}} _ {k} \right] _ {\pi (i), \pi (i)} + S _ {i j} = 0. \tag {38}
$$
For (37b), as  $\Theta_{ij} \neq 0$ , the pair  $(i,j)$  either satisfies  $(i,j) \in \mathcal{B}$  or  $\psi(i) = \psi(j) \stackrel{\Delta}{=} k$ . If  $(i,j) \in \mathcal{B}$ , then by using  $R_{ij} = T_{ij}$ , we have
$$
- \left[ \Theta^ {- 1} \right] _ {i j} + S _ {i j} - \Lambda_ {i j} = - R _ {i j} + T _ {i j} = 0. \tag {39}
$$
If nodes  $i$  and  $j$  are in the same cluster, then according to (21b) in Lemma A.2, we have  $\Theta_{ij} = [\widehat{\Theta}_k]_{\pi(i), \pi(j)} \neq 0$  such that
$$
- \left[ \boldsymbol {\Theta} ^ {- 1} \right] _ {i j} + S _ {i j} - \Lambda_ {i j} = - \left[ \widehat {\boldsymbol {R}} _ {k} \right] _ {\pi (i), \pi (j)} + S _ {i j} - \Lambda_ {i j} = 0. \tag {40}
$$
For (37c), the pair  $(i,j)$  either satisfies  $\psi(i) \neq \psi(j)$  or  $\psi(i) = \psi(j)$ . For first case, we have  $S_{ij} - \Lambda_{ij} \leq 0$  as  $(i,j) \notin \operatorname{supp}(\mathbf{T})$ . Consequently, we obtain
$$
- \left[ \Theta^ {- 1} \right] _ {i j} + S _ {i j} - \Lambda_ {i j} \leq - R _ {i j} \leq 0, \tag {41}
$$
as  $R \geq 0$ . For second case where  $\psi(i) = \psi(j)$ , we have  $\Theta_{ij} = [\widehat{\Theta}_k]_{\pi(i), \pi(j)} = 0$ . Then, applying (21c) in Lemma A.2, we have
$$
- \left[ \boldsymbol {\Theta} ^ {- 1} \right] _ {i j} + S _ {i j} - \Lambda_ {i j} = - \left[ \widehat {\boldsymbol {R}} _ {k} \right] _ {\pi (i), \pi (j)} + S _ {i j} - \Lambda_ {i j} \leq 0. \tag {42}
$$
Hence, together with the fact that  $\Theta$  is primal feasible and positive definite, we conclude that  $\Theta$  satisfies the KKT conditions of convex Problem (1). Therefore,  $\Theta$  is the optimal solution of Problem (1) and Theorem 3.3 is obtained.
C Extensions and Discussions
C.1 Extensions to traditional graphical lasso
The traditional graphical lasso is formulated as
$$
\underset {\boldsymbol {\Theta} \succ \mathbf {0}} {\text {m i n i m i z e}} - \log \det  (\boldsymbol {\Theta}) + \left\langle \boldsymbol {\Theta}, \boldsymbol {S} \right\rangle + \sum_ {i \neq j} \Lambda_ {i j} | \boldsymbol {\Theta} _ {i j} |. \tag {43}
$$
The KKT conditions of graphical lasso include:
$$
\begin{array}{l} - R _ {i j} + S _ {i j} = 0 \quad \text {i f} i = j, (44a) \\ - R _ {i j} + S _ {i j} - \Lambda_ {i j} \cdot \operatorname {s i g n} \left(\Theta_ {i j}\right) = 0 \quad \text {i f} i \neq j, \Theta_ {i j} \neq 0, (44b) \\ - \Lambda_ {i j} \leq - R _ {i j} + S _ {i j} - \leq \Lambda_ {i j} \quad \text {i f} i = j, \Theta_ {i j} = 0. (44c) \\ \end{array}
$$
In the realm of graphical lasso, the thresholded sample covariance matrix is usually defined as
$$
T _ {i j} = \left\{ \begin{array}{l l} 0 & \text {i f} | S _ {i j} | \leq \Lambda_ {i j}, \\ S _ {i j} - \Lambda_ {i j} \cdot \operatorname {s i g n} \left(S _ {i j}\right) & \text {i f} | S _ {i j} | > \Lambda_ {i j}. \end{array} \right. \tag {45}
$$
Our main results Theorem 3.3 can be generalized to graphical lasso with some modifications.
Theorem C.1. Suppose that  $\mathcal{P}$  is the bridge-block decomposition of the thresholded graph,  $\widehat{\Theta}_k$  is the optimal solution of  $k$ -th sub-problem, i.e.,
$$
\widehat {\boldsymbol {\Theta}} _ {k} = \arg \min  _ {\boldsymbol {\Theta} _ {k} \succ \mathbf {0}} - \log \det  \left(\boldsymbol {\Theta} _ {k}\right) + \left\langle \boldsymbol {\Theta} _ {k}, \boldsymbol {S} _ {\mathcal {V} _ {k}} \right\rangle + \sum_ {i \neq j} \left[ \boldsymbol {\Lambda} _ {\mathcal {V} _ {k}} \odot | \boldsymbol {\Theta} _ {k} | \right] _ {i j}. \tag {46}
$$
The decomposed form, which is given as
$$
\boldsymbol {\Theta} _ {i, j} ^ {\star} = \left\{ \begin{array}{l l} {[ \widehat {\boldsymbol {\Theta}} _ {k} ] _ {\pi (i), \pi (i)} + \zeta_ {i}} & \text {i f i = j \in \mathcal {V} _ {k} ,} \\ {[ \widehat {\boldsymbol {\Theta}} _ {k} ] _ {\pi (i), \pi (j)}} & \text {i f i \neq j a n d i , j \in \mathcal {V} _ {k} ,} \\ - T _ {i j} / \left(S _ {i i} S _ {j j} - T _ {i j} ^ {2}\right) & \text {i f (i , j) \in \mathcal {B} ,} \\ 0 & \text {o t h e r w i s e ,} \end{array} \right. \tag {47}
$$
in which  $\zeta_{i} = \frac{1}{S_{ii}}\sum_{(i,m)\in \mathcal{B}}\frac{T_{im}^{2}}{S_{ii}S_{m m} - T_{im}^{2}}$  and  $\zeta_{i} = 0$  if  $\forall m:(i,m)\notin \mathcal{B}$ , achieves optimality if  $\Theta \succ 0$  and the following inequality is satisfied:
$$
\left| - R _ {i j} + S _ {i j} \right| \leq \Lambda_ {i j}, \quad \forall i \neq j \text {a n d} i, j \text {b e l o n g t o d i f f e r e n c l u s t e r s}. \tag {48}
$$
It is important to note that the decomposed form (47) aligns with the structure of (7), with the key distinction lying in the computation of the thresholded matrix  $\pmb{T}$ . The proofs of Theorem C.1 can be conducted similar to Section B.
The condition (48) can be guaranteed under some circumstances. For example, we can set large values for  $\Lambda_{ij}$  whenever  $i \neq j$  and  $i, j$  are in different clusters. In this case, the thresholded graph is sparse and the condition (48) is satisfied. Nevertheless, in the majority of practical situations, verifying this condition poses a substantial challenge. Conversely, these conditions are inherently satisfied when  $\mathrm{MTP}_2$  constraints are applied.
C.2 Extensions to dense graphs
The proposed method operates under an implicit assumption of sparsity, which is a common characteristic of high-dimensional applications where the focus is on identifying only the most significant relationships among variables. However, it is possible to encounter scenarios where the thresholded graph is dense, making it difficult to identify any bridges for conducting bridge-block decomposition. In such cases, the proposed framework can still be useful, and this section will primarily focus on exploring its potential applications under these circumstances. We will discuss various strategies and techniques for adapting the proposed method to deal with dense graphs, enabling a broader applicability of the proposed bridge-block decomposition approach.
One of typical approaches to make the proposed method applicable is to tune the values of regularization coefficients. As the thresholded matrix is computed as
$$
\boldsymbol {T} = \max  (\mathbf {0}, \boldsymbol {S} - \boldsymbol {\Lambda}), \tag {49}
$$
we can always increase the value of  $\Lambda_{ij}$  to make the thresholded graph  $\operatorname{supp}(\mathbf{T})$  sufficiently sparse. When tuning the hyper-parameters is not recommended, a more general strategy is to use the solution of proposed framework as an initial point of numerical algorithms.
Warm-start algorithm: One of the main possible strengths of the proposed decomposed-form solution is that it can be treated as an initial point (warm-start) for the numerical algorithms specialized for solving Problem 1. To apply the decomposition approach, we first need to derive a suitable vertex partition of the graph based on the given data matrix  $S$  and regularization matrix  $\Lambda$ . There are several possible vertex partitioning techniques that can be used, including:
-  $k$ -edge-connected decomposition: This method divides a connected graph into its  $k$ -edge-connected sub-graphs, which can be useful for identifying bridges or bottlenecks in the graph structure.
-  $k$ -vertex-connected decomposition: This method decomposes a graph into its  $k$ -vertex-connected sub-graphs, which can be useful for identifying clusters or communities of vertices that are tightly connected to each other.
-  $k$ -core decomposition: This method identifies the maximal subgraphs of the graph in which each vertex has a degree of at least  $k$ , which can be useful for identifying the most densely connected regions of the graph.
- Community detection: This method identifies groups of vertices that are densely connected within the group and sparsely connected to vertices outside the group, which can be useful for identifying clusters or communities of vertices with similar properties or behavior.
Once we obtain a vertex partition  $\mathcal{P} = \{\mathcal{V}_1,\dots ,\mathcal{V}_K\}$ , where  $K$  refers to the number of clusters, we can compute the initial point using the following form: