SlowGuess/NeurIPS_2023_OCR huggingface dataset:
In conclusion, we have shown that, with probability at least  $1 - 2|\mathcal{S}_r|^2\exp \left(-\frac{m\alpha^2\Lambda_{\min}(\boldsymbol{H}_{\mathcal{S}_r\mathcal{S}_r})}{1152(1 - \alpha)^2|\mathcal{S}_r|^3}\right) - 2|\mathcal{S}_r|^2\exp \left(-\frac{m(\Lambda_{\min}(\boldsymbol{H}_{\mathcal{S}_r\mathcal{S}_r}))^2}{8|\mathcal{S}_r|^2}\right) - 2|\mathcal{S}_r^c ||\mathcal{S}_r| \exp \left(-\frac{m\alpha^2\Lambda_{\min}(\boldsymbol{H}_{\mathcal{S}_r\mathcal{S}_r})}{288\rho_{\max}^2|\mathcal{S}_r|^3}\right) - 2|\mathcal{S}_r^c ||\mathcal{S}_r| \exp \left(-\frac{m\alpha}{48\rho_{\max}^2|\mathcal{S}_r|^2}\right) - 2|\mathcal{S}_r|^2\exp \left(-\frac{m(\Lambda_{\min}(\boldsymbol{H}_{\mathcal{S}_r\mathcal{S}_r}))^2}{192|\mathcal{S}_r|^3}\right) - 2|\mathcal{S}_r|^2\exp \left(-\frac{m(\Lambda_{\min}(\boldsymbol{H}_{\mathcal{S}_r\mathcal{S}_r}))^2}{8|\mathcal{S}_r|^2}\right)$  and
$$
\begin{array}{l} \varepsilon \leqslant \min  (\frac {\alpha}{4 8 (1 - \alpha) | \mathcal {S} _ {r} |} \sqrt {\frac {\Lambda_ {\operatorname* {m i n}} (\boldsymbol {H} _ {\mathcal {S} _ {r} \mathcal {S} _ {r}})}{| \mathcal {S} _ {r} |}}, \frac {\Lambda_ {\operatorname* {m i n}} (\boldsymbol {H} _ {\mathcal {S} _ {r} \mathcal {S} _ {r}})}{1 6 | \mathcal {S} _ {r} | ^ {\frac {1}{2}}}, \frac {\alpha}{2 4 \rho_ {\operatorname* {m a x}} | \mathcal {S} _ {r} |} \sqrt {\frac {\Lambda_ {\operatorname* {m i n}} (\boldsymbol {H} _ {\mathcal {S} _ {r} \mathcal {S} _ {r}})}{| \mathcal {S} _ {r} |}}, \\ \frac {1}{4 \rho_ {\max } | \mathcal {S} _ {r} |} \sqrt {\frac {\alpha}{6}}, \frac {\Lambda_ {\min } (\boldsymbol {H} _ {\mathcal {S} _ {r} \mathcal {S} _ {r}})}{8 | \mathcal {S} _ {r} |} \sqrt {\frac {\alpha}{6 | \mathcal {S} _ {r} |}}, \frac {\Lambda_ {\min } (\boldsymbol {H} _ {\mathcal {S} _ {r} \mathcal {S} _ {r}})}{1 6 | \mathcal {S} _ {r} | ^ {\frac {1}{2}}}), \\ \end{array}
$$
the mutual incoherence condition holds for any worst-case distributions:
$$
\left\| \boldsymbol {H} _ {\mathcal {S} _ {r} ^ {\mathrm {c}} \mathcal {S} _ {r}} ^ {\mathbb {Q}} \left(\boldsymbol {H} _ {\mathcal {S} _ {r} \mathcal {S} _ {r}} ^ {\mathbb {Q}}\right) ^ {- 1} \right\| _ {B, 1, \infty} \leqslant 1 - \frac {\alpha}{2}.
$$
Simplifying the above expressions, with probability at least 1 -  $\mathcal{O}(\exp \left(-\frac{Cm}{\rho_{\max}^2|\mathcal{S}_r|^3} + \log |\mathcal{S}_r^c| + \log |\mathcal{S}_r|\right))$  and  $\varepsilon \leqslant \frac{C}{\rho_{\max}|\mathcal{S}_r|^{3/2}}$ ,
$$
\left\| \boldsymbol {H} _ {\mathcal {S} _ {r} ^ {c} \mathcal {S} _ {r}} ^ {\mathbb {Q}} \left(\boldsymbol {H} _ {\mathcal {S} _ {r} \mathcal {S} _ {r}} ^ {\mathbb {Q}}\right) ^ {- 1} \right\| _ {B, 1, \infty} \leqslant 1 - \frac {\alpha}{2},
$$
where  $C$  only depends on  $\alpha$ ,  $\Lambda_{\min}(H_{S_r S_r})$ .
Lemma 12. If Assumption 1 holds, then for any  $\mathbb{Q} \in \mathcal{A}_{\varepsilon}^{W_p}(\tilde{\mathbb{P}}_m)$  and  $\alpha \in (0,1]$ , with probability at least  $1 - |\mathcal{S}_r| \rho_r \exp \left(-\frac{m \mu^2}{2 \sigma^2}\right)$ ,  $\varepsilon \leqslant \frac{\mu}{\sigma}$  and  $\lambda_B^* > \frac{32 \mu \sqrt{\rho_r(1 - \alpha / 2)}}{\alpha}$ , we have
$$
\| \mathbb {E} _ {\mathbb {Q}} \mathcal {E} \left(\boldsymbol {X} _ {\bar {r}}\right) _ {\mathcal {S}, r} \boldsymbol {e} ^ {\intercal} \| _ {2, \infty} \leqslant \frac {\lambda_ {B} ^ {*} \alpha}{8 (1 - \alpha / 2)}.
$$
With probability at least  $1 - |\mathbf{C}\pmb{o}_r|\rho_r\exp \left(-\frac{m\mu^2}{2\sigma^2}\right)$ ,  $\varepsilon \leqslant \frac{\mu}{\sigma}$  and  $\lambda_B^* >\frac{32\mu\sqrt{\rho_{max}\rho_r}}{\alpha}$ , we have
$$
\left\| \mathbb {E} _ {\mathbb {Q}} \mathcal {E} \left(\boldsymbol {X} _ {\bar {r}}\right) \mathcal {S} _ {r} ^ {c} \boldsymbol {e} ^ {\intercal} \right\| _ {B, 2, \infty} \leqslant \frac {\lambda_ {B} ^ {*} \alpha}{8}.
$$
Proof. We start with  $\| \mathbb{E}_{\mathbb{Q}}\mathcal{E}(\pmb{X}_{\tilde{r}})_{\mathcal{S}_r}\pmb{e}^{\top}\|_{2,\infty}$ . After some algebraic manipulation, we find that
$$
\begin{array}{l} \| \mathbb {E} _ {\mathbb {Q}} \mathcal {E} \left(\boldsymbol {X} _ {\bar {r}}\right) _ {\mathcal {S} _ {r}} \boldsymbol {e} ^ {\intercal} \| _ {2, \infty} \leqslant \max  _ {i \in \mathcal {S} _ {r}} \| \mathbb {E} _ {\mathbb {Q}} \mathcal {E} \left(\boldsymbol {X} _ {\bar {r}}\right) _ {i} \boldsymbol {e} \| _ {2} \\ \leqslant \max  _ {i \in \mathcal {S} _ {r}} \sqrt {\rho_ {r}} \max  _ {j \in \rho_ {r}} | \mathbb {E} _ {\mathbb {Q}} \mathcal {E} (\boldsymbol {X} _ {\bar {r}}) _ {i} e _ {j} | \\ \leqslant \max  _ {i \in \mathcal {S} _ {r}} \sqrt {\rho_ {r}} \max  _ {j \in \rho_ {r}} \mathbb {E} _ {\mathbb {Q}} | \mathcal {E} (\boldsymbol {X} _ {\bar {r}}) _ {i} e _ {j} | \\ \leqslant \max  _ {i \in \mathcal {S} _ {r}} \sqrt {\rho_ {r}} \max  _ {j \in \rho_ {r}} \mathbb {E} _ {\mathbb {Q}} | e _ {j} |. \\ \end{array}
$$
Since  $|e_j|$  is a bounded random variable according to Assumption 1, we apply Hoeffding's inequality to get
$$
\operatorname {P r o b} \left(\mathbb {E} _ {\tilde {\mathbb {P}} _ {m}} \left| e _ {j} \right| \geqslant \mu + t\right) \leqslant \exp \left(- \frac {m t ^ {2}}{2 \sigma^ {2}}\right).
$$
Base on a similar argument as Equation (9), we can derive
$$
\mathbb {E} _ {\mathbb {Q}} \left| e _ {j} \right| - \mathbb {E} _ {\tilde {\mathbb {P}} _ {m}} \left| e _ {j} \right| \leqslant 2 \varepsilon \sigma ,
$$
$$
\operatorname {P r o b} \left(\mathbb {E} _ {\mathbb {Q}} \left| e _ {j} \right| \geqslant 2 \varepsilon \sigma + \mu + t\right) \leqslant \exp \left(- \frac {m t ^ {2}}{2 \sigma^ {2}}\right).
$$
Taking the union bound over all  $i \in S_r$  and  $j \in \rho_r$ , we find that
$$
\operatorname {P r o b} (\| \mathbb {E} _ {\mathbb {Q}} \mathcal {E} (\boldsymbol {X} _ {\bar {r}}) _ {\mathcal {S} _ {r}} \boldsymbol {e} ^ {\intercal} \| _ {2, \infty} \geqslant \sqrt {\rho_ {r}} (2 \varepsilon \sigma + \mu + t)) \leqslant | \mathcal {S} _ {r} | \rho_ {r} \exp (- \frac {m t ^ {2}}{2 \sigma^ {2}}).
$$
Setting  $t = \mu$  and  $\varepsilon \leqslant \frac{\mu}{\sigma}$  while requiring  $\lambda_B^* > \frac{32\mu\sqrt{\rho_r(1 - \alpha / 2)}}{\alpha}$ . With probability at least  $1 - |\mathcal{S}_r| \rho_r \exp \left(-\frac{m \mu^2}{2 \sigma^2}\right)$ , we have
$$
\left\| \mathbb {E} _ {\mathbb {Q}} \mathcal {E} \left(\boldsymbol {X} _ {\bar {r}}\right) _ {\mathcal {S} _ {r}} \boldsymbol {e} ^ {\intercal} \right\| _ {2, \infty} \leqslant \frac {\lambda_ {B} ^ {*} \alpha}{8 (1 - \alpha / 2)}. \tag {15}
$$
Then we consider  $\| \mathbb{E}_{\mathbb{Q}}\mathcal{E}(\boldsymbol{X}_{\bar{r}})_{S_{\hat{c}}^{\prime}}e^{\mathsf{T}}\|_{B,2,\infty}$ :
$$
\begin{array}{l} \| \mathbb {E} _ {\mathbb {Q}} \mathcal {E} \left(\boldsymbol {X} _ {\bar {r}}\right) _ {\mathcal {S} _ {r} ^ {c}} \boldsymbol {e} ^ {\intercal} \| _ {B, 2, \infty} \leqslant \max  _ {i \in \mathbf {C o} _ {r}} \| \mathbb {E} _ {\mathbb {Q}} \mathcal {E} \left(X _ {i}\right) \boldsymbol {e} ^ {\intercal} \| _ {2, 2} \\ \leqslant \max  _ {i \in \mathbf {C o} _ {r}} \sqrt {\rho_ {i} \rho_ {r}} \max  _ {j \in \rho_ {i}, k \in \rho_ {r}} | \mathbb {E} _ {\mathbb {Q}} \mathcal {E} (X _ {i}) _ {j} e _ {k} | \\ \leqslant \max  _ {i \in \mathbf {C o} _ {r}} \sqrt {\rho_ {i} \rho_ {r}} \max  _ {k \in \rho_ {r}} \mathbb {E} _ {\mathbb {Q}} | e _ {k} |. \\ \end{array}
$$
Similarly, applying Hoeffding's inequality and the Kantorovich-Rubinstein theorem gives us
$$
\operatorname {P r o b} \left(\| \mathbb {E} _ {\mathbb {Q}} \mathcal {E} \left(\boldsymbol {X} _ {\tilde {r}}\right) _ {\mathcal {S} _ {r} ^ {c}} \boldsymbol {e} ^ {\intercal} \| _ {B, 2, \infty} \geqslant \sqrt {\rho_ {\max} \rho_ {r}} (2 \varepsilon \sigma + \mu + t)\right) \leqslant | \mathbf {C o} _ {r} | \rho_ {r} \exp \left(- \frac {m t ^ {2}}{2 \sigma^ {2}}\right).
$$
Let  $t = \mu$ ,  $\varepsilon \leqslant \frac{\mu}{\sigma}$  and  $\lambda_B^* > \frac{32\mu\sqrt{\rho_{\max}\rho_r}}{\alpha}$  hold, we have, with probability at least  $1 - |\mathbf{C}\mathbf{o}_r| \rho_r \exp \left(-\frac{m\mu^2}{2\sigma^2}\right)$ ,
$$
\| \mathbb {E} _ {\mathbb {Q}} \mathcal {E} \left(\boldsymbol {X} _ {\bar {r}}\right) _ {\mathcal {S} _ {r} ^ {c}} \boldsymbol {e} ^ {\intercal} \| _ {B, 2, \infty} \leqslant \frac {\lambda_ {B} ^ {*} \alpha}{8}.
$$
Theorem 9. Given a Bayesian network  $(\mathcal{G},\mathbb{P})$  of  $n$  categorical random variables and its skeleton  $\mathcal{G}_{skel} := (\mathcal{V},\mathcal{E}_{skel})$ . Assume that the condition  $\| \pmb{W}^{*}\|_{B,2,1} \leqslant \bar{B}$  holds for some  $\bar{B} > 0$  associated with an optimal Lagrange multiplier  $\lambda_B^* > 0$  for  $\pmb{W}^{*}$  defined in Equation (1). Suppose that  $\hat{\pmb{W}}$  is a DRO risk minimizer of Equation (4) with a Wasserstein distance of order 1 and an ambiguity radius
$\varepsilon = \varepsilon_0 / m$  where  $m$  is the number of samples drawn i.i.d. from  $\mathbb{P}$ . Under Assumptions 1, 2, 3, 4, if the number of samples satisfies
where  $C$  only depends on  $\alpha, \Lambda$ , and if the Lagrange multiplier satisfies
(a) The optimal estimator  $\hat{W}$  is unique.
(b) All the non-neighbor nodes are excluded:  $\mathbf{C}\mathbf{o}_r\subseteq \hat{\mathbf{C}}\mathbf{o}_r$
(c) All the neighbor nodes are identified:  $\mathbf{N}\mathbf{e}_r\subseteq \hat{\mathbf{N}}\mathbf{e}_r$
(d) The true skeleton is successfully reconstructed:  $\mathcal{G}_{skel} = \hat{\mathcal{G}}_{skel}$ .
Proof. We prove the statements in this theorem in several steps. In order to prove (a) and (b), we will show that the DRO problem is strictly convex if true non-neighbors are known so that there is an optimal solution. Next we would like to demonstrate that this solution with a non-neighbor constraint is indeed unique for all the solutions without constraints. The proof for uniqueness comes with a conclusion that we do not accidentally include any edge between the current node and its non-neighbors. Next, to prove (c), we present a generalization bound for the DRO estimator in terms of its true risk, which leads to a  $\ell_{\infty}$  bound of the difference between the estimator  $\hat{W}$  and the true weight matrix  $W^{*}$ . Combined with the assumption on the minimum weight, it implies that we include all the neighbor nodes successfully. Finally, by taking a union bound for all the nodes, we could conclude that the correct skeleton is recovered with high probability, which proves (d).
(i) Given the true non-neighbors, there is a unique solution.
We start with the Wasserstein DRO problem, which we recapitulate here for convenience:
$$
\hat {\boldsymbol {W}} \in \arg \inf  _ {\boldsymbol {W}} \sup  _ {\mathbb {Q} \in \mathcal {A} _ {\varepsilon} ^ {W _ {p}} (\tilde {\mathbb {P}} _ {m})} \frac {1}{2} \mathbb {E} _ {\mathbb {Q}} \| \mathcal {E} (X _ {r}) - \boldsymbol {W} ^ {\intercal} \mathcal {E} (\boldsymbol {X} _ {\bar {r}}) \| _ {2} ^ {2}.
$$
The objective is convex because it is a supremum of convex functions.
For now, we assume that the non-neighbor nodes  $\mathbf{C}\mathbf{o}_r$  are given. We can then explicitly restrict  $\pmb{W}_{i} = \mathbf{0}$  for all  $i\in \mathbf{C}\mathbf{o}_r$ . The Hessian of  $\pmb{W}_{S_r}$  is a block diagonal matrix reads
$$
\nabla^ {2} R ^ {\mathbb {Q}} (\boldsymbol {W} _ {\mathcal {S} _ {r} \cdot}) = \left[ \begin{array}{c c c c} \boldsymbol {H} _ {\mathcal {S} _ {r} \mathcal {S} _ {r}} ^ {\mathbb {Q}} & \boldsymbol {0} & \dots & \boldsymbol {0} \\ \boldsymbol {0} & \boldsymbol {H} _ {\mathcal {S} _ {r} \mathcal {S} _ {r}} ^ {\mathbb {Q}} & \dots & \boldsymbol {0} \\ \vdots & \vdots & \ddots & \vdots \\ \boldsymbol {0} & \boldsymbol {0} & \dots & \boldsymbol {H} _ {\mathcal {S} _ {r} \mathcal {S} _ {r}} ^ {\mathbb {Q}} \end{array} \right] \in \mathbb {R} ^ {\rho_ {r} \rho_ {\mathbf {N e} _ {r}} \times \rho_ {r} \rho_ {\mathbf {N e} _ {r}}},
$$
$$
\boldsymbol {H} ^ {\mathbb {Q}} := \mathbb {E} _ {\mathbb {Q}} \left[ \boldsymbol {\mathcal {E}} \left(\boldsymbol {X} _ {\bar {r}}\right) \boldsymbol {\mathcal {E}} \left(\boldsymbol {X} _ {\bar {r}}\right) ^ {\intercal} \right] \in \mathbb {R} ^ {\rho_ {\bar {r}} \times \rho_ {\bar {r}}}
$$
is the covariance matrix of encodings of  $X_{\bar{r}}$  under some distribution  $\mathbb{Q}\in \mathcal{A}_{\varepsilon}^{W_p}(\tilde{\mathbb{P}}_m)$
Since  $\mathbf{W}_{\mathcal{S}_r^c}$  is fixed to be zero and  $\nabla^2 R^\mathbb{Q}(\mathbf{W}_{\mathcal{S}_r})$  is a block diagonal matrix, we focus on showing that  $H_{\mathcal{S}_r\mathcal{S}_r}^\mathbb{Q} > 0$ .
We apply Lemma 7 to get the bound
$$
\Lambda_ {\min } \left(\boldsymbol {H} _ {\mathcal {S} _ {r} \mathcal {S} _ {r}} ^ {\mathbb {Q}}\right) \geqslant \Lambda_ {\min } \left(\boldsymbol {H} _ {\mathcal {S} _ {r} \mathcal {S} _ {r}}\right) - 4 \varepsilon | \mathcal {S} _ {r} | ^ {\frac {1}{2}} - t,
$$
with probability at least  $1 - 2|\mathcal{S}_r|^2\exp \left(-\frac{mt^2}{2|\mathcal{S}_r|^2}\right)$ .  $\Lambda_{\min}(\pmb {H}_{\mathcal{S}_r\mathcal{S}_r}) - 4\varepsilon |\mathcal{S}_r|^{\frac{1}{2}} - t > 0$  will guarantee that the DRO problem in Equation (4) has a unique solution when the  $\pmb {W}_i = \pmb{0}$  is satisfied for non-neighbor nodes.
(ii) Given the true non-neighbors, the solution is optimal.
We would like to show that the solution to Equation (4) with true non-neighbor constraints is optimal. In this way, we do not recover any non-neighbor nodes in the skeleton. We adopt the primal-dual witness (PDW) [Wainwright, 2009] method to show optimality for the constrained unique solution.
Recall that we assume  $\| \pmb {W}\|_{B,2,1}\leqslant \bar{B}$ . To begin with, we write the dual problem as
$$
\hat {\boldsymbol {W}} \in \arg \inf  _ {\boldsymbol {W}} \sup  _ {\mathbb {Q} \in \mathcal {A} _ {\varepsilon} ^ {W _ {p}} (\tilde {\mathbb {P}} _ {m}), \| \boldsymbol {Z} \| _ {B, 2, \infty} \leqslant 1, \lambda_ {B} \geqslant 0} \frac {1}{2} \mathbb {E} _ {\mathbb {Q}} \| \mathcal {E} \left(\boldsymbol {X} _ {r}\right) - \boldsymbol {W} ^ {\intercal} \mathcal {E} \left(\boldsymbol {X} _ {\bar {r}}\right) \| _ {2} ^ {2} + \lambda_ {B} \left(\langle \boldsymbol {Z}, \boldsymbol {W} \rangle - \bar {B}\right) \tag {16}
$$
s.t.  $\forall i\in \mathbf{C}\mathbf{o}_r\quad \pmb {W}_i = \mathbf{0},$
where  $\lambda_B$  is the Lagrange multiplier for the norm constraint on  $\mathbf{W}$ .
$\tilde{W}$  is optimal if and only if there exists  $(\mathbb{Q}^{*},Z^{*},\lambda_{B}^{*})$  that satisfies the KKT condition:
$$
\mathbb {E} _ {\mathbb {Q} ^ {*}} \mathcal {E} \left(\boldsymbol {X} _ {\bar {r}}\right) \mathcal {E} \left(\boldsymbol {X} _ {\bar {r}}\right) ^ {\intercal} \tilde {\boldsymbol {W}} - \mathbb {E} _ {\mathbb {Q} ^ {*}} \mathcal {E} \left(\boldsymbol {X} _ {\bar {r}}\right) \mathcal {E} \left(\boldsymbol {X} _ {r}\right) ^ {\intercal} + \lambda_ {B} ^ {*} \boldsymbol {Z} ^ {*} = \boldsymbol {0}
$$
$$
\mathbb {Q} ^ {*} \in \mathcal {A} _ {\varepsilon} ^ {W _ {p}} (\tilde {\mathbb {P}} _ {m}), \| Z ^ {*} \| _ {B, 2, \infty} \leqslant 1, \lambda_ {B} ^ {*} \geqslant 0, \| \hat {\boldsymbol {W}} \| _ {B, 2, 1} \leqslant \bar {B}
$$
$$
\langle \boldsymbol {Z} ^ {*}, \hat {\boldsymbol {W}} \rangle = \| \hat {\boldsymbol {W}} \| _ {B, 2, 1}, \lambda_ {B} ^ {*} (\| \hat {\boldsymbol {W}} \| _ {B, 2, 1} - \bar {B}) = 0.
$$
Note that we assume that the constraint  $\| \pmb{W}\|_{B,2,1} \leqslant \bar{B}$  is active such that  $\lambda_B^* > 0$ . This assumption is only for convenience of theoretical analysis and not restrictive. If it is not active, we have  $\| \hat{\pmb{W}}\|_{B,2,1} = \check{B} < \bar{B}$  for some  $\check{B}$  and  $\lambda_B^* = 0$ , which leads to an unconstrained problem similar to the ordinary least square problem, which is known to suffer from overfitting. Instead, we are usually interested in solutions that have finite norms so we can always find  $\bar{B} = \check{B} - \epsilon < \check{B}$  for some small positive constant  $\epsilon > 0$  to make the constraint active and thus  $\lambda_B^* > 0$ .
Substituting  $\mathcal{E}(X_r) = \mathbf{W}^{*\top}\mathcal{E}(\mathbf{X}_{\hat{r}}) + \mathbf{e}$  into the first-order optimality condition yields
$$
\begin{array}{l} \mathbb {E} _ {\mathbb {Q} ^ {*}} \mathcal {E} \left(\boldsymbol {X} _ {\bar {r}}\right) \mathcal {E} \left(\boldsymbol {X} _ {\bar {r}}\right) ^ {\top} (\hat {\boldsymbol {W}} - \boldsymbol {W} ^ {*}) - \mathbb {E} _ {\mathbb {Q} ^ {*}} \mathcal {E} \left(\boldsymbol {X} _ {\bar {r}}\right) e ^ {\top} + \lambda_ {B} ^ {*} \boldsymbol {Z} ^ {*} = \boldsymbol {0} \\ \Longleftrightarrow \left[ \begin{array}{c c} \boldsymbol {H} _ {\mathcal {S} _ {r} \mathcal {S} _ {r}} ^ {\mathbb {Q} ^ {*}} & \boldsymbol {H} _ {\mathcal {S} _ {r} \mathcal {S} _ {r} ^ {c}} ^ {\mathbb {Q} ^ {*}} \\ \boldsymbol {H} _ {\mathcal {S} _ {r} ^ {c} \mathcal {S} _ {r}} ^ {\mathbb {Q} ^ {*}} & \boldsymbol {H} _ {\mathcal {S} _ {r} ^ {c} \mathcal {S} _ {r} ^ {c}} ^ {\mathbb {Q} ^ {*}} \end{array} \right] \left[ \begin{array}{c} \hat {\boldsymbol {W}} _ {\mathcal {S} _ {r}} - \boldsymbol {W} _ {\mathcal {S} _ {r}} ^ {*} \\ \boldsymbol {0} \end{array} \right] - \left[ \begin{array}{c} \mathbb {E} _ {\mathbb {Q} ^ {*}} \mathcal {E} (\boldsymbol {X} _ {\bar {r}}) _ {\mathcal {S} _ {r}} \boldsymbol {e} ^ {\intercal} \\ \mathbb {E} _ {\mathbb {Q} ^ {*}} \mathcal {E} (\boldsymbol {X} _ {\bar {r}}) _ {\mathcal {S} _ {r} ^ {c}} \boldsymbol {e} ^ {\intercal} \end{array} \right] + \lambda_ {B} ^ {*} \left[ \begin{array}{c} \boldsymbol {Z} _ {\mathcal {S} _ {r}} ^ {*} \\ \boldsymbol {Z} _ {\mathcal {S} _ {r} ^ {c}} ^ {*} \end{array} \right] = \left[ \begin{array}{c} \boldsymbol {0} \\ \boldsymbol {0} \end{array} \right]. \tag {17} \\ \end{array}
$$
Solving for  $Z_{\mathcal{S}_{\varepsilon}^{\bullet}}^{*}$ , we find that
$$
\lambda_ {B} ^ {*} \boldsymbol {Z} _ {\mathcal {S} _ {r} ^ {c}} ^ {*} = \lambda_ {B} ^ {*} \boldsymbol {H} _ {\mathcal {S} _ {r} ^ {c} \mathcal {S} _ {r}} ^ {\mathbb {Q} ^ {*}} \left(\boldsymbol {H} _ {\mathcal {S} _ {r} \mathcal {S} _ {r}} ^ {\mathbb {Q} ^ {*}}\right) ^ {- 1} \boldsymbol {Z} _ {\mathcal {S} _ {r}} ^ {*} - \boldsymbol {H} _ {\mathcal {S} _ {r} ^ {c} \mathcal {S} _ {r}} ^ {\mathbb {Q} ^ {*}} \left(\boldsymbol {H} _ {\mathcal {S} _ {r} \mathcal {S} _ {r}} ^ {\mathbb {Q} ^ {*}}\right) ^ {- 1} \mathbb {E} _ {\mathbb {Q} ^ {*}} \boldsymbol {\mathcal {E}} \left(\boldsymbol {X} _ {\bar {r}}\right) _ {\mathcal {S} _ {r}} \boldsymbol {e} ^ {\intercal} + \mathbb {E} _ {\mathbb {Q} ^ {*}} \boldsymbol {\mathcal {E}} \left(\boldsymbol {X} _ {\bar {r}}\right) _ {\mathcal {S} _ {r} ^ {c}} \boldsymbol {e} ^ {\intercal},
$$
which can be bounded such that
$$
\begin{array}{l} \lambda_ {B} ^ {*} \| \mathbf {Z} _ {\mathcal {S} _ {\mathcal {E}}} ^ {*} \| _ {B, 2, \infty} \\ = \| \lambda_ {B} ^ {*} H _ {\mathcal {S} _ {r} ^ {c} \mathcal {S} _ {r}} ^ {\mathbb {Q} ^ {*}} \left(H _ {\mathcal {S} _ {r} \mathcal {S} _ {r}} ^ {\mathbb {Q} ^ {*}}\right) ^ {- 1} Z _ {\mathcal {S} _ {r}.} ^ {*} - H _ {\mathcal {S} _ {r} ^ {c} \mathcal {S} _ {r}} ^ {\mathbb {Q} ^ {*}} \left(H _ {\mathcal {S} _ {r} \mathcal {S} _ {r}} ^ {\mathbb {Q} ^ {*}}\right) ^ {- 1} \mathbb {E} _ {\mathbb {Q} ^ {*}} \mathcal {E} \left(X _ {\bar {r}}\right) _ {\mathcal {S} _ {r}} e ^ {\intercal} + \mathbb {E} _ {\mathbb {Q} ^ {*}} \mathcal {E} \left(X _ {\bar {r}}\right) _ {\mathcal {S} _ {r} ^ {c}} e ^ {\intercal} \| _ {B, 2, \infty} \\ \leqslant \lambda_ {B} ^ {*} \| H _ {\mathcal {S} _ {r} ^ {c} \mathcal {S} _ {r}} ^ {\mathbb {Q} ^ {*}} \left(H _ {\mathcal {S} _ {r} \mathcal {S} _ {r}} ^ {\mathbb {Q} ^ {*}}\right) ^ {- 1} Z _ {\mathcal {S} _ {r}} ^ {*} \| _ {B, 2, \infty} + \| H _ {\mathcal {S} _ {r} ^ {c} \mathcal {S} _ {r}} ^ {\mathbb {Q} ^ {*}} \left(H _ {\mathcal {S} _ {r} \mathcal {S} _ {r}} ^ {\mathbb {Q} ^ {*}}\right) ^ {- 1} \mathbb {E} _ {\mathbb {Q} ^ {*}} \mathcal {E} \left(\boldsymbol {X} _ {\bar {r}}\right) _ {\mathcal {S} _ {r}} e ^ {\intercal} \| _ {B, 2, \infty} + \| \mathbb {E} _ {\mathbb {Q} ^ {*}} \mathcal {E} \left(\boldsymbol {X} _ {\bar {r}}\right) _ {\mathcal {S} _ {r} ^ {c}} e ^ {\intercal} \| _ {B, 2, \infty} \\ \leqslant \lambda_ {B} ^ {*} \| H _ {\mathcal {S} _ {r} ^ {c} \mathcal {S} _ {r}} ^ {\mathbb {Q} ^ {*}} \left(H _ {\mathcal {S} _ {r} \mathcal {S} _ {r}} ^ {\mathbb {Q} ^ {*}}\right) ^ {- 1} \| _ {B, 1, \infty} \| Z _ {\mathcal {S} _ {r}} ^ {*} \cdot \| _ {2, \infty} + \| H _ {\mathcal {S} _ {r} ^ {c} \mathcal {S} _ {r}} ^ {\mathbb {Q} ^ {*}} \left(H _ {\mathcal {S} _ {r} \mathcal {S} _ {r}} ^ {\mathbb {Q} ^ {*}}\right) ^ {- 1} \| _ {B, 1, \infty} \| \mathbb {E} _ {\mathbb {Q} ^ {*}} \mathcal {E} \left(\boldsymbol {X} _ {\bar {r}}\right) _ {\mathcal {S} _ {r}} e ^ {\intercal} \| _ {2, \infty} \\ + \left\| \mathbb {E} _ {\mathbb {Q} ^ {*}} \mathcal {E} \left(\boldsymbol {X} _ {\tilde {r}}\right) \mathcal {S} _ {\mathcal {C}} ^ {*} \boldsymbol {e} ^ {\intercal} \right\| _ {B, 2, \infty}. \\ \end{array}
$$
$$
\left\| \boldsymbol {Z} _ {\mathcal {S} _ {r}} ^ {*} \right\| _ {2, \infty} \leqslant \left\| \boldsymbol {Z} ^ {*} \right\| _ {B, 2, \infty} \leqslant 1.
$$
Recall that  $0 < \alpha \leqslant 1$  in Assumption 4. Based on Lemma 8 and Lemma 12, we may write
$$
\begin{array}{l} \lambda_ {B} ^ {*} \| \mathbf {Z} _ {\mathcal {S} _ {\xi}} ^ {*} \| _ {B, 2, \infty} \\ \leqslant \lambda_ {B} ^ {*} \| H _ {\mathcal {S} _ {r} ^ {c} \mathcal {S} _ {r}} ^ {\mathbb {Q} ^ {*}} \left(H _ {\mathcal {S} _ {r} \mathcal {S} _ {r}} ^ {\mathbb {Q} ^ {*}}\right) ^ {- 1} \| _ {B, 1, \infty} \| Z _ {\mathcal {S} _ {r}} ^ {*}, \| _ {2, \infty} + \| H _ {\mathcal {S} _ {r} ^ {c} \mathcal {S} _ {r}} ^ {\mathbb {Q} ^ {*}} \left(H _ {\mathcal {S} _ {r} \mathcal {S} _ {r}} ^ {\mathbb {Q} ^ {*}}\right) ^ {- 1} \| _ {B, 1, \infty} \| \mathbb {E} _ {\mathbb {Q} ^ {*}} \mathcal {E} \left(\boldsymbol {X} _ {\bar {r}}\right) _ {\mathcal {S} _ {r}} e ^ {\intercal} \| _ {2, \infty} \\ + \left\| \mathbb {E} _ {\mathbb {Q} ^ {*}} \mathcal {E} \left(\boldsymbol {X} _ {\bar {r}}\right) _ {\mathcal {S} _ {\bar {r}}} \boldsymbol {e} ^ {\top} \right\| _ {B, 2, \infty} \\ \leqslant \lambda_ {B} ^ {*} (1 - \frac {\alpha}{2}) + (1 - \frac {\alpha}{2}) (\frac {\lambda_ {B} ^ {*} \alpha}{8 (1 - \alpha / 2)}) + \frac {\lambda_ {B} ^ {*} \alpha}{8} \\ \leqslant \lambda_ {B} ^ {*} (1 - \frac {\alpha}{4}) \\ <   \lambda_ {B} ^ {*}, \\ \end{array}
$$
with high probability and certain conditions on  $\lambda_B^*$  and  $\varepsilon$ .
Henceforth,  $\| Z_{\mathcal{S}_r^c}^*\|_{B,2,\infty} < 1$  satisfies strict dual feasibility and we must have  $\| \hat{\boldsymbol{W}}_{\mathcal{S}_r^c}\|_{B,2,1} = 0$  according to complementary slackness:  $\langle \pmb {Z}^{*},\hat{\pmb{W}}\rangle = \| \hat{\pmb{W}}\|_{B,2,1}$ . In other words, we have
$$
\forall i \in \mathbf {C o} _ {r} \quad \hat {\boldsymbol {W}} _ {i} = \mathbf {0},
$$
with high probability. This guarantees that we do not recover any node that is not a neighbor of  $r$  with high probability.
(iii) Without information about the true skeleton, we have a unique and optimal solution.
We follow the proof of Lemma 11.2 in Hastie et al. [2015].
We have shown that  $\hat{\pmb{W}}$  satisfying  $\hat{\pmb{W}}_i = \mathbf{0}$ $\forall i\in \mathbf{C o}_r$  is an optimal solution with optimal dual variables  $\| Z_{\mathcal{S}_r^c}^*\|_{B,2,\infty} < 1$ .
To avoid clutter of notations, we define
$$
L ^ {\mathrm {D R O}} (\boldsymbol {W}) := \sup  _ {\mathbb {Q} \in \mathcal {A} _ {\varepsilon} ^ {W _ {p}} (\tilde {\mathbb {P}} _ {m})} \frac {1}{2} \mathbb {E} _ {\mathbb {Q}} \| \mathcal {E} (X _ {r}) - \boldsymbol {W} ^ {\intercal} \mathcal {E} (\boldsymbol {X} _ {\bar {r}}) \| _ {2} ^ {2}.
$$
Let  $(\bar{\mathbf{W}},\check{\lambda})$  be any other optimal solution to  $\inf_{\mathbf{W}}\sup_{\lambda}L^{\mathrm{DRO}}(\mathbf{W}) + \lambda (\| \mathbf{W}\|_{B,2,1} - \bar{B})$ . By definition,