SlowGuess/NeurIPS_2023_OCR huggingface dataset:
- Compute  $\hat{\Sigma}$ 's top eigenvector  $v$ .
- Calculate the max margin (i.e., maximum gap) of the one-dimensional projections of the data along the vectors  $\hat{\mu}, v$ . Let  $\hat{u}$  be the vector among these two with a larger margin.
return the vector  $\hat{u}$ .
B.2 Analysis
We demonstrate that Algorithm 2 operates within polynomial time and sample complexity. The details regarding sample complexity are presented in Theorem 2 (formal statement of Theorem 6). The time complexity is justified by the algorithm's process: it calculates the sample mean and the top eigenvector of the sample covariance matrix, both of which require polynomial time.
Theorem 6 (Sample Complexity for Isotropic Isoperimetric Distribution). Algorithm 2 with  $N = \tilde{O}(d^2\epsilon^{-6}\delta^{-2}\xi^{-1})$  samples learns the target isotropic isoperimetric distribution with  $\epsilon$ -margin to within TV distance  $\delta$  with probability  $1 - \xi$ .
The analytical approach is straightforward. Given that the component distributions are isotropic, the empirical mean will reveal the band if the removed band  $[a,b]$  stays on one side of the origin. Otherwise, when  $[a,b]$  spans across the origin, the variance along the component with the deleted band will increase. Consequently, this component emerges as the top principal component. Intriguing, this
property is "opposite" to the method used to identify low-dimensional convex concepts in [Vem10]. The latter relies on the Brascamp-Lieb inequality, where the variance of a restricted Gaussian is less than that of the original Gaussian.
To prove Theorem 6, we aim to quantify either the mean gap or the spectral gap (gap between the top two eigenvalues) of the uncentered covariance matrix. Specifically, Lemma 15 indicates that when  $0 \leq a < b$ , the mean along the direction  $u$  significantly deviates from zero. Meanwhile, Lemma 16 demonstrates that when  $a \leq 0 < b$ , there's a gap between the first and second eigenvalues of the uncentered covariance matrix. Subsequently, we employ Lemma 6 [SV13] to determine the sample complexity, and utilize the Davis-Kahan Theorem [DK70] (Lemma 7) to leverage the eigenvalue gap in identifying the pertinent vector  $u$ . We leave the proof of the lemmas in Section B.3.
For any  $x \in \mathbb{R}$ , we denote  $x_{i}$  as its  $i$ -th coordinate. We use  $\| x\|$  to denote its  $l_{2}$  norm. For a matrix  $A \in \mathbb{R}^{m\times n}$ , we denote its operator norm as  $\| A\|_{\mathrm{op}}$ . We denote the standard basis of  $\mathbb{R}^d$  by  $\{e_1,\dots ,e_d\}$ , and assume wlog that  $e_1 = u$  is the (unknown) normal vector to the band. Denote  $\Sigma = \mathbb{E}_{x\sim P}xx^{\top}$  as the uncentered covariance matrix of  $P$ , with eigenvalues  $\lambda_{1} \geq \lambda_{2} \geq \dots \geq \lambda_{d}$ .
Lemma 15 (Mean Gap). For  $0 \leq a < b$  and  $b - a \geq c\epsilon$  for constant  $c > 0$ , we have
$$
\underset {x \sim \hat {q}} {\mathbb {E}} x <   - \frac {\psi c ^ {2} \epsilon^ {3}}{2}, \operatorname {v a r} _ {x \sim \hat {q}} x \leq \frac {1}{2 \epsilon}.
$$
Lemma 16 (Spectral Gap of Covariance). If  $a \leq 0 < b$  and  $b > c\epsilon$  for constant  $c > 0$ , then the first and second eigenvalues of the uncentered covariance matrix  $\Sigma$  have the following gap
$$
\lambda_ {1} - \lambda_ {2} > C \epsilon^ {3} \lambda_ {1} f o r c o n s t a n t C > 0.
$$
Furthermore, the top eigenvector corresponds to  $u$ .
Now we are ready to prove Theorem 6.
Proof of Theorem 6. We can proceed with the assumption that  $|b| > |a|$ . If this condition is not met, we can redefine our interval by setting  $a' = -b$  and  $b' = -a$ . The proof can then be applied considering the distribution is restricted to  $\{x \in \mathbb{R}^d : u^\top x \leq a' \text{ or } u^\top x \geq b'\}$ . We will prove the theorem by considering two cases:  $0 \leq a < b$  and  $a \leq 0 < b$ .
We first consider the case when  $0 \leq a < b$ . Given that  $q_{1}$  is bounded by  $\tau$ , it follows that  $b - a > \epsilon / \tau$ . By Lemma 15, we know
$$
\underset {x \sim P} {\mathbb {E}} x _ {1} <   - \frac {\psi \epsilon^ {3}}{2 \tau^ {2}}, \underset {x \sim P} {\operatorname {v a r}} x _ {1} \leq \frac {1}{2 \epsilon},
$$
while for  $i\geq 2$  , we have
$$
\underset {x \sim P} {\mathbb {E}} x _ {i} = 0, \operatorname {v a r} _ {x \sim P} x _ {i} = 1.
$$
Given data  $x^{(1)}, \dots, x^{(N)}$ , let  $\hat{\mu} = \frac{1}{N}\sum_{j=1}^{N}x^{(j)}$  be the sample mean. Then by Chebyshev's Inequality,
$$
\mathbb {P} (\hat {\mu} _ {1} > - \frac {\psi \epsilon^ {3}}{4 \tau^ {2}}) \leq \frac {8 \tau^ {4}}{N \psi^ {2} \epsilon^ {7}}, \quad \mathbb {P} (\hat {\mu} _ {i} <   - \frac {\psi \epsilon^ {3} \delta}{4 \tau^ {2} \sqrt {d}}) \leq \frac {1 6 \tau^ {4} d}{N \psi^ {2} \epsilon^ {6} \delta^ {2}}, 2 \leq i \leq d
$$
Let  $0 < \xi < 1$ . So we know with sample size  $N_{1} = \frac{16\tau^{4}d^{2}}{\epsilon^{7}\delta^{2}\psi^{2}\xi}$ ,
$$
\mathbb {P} (\hat {\mu} _ {1} > - \frac {\psi \epsilon^ {3}}{4 \tau^ {2}}) \leq \frac {\psi^ {2} \delta^ {2} \xi}{2 d ^ {2}} <   \frac {\xi}{d}, \quad \mathbb {P} (\hat {\mu} _ {i} <   - \frac {\psi \epsilon^ {3} \delta}{4 \tau^ {2} \sqrt {d}}) \leq \frac {\epsilon \xi}{d} <   \frac {\xi}{d}
$$
$$
\begin{array}{l} \mathbb {P} (\sin \theta (\hat {\mu}, e _ {1}) \leq \delta) = \mathbb {P} \left(\frac {\hat {\mu} _ {1} ^ {2}}{\sum_ {i = 1} ^ {d} \hat {\mu} _ {i} ^ {2}} \geq 1 - \delta^ {2}\right) \\ \geq \mathbb {P} (\hat {\mu} _ {1} <   - \frac {\psi \epsilon^ {3}}{4 \tau^ {2}}, \hat {\mu} _ {i} > - \frac {\psi \epsilon^ {3} \delta}{4 \tau^ {2} \sqrt {d}}, 2 \leq i \leq d) \\ \geq 1 - \xi \\ \end{array}
$$
Secondly, we consider the case where  $a \leq 0 < b$ . Given that  $b - a > \epsilon / \tau$  and  $|b| > |a|$ , it results in  $b > \epsilon / (2\tau)$ . By Lemma 16, the top two eigenvalues of  $\Sigma$ , denoted as  $\lambda_1$  and  $\lambda_2$  satisfies
$$
\lambda_ {1} - \lambda_ {2} \geq C \epsilon^ {3} \lambda_ {1} \quad \text {f o r s o m e c o n s t a n t} C > 0
$$
By Lemma 6, with sample size  $N_{2} = \tilde{O}(d\epsilon_{1}^{-2})$ , with probability at least  $1 - \xi$
$$
\| \Sigma - \hat {\Sigma} \| _ {\mathrm {o p}} \leq \epsilon_ {1} \| \Sigma \| _ {\mathrm {o p}}
$$
By Lemma 7, we know for the top eigenvector  $v$  of  $\hat{\Sigma}$  satisfies
$$
\sin \theta (e _ {1}, v) \leq \frac {2 \| \Sigma - \hat {\Sigma} \| _ {\mathrm {o p}}}{C \epsilon^ {3} \lambda_ {1}} \leq \frac {2 \epsilon_ {1} \lambda_ {1}}{C \epsilon^ {3} \lambda_ {1}} = \frac {2 \epsilon_ {1}}{C \epsilon^ {3}}
$$
Choose  $\epsilon_{1} = C\epsilon^{3}\delta /2$ , and we will get  $\sin \theta (e_1,v)\leq \delta$ . The sample size we need is  $N_{2} = \tilde{O} (d\epsilon^{-6}\delta^{-2})$ . So with sample size  $N = \max (N_{1},N_{2}) = \tilde{O} (d^{2}\epsilon^{-6}\delta^{-2}\xi^{-1})$ , Algorithm 2 can recover  $e_1$  within TV distance  $\delta$  with probability  $1 - \xi$ .
B.3 Proofs
Proof. Since  $q_{1}$  is  $\psi$ -isoperimetric,  $\forall t \in [a,b]$ ,  $q_{1}(t) \geq \psi \int_{t}^{\infty} q_{1}(x) dx \geq \psi \epsilon$ . Then we have
$$
\int_ {a} ^ {b} x q _ {1} (x) d x > \psi \epsilon \int_ {a} ^ {b} x d x > \frac {c ^ {2} \psi \epsilon^ {3}}{2}
$$
By the definition of expectation, we have
$$
\mid \underset {x \sim \hat {q}} {\mathbb {E}} x \mid = \frac {\mid \int_ {\mathbb {R} \setminus [ a , b ]} x q _ {1} (x) d x \mid}{\int_ {\mathbb {R} \setminus [ a , b ]} q _ {1} (x) d x} = \frac {\mid \int_ {a} ^ {b} x q _ {1} (x) d x \mid}{\int_ {\mathbb {R} \setminus [ a , b ]} q _ {1} (x) d x} > \frac {c ^ {2} \epsilon^ {3}}{2}
$$
On the other hand, we can calculate the variance as follows.
$$
\operatorname * {v a r} _ {x \sim \hat {q}} x \leq \underset {x \sim \hat {q}} {\mathbb {E}} x ^ {2} = \frac {\int_ {\mathbb {R} \setminus [ a , b ]} x ^ {2} q _ {1} (x) d x}{\int_ {- \infty} ^ {a} q _ {1} (x) d x + \int_ {b} ^ {\infty} q _ {1} (x) d x} \leq \frac {\int_ {\mathbb {R}} x ^ {2} q _ {1} (x) d x}{2 \epsilon} = \frac {1}{2 \epsilon}
$$
Lemma 17 (Second Moment). For  $a, b$  satisfying  $a \leq 0 < b, b > c \epsilon$  for constant  $c > 0$ , we have
$$
\underset {x \sim \hat {q}} {\mathbb {E}} x ^ {2} > 1 + C \epsilon^ {2} \text {f o r c o n s t a n t} C > 0.
$$
Proof. By definition of  $\hat{q}$ , we know its density on the support  $x\in \mathbb{R}\backslash [a,b]$  is
$$
\hat {q} (x) = \frac {q _ {1} (x)}{\int_ {- \infty} ^ {a} q _ {1} (x) d x + \int_ {b} ^ {\infty} q _ {1} (x) d x}
$$
Then we calculate its second moment as follows.
$$
\underset {x \sim \hat {q}} {\mathbb {E}} x ^ {2} = \frac {\int_ {- a} ^ {\infty} x ^ {2} q _ {1} (x) d x + \int_ {b} ^ {\infty} x ^ {2} q _ {1} (x) d x}{\int_ {- a} ^ {\infty} q _ {1} (x) d x + \int_ {b} ^ {\infty} q _ {1} (x) d x}
$$
Define  $g(x) \coloneqq \int_{x}^{\infty}(t^{2} - 1)q_{1}(t)dt, x \geq 0$ . Its derivative is  $g'(x) = (1 - x^{2})q_{1}(x)$ . So we know  $g(x)$  is monotonically increasing when  $x \in [0,1]$ , and decreasing when  $x \geq 1$ . Since  $P_{1}$  is symmetric and isotropic, we know  $\int_{0}^{\infty} q_{1}(x) dx = \int_{0}^{\infty} x^{2} q_{1}(x) dx = 1/2$ . So we have  $g(0) = 0$ . This derives that  $g(x) \geq 0, \forall x \geq 0$ . In other words,  $\int_{-a}^{\infty} x^{2} q_{1}(x) dx \geq \int_{-a}^{\infty} q_{1}(x) dx$ .
For any  $x \in [c\epsilon, M]$ , we have  $g(x) \geq \min(g(c\epsilon), g(M))$ . Here we let  $M > 0$  such that  $\int_{M}^{\infty} p(x) \, dx = \epsilon$ . Then we can lower bound  $g(c\epsilon)$  as follows.
$$
g (c \epsilon) = g (0) + \int_ {0} ^ {c \epsilon} g ^ {\prime} (x) d x = \int_ {0} ^ {c \epsilon} (1 - x ^ {2}) q _ {1} (x) d x > c \epsilon (1 - c ^ {2} \epsilon^ {2}) q _ {1} (c \epsilon) > \psi c \epsilon^ {2} (1 - c ^ {2} \epsilon^ {2})
$$
If  $M \leq 1$ , we know  $|b| < M \leq 1$ . Then we have  $g(b) \geq g(c\epsilon)$ .
If  $M \geq 1 + \epsilon$ , we can lower bound  $g(M)$  as
$$
g (M) > (M ^ {2} - 1) \int_ {M} ^ {\infty} q _ {1} (x) d x > ((1 + \epsilon) ^ {2} - 1) \epsilon = \epsilon^ {3} + 2 \epsilon^ {2}
$$
Similarly we will get  $g(b) > \min(\psi c\epsilon^2 (1 - c^2\epsilon^2),\epsilon^3 +2\epsilon^2)$ . Finally if  $1 < M < 1 + \epsilon$ , there exists  $M' > 0$  such that  $\int_{M}^{M'}q_{1}(x)dx = \epsilon /2$ . Here  $M' - M > \epsilon /2$ . Then we have
$$
g (M ^ {\prime}) > \left(M ^ {\prime 2} - 1\right) \int_ {M ^ {\prime}} ^ {\infty} q _ {1} (x) d x > \left((1 + \epsilon / 2) ^ {2} - 1\right) \epsilon / 2 = \epsilon^ {3} / 8 + \epsilon^ {2} / 2
$$
In this case, we have  $g(b) > \min(g(\epsilon), g(M')) > \min(\psi c\epsilon^2(1 - c^2\epsilon^2), \epsilon^3/8 + \epsilon^2/2)$ . Therefore, we can lower bound the second moment of  $\hat{q}$  as follows.
$$
\begin{array}{l} \underset {x \sim \hat {q}} {\mathbb {E}} x ^ {2} > \frac {\int_ {- a} ^ {\infty} q _ {1} (x) d x + \int_ {b} ^ {\infty} q _ {1} (x) d x + g (b)}{\int_ {- a} ^ {\infty} q _ {1} (x) d x + \int_ {b} ^ {\infty} q _ {1} (x) d x} \\ > 1 + \frac {\min  (\psi c \epsilon^ {2} (1 - c ^ {2} \epsilon^ {2}) , \epsilon^ {3} / 8 + \epsilon^ {2} / 2)}{\int_ {- a} ^ {\infty} q _ {1} (x) d x + \int_ {b} ^ {\infty} q _ {1} (x) d x} \\ > 1 + \min  \left(\psi c \epsilon^ {2} \left(1 - c ^ {2} \epsilon^ {2}\right), \epsilon^ {3} / 8 + \epsilon^ {2} / 2\right) \\ > 1 + C \epsilon^ {3} \text {w h e r e} C = \min  (\psi c / 2, 1 / 8) \\ \end{array}
$$
Lemma 16 (Spectral Gap of Covariance). If  $a \leq 0 < b$  and  $b > c \epsilon$  for constant  $c > 0$ , then the first and second eigenvalues of the uncentered covariance matrix  $\Sigma$  have the following gap
Proof. We assume wlog that  $e_1 = u$ . Then the marginal distribution of  $P$  in  $e_1$  is  $\hat{q}$ , while the marginal distribution in  $e_i$  is  $q_i$  for any  $2 \leq i \leq d$ . Since  $q_i$  is isotropic, for any  $2 \leq i \leq d$ ,  $\mathbb{E} x_i^2 = 1$ . By Lemma 17, we have  $\mathbb{E} x_1^2 > 1 + C' \epsilon^3$  for constant  $C' > 0$ . Let  $g(v) := \mathbb{E} \frac{v^\top xx^\top v}{v^\top v}$ . Then we have  $g(e_1) > 1 + C' \epsilon^3$ , while  $g(e_i) = 1, \forall 2 \leq i \leq d$ . Then for any unit vector  $v = \sum_{i=1}^{d} \beta_i e_i$  satisfying  $\sum_{i=1}^{d} \beta_i^2 = 1$ , we have
$$
g (v) = \sum_ {i = 1} ^ {d} \beta_ {i} ^ {2} g \left(e _ {i}\right) \leq g \left(e _ {1}\right)
$$
Then we know the top eigenvalue of  $\Sigma$  is  $\lambda_1 > 1 + C\epsilon^3$ . Furthermore, the top eigenvector corresponds to  $e_1$ . Similarly, the second eigenvalue of  $\Sigma$  is  $\lambda_2 = \max_{v:v\perp e_1}g(v) = g(e_i) = 1, 2 \leq i \leq d$ . This implies that  $\lambda_1 - \lambda_2 > C\epsilon^3\lambda_1$  for constant  $C > 0$ .
C General Case: Affine Product Distribution with  $\epsilon$ -Margin
In this section, we first prove the main theorem in Section C.1 using the quantitative lemmas, and then prove all the lemmas. We begin with proving the two qualitative lemmas (Lemma 1 and Lemma 2) in Section C.2, and then prove the quantitative lemmas (Lemma 4 and Lemma 5) in the remaining section. For the quantitative part, we first consider the asymmetric case where  $|a + b| \geq \epsilon^5$ . In this case, contrastive mean leads to recovering  $u$ , as elaborated in Section C.3. Secondly, we consider the symmetric case characterized by  $a + b = 0$ , addressed in Section C.4. We show that we can recover  $u$  by calculating the top eigenvector of the re-weighted covariance matrix. Finally we extend the result to near-symmetric case where  $|a + b| < \epsilon^5$  in Section C.5.
Recall in Definition 1 that  $q$  is the symmetric one-dimensional isotropic logconcave density function, and  $\tilde{q}$  is the density obtained by restricting  $q$  to  $\mathbb{R} \backslash [a, b]$  for some unknown  $a < b$ . Denote  $\mu_1, \sigma_1^2$  as the mean and variance of  $\tilde{q}$ .  $\hat{q}$  is the density obtained after making  $\tilde{q}$  isotropic, with support  $\mathbb{R} \backslash [a', b']$ , where  $a' = \frac{a - \mu_1}{\sigma_1}$ ,  $b' = \frac{b - \mu_1}{\sigma_1}$ . The density  $\hat{q}$  on its support is
$$
\hat {q} (x) = \frac {\sigma_ {1} q (x \sigma_ {1} + \mu_ {1})}{\int_ {- \infty} ^ {a} q (x) d x + \int_ {b} ^ {\infty} q (x) d x}
$$
We denote the standard basis of  $\mathbb{R}^d$  by  $\{e_1, \dots, e_d\}$ , and assume wlog that  $e_1 = u$  is the (unknown) normal vector to the band. We write  $x_i := \langle x, e_i \rangle$  as  $x$ 's  $i$ -th coordinate. We assume in our proof that  $|b| > |a|$ . If this condition is not met, we can redefine our interval by setting  $a' = -b$  and  $b' = -a$ . The proof can then be applied considering the distribution is restricted to  $\{x \in \mathbb{R}^d : u^\top x \leq a' \text{ or } u^\top x \geq b'\}$ . For a vector  $x \in \mathbb{R}^d$ , we use  $\|x\|$  to denote its  $l_2$  norm. For a matrix  $A \in \mathbb{R}^{m \times n}$ , we denote its operator norm as  $\|A\|_{\mathrm{op}}$ .
Armed with two quantitative lemmas and the Davis-Kahan Theorem, we are now prepared to prove the main theorem. We state the formal version of Theorem 1 as follows.
Theorem 7 (Sample Complexity for Affine Product Distribution with  $\epsilon$ -Margin). Algorithm 1 with  $N = \tilde{O}(d^6 \epsilon^{-318} \delta^{-2} \xi^{-1})$  samples learns the target affine product distribution with  $\epsilon$ -margin to within TV distance  $\delta$  with probability  $1 - \xi$ .
Proof. Firstly we consider the case when  $|a + b| \geq \epsilon^5$ . Denote  $\alpha^* = \operatorname{argmax}_{\alpha} \{ |(\mu_{\alpha_1})_1|, |(\mu_{\alpha_2})_1| \}$  and  $\mu_{\alpha} = \mu_{\alpha^*}$ . By Lemma 4,  $|(\mu_{\alpha})_1| \geq C_1 \epsilon^{159} / d^2$ . Since for any negative  $\alpha$ , for any  $1 \leq i \leq d$ ,
$$
\operatorname {v a r} \left(\mu_ {\alpha}\right) _ {i} \leq \underset {y \sim P} {\mathbb {E}} e ^ {2 \alpha^ {*} \| y \| ^ {2}} y ^ {2} \leq \underset {y \sim P} {\mathbb {E}} y ^ {2} = 1
$$
By Chebyshev's Inequality, the re-weighted sample mean  $\tilde{\mu} = \frac{1}{N}\sum_{j=1}^{N}e^{\alpha^{*}\|y^{(j)}\|^{2}}y^{(j)}$  satisfies
$$
\mathbb {P} (| \tilde {\mu} _ {1} | \leq \frac {C _ {1} \epsilon^ {1 5 9}}{2 d ^ {2}}) \leq \frac {4 d ^ {4}}{N C _ {1} ^ {2} \epsilon^ {3 1 8}}, \quad \mathbb {P} (| \tilde {\mu} _ {i} | \geq \frac {C _ {1} \epsilon^ {1 5 9} \delta}{2 d ^ {2} \sqrt {d}}) \leq \frac {4 d ^ {5}}{N C _ {1} ^ {2} \epsilon^ {3 1 8} \delta^ {2}}, 2 \leq i \leq d.
$$
Let the sample size  $N_{1} = \frac{4d^{6}}{C_{1}^{2}\epsilon^{318}\delta^{2}\xi} = O(Cd^{6}\epsilon^{-318}\delta^{-2}\xi^{-1})$ , and we have
$$
\mathbb {P} (| \tilde {\mu} _ {1} | > \frac {C _ {1} \epsilon^ {1 5 9}}{2 d ^ {2}}) > 1 - \frac {\xi}{d}, \quad \mathbb {P} (| \tilde {\mu} _ {i} | <   \frac {C _ {1} \epsilon^ {1 5 9} \delta}{2 d ^ {2} \sqrt {d}}) > 1 - \frac {\xi}{d}, 2 \leq i \leq d.
$$
$$
\mathbb {P} (\sin \theta (\tilde {\mu}, e _ {1}) \leq \delta) = \mathbb {P} \left(\frac {\tilde {\mu} _ {1} ^ {2}}{\sum_ {i = 1} ^ {d} \tilde {\mu} _ {i} ^ {2}} \geq 1 - \delta^ {2}\right)
$$
$$
\begin{array}{l} \geq \mathbb {P} (| \tilde {\mu} _ {1} | \geq \frac {C _ {1} \epsilon^ {1 5 9}}{2 d ^ {2}}, | \tilde {\mu} _ {i} | \leq \frac {C _ {1} \epsilon^ {1 5 9} \delta}{2 d ^ {2} \sqrt {d}}, 2 \leq i \leq d) \\ \geq 1 - \xi \\ \end{array}
$$
This indicates that with probability  $1 - \xi$ , the re-weighted mean can output the vector  $\tilde{\mu}$  that is within angle  $\delta$  to the vector  $e_1$ .
Secondly, for the case when  $a$  and  $b$  are near-symmetric. Denote  $\Sigma$  as the re-weighted covariance matrix with eigenvalues  $\lambda_{i}$  and  $\tilde{\Sigma}$  as the empirical re-weighted covariance matrix with eigenvector  $v$ . By Lemma 5,  $\lambda_1 - \lambda_2 > C_2\epsilon^3\lambda_1$ . By Lemma 6, with sample size  $N_{2} = \tilde{O}(d\epsilon^{-6}\delta^{-2}\xi^{-1})$ , with probability  $1 - \xi$ ,
$$
\left\| \Sigma - \tilde {\Sigma} \right\| _ {\mathrm {o p}} \leq C _ {2} \epsilon^ {3} \delta \left\| \Sigma \right\| _ {\mathrm {o p}} / 2
$$
$$
\sin \theta (e _ {1}, v) \leq \frac {2 \| \Sigma - \tilde {\Sigma} \| _ {\mathrm {o p}}}{C _ {2} \epsilon^ {3} \lambda_ {1}} \leq \frac {C _ {2} \epsilon^ {3} \delta \lambda_ {1}}{C _ {2} \epsilon^ {3} \lambda_ {1}} = \delta
$$
So given  $N = \max (N_1,N_2) = \tilde{O} (d^6\epsilon^{318}\delta^{-2}\xi^{-1})$ , the algorithm learns the distribution  $P$  with probability  $1 - \xi$ .
C.2 Proofs of Qualitative Bounds
We present proofs of two qualitative lemmas: the contrastive mean (Lemma 1) and the contrastive covariance (Lemma 2). Their quantitative counterparts can be found in Section C.3 and Section C.5. To establish the contrastive mean, we invoke Descartes' Rule of Signs. For the proof concerning contrastive covariance, we introduce a novel monotonicity property on the moment ratio, as described in Lemma 3. We include the complete proof within this section.
Figure C.1: Coefficients of  $F(\alpha)$  ahead of  $e^{\alpha x^2}$  term in Lemma 1's proof. By combining  $e^{\alpha x^2}$  terms, we flip  $\hat{q}(x)$  horizontally. For  $a' > 0$ , the coefficient is negative when  $x \in (a', b')$  and non-negative outside the interval. For  $a' \leq 0$ , the it is negative when  $x \in (-a', b')$  and positive when  $x > b'$ .
Proof.  $|b| > |a|$  implies that  $\mu_1 < 0$ . For any  $x \geq 0$ , we have
$$
F (\alpha) = \underset {x \sim \hat {q}} {\mathbb {E}} e ^ {\alpha x _ {1} ^ {2}} x _ {1} \tag {C.1}
$$
By calculation, we have
$$
F (\alpha) = \int_ {\mathbb {R} \backslash [ a ^ {\prime}, b ^ {\prime} ]} e ^ {\alpha x ^ {2}} x \hat {q} (x) d x
$$
Then we rearrange  $F(\alpha)$  by combining  $e^{\alpha x^2}$  as in Figure C.1.
Similarly, we treat  $F(\alpha)$  as the integral of  $a(x)e^{\alpha x^2}$  for  $x \geq 0$ . For increasing  $x$ , the sign of  $a(x)$  changes twice. By Descartes' rule of signs,  $F(\alpha) = 0$  has at most two roots. In addition, we know  $F(0) = \frac{E}{P} x_1 = 0$  by definition of  $P$ . So  $\alpha = 0$  is one root of  $F(\alpha) = 0$ . So there is at most one nonzero root of  $F(\alpha) = 0$ . In other words, for any two distinct nonzero  $\alpha_1, \alpha_2$ , at least one of  $F(\alpha_1), F(\alpha_2)$  is nonzero. This implies that
Moment Ratio. To prove Lemma 2, we develop a new monotonicity property of the moment ratio of logconcave distributions. Moment ratio is specifically defined as the ratio of the fourth moment to the square of the second moment of truncated versions of the distribution. This measurement essentially reflects the uncentered kurtosis of the distribution. The formal definition is detailed in Definition 5.
Definition 4 (One-side  $t$ -restriction distribution). Let  $q$  be a distribution in one dimension with nonnegative support. For any  $t \geq 0$ , define  $q_{t}$  as the one-side  $t$ -restriction distribution on  $q$  obtained by restricting  $q$  to  $[t, \infty)$ .
Definition 5 (Moment Ratio). Let  $q$  be a distribution in one dimension with nonnegative support. For any  $t \geq 0$ , define  $q$ 's moment ratio as a function of  $t$ , given by
$$
\operatorname {m r} _ {q} (t) := \frac {\operatorname {v a r} _ {q _ {t}} \left(X ^ {2}\right)}{\left(\mathbb {E} _ {q _ {t}} X ^ {2}\right) ^ {2}}, \quad \text {w h e r e} q _ {t} \text {i s t h e o n e - s i d e} t \text {- r e s t r i c t i o n} q.
$$
We will prove the monotonicity of the moment ratio (Lemma 3) by reducing general logconcave distributions to exponential distributions. The monotonicity of the moment ratio for exponential distribution is detailed in Lemma 18.
Lemma 18 (Monotonicity of Moment Ratio of Exponential Distribution). Define  $h(x) = \beta e^{-\gamma x}$ ,  $x \geq 0, \beta, \gamma > 0$ . Denote  $N_{k}(t) = \int_{t}^{\infty} x^{k} h(x) \, dx$ . Then for any  $t \geq 0$ , we have
$$
t ^ {4} N _ {0} (t) N _ {2} (t) + N _ {2} (t) N _ {4} (t) - 2 t ^ {2} N _ {0} (t) N _ {4} (t) > 0.
$$
Proof. By calculation, we have
$$
N _ {0} (t) = \int_ {t} ^ {\infty} \beta e ^ {- \gamma x} d x = \frac {\beta}{\gamma} e ^ {- \gamma t}
$$
$$
N _ {1} (t) = \int_ {t} ^ {\infty} \beta x e ^ {- \gamma x} d x = \frac {\beta}{\gamma} t e ^ {- \gamma t} + \frac {1}{\gamma} N _ {0} (t) = \left(t + \frac {1}{\gamma}\right) \frac {\beta}{\gamma} e ^ {- \gamma t}
$$
$$
N _ {2} (t) = \int_ {t} ^ {\infty} \beta x ^ {2} e ^ {- \gamma x} d x = \frac {\beta}{\gamma} t ^ {2} e ^ {- \gamma t} + \frac {2}{\gamma} N _ {1} (t) = \left(t ^ {2} + \frac {2}{\gamma} t + \frac {2}{\gamma^ {2}}\right) \frac {\beta}{\gamma} e ^ {- \gamma t}
$$
$$
N _ {3} (t) = \int_ {t} ^ {\infty} \beta x ^ {3} e ^ {- \gamma x} d x = \frac {\beta}{\gamma} t ^ {3} e ^ {- \gamma t} + \frac {3}{\gamma} N _ {2} (t) = \left(t ^ {3} + \frac {3}{\gamma} t ^ {2} + \frac {6}{\gamma^ {2}} t + \frac {6}{\gamma^ {3}}\right) \frac {\beta}{\gamma} e ^ {- \gamma t}
$$
$$
N _ {4} (t) = \int_ {t} ^ {\infty} \beta x ^ {4} e ^ {- \gamma x} d x = \frac {\beta}{\gamma} t ^ {4} e ^ {- \gamma t} + \frac {4}{\gamma} N _ {3} (t) = \left(t ^ {4} + \frac {4}{\gamma} t ^ {3} + \frac {1 2}{\gamma^ {2}} t ^ {2} + \frac {2 4}{\gamma^ {3}} t + \frac {2 4}{\gamma^ {4}}\right) \frac {\beta}{\gamma} e ^ {- \gamma t}
$$
Then we can plug them and get
$$
\begin{array}{l} t ^ {4} N _ {0} (t) N _ {2} (t) + N _ {2} (t) N _ {4} (t) - 2 t ^ {2} N _ {0} (t) N _ {4} (t) \\ = \frac {\beta^ {2}}{\gamma^ {2}} e ^ {- 2 \gamma t} \left(t ^ {4} \left(t ^ {2} + \frac {2}{\gamma} t + \frac {2}{\gamma^ {2}}\right) + \left(- t ^ {2} + \frac {2}{\gamma} t + \frac {2}{\gamma^ {2}}\right) \cdot \left(t ^ {4} + \frac {4}{\gamma} t ^ {3} + \frac {1 2}{\gamma^ {2}} t ^ {2} + \frac {2 4}{\gamma^ {3}} t + \frac {2 4}{\gamma^ {4}}\right)\right) \\ = \frac {8 \beta^ {2}}{\gamma^ {2}} e ^ {- 2 \gamma t} \left(\frac {t ^ {3}}{\gamma^ {3}} + \frac {6 t ^ {2}}{\gamma^ {4}} + \frac {1 2 t}{\gamma^ {5}} + \frac {6}{\gamma^ {6}}\right) > 0 \\ \end{array}
$$
Next, we will prove the monotonicity of moment ratio for logconcave distributions.
Proof. Denote  $M_{k}(t) = \int_{t}^{\infty}x^{k}q(x)dx$ . By Definition 5,