SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$$
\nabla_ {x} \ln q _ {t} (x) = \sum_ {i = 1} ^ {K} w _ {i, t} ^ {*} (x) \mu_ {i, t} ^ {*} - x, \quad \text {w h e r e} \quad w _ {i, t} ^ {*} (x) = \frac {\exp (- \| x - \mu_ {i , t} ^ {*} \| ^ {2} / 2)}{\sum_ {j = 1} ^ {K} \exp (- \| x - \mu_ {j , t} ^ {*} \| ^ {2} / 2)}.
$$
For a mixture of two Gaussians, the score function simplifies to
$$
\nabla_ {x} \log q _ {t} (x) = \tanh (\mu_ {t} ^ {* \top} x) \mu_ {t} ^ {*} - x, \quad w h e r e \quad \mu_ {t} ^ {*} \triangleq \mu^ {*} \exp (- t)
$$
See Appendix A for the calculation.
Recall that  $\nabla_{x}\log q_{t}(x)$  is the minimizer for the score-matching objective given in Eq. (4). Therefore, we parametrize our student network architecture similarly to the optimal score function. Our student architecture for mixtures of  $K$  Gaussians is
$$
s _ {\theta_ {t}} (x) = \sum_ {i = 1} ^ {K} w _ {i, t} (x) \mu_ {i, t} - x, \quad \text {w h e r e} \quad w _ {i, t} (x) \triangleq \frac {\exp \left(- \| x - \mu_ {i , t} \| ^ {2} / 2\right)}{\sum_ {j = 1} ^ {K} \exp \left(- \| x - \mu_ {j , t} \| ^ {2} / 2\right)} \tag {9}
$$
$$
\mu_ {i, t} \triangleq \mu_ {i} \exp (- t).
$$
where  $\theta_{t} = \{\mu_{1,t},\mu_{2,t},\dots ,\mu_{K,t}\}$  denotes the set of parameters at the noise scale  $t$  . For mixtures of two Gaussians, we simplify the student architecture as follows:
$$
s _ {\theta_ {t}} (x) = \tanh  (\mu_ {t} ^ {\top} x) \mu_ {t} - x, \quad \text {w h e r e} \quad \mu_ {t} \triangleq \mu \exp (- t).
$$
As  $\theta_t$  only depends on  $\mu_t$  in the case of mixtures of two Gaussians, we simplify the notation of the score function from  $s_{\theta_t}(x)$  to  $s_{\mu_t}(x)$  in that case. We use  $\hat{\mu}_t$  and  $\hat{\mu}_t^*$  to denote the unit vector along the direction of  $\mu_t$  and  $\mu_t^*$  respectively. Note that we often use  $\mu_t$  (or  $\theta_t$ ) to denote the current iterate of gradient descent on the DDPM objective and  $\mu_t'$  to denote the iterate after taking a gradient descent step from  $\mu_t$ .
Expectation-Maximization (EM) algorithm. The EM algorithm is composed of two steps: the E-step and the M-step. For mixtures of Gaussians, the E-step computes the expected log-likelihood based on the current mean parameters and the M-step maximizes this expectation to find a new estimate of the parameters.
Fact 5 (See e.g., [DTZ17, YYS17, KC20] for more details). When  $X$  is the mixture of  $K$  Gaussian and  $\{\mu_1, \mu_2, \ldots, \mu_K\}$  are current estimates of the means, the population EM update for all  $i \in \{1, 2, \ldots, K\}$  is given by
$$
\mu_ {i} ^ {\prime} = \frac {\mathbb {E} _ {X} [ w _ {i} (X) X ]}{\mathbb {E} _ {X} [ w _ {i} (X) ]}, \text {w h e r e} w _ {i} (X) = \frac {\exp (- \| X - \mu_ {i} \| ^ {2} / 2)}{\sum_ {j = 1} ^ {K} \exp (- \| X - \mu_ {j} \| ^ {2} / 2)}.
$$
The EM update for mixtures of two Gaussians given in Eq. (7) simplifies to
$$
\mu^ {\prime} = \mathbb {E} _ {X \sim \mathcal {N} (\mu^ {*}, \operatorname {I d})} [ \tanh  (\mu^ {\top} X) X ].
$$
An analogous version of the EM algorithm, called the gradient EM algorithm, takes a gradient step in the direction of the M-step instead of optimizing the objective in the M-step fully.
Fact 6 (See e.g., [YYS17, SN21] for more details). For all  $i \in \{1, 2, \dots, K\}$ , the gradient EM-update for mixtures of  $K$  Gaussian is given by
$$
\mu_ {i} ^ {\prime} = \mu_ {i} + \eta \mathbb {E} _ {X} [ w _ {i} (X) (X - \mu_ {i}) ],
$$
where  $\eta$  is the learning rate.
2 Warmup: mixtures of two Gaussians with constant separation
In this section, we formally state our result for learning mixtures of two Gaussians with constant separation. This case highlights the main proof techniques, namely viewing gradient descent on the DDPM objective as power iteration and as the EM algorithm.
2.1 Result and algorithm
Theorem 7. There is an absolute constant  $c > 0$  such that the following holds. Suppose a mixture of two Gaussians with the mean parameter  $\mu^{*}$  satisfies  $\| \mu^{*}\| >c$ . Then, for any  $\varepsilon >0$ , there is a procedure that calls Algorithm 1 at two different noise scales  $t$  and outputs  $\tilde{\mu}$  such that  $\| \tilde{\mu} -\mu^{*}\| \leq \varepsilon$  with high probability. Moreover, the algorithm has time and sample complexity  $\mathrm{poly}(d) / \varepsilon^2$  (see Theorem C.1 for more precise quantitative bounds).
Algorithm. The algorithm has two stages. In the first stage we run gradient descent on the DDPM objective described in Algorithm 1 from a random Gaussian initialization and noise scale  $t_1$  for a fixed number of iterations  $H$  where  $t_1 = \Theta(\log d)$  ("high noise") and  $H = \mathrm{poly}(d, 1 / \varepsilon)$ . In the second stage, the procedure uses the output of the first step as initialization and runs Algorithm 1 at a "low noise" scale of  $t_2 = \Theta(1)$ .
2.2 Proof outline of Theorem 7
We provide a proof sketch of correctness of the above algorithm and summarize the main technical lemmas here. All proofs of the following lemmas can be found in Appendix C.
Part I: Analysis of high noise regime and connection to power iteration. We show that in the large noise regime, the negative gradient  $-\nabla L_{t}(s_{t})$  is well-approximated by  $2\mu_t^*\mu_t^{*\top}\mu_t - 3\| \mu_t\| ^2\mu_t$ . Recall that this result is the key to showing the resemblance between gradient descent and power iteration. Concretely, we show the following lemma:
Lemma 8 (See Lemma C.3 for more details). For  $t = \Theta (\log d)$ , the gradient descent update on the DDPM objective  $L_{t}(s_{t})$  can be approximated with  $2\mu_t^*\mu_t^{*\top}\mu_t - 3\| \mu_t\| ^2\mu_t$ :
$$
\left\| \left(- \nabla L _ {t} \left(s _ {t}\right)\right) - \left(2 \mu_ {t} ^ {*} \mu_ {t} ^ {* \top} \mu_ {t} - 3 \| \mu_ {t} \| ^ {2} \mu_ {t}\right) \right\| \leq \operatorname {p o l y} (1 / d).
$$
From Lemma 8, it immediately follows that  $\mu^{\prime}t$ , the result of taking a single gradient step starting from  $\mu_t$ , is well-approximated by the result of taking a single step of power iteration for a matrix whose leading eigenvector is  $\mu_t^*$ :
$$
\mu_ {t} ^ {\prime} = \mu_ {t} - \eta \nabla L _ {t} (s _ {\mu}) \approx (\mathrm {I d} (1 - 3 \eta \| \mu_ {t} \| ^ {2}) + 2 \mu_ {t} ^ {*} \mu_ {t} ^ {* \top}) \mu_ {t}.
$$
The second key element is to show that as a consequence of the above power iteration update, the gradient descent converges in angular distance to the leading eigenvector. Concretely, we show the following lemma:
Lemma 9 (Informal, see Lemma C.5 for more details). Suppose  $\mu_t'$  is the iterate after one step of gradient descent on the DDPM objective from  $\mu_t$ . Denote the angle between  $\mu_t$  and  $\mu_t^*$  to be  $\theta$  and between  $\mu_t'$  and  $\mu_t^*$  to be  $\theta'$ . In this case, we show that
$$
\tan \theta^ {\prime} = \max \left(\kappa_ {1} \tan \theta , \kappa_ {2}\right),
$$
where  $\kappa_{1} < 1$  and  $\kappa_{2} \leq 1 / \mathrm{poly}(d)$ .
Note  $\tan \theta^{\prime} < \tan \theta$  implies that  $\theta^{\prime} < \theta$  or equivalently  $\langle \hat{\mu}_t^\prime ,\hat{\mu}_t^*\rangle >\langle \hat{\mu}_t,\hat{\mu}_t^*\rangle$ . Thus, the above lemma shows that by taking a gradient step in the DDPM objective, the angle between  $\mu_t$  and  $\mu_t^*$  decreases. By iterating this, we obtain the following lemma:
Lemma 10 (Informal, see Lemma C.6 for more details). Running gradient descent from a random initialization on the DDPM objective  $L_{t}(s_{\mu})$  for  $t = O(\log d)$  gives  $\mu_t$  for which  $\langle \hat{\mu}_t, \hat{\mu}_t^* \rangle$  is  $\Omega(1)$ .
Note that we cannot keep running gradient descent at this high noise scale and hope to achieve  $\mu$  such that  $\| \mu -\mu^{*}\|$  is  $O(\varepsilon)$ . This is because Lemma 9 can only guarantee that the angle between  $\mu_{t}$  and  $\mu_t^*$  is  $O(\varepsilon)$ , but this does not imply  $\| \mu -\mu^{*}\|$  is  $O(\varepsilon)$ . Instead, as described in Part II, we will proceed with a smaller noise scale.
Part II: Analysis of low noise regime and connection to EM. In the low noise regime, we run Algorithm 1 using the output from Part I as our initialization. Our analysis here shows that whenever the initialization  $\mu_t$  satisfies the condition of  $\langle \hat{\mu}_t, \hat{\mu}_t^* \rangle$  being  $\Omega(1), \| \mu_t - \mu_t^* \|$  contracts after every gradient step. To start with, we show that the result of a population gradient step on the DDPM objective  $L_t(s_\mu)$  results in the following:
$$
\mu_ {t} ^ {\prime} = (1 - \eta) \mu_ {t} + \eta \mathbb {E} _ {x \sim \mathcal {N} (\mu_ {t} ^ {*}, \mathrm {I d})} [ \tanh (\mu_ {t} ^ {\top} x) x ] + \eta G (\mu_ {t}, \mu_ {t} ^ {*}),
$$
where  $\mu_t^\prime$  is the parameter after a gradient step,  $\eta$  is the learning rate, and function  $G$  is given by
$$
G (\mu , \mu^ {*}) = \mathbb {E} _ {x \sim \mathcal {N} (\mu^ {*}, \mathrm {I d})} [ - \frac {1}{2} \tanh ^ {\prime \prime} (\mu^ {\top} x) \| \mu \| ^ {2} x + \tanh ^ {\prime} (\mu^ {\top} x) \mu^ {\top} x x - \tanh ^ {\prime} (\mu^ {\top} x) \mu ].
$$
Note we use the population gradient here only for simplicity; in the Appendix we show that empirical estimates of the gradient suffice. After some calculation, we can show that
$$
\left\| \mu_ {t} ^ {\prime} - \mu_ {t} ^ {*} \right\| \leq (1 - \eta) \| \mu_ {t} - \mu_ {t} ^ {*} \| + \eta \| \mathbb {E} _ {x \sim \mathcal {N} \left(\mu_ {t} ^ {*}, \mathrm {I d}\right)} [ \tanh  \left(\mu_ {t} ^ {\top} x\right) x ] - \mu_ {t} ^ {*} \| + \eta \| G (\mu_ {t}, \mu_ {t} ^ {*}) \| . \tag {10}
$$
Using Fact 5, we know that  $\mathbb{E}_{x\sim \mathcal{N}(\mu_t^*,\mathrm{Id})}[\tanh (\mu_t^\top x)x]$  is precisely the result of one step of EM starting from  $\mu_{t}$ , and it is known [DTZ17] that the EM update contracts the distance between  $\mu_{t}$  and  $\mu_t^*$  as follows:
$$
\left\| \mathbb {E} _ {x \sim \mathcal {N} \left(\mu_ {t} ^ {*}, \mathrm {I d}\right)} \left[ \tanh  \left(\mu_ {t} ^ {\top} x\right) x \right] - \mu_ {t} ^ {*} \right\| \leq \lambda_ {1} \| \mu_ {t} - \mu_ {t} ^ {*} \| \quad \text {f o r s o m e} \lambda_ {1} <   1 \tag {11}
$$
It remains to control the second term in Eq. (10), for which we prove the following:
Lemma 11 (Informal, see Lemma C.9 for more details). When  $\| \mu^{*}\| = \Omega (1)$  and the noise scale  $t = \Theta (1)$ , then for every  $\mu$  with  $\langle \hat{\mu},\hat{\mu}^*\rangle$  being  $\Omega (1)$ , the following inequality holds:
$$
\| G (\mu_ {t}, \mu_ {t} ^ {*}) \| \leq \lambda_ {2} \| \mu_ {t} - \mu_ {t} ^ {*} \| \quad f o r s o m e \lambda_ {2} <   1.
$$
Combining Eq. (11) and Lemma 11 with Eq. (10), we have
$$
\left\| \mu_ {t} ^ {\prime} - \mu_ {t} ^ {*} \right\| \leq (1 - \eta \left(1 - \lambda_ {1} - \lambda_ {2}\right)) \| \mu_ {t} - \mu_ {t} ^ {*} \| \tag {12}
$$
We can set parameters to ensure that  $\lambda_1 + \lambda_2 < 1$  and therefore that  $\| \mu_t - \mu_t^*\|$  contracts with each gradient step. Applying Lemma 11 and Eq. (12), we obtain the following lemma summarizing the behavior of gradient descent on the DDPM objective in the low noise regime.
Lemma 12 (Informal). For any  $\varepsilon >0$  and for the noise scale  $t = \Theta (1)$ , starting from an initialization  $\mu_t$  for which  $\langle \hat{\mu}_t,\hat{\mu}_t^*\rangle = \Omega (1)$ , running gradient descent on the DDPM objective,  $L_{t}(s_{\mu})$  will give us mean parameter  $\tilde{\mu}$  such that  $\| \tilde{\mu} -\mu^{*}\| \leq O(\varepsilon)$ .
Combining Lemma 10 and Lemma 12, we obtain our first main result, Theorem 7, for learning mixtures of two Gaussians with constant separation. For the full technical details, see Appendix C.
3 Extensions: small separation and more components
3.1 Mixtures of two Gaussians with small separation
In this section, we briefly sketch how the ideas from Section 2 can be extended to give our second main result, namely on learning mixtures of two Gaussians even with small separation. We defer the full technical details to Appendix D.
Theorem 13. Suppose a mixture of two Gaussians has mean parameter  $\mu^{*}$  that satisfies  $\| \mu^{*}\| = \Omega (\frac{1}{\mathrm{poly}(d)})$ . Then, for any  $\varepsilon >0$ , there exists a modification of Algorithm 1 that provides an estimate  $\mu$  such that  $\| \mu -\mu^{*}\| \leq O(\varepsilon)$  with high probability. Moreover, the algorithm has time and sample complexity  $\mathrm{poly}(d) / \varepsilon^8$  (see Theorem D.1 for more precise quantitative bounds).
Algorithm modification. The algorithm that we analyze runs projected gradient descent on the DDPM objective but only in the high noise scale regime where  $t = O(\log d)$ . At each step, we project the iterate  $\mu$  to the ball of radius  $R$ , where  $R$  is an empirical estimate for  $\| \mu^{*} \|$  obtained by drawing samples  $x_{1}, \ldots, x_{n}$  from the data distribution and forming  $R \triangleq (\frac{1}{n} \sum_{i=1}^{n} \| x_{i} \|^{2} - d)^{1/2}$ .
Proof sketch. Lemma 9 and Lemma 10 apply even when the components of the mixture have small separation, and they show that running gradient descent on the DDPM objective results in  $\mu_t$  and  $\mu_t^*$  being  $O(1)$  close in angular distance. Although our analysis can be extended to show that gradient descent can achieve  $O(\varepsilon)$  angular distance, this does not guarantee that  $\| \mu_t - \mu_t^*\|$  is  $O(\varepsilon)$ . If in addition to being  $O(\varepsilon)$  close in angular distance, we also have that  $\| \mu_t\| \approx \| \mu_t^*\|$ , then it is easy to see that  $\| \mu_t - \mu_t^*\|$  is indeed  $O(\varepsilon)$ .
Observe that if  $R$  is approximately equal to  $\| \mu_t^* \|$ , then the projection step in our algorithm ensures that our final estimate  $\mu_t$  satisfies this additional condition of  $\| \mu_t \| \approx \| \mu_t^* \|$ . It is not hard to show that  $R^2$  is an unbiased estimate of  $\| \mu_t^* \|^2$ , so standard concentration shows that taking  $n = \mathrm{poly}(d, \frac{1}{\varepsilon})$  suffices to ensure that  $R$  is sufficiently close to  $\| \mu_t^* \|$ .
3.2 Mixtures of  $K$  Gaussians, from a warm start
In this section, we state our third main result, namely for learning mixtures of  $K$  Gaussians given by Eq. (6) from a warm start, and provide an overview of how the ideas from Section 2 can be extended to obtain this result.
Assumption 14. (Separation) For a mixture of  $K$  Gaussians given by Eq. (6), for every pair of components  $i, j \in \{1,2,\dots,K\}$  with  $i \neq j$ , we assume that the separation between their means  $\| \mu_i^* - \mu_j^* \| \geq C \sqrt{\log(\min(K,d))}$  for sufficiently large absolute constant  $C > 0$ .
Assumption 15. (Initialization) For each component  $i \in \{1,2,\ldots,K\}$ , we have an initialization  $\mu_i^{(0)}$  with the property that  $\| \mu_i^{(0)} - \mu_i^*\| \leq C' \sqrt{\log(\min(K,d))}$  for sufficiently small absolute constant  $C' > 0$ .
Theorem 16. Suppose a mixture of  $K$  Gaussians satisfies Assumption 14. Then, for any  $\varepsilon = \Theta(1 / \mathrm{poly}(d))$ , running gradient descent on the DDPM objective (Algorithm 1) at low noise scale  $t = O(1)$  and with initialization satisfying Assumption 15 results in mean parameters  $\{\mu_i\}_{i=1}^K$  such that with high probability, the mean parameters satisfy  $\| \mu_i - \mu_i^* \| \leq O(\varepsilon)$  for each  $i \in \{1,2,\dots,K\}$ . Additionally, the runtime and sample complexity of the algorithm is poly  $(d,1/\varepsilon)$  (see Theorem E.1 for more precise quantitative bounds).
We provide a brief overview of the proof here. The full proof can be found in Appendix E.
Proof sketch. For learning mixtures of two Gaussians, we have already established the connection between gradient descent on the DDPM objective and the EM algorithm. For mixtures of  $K$  Gaussians, however, in a local neighborhood around the ground truth parameters  $\theta^{*}$ , we show an equivalence between gradient EM (recall gradient EM performs one-step of gradient descent on the "M" step objective) and gradient descent on the DDPM objective. In particular, our main technical lemma (Lemma E.4) shows that for noise scale  $t = \Theta(1)$  and for any  $\mu_{i}$  that satisfies  $\| \mu_{i} - \mu_{i}^{*} \| \leq O(\sqrt{\log(\min(K,d))})$ , we have
$$
- \nabla_ {\mu_ {i, t}} L _ {t} \left(s _ {\theta_ {t}}\right) \approx \mathbb {E} _ {X _ {t}} \left[ w _ {i, t} \left(X _ {t}\right) \left(X _ {t} - \mu_ {i, t}\right) \right].
$$
Therefore, the iterate  $\mu_{i,t}^{\prime}$  resulting from a single gradient step on the DDPM objective  $L_{t}(s_{\theta_{t}})$  with learning rate  $\eta$  is given by
$$
\mu_ {1, t} ^ {\prime} = \mu_ {1, t} - \eta \nabla_ {\mu_ {1, t}} L _ {t} \left(s _ {\theta_ {t}}\right) \approx \mu_ {1, t} + \eta \mathbb {E} _ {X _ {t}} \left[ w _ {1, t} \left(X _ {t}\right) \left(X _ {t} - \mu_ {1, t}\right) \right]. \tag {13}
$$
Comparing Fact 6 with Eq. (13), we see the correspondence in this regime between gradient descent on the DDPM objective to gradient EM. Using this connection and an existing local convergence guarantee from the gradient EM literature [SN21, KC20], we obtain our main theorem for mixtures of  $K$  Gaussians. Full details can be found in Appendix E.
SC would like to thank Sinho Chewi, Khashayar Gatmiry, Frederic Koehler, and Holden Lee for enlightening discussions on sampling and score estimation. KS and AK are supported by the NSF AI Institute for Foundations of Machine Learning (IFML). SC is supported by NSF Award 2103300.
[AG22] Ahmed El Alaoui and Jason Gaitonde. Bounds on the covariance matrix of the sherrington-kirkpatrick model. arXiv preprint arXiv:2212.02445, 2022.
[BDD23] Joe Benton, George Deligiannidis, and Arnaud Doucet. Error bounds for flow matching methods. arXiv preprint arXiv:2305.16860, 2023.
[BDJ+22] Ainesh Bakshi, Ilias Diakonikolas, He Jia, Daniel M Kane, Pravesh K Kothari, and Santosh S Vempala. Robustly learning mixtures of k arbitrary gaussians. In Proceedings of the 54th Annual ACM SIGACT Symposium on Theory of Computing, pages 1234–1247, 2022.
[BKM17] David M Blei, Alp Kucukelbir, and Jon D McAuliffe. Variational inference: A review for statisticians. Journal of the American statistical Association, 112(518):859-877, 2017.
[BM11] Mohsen Bayati and Andrea Montanari. The dynamics of message passing on dense graphs, with applications to compressed sensing. IEEE Transactions on Information Theory, 57(2):764-785, 2011.
[BMR22] Adam Block, Youssef Mroueh, and Alexander Rakhlin. Generative modeling with denoising auto-encoders and Langevin sampling. arXiv preprint 2002.00107, 2022.
[BRST21] Joan Bruna, Oded Regev, Min Jae Song, and Yi Tang. Continuous lwe. In Proceedings of the 53rd Annual ACM SIGACT Symposium on Theory of Computing, pages 694-707, 2021.
[BS15] Mikhail Belkin and Kaushik Sinha. Polynomial learning of distribution families. SIAM Journal on Computing, 44(4):889-911, 2015.
[BWY17] Sivaraman Balakrishnan, Martin J Wainwright, and Bin Yu. Statistical guarantees for the em algorithm: From population to sample-based analysis. 2017.
$\left[\mathrm{CCL}^{+}23\right]$  Sitan Chen, Sinho Chewi, Holden Lee, Yanzhi Li, Jianfeng Lu, and Adil Salim. The probability flow ode is provably fast. arXiv preprint arXiv:2305.11798, 2023.
[Eld13] Ronen Eldan. Thin shell implies spectral gap up to polylog via a stochastic localization scheme. Geometric and Functional Analysis, 23(2):532-569, 2013.
[Eld20] Ronen Eldan. Taming correlations through entropy-efficient measure decompositions with applications to mean-field approximation. *Probability Theory and Related Fields*, 176(3-4):737-755, 2020.
[HJA20] Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in Neural Information Processing Systems, 33:6840-6851, 2020.
[HL18] Samuel B Hopkins and Jerry Li. Mixture models, robustness, and sum of squares proofs. In Proceedings of the 50th Annual ACM SIGACT Symposium on Theory of Computing, pages 1021-1034, 2018.
[HP15] Moritz Hardt and Eric Price. Tight bounds for learning a mixture of two gaussians. In Proceedings of the forty-seventh annual ACM symposium on Theory of computing, pages 753-760, 2015.
[Kab03] Yoshiyuki Kabashima. A cdma multiuser detection algorithm on the basis of belief propagation. Journal of Physics A: Mathematical and General, 36(43):11111, 2003.
[Kan21] Daniel M Kane. Robust learning of mixtures of gaussians. In Proceedings of the 2021 ACM-SIAM Symposium on Discrete Algorithms (SODA), pages 1246-1258. SIAM, 2021.
[KC20] Jeongyeol Kwon and Constantine Caramanis. The em algorithm gives sample-optimality for learning mixtures of well-separated gaussians. In Conference on Learning Theory, pages 2425–2487. PMLR, 2020.
[KMV10] Adam Tauman Kalai, Ankur Moitra, and Gregory Valiant. Efficiently learning mixtures of two gaussians. In Proceedings of the forty-second ACM symposium on Theory of computing, pages 553-562, 2010.
[KSS18] Pravesh K Kothari, Jacob Steinhardt, and David Steurer. Robust moment estimation and improved clustering via sum of squares. In Proceedings of the 50th Annual ACM SIGACT Symposium on Theory of Computing, pages 1035–1046, 2018.
[LL22] Allen Liu and Jerry Li. Clustering mixtures with almost optimal separation in polynomial time. In Proceedings of the 54th Annual ACM SIGACT Symposium on Theory of Computing, pages 1248-1261, 2022.
[LLT22] Holden Lee, Jianfeng Lu, and Yixin Tan. Convergence for score-based generative modeling with polynomial complexity. In Alice H. Oh, Alekh Agarwal, Danielle Belgrave, and Kyunghyun Cho, editors, Advances in Neural Information Processing Systems, 2022.
[LLT23] Holden Lee, Jianfeng Lu, and Yixin Tan. Convergence of score-based generative modeling for general data distributions. In International Conference on Algorithmic Learning Theory, pages 946–985. PMLR, 2023.
[LM23] Allen Liu and Ankur Moitra. Robustly learning general mixtures of gaussians. Journal of the ACM, 2023.
[ LWCC23] Gen Li, Yuting Wei, Yuxin Chen, and Yuejie Chi. Towards faster non-asymptotic convergence for diffusion-based generative models. arXiv preprint arXiv:2306.09251, 2023.
[LOWYL22] Xingchao Liu, Lemeng Wu, Mao Ye, and Qiang Liu. Let us build bridges: understanding and extending diffusion generative models. arXiv preprint arXiv:2208.14699, 2022.
[MM09] Marc Mezard and Andrea Montanari. Information, physics, and computation. Oxford University Press, 2009.
[WJ+08] Martin J Wainwright, Michael I Jordan, et al. Graphical models, exponential families, and variational inference. Foundations and Trends® in Machine Learning, 1(1-2):1-305, 2008.
[WY22] Andre Wibisono and Kaylee Y. Yang. Convergence in KL divergence of the inexact Langevin algorithm with application to score-based generative models. arXiv preprint 2211.01512, 2022.
[XHM16] Ji Xu, Daniel J Hsu, and Arian Maleki. Global analysis of expectation maximization for mixtures of two gaussians. Advances in Neural Information Processing Systems, 29, 2016.
[YYS17] Bowei Yan, Mingzhang Yin, and Purnamrita Sarkar. Convergence analysis of gradient em for multi-component gaussian mixture. arXiv preprint arXiv:1705.08530, 2017.
[ZLS20] Ruofei Zhao, Yuanzhi Li, and Yuekai Sun. Statistical convergence of the em algorithm on gaussian mixture models. 2020.