$\left[\mathrm{BDM}^{+}16\right]$  Dariusz Buraczewski, Ewa Damek, Thomas Mikosch, et al. Stochastic models with power-law tails. The equation  $X = AX + B$ . Cham: Springer, 2016.
[Bog06] VI Bogachev. Measure theory (second volume), 2006.
$\left[\mathrm{BSE}^{+}21\right]$  Melih Barsbey, Milad Sefidgaran, Murat A Erdogdu, Gael Richard, and Umut Simsekli. Heavy tails in sgd and compressibility of overparametrized neural networks. Advances in Neural Information Processing Systems, 34:29364-29378, 2021.
[BWMP12] A. Benveniste, S.S. Wilson, M. Metivier, and P. Priouret. Adaptive Algorithms and Stochastic Approximations. Stochastic Modelling and Applied Probability. Springer Berlin Heidelberg, 2012.
[CGZ19] Bugra Can, Mert Gurbuzbalaban, and Lingjiong Zhu. Accelerated linear convergence of stochastic momentum methods in Wasserstein distances. In International Conference on Machine Learning, pages 891–901. PMLR, 2019.
[DDB20] Aymeric Dieuleveut, Alain Durmus, and Francis Bach. Bridging the gap between constant step size stochastic gradient descent and markov chains, 2020.
[DMN+21] Alain Durmus, Eric Moulines, Alexey Naumov, Sergey Samsonov, Kevin Scaman, and Hoi-To Wai. Tight high probability bounds for linear stochastic approximation with fixed stepsize. Advances in Neural Information Processing Systems, 34:30063-30074, 2021.
[FG15] Nicolas Fournier and Arnaud Guillin. On the rate of convergence in wasserstein distance of the empirical measure. Probability Theory and Related Fields, 162(3):707-738, 2015.
[GSZ21] Mert Gurbuzbalaban, Umut Simsekli, and Lingjiong Zhu. The heavy-tail phenomenon in sgd. In International Conference on Machine Learning, pages 3964-3975. PMLR, 2021.
[HM21] Liam Hodgkinson and Michael Mahoney. Multiplicative noise and heavy tails in stochastic optimization. In International Conference on Machine Learning, pages 4262-4274. PMLR, 2021.
[HSKM22] Liam Hodgkinson, Umut Simsekli, Rajiv Khanna, and Michael Mahoney. Generalization bounds using lower tail exponents in stochastic optimizers. In International Conference on Machine Learning, pages 8774-8795. PMLR, 2022.
[KSH17] Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. Imagenet classification with deep convolutional neural networks. Communications of the ACM, 60(6):84-90, 2017.
[LBBH98] Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11):2278-2324, 1998.
[LW§22] Soon Hoe Lim, Yijun Wan, and Umut Şimsekli. Chaotic regularization and heavy-tailed limits for deterministic gradient descent. arXiv preprint arXiv:2205.11361, 2022.
[MM19] Charles H Martin and Michael W Mahoney. Traditional and heavy-tailed self regularization in neural network models. arXiv preprint arXiv:1901.08276, 2019.
[MMO15] Mohammad Mohammadi, Adel Mohammadpour, and Hiroaki Ogata. On estimating the tail index and the spectral measure of multivariate  $\alpha$ -stable distributions. *Metrika*, 78(5):549-561, 2015.
[PZ19] Victor M Panaretos and Yoav Zemel. Statistical aspects of wasserstein distances. Annual review of statistics and its application, 6:405-431, 2019.
$\left[\mathrm{RBG}^{+}22\right]$  Anant Raj, Melih Barsbey, Mert Gurbuzbalaban, Lingjiong Zhu, and Umut Şimsekli. Algorithmic stability of heavy-tailed stochastic gradient descent on least squares. arXiv preprint arXiv:2206.01274, 2022.