[3] L. Berrada, A. Zisserman, and M. P. Kumar. Training neural networks for and by interpolation. In Machine Learning, volume 119 of Proceedings of Machine Learning Research, 2020.
[4] E. Bingham, J. Chen, M. Jankowiak, F. Obermeyer, N. Pradhan, T. Karaletsos, R. Singh, P. Szerlip, P. Horsfall, and N. Goodman. Pyro: Deep universal probabilistic programming. arXiv:1810.09538, 2018.
[5] D. M. Blei, A. Kucukelbir, and J. D. McAuliffe. Variational inference: A review for statisticians. Journal of the American statistical Association, 112(518):859-877, 2017.
[6] B. Carpenter, A. Gelman, M. D. Hoffman, D. Lee, B. Goodrich, M. Betancourt, M. Brubaker, J. Guo, P. Li, and A. Riddell. Stan: A probabilistic programming language. Journal of Statistical Software, 76(1):1-32, 2017.
[7] K. Crammer, O. Dekel, J. Keshet, S. Shalev-Shwartz, and Y. Singer. Online passive-aggressive algorithms. Journal of Machine Learning Research, 7:551-585, 2006.
[8] A. K. Dhaka, A. Catalina, M. Welandawe, M. Riis Andersen, J. Huggins, and A. Vehtari. Challenges and Opportunities in High-dimensional Variational Inference. art. arXiv:2103.01085, March 2021.
[9] A. Dieng, D. Tran, R. Ranganath, J. Paisley, and D. Blei. Variational inference via  $\chi$  upper bound minimization. In Neural Information Processing Systems, 2017.
[10] J. Domke. Provable gradient variance guarantees for black-box variational inference. In Neural Information Processing Systems, volume 32, 2019.
[11] J. Domke. Provable smoothness guarantees for black-box variational inference. In International Conference on Machine Learning, volume 119, pages 2587-2596. PMLR, 2020.
[12] M. Dredze, K. Crammer, and F. Pereira. Confidence-weighted linear classification. In International Conference on Machine Learning., volume 307, pages 264-271, 2008.
[13] R. M. Gower, A. Defazio, and M. Rabbat. Stochastic polyak stepsize with a moving target. arXiv:2106.11851, 2021.
[14] A. Gunes Baydin, B. A. Pearlmutter, A. Andreyevich Radul, and J. M. Siskind. Automatic differentiation in machine learning: a survey. arXiv e-prints, art. arXiv:1502.05767, February 2015. doi: 10.48550/arXiv.1502.05767.
[15] M. Hoffman, D. Blei, C. Wang, and J. Paisley. Stochastic variational inference. Journal of Machine Learning Research, 14:1303-1347, 2013.
[16] A. Hyvarinen. Estimation of non-normalized statistical models by score matching. Journal of Machine Learning Research, 6:695-709, 2005.
[17] C. Jones and A. Pewsey. The Sinh-Arcsinh Normal Distribution. *Significance*, 16(2):6-7, 04 2019. ISSN 1740-9705.
[18] M. I. Jordan, Z. Ghahramani, T. S. Jaakkola, and L. K. Saul. An introduction to variational methods for graphical models. Machine learning, 37:183-233, 1999.
[19] D. P. Kingma and J. Ba. Adam: A method for stochastic optimization. In ICLR, 2015.
[20] J. Knoblauch, J. Jewson, and T. Damoulas. An optimization-centric view on Bayes' rule: Reviewing and generalizing variational inference. Journal of Machine Learning Research, 23: 1-109, 2022.
[21] A. Kucukelbir, D. Tran, R. Ranganath, A. Gelman, and D. M. Blei. Automatic Differentiation Variational Inference. art. arXiv:1603.00788, March 2016.
[22] W. Lin, M. Emtiyaz Khan, and M. Schmidt. Fast and Simple Natural-Gradient Variational Inference with Mixture of Exponential-family Approximations. arXiv e-prints, art. arXiv:1906.02914, June 2019. doi: 10.48550/arXiv.1906.02914.
[23] N. Loizou, S. Vaswani, I. Laradji, and S. Lacoste-Julien. Stochastic polyak step-size for sgd: An adaptive learning rate for fast convergence. International Conference on Artificial Intelligence and Statistics, 2021.
[24] M. Magnusson, P. Bürkner, and A. Vehtari. posteriordb: a set of posteriors for Bayesian inference and probabilistic programming, November 2022. URL https://github.com/stan-dev/posteriordb.