[1] I. Goodfellow, Y. Bengio, and A. Courville. Deep Learning. MIT Press, 2016.
[2] I. Loshchilov and F. Hutter. Decoupled weight decay regularization. In Proceedings of the International Conference on Learning Representations (ICLR'19), 2019. Published online: iclr.cc.
[3] Y. Chen, A. Huang, Z. Wang, I. Antonoglou, J. Schrittwieser, D. Silver, and N. de Freitas. Bayesian optimization in alphago. arXiv:1812.06855 [cs.LG], 2018.
[4] G. Melis, C. Dyer, and P. Blunsom. On the state of the art of evaluation in neural language models. In Proceedings of the International Conference on Learning Representations (ICLR'18), 2018. Published online: icldr.cc.
[5] B. Zhang, R. Rajan, L. Pineda, N. Lambert, A. Biedenkapp, K. Chua, F. Hutter, and R. Calandra. On the importance of Hyperparameter Optimization for model-based reinforcement learning. In A. Banerjee and K. Fukumizu, editors, Proceedings of the 24th International Conference on Artificial Intelligence and Statistics (AISTATS'21), pages 4015-4023. Proceedings of Machine Learning Research, 2021.
[6] R. Wightman, H. Touvron, and H. Jégou. Resnet strikes back: An improved training procedure in timm. In Ranzato et al. [45].
[7] J. Snoek, H. Larochelle, and R. Adams. Practical Bayesian optimization of machine learning algorithms. In P. Bartlett, F. Pereira, C. Burges, L. Bottou, and K. Weinberger, editors, Proceedings of the 25th International Conference on Advances in Neural Information Processing Systems (NeurIPS'12), pages 2960-2968. Curran Associates, 2012.
[8] M. Feurer, K. Eggensperger, S. Falkner, M. Lindauer, and F. Hutter. Auto-Sklearn 2.0: Hands-free automl via meta-learning. JMLR, 23(261):1-61, 2022.
[9] T. Brown, B. Mann, N. Ryder, M. Subbiah, J. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell, S. Agarwal, A. Herbert-Voss, G. Krueger, T. Henighan, R. Child, A. Ramesh, D. Ziegler, J. Wu, C. Winter, C. Hesse, M. Chen, E. Sigler, M. Litwin, S. Gray, B. Chess, J. Clark, C. Berner, S. McCandlish, A. Radford, I. Sutskever, and D. Amodei. Language models are few-shot learners. In Larochelle et al. [46], pages 1877-1901.
[10] Lambda. Demystifying gpt-3. https://lambdalabs.com/blog/demystifying-gpt-3, June 2020. Accessed on May 4, 2023.
[11] X. Bouthillier and G. Varoquaux. Survey of machine-learning experimental methods at NeurIPS2019 and ICLR2020. Research report [hal-02447823], Inria Saclay Ile de France, 2020.
[12] N. Hasebrook, F. Morsbach, N. Kannengieber, M. Zoller, J. Franke, M. Lindauer, F. Hutter, and A. Sunyaev. Practitioner motives to select hyperparameter optimization methods. arXiv preprint arXiv:2203.01717, 2022.
[13] F. Schneider, Z. Nado, N. Agarwal, G. E. Dahl, and P. Hennig. Hity workshop poll, neurips 2022. https://github.com/fsschneider/HITYWorkshopPoll, 2022.
[14] K. Van der Blom, A. Serban, H. Hoos, and J. Visser. Automl adoption in ml software. In K. Eggensperger, F. Hutter, E. LeDell, M. Lindauer, G. Shala, J. Vanschoren, and C. White, editors, ICML workshop on Automated Machine Learning (AutoML workshop 2021), 2021.
[15] Javier Gonzalez, Zhenwen Dai, Philipp Hennig, and Neil Lawrence. Batch bayesian optimization via local penalization. In Arthur Gretton and Christian C. Robert, editors, Proceedings of the 19th International Conference on Artificial Intelligence and Statistics, volume 51 of Proceedings of Machine Learning Research, pages 648-657, Cadiz, Spain, 09-11 May 2016. PMLR. URL https://proceedings.mlr.press/v51/gonzalez16a.html.