A uto ML for NLP (2023)

Abstract: Automated Machine Learning (AutoML) is an emerging field that has potential to impact how we build models in NLP. As an umbrella term that includes topics like hyperparameter optimization and neural architecture search, AutoML has recently become mainstream at major conferences such as NeurIPS, ICML, and ICLR. What does this mean to NLP? Currently, models are often built in an ad hoc process: we might borrow default hyperparameters from previous work and try a few variant architectures, but it is never guaranteed that final trained model is optimal. Automation can introduce rigor in this model-building process. This tutorial will summarize the main AutoML techniques and illustrate how to apply them to improve the NLP model-building process.

## tutorial content outline: this is a 3-hour tutorial. it is divided into two parts: reading list: the tutorial will be self-contained, so there is no required reading list. for a preview of the techniques we will cover, the audience is welcomed to refer to survey papers such as @xcite . we gave a similar tutorial titled "automl for machine translation" at amta 2022, a machine translation conference. the tutorial slides are available 3 . for the eacl tutorial, we will add discussion on recent uses of automl in various nlp applications, ranging from text classification to large language models. ## ethics statement there are at least three concerns relevant for responsible use of automl technology.