  Tuning complex machine learning systems is challenging. Machine learning
models typically expose a set of hyperparameters, be it regularization,
architecture, or optimization parameters, whose careful tuning is critical to
achieve good performance. To democratize access to such systems, it is
essential to automate this tuning process. This paper presents Amazon SageMaker
Automatic Model Tuning (AMT), a fully managed system for black-box optimization
at scale. AMT finds the best version of a machine learning model by repeatedly
training it with different hyperparameter configurations. It leverages either
random search or Bayesian optimization to choose the hyperparameter values
resulting in the best-performing model, as measured by the metric chosen by the
user. AMT can be used with built-in algorithms, custom algorithms, and Amazon
SageMaker pre-built containers for machine learning frameworks. We discuss the
core functionality, system architecture and our design principles. We also
describe some more advanced features provided by AMT, such as automated early
stopping and warm-starting, demonstrating their benefits in experiments.
