  Over the last decade, the long-running endeavour to automate high-level
processes in machine learning (ML) has risen to mainstream prominence,
stimulated by advances in optimisation techniques and their impact on selecting
ML models/algorithms. Central to this drive is the appeal of engineering a
computational system that both discovers and deploys high-performance solutions
to arbitrary ML problems with minimal human interaction. Beyond this, an even
loftier goal is the pursuit of autonomy, which describes the capability of the
system to independently adjust an ML solution over a lifetime of changing
contexts. However, these ambitions are unlikely to be achieved in a robust
manner without the broader synthesis of various mechanisms and theoretical
frameworks, which, at the present time, remain scattered across numerous
research threads. Accordingly, this review seeks to motivate a more expansive
perspective on what constitutes an automated/autonomous ML system, alongside
consideration of how best to consolidate those elements. In doing so, we survey
developments in the following research areas: hyperparameter optimisation,
multi-component models, neural architecture search, automated feature
engineering, meta-learning, multi-level ensembling, dynamic adaptation,
multi-objective evaluation, resource constraints, flexible user involvement,
and the principles of generalisation. We also develop a conceptual framework
throughout the review, augmented by each topic, to illustrate one possible way
of fusing high-level mechanisms into an autonomous ML system. Ultimately, we
conclude that the notion of architectural integration deserves more discussion,
without which the field of automated ML risks stifling both its technical
advantages and general uptake.
