  Recently, we witness a rapid increase in the use of machine learning in
self-adaptive systems. Machine learning has been used for a variety of reasons,
ranging from learning a model of the environment of a system during operation
to filtering large sets of possible configurations before analysing them. While
a body of work on the use of machine learning in self-adaptive systems exists,
there is currently no systematic overview of this area. Such overview is
important for researchers to understand the state of the art and direct future
research efforts. This paper reports the results of a systematic literature
review that aims at providing such an overview. We focus on self-adaptive
systems that are based on a traditional Monitor-Analyze-Plan-Execute feedback
loop (MAPE). The research questions are centred on the problems that motivate
the use of machine learning in self-adaptive systems, the key engineering
aspects of learning in self-adaptation, and open challenges. The search
resulted in 6709 papers, of which 109 were retained for data collection.
Analysis of the collected data shows that machine learning is mostly used for
updating adaptation rules and policies to improve system qualities, and
managing resources to better balance qualities and resources. These problems
are primarily solved using supervised and interactive learning with
classification, regression and reinforcement learning as the dominant methods.
Surprisingly, unsupervised learning that naturally fits automation is only
applied in a small number of studies. Key open challenges in this area include
the performance of learning, managing the effects of learning, and dealing with
more complex types of goals. From the insights derived from this systematic
literature review we outline an initial design process for applying machine
learning in self-adaptive systems that are based on MAPE feedback loops.
