  With the increase of research in self-adaptive systems, there is a need to
better understand the way research contributions are evaluated. Such insights
will support researchers to better compare new findings when developing new
knowledge for the community. However, so far there is no clear overview of how
evaluations are performed in self-adaptive systems. To address this gap, we
conduct a mapping study. The study focuses on experimental evaluations
published in the last decade at the prime venue of research in software
engineering for self-adaptive systems -- the International Symposium on
Software Engineering for Adaptive and Self-Managing Systems (SEAMS). Results
point out that specifics of self-adaptive systems require special attention in
the experimental process, including the distinction of the managing system
(i.e., the target of evaluation) and the managed system, the presence of
uncertainties that affect the system behavior and hence need to be taken into
account in data analysis, and the potential of managed systems to be reused
across experiments, beyond replications. To conclude, we offer a set of
suggestions derived from our study that can be used as input to enhance future
experiments in self-adaptive systems.
