  This work uses genetic programming to explore the space of continuous
optimisers, with the goal of discovering novel ways of doing optimisation. In
order to keep the search space broad, the optimisers are evolved from scratch
using Push, a Turing-complete, general-purpose, language. The resulting
optimisers are found to be diverse, and explore their optimisation landscapes
using a variety of interesting, and sometimes unusual, strategies.
Significantly, when applied to problems that were not seen during training,
many of the evolved optimisers generalise well, and often outperform existing
optimisers. This supports the idea that novel and effective forms of
optimisation can be discovered in an automated manner. This paper also shows
that pools of evolved optimisers can be hybridised to further increase their
generality, leading to optimisers that perform robustly over a broad variety of
problem types and sizes.
