title: Pushing The Limits of Sample-Efficient Optimisation
Creators: name: Cowen-Rivers, Alexander
Publisher: name: UNSPECIFIED
Year: 2023

description: Humans excel at confronting problems with little to no prior information about, and with few interactions reasoning over the problems to propose adequate solutions. In other terms, when a human encounters an unknown function, they are able to, with few samples, find a variable such that the evaluation of this variable in the unknown function produces a satisfactory result. However, as the unknown function becomes increasingly non-linear, as well as the design space x becomes increasingly abstract, it becomes harder for a human to utilise prior knowledge around this abstracted black-box function. Bayesian Optimisation, on the other hand, provides a principled approach to reasoning over the, typically expensive to evaluate, unknown function f and exploring regions of uncertainty in an efficient manner. Ubiquitous applications of Bayesian Optimisation range from hyper-parameter tuning, molecule design, sensor placement, antenna design and laser optimisation. Thus improvements in the performance of Bayesian Optimisation can have wide-ranging implications in many practical applications. Bayesian Optimisation also offers the great potential to enable systems to autonomously tune their hyper-parameters, as well as automatically design the machine learning architectures (AutoML). In this thesis, we want to advance the success of Bayesian optimisation algorithms through revisiting and contributing to the first two stages in detail. In the first stage, the surrogate model is chosen and constructed typically based on certain assumptions of the unknown function, such as whether the function is deterministic or stochastic? and if it's believed to be stochastic is its noise process homoscedastic or heteroscedastic? Do we believe the unknown function to be stationary or non-stationary? To assess the effect of these assumptions, we look at a broad range of applications of tuning machine learning models in typically studied domains. We find that through revisiting these initial assumptions imposed at the start of applying Bayesian Optimisation, we can construct a novel algorithm HEBO that achieves state-of-the-art performance compared to existing methods. HEBO is verified externally also, by the submission of our algorithm into the NeurIPS 2020 Black-box Optimisation challenge, whereby our proposed method achieved 1st place when evaluated on a wide variety of held-out tasks. We then visit the second stage of the optimisation process, and we look at new ways to optimise the acquisition functions by framing them in a mathematically equivalent compositional format, which allows for the application of a new family of compositional optimisers. We show that on synthetic and real-world experiments, that these compositional methods perform favourably in the majority of applications. Lastly, we attempt to carry through a Bayesian optimisation perspective towards safe sequential decision making and propose new acquisition functions to determine the fitness value for safe reinforcement learning, and evaluate them on a variety of challenging benchmarks such as constrained robotic car, constrained point robot, constrained robotic arm control, constrained pendulum and constrained double pendulum. Whereby all robotic actuators are tasked with reading a goal state whilst avoiding an unsafe region defined within the state space. We find that the incorporation of an acquisition function helps guide exploration and leading to improved sample complexity in acquiring safe policies in training and evaluation. To summarise, we have developed a new Bayesian optimisation algorithm that was successfully shown to be state-of-the-art internally against prior Black-box optimisation algorithms, as well as externally in the NeurIPS 2020 Black-box optimisation challenge. HEBO was shown to be two orders of magnitude more sample efficient than random search for certain black-box optimisation tasks. We also introduced novel formulations of popular acquisition functions in a mathematically equivalent compositional framework, allowing us to bridge the well-studied field of compositional optimisation together and show the success of doing so across commonly studied synthetic and real-world Bayesian optimisation benchmarks. Finally, we study the important problem of safe sequential decision making and construct novel acquisition functions that allow agents to safely explore their environments.