Title: Efficient Prior Selection in Gaussian Process Bandits with Thompson Sampling
Primary area: reinforcement learning
Keywords: Multi-armed bandits, Gaussian processes, Thompson sampling, Hyperparameter optimization, Bayesian optimization

Abstract: Gaussian process (GP) bandits provide a powerful framework for performing blackbox optimization of unknown functions. The characteristics of the unknown function depend heavily on the assumed GP prior. Most work in the literature assume that this prior is known but in practice this seldom holds. Instead, practitioners often rely on maximum likelihood estimation to select the hyperparameters of the prior - which lacks theoretical guarantees. In this work, we propose two algorithms for joint prior selection and regret minimization in GP bandits based on GP Thompson sampling (GP-TS): Prior-Elimination GP-TS (PE-GP-TS) and HyperPrior GP-TS (HP-GP-TS). We theoretically analyze the algorithms and establish upper bounds for the regret of HP-GP-TS. In addition, we demonstrate the effectiveness of our algorithms compared to the alternatives through experiments with synthetic and real-world data.

TL;DR: We propose, analyze and evaluate two algorithms for joint prior selection and regret minimization based on Gaussian process Thompson sampling.