Pablo Moreno-Muñoz - Model Recycling with Gaussian Processes
Channel: Secondmind
Date: 2022-08-15

Abstract:
Gaussian processes are well known for their modelling flexibility and robustness to overfitting, but also for their prohibitive computational cost in training and prediction. In practice, modern sparse approximations circumvent this problem via inducing inputs and variational inference. However, the utility of inducing points goes beyond the scalability problem. In this talk, I will present a new framework for transfer learning based on the idea of model recycling. Crucially, the use of inducing points and variational posteriors makes it possible to perform two intertwined tasks between pre-trained GP models: i) building meta-models from models and ii) updating models with new data, without revisiting any sample. The framework avoids undesired data centralisation, reduces computational cost and allows the transfer of uncertainty metrics after training. The method exploits the augmentation of high-dimensional integral operators based on the Kullback-Leibler divergence between stochastic processes. This introduces efficient lower bounds under all pre-trained sparse variational GPs, with different complexity and even likelihood model. In the talk, I will also show experimental results for two scenarios, one for building meta-models given a collection of already-fitted GPs and another for continual inference given streaming data.

References:
- P. Moreno-Muñoz, A. Artés-Rodríguez and M. A. Álvarez. Modular Gaussian Processes for Transfer Learning, In Advances in Neural Information Processing Systems (NeurIPS), 2021 
- P. Moreno-Muñoz, A. Artés-Rodríguez and M. A. Álvarez. Continual Multi-task Gaussian Processes, In arXiv:1911.00002, 2019

Speaker: 
Pablo Moreno-Muñoz (Technical University of Denmark), personal website can be found at https://pmorenoz.github.io/.

This talk was given at Secondmind Labs, as a part of our (virtual) research seminar. Our research seminar is where we exchange ideas with guest speakers, keeping you up to date with the latest developments and inspiring research topics. Occasionally, Secondmind researchers present their own work as well. You can find a complete list of speakers at https://www.secondmind.ai/labs/seminars/. Learn more about Secondmind Labs at https://www.secondmind.ai/labs/​