Physics Informed Deep Kernel Learning
Authors: Zheng Wang, Wei Xing, Robert Kirby, Shandian Zhe
Keywords: Deep Kernel, Bayesian Learning
Abstract: Deep kernel learning is a promising combination of deep neural networks and nonparametric function estimation. However, as a data-driven approach, the performance of deep kernel learning can still be restricted by scarce or insufficient data, especially in extrapolation tasks.  To address these limitations, we propose Physics Informed Deep Kernel Learning (PI-DKL) that exploits physics  knowledge represented by differential equations with latent sources. Specifically, we use the posterior function sample of the  Gaussian process as the surrogate for the solution of the differential equation, and construct  a generative component to integrate the equation in a principled Bayesian hybrid framework. For efficient and effective inference, we marginalize out the latent variables in the joint probability and derive a simple model evidence lower bound (ELBO), based on which we develop a stochastic collapsed inference algorithm. Our ELBO can be viewed as a nice, interpretable posterior regularization objective. On synthetic datasets and real-world applications, we show the advantage of our approach in both prediction accuracy and uncertainty quantification.