Gaussian process regression models under linear inequality conditions

In the last decades, Gaussian processes (GPs) have become one of the most attractive Bayesian framework due to their ability to perform both regression and classification tasks [1]. However, due to their pure data-driven nature, they do not account for the physical properties exhibited in real-world data (e.g. positivity, monotonicity), which can lead to more realistic data interpolation and uncertainty quantifications [2, 3]. Figure 1 shows an example where the target function satisfies both boundedness (i.e. 0 ≤ y(xi) ≤ 1, for all xi ∈ [0, 1]) and monotonicity constraints (i.e. y(xi−1) ≤ y(xi), if xi−1 ≤ xi). Our aim is to investigate deeper GP regression models under inequality constraints. 0.0 0.2 0.4 0.6 0.8 1.0 0 0 0 2 0 . 4 0 . 6 0 . 8 1 . 0 x y (x ) Testing data Training data Mean function Bounds CI 90% 0.0 0.2 0.4 0.6 0.8 1.0 0 0 0 2 0 . 4 0 . 6 0 . 8 1 . 0 x y (x ) Testing data Training data Mean function Bounds CI 90% Figure 1: GP models interpolating the Gaussian CDF x 7→ Φ(x−0.5 0.2 ): (left) classical (unconstrained) GP model, (right) constrained GP model satisfying both boundedness and monotonicity constraints. To the best of our knowledge, the only probabilistic model which satisfies specific inequalities everywhere in the input space is proposed in [2]. With this approach, the posterior converges to the one provided by splines methods [4]. Our work builds on this framework and our contributions are threefold. First, we extend their framework to deal with any linear inequality constraint (not only boundedness, monotonicity or convexity). Second, we suggest an efficient Hamiltonian Monte Carlo-based sampler to approximate the posterior distribution satisfying both interpolation and inequality constraints. Finally, we investigate theoretical and numerical properties of a constrained likelihood for covariance parameter estimation. The model was tested under both synthetic and real-world data in 1D or 2D. According to the experimental results under different types of inequality constraints, the proposed method fits properly the observations and provides realistic confidence intervals (see Figure 1). On a MascotNum Annual Conference, March 21-23 2018, Nantes, France 2D nuclear criticality safety dataset, it provides reliable results on both data prediction and uncertainty quantification satisfying both positivity and monotonicity conditions exhibited by the nuclear data. Table 1 shows the performances of the Gaussian models for interpolating the nuclear database using different number of training points n and using twenty different random Latin hypercube designs. The models are assessed using the Q criterion. One can observe that constrained GP models outperformed the unconstrained one in all the cases. Table 1: Assessment of the Gaussian models for interpolating the nuclear database using different number of training points n and using twenty different random Latin hypercube designs. Predictive accuracy is evaluated using the mean μ and the standard deviation σ of the Q results. MLE: maximum likelihood estimation. CMLE: constrained maximum likelihood estimation. n Q unconstrained GP + MLE constrained GP + MLE constrained GP + CMLE μ± σ μ± σ μ± σ 2 −0.128± 1.004 0.967 ± 0.026 0.952± 0.043 4 0.558± 0.260 0.981± 0.014 0.996 ± 0.006 6 0.858± 0.139 0.940± 0.059 0.995 ± 0.004 8 0.962± 0.035 0.995 ± 0.003 0.981± 0.011 We would like to present the results obtained during the 1st year of the PhD training at MascotNum Annual Conference 2018 for the poster session. We believe that an oral presentation would be more appropriate at a later edition of the conference, when the results of currently ongoing work could be presented. This ongoing work consists in the extension of our framework for higher dimension, and the investigation of the asymptotics for the constrained likelihood. We also believe that our presentation at the conference could be helpful to discuss possible research directions in the future.