ChePAN: Constrained Black-Box Uncertainty Modelling with Quantile Regression
Authors: Axel Brando, Joan Gimeno, Jose Antonio Rodriguez-Serrano, Jordi Vitria
Keywords: Uncertainty modelling, Deep Learning, Black Box, Aleatoric, Quantile Regression, Chebyshev Polynomial, Neural networks
Abstract: Most predictive systems currently in use do not report any useful information for auditing their associated uncertainty and evaluating the corresponding risk. Taking it for granted that their replacement may not be advisable in the short term, in this paper we propose a novel approach to modelling confidence in such systems while preserving their predictions. The method is based on the Chebyshev Polynomial Approximation Network (the ChePAN), a new way of modelling aleatoric uncertainty in a regression scenario. In the case addressed here, uncertainty is modelled by building conditional quantiles on top of the original pointwise forecasting system considered as a black box, i.e. without making assumptions about its internal structure. Furthermore, the ChePAN allows users to consistently choose how to constrain any predicted quantile with respect to the original forecaster. Experiments show that the proposed method scales to large size data sets and transfers the advantages of quantile regression to estimating black-box uncertainty.