Q UANTIFYING P OINT -P REDICTION U NCERTAINTY IN N EURAL N ETWORKS VIA R ESIDUAL E STIMATION WITH AN I/O K ERNEL
A BSTRACT
Neural Networks (NNs) have been extensively used for a wide spectrum of realworld regression tasks, where the goal is to predict a numerical outcome such as revenue, effectiveness, or a quantitative result.
In many such tasks, the point prediction is not enough: the uncertainty (i.e. risk or conﬁdence) of that prediction must also be estimated.
Standard NNs, which are most often used in such tasks, do not provide uncertainty information.
Existing approaches address this issue by combining Bayesian models with NNs, but these models are hard to implement, more expensive to train, and usually do not predict as accurately as standard NNs.
In this paper, a new framework (RIO) is developed that makes it possible to estimate uncertainty in any pretrained standard NN.
The behavior of the NN is captured by modeling its prediction residuals with a Gaussian Process, whose kernel includes both the NN’s input and its output.
The framework is evaluated in twelve realworld datasets, where it is found to (1) provide reliable estimates of uncertainty, (2) reduce the error of the point predictions, and (3) scale well to large datasets.
Given that RIO can be applied to any standard NN without modiﬁcations to model architecture or training pipeline, it provides an important ingredient for building real-world NN applications.
I NTRODUCTION
Nowadays, Neural Networks (NNs) are arguably the most popular machine learning tool among
Artiﬁcial Intelligence (AI) community.
Researchers and practitioners have applied NNs to a wide variety of ﬁelds, including manufacturing (Bergmann et al., 2014), bioinformatics (LeCun et al., 2015), physics (Baldi et al., 2014), ﬁnance (Niaki & Hoseinzade, 2013), chemistry (Anjos et al., 2015), healthcare (Shahid et al., 2019), etc.
Although standard NNs are good at making a point prediction (a single outcome) for supervised learning tasks, they are unable to provide uncertainty information about predictions.
For real-world decision makers, representing prediction uncertainty is of crucial importance (Krzywinski & Altman, 2013; Ghahramani, 2015).
For example, in the case of regression, providing a 95% conﬁdence interval around the prediction allows the decision maker to anticipate the possible outcomes with explicit probability.
In contrast, simply returning a single point prediction imposes increased risks on decision making, e.g., a predictively good but actually risky medical treatment may be overconﬁdently interpreted without uncertainty information.
Conventional Bayesian models such as Gaussian Processes (GP) (Rasmussen & Williams, 2006) offer a mathematically grounded approach to reason about the predictive uncertainty, but often come with a prohibitive computational cost and lower prediction performance compared to NNs (Gal & Ghahramani, 2016).
As a potential solution, considerable research has been devoted to the combination of Bayesian models and NNs (see Section 2 for a detailed review of such approaches), aiming to overcome the downsides of each.
However, from the classical Bayesian Neural Network (Neal, 1996), in which a distribution of weights is learned, to the recent Neural Processes (Garnelo et al., 2018a;b; Kim et al., 2019), in which a distribution over functions is deﬁned, all such methods require signiﬁcant modiﬁcations to the model infrastructure and training pipeline.
Compared to standard (non-Bayesian) NNs, these new models are often computationally slower to train and harder to implement (Gal & Ghahramani, 2016; Lakshminarayanan et al., 2017), creating tremendous difﬁculty for practical uses.
Gal & Ghahramani (2016) derived a theoretical tool to extract uncertainty information from dropout training, however, the method can only be applied to dropout models, and requires changes to their internal inference pipeline.
Quantifying point-prediction uncertainty in standard NNs, which are overwhelmingly popular in practical applications, still remains a challenging problem with signiﬁcant potential impact.
To circumvent the above issues, this paper presents a new framework that can quantitatively estimate the point-prediction uncertainty of standard NNs without any modiﬁcations to the model structure or training pipeline.
The proposed approach works as a supporting tool that can augment any pretrained
NN without retraining them.
The idea is to capture the behavior of the NN by estimating its prediction residuals using a modiﬁed GP, which uses a new composite (I/O) kernel that makes use of both inputs and outputs of the NNs.
The framework is referred to as RIO (for Residual estimation with an I/O kernel).
In addition to providing valuable uncertainty estimation, RIO has an interesting side-effect: It provides a way to reduce the error of the NN predictions.
Moreover, with the help of recent sparse GP models, RIO scales well to large datasets.
Since classiﬁcation problems can be treated as regression on class labels (Lee et al., 2018), this paper focuses on regression tasks.
In this setting, empirical studies are conducted with twelve real-world datasets, and an initial theoretical investigation is presented that characterizes the beneﬁts of RIO.
The results show that RIO exhibits reliable uncertainty estimations, more accurate point predictions, and better scalability compared to alternative approaches.
R ELATED W ORK
There has been signiﬁcant interest in combining NNs with probabilistic Bayesian models.
An early approach was Bayesian Neural Networks, in which a prior distribution is deﬁned on the weights and biases of a NN, and a posterior distribution is then inferred from the training data (MacKay, 1992;
Neal, 1996).
Notice that unlike these methods, RIO is concerned only with uncertainty over NN predictions, not over NN weights.
Traditional variational inference techniques have been applied to the learning procedure of Bayesian NN, but with limited success (Hinton & van Camp, 1993;
Barber & Bishop, 1998; Graves, 2011).
By using a more advanced variational inference method, new approximations for Bayesian NNs were achieved that provided similar prediction performance as dropout NNs (Blundell et al., 2015).
However, the main drawbacks of Bayesian NNs remain: prohibitive computational cost and difﬁcult implementation procedure compared to standard NNs.
Alternatives to Bayesian NNs have been developed recently.
One such approach introduces a training pipeline that incorporates ensembles of NNs and adversarial training (Lakshminarayanan et al., 2017).
Another approach, NNGP, considers a theoretical connection between NNs and GP to develop a model approximating the Bayesian inference process of wide deep neural networks (Lee et al., 2018).
Deep kernel learning (DKL) combines NNs with GP by using a deep NN embedding as the input to the GP kernel (Wilson et al., 2016).
In Iwata & Ghahramani (2017), NNs are used for the mean functions of GPs, and parameters of both NNs and GP kernels are simultaneously estimated by stochastic gradient descent methods.
Conditional Neural Processes (CNPs) combine the beneﬁts of NNs and GP, by deﬁning conditional distributions over functions given data, and parameterizing this dependence with a NN (Garnelo et al., 2018a).
Neural Processes (NPs) generalize deterministic
CNPs by incorporating a latent variable, strengthening the connection to approximate Bayesian and latent variable approaches (Garnelo et al., 2018b).
Attentive Neural Processes (ANPs) further extends
NPs by incorporating attention to overcome underﬁtting issues (Kim et al., 2019).
The above models all require signiﬁcant modiﬁcations to the original NN model and training pipeline.
Compared to standard NNs, they are also less computationally efﬁcient and more difﬁcult for practitioners to implement.
In the approach that shares the most motivation with RIO, Monte Carlo dropout was used to estimate the predictive uncertainty of dropout NNs (Gal & Ghahramani, 2016).
However, this method is restricted to dropout NNs, and also requires modiﬁcations to the NN inference process.
Different from all above-mentioned methods, which are independent alternatives to standard NNs,
[Figure]
RIO is designed as a supporting tool that can be applied on top of any pretrained NN.
RIO augments standard NNs without retraining or modifying any component of them.
T HE RIO F RAMEWORK
This section gives the general problem statement, develops the RIO framework, and discusses its scalability.
For background introductions of NNs, GP, and its more efﬁcient approximation, SVGP (Hensman et al., 2013; 2015), see Appendix B.
3. P ROBLEM S TATEMENT
Consider a training dataset D =
( X , y )
= { ( x i , y i ) } ni
=1 , and a pretrained standard NN that outputs a point prediction ˆ y i given x i .
The problem is two-fold: (1) Quantify the uncertainty in the predictions of the NN, and (2) calibrate the point predictions of the NN (i.e. make them more accurate).
3. F RAMEWORK O VERVIEW
RIO solves this problem by modeling the residuals between observed outcomes y and NN predictions
ˆ y using GP with a composite kernel.
The framework can be divided into two phases: the training phase and the deployment phase.
In the training phase, the residuals between observed outcomes and NN predictions on the training dataset are calculated as
[Equation]
Let r denote the vector of all residuals and ˆy denote the vector of all NN predictions.
A GP with a composite kernel is trained assuming r ∼ N (0 , K c (( X , ˆy ) , ( X , ˆy ))
+ σ 2 n I ) , where K c (( X , ˆy ) , ( X , ˆy )) denotes an n × n covariance matrix at all pairs of training points based on a composite kernel
[Equation]
Suppose a radial basis function (RBF) kernel is used for both k in and k out .
Then,
[Equation]
The training process of GP learns the hyperparameters σ 2in , l in , σ 2out , l out , and σ 2 n by maximizing the log marginal likelihood log p ( r |X , ˆy ) given by log p ( r |X , ˆy )
= − 1 2 r (cid:62) ( K c (( X , ˆy ) , ( X , ˆy ))
+ σ 2 n I ) r − 1 2 log | K c (( X , ˆy ) , ( X , ˆy ))
+ σ 2 n I |− n 2 log 2 π .
[Equation]
[Figure]
The ineffective neural network has introduced unnecessary complexity, so the residuals are modeled with high uncertainty.
In both cases, RIO matches the intuition for how uncertain the NN really is.
In the deployment phase, a test point x ∗ is input to the NN to get an output ˆ y ∗ .
The trained GP predicts the distribution of the residual as ˆ r ∗ |X , ˆy , r , x ∗ , ˆ y ∗ ∼ N (¯ˆ r ∗ , var(ˆ r ∗ )) , where
[Equation]
where k ∗ denotes the vector of kernel-based covariances (i.e., k c (( x ∗ , ˆ y ∗ ) ,
[Equation]
x
[Equation]
ˆ y i )) )
between ( x ∗ , ˆ y ∗ ) and the training points.
Interestingly, note that the predicted residuals can also be used to calibrate the point predictions of the NN, so that the ﬁnal calibrated prediction with uncertainty information is given by
[Equation]
In other words, RIO not only adds uncertainty estimation to a standard NN—it also provides a way to calibrate NN predictions, without any modiﬁcation to its architecture or training.
Figure 1 shows the overall procedure when applying the proposed framework in real-world applications.
Figure 2 shows example behavior of RIO that illustrates the intuition of the approach.
An algorithmic description of
RIO is provided in Appendix C.
3. S CALABILITY
RIO is scalable to large datasets (No. of data points ×
No. of features > 1M) by applying sparse
GP methods, e.g., SVGP (Hensman et al., 2013; 2015).
All the conclusions in previous sections still remain valid since sparse GP is simply an approximation of the original GP.
In the case of applying SVGP with a traditional optimizer, e.g., L-BFGS-B (Byrd et al., 1995; Zhu et al., 1997), the computational complexity is O ( nm 2 ) , and space complexity is O ( nm ) , where n is the number of data points and m is the number of inducing variables, compared to O ( n 3 ) and O ( n 2 ) for traditional
GP.
Experiments verify that the computational cost of RIO with SVGP is signiﬁcantly cheaper than other state-of-the-art approaches.
E MPIRICAL E VALUATION
Experiments in this section compare nine algorithms on 12 real-world datasets.
The algorithms include standard NN, the proposed RIO framework, four ablated variants of RIO, and three state-ofthe-art models that provide predictive uncertainty: SVGP (Hensman et al., 2013), Neural Network
Gaussian Process (NNGP) (Lee et al., 2018), and Attentive Neural Processes (ANP) (Kim et al., 2019).
In naming the RIO variants,“R” means estimating NN residuals then correcting NN outputs,“Y” means directly estimating outcomes, “I” means only using input kernel, “O” means only using output kernel, and “IO” means using I/O kernel.
For all RIO variants (including full RIO), SVGP is used as the GP component, but using the appropriate kernel and prediction target.
Therefore, “Y+I” amounts to original SVGP, and it is denoted as “SVGP” in all the experimental results.
All 12 datasets
[Table]
are real-world regression problems (Dua & Graff, 2017)
[Table]
and cover a wide variety of dataset sizes and feature dimensionalities.
Except for the “MSD” dataset, all other datasets are tested forindependent runs.
During each run, the dataset is randomly split into training set, validation set, and test set, and all algorithms are trained on the same split.
All RIO variants that involve an output kernel or residual estimation are based on the trained NN in the same run.
For “MSD”, since the dataset split is strictly predeﬁned by the provider, only 10 independent runs are conducted.
NNGP and ANP are only tested on the four smallest dataset (based on the product of dataset size and feature dimensionality) because they do not scale well to larger datasets.
It is notable that for all the RIO variants, no extensive hyperparameter tuning is conducted; the same default setup is used for all experiments, i.e., standard RBF kernel and 50 inducing points.
See Appendix D.1 for additional details of experimental setups and a link to downloadable source code.
Table 1 summarizes the numerical results from these experiments.
The main conclusions in terms of point-prediction error, uncertainty estimation, computational requirements, and ablation studies are summarized below.
Point-Prediction Error
The errors between point predictions of models and true outcomes of test points are measured using Root Mean Square Error (RMSE); the mean and standard deviation of
RMSEs over multiple experimental runs are shown in Table 1.
For models that return a probabilistic distribution, the mean of the distribution is the point prediction.
Although the main motivation of
RIO is to enhance pretrained NN rather than construct a new state-of-the-art prediction model from scratch, RIO performs the best or equals the best method (based on statistical tests) in 10 out ofdatasets.
RIO signiﬁcantly outperforms original NN in all 12 datasets, while original SVGP performs signiﬁcantly worse than NN in 7 datasets.
For the CT dataset, which has 386 input features, SVGP fails severely since the input kernel cannot capture the implicit correlation information.
ANP is unstable on the airfoil dataset because it scales poorly with dataset size.
Figure 3 compares NN, RIO
[Figure]
and SVGP in terms of prediction RMSE.
RIO is able to improve the NN predictions consistently regardless of how the dataset is split and how well the NN is trained.
Even in the situations where original NN is much worse than SVGP, RIO still successfully improves the NN performance into a level that is comparable or better than SVGP.
Appendix D.3 provides additional investigation into the behavior of RIO output corrections.
RIO exhibits diverse behaviors that generally move the original
NN predictions closer to the ground truth.
To conclude, applying RIO to NNs not only provides additional uncertainty information, but also reliably reduces the point-prediction error.
Uncertainty Estimation Average negative log predictive density (NLPD) is used to measure the quality of uncertainty estimation, which favours conservative models, but also effectively penalizes both over- and under-conﬁdent predictions (Qui ˜ nonero-Candela et al., 2006).
The mean and standard deviation of NLPDs over multiple experimental runs are shown in Table 1 (lower is better).
RIO performs the best or equals the best method (based on statistical tests) in 8 out of 12 datasets.
NNGP always yields a high NLPD; it returns over-conﬁdent predictions, because it does not include noise estimation in its original implementation.
For the yacht dataset, ANP achieves the best NLPD, but with high RMSE.
This is because ANP is able to correctly return high predictive variance when its prediction error is high.
For all other tested datasets, RIO variants consistently outperform ANP.
Among all RIO variants, the full RIO provides the most reliable overall predictive uncertainty.
The conclusion is that RIO successfully extracts useful uncertainty information from NN predictions.
Appendix D.2 evaluates uncertainty estimates on an additional metric: the true coverage probabilities of estimated conﬁdence intervals.
Although RIO also provides reasonable estimates for this metric in most cases, comparison among algorithms using this metric is discouraged due to its unreliability (see Appendix D.2 for examples).
Computation Time Table 1 shows the average wall clock time of each algorithm.
All algorithms are implemented using Tensorﬂow under the same running environment (see Appendix D.1 for implementation details).
The RIO variants scale well to increasing dataset sizes and feature dimensionalities.
L-BFGS-B converges especially quickly for R+I on the three highest dimensional datasets, presumably because the residuals are very well-behaved compared to the raw targets or NN output.
In contrast, ANP’s computation time increases signiﬁcantly with the scale of the dataset, and NNGP always needs very expensive computational budgets due to its costly grid search of hyperparameters.
Ablation Study The RIO variants with residual estimation generally perform better than its counterparts in both point-prediction error and uncertainty estimation.
This result conﬁrms the effectiveness of residual estimation, as suggested in Section 5.1.
Another important result is that Y+IO outperforms both Y+I (SVGP) and Y+O in most cases across all performance metrics, and RIO generally provides better performance than R+I and R+O in all respects.
This result, in turn, conﬁrms that the I/O kernel provides additional robustness, as suggested by the analysis in Section 5.2.
In sum, both residual estimation and the I/O kernel contribute substantially to the performance of the framework.
[Table]
Correlation between error and noise variance Table 2 shows the results of Spearman’s Rank
Correlation between RMSE and noise variance σ 2 n .
For each dataset, there are six pairs of data points, each of which contains the mean of RMSE and noise variance of a RIO variant.
A correlation value larger than 0.8 and p -value less than 0.05 indicate a very strong positive monotonic correlation.
For 10 out of 12 datasets, very strong positive monotonic correlation between RMSE and noise variance was observed.
This empirical result is in accordance with the theoretical prediction in Section 5.1.
Application to a large-scale vision problem To show RIO’s off-the-shelf applicability to modern deep convolutional architectures, it was applied to a recent pretrained NN for age estimation (Yang et al., 2018).
The pretrained NN and all data preprocessing were taken exactly from the ofﬁcial code release.
The model is a variant of DenseNet-121 (Huang et al., 2017), and uses the IMDB age estimation dataset, which contains ≈ 172K RGB images (Rothe et al., 2016).
The goal is to predict the age of the individual in each image.
The features for the GP input kernel were simply a global max pool of the CNN’s ﬁrst stage output.
From Table 3, RIO substantially improves upon the mean absolute error (MAE) of the pretrained NN, outperforms SVGP in terms of both MAE and NLPD, and yields realistic conﬁdence intervals (CIs).
SVGP effectively learns nothing, so it estimates almost all variance as noise, while RIO effectively augments the pretrained model.
[Table]
A NALYSIS OF RIO
This section presents an initial theoretical investigation into what makes RIO work, including why it is useful to ﬁt residuals, and why it is beneﬁcial to use an I/O kernel.
5. B ENEFITS OF FITTING NEURAL NETWORK RESIDUALS
Given a pretrained NN, this section provides a theoretical perspective on how ﬁtting its residuals with a GP can yield useful uncertainty information, while leading to prediction error lower than the NN or
GP alone.
The idea is that, due to its high expressivity, a pretrained NN may have learned complex structure that a GP would model poorly, when selecting a kernel that would capture this complexity is infeasible.
Fitting a GP to the residuals of the NN is easier, since this complex structure has been removed, and takes advantage of the predictive power of the NN, while providing useful uncertainty estimates.
Given a pretrained NN, when is ﬁtting its residuals with a GP a good idea?
To produce a model with uncertainty information, one could simply discard the NN and train a GP directly on D .
However, for problems with enough complexity, GP model selection (i.e., specifying an appropriate class of kernels) is challenging.
Fortunately, the pretrained NN may have learned a useful representation of this complexity, which can then be exploited.
Suppose instead that a GP tries to model this complexity, and this attempt leads to worse generalization.
This poor performance could come from many dimensions of the kernel selection process.
As an initial theoretical investigation, an ideal case is considered ﬁrst: suppose the GP optimizer optimally avoids modeling this complexity incorrectly by estimating the variance of such complexity as noise.
Such an estimate leads to the following decomposition:
[Equation]
where h ( · ) is the true signal, f ( · ) is the apparent signal of the GP, g ( · ) is its apparent (spurious) noise, and ξ i ∼ N (0 , σ 2 n ) is real noise.
Intuitively, due to its high expressivity, it is possible for a pretrained
NN ¯ h NN to correctly model part of g , along with part of f , resulting in the residuals
[Equation]
where r f ( · ) is now the apparent signal of the GP, and r g ( · ) is now its apparent noise.
In such a case, it will be easier for the GP to learn the the residuals than to learn h directly.
The uncertainty estimates of the GP will also be more precise, since the NN removes spurious noise.
Thus, Eq. 8 immediately predicts that the noise estimates for GP will be higher when ﬁtting h directly than when ﬁtting NN residuals.
This prediction is conﬁrmed in experiments (Table 1).
To analyze these behaviors, the above decomposition is used to identify a class of scenarios for which ﬁtting residuals leads to improvements (see Appendix A for additional details and proofs of following theorems).
For simplicity, suppose GP kernels are parameterized only by their signal variance β , i.e., kernels are of the form βk ( · , · ) for some kernel k .
Then, the following two assumptions make Eq.concrete: (1) GP is well-suited for f , i.e., f ( · ) ∼ GP (0 , k ( · , · )) ; (2) GP is ill-suited for g , i.e., g is independent of f and ε -indistinguishable for GP on D (see Lemma A.2 for an example of such a function).
Intuitively, this means the predictions of GP change by no more than ε in the presence of g .
Consider the residuals y
i
[Equation]
( x i ) + ξ i , where r f is the remaining
GP component, and r g is the remainder of g .
The following assumption ensures that r g captures any spurious complexity added by an imperfectly trained NN: r f ∼ GP (0 , αk ( · , · )) , for some α ∈ (0 , 1] .
Let ¯ h GP be the GP predictor trained directly on D , and ¯ h GP+NN = ¯ h NN + ¯ r GP be the ﬁnal predictor after ﬁtting residuals; let E h GP , E h NN , and E h GP+NN be the expected generalization errors of ¯ h GP , ¯ h NN , and ¯ h GP+NN , resp.
The advantage of ¯ h GP+NN becomes clear as g becomes complex:
Theorem 5.1 (Advantage of ﬁtting residuals) .
[Equation]
The experimental results in Section 4 support this result (Table 1).
First, in all experiments, GP+NN does indeed improve over the underlying NN.
Second, notice that the improvement of GP+NN over
GP is only guaranteed when E [ g 2 ( x )] − E [ r 2 g ( x )] is positive, i.e., when the NN successfully captures some underlying complexity.
In experiments, when the NN is deﬁcient, GP+NN often outperforms
GP, but not always.
For example, if the NN is overﬁt to zero training loss, and uses the same training data as the GP, the GP cannot provide beneﬁt.
However, when the NN is well-trained, using proper overﬁtting prevention, the improvements are consistent and signiﬁcant.
The prediction of Theorem 5.1 that lower apparent noise corresponds to improved error is conﬁrmed in experiments (Table 2).
Note that an optimal model would learn h exactly, and have a predictive variance of σ 2 n .
So, the improvement in error of ¯ h GP+NN over ¯ h GP also corresponds to a predictive variance that is closer to that of the optimal model, since some spurious noise has been removed by ¯ h NN .
Complementing this theoretical improvement in uncertainty estimation, the above setting also leads to a key practical property, which is illustrated in Figure 2:
Theorem 5.2.
The uncertainty of ¯ r GP is positively correlated with the variance of NN residuals.
This property matches the intuition that the GP’s variance should generally be higher for unstable
NNs, i.e., NNs with high residual variance, than for stable NNs.
Such a property is crucial to measuring the conﬁdence of NN predictions in practice.
Overall, the initial theoretical results in this section (and the appendix) show that if a problem is well-suited to be learned by an NN, and a practitioner has access to a pretrained NN for that problem, then training a GP to ﬁt the residuals of the NN can provide potential beneﬁts in uncertainty estimates, without sacriﬁcing prediction performance.
Further theoretical investigation regarding more complicated situations, e.g., misspeciﬁcation of GP kernels, is considered as one interesting direction for future work.
5. R OBUSTNESS OF THE I/O K ERNEL
This section provides a justiﬁcation for why a GP using the proposed I/O kernel is more robust than the standard GP, i.e., using the input kernel alone.
The key assumption is that the output of an NN can contain valuable information about its behavior, and, consequently, the structure of the target function.
Consider the setup in Section 5.1, but now with y i = f in ( x i ) + f out ( x i ) + ξ i , where f in ∼ GP (0 , k in ) and f out ∼ GP (0 , k out ) , with non-trivial RBF kernels k in , k out (as in Equation 3).
Let ¯ h NN be an NN of sufﬁcient complexity to be nontrivially non-afﬁne, in that there exists a positive-measure set of triples ( x , x (cid:48) , x (cid:48)(cid:48) )
s.t. (cid:107) x − x (cid:48) (cid:107) =
(cid:107) x − x (cid:48)(cid:48) (cid:107) , but (cid:107) ¯ h NN ( x ) − ¯ h NN ( x (cid:48) )
(cid:107) (cid:54) =
(cid:107) ¯ h NN ( x ) − ¯ h NN
( x (cid:48)(cid:48) )
(cid:107) .
Denote the generalization errors of the standard GP, GP with output kernel only, and GP with I/O kernel by E h
I , E h O , and E h
I / O , respectively.
The expected result follows (proof in Appendix A).
Theorem 5.3 (Advantage of I/O kernel) .
E h
I
O < E h
I and E h
I /
O < E h O .
The optimizer associated with the GP simultaneously optimizes the hyperparameters of both kernels, so the less useful kernel usually receives a smaller signal variance.
As a result, the I/O kernel is resilient to failures of either kernel.
In particular, the GP using I/
O kernel improves performance even in the case where the problem is so complex that Euclidean distance in the input space provides no useful correlation information or when the input space contains some noisy features.
Conversely, when the NN is a bad predictor, and h NN is no better noise, the standard GP with input kernel alone is recovered.
In other words, the I/O kernel is never worse than using the input kernel alone, and in practice it is often better.
This conclusion is conﬁrmed in the experiments in Section 4.
D ISCUSSION AND F UTURE D IRECTIONS
In addition to the reliable uncertainty estimation, accurate point prediction, and good scalability, demonstrated in Section 4, RIO provides other important beneﬁts.
RIO can be directly applied to any standard NN without modiﬁcation to the model architecture or training pipeline.
Moreover, retraining of the NN or change of inference process are not required.
The framework simply requires the outputs of an NN; it does not need to access any internal structure.
This feature makes the framework more accessible to practitioners in real-world applications, e.g., data scientists can train NNs using traditional pipelines, then directly apply RIO to the trained NNs.
RIO also provides robustness to a type of adversarial attack.
Consider a worst-case scenario, in which an adversary can arbitrarily alter the output of the NN with minuscule changes to the input.
It is well-known that there are NNs for which this is possible (Goodfellow et al., 2015).
In this case, with the help of the I/O kernel, the model becomes highly uncertain with respect to the output kernel.
A conﬁdent prediction then requires both input and output to be reasonable.
In the real world, a high degree of uncertainty may meet a threshold for disqualifying the prediction as outside the scope of the model’s ability.
There are several promising future directions for extending RIO:
First, applying RIO to reinforcement learning (RL) algorithms, which usually use standard NNs for reward predictions, would allow uncertainty estimation of the future rewards.
Agents can then directly employ efﬁcient exploration strategies, e.g., bandit algorithms (Thompson, 1933), rather than traditional stochastic approaches like ε -greedy.
Second, RIO applied to Bayesian optimization (BO) (Moˇckus, 1975) would make it possible to use standard NNs in surrogate modeling.
This approach can potentially improve the expressivity of the surrogate model and the scalability of BO.
Third, since RIO only requires access to the inputs and outputs of NNs, it could be directly applied to any existing prediction models, including hybrid and ensemble models.
For example, experiments in Appendix D.6 show that RIO achieves good results when directly applied to random forests.
This general applicability makes RIO a more general tool for real-world practitioners.
C ONCLUSION
The RIO framework both provides estimates of predictive uncertainty of neural networks, and reduces their point-prediction errors.
The approach captures NN behavior by estimating their residuals with an I/O kernel.
RIO is theoretically-grounded, performs well on several real-world problems, and, by using a sparse GP approximation, scales well to large datasets.
Remarkably, it can be applied directly to any pretrained NNs without modiﬁcations to model architecture or training pipeline.
Thus, RIO can be used to make NN regression practical and powerful in many real-world applications.
A A DDITIONAL D ETAILS AND P ROOFS FOR S ECTIONS 5.1 AND 5.
Section 5.1 provides a theoretical perspective on how using GP to ﬁt NN residuals can provide advantages over GP or NN alone.
The idea is to decompose the target function into two independent parts: one that is well-suited for a GP to learn, and the other that is difﬁcult for it to capture.
The following deﬁnition provides a formalization of this notion of difﬁculty.
Deﬁnition A.1 ( ε -indistinguishable) .
Let ¯ f A be the predictor of a learner A trained on { ( x i , y i ) } ni =1 , and ¯ h A that of A trained on D = { ( x i , y i + g ( x i ))
} n i =1 .
Then, if | ¯ f A ( x ) − ¯ h A ( x ) | < ε ∀ x , we say g is ε -indistinguishable for A on D .
In other words, the predictions of A change by no more than ε in the presence of g .
If ε is small compared to E [ g 2 ( x )] , then g contains substantial complexity that A cannot capture.
For notation, let ¯ ψ A denote the predictor of a learner
A trained on { ( x i , ψ ( x i ) + ξ i ) } ni =1 , with ξ i ∼ N (0 , σ 2 n ) .
Let E ψA =
E [( ψ ( x ) − ¯ ψ A ( x )) 2 ] denote the expected generalization error of A on ψ given training locations { x i } ni =1 .
Now, consider a dataset D
= { ( x i , y i ) } ni =1 with y
i
= h ( x i ) + ξ i = f
( x i )
+ g ( x i )
+ ξ i , where f ( · ) ∼ GP (0 , k ( · , · )) , ξ i ∼ N (0 , σ 2 n ) , and E [ g 2 ( x )]
= σ 2 g .
For simplicity, assume the
GP learner selects optimal hyperparameters for k and σ 2 n .
Suppose g is independent of f and εindistinguishable for GP on D , but a neural network ¯ h NN may have successfully learned some of g ’s structure.
These assumptions induce a concrete class of examples of the decomposition in Eq. 8.
To demonstrate the existence of such decompositions, Lemma A.2 provides a constructive example of such a function g .
Lemma A.2 (Construction of independent ε -indistinguishable functions for GP) .
Given a dataset { ( x i , f ( x i ) + ξ i ) } ni =1 , for any σ 2 g and ε > 0 , there exists g with E [ g 2 ( x )]
= σ 2 g , and { x i } 2 ni = n +1 , s.t.
g is ε -indistinguishable on { ( x i , f ( x i )
+ g ( x i )
+ ξ i ) } 2 ni =1 for GP with a continuous kernel k .
Proof.
Let x n + i
= x i + v ∀ i = 1 , . . .
, n , for a ﬁxed vector v .
Consider the linear smoother view of GP prediction (Rasmussen & Williams, 2006): ¯ f GP ( x ∗ ) =
w ( x ∗ ) (cid:62) y , where w ( x ∗ )
=
( K + σ 2 n I ) − 1 k ( x ∗ ) is the weight function (Silverman et al., 1984), and y is the target vector.
Since k and w ( x ∗ ) are continuous around each x
i , we can choose v such that ∀ x ∗
[Equation]
Let ¯ f GP be the predictor of GP trained on { ( x i , f ( x i ) + ξ i ) } 2 ni =1 , and ¯ h GP the predictor trained on { ( x i , f ( x i )
+ g ( x i )
+ ξ i ) } 2 ni
=1 = D .
Now, choose g so that
[Equation]
Then,
[Equation] [Equation]
Consider the residuals y i − ¯ h NN ( x i )
=
r ( x i ) + ξ i =
r f ( x i ) + r g ( x i ) + ξ i .
Here, r f is the remaining
GP component, i.e., r f ∼ GP (0 , αk ( · , · )) , for 0 < α ≤ 1 .
Similarly, r g is the remainder of g ε -indistinguishable for GP on { ( x i , r ( x i )+ ξ i ) } ni =1 , i.e., σ 2 g − E [ r 2 g ( x )]
= δ .
These two assumptions about r f and r g ensure that any new spurious complexity added by the NN is collected in r g , since the biggest advantage of the NN (its expressivity) is also its greatest source of risk.
The ﬁnal predictor after ﬁtting residuals is then ¯ h GP+NN = ¯ h NN + ¯ r GP .
The following sequence of results captures the improvement due to residual estimation.
Lemma A.3 (Generalization of GP on h ) .
[Equation]
Proof.
Let ∆ ¯ f GP ( x ) =
¯ h GP ( x ) − ¯ f GP ( x ) denote the change in GP prediction due to g .
Then,
[Equation]
where the last line makes use of the fact that f and g are independent.
Now,
[Equation]
and
E
[Equation]
, where (cid:12)(cid:12)
[Equation]
So, E
[Equation]
Lemma A.4 (Generalization of NN) .
E h NN = α E [ f 2 ( x )]
+ σ 2 g − δ.
Proof.
Making use of the fact that E [ r f ] = 0 ,
[Equation]
Now, r f ∼ GP (0 , αk ) =
⇒ E
[ r 2 f ( x )]
= α E [ f 2 ( x )] , and σ 2 g − E
[ r 2 g ( x )]
= δ = ⇒ E
[ r 2 g ( x )]
= σ 2 g − δ .
So, E h
NN = α E [ f 2 ( x )]
+ σ 2 g − δ.
Lemma A.5 (Generalization of GP ﬁtting residuals) .
[Equation]
Proof.
Let ∆¯ r f GP ( x ) =
¯ r GP ( x ) − ¯ r f GP ( x ) denote the change in GP prediction due to r g .
Then,
E
[Equation] [Equation]
( x
[Equation]
h GP+NN ( x )) 2 ] =
E [( h ( x ) − ¯ h NN ( x ) − ¯ r GP ( x )) 2 ] =
E [( r f ( x ) + r g ( x )) − (¯ r f GP ( x ) + ∆¯ r f GP ( x ))) 2 ] = E [( r f ( x ) − ¯ r f GP ( x )) + ( r g ( x ) − ∆¯ r f GP ( x ))) 2 ] =
E
[Equation]
GP
[Equation]
E [( r f ( x ) − ¯ r GP ( x ))
( r g ( x ) − ∆¯ r f GP ( x ))]
+ E [( r g ( x ) − ∆¯ r f GP ( x )) 2 ] =
E r f GP − 2 E
[( r f ( x ) − ¯ r GP ( x )∆¯ r f GP ( x )] + E [( r g ( x ) − ∆¯ r f GP ( x )) 2 ] .
Similar to the case of Lemma A.3,
[Equation]
f GP ( x ))] (cid:12)(cid:12) <
2 ε ( E r f GP ) 1 2 , and
[Equation]
where
[Equation]
So,
[Equation]
Lemma A.6 (Generalization of GP on f and r f ) .
From the classic result (Opper & Vivarelli, 1999;
Sollich, 1999; Rasmussen & Williams, 2006): Consider the eigenfunction expansion k ( x , x (cid:48) )
= (cid:80)
j λ j φ j ( x ) φ j ( x (cid:48) ) and (cid:82) k ( x , x (cid:48) )
φ i ( x ) p ( x ) d x = λ
i φ i ( x (cid:48) ) .
Let Λ be the diagonal matrix of the eigenvalues λ j
[Equation]
and Φ be the design matrix,
i
[Equation]
e., Φ ji = φ j ( x i ) .
Then,
[Equation]
Theorem
[Equation]
Proof.
From Lemmas A.3, A.4, and A.5 we have, resp.:
[Equation]
, and
[Equation]
From Lemma A.6, we have
[Equation]
and
[Equation]
So,
[Equation]
and
[Equation]
Theorem 5.2.
The variance of NN residuals is positively correlated with the uncertainty of r GP .
Proof.
Increases in E [ r 2 f ( x )] lead to increases in α ; increases in E [ r 2 g ( x )] lead to decreases in δ , and thus increases in the estimated noise level ˆ σ 2 n .
So, an increase in either E [ r 2 f ( x )] or E [ r 2 g ( x )] leads to an increase in αk (( x ∗ , ˆ y ∗ ) , ( x ∗ , ˆ y ∗ ))
− α k
(cid:62)∗ ( α K (( X , ˆy ) , ( X , ˆy ))
+ ˆ σ 2 n I ) − 1 α k ∗ , which is the predictive variance of r GP .
Theorem 5.3.
E h
I /
O < E h
I and E h
I /
O < E h O .
Proof.
[Equation]
These are true for all hyperparameter settings of k in or k out , since both are RBF kernels.
So, there is no hyperparameter setting of k in that yields k (cid:48) in ( x 1 , x 2 )
=
k in ( x 1 , x 2 ) + k out ( x 1 , x 2 ) ∀ x 1 , x 2 .
Similarly, there is no hyperparameter setting of k out that yields k (cid:48) out ( x 1 , x 2 )
=
k in ( x 1 , x 2 ) + k out ( x 1 , x 2 ) ∀ x 1 , x 2 .
Since, neither the input nor output kernel alone can correctly specify the kernel over all sets of positive measure, their generalization error is greater than the Bayes error, which is achieved by the correct kernel (Sollich, 2002), i.e., k in ( x 1 , x 2 ) + k out ( x 1 , x 2 ) .
B B ACKGROUND
This section reviews notation for Neural Networks, Gaussian Process, and its more efﬁcient approximation, SVGP.
The RIO method, introduced in Section 3 of the main paper, uses Gaussian Processes to estimate the uncertainty in neural network predictions and reduces their point-prediction errors.
B. N EURAL N ETWORKS
Neural Networks (NNs) learn a nonlinear transformation from input to output space based on a number of training examples.
Let D ⊆ R d in × R d out denote the training dataset with size n , and X = { x i : ( x i , y i ) ∈ D , x i =
[ x 1 i , x 2 i , . . .
, x d in i ] | i = 1 , 2 , . . . , n } and Y = { y i : ( x i , y i ) ∈ D , y i =
[ y 1 i , y 2 i , . .
, y d out i ] | i = 1 , 2 , . . . , n } denote the inputs and outputs (i.e., targets).
A fully-connected feed-forward neural network with L hidden layers of width N l (for layer l = 1 , 2 , . . .
, L ) performs the following computations: Let z jl denote the output value of j th node in l th hidden layer given input x i , then z jl = φ
( (cid:80) d in k =1 w j,kl x ki + b jl ) , for l = 1 and z jl = φ ( (cid:80) N l − 1 k =1 w j,kl z kl − 1 + b jl ) , for l = 2 , . . .
, L , where w j,kl denotes the weight on the connection from k th node in previous layer to j th node in l th hidden layer, b jl denotes the bias of j th node in l th hidden layer, and φ is a nonlinear activation function.
The output value of j th node in output layer is then given by ˆ y ji
= (cid:80) N L k =1 w j,k out z kL + b j out , where w j,k out denotes the weight on the connection from k th node in last hidden layer to j th node in output layer, and b j out denotes the bias of j th node in output layer.
A gradient-based optimizer is usually used to learn the weights and bias given a pre-deﬁned loss function, e.g., a squared loss function L = 1 n (cid:80) ni
=
( y i − ˆy i ) 2 .
For a standard NN, the learned parameters are ﬁxed, so the NN output ˆy i is also a ﬁxed point.
For a Bayesian NN, a distribution of the parameters is learned, so the NN output is a distribution of ˆy i .
However, a pretrained standard
NN needs to be augmented, e.g., with a Gaussian Process, to achieve the same result.
B. G AUSSIAN P ROCESS
A Gaussian Process (GP) is a collection of random variables, such that any ﬁnite collection of these variables follows a joint multivariate Gaussian distribution (Rasmussen & Williams, 2006).
Given a training dataset X = { x i | i = 1 , 2 , . . . , n } and y =
{ y i = f
( x i ) + (cid:15) | i = 1 , 2 , . . . , n } , where (cid:15) denotes additive independent identically distributed Gaussian noise, the ﬁrst step for GP is to ﬁt itself to these training data assuming y ∼ N (0 , K ( X , X ) + σ 2 n I ) , where N denotes a multivariate
Gaussian distribution with mean 0 and covariance matrix K ( X , X ) + σ 2 n I .
K ( X , X ) denotes the kernel-based covariance matrix at all pairs of training points with each entry k
i,j = k ( x i , x j ) , and σ 2 n denotes the noise variance of observations.
One commonly used kernel is the radial basis function (RBF) kernel, which is deﬁned as k ( x i , x j ) =
σ 2 f exp( − 12 l 2 f (cid:107) x
i − x j (cid:107) 2 ) .
The signal variance σ 2 f , length scale l f and noise variance σ 2 n are trainable hyperparameters.
The hyperparameters of the covariance function are optimized during the learning process to maximize the log marginal likelihood log p ( y |X ) .
After ﬁtting phase, the GP is utilized to predict the distribution of label y ∗ given a test point x ∗ .
This prediction is given by y ∗ |X , y , x ∗ ∼ N (¯ y ∗ , var( y ∗ )) with ¯ y ∗ =
k (cid:62)∗ ( K ( X , X ) + σ 2 n I ) − 1 y and var( y ∗ )
=
k ( x ∗ , x ∗ ) − k (cid:62) ∗ ( K ( X , X ) + σ 2 n I ) − 1 k ∗ , where k ∗ denotes the vector of kernel-based covariances (i.e., k ( x ∗ , x i ) ) between x ∗ and all the training points, and y denotes the vector of all training labels.
Unlike with NN, the uncertainty of the prediction of a GP is therefore explicitly quantiﬁed.
B. SVGP
The main limitation of the standard GP, as deﬁned above, is that it is excessively expensive in both computational and storage cost.
For a dataset with n data points, the inference of standard GP has time complexity O ( n 3 ) and space complexity O ( n 2 ) .
To circumvent this issue, sparse GP methods were developed to approximate the original GP by introducing inducing variables (Csat´o & Opper, 2002;
Seeger et al., 2003; Qui˜nonero Candela & Rasmussen, 2005; Titsias, 2009).
These approximation approaches lead to a computational complexity of O ( nm 2 ) and space complexity of O ( nm ) , where m is the number of inducing variables.
Following this line of work, SVGP (Hensman et al., 2013; 2015) further improves the scalability of the approach by applying Stochastic Variational Inference (SVI) technique, as follows:
Consider the same training dataset and GP as in section B.2, and assume a set of inducing variables as
Z = { z i | i = 1 , 2 , . . .
, m } and U = { u i =
f
( z i ) + (cid:15) | i = 1 , 2 , . . .
, m } ( f ( · ) and (cid:15) are unknown).
Algorithm
Require: ( X , y )
= { ( x i , y i ) } ni =1 : training data ˆy = { ˆ y i } ni =1 : NN predictions on training data x ∗ : data to be predicted ˆ y ∗ : NN prediction on x ∗
Ensure: ˆ y (cid:48)∗ ∼ N (ˆ y ∗ + ¯ˆ r ∗ , var(ˆ r ∗ )) : a distribution of calibrated prediction
Training Phase: 1: calculate residuals r = { r i
=
y
i − ˆ y i } ni = 2: for each optimizer step do 3: calculate covariance matrix K c (( X , ˆy ) , ( X , ˆy )) , where each entry is given by k c (( x i , ˆ y i ) , ( x j , ˆ y j ))
=
k in ( x i , x j ) + k out (ˆ y i , ˆ y j ) , for i, j = 1 , 2 , . . . , n 4: optimize GP hyperparameters by maximizing log marginal likelihood log p ( r |X , ˆy )
= − 12 r (cid:62) ( K c (( X , ˆy ) , ( X , ˆy ))
+ σ 2 n I ) r − 12 log | K c (( X , ˆy ) , ( X , ˆy ))
+ σ 2 n I | − n 2 log 2 π
Deployment Phase: 5: calculate residual mean ¯ˆ r ∗ =
k (cid:62)∗ ( K c (( X , ˆy ) , ( X , ˆy ))
+ σ 2 n I ) − 1 r and residual variance var(ˆ r ∗ ) =
k c (( x ∗ , ˆ y ∗ ) , ( x ∗ , ˆ y ∗ ))
− k (cid:62)∗ ( K c (( X , ˆy ) , ( X , ˆy ))
+ σ 2 n I ) − 1 k ∗ 6: return distribution of calibrated prediction ˆ y (cid:48)∗ ∼ N (ˆ y ∗ + ¯ˆ r ∗ , var(ˆ r ∗ ))
SVGP learns a variational distribution q ( U ) by maximizing a lower bound of log p ( y |X ) , where log p ( y |X )
=
log (cid:82) p ( y |U , X ) p
( U )
d U and p ( · ) denotes the probability density under original GP.
Trainable hyperparameters during the learning process include values of z i and hyperparameters of the covariance function of original GP.
Given a test point x ∗ , the predictive distribution is then given by p ( y ∗ | x ∗ ) = (cid:82) p ( y ∗ |U , x ∗ ) q ( U )d U , which still follows a Gaussian distribution.
One advantage of SVGP is that minibatch training methods (Le et al., 2011) can be applied in case of very large dataset.
Suppose the minibatch size is m (cid:48) and m (cid:28) m (cid:48) , then for each training step/iteration, the computational complexity is O ( m (cid:48) m 2 ) , and the space complexity is O ( m (cid:48) m ) .
For full details about
SVGP, see Hensman et al.
Since NNs typically are based on training with relatively large datasets, SVGP makes it practical to implement uncertainty estimates on NNs.
C P ROCEDURE OF RIO
This section provides an algorithmic description of RIO (see Algorithm 1).
D E MPIRICAL S TUDY D. E XPERIMENTAL S ETUPS
Dataset Description
In total, 12 real-world regression datasets from UCI machine learning repository (Dua & Graff, 2017) are tested.
Table 4 summarizes the basic information of these datasets.
For all the datasets except MSD, 20% of the whole dataset is used as test dataset and 80% is used as training dataset, and this split is randomly generated in each independent run.
For MSD, the ﬁrst 463715 samples are used as training dataset and the last 51630 samples are used as testing dataset according to the provider’s guideline.
During the experiments, all the datasets except for MSD are tested for 100 independent runs, and MSD datasets are tested for 10 independent runs.
For each independent run, the dataset is randomly split into training set, validation set, and test set (except for MSD, in which the dataset split is strictly predeﬁned by the provider), and the same random dataset split are used by all the tested algorithms to ensure fair comparisons.
All source codes for reproducing the experimental results are provided at: (https://github.com/leaf-ai/rio-paper).
Parametric Setup for Algorithms
• NN: For SC dataset, a fully connected feed-forward NN with 2 hidden layers, each with 128 hidden neurons, is used.
For CT dataset, a fully connected feed-forward NN withhidden layers, each with 256 hidden neurons, is used.
For MSD dataset, a fully connected
[Table]
feed-forward NN with 4 hidden layers, each with 64 hidden neurons, is used.
For all the remaining datasets, a fully connected feed-forward NN with 2 hidden layers, each withhidden neurons, is used.
The inputs to the NN are normalized to have mean 0 and standard deviation 1.
The activation function is ReLU for all the hidden layers.
The maximum number of epochs for training is 1000.
20% of the training data is used as validation data, and the split is random at each independent run.
An early stop is triggered if the loss on validation data has not be improved for 10 epochs.
The optimizer is RMSprop with learning rate 0.001, and the loss function is mean squared error (MSE).
• RIO, RIO variants and SVGP (Hensman et al., 2013): SVGP is used as an approximator to original GP in RIO and all the RIO variants.
For RIO, RIO variants and SVGP, the number of inducing points are 50 for all the experiments.
RBF kernel is used for both input and output kernel.
For RIO, RIO variants and SVGP, the signal variances and length scales of all the kernels plus the noise variance are the trainable hyperparameters.
The optimizer is L-BFGS-B with default parameters as in Scipy.optimize documentation (https://docs.scipy.org/doc/scipy/reference/optimize.minimize-lbfgsb.html), and the maximum number of iterations is set as 1000.
The training process runs until the L-BFGS-B optimizer decides to stop.
• NNGP (Lee et al., 2018):
For NNGP kernel, the depth is 2, and the activation function is ReLU.
n g = 101 , n v = 151 , and n c = 131 .
Following the learning process in original paper, a grid search is performed to search for the best values of σ 2 w and σ 2 b .
Same as in the original paper, a grid of 30 points evenly spaced from 0.1 to 5.0 (for σ 2 w ) and 30 points evenly spaced from 0 to 2.0 (for σ 2 b ) was evaluated.
The noise variance σ 2 (cid:15) is ﬁxed as 0.01.
The grid search process stops when Cholesky decomposition fails or all the 900 points are evaluated.
The best values found during the grid search will be used in the experiments.
No pre-computed lookup tables are utilized.
• ANP (Kim et al., 2019):
The parametric setups of ANP are following the recommendations in the original paper.
The attention type is multihead, the hidden size is 64, the max number of context points is 50, the context ratio is 0.8, the random kernel hyperparameters option is on.
The size of latent encoder is 64 × 64 × 64 × 64 , the number of latents is 64, the size of deterministic encoder is 64 × 64 × 64 × 64 , the size of decoder is 64 × 64 × 64 × 64 × 2 , and the deterministic path option is on.
Adam optimizer with learning rate 10 − 4 is used, and the maximum number of training iterations is 2000.
Performance Metrics
• To measure the point-prediction error, the Root Mean Square Error (RMSE) between the method predictions and true outcomes on test datasets are calculated for each independent experimental run.
After that, the mean and standard deviations of these RMSEs are used to measure the performance of the algorithms.
• To quantitatively measure the quality of uncertainty estimation, average negative log predictive density (NLPD) (Qui ˜ nonero-Candela et al., 2006) is used to measure the quality of
uncertainty estimation.
NLPD is given by
[Equation]
where ˆy i indicates the prediction results, x i is the input with true associated outcome y i , p ( · ) is the probability density function (PDF) of the returned distribution based on input x i .
• To investigate the behaviors of RIO variants during learning, the mean of estimated noise variance σ 2 n over all the independent runs are calculated.
• To compare the computation time of the algorithms, the training time (wall clock time) of NN, RIO, all the ablated RIO variants, SVGP and ANP are averaged over all the independent runs as the computation time.
It is notable that the computation time of all RIO variants does not include the training time of associated NN, because the NN is considered to be pretrained.
For NNGP, the wall clock time for the grid search is used.
In case that the grid search stops due to Cholesky decomposition failures, the computation time of NNGP will be estimated as the average running time of all the successful evaluations × 900, which is the supposed number of evaluations.
All the algorithms are implemented using Tensorﬂow, and tested in the exactly same python environment.
All the experiments are running on a machine with 16 Intel(R)
Xeon(R) CPU E5-2623 v4@2.60GHz and 128GB memory.
D. R ESULTS ON E STIMATED C ONFIDENCE I NTERVALS
Conﬁdence interval (CI) is a useful tool to estimate the distribution of outcomes with explicit probabilities in real-world applications.
In order to provide more insights for practitioners, the percentages of testing outcomes that are within the 95%/90%/68% CIs as estimated by each algorithm are calculated.
Ideally, these percentages should be as close to the estimated conﬁdence levels as possible, e.g., a perfect uncertainty estimator would have exactly 95% of testing outcomes within its estimated 95% CIs.
In practice, when two models have similar quality of CI estimations, the more conservative one is favoured.
Figure 4 and 5 show the distribution of the percentages that testing outcomes are within the estimated 95%/90%/68% CIs over all the independent runs for all the datasets and algorithms.
Figure 6, 7 and 8 compare the distributions of coverage percentages for 1%-99% CIs for RIO and SVGP.
The experimental data are extracted from the experiments that generate Table 1 in main paper.
In Figure 4, 5, 6, 7 and 8, the box extends from the lower to upper quartile values of the data (each data point represents an independent experimental run), with a line at the median.
extend from the to the
According to the results, RIO is able to provide reasonable CI estimations in most cases.
However, using this metric to compare algorithms regarding uncertainty estimation quality may be noisy and misleading.
A method that simply learns the distribution of the labels would perform well in CI coverage metric, but it cannot make any meaningful point-wise prediction.
More detailed examples are given as follows.
For “CT” dataset, SVGP has an extremely high RMSE of around 52 while RIO variants only have
RMSEs of 1.
However, SVGP still shows acceptable 95% and 90% CI coverage, and even has over-conﬁdent coverage for 68% CI.
After investigation, it was found that this happened only by chance, and was not due to the accurate CI estimations of SVGP.
Since SVGP is not able to extract any useful information from the high-dimensional input space, it simply treated all outcomes as noise.
As a result, SVGP shows a very large RMSE compared to other algorithms, and the mean of its predicted outcome distribution is always around 0.
Since SVGP treats everything as noise, the estimated noise variance is very high, and the estimated 95% CI based on this noise variance is overly high and covers all the test outcomes in most cases.
When the estimated 90% CI is evaluated, the large error in mean estimation and large error in noise variance estimation cancel mostly cancel each other out by chance, i.e., the estimated 90% CI is mistakenly shifted by erroneous mean
then the overly wide noise variance fortunately covers slightly more than 90% test outcomes.
The case of the estimated 68% CI is similar, but now the error in noise variance cannot fully cover the error in mean, so the coverage percentages are now below 68%, indicating over-conﬁdence.
One interesting observation is that SVGP tends to be more “conservative” for high conﬁdence levels ( > 90%), even in cases where they are “optimistic” for low conﬁdence levels.
After investigation,
[Figure]
n t a g e o f t e s t p o i n t s w
i
t
h
i
[Figure]
C
[Figure] [Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
p e r c e n t a g e o f t e s t p
o i n t s w
i
t
h
i
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
n
[Table]
f t
[Table]
t p
[Table]
i n
[Table]
t
[Table]
[Figure]
[Table]
[Figure] [Figure]
[Figure] [Figure]
this is because SVGP normally has an overly high noise variance estimation (also comes with a higher prediction RMSE in most cases), so it has a higher chance of covering more points when the increase in CI width (inﬂuenced by noise variance) surpasses the erroneous shift of mean (depending on prediction errors).
D. A DDITIONAL R ESULTS ON O UTPUT C ORRECTION
This section shows additional demonstrations of the detailed behaviors of RIO output correction.
The experimental data are extracted from the experiments that generate Table 1 in main paper.
Figure 9 plots the distributions of ground truth labels (outcomes), original NN predictions and predictions corrected after RIO for a randomly picked run for each dataset.
Figure 10 shows the pointwise comparisons between NN outputs and RIO-corrected outputs for the same experimental runs as in Figure 9.
Based on the results, it is clear that RIO tends to calibrate each NN prediction accordingly.
The distribution of outputs after RIO calibration may be a shift, or shrinkage, or expansion, or even more complex modiﬁcations of the original NN predictions, depending on how different are NN predictions from ground truths.
As a result, the distribution of RIO calibrated outputs are closer to the distribution of the ground truth.
One interesting behavior can be observed in Figure 9 for “protein” dataset: after applying RIO, the range of whiskers shrunk and the outliers disappeared, but the box (indicating 25 to 75 percentile of the data) expanded.
This behavior shows that RIO can customize its calibration to each point.
Another interesting behavior is that for “wine/r” dataset (see both Figureand 10), RIO shifts all the original NN outputs to lower values, which are closer to the distribution of the ground truth.
To better study the behaviors of RIO, a new performance metric is deﬁned, called improvement ratio (IR), which is the ratio between number of successful corrections (successfully reducing the prediction error) and total number of data points.
For each run on each dataset, this IR value is calculated, and the distribution of IR values over 100 independent runs (random dataset split except for MSD, random NN initialization and training) on each dataset is plotted in Figure 11.
According to the results, the IR values for RIO are above 0.5 in most cases.
For 7 datasets, IR values are above 0.
in all 100 independent runs.
For some runs in yacht, ENB/h, CT, and MSD, the IR values are above 0.8 or even above 0.9.
All these observations show that RIO is making meaningful corrections instead of random perturbations.
Results in Figure 11 also provides useful information for practitioners:
Although not all RIO calibrations improve the result, most of them do.
Same empirical analysis is also conducted for two RIO variants, namely R+I (predicting residuals with only input kernel) and R+O (predicting residuals with only output kernel).
Figure 12, 13 andshow results for R+I, and Figure 15, 16 and 17 show results for R+O. From the results, the output kernel is helpful in problems where input kernel does not work well (CT and MSD), and it also shows more robust performance in terms of improvement ratio (IR) in most datasets.
However it is still generally worse than full RIO.
More speciﬁcally, R+I shows an extremely low IR in CT dataset (see
Figure 14).
After investigation, this is because the input kernel itself is not able to learn anything from the complex high-dimensional input space, so it treats everything as noise.
As a result, it keeps the NN output unchanged during correction in most cases.
Applying output kernel instead solves the issue.
After comparing Figure 10, 13 and 16, it can be observed that the behaviors of RIO are either a mixture or selection between R+I and
R+O. This means RIO with I/O kernel is able to choose the best kernel among these two or combines both if needed.
D. E XPERIMENTAL R ESULTS WITH ARD
The experimental results shown in the main paper is based on the setup that all RIO variants are using standard RBF kernel without Automatic Relevance Determination (ARD).
To investigate the performance of RIO under more sophisticated setups, same experiments are run for all RIO variants with ARD mechanism turned on (all other experimental setups are identical with section D.1, except that the NN depth for MSD dataset is reduced from 4 hidden layers to 2, due to computation failure issue during Cholesky decomposition).
Table 5 shows the summarized experimental results.
From the results, RIO still performs the best or equals the best method (based on statistical tests) in 9 out of 12 datasets for both RMSE and NLPD metrics.
This clearly shows the effectiveness and robustness of RIO.
[Figure]
un d t r u t h
[Figure]
p r e d i c t i o n v a
[Figure]
r
[Figure]
d t r u t h
[Figure]
p r e d i c t i o n v
[Figure]
g r o un d t r u t h / p
r
e d i c t i o n v a l u
[Figure]
s
[Figure]
g r o un d t r u t h
[Figure]
p
[Figure]
i c t i o n v a l
[Figure]
r o
[Figure]
d t r u t h
[Figure]
p
[Figure]
e d i
[Figure]
t
[Figure] [Figure]
g r o un d t r u t h / p
r
e d i c t i o n v a l
[Table]
[Figure]
d t r u t h / p r e d i c t i o n v a l u e
[Figure]
g r o
[Figure]
d t r u t h / p r e d i c t i o n v a l u e s
[Figure]
g r o un d t r u t h / p
r
e d i c t i o n v a l u e s
[Figure]
[Table]
[Figure]
p e r c e n t a g e
o f s u cc e ss f u l i m p r
o
v e m e n t
D. E XPERIMENTAL R ESULTS WITH 10,000 M AXIMUM O PTIMIZER I ERATIONS
The experimental results shown in the main paper is based on the setup that all RIO variants are using
L-BFGS-B optimizer with a maximun number of iterations as 1,000.
To investigate the performance of RIO under larger computational budget, same experiments are run for all RIO variants with maximum number of optimizer iterations as 10,000 (all other experimental setups are identical with section D.1).
Table 6 shows the summarized experimental results for 8 smallest datasets.
According to the results, the rankings of the methods are very similar to those in Table 1 (in the main paper), and RIO still performs best in all metrics.
D. R ESULTS ON R ANDOM F ORESTS
In principle, RIO can be applied to any prediction model since it treats the pretrained model as a black box.
To demonstrate this extensibility, RIO is applied to another classical prediction model —
Random Forests (RF) (Ho, 1995; Breiman, 2001).
The same experimental setups as in section D.1 are used.
For RF, the number of estimators is set as 100 for all datasets.
To avoid overﬁtting, the minimum number of samples required to be at a leaf node is 10 for all datasets, and the max depth of the tree is 7 for MSD and 5 for all other datasets.
Table 7 shows the experimental results.
From Table 7, RIO performs the best or equals the best method (based on statistical tests) in 9 out of 12 datasets in terms of both RMSE and NLPD.
In addition, RIO signiﬁcantly improves the performance of original RF in 11 out of 12 datasets.
These results demonstrates the robustness and broad applicability of RIO.
[Figure]
o
[Figure]
d t r u t h
[Figure]
p r e d i c t i o n v a l
[Figure]
o
[Figure]
d t r u t h
[Figure]
r
[Figure]
d i c t i o n v
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
d t r u t h
[Figure]
p r
[Figure]
i
[Figure]
t i o n v a l
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table] [Figure] [Table] [Figure]
[Table]
[Figure] [Table] [Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
g r o un d t r u t h / p
r
e d i c t i o n v a
[Table] [Table]
[Figure]
[Table]
[Figure]
r o un d t r u t h / p
r
e d i c t i o n v a l u e s
[Figure]
R
[Figure] [Figure]
g r o un d t r u t h / p
r
e d i c t i o n v a l u e s
[Figure]
g r o
[Figure]
d t r u t h / p r e d i c t i o n v
[Figure]
[Figure]
p e r c e n t a g e
o f s u cc e ss f u l i m p r
o
v e m e n t
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
u t h
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
u
[Table]
[Figure]
[Table]
[Figure]
[Table]
n
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
t r u t h
[Table]
p
[Table]
n
[Table]
a l
[Table]
t r u t h p
[Table] [Table]
n
[Table]
[Figure]
r o un d t r u t h
[Figure]
p r e d i c t i o n v a l u e s
[Figure]
[Figure]
g r o un d t r u t h / p
r
e d i c t i o n v a l u e
[Figure]
g r o u n d t r u t h / p r
e d i c t i o n v
[Figure] [Figure]
p
[Figure]
r c e n t a g e
o f s u cc e ss f u l i m p r
o
v e m e n
[Figure] [Figure]
[Table]