Robust uncertainty estimates with out-of-distribution pseudo-inputs training
Year: 2021

Abstract:
Probabilistic models often use neural networks to control their predictive uncer1 tainty. However, when making out-of-distribution (OOD) predictions, the often2 uncontrollable extrapolation properties of neural networks yield poor uncertainty 3 predictions. Such models then don’t know what they don’t know, which directly lim4 its their robustness w.r.t unexpected inputs. To counter this, we propose to explicitly 5 train the uncertainty predictor where we are not given data to make it reliable. As 6 one cannot train without data, we provide mechanisms for generating pseudo-inputs 7 in informative low-density regions of the input space, and show how to leverage 8 these in a practical Bayesian framework that casts a prior distribution over the 9 model uncertainty. With a holistic evaluation, we demonstrate that this yields 10 robust and interpretable predictions of uncertainty while retaining state-of-the-art 11 performance on diverse tasks such as regression and generative modeling. 12

Probabilistic models often use neural networks to control their predictive uncer-1 tainty. However, when making out-of-distribution (OOD) predictions, the often-2 uncontrollable extrapolation properties of neural networks yield poor uncertainty3 predictions. Such models then don’t know what they don’t know, which directly lim-4 its their robustness w.r.t unexpected inputs. To counter this, we propose to explicitly5 train the uncertainty predictor where we are not given data to make it reliable. As6 one cannot train without data, we provide mechanisms for generating pseudo-inputs7 in informative low-density regions of the input space, and show how to leverage8 these in a practical Bayesian framework that casts a prior distribution over the9 model uncertainty. With a holistic evaluation, we demonstrate that this yields10 robust and interpretable predictions of uncertainty while retaining state-of-the-art11 performance on diverse tasks such as regression and generative modeling.12
1 Introduction13
Neural networks generally extrapolate arbitrarily [Xu et al., 2020], and high quality predictions are14 limited to regions of the input space where the networks have been trained. This is to be expected and15 is only problematic if the associated predictions are not accompanied with a well-calibrated measure16 of uncertainty. If a neural network is used for estimating such a measure of uncertainty, we, however,17 quickly run into trouble, as the reported uncertainty then exhibits arbitrary behaviour in regions with18 no training data. Alarmingly, these are exactly the regions where evaluating the uncertainty is most19 important to the safe deployment of machine learning models in real world applications [Amodei20 et al., 2016]. One potential solution is to avoid using directly the output of neural networks for21 predicting uncertainty, and let it emerge from another mechanism, e.g. an ensemble [Hansen and22 Salamon, 1990, Lakshminarayanan et al., 2017] or some notion of Monte Carlo [MacKay, 1992, Gal23 and Ghahramani, 2016]. Here we explore the alternative view that the networks should simply be24 trained where there is no data.25
Figure 1: Pseudo-inputs are generated out of distribution, and there we train towards a prior (grey density).
But can we train without data? The Bayesian formalism often26 does so implicitly: most conjugate priors can be seen as ad-27 ditional training data [Bishop, 2006], e.g. in Gaussian models,28 a mean prior N (µ0, 20) can be realised by additional training29 data of µ0 with 20 setting the amount of observations. Placing30 a prior over the output of a neural network can, thus, be inter-31 preted as additional training data. Unfortunately, this view is32 not practical as it implies additional data for all possible inputs33 to a neural network, resulting in infinite data. Our approach34 is simple: we locate regions of low data density in input space35 and implicitly place observations here in output space by min-36
Submitted to 35th Conference on Neural Information Processing Systems (NeurIPS 2021). Do not distribute.
imizing an appropriate KL divergence towards a prior (see Fig. 1). The result is a remarkably simple37 algorithm that drastically improves uncertainty estimates in both regression and generative modeling.38
1.1 Background and related work39
The predictive performance of machine learning models has drastically increased in the past decade,40 but the quality of the accompanying uncertainties have not followed. Uncertainties are reported as be-41 ing miscalibrated [Guo et al., 2017] and overconfident [Lakshminarayanan et al., 2017, Hendrycks and42 Gimpel, 2016]. Some models even see higher likelihoods of out-of-distribution than in-distribution43 data [Nalisnick et al., 2019, Nguyen et al., 2015, Louizos and Welling, 2017].44
Neural networks commonly output distributions which gives a notion of predictive uncertainty. Clas-45 sifiers trained with soft-max is an ever-present example of such. These predictions are generally ob-46 served to be overconfident [Lakshminarayanan et al., 2017, Hendrycks and Gimpel, 2016] and to carry47 little meaning outside the support of the training data [Skafte et al., 2019, Lee et al., 2017]. The latter48 is an artifact of the hard-to-control extrapolation that comes with neural networks [Xu et al., 2021].49 In general, since extrapolation is difficult to control, uncertainties predicted by neural networks will50 exhibit seemingly arbitrary behavior outside the support of the data, yielding untrustworthy results.51
Mean-variance networks for regression [Nix and Weigend, 1994] model the conditional target52 density as a normal p(y|x) = N y|µ(x), (x)2 with mean and variance predicted by neural53 networks. The reported predictive uncertainty is generally accurate in regions near training data,54 but otherwise unreliable [Hauberg, 2019]. To counter this, Arvanitidis et al. [2017] and Skafte et al.55 [2019] proposed variance network architectures to enforce a specified extrapolation value, but these56 heuristics tend to be difficult to tune, and lack principle. Mean-variance networks have seen a recent57 uptake within generative modeling, where they are applied as an encoder distribution in variational58 autoencders (VAEs) [Kingma and Welling, 2013, Rezende et al., 2014].59
Which uncertainty? A commonly called-upon dichotomy [Der Kiureghian and Ditlevsen, 2009]60 is that the uncertainty of a model’s prediction can be decomposed into the uncertainty of the model61 (epistemic) and of the data (aleatoric). The epistemic uncertainty can be lowered by increasing62 the amount of data, simplifying the model or otherwise reducing the complexity of the learning63 problem. The aleatoric uncertainty, on the other hand, is a property of the world, and cannot be64 changed; no prediction should ever be more certain than the uncertainty displayed by the associated65 data. Depending on the task at hand, we may be interested in different types of uncertainty: In active66 learning [Settles, 2012] and Bayesian optimization [Močkus, 1975] we request data for which we67 have high epistemic, but low aleatoric uncertainty to ensure maximal information gain; while for68 classification and regression we often just want to minimize the overall predictive uncertainty.69
Bayesian methods are often used to quantify uncertainty due to their explicit formulation of70 uncertainty. Gaussian processes (GPs) [Rasmussen and Williams, 2005] provide an elegant71 framework that provide state-of-the-art uncertainty estimates, but, alas, the corresponding mean72 predictions are often not up to the standards of neural networks. GPs are tightly linked to Bayesian73 neural networks (BNNs) [MacKay, 1992] that place a prior over the network weights and seek the74 corresponding posterior. Despite advances in variational approximations [Graves, 2011, Kingma75 and Welling, 2013, Blundell et al., 2015], expectation propagation [Hernández-Lobato and Adams,76 2015, Hasenclever et al., 2017], or Monte Carlo methods [Welling and Teh, 2011, Springenberg et al.,77 2016], training BNNs remains difficult. Furthermore, the predictive uncertainty seems dependent78 on the degree of approximation and is thus controlled by the available compute power.79
Ensemble methods have long been used to produce aggregated predictions with uncertainty estimates80 [Hansen and Salamon, 1990, Breiman, 1996]. Deep ensembles [Lakshminarayanan et al., 2017],81 a collection of differently initialized networks trained on the same data, are generally reported as82 state-of-the-art for uncertainty quantification in deep models [Thagaard et al., 2020, Ovadia et al.,83 2019]. As the models in the ensemble are trained on overlapping data, they are correlated, which84 influence the ensemble uncertainty in ways that remains unclear [Breiman, 2001]. Monte-Carlo85 dropout [Gal and Ghahramani, 2016] casts dropout training [Srivastava et al., 2014] as an ensemble86 model. It is computationally cheap, but experiments [Ovadia et al., 2019, Skafte et al., 2019] show87 that the increased correlation of ensemble elements amplifies the method’s overconfidence.88
Robustness to distribution shift is paramount to a well-behaved uncertainty predictor [Ovadia89 et al., 2019] and must be evaluated accordingly. For out-of-distribution detection, Liang et al.90
[2017] proposes a pre-processing perturbation step inspired by adversarial attacks [Goodfellow et al.,91 2014a] that helps the model distinguish in-distribution and out-of-distribution inputs. Hendrycks92 et al. [2018] used a Generative Adversarial Network (GAN) [Goodfellow et al., 2014b] to generate93 out-of-distribution pseudo-inputs whose inclusion in the training under an additional regularizing94 term in the loss function, called outlier exposure, enhances the predictor’s ability to discriminate95 out-of-distribution inputs [Lee et al., 2017, Dai et al., 2017].96
1.2 Robust uncertainty estimates97
Our work is strongly inspired by the critical assessment of the issues that undermine variance estima-98 tion ran by Skafte et al. [2019] and by the proposal of Stirn and Knowles [2020] which we detail here.99
Notation. Let the observed variable x 2 X follow the data generating distribution pdata(x), only100 known through the training dataset of N i.i.d samples Dtrain = {xn}Nn=1. In the case of supervised101 learning, the observed variables x = (x, y), with x 2 Rd being the input and y 2 Rd0 the target102 for the model, follow the joint decomposition pdata(x, y) = pdata(y|x)pdata(x). The proposed103 probabilistic model p✓(x), whose weights are indicated by ✓, aims to accurately emulate pdata(x).104
Practical problems in variance estimation. Gaussian likelihoods in the form of p✓(x) =105 N x|µ✓(x), ✓(x)2 are widely adopted to model continuous covariates. Real world data cannot be106 expected to be homoscedastic, i.e constant throughout input space, and thus the predictive uncertainty,107 ✓(x), most often uses neural networks to map continuously the observed x onto the parameter space.108 Beyond the well-known unreliable extrapolation properties of neural networks, this parametrisation109 of predictive uncertainty is hamstringed by serious defects. Firstly, the predictive variance scales the110 learning rates of the mean and variance updates by 1/2 ✓(x)2, resulting in a bias for data regions with111 low uncertainty [Nix and Weigend, 1994]. Secondly, the maximisation of the modeled likelihood is112 particularly sensitive to scarce data, as local gradient updates for the variance point towards the then113 undefined maximum likelihood estimate (MLE) [Skafte et al., 2019]. Lastly, and more worryingly, such114 model’s likelihood is ill-defined [Mattei and Frellsen, 2018a], as it can arbitrarily and without bound115 increase when the variance estimates collapse towards a detrimental 0. Overall, the naive maximisa-116 tion of model likelihood seems insufficient to generate robust and well-behaved uncertainty estimates.117
Student-t likelihood. The Bayesian formalism, by imposing to learn a parametrised distribution118 over the predictive uncertainty, offers an attractive view to approaching the problem of uncertainty119 estimation. Skafte et al. [2019] notably adopts a Gamma distributed precision, 1/ 2 = ⇠ (↵, ),120 as the conjugate of an unknown precision for a Gaussian likelihood, to yield a non-standard Student-t121 distributed marginal likelihood1. It is known to offer a more robust likelihood, especially in the scarce122 data regime [Gelman et al., 2013],123
p✓(x) = Z N (x|µ, ) ( |↵, )d = T ⇣ x|⌫ = 2↵, µ̂ = µ, ̂ = p /↵ ⌘ . (1)
Interestingly, its variance Var[x] = /(↵ 1) = ( /↵) · (↵/(↵ 1)) can be explicitly decomposed124 to an aleatoric term /↵ and an epistemic term1 ↵/(↵ 1) [Jørgensen, 2020, p16], and offers a125 direct verification of whether a model knows what it knows.126
Variational variance. Stirn and Knowles [2020] assumes a latent model precision . This is127 generated by a prior p( ) and its posterior is approximated variationally by the family of Gamma128 distributions, conditioned on the inputs to reflect heteroscedasticity. Through amortized variational129 inference (AVI) [Kingma and Welling, 2013] neural networks f map to the posterior parameters from130 data, q(z|f (x)). As such, variational variance preserves the modelling capacity and robustness of131 the non-standard Student-t marginal likelihood, without modifying its parameter architecture, while132 the definition of a prior over the latent precision induces a more robust training objective. Assuming133 the likelihood precision is the unique latent code, the evidence lower bound (ELBO),134
L(q; x) = Eq( ) [log p(x| )] DKL (q( |x) || p( ))
= 1
2
✓ (↵) log log(2⇡) ↵ (x µ)2 ◆ DKL (q( |x) || p( )) ,
(2)
takes the form of a regularised log-likelihood, exposing the benefits of the prior regularisation. It135 penalises predicted variances that would unrealistically get arbitrarily close to either the detrimental136
1See Section I. of the supplementary materials.
limits of 0 or 1, reducing the concerns regarding the ill-definition of the objective. Additionally,137 the scaling effect of the learning rates of the likelihood parameters is reduced. Naturally, the effect138 of the regularisation will be highly dependent on the prior selected. Here, because we are mostly139 interested in enforcing a constant desired uncertainty extrapolation, we adopt an homoscedastic140 Gamma distributed prior, p( ) = ( |a, b), that matches the level of uncertainty observed in data,141 and leave it for future practitioners to adopt the most adequate prior for the task at hand.142
2 Out-of-distribution pseudo-inputs training143
2.1 Dissipative loss144
In the variational variance formalism, due to AVI, the predictive uncertainty is controlled by ↵ and ,145 the independent neural networks parametrising the posterior distribution, Var[x] = (x)/(↵(x) 1).146 The unreliable extrapolation properties of neural networks therefore directly challenge the robustness147 of the method’s uncertainty estimates outside of its training support, limiting the applicability of the148 method. We consider that this flawed extrapolation is not inevitable.149
Inspired by outlier exposure [Hendrycks et al., 2018], we propose to include deliberately generated150 out-of-distribution pseudo-inputs, {x̂k}Kk=1 where x̂k ⇠ pout(x), in the training of our variational151 objective to constrain the extrapolation of the posterior parametrisation. The optimal variational152 objective q⇤ is chosen such that it minimises our proposed dissipative loss over the consolidated153 dataset D = Dtrain [Dout, where Dout = {x̂k}Kk=1,154
Loss(q;D) = h Lin(q;Dtrain) + Lout(q;Dout) i . (3)
The in-distribution component of the loss function Lin(q;D) naturally arises as the standard ELBO155 over the training set. The out-of-distribution component Lout(q;D) operates on a fundamentally156 different source of data. As the only information available regarding the pseudo-inputs is that they157 are out-of-distribution, we assert for them a constant, non-informative likelihood p(x̂| ) = c, that158 has thus no influence on optimisation. This is similar to the strategy of censoring [Lee and Wang,159 2003] where different likelihoods are used for observations with different properties. As a result, the160 dissipative loss becomes,161
Loss(q;D) = h X
x2Dtrain
Eq( |x) [p✓(x| )] DKL(q( |x) || p( )) X
x̂2Dout
DKL(q( |x̂) || p( )) i . (4)
It share the same motivating intuition as the confidence loss of Lee et al. [2017] and completes the162 variational variance formalism with a principled mechanism to learn robust variance estimates with163 the desired extrapolation properties. It indeed explicitly forces the predictor to match our high-entropy164 prior expectations on out-of-distribution samples while learning the low-entropy covariate dependent165 distribution, hence the name of dissipative. The reliance of the model’s predictive uncertainty166 on its mean predictions implies that it is primordial here to safeguard its generative performance.167 We guarantee it with the implementation of a split training procedure [Skafte et al., 2019]; the168 out-of-distribution regularisation is only applied after the model’s mean has been trained.169
2.2 Pseudo-input generators (PIGs)170
Minimising the posterior KL divergence out-of-distribution requires an efficient sampling procedure171 of pseudo-inputs. As exposed in Fig. 2, their generation should leverage a-priori knowledge about172 pdata(x) to resolve the undefined nature of pout(x). In this simple regression case, we show the173 predictive uncertainty of variational variance models trained on artificial heteroscedastic data. We174 use a prior uncertainty level that matches the maximum of the data uncertainty. As anticipated,175 without pseudo-inputs, the model extrapolates uncertainty to a constant, arbitrary level, and only the176 introduction of pseudo-inputs near the training data results in the desired uncertainty extrapolation.177 Reassuringly, this suggests that we do not need to regularise our model’s extrapolation in the entire178 out-of-distribution space. Instead, we can focus on the simpler task of generating pseudo-inputs179 in low-density regions of the input space that neighbours training data, as they can enforce correct180 extrapolation in the rest of the out-of-distribution space. Lee et al. [2017] gives supporting arguments.181
Recent contributions have relied on GANs for generating a useful representation of pout(x) [Lee182 et al., 2017, Dai et al., 2017]. Although conceptually intuitive, GANs incur a heavy computational183
burden and most likely induce serious practical challenges as a result of the instability of their training184 [Shrivastava et al., 2017]. Furthermore, as one need to understand what is in-distribution to model185 what it is not, we instead propose to directly leverage the information at hand about the data.186
Algorithm 1: Pseudo-Input Generator (PIG) 8k 2 [1,K], x̂k ⇠ pdata(x). iterations = 0. ✏ = 1; while (iterations < max_iterations) & (✏ > tolerance) do
compute 8k 2 [1,K],rxp(x)(x̂k); ✏ = maxk2[1,K]( rxp(x)(x̂k)); 8k 2 [1,K], x̂k = x̂k rxp(x)(x̂k); iterations = iterations + 1;
end Algorithm 1 gives a simple proce-187 dure for generating pseudo-inputs us-188 ing the data density. Pseudo-inputs189 are originally sampled from pdata(x),190 and their positions iteratively updated191 with gradient descent, with step size ,192 by following the directions that min-193 imise their likelihood under pdata(x),194 similarly to reversed adversarial steps195 [Goodfellow et al., 2014a].196
The procedure can run prior to training, in parallel for all x̂k with automatic differentiation, and197 thus results in limited additional complexity for the optimisation2. It relies on the availability of198 a differentiable density estimate of the data, which is, depending on the use case, either directly199 available (see Sec. 3.2), or can be approximated through a variety of methods such as Bayesian200 Gaussian mixture models [Bishop, 2006], or various normalising flows [Rezende and Mohamed,201 2015] based methods such as masked autoregressive flows [Papamakarios et al., 2017] (see Sec. 3.1).202 A caveat here is that depending on the PIG’s parameters, and on the quality of the density estimate203 available, pseudo-inputs might be generated in undesired regions of the input space, e.g uninformative204 density minima. In practice, we adopted conservative density estimates and parameters and did not205 observe any significant degradation of the predictive uncertainty due to the addition of pseudo-inputs.206
3 Experiments207
Holistic evaluation of uncertainty estimates. The ground truth for uncertainty estimates is usually208 unknown, making their evaluation non-trivial. Similarly as in Stirn and Knowles [2020], we propose209 to assess them using a collection of metrics. Calibration, which evaluates probabilistic predictions210 w.r.t the long-run frequencies that actually occur [Dawid, 1982] can be measured by proper scoring211 rules [Lakshminarayanan et al., 2017] such as the model log-likelihood log p✓(x| ). Additionally,212 the root mean squared error (RMSE) between the predictive and empirical variance, Var[x] 213 (Eq(z|x) [p✓(x| )] x)2, offers a quantification of the model’s awareness of its own uncertainty. It214 nevertheless requires an understanding of the model’s mean predictive performance, as commonly215 measured by the RMSE of the mean residuals, Eq( |x) [p✓(x| )] x. We further propose to evaluate216 the cooperation of mean and uncertainty estimates for the generation of credible samples, which217 constitutes a consistency check for the learned precision distribution [Gelman et al., 2013], by218 measuring the RMSE of sample residuals x⇤ x, with x⇤ ⇠ p✓(x). Finally, The ELBO, despite the219 absence of theoretical grounding for it [Blei et al., 2017], is commonly reported as an approximation220 of the marginal likelihood, and thus of the overall model’s predictive performance.221
2Running times are reported in Sec. IV. of the supplementary materials.
A complete assessment of a model’s uncertainty estimates further requires their evaluation under222 distributional shift [Ovadia et al., 2019], which we either introduce voluntarily through deliberate223 splitting of the training and test sets, as in Sec. 3.1, or by using test data from a different dataset224 altogether, as in Sec. 3.2.225
3.1 Regression226
In a regression setting where the proposed model must capture the conditioning between targets and227 inputs y|x, the precision of a Gaussian likelihood is the only assumed latent code.228
Faithfully to variational variance [Stirn and Knowles, 2020] we229 adopt a Gamma heteroscedastic variational posterior q ( |x) =230 ( |↵ (x), (x)) parametrised by the independent ↵ and 231 networks, with weights , uniquely conditioned on the inputs (see232 Fig. 3). This approximate posterior, independent of the targets, gives233 up on the dependency of the true posterior on both covariates to234 guarantee heteroscedasticity3.235
For strictly more than 2 degrees of freedom, or equivalently,236 ↵ (x) > 1, the marginal predictive probability p✓, (y|x) =237 T ⇣ y|2↵ (x), µ✓(x), p (x)/↵ (x) ⌘ , has its first two moments238
defined, E[y|x] = µ✓(x) and Var[y|x] = (x)/(↵ (x) 1), pro-239 viding explicit mean and uncertainty estimates with a single forward240
pass in the single layered, fully connected, ↵ , and µ✓ networks used here. To ensure definition of241 both the posterior distribution and of the marginal distribution’s variance, the parameter maps use a242 soft-plus activation on their last layer to ensure positivity, and the ↵ network is further shifted by 1.243
The unique dependence of the posterior on the inputs implies that the generation of pseudo-inputs244 should only rely on the input density. In a general regression setting, it is unknown, and we estimate245 it here prior to training with a Bayesian Gaussian mixture model [Bishop, 2006]. We refer to it246 henceforth as dissipative variational variance (d-VV). The specific implementation details are listed247 in Section II. of the supplementary materials.248
3.1.1 Toy regression249
The desiderata for our method are clear: capture of the data heteroscedasticity, extrapolation to a250 higher uncertainty level, no underestimation of the predictive uncertainty, and posterior extrapolation251 to the prior out-of-distribution. Skafte et al. [2019] first showed on the toy regression task, y =252 x sin(x)+0.3 ✏1+0.3x ✏2, where ✏1, ✏2 ⇠ N (0, 1), that amongst a collection of methods, only their253 proposed variance network architecture could realise our first three expectations. Fig. 4 demonstrates254 that our more principled approach also fulfills all of our requirements, without the need for arbitrarily255 enforcing the desired extrapolation in our architecture. The importance of out-of-distribution training256 is also revealed as the standard variational variance approach fails to produce uncertainty estimates257 that extrapolate correctly and are robust to distributional shift (bottom row of Fig. 4).258
3See Section II. of the supplementary materials for the expression of the true posterior.
Table 1: UCI benchmarks. Each square shows the performance of a given model (rows) on a given dataset (columns). The intensity of the colouring represents the certitude that the associated model performed best on the given dataset. Grey rows mean impossible evaluation for a metric.
Table 2: Evaluation of the generative modeling. For each dataset, we report mean ± std over 5 trials.
FashionMNIST SVHN CIFAR
log p(x) VAE 2215.54± 68.81 4304.90± 58.45 2930.64± 14.82 d-V3AE 2349.71± 11.80 4133.41± 64.28 2668.85± 13.23 RMSE(x, x̃) VAE 0.171± 0.003 0.097± 7e-4 0.154± 5e-4d-V3AE 0.158± 0.003 0.087± 0.002 0.129± 7e-4
Decomposition of the model and data uncertainty. Fig. 5259 presents the decomposition of the predictive uncertainty. The260 aleatoric component captures the heteroscedastic increase of261 uncertainty in the training data while the epistemic uncertainty,262 constant in distribution, extrapolates to higher values. The263 proposed method therefore demonstrates, to the best of our264 knowledge, a principled decomposition of uncertainty factors.265
3.1.2 UCI Benchmarks266
Real world regression datasets from the UCI repository4 are used to evaluate our model against267 curated baselines, analogically to the setup from Hernández-Lobato and Adams [2015] and Skafte268 et al. [2019]5. As revealed by the summarising Tab. 16, our method retains the mean predictive power269 of the variational variance method. The log-likelihood and RMSE of variance and sample residuals270 further show the improved calibration resulting from the imposition of a prior on the variance, as271 the VV methods generally outperform the MLE Student-t (VV (no prior)) that shares the same272 architecture. The table thus proves that holistically, the dissipative loss strengthens the variational273 variance model’s performance, which itself generally surpasses the chosen baselines.274
The robustness of the methods to distributional shift is further evaluated as in Foong et al. [2019]. For275 each input feature, a hole is created in the training data by assigning the middle third of observations276 to the test set, when sorted w.r.t that feature. Interestingly, we see that our method’s calibration277 slightly improves under the shift, highlighting the robustness benefits of the OOD prior regularisation.278
We note that both MC Dropout and the combined method of Skafte et al. [2019] generally perform279 well, confirming their interest for regression tasks requiring uncertainty quantification, but the280 former’s calibration is not robust to data shifts, as is also reported in Ovadia et al. [2019], and the281 latter is in practice difficult to tune and lacks principle.282
4https://archive.ics.uci.edu/ml/index.php 5See Sec. II. for details about the chosen baselines, datasets and implementations specifics. 6The full numbers are included in Sec. II. of the supplements.
3.2 Generative models283
We extend the evaluation of our proposal to the case of generative models through the lens of VAEs284 [Kingma and Welling, 2013, Rezende et al., 2014]. Variational auto-encoders infer a low dimensional285 latent encoding of the data z 2 RD, on which is conditioned the generative process p✓(x|z). Its286 predictive uncertainty, which evaluates the confidence of the model in its ability to adequately287 reconstruct inputs is known to be untrustworthy.288
In the case of continuous or seemingly continuous in-289 puts, the adoption of a Gaussian decoder p✓(x|z) =290 N x|µ✓(z), ✓(z)2 results in an ill-defined model likelihood291 [Mattei and Frellsen, 2018a] that encourages decoder variance292 collapse, making the training of the model notoriously harder293 [Skafte et al., 2019]. Most implementations therefore choose294 to fix the variance to a set level e.g ✓(z) = 0.1, or elude the295 challenge by adopting a Bernoulli likelihood.296
Motivated by our previous results, we now aim to demonstrate297 that VAEs, whose decoder is fitted with our method, are298 able to provide robust uncertainty estimates. Assuming a299 latent generative precision, the latent variables of the model300 are decomposed into z = {z, }, with z the latent input301 representations. The marginalisation of the Gamma distributed302 latent variance results in a Student-T decoder, as detailed in303
Eq. 1. The overall architecture of the variational variance variational auto encoder (V3AE) [Stirn304 and Knowles, 2020] is shown in Fig. 6, and yields, with the addition of our out-of-distribution305 pseudo-inputs training, the dissipative loss function7,306
Loss(q , ✓;Dtrain) = h X
x2Dtrain
L(q , ✓; x) Eqout(z) [DKL (q ( |z) || p( ))] i . (5)
Because only the decoder is regularised, the pseudo-inputs lie in the space of latent representations,307 Dout = {ẑk}Kk=1 2 RD. The distribution of training inputs is therefore readily accessible as the308 aggregate variational posterior q (z|Dtrain) = q (z|x1) · · · q (z|xN ). Here again, we rely on a split309 training procedure to leverage this perk; the encoder parameter maps µ✓ and ✓, as well as the310 decoder mean µ are first trained until convergence, allowing the generation of the out-of-distribution311 pseudo-inputs and subsequently, the training of the decoder variance.312
Image data. We evaluate the performance of our proposed dissipative313 V3AE (d-V3AE) against a fully Gaussian VAE on image data, coming from314 FashionMNIST, SVHN and CIFAR10. For both models, all parameter315 maps share the same underlying architecture, with the addition of either316 a softplus and/or a shifting last layer to ensure definition of both the317 variational and the generative distribution’s moments8.318
Tab. 2 compares model performance on two metrics, the log-likelihood319 and the RMSE between the original inputs x and reconstructed samples x̃,320 where x̃ ⇠ p✓(x| , z) , ( , z) ⇠ q ( , z|x). Unlike most previous imple-321 mentations, we focus on actual samples, and not the mean, of the generative322 distributions. This comparison emphasize the cooperation between the323 decoder’s mean and variance, allowing evaluation of the models’ uncer-324 tainty estimates. Our method both qualitatively (Fig. 7), and quantitatively325 improves on a Gaussian VAE’s sampling ability. The prior smoothens the326 uncertainty estimates, resulting in more realistic and less crisp samples.327 The log-likelihoods, evaluated at test time using truncation, i.e. ptrunc(x) =328 p✓(x)/(Fx(1) Fx(0)), to account for the finite support of data, reveal329 that our model can achieve a better fit, if the prior is selected correctly. In330
SVHN and CIFAR10, the presence of color channels complicates the selection process and challenges331 our choice of a single homoscedastic prior for all pixels and channels. We note that the dissipative loss332 also applies to classic VAEs with Bernoulli-only decoders; see Sec. III. of the supplements for details.333
7The derivation of the dissipative loss function is provided in Sec. III. of the supplementary materials. 8Implementation details are provided in Sec. III. of the supplementary materials.
Applications of robust generative uncertainty. In Figs. 8 & 9, the colouring of the 2D latent space334 represent the aggregated decoder variance Pdi=1( ✓(z)2)i. It is clear that our method displays more335 regular uncertainty estimates, and provides the extrapolation guarantees we strove for. Beyond in-336 creased robustness and better generative power, this unlocks meaningful out-of-distribution detection,337 beating previous state-of-the-art [Havtorn et al., 2021]. For Figs. 9 & 10, as argued in Mattei and338 Frellsen [2018b], we refit at test time the encoder of models trained on FashionMNIST on MNIST.339 The regularity and structure of the decoder variance rewards the encoder for learning to place represen-340 tations of OOD data outside of the region of in-distribution latent encodings, resulting in a model that341 is aware of its own inability to reconstruct plausible data, as displayed by the row x̃refit of d-V3AE.342
4 Conclusion343
We have introduced a novel loss, the dissipative loss, that leverages artificial out-of-distribution344 pseudo-inputs for learning robust uncertainty estimates. We demonstrate through a Bayesian approach345 that casts a prior distribution over the model’s variance a principled mechanism for controlling the346 extrapolation properties of neural networks governing the predictive uncertainty. Our experimental347 results reflect the benefits of our principled and scalable approach, displaying better calibrated and348 more robust uncertainty estimates, while matching the predictive power of known baselines. Finally,349 and most interestingly, our approach can instill into probabilistic models a notion of their own350 ignorance, increasing their ability to know what they don’t know.351
Limitations. The largest limitation of our approach is that it depends on an estimate of the density of352 the input data. In our experience, even coarse-grained densities are sufficient to significantly improve353 upon current approaches. However, as one rarely have guaranteed good estimates of the input density,354 our method cannot be approached as a black-box. One exception seems to be the application to VAEs,355 where the aggregated posterior, in our experience, always provide a suitable density estimate.356
Negative societal impact. Improving the ability of predictive models to assess their own confidence357 is solely a positive contribution as it can help alleviate potential consequences of incorrect predictions.358 We are therefore not aware of any potential negative impacts of our work.359
References360 Keyulu Xu, Mozhi Zhang, Jingling Li, Simon S Du, Ken-ichi Kawarabayashi, and Stefanie Jegelka.361 How neural networks extrapolate: From feedforward to graph neural networks. arXiv preprint362 arXiv:2009.11848, 2020.363
Dario Amodei, Chris Olah, Jacob Steinhardt, Paul Christiano, John Schulman, and Dan Mané.364 Concrete problems in ai safety. arXiv preprint arXiv:1606.06565, 2016.365
Lars Kai Hansen and Peter Salamon. Neural network ensembles. IEEE transactions on pattern366 analysis and machine intelligence, 12(10):993–1001, 1990.367
Balaji Lakshminarayanan, Alexander Pritzel, and Charles Blundell. Simple and scalable predictive368 uncertainty estimation using deep ensembles. In Advances in neural information processing369 systems, pages 6402–6413, 2017.370
David JC MacKay. A practical bayesian framework for backpropagation networks. Neural computa-371 tion, 4(3):448–472, 1992.372
Yarin Gal and Zoubin Ghahramani. Dropout as a bayesian approximation: Representing model373 uncertainty in deep learning. In international conference on machine learning, pages 1050–1059,374 2016.375
Christopher M. Bishop. Pattern Recognition and Machine Learning. Springer, August 2006. ISBN376 0387310738.377
Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q Weinberger. On calibration of modern neural378 networks. arXiv preprint arXiv:1706.04599, 2017.379
Dan Hendrycks and Kevin Gimpel. A baseline for detecting misclassified and out-of-distribution380 examples in neural networks. arXiv preprint arXiv:1610.02136, 2016.381
Eric Nalisnick, Akihiro Matsukawa, Yee Whye Teh, and Balaji Lakshminarayanan. Detecting out-of-382 distribution inputs to deep generative models using typicality. arXiv preprint arXiv:1906.02994,383 2019.384
Anh Nguyen, Jason Yosinski, and Jeff Clune. Deep neural networks are easily fooled: High confidence385 predictions for unrecognizable images. In Proceedings of the IEEE conference on computer vision386 and pattern recognition, pages 427–436, 2015.387
Christos Louizos and Max Welling. Multiplicative normalizing flows for variational bayesian neural388 networks. arXiv preprint arXiv:1703.01961, 2017.389
Nicki Skafte, Martin Jørgensen, and Søren Hauberg. Reliable training and estimation of variance390 networks. In Advances in Neural Information Processing Systems, pages 6326–6336, 2019.391
Kimin Lee, Honglak Lee, Kibok Lee, and Jinwoo Shin. Training confidence-calibrated classifiers for392 detecting out-of-distribution samples. arXiv preprint arXiv:1711.09325, 2017.393
Keyulu Xu, Mozhi Zhang, Jingling Li, Simon S. Du, Ken ichi Kawarabayashi, and Stefanie Jegelka.394 How neural networks extrapolate: From feedforward to graph neural networks, 2021.395
David A Nix and Andreas S Weigend. Estimating the mean and variance of the target probability396 distribution. In Proceedings of 1994 ieee international conference on neural networks (ICNN’94),397 volume 1, pages 55–60. IEEE, 1994.398
Søren Hauberg. Only bayes should learn a manifold (on the estimation of differential geometric399 structure from data), 2019.400
Georgios Arvanitidis, Lars Kai Hansen, and Søren Hauberg. Latent space oddity: on the curvature of401 deep generative models. arXiv preprint arXiv:1710.11379, 2017.402
Diederik P Kingma and Max Welling. Auto-encoding variational bayes. arXiv preprint403 arXiv:1312.6114, 2013.404
Danilo Jimenez Rezende, Shakir Mohamed, and Daan Wierstra. Stochastic backpropagation and405 approximate inference in deep generative models. arXiv preprint arXiv:1401.4082, 2014.406
Armen Der Kiureghian and Ove Ditlevsen. Aleatory or epistemic? does it matter? Structural safety,407 31(2):105–112, 2009.408
Burr Settles. Active learning. Synthesis lectures on artificial intelligence and machine learning, 6(1):409 1–114, 2012.410
Jonas Močkus. On bayesian methods for seeking the extremum. In Optimization techniques IFIP411 technical conference, pages 400–404. Springer, 1975.412
Carl Edward Rasmussen and Christopher K. I. Williams. Gaussian Processes for Machine Learning413 (Adaptive Computation and Machine Learning). The MIT Press, 2005.414
Alex Graves. Practical variational inference for neural networks. Advances in neural information415 processing systems, 24:2348–2356, 2011.416
Charles Blundell, Julien Cornebise, Koray Kavukcuoglu, and Daan Wierstra. Weight uncertainty in417 neural networks. arXiv preprint arXiv:1505.05424, 2015.418
José Miguel Hernández-Lobato and Ryan Adams. Probabilistic backpropagation for scalable learning419 of bayesian neural networks. In International Conference on Machine Learning, pages 1861–1869,420 2015.421
Leonard Hasenclever, Stefan Webb, Thibaut Lienart, Sebastian Vollmer, Balaji Lakshminarayanan,422 Charles Blundell, and Yee Whye Teh. Distributed bayesian learning with stochastic natural gradient423 expectation propagation and the posterior server. The Journal of Machine Learning Research, 18424 (1):3744–3780, 2017.425
Max Welling and Yee W Teh. Bayesian learning via stochastic gradient langevin dynamics. In426 Proceedings of the 28th international conference on machine learning (ICML-11), pages 681–688,427 2011.428
Jost Tobias Springenberg, Aaron Klein, Stefan Falkner, and Frank Hutter. Bayesian optimization429 with robust bayesian neural networks. Advances in neural information processing systems, 29:430 4134–4142, 2016.431
Leo Breiman. Bagging predictors. Machine learning, 24(2):123–140, 1996.432
Jeppe Thagaard, Søren Hauberg, Bert van der Vegt, Thomas Ebstrup, Johan D. Hansen, and Anders B.433 Dahl. Can you trust predictive uncertainty under real dataset shifts in digital pathology? In Medical434 Image Computing and Computer-Assisted Intervention (MICCAI), Lima, Peru, October 2020.435
Yaniv Ovadia, Emily Fertig, Jie Ren, Zachary Nado, David Sculley, Sebastian Nowozin, Joshua436 Dillon, Balaji Lakshminarayanan, and Jasper Snoek. Can you trust your model’s uncertainty?437 evaluating predictive uncertainty under dataset shift. In Advances in Neural Information Processing438 Systems, pages 13991–14002, 2019.439
Leo Breiman. Random forests. Machine learning, 45(1):5–32, 2001.440
Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov.441 Dropout: a simple way to prevent neural networks from overfitting. The journal of machine442 learning research, 15(1):1929–1958, 2014.443
Shiyu Liang, Yixuan Li, and Rayadurgam Srikant. Enhancing the reliability of out-of-distribution444 image detection in neural networks. arXiv preprint arXiv:1706.02690, 2017.445
Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial446 examples. arXiv preprint arXiv:1412.6572, 2014a.447
Dan Hendrycks, Mantas Mazeika, and Thomas Dietterich. Deep anomaly detection with outlier448 exposure. arXiv preprint arXiv:1812.04606, 2018.449
Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair,450 Aaron Courville, and Yoshua Bengio. Generative adversarial nets. Advances in neural information451 processing systems, 27:2672–2680, 2014b.452
Zihang Dai, Zhilin Yang, Fan Yang, William W Cohen, and Russ R Salakhutdinov. Good semi-453 supervised learning that requires a bad gan. In Advances in neural information processing systems,454 pages 6510–6520, 2017.455
Andrew Stirn and David A Knowles. Variational variance: Simple and reliable predictive variance456 parameterization. arXiv preprint arXiv:2006.04910, 2020.457
Pierre-Alexandre Mattei and Jes Frellsen. Leveraging the exact likelihood of deep latent variable458 models. In Advances in Neural Information Processing Systems, pages 3855–3866, 2018a.459
Andrew Gelman, John B Carlin, Hal S Stern, David B Dunson, Aki Vehtari, and Donald B Rubin.460 Bayesian data analysis. CRC press, 2013.461
Martin Jørgensen. Stochastic Representations with Gaussian Processes and Geometry. PhD thesis,462 Technical University of Denmark, 2020.463
Elisa T Lee and John Wang. Statistical methods for survival data analysis, volume 476. John Wiley464 & Sons, 2003.465
Ashish Shrivastava, Tomas Pfister, Oncel Tuzel, Joshua Susskind, Wenda Wang, and Russell Webb.466 Learning from simulated and unsupervised images through adversarial training. In Proceedings of467 the IEEE conference on computer vision and pattern recognition, pages 2107–2116, 2017.468
Danilo Rezende and Shakir Mohamed. Variational inference with normalizing flows. In International469 Conference on Machine Learning, pages 1530–1538. PMLR, 2015.470
George Papamakarios, Theo Pavlakou, and Iain Murray. Masked autoregressive flow for density471 estimation. arXiv preprint arXiv:1705.07057, 2017.472
A Philip Dawid. The well-calibrated bayesian. Journal of the American Statistical Association, 77473 (379):605–610, 1982.474
David M Blei, Alp Kucukelbir, and Jon D McAuliffe. Variational inference: A review for statisticians.475 Journal of the American statistical Association, 112(518):859–877, 2017.476
Andrew YK Foong, Yingzhen Li, José Miguel Hernández-Lobato, and Richard E Turner. ’in-477 between’uncertainty in bayesian neural networks. arXiv preprint arXiv:1906.11537, 2019.478
Jakob D. Havtorn, Jes Frellsen, Søren Hauberg, and Lars Maaløe. Hierarchical vaes know what they479 don’t know, 2021.480
P.-A Mattei and J Frellsen. Refit your encoder when new data comes by. In 3rd NeurIPS workshop481 on Bayesian Deep Learning, 2018b.482
Norman L Johnson, Samuel Kotz, and Narayanaswamy Balakrishnan. Continuous univariate distri-483 butions, volume 1, 2nd Edition. John wiley & sons, 1994.484
Christian Bauckhage. Computing the kullback-leibler divergence between two generalized gamma485 distributions. arXiv preprint arXiv:1401.6853, 2014.486
Checklist487 1. For all authors...488
(a) Do the main claims made in the abstract and introduction accurately reflect the paper’s489 contributions and scope? [Yes]490
(b) Did you describe the limitations of your work? [Yes] Yes, we have included a separate491 subsection on the limitations. See Section 4.492
(c) Did you discuss any potential negative societal impacts of your work? [Yes] Covered493 specifically in Section 4.494
(d) Have you read the ethics review guidelines and ensured that your paper conforms to495 them? [Yes] Yes.496
2. If you are including theoretical results...497 (a) Did you state the full set of assumptions of all theoretical results? [Yes] Our results are498 obtained through empirical evidence. We do, however, include relevant derivations in499 the supplementary material.500
(b) Did you include complete proofs of all theoretical results? [N/A]501 3. If you ran experiments...502
(a) Did you include the code, data, and instructions needed to reproduce the main ex-503 perimental results (either in the supplemental material or as a URL)? [Yes] Code is504 included in the supplementary material.505
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they506 were chosen)? [Yes] Yes, every experiment is accompanied by a separate settings file507 in yaml format.508
(c) Did you report error bars (e.g., with respect to the random seed after running experi-509 ments multiple times)? [Yes] Each experiment was run at least in triplicate.510
(d) Did you include the total amount of compute and the type of resources used (e.g., type511 of GPUs, internal cluster, or cloud provider)? [Yes] Yes, in supplementary material.512
4. If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...513 (a) If your work uses existing assets, did you cite the creators? [Yes]514 (b) Did you mention the license of the assets? [Yes] We ran our experiments on standard,515 well known datasets, reference the source, which itself includes licenses.516 (c) Did you include any new assets either in the supplemental material or as a URL? [Yes]517 Our code is available in the supplementary material518 (d) Did you discuss whether and how consent was obtained from people whose data you’re519 using/curating? [N/A] We did not run any experiments on personal data.520 (e) Did you discuss whether the data you are using/curating contains personally identifiable521 information or offensive content? [N/A]522 5. If you used crowdsourcing or conducted research with human subjects...523
(a) Did you include the full text of instructions given to participants and screenshots, if524 applicable? [N/A]525
(b) Did you describe any potential participant risks, with links to Institutional Review526 Board (IRB) approvals, if applicable? [N/A]527
(c) Did you include the estimated hourly wage paid to participants and the total amount528 spent on participant compensation? [N/A]529

Review summaries:
This paper considers the idea of generating pseudo-inputs (i.e. extra augmented samples) to improve OOD uncertainty estimates by training with an uncertainty-increasing ("dissipative") loss at these pseudo-inputs. The pseudo-inputs are generated by minimizing estimated density under the data distribution via a simple gradient algorithm. The idea is evaluated in supervised learning (regression) and generative modeling (VAE for images).

The authors propose an approach (called d-VV) for learning better uncertainty estimates for neural network models. The core idea is a variation (pun not intended) on the variational variance work by Stirn and Knowles (2020) where an additional "dissipative" term is added to ELBO, which enforces that the latent precision stay close to its prior outside the training distribution as well. This is done to ensure that the uncertainty estimates are accurate even on OOD inputs.  The authors go on to propose a method for emulating "informative" OOD samples to estimate this dissipative term. This is achieved by starting from data samples and taking negative gradient steps on some differentiable density estimate (which is presumed to be available). 

The paper proposes a method for learnable uncertainty estimation using neural networks without an explicit requirement of known out-of-distribution data. Instead, the model is trained to maximize model fit where there is explicit training data and maximize uncertainty on estimated “pseudo-inputs”. A key contribution of the method is in how the “pseudo-inputs” are generated and crafted. The authors propose a “Pseudo-Input Generator (PIG)” which generates samples based on the training data density -- locations with lower data density will receive more pseudo-inputs. Results are demonstrated on a variety of tasks with increasing complexity ranging from UCI benchmarks to image generation tasks.