Robust uncertainty estimates with out-of-distribution pseudo-inputs training
Authors: Pierre Segonne
Yevgen Zainchkovskyy
Søren Hauberg
Venue: NeurIPS 2021 Submitted (2021)

Abstract: Probabilistic models often use neural networks to control their predictive uncertainty.
However, when making out-of-distribution (OOD) predictions, the often uncontrollable
extrapolation properties of neural networks yield poor uncertainty predictions.
Such models then don’t know what they don’t know, which directly limits
their robustness w.r.t unexpected inputs. To counter this, we propose to explicitly train
the uncertainty predictor where we are not given data to make it reliable.
As one cannot train without data, we provide mechanisms for generating pseudo-inputs
in informative low-density regions of the input space, and show how to leverage these
in a practical Bayesian framework that casts a prior distribution over the model uncertainty.
With a holistic evaluation, we demonstrate that this yields robust and interpretable predictions
of uncertainty while retaining state-of-the-art performance on diverse tasks such as regression
and generative modeling.

29 a mean prior N (Hu, n can be realised by additional training data of Ho with of setting the amount of observations. Placing Robust uncertainty estimates with out-of-distribution pseudo-inputs training
Anonymous Author(s)
Affiliation Address email
Abstract
Probabilistic models often use neural networks to control their predictive uncer- tainty. However, when making uncontrollable extrapolation properties of neural networks yield poor uncertainty predictions. Such models then don't know what they don't 't know, which directly lim- its their robustness w.r.t unexpected inputs. To counter this, we propose explicitly train the uncertainty predictor where we are not given data to make it reliable. As cannot train without data, we provide mechanisms for generating pseudo-inpus. in informative low-density regions of the input space, and show how to leverage these in a practical Bayesian framework that casts a prior distribution over the model uncertainty. With a holistic evaluation, we demonstrate that this yields robust and interpretable predictions of uncertainty while retaining state-of-the-art performance on diverse tasks such as regression and generative modeling.
2
(OOD) predictions, the often-
4
8
10
11
12
Introduction
14
Neural networks generally extrapolate arbitrarily [Xu et al. 2020]. and high quality predictions are limited regions of the imput space where the networks have been trained. This is be expected and is only problematic if the associated predictions are not accompanied with well-calibrated measure of uncertainty. If neural network is used for estimating such a measure of uncertainty, we, however, quickly run into trouble, the reported uncertainty then exhibits arbitrary behaviour in regions with no training data. Alarmingly, these are exactly the regions where evaluating the uncertainty is most important to the safe deployment of machine learning models in real world applications [Amodei) al., 2016]. One potential solution is avoid using directly the output of neural networks for predicting uncertainty, and let it emerge from another mechanism, e.g. an ensemble [Hasee and Salamon 1990 Lakshminarayanan et al. 2017] or some notion of Monte Carlo (MacKay 1992 Gal and Ghahramani 2016]. Here we explore the alternative view that the networks should simply be trained where there is no data.
15
16
17
18
19
20
21
23
24
25
28
But can we train without data? The Bayesian formalism often does so implicitly: most conjugate priors can be seen as ad- ditional training data [Bishop 2006). e.g. in Gaussian models,
Data Poeudo-inputs
27
28
30
31 a prior over the output a neural network can, thus, inter- preted as additional training data. Unfortunately, this view is
33 not practical as it implies additional data for all possible inputs to neural network, resulting in infinite data. Our approach
Figure 1: Pseudo-inputs are gener- ated out of distribution, and there train towards prior (grey density).
35 simple: we locate regions of low data density in input space and implicitly place observations here in output space by min-
Submitted to 35th Conference on Neural Information Processing Systems (NeurIPS 2021). Do not distribute.

---

37
imizing an appropriate KL divergence towards a prior (see Fig. D The result is a remarkably simple algorithm that drastically improves uncertainty estimates in both regression and generative modeling.
38
39
1.1
Background and related work
40
The predictive performance of machine learning models has drastically increased in the past decade, but the quality of the accompanying uncertainties have not followed. Uncertainties are reported as be-
41
42
ing miscalibrated [Guo et al. 2017 and overconfident [Laksminarayanann al. [017 [endrycks and than in-distribution data Nalisnick et al. 2019. Nguyen et al. 2013 Louizos and Welling, 2017].
43
Gimpel| 2016). Some models even see higher likelihoods of
44
45
Neural networks commonly output distributions which gives a notion of predictive uncertainty. Clas- sifiers trained with soft-max is ever-preseed example of such. These predictions are generally ob- served to be overconfident little meaning outside the support of the training data [Skafte al. [209 Lee al. [017]. The latter is an artifact of the hard-to-control extrapolation that comes with neural networks [uu et al. 2021] In general, since extrapolation is difficult to control, uncertainties predicted by neural networks will exhibit seemingly arbitrary behavior outside the support of the data, yielding untrustworthy results.
46
47
makshminarayanaan et al. 2017 Hendrycks and Gimpel| 2016) and to carry
49
51
Veranvariance networks for regression Nix and Weigend 1994] model the conditional target density as a normal p(y]z) = N (vlu(x);aax))) with mean and variance predicted by neural networks. The reported predictive uncertainty generally accurate in regions
53
54
training data, but otherwise unreliable [Huberrg 2019 To counter this, Arvanitidis et al. [2017] and [Skatte et al. 2019] proposed variance network architectures to enforce a specified extrapolation value, but these heuristics tend to be difficult to tune, and lack principle. Mean-varian
55
56
57
networks have seen a recent uptake within generative modeling, where they are applied as encoder distribution in variational autoencders (VAEs) (Kngmma and Welling 2013 Rezende et al. 2014).
59
Which uncertainty? A commonly called-upon dichotomy DDer Kiureghian and Ditlevsen, 2009 that the uncertainty of model's prediction can be decomposed into the uncertainty the model (epistemic) and of the data (aleatoric). The epistemic uncertainty can be lowered by increasing the amount of data, simplifying the model or otherwise reducing the complexity of the learning problem. The alcatoric uncertainty, changed; no prediction should ever be more certain than the uncertainty displayed by the associated data. Depending on the task at hand, may be interested in different types of uncertainty: In active learning [Settles] 2012) and Bayesian optimization [Močkus| 1975) we request data for which we have high epistemic, but low aleatoric uncertainty ensure maximal information gain; while for classification and regression we often just want to minimize the overall predictive uncertainty.
61
63
64
the other hand, is property of the world, and cannot be
65
67
69
Bayesian methods are often used to quantify uncertainty due to their explicit formulation of uncertainty. Gaussian procexses (GPs) [Rasmussen and Williams) 2005] provide an elegant framework that provide state-of-the -fte---rttt-ttttt uncertainty estimates, but, alas, the corresponding mean predictions are often not up to the standards of neural networks. GPs are tightly linked to Bayesian neural networks (BNNs) MMacKay 1992] that place prior over the network weights and seek the corresponding posterior. Despite ad advances in variational approximations (Graves] 2011 Kingma and Welling 2013 Blundell et al., 2015). expectation propagation (Hernández-Lobarr and Adams
71
73
74
75
77
2015 Hasenclever et al.] 2017]. or Monte Carlo methods [Welling and Teh 201 |Springenberg et al.
2016 training BNNs remains difficult. Furthermore, the predictive uncertainty seems dependent
79
on the degree of approximation and is thus controlled by the available compute power.
80
Ensemble methods have long been used to produce aggregated predictions with uncertainty estimates Hansen and Salamon 1990 Breiman 1996). Deep ensembles [Lakshminarayanall et al. 2017). collection of differently initialized networks trained the same data, are generally reported as state-of-the-ar for uncertainty quantification in deep models [Thagaard et al., 2020 Ovadia et al. 2019]. As the models in the ensemble are trained on overlapping data, they are correlated, which influence the ensemble uncertainty in ways that remains unclear [Breiman 2001]. Monte-Carlo dropout Gal and Ghahramani 2016 casts dropout training (Srivastava et al. 2014] as an ensemble model. It is computationally cheap, but experiments [Ovadia et al.] 2019 Skatte et al. 2019 show that the increased correlation of ensemble elements amplifies the method's overconfidence.
81
83
84
85
87
88
89
Robustness to distribution shift paramount to well-behaved uncertainty predictor [Ovadia et al.
2019 and must be evaluated accordingly. For out-of-distribution detection, Liang ct al.
2

---

91
2011
proposes pre- processing perturbation step inspired by adversarial attacks (Goodfellow et al.
92
4
that helps the model distinguish in-distribution and -iittriibiitttoonn inputs. Hendryeks
93
et al.
2018 used Generative Adversarial Network (GAN) [Goodfellow et al. 2014b] to generate
94
out-of -f-distribution pseudo-inputs whose inclusion in the training under an additional regularizing
95
term in the loss function, called outlier exposure, enhances the predictor's ' ability to discriminate
96
-fu-t--distribution inputs [ee et al. 2017
Dai et al. 2017].
97
1.2
Robust uncertainty estimates
98
Our work is strongly inspired by the critical assessment of the issues that undermine variance estima- tion ran by [Skatte et al. 2019) and by the proposal of SStim and Knowles 2020) which we detail here.
99
Notation. Let the observed variable x E X follow the data generating distribution Pdata (x), only known through the training dataset of N i..d samples Dunin {xn J In the case of supervised learning, the observed variables for the model, follow the joint decomposition Piata (I. v) = Paata (y xx) (drssaa..)). The proposed probabilistic model (x))), whose weights are indicated by 0, aims accurately emulate paata (x).
100
(zz, y), with x R being the input and y E Rd the target
102
103
104
105
Practical problems in variance estimation. Gaussian likelihoods in the form of po(x) =
N (x) are widely adopted to model continuous covariates. Real world data cannot be expected to be homoscedastic, i.e constant throughout input space, and thus the predictive uncertainty, (x), most often uses neural networks to map continuously the observed x onto the parameter space, Beyond the well-known unreliable extrapolation properties of neural networks, this parametrisation of predictive uncertainty is hamstringed by serious defects. Firstly, the predictive variance scales the learning rates of the mean and variance updates by (x)r resulting in bias for data regions with low uncertainty [Nix and Weigend 1994]. Secondly, the maximisation of the modeled likelihood is particularly sensitive to scarce data, as local gradient updates for the variance point towards the then undefined maximum likelihood estimate (MLE) [Skafte ct al. 2019]. Lastly, and more worryingly, such model's ss likelihood is ill-defined (Mattei and Frellsen 2018a|, J. as it can arbitrarily and without bound increase when the variance estimates collapse towards detrimental O. Overall, the naive maximisa- tion of model likelihood seems insufficient to generate robust and well-behaved uncertainty estimates.
106
107
108
109
110
112
114
116
118
Student-t likelihood. The Bayesian formalism, by imposing to learn a parametrised distribution over the predictive uncertainty, offers an attractive view to approaching the problem of uncertainty estimation. Skafte et al. [019] notably adopts a Gamma distributed precision, 1//2 = À ~ [' B),
120
as the conjugate of unknown precision for Gaussian likelihood, to yield a non-standard Student-t It is known to offer a more robust likelihood, especially in the scarce
distributed marginal likelihood! [
122
data regime [Gelman al. 2013].
(x(x) - f A)(AN, = (x|x = (xx1x
=20; i= 1, = vB/a)
(1)
Interestingly, its variance Var(x] -8/(a= 1) (a/(a 1)) can be explicitly decomposed
124
125
to an aleatoric term 3/a and an epistemic tern a/(a - 1) (orgensen 2020 p16], and offers a direct verification of whether model knows what it knows.
126
127
Variational variance. Stirn and Knowles (2020) assumes a latent model precision This is generated by a prior p(A) and its posterior is approximated variationally by the family of Gamma distributions, conditioned on the inputs to reflect heteroscedasticity. Through amortized variational inference (AVI) (Kingma and Welling, 2013) neural networks fo map to the posterior parameters from data, (x)). As such, variational variance preserves the modelling capacity and robustness of the non-standard Student-t marginal likelihood, without modifying its parameter architecture, while the definition of prior over the latent precision induces a more robust training objective. Assuming the likelihood precision is the unique latent code, the evidence lower bound (ELBO),
129
131
132
133
134
L(g: x) = (x) logp(x A)] - DKL. (q(Alx) ((A))
1
(2)
2 ((a) log 3 log(2
n1}²
DKL (a(Xlx) lI(()))
135
takes the form of a regularised ogglikkelhhood, exposing the benefits of the prior regularisation. It 136 penalises predicted variances that would unrealistically get arbitrarily close to either the detrimental
See Section I of the supplementary materials.
3

---

137
limits of O or oo, reducing the concerns regarding the ill-definition of the objective. Additionally, the scaling effect of the learning rates of the likelihood parameters reduced. Naturally, the effect of the regularisation will be highly dependent on the prior selected. Here, because we are mostly interested enforeing a constant desired uncertainty extrapolation, we adopt an homoscedastic Gamma distributed prior, p()) [(A b), that matches the level of uncertainty observed in data, and leave it for future practitioners to adopt the most adequate prior for the task at hand.
138
139
140
12
Out-of-distribution pseudo-inputs training
143
144
2.1
Dissipative loss
145
In the variational variance formalism, due AVI, the predictive uncertainty is controlled by a and B. the independent neural networks parametrising the posterior distribution, Var(x] = (xx)/(a(x) 1).
146
147
The unreliable extrapolation properties of neural networks therefore directly challenge the robustness of the method's uncertainty estimates outside of its training support, limiting the applicability of the method. We consider that this flawed extrapolation is not inevitable.
148
149
150
Inspired by outlier exposure [Hendrycks et al. 2018 we propose to include deliberately generated out-of-distribution pseudo- innulss {== where X5: Pxur(x), in the training of our variational objective to constrain the extrapolation of the posterior parametrisation. The optimal variational
151
152
153
objective 9º is chosen such that minimises our proposed dissipative loss over the consolidated
dataset D = Duain U Dout. where Dout = {
154
Loss(g; D) = [in((g Demin) Louu Domn)
(3)
155
The in-distribution component of the loss function Lin(;; D) naturally arises as the standard ELBO over the training set. The out-of-distriutiion component Cou( D) operates on a fundamentally different source of data. As the only information available regarding the pseudo-inputs is that they are -utto--distributtion, we assert for them a constant, con-informative likelihood p(([A) = c, that
156
157
158
159
has thus no influence on optimisation. This is similar to the strategy of censoring (Lee and Wang 2003] where different likelihoods are used for observations with different properties. As result, the
160
dissipative loss becomes,
Loss(g; D)
Σ BxCAs) [po[] DKL(o(A)))) D
Σ (KL(G(AB)))I((A)) (4) RED
162
It share the same motivating intuition as the confidence loss of Lee et al. [2017] and completes the variational variance formalism with a principled mechanism to learn robust variance estimates with the desired extrapolation properties. It indeed explicitly forces the predictor to match our high-entropy prior expectations on out- disttiiiitiiitoion samples while learning the low-entropy covariate dependent distribution, hence the name of dissipative. The reliance of the model's ' predictive uncertainty on its mean predictions implies that it is primordial here to safeguard its generative performance. We guarantee it with the implementation of a split training procedure [Skafte et al., 2019: the out- diit----ittiiitionn regularisation is only applied after the model's mean has been trained.
163
165
167
168
189
170
2.2
Pseudo-input generators (PIGs)
Minimising the posterior KL divergence of pseudo-inputs. As exposed in Fig. 2 their generation should leverage a-priori knowledge about (x) to resolve the undefined nature of (x). In this simple regression case, we show the predictive uncertainty of variational variance models trained on artificial heteroscedastic data. We use prior uncertainty level that matches the maximum of the data uncertainty. As anticipated, without pseudo-inputs, the model extrapolates uncertainty to a constant, arbitrary level, and only the introduction of pseudo-inputs near the training data results in the desired uncertainty extrapolation. Reassuringly, this suggests that we do not need to regularise our model's extrapolation in the entire out-of out-of distrrbbint space. Instead, in low-density regions of the input space that neighbours training data, as they can enforce correct extrapolation in the rest of the out-of-distribution space. Lee al. (0177] gives supporting arguments.
requires efficient sampling procedure
172
173
174
176
178
can focus the simpler task of generating pseudo-
180
181
182
Recent contributions have relied on GANs for generating useful representation of Pxm [Lee Dai et al. 2017]. Although conceptually intuitive, GANs incur heavy computational
et al.
2017

---

(x)
Near Pls
Far-away Pls
No Pls
Data uncertainty
Prior uncertainty
(x))
{x,)
Figure 2: Effect of different pseudo-input distributions on the predictive uncertainty of variational variance models. Training data (black points) is generated uniformly on [-5, 5], with variance that scales as exp -0.5[]
The near pseudo-inputs. (green diamonds) are generated uniformly in [-10, -5] U (5, 10), while the far-away (blue diamonds) are [-200, -990] [190, 200]. Dashes amount for the empty space that separates far away pseudo-inputs.
184
burden and most likely induce serious practical challenges result of the instability of their training SShrivastaaa al. 2017]. Furthermore, as one need to understand what in-distribution to model what it is not, instead propose to directly leverage the information hand about the data.
185
187 Al Algorithm gives a simple proce- dure for generating pseudo-inputs us- ing the data density. Pseudo-inputs are originally sampled from Pdata (x), and their positions iteratively updated with gradient descent, with step size 8,
Algorithm I: Pseudo-Inpul Generator (PIG)
188
Vk E [1, K], Åa Pdatta (x). iterations 0. while (iterations max iterations) > tolerance) do compute Vk E [1, K]. Vxp(x)
189
190
191
=mxx V (xx)))xx)):
V [1, K]. &2 X - V (xp(x)(RR); iterations iterations
193
by following the directions that min- imise their likelihood under Pdata (x).
194
195
similarly to reversed adversarial steps
end
Goodfellow et al. 2014a]
197
The procedure can run prior to training, in parallel for all & with automatic differentiation, and thus results in limited additional complexity for the optimisation It relies on the availability of differentiable density estimate of the data, which is, depending on the use case, either directly available (see Sec. 5.2). or can be approximated through a variety of methods such as Bayesian
198
199
200
Gaussian mixture models [Bishop, 2006), or various normalising flows [Rezende and Mohamed
201
based methods such as masked autoregressive flows (Papamakarios et al. 2017 (see Sec. B.Ij. caveat here is that depending on the PIG's parameters, and on the quality of the density estimate available, pscudo-inputs might be generated in undesired regions of the input space, e.g uninformative density minima. In practice, we adopted conservative density estimates and parameters and did not observe any significant degradation of the predictive uncertainty due to the addition of pscudo-inputs.
2015
203
205
3
Experiments
207
208
Holistic evaluation of uncertainty estimates. The ground truth for uncertainty estimates is usually unknown, making their evaluation non-trivial. Similarly as in (Stinn and Knowles [2020] we propose to assess them using collection of metrics. Calibration, which evaluates probabilistic predictions w.r.t the long-run run frequencies that actually occur (Dawid] 1982 can be measured by proper scoring rules LLkshminaaayaaaa et al. 2017) such as the model log-likelihood log pe (x[A). Additionally, the root mean squared error (RMSE) between the predictive and empirical variance, Var(x) (Extt))) (x[A)) - x)2 offers quantification of the model's 's awareness of its own uncertainty. It nevertheless requires an understanding of the model's mean predictive performance, as commonly
209
210
211
212
213
214
215
216
measured by the RMSE of the mean residuals, Ey(A))) [x]Ax] x. We further propose to evaluate the cooperation of mean and uncertainty estimates for the generation of credible samples, which constitutes consistency check for the learned precision distribution [Gelman et al. 2013), by measuring the RMSE of sample residuals x* x, with x* ~ (x)... Finally, The ELBO, despite the absence of theoretical grounding for it [Blei et al. 2017] is commonly reported as approximation of the marginal likelihood, and thus the overall model's predictive performance.
217
219
221
Running times reported in Sec. IV. of the supplementary materials.
5

---

Cround truth
Prior
Variational variance
d VV (Ours)
Decle))
Figure 4: Toy regression results. On the left, the predictions surrounded by + standard deviations, with the training data of the bottom row presenting a shift. On the right are displayed the predictive uncertainty fit and the prior KI. divergence.
complete assessment model's uncertainty estimates further requires their evaluation under 222
distributional shift [Ovadia et al., 2019]. which we either introduce voluntarily through deliberate splitting the training and test sets, as Sec. 3.1 or by using test data from different dataset altogether, as in Sec. 3.2
224
226
3.1
Regression
In regression setting where the proposed model must capture the conditioning between targets and inputs y|a, the precision Gaussian likelihood the only assumed latent code.
228
229
Faithfully to variational variance (Stirn and Knowles, 2020 we
230
adopt Gamma heteroscedastic variational posterior (A
polx, X)
(x)a(()), (z))) parametrised by the independent 0%6 and Ag networks, with weights ø, uniquely conditioned on the inputs (see
232
Pr
Oin
Fig. 3 This approximate posterior, independent of the targets, gives
234
up on the dependency of the true posterior on both covariates guarantee heteroscedastictty]
For strictly more than 2 degrees of freedom, equivalently, (x(x) > 1, the marginal predictive probability Ps (v)x)
237
3h
238
(1) v (=(())/as(zz) has its first two moments defined, [[y|a] (x( and Var = B (x)((a(((x) - 1), pro- Figure 3: PGM for regression viding explicit mean and uncertainty estimates with a single forward
I...,
239
241
pass in the single layered, fully connected, ag. Bo and jig networks used here. To ensure definition of both the posterior distribution and of the marginal distribution's 's variance, the parameter maps use soft-plus activation on their last layer to ensure positivity, and the 0g network further shifted by 1.
243
244
The unique dependence of the posterior on the inputs implies that the generation of pseudo-inputs should only rely on the input density. In a general regression setting, unknown, and we estimate it here prior to training with Bayesian Gaussian mixture model (Bishop 2006). We refer to it henceforth as dissipative variational variance (d-VV). The specific implementation details are listed in Section II of the supplementary materials.
245
247
3.1.1 Toy regression
The desiderata for method clear: capture of the data heteroscedastieityy extrapolation to higher uncertainty level, no underestimation the predictive uncertainty, and posterior extrapolation to the prior out-of-distribution. [Skafte et al. [209 first showed the toy regression task, y =
251
253
x sin(x) + 0.3 c1 + 0.3 x 62, where 61, 62 ~ A (0. 1). that amongst a collection of methods, only their first three expectations. Fig. F demonstrates that our more principled approach also fulfills all our requirements, without the need for arbitrarily enforcing the desired extrapolation
proposed variance network architecture could realise
255
architecture. The importance of -at-o--distribution training also revealed as the standard variational variance approach fails to produce uncertainty estimates that extrapolate correctly and robust distributional shift (bottom of Fig.
257
See Section II of the supplementary materials for the expression of the true posterior.

---

Table 1: UCI benchmarks. Each square shows the performance of given model (rows) on given dataset (columns). The intensity of the colouring represents the certitude that the associated model performed best on the given dataset. Grey rows mean impossible evaluation for a metric.
kmppyla)
RMSB (el))
RMSE
Valfyle), (v pix))"
RMSE|p. 5)
uer
Shined ver
1:V-V VV PIG)
3: prior)
Mean-variance network
5: Deep ensemble
6: MC Dropout
7: Skaffe
Table 2: Evaluation of the generative modeling. For each dataset, we report mean + std over trials.
FashionMNIST
SVHN
CIFAR
VAE
2215.54 + 68.81
4304.90 + 58.45
2930.64 + 14.82 2668.85 13.23
log p()
d-V3AE
2349.71
11.80
4133.41 + 64.28
VAE
0.171
+ 0.003
0.097 + 7e-4
0.154 + 5e-4
RMSE(X, X)
d-V3AE
0.158 0.003
0.087 0.002
0.129 + 7e-4
V3/a Valla
Decomposition of the model and data uncertainty. Fig.
260
presents the decomposition of the predictive uncertainty. The aleatoric component captures the heteroscedastic increase of
261
282
uncertainty in the training data while the epistemic uncertainty, constant in distribution, extrapolates higher values. The
1.8
283
264
proposed method therefore demonstrates, to the best of our
0.4
285
knowledge, principled decomposition of uncertainty factors.
10
Figure 5: Alcatoric (yellow) and epistemic (dark) uncertainties.
3.1.2
UCI Benchmarks
Real world regression datasets from the UCI repository are used evaluate model against curated baselines, analogically to the setup from Hernández-Lobate and Adams (2015) [kkatee
268
al. 2019 As revealed by the summarising Tab. M our method retains the mean predictive power of the variational variance method. The log-likelihood and RMSE of variance and sample residuals further show the improved calibration resulting from the imposition of a prior the variance, as the VV methods generally outperform the MLE Student-t (VV (no prior)) that shares the same architecture. The table thus proves that holistically, the dissipative loss strengthens the variational variance model's performance, which itself generally surpasses the chosen baselines.
270
272
273
274
The robustness of the methods to distributional shift is further evaluated as in Foong al. 2019]. For each input feature, a hole is created the training data by assigning the middle third of observations the test set, when sorted w.r.t that feature. Interestingly, slightly improves under the shift, highlighting the robustness benefits of the OOD prior regularisation.
276
that our method's calibration
278
We note that both MC Dropout and the combined method of Skafte et al. 2019 generally perform well, confirming their interest for regression tasks requiring uncertainty quantification, but the former's calibration is not robust to data shifts, as is also reported in Ovadia et al. (201)) and the
280
282
latter is in practice difficult to tune and lacks principle.
"https//archive.iss.uu
5 SSee Sec. II. details about the chosen baselines, datasets and implementations specifies.
'The full numbers are included in Sec. II. of the supplements.
7

---

283
3.2
Generative models
284
We extend the evaluation of our proposal to the case of generative models through the lens of VAEs KKngma and Welling 2013 Rezende et al. 2014). Variational auto-encoders infer a low dimensional
285
latent encoding of the data 2 E K", on which is conditioned the generative process pe (x|2). Its predictive uncertainty, which evaluates the confidence of the model in its ability to adequately
286
286
reconstruct inputs is known to be untrustworthy.
289
In the case of continuous or seemingly continuous in- puts, the adoption of Gaussian decoder (x[2))
280
Zn
og
N (x)uo(=), ()) results in an ill-defined model likelihood
N
292
Matte1 and Frellsen 2018a that encourages decoder variance
283
collapse, making the training of the model notoriously harder
294
Skafte et al. 2019). Most implementations therefore choose
295
to fix the variance to set level e.g (2) = 0.1, or elude the
296
challenge by adopting a Bernoulli likelihood.
N
Motivated by our previous results, we aim to demonstrate that VAES, whose decoder is fitted with our method, are 298
%
1,..., NV
able to provide robust uncertainty estimates. Assuming latent generative precision, the latent variables of the model decomposed into
300
pa(x, 2]
90(2/b))
{2, A}. with the latent input representations. The marginalisation of the Gamma distributed
302
Figure 6: PGM for V3AE
303
latent variance results in Student-T decoder, as detailed in
304
F [T] The overall architecture of the variational variance variational auto encoder (V3AE) [Stirn
305
and Knowles 2020 is shown in Fig. 6 and yields, with the addition of our out-of-distribution pseudo-inputs training, the dissipative loss function
308
Loss(go. Duain) =
Σ L(gs, 0;x) (x))(=) [DKL (AI()
(5)
D
307
Because only the decoder is regularised, the pseudo-inputs lie in the space of latent representations, D {&x jKK IRP The distribution of training inputs is therefore readily accessible as the aggregate variational posterior (=(=/Dirin)) = (o(2|x1) ... yo (=(xx)) Here again, we rely on a split
308
309
310
training procedure to leverage this perk: the encoder parameter maps ne and σee as well as the decoder mean H& are first trained until convergence, allowing the generation of the out-of-distribution pseudo-inputs and subsequently, the training of the decoder variance.
311
312
313
Image data. We evaluate the performance of our proposed dissipative V3AE (d- V3AE) against fully Gaussian VAE image data, coming from MNNSI SVHN and CIFARIO. For both models, all parameter maps share the same underlying architecture, with the addition of either a softplus and/or a shifting last layer to ensure definition of both the variational and the generative distribution's 's moments
FaSS
Input
315
VAE d-V3AE
316
317
318
SVHN
319
Tab. 2 compares model performance on two metrics, the log-likelihood and the RMSE between the original inputs and reconstructed samples X, where x ~ (x]A, 2). =) ~ 4o (e((,, (A, 2 x|)). Unlike most previous imple- mentations, we focus on actual samples, and not the mean, of the generative distributions. This comparison emphasize the cooperation between the
3 9
Input
321
3
1
VAE
322
d-V3AE
323
CIFAR
decoder's 's mean and variance, allowing evaluation of the models' uncer- tainty estimates. Our method both qualitatively (Fig. Z and quantitatively improves Gaussian VAE's sampling ability. The prior smoothens the uncertainty estimates, resulting in more realistic and less crisp samples. The log- liellhoossss evaluated test time using truncation, i.e. Ptrunc (x)
325
Input
326
VAE
327
d-V3AE
328
329
Pe (x() - (O)), to account for the finite support of data, reveal
Figure/ 7: that our model can achieve better fit, if the prior is selected correctly. In samples
Generated
330
331
SVHN and CIFARIO, 10, the presence of color channels complicates the selection prócess and challenges
332
our choice of single homoscedastic prior for all pixels and channels. We note that the dissipative loss
333
also applies to classic VAEs with Bernoulli-only decoders; see Sec. III. of the supplements for details.
? The derivation the dissipative loss function is provided in Sec. of the supplementary materials.
Immllementation details are provided in Sec. II of the supplementary materials.
8

---

Var x =]
Varls (ar(
/ 2 7
VAB
0.20
0.0
0.35
7 2
0.30
Figure 8: Decoder's aggregated variance (left) and generated samples (right) from the latent space. Coloured points corre- spond to latent representations of test data, with per-class colours.
Figure Effect of encoder refitting on the latent repre- sentations (left) and corresponding samples (right). OOD inputs (first rows, x)) initially result in in-distribution sam- ples (sccond rows, Ø). The refitted encoder displaces the encodings (coloured trajectories). modifying the gener- ated samples (third rows, Freaa).
VAB
d-V3AE
Trae Poetove Doze
VAI AIROCC - 0.157
d-VIAE
AUROC - 0.522
Joghbss/dmss
Piles
Figure 10: Empirical densities of likelihoods for FashionMNIST (ID) and MNIST (OOD). The clear
separation of distributions offered by our method is reflected in the high AUROC shown on the right.
334
Applications of robust generative uncertainty. In Figs. & the colouring of the 2D latent space represent the aggregated decoder variance ΣØ= (o
335
clear that our method displays more regular uncertainty estimates, and provides the extrapolation guarantees we strove for. Beyond in- creased robustness and better generative power, this unlocks meaningful out-of-distribution detection.
336
337
heating previous tffte-of-the-art [Havtorn et al.) 202T]. For Figs. & TO as argued in Mattei and Frellsen [2018h), we refit test time the encoder of models trained on FaShionMNIST on MNIST. The regularity and structure of the decoder variance rewards the encoder for learning to place represen- tations of OOD data outside of the region of in-distribution latent encodings, resulting in model that aware of its own inability to reconstruct plausible data, as displayed by the row Trefit of d-V3AE.
339
341
4
Conclusion
344
We have introduced a novel loss, the dissipative loss, that leverages artificial out-of- distribbuttt pseudo-inputs for learning robust uncertainty estimates. We demonstrate through Bayesian approach that casts a prior distribution over the model's 's variance a principled mechanism for controlling the extrapolation properties of neural networks governing the predictive uncertainty. Our experimental results reflect the benefits of our principled and scalable approach, displaying better calibrated and more robust uncertainty estimates, while matching the predictive power of known baselines. Finally, and most interestingly, our approach can instill into probabilistic models a notion of their own ignorance, increasing their ability to know what they don't 't know.
346
348
349
350
351
352
Limitations. The largest limitation of our approach is that it depends on an estimate of the density of the input data. In our experience, even coarse- coarse-grained densities are sufficient significantly improve upon current approaches. However, as one rarely have guaranteed good estimates of the input density, method cannot be approached as a black-box. One exception seems be the application to VAEs, where the aggregated posterior, in our experience, always provide a suitable density estimate.
354
356
Negative societal impact. Improving the ability of predictive models to assess their own confidence 358
solely a positive contribution as can help alleviate potential consequences of incorrect predictions. We are therefore not aware of any potential negative impacts of our work.

---

References
380
361
Keyulu Xu, Mozhi Zhang, Jingling Li, Simon S Du, Ken-ichi Kawarabayashi, and Stefanie Jegelka. How neural networks extrapolate: From feedforward to graph neural networks. arXiv preprint arXiv:2009. 1848, 2020.
363
364
Dario Amodei, Chris Olah, Jacob Steinhardt, Paul Christiano, John Schulman, and Dan Mané. Concrete problems in ai safety. arXiv preprint ar0iv: (60606666 2016.
365
366
I Lars Kai Hansen and Peter Salamon. Neural network ensembles. IEEE transactions on pattern analysis and machine intelligence, 12(1009993-100, 1990.
368
Balaji Lakshminarayanan, Alexander Pritzel, and Charles Blundell. Simple and scalable predictive uncertainty estimation using deep ensembles. In Advances in neural information processing systems, pages 6402-6413, 2017.
370
371
David JC MacKay. A practical bayesian framework for backpropagation networks. Neural compula- fion, 4(3):448 472, 1992.
372
375
Yarin Gal and Zoubin Ghahramani. Dropout as a bayesian approximation: Representing model uncertainty in deep learning. In international conference on machine learning, pages 1050-1059, 2016.
375
376
Christopher M. Bishop. Pattern Recognition and Machine Learning. Springer, August 2006. ISBN 0387310738.
377
378
Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q Weinberger. On calibration of modern neural networks. arXiv preprint 1xXiv:7706004577 2017.
380
Dan Hendrycks and Kevin Gimpel. baseline for detecting misclassified and out-of-distribution examples in neural networks. arXiv preprint 1r1.v2161..02166, 2016.
381
Eric Nalisnick, Akihiro Matsukawa, Yee Whye Teh, and Balaji Lakshminarayanan. Detecting out-of- distribution inputs to deep generative models using typicality. arXiv preprint arXiv: 1ed:::006009944 2019.
383
385
Anh Nguyen, Jason Yosinski, and Jeff Clune. Deep neural networks are easily fooled: High confidence predictions for unrecognizable images. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 427-436 236,436,22 2015.
386
387
388
Christos Louizos and Max Welling. Multiplicative normalizing flows for variational bayesian neural networks. arXiv preprint /733:: 2017.
389
390
Nicki Skafte, Martin Jørgensen, and Søren Hauberg. Reliable training and estimation of variance networks. In Advances in Neural Information Processing Systems, pages 6326-6336 2019.
Kimin Lee, Honglak Lee, Kibok Lee, and Jinwoo Shin. Training confidence-calibrattee classifiers for detecting out-of-distribution samples. arXiv preprini 1rXiv:1711.09322. 2017.
393
394
Keyulu Xu, Mozhi Zhang, Jingling Li, Simon S. Du, Ken ichi Kawarabayashi, and Stefanie Jegelka. How neural networks extrapolate: From feedforward to graph neural networks, 2021.
395
396
David A Nix and Andreas Weigend. Estimating the mean and variance of the target probability distribution. In Proceedings of 1994 ieee international conference neural networks (ICNN"94), volume 1, pages 55-60. IEEE, 1994.
398
399
Spren Hauberg. Only bayes should learn manifold (on the estimation of differential geometric structure from data), 2019.
400
401
Georgios Arvanitidis, Lars Kai Hansen, and Søren Hauberg. Latent space oddity: on the curvature of deep generative models. arXiv preprint arXiv:1710.11332 2017.
402
403
Diederik P Kingma and Max Welling. 1rXiv:1312.6114,200 2013.
Auto-encoding variational bayes. arXiv preprint
10

---

405
Danilo Jimenez Rezende, Shakir Mohamed, and Daan Wierstra. Stochastic backpropagation and approximate inference deep generative models. arXiv preprint arX4::1401.4082, 2014.
406
407
Armen Der Kiureghian and Ove Ditlevsen. Aleatory or epistemic? does it matter? Structural safety, 31(2)105-112, 2009.
408
409
Burr Settles. Active learning. Synthesis lectures on artificial intelligence and machine learning, 6(1):
1-1 2012.
411
Jonas Močkus. On bayesian methods for seeking the extremum. In Optimization techniques IFIP technical conference, pages 400-404. Springer, 1975.
413
Carl Edward Rasmussen and Christopher (Adaptive Computation and Machine Learning). The MIT Press, 2005.
Williams. Gaussian Processes for Machine Learning
414
415
Alex Graves. Practical variational inference for neural networks, Advances in neural information processing systems, 24:2348-2356, 2011.
416
Charles Blundell, Julien Cornebise, Koray Kavukcuoglu, and Daan Wierstra. Weight uncertainty in neural networks. arXiv preprint arXiv: 2015.
418
José Miguel Hemáández-Lobao and Ryan Adams. Probabilistic backpropagation for scalable learning of bayesian neural networks. In International Conference on Machine Learning, pages 1861-1869, 2015.
420
422
Leonard Hasenclever, Stefan Webb, Thibaut Lienart, Sebastian Vollmer, Balaji Lakshminarayanan, Charles Blundell, and Yee Whye Teh. Distributed bayesian learning with stochastic natural gradient expectation propagation and the posterior server. The Journal of Machine Learming Research, 18 (1):3744-3780, 2017.
424
426
Max Welling and Yee W Teh. Bayesian learning via stochastic gradient langevin dynamics. In Proceedings of the 28th international conference on machine learning (ICML-I1), pages 681-688
427
428
2011.
429
Jost Tobias Springenberg, Aaron Klein, Stefan Falkner, and Frank Hutter. Bayesian optimization with robust bayesian neural networks. Advances in neural information processing systems, 29:
430
4134 4142, 2016.
432
Leo Breiman. Bagging predictors. Machine learning, 24(2):123-440, 1996.
433
Jeppe Thagaard, Søren Hauberg, Bert van der Vegt. Thomas Ebstrup, Johan D. Hansen, and Anders B. Dahl. Can you trust predictive uncertainty under real dataset shifts in digital pathology? In Medical Image Computing and Computer-Assisted Intervention (MICCAI), Lima, Peru, October 2020.
435
Yaniv Ovadia, Emily Fertig, Jie Ren, Zachary Nado, David Sculley, Sebastian Nowozin, Joshua Dillon, Balaji Lakshminarayanan, and Jasper Snock. Can you trust your model's uncertainty? evaluating predictive uncertainty under dataset shift. In Advances in Neural Information Processing Systems, pages 13991-14002, 2019.
437
439
440
Leo Breiman. Random forests. Machine learning, 45(1):5-32, 2001.
Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. Dropout: a simple way to prevent neural networks from overfitting. The journal of machine learning research, 15(1):1929 19588 2014.
442
443
444
Shiyu Liang, Yixuan Li, and Rayadurgam Srikant, Enhancing the reliability of out-of-distriibtioo image detection in neural networks. arXiv preprint (.Xx:17706.06600 2017.
445
446
Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. arXiv preprint arXiv: 1rXiv::442..5//1 /41.6572, 2014a.
448 Dan Hendrycks, Mantas Mazeika, and Thomas Dietterich. Deep anomaly detection with outlier exposure, arXiv preprint arXiv:1812.00 2018.

---

450
Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. Advances neural information processing systems, 27:2672-2688) 2014b.
451
Zihang Dai, Zhilin Yang, Fan Yang, William W Cohen, and Russ R Salakhutdinov. Good semi- supervised learning that requires a bad gan. In Advances in neural information processing systems, pages 65 2017.
454
Andrew Stir and David A Knowles. Variational variance: Simple and reliable predictive variance parameterization. arXiv preprint arXiv:2006. 04910, 2020.
457
458
Alexae-Alexandre Mattei and Jes Frellsen. Leveraging the exact likelihood of deep latent variable models. In Advances in Neural Information Processing Systems, pages 3855-3866, 2018a.
459
460
Andrew Gelman, John B Carlin, Hal $ Stern, David Dunson, Aki Vehtari, and Donald B Rubin. Bayesian data analysis. CRC press, 2013.
Martin Jørgensen. Stochastic Representations with Gaussian Processes and Geometry. PhD thesis, Technical University of Denmark, 2020.
463
464
Elisa Lee and John Wang. Statistical methods for survival data analysis, volume 476. John Wiley & Sons, 2003.
466
Ashish Shrivastava, Tomas Pfister, Oncel Tuzel, Joshua Susskind, Wenda Wang, and Russell Webb. Learning from simulated and unsupervised images through adversarial training. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 2107-2116, 2017.
487
468
469
Danilo Rezende and Shakir Mohamed. Variational inference with normalizing flows. International Conference on Machine Learning, pages 1530-1538. PMLR, 2015.
470
George Papamakarios, Theo Pavlakou, and Lain Murray. Masked autoregressive flow for density estimation. arXiv preprint arXiv:170S.07055 2017.
472
473
A Philip Dawid. The well-calibrated bayesian. Journal of the American Statistical Association, 77 (379):605-610, 1982.
475
David M Blei, Alp Kucukelbir, and Jon D McAuliffe. Variational inference: A review for statisticians. Journal of the American statistical Association, 112(518):859 877, 2017.
476
477
Andrew YK Foong, Yingzhen Li, José Miguel Hernández-Lobato, and Richard E Turner. 'in- between' uncertainty in bayesian neural networks. arXiv preprint arXiv: 1537, 2019.
478
479
Jakob D. Havtorn, Jes Frellsen, Søren Hauberg, and Lars Maaløe. Hicrarchical vaes know what they don't don't know, 2021.
P.-A Mattei and Frellsen. Refit your encoder when new data comes by. In 3rd NeurlPS workshop on Bayesian Deep Learning, 2018b.
482
483
Norman L Johnson, Samuel Kotz, and Narayanaswamy Balakrishnan. Continuous univariate distri- butions, volume 1, 2nd Edition. John wiley & sons, 1994.
484
485
Christian Bauckhage. Computing the kullback-leibler divergence between two generalized gamma distributions. arXiv preprint
486
66553, 2014.
12

---

Checklist
487
488
1. For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's
490
contributions and scope? [Yes]
481
(b) Did you describe the limitations of your work? [Yes] Yes, we have included separate
492
subsection on the limitations. See Section !!
493
(c) Did you discuss any potential negative societal impacts of your work? [Yes] Covered
specifically in Section I
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes] Yes.
496
497
2.
If you are including theoretical results...
498
(a) Did you state the full set of assumptions of all theoretical results? [Yes] Our results are obtained through empirical evidence. We do, however, include relevant derivations in the supplementary material.
499
500
501
(b) Did you include complete proofs of all theoretical results? [N/A]
502
3. If you ran experiments...
(a) Did you include the code, data, and instructions needed reproduce the main ex- perimental results (either in the supplemental material or as a URL)? [Yes] Code is included in the supplementary material.
604
508
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)? [Yes] Yes, every experiment is accompanied by a separate settings file in yaml format.
507
508
509
(c) Did you report error bars (c.g., with respect to the random seed after running experi- ments multiple times)? [Yes] Each experiment
run at least in triplicate.
(d) Did you include the total amount of compute and the type of resources used (e.g.. type of GPUs, internal cluster, or cloud provider)? [Yes] Yes, in supplementary material.
512
513
4. If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
514
(a) your work uses existing assets, did you cite the creators? [Yes]
515
(b) Did you mention the license of the assets? [Yes] We ran our experiments on standard, well known datasets, reference the source, which itself includes licenses.
517
(c) Did you include any new assets either in the supplemental material as URL? [Yes] Our code is available in the supplementary material
518
519
(d) Did you discuss whether and how consent was obtained from people whose data you're using/curating? [N/A] We did not run any experiments on personal data.
520
521
(c) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content? [N/A]
5. If you used crowdsourcing conducted research with human subjects...
524
(a) Did you include the full text of instructions given to participants and screenshots, if applicable? [N/A]
525
526
(b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable? [N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation? [N/A]
$29
13


Reviews:
[{"review_id": "8Pz3HoB5JRX", "reviewer": "Reviewer_7t1Z", "summary": "The paper proposes a method for learnable uncertainty estimation using neural networks without an explicit requirement of known out-of-distribution data. Instead, the model is trained to maximize model fit where there is explicit training data and maximize uncertainty on estimated “pseudo-inputs”. A key contribution of the method is in how the “pseudo-inputs” are generated and crafted. The authors propose a “Pseudo-Input Generator (PIG)” which generates samples based on the training data density -- locations with lower data density will receive more pseudo-inputs. Results are demonstrated on a variety of tasks with increasing complexity ranging from UCI benchmarks to image generation tasks. \n", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "**Positives**:\n\nA. Empirical results are both comprehensive (in terms of the datasets and data modalities tested), but also demonstrate strong advances of the proposed method. The authors do a great job at capturing a diverse range of tests, from UCI benchmarks, to synthetic regression data, and higher-dimensional image generation tasks. \n\nB. The result on generative models presented an excellent application of the proposed method beyond only prediction but also towards density estimation. While generative models frequently model aleatoric uncertainty, the ability and demonstration of also epistemic uncertainty is a great contribution. \n\nC. The optimization of placement of pseudo-inputs is a clever approach and contribution. Performing this optimization prior to training is a great observation. Did you experiment with an adaptive approach (updating throughout training) which aims to focus on certain regions of the data space depending on the model’s performance during training. \n\n**Negatives:**\n \nThe main two contributions, as stated by the authors, are (1) on how to generate reasonable pseudo-inputs in low-density regions of the input space which neighbors training data and (2) a dissipative loss which can jointly fit the in-domain training data while simultaneously maximizing uncertainty on out-of-distribution extrapolated regions, as determined by the pseudo inputs. \n\nE. I believe contribution (1) is novel and original; however, I think some additional discussion to other works which leverage contrastive priors to emulate the pseudo-inputs would be helpful. An example of such a work is [D. Hafner et al. “Noise contrastive priors for functional uncertainty” UAI 2020]\n\nF. Contribution (2) on the other hand, overlaps significantly with prior work which is not covered in the related work section (despite this section being otherwise very thorough and well written). Specifically I want to raise the author’s attention to the methods of prior networks ([A. Malinin et al. “Predictive Uncertainty Estimation via Prior Networks” NeurIPS 2019]) and evidential learning ([A. Amini et al. “Deep evidential regression” NeurIPS 2020] and [M. Sensoy et al. “Evidential deep learning for …” NeurIPS 2019]). Specifically, the proposed dissipative loss (Eq. 4) is extremely similar to the prior networks training objective which also aims to jointly fit in-domain data while inflating uncertainty on the OOD data. However, a key difference between the proposed method and prior network OOD objective is that prior networks assume the OOD data is given, whereas the proposed method provides an algorithm for generating these. Furthermore, evidential methods (cited above) also leverage in-domain data to “simulate” and act in place of explicit OOD data, but again follow a similar OOD objective to inflate uncertainty on the extrapolated regions. The novelty of the proposed method (and lack of comparisons to both evidential regression and prior networks) in light of these works is my main criticism of the work. Greater discussion explaining the relation to these works would help alleviate my concerns of the contribution. \n\n**Minor**: \n\nG. Figure 3 and 4 are out of order on the page. \n\nH. Add quantitative  x-tick values to Figure 2. The legend references quantitative x-domains but these are meaningless without proper labeling on the plot. \n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Yes", "review_text": "", "extra_scores": {"ethical_concerns": "N/A", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "4"}, "extra_text": {}}, {"review_id": "7wYQRt3KxKs", "reviewer": "Reviewer_egD5", "summary": "The authors propose using pseudo-inputs, drawn OOD, during training to address unwanted extrapolation properties.These pseudo-inputs encourage the model to remain close to its prior away from the training data. This algorithm is then evaluated on regression and generative modeling tasks.", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Originality: To the best of my knowledge, the idea of combining neural networks and pseudo-inputs is novel. Section 1, which discusses related work, was especially well written. One suggestion is that the definition of OOD that is used in the paper given Algorithm 1, seems to be input points that have low density under the data distribution. It might be useful to mention this earlier in the paper, when OOD is introduced.\n\nQuality: The regression experiments in 1D (Figs. 1, 2, and 4) are helpful to build intuition, but I did not find the argument in Sec. 2.2 that only some pseudo-inputs near the training data are needed to correctly extrapolate everywhere convincing in higher-dimensional input spaces.\n \nI think an ablation study wrt. to Algorithm 1 or PIGs would improve the paper. There are many ways to generate OOD pseudo-inputs, and it’s not clear whether Algorithm 1 is crucial to the dissipative loss. Similarly the sensitivity to the number of pseudo-inputs is not explored.\nClarity: \n- I think the connection between pseudo-inputs and priors in Gaussian processes could be elaborated on to improve the motivation.\n- What is the maximum in Algorithm 1 as this is over vectors? How sensitive is the output of the algorithm to the number of iterations and tolerance? \n- Table 1 is very difficult to read. It’s not clear which model performed best from the intensity of the coloring\n\nSignificance: I like how simple a prescription training on pseudo-inputs is, and the results presented are encouraging. I feel that they would be of interest to the NeurIPS and robust deep learning communities. The significance of the paper would be improved further by benchmarking the method on the OOD calibration tasks of Ovadia et al., but I understand this may be beyond the scope of this work.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Yes, the authors include two sections at the end of the paper to address this.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "6"}, "extra_text": {}}, {"review_id": "5GvSu288ysZ", "reviewer": "Reviewer_6v5r", "summary": "The authors propose an approach (called d-VV) for learning better uncertainty estimates for neural network models. The core idea is a variation (pun not intended) on the variational variance work by Stirn and Knowles (2020) where an additional \"dissipative\" term is added to ELBO, which enforces that the latent precision stay close to its prior outside the training distribution as well. This is done to ensure that the uncertainty estimates are accurate even on OOD inputs.\n\nThe authors go on to propose a method for emulating \"informative\" OOD samples to estimate this dissipative term. This is achieved by starting from data samples and taking negative gradient steps on some differentiable density estimate (which is presumed to be available).\n", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The paper's writing is quite good, and it is apparent by the extensive background section that the authors have a good command of the topic at hand. d-VV is well motivated and clearly exposed, and there is a clear effort from the authors to conduct a thorough empirical evaluation on the benchmarks they chose (although the results are not always communicated in the clearest fashion).\n\nTo me what holds the paper back is the lack of experiments on more \"realistic\" benchmarks: the largest datasets are SVHN/CIFAR and the biggest model is a 5 layer CNN. It appears that the main drawback of d-VV is that it necessitates the introduction of auxiliary models to (1) compute the variance through the latent code and (2) yield density estimates for deriving OOD pseudo inputs. Whether the associated computational overhead precludes the use of the method in large scale settings is not apparent in the current version of the paper.\n\nThis is why I think the paper is borderline, although I am willing to change my score to a more positive evaluation if some of the cons below are addressed.\n\n### Pros\n\n- Well written, good background and principled approach\n- Simple concept\n- Good empirical results\n\n### Cons\n\n- Possibly lots of computational\n- Only toy-ish experiments, would be interesting to test on more realistic settings with large models\n- Visualizations are hit or miss: they rely too much on color. While the holistic evaluation in Table 1 (which is actually a figure and not a table) is interesting, I think this section is missing a \"punchline table\" with the key results (in numerical form rather than as a heatmap).\n\n### Remarks\n\n- Eq. 4 would be clearer with proper bracketing of the sum over D_train\n- L567 in the appendix: \"a single hidden layer with 50 hidden layers\" (I assume you mean 50 hidden units)\n\n \n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "The authors addressed these concerns appropriately.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "4"}, "extra_text": {}}, {"review_id": "2prZT9X0oeY", "reviewer": "Reviewer_23bM", "summary": "This paper considers the idea of generating pseudo-inputs (i.e. extra augmented samples) to improve OOD uncertainty estimates by training with an uncertainty-increasing (\"dissipative\") loss at these pseudo-inputs. The pseudo-inputs are generated by minimizing estimated density under the data distribution via a simple gradient algorithm. The idea is evaluated in supervised learning (regression) and generative modeling (VAE for images).", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "While the idea presented in this paper seems reasonable, the paper comes off as incomplete scientific work as it lacks comparisons against many related methods that have been developed in this area (all using pseudo-inputs for better uncertainty). Since the paper does not present strong theoretical or conceptual advantages of their proposed method, a reader must be convinced of its merits purely from the empirical comparisons, which are currently insufficient to convince me in this paper.\n\nI find the related work too general and misleadingly presents the history as if nobody has already considered out-of-distribution pseudo-inputs training to improve uncertainty estimates.\nWhen in fact there are tons of related works that have been overlooked such as:\n\nMaximizing Overall Diversity for Improved Uncertainty Estimates in Deep Ensembles. \nJain et al. 2020.\nhttps://arxiv.org/abs/1906.07380\n\nThis paper also generates augmented data to improve uncertainty estimates by increasing their entropy at the augmented samples, and does not require a GAN. It even explicitly considered generating pseudo-inputs with low density as in your work. Should be conceptually and empirically compared against as it seems highly related.\n\nNoise Contrastive Priors for Functional Uncertainty. \nHafner et al. 2019\nhttps://arxiv.org/abs/1807.09289\n\nAlso Lee et al. (\"Confidence-calibrated...\") should be contrasted against far more, currently the only clear difference I can understand from reading your paper is that you do not require a GAN while Lee et al do, but I'm sure there are other key differences that should be described. The authors write that Lee et al require an extra GAN as a downside, but do not acknowledge their method requires an extra density estimator. Furthermore there is no discussion in this paper of the finding by Lee et al that \"boundary samples\" correspond to the best pseudo-inputs, whereas how close to the \"boundary\" pseudo inputs from your method will lie is unclear and may arbitrarily depend on hyperparameter values.\n\n\nOther related work includes:\n\nPrediction-Based Regularization Using Data Augmented Regression. \nHooker et al. 2012.\nhttp://faculty.bscb.cornell.edu/~hooker/darpaper.pdf\n\nPredictive Uncertainty Estimation via Prior Networks. \nMalinin et al. 2018\nhttps://arxiv.org/abs/1802.10501\n\nUncertainty in Neural Networks: Approximately Bayesian Ensembling\nPierce et al. 2020.\nhttps://arxiv.org/abs/1810.05546\n\nFurthermore there are a suspicious number of nan reported results for baseline methods, for which it is unclear why this is the case (and again suggests the chosen baselines in this work are not properly intended for the tasks considered).\n\nAlso how did you choose hyperparameters for baselines and your model in each task? For example in the MNIST vs FashionMNIST experiment, it seems possible to tweak hyperparameters to obtain the favorable AUROC curve depicted in Fig 10. There should be many more pairs of image datasets evaluated in this experiment using the same hyperparameters to control for hyperparameter settings and more convincingly demonstrate the superiority of your proposed method.\n\nFinally your method depends on a density estimator, so it would be good to show your chosen density estimator alone does not lead to good OOD detection and your improved VAE is truly needed. For example, perhaps a simple linear combination of density-estimate + standard VAE is just as good for OOD detection in which case your method would not really be necessary.\n\n### Update after reading author response to my original review ###\n\nI believe the paper will be much stronger after the inclusion of the additional baselines and major updates to the related work to properly reflect the historical context of this contribution. However I think this stronger paper should be submitted to the next conference for a proper review, as there are simply too many critical updates made in the revision for me to recommend acceptance based only on the brief descriptions of the changes through the comment system. In particular, the revision needs to be carefully written in such a way that precisely describes what previously existed before and what exactly is a new contribution in this paper, without over-claiming anything. \n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "I believe the authors have adequately addressed societal impact & limitations of their work.", "review_text": "", "extra_scores": {"ethical_concerns": "I don't have any remaining omitted ethical concerns regarding this paper.", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "4"}, "extra_text": {}}]

Author rebuttal:

Meta-review: The idea of the paper is interesting, however the submitted work fails to acknowledge a fair amount of related work, and the experimental evaluation fails to incorporate sufficient relevant baselines (of which there are many). Although the authors attempted to address this in rebuttal, incorporating this into the paper would require significant changes. The reviewers also noted a number of other concerns (more extensive experiments, clarity) that can be addressed at the same time. Thus I recommend the authors revise their manuscript and resubmit with these improvements.
Decision: Reject