Propagating Distributions through Neural Networks
Authors: Felix Petersen
Christian Borgelt
Mikhail Yurochkin
Hilde Kuehne
Oliver Deussen
Venue: ICLR 2022 Submitted (2022)
Keywords: propagating distributions
uncertainty quantification

Abstract: We propose a new approach to propagating probability distributions through neural networks. To handle non-linearities, we use local linearization and show this to be an optimal approximation in terms of total variation for ReLUs. We demonstrate the advantages of our method over the moment matching approach popularized in prior works. In addition, we formulate new loss functions for training neural networks based on distributions. To demonstrate the utility of propagating distributions, we apply it to quantifying prediction uncertainties. In regression tasks we obtain calibrated confidence intervals, and in a classification setting we improve selective prediction on out-of-distribution data. We also show empirically that training with our uncertainty aware losses improve robustness to random and adversarial noise.

Under review as a conference paper at ICLR 2022
PROPAGATING DISTRIBUTIONS THROUGH NEURAL NETWORKS
Anonymous authors Paper under double-blind review
ABSTRACT
We propose new approach to propagating probability distributions through neural networks, To handle non-linearities, we use local linearization and show this to be an optimal approximation in terms of total variation for ReLUs. We demonstrate the advantages of our method over the moment matching approach popularized in prior works. In addition, we formulate new loss functions for training neural networks based on distributions. To demonstrate the utility of propagating distributions, we apply it to quantifying prediction uncertainties. In regression tasks we obtain calibrated confidence intervals, and in a classification setting we improve selective prediction out-of- distribbution data. We also show empirically that training with our uncertainty aware losses improve robustness to random and adversarial noise.
INTRODUCTION
Neural networks are routinely used in applications affecting our daily lives, including safety-critical domains. As result, quantifying uncertainties in neural network decisions and improving their robustness against noise has become an important problem. A prominent example is autonomous driving (]. where neural network is not only supposed to detect and classify various objects like other cars or pedestrians on the road, but also to know how certain it is about this decision and allow, e.g., for human assistance for uncertain cases. As pointed out by many works, e.g.. by Kendall and Gal (2) or by Kiureghian et al. (3). prediction uncertainties can arise from two different sources: systematic uncertainty in the data, which is referred to as epistemic uncertainty. or random uncertainties in the data, e.g., because of noisy sensors, which is referred to as aleatoric uncertainty (4). Both types of uncertainty, epistemic and aleatoric, have received considerable attention. Works focusing on epistemic uncertainties include out-of-distribution (OOD) detection [5) e.g., via orthogonal certificates (6) or via Bayesian neural networks 7-10 Other works focus on aleatoric uncertainties, e.g., via uncertainty propagation [I 11-13 via ensembles (14), via simultaneous quantile regression (6), or also via Bayesian neural networks [15). Note that many of the aforementioned methods can be applied to quantify both types of uncertainties. Similarly, in this work, we focus on both aleatoric and epistemic uncertainties.
We consider the problem of evaluating f(x + c), where f is neural network, x: is an input data point, and e is a random noise variable. Therefore, f(x + <) is a random variable, where its mean used for prediction, and its variance for quantifying the uncertainty. This perspective allows assessing the sensitivity of the neural network for uncertainty quantification. Aleatoric uncertainty due to input measurement errors can be estimated by modeling how much prediction changes under a respective input uncertainty. Incorporating the sensitivity of a neural network during training can also allow epistemic uncertainty quantification: when the output variance regularized (as in our proposed loss function), the variance of predictions for observed data is minimized. Thus, for the regions where data does not provide sufficient information, the variance can be larger. We visualize the uncertainty quantification on toy problem in Fig. [ on the left our method identifies high uncertainty in the regions where samples from two classes overlap corresponding to aleatoric uncertainty; on the right, we capture epistemic uncertainty due to many possible decision boundaries separating the two classes. See figure 6 in Hüllermeier et al. [16] for an analogous example with linear models.
Due to the high complexity of neural networks, specifically many non-linear functions, it is intractable to compute the distribution of f(= + e) analytically. A straightforward way to approximate the mean and (co-) (co-)yariances of [x +e) is to use the Monte Carlo method. Unfortunately, this is not suitable for

---

Under review as a conference paper at ICLR 2022
Figure Aleatoric (left) and epistemic (right) uncertainty estimation. Yellow indicates a large estimated uncertainty and blue indicates small estimated uncertainty.
the problems we consider: evaluating a neural network f many times computationally prohibitive, and the quality of the approximation for the (co-) vvrrannrs as well as the quality of the gradients deteriorates quickly when the data is nigh-dimensional, e.g., images, Instead, we consider analytical parametric approximations of [ar + e). The key challenge is to approximate the transformations of distributions by non-linearities such as ReI.U. The widely used moment matching technique [17) computes mean and variance of the corresponding transformed distribution and uses Gaussian distribution with the same mean and variance as the approximation. Moment matching can also be applied for other distributions of the exponential distribution family (18). However, this procedure requires that the moments are defined and finite, which is not always the case, e.g., for Cauchy distributions. Also, cannot be applied conventionally pre-trained deep neural networks. In addition, in our empirical studies, we observed that training with moment matching is numerically unstable, requiring careful hyperparameter tuning obtain meaningful results.
Contributions
We propose an approximation of a distribution transformed by a ReLU non-lincarity that minimizes the total variation distance to the true distribution. Empirically, we show that this approximation is also effective in approximating transformations with other popular activation functions, and applicable multivariate and multilayer cases.
We propose a new loss function for learning with Gaussian and Cauchy input distributions.
We show that our method can quantify aleatoric uncertainty by obtaining calibrated confidence intervals in regression and epistemic uncertainty by improving selective prediction in classification.
We demonstrate that method improves robustness against random and adversarial noise.
Our PyTorch-based
19 framework for propagating distributions will be publicly available.
2
RELATED WORK
Several approaches have been suggested to introduce some treatment of uncertainty into neural networks. They can be grouped into three broad categories, although there are also hybrid approaches.
The first category tries model uncertainty via sampling. Here, we find, among others: Variational inference for neural networks [9], tractable approximation to Bayesian inference for neural networks;
Bayes by Backprop [8), which learns a probability distribution by Monte Carlo sampling to introduce uncertainty in the weights of the network; Monte Carlo dropout (20). which estimates models'
prediction uncertainties by applying dropout (2I) at test time. An overview of current techniques for Bayesian deep learning in the context of computer vision models can be found in Gustafsson et al. [22]. For aleatoric uncertainties Bouchacourt et al. (23) minimize the dissimilarity coefficient between the true and an estimated distribution (modeled by neural network). The drawback of these methods the computationally expensive inference, i.c., to make a single prediction they require many forward passes of neural network.
2

---

Under review as a conference paper at ICLR 2022
The second category of approaches concentrates modeling uncertainties without sampling, i..., analytically. These works approximately propagate normal distributions at individual training samples
through the network to model the network response to perturbed inputs (TT 12 18 24]. Wang er
al. (18) propose Natural- Paramm-rr networks, which allow using exponential-fane
distributions
to model weights and neurons. Similarly, Wang et al. [25] and Postels er al. (26) uncertainty
propagation to sampling-free approximate dropout. These methods use moment matching for propagating parametric probability distributions through neural networks, we will discuss in greater detail subsequently. Wu et al. (27) propose an approximation to moment matching of Gaussian distributions with covariance for ReLU non-linearities to provide deterministic variational inference approximation (DVIA) for Bayesian neural networks.
The third category of approaches estimates uncertainties by training neural network to predict not only an output value but also the output value's 's uncertainty. Lakshminarayanan et al. [14] use neural networks for predicting Gaussian distributions instead of individual values and extend this idea with deep ensembles. Tagasovska et al. [6] learn conditional quantiles for aleatoric uncertainty estimation and propose orthonormal certificates for quantifying epistemic uncertainty. We use several methods from this category as baselines in our experiments.
Our approach falls into the second category and is most similar to Gast et al. [T]] and Shekhovtsov et al. [12] Our approach differs in how distributions are passed through non non-lincarities such as ReLUs. Previous work relies on moment matching [17]. also known as assumed density filtering. Moment matching is technique where the first two moments of distribution, such as the output distribution of ReLU (i.e., the mean and variance) are computed and used as parameters for normal distribution approximate the true distribution. As it otherwise usually intractable, moment matching assumes diagonal covariances. In our approach, we propagate normal distributions by an approximation that minimizes the total variation (TV) distance for ReLU activations. We find that our approximation is faster to compute, a better approximation with respect to TV, and allows propagating full covariances. Further, as we do not rely on moment matching, we can also propagate Cauchy distributions for which the moments are not finitely defined. Another advantage of our method is that it can be applied to pre-trained models. On the other hand, moment matching changes the mean such that there are deviations from what a network without uncertainty propagation would compute, leading to poor predictions when applied to a pre-trained network. Lastly, instead of approximating the distribution of the output classes for the loss function [11 12]. we propose new loss function suitable for distributional outputs.
3 PROPAGATING DISTRIBUTIONS THROUGH NEURAL NETWORKS
For propagating parametric distributions through neural networks, we consider affine transformations and non-lincarities, such ReLU, separately. For affine transformations, exact computation of the parametric output distribution is possible due to the reproductive property of Gaussian and use local linearization and show that this approximation
Cauchy distributions. For non-linearities, minimizes the TV distance to the true intractable distribution for ReLUs. Recall that in our setting, the input to neural network is a random variable and the weights are learned parameters.
3.1 AFFINE TRANSFORMATIONS
Fully connected layers as well as convolutional layers are affine transformations of their inputs.
For fully connected layers, we the notation y =AA + b where A mm is the weight matrix and b E IR 1 is the bias vector with n, m E N4.
As convolutional layers can be expressed as fully connected layers, discuss them in greater detail in Supplementary Material and only discuss fully connected layers here.
Given maltivariate normal distribution X (n, E). / E Rixa E Runn X can be transformed by a fully connected layer via μ PAA b and + AEAT.
Given a multivariate normal distribution without covariances X NN o² u Rixa R Xn. X can be transformed by a fully connected layer as follows: μ + UA + and d² H 2 (A2)T where 22 denotes the element-wise square.
3

---

Under review as a conference paper at ICLR 2022
Given maltivariate Cauchy distribution X Cl 7), To R
7 E Rixn X can be transformed by a fully connected layer by zo + A + b and 7 + 7 Abs(A))
Notably, in all cases, the location p/zo xo does coincide with the values propagated in conventional network layers. This allows applying this method directly to conventionally trained neural networks
Average pooling down-samples by averaging the values of pooling regions. As this is a linear combination, it can be expressed using matrix multiplications and thus needs no special treatment.
3.2 NON-LINEAR TRANSFORMATIONS
To handle non-linearities, utilize local linearization. That is, we transform the mean median and the variance / scale as follows:
(H,0) - (f(1),
(1)
for univariate distributions and as
(H. Σ) + (f(m), (())
(2)
for multivariate distributions. ReLU is the most common non-linearity for neural networks, and we now study it in more detail. Following local linearization (Eq. [ our approximation for transforming distributions with ReLUs
(1, 1 2
ReLU (U,
(0,0) otherwise
for distributions parameterized via μ and a. In fact, for Gaussian and Cauchy distributions, this approximation is optimal wrt. TV. The TV between two probability distributions, i.e., between the true distribution Q and an approximation P, is defined as
TV(P,Q) = suu J (p (p-9) ( du 2 / gl dv. .v..
(4)
A
To motivate our choice of the approximation quality metric, we note that TV is proper distance metric on probability distributions. Specifically, TV of 0 implies that distributions are the same, i.e, all their moments (including means) are equal. Devroye el al. [28] show that TV upper bounds the maximum of differences between moments for Gaussian distributions.
In the following theorem, we formalize that our approximation of the ReLU non-linearity minimizes the TV. Note that when parameterizing a Gaussian or Cauchy distribution with (0, 0), we are referring to Dirac's d distribution. An illustration of the transformed distributions and their approximations can be found in Supplementary Material A
Theorem 1. Local linearization provides the best Gaussian approximation of Gaussian distribution transformed by a ReLU non-linearity with respect to the total variation:
P p20 0 (0,00 otherwise
(,,
arg min (V(P,Q) (0,ö) where P PNN(ū Q ReLU(N((,,
(5)
Proof. We distinguish 3 cases, μ < 0, p 0, and μ >
(u 0) In the first case, the true distribution has probability mass of > 0.5 at 0 because all values below 0 will be mapped to 0 and CDF > 0.5. As the approximation is parameterized as (0, 0). it has probability mass of O. Therefore, the TV is || pol < 0.5. All parameterized distributions where o > have no probability mass at 0 and thus a TV of at least |0 P0-pol Po > 0.5.
(n == In this case, _e0h lp- ql du = because the distributions are equal on this domain. The true distribution has a probability mass of Po 0.5 O. therefore the TV 00 Pol 0.5. In fact, all distributions with 0 have a TV of at least 0.5 because |0- pol 0.5. The distribution parameterized by (0, 0) has the same TV at this point.
(n 0) In this case, Po CDF(O) 0.5. Further, again f 1p - ql dv 0. Thus, the TV of our distribution is 10 Pol Po 0.5. All distributions with > O have TV of at least po because [0- - -Pol = po- The distribution parameterized by (0, 0) has a TV of - pol > 0.5.
Thus, the approximation by linearization (Fq. is the optimal approximation wrt. total variation. n
4

---

Under review as a conference paper at ICLR 2022
Corollary 1. Theorem aalso applies to Cauchy distributions parameterized via 2o. instead of p. Proof. The proof of Theorem alsso applies here.
To demonstrate the accu-
Table I:
Simulation of propagating normal distributions through
racy of this approximation in the multi-dimensional and multi-layer case, we simulate multi-layer neural networks with ReLU ac- tivations and evaluate the
neural net. Reported is the intersection of probability mass (1 TV).
With Cov. (Ours)
DVIA 22
Without Cov. (Ours)
Moment Matching
0.1
0.9791 1 0.0202
0,9720 +: 0.0228
0.2361 0.0250
0.2611 0.0256
1
0.8747 + 0.0546
0.8519 + 0.0543
0.2243 = 0.0237
0.2195 + 0,0294
10
0.7586 + 0.0407
0.7035 + 0.0366
0.2186 = 0.0244
0.1976 + 0.0179
TV between parametric ap- proximations and Monte Carlo approximation with 10 samples representing the oracle. We use neural network with hidden layers and 100 neurons per layer with ReLU non-linearities trained with softmax cross-entropy on the Iris data set. This data set has small input and output dimensionalities, allowing us use Monte Carlo as a feasible and accurate oracle estimator of the truth. The results of this simulation are displayed in Tab.
100
0.6877 + 0.0333
0.5845 + 0.0479
0.2261 0.0282
0.1724 0.0111
1000
0.6808 + 0.0318
0.5318 + 0.0516
0.2193 = 0.0248
0.1706 0.0109
The simulation shows that propagating the covariances produces the best estimates of the output distribution. Further, the simulation shows that when discarding covariances after each layer, on average, the approximation from Eq. perrfrrss better than moment matching. Note that moment matching with covariances is intractable. Therefore, we use the deterministic variational inference approximation method (DVIA) by Wu et al. (27) in this case as a baseline, As DVIA has significantly larger computational complexity (its computational overhead is linear in the size of the largest layer; see Section 5.3 for additional details), we can compare our method only in these simulation experiments DVTA. Overall, all methods are better for small input standard deviations a. The reason for this is that larger variances cause a larger part of the distribution to be mapped to 0 (for positive means) or to positive values (for negative means), which the sources of error in the approximations. Results for additional architectures and non-lincarities are presented in Supplementary Material C.I There, we evaluate our method also for Leaky-ReLL GELU 29 and SiLU 30
LEARNING WITH UNCERTAINTY PROPAGATION
With the ability propagate parametric distributions through neural networks, can shift our focus to learning with such uncertainty propagation. Recall that the output of a neural network is now (multivariate) Gaussian corresponding to a neural network's 's output into the loss function. This is simple to achieve for regression, i.e., by using oigliielloood as the loss, however, for classification it is more challenging.
Cauchy distribution. We want to incorporate the covariances
4.1 REGRESSION
For regression (possibly with multiple outputs), we use the probability density of the predicted distribution at the corresponding value as our training objective. For normal distributions, our objective is maximizing the probability density by minimizing the negative log likelihood
log det(E) + (y - (y-
(6)
Here, the l dimensional prediction is μ with covariance matrix Σ and the ground truth value is y. For Cauchy distributions, the respective probability density can be maximized analogously.
4.2 CLASSIFICATION
More interesting than regression is the classification case, for which we propose new loss function. Previous work has proposed using moment matching of softmax (TT), Dirichlet outputs (TT), and an approximation to the (n 1)-viiitte logistic distribution [1) Instead of finding surrogate that incorporates the variance of the prediction, we use exact probabilities for classification. This allows incorporating not only the variances but also the covariances of neural network's 's prediction.
To arrive at our loss, we compute the probability of correct classification, i.e., the probability that the score for certain class is the maximum among all classes. As the exact probability of score among n > 2 scores being the maximum intractable, we resort to computing the pairwise exact probability of correct classification, which also allows us consider their covariance.
5

---

Under review as a conference paper at ICLR 2022
The exact probability of pairwise correct classification for two random variables X and Y is
P(X Y) P(X Y 0) J x CDFy-x(0)
For a multivariate normal distribution (X,Y) ~ ((((x, Σ) with covariance matrix Σ. P(X Y) = 1 + erf
Px -00x)))
(8)
22xx))
For independent normal distributions X ~ NNxxx xx) and Y~ NNuu,, og)
P(X > Y) = { erf
(2(03+003) )
ux-MY
(9)
For Cauchy distributions X ~ C(xx, 7x) and Y C(y,,T))
P(X Y) = aarctan
Ex-Ey +x+
(10)
With that, our classification training objective is maximizing where Ze takes on the distribution of the correct class score and Ze takes on the distributions of the corresponding (among k 1) erroneous class scores. We refer to this loss function the pairwise Gaussian and the pairwise Cauchy, respectively.
R- %z),
(11)
EXPERIMENTS
First, we investigate how well our method quantifies aleatoric uncertainty UCI regression tasks. Second, we test how well our method can estimate epistemic uncertainty for detection of out-of- distribution test data. Third, investigate robustness against Gaussian noise and adversarial attacks.
5.1 ALEATORIC UNCERTAINTY QUANTIFICATION
To evaluate aleatoric uncertainty predicted though uncertainty propagation, we examine its capability produce calibrated prediction intervals on eight UCI regression tasks. We follow the experimental setting of Tagasovska el al. (6] and compare our method to their Conditional Quantiles method as well as to their best baseline, Conditional Gaussians (14). Conditional Quantiles enable learning all conditional quantiles of a given target variable. Conditional Gaussian fits a conditional normal distribution, i.e., a neural network with two outputs, the mean and the variance. In comparison, our approach does not let a neural network predict the variance, but instead propagates input uncertainties through a neural network and computes calibrated prediction intervals from the output covariances.
We use the same model architecture, parame-prramttess and evaluation metrics Tagasovska et al. (6) for all three methods. The evaluation metrics are Prediction Interval Coverage Probability (PICP), i... the fraction of test data points falling into the predicted intervals, and the Mean Prediction Interval Width (MPIW). In Tab. [2] following Tagasovska et al. [6), we report the test PICP and MPIW of those models where the validation PICP lies between 92.5% and 97.5%. The goal is to achieve a narrow interval (small MPIW) while the optimal test PICP is 95%. Our method achieves the narrowest well-calibrated prediction intervals. Specifically, for 5 out of 8 data sets, method has the narrowest intervals while Conditional Gaussian as well as Conditional Quantile each achieve the narrowest well-calibrated prediction intervals on only 2 of the data sets. Notably, for the 'naval' task, our method has the cest-calibrated coverage intervals while having by far the smallest MPIW.
Table 2: Results for the aleatoric uncertainty experiment. The task is to compute calibrated prediction intervals for 8 UCI data sets. Reported are test PICP and MPIW in parentheses. For MPIW lower is better. All results are averaged over 20 runs. Prior methods are duplicated from Tagasovska et al. 6
Data Sct
Uncertainty Propagation
Conditional Gaussian
Conditional Quantile
concrete
0.92 0.03 (0.25 + 0.02)
0.94 0.04 0.03 (0.32 0.09)
0.94 - 0.03 (0.31 + 0.06)
power
0.94 094=00 0.01 (0.20 + 0.00)
0.94 (0.18 0.00)
0.93 = 0.01 (0.18 : 0.01)
0.92
(0.45 = 0.03)
0.94 (0.49 0.03)
0.93 0.03 (0.45 0.04)
yacht
0.93 0.04 (0.06 0.01)
0.03 (0.03 0.01)
0.93 0.06 (0.06 0.04)
naval
0.94 0.02 (0.02 +: 0.00)
0.96 0.01 (0.15 0.25)
0.95 0.02 (0.
0.09)
energy
0.91 0.05 (0.05 0.01)
0.94
(0.12
0.94 0.03 (0.08 0.03)
boston
0.93 0.04 (0.28 + 0.02)
0.94
(0.55
10.05:55000 0.20)
0.92 = 0.06 +
kinšum
0.95 = 0.01 (0.24
0.93 + 0.01 (0.20 0.01)
0.93 = 0.01 (0.23

---

Under review as a conference paper at ICLR 2022
04
Thdadi Dai a/Orameaa Conticaas Pedes Podiless
Triked of NSS Breadee End Pareio Polec habise
02
32
Figure 2: Selective prediction on MNIST with EMNIST letters as OOD data. Left: risk-coverage plots for off-the-shelf models trained with softmax cross-entropy, Right: models trained with uncertainty propagation. The grey line indicates perfect prediction. Results averaged over 10 runs.
Table 3: Selective prediction settings including the cove-cove AUC for Fig.
Training Objective
Uncertainty Prop.
Selective Prediction
Risk Cise-Coeerege AUC
Softmax CE
Softmax Entropy
21.4%
Softmax CE
Our Propagation
Pairwise Gaussian
20.4%
Softmax CE
Our Propagation
Pairwise Cauchy
19.2%
Softmax CE
Our Propagation
Dirichlet Dist.
32.0%
Softmax CF
Orthonormal Cert.
24.2%
Pairwise Gaussian
Our Propagation
Softmax Entropy
20.6%
Pairwise Gaussian
Our Propagation
Pairwise Gaussian
19.0%
Pairwise Gaussian
Our Propagation
Pairwise Cauchy
18.3%
Dirichlet Dist.
Moment Matching
Dirichlet Dist.
19.2%
Dirichlet Dist.
Moment Matching
Pairwise Gaussian
19.0%
Perfect Prediction
12.5%
5.2 EPISTEMIC UNCERTAINTY FOR OUT-OF- DITTOF-DISTRIBUTIOO SELECTIVE PREDICTION
Selective prediction 31 is formulation where instead of only predicting class, neural network can also abstain from prediction if is not certain. We benchmark selective prediction on MNIST 32 and use EMNIST letters [33] nut-of-distributio data. EMNIST letters is data set that contains letters from to 7. in the same format as MNIST. We train neural network on the MNIST training data set and then combine the MNIST test data set (10 000 images) with 10 000 images from the EMNIST letter data set. This gives us a test data set of 20 000 images, 50% of which are out-of-distribution samples and for which the model should abstain from prediction.
In the first setting, we train a neural network with conventional softmax cross-entropy to simulate an existing off-the-shelf network. In the second setting, train the neural network using uncertainty propagation with the pairwise Gaussian loss (ours) as well as using moment matching propagation and Dirichlet outputs (1T). We evaluate using five methods to compute certainty scores. First, the softmax entropy of the prediction, and apply Orthonormal Certificates from Tagasovska et al. (6). Second, we propagate distribution with 0.1 through the network obtain the covariances. Using these covariances, we use our pairwise Gaussian probabilities as well as the categorical probabilities of the Dirichlet outputs proposed by Gast al. [TT] compute the entropies of the predictions. In Fig. 2 we provide risk-coverage plots [3] of the selective prediction based on these scores. Risk-coverage plots report the empirical risk (i.e., the error) for each degree of coverage a. That is, we select the most certain predictions and report the error. This corresponds to setting, where predictor can abstain from prediction arca-under- unw-en-ee---uureeeeeeee (AUC) to quantify the overall selective prediction performance. Smaller AUC implics that the network is accurate on in-distribution test data, while abstaining from making a wrong prediction ditto--distribution examples. In this experiment, correct prediction on OOD data is possible because the classifier can only predict numbers while the OOD data consists letters.
a of the cases. We use risk risk-coverage
We evaluate variety of combinations of training losses, uncertainty propagation methods, and confi- dence scores for selective prediction. See Tab. for the summary and the corresponding risk-coverage AUC results. We find that training with uncertainty-awar training objective (pairwise Gaussian
7

---

Under review as a conference paper at ICLR 2022
and Cauchy losses and Dirichlet outputs) improves the out-of distriibiiitttonn detection substantially. Further, we find that the pairwise Cauchy scores achieve the best out-of-distrrbution detection on a mre-triined model. Orthonormal certificates (OC) estimate null-space of data in the latent space to detect out-of- -if-riiitiibttion examples. In our setting, digits and letters may share similar null-space, making OC not effective. The Dirichtlet distribution does not perform well on a pretrained model because it is not designed for this scenario but instead designed to work in accordance with moment matching. We also observe that, for model trained with Dirichlet outputs and moment matching uncertainty propagation, selective prediction confidence scores of pairwise Gaussian and pairwise Cauchy offer an improvement in comparison to the Dirichlet output based scores. This suggest that pairwise Cauchy scores are beneficial across various training approaches. The overall best accuracy (18.3% AUC) is achieved by training with an uncertainty-aware objective and evaluating using pairwise Cauchy. We also report AUC of perfect predictor, i.e., one that always abstains on out-of-distribution data and always predicts correctly in-distribution.
1950
Concty deeshase
Eatlhctio
Nanual Seriaiin
Serval Conticaion
43
Figure 3: Robustness of CNNs on the MNIST data set against Gaussian noise (left) as well as PGD adversarial noise (right). The continuous lines show the test accuracy and the dashed lines show the robust accuracy. The black lines indicate the softmax cross-entropy trained baseline. Results are averaged over 3 runs. Results on CIFAR-IO and further settings are in Supplementary Material
5.3 ROBUSTNESS AGAINST GAUSSIAN AND ADVERSARIAL NOISE
Robustness of neural networks has received sigmificant attention in the past years (3- There are many methods for robustifying neural networks against random noise in the inputs adversarially crafted perturbations. Our method is not explicitly designed for these purposes, however, here we empirically demonstrate that it achieves meaningful robustness improvements. This is an additional benefit to its main purpose of quantifying uncertainties.
We evaluate the robustness of classification models trained with uncertainty propagation the MNIST (32] and CIFAR- 10 (37) data sets. We consider the following training scenarios: First, we propagate normal distributions with and without covariance and train using the pairwise Gaussian loss. Second, we train with Cauchy distribution propagation using the pairwise Cauchy loss. As for baselines, we compare to training with softmax cross-entropy as well as moment matching propagation with the Dirichlet output loss (TT). The network architectures, hyper-parameters, as well as the results on CIFAR-10 10 are presented Supplementary Material C.2
Random Gaussian Noise To evaluate the random noise robustness of models trained with uncer- tainty propagation, we add random Gaussian per-pixel noise during evaluation. Specifically, we use input noise with a 0.5 (see Supplementary Material C.2 for E (0.25. 0.75)) and clamp the image to the original pixel value range (between 0 and 1). In Fig. (left,, we display results for input standard deviations scales from 10 to 10a. For every input standard deviation, we train separate models. For each method, the range of good hyperparameters is around 4 6 orders of magnitude. We observe, when training with uncertainty propagation and the pairwise Gaussian Cauchy losses, the models substantially
robust than with conventional training the MNIST data set.
Adversarial Noise To evaluate the adversarial robustness of models trained with uncertainty prop- agation, we use the projected gradient descent (PGD) attack (35] as is a popular and powerful attack. In Fig. 3 (right), demonstrate the adversarial robustness of CNNs distributions. We can see that our approach is competitive with the accuracy of the softmax cross-entropy loss and even outperform it. For measuring the adversarial robustness, use Ta bounded PGD, for
8

---

Under review as a conference paper at ICLR 2022
MNIST with 16/255 (and with 8/255 in the supplementary material), and for CIFAR with e E (3/255, 4/255] (results for CIFAR are presented in the supplementary material). Note that our uncertainty propagation, in combination with pairwise probability classification losses, outperform both baselines in each case. We find that the adversarially most robust training method is propagating normal distributions with covariances, where achieve robust accuracy of 85% with c 16/255 on the MNIST data set compared to robust accuracy of 14% for models trained with softmax cross- entropy. Note that the baseline Dirichlet loss with moment-matching [T] offers no robustness gains in this experiment. We observe that, when training with normal distributions without covariances, the models are only robust against adversarial noise for rather large input standard deviations. At the same time, this effect is inverse for random Gaussian noise, where robustness against random noise is achieved when input standard deviations are rather small.
We perform PGD on our models with our objective functions as described in Eq. 11 and PGD) on the cross-entropy trained models with the softmax cross-entropy objective. In Supplementary Material C.2 provide results where softmax cross-entropy based objective is used by the adversary: our models are more robust against the cross-entropy based attack than against the true objective based attacks which reported in the plot. Our methods are not robust as methods specifically designed for that purpose, e.g. adversarial training (35). However, our methods provide a substantial improvement over training with softmax cross-entropy.
5.4 PROPAGATING DISTRIBUTIONS THROUGII RESNETS
To validate that method also performs well for large architectures, we apply it to learning CIFAR-10 with ResNet-18 and ResNet-34 (38] architectures. Here, we train each model using Adam (39) for 400 epochs and compare same setting with the softmax cross-entropy loss in Tab. The results validate that training with the Pairwise Gaussian loss (w/ covariances) does substantially damage the performance in comparsion to softmax cross-entropy.
Table 4: CIFAR-10 performance with ResNets.
Model
Softmax CE
Pairwise Gaussian (w/ cov.)
ResNet-18
90.1% 0.2%
89.5% 0.0%
the
ResNet-34
90.5% 0.2%
89.6% 0.2%
5.5 RUNTIME ANALYSIS
In Tab. 5 we report runtimes for prop- agating distributions. For small mod- els. propagating distributions without covariances is almost without over- head while propagating covariances is slower. Moment matching is slightly more expensive than the proposed propagation, which is because mo- ment matching requires evaluating ad- ditional (scalar) functions at the non-linearities. The complexity (as a factor of the cost of propagating single point through the neural network) of propagating distributions with covariances is linear in the number of outputs, while for propagating distributions without covariances it is constant. As reference, exact propagation of the distributions (and not parametric approximation as in this work) has an exponential runtime factor in the number of neurons (40). The runtime factor of DVIA [77 is linear in the number of neurons in the largest layer, which makes experiments exceeding Table I iinfcsibbl. For example, in the robustness experiments on MNIST dataset in Section [.3 the dimension of the largest hidden layer 12 500, i.c., computational time of a single forward pass with DVIA is over 12 500 regular forward passes.
Table 5: Computational Cost Benchmark. Times per epoch on CIFAR-10 with batch size of 128 on a single V100 GPU.
Model
regular
moment matching dist. w/o cov.
dist. w/ cov.
Layer CNN
6.29%
6.33x
6.31s
ResNet-18
9.85s
30.7s*
28.5s
251s
as moment matching for layers of ResNet (such MaxPool for more than two inputs) is very expensive and does not have a clased form solution in the literature,
our approximation for layers.
CONCLUSION
We studied the problem of uncertainty propagation in neural networks. Uncertainty propagation allows quantifying both epistemic and aleatoric uncertainties, and promotes robustness to input noises, as demonstrated in our empirical studies. Our local linearization based methods, together with the pairwise Gaussian and Cauchy loss functions, performed well. Extending our theoretical results to better understand the empirical success could be an interesting direction for future work.

---

Under review as a conference paper at ICLR 2022
REPRODUCIBILITY STATEMENT
We will make the source code and experiments of this work publicly available to foster future research in this direction. All data sets are publicly available. We specify all necessary hyperparameters for each experiment. Each experiment can be reproduced on a single GPU.
REFERENCES
[ A. Loquercio, M. Segu, and D. Scaramuzza, "A general framework for uncertainty estimation in deep leammin," IEEE Robotics and Automation Letters, vol. 5, no. 2, pp. 3153-3106 2020.
[2] A. Kendall and Y. Gal, "What uncertainties do we need in bayesian deep learning for computer vision?"
In Proc. 31 Conf. on Neural Information Processing Systems (NIPS 2017, Long Beach, CA), 2017,
pp. 5574-5584.
[3] Der Kiureghian and Ditlevsen, "Aleatory or epistemic? does matter?" Structural safety, vol. 31,
no. 2. pp. 105-112, 2009.
[4] Y. Gal, "Uncertainty deep learning," University of Cambridge, vol. I, no. 3, 2016.
[5] D. Hendryeks and Gimpel, baseline for detecting misclassified and out-of-distributio examples in
neural networks," Proceedings of International Conference on Learning Representations, 2017.
[6] Tagasovska and Lopez-Paz. "Single-model uncertainties for deep learning." Proc. Neural Information Processing Systems (NIPS), 2019.
[7] S. Ghosh, E. M. D. Fave, and J. Yedidia, " "Assumed density filtering methods for leamning bayesian neural networks," " in Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence, ser. AAAI Phoenix, Arizona: AAAI Press, 2016, pp. 1589-1.595.
[8] C. Blundell. J. Cornebise, K. Kavukcuoglu, and D. Wierstra, "Weight uncertainty in neural networks." in Proc. 32nd Int. Conf. on Machine Learning (ICMI. 2015, Lille, France), MI. Research Press, 2015, pp. 1613-1622.
[9] A. Graves, "Practical variational inference for neural networks," in Proc. 25th Conf. Neural Informa- tion Processing Systems (NIPS 2011, Granada, Spain). 2011, pp. 1348-2356
[10] A. Buchholz, F. Wenzel, and S. Mandt, "Quasi-Monte Carlo variational inference," J. Dy and A. Krause, Eds., ser. Proceedings Machine Learning Research, vol. 80, Stockholmsmässan, Stockholm Sweden: PMLR, Oct. 2018, pp. 668-677. [Online]. Available: http::/ppoocceeinnggg./ ml.. press v buchhol21 1eto022laaa. httll
[11] J. Gast and S. Roth, "Lightweight probabilistic deep networks." in Proc. IEEE Int. Conf. Computer Vision and Puttern Recognition (CVPR 2018, Salt Lake City, UT), Piscataway, NJ, USA: IEEE Press, 2018.
[12] A. Shekhovtsov and B. Flach, "Feed-forward propagation in probabilistic neural networks with categorical and max layers," in Proc. Int. Conf. on Learning Representations (ICLR 2019, New Orleans, LA), 2019.
[13] J. Thiagarajan, 1. Kim, R. Anirudh, and P. Bremer, "Understanding deep neural networks through input uncertainties." in ICASSP 2019 2019 IEEE International Conference Acoustics. Speech and Signal Processing (ICASSP), 2019. pp. 2812-2816.
[14] B. Lakshminarayanan, A. Pritzel, and Blundell, "Simple and scalable predictive uncertainty estimation using deep ensembles," " in Proc. Neural Information Processing Systems (NIPS), 2017.
[15] E. Goan and C. Fookes, "Bayesian neural networks: An introduction and survey," in Case Studies Applied Bayesian Data Science, Springer, 2020, pp. 45-87.
[16] E. Hüllermeier and W. Waegeman, "Aleatoric and epistemic uncertainty machine learning: An introduction to concepts and methods," Machine Learning, vol. 110, no. 3, pp. 457-506, 2021. [17] Frey and G. E. Hinton, "Variational learning in nonlinear gaussian belief networks." Neural Computation, vol. 11, pp. 193-213, 1999.
[18] Wang, Shi, and D.-Y. Yeung, "Natural-parameter networks: A class probabilistic neural networks." in Proc. 3hth Conf. on Neural Information Processing Systems (NIPS 2016, Barcelona, Spain), 2016,
pp. 118-126.
[19]
A. Paszke, S. Gross, F. Massa, A. Lerer, J. Bradbury, G. Chanan, T. Killeen, Z. Lin, N. Gimelshein, L. Antiga, A. Desmaison, A. Kopf. E. Yang. Z. DeVito, M. Raison, A. Tejani, S. Chilamkurthy, B. Steiner, L. Fang, J. Bai, and S. Chintala, "Pytorch: An imperative style, high-performance deep learning library." in Proc. Neural Information Processing Systems (NIPS), 2019, pp. 8024-8035.
[20] Y. Gal and Z. Ghahramani, "Dropout as a bayesian approximation: Representing model uncertainty in deep learning." in international conference on machine learning, PMLR, 2016, pp. 1050-1055 1059.
[21]
Srivastava, Hinton, Krizhevsky, Sutskever, and R. Salakhutdinov, "Dropout: A simple way to prevent neural networks from overfitting." The journal of machine learning research, vol. 15, no. 1. pp. 1929-1958, 2014.
10

---

Under review as a conference paper at ICLR 2022
[22] F. K. Gustafsson, M. Danelljan, and T. B. Schon, "Evaluating scalable bayesian deep learning methods for robust computer vision," " 2020 JEEE/CVF Conference on Computer Vision and Pottern Recognition Workshops (CVPRW), 2020, pp. 1289-1298.
[23] D. Bouchacourt. M. Kumar, and Nowozin, "Disco nets: Dissimilarity coefficient networks." in Pruc. 30th Conf. on Neural Information Processing Systems (NIPS 2016, Burcelona, Spain), 2016, pp. 352-360. [24] J. Jin, A. Dundar, and F. Culutciello, "Rohust convolutional neural networks under adversarial noise," in Workshop Track Int. Conf. on Learning Representations (ICLR 2016, San Juan, Puerto Rico), 2016.
[25] S. Wang and C. Manning, "Fast dropout training." in International Conference on Machine Learning (ICML), PMLR, 2013, pp. 118-126.
[26] J. Postels, F. Ferroni, H. Coskun, N. Navab, and F. Tombari, "Sampling-free epistemic uncertainty estimation using approximated variance propagation," in Proc. International Conference on Computer Vision (ICCV), 2019, pp.
[27] A. Wu, S. Nowozin, E. Meeds, R. E. Turner, J. Hernandez-Lobato, and A. L. Gaunt, "Deterministic variational inference for robust bayesian neural networks," in International Conference on Learning Representations (ICLR), 2019.
[28] L. Devroye, A. Mchrabian, and T. Reddad, "The total variation distance between high-dimensional gaussians," " arXiv preprint 1810.088 2018.
[29]
Hendrycks and Gimpel, "Gaussian error linear units (gelus)." arXiv preprint arXiv:1606.0841:
2016.
[30] S. Elfwing, E. Uchibe, and K. Doya, geggoodd-eighhte linear units neural network function approximation reinforcement learning. arxiv e- e-prints (2017)," arXiv preprint 1rXiv:1702.03118 2017.
[31] Y. Geifman and R. EI- Yaniv, "Selective classification for deep neural networks," " in Proc. Neural Information Processing Systems (NIPS), 2017.
[32] Y. LcCun, C. Cortes, and C. Burges, "Mnist handwritten digit database," " ATT Labs, vol. 2, 2010. [Online). Available: hate::hhtt///tt htttp://yann) wwan.yannn... Iaamnnneeece lecun com/ exdb/mmmm mst
[33] G. Cohen, S. Afshar, J. Tapson, and A. van Schaik, "EMNIST: an extension of MNIST to handwritten letters," arXiv preprint arXiv:1702.05333
2017.
[34] N. Carlini, A. Athalye, N. Papernot, W. Brendel, J. Rauber, D. Tsipras, I. Goodfellow, A. Madry, and
A. Kurakin, "On evaluating adversarial robustness," arXiv preprint arXiv: 0902/99000009 2019.
[35] A. Madry, A. Makelov, L. Schmidt, D. Tsipras, and A. Vladu, "Towards deep learning models resistant to adversarial attacks," arXiv preprint 1706 06083, 2017.
[36]
E. Wong and Z. Kolter. "Provable defenses against adversarial examples via the convex outer adversarial polytope," in International Conference on Machine Learning (ICML), 2018
[37] A. Krizhevsky, V. Nair, and G. Hinton, "Cifar-10 (canadian institute for advanced research)," 2009. [Online]. Available: http::/
GS. wwwp:/ontteedd/errrreeea
far. html]
[38] K. He. X. Zhang, S. Ren, and J. Sun, "Deep residual learning for image recognition." " in Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 770-778.
[39] D. Kingma and J. Ba, "Adam: A method for stochastic optimization." in International Conference on Learning Representations (ICLR), 2015.
[40] R. Balestriero. S. Paris, and R. G. Baraniuk, "Analytical Probability Distributions and EM-Learning for
Deep Generative Networks," " in Proc. Neural Information Processing Systems (NIPS), 2020.

---

Under review as a conference paper at ICLR 2022
Supplementary Material for Propagating Distributions through Neural Networks
A
ILLUSTRATION OF PARAMETRIC APPROXIMATIONS OF RELU
J 1.0
1.0
R (6- 07
1.0
1.0
V2T0
2x
-
H 1.0
μ 1.0
④-- 14)
G 1.0
1.0
V/xg
V250
Figure 4: Illustration of parametric approximations of ReLU. In each plot, the gray distribution is the input and the green distribution is the true output distribution. The red distributions (top) are our local linearization approximation. The orange distributions (bottom) are produced by moment matching.
B TRANSFORMATION OF CONVOLUTIONS
For two-dimensional convolutions, we use the notation Y X W + B where X E R is the input (image), W E X xk: is the weight tensor, and B, Y E XRexxmamm are the bias and output tensors with co. C1, ko, k1, no, n1, Mo. 771, mo, my E N4. Further, let XU Iim xm, Xko be the unfolded sliding local blocks from tensor X such that Yormom =
Enghoks
XU
W Warcokoks + B X+
Normal Distribution with Covariances For the convolutional layer, E X E e X Xni Xngg X Note that E E Xm xi Xig ko xkj mma Xm xxxxxxxxxi x is the unfolded slid-
ing local blocks covariance of the covariance tensor Σ. / + * W + B and Ceymomie mbmm + Ee
W
U Wasokkok momse minmss/ 66 W WeGGGMAA
Normal Distribution without Covariances J x W B and σ² + o² W
For the convolutional layer, μa, X / +
Cauchy Distribution For the convolutional layer, æo, 7 x 20 H Zo W + and 7 Abs(W).
Einstein summation notation. The unfolded sliding local blocks are equivalent torch. un fold.
12

---

Under review as a conference paper at ICLR 2022
C ADDITIONAL EXPERIMENTS
C.1 SIMULATION OF PROPAGATING NORMAL DISTRIBUTIONS
Tab. show additional simulations with and hidden layers as well as ReLU, Leaky-ReLU, GELU, SILU, and logistic sigmoid activations. Here, the reported metric is intersection of probability mass, i.e., TV. The baseline computed via 10 samples.
Tab. 23 considers the average ratio between predicted and true standard deviations.
As there we did not find a moment matching method for GELU and SiLU in the literature, we omit moment matching in these cases. DVIA [27] is only applicable to ReLU among the non-linearitier we consider. For moment matching with logistic sigmoid, we use numerical integration as it does not have closed form solution.
Table 6: Simulation of propagating normal distributions. The network is 2 layer ReLU activated network with dimensions 4 --000-3, i.e., 1 ReLU activation.
With Covariances
DVIA 27
Without Covariances
Moment Matching
0.1
0.9874 + 0.0079
0.9875 = 0.0074
0.4574 + 0.0275
0.4532 + 0.0288
0.9578 + 0.0188
0.9619 1 0.0165
0.4643 + 0.0235
0.4439 1 0.0238
10
0.8648 + 0,0206
0.8982 = 0.0247
0.4966 + 0.0232
0.4580 = 0.0136
100
0.8157
0.0231
0.8608 = 0.0353
0.5034 0.0248
0.4640 = 0.0178
1000
0.8103 0.0236
0.8555 + 0.0372
0.5041 0.0254
0.4640 + 0.0182
Table 7: Simulation of propagating normal distributions. The network is a 3 layer ReLU activated network with dimensions --100-100-3, i..., 2 ReLU activations.
With Covariances
DVIA 27
Without Covariances
Moment Matching
0.1
0.9861 + 0.0099
0,9840 = 0.0112
0.3070 + 0.0185
0.3111 + 0.0193
0.9259 0.0279
0.9247 = 0.0270
0.3123 1 0.0133
0.3004 1 0.0136
0.8093 0.0234
0.8276 + 0.0255
0.3463 0.0206
0.2972 + 0.0170
100
0.7439 + 0.0257
0.8152 + 0.0347
0.3931 + 0.0207
0.3065 + 0.0237
1000
0.7373 1: 0.0259
0.8061 + 0.0384
0.3981 1: 0.0188
0.3079 +:201 0.0249
Table 8: Simulation of propagating normal distributions. The network is 5 layer ReLU activated network with dimensions 4-000-100010001100-3 i.e., ReLU activations.
With Covariances
DVIA 27
Without Covariances
Moment Matching
0.1
0.9791 0.0202
0.9720 + 0.0228
0.2361 0.0250
0.2611 + 0.0256
0.8747 + 0.0546
0.8519 = 0.0543
0.2243 + 0.0237
0.2195 + 0.0294
10
0.7586 1 0.0407
0.7035 1 0.0366
0.2186 1 0.0244
0.1976 1 0.0179
100
0.6877
0.0333
0.5845 = 0.0479
0.2261 = 0.0282
0.1724 + 0.0111
1000
0.6808 0.0318
0.5318 = 0.0516
0.2193 0.0248
0.1706 + 0.0109
Table 9: Simulation of propagating normal distributions. The network is a 7 layer ReLU activated network with dimensions 4 -0000-10-11001000000-00000000000 i... 6 ReLU activations.
With Covariances
DVIA [27
Without Covariances
Moment Matching
0.1
0.9732 + 0.0219
0.9660 + 0.0226
0.2196 + 0.0208
0.2601 + 0.0323
0.8494 + 0.0853
0,8166 = 0.0986
0.2292 + 0.0303
0.2236 + 0.0368
0.7743 0.0477
0.6309 = 0.0553
0.2420 0.0295
0.2327 =:2 0.0571
100
0.7077
0.0348
0.5265 + 0.0975
0.3525 0.2337
0.1800 + 0.0188
1000
0.7013 + 0.0334
0.5166 + 0.1223
0.4422 + 0.2450
0.1764 + 0.0175
13

---

Under review as a conference paper at ICLR 2022
Table 10: Simulation of propagating normal distributions. The network is 2 layer Leaky-ReLU activated network with dimensions 4-100-3, i.e., Leaky-ReLU activation with negative slope = 0.1.
With Covariances
Without Covariances
Moment Matching
0.1
0.9852 + 0.0065
0.4484 + 0.0204
0.4481 + 0.0208
1
0.9624 + 0.0152
0.4547 + 0.0183
0,4417 + 0.0172
10
0,8876 = 0.0194
0.4838 1 0.0198
0,4454 + 0.0118
100
0.8458 = 0.0226
0.4912 + 0.0203
0.4581 0.0139
1000
0.8411 = 0.0232
0.4918 + 0.0207
0.4594 + 0.0141
Table 11: Simulation of propagating normal distributions. The network is 3 layer Leaky-ReLU activated network with dimensions 1-100-11002- i.e., 2 Leaky-ReLL activations with negative slope a = 0.1.
With Covariances
Without Covariances
Moment Matching
0.1
0.9806 + 0.0085
0.3020 + 0.0162
0.3063 + 0.0171
1
0.9324 + 0.0255
0.3055 1 0.0129
0.3007 1: 0.0124
10
0.8321 = 0.0254
0.3343 + 0.0180
0.2963 + 0.0147
100
0.7716 = 0.0294
0.3769 + 0.0203
0.3040 0.0189
1000
0.7651 + 0.0297
0.3825 + 0.0190
0.3056 0.0198
Table 12: Simulation of propagating normal distributions. The network is 5 layer Leaky-ReLJ activated network with dimensions 11:00-100-1000000 i... Leaky-ReLU activations with negative slope a = 0.1.
With Covariances
Without Covariances
Moment Matching
0.1
0.9689 + 0.0163
0.2325 + 0.0224
0.2566 + 0.0249
1
0.8856 + 0.0460
0.2260 + 0.0197
0.2218 + 0.0236
10
0.7638 = 0.0393
0.2146 + 0.0220
0.1975 = 0.0169
100
0.6912 + 0.0352
0.2364 + 0.0227
0.1756 0.0112
1000
0.6835 + 0.0337
0.2361
+ 0.0216
0.1743 0.0116
Table 13: Simulation of propagating normal distributions. The network is 7 layer Leaky-ReLU activated network with dimensions 11100110-100-110-10--1111 i... Leaky-ReLU activations with negative slope = 0.1.
With Covariances
Without Covariances
Moment Matching
0.1
0.9585 + 0.0203
0.2197 + 0.0199
0.2519 + 0.0292
1
0,8573 = 0.0720
0.2251 i:2 0.0289
0.2220 + 0.0351
10
0.7701 + 0.0465
0.2332 + 0.0301
0.2243 + 0.0259
100
0.7047 + 0.0378
0.2940 + 0.1864
0.1779 0.0171
1000
0.6979 + 0.0371
0.3518 1 0.2209
0.1746 0.0159
14

---

Under review as a conference paper at ICLR 2022
Table 14: Simulation of propagating normal distri- butions. The network is 2 layer SiLU activated network with dimensions 1-100-1000 i.c. 1 SiLU activation.
Table 18: Simulation of propagating normal dis- tributions. The network is a layer GELU acti- vated network with dimensions 4- 4-100-3, i..., GELU activation.
With Covariances
Without Covariances
With Covariances
Without Covariances
0.1
0.9910 = 0.0025
0.4491 + 0.0192
0.1
0.9875 + 0.0075
0.4567 + 0.0317
I
0.9813 = 0.0076
0,4550 + 0.0158
0.9732 + 0.0096
0.4682 = 0.0225
10
0.8635 0.0180
0.5158 = 0.0251
10
0,8530 + 0.0185
0.5204 = 0.0266
100
0.8104
0.0201
0.5328 0.0282
100
0.8039 + 0.0211
0.5251 0.0278
1000
0.8050 0.0205
0.5347 0.0288
1000
0.7987 0.0216
0.5255 + 0.0282
Table 15: Simulation of propagating normal distri- butions. The network layer SiLU activated network with dimensions 4 4-11-0 1-100-10000 100 11:0003330 i.e., 2
Table 19: Simulation of propagating normal distri- butions. The network is a layer GELU activated network with dimensions 1--00-111 i.e., GELU activations.
SiLU activations.
a
With Covariances
Without Covariances
With Covariances
Without Covariances
0.1
0.9876 0.0051
0.3056 0.0178
0.9860 0.0054
0.3065 0.0244
I
0.9658 = 0.0121
0.3076 + 0.0122
0.9453 + 0.0169
0.3129 = 0.0164
10
0.8067 = 0.0136
0.3531 + 0.0189
10
0.8024 + 0.0128
0.3560 + 0.0176
100
0.7425 = 0.0162
0.3806 + 0.0261
100
1.7446+0.000
0.3966 + 0.0237
1000
0.7365 0.0167
0.3829 1 0.0233
1000
0.7390 + 0.0159
0.4004 + 0.0222
Table 16: Simulation of propagating nor- mal distributions. The network 5 layer SiLU activated network with dimensions 4-100 11:00-00-0000 1000-10--0,,,..0 i.e., 4 SiLU activa-
Table 20: Simulation of propagating nor- mal distributions. The network is layer GELU activated network with dimensions 4 4-100- 1000010-111,,,,, i.e., 4 GELU acti- vations.
tions.
With Covariances
Without Covariances
With Covariances
Without Covariances
0.1
0,9704 = 0.0126
0,2471 0.0277
0.1
0.9706 + 0.0104
0.2389 + 0.0287
I
0.9174 = 0.0343
0,2414 + 0.0096
0.8916 1 0.0359
0.2442 = 0.0131
10
0.7478 0.0342
0.2357 + 0.0141
10
0.7607 0.0247
0.2460 = 0.0220
100
0.68-46 0.0241
0.2453 1 0.0254
100
0.6975 1 0.0184
0.2532 + 0.0295
1000
0.6789 = 0.0238
0,2371 0.0236
1000
0.6915 0.0193
0.2465 0.0288
Table 17: Simulation of propagating nor- mal distributions. The network a layer SiLU activated network with dimensions
Table 21: Simulation of propagating nor-
mal distributions. The network is layer GELU activated network with dimensions 1-100-100- 100010000000000 100-100-00000 100- 1000-100-0000 3, i... GELU activations.
4-100-100- 1100110--0000 100-100 3, i.e., 6 SiLU activations.
With Covariances
Without Covariances
With Covariances
Without Covariances
0.1
0.9147 + 0.0421
0.2326 + 0.0345
0.1
0.9262 + 0.0294
0.2261 +: 0.0352
1
0.8248 0.0777
0.2348 + 0.1062
0.8387 + 0.0858
0.2710 + 0.1547
10
0.7293 = 0.0388
0.2762 0.1315
10
0.7342 0.0266
0.3557 0.2025
0.6726 = 0.0411
0.2326 0.0282
100
0.6898
0.0218
0.2244 0.0259
1000
0.6675 + 0.0417
0.2363 +:: 0.0309
1000
0.6854 4: 0.0220
0.2235 +: 0.0264
15

---

Under review as a conference paper at ICLR 2022
Table 22: Simulation of propagating normal distributions. The network is a 5 layer Logistic Sigmoid activated network with dimensions 1-100110011001100 4 Logistic Sigmoid activations.
With Covariances
Without Covariances
Moment Matching
0.1
0.9890
0.7809
0.7793
1
0.9562
0.7880
0.7912
10
0,8647
0.8656
0.7674
100
0.8443
0.8442
0,8027
1000
0.8440
0.8440
0.8070
Table 23: Simulation of propagating normal distributions. The network is 5 layer ReLU activated network with dimensions 4 average ratio between the output standard deviations. The 3 values, correspond to the three output dimensions of the Iris model.
i.e., 4 ReLU activations. Displayed is the
A value of 1 is optimal. We find that both our method with covariances as well as DVIA achieve a good accuracy in this setting, while the methods which do not consider covariances (ours (w/o cov.) and Moment Matching) underestimate the output standard deviations by large factor. For small input standard deviations, our method w/ cov. as well as DVIA perform better. For large input standard deviations, method tends to overestimate the output standard deviation, while DVIA underestimates the standard deviation. The ratios for our method and DVIA (while going into opposite directions) similar, e.g., 2.6095 for method and 1/0.3375-2.9629 for DVIA.
With Cov.
DVIA
Without Cov.
Moment Matching
0.1
0.9933
1.0043
0.9954
0.0180
0.0112
0.0255
0.0373
0.0211
1
0.9248
1.0184
0.9598
0.8462
0.9256
0.8739
0.0103
0.0170
0.0100
0.0101
0.0166
0.0097
10
1.9816
2.2526
2.0629
0.6035
0.6664
0.6172
0.0207
0.0359
0.0205
0.0166
0.0269
0.0157
100
2.5404
2.8161
2.6217
0.3721
0.4194
0.3849
0.0264
0.0450
0.0260
0.0186
0.0296
0.0174
1000
2.6095
2.8857
2.6906
0.3375
0.3839
0.3503
0.0271
0.0462
0.0267
0.0188
0.0298
0.0175
16

---

Under review as a conference paper at ICLR 2022
deziiaie Ncand thee Ove Nomaa derhidio Co
37
Figure 5: Robustness of CNNs on the MNIST data set against Gaussian noise (left) as well as PGD adversarial noise (right). While Fig. 3 presents 0.75 and 8/255 255 The continuous lines show the test accuracy and the dashed lines show the robust accuracy. The black lines indicate the softmax cross-entropy trained baseline. Results are averaged over 3 runs.
0.5 and 16/ 16/255, this figure presents
C.2 GAUSSIAN AND ADVERSARIAL NOISE ROBUSTNESS
In Fig. present extension Fig- S which demonstrates random and adversarial noise robustness for alternative noise intensities.
we present additional demonstration of the adversarial and random noise robustness of CNNs on distributions. Note that we do not include results for normal distributions with covariances (as in Figures 3 and 5). Here, the emphasis is set on using both MNIST and CIFAR.
In Fig. and 10
In Fig. we show the robustness where a cross-entropy based objective (green) is used by the adversary. A cross-entropy based objective could be used in practice in multiple scenarios: For example, publishing just the weights of the model without information about the objective function, or by using standard library for adversarial attacks that does not require or support specification of the training objective. We emphasize that our models are more robust against the cross cross-entropy based attack (CE-PGD) than the pairwise distribution loss based attack (PA-PGD) which reported in all other plots.
Note that the phenomenon of vanishing gradients can occur in our models (as it can also occur in any cross-entropy based model). Thus, decided to consider all attempted attacks, where the gradient is zero or zero for some of the pixels in the input image, as successful attacks because the attack might be mitigated by lack of gradients. In Fig. 7 compare conventional robustness robustness with the additional Gradient- Non- (GNZ) requirement. We emphasize that we use the GNZ requirement also in all other plots.
For evaluating the robustness against random noise, we use input noise with a E {0.25, 0.5, 0.75) and clamp the image to the original pixel value range.
C.3 PROPAGATING WITHOUT COVARIANCES
We analyze the accuracy of fully connected neural network (FCNN) where normal distribution is introduced in the keth out of layers. Here, the normal distribution modeled without covariances. The result is displayed in Fig. B. where we can observe that there is an optimal scale such that the accuracy is as large as possible. Further, we can see the behavior if the distribution is not introduced in the first layer, but in later layer instead. That is, the first layers are conventional layers and the distribution introduced after a certain number of layers. Here, we can see that the more layers propagate distributions rather than single values, the larger the optimal standard deviation is. The reason for this that the scale of the distribution decays with each layer such that for deep models a larger input standard deviation necessary get an adequate output standard deviation for training the model. This is because we modeled the distributions without covariances, which demonstrates that propagating without covariances underestimates the variances.
17

---

Under review as a conference paper at ICLR 2022
CE-based vs. PA-based PGD. (MNIST data set)
1.0
0.8
0.6
Accuracy
0.4
PA-PGD = 8/255
CE-PGD c 8/255
0.2
Nornal distribution
Cross-Entropy
10
10-3
10
10
10
10
10
Input during Training
Figure 6: Softmax Cross-Entrope (CE) pairwise distribution loss (PA) based PGD attack. Averaged
over runs.
PGD with vs. without GNZ Requirement. (CIFAR- 10 data set)
Accuracy
PA-PCD w/ GNZ. 3/255
0.6
PA-PGD w/o GNZ 3/255
0.5
Nomal distribution
Cross- Eooss-Entropy
0.4
0.3
0.1
0.0
10
10-3
10-2
10
10
10a
10a
10
Input during Training
Figure Effect of the gradient-non-zero (GNZ) requirement. Averaged over runs.
Introduction of dist. in kth layer. (CIFAR- 10 data set)
0.5
Accuracy
0.3
0.2
0.1
CE
10
10-5
10
10
10a
10a
10a
10a
Inpul during Training
Figure 8: The normal distribution is introduced in the kth layer of a layer fully connected network (k E 1,,1,2.3, 4,53). Averaged over runs.
18

---

Under review as a conference paper at ICLR 2022
Adversarial Robustness Cauchy Dist. (MNIST data set)
Adversasial Robustmess Nurmal Dist. (MINITT data set)
0.3
0.8
Necsney
0.
0.6
Kilaa 8,255
Nerná disiitttion
PCDD 5/355
CrossEniopy
PCD 16,256
012
dsstiution
0.2
Croso-arryy
10
10
10
10"d
10
10"
10'
10"
107
lapur doring Trannig
darinn Thoeinng
Robustness Cauchy (CIFAR- scl)
Robusiness (CIFAR-
0.7
0.7
Accunsy
05
Accanay
3/225
0/250
PGD
4/255
0.4
KGD 4/255
0.3
Cancly fattthation
0.3
Nornel dienthation
Erropp
0.2
0.2
0.1
0.1
0.0
00
10
10-1 lapor during Traning
10
10
10
10
10
30'
102
107
dating Trannig
Figure 9: Accuracy and adversarial robustness under PGD attack. Left: Cauchy distribution. Right: Normal distribution without propagating covariances. Top: MNIST data set. Bottom: CIFAR-10 data set. Averaged over 3 runs.
Noise Robusmess Canchy Dist. (MNIST data
Noise Robustness Namaa Dist. (MNIST data set)
08
0.8
0.4
Accanay
0.4
Necatasy
App btile roter
0.2
Caucly dstilitionn
0.2
Nornal disaittiin
Coons-Entopy
ChonsEniopp
10
lapor duting Traniag
Robustness Canchy (CIFAR-10
Dist, (CIFAR- set)
0.7
0.7
0.6
Apc. under paks 0.5
0.3
Neeovi desebattinn
Adc. Hnde
Closs Emeoy
Ace. under neise 0.
0.4
Cancly flstthation
0.3
0.2
0.2
0.1
0.1
0.
0.0
10
10"
10
10-1
10
10-
100
10--
10
16P
100
110
107
10
Japor daring Tranisg
Eupor:
daring Trainig
Figure 10: Accuracy and robustness against random normal noise. Left: Cauchy distribution. Right: Normal distribution without propagating covariances. Top: MNIST data set. Bottom: CIFAR-10 data set. Averaged over 3 runs.
19

---

Under review as a conference paper at ICLR 2022
D IMPLEMENTATION DETAILS
D.1 SIMULATION OF PROPAGATING NORMAL DISTRIBUTIONS
We trained fully-connected neural network with I, 2, or 4 hidden layers and ReLU / Leaky-ReL.U activations for 5000 epochs on the Iris data set via the softmax cross-entropy loss. Here, each hidden layer has 100 hidden neurons. Results are averaged over 10 runs, and 10 inputs are propagated for each these runs. Across all methods propagating distributions, each Monte Carlo baseline as well as model weights are shared.
D.2 ALEATORIC UNCERTAINTY QUANTIFICATION
For the aleatoric uncertainty quantification experiment, we use the same hyperparameter and settings as Tagasovska et al. (6]. That is, we use a network with ReLU activated hidden layer, with 64 hidden neurons and train it for 5000 epochs. We perform this for 20 seeds and for a learning rate 7 E 2,00-,1000 10 For the input standard deviation, (10-8, 10-7, 10-6; 10-6, 10-14, 10-03 10-2,10-1____----010101110110111 10") and then (for each data set) used 11 vari- ances resolution of 100.1 around the best initial variance.
and weight decay e {0, 10 10 2, 2,10-1,111 made single initial run with input variance o²
-.104, 4]
D.3 EPISTEMIC UNCERTAINTY FOR SELECTIVE PREDICTION ON OUT-OF-DISTRIBUTION DATA
Here we use the same network architecture as for the robustness experiments in the next subsection (A CNN composed of two U-activated convolutional layers with a kemel size of 3, stride of 2, and hidden sizes of 54 and 8 followed by a convolutional layer mapping it to an output size of 10) and also train it for 100 epochs, however, learning rate of 10-3. For training with uncertainty propagation, we use an input standard deviation of o 0.1. For training with uncertainty propagation via moment matching and the Dirichlet loss as in Gast al. (TT), we use input standard deviation of a = 0.01, which is what they use in their experiments and also performs best in our experiment.
D.4 GAUSSIAN AND ADVERSARIAL. NOISE ROBUSTNESS
Here, our CNN is composed of two ReL.-active convolutional layers with a kernel size of 3, a stride of and hidden sizes of 64 and 128. After flattening, this followed convolutional layer mapping it to an output size of 10. We have trained all models for 100 epochs with an Adam optimizer, a learning rate of 10-4, and batch size of 128.
D.5 PROPAGATING WITHOUT COVARIANCES
The architecture of the 5-layer ReI.t-activated FCNN is 784-256-256-256-25--1
D.6 EVALUATION METICS
D.6.1 COMPUTATION OF TOTAL VARIATION IN THE SIMULATION EXPERIMENTS
We compute the total variation using the following procedure: We sample 10t samples from the continuous propagated distribution. We bin the distributions into 10 10 x 10 bins (the output is 3-dimensional) and compute the total variation for the bins. Here, we have an average of 1000 samples per bin. For each setting, we compute this for 10 input points and on 10 models (trained with different seeds) each, the results are averaged over 100 data point model configurations. Between the different methods, we use the same seeds and oracles. To validate that this is reliable, we also tested propagating 10" samples, which yielded very similar results, but just took longer to compute the oracle. We also validated the evaluation methods by using a higher bin resolution, which also yielded very similar results.
20

---

Under review as a conference paper at ICLR 2022
D.6.2 COMPUTATION OF MPIW
Given a predicted mean and standard deviation, the Prediction Interval Width (for 95%) can be computed directly. In a normal distribution, 68.27% of samples lie within 1 standard deviation of the mean. For covering 95% of Gaussian distribution, we need cover 11.96 standard deviations. Therefore, the PIW is 3.92 standard deviations. We computed the reported MPIW on the test data set. This setup follows the setup by Tagasovska et [6].
D.7 MAXPOOL
For MaxPool, apply Equation I such that (μ, σ)) + (max(μ), arg max(() o) where arg yields a one-hot vector. The equation maxpools all inputs to one output; applying it to subsets of neurons correspondingly maxpools the respective subsets.
21


Reviews:
[{"review_id": "j6I8G5ZrsK4", "reviewer": "Reviewer_cdSE", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The paper presents a simple new technique to propagate distributions through neural networks. A nice feature of the method is that it can be readily used for pre-trained models. To justify the proposed method, a theoretical argument is provided that for the ReLU activation function a total variation distance is minimized. The paper is mostly clear and well written but lacks some details (see below).\n\n- Regarding experiments showing the total variation metric (i.e., Table 1 and several tables in the supplementary):\n\nHow is the total variation between 10^6 (discrete) samples and the (continuous) output of probability propagation computed? I have another concern about the significance of the total variation distance. In particular, it is mentioned in the introduction that f(x + \\epsilon) is a random variable where the variance is used to quantify its uncertainty. How well are the prediction uncertainties between the proposed method and sampling 10^6 models in these experiments aligned? Or can the total variation distance be used to give guarantees about the predicted uncertainty in these experiments?\n\n- Regarding activation functions:\n\nHow well does the proposed method perform with other activation functions. The experiments in the main paper are only concerned with ReLU activations and the experiments in the supplementary material are concerned with ReLU-like activation functions (leaky ReLU, GELU, ...). Does the proposed method also produce reasonable results for saturating activation functions such as tanh or the logistic sigmoid?\n\n- Failure modes:\n\nThe paper presents mostly results where the method succeeds although there are some obvious failure modes. For instance, given a Gaussian input with either a mean very close to zero or a very large variance, the approximation made by the proposed method is not very accurate. However, there are some experiments where the injected variances are rather large (e.g., Table 1 or Figure 3) but still the method does not seem to break down. How can you explain this?\n\nFurthermore, I see another failure mode regarding the proposed objective for classification where the sum over all classes is computed. I do not see why this objective is enforcing the model to make a correct prediction. If P(Z_c > Z_e) is close to one for all classes except for a single class where P(Z_c > Z_e) < 0.5, then a wrong prediction will be made but still a high score is assigned by the proposed loss. If, on the other side, P(Z_c > Z_e) is slightly above 0.5 for all classes, then the correct class will be predicted but only a low score is assigned by the proposed loss. This might be more severe for classification tasks with a very large number of classes (e.g., ImageNet). In my opinion, it would make more sense to define the score by maximizing \"min_e P(Z_c > Z_e)\". This is similar to the concept of a probabilistic margin, e.g., see [1].\n\n- Regarding presentation:\n\nPlease emphasize somewhere in the paper that the only point where stochasticity is injected is at the inputs (i.e., no weight uncertainty, no injected noise at hidden neurons, etc). Although I (now) see that this is clearly stated in the introduction, I would really appreciate if a formal definition of the considered setup is made at the beginning of Section 3. If space is an issue, I think Figure 1 is not that important and can be moved into the supplementary material. \n\n- Regarding experiments:\n\n  - How do you set the input variances in your experiments in Section 5.1? Is the input variance tuned so that the PICP is close to 95%? Is the input standard deviation in Section 5.2 always 0.1?\n  - How is the prediction interval in Section 5.1 computed? Is the MPIW computed at the test data? In general, I would appreciate a clearer explanation of the used metrics and the experimental setup in Section 5.1.\n  - In Section 5.2 it is stated that training is performed using both Pairwise Gaussian and Pairwise Cauchy. However, I do not see any results for training with Pairwise Cauchy.\n  - What architecture is used in Section 5.2?\n  - Results on Cifar-10: It would be interesting to see how well prediction uncertainties work for CNNs? For instance, a similar experiment as in Section 5.2 for out-of-distribution detection would be interesting (e.g., using samples from Cifar-100 for out-of-distribution data).\n  - An evaluation of \"in-between uncertainties\" [2] would be interesting.\n\n\n- Regarding computational complexity:\n\nSection 5.5 states that the complexity for propagating distributions with and without covariances is linear and constant in the number of outputs, respectively. How can the complexity be linear if there are quadratically many entries in the covariance matrix? Similarly, how can the complexity be constant if we need to compute a variance entry for each output?\n\n- Notation:\n\n  - x used multiple times (e.g., (i) as input value of a neural network in the introduction and (ii) as parameter of the Cauchy distribution)\n  - \\sigma used multiple times (e.g., (i) as standard deviation of a Gaussian and (ii) as standard deviation of the input noise)\n\n- References:\n  - [1] Roth et al., Hybrid Generative-Discriminative Training of Gaussian Mixture Models, Pattern Recognition Letters\n  - [2] Foong et al., 'In-Between' Uncertainty in Bayesian Neural Networks, ICML 2019 Workshop on Uncertainty and Robustness in Deep Learning", "summary_of_the_paper": "The paper proposes a method to propagate uncertainties through neural networks by performing a first-order approximation at nonlinear activation functions. By injecting uncertainty into the inputs, prediction uncertainties are obtained. These prediction uncertainties are exploited at training time by using new uncertainty-aware loss functions, and at test-time to obtained prediction uncertainties, out-of-distribution detection, and adversarial robustness.", "summary_of_the_review": "The proposed method is a simple and effective method to propagate distributions through a neural network. I am fine with the experimental results. However, I am not voting for accept right now since in my opinion there are some important details missing (as outlined in my review above) that are required to reproduce results and that would improve the quality of the paper. I am also missing a discussion of failure modes.\n\n--- POST REBUTTAL ---\n\nI have updated my rating from 5 to 6.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "cceuuffqsu", "reviewer": "Reviewer_bo8J", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The research problem of improving uncertainty estimates is very interesting and useful. However, the idea of propagating probability distribution through neural networks is not new. Specifically, the introduction section is a bit inaccurate and misleading. The authors claim that they focus on obtaining the propagated distribution in analytical form, or closed form, and that prior work [17] focuses only on Gaussian distributions (note that [17] is very early work from 1999). This is not true. For example, natural-parameter networks, or NPN [23] as cited in the paper, as well as its variants [12,26], considers exactly the same problem, and already handles distributions beyond Gaussian distributions; in fact their methods could handle any exponential-family distributions (including Gaussian ones) as well as various activation functions beyond ReLU. \n\nThis brings me to another of my concerns, which is the lack of proper baselines. Given the similarity the authors should at least include baselines from [23, 12, 26]. That would make the paper more convincing and maybe highlight what local linearization is a better solution. Currently, it seems [27] is the only modern baseline method (and note that [27] did not establish its improvement over methods such as [23, 12, 26]).\n\nIn terms of technical novelty, since the results for linear transformations are well known, the main novelty rests on the nonlinear part, i.e., the use of local linearization. Here the authors are able to prove that in terms of TV, local linearization is optimal in the Gaussian/ReLU case and Cauchy/ReLU case. In the introduction, the author claim that they are able to be ‘more general’ than some previous methods that can only handle Gaussian distributions. It is a bit disappointing to see that the proposed version can only handle Gaussian and Cauchy distributions and it has to be ReLU activation. \n\nIn Section 4.2, the authors mentioned that they can compute the exact probability, which is interesting. The trick is to change the softmax into a pariwise Gaussian comparison. However, it seems a lot of details are missing.\n\nTable 4 is a bit confusing. It shows that the proposed method actually underperforms the vanilla softmax CE loss. It is also unclear what is the setting for Section 5.4. For example, what is the input variance for ‘Pairwise Gaussian (PG)’, does PG produce better uncertainty estimates, etc.\n\nMinor:\nGrammar issue: Page 4: Note that when we … we refer to Dirac’s delta distribution.\n\n\n", "summary_of_the_paper": "This paper proposed an approach to propagate probability distribution through neural networks. In particular, the authors use local linearization to handle nonlinearity and show its optimality in terms of total variation for ReLU. Empirically they show that the proposed method can provide calibrated confidence intervals for regression problems and improve detection of OOD data in classification problems compared to some baselines. ", "summary_of_the_review": "Overall, the paper proposed a method for propagating uncertainty through neural network that could potentially be useful. However, it seems some important baselines are missing, the introduction is somehow misleading, the proposed method only works in very limited cases, i.e., Gaussian+ReLU and Cauchy+ReLU (while theoretical results on other activation such as sigmoid/tanh/other ReLU variants are not provided).  ", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "9bibwNBJMyf", "reviewer": "Reviewer_rdMQ", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strength:\n1. The proposed method is very simple, and has theoretical justifications with respect to total variation. The method is shown to be applied to relatively larger neural network structures where some previous methods are intractable.\n\n2. The improvements on the networks' robustness are interesting. This may bring more focus about studying the robustness and uncertainty of a network together.\n\nWeakness:\n1. The proposed method is only shown to have theoretical justifications for ReLU non-linearity. I think to extend it to more general non-linearities or give some more empirical analysis about the approximation, at least for the class of piece-wise linear functions, is important for this work. Also, it is not clear how to apply the method on max-pooling layers.\n\n2. Even though the method is simple, and efficient to some extend, the running time is still quite significant for large Resnet structures (with covariance). This limits the applicability of the proposed method to more realistic settings and datasets.\n\n3. The experimental settings are quite limited. Especially for the out-of-distribution data prediction, I think it's worth investigating more datasets, and different ways of introducing OOD data: for example, separating a dataset into different classes, or creating certain artifacts on a portion of the data.\n\n4. Given the network structure that the method can apply, the network is a piecewise linear model overall. Particularly, the model is a linear model for every data point. What will be the performance of the method if we propagate the distribution through every local linear model?", "summary_of_the_paper": "The paper proposes a simple method that propagates the uncertainty in the data through a neural network with ReLU non-linearity. The proposed method simply transforms the mean and covariance of the input Gaussian/Cauchy distribution through linear layers, and applies a local linear approximation for the ReLU activation. The proposed transformation is shown to be optimal under the total variation criterion. The paper applies the method to train neural networks with regression or classification targets, and show that the proposed method can well estimate the uncertainty of the network's output. Moreover, they also show some improvements in the robustness of the trained networks.", "summary_of_the_review": "The main strength of the paper is that the proposed method is simple. The main weakness is that the method is only applicable to limited kinds of layers of neural networks, and the paper lacks more comprehensive experiments to demonstrate the method's performance.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "-D8_Tr7G9R9", "reviewer": "Reviewer_H6dp", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths: \n- Interesting theoretical result showing that the local linearization is in fact an optimal approximation in terms of total variation for ReLU networks\n- Nice application of the method to study uncertainty quantification in neural networks\n\nWeaknesses: \n- Only consider the case where the activation function is ReLU (going beyond ReLU, can the linearization method still give reasonable approximations, under some conditions? Would be nice to have some theoretical results in this direction)\n\nComments/questions:\n- What is the role of depth in the uncertainty quantification and improvement in robustness to random/adversarial perturbations? \n- Can one formulate new loss function for learning with other non-Gaussian distributions (other than only Cauchy)?\n- On page 3 (beginning of Section 3.1), is there a typo? There, A is a m by n matrix and b is a 1 by n vector?", "summary_of_the_paper": "This paper studies the problem of propagating probability distributions through neural networks and applies the results to quantify prediction uncertainties. It proposes a local linearization method to approximate a distribution transformed by a ReLU network, as well as new loss function for learning with distribution-valued inputs. It also provides empirical results, showing that the method can quantify two kinds of uncertainty (aleatoric and epistemic) in classification and regression tasks, and training with the new loss function can improve robustness to random and adversarial perturbations.", "summary_of_the_review": "Overall, the strengths of the paper outweigh the weaknesses, although going beyond the ReLU activation (see Weaknesses above) could significantly improve the quality of the paper. Therefore, I am inclined to accept the paper.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper discusses propagating input uncertainty through non-linear layers by a simple local linearization approach. This is a straightforward idea and the authors explain how this is an optimal approximation of the propagated distribution for Total Variation and reLU non-linearity (for a single layer). This is an interesting (if quite limited) theoretical result. What is not clearly stated is that this result only holds for a single layer. It does not mean that the local approach is the best way to approximate (in the total variation sense) a distribution passed through multiple reLU layers.

By repeating the procedure, the authors are able to define closed form objective functions for noise-robust training of deep networks.

The reviewers found this an interesting paper and there was a good effort by the authors to improve the results. However, technical innovation is modest and reviewer doubts still remain. For that reason, clarity of presentation is critically important. The overall numerical score isn't convincing and with one reviewer remaining very unconvinced.

I agree with the reviewers that the technical contribution is quite limited and I would argue is not particularly well explained. For example, a simple alternative would be to use a "global" linearization in which one can consider the network function $f$ as a whole, and then simply linearize this (rather than linearizing each layer). Indeed, the way that the paper is written, this would be a natural interpretation since $f$ is defined in the introduction as the network function, but is used later differently (eg section 3.2) as the transfer function. The approach is to recursively compute a new mean and covariance for each layer, propagating these through the network (similar to moment matching approaches). It would have helped if the authors had made pseudocode for their approach. It would be natural and interesting to compare to the simple global linearization approach (which is computationally faster).

The presentation of results and experiments could be improved. For example, in figure 3, it is not clear (nor is it explained in the text) what the definition of "robust accuracy" is.  

Given the modest technical innovation, I also feel that the clarity of presentation isn't yet at the level that would merit acceptance. The paper would also benefit from some deeper insight into why the approach might perform better than other approaches (such as local moment matching) at the network level (rather than a single layer).
Decision: Reject