Paper: Uncertainty Quantification via Stable Distribution Propagation (ICLR-2024)

Reviewer comment (Writing):
The theoretical part of the paper only discusses uncertainty in terms of input/output/predictive uncertainties, while in Section 4 suddenly the terms epistemic and aleatoric uncertainty appear without any prior introduction to them.

Author response:
Thank you for pointing this out. We have replaced the terms aleatoric and epistemic uncertainties with data and model uncertainties, respectively, to be in harmony with the prior manuscript sections.

Reviewer comment (Writing):
Section 1, first paragraph. The abbreviation PNN has not been introduced at that stage of the paper.

Author response:
We have added the full form as Probabilistic Neural Network (PNN) on its first use.

Reviewer comment (Writing):
UCI and Iris are never cited.

Author response:
We have duly added the citations suggested by the referee. Additionally, we have also added citations for the MNIST and the EMNIST datasets used in the manuscript.

Reviewer comment (Presentation):
How are the Table 1 and Table 3 baselines tables to be read if they refer to multiple baselines?

Author response:
In Table 1, we use marginal moment matching (aka. moment matching / assumed density filtering), which relies on two equations, which are shared among all cited references, i.e., each approach collapses to the same equation / implementation. 
We included multiple references to illustrate that the method is still commonly used in many forms and applications today, while also providing attribution to its original use by Frey & Hinton in the 90s.

In Table 3, we followed "Tagasovska et al. [9]" in applying Deep Ensembles (Lakshminarayanan et al. [21]) to the UCI data uncertainty experiment.
However, as the best choice of Deep Ensembles was indeed using only 1 PNN (which makes sense because it is a data uncertainty experiment), we refer to it as PNN, for which the original reference is [34], and the results are indeed the same for each method.

Reviewer comment (Experiments):
How does the method perform with deeper networks? E.g., the ResNet-18 used in Sec4.3? It would be interesting to see (i) a comparison of this depth where SDP is still viable, and (ii) a deeper setup to which only marginal MM and marginal SDP.

Author response:
We agree with this concern, and have therefore extended our evaluations to ResNet-18 (Table 33) as well as other deeper networks (10, 15, 20, 25 layers; Tables 36-39).
We included the results in Supplementary Material D of the revision.
We find that SDP outperforms moment matching in each case by a substantial margin, and even marginal SDP outperforms moment matching in most cases.
Further, SDP also consistently outperformed DVIA for 10, 15, 20, and 25 layers.

Reviewer comment (Writing):
The term "PNN" is used without explaining it first in the Introduction.

Author response:
Thank you for pointing out the typos, we have fixed each of them.

Reviewer comment (Writing):
On Page 5, the phrase "the the" is repeated by mistake.

Author response:
Thank you for pointing out the typos, we have fixed each of them.

Reviewer comment (Experiments):
Table 2 used very small networks with only 4 layers. The biggest networks in the extra section have only 7 layers. This doesn't help us understand how the method works on big (practical) networks.

Author response:
To address this concern, we have run additional experiments. In particular, we have considered 10, 15, 20, and 25 layers in Tables 36-39.
Moreover, we have run a similar experiment for ResNet-18 and report the results in Table 33.
We find that SDP (i.e., with covariance matrix) outperforms moment matching in each case by a substantial margin, and even marginal SDP outperforms moment matching in most cases.

Reviewer comment (Reproducibility):
There's no code provided.

Author response:
We have included a PyTorch implementation of SDP in Supplementary Material E, and we will be publishing the code upon publication of the paper, also including marginal SDP and experiments.

Reviewer comment (Presentation):
Reference [16] has no authors mentioned in the paper.

Author response:
Thank you for pointing out the typos, we have fixed each of them.

Reviewer comment (Reproducibility):
How to choose the distribution which to propagate?

Author response:
The knowledge of the input uncertainty distribution would be based on the user's prior knowledge of the problem. In a case where such domain knowledge is absent, the user can choose the maximum entropy prior distribution (e.g., Gaussian) and estimate its parameters via cross validation.

Reviewer comment (Evaluation):
Figure 3 — which measure of uncertainty was used?

Author response:
The plot displays the means and standard deviations of the risk-coverage between all 10 seeds. Please elaborate and let us know in case this does not answer your question.

Reviewer comment (Theory):
Theorem 1 has limited applicability. It may not even be applicable to some of the authors' own methods.

Author response:
First, we would like to remark that Theorem 1 (+Corollary 2 and Corollary 3) apply generally to $\alpha$-stable distributions.
You are indeed correct that the Theorem 1 applies to univariate distributions. This is similar to how a result of "moment matching is an optimal estimator of the moments" is also only applicable in the univariate case. For this reason, we included extensive numerical studies to evaluate the behavior empirically (Tables 1-2, 6-31, 33, 36-39).

Reviewer comment (Theory):
As far as I can tell, Theorem 1 only works for univariate Gaussian distributions. Thus, this optimal approximation result does not apply to propagating the full covariance (including the SDP proposed in this paper).

Author response:
First, we would like to remark that Theorem 1 (+Corollary 2 and Corollary 3) apply generally to $\alpha$-stable distributions.
You are indeed correct that the Theorem 1 applies to univariate distributions. This is similar to how a result of "moment matching is an optimal estimator of the moments" is also only applicable in the univariate case. For this reason, we included extensive numerical studies to evaluate the behavior empirically (Tables 1-2, 6-31, 33, 36-39).

Reviewer comment (Theory):
As a compromise for computation efficiency, Section 3.3 proposes linearizing the entire network, including the ReLU activation. The covariance of the output distribution is computed by simply multiplying the Jacobian on the input covariance. Thus, the optimal approximation result in Theorem 1 does not apply either.

Author response:
We would like to clarify this confusion. Section 3.3 is not a compromise for computation efficiency. Instead, it shows how to efficiently compute SDP exactly and without introducing an approximation here. We added a clarification to the paper.

Reviewer comment (Evaluation):
Some evaluation metrics are not easy to compare. For example, Table 3 reports the prediction interval coverage probability (PICP, the higher the better) and the mean prediction interval width (MPIW, the lower the better). Each method has different PICP and MPIW which makes it hard to distinguish which one is the best. Since the network outputs a probability distribution, it might be better to report the test log likelihood, which captures the prediction accuracy and the length of the prediction interval simultaneously in a single metric.

Author response:
The objective of the experiment in the table reported was to evaluate the sharpness of best calibrated prediction intervals from different UQ methods. This test has been used by prior authors. The negative log likelihood would not inform about the calibration of the prediction interval for this comparison.

That said, we completely agree with the reviewer that the test negative log likelihood is the metric that captures both accuracy and width of prediction intervals simultaneously. As instructed by the reviewer, we have added test negative log likelihood comparisons for the experiment in Table 3 in the Supplementary Material (Table 34).

Reviewer comment (Experiments):
Comparison to other models seems a bit limited. DVIA is only used in a toy example due to its prohibited cost, Section 4.1 only has 2 baselines, none of which include, for example, Bayesian neural networks.

Author response:
Regarding uncertainty propagation experiments, DVIA and marginal moment matching are to our knowledge the only existing methods for propagating input uncertainties through neural networks. If applicable, please point us to a respective reference for any other propagation methods.

Regarding Section 4.1, we remark that Bayesian neural networks are for estimating model uncertainty and not for estimating data uncertainty. Accordingly, their applicability in this experiment is limited. Further, MC Estimate is a quite strong baseline, as it is an unbiased estimator and we remark that the column PNN includes deep ensembles, which only collapse to PNN because deep ensembles performs best using only one model (as it is a data uncertainty experiment).

Reviewer comment (Writing):
End of the first paragraph. PNN is not defined.

Author response:
Thank you, we have added the full form as Probabilistic Neural Network (PNN) on its first use.

Reviewer comment (Writing):
Section 1. What is Y?

Author response:
Y is the true output distribution, which is defined via $f(x+\epsilon)\sim Y$. We have clarified this in the revision.

Reviewer comment (Presentation):
Ref 16 does not have authors.

Author response:
Thank you, we fixed reference 16.

Reviewer comment (Novelty):
I believe Section 2 is missing an important, one of the first works in the area. Hernández-Lobato, J.M. and Adams, R., 2015. Probabilistic backpropagation for scalable learning of Bayesian neural networks. In International conference on machine learning (pp. 1861-1869).

Author response:
Thank you, we included it in the revised manuscript.

Reviewer comment (Experiments):
Table 1. It is not very clear what marginal moment matching with several references means. Did the authors try all these references and report the best results among them?

Author response:
Moment matching, assumend density filtering, etc., all use the same equation for propagating moments, and therefore all references collapse to the same equation / method.
We refer to it as marginal moment matching because we also explicitly discuss full moment matching (which is computationally intractible, but DVIA provides an approximation to full moment matching).

Reviewer comment (Writing):
Elaboration of the idea of pairwise probability of correct classification is required in Section 3.5.

Author response:
Historically, one-vs-one settings have had a long tradition of being applied to multi-class classification problems.
In particular, the ground truth class score is compared to each of the false class scores, a loss is computed for each of these comparisons, and then these losses are averaged.
Historically, this gave rise to the multivariate logistic losses, which later was improved to softmax-based losses.
The "pairwaise probability of correct classification" is defined in Equations 7-9, i.e., it is the probability that the score for the true class is greater than the score for a respective false class. We clarified the description in the paper.

Reviewer comment (Theory):
The theory only works for certain types of activations and distributions. This should be acknowledged and discussed more clearly in the paper.

Author response:
Regarding the theoretical result, yes, it only applies to ReLU non-linearities for stable distributions, but we remark that the approach performs well on various other non-linear functions (Leaky-ReLU, sigmoid, SiLU, GeLU, MaxPool) empirically as shown, e.g., in Tables 18-31 and Table 33. 
We have included an additional statement in the introduction regarding the restriction on input distributions.

Reviewer comment (Experiments):
The computational considerations remain somewhat unclear to me. In the experiments section, training a resnet appears to take 30x longer. There is a simpler mar-SDP method that is faster, but it is unclear to me where it is evaluated and what are the pros/cons of going between SDP and mar-SDP. Also, the methods section seems to suggest that computational cost is not a big concern (saying things like "the Jacobian easy to compute"), but it's not clear to me why that is the case, especially given the empirical results.

Author response:
Regarding the computational considerations, the Jacobian is relatively easy to compute in comparison to other approaches that either require exponential complexity (computing it exactly) or cubed complexity in the number of activations in the neural network (DVIA). 
The computational cost of computing the Jacobian is typically proportional to m propagations where m is the number of classes / output dimension.
The reason for why it was 25 times slower than regular training was only an implementation reason. 
With our current implementation and a more recent PyTorch version than originally, SDP takes only 4.3 times longer compared to the default on ResNet-18 with CIFAR-10.

Reviewer comment (Experiments):
The experiments are on very simple datasets (UCI, MNIST). I wonder if the computational cost is something that makes it hard to test on bigger datasets.

Author response:
The reason for us to limit the uncertainty propagation experiments primarily to UCI is actually the large computational cost of obtaining the ground truth oracles (which we do by propagating large numbers of samples).
We now have uncertainty propagation experiments for ResNet-18 on CIFAR-10 (Table 33), where computing the oracle took many hours compared to the distribution propagation methods, which took seconds.

Reviewer comment (Experiments):
The experimental results are not entirely convincing. On the UCI table, while directionally things look right, the numbers are often within error bars. The baselines are also not very convincing: in the UCI table, the MC method is known to not be very good in practice. There are probably stronger baselines such as deep ensembles (Lakshminarayanan et al., NeurIPS 2017), or post-hoc recalibration (Kuleshov et al., ICML 2018), or more recent methods for quantile estimation (Si et al, ICLR 2022). (Perhaps SDP can be combined with these methods)? The second experiment has even fewer baselines, and the table is missing error bars (although they seem to be in the figure).

Author response:
Regarding, "MC Estimate", we want to point out that this is not "MC Dropout", which may have caused confusion.
With "MC Estimate", we refer to the procedure of sampling k inputs from the input distribution, propagating each sample through the network, and then computing mean and covariance matrix based on these samples. This is quite a strong method as it is an unbiased estimator.
We have added a clarification in the paper for Table 3.
Further, we would like to point out that, for data uncertainty, the best deep ensemble is an ensemble with only 1 model, which makes it exactly equivalent to a PNN (the ensembling helps with model uncertainty, but not for data uncertainty). 
Indeed, we ran deep ensembles (with an additional option of only 1 model), and found that 1 model performs best, which is why we labelled it PNN, but still cite the original PNN as well as (Lakshminarayanan et al., NeurIPS 2017) at this point in Table 3. 
This aligns with the results in Tagasovska et al. (2019), where we adopted the experiment from.
We included references to the methods you mentioned.
For post-hoc recalibration, we want to remark that recalibration is orthogonal to the choice of the "base UQ method", which means it could be applied on top of each of the methods in Table 3, including out methods.

Regaring the experiments in Table 4, we have now included a Table with standard deviations (Table 35).
The experiment has a number of baselines, in particular, softmax entropy, dirichlet outputs / distributions, moment matching, and orthonormal certificates, as well as combinations of these methods.

Reviewer comment (Reproducibility):
How is the method applied end-to-end to a neural net?

Author response:
Thank you for the suggestion. In our revision, we have now included a pseudo-code implementation as well as an optimized PyTorch implementation of SDP in Appendix E to demonstrate how the method is applied end-to-end to a neural net.
We remark that for marginal SDP (or marginal moment matching) each layer of the neural network requires separate implementations as these do not allow for end-to-end implementations, which we will provide upon publication.

Reviewer comment (Reproducibility):
Can the authors provide a more detailed discussion of computational considerations?

Author response:
Regarding the computational considerations, the Jacobian is relatively easy to compute in comparison to other approaches that either require exponential complexity (computing it exactly) or cubed complexity in the number of activations in the neural network (DVIA). 
The computational cost of computing the Jacobian is typically proportional to m propagations where m is the number of classes / output dimension.
The reason for why it was 25 times slower than regular training was only an implementation reason. 
With our current implementation and a more recent PyTorch version than originally, SDP takes only 4.3 times longer compared to the default on ResNet-18 with CIFAR-10.

Reviewer comment (Theory):
What are the effects of assumptions on the distribution and the activation type?

Author response:
Regarding the theoretical result, yes, it only applies to ReLU non-linearities for stable distributions, but we remark that the approach performs well on various other non-linear functions (Leaky-ReLU, sigmoid, SiLU, GeLU, MaxPool) empirically as shown, e.g., in Tables 18-31 and Table 33. 
We have included an additional statement in the introduction regarding the restriction on input distributions.

Reviewer comment (Experiments):
How does the method compare to stronger baselines?

Author response:
Regarding, "MC Estimate", we want to point out that this is not "MC Dropout", which may have caused confusion.
With "MC Estimate", we refer to the procedure of sampling k inputs from the input distribution, propagating each sample through the network, and then computing mean and covariance matrix based on these samples. This is quite a strong method as it is an unbiased estimator.
We have added a clarification in the paper for Table 3.
Further, we would like to point out that, for data uncertainty, the best deep ensemble is an ensemble with only 1 model, which makes it exactly equivalent to a PNN (the ensembling helps with model uncertainty, but not for data uncertainty). 
Indeed, we ran deep ensembles (with an additional option of only 1 model), and found that 1 model performs best, which is why we labelled it PNN, but still cite the original PNN as well as (Lakshminarayanan et al., NeurIPS 2017) at this point in Table 3. 
This aligns with the results in Tagasovska et al. (2019), where we adopted the experiment from.
We included references to the methods you mentioned.
For post-hoc recalibration, we want to remark that recalibration is orthogonal to the choice of the "base UQ method", which means it could be applied on top of each of the methods in Table 3, including out methods.

Regaring the experiments in Table 4, we have now included a Table with standard deviations (Table 35).
The experiment has a number of baselines, in particular, softmax entropy, dirichlet outputs / distributions, moment matching, and orthonormal certificates, as well as combinations of these methods.

Reviewer comment (Experiments):
Can the method be combined with other UQ methods?

Author response:
Beyond the combination of SDP with PNNs, SDP could be combined also with other UQ methods that complement each other.
For example, the hybrid SDP+PNN method with ensembling and adversarial training along the lines of Deep Ensembles to account for both data and model uncertainties.
Further, post-hoc recalibration could also be combined with SDP.