SlowGuess/NeurIPS_2023_OCR huggingface dataset:
In this section, we compare PICProp with Bayesian PINNs (BPINNs) [34] for the problem of Section 5.1. For BPINNs, the inference method for posterior estimation is Hamiltonian Monte Carlo (HMC) [4]. Although provided here for completeness, we believe that such a comparison can also be misleading because of the many differences between the two methods. The first difference pertains to the fact that PICProp does not consider parameter uncertainty. Further, the assumptions for these two methods are different: BPINN requires heavy assumptions about the data likelihood, for instance, while PICProp only requires the noise to be mean-zero. Furthermore, PICProp produces joint CIs for the whole domain, while the BPINN intervals are marginal. Finally, although supported by a sound Bayesian justification, BPINNs are not supported by a strict confidence guarantee for the produced intervals, in contrast to Theorem 4.1 that supports PICProp.
We summarize and contrast the two approaches in the following: BPINNs assume a NN parameter prior, and subsequently approximate and sample the parameter posterior to obtain samples from the solution posterior. This is done in order to fit a solution posterior distribution and subsequently compute its statistical intervals. In contrast, the problem of UQ is rethought ab initio in our paper. PICProp first considers either CIs given at the boundaries or data collected at the boundaries and estimates the corresponding CIs. Subsequently, the boundary CIs are propagated to the interior domain with mild assumptions and with the theoretical guarantee that the obtained CIs are indeed CIs according to the definition.
From the results summarized in L.1, we observe from Figure L.1 that the PICProp CIs are more conservative than the BPINN. From a design perspective, conservative uncertainty estimation is the preferred route for critical and risk-sensitive applications [7, 13], although it can be more costly. In contrast, most widely used UQ methods, such as deep ensembles [18] and Monte-Carlo dropout [28, 10], are known to be overconfident in practice [9, 7]. Finally, note that the BPINN results of Figure L.1 were obtained by using the software NeuralUQ developed in [38].
Figure K.5: Pedagogical example, Hoeffding: Although the results of both SimPICProp and EffiPICProp match PICProp results on the query points, EffiPICProp generalizes better on unseen locations.
Figure K.6: 2-D Poisson equation: Slices of CI predictions for  $y \in \{-0.5, 0.0, 0.5, 1.0\}$ .
Figure K.7: 2-D Poisson equation: Heatmaps of upper and lower CIs predicted by the exhaustive search (ES) and EffiPICProp.
(a)  $t = 0.0$
(b)  $t = 0.25$
(c)  $t = 0.5$
Figure K.8: Burgers equation: Meta-learning curves for  $x = 0$  demonstrating the stability of our method.
(d)  $t = 0.75$
(b)  $t = 0.25$
Figure K.9: Burgers equation: Meta-learning curves for  $x = 0.2$  demonstrating the stability of our method.
(c)  $t = 0.5$
Chi-squared
Figure L.1: Pedagogical example: The comparison between intervals produced by EffiPICProp and BPINNs [34] + HMC [4]. Noise on the boundary is Gaussian with known standard deviation  $\sigma = 0.05$ .