SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Florian Seligmann*
Philipp Becker
FZI Research Center for Information Technology
Michael Volpp
Bosch Center for Artificial Intelligence
Gerhard Neumann
Bayesian deep learning (BDL) is a promising approach to achieve well-calibrated predictions on distribution-shifted data. Nevertheless, there exists no large-scale survey that evaluates recent SOTA methods on diverse, realistic, and challenging benchmark tasks in a systematic manner. To provide a clear picture of the current state of BDL research, we evaluate modern BDL algorithms on real-world datasets from the WILDS collection containing challenging classification and regression tasks, with a focus on generalization capability and calibration under distribution shift. We compare the algorithms on a wide range of large, convolutional and transformer-based neural network architectures. In particular, we investigate a signed version of the expected calibration error that reveals whether the methods are over- or underconfident, providing further insight into the behavior of the methods. Further, we provide the first systematic evaluation of BDL for fine-tuning large pre-trained models, where training from scratch is prohibitively expensive. Finally, given the recent success of Deep Ensembles, we extend popular single-mode posterior approximations to multiple modes by the use of ensembles. While we find that ensembling single-mode approximations generally improves the generalization capability and calibration of the models by a significant margin, we also identify a failure mode of ensembles when finetuning large transformer-based language models. In this setting, variational inference based approaches such as last-layer Bayes By Backprop outperform other methods in terms of accuracy by a large margin, while modern approximate inference algorithms such as SWAG achieve the best calibration.
Real-world applications of deep learning require accurate estimates of the model's predictive uncertainty [2, 21, 38]. This is particularly relevant in safety-critical applications of deep learning, such as medical applications [95] and self-driving cars [27]. Therefore, we want our models to be calibrated: A model should be confident about its prediction if and only if the prediction will likely be correct. Only then it is sensible to rely on high-confidence predictions, and, e.g., to contact a human expert in the low-confidence regime [4]. Calibration is particularly relevant when models are evaluated on out-of-distribution (o.o.d.) data, i.e. on inputs that are very different from the training data, and hence, the model cannot always make accurate predictions. However, typical deep neural networks are highly overconfident on o.o.d. data [28, 72].
* Correspondence to florian.seligmann@student.kit.edu.
Bayesian deep learning (BDL) promises to fix this overconfidence problem by marginalizing over the posterior of the model's parameters. This process takes all explanations that are compatible with the training data into account. As desired, explanations will disagree on o.o.d. data, so that predictions will have low confidence in this regime. While computing the exact parameter posterior in BDL is infeasible, many approximate inference procedures exist to tackle this problem, aiming at making BDL applicable to real-world problems. Yet, recent BDL algorithms are typically only evaluated on the comparatively small and curated MNIST [52], UCI [16], and CIFAR [48] datasets with artificial o.o.d. splits. Existing BDL surveys [19, 20, 29, 72] concentrate on a few popular but relatively old algorithms such as Bayes By Backprop, Deep Ensembles, and Monte Carlo Dropout. In the light of recent calls for more realistic benchmarks of state-of-the-art (SOTA) algorithms [1] – with some experts going as far as calling the current state of BDL a “replication crisis”[2] – we aim to provide a large-scale evaluation of recent BDL algorithms on complex tasks with large, diverse neural networks.
Contributions. i) We systematically evaluate a comprehensive selection of modern, scalable BDL algorithms on large image- and text-based classification and regression datasets from the WILDS collection [47] that originate from real-world, safety-critical applications of deep learning (Section 5). In the spirit of Ovadia et al. [72], we focus on generalization capability and calibration on o.o.d. data, but consider more diverse and modern algorithms (Section 3) on realistic datasets with distribution shift. In particular, we include recent advances in variational inference such as natural gradient descent (iVON [53]) and low-rank posterior approximations (Rank-1 VI [17]). Furthermore, we use modern neural network architectures such as various ResNets [32], a DenseNet [35], and a transformer architecture [87]. ii) We present the first systematic evaluation of BDL for finetuning large pre-trained models, a setting that has recently gained attention in the context of BDL [79, 81]. We show that using BDL for finetuning gives a significant performance boost across a wide variety of tasks compared to standard deterministic finetuning (Section 5). iii) Inspired by the success of Deep Ensembles [51], we systematically evaluate the benefit of ensembling single-mode posterior approximations [4] (Section 5). iv) We use a signed extension of the expected calibration error (ECE) called the signed expected calibration error (sECE) that can differentiate between overconfidence and underconfidence, allowing us to better understand in which ways models are miscalibrated (Section 4). v) We compare the posterior approximation quality of the considered algorithms using the HMC samples from Izmailov et al. [37] (Section 5) and show that modern single-mode BDL algorithms approximate the parameter posterior better than Deep Ensembles, with further gains being achieved by ensembling these algorithms. Overall, our work is similar in spirit to Band et al. [4], but we compare the algorithms on more diverse datasets and focus on pure calibration metrics, thereby revealing failure modes of SOTA BDL algorithms that are not yet present in the literature. We provide code for all implemented algorithms and all evaluations<sup>3</sup>.
Several recent publications [1, 25] review the SOTA in uncertainty quantification using Bayesian models without providing experimental results. Yao et al. [93] compare a wide range of Markov Chain Monte Carlo [31] and approximate inference [6] methods on toy classification and regression datasets. Ovadia et al. [72] perform a large-scale experimental evaluation of a small selection of popular BDL algorithms on o.o.d. data and conclude that Deep Ensembles [51] perform best while stochastic variational inference [26] performs worst. Filos et al. [19] use a similar selection of algorithms but only evaluate on a single, large computer vision task not considering o.o.d. data. Foong et al. [20] artificially create o.o.d. splits for UCI datasets [16] and again find that variational inference performs worse than the Laplace approximation [57]. Mukhoti and Gal [63] and Gustafsson et al. [29] compare Monte Carlo Dropout [22] and Deep Ensembles in the context of semantic segmentation and depth completion, but, again, do not consider o.o.d. data. Nado et al. [65] evaluate many popular BDL algorithms on a small number of mostly artificially created o.o.d. image and text classification tasks. The work of Band et al. [4] is the most similar to ours, as they evaluate several BDL algorithms, including ensembles of single-mode posterior approximations, on two large image-classification datasets and consider o.o.d. data. Compared to Band et al. [4], we evaluate a different set of algorithms such as SWAG [59] and natural gradient descent variational inference [53]
$^{2}$ https://nips.cc/Conferences/2021/Schedule?showEvent=21827
<sup>3</sup>https://github.com/Feuermagier/Beyond_Depen_Ensembles
on a more diverse selection of datasets and network architectures, including transformer-based models and finetuning tasks, thereby revealing new failure modes of SOTA BDL methods. Competitions such as Wilson et al. [90] and Malinin et al. [60] also provide insights into the performance of different algorithms. However, the employed algorithms are typically highly tuned and modified for the specific tasks and thus of limited use to assess the general quality of the underlying methods in more diverse settings. Importantly, all winners of Wilson et al. [90] use ensemble-based algorithms.
3 Bayesian Deep Learning Algorithms
We assume a neural network with parameters  $\pmb{\theta}$  that models the likelihood  $p(\pmb{y} \mid \pmb{x}, \pmb{\theta})$  of an output  $\pmb{y}$  given an input  $\pmb{x}$ . Treating  $\pmb{\theta}$  as a random variable, the parameter posterior  $p(\pmb{\theta} \mid \mathcal{D})$  given a training dataset  $\mathcal{D} = \{(x_i, y_i) \mid i = 1, \dots, N\}$  of input-output pairs is defined by Bayes' theorem as
$$
p (\boldsymbol {\theta} \mid \mathcal {D}) = \frac {\prod_ {i} p \left(\boldsymbol {y} _ {i} \mid \boldsymbol {x} _ {i} , \boldsymbol {\theta}\right) p (\boldsymbol {\theta})}{\int \prod_ {i} p \left(\boldsymbol {y} _ {i} \mid \boldsymbol {x} _ {i} , \boldsymbol {\theta}\right) p (\boldsymbol {\theta}) \mathrm {d} \boldsymbol {\theta}}, \tag {1}
$$
where  $p(\boldsymbol{\theta})$  is a prior over parameters. The posterior  $p(\boldsymbol{\theta} \mid \mathcal{D})$  assigns higher probability to parameter vectors that fit the training data well and conform to our prior beliefs. Using  $p(\boldsymbol{\theta} \mid \mathcal{D})$ , a prediction  $\boldsymbol{y}$  given an input vector  $\boldsymbol{x}$  is defined as
$$
p (\boldsymbol {y} \mid \boldsymbol {x}, \mathcal {D}) = \int p (\boldsymbol {y} \mid \boldsymbol {x}, \boldsymbol {\theta}) p (\boldsymbol {\theta} \mid \mathcal {D}) \mathrm {d} \boldsymbol {\theta} = \underset {\boldsymbol {\theta} \sim p (\boldsymbol {\theta} \mid \mathcal {D})} {\mathbb {E}} [ p (\boldsymbol {y} \mid \boldsymbol {x}, \boldsymbol {\theta}) ]. \tag {2}
$$
This so-called Bayesian model average (BMA) [6, 58, 64, 89] encompasses the information of all explanations of the training data that are consistent with the parameter posterior. The BMA is especially valuable when dealing with large neural networks that are typically underspecified by the training data, where marginalizing over parameters can mitigate overfitting and promises significant accuracy and calibration gains [89]. While recent work has shown problems with certain types of covariate shift [36] and o.o.d. data [11], BDL not only promises calibration but also generalization gains [89].
3.1 Scalable Approximations for Bayesian Deep Learning
As computing the marginalization integral defining the normalization constant of the parameter posterior (Equation (1)) is intractable for neural networks, we have to resort to approximations. Approximate inference algorithms approximate the posterior, either by sampling from it or by computing an approximate distribution. Sampling-based Markov Chain Monte Carlo (MCMC) methods [31] such as Hamiltonian Monte Carlo (HMC) [67] sample directly from the true posterior and are therefore asymptotically exact. However, they are computationally very expensive and hence typically intractable in the context of BDL. Deterministic methods such as variational inference construct local approximations at a mode of the parameter posterior and are generally more computationally performant than MCMC [37], as they transform the posterior inference problem into an optimization problem that can be efficiently solved with standard gradient-based optimization techniques [6, 58, 64]. Therefore, we focus on these algorithms
in this work. This framework also encompasses standard deep learning, which is equivalent to a "Maximum A Posteriori" (MAP) estimate, i.e., a point estimate at the posterior maximum. In this section, we give a brief overview of the algorithms that we evaluate. See Appendix A for more detailed explanations and Appendix D for implementation details.
(a) MAP
(b) Single-Mode
(c) Ensemble
(d) MultiX
Figure 1: Posterior Approximation Types. MAP approximates a single posterior mode with a point estimate, while probabilistic single-mode approximations additionally capture the shape of the mode. Deep Ensembles approximate multiple modes with a mixture of point estimates. Likewise, MultiX employs a mixture of single-mode approximations to capture the shape of multiple modes. Figure adapted from Wilson and Izmailov [89].
Variational Inference. Variational inference (VI) minimizes the Kullback-Leibler divergence [50] between the approximate posterior and the true posterior [26]. Bayes By Backprop (BBB) [7] approximates the posterior with a diagonal Gaussian distribution and optimizes the mean and variance parameters with Stochastic Gradient Descent (SGD) [42]. Rank-1 variational inference (Rank-1 VI) [17] in contrast uses a low-rank posterior approximation, which reduces the number of additional parameters and allows the use of multiple components in the low-rank subspace. The improved Variational Online Newton (iVON) algorithm [53] still uses a diagonal Gaussian posterior but uses second-order information to better optimize the distribution parameters with natural gradients. Stein Variational Gradient Descent (SVGD) [55] is a non-parametric VI algorithm that approximates the posterior with multiple point estimates. SVGD is similar to a Deep Ensemble (see below) but adds repulsive forces between the particles to push them away from each other in parameter space.
Other Algorithms. Lakshminarayanan et al. [51] introduce Deep Ensembles that approximate the posterior with a few, typically five to ten, independently trained MAP models. As such, Deep Ensembles were originally considered a competing approach to Bayesian models [51] but can be viewed as Bayesian as they form a sum of delta distributions that approximate the posterior [89]. We follow this interpretation. The Laplace approximation [57] approximates the posterior with a second-order Taylor expansion around the parameters of a MAP model. We only consider the last-layer Laplace approximation [13] with diagonal and Kronecker-factorized [74] posterior approximations, which Daxberger et al. [13] find to achieve the best tradeoff between performance and calibration. Monte Carlo Dropout (MCD) [22] utilizes the probabilistic nature of dropout units that are part of many common network architectures to construct an approximation of a posterior mode. Stochastic Weight Averaging-Gaussian (SWAG) [59] periodically stores the parameters during SGD training and uses them to build a low-rank Gaussian posterior approximation. Finally, we evaluate Spectrally-Normalized Gaussian Processes (SNGP) [54] as a Bayesian baseline that does not infer a distribution over the model's parameters, but replaces the last layer by a Gaussian Process. The results for SNGP are therefore not directly comparable to the other algorithms' results.
3.2 MultiX
While single-mode posterior approximations such as BBB and SWAG capture the shape of a single mode of the parameter posterior, Deep Ensembles cover multiple modes but approximate each with a single point estimate. Hence, ensembling single-mode approximations promises even better posterior coverage and therefore improved uncertainty estimates (see Figure 1). This concept is not new: Tomczak et al. [86] experiment with an ensemble of BBB models on small datasets. Cobb et al. [10] use an ensemble of Concrete Dropout [23] models and Filos et al. [19] use MCD models. Both report accuracy improvements compared to a Deep Ensemble. Wilson and Izmailov [89] introduce MultiSWAG, an ensemble of SWAG [59] models. The winning teams of Wilson et al. [90] also show that ensembling Bayesian neural networks yields good posterior approximations. Similar to Band et al. [4] and Mehrtens et al. [61], we ensemble all considered single-mode posterior approximations (Section 3.1) except for SNGP to assess the performance gains on a per-algorithm basis. We use the term "MultiX" to refer to an ensemble of models trained with algorithm "X". We make an exception for "MultiMAP", which we keep referring to as Deep Ensemble for consistency with the existing literature.
4 Calibration Metrics
A calibrated model is defined as a model that makes confident predictions if and only if they will likely be accurate. While this definition directly implies a calibration metric for classification tasks [66], it has to be adapted for regression tasks, as "being accurate" is not a binary property in the regression case.
4.1 Unsigned Calibration metrics
Calibrated Classification. The calibration of a classification model can be measured with the expected calibration error (ECE) [28, 66]. By partitioning the interval  $[0,1]$  into  $M$  equally spaced bins and grouping the model's predictions into those bins based on their confidence values, we can calculate the average accuracy and confidence of each bin. The expected calibration error is then
given by  $\mathrm{ECE} = \sum_{m=1}^{M} |B_m| / |\mathcal{D}'| \mathrm{acc}(B_m) - \mathrm{conf}(B_m)|$  where  $B_m$  is the set of predictions in the  $m$ -th bin, and  $\mathrm{acc}(B_m)$  and  $\mathrm{conf}(B_m)$  are the average accuracy and confidence of the predictions in  $B_m$  (see Appendix B for details). An ECE of zero indicates perfect calibration.
Calibrated Regression. The confidence intervals of the predictive distribution can be used to measure the calibration of a regression model. Selecting  $M$  confidence levels  $\rho_{m}$  allows the computation of a calibration error based on the observed probability  $p_{\mathrm{obs}}(\rho_m)$ , calculated as the fraction of predictions that fall into the  $\rho_{m}$ -confidence interval of their respective predictive distributions:  $\mathrm{QCE} = 1 / M\sum_{m = 1}^{M}|p_{\mathrm{obs}}(\rho_m) - \rho_m|$ . We refer to this as the quantile calibration error (QCE), which simply replaces the quantiles in the definition of the calibration error from Kuleshov et al. [49] by confidence intervals. Using the confidence intervals allows a simpler interpretation of the resulting reliability diagrams (see Appendix B).
4.2 Signed Calibration Metrics
Models can be miscalibrated in two distinct ways: Overconfident models make inaccurate predictions with high confidence, and underconfident models make accurate predictions with low confidence. Arguably, overconfidence is worse in practice when applicants want to rely on the model's confidence to assess whether they can trust a prediction, for example in safety-critical applications of deep learning. However, none of the presented metrics can differentiate between overconfidence and underconfidence. Until now, this information was only apparent in reliability diagrams [28]. We propose two simple extensions of the ECE and the QCE that condense the information about overconfidence and underconfidence into a single scalar value by removing the absolute values: sECE and sQCE. We define these signed calibration metrics as
$$
\mathrm {s E C E} = \sum_ {m = 1} ^ {M} \frac {\left| B _ {m} \right|}{\left| \mathcal {D} ^ {\prime} \right|} \left(\operatorname {a c c} \left(B _ {m}\right) - \operatorname {c o n f} \left(B _ {m}\right)\right) \quad \text {a n d} \quad \mathrm {s Q C E} = \frac {1}{M} \sum_ {m = 1} ^ {M} \left(p _ {\mathrm {o b s}} \left(\rho_ {m}\right) - \rho_ {m}\right). \tag {3}
$$
A positive signed calibration error indicates that a model makes predominantly underconfident predictions and a negative signed calibration error indicates predominantly overconfident predictions. Perfectly calibrated models have a sECE/sQCE of zero. For models that are overconfident for some inputs but underconfident for others the signed calibration metrics may be zero, even though the model is not perfectly calibrated. This is typically not an issue in practice, as our experiments in Appendix C show that the absolute value of the signed metrics is usually very close to the absolute value of the corresponding unsigned metric, as most models are either overconfident or underconfident for nearly all predictions. Nevertheless, we always report the signed calibration metrics together with the unsigned calibration metrics to avoid any ambiguity.
For our comparison of the BDL algorithms introduced in Section 3.1, we focus on i) the ability of the models to generalize to realistic distribution-shifted data, ii) the calibration of the models under distribution shift, and iii) how well the models approximate the true parameter posterior. To assess the generalization capability and calibration of the models under realistic distribution shift, we use a subset of the WILDS dataset collection [47]. We assess the posterior approximation quality by comparing the model's predictive distributions to those of the HMC approximation provided by Izmailov et al. [37] for CIFAR-10 [48]. See Section 5.1 and Section 5.2 for details about the task, Section 5.3 for generalization results, Section 5.4 for calibration results, and Section 5.5 for posterior approximation quality results. All results for WILDS are directly comparable to the respective non-BDL o.o.d. detection algorithms on the WILDS leaderboard<sup>4</sup>, since we strictly follow their training and evaluation protocol. Whenever the BDL models' accuracy is competitive with the best performing algorithm on the WILDS leaderboard, the WILDS result is marked in the respective plot. We also report results for a subset of the smaller UCI [16] and UCI-Gap [20] tabular regression datasets in Appendix G.1. Appendix F contains information about the used computational resources and training times. Details regarding hyperparameters, training procedures, and additional results can be found in Appendix G. The results on all datasets are reported with a  $95\%$  confidence interval.
<sup>4</sup>https://wilds.stanford.edu/leaderboard/, last accessed on August 31, 2023
5.1 The WILDS Datasets
WILDS consists of ten diverse datasets that originate from real-world applications of deep learning in which models need to perform well under distribution shift. Standard o.o.d. datasets such as MNIST-C [62], CIFAR-10-C [33] and UCI-Gap [20] create distribution-shifted data by selectively removing data from the training split or artificially adding data corruptions onto the data in the evaluation split. WILDS represents real-world distribution shifts and is therefore more suitable for an application-oriented evaluation of BDL. We systematically evaluate all considered algorithms (see Section 3.1) on six of the ten datasets: The image-based regression task POVERTYMAP, the image classification tasks IWILDCAM, FMOW, and RXRx1, and the text classification tasks CIVILCOMMENTS and AMAZON. Aside from POVERTYMAP, all datasets are finetuning tasks, where we initialize the model's parameters from a model that has been pre-trained on a similar task. We also evaluate some algorithms on the CAMELYON17 image classification dataset but find that the performance degradation on the o.o.d. evaluation split is to a large part a consequence of the use of batch normalization rather than the o.o.d. data, making the dataset less interesting for a fair comparison on o.o.d. data (see Appendix E for details).
As we want to evaluate the posterior approximation, generalization, and calibration capability of all models given the true parameter posterior, none of our models use the metadata (e.g. location, time) associated with the input data, nor do we consider approaches that are specifically designed for o.o.d. generalization or augment the dataset, for example by re-weighting underrepresented classes, contrary to the algorithms evaluated by Koh et al. [47].