Input-gradient space particle inference for neural network ensembles
Authors: Trung Trinh
Markus Heinonen
Luigi Acerbi
Samuel Kaski
Venue: ICLR 2024 spotlight (2024)
Keywords: deep ensembles
diversity
input gradient
robustness
covariate shift
particle variational inference
Primary area: probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)
TL;DR: We learn an ensemble of neural networks that is diverse with respect to their input gradients.

Abstract: Deep Ensembles (DEs) demonstrate improved accuracy, calibration and robustness to perturbations over single neural networks partly due to their functional diversity. Particle-based variational inference (ParVI) methods enhance diversity by formalizing a repulsion term based on a network similarity kernel. However, weight-space repulsion is inefficient due to over-parameterization, while direct function-space repulsion has been found to produce little improvement over DEs. To sidestep these difficulties, we propose First-order Repulsive Deep Ensemble (FoRDE), an ensemble learning method based on ParVI, which performs repulsion in the space of first-order input gradients. As input gradients uniquely characterize a function up to translation and are much smaller in dimension than the weights, this method guarantees that ensemble members are functionally different. Intuitively, diversifying the input gradients encourages each network to learn different features, which is expected to improve the robustness of an ensemble. Experiments on image classification datasets and transfer learning tasks show that FoRDE significantly outperforms the gold-standard DEs and other ensemble methods in accuracy and calibration under covariate shift due to input perturbations.

Published as a conference paper at ICLR 2024
INPUT-GRADIENT SPACE PARTICLE INFERENCE FOR NEURAL NETWORK ENSEMBLES
Trung Trinh Markus Heinonen' Luigi Acerbi
Samuel Kaskit.5
1 Department of Computer Science, Aalto University, Finland Department of Computer Science, University of Helsinki, Finland DDeparmentt of Computer Science, University of Manchester, United Kingdom {t rung. trinh, markus.. o heinonen, samuel. kaski (aski)@aalla eallto. fi, luigi. weeerbi@@essnnki fi
ABSTRACT
Deep Ensembles (DEs) demonstrate improved accuracy, calibration and robustness to perturbations over single neural networks partly due to their functional diversity. Particle-base based variational inference (ParVI) methods enhance diversity by formaliz- ing repulsion term based on network similarity kernel. However, weight-space repulsion is inefficient due to over-parann peeeraraamttrinztionn, while direct function-space repulsion has been found to produce little improvement over DEs. To sidestep these difficulties, we propose First-order Repulsive Deep Ensemble (FoRDE), an ensemble learning method based on ParVI, which performs repulsion in the space of first-order input gradients. As input gradients uniquely characterize a function up translation and are much smaller in dimension than the weights, this method guarantees that ensemble members are functionally different. Intuitively, diver- sifying the input gradients encourages each network to learn different features, which is expected to improve the robustness of an ensemble. Experiments on image classification datasets and transfer learning tasks show that FoRDE significantly outperforms the gold-standard DEs and other ensemble methods in accuracy and calibration under covariate shift due input perturbations.
INTRODUCTION
Ensemble methods, which combine predictions from multiple models, are a well-known strategy in machine learning (Dietterich, 2000) to boost predictive performance (Lakshminarayanan et al., 2017), uncertainty estimation (Ovadia et al., 2019), robustness to adversarial attacks (Pang et al., 2019) and corruptions (Hendrycks & Dietterich, 2019). Deep ensembles (DEs) combine multiple neural networks from independent weight initializations (Lakshminarayanan DEs are simple to implement and have promising performance, their weight-based diversity does necessarily translate into useful functional diversity (Rame & Cord, 2021; D' Angelo & Fortuin, 2021: Yashima et al., 2022).
al., 2017). While
Particle-based variational inference (ParVI) (Liu & Wang, 2016; Chen et al., 2018: Liu al., 2019; Shen et al., 2021) has recently emerged as a direction to promote diversity in neural ensembles from the Bayesian perspective (Wang al., 2019; D' Angelo & Fortuin, 2021). Notably, the ParVI update rule adds kernelized repulsion term k(f. between the ensemble networks f, f' for explicit control of the ensemble diversity. Typically repulsion is done in the weight space capture different regions in the weight posterior. However, due to the rneeettramtttizotioonn of neural networks, alternative approach to define the repulsion
weight-space repulsion suffers from redundancy. in function space (Wang al., 2019; D' Angelo & Fortuin, 2021), which requires the challenging computation of a kernel between functions. Previous works avoided this issue by comparing functions only on training inputs, which leads to underfitting (D' Angelo et al., 2021). Neither weight nor function space repulsion has led to significant improvements over vanilla DEs.
From functional perspective, model can also be uniquely represented, up translation, using its first-order derivatives, i.., input gradients f. Promoting diversity in this third view input gradients has notable advantages:

---

Published as a conference paper at ICLR 2024
Figure I: Input-gradient repulsion increases functional diversity. An illustration of input gradient repulsion ID regression with neural networks. Left: At some point during training, the models fit well the training samples yet exhibit low functional diversity. Middle: As training proceeds, at each data point, the repulsion term gradually pushes the input gradients (represented by the arrows) away from each other unit sphere. Right: As result, the end of training, the ensemble has gained functional diversity.
1. each ensemble member is guaranteed to correspond to a different function;
2. input gradients have smaller dimensions than weights and thus are more amenable to kernel comparisons:
3. unlike function-space repulsion, input-gradies repulsion does lead training point underfitting (See Fig. and the last panel of Fig. 2);
4. each ensemble member is encouraged to learn different features, which can improve robust- ness.
In this work, propose ParVI neural network ensemble that promotes diversity in their input gradients, called First-order Repulsive deep ensemble (FoRDE). Furthermore, we devise data- dependent kernel that allows FoRDE outperform other ensemble methods under input corruptions on image classification tasks. Our code is available at ht /ts::////hhht cwm.cooooaaacccaec. PMMMPML/ FORDE.
BACKGROUND
Bayesian neural networks In Bayesian neural network (BNN), we treat the model's weights 0 as random variables with prior p(0). Given dataset D {(x,, Un ) and likelihood function ply|x, 0) per data point, we infer the posterior over weights (() using Bayes' rule
D() p(D(0)p(0) Jo ((D)))(())
p(0) II P(yn Nn. So P(0) II p(yn X 0)d0
where the likelihood is assumed to factorize over data. To make a prediction on a test sample x", we integrate the inferred posterior in Eq. (1), practice called Bayesian model averaging (BMA):
p(yxx", D) - p = Ep(o [p(v|x", 8)]
However, computing the integral in Eq. (2) is intractable for BNNs. Various approximate inference methods have been developed for BNNs, including variational inference (VI) (Graves, 2011; Blundell et al., 2015), Markov chain Monte Carlo (MCMC) (Neal, 2012; Welling & Teh, 2011: Zhang et al., 2020) and more recently ParVI (Liu & Wang, 2016; Wang et al., 2019; D' Angelo & Fortuin, 20211 l).
Deep ensembles As opposed to BNNs, which attempt to learn the posterior distribution, DEs (Lakshminarayanan et al., 2017) consist of multiple aakiimum-a-posteed (MAP) estimates trained from independent random initializations. They can capture diverse functions that explain the data under different random conditions will likely converge different
well, independent training modes in the posterior landscape. DEs have been shown to be better than BNNs both accuracy and uncertainty estimation (Ovadia et al., 2019; Ashukha al., 2020; Gustafsson al., 2020).
2

---

Published as a conference paper at ICLR 2024
Deep ensemble
Weight-RDE
Function-Ree -DD
FoRDE (ours)
2
1
0
-1
-10
-10
-10
-10
Figure 2: Input gradient ensembles (FoRDE) capture higher uncertainty than baselines. Each
panel shows predictive uncertainty in ID regression for different (repulsive) deep ensemble methods.
Deep ensemble
Weight-RDE
Function-RDE
FoRDE (ours)
0.0
2.5
-5.0
-0.2
0.0
Figure 3: Uncertainty of FoRDE is high in all input regions outside the training data, and is higher than baselines. Each panel shows the entropy of the predictive posteriors 2D classification.
Particle-based variational inference for neural network ensembles ParVI methods (Liu & Wang, 2016; Chen et al., 2018; Liu et al., 2019; Shen et al., 2021) have been studied recently to formalize neural network ensembles. They approximate the target posterior using set of samples, or particles, by deterministically transporting these particles to the target distribution (Liu & Wang, 2016). ParVI methods are expected to be more efficient than MCMC as they take into account the interactions between particles in their update rules (Liu et al., 2019). These repulsive interactions are driven by kernel which measures the pairwise similarities between particles, i.c., networks (Liu et al., 2019).
The current approaches compare networks in weight space 0 in function space (:; Weight-space repulsion is ineffective due difficulties in comparing extremely -igh-dimensional weight vectors and the existence of weight symmetries (Fort et al., 2019; Entezari al., 2022). Previous studies show that weight-space ParVI does not improve performance over plain DEs (D' Angelo Fortuin, 2021; Yashima et al., 2022). Comparing neural networks via function kernel is also challenging since functions are infinite-dimensionall objects. Previous works resort to comparing functions only on subset of the input space (Wang al., 2019; Angelo & Fortuin, 2021). Comparing functions over training data leads to underfitting (D' Angelo & Fortuin, 2021; Yashima et al., 2022). likely because these inputs have known labels, leaving no room for diverse predictions without impairing performance.
3 FORDE: FIRST-ORDER REPULSIVE DEEP ENSEMBLES
In this section, present framework perform ParVI in the input-gradient space. We start by summarizing Wasserstein gradient descent (WGD) in Section 3.1. and show how to apply WGD for input-gred saauut-rrdienn-spaccee repulsion in Section 3.2. We then discuss hyperparameter selection for the input-gradient kernel in Section 3.3, and outline practical considerations in Section 3.4.
set of M neural networks {ʃi + f(x: 0,) ).) We focus on the supervised classification setting: given labelled dataset D {(x., yn > with C classes and inputs x,, E IR" we approximate the posterior p(0(D) using the M particles. The output fx; f) for input x a vector of size C whose y-th entry [xx ,),, is the logit of the y-th class.
Throughout this paper, we assume a of M weight particles (0.) (0 corresponding
3

---

Published as a conference paper at ICLR 2024
3.1 WASSERSTEIN GRADIENT DESCENT
Following D' Angelo & Fortuin (2021), we use a ParVI method called Wasserstein gradient descent (WGD) (Liu al., 2019; Wang al., 2022). Given an intractable target posterior distribution p(1(D) and a set of particles [0:),= from distribution the goal of WGD is to find the particle distribution p².
= argmin ,L (p),
(3)
(EP:(())
where (p) is shorthand for the standard Kullback-Leible divergence
(p) p,))) log p(0) - log r(0)],
(4)
and (2) is the Wasserstein space equipped with the Wasserstein distance W2 (Ambrosio et al., 2005; Villani, 2009). WGD solves the problem in Eq. (3) using a Wasserstein gradient flow (pr)e-
which is roughly the family of steepest descent curves of KL, (-) in P2(0). The tangent vector of
this gradient flow at time f is
(0) = Vogg x(0) V log (e(
(5)
whenever P: is absolutely continuous (Villani, 2009; Liu et al., 2019). Intuitively, (0) points to the direction where the probability mass 0 of Pr should be transported in order to bring Pe closer Since Eq. (5) requires the analytical form of Pe which we do not have access to, we use kernel density
estimation (KDE) obtain tractable approximation P induced by particles (0(²) JM at time t,
M
(0) α
(0,
(6)
where k is positive semi-definite kernel. Then, the gradient of the approximation is
= Vok (ok (o) v log Po(
EM
(7)
ΣN M (0,0
Using Eq. (7) in Eq. (5), obtain practical update rule for each particle g(t) of p.:
Σ ii=l
V Bi ((())),(()))
g(++1) g(t) nt
Voco log (8(0)
(8)
M 1e ((()) 060))
driving Force
repulsion force
where > 0 is the step size at optimization time t. Intuitively, we can interpret the first term in the particle gradient as the driving force directing the particles towards high density regions of the posterior, while the second term is the repulsion force pushing the particles away from each other.
3.2 DEFINING THE KERNEL FOR WGD IN INPUT GRADIENT SPACE
We propose to use kernel comparing the input gradients of the particles,
k(O,,0,) 0,) def ((xx))~p(xxx) (xx((xxx)) Vxf(x; 0)))
(9)
where is a base kernel between gradients Vxf(x: f)y that are of same size as the inputs In essence, we define i as the expected similarity between the input gradients of two networks with respect the data distribution p(x, y)- Interestingly, by using the kernel k the KDE approximation I of the particle distribution not only depends on the particles themselves but also depends on the data distribution. We approximate the kernel using the training samples, with linear complexity:
N
A(O, 0,) = (0. = N
Σ K ((Vxf(Xn;8))) ym: Vx fXxx 0z)..)) n=1
(10)
The kernel only compares the gradients of the true label V VxI(xn; 0) p.,. as opposed to the entire Jacobian matrix (x, 8), as our motivation to encourage each particle to learn different features that could explain the training sample (x..3n) well. This approach also reduces computational complexity, since automatic differentiation libraries such as JAX (Bradbury et al., 2018) or Pytorch (Paszke et al., 2019) would require C passes, one per class, to calculate the full Jacobian.
4

---

Published as a conference paper at ICLR 2024
Choosing the base kernel We choose the RBF kernel on the unit sphere our base kernel R:
Vx 0) [xx E E
x(s - exp -)
(1I)
where S. denote the two normalized gradients of two particles with respect to one input, and E E RDx is diagonal matrix containing squared lengthscales. We design K to be norm-agnostic since the norm of the true label gradient Vf f)
ll2 fluctuates during training and as training converges, the log-probability f(x,;; p,. log P(Un Xx, 0) will approach log 1, leading to the norm [ f) ll2 approaching O due to the saturation of the log-softmax activation. Furthermore, comparing the normed input gradients between ensemble members teaches them to learn complemen- tary explanatory patterns from the training samples, which could improve robustness of the ensemble. The RBF kernel is an apt kernel to compare unit vectors (Jayasumana al., 2014), and we can control the variances of the gradients along imput dimensions via the square lengthscales E.
3.3 SELECTING THE LENGTIISCALES FOR THE BASE KERNEL
In this section, we present a method to select the lengthscales for the base kernel. These lengthscales are important for the performance of FoRDE, since they control how much repulsion force is applied in each dimension of the input- gruuigg space. The dimension wise repulsion is (Liu & Wang, 2016)
kss s': 3d 8/4 k(s EEE, Edd
(12)
asd
where see that along the d-th dimension the inverse square lengthscale Edd controls the strength of the repulsion Vs,n(s, E): smaller lengthscale corresponds to a stronger force." Additionally, since the repulsion is restricted to the unit sphere in the input-gradient space, increasing distance in one dimension decreases the distance in other dimensions. As result, the repulsion motivates the ensemble members to depend more on dimensions with stronger repulsion in the input gradient space for their predictions, while focusing less on dimensions with weaker repulsion. One should then apply stronger repulsion dimensions of data manifold with higher variances.
To realize the intuition above in FORDE, we first apply Principal Component Analysis (PCA) to discover dominant features in the data. In PCA, we calculate the eigendecomposition
C UAUT
DxD
(13)
of the covariance matrix C = XTXX(N 1) of the centered training samples X E N to get eigenvectors and eigenvalues (ua, Aa The d-th eigenvalue Xa is the variance of the data along eigenvector ug, offering natural choice of inverse eigenvalues X- as the squared lengthscales Edd of the principal components. Let x = UTx denote the representation of the input in eigenbasis U = jn U2 ... up). We compute the gradient kernel in PCA basis U and set Σ = A
k(s,s') def cxp (-2(0Ts UT
(T) - - UTs')
(14)
where A is diagonal eigenvalue matrix. While setting the square inverse lengthscales equal the eigenvalues seems problematic at first glance, since large eigenvalues will push the kernel x towards O, this problem is avoided in practice since we also employ the median heuristic, which introduces in the kernel global bandwidth scaling term that adapts to the current pairwise distance between particles, as discussed below in Section 3.4.
Connection to the EmpCov prior Recently, Izmailov et al. (2021a) proposed the EmpCov prior for the weight columns of the first layer of a BNN:
N(0, + cI),
WERD
(15)
where a > 0 determines the prior scale and e > 0 is a small constant ensuring a positive definite covariance. The prior encourages first layer weights to vary more along higher variance data dimensions. Samples from this prior will have large input gradients along high variance input dimensions. In this sense, the EmpCov prior has a similar effect to the kernel in Eq. (14) on ensemble members. The difference is that while Izmailov et al. (2021a a) incorporates knowledge of the data manifold into the prior, we embed this knowledge into our approximate posterior via the kernel.
'erre we assume that the lengthscales are set appropriately that the kemel x does not vanish, which is truc since we the median heuristic during training (Section 3.4).
5

---

Published as a conference paper at ICLR 2024
3.4 PRACTICAL CONSIDERATIONS
In this section, we detail two important considerations to make FoRDES work in practice. We include the full training algorithm in Appendix C.I.
Mini-batching To make FoRDE amenable to mini-batch gradient optimization, we adapt the kernel Eq. (10) mini-batch of samples B {(x6. 36))
B
(0, 02)
K Vx/(Xxx; (xb; Os) Vx (xx(xx;;)jo))
(16)
In principle, this kernel in the update rule in Eq. (8) leads biased stochastic gradients of the repulsion term because the average over batch samples in Eq. (16) is inside the logarithm. However, in practice, found no convergence issues in our experiments.
Median heuristics Since perform particle optimization with an RBF kernel following earlier works (Liu Wang, 2016; Liu al., 2019), we adopt the median heuristic (Schölkopf et al., 2002). Besides the lengthscales, we introduce global bandwidth h in our base kernel in Eq. (11):
(8i, Sj; Σ) exp
2h (s. 8,) T (s: -ss (8,-))) )
S;
Vxf(x; 0,),
E ER. (17)
V 0,)g212
During training, the bandwidth h is adaptively set to med? (2 log M), where med? is the median of the pairwise distance (s; s,) Σ (ss sy) between the weight samples {0:)
3.5 COMPUTATIONAL COMPLEXITY
Compared to DES, FoRDES take roughly three times longer to train. In addition to a pass to calculate the ligllike,hoood, need additional forward-backward pass calculate the input gradients, and another backward pass to calculate the gradients of the input gradient repulsion practice: in RESNET 8/CIFAR-
with respect the weights. This analysis is confirmed experiments of Section 5.2 with an ensemble size of 10, a DE took ~31 seconds per epoch on an Nvidia 1100 GPU, while FoRDE took 1111 seconds per epoch.
RELATED WORKS
Besides the ParVI methods mentioned in Section discuss additional related works below.
Diversifying input gradients of ensembles Local independent training (LITT) (Ross et al., 2018) 8) orthogonalizes the input gradients ensemble members by minimizing their pairwise squared cosine similarities, and thus closely resembles our input-gradies gradients on a hyper-sphere. However, their goal is to find a maximal set of models that accurately predict the data using different sets of distinct features, while our goal is to induce functional diversity in an ensemble. Furthermore. we formulate our kernelized repulsion term based on the ParVI framework, allowing to choose hyperparameter settings (orthogonal basis and lengthscales) that imbue the ensemble with beneficial biases (such as robustness to corruption).
repulsion term which diversifies input
Gradient-based attribution methods for deep models One application of input gradients is to build attribution (or saliency) maps, which assign importance to visually-interpetabbe input features for specified output (Simonyan et al., 2014: Bach et al., 2015; Shrikumar et al., 2016; Sundararajan et al., 2017; Shrikumar et al., 2017). Our method intuitively utilizes the attribution perspective of input gradients to encourage ensemble members learn different patterns from training data.
Improving corruption robustness of BNNs Previous works have evaluated the predictive uncer- tainty of BNNs under covariate shift (Ovadia et al., 2019; Izmailov et al., 2021b), with Izmailov et al. (2021b) showing that standard BNNs with high- figh-iideity posteriors perform worse than MAP solutions on under corruptions. Izmailoy et al. (2021a) attributed this phenomenon to the lack of posterior contraction in the null space of the data manifold and proposed the EmpCov prior

---

Published as a conference paper at ICLR 2024
remedy. As discussed in Section 3.3, the PCA kernel works in the same manner the EmpCov prior and thus significantly improves robustness FoRDE against corruptions. Trinh et al. (2022) studied teight-based BNNs, and showed that by
the robustness of node-BNNs, increasing the entropy the posterior, node-BNNs become more robust against corruptions. Wang & Aitchison (2023) allow BNNs adapt the distribution shift at test time by using test data statistics.
efficient alternative
Table 1: FORDE-PCA achieves the best performance under corruptions while FoRDE-Identity outperforms baselines on clean data. FORDE- Tunee outperforms baselines on both clean and corrupted data. Results of RESNET 88/ CIFAR- 100 averaged over 5 seeds. Each ensemble has 10 members. cA, cNLL and cECE are accuracy, NLL., and ECE on CIFAR- 100-C.
METHOD
N
ACCURACY (%)
ECE
CA/CNEL CRCE
NODE-BNNS
0.74
79.7.10.3
0.004.00.002
54.8 1,96 0.00 53.0 2.03 0.005
SWAG
0.73+0.01
79.4+0.1
0.038+00.01
DEEP ENSEMBLES
0.70=0.00
81.8:0.2
0.041=0.0003
54.3 1.99 0.05
WEIGHT-
0.70
81.7.0033
0.043.10.004
2.02 0.006
FUNCTION-RID -DDE
0.76=0.02
80.1+0.4
0.042++0.005
51.9 2.08 0.00
FEATURE-RDE
0.75-0.04
82.1=0.3
0.072:10.033
54.8 1 0.006
LIT
0.70=0.00
81.9+0.2
0.040++0.00
54.41 1.98 0.00
FORDE- PCA (OURS)
0.71
814.002
0,039
56.1 1.90 0.05
FORDE-IDENTIE (OURS) 0.70=0.00
82.1
0.043+0.001
54.1 2.02 0.005
FORDE-TUNED (OURS) 0.70=0.00
82.1=0.2
0.044+0.002
55.3 /.005
Table 2: FORDE-PCA achieves the best performance under corruptions while FoRDE-Identity has the best NLL clean data. FoRDE-Tuned outperforms most baselines on both clean and corrupted data. Results of RESNET 18 CIFAR- 10 averaged over seeds. Each ensemble has 10 members. cA, eNLL and cECE accuracy, NLL. and ECE on CIFAR- 10-C.
METHOD
NLL 1
ACCURACY (%)
ECE
CA/CNLL / CECE
NODE-BNNS
0.127.10.00 10.009
95.9.10.3
0.006.0.000
78.2 0.09
SWAG
0.124:10.001
96.910.1
0.005110.001
7 0.07
DEEP ENSEMBLES
0.117
96.3
0.005
7 0.08
WEIGHT-RDE
0,117
96.2.10.1
0.005
7..0 0.08
-DDE
0.128+00.001
95.8+-0.2
0.006+0.000
7 0.08
PEATURE-RDE
0.116::0.001
96.4:10.1
0.004
0 0.08
LIT
0.116=00.001
96.4+-0.1
0.004=0.001
0.09
FORDE-PCA (OURS)
0.125++0 00000
96.1+-0.1
0.006+0.000
80.5 / 0.71 0.07
FORDE-IDENTITY (OURS)
0.113 LO. L0.002
96.3.10.1
0.005 10.001
78.0/ 0.80 0.08
FORDE-TUNED (OURS) 0.114+0.002
96.4+-0.1
0.005=0.001
79.1 / 0.07
Table 3: FoRDE outperforms EmpCov priors under corruptions, while maintaining competitive
performance on clean data. Results of RESNETI8 on CIFAR- 10 evaluated over seeds. Each ensemble has 10 members. cA, cNLL and cECE are accuracy, NLL, and ECE on CIFAR- 10-C. Here we use the EmpCov prior for all methods except FoRDE.
METHOD
NLL
ACCURACY (%)
ECE 1
CA/ CECE
DEEP ENSEMBLES
0..0010.001
96.2110.1
0.00010.001
78.7/0.76/0.00 0.08
WEIGHT- RDD
0.120=0.001
96.0-0.1
0.006-10.001
78.8/0.76/0.00 0.08
FUNCTION- RDE
0.132-0.000
95.6=0.3
0.007-0.001
77.8/0.79
FEATURE- RDE
0.118=0.001
96.2+:0.1
0.005
78.9/0.74/0.00
FORDE-PCAL (OURS)
0.105
96.1-001
0.006-10.001
80.5 0.71 0.07 0.07
EXPERIMENTS
5.1 ILLUSTRATING FUNCTIONAL DIVERSITY
To show that FoRDE does produce better functional diversity than plain DE and other repulsive DE approaches, repeated the 1D regression of Izmailov et al. (2019) and the 2D classification
7

---

Published as a conference paper at ICLR 2024
experiments of D' Angelo & Fortuin (2021). We ensembles of 16 networks for these experiments. Fig. shows that FoRDE exhibits higher predictive uncertainty in the imput regions outside the training data compared to the baselines in ID regression. For the 2D classification task, visualize the entropy of the predictive posteriors in Fig. 3, which also shows that FoRDE has higher uncertainty than the baselines. Furthermore, FORDE is the only method that exhibits high uncertainty in all input regions outside the training data, property mainly observed in predictive uncertainty of Gaussian processes (Rasmussen & Williams, 2006).
5.2 COMPARISONS TO OTHER REPULSIVE DE METHODS AND BNNs
We report performance of FoRDE against other methods on CIFAR- CIFAR-10/100 (Krizhevsky, 2009) in Tables 1-2 and TINYIMAGENET (Le & Yang, 2015) in Appendix A. Besides the PCA lengthscales introduced in Section 3.3, we experiment with the identity lengthscales Σ = and with tuned lengthscales where we take the weighted average of the PCA lengthscales and the identity lengthscales. Details on lengthscale tuning are presented in Appendix D.4. For the repulsive DE (RDF) baselines, we choose weight RDE (D'Aneelo& & Fortuin, 2021), function RDE (D' Angelo & Fortuin, 2021) and feature RDE (Yashima ct al., 2022). We also include LIT (Ross et al., 2018), node-BNNs (Trinh et al., 2022) and SWAG (Maddox et al., 2019) baselines. We use an ensemble size of 10. We use standard performance metrics of expected calibration error (ECE) (Naeini et al., 2015), negative log- likelihood (NLL) and predictive accuracy. For evaluations on input perturbations, we use CIFAR- 10CAR-10/100 10/100-C and TINYIMAGENET-C provided by Hendrycks & Gimpel (2017), which are datasets of corrupted test images containing 19 image corruption types across 5 levels of severity, and we report the accuracy, NI.L and ECE averaged over all corruption types and severity levels (denoted cA, cNLL and cECE in Tables 1-4). We use RESNET 18 (He et al., 201 for CIFAR- 10-RR-0011000000 and PREACTRESNET 1 8 (He et al., 2016b) for TINYIMAGENET. Experimental details are included in Appendix C.2.
Tables and show that FORDE-PCA outperforms other methods under input corruptions across all metrics, while maintaining competitive performance on clean data. For instance, FoRDE-PCA shows a +1.3% gain on CIFAR- I 10-C and +2.4% gain on CIFAR- 10-C in accuracy compared the second-best results. As stated in Section 3.3, the PCA kernel encourages FoRDE to rely more on features with high variances in the data manifold to make predictions, while being less dependent on features with low variances. This effect has been shown in Izmailov et al. (2021a) to boost model robustness against perturbations, which explains why FoRDE with the PCA kernel performs better than the baselines on input corruptions.
On the other hand, Tables 1 and show that FoRDE with identity lengthscales outperforms the baselines in terms of NLL CIFAR- 10 and has the best accuracy on 1IFAR-100.. However, FoRDE- FoRDE-Ldentity is slightly less robust than DE against corruptions. We suspect that with the identity lengthscales, FoRDE also learns to rely on low-variance features to make predictions, which is harmful to performance under corruptions (Izmailov et al., 2021a).
Finally, Tables 1 and 2 show that FoRDE with tuned lengthscales (FORDE-Tuned) outperforms the baselines on both clean and corrupted data in most cases, suggesting that the optimal lengthscales for good performance on both clean and corrupted data lie somewhere between the identity lengthscales and the PCA lengthscales. Additional results on lengthscale tuning are presented in Appendix D.4.
5.3 COMPARISONS TO EMPCOV PRIOR
As stated in Section 3.3, our approach is similar to the EmpCov prior (Izmailov et al., 2021a). We thus perform comparisons against ensemble methods where the EmpCov prior is defined for the first layer instead of the standard isotropic Gaussian prior. We report the results of RESNET 18/CIFAR- 10 in Table 3, where we use the EmpCov prior for all ensemble methods except FORDE. Comparing Table to Table 2 indicates that the EmpCov prior slightly improves robustness of the baseline ensemble methods against corruptions, while also leading to a small reduction in performance on clean data. These small improvements in robustness are not surprising, since for ensemble methods consisting of approximate MAP solutions, the isotropic Gaussian prior already minimizes the influences of low-variance data features the ensemble's 's predictions (Izmailov et al., 2021a). We argue that besides minimizing the influences of low variance features on predictions via the PCA kernel, FoRDE also encourages its members to learn complementary patterns that can explain the data well, and these
8

---

Published as a conference paper at ICLR 2024
Accuracy
Epistemic unrtainity
CIAR-10
95
0.4
0.4
0.2
85
0.2
0.000
0.0
CIAR-10
CINC10
CHAR-10-C
CHIAR-10
CINICIO
CIAR-1D-C
CHAR-10 CNCC10 DIAR-11-C
CHHAR-100
CIFAR-100
90
80
0.3
0.02
0.0
0.00
60
0.0
OFAR-100
CIFAR-100-C
CIFAR-100
CIFAR-100-C
CIAAN-100
CIFAR-100-C
CIFA-11
Weight
Function:
Figure FoRDE outperforms competing methods transfer learning. First three columns: report NLL., ECE and accuracy on in-distribution test sets and under covariate shift. For CIFAR-10, we use CINIC 10 (Darlow al., 2018) evaluate models under natural shift and 1FFAR-10-6 for corruption shift. For CIFAR-100, we evaluate on CIFAR- 100-C. FoRDE performs better than the baselines in all cases. Last column: We evaluate functional diversity by calculating epistemic uncertainty of ensembles on out-of-distribution (OOD) datasets using the formula in Depeweg et al. (2018). We use CIFAR- 100 as the OOD test set for CIFAR- 10 and we use CIFAR- 10 and CINIC 10 as OOD test sets for CIFAR- 100. FoRDE exhibits higher functional diversity than the baselines.
two effects act in synergy improve the robustness of the resulting ensemble. Thus, Table 3 shows that FoRDE still more robust against corruptions than the baseline methods with the EmpCov prior.
5.4 TRANSFER LEARNING EXPERIMENTS
To show the practicality of FORDE, we evaluated its performance in transfer learning scenario. We use the outputs of the last hidden layer of Vision Transformer model pretrained IMAGENET-2 as input features and train ensembles of 10 networks. We report the results on CIFAR- 10 the first row and CIFAR- I (0 in the second row in Fig. 4. Overall, Fig. 4 shows that FoRDE is better than the baselines across all cases. See Appendix C.3 for experimental details.
6 DISCUSSION
In this section, we outline directions to further improve FORDE.
Reducing computational complexity One major drawback of FoRDEs is the high computational complexity discussed in Section 3.5. To circumvent this problem, could either (i) only caleulate the repulsion term after every >> 1 epochs, or (ii) using only a subset of batch samples at each iteration calculate the repulsion term.
Reducing the number of lengthscale parameters Here the RBF kernel which requires us to choose appropriate lengthscales for good performance. To avoid this problem, could explore other kernels suitable for unit vector comparisons, such those introduced in Jayasumana et al. (2014). Another solution is to study dimensionality reduction techniques for input gradients before calculating the kernel, which can reduce the number lengthscales be
base kernel,
CONCLUSION
In this work, we proposed FoRDE, an ensemble learning method that promotes diversity the input- gradient space among ensemble members. We detailed the update rule and devised data-dependent kernel suitable for imput-gradien repulsion. Experiments on image classification and transfer learning tasks show that FoRDE outperforms other ensemble methods under covariate shift. Future directions include more efficient implementations of the method and reducing the burden of hyperparameter selection as discussed in Section 6.

---

Published as a conference paper at ICLR 2024
ACKNOWLEDGMENTS
This work was supported by the Research Council of Finland (Flagship programme: Finnish Center for Artificial Intelligence FCAI and decision no. 359567, 345604 and 341763), ELISE Networks of Excellence Centres (EU Horizon: 2020 grant agreement 951847) and UKRI Turing AI World-Leading Researcher Fellowship (EP/W002973/1), We acknowledge the computational resources provided by Aalto Science-IT project and CSC-IT Center for Science, Finland.
ETHICS STATEMENT
Our paper introduces a new ensemble learning method for neural networks, allowing deep learning models be more reliable practice. Therefore, we believe that our work contributes towards making neural networks safer and more reliable to use in real-world applications, especially those that are safety-critical. Our technique per se does not directly deal with issues such fairness, bias or other potentially harmful societal impacts, which may be caused by improper usages of machine learning or deep learning systems (Mehrabi al., 2021). These issues would need be adequately considered when constructing the datasets and designing specific deep learning applications.
REPRODUCIBILITY STATEMENT
For the purpose of reproducibility of our results with new ensemble learning method, we have included in the Appendix detailed descriptions of the training algorithm. For each experiment, we include in the Appendix details about the neural network architecture, datasets, data augmentation procedures and hyperparameter settings. All datasets used for our experiments are publicly available. We have included our codes in the supplementary material and we provide instructions on how to run our experiments in README md available in the provided codebase. For the transfer learning experiments, we used publicly available pretrained models which we have mentioned in the Appendix.
REFERENCES
Luigi Ambrosio, Nicola Gigli, and Giuseppe Savaré. Gradient flows: in metric spaces and in the space of probability measures. Springer, 2005.
Arsenii Ashukha, Alexander Lyzhov, Dmitry Molchanov, and Dmitry Vetrov. Pitfalls of in-domain uncertainty estimation and ensembling in deep learning. In International Conference Learning Representations, 2020.
Sebastian Bach, Alexander Binder, Grégoire Montavon, Frederick Klauschen, Klaus-Robert Robert Müller, and Wojeiech Samek. On pixel-wise explanations for non-lincar classifier decisions by layer-wise relevance propagation. PloS ONE, 10(7):e0300140 2015.
Charles Blundell, Julien Cornebise, Koray Kavukcuoglu, and Daan Wierstra. Weight uncertainty in neural network. In International conference machine learning. 2015.
James Bradbury, Roy Frostig, Peter Hawkins, Matthew James Johnson, Chris Leary, Dougal Maclau- rin, George Necula, Adam Paszke, Jake VanderPlas, Skye Wanderman-Milne, and Qiao Zhang. JAX: composable transformations Python+NumPy programs, 2018.
Changyou Chen, Ruiyi Zhang, Wenlin Wang, Bai Li, and Liqun Chen. A unified particle-optimization framework for scalable Bayesian sampling. In Uncertainty Artificial Intelligence, 2018.
Francesco D' Angelo and Vincent Fortuin. Repulsive deep ensembles are Bayesian. In Advances in Neural Information Processing Systems, 2021.
Francesco D' Angelo, Vincent Fortuin, and Florian Wenzel. On Stein variational neural network
ensembles. In ICML workshop Uncertainty and Robustness in Deep Learning, 2021.
Luke N Darlow, Elliot Crowley, Antreas Antoniou, and Amos Storkey. Cinic- is not imagenet or cifar-10. 10. arXiv preprint urXiv:1810.08:: 2018.
10

---

Published as a conference paper at ICLR 2024
Stefan Depeweg, lose-Miguel Hernandez-Lobate Finale Deshi,Velez, and Steffen Udluft. Decompo- sition of uncertainty in Bayesian deep learning for efficient and risk-sensitive learning. In Jennifer Dy and Andreas Krause (eds.). Proceedings of the 35th International Conference Machine Learning, volume 80 of Proceedings of Machine Learning Research, pp. 1184-1193. PMLR, 10-15 Jul 2018. URL https: ppwppeoeeedeecee ngs. www.mmlrr...//.mmmmmmmmm /80 /eppweell 1 Ba html.
Thomas G. Dietterich. Ensemble methods in machine learning. In International Workshop on Multiple Classifier Systems, 2000.
Rahim Entezari, Hanie Sedghi, Olga Saukh, and Behnam Neyshabur. The role of permutation invariance linear mode connectivity neural networks. International Conference on Learning Representations, 2022.
Stanislav Fort, Huiyi Hu, and Balaji Lakshminarayanan. Deep ensembles: A loss landscape perspec- tive. In NeurlPS workshop Bayesian Deep Learning, 2019.
Alex Graves. Practical variational inference for neural networks. In Advances in Neural Information Processing Systems, 2011.
Fredrik K Gustafsson, Martin Danelljan, and Thomas B Schon. Evaluating scalable bayesian deep learning methods for robust computer vision. In IEEE/CVF Conference Computer Vision and Pattern Recognition workshops, 2020.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image
recognition. In IEEE conference on Computer Vision and Pattern Recognition, 2016a.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Identity mappings in deep residual networks. In European Conference on Computer Vision, 2016b.
Dan Hendrycks and Thomas Dietterich. Benchmarking neural network robustness common corruptions and perturbations. In International Conference on Learning Representations, 2019.
Dan Hendrycks and Kevin Gimpel. baseline for detecting misclassified and distr--dist examples in neural networks. In International Conference on Learning Representations, 2017.
Pavel Izmailov, Wesley Maddox, Polina Kirichenko, Timur Garipov, Dmitry Vetrov, and Andrew Gor- don Wilson. Subspace inference for bayesian deep learning. Uncertainty in Artificial Intelligence (UAI), 2019.
Pavel Izmailov, Patrick Nicholson, Sanae Lotfi, and Andrew G Wilson. Dangers of Bayesian model averaging under covariate shift. In Advances in Neural Information Processing Systems, 202 la.
Pavel Izmailov, Sharad Vikram, Matthew D Hoffman, and Andrew Gordon Wilson. What are bayesian neural network posteriors really like? In International Conference on Machine Learning, 2021b.
Sadeep Jayasumana, Richard Hartley Mathieu Salzmann, Hongdong Li, and Mehrtash Harandi. Optimizing over radial kernels on compact manifolds. In IEEE Conference on Computer Vision and Pattern Recognition, 2014.
Alex Krizhevsky. Learning multiple layers of features from tiny images. Technical report, 2009.
Balaji Lakshminarayanan, Alexander Pritzel, and Charles Blundell. Simple and scalable predictive uncertainty estimation using deep ensembles. In Advances in Neural Information Processing Systems, 2017.
Ya Le and Xuan S. Yang. Tiny ImageNet visual recognition challenge. 2015.
Chang Liu, Jingwei Zhuo, Pengyu Cheng, Ruiyi Zhang, and Jun Zhu. Understanding and accelerating particle-based variational inference. In International Conference Machine Learning, 2019.
Qiang Liu and Dilin Wang. Stein variational gradient descent: A general purpose Bayesian inference algorithm. In Advances in Neural Information Processing Systems, 2016.

---

Published as a conference paper at ICLR 2024
Wesley Maddox, Pavel Izmailov, Timur Garipov, Dmitry P Vetrov, and Andrew Gordon Wilson A simple baseline for Bayesian uncertainty deep learning. In Advances in Neural Information Processing Systems, 2019.
Ninareh Mehrabi, Fred Morstatter, Nripsuta Saxena, Kristina Lerman, and Aram Galsryan. A survey on bias and fairness in machine learning. ACM Comput. Surv., 54(6), jul 2021. ISSN 0360-0300. doi: 10.1 10.1145/3457677. URL https: //oii , org/10. /1/. 1144/11457660
Mahdi Pakdaman Naeini, Gregory Cooper, and Milos Hauskrecht. Obtaining well calibrated probabilities using Bayesian binning. In AAAI Conference on Artificial Intelligence, 2015.
Radford M Neal. Bayesian learning for neural networks, volume 118 of Lecture Notes in Statistics. Springer, 2012.
Yaniv Ovadia. Emily Fertig. Jie Ren, Zachary Nado, David Sculley, Sebastian Nowozin, Joshua Dillon, Balaji Lakshminarayanan, and Jasper Snoek. Can you trust your model's uncertainty? Evaluating predictive uncertainty under dataset shift. In Advances in Neural Information P Processing Systems, 2019.
Tianyu Pang, Kun Xu, Chao Du, Ning Chen, and Jun Zhu. Improving adversarial robustness via promoting ensemble diversity. In International Conference on Machine Learning, 2019.
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Kopf, Edward Yang. Zachary Vetito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. Pytorch: An imperative style, high-performance deep learning library. Advances in Neural Information Processing Systems, 2019.
Alexandre Rame and Matthicu Cord. DICE: Diversity in deep ensembles via conditional redundancy adversarial estimation. In International Conference Learning Representations, 2021.
Carl Edward Rasmussen and Christopher K. I. Williams. Gaussian processes for machine learning. Adaptive computation and machine learning. MIT Press, 2006. ISBN 026218253X.
Andrew Slavin Ross, Weiwei Pan, and Finale Doshi-Velez Learning qualitatively diverse and interpretable rules for classification. arXiv preprint /8X6v:/806..000
2018
Bernhard Schölkopf, Alexander Smola, Francis Bach, et al. Learning with kernels: support vector machines, regularization, optimization, and beyond. MIT press, 2002.
Zheyang Shen, Markus Heinonen, and Samuel Kaski. De-randomizing MCMC dynamics with the diffusion Stein operator. In Advances in Neural Information Processing Systems, 2021.
Avanti Shrikumar, Peyton Greenside, Anna Shcherbina, and Anshul Kundaje. Not just black box: Learning important features through propagating activation differences. arXiv, 2016.
Avanti Shrikumar, Peyton Greenside, and Anshul Kundaje. Learning important features through propagating activation differences. In International conference on machine learning, 2017.
Karen Simonyan, Andrea Vedaldi, and Andrew Zisserman. Deep inside convolutional networks:
Visualising image classification models and saliency maps. In International Conference on Learning Representations Workshop, 2014.
Mukund Sundararajan, Ankur Taly, and Qiqi Yan. Axiomatic attribution for deep networks. In International conference on machine learning, 2017.
Trung Q Trinh, Markus Heinonen, Luigi Acerbi, and Samuel Kaski. Tackling covariate shift with node-based Bayesian neural networks. In International Conference on Machine Learning, 2022.
Cédric Villani. Optimal transport: old and new, volume 338 of Grundlehren der mathematischen Wissenschaften. Springer, 2009.
Xi Wang and Laurence Aitchison. Robustness to corruption in pre-trained Bayesian neural networks. In International Conference on Learning Representations, 2023.
12

---

Published as a conference paper at ICLR 2024
Yifei Wang, Peng Chen, and Wuchen Li. Projected Wasserstein gradient descent for high-dimensional
Bayesian inference. SIAM/ASA Journal on Uncertainty Quantification, 10(4):1513-13 1532, 2022.
Ziyu Wang, Tongzheng Ren, Jun Zhu, and Bo Zhang. Function space particle optimization for
Bayesian neural networks. In International Conference on Learning Representations, 2019.
Max Welling and Yee W Teh. Bayesian learning via stochastic gradient Langevin dynamics. In International Conference on Machine Learning. 2011.
Shingo Yashima, Teppei Suzuki, Kohta Ishikawa, Ikuro Sato, and Rei Kawakami. Feature space particle inference for neural network ensembles. International Conference on Machine Learning, 2022.
Sergey Zagoruyko and Nikos Komodakis. Wide residual networks. In British Machine Vision Conference, 2016.
Ruqi Zhang, Chunyuan Li, Jianyi Zhang, Changyou Chen, and Andrew Gordon Wilson. Cyclical stochastic gradient MCMC for Bayesian deep learning. In International Conference on Learning Representations, 2020.
13

---

Published as a conference paper at ICLR 2024
A RESULTS ON TINYIMAGENET
Table FORDE-PCA performs best under corruptions while having competitive performance on clean data. Results of PREACTRESNET 18 on TINYIMAGENET evaluated over seeds. Each ensemble has 10 members. cA, cNLL and cECE are accuracy, NLL, and ECE on TINYIMAGENET-C.
METHOD
NLL L
ACCURACY 1
ECE
CA/C CECE
NODE- NODE-BNNS
1.39:+0.01
67.610.3
0.114:0.004
30.4/3.40 0.05
SWAG
1.39:1001
66.610.3
0.020
28.4/3.72
DEEP ENSEMBLES
1.15:10.00
71.61:.00
0.035 0.02
31.8/3.38 00009 0.09
WHIGHT-RDE
1.15-10.01
71.510.00
0.036
31.7/3.39/0.. /0009
FUNCTION-RDD RDE
1.21
70.2+00.5
0.036
31.1/3.43 00.10
FEATURE-RDS RDE
1.24::0.01
72.0=0.1
0.100:t0.003
/ 0.09
LIT
1.15+0000
71.5+0000
0.035++0.002
31.2 / 3.40 0.11
FORDE-PCA (OURS)
1.16+0.00
71.4+0.0
0.003+0.002
32.2 / 3.28 0.08
B PERFORMANCE UNDER DIFFERENT ENSEMBLE SIZES
We report the NLL of FoRDE and DE under different ensemble sizes NIFAAR100/10 and CIFAR- 10/ 10/100-C in Figs. 5-6. We use the WIDERESNET 6xx (Zagoruyko & Komodakis, 2016) architecture for this experiment. These figures show that both methods enjoy significant improvements in perfor- mance as the ensemble size increases. While Fig. 5a and Fig. 6a show that FoRDE underperforms DE on clean images, Fig. 5b and Fig. 6b show that FoRDE significantly outperforms DE corrupted images, such that FoRDE with 10 members has the same or better corruption robustness of DE with 30 members.
Method
0.66
FORDE
2.20
DE
0.65
2.15
2.10
0.64
2.05
10
20 Ensemble size
20 Ensemble size
30
(a) CIFAR- 1000 (clcan)
(b) CIFAR- 100-- (corrupted)
Figure 5: FoRDE is competitive on in-distribution and outperforms DEs under domain shifts by corruption. Performance of WIDERESNETI6X4 on CIFAR-100 100 over 5 seeds.
C TRAINING PROCEDURE
C.I TRAINING ALGORITHM FOR FORDE
We describe the training algorithm of FoRDE in Algorithm
C.2 EXPERIMENTAL DETAILS FOR IMAGE CLASSIFICATION EXPERIMENTS
For all the experiments, we used SGD with Nesterov momentum as our optimizer, and we set the momemtum coefficient to 0.9. We used weight decay À of 5 x 10-4 and we set the learning rate to 10-1 We used a batch size of 128 and we set e in Algorithm to 10 12, We used 15 bins to calculate ECE during evaluation.
14

---

Published as a conference paper at ICLR 2024
Algorithm FoRDE
1: Input: training data D, orthonormal basis U, diagonal matrix of squared lengthscales Σ, a neural
network ensemble (ff (.)) of size M, positive sealar c, number of iterations T, step sizes
(Th Vo))zz. weight decay X
2: Output: optimized parameters {0 { AM
3: Initialize parameters {0(0) ).M
4: for 1 to 7 do
5: Draw mini-batee (xu. yb 1
D.
6: for = 1 to B do
7:
for i 1 to M do
Calculate the normalized input gradients for each 0; (Eq. (11))
8:
V Vx. (x1; 0(())
Si,b
(18)
x,, (xt; 03())
yn + x2
9:
end for
10:
for i = do
Calculate the pairwise squared distance in Eq. (14)
11:
for j to M do
1
di.j.b -- 2 (SL. 83,6) "
JUEU (si.6 83.6)
(19)
end for end for
13:
14:
Calculate global bandwidth per batch sample using the median heuristic (Eq. (17)):
hg <- median(((4,;))) M.M
(2
(20)
15:
end for
16:
for = to do
Calculate the pairwise kernel similarity using Eq. (16) and Eq. (17)
17:
for j = 1 to M do
1
ki
2 exp ( di.,h/hb) b=1
(21)
B
18:
end for
19:
end for
20:
for --i-1 to M do
21:
Calculate the gradient of the repulsion term using Eq. (7):
M
Rij
rop
(22)
kig
22:
Calculate the gradient data of the cross-entropy loss with respect to 0,.
23:
Calculate the update vector in Eq. (8):
(8)
1
(g.ata
data
g rep)
(23)
B
24:
Update the parameters and apply weight decay:
((++ - 800) + ne (v - -20(0)
(24)
end for 26: end for
15

---

Published as a conference paper at ICLR 2024
0.85
0.1250
0.80
0.1225
Method
0.1200
FORDE
NLL
0.75
0.1175
0.70
0.1150
0.65
10
20
30
10
20 Ensemble size
30
Ensemble size
CIFAR-10 (clean)
(b) CIFAR- 10-AR-O--C (corrupted)
Figure 6: FoRDE is competitive on in-distribution and outperforms DEs under domain shifts by corruption. Performance of WIDERESNETI6X4 on CIFAR-10 over 5 seeds.
the standard data augmentation procedure, which includes input normalization, random cropping and random horizontal flipping. We ran each experiments for 300 epochs. We decreased the learning rate 7 linearly from 10-1 to 10-3 from epoch 150 to epoch 270. For evaluation, we used all available types for corruptions and all levels of severity in CIFAR- -C---.
On CIFAR-10 10 and CIFAR-
00, -000
On TINYIMAGENET, we use the standard data augmentation procedure, which includes input normal- ization, random cropping and random horizontal flipping. We ran each experiments for 150 epochs. We decreased the learning rate linearly from 10-1 to 10-3 from epoch 75 to epoch 135. For evaluation, we used all available types for corruptions and all levels of severity in TINYIMAGENET-C.
For weight-RDE and FORDE, we only imposed prior on the weights via the weight decay parameter. For Maturre-DEE and function-RDes we followed the recommended priors in Yashima al. (2022). For feature-RDE, we used Cauchy prior with prior scale of 10-3 for CIFAR-10 and a prior scale of x 10-3 for both CIFAR-100 and TINYIMAGENET, and we used a projection dimension 5. For function-RDE. we used Cauchy prior with prior scale of 10 for all datasets.
C.3 EXPERIMENTAL DETAILS TRANSFER LEARNING EXPERIMENTS
We extracted the outputs of the last hidden layer of Vision Transformer model pretrained on the ImageNet-21K dataset ggoogle/viie base phaaee-pacch 6-224-in2 checkpoint in the cransformen rs package from huggi wngginggann ng and use them as input features, and we trained ensembles of 10 ReLU networks with hidden layers and batch normalization.
For all the experiments, we used SGD with Nesterov momentum as our optimizer, and we set the momemtum coefficient to 0.9. We used a batch size 256, and we annealed the learning rate from 0.2 to 0.002 during training. We used a weight decay of 5 10-4, We used 15 bins to calculate ECE for evaluation. For OOD experiments, we calculated epistemic uncertainty on the test sets of CIFAR- 10/100 and CINIC10. For evaluation natural corruptions, used all available types for corruptions and all levels of severity in CIFAR-10/ 10FAR-10/100-C.
D ADDITIONAL RESULTS
D.1 INPUT GRADIENT DIVERSITY AND FUNCTIONAL DIVERSITY
To show that FoRDE indeed produces ensembles with higher input gradient diversity among member models, which in turn leads to higher functional diversity than DE, we visualize the input gradient distance and epistemic uncertainty of FoRDE and DE in Fig. 7. To measure the differences between input gradients, we cosine distance, defined as cos(u, u, v) where cos(u, v) is the cosine similarity between two vectors u and v. To quantify functional diversity, we calculate the epistemic uncertainty using the formula in Depeweg al. (2018), similar to the transfer learning experiments. Fig. 7 shows that FoRDE has higher gradient distances among members compared to DE, while also having higher epistemic uncertainty across all levels of corruption severity. Intuitively, the test.
16

---

Published as a conference paper at ICLR 2024
inputs become more corrupted, epistemic uncertainty of both FoRDE and DE increases, and the input gradients between member models become more dissimilar for both methods. These results suggest that there could be connection between input gradient diversity and functional diversity in neural network ensembles.
ResNet18 / CIFAR-100
Gradient cosine distance
Epistemic uncertainty t
0.3
0.6
0.2
0.4
0.2
0.1
0.0
0.0
4
5
0
1
2
3
4
Severity
Severity
DE
FORDE-PCA
Figure 7: FoRDE has higher gradient distance as well as higher epistemic uncertainty Results of RESNETI8 8 CIFAR100 over seeds under different levels of corruption severity, where 0 mean corruption.
D.2 PERFORMANCE UNDER CORRUPTIONS
We plot performance of all methods under the RESNET 8/CIFAR-C setting in Figs. 8 and 9, These figures show that FoRDE achieves the best performance across all metrics under all corruption severities.
0.4
80
60
Accuracy(S8)
3
ECE
0.2
40
0.1
20
0.0
1
3
5
1
2
4
2
5
Corruption severity
Corruption severity
Corruption severity
DE
DE-EmpCov
feature-RDE
FORDE-PCA
Figure 8: FoRDE performs better than baselines across all metrics and under corruption severities. Results for RESNET 8/CIFAR- 100-c. Each ensemble has 10 members.
D.3 COMPARISON BETWEEN FORDE-PCA AND EMPCOV PRIOR
In Section 3.3, we discussed a possible connection between FORDE-PCA and the EmpCov prior (Iz- mailov et al., 2021a) Here, we further visualize performance of FoRDE-PCA, DE with EmpCov prior and vanilla DE on different types of corruptions and levels of severity for the RESNET 8/CIFAR 1 setting in Fig. 10. This figure also includes results of FoRDE-PCA with EmpCov prior to show that these two approaches can be combined together to further boost corruption robustness an ensemble. Overall, Fig. 10 shows that FoRDE-PCA and DE- DE-EmpCov have similar behaviors on the majority of the corruption types, meaning that DE- Oo-EmpCov is more or less robust than DE on a corruption type then does FORDE-PCA. The exceptions the blur corruption types ( (mmotoon, glass, zoom,
17

---

Published as a conference paper at ICLR 2024
100
0.4
Accuracy(%%
0.3
-2
ECE
0.2
40
0.1
2
4
5
2
5
2
3
5
Corruption severity
Corruption severity
Corruption severity
DE
DE-EmpCov
feature-RDE
FORDE-PCA
Figure 9: FoRDE performs better than baselines across all metrics and under all corruption severities. Results for RESNET 8/CIFAR- -0--.. Each ensemble has 10 members.
defocus, -blus)) where DE-EmpCov is less robust than vanilla DE while FORDE-PCA exhibits better robustness than DE. Finally, by combining FORDE-PCA and EmpCov prior together, we achieve the best robustness on average.
CIFAR-10:C
pixelate
comp_cppressoo
impulse_noise
Accuracy (9)
50
25
100
brightness
motion
Accurary (KG
75
50
100
gloss blur
zoom Nur
Snow
defocus blur
saturate
Accurary
fog
shot_noiss
gaussian blur
elastic_transfamm
Average
Accuraty (8)
50
25
Severity
Severity
Severity
Severity
Severity
DE EmpCov
FORDE PCA
FORDE FORDE-PCA EMMDCCC
Figure 10: FORDE-PCA and EmpCov prior behave similarly most of the corruption types Here we visualize accuracy for each of the 19 corruption types in CIFAR-10-C in the first 19 panels, while the last panel (bottom right) shows the average accuracy. Both FoRDE-PCA and DE-EmpCov more robust than plain DE most of the corruption types, with the exception of contrast where both FoRDE-PCA and Em-Emp are less robust than DE. On the other hand, on the blur corruption types (fmotion, glass, zoom, defocus, gaussian) -blur), DE-EmpCov is less robust than vanilla DE while FoRDE-PCA exhibits better robustness than DE.
D.4 TUNING THE LENCTHSCALES FOR THE RBF KERNEL
In this section, we show how to tune the lengthscales for the RBF kernel by taking the weighted average of the identity lengthscales and the PCA lengthscales introduced in Section 3.3. Particularly,
18

---

Published as a conference paper at ICLR 2024
using the notation of Section 3.3, define the diagonal lengthscale matrix Ex:
= + a
(25)
where A is diagonal matrix containing the eigenvalues from applying PCA on the training data as defined in Section 3.3. We then visualize the accuracy of FoRDE-PCA trained under different {0.0, 0.1, 0.2, 0.4, 0.8, 1.0] in Fig. 11 for the RESNET I 18/CIFAR- 100 setting and in Fig. 12 for the RESNET 8/CIFAR- 10 setting. Fig. 11 shows that indeed we can achieve good performance on both clean and corrupted data by choosing lengthscale setting somewhere between the identity lengthscales and the PCA lengthscales, which a = 0.4 this experiment. A similar phenomenon observed in Fig. 12, where 0.2 achieves the best results on both clean and corrupted data.
ResNet18 / CIFAR-100
Clean data
Severity level 1, 2, 3
Severity level 4,
Accuracy (%)
63.5
44.5
82.0
63.0
44.0
43.5
81.5
62.5
43.0
0.0
0.5
1.0
0.0
0.5 1.0
0.0
0.5
1.0
a
a
Figure 11: When moving from the identity lengthscales to the PCA lengthscales, FoRDE becomes more robust against natural corruptions, while exhibiting small performance degradation on clean data. Results are averaged over 3 seeds. Blue lines show performance of FoRDE, while orange dotted lines indicate the average accuracy of DE for comparison. At the identity lengthscales, FoRDE has higher accuracy than DE on in-distribution data but are slightly less robust against corruptions than DE. As we move from the identity lengthscales to the PCA lengthscales, FoRDE becomes more and more robust against corruptions, while showing small decrease in in-distribution performance. Here we can see that 0.4 achieves good balance between in-distribution accuracy and corruption robustness.
ResNet18 / CIFAR-10
Clean data
Severity level 1,
Severity level
Accuracy (%)
96.4
70
96.2
68
96.1
0.0
0.5
1.0
0.0
0.5
0.0
0.5
1.0
Figure 12: When moving from the identity lengthscales to the PCA lengthscales, FoRDE becomes more robust against natural corruptions, while exhibiting small performance degradation clean data. Results are averaged over 3 seeds. Blue lines show performance of FoRDE, while orange dotted lines indicate the average accuracy of DE for comparison. has higher accuracy than DE on in in-disribbttonn data but are slightly less robust against corruptions than DE. As we move from the identity lengthscales to the PCA lengthscales, FoRDE becomes more and more robust against corruptions, while showing small decrease in in-distribution performance. Here we can see that a = 0.2 achieves good balance between in-distribution accuracy and corruption robustness.
the identity lengthscales. FoRDE
19


Reviews:
[{"review_id": "oCEMJWyNqy", "reviewer": "Reviewer_VQHQ", "summary": "The paper proposes a novel method for ensembling deep models that ensures diversity of the ensemble members. The paper continues the line of work in particle-based variational inference transforming the repulsion step of this approach into an input gradient space. This is different from the existing works that have done this step in weight and function spaces.", "questions": "What exact corruption has been used in reported corruption experiments?", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 4, "strengths": "* A novel method for an important problem of ensembling\n* Thorough empirical evaluation and comparison to the existing methods\n* Drawing connections with the existing methods\n* The paper is mostly well written and easy to follow\n* Runtime analysis presented", "weaknesses": "* Some presentation unclearness (see details below)\n* Some transformations between theory in Section 3 and steps in Algorithm (in Appendix) are not obvious\n\n\n1. What corruption is considered? CIFAR-10/100-C datasets have several types of corruptions each of which has several level of severity of corruptions. No confidence intervals (+-) for corruption results. \n2. Section 3.1 doesn't address that the target distribution \\pi is not available, or am I missing something? \n3. It would help to clear some confusion of how Algorithm comes in place if steps in Algorithm would be linked to equations in Section 3. \n4. Section 3.4. \"However, in practice we found no performance degradation nor convergence issues in our experiments\" - though the convergence issues can easily be observed, in order to see no performance degradation one would need to compare the performance with and without mini-batches. This experiment is not presented in the paper (including Appendix). \n5. Though the code is provided, some implementation details in text are missing. For example, ECE computation details such as a number of bins. Or details of OOD experiments: what portion of OOD data (CIFAR-100 for CIFAR-100 and vice versa) was used. \n6. No reference for CINIC10 dataset", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "HVMMI5Uc1E", "reviewer": "Reviewer_eVm5", "summary": "This paper points out that while the repulsion in the existing weight-space or function-space repulsive deep ensembles has been theoretically well-motivated, it does not lead to a practical performance improvement compared to vanilla deep ensembles. Rather than relying on repulsion in weight or function space, the authors employ a kernel comparing input gradients of particles and propose First-order Repulsive Deep Ensembles (FoRDE). Experimental results clearly indicate that FoRDE outperforms baseline methods, particularly when dealing with corrupted data.", "questions": "1. The paper mentions the reasons for the ineffectiveness of weight-space repulsion: (1) \"Typically repulsion is done in the weight space to capture different regions in the weight posterior. However, due to the over-parameterization of neural networks, weight-space repulsion suffers from redundancy.\" (2) \"Weight-space repulsion is ineffective due to difficulties in comparing extremely high-dimensional weight vectors and the existence of weight symmetries (Fort et al., 2019; Entezari et al., 2022).\" Could you provide a more detailed explanation of this?\n\n2. The paper outlines the advantages of ensemble methods in four specific areas: (1) predictive performance, (2) uncertainty estimation, (3) robustness to adversarial attacks, and (4) corruptions. In the experimental results, it delves into (1) using ACC, (2) using NLL and ECE, and (4) using cA, cNLL, and cECE. Did you carry out any experiments regarding (3) by any chance? Considering that the current experimental results are somewhat lacking in (1) and (2), it might be worthwhile to focus more on (3) and (4).\n\n3. FoRDE-PCA exhibits robust performance in addressing common corruptions (although it shows a minor decrease in its in-distribution performance). Hence, I would like to suggest providing more detailed experimental results concerning common corruptions, e.g., if it operates similarly to EmpCov (Izmailov et al., 2021), it is worth exploring whether the most beneficial corruption type aligns as well.\n\n---\nFort et al., 2019, Deep ensembles: A loss landscape perspective.  \nEntezari et al., 2022, The role of permutation invariance in linear mode connectivity of neural networks.  \nIzmailov et al., 2021,  Dangers of Bayesian model averaging under covariate shift.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1. I have experienced that although repulsive deep ensembles are theoretically well-grounded, they do not result in performance enhancements in practice. In this regard, this paper is well-motivated, as it states, \"Neither weight nor function space repulsion has led to significant improvements over vanilla DEs.\"\n2. The paper provides a comprehensive overview of the literature concerning repulsive deep ensembles. Also, the proposed approach is meticulously detailed in a step-by-step manner, as well as its practical considerations.\n3. The connection to the EmpCov prior (Izmailov et al., 2021) further clarifies why the proposed FoRDE-PCA algorithm performs well for data under common corruptions.\n\n---\nIzmailov et al., 2021,  Dangers of Bayesian model averaging under covariate shift.", "weaknesses": "Despite the critique that neither weight nor function space repulsion yielded significant improvements compared to vanilla DEs, the FoRDE algorithm introduced in this context still did not result in a substantial performance enhancement over vanilla DEs. In particular, FoRDE-Identity demonstrates a performance similar to that of vanilla DE, while FoRDE-PCA excels in performance under corruption but significantly diminishes its in-distribution performance.\n\nThe authors seem to have recognized this aspect; \"Hence, we believe that the optimal lengthscales for good performance on both clean and corrupted data lie somewhere between unit lengthscales (the identity) and using the inverse square root eigenvalues as lengthscales.\" For this paper to be considered complete, it should not just acknowledge such ideal lengthscales but also offer experimental evidence of their practical identification.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "QfcL7C3tIC", "reviewer": "Reviewer_XVK8", "summary": "This paper is concerned with adapting particle based variational inference for improved training of neural network ensembles.  The authors attempt to circumvent problems that have affected previous attempts to use particle based variational inference for ensembles, with a lack of effective repulsion in weight space (intended to promote functional diversity) chief among them.  This paper proposes instead to enforce diversity in the input gradients rather than in weight space, by using Wasserstein gradient descent along with an RBF kernel defined over the input gradients to guide the particles during training.  They compare against deep ensembles and other BNNs on accuracy, calibration, and robustness to covariate shift.", "questions": "- Regarding the motivation of the RBF kernel in the \\textbf{Choosing the base kernel} paragraph of section 3.2, they are good arguments for using the RBF kernel, but are there others that were considered? As the authors suggest in section 3.3 and 3.4, choosing the length scales for RBF presents its own problem.  Could this be circumvented by employing a simpler base kernel? \n- Again in section 3.3, is the median heuristic required? I’m a little unsure at the outset why this is the solution chosen over any others that would reduce the effect of the dominant eigenvalues.", "limitations": "", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "- The paper is very well written.  The potential advantages of moving to input gradient based diversity management are well introduced & well motivated, and the explanations are largely self-contained, which is no small feat considering page restrictions for conferences.\n- In particular, the main contribution section (section 3) is *so* well written.  It takes time to lead the reader from the wider view of Wasserstein gradient descent, to input space gradients, and the more narrow questions of choice of kernels, and their tradeoffs.  Of all the papers I reviewed, this was by far the most enjoyable and informative to read.  Bravo for taking the time to write so clearly.", "weaknesses": "- One thing I often worry about is that the experiments are performed only in the vision domain, on over-hygenic datasets.  While I don't want to discount the amount of work needed to extend to other domains, projects like [WILDS](https://wilds.stanford.edu/) make this easier, and build confidence that demonstrated success isn't due to some quirk of CIFAR datasets.\n- One other complaint that to the authors' credit they highlight in section 3.5 is the cost of computing FoRDEs.  At a 3x computational premium to DEs, the penalty paid in compute seems to be the largest drawback of FoRDE with respect to DEs.  Do the authors have any ideas for reducing this burden? DEs themselves are expensive in both space and time to compute.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "b1eQn1voxm", "reviewer": "Reviewer_aXJD", "summary": "Most prior research employing particle-based variational inference (ParVI) has proven to be inefficient and has not significantly improved performance. To tackle these issues, this study presents a new ParVI approach known as the First-order Repulsive Deep Ensemble (FoRDE), which integrates repulsion principles into the realm of first-order input gradients.", "questions": "See the weakness section\n\nRecommend\n- It is recommended to include an ethics statement and a reproducibility statement right after the main paper.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "- The idea of incorporating repulsion into first-order input gradients(not a function space or a weight space repulsion which are quite common in Bayesian Neural Network literature) to enhance functional diversity is new to the community and intriguing.\n- The paper is well-written, ensuring it is easy to read and understand.", "weaknesses": "- The scale of experiments are quite small to show the effectiveness of FoRDE.\n- The overall performance gain compared to other baselines looks quite marginal for the out-of-distribution datasets, especially for the TinyImageNet which is the largest dataset. And shows lower performance compared to the other baselines for the in-distribution datasets.\n- Having empirical or theoretical evidence to demonstrate the effectiveness of FoRDE in enhancing input gradient diversity would be beneficial.\n- Providing empirical results that illustrate how the improved input gradient diversity effectively changes into enhanced functional diversity in deep neural network scenarios would be valuable. \n- Additional hyperparameters for the kernel would be another burden for this method.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (spotlight)