Disentangling the Predictive Variance of Deep Ensembles through the Neural Tangent Kernel
Abstract
Identifying unfamiliar inputs, also known as out-of-distribution (OOD) detection, is a crucial property of any decision making process.
A simple and empirically validated technique is based on deep ensembles where the variance of predictions over different neural networks acts as a substitute for input uncertainty.
Nevertheless, a theoretical understanding of the inductive biases leading to the performance of deep ensemble’s uncertainty estimation is missing.
To improve our description of their behavior, we study deep ensembles with large layer widths operating in simplified linear training regimes, in which the functions trained with gradient descent can be described by the neural tangent kernel.
We identify two sources of noise, each inducing a distinct inductive bias in the predictive variance at initialization.
We further show theoretically and empirically that both noise sources affect the predictive variance of non-linear deep ensembles in toy models and realistic settings after training.
Finally, we propose practical ways to eliminate part of these noise sources leading to significant changes and improved OOD detection in trained deep ensembles.
Introduction
Modern artificial intelligence uses intricate deep neural networks to process data, make predictions and take actions.
One of the crucial steps toward allowing these agents to act in the real world is to incorporate a reliable mechanism for estimating uncertainty – in particular when human lives are at risk [1, 2].
Although the ongoing success of deep learning is remarkable, the increasing data, model and training algorithm complexity make a thorough understanding of their inner workings increasingly difficult.
This applies when trying to understand when and why a system is certain or uncertain about a given output and is therefore the topic of numerous publications [3–10].
Principled mechanisms for uncertainty quantification would rely on Bayesian inference with an appropriate prior.
This has led to the development of (approximate) Bayesian inference methods for deep neural networks [11–15].
Simply aggregating an ensemble of models [16] and using the disagreement of their predictions as a substitute for uncertainty has gained popularity.
However, the theoretical justification of deep ensembles remains a matter of debate, see Wilson and Izmailov [17].
Although a link between Bayesian inference and deep ensembles can be obtained, see [18, 19], an understanding of the widely adopted standard deep ensemble and it’s predictive distribution is still missing [20, 21].
Note that even for principled Bayesian approaches there is no valid theoretical or practical OOD guarantee without a proper definition of out-of-distribution data [22].
One avenue to simplify the analyses of deep neural networks that gained a lot of attention in recent years is to increase the layer width to infinity [23, 24] or to very large values [25, 26].
In the former regime, an intriguing equivalence of infinitely wide deep networks at initialization and Gaussian processes allows for exact Bayesian inference and therefore principled uncertainty estimation.
Although it is not possible to generally derive a Bayesian posterior for trained infinite or finite layer width networks, the resulting model predictions can be expressed analytically by kernels.
Given this favorable mathematical description, the question of how powerful and similar these models are compared to their arguably black-box counterparts arises, with e.g. moderate width, complex optimizers and training stochasticity [25, 27–32].
In this paper, we leverage this tractable description of trained neural networks and take a first step towards understanding the predictive distribution of neural networks ensembles with large but finite width.
Building on top of the various studies mentioned, we do so by studying the case where these networks can be described by a kernel and study the effect of two distinct noise sources stemming from the network initialization: The noise in the functional initialization of the network and the initialization noise of the gradient, which affects the training and therefore the kernel.
As we will show, these noise sources will affect the predictive distributions differently and influence the network’s generalization on in- and out-of-distribution data.
Our contributions are the following:
• We provide a first order approximation of the predictive variance of an ensemble of linearly trained, finite-width neural networks.
We identify interpretable terms in the refined variance description, originating from 2 distinct noise sources, and further provide their analytical expression for single layer neural networks with ReLU non-linearities.
• We show theoretically that under mild assumptions these refined variance terms survive nonlinear training for sufficiently large width, and therefore contribute to the predictive variance of non-linearly trained deep ensembles.
Crucially, our result suggests that any finer description of the predictive variance of a linearized ensemble can be erased by nonlinear training.
• We conduct empirical studies validating our theoretical results, and investigate how the different variance terms influence generalization on in - and out-of-distribution.
We highlight the practical implications of our theory by proposing simple methods to isolate noise
sources in realistic settings which can lead to improved OOD detection.
Neural network ensembles and their relations to kernels
Let f θ = f ( · , θ ) : R h
→ R h L denote a neural network parameterized by the weights θ ∈ R n .
The weights consist of weight matrices and bias vectors { ( W l , b l ) } Ll =1 describing the following feed-forward computation beginning with the input data x 0 :
[Equation]
Here h l is the dimension of the vector x l and ϕ is a pointwise non-linearity such as the softplus log(1 + e x ) or Rectified Linear Unit i.e. max(0 , x ) (ReLU) [33].
We follow Jacot et al.
[24] and use σ w = √ 2 to control the standard deviation of the initialised weights W lij , b li ∼ N (0 , 1) .
Given a set of N datapoints X
= ( x i )
≤ i ≤ N ∈ R N × h 0 and targets Y =
( y i )
≤ i ≤ N ∈ R
N
×
h
L , we consider regression problems with the goal of finding θ ∗ which minimizes the mean squared error (MSE) loss
L ( θ ) = 12 (cid:80)
Ni =
∥ f
( x i , θ ) − y i ∥ 22 .
For ease of notation, we denote by f ( X , θ ) ∈
R N · h L the vectorized evaluation of f on each datapoint and
Y ∈ R N · h L the target vector for the entire dataset.
As the widths of the hidden layers grow towards infinity, the distribution of outputs at initialization f ( x, θ 0 ) converges to a multivariate gaussian distribution due to the Central Limit
Theorem [23].
The resulting function can then accurately be described as a zero-mean Gaussian process, coined Neural Network Gaussian Process (NNGP), where the covariance of a pair of output neurons i, j for data x and x ′ is given by the kernel
[Equation]
with h = min( h 1 , ..., h L − 1 ) .
This equivalence can be used to analytically compute the Bayesian posterior of infinitely wide Bayesian neural networks [34].
On the other hand infinite width models trained via gradient descent (GD) can be described by the
Neural Tangent Kernel (NTK).
Given θ , the NTK Θ θ of f θ is a matrix in R N · h L × R N · h L with the ( i, j ) -entry given as the following dot product
[Equation]
where we consider without loss of generality the output dimension of f to be h L = 1 for ease of notation.
Furthermore, we denote Θ θ ( X , X ) :
= ∇ θ f
( X , θ ) ∇ θ f
( X , θ ) T the matrix and Θ θ ( x ′ , X ) :
= ∇ θ f ( x ′ , θ ) ∇ θ f ( X , θ ) T the vector form of the NTK while highlighting the dependencies on different datapoints.
Lee et al.
[25] showed that for sufficiently wide networks under common parametrizations, the gradient descent dynamics of the model with a sufficiently small learning rate behaves closely to its linearly trained counterpart, i.e. its first-order Taylor expansion in parameter space.
In this gradient flow regime, after training on the mean squared error converges, we can rewrite the predictions of the linearly trained models in the following closed-form:
[Equation]
where Q θ 0 ( x, X ) :
=
Θ θ 0 ( x, X )Θ θ
( X , X ) − 1 with Θ θ
the NTK at initialization, i.e. of f ( ., θ 0 ) .
The linearization error throughout training sup t ≥
∥ f lin t ( x ) − f t ( x ) ∥ is further shown to decrease with the width of the network, bounded by O ( h − 12 ) .
Note that one can also linearize the dynamics without increasing the width of a neural network but by simply changing its output scaling [26].
When moving from finite to the infinite width limit the training of a multilayer perceptron (MLP) can again be described with the NTK, which now converges to a deterministic kernel Θ ∞ [24], a result which extends to convolutional neural networks [27] and other common architectures [35, 36].
A fully trained neural network model can then be expressed as
[Equation]
where f ( {X , x } , θ 0 ) ∼ N (0 , K ( {X , x } , {X , x } )) .
2.1 Predictive distribution of linearly trained deep ensembles
In this Section, we study in detail the predictive distribution of ensembles of linearly trained models, i.e. the distribution of f lin ( x ) given x over random initializations θ 0 .
In particular, for a given data x , we are interested in the mean E [ f ( x )] and variance V [ f ( x )] of trained models over random initialization.
The former is typically used for the prediction of a deep ensemble, while the latter is used for estimating model or epistemic uncertainty in order to do e.g. OOD detection or exploration.
To start, we describe the simpler case of the infinite width limit and a deterministic NTK, which allows us to compute the mean and variance of the solutions found by training easily:
[Equation]
where we introduced Q ∞ ( x, X )
=
Θ ∞ ( x, X )Θ ∞ ( X , X ) − 1 .
For finite width linearly trained networks, the kernel is no longer deterministic, and its stochasticity influences the predictive distribution.
Because there is probability mass assigned to the neighborhood of rare events where the NTK kernel matrix is not invertible, the expectation and variance over parameter initialization of the expression in equation 4 diverges to infinity.
Fortunately, due to the convergence in probability of the empirical NTK to the infinite width counterpart [24], we know these singularities become rarer and ultimately vanish as the width increases to infinity.
Intuitively, we should therefore be able to assign meaningful, finite values to these undefined quantities, which ignores these rare singularities.
The delta method [37] in statistics formalizes this intuition, by using Taylor approximation to smooth out the singularities before computing the mean or variance.
When the probability mass of the empirical NTK is highly concentrated in a small radius around the limiting NTK, the expression 4 is roughly linear w.r.t the NTK entries.
Given this observation, we prove (see Appendix A.2) the following result, and justify that the obtained expression is informative of the empirical predictive mean and variance of deep ensembles.
Rewriting equation 4 into
[Equation]
where ¯ Q ( x, X ) = ¯Θ( x, X )¯Θ( X , X ) − 1 and ¯Θ =
E (Θ θ 0 ) , we state:
Proposition 2.1.
For one hidden layer networks parametrized as in equation 1, given an input x and training data ( X , Y ) , when increasing the hidden layer width h , we have the following convergence in distribution over random initialization θ 0 :
[Equation]
where Z(x) is the linear combination of 2 Chi-Square distributions, such that
[Equation]
where
[Equation]
We omit the dependence of θ 0 on the width h for notational simplicity.
While the expectation or variance of equation 4 for any finite width is undefined, their empirical mean and variance are with high probability indistinguishable from that of the above limiting distribution (see Lemma A.1).
Note that the above proposition assumes the noise in Θ θ 0 to be decorrelated from f ( x, θ 0 ) , which can hold true under specific constructions of the network that are of practical interest as we will see in the following (c.f. Appendix A.3.2).
Given Proposition 2.1, we now describe the approximate variance of f lin ( x ) for L = 2 , which we can extend to the general L > 2 case using an informal argument (see A.2.2):
Proposition 2.2.
Let f be a neural network with identical width of all hidden layers, h 1 =
h 2 = ...
=
h L − 1 = h .
We assume ∥
Θ θ 0 − ¯Θ ∥ 2 F = O p ( 1 h ) .
Then,
[Equation]
where
[Equation]
[-[¯Θ( X , X ) − 1 Θ θ 
-]
and V res ( x ) =
O ( h − 2 ) as well as ¯ K the expectation over initializations of the finite width counterpart of the NNGP kernel.
Several observations can be made: First, the above expression only involves the first and second moments of the empirical, finite width NTK, as well as the first moment of the NNGP kernel.
These terms can be analytically computed in some settings.
We provide in Appendix A.4.3 some of the moments for the special case of a 1-hidden layer ReLU network, and show the analytical expression correspond to empirical findings.
Second, the decomposition demonstrates the interplay of 2 distinct noise sources in the predictive variance:
[Figure]
• V a is the variance associated to the expression in the first line of equation 7.
Intuitively, it is the finite width counterpart of the predictive variance of the infinite width model (equation 6), as it assumes the NTK is deterministic. The variance stems entirely from the functional noise at initialization and converges to the infinite width predictive variance as the width increases.
• V c and V i stem from the second line of equation 7.
V c is a first-order approximation of the predictive variance of a linearly trained network with pure kernel noise , without functional noise i.e. V c ≈ V
[ Q θ 0 ( x, X ) Y ] .
On the other hand, V i depends on the interplay between the 2 noises, and can be identified as the predictive variance of a deep ensemble with a deterministic NTK ¯Θ and a new functional prior g ( x ) =
Θ θ 0 ( x, X )¯Θ( X , X ) − 1 f
( X , θ 0 ) .
Intuitively, this new functional prior can be seen as a data-specific inductive bias on the NTK formulation of the predictive variance (see Appendix A.3.1 for more details). • V cor is a covariance term between the 2 terms in equation 7 and also contains the correlation terms between Θ θ {+
+} and f ( x, θ 0 ) .
In general, its analytical expression is challenging to obtain as it requires the 4th moments of the finite width NNGP kernel fluctuation.
Here, we provide its expression under the same simplifying assumption that the noise in Θ θis decorrelated from f ( x, θ 0 ) .
We therefore do not attempt to describe it in general, and focus in our empirical Section on the terms that are tractable and can be easily isolated for practical purposes.
Each of V c , V i , V cor decay in O ( h − 1 ) , which, together with V a , provide a first-order approximation of the predictive variance of f lin ( x ) .
Note that V a and V c are of particular interest, as removing either the kernel or the functional noise at initialization will collapse the predictive variance of the trained ensemble to either one of these 2 terms.
2.2 Predictive distribution of standard deep ensemble of large width
An important question at this point is to which extent our analysis for linearly trained models applies to a fully non-linearly trained deep ensemble.
Indeed, if the linear approximation error is at least of the same order of magnitude as the higher-order correction in the variance term, the latter can be ’erased’ by training.
Building on top of previous work, we show that, under the assumption of an empirically supported conjecture [38], for one hidden layer networks trained on the Mean Squared Error (MSE) loss, the perturbation in the prediction from non-linear training is asymptotically dominated by the refined predictive variance terms of the linearly trained ensemble we described in Section 2.1.
Proposition 2.3.
Let f be a neural network with identical width of all hidden layers, h 1 =
h 2 = ...
=
h L − 1 = h , and such that ϕ ′ is bounded and Lipschitz continuous on R .
Let the training data ( X , Y ) contained in some compact set, such that the NTK of f on X is invertible. Let f t be the model trained on the MSE loss with gradient flow at timestep t with some learning rate. Assuming [Equation]
Then, ∀ x, ∀ δ > 0 , ∃ C, H : ∀ h > H ,
[Equation]
1024 h .
V [ f ( x ) ]
V V a
V c
V i 1 /h   256 h .
V [ f ( x ) ]
V V a
V c
V i 1 /h    1024 h 0.
.005 V [ f ( x ) ]
V res 1 /h   256 h 0.
.005 V [ f ( x ) ]
V res 1 /h
In particular for one hidden layer networks, after training,
[Equation]
where ˆ V denotes the sample variance with some fixed sample size.
The proof can be found in Appendix A.1.1.
While only the bound sup t ∥
Θ θ 0 − Θ θ t ∥ F = O ( 1 √ h ) has been proven in previous works [25], many empirical studies including those in the present work (see
Appendix Fig.
5, Table 3) have shown that the bound decreases faster in practice, on the order of ( h − 1 ) [25, 38].
Note that this result suggests the approximation provided in Proposition 2.2 is as good as it gets for describing the predictive variance of non-linearly trained ensembles: the higher order terms would be of a smaller order of magnitude as the non-linear correction to the training, rendering any finer approximation pointless.
Disentangling deep ensemble variance in practice
The goal of this Section is to validate our theoretical findings in experiments.
First, we aim to show qualitatively and quantitatively that the variance of linearly trained neural networks is well approximated by the decomposition introduced in Proposition 2.2.
To do so, we investigate ensembles of linearly trained models and analyze their behavior in toy models and on common computer vision classification datasets.
We then extend our analyses to fully-trained non-linear deep neural networks optimized with (stochastic) gradient descent in parameter space.
Here, we confirm empirically the strong influence of the variance description of linearly trained models in these less restrictive settings while being trained to very low training loss.
Therefore we showcase the improved understanding of deep ensembles through their linearly trained counterpart and highlight the practical relevance of our study by observing significant OOD detection performance differences of models when removing noise sources in various settings.
3.1 Disentangling noise sources in kernel models
To isolate the different terms in Proposition 2.2, we construct, from a given linearly trained network f lin , three additional linearly trained models:
[Equation]
where g ( x, θ 0 )
=
Θ θ 0 ( x, X )¯Θ( X , X ) − 1 f
( X , θ 0 ) .
Note that the predictive variance over random initialization of these functions corresponds to respectively V c , V a , V i as defined in Section 2.1.
As one can see, we can simply remove the initialization noise from f lin by subtracting the initial (noisy) function f ( x, θ 0 ) before training resulting in a centered model f lin-c .
Equivalently, we can remove noise that originates from the kernel by using the empirical average over kernels resulting in model f lin-a .
Finally, we can isolate f lin-i by the same averaging trick as in f lin-a but use as functional noise g ( x, θ 0 ) which can be precomputed and added to f lin-c before training.
Note that we neglect the terms involving covariance terms and focus on the parts which are easy to isolate, for linearly trained as well as for standard models.
This will later allow us to study practical ways to subtract important parts of the predictive distribution for neural networks leading for example to significant
OOD detection performance differences.
Now we explore the differences and similarities of these disentangled functions and their respective predictive distributions.
3.1.1 Visualizations on a star-shaped toy dataset
To qualitatively visualize the different terms, we construct a two-way star-shaped regression problem on a 2d-plane depicted in Figure 1.
After training an ensemble we visualize its predictive variance on the input space.
Our first goal is to visualize qualitative differences in the predictive variance of ensembles consisting of f lin and the 3 disentangled models from above.
We train a large ensemble of size 300 where each model is a one-layer ReLU neural network with hidden dimensionand 1 hidden layer.
As suggested analytically for one hidden layer ReLU networks (see Appendix A.4.3), for example V [ f lin-c ( x )] depends on the angle of the datapoints while V [ f lin ( x )] depicts a superposition of the 3 isolated variances.
We stress that the interaction of the variance terms in shaping V [ f lin ( x )] is dependent on the data and architecture and can drastically vary across setups.
3.1.2 Disentangling linearly trained / kernel ensembles for MNIST and CIFAR
Next, we move to a quantitative analysis of the asymptotic behavior of the various variance terms, as we increase the hidden layer size.
In Figure 2, we analyze the predictive variance of the kernel models based on MLPs and Convolutional Neural Networks (CNN) for various depths and widths and on subsets of MNIST [39] and CIFAR10.
As before, we construct a binary classification task through a
MSE loss with dataset size of N = 100 and confirm, shown in Figure 2, that V c , V i decay by 1 /h over all of our experiments.
Crucially, we see that they contribute to the overall variance V even for relatively large widths.
We further observe a decay in 1 /h 2 of the residual term as predicted by
Proposition 2.2.
Again, the variance magnitude and therefore the influence on f lin of the disentangled parts is highly architecture and dataset-dependent.
Note that the small size of the datasets comes from the necessity to compute the inverse of the kernels for every ensemble member, see Appendix B for a additional analysis on larger datasets and scaling plots of V cor .
In Table 1, we quantify the previously observed qualitative difference of the various predictive variances by evaluating their performance on out-of-distribution detection tasks, where high predictive variance is used as a proxy for detecting out-of-distribution data.
We focus our attention on analysing
V [ f lin-c ( x )] and V [ f lin-a ( x )] , as they are the variance terms containing purely the functional and kernel noise, respectively.
As an evaluation metric, we follow numerous studies and compute the area under the receiver operating characteristics curve (AUROC, c.f. Appendix B).
We fit a linearized ensemble on a larger subset of the standard 10-way classification MNIST and CIFAR10 datasets using MSE loss.
When training our ensembles on MNIST, we test and average the OOD detection performance on FashionMNIST (FM) [40], E-MNIST (EM) [41] and K-MNIST (KM) [42].
When training our ensembles on CIFAR10, we compute the AUROC for SVHN [43], LSUN [44], TinyImageNet (TIN) and CIFAR100 (C100), see Appendix Table 4 for AUROC values for all datasets. [Table]
The results show significant differences in variance magnitude and AUROC values.
While we do not claim competitive OOD performance, we aim to highlight the differences in behavior of the isolated functions developed above: we see for instance that for (MLP, MNIST, N=1000), f lin-a generally performs better than f lin in OOD detection.
Indeed, the overall worse performance of V [ f lin-c ( x )] seems to be affecting that of V [ f lin ( x )] which contains both terms.
On the other hand, we see that for the setup (CNN, CIFAR10, N=1000) V [ f lin ( x )] is not well described by this interpolation argument, which highlights the influence of the other variance terms described in Proposition 2.2.
Furthermore, the OOD detection capabilities of each function seem to be highly dependent on the particular data considered: Ensembles of f lin-c are relatively good at identifying SVHN data as OOD, while being poor at identifying LSUN and iSUN data.
These observations highlight the particular inductive bias of each variance term for OOD detection on different datasets.
We further report the test set generalization of the ensemble mean of different functions, highlighting the diversity in the predictive mean of these models as well.
Note that for N > = 1000 we trained the ensembles in linear fashion with gradient flow (which coincides with the kernel expression) up until the MSE training error was smaller than 0 .
3.2 Does the refined variance description generalize to standard gradient descent in practice?
In this Section, we start with empirical verification of Proposition 2.3 and show that the bound in equation 10 holds in practice.
Given this verification, we then propose equivalent disentangled models as those previously defined but in the non-linear setting, and 1) show significant differences in their predictive distribution but also 2) investigate to which extent improvements in OOD detection translate from kernel / linearly to fully non-linearly trained models.
We stress that we do not consider early stopped models and aim to connect the kernel with the gradient descent models faithfully.
3.2.1 Survival of the kernel noise after training
To validate Proposition 2.3, we first introduce f gd ( x ) =
f ( x,θ t ) , a model trained with standard gradient descent of t steps i.e. θ t = θ 0 − (cid:80) t − 1 i =
η ∇ θ f
( X , θ i )
( Y − f
( X , θ i )) .
To empirically verify
Proposition 2.3, we introduce the following ratio
[Equation]
where the empirical variances are computed over random initialization, and the expectation over some data distribution which we choose to be the union of the test-set and the various OOD datasets.
Given a datapoint x , the term inside the log measures the ratio between the discrepancy of the variance between the linearized and non-linear ensemble, against the refined variance terms.
R ( f ) is then the geometric mean of this ratio over the whole dataset.
Proposition 2.3 predicts that the ratio remains bounded as the width increases, suggesting that the refined terms contribute to the final predictive variance of the non-linear model in a non negligible manner.
We empirically verify this prediction for various depths in Fig.
3 and Appendix Figure 6, for functions trained on subsets
MNIST and CIFAR10.
Note that for all our experiments we also empirically verify the assumption from Proposition 2.3 (see Appendix Figure 5, Table 3).
3.2.2 Disentangling noise sources in gradient descent non-linear models
h .
R ( f ) L =
L = L =
Motivated by the empirical verification of Proposition 2.3, we now aim to isolate different noise sources in non-linear models trained with gradient descent.
Starting from a non-linear network f gd , we follow the same strategy as before and silence the functional initialization noise by centering the network (referred as f gd-c ( x ) ) by simply subtracting the function at initialization.
On the other hand, we remove the kernel noise with a simple trick: We first sample a random weight θ c 0 once, and use it as the weight initialization for all ensemble members.
A function noise is added by first removing the function initialization from θ c 0 , and adding that of a second random network which is not trained.
The resulting functions (referred as f gd-a ( x ) ) will induce and ensemble which will only differ in their functional initialization while having the same Jacobian.
[Equation]
We furthermore introduce f gd-i ( x ) , the non linear counterpart to f lin-i ( x ) , which we construct similarly to f gd-a ( x ) but using g ( x, θ 0 , θ c 0 ) =
Θ θ 0 ( x, X )Θ θ c
( X , X ) − 1 f
( X , θ 0 ) as the function initialization instead of f ( x, θ 0 )
(see Section 2.1 and Appendix A.3.1 for the justification).
Unlike f gd-a and f gd-c , constructing f gd-i requires the inversion of large matrices due to the way g is defined, a challenging task for realistic settings.
While its practical use is thus limited, we introduce it to illustrate the correspondence of correspondence of the predictive variance of linearized vs non-linear deep ensemble.
Given these simple modifications of f gd , we rerun the experiments conducted for the linearly trained models for moderate dataset sizes (N=1000).
We observe close similarities in the OOD detection capabilities as well as predictive variance between the introduced non-linearly trained ensembles and their linearly trained counterparts.
We further train these models on the full MNIST dataset (N=50000) for which we show the same trend in Appendix Table 5.
We also include the ensemble’ performance when trained on the full CIFAR10 dataset.
Intriguingly, the relative performance of the ensemble is somewhat preserved in both settings between N=1000 and N=50000, even when training with SGD, promoting the use of quick, linear training on subset of data as a proxy for the
OOD performance of a fully trained deep ensemble.
Similar to the case of (MLP, MNIST, N=1000/50000), we observe that f gd ensemble performance is an interpolation of f gd-c and f gd-a which interestingly performs often favorably, on different OOD data.
To understand if the noise introduced by SGD impacts the predictive distribution of our disentangled ensembles, we compared the behavior of f gd and f sgd in the lower data regime of N = 1000 .
Intriguingly, we show in Appendix Table 6 that no significant empirical difference between GD and SGD models can be observed and hypothesize that noise sources discussed in this study are more important in our approximately linear training regimes.
To speed up experiments we used (S)GD with momentum (0.9) in all experiments of this subsection.
3.2.3 Removing noise of models possibly far away from the linear regime
Finally, we investigate the OOD performance of the previously introduced model variants f sgd , f sgd-c and f sgd-d in more realistic settings.
To do so we train the commonly used WideResNet 28-10 [45] on
CIFAR10 with BatchNorm [46] Layers and cross-entropy (CE) loss for 20 epochs with batchsize of 128, without data augmentation (see Table 3.2.3).
These network and training algorithm choices are considered crucial to achieving state-of-the-art and superior performance compared to their linearly trained counterparts.
Strikingly, we notice that our model variants, which each isolate a different initial noise source, significantly affect the OOD capabilities of the final models when the training loss is virtually 0 - as in all of our experiments.
This indicates that the discussed noise sources influence the ensemble’s predictive variance long throughout training.
We provide similar results for
CIFAR100 and FashionMNIST in Table B.1 and B.1 of the Appendix B. We stress that we do not claim that our theoretical assumptions hold in this setup.
[Table]
Conclusion
The generalization on in-and out-of-distribution data of deep neural network ensembles is poorly understood.
This is particularly worrying since deep ensembles are widely used in practice when trying to asses if data is out-of-distribution.
In this study, we try to provide insights into the sources of noise stemming from initialization that influence the predictive distribution of trained deep ensembles.
By focusing on large-width models we are able to characterize two distinct sources of noise and describe an analytical first-order approximation of the predictive variance in some restricted settings.
We then show theoretically and empirically how parts of this refined predictive variance description in the linear training regime survive and impact the predictive distribution of non-linearly trained deep ensembles.
This allows us to extrapolate insights of the tractable linearly trained deep ensembles into the non-linear regime which can lead to improved out-of-distribution detection of deep ensembles by eliminating potentially unfavorable noise sources.
Although our theoretical analysis relies on the closeness to linear gradient descent which has shown to result in less powerful models in practice, we hope that our surprising empirical success of noise disentanglement sparks further research into using the lens of linear gradient descent to understand the mysteries of deep learning.
Acknowledgments and Disclosure of Funding
Seijin Kobayashi was supported by the Swiss National Science Foundation (SNF) grant CRSII5_173721.
Pau Vilimelis Aceituno was supported by the ETH Postdoctoral Fellowship program (007113).
Johannes von Oswald was funded by the Swiss Data Science Center (J.v.O. P18-03).
We thank Christian Henning, Frederik Benzing and Yassir Akram for helpful comments and discussions.
Seijin Kobayashi and Johannes von Oswald are grateful for Angelika Steger’s and João Sacramento’s overall support and guidance.
A Appendix A.1 Proof of Proposition 2.
Lemma A.1.
Let ( X n ) n ∈
N be a sequence of random variables with infinite mean and variance.
Let us assume that there exists a random variable X with finite mean and variance such that X n d → X , where d denotes convergence in distribution.
Given the number of samples K , let the empirical mean and variance random variables (ˆ µ n ) n ∈ N , (ˆ σ n ) n ∈ N defined as
[Equation]
where the X n,k are i.i.d samples.
Then we have
[Equation]
as well as
[Equation]
where ˆ µ, ˆ σ are resp.
the empirical mean and variance of the limiting distribution X .
The proof is a straightforward application of the continuous mapping theorem.
Lemma A.2.
Let ( X n ) n ∈ N , ( Y n ) n ∈ N be a sequence of random variables such that | X n | =
O p (1) and | Y n | =
O p (1) .
Let us assume that there exists a real valued sequence ( a n ) n ∈ N such that | X n − Y n | =
O p ( a n ) .
Given K , let the empirical variance random variables (ˆ σ Xn ) n ∈ N defined as
[Equation]
We define similarly (ˆ σ Yn ) n ∈ N .
Then,
[Equation]
Proof.
[Equation]
n
[Equation]
k )
[Equation]
[Equation]
| =
[Equation]
|
[Equation] [Equation]
|
[Equation]
[Equation]
|
[Equation]
k
[Equation] [Equation] [Equation] ) =
O p
[Equation]
K (cid:88)
k
[Equation]
| X n
[Equation]
k − Y n
[Equation]
k | ) =
O p
[Equation]
a n )
A.1.1 Discrepancy between non-linear and linearly trained neural network during training
We adapt Theorem H.1.
from [25] to show that the discrepancy between the original and the linearly trained network for the MSE loss that we consider is bounded as sup t ∥ f lint ( x ) − f ( x, θ t ) ∥ 22 ≤
O ( h − 2 ) as well.
The proof is an adaptation of the one in [25] with very minor differences.
We piggyback on the main result in their proof which was obtained with Grönwall’s inequality, which requires the continuity of the derivative of the activation function.
Let a neural network f such the width of all hidden layers are identical, h 1 =
h 2 = ...
=
h L − 1 = h , and such that ϕ ′ is bounded and Lipschitz continuous on R .
Let the training data ( X , Y ) contained in some compact set, such that the NTK of f on X is invertible.
Let f t the model trained on the MSE loss with gradient flow at timestep t with some learning rate.
Assumption A.3.
∀ δ > 0 , ∃ C, N : ∀ h > H , with probability at least 1 − δ over random initialization,
[Equation]
Proposition A.4.
Under assumption A.3, when trained with gradient flow on the MSE loss, we have ∀ x, ∀ δ > 0 , ∃ C, H : ∀ h > H ,
[Equation]
Proof.
Let g lin ( t ) =
f lint ( X ) − Y and g ( t ) =
f t ( X ) − Y .
Starting from equation (S118) from [25], we have
[Equation]
(15) where σ t = sup 0 ≤ s ≤ t ∥
Θ θ t − Θ θ 0 ∥ op , η 0 the learning rate and λ 0 is the smallest eigenvalue of Θ θ 0 .
Because the functions are trained on MSE loss L , we have
[Equation]
Integrating and taking the L2 norm,
[Equation]
[Equation]
where we used ∥ g (0) ∥
=
∥
g lin ( t ′ )
∥ ≤ ∥ g lin (0) ∥
e − λ 0 η 0 t ′ the triangular inequality and equation 15.
Because g (0) converges in distribution to a mean zero gaussian distribution, and because Θ θconverges in probability to Θ ∞ , we can find H such that ∀ h > H , with probability at least 1 − δ ′ ,
[Equation] [Equation]
and
[Equation]
where C > 0 is a constant.
Because the NTK at initialization converges in probaility to Θ ∞ assumed to be invertible, there exists
H ′ such that
[Equation]
>
[Equation]
Where λ min is the smallest eigenvalue of Θ ∞ .
Thus ∥ Θ θ 0 − Θ ∞ ∥ op ≤ λ min 2 , and so λ 0 > λ min
From assumption A.3, let us fix H ′′ , C ′ such that ∀ h > H ′′ , with probability at least 1 − δ ′
[Equation]
[Equation]
and
[Equation]
And therefore ∀ h > max( H ′ , H ′′ , 2 C ′ λmin ) , with probability at least 1 − δ ′ , σ t < λ 0 , and therefore (cid:82) t 0 t ′
e − λ 0 η 0 t ′ + σ t ′ η 0 t ′ is bounded by some C ′′ .
Putting everything together, ∀ n > max( H, H ′ , H ′′ , 2 C ′ λmin ) , with probability at least 1 − 3 δ ′ ,
[Equation]
30) (31) with K some constant.
By taking δ ′ =
δ 3 we get the result that ∥ f lint ( x ) − f t ( x )) ∥
=
O p ( 1 h )
Finally, using Lemma A.2 and the fact that f lin ( x ) and f ( x ) are bounded with high probability since they both converge in distribution to a gaussian with finite variance, we have, at the end of training,
[Equation]
for some finite sample empirical variance.
It remains to show that ∀ x, ∀ δ, ∃ C > 0 , H > 0 : ∀ h > H ,
[Equation]
i.e.
[Equation]
Following Proposition 2.1 and Lemma A.1, we have
[Equation]
where Z is a linear combination of 2 chi-square distribution with finite and no-zero moments, which proves the result.
A.2 Delta method A.2.1 Proof of Proposition 2.
We start with the special case of a single hidden layer neural network.
We provide the following
Lemma, which is a slight variation of the Delta method.
Lemma A.5.
Let X h ∈ R n , Y h ∈ R n be two sequences of multivariate independent random variables that satisfy X h dist.
→ N ( µ, Σ 1 ) and √ h ( Y h − ¯ Y ) dist.
→ N (0 , Σ 2 ) in distribution for some constant ¯ Y .
Let a function g : R n → R n with continuous partial derivative.
Then,
[Equation]
34) such that Z is a linear combination of 2 Chi-square distributions, and
[Equation]
[Equation]
[Equation]
Proof.
By applying the multivariate delta method, we have
[Equation]
Given the independence assumption of X h and Y h , we have the independence of X h and √ h (cid:2) g
( Y h )
T − g ( ¯ Y ) T (cid:3) , and therefore ( X h , √ h (cid:2) g ( Y h )
T − g ( ¯ Y ) T (cid:3) ) converge in distribution to the
Cartesian product of their respective limiting random variables.
Using the continuity of the dotproduct operation, and applying again the continuous mapping theorem, we have
[Equation]
(38) where G 1 , G 2 are normally distributed multivariate random variables with (mean, covariance) resp.
(0 , ∇ T g ( ¯ Y )Σ 2 ∇ g ( ¯ Y )) and ( µ, Σ 1 ) .
Note that if the X h are constant, or converge to a constant value, the limiting distribution
Z is a
Gaussian distribution.
In general however, given the independence of G 1 and G 2 , Z as the product of 2 independent Gaussians is a linear combination of two Chi-square distributions.
Finally, we have
[Equation] [Equation]
which concludes the lemma.
Let us now prove Proposition 2.1.
For one hidden layer networks, given a width h , it is straightforward to see (see A.4) that the empirical NTK Θ h (whereby the weight initialization is a random variable) is the sum of h i.i.d.
random variables which mean equals the infinite width NTK Θ ∞ , i.e ∀ ( X , X ′ ) ,
[Equation] [Equation]
Proposition A.6.
For one hidden layer networks,
[Equation]
where Z is the linear combination of 2 Chi-Square distributions, and
E
[ Z ] =
[Equation]
Proof.
Following the Central Limit Theorem, we have the following convergence in distribution:
[Equation]
where Σ is the covariance matrix between the entries of ˆΘ( X , X ′ ) .
Let the function
[Equation]
for any invertible block matrix W , and vector v .
Note that g (Θ h ( X , X ) , Θ h
( X , x ))
( Y − f
( X , θ 0 )) is the prediction of a linearly trained neural network evaluated on x trained on X , given a functional initialization f ( ., θ 0 ) and
NTK Θ h .
We wish to estimate the asymptotic behavior of the expectation and variance of this quantity in the limit of h → ∞ .
However, these moments are not always defined because the support of Θ h ( X , X ) contains non invertible instances of the gram schmidt matrix (e.g. all weights initialized at 0), which induces divergent moments.
However, because of the convergence in probability of Θ h to Θ ∞ (which is invertible by assumption), the event of such singularities becomes rarer as h increases, and the delta method allows us to get the asymptotic expectation and variance.
Using the fact that g has continuous first partial derivatives, and the independence of f ( X , θ 0 ) and Θ h , following Lemma A.5,
[Equation]
[Equation]
with Z being the linear combination of 2 Chi-Square distributions, and
[Equation]
where, by vectorizing matrices and using g k the k -th entry of the value of g , Σ i,j = ∇ TW g i (Θ ∞ ( X , X ) , Θ ∞ ( X , x ))
C ov [ vect (ˆΘ( X , X ))] ∇ W g j (Θ ∞ ( X , X ) , Θ ∞ ( X , x ))
+ ∇ Tv g
i (Θ ∞ ( X , X ) , Θ ∞ ( X , x ))
C ov [ˆΘ( X , x )] ∇ v g j (Θ ∞ ( X , X ) , Θ ∞ ( X , x ))
+ ∇ TW g
i
Θ ∞ ( X , X ) , Θ ∞ ( X , x ))
C ov [
[Equation] (ˆΘ( X , X [Equation] ) , ˆΘ( X , x )] ∇ v g j (Θ ∞ ( X , X ) , Θ ∞ ( X , x ))
+
[Equation]
g j (Θ ∞ ( X , X ) , Θ ∞ ( X , x ))
Using ∇ v g k ( W, v )
=
W − 1 u k and ∇ W g k = vect ( − W − T vu Tk W − T ) , where u k is the vectoreverywhere except for the k -th position which is 1, the expression can be rewritten as
[Equation]
Finally, we can notice that the following expression equals that of
[Equation]
which concludes the proof by using E [ˆΘ] =
Θ ∞ .
A.2.2 Approximation in the general case
In the general case, we can no longer apply the central limit theorem to asymptotically describe the
NTK as a gaussian.
Nonetheless, the delta method is often used in a form that is essentially identical to that above, but without the asymptotically normal assumption, so long as the fluctuation of the variable around the mean vanishes, i.e. ∥ Θ θ 0 ( {X , x } , {X , x } ) − ¯Θ( {X , x } , {X , x } )
∥ F = o p (1) .
Using the identity M − 1 = ¯ M − 1 + ¯ M − 1 ( ¯ M − M ) ¯ M − 1 + ¯ M − 1 (cid:2) ( ¯ M − M ) ¯ M − 1 (cid:3) 2 ¯ MM − 1 for any pair of invertible matrices M, ¯ M , we can rewrite f lin as f lin ( x ) =
f
( x ) + Θ θ 0 ( x )Θ − 1 θ
( Y − f )
= f
( x ) + Θ θ 0 ( x )[¯Θ − 1 + ¯Θ − 1 (¯Θ − Θ θ 0 )
¯Θ − 1 + ¯Θ − 1 (cid:2) (¯Θ − Θ θ 0 )¯Θ − 1 (cid:3)
¯ΘΘ − 1 θ 0 ]
( Y − f )
= f
( x ) + ¯Θ( x )[¯Θ − 1 + ¯Θ − 1 (¯Θ − Θ θ 0 )¯Θ − 1 + ¯Θ − 1 (cid:2) (¯Θ − Θ θ 0 )
¯Θ − 1 (cid:3)
¯ΘΘ − 1 θ 0 ]
( Y − f )
+ [Θ θ 0 ( x ) − ¯Θ( x )]
[¯Θ − 1 + ¯Θ − 1 (¯Θ − Θ θ 0 )¯Θ − 1 + ¯Θ − 1 (cid:2) (¯Θ − Θ θ 0 )¯Θ − 1 (cid:3)
¯ΘΘ − 1 θ 0 ]
( Y − f )
= f
( x ) + ¯Θ( x )¯Θ − 1 ( Y − f )
+ ¯ Θ( x ) ¯ Θ − 1 ( ¯ Θ − Θ θ 0 )
¯ Θ − 1 ( Y − f )
+ (Θ θ 0 ( x ) − ¯ Θ
( x ))
¯ Θ − 1 ( Y − f ) + (cid:2)
Θ θ 0 ( x ) − ¯Θ( x )¯Θ − 1 Θ θ 0 (cid:3) ¯Θ − 1 (¯Θ − Θ θ 0 )¯Θ − 1 ( Y − f )
+ o p ( ∥ Θ θ
( {X , x } , {X , x } ) − ¯Θ( {X , x } , {X , x } ) ∥ 2 F ) where we note Θ θ 0 =
Θ θ
( X , X ) , Θ θ 0 ( x ) =
Θ θ
( X , x ) (resp.
for ¯Θ , f ) for ease of notation.
For sufficiently large width, with high probability the remainder term will be negligible.
Keeping the empirical mean and variance in mind, we can now take the expectation and variance ignoring the rare singularities.
[Equation]
[Table]
[Equation]
[Table]
[Equation]
[Table]
[Equation]
[Equation]
[Table]
[Equation]
Assuming the fluctuation of Θ θ 0 around its mean is in the order of O ( h − 14 ) (see A.2.3)
[Equation]
we have V c ( x ) , V i ( x ) , V cor ( x ) all of order O ( 1 h ) .
While the variance of the true residual might not be finite for the same reason as why V [ f ( x )] is not, expanding the approximation to one order higher yields
V res ( x ) ≈ O ( 1 h 2 ) .
A.2.3 Fluctuation of the NTK initialization
For one hidden layer networks, we can bound the fluctuation of the NTK at initialization using the
Central Limit Theorem, which yields V [Θ θ 0 ]
=
O ( 1 √ h ) .
While we do not provide a proof, a similar heuristic argument presented in Appendix C. of [38] can be used to argue for the same bound in the general case of arbitrary depth networks.
A.3 Other remarks A.3.1 Interpretation of variance terms
Given a centered functional initialization g and an NTK ¯Θ , a fully trained neural network has the functional expression
[Equation]
The predictive variance is then
[Equation]
[Equation] [Equation]
[Equation]
For any Θ θ 0 , and f centered and decorrelated, by defining g ( x ) =
Θ θ 0 ( x, X )¯Θ( X , X ) − 1 f
( X , θ 0 ) [Equation] we get
V
[Equation]
which is identical to V i ( x ) from Proposition 2.1.
A.3.2 Noise correlation
In general, the noise in the NTK and in the functional initialization are related, as they come from the same weight initialization θ 0 .
Therefore, the analytical expression in Proposition 2.2 would have another covariance term.
This covariance term disappears as we consider the models f gd-c and f gd-a defined in Section 3.2.2, which still manage to describe the predictive variance of the full model f as the interpolation of the two.
We can nevertheless construct a model such that the 2 noises are decorrelated for validating the taylor expansion from Prop 2.1, by sampling 2 initialization independently, θ f 0 and θ k 0 , and defining the model as such:
[Equation]
We use such f lin-d to compute the predictive variance in Fig.
2 and 4.
A.4 Moments of the Neural Tangent Kernel
We consider the MLP defined in Section 2.
We will consider the case where the output dimension h L is 1 for ease of notation, but the derivations can be trivially extended into the multiple dimension case.
The NTK is defined as:
[Equation]
(65) where w l,i is the i -th column of W l , and b l,i the i -th element of b l .
Denoting by z l,i the i -th pre-activation in layer l for the input x , we have
[Equation]
−
And thus
[Equation]
where the x ′ l , z ′ l are the counterparts of x l and z l evaluated at x ′ .
For the single hidden layer case (e.g. L = 2 ), the above expression can be simplified into
[Equation]
where h = h 1 , d = h 0 , w 2 , 0 ,i the i -th element of w 2 , 0 , and z i =
z 1 ,i .
A.4.1 NTK first moment for 1 hidden layer MLP
For a single hidden MLP, we have
[Equation]
(68) which is identical to the infinite width deterministic NTK [24].
A.4.2 NTK
second moment
for 1 hidden layer MLP
Here, we assume the network to be a 1 hidden layer MLP.
We then have
[Equation]
[Equation]
and thus, using C ov
[Equation]
z i , z
[Equation] [Equation]
ov
[Equation]
w 2 , 0 ,i ,
[Equation]
[Equation]
= 0 for any i ̸ =
j
[Equation]
Since the z i and w 2 ,
[Equation]
i are further identically distributed, by denoting by z , w the random variables respectively drawn from the same distributions,
[Equation]
In particular
[Equation]
we have
[Equation] [Equation]
A.4.3 Special case of ReLU activation
We now give the analytical expression of the first and second moments of the NTK for the 1 hidden layer MLP ReLU activation that are required to compute the predictive variance for a single training data setting.
For simplicity, we assume the bias to be initialized to 0.
[Equation]
Following the previous notation, given x , the hidden activations are i.i.d.
random variables z = σ w √ d w T 1 x where w 1 is a univariate standard Gaussian random variable.
σ w is typically chosen to be √ 2 for ReLU activations.
We can rewrite a multivariate standard gaussian random variable as w 1 = r.u where r = || w 1 ||is a real valued random variable distributed such that its squared value follows the Chi-squared distribution of degree dim ( w 1 )
=
d and u = w 1 || w
|| 2 is a multivariate random variable uniformly distributed on the unit sphere.
The 2 random variables are furthermore independent.
Let x, x ′ ∈ R d .
We denote by θ = arccos( x T x ′
|| x |||| x ′ || )
the angle between the vectors.
We define
S x,x ′ = { u ∈ R d s.t.
||
u
||
= 1 , u T x > 0 , u T x ′ > 0 } .
We then have
[Equation]
Where u ∥ is the component of u which is in the 2-dimensional subspace spanned by x, x ′ if θ ̸ = 0 , and any 2-dimensional subspace including x otherwise.
β = sign ( u T || y ) .
arccos( u T
||
v ||
v || ) , with v = x
|| x
||
+ x ′ ||
x ′ ||
and y a unit vector in the subspace orthogonal to v , is its angle in the subspace, uniformly distributed on [ − π, π ] .
ϕ ( z ) ϕ
( z ′ ) is thus the product of 3 independent distribution, a random variable from a Chi-squared distribution of degree d, another one which depends on β , and finally on || u
∥
|| .
Furthermore we have
[Equation]
ϕ ′
( z ) ϕ ′
( z ′ ) is thus a Bernouilli distribution of probability p =
π − θ 2 π .
Let us now compute the various quantities required for the predictive variance:
[Equation]
where we used cos( β + θ 2 ) cos( β − θ 2 )
=
12 [cos(2 β ) + cos( θ )] and E ( χ 2 d ) =
d .
[Equation]
where we used V ( χ 2 d ) = 2 d and C d = E
[ || u ∥
[Equation]
Likewise,
[Equation]
as given by the Bernouilli distribution
[Equation]
Finally,
[Equation]
and by using 1 u ∈ S x,x ′ 1 u ∈ S x,x = 1 u ∈ S x,x ′ as well as E [ ϕ ′ ( z ) 2 ] = 12 ,
[Equation]
[Equation]
Putting everything together in eq 68,72 and
[Equation]
and using
[Equation]
gives us the analytical expression of the following variance terms:
[Equation]
In particular, when ∥ x ∥ , ∥ x ′ ∥ << √ d , the first-order approximation of V [ f lin − c ( x ′ )] becomes a function which only depends on θ , which could be seen on Fig.
We analytically validate the expression in Fig.
B Appendix: empirical results
In this Appendix Section, we provide more data on similar experiments described in Section 2 and 3 of the manuscript.
Generally, we conducted our experiments on 4 Linux servers with 8 Nvidia
RTX 3090 GPUs with 24 GB memory each.
The presented experiments are compute-intensive which led to experiments validating our theoretical propositions on rather small networks and datasets.
During the development, we conducted many scans over ensemble width and depth as well as datasets over the course of several months.
Despite heavily relying on PyTorch, we conducted NTK kernel experiments with the following Github codebase.
We thank the authors for providing this excellent resource (https://github.com/google/neural-tangents).
Further details about our general setup and training specifications are not described in the text.
Missing details may be described in the accompanied code.
• We choose a learning rate η = 0 .
1 and trained all of our models with gradient descent and momentum (0.9) for all (linearized) training experiments.
Although the learning rate is relatively high, we saw that the models trained with gradient descent align very well with the kernel models.
• For the CNN, we always use filter size of 3 and padding.
Every 2nd layer, we use a stride of 2.
Before the last layer, we flatten the features and linearly project to the output.
We always use the NTK initialization as introduced above.
• For the SGD results in Table 1, we used a batchsize of 1000.
• Whenever we used kernel models and a small dataset (N=100), we restricted the problem to be a binary classification problem.
• For the WRN 28-10 experiments, we used learning rate η = 0 . 03 and batchsize 128 the standard network specifications as in https://github.com/hysts/pytorch_wrn.
• For the AUROC computation, we used the standard method from the SciPy package.
B.1 Additional empirical results
The following results are presented in the Appendix:
• Figure 5 and Table 3: Confirmation of the assumption used in Proposition 2.3 for MLP and CNN respectively trained on a subset of MNIST and CIFAR10.
• Figure 6: R ( f ) for CNNs trained on a subset of CIFAR10 in support of Proposition 2.3. • Table AUROC • Table
: Test set accuracy and AUROC (stochastic) linearly trained as well as models trained with (stochastic) gradient descent on a subset of MNIST (N=1000). We observe tiny differences between the stochastic and its non-stochastic counterpart
For computing the AUROC values that play a central part in our empirical evaluation we simply collect predictions from in-distribution i.e. the test dataset of the corresponding training dataset as well as predictions from the out-of-distribution datasets which vary across setups, see above.
To compute per in- and out-of-distribution pair, we compute the auroc values with the help of the publicly available sklearn package and its metrics.roc_auc_score function.
We report the average over the pairs.
[Figure]
V i
V c
V cor
[Figure]
1/h
V i
V c
V cor
[Figure] [Figure]
[Table] [Table]
trained an a subset (N=1000) of CIFAR10.
We indicate small standard deviations σ obtained overensembles of size E with ± . 00 .
In all experiments, the various disentangled models show significant differences in behavior.
All linearly trained models follow the gradient descent models behavior tightly.
When optimizing with SGD, isolating initial noise sources still affect the ensemble behavior significantly and can lead to improved OOD detection as well as test set accuracy in this more realistic settings.
[Table]
In all experiments
[Table]
the various disentangled models show significant differences in behavior.
linearly trained models follow the gradient descent models behavior tightly even in this regime of full
[Table]
[Figure]
[Table]
[Table]
[Table]
[Figure]