Title: Joint Training of Deep Ensembles Fails Due to Learner Collusion
Venue: NeurIPS 2023
Accepted: yes
Keywords: Deep Ensembles

Deep Learning
Abstract: Ensembles of machine learning models have been well established as a powerful method of improving performance over a single model. Traditionally, ensembling algorithms train their base learners independently or sequentially with the goal of optimizing their joint performance. In the case of deep ensembles of neural networks, we are provided with the opportunity to directly optimize the true objective: the joint performance of the ensemble as a whole. Surprisingly, however, directly minimizing the loss of the ensemble appears to rarely be applied in practice. Instead, most previous research trains individual models independently with ensembling performed _post hoc_. In this work, we show that this is for good reason - _joint optimization of ensemble loss results in degenerate behavior_. We approach this problem by decomposing the ensemble objective into the strength of the base learners and the diversity between them. We discover that joint optimization results in a phenomenon in which base learners collude to artificially inflate their apparent diversity. This pseudo-diversity fails to generalize beyond the training data, causing a larger generalization gap. We proceed to comprehensively demonstrate the practical implications of this effect on a range of standard machine learning tasks and architectures by smoothly interpolating between independent training and joint optimization.

Joint Training of Deep Ensembles Fails Due to Learner Collusion

Abstract

Ensembles of machine learning models have been well established as a powerful method of improving performance over a single model. Traditionally, ensembling algorithms train their base learners independently or sequentially with the goal of optimizing their joint performance. In the case of deep ensembles of neural networks, we are provided with the opportunity to directly optimize the true objective: the joint performance of the ensemble as a whole. Surprisingly, however, directly minimizing the loss of the ensemble appears to rarely be applied in practice. Instead, most previous research trains individual models independently with ensembling performed post hoc . In this work, we show that this is for good reason -joint optimization of ensemble loss results in degenerate behavior . We approach this problem by decomposing the ensemble objective into the strength of the base learners and the diversity between them. We discover that joint optimization results in a phenomenon in which base learners collude to artificially inflate their apparent diversity. This pseudo-diversity fails to generalize beyond the training data, causing a larger generalization gap. We proceed to comprehensively demonstrate the practical implications of this effect on a range of standard machine learning tasks and architectures by smoothly interpolating between independent training and joint optimization. 1

1 Introduction

Deep ensembles [1] have proven to be a remarkably effective method for improving performance over a single deep learning model. This success has been attributed to deep ensembles exploring multiple modes in the loss landscape [2], in contrast to variational Bayesian methods, which generally attempt to better explore a single mode [e.g. 3]. Recent work has proposed an alternative view suggesting that deep ensembles' success can be explained as being an effective method of scaling smaller models to match the performance of a single larger model [4]. Regardless of the specific mechanism, deep ensembles' empirical success has been at the heart of numerous state-of-the-art deep learning solutions in recent years [e.g. 5, 6, 7].

Historically, non-neural network ensembles such as a bagging predictor [8] or a random forest [9] were generally trained independently and aggregated at test time. Sequential training methods such as boosting [10] were also developed to produce more collaborative ensembles. Training individually and evaluating jointly was sensible for these methods, in which the base learners were typically

decision trees that require individual training. Formally, for a single example x , a loss function L and denoting the prediction of learner j on x as f j ∈ R d with target y ∈ R d , we define the joint objective as L ( 1 M ∑ M j =1 f j , y ) and the independent objective as 1 M ∑ M j =1 L ( f j , y ) . 2 Interestingly, modern deep ensembles are also generally trained independently and evaluated jointly. This is less intuitive as the performance of the ensemble as a whole (i.e. the joint objective) is the true objective of interest and jointly training a deep ensemble is quite natural for the backpropagation algorithm. Indeed, one might expect there to be no need for the proxy objective of independent training in this case, as the joint objective should find a solution in which individual learners collaborate to minimize the ensemble's loss. In practice, however, independent training generalizes better than joint training in this context. A phenomenon that we demonstrate in Section 3 and investigate throughout this work.

In Section 4, we demonstrate that ensemble diversity is the foundation upon which understanding the limitations of joint training sits. We show that any twice differentiable joint loss function of an ensemble can be decomposed into the individual losses and a second term, which can be interpreted as ensemble diversity. This diversity term encourages some level of ambiguity among the base learners' predictions, a generalization of well-established work in the regression setting [11]. Mathematically, this term is exactly the difference between independent and joint training and, therefore, must encapsulate the disparity in performance. In Section 5, we introduce a scalar weighting on the diversity term allowing us to smoothly interpolate between the extremes of fully independent and fully joint training. y

Then, in Section 6, we diagnose and empirically verify a precise cause of the sub-optimal performance of jointly trained ensembles. We show that the additional requirement for diversity in the training objective can be trivially achieved by an adverse effect we term learner collusion . This consists of a set of base learners artificially biasing their outputs in order to trivially achieve diversity while ensuring that these biases cancel in aggregate as illustrated in Figure 1. We perform extensive experiments verifying this hypothesis. We then show that although training loss can be driven down by learner collusion, this does not generalize beyond the training set, resulting in a larger generalization gap than independent training. Finally, we examine the practical implications of learner collusion on standard machine learning benchmarks.

y

1

2

¯

f

f

1

f

2

f

3

f

4

Learner collusion

This work addresses the pervasive yet relatively understudied phenomenon that directly optimizing a deep ensemble fails . (1) We unify various strands of the ensemble literature to provide a deep, practical categorization of ensemble training and diversification. (2) We present and verify a comprehensive explanation for this counter-intuitive limitation of deep ensembles with both theoretical and empirical contributions. (3) Weinvestigate the catastrophic consequences of this issue on practical machine learning tasks. As this investigation takes a somewhat non-standard format, we provide an abridged summary of this work and its technical contributions in Appendix A.

2 Background

Independent training. This refers to ensembles 3 in which each base learner is optimized independently, with no influence from the other ensemble members either explicit or implicit . This includes many non-neural ensemble techniques in which diversity between the base learners results in ensemble performance better than any individual. This diversity is often achieved by sampling the input space [e.g. bagging in 8] and the feature space [e.g. random forest in 9]. In the case of deep ensembles of neural networks, independent training is also generally applied, with diversity achieved using random weight initialization and stochastic batching [1, 2]. Interestingly, bootstrap sampling of the inputs is ineffective in the case of deep ensembles [12]. Several notable state-of-the-art results

have been achieved by ensembling several independently trained but individually performant neural networks. For example, the groundbreaking results in the ImageNet challenge were often achieved by independently trained ensembles such as VGG [13] GoogLeNet [5], and AlexNet [14]. Seq2Seq ensembles of long short-term memory networks [15] achieved remarkable performance for neural translation. More recently in the tabular domain, ensembles of transformers have outperformed state-of-the-art non-neural methods [7].

Ensemble-aware individual training. This refers to the case where each base learner is still being optimized as an individual, but with some additional technique being applied to coordinate the ensemble members . A prominent approach is sequentially trained, boosting ensembles in which each new member attempts to improve upon the errors of the existing members (e.g. AdaBoost [10] & gradient boosting [16]). In the case of deep ensembles, an alternative approach consists of training each base learner with a standard individual loss with the addition of an ensemble-level regularization term designed to manage diversity. This often consists of an f-divergence between the learners' predictive distributions. Examples include the pairwise χ 2 -divergence [17], KL -divergence, but between the learner and the ensemble [18], and a computationally efficient, kernelized approximation of Renyi-entropy [19]. Another prominent regularization approach is based on the seminal idea of negative correlation learning (NCL) which penalizes individual learners through a correlation penalty on error distributions [20, 21]. This penalty weakens the relationship with other learners and controls the trade-off between bias, variance, and covariance in ensemble learning [22]. Recent works have adapted NCL for improved computational efficiency with large base learners [23] and attempt to generalize NCL beyond the regression setting [24]. [25] encourages ensemble diversity in weight and functional space using a 'repulsive' term while other works show that a single learner can be viewed as an ensemble with diversity encouraged implicitly (e.g. [26, 27]). The efficient BatchEnsemble [28] can also be characterized as an ensemble-aware individual training approach.

Joint training. This refers to the case where the ensemble's predictions are optimized directly as if it were a single model . Intuitively, one might naturally expect joint training to outperform the previously discussed methods as it directly optimizes (at training time) the test metric of all three approaches ensemble loss . Empirically, however, this is not the case as joint training achieves inferior performance in general. [29] reported poor results when joint training averages the base learners' output scores but excellent performance when they are averaged at the log-probability or loss levels. However, later work showed that the two alleged success cases were, in fact, implementing independent training rather than joint training unbeknownst to the authors [30]. This later work also reported poor generalization for score-averaged ensembles. We provide a detailed clarification and gradient analysis in Appendix E. Elsewhere, [31] experienced poor performance from joint training for both score and probability averaging. Concurrently to this work, [32, 33] also found that joint training consistently resulted in worse empirical performance. To put the matter to rest, we demonstrate these issues with joint training in the next section.

3 Empirically Evaluating Joint Training

In this section, we briefly demonstrate the effect that we investigate throughout this work. That is, when evaluating the ensemble loss at test time, optimizing for that same ensemble loss objective at training time (i.e. joint training) is less effective than optimizing each of the base learners independently (i.e. independent training). We do this by performing a straightforward comparison of the two methods. We compare the test set performance of independent and joint training on ImageNet [34]. We consider several architectures as base learners of an ensemble of various sizes determined by computational limitations. We report test set top-1 and top-5 accuracy for both independently and jointly trained ensembles. These results are included in Table 1. Complete experimental details for all experiments in this work are provided in Appendix B.

Here, we observe the counter-intuitive phenomenon that independently trained ensembles consistently outperform jointly trained ensembles, despite all models being evaluated using the joint objective. This is consistent with previous works that have attempted to apply joint training [30, 31]. In the following sections, we will proceed to investigate and explain this effect through the lens of ensemble diversity.

Summary: Joint training performs considerably worse than independent training in practice.

4 Diversity in Ensemble Learning

To understand the limitations of joint training, we will need to examine the joint objective through the lens of diversity. In this section, we argue for a unified definition of diversity which generalizes much of the existing literature on the subject.

4.1 Preliminaries

We begin with some notation. We consider ensembles made up of M individual learners. The j 'th individual learner maps an input x ∈ R p to a d -dimensional output space such that f j : R p → R d and attempts to learn some regression target(s) y ∈ R d or, in the case of classification, labels which lie on a probability simplex ∆ d . Without loss of generality, we can refer to y ∈ Y . We primarily consider combining individual learner predictions into a single ensemble prediction as a weighted average ∑ M j =1 w j f j ( x ) = ¯ f ( x ) where w j ∈ R denotes the scalar weights. Although not strictly required for much of this work, we will generally assume that these weights satisfy the properties ∑ M j =1 w j = 1 and w j ≥ 0 ∀ j , ensuring that each individual learner can be interpreted as predicting the target. Scalars are denoted in plain font and vectors in bold. We generally drop the dependence on x in our notation such that individual and ensemble predictions are denoted f j and ¯ f respectively. Finally, we formalize the respective losses of the ensemble and the base learners in Definition 4.1.

Definition 4.1 (Ensemble and Individual Loss) . For a given loss function L : Y 2 → R + , we define:

(a) The overall ensemble loss ERR := L ( ¯ f , y ) .

(b) The weighted individual loss of the base learners ERR := ∑ M j =1 w j L ( f j , y ) .

Intuitively, ERR measures the predictive performance of the ensemble, while ERR measures the aggregate predictive performance of the individual ensemble members.

4.2 A Generalized Diversity Formulation

Diversity in regression ensembles. The prevailing view of diversity in regression ensembles began in [11], which showed that the mean squared error of a regression ensemble can be decomposed into the mean squared error of the individual learners and an additional ambiguity term which measures the variance of the individual learners around the ensemble prediction showing

This decomposition provides an intuitive definition of diversity whilst also showing that a jointly trained ensemble, that is one that simply optimizes ERR, encourages diversity by default due to it being baked into the objective . Later work from the evolutionary computation literature proposed an apparently alternative measure of diversity called Negative Correlation Learning, which argues

that lower correlation in individual errors is equivalent to higher diversity [21]. The apparent disagreement between these two seemingly valid views on diversity was resolved when it was shown that they are in fact equivalent up to a scaling factor of the diversity term [22].

Generalized diversity. Motivated by the regression case, we will argue that the difference between the individual and ensemble errors has a natural interpretation as the diversity among the predictions of the ensemble members for any twice (or more) differentiable loss function . In fact, this generalizes several existing notions of diversity proposed throughout the literature for specific cases. Specifically, we define diversity as in Definition 4.2 and will proceed to justify this definition in what follows.

Definition 4.2 (Diversity) . The diversity of an ensemble of learners (DIV) is the difference between the weighted average loss of individual learners and the ensemble loss:

Diversity in classification ensembles. It is natural to next verify that this definition is appropriate in the classification setting. We proceed by concretely deriving the diversity term in this setting. There are two standard methods for aggregating base learners' predictions into a single ensemble prediction in classification. Averaging at the score (preactivation) level and averaging at the probability level. We address each of these in turn.

1 ⃝ Score averaging . We consider a classification task with target distribution y ∈ Y ∈ ∆ d with d classes, and consider ensembling by averaging the base learners' scores, before applying the normalizing map ϕ (typically a softmax). We define p j : X → ∆ d ≡ ϕ ◦ f j as the normalized predictive distribution for learner j and ¯ q = ϕ ◦ ¯ f the predictive distribution for the ensemble, where ¯ f = ∑ M j =1 w j f j as before. In this setting, it can be shown [41] that

where D KL is the KL -divergence. Intuitively, this diversity term measures the weighted divergence between each ensemble member's predictive distribution and the full ensemble's predictive distribution, a sensible expression for diversity. It should be noted that this decomposition holds for normalizing functions beyond the softmax [30]. Specifically, this exact expression holds for all exponential family normalization functions with score averaging. This allows the diversity decomposition described in this work to apply to disparate settings as demonstrated by its recent application in modeling diversity among the Poisson spike train outputs of spiking neural networks [42].

2 ⃝ Probability averaging . This setting diverges from score averaging by setting the ensemble predictive distribution ¯ p to be the average of the base learners' probabilities such that ¯ p = ∑ M j =1 w j ( ϕ ◦ f j ) . This is a more natural strategy, averaging the scale-invariant probabilities while also providing each base learner with unique gradients during backpropagation (see a detailed gradient analysis in Appendix E). The resulting expression for diversity is included in Theorem 4.3.

Theorem 4.3. For a probability averaged ensemble classifier, trained using cross-entropy loss L CE with k ⋆ ∈ K denoting the ground truth class, diversity is given by

Remark 4.4 . It is not immediately obvious that this expression has a natural interpretation as diversity. However, in Appendix C.3 we show it to be well aligned with the notion of diversity used throughout this work by analysis of the distribution of p j ( k ⋆ ) around ¯ p ( k ⋆ ).

Beyond the KL-divergence family, alternative loss functions also result in sensible diversity expressions. Brier score, expressed as (¯ p -y ) T (¯ p -y ) , can be shown to obtain DIV = ∑ j w j ( ¯ p j -p ) T (¯ p -p j ) , the variance of the base learner probabilities around the ensemble probabilities [4].

For 0-1 loss and the case of majority voting, diversity is also measured by the disagreement between the (categorical) predictions of the base learners and the ensemble prediction [43].

Diversity in a general setting. The previous paragraphs verified the diversity definition introduced in Definition 4.2 to be a sensible interpretation for several common loss functions. A more general question can be asked Is DIV reasonable for any choice of loss functions? The answer is yes , and this turns out to be no coincidence. To demonstrate why, we analyze the properties of the diversity expression for any twice differentiable loss function and discover a general form that permits a clear interpretation as diversity. To do so, we present Theorem 4.5, a multidimensional generalization of one proposed in [44] which, in turn, generalizes our notion of diversity and analysis of joint training to practically any loss function.

Theorem 4.5. [Diversity for General Loss] For any loss function L : Y 2 → R + with dim( Y ) ∈ N ∗ that is at least twice differentiable, the loss of the ensemble can be decomposed into

where H L f is the Hessian of L with respect to its first argument and some c j ∈ [0 , 1] for all j ∈ [ M ] .

Here, the measure of diversity is obtained by performing a Taylor expansion on the loss of the output of f i near the ensemble output ¯ f and using Lagrange's remainder theorem to obtain an exact equality . Additionally, f ∗ i is an uncertain number, which approaches a constant in the limit. This diversity admits the same interpretation as the variance of learners around the ensemble, but is now weighted by a term dependent on the second derivative of the loss function.

One can easily verify that this generalized definition encompasses the uni-dimensional MSE ambiguity decomposition in Equation (1), where L ′′ ( f ∗ i , y ) = 2 . Similarly, the multidimensional MSE L ( f , y ) = ∥ f -y ∥ 2 2 leads to the Hessian H L f = 2 I , where I is the identity matrix on Y . From this, we get the multi-dimensional generalization DIV = ∑ j w j ∥ f j -¯ f ∥ 2 ≥ 0 . Importantly, diversity has the same non-negative property ( DIV ≥ 0 ) for any loss function with semi-positive definite Hessian H L f ⪰ 0 , such that ensemble performance is always better than individual learner performance. Another typical example is the classification loss L ( f , y ) = -y ⊺ log f . Here, it is trivial to show that H L f ( f , y ) = diag( y ⊘ f 2 ) , where ⊘ denotes the component-wise division. Again, this Hessian matrix is positive semi-definite for y , f ∈ ∆ d so that DIV ≥ 0 . Revisiting our previous question, we verify that, for at least twice-differentiable loss functions (e.g. logistic loss: L = log(1 + exp -yf ) & exponential loss: L = exp -yf ), DIV admits a generalized interpretation as the variance of individual predictors around ensemble prediction. This makes a robust case for Definition 4.2, which provides a principled estimate of diversity (as opposed to heuristic measures) and can be easily obtained by calculating the difference between ensemble loss and weighted average learner loss.

Summary: The difference between independent loss and joint loss has a natural interpretation as diversity, providing a robust generalization of much of the existing literature.

5 An Augmented Objective

Given this generalized definition of diversity, one might naturally consider placing a scalar weighting on its contribution, resulting in an augmented training objective. In fact, such an approach has already been applied for special cases of diversity including regression [22], score averaging [30], and, concurrently to this work, in [33, 32]. Throughout our experiments, we will consider this augmented objective in Equation (2), allowing us to analyze the effects of smoothly transitioning between independent and joint training.

When β = 0 , this amounts to training each learner independently (i.e. independent training ) and β = 1 corresponds to optimizing the loss of the ensemble as a whole (i.e. joint training ). Clearly, intermediate values of β can be interpreted as interpolating between the two.

When examining Equation (2), one might be tempted to encourage more diversity by setting β > 1 . Previous work by Brown et al. [45] showed that, in the case of regression, optimization can become degenerate for larger values of β , proving an upper bound of β < M M -1 . More recently, [30] showed that in the case of score averaged ensembles using their modular loss , the ensemble loss ERR was unbounded from below for β > 1 - i.e. DIV can trivially be increased faster than its corresponding increase in ERR, thus (superficially) minimizing the loss. In Theorem 5.1 we generalize this same bound on β to all loss functions and further include the case of probability-averaged ensembles.

Theorem 5.1. [Pathological Behavior for β > 1 ] We consider a target set Y = R d or Y ⊂ R d being a compact and convex subset of R d . Let L : Y 2 → R + be a continuous and finite function on the interior int( Y 2 ) . We assume that the loss is not bounded from above on Y 2 in such a way that there exists y ∈ int( Y ) and a sequence ( y t ) t ∈ N ⊂ Y such that L ( y t , y ) → + ∞ for t → ∞ . If β > 1 , then the augmented loss L β defined in Equation (2) is not bounded from below on Y 2 .

Remark 5.2 . Note that we consider the case where Y is a compact and convex subset of R d to include classifiers, for which the target set is a probability simplex ∆ d -1 .

As a consequence of the above theorem, one needs to impose the tighter bound β ≤ 1 in order to have a well-defined optimization objective. Furthermore, intermediate values where β ∈ (0 , 1) may have an additional interpretation beyond interpolating between independent and joint training. In Appendix D we show that, in the regression setting, L β is exactly equivalent to applying independent training while adjusting the targets to account for the errors of the ensemble as measured by its gradient. We derive an exact equivalence between β in L β and the magnitude of the adjustment of the target in this gradient-adjusted target setting.

Summary: An interpolation between independent training and joint training is a well-motivated and useful objective but, to avoid pathological behavior, one should not extrapolate to higher diversity.

6 Why Does Joint Training Fail?

Given the universal interpretation of diversity developed in the previous sections paired with its exclusive impact on the joint training objective, we now turn our attention to answering the question -why does joint training of deep ensembles perform worse than independent training? Some previous works have attempted to partially address this question. [29] suggested that score averaging fails as the softmax function 'distorts' the averages. However, little evidence exists to support this claim. When faced with the same issue, [31] suggested the issue could be due to score averaged ensemble members receiving identical gradient values. We address this point in a gradient analysis in Appendix E, where we (a) point out that the optimal solution to joint objective would still achieve equal or lower loss for score averaging, and (b) show that probability averaged ensemble members receive variable gradient expressions and still suffer from poor performance. Finally, [30] noticed vastly different test set performances among the base learners which they referred to as model dominance . Indeed, this is a symptom of the underlying issue with joint training which we describe and verify throughout the rest of this section. The explanation we propose for the poor empirical results achieved by joint training consists of two claims:

Claim 1: the additional diversity term in the joint learning objective (Equation (2)) results in a dual objective. The DIV term can always be artificially inflated if multiple learners collude to bias their predictive distributions in opposing directions which cancel out in aggregate and, therefore, is easily maximized across the training data. We refer to this phenomenon as learner collusion .

Claim 2: while learner collusion can still minimize the loss on the training data, this solution may contain little genuine diversity resulting in a solution that achieves worse generalization than independent training. In Appendix F, we provide a detailed illustrative example of learner collusion for the regression case from an optimization perspective.

In Section 6.1 we empirically demonstrate this learner collusion effect is emergent. Then, in Section 6.2, we show that the low training loss achieved by learner collusion generalizes poorly onto unseen data. Finally, in Section 6.3, we establish the practical implications of learner collusion on standard tasks.

6.1 Diagnosing Learner Collusion

In this section, we focus on claim 1 and empirically investigate the extent to which learner collusion does occur. Recall that the effect being investigated is that a subset of non-diverse base learners can trivially adjust their similar, or even identical, predictions to appear diverse (i.e. Figure 1). If learner collusion does indeed take place we suggest that the following three effects should be observable empirically: (1) the relative magnitude of diversity should be excessive relative to the ensemble loss, (2) if we remove constant output biases from the individual learners, the ensemble diversity should decrease significantly, and (3) learners should become highly dependent on each other such that removing a small subset of base learners is detrimental to the ensemble performance. We investigate each of these effects in turn.

We perform these experiments on four tabular datasets of the style upon which ensemble methods typically excel [46]. Here we focus on regression as it offers the most controlled setting to isolate the learner collusion phenomenon (see Appendix F), but in later experiments we demonstrate its effects in the classification setting too. We smoothly interpolate between independent and joint training using the augmented objective controlled by β from Section 5. We report mean ± one standard deviation evaluated on a test set throughout. An extended description of the experimental protocol for these experiments is provided in Appendix B.

Diversity explosion. We begin with the most simple expected effect of learner collusion, that the relative magnitude of diversity is excessive with respect to the ensemble loss. The results are included in Figure 2 where we note a sharp increase in diversity for the joint objective. Specifically, we note that mean test set diversity remains relatively low for most values of β but quickly explodes as β → 1 to levels that are often significantly greater than the mean squared error of the ensemble predictions. This indicates that the base learners' variation around their own mean is much greater than the ensemble's variation around the label.

0.244

0.261

0.278

MSE

Bioconcentration

0

.

0

0

.

2

0

.

4

0

.

6

0

.

8

1

.

0

β

0.013

0.626

1.238

Diversity

0.244

0.261

0.278

MSE

Bioconcentration

0

.

0

0

.

2

0

.

4

0

.

6

0

.

8

1

.

0

β

0.013

0.626

1.238

Diversity

0.108

0.119

0.131

Weather

0

.

0

0

.

2

0

.

4

0

.

6

0

.

8

1

.

0

β

0.019

1.239

2.459

0.108

0.119

0.131

Weather

0

.

0

0

.

2

0

.

4

0

.

6

0

.

8

1

.

0

β

0.019

1.239

2.459

2.04

2.332

2.623

Facebook

0

.

0

0

.

2

0

.

4

0

.

6

0

.

8

1

.

0

β

0.068

1.067

2.066

2.04

2.332

2.623

Facebook

0

.

0

0

.

2

0

.

4

0

.

6

0

.

8

1

.

0

β

0.068

1.067

2.066

0.316

0.374

0.431

Abalone

0

.

0

0

.

2

0

.

4

0

.

6

0

.

8

1

.

0

β

0.003

0.379

0.755

0.316

0.374

0.431

Abalone

0

.

0

0

.

2

0

.

4

0

.

6

0

.

8

1

.

0

β

0.003

0.379

0.755

Debiased diversity. We now proceed to the second postulated effect of learner collusion. Although learners could perform learner collusion on a per-example basis, the most straightforward form of this behavior would be learners that bias their predictions without any dependence on the example at hand. To investigate the extent to which this trivial form of collusion does occur we apply a post hoc debiasing to each of the individual learners in the ensemble which affects the trade-off between ERR and DIV without affecting ERR. Specifically, we adjust learner j 's predictions such that

In Appendix C.1 we prove that the debiasing term b j is the optimal diversity minimizer that keeps the ensemble predictions unchanged. In Figure 3 we show that applying this technique significantly reduces the ensemble's apparent diversity to far more reasonable values. In general, we find that this typically removes around half of the ensemble's diversity at the joint training region ( β = 1 ). While example-dependent collusion could also emerge, these results already indicate that a significant proportion of the apparent diversity in jointly trained ensembles is due to an artificial biasing of outputs.

0

.

0

0

.

5

1

.

0

β

0

24

48

% Diversity

Decrease

Weather

0

.

0

0

.

5

1

.

0

β

0

31

61

Skillcraft

0

.

0

0

.

5

1

.

0

β

0

22

44

Facebook

0

.

0

0

.

5

1

.

0

β

0

33

67

Abalone

Learner codependence: The final effect we predicted to emerge due to learner collusion is an excessive codependence among the base learners. If individual learners are engaging in learner collusion, the removal of a subset of these learners at test time should be catastrophic to the performance of the ensemble. This is because a learner producing excessive errors needs to be counterbalanced by other learners performing similar errors in the opposite direction, otherwise the ensemble predictions will be influenced by the poorly performing base learners. To test this we consider dropping a fraction of the individual learners at test time and evaluate the resulting ensemble performance. This is similar to experiments performed in [30], although from a very different perspective. In Figure 4 we present the results of randomly applying various levels of test time base learner dropout 100 times for each model and report the average resulting increase in test set error. These results clearly show that such a procedure is detrimental to ensemble performance at precisely the levels of β at which we have hypothesized learner collusion to occur, while having only minimal impact elsewhere.

0

.

0

0

.

2

0

.

4

0

.

6

0

.

8

1

.

0

β

0

0.24

0.47

Increase in

test error

Weather

With 0.2 dropout

With 0.4 dropout

With 0.6 dropout

0

.

0

0

.

2

0

.

4

0

.

6

0

.

8

1

.

0

β

0

0.14

0.29

Skillcraft

0

.

0

0

.

2

0

.

4

0

.

6

0

.

8

1

.

0

β

0

0.24

0.47

Facebook

0

.

0

0

.

2

0

.

4

0

.

6

0

.

8

1

.

0

β

0

0.08

0.16

Abalone

Summary: Extensive empirical evidence confirms that training with the joint objective results in learner collusion.

6.2 Learner Collusion Generalizes Poorly

Given that we have shown that learner collusion emerges from joint training, we now proceed to examine the second claim. That is, although learner collusion does not prevent the training loss from being minimized, the solution it obtains will generalize more poorly than an independently trained ensemble with non-trivial diversity. This claim is more straightforward to investigate and only requires a comparison of the difference between the training and testing loss curves over the course of training. The difference between the two, which we refer to as the generalization gap , captures the drop in performance when a model transitions from training to testing data.

For this experiment, we return to the more standard machine learning benchmark of classification on the CIFAR tasks as introduced in Section 3. On each dataset we train a joint model (ERR) and an independent model (ERR) and report the generalization gap of ensemble loss for both methods over the course of training. We repeat this six times and include a shaded region representing ± 2 standard errors. The results are included in Figure 5 where we consistently find that joint training displays a significantly larger generalization gap as training proceeds. This is strongly suggestive of a much greater tendency towards overfitting in the jointly trained model while the independently trained model's generalization gap plateaus at a much lower level. We provide

0

50

100

0

.

0

0

.

2

0

.

4

0

.

6

0

.

8

CIFAR-10

Joint

Independent

0

50

100

0

1

2

3

CIFAR-100

Epoch

Generalization Gap

further analysis of the distinct training and testing curves in Appendix G.

Summary: Learner collusion caused by joint training (ERR) results in greater overfitting exhibited by a larger generalization gap.

6.3 Practical Implications

We have now described the phenomenon of learner collusion caused by the joint training of deep ensembles, empirically verified its existence, and demonstrated how it results in poorer generalization. We complete this analysis with an empirical investigation of how learner collusion manifests in the ensemble performance on standard machine learning tasks. We use the augmented objective, L β as described in Section 5, to examine the effects of smoothly increasing the level of joint training in the objective on CIFAR-10/CIFAR-100 with ResNet architectures. We also examine the validation performance of the ensemble and base learners throughout training on ImageNet. Results are provided in Figure 6. We also provide extensive additional results in Appendix G confirming the generality of this degeneracy. There we consider: larger ensembles, score-averaging, alternative models, additional datasets, and ERR as a metric. Complete experimental details are provided in Appendix B.

0

.

0

0

.

5

1

.

0

0

.

93

0

.

94

0

.

95

CIFAR-10

ResNet-18

ResNet-34

0

.

0

0

.

5

1

.

0

0

.

700

0

.

725

0

.

750

0

.

775

CIFAR-100

Level of diversity

β

0

100

200

0

.

0

0

.

2

0

.

4

0

.

6

ImageNet

Ensemble

β

= 0

Base learners

β

= 0

Ensemble

β

= 1

Base learners

β

= 1

0

100

200

0

.

0

0

.

2

0

.

4

0

.

6

ImageNet

Training iteration

Consistently across all experiments, we find that joint training ( β = 1 ) results in the worst test set performance. We also note the poor performance among base learners in this regime, highlighting that inflating diversity comes at the cost of degenerate individual performance. Interestingly, across all of our experiments learner collusion typically emerges as β → 1 implying that partial joint training may be a feasible direction for future work. However, in general, independent training emerges as a primary choice of training strategy for deep ensembles.

7 Conclusion

In this work, we have provided a detailed account explaining the unexpected observation that jointly trained deep ensembles are generally outperformed by their independently trained counterparts despite joint training being the objective of interest at test time . We presented a comprehensive view of diversity in machine learning ensembles through which we analyzed the joint training objective. From this perspective, we uncovered a learner collusion effect in which non-diverse base learners collude to artificially inflate their apparent diversity. Empirically, we both isolated this phenomenon and demonstrated its catastrophic effect on model generalization. Finally, the practical consequences of this limitation were exposed for standard machine learning settings. Note that we also provide a visual summary of this work and its technical contributions in Appendix A. We hope that future work will investigate how to optimize deep ensembles to operate collaboratively whilst avoiding the degeneracies of learner collusion. In Appendix G.7 we include a negative result of our initial attempts in this direction. However, several promising avenues remain open (e.g. jointly optimizing worst-case bounds instead [47, 48]).

Acknowledgments

We would like to thank Alicia Curth, Fergus Imrie, and the anonymous reviewers for very insightful comments and discussions on earlier drafts of this paper. AJ gratefully acknowledges funding from the Cystic Fibrosis Trust. TL is funded by AstraZeneca. JC is funded by Aviva. This work was supported by Azure sponsorship credits granted by Microsoft's AI for Good Research Lab.

References

Balaji Lakshminarayanan, Alexander Pritzel, and Charles Blundell. Simple and scalable predictive uncertainty estimation using deep ensembles. Advances in neural information processing systems , 30, 2017.

Stanislav Fort, Huiyi Hu, and Balaji Lakshminarayanan. Deep ensembles: A loss landscape perspective. arXiv preprint arXiv:1912.02757 , 2019.

Wesley J Maddox, Pavel Izmailov, Timur Garipov, Dmitry P Vetrov, and Andrew Gordon Wilson. A simple baseline for bayesian uncertainty in deep learning. Advances in Neural Information Processing Systems , 32, 2019.

Taiga Abe, E Kelly Buchanan, Geoff Pleiss, Richard Zemel, and John P Cunningham. Deep ensembles work, but are they necessary? arXiv preprint arXiv:2202.06985 , 2022.

Christian Szegedy, Wei Liu, Yangqing Jia, Pierre Sermanet, Scott Reed, Dragomir Anguelov, Dumitru Erhan, Vincent Vanhoucke, and Andrew Rabinovich. Going deeper with convolutions. In Proceedings of the IEEE conference on computer vision and pattern recognition , pages 1-9, 2015.

Mitchell Wortsman, Gabriel Ilharco, Samir Ya Gadre, Rebecca Roelofs, Raphael GontijoLopes, Ari S Morcos, Hongseok Namkoong, Ali Farhadi, Yair Carmon, Simon Kornblith, et al. Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time. In International Conference on Machine Learning , pages 2396523998. PMLR, 2022.

Yury Gorishniy, Ivan Rubachev, Valentin Khrulkov, and Artem Babenko. Revisiting deep learning models for tabular data. Advances in Neural Information Processing Systems , 34: 18932-18943, 2021.

Leo Breiman. Bagging predictors. Machine learning , 24(2):123-140, 1996.

Leo Breiman. Random forests. Machine learning , 45(1):5-32, 2001.

Yoav Freund, Robert E Schapire, et al. Experiments with a new boosting algorithm. In icml , volume 96, pages 148-156. Citeseer, 1996.

Anders Krogh and Jesper Vedelsby. Neural network ensembles, cross validation, and active learning. Advances in neural information processing systems , 7, 1994.

Jeremy Nixon, Balaji Lakshminarayanan, and Dustin Tran. Why are bootstrapped deep ensembles not better? In 'I Can't Believe It's Not Better!'NeurIPS 2020 workshop , 2020.

Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. Imagenet classification with deep convolutional neural networks. Communications of the ACM , 60(6):84-90, 2017.

Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 , 2014.

Ilya Sutskever, Oriol Vinyals, and Quoc V Le. Sequence to sequence learning with neural networks. Advances in neural information processing systems , 27, 2014.

Jerome H Friedman. Greedy function approximation: a gradient boosting machine. Annals of statistics , pages 1189-1232, 2001.

Hendrik Alexander Mehrtens, Camila Gonz´ alez, and Anirban Mukhopadhyay. Improving robustness and calibration in ensembles with diversity regularization. arXiv preprint arXiv:2201.10908 , 2022.

Xiaohui Zhang, Daniel Povey, and Sanjeev Khudanpur. A diversity-penalizing ensemble training method for deep learning. In Sixteenth Annual Conference of the International Speech Communication Association , 2015.

Hong-Jie Xing and Xi-Zhao Wang. Selective ensemble of svdds with renyi entropy based diversity measure. Pattern Recognition , 61:185-196, 2017.

Bruce E Rosen. Ensemble learning using decorrelated neural networks. Connection science , 8 (3-4):373-384, 1996.

Yong Liu and Xin Yao. Ensemble learning via negative correlation. Neural networks , 12(10): 1399-1404, 1999.

Gavin Brown, Jeremy L Wyatt, Peter Tino, and Yoshua Bengio. Managing diversity in regression ensembles. Journal of machine learning research , 6(9), 2005.

Zenglin Shi, Le Zhang, Yun Liu, Xiaofeng Cao, Yangdong Ye, Ming-Ming Cheng, and Guoyan Zheng. Crowd counting with deep negative correlation learning. In Proceedings of the IEEE conference on computer vision and pattern recognition , pages 5382-5390, 2018.

Sebastian Buschj¨ ager, Lukas Pfahler, and Katharina Morik. Generalized negative correlation learning for deep ensembling. arXiv preprint arXiv:2011.02952 , 2020.

Francesco D'Angelo and Vincent Fortuin. Repulsive deep ensembles are bayesian. Advances in Neural Information Processing Systems , 34:3451-3465, 2021.

Pierre Baldi and Peter J Sadowski. Understanding dropout. Advances in neural information processing systems , 26, 2013.

Alan Jeffares, Tennison Liu, Jonathan Crabb´ e, Fergus Imrie, and Mihaela van der Schaar. Tangos: Regularizing tabular neural networks through gradient orthogonalization and specialization. arXiv preprint arXiv:2303.05506 , 2023.

Yeming Wen, Dustin Tran, and Jimmy Ba. Batchensemble: an alternative approach to efficient ensemble and lifelong learning. arXiv preprint arXiv:2002.06715 , 2020.

Anuvabh Dutt, Denis Pellerin, and Georges Qu´ enot. Coupled ensembles of neural networks. Neurocomputing , 396:346-357, 2020.

Andrew Webb, Charles Reynolds, Wenlin Chen, Henry Reeve, Dan Iliescu, Mikel Lujan, and Gavin Brown. To ensemble or not ensemble: When does end-to-end training fail? In Joint European Conference on Machine Learning and Knowledge Discovery in Databases , pages 109-123. Springer, 2020.

Stefan Lee, Senthil Purushwalkam, Michael Cogswell, David Crandall, and Dhruv Batra. Why m heads are better than one: Training a diverse ensemble of deep networks. arXiv preprint arXiv:1511.06314 , 2015.

Taiga Abe, E Kelly Buchanan, Geoff Pleiss, and John P Cunningham. Pathologies of predictive diversity in deep ensembles. arXiv preprint arXiv:2302.00704 , 2023.

Taiga Abe, E Kelly Buchanan, Geoff Pleiss, and John Patrick Cunningham. The best deep ensembles sacrifice predictive diversity. In I Can't Believe It's Not Better Workshop: Understanding Deep Learning Through Empirical Falsification , 2022.

Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, et al. Imagenet large scale visual recognition challenge. International journal of computer vision , 115:211-252, 2015.

Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition , pages 770-778, 2016.

Gao Huang, Zhuang Liu, Laurens Van Der Maaten, and Kilian Q Weinberger. Densely connected convolutional networks. In Proceedings of the IEEE conference on computer vision and pattern recognition , pages 4700-4708, 2017.

Ningning Ma, Xiangyu Zhang, Hai-Tao Zheng, and Jian Sun. Shufflenet v2: Practical guidelines for efficient cnn architecture design. In Proceedings of the European conference on computer vision (ECCV) , pages 116-131, 2018.

Forrest N Iandola, Song Han, Matthew W Moskewicz, Khalid Ashraf, William J Dally, and Kurt Keutzer. Squeezenet: Alexnet-level accuracy with 50x fewer parameters and¡ 0.5 mb model size. arXiv preprint arXiv:1602.07360 , 2016.

Andrew Howard, Mark Sandler, Grace Chu, Liang-Chieh Chen, Bo Chen, Mingxing Tan, Weijun Wang, Yukun Zhu, Ruoming Pang, Vijay Vasudevan, et al. Searching for mobilenetv3. In Proceedings of the IEEE/CVF international conference on computer vision , pages 1314-1324, 2019.

Sachin Mehta and Mohammad Rastegari. Mobilevit: light-weight, general-purpose, and mobile-friendly vision transformer. arXiv preprint arXiv:2110.02178 , 2021.

Tom Heskes. Selecting weighting factors in logarithmic opinion pools. Advances in neural information processing systems , 10, 1997.

Georgiana Neculae, Oliver Rhodes, and Gavin Brown. Ensembles of spiking neural networks. arXiv preprint arXiv:2010.14619 , 2020.

Gavin Brown and Ludmila I Kuncheva. 'good' and 'bad' diversity in majority vote ensembles. In International workshop on multiple classifier systems , pages 124-133. Springer, 2010.

Zhengshen Jiang, Hongzhi Liu, Bin Fu, and Zhonghai Wu. Generalized ambiguity decompositions for classification with applications in active learning and unsupervised ensemble pruning. In Proceedings of the AAAI Conference on Artificial Intelligence , volume 31, 2017.

Gavin Brown, Jeremy Wyatt, Rachel Harris, and Xin Yao. Diversity creation methods: a survey and categorisation. Information fusion , 6(1):5-20, 2005.

L´ eo Grinsztajn, Edouard Oyallon, and Ga¨ el Varoquaux. Why do tree-based models still outperform deep learning on tabular data? arXiv preprint arXiv:2207.08815 , 2022.

Andr´ es Masegosa, Stephan Lorenzen, Christian Igel, and Yevgeny Seldin. Second order pacbayesian bounds for the weighted majority vote. Advances in Neural Information Processing Systems , 33:5263-5273, 2020.

Luis A Ortega, Rafael Caba˜ nas, and Andres Masegosa. Diversity and generalization in neural network ensembles. In International Conference on Artificial Intelligence and Statistics , pages 11720-11743. PMLR, 2022.

Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. Pytorch: An imperative style, high-performance deep learning library. Advances in neural information processing systems , 32, 2019.

Arthur Asuncion and David Newman. Uci machine learning repository, 2007.

Francesca Grisoni, Viviana Consonni, Sara Villa, Marco Vighi, and Roberto Todeschini. Qsar models for bioconcentration: is the increase in the complexity justified by more accurate predictions? Chemosphere , 127:171-179, 2015.

Dongjin Cho, Cheolhee Yoo, Jungho Im, and Dong-Hyun Cha. Comparative assessment of various machine learning-based bias correction methods for numerical weather prediction model forecasts of extreme air temperatures in urban areas. Earth and Space Science , 7(4): e2019EA000740, 2020.

S´ ergio Moro, Paulo Rita, and Bernardo Vala. Predicting social media performance metrics and evaluation of the impact on brand building: A data mining approach. Journal of Business Research , 69(9):3341-3351, 2016.

Samuel George Waugh. Extending and benchmarking Cascade-Correlation: extensions to the Cascade-Correlation architecture and benchmarking of feed-forward supervised artificial neural networks . PhD thesis, University of Tasmania, 1995.

Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 , 2014.

Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009.

Milton Abramowitz, Irene A Stegun, and Robert H Romer. Handbook of mathematical functions with formulas, graphs, and mathematical tables, 1988.

Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Bo Wu, and Andrew Y Ng. Reading digits in natural images with unsupervised feature learning. 2011.

A Paper Summary

Background - Section 2

Categorizes the three forms of ensemble training:

independent training, ensemble-aware individual

training and joint training (the focus of this work)

and relates these concepts to existing literature.

Empirically Evaluating Joint Training - Section 3

Empirically demonstrates the surprising effect in-

vestigated throughout this work -

the joint training

of deep ensembles fails in practice.

•

Results of an empirical comparison

between the two objectives in Table 1.

- Section contributions

- Technical contributions

Diversity in Ensemble Learning - Section 4

Introduces and justifies a unified definition of en-

semble diversity which acts as a foundation for

understanding the limitations of joint training due

to its role in the joint objective.

•

An expression for diversity in the case of

probability averaging in Theorem 4.3.

•

A general expression for diversity for any

twice differentiable loss function in

Theorem 4.5.

An Augmented Objective - Section 5

Describes an objective that acts as an interpolation

between independent training and joint training and

shows this to be well-motivated and a useful tool

for later analysis.

•

A tight upper bound on diversity in the

learning objective in Theorem 5.1.

•

An equivalence between the augmented

objective

L

β

and gradient adjusted

independent training in Appendix D.

Hypothesizes an explanation for the failure of joint

training -

learner collusion

.

Comprehensively

demonstrates the existence, generalization conse-

quences and practical implications of this effect.

•

A gradient analysis of ensemble

•

Empirical discovery of

learner collusion

in

Figures 2, 3 & 4.

•

Demonstration of generalization

consequences in Figure 5.

•

Investigation of collusion implication on

practical benchmark tasks in Figure 6.

B Experimental Details

Empirically Evaluating Joint Training (Section 3) . This section compares training using crossentropy loss as the objective applied to each learner's predictions independently (independent training) and to the ensemble's aggregated predictions (joint training). As described in the main text, we compare the test set performance of these two training methods using various popular architectures on ImageNet [34] evaluating the joint objective on the validation set. Specifically, we use the architectures cited in Table 1 with ensemble size determined by computational limitations. We use implementations from [49] except for MobileViT where the implementation is from [40]. Base learners' are aggregated using their probability outputs (i.e. probability averaging in Section 4.2). We use the standard ImageNet training-validation splits. For all models, we adjusted the following hyperparameters to optimize performance: learning rate, learning rate scheduler (and corresponding decay factor, gamma and frequency), momentum, and weight decay. Batch size was typically set to 128 but was divided by a multiple of 2 for larger base learners. We train models with early stopping and patience of 15 test set evaluations which occur after every 2000 batches. We apply stochastic

gradient descent as our optimizer. We apply standard preprocessing and augmentations consisting of input standardization, random crops, and random horizontal flips.

Diagnosing Learner Collusion (Section 6.1) . These experiments investigate the performance of ensembles of multi-layer perceptrons on four regression datasets from the UCI machine learning repository [50]. These are Bioconcentration [51], Weather [52], Facebook [53], and Abalone [54]. Standard preprocessing is applied to each dataset including standardization of the features and targets, one hot encoding of categorical features, and log transformation of skewed features. Due to the smaller size of these tabular datasets, these experiments are less computationally expensive and, therefore, we perform hyperparameter optimization using Bayesian optimization for each run. For each run, we apply 20 iterations of Bayesian optimization searching over the following parameters. Learning rate ∈ { 0 . 01 , 0 . 005 , 0 . 001 } , batch size ∈ { 16 , 32 , 64 } , weight decay ∈ { 0 , 0 . 01 , 0 . 001 } , number of base learners ∈ { 5 , 10 , 15 , 20 } , hidden layer neurons ∈ { 10 , 20 , 30 } , and number of hidden layers ∈ { 1 , 2 , 3 } . Additionally, the ReLU activation function is used with mean squared error loss. At each run, 20% of the data is randomly used for testing with the remaining data being further split into training and validation with an 80/20 ratio. Each ensemble is trained with early stopping with patience of 5 epochs. The best-performing hyperparameters are selected based on validation set performance and then retrained on the train/validation data for all values of β ∈ { 0 , 0 . 1 , . . . , 0 . 9 , 1 } and evaluated on the test set. This is repeated for 10 runs. The mean ± a standard deviation is provided in our results.

In Figure 2 we report test set ERR and DIV as MSE and Diversity respectively. In Figure 3 we report the percentage decrease in test set DIV for each value of β after the debiasing process described in the main text. In Figure 4, for each model being evaluated on the test set we randomly drop a percentage of the base learners p ∈ { 0 . 2 , 0 . 4 , 0 . 6 } and report the mean increase in test set ERR (ensemble mean squared error).

Learner Collusion Generalizes Poorly (Section 6.2) . This section uses a similar experimental setting as Section 6.3 on the CIFAR datasets, as described in detail in the next paragraph. The key difference is using Adam optimizer [55] without learning rate decay to evaluate performance smoothly over the course of training. We train for a fixed period of 100 training epochs. We note that this number of training epochs is only for our analysis as the best validation performance generally occurs significantly earlier. We also only use the standard training/testing split as a third split is not required. In Figure 5 we report the difference in ERR measured on the training set vs the test set over the course of training and refer to this metric as the generalization gap . This can be expressed as L test ( ¯ f , y ) - L train ( ¯ f , y ) where the loss function is cross-entropy loss. Further results from this experiment are provided in Appendix G.

Practical Implications (Section 6.3) . In Figure 6 (right) we use the same experimental setup as in Section 3. In the two left-side plots we train and evaluate on CIFAR-10 and CIFAR-100 [56]. We train ensembles of either ResNet-18 or ResNet-34 [35] as base learners with each ensemble consisting of five learners. Base learners are again aggregated using their probability outputs (i.e. probability averaging in Section 4.2). We use the standard training and testing splits with an additional 80/20 split on the training data to provide a validation set in order to ensure our test set performance is independent of implicit bias from the training process. We use a similar learning schedule as proposed in [35]. Specifically, we train for 80 epochs using stochastic gradient descent with momentum of 0.9. The learning rate is initialized at 0.1 and exponentially decays at a rate of 0.2 at epochs [30 , 50 , 70] . We apply weight decay of 5 e -4 . Validation performance is evaluated at each epoch with the best-performing model across all 80 epochs evaluated on the test set. A batch size of 128 was used. We repeat each experiment for a total of six runs and report the mean and two standard errors in our results. We apply standard preprocessing and augmentations consisting of input standardization, random crops, random horizontal flips, and random rotations. The key difference in the left figure is that instead of only training deep ensembles using independent training and joint training, we now also consider a hybrid objective using the augmented objective for Section 5. We train this objective for β ∈ { 0 , 0 . 2 , 0 . 4 , 0 . 6 , 0 . 8 , 1 } and report accuracy evaluated on the test set in the figure. In the right figure, we provide the validation set accuracy of both ImageNet models throughout training. We provide both the performance of the individual learners and the aggregate ensemble performance.

Additional experiments from this section are provided in Appendix G. Unless otherwise stated, results reported in the appendix are aggregated over three runs.

Compute . All experiments are run on Azure NCv3-series VMs powered by NVIDIA Tesla V100 GPUs with 16GB of GPU VRAM.

C Proofs

C.1 Optimal Debiasing

We hypothesize that the predictions of each learner in the ensemble is associated with a bias term b j which cancel in aggregate but significantly affect our measure of diversity. This is the most simple conceivable form of learner collusion and in this section we prove that the debiasing term used in Section C.1 is optimal for this purpose.

We begin by noting that for these biasing terms to not affect the aggregated ensemble prediction we require that ∑ j ( f j -b j ) = ∑ j f j and therefore we require that ∑ j b j = 0 . Then we wish to find the set of such bias terms that minimize the standard diversity, DIV , of the ensemble over the data. Combining these two requirements we arrive at the optimization objective in Equation (3) for an ensemble of size M with N examples.

By the method of Lagrange multipliers we obtain M + 1 equations for the M values b j and the Lagrange multiplier λ

Solving equation 4 for an arbitrary b j , we obtain

And subbing into equation 5

Finally, we can substitute λ = 0 back into equation 6 to obtain the solution

C.2 General Diversity Expression for Multidimensional Outputs

Theorem 4.5. [Diversity for General Loss] For any loss function L : Y 2 → R + with dim( Y ) ∈ N ∗ that is at least twice differentiable, the loss of the ensemble can be decomposed into

where H L f is the Hessian of L with respect to its first argument and some c j ∈ [0 , 1] for all j ∈ [ M ] .

Proof. We follow the same approach taken in [44] but extend their proof to account for a multidimensional output space. For each j ∈ [ M ] , we define the curve γ j : [0 , 1] → Y by γ j ( t ) = L [ ¯ f + t · ( f j -¯ f ) , y ] for all t ∈ [0 , 1] . We note that L ( f j , y ) = γ j (1) . Since the loss L is at least twice differentiable, we can perform a first-order Taylor expansion around t = 0 with Lagrange remainder:

with f ∗ j defined as in the theorem statement. We note that this expression for γ j (1) is exact for any twice differentiable loss thanks to the Lagrange's remainder theorem (see e.g. p.878 of [57]). We deduce the following formula for the average loss over the learners:

where the second term vanishes due to the identity ∑ M j =1 w j f j = ¯ f . Since DIV = ERR -ERR , this proves the theorem.

C.3 Probability Averaged Diversity Term for Classification

Theorem 4.3. For a probability averaged ensemble classifier, trained using cross-entropy loss L CE with k ⋆ ∈ K denoting the ground truth class, diversity is given by

Proof.

Interpretation. This expression has a sensible interpretation as the diversity of a probabilityaveraged ensemble. Examining this expression of diversity:

where k ⋆ is the true class, and we note that this expression's only non-zero contribution comes from the predictions of the ground-truth class. We now view p j ( k ⋆ ) as coming from a distribution of probabilities and assume that p j ( k ⋆ ) is distributed around ¯ p ( k ⋆ ) following a Beta distribution with its mean at ¯ p ( k ⋆ ) , i.e. p j ( k ⋆ ) ∼ Beta ( α, β ) , where E [ p j ( k ⋆ )] = ¯ p ( k ⋆ ) . Consequently, we can examine the expected value of the term with respect to ensemble learners as follows:

where the approximation is from taking a second order Taylor expansion around p j 0 ( k ⋆ ) = E p j ( k ⋆ ) [ p j ( k ⋆ )] to approximate E p j ( k ⋆ ) [log p j ( k ⋆ )] . The fourth line is due to a reparameterization of the Beta distribution in terms of mean µ and precision ϕ parameters with α = ϕ · µ and β = ϕ (1 -µ ) . Using this reparameterization, we arrive at an expression that emits an intuitive interpretation- that for a fixed ¯ p , the smaller the value of ϕ (conversely, the larger the variance), the larger the expected value of diversity.

Furthermore, we might ask if this expression of diversity in Theorem 4.3 is connected to an information-theoretic quantity as in the score averaging case . While this expression does not take the form of any divergence term that the authors of this work are aware of, the following link can be made.

Therefore, as the ensemble predictive distribution approaches the label distribution, the diversity term approaches the weighted average of the KL-divergence between the ensemble predictive distribution and each of the base learner predictive distributions, analogous to the equivalent expression of the score averaged ensemble.

C.4 Pathological loss for β > 1

Theorem 5.1. [Pathological Behavior for β > 1 ] We consider a target set Y = R d or Y ⊂ R d being a compact and convex subset of R d . Let L : Y 2 → R + be a continuous and finite function on the interior int( Y 2 ) . We assume that the loss is not bounded from above on Y 2 in such a way that there exists y ∈ int( Y ) and a sequence ( y t ) t ∈ N ⊂ Y such that L ( y t , y ) → + ∞ for t → ∞ . If β > 1 , then the augmented loss L β defined in Equation (2) is not bounded from below on Y 2 .

Proof. Case 1. We first consider the case where Y = R d . Without loss of generality, we consider an ensemble such that w 1 , w 2 > 0 . We fix f j = 0 ∈ Y for all 2 < j ≤ M . We then consider the following sequence of learners: ( f 1 ,t ) t ∈ N = ( y t ) t ∈ N and ( f 2 ,t ) t ∈ N = ( -w 1 w -1 2 y t ) t ∈ N . Wenote that for all t ∈ N , the ensemble prediction equals zero ¯ f t = ( w 1 y t -w 2 w -1 2 w 1 y t +0) = 0 . The ensemble loss is therefore constant and finite ERR t →L (0 , y ) < + ∞ as t →∞ since (0 , y ) ∈ int( Y 2 ) . On the other hand, we have that the average loss diverges as ERR t ≥ w 1 L ( y t , y ) → + ∞ as t →∞ . We deduce that the augmented loss is not bounded from below when β > 1 as β · ERR t +(1 -β ) · ERR t →-∞ for t →∞ .

Case 2. We now consider the case where Y ⊂ R d is a compact and convex subset of R d . Without loss of generality, we consider an ensemble such that 0 < w 1 < 1 . Since Y is compact, it is closed and admits a topological boundary that we denote ∂ Y . Since the loss is finite on the interior int( Y 2 ) , our assumption implies that the loss admits a singular point ( y ∞ , y ) with y ∞ ∈ ∂ Y . We fix f j = y for all 1 < j ≤ M . For the first learner, we consider a sequence ( f 1 ,t ) t ∈ N + with f 1 ,t = t -1 · y +(1 -t -1 ) · y ∞ . Wetrivially have that f 1 ,t → y ∞ for t →∞ , so that L ( f 1 ,t , y ) →∞ . Hence, once again, we have that the average loss diverges as ERR t ≥ w 1 L ( f 1 ,t , y ) → + ∞ as t → ∞ . It remains to show that the ensemble loss ERR is finite. We note that the ensemble prediction also defines a sequence ( ¯ f t ) t ∈ N + such that for all t ∈ N +

We deduce that ¯ f t ∈ Y as it is a convex combination of y and y ∞ . Furthermore, we trivially have that ¯ f t → ¯ f ∞ = (1 -w 1 ) · y + w 1 · y ∞ as t → ∞ . We shall now demonstrate that ¯ f ∞ ∈ int( Y ) and conclude. We note that, since y ∈ int( Y ) , there exists an open ball B [ y , r ] of radius r ∈ R + centered at y such that B [ y , r ] ⊆ Y . Let us now consider the open ball B [ ¯ f ∞ , (1 -w 1 ) · r ] . We note that any y ′ ∈ B [ ¯ f ∞ , (1 -w 1 ) · r ] is of the form

where v ∈ B [0 , r ] . Clearly, ( y + v ) ∈ B [ y , r ] ⊆ Y , hence y ′ ∈ Y as it is a convex combination of two elements of Y . We deduce that B [ f ∞ , (1 -w 1 ) · r ] ⊆ Y , which implies that f ∞ ∈ int( Y ) . We conclude that ERR t →L ( ¯ f ∞ , y ) by continuity of the loss on int( Y 2 ) . We also know that L ( ¯ f ∞ , y ) is finite as ( ¯ f ∞ , y ) ∈ int( Y 2 ) . We deduce that the augmented loss is not bounded from below when β > 1 as β · ERR t +(1 -β ) · ERR t →-∞ for t →∞ .

D Gradient Adjusted Targets

Throughout this work we have discussed the two extremes of training neural network ensembles. Independent training , where each ensemble member is trained independently and joint training where the predictions of the ensemble as a whole is optimized. We have also shown that interpolating between these two extremes via the parameter β can result in an effective hybridization. In this section, we provide an alternative interpretation of these intermediate values of β as an optimization objective. In particular, we show that the hybrid objective can in fact be interpreted as an independently trained ensemble in which the targets y are adjusted by the gradient of the ensemble.

Specifically, in the case of mean squared error we can adjust the true target y by a step in the opposite direction to the gradient. Then each individual learner f i will have its target adjusted to compensate for the errors of the full ensemble. The size of this step is determined by the parameter α . This results in the gradient adjusted target (GAT) objective in Equation (7). Note that throughout this section we consider scalar regression but, of course, these results would hold for multiple outputs by applying the same reasoning to each output.

Conveniently, it can be shown that applying individual training with this GAT objective is equivalent to using the hybrid objective described in equation 2 where the step size α has a similar effect to the interpolation parameter β .

Theorem D.1 (Residual adjusted mean squared error) . The ensemble gradient adjusted mean squared error objective

is proportional to ERR -(1 -1 ( α +1) 2 ) · DIV.

Proof.

When we take no gradient step at α = 0 , this is equivalent to β = 0 or independent training. As we take a larger gradient step we approach joint training noting that as α →∞ we find β → 1 . Should we choose to take a step of exactly the gradient itself (i.e. α = 1 ), it is equivalent to hybrid training with β = 0 . 75 .

̸

Alternatively, we might wish to only consider the M -1 other members of the ensemble in our gradient step g . In this case we might consider ¯ f -i = 1 M -1 ∑ M -1 j = i f j rather than ¯ f . Again, we find that this can be expressed in terms of ERR and DIV. We begin with a prerequisite in Lemma D.2 and then proceed in Theorem D.3 to show that there is also an exact equivalence in the M -1 case to augmented objective L β described in Section 5.

Lemma D.2.

Proof.

Theorem D.3 ( M -1 residual adjusted mean squared error) . The M -1 ensemble gradient adjusted mean squared error objective

̸

is equivalent to ERR -γ · DIV up to a scalar constant where γ = α ( α +1) 2 · M (( α +2) M -2( α +1)) ( M -1) 2 .

Proof.

̸

While the scalar γ might initially appear unwieldy, at closer inspection it performs a similar function to the full ensemble gradient equivalent in Proposition D.1. The key difference being that this term also accounts for the ensemble size M and ensures that smaller ensembles have a higher weighting on diversity. This relationship is visually depicted in Figure 8 where we again observe a sensible connection to hybrid training at various values of β .

0

20

40

60

80

100

Ensemble size

M

0

.

0

0

.

2

0

.

4

0

.

6

0

.

8

1

.

0

γ

α

= 1

α

= 0.75

α

= 0.5

α

= 0.25

α

= 0

̸

Investigating the relationship of β and diversity via GAT. In the regression experiments in Figure 2 one might intuitively have expected diversity to increase linearly with β . Instead, we observe a sharp increase in diversity as β → 1 . The GAT perspective described in this section may provide some insight into why this relationship takes this form. Recall that the β term from the augmented objective in Equation (2) is related to the α term the GAT objective in Theorem D.1 according to β = (1 -1 (1+ α ) 2 ) . We include a visualization of this relationship in Figure 9. This illustrates that the gradient step α grows exponentially as β → 1 resulting in each individual learner having their targets biased excessively (given that the targets are standardized in these experiments) to account for the ensemble errors. This exponential increase in the bias in the targets would appear to be related to the simultaneous increase in diversity at the same point.

0

.

0

0

.

2

0

.

4

0

.

6

0

.

8

1

.

0

β

0

5

10

15

20

25

30

α

E Gradient Analysis of Classification Ensemble Averaging Methods

On the claims of [31]. We begin by taking a closer look at the claims of [31] where, using their notation, they presented the following equation for an equally weighted ensemble using either output averaging or probability averaging.

where µ ( x 1 , · · · , x N ) = 1 N ∑ N i =1 x i represents the averaging of a set of N model outputs and ℓ is some loss function.

The authors claimed that, because this expression does not depend on i , the 'gradients backpropagated into all ensemble members are identical'. On this basis, they argued that this 'lack of gradient diversity' combined with numerical stability issues explains the poor performance of joint training. We wish to make two points on this claim. (1) Contrary to what is claimed, this constant gradient effect is only true in the case of score averaging and is not true for probability averaging which also performs poorly under joint training. (2) Even in the case of score averaging, although all ensemble members receive the same gradient signal, that signal is a function of the ensemble performance under joint training and the individual performance under independent training. Therefore the joint training optimal solution would still maximize ensemble performance if we could obtain the optimal solution .

We begin with the first point (1) . If each x i represents the scores of each ensemble member and we presume that the softmax activation ϕ is baked into ℓ , then this is the score-averaged classification setting, and the author's expression holds. However, if we wish to consider the probability averaged setting, then the averaging function should in fact be µ ( x 1 , · · · , x N ) = 1 N ∑ N i =1 ϕ ( x i ) and the derivative takes the following form.

It is clear that the final term in this expression is indeed a function of x i and therefore each learner can receive unique gradients in the case of probability averaging.

With respect to the second point (2) , we can simply note that the loss term ℓ is a function of just x i in independent training and all x 1 , · · · , x N in joint training. Concretely, we note that the gradient with respect to learner i should be expressed as ∂ℓ ( x i ) ∂ x i for independent training and as ∂ℓ ( µ ( x 1 , ··· , x N )) ∂ x i for joint training. This should clarify that although the learners could potentially have constant gradients in the case of joint training, they are still optimizing the correct objective and, therefore, the constant nature of these gradients would not in itself explain why the joint training objective results in poorer ensemble performance. We argue that a better explanation lies in the observation that the additional diversity term in the joint objective surprisingly results in poorer solutions is the observation that these multivariate entangled objectives result in an optimization problem that is significantly more challenging to solve in practice. In other words, individual training performs better as it results in a better solution to a slightly sub-optimal objective.

A gradient analysis of cross-entropy loss. Here we extend the previous gradient analysis by comparing the exact gradients obtained for independent training, probability-averaged joint training and score-averaged joint training in the case of cross-entropy loss. We use the following notation. f j,k is the pre-activation output, or score, for learner j of class k . ϕ is the softmax function with ϕ ( f j ) k referencing the the k 'th element of the softmax function applied to the vector f j . L is the loss function.

We summarize the objectives for the three cases in Table 2. The columns correspond to the training method (independent vs joint) paired with the output averaging method (probability vs score). The first row corresponds to a general loss function, while the others are the specific cases of crossentropy loss. The third row is simply re-expressing the second row by considering the true class k ⋆ such that y k ⋆ = 1 .

Now we evaluate the gradient of each training method and output averaging combination with respect to an arbitrary learner i for an arbitrary class k . Webegin with the case of independent training.

̸

Next we examine the equivalent gradients of the joint training objective with probability averaged predictions.

̸

Finally, we calculate the same gradients for the case of score averaging.

̸

From these gradients, we can draw the following conclusions for a given ensemble member f i .

Independent training - Each member receives a variable gradient that is a function of their own predictions but not the predictions of the rest of the ensemble. The gradient for learner f i takes the form g ( f i ) .

Joint training with probability averaging - Each member receives a variable gradient that is a function of their own predictions and the predictions of the other ensemble members. The gradient for learner f i takes the form g ( i, f 1 , · · · , f M ) .

Joint training with score averaging - Each member receives a constant gradient that is a function of their own predictions and the predictions of the other ensemble members. The gradient for learner f i takes the form g ( ¯ f ) .

F Learner Collusion in Regression

In this section, we provide an illustrative example of the learner collusion effect in the case of regression with mean squared error. We begin by recalling the relevant decomposition from Equation (1) for an equally weighted ensemble.

Without loss of generality, we consider each base learner to predict some ensemble prediction f ∈ R and a learner-dependent scalar adjustment c j ∈ R such that f j = f + c j . We also write 1 M ∑ j c j = ¯ c . It is straightforward to check that Equation (11) can then be expressed as

Setting aside f , we focus on the optimization of the c j 's. We note that in the setting of independent training where we only minimize ERR, the unique minimum is achieved when c j = 0 ∀ j . However, when we add the additional DIV term, this changes. When optimizing the joint objective, ERR, any solution satisfying ¯ c = y -f will minimize this loss. This is exactly learner collusion. We

note that as we optimize on training data, loss typically approaches zero, and ¯ c → 0 . Furthermore, if we were to require that the two terms ERR and DIV should be relatively, approximately equal, we note that this would only occur as the c j terms become large. Specifically, ERR ≈ DIV as | c j |→∞ . While this analysis illustrates how artificial diversity may emerge in the regression setting, we note that this is still not guaranteed to result in poor test performance in all cases. One such example is the Facebook dataset in Figure 2 where the test mean squared error is largely unaffected by this excessive diversity. In this case, it appears that almost the entire predictive is contained within a single feature resulting in no benefit from true diversity and, therefore, no substantial impact due to learner collusion

Finally, we highlight that during independent training, the learners are not aware of each other's predictions and, therefore, the premise that there would be a single shared ensemble prediction can not hold. In this case, all diversity will be due to external factors such as random initialization and improved performance over a single model can be explained by standard ensemble theory [e.g. 1, 2]. In the case of joint training, the premise that the ensemble produces a single prediction disguised as multiple predictions (i.e. f j = f + c j where 1 M ∑ j c j = 0 ) is plausible and even likely. However, if this is the case, then clearly there is no genuine diversity among the learners implying that overfitting and poor generalization may become more likely.

G Additional Results

G.1 Generalization Gap Loss Plots

This section provides an extended analysis of the experiments investigating the generalization consequences of learner collusion discussed in Section 6.2. Figure 10 provides the same generalization gap plots from Figure 5, but decomposed into the training and testing loss. For analysis purposes, we trained these models for 100 epochs, but this plot highlights that convergence occurs much earlier. These results reveal that the generalization gap is caused by both a slower training set convergence in addition to a greater test set loss. This slower convergence may in part be due to the two terms of the joint objective ( β = 1 ) competing against each other in the direction of their gradient updates. More concerning is the larger test set loss. This indicates that the solution achieved as training loss eventually becomes low, generalizes much more poorly than the solution achieved by the independent training objective ( β = 0 ).

0

50

100

0

.

00

0

.

25

0

.

50

0

.

75

1

.

00

CIFAR-10 (

β

= 0)

Train loss

Test loss

0

50

100

0

.

00

0

.

25

0

.

50

0

.

75

1

.

00

CIFAR-10 (

β

= 1)

0

50

100

0

1

2

3

CIFAR-100 (

β

= 0)

0

50

100

0

1

2

3

CIFAR-100 (

β

= 1)

Epoch

Loss

We also provide a decomposition of the loss into individual error ( ERR ) and diversity ( DIV ) on both the training set (Figure 11) and the test set (Figure 12). From the training plots, it appears that the joint objective ( β = 1 ) maximizes diversity early in training before beginning to minimize individual error later. This aligns with the learner collusion effect we describe in Section 6.1 as it is trivial to inflate diversity while minimizing the individual error requires solving the true objective of the task. The test set decomposition in Figure 12 reinforces that the solutions obtained with joint training generalize poorly beyond the training data.

0

50

100

0

.

0

0

.

5

1

.

0

1

.

5

2

.

0

CIFAR-10 (

β

= 0)

DIV

ERR

0

50

100

0

.

0

0

.

5

1

.

0

1

.

5

2

.

0

CIFAR-10 (

β

= 1)

0

50

100

0

1

2

3

4

5

CIFAR-100 (

β

= 0)

0

50

100

0

1

2

3

4

5

CIFAR-100 (

β

= 1)

Epoch

Value

0

50

100

0

1

2

3

CIFAR-10 (

β

= 0)

DIV

ERR

0

50

100

0

1

2

3

CIFAR-10 (

β

= 1)

0

50

100

0

1

2

3

4

5

CIFAR-100 (

β

= 0)

0

50

100

0

1

2

3

4

5

CIFAR-100 (

β

= 1)

Epoch

Value

G.2 Larger Ensembles

In Figure 13 we reproduce the experiments with ResNet-18 base learners from Figure 6 (left) but with a larger ensemble such that M = 10 . We observe that, while there is a small improvement in test accuracy, the shape of the curves with increasing β are consistent with those of the smaller ensemble.

0

.

0

0

.

2

0

.

4

0

.

6

0

.

8

1

.

0

0

.

93

0

.

94

0

.

95

CIFAR-10

ResNet-18

0

.

0

0

.

2

0

.

4

0

.

6

0

.

8

1

.

0

0

.

70

0

.

72

0

.

74

0

.

76

0

.

78

CIFAR-100

Level of diversity

β

G.3 Score Averaging

Figure 14 includes the results of repeating the experiments in the top row of Figure 6 using scoreaveraging rather than probability averaging as described in Section 4.2. Similarly to previous works, we find that joint training performs poorly in this setting too. We also note that test set performance is similar to that of probability averaging on both datasets.

G.4 CNN Ensembles

We repeat the experiments described in Section 6.3 with ensembles consisting of base learners with a vanilla convolutional neural network architecture. The architecture of each of these base learners is provided in Table 3. Dropout with probability 0 . 1 is applied throughout the network during training. Furthermore, we apply Adam optimizer [55] with learning rate 0.001 and early stopping

0

.

0

0

.

2

0

.

4

0

.

6

0

.

8

1

.

0

0

.

925

0

.

930

0

.

935

0

.

940

0

.

945

CIFAR-10

ResNet-18

0

.

0

0

.

2

0

.

4

0

.

6

0

.

8

1

.

0

0

.

70

0

.

72

0

.

74

0

.

76

CIFAR-100

Level of diversity

β

with patience of 5 epochs. Otherwise, all experimental parameters are consistent with those previous experiments. We apply this ensemble to CIFAR-10 with ensembles of size M = 5 and M = 10 with results provided in Figure 15. The results of this experiment are consistent with those of previous architectures and we again note the poor test set performance of the jointly trained model ( β = 1 ). Interestingly, we find in both cases that the lowest test loss is achieved at interpolating values of β as achieved by the augmented objective L β as described in Section 5.

0

.

0

0

.

2

0

.

4

0

.

6

0

.

8

1

.

0

0

.

68

0

.

70

0

.

72

M

= 5

CNN

0

.

0

0

.

2

0

.

4

0

.

6

0

.

8

1

.

0

0

.

68

0

.

70

0

.

72

M

= 10

0

.

0

0

.

2

0

.

4

0

.

6

0

.

8

1

.

0

0

.

85

0

.

90

0

.

95

1

.

00

CNN

0

.

0

0

.

2

0

.

4

0

.

6

0

.

8

1

.

0

0

.

85

0

.

90

0

.

95

1

.

00

Level of diversity

β

Negative log-liklihood

G.5 SVHN with VGG architecture

In this experiment we repeat the experiments in Section 6.3 on the SVHN dataset [58] using a VGG style architecture [14]. We report the exact architecture in Table 4. Again, we use ensembles of size 5. We use the same optimization protocol as applied to the ResNet-based ensembles as reported in Appendix B. Consistent with all previous experiments we observe that joint training again results in degenerate performance in this setting.

G.6 ERR as a metric

For completeness we report the ERR test set performance corresponding to the experiments in Figure 6 (left). We note that this is exactly the ensemble loss which is optimized at β = 1 . This is useful to report as it is the exact quantity being optimized during training. Consistent with all previous results, performance is worst as β → 1 (i.e. as we approach optimizing for the metric being reported).

0

.

0

0

.

2

0

.

4

0

.

6

0

.

8

1

.

0

0

.

9575

0

.

9600

0

.

9625

0

.

9650

0

.

9675

SVHN (accuracy)

VGG

0

.

0

0

.

2

0

.

4

0

.

6

0

.

8

1

.

0

0

.

14

0

.

16

0

.

18

Negative log-likelihood

SVHN (loss)

Level of diversity

β

G.7 Learner dropout as a resolution

In this section we provide a negative result in which we attempt to resolve learner collusion by randomly removing a subset of base learners from the ensemble at each batch during training. The hypothesis is that if we drop a sufficiently large portion of base learners but still perform joint training on the remainder, this should remove the ability to deterministically collude and, therefore, cause inflated diversity to be actively harmful even on the training data. One might hope that this would cause the ensemble to avoid this degeneracy. In what follows we provide the details of this experiment.

We repeat the setup on CIFAR-10 with ResNet-18 architecture as described earlier in the paper. We train each model using the joint objective, but at each batch we drop a proportion p ∈ [0 , 0 . 2 , 0 . 4 , 0 . 6] of randomly selected learners from the ensemble. We then investigate if this (a) reduces collusion and (b) improves ensemble performance. The results of this experiment are included in Figure 18. We find that dropping learners does indeed significantly reduce the diversity score indicating a reduction in learner collusion (although a reasonably large proportion of learners is required to be dropped). Unfortunately, the improvement in performance by reducing collusion is negated by a decrease in individual performance due to the base learners becoming weaker on average. The reason why is apparent once we notice that by dropping a base learner from some proportion of batches, this is exactly equivalent to bootstrapping. While bootstrapping has been effective for ensembles of weak learners (e.g. random forest) it has already been shown to be harmful for deep ensem-

0

.

0

0

.

2

0

.

4

0

.

6

0

.

8

1

.

0

0

.

150

0

.

175

0

.

200

0

.

225

0

.

250

CIFAR-10

ResNet-18

ResNet-34

0

.

0

0

.

2

0

.

4

0

.

6

0

.

8

1

.

0

0

.

8

1

.

0

1

.

2

1

.

4

CIFAR-100

Level of diversity

β

Negative log-liklihood

bles [12]. Unfortunately, this resolution appears to only reduce the effect of learner collusion by simultaneously harming the performance of the ensemble due to bootstrapping.

Proposing methods to overcome learner collusion is a valuable but non-trivial direction for future work that we and, we hope, others in the research community will pursue. We believe that the comprehensive diagnosis of the issue we have presented in this work will provide a foundation for future methodological progress.

0.0

0.2

0.4

0.6

Learner dropout rate

0.91

0.92

0.93

ERR

DIV

0.0

0.2

0.4

0.6

Learner dropout rate

0.1

0.2

0.3

0.4

Loss

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors aim to answer why joint training of ensembles does not work as well as separately training the members of the ensembles, before ensembling them. This is a well-known empirical fact, but the authors attempt to give a theoretical understanding of it, which is novel to my knowledge.\n\nTo reach their objective, they consider an expression for diversity (DIV) for probability averaging, which they motivate thru a (known) regression example. Then they generalize that expression to any twice differentiable loss function. \n\nThe authors study DIV in more depth by (upper) bounding it, then using it as a tunable regularization tool to interpolate between joint training and separate training of ensembles. \n\nThen the authors hypothesize that jointly training the ensemble may artificially bump up DIV without yielding proper generalization. They say that this happens because of \"learner collusion\" a phenomenon where two learners are displaced artificially by a constant bias term without changing the overall performance of the ensemble. They devise empirical measures to test that hypothesis (by studying generalization gaps, diversity explosion, learner codependence etc.)\n\nSTRENGTHS:\nS1: The paper is really well written. Thanks to the authors for building my reading experience so well.\n\nS2: The explanation of the phenomenon (to my knowledge) is novel, and gives me a new way of thinking about the well-known failure case of jointly training ensembles. \n\nS3: It is great that the authors have theory that is paralleled with experiments on real-world data.\n\nWEAKNESSES:\nW1: Not sure I understand what happens when we vary $\\beta$. I think the paper can improve in a more detailed discussion about a more thorough study of the effect of collusion in the intermediate values of $\\beta$. See my Q2 in the questions below.\n\nW2: Typically, we expect to be able to devise better models after we have a better understanding of an underside phenomenon. The paper lacks a discussion about this. It's fine to leave it as future work, but it would be useful to discuss some directions a bit.\n\nQUESTIONS:\nQ1: In Figure 2, why does Facebook seem to be different from the overall trend? \n\nQ2: If we get $\\beta \\approx 1$, then great, we have increased diversity and this gives us some hints as to why joint training might fail. However, as we vary $\\beta$, I do not gain any intuition. Is the collusion effect a monotonic one? It seems so from Figure 6. If there is some $\\beta > 0$ then do we always have that effect? Shouldn't intuition tell us that there should be a \"sweet spot\" for $\\beta$? If not, then why would we be interested in training learners jointly to any extent? For example, could we do something more explicit to prevent the learner's collusion?\n\nLIMITATIONS:\nThe authors have adequately addressed the limitations.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-06-22"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nEnsembles are a simple but powerful way to improve model performance. Typically, ensembles are used to improve performance by training each model independently and then using them jointly. However, unlike previous ML methods that require ensemble members to be trained individually, Deep Ensemble, an ensemble of deep learning models, can also be trained jointly. Considering that joint performance is the true objective of Deep Ensemble, it seems more natural to train jointly rather than individually. However, in practice, joint training leads to poor performance and poor generalization. The authors show theoretically and empirically that this is due to ensemble diversity. From this, they propose an augmented objective $\\mathcal{L}^\\beta$ that allows for a linear interpolation between joint and independent training. Finally, they hypothesize that this is due to learner collusion, a phenomenon where each model has a bias when jointly training an ensemble, and show through experiments that this is indeed the case.\n\nSTRENGTHS:\n- The notation is well organized and the formulas are easy to follow.\n- The problems covered in previous studies are well summarized and theoretically explained in Sections 2, 4, and 5.\n- The reason why joint training of Deep Ensemble fails is effectively explained through *learner collusion* and is well supported experimentally.\n\nWEAKNESSES:\n- The authors do not propose a way to prevent learner collusion while doing joint training, so there is not much contribution in terms of practicality. The possibility of partial joint training by weighting between independent training and joint training using an augmented objective is presented as a future work, which is already analyzed in various ways in Webb et al. (2020).\n- (Minor) The overall formatting of the paper is less polished and a bit difficult to read.\n    - Figures are scattered throughout the page. If possible, the figure should be located at the top of the page.\n    - The text in the figure is too small to read the results. Increase the size of the text so that it is not too different from the size of the main text.\n    - The lack of titles in the references makes it very difficult to identify the relevant papers. **Please fix this.**\n-----\n(Webb et al., 2020) [To Ensemble or Not Ensemble: When does End-To-End Training Fail?](https://arxiv.org/abs/1902.04422)\n\nQUESTIONS:\n- Do you think it would be beneficial to drop a different fraction of base learners each step during the joint training of the ensemble to avoid learner codependency?\n- Minor comments:\n  - As mentioned above, the formatting of the reference is incorrect. At least the title of the paper should be visible.\n  - NeurIPS style rules do not allow vertical lines in tables. I would recommend removing the background color as well.\n  - Overall, I like the paper, but you spend too much space organizing and formalizing previous research; I think it would be better to add the experiments in the appendix to the main text.\n  - In Figure 6, it looks better to add the ImageNet title to the two graphs on the right for consistency.\n\nLIMITATIONS:\nI see no potential negative societal impact from this paper.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-04"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper explores the joint training of deep ensembles, wherein the ensemble error is directly optimized during training. The authors find that joint training leads to poor performance, which they posit it due to phenomenon they call \"learner collusion\", where base learners artificially inflate their diversity, at the expense of test performance. The authors provide a new decomposition of a broad class of loss functions in terms of the average error rate, the ensemble error rate, and a generalized notion of diversity. The authors then perform an experimental evaluation into the prevalence of learner collusion during joint training, using an augmented objective function to probe the effect of explicitly encouraging diversity during training.\n\nSTRENGTHS:\nOverall, the paper is very well-written and easy to follow, and in my view the topic is very relevant to the community. \n\nTo me, the most compelling experimental evidence for the learner collusion phenomenon are those presented under \"learner codependence\"/in Figure 4. From these results it is very clear that something distinct is happening in the joint training regime, and that the individual models are exploitating the joint loss. The definition of the augmented objective $\\mathcal{L}^\\beta$ seems like a very useful tool for probing these and other related phenomena.\n\nWEAKNESSES:\nUnless I am misunderstanding, the conclusions drawn from Figure 2 do not seem to strongly support the claim that joint training significantly harms ensemble performance. Indeed, for the Facebook task, it seems the method actually helps, while for others it does not. This is in contrast to the CIFAR results in Table 1/Figure 6, where the degredation in performance is significant. Could the authors provide some clarification as the intended conclusion from these experiments?\n\nThere are a number of other papers in the literature that study some form of the gap (in the authors' notation) $\\bar{\\text{ERR}} - \\text{ERR}$, and express and/or bound it in terms of various forms of \"diversity\", depending on the given loss function. For example Ortega et al., 2022, Abe et al., 2022 and Masegosa et al., 2020 give explicit expressions for the average - ensemble error gap in terms of diversity metrics that are easily interpretable. It's unclear to me that the form in Theorem 4.5 adds much to this literature, and indeed the stated expression for diversity doesn't seem to be used in the remainder of the paper.\n\nOverall, while I think the paper is well-written and the topic is relevant, I think there is 1) insufficient use/novelty in the theoretical results and 2) somewhat incomplete conclusions to be drawn from the empirical results to recommend accepting at the the current stage. In particular, while it certainly appears like some type of \"learner collusion\" is happening, it seems very important to understand why this only appears at $\\beta=1$, and exactly what impact this could have on the ensemble error.   \n\nAs a minor point, there seems to be an issue with the references -- they don't seem to contain any titles of the referred papers. \n\n**References:**\n\nTaiga Abe, E Kelly Buchanan, Geoff Pleiss, and John Patrick Cunningham. The best deep\nensembles sacrifice predictive diversity, 2022.\n\nLuis A. Ortega, Rafael Cabañas, and Andres Masegosa. Diversity and generalization in neural\nnetwork ensembles, 2022.\n\nAndres Masegosa, Stephan Lorenzen, Christian Igel, and Yevgeny Seldin. Second order PAC-\nBayesian bounds for the weighted majority vote, 2020.\n\nQUESTIONS:\nI find the distinct change in behavior of the various metrics at $\\beta=1$ very interesting; it seems the behavior is relatively benign for $\\beta < 1$, and yet there appears to be a distinct transition at $\\beta = 1$. Do the authors have any hypotheses as to why this occurs?\n\nLIMITATIONS:\nN/A", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-06"}

comments: SUMMARY:
This paper mainly studies the reason behind the failure of jointly training deep ensembles. It discovers that joint optimization results in a phenomenon in which base learners collude to artificially inflate their apparent diversity. Both theoretical and empirical evidence are provided further to verify the hypothesis.

STRENGTHS:
(1) This paper attempts to study a seemingly under-explored question, that is, why joint training of ensembles fails to generalize better than individual training and ensemble. This research topic is very important and helps us to understand the foundations of the deep ensemble.  

(2) Both theoretical proof and empirical proof are provided.

WEAKNESSES:
(1) One big issue of this paper is that all references do not contain titles, which to be honest, I have never seen in top-tier conferences like NeurIPS. 

(2) I do not understand why joint training is easier to train but results in poor generalization. What does "easier to train" refer to? 

(3) What joint training approaches are used in the paper? 

(4) The empirical evaluation isn't very convincing to me. For instance, how many ensemble members are we using for "Learner codependence"? It is not surprising that the performance degrades to me if we drop ensemble members during testing, especially if our overall ensemble members is not sufficient.

QUESTIONS:
Please see my above weaknesses.

LIMITATIONS:
N/a
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 2
DATE: 2023-07-07