Title: Scalable Ensemble Diversification for OOD Generalization and Detection
Venue: NeurIPS 2024
Accepted: no
Keywords: diverse

ensemble

scalable

robustness

uncertainty

OOD detection

OOD generalization
Abstract: Training a diverse ensemble of models has several practical application scenarios, such as model selection for out-of-distribution (OOD) generalization and the detection of OOD samples via Bayesian principles. Previous approaches to diverse ensemble training have relied on the framework of letting the models make the correct predictions for the given in-distribution (ID) data while letting them come up with different hypotheses for the OOD data. As such, they require well-separated ID and OOD datasets to ensure a performant and diverse ensemble and have only been verified in smaller-scale lab environments where such a separation is readily available. In this work, we propose a framework, Scalable Ensemble Diversification (SED), for scaling up existing diversification methods to large-scale datasets and tasks (e.g. ImageNet), where the ID-OOD separation may not be available. SED automatically identifies OOD samples within the large-scale ID dataset on the fly and encourages the ensemble to make diverse hypotheses on them. To make SED more suitable for large-scale applications, we propose an algorithm to speed up the expensive pairwise disagreement computation. We verify the resulting diversification of the ensemble on ImageNet and demonstrate the benefit of diversification on the OOD generalization and OOD detection tasks. In particular, for OOD detection, we propose a novel uncertainty score estimator based on the diversity of ensemble hypotheses, which lets SED surpass all the considered baselines in OOD detection task. Code will be available soon.

Abstract

Training a diverse ensemble of models has several practical application scenarios, such as model selection for out-of-distribution (OOD) generalization and the detection of OOD samples via Bayesian principles. Previous approaches to diverse ensemble training have relied on the framework of letting the models make the correct predictions for the given in-distribution (ID) data while letting them come up with different hypotheses for the OOD data. As such, they require well-separated ID and OOD datasets to ensure a performant and diverse ensemble and have only been verified in smaller-scale lab environments where such a separation is readily available. In this work, we propose a framework, Scalable Ensemble Diversification (SED), for scaling up existing diversification methods to large-scale datasets and tasks (e.g. ImageNet), where the ID-OOD separation may not be available. SED automatically identifies OOD samples within the large-scale ID dataset on the fly and encourages the ensemble to make diverse hypotheses on them. To make SED more suitable for large-scale applications, we propose an algorithm to speed up the expensive pairwise disagreement computation. We verify the resulting diversification of the ensemble on ImageNet and demonstrate the benefit of diversification on the OOD generalization and OOD detection tasks. In particular, for OOD detection, we propose a novel uncertainty score estimator based on the diversity of ensemble hypotheses, which lets SED surpass all the considered baselines in OOD detection task. Code will be available soon.

1 Introduction 21

Training a diverse ensemble of models is useful in multiple applications. Diverse ensembles are used 22

to enhance out-of-distribution (OOD) generalization, where strong spurious features learned from 23

the in-distribution (ID) training data hinder generalization [30, 31, 28, 23]. By learning multiple 24

hypotheses, the ensemble is given a chance to learn causal features that are otherwise overshadowed 25

by the prominent spurious features [39, 4]. In Bayesian machine learning, diversification of the 26

27

posterior samples has been studied as a means to improve the precision and efficiency of sample

uncertainty estimates [5, 37]. 28

A common strategy to train a diverse ensemble is to introduce two objectives: one for the main 29

task and one for diversification [29, 5, 28, 23]. The main task loss, such as the cross-entropy loss 30

for classification, encourages the hypotheses to solve the task on the labeled ID training set. The 31

diversification loss encourages the hypotheses to diversify the responses on an unlabelled OOD 32

dataset [28, 23] (Figure 1). The datasets for the objectives are separated to avoid contradictory 33

objectives: prediction diversification on the ID set will encourage wrong answers if there is only one 34

correct label. 35

This strategy, however, requires a separate OOD dataset where the hypotheses may make diverse 36 predictions without harming the main task performance on the ID training samples. Previous work 37 has thus been tested on hypothetical lab settings where the spurious and causal features can easily be 38 controlled to secure separate ID and OOD datasets for diverse ensemble training. It is not clear yet 39

how one could diversify an ensemble of models for realistic, uncontrolled, and large-scale applications 40

(e.g. ImageNet scale) where collecting a separate OOD dataset can be very costly, if not impossible. 41

To address the scalability challenge, 42 we propose a novel diversification 43 framework, Scalable Ensemble Diver44 sification (SED, Figure 1). We intro45 duce three ingredients. (1) OOD sam46 ples are dynamically selected from the 47 ID training samples, on which the mod48 els are trained to make different predic49 tions. (2) At each iteration, a subset of 50 model pairs are stochastically selected 51 to construct the disagreement objec52 tive, rather than the full list of model 53 pairs. (3) Deep networks are trained 54 to diversify only a few layers at the 55 end, rather than the full networks. This 56 framework allows scaling up existing 57 ensemble diversification methods. In 58 this work, we focus on scaling up the 59 Agree to Disagree (A2D) method [28]. 60

61

62

63

64

65

66

67

68

69

70

71

72

73

74

75

76

77

78

79

80

Existing work assumes existence of ID and OOD datasets.

Our Scalable Ensemble Diversification (SED) only requires a single ID dataset.

Diverse model ensemble

Classify

2 datasets

training objectives

Disagree

ID dataset

OOD dataset

Diverse model ensemble

1 dataset

training objectives

Classify

Disagree

Identify OOD samples

ID dataset

OOD samples

We verify that SEDdiversifies a model ensemble trained on ImageNet. We demonstrate the benefit of diversification on OOD generalization and OOD

detection tasks. For the former, we showcase the usage of SED-diversified ensemble in three variants: (a) vanilla ensemble of prediction probabilities [22], (b) an average of the model weights through model soup [38], and (c) the oracle selection of the individual models for each OOD test set [23, 30]. In all three cases, SEDachieves a superior generalization to OOD datasets like ImageNet-A/R/C, OpenImages, and iNaturalist.

For OOD detection, we seek multiple ways to use the SED-diversified ensemble: (a) treating them as samples of the Bayesian posterior and (b) using our novel OODness estimate of Predictive Diversity Score (PDS) that measures the diversity of predictions from an ensemble. We show that PDS provides a superior detection of OOD samples like ImageNet-A/R/C, OpenImages, and iNaturalist.

Our contributions are

Scalable Ensemble Diversification (SED) framework that scales up existing ensemble methods;

Predictive Diversity Score (PDS) that computes the OODness score for samples based on ensemble prediction diversity;

First demonstration of the ensemble diversification and its application to OOD generalization and detection at ImageNet level.

The code will be released with the next versions of the manuscript. 81

2 Related work 82

In this section, we give a short overview of ensembling methods. At first, we speak about ensembles 83

84

in general and the role of diversity in them (§ 2.1), then we focus on ensembling methods for neural

networks and separate them into two big groups. The first group includes algorithms that use loss 85

regularizers (§ 2.2) and the second group covers works that do not modify the training loss (§ 2.3). 86

87

88

89

90

91

92

93

94

95

96

97

98

99

100

101

102

103

104

105

106

107

108

109

110

111

112

113

114

115

116

117

118

119

120

121

122

123

124

125

126

127

128

129

130

131

132

133

2.1 Ensembles as a technique

Ensembling is a powerful technique of aggregating the outputs of multiple models to make more accurate predictions and it has been around for decades [12, 21, 18, 2, 3]. It is well known that diversity in ensemble members' outputs leads to better performance of the ensemble compared to the performance of a single model [21] because ensemble members make independent errors [12, 11]. Therefore, one way to reduce DNNs' reliance on spurious correlations is to train multiple models on the same task and make them diverse in terms of errors they make so that their ensemble is less dependent on such correlations.

2.2 Neural network ensembles that promote diversity through loss regularizers

Diversity in models can be induced by supplying training loss with a suitable regularizer.

Such regularizers can diversify models' weights [5, 7, 34, 6], features [39, 4], input gradients [29, 30, 31, 33] and outputs [25, 5, 28, 23].

Notably, in [5] authors showed that regularizer of a certain structure that repulses ensemble members' weights or outputs leads to ensembles that provide a better approximation of Bayesian Model Averaging. This idea was later extended by works that repulse ensemble members' features [39] and input gradients [33].

Since the ensemble performs better due to the diversity of errors that ensemble members make [21] we want those members to give pairwise different outputs for the same inputs. Unfortunately, diversity in weights space, input gradient space, or features space does not guarantee such property without additional assumptions due to functional symmetry which means that models can be different in terms of their weights or feature maps and input gradients they produce but still give the same outputs for a given input. That is why we are focused on methods that diversify models' outputs, specifically [28, 23] which are state-of-the-art according to [1] and use regularizer of repulsive nature conceptually similar to [5].

2.3 Neural network ensembles that promote diversity without modifying loss

In addition to loss regularizers, there were an uncountable number of different ways to induce diversity in ensembles of neural networks that did not modify the training loss. The most straightforward approach of independently training multiple models of the same architecture by changing only random seeds is called Deep Ensemble [22] which was extended from the Bayesian perspective in [37]. Another solution is to construct an ensemble from models trained with different hyperparameters [36], augmentations [24], or architectures [40]. More computationally efficient direction allows training only one base model inducing diversity by ensembling either checkpoints saved in different local minima along the training trajectory of this base model [19] or models produced by the base model after applying dropout [10] or masking [9] to it. The mixture of experts paradigm can also be viewed as an ensemble diversification technique [41] where diversification happens due to assigning different training samples to different ensemble members.

Despite their conceptual simplicity Deep Ensembles [22] and ensembles of models trained with different hyperparameters [36] are strong baselines for OOD detection [27] and OOD generalization tasks, especially when combined with model souping techniques [38]. That is why we selected them as baselines for our experiments.

3 Method

We present our main technical contributions, Scalable Ensemble Diversification (SED, §3.2) and the Predictive Diversity Score (PDS, §3.3).

3.1 Preliminaries

We cover background materials before introducing our main technical contributions. We work with a training set D := { x n , y n } N n =1 , which we refer to as the in-distribution (ID) dataset. For prior diversification methods, we also assume the existence of a separate, unlabeled out-of-distribution

134

135

136

137

138

139

140

141

142

143

144

145

146

147

148

149

150

151

152

153

154

155

156

157

158

159

160

161

162

163

164

165

166

167

168

169

170

171

172

173

174

175

176

(OOD) dataset D ood := { x ood n } N ood n =1 . We write f ( · , θ ) for a deep neural network classifier parametrized by θ . f ( x ; θ ) ∈ R C indicates the logit outputs for C classes for input x . We write p ( x ) := Softmax( f ( x )) ∈ [0 , 1] C for the probability outputs. We consider an ensemble { f 1 , · · · , f M } of M models.

3.1.1 Existing ensemble diversification approach

We introduce an existing approach for diversifying an ensemble of models [28, 23]. Two objectives are imposed upon the ensemble of models: the main task loss and the diversification regularization.

For the main task, the community has focused on the classification task. The cross-entropy loss -log p y ( x ; θ ) is used to train the model ensemble { f 1 , · · · , f M } on the ID dataset D :

This encourages each member of the ensemble to behave similarly on the ID dataset.

Different diversification schemes use different diversification regularization loss L div applied on pairs ( f m , f l ) of ensemble members. The diversification objective is commonly optimized on the OOD dataset D ood to encourage the training of multiple hypotheses on the OOD samples while avoiding clashes with the main task objective. In this work, we focus on the Agree to Disagree [28] method. The diversification loss for a pair ( p m , p l ) is defined as:

where ˆ y := arg max c p c m ( x ) is the predicted class for the first model p m . One may symmetrically define ˆ y to be the prediction for the second model p l ; in practice, it does not make a difference [28]. Note that the diversification loss favors p l to predict a lower likelihood for the prediction by p m , p l ˆ y ( x ) , and vice versa. For M models in an ensemble, A2D is applied on the OOD dataset D ood for every pair of models ( p m , p l ) :

3.2 Scalable Ensemble Diversification (SED)

We present Scalable Ensemble Diversification (SED) that addresses the limitation of the existing ensemble diversification framework that requires a separate OOD dataset. We introduce two main components of SED: dynamic selection of OOD samples within the ID dataset (§3.2.1) and the stochastic selection of pairs to diverge in the optimization iterations (§3.2.2).

3.2.1 Dynamic selection of OOD samples

If only the ID training dataset is present, it is difficult to induce diversity in ensemble members, as they are uniformly incentivized to solve the main task objective: given x , predict y . Hence, previous approaches have introduced a qualitatively disjoint unlabeled set, which we refer to as the OOD dataset, where the ensemble members are encouraged to disagree with each other. The clear separation of ID and OOD datasets for the two objectives matters for ensuring a good balance between the main task performance and the diversity of hypotheses.

Previous works like Pagliardini et al. [28], Lee et al. [23] have performed experiments on small-scale datasets where factors are well-controlled and clean versions of OOD datasets are readily available. Examples include Waterbirds, Camelyon17, CelebA, MultiNLI, C-MNIST, and the Office-Home datasets. For example, for Waterbirds, the ID dataset is set as the cases where the bird's habitat matches with the visual background and the OOD dataset corresponds to the complementary case.

While conceptually desirable, collecting a separate OOD dataset can be highly cumbersome and expensive. For a large-scale dataset like ImageNet, it is highly non-obvious how one could build a corresponding OOD dataset where the underlying feature-label correlations are different from the ID training dataset.

To address this challenge, we consider dynamically identifying an OOD subset of the ID dataset and letting the ensemble diverge on this subset. The desiderata for the identification of OOD samples

within the ID dataset are twofold: (a) we wish to discriminate samples where the ensemble members 177 make mistakes and (b) we only trust the ensemble prediction for the OOD sample identification when 178 the ensemble is sufficiently trained. 179

We define the sample-wise weight α n on each ID sample ( x n , y n ) ∈ D that satisfy the two conditions: 180

where CE ( f 1 , · · · , f M ; x n , y n ) := CE ( 1 M ∑ m f m ( x n ) , y n ) is the loss on the logit-averaged prediction and B is a minibatch that contains the sample ( x n , y n ) . α n is a weight proportional to the ensemble loss on the sample; we thus meet the condition (a). The normalization is designed to handle the condition (b). To see this, consider the batch-wise weight

Note that α B is now inversely proportional to the average cross-entropy loss of the ensemble on the batch B . Thus, the overall level of α n for n ∈ B is lower for earlier iterations of the ensemble training, where the predictions from the models are not trustworthy yet.

With this definition of sample-wise weight α n for the diversification objective, we define the SED objective with the A2D loss for the diversification kernel:

where λ > 0 controls the overall weight of the diversification term. Note that, compared to Equation 3, this formulation does not rely on the OOD dataset D ood . Instead, all ID samples are treated as potential OOD samples, where their OODness is softly determined via α n . This enables a seamless adaptation of existing ensemble diversification methods to a relaxed setting where a separate OOD dataset is unavailable.

3.2.2 Further tricks for scalability

Ensemble diversification algorithms are often based on pairwise similarities of the members. Pairwise similarity computation scales quadratically with the size of the ensemble M . The second term of Equation 6 is an example of this. This is potentially a hurdle when ensemble diversification is to be applied to M ≥ 10 , and the data and parameter sizes are in the order of millions (e.g. ImageNet).

We address this computational challenge by computing the summation of pairwise distances as a stochastic sum. For every minibatch B of SGD iterations, we uniformly-iid sample a subset I of { 1 , · · · , M } to compute the diversification term in Equation 6. The procedure is illustrated in the figure on the right.

To further speed up the SED training, we consider diversifying only a subset of layers, while freezing the other layers. In our experiments, ensemble members share the same frozen feature extractor of Deit3b [32] pretrained on ImageNet-21k [8] and we diversify only the last two layers of the models.

3.3 Predictive Diversity Score (PDS) for OOD Detection

We demonstrate several benefits of the diversified ensembles in §4. One of them is the possibility of using them for detecting OOD samples through the notion of epistemic uncertainty [13]. Given an ensemble of models, a simple baseline for OOD detection is to compute the predictive uncertainty of the Bayesian Model Averaging (BMA) by treating the ensemble members as samples of the posterior p ( θ |D ) [22, 37]:

181

182

183

184

185

186

187

188

189

190

191

192

193

194

195

196

197

198

199

200

201

202

203

204

205

206

207

208

209

210

211

212

213

214

215

216

217

Model 1

Iteration K:

randomly select 1 and 3

Input

Forward pass

Model 2

Model 3

Model 1

randomly select 1 and 2

Model 2

Model 3

This notion of epistemic uncertainty does not directly exploit the potential diversity in individual 218 models of the ensemble because it averages out the predictions along the model index m . 219

We propose a novel measure for epistemic uncertainty, Predictive Diversity Score (PDS), that directly 220 measures the prediction diversity of the individual members. The formulation is given below: 221

PDS is a continuous relaxation of the number of unique argmax predictions within an ensemble of models. To see this, consider the special case where p m ∈ { 0 , 1 } are one-hot vectors. Then, max m p c m ( x ) is 1 if any of m predicts c and 0 otherwise. Thus, ∑ c max m p c m ( x ) computes the number of classes that at least one of the ensemble members predicts. We show that, with our diverse ensembles, PDS outperforms the DE baseline for the OOD detection task (§4.4).

4 Experiments

We verify our contributions, Scalable Ensemble Diversification (SED, §3.2) and Predictive Diversity Score (PDS, §3.3), on ImageNet-scale tasks and datasets. We first verify that SED diversifies the ensemble (§4.2). Then, we demonstrate the application of diversified ensemble to OOD generalization (§4.3) and OOD detection (§4.4) tasks.

4.1 Experimental setup

We task the ensemble with the OOD generalization and OOD detection tasks.

Training settings. For both tasks, we train an ensemble of models with the SED framework with the A2D [28] diversity regularization using AdamW optimizer [26]. We use the default settings of a batch size of 16 , learning rate 10 -3 , weight decay 0 . 01 , and the number of epochs 10 . The overall diversity weight λ is set to 0 . 1 and the stochastic pairing is done for |I| = 2 models for each SGD batch. We use Deit3b [32] network pretrained on ImageNet21k [8] for all the experiments. Following the speed-up trick in §3.2.2, we use only the last 2 layers of the network. For the in-distribution (ID) dataset where the ensemble is trained to diversify, we use the training split of ImageNet with |D| = 1 , 281 , 167 . All experiments were ran on RTX2080Ti GPUs with 12GB vRAM and 40GB RAM, each experiment took from 2 to 12 hours depending on the complexity of the training.

Baselines. For naive ensemble training, we consider the deep ensemble [22] where each ensemble member independently with different random seeds that control the weight initialization and SGD batch shuffling. To match the resource usage of our SED, where we diversify only the last 2 layers of the network, we consider the shallow ensemble variant, which is the deep ensemble where only the last 2 layers are trained. We further consider a viable diversification scheme that performs deep ensemble with varying hyperparameters [36]. In addition to that, we reimplement A2D [28] and DivDis [23] algorithms and apply them without stochastic model sampling to do classification on labeled samples from ImageNet-Train and disagreement on unlabeled samples from ImageNet-R. For A2D we use frozen feature extractor and a parallel variant of their method which means that all ensemble members are trained simultaneously and not sequentially. The computational complexity of both these approaches scales quadratically with ensemble size which is why they are called Naive A2D and Naive DivDis respectively.

Evaluation benchmarks. The generalization performances of the ImageNet-trained ensembles are measured on multiple test datasets, ranging from the in-distribution validation split of ImageNet with 50,000 samples to OOD datasets like ImageNet-A ( A [17], 7.5k images & 200 classes), ImageNet-R ( A [16], 30k images, 200 classes), ImageNet-C ( C-i for corruption strength i [14], 50k images, 1k classes). OpenImages-O ( OI [35], 17k images, unlabeled), and iNaturalist ( iNat [20], 10k images, unlabeled). For OOD detection, we task the ensemble with the detection of the above OOD datasets against the ImageNet validation split.

Evaluation metrics. For OOD generalization, we use the accuracy. For OOD detection, we use the area under the ROC curve, following [15].

222

223

224

225

226

227

228

229

230

231

232

233

234

235

236

237

238

239

240

241

242

243

244

245

246

247

248

249

250

251

252

253

254

255

256

257

258

259

260

261

262

263

264

265

266

267

268

269

270

271

272

273

274

275

276

277

278

279

280

281

282

283

284

285

286

287

288

289

290

291

GT

Cowboy hat

Sea lion

Scuba diver

Great shark

Weimaraner

SED

Cowboy hat

Sea lion

Scuba diver

Great shark

Weimaraner

Comic book

Otter

Jellyfish

Killer whale

Vizsla

PDS

0.300

0.300

0.294

0.292

0.292

GT

Pomegranate

Zebra

Pomegranate

Pomegranate

SED

Pomegranate

Zebra

Pomegranate

Pomegranate

PDS

0.216

0.216

0.216

0.216

0.216

LeneDauaaard

4.2 Diversification

We start with the question of whether Scalable Ensemble Diversification (SED) truly diversify the ensemble at the ImageNet scale. To measure the diversity of the ensemble, we compute the number of unique predictions for each sample for the committee of models ( # unique).

Table 1 shows the # unique values for the IN-Val as well as multiple OOD datasets. We observe that the deep ensemble baseline does not increase the diversity dramatically (e.g. 1.09 for C-1) beyond no-diversity values (1.0). Diversification tricks like hyperparameter diversification (1.11 for C-1) or Naive A2D (1.04 for C-1) and DivDis (1.04 for C-1) do not improve the prediction diversity dramatically. On the other hand, our SED increases the prediction diversity across the board (e.g. 5.00 for C-1).

Qualitative results on ImageNet-R further verify the ability of SED to diversify the ensemble (Figure 2). As a measure for diversity, we use the Predictive Diversity Score (PDS) in §3.3. We observe that the samples inducing the highest diversity (high PDS scores) are indeed ambiguous: for the first image, where the 'cowboy hat' is the ground truth category, we observe that 'comic book' is also a valid label for the image style. On the other hand, samples with low PDS exhibit clearer image-to-category relationship.

4.3 OODGeneralization

We examine the first application of diversified ensembles: OOD generalization. We hypothesize that the superior diversification ability verified in §4.2 leads to greater OOD generalization due to the consideration of more robust hypotheses that do not rely on obvious spurious correlations.

Ensemble aggregation for OOD generalization. As a means to exploit such robust hypotheses, we consider 3 aggregation strategies. (1) Oracle selection : the best-performing individual model is chosen from an ensemble [28, 30]. Final prediction is given by f ( x ; θ m ⋆ ) where

292

293

294

295

296

297

298

299

300

301

302

303

304

305

306

307

308

309

310

311

312

313

314

315

316

317

318

319

320

321

322

323

324

325

326

327

328

329

m ⋆ := arg max m Acc( f m , D ood ) . (2) Prediction ensemble is a vanilla prediction ensemble where the logit values are averaged: 1 M ∑ m f m ( x ) [38]. (3) Uniform soup [38] averages the weights themselves. Final prediction is given by f ( x ; 1 M ∑ m θ m ) .

SED improves OOD generalization for ensembles. We show the OOD generalization performances of ensembles in Table 2, for the three ensemble prediction aggregation strategies described above. We observe that our SED framework (SED-A2D) results in superior OOD generalization performances for all three strategies. SED-A2D is particularly strong in prediction ensemble (e.g. 48.1% for M = 5 and 54.4% for M = 50 on ImageNet-R) and uniform soup (e.g. 46.1% for M = 5 and 46.5% for M = 50 on ImageNet-R). We contend that the increased ensemble diversity contributes to the improvements in OOD generalization. We also remark that the SED framework (SED-A2D) envelops the performance of Naive A2D in this ImageNet-scale experiment. Together with the superiority of computational efficiency (as discussed at the end of § 4.4) of SED-A2D over the Naive A2D, this demonstrates that SED fulfills its purpose of scaling up ensemble diversification methods like A2D.

Deep ensemble is a strong baseline. We also note that deep ensemble, particularly with diverse hyperparameters, provides a strong baseline, outperforming dedicated diversification methodologies under the oracle selection strategy when M = 5 . It also provides a good balance between ID (ImageNet validation split) and OOD generalization.

4.4 OODDetection

We study the impact of ensemble diversification on OOD detection capabilities of an ensemble. Once an ensemble is trained, we compute the epistemic uncertainty, or likelihood of the sample being OOD, following two schemes, η BMA and η PDS introduced in §3.3.

SED and PDS together lead to superior OOD detection performances. We show the OOD detection results in Table 3. For the BMA scores, deep ensemble remains a strong baseline. In particular, when the hyperparameters are varied ('+Diverse HPs'), the detection AUROC reaches the maximal performances among the ensembles using the BMA scores. The quality of PDS is more sensitive to the ensemble diversity, as seen in the jump from the deep ensemble (e.g. 0.589 for OI) to the diverse-HP variant (0.889). However, when the ensemble

is sufficiently diverse, such as when trained 330 with SED-A2D, the PDS leads to high-quality OODness scores. SED-A2D with PDS achieves the 331 best AUROC across the board, including the BMA variants. 332

Impact of diversification parameter λ . We further study the impact of ensemble diversification on the OOD detection with the PDS estimator. In Figure 3, we observe that strengthening the diversification objective (higher λ ) indeed leads to greater diversity (higher PDS), with a jump at around λ ∈ [10 -1 , 10 1 ] . This range corresponds to the jump in the OOD detection performance (higher AUROC).

Influence of ensemble size. How ensemble size influences performance of our method? We can see that increasing ensemble size helps to improve AUROC for OOD detection on C-1 (Figure 4). Increasing ensemble size marginally helps, but using 5 models provides already a significant improvement over the smallest possible ensemble of size 2. It is also important to mention, that SED framework is computationally more efficient w.r.t. ensemble size M than Naive

A2D and Naive DivDis: since we train ensembles for the fixed number of epochs, training complexity for SED is O (1) thanks to stochastic model pairs selection, while for Naive A2D and Naive DivDis it is O ( M 2 ) .

5 Conclusion

Ensemble diversification has many implications for treating one of the ultimate goals of machine learning, handling out-of-distribution (OOD) samples. By training a large number of plausible hypotheses on an in-distribution (ID) dataset, an OOD-generalizable hypothesis may appear. Moreover, the diversity of hypotheses lets us distinguish ID samples from OOD samples by measuring the degree of divergence in ensemble members' predictions. Despite conceptual benefits, diverse-ensemble training has previously remained a lab-bound concept for several reasons. First, previous approaches required a separate OOD dataset that may nurture diverse hypotheses. Second, computational complexities of previous pairwise diversification objectives increase quadratically with the ensemble size.

We have addressed the challenges through the novel Scalable Ensemble Diversification (SED) framework. SED identifies the OOD-like samples from a single dataset, bypassing the need to prepare a separate OOD dataset. SED also employs a stochastic pair selection algorithm which reduces the quadratic complexity of previous approaches to a constant cost per SGD iteration. We have demonstrated good performances by SED on the OOD generalization and detection tasks, both at the ImageNet scale, a largely underexplored regime in the ensemble diversification community. In particular, for OOD detection, our novel diversity measure of Predictive Diversity Score (PDS) amplifies the benefits of diverse ensembles for OOD detection. The code to reproduce the results of our experiments will provided with the next revision of the manuscript.

Limitations

We do not provide theoretical justification for the method. Our experiments were conducted on models with a frozen feature extractor.

333

334

335

336

337

338

339

340

341

342

343

344

345

346

347

348

349

350

351

352

353

354

355

356

357

358

359

360

361

362

363

364

365

366

367

368

369

370

371

372

373

374

375

376

377

378

379

380

381

382

383

384

385

386

387

388

389

390

391

392

References

H. L. Benoit, L. Jiang, A. Atanov, O. F. Kar, M. Rigotti, and A. Zamir. Unraveling the key components of OOD generalization via diversification. In The Twelfth International Conference on Learning Representations , 2024. URL https://openreview.net/forum?id=Lvf7GnaLru .

L. Breiman. Bagging predictors. Machine Learning , 24(2):123-140, Aug 1996. ISSN 1573-0565. doi: 10.1007/BF00058655. URL https://doi.org/10.1007/BF00058655 .

L. Breiman. Random forests. Machine Learning , 45(1):5-32, Oct 2001. ISSN 1573-0565. doi: 10.1023/A: 1010933404324. URL https://doi.org/10.1023/A:1010933404324 .

A. S. Chen, Y. Lee, A. Setlur, S. Levine, and C. Finn. Project and probe: Sample-efficient domain adaptation by interpolating orthogonal features. arXiv preprint arXiv:2302.05441 , 2023.

F. D'Angelo and V. Fortuin. Repulsive deep ensembles are bayesian. Advances in Neural Information Processing Systems , 34:3451-3465, 2021.

A. de Mathelin, F. Deheeger, M. Mougeot, and N. Vayatis. Maximum weight entropy. arXiv preprint arXiv:2309.15704 , 2023.

A. de Mathelin, F. Deheeger, M. Mougeot, and N. Vayatis. Deep anti-regularized ensembles provide reliable out-of-distribution uncertainty quantification, 2023.

J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei. Imagenet: A large-scale hierarchical image database. In 2009 IEEE Conference on Computer Vision and Pattern Recognition , pages 248-255, 2009. doi: 10.1109/CVPR.2009.5206848.

N. Durasov, T. Bagautdinov, P. Baque, and P. Fua. Masksembles for uncertainty estimation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 13539-13548, 2021.

Y. Gal and Z. Ghahramani. Dropout as a bayesian approximation: Representing model uncertainty in deep 393 learning. In international conference on machine learning , pages 1050-1059. PMLR, 2016. 394

I. Goodfellow, Y. Bengio, and A. Courville. Deep Learning . MIT Press, 2016. http://www. 395 deeplearningbook.org . 396

L. Hansen and P. Salamon. Neural network ensembles. IEEE Transactions on Pattern Analysis and 397 Machine Intelligence , 12(10):993-1001, 1990. doi: 10.1109/34.58871. 398

399

400

401

J. C. Helton, J. D. Johnson, and W. L. Oberkampf. An exploration of alternative approaches to the representation of uncertainty in model predictions. Reliability Engineering & System Safety , 85(1-3): 39-71, 2004.

402

403

404

405

406

407

408

409

410

411

412

413

414

415

416

417

418

419

420

421

422

423

424

425

426

427

428

D. Hendrycks and T. Dietterich. Benchmarking neural network robustness to common corruptions and perturbations. In International Conference on Learning Representations , 2019. URL https:// openreview.net/forum?id=HJz6tiCqYm .

D. Hendrycks and K. Gimpel. A baseline for detecting misclassified and out-of-distribution examples in neural networks. In International Conference on Learning Representations , 2017. URL https: //openreview.net/forum?id=Hkg4TI9xl .

D. Hendrycks, S. Basart, N. Mu, S. Kadavath, F. Wang, E. Dorundo, R. Desai, T. Zhu, S. Parajuli, M. Guo, et al. The many faces of robustness: A critical analysis of out-of-distribution generalization. In Proceedings of the IEEE/CVF international conference on computer vision , pages 8340-8349, 2021.

D. Hendrycks, K. Zhao, S. Basart, J. Steinhardt, and D. Song. Natural adversarial examples. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 15262-15271, 2021.

T. K. Ho. Random decision forests. In Proceedings of 3rd International Conference on Document Analysis and Recognition , volume 1, pages 278-282 vol.1, 1995. doi: 10.1109/ICDAR.1995.598994.

G. Huang, Y. Li, G. Pleiss, Z. Liu, J. E. Hopcroft, and K. Q. Weinberger. Snapshot ensembles: Train 1, get mfor free. arXiv preprint arXiv:1704.00109 , 2017.

R. Huang and Y. Li. Mos: Towards scaling out-of-distribution detection for large semantic space. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 8710-8719, 2021.

A. Krogh and J. Vedelsby. Neural network ensembles, cross validation, and active learning. In G. Tesauro, D. Touretzky, and T. Leen, editors, Advances in Neural Information Processing Systems , volume 7. MIT Press, 1994. URL https://proceedings.neurips.cc/paper_files/paper/1994/ file/b8c37e33defde51cf91e1e03e51657da-Paper.pdf .

B. Lakshminarayanan, A. Pritzel, and C. Blundell. Simple and scalable predictive uncertainty estimation using deep ensembles. In I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, editors, Advances in Neural Information Processing Systems , volume 30. Curran Associates, Inc., 2017. URL https://proceedings.neurips.cc/paper_files/paper/2017/file/ 9ef2ed4b7fd2c810847ffa5fa85bce38-Paper.pdf .

429

430

431

432

433

434

435

436

437

438

439

440

441

442

443

444

445

446

447

448

449

450

451

452

453

454

455

456

457

458

459

460

461

462

463

464

465

466

467

468

469

470

471

472

473

474

475

476

477

478

479

480

481

482

483

Y. Lee, H. Yao, and C. Finn. Diversify and disambiguate: Out-of-distribution robustness via disagreement. In The Eleventh International Conference on Learning Representations , 2023. URL https://openreview.net/forum?id=RVTOp3MwT3n .

Z. Li, I. Evtimov, A. Gordo, C. Hazirbas, T. Hassner, C. C. Ferrer, C. Xu, and M. Ibrahim. A whac-a-mole dilemma: Shortcuts come in multiples where mitigating one amplifies others. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 20071-20082, 2023.

Y. Liu and X. Yao. Simultaneous training of negatively correlated neural networks in an ensemble. IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics) , 29(6):716-725, 1999.

I. Loshchilov and F. Hutter. Decoupled weight decay regularization. In International Conference on Learning Representations , 2019. URL https://openreview.net/forum?id=Bkg6RiCqY7 .

Y. Ovadia, E. Fertig, J. Ren, Z. Nado, D. Sculley, S. Nowozin, J. Dillon, B. Lakshminarayanan, and J. Snoek. Can you trust your model's uncertainty? evaluating predictive uncertainty under dataset shift. Advances in neural information processing systems , 32, 2019.

M. Pagliardini, M. Jaggi, F. Fleuret, and S. P. Karimireddy. Agree to disagree: Diversity through disagreement for better transferability. In The Eleventh International Conference on Learning Representations , 2023. URL https://openreview.net/forum?id=K7CbYQbyYhY .

A. Ross, W. Pan, L. Celi, and F. Doshi-Velez. Ensembles of locally independent prediction models. In Proceedings of the AAAI Conference on Artificial Intelligence , volume 34, pages 5527-5536, 2020.

D. Teney, E. Abbasnejad, S. Lucey, and A. van den Hengel. Evading the simplicity bias: Training a diverse set of models discovers solutions with superior ood generalization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages 16761-16772, June 2022.

D. Teney, M. Peyrard, and E. Abbasnejad. Predicting is not understanding: Recognizing and addressing underspecification in machine learning. In S. Avidan, G. Brostow, M. Cissé, G. M. Farinella, and T. Hassner, editors, Computer Vision - ECCV 2022 , pages 458-476, Cham, 2022. Springer Nature Switzerland. ISBN 978-3-031-20050-2.

H. Touvron, M. Cord, and H. Jégou. Deit iii: Revenge of the vit. In European conference on computer vision , pages 516-533. Springer, 2022.

T. Trinh, M. Heinonen, L. Acerbi, and S. Kaski. Input-gradient space particle inference for neural network ensembles. In International Conference on Learning Representations , 2024.

H. Wang and Q. Ji. Diversity-enhanced probabilistic ensemble for uncertainty estimation. In R. J. Evans and I. Shpitser, editors, Proceedings of the Thirty-Ninth Conference on Uncertainty in Artificial Intelligence , volume 216 of Proceedings of Machine Learning Research , pages 2214-2225. PMLR, 31 Jul-04 Aug 2023. URL https://proceedings.mlr.press/v216/wang23c.html .

H. Wang, Z. Li, L. Feng, and W. Zhang. Vim: Out-of-distribution with virtual-logit matching. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 4921-4930, 2022.

F. Wenzel, J. Snoek, D. Tran, and R. Jenatton. Hyperparameter ensembles for robustness and uncertainty quantification. Advances in Neural Information Processing Systems , 33:6514-6527, 2020.

A. G. Wilson and P. Izmailov. Bayesian deep learning and a probabilistic perspective of generalization. Advances in neural information processing systems , 33:4697-4708, 2020.

M. Wortsman, G. Ilharco, S. Y. Gadre, R. Roelofs, R. Gontijo-Lopes, A. S. Morcos, H. Namkoong, A. Farhadi, Y. Carmon, S. Kornblith, and L. Schmidt. Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time. In K. Chaudhuri, S. Jegelka, L. Song, C. Szepesvari, G. Niu, and S. Sabato, editors, Proceedings of the 39th International Conference on Machine Learning , volume 162 of Proceedings of Machine Learning Research , pages 23965-23998. PMLR, 17-23 Jul 2022. URL https://proceedings.mlr.press/v162/wortsman22a.html .

S. Yashima, T. Suzuki, K. Ishikawa, I. Sato, and R. Kawakami. Feature space particle inference for neural network ensembles. In International Conference on Machine Learning , pages 25452-25468. PMLR, 2022.

S. Zaidi, A. Zela, T. Elsken, C. C. Holmes, F. Hutter, and Y. Teh. Neural ensemble search for uncertainty estimation and dataset shift. Advances in Neural Information Processing Systems , 34:7898-7911, 2021.

T. Zhou, S. Wang, and J. A. Bilmes. Diverse ensemble evolution: Curriculum data-model marriage. In S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, and R. Garnett, editors, Advances in Neural Information Processing Systems , volume 31. Curran Associates, Inc., 2018. URL https://proceedings.neurips.cc/paper_files/paper/2018/file/ 3070e6addcd702cb58de5d7897bfdae1-Paper.pdf .

484

485

486

487

488

489

490

491

492

493

494

495

496

497

498

499

500

501

502

503

504

505

506

507

508

509

510

511

512

513

514

515

516

517

518

519

520

521

522

523

524

525

526

527

528

529

530

531

532

533

534

NeurIPS Paper Checklist

1. Claims

Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?

Answer: [Yes]

Justification: Please refer to § 4

Guidelines:

The answer NA means that the abstract and introduction do not include the claims made in the paper.

The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.

The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.

It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.

2. Limitations

Question: Does the paper discuss the limitations of the work performed by the authors?

Answer: [Yes]

Justification: Please refer to § 5

Guidelines:

The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.

The authors are encouraged to create a separate "Limitations" section in their paper.

The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.

The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.

The authors should reflect on the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.

The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.

If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.

While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.

3. Theory Assumptions and Proofs

Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?

Answer: [NA]

Justification: The paper contains no theoretical results.

Guidelines:

The answer NA means that the paper does not include theoretical results.

All the theorems, formulas, and proofs in the paper should be numbered and crossreferenced.

All assumptions should be clearly stated or referenced in the statement of any theorems.

The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.

Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.

Theorems and Lemmas that the proof relies upon should be properly referenced.

4. Experimental Result Reproducibility

Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?

Answer: [Yes]

Justification: Please refer to § 4

Guidelines:

The answer NA means that the paper does not include experiments.

If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.

If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.

Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.

While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example

(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.

(b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.

(c) If the contribution is a new model (e.g., a large language model), then there should either be a way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).

(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.

5. Open access to data and code

Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?

Answer: [Yes]

Justification: Code will be available soon, please refer to § 4.1.

Guidelines:

The answer NA means that paper does not include experiments requiring code.

Please see the NeurIPS code and data submission guidelines ( https://nips.cc/ public/guides/CodeSubmissionPolicy ) for more details.

While we encourage the release of code and data, we understand that this might not be possible, so 'No' is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).

The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines ( https: //nips.cc/public/guides/CodeSubmissionPolicy ) for more details.

The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.

The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.

At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).

Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.

6. Experimental Setting/Details

Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?

Answer: [Yes]

Justification: please refer to § 4.1.

Guidelines:

The answer NA means that the paper does not include experiments.

The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.

The full details can be provided either with the code, in appendix, or as supplemental material.

7. Experiment Statistical Significance

Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?

Answer: [No]

Justification: Error bars are not reported because their magnitude was below the rounding error or roughly around it for the majority of experiments.

Guidelines:

The answer NA means that the paper does not include experiments.

The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).

The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)

The assumptions made should be given (e.g., Normally distributed errors).

640

641

642

643

644

645

646

647

648

649

650

651

652

653

654

655

656

657

658

659

660

661

662

663

664

665

666

667

668

669

670

671

672

673

674

675

676

677

678

679

680

681

682

683

684

685

686

687

688

It should be clear whether the error bar is the standard deviation or the standard error of the mean.

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.

For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).

If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.

8. Experiments Compute Resources

Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?

Answer: [Yes]

Justification: please refer to § 4.1.

Guidelines:

The answer NA means that the paper does not include experiments.

The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.

The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.

The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).

9. Code Of Ethics

Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines ?

Answer: [Yes]

Justification: we followed the Code to the best of our knowledge.

Guidelines:

The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.

If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.

The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).

10. Broader Impacts

Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?

Answer: [NA]

Justification: We believe that this work has no societal impact.

Guidelines:

The answer NA means that there is no societal impact of the work performed.

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.

Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.

689 690 691 692 693 694 695 696 697 698 699 700 701 702 703 704 705 706 707 708 709 710 711 712 713 714 715 716 717 718 719 720 721 722 723 724 725 726 727 728 729 730 731 732 733 734 735 736 737 738 739 740 741

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.

The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.

If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).

11. Safeguards

Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?

Answer: [NA]

Justification: We believe that our paper does not pose such risks as we train models for ImageNet classification.

Guidelines:

The answer NA means that the paper poses no such risks.

Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.

Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.

We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.

12. Licenses for existing assets

Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?

Answer: [No]

Justification: we were unable to find the license for the dataset we used.

Guidelines:

The answer NA means that the paper does not use existing assets.

The authors should cite the original paper that produced the code package or dataset.

The authors should state which version of the asset is used and, if possible, include a URL.

The name of the license (e.g., CC-BY 4.0) should be included for each asset.

For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.

742

743

744

745

746

747

748

749

750

751

752

753

754

755

756

757

758

759

760

761

762

763

764

765

766

767

768

769

770

771

772

773

774

775

776

777

778

779

780

781

782

783

784

785

786

787

788

789

790

791

If this information is not available online, the authors are encouraged to reach out to the asset's creators.

13. New Assets

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?

Answer: [NA]

Justification: the paper does not release new assets.

Guidelines:

The answer NA means that the paper does not release new assets.

Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.

The paper should discuss whether and how consent was obtained from people whose asset is used.

At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.

14. Crowdsourcing and Research with Human Subjects

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?

Answer: [NA]

Justification: the paper does not involve crowdsourcing nor research with human subjects.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.

According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.

15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?

Answer: [NA]

Justification: the paper does not involve crowdsourcing nor research with human subjects.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.

For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper presents SED, a method for scaling up existing diversification methods to large-scale datasets and tasks. SED identifies the OOD-like samples from a single dataset, bypassing the need to prepare a separate OOD dataset. Experimental results demonstrated good performances by SED on the OOD generalization and detection tasks.\n\nSTRENGTHS:\n1. SED scales up existing ensemble methods.\n2. According to the author's experimental results, SED demonstrates its application to OOD generalization and detection at ImageNet level.\n3. Simple method, and easy to understand.\n\nWEAKNESSES:\n1. I am very confused about the dataset division for OOD detection task in the paper. The distribution should refer to “label distribution” in OOD detection [1], which means that OOD samples should not have overlapping labels w.r.t. training data. In the paper, the ID dataset is ImageNet-1K, while the OOD dataset for the OOD detection task includes ImageNet-C (Table 3). Their label spaces overlap, which is clearly incorrect. I don't believe the experiments conducted in this paper fall under the category of OOD detection. I suggest the authors refer to relevant literature on OOD detection, such as OpenOOD [1].\n\n2. The ablation studies are insufficient. For instance, the number of layers being diversified is a hyperparameter. I believe conducting ablation experiments on this would make the paper more solid.\n\n3. I think the experiments in the paper are not comprehensive enough. For example, how does it perform on small-scale datasets? Although it may not be fair to compare with methods using real OOD datasets, this could provide insights into SED's performance from multiple perspectives.\n\n4. The comparative methods in the paper are not comprehensive enough. How does it perform compared to existing OOD generalization and OOD detection methods? If SED is complementary to existing methods, how much improvement can it bring?\n\n5. The paper claims to speed up pairwise divergence computation, but no results are shown. Could authors demonstrate specifically how much speedup was achieved?\n\n6. typos:\n6.1 Line 61: \"We verify that SEDdiversifies a model...\"\n6.2 Line 68: \"In all three cases, SEDachieves a superior generalization...\"\n\n[1] Yang et al, OpenOOD: Benchmarking Generalized Out-of-Distribution Detection, IJCV 2024.\n\nQUESTIONS:\n1. Where is the batch-wise weight $\\alpha_B$ used in the method?\n\nLIMITATIONS:\nI think the author should discuss more about the limitations of the method proposed in the article, such as the computational time required for the method proposed in the article compared with other methods.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2024-07-08"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper presents a new method which directly encourages ensemble diversification on selected ID datapoints without the need for a separate OOD dataset. They also introduce a new measure of epistemic uncertainty which measures the diversity of the final predictions of each model, and suggest a speedup of comparing pairwise disagreement via random sampling.\n\nSTRENGTHS:\n- The paper is well-written, and the presentation of the method is easy to understand. \n- The experiments cover a wide range of OOD datasets.\n- The methods are intuitive and can be inexpensively applied to existing ensemble diversification algorithms.\n- SED-A2D outperforms other baselines when using uniform soup or prediction ensembles for OOD generalization, and also achieves the highest AUROC for OOD detection.\n\nWEAKNESSES:\n- It appears that utilizing this new training objective leads to a loss in ID accuracy, since it encourages members of the ensembles to diverge. This tradeoff between ID accuracy and OOD accuracy may not be desirable in many settings. Overall, the paper emphasizes the improved OOD performance but does not show its impact on ID data for many experiments, such as the ablation studies for OOD detection, model diversity, etc.\n- The stochastic computation of pairwise disagreement seems incremental, and there is no work comparing this stochastic implementation with the traditional expensive one. It would be helpful to include an ablation study to understand the accuracy vs performance tradeoff.\n- There are many other methods for OOD detection beyond MSP with BMA (eg [1]). How PDS does compare against other baselines?\n- Deep ensembles remain competitive in many settings, and the best values for C-1 and C-5 OOD generalization are still achieved using ensembles. \n\n[1] Xia and Bouganis: On the Usefulness of Deep Ensemble Diversity for Out-of-Distribution Detection https://arxiv.org/abs/2207.07517\n\nQUESTIONS:\n- The selection of OOD samples does not appear to be tied to a specific method. Have you tried applying this method to other like DivDis?\n- It's interesting that diversifying on ID samples improves OOD generalization compared to diversifying on ImageNet-R (Table 2). What was the setup for these experiments? Did the ensembles see fewer \"OOD\" datapoints in the latter?\n- In Table 1, I'm surprised that SED-A2D has each model giving a different prediction for C-1, but OOD detection AUROC using PDS is quite low. Does this imply that SED-A2D also has very high disagreement on ID data?\n\nLIMITATIONS:\n- The approach sacrifices ID accuracy for OOD generalization/detection.\n- The experiments only showed the result of finetuning the last two layers.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-10"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper aims to train a diverse ensemble of models via a framework called Scalable Ensemble Diversification. This framework does not require an additional dataset of OOD inputs, as it identifies OOD samples from a given ID dataset. It then encourages the ensemble to return diverse predictions (disagreement) on these OOD samples. Furthermore, the framework makes use of stochastic summation to speed up the disagreement computation. Results are shown for different tasks like generalisation, OOD detection on different OOD datasets.\n\nSTRENGTHS:\n- The high level idea of removing the need for a separate OOD dataset and speeding up the diversification computation can be useful in practice.\n- The writing is clear and easy to follow.\n\nWEAKNESSES:\n1. It is not clear why the method works, additional ablations studies would be useful.   \n    - Naive A2D (and DivDis) uses IN-R data to compute the disagreement loss, which could give the method an advantage as it has access to the OOD data. However, it has a lower accuracy on IN-R. There seems to be two methodological differences between A2D and SED-A2D, the OOD data and use of stochastic sum. Given that A2D computes the full pairwise disagreement and stochastic sum is meant to reduce cost rather than improve performance, why does SED-A2D perform better? It would have been useful to compare two methods that only differs on the OOD data used. E.g., SED-A2D without the stochastic sum.   \n    - From eqn 6, it looks like the two terms have contradicting objectives. For a “OOD” point, the first term encourages all models to classify the point correctly, but the second term encourages models to have different predictions on the same point. These objectives can be challenging to balance.   \n2. The writing clearly explains the method or setup, but sometimes stops short of giving further insights. For example,  \n    - Further analysis of experimental results   \n        - Table 2 why does having more ensemble component (5→50) make the SED-A2D results worse? Similar trends can also be seen in Tab 4 for C-1 or C-5. Could it be because the stochastic sum does not scale with more models?   \n        - Why was #unique used in Tab 1 to measure diversity when the Predictive Diversity Score was just introduced?  \n        - Why does oracle selection perform worse compared to simple average in Tab 2? I would expect otherwise given that there is privilege information.   \n    - Components of the method can be better motivated   \n        - Why was the A2D loss chosen instead of other losses e.g. DivDis?   \n        - Why is optimizing Eqn 6 preferable to e.g., forming an OOD dataset from the ID data based on the errors of DeiT or even from the errors from an ensemble of models, similar to imagnet-a, and using existing techniques like [23,28].   \n    - “collecting a separate OOD dataset can be very costly, if not impossible”. \n        - There are cheap ways to introduce OOD samples to an ID dataset, e.g., simple augmentations/transformations to the input. Why are these methods not preferable?  \n4. One of the main contributions involves speeding up the disagreement computation. There does not seem to be experimental details or results on this. E.g., a subset of models is chosen, what is the size of this subset? How does performance for generalization/detection change with and without this speedup?\n\nQUESTIONS:\n1. It would be interesting to see if using eval datasets other than IN-R as OOD data results in similar trends in Tab 2. It seems like deep ensemble and its variant and SED-A2D, i.e., the methods that do not make use of external OOD data, tend to perform better. \n2. Instead of computing the detection scores, how does the proposed diversity measure correlate with correct predictions? \n3. In table 1, what is the IN-Val #unique?\n\nLIMITATIONS:\nYes the limitations were adequately addressed.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2024-07-11"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nEnsembles of diverse models have shown promising signs for out-of-distribution (OOD) generalization.\nTo boost diversity, some methods require a set of OOD examples for measuring the disagreement among models.\nThe desired OOD examples, however, can be difficult to obtain in practice.\nThis paper proposes to dynamically draw OOD samples from the training data during training.\nThis is done by assigning a higher OOD score to examples with a greater loss in each mini-batch.\nTo make the diversification process across multiple models more efficient, the authors propose a stochastic approach that only diversifies a small sample of models at each iteration.\nThe resulting diversified models give rise to the notion of a diversity score for uncertainty estimation and can be used for OOD detection.\n\nSTRENGTHS:\n- The paper introduces several reasonable improvements to a state-of-the-art method, A2D, making it more scalable and practically feasible.\n- The empirical performance looks good. It is a bit surprising that the proposed method can outperform A2D which has access to “true” OOD datasets.\n\nWEAKNESSES:\n- The notion of “OOD samples” in an ID dataset is confusing. The actual implementation, i.e. assigning a higher “OOD-ness” weight to training examples with a greater loss, is more like identifying “hard” training examples rather than just OOD samples. Calling them “OOD samples” somewhat obfuscate their nature. They are not arbitrary OOD samples but hard samples within the support of the ID dataset. It is not obvious why diversifying models’ predictions on such samples would help. Is such prediction diversification always conducive to OOD generalization? If not, when would the proposed method work or break? These relevant theoretical questions are not answered satisfactorily in the current manuscript.\n- The connection between SED and PDS is weak; PDS is not well justified. The A2D diversification loss can also be seen as a measure for prediction diversity, like PDS. Why choose PDS instead for OOD detection? Furthermore, is PDS really a good measure for epistemic uncertainty? Imagine two cases. In the first case, two models confidently (with probability 1) predict the same class for an input example, while in the second case, the two models assign uniform probability to all classes for another example. The PDS for these two examples are exactly the same, yet the models are much less confident (or more uncertain) in the second case. Meanwhile, BMA does not have this issue.\n- The baselines are relatively limited. There are many other diversification methods which do not require a separate OOD dataset [1, 2, 3]. How does the proposed method compare with these methods? Can the authors also comment on why BMS is the only considered baseline for OOD detection?\n- The definition of #unique values is not very clear. Table 1 shows SED-A2D has extremely large #unique values. On C-1 dataset, the value is 5, the maximum possible value. If my understanding is correct, does this suggest that all the 5 models disagree with each other on every C-1 example? If so, this suggests that for many examples, 4 out of 5 models are probably wrong. Why is this more of a good sign than a bad one?\n\n[1] Rame, Alexandre, et al. \"Diverse weight averaging for out-of-distribution generalization.\" Advances in Neural Information Processing Systems 35 (2022): 10821-10836.  \n[2] Chu, Xu, et al. \"Dna: Domain generalization with diversified neural averaging.\" International conference on machine learning. PMLR, 2022.  \n[3] Lin, Yong, et al. \"Spurious feature diversification improves out-of-distribution generalization.\" arXiv preprint arXiv:2309.17230 (2023).\n\nQUESTIONS:\nPlease see the weaknesses.\n\nLIMITATIONS:\nThe authors only briefly mentioned two limitations of the work. I don't notice any potential negative societal impact.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2024-07-12"}

comments: SUMMARY:
The paper proposes Scalable Ensemble Diversification (SED) to extend existing diversification methods to large-scale datasets and tasks where ID-OOD separation may not be possible, and also propose Predictive Diversity Score (PDS) as a novel measure for epistemic uncertainty. Extensive analysis and experiments support the effectiveness of the proposed modules.

STRENGTHS:
The logic of this paper is very clear, the motivation is reasonable, and the proposed method has been proven to be effective in analysis and experiments. The figures and tables in the paper are also relatively clear.

WEAKNESSES:
1. Although the experiments are diverse, I am not sure if the comparison is comprehensive. Can more explanation and discussion be added?

2. The feature extractor used is frozen. Is the proposed method robust enough to different feature extractors? What will the performance be if the feature extractor is also involved in the training?

QUESTIONS:
The proposed framework is interesting, but can it potentially be extended to improve a wide range of OOD tasks?

LIMITATIONS:
The authors have addressed the limitations.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 2
DATE: 2024-07-13