Distributionally Robust Ensemble of Lottery Tickets Towards Calibrated  Sparse Network Training
Authors: Hitesh Sapkota
Dingrong Wang
ZHIQIANG TAO
Qi Yu
Venue: NeurIPS 2023 poster (2023)
Keywords: sparse network training
model calibration
TL;DR: We propose a novel Distributionally Robust Optimization (DRO) framework to achieve an ensemble of lottery tickets towards calibrated network sparsification.

Abstract: The recently developed sparse network training methods, such as Lottery Ticket Hypothesis (LTH) and its variants, have shown impressive learning capacity by finding sparse sub-networks from a dense one. While these methods could largely sparsify deep networks, they generally focus more on realizing comparable accuracy to dense counterparts yet neglect network calibration. However, how to achieve calibrated network predictions lies at the core of improving model reliability, especially when it comes to addressing the overconfident issue and out-of-distribution cases. In this study, we propose a novel Distributionally Robust Optimization (DRO) framework to achieve an ensemble of lottery tickets towards calibrated network sparsification. Specifically, the proposed DRO ensemble aims to learn multiple diverse and complementary sparse sub-networks (tickets) with the guidance of uncertainty sets, which encourage tickets to gradually capture different data distributions from easy to hard and naturally complement each other. We theoretically justify the strong calibration performance by showing how the proposed robust training process guarantees to lower the confidence of incorrect predictions. Extensive experimental results on several benchmarks show that our proposed lottery ticket ensemble leads to a clear calibration improvement without sacrificing accuracy and burdening inference costs. Furthermore, experiments on OOD datasets demonstrate the robustness of our approach in the open-set environment.

Distributionally Robust Ensemble of Lottery Tickets Towards Calibrated Sparse Network Training
Hitesh Sapkota Dingrong Wang
Zhiqiang Tao Qi Yu
Rochester Institute of Technology (hxs1943, dw7445, zhiqiang. tao, qi.juborit.ed
Abstract
The recently developed sparse network training methods, such as Lottery Ticket Hy- pothesis (LTHH) and its variants, have shown impressive learning capacity by finding sparse sub-networks from dense one. While these methods could largely sparsify deep networks, they generally focus more on realizing comparable accuracy to dense counterparts yet neglect network calibration. However, how to achieve cali- brated network predictions lies at the core of improving model reliability, especially when it comes addressing the overconfident issue and out-of-distribution cases. In this study, we propose novel Distributionally Robust Optimization (DRO) framework achieve an ensemble of lottery tickets towards calibrated network sparsification. Specifically, the proposed DRO ensemble aims to learn multiple diverse and complementary sparse sub-networks (tickets) with the guidance of uncertainty sets, which encourage tickets to gradually capture different data distri- butions from easy to hard and naturally complement each other. We theoretically justify the strong calibration performance by showing how the proposed robust training process guarantees to lower the confidence incorrect predictions. Ex- tensive experimental results several benchmarks show that proposed lottery ticket ensemble leads to clear calibration improvement without sacrificing accu- racy and burdening inference costs. Furthermore, experiments on OOD datasets demonstrate the robustness of our approach in the open-set environment.
Introduction
While there is remarkable progress in developing deep neural networks with densely connected layers, most of these dense networks have poor calibration performance [9], limiting their applicability in safety-critical domains like self-driving cars [4] and medical diagnosis [1]]. The poor calibration is mainly due to the fact that there exists good number of wrongly classified data samples (i.e., low accuracy) with high confidence resulting from the memorization effect introduced by an over- parameterized architecture (27). Recent sparse network training methods, such Lottery Ticket Hypothesis (LTH)) 6 and its variants 3 38
generally assume that there exists a sparse sub-network (i.e., lottery ticket) in randomly initialized dense network, which could be trained in isolation and also match the performance of its dense counterpart network in terms of accuracy. While these methods may, some extent, alleviate the overconfident issue, most of them require pre-training of dense network followed by multi-step iterative pruning, making the overall training process highly costly, especially for large dense networks, Even for techniques that do rely on pre- vre-trainig and iterative pruning (e.g., Edge Popup or EP (25)). their learning goal focuses on pushing the accuracy up to the original dense networks and hence may still exhibit a severely over-fitting behavior, leading to poor calibration performance as demonstrated in Figure (b).
18
16
351
Inspired by the recent success of using ensembles estimate uncertainties solution to realize well-calibrated predictions would be training multiple sparse sub-networks and
[13 34 potential
*Corresponding author
37th Conference on Neural Information Processing Systems (NcurIPS 2023).

---

ECE 10.99
ECE 24.0
ECE: 22
ECE 22
Accuuscy
Accaacy
Cartidenca
Carfieecs
Cartibence
Contidence
(a) Dense network
(b) EP sparse sub-network (c) EP Sparse sub-network (d) EP network ensemble
Accurayy
Accuracy
Cunterica
Cartdence
(c) Sparse sub-network 1
(f) Sparse sub-network 2 (g) Sparse sub-network 3 (h) DRO ensemble of (e-g)
Figure 1: Calibration performance by expected calibration error (ECH) on Cifar100 dataset with ResNet101 architecture with density K. 15%. EP refers to the Edge Popup algorithm [25]
building an ensemble from them. As such, by leveraging accurate uncertainty quantification, the ensemble is expected to achieve better calibration. However, existing ensemble models of sparse networks rely on pre- pre-training and iterative fine-tuning for learning each sub-network [18 (5), leading to a significant overhead for building the entire ensemble. Furthermore, an ensemble of independently trained sparse sub-networks does not necessarily improve the calibration performance. Since these networks are trained in a similar fashion from the same training data distribution, they could be strongly correlated such that the ensemble model will potentially inherit the overfitting behavior of each sub-network as shown in Figure (c). Therefore, the calibration capacity of sparse sub-network ensemble can be compromised as shown empirically in Figure (d))
To further enhance the calibration of the ensemble, it is critical to ensure sufficient diversity among sparse sub-networks so that they are able to complement each other. One natural way to achieve diversity is to allow each sparse sub-network (ticket) to primarily focus on a specific part of training data distribution. This inspires to leverage the AdaBoost (28) framework that sequentially finds tickets by manipulating training data distribution based on errors. By this means, the AdaBoost facilitates the training for a sequence of complementary sparse sub-networks. However, the empirical analysis (see Table I reveals that in the AdaBoost ensemble, most sub-networks (except for the first one) severely under-fit data leading poor generalization ability. This is mainly because of the overfitting behavior of the first sub-network, which assigns very low training losses to the majority of data samples, making the subsequent sub-networks concentrate on very rare difficult samples that likely to be outliers or noises. Hence, directly learning from these difficult samples without having global knowledge of the entire training distribution will result in the failure of subsequent training tickets and also hurt the overall calibration.
To this end, we need a more robust learning process for proper training of complementary sparse sub- networks, each of which can be learned in an efficient way to ensure the cost-effective construction of the entire ensemble. We propose Distributionally Robust Optimization (DRO) framework to schedule learning an ensemble of lottery tickets (sparse sub-networks) with complimentary calibration behaviors that contribute an overall well-calibrated ensemble as shown in Figure (e-h). Our technique directly searches sparse sub-networks in a randomly initialized dense network without pre-training iterative pruning. Unlike the AdaBoost ensemble, the proposed ensemble ticket method starts from the original training distribution and eventually allows learning each sub-network from different parts of the training distribution enrich diversity. This is also fundamentally different from existing sparse ensemble models [18 (35), which attempt to obtain diverse sub-networks in a heuristic way by relying on different learning rates. As result, these models offer no guaranteed complementary behavior among sparse sub- networks to cover a different part of training data, which essential to alleviate the overfitting behavior of the learned sparse sub-networks. In contrast, we realize a principled scheduling process by changing the uncertainty set of DRO, where small set pushes sub-networks learning with easy data samples and a large set focuses on the difficult ones (see Figure 2). By this means, the ticket ensemble governed by our DRO framework could work complementary and lead to much better calibration ability as demonstrated in Figure IT)). On the one hand, we hypothesize that the ticket found with easy data samples will tend to be learned and

---

overfitted easily, resulting in overconfident predictions TiggueTTe). On the other hand, the ticket focused on more difficult data samples will be less likely to overfit and may become conservative and give under-confident predictions. Thus, it is natural to form an ensemble of such lottery tickets to complement each other making calibrated predictions. As demonstrated in Figure (h),, owing to the diversity in the sparse sub-networks (e-g), the DRO ensemble exhibits better calibration ability. It is also worth noting that under the DRO framework, our sparse sub-networks already improve the calibration ability shown in Figure (f-g), which is further confirmed by our theoretical results.
Experiments conducted on three benchmark datasets demonstrate the effectiveness of our proposed technique compared to sparse counterparts and dense networks. Furthermore, we show through the experimentation that because of the better calibration, our model is being able to perform well on the distributionally shifted datasets [6] (CIFAR 10-C and CIFAR (0--). The experiments also demonstrate that our proposed DRO ensemble framework can better detect open-set samples on varying confidence thresholds. The contribution of this work can be summarized as follows:
a new sparse ensemble framework that combines multiple sparse sub-networks to achieve better calibration performance without dense network training and iterative pruning. a distributionally robust optimization framework that schedules the learning of an ensemble complementary sub-networks (tickets),
theoretical justification of the strong calibration performance by showing how the proposed robust training process guarantees to lower the confidence of incorrect predictions in Theorem 2 extensive empirical evidence on the effectiveness of the proposed lottery ticket ensemble in terms of competitive classification accuracy and improved open-set detection performance.
2 Related Work
Sparse networks training. Sparse network training has received increasing attention in recent years. Representative techniques include lottery ticket hypothesis (LTH) [6] and its variants [3] [3]. To avoid training dense network, supermasks have been used to find the winning ticket in the dense network without training network weights (38]. Edge-Popup (EP) extends this idea by leveraging training scores associated with the neural network weights and only weights with top scores are used for predictions. There are two key limitations to most existing LTH techniques. First, most of them require pre-training of dense network followed by multi-step iterative pruning making the overall training process expensive. Second, their learning objective remains as improving the accuracy up to the original dense networks and may still suffer from over-fitting (as shown in Figure
Sparse network ensemble. There are recent advancements in building ensembles from sparse networks. A pruning and regrowing strategy has been developed in model, called CigL [[6]. where dropout serves as an implicit ensemble to improve the calibration performance. Cigl. requires weight updates and performs pruning and growing for multiple rounds, leading to a high training cost. Additionally, dropping many weights may lead to a performance decrease, which prevents building highly sparse networks. This idea has been further extended by using different learning rates to generate different typologies of the network structure for each sparse network [18, 35). While diversity among sparse networks can be achieved, there is no guarantee that this can improve the calibration performance of the final ensemble. In fact, different networks may still learn from the training data in similar way. Hence, the learned networks may exhibit similar overfitting behavior with a high correlation, making it difficult to generate well-calibrated ensemble. In contrast, the proposed DRO ensemble schedules different sparse networks learn from complementary parts of the training distribution, leading to improved calibration with theoretical guarantees.
Model calibration. Various attempts have been proposed make the deep models more reliable either through calibration g [24]]]]] 32 or uncertainty quantification [0]. cost-calibration techniques have been commonly used, including temperature scaling (24 9 using regularization penalize overconfident predictions (23] Recent studies show that post-hoe calibration falls short of providing reliable predictions (22). Most existing techniques require additional post-processing steps and an additional validation dataset. In our setting, we aim to improve the calibration ability of sparse networks without introducing additional post-calibration steps or validation dataset.
3 Methodology
Let D.v = (X,Y) = ((x1, (1,, ..., (XN, yN)} be a set of training samples where each x, E RD is D-dimensional feature vector and yn E [1, be associated label with C' total classes. Let M be
3

---

122
Uniform
Distribution
(a] Sparse Subnetwark 1 (Learning from Easlest Samples)
(b) Sparse Subnetwork 2 (Learning from Moderate Samples)
(c) Sparse Subnetwork 3 from Difficult Samples)
(d) Proposed DRO Ensemble (Sparse Network)
Figure 2:
Robust ensemble where y) defines the size of an uncertainty with ?p S 1/2 S 7/3-
the total number of base learners used in the given ensemble technique. Further, consider K. to be the density ratio in the given network, which denotes the percentage of weights we keep during the training process. The major notations are summarizzed in the Appendix.
3.1 Preliminaries
Edge-Popup (EP) 255 EP finds a lottery ticket (sparse sub-network) from randomly initialized dense network based the score values learned from training data. Specifically, to find the sub- network with density IC, the algorithm optimizes the scores associated with each weight in the dense network. During the forward pass, the top-K weights in each layer are selected based on their scores. During the backward pass, scores associated with all weights are updated, which allows potentially useful weights that ignored in previous forward passes to be re-considered.
Expected calibration error. Expected Calibration Error (ECE) measures the correspondence between predicted probability and empirical accuracy 20 Specifically, mis-calibration is computed based on the difference in expectation between confidence and accuracy: Eş [I = ylp = p) - pl]. In practice, we approximate the expectation by partitioning confidences into T bins (equally spaced) and take the weighted average on the absolute difference between each bins' accuracy and confidence.
Let B, denote the t-th beam and we have ECE = -''
[.l |ace( B.) con conf(B.)).
3.2 Distributionally Robust Ensemble (DRE)
As motivated in the introduction, further enhance the calibration of deep ensemble, it instru- mental to introduce sufficient diversity among the component sparse sub-networks so that they can complement each other when forming the ensemble. One way to achieve diversity is to allow each sparse sub-network to primarily focus on a specific part of the training data distribution. Figure 2 provides an illustration of this idea, where the training data can imagined to follow a multivariate Gaussian distribution with the red dot representing its mean. In this case, the first sub-network will learn the most common patterns by focusing on the training data close to the mean. The subsequent sub-networks will then learn relatively rare patterns by focusing other parts of the training data (e.g., two or three standard deviations from the mean).
AdaBoost ensemble. The above idea inspires us to leverage the AdaBoost framework 22 to manipulate the training distribution that allows us train a sequence complementary sparse sub- networks. In particular, where each data sample has an equal probability to be sampled. In this way, the first sparse sub- network can learn the common patterns from the most representative training samples. Starting from the second sub-network. the training distribution is changed according to the losses suffered from the previous sub-network during the last round of training. This allows the later sub-networks to focus on the difficult data samples by following the spirit of AdaBoost.
train the first sparse sub-network from the original training distribution,
However, our empirical results reveal that in the AdaBoost ensemble, most sub-networks (except for the first one) severely underfit the training data, leading to a rather poor generalization capability. This is caused by the overfitting behavior of the first sparse sub-network. which assigns very small training losses to a majority of data samples. a result, the subsequent sub-networks can only focus on a limited number of training samples that correspond to relatively rare patterns (or even outliers and noises) in the training data. Directly learning from these difficult data samples without general knowledge of the entire training distribution will result in the failure of training the sub-networks.

---

Distributionally robust ensemble (DRE). To tackle the challenge outlined above, we need more robust learning process ensure proper training of complementary sparse sub-networks. Different from the AdaBoost ensemble, the training of all sub-networks starts from the original training distribution in the DRO framework. Meanwhile, focus on learning from different parts of the training distribution to ensure the desired diverse and complementary behavior. Let ((x,, Q) denote the loss associated with the n data sample with θ being the parameters in the sparse sub-network. Then, the total loss is given by
also allows each sub-network to eventually
N Σ
C RRbust (0)
max 2x2ifRobutt
2x θ)
(1)
The uncertainty set defined to assign weights z is given
RRobust
FZZER" i 1, =2== 1.2 10=20011
1 N =n}
(2)
where (2(2g]q) is f-divergence between two distributions and q and controls the size of the uncertainty set and 1 E 1NT is N-dimensional unit vector. Depending on the 1) value, the above robust framework instantiates different sub-networks. For example, by making 7) + oc, we have 1, 2 2 0, (zll S x) In this case, we train a sub-network by only using the most difficult sample in the training set. On the other extreme with 7) + O, we have RRoobust = {z E R-V 1 1, 2 0, (z n < <0). which assigns equal weights to all data samples. So, the sub-network learns from the original training distribution.
URoobus- = {2 E R x
To fully leverage the key properties of the robust loss function as described above, we propose to perform distributionally robust ensembling learning generate diverse set of sparse sub- networks with well-controlled overfitting behavior that can collectively achieve superior calibration performance. The training process starts with a relatively small r) value to ensure that the initially generated sub-networks can adequately capture the general patterns from the most representative data samples in the original training distribution. The training proceeds by gradually increasing the value, which allows the subsequent sub-networks to focus on relatively rare and more difficult data samples. As result, the later generated sub-networks tend to produce less confident predictions that complement the sub-networks generated in the earlier phase of the training process. This diverse and complementary behavior among different sparse bub-networks is clearly illustrated in Figure D (c)-(g). During the ensemble phase, we combine the predictions of different sub-networks in the logit space by taking the mean and then performing the softmax. In this way, the sparse sub-networks with high 1) values help lower the overall confidence score, especially those wrongly predicted data samples. Furthermore, the sub-networks with lower 1) values help to bring up the confidence score of correctly predicted data samples. Thus, the overall confidence score will be well compensated, resulting in better calibrated ensemble.
3.3 Theoretical Analysis
In this section, we theoretically justify why the proposed DRE framework improves the calibration performance by extending the recently developed theoretical framework on multi-view learning M. In particular, we will show how it can effectively lower the model's false confidence its wrong predictions resulting from spurious correlations. To this end, we first define the problem setup that includes some key concepts used in our theoretical analysis. We then formally show that DRO helps to tackle the spurious correlations by learning from less frequent features that characterize difficult data samples in a training dataset. This important property further guarantees better calibration performance of DRO as we show in the main theorem. It is worth to note that our theoretical analysis primarily from the spurious correlation perspective. This is only of the potential sources that can lead to over-confidence, resulting in poor-calibration in neural networks. The goal is offer deeper insights why the proposed approach is able to improve the calibration performance resulting from spurious correlations by effectively lowering the model's 's false confidence on its wrong predictions.
Problem setup. Assume that each data sample x,, E RRD is divided into P total patches, where each patch a d-dimensional vector. For the sake of simplicity, let us assume each class e E [1, C has two characterizing (major) features V. (ve. V with L 2. For example, the features for Cars could be Headlights and Tires. Let D and D denote the set of single-view and multi-view data
5

---

samples, respectively, which are formally defined as
{ Vn E D if one of Va,1 Ve,2 appears along with some noise features [xx, yn E D both Va,1 and Ve,2 appears along with some noise features
The noise features (also called minor features) refer to those that do not characterize (or differentiate) given class (e.g., being part of the background). In important applications like computer vision, images supporting such "multi-view" structure very common []] For example, for most car images, we can observe all main features, such as Wheels, Tires, and Headlights so they belong to D. Meanwhile, there may also be images, where multiple features are missing. For example, if the car image is taken from the front, the tire and wheel features may not be captured. In most real-world datasets, such single-view data samples are usually much limited as compared to their multi-view counterparts. The Appendix provides concrete examples of both single and multi-view images. Let us consider (x, y) E D with the major feature Ve,1 where y = c. Then each patch x" E Rd can be expressed as
x" === aLv
Σ a +
V'EUVVE
where U= = [Ve,1, )2 is collection of all features, a² is the weight allocated feature Va. anv'' E [0,-]l is the weight allocated to the noisy feature v' that is not present in feature set V. i.e. v' EVV... and e" ~ (0, (oP)²11) is a random Gaussian noise. In 4 patch x2 in a single-view sample x also contains set of minor (noise) features presented from other classes iee, v' E U in :vvv' ,"VV'vv'''' E EV\ve. However, since the vingle-view data samples are usually sparse in the training data, may prevent the model from accumulating large a" for Ve,1 shown Lemma I below. In contrast, some noise v' may be selected as the dominant feature (due to spurious correlations) to minimize the errors of specific training samples, leading to potential overfitting of the model.
addition to the main feature Va,I- Since Ve,/ characterizes class we have a² > a..V
We further assume that the network contains H convolutional layers, which outputs F(x; Q) (xx((x)... RC, The logistic output for the dd class can be represented
F.(x) = Σ
ReLU((( x")]
(5)
kE(H PE[P]
where Oc,r denote the th convolution layer (feature map) associated with class Under the above data and network setting, we propose the following lemma.
Lemma Let Ve,1 be the main feature vector present in the single-view data D Assume that number of single-view data samples containing feature Vo, is limited as compared with the rest, i.e., Nvc. < Nuvee. Then, at any iteration >> we have
N
(0 Ve.1) (0 VeJ) max ZEU
Σ 2n [[oy=c(Venn(xa)] + (1 (F(..)
(6)
where K. is dataset specific constant, 8 is the learning rate, SOFT. is the softmax output for class and Vx,n,(x,,) = EpEPs,,, (x,) ReLU (0e, aP) with (x,) being the collection of patches containing feature Ve.1 1 in Xz. The set U is an uncertainty set that assigns weight to each data sample based on it loss. In particular, the uncertainty set under DRO is given as in 2 and we
further define the uncertainty set under ERM: FRR (z RN 2n v [1, N]} Learning via the robust loss in (1) leads to stronger correlation between the network weights Ge. and the single-view data feature Ne,ti
1(0 Ve.) Robust > ((0) VeI)/EEM: 0
Remark. The robust loss Robuss forces the model to learn from the single-view samples (according to the loss) by assigning a higher weight. As a result, the network weights will be adjusted to increase the correlation with the view data features Ve,1 due to Lemma] [] In contrast, for standard ERM, weight is uniformly assigned all samples. Due to the sparse single-view data features (which also makes them more difficult to learn from, leading to larger loss), the model does not grow sufficient

---

correlation with Ve, In this case, the ERM model instead learns to memorize some noisy feature v' introduced through certain spurious correlations. For testing data sample, the ERM model may confidently assign it to an incorrect class k according to the noise feature v'. In the theorem below, we show how the robust training proces can effectively lower the confidence of incorrect predictions, leading to an improved calibration performance.
Theorem 2. Given a new testing sample x E D containing Ve,1 as the main feature and a dominant noise feature v' that learned due to memorization, we have
{SOFT&(X)) Robust < { (SOFTx(X)) ERM
(8)
where v' is assumed to be main feature characterizing class k.
Remark. For ERM, due to the impact of the dominate noisy feature v', it assigns a large probability class k since v' one of its major features, leading high confidence for an incorrect prediction. In contrast, the robust learning process allows the model to learn a stronger correlation with the main feature Ve.I as shown in Lemma] 1 Thus, the model is less impacted by the noise feature v', resulting in reduced confidence in predicting the wrong class k. Such a key property guarantees an improved calibration performance, which clearly verified by our empirical evaluation. It is also worth noting that Theorem 2 does not necessarily lead to better classification accuracy. This is because K only ensures that that the false confidence is lower than an ERM model, but there is no guarantee that (SOFTx(x)) Robust Robust < [SOFTx(x)) Robust- It should be noted that our DRE framework
ensures diverse sparse sub-network focusing on different single-vies view data samples from different classes. As such, an ensemble of those diverse sparse subnetworks provides maximum coverage of all features (even the weaker one) and therefore can ultimately improve the calibration performance. The detailed proofs are provided in the Appendix.
4 Experiments
We perform extensive experimentation to evaluate the distributionally robust ensemble of sparse sub-networks. Specifically, we test the ability of our proposed technique in terms of calibration and classification accuracy. For this, we consider three settings: (a) general classification, (b) out-of- distribution setting where we have in-domain data but with different distributions, and (c) open-set detection, where have unknown samples from new domains.
4.1 Experimental Settings
Dataset description. For the general classification setting, we consider three real- world datasets: Cifar10, Cifar100 12], and TinylmageNe [14]. For the out-of-distribution setting, we consider the corrupted version of the Cifar10 and Cifarl 100 datasets which are named Cifar10-C and Cifar100- 10. It should be noted that in this setting, we train all models in clean dataset and perform testing in the corrupted datasets. For open-set detection, we use the SVHN dataset [21] as the open-set dataset and Cifarl0 and Cifarl 100 as the close-set data. A more detailed description of each dataset is presented in the Appendix.
Evaluation metrics. To assess the model performance in the first two settings, we report the classification accuracy (ACC) along with the Expected Calibration Error (ECE). In the case of open-set detection, we report open-set detection for different confidence thresholds.
Implementation details. In all experiments, we use family of ResNet architectures with two density levels: 9% and 15%. To construct an ensemble, we learn 3 sparse sub-networks each with a density of 3% for the total of 9% density and that of 5% density for the total of density 15% All experiments are conducted with the 200 total epochs with an initial learning rate of 0.1 and a cosine scheduler function to decay the learning rate over time. The last-epoch model is taken for all analyses. For the training loss, we use the EP-loss in our DRO ensemble that optimizes the scores for each weight and finally selects the sub-network from the initialized dense network for the final prediction. The selection is performed based on the optimized scores. More detailed information about the training process and hyperparameter settings can be found in the Appendix.
4.2 Performance Comparison
In our comparison study, we include baselines that are relevant to our technique and therefore we primarily focus on the L.T--based techniques. Specifically, we include the initial lottery ticket hypothesis (LTH) [6] that iteratively performs pruning from a dense network until the randomly
7

---

Table Accuracy and ECE performance with 9% density for Cifarl0 and Cifar100.
Cifar10
Cifar100
Training Type
Approach
ResNet50
ResNer111
ResNet101
ResNer152
ACC
ECE
ACC
ECE
ACC
ECE
ACC
ECE
Denset
94.82
95.12
5.99
76.40
16.89
77.97
16.73
Dense Pre-training
Pruning 17
93.45
5.31
93.67
6.14
75.11
15.89
75.12
16.24
LTH 6
92.65
3.68
92.87
6.02
74.09
15.45
74.41
16.12
DLTH 3
93:27
5.87
95.12 7.09
77.29
16.64
77.86
17.26
Mixup
92.86
93.06
6.01
74.15
15.41
74.28
16.05
Sparse Training
CigL 76
92.39
5.06
93.41
4.60
76.40
9.30
76,46
9.91
DST Ensemble [8
88.87
2.02
84,93
0.8
63.57
7.23
63.22
6.18
Sup-ticket 35
94.52
3.30
95.04
3.10
78.28
10:20
78.60
10.50
Mask Training
AdaBoost
93.12
5.13
94.15
5.46
75.15
22.96
75.89
24.54
EP 25
94.20
94.35
4.03
75.05
14.62
75.68
14.41
SNE
94.70
2.51
94,48
3.51
75.69
9.02
75.22
10.89
DRE (Ours)
94.60
0.7
94.28
0.7
74.68
1.20
74.37
2.09
Table 2: Accuracy and ECE on TinylmageNet.
K.
9%
K. 15%
Training Type
Approach
ResNat101
WideResNet101
ResNet101
WideResNet101
ACC
ECE
ACC
ECE
ACC
ECE
ACC
ECE
Dense
71.28
15.58
72.57
71.28
15.58
72.57
16.96
Pre-training
Pruming
68.85
14.72
69.78
16.38
70.24
14.24
15.36
LTH [
69.23
13.97
69.13
15.34
70.16
13.63
70.25
14.24
DLTH 3
70.12
71.36
71.68
72.97
17.21
Mixup [32
69.34
14.24
69.25
15.59
70.28
14.31
70.39
14.57
Mask Training
AdaBoost
69.52
17:23
68.66
19.46
70.12
16.57
70.24
18.35
EP 25
69.88
10.78
71.57
9.82
70.46
11.99
70.71
12.41
SNE
71.28
4.64
73.32
5.48
72.20
6.57
74.56
6.55
DRE (Ours)
71.68
3.48
74.04
2.82
72.00
1.52
73.72
1.08
Table 3: Accuracy and ECE performance on out-of-distribution datasets.
Cifar10
Ci
Training Type
Approach
ResNet50
ResNet101
ResNet101
ResNet152
ACC
ECE
ACC
ECE
ACC
ECE
ACC
ECE
Densel
79.65
19.63
79.65
19.63
54.75
35.32
54.75
35.32
Dense Pre-training
I.1 Pruning 17
77.34
17.95
76.39
17.89
52.06
31.45
51.67
30.98
LTH 6
75.85
17.88
76.15
17.62
50.79
31.23
51.35
30.56
DL.H
79.67
21.74
80.12
20.31
54.82
37.55
55.12
35.74
Mixup
76.35
17.74
76.88
17.55
51.36
31.12
51.92
30.35
Sparse Training
Cigl. 16
70.80
21.04
04.8441
21.42
49.42
25.86
$1.49
24.13
Sup-ticket
35
72.89
17.80
73.01
18.82
48.80
24.99
48.81
25.62
Mask Training
AdaBoost
75.94
22.96
74.55
21.46
51.36
38.45
$1.25
38.34
EP 25
77.58
17.82
77.73
17.46
52.18
30.60
52.14
29.48
SNE
78.93
15.73
78.61
15.56
54.74
24.22
54.00
20.54
DRE (Ours)
78.57
10.92
78.00
10.19
54.11
14.28
53,21
8.13
initialized sub-network with given density is reached. Once the sub-network is found, the model trains the sub-network using the training dataset. Similarly, we also include L1 praning [17. We also include three approaches Cigl. (16), Sup-ticket (35). DST Ensemble [T8] which are based on the pruning and regrowing sparse network training strategies. From Venkatesh et al.
(3 we consider MixUp strategy as a comparison baseline as does not require multi-step forward passes. A dense network is also included as a reference (denoted as Dense'), ). Furthermore, we report the performance obtained using the EP algorithm 25 on a single model with given density. Finally, we also include the deep ensemble technique (i.e., Sparse Network Ensemble (SNE), where each base model is randomly initialized and independently trained. The approaches that require pre- pre-trannin of dense network are categorized under the Dense Pre-training category. Those performing sparse network training but actually updating the network parameters are grouped as Sparse Training. It should be noted that sparse training techniques still require iterative pruning and regrowing. Finally, techniques that attempt search the best initialized sparse sub-network through mask update (e.g., EP) are grouped as Mask Training.
8

---

Sparte Hebwork
Sparte Rebrork
CP Sparte Retwork
Sparte Ratterr
Sparce Nettoark
Bissenbble
Sparse Netwerk
Sparse
60
P1-Skore
40
16
20
20
20
0.59
0.29
5.50
0.00
0.7
0.33
0.35
0.60
0.70
0.00
0.53
0.20
0.23
6
0.00
0.85
0.150
0.20
0.23
0.50
0.40
050
0.00
Confidence
Connience
Conndence
Conndence
(a) CIFAR1O (K 15%)
(b) CIFARIO (AC 9%)
(c) CIFAR100 (K= 15%) (d) CIFARIOO 100 (A== 9%)
Figure 3: Open-set detection performance on different confidence thresholds.
General classification setting. In this setting, we consider clean Cifar10, Cifar100, and TinyIm- ageNer datasets. Tables 1 and 6 (in the Appendix) show the accuracy and calibration error for different models with density 9% and 15%. It should be noted that for the TinyImageNe dataset, we could not run the Sparse Training techniques due to the computation issue (i.e., memory overflow). This may be because sparse training techniques require maintaining additional parameters for the pruning and regrowing strategy. In the Appendix, with those baselines on lower architecture size. There are three key observations we can infer from the experimental results. First, sparse networks are able maintain or improve the generalization performance (in terms of accuracy) with better calibration, which can be seen by comparing dense network performance with the edge-popup algorithm. Second, the ensemble in general helps to further lower the calibration error (lower the better). For example, in all datasets, standard ensemble (SNE) consistently improves the EP model. Finally, the proposed DRE significantly improves the calibration performance by diversifying base learners and allow each sparse sub-network to focus on different parts of the training data. The strong calibration performance provides clear empirical evidence to justify our theoretical results.
have made comparison of the proposed DRE
ditt----ditriiutionn classification setting. In this setting, we assess the effectiveness of the proposed techniques on out-of-distribution samples. Specifically, [10] provide the Cifar10-C and Cifar100-c validation datasets which are different than that of the original clean datasets. They apply different corruptions (such as blurring noise, and compression) to shift the distribution of the datasets. We assess those corrupted datasets using the models trained using the clean dataset. Table 3 shows the performance using different architectures. In this setting, we have not included DST Ensemble, because: (a) its accuracy far below the SOTA performance, and (b) same training mechanism as that of the Sup-ticket, whose performance is reported. As shown, the proposed DRE provides much better calibration performance even with the out of distribution datasets.
Open-set detection setting. In this setting, we demonstrate the ability of our proposed DRO ensemble in detecting open-set samples. For this, we the SVHN dataset an open-set dataset. Specifically, we have a better calibration, we would be able to better differentiate the open-set samples based on the confidence threshold. For this, we randomly consider 20% of the total testing in-distribution dataset as the open-set samples from the SVHN dataset. The reason for only choosing a subset of the dataset is to imitate the practical scenario where we have very few open-set samples compared to the close-set samples. We treat the open-set samples as the positive and in-distribution (close-set) ones as the negative. Since this is a binary detection problem, we compute the F-score [8] at various thresholds, which considers both precision and recall. Figure 3 shows the performance for the proposed technique along with comparative baselines. As shown, our proposed DRE (refereed as DRO Ensemble) always stays on the top for various confidence thresholds which demonstrates that strong calibration performance can benefit DRE for open-set detection as compared other baselines.
4.3 Ablation Study
In this section, we investigate the impact of the backbone architecture along with the size of the ensemble. Additional ablation studies are presented in Appendix D.9
Performance analysis of different backbones. Table 4 (a) reports the performance of Cifar10 from both DRE and EP using different backbone architectures. In case of WideResNet28- 10, the calibration error is low without sacrificing the accuracy. also demonstrates that the superior performance of DRE is not limited to specific backbone. In case of ViT. DRE still achieves a much lower calibration error than EP. However, using ViT as a backbone, the accuracy from both EP and DRE is lower and ECE is higher than other backbones. Existing studies show that without pretraining, the lack of useful

---

Table ACC and ECE with different backbones and number of subnetworks
WideResNer28-10
VIT
ResNer101
ResNet152
Approach
Approach
ACC
ECE
ACC
ECE
ACC
ECE
ACC
EP
94.12
4.53
86.
10.01
DRF (M 3) 94.87
1.71
94.74
1.34
DRE
93.98 1.93
85.53 4.18
DRE (M = 94.79
0.84
94.69
0.62
(a) Different backbones Cifarl0 Dataset.
(b) Different M values Cifar10 with K. = 15%.
inductive biases for VIT cause performance drop [1]. Since no pretraining is conducted in both EP and DRE, it causes a lower accuracy (and a higher ECE).
Impact of number of sparse-sub-networrs sparse sub-networks. It should noted that our work not limited only for M = 3. We can instead increase the M value. For example, Table (b) shows the performance for ensemble model with M = 5, where each sub-network is trained with I = 3%, leading to a total K = 15%. We also show the performance with M 3, where each sub-network is trained with K. 5%. As can be seen, if there is sufficient learning capacity for each sub-network, the ECE score can further improve with the increase of M.
In this analysis, study the impact of number of
4.4 Qualitative Analysis
In this section, provide illustrative examples further justify why the proposed DRE is better calibrated as compared to existing baselines. Figure 4 (a)-(d) show the confidence values for the wrongly classified samples by different sparse sub-networks in the DRE ensemble. As can be seen, each sparse sub-network provides confidence values in different ranges, where the sub-network in (a) learned from most representative samples and the one (c) from the most difficult ones. As these sub-networks are complementary with each other, the DRE has a much better confidence distribution for incorrect samples than the baselines. In contrast, as shown in Figure of Appendix [D12] all the baselines, including the dense network, EP sparse networks, and the SNE, tend to allocate much higher confidence values to wrongly classified examples, leading to poor calibration. case of correctly classified samples, the proposed DRE generates confident predictions and thereby not compromising the calibration performance, which is further discussed Appendix D.12
800
500
400
Number of Samplee
400
Number at Sample le
500
300
400
400
200
200
300
200
100
100
9.0
0.5
1.0
0.0
0.5
1.0
0.0
0.5
1.0
6.0
0.5
1.0
Confidence
Confiilence
(a) Sparse sub sub-network
(b) Sparse sub-network 2
(c) Sparse sub-network
(d) DRE
Figure Confidence scores of incorrectly classified samples in CIFAR 100 with ResNet101.
5
Conclusion
In this paper, we proposed novel DRO framework, called DRE, that achieves an ensemble of lottery tickets towards calibrated network sparsification. Specifically, with the guidance of uncertainty sets under the DRO framework, the proposed DRE aims to learn multiple diverse and complementary sparse sub-networks (tickets) where uncertainty sets encourage tickets to gradually capture different data distributions from easy hard and naturally complement each other. We have theoretically justified the strong calibration performance by demonstrating how the proposed robust training process guarantees to lower the confidence incorrect predictions. The extensive evaluation shows that the proposed DRE leads to significant calibration improvement without sacrificing the accuracy and burdening inference cost. Furthermore, experiments on OOD and open-set datasets show its effectiveness in terms of generalization and novelty detection capability, respectively.
10

---

Acknowledgement
This research was supported in part by an NSF award 185-18 and an ONR award N00014- 18-1-2875. The views and conclusions contained in this paper those of the authors and should not be interpreted as representing any funding agency.
References
[1] Zeyuan Allen-Zhu and Yuanzhi Li. Towards understanding ensemble, knowledge distillation and self-distillation in deep learning. In The Eleventh International Conference on Learning Representations, 2023.
[2] Yue Bai, Huan Wang, Xu Ma, Yitian Zhang, Zhiqiang Tao, and Yun Fu. Parameter-efficient masking networks. In Advances in Neural Information Processing Systems, 2022.
[3] Yue Bai, Huan Wang, Zhiqiang Tao, Kunpeng Li, and Yun Fu. Dual lottery ticket hypothesis. In International Conference on Learning Representutions, 2022.
[4] Mariusz Bojarski, Davide Del Testa, Daniel Dworakowski, Bernhard Firner, Beat Flepp, Prasoon Goyal, Lawrence D. Jackel, Mathew Monfort, Urs Muller, Jiakai Zhang, Xin Zhang, Jake Zhao, and Karol Zieba. End to end learning for self-driving cars, 2016.
[5] Tianlong Chen, Zhenyu Zhang, Jun Wu, Randy Huang, Sijia Liu, Shiyu Chang, and Zhangyang Wang. Can you win everything with lottery ticket? Transactions on Machine Learning Research, 2022.
[6] Jonathan Frankle and Michael Carbin, The lottery ticket hypothesis: Finding sparse, trainable neural networks. 2018.
[7] Yarin Gal and Zoubin Ghahramani. Dropout as a bayesian approximation: Representing model uncertainty in deep learning, 2015.
[8] Cyril Goutte and Eric Gaussier. A probabilistic interpretation of precision, recall and f-score, with implication for evaluation. David E. Losada and Juan M. Fernández-Luna, editors, Advances in Information Retrieval, pages 345-359, Berlin, Heidelberg, 2005. Springer Berlin Heidelberg.
[9] Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q. Weinberger. On calibration of modern neural networks. In Proceedings of the 34th International Conference on Machine Learning Volume 70, ICML' 17, page 1321-1330. JMLR.org, 2017.
[10] Dan Hendrycks and Thomas Dietterich. Benchmarking neural network robustness to common corruptions and perturbations. Proceedings of the International Conference Learning Representations, 2019.
[11] Xiaoqian Jiang, Melanie Osl, Jihoon Kim, and Lucila Ohno-Machadae Calibrating predictive model estimates support personalized medicine. Journal of the American Medical Informaties Association JAMIA, 19:263 274, 2011.
[12] Alex Krizhevsky. Learning multiple layers of features from tiny images. 2009.
[13] Balaji Lakshminarayanan, Alexander Pritzel, and Charles Blundell. Simple and scalable predictive uncertainty estimation using deep ensembles. In Proceedings of the 31 International Conference on Neural Information Processing Systems, NIPS' 17, page 6405-6416, Red Hook, NY, USA, 2017. Curran Associates Inc.
[14] Ya Le and Xuan S. Yang. Tiny imagenet visual recognition challenge. 2015.
[15] Namhoon Lee, Thalaiyasingam Ajanthan, and Philip S. Torr. SNIP: single-shot network pruning based on connection sensitivity. CoRR, Abs/1800.02340, 2018.
[16] Bowen Lei, Ruqi Zhang, Dongkuan Xu, and Bani Mallick. Calibrating the rigged lottery: Mak- ing all tickets reliable. In The Eleventh International Conference on Learning Representations, 2023.

---

[17] Hao Li, Asim Kadav, Igor Durdanovic, Hanan Samet, and Hans Peter Graf. Pruning filters for efficient convnets, 2016.
[18] Shiwei Liu, Tianlong Chen, Zahra Atashgahi, Xiaohan Chen, Ghada Sokar, Elena Mocanu, Mykola Pechenizkiy, Zhangyang Wang, and Decebal Constantin Mocanu. Deep ensembling with no overhead for either training or testing: The all-round blessings of dynamic sparsity, 2022.
[19] Hesham Mostafa and Xin Wang. Parameter efficient training of deep convolutional neural networks by dynamic sparse reparameterization. ArXiv, abs/1902.059677, 2019.
[20] Mahdi Pakdaman Naeini, Gregory F. Cooper, and Milos Hauskrecht. Obtaining well calibrated probabilities using bayesian binning. In Proceedings of the Twenty-Ninth AAAI Conference on Artificial Intelligence, AAAI' 15, page 2901-2907. AAAI Press, 2015.
[21] Yuval Netzer, Tao Wang, Adam Coates, A. Bissacco, Bo Wu, and A. Ng. Reading digits in natural images with unsupervised feature learning. 2011.
[22] Yaniv Ovadia, Emily Fertig, Jie Ren, Zachary Nado, D Sculley, Sebastian Nowozin, Joshua V. Dillon, Balaji Lakshminarayanan, and Jasper Snoek. Can you trust your model's uncertainty? evaluating predictive uncertainty under dataset shift, 2019.
[23] Gabriel Pereyra, George Tucker, Jan Chorowski, Lukasz Kaiser, and Geoffrey Hinton. Regular- izing neural networks by penalizing confident output distributions, 2017.
[24] John Platt and Nikos Karampatziakis. Probabilistic outputs for svms and comparisons to regularized likelihood methods. 2007.
[25] Vivek Ramanujan, Mitchell Wortsman, Aniruddha Kembhavi, Ali Farhadi, and Mohammad Rastegari. What's 's hidden in a randomly weighted neural network?, 2019.
[26] Shiori Sagawa, Pang Wei Koh, Tatsunori B. Hashimoto, and Percy Liang. Distributionally robust neural networks for group shifts: On the importance of regularization for worst-case generalization. CoRR, abs/1911.0873. 2019.
[27] Shiori Sagawa, Aditi Raghunathan, Pang Wei Koh, and Percy Liang. An investigation of why overparameterization exacerbates spurious correlations, 2020.
[28] Robert E Schapire. Explaining adaboost. In Empirical inference, pages 37-52. Springer, 2013.
[29] Ramprasaath R. Selvaraju, Abhishek Das, Ramakrishna Vedantam, Michael Cogswell, Devi Parikh, and Dhruv Batra. Grad-cam: Why did you say that? visual explanations from deep networks via gradient-based localization. CoRR, abs/1610.02391, 2016.
[30] Murat Sensoy, Lance Kaplan, and Melih Kandemir. Evidential deep learning to quantify classification uncertainty, 2018
[31] Christian Szegedy, Vincent Vanhoucke, Sergey loffe, Jonathon Shlens, and Zbigniew Wojna. Rethinking the inception architecture for computer vision, 2015.
[32] Bindya Venkatesh, Jayaraman J. Thiagarajan. Kowshik Thopalli, and Prasanna Sattigeri. Cali- brate and prune: Improving reliability of lottery tickets through prediction calibration, 2020.
[33] Chaoqi Wang, Guodong Zhang, and Roger Grosse. Picking winning tickets hefore training by preserving gradient flow. 2020.
[34] Andrew G Wilson and Pavel Izmailov. Bayesian deep learning and a probabilistic perspective of generalization. In H. Larochelle, M. Ranzato, R. Hadsell, M. F. Balcan, and H. Lin, editors, Advances in Neural Information Processing Systems, volume 33, pages 4697-4708, 2020.
[35] Lu Yin, Vlado Menkovski, Meng Fang, Tianjin Huang, Yulong Pei, Mykola Pechenizkiy, Decebal Constantin Mocanu, and Shiwei Liu. Superposing many tickets into one: performance booster for sparse neural network training, 2022.
12

---

[36] Geng Yuan, Xiaolong Ma, Wei Niu, Zhengang Li, Zhenglun Kong, Ning Liu. Yifan Gong, Zheng Zhan, Chaoyang He, Qing Jin, Siyue Wang, Minghai Qin, Bin Ren, Yanzhi Wang, Sijia Liu, and Xue Lin. MEST: accurate and fast memory-economic sparse training framework on the edge. CoRR, abs/21 10. 14032, 2021.
[37] Jize Zhang, Bhavya Kailkhura, and T. Yong-Jin Han. Mix-n-mateen n-mahhh: Ensemble and compositional methods for uncertainty calibration in deep learning, 2020.
[38] Hattie Zhou, Janice Lan, Rosanne Liu, and Jason Yosinski. Deconstructing lottery tickets: Zeros, signs, and the supermask, 2019.
13

---

Supplementary Materials
Appendix
Table of Contents
Summary of Notations
B Robust Loss Optimization in DRO
B.1 Robust Loss Optimization
B.2 Hyperparameter settings
C Proofs of Main Theoretical Results
16
C.I Proof of Lemma !
C.2 Proof Theorem
D Experimental Details and Additional Results
18
D.1 Detailed Dataset Description
8
D.2 Hardware Details for Experimentation
D.3 Single-view and Multi-view Examples
D.4 Additional Result on Cifarl0 and Cifar100
D.5 Additional Baseline Results on Sparse Training Methods
D.6 Additional Baseline Results TinyImageNet
D.7 Performance from Ensemble Members
20
D.8 Comparison with Common Calibration Techniques
20
D.9 Ablation Study
21
D.10 Parameter Size and Inference Speed
D.11 Diversity on Sparse Sub-networks
D.12 Qualitative Analysis
E Broader Impact, Limitations, and Future Work
E.1
Broader Impact
E.2 Limitations and Future Works
F Source Code
25
14

---

Organization of Appendix
In this appendix, first present table summarizing the major notations used by the main paper in Appendix A Next, we provide detailed information about the training process and hyperaprameters setting We provide the detailed proof of Lemma [ and Theorem in Section C After that, we provide additional experimental details and results in Appendix D Finally, we discuss the broader impacts, limitations, and future work of our DRE technique Appendix The link to the source code can be found in the end of the Appendix.
A Summary of Notations
Table beloww shows the major notations used main paper. We further assign each notation into one of Tour major categories: dataset, DRO formulation, sparse training, and theoretical results.
Table 5: Symbols with Descriptions.
Symbol Group
Notation
Description
X
Set of training images Set of training class labels
Y
C
Total classes
Dataset
Predicted class label
N
Total number of training samples
D
Dimensionality of cach data sample
Dj
f f-divergence
DRO
Parameter controlling size of uncertainty set DRO framework
Weight ussociated with n² data sample
M
Number of spurse sub-networks
KC
Density of the given network
Sparse Training
Parameter associated with given neural network
Confidence associated with predicted class
1(x.. +)
Loss ussociated with n data sample
13
Learning rate of the given network
P
Total number of patches in each data sample
d
Dimensionality of each patch
Ve.2
Major 7th feature associated with class
Total numher of features each class class
Theoretical Results
D
Collection single-view data samples
D
Collection of multi-view data sumples Collection of features
U
Number of convolution layers
Fx(x)
Logistic output for the e* class for the data sample Collection of patches containing feature Va. in sample Softmax output for class
Pv.. SOFT.
B Robust Loss Optimization in DRO
In this section, we first provide detailed description on how optimize the robust loss function in We then explain how to set uncertainty set by choosing a proper hyperparameter.
B.1 Robust Loss Optimization
The optimization problem specified in (I involves an inequality constraint so directly solving it may incur higher computational overhead. Therefore, we consider regularized version of the robust loss to train each base learner by using the following loss:
N
Robust
max
=n),.( -DD
1
#20.x 1
NV
where (+) = I(x,,, θ). Solving the above maximization problem leads to a closed-form solution for z" shown by the following lemma:
15

---

Lemma 3. Assuming that Dj is the KL divergence, then solving 9
leads to the following solution
N
CRobust
Σ
2
(9)
(10)
n=1
where in is given by
exp
x
2,
(11)
N exp
(((0)
2j=1
It can be verified that there is a one-to-one correspondence between 7) in 2 and À in (9. Given their roles in the corresponding equations, large 7) implies small À and small 1j implies large A.
B.2 Hyperparameter settings
The hyperparameter in the regularization term chosen based on the difficulty of a dataset. Specifi- cally, for DRE, always consider the À - oo for the first sparse sub-network which is equivalent to Expected Risk Minimization (ERM). For the second and third sub-networks, we choose this hyperparameter based on the difficulty of data samples. It should be noted that need to set higher values for more difficult datasets as difficult samples are more common on those datasets. Using this notion, for Cifar10, we choose small values that the model focus the difficult samples that are few. For this, we choose X = 10 for the second sparse sub-network and À 500 for the third sparse sub-network. Considering Cifar is samples and therefore higher value is preferred. For this, we choose À = 50 for the second sparse sub-network and À 500 for the third one. In the case of TinyImageNet, we have many difficult samples and therefore we choose relatively large À values. Specifically, we choose À = 100 for the second sparse sub-network and 1, 000, 000 for the third sparse sub-network.
difficult, we would have more difficult
C Proofs of Main Theoretical Results
In this section, we provide detailed proofs of the theoretical results presented the main paper.
C.1 Proof of Lemma
Proof. For yn c, with respect to data sample (xn, 3n the gradient can be evaluated
Ve.. 1(0; Xn, Vn)
SOFT (F(x.(
(12)
PE[P]
Assume that the given sample has major feature Ve.,, taking dot product with respect to Va,1 on both side of [12) leads
Vec 1(0;x,, Vn), Vo.) [ - SOFT. Σ PE(P)
(ReLu
xx Ve.)
(13)
Let's further assume that the feature set is orthonormal: Vc, d. Al [L], Vvaull2 = and Ve,1 1 Vel
when (c,1) + (d), Using x" = a'va, + Eveeuv p. v' d given in we have
Ve, Mn), Volt) = [1 SOFT
ReLU (0 x
Σ PE(M
(e, vol))
(x...
(14)
It should be noted that the term i.e., E Euvv, a.r properties of the feature set. Let represent the second term by EpELP (e", v..) Then, have
p (v', Vo.) becomes zero due to the orthogonal
(- ((0x), Vali) = (1 SOFT, SOFT,(F(xx,))
ReLui x²) + # (15)
16

---

Furthermore, let us define (x)a(x;) = EpePr,,(x)) (e, then above equation further reduces to following
{ Ve.. 10;; xn: yn); Val) = (1 SOFT, (F...
(16)
Recall the above equation is the gradient with respect to the n data sample. Considering the gradient with respect all data samples with Yn = c, and us consider the total loss, where the weight on of each loss is assigned according to distribution specified by the uncertainty set U. Then, the total gradient is
(() Y), Vea)
max ZEN
2n
+ x) (VC,)(()
SOFT,
(17)
Now using the standard gradient update rule with 3 being the learning rate, we have
(O Ve.)) = (0 Ve.1) + B maxx ZELL
(Vass() + x1(1 - SOFT SFFT.(F(X.)))
(18)
Let X E D be the most difficult sample having Vo, as the main feature. Also, consider Xn E D to be the easy sample with Vo. c, y/ Then, we have
SOFT ((xx)))] 2 [(1 (OFT. Vn E [1, NJ. yn =c
(19)
Using above property, we can write the following using
18
(0' cA Ve.) + 3 max
2n LL,,=o(Van, x)) (FFT.(F(x...)
S (0L C/i Ve.1) + 2N (1 SOFT.
(20)
On the r.h.s., we have Zn = N for ERM, which assigns equal weights to all samples. Under the assumption of Nv.. < &Nuive... the contribution of the Nv.. on overall gradient will be negligible. In contrast, for the DRO framework, using (II) we have
2B
(21)
Σn 1j exp
((0) X
Since (O) (0) AA > 0,1 co, we have 2k J.
Using r.... of 20 and incorporating
2 N for ERM and 2 N we have
{(0), Vo.1) + 31 SOFT (F(x 1)) S ((0' Vel)
va,) + - -1- (F(( (x)))) Robust (22)
This subsequently leads to the following:
( Ve.) Robust > (10) Ve.) VellbenNtt 0
(23)
which completes the proof of Lemma]
C.2 Proof of Theorem
Let x D& from class with Ve, as the main feature and v' as the dominant feature learned through the memorization. Also consider to be the main feature characterizing class Then for any class c', we can define the following
exp Fer (x))
SOFT. (x)
(24)
Ejeloy exp(F,) (x)))
In the above equation, F2 (x) can be written as
(x) = Σ Σ
ReLU Re xP)
(25)
hE[H] pE[P]
17

---

Substituting x" from 4
we have
(x E Σ
ReLU a" (Bech Vedl+
E VEEIVA
a a (0a.h. (O,,,,
(26)
PEIP)
Substituting d' by k. we have
Fx(x)= Σ Σ
ReLU a (O Vall +
E a (Ok, v²) (Ok,
(27)
PE(P)
v'inUlve
In case of ERM, the Ve,1 signal is fairly weak during the training process due to Nvo, & Noweer Therefore, the term (Ok,s. Ve,1) is negligible. Also, the last term (Qx, eP) is also small as this corresponds the Gaussian noise. For the second term 3v' for which (MR.h, e') is very high because of the spurious correlation. In contrast, for the robust loss, using Lemma T the model learns stronger correlation with the true class parameter and therefore (6e,,, Ve.i) 1) is high. As such, both terms (Ok,h, Ve,1) as well as (ME,,, v'), Vel becomes low. As a result, we have
(F)())) (F) Robust
(28)
Substituting this inequality to (24) we have
(SOFTx(X)) Nobust (SOFTx(X)) ERM
(29)
This completes the proof of Theorem
2
D Experimental Details and Additional Results
In this section, first provide detailed description of datasets used in experimentation followed by hardware description of our experimentation. We then provide examples of single-view and multi-view data samples. Next, we provide additional experimental results and baselines Cifar10 and Cifar100 100 datasets. After that, we provide additional baseline results TinyImageNe We also compare individual ensemble members' performance followed by model performance with different calibration techniques commonly used in dense networks. Then, we perform an in-depth ablation study. Parameter size and inference speed are discussed in the subsequent subsection. We also further investigate the diversity of the sparse subnetworks. Finally, we provide detailed qualitative analysis to support our proposed claim.
D.1 Detailed Dataset Description
For general classification setting, we consider Cifar10, Ci 12]. and TinyImageNet [4] datasets. For the out of distribution setting, consider corrupted version of Cifar10 and Cifar100, which are named as CifarlO-C and Cifar100-4 [10], respectively. Finally, for open-set detection, we leverage
SVHN 21 the open-set dataset. The detailed description of each dataset is given below:
Cifar10 consists of total classes, each consisting of 5,000 training samples and 1,000 testing (evaluation) samples. Each image is a colored image with size 32 x 32.
Cifar100 consists 20 super classes where each super-class consists of classes resulting into total 100 classes. Each class consists of 500 training samples and 100 testing samples. Each image is a colored image with size 32 x 32.
TinylmageNet consists of 200 classes with 1,000,000 samples where each class has 500 training images, 50 validation images, and 50 test images. Each image is a colored image with size 64 x 64. CifarlO-C consists of fifteen different types of corruptions applied dataset where each corruption has 5 severity levels, ranging from I to 5 with being least severe and 5 being most severe. The corruptions include Gaussian noise, shot noise, impulse noise, defocus blur, forsted glass blur, motion blur, zoom blur, snow, frost, fog, brightness, contrast, elastic, pixelate, and JPEG.
the Cifar 10 clean testing
Cifarl0-C. consists of fifteen different corruptions applied on the Cifar 100 clean testing dataset. SVHN consists of 10 classes with digit I class 1, digit class and digit 0 class 10. These are original, variable-resolution. colored house-number images with character level bounding boxes. We use this dataset the open-set dataset in our experimentation.
18

---

Single View
Multi View
Multi View (Present: Tire, Door handle
(Present: Headlight, Missing: Tire, Door handle)
(Present: Headlight, Tire, Door handle)
Missing: Headlight)
Headlight
Tire
Door handle
Figure Examples of single-view and multi-view samples.
D.2
Hardware Details for Experimentation
All experimentations are conducted using NVIDIA RTX A6000 GPU with 48GB memory requiring 300 Watt power. For GPU, CUDA Version: 11.6, Driver Version: 510.108.0 and NVIDIA-SMI 510. 110.108.00 03 is used. In terms of CPU, our experimentation uses an Intel(R) Xeon(R) Gold 6326 CPU 2.90GHz with 64-bit system and an x86 64 architecture.
D.3
Single-view and Multi-view Examples
Figure 5 show the three example images, where the first image is representative single-view data sample whereas the last two are multi-view samples. In this example, we consider three major features for cars: i.e., Tire, Headlight, and Door handle. As only headlight feature is present in the first image, it belongs to the single-view category. For the second and third images, multiple features presented and therefore regard those images multi-view data samples.
Table Accuracy and ECE performance with 15% density for Cifar 10 and Cifar 100 dataset.
Cifarl0
Cifar100
Training Type
Approach
ResNet50
ResNet101
ResNet101
ResNet152
ACC
ECE ACC E33
ACC
E33
ACC
E33
Densel
94.82 5.87 95.12 5.99
76.40
16.89
77.97
16.73
Dense Training
LJ Pruming
93.88 5.69 94.23 5.88
75.53
15.52
75.83
15.78
LTH
92.97 4.03 93.15 5.69
74.36
15.13
74.77
15.22
DLTH
95.15
6.21 95.65 6.96
77.98
16.24
78.23
16.54
Mixup
93.22 4.02
93.38 5.68
74.48
15.10
74.68
15.16
Sparse Training
CigL
92.25 4.67 93.34
4.59
77.88
10.16
77.27
10.62
DST Ensemble
89.57
2.10 88.64
1.34
64.57
9.76
64.75
9.27
Sup-ticket
94.65
3.20
94.95
78.68
10.16
78.95
10.32
Mask Training
AdaBoost
94.07 5,65
94.76
5.14
75.98
23.55
76.28
24.27
94.41
3.90
94.42
4.07
75.66
14.79
76.05
14.79
SNE
94.85
94.96
3.18
76.82
11.12
77.23
11.63
DRE
94.87 1.71
94.74
1.34
75.86
4.90
76.46
5.81
D.4 Additional Result on Cifar10 and Cifar100
Table shows experimental result on Cifar10 and C0 datasets with 15% density. shown, the proposed technique has a far superior performance in terms of the ECE score compared to competitive baselines. This is consistent with the results with a 9% density as presented in the main paper, which further justifies the effectiveness of our proposed technique.
19

---

D.5 Additional Baseline Results on Sparse Training Methods
Apart from the baselines included in the main paper, there are other sparse training methods without the need for iterative gruning/rrowing [5]] 19 33 36]. However, as all these methods primarily focus on pushing the accuracy up to the original dense networks, they still suffer from a severely overfitting behavior, leading to poor calibration performance as shown in Table
Table 7: ECE and Accuracy for other sparse baselines.
Cifar10
Cifar100
Approach
9%
15%
9%
15%
ACC
333
ACC
ECE
ACC
E3E
ACC
333
SNIP [55
93.45
4.10
94.12
3.86
52.99
10.97
54.40
10.56
Dynamic Sparse 19
91.11
5.73
92.09
5.33
70.06
15.56
71.23
14.29
GraSP 33
92.73
4.78
93.16
4.54
72.60
17.07
73.07
15.95
MEST 36
92.94
4.67
93.50
4.40
72.19
16.47
73.49
15.81
EP
94.20
3.97
94.41
3.90
75.07
14.62
75.66
14.79
DRE
94.60
0.7
94.87
1.71
74.68
1.20
75,86
4.90
D.6 Additional Baseline Results on TinyImageNet
As mentioned in the main paper, the computational issue (i.e., memory overflow) makes it impossible to
Table 8: Additional baseline results on Tiny-
ImageNet using ResNet50 with KC = 15%.
run sparse learning techniques iee. Cigl. [6). DST
Ensemble [18. and Sup-ticket (35) on the ResNet101
Training Type
Approach
ACC E33
and Wete architectures to make a fair com- parison. Therefore, in this section, we pick lower capacity model (ResNet50) and compare the perfor- mance. Even for the ResNet50 architecture, CigL still runs into the memory overflow issue with a batch
Sparse Training
DST Ensemble
72.00 2.94
Sup-ticket
68.68
10.96
Mask Training
DRE
71.57 1.51
size of 128. Furthermore, lowering the batch size (e.g., 16) makes the training process extremely slow even using a 48Gb GPU, where each training epoch takes more than half an hour, making model training extremely difficult. Therefore, we did not report the performance of CigL. It should be noted that Cigl. can be trained on Cifar 10 and Cifar 100 because of lower dimension of the input images and have already reported its performance in the main paper. Table shows the performance of DRE along with those from DST Ensemble and Sup-ticket on ResNet50. It is clear that DRE achieves better performance compared these baselines.
D.7 Performance from Ensemble Members
Table 9: Different sub-network performance on Cifar dataset.
We investigate how performance varies in different sparse sub-networks. We use 100 as example and Table&rrepor report the individual sub-network perfor- mance on both accuracy and ECE. While each sparse sub-network is a relatively weaker learner (which expected), they contribute to the final ensemble model in a complementary way, leading to better ECE score as well as accuracy.
ResNet101
ResNet152
Suhnetworks
ACC
ECE
ACC
E3E
Subetwork (3%)
68.22
14.35
69.65
13.31
Subetwork 2 (3%)
69.03
1.39
70.00
3.39
Subetwork 3 (3%)
72.86
11.96
70.24
14.78
DRE
74.68
1.20
74.37
2.09
D.8 Comparison with Common Calibration Techniques
In this section, we investigate whether existing calibration techniques designed for training dense networks can be leveraged further improve the calibration performance of sparse networks. How- ever, most of these techniques (e.g., temperature scaling and m-atch) are post hoc techniques, which require a separate validation set to fine-tune the parameters. This means we need to further divide the training data into training and validation sets, which may negatively impact the general- ization capability the trained model (due to less training data). To make comparison, we pick Temperature Scaling (TS) [9. Label Smoothing (LS) [3], and few other techniques proposed in
20

---

4.8
3.51
1.26
1.08
4.85
6.59
12.09
10.06
7.02
6.24
1.48
1.89
3.51
1.99
1.02
1.59
5.45
6.98
10
10.06
7.99
5.69
4.85
1.34
2.69
1.26
1.02
2.41
2.98
6.62
7.99
50
7.02
5.69
3.06
2.79
2.42
3.98
A2
100
1.08
1.59
2.90
4.27
7.07
8.32
100
6.24
4.85
2.79
1.4
2.64
4.15
1000
4.85
5.45
6.62
7.07
10.18
10.38
1000
1.48
1.34
2.42
2.64
6.03
7.02
5000
6.59
6.98
7.99
6.32
10.38
12.17
5000
1.89
2.69
3.98
4.15
7.02
9.58
100
1000
Sabel
100
1000
5000
A
A
(a) AC
5%
(b) K = 3%
100 Figure 6: (a-b) Impact of À on ECE using ResNet101 architecture on Cifar 100 dataset.
37 including Ensemble Temperature Scaling (ETS) and Isotonoic Regression One All combined with Temperature Scaling (IROvA-TS). We apply these calibration techniques on the top of the HP algorithm. Specifically, as LS does not require separate validation set, train it the full training dataset using the LS loss (with = 0.1). Other calibration techniques require separate validation and therefore we divide training data into training and validation with a 80:20 ratio. EP (No Validation) uses the full training dataset whereas EP (Validation) is trained using 80% of the training data. Once the model is trained with 80% of training data using EP, we further calibrate it using the aforementioned calibration techniques. Table 10 shows the results. There are two key observations: (i) the classification accuracy decreases for all calibration techniques at the expense of improving calibration performance as they require separate validation set, and (ii) DRE achieves the best ECE all cases, which further justifies its strong calibration performance.
Table 10: Different calibration techniques on the top of EP Algorithm with K. = 9%.
Cifar10
Cifar100
Approach
ResNet50
ResNet101
ResNet101
ResNet152
ACC
E33
ACC
333
ACC
E3E
ACC
333
TS
93.42 0.96 93.42
1.37
73.06
1.72
73.40
2.45
ETS
93.42
0.97
93.42
1.37
73.06
1.76
73.40
2.40
IROvA-TS
89.90
1.45
88.69
0.89
60.87
1.56
60.77
2.86
LS
94.06
7.56
94.21
7.41
75.96
9.36
76.40
7.71
EP (No Validation)
94.20
3.97
94.35
4.03
75.05
14.62
75.68
14.41
EP (Validation)
93.42
4.46
93.42
4.83
73.06
15.56
73.40
15.88
DRE
94.60
0.7
94.28
0.7
74.68
1.20
74.37
2.09
D.9
Ablation Study
In this section, we first show the impact values on the prediction and calibration performance. We then compare the performance of proposed DRE with respect to dense ensemble techniques. Finally, we show how proposed DRE avoids learning from noisy features by considering the dataset containing explicit spurious features.
Impact the uncertainty set size. For simplicity, we always keep one sparse sub-network in our framework to be with λ₁ - co. The ECE performance with respect to different sets of À value for the remaining sub-networks is shown using the heatmap given Figure (a-b). As can be seen, important to choose X2 and 0g with very distinct values to achieve low calibration error.
21

---

Sparse Ensemble vs Dense Ensemble. To more clearly demonstrate the advantage of using the proposed DRE compared to the dense ensemble, we have conducted additional experimentation and present the results in Table [1 The Dense Ensemble (w/o DRO) refers to the one, where we ensemble multiple dense networks and each one is trained using the standard ERM loss. The Dense Ensemble (w/ DRO) is the one, where we train multiple dense networks but using the DRO loss given by Eq. ① As can be seen, the proposed sparse ensemble (i.e., DRE) clearly outperforms the dense ensemble to a large extent. It should be noted that dense ensemble (with DRO) only achieves a slightly better ECE score as compared with dense ensemble (w/o DRO). This is because, it is more difficult to further diversify different dense networks with the exactly same architectures (i.e., nodes and connections). In contrast, using sparse training, we can naturally pick very distinct sparse subnetworks from the original dense network to increase the diversity, where each subnetwork is already better calibrated because of the reason explained above. Additionally, thanks to the distributionally robust ensemble, we can further diversify the learned subnetworks leading much better calibration performance. This result also helps to further justify our key motivation for developing sparse ensembles by showing how dense networks are poorly calibrated, resulting from the memorization effect introduced by the perr-paraeterized architecture.
Table 11: BCE and Accuracy comparison between dense ensemble and DRE.
Ci
C0far
Approach
ResNet50
ResNet101
ResNet101
ResNet152
ACC
E3E
ACC
ECE
ACC
ECE
ACC
ECE
Dense Ensemble (w/ DRO)
94.99
3.95
95.29
3.69
78.38
9.11
78.40 9.01
Dense Ensemble (w/ DRO)
94.35
3.90
94.30
2.97
77.69
8.23
77.79 7.57
DRE
94.87
1.71
94.74
1.34
75.86
4.90
76.46 5.81
Table 12: Training, validation, and testing data distribution in the waterbird dataset.
Background
Training
Validation
Testing
Waterbird Landbird
Waterbird
Landbird
Waterbird
Landbird
Waterbackground
1057
184
133
466
642
2225
Landbackground 56
3498
133
467
642
2255
Performance with respect to explicit spurious features. To demonstrate how our proposed approach improves the calibration by avoiding the noisy (spurious) features, we conduct additional experiments on the waterbird dataset (26), which contains explicit spurious correlations. Specifically, in this dataset, there are two classes: (a) waterbird and (b) landbird. Most of the waterbirds images are taken in the water background whereas landbirds images are taken in the land background. Hence, the model will have a tendency to make an association between the background and the type of bird instead of focusing on the true underlying features of the birds (e.g., color of the feature, etc.). Table 12 summarizes the the data distribution. There are limited data samples without the spurious correlation in the training set and therefore the model is likely to predict based on the background instead using the true features. Compared to the training set, the validation and testing sets are less skewed and therefore evaluation on testing set will no longer be favored by only focusing on the spurious correlation.
Table [3 shows the performance from the sparse network ensemble SNE and DRE with a 15% total sparsity. In the table. Original indicates
Table 13: Performance on waterbird dataset.
Approach
Original
Spurious Only
Non-spurious Only
the original testing set provided in Table [2]
ACC
ECE
ACC ECE
ACC ECE
whereas Spurious Only the one where only data samples with spurious correlations (i.e., wa- terbird on a water background and landbird on a land background) are considered and Non-spurious only indicates the samples without spurious correlations (i.e., waterbird on a land background and landbird on a water background). There are two key observations. First, SNE performs similarly with the DRE in case of samples holding spurious correlations (i.e., spurious only) where the overconfident predictions are usually preferred as they most likely to be correct benefiting from the spurious correlation. Second, in the case
SNE
77.64 13.78
85.95
9.50
64.17
24.89
DRE
77.83 8.32
84.76
8.67
66.30 19.96
22

---

of non-spurious only, DRE achieves better performance both in terms of accuracy and ECE, which justifies that our model indeed learns from important features instead spurious correlations. In the original test set, because of the large number of samples holding spurious correlations, we do not see a clear advantage in terms of accuracy. However, DRE still achieves a clearly better calibration performance compared to SNE.
We have further visualized the the heatmap convolution 4 layer using the Grad-Cam technique. As shown in the Figure 7 (a), the sparse sub-network in the SNE focuses on the water background instead of focusing on actual landbird object. This is because, during training process, the sparse sub-netue uub-network in SNE is likely to learn to associate the spurious background feature with the true label. Specifically, the model learns to predict landbird whenever there a land background and waterbird whenever there is a water background. In contrast, using the DRE technique, as demonstrated through the heatmap, the sparse sub-network focuses on the actual object instead of the background. worth mentioning that, because of the overfitting phenomenon and lack of systematic way for diversification, each sparse sub-network the SNE behaves in similar way by focusing on the spurious feature instead of the actual object. In contrast, in the case of DRE, each sparse sub-network controlled by the parameter in Eq. (2). Specifically, we use a low r) value (i.e., + 0) for one of the sparse sub-networks, which will be similar to that of the sparse sub-network obtained using the SNE. However, for the higher value, it will focus on learning from more difficult samples, including those not holding the spurious correlations. As such, the sparse sub-nes network is forced to learn from the actual object instead of through the background. Therefore, the model focuses mostly on actual objects demonstrated in Figure (b). When these diverse sparse sub-networks are combined in the DRE, achieves a better calibration without being confidently wrong like the SNE.
(a) SNE sparse sub-network
(b) DRE sparse sub-network
Figure 7: Visualization of the layer 4 convolution using the Grad-Cam technique 229 on a landbird data sample: (a) SNE focuses on the water background, (b) DRE focuses on the actual object.
D.10 Parameter Size and Inference Speed
Table 14: Parameter size and inference speed.
We compare parameter size and inference speed of different types of sparse networks. Table shows the FLOPS along with number of param- eters associated with each technique. As can be seen, the proposed DRE has comparable parameter size as that of the sparse network en- semble. terms of computational times, our approach is comparable to the sparse network ensemble. Compared to a dense network, our technique has much smaller parameter size with less FLOPS.
ResNet50
ResNet101
Approach
Params
Flops (x10P)
Params
Flops (x (x10%)
Densel
23.6M
4.14
42.5M
7.88
SNE DRE
3.5M
1.31
6.3M
2.53
3.5M
1.31
6.3M
2.53
D.11 Diversity on Sparse Sub-networks
To justify our claim that our technique ensures the diverse sparse sub-networks, we adapt the disagreement metric (daist) from [18). This metric measures the disagreement among sub-networks terms of class label prediction. Table 15 below shows the results for Cifar and Cifar 100 datasets. As shown, compared to Sparse Network Ensemble, DRE achieves higher disagreement which implies that the sparse sub-networks are more diverse.
D.12 Qualitative Analysis
As demonstrated in the qualitative analysis section of the main paper, approach is able to generate the less confident prediction in the case incorrectly samples and thereby making model better calibrated. Figure (a)-(d) show the confidence values of the wrongly classified samples from
23

---

Table 15: Accuracy. ECE, and prediction disagreement performance with a 15% density.
Cifar10
Cifar100
Approach
ResNet50
ResNer101
ResNet101
ResNet152
ACC
ECE daist
ACC
ECE
daist
ACC
ECE
daist
ACC
ECE
ddist
SNE
94.85
3.05
0.048
94.96
3.18
0.049
76.82
11.12
0.20
77.23 11.63
0.20
DRE (Ours)
94.87
1.71 0.088
94.74
1.34
0.069
75.86
4.90
0.24
76.46
5.81
0.24
of tamples
Number of Samples
Semplex
600
500
800
600
400
400
300
200
200
200
200
100
0.0
0.0
0.0
0.0
Confidence
Confidence
Confidence
Confidence
(a)
Dense network
(b) EP sparse network
(c) sparse network
(d) SNE
6000
7000
6000
6000
of Samples
5000
Camplee
5000
of Camples
5000
Samplex
5000
0000
6000
3000
3000
3000
2000
2000
2000
1000
1000
1000
0.0
0.0
Confislence
Confidence
Confidence
Confislence
(c) Dense network
(f) EP sparse network
(g) EP sparse network
(h) Sparse ensemble
Murbber of Samples
6000
5000
Number of Samples
2500
5000
5000
4000
5000
1500
3000
Wamber
2000
2000
2000
1000
1000
1000
0.0
0.0
0.5
1.0
0.5
1.0
0.0
0.5
1.0
(i) DRO sparse network (i) DRO sparse network (k) DRO sparse network 3
(I) DRE
Figure 8: Confidence scores of correctly classified samples in baseline models (a)-(d); confidence scores of correctly classified samples in CIFAR100 with ResNet101: (c)-(1)
the baseline models, which are concentrated on the higher end as compared with DRE. We further show that DRE behavior in the correctly classified samples in Figure & (e)-(I). As can be seen, confidence score of correctly classified data samples from the CIFAR dataset with different techniques, As shown, our DRE technique remains confident on the correct data samples while being not confident on the incorrect data samples. This result shows our approach well calibrated and trustworthy compared with the competitive baselines. In summary, uncertain for incorrect samples while being confident on the correct samples resulting in a much improved calibration.
proposed technique remains
E Broader Impact, Limitations, and Future Work
In this section, we first describe the potential broader impacts of our work. We then discuss the limitations and identify some possible future directions.

---

E.1
Broader Impact
Sparse network training provides a highly promising way to significantly reduce the computational cost for training large-scale deep neural networks without sacrificing their predictive power. Besides energy savings, also opens the gate for deploying deep neural networks lightweight computing or edge devices that can further broaden the applications of Al in more diverse and resource constrained settings. The proposed robust ensemble framework provides a general solution to achieve calibrated training of deep learning models. As a result, the trained model is expected to provide more reliable uncertainty predictions, which could be important step towards using Al in safety-critical domains.
E.2 Limitations and Future Works
As an ensemble model, DRE involves multiple base learners (i.e., sparse sub-networks). Consequently, may lead more computational overhead. This could create issues for real-time application as during the inference time, the input needs to be passed through all base learners to get the final output, which can slow down the prediction speed. straightforward way speed up the inference process is to execute all the base learners in parallel, which still incurs additional computational overhead. One interesting future direction to investigate knowledge distillation and train a single sparse network from the ensemble model. Theoretical evidence [ shows that knowledge distillation has the potential to largely maintain ensemble performance while providing promising way to train a single sparse network with an even higher sparsity level and improved inference speed.
F
Source Code
For the source code of this paper, please click here
25


Reviews:
[{"review_id": "SiSI3w4HDE", "reviewer": "Reviewer_TYXu", "summary": "The authors propose a method of sparse training of deep neural networks with the objective of confidence calibration. The method is based on learning an ensemble of sparse models where each model begins with the same training dataset, and is increasingly diversified such that each model in the ensemble is trained on different sets of hard/rare data samples. The authors walk through a theoretical argument on why they believe this helps to improve confidence calibration based on prior work on understanding spurious correlations and training. Results are demonstrated on Res/Nets with CIFAR-10, CIFAR-100 and Tiny ImageNet for ensemble models of 91% and 85% sparsity (considered over all models in ensemble). Results are compared with sparse training baselines, such as L1 pruning, CigL, Sup-Ticket and DST Ensemble, along with Sparse Network Ensemble and finally a single dense model.", "questions": "- What is the motivation of using sparse v.s. dense ensembles in the proposed method?\n- What is the effect of the different aspects of the proposed method? i.e. robust loss, sparse v.s. dense training, and ensembles.\n- Why are dense ensembles not a baseline?", "limitations": "The authors did not AFAIK address any such limitations, although improving confidence calibration of neural networks would have positive societal impact overall I believe.", "rating": 5, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "- The authors focus on confidence calibration, and evaluate their results not just in a typical classification context, but also in out-of-distribution and open-set distributions where the effect of confidence calibration is well-motivated and clear.\n- Results are on reasonable models and datasets, with mostly appropriate baselines (at least on the sparsity side)\n- Results suggest compared to dense and most sparse baselines the proposed method improves confidence calibration significantly.\n- The proposed \"Robust loss\" which focuses on ensembles learning from a diverse set of difficult examples found during training is intuitive.\n- Reasonable presentation overall, although could be cleaned up and made easier to understand.", "weaknesses": "- I found the motivation of using sparse ensembles (v.s. a dense ensemble) lacking, with no clear reason why the proposed method would not work for a dense ensemble also, and notably the lack of a dense ensemble as a baseline. If the motivation is improved training performance, there is no description or qualification of the improved performance, with a comparison of the dense baseline to understand any potential tradeoffs, if the motivation is improved calibration compared to dense ensemble, then similarly this should be also be explicitly shown.\n- No ablation analysis of different effects of various aspects of the proposed method, i.e. sparse ensemble v.s. dense ensemble v.s. proposed robust loss, in particular sparse ensemble v.s. robust loss being the most important.\n- I found the \"Theoretical Analysis\" to be quite far from theoretical, and more motivation for the proposed method than a proof of any sort. Indeed considering how poorly spurious correlations are understood in the field, I believe the author's apparent claims that (a) the over-confidence of neural networks is due to spurious correlations alone, and (b) the approach proposed by the authors addresses spurious correlations can only be an over-claim. While this section is fine as a motivation for the method, this cannot be claimed to be a proof or formal demonstration that the method reduces spurious correlations in general.\n- What appears to be one of the most important/strongest baselines (DST Ensembles) is missing from all but the CIFAR-10 and CIFAR-100 classification results. \n- For TinyImageNet a sparse ensemble without the training data curriculum appears to have a relatively similar effect at the highest sparsity level as the proposed method on improving confidence calibration drastically, again causing me to wonder about ablation analysis of the method.\n\nMinor issues:\n- Writing is fairly verbose and in some cases a bit repetitive, could be made more easy to understand/compact.\n- Confusing usage of \"density\" rather than \"sparsity\" unlike most of the literature in the field.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We would like to thank the reviewer for the valuable comments/suggestions. We summarize our responses as follows.\n\n\n**Q1: Motivation of using sparse ensemble.**\n\n\nPlease refer to our answer to Q1 in the general response. In addition to a better calibration performance, sparse ensembles also achieve a significant computational advantage when compared with dense ensembles. Specifically, using DRE we only use 15\\% of the weights present in a single dense network leading to a significant reduction in the computational cost. As shown in Table 10 of the paper, using a dense ensemble with 3 dense networks, we will have a total of 12.42 ($\\times 10^{9}$) FLOPs whereas DRE only uses 1.31 ($\\times 10^{9}$). \n\n\n\n\n\n\n**Q2:  Ablation study on sparse ensemble vs deep ensemble.**\n\nThank you for this suggestion. In our response to Q1, we have compared the proposed sparse ensemble DRE with (1) a standard dense ensemble with each dense network trained using the ERM loss and (2) a dense ensemble trained using the proposed robust loss. The results ((Table 1 in attached file)) confirm that DRE is able to achieve a much better calibration performance than dense ensembles. We explain the reason for the performance difference in our answer to Q1 as well. \n\n\n\n\n**Q3: Importance of theoretical Analysis.**\n\nPlease refer to our answer to Q2 in the general response. To add more empirical evidence that supports our theoretical contribution, we have conducted additional experiments on the WaterBird dataset that contains explicit spurious correlations [7]. As shown in Table 4 (b) (in **the attached pdf file of the general rebuttal**), the proposed DRE greatly improves the calibration performance compared to the sparse network ensemble (SNE) in testing samples, where there is no spurious correlations (last two columns) between the background and the bird in the image. These are the samples with waterbirds in a land background or landbirds in a water background. The result justifies that our model indeed learns from important features instead of spurious correlations. In contrast, in the case of the testing samples holding spurious correlations, i.e., waterbird in a water background and landbird in a land background (Spurious columns in Table 4 (b)), DRE and SNE achieve a comparable performance. This is because overconfident predictions are preferred on these testing samples as they are most likely to be correct benefiting from the spurious correlation.\n\n\n**Q4. DST Ensemble Baseline.**\n\nThank you for the comment. We would like to clarify that we have included the comparison with the DST ensemble for Cifar10, Cifar100, as well as TinyImageNet. For the classification task, the \n DST ensemble result for TinyImageNet is reported  in Table 6 of the Appendix. Furthermore, in the case of the out-of-distribution experiments on Cifar10 and Cifar100, the reason for not including the DST ensemble method is provided in lines 342-344. Specifically, in the case of the Cifar100-C, the accuracy using DST is more than **11\\%** lower than that of the DRE in both architectures whereas in the case of Cifar10-C, the accuracy using DST is more than **6\\%** lower than that of the DRE using both architectures. \n\n **Q5. For TinyImageNet a sparse ensemble without the training data curriculum appears to have a relatively similar effect at the highest sparsity level as the proposed method on improving confidence calibration drastically...**\n\n Thank you for pointing out this important observation. First of all, this provides another important empirical evidence that, compared to the dense networks, sparse models and the ensemble thereof (i.e., SNE) help to improve calibration. Second, while SNE achieves significant improvement as compared with other baselines, the proposed DRE further outperforms SNE to a large extent in most cases. For example, on the WideResNet101 backbone, the ECE score of SNE is still two times of DRE; for the sparsity level at 15\\%, the ECE scores of SNE are 5-6 larger than those of DRE. The results are consistent over all datasets on all types of backbones. \n \n \n\n**Q6. The authors did not AFAIK address any such limitations...**\n\nWe apologize for the confusion regarding limitations. In Appendix E.2, we have discussed the limitations and potential future extensions of our work.  \n"}, {"review_id": "UhqbAHRA66", "reviewer": "Reviewer_xGvL", "summary": "The paper utilizes Distributionally Robust Optimization (DRO) framework to achieve an ensemble of lottery subnetworks for better calibration performance. Recently developed sparse network training methods, such as Lottery Ticket Hypothesis (LTH) and its variants, largely focus on sparsifying deep networks and realizing comparable accuracy to dense counterparts but neglect network calibration. The proposed DRO ensemble aims to learn multiple diverse and complementary sparse sub-networks with the guidance of uncertainty sets, which encourage tickets to gradually capture different data distributions from easy to hard and naturally complement each other. The authors theoretically justify the strong calibration performance of their proposed robust training process and show extensive experimental results on several benchmarks, demonstrating clear calibration improvement without sacrificing accuracy or burdening inference costs. Experiments on out-of-distribution (OOD) datasets also demonstrate the robustness of their approach in the open-set environment.", "questions": "Please refer to Weakness.", "limitations": "Yes.", "rating": 5, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The idea of using multiple subnetworks to the ensemble for better calibration performance is novel, and the procedure does not need to first train the dense network.  The authors also give a theoretical analysis of the method.  The authors conduct extensive experiments to validate the proposed method. ", "weaknesses": "For the out-of-distribution comparison, the authors miss some important methods to compare such as  [1]. All the experiments are conducted on ResNet architectures, could the authors provide results for some other architectures? And could the proposed method be conducted with filter-level subnetworks?\n\n[1] Zhang, Dinghuai, et al. \"Can subnetwork structure be the key to out-of-distribution generalization?.\" International Conference on Machine Learning. PMLR, 2021.\n[2] Zhou, Xiao, et al. \"Sparse invariant risk minimization.\" International Conference on Machine Learning. PMLR, 2022.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We would like to thank the reviewer for the valuable comments/suggestions. We summarize our responses as follows.\n\n**Q1: Comparison with OOD Works [1, 2].** \n\nThank you for pointing out these important references, which we would like to include and discuss in the related works section of the revised paper. Here, we would like to highlight some key differences from our work.\n\n- First, the training procedure of [1] also follows the dense pretraining  paradigm, where dense network pretraining is performed followed by the pruning and finetuning of the pruned network. This process is similar to the LTH work, which we have already included as a comparison baseline. As we can see from Table 2, dense pretraining techniques tend to perform poorly on OOD data leading to a lower ECE score. The underlying reason for the poor calibration on OOD data is associated with the dense network pretraining and is also explained by [2]. Through dense network training, the model may have already picked the spurious and noisy features while paying less attention to the important ones. As such, during the pruning phase, the subnetwork is searched by giving preference to the spurious and noisy features, leading to a suboptimal calibration performance. Compared to this, our proposed DRE does not rely on the pretrained dense network. The chance of finding sparse subnetworks that capture important features is also enhanced thanks to the DRO based training process.\n\n\n- In the case of [2], the authors propose Sparse Invariant Risk Minimization (SIRM) that applies sparse techniques on the top of IRM to extract the invariant features in the desired sparse subnetwork. This paper demonstrates that IRM alone is not enough to avoid spurious features and therefore sparsity is required during the training process to learn from the invariant features. As such, the resulting subnetwork will have an enhanced generalization capability.  Notably, the framework in [2] can not be compared directly with our DRE technique as IRM training requires datasets from multiple environments. Meanwhile, SIRM is a generic technique that can be integrated with any desired sparse training strategies. Therefore, SIRM is complementary and can be used together with our DRE framework.\n\n\n\n**Q2: All the experiments are conducted on ResNet architectures, could the authors provide results for some other architectures?**\n\nThank you for raising this great point! In fact, we have conducted an ablation study that includes investigates the impact of different architectures, and the results are summarized in Table 9 of Appendix D.8.  Using a vision transformer (i.e., ViT), DRE still achieves a much lower calibration error than EP. However, using ViT as a backbone, the accuracy from both EP and DRE is lower and ECE is higher than other backbones. It has been shown by existing works that without pretraining, the lack of useful inductive biases for ViT can cause a performance drop. Since no pretraining is conducted in both EP and DRE, it causes a lower accuracy (and a higher ECE).\n\n\n**Q3: Proposed method on filter-level subnetworks.**\n\nThank you for this thoughtful comment! In our experimentation, backbones contain convolution layers and we have explicitly enforced sparsity in each convolution module. If we understand this question correctly, we believe filter-level subnetworks indicate the sparse subnetworks and if so, we have already considered filter-level sparsity in our framework. One of the closest works that analyze the implicit sparsity in the filter level is [3]. In this paper, the authors empirically show the sparsification of convolution neural networks mainly with respect to (a) Regularizer (weight decay and $L_2$),  (b) Optimizer (SGD, Adam, Adadelta, and Adagrad), and (c) Difficulty of the task. They found that the adaptive optimizers (e.g., Adam, Adadelta, and Adagrad) learn sparser network representation compared to SGD. Also, no sparsity is observed in the absence of regularizers like $L_2$ and weight decay. Also, the sparsity depends on the interplay between the regularizer and optimizer. For example, $L_2$ shows a higher sparsity for comparable performance than weight decay when using the Adam optimizer, whereas using SGD the difference is not significant using weight decay and $L_2$. Furthermore, they showed that it is difficult to obtain a sparser network for difficult tasks whereas higher sparsity can be more easily obtained using less difficult tasks (e.g., Cifar10 generates higher sparsity compared to Cifar100). However, the analysis does not seem to align with our goal, which aims to improve the calibration ability of sparse networks whereas, [3] primarily focuses on analyzing the implicit sparsity achieved by considering different factors. We have conducted an experimentation considering different optimizers and weight decay coefficient in the Cifar10 dataset with ResNet50 architecture As shown in Table 4 (a) (in **the attached pdf file of the general rebuttal**), simply using the different optimizers with varied weight decay coefficients does not help to improve the calibration. \n\n**References**\n\n- [1] Zhang, Dinghuai, et al. \"Can subnetwork structure be the key to out-of-distribution generalization?.\" International Conference on Machine Learning. PMLR, 2021. \n\n- [2] Zhou, Xiao, et al. \"Sparse invariant risk minimization.\" International Conference on Machine Learning. PMLR, 2022.\n\n    \n- [3] Mehta et al. \"On Implicit Filter Level Sparsity in Convolutional Neural Networks\". CVPR2019."}, {"review_id": "mvsojsS8xA", "reviewer": "Reviewer_RrYi", "summary": "In this paper, the author proposes a Distributionally Robust Optimization (DRO) framework, which utilizes the ensemble of multiple sparse sub-networks to improve the network calibration. The author argues that the previous ensemble method, i.e., AdaBoost, will make the sub-network severely underfit the training data, leading to a rather poor generalization capability. To solve this problem, the author proposes Distributionally robust ensemble (DRE) method to obtain complementary sparse sub-networks.", "questions": "Please refer to the weakness part.", "limitations": "N/A", "rating": 3, "confidence": 5, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "* The proposed method is evaluated on different datasets and networks.\n\n* The proposed method achieves better accuracy than the baseline works compared.", "weaknesses": "* The author argues that the current sparse training works (LTH and EP) have two limitations, which are: (1) the requirement of pretraining a dense network and (2) the learning objective remains as improving the accuracy up to the original dense networks.\nHowever, the author seems completely ignore the very popular (static/dynamic) sparse training methods, which directly train a sparse network from scratch and does not require extra training epochs for iterative pruning and growing, e.g., [1] [2] [3] [4] [5]. It seems that the author argued limitations are not in those sparse training methods.\n\n[1] SNIP: SINGLE-SHOT NETWORK PRUNING BASED ON CONNECTION SENSITIVITY\n\n[2] Parameter Efficient Training of Deep Convolutional Neural Networks by Dynamic Sparse Reparameterization\n\n[3] PICKING WINNING TICKETS BEFORE TRAINING BY PRESERVING GRADIENT FLOW\n\n[4] Rigging the Lottery: Making All Tickets Winners\n\n[5] MEST: Accurate and Fast Memory-Economic Sparse Training Framework on the Edge\n\n\n* It is not clear what the content of Fig.1 means. There are no legends for those strips.\nIt would make the paper easier to read if the author can explain what the “network calibration” means at the beginning of the introduction section.\n\n\n* I think this is yet another paper that abuses term of “Lottery Tickets”. I really cannot find the connection between the proposed method and the Lottery Tickets hypothesis. This makes reader feel confusing.\n\n\n* I don’t think it is fair to compare the proposed method with the (original) LTH method. \n\n* The author argues that the methods using pretraining – pruning are costly. But the author does not provide comparison results about the training costs (FLOPs) of the proposed method. It is not clear the 200 training epochs is for each sub-network or for the entire training process.\n\n* It is inappropriate to call the inference FLOPs the inference speed.\n\n* It seems that the prior work [6] is highly related to the proposed method. But the author does not provide any discussion about it. (Only compared with it in the results part.)\n\n[6] Calibrate and prune: Improving reliability of lottery tickets through prediction calibration.\n", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We would like to thank the reviewer for the valuable comments/suggestions. We summarize our responses as follows.\n\n\n**Q1: Authors completely ignore very popular (static/dynamic) sparse training methods.**\n\nPlease refer to the answer to Q3 of the general response. To more clearly demonstrate this critical limitation from existing sparse training methods, we conduct additional experiments to evaluate the calibration performance of all the methods suggested by the reviewer. The results are summarized in  Table 2 (refer to **the attached pdf in the general rebuttal**). It is clear that all these methods achieve an ECE score similar to another representative sparse training method EP, which is much worse than the proposed DRE framework. Therefore, our work proposes a novel contribution to achieve calibrated sparse network training that is orthogonal and complementary to existing sparse training methods.\n\n\n\n**Q2: Clarity of Figure 1.**\n\nIn Figure 1, we use the standard Expected Calibration Error (ECE) plot, which is commonly used to visualize the calibration behavior of a model (see Figures 1 \\& 4 in [7] as an example). In the ECE plot, the dashed diagonal line indicates that the model is perfectly calibrated where the model's accuracy exactly matches its confidence. The shaded area using the strips indicates that the model's confidence is higher than accuracy, which implies the overfitting behavior. The sky blue shows the model's accuracy for the given confidence value. If the accuracy exceeds the diagonal line, it means that the model's accuracy is higher than its confidence, which implies that the model tends to provide less confidence predictions.\n\n\n\n**Q3: Connection between the proposed method and the Lottery Tickets hypothesis.**\n\nWe would like to cite several relevant works from existing literature to clearly show the connection between the proposed method and LTH. According to [8] and [9], the strong Lottery Tickets Hypothesis states that \"there exists a subnetwork in a randomly initialized neural network such that it already achieves almost the same accuracy as a fully trained network, without any optimization of the weights of the network\". The proposed method builds upon Edge Popup, which directly finds a sparse sub-network from a randomly initialized dense network without pre-training and iterative pruning. Therefore, it matches the definition of strong Lottery Tickets Hypothesis. \n\n\n**Q4: It is not fair to compare the proposed method with LTH method.**\n\nAs a representative sparse network training method, we empirically show that LTH also suffers from poor calibration performance through our comparison. Furthermore, compared with LTH, our technique does not involve pre-training followed by iterative pruning and finetuning, making it computationally more efficient. Since we keep all other factors (e.g., data split) the same, we would like to kindly clarify that this is a fair comparison. \n\n\n**Q5: Training cost of the proposed method.** \n\nWe would like to make it clear that each sparse subnetwork is trained using 200 epochs. It should be noted that there is no dependency among those sparse subnetworks (their focus on the data space is automatically adjusted by the DRO framework) and therefore can be run independently making overall training time more efficient. In contrast to our work, LTH variants (especially methods using pretraining-pruning) typically involve sequential process and therefore cannot be run parallel making them time-consuming. \n\n\n**Q6: Inappropriate to call the inference FLOPs the inference speed.**\n\n\nThank you for the suggestion. We will fix the issue and change to inference FLOPs in the revised paper.\n\n**Q7: Discussion about prior work in [6].**\n\nThank you for the suggestion and we will add a discussion of [6]. We would also like to clarify that [6] directly applies several commonly known calibration techniques on the top of the standard LTH model. Since it does not introduce any new technique, we only briefly discuss it when describing the performance comparison (lines 311-312). \n\n\n\n**References**\n\n- [1] SNIP: SINGLE-SHOT NETWORK PRUNING BASED ON CONNECTION SENSITIVITY\n\n- [2] Parameter Efficient Training of Deep Convolutional Neural Networks by Dynamic Sparse Reparameterization\n\n- [3] PICKING WINNING TICKETS BEFORE TRAINING BY PRESERVING GRADIENT FLOW\n\n- [4] Rigging the Lottery: Making All Tickets Winners\n\n- [5] MEST: Accurate and Fast Memory-Economic Sparse Training Framework on the Edge\n\n- [6] Calibrate and prune: Improving reliability of lottery tickets through prediction calibration.\n\n- [7] Guo et al. \"On Calibration of Modern Neural Networks\". ICML2017.\n\n- [8] Ramanujan et al. \"What’s Hidden in a Randomly Weighted Neural Network?\". CVPR2020.\n\n- [9] Chijiwa et al. \"Pruning Randomly Initialized Neural Networks with Iterative Randomization\". NeurIPS2021.\n"}, {"review_id": "DaBE4qiklI", "reviewer": "Reviewer_fvyR", "summary": "- The authors proposed a novel Distributionally Robust Optimization (DRO) framework to achieve an ensemble of lottery tickets toward calibrated network sparsification.\n- The proposed DRO ensemble aimed to learn multiple diverse and complementary sparse sub-networks with the guidance of uncertainty sets, which encourage winning tickets to capture different data distributions from easy to hard gradually.\n- The authors theoretically justified the strong calibration performance by showing how the proposed DRO guarantees to lower the confidence of incorrect predictions.\n- Extensive experimental results on several benchmarks demonstrated that the proposed DRO leads to a clear calibration improvement without sacrificing accuracy and burdening inference costs.", "questions": "(-) line 155 (typos): to follow a multivariate Gaussian distribution with the red dot representing its mean? → gray dot?\n(-) line 164: I could not understand the following sentence: “Starting from the second sub-network, the training distribution is changed according to the losses.”\n\n(-) The subnetwork’s feature-level analysis could help understand this work.", "limitations": "Please, see above the weaknesses and questions.", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "(+) The authors proposed a novel sparse ensemble framework (DRO) that combines competitive sparse sub-networks to achieve better calibration performance with the scheduling of the learning of complementary ensemble sub-networks (tickets).\n\n(+) The proposed robust training process guaranteed to lower the confidence of incorrect predictions and strong calibration performances.\n\n(+) Extensive empirical results demonstrated the proposed lottery ticket ensemble's effectiveness in competitive classification and open-set detection.", "weaknesses": "(-)The proposed DRO conducted experiments using (M==3) subnetworks and showed each sub-networks performance and confidence scores (Appendix). However, there is no ablation study on these subnetworks regarding V or V’ feature representations. These representations could help understand the subnetwork’s ensemble better.\n\n(-) Line 258: (Regarding feature representations), is there any evidence or observation that the ERM model learns to memorize some noisy feature v’ introduced through specific spurious correlations?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We would like to thank the reviewer for the valuable comments/suggestions. We summarize our responses as follows.\n\n**Q1: Ablation study regarding V or V’ feature representations.**\n\nThank you for this great suggestion! Following the reviewer's idea, we conduct additional experiments on the Waterbird dataset [1], which contains explicit spurious correlations. Specifically, in this dataset, there are two classes: (a) waterbird and (b) landbird. Most of the waterbirds images are taken in the water background whereas landbirds images are taken in the land background. Hence, the model will  have a tendency to make an association between the background and the type of bird instead of focusing on the true underlying features of the birds (e.g., color of the feature, etc). \nThe Table 3 (in the **attached pdf of the general rebuttal**) summarizes the the data distribution. There are limited data samples without the spurious correlation in the training set and therefore the model is likely to predict based on the background instead using the true features. Compared to the training set, validation and testing sets  are less skewed and therefore evaluation on testing set will no longer be favored by only focusing on the spurious correlation. \n\n\n\n\nThe Table 4 (b) (in the **attached pdf of the general rebuttal**)  shows the performance from  the sparse network ensemble SNE and DRE with a 15\\% total sparsity. In the table, Original indicates the original testing set provided in  Table 3 of the attached pdf file  whereas, spurious is the one where only data samples with spurious correlations (i.e., waterbird on a water background and land bird on a land background) are considered and non-spurious only indicates the samples without spurious correlation (i.e., waterbird on a land background and a landbird on water background). There are two key observations. First, SNE performs similarly with the DRE in case of samples holding spurious correlations (i.e., spurious only) where the overconfident predictions are usually preferred as they are most likely to be correct benefiting from the spurious correlation. Second, in the case of non-spurious only, DRE achieves better performance both in terms of accuracy and ECE, which justifies that our model indeed learns from important features instead of spurious correlations. In the original test set, because of the large number of samples holding spurious correlation, we do not see a clear advantage in terms of accuracy. However, DRE still achieves a clearly better calibration performance compared to SNE. \n\n\n\n\n\n\n\n**Q2: Evidence or observation that the ERM model learns to memorize some noisy feature v’ introduced through specific spurious correlations**\n\nThanks you for this insightful comments. In fact, we have provided the evidence that the ERM model learns to memorize noisy features introduced through the spurious correlations in Figure 6 of Appendix D.11. We show the number of incorrectly classified samples with respect to confidence score using different techniques. Plots a-d use the ERM model, where a majority of samples are concentrated in the high confidence region despite being incorrect. This is because, the model learns to pick the noisy features (i.e., spurious correlations which cause overfitting) in addition to the important learning signals during the training process. As such, during testing, whenever such noise features occur, the model produces overconfident predictions while ignoring the true signals. In contrast, using DRE (plot h) and DRO (plots f and g), the model becomes less confident in those wrong cases, and therefore a majority of the wrong samples are concentrated in the low-confidence region. This is because the DRO technique forces the model to learn from the important signals instead of spurious correlations. As such, the model does not learn to produce many overconfident predictions as by ERM.  \n\n**Q3: Clarity on sentence in line 164**\n\n\nIn the case of AdaBoost, we start by training the first sub-network. Then, based on its performance, we assign the importance of each data sample (based on their loss) and train the second sub-network. This means, a higher loss resulting from the first sub-network results in a larger weight to the given sample for the second sub-network, where weight indicates the probability of a given data point being sampled during training. In other words, difficult samples will appear more frequently in the second sub-network compared to the easier one. To train the third sub-network, we again assign higher weights to samples with higher losses from the second sub-network and perform training based on new data distribution based on new weights. We repeat this process. We will make this clear in the revised paper.\n\n**Q4: Typo**\n\nThanks for carefully checking our paper and identifying the typo. We will fix the issue and improve the presentation of the revised paper. \n\n**Q5: Subnetwork's feature-level analysis**\n\nThis is a great suggestion! We have visualized the features of a (non-DRO) subnetwork chooses to focus and made a comparison with features focused by a subnetwork trained through the DRO framework. Please refer to Figure 1 in **the pdf file attached with the general rebuttal**. \n\n\n**References**\n\n- [1] Sagawa et al. \"Distributionally Robust Neural Networks for Group Shifts: On the Importance of Regularization for Worst-Case Generalization\". ICLR2020."}]

Author rebuttal:
First of all, we would like to thank all the reviewers for spending time to review our paper and providing many constructive suggestions and comments. Here, we summarize our responses to some major questions raised by reviewers:

**Q1: Motivation of using sparse ensemble (Reviewer TYXu)**

Besides a high computational cost, one key motivation for developing sparse ensembles is the poor calibration of dense networks resulting from the memorization effect introduced by an over parameterized architecture as mentioned at the beginning of the introduction section. Such phenomenon has been commonly observed and discussed in recent literature (e.g., refs [9, 24] in the paper along with some other references such as [1] and [2] suggested by other reviewers). Our empirical study has further confirmed this across different datasets and backbone architectures. For example, ref [24] theoretically justifies that overparameterization in dense networks exacerbates spurious correlation leading to poor calibration; Refs [1] and [2] (see references for details)  further demonstrate that dense networks are more likely to overfit leading to poor generalization (and calibration). To avoid  overfitting,  [1] and [2] also resort to training sparse networks. Our empirical evaluation shows that sparse network training can consistently improve the calibration performance over multiple datasets in multiple architectures. As can be seen in Tables 1-3 of the paper, sparse networks achieve better ECE scores than their dense counterparts in most settings. 

To more clearly demonstrate the advantage of using the proposed DRE compared to a dense ensemble, we have conducted additional experimentation and present the results in Tables 1 presented in **the attached pdf file**. The Dense Ensemble (w/o DRO) refers to the one where we ensemble multiple dense networks, where each one is trained using the standard ERM loss. The Dense Ensemble (w/ DRO) is the one where we train multiple dense networks but using the DRO loss (i.e., Eq. 1). As can be seen, the proposed sparse ensemble (i.e., DRE) clearly outperforms the dense ensembles to a large extent. It is also interesting to note that a dense ensemble (with DRO) only achieves a slightly better ECE score as compared with a dense ensemble (w/o DRO). This is because, it is more difficult to further diversify dense networks with the exactly same architectures (i.e., nodes and connections). In contrast, using sparse training, we can naturally pick very distinct sparse subnetworks from the original dense network to increase the diversity, where each subnetwork is already better calibrated because of the reason explained above. Additionally, thanks to the distributionally robust ensemble, we can further diversify the learned subnetworks leading to much better calibration performance.

**Q2: Importance of theoretical Analysis. (Reviewer TYXu)**

We agree with the reviewer that our theoretical analysis can further strengthen our motivation. We also acknowledge that spurious correlation is only one potential source that can lead to over-confidence in neural networks (but we did not claim that *over-confidence of neural networks is due to spurious correlations alone*). Our analysis primarily focuses on this important perspective, which identifies concrete theoretical evidence on this important connection. The theoretical findings have been further confirmed in our empirical evaluations on multiple datasets over a diverse set of backbone architectures. Therefore, we believe our work is an important step forward in addressing this highly important issue by providing a theoretically sound and empirically feasible solution to effectively lower the model’s false confidence on its wrong predictions resulting from spurious correlations, as evidenced by the improved overall calibration performance. The analysis of other sources that contribute to overconfident predictions remains as an important topic for future research.  

**Q3: Authors completely ignore very popular (static/dynamic) sparse training methods. (Reviewer RrYi)**

Thank you for pointing out these sparse training methods without the need for iterative pruning/growing. To most clearly differentiate these works from our main technical contribution, we would like to re-state our primary focus, which is to **achieve calibrated sparse network training**, as indicated by the title of our paper. To this end, we propose a novel Distributionally Robust Optimization (DRO)
 framework to achieve an ensemble of lottery tickets towards calibrated network sparsification (see lines 8-10 of the abstract). Like the reviewer, we clearly recognize the existence of sparse training techniques that do not rely on pre-training and iterative pruning (see lines 33-34). We use Edge Popup (EP) as a representative of this group of methods (and the plural implies that EP is only one of such methods). However, like EP, all these methods focus on pushing the accuracy up to the original dense networks and hence still suffer from a severely overfitting behavior, leading to a poor calibration performance. This is the *novel technical gap* that we identify and aim to address using our proposed Distributionally Robust Ensemble (DRE). We have empirically shown the poor calibration performance from EP (along with some other representative sparse training models, including [3] as mentioned by the reviewer) in both Figure 1 and our experimental results in Tables 1-3 in the main paper and Tables 5-8 in the Appendix. 


**References**

- [1] Zhang, Dinghuai, et al. "Can subnetwork structure be the key to out-of-distribution generalization?." International Conference on Machine Learning, 2021.

- [2] Zhou, Xiao, et al. "Sparse invariant risk minimization." International Conference on Machine Learning, 2022.

- [3] Evci, U., et al. Rigging the lottery: Making all tickets winners. In International Conference on Machine Learning, 2020. 


Meta-review: The paper received 4 reviews, and the authors did a solid job in addressing all the reviewer comments and suggestions. This includes running new experiments to analyze performance and including more comparative methods. After the rebuttal and the discussion period, three reviewers were leaning to accept the paper. The rebuttal has addressed the majority of the reviewer concerns and the remaining issues seem to be minor. 

Reviewer TYXu raised a fair concern about the theoretical analysis and the claims being made based on it. This reviewer decided to raise their score after the rebuttal/discussion phase under the assumption that the authors phrase their claims on the theoretical analysis (as the rebuttal suggests) to be more realistic as to what is being shown in the paper and how it relates to the proposed method. This is a fair request and the authors are encouraged to address this in their final version.
Decision: Accept (poster)