SlowGuess/NeurIPS_2023_OCR huggingface dataset:
In this section, we investigate whether existing calibration techniques designed for training dense networks can be leveraged to further improve the calibration performance of sparse networks. However, most of these techniques (e.g., temperature scaling and mix-n-match) are post hoc techniques, which require a separate validation set to fine-tune the parameters. This means we need to further divide the training data into training and validation sets, which may negatively impact the generalization capability of the trained model (due to less training data). To make a comparison, we pick Temperature Scaling (TS) [9], Label Smoothing (LS) [31], and a few other techniques proposed in
(a)  $\mathcal{K} = 5\%$
Figure 6: (a-b) Impact of  $\lambda$  on ECE using ResNet101 architecture on Cifar100 dataset.
(b)  $\mathcal{K} = 3\%$
[37], including Ensemble Temperature Scaling (ETS) and Isotonoic Regression One vs All combined with Temperature Scaling (IROvA-TS). We apply these calibration techniques on the top of the EP algorithm. Specifically, as LS does not require a separate validation set, we train it on the full training dataset using the LS loss (with  $\epsilon = 0.1$ ). Other calibration techniques require a separate validation set and therefore we divide training data into training and validation with a 80:20 ratio. EP (No Validation) uses the full training dataset whereas EP (Validation) is trained using  $80\%$  of the training data. Once the model is trained with  $80\%$  of training data using EP, we further calibrate it using the aforementioned calibration techniques. Table 10 shows the results. There are two key observations: (i) the classification accuracy decreases for all calibration techniques at the expense of improving calibration performance as they require a separate validation set, and (ii) DRE achieves the best ECE in all cases, which further justifies its strong calibration performance.
Table 10: Different calibration techniques on the top of EP Algorithm with  $\mathcal{K} = 9\%$
Approach	Cifar10	Cifar100
ResNet50	ResNet101	ResNet101	ResNet152
ACC	ECE	ACC	ECE	ACC	ECE	ACC	ECE
TS	93.42	0.96	93.42	1.37	73.06	1.72	73.40	2.45
ETS	93.42	0.97	93.42	1.37	73.06	1.76	73.40	2.40
IROvA-TS	89.90	1.45	88.69	0.89	60.87	1.56	60.77	2.86
LS	94.06	7.56	94.21	7.41	75.96	9.36	76.40	7.71
EP (No Validation)	94.20	3.97	94.35	4.03	75.05	14.62	75.68	14.41
EP (Validation)	93.42	4.46	93.42	4.83	73.06	15.56	73.40	15.88
DRE	94.60	0.7	94.28	0.7	74.68	1.20	74.37	2.09
D.9 Ablation Study
In this section, we first show the impact of  $\lambda$  values on the prediction and calibration performance. We then compare the performance of proposed DRE with respect to dense ensemble techniques. Finally, we show how proposed DRE avoids learning from noisy features by considering the dataset containing explicit spurious features.
Impact of the uncertainty set size. For simplicity, we always keep one sparse sub-network in our framework to be with  $\lambda_1\rightarrow \infty$ . The ECE performance with respect to different sets of  $\lambda$  value for the remaining sub-networks is shown using the heatmap given in Figure 6 (a-b). As can be seen, it is important to choose  $\lambda_{2}$  and  $\lambda_{3}$  with very distinct values to achieve a low calibration error.
Sparse Ensemble vs Dense Ensemble. To more clearly demonstrate the advantage of using the proposed DRE compared to the dense ensemble, we have conducted additional experimentation and present the results in Table 11. The Dense Ensemble (w/o DRO) refers to the one, where we ensemble multiple dense networks and each one is trained using the standard ERM loss. The Dense Ensemble (w/ DRO) is the one, where we train multiple dense networks but using the DRO loss given by Eq. (1). As can be seen, the proposed sparse ensemble (i.e., DRE) clearly outperforms the dense ensemble to a large extent. It should be noted that a dense ensemble (with DRO) only achieves a slightly better ECE score as compared with dense ensemble (w/o DRO). This is because, it is more difficult to further diversify different a dense networks with the exactly same architectures (i.e., nodes and connections). In contrast, using sparse training, we can naturally pick very distinct sparse subnetworks from the original dense network to increase the diversity, where each subnetwork is already better calibrated because of the reason explained above. Additionally, thanks to the distributionally robust ensemble, we can further diversify the learned subnetworks leading to much better calibration performance. This result also helps to further justify our key motivation for developing sparse ensembles by showing how dense networks are poorly calibrated, resulting from the memorization effect introduced by the over-parameterized architecture.
Table 11: ECE and Accuracy comparison between dense ensemble and DRE.
Approach	Cifar10	Cifar100
ResNet50	ResNet101	ResNet101	ResNet152
ACC	ECE	ACC	ECE	ACC	ECE	ACC	ECE
Dense Ensemble (w/o DRO)	94.99	3.95	95.29	3.69	78.38	9.11	78.40	9.01
Dense Ensemble (w/ DRO)	94.35	3.90	94.30	2.97	77.69	8.23	77.79	7.57
DRE	94.87	1.71	94.74	1.34	75.86	4.90	76.46	5.81
Table 12: Training, validation, and testing data distribution in the waterbird dataset.
Background	Training	Validation	Testing
Waterbird	Landbird	Waterbird	Landbird	Waterbird	Landbird
Waterbackground	1057	184	133	466	642	2225
Landbackground	56	3498	133	467	642	2255
Performance with respect to explicit spurious features. To demonstrate how our proposed approach improves the calibration by avoiding the noisy (spurious) features, we conduct additional experiments on the waterbird dataset [26], which contains explicit spurious correlations. Specifically, in this dataset, there are two classes: (a) waterbird and (b) landbird. Most of the waterbirds images are taken in the water background whereas landbirds images are taken in the land background. Hence, the model will have a tendency to make an association between the background and the type of bird instead of focusing on the true underlying features of the birds (e.g., color of the feature, etc.). Table 12 summarizes the data distribution. There are limited data samples without the spurious correlation in the training set and therefore the model is likely to predict based on the background instead using the true features. Compared to the training set, the validation and testing sets are less skewed and therefore evaluation on testing set will no longer be favored by only focusing on the spurious correlation.
Table 13 shows the performance from the sparse network ensemble SNE and DRE with a  $15\%$  total sparsity. In the table, Original indicates the original testing set provided in Table 12, whereas Spurious Only is the one where only data samples with spurious correlations (i.e., waterbird on a water background and landbird on
Table 13: Performance on waterbird dataset.
Approach	Original	Spurious Only	Non-spurious Only
ACC	ECE	ACC	ECE	ACC	ECE
SNE	77.64	13.78	85.95	9.50	64.17	24.89
DRE	77.83	8.32	84.76	8.67	66.30	19.96
a land background) are considered and Non-spurious only indicates the samples without spurious correlations (i.e., waterbird on a land background and a landbird on a water background). There are two key observations. First, SNE performs similarly with the DRE in case of samples holding spurious correlations (i.e., spurious only) where the overconfident predictions are usually preferred as they are most likely to be correct benefiting from the spurious correlation. Second, in the case
of non-spurious only, DRE achieves better performance both in terms of accuracy and ECE, which justifies that our model indeed learns from important features instead of spurious correlations. In the original test set, because of the large number of samples holding spurious correlations, we do not see a clear advantage in terms of accuracy. However, DRE still achieves a clearly better calibration performance compared to SNE.
We have further visualized the heatmap of convolution 4 layer using the Grad-Cam technique. As shown in the Figure 7 (a), the sparse sub-network in the SNE focuses on the water background instead of focusing on the actual landbird object. This is because, during training process, the sparse sub-network in SNE is likely to learn to associate the spurious background feature with the true label. Specifically, the model learns to predict landbird whenever there is a land background and waterbird whenever there is a water background. In contrast, using the DRE technique, as demonstrated through the heatmap, the sparse sub-network focuses on the actual object instead of the background. It is worth mentioning that, because of the overfitting phenomenon and lack of a systematic way for diversification, each sparse sub-network in the SNE behaves in a similar way by focusing on the spurious feature instead of the actual object. In contrast, in the case of DRE, each sparse sub-network is controlled by the  $\eta$  parameter in Eq. (2). Specifically, we use a low  $\eta$  value (i.e.,  $\eta \rightarrow 0$ ) for one of the sparse sub-networks, which will be similar to that of the sparse sub-network obtained using the SNE. However, for the higher  $\eta$  value, it will focus on learning from more difficult samples, including those not holding the spurious correlations. As such, the sparse sub-network is forced to learn from the actual object instead of through the background. Therefore, the model focuses mostly on actual objects as demonstrated in Figure 7 (b). When these diverse sparse sub-networks are combined in the DRE, it achieves a better calibration without being confidently wrong like in the SNE.
(a) SNE sparse sub-network
Figure 7: Visualization of the layer 4 convolution using the Grad-Cam technique [29] on a landbird data sample: (a) SNE focuses on the water background, (b) DRE focuses on the actual object.
(b) DRE sparse sub-network
D.10 Parameter Size and Inference Speed
We compare parameter size and inference speed of different types of sparse networks. Table 14 shows the FLOPS along with number of parameters associated with each technique. As can be seen, the proposed DRE has a comparable parameter size as that of the sparse network ensemble. In terms of computational times, our
approach is comparable to the sparse network ensemble. Compared to a dense network, our technique has a much smaller parameter size with less FLOPS.
Table 14: Parameter size and inference speed.
Approach	ResNet50	ResNet101
Params	Flops (×109)	Params	Flops (×109)
Dense†	23.6M	4.14	42.5M	7.88
SNE	3.5M	1.31	6.3M	2.53
DRE	3.5M	1.31	6.3M	2.53
D.11 Diversity on Sparse Sub-networks
To justify our claim that our technique ensures the diverse sparse sub-networks, we adapt the disagreement metric  $(d_{dist})$  from [18]. This metric measures the disagreement among sub-networks in terms of class label prediction. Table 15 below shows the results for Cifar10 and Cifar100 datasets. As shown, compared to Sparse Network Ensemble, DRE achieves higher disagreement which implies that the sparse sub-networks are more diverse.
D.12 Qualitative Analysis
As demonstrated in the qualitative analysis section of the main paper, our approach is able to generate the less confident prediction in the case of incorrectly samples and thereby making model better calibrated. Figure 8 (a)-(d) show the confidence values of the wrongly classified samples from
Table 15: Accuracy, ECE, and prediction disagreement performance with a  $\mathcal{K} = {15}\%$  density.
Approach	Cifar10	Cifar100
ResNet50	ResNet101	ResNet101	ResNet152
ACC	ECE	d_dist	ACC	ECE	d_dist	ACC	ECE	d_dist	ACC	ECE	d_dist
SNE	94.85	3.05	0.048	94.96	3.18	0.049	76.82	11.12	0.20	77.23	11.63	0.20
DRE (Ours)	94.87	1.71	0.088	94.74	1.34	0.069	75.86	4.90	0.24	76.46	5.81	0.24
(e) Dense network
(i) DRO sparse network
Figure 8: Confidence scores of correctly classified samples in baseline models (a)-(d); confidence scores of correctly classified samples in CIFAR100 with ResNet101: (e)-(l)
(b) EP sparse network 1
(f) EP sparse network 1
(j) DRO sparse network 2
(c) EP sparse network 2
(g) EP sparse network 2
(k) DRO sparse network 3
(d) SNE
(h) Sparse ensemble
(1) DRE
the baseline models, which are concentrated on the higher end as compared with DRE. We further show that DRE behavior in the correctly classified samples in Figure 8 (e)-(l). As can be seen, the confidence score of correctly classified data samples from the CIFAR100 dataset with different techniques. As shown, our DRE technique remains confident on the correct data samples while being not confident on the incorrect data samples. This result shows our approach is well calibrated and trustworthy compared with the competitive baselines. In summary, our proposed technique remains uncertain for incorrect samples while being confident on the correct samples resulting in a much improved calibration.
E Broader Impact, Limitations, and Future Work
In this section, we first describe the potential broader impacts of our work. We then discuss the limitations and identify some possible future directions.
E.1 Broader Impact
Sparse network training provides a highly promising way to significantly reduce the computational cost for training large-scale deep neural networks without sacrificing their predictive power. Besides energy savings, it also opens the gate for deploying deep neural networks to lightweight computing or edge devices that can further broaden the applications of AI in more diverse and resource constrained settings. The proposed robust ensemble framework provides a general solution to achieve calibrated training of deep learning models. As a result, the trained model is expected to provide more reliable uncertainty predictions, which could be an important step towards using AI in safety-critical domains.
E.2 Limitations and Future Works
As an ensemble model, DRE involves multiple base learners (i.e., sparse sub-networks). Consequently, it may lead to more computational overhead. This could create issues for real-time application as during the inference time, the input needs to be passed through all base learners to get the final output, which can slow down the prediction speed. A straightforward way to speed up the inference process is to execute all the base learners in parallel, which still incurs additional computational overhead. One interesting future direction is to investigate knowledge distillation and train a single sparse network from the ensemble model. Theoretical evidence [1] shows that knowledge distillation has the potential to largely maintain the ensemble performance while providing a promising way to train a single sparse network with an even higher sparsity level and improved inference speed.
F Source Code
For the source code of this paper, please click here.