Paper: Diversifying Deep Ensembles: A Saliency Map Approach for Enhanced OOD Detection, Calibration, and Accuracy (ICLR-2024)

Reviewer comment (Experiments):
SDDE performs only with CNNs because it GradCAM that is applied to CNN layers.

Author response:
We appreciate your comment regarding the application of SDDE solely with CNNs due to the use of GradCAM which is typically applied to CNN layers. However, we would like to highlight that there are saliency maps for transformers [1] and GradCAM, being a versatile method for visualizing feature maps, can be applied to features of any dimensionality.

In the context of computer vision tasks, which is our focus in this study, CNNs continue to be a stalwart choice [2] due to their strong performance, popularity in the research community, and computational efficiency. They are often favored for their speed compared to alternatives, making them a viable and relevant model to use in our studies with SDDE and GradCAM. Moreover, CNN architectures are a part of OpenOOD benchmark, which we follow in our paper. We hope this clarifies our choice of models.

Reviewer comment (Experiments):
The main hypothesis that ensemble diversity is proportionally related to the diversity of saliency maps, might be invalid for other datasets. The validation process proposed in the paper might not work for other cases.

Author response:
Thank you for your insights. We acknowledge the concern regarding the validity of our hypothesis across varied datasets. However, our work is based on the well-tested OpenOOD benchmark, ensuring diverse domain coverage.

Reviewer comment (Experiments):
SDDE can be applied to classification algorithms because CAMs are computed on the predicted classes.

Author response:
We agree with the remark about the applicability of SDDE to classification algorithms due to the computation of CAMs on predicted classes. Moreover, in our work we follow existing open-source benchmark for OOD detection (OpenOOD [4]), that is constituted of image classification tasks. Furthermore, we'd like to note that there exist GradCAM-based saliency maps methods adaptable for various tasks e.g. regression, pose estimation [3]. This proves that our approach is not limited by classification-based tasks.

Reviewer comment (Evaluation):
Table 1 is misleading. The authors adopted additional diversity metrics, but it is obvious that their method looks to have better scores because they specifically added an additional loss function for diversity, which is based on cosine similarity.

Author response:
Thank you for your valuable feedback. We want to clarify that all of the baselines we considered (NCL, APD, DICE) also introduce some form of loss function to diversify their predictions, and we have not modified these baselines with our loss function.

The main purpose of Table 1 is to provide a comparison among the diversity metrics of different methods. Our method aims to diversify saliency maps, which is different from most existing methods that increase the diversity in the prediction space. While diversification in the prediction space often has a trade-off with performance, our approach has shown that diversifying features can achieve balanced performance. We hope this clarifies our position and sheds light on the rationale underlying our approach.

Reviewer comment (Presentation):
The paper requires re-writing. The final method named SDDE_{OOD} is presented at the end of the paper, right before Section 6. The authors should describe this final method in Section 4.

Author response:
Thank you for your opinion. We would like to emphasize that SDDE_{OOD} is just a one piece of our pack of experiments. Consequently, we want to get attention of a reader to saliency map diversification without external data as the main topic of out work.

Reviewer comment (Evaluation):
What is the total training time of the entire framework when compared to previous approaches? I think it takes more time and has more FLOPs because of the CAMs computation.

Author response:
You rightly pointed out that our approach raises questions about its computational cost compared to other methods for improving ensemble diversity. To address this concern, we have conducted a comparative analysis of the computational cost of SDDE training compared to other methods. According to our measurements on the NVIDIA V100 GPU, with training details described in Section 5.1, training a single SDDE model takes 2.9 hours using a single V100 GPU. Deep Ensemble takes 6.4 hours, and Deep Ensemble without adversarial loss takes 1.4 hours. The inference time is not affected by diversity loss and remains identical for all ensemble methods. Similarly, during training, SDDE, Deep Ensemble, and Deep Ensemble without adversarial loss consume 6.2 GB, 6.8 GB, and 3.9 GB of GPU memory, respectively. We have included this information to our manuscript (see Appendix G).

Reviewer comment (Evaluation):
The authors followed the experimental setup and training procedure from the OpenOOD benchmark (Zhang et al., 2023). I am confused as to why they did not also follow the same evaluation setup from the OpenOOD.

Author response:
**[A1]**. We would like to kindly inform you that we have strictly followed the evaluation procedure delineated in OpenOOD v1.5 [1] for conducting our research. The only modification we have implemented is the multi-seed evaluation for ImageNet. We believe this slight adjustment is necessary for the depth and accuracy of our study.

Reviewer comment (Experiments):
The authors miss several state-of-the-art OOD methods [1-4] for comparison. 
[1] Yue Song, Nicu Sebe, and Wei Wang. Rankfeat: Rank-1 feature removal for out-of-distribution detection. NIPS 2022.  
[2] Andrija Djurisic, Nebojsa Bozanic, Arjun Ashok, and Rosanne Liu. Extremely simple activation shaping for out-of-distribution detection. ICLR 2023.  
[3] Jinsong Zhang, Qiang Fu, Xu Chen, Lun Du, Zelin Li, Gang Wang, xiaoguang Liu, Shi Han, and Dongmei Zhang. Out-of-distribution detection based on in-distribution data patterns memorization with modern hopfield energy. ICLR 2023.  
[4] Yiyou Sun, Yifei Ming, Xiaojin Zhu, and Yixuan Li. Out-of-distribution detection with deep nearest neighbors. ICML, 2022.

Author response:
**[A2]**. In our paper, we have proposed a novel ensembling method and focused on comparisons with other ensemble approaches. The mentioned methods, to our understanding, target single-model designs, and have already been implemented in the OpenOOD 1.5 paper. As we run the same evaluation pipeline as in the OpenOOD benchmark, our results can be compared with the original OpenOOD paper. Thus, our approach outperforms the ASH [2] method, which is included in OpenOOD (C10 Near: 75.27% → 92.06%, C10 Far: 78.49% → 94.60%, C100 Near: 78.20% → 92.06%, C100 Far: 80.58% → 94.60%). We hope this elucidates our considerations regarding the methods we've chosen to compare in our study.

Reviewer comment (Theory):
As per my understanding, saliency maps should highlight the object regions to help classification. If we make them highlight different regions, as done in Fig.1, it defeats the purpose of saliency maps. I don't agree with the idea that we should diversify saliency maps spatially, to the extent they start highlighting backgrounds.

Author response:
We respect your perspective on the traditional function of saliency maps; however, we would like to offer a different viewpoint supported by our experimental results. Our approach, termed SDDE, might appear unconventional at first glance, but it's crucial to note that it doesn't compromise the performance of individual models. This is clearly demonstrated in Table 1 of our manuscript, where the accuracy of individual models remains intact despite the diversification of saliency maps. We thus conclude that even models, focusing on backgrounds and separate parts of the image, can achieve competitive prediction accuracy.

Reviewer comment (Theory):
Why do authors think diversifying saliency maps is the same as diversifying features?

Author response:
Thank you for your thoughtful question. To address your query, we conducted an analysis of the relationship between the diversity in saliency maps and the diversity in the features of the models' penultimate layers. This involved measuring the cosine similarities between the saliency maps and these features. The results of this analysis are presented in Figure 8 of Appendix F of our manuscript.

Our findings indicate a clear correlation: as similarities in the feature space increase, so do the similarities in saliency maps. This observation led us to conclude that by diversifying the saliency maps, we indirectly encourage the ensemble models to utilize a broader range of features for their predictions. This inference bridges the gap between the diversification of saliency maps and features in our approach. We understand the importance of this clarification and have thus incorporated the relevant details into our manuscript (Appendix F).

Thank you once again for your valuable feedback, which has helped us improve the clarity and depth of our research. We kindly ask you to reconsider your evaluation of our manuscript, taking into account the additional information provided regarding our methods and results. We look forward to your continued input and further discussions.

Reviewer comment (Novelty):
Originality: The diversity loss is combined with cross-entropy during training in a single optimization objective. This additional component sounds like an incremental change. More deep investigation on the incentive of using this combination is required.

Author response:
It is indeed a common practice to utilize cross-entropy in classification tasks. However, the introduction of the SDDE loss is not a mere incremental improvement to CE. Our approach fundamentally alters the optimization landscape by integrating a diversification component, which is not typically considered in standard CE-based methods. This integration goes beyond the conventional usage of CE and contributes to the novelty of our work. The concept of introducing diversification losses is indeed known in the context of ensemble methods [1, 2, 3]. However, our work presents a novel perspective on this concept. We have developed a unique approach to diversification that differs from traditional methods used in ensemble models.

Considering your valuable feedback and the clarifications we've provided about our approach and its impact, we kindly request that you reassess the evaluation of our manuscript.

[1] Changjian Shui, Azadeh Sadat Mozafari, Jonathan Marek, Ihsen Hedhli, and Christian Gagn ́e. Di-versity regularization in deep ensembles. 2018.

[2] Tianyu Pang, Kun Xu, Chao Du, Ning Chen, and Jun Zhu. Improving adversarial robustness via promoting ensemble diversity. In International Conference on Machine Learning, pp. 4970–4979. PMLR, 2019.

[3] Alexandre Ram ́e and Matthieu Cord. Dice: Diversity in deep ensembles via conditional redundancy adversarial estimation. In ICLR 2021-9th International Conference on Learning Representations, 2021.

Reviewer comment (Evaluation):
Significance: The proposed method does not significantly outperforms the other state of the art technologies. In some of the metrics, the proposed method seems to work well but not all or large metrics.

Author response:
Thank you for your feedback. While we respect your view, we politely disagree with the assessment. In our paper, we have proposed a novel ensembling method and focused on comparisons with other ensemble approaches. Our results indicate meaningful improvements over current methods, specifically in the area of OOD detection tasks, even on large-scale datasets; please, refer to Tables 3 and 4 Near/Far OOD detection scores: C10 Near: 91.22% → 92.06%, C10 Far: 93.55% → 94.60%, C100 Near: 83.18% → 83.65%, C100 Far: 82.15% → 82.39%, ImageNet Far: 82.85% → 85.98%. We believe the margin of improvement we demonstrate is significant and worthy of consideration. It's our hope that this will be taken into account in the evaluation.

Reviewer comment (Theory):
To explain the convergence property of the combined solution in the paper.

Author response:
Our research leverages a well-established setup for optimizers and models using SGD and ResNets, which is grounded in a significant body of previous research. With respect to your question on convergence properties, this is still an open problem in the research community and it transcends the scope of our study [4]. While it is a critically important topic, many pragmatic studies, including ours, do not go in-depth into this subject because the focus is primarily on real-world applications and empirical approach. We appreciate your understanding of these constraints and your important role in helping us improve our contributions in this field.