From: ICLR.cc/2023/Conference/Paper5146/Authors
title: Submission Withdrawn by the Authors

From: ICLR.cc/2023/Conference/Paper5146/Reviewer_uVrN
title: Post-rebuttal update
comment: Thanks to the authors for the response. There has been a noticeable improvement in the revised paper. The revision has clearly explained the motivation of running certification on all examples (including possibly misclassified examples). But there are still weaknesses on the experiments and empirical performance. I have updated my review and raised my recommendation to 3.

From: ICLR.cc/2023/Conference/Paper5146/Authors
title: Updated Manuscript
comment: Based on feedback from the reviewers, we have made some improvements to our work. The changes are highlighted in blue.

**Modifications**
* add a short overview of formal verification methods (Section 1).
* extend the motivation with a detailed description and a graphical example (Section 1).
* adjust the description of the contribution to highlight the empirical results (Section 1).
* improve the description of the results for the ID analysis (Section 3.2).
* add EMINST (letters) dataset to the grayscale category (Section 4.2).

We thank the reviewers for their valuable feedback, and hope that the updated version address most of your doubts.

From: ICLR.cc/2023/Conference/Paper5146/Authors
title: Response to Reviewer 7Uwc
comment: Thank you for your interesting questions, which we will answer in the following.

**Q1:** *“What is the motivation of this paper?”*

**A1:** Robustness certificates (or formal verification methods) ensure that a given sample is certifiably robust to a predefined perturbation. But what does this mean in practice? Can we trust the prediction or not? Can we use robustness as an additional safety metric? In our analysis we want to explore how to relate certifiable robustness to correctness at operation time, i.e., when the ground truth labels are not given. For this reason, we evaluate robustness certificates on both in- and out-of-distribution cases.  The two questions we address in our work are:
1. “Can we guarantee that a certified sample has been classified correctly?” \
Since robustness certificates ensure the stability of the prediction near the input, this question seeks to understand if certified robustness is useful in judging whether a sample has been classified correctly or not. 
2. “Can we detect if a sample is from a different distribution with respect to the training distribution?” \
Similarly, to the previous question, this question seeks to understand whether certified robustness is useful in judging whether a sample is from the training distribution or not.

**Q2:** *“What is the novelty of this paper?”*

**A2:** Our contribution is to conduct a practical evaluation of formal verification methods. To the best of our knowledge, there is no previous work conducting such an evaluation to answer the questions above.

**Q3:** *“What do the experimental results indicate?”*

**A3:** The results indicate two main outcomes:
1. In our in-distribution analysis, we showed that there is a strong correlation between certified robustness and accuracy for clean and perturbed samples. In this context, formal verification methods prove to be a useful error-reduction metric leading to an overall increase in reliability.
2. Completely different are the results for the OOD analysis, where robustness certificates demonstrate their unreliability compared to standard OOD detection methods.

From: ICLR.cc/2023/Conference/Paper5146/Authors
title: Response to Reviewer uVrN
comment: Thank you for your interesting comments. By answering you, we try to bring out a different point of view.

**Q1:** *“The study is based on a wrong understanding of certified robustness. Robustness certification only needs to be conducted on correctly classified examples. […]”*

**A1:** The evaluation of robustness certificates only on correctly classified examples assumes that the labels are already known. This assumption holds in the well-established application of certification methods towards the evaluation of a model's robustness based on the certifiability of samples from a test set for which the labels are known. In contrast to that, we are interested in the possibility of applying those methods at operation time to new samples for which the ground truth labels, and therefore the correctness of the prediction is not known. This is an aspect so far unexplored, but of high practical relevance since it might open the door for using certificates during operation. We conduct this study to understand if there is a link between the stability of the model around an input (indicated by a successful certification) and the correctness of the prediction of the model. Therefore, if correctly classified samples have a higher chance of also being certified, we could leverage this result to build more reliable predictors by utilizing the certification result as an additional signal - similar to uncertainty and other monitoring approaches applied at operation time.

From: ICLR.cc/2023/Conference/Paper5146/Authors
title: Response to Reviewer YRTu
comment: Thank you for appreciating that our empirical observations are very insightful and that our experiments are well-designed, as well as the valuable comments which we will address in the following.

**Q1:** *“The paper does not well state the motivations and background on why we need exploration to applying certification for correctness/OOD detection. [...]”*

**A1:** Yes, we agree that a more extended discussion of why we need such an exploration would be helpful in drawing the big picture. Therefore, we extend it slightly here. The main motivation for this paper is to understand how formal verification methods can be applied in practice. Robust certification of a sample ensures the stability of the prediction in the vicinity of the input. But then, can we trust the prediction? If not, how much does this increase our confidence in correctness? Why should we use robustness certificates in practice, given that a misclassified sample can be robustly certified? And if we used them, what is the optimal adversarial budget for achieving a good trade-off between accuracy and robustness?

**Q2:** *“[...] It would be helpful to include certifiably trained networks and adversarially training networks with different epsilons to better observe the trends.”*

**A2:** Yes, we agree that a more extensive evaluation would be helpful to better observe this trend. Unfortunately, robustness certificates are slow in practice, increasing our time to evaluate larger or differently trained models.

**Q3:** *“Only geometric robustness is evaluated in the paper. [..]”*

**A3:** We want to highlight that not only geometric robustness certificates are evaluated but as shown Fig. 3, we utilized infinity-norm certificates on clean ID. Additionally in Table 2, we calculated infinity-norm certificates on perturbed ID samples. Ultimately, the analysis on OOD samples is conducted purely on infinity-norm certificates.

From: ICLR.cc/2023/Conference/Paper5146/Authors
title: Response to Reviewer WTte
comment: Thank you for appreciating that our evaluation between correctness and certified robustness is novel, as well as the valuable feedback. Below we address the detailed comments.

**Q1:** *“[…], there is no analysis of whether and how the studied AUC increases confidence on the correctness of a model's prediction in comparison to considering the network accuracy in isolation.”*

**A1:** Yes, we agree that a more detailed discussion of the results in Section 3.2 would help in drawing conclusions about the perturbed samples. The AUC is used as a metric to select possibly optimal models and to discard suboptimal ones regardless of class distribution. In our analysis, we emphasize the fact that AUC is directly and naturally related to model accuracy.  This claim is supported by the empirical results presented in Table 3, whereby perturbing the test samples, a similar decrease in accuracy and robustness is observed at the same time. This result is intended to state that distributional shifts (or perturbed ID samples) are as difficult to verify as the network's generalization capacity is lower. This is independent of the training procedure. On the one hand, adversarially-trained networks obtain higher AUCs than plain models, regardless of accuracy (mirroring the results for the unperturbed distribution samples). On the other hand, the AUC of adversarially-trained networks similarly decreases with respect to accuracy. Validating the fact that accuracy is correlated with robustness.

**Q2:** *“The paper concludes that formal verification methods should be combined with OOD detectors. I think this should be explained in more detail. In my view the two problems are orthogonal. […]”*

**A2:** As described on related work, adversarial attacks are performed for both in- and out-of-distribution data. Robustness certificates (or formal verification methods) ensure that a given sample is certifiably robust to a predefined perturbation. But what does this mean in practice? Can we trust the prediction or not? Can we use robustness as an additional safety metric? In our analysis we want to explore how to relate certifiable robustness to correctness at operational time, i.e., when the ground truth labels are not given. For this reason, we evaluate robustness certificates on both in- and out-of-distribution cases. Given an OOD sample that gets high confidence (i.e., is not detected by standard OOD detectors), there are two possible outcomes: certified or uncertified. If the number of certified OOD samples is greater than the number of certified ID samples, then we cannot safely use this method. In our analysis, we show that for OE-trained models, the number of OOD certified samples is greater than the number ID samples, posing security problems.

From: ICLR.cc/2023/Conference/Paper5146/Reviewer_WTte
confidence: 3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.
summary_of_the_paper: The paper presents an empirical analysis of the relationship between prediction
and robustness accuracies of neural networks. The analysis is carried out w.r.t
to distributional shifts (geometrically transformed samples), where the paper
concludes that robustness and prediction accuracies are strongly correlated.
The paper also presents a theoretical and empirical analysis that shows that
out of distribution samples are ascertained robust by formal verification
methods.
strength_and_weaknesses: + First paper I am aware of that considers that discusses in some depth the
correlation between robustness and accuracy. 

- The paper claims that robustness is strongly related to accuracy. What it
  concretely shows is that the  AUC w.r.t the ratio of correctly classified and
  certified images and the ratio of incorrectly classified and certified images
  decreases with decreased accuracy. It does not explain however the
  significance of this. That is, there is no analysis of whether and how the
  studied AUC increases confidence on the correctness of a model's prediction
  in comparison to considering the network accuracy in isolation.

- The paper concludes that formal verification methods should be combined with
  OOD detectors. I think this should be explained in more detail. In my view
  the two problems are orthogonal. That is, the formal verification problem is
  to show that a model has the same classification result for a set of inputs,
  where the set of inputs is a parameter to the problem; i.e., irrespective of
  whether this set is within distribution or out of distribution the
  verification analysis shows how robust the model is to its predictions.
clarity,_quality,_novelty_and_reproducibility: The paper is not particularly novel as it mostly concerns empirical
observations whose significance is in my opinion not adequately discussed. The
presentation can be improved by properly situating the several concepts of the
empirical analysis to the overall aim of the paper. For instance, the
interpretation of the AUC studied  and the significance of the difference in
performance of certificates (as this is measured by the AUC) is never
discussed.
summary_of_the_review: The paper makes some empirical observations regarding prediction and robustness
accuracies but the significance of these observations is not entirely clear.
correctness: 3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.
technical_novelty_and_significance: 2: The contributions are only marginally significant or novel.
empirical_novelty_and_significance: 2: The contributions are only marginally significant or novel.
flag_for_ethics_review: NO.
recommendation: 3: reject, not good enough

From: ICLR.cc/2023/Conference/Paper5146/Reviewer_YRTu
confidence: 5: You are absolutely certain about your assessment. You are very familiar with the related work and checked the math/other details carefully.
summary_of_the_paper: This paper explores geometric certification methods in terms of in-distribution and distribution analysis. For in-distribution analysis, the authors show a strong correlation between certified robustness and accuracy, and thus certification methods can be used to detect correct predictions. For out-of-distribution analysis, the authors show inferior performance in detecting OOD samples compared to SOTA OOD detectors since OOD samples far from the training distribution are easy to be certified.
strength_and_weaknesses: ### Strength
- This is an empirical analysis paper with well-designed experiments to illustrate how certification methods perform in terms of correctness and OOD detections.
- Both empirical observations are very insightful.

### Weakness
- The paper does not well state the motivations and background on why we need exploration to applying certification for correctness/OOD detection. The current version could be very confusing for most common readers.
- For in-distribution analysis, the main claim is related to how the model is certification-inclined. It would be helpful to include certifiably trained networks and adversarially training networks with different epsilons to better observe the trends.
- Only geometric robustness is evaluated in the paper. Could we observe the same trends for the regular robustness definitions with lp norm balls? If not, what are the intrinsic differences? A discussion would be very helpful.

clarity,_quality,_novelty_and_reproducibility: The paper proposes new observations supported with good experiments designed.
summary_of_the_review: Overall, this paper proposes several interesting observations regarding certification methods in terms of correctness and OOD detections, potentially leading to more general usage of certification methods. However, the paper should better state the goal and background of the paper.
correctness: 4: All of the claims and statements are well-supported and correct.
technical_novelty_and_significance: 3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.
empirical_novelty_and_significance: 3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.
flag_for_ethics_review: NO.
recommendation: 6: marginally above the acceptance threshold

From: ICLR.cc/2023/Conference/Paper5146/Reviewer_uVrN
confidence: 4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.
summary_of_the_paper: This work tries to study the relationship between certified robustness and the correctness of prediction, on both in-distribution and out-of-distribution data.

The motivation of the paper has been substantially updated during the rebuttal period – how certifications may be used at test time when ground-truth is unknown.


strength_and_weaknesses: Weaknesses:

- The study is based on a wrong understanding of certified robustness. Robustness certification only needs to be conducted on correctly classified examples. It is meaningless to run a certification on misclassified examples or samples where an adversarial perturbation can be easily found. The goal of robustness certification is to certify whether the prediction under perturbation is certified to be the ground-truth rather than just the model’s prediction on the clean input. Since this paper uses GPUPoly, the authors may check GPUPoly’s paper which says “The goal of certification then is to prove that **all images in this region are classified correctly** by the network”. 

- I do not see any useful contribution. It is unclear what important message the paper wants to convey by running robustness certification on misclassified examples. 

Many arguments are wrong and not meaningful:

>“In short, the more certification-inclined a network is, the better we can use verification methods as a metric to distinguish between correctly and incorrectly classified samples.” 

>  “ In this analysis, we want to deter- mine if incorrectly classified samples will be robustly certified or not”. 

People do not need verification to tell whether an example is correctly classified (this is like using a complex method to solve a simple problem while providing no benefit). And people usually don’t run verification for misclassified examples. 

> “This is a major problem for the practical use of robustness certificates, since samples are likely to be certified quite far from the training distribution.” 

This problem does not exist if certified robustness is correctly used.

> “In such analysis, we proved theoretically that samples far enough from the training distribution are easily certified for ReLU classifiers.” 

Again, misclassified examples should not be considered as certified. The implications of these results are unclear.


==============

**Updated review based on the revised and improved paper**

Strengths:
* This paper reveals a correlation between certification and correctness on in-distribution (ID) data, which implies that certification may potentially be used as an indicator to reflect whether the prediction is likely to be correct, when the ground-truth is unknown.
* The paper also conducted experiments on OOD data and compared with existing OOD detection methods. 

Weaknesses:
* On ID data, the paper only illustrates a correlation between certification and correctness. There lack an experiment in a scenario where this correlation may be utilized. E.g., if certification may be used to predict the correctness of the classifier, there lacks an experiment showing the quality of such a prediction and comparison with baselines (e.g., using adversarial attacks).
* On OOD data, experiments show that certification generally cannot outperform existing OOD detection methods on OOD detection.
* GPUPoly is costly (“the incredibly long run time of validating a large amount of samples” mention in this paper), which is also a weakness of using certification for correctness prediction or OOD detection, compared to other methods. Originally, certification methods were proposed to obtain safety guarantees, so a larger time cost is acceptable given the benefits from the guarantees, while the paper is using certification methods in a non-certified way (e.g., OOD detection using certifications is not certified).  


clarity,_quality,_novelty_and_reproducibility: 
- Clarity and quality: ~~It is unclear what meaningful contribution this paper aims to make. Many arguments are wrong.~~ Writing is mostly clear after the rebuttal.  
- Novelty: ~~Not novel.~~ Potentially using certification for predicting the correctness or OOD detection is novel. The motivation has been further clarified in the rebuttal period. 
- Reproducibility: Major implementation details have been provided.

summary_of_the_review: ~~This work is based on a wrong understanding of certified robustness. Issues studied in the paper do not exist if certified robustness is correctly used. Implications of the results in the paper are unknown. I don’t see any significant contribution.~~

In the rebuttal and the revised paper, the authors have clearly explained the motivation of the work. Now the method is interesting and novel. However, some experiments are missing, and the performance of the method is not good enough.

correctness: 3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.
technical_novelty_and_significance: 2: The contributions are only marginally significant or novel.
empirical_novelty_and_significance: 2: The contributions are only marginally significant or novel.
flag_for_ethics_review: NO.
recommendation: 3: reject, not good enough

From: ICLR.cc/2023/Conference/Paper5146/Reviewer_7Uwc
confidence: 4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.
summary_of_the_paper: This paper experimentally investigates certified robustness and OOD detection for relu networks. 
This paper is very badly written, and badly organized. 
I don't understand the motivation and the final outcome of the paper. 
strength_and_weaknesses: I don't see any strength of this paper. 

Weakness, to name a few:
1. What is the motivation of this paper? 
2. What is the novelty of this paper? 
3. What do the experimental results indicate?

The authors mentioned that their top main contribution is "first study on the relationship between correctness and certified robustness". Honestly, I don't think that is an attractive contribution, if there are no other additional explanations. 
clarity,_quality,_novelty_and_reproducibility: Clarity: this is almost the most confusing paper that I have read during the past 10 years.

Quality: Bad quality. 

Novelty: No novelty. 

Reproducibility: I did not check the experiments. 
summary_of_the_review: Clear reject. 
correctness: 1: The main claims of the paper are incorrect or not at all supported by theory or empirical results.
technical_novelty_and_significance: 1: The contributions are neither significant nor novel.
empirical_novelty_and_significance: 1: The contributions are neither significant nor novel.
flag_for_ethics_review: NO.
details_of_ethics_concerns: None
recommendation: 1: strong reject