Venue: NeurIPS 2021 Conference

Reviews:

final_score: rating: 7: Good paper, accept
review_content: summary: The paper presents a novel way for adversarial attacks.  This work studies the problem of attacking networks to change their confidence without affecting the accuracy. Such attacks increase the confidence of the network on wrong predictions and decrease the confidence of the networks on correct predictions. Since the accuracy of the network is not the target, this enables the. attacker to use smaller attacks that might not be easily detected. The paper studies both white-box and black-box attacks on different kinds of uncertainty estimation methods including MC-dropout, SelectiveNets, and Ensemble methods. Finally, the paper presents extensive empirical evaluations that support the proposed method.
main_review: - originality: The problem is novel and different from the works that have studied adversarial attacks which aim to drive the network to generate wrong predictions. 
- quality: The authors conducted thorough experiments and evaluated different attacks and uncertainty methods.
- Clarity: The paper is well written and easy to follow.
- Significance: This work is important as the effect of attacks on network confidence can affect the practical deployment of deep learning models especially in critical applications.
limitations_and_societal_impact: I do not see concerns regarding the societal impacts. This kind of work is important to develop more robust models.
review_title: null

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: This paper proposes an adversarial attack on uncertainty estimation that does not change the predicted label of a classifier. Rather than perturbing an image to flip its predicted label, this method moves incorrectly classified images further away from the decision boundary (with respect to some notion of distance that depends on the uncertainty method being attacked), thus artificially reducing the model's uncertainty in these images.
main_review: The presented idea is clearly presented and properly contextualized, and the experimental analysis of the attack is thorough and considers a wide variety of uncertainty estimation algorithms, showing that they are all vulnerable to the proposed attack. I am not an expert on uncertainty estimation, but from my experience both the paper and the evaluation are technically sound.

My primary concern is with the novelty and technical depth of the approach, combined with a lack of showcased "real-world" applications. In particular, technique-wise, the method is just performing weak adversarial attacks (FGSM-based with small epsilon, which are known to preserve the predicted class) that are untargeted on the correctly classified images, and targeted (towards the predicted class) on the incorrectly classified images. Such attacks have been thoroughly studied in the adversarial examples literature, see e.g. https://arxiv.org/abs/1608.04644 or https://arxiv.org/abs/1902.06705, and in general given any "reasonable" (differentiable) loss function derived from the output of an image classifier, it's not really surprising that images can be imperceptibly perturbed to maximize or minimize it.

For this reason, I think that the majority of the novelty here is in the concept/idea of attacking uncertainty estimation, rather in the technical approach. This is (in my opinion) completely fine, but in the absence of a fundamentally new technique, I think that the paper should demonstrate more convincing "real-world" applications of their proposed attacks, rather than focusing on the controlled settings shown in the current evaluation section. For example: are there deployed systems that take action based on uncertainty estimates? If so, does the proposed attack cause misbehavior in these systems? 

More specific/detailed comments:
- [Minor] I think "mislabeled instance" in Figure 1 actually be "misclassified instance"
- [Minor] Although it is commonly accepted/well-known, I think along with Figure 2 the paper should provide a quantitative summary (e.g., a simple confidence histogram) illustrating that indeed, on unaltered images, classifiers are more confidence when predicting the correct label than the incorrect label.
- The related work section is rather light on literature relating to adversarial attacks, of which a variety have been proposed, many with the explicit goal of maximizing or minimizing distance to the decision boundary (e.g., the CW attacks, MMA attack, etc.). These attacks and their relation to the proposed methods should be discussed in more depth.
- The definition of RC and AURC curves should not be entirely relegated to the Appendix, since it is central to the evaluation of the proposed method. Appendix A is probably too detailed to put into the paper as-is, but the paper should stand alone and thus would benefit from a short paragraph explaining RC, AURC, and their relevance to evaluating uncertainty estimation.
- [Minor] Based on the motivation section, I thought that the goal was to find the *smallest* epsilon the would ruin the image ordering by confidence, but the algorithm actually searches for the largest epsilon that does not flip the decision boundary. What is the motivation for choosing epsilon in this way (i.e., searching downwards and not upwards)?


limitations_and_societal_impact: N/A
review_title: null

final_score: rating: 7: Good paper, accept
review_content: summary: The paper presents an interesting method to create adversarial examples that do not change the class label, but rather flips the confidence in that label.  In other words, the accuracy of the classification system is not changed.  Rather, the idea is that the attack will force a system with well calibrated confidence to become confidently wrong and unconfidently correct rather than the desired behavior of being unconfidently wrong and confidently correct. The overall approach is basically the same as a standard label adversarial attack (i.e., follow the gradient), but at a lower magnitude as not to change the label.  The other minor difference is to determine the direction of the gradient depending on the ground truth label of the test sample.  That is increase confidence if the classification label prediction is incorrect and reduce the confidence otherwise. The paper does include extensive experimentation over various neural network models including uncertainty-aware models such as ensembles, MC dropout and SelectiveNet. 
main_review: At one level, the proposed method seems to be a corollary of creation of adversarial examples, i.e., standard attack at low magnitude to mess with confidence but not change the label. The slight innovation is to use knowledge of the ground truth and predicted labels to determine which direction to follow the gradient.   This leads to my main concern of the paper.  It requires knowledge of the ground truth.  To create one adversarial example, this might not be a problem as one may argue that through costly manual intervention an attacker can determine the ground truth.  However, for this method to be effective in terms of reducing human trust in the classifier label recommendations, the attack has to be executed at scale so that the confidence is flipped for most test samples.  Now knowledge of the ground truth by the attacker at scale seems too costly. Perhaps an attacker can simply flip the reported confidence without knowing the ground truth. It would be interesting to evaluate the efficacy of such a more realistic method.

The experiments do provide very interesting insights including the efficacy of choosing epsilon and the resiliency of ensemble methods.

The experiments focus on aleatoric uncertainty for the ensemble and MC dropout methods in focusing on softmax and entropy.  In other words, the focus in on the posterior predictive (or sample mean of the label probabilities).  It would be interesting to understand how the attacks work on epistemic uncertainty which could be measured as the spread or variance of the label probabilities.

There seems to be a discrepancy between the AURC numbers in Table 1 and Figure 4 for the EfficientNetB0 model. 

In light of the authors' response that the more practical attacks will be discussed and the paradigm shift the paper offers in terms of adversarial examples, I can recommend acceptance of the paper.
limitations_and_societal_impact: The paper is about possible attacks to uncertainty-aware machine learning classifiers that could reduce human trust in the system while the reported accuracy of the system looks good.  Understanding this vulnerability is the first step of designing machine learning systems that will be robust to such a vulnerability.  There does not seem to be any negative societal impact of this work.
review_title: null

Meta-review: The paper presents a novel approach that has been very well received by the reviewers. Also, the main concerns that were raised after the initial review have been addressed in the rebuttal and there is consensus in accepting the work.
I personally find the proposed method very interesting as it deals with uncertainty estimation and does so in an elegant way.
Decision: Accept (Poster)