Venue: ICLR 2022 Conference

Reviews:

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: The authors propose an attack that could break 4 adversarial detection methods published recently. Traditionally, attacks against detection methods have attempted to maximize the loss for both classification and detection simultaneously. However, using a toy example the authors show that this is suboptimal, as it may not find the worst-case adversaries. The authors propose to minimize the loss (targeted attack setting) iteratively by optimizing either only for the classification pipeline or the detection pipeline at a time.  The attack first considers the classification loss and further tries to fool the detection pipeline until the classification prediction remains incorrect. The authors also propose a variant of the attack by considering gradient steps for the classification pipeline to be orthogonal to the gradients of the detection pipeline and vice versa. Finally, the paper shows that these two proposed attacks completely circumvent four recent adversarial detection methods.
main_review: Strengths:

-  The proposed approach is well motivated and novel. Empirically the contributions of the paper are good.
-  Experiments are well designed. The ROC curves help in better understanding.
-  The discussed defences have been reproduced well.

Weaknesses:

-  A more detailed discussion on the past works which broke adversarial-detection defences needs to be done in order to better highlight the novelty of the proposed approach. Could the authors clarify why these detection methods cannot be broken using existing adaptive attacks? [1,2] 
-  The clarity on the discussion related to the four defences that are broken could be improved. It is difficult to understand the defence approaches from Section 4 of the paper.
-  Could the authors clarify why an epsilon bound of 0.01 was used to draw the ROC curves? Could the authors share the curves for an epsilon bound of 0.031 (8/255)?

[1] Carlini et al., Adversarial Examples Are Not Easily Detected: Bypassing Ten Detection Methods
[2] Tramer et al., On Adaptive Attacks to Adversarial Example Defenses

##### Update post rebuttal #####

The authors' rebuttal sufficiently addresses my concerns and I am happy to update my recommendation to "Accept". 



summary_of_the_review: The proposed attack seems interesting. I think the main problem with the paper is with respect to the clarity in writing, especially in Section-4. Further, the past methods which broke detection methods need to be discussed in detail to highlight the significance of the proposed method.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: This paper targets on attacking the defensive mechanism of adversarial examples detection. It proposes a new optimization algorithm to simultaneously meet two different requirements. It verifies its effectiveness on several state-of-the-art adversarial example detection methods.
main_review: ### Pros
Breaking the defenses of adversarial example detection is an important yet underdeveloped field. The approach proposed in this paper is straightforward and effective in practice. The experimental evaluations seem reasonable.

### Cons
The proposed optimization formulation is based on intuition. It would be better if the author can provide theoretical evidences that can explain its effectiveness. 

Currently, the empirical evaluations mainly shows its performance against the detection defense. It can be improved by adding more analytical results.




summary_of_the_review: I would love to recommend an accept for this paper, considering its topic of research, effectiveness, and novelty.

In addition, I want to clarify that I served as a reviewer of this paper for its last submission. I am glad to see that the authors followed my suggestions and made several improvements on their writing. For the last submission, I give a weak accept score in the final stage. Considering that the paper has been improved on several aspects, I raise my score to accept.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: This paper proposes two techniques for generating adversarial examples: Selective Projected Gradient Descent (SPGD) and Orthogonal Projected Gradient Descent (OPGD). In order to fool both the victim model $f$ and a detector $g$, SPGD selectively optimise either $f$ or $g$ depending on whether the modified input is misclassified as the target class, while OPGD further orthogonalizes the gradients. Evaluation on four previously unbroken, state-of-the-art defence methods demonstrate the effectiveness of the proposed attacks.
main_review: I find this paper pleasant to read in general: SPGD and OPGD are well motivated and clearly explained, while the experimental verification is sufficient and convincing. I only have several minor comments:

1. How effective are SPGD and OPGD against an ensemble of detectors instead of just one?
2. For Honeypot Defence and Sensitivity Inconsistency Detector, what is the impact of the threshold $\phi$ on the effectiveness of SPGD and OPGD?
3. Why is OPGD much less effective than SPGD against Dense Layer Analysis when $\epsilon$=0.01 (Figure 3)?
4. Why isn’t SPGD evaluated against Steganalysis?
summary_of_the_review: The proposed attacks for creating adversarial samples are simple but effective, and the evaluation is sufficient.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper considers the problem of finding adversarial examples that simultaneously defeat a detector of adversarial examples. An argument is made that existing attacks often achieve one goal at the expense of another. This argument motivates the proposal of two attack techniques. These are evaluated against four existing detection-based defence methods, with successful results.
main_review: The main proposed approaches are a fairly simple update to existing attack methods, though appear to be quite effective.

The presentation of the paper was generally clear, but with some areas of issue. For example
-	Section 2.4 was a bit confusing. It was not clear (to me) from the preceding discussion that the focus was to be on targeted attacks. For example, in equation (1), the minimisation of the loss suggests that this is targeted (to class t). Yet in section 2.2, t was used as the valid class of the data point x.
-	In Section 3, however, it seems that t is the valid class, as the first equation is targeting attacks which result in distinct classifications from t. Also reference in the text just above  the bold heading refers to equation2 as maximising the loss (equation 2 is currently a minimum) – which it would do if t were the valid class. 
-	In Section 3.1, the function f as defined is –epsilon at x=e and positive (for small epsilon) at x=0. I think the 1 should probably be something like exp(-1) so that it evaluates to –epsilon at the origin. As it stands this section does not make sense.
-	It does not seem clear in section 3.1, even if it were corrected, why a Lagrangian formulation would not work. (Note that that is an independent statement of whether gradient descent would work). Also it is said that where the gradient of h vanishes wrt x the loss function has a non-zero slope and so the loss can be further minimised. But if one respects the constraint g(x)<0 this may not be true. In fact, surely the solution to minimising f st g<0 is the point on the y-axis at the apex of g’s circle. At this point, the gradient of f will point upwards, but one cannot follow that without violating the constraint. I am not clear what the authors are arguing exactly.
-	For the methods proposed in 4.1 and 4.2, it is not made made clear exactly how the proposed gradients are to be used. Obviously, presumably in a PGD style attack. (But even the PGD attack in section 2.4 is imprecisely specified as it is not made clear exactly what x_0 is chosen to be nor how may steps are to be performed, etc.) 
-	It is stated that the proposed attack is “easier to analyse” than prior approaches. Yet no analysis really has been done.

The experimental results seem to be quite convincing (see comments below also), though it would be good to have seen ROC curves for other (standard) attacks included in the figures for comparison (the only data from other attacks is a simple reproduction of the SN@5 value from the original paper (for usually only one epsilon value). Also, given the defence in 5.4 was reimplemented, it would have been good to see a statement that the original results of the original authors were reproduced, and that those authors had also confirmed the attacks, as was said in at least two of the other cases.

The experiments conducted seem to be fair and thorough, in the sense that the attacks are evaluated against the extant defences with both the threat model originally considered for that defence as well as with common parameter choices. In most cases, the original code of the defence’s authors is used, and also in most cases it seems the authors confirmed their attacks as successful with the defence’s authors.

A side query: I accept that the goal is to not have the gradient of one target conflict with the other (i.e. f and g). But to make the gradient used purely orthogonal to the other is a bit extreme and throws away advantageous cases where the change is in the correct direction for both at the same time. Would it not be possible to instead simply use the orthogonal gradient in cases where its projection is in the negative direction on the other gradient, and use it unchanged otherwise, to get the best of both worlds.

Though the results are good, the issues noted above with regard to clarity and clear motivation mean that my score (for now) is lowered.

Some minor comments:
-	Should “defences” in the third paragraph of the Introduction be “attacks”?
-	In section 3.1, h(x,lamda)=0 uses vector notation on the 0. Isn’t h() a scalar?
-	I found the notation confusing for projections (it may just be me!). But the paper seems to use proj_A B to denote the projection of the vector A along the direction of the vector B. This would seem counterintuitive i.e. A and B wrong way round? Not sure what is standard here, so I could be wrong.
-	In section 4.2, “... exactly opposite, that is a situation where grad f = grad g”. Surely you mean “- grad g”?
-	Section 5: “Three of the case study” should be “Three of the case studies”
-	The text “The equivalence ...” after equation (5) is trivial and need not be said.
-	FLD not defined??
-	In section 5.4, the specification of the two ranges of features is the same. Should some be diagonal?
-	“non-diiferentiable”
-	“any every”
-	Why is only one of the proposed approaches used in 5.4?
-	In the conclusion it is claimed that 0% detection was achieved with 0% accuracy at 5% FPR. This was mentioned in one case, but not clearly achieved in the others? Or at least it wasn’t noted explicitly for the other cases. 

summary_of_the_review: Overall, the two proposed techniques, though a relatively simple update to existing approaches, seem to perform well and provide an effective attack against state of the art adversarial attack detection techniques. However, the presentation of the paper is a bit confused in places and the motivation for the approach a bit unclearly argued. It is hard to argue against such convincing numerical results, so with some rework to address the issues raised I think my recommendation for the paper could increase.

Edit: Score increased after author response and revision.
review_title: null

Meta-review: The authors propose two new variants of (projected) gradient descent for attacking a classifier and a detector simultaneously. Using these two new variants they are able to break four recent detection methods for adversarial samples.

Strength:
- All the reviewer acknowledge that breaking these four defenses is a valuable contribution.

Weakness:
- From a technical perspective the paper is rather simple and no theoretical support for the suggested variants is provided. The justification is rather handwavy. From an optimization perspective it is unclear why not a simple penalty-based approach would have given the same results or would even work better. The choices maded in this paper seem a bit arbitrary and are mainly justified by the fact that they work for the four detectors
- the honeypot defense was already broken in 
A Partial Break of the Honeypots Defense to Catch Adversarial Attacks, Nicholas Carlini, arXiv:2009.10975

Minor:
- the authors should update the references, several papers have appeared in the meantime

While I appreciate the contribution of the broken defenses, in terms of technical contribution and discussion of the methods this paper is borderline.
Decision: Accept (Poster)