Venue: ICLR 2021 Conference

Reviews:

final_score: rating: 7: Good paper, accept
review_content: This work starts by questioning the apparent robustness of quantized networks and demonstrates that such robustness is more so a failure of the attack algorithm in picking up the gradient signal. The authors address this by tuning a scalar multiplier applied to the network logits, which doesn’t modify the model’s decision boundary. Through analyzing the Jacobian, two approaches are proposed to determine the scalar $\beta$ without tuning it by performing the attack. This approach is quite effective on quantized networks and even provides significant improvement on floating-point networks combining with existing attacks like FGSM and PGD. The proposed modification might seem trivial at first, but it constitutes an important factor the community hasn’t taken notice of, to the best of my knowledge.

A few questions: 1) I don’t see any mentions of tuning the attack step size; if we set the new $\eta$ to $\eta/\beta$, we can keep the Jacobian intact and isolate the effect of temperature scaling for XENT. 2) how about sweeping $\beta$ and plotting against adversarial accuracy? This is surely expensive, but it would paint a clearer picture of the optimality of $\beta$ found using the proposed approaches. This can be done in tandem with an attack step of $\eta/\beta$.

I like the result overall, though the effect of $\beta$ on the Jacobian and the softmax can and should be separated, if my understanding is correct. The proposed approaches for determining $\beta$ largely depend on the Jacobian; therefore, there should be more investigation on if scaling the Jacobian correctly is more important than getting good error signals from softmax.
review_title: Interesting observation from quantized networks results in general, stronger adversarial attacks

final_score: rating: 6: Marginally above acceptance threshold
review_content: **Update**: Thanks to the authors for addressing my comments. As it was pointed out by the authors, temperature rescaling is mostly applicable to non-linear loss functions. For linear loss functions, temperature scaling only linear rescales the gradients. The difference between the proposed PGD++ attack and PGD with linear DLR loss is small (see the author's response to AR4). The improvements are most significant for FGSM but FGSM is not recommended for the robustness evaluation. Given the limited technical novelty and small improvements for linear loss functions, my score remains unchanged.

###### Summary
The paper studies the robustness of binary neural networks (BNNS), which at first look have higher robustness than full-precious neural networks. The authors highlight the problem of poor signal propagation in BNNs, which makes gradient-based attacks difficult. To address this issue, the authors proposed a 1) single scalar rescaling of the jacobian to improve the signal propagation; 2) parameter-free hessian norm scaling technique. In the experiments, the authors demonstrated that the modified attacks reduce the accuracy of BNNs to zero and outperform existing gradient-based attacks against floating-point networks.

###### Reasons for score

I vote for a weak acceptance of this paper. The paper shows that BNNs are not robust and introduce an interesting gradient rescaling technique, which can also be used to attack full-precision networks. The rescaling technique is well explained, easy to apply for any existing attacks, and has low computational overhead. However, as I will discuss below, I see some problems comparing the proposed attack against a well-tuned PGD attack.

###### Pros:
1) The paper studies the robustness of BNNs. The robustness of BNNs is not well studied, and understanding the robustness of BNNs is an important research direction.
2) The authors highlight the issue of signal propagation in BNNs. To address this issue, they devise a novel low-computational complexity technique.
3) Experimental results for BNNs and full-precious models demonstrate that the modified attack is effective.

###### Concerns and questions:
- In the experiments, the authors use a single parameter for the step size for both PGD and FGSM attacks. On the other hand, the proposed method computes an optimal rescaling to achieve good signal propagation. Even though the proposed technique has a low computational budget, I believe the authors should do a grid search for the optimal step size for PGD and FGSM attacks for a fair comparison.
- In the experiments, PGD L2 attack was unable to reduce the naturally trained models' accuracy to 0. This seems strange and unlikely as gradient shattering should not happen for naturally trained models. Can the authors explain these results? Is it possible that there might be an implementation issue?

###### Comments and suggestions:
- The proposed technique amplifies the error signal in a nonlinear way for nonlinear losses such as cross-entropy. However, for other losses such as multiclass-hinge loss, CW loss, the proposed method will simply linearly rescale the error signal. Attacking CW loss might be useful as it avoids the issues of saturated softmax gradients. Will this technique be useful for attacking with CW loss? 
- The authors claim that this method improves the efficiency of white-box attacks against full precision models. Is it possible for the authors to get the results for mnist_challenge and cifar10_challenge to see if the method outperforms an optimally tuned PGD attack?

review_title: The paper studies the robustness of binary neural networks. It highlights the issue of signal propagation in BNNs. To mitigate this issue, the authors propose a temperature rescaling technique.

final_score: rating: 5: Marginally below acceptance threshold
review_content: This paper studies the robustness of quantized neural networks against adversarial attacks. The authors use some slight modification of existing methods to successfully increase the attack success rate. In general, I think the idea is interesting. But I have some concerns that need to be addressed:
1. I am not fully convinced by the arguments made at the beginning of Section 4. The authors claim that poor signal propagation such as gradient vanishing or gradient exploding should be a problem for adversarial attacks. However, I do not think the reasonings provided here is specifically for binarized neural networks. Equations (3) and (4) also works in regular full precision networks. I do not think there are any problems which only present in BNNs, so the arguments here are not strong enough. If poor signal propagation is a problem for attacks, why we don't see that in full precision networks? More discussions on this are welcomed.
2. ResNet and DenseNet REF models in Table 3 seem to be surprisingly robust under PGD L2 attacks (column 3). This adversarial accuracy seems to be comparable to models with adversarial training. I think the authors need to provide some explanation on this. 
3. (Minor) Please refrain from only using color to distinguish curves/bars in figures as it may not be friendly to readers with color blindness.
4. (Minor) The authors may need to re-organize some sections to make the paper easier to follow, for example, the "related works" section before the "experiments" section.
review_title: Official Blind Review

final_score: rating: 5: Marginally below acceptance threshold
review_content: The paper identifies the gradient vanishing issue in the robustness of binary quantized networks. Therefore, it proposes to use temperature scaling approach in the attack generation. It has two methods for the temperature scale: (1) singular values of the input-output Jacobian and (2) maximizing the norm of the Hessian of the loss.

------------Updates after rebuttal------------- 
Thanks the authors for answering my questions. However, I don't think my comments are well addressed. Even though the paper [d] may not provide public available code, the authors could either use results from the paper [d] or implement the proposed attack on the models used by [d] to see the difference.

Strengths:

+ The proposed method work well on adv trained models and floating-point models.

+ Practical approach by a simple modification to existing gradient based attacks.


Weaknesses:

- Binary quantization is not a well accepted method, since it can in general introduce >5% accuracy loss. There are a lot more valuable quantization schemes to investigate such as low-bit-with fixed point, power of 2, and additive power of 2 (Y. Li, X. Dong, and W. Wang, “Additive powers-of-two quantization: An efficient non-uniform discretization for neural networks,” in International Conference on Learning Representations, 2020).

- The novelty is limited, since it brings the temperature scaling approach, an existing method, to the problem of attacking binary quantized models.

- The paper writing is not constructed for easy understanding. 


Comments and questions:

1. I would like to see comparisons with other attacks that are particularly designed for quantized models.

2. The third paragraph in Introduction, the paper tries to justify two techniques, but they are still not well motivated. 

3. The fourth paraph in Introduction mentions both full precision networks and floating-point networks. What’s the difference between these two?

4. Table 1 results are surprising. Is the same observation made by other ref works? 

5. The method is to replace the softmax with a monotonic function (softmax with a single scalar) during the attack generation. Then for testing the attack success rate, I think the neural network should still use the original softmax (without scalar). Then the attack success rate won’t be degraded?
review_title: This paper brings the temperature scaling method to attack generation mainly on BNN.

final_score: rating: 6: Marginally above acceptance threshold
review_content: **Update** : Since most of my issues have been addressed, I have changed my rating from 4 to 6

Summary:

This paper studies the robustness of quantized networks against gradient-based adversarial attacks (for L2 and Linf norms), showing how quantized models suffer from gradient vanishing, giving a false sense of security via gradient masking. To circumvent this issue, the authors propose temperature scaling approaches that can overcome this masking, achieving near-perfect perfect success in crafting adversarial inputs for these models.  

##########################################################################

Reasons for score: 

The paper's ultimate goal is to get better gradient-based attack performance on quantized (binarized, in this case) networks. However, key steps that should have been tried first for benchmarking such as adaptive PGD attacks have not been performed. Moreover, it is not clear what benefit the proposed method has in this scenario compared to gradient-free attacks like Boundary++.  The paper's contributions, although including some nice analyses on temperature scaling based solutions, are too weak to be accepted in their current form.

 
##########################################################################

Pros: 
  
- Improvement in attack success rates for full-precision networks, even for FGSM, seems like an exciting result. Further analyses and methods on top of this could be used to further increase the strength of these first-order gradient attacks.

- Jacobian and Hessian based detailed analyses of temperature scaling, and what different solutions correspond to in terms of robustness is quite insightful and interesting.  

##########################################################################

Cons: 

- Gradient masking is a relatively well-known phenomenon in adversarial machine learning. In cases when normal first-order gradient attacks fail, techniques like adaptive PGD attacks, gradient-free attacks, or even black-box transfer attacks are some straightforward methods to overcome gradient masking. Thus, it is not clear why the authors did not try non-gradient attacks before jumping to a complicated algorithm. At the very least, those attacks (like Boundary++) should at least be part of benchmarks for comparison. 
  - For starters, please refer to 'Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks': they have a [publicly available implementation](https://github.com/fra31/auto-attack ) as well
  - All of this is crucial, especially since the paper claims (Section 4) that "we would like to point out that the focus of this paper is to improve gradient-based attacks on already trained BNNs"

- In general, investigating if a model exhibits gradient masking is not a contribution: standard checks like comparing transfer rates, multi-step to single-step performance, attack rates for increasing attack budgets, etc are often used to check for gradient masking. 

- Figure 1: For which norm are these numbers reported? Without knowing the norm, it is hard to say if Figure b is a sign of gradient masking or not.

- Section 2.2 "..these attacks have been further strengthened by a random initial step". This is partially true: the real benefit comes from having multiple random restarts. Having just one random initialization by itself is not that useful. Please re-run evaluation experiments with random restarts (20 is a good number).

- Section 3: What does "adversarial accuracy" refer to? Is it accuracy on perturbed inputs f(x') = y, or success rate of the adversary when trying to change predictions aka f(x) ~= f(x')? Please clarify

- Section 3.1 "... clearly indicate gradient masking issues.." please elaborate: not every reader will be familiar with the set of checks used for gradient masking.

- Issues with Cross-Entropy based loss and how they promote certain magnitudes of logit values are not new. The authors might want to have a look at Section 4.1 of [Reliable Evaluation of Adversarial Robustness with an Ensemble of Diverse
Parameter-free Attacks](https://arxiv.org/pdf/2003.01690.pdf) to see if there are similarities/differences in the proposed temperature-based variant, and how the proposed method is better than the one in the Difference of Logits-Ratio based loss? This work seems to be a key and relevant part of related work and should be included in comparisons/benchmarking.
 
- "implementation of our algorithm will be released upon publication". Please anonymize and attach the code in response. 

- The benefit of using the proposed FGSM++/PGD++ attacks on full-precision models trained with adversarial robustness seems to be negligible (Table 4), and should not be overstated in results. Also, since these attacks all have random seeds, please perform experiments multiple times for statistical significance and report summary statistics. 

##########################################################################

Minor Edits:

- Section 2.2 "..perturbations to the images..." the definition here is for adversarial examples in general, and should thus be "perturbations to data"

- Section 2.2 "Gradient-based attacks can be... written as Projected Gradient Descent (PGD)" this is true only for first-order gradient-based attacks, not all gradient-based attacks (examples JSMA). Please correct.

- Section 4.1 "...since most of the modern networks consist of ReLU nonlinearities" this can (and often is) circumvented using Fake-ReLU. Example implementation [here](https://github.com/MadryLab/robustness/blob/89bdf8088a8f4bd4a8b86925a2801069ec281fee/robustness/tools/custom_modules.py#L5)

- Section 5 "...and they hypothesize that linear networks would be robust to adversarial attacks." this is not their conclusion, and seems to be out of context.

- Section 6 should preferably be either towards the end or at the beginning? Not clear why it is in the middle of other sections


Please address and clarify the cons above 
review_title: Some interesting techniques proposed, but contributions are too weak

Meta-review: The paper studies the robustness of binary neural networks (BNNS), showing how quantized models suffer from gradient vanishing. To solve this issue, the authors propose temperature scaling approaches that can overcome this masking, achieving near-perfect perfect success in crafting adversarial inputs for these models. The problem is interesting and important. However, the major concerns are that the technical novelty is limited raised by two Reviewers, small improvements for linear loss functions. The most related work is not compared in the experiment. 

Decision: Reject