Venue: ICLR 2021 Conference

Reviews:

final_score: rating: 3: Clear rejection
review_content: This paper presents a new method to defend black-box attack based on an ensemble of sign activation neural networks. The authors demonstrate their method has much higher minimum distortion using HopSkipJump attack.

However, I have many concerns regarding this paper:

-The paper organization is very weird and confusing. The author did not put their main algorithm into the main paper. Instead, they put many unimportant results (e.g. Table 1) into the main paper. If there is not enough space the author should use simpler sentences to describe their algorithm and put some results into supp. Also, the figure and table style (i.e., unbounded table, screenshot figures) makes me feel this is an undergrad project report instead of an ICLR submission.

-Besides transfer attack, the authors only evaluate the black-block robustness using HopSkipJump attack. Their claim is "Compared to other boundary attack methods it is known to give the best estimate of a datapoint’s minimum adversarial distortion." 
Is there any paper support this claim? I don't believe one attack method is universally better than other method among all datasets.
I think the authors should evaluate a set of attack methods instead of only one method otherwise the results are not convincing to me.

-What is the purpose of providing detailed results of 10 random images in Table 2, 3? Those results not only occupied a lot of space but also did not provide any useful insight. An average number of the entire dataset is enough.

-Citing issue: when referencing a paper the author should use the published version not the arxiv version if the cited paper is published.
E.g., Angus Galloway, Graham W Taylor, and Medhat Moussa. Attacking binarized neural networks. arXiv preprint arXiv:1711.00449, 2017.
should be
Galloway, Angus, Graham W. Taylor, and Medhat Moussa. Attacking Binarized Neural Networks. International Conference on Learning Representations. 2018.


-The baseline comparison are all undefended networks. The author should compare to some other blackbox defense methods. 
review_title: Unconvincing evaluation

final_score: rating: 5: Marginally below acceptance threshold
review_content: The paper proposes an architecture (ensemble of networks) aiming at being robust against black-box attacks, based on the idea that crafting an adversarial example able to fool enough individual networks such that the majority vote changes is a more difficult task.  The paper presents ways of training such ensembles and provides several sets of experiments showing the advantage of the approach. It also contains an observation on "non-transferability", counting how many co-networks are fooled when only one is targetted by the blackbox attack. It turns out that this amount is lower for the proposed scheme. 

The algorithms are postponed to supplementary material and the paper itself mainly report the experimental part. It concerns : 
-> training time
-> minimum adversarial distorsion (l2 and l_inf)
-> transferabiliy
-> different tasks (images, text and ecg)

All tested algorithms are ensembles and dataset are subparts of actual datasets.

Comments: 
* Table 1 : training time of a single run : should not it be avaraged on several runs somehow?  
* Table 2, 3, 5 : I would find it more readable if some remarkable values were bold for instance
* Table 4 : since all models don't have the same number of weak learners, I suggest that the presentation of the table recalls it, it * has a great impact on values here
* (a verb is missing on top of page 6. )
Figure 2 : hard to read. Top lines appears to be  some kind of reference to be compared to and the bottom lines the proposed methods. I suggest that one color is assigned to each algorithm and different line style for with or without GN. Also I think that there's a typo in the legend (SCDCE-BNN-> SCDCE-GN?) 
* biblio : I did not check each paper on arxiv but I'm quite sure that a significative part of arvix references have some published references.  Citing preprints is ok for recent work only. The reference by Alex Krizhesky seems to lack information too. 

Overall, I'm not really convinced by the paper in its present form, although I recognize that the results show some interesting properties for robustness.  Experiment on running time seems a little misleading, and shows that the most efficient variant is also significantly slower. I'm actually not against slower methods, but when I read this part I have a feeling of "is it right?" 
I also think that there would be enough space to provide more information on algorithms in the main paper. 
The transferability idea seems to be an interesting point, but it has to be further developped. How to apply this observation to the actual blackbox attack, which, as far as I understand, does not attack weak learners one by one?
review_title: an experimental paper that could be improved

final_score: rating: 4: Ok but not good enough - rejection
review_content: 

This paper proposes training an ensemble of binary neural networks with sign activations using gradient-free stochastic coordinate descent algorithm. 
The nature of the training method and binary networks leads to robust models with non-transferable attacks.

----
*Pros*:
I appreciated 
- the non-transferability experiment which demonstrated that the trained ensemble is diverse and sort-of "orthogonal".
- the use of minimum distortion as a measure of the model's robustness.

*Cons*:
- It looks like the method is not scalable. Experiments were carried out on a single-hidden-layer network of 20 nodes (would really like to see experiments on bigger models)
- Only one black-box attack is used to demonstrate the effectiveness of the method and that too under one perturbation threat (l2-norm). There are multiple black-box attacks that exploit different aspects of model vulnerabilities. It cant be claimed that the method is robust based on a single-attack single-threat.
- The scale of experiments (10 images) does not provide significant evidence that this method is effective and the authors acknowledge that in Section 3.6


Remarks/Comments:
- Page 2, Line2: "by with gradient" -> "with gradient".
- Sec 3.2: "divide by 9900" -> "divide by 99".
- Table 1: What does "single run" mean in the caption? Is it a single forward-pass?
- Fig 2: please make it more legible with markers (colors are not enough).


Based on the above, I suggest that the authors redesign their experiment to support the paper's proposition in terms of the method scalability and robustness against more attacks & perturbation settings. Perhaps making the experiments more scalable by reducing the compute budget of HopSkipJump (1000 random init x 10 times x 100 iterations)

**Post-Rebuttal**: No change in my score, please read my comments in the thread below.

review_title: Lots of work is needed to improve the paper

Meta-review: The paper proposes an algorithm to defend against black-box attacks. All the reviewers think the current experiments are not convincing enough, and the method seems to have some issues (e.g., not scalable). 
Decision: Reject