Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: This paper studies query efficient black-box attacks, which aims to generate black-box adversarial examples using least number of queries to the unknown victim model. The authors proposed to leverage ensemble of surrogate models to generate some candidate perturbations and then query the unknown model. The query feedback is then used to update the weights of individual surrogate models to produce better candidate perturbations for the next query. The empirical results demonstrate that the proposed approach can obtain reasonably high success with handful of queries. 
strengths_and_weaknesses: The presentation of the paper is very clear and easy to follow. However, the paper lacks comprehensive review of the related works and also the adopted baselines are not convincing enough. Further, the experiments should be designed to illustrate the importance of each module in the attack process. More specifically,
1) many closely related works (especially attacks that combine transfer and query based attacks) are missing from the paper. A few samples are listed below [1], [2], [3], [4] and the authors are encouraged to conduct a more comprehensive survey on related works and include more recent baselines. For example, the listed baselines [1], [2] can be directly adopted as baselines for this paper and the combination method proposed by Suya et al. [3] might be coupled with the strong (should still be the state-of-the-art) query based attack [4] to form another strong baseline. The key is also to ensure that all baselines are using the same model ensembles. Since the nature of the proposed attack in this paper is "adaptive" transfer attacks using query feedback, it is really interesting to see if it is always advantageous over attacks that combine transfer and query based attacks directly. 
[1] Yang et al., "Learning Black-Box Attackers with Transferable Priors and Query Feedback", NeurIPS 2021.
[2] Ma et a.., "Simulating Unknown Target Models for Query-Efficient Black-box Attacks", CVPR 2021
[3] Suya et al., "Hybrid Batch Attacks: Finding Black-box Adversarial Examples with Limited Queries", USENIX Security 2020.
[4] Andriushchenko et al., "Square Attack: a query-efficient black-box adversarial attack via random search", ECCV 2020.
2) It is quite surprising to me that, targeted attacks even using the baselines only cost less than 100 queries in most cases. It may be because the selected target class for misclassification is still not hard enough. A more comprehensive way of evaluation might be to include three types of target classes: 1) easiest class (second highest class in original confidence scores), 2) random class and 3) hardest class (lowest class in the original confidence scores). 
3) at least the impact of ensemble diversity should be carefully measured. The current attack setup assumes the attacker has access to models that are trained on the exact same training data as the black-box model and also the ensemble set contains diverse enough architectures, which may not always hold in practice. 
4) In the attack results, maybe it is also better to report the direct transfer rates when different ensembles have equal weights. With this, we can see how much improvement is obtained by updating the surrogate model weights. 
questions: Most of the suggestions for improving the paper are listed in the "strengths and weaknesses" section.
limitations: Yes.
review_title: null

final_score: rating: 4: Borderline reject: Technically solid paper where reasons to reject, e.g., limited evaluation, outweigh reasons to accept, e.g., good evaluation. Please use sparingly.
review_content: summary: This paper proposes an ensemble search strategy for black-box adversarial attacks, which updates the weight of each ensemble member according to the queried feedback from target models. The empirical evaluation is done on the NeurIPS 2017 competition dataset (with 1,000 ImageNet-like images), using different model architectures. The results show that the query efficiency is largely improved.
strengths_and_weaknesses: Strengths:
- Clear writing and explanation of the proposed ensemble search strategy.
- The ensemble search strategy is simple but effective, leading to a significant empirical improvement.
- Commercial Google Cloud Vision API is also tested.

Weaknesses:
- My main concern is that in Table 1, the average query times of the proposed method (Ours) is around 1~2, which means the ensemble weights are only slightly updated. This makes me wonder if the effectiveness comes from the ensemble rather than the weight searching strategy.
- The architectures of the target model (e.g., VGG, DenseNet, ResNext in Figure 2) are included in the substitute ensemble, thus the ensemble search strategy is more like 'guessing the architecture of the target model'. This is not the common case in practice, because the substitute ensemble usually may not involve the architecture of the target model. More ablation studies on this aspect would be helpful.

questions: Please answer the listed weaknesses. My final rate may depend on the authors' response. 
limitations: Yes
review_title: null

final_score: rating: 4: Borderline reject: Technically solid paper where reasons to reject, e.g., limited evaluation, outweigh reasons to accept, e.g., good evaluation. Please use sparingly.
review_content: summary: This paper proposes a hybrid black-box attack on deep learning models that uses a combination of transfer and query-based attacks to reduce the number of queries while still maintaining a high targeted attack success rate. The main method involves creating a weighted ensemble of surrogate models, which is then used to generate local adversarial examples. These are tested against the victim model and the ensemble weights are updated using a form of coordinate descent based on the success of the locally generated adversarial examples. Experiments are carried out on non-robust models trained on the Imagenet dataset using standard Lp threat models. Both targeted and untargeted attacks are used. Real-world experiments on the Google Cloud Vision API also demonstrate the effectiveness of the method.
strengths_and_weaknesses: **Strengths**
+ The design of the proposed attack which involves optimizing over the weights in an ensemble of models to generate transferable adversarial examples ensures, by construction, that the number of queries used will be limited due to the significant amount of prior information leveraged to construct the adversarial examples. This is borne out by the empirical results, which use a small number of queries even for high-dimensional Imagenet data.
+ The attack description is clear and well-written. The attack is easy to implement given a number of trained models with adversarial examples that transfer to the model under consideration.
+ The attack is shown to be effective against different architectures.
**Weaknesses**
- My main concern with the paper is that the evaluation is rather thin. The experiments are carried out on a single dataset, and only models with non-robust training are considered. Also, only a single modality (image classification) and vision API (Google Cloud) is considered. For a more thorough evaluation, I would recommend that the authors do the following:
     - Attempt black-box attacks on another modality such as object recognition or segmentation. This will also enable the use of a different dataset such as MS-COCO.
    - Explore the space of attacks on robustly trained models. Does using robust models in the ensemble help or hinder the generation of effective adversarial examples? Does the number of queries needed to attack a robust model increase as the model's robustness increases (with respect to a specified threat model)?
   - Carry out a comparative evaluation of the attack's effectiveness on different vision APIs. This is also important to determine the relation between the chosen surrogate models and the target model.
   - Provide further explanations of the link between the models chosen in the ensemble and the target model. It appears that the models chosen to create the ensemble are very close to those targeted. What happens when the models in the ensemble are trained on a different dataset, for example, or a smaller subset of the same dataset?
- The paper is missing a reference to very closely related work from Suya et al. (Hybrid Batch Attacks: Finding Black-box Adversarial Examples with Limited Queries).
questions: - Line 262-263: Does the whitebox version involve taking gradients with respect to the weight vector $\mathbf{w}$? How is this gradient computed?
- Line 267-268: For the hard-label attacks using a surrogate victim model, is score based access to the surrogate model assumed? What is the link between the architecture of the surrogate and true victim models?
- Comparison with existing hard label attacks: Is there a reason there is no direct comparison to existing hard-label attacks such as the Hop-Skip-Jump attack (https://arxiv.org/abs/1904.02144)?

limitations: - While I appreciate that the authors discuss the runtime of the perturbation machine as a possible source of computational expense, I would have liked to see direct timing comparisons between the proposed attack and existing black-box attacks. This is particularly important since the per-query time for this attack is likely to be higher than attacks using simpler queries.
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: This paper proposes a new method in generating efficient adversarial examples by a set of surrogate models. They apply PGD over the losses computed by these surrogates and a group of weights to adjust the effect of different models. In the experiments, they can find the adversarial example for each input within only a few queries, which is 30x faster than previous methods.

strengths_and_weaknesses: The paper is easy to understand and the structure is clear. Overall, the presentation is also good. 
The idea of this paper makes sense and is straightforward.

The novelty of this paper or some analysis seems a little limited and not enough, which i mean, brings limited inspiration for our future studies. To solve this, a comprehensive theoretical analysis or more experimental analysis maybe needed.


questions: Here are some suggestions which may help improve this paper.

1. In equation (6), the author actually uses the target label y*, which is different from previous expression trying to maximize the loss. It is suggesting that the definition of y* is re-emphasized here for clarification. Similar problem can be found in the expression of PGD optimization.

2. The main idea or the advantage of proposed method is to leverage a variety of existing models to generate adversarial examples that could fool all the surrogate models. Such examples also intend to fool the targeted victim models in large probability. The trick of the method is to transfer the consumed queries in black-box attack to white-box attack performed by a variety of surrogate models. Thus, the number of T consumed by surrogate models should be well-studied and discussed. In experiments, for more comprehensive results, it is suggesting that an appendix column to show the number of T is added in the table to make it clearer or fairer.

3. why use coordinate gradient descent for updating w instead of applying SGD over all elements in w?

4. In the appendix, the authors also study the transferability of their generated adversarial examples in the task of object detection, which is good and convincing. I got another concern that, together the black-box classification task in the first phase, is data used for training surrogates the same as the data used in attack? I mean, for example, can the surrogate be trained in the CIFAR, and the attack is performed in the ImageNet. Can we assume the attacked data is unknown in advance and different from the data used in training surrogate?
limitations: See above.
review_title: null

Meta-review: metareview: This paper proposes BASES, a query-efficient black-box adversarial attack by first generating adversarial perturbation with gradient-based attack using a weighted ensemble of surrogate models. The perturbed image is used to query the target model and its feedback is used to update the weights via zeroth-order optimization. The method is simple, intuitive and well-presented, and the authors show that it achieves a high attack success rate using a very limited query budget. The method can be used for both score-based and hard-label attacks, and experiment on Google Cloud Vision demonstrates real-world applicability.

The most common concern among reviewers is the method’s dependence on ensemble diversity. AC agrees that this is an inherent limitation as Figure 3 in the paper shows drastically reduced attack success rate when using a smaller set of surrogate models. However, reviewers wjHs and vphn argued that the paper’s contributions outweigh this limitation. AC agrees with this characterization and recommends acceptance, and encourages the authors to incorporate additional results from the rebuttal in the camera ready version.

confidence: Certain
Decision: Accept