Venue: NeurIPS 2021 Conference

Reviews:

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: The authors propose a meta-learning framework to improve the query efficiency for Square Attack.

main_review: This paper focus on improving the query efficiency of a score-based black-box attack named Square Attack.
Specifically, the authors make some hyper-parameters of Square Attack (i.e., square size and the color channel) to be adaptive.
The authors use current step size, the history of loss values and the success frequencies of different color channels as inputs, and design MLPs to map from these inputs to better values of square size and color channel.
The MLPs are then trained using a meta-learning formulation, to make the learned strategies (parameterized by MLPs) to have better generalization performance on unseen images and models.

Strengths:
* The proposed method seems to be technically sound and novel. The square size and color channel schedules in Square Attack are crucial for query efficiency, but these parameters require significant manual design. The authors propose to directly learn these parameters via meta-learning, such that the learned schedules can work well on unseen images and models.
* The controllers are trained on a white-box ResNet-18 model, and are evaluated on black-box RobustBench models. As such, an attacker does not need to know the detailed architecture of the target model, making the proposed method more practical.

Weakness:
* The authors claim in the abstract that the controller "can be plugged into random search methods", however, the proposed method seems to be deeply coupled with one particular attack (i.e., Square Attack). Can the proposed method be easily integrated with other attacks?
* In line 38-39, the authors said "...propose a method to ...reduce the amount of manual design in random search based attacks.". However, the proposed two controllers themselves also require significant manual design (in Section 3.3).
* In Table 1, although the proposed method has better query efficiency than competitive methods, the differences are not great in my opinion, given the fact that competitive methods use manually designed schedules while the proposed method use learned schedules. I wonder whether the advantage of the proposed method is more clear in extremely low query budget regime (e.g., < 150 queries), since in the early stage, the accuracies have not been saturated, thus a good search distribution could possibly make more contribution to the query efficiency.

limitations_and_societal_impact: Yes
review_title: null

final_score: rating: 5: Marginally below the acceptance threshold
review_content: summary: The paper proposes a meta-learning approach to produce optimistic hyperparameters for score-based black-box adversarial attack, especially for Square Attack. A controller is trained on the source network to find the optimistic schedule of hyperparamters. The success rate is improved using the controller.
main_review: **Pros**

1. Meta learning is a reasonable and promising way to improve the query efficiency and success rate of black-box attack. The attacker can use the information from the query and adjust itself to gain better performance.
2. The experiments, despite quite inadequate, do show that the hyperparameters of the Square Attack is sensitive to some hyperparameters and tuned hyperparameters can improve the query efficiency. 

**Cons**

1. The paper fails to prove that the controller is able to generalize to unseen models. The models used in Table 1 are all adversarially trained model with similar loss landscapes. And the controller is trained on an adversarially trained model (Resnet18), which is similar to the models used for evaluation. I think the author should show that the controller is able to generalize to different settings such as attack for undefended models and targeted attack. 
2. The author uses the information of the source model. Although the information is quite limited, the method should be compared with the methods combined transfer-based attack and score-based attack such as [1][2].
3. The author uses two inputs for the MLP. I think there should be an ablation study to show the how each input affect the finall success rate and query efficiency.
4. While the controller is promising direction for black-box attack, the authors only show its effectiveness on Square Attack. I would be better to show the controller is able to control other hyperparameters in different black-box attacks.


**Minors**

The notation $\mathcal{D}_\omega$ should be changed. It is easy to be mixed up with the notation $D$.

[1] Zhichao Huang, Tong Zhang. Black-box adversarial attack with transferable model-based embedding, ICLR 2020

[2] Yusuke Tashiro, Yang Song, Stefano Ermon. Diversity Can Be Transferred: Output Diversification for White- and Black-box Attacks, NeurIPS 2020
limitations_and_societal_impact: See above.
review_title: null

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: This paper proposes meta-learning a black box adversarial attack strategy. Specifically, the authors propose meta-learning controllers that output the parameters of a square attack. Experiments on CIFAR-10 and ImageNet demonstrate the effectiveness of a meta-learned square attack relative to a non meta-learned baseline square attack.
main_review: **Originality**
As the authors point out, prior works have considered meta-learning attacks on neural networks. The authors propose meta-learning parameters of an existing black box attack. Nevertheless, the specific approach used by the authors appears novel.

**Quality**
The effectiveness of the proposed meta-learned square attack is evaluated experimentally on CIFAR-10 and ImageNet. However, there are some issues with the experiments:

First, the proposed method is framed as a meta-learned attack. However, the authors meta-train on a single dataset (CIFAR-10). Meta-learning typically refers to scenarios where meta-training occurs on a *set* of datasets and meta-testing occurs on a *different* set of datasets. The method proposed by the authors might be better framed as hyperparameter optimization or transfer learning rather than meta-learning. Alternatively, the authors might consider meta-training over a set of datasets and meta-testing on a different set of datasets.

Second, the improvements over baselines appear relatively incremental. In Table 1, it appears that meta-learning reduces the number of queries to achieve a certain accuracy by a factor of 2 or less. If the authors could reduce the number of queries by an order of magnitude of more, this would significantly improve the empirical experiments.

Relatedly, how does the proposed attack compared to other black box attacks (other than the square attack)? In particular, is the proposed method the state-of-the-art black box attack on CIFAR-10 or ImageNet (compared to previously proposed black box attacks)?

**Clarity**
The paper is well-written, with concepts introduced at appropriate points. Figures and tables are clear and appropriately sized. Experiments are adequately described.

**Significance**
Overall, the conceptual and empirical contributions in this paper appear relatively limited. Improvements over previous baselines seem to be small. If the authors can demonstrate that their results are a state-of-the-art black box attack, this would likely have practical value for neural network attackers and defenders. In general, the significance of this work depends strongly on the empirical improvement over prior works.
limitations_and_societal_impact: The authors describe the limitations and social impact of this work in section 5. However, the authors do not adequately potential strategies that could be used to limit the negative social impact of adversarial attacks. This is particularly important if proposed method is a state-of-the-art black box adversarial attack. The authors may want to mention a few specific mitigation strategies to limit the negative impacts of their proposed attack.
review_title: null

final_score: rating: 5: Marginally below the acceptance threshold
review_content: summary: The paper proposes a novel technique to improve query efficiency of black-box random search based adversarial attacks. The proposed method utilizes a controller which manages scheduling and proposal distribution for random search. The controller is trained by gradient-based meta-learning on surrogate models. Empirical results show that the Square Attack with the proposed method improves query efficiency compared to the original setting on CIFAR10 without manual fine-tuning.
main_review: Pros:
- The paper is well motivated.
- The characteristics of learned controller are useful and interesting.

Cons:
- The advantage over existing hyperparameter search methods is not clear.
- The proposed method needs the task specific controller design.
- Additional data and training are a strong assumption but not well explained.
- Experiments are not held in various settings.

While the motivation is clear and some of the results are interesting, it seems to need more explanations and experiments for the following points:

- The proposed method in this paper is similar to hyperparameter search (optimization) methods by Bayesian optimization or meta-learning, but the relationship with these methods is not explained. When random search based attacks only include simple hyperparameters like step size, these methods can also be used. In this case, does the proposed method have advantages over existing methods?  
In addition, existing hyperparameter search methods would be applicable to the Square Attack under assumptions of simple distributions (for example, linear or quadratic size schedule). There is no empirical comparison with hyperparameter search methods, and it is not clear that the proposed method is better than the existing methods.

- The method needs the controller design which is specific to the task. The controller in Section 3.3 seems carefully designed and not simple. While the proposed method can be used without manual fine-tuning, the complex hand-designed architecture makes the advantage diminishing.

- The requirement of source models (i.e., surrogate models) and additional training is a strong assumption, and the assumption should be more explained. Although the authors explain this point by “on model with white-box access” in the abstract and introduction, the relationship between the white-box model and black-box model is not clear. In addition, while there are many existing methods which exploit surrogate models to improve the query efficiency, the paper does not mention the existing methods.

- In the experiment, the paper uses an adversarially trained ResNet model as source model, and all the target models are also adversarially trained ResNet-based models. This setting is not enough to show the strength of the proposed method. As the authors mentioned in line 66, the similarity between source and target models often affects the attack performance when the attack exploits source (surrogate) models. When the source model is trained with different architectures/training settings (e.g., DenseNet and naturally trained model), does the proposed method still outperform the baseline?

- In addition to the above similarity between the source model and target models, the experiments are held in only 1 dataset and 1 attack. Moreover, the improvement is not significant and there are still non-negligible margins between the result and robust accuracy. Therefore, it seems to need more experiments to show the advantage of the proposed method.

limitations_and_societal_impact: The authors have described limitations about the usage of surrogate models. 

The authors have addressed societal impacts in Section 5.
review_title: null

Meta-review: First, we would like to commend the authors on an interesting submission and a heroic effort in the response period, including several experimental findings to help address reviewer concerns. It is extremely difficult to make a decision for this paper --- it is on the borderline. The scores are all borderline, tending on average towards reject, without a clear champion. However, the authors do make a reasonably good case that they are proposing a query efficient black box attack. In the rebuttal they also attempt to address ethical concerns. On the other hand, there are remaining concerns about the technical contribution, particularly how specific the approach is to the square attack. The questions are mostly around how applicable the ideas are to other attacks, and how much effort would be required to design controllers for other attacks. In any case, the effort the reviewers have made in responding to reviewers should be highly useful in subsequent revisions. 
Decision: Accept (Poster)