Venue: NeurIPS 2022 Track Datasets and Benchmarks

Reviews:

final_score: rating: 8: Top 50% of accepted papers, clear accept
review_content: summary and contributions: The paper deals with physical triggers for backdoor attacks. Physical triggers are real-world objects present in images at their creation and such backdoors have been shown to successfully evade existing defenses for object and facial recognition [31-CVPR21] (work from the same group). This paper deals with a cheap way to generate datasets with physical triggers. The key insight is to use existing widely used image datasets and relabel some trigger objects in them. 

strengths: * This is an important and relatively unexplored area of work (physical triggers for backdoor attacks).
* The work has a clear intuition.
* The design using network centrality measures is well done.
* The evaluation seems to back up the intuition.

weaknesses: * I would have liked to see further discussion of how applicable this technique is to the more general class of backdoor attacks and defenses, beyond physical triggers.
* It was not clear to me how the work generalizes to more number of trigger classes of objects.

correctness: The dataset is soundly constructed. There is good support for reproducibility through the run_on_gpus_centrality_ablate.py script that is set up to reproduce the results from Figures 5 and 13 in the paper. I wondered about reproducibility for Figures 6 and 7 though. 

For ease of use, the authors should provide a repository with images with the trigger objects inserted, so that one can readily train a model using the dataset. 

clarity: The paper is well structured and well written. 

relation to prior work: This is a relatively new area. The most closely related work is from the authors' group: [31].

There are a few other works in backdoors in images that the authors can discuss [A][B].
[A] Li, Yiming, Tongqing Zhai, Baoyuan Wu, Yong Jiang, Zhifeng Li, and Shutao Xia. "Rethinking the trigger of backdoor attack." arXiv preprint arXiv:2004.04692 (2020).
[B] Lin, Junyu, Lei Xu, Yingqi Liu, and Xiangyu Zhang. "Composite backdoor attack for deep neural network by mixing existing benign features." In Proceedings of the 2020 ACM SIGSAC Conference on Computer and Communications Security, pp. 113-131. 2020.

additional feedback: What about backdoor defenses against non-physical triggers? Do they transfer over well to the physical world backdoors? [30] is a representative sample of this. What is fundamental about these triggers that escape detection by defenses?

Lines 167-172: You could consider multiple trigger objects rather than a single one. What is the interaction of these multiple triggers? Is it always best to pick the top-k or combinatorial effects of two (or more) trigger classes that are individually lower ranked is more significant?

What is the relative contribution of these two factors in choosing the trigger class --- centrality and MIS? While an experiment is given (Table 1, lines 270-275), some intuition about what helps how much would be good.

There is good comparison of performance of existing backdoor defenses against their generated datasets.
review_title: Method to automatically generate physical backdoor attack datasets

final_score: rating: 4: Ok but not good enough - rejection
review_content: summary and contributions: This paper presents a novel idea for facilitating the generation of physical backdoors. Using many co-occurring objects in a public dataset as triggers significantly reduces the cost of generating physical backdoors. Using graphs to represent the occurrence relationships between objects and find important nodes and corresponding MIS shows promising results.
strengths: This idea alleviates the high trigger generation costs in physical backdoors is good.


weaknesses: This paper lacks a comparison with past physical attack methods. 

The number of defense methods used in this paper is small, and these methods are old. 

This paper does not include experimental results for the other two scenarios declared in the section "Other usage scenarios", which makes us unsure whether the algorithms combine well with other physical attacks in more general cases.


----- 

Considering there is one month for rebuttal, I don't think the authors have made good efforts to make this work better. Ignoring the reviewers suggestions is not a smart rebuttal strategy. 

I also carefully read other reviewers' comments and the authors' responses. I get a similar opinion, and several important and good suggestions/concerns are not followed/addressed. 

I keep the initial score.
correctness: Correct. 
clarity: could be improved. 
relation to prior work: yes
additional feedback: no
review_title: Valuable work but need improvements

final_score: rating: 6: Marginally above acceptance threshold
review_content: summary and contributions: The paper proposed a method to obtain the Natural Backdoor Dataset by modifying the labels of specific samples in the open source dataset. The paper develops some criteria to select appropriate multi-object pictures to ensure that the trigger can be recognized and triggered by multiple classes. The paper proves the existence of trigger selected according to the measures and provides the results of the post-attack model and backdoor defense against natural backdoor.
strengths: 1. This paper selects the hidden trigger in the open source dataset through multi-label data, which greatly improves the method of obtaining attack data in the previous physical Backdoor method.
2. This paper ensures the co-occurence of trigger and other types of data by designing measures.
3. This paper has done sufficient data experiments to verify the existence and feasibility of Natural Backdoor
weaknesses: 1.Lack of standardized dataset. As this paper is in the track of Dataset & Benchmark, an accessible dataset is needed for fair evaluation. 
2.Although the method of establishing the backdoor in this paper is different from the previous physical backdoor, it does not compare with the attack situation after establishing the backdoor before.
3.In sec 5.4, some backdoor defense method are not considered. [1] can identify the backdoor data. [2,3,4] can mitigate backdoor attack. However, the above backdoor defense methods are not used to make experiments to show whether natural backdoor can be identified or mitigated.
4.The performance of trigger selected for different measures is not stable enough in the two data sets in Fig. 5,13. Whether it is necessary to select appropriate selection indicators for different data sets

[1] Guo J, Li A, Liu C. Aeva: Black-box backdoor detection using adversarial extreme value analysis[J]. arXiv preprint arXiv:2110.14880, 2021.
[2]Huang K, Li Y, Wu B, et al. Backdoor defense via decoupling the training process[J]. arXiv preprint arXiv:2202.03423, 2022.
[3]Wu D, Wang Y. Adversarial neuron pruning purifies backdoored deep models[J]. Advances in Neural Information Processing Systems, 2021, 34: 16913-16925.
[4]Li Y, Lyu X, Koren N, et al. Neural attention distillation: Erasing backdoor triggers from deep neural networks[J]. arXiv preprint arXiv:2101.05930, 2021.

correctness: Yes, this paper selects the data that meet the selection criteria from the multi-label open source data set and then modifies the labels of these data.
clarity: The citation in the supplementary material is false. For example, wrong citation in line 483 "[18] uses light-based reflections as backdoor triggers."
relation to prior work: Yes, this paper compares the previous setting method of physical backdoor and some other backdoor method. However, the article lacks the introduction of the backdoor defense and some related work. The backdoor defense method in Sec 5.4 is lacked.



additional feedback: No.
review_title: Natural Backdoor selected through multi-label

final_score: rating: 7: Good paper, accept
review_content: summary and contributions: This contribution proposes a methodology to poison existing datasets with the objective to add backdoors in object classification and recognition models. The method relies on the identification of categories that could serve as potential triggers in backdoored models, by searching for physically co-located objects in datasets using different centrality metrics and graph algorithms. 
This automated process allows for the generation of poisoned datasets, without the need to manually construct triggers in physical conditions. This has the potential to favor research works on backdoor attacks in machine learning models.
strengths: The main strength of the contribution is the versatility of the tool that allows for the generation of multiple datasets of reasonable size for the study of backdoors in machine learning models. The paper carefully assesses the soundness of the approach through the study of 3 properties, and in particular their resistance to existing defensive mechanisms. This work provides a meaningful tool for researchers and fills a gap in the field of adversarial machine learning on this special types of attacks. It could help design better security mechanisms and provide benchmarking tools for researchers of the field.

**Update after the rebuttal period**: The authors took the time to carefully respond to my comments and address the weaknesses I mentioned in my initial review. I'm therefore updating my rating (5->7) to take into account the changes in the revised version.
weaknesses: The paper presents in my opinion three main weaknesses:

1. The title of the paper does not reflect the content. First, it is not a dataset but rather a methodology. Second, it is limited to multi-object image datasets (even if this could be extended to more broad categories of data, but this point is not developed in the paper). Third, the term "natural backdoor" is ambiguous and does not describe well the hypothesis on which the authors rely to construct their methodology.

2. The scope of the paper is too narrow to be fully relevant for the study of backdoored models. It is too restrictive in terms of application (multi-object image classification) and the datasets that are obtained might be too limited in terms of size and diversity, challenging the claim of scalability made by the authors.

3. There is finally little discussion about the cybersecurity context relevant for this type of datasets. In particular the threat models in which physical triggers as known objects are relevant is not clear. Obvious defensive mechanisms, such as a sanity check on the labels, are not mentioned and may be able to detect poisoning of the data. The definition of a proper cybersecurity context would help the cybersecurity community to adopt this approach and extend this work beyond the machine learning community, with the aim to address specific security issues connected to backdoored models.

4. The codebase relies on the python package 'graph-tool' which is not actively maintained. This leads to conflicts between packages (as acknowledged by the authors in the github project description) and with python version ('graph-tools that and does not offer support for version above python 3.3. This makes the integration of the codebase in projects more challenging, and limits the dissemination of the work.
correctness: The different steps of the methodology seem reasonable and globally well-justified. The code cannot be straightforwardly reproduced because of various conflicts, limiting the verification of the results made in the paper.
clarity: The paper is well-written and easy to read.
relation to prior work: The related work section clearly discussed the difference between the proposed work and previous works on backdoors.
additional feedback: Despite the good work put forward by the authors, the submission fails in my opinion to provide a tool relevant for the machine learning and cybersecurity communities. While research works based on this tool may be relevant, it may be too specific and not of sufficient quality to benefit other researchers in this current state of development. The paper also fails to set the scope in a clear manner and does not provide sufficient motivations and contexts to be adopted by interested researchers.
review_title: Promising yet unfinished contribution to research on backdoor attacks 

final_score: rating: 5: Marginally below acceptance threshold
review_content: summary and contributions: This paper proposes a method to build natural backdoor datasets with physical triggers. They utilize the fact that existing computer vision datasets already contain many co-occurring objects. The way they build the dataset is to 
1) identify natural backdoor triggers via graph centrality to find high coverage and frequent objects as effective triggers.
2) find the co-occurrence images with triggers and normal objects, and modify the images labels.

They evaluate the datasets through three aspects: existence, efficacy and defense resistance. 

This is an interesting and useful work in backdoor attack.

strengths: 1. The created dataset is very useful in backdoor scenario, and an interesting research direction. Also, the dataset itself is constructed in a sound way.
2. Good evaluations in section 5. The paper provides trigger efficiency analysis and defense resistance analysis, which is helpful.
3. The physical object backdoor attack is not wildly explored in previous research, so this paper definitely provides an useful and necessary task for backdoor community.
weaknesses: 1. This method requires multi-label datasets (if I understand correctly). What if the dataset itself only have single label, without any annotations of other objects labels? If the method can generalize to the above senario, it would make a more interesting and practical case.
2. In Table1, Figure7, the 'trigger accuracy' indicates the Attack success rate (ASR)? If so, should it be more than 90%? Otherwise, what is the ASR for the backdoor attack?
3. The poison data injection rate is 0.2,which is relatively high in backdoor attack tasks. In Figure 6, when the injection rate is low, the performance is not good. This indicates the trojan attack might be not very stealthy. A high backdoor attack performance with low injection rate would build a more competative method.

correctness: The dataset is constructed in a sound way. 
clarity: The paper is well written. 
relation to prior work: This paper mensioned a previous work [31], which is a physical object attack paper. 
additional feedback: Nope.
review_title: This paper propose a way to build the backdoor datasets based on existing multi-label datasets.

final_score: rating: 4: Ok but not good enough - rejection
review_content: summary and contributions: This paper presents natural backdoor datasets, investigating the backdoor attack in the physical world. Then conduct experiments to verify the feasibility of physical backdoor attacks based on the constructed dataset. Develop algorithms (tools) to curate natural backdoor datasets from existing object recognition datasets.
strengths: - Propose effective algorithms to construct natural backdoor datasets from existing large-scale multi-label object recognition datasets.
- Conduct experiments to reveal the harm and feasibility of physical backdoor attacks. 
- The constructed dataset has great potential significance beyond the utilization mentioned in this paper.
weaknesses: Major points:
- Although this paper carefully designs algorithms to identify co-occurrence objects and natural backdoor triggers based on graph analysis,  no human post-inspection has been conducted. Thus, we don't have a clear understanding of the label noise existing in the constructed datasets, which harms the credibility. 
- This paper analyzes the constructed datasets from three perspectives, including existence, efficacy, and defense resistance. However, I think the analysis stays on the surface. Detailed and in-depth analysis of proposed datasets is needed especially. 
- The story in this paper follows the common pattern (poisoned dataset -> attack -> adopting existing defense methods). Although this may not be a drawback, no interesting findings and conclusions are presented in the paper.  The concept of natural backdoor triggers may be relevant to spurious correlation, causal inference, and adversarial robustness. 
- Similar dataset has been proposed in this [paper](https://openreview.net/forum?id=MTex8qKavoS)



Minor points:
- A discussion of potential defense methods against physical backdoor attacks will increase the depth of the paper.
- The table format is problematic I think. The captions should be put on top of the table. 
correctness: The claims are correct. The dataset is constructed in a sound way. 
clarity: Yes
relation to prior work: Yes
additional feedback: No
review_title: Good research problem but no interesting and in-depth analysis (conclusions) is presented in the paper

Meta-review: metareview: The main contribution of the work is a versatile tool to generate natural backdoor datasets that can be used to benchmark future models and backdoor defenses. The reviewers acknowledge that having a low-cost approach to generating these physical backdoor datasets is important since there is a typically a high cost to introduce physical triggers in realistic training data.

However, some concerns were raised over the fact that the authors provide a technique to generate natural backdoor datasets, but do not curate a standardized benchmark for the community.  Additionally, the authors do not use human verification of labels, which some reviewers argue harms the credibility of the constructed datasets.

The authors do release sample datasets for the community, which I think addresses the concern regarding lack of benchmark datasets. I would additionally encourage the authors to spend effort validating that the techniques to generate such datasets are producing sensible labels, so that the community has increased trust in both the tool and the released benchmarks.
confidence: 3: The area chair is somewhat confident
Decision: Accept