Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: The authors proposed a novel backdoor defense, which used first-order gradient to identify bad neurons using clean dataset. Also, a new Adaptive Regularization (AR) mechanism is used to fine-tune the model using detected bad neurons. They performed experiments on ten different backdoor attacks with three benchmark datasets. Their proposed method has performed better than other six state-of-the-art defense methods. 
strength_and_weaknesses: Strong Points:

1. A novel a gradient-based detection method of bad neurons.
2. The method focused on the last fully connected layers to remove backdoor. 
3. A new Adaptive Regularization (AR) mechanism is proposed to assist in purifying these identified bad neurons via fine-tuning.
4. Extensive experiment results illustrated the good performances. 

Weak point:

There is a paper published at CCS 2021: “AI-Lancet: Locating Error-inducing Neurons to Optimize Neural Networks”. This paper also considered gradient during finding bad neurons. They also achieved good results with respect to accuracy and ASR. Authors have not mentioned this paper nor compared with. They should include this paper for comparison to prove the effectiveness of their method. 

clarity,_quality,_novelty_and_reproducibility: Clarity, Quality, Novelty And Reproducibility are acceptable. 
summary_of_the_review: The authors used clean data to detect bad neurons and designed a new Adaptive Regularization (AR) mechanism for finetuning. However, they failed to mention and compare a very similar approach published in CCS’21, which also achieved good clean accuracy, ASR. Since these two algorithms are belongs to the same category. Lacking this comparison makes its contribution not clear.
review_title: null

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: This paper proposes a measure to distinguish benign neurons and malicious neurons. Based on the measure, the authors identify the malicious neurons and update their parameters. Experimental results show the effectiveness of the proposed method.
strength_and_weaknesses: **Strength**
1. The empirical results demonstrate the effectiveness of the proposed method.
2. If I understand correctly, the author defines the importance of a weight parameter using the element-wise product of the weights and their gradients. There is a similar formulation (element-wise product of the input and the gradient) in Saliency Maps [1, 2]. If we could find the similarities between them, it must be very interesting.

**Weakness**
1. This paper is written badly and requires major revision. I have a terrible time understanding this paper. For example, when citing related works, please use brackets “(” & “)” in most cases to include them, i.e., use \citep in Latex.

2. It is difficult to understand the formulation of Eqn (2). 
- Since DNN has an extremely large number of parameters, close-to-zero initialization still result in different loss compared to zero-initialization. Why could we replace  $L_{D_c}(w_0)$ with  $L_{D_c}(0)$?
- In this formulation, BS is defined for the whole model and every neuron share this value inside the model. So how can we distinguish neurons using this shared BS? Is the neuron-wise BS defined as the sum of the products of the parameters and their derivatives inside a specific neuron?

3. Where is Figure 4 in Sec 5.4?

4. If this paper regards it as a pruning method, it neglects an important work, ANP [3]. Since this is another pruning-based method, this paper should compare it.

[1] Avanti Shrikumar, Peyton Greenside, Anna Shcherbina, and Anshul Kundaje. Not just a black box: Learning important features through propagating activation differences. arXiv preprint arXiv:1605.01713, 2016.

[2] Julius Adebayo, Justin Gilmer, Michael Muelly, Ian Goodfellow, Moritz Hardt, Been Kim. Sanity Checks for Saliency Maps. In NeurIPS, 2018.

[3] Dongxian Wu and Yisen Wang. Adversarial Neuron Pruning Purifies Backdoored Deep Models. In NeurIPS, 2021.

clarity,_quality,_novelty_and_reproducibility: This paper misses many details to help readers to understand. Thus, it requires a major revision.

summary_of_the_review: This paper misses many details and is hard to understand. Even if the proposed method might be promising, the author should clarify it well. In my opinion, it requires a major revision. Thus, I recommend rejecting this paper.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: The paper proposes a novel backdoor defense method (called WIPER) that includes: i) a new metric, called benign salience, to identify backdoored neurons; and ii) a new adaptive regularization mechanism to assist in purifying the identified bad neurons. 
strength_and_weaknesses: Strengths: 
- empirical evaluation and effectiveness, i.e. the proposed method yields better quantitative results than the included competitor methods.
- relevant application: the fact of developing/designing safer and more resilient neural models is always welcome. 

Weaknesses: 
- not sure if technical/methodological novelty of this work with respect to the state-of-the-art is enough.
- not sure if authors are comparing (quantitatively and qualitatively) with all relevant prior work. 
- the organization, presentation and writing of the paper.
clarity,_quality,_novelty_and_reproducibility: I think overall the paper is interesting and is acceptably clear. The main ideas and goals are clearly stated, and the results are positive. In my opinion, the experimental evaluation of the proposed method that tries to answer the 4 research questions posed is one of the main assets of the paper. However, I'm hesitant about the novelty and technical and methodological contribution of the proposed algorithm. 

In terms of reproducibility, I'd say that the work is reproducible based on the main manuscript, the supplementary materials and the source code provided. 

Finally, there are several aspects related to the presentation/organization of the paper that could be improved:
- the writing of the paper (to avoid typos like "priro" -> "prior" or "decrease parameters to 0.." -> "decrease parameters to 0.")
- Table 4 is not referenced/commented in the body of the manuscript. 
- there is no global figure to introduce the whole approach and problematic in a visual manner (including an illustration of a backdoor attack). 
- the algorithm employed to describe the proposed paper is included in Supplementary Materials, and not in the main paper.
- captions of some figures are not particularly informative. For instance, in Figure 1. 
summary_of_the_review: See "Clarity, Quality, Novelty And Reproducibility" Section of this review.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: The paper presents a neuron-based method to defense against backdoor attacks. They first identify less important neurons in the network and then purify them using a fine-tuning based method. The paper contributes to both the selection of less important neurons and in purifying the network. Compared to the common method of selecting less important neurons based on the activation magnitude, authors argued that activation ignores the strong connections that a neuron may have and may result in selecting important neurons as the least important neurons. The authors proposed Benign salience (BS) that ranks the neurons based on how they contribute to reducing the loss. The neurons that contributed the least are identified as bad neurons. The bad neurons are then purified using an improved fine-tuning strategy. The experiments show that the proposed method outperforms all the baseline methods.
strength_and_weaknesses: The results are promising.
clarity,_quality,_novelty_and_reproducibility: I had clarity issues while reading the paper. A major line of work identifying important neurons is missing.
summary_of_the_review: - I found the writing of the paper a bit problematic. It mentions WIPER as their method but then mainly presents BS as their method. I did not get whether WIPER refers to only improved fine-tuning strategy or it refers to BS plus improved fine-tuning strategy.

- The paper mainly ignores various methods to identify the role of a neuron with respect to the prediction such as integrated gradient, deep lift, layer-wise relevance back propagation. Looking at just activation is indeed not the best way to identify important neurons but I would expect authors to compare their proposed method with the methods from the literature.

- Figure 2 (left): The fine neurons that authors are getting could be the result of the BadNet method that it learns to cleanly separate the poison neurons. Testing gradient based methods on this model would clarify the benefit of BS in comparison to other methods.

- (Section 4.2) I did not fully get the description of L1 and L2. L1 brings sparsity so it should turn the neurons will low weight/activations to zero quickly and L2 normalizes the spikes in the scores and it won't turn neurons to zero. Section 5.4 seems to be correctly discussing the effect of L1 and L2.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: This work proposed a new backdoor defense method by identifying bad neurons in a backdoor model, according to a new metric called benign salience. 

The main concerns from the reviewers include:
1. Lack of important related methods: 1) the work with similar idea published in CCS’21, which also utilized gradient to identify bad neurons; 2) various methods that measure the contribution of neuron; 3) some advanced backdoor defense methods, like ANP (and several other advanced methods). Although the authors added partial evaluations, they are insufficient to verify the superior performance of the proposed method. The lack of so many related works from different perspectives reveals that the work was not well prepared before submission. 
2. The writing is very poor. 

Moreover, the behind assumption of identifying bad neurons is that the neurons could be separated into good and bad neurons. However, the adaptive attack could produce a backdoor model within which the activation paths of both poisoned and clean samples are largely overlapped, such that the proposed method is expected to be evaded. It should also be discussed and studied. 
justification_for_why_not_higher_score: The lack of several related and important related work downgrade the novelty and effectiveness of the proposed method, as well as the completeness of the manuscript. The writing is very poor. No study of the adaptive attack.
Decision: Reject