Venue: NeurIPS 2021 Conference

Reviews:

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: The proposed work identifies that backdoor attack examples converge fairly quickly compared to clean examples during training, due to the nature of their objective. Using this, the authors identify backdoor examples and use a gradient ascent method to unlearn the backdoor, while gradient descent is used for clean examples. The proposed method is shown to be effective in reducing Attack Success Rate (ASR) while increasing Clean Accuracy (CA) on multiple datasets and attacks. 
main_review: Strengths:

— The proposed work is shown to be effective against multiple attacks for CIFAR10,GTSRB and ImageNet subset dataset.

— The paper is well written and easy to understand.

— The proposed method is compared against different unlearning and detection methods, showing best performance.

Weaknesses:

— While the authors consider different class of attacks, it is not known if the training loss observation holds for other forms of optimization or feature-collision based attacks [15,17,19]. 

Additional comments:

— In L141, it is mentioned that 6 backdoor attacks are used to poison 10% of CIFAR10 training data. Does that mean the poisoned dataset consists of all 6 attacks? If so, it would be interesting to see if transferability across attacks hold in this scenario- training data consists of 5 backdoor attacks and test set backdoor data comes from the 6th. 

Due to the gradient ascent procedure, the network might unlearn only those 5 backdoors. As a result, the Re-training from scratch method (presented in Table 2) might perform better compared to ABL on ASR. An experiment in this setting would help understand the effect of L_GGA.

— How does Tuning epoch affect the weaker SIG attack?
limitations_and_societal_impact: The authors discuss limitations and potential negative impacts.
review_title: null

final_score: rating: 5: Marginally below the acceptance threshold
review_content: summary: This paper observes that deep neural networks learn backdoored data faster than
benign samples. Based on this finding, the paper proposes Anti-Backdoor Learning
(ABL), which can learn a benign model on poisoned datasets. Specifically, at the
beginning of model training, ABL maximizes the training loss gap between clean
samples and backdoored examples and detects backdoored data based on the
differences in their loss values. Then, ABL unlearns backdoored model with the
detected data.
main_review: 
This paper finds that deep neural models learn backdoored samples faster than
benign samples, which is interesting. However, it lacks sufficient analysis to
support this conclusion. For example, the paper only shows the results on
ResNet-18 and CIFAR-10 in figure 2 to support the conclusion. One model and one
data set are not representative enough.

Also, the observation is drawn from simple backdoor attacks. I suggest training
them on complicated backdoor attacks, such as feature space backdoor and latent
space backdoor, e.g., Cheng et al. AAAI 2021, Lin et al. CCS 2020, Yao et al.
CCS 2019.

Moreover, the paper assumes that data poisoning happens at the beginning of the
model training. In the data poisoning attack scenario, backdoor attacks can
happen at any time. If the attacker poisons the data dynamically, how would this
defense effectively defend such attacks?

It is unclear to me how this approach can generalize to other threat models,
such as federated learning.

Finally, it seems the selection of the fixed loss value is quite important and
hard to decide. If the fixed loss value is large, the training loss of the model
cannot decrease after it reaches the fixed loss value and the accuracy of the
model cannot increase. If the fixed loss value is small, it is harder to detect
backdoored samples. Based on my understanding, the fixed loss value needs to be
adjusted based on the validation dataset but it is hard to create a validation
dataset for poisoned datasets because we do not know whether the attack exists
and the attack type. The paper can add a discussion about this problem.

A minor comment:

In Table 1, it would be helpful if the paper can provide results of the SIG
attack on ImageNet subset dataset because it seems ABL may perform worse when
defense against SIG attack (from the results on GTSRB).

In "Effectiveness with Different Turning Epochs", the paper claims that epoch 20
achieves the best results for all datasets but only shows results on CIFAR-10.
It would be better if the paper can show results on other datasets.

Updates after rebuttal: 

Thank the authors for providing new experiments and justify their design/arguments. They addressed most of my concerns. I would like to raise my score but I still have a concern that when the trigger is complex enough (i.e., close to benign features learned by neural networks), the model will take longer time to learn, in which case, the foundation of this defense is broken.
limitations_and_societal_impact: The work uses existing assets but did not discuss license issues.
review_title: null

final_score: rating: 7: Good paper, accept
review_content: summary: Authors have introduced the concept of anti-backdoor learning to learn clean models using backdoor-poisoned data. They have proposed Anti-Backdoor Learning (ABL) to automatically break backdoor attack during training with poisoned data. They have identified two weaknesses from backdoor attack. First weakness is that models learn from poisoned data at a much faster rate than these learn from clean data. The stronger the attack, the faster the models converge to a backdoor. The second weakness is that the backdoor task is tied to a specific class (the backdoor target class). Based on the two weaknesses, the authors designed a two-step method to learn a clean model with poisoned data.  The first step is to isolate poisoned samples from the training dataset, and the second step is to use the isolated samples to unlearn backdoor. The first step happened in the early training stage and the second stage takes over the training after the first stage. As compared to other state of the arts, the isolation step is very precise, and in many case, the precision is easily to reach 100%. The second step is very effective to unlearn backdoors. Even if only 1% of the poisoned samples are isolated, it can counter up to 70% poisoned samples to learn a clean model. Overall, this is a good paper and the proposed method outperformed the competing methods under the authors’ experimental setting. 
main_review: Originality: Are the tasks or methods new? Is the work a novel combination of well-known techniques? (This can be valuable!) Is it clear how this work differs from previous contributions? Is related work adequately cited?

This is a new method as long as I am aware of, and I think references are adequate. 

Quality: Is the submission technically sound? Are claims well supported (e.g., by theoretical analysis or experimental results)? Are the methods used appropriate? Is this a complete piece of work or work in progress? Are the authors careful and honest about evaluating both the strengths and weaknesses of their work?

The paper is technically sound. Claims are well supported by their experimental results. However, the models being used in the paper are way below the performances of the state of the arts. For example, state of the arts on CIFAR10 are way above 90% (close to 100%), and the models being used in this paper obtained less than 90% of accuracies. How would this affect your conclusions? 

Clarity: Is the submission clearly written? Is it well organized? (If not, please make constructive suggestions for improving its clarity.) Does it adequately inform the reader? (Note that a superbly written paper provides enough information for an expert reader to reproduce its results.)

The paper is well organized, pleasant to read and easy to follow. 

Significance: Are the results important? Are others (researchers or practitioners) likely to use the ideas or build on them? Does the submission address a difficult task in a better way than previous work? Does it advance the state of the art in a demonstrable way? Does it provide unique data, unique conclusions about existing data, or a unique theoretical or experimental approach?

I believe the solution provided by the paper is significant. However, the authors did not discuss the computational complexities. For example, if the user gives a pass of poisoned sample detection before using the data to train a model, how much extra time s/he needs to spend, assuming the poisoned data screening process is effective? This information will give important information of the importance of the paper. 

limitations_and_societal_impact: In addition to the limitations mentioned by the authors, here are some other concerns of this reviewer: all the showcases are based on the relatively simple attack: BadNets. For example, The Stress Testing showed that with a fixing 1% Isolation rate, the proposed method is still very effective with up to poising rate of 70%, which is fantastic meaning that the 1% identified poisoned samples can overpower the other unidentified 69% poisoned samples with the proposed method. Is it true for other relatively complicated attacks?
review_title: null

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: This paper observes that models converge much faster on backdoored data than clean data. Within a few epochs, the training loss for backdoored data will decrease to zero, but not for clean data. Based on such an observation, this paper aims to train clean models on poisoned data without knowing the backdoored portion in the training set. Specifically, it proposes a new training procedure called anti-backdoor learning (ABL), which splits the traditional training procedure into two stages. In the first stage, ABL uses a threshold to prevent loss values for individual samples being too small by flipping the sign of the loss. It then selects a small portion (1%) of the training set with small losses as the backdoored set and the remaining as the clean set. In the second stage, ABL applies gradient ascent on the backdoored set and gradient descent on the clean set. The evaluation is conducted on three standard benchmarks and ABL is compared with three existing backdoor removal baselines. The experimental results show ABL has the best performance on both reducing attack success rate and retaining normal accuracy in most cases. A list of studies on different components of ABL and comparisons with other alternative methods are interesting and informative.
main_review: The observation on the training loss of backdoored data is interesting. The proposed training procedure leverages this observation to counter backdoor effects, and the results show the effectiveness of the proposed technique. This work may have potentials in defending against backdoor attacks in general. I have a few concerns regarding the observation, the proposed method and experiments. It would be appreciated if the authors could elaborate in the following points.

1. The observation of the distinctive train losses between backdoored data and clean data is quite interesting. But the six attacks studied in Figure 2 all seem to be universal attacks, where the backdoor flips all other classes to one single target class. How about label-specific attacks where only one or two victim classes are compromised? Can we still have similar observations for the training loss as in Figure 2? If so, the proposed method would probably be much more general and useful. Also, 5 out of 6 studied attacks are static attacks. Although Dynamic attack stamps input-specific triggers, the type of triggers is still at the pixel-level, where such patterns might easily be learned by models. This could be the reason of the distinctive train losses. To further understand different types of backdoors, filter attacks would be a good candidate for the study as such attacks universally transform the input images and generate more natural backdoored data. It would be interesting to see whether such natural-looking backdoored data are still distinguishable from clean data based on their training losses.

2. From Figure 2, by just looking at the training loss for the first a few epochs, backdoored data are already clearly separated from clean data. Why does ABL only select a small portion (1%) as the backdoored data? Why cannot we just directly use a threshold, for instance 0.4, to remove all the backdoored data? In Figure 2, the 0.4 threshold at epoch 5 can clearly separate the two sets (backdoored and clean). Are there some poisoned images still having large training loss after a few epochs or clean images having very small loss? If this is the case, Figure 2 does not well reflect the true training loss. Could the authors clarify this?

3. The proposed local gradient ascent loss is very similar to the flooding loss. The results in Figure 9 in Appendix shows these two also have very similar performance. Since the evaluation is only conducted on 4 attacks on CIFAR-10, it is hard to tell whether the proposed loss is indeed better than the flooding loss. Also, the flooding level is set to 0.5 as stated in the appendix. Is this the best parameter for the flooding loss? Could there be a better parameter? I would suggest the authors to extend the evaluation on the comparison with the flooding loss as the proposed loss has a very similar characteristic and performance.

4. In the introduction, the authors motivate their work by claiming "a high attack success rate can still be achieved on CIFAR-10 by different attacks even though the poisoning rate is less than 1%". This is very intriguing as such a low poisoning rate might make defense harder. However, in the evaluation, this paper uses 10% poisoning rate as the default setting. Even when studying the performance of ABL against different poisoning rates, this paper only increases the poisoning rate. I am interested in whether ABL is still effective if a very low poisoning rate is chosen. I would like to see ABL's performance on the poisoning rates studied in Figure 1.

5. Have the authors considered adaptive attacks? For instance, the adversary can intentionally generate poisoned data having training loss similar to normal data. Will the proposed training procedure still be effective defending against backdoor attacks?

6. Could the authors explain why ABL has a higher clean accuracy on the clean-label poisoned training set (89.03%) than on the clean training set (88.41%)?
limitations_and_societal_impact: The authors have discussed the limitations of ABL against Blend and SIG attacks. But there are missing discussions on the applicability of ABL on label-specific attacks, filter-like attacks. Please see details in the main review.

The authors have addressed the potential negative societal impact.
review_title: null

Meta-review: I read the paper and considered the discussions carefully, especially that of the authors with 9SWb who brought up many thoughtful comments and criticisms. I believe that the concerns and criticisms have been largely addressed by the authors during the rebuttal, which I will discuss below.

I will now discuss the merits of the paper:
1. Well-established threat model. The threat scenario of classical (non-federated) training on a dataset poisoned such that inserting the backdoor during inference will result in the target class is a well-established scenario. It is also one of the simplest scenarios, which means the attack is more likely to be deployed in the real world.
2. Novelty. To mine and all the reviewers' agreement, the observations and methods presented in this paper are quite novel. Compared to other backdoor defense methods which often aim to learn the backdoor pattern from among a large search space, I believe the easy-learning detection and backdoor unlearning methods are quite elegant since they exploit what seems to be a fundamental weakness of backdoors (that to be strong they must be easy to learn) and can be incorporated directly into the training procedure (as opposed to a separate stage).
3. Results. The authors have done a comprehensive study on 3 datasets and 6 backdoor attacks, and included some well-established datasets/architectures for comparison with prior art (i.e. resnet18 on cifar, along with resnet34 upon the request of 9SWb). The authors covered 4 additional attacks upon the request of reviewer 9SWb who subsequently raised their score from 3 to 5.
4. The authors provided ample ablation studies to provide better insight into why their method works, which helps the readers obtain some learnings from this paper.

There were some concerns about different threat models (e.g. adversary having control over more things than just a fraction of the dataset) and different training settings (federated learning). While interesting, these are outside the scope of the paper and that the current scope is sufficiently interesting.

I therefore recommend acceptance.
Decision: Accept (Poster)