Venue: NeurIPS 2022 Workshop TSRML

Reviews:

final_score: rating: 5: Marginally below acceptance threshold
review_content: summary: This paper proposes a novel method for poisoning the generative replayers in continual learning in order to prompte catastrophic forgetting. The authors take advantage of input-aware backdoor attack to create their poisons. Variations of the MNIST and CIFAR-10 datasets are utilized to evaluate the proposed method.
strengths: - The paper is well-written and easy to follow.
- The idea of poisoning the replayer in continual learning is interesting and novel.
- The method is effective on the MNIST and CIFAR-10 datasets.

weaknesses: - Recently, powerful backdoor attacks and defenses have been introduced. I think the authors should cite a few relevant works (e.g. [1-4])

- The poisoning setting is unrealistic because first of all it is not a clean-label attack and, second, the victim classifier is not trained from scratch on the poisons [2,5].

- The poison budget is too high. Usually, a realistic poison budget is less than 1%. Even in "Input-Aware Dynamic Backdoor Attack" the poison budget is  $ρ_b = ρ_c = 0.1$. 

- Additional evaluations must be conducted on more complex datasets, such as ImageNet (or Tiny ImageNet) and CIFAR-100. MNIST is usually regarded as a toy dataset.

- The attack must be evaluated against more advanced defense methods such as [3,4].

- Please include replayed and poisoned images of the CIFAR-10 dataset. 

- Please also report the numerical test accuracy. This will facilitate comparison.





1- Saha, Aniruddha, Akshayvarun Subramanya, and Hamed Pirsiavash. "Hidden trigger backdoor attacks."

2- Souri, Hossein, Micah Goldblum, Liam Fowl, Rama Chellappa, and Tom Goldstein. "Sleeper agent: Scalable hidden trigger backdoors for neural networks trained from scratch."

3- Gao, Yansong, Change Xu, Derui Wang, Shiping Chen, Damith C. Ranasinghe, and Surya Nepal. "Strip: A defence against trojan attacks on deep neural networks." 

4- Borgnia, Eitan, Jonas Geiping, Valeriia Cherepanova, Liam Fowl, Arjun Gupta, Amin Ghiasi, Furong Huang, Micah Goldblum, and Tom Goldstein. "Dp-instahide: Provably defusing poisoning and backdoor attacks with differentially private data augmentations."

5- Geiping, Jonas, Liam Fowl, W. Ronny Huang, Wojciech Czaja, Gavin Taylor, Michael Moeller, and Tom Goldstein. "Witches' brew: Industrial scale data poisoning via gradient matching." 

overall_recommendation: The proposed method for poisoning generative replayers is novel and intriguing, but the attack setting is unrealistic and has not been evaluated on complex datasets.
review_title: A good poising attack on generative replayers in continual learning, but the attack is not very stealthy and evaluations are limited.

final_score: rating: 7: Good paper, accept
review_content: summary: This paper proposes a poisoning attack on generative replayer in continual learning. This attack promotes catastrophic forgetting, while the accuracy of the current task is not reduced. 
strengths: The first poisoning attack promotes catastrophic forgetting in continual learning. 

A trigger at test-time, which can circumvent defenses. 



weaknesses: As mentioned: the theoretical analysis of the trigger-discarding property is missing. 

Writing: 
 - Line 21 dot after the word "green" is incorrect. 
 - Figure 8, 10, 11, and 12 captions "." is missing. 
overall_recommendation: This paper proposes a novel poisoning attack that promotes catastrophic forgetting in continual learning. This field has not been studied very well and is therefore very valuable. The author(s) is/(are) encouraged to continue on this work.
review_title: CIAP - a novel poisoning attack on continuing learning.

Meta-review: The setting of the paper is novel, however the authors should try to improve the paper (add citations, report more metrics, add more datasets) based on reviewer's comments in the final version.
Decision: Accept