Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: This paper approached the data poisoning problem from a feature perspective. It argued that the backdoor is no more than the strongest feature in the dataset and thus can be detected based on its high sensitivity. The authors proposed using datamodels to approximate the sensitivity of features and tested the methods on dirty-label and clean-label attacks with either fixed pattern or m-way triggers. Compared to Inverse Self-Paced Learning (ISPL), Spectral Signatures and SPECTRE on CIFAR-10 and ResNet-9, the proposed algorithm successfully defended seven out of eight attacks. Still, it didn't outperform these existing methods on poisoned accuracy. 
strength_and_weaknesses: __Strength__:

* This paper is well-structured with all implementation details provided. 
* Intuitition of theoritical claims is always given to help the audience understand the concepts. 
* The idea of using datamodel for defending backdoor attacks is novel, and the proposed algorithm is very simple after the problem is provably reduced to a maximum-sum sub-matrix problem. 

__Weaknesses__:

* Basic notions should be better defined and explained in the data poisoning setting. 
* Connections between definitions and assumptions in Section 3 are weak and hard to follow. 
* As a result of the bullet point above, the theoritical intuitions cannot convincably motivate the algorithm. 
* The experiment section lacks more up-to-date attacks and defenses and more larger datasets and models. The results are also not significantly better than compared methods.
clarity,_quality,_novelty_and_reproducibility: __Clarity:__

($+$) This paper has a well-organized structure and is concisely written.

($-$) The main formulation seems to be borrowed from (Ilyas et al., 2022) which is not properly referred to. For example, definition 2 (margin function) is the same as Eq. 7 (correct-class margin) in (Ilyas et al., 2022). As this metric is heuristically chosen by (Ilyas et al., 2022), I believe it should be cited. 

($-$) Definitions are mostly not written formally in math but only with a line of words. It is helpful to have words to help the audience understand the concepts but they should only follow the mathematical formulations, not by themselves alone. (Ilyas et al., 2022) also used similar text definitions, but always accompanied with mathematical notations. 

($-$) Some definitions/notations are confusing to me:
  - Informal Assumption 1: "We assume that adding a backdoor example ($x:\phi_p(x) = 1$) to the training set changes predictions on the backdoor examples  $\phi_p(S)$ more than adding an example $x : \phi_(x) = 1$ changes predictions on  $\phi_(S)$."
    - $\phi_p(S)$ is not formally defined. Is it possible that $\phi_(x)=\phi_p(x) \ \forall x\in\mathcal{X}$? In this case, does this statement still hold? 
  - Definition 3: 
    - I don't know if "$S'\sim\alpha S$" is a valid expression for "sampling a random α-fraction subset of S." $S'\sim S, |S'|=\alpha |S|$ could be a better one. 
    - A definition of $k$ should be given near Definition 3. 
  - Assumption 1: 
    - I didn't get what $p$ is given the definition $p:=|\phi_p(S)|=|\phi(S)|$ . Why is $|\phi_p(S)|=|\phi(S)|$? Does it mean that a feature $\phi$ is considered in this assumption if and only if $|\phi_p(S)|=|\phi(S)|$? 
    - $S_p$ is defined but not used in this assumption. 
    - $\delta$ is not defined. 

($\cdot$) I suggest adding a pseudocode for the algorithm. 

__Quality__:

($\cdot$) I have a few questions regarding how the formal assumption, which is claimed as one of the contributions of this paper, is derived. 
  * In the context of data poisoning, I don't know how I should understand the margin. If I understood the definition of margin correctly, it should be that the higher the margin, the better job the model is doing, right? Then why in Figure 2 do we see the average margin of the poisoned image increase when there are more poisoned samples? Can the authors help provide some intuitions about this increase? 
  * Also for Figure 2, why should we look at the change in the margin of clean images when the number of poison images increases? 
  * Why does Assumption 1 _sum_ sensitivity over the supports? I didn't follow how we arrive at Assumption 1 from Definition 3. It would be helpful if the authors can explain more. 

($-$) Backdoor attacks and defenses considered in the experiments are limited and outdated. The model and the dataset options are also very limited: the only model used in the experiments is ResNet-9 and the only dataset tested is CIFAR-10. Experiments on more and larger models and datasets are needed. 

__Novelty__:

($+$) Defending against backdoor attacks seems to be a novel usage of datamodels. 

($-$) The empirical results are not significant compared to existing defenses like ISPL. The theoretical contribution is also marginally novel.

__Reproducibility__:

($+$) Very detailed information is provided for reproducing the results (though the code is not attached). 
summary_of_the_review: In my opinion, this is a work in progress rather than a complete work that meets the conference standard. The theoretical part is lacking in both novelty and clarity, and there should also be more experiments on the SOTA attacks and defenses with larger and more models and datasets to empirically validate the proposed algorithm. Therefore, I recommend rejection and encourage the authors to re-submit in the future after careful revision.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: The authors introduce a defense to backdoor attacks based on detecting "strong" features in training data, and removing backdoored training data by filtering along these lines.
strength_and_weaknesses: ### Strengths:
1. I like how the authors began their work in a principled way by discussing difficulties with categorizing backdoor attacks.
2. The approach seems fairly intuitive and well motivated.

### Weaknesses:
1. I don't think the authors can claim contribution for presenting a "new perspective" on backdoor attacks as it is common knowledge that backdoor triggers can be discriminative features of the data. Also, I think this analysis applies more to the "clean label" setting - a point I would make clearer in the manuscript.
2. The work only deals with rudimentary backdoor attacks. It would have been nice to test on modern backdoor attacks - some of which even disguise the "trigger" e.g. [1,2].
3.  The empirical results seem lukewarm - compared to some older defenses, the proposed approach wins out, but often it is not the most successful defense. 
4. There is a lot of clunkiness with the definitions/assumptions. For example:
    * You might be missing an $S'$ in the Definition 2 - $f(x;S')$ vs. $f(x;S)$
    * I don't like the presentation for Definition 1. If you're calling $\phi$ the feature $\phi:S \rightarrow \{0, 1\}$, then define $\textit{another}$ symbol to refer to the preimage of $\{1\}$. When you reuse $\phi$ to define $\phi(S) = \{ x | \overline{\phi(x)} = 1 \}$ Also, what is $\overline{\phi(x)\}$? Also what is feature $f$? Is this a typo?
    * Why have $\mathcal{S} = (X \times Y)^n$? This seems unnecessary. Just define $\mathcal{S}$ as $X \times Y$ for some input space $X$ and label space $Y$.

[1] Saha, Aniruddha, Akshayvarun Subramanya, and Hamed Pirsiavash. "Hidden trigger backdoor attacks." Proceedings of the AAAI conference on artificial intelligence. Vol. 34. No. 07. 2020.

[2] Souri, Hossein, et al. "Sleeper agent: Scalable hidden trigger backdoors for neural networks trained from scratch." arXiv preprint arXiv:2106.08970 (2021).
clarity,_quality,_novelty_and_reproducibility: The work was fairly clear, but seems to contain several typos. The work is novel as far as I know.
summary_of_the_review: The lukewarm results and less than ideal presentation makes me lean toward rejection.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: The paper gives a new perspective on backdoor attacks and how to identify and defend them. In back-door attacks, an adversary first performs a poisoning attack, usually by adding a specific pattern to examples that are then labeled carefully. Then, at the test time, the adversary can control the predicted label by minimally modifying the example (e.g., by again adding the pattern).

The paper focuses on the most common way of doing the backdoors (i.e., by adding patterns) and makes the following contributions:

1. Aiming to identify what backdoor attacks are: The paper starts by making the case that “feature adding” attacks are, in general, hard to identify as those features look like other features. Then the paper goes ahead and (somehow contradicts this thesis and) says that such adversarial features have special forms of features that could be identified as follows. The paper defines a notion of “senstiivty” for test examples with respect to a specific feature (where a feature is generally thought of as a boolean function over the training set) for a given data set and an integer k. Intuitively, this captures how much the answer to that test flips, when we go from “image” set of size k to “image” set of size k+1, where “image set” is the set of examples from the training set that have the feature.

2. Preventing the backdoor attacks: once we accept the thesis of item (1), that backdoor attacks will be identifiable by finding the “feature” with maximum “strength”. Then, the paper shows a heuristic that aims to identify such feature by (heavily) relying on the recent work of “data models” (from 2022) that allows one to predict the change of labels through a vector in R^|S| where S is the training set. This vector, once used in an inner product with the characteristic function of a subset, will give the result of training on that subset. Using this strong tool (which is conjectured to work well for neural nets) the paper defines an optimization task that heuristically solves to identify a feature/subset that is adversarial.

3. Finally, the paper performs some experiments to see how well their defense against two *previously known* attacks.

strength_and_weaknesses: The strength is to give a new perspective on backdoor attacks. The perspective is intuitive and might lead to useful defenses.

Weakness: There is basically no real theoretical result in this paper. Everything is based on (numerous) assumptions, which expectedly confirms that the defense works, but there are so many assumptions that are thrown without much deep justifications.

One can argue that the theoretical part would not matter as long as this defense truly works in practice. I agree with that sentiment, but here comes the second (main) weakness:

The paper only studies previous attacks, while clearly an adversary might have this defense in mind when attacking. This has happened over and over in adversarial learning tha defenses carefully study previous attacks, while attacks are defined adaptively. At the bare minimum, the paper should try to experiment with attackers who try to, e.g., keep the margin and sensitivity bounds of their adversarially added examples below a certain threshold to fool this defense. That is why I cannot support this paper for the top venue of ICLR if it does not follow this (by now well known) standard to judge defenses.

Another weakness is that: the paper really focuses on a specific type of backdoor attacks. In general, a backdoor attack can do *any* form of poisoning as long as (1) it has few poison points (2) it can control the prediction by a small amount of perturbation. This paper, on the other hand, focuses only on the “most common” way of doing the backdoor, which is by adding a feature to a bunch of examples.

clarity,_quality,_novelty_and_reproducibility: The novelty is in the new perspective on (the most common form of) backdoor attacks as well as using the “data model” tool from the recent work of Ilyas et al. for their heuristic defense, which looks like a nice application of this tool.

The theoretical section is too dense and many parts of it seem not fully baked to me. 

Some comments about the writing that might be helpful to the authors:
You keep mentioning the “indistinguishability” claim, but it is never formalized.
At the minimum, you could discuss the connection to “clean label” attacks. See here for the literature:
https://arxiv.org/pdf/2012.10544.pdf

You mention the “robust statistics” as the default source of literature on poisoning attacks. This is not true. You are working in the supervised setting, and there is already a large body of work that is much more relevant to your setting. For example, look up the words ``malicious noise” or “nasty noise” from the classical works Valiant and others.

I do not understand the claims about why the claim about the “slopes” in Figure 2 holds. I think the slopes are always there and exist, but it can be the case that in Def 3, the event that you condition on has zero probability. So something does not match.

summary_of_the_review: This paper gives a new interesting perspective on backdoor attacks. It then uses this perspective to come up with a new heuristic defense. The theory part is a bit sketchy and the experimental part does not aim for studying attacks that have this defense in mind.
review_title: null

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: This paper proposes a new perspective of looking at backdoor attacks as features
rather than outliers. It states that backdoor attacks are impossible to detect
from natural features without structural information on training data
distribution. It also assumes that a backdoor attack is the strongest feature
and designs a defense framework based on this assumption.
strength_and_weaknesses: It is an interesting perspective to look at backdoors.
The paper presentation is good.

Its definition seems to be oversimplified.
The assumption it makes seems to be wrong.
Evaluation is weak.
clarity,_quality,_novelty_and_reproducibility: 
The perspective of defining backdoor attacks as the strongest features is
interesting.

Its definitions of features and feature supports seem to be oversimplified. In
section 3, this paper defines a feature as a map from an input to a binary
value of either 0 or 1 and defines its feature support as whether the feature
is present. These definitions overly simplify the problem and discussion. In
practice, the presence of a feature is hard to measure or define -- moreover,
a feature is also hard to define or measure. Using a single scalar value to
denote its existence or not is overly simple. Moreover, the feature itself is
an ambiguous concept. In many cases, humans cannot even determine if a feature
exists or not without enough context. For example, it mentions feature support
of a dog's feature can be whether it contains "dog fur" -- my question is, do
you consider a short white line in an image as dog fur or not? These
properties of its defined feature support also affect the decision of whether
a feature is present. This paper directly defines a feature as a
representation of having something or not, which to me, tries to convert the
analysis of a DNN on image data to the analysis of tabular data. I am not
convinced that an image in DNN can be analyzed this way. If so, some hard
problems in DNN will become easy.

Another fundamental issue of this work is that it assumes that backdoor
attacks are the strongest features without considering the learning process.
This paper assumes that backdoors as stronger features than natural features
originally occur in input samples. This is wrong. I want to point out that
many datasets do have strong natural features. In backdoor research, this is
known as the natural backdoor. The research goes back to ABS by Liu et al. CCS
2019 where the authors show natural backdoors for deers. Further, existing
work Ex-ray by Liu et al. CVPR 2022 have shown that natural backdoors, whose
feature are not as strong as input features, can still manipulate models to
perform successful backdoor attacks. Another line of work is trying to develop
a robust learning method, e.g., ABL by Li et al. NeurIPS 2021, in the face of
poisoning data. They also prove that the learning process can pick up more
desirable features in training to produce more robust models. This also shows
that the assumption of this work does not hold in practice.

One reason I think its experiment result is good is that it only considers
patch triggers which are easier to present backdoor features at the input. It
is important to consider different types of backdoor triggers like Blended
attack [1], WaNet [2], and Invisible attack [3], whose features at input space
are not obvious.

[1] Chen et al. "Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoning." arXiv 2017
[2] Nguyen et al. "WaNet - Imperceptible Warping-based Backdoor Attack" ICLR 2021
[3] Li et al. "Invisible Backdoor Attack with Sample-Specific Triggers" ICCV 2021

summary_of_the_review: This paper defines features in a limited way and assumes backdoor attacks as
the strongest features without model learnability consideration. These seem to
be fundamental limitations.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: Under the assumption that a backdoor attack corresponds to the strongest feature in the training data, this paper develops a new framework for detecting backdoor attacks. The proposed algorithm is evaluated both theoretically and experimentally.

Strengths:
a new perspective on backdoor attacks
The paper is well-motivated

Weaknesses:
Lack experiments on SOTA attacks
The analysis relies on too strong assumptions

Even after the author response, the concerns remain unresolved. Therefore, I have to recommend rejection.
Decision: Reject