Venue: ICLR 2022 Conference

Reviews:

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: This paper proposed a post-training BA detection approach for two-class classification problems, which can also be extended to scenarios with multiple attacks. The approach is derived based on a proposed notion of 'expected transferability', which works with a principled threshold that is irrespective of the domain nor the network or attack configurations. Empirical experiments verified the effectiveness of their approach.
main_review: This paper proposed a novel and working solution to a tough problem of backdoor attacks detection with binary or multiple labels. 

Strengths: 
* A novel notion of detection statistics called 'expected transferability' (ET) is proposed to help in detecting the target labels.
* A domain/attack/network agnostic threshold (1/2) is used which is principled and has theoretical implications.
* This approach is extendable to address more challenging scenarios, with a potential computation cost. 

Concerns:
* One argued contribution of the work is that it does not need 'the training set of the classifier to be inspected'. However, the approach still requires collecting an individual set of clean samples. I think it would be better to remove this claim.
* My major concern for this problem is that it seems very slow for the proposed algorithm to converge, as $ET$ is estimated by iterating each class $i$ and each of its samples $x_n^{(i)}$, until transferable set remain unchanged. Since the same procedure needs to be done for each class label, this issue might propagate with more class labels or clean samples. 

Minor comments:
Page 4, definition 3.3 , the expectation $E$ should be over $X, Y \sim P_i$ instead of  $X \sim P_i$?
summary_of_the_review: This paper proposed a novel and working solution to a tough problem of backdoor attacks detection with binary or multiple labels. The solution is well motivated and theoretically justified. Although the convergence rate might be a potential drawback of this approach, it still makes solid contributions to the community.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This paper proposes a new engineered reverse-engine based backdoor model
detection method. It has stronger assumptions including: 1) no access to
training data; 2) no access to benign reference model; 3) 2 or multi-class
models. To detect backdoors in such scenarios, it leverages a new statistical
metric known as the expected transferability, which can derive a fixed threshold
for backdoored models detection. Results on six datasets show that the proposed
method is effective for detecting backdoored models.
main_review: * Related work and novelty claims

It appears to me that the paper does not include discussions on recent work. Its
key selling point is that the proposed method works under three conditions: no
access to training data, reference model, and binary classifier. 

Many existing work also work under these constraints, such as ABS proposed by
Liu et al. in CCS 2019. More broadly, the IARPA TrojAI competition include not
only these constraints but more, such as constrained time to finish scanning one
model etc. It involves using a private dataset for training backdoor models
(a.k.a., detectors cannot leverage training data or train their reference
models), and also many tasks are binary classifiers, e.g., sentiment analysis.
Many submitted solutions are based on reverse-engineering. Also, as a
requirement of this competition, all solutions are open sourced.

Thus, to me, I think many claims in this paper should be altered. Moreover, I
think the paper should compare with some of these solutions. I would suggest
using the TrojAI dataset (also open sourced on NIST web page) as their benchmark
and compare with the leaderboard results (also open to access on NIST web page)
so that we can better understand the effectiveness of proposed solution.

* Design of ET and analysis

Despite the formal representation of the method and discussion (which is
appreciated), the design of this method is based on empirical observations on
the transferability of input samples. As such, there is no guarantee that the
proposed method will work and the threshold value 1/2 is general.In fact, as
shown in Figure 2, the proposed method also suffers some FP/FN, namely, some ETs
for benign classes are larger 0.5 and ET of backdoor target classes can be
smaller than 0.5.

To me, it is important to understand the applicable scenarios of the proposed
method. Namely, why these cases fail? I personally suspect it is because of the
transferability assumption. Is it? I would strongly suggest putting such
assumptions upfront so that we can understand the pros and cons of this method.

Another concern I have is that, many existing method does require tuning. But if
we can tune them to get better results, what is the advantage of the proposed
method? More importantly, are there backup plans when the method fails? Or can
we know that when it will work or not? Otherwise, I do not see the fundamental
benefits of using this method.

* Evaluation for multi-class, multi-attack scenario is not sufficient.

In Section 4.4, this paper provides the evaluation results on multi-class,
multi-attack scenarios. However, it only trains one model for each attack
settings and a single clean model for each dataset. There is no baseline method,
and I am not sure if the proposed method can truly generalize to other datasets
and models.

* Countermeasures and ethical statement

There is no countermeasures discussed and no ethical statement.

* CIFAR-10

Why the maximum ET for CIFAR-100 is much larger than that of other datasets? As
shown in Table. 2, an interesting phenomenon is that the maximum ET statistic
for CIFAE-100 is much larger than that of other datasets (i.e., CIFAR-10,
STL-10). It would be better if this paper could provide some discussion about
this phenomenon.

* Code and data

Will the code and data be released?

summary_of_the_review: My main concern is the it lacks discussion and comparison with works that works
under the same setting. Its analysis is not sufficient to understand the
applicable scenarios and its key benefit.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: The paper proposes a class-specific metric called Expected Transferability (ET) that can determine if a classifier is targeted by a backdoor attack (or not). The threat model assumes no access to the model's training data. By the virtue of being class-specific, the metric allows one to examine classifiers regardless of the number of output classes. The author reason theoretical what values of ET indicates the presence (or absence) of backdoor attacks. Then they empirically demonstrate how this value can be estimated and leveraged.

main_review: What I liked:
- The paper is well organized.
- The use of a class-specific metric to figure out if a classifier is prone to backdoors, given existing works focus on inter-class metrics, is a novel idea. (Although, I have not been proactive in following work on these lines.)
- I liked the idea of theoretically motivating the value for a metric and then empirically demonstrating its usefulness.
- The authors conduct several empirical studies.

Weakness/Clarification:
- Some parts of the theoretical analysis to determine the value of ET seem to be hand-wavy. First, while it is intuitively understandable why Property 3.1 might hold in the context of non-backdoor-attack scenarios, it is not clear whether it will always hold. Second, there is no good definition of how small $\epsilon$ has to be for the reasoning to hold.
- The authors argue that existing approaches cannot work for 2-class classification problems (as their sample complexity is $O(K^2)$, $O(K)$ where $K$ is the number of classes). While I get that a method that needs fewer examples is better, how many samples are too few or too many? The author's approach uses 20 samples (why 20?), is that constant or is it ~$O(K^3)/O(K^4)$ for the 2-class classification task. Is there a theoretical bound here? 
- In the empirical section, there are several cases where a backdoor-poisoned classifier is determined to be benign as per the ET metric (16/20 for $A_3$ in Table 1) or vice-versa (16/20 for $C_4$). Is there any particular reason as to why this happens? Also, is there a confidence value associated with the metric (for example, would values far away from 1/2 strongly indicate that it is benign or backdoor-poisoned, or does no such correlation hold)? Looking at Fig 2, Col 3, I notice a few oranges close to 1, which signals that there are no guarantees of any kind.
- How important is it to have a constant threshold (for ET-based methods) vs. estimate the boundary with a few samples (for other baselines)? Also, from Fig 2, it seems L2, when an optimal threshold value is used, might have better separability than E2. Yet, the ROC curve tells ET is much better than L2; what was the threshold used for L2 and how was is calculated?
- Why do the authors simply show the max ET values in Table 2? I would be more interested to see aggregate statistics (similar to Table 1). The latter would give a better idea of how the ET-based approach performs on average (as opposed to in the best case, that too if values furthest from 1/2 imply best).
summary_of_the_review: See above.

*[Update]* Having read the rebuttal (and looking at the additions to the revised paper), I would like to thank the authors for showcasing an honest effort to answer/address many of the comments/clarifications I initially had. I have updated my score to reflect the same.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: This paper proposes a novel post-training backdoor attack detection algorithm using reverse-engineering defense. The authors design a new statistic called expected transferability (ET) for detection. The advantage of ET is that the detection threshold is a constant (1/2) and is independent of classifier domain and attack algorithms, so the proposed detection algorithm can be applied to a wide range of backdoor attacks on different databases.
main_review: Pros:
1. The math derivation of ET and detection threshold is convincing. With a constant threshold, ET can be used to detect different backdoor attacks in different domains.
2. The proposed BA detection algorithm using empirical estimate of ET can be applied to classifiers with arbitrary number of classes including two-class cases. 
3. Authors conducted comprehensive experiments on six benchmark databases using different backdoor patterns to demonstrate the performance of proposed algorithm.

Cons:

The technical part of proposed BA detection algorithm is interesting and convincing for me. I am curious about the patience parameter to determine convergence. Does the choice of 4 hold true for ensembles A1 to A10, C1 to C6 in the experiment? Does the number of clean images used during detection affect the detection accuracy? Is the a minimum number of clean images required for an effective detection?
summary_of_the_review: I will vote for accepting this paper. The proposed ET statistic is novel and technically convincing. Extensive experiments have been conducted to evaluate proposed BA detection algorithm.
review_title: null

Meta-review: The paper presents a new method for detection of backdoor attacks under strong limitations such as the lack of access to training data and the reference benign model. Its main idea is to utilize a new expected transferability statistic that can be used for detection in broad range of application domains. The effectiveness of the proposed approach is demonstrated experimentally.
Decision: Accept (Poster)