Revised: estimation is generally much smaller than for clean cases (as shown in Fig. 11).

Revised: Thus, the actual execution time when there is an attack should be much smaller than the time shown in Fig.

Revised: K I NFLUENCE OF B ACKDOOR P ATTERN R EVERSE -E NGINEERING ON D ETECTION P ERFORMANCE

Revised: Like most existing REDs, our method cannot achieve 100% detection accuracy in practice.

Revised: As shown in both Tab. 1 and Fig. 2, out method, though achieving generally good detection accuracy, suffers from a few false negatives and false positives.

Revised: One main reason for the false negatives is that the existing BP reverse-engineering techniques used in our detection framework cannot not always recover the key features of the true BP used by the attacker 6 .

Revised: In such cases, the “non-transfer probability” for a backdoor target class will be large, as if there were no attacks; and ET will likely be less than 12 .

Revised: In Fig.

Revised: 15a, we show an example of such failure of BP reverse-engineering.

Revised: We consider a classiﬁer being attacked associated with a two-class domain generated from CIFAR-100 that evaded our detection (from a blue bar less thanin the second ﬁgure of the left row in Fig. 2).

Revised: For a random sample, we observed that the estimated

Revised: BP is visually uncorrelated with the true BP used by the attacker.

Revised: For comparison, we also show an example for a successful BP reverse-engineering in Fig.

Revised: 15b, where the estimated pattern contains some key features of the true BP used by the attacker; the ET statistic for this class is larger thanand the attack is successfully detected.

Revised: We notice that most false positives happen when applying RE-PR (i.e. the reverse-engineering method in Wang et al.

Revised: (2019) for patch replacement BPs) to clean classiﬁers.

Revised: As introduced in Sec.

Revised: D.6, RE-PR searches for a small image patch that induces high group misclassiﬁcation to a putative target class.

Revised: But it is possible for some domains and some classes, there are common key features associated with the class that is easy to be reverse-engineered on a small spatial support/mask.

Revised: Such features, if reverse-engineered on one sample, will likely also induce another sample to be misclassiﬁed.

Revised: Although this hypothesis requires further validation in the future, we have shown empirically that the false detections have a low frequency, given that our experiments are performed on a large number of different two-class domains generated from six benchmark datasets.

Revised: L A DDITIONAL E XPERIMENTS FOR M ULTI -C LASS S CENARIOS WITH A RBITRARY N UMBER OF A TTACKS

Revised: In Sec. 4.4, we showed the performance of our ET framework against BAs for multi-class scenarios with arbitrary number of attacks.

Revised: We considered the original domains of CIFAR-10, CIFAR-100,

Revised: 1 attack 2 attacks clean

Revised: ET (RE-AP) 10/ 10/ 10/

Revised: RED-AP (original) 10/ 0/ 9/

Revised: RED-AP (MAD) 6/ 2/ 7/

Revised: [Figure]

Revised: and STL-10.

Revised: For each domain, we showed that the maximum ET statistic over all classes is larger than 12 if there is an attack, and less than 12 if there is no attack.

Revised: Here, we further investigate the capability of our ET framework on more multi-class domains.

Revised: In particular, we generate 10 ﬁve-class domains from CIFAR-10, with the ﬁve classes for each domain randomly selected from the original ten classes of CIFAR-10.

Revised: For each domain, we create an attack instance with one attack, an attack instance with two attacks, and a clean instance.

Revised: The protocols for attack creation, classiﬁer training, and defense conﬁgurations are the same as in Sec. 4.4.

Revised: Moreover, we compare our ET (with RE-AP) with the existing RED proposed by Xiang et al.

Revised: (2020) (with its original protocol including a conﬁdence threshold 0.05 (indicating a conﬁdence 0.95)), as well as the same RED proposed by Xiang et al.

Revised: but with the anomaly detection method changed to the one based on median absolute deviation (MAD) proposed by Wang et al.

Revised: (2019) (with the same threshold 2 (also indicating a conﬁdence 0.95) used by Wang et al.

Revised: For simplicity, we name these two methods as “RED-AP (original)” and “RED-AP (MAD)” respectively.

Revised: All three methods use 3 clean images per class for detection.

Revised: In Tab. 10, we show the detection accuracy of our ET compared with RED-AP (original) and REDAP (MAD).

Revised: Our ET detects all attacks with no false detections.

Revised: For each of the three ensembles with ten classiﬁers, the histogram of the maximum ET statistic over all the ﬁve classes is shown in

Revised: Fig.

Revised: On the other hand, RED-AP (original) achieved good accuracy to detect classiﬁers withattack, with very low false detection rate, but it fails to detect any classiﬁers with 2 attacks.

Revised: This is because the anomaly detection setting of RED-AP (original) relies on the 1-attack assumption to estimate a null distribution for non-backdoor class pairs.

Revised: For RED-AP (MAD), there is also a clear gap in performance compared with our ET.

Revised: This is because the median of the ﬁve statistics is largely biased by statistics associated with backdoor target classes.