Original: (1998), images for backdoor pattern reverse-engineering are synthesized by model inversion Fredrikson et al.
Revised: (1998), images for backdoor pattern reverse-engineering are synthesized by model inversion Fredrikson et al.

Original: However, images generated in such a way are not guaranteed to be visually typical to their designated classes, especially for complicated domains like ImageNet.
Revised: However, images generated in such a way are not guaranteed to be visually typical to their designated classes, especially for complicated domains like ImageNet.

Original: Thus we ask the following question:
Revised: Thus we ask the following question

Original: Can our ET framework be generalized to involve backdoor pattern reverse-engineering using synthesized images?

Original: [Figure]
Revised: [Figure]

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Revised: Can our ET framework be generalized to involve backdoor pattern reverse-engineering using synthesized images?

Original: Here, we consider the 20 two-class domains generated from MNIST due to this domain’s simplicity.
Revised: Here, we consider the 20 two-class domains generated from MNIST due to this domain’s simplicity.

Original: We apply RE-AP to the 20 classiﬁers being attacked in A 6 and the 20 clean classiﬁers in C 6 , respectively.
Revised: We apply RE-AP to the 20 classiﬁers being attacked in A 6 and the 20 clean classiﬁers in C 6 , respectively.

Original: The clean images used for detection are generated using a simpler version of the model inversion method used by Chen et al.
Revised: The clean images used for detection are generated using a simpler version of the model inversion method used by Chen et al.

Original: To synthesize an image for detection, we ﬁrst initialize an all-zero image added with some small random positive noise.
Revised: To synthesize an image for detection, we ﬁrst initialize an all-zero image added with some small random positive noise.

Original: Then, we maximize (over the image values) the posterior of the designated class of the image using the classiﬁer to be inspected, until the posterior is greater than 0.9.
Revised: Then, we maximize (over the image values) the posterior of the designated class of the image using the classiﬁer to be inspected, until the posterior is greater than 0.9.

Original: Examples for our synthesized images are shown in Fig.
Revised: Examples for our synthesized images are shown in Fig.

Original: Note that without the “auxiliary constraints” on images during their generation process (Chen et al.
Revised: Note that without the “auxiliary constraints” on images during their generation process (Chen et al.

Original: , the generated images are usually atypical to their designated classes.
Revised: , the generated images are usually atypical to their designated classes.

Original: In Fig. 13, we show the ET statistic for the 80 classes associated with the 40 binary classiﬁers ( classiﬁers being attacked and 20 clean classiﬁers).
Revised: In Fig. 13, we show the ET statistic for the 80 classes associated with the 40 binary classiﬁers ( classiﬁers being attacked and 20 clean classiﬁers).

Original: Among these 80 classes, there are 60 backdoor target classes and 20 non-target classes (please see the settings in Sec. 4.1).
Revised: Among these 80 classes, there are 60 backdoor target classes and 20 non-target classes (please see the settings in Sec. 4.1).

Original: Despite one backdoor target class evading our detection and one ET for a backdoor target class lying close to the threshold 12 , the separation of ET for backdoor target classes and non-target classes is even better than that in the bottom left ﬁgure in Fig.
Revised: Despite one backdoor target class evading our detection and one ET for a backdoor target class lying close to the threshold 12 , the separation of ET for backdoor target classes and non-target classes is even better than that in the bottom left ﬁgure in Fig.

Original: 2 (where backdoor pattern reverse-engineering is performed on typical
Revised: 2 (where backdoor pattern reverse-engineering is performed on typical

Original: MNIST images).
Revised: MNIST images).

Original: The possible reasons maybe:
Revised: The possible reasons maybe:

Original: • For non-attack cases, two independently synthesized samples predicted to the same class will likely be both atypical to their predicted class.
Revised: • For non-attack cases, two independently synthesized samples predicted to the same class will likely be both atypical to their predicted class.

Original: They will likely share fewer features associated with their predicted class than two independent typical samples from this class.
Revised: They will likely share fewer features associated with their predicted class than two independent typical samples from this class.

Original: They can also be located anywhere in the input space.
Revised: They can also be located anywhere in the input space.

Original: Thus, they will be more likely to be “mutually not transferable”.
Revised: Thus, they will be more likely to be “mutually not transferable”.

Original: • The synthesized samples are also far from the data manifold of classes other than the class they are predicted to.
Revised: • The synthesized samples are also far from the data manifold of classes other than the class they are predicted to.

Original: So the minimum perturbation size required to induce them to be misclassiﬁed will have a large norm – possibly larger than the norm of the backdoor pattern when there is an attack.
Revised: So the minimum perturbation size required to induce them to be misclassiﬁed will have a large norm – possibly larger than the norm of the backdoor pattern when there is an attack.

Original: If this is the case, the ET statistic will be exactly 1 based on Thm.
Revised: If this is the case, the ET statistic will be exactly 1 based on Thm.

Original: Future works along this research line include further investigation of the phenomenon we observed, and also improvement to the model inversion techniques, such that more complicated domains can be handled.
Revised: Future works along this research line include further investigation of the phenomenon we observed, and also improvement to the model inversion techniques, such that more complicated domains can be handled.

Original: J C OMPUTATIONAL C OMPLEXITY
Revised: J C OMPUTATIONAL C OMPLEXITY

Original: Here, we discuss the computational complexity of Alg.
Revised: Here, we discuss the computational complexity of Alg.

Original: More generally, we consider Alg.
Revised: More generally, we consider Alg.

Original: 1 for multi-class scenarios.
Revised: 1 for multi-class scenarios.

Original: Let K be the number of classes, N be the number of samples per class for detection, and T be the maximum number of forward/backward propagations for backdoor pattern reverse-engineering (i.e. “solving problem (3)” in line 8 of Alg. 1).
Revised: Let K be the number of classes, N be the number of samples per class for detection, and T be the maximum number of forward/backward propagations for backdoor pattern reverse-engineering (i.e. “solving problem (3)” in line 8 of Alg. 1).

Original: For each of the K classes, we compute the p ( i ) n quantity (line 12 of Alg. 1) for each of the KN samples used for detection.
Revised: For each of the K classes, we compute the p ( i ) n quantity (line 12 of Alg. 1) for each of the KN samples used for detection.

Original: To compute each p ( i ) n , the transferable set (line 9 of Alg. 1) is updated at most ( KN − 1) × τ times; and each updating involves at most T forward/backward propagations.
Revised: To compute each p ( i ) n , the transferable set (line 9 of Alg. 1) is updated at most ( KN − 1) × τ times; and each updating involves at most T forward/backward propagations.

Original: Thus, the theoretical upper bound of the number of forward/backward propagations for Alg. 1 is of the order
Revised: Thus, the theoretical upper bound of the number of forward/backward propagations for Alg. 1 is of the order

Original: O
Revised: O

Original: ( K 3 N 2 Tτ ) where τ is the patience parameter.
Revised: ( K 3 N 2 Tτ ) where τ is the patience parameter.

Original: However, the actual complexity of our method in practice is much lower than the theoretical bound.
Revised: However, the actual complexity of our method in practice is much lower than the theoretical bound.

Original: First, the purpose of using a sufﬁciently large number of samples for detection is to reduce the variance of the estimated ET (see Apdx.
Revised: First, the purpose of using a sufﬁciently large number of samples for detection is to reduce the variance of the estimated ET (see Apdx.

Original: G for more discussion and empirical results).
Revised: G for more discussion and empirical results).

Original: Thus, for sufﬁciently large K , we can set N = 1 and use only K samples for detection; or even use fewer samples randomly selected from these K samples.
Revised: Thus, for sufﬁciently large K , we can set N = 1 and use only K samples for detection; or even use fewer samples randomly selected from these K samples.

Original: Second, the actual number of iterations for updating the transferable set is much smaller than ( KN − 1) × τ in practice.
Revised: Second, the actual number of iterations for updating the transferable set is much smaller than ( KN − 1) × τ in practice.

Original: In Apdx.
Revised: In Apdx.

Original: H, we discussed the inﬂuence of the choice of τ on the estimated ET statistic and provided empirical results.
Revised: H, we discussed the inﬂuence of the choice of τ on the estimated ET statistic and provided empirical results.

Original: For attack cases, p ( i ) n reaches 1 (and thus terminates the updating of the transferable set) very quickly (even in one or two iterations) for most samples (see (a)(b)(e)(f) of Fig. 11).
Revised: For attack cases, p ( i ) n reaches 1 (and thus terminates the updating of the transferable set) very quickly (even in one or two iterations) for most samples (see (a)(b)(e)(f) of Fig. 11).

Original: For non-attack cases, convergence is also reached quickly for most samples (see (c)(d)(g)(h) of Fig. 11).
Revised: For non-attack cases, convergence is also reached quickly for most samples (see (c)(d)(g)(h) of Fig. 11).

Original: In these examples, 20 images are used for estimating the ET statistic with patience τ = 8 .
Revised: In these examples, 20 images are used for estimating the ET statistic with patience τ = 8 .

Original: Thus the theoretical maximum number of iterations for the transferable set updating for an image is (20 − 1) × 8 = 152 , which is several times larger than the actual maximum number of iterations ( < 25).
Revised: Thus the theoretical maximum number of iterations for the transferable set updating for an image is (20 − 1) × 8 = 152 , which is several times larger than the actual maximum number of iterations ( < 25).

Original: In Fig.
Revised: In Fig.

Original: 14, we show the curve of the execution time growing with the number of samples used for detection.
Revised: 14, we show the curve of the execution time growing with the number of samples used for detection.

Original: Speciﬁcally, we apply our detector to the clean binary classiﬁers in C 1 and record the average execution time, with the number of images for detection varying in [2, 5, 10, 15, 20].
Revised: Speciﬁcally, we apply our detector to the clean binary classiﬁers in C 1 and record the average execution time, with the number of images for detection varying in [2, 5, 10, 15, 20].

Original: Execution time is measured on a dual card RTX2080-Ti (11GB) GPU.
Revised: Execution time is measured on a dual card RTX2080-Ti (11GB) GPU.

Original: Comparing with Fig.where we show the effectiveness of our detector with only a few samples for detection, the actual time required for our detector to achieve good performance is only a few minutes.
Revised: Comparing with Fig. [Figure] where we show the effectiveness of our detector with only a few samples for detection, the actual time required for our detector to achieve good performance is only a few minutes.

Original: Moreover, for attack cases, the total number of iterations (for all samples used for detection) required for ET[Figure] estimation is generally much smaller than for clean cases (as shown in Fig. 11).
Revised: Moreover, for attack cases, the total number of iterations (for all samples used for detection) required for ET estimation is generally much smaller than for clean cases (as shown in Fig. 11).

Original: Thus, the actual execution time when there is an attack should be much smaller than the time shown in Fig.
Revised: Thus, the actual execution time when there is an attack should be much smaller than the time shown in Fig.

Original: K I NFLUENCE OF B ACKDOOR P ATTERN R EVERSE -E NGINEERING ON D ETECTION P ERFORMANCE
Revised: K I NFLUENCE OF B ACKDOOR P ATTERN R EVERSE -E NGINEERING ON D ETECTION P ERFORMANCE

Original: Like most existing REDs, our method cannot achieve 100% detection accuracy in practice.
Revised: Like most existing REDs, our method cannot achieve 100% detection accuracy in practice.

Original: As shown in both Tab. 1 and Fig. 2, out method, though achieving generally good detection accuracy, suffers from a few false negatives and false positives.
Revised: As shown in both Tab. 1 and Fig. 2, out method, though achieving generally good detection accuracy, suffers from a few false negatives and false positives.

Original: One main reason for the false negatives is that the existing BP reverse-engineering techniques used in our detection framework cannot not always recover the key features of the true BP used by the attacker 6 .
Revised: One main reason for the false negatives is that the existing BP reverse-engineering techniques used in our detection framework cannot not always recover the key features of the true BP used by the attacker 6 .

Original: In such cases, the “non-transfer probability” for a backdoor target class will be large, as if there were no attacks; and ET will likely be less than 12 .
Revised: In such cases, the “non-transfer probability” for a backdoor target class will be large, as if there were no attacks; and ET will likely be less than 12 .

Original: In Fig.
Revised: In Fig.

Original: 15a, we show an example of such failure of BP reverse-engineering.
Revised: 15a, we show an example of such failure of BP reverse-engineering.

Original: We consider a classiﬁer being attacked associated with a two-class domain generated from CIFAR-100 that evaded our detection (from a blue bar less thanin the second ﬁgure of the left row in Fig. 2).
Revised: We consider a classiﬁer being attacked associated with a two-class domain generated from CIFAR-100 that evaded our detection (from a blue bar less thanin the second ﬁgure of the left row in Fig. 2).

Original: For a random sample, we observed that the estimated
Revised: For a random sample, we observed that the estimated

Original: BP is visually uncorrelated with the true BP used by the attacker.
Revised: BP is visually uncorrelated with the true BP used by the attacker.

Original: For comparison, we also show an example for a successful BP reverse-engineering in Fig.
Revised: For comparison, we also show an example for a successful BP reverse-engineering in Fig.

Original: 15b, where the estimated pattern contains some key features of the true BP used by the attacker; the ET statistic for this class is larger thanand the attack is successfully detected.
Revised: 15b, where the estimated pattern contains some key features of the true BP used by the attacker; the ET statistic for this class is larger thanand the attack is successfully detected.

Original: We notice that most false positives happen when applying RE-PR (i.e. the reverse-engineering method in Wang et al.
Revised: We notice that most false positives happen when applying RE-PR (i.e. the reverse-engineering method in Wang et al.

Original: (2019) for patch replacement BPs) to clean classiﬁers.
Revised: (2019) for patch replacement BPs) to clean classiﬁers.

Original: As introduced in Sec.
Revised: As introduced in Sec.

Original: D.6, RE-PR searches for a small image patch that induces high group misclassiﬁcation to a putative target class.
Revised: D.6, RE-PR searches for a small image patch that induces high group misclassiﬁcation to a putative target class.

Original: But it is possible for some domains and some classes, there are common key features associated with the class that is easy to be reverse-engineered on a small spatial support/mask.
Revised: But it is possible for some domains and some classes, there are common key features associated with the class that is easy to be reverse-engineered on a small spatial support/mask.

Original: Such features, if reverse-engineered on one sample, will likely also induce another sample to be misclassiﬁed.
Revised: Such features, if reverse-engineered on one sample, will likely also induce another sample to be misclassiﬁed.

Original: Although this hypothesis requires further validation in the future, we have shown empirically that the false detections have a low frequency, given that our experiments are performed on a large number of different two-class domains generated from six benchmark datasets. 
Revised: Although this hypothesis requires further validation in the future, we have shown empirically that the false detections have a low frequency, given that our experiments are performed on a large number of different two-class domains generated from six benchmark datasets. [Table] [Figure]

Original: L A DDITIONAL E XPERIMENTS FOR M ULTI -C LASS S CENARIOS WITH A RBITRARY N UMBER OF A TTACKS
Revised: L A DDITIONAL E XPERIMENTS FOR M ULTI -C LASS S CENARIOS WITH A RBITRARY N UMBER OF A TTACKS

Original: In Sec. 4.4, we showed the performance of our ET framework against BAs for multi-class scenarios with arbitrary number of attacks.
Revised: In Sec. 4.4, we showed the performance of our ET framework against BAs for multi-class scenarios with arbitrary number of attacks.

Original: We considered the original domains of CIFAR-10, CIFAR-100, 1 attack 2 attacks clean ET (RE-AP) 10/ 10/ 10/ RED-AP (original) 10/ 0/ 9/
Revised: We considered the original domains of CIFAR-10, CIFAR-100, and STL-10.

Original: RED-AP (MAD) 6/ 2/ 7/ [Figure] and STL-10. For each domain, we showed that the maximum ET statistic over all classes is larger than 12 if there is an attack, and less than 12 if there is no attack.
Revised: For each domain, we showed that the maximum ET statistic over all classes is larger than 12 if there is an attack, and less than 12 if there is no attack.

Original: Here, we further investigate the capability of our ET framework on more multi-class domains.
Revised: Here, we further investigate the capability of our ET framework on more multi-class domains.

Original: In particular, we generate 10 ﬁve-class domains from CIFAR-10, with the ﬁve classes for each domain randomly selected from the original ten classes of CIFAR-10.
Revised: In particular, we generate 10 ﬁve-class domains from CIFAR-10, with the ﬁve classes for each domain randomly selected from the original ten classes of CIFAR-10.

Original: For each domain, we create an attack instance with one attack, an attack instance with two attacks, and a clean instance.
Revised: For each domain, we create an attack instance with one attack, an attack instance with two attacks, and a clean instance.

Original: The protocols for attack creation, classiﬁer training, and defense conﬁgurations are the same as in Sec. 4.4.
Revised: The protocols for attack creation, classiﬁer training, and defense conﬁgurations are the same as in Sec. 4.4.

Original: Moreover, we compare our ET (with RE-AP) with the existing RED proposed by Xiang et al.
Revised: Moreover, we compare our ET (with RE-AP) with the existing RED proposed by Xiang et al.

Original: (2020) (with its original protocol including a conﬁdence threshold 0.05 (indicating a conﬁdence 0.95)), as well as the same RED proposed by Xiang et al.
Revised: (2020) (with its original protocol including a conﬁdence threshold 0.05 (indicating a conﬁdence 0.95)), as well as the same RED proposed by Xiang et al.

Original: but with the anomaly detection method changed to the one based on median absolute deviation (MAD) proposed by Wang et al.
Revised: but with the anomaly detection method changed to the one based on median absolute deviation (MAD) proposed by Wang et al.

Original: (2019) (with the same threshold 2 (also indicating a conﬁdence 0.95) used by Wang et al.
Revised: (2019) (with the same threshold 2 (also indicating a conﬁdence 0.95) used by Wang et al.

Original: For simplicity, we name these two methods as “RED-AP (original)” and “RED-AP (MAD)” respectively.
Revised: For simplicity, we name these two methods as “RED-AP (original)” and “RED-AP (MAD)” respectively.

Original: All three methods use 3 clean images per class for detection.
Revised: All three methods use 3 clean images per class for detection.

Original: In Tab. 10, we show the detection accuracy of our ET compared with RED-AP (original) and REDAP (MAD).
Revised: In Tab. 10, we show the detection accuracy of our ET compared with RED-AP (original) and REDAP (MAD).

Original: Our ET detects all attacks with no false detections.
Revised: Our ET detects all attacks with no false detections.

Original: For each of the three ensembles with ten classiﬁers, the histogram of the maximum ET statistic over all the ﬁve classes is shown in
Revised: For each of the three ensembles with ten classiﬁers, the histogram of the maximum ET statistic over all the ﬁve classes is shown in

Original: Fig.
Revised: Fig.

Original: On the other hand, RED-AP (original) achieved good accuracy to detect classiﬁers withattack, with very low false detection rate, but it fails to detect any classiﬁers with 2 attacks.
Revised: On the other hand, RED-AP (original) achieved good accuracy to detect classiﬁers withattack, with very low false detection rate, but it fails to detect any classiﬁers with 2 attacks.

Original: This is because the anomaly detection setting of RED-AP (original) relies on the 1-attack assumption to estimate a null distribution for non-backdoor class pairs.
Revised: This is because the anomaly detection setting of RED-AP (original) relies on the 1-attack assumption to estimate a null distribution for non-backdoor class pairs.

Original: For RED-AP (MAD), there is also a clear gap in performance compared with our ET.
Revised: For RED-AP (MAD), there is also a clear gap in performance compared with our ET.

Original: This is because the median of the ﬁve statistics is largely biased by statistics associated with backdoor target classes.
Revised: This is because the median of the ﬁve statistics is largely biased by statistics associated with backdoor target classes.