Paper: DRMGuard: Defending Deep Regression Models against Backdoor Attacks (ICLR-2024)

Reviewer comment (Novelty):
The novelty is limited. The trigger reverse method is similar to Neural Cleanse.

Author response:
Thanks for your comments. **Our method is significantly different from Neural Cleanse (NC)**, as we have discussed in the third paragraph of the introduction and the third paragraph of the related work. Below, we briefly discuss the differences again.

NC is designed for backdoored deep classification models (DCMs). First, different from DCMs for which the output space is discretized into a few class labels, the output space of DRMs is continuous. Thus, **it is infeasible to enumerate all the potential target vectors by NC to determine the infected target vector.** 

Second, different from DCMs that adopt $\arg\max$ to obtain the final output, DRMs do not need $\arg\max$. This makes the backdoor behavior of the backdoored DRMs distinct from that of the backdoored DCMs. Specifically, for a backdoored DCM, the backdoor behavior is often triggered by the activation values of several neurons in the feature space, whereas for a backdoored DRM, it is triggered by the activation values of all the neurons. **Our method introduces feature-space regularization term based on the unique feature-space characteristics of backdoored DRMs, while NC does not utilize the feature-space characteristics of backdoored DCM for reverse engineering.**

Our work is the first to detect backdoored DRMs in the image domain. We apply our method to a DRM to reverse engineer a potential trigger function, based on which we decide whether the model has been injected with a backdoor or not. A major challenge to reverse engineering the potential trigger function in the regression domain is that the output is defined in the continuous space. To address this challenge, we formulate reverse engineering as an optimization problem, which is based on both unique output-space and feature-space characteristics of the backdoored DRMs that are observed in this paper.

These differences make our work distinct from NC as well as all existing defenses that are designed for backdoored DCMs.

Reviewer comment (Novelty):
This paper should emphasize the importance of the backdoor in the regression models. Why is it important in the regression area?

Author response:
Thanks for your comments. We have discussed the motivation and importance of defending deep regression models against backdoor attacks in the first paragraph of the introduction section and the first paragraph of the evaluation section. Below, we briefly summarize the importance again.

**Deep regression models are used in many safety-critical applications**, such as driver attention monitoring, head pose estimation, and facial landmark detection, among many others. In our evaluation, we consider gaze estimation and head pose estimation as two representative applications. Specifically, gaze estimation tracks where the subject is looking, and plays a key role in a series of safety-critical applications, such as user authentication, driver distraction detection, and user attention monitoring in AR/VR. Similarly, head pose estimation has also been used in many daily applications, such as the driver assistance system, pedestrian attention monitoring, and aliveness detection in facial recognition. 

Unfortunately, DRMs are also vulnerable to backdoor attacks, and existing defenses designed for classification models are ineffective in detecting backdoored regression models. **This research problem has been largely overlooked by the community and we are the first to detect backdoored DRMs in the image domain.**

Reviewer comment (Experiments):
The number of categories of compared methods and backdoor attacks is clearly below ICLR acceptance threshold.

Author response:
Thanks for your comments. Since **there is no existing defense designed for DRMs in the image domain**, we selected the most relevant and representative defenses originally designed for DCMs and adapted them to the regression problem for comparative analysis. Specifically, we compare DRMGuard with four compared methods, i.e., Neural Cleanse, FeatureRE [1], Fine-pruning [2], and ANP [3], and show that DRMGuard can defend against six backdoor attacks.

[1] Zhenting Wang, et al. "Rethinking the Reverse-engineering of Trojan Triggers." NeurIPS 2022.

[2] Kang Liu, et al. "Fine-pruning: Defending against backdooring attacks on deep neural networks." International symposium on research in attacks, intrusions, and defenses. Cham: Springer International Publishing, 2018.

[3] Dongxian Wu, and Yisen Wang. "Adversarial neuron pruning purifies backdoored deep models." NeurIPS 2021.

Reviewer comment (Evaluation):
Time complexity. I notice that the authors did not report the running time of DRMGuard. I am not sure whether the variance calculation is time-consuming and could be scaled to higher dimensional conditions.

Author response:
**Response:** Thanks for your constructive comment. To investigate the processing time of our method, we perform experiments on a server installed with an NVIDIA A10 GPU. The input size for the generative model $G_{\theta}$ and the deep regression model (DRM) is $224\times 224 \times3$. The generative model consists of 14 convolutional layers, and the DRM is implemented by ResNet18. The batch size is 50. **Given a DRM, DRMGuard takes about 16 minutes to train $G_{\theta}$ for 2000 training steps.** We also measure the processing time in calculating the variance when the output is a 1000-dimensional vector. We repeat this process 1000 times, and **the average time used to calculate the variance over 50 1000-dimensional vectors is $5.91\times 10^{-2}$ ms**. We think the results show that DRMGuard can be scaled to higher dimensional conditions.

Reviewer comment (Experiments):
Extension to classification models (not very important). In my opinion, the key of this work is that we could use the variance of deep representations for trigger inverse optimization when the candidate labels are uncountable. I think this method could be extended to deep classification models. I suggest the authors conducting this method in classification experiments.

Author response:
**Response:** This is a very insightful suggestion, and we really appreciate it. We analyze the behaviors of backdoored deep classification models (DCMs) and design a prototype to show that the idea can also be extended to DCMs. By doing so, we can perform trigger inverse optimization for backdoored DCMs without enumerating all the labels.

Specifically, we train a backdoored DCM on Cifar10 by BadNet. We use  $\lbrace x_i \rbrace _{i=1}^N$ and $\lbrace\mathcal{A}(x_i)\rbrace _{i=1}^N$ to denote a set of benign images and poisoned images, respectively, where $\mathcal{A}(\cdot)$ is the function to apply the trigger to the image. We feed $\lbrace x_i \rbrace _{i=1}^N$ and $\lbrace\mathcal{A}(x_i)\rbrace _{i=1}^N$ to the backdoored DCM $C(\cdot)$, followed by a softmax function $S(\cdot)$ to obtain the probability vectors $\lbrace S(C(x_i))\rbrace _{i=1}^N$ and $\lbrace S(C(A(x_i)))\rbrace _{i=1}^N$. We observe that poisoned images lead to similar probability vectors for the backdoored DCM. Formally, we have the following observation: $\frac{1}{d}\sum _{j=1}^d \sigma^2(\lbrace S_j(C(A(x_i))) \rbrace _{i=1}^N) << \frac{1}{d}\sum _{j=1}^d \sigma^2(\lbrace S_j(C(x_i)) \rbrace _{i=1}^N)$, where $S_j(C(A(x_i)))$ and $S_j(C(x_i))$ is the $j$th component of $S(C(A(x_i)))$ and $S(C(x_i))$ respectively. Specifically, we observe that $\frac{1}{d}\sum _{j=1}^d \sigma^2(\lbrace S_j(C(A(x_i))) \rbrace _{i=1}^N) \approx 9\times 10^{-12}$  and $\frac{1}{d}\sum _{j=1}^d \sigma^2(\lbrace S_j(C(x_i)) \rbrace _{i=1}^N) \approx7\times 10^{-2}$.

Based on this observation, we propose the following optimization objective to generalize DRMGuard from the regression domain to the classification domain:
$$
\theta^{*}=\min_{\theta} \frac{\lambda_1}{d}\sum _{j=1}^d {\sigma^2(\lbrace S_j(C(G _{\theta}(x_i))) \rbrace _{i=1}^N)} + \frac{1}{N} \sum _{i=1}^N \| G _{\theta}(x_i)-x_i \|_1
$$

The first objective in the optimization problem aims to reverse engineer the poisoned images $\lbrace G _{\theta}(x_i)\rbrace _{i=1}^N$ that lead to similar probability vectors, regardless of their actual contents. The second optimization term ensures the transformed image  $G _{\theta}(x_i)$ is similar to the original image $x_i$. We remove the feature-space regularization term designed for DRMs, since backdoored DRMs and backdoored DCMs have different feature-space characteristics, as we discussed in our paper.

By solving the above optimization problem, $G_{\theta}$ can reverse engineer the poisoned images without enumerating each label. Also, the probability vectors can converge to the vector that leads to the target label. We train ten backdoored DCMs by BadNets and ten benign DCMs on Cifar10. We report backdoor identification results in the following table.
| TP | FP | FN | TN | Acc |
|:--:|:--:|:--:|:--:|:---:|
|  7 |  0 |  3 | 10 | 85% |

**We also show reversed poisoned images in Appendix A.9 of the updated supplementary.** We think the results show that our idea of minimizing the variance can be extended to the classification domain for reverse engineering.