Paper: EigenGuard: Backdoor Defense in Eigenspace (ICLR-2024)

Reviewer comment (Novelty):
The findings are not novel. SM Moosavi-Dezfooli et al.[Ref-1] present that universal perturbations exhibit a concentrated behavior within the spectral space. Backdoor triggers, as a special type of universal perturbation, exhibit a similar property, which appears to lack significance.

Author response:
**1. About the difference between our paper and Universal Perturbations:**

Although trigger features can be regarded as a kind of universal perturbation that lies in eigenspace corresponding to the top singular value, the former work doesn’t show that trigger features are low rank and can only lie in top eigenspace. Furthermore, the relationship between the feature dimensions and backdoor accuracy during training as shown in Figure 1 is only unevaluatedin our paper. Therefore, the new findings of our paper are that the trigger features can only located in the top low-rank subspaces. Such a phenomenon may be caused by the low poison rate of the trigger samples. 

Furthermore, we also designed our new method by damping top singular values and using features corresponding to more eigenspace for prediction. Therefore, models with our module can naturally defend against many backdoor attacks.

Reviewer comment (Evaluation):
After progressively eliminating the dimension of the feature space by setting the top singular values of features SVD decomposition to 0, why is it necessary to lift the original small singular values and generate new features? Doesn't this manipulation lead to a degradation of the prediction accuracy for clean or natural images?

Author response:
**4. Why not just let top singular values to zero?**

Just removing the top eigenspace will make the natural accuracy drop because features corresponding to top singular values are also important for natural classification, as shown in the following experiments:

| Methods | ACC. | ASR. |
| -------- | ------- | ------- |
| Remove top 10 Singulars  |  $85.4\%$   |  $4.2\%$   |
| Ours | $92.6\%$     |  $5.3\%$   |


Our method tries to avoid trigger features dominating the prediction. According to our theory, if the trigger features not dominating, which means the corresponding singular value is not large, the rest natural features will help models predict other classes instead of natural classes.

Reviewer comment (Experiments):
What is the rationale for scaling $\sigma_k$ by 0.001 in Algorithm 1, and how is this scale factor determined?

Author response:
**5. About the 0.001 scaling factor.**

This parameter will not influence much, we don’t set it to zero because we think the gradient corresponds to the latter eigenspace may also help. But in fact, such a hyper-parameter will not influence much.

Reviewer comment (Theory):
The paper lacks theoretical proof of the proposed method.

Author response:
**6. About the theoretical proof.**

Our theory illustrates that damping the dominating trigger features may make the models use natural features to make correct predictions. It is also consistent with our method, we make the top singular values smaller to avoid the triggers dominating classifications.

Reviewer comment (Experiments):
The evaluation lacks comprehensiveness and does not include comparisons with some related works for defenses:  
- Zhenting Wang, Kai Mei, Hailun Ding, Juan Zhai, and Shiqing Ma. Rethinking the reverse-engineering of trojan triggers. In Advances in Neural Information Processing Systems, 2022.  
- Jonathan Hayase and Weihao Kong. Spectre: Defending against backdoor attacks using robust covariance estimation. In International Conference on Machine Learning, 2020.

Author response:
**7. About more baselines:**

As for baselines, we choose the most effective attacks and defenses as baselines according to BackdoorBench, since we use its setting for evaluation. We also add some defense methods(NAD[1], Spectral signature (SS)[2]) and attack methods (WaNet[3], SSBA[4]) on CIFAR-10. The full table is shown below:

|                     | | Types             | None                 | FT                     | ANP                   | ***NAD***                    | ***SS***                    | ABL                   | EigenGuard            |
|---------------------|-------------------|-------------------|----------------------|------------------------|-----------------------|------------------------|-----------------------|-----------------------|-----------------------|
| C10                 | ASR               | BadNets              | $100\%$                | $3.0\%$               | $0.5\%$                | $6.7\%$  | $99.7\%$ | $3.1\%$  | $5.3\%$             |
|                     |                   | Blend                | $100\%$                | $10.2\%$              | $0.5\%$                | $3.8\%$  | $100\%$  | $15.2\%$ | $0.4\%$             |
|                     |                   | CLB                  | $100\%$                | $1.2\%$               | $4.0\%$                | $21.7\%$ | $88.3\%$ | $0.1\%$ | $1.0\%$              |
|                     |                   | SIG                  | $94.2\%$               | $0.4\%$               | $0.3\%$                | $6.8\%$  | $89.1\%$ | $0.01\%$   | $2.8\%$           |
|                     |                   | ***WaNet***                | $92.3\%$  | $15.1\%$ | $1.7\%$   | $28.9\%$ | $90.1\%$ | $2.3\%$  | $8.8\%$|
|                     |                   | ***SSBA***                 | $100\%$   | $23.7\%$ | $0.9\%$   | $100\%$  | $88.3\%$ | $4.4\%$  | $17.1\%$|
|                     | ACC               | BadNets              | $93.7\%$               | $87.2\%$              | $90.2\%$        x       | $90.1\%$ | $92.3\%$ | $89.1\%$  | $92.6\%$             |
|                     |                   | Blend                | $94.8\%$               | $88.9\%$              | $93.4\%$               | $93.3\%$ | $93.1\%$ | $88.7\%$  | $93.5\%$            |
|                     |                   | CLB                  | $93.8\%$               | $91.9\%$              | $92.7\%$               | $91.8\%$ | $92.4\%$ | $89.3\%$  | $93.3\%$            |
|                     |                   | SIG                  | $93.6\%$               | $91.6\%$              | $93.4\%$               | $92.1\%$ | $91.9\%$ | $89.0\%$  | $92.9\%$            |
|                     |                   | ***WaNet***                | $92.7\%$  | $93.5\%$ | $90.5\%$  | $92.2\%$ | $87.4\%$ | $88.5\%$ | $92.7\%$|
|                     |                   | ***SSBA***   | $92.8\%$  | $89.7\%$ | $88.7\%$  | $92.3\%$ | $71.5\%$ | $85.6\%$ | $93.1\%$|
| C100  | ASR             | BadNets              | $99.9\%$               | $8.9\%$               | $6.6\%$                | -        | -        | $8.8\%$               | $7.8\%$ |
|                     |                   | Blend                | $100\%$                | $78.1\%$              | $3.1\%$                | -        | -        | $0.5\%$               |  $0.4\%$ |
|                     |                   | SIG                  | $87.5\%$               | $78.6\%$              | $55.5\%$               | -        | -        | $2.1\%$               |  $0.4\%$ |
|                     |                   | ***CLB***     | $100\%$   | $5.1\%$  | $6.7\%$  | -        | -        | $4.3\%$  | $3.8\%$|
|                     | ACC | BadNets              | $71.8\%$               | $68.2\%$              | $69.7\%$               | -        | -        | $66.8\%$              | $74.1\%$ |
|                     |                   | Blend                | $73.7\%$               | $66.8\%$              | $67.4\%$               | -        | -        | $62.3\%$              | $74.8\%$ |
|                     |                   | SIG                  | $74.5\%$               | $60.7\%$              | $63.1\%$               | -        | -        | $65.3\%$              | $74.9\%$ |
|                     |                   | ***CLB***     | $74.6\%$  | $62.1\%$ | $64.7\%$  | -        | -        | $66.2\%$ | $74.5\%$ |

Reviewer comment (Experiments):
Is the proposed method able to purify more complex attacks? (See weakness missing literatures)

Author response:
**7. About more baselines:**

As for baselines, we choose the most effective attacks and defenses as baselines according to BackdoorBench, since we use its setting for evaluation. We also add some defense methods(NAD[1], Spectral signature (SS)[2]) and attack methods (WaNet[3], SSBA[4]) on CIFAR-10. The full table is shown below:

The results show that our method can perform well on different attacks. Apart from that, we also noticed that the spectral signature cannot perform well under the backdoor bench’s setting. 

Apart from that, we also noticed that the spectral signature cannot perform well under the backdoor bench’s setting. The phenomenon may caused by its dataset split being parametric sensitive and the undetected few backdoor samples that are not detected can also lead to backdoor behavior in backdoor training, which also demonstrates that the detection and retraining defense methods are unstable.

Reviewer comment (Experiments):
The paper could be strengthened by addressing the potential of adaptive attacks, especially when attackers know the EigenGuard defense.

Author response:
**1. About the possible adaptive attacks:**

A possible adaptive attack is trying to expand the trigger feature space and make it as complicated as the natural features. Then only damping top eigenspaces for features is not useful because trigger features are varied enough and no natural features are used for classification. 

However, we think it is not easy to achieve because too complicated backdoor features will also make the backdoor not easy to implement. Because the success of backdoor attacks benefits from the simple shortcuts between trigger features and labels. Therefore, the attack may be less effective when trigger features are complicated.

Reviewer comment (Experiments):
The paper would benefit from discussing the effectiveness of EigenGuard against self-supervised learning backdoor attacks [3-5], as the proposed defense operates on the output of the encoder (i.e., f). This exploration could significantly enhance the robustness and applicability of the proposed method.

Author response:
**3. About EigenGuard on the self-supervised learning backdoor attacks:**

Thanks for your advice, we also evaluate the performance of CTRL on ResNet-18 with SimCLR on CIFAR-10. The results are as follows:

| Methods | ACC. | ASR. |
| -------- | ------- | ------- |
| Ori  |  $80.5\%$   |  $85.3\%$   |
| EigenGuard | $80.3\%$     |  $22.1\%$   |

From the table, one can see that our EigenGuard can also effectively reduce the backdoor performance of SSL attacks.

Reviewer comment (Experiments):
To bolster the generalizability of the findings, it would be advantageous to evaluate the effectiveness of EigenGuard across a variety of model architectures.

Author response:
**4. About model architecture:**

Our EigenGuard can be used in many model architectures with effective performance:

|  |  | ResNet-18 | ResNet-34 | VGG-16 | MobileNetV2 |
| --- | --- | --- | --- | --- | --- |
| ASR | BadNets | $ 5.3\%$ | $2.1\%$ | $0.6\%$ | $0.5\%$ |
|  | Blend | $0.4\%$ | $3.8\%$ | $3.4\%$ | $3.3\%$ |
|  | CL | $1.0\%$ | $1.2\%$ | $1.0\%$ | $0.4\%$ |
|  | SIG | $2.8\%$ | $0.4\%$ | $0.7\%$ | $0.6\%$ |
| ACC | BadNets | $ 92.6\%$ | $93.1\%$ | $93.5\%$ | $92.1\%$ |
|  | Blend | $91.1\%$ | $92.1\%$ | $92.4\%$ | $91.7\%$ |
|  | CL | $93.3\%$ | $93.7\%$ | $94.1\%$ | $92.1\%$ |
|  | SIG | $92.9\%$ | $94.1\%$ | $94.6\%$ | $94.3\%$ |

Reviewer comment (Novelty):
The theoretical foundation of the paper seems derivative, lacking novelty.

Author response:
**1. About our theoretical analysis:**

Although our toy model is simple, our theory can clearly illustrate that damping the dominating trigger features will lead the models to use natural features to make correct predictions. It is consistent with our method, we make the top singular values smaller to avoid the triggers dominating classifications.

Reviewer comment (Novelty):
The paper does not provide a comprehensive review of related works and omits comparisons with well-established baselines.

Author response:
**2. About the baselines in our paper:**

As for the choice of baselines, we choose the most effective attacks and defenses as baselines according to BackdoorBench, since we use its setting for evaluation. We also add some defense methods (NAD[1] and Spectral Signature [2]) and attack methods(WaNet[3] and SSBA[4]) on CIFAR-10. The results show that our method can perform well on different attacks. The full table is shown below:

|                     | | Types             | None                 | FT                     | ANP                   | ***NAD***                    | ***SS***                    | ABL                   | EigenGuard            |
|---------------------|-------------------|-------------------|----------------------|------------------------|-----------------------|------------------------|-----------------------|-----------------------|-----------------------|
| C10                 | ASR               | BadNets              | $100\%$                | $3.0\%$               | $0.5\%$                | $6.7\%$  | $99.7\%$ | $3.1\%$  | $5.3\%$             |
|                     |                   | Blend                | $100\%$                | $10.2\%$              | $0.5\%$                | $3.8\%$  | $100\%$  | $15.2\%$ | $0.4\%$             |
|                     |                   | CLB                  | $100\%$                | $1.2\%$               | $4.0\%$                | $21.7\%$ | $88.3\%$ | $0.1\%$ | $1.0\%$              |
|                     |                   | SIG                  | $94.2\%$               | $0.4\%$               | $0.3\%$                | $6.8\%$  | $89.1\%$ | $0.01\%$   | $2.8\%$           |
|                     |                   | ***WaNet***                | $92.3\%$  | $15.1\%$ | $1.7\%$   | $28.9\%$ | $90.1\%$ | $2.3\%$  | $8.8\%$|
|                     |                   | ***SSBA***                 | $100\%$   | $23.7\%$ | $0.9\%$   | $100\%$  | $88.3\%$ | $4.4\%$  | $17.1\%$|
|                     | ACC               | BadNets              | $93.7\%$               | $87.2\%$              | $90.2\%$        x       | $90.1\%$ | $92.3\%$ | $89.1\%$  | $92.6\%$             |
|                     |                   | Blend                | $94.8\%$               | $88.9\%$              | $93.4\%$               | $93.3\%$ | $93.1\%$ | $88.7\%$  | $93.5\%$            |
|                     |                   | CLB                  | $93.8\%$               | $91.9\%$              | $92.7\%$               | $91.8\%$ | $92.4\%$ | $89.3\%$  | $93.3\%$            |
|                     |                   | SIG                  | $93.6\%$               | $91.6\%$              | $93.4\%$               | $92.1\%$ | $91.9\%$ | $89.0\%$  | $92.9\%$            |
|                     |                   | WaNet                | $92.7\%$  | $93.5\%$ | $90.5\%$  | $92.2\%$ | $87.4\%$ | $88.5\%$ | $92.7\%$|
|                     |                   | SSBA    | $92.8\%$  | $89.7\%$ | $88.7\%$  | $92.3\%$ | $71.5\%$ | $85.6\%$ | $93.1\%$|
| C100  | ASR             | BadNets              | $99.9\%$               | $8.9\%$               | $6.6\%$                | -        | -        | $8.8\%$               | $7.8\%$ |
|                     |                   | Blend                | $100\%$                | $78.1\%$              | $3.1\%$                | -        | -        | $0.5\%$               |  $0.4\%$ |
|                     |                   | SIG                  | $87.5\%$               | $78.6\%$              | $55.5\%$               | -        | -        | $2.1\%$               |  $0.4\%$ |
|                     |                   | ***CLB***     | $100\%$   | $5.1\%$  | $6.7\%$   | -        | -        | $4.3\%$  | $3.8\%$|
|                     | ACC | BadNets              | $71.8\%$               | $68.2\%$              | $69.7\%$               | -        | -        | $66.8\%$              | $74.1\%$ |
|                     |                   | Blend                | $73.7\%$               | $66.8\%$              | $67.4\%$               | -        | -        | $62.3\%$              | $74.8\%$ |
|                     |                   | SIG                  | $74.5\%$               | $60.7\%$              | $63.1\%$               | -        | -        | $65.3\%$              | $74.9\%$ |
|                     |                   | ***CLB***     | $74.6\%$  | $62.1\%$ | $64.7\%$  | -        | -        | $66.2\%$ | $74.5\%$ |

Reviewer comment (Writing):
Some terms such as **$A$** in formula 1 and $k, \sigma_k$ in algorithm 1 are unclear.

Author response:
**5. About A and k and $\sigma_k$.**

A is just a matrix for calculating effective ranks. In our following evaluations, we change a feature tensor of (batch_size, channel, height, width) to be a (batch_size, channel*height*width) matrix and calculate the effective ranks to explore the rank of the trigger feature space.

k here is the index, and $\sigma_k$ means the $k$-th singular value.

Reviewer comment (Experiments):
It is confusing why the CLB is not considered for experiments (e.g., Cifar100 and GTSRB).

Author response:
**6. About the CLB attack.**

We also evaluate the CLB attacks on CIFAR-100 and GTSRB datasets as shown in the above table. And the results show that our methods are also effective.

Reviewer comment (Evaluation):
From Table 3, it's observed that some models, when defended with EigenGuard, exhibit improved accuracy compared to when no defense is applied. Is there a rationale behind this outcome?

Author response:
**7. About the improvement of the natural accuracy.**

I think the improvement of the natural accuracy is caused by the EigenGuard module can also regularize the neural network by damping the top singular values. Therefore, our EigenGuard can make neural networks’ activation to be well-conditioned. It can avoid overfitting in some circumstances and lead to better results.

Reviewer comment (Theory):
The paper emphasizes over and over about the relationship of the backdoor triggers with a low-rank space. For instance, it reads: "_One plausible explanation for this observation is the limited effective subspace associated with the trigger. This suggests that the trigger features are distributed in a low-dimensional subspace._" However, these explanations are just a restatement of the actual method used in the paper, not based on step-by-step intuitive reasonings. The reviewer encourages the authors to re-write these statements and try not to rely on the observations in the figures but on the intuitions and explain why this method should work.

Author response:
**2. About “However, these explanations are just a restatement of the actual method used in the paper, not based on step-by-step intuitive reasonings. I highly encourage the authors to re-write these statements and try not to rely on the observations in the figures but on the intuitions and explain why this method should work.”**

Our method is proposed based on our findings on the features’ eigenspace during training on different samples and tries to propose a new module to avoid such behaviors.

First, we find that the trigger features can only concentrated in a low-rank subspace according to Figure 1 and Table 2. Then we find that trigger features are mainly distributed in the top singular space in Figure 2, which means the backdoor predictions are made because trigger features contribute a lot to the prediction due to their large singular value. 

Due to these findings, we propose our EigenGuard which damps the magnitude of trigger features while involving more natural features for prediction. Then we can defend against several backdoor attacks without any additional treatments during training. It’s a natural finding to solution way for research.

Reviewer comment (Writing):
The observations made in Figure 2, the explanation of the paper about this figure, and the actual methodology seem different. In particular, after plotting the t-SNE of different singular values in Figure 2, the paper says: "_However, from the middle t-SNE figure, one can see that the pink dots represent backdoor images distributed uniformly in the space and overlap with other color dots. Thereby, the network cannot classify these samples as trigger classes since they are similar to samples belonging to different natural classes._" From this explanation it seems that the natural way of dealing with backdoors is to remove the dominant singular values. However, as shown in Algorithm 1 in the Spectral Filter, the proposed method actually preserves those singular values and dampens the effects of the remaining ones. This apparent contradiction suggests there may be a misunderstanding that needs to be resolved.

Author response:
**3.About the “inconsistency“ between Figure 2 and our method:**

Here we only damp top eigenvalues instead of dropping them because $\sigma_k$ is much smaller than $\sigma_1$, which is consistent with our findings in Figure 2. From Figure 2(a), one can see that the trigger features are large enough to dominate the classification of backdoor samples, so making it smaller can decrease its impact. In other words, we try to average the backdoor features in Figure 2 (a) and (b) to make the backdoor samples hard to classify.

The reason why we not simply removing the top eigenvalues is such a method will make natural classes hard to classify as we can see in Figure 2(b) and lead to a huge drop in natural accuracy as listed in the following experiments on CIFAR-10 with BadNets:

| Methods | ACC. | ASR. |
| -------- | ------- | ------- |
| Remove top 10 Singulars  |  $85.4\%$   |  $4.2\%$   |
| Ours | $92.6\%$     |  $5.3\%$   |

Also, as shown in Figure 4, the EigenGuard module can make the features perform as we wish.

Reviewer comment (Novelty):
There are two important related works that this paper needs to discuss its relationship with in more detail. First, the method of Spectral Signatures [1] also uses SVD in the feature space of neural nets to filter samples that are poisoned. Second, Collider [2] uses local intrinsic dimensionality (LID) to argue that backdoors reside in a locally high-dimension manifold. The current work argues that backdoors reside in a low-dimensional sub-space (even though it does so rather informally), making it important to clarify its stance with respect to these prior works.

Author response:
**4. About our methods’ relationship with Spectral Signature and Local intrinsic dimensionality.**

There are two differences between ours and theirs:

(1) Our method is a plug-in module for neural architectures, neural networks can naturally defend against some backdoor attacks with end-to-end natural training by inserting our EigenGuard model into the model’s architecture. The other two methods only try to use the poisoned model's behavior to detect the trigger images and then try to purify the datasets to prevent backdoor attacks, which need complicated post-stage retraining stages.

(2) Although both works analyze the feature behaviors, ours tries to detect the feature’s spectral influence during training while others only analyze the model's feature behavior after training.

Reviewer comment (Theory):
The theoretical contributions seem too abstract. It is unclear how the provided theory is supporting the proposed method, as having a set of segregated feature vectors for backdoored vs. clean data seems too artificial and not directly related to EigenGuard. This section should be re-written to make its connections with the proposed method clearer.

Author response:
**5. About the theory:**
Our theory tries to demonstrate that damping the dominating trigger features will make the models use natural features to make correct predictions. It is also consistent with our method, we make the top singular values smaller to avoid the triggers dominating classifications.