Paper: Non-uniform Noise Injection For Enhancing DNN Adversarial Robustness And Efficiency (ICLR-2024)

Reviewer comment (Theory):
It is stated that "A neuron is regarded as essential if its approximation from ˜z is larger than the predefined threshold." However, I am unsure why this makes it an important neuron. ˜z is used to reduce the value error through the MSE of z, meaning that the larger the z value, the more it will affect the size of ˜z. Therefore, an explanation is needed on why neurons with larger ˜z values are considered important.

Author response:
**(1) Why neurons with larger ˜z values are considered important.** 

Intuitively, activations with a larger magnitude have more impact on overall model inference; hence important. We use the approximate values as proxy to choose neurons with large magnitude.

Reviewer comment (Experiments):
The experiment tested both top-K injection and N:M injection methods, but there was no experiment that simply added noise values to a random N% of neurons. Therefore, there is a lack of validity in claiming that this method is effective.

Author response:
**(2) Add noise values to a random N\% of neurons. And control group that adds different noise to those neurons as a comparison to the approximated values used for detection.**

This is part of the ablation experiment that we missed to support our claim more rigorously, and we will include this experiment in future enhancements.

Reviewer comment (Experiments):
During the experimental process, in the experiment where noise was added to non-core neurons, I think there needs to be a control group that adds different noise to those neurons as a comparison to the approximated values used for detection. It is necessary to verify whether the method of injecting this noise actually contributes to performance improvement or simply enhances computational efficiency.

Author response:
**(2) Add noise values to a random N\% of neurons. And control group that adds different noise to those neurons as a comparison to the approximated values used for detection.**

This is part of the ablation experiment that we missed to support our claim more rigorously, and we will include this experiment in future enhancements.

Reviewer comment (Experiments):
Too much limited and insufficient experiments: There are no state-of-the-art attacks such AutoAttack [1] and no state-of-the-art defense baselines such as AWP [2], SCORE [3], and ADML [4]. In addition, based on ADML, not only CNN structure and Transformer structures should be needed to validate.

Author response:
**(3) Insufficient experiments.**

Your constructive comments are much appreciated. We will include a comparison of these robustness-related methods in future experiments.

Reviewer comment (Evaluation):
The threshold and the proportion of non-essential neurons should also be reported. It is also helpful to provide the distribution of the difference between $z$ and $\tilde{z}$, and study the relationship between accuracies and the choice of threshold.

Author response:
**(1) The threshold and the proportion of non-essential neurons.**

It indeed was a writing mistake. The threshold is not predefined, it is found during the top-k process. This threshold is the smallest number in the top-k, and the purpose of the threshold is to facilitate the generation of the mask. We will revise the manuscript to avoid ambiguity.

**(2) The distribution of the difference between z and ~z.**

Thank you for your valuable advice. We will take this into consideration in our future improvements to explore the underlying reasons and to support our approach even more.

Reviewer comment (Theory):
The theorem does not seem to directly link to the results. Based on the theorem, the clean accuracy can be higher than that without the noise injection, and it does not explain the consistent minor drop in clean accuracy.

Author response:
**(3) Theorem explanation.**

Our theoretical proof is divided into two parts, the first part is to show that this type of noise injection is naturally brought about by the dimensionality reduction method **random projection**, which has no effect on the distance between vectors and vectors in the high-dimensional space and therefore has no effect on the model's performance in clean accuracy. The second part shows that this kind of noise injection can be regarded as a kind of Gaussian noise, which improves the robustness, but too much noise injection affects the performance of the model in clean accuracy. Therefore, we chose the non-uniform noise injection that embodies a kind of trade-off.

Reviewer comment (Theory):
The learning of $\tilde{W}$ and $\tilde{b}$ minimizes the MSE. Is it possible that the optimal learned solution is $W=\tilde{W}P$ and $b=\tilde{b}$?

Author response:
**(4) Optimal learned solution of approximation weight and bias.**

Thank you for your careful consideration. Essentially, this approach is slightly different from the optimization approach using MSE, mainly is that the difference in the optimization objective has an impact on the convergence of the model training, and the way MSE is chosen is a single objective optimization, which makes the model converge better. If choosing weights and biases, the optimization objective becomes four, and the hyperparameters are more difficult to choose. In addition, we want to make the activation having little influence the result, so we do the optimization in this way. We will try the method you mentioned.

Reviewer comment (Experiments):
Are there any specific reasons that $\epsilon=4/255$ in Table 1 while $\epsilon=8/255$ in Table 3?

Author response:
**(5) Adversarial attack epsilon choice.**

In Table 1, this validation of adversarial robustness does not use adversarial training, which means that the neural network is not pre-fitted on the adversarial samples, and thus in case of larger attack intensity epsilon, the robustness accuracy of both our method and the original model will be very low, which does not reflect the enhancement of our model in adversarial robustness. Instead, Table 3 is trained for confrontation and can withstand higher intensity attacks epsilon(e.g. 8/255).

Reviewer comment (Writing):
I find the paper a little hard to follow. Specifically, I cannot determine what “irregular 50%” and “structured/unstructured” mean in the paper’s context. It is very likely because I don’t have sufficient background knowledge and I would appreciate it if the authors could refer me to 2-3 most related (and preferably recent) papers that can help me gain the basics. But for now, I tend to believe the authors fail to make the paper easy to follow.

Author response:
**(1) “irregular 50\%” and “structured/unstructured” mean in the paper’s context.**

Thank you for your advice. We realize that the lack of explanation of sparsity can be troubling to readers. So we need to explain the irregular as well as the structured/unstructured in the manuscript. `irregular/unstructured' and 'structured' both can be seen as a kind of sparsity. ‘irregular/unstructured’ means that the **activation pattern** after sparsification (top-k) is irregular[1]. 'structured' means for a fixed block, the sparsity is determediated[1] [2]. It's like the one shown in Figure 2 in the text. The difference between the two forms of sparsity is mainly in the efficiency of hardware execution; structured sparsity is more efficiently executed due to the fact that it can be chunked for the extraction of deterministic sparse values.

Reviewer comment (Evaluation):
AutoAttack is not included in threat models, which I believe is necessary to show the adversarial robustness of a new method.

Author response:
**(2) Autoattack to show the adversarial robustness.**

The autoattack method is the state-of-the-art method in recent years, in our experiments we ignored this method because of insufficient research, in the future experiments we will add the validation of this method, thank you for your suggestion.

Reviewer comment (Evaluation):
As it is claimed that the proposed method enhances adversarial robustness and execution efficiency simultaneously, it is important to show directly how much the efficiency is improved directly (e.g., throughput) compared to the baselines.

Author response:
**(4) Efficiency criteria.**

We will consider more measures that indicate model efficiency.

Reviewer comment (Theory):
Can you intuitively explain why “a neuron is regarded as essential if its approximation from $\widetilde{z}$ is larger than the predefined threshold”?

Author response:
**(5) A neuron is regarded as essential if its approximation is larger than the predefined threshold.**

This was a writing mistake that causes confusion. The threshold is not predefined, it is found during the top-k process. This threshold is the smallest number in the top-k, and the purpose of the threshold is to facilitate the generation of the mask. We will revise the manuscript to avoid ambiguity.